2024-04-25T10:22:19.606 INFO:root:teuthology version: 0.0.1.dev269+g9481b1d 2024-04-25T10:22:19.606 DEBUG:teuthology.run:Teuthology command: teuthology --archive /home/teuthworker/archive/pdonnell-2024-04-25_10:00:23-fs-wip-pdonnell-testing-20240425.015853-debug-distro-default-smithi/7673076 --description fs/functional/{begin/{0-install 1-ceph 2-logrotate 3-modules} clusters/1a3s-mds-4c-client conf/{client mds mgr mon osd} distro/{centos_latest} mount/fuse objectstore/bluestore-bitmap overrides/{ignorelist_health ignorelist_wrongly_marked_down no_client_pidfile pg_health} subvol_versions/create_subvol_version_v1 tasks/alternate-pool} --name pdonnell-2024-04-25_10:00:23-fs-wip-pdonnell-testing-20240425.015853-debug-distro-default-smithi --verbose --owner scheduled_pdonnell@teuthology -- /home/teuthworker/archive/pdonnell-2024-04-25_10:00:23-fs-wip-pdonnell-testing-20240425.015853-debug-distro-default-smithi/7673076/orig.config.yaml 2024-04-25T10:22:19.654 DEBUG:teuthology.report:Pushing job info to https://paddles.front.sepia.ceph.com/ 2024-04-25T10:22:19.745 INFO:teuthology.run:Config: archive_path: /home/teuthworker/archive/pdonnell-2024-04-25_10:00:23-fs-wip-pdonnell-testing-20240425.015853-debug-distro-default-smithi/7673076 branch: wip-pdonnell-testing-20240425.015853-debug description: fs/functional/{begin/{0-install 1-ceph 2-logrotate 3-modules} clusters/1a3s-mds-4c-client conf/{client mds mgr mon osd} distro/{centos_latest} mount/fuse objectstore/bluestore-bitmap overrides/{ignorelist_health ignorelist_wrongly_marked_down no_client_pidfile pg_health} subvol_versions/create_subvol_version_v1 tasks/alternate-pool} email: pdonnell@redhat.com first_in_suite: false job_id: '7673076' kernel: kdb: true sha1: distro last_in_suite: false log-rotate: ceph-mds: 10G ceph-osd: 10G machine_type: smithi mgrmodules: sequential: - print: Enabling mgr modules name: pdonnell-2024-04-25_10:00:23-fs-wip-pdonnell-testing-20240425.015853-debug-distro-default-smithi no_nested_subset: false nuke-on-error: true openstack: - volumes: count: 4 size: 20 - machine: disk: 200 os_type: centos os_version: 9.stream overrides: admin_socket: branch: wip-pdonnell-testing-20240425.015853-debug ceph: conf: client: client mount timeout: 600 debug client: 20 debug ms: 1 pid file: '' rados mon op timeout: 900 rados osd op timeout: 900 mds: debug mds: 20 debug mds balancer: 20 debug ms: 1 mds debug frag: true mds debug scatterstat: true mds op complaint time: 180 mds verify scatter: true osd op complaint time: 180 rados mon op timeout: 900 rados osd op timeout: 900 mgr: debug client: 20 debug mgr: 20 debug ms: 1 mon: debug mon: 20 debug ms: 1 debug paxos: 20 mon down mkfs grace: 300 mon op complaint time: 120 osd: bdev async discard: true bdev enable discard: true bluestore allocator: bitmap bluestore block size: 96636764160 bluestore fsck on mount: true debug bluefs: 1/20 debug bluestore: 1/20 debug ms: 1 debug osd: 20 debug rocksdb: 4/10 mon osd backfillfull_ratio: 0.85 mon osd full ratio: 0.9 mon osd nearfull ratio: 0.8 osd failsafe full ratio: 0.95 osd objectstore: bluestore osd op complaint time: 180 flavor: default fs: xfs log-ignorelist: - \(MDS_ALL_DOWN\) - \(MDS_UP_LESS_THAN_MAX\) - FS_DEGRADED - FS_INLINE_DATA_DEPRECATED - FS_WITH_FAILED_MDS - MDS_ALL_DOWN - MDS_DAMAGE - MDS_DEGRADED - MDS_FAILED - MDS_INSUFFICIENT_STANDBY - MDS_UP_LESS_THAN_MAX - filesystem is online with fewer MDS than max_mds - POOL_APP_NOT_ENABLED - do not have an application enabled - overall HEALTH_ - Replacing daemon - deprecated feature inline_data - overall HEALTH_ - \(OSD_DOWN\) - \(OSD_ - but it is still running - is not responding - PG_AVAILABILITY - PG_DEGRADED - Reduced data availability - bad backtrace - object missing on disk - error reading table object - error reading sessionmap - unmatched fragstat - unmatched rstat - was unreadable, recreating it now - Scrub error on inode - Metadata damage detected - MDS_FAILED - MDS_DAMAGE sha1: 994e2f20102a20fa9a27966c14b625cc01884df8 ceph-deploy: bluestore: true conf: client: log file: /var/log/ceph/ceph-$name.$pid.log mon: {} osd: bdev async discard: true bdev enable discard: true bluestore block size: 96636764160 bluestore fsck on mount: true debug bluefs: 1/20 debug bluestore: 1/20 debug rocksdb: 4/10 mon osd backfillfull_ratio: 0.85 mon osd full ratio: 0.9 mon osd nearfull ratio: 0.8 osd failsafe full ratio: 0.95 osd objectstore: bluestore fs: xfs install: ceph: flavor: default sha1: 994e2f20102a20fa9a27966c14b625cc01884df8 selinux: allowlist: - scontext=system_u:system_r:getty_t:s0 subvolume_version: 1 thrashosds: bdev_inject_crash: 2 bdev_inject_crash_probability: 0.5 workunit: branch: wip-pdonnell-testing-20240425.015853-debug sha1: 994e2f20102a20fa9a27966c14b625cc01884df8 owner: scheduled_pdonnell@teuthology priority: 85 repo: https://git.ceph.com/ceph-ci.git roles: - - mon.a - mgr.y - mds.a - mds.c - osd.0 - osd.1 - osd.2 - osd.3 - client.0 - client.2 - - mon.b - mon.c - mgr.x - mds.b - mds.d - osd.4 - osd.5 - osd.6 - osd.7 - client.1 - client.3 seed: 1880 sha1: 994e2f20102a20fa9a27966c14b625cc01884df8 sleep_before_teardown: 0 subset: 178/512 suite: fs suite_branch: wip-pdonnell-testing-20240425.015853-debug suite_path: /home/teuthworker/src/git.ceph.com_ceph-c_994e2f20102a20fa9a27966c14b625cc01884df8/qa suite_relpath: qa suite_repo: https://git.ceph.com/ceph-ci.git suite_sha1: 994e2f20102a20fa9a27966c14b625cc01884df8 syslog: ignorelist: - WARNING*.*check_session_state - WARNING*.*__ceph_remove_cap targets: smithi037.front.sepia.ceph.com: ecdsa-sha2-nistp256 AAAAE2VjZHNhLXNoYTItbmlzdHAyNTYAAAAIbmlzdHAyNTYAAABBBNb4E+ULhZgytXCYeSyIeWUc2qPf8fZagTkx7ptrATcZuSzhYFE+9U7RVP8J8zk9PX9bEZMENSTJ0lvQlJJV8J0= smithi203.front.sepia.ceph.com: ecdsa-sha2-nistp256 AAAAE2VjZHNhLXNoYTItbmlzdHAyNTYAAAAIbmlzdHAyNTYAAABBBKr9A/WsOZj+JJQP5Nth08arVsp53+mJPrPi+tzLz/Io/71y6YjlnTskBD+YCBw5RcNSZrKq/1pbw1+Nkh7cSgg= tasks: - install: extra_packages: deb: - python3-cephfs - cephfs-shell - cephfs-top - cephfs-mirror rpm: - python3-cephfs - cephfs-shell - cephfs-top - cephfs-mirror extra_system_packages: deb: - bison - flex - libelf-dev - libssl-dev - network-manager - iproute2 - util-linux - dump - indent - libaio-dev - libtool-bin - uuid-dev - xfslibs-dev - postgresql - postgresql-client - postgresql-common - postgresql-contrib rpm: - bison - flex - elfutils-libelf-devel - openssl-devel - NetworkManager - iproute - util-linux - libacl-devel - libaio-devel - libattr-devel - libtool - libuuid-devel - xfsdump - xfsprogs - xfsprogs-devel - libaio-devel - libtool - libuuid-devel - xfsprogs-devel - postgresql - postgresql-server - postgresql-contrib - ceph: create_rbd_pool: false - sequential: - mgrmodules - ceph-fuse: null - cephfs_test_runner: modules: - tasks.cephfs.test_recovery_pool teuthology: fragments_dropped: [] meta: {} postmerge: - local function is_kupstream() return false end - local function is_kdistro() return false end - local function is_fuse() return true end - local function syntax_version() return '' end teuthology_branch: main teuthology_sha1: 9481b1d62f50e7d0a4f3dd83adf6945b08d5ff17 timestamp: 2024-04-25_10:00:23 tube: smithi user: pdonnell verbose: false worker_log: /home/teuthworker/archive/worker_logs/dispatcher.smithi.2226885 2024-04-25T10:22:19.745 INFO:teuthology.run:suite_path is set to /home/teuthworker/src/git.ceph.com_ceph-c_994e2f20102a20fa9a27966c14b625cc01884df8/qa; will attempt to use it 2024-04-25T10:22:19.746 INFO:teuthology.run:Found tasks at /home/teuthworker/src/git.ceph.com_ceph-c_994e2f20102a20fa9a27966c14b625cc01884df8/qa/tasks 2024-04-25T10:22:19.746 INFO:teuthology.run_tasks:Running task internal.check_packages... 2024-04-25T10:22:19.748 INFO:teuthology.task.internal:Checking packages... 2024-04-25T10:22:19.773 INFO:teuthology.task.internal:Checking packages for os_type 'centos', flavor 'default' and ceph hash '994e2f20102a20fa9a27966c14b625cc01884df8' 2024-04-25T10:22:19.774 WARNING:teuthology.packaging:More than one of ref, tag, branch, or sha1 supplied; using branch 2024-04-25T10:22:19.774 INFO:teuthology.packaging:ref: None 2024-04-25T10:22:19.774 INFO:teuthology.packaging:tag: None 2024-04-25T10:22:19.774 INFO:teuthology.packaging:branch: wip-pdonnell-testing-20240425.015853-debug 2024-04-25T10:22:19.774 INFO:teuthology.packaging:sha1: 994e2f20102a20fa9a27966c14b625cc01884df8 2024-04-25T10:22:19.774 DEBUG:teuthology.packaging:Querying https://shaman.ceph.com/api/search?status=ready&project=ceph&flavor=default&distros=centos%2F9%2Fx86_64&ref=wip-pdonnell-testing-20240425.015853-debug 2024-04-25T10:22:19.927 INFO:teuthology.task.internal:Found packages for ceph version 19.0.0-3291.g994e2f20 2024-04-25T10:22:19.928 INFO:teuthology.run_tasks:Running task internal.buildpackages_prep... 2024-04-25T10:22:19.935 INFO:teuthology.task.internal:no buildpackages task found 2024-04-25T10:22:19.935 INFO:teuthology.run_tasks:Running task internal.save_config... 2024-04-25T10:22:19.993 INFO:teuthology.task.internal:Saving configuration 2024-04-25T10:22:20.011 INFO:teuthology.run_tasks:Running task internal.check_lock... 2024-04-25T10:22:20.018 INFO:teuthology.task.internal.check_lock:Checking locks... 2024-04-25T10:22:20.041 DEBUG:teuthology.task.internal.check_lock:machine status is {'name': 'smithi037.front.sepia.ceph.com', 'description': '/home/teuthworker/archive/pdonnell-2024-04-25_10:00:23-fs-wip-pdonnell-testing-20240425.015853-debug-distro-default-smithi/7673076', 'up': True, 'machine_type': 'smithi', 'is_vm': False, 'vm_host': None, 'os_type': 'centos', 'os_version': '9', 'arch': 'x86_64', 'locked': True, 'locked_since': '2024-04-25 10:17:05.251454', 'locked_by': 'scheduled_pdonnell@teuthology', 'mac_address': None, 'ssh_pub_key': 'ecdsa-sha2-nistp256 AAAAE2VjZHNhLXNoYTItbmlzdHAyNTYAAAAIbmlzdHAyNTYAAABBBNb4E+ULhZgytXCYeSyIeWUc2qPf8fZagTkx7ptrATcZuSzhYFE+9U7RVP8J8zk9PX9bEZMENSTJ0lvQlJJV8J0='} 2024-04-25T10:22:20.062 DEBUG:teuthology.task.internal.check_lock:machine status is {'name': 'smithi203.front.sepia.ceph.com', 'description': '/home/teuthworker/archive/pdonnell-2024-04-25_10:00:23-fs-wip-pdonnell-testing-20240425.015853-debug-distro-default-smithi/7673076', 'up': True, 'machine_type': 'smithi', 'is_vm': False, 'vm_host': None, 'os_type': 'centos', 'os_version': '9', 'arch': 'x86_64', 'locked': True, 'locked_since': '2024-04-25 10:17:05.253195', 'locked_by': 'scheduled_pdonnell@teuthology', 'mac_address': None, 'ssh_pub_key': 'ecdsa-sha2-nistp256 AAAAE2VjZHNhLXNoYTItbmlzdHAyNTYAAAAIbmlzdHAyNTYAAABBBKr9A/WsOZj+JJQP5Nth08arVsp53+mJPrPi+tzLz/Io/71y6YjlnTskBD+YCBw5RcNSZrKq/1pbw1+Nkh7cSgg='} 2024-04-25T10:22:20.062 INFO:teuthology.run_tasks:Running task internal.add_remotes... 2024-04-25T10:22:20.068 INFO:teuthology.task.internal:roles: ubuntu@smithi037.front.sepia.ceph.com - ['mon.a', 'mgr.y', 'mds.a', 'mds.c', 'osd.0', 'osd.1', 'osd.2', 'osd.3', 'client.0', 'client.2'] 2024-04-25T10:22:20.068 INFO:teuthology.task.internal:roles: ubuntu@smithi203.front.sepia.ceph.com - ['mon.b', 'mon.c', 'mgr.x', 'mds.b', 'mds.d', 'osd.4', 'osd.5', 'osd.6', 'osd.7', 'client.1', 'client.3'] 2024-04-25T10:22:20.068 INFO:teuthology.run_tasks:Running task console_log... 2024-04-25T10:22:20.192 DEBUG:teuthology.exit:Installing handler: Handler(exiter=, func=.kill_console_loggers at 0x7f9273d78f70>, signals=[15]) 2024-04-25T10:22:20.192 INFO:teuthology.run_tasks:Running task internal.connect... 2024-04-25T10:22:20.199 INFO:teuthology.task.internal:Opening connections... 2024-04-25T10:22:20.199 DEBUG:teuthology.task.internal:connecting to ubuntu@smithi037.front.sepia.ceph.com 2024-04-25T10:22:20.201 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi037.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-04-25T10:22:20.286 DEBUG:teuthology.task.internal:connecting to ubuntu@smithi203.front.sepia.ceph.com 2024-04-25T10:22:20.287 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi203.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-04-25T10:22:20.362 INFO:teuthology.run_tasks:Running task internal.push_inventory... 2024-04-25T10:22:20.369 DEBUG:teuthology.orchestra.run.smithi037:> uname -m 2024-04-25T10:22:20.388 INFO:teuthology.orchestra.run.smithi037.stdout:x86_64 2024-04-25T10:22:20.388 DEBUG:teuthology.orchestra.run.smithi037:> cat /etc/os-release 2024-04-25T10:22:20.446 INFO:teuthology.orchestra.run.smithi037.stdout:NAME="CentOS Stream" 2024-04-25T10:22:20.446 INFO:teuthology.orchestra.run.smithi037.stdout:VERSION="9" 2024-04-25T10:22:20.447 INFO:teuthology.orchestra.run.smithi037.stdout:ID="centos" 2024-04-25T10:22:20.447 INFO:teuthology.orchestra.run.smithi037.stdout:ID_LIKE="rhel fedora" 2024-04-25T10:22:20.447 INFO:teuthology.orchestra.run.smithi037.stdout:VERSION_ID="9" 2024-04-25T10:22:20.447 INFO:teuthology.orchestra.run.smithi037.stdout:PLATFORM_ID="platform:el9" 2024-04-25T10:22:20.447 INFO:teuthology.orchestra.run.smithi037.stdout:PRETTY_NAME="CentOS Stream 9" 2024-04-25T10:22:20.447 INFO:teuthology.orchestra.run.smithi037.stdout:ANSI_COLOR="0;31" 2024-04-25T10:22:20.447 INFO:teuthology.orchestra.run.smithi037.stdout:LOGO="fedora-logo-icon" 2024-04-25T10:22:20.447 INFO:teuthology.orchestra.run.smithi037.stdout:CPE_NAME="cpe:/o:centos:centos:9" 2024-04-25T10:22:20.447 INFO:teuthology.orchestra.run.smithi037.stdout:HOME_URL="https://centos.org/" 2024-04-25T10:22:20.447 INFO:teuthology.orchestra.run.smithi037.stdout:BUG_REPORT_URL="https://bugzilla.redhat.com/" 2024-04-25T10:22:20.447 INFO:teuthology.orchestra.run.smithi037.stdout:REDHAT_SUPPORT_PRODUCT="Red Hat Enterprise Linux 9" 2024-04-25T10:22:20.447 INFO:teuthology.orchestra.run.smithi037.stdout:REDHAT_SUPPORT_PRODUCT_VERSION="CentOS Stream" 2024-04-25T10:22:20.448 INFO:teuthology.lock.ops:Updating smithi037.front.sepia.ceph.com on lock server 2024-04-25T10:22:20.473 DEBUG:teuthology.orchestra.run.smithi203:> uname -m 2024-04-25T10:22:20.491 INFO:teuthology.orchestra.run.smithi203.stdout:x86_64 2024-04-25T10:22:20.491 DEBUG:teuthology.orchestra.run.smithi203:> cat /etc/os-release 2024-04-25T10:22:20.548 INFO:teuthology.orchestra.run.smithi203.stdout:NAME="CentOS Stream" 2024-04-25T10:22:20.549 INFO:teuthology.orchestra.run.smithi203.stdout:VERSION="9" 2024-04-25T10:22:20.549 INFO:teuthology.orchestra.run.smithi203.stdout:ID="centos" 2024-04-25T10:22:20.549 INFO:teuthology.orchestra.run.smithi203.stdout:ID_LIKE="rhel fedora" 2024-04-25T10:22:20.549 INFO:teuthology.orchestra.run.smithi203.stdout:VERSION_ID="9" 2024-04-25T10:22:20.549 INFO:teuthology.orchestra.run.smithi203.stdout:PLATFORM_ID="platform:el9" 2024-04-25T10:22:20.549 INFO:teuthology.orchestra.run.smithi203.stdout:PRETTY_NAME="CentOS Stream 9" 2024-04-25T10:22:20.549 INFO:teuthology.orchestra.run.smithi203.stdout:ANSI_COLOR="0;31" 2024-04-25T10:22:20.549 INFO:teuthology.orchestra.run.smithi203.stdout:LOGO="fedora-logo-icon" 2024-04-25T10:22:20.549 INFO:teuthology.orchestra.run.smithi203.stdout:CPE_NAME="cpe:/o:centos:centos:9" 2024-04-25T10:22:20.549 INFO:teuthology.orchestra.run.smithi203.stdout:HOME_URL="https://centos.org/" 2024-04-25T10:22:20.549 INFO:teuthology.orchestra.run.smithi203.stdout:BUG_REPORT_URL="https://bugzilla.redhat.com/" 2024-04-25T10:22:20.549 INFO:teuthology.orchestra.run.smithi203.stdout:REDHAT_SUPPORT_PRODUCT="Red Hat Enterprise Linux 9" 2024-04-25T10:22:20.549 INFO:teuthology.orchestra.run.smithi203.stdout:REDHAT_SUPPORT_PRODUCT_VERSION="CentOS Stream" 2024-04-25T10:22:20.550 INFO:teuthology.lock.ops:Updating smithi203.front.sepia.ceph.com on lock server 2024-04-25T10:22:20.573 INFO:teuthology.run_tasks:Running task internal.serialize_remote_roles... 2024-04-25T10:22:20.582 INFO:teuthology.run_tasks:Running task internal.check_conflict... 2024-04-25T10:22:20.623 INFO:teuthology.task.internal:Checking for old test directory... 2024-04-25T10:22:20.624 DEBUG:teuthology.orchestra.run.smithi037:> test '!' -e /home/ubuntu/cephtest 2024-04-25T10:22:20.626 DEBUG:teuthology.orchestra.run.smithi203:> test '!' -e /home/ubuntu/cephtest 2024-04-25T10:22:20.642 INFO:teuthology.run_tasks:Running task internal.check_ceph_data... 2024-04-25T10:22:20.684 INFO:teuthology.task.internal:Checking for non-empty /var/lib/ceph... 2024-04-25T10:22:20.685 DEBUG:teuthology.orchestra.run.smithi037:> test -z $(ls -A /var/lib/ceph) 2024-04-25T10:22:20.687 DEBUG:teuthology.orchestra.run.smithi203:> test -z $(ls -A /var/lib/ceph) 2024-04-25T10:22:20.713 INFO:teuthology.run_tasks:Running task internal.vm_setup... 2024-04-25T10:22:20.769 INFO:teuthology.run_tasks:Running task kernel... 2024-04-25T10:22:20.783 INFO:teuthology.task.kernel:normalize config orig: {'kdb': True, 'sha1': 'distro'} 2024-04-25T10:22:20.783 INFO:teuthology.task.kernel:config {'mon.a': {'kdb': True, 'sha1': 'distro'}, 'mon.b': {'kdb': True, 'sha1': 'distro'}}, timeout 300 2024-04-25T10:22:20.784 DEBUG:teuthology.orchestra.run.smithi037:> test -f /run/.containerenv -o -f /.dockerenv 2024-04-25T10:22:20.784 DEBUG:teuthology.orchestra.run.smithi203:> test -f /run/.containerenv -o -f /.dockerenv 2024-04-25T10:22:20.801 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-04-25T10:22:20.801 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-04-25T10:22:20.802 DEBUG:teuthology.orchestra.run.smithi037:> uname -r 2024-04-25T10:22:20.802 DEBUG:teuthology.orchestra.run.smithi203:> uname -r 2024-04-25T10:22:20.860 INFO:teuthology.orchestra.run.smithi203.stdout:5.14.0-437.el9.x86_64 2024-04-25T10:22:20.860 INFO:teuthology.task.kernel:Running kernel on smithi203: 5.14.0-437.el9.x86_64 2024-04-25T10:22:20.860 DEBUG:teuthology.orchestra.run.smithi203:> sudo yum install -y kernel 2024-04-25T10:22:20.861 INFO:teuthology.orchestra.run.smithi037.stdout:5.14.0-437.el9.x86_64 2024-04-25T10:22:20.861 INFO:teuthology.task.kernel:Running kernel on smithi037: 5.14.0-437.el9.x86_64 2024-04-25T10:22:20.862 DEBUG:teuthology.orchestra.run.smithi037:> sudo yum install -y kernel 2024-04-25T10:22:23.634 INFO:teuthology.orchestra.run.smithi037.stdout:CentOS Stream 9 - BaseOS 7.3 MB/s | 8.0 MB 00:01 2024-04-25T10:22:24.670 INFO:teuthology.orchestra.run.smithi203.stdout:CentOS Stream 9 - BaseOS 3.8 MB/s | 8.0 MB 00:02 2024-04-25T10:22:26.851 INFO:teuthology.orchestra.run.smithi037.stdout:CentOS Stream 9 - AppStream 9.2 MB/s | 19 MB 00:02 2024-04-25T10:22:27.213 INFO:teuthology.orchestra.run.smithi203.stdout:CentOS Stream 9 - AppStream 13 MB/s | 19 MB 00:01 2024-04-25T10:22:31.677 INFO:teuthology.orchestra.run.smithi037.stdout:CentOS Stream 9 - CRB 5.7 MB/s | 6.2 MB 00:01 2024-04-25T10:22:33.113 INFO:teuthology.orchestra.run.smithi203.stdout:CentOS Stream 9 - CRB 2.8 MB/s | 6.2 MB 00:02 2024-04-25T10:22:33.370 INFO:teuthology.orchestra.run.smithi037.stdout:CentOS Stream 9 - Extras packages 33 kB/s | 16 kB 00:00 2024-04-25T10:22:34.471 INFO:teuthology.orchestra.run.smithi037.stdout:Extra Packages for Enterprise Linux 21 MB/s | 21 MB 00:00 2024-04-25T10:22:34.706 INFO:teuthology.orchestra.run.smithi203.stdout:CentOS Stream 9 - Extras packages 64 kB/s | 16 kB 00:00 2024-04-25T10:22:36.202 INFO:teuthology.orchestra.run.smithi203.stdout:Extra Packages for Enterprise Linux 15 MB/s | 21 MB 00:01 2024-04-25T10:22:39.771 INFO:teuthology.orchestra.run.smithi037.stdout:lab-extras 49 kB/s | 1.7 kB 00:00 2024-04-25T10:22:41.282 INFO:teuthology.orchestra.run.smithi203.stdout:lab-extras 15 kB/s | 1.7 kB 00:00 2024-04-25T10:22:41.753 INFO:teuthology.orchestra.run.smithi037.stdout:Package kernel-5.14.0-239.el9.x86_64 is already installed. 2024-04-25T10:22:41.753 INFO:teuthology.orchestra.run.smithi037.stdout:Package kernel-5.14.0-437.el9.x86_64 is already installed. 2024-04-25T10:22:41.841 INFO:teuthology.orchestra.run.smithi037.stdout:Dependencies resolved. 2024-04-25T10:22:41.842 INFO:teuthology.orchestra.run.smithi037.stdout:Nothing to do. 2024-04-25T10:22:41.842 INFO:teuthology.orchestra.run.smithi037.stdout:Complete! 2024-04-25T10:22:41.948 DEBUG:teuthology.orchestra.run.smithi037:> echo no | sudo yum reinstall kernel || true 2024-04-25T10:22:42.470 INFO:teuthology.orchestra.run.smithi037.stdout:Last metadata expiration check: 0:00:03 ago on Thu 25 Apr 2024 10:22:39 AM UTC. 2024-04-25T10:22:42.594 INFO:teuthology.orchestra.run.smithi037.stdout:Dependencies resolved. 2024-04-25T10:22:42.622 INFO:teuthology.orchestra.run.smithi037.stdout:================================================================================ 2024-04-25T10:22:42.622 INFO:teuthology.orchestra.run.smithi037.stdout: Package Architecture Version Repository Size 2024-04-25T10:22:42.622 INFO:teuthology.orchestra.run.smithi037.stdout:================================================================================ 2024-04-25T10:22:42.623 INFO:teuthology.orchestra.run.smithi037.stdout:Reinstalling: 2024-04-25T10:22:42.623 INFO:teuthology.orchestra.run.smithi037.stdout: kernel x86_64 5.14.0-437.el9 baseos 255 k 2024-04-25T10:22:42.623 INFO:teuthology.orchestra.run.smithi037.stdout: 2024-04-25T10:22:42.623 INFO:teuthology.orchestra.run.smithi037.stdout:Transaction Summary 2024-04-25T10:22:42.623 INFO:teuthology.orchestra.run.smithi037.stdout:================================================================================ 2024-04-25T10:22:42.623 INFO:teuthology.orchestra.run.smithi037.stdout: 2024-04-25T10:22:42.623 INFO:teuthology.orchestra.run.smithi037.stdout:Total download size: 255 k 2024-04-25T10:22:42.623 INFO:teuthology.orchestra.run.smithi037.stdout:Installed size: 0 2024-04-25T10:22:42.623 INFO:teuthology.orchestra.run.smithi037.stderr:Operation aborted. 2024-04-25T10:22:42.671 INFO:teuthology.orchestra.run.smithi037.stdout:Is this ok [y/N]: 2024-04-25T10:22:42.672 DEBUG:teuthology.orchestra.run.smithi037:> sudo yum reinstall -y kernel || true 2024-04-25T10:22:43.164 INFO:teuthology.orchestra.run.smithi203.stdout:Package kernel-5.14.0-239.el9.x86_64 is already installed. 2024-04-25T10:22:43.164 INFO:teuthology.orchestra.run.smithi203.stdout:Package kernel-5.14.0-437.el9.x86_64 is already installed. 2024-04-25T10:22:43.188 INFO:teuthology.orchestra.run.smithi037.stdout:Last metadata expiration check: 0:00:04 ago on Thu 25 Apr 2024 10:22:39 AM UTC. 2024-04-25T10:22:43.254 INFO:teuthology.orchestra.run.smithi203.stdout:Dependencies resolved. 2024-04-25T10:22:43.255 INFO:teuthology.orchestra.run.smithi203.stdout:Nothing to do. 2024-04-25T10:22:43.255 INFO:teuthology.orchestra.run.smithi203.stdout:Complete! 2024-04-25T10:22:43.310 INFO:teuthology.orchestra.run.smithi037.stdout:Dependencies resolved. 2024-04-25T10:22:43.310 INFO:teuthology.orchestra.run.smithi037.stdout:================================================================================ 2024-04-25T10:22:43.310 INFO:teuthology.orchestra.run.smithi037.stdout: Package Architecture Version Repository Size 2024-04-25T10:22:43.310 INFO:teuthology.orchestra.run.smithi037.stdout:================================================================================ 2024-04-25T10:22:43.311 INFO:teuthology.orchestra.run.smithi037.stdout:Reinstalling: 2024-04-25T10:22:43.311 INFO:teuthology.orchestra.run.smithi037.stdout: kernel x86_64 5.14.0-437.el9 baseos 255 k 2024-04-25T10:22:43.311 INFO:teuthology.orchestra.run.smithi037.stdout: 2024-04-25T10:22:43.311 INFO:teuthology.orchestra.run.smithi037.stdout:Transaction Summary 2024-04-25T10:22:43.311 INFO:teuthology.orchestra.run.smithi037.stdout:================================================================================ 2024-04-25T10:22:43.311 INFO:teuthology.orchestra.run.smithi037.stdout: 2024-04-25T10:22:43.311 INFO:teuthology.orchestra.run.smithi037.stdout:Total download size: 255 k 2024-04-25T10:22:43.312 INFO:teuthology.orchestra.run.smithi037.stdout:Installed size: 0 2024-04-25T10:22:43.312 INFO:teuthology.orchestra.run.smithi037.stdout:Downloading Packages: 2024-04-25T10:22:43.358 DEBUG:teuthology.orchestra.run.smithi203:> echo no | sudo yum reinstall kernel || true 2024-04-25T10:22:43.841 INFO:teuthology.orchestra.run.smithi037.stdout:kernel-5.14.0-437.el9.x86_64.rpm 1.0 MB/s | 255 kB 00:00 2024-04-25T10:22:43.841 INFO:teuthology.orchestra.run.smithi037.stdout:-------------------------------------------------------------------------------- 2024-04-25T10:22:43.841 INFO:teuthology.orchestra.run.smithi037.stdout:Total 482 kB/s | 255 kB 00:00 2024-04-25T10:22:43.860 INFO:teuthology.orchestra.run.smithi037.stdout:Running transaction check 2024-04-25T10:22:43.861 INFO:teuthology.orchestra.run.smithi203.stdout:Last metadata expiration check: 0:00:02 ago on Thu 25 Apr 2024 10:22:41 AM UTC. 2024-04-25T10:22:43.983 INFO:teuthology.orchestra.run.smithi203.stdout:Dependencies resolved. 2024-04-25T10:22:43.992 INFO:teuthology.orchestra.run.smithi203.stdout:================================================================================ 2024-04-25T10:22:43.992 INFO:teuthology.orchestra.run.smithi203.stdout: Package Architecture Version Repository Size 2024-04-25T10:22:43.993 INFO:teuthology.orchestra.run.smithi203.stdout:================================================================================ 2024-04-25T10:22:43.993 INFO:teuthology.orchestra.run.smithi203.stdout:Reinstalling: 2024-04-25T10:22:43.993 INFO:teuthology.orchestra.run.smithi203.stdout: kernel x86_64 5.14.0-437.el9 baseos 255 k 2024-04-25T10:22:43.993 INFO:teuthology.orchestra.run.smithi203.stdout: 2024-04-25T10:22:43.993 INFO:teuthology.orchestra.run.smithi203.stdout:Transaction Summary 2024-04-25T10:22:43.993 INFO:teuthology.orchestra.run.smithi203.stdout:================================================================================ 2024-04-25T10:22:43.993 INFO:teuthology.orchestra.run.smithi203.stdout: 2024-04-25T10:22:43.993 INFO:teuthology.orchestra.run.smithi203.stdout:Total download size: 255 k 2024-04-25T10:22:43.993 INFO:teuthology.orchestra.run.smithi203.stdout:Installed size: 0 2024-04-25T10:22:43.993 INFO:teuthology.orchestra.run.smithi203.stderr:Operation aborted. 2024-04-25T10:22:44.037 INFO:teuthology.orchestra.run.smithi037.stdout:Transaction check succeeded. 2024-04-25T10:22:44.037 INFO:teuthology.orchestra.run.smithi037.stdout:Running transaction test 2024-04-25T10:22:44.042 INFO:teuthology.orchestra.run.smithi037.stdout:Transaction test succeeded. 2024-04-25T10:22:44.042 INFO:teuthology.orchestra.run.smithi037.stdout:Running transaction 2024-04-25T10:22:44.063 INFO:teuthology.orchestra.run.smithi203.stdout:Is this ok [y/N]: 2024-04-25T10:22:44.063 DEBUG:teuthology.orchestra.run.smithi203:> sudo yum reinstall -y kernel || true 2024-04-25T10:22:44.180 INFO:teuthology.orchestra.run.smithi037.stdout: Preparing : 1/1 2024-04-25T10:22:44.291 INFO:teuthology.orchestra.run.smithi037.stdout: Reinstalling : kernel-5.14.0-437.el9.x86_64 1/2 2024-04-25T10:22:44.291 INFO:teuthology.orchestra.run.smithi037.stdout: Cleanup : kernel-5.14.0-437.el9.x86_64 2/2 2024-04-25T10:22:44.554 INFO:teuthology.orchestra.run.smithi203.stdout:Last metadata expiration check: 0:00:03 ago on Thu 25 Apr 2024 10:22:41 AM UTC. 2024-04-25T10:22:44.670 INFO:teuthology.orchestra.run.smithi203.stdout:Dependencies resolved. 2024-04-25T10:22:44.670 INFO:teuthology.orchestra.run.smithi203.stdout:================================================================================ 2024-04-25T10:22:44.671 INFO:teuthology.orchestra.run.smithi203.stdout: Package Architecture Version Repository Size 2024-04-25T10:22:44.671 INFO:teuthology.orchestra.run.smithi203.stdout:================================================================================ 2024-04-25T10:22:44.671 INFO:teuthology.orchestra.run.smithi203.stdout:Reinstalling: 2024-04-25T10:22:44.671 INFO:teuthology.orchestra.run.smithi203.stdout: kernel x86_64 5.14.0-437.el9 baseos 255 k 2024-04-25T10:22:44.671 INFO:teuthology.orchestra.run.smithi203.stdout: 2024-04-25T10:22:44.671 INFO:teuthology.orchestra.run.smithi203.stdout:Transaction Summary 2024-04-25T10:22:44.671 INFO:teuthology.orchestra.run.smithi203.stdout:================================================================================ 2024-04-25T10:22:44.671 INFO:teuthology.orchestra.run.smithi203.stdout: 2024-04-25T10:22:44.671 INFO:teuthology.orchestra.run.smithi203.stdout:Total download size: 255 k 2024-04-25T10:22:44.671 INFO:teuthology.orchestra.run.smithi203.stdout:Installed size: 0 2024-04-25T10:22:44.671 INFO:teuthology.orchestra.run.smithi203.stdout:Downloading Packages: 2024-04-25T10:22:44.950 INFO:teuthology.orchestra.run.smithi037.stdout: Running scriptlet: kernel-5.14.0-437.el9.x86_64 2/2 2024-04-25T10:22:44.950 INFO:teuthology.orchestra.run.smithi037.stdout: Verifying : kernel-5.14.0-437.el9.x86_64 1/2 2024-04-25T10:22:45.118 INFO:teuthology.orchestra.run.smithi203.stdout:kernel-5.14.0-437.el9.x86_64.rpm 865 kB/s | 255 kB 00:00 2024-04-25T10:22:45.119 INFO:teuthology.orchestra.run.smithi203.stdout:-------------------------------------------------------------------------------- 2024-04-25T10:22:45.119 INFO:teuthology.orchestra.run.smithi203.stdout:Total 570 kB/s | 255 kB 00:00 2024-04-25T10:22:45.139 INFO:teuthology.orchestra.run.smithi203.stdout:Running transaction check 2024-04-25T10:22:45.311 INFO:teuthology.orchestra.run.smithi203.stdout:Transaction check succeeded. 2024-04-25T10:22:45.312 INFO:teuthology.orchestra.run.smithi203.stdout:Running transaction test 2024-04-25T10:22:45.316 INFO:teuthology.orchestra.run.smithi203.stdout:Transaction test succeeded. 2024-04-25T10:22:45.317 INFO:teuthology.orchestra.run.smithi203.stdout:Running transaction 2024-04-25T10:22:45.439 INFO:teuthology.orchestra.run.smithi203.stdout: Preparing : 1/1 2024-04-25T10:22:45.519 INFO:teuthology.orchestra.run.smithi203.stdout: Reinstalling : kernel-5.14.0-437.el9.x86_64 1/2 2024-04-25T10:22:45.519 INFO:teuthology.orchestra.run.smithi203.stdout: Cleanup : kernel-5.14.0-437.el9.x86_64 2/2 2024-04-25T10:22:45.678 INFO:teuthology.orchestra.run.smithi037.stdout: Verifying : kernel-5.14.0-437.el9.x86_64 2/2 2024-04-25T10:22:45.679 INFO:teuthology.orchestra.run.smithi037.stdout: 2024-04-25T10:22:45.679 INFO:teuthology.orchestra.run.smithi037.stdout:Reinstalled: 2024-04-25T10:22:45.679 INFO:teuthology.orchestra.run.smithi037.stdout: kernel-5.14.0-437.el9.x86_64 2024-04-25T10:22:45.679 INFO:teuthology.orchestra.run.smithi037.stdout: 2024-04-25T10:22:45.679 INFO:teuthology.orchestra.run.smithi037.stdout:Complete! 2024-04-25T10:22:45.839 DEBUG:teuthology.orchestra.run.smithi037:> rpm -q kernel | sort -rV | head -n 1 2024-04-25T10:22:45.887 INFO:teuthology.orchestra.run.smithi037.stdout:kernel-5.14.0-437.el9.x86_64 2024-04-25T10:22:45.887 DEBUG:teuthology.task.kernel:get_latest_image_version_rpm: 5.14.0-437.el9.x86_64 2024-04-25T10:22:45.887 INFO:teuthology.task.kernel:Newest distro kernel installed and running 2024-04-25T10:22:45.887 INFO:teuthology.task.kernel:Enabling kdb on mon.a... 2024-04-25T10:22:45.887 DEBUG:teuthology.orchestra.run.smithi037:> echo ttyS1 | sudo tee /sys/module/kgdboc/parameters/kgdboc 2024-04-25T10:22:45.921 INFO:teuthology.orchestra.run.smithi037.stdout:ttyS1 2024-04-25T10:22:45.942 DEBUG:teuthology.parallel:result is None 2024-04-25T10:22:46.320 INFO:teuthology.orchestra.run.smithi203.stdout: Running scriptlet: kernel-5.14.0-437.el9.x86_64 2/2 2024-04-25T10:22:46.321 INFO:teuthology.orchestra.run.smithi203.stdout: Verifying : kernel-5.14.0-437.el9.x86_64 1/2 2024-04-25T10:22:46.997 INFO:teuthology.orchestra.run.smithi203.stdout: Verifying : kernel-5.14.0-437.el9.x86_64 2/2 2024-04-25T10:22:46.997 INFO:teuthology.orchestra.run.smithi203.stdout: 2024-04-25T10:22:46.997 INFO:teuthology.orchestra.run.smithi203.stdout:Reinstalled: 2024-04-25T10:22:46.997 INFO:teuthology.orchestra.run.smithi203.stdout: kernel-5.14.0-437.el9.x86_64 2024-04-25T10:22:46.998 INFO:teuthology.orchestra.run.smithi203.stdout: 2024-04-25T10:22:46.998 INFO:teuthology.orchestra.run.smithi203.stdout:Complete! 2024-04-25T10:22:47.173 DEBUG:teuthology.orchestra.run.smithi203:> rpm -q kernel | sort -rV | head -n 1 2024-04-25T10:22:47.220 INFO:teuthology.orchestra.run.smithi203.stdout:kernel-5.14.0-437.el9.x86_64 2024-04-25T10:22:47.220 DEBUG:teuthology.task.kernel:get_latest_image_version_rpm: 5.14.0-437.el9.x86_64 2024-04-25T10:22:47.220 INFO:teuthology.task.kernel:Newest distro kernel installed and running 2024-04-25T10:22:47.220 INFO:teuthology.task.kernel:Enabling kdb on mon.b... 2024-04-25T10:22:47.221 DEBUG:teuthology.orchestra.run.smithi203:> echo ttyS1 | sudo tee /sys/module/kgdboc/parameters/kgdboc 2024-04-25T10:22:47.260 INFO:teuthology.orchestra.run.smithi203.stdout:ttyS1 2024-04-25T10:22:47.273 DEBUG:teuthology.parallel:result is None 2024-04-25T10:22:47.274 INFO:teuthology.run_tasks:Running task internal.base... 2024-04-25T10:22:47.281 INFO:teuthology.task.internal:Creating test directory... 2024-04-25T10:22:47.281 DEBUG:teuthology.orchestra.run.smithi037:> mkdir -p -m0755 -- /home/ubuntu/cephtest 2024-04-25T10:22:47.284 DEBUG:teuthology.orchestra.run.smithi203:> mkdir -p -m0755 -- /home/ubuntu/cephtest 2024-04-25T10:22:47.328 INFO:teuthology.run_tasks:Running task internal.archive_upload... 2024-04-25T10:22:47.335 INFO:teuthology.run_tasks:Running task internal.archive... 2024-04-25T10:22:47.384 INFO:teuthology.task.internal:Creating archive directory... 2024-04-25T10:22:47.385 DEBUG:teuthology.orchestra.run.smithi037:> install -d -m0755 -- /home/ubuntu/cephtest/archive 2024-04-25T10:22:47.387 DEBUG:teuthology.orchestra.run.smithi203:> install -d -m0755 -- /home/ubuntu/cephtest/archive 2024-04-25T10:22:47.420 INFO:teuthology.run_tasks:Running task internal.coredump... 2024-04-25T10:22:47.429 INFO:teuthology.task.internal:Enabling coredump saving... 2024-04-25T10:22:47.429 DEBUG:teuthology.orchestra.run.smithi037:> install -d -m0755 -- /home/ubuntu/cephtest/archive/coredump && sudo sysctl -w kernel.core_pattern=/home/ubuntu/cephtest/archive/coredump/%t.%p.core && echo kernel.core_pattern=/home/ubuntu/cephtest/archive/coredump/%t.%p.core | sudo tee -a /etc/sysctl.conf 2024-04-25T10:22:47.458 DEBUG:teuthology.orchestra.run.smithi203:> install -d -m0755 -- /home/ubuntu/cephtest/archive/coredump && sudo sysctl -w kernel.core_pattern=/home/ubuntu/cephtest/archive/coredump/%t.%p.core && echo kernel.core_pattern=/home/ubuntu/cephtest/archive/coredump/%t.%p.core | sudo tee -a /etc/sysctl.conf 2024-04-25T10:22:47.488 INFO:teuthology.orchestra.run.smithi037.stdout:kernel.core_pattern = /home/ubuntu/cephtest/archive/coredump/%t.%p.core 2024-04-25T10:22:47.490 INFO:teuthology.orchestra.run.smithi203.stdout:kernel.core_pattern = /home/ubuntu/cephtest/archive/coredump/%t.%p.core 2024-04-25T10:22:47.499 INFO:teuthology.orchestra.run.smithi203.stdout:kernel.core_pattern=/home/ubuntu/cephtest/archive/coredump/%t.%p.core 2024-04-25T10:22:47.499 INFO:teuthology.orchestra.run.smithi037.stdout:kernel.core_pattern=/home/ubuntu/cephtest/archive/coredump/%t.%p.core 2024-04-25T10:22:47.501 INFO:teuthology.run_tasks:Running task internal.sudo... 2024-04-25T10:22:47.508 INFO:teuthology.task.internal:Configuring sudo... 2024-04-25T10:22:47.508 DEBUG:teuthology.orchestra.run.smithi037:> sudo sed -i.orig.teuthology -e 's/^\([^#]*\) \(requiretty\)/\1 !\2/g' -e 's/^\([^#]*\) !\(visiblepw\)/\1 \2/g' /etc/sudoers 2024-04-25T10:22:47.543 DEBUG:teuthology.orchestra.run.smithi203:> sudo sed -i.orig.teuthology -e 's/^\([^#]*\) \(requiretty\)/\1 !\2/g' -e 's/^\([^#]*\) !\(visiblepw\)/\1 \2/g' /etc/sudoers 2024-04-25T10:22:47.568 INFO:teuthology.run_tasks:Running task internal.syslog... 2024-04-25T10:22:47.577 INFO:teuthology.task.internal.syslog:Starting syslog monitoring... 2024-04-25T10:22:47.578 DEBUG:teuthology.orchestra.run.smithi037:> mkdir -p -m0755 -- /home/ubuntu/cephtest/archive/syslog 2024-04-25T10:22:47.612 DEBUG:teuthology.orchestra.run.smithi203:> mkdir -p -m0755 -- /home/ubuntu/cephtest/archive/syslog 2024-04-25T10:22:47.626 DEBUG:teuthology.orchestra.run.smithi037:> install -m 666 /dev/null /home/ubuntu/cephtest/archive/syslog/kern.log 2024-04-25T10:22:47.711 DEBUG:teuthology.orchestra.run.smithi037:> sudo chcon system_u:object_r:var_log_t:s0 /home/ubuntu/cephtest/archive/syslog/kern.log 2024-04-25T10:22:47.764 DEBUG:teuthology.orchestra.run.smithi037:> install -m 666 /dev/null /home/ubuntu/cephtest/archive/syslog/misc.log 2024-04-25T10:22:47.847 DEBUG:teuthology.orchestra.run.smithi037:> sudo chcon system_u:object_r:var_log_t:s0 /home/ubuntu/cephtest/archive/syslog/misc.log 2024-04-25T10:22:47.887 DEBUG:teuthology.orchestra.run.smithi037:> set -ex 2024-04-25T10:22:47.888 DEBUG:teuthology.orchestra.run.smithi037:> sudo dd of=/etc/rsyslog.d/80-cephtest.conf 2024-04-25T10:22:47.953 DEBUG:teuthology.orchestra.run.smithi203:> install -m 666 /dev/null /home/ubuntu/cephtest/archive/syslog/kern.log 2024-04-25T10:22:47.994 DEBUG:teuthology.orchestra.run.smithi203:> sudo chcon system_u:object_r:var_log_t:s0 /home/ubuntu/cephtest/archive/syslog/kern.log 2024-04-25T10:22:48.041 DEBUG:teuthology.orchestra.run.smithi203:> install -m 666 /dev/null /home/ubuntu/cephtest/archive/syslog/misc.log 2024-04-25T10:22:48.122 DEBUG:teuthology.orchestra.run.smithi203:> sudo chcon system_u:object_r:var_log_t:s0 /home/ubuntu/cephtest/archive/syslog/misc.log 2024-04-25T10:22:48.162 DEBUG:teuthology.orchestra.run.smithi203:> set -ex 2024-04-25T10:22:48.162 DEBUG:teuthology.orchestra.run.smithi203:> sudo dd of=/etc/rsyslog.d/80-cephtest.conf 2024-04-25T10:22:48.225 DEBUG:teuthology.orchestra.run.smithi037:> sudo service rsyslog restart 2024-04-25T10:22:48.227 DEBUG:teuthology.orchestra.run.smithi203:> sudo service rsyslog restart 2024-04-25T10:22:48.263 INFO:teuthology.orchestra.run.smithi037.stderr:Redirecting to /bin/systemctl restart rsyslog.service 2024-04-25T10:22:48.305 INFO:teuthology.orchestra.run.smithi203.stderr:Redirecting to /bin/systemctl restart rsyslog.service 2024-04-25T10:22:48.650 INFO:teuthology.run_tasks:Running task internal.timer... 2024-04-25T10:22:48.658 INFO:teuthology.task.internal:Starting timer... 2024-04-25T10:22:48.658 INFO:teuthology.run_tasks:Running task pcp... 2024-04-25T10:22:48.667 INFO:teuthology.run_tasks:Running task selinux... 2024-04-25T10:22:48.676 DEBUG:teuthology.task:Applying overrides for task selinux: {'allowlist': ['scontext=system_u:system_r:getty_t:s0']} 2024-04-25T10:22:48.676 DEBUG:teuthology.orchestra.run.smithi037:> sudo service auditd rotate 2024-04-25T10:22:48.768 INFO:teuthology.orchestra.run.smithi037.stdout:Rotating logs: 2024-04-25T10:22:48.770 DEBUG:teuthology.orchestra.run.smithi203:> sudo service auditd rotate 2024-04-25T10:22:48.825 INFO:teuthology.orchestra.run.smithi203.stdout:Rotating logs: 2024-04-25T10:22:48.827 DEBUG:teuthology.task.selinux:Getting current SELinux state 2024-04-25T10:22:48.827 DEBUG:teuthology.orchestra.run.smithi037:> /usr/sbin/getenforce 2024-04-25T10:22:48.847 INFO:teuthology.orchestra.run.smithi037.stdout:Permissive 2024-04-25T10:22:48.847 DEBUG:teuthology.orchestra.run.smithi203:> /usr/sbin/getenforce 2024-04-25T10:22:48.864 INFO:teuthology.orchestra.run.smithi203.stdout:Permissive 2024-04-25T10:22:48.865 DEBUG:teuthology.task.selinux:Existing SELinux modes: {'ubuntu@smithi037.front.sepia.ceph.com': 'permissive', 'ubuntu@smithi203.front.sepia.ceph.com': 'permissive'} 2024-04-25T10:22:48.865 DEBUG:teuthology.orchestra.run.smithi037:> sudo grep -a 'avc: .*denied' /var/log/audit/audit.log | grep -av -e 'comm="dmidecode"' -e chronyd.service -e 'name="cephtest"' -e scontext=system_u:system_r:nrpe_t:s0 -e scontext=system_u:system_r:pcp_pmlogger_t -e scontext=system_u:system_r:pcp_pmcd_t:s0 -e 'comm="rhsmd"' -e scontext=system_u:system_r:syslogd_t:s0 -e tcontext=system_u:system_r:nrpe_t:s0 -e 'comm="updatedb"' -e 'comm="smartd"' -e 'comm="rhsmcertd-worke"' -e 'comm="setroubleshootd"' -e 'comm="rpm"' -e tcontext=system_u:object_r:container_runtime_exec_t:s0 -e 'comm="ksmtuned"' -e 'comm="sssd"' -e 'comm="sss_cache"' -e context=system_u:system_r:NetworkManager_dispatcher_t:s0 -e scontext=system_u:system_r:getty_t:s0 2024-04-25T10:22:48.914 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-04-25T10:22:48.914 DEBUG:teuthology.orchestra.run.smithi203:> sudo grep -a 'avc: .*denied' /var/log/audit/audit.log | grep -av -e 'comm="dmidecode"' -e chronyd.service -e 'name="cephtest"' -e scontext=system_u:system_r:nrpe_t:s0 -e scontext=system_u:system_r:pcp_pmlogger_t -e scontext=system_u:system_r:pcp_pmcd_t:s0 -e 'comm="rhsmd"' -e scontext=system_u:system_r:syslogd_t:s0 -e tcontext=system_u:system_r:nrpe_t:s0 -e 'comm="updatedb"' -e 'comm="smartd"' -e 'comm="rhsmcertd-worke"' -e 'comm="setroubleshootd"' -e 'comm="rpm"' -e tcontext=system_u:object_r:container_runtime_exec_t:s0 -e 'comm="ksmtuned"' -e 'comm="sssd"' -e 'comm="sss_cache"' -e context=system_u:system_r:NetworkManager_dispatcher_t:s0 -e scontext=system_u:system_r:getty_t:s0 2024-04-25T10:22:48.938 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-04-25T10:22:48.939 INFO:teuthology.task.selinux:Putting SELinux into permissive mode 2024-04-25T10:22:48.939 DEBUG:teuthology.orchestra.run.smithi037:> sudo /usr/sbin/setenforce permissive 2024-04-25T10:22:48.981 DEBUG:teuthology.orchestra.run.smithi203:> sudo /usr/sbin/setenforce permissive 2024-04-25T10:22:49.006 INFO:teuthology.run_tasks:Running task ansible.cephlab... 2024-04-25T10:22:49.016 DEBUG:teuthology.repo_utils:Setting repo remote to https://git.ceph.com/ceph-cm-ansible.git 2024-04-25T10:22:49.021 INFO:teuthology.repo_utils:Fetching git.ceph.com_ceph-cm-ansible_main from origin 2024-04-25T10:22:49.152 DEBUG:teuthology.repo_utils:Resetting repo at /home/teuthworker/src/git.ceph.com_ceph-cm-ansible_main to origin/main 2024-04-25T10:22:49.168 INFO:teuthology.task.ansible:Playbook: [{'import_playbook': 'ansible_managed.yml'}, {'import_playbook': 'teuthology.yml'}, {'hosts': 'testnodes', 'tasks': [{'set_fact': {'ran_from_cephlab_playbook': True}}]}, {'import_playbook': 'testnodes.yml'}, {'import_playbook': 'container-host.yml'}, {'import_playbook': 'cobbler.yml'}, {'import_playbook': 'paddles.yml'}, {'import_playbook': 'pulpito.yml'}, {'hosts': 'testnodes', 'become': True, 'tasks': [{'name': 'Touch /ceph-qa-ready', 'file': {'path': '/ceph-qa-ready', 'state': 'touch'}, 'when': 'ran_from_cephlab_playbook|bool'}]}] 2024-04-25T10:22:49.169 DEBUG:teuthology.task.ansible:Running ansible-playbook -v --extra-vars '{"ansible_ssh_user": "ubuntu"}' -i /etc/ansible/hosts --limit smithi037.front.sepia.ceph.com,smithi203.front.sepia.ceph.com /home/teuthworker/src/git.ceph.com_ceph-cm-ansible_main/cephlab.yml 2024-04-25T10:26:20.906 DEBUG:teuthology.task.ansible:Reconnecting to [Remote(name='ubuntu@smithi037.front.sepia.ceph.com'), Remote(name='ubuntu@smithi203.front.sepia.ceph.com')] 2024-04-25T10:26:20.908 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@smithi037.front.sepia.ceph.com' 2024-04-25T10:26:20.909 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi037.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-04-25T10:26:21.013 DEBUG:teuthology.orchestra.run.smithi037:> true 2024-04-25T10:26:21.090 INFO:teuthology.orchestra.remote:Successfully reconnected to host 'ubuntu@smithi037.front.sepia.ceph.com' 2024-04-25T10:26:21.091 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@smithi203.front.sepia.ceph.com' 2024-04-25T10:26:21.091 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi203.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-04-25T10:26:21.164 DEBUG:teuthology.orchestra.run.smithi203:> true 2024-04-25T10:26:21.248 INFO:teuthology.orchestra.remote:Successfully reconnected to host 'ubuntu@smithi203.front.sepia.ceph.com' 2024-04-25T10:26:21.248 INFO:teuthology.run_tasks:Running task clock... 2024-04-25T10:26:21.263 INFO:teuthology.task.clock:Syncing clocks and checking initial clock skew... 2024-04-25T10:26:21.263 INFO:teuthology.orchestra.run:Running command with timeout 360 2024-04-25T10:26:21.264 DEBUG:teuthology.orchestra.run.smithi037:> sudo systemctl stop ntp.service || sudo systemctl stop ntpd.service || sudo systemctl stop chronyd.service ; sudo ntpd -gq || sudo chronyc makestep ; sudo systemctl start ntp.service || sudo systemctl start ntpd.service || sudo systemctl start chronyd.service ; PATH=/usr/bin:/usr/sbin ntpq -p || PATH=/usr/bin:/usr/sbin chronyc sources || true 2024-04-25T10:26:21.266 INFO:teuthology.orchestra.run:Running command with timeout 360 2024-04-25T10:26:21.266 DEBUG:teuthology.orchestra.run.smithi203:> sudo systemctl stop ntp.service || sudo systemctl stop ntpd.service || sudo systemctl stop chronyd.service ; sudo ntpd -gq || sudo chronyc makestep ; sudo systemctl start ntp.service || sudo systemctl start ntpd.service || sudo systemctl start chronyd.service ; PATH=/usr/bin:/usr/sbin ntpq -p || PATH=/usr/bin:/usr/sbin chronyc sources || true 2024-04-25T10:26:21.296 INFO:teuthology.orchestra.run.smithi037.stderr:Failed to stop ntp.service: Unit ntp.service not loaded. 2024-04-25T10:26:21.311 INFO:teuthology.orchestra.run.smithi037.stderr:Failed to stop ntpd.service: Unit ntpd.service not loaded. 2024-04-25T10:26:21.315 INFO:teuthology.orchestra.run.smithi203.stderr:Failed to stop ntp.service: Unit ntp.service not loaded. 2024-04-25T10:26:21.329 INFO:teuthology.orchestra.run.smithi203.stderr:Failed to stop ntpd.service: Unit ntpd.service not loaded. 2024-04-25T10:26:21.335 INFO:teuthology.orchestra.run.smithi037.stderr:sudo: ntpd: command not found 2024-04-25T10:26:21.347 INFO:teuthology.orchestra.run.smithi037.stdout:506 Cannot talk to daemon 2024-04-25T10:26:21.351 INFO:teuthology.orchestra.run.smithi203.stderr:sudo: ntpd: command not found 2024-04-25T10:26:21.361 INFO:teuthology.orchestra.run.smithi037.stderr:Failed to start ntp.service: Unit ntp.service not found. 2024-04-25T10:26:21.361 INFO:teuthology.orchestra.run.smithi203.stdout:506 Cannot talk to daemon 2024-04-25T10:26:21.374 INFO:teuthology.orchestra.run.smithi037.stderr:Failed to start ntpd.service: Unit ntpd.service not found. 2024-04-25T10:26:21.375 INFO:teuthology.orchestra.run.smithi203.stderr:Failed to start ntp.service: Unit ntp.service not found. 2024-04-25T10:26:21.388 INFO:teuthology.orchestra.run.smithi203.stderr:Failed to start ntpd.service: Unit ntpd.service not found. 2024-04-25T10:26:21.429 INFO:teuthology.orchestra.run.smithi037.stderr:bash: line 1: ntpq: command not found 2024-04-25T10:26:21.432 INFO:teuthology.orchestra.run.smithi037.stdout:MS Name/IP address Stratum Poll Reach LastRx Last sample 2024-04-25T10:26:21.432 INFO:teuthology.orchestra.run.smithi037.stdout:=============================================================================== 2024-04-25T10:26:21.432 INFO:teuthology.orchestra.run.smithi037.stdout:^? hv01.front.sepia.ceph.com 0 6 0 - +0ns[ +0ns] +/- 0ns 2024-04-25T10:26:21.432 INFO:teuthology.orchestra.run.smithi037.stdout:^? hv02.front.sepia.ceph.com 0 6 0 - +0ns[ +0ns] +/- 0ns 2024-04-25T10:26:21.432 INFO:teuthology.orchestra.run.smithi037.stdout:^? hv03.front.sepia.ceph.com 0 6 0 - +0ns[ +0ns] +/- 0ns 2024-04-25T10:26:21.433 INFO:teuthology.orchestra.run.smithi037.stdout:^? hv04.front.sepia.ceph.com 0 6 0 - +0ns[ +0ns] +/- 0ns 2024-04-25T10:26:21.442 INFO:teuthology.orchestra.run.smithi203.stderr:bash: line 1: ntpq: command not found 2024-04-25T10:26:21.446 INFO:teuthology.orchestra.run.smithi203.stdout:MS Name/IP address Stratum Poll Reach LastRx Last sample 2024-04-25T10:26:21.446 INFO:teuthology.orchestra.run.smithi203.stdout:=============================================================================== 2024-04-25T10:26:21.446 INFO:teuthology.orchestra.run.smithi203.stdout:^? hv01.front.sepia.ceph.com 0 6 0 - +0ns[ +0ns] +/- 0ns 2024-04-25T10:26:21.446 INFO:teuthology.orchestra.run.smithi203.stdout:^? hv02.front.sepia.ceph.com 0 6 0 - +0ns[ +0ns] +/- 0ns 2024-04-25T10:26:21.446 INFO:teuthology.orchestra.run.smithi203.stdout:^? hv03.front.sepia.ceph.com 0 6 0 - +0ns[ +0ns] +/- 0ns 2024-04-25T10:26:21.446 INFO:teuthology.orchestra.run.smithi203.stdout:^? hv04.front.sepia.ceph.com 0 6 0 - +0ns[ +0ns] +/- 0ns 2024-04-25T10:26:21.447 INFO:teuthology.run_tasks:Running task install... 2024-04-25T10:26:21.455 DEBUG:teuthology.task.install:project ceph 2024-04-25T10:26:21.456 DEBUG:teuthology.task.install:INSTALL overrides: {'ceph': {'flavor': 'default', 'sha1': '994e2f20102a20fa9a27966c14b625cc01884df8'}} 2024-04-25T10:26:21.456 DEBUG:teuthology.task.install:config {'extra_packages': {'deb': ['python3-cephfs', 'cephfs-shell', 'cephfs-top', 'cephfs-mirror'], 'rpm': ['python3-cephfs', 'cephfs-shell', 'cephfs-top', 'cephfs-mirror']}, 'extra_system_packages': {'deb': ['bison', 'flex', 'libelf-dev', 'libssl-dev', 'network-manager', 'iproute2', 'util-linux', 'dump', 'indent', 'libaio-dev', 'libtool-bin', 'uuid-dev', 'xfslibs-dev', 'postgresql', 'postgresql-client', 'postgresql-common', 'postgresql-contrib'], 'rpm': ['bison', 'flex', 'elfutils-libelf-devel', 'openssl-devel', 'NetworkManager', 'iproute', 'util-linux', 'libacl-devel', 'libaio-devel', 'libattr-devel', 'libtool', 'libuuid-devel', 'xfsdump', 'xfsprogs', 'xfsprogs-devel', 'libaio-devel', 'libtool', 'libuuid-devel', 'xfsprogs-devel', 'postgresql', 'postgresql-server', 'postgresql-contrib']}, 'flavor': 'default', 'sha1': '994e2f20102a20fa9a27966c14b625cc01884df8'} 2024-04-25T10:26:21.456 INFO:teuthology.task.install:Using flavor: default 2024-04-25T10:26:21.462 DEBUG:teuthology.task.install:Package list is: {'deb': ['ceph', 'cephadm', 'ceph-mds', 'ceph-mgr', 'ceph-common', 'ceph-fuse', 'ceph-test', 'ceph-volume', 'radosgw', 'python3-rados', 'python3-rgw', 'python3-cephfs', 'python3-rbd', 'libcephfs2', 'libcephfs-dev', 'librados2', 'librbd1', 'rbd-fuse'], 'rpm': ['ceph-radosgw', 'ceph-test', 'ceph', 'ceph-base', 'cephadm', 'ceph-immutable-object-cache', 'ceph-mgr', 'ceph-mgr-dashboard', 'ceph-mgr-diskprediction-local', 'ceph-mgr-rook', 'ceph-mgr-cephadm', 'ceph-fuse', 'ceph-volume', 'librados-devel', 'libcephfs2', 'libcephfs-devel', 'librados2', 'librbd1', 'python3-rados', 'python3-rgw', 'python3-cephfs', 'python3-rbd', 'rbd-fuse', 'rbd-mirror', 'rbd-nbd']} 2024-04-25T10:26:21.462 INFO:teuthology.task.install:extra packages: {'deb': ['python3-cephfs', 'cephfs-shell', 'cephfs-top', 'cephfs-mirror'], 'rpm': ['python3-cephfs', 'cephfs-shell', 'cephfs-top', 'cephfs-mirror']} 2024-04-25T10:26:21.462 DEBUG:teuthology.task.install.rpm:_update_package_list_and_install: config is {'branch': None, 'cleanup': None, 'debuginfo': None, 'downgrade_packages': [], 'exclude_packages': [], 'extra_packages': {'deb': ['python3-cephfs', 'cephfs-shell', 'cephfs-top', 'cephfs-mirror'], 'rpm': ['python3-cephfs', 'cephfs-shell', 'cephfs-top', 'cephfs-mirror']}, 'extra_system_packages': {'deb': ['bison', 'flex', 'libelf-dev', 'libssl-dev', 'network-manager', 'iproute2', 'util-linux', 'dump', 'indent', 'libaio-dev', 'libtool-bin', 'uuid-dev', 'xfslibs-dev', 'postgresql', 'postgresql-client', 'postgresql-common', 'postgresql-contrib'], 'rpm': ['bison', 'flex', 'elfutils-libelf-devel', 'openssl-devel', 'NetworkManager', 'iproute', 'util-linux', 'libacl-devel', 'libaio-devel', 'libattr-devel', 'libtool', 'libuuid-devel', 'xfsdump', 'xfsprogs', 'xfsprogs-devel', 'libaio-devel', 'libtool', 'libuuid-devel', 'xfsprogs-devel', 'postgresql', 'postgresql-server', 'postgresql-contrib']}, 'extras': None, 'enable_coprs': [], 'flavor': 'default', 'install_ceph_packages': True, 'packages': {}, 'project': 'ceph', 'repos_only': False, 'sha1': '994e2f20102a20fa9a27966c14b625cc01884df8', 'tag': None, 'wait_for_package': False} 2024-04-25T10:26:21.463 DEBUG:teuthology.packaging:Querying https://shaman.ceph.com/api/search?status=ready&project=ceph&flavor=default&distros=centos%2F9%2Fx86_64&sha1=994e2f20102a20fa9a27966c14b625cc01884df8 2024-04-25T10:26:21.465 DEBUG:teuthology.task.install.rpm:_update_package_list_and_install: config is {'branch': None, 'cleanup': None, 'debuginfo': None, 'downgrade_packages': [], 'exclude_packages': [], 'extra_packages': {'deb': ['python3-cephfs', 'cephfs-shell', 'cephfs-top', 'cephfs-mirror'], 'rpm': ['python3-cephfs', 'cephfs-shell', 'cephfs-top', 'cephfs-mirror']}, 'extra_system_packages': {'deb': ['bison', 'flex', 'libelf-dev', 'libssl-dev', 'network-manager', 'iproute2', 'util-linux', 'dump', 'indent', 'libaio-dev', 'libtool-bin', 'uuid-dev', 'xfslibs-dev', 'postgresql', 'postgresql-client', 'postgresql-common', 'postgresql-contrib'], 'rpm': ['bison', 'flex', 'elfutils-libelf-devel', 'openssl-devel', 'NetworkManager', 'iproute', 'util-linux', 'libacl-devel', 'libaio-devel', 'libattr-devel', 'libtool', 'libuuid-devel', 'xfsdump', 'xfsprogs', 'xfsprogs-devel', 'libaio-devel', 'libtool', 'libuuid-devel', 'xfsprogs-devel', 'postgresql', 'postgresql-server', 'postgresql-contrib']}, 'extras': None, 'enable_coprs': [], 'flavor': 'default', 'install_ceph_packages': True, 'packages': {}, 'project': 'ceph', 'repos_only': False, 'sha1': '994e2f20102a20fa9a27966c14b625cc01884df8', 'tag': None, 'wait_for_package': False} 2024-04-25T10:26:21.465 DEBUG:teuthology.packaging:Querying https://shaman.ceph.com/api/search?status=ready&project=ceph&flavor=default&distros=centos%2F9%2Fx86_64&sha1=994e2f20102a20fa9a27966c14b625cc01884df8 2024-04-25T10:26:21.608 INFO:teuthology.task.install.rpm:Pulling from https://3.chacra.ceph.com/r/ceph/wip-pdonnell-testing-20240425.015853-debug/994e2f20102a20fa9a27966c14b625cc01884df8/centos/9/flavors/default/ 2024-04-25T10:26:21.608 INFO:teuthology.task.install.rpm:Package version is 19.0.0-3291.g994e2f20 2024-04-25T10:26:21.719 INFO:teuthology.task.install.rpm:Pulling from https://3.chacra.ceph.com/r/ceph/wip-pdonnell-testing-20240425.015853-debug/994e2f20102a20fa9a27966c14b625cc01884df8/centos/9/flavors/default/ 2024-04-25T10:26:21.719 INFO:teuthology.task.install.rpm:Package version is 19.0.0-3291.g994e2f20 2024-04-25T10:26:21.840 INFO:teuthology.packaging:Writing yum repo: [ceph] name=ceph packages for $basearch baseurl=https://3.chacra.ceph.com/r/ceph/wip-pdonnell-testing-20240425.015853-debug/994e2f20102a20fa9a27966c14b625cc01884df8/centos/9/flavors/default/$basearch enabled=1 gpgcheck=0 type=rpm-md [ceph-noarch] name=ceph noarch packages baseurl=https://3.chacra.ceph.com/r/ceph/wip-pdonnell-testing-20240425.015853-debug/994e2f20102a20fa9a27966c14b625cc01884df8/centos/9/flavors/default/noarch enabled=1 gpgcheck=0 type=rpm-md [ceph-source] name=ceph source packages baseurl=https://3.chacra.ceph.com/r/ceph/wip-pdonnell-testing-20240425.015853-debug/994e2f20102a20fa9a27966c14b625cc01884df8/centos/9/flavors/default/SRPMS enabled=1 gpgcheck=0 type=rpm-md 2024-04-25T10:26:21.840 DEBUG:teuthology.orchestra.run.smithi037:> set -ex 2024-04-25T10:26:21.841 DEBUG:teuthology.orchestra.run.smithi037:> sudo dd of=/etc/yum.repos.d/ceph.repo 2024-04-25T10:26:21.868 INFO:teuthology.task.install.rpm:Installing packages: ceph-radosgw, ceph-test, ceph, ceph-base, cephadm, ceph-immutable-object-cache, ceph-mgr, ceph-mgr-dashboard, ceph-mgr-diskprediction-local, ceph-mgr-rook, ceph-mgr-cephadm, ceph-fuse, ceph-volume, librados-devel, libcephfs2, libcephfs-devel, librados2, librbd1, python3-rados, python3-rgw, python3-cephfs, python3-rbd, rbd-fuse, rbd-mirror, rbd-nbd, python3-cephfs, cephfs-shell, cephfs-top, cephfs-mirror, bison, flex, elfutils-libelf-devel, openssl-devel, NetworkManager, iproute, util-linux, libacl-devel, libaio-devel, libattr-devel, libtool, libuuid-devel, xfsdump, xfsprogs, xfsprogs-devel, libaio-devel, libtool, libuuid-devel, xfsprogs-devel, postgresql, postgresql-server, postgresql-contrib, bison, flex, elfutils-libelf-devel, openssl-devel, NetworkManager, iproute, util-linux, libacl-devel, libaio-devel, libattr-devel, libtool, libuuid-devel, xfsdump, xfsprogs, xfsprogs-devel, libaio-devel, libtool, libuuid-devel, xfsprogs-devel, postgresql, postgresql-server, postgresql-contrib, bison, flex, elfutils-libelf-devel, openssl-devel, NetworkManager, iproute, util-linux, libacl-devel, libaio-devel, libattr-devel, libtool, libuuid-devel, xfsdump, xfsprogs, xfsprogs-devel, libaio-devel, libtool, libuuid-devel, xfsprogs-devel, postgresql, postgresql-server, postgresql-contrib on remote rpm x86_64 2024-04-25T10:26:21.868 DEBUG:teuthology.orchestra.run.smithi037:> if test -f /etc/yum.repos.d/ceph.repo ; then sudo sed -i -e ':a;N;$!ba;s/enabled=1\ngpg/enabled=1\npriority=1\ngpg/g' -e 's;ref/[a-zA-Z0-9_-]*/;sha1/994e2f20102a20fa9a27966c14b625cc01884df8/;g' /etc/yum.repos.d/ceph.repo ; fi 2024-04-25T10:26:21.934 DEBUG:teuthology.orchestra.run.smithi037:> sudo touch -a /etc/yum/pluginconf.d/priorities.conf ; test -e /etc/yum/pluginconf.d/priorities.conf.orig || sudo cp -af /etc/yum/pluginconf.d/priorities.conf /etc/yum/pluginconf.d/priorities.conf.orig 2024-04-25T10:26:22.010 DEBUG:teuthology.orchestra.run.smithi037:> grep check_obsoletes /etc/yum/pluginconf.d/priorities.conf && sudo sed -i 's/check_obsoletes.*0/check_obsoletes = 1/g' /etc/yum/pluginconf.d/priorities.conf || echo 'check_obsoletes = 1' | sudo tee -a /etc/yum/pluginconf.d/priorities.conf 2024-04-25T10:26:22.047 INFO:teuthology.packaging:Writing yum repo: [ceph] name=ceph packages for $basearch baseurl=https://3.chacra.ceph.com/r/ceph/wip-pdonnell-testing-20240425.015853-debug/994e2f20102a20fa9a27966c14b625cc01884df8/centos/9/flavors/default/$basearch enabled=1 gpgcheck=0 type=rpm-md [ceph-noarch] name=ceph noarch packages baseurl=https://3.chacra.ceph.com/r/ceph/wip-pdonnell-testing-20240425.015853-debug/994e2f20102a20fa9a27966c14b625cc01884df8/centos/9/flavors/default/noarch enabled=1 gpgcheck=0 type=rpm-md [ceph-source] name=ceph source packages baseurl=https://3.chacra.ceph.com/r/ceph/wip-pdonnell-testing-20240425.015853-debug/994e2f20102a20fa9a27966c14b625cc01884df8/centos/9/flavors/default/SRPMS enabled=1 gpgcheck=0 type=rpm-md 2024-04-25T10:26:22.048 DEBUG:teuthology.orchestra.run.smithi203:> set -ex 2024-04-25T10:26:22.048 DEBUG:teuthology.orchestra.run.smithi203:> sudo dd of=/etc/yum.repos.d/ceph.repo 2024-04-25T10:26:22.073 INFO:teuthology.task.install.rpm:Installing packages: ceph-radosgw, ceph-test, ceph, ceph-base, cephadm, ceph-immutable-object-cache, ceph-mgr, ceph-mgr-dashboard, ceph-mgr-diskprediction-local, ceph-mgr-rook, ceph-mgr-cephadm, ceph-fuse, ceph-volume, librados-devel, libcephfs2, libcephfs-devel, librados2, librbd1, python3-rados, python3-rgw, python3-cephfs, python3-rbd, rbd-fuse, rbd-mirror, rbd-nbd, python3-cephfs, cephfs-shell, cephfs-top, cephfs-mirror, bison, flex, elfutils-libelf-devel, openssl-devel, NetworkManager, iproute, util-linux, libacl-devel, libaio-devel, libattr-devel, libtool, libuuid-devel, xfsdump, xfsprogs, xfsprogs-devel, libaio-devel, libtool, libuuid-devel, xfsprogs-devel, postgresql, postgresql-server, postgresql-contrib, bison, flex, elfutils-libelf-devel, openssl-devel, NetworkManager, iproute, util-linux, libacl-devel, libaio-devel, libattr-devel, libtool, libuuid-devel, xfsdump, xfsprogs, xfsprogs-devel, libaio-devel, libtool, libuuid-devel, xfsprogs-devel, postgresql, postgresql-server, postgresql-contrib, bison, flex, elfutils-libelf-devel, openssl-devel, NetworkManager, iproute, util-linux, libacl-devel, libaio-devel, libattr-devel, libtool, libuuid-devel, xfsdump, xfsprogs, xfsprogs-devel, libaio-devel, libtool, libuuid-devel, xfsprogs-devel, postgresql, postgresql-server, postgresql-contrib, bison, flex, elfutils-libelf-devel, openssl-devel, NetworkManager, iproute, util-linux, libacl-devel, libaio-devel, libattr-devel, libtool, libuuid-devel, xfsdump, xfsprogs, xfsprogs-devel, libaio-devel, libtool, libuuid-devel, xfsprogs-devel, postgresql, postgresql-server, postgresql-contrib on remote rpm x86_64 2024-04-25T10:26:22.073 DEBUG:teuthology.orchestra.run.smithi203:> if test -f /etc/yum.repos.d/ceph.repo ; then sudo sed -i -e ':a;N;$!ba;s/enabled=1\ngpg/enabled=1\npriority=1\ngpg/g' -e 's;ref/[a-zA-Z0-9_-]*/;sha1/994e2f20102a20fa9a27966c14b625cc01884df8/;g' /etc/yum.repos.d/ceph.repo ; fi 2024-04-25T10:26:22.076 INFO:teuthology.orchestra.run.smithi037.stdout:check_obsoletes = 1 2024-04-25T10:26:22.077 DEBUG:teuthology.orchestra.run.smithi037:> sudo yum clean all 2024-04-25T10:26:22.137 DEBUG:teuthology.orchestra.run.smithi203:> sudo touch -a /etc/yum/pluginconf.d/priorities.conf ; test -e /etc/yum/pluginconf.d/priorities.conf.orig || sudo cp -af /etc/yum/pluginconf.d/priorities.conf /etc/yum/pluginconf.d/priorities.conf.orig 2024-04-25T10:26:22.211 DEBUG:teuthology.orchestra.run.smithi203:> grep check_obsoletes /etc/yum/pluginconf.d/priorities.conf && sudo sed -i 's/check_obsoletes.*0/check_obsoletes = 1/g' /etc/yum/pluginconf.d/priorities.conf || echo 'check_obsoletes = 1' | sudo tee -a /etc/yum/pluginconf.d/priorities.conf 2024-04-25T10:26:22.273 INFO:teuthology.orchestra.run.smithi203.stdout:check_obsoletes = 1 2024-04-25T10:26:22.275 DEBUG:teuthology.orchestra.run.smithi203:> sudo yum clean all 2024-04-25T10:26:22.311 INFO:teuthology.orchestra.run.smithi037.stdout:48 files removed 2024-04-25T10:26:22.338 DEBUG:teuthology.orchestra.run.smithi037:> sudo yum -y install ceph-radosgw ceph-test ceph ceph-base cephadm ceph-immutable-object-cache ceph-mgr ceph-mgr-dashboard ceph-mgr-diskprediction-local ceph-mgr-rook ceph-mgr-cephadm ceph-fuse ceph-volume librados-devel libcephfs2 libcephfs-devel librados2 librbd1 python3-rados python3-rgw python3-cephfs python3-rbd rbd-fuse rbd-mirror rbd-nbd python3-cephfs cephfs-shell cephfs-top cephfs-mirror bison flex elfutils-libelf-devel openssl-devel NetworkManager iproute util-linux libacl-devel libaio-devel libattr-devel libtool libuuid-devel xfsdump xfsprogs xfsprogs-devel libaio-devel libtool libuuid-devel xfsprogs-devel postgresql postgresql-server postgresql-contrib bison flex elfutils-libelf-devel openssl-devel NetworkManager iproute util-linux libacl-devel libaio-devel libattr-devel libtool libuuid-devel xfsdump xfsprogs xfsprogs-devel libaio-devel libtool libuuid-devel xfsprogs-devel postgresql postgresql-server postgresql-contrib bison flex elfutils-libelf-devel openssl-devel NetworkManager iproute util-linux libacl-devel libaio-devel libattr-devel libtool libuuid-devel xfsdump xfsprogs xfsprogs-devel libaio-devel libtool libuuid-devel xfsprogs-devel postgresql postgresql-server postgresql-contrib bison flex elfutils-libelf-devel openssl-devel NetworkManager iproute util-linux libacl-devel libaio-devel libattr-devel libtool libuuid-devel xfsdump xfsprogs xfsprogs-devel libaio-devel libtool libuuid-devel xfsprogs-devel postgresql postgresql-server postgresql-contrib 2024-04-25T10:26:22.497 INFO:teuthology.orchestra.run.smithi203.stdout:48 files removed 2024-04-25T10:26:22.524 DEBUG:teuthology.orchestra.run.smithi203:> sudo yum -y install ceph-radosgw ceph-test ceph ceph-base cephadm ceph-immutable-object-cache ceph-mgr ceph-mgr-dashboard ceph-mgr-diskprediction-local ceph-mgr-rook ceph-mgr-cephadm ceph-fuse ceph-volume librados-devel libcephfs2 libcephfs-devel librados2 librbd1 python3-rados python3-rgw python3-cephfs python3-rbd rbd-fuse rbd-mirror rbd-nbd python3-cephfs cephfs-shell cephfs-top cephfs-mirror bison flex elfutils-libelf-devel openssl-devel NetworkManager iproute util-linux libacl-devel libaio-devel libattr-devel libtool libuuid-devel xfsdump xfsprogs xfsprogs-devel libaio-devel libtool libuuid-devel xfsprogs-devel postgresql postgresql-server postgresql-contrib bison flex elfutils-libelf-devel openssl-devel NetworkManager iproute util-linux libacl-devel libaio-devel libattr-devel libtool libuuid-devel xfsdump xfsprogs xfsprogs-devel libaio-devel libtool libuuid-devel xfsprogs-devel postgresql postgresql-server postgresql-contrib bison flex elfutils-libelf-devel openssl-devel NetworkManager iproute util-linux libacl-devel libaio-devel libattr-devel libtool libuuid-devel xfsdump xfsprogs xfsprogs-devel libaio-devel libtool libuuid-devel xfsprogs-devel postgresql postgresql-server postgresql-contrib bison flex elfutils-libelf-devel openssl-devel NetworkManager iproute util-linux libacl-devel libaio-devel libattr-devel libtool libuuid-devel xfsdump xfsprogs xfsprogs-devel libaio-devel libtool libuuid-devel xfsprogs-devel postgresql postgresql-server postgresql-contrib 2024-04-25T10:26:23.533 INFO:teuthology.orchestra.run.smithi037.stdout:ceph packages for x86_64 91 kB/s | 84 kB 00:00 2024-04-25T10:26:23.631 INFO:teuthology.orchestra.run.smithi203.stdout:ceph packages for x86_64 99 kB/s | 84 kB 00:00 2024-04-25T10:26:24.217 INFO:teuthology.orchestra.run.smithi037.stdout:ceph noarch packages 18 kB/s | 12 kB 00:00 2024-04-25T10:26:24.265 INFO:teuthology.orchestra.run.smithi203.stdout:ceph noarch packages 20 kB/s | 12 kB 00:00 2024-04-25T10:26:24.726 INFO:teuthology.orchestra.run.smithi037.stdout:ceph source packages 4.0 kB/s | 1.9 kB 00:00 2024-04-25T10:26:24.799 INFO:teuthology.orchestra.run.smithi203.stdout:ceph source packages 3.8 kB/s | 1.9 kB 00:00 2024-04-25T10:26:26.874 INFO:teuthology.orchestra.run.smithi203.stdout:CentOS Stream 9 - BaseOS 3.9 MB/s | 8.0 MB 00:02 2024-04-25T10:26:27.118 INFO:teuthology.orchestra.run.smithi037.stdout:CentOS Stream 9 - BaseOS 3.4 MB/s | 8.0 MB 00:02 2024-04-25T10:26:29.317 INFO:teuthology.orchestra.run.smithi203.stdout:CentOS Stream 9 - AppStream 14 MB/s | 19 MB 00:01 2024-04-25T10:26:30.960 INFO:teuthology.orchestra.run.smithi037.stdout:CentOS Stream 9 - AppStream 6.9 MB/s | 19 MB 00:02 2024-04-25T10:26:35.303 INFO:teuthology.orchestra.run.smithi037.stdout:CentOS Stream 9 - CRB 9.6 MB/s | 6.2 MB 00:00 2024-04-25T10:26:36.543 INFO:teuthology.orchestra.run.smithi203.stdout:CentOS Stream 9 - CRB 1.7 MB/s | 6.2 MB 00:03 2024-04-25T10:26:37.054 INFO:teuthology.orchestra.run.smithi037.stdout:CentOS Stream 9 - Extras packages 30 kB/s | 16 kB 00:00 2024-04-25T10:26:37.935 INFO:teuthology.orchestra.run.smithi203.stdout:CentOS Stream 9 - Extras packages 68 kB/s | 16 kB 00:00 2024-04-25T10:26:39.066 INFO:teuthology.orchestra.run.smithi037.stdout:Extra Packages for Enterprise Linux 11 MB/s | 21 MB 00:01 2024-04-25T10:26:42.248 INFO:teuthology.orchestra.run.smithi203.stdout:Extra Packages for Enterprise Linux 5.0 MB/s | 21 MB 00:04 2024-04-25T10:26:44.289 INFO:teuthology.orchestra.run.smithi037.stdout:lab-extras 30 kB/s | 1.7 kB 00:00 2024-04-25T10:26:46.499 INFO:teuthology.orchestra.run.smithi037.stdout:Package librados2-2:16.2.4-5.el9.x86_64 is already installed. 2024-04-25T10:26:46.499 INFO:teuthology.orchestra.run.smithi037.stdout:Package librbd1-2:16.2.4-5.el9.x86_64 is already installed. 2024-04-25T10:26:46.510 INFO:teuthology.orchestra.run.smithi037.stdout:Package NetworkManager-1:1.47.4-1.el9.x86_64 is already installed. 2024-04-25T10:26:46.511 INFO:teuthology.orchestra.run.smithi037.stdout:Package iproute-6.2.0-5.el9.x86_64 is already installed. 2024-04-25T10:26:46.512 INFO:teuthology.orchestra.run.smithi037.stdout:Package util-linux-2.37.4-18.el9.x86_64 is already installed. 2024-04-25T10:26:46.513 INFO:teuthology.orchestra.run.smithi037.stdout:Package libacl-devel-2.3.1-4.el9.x86_64 is already installed. 2024-04-25T10:26:46.514 INFO:teuthology.orchestra.run.smithi037.stdout:Package libattr-devel-2.5.1-3.el9.x86_64 is already installed. 2024-04-25T10:26:46.517 INFO:teuthology.orchestra.run.smithi037.stdout:Package xfsdump-3.1.12-4.el9.x86_64 is already installed. 2024-04-25T10:26:46.517 INFO:teuthology.orchestra.run.smithi037.stdout:Package xfsprogs-6.3.0-1.el9.x86_64 is already installed. 2024-04-25T10:26:46.581 INFO:teuthology.orchestra.run.smithi037.stdout:Dependencies resolved. 2024-04-25T10:26:46.591 INFO:teuthology.orchestra.run.smithi037.stdout:======================================================================================= 2024-04-25T10:26:46.591 INFO:teuthology.orchestra.run.smithi037.stdout: Package Arch Version Repository Size 2024-04-25T10:26:46.591 INFO:teuthology.orchestra.run.smithi037.stdout:======================================================================================= 2024-04-25T10:26:46.591 INFO:teuthology.orchestra.run.smithi037.stdout:Installing: 2024-04-25T10:26:46.591 INFO:teuthology.orchestra.run.smithi037.stdout: bison x86_64 3.7.4-5.el9 appstream 943 k 2024-04-25T10:26:46.591 INFO:teuthology.orchestra.run.smithi037.stdout: ceph x86_64 2:19.0.0-3291.g994e2f20.el9 ceph 6.6 k 2024-04-25T10:26:46.591 INFO:teuthology.orchestra.run.smithi037.stdout: ceph-base x86_64 2:19.0.0-3291.g994e2f20.el9 ceph 5.4 M 2024-04-25T10:26:46.591 INFO:teuthology.orchestra.run.smithi037.stdout: ceph-fuse x86_64 2:19.0.0-3291.g994e2f20.el9 ceph 863 k 2024-04-25T10:26:46.592 INFO:teuthology.orchestra.run.smithi037.stdout: ceph-immutable-object-cache x86_64 2:19.0.0-3291.g994e2f20.el9 ceph 147 k 2024-04-25T10:26:46.592 INFO:teuthology.orchestra.run.smithi037.stdout: ceph-mgr x86_64 2:19.0.0-3291.g994e2f20.el9 ceph 1.5 M 2024-04-25T10:26:46.592 INFO:teuthology.orchestra.run.smithi037.stdout: ceph-mgr-cephadm noarch 2:19.0.0-3291.g994e2f20.el9 ceph-noarch 142 k 2024-04-25T10:26:46.592 INFO:teuthology.orchestra.run.smithi037.stdout: ceph-mgr-dashboard noarch 2:19.0.0-3291.g994e2f20.el9 ceph-noarch 3.5 M 2024-04-25T10:26:46.592 INFO:teuthology.orchestra.run.smithi037.stdout: ceph-mgr-diskprediction-local noarch 2:19.0.0-3291.g994e2f20.el9 ceph-noarch 7.4 M 2024-04-25T10:26:46.592 INFO:teuthology.orchestra.run.smithi037.stdout: ceph-mgr-rook noarch 2:19.0.0-3291.g994e2f20.el9 ceph-noarch 49 k 2024-04-25T10:26:46.592 INFO:teuthology.orchestra.run.smithi037.stdout: ceph-radosgw x86_64 2:19.0.0-3291.g994e2f20.el9 ceph 10 M 2024-04-25T10:26:46.592 INFO:teuthology.orchestra.run.smithi037.stdout: ceph-test x86_64 2:19.0.0-3291.g994e2f20.el9 ceph 43 M 2024-04-25T10:26:46.592 INFO:teuthology.orchestra.run.smithi037.stdout: ceph-volume noarch 2:19.0.0-3291.g994e2f20.el9 ceph-noarch 273 k 2024-04-25T10:26:46.592 INFO:teuthology.orchestra.run.smithi037.stdout: cephadm noarch 2:19.0.0-3291.g994e2f20.el9 ceph-noarch 762 k 2024-04-25T10:26:46.592 INFO:teuthology.orchestra.run.smithi037.stdout: cephfs-mirror x86_64 2:19.0.0-3291.g994e2f20.el9 ceph 224 k 2024-04-25T10:26:46.592 INFO:teuthology.orchestra.run.smithi037.stdout: cephfs-shell x86_64 2:19.0.0-3291.g994e2f20.el9 ceph 27 k 2024-04-25T10:26:46.593 INFO:teuthology.orchestra.run.smithi037.stdout: cephfs-top noarch 2:19.0.0-3291.g994e2f20.el9 ceph-noarch 18 k 2024-04-25T10:26:46.593 INFO:teuthology.orchestra.run.smithi037.stdout: elfutils-libelf-devel x86_64 0.190-2.el9 appstream 21 k 2024-04-25T10:26:46.593 INFO:teuthology.orchestra.run.smithi037.stdout: flex x86_64 2.6.4-9.el9 appstream 313 k 2024-04-25T10:26:46.593 INFO:teuthology.orchestra.run.smithi037.stdout: libaio-devel x86_64 0.3.111-13.el9 appstream 11 k 2024-04-25T10:26:46.593 INFO:teuthology.orchestra.run.smithi037.stdout: libcephfs-devel x86_64 2:19.0.0-3291.g994e2f20.el9 ceph 32 k 2024-04-25T10:26:46.593 INFO:teuthology.orchestra.run.smithi037.stdout: libcephfs2 x86_64 2:19.0.0-3291.g994e2f20.el9 ceph 756 k 2024-04-25T10:26:46.593 INFO:teuthology.orchestra.run.smithi037.stdout: librados-devel x86_64 2:19.0.0-3291.g994e2f20.el9 ceph 127 k 2024-04-25T10:26:46.593 INFO:teuthology.orchestra.run.smithi037.stdout: libtool x86_64 2.4.6-45.el9 appstream 582 k 2024-04-25T10:26:46.593 INFO:teuthology.orchestra.run.smithi037.stdout: libuuid-devel x86_64 2.37.4-18.el9 appstream 25 k 2024-04-25T10:26:46.593 INFO:teuthology.orchestra.run.smithi037.stdout: openssl-devel x86_64 1:3.2.1-1.el9 appstream 4.4 M 2024-04-25T10:26:46.593 INFO:teuthology.orchestra.run.smithi037.stdout: postgresql x86_64 13.14-1.el9 appstream 1.6 M 2024-04-25T10:26:46.593 INFO:teuthology.orchestra.run.smithi037.stdout: postgresql-contrib x86_64 13.14-1.el9 appstream 911 k 2024-04-25T10:26:46.593 INFO:teuthology.orchestra.run.smithi037.stdout: postgresql-server x86_64 13.14-1.el9 appstream 5.8 M 2024-04-25T10:26:46.593 INFO:teuthology.orchestra.run.smithi037.stdout: python3-cephfs x86_64 2:19.0.0-3291.g994e2f20.el9 ceph 161 k 2024-04-25T10:26:46.594 INFO:teuthology.orchestra.run.smithi037.stdout: python3-rados x86_64 2:19.0.0-3291.g994e2f20.el9 ceph 321 k 2024-04-25T10:26:46.594 INFO:teuthology.orchestra.run.smithi037.stdout: python3-rbd x86_64 2:19.0.0-3291.g994e2f20.el9 ceph 298 k 2024-04-25T10:26:46.594 INFO:teuthology.orchestra.run.smithi037.stdout: python3-rgw x86_64 2:19.0.0-3291.g994e2f20.el9 ceph 100 k 2024-04-25T10:26:46.594 INFO:teuthology.orchestra.run.smithi037.stdout: rbd-fuse x86_64 2:19.0.0-3291.g994e2f20.el9 ceph 86 k 2024-04-25T10:26:46.594 INFO:teuthology.orchestra.run.smithi037.stdout: rbd-mirror x86_64 2:19.0.0-3291.g994e2f20.el9 ceph 3.0 M 2024-04-25T10:26:46.594 INFO:teuthology.orchestra.run.smithi037.stdout: rbd-nbd x86_64 2:19.0.0-3291.g994e2f20.el9 ceph 169 k 2024-04-25T10:26:46.594 INFO:teuthology.orchestra.run.smithi037.stdout: xfsprogs-devel x86_64 6.3.0-1.el9 appstream 94 k 2024-04-25T10:26:46.594 INFO:teuthology.orchestra.run.smithi037.stdout:Upgrading: 2024-04-25T10:26:46.594 INFO:teuthology.orchestra.run.smithi037.stdout: librados2 x86_64 2:19.0.0-3291.g994e2f20.el9 ceph 3.3 M 2024-04-25T10:26:46.594 INFO:teuthology.orchestra.run.smithi037.stdout: librbd1 x86_64 2:19.0.0-3291.g994e2f20.el9 ceph 3.1 M 2024-04-25T10:26:46.594 INFO:teuthology.orchestra.run.smithi037.stdout: openssl x86_64 1:3.2.1-1.el9 baseos 1.3 M 2024-04-25T10:26:46.594 INFO:teuthology.orchestra.run.smithi037.stdout: openssl-libs x86_64 1:3.2.1-1.el9 baseos 2.4 M 2024-04-25T10:26:46.594 INFO:teuthology.orchestra.run.smithi037.stdout:Installing dependencies: 2024-04-25T10:26:46.595 INFO:teuthology.orchestra.run.smithi037.stdout: abseil-cpp x86_64 20211102.0-3.el9 epel 548 k 2024-04-25T10:26:46.595 INFO:teuthology.orchestra.run.smithi037.stdout: automake noarch 1.16.2-8.el9 appstream 689 k 2024-04-25T10:26:46.595 INFO:teuthology.orchestra.run.smithi037.stdout: boost-program-options x86_64 1.75.0-8.el9 appstream 107 k 2024-04-25T10:26:46.595 INFO:teuthology.orchestra.run.smithi037.stdout: ceph-common x86_64 2:19.0.0-3291.g994e2f20.el9 ceph 20 M 2024-04-25T10:26:46.595 INFO:teuthology.orchestra.run.smithi037.stdout: ceph-grafana-dashboards noarch 2:19.0.0-3291.g994e2f20.el9 ceph-noarch 34 k 2024-04-25T10:26:46.595 INFO:teuthology.orchestra.run.smithi037.stdout: ceph-mds x86_64 2:19.0.0-3291.g994e2f20.el9 ceph 2.3 M 2024-04-25T10:26:46.595 INFO:teuthology.orchestra.run.smithi037.stdout: ceph-mgr-modules-core noarch 2:19.0.0-3291.g994e2f20.el9 ceph-noarch 249 k 2024-04-25T10:26:46.595 INFO:teuthology.orchestra.run.smithi037.stdout: ceph-mon x86_64 2:19.0.0-3291.g994e2f20.el9 ceph 4.9 M 2024-04-25T10:26:46.595 INFO:teuthology.orchestra.run.smithi037.stdout: ceph-osd x86_64 2:19.0.0-3291.g994e2f20.el9 ceph 18 M 2024-04-25T10:26:46.595 INFO:teuthology.orchestra.run.smithi037.stdout: ceph-prometheus-alerts noarch 2:19.0.0-3291.g994e2f20.el9 ceph-noarch 16 k 2024-04-25T10:26:46.595 INFO:teuthology.orchestra.run.smithi037.stdout: ceph-selinux x86_64 2:19.0.0-3291.g994e2f20.el9 ceph 25 k 2024-04-25T10:26:46.595 INFO:teuthology.orchestra.run.smithi037.stdout: flexiblas x86_64 3.0.4-8.el9 appstream 33 k 2024-04-25T10:26:46.595 INFO:teuthology.orchestra.run.smithi037.stdout: flexiblas-netlib x86_64 3.0.4-8.el9 appstream 3.0 M 2024-04-25T10:26:46.595 INFO:teuthology.orchestra.run.smithi037.stdout: flexiblas-openblas-openmp x86_64 3.0.4-8.el9 appstream 18 k 2024-04-25T10:26:46.595 INFO:teuthology.orchestra.run.smithi037.stdout: fuse x86_64 2.9.9-15.el9 baseos 80 k 2024-04-25T10:26:46.596 INFO:teuthology.orchestra.run.smithi037.stdout: gperftools-libs x86_64 2.9.1-2.el9 epel 309 k 2024-04-25T10:26:46.596 INFO:teuthology.orchestra.run.smithi037.stdout: grpc-data noarch 1.46.7-10.el9 epel 19 k 2024-04-25T10:26:46.596 INFO:teuthology.orchestra.run.smithi037.stdout: libarrow x86_64 9.0.0-10.el9 epel 4.4 M 2024-04-25T10:26:46.596 INFO:teuthology.orchestra.run.smithi037.stdout: libarrow-doc noarch 9.0.0-10.el9 epel 26 k 2024-04-25T10:26:46.596 INFO:teuthology.orchestra.run.smithi037.stdout: libcephsqlite x86_64 2:19.0.0-3291.g994e2f20.el9 ceph 158 k 2024-04-25T10:26:46.596 INFO:teuthology.orchestra.run.smithi037.stdout: libgfortran x86_64 11.4.1-3.el9 baseos 803 k 2024-04-25T10:26:46.596 INFO:teuthology.orchestra.run.smithi037.stdout: liboath x86_64 2.6.7-2.el9 epel 49 k 2024-04-25T10:26:46.596 INFO:teuthology.orchestra.run.smithi037.stdout: libpmemobj x86_64 1.12.1-1.el9 appstream 160 k 2024-04-25T10:26:46.596 INFO:teuthology.orchestra.run.smithi037.stdout: libquadmath x86_64 11.4.1-3.el9 baseos 195 k 2024-04-25T10:26:46.596 INFO:teuthology.orchestra.run.smithi037.stdout: librabbitmq x86_64 0.11.0-7.el9 appstream 45 k 2024-04-25T10:26:46.596 INFO:teuthology.orchestra.run.smithi037.stdout: libradosstriper1 x86_64 2:19.0.0-3291.g994e2f20.el9 ceph 497 k 2024-04-25T10:26:46.596 INFO:teuthology.orchestra.run.smithi037.stdout: librdkafka x86_64 1.6.1-102.el9 appstream 662 k 2024-04-25T10:26:46.596 INFO:teuthology.orchestra.run.smithi037.stdout: librgw2 x86_64 2:19.0.0-3291.g994e2f20.el9 ceph 5.2 M 2024-04-25T10:26:46.596 INFO:teuthology.orchestra.run.smithi037.stdout: libunwind x86_64 1.6.2-1.el9 epel 67 k 2024-04-25T10:26:46.596 INFO:teuthology.orchestra.run.smithi037.stdout: libzstd-devel x86_64 1.5.1-2.el9 appstream 47 k 2024-04-25T10:26:46.597 INFO:teuthology.orchestra.run.smithi037.stdout: lttng-ust x86_64 2.12.0-6.el9 appstream 292 k 2024-04-25T10:26:46.597 INFO:teuthology.orchestra.run.smithi037.stdout: lua x86_64 5.4.4-4.el9 appstream 188 k 2024-04-25T10:26:46.597 INFO:teuthology.orchestra.run.smithi037.stdout: lua-devel x86_64 5.4.4-4.el9 crb 22 k 2024-04-25T10:26:46.597 INFO:teuthology.orchestra.run.smithi037.stdout: luarocks noarch 3.9.2-1.el9 epel 151 k 2024-04-25T10:26:46.597 INFO:teuthology.orchestra.run.smithi037.stdout: openblas x86_64 0.3.21-2.el9 appstream 35 k 2024-04-25T10:26:46.597 INFO:teuthology.orchestra.run.smithi037.stdout: openblas-openmp x86_64 0.3.21-2.el9 appstream 4.7 M 2024-04-25T10:26:46.597 INFO:teuthology.orchestra.run.smithi037.stdout: parquet-libs x86_64 9.0.0-10.el9 epel 838 k 2024-04-25T10:26:46.597 INFO:teuthology.orchestra.run.smithi037.stdout: postgresql-private-libs x86_64 13.14-1.el9 appstream 133 k 2024-04-25T10:26:46.597 INFO:teuthology.orchestra.run.smithi037.stdout: protobuf x86_64 3.14.0-13.el9 appstream 1.0 M 2024-04-25T10:26:46.597 INFO:teuthology.orchestra.run.smithi037.stdout: protobuf-compiler x86_64 3.14.0-13.el9 crb 863 k 2024-04-25T10:26:46.597 INFO:teuthology.orchestra.run.smithi037.stdout: python3-asyncssh noarch 2.13.2-5.el9 epel 548 k 2024-04-25T10:26:46.597 INFO:teuthology.orchestra.run.smithi037.stdout: python3-attrs noarch 20.3.0-7.el9 appstream 87 k 2024-04-25T10:26:46.598 INFO:teuthology.orchestra.run.smithi037.stdout: python3-babel noarch 2.9.1-2.el9 appstream 6.0 M 2024-04-25T10:26:46.598 INFO:teuthology.orchestra.run.smithi037.stdout: python3-bcrypt x86_64 3.2.2-1.el9 epel 43 k 2024-04-25T10:26:46.598 INFO:teuthology.orchestra.run.smithi037.stdout: python3-cachetools noarch 4.2.4-1.el9 epel 32 k 2024-04-25T10:26:46.598 INFO:teuthology.orchestra.run.smithi037.stdout: python3-ceph-argparse x86_64 2:19.0.0-3291.g994e2f20.el9 ceph 45 k 2024-04-25T10:26:46.598 INFO:teuthology.orchestra.run.smithi037.stdout: python3-ceph-common x86_64 2:19.0.0-3291.g994e2f20.el9 ceph 135 k 2024-04-25T10:26:46.598 INFO:teuthology.orchestra.run.smithi037.stdout: python3-certifi noarch 2021.10.8-2.el9 epel 15 k 2024-04-25T10:26:46.598 INFO:teuthology.orchestra.run.smithi037.stdout: python3-cffi x86_64 1.14.5-5.el9 baseos 253 k 2024-04-25T10:26:46.598 INFO:teuthology.orchestra.run.smithi037.stdout: python3-cheroot noarch 8.6.0-4.el9 epel 172 k 2024-04-25T10:26:46.598 INFO:teuthology.orchestra.run.smithi037.stdout: python3-cherrypy noarch 18.6.1-2.el9 epel 358 k 2024-04-25T10:26:46.598 INFO:teuthology.orchestra.run.smithi037.stdout: python3-cmd2 noarch 2.4.2-2.el9 epel 303 k 2024-04-25T10:26:46.598 INFO:teuthology.orchestra.run.smithi037.stdout: python3-colorama noarch 0.4.4-12.el9 epel 35 k 2024-04-25T10:26:46.598 INFO:teuthology.orchestra.run.smithi037.stdout: python3-cryptography x86_64 36.0.1-4.el9 baseos 1.2 M 2024-04-25T10:26:46.598 INFO:teuthology.orchestra.run.smithi037.stdout: python3-devel x86_64 3.9.18-3.el9 appstream 244 k 2024-04-25T10:26:46.598 INFO:teuthology.orchestra.run.smithi037.stdout: python3-google-auth noarch 1:2.29.0-1.el9 epel 206 k 2024-04-25T10:26:46.599 INFO:teuthology.orchestra.run.smithi037.stdout: python3-grpcio x86_64 1.46.7-10.el9 epel 2.0 M 2024-04-25T10:26:46.599 INFO:teuthology.orchestra.run.smithi037.stdout: python3-grpcio-tools x86_64 1.46.7-10.el9 epel 144 k 2024-04-25T10:26:46.599 INFO:teuthology.orchestra.run.smithi037.stdout: python3-jaraco noarch 8.2.1-3.el9 epel 11 k 2024-04-25T10:26:46.599 INFO:teuthology.orchestra.run.smithi037.stdout: python3-jaraco-classes noarch 3.2.1-5.el9 epel 18 k 2024-04-25T10:26:46.599 INFO:teuthology.orchestra.run.smithi037.stdout: python3-jaraco-collections noarch 3.0.0-8.el9 epel 23 k 2024-04-25T10:26:46.599 INFO:teuthology.orchestra.run.smithi037.stdout: python3-jaraco-functools noarch 3.5.0-2.el9 epel 19 k 2024-04-25T10:26:46.599 INFO:teuthology.orchestra.run.smithi037.stdout: python3-jaraco-text noarch 3.2.0-6.el9 epel 20 k 2024-04-25T10:26:46.599 INFO:teuthology.orchestra.run.smithi037.stdout: python3-jinja2 noarch 2.11.3-5.el9 appstream 248 k 2024-04-25T10:26:46.599 INFO:teuthology.orchestra.run.smithi037.stdout: python3-jsonpatch noarch 1.21-16.el9 appstream 26 k 2024-04-25T10:26:46.599 INFO:teuthology.orchestra.run.smithi037.stdout: python3-jsonpointer noarch 2.0-4.el9 appstream 19 k 2024-04-25T10:26:46.599 INFO:teuthology.orchestra.run.smithi037.stdout: python3-kubernetes noarch 1:26.1.0-2.el9 epel 1.0 M 2024-04-25T10:26:46.599 INFO:teuthology.orchestra.run.smithi037.stdout: python3-logutils noarch 0.3.5-21.el9 epel 46 k 2024-04-25T10:26:46.599 INFO:teuthology.orchestra.run.smithi037.stdout: python3-mako noarch 1.1.4-6.el9 appstream 172 k 2024-04-25T10:26:46.599 INFO:teuthology.orchestra.run.smithi037.stdout: python3-markupsafe x86_64 1.1.1-12.el9 appstream 35 k 2024-04-25T10:26:46.599 INFO:teuthology.orchestra.run.smithi037.stdout: python3-more-itertools noarch 8.12.0-2.el9 epel 79 k 2024-04-25T10:26:46.600 INFO:teuthology.orchestra.run.smithi037.stdout: python3-natsort noarch 7.1.1-5.el9 epel 58 k 2024-04-25T10:26:46.600 INFO:teuthology.orchestra.run.smithi037.stdout: python3-numpy x86_64 1:1.20.1-5.el9 appstream 5.1 M 2024-04-25T10:26:46.600 INFO:teuthology.orchestra.run.smithi037.stdout: python3-numpy-f2py x86_64 1:1.20.1-5.el9 appstream 250 k 2024-04-25T10:26:46.600 INFO:teuthology.orchestra.run.smithi037.stdout: python3-oauthlib noarch 3.1.1-5.el9 appstream 222 k 2024-04-25T10:26:46.600 INFO:teuthology.orchestra.run.smithi037.stdout: python3-packaging noarch 20.9-5.el9 appstream 77 k 2024-04-25T10:26:46.600 INFO:teuthology.orchestra.run.smithi037.stdout: python3-pecan noarch 1.4.2-3.el9 epel 272 k 2024-04-25T10:26:46.600 INFO:teuthology.orchestra.run.smithi037.stdout: python3-ply noarch 3.11-14.el9 baseos 106 k 2024-04-25T10:26:46.600 INFO:teuthology.orchestra.run.smithi037.stdout: python3-portend noarch 3.1.0-2.el9 epel 16 k 2024-04-25T10:26:46.600 INFO:teuthology.orchestra.run.smithi037.stdout: python3-prettytable noarch 0.7.2-27.el9 appstream 42 k 2024-04-25T10:26:46.600 INFO:teuthology.orchestra.run.smithi037.stdout: python3-protobuf noarch 3.14.0-13.el9 appstream 269 k 2024-04-25T10:26:46.600 INFO:teuthology.orchestra.run.smithi037.stdout: python3-pyOpenSSL noarch 21.0.0-1.el9 epel 90 k 2024-04-25T10:26:46.600 INFO:teuthology.orchestra.run.smithi037.stdout: python3-pyasn1 noarch 0.4.8-6.el9 appstream 159 k 2024-04-25T10:26:46.600 INFO:teuthology.orchestra.run.smithi037.stdout: python3-pyasn1-modules noarch 0.4.8-6.el9 appstream 279 k 2024-04-25T10:26:46.601 INFO:teuthology.orchestra.run.smithi037.stdout: python3-pycparser noarch 2.20-6.el9 baseos 135 k 2024-04-25T10:26:46.601 INFO:teuthology.orchestra.run.smithi037.stdout: python3-pyperclip noarch 1.8.2-2.el9 epel 28 k 2024-04-25T10:26:46.601 INFO:teuthology.orchestra.run.smithi037.stdout: python3-pytz noarch 2021.1-5.el9 appstream 51 k 2024-04-25T10:26:46.601 INFO:teuthology.orchestra.run.smithi037.stdout: python3-repoze-lru noarch 0.7-16.el9 epel 31 k 2024-04-25T10:26:46.601 INFO:teuthology.orchestra.run.smithi037.stdout: python3-requests noarch 2.25.1-8.el9 baseos 125 k 2024-04-25T10:26:46.601 INFO:teuthology.orchestra.run.smithi037.stdout: python3-requests-oauthlib noarch 1.3.0-12.el9 appstream 54 k 2024-04-25T10:26:46.601 INFO:teuthology.orchestra.run.smithi037.stdout: python3-routes noarch 2.5.1-5.el9 epel 188 k 2024-04-25T10:26:46.601 INFO:teuthology.orchestra.run.smithi037.stdout: python3-rsa noarch 4.9-2.el9 epel 59 k 2024-04-25T10:26:46.601 INFO:teuthology.orchestra.run.smithi037.stdout: python3-scipy x86_64 1.6.2-8.el9 appstream 16 M 2024-04-25T10:26:46.601 INFO:teuthology.orchestra.run.smithi037.stdout: python3-tempora noarch 5.0.0-2.el9 epel 36 k 2024-04-25T10:26:46.601 INFO:teuthology.orchestra.run.smithi037.stdout: python3-toml noarch 0.10.2-6.el9 appstream 42 k 2024-04-25T10:26:46.601 INFO:teuthology.orchestra.run.smithi037.stdout: python3-typing-extensions noarch 4.4.0-2.el9 epel 51 k 2024-04-25T10:26:46.601 INFO:teuthology.orchestra.run.smithi037.stdout: python3-urllib3 noarch 1.26.5-5.el9 baseos 215 k 2024-04-25T10:26:46.601 INFO:teuthology.orchestra.run.smithi037.stdout: python3-wcwidth noarch 0.2.5-8.el9 appstream 44 k 2024-04-25T10:26:46.602 INFO:teuthology.orchestra.run.smithi037.stdout: python3-webob noarch 1.8.7-6.el9 epel 230 k 2024-04-25T10:26:46.602 INFO:teuthology.orchestra.run.smithi037.stdout: python3-websocket-client noarch 1.2.3-2.el9 epel 90 k 2024-04-25T10:26:46.602 INFO:teuthology.orchestra.run.smithi037.stdout: python3-werkzeug noarch 2.0.3-3.el9.1 epel 427 k 2024-04-25T10:26:46.602 INFO:teuthology.orchestra.run.smithi037.stdout: python3-zc-lockfile noarch 2.0-10.el9 epel 20 k 2024-04-25T10:26:46.602 INFO:teuthology.orchestra.run.smithi037.stdout: qatlib x86_64 24.02.0-1.el9 appstream 221 k 2024-04-25T10:26:46.602 INFO:teuthology.orchestra.run.smithi037.stdout: qatzip-libs x86_64 1.2.0-1.el9 appstream 47 k 2024-04-25T10:26:46.602 INFO:teuthology.orchestra.run.smithi037.stdout: re2 x86_64 1:20211101-3.el9 epel 192 k 2024-04-25T10:26:46.602 INFO:teuthology.orchestra.run.smithi037.stdout: socat x86_64 1.7.4.1-5.el9 appstream 305 k 2024-04-25T10:26:46.602 INFO:teuthology.orchestra.run.smithi037.stdout: thrift x86_64 0.15.0-2.el9 epel 1.6 M 2024-04-25T10:26:46.602 INFO:teuthology.orchestra.run.smithi037.stdout: uuid x86_64 1.6.2-55.el9 appstream 58 k 2024-04-25T10:26:46.602 INFO:teuthology.orchestra.run.smithi037.stdout: xmlstarlet x86_64 1.6.1-20.el9 appstream 64 k 2024-04-25T10:26:46.602 INFO:teuthology.orchestra.run.smithi037.stdout: zlib-devel x86_64 1.2.11-41.el9 appstream 45 k 2024-04-25T10:26:46.602 INFO:teuthology.orchestra.run.smithi037.stdout:Installing weak dependencies: 2024-04-25T10:26:46.602 INFO:teuthology.orchestra.run.smithi037.stdout: qatlib-service x86_64 24.02.0-1.el9 appstream 36 k 2024-04-25T10:26:46.603 INFO:teuthology.orchestra.run.smithi037.stdout: 2024-04-25T10:26:46.603 INFO:teuthology.orchestra.run.smithi037.stdout:Transaction Summary 2024-04-25T10:26:46.603 INFO:teuthology.orchestra.run.smithi037.stdout:======================================================================================= 2024-04-25T10:26:46.603 INFO:teuthology.orchestra.run.smithi037.stdout:Install 148 Packages 2024-04-25T10:26:46.603 INFO:teuthology.orchestra.run.smithi037.stdout:Upgrade 4 Packages 2024-04-25T10:26:46.603 INFO:teuthology.orchestra.run.smithi037.stdout: 2024-04-25T10:26:46.603 INFO:teuthology.orchestra.run.smithi037.stdout:Total download size: 214 M 2024-04-25T10:26:46.603 INFO:teuthology.orchestra.run.smithi037.stdout:Downloading Packages: 2024-04-25T10:26:47.303 INFO:teuthology.orchestra.run.smithi203.stdout:lab-extras 25 kB/s | 1.7 kB 00:00 2024-04-25T10:26:47.323 INFO:teuthology.orchestra.run.smithi037.stdout:(1/152): ceph-19.0.0-3291.g994e2f20.el9.x86_64. 28 kB/s | 6.6 kB 00:00 2024-04-25T10:26:47.790 INFO:teuthology.orchestra.run.smithi037.stdout:(2/152): ceph-fuse-19.0.0-3291.g994e2f20.el9.x8 1.8 MB/s | 863 kB 00:00 2024-04-25T10:26:47.915 INFO:teuthology.orchestra.run.smithi037.stdout:(3/152): ceph-immutable-object-cache-19.0.0-329 1.1 MB/s | 147 kB 00:00 2024-04-25T10:26:48.657 INFO:teuthology.orchestra.run.smithi037.stdout:(4/152): ceph-mds-19.0.0-3291.g994e2f20.el9.x86 3.0 MB/s | 2.3 MB 00:00 2024-04-25T10:26:49.049 INFO:teuthology.orchestra.run.smithi037.stdout:(5/152): ceph-base-19.0.0-3291.g994e2f20.el9.x8 2.8 MB/s | 5.4 MB 00:01 2024-04-25T10:26:49.108 INFO:teuthology.orchestra.run.smithi037.stdout:(6/152): ceph-mgr-19.0.0-3291.g994e2f20.el9.x86 3.3 MB/s | 1.5 MB 00:00 2024-04-25T10:26:49.445 INFO:teuthology.orchestra.run.smithi203.stdout:Package librados2-2:16.2.4-5.el9.x86_64 is already installed. 2024-04-25T10:26:49.446 INFO:teuthology.orchestra.run.smithi203.stdout:Package librbd1-2:16.2.4-5.el9.x86_64 is already installed. 2024-04-25T10:26:49.457 INFO:teuthology.orchestra.run.smithi203.stdout:Package NetworkManager-1:1.47.4-1.el9.x86_64 is already installed. 2024-04-25T10:26:49.458 INFO:teuthology.orchestra.run.smithi203.stdout:Package iproute-6.2.0-5.el9.x86_64 is already installed. 2024-04-25T10:26:49.458 INFO:teuthology.orchestra.run.smithi203.stdout:Package util-linux-2.37.4-18.el9.x86_64 is already installed. 2024-04-25T10:26:49.459 INFO:teuthology.orchestra.run.smithi203.stdout:Package libacl-devel-2.3.1-4.el9.x86_64 is already installed. 2024-04-25T10:26:49.461 INFO:teuthology.orchestra.run.smithi203.stdout:Package libattr-devel-2.5.1-3.el9.x86_64 is already installed. 2024-04-25T10:26:49.463 INFO:teuthology.orchestra.run.smithi203.stdout:Package xfsdump-3.1.12-4.el9.x86_64 is already installed. 2024-04-25T10:26:49.464 INFO:teuthology.orchestra.run.smithi203.stdout:Package xfsprogs-6.3.0-1.el9.x86_64 is already installed. 2024-04-25T10:26:49.525 INFO:teuthology.orchestra.run.smithi203.stdout:Dependencies resolved. 2024-04-25T10:26:49.534 INFO:teuthology.orchestra.run.smithi203.stdout:======================================================================================= 2024-04-25T10:26:49.534 INFO:teuthology.orchestra.run.smithi203.stdout: Package Arch Version Repository Size 2024-04-25T10:26:49.534 INFO:teuthology.orchestra.run.smithi203.stdout:======================================================================================= 2024-04-25T10:26:49.534 INFO:teuthology.orchestra.run.smithi203.stdout:Installing: 2024-04-25T10:26:49.534 INFO:teuthology.orchestra.run.smithi203.stdout: bison x86_64 3.7.4-5.el9 appstream 943 k 2024-04-25T10:26:49.534 INFO:teuthology.orchestra.run.smithi203.stdout: ceph x86_64 2:19.0.0-3291.g994e2f20.el9 ceph 6.6 k 2024-04-25T10:26:49.534 INFO:teuthology.orchestra.run.smithi203.stdout: ceph-base x86_64 2:19.0.0-3291.g994e2f20.el9 ceph 5.4 M 2024-04-25T10:26:49.534 INFO:teuthology.orchestra.run.smithi203.stdout: ceph-fuse x86_64 2:19.0.0-3291.g994e2f20.el9 ceph 863 k 2024-04-25T10:26:49.534 INFO:teuthology.orchestra.run.smithi203.stdout: ceph-immutable-object-cache x86_64 2:19.0.0-3291.g994e2f20.el9 ceph 147 k 2024-04-25T10:26:49.534 INFO:teuthology.orchestra.run.smithi203.stdout: ceph-mgr x86_64 2:19.0.0-3291.g994e2f20.el9 ceph 1.5 M 2024-04-25T10:26:49.535 INFO:teuthology.orchestra.run.smithi203.stdout: ceph-mgr-cephadm noarch 2:19.0.0-3291.g994e2f20.el9 ceph-noarch 142 k 2024-04-25T10:26:49.535 INFO:teuthology.orchestra.run.smithi203.stdout: ceph-mgr-dashboard noarch 2:19.0.0-3291.g994e2f20.el9 ceph-noarch 3.5 M 2024-04-25T10:26:49.535 INFO:teuthology.orchestra.run.smithi203.stdout: ceph-mgr-diskprediction-local noarch 2:19.0.0-3291.g994e2f20.el9 ceph-noarch 7.4 M 2024-04-25T10:26:49.535 INFO:teuthology.orchestra.run.smithi203.stdout: ceph-mgr-rook noarch 2:19.0.0-3291.g994e2f20.el9 ceph-noarch 49 k 2024-04-25T10:26:49.535 INFO:teuthology.orchestra.run.smithi203.stdout: ceph-radosgw x86_64 2:19.0.0-3291.g994e2f20.el9 ceph 10 M 2024-04-25T10:26:49.535 INFO:teuthology.orchestra.run.smithi203.stdout: ceph-test x86_64 2:19.0.0-3291.g994e2f20.el9 ceph 43 M 2024-04-25T10:26:49.535 INFO:teuthology.orchestra.run.smithi203.stdout: ceph-volume noarch 2:19.0.0-3291.g994e2f20.el9 ceph-noarch 273 k 2024-04-25T10:26:49.535 INFO:teuthology.orchestra.run.smithi203.stdout: cephadm noarch 2:19.0.0-3291.g994e2f20.el9 ceph-noarch 762 k 2024-04-25T10:26:49.535 INFO:teuthology.orchestra.run.smithi203.stdout: cephfs-mirror x86_64 2:19.0.0-3291.g994e2f20.el9 ceph 224 k 2024-04-25T10:26:49.535 INFO:teuthology.orchestra.run.smithi203.stdout: cephfs-shell x86_64 2:19.0.0-3291.g994e2f20.el9 ceph 27 k 2024-04-25T10:26:49.535 INFO:teuthology.orchestra.run.smithi203.stdout: cephfs-top noarch 2:19.0.0-3291.g994e2f20.el9 ceph-noarch 18 k 2024-04-25T10:26:49.535 INFO:teuthology.orchestra.run.smithi203.stdout: elfutils-libelf-devel x86_64 0.190-2.el9 appstream 21 k 2024-04-25T10:26:49.535 INFO:teuthology.orchestra.run.smithi203.stdout: flex x86_64 2.6.4-9.el9 appstream 313 k 2024-04-25T10:26:49.535 INFO:teuthology.orchestra.run.smithi203.stdout: libaio-devel x86_64 0.3.111-13.el9 appstream 11 k 2024-04-25T10:26:49.535 INFO:teuthology.orchestra.run.smithi203.stdout: libcephfs-devel x86_64 2:19.0.0-3291.g994e2f20.el9 ceph 32 k 2024-04-25T10:26:49.536 INFO:teuthology.orchestra.run.smithi203.stdout: libcephfs2 x86_64 2:19.0.0-3291.g994e2f20.el9 ceph 756 k 2024-04-25T10:26:49.536 INFO:teuthology.orchestra.run.smithi203.stdout: librados-devel x86_64 2:19.0.0-3291.g994e2f20.el9 ceph 127 k 2024-04-25T10:26:49.536 INFO:teuthology.orchestra.run.smithi203.stdout: libtool x86_64 2.4.6-45.el9 appstream 582 k 2024-04-25T10:26:49.536 INFO:teuthology.orchestra.run.smithi203.stdout: libuuid-devel x86_64 2.37.4-18.el9 appstream 25 k 2024-04-25T10:26:49.536 INFO:teuthology.orchestra.run.smithi203.stdout: openssl-devel x86_64 1:3.2.1-1.el9 appstream 4.4 M 2024-04-25T10:26:49.536 INFO:teuthology.orchestra.run.smithi203.stdout: postgresql x86_64 13.14-1.el9 appstream 1.6 M 2024-04-25T10:26:49.536 INFO:teuthology.orchestra.run.smithi203.stdout: postgresql-contrib x86_64 13.14-1.el9 appstream 911 k 2024-04-25T10:26:49.536 INFO:teuthology.orchestra.run.smithi203.stdout: postgresql-server x86_64 13.14-1.el9 appstream 5.8 M 2024-04-25T10:26:49.536 INFO:teuthology.orchestra.run.smithi203.stdout: python3-cephfs x86_64 2:19.0.0-3291.g994e2f20.el9 ceph 161 k 2024-04-25T10:26:49.536 INFO:teuthology.orchestra.run.smithi203.stdout: python3-rados x86_64 2:19.0.0-3291.g994e2f20.el9 ceph 321 k 2024-04-25T10:26:49.536 INFO:teuthology.orchestra.run.smithi203.stdout: python3-rbd x86_64 2:19.0.0-3291.g994e2f20.el9 ceph 298 k 2024-04-25T10:26:49.536 INFO:teuthology.orchestra.run.smithi203.stdout: python3-rgw x86_64 2:19.0.0-3291.g994e2f20.el9 ceph 100 k 2024-04-25T10:26:49.536 INFO:teuthology.orchestra.run.smithi203.stdout: rbd-fuse x86_64 2:19.0.0-3291.g994e2f20.el9 ceph 86 k 2024-04-25T10:26:49.536 INFO:teuthology.orchestra.run.smithi203.stdout: rbd-mirror x86_64 2:19.0.0-3291.g994e2f20.el9 ceph 3.0 M 2024-04-25T10:26:49.537 INFO:teuthology.orchestra.run.smithi203.stdout: rbd-nbd x86_64 2:19.0.0-3291.g994e2f20.el9 ceph 169 k 2024-04-25T10:26:49.537 INFO:teuthology.orchestra.run.smithi203.stdout: xfsprogs-devel x86_64 6.3.0-1.el9 appstream 94 k 2024-04-25T10:26:49.537 INFO:teuthology.orchestra.run.smithi203.stdout:Upgrading: 2024-04-25T10:26:49.537 INFO:teuthology.orchestra.run.smithi203.stdout: librados2 x86_64 2:19.0.0-3291.g994e2f20.el9 ceph 3.3 M 2024-04-25T10:26:49.537 INFO:teuthology.orchestra.run.smithi203.stdout: librbd1 x86_64 2:19.0.0-3291.g994e2f20.el9 ceph 3.1 M 2024-04-25T10:26:49.537 INFO:teuthology.orchestra.run.smithi203.stdout: openssl x86_64 1:3.2.1-1.el9 baseos 1.3 M 2024-04-25T10:26:49.537 INFO:teuthology.orchestra.run.smithi203.stdout: openssl-libs x86_64 1:3.2.1-1.el9 baseos 2.4 M 2024-04-25T10:26:49.537 INFO:teuthology.orchestra.run.smithi203.stdout:Installing dependencies: 2024-04-25T10:26:49.537 INFO:teuthology.orchestra.run.smithi203.stdout: abseil-cpp x86_64 20211102.0-3.el9 epel 548 k 2024-04-25T10:26:49.537 INFO:teuthology.orchestra.run.smithi203.stdout: automake noarch 1.16.2-8.el9 appstream 689 k 2024-04-25T10:26:49.537 INFO:teuthology.orchestra.run.smithi203.stdout: boost-program-options x86_64 1.75.0-8.el9 appstream 107 k 2024-04-25T10:26:49.537 INFO:teuthology.orchestra.run.smithi203.stdout: ceph-common x86_64 2:19.0.0-3291.g994e2f20.el9 ceph 20 M 2024-04-25T10:26:49.537 INFO:teuthology.orchestra.run.smithi203.stdout: ceph-grafana-dashboards noarch 2:19.0.0-3291.g994e2f20.el9 ceph-noarch 34 k 2024-04-25T10:26:49.537 INFO:teuthology.orchestra.run.smithi203.stdout: ceph-mds x86_64 2:19.0.0-3291.g994e2f20.el9 ceph 2.3 M 2024-04-25T10:26:49.538 INFO:teuthology.orchestra.run.smithi203.stdout: ceph-mgr-modules-core noarch 2:19.0.0-3291.g994e2f20.el9 ceph-noarch 249 k 2024-04-25T10:26:49.538 INFO:teuthology.orchestra.run.smithi203.stdout: ceph-mon x86_64 2:19.0.0-3291.g994e2f20.el9 ceph 4.9 M 2024-04-25T10:26:49.538 INFO:teuthology.orchestra.run.smithi203.stdout: ceph-osd x86_64 2:19.0.0-3291.g994e2f20.el9 ceph 18 M 2024-04-25T10:26:49.538 INFO:teuthology.orchestra.run.smithi203.stdout: ceph-prometheus-alerts noarch 2:19.0.0-3291.g994e2f20.el9 ceph-noarch 16 k 2024-04-25T10:26:49.538 INFO:teuthology.orchestra.run.smithi203.stdout: ceph-selinux x86_64 2:19.0.0-3291.g994e2f20.el9 ceph 25 k 2024-04-25T10:26:49.538 INFO:teuthology.orchestra.run.smithi203.stdout: flexiblas x86_64 3.0.4-8.el9 appstream 33 k 2024-04-25T10:26:49.538 INFO:teuthology.orchestra.run.smithi203.stdout: flexiblas-netlib x86_64 3.0.4-8.el9 appstream 3.0 M 2024-04-25T10:26:49.538 INFO:teuthology.orchestra.run.smithi203.stdout: flexiblas-openblas-openmp x86_64 3.0.4-8.el9 appstream 18 k 2024-04-25T10:26:49.538 INFO:teuthology.orchestra.run.smithi203.stdout: fuse x86_64 2.9.9-15.el9 baseos 80 k 2024-04-25T10:26:49.538 INFO:teuthology.orchestra.run.smithi203.stdout: gperftools-libs x86_64 2.9.1-2.el9 epel 309 k 2024-04-25T10:26:49.538 INFO:teuthology.orchestra.run.smithi203.stdout: grpc-data noarch 1.46.7-10.el9 epel 19 k 2024-04-25T10:26:49.538 INFO:teuthology.orchestra.run.smithi203.stdout: libarrow x86_64 9.0.0-10.el9 epel 4.4 M 2024-04-25T10:26:49.538 INFO:teuthology.orchestra.run.smithi203.stdout: libarrow-doc noarch 9.0.0-10.el9 epel 26 k 2024-04-25T10:26:49.538 INFO:teuthology.orchestra.run.smithi203.stdout: libcephsqlite x86_64 2:19.0.0-3291.g994e2f20.el9 ceph 158 k 2024-04-25T10:26:49.539 INFO:teuthology.orchestra.run.smithi203.stdout: libgfortran x86_64 11.4.1-3.el9 baseos 803 k 2024-04-25T10:26:49.539 INFO:teuthology.orchestra.run.smithi203.stdout: liboath x86_64 2.6.7-2.el9 epel 49 k 2024-04-25T10:26:49.539 INFO:teuthology.orchestra.run.smithi203.stdout: libpmemobj x86_64 1.12.1-1.el9 appstream 160 k 2024-04-25T10:26:49.539 INFO:teuthology.orchestra.run.smithi203.stdout: libquadmath x86_64 11.4.1-3.el9 baseos 195 k 2024-04-25T10:26:49.539 INFO:teuthology.orchestra.run.smithi203.stdout: librabbitmq x86_64 0.11.0-7.el9 appstream 45 k 2024-04-25T10:26:49.539 INFO:teuthology.orchestra.run.smithi203.stdout: libradosstriper1 x86_64 2:19.0.0-3291.g994e2f20.el9 ceph 497 k 2024-04-25T10:26:49.539 INFO:teuthology.orchestra.run.smithi203.stdout: librdkafka x86_64 1.6.1-102.el9 appstream 662 k 2024-04-25T10:26:49.539 INFO:teuthology.orchestra.run.smithi203.stdout: librgw2 x86_64 2:19.0.0-3291.g994e2f20.el9 ceph 5.2 M 2024-04-25T10:26:49.539 INFO:teuthology.orchestra.run.smithi203.stdout: libunwind x86_64 1.6.2-1.el9 epel 67 k 2024-04-25T10:26:49.539 INFO:teuthology.orchestra.run.smithi203.stdout: libzstd-devel x86_64 1.5.1-2.el9 appstream 47 k 2024-04-25T10:26:49.539 INFO:teuthology.orchestra.run.smithi203.stdout: lttng-ust x86_64 2.12.0-6.el9 appstream 292 k 2024-04-25T10:26:49.539 INFO:teuthology.orchestra.run.smithi203.stdout: lua x86_64 5.4.4-4.el9 appstream 188 k 2024-04-25T10:26:49.539 INFO:teuthology.orchestra.run.smithi203.stdout: lua-devel x86_64 5.4.4-4.el9 crb 22 k 2024-04-25T10:26:49.539 INFO:teuthology.orchestra.run.smithi203.stdout: luarocks noarch 3.9.2-1.el9 epel 151 k 2024-04-25T10:26:49.540 INFO:teuthology.orchestra.run.smithi203.stdout: openblas x86_64 0.3.21-2.el9 appstream 35 k 2024-04-25T10:26:49.540 INFO:teuthology.orchestra.run.smithi203.stdout: openblas-openmp x86_64 0.3.21-2.el9 appstream 4.7 M 2024-04-25T10:26:49.540 INFO:teuthology.orchestra.run.smithi203.stdout: parquet-libs x86_64 9.0.0-10.el9 epel 838 k 2024-04-25T10:26:49.540 INFO:teuthology.orchestra.run.smithi203.stdout: postgresql-private-libs x86_64 13.14-1.el9 appstream 133 k 2024-04-25T10:26:49.540 INFO:teuthology.orchestra.run.smithi203.stdout: protobuf x86_64 3.14.0-13.el9 appstream 1.0 M 2024-04-25T10:26:49.540 INFO:teuthology.orchestra.run.smithi203.stdout: protobuf-compiler x86_64 3.14.0-13.el9 crb 863 k 2024-04-25T10:26:49.540 INFO:teuthology.orchestra.run.smithi203.stdout: python3-asyncssh noarch 2.13.2-5.el9 epel 548 k 2024-04-25T10:26:49.540 INFO:teuthology.orchestra.run.smithi203.stdout: python3-attrs noarch 20.3.0-7.el9 appstream 87 k 2024-04-25T10:26:49.540 INFO:teuthology.orchestra.run.smithi203.stdout: python3-babel noarch 2.9.1-2.el9 appstream 6.0 M 2024-04-25T10:26:49.540 INFO:teuthology.orchestra.run.smithi203.stdout: python3-bcrypt x86_64 3.2.2-1.el9 epel 43 k 2024-04-25T10:26:49.540 INFO:teuthology.orchestra.run.smithi203.stdout: python3-cachetools noarch 4.2.4-1.el9 epel 32 k 2024-04-25T10:26:49.540 INFO:teuthology.orchestra.run.smithi203.stdout: python3-ceph-argparse x86_64 2:19.0.0-3291.g994e2f20.el9 ceph 45 k 2024-04-25T10:26:49.540 INFO:teuthology.orchestra.run.smithi203.stdout: python3-ceph-common x86_64 2:19.0.0-3291.g994e2f20.el9 ceph 135 k 2024-04-25T10:26:49.541 INFO:teuthology.orchestra.run.smithi203.stdout: python3-certifi noarch 2021.10.8-2.el9 epel 15 k 2024-04-25T10:26:49.541 INFO:teuthology.orchestra.run.smithi203.stdout: python3-cffi x86_64 1.14.5-5.el9 baseos 253 k 2024-04-25T10:26:49.541 INFO:teuthology.orchestra.run.smithi203.stdout: python3-cheroot noarch 8.6.0-4.el9 epel 172 k 2024-04-25T10:26:49.541 INFO:teuthology.orchestra.run.smithi203.stdout: python3-cherrypy noarch 18.6.1-2.el9 epel 358 k 2024-04-25T10:26:49.541 INFO:teuthology.orchestra.run.smithi203.stdout: python3-cmd2 noarch 2.4.2-2.el9 epel 303 k 2024-04-25T10:26:49.541 INFO:teuthology.orchestra.run.smithi203.stdout: python3-colorama noarch 0.4.4-12.el9 epel 35 k 2024-04-25T10:26:49.541 INFO:teuthology.orchestra.run.smithi203.stdout: python3-cryptography x86_64 36.0.1-4.el9 baseos 1.2 M 2024-04-25T10:26:49.541 INFO:teuthology.orchestra.run.smithi203.stdout: python3-devel x86_64 3.9.18-3.el9 appstream 244 k 2024-04-25T10:26:49.541 INFO:teuthology.orchestra.run.smithi203.stdout: python3-google-auth noarch 1:2.29.0-1.el9 epel 206 k 2024-04-25T10:26:49.541 INFO:teuthology.orchestra.run.smithi203.stdout: python3-grpcio x86_64 1.46.7-10.el9 epel 2.0 M 2024-04-25T10:26:49.541 INFO:teuthology.orchestra.run.smithi203.stdout: python3-grpcio-tools x86_64 1.46.7-10.el9 epel 144 k 2024-04-25T10:26:49.541 INFO:teuthology.orchestra.run.smithi203.stdout: python3-jaraco noarch 8.2.1-3.el9 epel 11 k 2024-04-25T10:26:49.541 INFO:teuthology.orchestra.run.smithi203.stdout: python3-jaraco-classes noarch 3.2.1-5.el9 epel 18 k 2024-04-25T10:26:49.541 INFO:teuthology.orchestra.run.smithi203.stdout: python3-jaraco-collections noarch 3.0.0-8.el9 epel 23 k 2024-04-25T10:26:49.541 INFO:teuthology.orchestra.run.smithi203.stdout: python3-jaraco-functools noarch 3.5.0-2.el9 epel 19 k 2024-04-25T10:26:49.541 INFO:teuthology.orchestra.run.smithi203.stdout: python3-jaraco-text noarch 3.2.0-6.el9 epel 20 k 2024-04-25T10:26:49.542 INFO:teuthology.orchestra.run.smithi203.stdout: python3-jinja2 noarch 2.11.3-5.el9 appstream 248 k 2024-04-25T10:26:49.542 INFO:teuthology.orchestra.run.smithi203.stdout: python3-jsonpatch noarch 1.21-16.el9 appstream 26 k 2024-04-25T10:26:49.542 INFO:teuthology.orchestra.run.smithi203.stdout: python3-jsonpointer noarch 2.0-4.el9 appstream 19 k 2024-04-25T10:26:49.542 INFO:teuthology.orchestra.run.smithi203.stdout: python3-kubernetes noarch 1:26.1.0-2.el9 epel 1.0 M 2024-04-25T10:26:49.542 INFO:teuthology.orchestra.run.smithi203.stdout: python3-logutils noarch 0.3.5-21.el9 epel 46 k 2024-04-25T10:26:49.542 INFO:teuthology.orchestra.run.smithi203.stdout: python3-mako noarch 1.1.4-6.el9 appstream 172 k 2024-04-25T10:26:49.542 INFO:teuthology.orchestra.run.smithi203.stdout: python3-markupsafe x86_64 1.1.1-12.el9 appstream 35 k 2024-04-25T10:26:49.542 INFO:teuthology.orchestra.run.smithi203.stdout: python3-more-itertools noarch 8.12.0-2.el9 epel 79 k 2024-04-25T10:26:49.542 INFO:teuthology.orchestra.run.smithi203.stdout: python3-natsort noarch 7.1.1-5.el9 epel 58 k 2024-04-25T10:26:49.542 INFO:teuthology.orchestra.run.smithi203.stdout: python3-numpy x86_64 1:1.20.1-5.el9 appstream 5.1 M 2024-04-25T10:26:49.542 INFO:teuthology.orchestra.run.smithi203.stdout: python3-numpy-f2py x86_64 1:1.20.1-5.el9 appstream 250 k 2024-04-25T10:26:49.542 INFO:teuthology.orchestra.run.smithi203.stdout: python3-oauthlib noarch 3.1.1-5.el9 appstream 222 k 2024-04-25T10:26:49.542 INFO:teuthology.orchestra.run.smithi203.stdout: python3-packaging noarch 20.9-5.el9 appstream 77 k 2024-04-25T10:26:49.542 INFO:teuthology.orchestra.run.smithi203.stdout: python3-pecan noarch 1.4.2-3.el9 epel 272 k 2024-04-25T10:26:49.543 INFO:teuthology.orchestra.run.smithi203.stdout: python3-ply noarch 3.11-14.el9 baseos 106 k 2024-04-25T10:26:49.543 INFO:teuthology.orchestra.run.smithi203.stdout: python3-portend noarch 3.1.0-2.el9 epel 16 k 2024-04-25T10:26:49.543 INFO:teuthology.orchestra.run.smithi203.stdout: python3-prettytable noarch 0.7.2-27.el9 appstream 42 k 2024-04-25T10:26:49.543 INFO:teuthology.orchestra.run.smithi203.stdout: python3-protobuf noarch 3.14.0-13.el9 appstream 269 k 2024-04-25T10:26:49.543 INFO:teuthology.orchestra.run.smithi203.stdout: python3-pyOpenSSL noarch 21.0.0-1.el9 epel 90 k 2024-04-25T10:26:49.543 INFO:teuthology.orchestra.run.smithi203.stdout: python3-pyasn1 noarch 0.4.8-6.el9 appstream 159 k 2024-04-25T10:26:49.543 INFO:teuthology.orchestra.run.smithi203.stdout: python3-pyasn1-modules noarch 0.4.8-6.el9 appstream 279 k 2024-04-25T10:26:49.543 INFO:teuthology.orchestra.run.smithi203.stdout: python3-pycparser noarch 2.20-6.el9 baseos 135 k 2024-04-25T10:26:49.543 INFO:teuthology.orchestra.run.smithi203.stdout: python3-pyperclip noarch 1.8.2-2.el9 epel 28 k 2024-04-25T10:26:49.543 INFO:teuthology.orchestra.run.smithi203.stdout: python3-pytz noarch 2021.1-5.el9 appstream 51 k 2024-04-25T10:26:49.543 INFO:teuthology.orchestra.run.smithi203.stdout: python3-repoze-lru noarch 0.7-16.el9 epel 31 k 2024-04-25T10:26:49.543 INFO:teuthology.orchestra.run.smithi203.stdout: python3-requests noarch 2.25.1-8.el9 baseos 125 k 2024-04-25T10:26:49.543 INFO:teuthology.orchestra.run.smithi203.stdout: python3-requests-oauthlib noarch 1.3.0-12.el9 appstream 54 k 2024-04-25T10:26:49.543 INFO:teuthology.orchestra.run.smithi203.stdout: python3-routes noarch 2.5.1-5.el9 epel 188 k 2024-04-25T10:26:49.543 INFO:teuthology.orchestra.run.smithi203.stdout: python3-rsa noarch 4.9-2.el9 epel 59 k 2024-04-25T10:26:49.544 INFO:teuthology.orchestra.run.smithi203.stdout: python3-scipy x86_64 1.6.2-8.el9 appstream 16 M 2024-04-25T10:26:49.544 INFO:teuthology.orchestra.run.smithi203.stdout: python3-tempora noarch 5.0.0-2.el9 epel 36 k 2024-04-25T10:26:49.544 INFO:teuthology.orchestra.run.smithi203.stdout: python3-toml noarch 0.10.2-6.el9 appstream 42 k 2024-04-25T10:26:49.544 INFO:teuthology.orchestra.run.smithi203.stdout: python3-typing-extensions noarch 4.4.0-2.el9 epel 51 k 2024-04-25T10:26:49.544 INFO:teuthology.orchestra.run.smithi203.stdout: python3-urllib3 noarch 1.26.5-5.el9 baseos 215 k 2024-04-25T10:26:49.544 INFO:teuthology.orchestra.run.smithi203.stdout: python3-wcwidth noarch 0.2.5-8.el9 appstream 44 k 2024-04-25T10:26:49.544 INFO:teuthology.orchestra.run.smithi203.stdout: python3-webob noarch 1.8.7-6.el9 epel 230 k 2024-04-25T10:26:49.544 INFO:teuthology.orchestra.run.smithi203.stdout: python3-websocket-client noarch 1.2.3-2.el9 epel 90 k 2024-04-25T10:26:49.544 INFO:teuthology.orchestra.run.smithi203.stdout: python3-werkzeug noarch 2.0.3-3.el9.1 epel 427 k 2024-04-25T10:26:49.544 INFO:teuthology.orchestra.run.smithi203.stdout: python3-zc-lockfile noarch 2.0-10.el9 epel 20 k 2024-04-25T10:26:49.544 INFO:teuthology.orchestra.run.smithi203.stdout: qatlib x86_64 24.02.0-1.el9 appstream 221 k 2024-04-25T10:26:49.544 INFO:teuthology.orchestra.run.smithi203.stdout: qatzip-libs x86_64 1.2.0-1.el9 appstream 47 k 2024-04-25T10:26:49.544 INFO:teuthology.orchestra.run.smithi203.stdout: re2 x86_64 1:20211101-3.el9 epel 192 k 2024-04-25T10:26:49.544 INFO:teuthology.orchestra.run.smithi203.stdout: socat x86_64 1.7.4.1-5.el9 appstream 305 k 2024-04-25T10:26:49.544 INFO:teuthology.orchestra.run.smithi203.stdout: thrift x86_64 0.15.0-2.el9 epel 1.6 M 2024-04-25T10:26:49.545 INFO:teuthology.orchestra.run.smithi203.stdout: uuid x86_64 1.6.2-55.el9 appstream 58 k 2024-04-25T10:26:49.545 INFO:teuthology.orchestra.run.smithi203.stdout: xmlstarlet x86_64 1.6.1-20.el9 appstream 64 k 2024-04-25T10:26:49.545 INFO:teuthology.orchestra.run.smithi203.stdout: zlib-devel x86_64 1.2.11-41.el9 appstream 45 k 2024-04-25T10:26:49.545 INFO:teuthology.orchestra.run.smithi203.stdout:Installing weak dependencies: 2024-04-25T10:26:49.545 INFO:teuthology.orchestra.run.smithi203.stdout: qatlib-service x86_64 24.02.0-1.el9 appstream 36 k 2024-04-25T10:26:49.545 INFO:teuthology.orchestra.run.smithi203.stdout: 2024-04-25T10:26:49.545 INFO:teuthology.orchestra.run.smithi203.stdout:Transaction Summary 2024-04-25T10:26:49.545 INFO:teuthology.orchestra.run.smithi203.stdout:======================================================================================= 2024-04-25T10:26:49.545 INFO:teuthology.orchestra.run.smithi203.stdout:Install 148 Packages 2024-04-25T10:26:49.545 INFO:teuthology.orchestra.run.smithi203.stdout:Upgrade 4 Packages 2024-04-25T10:26:49.545 INFO:teuthology.orchestra.run.smithi203.stdout: 2024-04-25T10:26:49.546 INFO:teuthology.orchestra.run.smithi203.stdout:Total download size: 214 M 2024-04-25T10:26:49.546 INFO:teuthology.orchestra.run.smithi203.stdout:Downloading Packages: 2024-04-25T10:26:49.875 INFO:teuthology.orchestra.run.smithi037.stdout:(7/152): ceph-mon-19.0.0-3291.g994e2f20.el9.x86 5.9 MB/s | 4.9 MB 00:00 2024-04-25T10:26:50.771 INFO:teuthology.orchestra.run.smithi203.stdout:(1/152): ceph-19.0.0-3291.g994e2f20.el9.x86_64. 20 kB/s | 6.6 kB 00:00 2024-04-25T10:26:51.172 INFO:teuthology.orchestra.run.smithi203.stdout:(2/152): ceph-fuse-19.0.0-3291.g994e2f20.el9.x8 2.1 MB/s | 863 kB 00:00 2024-04-25T10:26:51.183 INFO:teuthology.orchestra.run.smithi037.stdout:(8/152): ceph-common-19.0.0-3291.g994e2f20.el9. 4.8 MB/s | 20 MB 00:04 2024-04-25T10:26:51.221 INFO:teuthology.orchestra.run.smithi203.stdout:(3/152): ceph-immutable-object-cache-19.0.0-329 2.9 MB/s | 147 kB 00:00 2024-04-25T10:26:51.300 INFO:teuthology.orchestra.run.smithi037.stdout:(9/152): ceph-radosgw-19.0.0-3291.g994e2f20.el9 7.1 MB/s | 10 MB 00:01 2024-04-25T10:26:51.325 INFO:teuthology.orchestra.run.smithi037.stdout:(10/152): ceph-selinux-19.0.0-3291.g994e2f20.el 177 kB/s | 25 kB 00:00 2024-04-25T10:26:51.442 INFO:teuthology.orchestra.run.smithi037.stdout:(11/152): cephfs-mirror-19.0.0-3291.g994e2f20.e 1.9 MB/s | 224 kB 00:00 2024-04-25T10:26:51.551 INFO:teuthology.orchestra.run.smithi037.stdout:(12/152): cephfs-shell-19.0.0-3291.g994e2f20.el 249 kB/s | 27 kB 00:00 2024-04-25T10:26:51.668 INFO:teuthology.orchestra.run.smithi037.stdout:(13/152): libcephfs-devel-19.0.0-3291.g994e2f20 272 kB/s | 32 kB 00:00 2024-04-25T10:26:51.755 INFO:teuthology.orchestra.run.smithi203.stdout:(4/152): ceph-mds-19.0.0-3291.g994e2f20.el9.x86 4.2 MB/s | 2.3 MB 00:00 2024-04-25T10:26:52.076 INFO:teuthology.orchestra.run.smithi037.stdout:(14/152): libcephfs2-19.0.0-3291.g994e2f20.el9. 1.8 MB/s | 756 kB 00:00 2024-04-25T10:26:52.177 INFO:teuthology.orchestra.run.smithi037.stdout:(15/152): libcephsqlite-19.0.0-3291.g994e2f20.e 1.5 MB/s | 158 kB 00:00 2024-04-25T10:26:52.189 INFO:teuthology.orchestra.run.smithi203.stdout:(5/152): ceph-mgr-19.0.0-3291.g994e2f20.el9.x86 3.5 MB/s | 1.5 MB 00:00 2024-04-25T10:26:52.269 INFO:teuthology.orchestra.run.smithi037.stdout:(16/152): librados-devel-19.0.0-3291.g994e2f20. 1.4 MB/s | 127 kB 00:00 2024-04-25T10:26:52.444 INFO:teuthology.orchestra.run.smithi037.stdout:(17/152): ceph-osd-19.0.0-3291.g994e2f20.el9.x8 5.3 MB/s | 18 MB 00:03 2024-04-25T10:26:52.478 INFO:teuthology.orchestra.run.smithi037.stdout:(18/152): libradosstriper1-19.0.0-3291.g994e2f2 2.3 MB/s | 497 kB 00:00 2024-04-25T10:26:52.586 INFO:teuthology.orchestra.run.smithi037.stdout:(19/152): python3-ceph-argparse-19.0.0-3291.g99 417 kB/s | 45 kB 00:00 2024-04-25T10:26:52.687 INFO:teuthology.orchestra.run.smithi037.stdout:(20/152): python3-ceph-common-19.0.0-3291.g994e 1.3 MB/s | 135 kB 00:00 2024-04-25T10:26:52.803 INFO:teuthology.orchestra.run.smithi037.stdout:(21/152): python3-cephfs-19.0.0-3291.g994e2f20. 1.4 MB/s | 161 kB 00:00 2024-04-25T10:26:52.897 INFO:teuthology.orchestra.run.smithi203.stdout:(6/152): ceph-base-19.0.0-3291.g994e2f20.el9.x8 2.2 MB/s | 5.4 MB 00:02 2024-04-25T10:26:52.912 INFO:teuthology.orchestra.run.smithi037.stdout:(22/152): python3-rados-19.0.0-3291.g994e2f20.e 2.9 MB/s | 321 kB 00:00 2024-04-25T10:26:53.038 INFO:teuthology.orchestra.run.smithi037.stdout:(23/152): python3-rbd-19.0.0-3291.g994e2f20.el9 2.3 MB/s | 298 kB 00:00 2024-04-25T10:26:53.147 INFO:teuthology.orchestra.run.smithi037.stdout:(24/152): python3-rgw-19.0.0-3291.g994e2f20.el9 920 kB/s | 100 kB 00:00 2024-04-25T10:26:53.255 INFO:teuthology.orchestra.run.smithi037.stdout:(25/152): rbd-fuse-19.0.0-3291.g994e2f20.el9.x8 796 kB/s | 86 kB 00:00 2024-04-25T10:26:53.598 INFO:teuthology.orchestra.run.smithi203.stdout:(7/152): ceph-mon-19.0.0-3291.g994e2f20.el9.x86 3.5 MB/s | 4.9 MB 00:01 2024-04-25T10:26:53.922 INFO:teuthology.orchestra.run.smithi037.stdout:(26/152): rbd-mirror-19.0.0-3291.g994e2f20.el9. 4.6 MB/s | 3.0 MB 00:00 2024-04-25T10:26:53.997 INFO:teuthology.orchestra.run.smithi037.stdout:(27/152): librgw2-19.0.0-3291.g994e2f20.el9.x86 3.3 MB/s | 5.2 MB 00:01 2024-04-25T10:26:54.023 INFO:teuthology.orchestra.run.smithi037.stdout:(28/152): rbd-nbd-19.0.0-3291.g994e2f20.el9.x86 1.7 MB/s | 169 kB 00:00 2024-04-25T10:26:54.106 INFO:teuthology.orchestra.run.smithi037.stdout:(29/152): ceph-grafana-dashboards-19.0.0-3291.g 315 kB/s | 34 kB 00:00 2024-04-25T10:26:54.148 INFO:teuthology.orchestra.run.smithi037.stdout:(30/152): ceph-mgr-cephadm-19.0.0-3291.g994e2f2 1.1 MB/s | 142 kB 00:00 2024-04-25T10:26:54.298 INFO:teuthology.orchestra.run.smithi203.stdout:(8/152): ceph-common-19.0.0-3291.g994e2f20.el9. 5.1 MB/s | 20 MB 00:03 2024-04-25T10:26:54.406 INFO:teuthology.orchestra.run.smithi203.stdout:(9/152): ceph-selinux-19.0.0-3291.g994e2f20.el9 233 kB/s | 25 kB 00:00 2024-04-25T10:26:55.074 INFO:teuthology.orchestra.run.smithi203.stdout:(10/152): ceph-osd-19.0.0-3291.g994e2f20.el9.x8 8.2 MB/s | 18 MB 00:02 2024-04-25T10:26:55.165 INFO:teuthology.orchestra.run.smithi203.stdout:(11/152): cephfs-mirror-19.0.0-3291.g994e2f20.e 2.4 MB/s | 224 kB 00:00 2024-04-25T10:26:55.249 INFO:teuthology.orchestra.run.smithi203.stdout:(12/152): cephfs-shell-19.0.0-3291.g994e2f20.el 323 kB/s | 27 kB 00:00 2024-04-25T10:26:55.332 INFO:teuthology.orchestra.run.smithi203.stdout:(13/152): libcephfs-devel-19.0.0-3291.g994e2f20 381 kB/s | 32 kB 00:00 2024-04-25T10:26:55.482 INFO:teuthology.orchestra.run.smithi037.stdout:(31/152): ceph-mgr-diskprediction-local-19.0.0- 5.5 MB/s | 7.4 MB 00:01 2024-04-25T10:26:55.541 INFO:teuthology.orchestra.run.smithi203.stdout:(14/152): libcephfs2-19.0.0-3291.g994e2f20.el9. 3.5 MB/s | 756 kB 00:00 2024-04-25T10:26:55.565 INFO:teuthology.orchestra.run.smithi037.stdout:(32/152): ceph-mgr-modules-core-19.0.0-3291.g99 2.9 MB/s | 249 kB 00:00 2024-04-25T10:26:55.608 INFO:teuthology.orchestra.run.smithi203.stdout:(15/152): libcephsqlite-19.0.0-3291.g994e2f20.e 2.3 MB/s | 158 kB 00:00 2024-04-25T10:26:55.667 INFO:teuthology.orchestra.run.smithi203.stdout:(16/152): librados-devel-19.0.0-3291.g994e2f20. 2.1 MB/s | 127 kB 00:00 2024-04-25T10:26:55.682 INFO:teuthology.orchestra.run.smithi037.stdout:(33/152): ceph-mgr-dashboard-19.0.0-3291.g994e2 2.2 MB/s | 3.5 MB 00:01 2024-04-25T10:26:55.707 INFO:teuthology.orchestra.run.smithi037.stdout:(34/152): ceph-mgr-rook-19.0.0-3291.g994e2f20.e 347 kB/s | 49 kB 00:00 2024-04-25T10:26:55.766 INFO:teuthology.orchestra.run.smithi037.stdout:(35/152): ceph-prometheus-alerts-19.0.0-3291.g9 197 kB/s | 16 kB 00:00 2024-04-25T10:26:55.791 INFO:teuthology.orchestra.run.smithi037.stdout:(36/152): ceph-volume-19.0.0-3291.g994e2f20.el9 3.2 MB/s | 273 kB 00:00 2024-04-25T10:26:55.792 INFO:teuthology.orchestra.run.smithi203.stdout:(17/152): libradosstriper1-19.0.0-3291.g994e2f2 3.9 MB/s | 497 kB 00:00 2024-04-25T10:26:55.850 INFO:teuthology.orchestra.run.smithi037.stdout:(37/152): cephfs-top-19.0.0-3291.g994e2f20.el9. 313 kB/s | 18 kB 00:00 2024-04-25T10:26:56.000 INFO:teuthology.orchestra.run.smithi037.stdout:(38/152): fuse-2.9.9-15.el9.x86_64.rpm 535 kB/s | 80 kB 00:00 2024-04-25T10:26:56.050 INFO:teuthology.orchestra.run.smithi037.stdout:(39/152): cephadm-19.0.0-3291.g994e2f20.el9.noa 2.6 MB/s | 762 kB 00:00 2024-04-25T10:26:56.159 INFO:teuthology.orchestra.run.smithi037.stdout:(40/152): libgfortran-11.4.1-3.el9.x86_64.rpm 5.0 MB/s | 803 kB 00:00 2024-04-25T10:26:56.226 INFO:teuthology.orchestra.run.smithi037.stdout:(41/152): python3-cffi-1.14.5-5.el9.x86_64.rpm 3.7 MB/s | 253 kB 00:00 2024-04-25T10:26:56.270 INFO:teuthology.orchestra.run.smithi037.stdout:(42/152): libquadmath-11.4.1-3.el9.x86_64.rpm 891 kB/s | 195 kB 00:00 2024-04-25T10:26:56.275 INFO:teuthology.orchestra.run.smithi203.stdout:(18/152): librgw2-19.0.0-3291.g994e2f20.el9.x86 11 MB/s | 5.2 MB 00:00 2024-04-25T10:26:56.326 INFO:teuthology.orchestra.run.smithi203.stdout:(19/152): python3-ceph-argparse-19.0.0-3291.g99 909 kB/s | 45 kB 00:00 2024-04-25T10:26:56.337 INFO:teuthology.orchestra.run.smithi037.stdout:(43/152): python3-cryptography-36.0.1-4.el9.x86 11 MB/s | 1.2 MB 00:00 2024-04-25T10:26:56.362 INFO:teuthology.orchestra.run.smithi037.stdout:(44/152): python3-ply-3.11-14.el9.noarch.rpm 1.1 MB/s | 106 kB 00:00 2024-04-25T10:26:56.376 INFO:teuthology.orchestra.run.smithi203.stdout:(20/152): python3-ceph-common-19.0.0-3291.g994e 2.6 MB/s | 135 kB 00:00 2024-04-25T10:26:56.396 INFO:teuthology.orchestra.run.smithi037.stdout:(45/152): python3-pycparser-2.20-6.el9.noarch.r 2.3 MB/s | 135 kB 00:00 2024-04-25T10:26:56.435 INFO:teuthology.orchestra.run.smithi203.stdout:(21/152): python3-cephfs-19.0.0-3291.g994e2f20. 2.7 MB/s | 161 kB 00:00 2024-04-25T10:26:56.455 INFO:teuthology.orchestra.run.smithi037.stdout:(46/152): python3-requests-2.25.1-8.el9.noarch. 1.3 MB/s | 125 kB 00:00 2024-04-25T10:26:56.471 INFO:teuthology.orchestra.run.smithi037.stdout:(47/152): python3-urllib3-1.26.5-5.el9.noarch.r 2.8 MB/s | 215 kB 00:00 2024-04-25T10:26:56.527 INFO:teuthology.orchestra.run.smithi203.stdout:(22/152): python3-rados-19.0.0-3291.g994e2f20.e 3.4 MB/s | 321 kB 00:00 2024-04-25T10:26:56.572 INFO:teuthology.orchestra.run.smithi037.stdout:(48/152): automake-1.16.2-8.el9.noarch.rpm 5.8 MB/s | 689 kB 00:00 2024-04-25T10:26:56.614 INFO:teuthology.orchestra.run.smithi037.stdout:(49/152): bison-3.7.4-5.el9.x86_64.rpm 6.5 MB/s | 943 kB 00:00 2024-04-25T10:26:56.619 INFO:teuthology.orchestra.run.smithi203.stdout:(23/152): python3-rbd-19.0.0-3291.g994e2f20.el9 3.2 MB/s | 298 kB 00:00 2024-04-25T10:26:56.664 INFO:teuthology.orchestra.run.smithi037.stdout:(50/152): elfutils-libelf-devel-0.190-2.el9.x86 432 kB/s | 21 kB 00:00 2024-04-25T10:26:56.681 INFO:teuthology.orchestra.run.smithi037.stdout:(51/152): boost-program-options-1.75.0-8.el9.x8 985 kB/s | 107 kB 00:00 2024-04-25T10:26:56.686 INFO:teuthology.orchestra.run.smithi203.stdout:(24/152): python3-rgw-19.0.0-3291.g994e2f20.el9 1.5 MB/s | 100 kB 00:00 2024-04-25T10:26:56.731 INFO:teuthology.orchestra.run.smithi037.stdout:(52/152): flexiblas-3.0.4-8.el9.x86_64.rpm 654 kB/s | 33 kB 00:00 2024-04-25T10:26:56.745 INFO:teuthology.orchestra.run.smithi203.stdout:(25/152): rbd-fuse-19.0.0-3291.g994e2f20.el9.x8 1.4 MB/s | 86 kB 00:00 2024-04-25T10:26:56.765 INFO:teuthology.orchestra.run.smithi037.stdout:(53/152): flex-2.6.4-9.el9.x86_64.rpm 3.1 MB/s | 313 kB 00:00 2024-04-25T10:26:56.815 INFO:teuthology.orchestra.run.smithi037.stdout:(54/152): flexiblas-openblas-openmp-3.0.4-8.el9 365 kB/s | 18 kB 00:00 2024-04-25T10:26:56.857 INFO:teuthology.orchestra.run.smithi037.stdout:(55/152): libaio-devel-0.3.111-13.el9.x86_64.rp 267 kB/s | 11 kB 00:00 2024-04-25T10:26:56.924 INFO:teuthology.orchestra.run.smithi037.stdout:(56/152): libpmemobj-1.12.1-1.el9.x86_64.rpm 2.4 MB/s | 160 kB 00:00 2024-04-25T10:26:56.983 INFO:teuthology.orchestra.run.smithi037.stdout:(57/152): flexiblas-netlib-3.0.4-8.el9.x86_64.r 12 MB/s | 3.0 MB 00:00 2024-04-25T10:26:57.033 INFO:teuthology.orchestra.run.smithi037.stdout:(58/152): librabbitmq-0.11.0-7.el9.x86_64.rpm 417 kB/s | 45 kB 00:00 2024-04-25T10:26:57.070 INFO:teuthology.orchestra.run.smithi203.stdout:(26/152): rbd-mirror-19.0.0-3291.g994e2f20.el9. 9.3 MB/s | 3.0 MB 00:00 2024-04-25T10:26:57.100 INFO:teuthology.orchestra.run.smithi037.stdout:(59/152): librdkafka-1.6.1-102.el9.x86_64.rpm 5.5 MB/s | 662 kB 00:00 2024-04-25T10:26:57.121 INFO:teuthology.orchestra.run.smithi203.stdout:(27/152): rbd-nbd-19.0.0-3291.g994e2f20.el9.x86 3.3 MB/s | 169 kB 00:00 2024-04-25T10:26:57.142 INFO:teuthology.orchestra.run.smithi037.stdout:(60/152): libtool-2.4.6-45.el9.x86_64.rpm 5.2 MB/s | 582 kB 00:00 2024-04-25T10:26:57.163 INFO:teuthology.orchestra.run.smithi203.stdout:(28/152): ceph-grafana-dashboards-19.0.0-3291.g 823 kB/s | 34 kB 00:00 2024-04-25T10:26:57.167 INFO:teuthology.orchestra.run.smithi037.stdout:(61/152): libuuid-devel-2.37.4-18.el9.x86_64.rp 381 kB/s | 25 kB 00:00 2024-04-25T10:26:57.217 INFO:teuthology.orchestra.run.smithi037.stdout:(62/152): libzstd-devel-1.5.1-2.el9.x86_64.rpm 631 kB/s | 47 kB 00:00 2024-04-25T10:26:57.268 INFO:teuthology.orchestra.run.smithi037.stdout:(63/152): lttng-ust-2.12.0-6.el9.x86_64.rpm 2.8 MB/s | 292 kB 00:00 2024-04-25T10:26:57.296 INFO:teuthology.orchestra.run.smithi203.stdout:(29/152): ceph-radosgw-19.0.0-3291.g994e2f20.el 2.7 MB/s | 10 MB 00:03 2024-04-25T10:26:57.309 INFO:teuthology.orchestra.run.smithi037.stdout:(64/152): lua-5.4.4-4.el9.x86_64.rpm 2.0 MB/s | 188 kB 00:00 2024-04-25T10:26:57.321 INFO:teuthology.orchestra.run.smithi203.stdout:(30/152): ceph-mgr-cephadm-19.0.0-3291.g994e2f2 897 kB/s | 142 kB 00:00 2024-04-25T10:26:57.335 INFO:teuthology.orchestra.run.smithi037.stdout:(65/152): openblas-0.3.21-2.el9.x86_64.rpm 527 kB/s | 35 kB 00:00 2024-04-25T10:26:57.585 INFO:teuthology.orchestra.run.smithi037.stdout:(66/152): openblas-openmp-0.3.21-2.el9.x86_64.r 17 MB/s | 4.7 MB 00:00 2024-04-25T10:26:57.644 INFO:teuthology.orchestra.run.smithi037.stdout:(67/152): openssl-devel-3.2.1-1.el9.x86_64.rpm 14 MB/s | 4.4 MB 00:00 2024-04-25T10:26:57.762 INFO:teuthology.orchestra.run.smithi037.stdout:(68/152): postgresql-contrib-13.14-1.el9.x86_64 7.6 MB/s | 911 kB 00:00 2024-04-25T10:26:57.811 INFO:teuthology.orchestra.run.smithi037.stdout:(69/152): postgresql-13.14-1.el9.x86_64.rpm 7.0 MB/s | 1.6 MB 00:00 2024-04-25T10:26:57.844 INFO:teuthology.orchestra.run.smithi037.stdout:(70/152): postgresql-private-libs-13.14-1.el9.x 1.6 MB/s | 133 kB 00:00 2024-04-25T10:26:57.914 INFO:teuthology.orchestra.run.smithi203.stdout:(31/152): ceph-mgr-diskprediction-local-19.0.0- 12 MB/s | 7.4 MB 00:00 2024-04-25T10:26:57.964 INFO:teuthology.orchestra.run.smithi203.stdout:(32/152): ceph-mgr-modules-core-19.0.0-3291.g99 4.9 MB/s | 249 kB 00:00 2024-04-25T10:26:58.014 INFO:teuthology.orchestra.run.smithi203.stdout:(33/152): ceph-mgr-rook-19.0.0-3291.g994e2f20.e 986 kB/s | 49 kB 00:00 2024-04-25T10:26:58.056 INFO:teuthology.orchestra.run.smithi203.stdout:(34/152): ceph-prometheus-alerts-19.0.0-3291.g9 396 kB/s | 16 kB 00:00 2024-04-25T10:26:58.115 INFO:teuthology.orchestra.run.smithi203.stdout:(35/152): ceph-mgr-dashboard-19.0.0-3291.g994e2 4.3 MB/s | 3.5 MB 00:00 2024-04-25T10:26:58.157 INFO:teuthology.orchestra.run.smithi203.stdout:(36/152): ceph-volume-19.0.0-3291.g994e2f20.el9 2.7 MB/s | 273 kB 00:00 2024-04-25T10:26:58.191 INFO:teuthology.orchestra.run.smithi203.stdout:(37/152): cephadm-19.0.0-3291.g994e2f20.el9.noa 9.9 MB/s | 762 kB 00:00 2024-04-25T10:26:58.216 INFO:teuthology.orchestra.run.smithi203.stdout:(38/152): cephfs-top-19.0.0-3291.g994e2f20.el9. 311 kB/s | 18 kB 00:00 2024-04-25T10:26:58.295 INFO:teuthology.orchestra.run.smithi037.stdout:(71/152): ceph-test-19.0.0-3291.g994e2f20.el9.x 6.2 MB/s | 43 MB 00:06 2024-04-25T10:26:58.362 INFO:teuthology.orchestra.run.smithi037.stdout:(72/152): protobuf-3.14.0-13.el9.x86_64.rpm 2.0 MB/s | 1.0 MB 00:00 2024-04-25T10:26:58.387 INFO:teuthology.orchestra.run.smithi037.stdout:(73/152): python3-attrs-20.3.0-7.el9.noarch.rpm 956 kB/s | 87 kB 00:00 2024-04-25T10:26:58.408 INFO:teuthology.orchestra.run.smithi203.stdout:(39/152): fuse-2.9.9-15.el9.x86_64.rpm 371 kB/s | 80 kB 00:00 2024-04-25T10:26:58.496 INFO:teuthology.orchestra.run.smithi037.stdout:(74/152): postgresql-server-13.14-1.el9.x86_64. 8.4 MB/s | 5.8 MB 00:00 2024-04-25T10:26:58.500 INFO:teuthology.orchestra.run.smithi203.stdout:(40/152): libquadmath-11.4.1-3.el9.x86_64.rpm 2.1 MB/s | 195 kB 00:00 2024-04-25T10:26:58.521 INFO:teuthology.orchestra.run.smithi037.stdout:(75/152): python3-devel-3.9.18-3.el9.x86_64.rpm 1.8 MB/s | 244 kB 00:00 2024-04-25T10:26:58.525 INFO:teuthology.orchestra.run.smithi203.stdout:(41/152): libgfortran-11.4.1-3.el9.x86_64.rpm 2.5 MB/s | 803 kB 00:00 2024-04-25T10:26:58.571 INFO:teuthology.orchestra.run.smithi037.stdout:(76/152): python3-jsonpatch-1.21-16.el9.noarch. 513 kB/s | 26 kB 00:00 2024-04-25T10:26:58.592 INFO:teuthology.orchestra.run.smithi203.stdout:(42/152): python3-cffi-1.14.5-5.el9.x86_64.rpm 2.7 MB/s | 253 kB 00:00 2024-04-25T10:26:58.596 INFO:teuthology.orchestra.run.smithi037.stdout:(77/152): python3-jinja2-2.11.3-5.el9.noarch.rp 2.4 MB/s | 248 kB 00:00 2024-04-25T10:26:58.626 INFO:teuthology.orchestra.run.smithi203.stdout:(43/152): python3-cryptography-36.0.1-4.el9.x86 12 MB/s | 1.2 MB 00:00 2024-04-25T10:26:58.638 INFO:teuthology.orchestra.run.smithi037.stdout:(78/152): python3-jsonpointer-2.0-4.el9.noarch. 287 kB/s | 19 kB 00:00 2024-04-25T10:26:58.651 INFO:teuthology.orchestra.run.smithi203.stdout:(44/152): python3-ply-3.11-14.el9.noarch.rpm 1.8 MB/s | 106 kB 00:00 2024-04-25T10:26:58.680 INFO:teuthology.orchestra.run.smithi037.stdout:(79/152): python3-mako-1.1.4-6.el9.noarch.rpm 2.0 MB/s | 172 kB 00:00 2024-04-25T10:26:58.685 INFO:teuthology.orchestra.run.smithi203.stdout:(45/152): python3-pycparser-2.20-6.el9.noarch.r 2.2 MB/s | 135 kB 00:00 2024-04-25T10:26:58.706 INFO:teuthology.orchestra.run.smithi037.stdout:(80/152): python3-markupsafe-1.1.1-12.el9.x86_6 519 kB/s | 35 kB 00:00 2024-04-25T10:26:58.710 INFO:teuthology.orchestra.run.smithi203.stdout:(46/152): python3-requests-2.25.1-8.el9.noarch. 2.1 MB/s | 125 kB 00:00 2024-04-25T10:26:58.744 INFO:teuthology.orchestra.run.smithi203.stdout:(47/152): python3-urllib3-1.26.5-5.el9.noarch.r 3.6 MB/s | 215 kB 00:00 2024-04-25T10:26:58.802 INFO:teuthology.orchestra.run.smithi203.stdout:(48/152): automake-1.16.2-8.el9.noarch.rpm 7.3 MB/s | 689 kB 00:00 2024-04-25T10:26:58.806 INFO:teuthology.orchestra.run.smithi037.stdout:(81/152): python3-babel-2.9.1-2.el9.noarch.rpm 13 MB/s | 6.0 MB 00:00 2024-04-25T10:26:58.856 INFO:teuthology.orchestra.run.smithi037.stdout:(82/152): python3-numpy-f2py-1.20.1-5.el9.x86_6 1.6 MB/s | 250 kB 00:00 2024-04-25T10:26:58.861 INFO:teuthology.orchestra.run.smithi203.stdout:(49/152): bison-3.7.4-5.el9.x86_64.rpm 7.9 MB/s | 943 kB 00:00 2024-04-25T10:26:58.907 INFO:teuthology.orchestra.run.smithi037.stdout:(83/152): python3-oauthlib-3.1.1-5.el9.noarch.r 2.2 MB/s | 222 kB 00:00 2024-04-25T10:26:58.932 INFO:teuthology.orchestra.run.smithi037.stdout:(84/152): python3-packaging-20.9-5.el9.noarch.r 1.0 MB/s | 77 kB 00:00 2024-04-25T10:26:58.982 INFO:teuthology.orchestra.run.smithi037.stdout:(85/152): python3-prettytable-0.7.2-27.el9.noar 564 kB/s | 42 kB 00:00 2024-04-25T10:26:59.020 INFO:teuthology.orchestra.run.smithi203.stdout:(50/152): boost-program-options-1.75.0-8.el9.x8 492 kB/s | 107 kB 00:00 2024-04-25T10:26:59.032 INFO:teuthology.orchestra.run.smithi037.stdout:(86/152): python3-protobuf-3.14.0-13.el9.noarch 2.6 MB/s | 269 kB 00:00 2024-04-25T10:26:59.045 INFO:teuthology.orchestra.run.smithi203.stdout:(51/152): elfutils-libelf-devel-0.190-2.el9.x86 117 kB/s | 21 kB 00:00 2024-04-25T10:26:59.074 INFO:teuthology.orchestra.run.smithi037.stdout:(87/152): python3-pyasn1-0.4.8-6.el9.noarch.rpm 1.7 MB/s | 159 kB 00:00 2024-04-25T10:26:59.079 INFO:teuthology.orchestra.run.smithi203.stdout:(52/152): flex-2.6.4-9.el9.x86_64.rpm 5.2 MB/s | 313 kB 00:00 2024-04-25T10:26:59.104 INFO:teuthology.orchestra.run.smithi203.stdout:(53/152): flexiblas-3.0.4-8.el9.x86_64.rpm 557 kB/s | 33 kB 00:00 2024-04-25T10:26:59.158 INFO:teuthology.orchestra.run.smithi037.stdout:(88/152): python3-numpy-1.20.1-5.el9.x86_64.rpm 11 MB/s | 5.1 MB 00:00 2024-04-25T10:26:59.171 INFO:teuthology.orchestra.run.smithi203.stdout:(54/152): flexiblas-openblas-openmp-3.0.4-8.el9 274 kB/s | 18 kB 00:00 2024-04-25T10:26:59.183 INFO:teuthology.orchestra.run.smithi037.stdout:(89/152): python3-pytz-2021.1-5.el9.noarch.rpm 471 kB/s | 51 kB 00:00 2024-04-25T10:26:59.208 INFO:teuthology.orchestra.run.smithi037.stdout:(90/152): python3-pyasn1-modules-0.4.8-6.el9.no 1.6 MB/s | 279 kB 00:00 2024-04-25T10:26:59.230 INFO:teuthology.orchestra.run.smithi203.stdout:(55/152): libaio-devel-0.3.111-13.el9.x86_64.rp 191 kB/s | 11 kB 00:00 2024-04-25T10:26:59.234 INFO:teuthology.orchestra.run.smithi037.stdout:(91/152): python3-requests-oauthlib-1.3.0-12.el 712 kB/s | 54 kB 00:00 2024-04-25T10:26:59.284 INFO:teuthology.orchestra.run.smithi037.stdout:(92/152): python3-toml-0.10.2-6.el9.noarch.rpm 553 kB/s | 42 kB 00:00 2024-04-25T10:26:59.301 INFO:teuthology.orchestra.run.smithi037.stdout:(93/152): python3-wcwidth-0.2.5-8.el9.noarch.rp 662 kB/s | 44 kB 00:00 2024-04-25T10:26:59.313 INFO:teuthology.orchestra.run.smithi203.stdout:(56/152): libpmemobj-1.12.1-1.el9.x86_64.rpm 1.9 MB/s | 160 kB 00:00 2024-04-25T10:26:59.343 INFO:teuthology.orchestra.run.smithi037.stdout:(94/152): qatlib-service-24.02.0-1.el9.x86_64.r 865 kB/s | 36 kB 00:00 2024-04-25T10:26:59.364 INFO:teuthology.orchestra.run.smithi203.stdout:(57/152): flexiblas-netlib-3.0.4-8.el9.x86_64.r 11 MB/s | 3.0 MB 00:00 2024-04-25T10:26:59.376 INFO:teuthology.orchestra.run.smithi037.stdout:(95/152): qatlib-24.02.0-1.el9.x86_64.rpm 2.3 MB/s | 221 kB 00:00 2024-04-25T10:26:59.389 INFO:teuthology.orchestra.run.smithi203.stdout:(58/152): librabbitmq-0.11.0-7.el9.x86_64.rpm 600 kB/s | 45 kB 00:00 2024-04-25T10:26:59.427 INFO:teuthology.orchestra.run.smithi037.stdout:(96/152): qatzip-libs-1.2.0-1.el9.x86_64.rpm 566 kB/s | 47 kB 00:00 2024-04-25T10:26:59.431 INFO:teuthology.orchestra.run.smithi203.stdout:(59/152): librdkafka-1.6.1-102.el9.x86_64.rpm 9.7 MB/s | 662 kB 00:00 2024-04-25T10:26:59.465 INFO:teuthology.orchestra.run.smithi203.stdout:(60/152): libtool-2.4.6-45.el9.x86_64.rpm 7.6 MB/s | 582 kB 00:00 2024-04-25T10:26:59.477 INFO:teuthology.orchestra.run.smithi037.stdout:(97/152): socat-1.7.4.1-5.el9.x86_64.rpm 3.0 MB/s | 305 kB 00:00 2024-04-25T10:26:59.490 INFO:teuthology.orchestra.run.smithi203.stdout:(61/152): libuuid-devel-2.37.4-18.el9.x86_64.rp 433 kB/s | 25 kB 00:00 2024-04-25T10:26:59.502 INFO:teuthology.orchestra.run.smithi037.stdout:(98/152): uuid-1.6.2-55.el9.x86_64.rpm 768 kB/s | 58 kB 00:00 2024-04-25T10:26:59.534 INFO:teuthology.orchestra.run.smithi203.stdout:(62/152): libzstd-devel-1.5.1-2.el9.x86_64.rpm 810 kB/s | 47 kB 00:00 2024-04-25T10:26:59.549 INFO:teuthology.orchestra.run.smithi203.stdout:(63/152): lttng-ust-2.12.0-6.el9.x86_64.rpm 4.9 MB/s | 292 kB 00:00 2024-04-25T10:26:59.561 INFO:teuthology.orchestra.run.smithi037.stdout:(99/152): xfsprogs-devel-6.3.0-1.el9.x86_64.rpm 1.1 MB/s | 94 kB 00:00 2024-04-25T10:26:59.582 INFO:teuthology.orchestra.run.smithi203.stdout:(64/152): lua-5.4.4-4.el9.x86_64.rpm 3.1 MB/s | 188 kB 00:00 2024-04-25T10:26:59.586 INFO:teuthology.orchestra.run.smithi037.stdout:(100/152): xmlstarlet-1.6.1-20.el9.x86_64.rpm 762 kB/s | 64 kB 00:00 2024-04-25T10:26:59.608 INFO:teuthology.orchestra.run.smithi203.stdout:(65/152): openblas-0.3.21-2.el9.x86_64.rpm 600 kB/s | 35 kB 00:00 2024-04-25T10:26:59.679 INFO:teuthology.orchestra.run.smithi037.stdout:(101/152): zlib-devel-1.2.11-41.el9.x86_64.rpm 383 kB/s | 45 kB 00:00 2024-04-25T10:26:59.875 INFO:teuthology.orchestra.run.smithi203.stdout:(66/152): openblas-openmp-0.3.21-2.el9.x86_64.r 16 MB/s | 4.7 MB 00:00 2024-04-25T10:26:59.912 INFO:teuthology.orchestra.run.smithi037.stdout:(102/152): python3-scipy-1.6.2-8.el9.x86_64.rpm 22 MB/s | 16 MB 00:00 2024-04-25T10:26:59.929 INFO:teuthology.orchestra.run.smithi037.stdout:(103/152): lua-devel-5.4.4-4.el9.x86_64.rpm 65 kB/s | 22 kB 00:00 2024-04-25T10:26:59.933 INFO:teuthology.orchestra.run.smithi203.stdout:(67/152): openssl-devel-3.2.1-1.el9.x86_64.rpm 14 MB/s | 4.4 MB 00:00 2024-04-25T10:27:00.025 INFO:teuthology.orchestra.run.smithi203.stdout:(68/152): postgresql-13.14-1.el9.x86_64.rpm 11 MB/s | 1.6 MB 00:00 2024-04-25T10:27:00.051 INFO:teuthology.orchestra.run.smithi203.stdout:(69/152): postgresql-contrib-13.14-1.el9.x86_64 7.6 MB/s | 911 kB 00:00 2024-04-25T10:27:00.084 INFO:teuthology.orchestra.run.smithi203.stdout:(70/152): postgresql-private-libs-13.14-1.el9.x 2.2 MB/s | 133 kB 00:00 2024-04-25T10:27:00.155 INFO:teuthology.orchestra.run.smithi037.stdout:(104/152): abseil-cpp-20211102.0-3.el9.x86_64.r 2.2 MB/s | 548 kB 00:00 2024-04-25T10:27:00.179 INFO:teuthology.orchestra.run.smithi037.stdout:(105/152): gperftools-libs-2.9.1-2.el9.x86_64.r 1.2 MB/s | 309 kB 00:00 2024-04-25T10:27:00.221 INFO:teuthology.orchestra.run.smithi037.stdout:(106/152): grpc-data-1.46.7-10.el9.noarch.rpm 468 kB/s | 19 kB 00:00 2024-04-25T10:27:00.235 INFO:teuthology.orchestra.run.smithi203.stdout:(71/152): protobuf-3.14.0-13.el9.x86_64.rpm 6.7 MB/s | 1.0 MB 00:00 2024-04-25T10:27:00.310 INFO:teuthology.orchestra.run.smithi203.stdout:(72/152): python3-attrs-20.3.0-7.el9.noarch.rpm 1.1 MB/s | 87 kB 00:00 2024-04-25T10:27:00.347 INFO:teuthology.orchestra.run.smithi037.stdout:(107/152): libarrow-9.0.0-10.el9.x86_64.rpm 26 MB/s | 4.4 MB 00:00 2024-04-25T10:27:00.372 INFO:teuthology.orchestra.run.smithi037.stdout:(108/152): libarrow-doc-9.0.0-10.el9.noarch.rpm 175 kB/s | 26 kB 00:00 2024-04-25T10:27:00.414 INFO:teuthology.orchestra.run.smithi037.stdout:(109/152): liboath-2.6.7-2.el9.x86_64.rpm 1.1 MB/s | 49 kB 00:00 2024-04-25T10:27:00.439 INFO:teuthology.orchestra.run.smithi037.stdout:(110/152): libunwind-1.6.2-1.el9.x86_64.rpm 1.0 MB/s | 67 kB 00:00 2024-04-25T10:27:00.464 INFO:teuthology.orchestra.run.smithi037.stdout:(111/152): luarocks-3.9.2-1.el9.noarch.rpm 2.9 MB/s | 151 kB 00:00 2024-04-25T10:27:00.506 INFO:teuthology.orchestra.run.smithi037.stdout:(112/152): parquet-libs-9.0.0-10.el9.x86_64.rpm 12 MB/s | 838 kB 00:00 2024-04-25T10:27:00.531 INFO:teuthology.orchestra.run.smithi037.stdout:(113/152): python3-asyncssh-2.13.2-5.el9.noarch 8.2 MB/s | 548 kB 00:00 2024-04-25T10:27:00.536 INFO:teuthology.orchestra.run.smithi203.stdout:(73/152): postgresql-server-13.14-1.el9.x86_64. 12 MB/s | 5.8 MB 00:00 2024-04-25T10:27:00.557 INFO:teuthology.orchestra.run.smithi037.stdout:(114/152): python3-bcrypt-3.2.2-1.el9.x86_64.rp 883 kB/s | 43 kB 00:00 2024-04-25T10:27:00.582 INFO:teuthology.orchestra.run.smithi037.stdout:(115/152): python3-cachetools-4.2.4-1.el9.noarc 641 kB/s | 32 kB 00:00 2024-04-25T10:27:00.607 INFO:teuthology.orchestra.run.smithi037.stdout:(116/152): python3-certifi-2021.10.8-2.el9.noar 300 kB/s | 15 kB 00:00 2024-04-25T10:27:00.633 INFO:teuthology.orchestra.run.smithi037.stdout:(117/152): python3-cheroot-8.6.0-4.el9.noarch.r 3.3 MB/s | 172 kB 00:00 2024-04-25T10:27:00.711 INFO:teuthology.orchestra.run.smithi203.stdout:(74/152): python3-devel-3.9.18-3.el9.x86_64.rpm 1.4 MB/s | 244 kB 00:00 2024-04-25T10:27:00.717 INFO:teuthology.orchestra.run.smithi037.stdout:(118/152): python3-cherrypy-18.6.1-2.el9.noarch 3.2 MB/s | 358 kB 00:00 2024-04-25T10:27:00.742 INFO:teuthology.orchestra.run.smithi037.stdout:(119/152): python3-cmd2-2.4.2-2.el9.noarch.rpm 2.7 MB/s | 303 kB 00:00 2024-04-25T10:27:00.767 INFO:teuthology.orchestra.run.smithi037.stdout:(120/152): python3-colorama-0.4.4-12.el9.noarch 712 kB/s | 35 kB 00:00 2024-04-25T10:27:00.784 INFO:teuthology.orchestra.run.smithi037.stdout:(121/152): python3-google-auth-2.29.0-1.el9.noa 4.8 MB/s | 206 kB 00:00 2024-04-25T10:27:00.836 INFO:teuthology.orchestra.run.smithi203.stdout:(75/152): python3-babel-2.9.1-2.el9.noarch.rpm 11 MB/s | 6.0 MB 00:00 2024-04-25T10:27:00.842 INFO:teuthology.orchestra.run.smithi037.stdout:(122/152): python3-grpcio-1.46.7-10.el9.x86_64. 27 MB/s | 2.0 MB 00:00 2024-04-25T10:27:00.862 INFO:teuthology.orchestra.run.smithi203.stdout:(76/152): python3-jinja2-2.11.3-5.el9.noarch.rp 1.6 MB/s | 248 kB 00:00 2024-04-25T10:27:00.868 INFO:teuthology.orchestra.run.smithi037.stdout:(123/152): python3-grpcio-tools-1.46.7-10.el9.x 1.7 MB/s | 144 kB 00:00 2024-04-25T10:27:00.884 INFO:teuthology.orchestra.run.smithi037.stdout:(124/152): python3-jaraco-8.2.1-3.el9.noarch.rp 258 kB/s | 11 kB 00:00 2024-04-25T10:27:00.903 INFO:teuthology.orchestra.run.smithi203.stdout:(77/152): python3-jsonpatch-1.21-16.el9.noarch. 382 kB/s | 26 kB 00:00 2024-04-25T10:27:00.910 INFO:teuthology.orchestra.run.smithi037.stdout:(125/152): python3-jaraco-classes-3.2.1-5.el9.n 422 kB/s | 18 kB 00:00 2024-04-25T10:27:00.920 INFO:teuthology.orchestra.run.smithi203.stdout:(78/152): python3-jsonpointer-2.0-4.el9.noarch. 327 kB/s | 19 kB 00:00 2024-04-25T10:27:00.927 INFO:teuthology.orchestra.run.smithi037.stdout:(126/152): python3-jaraco-collections-3.0.0-8.e 552 kB/s | 23 kB 00:00 2024-04-25T10:27:00.944 INFO:teuthology.orchestra.run.smithi037.stdout:(127/152): python3-jaraco-functools-3.5.0-2.el9 581 kB/s | 19 kB 00:00 2024-04-25T10:27:00.961 INFO:teuthology.orchestra.run.smithi037.stdout:(128/152): python3-jaraco-text-3.2.0-6.el9.noar 583 kB/s | 20 kB 00:00 2024-04-25T10:27:00.962 INFO:teuthology.orchestra.run.smithi203.stdout:(79/152): python3-mako-1.1.4-6.el9.noarch.rpm 2.9 MB/s | 172 kB 00:00 2024-04-25T10:27:00.979 INFO:teuthology.orchestra.run.smithi203.stdout:(80/152): python3-markupsafe-1.1.1-12.el9.x86_6 592 kB/s | 35 kB 00:00 2024-04-25T10:27:01.003 INFO:teuthology.orchestra.run.smithi037.stdout:(129/152): python3-kubernetes-26.1.0-2.el9.noar 17 MB/s | 1.0 MB 00:00 2024-04-25T10:27:01.020 INFO:teuthology.orchestra.run.smithi037.stdout:(130/152): python3-logutils-0.3.5-21.el9.noarch 790 kB/s | 46 kB 00:00 2024-04-25T10:27:01.054 INFO:teuthology.orchestra.run.smithi037.stdout:(131/152): python3-more-itertools-8.12.0-2.el9. 1.5 MB/s | 79 kB 00:00 2024-04-25T10:27:01.079 INFO:teuthology.orchestra.run.smithi037.stdout:(132/152): python3-natsort-7.1.1-5.el9.noarch.r 982 kB/s | 58 kB 00:00 2024-04-25T10:27:01.080 INFO:teuthology.orchestra.run.smithi203.stdout:(81/152): python3-numpy-f2py-1.20.1-5.el9.x86_6 2.4 MB/s | 250 kB 00:00 2024-04-25T10:27:01.104 INFO:teuthology.orchestra.run.smithi037.stdout:(133/152): python3-pecan-1.4.2-3.el9.noarch.rpm 5.3 MB/s | 272 kB 00:00 2024-04-25T10:27:01.130 INFO:teuthology.orchestra.run.smithi037.stdout:(134/152): python3-portend-3.1.0-2.el9.noarch.r 332 kB/s | 16 kB 00:00 2024-04-25T10:27:01.155 INFO:teuthology.orchestra.run.smithi037.stdout:(135/152): python3-pyOpenSSL-21.0.0-1.el9.noarc 1.8 MB/s | 90 kB 00:00 2024-04-25T10:27:01.172 INFO:teuthology.orchestra.run.smithi203.stdout:(82/152): python3-oauthlib-3.1.1-5.el9.noarch.r 2.4 MB/s | 222 kB 00:00 2024-04-25T10:27:01.180 INFO:teuthology.orchestra.run.smithi037.stdout:(136/152): python3-pyperclip-1.8.2-2.el9.noarch 565 kB/s | 28 kB 00:00 2024-04-25T10:27:01.205 INFO:teuthology.orchestra.run.smithi037.stdout:(137/152): python3-repoze-lru-0.7-16.el9.noarch 613 kB/s | 31 kB 00:00 2024-04-25T10:27:01.239 INFO:teuthology.orchestra.run.smithi203.stdout:(83/152): python3-packaging-20.9-5.el9.noarch.r 1.1 MB/s | 77 kB 00:00 2024-04-25T10:27:01.281 INFO:teuthology.orchestra.run.smithi037.stdout:(138/152): python3-routes-2.5.1-5.el9.noarch.rp 1.8 MB/s | 188 kB 00:00 2024-04-25T10:27:01.306 INFO:teuthology.orchestra.run.smithi037.stdout:(139/152): python3-rsa-4.9-2.el9.noarch.rpm 589 kB/s | 59 kB 00:00 2024-04-25T10:27:01.314 INFO:teuthology.orchestra.run.smithi203.stdout:(84/152): python3-numpy-1.20.1-5.el9.x86_64.rpm 15 MB/s | 5.1 MB 00:00 2024-04-25T10:27:01.331 INFO:teuthology.orchestra.run.smithi037.stdout:(140/152): python3-tempora-5.0.0-2.el9.noarch.r 715 kB/s | 36 kB 00:00 2024-04-25T10:27:01.339 INFO:teuthology.orchestra.run.smithi203.stdout:(85/152): python3-prettytable-0.7.2-27.el9.noar 422 kB/s | 42 kB 00:00 2024-04-25T10:27:01.356 INFO:teuthology.orchestra.run.smithi037.stdout:(141/152): python3-typing-extensions-4.4.0-2.el 1.0 MB/s | 51 kB 00:00 2024-04-25T10:27:01.373 INFO:teuthology.orchestra.run.smithi203.stdout:(86/152): python3-protobuf-3.14.0-13.el9.noarch 4.5 MB/s | 269 kB 00:00 2024-04-25T10:27:01.382 INFO:teuthology.orchestra.run.smithi037.stdout:(142/152): python3-webob-1.8.7-6.el9.noarch.rpm 4.5 MB/s | 230 kB 00:00 2024-04-25T10:27:01.398 INFO:teuthology.orchestra.run.smithi203.stdout:(87/152): python3-pyasn1-0.4.8-6.el9.noarch.rpm 2.7 MB/s | 159 kB 00:00 2024-04-25T10:27:01.407 INFO:teuthology.orchestra.run.smithi037.stdout:(143/152): python3-websocket-client-1.2.3-2.el9 1.8 MB/s | 90 kB 00:00 2024-04-25T10:27:01.432 INFO:teuthology.orchestra.run.smithi203.stdout:(88/152): python3-pyasn1-modules-0.4.8-6.el9.no 4.7 MB/s | 279 kB 00:00 2024-04-25T10:27:01.432 INFO:teuthology.orchestra.run.smithi037.stdout:(144/152): python3-werkzeug-2.0.3-3.el9.1.noarc 8.4 MB/s | 427 kB 00:00 2024-04-25T10:27:01.457 INFO:teuthology.orchestra.run.smithi203.stdout:(89/152): python3-pytz-2021.1-5.el9.noarch.rpm 868 kB/s | 51 kB 00:00 2024-04-25T10:27:01.458 INFO:teuthology.orchestra.run.smithi037.stdout:(145/152): python3-zc-lockfile-2.0-10.el9.noarc 406 kB/s | 20 kB 00:00 2024-04-25T10:27:01.483 INFO:teuthology.orchestra.run.smithi037.stdout:(146/152): re2-20211101-3.el9.x86_64.rpm 3.7 MB/s | 192 kB 00:00 2024-04-25T10:27:01.500 INFO:teuthology.orchestra.run.smithi203.stdout:(90/152): python3-requests-oauthlib-1.3.0-12.el 793 kB/s | 54 kB 00:00 2024-04-25T10:27:01.525 INFO:teuthology.orchestra.run.smithi037.stdout:(147/152): thrift-0.15.0-2.el9.x86_64.rpm 24 MB/s | 1.6 MB 00:00 2024-04-25T10:27:01.558 INFO:teuthology.orchestra.run.smithi203.stdout:(91/152): python3-toml-0.10.2-6.el9.noarch.rpm 722 kB/s | 42 kB 00:00 2024-04-25T10:27:01.558 INFO:teuthology.orchestra.run.smithi037.stdout:(148/152): protobuf-compiler-3.14.0-13.el9.x86_ 459 kB/s | 863 kB 00:01 2024-04-25T10:27:01.617 INFO:teuthology.orchestra.run.smithi203.stdout:(92/152): python3-wcwidth-0.2.5-8.el9.noarch.rp 758 kB/s | 44 kB 00:00 2024-04-25T10:27:01.675 INFO:teuthology.orchestra.run.smithi203.stdout:(93/152): qatlib-24.02.0-1.el9.x86_64.rpm 3.7 MB/s | 221 kB 00:00 2024-04-25T10:27:01.726 INFO:teuthology.orchestra.run.smithi203.stdout:(94/152): qatlib-service-24.02.0-1.el9.x86_64.r 718 kB/s | 36 kB 00:00 2024-04-25T10:27:01.785 INFO:teuthology.orchestra.run.smithi203.stdout:(95/152): qatzip-libs-1.2.0-1.el9.x86_64.rpm 813 kB/s | 47 kB 00:00 2024-04-25T10:27:01.826 INFO:teuthology.orchestra.run.smithi037.stdout:(149/152): openssl-3.2.1-1.el9.x86_64.rpm 5.1 MB/s | 1.3 MB 00:00 2024-04-25T10:27:01.851 INFO:teuthology.orchestra.run.smithi203.stdout:(96/152): socat-1.7.4.1-5.el9.x86_64.rpm 4.5 MB/s | 305 kB 00:00 2024-04-25T10:27:01.919 INFO:teuthology.orchestra.run.smithi203.stdout:(97/152): uuid-1.6.2-55.el9.x86_64.rpm 865 kB/s | 58 kB 00:00 2024-04-25T10:27:01.978 INFO:teuthology.orchestra.run.smithi203.stdout:(98/152): xfsprogs-devel-6.3.0-1.el9.x86_64.rpm 1.6 MB/s | 94 kB 00:00 2024-04-25T10:27:01.992 INFO:teuthology.orchestra.run.smithi037.stdout:(150/152): openssl-libs-3.2.1-1.el9.x86_64.rpm 14 MB/s | 2.4 MB 00:00 2024-04-25T10:27:02.036 INFO:teuthology.orchestra.run.smithi203.stdout:(99/152): xmlstarlet-1.6.1-20.el9.x86_64.rpm 1.1 MB/s | 64 kB 00:00 2024-04-25T10:27:02.076 INFO:teuthology.orchestra.run.smithi037.stdout:(151/152): librbd1-19.0.0-3291.g994e2f20.el9.x8 5.6 MB/s | 3.1 MB 00:00 2024-04-25T10:27:02.094 INFO:teuthology.orchestra.run.smithi203.stdout:(100/152): zlib-devel-1.2.11-41.el9.x86_64.rpm 769 kB/s | 45 kB 00:00 2024-04-25T10:27:02.145 INFO:teuthology.orchestra.run.smithi203.stdout:(101/152): lua-devel-5.4.4-4.el9.x86_64.rpm 447 kB/s | 22 kB 00:00 2024-04-25T10:27:02.245 INFO:teuthology.orchestra.run.smithi203.stdout:(102/152): protobuf-compiler-3.14.0-13.el9.x86_ 8.5 MB/s | 863 kB 00:00 2024-04-25T10:27:02.376 INFO:teuthology.orchestra.run.smithi037.stdout:(152/152): librados2-19.0.0-3291.g994e2f20.el9. 3.7 MB/s | 3.3 MB 00:00 2024-04-25T10:27:02.381 INFO:teuthology.orchestra.run.smithi037.stdout:-------------------------------------------------------------------------------- 2024-04-25T10:27:02.381 INFO:teuthology.orchestra.run.smithi037.stdout:Total 14 MB/s | 214 MB 00:15 2024-04-25T10:27:02.704 INFO:teuthology.orchestra.run.smithi203.stdout:(103/152): abseil-cpp-20211102.0-3.el9.x86_64.r 1.2 MB/s | 548 kB 00:00 2024-04-25T10:27:02.813 INFO:teuthology.orchestra.run.smithi203.stdout:(104/152): gperftools-libs-2.9.1-2.el9.x86_64.r 2.8 MB/s | 309 kB 00:00 2024-04-25T10:27:02.879 INFO:teuthology.orchestra.run.smithi203.stdout:(105/152): grpc-data-1.46.7-10.el9.noarch.rpm 293 kB/s | 19 kB 00:00 2024-04-25T10:27:03.155 INFO:teuthology.orchestra.run.smithi203.stdout:(106/152): libarrow-9.0.0-10.el9.x86_64.rpm 16 MB/s | 4.4 MB 00:00 2024-04-25T10:27:03.222 INFO:teuthology.orchestra.run.smithi203.stdout:(107/152): libarrow-doc-9.0.0-10.el9.noarch.rpm 390 kB/s | 26 kB 00:00 2024-04-25T10:27:03.414 INFO:teuthology.orchestra.run.smithi203.stdout:(108/152): python3-scipy-1.6.2-8.el9.x86_64.rpm 8.3 MB/s | 16 MB 00:01 2024-04-25T10:27:03.431 INFO:teuthology.orchestra.run.smithi203.stdout:(109/152): liboath-2.6.7-2.el9.x86_64.rpm 233 kB/s | 49 kB 00:00 2024-04-25T10:27:03.523 INFO:teuthology.orchestra.run.smithi203.stdout:(110/152): luarocks-3.9.2-1.el9.noarch.rpm 1.6 MB/s | 151 kB 00:00 2024-04-25T10:27:03.537 INFO:teuthology.orchestra.run.smithi037.stdout:Running transaction check 2024-04-25T10:27:03.615 INFO:teuthology.orchestra.run.smithi203.stdout:(111/152): parquet-libs-9.0.0-10.el9.x86_64.rpm 8.9 MB/s | 838 kB 00:00 2024-04-25T10:27:03.697 INFO:teuthology.orchestra.run.smithi037.stdout:Transaction check succeeded. 2024-04-25T10:27:03.698 INFO:teuthology.orchestra.run.smithi037.stdout:Running transaction test 2024-04-25T10:27:03.706 INFO:teuthology.orchestra.run.smithi203.stdout:(112/152): python3-asyncssh-2.13.2-5.el9.noarch 5.9 MB/s | 548 kB 00:00 2024-04-25T10:27:03.782 INFO:teuthology.orchestra.run.smithi203.stdout:(113/152): libunwind-1.6.2-1.el9.x86_64.rpm 183 kB/s | 67 kB 00:00 2024-04-25T10:27:03.807 INFO:teuthology.orchestra.run.smithi203.stdout:(114/152): python3-bcrypt-3.2.2-1.el9.x86_64.rp 433 kB/s | 43 kB 00:00 2024-04-25T10:27:03.891 INFO:teuthology.orchestra.run.smithi203.stdout:(115/152): python3-cachetools-4.2.4-1.el9.noarc 297 kB/s | 32 kB 00:00 2024-04-25T10:27:03.916 INFO:teuthology.orchestra.run.smithi203.stdout:(116/152): python3-certifi-2021.10.8-2.el9.noar 138 kB/s | 15 kB 00:00 2024-04-25T10:27:03.983 INFO:teuthology.orchestra.run.smithi203.stdout:(117/152): python3-cheroot-8.6.0-4.el9.noarch.r 1.8 MB/s | 172 kB 00:00 2024-04-25T10:27:04.075 INFO:teuthology.orchestra.run.smithi203.stdout:(118/152): python3-cmd2-2.4.2-2.el9.noarch.rpm 3.2 MB/s | 303 kB 00:00 2024-04-25T10:27:04.167 INFO:teuthology.orchestra.run.smithi203.stdout:(119/152): python3-colorama-0.4.4-12.el9.noarch 385 kB/s | 35 kB 00:00 2024-04-25T10:27:04.200 INFO:teuthology.orchestra.run.smithi203.stdout:(120/152): python3-cherrypy-18.6.1-2.el9.noarch 1.2 MB/s | 358 kB 00:00 2024-04-25T10:27:04.268 INFO:teuthology.orchestra.run.smithi203.stdout:(121/152): python3-google-auth-2.29.0-1.el9.noa 2.0 MB/s | 206 kB 00:00 2024-04-25T10:27:04.359 INFO:teuthology.orchestra.run.smithi203.stdout:(122/152): python3-grpcio-tools-1.46.7-10.el9.x 1.5 MB/s | 144 kB 00:00 2024-04-25T10:27:04.452 INFO:teuthology.orchestra.run.smithi203.stdout:(123/152): python3-jaraco-8.2.1-3.el9.noarch.rp 116 kB/s | 11 kB 00:00 2024-04-25T10:27:04.519 INFO:teuthology.orchestra.run.smithi203.stdout:(124/152): python3-jaraco-classes-3.2.1-5.el9.n 265 kB/s | 18 kB 00:00 2024-04-25T10:27:04.560 INFO:teuthology.orchestra.run.smithi203.stdout:(125/152): python3-grpcio-1.46.7-10.el9.x86_64. 5.7 MB/s | 2.0 MB 00:00 2024-04-25T10:27:04.586 INFO:teuthology.orchestra.run.smithi203.stdout:(126/152): python3-jaraco-collections-3.0.0-8.e 346 kB/s | 23 kB 00:00 2024-04-25T10:27:04.628 INFO:teuthology.orchestra.run.smithi203.stdout:(127/152): python3-jaraco-functools-3.5.0-2.el9 290 kB/s | 19 kB 00:00 2024-04-25T10:27:04.653 INFO:teuthology.orchestra.run.smithi203.stdout:(128/152): python3-jaraco-text-3.2.0-6.el9.noar 293 kB/s | 20 kB 00:00 2024-04-25T10:27:04.720 INFO:teuthology.orchestra.run.smithi203.stdout:(129/152): python3-logutils-0.3.5-21.el9.noarch 695 kB/s | 46 kB 00:00 2024-04-25T10:27:04.754 INFO:teuthology.orchestra.run.smithi203.stdout:(130/152): python3-kubernetes-26.1.0-2.el9.noar 8.1 MB/s | 1.0 MB 00:00 2024-04-25T10:27:04.787 INFO:teuthology.orchestra.run.smithi203.stdout:(131/152): python3-more-itertools-8.12.0-2.el9. 1.1 MB/s | 79 kB 00:00 2024-04-25T10:27:04.821 INFO:teuthology.orchestra.run.smithi203.stdout:(132/152): python3-natsort-7.1.1-5.el9.noarch.r 861 kB/s | 58 kB 00:00 2024-04-25T10:27:04.863 INFO:teuthology.orchestra.run.smithi203.stdout:(133/152): python3-pecan-1.4.2-3.el9.noarch.rpm 3.5 MB/s | 272 kB 00:00 2024-04-25T10:27:04.888 INFO:teuthology.orchestra.run.smithi203.stdout:(134/152): python3-portend-3.1.0-2.el9.noarch.r 246 kB/s | 16 kB 00:00 2024-04-25T10:27:04.930 INFO:teuthology.orchestra.run.smithi203.stdout:(135/152): python3-pyOpenSSL-21.0.0-1.el9.noarc 1.3 MB/s | 90 kB 00:00 2024-04-25T10:27:04.956 INFO:teuthology.orchestra.run.smithi203.stdout:(136/152): python3-pyperclip-1.8.2-2.el9.noarch 421 kB/s | 28 kB 00:00 2024-04-25T10:27:04.998 INFO:teuthology.orchestra.run.smithi203.stdout:(137/152): python3-repoze-lru-0.7-16.el9.noarch 460 kB/s | 31 kB 00:00 2024-04-25T10:27:05.031 INFO:teuthology.orchestra.run.smithi203.stdout:(138/152): python3-routes-2.5.1-5.el9.noarch.rp 2.4 MB/s | 188 kB 00:00 2024-04-25T10:27:05.065 INFO:teuthology.orchestra.run.smithi203.stdout:(139/152): python3-rsa-4.9-2.el9.noarch.rpm 885 kB/s | 59 kB 00:00 2024-04-25T10:27:05.124 INFO:teuthology.orchestra.run.smithi203.stdout:(140/152): python3-tempora-5.0.0-2.el9.noarch.r 389 kB/s | 36 kB 00:00 2024-04-25T10:27:05.149 INFO:teuthology.orchestra.run.smithi203.stdout:(141/152): python3-typing-extensions-4.4.0-2.el 611 kB/s | 51 kB 00:00 2024-04-25T10:27:05.199 INFO:teuthology.orchestra.run.smithi203.stdout:(142/152): python3-webob-1.8.7-6.el9.noarch.rpm 3.0 MB/s | 230 kB 00:00 2024-04-25T10:27:05.224 INFO:teuthology.orchestra.run.smithi203.stdout:(143/152): python3-websocket-client-1.2.3-2.el9 1.2 MB/s | 90 kB 00:00 2024-04-25T10:27:05.283 INFO:teuthology.orchestra.run.smithi203.stdout:(144/152): python3-werkzeug-2.0.3-3.el9.1.noarc 5.0 MB/s | 427 kB 00:00 2024-04-25T10:27:05.308 INFO:teuthology.orchestra.run.smithi203.stdout:(145/152): python3-zc-lockfile-2.0-10.el9.noarc 239 kB/s | 20 kB 00:00 2024-04-25T10:27:05.350 INFO:teuthology.orchestra.run.smithi203.stdout:(146/152): re2-20211101-3.el9.x86_64.rpm 2.8 MB/s | 192 kB 00:00 2024-04-25T10:27:05.417 INFO:teuthology.orchestra.run.smithi203.stdout:(147/152): thrift-0.15.0-2.el9.x86_64.rpm 15 MB/s | 1.6 MB 00:00 2024-04-25T10:27:05.529 INFO:teuthology.orchestra.run.smithi037.stdout:Transaction test succeeded. 2024-04-25T10:27:05.530 INFO:teuthology.orchestra.run.smithi037.stdout:Running transaction 2024-04-25T10:27:05.742 INFO:teuthology.orchestra.run.smithi203.stdout:(148/152): librbd1-19.0.0-3291.g994e2f20.el9.x8 9.5 MB/s | 3.1 MB 00:00 2024-04-25T10:27:05.959 INFO:teuthology.orchestra.run.smithi203.stdout:(149/152): librados2-19.0.0-3291.g994e2f20.el9. 5.5 MB/s | 3.3 MB 00:00 2024-04-25T10:27:06.068 INFO:teuthology.orchestra.run.smithi203.stdout:(150/152): openssl-3.2.1-1.el9.x86_64.rpm 4.1 MB/s | 1.3 MB 00:00 2024-04-25T10:27:06.323 INFO:teuthology.orchestra.run.smithi203.stdout:(151/152): openssl-libs-3.2.1-1.el9.x86_64.rpm 6.6 MB/s | 2.4 MB 00:00 2024-04-25T10:27:07.660 INFO:teuthology.orchestra.run.smithi203.stdout:(152/152): ceph-test-19.0.0-3291.g994e2f20.el9. 3.3 MB/s | 43 MB 00:13 2024-04-25T10:27:07.665 INFO:teuthology.orchestra.run.smithi203.stdout:-------------------------------------------------------------------------------- 2024-04-25T10:27:07.666 INFO:teuthology.orchestra.run.smithi203.stdout:Total 12 MB/s | 214 MB 00:18 2024-04-25T10:27:08.784 INFO:teuthology.orchestra.run.smithi203.stdout:Running transaction check 2024-04-25T10:27:08.928 INFO:teuthology.orchestra.run.smithi203.stdout:Transaction check succeeded. 2024-04-25T10:27:08.928 INFO:teuthology.orchestra.run.smithi203.stdout:Running transaction test 2024-04-25T10:27:10.653 INFO:teuthology.orchestra.run.smithi203.stdout:Transaction test succeeded. 2024-04-25T10:27:10.654 INFO:teuthology.orchestra.run.smithi203.stdout:Running transaction 2024-04-25T10:27:11.631 INFO:teuthology.orchestra.run.smithi037.stdout: Preparing : 1/1 2024-04-25T10:27:11.722 INFO:teuthology.orchestra.run.smithi037.stdout: Upgrading : openssl-libs-1:3.2.1-1.el9.x86_64 1/156 2024-04-25T10:27:11.769 INFO:teuthology.orchestra.run.smithi037.stdout: Installing : librabbitmq-0.11.0-7.el9.x86_64 2/156 2024-04-25T10:27:11.832 INFO:teuthology.orchestra.run.smithi037.stdout: Installing : librdkafka-1.6.1-102.el9.x86_64 3/156 2024-04-25T10:27:11.873 INFO:teuthology.orchestra.run.smithi037.stdout: Installing : thrift-0.15.0-2.el9.x86_64 4/156 2024-04-25T10:27:11.898 INFO:teuthology.orchestra.run.smithi037.stdout: Installing : python3-more-itertools-8.12.0-2.el9.noarch 5/156 2024-04-25T10:27:11.942 INFO:teuthology.orchestra.run.smithi037.stdout: Installing : python3-jaraco-8.2.1-3.el9.noarch 6/156 2024-04-25T10:27:12.294 INFO:teuthology.orchestra.run.smithi037.stdout: Installing : lttng-ust-2.12.0-6.el9.x86_64 7/156 2024-04-25T10:27:12.416 INFO:teuthology.orchestra.run.smithi037.stdout: Upgrading : librados2-2:19.0.0-3291.g994e2f20.el9.x86_64 8/156 2024-04-25T10:27:12.545 INFO:teuthology.orchestra.run.smithi037.stdout: Running scriptlet: librados2-2:19.0.0-3291.g994e2f20.el9.x86_64 8/156 2024-04-25T10:27:12.575 INFO:teuthology.orchestra.run.smithi037.stdout: Installing : libcephfs2-2:19.0.0-3291.g994e2f20.el9.x86_64 9/156 2024-04-25T10:27:12.668 INFO:teuthology.orchestra.run.smithi037.stdout: Running scriptlet: libcephfs2-2:19.0.0-3291.g994e2f20.el9.x86_64 9/156 2024-04-25T10:27:12.696 INFO:teuthology.orchestra.run.smithi037.stdout: Installing : python3-rados-2:19.0.0-3291.g994e2f20.el9.x86_64 10/156 2024-04-25T10:27:12.855 INFO:teuthology.orchestra.run.smithi037.stdout: Installing : postgresql-private-libs-13.14-1.el9.x86_64 11/156 2024-04-25T10:27:12.902 INFO:teuthology.orchestra.run.smithi037.stdout: Installing : postgresql-13.14-1.el9.x86_64 12/156 2024-04-25T10:27:12.930 INFO:teuthology.orchestra.run.smithi037.stdout: Installing : libcephsqlite-2:19.0.0-3291.g994e2f20.el9.x86_64 13/156 2024-04-25T10:27:13.028 INFO:teuthology.orchestra.run.smithi037.stdout: Running scriptlet: libcephsqlite-2:19.0.0-3291.g994e2f20.el9.x86_64 13/156 2024-04-25T10:27:13.055 INFO:teuthology.orchestra.run.smithi037.stdout: Installing : libradosstriper1-2:19.0.0-3291.g994e2f20.el9.x86 14/156 2024-04-25T10:27:13.118 INFO:teuthology.orchestra.run.smithi037.stdout: Running scriptlet: libradosstriper1-2:19.0.0-3291.g994e2f20.el9.x86 14/156 2024-04-25T10:27:13.140 INFO:teuthology.orchestra.run.smithi037.stdout: Installing : qatlib-service-24.02.0-1.el9.x86_64 15/156 2024-04-25T10:27:13.164 INFO:teuthology.orchestra.run.smithi037.stdout: Running scriptlet: qatlib-service-24.02.0-1.el9.x86_64 15/156 2024-04-25T10:27:13.358 INFO:teuthology.orchestra.run.smithi037.stdout: Running scriptlet: qatlib-24.02.0-1.el9.x86_64 16/156 2024-04-25T10:27:13.390 INFO:teuthology.orchestra.run.smithi037.stdout: Installing : qatlib-24.02.0-1.el9.x86_64 16/156 2024-04-25T10:27:13.503 INFO:teuthology.orchestra.run.smithi037.stdout: Installing : re2-1:20211101-3.el9.x86_64 17/156 2024-04-25T10:27:13.597 INFO:teuthology.orchestra.run.smithi037.stdout: Installing : libarrow-9.0.0-10.el9.x86_64 18/156 2024-04-25T10:27:13.634 INFO:teuthology.orchestra.run.smithi037.stdout: Installing : python3-werkzeug-2.0.3-3.el9.1.noarch 19/156 2024-04-25T10:27:13.675 INFO:teuthology.orchestra.run.smithi037.stdout: Installing : liboath-2.6.7-2.el9.x86_64 20/156 2024-04-25T10:27:13.750 INFO:teuthology.orchestra.run.smithi037.stdout: Installing : python3-pytz-2021.1-5.el9.noarch 21/156 2024-04-25T10:27:13.799 INFO:teuthology.orchestra.run.smithi037.stdout: Installing : python3-pyasn1-0.4.8-6.el9.noarch 22/156 2024-04-25T10:27:13.843 INFO:teuthology.orchestra.run.smithi037.stdout: Installing : python3-markupsafe-1.1.1-12.el9.x86_64 23/156 2024-04-25T10:27:13.875 INFO:teuthology.orchestra.run.smithi037.stdout: Installing : protobuf-3.14.0-13.el9.x86_64 24/156 2024-04-25T10:27:13.902 INFO:teuthology.orchestra.run.smithi037.stdout: Installing : lua-5.4.4-4.el9.x86_64 25/156 2024-04-25T10:27:13.974 INFO:teuthology.orchestra.run.smithi037.stdout: Installing : flexiblas-3.0.4-8.el9.x86_64 26/156 2024-04-25T10:27:14.035 INFO:teuthology.orchestra.run.smithi037.stdout: Installing : python3-urllib3-1.26.5-5.el9.noarch 27/156 2024-04-25T10:27:14.064 INFO:teuthology.orchestra.run.smithi037.stdout: Installing : python3-requests-2.25.1-8.el9.noarch 28/156 2024-04-25T10:27:14.108 INFO:teuthology.orchestra.run.smithi037.stdout: Installing : libquadmath-11.4.1-3.el9.x86_64 29/156 2024-04-25T10:27:14.185 INFO:teuthology.orchestra.run.smithi037.stdout: Installing : libgfortran-11.4.1-3.el9.x86_64 30/156 2024-04-25T10:27:14.218 INFO:teuthology.orchestra.run.smithi037.stdout: Installing : python3-ceph-common-2:19.0.0-3291.g994e2f20.el9. 31/156 2024-04-25T10:27:14.262 INFO:teuthology.orchestra.run.smithi037.stdout: Installing : python3-ceph-argparse-2:19.0.0-3291.g994e2f20.el 32/156 2024-04-25T10:27:14.290 INFO:teuthology.orchestra.run.smithi037.stdout: Installing : python3-cephfs-2:19.0.0-3291.g994e2f20.el9.x86_6 33/156 2024-04-25T10:27:14.362 INFO:teuthology.orchestra.run.smithi037.stdout: Installing : lua-devel-5.4.4-4.el9.x86_64 34/156 2024-04-25T10:27:14.414 INFO:teuthology.orchestra.run.smithi037.stdout: Installing : luarocks-3.9.2-1.el9.noarch 35/156 2024-04-25T10:27:14.500 INFO:teuthology.orchestra.run.smithi037.stdout: Installing : protobuf-compiler-3.14.0-13.el9.x86_64 36/156 2024-04-25T10:27:14.646 INFO:teuthology.orchestra.run.smithi037.stdout: Installing : python3-mako-1.1.4-6.el9.noarch 37/156 2024-04-25T10:27:14.708 INFO:teuthology.orchestra.run.smithi037.stdout: Installing : python3-pyasn1-modules-0.4.8-6.el9.noarch 38/156 2024-04-25T10:27:15.263 INFO:teuthology.orchestra.run.smithi037.stdout: Installing : python3-rsa-4.9-2.el9.noarch 39/156 2024-04-25T10:27:15.347 INFO:teuthology.orchestra.run.smithi037.stdout: Installing : python3-babel-2.9.1-2.el9.noarch 40/156 2024-04-25T10:27:15.382 INFO:teuthology.orchestra.run.smithi037.stdout: Installing : python3-jinja2-2.11.3-5.el9.noarch 41/156 2024-04-25T10:27:15.425 INFO:teuthology.orchestra.run.smithi037.stdout: Installing : qatzip-libs-1.2.0-1.el9.x86_64 42/156 2024-04-25T10:27:15.471 INFO:teuthology.orchestra.run.smithi037.stdout: Installing : librados-devel-2:19.0.0-3291.g994e2f20.el9.x86_6 43/156 2024-04-25T10:27:15.514 INFO:teuthology.orchestra.run.smithi037.stdout: Installing : python3-jaraco-classes-3.2.1-5.el9.noarch 44/156 2024-04-25T10:27:15.681 INFO:teuthology.orchestra.run.smithi037.stdout: Installing : socat-1.7.4.1-5.el9.x86_64 45/156 2024-04-25T10:27:15.732 INFO:teuthology.orchestra.run.smithi037.stdout: Upgrading : openssl-1:3.2.1-1.el9.x86_64 46/156 2024-04-25T10:27:15.788 INFO:teuthology.orchestra.run.smithi037.stdout: Installing : python3-zc-lockfile-2.0-10.el9.noarch 47/156 2024-04-25T10:27:15.862 INFO:teuthology.orchestra.run.smithi037.stdout: Installing : python3-websocket-client-1.2.3-2.el9.noarch 48/156 2024-04-25T10:27:15.899 INFO:teuthology.orchestra.run.smithi037.stdout: Installing : python3-webob-1.8.7-6.el9.noarch 49/156 2024-04-25T10:27:15.936 INFO:teuthology.orchestra.run.smithi037.stdout: Installing : python3-typing-extensions-4.4.0-2.el9.noarch 50/156 2024-04-25T10:27:15.984 INFO:teuthology.orchestra.run.smithi037.stdout: Installing : python3-repoze-lru-0.7-16.el9.noarch 51/156 2024-04-25T10:27:16.018 INFO:teuthology.orchestra.run.smithi037.stdout: Installing : python3-routes-2.5.1-5.el9.noarch 52/156 2024-04-25T10:27:16.063 INFO:teuthology.orchestra.run.smithi037.stdout: Installing : python3-pyperclip-1.8.2-2.el9.noarch 53/156 2024-04-25T10:27:16.105 INFO:teuthology.orchestra.run.smithi037.stdout: Installing : python3-natsort-7.1.1-5.el9.noarch 54/156 2024-04-25T10:27:16.247 INFO:teuthology.orchestra.run.smithi037.stdout: Installing : python3-logutils-0.3.5-21.el9.noarch 55/156 2024-04-25T10:27:16.293 INFO:teuthology.orchestra.run.smithi037.stdout: Installing : python3-pecan-1.4.2-3.el9.noarch 56/156 2024-04-25T10:27:16.323 INFO:teuthology.orchestra.run.smithi037.stdout: Installing : python3-colorama-0.4.4-12.el9.noarch 57/156 2024-04-25T10:27:16.365 INFO:teuthology.orchestra.run.smithi037.stdout: Installing : python3-certifi-2021.10.8-2.el9.noarch 58/156 2024-04-25T10:27:16.462 INFO:teuthology.orchestra.run.smithi037.stdout: Installing : python3-cachetools-4.2.4-1.el9.noarch 59/156 2024-04-25T10:27:16.498 INFO:teuthology.orchestra.run.smithi037.stdout: Installing : python3-google-auth-1:2.29.0-1.el9.noarch 60/156 2024-04-25T10:27:16.538 INFO:teuthology.orchestra.run.smithi037.stdout: Installing : libunwind-1.6.2-1.el9.x86_64 61/156 2024-04-25T10:27:16.563 INFO:teuthology.orchestra.run.smithi037.stdout: Installing : gperftools-libs-2.9.1-2.el9.x86_64 62/156 2024-04-25T10:27:16.588 INFO:teuthology.orchestra.run.smithi037.stdout: Installing : libarrow-doc-9.0.0-10.el9.noarch 63/156 2024-04-25T10:27:16.642 INFO:teuthology.orchestra.run.smithi203.stdout: Preparing : 1/1 2024-04-25T10:27:16.666 INFO:teuthology.orchestra.run.smithi037.stdout: Installing : grpc-data-1.46.7-10.el9.noarch 64/156 2024-04-25T10:27:16.781 INFO:teuthology.orchestra.run.smithi203.stdout: Upgrading : openssl-libs-1:3.2.1-1.el9.x86_64 1/156 2024-04-25T10:27:16.804 INFO:teuthology.orchestra.run.smithi037.stdout: Installing : abseil-cpp-20211102.0-3.el9.x86_64 65/156 2024-04-25T10:27:16.837 INFO:teuthology.orchestra.run.smithi037.stdout: Installing : python3-grpcio-1.46.7-10.el9.x86_64 66/156 2024-04-25T10:27:16.865 INFO:teuthology.orchestra.run.smithi037.stdout: Installing : zlib-devel-1.2.11-41.el9.x86_64 67/156 2024-04-25T10:27:16.899 INFO:teuthology.orchestra.run.smithi037.stdout: Installing : xmlstarlet-1.6.1-20.el9.x86_64 68/156 2024-04-25T10:27:16.939 INFO:teuthology.orchestra.run.smithi037.stdout: Installing : uuid-1.6.2-55.el9.x86_64 69/156 2024-04-25T10:27:16.975 INFO:teuthology.orchestra.run.smithi203.stdout: Installing : librabbitmq-0.11.0-7.el9.x86_64 2/156 2024-04-25T10:27:16.978 INFO:teuthology.orchestra.run.smithi037.stdout: Installing : python3-wcwidth-0.2.5-8.el9.noarch 70/156 2024-04-25T10:27:17.008 INFO:teuthology.orchestra.run.smithi037.stdout: Installing : python3-toml-0.10.2-6.el9.noarch 71/156 2024-04-25T10:27:17.042 INFO:teuthology.orchestra.run.smithi203.stdout: Installing : librdkafka-1.6.1-102.el9.x86_64 3/156 2024-04-25T10:27:17.082 INFO:teuthology.orchestra.run.smithi203.stdout: Installing : thrift-0.15.0-2.el9.x86_64 4/156 2024-04-25T10:27:17.107 INFO:teuthology.orchestra.run.smithi203.stdout: Installing : python3-more-itertools-8.12.0-2.el9.noarch 5/156 2024-04-25T10:27:17.126 INFO:teuthology.orchestra.run.smithi037.stdout: Installing : python3-jaraco-functools-3.5.0-2.el9.noarch 72/156 2024-04-25T10:27:17.151 INFO:teuthology.orchestra.run.smithi203.stdout: Installing : python3-jaraco-8.2.1-3.el9.noarch 6/156 2024-04-25T10:27:17.277 INFO:teuthology.orchestra.run.smithi037.stdout: Installing : python3-jaraco-text-3.2.0-6.el9.noarch 73/156 2024-04-25T10:27:17.319 INFO:teuthology.orchestra.run.smithi037.stdout: Installing : python3-jaraco-collections-3.0.0-8.el9.noarch 74/156 2024-04-25T10:27:17.361 INFO:teuthology.orchestra.run.smithi037.stdout: Installing : python3-tempora-5.0.0-2.el9.noarch 75/156 2024-04-25T10:27:17.454 INFO:teuthology.orchestra.run.smithi037.stdout: Installing : python3-portend-3.1.0-2.el9.noarch 76/156 2024-04-25T10:27:17.487 INFO:teuthology.orchestra.run.smithi203.stdout: Installing : lttng-ust-2.12.0-6.el9.x86_64 7/156 2024-04-25T10:27:17.501 INFO:teuthology.orchestra.run.smithi037.stdout: Installing : python3-protobuf-3.14.0-13.el9.noarch 77/156 2024-04-25T10:27:17.531 INFO:teuthology.orchestra.run.smithi037.stdout: Installing : python3-grpcio-tools-1.46.7-10.el9.x86_64 78/156 2024-04-25T10:27:17.579 INFO:teuthology.orchestra.run.smithi037.stdout: Installing : python3-prettytable-0.7.2-27.el9.noarch 79/156 2024-04-25T10:27:17.617 INFO:teuthology.orchestra.run.smithi203.stdout: Upgrading : librados2-2:19.0.0-3291.g994e2f20.el9.x86_64 8/156 2024-04-25T10:27:17.689 INFO:teuthology.orchestra.run.smithi037.stdout: Installing : python3-packaging-20.9-5.el9.noarch 80/156 2024-04-25T10:27:17.733 INFO:teuthology.orchestra.run.smithi203.stdout: Running scriptlet: librados2-2:19.0.0-3291.g994e2f20.el9.x86_64 8/156 2024-04-25T10:27:17.742 INFO:teuthology.orchestra.run.smithi037.stdout: Installing : python3-oauthlib-3.1.1-5.el9.noarch 81/156 2024-04-25T10:27:17.760 INFO:teuthology.orchestra.run.smithi203.stdout: Installing : libcephfs2-2:19.0.0-3291.g994e2f20.el9.x86_64 9/156 2024-04-25T10:27:17.835 INFO:teuthology.orchestra.run.smithi203.stdout: Running scriptlet: libcephfs2-2:19.0.0-3291.g994e2f20.el9.x86_64 9/156 2024-04-25T10:27:17.873 INFO:teuthology.orchestra.run.smithi203.stdout: Installing : python3-rados-2:19.0.0-3291.g994e2f20.el9.x86_64 10/156 2024-04-25T10:27:18.028 INFO:teuthology.orchestra.run.smithi203.stdout: Installing : postgresql-private-libs-13.14-1.el9.x86_64 11/156 2024-04-25T10:27:18.078 INFO:teuthology.orchestra.run.smithi203.stdout: Installing : postgresql-13.14-1.el9.x86_64 12/156 2024-04-25T10:27:18.106 INFO:teuthology.orchestra.run.smithi203.stdout: Installing : libcephsqlite-2:19.0.0-3291.g994e2f20.el9.x86_64 13/156 2024-04-25T10:27:18.211 INFO:teuthology.orchestra.run.smithi203.stdout: Running scriptlet: libcephsqlite-2:19.0.0-3291.g994e2f20.el9.x86_64 13/156 2024-04-25T10:27:18.240 INFO:teuthology.orchestra.run.smithi203.stdout: Installing : libradosstriper1-2:19.0.0-3291.g994e2f20.el9.x86 14/156 2024-04-25T10:27:18.303 INFO:teuthology.orchestra.run.smithi203.stdout: Running scriptlet: libradosstriper1-2:19.0.0-3291.g994e2f20.el9.x86 14/156 2024-04-25T10:27:18.325 INFO:teuthology.orchestra.run.smithi203.stdout: Installing : qatlib-service-24.02.0-1.el9.x86_64 15/156 2024-04-25T10:27:18.345 INFO:teuthology.orchestra.run.smithi203.stdout: Running scriptlet: qatlib-service-24.02.0-1.el9.x86_64 15/156 2024-04-25T10:27:18.507 INFO:teuthology.orchestra.run.smithi037.stdout: Installing : python3-requests-oauthlib-1.3.0-12.el9.noarch 82/156 2024-04-25T10:27:18.525 INFO:teuthology.orchestra.run.smithi203.stdout: Running scriptlet: qatlib-24.02.0-1.el9.x86_64 16/156 2024-04-25T10:27:18.554 INFO:teuthology.orchestra.run.smithi037.stdout: Installing : python3-kubernetes-1:26.1.0-2.el9.noarch 83/156 2024-04-25T10:27:18.558 INFO:teuthology.orchestra.run.smithi203.stdout: Installing : qatlib-24.02.0-1.el9.x86_64 16/156 2024-04-25T10:27:18.605 INFO:teuthology.orchestra.run.smithi037.stdout: Installing : python3-jsonpointer-2.0-4.el9.noarch 84/156 2024-04-25T10:27:18.669 INFO:teuthology.orchestra.run.smithi203.stdout: Installing : re2-1:20211101-3.el9.x86_64 17/156 2024-04-25T10:27:18.709 INFO:teuthology.orchestra.run.smithi037.stdout: Installing : python3-jsonpatch-1.21-16.el9.noarch 85/156 2024-04-25T10:27:18.756 INFO:teuthology.orchestra.run.smithi037.stdout: Installing : python3-devel-3.9.18-3.el9.x86_64 86/156 2024-04-25T10:27:18.771 INFO:teuthology.orchestra.run.smithi203.stdout: Installing : libarrow-9.0.0-10.el9.x86_64 18/156 2024-04-25T10:27:18.807 INFO:teuthology.orchestra.run.smithi203.stdout: Installing : python3-werkzeug-2.0.3-3.el9.1.noarch 19/156 2024-04-25T10:27:18.846 INFO:teuthology.orchestra.run.smithi037.stdout: Installing : python3-attrs-20.3.0-7.el9.noarch 87/156 2024-04-25T10:27:18.852 INFO:teuthology.orchestra.run.smithi203.stdout: Installing : liboath-2.6.7-2.el9.x86_64 20/156 2024-04-25T10:27:18.882 INFO:teuthology.orchestra.run.smithi037.stdout: Installing : python3-cmd2-2.4.2-2.el9.noarch 88/156 2024-04-25T10:27:18.927 INFO:teuthology.orchestra.run.smithi203.stdout: Installing : python3-pytz-2021.1-5.el9.noarch 21/156 2024-04-25T10:27:18.967 INFO:teuthology.orchestra.run.smithi203.stdout: Installing : python3-pyasn1-0.4.8-6.el9.noarch 22/156 2024-04-25T10:27:19.012 INFO:teuthology.orchestra.run.smithi203.stdout: Installing : python3-markupsafe-1.1.1-12.el9.x86_64 23/156 2024-04-25T10:27:19.052 INFO:teuthology.orchestra.run.smithi203.stdout: Installing : protobuf-3.14.0-13.el9.x86_64 24/156 2024-04-25T10:27:19.067 INFO:teuthology.orchestra.run.smithi037.stdout: Installing : openblas-0.3.21-2.el9.x86_64 89/156 2024-04-25T10:27:19.080 INFO:teuthology.orchestra.run.smithi203.stdout: Installing : lua-5.4.4-4.el9.x86_64 25/156 2024-04-25T10:27:19.095 INFO:teuthology.orchestra.run.smithi037.stdout: Installing : openblas-openmp-0.3.21-2.el9.x86_64 90/156 2024-04-25T10:27:19.151 INFO:teuthology.orchestra.run.smithi203.stdout: Installing : flexiblas-3.0.4-8.el9.x86_64 26/156 2024-04-25T10:27:19.176 INFO:teuthology.orchestra.run.smithi037.stdout: Installing : flexiblas-openblas-openmp-3.0.4-8.el9.x86_64 91/156 2024-04-25T10:27:19.213 INFO:teuthology.orchestra.run.smithi203.stdout: Installing : python3-urllib3-1.26.5-5.el9.noarch 27/156 2024-04-25T10:27:19.250 INFO:teuthology.orchestra.run.smithi203.stdout: Installing : python3-requests-2.25.1-8.el9.noarch 28/156 2024-04-25T10:27:19.294 INFO:teuthology.orchestra.run.smithi203.stdout: Installing : libquadmath-11.4.1-3.el9.x86_64 29/156 2024-04-25T10:27:19.370 INFO:teuthology.orchestra.run.smithi203.stdout: Installing : libgfortran-11.4.1-3.el9.x86_64 30/156 2024-04-25T10:27:19.407 INFO:teuthology.orchestra.run.smithi203.stdout: Installing : python3-ceph-common-2:19.0.0-3291.g994e2f20.el9. 31/156 2024-04-25T10:27:19.456 INFO:teuthology.orchestra.run.smithi203.stdout: Installing : python3-ceph-argparse-2:19.0.0-3291.g994e2f20.el 32/156 2024-04-25T10:27:19.487 INFO:teuthology.orchestra.run.smithi203.stdout: Installing : python3-cephfs-2:19.0.0-3291.g994e2f20.el9.x86_6 33/156 2024-04-25T10:27:19.577 INFO:teuthology.orchestra.run.smithi203.stdout: Installing : lua-devel-5.4.4-4.el9.x86_64 34/156 2024-04-25T10:27:19.633 INFO:teuthology.orchestra.run.smithi203.stdout: Installing : luarocks-3.9.2-1.el9.noarch 35/156 2024-04-25T10:27:19.708 INFO:teuthology.orchestra.run.smithi203.stdout: Installing : protobuf-compiler-3.14.0-13.el9.x86_64 36/156 2024-04-25T10:27:19.851 INFO:teuthology.orchestra.run.smithi203.stdout: Installing : python3-mako-1.1.4-6.el9.noarch 37/156 2024-04-25T10:27:19.862 INFO:teuthology.orchestra.run.smithi037.stdout: Installing : flexiblas-netlib-3.0.4-8.el9.x86_64 92/156 2024-04-25T10:27:19.902 INFO:teuthology.orchestra.run.smithi203.stdout: Installing : python3-pyasn1-modules-0.4.8-6.el9.noarch 38/156 2024-04-25T10:27:19.990 INFO:teuthology.orchestra.run.smithi037.stdout: Installing : python3-numpy-1:1.20.1-5.el9.x86_64 93/156 2024-04-25T10:27:20.443 INFO:teuthology.orchestra.run.smithi203.stdout: Installing : python3-rsa-4.9-2.el9.noarch 39/156 2024-04-25T10:27:20.523 INFO:teuthology.orchestra.run.smithi203.stdout: Installing : python3-babel-2.9.1-2.el9.noarch 40/156 2024-04-25T10:27:20.552 INFO:teuthology.orchestra.run.smithi203.stdout: Installing : python3-jinja2-2.11.3-5.el9.noarch 41/156 2024-04-25T10:27:20.594 INFO:teuthology.orchestra.run.smithi203.stdout: Installing : qatzip-libs-1.2.0-1.el9.x86_64 42/156 2024-04-25T10:27:20.633 INFO:teuthology.orchestra.run.smithi203.stdout: Installing : librados-devel-2:19.0.0-3291.g994e2f20.el9.x86_6 43/156 2024-04-25T10:27:20.674 INFO:teuthology.orchestra.run.smithi203.stdout: Installing : python3-jaraco-classes-3.2.1-5.el9.noarch 44/156 2024-04-25T10:27:20.823 INFO:teuthology.orchestra.run.smithi203.stdout: Installing : socat-1.7.4.1-5.el9.x86_64 45/156 2024-04-25T10:27:20.878 INFO:teuthology.orchestra.run.smithi203.stdout: Upgrading : openssl-1:3.2.1-1.el9.x86_64 46/156 2024-04-25T10:27:20.933 INFO:teuthology.orchestra.run.smithi203.stdout: Installing : python3-zc-lockfile-2.0-10.el9.noarch 47/156 2024-04-25T10:27:21.006 INFO:teuthology.orchestra.run.smithi203.stdout: Installing : python3-websocket-client-1.2.3-2.el9.noarch 48/156 2024-04-25T10:27:21.037 INFO:teuthology.orchestra.run.smithi203.stdout: Installing : python3-webob-1.8.7-6.el9.noarch 49/156 2024-04-25T10:27:21.073 INFO:teuthology.orchestra.run.smithi203.stdout: Installing : python3-typing-extensions-4.4.0-2.el9.noarch 50/156 2024-04-25T10:27:21.120 INFO:teuthology.orchestra.run.smithi203.stdout: Installing : python3-repoze-lru-0.7-16.el9.noarch 51/156 2024-04-25T10:27:21.156 INFO:teuthology.orchestra.run.smithi203.stdout: Installing : python3-routes-2.5.1-5.el9.noarch 52/156 2024-04-25T10:27:21.199 INFO:teuthology.orchestra.run.smithi203.stdout: Installing : python3-pyperclip-1.8.2-2.el9.noarch 53/156 2024-04-25T10:27:21.241 INFO:teuthology.orchestra.run.smithi203.stdout: Installing : python3-natsort-7.1.1-5.el9.noarch 54/156 2024-04-25T10:27:21.295 INFO:teuthology.orchestra.run.smithi037.stdout: Installing : python3-numpy-f2py-1:1.20.1-5.el9.x86_64 94/156 2024-04-25T10:27:21.378 INFO:teuthology.orchestra.run.smithi203.stdout: Installing : python3-logutils-0.3.5-21.el9.noarch 55/156 2024-04-25T10:27:21.383 INFO:teuthology.orchestra.run.smithi037.stdout: Installing : python3-scipy-1.6.2-8.el9.x86_64 95/156 2024-04-25T10:27:21.412 INFO:teuthology.orchestra.run.smithi037.stdout: Installing : libzstd-devel-1.5.1-2.el9.x86_64 96/156 2024-04-25T10:27:21.422 INFO:teuthology.orchestra.run.smithi203.stdout: Installing : python3-pecan-1.4.2-3.el9.noarch 56/156 2024-04-25T10:27:21.445 INFO:teuthology.orchestra.run.smithi037.stdout: Installing : libuuid-devel-2.37.4-18.el9.x86_64 97/156 2024-04-25T10:27:21.452 INFO:teuthology.orchestra.run.smithi203.stdout: Installing : python3-colorama-0.4.4-12.el9.noarch 57/156 2024-04-25T10:27:21.494 INFO:teuthology.orchestra.run.smithi203.stdout: Installing : python3-certifi-2021.10.8-2.el9.noarch 58/156 2024-04-25T10:27:21.589 INFO:teuthology.orchestra.run.smithi203.stdout: Installing : python3-cachetools-4.2.4-1.el9.noarch 59/156 2024-04-25T10:27:21.627 INFO:teuthology.orchestra.run.smithi203.stdout: Installing : python3-google-auth-1:2.29.0-1.el9.noarch 60/156 2024-04-25T10:27:21.668 INFO:teuthology.orchestra.run.smithi203.stdout: Installing : libunwind-1.6.2-1.el9.x86_64 61/156 2024-04-25T10:27:21.693 INFO:teuthology.orchestra.run.smithi203.stdout: Installing : gperftools-libs-2.9.1-2.el9.x86_64 62/156 2024-04-25T10:27:21.718 INFO:teuthology.orchestra.run.smithi203.stdout: Installing : libarrow-doc-9.0.0-10.el9.noarch 63/156 2024-04-25T10:27:21.782 INFO:teuthology.orchestra.run.smithi037.stdout: Installing : libpmemobj-1.12.1-1.el9.x86_64 98/156 2024-04-25T10:27:21.793 INFO:teuthology.orchestra.run.smithi203.stdout: Installing : grpc-data-1.46.7-10.el9.noarch 64/156 2024-04-25T10:27:21.824 INFO:teuthology.orchestra.run.smithi037.stdout: Upgrading : librbd1-2:19.0.0-3291.g994e2f20.el9.x86_64 99/156 2024-04-25T10:27:21.899 INFO:teuthology.orchestra.run.smithi037.stdout: Running scriptlet: librbd1-2:19.0.0-3291.g994e2f20.el9.x86_64 99/156 2024-04-25T10:27:21.920 INFO:teuthology.orchestra.run.smithi203.stdout: Installing : abseil-cpp-20211102.0-3.el9.x86_64 65/156 2024-04-25T10:27:21.929 INFO:teuthology.orchestra.run.smithi037.stdout: Installing : python3-rbd-2:19.0.0-3291.g994e2f20.el9.x86_64 100/156 2024-04-25T10:27:21.951 INFO:teuthology.orchestra.run.smithi203.stdout: Installing : python3-grpcio-1.46.7-10.el9.x86_64 66/156 2024-04-25T10:27:21.965 INFO:teuthology.orchestra.run.smithi037.stdout: Installing : boost-program-options-1.75.0-8.el9.x86_64 101/156 2024-04-25T10:27:22.034 INFO:teuthology.orchestra.run.smithi203.stdout: Installing : zlib-devel-1.2.11-41.el9.x86_64 67/156 2024-04-25T10:27:22.272 INFO:teuthology.orchestra.run.smithi203.stdout: Installing : xmlstarlet-1.6.1-20.el9.x86_64 68/156 2024-04-25T10:27:22.412 INFO:teuthology.orchestra.run.smithi203.stdout: Installing : uuid-1.6.2-55.el9.x86_64 69/156 2024-04-25T10:27:22.451 INFO:teuthology.orchestra.run.smithi203.stdout: Installing : python3-wcwidth-0.2.5-8.el9.noarch 70/156 2024-04-25T10:27:22.472 INFO:teuthology.orchestra.run.smithi037.stdout: Installing : parquet-libs-9.0.0-10.el9.x86_64 102/156 2024-04-25T10:27:22.481 INFO:teuthology.orchestra.run.smithi203.stdout: Installing : python3-toml-0.10.2-6.el9.noarch 71/156 2024-04-25T10:27:22.502 INFO:teuthology.orchestra.run.smithi037.stdout: Installing : librgw2-2:19.0.0-3291.g994e2f20.el9.x86_64 103/156 2024-04-25T10:27:22.516 INFO:teuthology.orchestra.run.smithi203.stdout: Installing : python3-jaraco-functools-3.5.0-2.el9.noarch 72/156 2024-04-25T10:27:22.550 INFO:teuthology.orchestra.run.smithi203.stdout: Installing : python3-jaraco-text-3.2.0-6.el9.noarch 73/156 2024-04-25T10:27:22.557 INFO:teuthology.orchestra.run.smithi037.stdout: Running scriptlet: librgw2-2:19.0.0-3291.g994e2f20.el9.x86_64 103/156 2024-04-25T10:27:22.586 INFO:teuthology.orchestra.run.smithi037.stdout: Installing : python3-rgw-2:19.0.0-3291.g994e2f20.el9.x86_64 104/156 2024-04-25T10:27:22.591 INFO:teuthology.orchestra.run.smithi203.stdout: Installing : python3-jaraco-collections-3.0.0-8.el9.noarch 74/156 2024-04-25T10:27:22.625 INFO:teuthology.orchestra.run.smithi203.stdout: Installing : python3-tempora-5.0.0-2.el9.noarch 75/156 2024-04-25T10:27:22.714 INFO:teuthology.orchestra.run.smithi203.stdout: Installing : python3-portend-3.1.0-2.el9.noarch 76/156 2024-04-25T10:27:22.773 INFO:teuthology.orchestra.run.smithi203.stdout: Installing : python3-protobuf-3.14.0-13.el9.noarch 77/156 2024-04-25T10:27:22.804 INFO:teuthology.orchestra.run.smithi203.stdout: Installing : python3-grpcio-tools-1.46.7-10.el9.x86_64 78/156 2024-04-25T10:27:22.852 INFO:teuthology.orchestra.run.smithi203.stdout: Installing : python3-prettytable-0.7.2-27.el9.noarch 79/156 2024-04-25T10:27:22.950 INFO:teuthology.orchestra.run.smithi203.stdout: Installing : python3-packaging-20.9-5.el9.noarch 80/156 2024-04-25T10:27:23.006 INFO:teuthology.orchestra.run.smithi203.stdout: Installing : python3-oauthlib-3.1.1-5.el9.noarch 81/156 2024-04-25T10:27:23.740 INFO:teuthology.orchestra.run.smithi203.stdout: Installing : python3-requests-oauthlib-1.3.0-12.el9.noarch 82/156 2024-04-25T10:27:23.787 INFO:teuthology.orchestra.run.smithi203.stdout: Installing : python3-kubernetes-1:26.1.0-2.el9.noarch 83/156 2024-04-25T10:27:23.820 INFO:teuthology.orchestra.run.smithi203.stdout: Installing : python3-jsonpointer-2.0-4.el9.noarch 84/156 2024-04-25T10:27:23.922 INFO:teuthology.orchestra.run.smithi203.stdout: Installing : python3-jsonpatch-1.21-16.el9.noarch 85/156 2024-04-25T10:27:23.985 INFO:teuthology.orchestra.run.smithi203.stdout: Installing : python3-devel-3.9.18-3.el9.x86_64 86/156 2024-04-25T10:27:24.076 INFO:teuthology.orchestra.run.smithi203.stdout: Installing : python3-attrs-20.3.0-7.el9.noarch 87/156 2024-04-25T10:27:24.106 INFO:teuthology.orchestra.run.smithi203.stdout: Installing : python3-cmd2-2.4.2-2.el9.noarch 88/156 2024-04-25T10:27:24.287 INFO:teuthology.orchestra.run.smithi203.stdout: Installing : openblas-0.3.21-2.el9.x86_64 89/156 2024-04-25T10:27:24.321 INFO:teuthology.orchestra.run.smithi203.stdout: Installing : openblas-openmp-0.3.21-2.el9.x86_64 90/156 2024-04-25T10:27:24.408 INFO:teuthology.orchestra.run.smithi203.stdout: Installing : flexiblas-openblas-openmp-3.0.4-8.el9.x86_64 91/156 2024-04-25T10:27:24.937 INFO:teuthology.orchestra.run.smithi037.stdout: Running scriptlet: ceph-common-2:19.0.0-3291.g994e2f20.el9.x86_64 105/156 2024-04-25T10:27:24.966 INFO:teuthology.orchestra.run.smithi037.stdout: Installing : ceph-common-2:19.0.0-3291.g994e2f20.el9.x86_64 105/156 2024-04-25T10:27:25.060 INFO:teuthology.orchestra.run.smithi203.stdout: Installing : flexiblas-netlib-3.0.4-8.el9.x86_64 92/156 2024-04-25T10:27:25.188 INFO:teuthology.orchestra.run.smithi203.stdout: Installing : python3-numpy-1:1.20.1-5.el9.x86_64 93/156 2024-04-25T10:27:25.559 INFO:teuthology.orchestra.run.smithi037.stdout: Running scriptlet: ceph-common-2:19.0.0-3291.g994e2f20.el9.x86_64 105/156 2024-04-25T10:27:25.603 INFO:teuthology.orchestra.run.smithi037.stdout: Installing : ceph-base-2:19.0.0-3291.g994e2f20.el9.x86_64 106/156 2024-04-25T10:27:25.695 INFO:teuthology.orchestra.run.smithi037.stdout: Running scriptlet: ceph-base-2:19.0.0-3291.g994e2f20.el9.x86_64 106/156 2024-04-25T10:27:25.695 INFO:teuthology.orchestra.run.smithi037.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph.target → /usr/lib/systemd/system/ceph.target. 2024-04-25T10:27:25.695 INFO:teuthology.orchestra.run.smithi037.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-crash.service → /usr/lib/systemd/system/ceph-crash.service. 2024-04-25T10:27:25.695 INFO:teuthology.orchestra.run.smithi037.stdout: 2024-04-25T10:27:25.741 INFO:teuthology.orchestra.run.smithi037.stdout: Installing : ceph-selinux-2:19.0.0-3291.g994e2f20.el9.x86_64 107/156 2024-04-25T10:27:26.436 INFO:teuthology.orchestra.run.smithi203.stdout: Installing : python3-numpy-f2py-1:1.20.1-5.el9.x86_64 94/156 2024-04-25T10:27:26.535 INFO:teuthology.orchestra.run.smithi203.stdout: Installing : python3-scipy-1.6.2-8.el9.x86_64 95/156 2024-04-25T10:27:26.563 INFO:teuthology.orchestra.run.smithi203.stdout: Installing : libzstd-devel-1.5.1-2.el9.x86_64 96/156 2024-04-25T10:27:26.597 INFO:teuthology.orchestra.run.smithi203.stdout: Installing : libuuid-devel-2.37.4-18.el9.x86_64 97/156 2024-04-25T10:27:26.918 INFO:teuthology.orchestra.run.smithi203.stdout: Installing : libpmemobj-1.12.1-1.el9.x86_64 98/156 2024-04-25T10:27:26.952 INFO:teuthology.orchestra.run.smithi203.stdout: Upgrading : librbd1-2:19.0.0-3291.g994e2f20.el9.x86_64 99/156 2024-04-25T10:27:27.043 INFO:teuthology.orchestra.run.smithi203.stdout: Running scriptlet: librbd1-2:19.0.0-3291.g994e2f20.el9.x86_64 99/156 2024-04-25T10:27:27.073 INFO:teuthology.orchestra.run.smithi203.stdout: Installing : python3-rbd-2:19.0.0-3291.g994e2f20.el9.x86_64 100/156 2024-04-25T10:27:27.110 INFO:teuthology.orchestra.run.smithi203.stdout: Installing : boost-program-options-1.75.0-8.el9.x86_64 101/156 2024-04-25T10:27:27.597 INFO:teuthology.orchestra.run.smithi203.stdout: Installing : parquet-libs-9.0.0-10.el9.x86_64 102/156 2024-04-25T10:27:27.630 INFO:teuthology.orchestra.run.smithi203.stdout: Installing : librgw2-2:19.0.0-3291.g994e2f20.el9.x86_64 103/156 2024-04-25T10:27:27.701 INFO:teuthology.orchestra.run.smithi203.stdout: Running scriptlet: librgw2-2:19.0.0-3291.g994e2f20.el9.x86_64 103/156 2024-04-25T10:27:27.731 INFO:teuthology.orchestra.run.smithi203.stdout: Installing : python3-rgw-2:19.0.0-3291.g994e2f20.el9.x86_64 104/156 2024-04-25T10:27:29.954 INFO:teuthology.orchestra.run.smithi203.stdout: Running scriptlet: ceph-common-2:19.0.0-3291.g994e2f20.el9.x86_64 105/156 2024-04-25T10:27:30.004 INFO:teuthology.orchestra.run.smithi203.stdout: Installing : ceph-common-2:19.0.0-3291.g994e2f20.el9.x86_64 105/156 2024-04-25T10:27:30.569 INFO:teuthology.orchestra.run.smithi203.stdout: Running scriptlet: ceph-common-2:19.0.0-3291.g994e2f20.el9.x86_64 105/156 2024-04-25T10:27:30.616 INFO:teuthology.orchestra.run.smithi203.stdout: Installing : ceph-base-2:19.0.0-3291.g994e2f20.el9.x86_64 106/156 2024-04-25T10:27:30.700 INFO:teuthology.orchestra.run.smithi203.stdout: Running scriptlet: ceph-base-2:19.0.0-3291.g994e2f20.el9.x86_64 106/156 2024-04-25T10:27:30.700 INFO:teuthology.orchestra.run.smithi203.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph.target → /usr/lib/systemd/system/ceph.target. 2024-04-25T10:27:30.700 INFO:teuthology.orchestra.run.smithi203.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-crash.service → /usr/lib/systemd/system/ceph-crash.service. 2024-04-25T10:27:30.700 INFO:teuthology.orchestra.run.smithi203.stdout: 2024-04-25T10:27:30.734 INFO:teuthology.orchestra.run.smithi203.stdout: Installing : ceph-selinux-2:19.0.0-3291.g994e2f20.el9.x86_64 107/156 2024-04-25T10:27:37.535 INFO:teuthology.orchestra.run.smithi037.stdout: Running scriptlet: ceph-selinux-2:19.0.0-3291.g994e2f20.el9.x86_64 107/156 2024-04-25T10:27:37.535 INFO:teuthology.orchestra.run.smithi037.stdout:skipping the directory /sys 2024-04-25T10:27:37.536 INFO:teuthology.orchestra.run.smithi037.stdout:skipping the directory /proc 2024-04-25T10:27:37.536 INFO:teuthology.orchestra.run.smithi037.stdout:skipping the directory /mnt 2024-04-25T10:27:37.536 INFO:teuthology.orchestra.run.smithi037.stdout:skipping the directory /var/tmp 2024-04-25T10:27:37.536 INFO:teuthology.orchestra.run.smithi037.stdout:skipping the directory /home 2024-04-25T10:27:37.536 INFO:teuthology.orchestra.run.smithi037.stdout:skipping the directory /root 2024-04-25T10:27:37.536 INFO:teuthology.orchestra.run.smithi037.stdout:skipping the directory /tmp 2024-04-25T10:27:37.536 INFO:teuthology.orchestra.run.smithi037.stdout: 2024-04-25T10:27:37.717 INFO:teuthology.orchestra.run.smithi037.stdout: Installing : ceph-volume-2:19.0.0-3291.g994e2f20.el9.noarch 108/156 2024-04-25T10:27:37.734 INFO:teuthology.orchestra.run.smithi037.stdout: Running scriptlet: ceph-volume-2:19.0.0-3291.g994e2f20.el9.noarch 108/156 2024-04-25T10:27:37.734 INFO:teuthology.orchestra.run.smithi037.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-04-25T10:27:37.734 INFO:teuthology.orchestra.run.smithi037.stdout:Invalid unit name "ceph-volume@*.service" escaped as "ceph-volume@\x2a.service". 2024-04-25T10:27:37.735 INFO:teuthology.orchestra.run.smithi037.stdout: 2024-04-25T10:27:39.370 INFO:teuthology.orchestra.run.smithi037.stdout: Installing : ceph-osd-2:19.0.0-3291.g994e2f20.el9.x86_64 109/156 2024-04-25T10:27:39.398 INFO:teuthology.orchestra.run.smithi037.stdout: Running scriptlet: ceph-osd-2:19.0.0-3291.g994e2f20.el9.x86_64 109/156 2024-04-25T10:27:39.398 INFO:teuthology.orchestra.run.smithi037.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-04-25T10:27:39.398 INFO:teuthology.orchestra.run.smithi037.stdout:Invalid unit name "ceph-osd@*.service" escaped as "ceph-osd@\x2a.service". 2024-04-25T10:27:39.398 INFO:teuthology.orchestra.run.smithi037.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-osd.target → /usr/lib/systemd/system/ceph-osd.target. 2024-04-25T10:27:39.398 INFO:teuthology.orchestra.run.smithi037.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-osd.target → /usr/lib/systemd/system/ceph-osd.target. 2024-04-25T10:27:39.399 INFO:teuthology.orchestra.run.smithi037.stdout: 2024-04-25T10:27:39.638 INFO:teuthology.orchestra.run.smithi037.stdout: Installing : ceph-mds-2:19.0.0-3291.g994e2f20.el9.x86_64 110/156 2024-04-25T10:27:39.662 INFO:teuthology.orchestra.run.smithi037.stdout: Running scriptlet: ceph-mds-2:19.0.0-3291.g994e2f20.el9.x86_64 110/156 2024-04-25T10:27:39.662 INFO:teuthology.orchestra.run.smithi037.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-04-25T10:27:39.662 INFO:teuthology.orchestra.run.smithi037.stdout:Invalid unit name "ceph-mds@*.service" escaped as "ceph-mds@\x2a.service". 2024-04-25T10:27:39.662 INFO:teuthology.orchestra.run.smithi037.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-mds.target → /usr/lib/systemd/system/ceph-mds.target. 2024-04-25T10:27:39.663 INFO:teuthology.orchestra.run.smithi037.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-mds.target → /usr/lib/systemd/system/ceph-mds.target. 2024-04-25T10:27:39.663 INFO:teuthology.orchestra.run.smithi037.stdout: 2024-04-25T10:27:40.147 INFO:teuthology.orchestra.run.smithi037.stdout: Installing : ceph-mon-2:19.0.0-3291.g994e2f20.el9.x86_64 111/156 2024-04-25T10:27:40.171 INFO:teuthology.orchestra.run.smithi037.stdout: Running scriptlet: ceph-mon-2:19.0.0-3291.g994e2f20.el9.x86_64 111/156 2024-04-25T10:27:40.171 INFO:teuthology.orchestra.run.smithi037.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-04-25T10:27:40.171 INFO:teuthology.orchestra.run.smithi037.stdout:Invalid unit name "ceph-mon@*.service" escaped as "ceph-mon@\x2a.service". 2024-04-25T10:27:40.172 INFO:teuthology.orchestra.run.smithi037.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-mon.target → /usr/lib/systemd/system/ceph-mon.target. 2024-04-25T10:27:40.172 INFO:teuthology.orchestra.run.smithi037.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-mon.target → /usr/lib/systemd/system/ceph-mon.target. 2024-04-25T10:27:40.172 INFO:teuthology.orchestra.run.smithi037.stdout: 2024-04-25T10:27:40.286 INFO:teuthology.orchestra.run.smithi037.stdout: Installing : automake-1.16.2-8.el9.noarch 112/156 2024-04-25T10:27:40.344 INFO:teuthology.orchestra.run.smithi037.stdout: Installing : python3-ply-3.11-14.el9.noarch 113/156 2024-04-25T10:27:40.408 INFO:teuthology.orchestra.run.smithi037.stdout: Installing : python3-pycparser-2.20-6.el9.noarch 114/156 2024-04-25T10:27:40.606 INFO:teuthology.orchestra.run.smithi037.stdout: Installing : python3-cffi-1.14.5-5.el9.x86_64 115/156 2024-04-25T10:27:40.657 INFO:teuthology.orchestra.run.smithi037.stdout: Installing : python3-cryptography-36.0.1-4.el9.x86_64 116/156 2024-04-25T10:27:40.738 INFO:teuthology.orchestra.run.smithi037.stdout: Installing : python3-pyOpenSSL-21.0.0-1.el9.noarch 117/156 2024-04-25T10:27:40.827 INFO:teuthology.orchestra.run.smithi037.stdout: Installing : python3-cheroot-8.6.0-4.el9.noarch 118/156 2024-04-25T10:27:40.971 INFO:teuthology.orchestra.run.smithi037.stdout: Installing : python3-cherrypy-18.6.1-2.el9.noarch 119/156 2024-04-25T10:27:41.015 INFO:teuthology.orchestra.run.smithi037.stdout: Installing : python3-asyncssh-2.13.2-5.el9.noarch 120/156 2024-04-25T10:27:41.047 INFO:teuthology.orchestra.run.smithi037.stdout: Installing : python3-bcrypt-3.2.2-1.el9.x86_64 121/156 2024-04-25T10:27:41.074 INFO:teuthology.orchestra.run.smithi037.stdout: Installing : fuse-2.9.9-15.el9.x86_64 122/156 2024-04-25T10:27:41.528 INFO:teuthology.orchestra.run.smithi037.stdout: Running scriptlet: cephadm-2:19.0.0-3291.g994e2f20.el9.noarch 123/156 2024-04-25T10:27:41.547 INFO:teuthology.orchestra.run.smithi037.stdout: Installing : cephadm-2:19.0.0-3291.g994e2f20.el9.noarch 123/156 2024-04-25T10:27:41.580 INFO:teuthology.orchestra.run.smithi037.stdout: Installing : ceph-prometheus-alerts-2:19.0.0-3291.g994e2f20.e 124/156 2024-04-25T10:27:41.643 INFO:teuthology.orchestra.run.smithi037.stdout: Installing : ceph-grafana-dashboards-2:19.0.0-3291.g994e2f20. 125/156 2024-04-25T10:27:41.670 INFO:teuthology.orchestra.run.smithi037.stdout: Installing : ceph-mgr-cephadm-2:19.0.0-3291.g994e2f20.el9.noa 126/156 2024-04-25T10:27:42.520 INFO:teuthology.orchestra.run.smithi203.stdout: Running scriptlet: ceph-selinux-2:19.0.0-3291.g994e2f20.el9.x86_64 107/156 2024-04-25T10:27:42.520 INFO:teuthology.orchestra.run.smithi203.stdout:skipping the directory /sys 2024-04-25T10:27:42.520 INFO:teuthology.orchestra.run.smithi203.stdout:skipping the directory /proc 2024-04-25T10:27:42.521 INFO:teuthology.orchestra.run.smithi203.stdout:skipping the directory /mnt 2024-04-25T10:27:42.521 INFO:teuthology.orchestra.run.smithi203.stdout:skipping the directory /var/tmp 2024-04-25T10:27:42.521 INFO:teuthology.orchestra.run.smithi203.stdout:skipping the directory /home 2024-04-25T10:27:42.521 INFO:teuthology.orchestra.run.smithi203.stdout:skipping the directory /root 2024-04-25T10:27:42.521 INFO:teuthology.orchestra.run.smithi203.stdout:skipping the directory /tmp 2024-04-25T10:27:42.521 INFO:teuthology.orchestra.run.smithi203.stdout: 2024-04-25T10:27:42.701 INFO:teuthology.orchestra.run.smithi203.stdout: Installing : ceph-volume-2:19.0.0-3291.g994e2f20.el9.noarch 108/156 2024-04-25T10:27:42.717 INFO:teuthology.orchestra.run.smithi203.stdout: Running scriptlet: ceph-volume-2:19.0.0-3291.g994e2f20.el9.noarch 108/156 2024-04-25T10:27:42.717 INFO:teuthology.orchestra.run.smithi203.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-04-25T10:27:42.717 INFO:teuthology.orchestra.run.smithi203.stdout:Invalid unit name "ceph-volume@*.service" escaped as "ceph-volume@\x2a.service". 2024-04-25T10:27:42.717 INFO:teuthology.orchestra.run.smithi203.stdout: 2024-04-25T10:27:42.772 INFO:teuthology.orchestra.run.smithi037.stdout: Running scriptlet: ceph-mgr-cephadm-2:19.0.0-3291.g994e2f20.el9.noa 126/156 2024-04-25T10:27:42.814 INFO:teuthology.orchestra.run.smithi037.stdout: Installing : ceph-mgr-dashboard-2:19.0.0-3291.g994e2f20.el9.n 127/156 2024-04-25T10:27:43.838 INFO:teuthology.orchestra.run.smithi037.stdout: Running scriptlet: ceph-mgr-dashboard-2:19.0.0-3291.g994e2f20.el9.n 127/156 2024-04-25T10:27:43.873 INFO:teuthology.orchestra.run.smithi037.stdout: Installing : ceph-mgr-diskprediction-local-2:19.0.0-3291.g994 128/156 2024-04-25T10:27:43.982 INFO:teuthology.orchestra.run.smithi037.stdout: Running scriptlet: ceph-mgr-diskprediction-local-2:19.0.0-3291.g994 128/156 2024-04-25T10:27:44.164 INFO:teuthology.orchestra.run.smithi037.stdout: Installing : ceph-mgr-modules-core-2:19.0.0-3291.g994e2f20.el 129/156 2024-04-25T10:27:44.192 INFO:teuthology.orchestra.run.smithi037.stdout: Installing : ceph-mgr-2:19.0.0-3291.g994e2f20.el9.x86_64 130/156 2024-04-25T10:27:44.217 INFO:teuthology.orchestra.run.smithi037.stdout: Running scriptlet: ceph-mgr-2:19.0.0-3291.g994e2f20.el9.x86_64 130/156 2024-04-25T10:27:44.217 INFO:teuthology.orchestra.run.smithi037.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-04-25T10:27:44.217 INFO:teuthology.orchestra.run.smithi037.stdout:Invalid unit name "ceph-mgr@*.service" escaped as "ceph-mgr@\x2a.service". 2024-04-25T10:27:44.217 INFO:teuthology.orchestra.run.smithi037.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-mgr.target → /usr/lib/systemd/system/ceph-mgr.target. 2024-04-25T10:27:44.217 INFO:teuthology.orchestra.run.smithi037.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-mgr.target → /usr/lib/systemd/system/ceph-mgr.target. 2024-04-25T10:27:44.217 INFO:teuthology.orchestra.run.smithi037.stdout: 2024-04-25T10:27:44.267 INFO:teuthology.orchestra.run.smithi037.stdout: Installing : ceph-mgr-rook-2:19.0.0-3291.g994e2f20.el9.noarch 131/156 2024-04-25T10:27:44.279 INFO:teuthology.orchestra.run.smithi037.stdout: Running scriptlet: ceph-mgr-rook-2:19.0.0-3291.g994e2f20.el9.noarch 131/156 2024-04-25T10:27:44.287 INFO:teuthology.orchestra.run.smithi203.stdout: Installing : ceph-osd-2:19.0.0-3291.g994e2f20.el9.x86_64 109/156 2024-04-25T10:27:44.313 INFO:teuthology.orchestra.run.smithi203.stdout: Running scriptlet: ceph-osd-2:19.0.0-3291.g994e2f20.el9.x86_64 109/156 2024-04-25T10:27:44.313 INFO:teuthology.orchestra.run.smithi203.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-04-25T10:27:44.313 INFO:teuthology.orchestra.run.smithi203.stdout:Invalid unit name "ceph-osd@*.service" escaped as "ceph-osd@\x2a.service". 2024-04-25T10:27:44.314 INFO:teuthology.orchestra.run.smithi203.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-osd.target → /usr/lib/systemd/system/ceph-osd.target. 2024-04-25T10:27:44.314 INFO:teuthology.orchestra.run.smithi203.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-osd.target → /usr/lib/systemd/system/ceph-osd.target. 2024-04-25T10:27:44.314 INFO:teuthology.orchestra.run.smithi203.stdout: 2024-04-25T10:27:44.382 INFO:teuthology.orchestra.run.smithi037.stdout: Installing : ceph-2:19.0.0-3291.g994e2f20.el9.x86_64 132/156 2024-04-25T10:27:44.434 INFO:teuthology.orchestra.run.smithi037.stdout: Installing : ceph-fuse-2:19.0.0-3291.g994e2f20.el9.x86_64 133/156 2024-04-25T10:27:44.538 INFO:teuthology.orchestra.run.smithi203.stdout: Installing : ceph-mds-2:19.0.0-3291.g994e2f20.el9.x86_64 110/156 2024-04-25T10:27:44.561 INFO:teuthology.orchestra.run.smithi203.stdout: Running scriptlet: ceph-mds-2:19.0.0-3291.g994e2f20.el9.x86_64 110/156 2024-04-25T10:27:44.562 INFO:teuthology.orchestra.run.smithi203.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-04-25T10:27:44.562 INFO:teuthology.orchestra.run.smithi203.stdout:Invalid unit name "ceph-mds@*.service" escaped as "ceph-mds@\x2a.service". 2024-04-25T10:27:44.562 INFO:teuthology.orchestra.run.smithi203.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-mds.target → /usr/lib/systemd/system/ceph-mds.target. 2024-04-25T10:27:44.562 INFO:teuthology.orchestra.run.smithi203.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-mds.target → /usr/lib/systemd/system/ceph-mds.target. 2024-04-25T10:27:44.562 INFO:teuthology.orchestra.run.smithi203.stdout: 2024-04-25T10:27:45.022 INFO:teuthology.orchestra.run.smithi203.stdout: Installing : ceph-mon-2:19.0.0-3291.g994e2f20.el9.x86_64 111/156 2024-04-25T10:27:45.045 INFO:teuthology.orchestra.run.smithi203.stdout: Running scriptlet: ceph-mon-2:19.0.0-3291.g994e2f20.el9.x86_64 111/156 2024-04-25T10:27:45.045 INFO:teuthology.orchestra.run.smithi203.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-04-25T10:27:45.045 INFO:teuthology.orchestra.run.smithi203.stdout:Invalid unit name "ceph-mon@*.service" escaped as "ceph-mon@\x2a.service". 2024-04-25T10:27:45.045 INFO:teuthology.orchestra.run.smithi203.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-mon.target → /usr/lib/systemd/system/ceph-mon.target. 2024-04-25T10:27:45.045 INFO:teuthology.orchestra.run.smithi203.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-mon.target → /usr/lib/systemd/system/ceph-mon.target. 2024-04-25T10:27:45.045 INFO:teuthology.orchestra.run.smithi203.stdout: 2024-04-25T10:27:45.144 INFO:teuthology.orchestra.run.smithi203.stdout: Installing : automake-1.16.2-8.el9.noarch 112/156 2024-04-25T10:27:45.202 INFO:teuthology.orchestra.run.smithi203.stdout: Installing : python3-ply-3.11-14.el9.noarch 113/156 2024-04-25T10:27:45.266 INFO:teuthology.orchestra.run.smithi203.stdout: Installing : python3-pycparser-2.20-6.el9.noarch 114/156 2024-04-25T10:27:45.422 INFO:teuthology.orchestra.run.smithi037.stdout: Installing : libtool-2.4.6-45.el9.x86_64 134/156 2024-04-25T10:27:45.444 INFO:teuthology.orchestra.run.smithi037.stdout: Installing : ceph-radosgw-2:19.0.0-3291.g994e2f20.el9.x86_64 135/156 2024-04-25T10:27:45.453 INFO:teuthology.orchestra.run.smithi203.stdout: Installing : python3-cffi-1.14.5-5.el9.x86_64 115/156 2024-04-25T10:27:45.469 INFO:teuthology.orchestra.run.smithi037.stdout: Running scriptlet: ceph-radosgw-2:19.0.0-3291.g994e2f20.el9.x86_64 135/156 2024-04-25T10:27:45.469 INFO:teuthology.orchestra.run.smithi037.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-04-25T10:27:45.469 INFO:teuthology.orchestra.run.smithi037.stdout:Invalid unit name "ceph-radosgw@*.service" escaped as "ceph-radosgw@\x2a.service". 2024-04-25T10:27:45.469 INFO:teuthology.orchestra.run.smithi037.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-radosgw.target → /usr/lib/systemd/system/ceph-radosgw.target. 2024-04-25T10:27:45.469 INFO:teuthology.orchestra.run.smithi037.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-radosgw.target → /usr/lib/systemd/system/ceph-radosgw.target. 2024-04-25T10:27:45.469 INFO:teuthology.orchestra.run.smithi037.stdout: 2024-04-25T10:27:45.499 INFO:teuthology.orchestra.run.smithi203.stdout: Installing : python3-cryptography-36.0.1-4.el9.x86_64 116/156 2024-04-25T10:27:45.520 INFO:teuthology.orchestra.run.smithi037.stdout: Installing : ceph-immutable-object-cache-2:19.0.0-3291.g994e2 136/156 2024-04-25T10:27:45.544 INFO:teuthology.orchestra.run.smithi037.stdout: Running scriptlet: ceph-immutable-object-cache-2:19.0.0-3291.g994e2 136/156 2024-04-25T10:27:45.544 INFO:teuthology.orchestra.run.smithi037.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-04-25T10:27:45.544 INFO:teuthology.orchestra.run.smithi037.stdout:Invalid unit name "ceph-immutable-object-cache@*.service" escaped as "ceph-immutable-object-cache@\x2a.service". 2024-04-25T10:27:45.544 INFO:teuthology.orchestra.run.smithi037.stdout: 2024-04-25T10:27:45.578 INFO:teuthology.orchestra.run.smithi203.stdout: Installing : python3-pyOpenSSL-21.0.0-1.el9.noarch 117/156 2024-04-25T10:27:45.674 INFO:teuthology.orchestra.run.smithi037.stdout: Installing : cephfs-mirror-2:19.0.0-3291.g994e2f20.el9.x86_64 137/156 2024-04-25T10:27:45.674 INFO:teuthology.orchestra.run.smithi037.stdout: Running scriptlet: cephfs-mirror-2:19.0.0-3291.g994e2f20.el9.x86_64 137/156 2024-04-25T10:27:45.674 INFO:teuthology.orchestra.run.smithi037.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-04-25T10:27:45.674 INFO:teuthology.orchestra.run.smithi037.stdout:Invalid unit name "cephfs-mirror@*.service" escaped as "cephfs-mirror@\x2a.service". 2024-04-25T10:27:45.674 INFO:teuthology.orchestra.run.smithi037.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/cephfs-mirror.target → /usr/lib/systemd/system/cephfs-mirror.target. 2024-04-25T10:27:45.674 INFO:teuthology.orchestra.run.smithi037.stdout:Created symlink /etc/systemd/system/ceph.target.wants/cephfs-mirror.target → /usr/lib/systemd/system/cephfs-mirror.target. 2024-04-25T10:27:45.674 INFO:teuthology.orchestra.run.smithi037.stdout: 2024-04-25T10:27:45.675 INFO:teuthology.orchestra.run.smithi203.stdout: Installing : python3-cheroot-8.6.0-4.el9.noarch 118/156 2024-04-25T10:27:45.815 INFO:teuthology.orchestra.run.smithi203.stdout: Installing : python3-cherrypy-18.6.1-2.el9.noarch 119/156 2024-04-25T10:27:45.858 INFO:teuthology.orchestra.run.smithi203.stdout: Installing : python3-asyncssh-2.13.2-5.el9.noarch 120/156 2024-04-25T10:27:45.890 INFO:teuthology.orchestra.run.smithi203.stdout: Installing : python3-bcrypt-3.2.2-1.el9.x86_64 121/156 2024-04-25T10:27:45.918 INFO:teuthology.orchestra.run.smithi203.stdout: Installing : fuse-2.9.9-15.el9.x86_64 122/156 2024-04-25T10:27:45.947 INFO:teuthology.orchestra.run.smithi037.stdout: Installing : rbd-mirror-2:19.0.0-3291.g994e2f20.el9.x86_64 138/156 2024-04-25T10:27:45.971 INFO:teuthology.orchestra.run.smithi037.stdout: Running scriptlet: rbd-mirror-2:19.0.0-3291.g994e2f20.el9.x86_64 138/156 2024-04-25T10:27:45.971 INFO:teuthology.orchestra.run.smithi037.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-04-25T10:27:45.971 INFO:teuthology.orchestra.run.smithi037.stdout:Invalid unit name "ceph-rbd-mirror@*.service" escaped as "ceph-rbd-mirror@\x2a.service". 2024-04-25T10:27:45.971 INFO:teuthology.orchestra.run.smithi037.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-rbd-mirror.target → /usr/lib/systemd/system/ceph-rbd-mirror.target. 2024-04-25T10:27:45.971 INFO:teuthology.orchestra.run.smithi037.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-rbd-mirror.target → /usr/lib/systemd/system/ceph-rbd-mirror.target. 2024-04-25T10:27:45.972 INFO:teuthology.orchestra.run.smithi037.stdout: 2024-04-25T10:27:46.380 INFO:teuthology.orchestra.run.smithi203.stdout: Running scriptlet: cephadm-2:19.0.0-3291.g994e2f20.el9.noarch 123/156 2024-04-25T10:27:46.408 INFO:teuthology.orchestra.run.smithi203.stdout: Installing : cephadm-2:19.0.0-3291.g994e2f20.el9.noarch 123/156 2024-04-25T10:27:46.441 INFO:teuthology.orchestra.run.smithi203.stdout: Installing : ceph-prometheus-alerts-2:19.0.0-3291.g994e2f20.e 124/156 2024-04-25T10:27:46.502 INFO:teuthology.orchestra.run.smithi203.stdout: Installing : ceph-grafana-dashboards-2:19.0.0-3291.g994e2f20. 125/156 2024-04-25T10:27:46.532 INFO:teuthology.orchestra.run.smithi203.stdout: Installing : ceph-mgr-cephadm-2:19.0.0-3291.g994e2f20.el9.noa 126/156 2024-04-25T10:27:47.574 INFO:teuthology.orchestra.run.smithi203.stdout: Running scriptlet: ceph-mgr-cephadm-2:19.0.0-3291.g994e2f20.el9.noa 126/156 2024-04-25T10:27:47.608 INFO:teuthology.orchestra.run.smithi203.stdout: Installing : ceph-mgr-dashboard-2:19.0.0-3291.g994e2f20.el9.n 127/156 2024-04-25T10:27:48.589 INFO:teuthology.orchestra.run.smithi203.stdout: Running scriptlet: ceph-mgr-dashboard-2:19.0.0-3291.g994e2f20.el9.n 127/156 2024-04-25T10:27:48.617 INFO:teuthology.orchestra.run.smithi203.stdout: Installing : ceph-mgr-diskprediction-local-2:19.0.0-3291.g994 128/156 2024-04-25T10:27:48.721 INFO:teuthology.orchestra.run.smithi203.stdout: Running scriptlet: ceph-mgr-diskprediction-local-2:19.0.0-3291.g994 128/156 2024-04-25T10:27:48.893 INFO:teuthology.orchestra.run.smithi203.stdout: Installing : ceph-mgr-modules-core-2:19.0.0-3291.g994e2f20.el 129/156 2024-04-25T10:27:48.920 INFO:teuthology.orchestra.run.smithi203.stdout: Installing : ceph-mgr-2:19.0.0-3291.g994e2f20.el9.x86_64 130/156 2024-04-25T10:27:48.943 INFO:teuthology.orchestra.run.smithi203.stdout: Running scriptlet: ceph-mgr-2:19.0.0-3291.g994e2f20.el9.x86_64 130/156 2024-04-25T10:27:48.943 INFO:teuthology.orchestra.run.smithi203.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-04-25T10:27:48.943 INFO:teuthology.orchestra.run.smithi203.stdout:Invalid unit name "ceph-mgr@*.service" escaped as "ceph-mgr@\x2a.service". 2024-04-25T10:27:48.943 INFO:teuthology.orchestra.run.smithi203.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-mgr.target → /usr/lib/systemd/system/ceph-mgr.target. 2024-04-25T10:27:48.943 INFO:teuthology.orchestra.run.smithi203.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-mgr.target → /usr/lib/systemd/system/ceph-mgr.target. 2024-04-25T10:27:48.943 INFO:teuthology.orchestra.run.smithi203.stdout: 2024-04-25T10:27:48.987 INFO:teuthology.orchestra.run.smithi203.stdout: Installing : ceph-mgr-rook-2:19.0.0-3291.g994e2f20.el9.noarch 131/156 2024-04-25T10:27:48.998 INFO:teuthology.orchestra.run.smithi203.stdout: Running scriptlet: ceph-mgr-rook-2:19.0.0-3291.g994e2f20.el9.noarch 131/156 2024-04-25T10:27:49.101 INFO:teuthology.orchestra.run.smithi203.stdout: Installing : ceph-2:19.0.0-3291.g994e2f20.el9.x86_64 132/156 2024-04-25T10:27:49.152 INFO:teuthology.orchestra.run.smithi203.stdout: Installing : ceph-fuse-2:19.0.0-3291.g994e2f20.el9.x86_64 133/156 2024-04-25T10:27:50.085 INFO:teuthology.orchestra.run.smithi203.stdout: Installing : libtool-2.4.6-45.el9.x86_64 134/156 2024-04-25T10:27:50.115 INFO:teuthology.orchestra.run.smithi203.stdout: Installing : ceph-radosgw-2:19.0.0-3291.g994e2f20.el9.x86_64 135/156 2024-04-25T10:27:50.138 INFO:teuthology.orchestra.run.smithi203.stdout: Running scriptlet: ceph-radosgw-2:19.0.0-3291.g994e2f20.el9.x86_64 135/156 2024-04-25T10:27:50.138 INFO:teuthology.orchestra.run.smithi203.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-04-25T10:27:50.138 INFO:teuthology.orchestra.run.smithi203.stdout:Invalid unit name "ceph-radosgw@*.service" escaped as "ceph-radosgw@\x2a.service". 2024-04-25T10:27:50.138 INFO:teuthology.orchestra.run.smithi203.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-radosgw.target → /usr/lib/systemd/system/ceph-radosgw.target. 2024-04-25T10:27:50.138 INFO:teuthology.orchestra.run.smithi203.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-radosgw.target → /usr/lib/systemd/system/ceph-radosgw.target. 2024-04-25T10:27:50.138 INFO:teuthology.orchestra.run.smithi203.stdout: 2024-04-25T10:27:50.182 INFO:teuthology.orchestra.run.smithi203.stdout: Installing : ceph-immutable-object-cache-2:19.0.0-3291.g994e2 136/156 2024-04-25T10:27:50.204 INFO:teuthology.orchestra.run.smithi203.stdout: Running scriptlet: ceph-immutable-object-cache-2:19.0.0-3291.g994e2 136/156 2024-04-25T10:27:50.204 INFO:teuthology.orchestra.run.smithi203.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-04-25T10:27:50.204 INFO:teuthology.orchestra.run.smithi203.stdout:Invalid unit name "ceph-immutable-object-cache@*.service" escaped as "ceph-immutable-object-cache@\x2a.service". 2024-04-25T10:27:50.204 INFO:teuthology.orchestra.run.smithi203.stdout: 2024-04-25T10:27:50.249 INFO:teuthology.orchestra.run.smithi203.stdout: Installing : cephfs-mirror-2:19.0.0-3291.g994e2f20.el9.x86_64 137/156 2024-04-25T10:27:50.272 INFO:teuthology.orchestra.run.smithi203.stdout: Running scriptlet: cephfs-mirror-2:19.0.0-3291.g994e2f20.el9.x86_64 137/156 2024-04-25T10:27:50.272 INFO:teuthology.orchestra.run.smithi203.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-04-25T10:27:50.272 INFO:teuthology.orchestra.run.smithi203.stdout:Invalid unit name "cephfs-mirror@*.service" escaped as "cephfs-mirror@\x2a.service". 2024-04-25T10:27:50.272 INFO:teuthology.orchestra.run.smithi203.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/cephfs-mirror.target → /usr/lib/systemd/system/cephfs-mirror.target. 2024-04-25T10:27:50.272 INFO:teuthology.orchestra.run.smithi203.stdout:Created symlink /etc/systemd/system/ceph.target.wants/cephfs-mirror.target → /usr/lib/systemd/system/cephfs-mirror.target. 2024-04-25T10:27:50.272 INFO:teuthology.orchestra.run.smithi203.stdout: 2024-04-25T10:27:50.485 INFO:teuthology.orchestra.run.smithi037.stdout: Installing : ceph-test-2:19.0.0-3291.g994e2f20.el9.x86_64 139/156 2024-04-25T10:27:50.526 INFO:teuthology.orchestra.run.smithi037.stdout: Installing : rbd-fuse-2:19.0.0-3291.g994e2f20.el9.x86_64 140/156 2024-04-25T10:27:50.567 INFO:teuthology.orchestra.run.smithi037.stdout: Installing : rbd-nbd-2:19.0.0-3291.g994e2f20.el9.x86_64 141/156 2024-04-25T10:27:50.598 INFO:teuthology.orchestra.run.smithi037.stdout: Installing : xfsprogs-devel-6.3.0-1.el9.x86_64 142/156 2024-04-25T10:27:50.600 INFO:teuthology.orchestra.run.smithi203.stdout: Installing : rbd-mirror-2:19.0.0-3291.g994e2f20.el9.x86_64 138/156 2024-04-25T10:27:50.623 INFO:teuthology.orchestra.run.smithi203.stdout: Running scriptlet: rbd-mirror-2:19.0.0-3291.g994e2f20.el9.x86_64 138/156 2024-04-25T10:27:50.623 INFO:teuthology.orchestra.run.smithi203.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-04-25T10:27:50.623 INFO:teuthology.orchestra.run.smithi203.stdout:Invalid unit name "ceph-rbd-mirror@*.service" escaped as "ceph-rbd-mirror@\x2a.service". 2024-04-25T10:27:50.623 INFO:teuthology.orchestra.run.smithi203.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-rbd-mirror.target → /usr/lib/systemd/system/ceph-rbd-mirror.target. 2024-04-25T10:27:50.623 INFO:teuthology.orchestra.run.smithi203.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-rbd-mirror.target → /usr/lib/systemd/system/ceph-rbd-mirror.target. 2024-04-25T10:27:50.624 INFO:teuthology.orchestra.run.smithi203.stdout: 2024-04-25T10:27:50.625 INFO:teuthology.orchestra.run.smithi037.stdout: Installing : elfutils-libelf-devel-0.190-2.el9.x86_64 143/156 2024-04-25T10:27:50.828 INFO:teuthology.orchestra.run.smithi037.stdout: Installing : cephfs-shell-2:19.0.0-3291.g994e2f20.el9.x86_64 144/156 2024-04-25T10:27:50.871 INFO:teuthology.orchestra.run.smithi037.stdout: Installing : postgresql-contrib-13.14-1.el9.x86_64 145/156 2024-04-25T10:27:50.901 INFO:teuthology.orchestra.run.smithi037.stdout: Installing : libcephfs-devel-2:19.0.0-3291.g994e2f20.el9.x86_ 146/156 2024-04-25T10:27:51.516 INFO:teuthology.orchestra.run.smithi037.stdout: Running scriptlet: postgresql-server-13.14-1.el9.x86_64 147/156 2024-04-25T10:27:51.557 INFO:teuthology.orchestra.run.smithi037.stdout: Installing : postgresql-server-13.14-1.el9.x86_64 147/156 2024-04-25T10:27:51.578 INFO:teuthology.orchestra.run.smithi037.stdout: Running scriptlet: postgresql-server-13.14-1.el9.x86_64 147/156 2024-04-25T10:27:52.495 INFO:teuthology.orchestra.run.smithi037.stdout: Installing : cephfs-top-2:19.0.0-3291.g994e2f20.el9.noarch 148/156 2024-04-25T10:27:52.587 INFO:teuthology.orchestra.run.smithi037.stdout: Installing : openssl-devel-1:3.2.1-1.el9.x86_64 149/156 2024-04-25T10:27:52.620 INFO:teuthology.orchestra.run.smithi037.stdout: Installing : libaio-devel-0.3.111-13.el9.x86_64 150/156 2024-04-25T10:27:52.714 INFO:teuthology.orchestra.run.smithi037.stdout: Installing : flex-2.6.4-9.el9.x86_64 151/156 2024-04-25T10:27:52.758 INFO:teuthology.orchestra.run.smithi037.stdout: Installing : bison-3.7.4-5.el9.x86_64 152/156 2024-04-25T10:27:52.758 INFO:teuthology.orchestra.run.smithi037.stdout: Cleanup : librbd1-2:16.2.4-5.el9.x86_64 153/156 2024-04-25T10:27:52.842 INFO:teuthology.orchestra.run.smithi037.stdout: Running scriptlet: librbd1-2:16.2.4-5.el9.x86_64 153/156 2024-04-25T10:27:52.875 INFO:teuthology.orchestra.run.smithi037.stdout: Cleanup : openssl-1:3.0.7-27.el9.x86_64 154/156 2024-04-25T10:27:52.875 INFO:teuthology.orchestra.run.smithi037.stdout: Cleanup : librados2-2:16.2.4-5.el9.x86_64 155/156 2024-04-25T10:27:52.952 INFO:teuthology.orchestra.run.smithi037.stdout: Running scriptlet: librados2-2:16.2.4-5.el9.x86_64 155/156 2024-04-25T10:27:53.026 INFO:teuthology.orchestra.run.smithi037.stdout: Cleanup : openssl-libs-1:3.0.7-27.el9.x86_64 156/156 2024-04-25T10:27:54.896 INFO:teuthology.orchestra.run.smithi203.stdout: Installing : ceph-test-2:19.0.0-3291.g994e2f20.el9.x86_64 139/156 2024-04-25T10:27:54.938 INFO:teuthology.orchestra.run.smithi203.stdout: Installing : rbd-fuse-2:19.0.0-3291.g994e2f20.el9.x86_64 140/156 2024-04-25T10:27:54.979 INFO:teuthology.orchestra.run.smithi203.stdout: Installing : rbd-nbd-2:19.0.0-3291.g994e2f20.el9.x86_64 141/156 2024-04-25T10:27:55.010 INFO:teuthology.orchestra.run.smithi203.stdout: Installing : xfsprogs-devel-6.3.0-1.el9.x86_64 142/156 2024-04-25T10:27:55.038 INFO:teuthology.orchestra.run.smithi203.stdout: Installing : elfutils-libelf-devel-0.190-2.el9.x86_64 143/156 2024-04-25T10:27:55.225 INFO:teuthology.orchestra.run.smithi203.stdout: Installing : cephfs-shell-2:19.0.0-3291.g994e2f20.el9.x86_64 144/156 2024-04-25T10:27:55.277 INFO:teuthology.orchestra.run.smithi203.stdout: Installing : postgresql-contrib-13.14-1.el9.x86_64 145/156 2024-04-25T10:27:55.308 INFO:teuthology.orchestra.run.smithi203.stdout: Installing : libcephfs-devel-2:19.0.0-3291.g994e2f20.el9.x86_ 146/156 2024-04-25T10:27:55.927 INFO:teuthology.orchestra.run.smithi203.stdout: Running scriptlet: postgresql-server-13.14-1.el9.x86_64 147/156 2024-04-25T10:27:55.972 INFO:teuthology.orchestra.run.smithi203.stdout: Installing : postgresql-server-13.14-1.el9.x86_64 147/156 2024-04-25T10:27:55.992 INFO:teuthology.orchestra.run.smithi203.stdout: Running scriptlet: postgresql-server-13.14-1.el9.x86_64 147/156 2024-04-25T10:27:56.834 INFO:teuthology.orchestra.run.smithi203.stdout: Installing : cephfs-top-2:19.0.0-3291.g994e2f20.el9.noarch 148/156 2024-04-25T10:27:56.918 INFO:teuthology.orchestra.run.smithi203.stdout: Installing : openssl-devel-1:3.2.1-1.el9.x86_64 149/156 2024-04-25T10:27:56.959 INFO:teuthology.orchestra.run.smithi203.stdout: Installing : libaio-devel-0.3.111-13.el9.x86_64 150/156 2024-04-25T10:27:57.050 INFO:teuthology.orchestra.run.smithi203.stdout: Installing : flex-2.6.4-9.el9.x86_64 151/156 2024-04-25T10:27:57.089 INFO:teuthology.orchestra.run.smithi203.stdout: Installing : bison-3.7.4-5.el9.x86_64 152/156 2024-04-25T10:27:57.089 INFO:teuthology.orchestra.run.smithi203.stdout: Cleanup : librbd1-2:16.2.4-5.el9.x86_64 153/156 2024-04-25T10:27:57.165 INFO:teuthology.orchestra.run.smithi203.stdout: Running scriptlet: librbd1-2:16.2.4-5.el9.x86_64 153/156 2024-04-25T10:27:57.197 INFO:teuthology.orchestra.run.smithi203.stdout: Cleanup : openssl-1:3.0.7-27.el9.x86_64 154/156 2024-04-25T10:27:57.198 INFO:teuthology.orchestra.run.smithi203.stdout: Cleanup : librados2-2:16.2.4-5.el9.x86_64 155/156 2024-04-25T10:27:57.285 INFO:teuthology.orchestra.run.smithi203.stdout: Running scriptlet: librados2-2:16.2.4-5.el9.x86_64 155/156 2024-04-25T10:27:57.354 INFO:teuthology.orchestra.run.smithi203.stdout: Cleanup : openssl-libs-1:3.0.7-27.el9.x86_64 156/156 2024-04-25T10:27:58.750 INFO:teuthology.orchestra.run.smithi037.stdout: Running scriptlet: openssl-libs-1:3.0.7-27.el9.x86_64 156/156 2024-04-25T10:27:58.751 INFO:teuthology.orchestra.run.smithi037.stdout: Verifying : ceph-2:19.0.0-3291.g994e2f20.el9.x86_64 1/156 2024-04-25T10:27:58.751 INFO:teuthology.orchestra.run.smithi037.stdout: Verifying : ceph-base-2:19.0.0-3291.g994e2f20.el9.x86_64 2/156 2024-04-25T10:27:58.751 INFO:teuthology.orchestra.run.smithi037.stdout: Verifying : ceph-common-2:19.0.0-3291.g994e2f20.el9.x86_64 3/156 2024-04-25T10:27:58.751 INFO:teuthology.orchestra.run.smithi037.stdout: Verifying : ceph-fuse-2:19.0.0-3291.g994e2f20.el9.x86_64 4/156 2024-04-25T10:27:58.751 INFO:teuthology.orchestra.run.smithi037.stdout: Verifying : ceph-immutable-object-cache-2:19.0.0-3291.g994e2 5/156 2024-04-25T10:27:58.751 INFO:teuthology.orchestra.run.smithi037.stdout: Verifying : ceph-mds-2:19.0.0-3291.g994e2f20.el9.x86_64 6/156 2024-04-25T10:27:58.751 INFO:teuthology.orchestra.run.smithi037.stdout: Verifying : ceph-mgr-2:19.0.0-3291.g994e2f20.el9.x86_64 7/156 2024-04-25T10:27:58.751 INFO:teuthology.orchestra.run.smithi037.stdout: Verifying : ceph-mon-2:19.0.0-3291.g994e2f20.el9.x86_64 8/156 2024-04-25T10:27:58.751 INFO:teuthology.orchestra.run.smithi037.stdout: Verifying : ceph-osd-2:19.0.0-3291.g994e2f20.el9.x86_64 9/156 2024-04-25T10:27:58.751 INFO:teuthology.orchestra.run.smithi037.stdout: Verifying : ceph-radosgw-2:19.0.0-3291.g994e2f20.el9.x86_64 10/156 2024-04-25T10:27:58.751 INFO:teuthology.orchestra.run.smithi037.stdout: Verifying : ceph-selinux-2:19.0.0-3291.g994e2f20.el9.x86_64 11/156 2024-04-25T10:27:58.751 INFO:teuthology.orchestra.run.smithi037.stdout: Verifying : ceph-test-2:19.0.0-3291.g994e2f20.el9.x86_64 12/156 2024-04-25T10:27:58.751 INFO:teuthology.orchestra.run.smithi037.stdout: Verifying : cephfs-mirror-2:19.0.0-3291.g994e2f20.el9.x86_64 13/156 2024-04-25T10:27:58.752 INFO:teuthology.orchestra.run.smithi037.stdout: Verifying : cephfs-shell-2:19.0.0-3291.g994e2f20.el9.x86_64 14/156 2024-04-25T10:27:58.752 INFO:teuthology.orchestra.run.smithi037.stdout: Verifying : libcephfs-devel-2:19.0.0-3291.g994e2f20.el9.x86_ 15/156 2024-04-25T10:27:58.752 INFO:teuthology.orchestra.run.smithi037.stdout: Verifying : libcephfs2-2:19.0.0-3291.g994e2f20.el9.x86_64 16/156 2024-04-25T10:27:58.752 INFO:teuthology.orchestra.run.smithi037.stdout: Verifying : libcephsqlite-2:19.0.0-3291.g994e2f20.el9.x86_64 17/156 2024-04-25T10:27:58.752 INFO:teuthology.orchestra.run.smithi037.stdout: Verifying : librados-devel-2:19.0.0-3291.g994e2f20.el9.x86_6 18/156 2024-04-25T10:27:58.752 INFO:teuthology.orchestra.run.smithi037.stdout: Verifying : libradosstriper1-2:19.0.0-3291.g994e2f20.el9.x86 19/156 2024-04-25T10:27:58.752 INFO:teuthology.orchestra.run.smithi037.stdout: Verifying : librgw2-2:19.0.0-3291.g994e2f20.el9.x86_64 20/156 2024-04-25T10:27:58.752 INFO:teuthology.orchestra.run.smithi037.stdout: Verifying : python3-ceph-argparse-2:19.0.0-3291.g994e2f20.el 21/156 2024-04-25T10:27:58.752 INFO:teuthology.orchestra.run.smithi037.stdout: Verifying : python3-ceph-common-2:19.0.0-3291.g994e2f20.el9. 22/156 2024-04-25T10:27:58.752 INFO:teuthology.orchestra.run.smithi037.stdout: Verifying : python3-cephfs-2:19.0.0-3291.g994e2f20.el9.x86_6 23/156 2024-04-25T10:27:58.752 INFO:teuthology.orchestra.run.smithi037.stdout: Verifying : python3-rados-2:19.0.0-3291.g994e2f20.el9.x86_64 24/156 2024-04-25T10:27:58.752 INFO:teuthology.orchestra.run.smithi037.stdout: Verifying : python3-rbd-2:19.0.0-3291.g994e2f20.el9.x86_64 25/156 2024-04-25T10:27:58.753 INFO:teuthology.orchestra.run.smithi037.stdout: Verifying : python3-rgw-2:19.0.0-3291.g994e2f20.el9.x86_64 26/156 2024-04-25T10:27:58.753 INFO:teuthology.orchestra.run.smithi037.stdout: Verifying : rbd-fuse-2:19.0.0-3291.g994e2f20.el9.x86_64 27/156 2024-04-25T10:27:58.753 INFO:teuthology.orchestra.run.smithi037.stdout: Verifying : rbd-mirror-2:19.0.0-3291.g994e2f20.el9.x86_64 28/156 2024-04-25T10:27:58.753 INFO:teuthology.orchestra.run.smithi037.stdout: Verifying : rbd-nbd-2:19.0.0-3291.g994e2f20.el9.x86_64 29/156 2024-04-25T10:27:58.753 INFO:teuthology.orchestra.run.smithi037.stdout: Verifying : ceph-grafana-dashboards-2:19.0.0-3291.g994e2f20. 30/156 2024-04-25T10:27:58.765 INFO:teuthology.orchestra.run.smithi037.stdout: Verifying : ceph-mgr-cephadm-2:19.0.0-3291.g994e2f20.el9.noa 31/156 2024-04-25T10:27:58.765 INFO:teuthology.orchestra.run.smithi037.stdout: Verifying : ceph-mgr-dashboard-2:19.0.0-3291.g994e2f20.el9.n 32/156 2024-04-25T10:27:58.765 INFO:teuthology.orchestra.run.smithi037.stdout: Verifying : ceph-mgr-diskprediction-local-2:19.0.0-3291.g994 33/156 2024-04-25T10:27:58.765 INFO:teuthology.orchestra.run.smithi037.stdout: Verifying : ceph-mgr-modules-core-2:19.0.0-3291.g994e2f20.el 34/156 2024-04-25T10:27:58.765 INFO:teuthology.orchestra.run.smithi037.stdout: Verifying : ceph-mgr-rook-2:19.0.0-3291.g994e2f20.el9.noarch 35/156 2024-04-25T10:27:58.765 INFO:teuthology.orchestra.run.smithi037.stdout: Verifying : ceph-prometheus-alerts-2:19.0.0-3291.g994e2f20.e 36/156 2024-04-25T10:27:58.765 INFO:teuthology.orchestra.run.smithi037.stdout: Verifying : ceph-volume-2:19.0.0-3291.g994e2f20.el9.noarch 37/156 2024-04-25T10:27:58.765 INFO:teuthology.orchestra.run.smithi037.stdout: Verifying : cephadm-2:19.0.0-3291.g994e2f20.el9.noarch 38/156 2024-04-25T10:27:58.765 INFO:teuthology.orchestra.run.smithi037.stdout: Verifying : cephfs-top-2:19.0.0-3291.g994e2f20.el9.noarch 39/156 2024-04-25T10:27:58.765 INFO:teuthology.orchestra.run.smithi037.stdout: Verifying : fuse-2.9.9-15.el9.x86_64 40/156 2024-04-25T10:27:58.766 INFO:teuthology.orchestra.run.smithi037.stdout: Verifying : libgfortran-11.4.1-3.el9.x86_64 41/156 2024-04-25T10:27:58.766 INFO:teuthology.orchestra.run.smithi037.stdout: Verifying : libquadmath-11.4.1-3.el9.x86_64 42/156 2024-04-25T10:27:58.766 INFO:teuthology.orchestra.run.smithi037.stdout: Verifying : python3-cffi-1.14.5-5.el9.x86_64 43/156 2024-04-25T10:27:58.766 INFO:teuthology.orchestra.run.smithi037.stdout: Verifying : python3-cryptography-36.0.1-4.el9.x86_64 44/156 2024-04-25T10:27:58.766 INFO:teuthology.orchestra.run.smithi037.stdout: Verifying : python3-ply-3.11-14.el9.noarch 45/156 2024-04-25T10:27:58.766 INFO:teuthology.orchestra.run.smithi037.stdout: Verifying : python3-pycparser-2.20-6.el9.noarch 46/156 2024-04-25T10:27:58.766 INFO:teuthology.orchestra.run.smithi037.stdout: Verifying : python3-requests-2.25.1-8.el9.noarch 47/156 2024-04-25T10:27:58.766 INFO:teuthology.orchestra.run.smithi037.stdout: Verifying : python3-urllib3-1.26.5-5.el9.noarch 48/156 2024-04-25T10:27:58.766 INFO:teuthology.orchestra.run.smithi037.stdout: Verifying : automake-1.16.2-8.el9.noarch 49/156 2024-04-25T10:27:58.766 INFO:teuthology.orchestra.run.smithi037.stdout: Verifying : bison-3.7.4-5.el9.x86_64 50/156 2024-04-25T10:27:58.766 INFO:teuthology.orchestra.run.smithi037.stdout: Verifying : boost-program-options-1.75.0-8.el9.x86_64 51/156 2024-04-25T10:27:58.766 INFO:teuthology.orchestra.run.smithi037.stdout: Verifying : elfutils-libelf-devel-0.190-2.el9.x86_64 52/156 2024-04-25T10:27:58.766 INFO:teuthology.orchestra.run.smithi037.stdout: Verifying : flex-2.6.4-9.el9.x86_64 53/156 2024-04-25T10:27:58.767 INFO:teuthology.orchestra.run.smithi037.stdout: Verifying : flexiblas-3.0.4-8.el9.x86_64 54/156 2024-04-25T10:27:58.767 INFO:teuthology.orchestra.run.smithi037.stdout: Verifying : flexiblas-netlib-3.0.4-8.el9.x86_64 55/156 2024-04-25T10:27:58.767 INFO:teuthology.orchestra.run.smithi037.stdout: Verifying : flexiblas-openblas-openmp-3.0.4-8.el9.x86_64 56/156 2024-04-25T10:27:58.767 INFO:teuthology.orchestra.run.smithi037.stdout: Verifying : libaio-devel-0.3.111-13.el9.x86_64 57/156 2024-04-25T10:27:58.767 INFO:teuthology.orchestra.run.smithi037.stdout: Verifying : libpmemobj-1.12.1-1.el9.x86_64 58/156 2024-04-25T10:27:58.767 INFO:teuthology.orchestra.run.smithi037.stdout: Verifying : librabbitmq-0.11.0-7.el9.x86_64 59/156 2024-04-25T10:27:58.767 INFO:teuthology.orchestra.run.smithi037.stdout: Verifying : librdkafka-1.6.1-102.el9.x86_64 60/156 2024-04-25T10:27:58.767 INFO:teuthology.orchestra.run.smithi037.stdout: Verifying : libtool-2.4.6-45.el9.x86_64 61/156 2024-04-25T10:27:58.767 INFO:teuthology.orchestra.run.smithi037.stdout: Verifying : libuuid-devel-2.37.4-18.el9.x86_64 62/156 2024-04-25T10:27:58.767 INFO:teuthology.orchestra.run.smithi037.stdout: Verifying : libzstd-devel-1.5.1-2.el9.x86_64 63/156 2024-04-25T10:27:58.767 INFO:teuthology.orchestra.run.smithi037.stdout: Verifying : lttng-ust-2.12.0-6.el9.x86_64 64/156 2024-04-25T10:27:58.767 INFO:teuthology.orchestra.run.smithi037.stdout: Verifying : lua-5.4.4-4.el9.x86_64 65/156 2024-04-25T10:27:58.767 INFO:teuthology.orchestra.run.smithi037.stdout: Verifying : openblas-0.3.21-2.el9.x86_64 66/156 2024-04-25T10:27:58.767 INFO:teuthology.orchestra.run.smithi037.stdout: Verifying : openblas-openmp-0.3.21-2.el9.x86_64 67/156 2024-04-25T10:27:58.768 INFO:teuthology.orchestra.run.smithi037.stdout: Verifying : openssl-devel-1:3.2.1-1.el9.x86_64 68/156 2024-04-25T10:27:58.768 INFO:teuthology.orchestra.run.smithi037.stdout: Verifying : postgresql-13.14-1.el9.x86_64 69/156 2024-04-25T10:27:58.768 INFO:teuthology.orchestra.run.smithi037.stdout: Verifying : postgresql-contrib-13.14-1.el9.x86_64 70/156 2024-04-25T10:27:58.768 INFO:teuthology.orchestra.run.smithi037.stdout: Verifying : postgresql-private-libs-13.14-1.el9.x86_64 71/156 2024-04-25T10:27:58.768 INFO:teuthology.orchestra.run.smithi037.stdout: Verifying : postgresql-server-13.14-1.el9.x86_64 72/156 2024-04-25T10:27:58.768 INFO:teuthology.orchestra.run.smithi037.stdout: Verifying : protobuf-3.14.0-13.el9.x86_64 73/156 2024-04-25T10:27:58.768 INFO:teuthology.orchestra.run.smithi037.stdout: Verifying : python3-attrs-20.3.0-7.el9.noarch 74/156 2024-04-25T10:27:58.768 INFO:teuthology.orchestra.run.smithi037.stdout: Verifying : python3-babel-2.9.1-2.el9.noarch 75/156 2024-04-25T10:27:58.768 INFO:teuthology.orchestra.run.smithi037.stdout: Verifying : python3-devel-3.9.18-3.el9.x86_64 76/156 2024-04-25T10:27:58.768 INFO:teuthology.orchestra.run.smithi037.stdout: Verifying : python3-jinja2-2.11.3-5.el9.noarch 77/156 2024-04-25T10:27:58.768 INFO:teuthology.orchestra.run.smithi037.stdout: Verifying : python3-jsonpatch-1.21-16.el9.noarch 78/156 2024-04-25T10:27:58.768 INFO:teuthology.orchestra.run.smithi037.stdout: Verifying : python3-jsonpointer-2.0-4.el9.noarch 79/156 2024-04-25T10:27:58.768 INFO:teuthology.orchestra.run.smithi037.stdout: Verifying : python3-mako-1.1.4-6.el9.noarch 80/156 2024-04-25T10:27:58.769 INFO:teuthology.orchestra.run.smithi037.stdout: Verifying : python3-markupsafe-1.1.1-12.el9.x86_64 81/156 2024-04-25T10:27:58.769 INFO:teuthology.orchestra.run.smithi037.stdout: Verifying : python3-numpy-1:1.20.1-5.el9.x86_64 82/156 2024-04-25T10:27:58.769 INFO:teuthology.orchestra.run.smithi037.stdout: Verifying : python3-numpy-f2py-1:1.20.1-5.el9.x86_64 83/156 2024-04-25T10:27:58.769 INFO:teuthology.orchestra.run.smithi037.stdout: Verifying : python3-oauthlib-3.1.1-5.el9.noarch 84/156 2024-04-25T10:27:58.769 INFO:teuthology.orchestra.run.smithi037.stdout: Verifying : python3-packaging-20.9-5.el9.noarch 85/156 2024-04-25T10:27:58.769 INFO:teuthology.orchestra.run.smithi037.stdout: Verifying : python3-prettytable-0.7.2-27.el9.noarch 86/156 2024-04-25T10:27:58.769 INFO:teuthology.orchestra.run.smithi037.stdout: Verifying : python3-protobuf-3.14.0-13.el9.noarch 87/156 2024-04-25T10:27:58.769 INFO:teuthology.orchestra.run.smithi037.stdout: Verifying : python3-pyasn1-0.4.8-6.el9.noarch 88/156 2024-04-25T10:27:58.769 INFO:teuthology.orchestra.run.smithi037.stdout: Verifying : python3-pyasn1-modules-0.4.8-6.el9.noarch 89/156 2024-04-25T10:27:58.769 INFO:teuthology.orchestra.run.smithi037.stdout: Verifying : python3-pytz-2021.1-5.el9.noarch 90/156 2024-04-25T10:27:58.769 INFO:teuthology.orchestra.run.smithi037.stdout: Verifying : python3-requests-oauthlib-1.3.0-12.el9.noarch 91/156 2024-04-25T10:27:58.769 INFO:teuthology.orchestra.run.smithi037.stdout: Verifying : python3-scipy-1.6.2-8.el9.x86_64 92/156 2024-04-25T10:27:58.769 INFO:teuthology.orchestra.run.smithi037.stdout: Verifying : python3-toml-0.10.2-6.el9.noarch 93/156 2024-04-25T10:27:58.770 INFO:teuthology.orchestra.run.smithi037.stdout: Verifying : python3-wcwidth-0.2.5-8.el9.noarch 94/156 2024-04-25T10:27:58.770 INFO:teuthology.orchestra.run.smithi037.stdout: Verifying : qatlib-24.02.0-1.el9.x86_64 95/156 2024-04-25T10:27:58.770 INFO:teuthology.orchestra.run.smithi037.stdout: Verifying : qatlib-service-24.02.0-1.el9.x86_64 96/156 2024-04-25T10:27:58.770 INFO:teuthology.orchestra.run.smithi037.stdout: Verifying : qatzip-libs-1.2.0-1.el9.x86_64 97/156 2024-04-25T10:27:58.770 INFO:teuthology.orchestra.run.smithi037.stdout: Verifying : socat-1.7.4.1-5.el9.x86_64 98/156 2024-04-25T10:27:58.770 INFO:teuthology.orchestra.run.smithi037.stdout: Verifying : uuid-1.6.2-55.el9.x86_64 99/156 2024-04-25T10:27:58.770 INFO:teuthology.orchestra.run.smithi037.stdout: Verifying : xfsprogs-devel-6.3.0-1.el9.x86_64 100/156 2024-04-25T10:27:58.770 INFO:teuthology.orchestra.run.smithi037.stdout: Verifying : xmlstarlet-1.6.1-20.el9.x86_64 101/156 2024-04-25T10:27:58.770 INFO:teuthology.orchestra.run.smithi037.stdout: Verifying : zlib-devel-1.2.11-41.el9.x86_64 102/156 2024-04-25T10:27:58.770 INFO:teuthology.orchestra.run.smithi037.stdout: Verifying : lua-devel-5.4.4-4.el9.x86_64 103/156 2024-04-25T10:27:58.770 INFO:teuthology.orchestra.run.smithi037.stdout: Verifying : protobuf-compiler-3.14.0-13.el9.x86_64 104/156 2024-04-25T10:27:58.770 INFO:teuthology.orchestra.run.smithi037.stdout: Verifying : abseil-cpp-20211102.0-3.el9.x86_64 105/156 2024-04-25T10:27:58.771 INFO:teuthology.orchestra.run.smithi037.stdout: Verifying : gperftools-libs-2.9.1-2.el9.x86_64 106/156 2024-04-25T10:27:58.771 INFO:teuthology.orchestra.run.smithi037.stdout: Verifying : grpc-data-1.46.7-10.el9.noarch 107/156 2024-04-25T10:27:58.771 INFO:teuthology.orchestra.run.smithi037.stdout: Verifying : libarrow-9.0.0-10.el9.x86_64 108/156 2024-04-25T10:27:58.771 INFO:teuthology.orchestra.run.smithi037.stdout: Verifying : libarrow-doc-9.0.0-10.el9.noarch 109/156 2024-04-25T10:27:58.771 INFO:teuthology.orchestra.run.smithi037.stdout: Verifying : liboath-2.6.7-2.el9.x86_64 110/156 2024-04-25T10:27:58.771 INFO:teuthology.orchestra.run.smithi037.stdout: Verifying : libunwind-1.6.2-1.el9.x86_64 111/156 2024-04-25T10:27:58.771 INFO:teuthology.orchestra.run.smithi037.stdout: Verifying : luarocks-3.9.2-1.el9.noarch 112/156 2024-04-25T10:27:58.771 INFO:teuthology.orchestra.run.smithi037.stdout: Verifying : parquet-libs-9.0.0-10.el9.x86_64 113/156 2024-04-25T10:27:58.771 INFO:teuthology.orchestra.run.smithi037.stdout: Verifying : python3-asyncssh-2.13.2-5.el9.noarch 114/156 2024-04-25T10:27:58.771 INFO:teuthology.orchestra.run.smithi037.stdout: Verifying : python3-bcrypt-3.2.2-1.el9.x86_64 115/156 2024-04-25T10:27:58.771 INFO:teuthology.orchestra.run.smithi037.stdout: Verifying : python3-cachetools-4.2.4-1.el9.noarch 116/156 2024-04-25T10:27:58.771 INFO:teuthology.orchestra.run.smithi037.stdout: Verifying : python3-certifi-2021.10.8-2.el9.noarch 117/156 2024-04-25T10:27:58.771 INFO:teuthology.orchestra.run.smithi037.stdout: Verifying : python3-cheroot-8.6.0-4.el9.noarch 118/156 2024-04-25T10:27:58.772 INFO:teuthology.orchestra.run.smithi037.stdout: Verifying : python3-cherrypy-18.6.1-2.el9.noarch 119/156 2024-04-25T10:27:58.772 INFO:teuthology.orchestra.run.smithi037.stdout: Verifying : python3-cmd2-2.4.2-2.el9.noarch 120/156 2024-04-25T10:27:58.772 INFO:teuthology.orchestra.run.smithi037.stdout: Verifying : python3-colorama-0.4.4-12.el9.noarch 121/156 2024-04-25T10:27:58.772 INFO:teuthology.orchestra.run.smithi037.stdout: Verifying : python3-google-auth-1:2.29.0-1.el9.noarch 122/156 2024-04-25T10:27:58.772 INFO:teuthology.orchestra.run.smithi037.stdout: Verifying : python3-grpcio-1.46.7-10.el9.x86_64 123/156 2024-04-25T10:27:58.772 INFO:teuthology.orchestra.run.smithi037.stdout: Verifying : python3-grpcio-tools-1.46.7-10.el9.x86_64 124/156 2024-04-25T10:27:58.772 INFO:teuthology.orchestra.run.smithi037.stdout: Verifying : python3-jaraco-8.2.1-3.el9.noarch 125/156 2024-04-25T10:27:58.772 INFO:teuthology.orchestra.run.smithi037.stdout: Verifying : python3-jaraco-classes-3.2.1-5.el9.noarch 126/156 2024-04-25T10:27:58.772 INFO:teuthology.orchestra.run.smithi037.stdout: Verifying : python3-jaraco-collections-3.0.0-8.el9.noarch 127/156 2024-04-25T10:27:58.772 INFO:teuthology.orchestra.run.smithi037.stdout: Verifying : python3-jaraco-functools-3.5.0-2.el9.noarch 128/156 2024-04-25T10:27:58.772 INFO:teuthology.orchestra.run.smithi037.stdout: Verifying : python3-jaraco-text-3.2.0-6.el9.noarch 129/156 2024-04-25T10:27:58.772 INFO:teuthology.orchestra.run.smithi037.stdout: Verifying : python3-kubernetes-1:26.1.0-2.el9.noarch 130/156 2024-04-25T10:27:58.772 INFO:teuthology.orchestra.run.smithi037.stdout: Verifying : python3-logutils-0.3.5-21.el9.noarch 131/156 2024-04-25T10:27:58.772 INFO:teuthology.orchestra.run.smithi037.stdout: Verifying : python3-more-itertools-8.12.0-2.el9.noarch 132/156 2024-04-25T10:27:58.773 INFO:teuthology.orchestra.run.smithi037.stdout: Verifying : python3-natsort-7.1.1-5.el9.noarch 133/156 2024-04-25T10:27:58.773 INFO:teuthology.orchestra.run.smithi037.stdout: Verifying : python3-pecan-1.4.2-3.el9.noarch 134/156 2024-04-25T10:27:58.773 INFO:teuthology.orchestra.run.smithi037.stdout: Verifying : python3-portend-3.1.0-2.el9.noarch 135/156 2024-04-25T10:27:58.773 INFO:teuthology.orchestra.run.smithi037.stdout: Verifying : python3-pyOpenSSL-21.0.0-1.el9.noarch 136/156 2024-04-25T10:27:58.773 INFO:teuthology.orchestra.run.smithi037.stdout: Verifying : python3-pyperclip-1.8.2-2.el9.noarch 137/156 2024-04-25T10:27:58.773 INFO:teuthology.orchestra.run.smithi037.stdout: Verifying : python3-repoze-lru-0.7-16.el9.noarch 138/156 2024-04-25T10:27:58.773 INFO:teuthology.orchestra.run.smithi037.stdout: Verifying : python3-routes-2.5.1-5.el9.noarch 139/156 2024-04-25T10:27:58.773 INFO:teuthology.orchestra.run.smithi037.stdout: Verifying : python3-rsa-4.9-2.el9.noarch 140/156 2024-04-25T10:27:58.773 INFO:teuthology.orchestra.run.smithi037.stdout: Verifying : python3-tempora-5.0.0-2.el9.noarch 141/156 2024-04-25T10:27:58.773 INFO:teuthology.orchestra.run.smithi037.stdout: Verifying : python3-typing-extensions-4.4.0-2.el9.noarch 142/156 2024-04-25T10:27:58.773 INFO:teuthology.orchestra.run.smithi037.stdout: Verifying : python3-webob-1.8.7-6.el9.noarch 143/156 2024-04-25T10:27:58.773 INFO:teuthology.orchestra.run.smithi037.stdout: Verifying : python3-websocket-client-1.2.3-2.el9.noarch 144/156 2024-04-25T10:27:58.773 INFO:teuthology.orchestra.run.smithi037.stdout: Verifying : python3-werkzeug-2.0.3-3.el9.1.noarch 145/156 2024-04-25T10:27:58.774 INFO:teuthology.orchestra.run.smithi037.stdout: Verifying : python3-zc-lockfile-2.0-10.el9.noarch 146/156 2024-04-25T10:27:58.774 INFO:teuthology.orchestra.run.smithi037.stdout: Verifying : re2-1:20211101-3.el9.x86_64 147/156 2024-04-25T10:27:58.774 INFO:teuthology.orchestra.run.smithi037.stdout: Verifying : thrift-0.15.0-2.el9.x86_64 148/156 2024-04-25T10:27:58.774 INFO:teuthology.orchestra.run.smithi037.stdout: Verifying : librados2-2:19.0.0-3291.g994e2f20.el9.x86_64 149/156 2024-04-25T10:27:58.774 INFO:teuthology.orchestra.run.smithi037.stdout: Verifying : librados2-2:16.2.4-5.el9.x86_64 150/156 2024-04-25T10:27:58.774 INFO:teuthology.orchestra.run.smithi037.stdout: Verifying : librbd1-2:19.0.0-3291.g994e2f20.el9.x86_64 151/156 2024-04-25T10:27:58.774 INFO:teuthology.orchestra.run.smithi037.stdout: Verifying : librbd1-2:16.2.4-5.el9.x86_64 152/156 2024-04-25T10:27:58.774 INFO:teuthology.orchestra.run.smithi037.stdout: Verifying : openssl-1:3.2.1-1.el9.x86_64 153/156 2024-04-25T10:27:58.774 INFO:teuthology.orchestra.run.smithi037.stdout: Verifying : openssl-1:3.0.7-27.el9.x86_64 154/156 2024-04-25T10:27:58.774 INFO:teuthology.orchestra.run.smithi037.stdout: Verifying : openssl-libs-1:3.2.1-1.el9.x86_64 155/156 2024-04-25T10:28:00.907 INFO:teuthology.orchestra.run.smithi037.stdout: Verifying : openssl-libs-1:3.0.7-27.el9.x86_64 156/156 2024-04-25T10:28:00.907 INFO:teuthology.orchestra.run.smithi037.stdout: 2024-04-25T10:28:00.907 INFO:teuthology.orchestra.run.smithi037.stdout:Upgraded: 2024-04-25T10:28:00.907 INFO:teuthology.orchestra.run.smithi037.stdout: librados2-2:19.0.0-3291.g994e2f20.el9.x86_64 2024-04-25T10:28:00.907 INFO:teuthology.orchestra.run.smithi037.stdout: librbd1-2:19.0.0-3291.g994e2f20.el9.x86_64 2024-04-25T10:28:00.907 INFO:teuthology.orchestra.run.smithi037.stdout: openssl-1:3.2.1-1.el9.x86_64 2024-04-25T10:28:00.907 INFO:teuthology.orchestra.run.smithi037.stdout: openssl-libs-1:3.2.1-1.el9.x86_64 2024-04-25T10:28:00.907 INFO:teuthology.orchestra.run.smithi037.stdout:Installed: 2024-04-25T10:28:00.907 INFO:teuthology.orchestra.run.smithi037.stdout: abseil-cpp-20211102.0-3.el9.x86_64 2024-04-25T10:28:00.907 INFO:teuthology.orchestra.run.smithi037.stdout: automake-1.16.2-8.el9.noarch 2024-04-25T10:28:00.908 INFO:teuthology.orchestra.run.smithi037.stdout: bison-3.7.4-5.el9.x86_64 2024-04-25T10:28:00.908 INFO:teuthology.orchestra.run.smithi037.stdout: boost-program-options-1.75.0-8.el9.x86_64 2024-04-25T10:28:00.908 INFO:teuthology.orchestra.run.smithi037.stdout: ceph-2:19.0.0-3291.g994e2f20.el9.x86_64 2024-04-25T10:28:00.908 INFO:teuthology.orchestra.run.smithi037.stdout: ceph-base-2:19.0.0-3291.g994e2f20.el9.x86_64 2024-04-25T10:28:00.908 INFO:teuthology.orchestra.run.smithi037.stdout: ceph-common-2:19.0.0-3291.g994e2f20.el9.x86_64 2024-04-25T10:28:00.908 INFO:teuthology.orchestra.run.smithi037.stdout: ceph-fuse-2:19.0.0-3291.g994e2f20.el9.x86_64 2024-04-25T10:28:00.908 INFO:teuthology.orchestra.run.smithi037.stdout: ceph-grafana-dashboards-2:19.0.0-3291.g994e2f20.el9.noarch 2024-04-25T10:28:00.908 INFO:teuthology.orchestra.run.smithi037.stdout: ceph-immutable-object-cache-2:19.0.0-3291.g994e2f20.el9.x86_64 2024-04-25T10:28:00.908 INFO:teuthology.orchestra.run.smithi037.stdout: ceph-mds-2:19.0.0-3291.g994e2f20.el9.x86_64 2024-04-25T10:28:00.908 INFO:teuthology.orchestra.run.smithi037.stdout: ceph-mgr-2:19.0.0-3291.g994e2f20.el9.x86_64 2024-04-25T10:28:00.908 INFO:teuthology.orchestra.run.smithi037.stdout: ceph-mgr-cephadm-2:19.0.0-3291.g994e2f20.el9.noarch 2024-04-25T10:28:00.908 INFO:teuthology.orchestra.run.smithi037.stdout: ceph-mgr-dashboard-2:19.0.0-3291.g994e2f20.el9.noarch 2024-04-25T10:28:00.908 INFO:teuthology.orchestra.run.smithi037.stdout: ceph-mgr-diskprediction-local-2:19.0.0-3291.g994e2f20.el9.noarch 2024-04-25T10:28:00.908 INFO:teuthology.orchestra.run.smithi037.stdout: ceph-mgr-modules-core-2:19.0.0-3291.g994e2f20.el9.noarch 2024-04-25T10:28:00.908 INFO:teuthology.orchestra.run.smithi037.stdout: ceph-mgr-rook-2:19.0.0-3291.g994e2f20.el9.noarch 2024-04-25T10:28:00.909 INFO:teuthology.orchestra.run.smithi037.stdout: ceph-mon-2:19.0.0-3291.g994e2f20.el9.x86_64 2024-04-25T10:28:00.909 INFO:teuthology.orchestra.run.smithi037.stdout: ceph-osd-2:19.0.0-3291.g994e2f20.el9.x86_64 2024-04-25T10:28:00.909 INFO:teuthology.orchestra.run.smithi037.stdout: ceph-prometheus-alerts-2:19.0.0-3291.g994e2f20.el9.noarch 2024-04-25T10:28:00.909 INFO:teuthology.orchestra.run.smithi037.stdout: ceph-radosgw-2:19.0.0-3291.g994e2f20.el9.x86_64 2024-04-25T10:28:00.909 INFO:teuthology.orchestra.run.smithi037.stdout: ceph-selinux-2:19.0.0-3291.g994e2f20.el9.x86_64 2024-04-25T10:28:00.909 INFO:teuthology.orchestra.run.smithi037.stdout: ceph-test-2:19.0.0-3291.g994e2f20.el9.x86_64 2024-04-25T10:28:00.909 INFO:teuthology.orchestra.run.smithi037.stdout: ceph-volume-2:19.0.0-3291.g994e2f20.el9.noarch 2024-04-25T10:28:00.909 INFO:teuthology.orchestra.run.smithi037.stdout: cephadm-2:19.0.0-3291.g994e2f20.el9.noarch 2024-04-25T10:28:00.909 INFO:teuthology.orchestra.run.smithi037.stdout: cephfs-mirror-2:19.0.0-3291.g994e2f20.el9.x86_64 2024-04-25T10:28:00.909 INFO:teuthology.orchestra.run.smithi037.stdout: cephfs-shell-2:19.0.0-3291.g994e2f20.el9.x86_64 2024-04-25T10:28:00.909 INFO:teuthology.orchestra.run.smithi037.stdout: cephfs-top-2:19.0.0-3291.g994e2f20.el9.noarch 2024-04-25T10:28:00.909 INFO:teuthology.orchestra.run.smithi037.stdout: elfutils-libelf-devel-0.190-2.el9.x86_64 2024-04-25T10:28:00.909 INFO:teuthology.orchestra.run.smithi037.stdout: flex-2.6.4-9.el9.x86_64 2024-04-25T10:28:00.909 INFO:teuthology.orchestra.run.smithi037.stdout: flexiblas-3.0.4-8.el9.x86_64 2024-04-25T10:28:00.909 INFO:teuthology.orchestra.run.smithi037.stdout: flexiblas-netlib-3.0.4-8.el9.x86_64 2024-04-25T10:28:00.909 INFO:teuthology.orchestra.run.smithi037.stdout: flexiblas-openblas-openmp-3.0.4-8.el9.x86_64 2024-04-25T10:28:00.910 INFO:teuthology.orchestra.run.smithi037.stdout: fuse-2.9.9-15.el9.x86_64 2024-04-25T10:28:00.910 INFO:teuthology.orchestra.run.smithi037.stdout: gperftools-libs-2.9.1-2.el9.x86_64 2024-04-25T10:28:00.910 INFO:teuthology.orchestra.run.smithi037.stdout: grpc-data-1.46.7-10.el9.noarch 2024-04-25T10:28:00.910 INFO:teuthology.orchestra.run.smithi037.stdout: libaio-devel-0.3.111-13.el9.x86_64 2024-04-25T10:28:00.910 INFO:teuthology.orchestra.run.smithi037.stdout: libarrow-9.0.0-10.el9.x86_64 2024-04-25T10:28:00.910 INFO:teuthology.orchestra.run.smithi037.stdout: libarrow-doc-9.0.0-10.el9.noarch 2024-04-25T10:28:00.910 INFO:teuthology.orchestra.run.smithi037.stdout: libcephfs-devel-2:19.0.0-3291.g994e2f20.el9.x86_64 2024-04-25T10:28:00.910 INFO:teuthology.orchestra.run.smithi037.stdout: libcephfs2-2:19.0.0-3291.g994e2f20.el9.x86_64 2024-04-25T10:28:00.910 INFO:teuthology.orchestra.run.smithi037.stdout: libcephsqlite-2:19.0.0-3291.g994e2f20.el9.x86_64 2024-04-25T10:28:00.910 INFO:teuthology.orchestra.run.smithi037.stdout: libgfortran-11.4.1-3.el9.x86_64 2024-04-25T10:28:00.910 INFO:teuthology.orchestra.run.smithi037.stdout: liboath-2.6.7-2.el9.x86_64 2024-04-25T10:28:00.913 INFO:teuthology.orchestra.run.smithi037.stdout: libpmemobj-1.12.1-1.el9.x86_64 2024-04-25T10:28:00.913 INFO:teuthology.orchestra.run.smithi037.stdout: libquadmath-11.4.1-3.el9.x86_64 2024-04-25T10:28:00.913 INFO:teuthology.orchestra.run.smithi037.stdout: librabbitmq-0.11.0-7.el9.x86_64 2024-04-25T10:28:00.913 INFO:teuthology.orchestra.run.smithi037.stdout: librados-devel-2:19.0.0-3291.g994e2f20.el9.x86_64 2024-04-25T10:28:00.913 INFO:teuthology.orchestra.run.smithi037.stdout: libradosstriper1-2:19.0.0-3291.g994e2f20.el9.x86_64 2024-04-25T10:28:00.914 INFO:teuthology.orchestra.run.smithi037.stdout: librdkafka-1.6.1-102.el9.x86_64 2024-04-25T10:28:00.914 INFO:teuthology.orchestra.run.smithi037.stdout: librgw2-2:19.0.0-3291.g994e2f20.el9.x86_64 2024-04-25T10:28:00.914 INFO:teuthology.orchestra.run.smithi037.stdout: libtool-2.4.6-45.el9.x86_64 2024-04-25T10:28:00.914 INFO:teuthology.orchestra.run.smithi037.stdout: libunwind-1.6.2-1.el9.x86_64 2024-04-25T10:28:00.914 INFO:teuthology.orchestra.run.smithi037.stdout: libuuid-devel-2.37.4-18.el9.x86_64 2024-04-25T10:28:00.914 INFO:teuthology.orchestra.run.smithi037.stdout: libzstd-devel-1.5.1-2.el9.x86_64 2024-04-25T10:28:00.914 INFO:teuthology.orchestra.run.smithi037.stdout: lttng-ust-2.12.0-6.el9.x86_64 2024-04-25T10:28:00.914 INFO:teuthology.orchestra.run.smithi037.stdout: lua-5.4.4-4.el9.x86_64 2024-04-25T10:28:00.914 INFO:teuthology.orchestra.run.smithi037.stdout: lua-devel-5.4.4-4.el9.x86_64 2024-04-25T10:28:00.915 INFO:teuthology.orchestra.run.smithi037.stdout: luarocks-3.9.2-1.el9.noarch 2024-04-25T10:28:00.915 INFO:teuthology.orchestra.run.smithi037.stdout: openblas-0.3.21-2.el9.x86_64 2024-04-25T10:28:00.915 INFO:teuthology.orchestra.run.smithi037.stdout: openblas-openmp-0.3.21-2.el9.x86_64 2024-04-25T10:28:00.915 INFO:teuthology.orchestra.run.smithi037.stdout: openssl-devel-1:3.2.1-1.el9.x86_64 2024-04-25T10:28:00.915 INFO:teuthology.orchestra.run.smithi037.stdout: parquet-libs-9.0.0-10.el9.x86_64 2024-04-25T10:28:00.915 INFO:teuthology.orchestra.run.smithi037.stdout: postgresql-13.14-1.el9.x86_64 2024-04-25T10:28:00.915 INFO:teuthology.orchestra.run.smithi037.stdout: postgresql-contrib-13.14-1.el9.x86_64 2024-04-25T10:28:00.915 INFO:teuthology.orchestra.run.smithi037.stdout: postgresql-private-libs-13.14-1.el9.x86_64 2024-04-25T10:28:00.915 INFO:teuthology.orchestra.run.smithi037.stdout: postgresql-server-13.14-1.el9.x86_64 2024-04-25T10:28:00.915 INFO:teuthology.orchestra.run.smithi037.stdout: protobuf-3.14.0-13.el9.x86_64 2024-04-25T10:28:00.916 INFO:teuthology.orchestra.run.smithi037.stdout: protobuf-compiler-3.14.0-13.el9.x86_64 2024-04-25T10:28:00.916 INFO:teuthology.orchestra.run.smithi037.stdout: python3-asyncssh-2.13.2-5.el9.noarch 2024-04-25T10:28:00.916 INFO:teuthology.orchestra.run.smithi037.stdout: python3-attrs-20.3.0-7.el9.noarch 2024-04-25T10:28:00.916 INFO:teuthology.orchestra.run.smithi037.stdout: python3-babel-2.9.1-2.el9.noarch 2024-04-25T10:28:00.916 INFO:teuthology.orchestra.run.smithi037.stdout: python3-bcrypt-3.2.2-1.el9.x86_64 2024-04-25T10:28:00.916 INFO:teuthology.orchestra.run.smithi037.stdout: python3-cachetools-4.2.4-1.el9.noarch 2024-04-25T10:28:00.916 INFO:teuthology.orchestra.run.smithi037.stdout: python3-ceph-argparse-2:19.0.0-3291.g994e2f20.el9.x86_64 2024-04-25T10:28:00.917 INFO:teuthology.orchestra.run.smithi037.stdout: python3-ceph-common-2:19.0.0-3291.g994e2f20.el9.x86_64 2024-04-25T10:28:00.917 INFO:teuthology.orchestra.run.smithi037.stdout: python3-cephfs-2:19.0.0-3291.g994e2f20.el9.x86_64 2024-04-25T10:28:00.917 INFO:teuthology.orchestra.run.smithi037.stdout: python3-certifi-2021.10.8-2.el9.noarch 2024-04-25T10:28:00.917 INFO:teuthology.orchestra.run.smithi037.stdout: python3-cffi-1.14.5-5.el9.x86_64 2024-04-25T10:28:00.917 INFO:teuthology.orchestra.run.smithi037.stdout: python3-cheroot-8.6.0-4.el9.noarch 2024-04-25T10:28:00.917 INFO:teuthology.orchestra.run.smithi037.stdout: python3-cherrypy-18.6.1-2.el9.noarch 2024-04-25T10:28:00.917 INFO:teuthology.orchestra.run.smithi037.stdout: python3-cmd2-2.4.2-2.el9.noarch 2024-04-25T10:28:00.918 INFO:teuthology.orchestra.run.smithi037.stdout: python3-colorama-0.4.4-12.el9.noarch 2024-04-25T10:28:00.918 INFO:teuthology.orchestra.run.smithi037.stdout: python3-cryptography-36.0.1-4.el9.x86_64 2024-04-25T10:28:00.918 INFO:teuthology.orchestra.run.smithi037.stdout: python3-devel-3.9.18-3.el9.x86_64 2024-04-25T10:28:00.918 INFO:teuthology.orchestra.run.smithi037.stdout: python3-google-auth-1:2.29.0-1.el9.noarch 2024-04-25T10:28:00.918 INFO:teuthology.orchestra.run.smithi037.stdout: python3-grpcio-1.46.7-10.el9.x86_64 2024-04-25T10:28:00.918 INFO:teuthology.orchestra.run.smithi037.stdout: python3-grpcio-tools-1.46.7-10.el9.x86_64 2024-04-25T10:28:00.918 INFO:teuthology.orchestra.run.smithi037.stdout: python3-jaraco-8.2.1-3.el9.noarch 2024-04-25T10:28:00.918 INFO:teuthology.orchestra.run.smithi037.stdout: python3-jaraco-classes-3.2.1-5.el9.noarch 2024-04-25T10:28:00.918 INFO:teuthology.orchestra.run.smithi037.stdout: python3-jaraco-collections-3.0.0-8.el9.noarch 2024-04-25T10:28:00.918 INFO:teuthology.orchestra.run.smithi037.stdout: python3-jaraco-functools-3.5.0-2.el9.noarch 2024-04-25T10:28:00.918 INFO:teuthology.orchestra.run.smithi037.stdout: python3-jaraco-text-3.2.0-6.el9.noarch 2024-04-25T10:28:00.918 INFO:teuthology.orchestra.run.smithi037.stdout: python3-jinja2-2.11.3-5.el9.noarch 2024-04-25T10:28:00.918 INFO:teuthology.orchestra.run.smithi037.stdout: python3-jsonpatch-1.21-16.el9.noarch 2024-04-25T10:28:00.919 INFO:teuthology.orchestra.run.smithi037.stdout: python3-jsonpointer-2.0-4.el9.noarch 2024-04-25T10:28:00.919 INFO:teuthology.orchestra.run.smithi037.stdout: python3-kubernetes-1:26.1.0-2.el9.noarch 2024-04-25T10:28:00.919 INFO:teuthology.orchestra.run.smithi037.stdout: python3-logutils-0.3.5-21.el9.noarch 2024-04-25T10:28:00.919 INFO:teuthology.orchestra.run.smithi037.stdout: python3-mako-1.1.4-6.el9.noarch 2024-04-25T10:28:00.919 INFO:teuthology.orchestra.run.smithi037.stdout: python3-markupsafe-1.1.1-12.el9.x86_64 2024-04-25T10:28:00.920 INFO:teuthology.orchestra.run.smithi037.stdout: python3-more-itertools-8.12.0-2.el9.noarch 2024-04-25T10:28:00.920 INFO:teuthology.orchestra.run.smithi037.stdout: python3-natsort-7.1.1-5.el9.noarch 2024-04-25T10:28:00.920 INFO:teuthology.orchestra.run.smithi037.stdout: python3-numpy-1:1.20.1-5.el9.x86_64 2024-04-25T10:28:00.920 INFO:teuthology.orchestra.run.smithi037.stdout: python3-numpy-f2py-1:1.20.1-5.el9.x86_64 2024-04-25T10:28:00.920 INFO:teuthology.orchestra.run.smithi037.stdout: python3-oauthlib-3.1.1-5.el9.noarch 2024-04-25T10:28:00.920 INFO:teuthology.orchestra.run.smithi037.stdout: python3-packaging-20.9-5.el9.noarch 2024-04-25T10:28:00.921 INFO:teuthology.orchestra.run.smithi037.stdout: python3-pecan-1.4.2-3.el9.noarch 2024-04-25T10:28:00.921 INFO:teuthology.orchestra.run.smithi037.stdout: python3-ply-3.11-14.el9.noarch 2024-04-25T10:28:00.921 INFO:teuthology.orchestra.run.smithi037.stdout: python3-portend-3.1.0-2.el9.noarch 2024-04-25T10:28:00.921 INFO:teuthology.orchestra.run.smithi037.stdout: python3-prettytable-0.7.2-27.el9.noarch 2024-04-25T10:28:00.921 INFO:teuthology.orchestra.run.smithi037.stdout: python3-protobuf-3.14.0-13.el9.noarch 2024-04-25T10:28:00.921 INFO:teuthology.orchestra.run.smithi037.stdout: python3-pyOpenSSL-21.0.0-1.el9.noarch 2024-04-25T10:28:00.921 INFO:teuthology.orchestra.run.smithi037.stdout: python3-pyasn1-0.4.8-6.el9.noarch 2024-04-25T10:28:00.922 INFO:teuthology.orchestra.run.smithi037.stdout: python3-pyasn1-modules-0.4.8-6.el9.noarch 2024-04-25T10:28:00.922 INFO:teuthology.orchestra.run.smithi037.stdout: python3-pycparser-2.20-6.el9.noarch 2024-04-25T10:28:00.922 INFO:teuthology.orchestra.run.smithi037.stdout: python3-pyperclip-1.8.2-2.el9.noarch 2024-04-25T10:28:00.922 INFO:teuthology.orchestra.run.smithi037.stdout: python3-pytz-2021.1-5.el9.noarch 2024-04-25T10:28:00.922 INFO:teuthology.orchestra.run.smithi037.stdout: python3-rados-2:19.0.0-3291.g994e2f20.el9.x86_64 2024-04-25T10:28:00.922 INFO:teuthology.orchestra.run.smithi037.stdout: python3-rbd-2:19.0.0-3291.g994e2f20.el9.x86_64 2024-04-25T10:28:00.923 INFO:teuthology.orchestra.run.smithi037.stdout: python3-repoze-lru-0.7-16.el9.noarch 2024-04-25T10:28:00.923 INFO:teuthology.orchestra.run.smithi037.stdout: python3-requests-2.25.1-8.el9.noarch 2024-04-25T10:28:00.923 INFO:teuthology.orchestra.run.smithi037.stdout: python3-requests-oauthlib-1.3.0-12.el9.noarch 2024-04-25T10:28:00.923 INFO:teuthology.orchestra.run.smithi037.stdout: python3-rgw-2:19.0.0-3291.g994e2f20.el9.x86_64 2024-04-25T10:28:00.923 INFO:teuthology.orchestra.run.smithi037.stdout: python3-routes-2.5.1-5.el9.noarch 2024-04-25T10:28:00.923 INFO:teuthology.orchestra.run.smithi037.stdout: python3-rsa-4.9-2.el9.noarch 2024-04-25T10:28:00.923 INFO:teuthology.orchestra.run.smithi037.stdout: python3-scipy-1.6.2-8.el9.x86_64 2024-04-25T10:28:00.923 INFO:teuthology.orchestra.run.smithi037.stdout: python3-tempora-5.0.0-2.el9.noarch 2024-04-25T10:28:00.923 INFO:teuthology.orchestra.run.smithi037.stdout: python3-toml-0.10.2-6.el9.noarch 2024-04-25T10:28:00.923 INFO:teuthology.orchestra.run.smithi037.stdout: python3-typing-extensions-4.4.0-2.el9.noarch 2024-04-25T10:28:00.923 INFO:teuthology.orchestra.run.smithi037.stdout: python3-urllib3-1.26.5-5.el9.noarch 2024-04-25T10:28:00.923 INFO:teuthology.orchestra.run.smithi037.stdout: python3-wcwidth-0.2.5-8.el9.noarch 2024-04-25T10:28:00.923 INFO:teuthology.orchestra.run.smithi037.stdout: python3-webob-1.8.7-6.el9.noarch 2024-04-25T10:28:00.924 INFO:teuthology.orchestra.run.smithi037.stdout: python3-websocket-client-1.2.3-2.el9.noarch 2024-04-25T10:28:00.924 INFO:teuthology.orchestra.run.smithi037.stdout: python3-werkzeug-2.0.3-3.el9.1.noarch 2024-04-25T10:28:00.924 INFO:teuthology.orchestra.run.smithi037.stdout: python3-zc-lockfile-2.0-10.el9.noarch 2024-04-25T10:28:00.924 INFO:teuthology.orchestra.run.smithi037.stdout: qatlib-24.02.0-1.el9.x86_64 2024-04-25T10:28:00.924 INFO:teuthology.orchestra.run.smithi037.stdout: qatlib-service-24.02.0-1.el9.x86_64 2024-04-25T10:28:00.924 INFO:teuthology.orchestra.run.smithi037.stdout: qatzip-libs-1.2.0-1.el9.x86_64 2024-04-25T10:28:00.924 INFO:teuthology.orchestra.run.smithi037.stdout: rbd-fuse-2:19.0.0-3291.g994e2f20.el9.x86_64 2024-04-25T10:28:00.924 INFO:teuthology.orchestra.run.smithi037.stdout: rbd-mirror-2:19.0.0-3291.g994e2f20.el9.x86_64 2024-04-25T10:28:00.924 INFO:teuthology.orchestra.run.smithi037.stdout: rbd-nbd-2:19.0.0-3291.g994e2f20.el9.x86_64 2024-04-25T10:28:00.924 INFO:teuthology.orchestra.run.smithi037.stdout: re2-1:20211101-3.el9.x86_64 2024-04-25T10:28:00.924 INFO:teuthology.orchestra.run.smithi037.stdout: socat-1.7.4.1-5.el9.x86_64 2024-04-25T10:28:00.924 INFO:teuthology.orchestra.run.smithi037.stdout: thrift-0.15.0-2.el9.x86_64 2024-04-25T10:28:00.924 INFO:teuthology.orchestra.run.smithi037.stdout: uuid-1.6.2-55.el9.x86_64 2024-04-25T10:28:00.925 INFO:teuthology.orchestra.run.smithi037.stdout: xfsprogs-devel-6.3.0-1.el9.x86_64 2024-04-25T10:28:00.925 INFO:teuthology.orchestra.run.smithi037.stdout: xmlstarlet-1.6.1-20.el9.x86_64 2024-04-25T10:28:00.925 INFO:teuthology.orchestra.run.smithi037.stdout: zlib-devel-1.2.11-41.el9.x86_64 2024-04-25T10:28:00.925 INFO:teuthology.orchestra.run.smithi037.stdout: 2024-04-25T10:28:00.925 INFO:teuthology.orchestra.run.smithi037.stdout:Complete! 2024-04-25T10:28:01.309 DEBUG:teuthology.parallel:result is None 2024-04-25T10:28:03.466 INFO:teuthology.orchestra.run.smithi203.stdout: Running scriptlet: openssl-libs-1:3.0.7-27.el9.x86_64 156/156 2024-04-25T10:28:03.466 INFO:teuthology.orchestra.run.smithi203.stdout: Verifying : ceph-2:19.0.0-3291.g994e2f20.el9.x86_64 1/156 2024-04-25T10:28:03.466 INFO:teuthology.orchestra.run.smithi203.stdout: Verifying : ceph-base-2:19.0.0-3291.g994e2f20.el9.x86_64 2/156 2024-04-25T10:28:03.467 INFO:teuthology.orchestra.run.smithi203.stdout: Verifying : ceph-common-2:19.0.0-3291.g994e2f20.el9.x86_64 3/156 2024-04-25T10:28:03.467 INFO:teuthology.orchestra.run.smithi203.stdout: Verifying : ceph-fuse-2:19.0.0-3291.g994e2f20.el9.x86_64 4/156 2024-04-25T10:28:03.467 INFO:teuthology.orchestra.run.smithi203.stdout: Verifying : ceph-immutable-object-cache-2:19.0.0-3291.g994e2 5/156 2024-04-25T10:28:03.467 INFO:teuthology.orchestra.run.smithi203.stdout: Verifying : ceph-mds-2:19.0.0-3291.g994e2f20.el9.x86_64 6/156 2024-04-25T10:28:03.467 INFO:teuthology.orchestra.run.smithi203.stdout: Verifying : ceph-mgr-2:19.0.0-3291.g994e2f20.el9.x86_64 7/156 2024-04-25T10:28:03.467 INFO:teuthology.orchestra.run.smithi203.stdout: Verifying : ceph-mon-2:19.0.0-3291.g994e2f20.el9.x86_64 8/156 2024-04-25T10:28:03.467 INFO:teuthology.orchestra.run.smithi203.stdout: Verifying : ceph-osd-2:19.0.0-3291.g994e2f20.el9.x86_64 9/156 2024-04-25T10:28:03.467 INFO:teuthology.orchestra.run.smithi203.stdout: Verifying : ceph-radosgw-2:19.0.0-3291.g994e2f20.el9.x86_64 10/156 2024-04-25T10:28:03.467 INFO:teuthology.orchestra.run.smithi203.stdout: Verifying : ceph-selinux-2:19.0.0-3291.g994e2f20.el9.x86_64 11/156 2024-04-25T10:28:03.467 INFO:teuthology.orchestra.run.smithi203.stdout: Verifying : ceph-test-2:19.0.0-3291.g994e2f20.el9.x86_64 12/156 2024-04-25T10:28:03.467 INFO:teuthology.orchestra.run.smithi203.stdout: Verifying : cephfs-mirror-2:19.0.0-3291.g994e2f20.el9.x86_64 13/156 2024-04-25T10:28:03.467 INFO:teuthology.orchestra.run.smithi203.stdout: Verifying : cephfs-shell-2:19.0.0-3291.g994e2f20.el9.x86_64 14/156 2024-04-25T10:28:03.467 INFO:teuthology.orchestra.run.smithi203.stdout: Verifying : libcephfs-devel-2:19.0.0-3291.g994e2f20.el9.x86_ 15/156 2024-04-25T10:28:03.468 INFO:teuthology.orchestra.run.smithi203.stdout: Verifying : libcephfs2-2:19.0.0-3291.g994e2f20.el9.x86_64 16/156 2024-04-25T10:28:03.468 INFO:teuthology.orchestra.run.smithi203.stdout: Verifying : libcephsqlite-2:19.0.0-3291.g994e2f20.el9.x86_64 17/156 2024-04-25T10:28:03.468 INFO:teuthology.orchestra.run.smithi203.stdout: Verifying : librados-devel-2:19.0.0-3291.g994e2f20.el9.x86_6 18/156 2024-04-25T10:28:03.468 INFO:teuthology.orchestra.run.smithi203.stdout: Verifying : libradosstriper1-2:19.0.0-3291.g994e2f20.el9.x86 19/156 2024-04-25T10:28:03.468 INFO:teuthology.orchestra.run.smithi203.stdout: Verifying : librgw2-2:19.0.0-3291.g994e2f20.el9.x86_64 20/156 2024-04-25T10:28:03.468 INFO:teuthology.orchestra.run.smithi203.stdout: Verifying : python3-ceph-argparse-2:19.0.0-3291.g994e2f20.el 21/156 2024-04-25T10:28:03.468 INFO:teuthology.orchestra.run.smithi203.stdout: Verifying : python3-ceph-common-2:19.0.0-3291.g994e2f20.el9. 22/156 2024-04-25T10:28:03.468 INFO:teuthology.orchestra.run.smithi203.stdout: Verifying : python3-cephfs-2:19.0.0-3291.g994e2f20.el9.x86_6 23/156 2024-04-25T10:28:03.468 INFO:teuthology.orchestra.run.smithi203.stdout: Verifying : python3-rados-2:19.0.0-3291.g994e2f20.el9.x86_64 24/156 2024-04-25T10:28:03.468 INFO:teuthology.orchestra.run.smithi203.stdout: Verifying : python3-rbd-2:19.0.0-3291.g994e2f20.el9.x86_64 25/156 2024-04-25T10:28:03.468 INFO:teuthology.orchestra.run.smithi203.stdout: Verifying : python3-rgw-2:19.0.0-3291.g994e2f20.el9.x86_64 26/156 2024-04-25T10:28:03.468 INFO:teuthology.orchestra.run.smithi203.stdout: Verifying : rbd-fuse-2:19.0.0-3291.g994e2f20.el9.x86_64 27/156 2024-04-25T10:28:03.468 INFO:teuthology.orchestra.run.smithi203.stdout: Verifying : rbd-mirror-2:19.0.0-3291.g994e2f20.el9.x86_64 28/156 2024-04-25T10:28:03.468 INFO:teuthology.orchestra.run.smithi203.stdout: Verifying : rbd-nbd-2:19.0.0-3291.g994e2f20.el9.x86_64 29/156 2024-04-25T10:28:03.468 INFO:teuthology.orchestra.run.smithi203.stdout: Verifying : ceph-grafana-dashboards-2:19.0.0-3291.g994e2f20. 30/156 2024-04-25T10:28:03.480 INFO:teuthology.orchestra.run.smithi203.stdout: Verifying : ceph-mgr-cephadm-2:19.0.0-3291.g994e2f20.el9.noa 31/156 2024-04-25T10:28:03.480 INFO:teuthology.orchestra.run.smithi203.stdout: Verifying : ceph-mgr-dashboard-2:19.0.0-3291.g994e2f20.el9.n 32/156 2024-04-25T10:28:03.480 INFO:teuthology.orchestra.run.smithi203.stdout: Verifying : ceph-mgr-diskprediction-local-2:19.0.0-3291.g994 33/156 2024-04-25T10:28:03.480 INFO:teuthology.orchestra.run.smithi203.stdout: Verifying : ceph-mgr-modules-core-2:19.0.0-3291.g994e2f20.el 34/156 2024-04-25T10:28:03.480 INFO:teuthology.orchestra.run.smithi203.stdout: Verifying : ceph-mgr-rook-2:19.0.0-3291.g994e2f20.el9.noarch 35/156 2024-04-25T10:28:03.480 INFO:teuthology.orchestra.run.smithi203.stdout: Verifying : ceph-prometheus-alerts-2:19.0.0-3291.g994e2f20.e 36/156 2024-04-25T10:28:03.481 INFO:teuthology.orchestra.run.smithi203.stdout: Verifying : ceph-volume-2:19.0.0-3291.g994e2f20.el9.noarch 37/156 2024-04-25T10:28:03.481 INFO:teuthology.orchestra.run.smithi203.stdout: Verifying : cephadm-2:19.0.0-3291.g994e2f20.el9.noarch 38/156 2024-04-25T10:28:03.481 INFO:teuthology.orchestra.run.smithi203.stdout: Verifying : cephfs-top-2:19.0.0-3291.g994e2f20.el9.noarch 39/156 2024-04-25T10:28:03.481 INFO:teuthology.orchestra.run.smithi203.stdout: Verifying : fuse-2.9.9-15.el9.x86_64 40/156 2024-04-25T10:28:03.481 INFO:teuthology.orchestra.run.smithi203.stdout: Verifying : libgfortran-11.4.1-3.el9.x86_64 41/156 2024-04-25T10:28:03.481 INFO:teuthology.orchestra.run.smithi203.stdout: Verifying : libquadmath-11.4.1-3.el9.x86_64 42/156 2024-04-25T10:28:03.481 INFO:teuthology.orchestra.run.smithi203.stdout: Verifying : python3-cffi-1.14.5-5.el9.x86_64 43/156 2024-04-25T10:28:03.481 INFO:teuthology.orchestra.run.smithi203.stdout: Verifying : python3-cryptography-36.0.1-4.el9.x86_64 44/156 2024-04-25T10:28:03.481 INFO:teuthology.orchestra.run.smithi203.stdout: Verifying : python3-ply-3.11-14.el9.noarch 45/156 2024-04-25T10:28:03.481 INFO:teuthology.orchestra.run.smithi203.stdout: Verifying : python3-pycparser-2.20-6.el9.noarch 46/156 2024-04-25T10:28:03.481 INFO:teuthology.orchestra.run.smithi203.stdout: Verifying : python3-requests-2.25.1-8.el9.noarch 47/156 2024-04-25T10:28:03.481 INFO:teuthology.orchestra.run.smithi203.stdout: Verifying : python3-urllib3-1.26.5-5.el9.noarch 48/156 2024-04-25T10:28:03.481 INFO:teuthology.orchestra.run.smithi203.stdout: Verifying : automake-1.16.2-8.el9.noarch 49/156 2024-04-25T10:28:03.481 INFO:teuthology.orchestra.run.smithi203.stdout: Verifying : bison-3.7.4-5.el9.x86_64 50/156 2024-04-25T10:28:03.481 INFO:teuthology.orchestra.run.smithi203.stdout: Verifying : boost-program-options-1.75.0-8.el9.x86_64 51/156 2024-04-25T10:28:03.482 INFO:teuthology.orchestra.run.smithi203.stdout: Verifying : elfutils-libelf-devel-0.190-2.el9.x86_64 52/156 2024-04-25T10:28:03.482 INFO:teuthology.orchestra.run.smithi203.stdout: Verifying : flex-2.6.4-9.el9.x86_64 53/156 2024-04-25T10:28:03.482 INFO:teuthology.orchestra.run.smithi203.stdout: Verifying : flexiblas-3.0.4-8.el9.x86_64 54/156 2024-04-25T10:28:03.482 INFO:teuthology.orchestra.run.smithi203.stdout: Verifying : flexiblas-netlib-3.0.4-8.el9.x86_64 55/156 2024-04-25T10:28:03.482 INFO:teuthology.orchestra.run.smithi203.stdout: Verifying : flexiblas-openblas-openmp-3.0.4-8.el9.x86_64 56/156 2024-04-25T10:28:03.482 INFO:teuthology.orchestra.run.smithi203.stdout: Verifying : libaio-devel-0.3.111-13.el9.x86_64 57/156 2024-04-25T10:28:03.482 INFO:teuthology.orchestra.run.smithi203.stdout: Verifying : libpmemobj-1.12.1-1.el9.x86_64 58/156 2024-04-25T10:28:03.482 INFO:teuthology.orchestra.run.smithi203.stdout: Verifying : librabbitmq-0.11.0-7.el9.x86_64 59/156 2024-04-25T10:28:03.482 INFO:teuthology.orchestra.run.smithi203.stdout: Verifying : librdkafka-1.6.1-102.el9.x86_64 60/156 2024-04-25T10:28:03.482 INFO:teuthology.orchestra.run.smithi203.stdout: Verifying : libtool-2.4.6-45.el9.x86_64 61/156 2024-04-25T10:28:03.482 INFO:teuthology.orchestra.run.smithi203.stdout: Verifying : libuuid-devel-2.37.4-18.el9.x86_64 62/156 2024-04-25T10:28:03.482 INFO:teuthology.orchestra.run.smithi203.stdout: Verifying : libzstd-devel-1.5.1-2.el9.x86_64 63/156 2024-04-25T10:28:03.483 INFO:teuthology.orchestra.run.smithi203.stdout: Verifying : lttng-ust-2.12.0-6.el9.x86_64 64/156 2024-04-25T10:28:03.483 INFO:teuthology.orchestra.run.smithi203.stdout: Verifying : lua-5.4.4-4.el9.x86_64 65/156 2024-04-25T10:28:03.483 INFO:teuthology.orchestra.run.smithi203.stdout: Verifying : openblas-0.3.21-2.el9.x86_64 66/156 2024-04-25T10:28:03.483 INFO:teuthology.orchestra.run.smithi203.stdout: Verifying : openblas-openmp-0.3.21-2.el9.x86_64 67/156 2024-04-25T10:28:03.483 INFO:teuthology.orchestra.run.smithi203.stdout: Verifying : openssl-devel-1:3.2.1-1.el9.x86_64 68/156 2024-04-25T10:28:03.483 INFO:teuthology.orchestra.run.smithi203.stdout: Verifying : postgresql-13.14-1.el9.x86_64 69/156 2024-04-25T10:28:03.483 INFO:teuthology.orchestra.run.smithi203.stdout: Verifying : postgresql-contrib-13.14-1.el9.x86_64 70/156 2024-04-25T10:28:03.483 INFO:teuthology.orchestra.run.smithi203.stdout: Verifying : postgresql-private-libs-13.14-1.el9.x86_64 71/156 2024-04-25T10:28:03.483 INFO:teuthology.orchestra.run.smithi203.stdout: Verifying : postgresql-server-13.14-1.el9.x86_64 72/156 2024-04-25T10:28:03.483 INFO:teuthology.orchestra.run.smithi203.stdout: Verifying : protobuf-3.14.0-13.el9.x86_64 73/156 2024-04-25T10:28:03.483 INFO:teuthology.orchestra.run.smithi203.stdout: Verifying : python3-attrs-20.3.0-7.el9.noarch 74/156 2024-04-25T10:28:03.483 INFO:teuthology.orchestra.run.smithi203.stdout: Verifying : python3-babel-2.9.1-2.el9.noarch 75/156 2024-04-25T10:28:03.483 INFO:teuthology.orchestra.run.smithi203.stdout: Verifying : python3-devel-3.9.18-3.el9.x86_64 76/156 2024-04-25T10:28:03.483 INFO:teuthology.orchestra.run.smithi203.stdout: Verifying : python3-jinja2-2.11.3-5.el9.noarch 77/156 2024-04-25T10:28:03.484 INFO:teuthology.orchestra.run.smithi203.stdout: Verifying : python3-jsonpatch-1.21-16.el9.noarch 78/156 2024-04-25T10:28:03.484 INFO:teuthology.orchestra.run.smithi203.stdout: Verifying : python3-jsonpointer-2.0-4.el9.noarch 79/156 2024-04-25T10:28:03.484 INFO:teuthology.orchestra.run.smithi203.stdout: Verifying : python3-mako-1.1.4-6.el9.noarch 80/156 2024-04-25T10:28:03.484 INFO:teuthology.orchestra.run.smithi203.stdout: Verifying : python3-markupsafe-1.1.1-12.el9.x86_64 81/156 2024-04-25T10:28:03.484 INFO:teuthology.orchestra.run.smithi203.stdout: Verifying : python3-numpy-1:1.20.1-5.el9.x86_64 82/156 2024-04-25T10:28:03.484 INFO:teuthology.orchestra.run.smithi203.stdout: Verifying : python3-numpy-f2py-1:1.20.1-5.el9.x86_64 83/156 2024-04-25T10:28:03.484 INFO:teuthology.orchestra.run.smithi203.stdout: Verifying : python3-oauthlib-3.1.1-5.el9.noarch 84/156 2024-04-25T10:28:03.484 INFO:teuthology.orchestra.run.smithi203.stdout: Verifying : python3-packaging-20.9-5.el9.noarch 85/156 2024-04-25T10:28:03.484 INFO:teuthology.orchestra.run.smithi203.stdout: Verifying : python3-prettytable-0.7.2-27.el9.noarch 86/156 2024-04-25T10:28:03.484 INFO:teuthology.orchestra.run.smithi203.stdout: Verifying : python3-protobuf-3.14.0-13.el9.noarch 87/156 2024-04-25T10:28:03.484 INFO:teuthology.orchestra.run.smithi203.stdout: Verifying : python3-pyasn1-0.4.8-6.el9.noarch 88/156 2024-04-25T10:28:03.484 INFO:teuthology.orchestra.run.smithi203.stdout: Verifying : python3-pyasn1-modules-0.4.8-6.el9.noarch 89/156 2024-04-25T10:28:03.484 INFO:teuthology.orchestra.run.smithi203.stdout: Verifying : python3-pytz-2021.1-5.el9.noarch 90/156 2024-04-25T10:28:03.484 INFO:teuthology.orchestra.run.smithi203.stdout: Verifying : python3-requests-oauthlib-1.3.0-12.el9.noarch 91/156 2024-04-25T10:28:03.485 INFO:teuthology.orchestra.run.smithi203.stdout: Verifying : python3-scipy-1.6.2-8.el9.x86_64 92/156 2024-04-25T10:28:03.485 INFO:teuthology.orchestra.run.smithi203.stdout: Verifying : python3-toml-0.10.2-6.el9.noarch 93/156 2024-04-25T10:28:03.485 INFO:teuthology.orchestra.run.smithi203.stdout: Verifying : python3-wcwidth-0.2.5-8.el9.noarch 94/156 2024-04-25T10:28:03.485 INFO:teuthology.orchestra.run.smithi203.stdout: Verifying : qatlib-24.02.0-1.el9.x86_64 95/156 2024-04-25T10:28:03.485 INFO:teuthology.orchestra.run.smithi203.stdout: Verifying : qatlib-service-24.02.0-1.el9.x86_64 96/156 2024-04-25T10:28:03.485 INFO:teuthology.orchestra.run.smithi203.stdout: Verifying : qatzip-libs-1.2.0-1.el9.x86_64 97/156 2024-04-25T10:28:03.485 INFO:teuthology.orchestra.run.smithi203.stdout: Verifying : socat-1.7.4.1-5.el9.x86_64 98/156 2024-04-25T10:28:03.485 INFO:teuthology.orchestra.run.smithi203.stdout: Verifying : uuid-1.6.2-55.el9.x86_64 99/156 2024-04-25T10:28:03.485 INFO:teuthology.orchestra.run.smithi203.stdout: Verifying : xfsprogs-devel-6.3.0-1.el9.x86_64 100/156 2024-04-25T10:28:03.485 INFO:teuthology.orchestra.run.smithi203.stdout: Verifying : xmlstarlet-1.6.1-20.el9.x86_64 101/156 2024-04-25T10:28:03.485 INFO:teuthology.orchestra.run.smithi203.stdout: Verifying : zlib-devel-1.2.11-41.el9.x86_64 102/156 2024-04-25T10:28:03.485 INFO:teuthology.orchestra.run.smithi203.stdout: Verifying : lua-devel-5.4.4-4.el9.x86_64 103/156 2024-04-25T10:28:03.485 INFO:teuthology.orchestra.run.smithi203.stdout: Verifying : protobuf-compiler-3.14.0-13.el9.x86_64 104/156 2024-04-25T10:28:03.485 INFO:teuthology.orchestra.run.smithi203.stdout: Verifying : abseil-cpp-20211102.0-3.el9.x86_64 105/156 2024-04-25T10:28:03.486 INFO:teuthology.orchestra.run.smithi203.stdout: Verifying : gperftools-libs-2.9.1-2.el9.x86_64 106/156 2024-04-25T10:28:03.486 INFO:teuthology.orchestra.run.smithi203.stdout: Verifying : grpc-data-1.46.7-10.el9.noarch 107/156 2024-04-25T10:28:03.486 INFO:teuthology.orchestra.run.smithi203.stdout: Verifying : libarrow-9.0.0-10.el9.x86_64 108/156 2024-04-25T10:28:03.486 INFO:teuthology.orchestra.run.smithi203.stdout: Verifying : libarrow-doc-9.0.0-10.el9.noarch 109/156 2024-04-25T10:28:03.486 INFO:teuthology.orchestra.run.smithi203.stdout: Verifying : liboath-2.6.7-2.el9.x86_64 110/156 2024-04-25T10:28:03.486 INFO:teuthology.orchestra.run.smithi203.stdout: Verifying : libunwind-1.6.2-1.el9.x86_64 111/156 2024-04-25T10:28:03.486 INFO:teuthology.orchestra.run.smithi203.stdout: Verifying : luarocks-3.9.2-1.el9.noarch 112/156 2024-04-25T10:28:03.486 INFO:teuthology.orchestra.run.smithi203.stdout: Verifying : parquet-libs-9.0.0-10.el9.x86_64 113/156 2024-04-25T10:28:03.486 INFO:teuthology.orchestra.run.smithi203.stdout: Verifying : python3-asyncssh-2.13.2-5.el9.noarch 114/156 2024-04-25T10:28:03.486 INFO:teuthology.orchestra.run.smithi203.stdout: Verifying : python3-bcrypt-3.2.2-1.el9.x86_64 115/156 2024-04-25T10:28:03.486 INFO:teuthology.orchestra.run.smithi203.stdout: Verifying : python3-cachetools-4.2.4-1.el9.noarch 116/156 2024-04-25T10:28:03.486 INFO:teuthology.orchestra.run.smithi203.stdout: Verifying : python3-certifi-2021.10.8-2.el9.noarch 117/156 2024-04-25T10:28:03.486 INFO:teuthology.orchestra.run.smithi203.stdout: Verifying : python3-cheroot-8.6.0-4.el9.noarch 118/156 2024-04-25T10:28:03.486 INFO:teuthology.orchestra.run.smithi203.stdout: Verifying : python3-cherrypy-18.6.1-2.el9.noarch 119/156 2024-04-25T10:28:03.487 INFO:teuthology.orchestra.run.smithi203.stdout: Verifying : python3-cmd2-2.4.2-2.el9.noarch 120/156 2024-04-25T10:28:03.487 INFO:teuthology.orchestra.run.smithi203.stdout: Verifying : python3-colorama-0.4.4-12.el9.noarch 121/156 2024-04-25T10:28:03.487 INFO:teuthology.orchestra.run.smithi203.stdout: Verifying : python3-google-auth-1:2.29.0-1.el9.noarch 122/156 2024-04-25T10:28:03.487 INFO:teuthology.orchestra.run.smithi203.stdout: Verifying : python3-grpcio-1.46.7-10.el9.x86_64 123/156 2024-04-25T10:28:03.487 INFO:teuthology.orchestra.run.smithi203.stdout: Verifying : python3-grpcio-tools-1.46.7-10.el9.x86_64 124/156 2024-04-25T10:28:03.487 INFO:teuthology.orchestra.run.smithi203.stdout: Verifying : python3-jaraco-8.2.1-3.el9.noarch 125/156 2024-04-25T10:28:03.487 INFO:teuthology.orchestra.run.smithi203.stdout: Verifying : python3-jaraco-classes-3.2.1-5.el9.noarch 126/156 2024-04-25T10:28:03.487 INFO:teuthology.orchestra.run.smithi203.stdout: Verifying : python3-jaraco-collections-3.0.0-8.el9.noarch 127/156 2024-04-25T10:28:03.487 INFO:teuthology.orchestra.run.smithi203.stdout: Verifying : python3-jaraco-functools-3.5.0-2.el9.noarch 128/156 2024-04-25T10:28:03.487 INFO:teuthology.orchestra.run.smithi203.stdout: Verifying : python3-jaraco-text-3.2.0-6.el9.noarch 129/156 2024-04-25T10:28:03.487 INFO:teuthology.orchestra.run.smithi203.stdout: Verifying : python3-kubernetes-1:26.1.0-2.el9.noarch 130/156 2024-04-25T10:28:03.487 INFO:teuthology.orchestra.run.smithi203.stdout: Verifying : python3-logutils-0.3.5-21.el9.noarch 131/156 2024-04-25T10:28:03.487 INFO:teuthology.orchestra.run.smithi203.stdout: Verifying : python3-more-itertools-8.12.0-2.el9.noarch 132/156 2024-04-25T10:28:03.487 INFO:teuthology.orchestra.run.smithi203.stdout: Verifying : python3-natsort-7.1.1-5.el9.noarch 133/156 2024-04-25T10:28:03.488 INFO:teuthology.orchestra.run.smithi203.stdout: Verifying : python3-pecan-1.4.2-3.el9.noarch 134/156 2024-04-25T10:28:03.488 INFO:teuthology.orchestra.run.smithi203.stdout: Verifying : python3-portend-3.1.0-2.el9.noarch 135/156 2024-04-25T10:28:03.488 INFO:teuthology.orchestra.run.smithi203.stdout: Verifying : python3-pyOpenSSL-21.0.0-1.el9.noarch 136/156 2024-04-25T10:28:03.488 INFO:teuthology.orchestra.run.smithi203.stdout: Verifying : python3-pyperclip-1.8.2-2.el9.noarch 137/156 2024-04-25T10:28:03.488 INFO:teuthology.orchestra.run.smithi203.stdout: Verifying : python3-repoze-lru-0.7-16.el9.noarch 138/156 2024-04-25T10:28:03.488 INFO:teuthology.orchestra.run.smithi203.stdout: Verifying : python3-routes-2.5.1-5.el9.noarch 139/156 2024-04-25T10:28:03.488 INFO:teuthology.orchestra.run.smithi203.stdout: Verifying : python3-rsa-4.9-2.el9.noarch 140/156 2024-04-25T10:28:03.488 INFO:teuthology.orchestra.run.smithi203.stdout: Verifying : python3-tempora-5.0.0-2.el9.noarch 141/156 2024-04-25T10:28:03.488 INFO:teuthology.orchestra.run.smithi203.stdout: Verifying : python3-typing-extensions-4.4.0-2.el9.noarch 142/156 2024-04-25T10:28:03.488 INFO:teuthology.orchestra.run.smithi203.stdout: Verifying : python3-webob-1.8.7-6.el9.noarch 143/156 2024-04-25T10:28:03.488 INFO:teuthology.orchestra.run.smithi203.stdout: Verifying : python3-websocket-client-1.2.3-2.el9.noarch 144/156 2024-04-25T10:28:03.488 INFO:teuthology.orchestra.run.smithi203.stdout: Verifying : python3-werkzeug-2.0.3-3.el9.1.noarch 145/156 2024-04-25T10:28:03.488 INFO:teuthology.orchestra.run.smithi203.stdout: Verifying : python3-zc-lockfile-2.0-10.el9.noarch 146/156 2024-04-25T10:28:03.488 INFO:teuthology.orchestra.run.smithi203.stdout: Verifying : re2-1:20211101-3.el9.x86_64 147/156 2024-04-25T10:28:03.489 INFO:teuthology.orchestra.run.smithi203.stdout: Verifying : thrift-0.15.0-2.el9.x86_64 148/156 2024-04-25T10:28:03.489 INFO:teuthology.orchestra.run.smithi203.stdout: Verifying : librados2-2:19.0.0-3291.g994e2f20.el9.x86_64 149/156 2024-04-25T10:28:03.489 INFO:teuthology.orchestra.run.smithi203.stdout: Verifying : librados2-2:16.2.4-5.el9.x86_64 150/156 2024-04-25T10:28:03.489 INFO:teuthology.orchestra.run.smithi203.stdout: Verifying : librbd1-2:19.0.0-3291.g994e2f20.el9.x86_64 151/156 2024-04-25T10:28:03.489 INFO:teuthology.orchestra.run.smithi203.stdout: Verifying : librbd1-2:16.2.4-5.el9.x86_64 152/156 2024-04-25T10:28:03.489 INFO:teuthology.orchestra.run.smithi203.stdout: Verifying : openssl-1:3.2.1-1.el9.x86_64 153/156 2024-04-25T10:28:03.489 INFO:teuthology.orchestra.run.smithi203.stdout: Verifying : openssl-1:3.0.7-27.el9.x86_64 154/156 2024-04-25T10:28:03.489 INFO:teuthology.orchestra.run.smithi203.stdout: Verifying : openssl-libs-1:3.2.1-1.el9.x86_64 155/156 2024-04-25T10:28:05.451 INFO:teuthology.orchestra.run.smithi203.stdout: Verifying : openssl-libs-1:3.0.7-27.el9.x86_64 156/156 2024-04-25T10:28:05.452 INFO:teuthology.orchestra.run.smithi203.stdout: 2024-04-25T10:28:05.452 INFO:teuthology.orchestra.run.smithi203.stdout:Upgraded: 2024-04-25T10:28:05.452 INFO:teuthology.orchestra.run.smithi203.stdout: librados2-2:19.0.0-3291.g994e2f20.el9.x86_64 2024-04-25T10:28:05.452 INFO:teuthology.orchestra.run.smithi203.stdout: librbd1-2:19.0.0-3291.g994e2f20.el9.x86_64 2024-04-25T10:28:05.452 INFO:teuthology.orchestra.run.smithi203.stdout: openssl-1:3.2.1-1.el9.x86_64 2024-04-25T10:28:05.452 INFO:teuthology.orchestra.run.smithi203.stdout: openssl-libs-1:3.2.1-1.el9.x86_64 2024-04-25T10:28:05.452 INFO:teuthology.orchestra.run.smithi203.stdout:Installed: 2024-04-25T10:28:05.452 INFO:teuthology.orchestra.run.smithi203.stdout: abseil-cpp-20211102.0-3.el9.x86_64 2024-04-25T10:28:05.452 INFO:teuthology.orchestra.run.smithi203.stdout: automake-1.16.2-8.el9.noarch 2024-04-25T10:28:05.452 INFO:teuthology.orchestra.run.smithi203.stdout: bison-3.7.4-5.el9.x86_64 2024-04-25T10:28:05.453 INFO:teuthology.orchestra.run.smithi203.stdout: boost-program-options-1.75.0-8.el9.x86_64 2024-04-25T10:28:05.453 INFO:teuthology.orchestra.run.smithi203.stdout: ceph-2:19.0.0-3291.g994e2f20.el9.x86_64 2024-04-25T10:28:05.453 INFO:teuthology.orchestra.run.smithi203.stdout: ceph-base-2:19.0.0-3291.g994e2f20.el9.x86_64 2024-04-25T10:28:05.453 INFO:teuthology.orchestra.run.smithi203.stdout: ceph-common-2:19.0.0-3291.g994e2f20.el9.x86_64 2024-04-25T10:28:05.453 INFO:teuthology.orchestra.run.smithi203.stdout: ceph-fuse-2:19.0.0-3291.g994e2f20.el9.x86_64 2024-04-25T10:28:05.453 INFO:teuthology.orchestra.run.smithi203.stdout: ceph-grafana-dashboards-2:19.0.0-3291.g994e2f20.el9.noarch 2024-04-25T10:28:05.453 INFO:teuthology.orchestra.run.smithi203.stdout: ceph-immutable-object-cache-2:19.0.0-3291.g994e2f20.el9.x86_64 2024-04-25T10:28:05.453 INFO:teuthology.orchestra.run.smithi203.stdout: ceph-mds-2:19.0.0-3291.g994e2f20.el9.x86_64 2024-04-25T10:28:05.453 INFO:teuthology.orchestra.run.smithi203.stdout: ceph-mgr-2:19.0.0-3291.g994e2f20.el9.x86_64 2024-04-25T10:28:05.453 INFO:teuthology.orchestra.run.smithi203.stdout: ceph-mgr-cephadm-2:19.0.0-3291.g994e2f20.el9.noarch 2024-04-25T10:28:05.453 INFO:teuthology.orchestra.run.smithi203.stdout: ceph-mgr-dashboard-2:19.0.0-3291.g994e2f20.el9.noarch 2024-04-25T10:28:05.453 INFO:teuthology.orchestra.run.smithi203.stdout: ceph-mgr-diskprediction-local-2:19.0.0-3291.g994e2f20.el9.noarch 2024-04-25T10:28:05.454 INFO:teuthology.orchestra.run.smithi203.stdout: ceph-mgr-modules-core-2:19.0.0-3291.g994e2f20.el9.noarch 2024-04-25T10:28:05.454 INFO:teuthology.orchestra.run.smithi203.stdout: ceph-mgr-rook-2:19.0.0-3291.g994e2f20.el9.noarch 2024-04-25T10:28:05.454 INFO:teuthology.orchestra.run.smithi203.stdout: ceph-mon-2:19.0.0-3291.g994e2f20.el9.x86_64 2024-04-25T10:28:05.454 INFO:teuthology.orchestra.run.smithi203.stdout: ceph-osd-2:19.0.0-3291.g994e2f20.el9.x86_64 2024-04-25T10:28:05.454 INFO:teuthology.orchestra.run.smithi203.stdout: ceph-prometheus-alerts-2:19.0.0-3291.g994e2f20.el9.noarch 2024-04-25T10:28:05.454 INFO:teuthology.orchestra.run.smithi203.stdout: ceph-radosgw-2:19.0.0-3291.g994e2f20.el9.x86_64 2024-04-25T10:28:05.454 INFO:teuthology.orchestra.run.smithi203.stdout: ceph-selinux-2:19.0.0-3291.g994e2f20.el9.x86_64 2024-04-25T10:28:05.454 INFO:teuthology.orchestra.run.smithi203.stdout: ceph-test-2:19.0.0-3291.g994e2f20.el9.x86_64 2024-04-25T10:28:05.454 INFO:teuthology.orchestra.run.smithi203.stdout: ceph-volume-2:19.0.0-3291.g994e2f20.el9.noarch 2024-04-25T10:28:05.454 INFO:teuthology.orchestra.run.smithi203.stdout: cephadm-2:19.0.0-3291.g994e2f20.el9.noarch 2024-04-25T10:28:05.454 INFO:teuthology.orchestra.run.smithi203.stdout: cephfs-mirror-2:19.0.0-3291.g994e2f20.el9.x86_64 2024-04-25T10:28:05.455 INFO:teuthology.orchestra.run.smithi203.stdout: cephfs-shell-2:19.0.0-3291.g994e2f20.el9.x86_64 2024-04-25T10:28:05.455 INFO:teuthology.orchestra.run.smithi203.stdout: cephfs-top-2:19.0.0-3291.g994e2f20.el9.noarch 2024-04-25T10:28:05.455 INFO:teuthology.orchestra.run.smithi203.stdout: elfutils-libelf-devel-0.190-2.el9.x86_64 2024-04-25T10:28:05.455 INFO:teuthology.orchestra.run.smithi203.stdout: flex-2.6.4-9.el9.x86_64 2024-04-25T10:28:05.455 INFO:teuthology.orchestra.run.smithi203.stdout: flexiblas-3.0.4-8.el9.x86_64 2024-04-25T10:28:05.455 INFO:teuthology.orchestra.run.smithi203.stdout: flexiblas-netlib-3.0.4-8.el9.x86_64 2024-04-25T10:28:05.455 INFO:teuthology.orchestra.run.smithi203.stdout: flexiblas-openblas-openmp-3.0.4-8.el9.x86_64 2024-04-25T10:28:05.455 INFO:teuthology.orchestra.run.smithi203.stdout: fuse-2.9.9-15.el9.x86_64 2024-04-25T10:28:05.455 INFO:teuthology.orchestra.run.smithi203.stdout: gperftools-libs-2.9.1-2.el9.x86_64 2024-04-25T10:28:05.455 INFO:teuthology.orchestra.run.smithi203.stdout: grpc-data-1.46.7-10.el9.noarch 2024-04-25T10:28:05.455 INFO:teuthology.orchestra.run.smithi203.stdout: libaio-devel-0.3.111-13.el9.x86_64 2024-04-25T10:28:05.455 INFO:teuthology.orchestra.run.smithi203.stdout: libarrow-9.0.0-10.el9.x86_64 2024-04-25T10:28:05.455 INFO:teuthology.orchestra.run.smithi203.stdout: libarrow-doc-9.0.0-10.el9.noarch 2024-04-25T10:28:05.456 INFO:teuthology.orchestra.run.smithi203.stdout: libcephfs-devel-2:19.0.0-3291.g994e2f20.el9.x86_64 2024-04-25T10:28:05.456 INFO:teuthology.orchestra.run.smithi203.stdout: libcephfs2-2:19.0.0-3291.g994e2f20.el9.x86_64 2024-04-25T10:28:05.456 INFO:teuthology.orchestra.run.smithi203.stdout: libcephsqlite-2:19.0.0-3291.g994e2f20.el9.x86_64 2024-04-25T10:28:05.456 INFO:teuthology.orchestra.run.smithi203.stdout: libgfortran-11.4.1-3.el9.x86_64 2024-04-25T10:28:05.456 INFO:teuthology.orchestra.run.smithi203.stdout: liboath-2.6.7-2.el9.x86_64 2024-04-25T10:28:05.456 INFO:teuthology.orchestra.run.smithi203.stdout: libpmemobj-1.12.1-1.el9.x86_64 2024-04-25T10:28:05.456 INFO:teuthology.orchestra.run.smithi203.stdout: libquadmath-11.4.1-3.el9.x86_64 2024-04-25T10:28:05.456 INFO:teuthology.orchestra.run.smithi203.stdout: librabbitmq-0.11.0-7.el9.x86_64 2024-04-25T10:28:05.456 INFO:teuthology.orchestra.run.smithi203.stdout: librados-devel-2:19.0.0-3291.g994e2f20.el9.x86_64 2024-04-25T10:28:05.456 INFO:teuthology.orchestra.run.smithi203.stdout: libradosstriper1-2:19.0.0-3291.g994e2f20.el9.x86_64 2024-04-25T10:28:05.456 INFO:teuthology.orchestra.run.smithi203.stdout: librdkafka-1.6.1-102.el9.x86_64 2024-04-25T10:28:05.456 INFO:teuthology.orchestra.run.smithi203.stdout: librgw2-2:19.0.0-3291.g994e2f20.el9.x86_64 2024-04-25T10:28:05.456 INFO:teuthology.orchestra.run.smithi203.stdout: libtool-2.4.6-45.el9.x86_64 2024-04-25T10:28:05.457 INFO:teuthology.orchestra.run.smithi203.stdout: libunwind-1.6.2-1.el9.x86_64 2024-04-25T10:28:05.457 INFO:teuthology.orchestra.run.smithi203.stdout: libuuid-devel-2.37.4-18.el9.x86_64 2024-04-25T10:28:05.457 INFO:teuthology.orchestra.run.smithi203.stdout: libzstd-devel-1.5.1-2.el9.x86_64 2024-04-25T10:28:05.457 INFO:teuthology.orchestra.run.smithi203.stdout: lttng-ust-2.12.0-6.el9.x86_64 2024-04-25T10:28:05.457 INFO:teuthology.orchestra.run.smithi203.stdout: lua-5.4.4-4.el9.x86_64 2024-04-25T10:28:05.457 INFO:teuthology.orchestra.run.smithi203.stdout: lua-devel-5.4.4-4.el9.x86_64 2024-04-25T10:28:05.457 INFO:teuthology.orchestra.run.smithi203.stdout: luarocks-3.9.2-1.el9.noarch 2024-04-25T10:28:05.457 INFO:teuthology.orchestra.run.smithi203.stdout: openblas-0.3.21-2.el9.x86_64 2024-04-25T10:28:05.457 INFO:teuthology.orchestra.run.smithi203.stdout: openblas-openmp-0.3.21-2.el9.x86_64 2024-04-25T10:28:05.457 INFO:teuthology.orchestra.run.smithi203.stdout: openssl-devel-1:3.2.1-1.el9.x86_64 2024-04-25T10:28:05.457 INFO:teuthology.orchestra.run.smithi203.stdout: parquet-libs-9.0.0-10.el9.x86_64 2024-04-25T10:28:05.457 INFO:teuthology.orchestra.run.smithi203.stdout: postgresql-13.14-1.el9.x86_64 2024-04-25T10:28:05.457 INFO:teuthology.orchestra.run.smithi203.stdout: postgresql-contrib-13.14-1.el9.x86_64 2024-04-25T10:28:05.457 INFO:teuthology.orchestra.run.smithi203.stdout: postgresql-private-libs-13.14-1.el9.x86_64 2024-04-25T10:28:05.458 INFO:teuthology.orchestra.run.smithi203.stdout: postgresql-server-13.14-1.el9.x86_64 2024-04-25T10:28:05.458 INFO:teuthology.orchestra.run.smithi203.stdout: protobuf-3.14.0-13.el9.x86_64 2024-04-25T10:28:05.458 INFO:teuthology.orchestra.run.smithi203.stdout: protobuf-compiler-3.14.0-13.el9.x86_64 2024-04-25T10:28:05.458 INFO:teuthology.orchestra.run.smithi203.stdout: python3-asyncssh-2.13.2-5.el9.noarch 2024-04-25T10:28:05.458 INFO:teuthology.orchestra.run.smithi203.stdout: python3-attrs-20.3.0-7.el9.noarch 2024-04-25T10:28:05.458 INFO:teuthology.orchestra.run.smithi203.stdout: python3-babel-2.9.1-2.el9.noarch 2024-04-25T10:28:05.458 INFO:teuthology.orchestra.run.smithi203.stdout: python3-bcrypt-3.2.2-1.el9.x86_64 2024-04-25T10:28:05.458 INFO:teuthology.orchestra.run.smithi203.stdout: python3-cachetools-4.2.4-1.el9.noarch 2024-04-25T10:28:05.458 INFO:teuthology.orchestra.run.smithi203.stdout: python3-ceph-argparse-2:19.0.0-3291.g994e2f20.el9.x86_64 2024-04-25T10:28:05.458 INFO:teuthology.orchestra.run.smithi203.stdout: python3-ceph-common-2:19.0.0-3291.g994e2f20.el9.x86_64 2024-04-25T10:28:05.458 INFO:teuthology.orchestra.run.smithi203.stdout: python3-cephfs-2:19.0.0-3291.g994e2f20.el9.x86_64 2024-04-25T10:28:05.458 INFO:teuthology.orchestra.run.smithi203.stdout: python3-certifi-2021.10.8-2.el9.noarch 2024-04-25T10:28:05.459 INFO:teuthology.orchestra.run.smithi203.stdout: python3-cffi-1.14.5-5.el9.x86_64 2024-04-25T10:28:05.459 INFO:teuthology.orchestra.run.smithi203.stdout: python3-cheroot-8.6.0-4.el9.noarch 2024-04-25T10:28:05.459 INFO:teuthology.orchestra.run.smithi203.stdout: python3-cherrypy-18.6.1-2.el9.noarch 2024-04-25T10:28:05.459 INFO:teuthology.orchestra.run.smithi203.stdout: python3-cmd2-2.4.2-2.el9.noarch 2024-04-25T10:28:05.459 INFO:teuthology.orchestra.run.smithi203.stdout: python3-colorama-0.4.4-12.el9.noarch 2024-04-25T10:28:05.459 INFO:teuthology.orchestra.run.smithi203.stdout: python3-cryptography-36.0.1-4.el9.x86_64 2024-04-25T10:28:05.459 INFO:teuthology.orchestra.run.smithi203.stdout: python3-devel-3.9.18-3.el9.x86_64 2024-04-25T10:28:05.459 INFO:teuthology.orchestra.run.smithi203.stdout: python3-google-auth-1:2.29.0-1.el9.noarch 2024-04-25T10:28:05.459 INFO:teuthology.orchestra.run.smithi203.stdout: python3-grpcio-1.46.7-10.el9.x86_64 2024-04-25T10:28:05.459 INFO:teuthology.orchestra.run.smithi203.stdout: python3-grpcio-tools-1.46.7-10.el9.x86_64 2024-04-25T10:28:05.459 INFO:teuthology.orchestra.run.smithi203.stdout: python3-jaraco-8.2.1-3.el9.noarch 2024-04-25T10:28:05.459 INFO:teuthology.orchestra.run.smithi203.stdout: python3-jaraco-classes-3.2.1-5.el9.noarch 2024-04-25T10:28:05.459 INFO:teuthology.orchestra.run.smithi203.stdout: python3-jaraco-collections-3.0.0-8.el9.noarch 2024-04-25T10:28:05.459 INFO:teuthology.orchestra.run.smithi203.stdout: python3-jaraco-functools-3.5.0-2.el9.noarch 2024-04-25T10:28:05.460 INFO:teuthology.orchestra.run.smithi203.stdout: python3-jaraco-text-3.2.0-6.el9.noarch 2024-04-25T10:28:05.460 INFO:teuthology.orchestra.run.smithi203.stdout: python3-jinja2-2.11.3-5.el9.noarch 2024-04-25T10:28:05.460 INFO:teuthology.orchestra.run.smithi203.stdout: python3-jsonpatch-1.21-16.el9.noarch 2024-04-25T10:28:05.460 INFO:teuthology.orchestra.run.smithi203.stdout: python3-jsonpointer-2.0-4.el9.noarch 2024-04-25T10:28:05.460 INFO:teuthology.orchestra.run.smithi203.stdout: python3-kubernetes-1:26.1.0-2.el9.noarch 2024-04-25T10:28:05.460 INFO:teuthology.orchestra.run.smithi203.stdout: python3-logutils-0.3.5-21.el9.noarch 2024-04-25T10:28:05.460 INFO:teuthology.orchestra.run.smithi203.stdout: python3-mako-1.1.4-6.el9.noarch 2024-04-25T10:28:05.460 INFO:teuthology.orchestra.run.smithi203.stdout: python3-markupsafe-1.1.1-12.el9.x86_64 2024-04-25T10:28:05.460 INFO:teuthology.orchestra.run.smithi203.stdout: python3-more-itertools-8.12.0-2.el9.noarch 2024-04-25T10:28:05.460 INFO:teuthology.orchestra.run.smithi203.stdout: python3-natsort-7.1.1-5.el9.noarch 2024-04-25T10:28:05.460 INFO:teuthology.orchestra.run.smithi203.stdout: python3-numpy-1:1.20.1-5.el9.x86_64 2024-04-25T10:28:05.460 INFO:teuthology.orchestra.run.smithi203.stdout: python3-numpy-f2py-1:1.20.1-5.el9.x86_64 2024-04-25T10:28:05.460 INFO:teuthology.orchestra.run.smithi203.stdout: python3-oauthlib-3.1.1-5.el9.noarch 2024-04-25T10:28:05.460 INFO:teuthology.orchestra.run.smithi203.stdout: python3-packaging-20.9-5.el9.noarch 2024-04-25T10:28:05.461 INFO:teuthology.orchestra.run.smithi203.stdout: python3-pecan-1.4.2-3.el9.noarch 2024-04-25T10:28:05.461 INFO:teuthology.orchestra.run.smithi203.stdout: python3-ply-3.11-14.el9.noarch 2024-04-25T10:28:05.461 INFO:teuthology.orchestra.run.smithi203.stdout: python3-portend-3.1.0-2.el9.noarch 2024-04-25T10:28:05.461 INFO:teuthology.orchestra.run.smithi203.stdout: python3-prettytable-0.7.2-27.el9.noarch 2024-04-25T10:28:05.461 INFO:teuthology.orchestra.run.smithi203.stdout: python3-protobuf-3.14.0-13.el9.noarch 2024-04-25T10:28:05.461 INFO:teuthology.orchestra.run.smithi203.stdout: python3-pyOpenSSL-21.0.0-1.el9.noarch 2024-04-25T10:28:05.461 INFO:teuthology.orchestra.run.smithi203.stdout: python3-pyasn1-0.4.8-6.el9.noarch 2024-04-25T10:28:05.461 INFO:teuthology.orchestra.run.smithi203.stdout: python3-pyasn1-modules-0.4.8-6.el9.noarch 2024-04-25T10:28:05.461 INFO:teuthology.orchestra.run.smithi203.stdout: python3-pycparser-2.20-6.el9.noarch 2024-04-25T10:28:05.461 INFO:teuthology.orchestra.run.smithi203.stdout: python3-pyperclip-1.8.2-2.el9.noarch 2024-04-25T10:28:05.461 INFO:teuthology.orchestra.run.smithi203.stdout: python3-pytz-2021.1-5.el9.noarch 2024-04-25T10:28:05.461 INFO:teuthology.orchestra.run.smithi203.stdout: python3-rados-2:19.0.0-3291.g994e2f20.el9.x86_64 2024-04-25T10:28:05.461 INFO:teuthology.orchestra.run.smithi203.stdout: python3-rbd-2:19.0.0-3291.g994e2f20.el9.x86_64 2024-04-25T10:28:05.461 INFO:teuthology.orchestra.run.smithi203.stdout: python3-repoze-lru-0.7-16.el9.noarch 2024-04-25T10:28:05.462 INFO:teuthology.orchestra.run.smithi203.stdout: python3-requests-2.25.1-8.el9.noarch 2024-04-25T10:28:05.462 INFO:teuthology.orchestra.run.smithi203.stdout: python3-requests-oauthlib-1.3.0-12.el9.noarch 2024-04-25T10:28:05.462 INFO:teuthology.orchestra.run.smithi203.stdout: python3-rgw-2:19.0.0-3291.g994e2f20.el9.x86_64 2024-04-25T10:28:05.462 INFO:teuthology.orchestra.run.smithi203.stdout: python3-routes-2.5.1-5.el9.noarch 2024-04-25T10:28:05.462 INFO:teuthology.orchestra.run.smithi203.stdout: python3-rsa-4.9-2.el9.noarch 2024-04-25T10:28:05.462 INFO:teuthology.orchestra.run.smithi203.stdout: python3-scipy-1.6.2-8.el9.x86_64 2024-04-25T10:28:05.462 INFO:teuthology.orchestra.run.smithi203.stdout: python3-tempora-5.0.0-2.el9.noarch 2024-04-25T10:28:05.462 INFO:teuthology.orchestra.run.smithi203.stdout: python3-toml-0.10.2-6.el9.noarch 2024-04-25T10:28:05.462 INFO:teuthology.orchestra.run.smithi203.stdout: python3-typing-extensions-4.4.0-2.el9.noarch 2024-04-25T10:28:05.462 INFO:teuthology.orchestra.run.smithi203.stdout: python3-urllib3-1.26.5-5.el9.noarch 2024-04-25T10:28:05.462 INFO:teuthology.orchestra.run.smithi203.stdout: python3-wcwidth-0.2.5-8.el9.noarch 2024-04-25T10:28:05.462 INFO:teuthology.orchestra.run.smithi203.stdout: python3-webob-1.8.7-6.el9.noarch 2024-04-25T10:28:05.462 INFO:teuthology.orchestra.run.smithi203.stdout: python3-websocket-client-1.2.3-2.el9.noarch 2024-04-25T10:28:05.463 INFO:teuthology.orchestra.run.smithi203.stdout: python3-werkzeug-2.0.3-3.el9.1.noarch 2024-04-25T10:28:05.463 INFO:teuthology.orchestra.run.smithi203.stdout: python3-zc-lockfile-2.0-10.el9.noarch 2024-04-25T10:28:05.463 INFO:teuthology.orchestra.run.smithi203.stdout: qatlib-24.02.0-1.el9.x86_64 2024-04-25T10:28:05.463 INFO:teuthology.orchestra.run.smithi203.stdout: qatlib-service-24.02.0-1.el9.x86_64 2024-04-25T10:28:05.463 INFO:teuthology.orchestra.run.smithi203.stdout: qatzip-libs-1.2.0-1.el9.x86_64 2024-04-25T10:28:05.463 INFO:teuthology.orchestra.run.smithi203.stdout: rbd-fuse-2:19.0.0-3291.g994e2f20.el9.x86_64 2024-04-25T10:28:05.463 INFO:teuthology.orchestra.run.smithi203.stdout: rbd-mirror-2:19.0.0-3291.g994e2f20.el9.x86_64 2024-04-25T10:28:05.463 INFO:teuthology.orchestra.run.smithi203.stdout: rbd-nbd-2:19.0.0-3291.g994e2f20.el9.x86_64 2024-04-25T10:28:05.463 INFO:teuthology.orchestra.run.smithi203.stdout: re2-1:20211101-3.el9.x86_64 2024-04-25T10:28:05.463 INFO:teuthology.orchestra.run.smithi203.stdout: socat-1.7.4.1-5.el9.x86_64 2024-04-25T10:28:05.463 INFO:teuthology.orchestra.run.smithi203.stdout: thrift-0.15.0-2.el9.x86_64 2024-04-25T10:28:05.463 INFO:teuthology.orchestra.run.smithi203.stdout: uuid-1.6.2-55.el9.x86_64 2024-04-25T10:28:05.463 INFO:teuthology.orchestra.run.smithi203.stdout: xfsprogs-devel-6.3.0-1.el9.x86_64 2024-04-25T10:28:05.463 INFO:teuthology.orchestra.run.smithi203.stdout: xmlstarlet-1.6.1-20.el9.x86_64 2024-04-25T10:28:05.464 INFO:teuthology.orchestra.run.smithi203.stdout: zlib-devel-1.2.11-41.el9.x86_64 2024-04-25T10:28:05.464 INFO:teuthology.orchestra.run.smithi203.stdout: 2024-04-25T10:28:05.464 INFO:teuthology.orchestra.run.smithi203.stdout:Complete! 2024-04-25T10:28:05.871 DEBUG:teuthology.parallel:result is None 2024-04-25T10:28:05.872 DEBUG:teuthology.packaging:Querying https://shaman.ceph.com/api/search?status=ready&project=ceph&flavor=default&distros=centos%2F9%2Fx86_64&sha1=994e2f20102a20fa9a27966c14b625cc01884df8 2024-04-25T10:28:06.030 DEBUG:teuthology.orchestra.run.smithi037:> rpm -q ceph --qf '%{VERSION}-%{RELEASE}' 2024-04-25T10:28:06.057 INFO:teuthology.orchestra.run.smithi037.stdout:19.0.0-3291.g994e2f20.el9 2024-04-25T10:28:06.057 INFO:teuthology.packaging:The installed version of ceph is 19.0.0-3291.g994e2f20.el9 2024-04-25T10:28:06.057 INFO:teuthology.task.install:The correct ceph version 19.0.0-3291.g994e2f20 is installed. 2024-04-25T10:28:06.059 DEBUG:teuthology.packaging:Querying https://shaman.ceph.com/api/search?status=ready&project=ceph&flavor=default&distros=centos%2F9%2Fx86_64&sha1=994e2f20102a20fa9a27966c14b625cc01884df8 2024-04-25T10:28:06.196 DEBUG:teuthology.orchestra.run.smithi203:> rpm -q ceph --qf '%{VERSION}-%{RELEASE}' 2024-04-25T10:28:06.218 INFO:teuthology.orchestra.run.smithi203.stdout:19.0.0-3291.g994e2f20.el9 2024-04-25T10:28:06.218 INFO:teuthology.packaging:The installed version of ceph is 19.0.0-3291.g994e2f20.el9 2024-04-25T10:28:06.218 INFO:teuthology.task.install:The correct ceph version 19.0.0-3291.g994e2f20 is installed. 2024-04-25T10:28:06.220 INFO:teuthology.task.install.util:Shipping valgrind.supp... 2024-04-25T10:28:06.220 DEBUG:teuthology.orchestra.run.smithi037:> set -ex 2024-04-25T10:28:06.220 DEBUG:teuthology.orchestra.run.smithi037:> sudo dd of=/home/ubuntu/cephtest/valgrind.supp 2024-04-25T10:28:06.253 DEBUG:teuthology.orchestra.run.smithi203:> set -ex 2024-04-25T10:28:06.253 DEBUG:teuthology.orchestra.run.smithi203:> sudo dd of=/home/ubuntu/cephtest/valgrind.supp 2024-04-25T10:28:06.283 INFO:teuthology.task.install.util:Shipping 'daemon-helper'... 2024-04-25T10:28:06.284 DEBUG:teuthology.orchestra.run.smithi037:> set -ex 2024-04-25T10:28:06.284 DEBUG:teuthology.orchestra.run.smithi037:> sudo dd of=/usr/bin/daemon-helper 2024-04-25T10:28:06.322 DEBUG:teuthology.orchestra.run.smithi037:> sudo chmod a=rx -- /usr/bin/daemon-helper 2024-04-25T10:28:06.390 DEBUG:teuthology.orchestra.run.smithi203:> set -ex 2024-04-25T10:28:06.390 DEBUG:teuthology.orchestra.run.smithi203:> sudo dd of=/usr/bin/daemon-helper 2024-04-25T10:28:06.415 DEBUG:teuthology.orchestra.run.smithi203:> sudo chmod a=rx -- /usr/bin/daemon-helper 2024-04-25T10:28:06.481 INFO:teuthology.task.install.util:Shipping 'adjust-ulimits'... 2024-04-25T10:28:06.481 DEBUG:teuthology.orchestra.run.smithi037:> set -ex 2024-04-25T10:28:06.481 DEBUG:teuthology.orchestra.run.smithi037:> sudo dd of=/usr/bin/adjust-ulimits 2024-04-25T10:28:06.510 DEBUG:teuthology.orchestra.run.smithi037:> sudo chmod a=rx -- /usr/bin/adjust-ulimits 2024-04-25T10:28:06.578 DEBUG:teuthology.orchestra.run.smithi203:> set -ex 2024-04-25T10:28:06.578 DEBUG:teuthology.orchestra.run.smithi203:> sudo dd of=/usr/bin/adjust-ulimits 2024-04-25T10:28:06.601 DEBUG:teuthology.orchestra.run.smithi203:> sudo chmod a=rx -- /usr/bin/adjust-ulimits 2024-04-25T10:28:06.669 INFO:teuthology.task.install.util:Shipping 'stdin-killer'... 2024-04-25T10:28:06.669 DEBUG:teuthology.orchestra.run.smithi037:> set -ex 2024-04-25T10:28:06.669 DEBUG:teuthology.orchestra.run.smithi037:> sudo dd of=/usr/bin/stdin-killer 2024-04-25T10:28:06.696 DEBUG:teuthology.orchestra.run.smithi037:> sudo chmod a=rx -- /usr/bin/stdin-killer 2024-04-25T10:28:06.764 DEBUG:teuthology.orchestra.run.smithi203:> set -ex 2024-04-25T10:28:06.764 DEBUG:teuthology.orchestra.run.smithi203:> sudo dd of=/usr/bin/stdin-killer 2024-04-25T10:28:06.787 DEBUG:teuthology.orchestra.run.smithi203:> sudo chmod a=rx -- /usr/bin/stdin-killer 2024-04-25T10:28:06.853 INFO:teuthology.run_tasks:Running task ceph... 2024-04-25T10:28:06.945 INFO:tasks.ceph:Making ceph log dir writeable by non-root... 2024-04-25T10:28:06.945 DEBUG:teuthology.orchestra.run.smithi037:> sudo chmod 777 /var/log/ceph 2024-04-25T10:28:06.948 DEBUG:teuthology.orchestra.run.smithi203:> sudo chmod 777 /var/log/ceph 2024-04-25T10:28:06.973 INFO:tasks.ceph:Disabling ceph logrotate... 2024-04-25T10:28:06.973 DEBUG:teuthology.orchestra.run.smithi037:> sudo rm -f -- /etc/logrotate.d/ceph 2024-04-25T10:28:07.016 DEBUG:teuthology.orchestra.run.smithi203:> sudo rm -f -- /etc/logrotate.d/ceph 2024-04-25T10:28:07.041 INFO:tasks.ceph:Creating extra log directories... 2024-04-25T10:28:07.041 DEBUG:teuthology.orchestra.run.smithi037:> sudo install -d -m0777 -- /var/log/ceph/valgrind /var/log/ceph/profiling-logger 2024-04-25T10:28:07.083 DEBUG:teuthology.orchestra.run.smithi203:> sudo install -d -m0777 -- /var/log/ceph/valgrind /var/log/ceph/profiling-logger 2024-04-25T10:28:07.114 INFO:tasks.ceph:Setting up log rotation with {'ceph-mds': '10G', 'ceph-osd': '10G'} 2024-04-25T10:28:07.114 INFO:tasks.ceph:writing logrotate stanza for ceph-mds 2024-04-25T10:28:07.115 INFO:tasks.ceph:writing logrotate stanza for ceph-osd 2024-04-25T10:28:07.115 DEBUG:teuthology.orchestra.run.smithi037:> set -ex 2024-04-25T10:28:07.115 DEBUG:teuthology.orchestra.run.smithi037:> dd of=/home/ubuntu/cephtest/logrotate.ceph-test.conf 2024-04-25T10:28:07.164 DEBUG:teuthology.orchestra.run.smithi037:> sudo mv /home/ubuntu/cephtest/logrotate.ceph-test.conf /etc/logrotate.d/ceph-test.conf && sudo chmod 0644 /etc/logrotate.d/ceph-test.conf && sudo chown root.root /etc/logrotate.d/ceph-test.conf 2024-04-25T10:28:07.277 DEBUG:teuthology.orchestra.run.smithi037:> sudo chcon system_u:object_r:etc_t:s0 /etc/logrotate.d/ceph-test.conf 2024-04-25T10:28:07.302 DEBUG:teuthology.orchestra.run.smithi203:> set -ex 2024-04-25T10:28:07.302 DEBUG:teuthology.orchestra.run.smithi203:> dd of=/home/ubuntu/cephtest/logrotate.ceph-test.conf 2024-04-25T10:28:07.317 DEBUG:teuthology.orchestra.run.smithi203:> sudo mv /home/ubuntu/cephtest/logrotate.ceph-test.conf /etc/logrotate.d/ceph-test.conf && sudo chmod 0644 /etc/logrotate.d/ceph-test.conf && sudo chown root.root /etc/logrotate.d/ceph-test.conf 2024-04-25T10:28:07.425 DEBUG:teuthology.orchestra.run.smithi203:> sudo chcon system_u:object_r:etc_t:s0 /etc/logrotate.d/ceph-test.conf 2024-04-25T10:28:07.469 INFO:tasks.ceph:Creating ceph cluster ceph... 2024-04-25T10:28:07.469 INFO:tasks.ceph:config {'conf': {'client': {'client mount timeout': 600, 'debug client': 20, 'debug ms': 1, 'pid file': '', 'rados mon op timeout': 900, 'rados osd op timeout': 900}, 'mds': {'debug mds': 20, 'debug mds balancer': 20, 'debug ms': 1, 'mds debug frag': True, 'mds debug scatterstat': True, 'mds op complaint time': 180, 'mds verify scatter': True, 'osd op complaint time': 180, 'rados mon op timeout': 900, 'rados osd op timeout': 900}, 'mgr': {'debug client': 20, 'debug mgr': 20, 'debug ms': 1}, 'mon': {'debug mon': 20, 'debug ms': 1, 'debug paxos': 20, 'mon down mkfs grace': 300, 'mon op complaint time': 120}, 'osd': {'bdev async discard': True, 'bdev enable discard': True, 'bluestore allocator': 'bitmap', 'bluestore block size': 96636764160, 'bluestore fsck on mount': True, 'debug bluefs': '1/20', 'debug bluestore': '1/20', 'debug ms': 1, 'debug osd': 20, 'debug rocksdb': '4/10', 'mon osd backfillfull_ratio': 0.85, 'mon osd full ratio': 0.9, 'mon osd nearfull ratio': 0.8, 'osd failsafe full ratio': 0.95, 'osd objectstore': 'bluestore', 'osd op complaint time': 180}}, 'fs': 'xfs', 'mkfs_options': None, 'mount_options': None, 'skip_mgr_daemons': False, 'log_ignorelist': ['\\(MDS_ALL_DOWN\\)', '\\(MDS_UP_LESS_THAN_MAX\\)', 'FS_DEGRADED', 'FS_INLINE_DATA_DEPRECATED', 'FS_WITH_FAILED_MDS', 'MDS_ALL_DOWN', 'MDS_DAMAGE', 'MDS_DEGRADED', 'MDS_FAILED', 'MDS_INSUFFICIENT_STANDBY', 'MDS_UP_LESS_THAN_MAX', 'filesystem is online with fewer MDS than max_mds', 'POOL_APP_NOT_ENABLED', 'do not have an application enabled', 'overall HEALTH_', 'Replacing daemon', 'deprecated feature inline_data', 'overall HEALTH_', '\\(OSD_DOWN\\)', '\\(OSD_', 'but it is still running', 'is not responding', 'PG_AVAILABILITY', 'PG_DEGRADED', 'Reduced data availability', 'bad backtrace', 'object missing on disk', 'error reading table object', 'error reading sessionmap', 'unmatched fragstat', 'unmatched rstat', 'was unreadable, recreating it now', 'Scrub error on inode', 'Metadata damage detected', 'MDS_FAILED', 'MDS_DAMAGE'], 'cpu_profile': set(), 'cluster': 'ceph', 'mon_bind_msgr2': True, 'mon_bind_addrvec': True} 2024-04-25T10:28:07.470 INFO:tasks.ceph:ctx.config {'archive_path': '/home/teuthworker/archive/pdonnell-2024-04-25_10:00:23-fs-wip-pdonnell-testing-20240425.015853-debug-distro-default-smithi/7673076', 'branch': 'wip-pdonnell-testing-20240425.015853-debug', 'description': 'fs/functional/{begin/{0-install 1-ceph 2-logrotate 3-modules} clusters/1a3s-mds-4c-client conf/{client mds mgr mon osd} distro/{centos_latest} mount/fuse objectstore/bluestore-bitmap overrides/{ignorelist_health ignorelist_wrongly_marked_down no_client_pidfile pg_health} subvol_versions/create_subvol_version_v1 tasks/alternate-pool}', 'email': 'pdonnell@redhat.com', 'first_in_suite': False, 'job_id': '7673076', 'kernel': {'kdb': True, 'sha1': 'distro'}, 'last_in_suite': False, 'log-rotate': {'ceph-mds': '10G', 'ceph-osd': '10G'}, 'machine_type': 'smithi', 'mgrmodules': {'sequential': [{'print': 'Enabling mgr modules'}]}, 'name': 'pdonnell-2024-04-25_10:00:23-fs-wip-pdonnell-testing-20240425.015853-debug-distro-default-smithi', 'no_nested_subset': False, 'nuke-on-error': True, 'openstack': [{'volumes': {'count': 4, 'size': 20}}, {'machine': {'disk': 200}}], 'os_type': 'centos', 'os_version': '9.stream', 'overrides': {'admin_socket': {'branch': 'wip-pdonnell-testing-20240425.015853-debug'}, 'ceph': {'conf': {'client': {'client mount timeout': 600, 'debug client': 20, 'debug ms': 1, 'pid file': '', 'rados mon op timeout': 900, 'rados osd op timeout': 900}, 'mds': {'debug mds': 20, 'debug mds balancer': 20, 'debug ms': 1, 'mds debug frag': True, 'mds debug scatterstat': True, 'mds op complaint time': 180, 'mds verify scatter': True, 'osd op complaint time': 180, 'rados mon op timeout': 900, 'rados osd op timeout': 900}, 'mgr': {'debug client': 20, 'debug mgr': 20, 'debug ms': 1}, 'mon': {'debug mon': 20, 'debug ms': 1, 'debug paxos': 20, 'mon down mkfs grace': 300, 'mon op complaint time': 120}, 'osd': {'bdev async discard': True, 'bdev enable discard': True, 'bluestore allocator': 'bitmap', 'bluestore block size': 96636764160, 'bluestore fsck on mount': True, 'debug bluefs': '1/20', 'debug bluestore': '1/20', 'debug ms': 1, 'debug osd': 20, 'debug rocksdb': '4/10', 'mon osd backfillfull_ratio': 0.85, 'mon osd full ratio': 0.9, 'mon osd nearfull ratio': 0.8, 'osd failsafe full ratio': 0.95, 'osd objectstore': 'bluestore', 'osd op complaint time': 180}}, 'flavor': 'default', 'fs': 'xfs', 'log-ignorelist': ['\\(MDS_ALL_DOWN\\)', '\\(MDS_UP_LESS_THAN_MAX\\)', 'FS_DEGRADED', 'FS_INLINE_DATA_DEPRECATED', 'FS_WITH_FAILED_MDS', 'MDS_ALL_DOWN', 'MDS_DAMAGE', 'MDS_DEGRADED', 'MDS_FAILED', 'MDS_INSUFFICIENT_STANDBY', 'MDS_UP_LESS_THAN_MAX', 'filesystem is online with fewer MDS than max_mds', 'POOL_APP_NOT_ENABLED', 'do not have an application enabled', 'overall HEALTH_', 'Replacing daemon', 'deprecated feature inline_data', 'overall HEALTH_', '\\(OSD_DOWN\\)', '\\(OSD_', 'but it is still running', 'is not responding', 'PG_AVAILABILITY', 'PG_DEGRADED', 'Reduced data availability', 'bad backtrace', 'object missing on disk', 'error reading table object', 'error reading sessionmap', 'unmatched fragstat', 'unmatched rstat', 'was unreadable, recreating it now', 'Scrub error on inode', 'Metadata damage detected', 'MDS_FAILED', 'MDS_DAMAGE'], 'sha1': '994e2f20102a20fa9a27966c14b625cc01884df8'}, 'ceph-deploy': {'bluestore': True, 'conf': {'client': {'log file': '/var/log/ceph/ceph-$name.$pid.log'}, 'mon': {}, 'osd': {'bdev async discard': True, 'bdev enable discard': True, 'bluestore block size': 96636764160, 'bluestore fsck on mount': True, 'debug bluefs': '1/20', 'debug bluestore': '1/20', 'debug rocksdb': '4/10', 'mon osd backfillfull_ratio': 0.85, 'mon osd full ratio': 0.9, 'mon osd nearfull ratio': 0.8, 'osd failsafe full ratio': 0.95, 'osd objectstore': 'bluestore'}}, 'fs': 'xfs'}, 'install': {'ceph': {'flavor': 'default', 'sha1': '994e2f20102a20fa9a27966c14b625cc01884df8'}}, 'selinux': {'allowlist': ['scontext=system_u:system_r:getty_t:s0']}, 'subvolume_version': 1, 'thrashosds': {'bdev_inject_crash': 2, 'bdev_inject_crash_probability': 0.5}, 'workunit': {'branch': 'wip-pdonnell-testing-20240425.015853-debug', 'sha1': '994e2f20102a20fa9a27966c14b625cc01884df8'}}, 'owner': 'scheduled_pdonnell@teuthology', 'priority': 85, 'repo': 'https://git.ceph.com/ceph-ci.git', 'roles': [['mon.a', 'mgr.y', 'mds.a', 'mds.c', 'osd.0', 'osd.1', 'osd.2', 'osd.3', 'client.0', 'client.2'], ['mon.b', 'mon.c', 'mgr.x', 'mds.b', 'mds.d', 'osd.4', 'osd.5', 'osd.6', 'osd.7', 'client.1', 'client.3']], 'seed': 1880, 'sha1': '994e2f20102a20fa9a27966c14b625cc01884df8', 'sleep_before_teardown': 0, 'subset': '178/512', 'suite': 'fs', 'suite_branch': 'wip-pdonnell-testing-20240425.015853-debug', 'suite_path': '/home/teuthworker/src/git.ceph.com_ceph-c_994e2f20102a20fa9a27966c14b625cc01884df8/qa', 'suite_relpath': 'qa', 'suite_repo': 'https://git.ceph.com/ceph-ci.git', 'suite_sha1': '994e2f20102a20fa9a27966c14b625cc01884df8', 'syslog': {'ignorelist': ['WARNING*.*check_session_state', 'WARNING*.*__ceph_remove_cap']}, 'targets': {'smithi037.front.sepia.ceph.com': 'ecdsa-sha2-nistp256 AAAAE2VjZHNhLXNoYTItbmlzdHAyNTYAAAAIbmlzdHAyNTYAAABBBNb4E+ULhZgytXCYeSyIeWUc2qPf8fZagTkx7ptrATcZuSzhYFE+9U7RVP8J8zk9PX9bEZMENSTJ0lvQlJJV8J0=', 'smithi203.front.sepia.ceph.com': 'ecdsa-sha2-nistp256 AAAAE2VjZHNhLXNoYTItbmlzdHAyNTYAAAAIbmlzdHAyNTYAAABBBKr9A/WsOZj+JJQP5Nth08arVsp53+mJPrPi+tzLz/Io/71y6YjlnTskBD+YCBw5RcNSZrKq/1pbw1+Nkh7cSgg='}, 'tasks': [{'internal.check_packages': None}, {'internal.buildpackages_prep': None}, {'internal.save_config': None}, {'internal.check_lock': None}, {'internal.add_remotes': None}, {'console_log': None}, {'internal.connect': None}, {'internal.push_inventory': None}, {'internal.serialize_remote_roles': None}, {'internal.check_conflict': None}, {'internal.check_ceph_data': None}, {'internal.vm_setup': None}, {'kernel': {'kdb': True, 'sha1': 'distro'}}, {'internal.base': None}, {'internal.archive_upload': None}, {'internal.archive': None}, {'internal.coredump': None}, {'internal.sudo': None}, {'internal.syslog': None}, {'internal.timer': None}, {'pcp': None}, {'selinux': None}, {'ansible.cephlab': None}, {'clock': None}, {'install': {'extra_packages': {'deb': ['python3-cephfs', 'cephfs-shell', 'cephfs-top', 'cephfs-mirror'], 'rpm': ['python3-cephfs', 'cephfs-shell', 'cephfs-top', 'cephfs-mirror']}, 'extra_system_packages': {'deb': ['bison', 'flex', 'libelf-dev', 'libssl-dev', 'network-manager', 'iproute2', 'util-linux', 'dump', 'indent', 'libaio-dev', 'libtool-bin', 'uuid-dev', 'xfslibs-dev', 'postgresql', 'postgresql-client', 'postgresql-common', 'postgresql-contrib'], 'rpm': ['bison', 'flex', 'elfutils-libelf-devel', 'openssl-devel', 'NetworkManager', 'iproute', 'util-linux', 'libacl-devel', 'libaio-devel', 'libattr-devel', 'libtool', 'libuuid-devel', 'xfsdump', 'xfsprogs', 'xfsprogs-devel', 'libaio-devel', 'libtool', 'libuuid-devel', 'xfsprogs-devel', 'postgresql', 'postgresql-server', 'postgresql-contrib']}, 'flavor': 'default', 'sha1': '994e2f20102a20fa9a27966c14b625cc01884df8'}}, {'ceph': {'create_rbd_pool': False, 'conf': {'client': {'client mount timeout': 600, 'debug client': 20, 'debug ms': 1, 'pid file': '', 'rados mon op timeout': 900, 'rados osd op timeout': 900}, 'mds': {'debug mds': 20, 'debug mds balancer': 20, 'debug ms': 1, 'mds debug frag': True, 'mds debug scatterstat': True, 'mds op complaint time': 180, 'mds verify scatter': True, 'osd op complaint time': 180, 'rados mon op timeout': 900, 'rados osd op timeout': 900}, 'mgr': {'debug client': 20, 'debug mgr': 20, 'debug ms': 1}, 'mon': {'debug mon': 20, 'debug ms': 1, 'debug paxos': 20, 'mon down mkfs grace': 300, 'mon op complaint time': 120}, 'osd': {'bdev async discard': True, 'bdev enable discard': True, 'bluestore allocator': 'bitmap', 'bluestore block size': 96636764160, 'bluestore fsck on mount': True, 'debug bluefs': '1/20', 'debug bluestore': '1/20', 'debug ms': 1, 'debug osd': 20, 'debug rocksdb': '4/10', 'mon osd backfillfull_ratio': 0.85, 'mon osd full ratio': 0.9, 'mon osd nearfull ratio': 0.8, 'osd failsafe full ratio': 0.95, 'osd objectstore': 'bluestore', 'osd op complaint time': 180}}, 'flavor': 'default', 'fs': 'xfs', 'log-ignorelist': ['\\(MDS_ALL_DOWN\\)', '\\(MDS_UP_LESS_THAN_MAX\\)', 'FS_DEGRADED', 'FS_INLINE_DATA_DEPRECATED', 'FS_WITH_FAILED_MDS', 'MDS_ALL_DOWN', 'MDS_DAMAGE', 'MDS_DEGRADED', 'MDS_FAILED', 'MDS_INSUFFICIENT_STANDBY', 'MDS_UP_LESS_THAN_MAX', 'filesystem is online with fewer MDS than max_mds', 'POOL_APP_NOT_ENABLED', 'do not have an application enabled', 'overall HEALTH_', 'Replacing daemon', 'deprecated feature inline_data', 'overall HEALTH_', '\\(OSD_DOWN\\)', '\\(OSD_', 'but it is still running', 'is not responding', 'PG_AVAILABILITY', 'PG_DEGRADED', 'Reduced data availability', 'bad backtrace', 'object missing on disk', 'error reading table object', 'error reading sessionmap', 'unmatched fragstat', 'unmatched rstat', 'was unreadable, recreating it now', 'Scrub error on inode', 'Metadata damage detected', 'MDS_FAILED', 'MDS_DAMAGE'], 'sha1': '994e2f20102a20fa9a27966c14b625cc01884df8', 'cluster': 'ceph'}}, {'sequential': ['mgrmodules']}, {'ceph-fuse': None}, {'cephfs_test_runner': {'modules': ['tasks.cephfs.test_recovery_pool']}}], 'teuthology': {'fragments_dropped': [], 'meta': {}, 'postmerge': ['local function is_kupstream() return false end', 'local function is_kdistro() return false end', 'local function is_fuse() return true end', "local function syntax_version() return '' end"]}, 'teuthology_branch': 'main', 'teuthology_sha1': '9481b1d62f50e7d0a4f3dd83adf6945b08d5ff17', 'timestamp': '2024-04-25_10:00:23', 'tube': 'smithi', 'user': 'pdonnell', 'verbose': False, 'worker_log': '/home/teuthworker/archive/worker_logs/dispatcher.smithi.2226885'} 2024-04-25T10:28:07.470 DEBUG:teuthology.orchestra.run.smithi037:> install -d -m0755 -- /home/ubuntu/cephtest/ceph.data 2024-04-25T10:28:07.473 DEBUG:teuthology.orchestra.run.smithi203:> install -d -m0755 -- /home/ubuntu/cephtest/ceph.data 2024-04-25T10:28:07.519 DEBUG:teuthology.orchestra.run.smithi037:> sudo install -d -m0777 -- /var/run/ceph 2024-04-25T10:28:07.529 DEBUG:teuthology.orchestra.run.smithi203:> sudo install -d -m0777 -- /var/run/ceph 2024-04-25T10:28:07.583 DEBUG:teuthology.orchestra.run.smithi037:> set -ex 2024-04-25T10:28:07.583 DEBUG:teuthology.orchestra.run.smithi037:> dd if=/scratch_devs of=/dev/stdout 2024-04-25T10:28:07.609 DEBUG:teuthology.misc:devs=['/dev/vg_nvme/lv_1', '/dev/vg_nvme/lv_2', '/dev/vg_nvme/lv_3', '/dev/vg_nvme/lv_4'] 2024-04-25T10:28:07.609 DEBUG:teuthology.orchestra.run.smithi037:> stat /dev/vg_nvme/lv_1 2024-04-25T10:28:07.665 INFO:teuthology.orchestra.run.smithi037.stdout: File: /dev/vg_nvme/lv_1 -> ../dm-0 2024-04-25T10:28:07.665 INFO:teuthology.orchestra.run.smithi037.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2024-04-25T10:28:07.665 INFO:teuthology.orchestra.run.smithi037.stdout:Device: 5h/5d Inode: 664 Links: 1 2024-04-25T10:28:07.665 INFO:teuthology.orchestra.run.smithi037.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2024-04-25T10:28:07.665 INFO:teuthology.orchestra.run.smithi037.stdout:Context: system_u:object_r:device_t:s0 2024-04-25T10:28:07.665 INFO:teuthology.orchestra.run.smithi037.stdout:Access: 2024-04-25 10:27:53.394298062 +0000 2024-04-25T10:28:07.665 INFO:teuthology.orchestra.run.smithi037.stdout:Modify: 2024-04-25 10:26:02.573565682 +0000 2024-04-25T10:28:07.665 INFO:teuthology.orchestra.run.smithi037.stdout:Change: 2024-04-25 10:26:02.573565682 +0000 2024-04-25T10:28:07.665 INFO:teuthology.orchestra.run.smithi037.stdout: Birth: 2024-04-25 10:26:02.573565682 +0000 2024-04-25T10:28:07.666 DEBUG:teuthology.orchestra.run.smithi037:> sudo dd if=/dev/vg_nvme/lv_1 of=/dev/null count=1 2024-04-25T10:28:07.728 INFO:teuthology.orchestra.run.smithi037.stderr:1+0 records in 2024-04-25T10:28:07.728 INFO:teuthology.orchestra.run.smithi037.stderr:1+0 records out 2024-04-25T10:28:07.729 INFO:teuthology.orchestra.run.smithi037.stderr:512 bytes copied, 0.000261574 s, 2.0 MB/s 2024-04-25T10:28:07.730 DEBUG:teuthology.orchestra.run.smithi037:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_1 2024-04-25T10:28:07.787 DEBUG:teuthology.orchestra.run.smithi037:> stat /dev/vg_nvme/lv_2 2024-04-25T10:28:07.842 INFO:teuthology.orchestra.run.smithi037.stdout: File: /dev/vg_nvme/lv_2 -> ../dm-1 2024-04-25T10:28:07.842 INFO:teuthology.orchestra.run.smithi037.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2024-04-25T10:28:07.843 INFO:teuthology.orchestra.run.smithi037.stdout:Device: 5h/5d Inode: 683 Links: 1 2024-04-25T10:28:07.843 INFO:teuthology.orchestra.run.smithi037.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2024-04-25T10:28:07.843 INFO:teuthology.orchestra.run.smithi037.stdout:Context: system_u:object_r:device_t:s0 2024-04-25T10:28:07.843 INFO:teuthology.orchestra.run.smithi037.stdout:Access: 2024-04-25 10:27:53.395298049 +0000 2024-04-25T10:28:07.843 INFO:teuthology.orchestra.run.smithi037.stdout:Modify: 2024-04-25 10:26:02.914561144 +0000 2024-04-25T10:28:07.843 INFO:teuthology.orchestra.run.smithi037.stdout:Change: 2024-04-25 10:26:02.914561144 +0000 2024-04-25T10:28:07.843 INFO:teuthology.orchestra.run.smithi037.stdout: Birth: 2024-04-25 10:26:02.914561144 +0000 2024-04-25T10:28:07.843 DEBUG:teuthology.orchestra.run.smithi037:> sudo dd if=/dev/vg_nvme/lv_2 of=/dev/null count=1 2024-04-25T10:28:07.906 INFO:teuthology.orchestra.run.smithi037.stderr:1+0 records in 2024-04-25T10:28:07.906 INFO:teuthology.orchestra.run.smithi037.stderr:1+0 records out 2024-04-25T10:28:07.906 INFO:teuthology.orchestra.run.smithi037.stderr:512 bytes copied, 0.000212466 s, 2.4 MB/s 2024-04-25T10:28:07.908 DEBUG:teuthology.orchestra.run.smithi037:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_2 2024-04-25T10:28:07.964 DEBUG:teuthology.orchestra.run.smithi037:> stat /dev/vg_nvme/lv_3 2024-04-25T10:28:08.019 INFO:teuthology.orchestra.run.smithi037.stdout: File: /dev/vg_nvme/lv_3 -> ../dm-2 2024-04-25T10:28:08.019 INFO:teuthology.orchestra.run.smithi037.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2024-04-25T10:28:08.020 INFO:teuthology.orchestra.run.smithi037.stdout:Device: 5h/5d Inode: 702 Links: 1 2024-04-25T10:28:08.020 INFO:teuthology.orchestra.run.smithi037.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2024-04-25T10:28:08.020 INFO:teuthology.orchestra.run.smithi037.stdout:Context: system_u:object_r:device_t:s0 2024-04-25T10:28:08.020 INFO:teuthology.orchestra.run.smithi037.stdout:Access: 2024-04-25 10:27:53.395298049 +0000 2024-04-25T10:28:08.020 INFO:teuthology.orchestra.run.smithi037.stdout:Modify: 2024-04-25 10:26:03.268556434 +0000 2024-04-25T10:28:08.020 INFO:teuthology.orchestra.run.smithi037.stdout:Change: 2024-04-25 10:26:03.268556434 +0000 2024-04-25T10:28:08.020 INFO:teuthology.orchestra.run.smithi037.stdout: Birth: 2024-04-25 10:26:03.268556434 +0000 2024-04-25T10:28:08.020 DEBUG:teuthology.orchestra.run.smithi037:> sudo dd if=/dev/vg_nvme/lv_3 of=/dev/null count=1 2024-04-25T10:28:08.083 INFO:teuthology.orchestra.run.smithi037.stderr:1+0 records in 2024-04-25T10:28:08.083 INFO:teuthology.orchestra.run.smithi037.stderr:1+0 records out 2024-04-25T10:28:08.083 INFO:teuthology.orchestra.run.smithi037.stderr:512 bytes copied, 0.000269714 s, 1.9 MB/s 2024-04-25T10:28:08.084 DEBUG:teuthology.orchestra.run.smithi037:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_3 2024-04-25T10:28:08.141 DEBUG:teuthology.orchestra.run.smithi037:> stat /dev/vg_nvme/lv_4 2024-04-25T10:28:08.196 INFO:teuthology.orchestra.run.smithi037.stdout: File: /dev/vg_nvme/lv_4 -> ../dm-3 2024-04-25T10:28:08.197 INFO:teuthology.orchestra.run.smithi037.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2024-04-25T10:28:08.197 INFO:teuthology.orchestra.run.smithi037.stdout:Device: 5h/5d Inode: 722 Links: 1 2024-04-25T10:28:08.197 INFO:teuthology.orchestra.run.smithi037.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2024-04-25T10:28:08.197 INFO:teuthology.orchestra.run.smithi037.stdout:Context: system_u:object_r:device_t:s0 2024-04-25T10:28:08.197 INFO:teuthology.orchestra.run.smithi037.stdout:Access: 2024-04-25 10:27:53.396298035 +0000 2024-04-25T10:28:08.197 INFO:teuthology.orchestra.run.smithi037.stdout:Modify: 2024-04-25 10:26:03.615551817 +0000 2024-04-25T10:28:08.197 INFO:teuthology.orchestra.run.smithi037.stdout:Change: 2024-04-25 10:26:03.615551817 +0000 2024-04-25T10:28:08.197 INFO:teuthology.orchestra.run.smithi037.stdout: Birth: 2024-04-25 10:26:03.615551817 +0000 2024-04-25T10:28:08.197 DEBUG:teuthology.orchestra.run.smithi037:> sudo dd if=/dev/vg_nvme/lv_4 of=/dev/null count=1 2024-04-25T10:28:08.260 INFO:teuthology.orchestra.run.smithi037.stderr:1+0 records in 2024-04-25T10:28:08.260 INFO:teuthology.orchestra.run.smithi037.stderr:1+0 records out 2024-04-25T10:28:08.260 INFO:teuthology.orchestra.run.smithi037.stderr:512 bytes copied, 0.000240954 s, 2.1 MB/s 2024-04-25T10:28:08.261 DEBUG:teuthology.orchestra.run.smithi037:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_4 2024-04-25T10:28:08.317 INFO:tasks.ceph:osd dev map: {'osd.0': '/dev/vg_nvme/lv_1', 'osd.1': '/dev/vg_nvme/lv_2', 'osd.2': '/dev/vg_nvme/lv_3', 'osd.3': '/dev/vg_nvme/lv_4'} 2024-04-25T10:28:08.317 DEBUG:teuthology.orchestra.run.smithi203:> set -ex 2024-04-25T10:28:08.317 DEBUG:teuthology.orchestra.run.smithi203:> dd if=/scratch_devs of=/dev/stdout 2024-04-25T10:28:08.333 DEBUG:teuthology.misc:devs=['/dev/vg_nvme/lv_1', '/dev/vg_nvme/lv_2', '/dev/vg_nvme/lv_3', '/dev/vg_nvme/lv_4'] 2024-04-25T10:28:08.333 DEBUG:teuthology.orchestra.run.smithi203:> stat /dev/vg_nvme/lv_1 2024-04-25T10:28:08.388 INFO:teuthology.orchestra.run.smithi203.stdout: File: /dev/vg_nvme/lv_1 -> ../dm-0 2024-04-25T10:28:08.389 INFO:teuthology.orchestra.run.smithi203.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2024-04-25T10:28:08.389 INFO:teuthology.orchestra.run.smithi203.stdout:Device: 5h/5d Inode: 665 Links: 1 2024-04-25T10:28:08.389 INFO:teuthology.orchestra.run.smithi203.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2024-04-25T10:28:08.389 INFO:teuthology.orchestra.run.smithi203.stdout:Context: system_u:object_r:device_t:s0 2024-04-25T10:28:08.389 INFO:teuthology.orchestra.run.smithi203.stdout:Access: 2024-04-25 10:27:57.719430652 +0000 2024-04-25T10:28:08.389 INFO:teuthology.orchestra.run.smithi203.stdout:Modify: 2024-04-25 10:25:58.296091313 +0000 2024-04-25T10:28:08.389 INFO:teuthology.orchestra.run.smithi203.stdout:Change: 2024-04-25 10:25:58.296091313 +0000 2024-04-25T10:28:08.389 INFO:teuthology.orchestra.run.smithi203.stdout: Birth: 2024-04-25 10:25:58.296091313 +0000 2024-04-25T10:28:08.389 DEBUG:teuthology.orchestra.run.smithi203:> sudo dd if=/dev/vg_nvme/lv_1 of=/dev/null count=1 2024-04-25T10:28:08.451 INFO:teuthology.orchestra.run.smithi203.stderr:1+0 records in 2024-04-25T10:28:08.451 INFO:teuthology.orchestra.run.smithi203.stderr:1+0 records out 2024-04-25T10:28:08.451 INFO:teuthology.orchestra.run.smithi203.stderr:512 bytes copied, 0.00021341 s, 2.4 MB/s 2024-04-25T10:28:08.452 DEBUG:teuthology.orchestra.run.smithi203:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_1 2024-04-25T10:28:08.508 DEBUG:teuthology.orchestra.run.smithi203:> stat /dev/vg_nvme/lv_2 2024-04-25T10:28:08.564 INFO:teuthology.orchestra.run.smithi203.stdout: File: /dev/vg_nvme/lv_2 -> ../dm-1 2024-04-25T10:28:08.564 INFO:teuthology.orchestra.run.smithi203.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2024-04-25T10:28:08.564 INFO:teuthology.orchestra.run.smithi203.stdout:Device: 5h/5d Inode: 686 Links: 1 2024-04-25T10:28:08.564 INFO:teuthology.orchestra.run.smithi203.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2024-04-25T10:28:08.564 INFO:teuthology.orchestra.run.smithi203.stdout:Context: system_u:object_r:device_t:s0 2024-04-25T10:28:08.564 INFO:teuthology.orchestra.run.smithi203.stdout:Access: 2024-04-25 10:27:57.720430628 +0000 2024-04-25T10:28:08.564 INFO:teuthology.orchestra.run.smithi203.stdout:Modify: 2024-04-25 10:25:58.629082603 +0000 2024-04-25T10:28:08.564 INFO:teuthology.orchestra.run.smithi203.stdout:Change: 2024-04-25 10:25:58.629082603 +0000 2024-04-25T10:28:08.564 INFO:teuthology.orchestra.run.smithi203.stdout: Birth: 2024-04-25 10:25:58.629082603 +0000 2024-04-25T10:28:08.565 DEBUG:teuthology.orchestra.run.smithi203:> sudo dd if=/dev/vg_nvme/lv_2 of=/dev/null count=1 2024-04-25T10:28:08.626 INFO:teuthology.orchestra.run.smithi203.stderr:1+0 records in 2024-04-25T10:28:08.626 INFO:teuthology.orchestra.run.smithi203.stderr:1+0 records out 2024-04-25T10:28:08.626 INFO:teuthology.orchestra.run.smithi203.stderr:512 bytes copied, 0.000184529 s, 2.8 MB/s 2024-04-25T10:28:08.627 DEBUG:teuthology.orchestra.run.smithi203:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_2 2024-04-25T10:28:08.682 DEBUG:teuthology.orchestra.run.smithi203:> stat /dev/vg_nvme/lv_3 2024-04-25T10:28:08.736 INFO:teuthology.orchestra.run.smithi203.stdout: File: /dev/vg_nvme/lv_3 -> ../dm-2 2024-04-25T10:28:08.736 INFO:teuthology.orchestra.run.smithi203.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2024-04-25T10:28:08.736 INFO:teuthology.orchestra.run.smithi203.stdout:Device: 5h/5d Inode: 702 Links: 1 2024-04-25T10:28:08.736 INFO:teuthology.orchestra.run.smithi203.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2024-04-25T10:28:08.736 INFO:teuthology.orchestra.run.smithi203.stdout:Context: system_u:object_r:device_t:s0 2024-04-25T10:28:08.736 INFO:teuthology.orchestra.run.smithi203.stdout:Access: 2024-04-25 10:27:57.720430628 +0000 2024-04-25T10:28:08.736 INFO:teuthology.orchestra.run.smithi203.stdout:Modify: 2024-04-25 10:25:58.974073580 +0000 2024-04-25T10:28:08.736 INFO:teuthology.orchestra.run.smithi203.stdout:Change: 2024-04-25 10:25:58.974073580 +0000 2024-04-25T10:28:08.737 INFO:teuthology.orchestra.run.smithi203.stdout: Birth: 2024-04-25 10:25:58.974073580 +0000 2024-04-25T10:28:08.737 DEBUG:teuthology.orchestra.run.smithi203:> sudo dd if=/dev/vg_nvme/lv_3 of=/dev/null count=1 2024-04-25T10:28:08.798 INFO:teuthology.orchestra.run.smithi203.stderr:1+0 records in 2024-04-25T10:28:08.798 INFO:teuthology.orchestra.run.smithi203.stderr:1+0 records out 2024-04-25T10:28:08.798 INFO:teuthology.orchestra.run.smithi203.stderr:512 bytes copied, 0.000189918 s, 2.7 MB/s 2024-04-25T10:28:08.799 DEBUG:teuthology.orchestra.run.smithi203:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_3 2024-04-25T10:28:08.855 DEBUG:teuthology.orchestra.run.smithi203:> stat /dev/vg_nvme/lv_4 2024-04-25T10:28:08.910 INFO:teuthology.orchestra.run.smithi203.stdout: File: /dev/vg_nvme/lv_4 -> ../dm-3 2024-04-25T10:28:08.910 INFO:teuthology.orchestra.run.smithi203.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2024-04-25T10:28:08.910 INFO:teuthology.orchestra.run.smithi203.stdout:Device: 5h/5d Inode: 721 Links: 1 2024-04-25T10:28:08.910 INFO:teuthology.orchestra.run.smithi203.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2024-04-25T10:28:08.910 INFO:teuthology.orchestra.run.smithi203.stdout:Context: system_u:object_r:device_t:s0 2024-04-25T10:28:08.910 INFO:teuthology.orchestra.run.smithi203.stdout:Access: 2024-04-25 10:27:57.720430628 +0000 2024-04-25T10:28:08.910 INFO:teuthology.orchestra.run.smithi203.stdout:Modify: 2024-04-25 10:25:59.302065002 +0000 2024-04-25T10:28:08.911 INFO:teuthology.orchestra.run.smithi203.stdout:Change: 2024-04-25 10:25:59.302065002 +0000 2024-04-25T10:28:08.911 INFO:teuthology.orchestra.run.smithi203.stdout: Birth: 2024-04-25 10:25:59.302065002 +0000 2024-04-25T10:28:08.911 DEBUG:teuthology.orchestra.run.smithi203:> sudo dd if=/dev/vg_nvme/lv_4 of=/dev/null count=1 2024-04-25T10:28:08.972 INFO:teuthology.orchestra.run.smithi203.stderr:1+0 records in 2024-04-25T10:28:08.972 INFO:teuthology.orchestra.run.smithi203.stderr:1+0 records out 2024-04-25T10:28:08.972 INFO:teuthology.orchestra.run.smithi203.stderr:512 bytes copied, 0.000225158 s, 2.3 MB/s 2024-04-25T10:28:08.973 DEBUG:teuthology.orchestra.run.smithi203:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_4 2024-04-25T10:28:09.029 INFO:tasks.ceph:osd dev map: {'osd.4': '/dev/vg_nvme/lv_1', 'osd.5': '/dev/vg_nvme/lv_2', 'osd.6': '/dev/vg_nvme/lv_3', 'osd.7': '/dev/vg_nvme/lv_4'} 2024-04-25T10:28:09.029 INFO:tasks.ceph:remote_to_roles_to_devs: {Remote(name='ubuntu@smithi037.front.sepia.ceph.com'): {'osd.0': '/dev/vg_nvme/lv_1', 'osd.1': '/dev/vg_nvme/lv_2', 'osd.2': '/dev/vg_nvme/lv_3', 'osd.3': '/dev/vg_nvme/lv_4'}, Remote(name='ubuntu@smithi203.front.sepia.ceph.com'): {'osd.4': '/dev/vg_nvme/lv_1', 'osd.5': '/dev/vg_nvme/lv_2', 'osd.6': '/dev/vg_nvme/lv_3', 'osd.7': '/dev/vg_nvme/lv_4'}} 2024-04-25T10:28:09.029 INFO:tasks.ceph:Generating config... 2024-04-25T10:28:09.031 INFO:tasks.ceph:[client] client mount timeout = 600 2024-04-25T10:28:09.031 INFO:tasks.ceph:[client] debug client = 20 2024-04-25T10:28:09.031 INFO:tasks.ceph:[client] debug ms = 1 2024-04-25T10:28:09.031 INFO:tasks.ceph:[client] pid file = 2024-04-25T10:28:09.031 INFO:tasks.ceph:[client] rados mon op timeout = 900 2024-04-25T10:28:09.031 INFO:tasks.ceph:[client] rados osd op timeout = 900 2024-04-25T10:28:09.031 INFO:tasks.ceph:[mds] debug mds = 20 2024-04-25T10:28:09.031 INFO:tasks.ceph:[mds] debug mds balancer = 20 2024-04-25T10:28:09.031 INFO:tasks.ceph:[mds] debug ms = 1 2024-04-25T10:28:09.031 INFO:tasks.ceph:[mds] mds debug frag = True 2024-04-25T10:28:09.031 INFO:tasks.ceph:[mds] mds debug scatterstat = True 2024-04-25T10:28:09.031 INFO:tasks.ceph:[mds] mds op complaint time = 180 2024-04-25T10:28:09.031 INFO:tasks.ceph:[mds] mds verify scatter = True 2024-04-25T10:28:09.032 INFO:tasks.ceph:[mds] osd op complaint time = 180 2024-04-25T10:28:09.032 INFO:tasks.ceph:[mds] rados mon op timeout = 900 2024-04-25T10:28:09.032 INFO:tasks.ceph:[mds] rados osd op timeout = 900 2024-04-25T10:28:09.032 INFO:tasks.ceph:[mgr] debug client = 20 2024-04-25T10:28:09.032 INFO:tasks.ceph:[mgr] debug mgr = 20 2024-04-25T10:28:09.032 INFO:tasks.ceph:[mgr] debug ms = 1 2024-04-25T10:28:09.032 INFO:tasks.ceph:[mon] debug mon = 20 2024-04-25T10:28:09.032 INFO:tasks.ceph:[mon] debug ms = 1 2024-04-25T10:28:09.032 INFO:tasks.ceph:[mon] debug paxos = 20 2024-04-25T10:28:09.032 INFO:tasks.ceph:[mon] mon down mkfs grace = 300 2024-04-25T10:28:09.032 INFO:tasks.ceph:[mon] mon op complaint time = 120 2024-04-25T10:28:09.032 INFO:tasks.ceph:[osd] bdev async discard = True 2024-04-25T10:28:09.032 INFO:tasks.ceph:[osd] bdev enable discard = True 2024-04-25T10:28:09.032 INFO:tasks.ceph:[osd] bluestore allocator = bitmap 2024-04-25T10:28:09.032 INFO:tasks.ceph:[osd] bluestore block size = 96636764160 2024-04-25T10:28:09.033 INFO:tasks.ceph:[osd] bluestore fsck on mount = True 2024-04-25T10:28:09.033 INFO:tasks.ceph:[osd] debug bluefs = 1/20 2024-04-25T10:28:09.033 INFO:tasks.ceph:[osd] debug bluestore = 1/20 2024-04-25T10:28:09.033 INFO:tasks.ceph:[osd] debug ms = 1 2024-04-25T10:28:09.033 INFO:tasks.ceph:[osd] debug osd = 20 2024-04-25T10:28:09.033 INFO:tasks.ceph:[osd] debug rocksdb = 4/10 2024-04-25T10:28:09.033 INFO:tasks.ceph:[osd] mon osd backfillfull_ratio = 0.85 2024-04-25T10:28:09.033 INFO:tasks.ceph:[osd] mon osd full ratio = 0.9 2024-04-25T10:28:09.033 INFO:tasks.ceph:[osd] mon osd nearfull ratio = 0.8 2024-04-25T10:28:09.033 INFO:tasks.ceph:[osd] osd failsafe full ratio = 0.95 2024-04-25T10:28:09.033 INFO:tasks.ceph:[osd] osd objectstore = bluestore 2024-04-25T10:28:09.033 INFO:tasks.ceph:[osd] osd op complaint time = 180 2024-04-25T10:28:09.033 INFO:tasks.ceph:Setting up mon.a... 2024-04-25T10:28:09.034 DEBUG:teuthology.orchestra.run.smithi037:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool --create-keyring /etc/ceph/ceph.keyring 2024-04-25T10:28:09.114 INFO:teuthology.orchestra.run.smithi037.stdout:creating /etc/ceph/ceph.keyring 2024-04-25T10:28:09.117 DEBUG:teuthology.orchestra.run.smithi037:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool --gen-key --name=mon. /etc/ceph/ceph.keyring 2024-04-25T10:28:09.172 DEBUG:teuthology.orchestra.run.smithi037:> sudo chmod 0644 /etc/ceph/ceph.keyring 2024-04-25T10:28:09.196 DEBUG:tasks.ceph:Ceph mon addresses: [('mon.a', '172.21.15.37'), ('mon.b', '172.21.15.203'), ('mon.c', '[v2:172.21.15.203:3301,v1:172.21.15.203:6790]')] 2024-04-25T10:28:09.197 DEBUG:tasks.ceph:writing out conf {'global': {'chdir': '', 'pid file': '/var/run/ceph/$cluster-$name.pid', 'auth supported': 'cephx', 'filestore xattr use omap': 'true', 'mon clock drift allowed': '1.000', 'osd crush chooseleaf type': '0', 'auth debug': 'true', 'ms die on old message': 'true', 'ms die on bug': 'true', 'mon max pg per osd': '10000', 'mon pg warn max object skew': '0', 'osd_pool_default_pg_autoscale_mode': 'off', 'osd pool default size': '2', 'mon osd allow primary affinity': 'true', 'mon osd allow pg remap': 'true', 'mon warn on legacy crush tunables': 'false', 'mon warn on crush straw calc version zero': 'false', 'mon warn on no sortbitwise': 'false', 'mon warn on osd down out interval zero': 'false', 'mon warn on too few osds': 'false', 'mon_warn_on_pool_pg_num_not_power_of_two': 'false', 'mon_warn_on_pool_no_redundancy': 'false', 'mon_allow_pool_size_one': 'true', 'osd pool default erasure code profile': 'plugin=jerasure technique=reed_sol_van k=2 m=1 crush-failure-domain=osd', 'osd default data pool replay window': '5', 'mon allow pool delete': 'true', 'mon cluster log file level': 'debug', 'debug asserts on shutdown': 'true', 'mon health detail to clog': 'false', 'mon host': '172.21.15.37,172.21.15.203,[v2:172.21.15.203:3301,v1:172.21.15.203:6790]'}, 'osd': {'osd journal size': '100', 'osd scrub load threshold': '5.0', 'osd scrub max interval': '600', 'osd mclock profile': 'high_recovery_ops', 'osd recover clone overlap': 'true', 'osd recovery max chunk': '1048576', 'osd debug shutdown': 'true', 'osd debug op order': 'true', 'osd debug verify stray on activate': 'true', 'osd debug trim objects': 'true', 'osd open classes on start': 'true', 'osd debug pg log writeout': 'true', 'osd deep scrub update digest min age': '30', 'osd map max advance': '10', 'journal zero on create': 'true', 'filestore ondisk finisher threads': '3', 'filestore apply finisher threads': '3', 'bdev debug aio': 'true', 'osd debug misdirected ops': 'true', 'bdev async discard': True, 'bdev enable discard': True, 'bluestore allocator': 'bitmap', 'bluestore block size': 96636764160, 'bluestore fsck on mount': True, 'debug bluefs': '1/20', 'debug bluestore': '1/20', 'debug ms': 1, 'debug osd': 20, 'debug rocksdb': '4/10', 'mon osd backfillfull_ratio': 0.85, 'mon osd full ratio': 0.9, 'mon osd nearfull ratio': 0.8, 'osd failsafe full ratio': 0.95, 'osd objectstore': 'bluestore', 'osd op complaint time': 180}, 'mgr': {'debug ms': 1, 'debug mgr': 20, 'debug mon': '20', 'debug auth': '20', 'mon reweight min pgs per osd': '4', 'mon reweight min bytes per osd': '10', 'mgr/telemetry/nag': 'false', 'debug client': 20}, 'mon': {'debug ms': 1, 'debug mon': 20, 'debug paxos': 20, 'debug auth': '20', 'mon data avail warn': '5', 'mon mgr mkfs grace': '240', 'mon reweight min pgs per osd': '4', 'mon osd reporter subtree level': 'osd', 'mon osd prime pg temp': 'true', 'mon reweight min bytes per osd': '10', 'auth mon ticket ttl': '660', 'auth service ticket ttl': '240', 'mon_warn_on_insecure_global_id_reclaim': 'false', 'mon_warn_on_insecure_global_id_reclaim_allowed': 'false', 'mon_down_mkfs_grace': '2m', 'mon_warn_on_filestore_osds': 'false', 'mon down mkfs grace': 300, 'mon op complaint time': 120}, 'client': {'rgw cache enabled': 'true', 'rgw enable ops log': 'true', 'rgw enable usage log': 'true', 'log file': '/var/log/ceph/$cluster-$name.$pid.log', 'admin socket': '/var/run/ceph/$cluster-$name.$pid.asok', 'client mount timeout': 600, 'debug client': 20, 'debug ms': 1, 'pid file': '', 'rados mon op timeout': 900, 'rados osd op timeout': 900}, 'mon.a': {}, 'mon.b': {}, 'mon.c': {}, 'mds.a': {}, 'mds.c': {}, 'mds.b': {}, 'mds.d': {}, 'mds': {'debug mds': 20, 'debug mds balancer': 20, 'debug ms': 1, 'mds debug frag': True, 'mds debug scatterstat': True, 'mds op complaint time': 180, 'mds verify scatter': True, 'osd op complaint time': 180, 'rados mon op timeout': 900, 'rados osd op timeout': 900}} 2024-04-25T10:28:09.197 DEBUG:teuthology.orchestra.run.smithi037:> set -ex 2024-04-25T10:28:09.197 DEBUG:teuthology.orchestra.run.smithi037:> dd of=/home/ubuntu/cephtest/ceph.tmp.conf 2024-04-25T10:28:09.251 DEBUG:teuthology.orchestra.run.smithi037:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage monmaptool -c /home/ubuntu/cephtest/ceph.tmp.conf --create --clobber --enable-all-features --add a 172.21.15.37 --add b 172.21.15.203 --addv c '[v2:172.21.15.203:3301,v1:172.21.15.203:6790]' --print /home/ubuntu/cephtest/ceph.monmap 2024-04-25T10:28:09.346 INFO:teuthology.orchestra.run.smithi037.stdout:monmaptool: monmap file /home/ubuntu/cephtest/ceph.monmap 2024-04-25T10:28:09.346 INFO:teuthology.orchestra.run.smithi037.stdout:monmaptool: generated fsid f4acda28-be34-492a-b286-994afd36f61d 2024-04-25T10:28:09.346 INFO:teuthology.orchestra.run.smithi037.stdout:setting min_mon_release = quincy 2024-04-25T10:28:09.346 INFO:teuthology.orchestra.run.smithi037.stdout:epoch 0 2024-04-25T10:28:09.346 INFO:teuthology.orchestra.run.smithi037.stdout:fsid f4acda28-be34-492a-b286-994afd36f61d 2024-04-25T10:28:09.346 INFO:teuthology.orchestra.run.smithi037.stdout:last_changed 2024-04-25T10:28:09.345303+0000 2024-04-25T10:28:09.347 INFO:teuthology.orchestra.run.smithi037.stdout:created 2024-04-25T10:28:09.345303+0000 2024-04-25T10:28:09.347 INFO:teuthology.orchestra.run.smithi037.stdout:min_mon_release 17 (quincy) 2024-04-25T10:28:09.347 INFO:teuthology.orchestra.run.smithi037.stdout:election_strategy: 1 2024-04-25T10:28:09.347 INFO:teuthology.orchestra.run.smithi037.stdout:0: [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] mon.a 2024-04-25T10:28:09.347 INFO:teuthology.orchestra.run.smithi037.stdout:1: [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] mon.b 2024-04-25T10:28:09.347 INFO:teuthology.orchestra.run.smithi037.stdout:2: [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] mon.c 2024-04-25T10:28:09.347 INFO:teuthology.orchestra.run.smithi037.stdout:monmaptool: writing epoch 0 to /home/ubuntu/cephtest/ceph.monmap (3 monitors) 2024-04-25T10:28:09.348 DEBUG:teuthology.orchestra.run.smithi037:> rm -- /home/ubuntu/cephtest/ceph.tmp.conf 2024-04-25T10:28:09.363 INFO:tasks.ceph:Writing /etc/ceph/ceph.conf for FSID f4acda28-be34-492a-b286-994afd36f61d... 2024-04-25T10:28:09.364 DEBUG:teuthology.orchestra.run.smithi037:> sudo mkdir -p /etc/ceph && sudo chmod 0755 /etc/ceph && sudo tee /etc/ceph/ceph.conf && sudo chmod 0644 /etc/ceph/ceph.conf > /dev/null 2024-04-25T10:28:09.406 DEBUG:teuthology.orchestra.run.smithi203:> sudo mkdir -p /etc/ceph && sudo chmod 0755 /etc/ceph && sudo tee /etc/ceph/ceph.conf && sudo chmod 0644 /etc/ceph/ceph.conf > /dev/null 2024-04-25T10:28:09.445 INFO:teuthology.orchestra.run.smithi037.stdout:[global] 2024-04-25T10:28:09.445 INFO:teuthology.orchestra.run.smithi037.stdout: chdir = "" 2024-04-25T10:28:09.445 INFO:teuthology.orchestra.run.smithi037.stdout: pid file = /var/run/ceph/$cluster-$name.pid 2024-04-25T10:28:09.445 INFO:teuthology.orchestra.run.smithi037.stdout: auth supported = cephx 2024-04-25T10:28:09.445 INFO:teuthology.orchestra.run.smithi037.stdout: 2024-04-25T10:28:09.445 INFO:teuthology.orchestra.run.smithi037.stdout: filestore xattr use omap = true 2024-04-25T10:28:09.446 INFO:teuthology.orchestra.run.smithi037.stdout: 2024-04-25T10:28:09.446 INFO:teuthology.orchestra.run.smithi037.stdout: mon clock drift allowed = 1.000 2024-04-25T10:28:09.446 INFO:teuthology.orchestra.run.smithi037.stdout: 2024-04-25T10:28:09.446 INFO:teuthology.orchestra.run.smithi037.stdout: osd crush chooseleaf type = 0 2024-04-25T10:28:09.446 INFO:teuthology.orchestra.run.smithi037.stdout: auth debug = true 2024-04-25T10:28:09.446 INFO:teuthology.orchestra.run.smithi037.stdout: 2024-04-25T10:28:09.446 INFO:teuthology.orchestra.run.smithi037.stdout: ms die on old message = true 2024-04-25T10:28:09.446 INFO:teuthology.orchestra.run.smithi037.stdout: ms die on bug = true 2024-04-25T10:28:09.446 INFO:teuthology.orchestra.run.smithi037.stdout: 2024-04-25T10:28:09.446 INFO:teuthology.orchestra.run.smithi037.stdout: mon max pg per osd = 10000 # >= luminous 2024-04-25T10:28:09.446 INFO:teuthology.orchestra.run.smithi037.stdout: mon pg warn max object skew = 0 2024-04-25T10:28:09.446 INFO:teuthology.orchestra.run.smithi037.stdout: 2024-04-25T10:28:09.446 INFO:teuthology.orchestra.run.smithi037.stdout: # disable pg_autoscaler by default for new pools 2024-04-25T10:28:09.446 INFO:teuthology.orchestra.run.smithi037.stdout: osd_pool_default_pg_autoscale_mode = off 2024-04-25T10:28:09.447 INFO:teuthology.orchestra.run.smithi037.stdout: 2024-04-25T10:28:09.447 INFO:teuthology.orchestra.run.smithi037.stdout: osd pool default size = 2 2024-04-25T10:28:09.447 INFO:teuthology.orchestra.run.smithi037.stdout: 2024-04-25T10:28:09.447 INFO:teuthology.orchestra.run.smithi037.stdout: mon osd allow primary affinity = true 2024-04-25T10:28:09.447 INFO:teuthology.orchestra.run.smithi037.stdout: mon osd allow pg remap = true 2024-04-25T10:28:09.447 INFO:teuthology.orchestra.run.smithi037.stdout: mon warn on legacy crush tunables = false 2024-04-25T10:28:09.447 INFO:teuthology.orchestra.run.smithi037.stdout: mon warn on crush straw calc version zero = false 2024-04-25T10:28:09.447 INFO:teuthology.orchestra.run.smithi037.stdout: mon warn on no sortbitwise = false 2024-04-25T10:28:09.447 INFO:teuthology.orchestra.run.smithi037.stdout: mon warn on osd down out interval zero = false 2024-04-25T10:28:09.447 INFO:teuthology.orchestra.run.smithi037.stdout: mon warn on too few osds = false 2024-04-25T10:28:09.447 INFO:teuthology.orchestra.run.smithi037.stdout: mon_warn_on_pool_pg_num_not_power_of_two = false 2024-04-25T10:28:09.447 INFO:teuthology.orchestra.run.smithi037.stdout: mon_warn_on_pool_no_redundancy = false 2024-04-25T10:28:09.447 INFO:teuthology.orchestra.run.smithi037.stdout: mon_allow_pool_size_one = true 2024-04-25T10:28:09.447 INFO:teuthology.orchestra.run.smithi037.stdout: 2024-04-25T10:28:09.447 INFO:teuthology.orchestra.run.smithi037.stdout: osd pool default erasure code profile = plugin=jerasure technique=reed_sol_van k=2 m=1 crush-failure-domain=osd 2024-04-25T10:28:09.448 INFO:teuthology.orchestra.run.smithi037.stdout: 2024-04-25T10:28:09.448 INFO:teuthology.orchestra.run.smithi037.stdout: osd default data pool replay window = 5 2024-04-25T10:28:09.448 INFO:teuthology.orchestra.run.smithi037.stdout: 2024-04-25T10:28:09.448 INFO:teuthology.orchestra.run.smithi037.stdout: mon allow pool delete = true 2024-04-25T10:28:09.448 INFO:teuthology.orchestra.run.smithi037.stdout: 2024-04-25T10:28:09.448 INFO:teuthology.orchestra.run.smithi037.stdout: mon cluster log file level = debug 2024-04-25T10:28:09.448 INFO:teuthology.orchestra.run.smithi037.stdout: debug asserts on shutdown = true 2024-04-25T10:28:09.448 INFO:teuthology.orchestra.run.smithi037.stdout: mon health detail to clog = false 2024-04-25T10:28:09.448 INFO:teuthology.orchestra.run.smithi037.stdout: mon host = "172.21.15.37,172.21.15.203,[v2:172.21.15.203:3301,v1:172.21.15.203:6790]" 2024-04-25T10:28:09.448 INFO:teuthology.orchestra.run.smithi037.stdout: fsid = f4acda28-be34-492a-b286-994afd36f61d 2024-04-25T10:28:09.448 INFO:teuthology.orchestra.run.smithi037.stdout: 2024-04-25T10:28:09.448 INFO:teuthology.orchestra.run.smithi037.stdout:[osd] 2024-04-25T10:28:09.448 INFO:teuthology.orchestra.run.smithi037.stdout: osd journal size = 100 2024-04-25T10:28:09.448 INFO:teuthology.orchestra.run.smithi037.stdout: 2024-04-25T10:28:09.448 INFO:teuthology.orchestra.run.smithi037.stdout: osd scrub load threshold = 5.0 2024-04-25T10:28:09.449 INFO:teuthology.orchestra.run.smithi037.stdout: osd scrub max interval = 600 2024-04-25T10:28:09.449 INFO:teuthology.orchestra.run.smithi037.stdout: osd mclock profile = high_recovery_ops 2024-04-25T10:28:09.449 INFO:teuthology.orchestra.run.smithi037.stdout: 2024-04-25T10:28:09.449 INFO:teuthology.orchestra.run.smithi037.stdout: osd recover clone overlap = true 2024-04-25T10:28:09.449 INFO:teuthology.orchestra.run.smithi037.stdout: osd recovery max chunk = 1048576 2024-04-25T10:28:09.449 INFO:teuthology.orchestra.run.smithi037.stdout: 2024-04-25T10:28:09.449 INFO:teuthology.orchestra.run.smithi037.stdout: osd debug shutdown = true 2024-04-25T10:28:09.449 INFO:teuthology.orchestra.run.smithi037.stdout: osd debug op order = true 2024-04-25T10:28:09.449 INFO:teuthology.orchestra.run.smithi037.stdout: osd debug verify stray on activate = true 2024-04-25T10:28:09.449 INFO:teuthology.orchestra.run.smithi037.stdout: osd debug trim objects = true 2024-04-25T10:28:09.449 INFO:teuthology.orchestra.run.smithi037.stdout: 2024-04-25T10:28:09.449 INFO:teuthology.orchestra.run.smithi037.stdout: osd open classes on start = true 2024-04-25T10:28:09.449 INFO:teuthology.orchestra.run.smithi037.stdout: osd debug pg log writeout = true 2024-04-25T10:28:09.449 INFO:teuthology.orchestra.run.smithi037.stdout: 2024-04-25T10:28:09.450 INFO:teuthology.orchestra.run.smithi037.stdout: osd deep scrub update digest min age = 30 2024-04-25T10:28:09.450 INFO:teuthology.orchestra.run.smithi037.stdout: 2024-04-25T10:28:09.450 INFO:teuthology.orchestra.run.smithi037.stdout: osd map max advance = 10 2024-04-25T10:28:09.450 INFO:teuthology.orchestra.run.smithi037.stdout: 2024-04-25T10:28:09.450 INFO:teuthology.orchestra.run.smithi037.stdout: journal zero on create = true 2024-04-25T10:28:09.450 INFO:teuthology.orchestra.run.smithi037.stdout: 2024-04-25T10:28:09.450 INFO:teuthology.orchestra.run.smithi037.stdout: filestore ondisk finisher threads = 3 2024-04-25T10:28:09.450 INFO:teuthology.orchestra.run.smithi037.stdout: filestore apply finisher threads = 3 2024-04-25T10:28:09.450 INFO:teuthology.orchestra.run.smithi037.stdout: 2024-04-25T10:28:09.450 INFO:teuthology.orchestra.run.smithi037.stdout: bdev debug aio = true 2024-04-25T10:28:09.450 INFO:teuthology.orchestra.run.smithi037.stdout: osd debug misdirected ops = true 2024-04-25T10:28:09.450 INFO:teuthology.orchestra.run.smithi037.stdout: bdev async discard = True 2024-04-25T10:28:09.450 INFO:teuthology.orchestra.run.smithi037.stdout: bdev enable discard = True 2024-04-25T10:28:09.450 INFO:teuthology.orchestra.run.smithi037.stdout: bluestore allocator = bitmap 2024-04-25T10:28:09.451 INFO:teuthology.orchestra.run.smithi037.stdout: bluestore block size = 96636764160 2024-04-25T10:28:09.451 INFO:teuthology.orchestra.run.smithi037.stdout: bluestore fsck on mount = True 2024-04-25T10:28:09.451 INFO:teuthology.orchestra.run.smithi037.stdout: debug bluefs = 1/20 2024-04-25T10:28:09.451 INFO:teuthology.orchestra.run.smithi037.stdout: debug bluestore = 1/20 2024-04-25T10:28:09.451 INFO:teuthology.orchestra.run.smithi037.stdout: debug ms = 1 2024-04-25T10:28:09.451 INFO:teuthology.orchestra.run.smithi037.stdout: debug osd = 20 2024-04-25T10:28:09.451 INFO:teuthology.orchestra.run.smithi037.stdout: debug rocksdb = 4/10 2024-04-25T10:28:09.451 INFO:teuthology.orchestra.run.smithi037.stdout: mon osd backfillfull_ratio = 0.85 2024-04-25T10:28:09.451 INFO:teuthology.orchestra.run.smithi037.stdout: mon osd full ratio = 0.9 2024-04-25T10:28:09.451 INFO:teuthology.orchestra.run.smithi037.stdout: mon osd nearfull ratio = 0.8 2024-04-25T10:28:09.451 INFO:teuthology.orchestra.run.smithi037.stdout: osd failsafe full ratio = 0.95 2024-04-25T10:28:09.451 INFO:teuthology.orchestra.run.smithi037.stdout: osd objectstore = bluestore 2024-04-25T10:28:09.451 INFO:teuthology.orchestra.run.smithi037.stdout: osd op complaint time = 180 2024-04-25T10:28:09.451 INFO:teuthology.orchestra.run.smithi037.stdout: 2024-04-25T10:28:09.451 INFO:teuthology.orchestra.run.smithi037.stdout:[mgr] 2024-04-25T10:28:09.452 INFO:teuthology.orchestra.run.smithi037.stdout: debug ms = 1 2024-04-25T10:28:09.452 INFO:teuthology.orchestra.run.smithi037.stdout: debug mgr = 20 2024-04-25T10:28:09.452 INFO:teuthology.orchestra.run.smithi037.stdout: debug mon = 20 2024-04-25T10:28:09.452 INFO:teuthology.orchestra.run.smithi037.stdout: debug auth = 20 2024-04-25T10:28:09.452 INFO:teuthology.orchestra.run.smithi037.stdout: mon reweight min pgs per osd = 4 2024-04-25T10:28:09.452 INFO:teuthology.orchestra.run.smithi037.stdout: mon reweight min bytes per osd = 10 2024-04-25T10:28:09.452 INFO:teuthology.orchestra.run.smithi037.stdout: mgr/telemetry/nag = false 2024-04-25T10:28:09.452 INFO:teuthology.orchestra.run.smithi037.stdout: debug client = 20 2024-04-25T10:28:09.452 INFO:teuthology.orchestra.run.smithi037.stdout: 2024-04-25T10:28:09.452 INFO:teuthology.orchestra.run.smithi037.stdout:[mon] 2024-04-25T10:28:09.452 INFO:teuthology.orchestra.run.smithi037.stdout: debug ms = 1 2024-04-25T10:28:09.452 INFO:teuthology.orchestra.run.smithi037.stdout: debug mon = 20 2024-04-25T10:28:09.452 INFO:teuthology.orchestra.run.smithi037.stdout: debug paxos = 20 2024-04-25T10:28:09.452 INFO:teuthology.orchestra.run.smithi037.stdout: debug auth = 20 2024-04-25T10:28:09.453 INFO:teuthology.orchestra.run.smithi037.stdout: mon data avail warn = 5 2024-04-25T10:28:09.453 INFO:teuthology.orchestra.run.smithi037.stdout: mon mgr mkfs grace = 240 2024-04-25T10:28:09.453 INFO:teuthology.orchestra.run.smithi037.stdout: mon reweight min pgs per osd = 4 2024-04-25T10:28:09.453 INFO:teuthology.orchestra.run.smithi037.stdout: mon osd reporter subtree level = osd 2024-04-25T10:28:09.453 INFO:teuthology.orchestra.run.smithi037.stdout: mon osd prime pg temp = true 2024-04-25T10:28:09.453 INFO:teuthology.orchestra.run.smithi037.stdout: mon reweight min bytes per osd = 10 2024-04-25T10:28:09.453 INFO:teuthology.orchestra.run.smithi037.stdout: 2024-04-25T10:28:09.453 INFO:teuthology.orchestra.run.smithi037.stdout: # rotate auth tickets quickly to exercise renewal paths 2024-04-25T10:28:09.453 INFO:teuthology.orchestra.run.smithi037.stdout: auth mon ticket ttl = 660 # 11m 2024-04-25T10:28:09.453 INFO:teuthology.orchestra.run.smithi037.stdout: auth service ticket ttl = 240 # 4m 2024-04-25T10:28:09.453 INFO:teuthology.orchestra.run.smithi037.stdout: 2024-04-25T10:28:09.453 INFO:teuthology.orchestra.run.smithi037.stdout: # don't complain about insecure global_id in the test suite 2024-04-25T10:28:09.453 INFO:teuthology.orchestra.run.smithi037.stdout: mon_warn_on_insecure_global_id_reclaim = false 2024-04-25T10:28:09.453 INFO:teuthology.orchestra.run.smithi037.stdout: mon_warn_on_insecure_global_id_reclaim_allowed = false 2024-04-25T10:28:09.453 INFO:teuthology.orchestra.run.smithi037.stdout: 2024-04-25T10:28:09.454 INFO:teuthology.orchestra.run.smithi037.stdout: # 1m isn't quite enough 2024-04-25T10:28:09.454 INFO:teuthology.orchestra.run.smithi037.stdout: mon_down_mkfs_grace = 2m 2024-04-25T10:28:09.454 INFO:teuthology.orchestra.run.smithi037.stdout: 2024-04-25T10:28:09.454 INFO:teuthology.orchestra.run.smithi037.stdout: mon_warn_on_filestore_osds = false 2024-04-25T10:28:09.454 INFO:teuthology.orchestra.run.smithi037.stdout: mon down mkfs grace = 300 2024-04-25T10:28:09.454 INFO:teuthology.orchestra.run.smithi037.stdout: mon op complaint time = 120 2024-04-25T10:28:09.454 INFO:teuthology.orchestra.run.smithi037.stdout: 2024-04-25T10:28:09.454 INFO:teuthology.orchestra.run.smithi037.stdout:[client] 2024-04-25T10:28:09.454 INFO:teuthology.orchestra.run.smithi037.stdout: rgw cache enabled = true 2024-04-25T10:28:09.454 INFO:teuthology.orchestra.run.smithi037.stdout: rgw enable ops log = true 2024-04-25T10:28:09.454 INFO:teuthology.orchestra.run.smithi037.stdout: rgw enable usage log = true 2024-04-25T10:28:09.454 INFO:teuthology.orchestra.run.smithi037.stdout: log file = /var/log/ceph/$cluster-$name.$pid.log 2024-04-25T10:28:09.454 INFO:teuthology.orchestra.run.smithi037.stdout: admin socket = /var/run/ceph/$cluster-$name.$pid.asok 2024-04-25T10:28:09.454 INFO:teuthology.orchestra.run.smithi037.stdout: client mount timeout = 600 2024-04-25T10:28:09.454 INFO:teuthology.orchestra.run.smithi037.stdout: debug client = 20 2024-04-25T10:28:09.455 INFO:teuthology.orchestra.run.smithi037.stdout: debug ms = 1 2024-04-25T10:28:09.455 INFO:teuthology.orchestra.run.smithi037.stdout: pid file = "" 2024-04-25T10:28:09.455 INFO:teuthology.orchestra.run.smithi037.stdout: rados mon op timeout = 900 2024-04-25T10:28:09.455 INFO:teuthology.orchestra.run.smithi037.stdout: rados osd op timeout = 900 2024-04-25T10:28:09.455 INFO:teuthology.orchestra.run.smithi037.stdout:[mon.a] 2024-04-25T10:28:09.455 INFO:teuthology.orchestra.run.smithi037.stdout:[mon.b] 2024-04-25T10:28:09.455 INFO:teuthology.orchestra.run.smithi037.stdout:[mon.c] 2024-04-25T10:28:09.455 INFO:teuthology.orchestra.run.smithi037.stdout:[mds.a] 2024-04-25T10:28:09.455 INFO:teuthology.orchestra.run.smithi037.stdout:[mds.c] 2024-04-25T10:28:09.455 INFO:teuthology.orchestra.run.smithi037.stdout:[mds.b] 2024-04-25T10:28:09.455 INFO:teuthology.orchestra.run.smithi037.stdout:[mds.d] 2024-04-25T10:28:09.455 INFO:teuthology.orchestra.run.smithi037.stdout:[mds] 2024-04-25T10:28:09.455 INFO:teuthology.orchestra.run.smithi037.stdout: debug mds = 20 2024-04-25T10:28:09.455 INFO:teuthology.orchestra.run.smithi037.stdout: debug mds balancer = 20 2024-04-25T10:28:09.456 INFO:teuthology.orchestra.run.smithi037.stdout: debug ms = 1 2024-04-25T10:28:09.456 INFO:teuthology.orchestra.run.smithi037.stdout: mds debug frag = True 2024-04-25T10:28:09.456 INFO:teuthology.orchestra.run.smithi037.stdout: mds debug scatterstat = True 2024-04-25T10:28:09.456 INFO:teuthology.orchestra.run.smithi037.stdout: mds op complaint time = 180 2024-04-25T10:28:09.456 INFO:teuthology.orchestra.run.smithi037.stdout: mds verify scatter = True 2024-04-25T10:28:09.456 INFO:teuthology.orchestra.run.smithi037.stdout: osd op complaint time = 180 2024-04-25T10:28:09.456 INFO:teuthology.orchestra.run.smithi037.stdout: rados mon op timeout = 900 2024-04-25T10:28:09.456 INFO:teuthology.orchestra.run.smithi037.stdout: rados osd op timeout = 900 2024-04-25T10:28:09.457 INFO:teuthology.orchestra.run.smithi203.stdout:[global] 2024-04-25T10:28:09.457 INFO:teuthology.orchestra.run.smithi203.stdout: chdir = "" 2024-04-25T10:28:09.457 INFO:teuthology.orchestra.run.smithi203.stdout: pid file = /var/run/ceph/$cluster-$name.pid 2024-04-25T10:28:09.457 INFO:teuthology.orchestra.run.smithi203.stdout: auth supported = cephx 2024-04-25T10:28:09.457 INFO:teuthology.orchestra.run.smithi203.stdout: 2024-04-25T10:28:09.457 INFO:teuthology.orchestra.run.smithi203.stdout: filestore xattr use omap = true 2024-04-25T10:28:09.457 INFO:teuthology.orchestra.run.smithi203.stdout: 2024-04-25T10:28:09.457 INFO:teuthology.orchestra.run.smithi203.stdout: mon clock drift allowed = 1.000 2024-04-25T10:28:09.457 INFO:teuthology.orchestra.run.smithi203.stdout: 2024-04-25T10:28:09.457 INFO:teuthology.orchestra.run.smithi203.stdout: osd crush chooseleaf type = 0 2024-04-25T10:28:09.457 INFO:teuthology.orchestra.run.smithi203.stdout: auth debug = true 2024-04-25T10:28:09.457 INFO:teuthology.orchestra.run.smithi203.stdout: 2024-04-25T10:28:09.457 INFO:teuthology.orchestra.run.smithi203.stdout: ms die on old message = true 2024-04-25T10:28:09.458 INFO:teuthology.orchestra.run.smithi203.stdout: ms die on bug = true 2024-04-25T10:28:09.458 INFO:teuthology.orchestra.run.smithi203.stdout: 2024-04-25T10:28:09.458 INFO:teuthology.orchestra.run.smithi203.stdout: mon max pg per osd = 10000 # >= luminous 2024-04-25T10:28:09.458 INFO:teuthology.orchestra.run.smithi203.stdout: mon pg warn max object skew = 0 2024-04-25T10:28:09.458 INFO:teuthology.orchestra.run.smithi203.stdout: 2024-04-25T10:28:09.458 INFO:teuthology.orchestra.run.smithi203.stdout: # disable pg_autoscaler by default for new pools 2024-04-25T10:28:09.458 INFO:teuthology.orchestra.run.smithi203.stdout: osd_pool_default_pg_autoscale_mode = off 2024-04-25T10:28:09.458 INFO:teuthology.orchestra.run.smithi203.stdout: 2024-04-25T10:28:09.458 INFO:teuthology.orchestra.run.smithi203.stdout: osd pool default size = 2 2024-04-25T10:28:09.458 INFO:teuthology.orchestra.run.smithi203.stdout: 2024-04-25T10:28:09.458 INFO:teuthology.orchestra.run.smithi203.stdout: mon osd allow primary affinity = true 2024-04-25T10:28:09.458 INFO:teuthology.orchestra.run.smithi203.stdout: mon osd allow pg remap = true 2024-04-25T10:28:09.458 INFO:teuthology.orchestra.run.smithi203.stdout: mon warn on legacy crush tunables = false 2024-04-25T10:28:09.458 INFO:teuthology.orchestra.run.smithi203.stdout: mon warn on crush straw calc version zero = false 2024-04-25T10:28:09.459 INFO:teuthology.orchestra.run.smithi203.stdout: mon warn on no sortbitwise = false 2024-04-25T10:28:09.459 INFO:teuthology.orchestra.run.smithi203.stdout: mon warn on osd down out interval zero = false 2024-04-25T10:28:09.459 INFO:teuthology.orchestra.run.smithi203.stdout: mon warn on too few osds = false 2024-04-25T10:28:09.459 INFO:teuthology.orchestra.run.smithi203.stdout: mon_warn_on_pool_pg_num_not_power_of_two = false 2024-04-25T10:28:09.459 INFO:teuthology.orchestra.run.smithi203.stdout: mon_warn_on_pool_no_redundancy = false 2024-04-25T10:28:09.459 INFO:teuthology.orchestra.run.smithi203.stdout: mon_allow_pool_size_one = true 2024-04-25T10:28:09.459 INFO:teuthology.orchestra.run.smithi203.stdout: 2024-04-25T10:28:09.459 INFO:teuthology.orchestra.run.smithi203.stdout: osd pool default erasure code profile = plugin=jerasure technique=reed_sol_van k=2 m=1 crush-failure-domain=osd 2024-04-25T10:28:09.459 INFO:teuthology.orchestra.run.smithi203.stdout: 2024-04-25T10:28:09.459 INFO:teuthology.orchestra.run.smithi203.stdout: osd default data pool replay window = 5 2024-04-25T10:28:09.459 INFO:teuthology.orchestra.run.smithi203.stdout: 2024-04-25T10:28:09.459 INFO:teuthology.orchestra.run.smithi203.stdout: mon allow pool delete = true 2024-04-25T10:28:09.459 INFO:teuthology.orchestra.run.smithi203.stdout: 2024-04-25T10:28:09.459 INFO:teuthology.orchestra.run.smithi203.stdout: mon cluster log file level = debug 2024-04-25T10:28:09.459 INFO:teuthology.orchestra.run.smithi203.stdout: debug asserts on shutdown = true 2024-04-25T10:28:09.460 INFO:teuthology.orchestra.run.smithi203.stdout: mon health detail to clog = false 2024-04-25T10:28:09.460 INFO:teuthology.orchestra.run.smithi203.stdout: mon host = "172.21.15.37,172.21.15.203,[v2:172.21.15.203:3301,v1:172.21.15.203:6790]" 2024-04-25T10:28:09.460 INFO:teuthology.orchestra.run.smithi203.stdout: fsid = f4acda28-be34-492a-b286-994afd36f61d 2024-04-25T10:28:09.460 INFO:teuthology.orchestra.run.smithi203.stdout: 2024-04-25T10:28:09.460 INFO:teuthology.orchestra.run.smithi203.stdout:[osd] 2024-04-25T10:28:09.460 INFO:teuthology.orchestra.run.smithi203.stdout: osd journal size = 100 2024-04-25T10:28:09.460 INFO:teuthology.orchestra.run.smithi203.stdout: 2024-04-25T10:28:09.460 INFO:teuthology.orchestra.run.smithi203.stdout: osd scrub load threshold = 5.0 2024-04-25T10:28:09.460 INFO:teuthology.orchestra.run.smithi203.stdout: osd scrub max interval = 600 2024-04-25T10:28:09.460 INFO:teuthology.orchestra.run.smithi203.stdout: osd mclock profile = high_recovery_ops 2024-04-25T10:28:09.460 INFO:teuthology.orchestra.run.smithi203.stdout: 2024-04-25T10:28:09.460 INFO:teuthology.orchestra.run.smithi203.stdout: osd recover clone overlap = true 2024-04-25T10:28:09.460 INFO:teuthology.orchestra.run.smithi203.stdout: osd recovery max chunk = 1048576 2024-04-25T10:28:09.460 INFO:teuthology.orchestra.run.smithi203.stdout: 2024-04-25T10:28:09.460 INFO:teuthology.orchestra.run.smithi203.stdout: osd debug shutdown = true 2024-04-25T10:28:09.460 INFO:teuthology.orchestra.run.smithi203.stdout: osd debug op order = true 2024-04-25T10:28:09.461 INFO:teuthology.orchestra.run.smithi203.stdout: osd debug verify stray on activate = true 2024-04-25T10:28:09.461 INFO:teuthology.orchestra.run.smithi203.stdout: osd debug trim objects = true 2024-04-25T10:28:09.461 INFO:teuthology.orchestra.run.smithi203.stdout: 2024-04-25T10:28:09.461 INFO:teuthology.orchestra.run.smithi203.stdout: osd open classes on start = true 2024-04-25T10:28:09.461 INFO:teuthology.orchestra.run.smithi203.stdout: osd debug pg log writeout = true 2024-04-25T10:28:09.461 INFO:teuthology.orchestra.run.smithi203.stdout: 2024-04-25T10:28:09.461 INFO:teuthology.orchestra.run.smithi203.stdout: osd deep scrub update digest min age = 30 2024-04-25T10:28:09.461 INFO:teuthology.orchestra.run.smithi203.stdout: 2024-04-25T10:28:09.461 INFO:teuthology.orchestra.run.smithi203.stdout: osd map max advance = 10 2024-04-25T10:28:09.461 INFO:teuthology.orchestra.run.smithi203.stdout: 2024-04-25T10:28:09.461 INFO:teuthology.orchestra.run.smithi203.stdout: journal zero on create = true 2024-04-25T10:28:09.461 INFO:teuthology.orchestra.run.smithi203.stdout: 2024-04-25T10:28:09.461 INFO:teuthology.orchestra.run.smithi203.stdout: filestore ondisk finisher threads = 3 2024-04-25T10:28:09.461 INFO:teuthology.orchestra.run.smithi203.stdout: filestore apply finisher threads = 3 2024-04-25T10:28:09.461 INFO:teuthology.orchestra.run.smithi203.stdout: 2024-04-25T10:28:09.461 INFO:teuthology.orchestra.run.smithi203.stdout: bdev debug aio = true 2024-04-25T10:28:09.462 INFO:teuthology.orchestra.run.smithi203.stdout: osd debug misdirected ops = true 2024-04-25T10:28:09.462 INFO:teuthology.orchestra.run.smithi203.stdout: bdev async discard = True 2024-04-25T10:28:09.462 INFO:teuthology.orchestra.run.smithi203.stdout: bdev enable discard = True 2024-04-25T10:28:09.462 INFO:teuthology.orchestra.run.smithi203.stdout: bluestore allocator = bitmap 2024-04-25T10:28:09.462 INFO:teuthology.orchestra.run.smithi203.stdout: bluestore block size = 96636764160 2024-04-25T10:28:09.462 INFO:teuthology.orchestra.run.smithi203.stdout: bluestore fsck on mount = True 2024-04-25T10:28:09.462 INFO:teuthology.orchestra.run.smithi203.stdout: debug bluefs = 1/20 2024-04-25T10:28:09.462 INFO:teuthology.orchestra.run.smithi203.stdout: debug bluestore = 1/20 2024-04-25T10:28:09.462 INFO:teuthology.orchestra.run.smithi203.stdout: debug ms = 1 2024-04-25T10:28:09.462 INFO:teuthology.orchestra.run.smithi203.stdout: debug osd = 20 2024-04-25T10:28:09.462 INFO:teuthology.orchestra.run.smithi203.stdout: debug rocksdb = 4/10 2024-04-25T10:28:09.462 INFO:teuthology.orchestra.run.smithi203.stdout: mon osd backfillfull_ratio = 0.85 2024-04-25T10:28:09.462 INFO:teuthology.orchestra.run.smithi203.stdout: mon osd full ratio = 0.9 2024-04-25T10:28:09.462 INFO:teuthology.orchestra.run.smithi203.stdout: mon osd nearfull ratio = 0.8 2024-04-25T10:28:09.463 INFO:teuthology.orchestra.run.smithi203.stdout: osd failsafe full ratio = 0.95 2024-04-25T10:28:09.463 INFO:teuthology.orchestra.run.smithi203.stdout: osd objectstore = bluestore 2024-04-25T10:28:09.463 INFO:teuthology.orchestra.run.smithi203.stdout: osd op complaint time = 180 2024-04-25T10:28:09.463 INFO:teuthology.orchestra.run.smithi203.stdout: 2024-04-25T10:28:09.463 INFO:teuthology.orchestra.run.smithi203.stdout:[mgr] 2024-04-25T10:28:09.463 INFO:teuthology.orchestra.run.smithi203.stdout: debug ms = 1 2024-04-25T10:28:09.463 INFO:teuthology.orchestra.run.smithi203.stdout: debug mgr = 20 2024-04-25T10:28:09.463 INFO:teuthology.orchestra.run.smithi203.stdout: debug mon = 20 2024-04-25T10:28:09.463 INFO:teuthology.orchestra.run.smithi203.stdout: debug auth = 20 2024-04-25T10:28:09.463 INFO:teuthology.orchestra.run.smithi203.stdout: mon reweight min pgs per osd = 4 2024-04-25T10:28:09.463 INFO:teuthology.orchestra.run.smithi203.stdout: mon reweight min bytes per osd = 10 2024-04-25T10:28:09.463 INFO:teuthology.orchestra.run.smithi203.stdout: mgr/telemetry/nag = false 2024-04-25T10:28:09.463 INFO:teuthology.orchestra.run.smithi203.stdout: debug client = 20 2024-04-25T10:28:09.463 INFO:teuthology.orchestra.run.smithi203.stdout: 2024-04-25T10:28:09.463 INFO:teuthology.orchestra.run.smithi203.stdout:[mon] 2024-04-25T10:28:09.463 INFO:teuthology.orchestra.run.smithi203.stdout: debug ms = 1 2024-04-25T10:28:09.463 INFO:teuthology.orchestra.run.smithi203.stdout: debug mon = 20 2024-04-25T10:28:09.464 INFO:teuthology.orchestra.run.smithi203.stdout: debug paxos = 20 2024-04-25T10:28:09.464 INFO:teuthology.orchestra.run.smithi203.stdout: debug auth = 20 2024-04-25T10:28:09.464 INFO:teuthology.orchestra.run.smithi203.stdout: mon data avail warn = 5 2024-04-25T10:28:09.464 INFO:teuthology.orchestra.run.smithi203.stdout: mon mgr mkfs grace = 240 2024-04-25T10:28:09.464 INFO:teuthology.orchestra.run.smithi203.stdout: mon reweight min pgs per osd = 4 2024-04-25T10:28:09.464 INFO:teuthology.orchestra.run.smithi203.stdout: mon osd reporter subtree level = osd 2024-04-25T10:28:09.464 INFO:teuthology.orchestra.run.smithi203.stdout: mon osd prime pg temp = true 2024-04-25T10:28:09.464 INFO:teuthology.orchestra.run.smithi203.stdout: mon reweight min bytes per osd = 10 2024-04-25T10:28:09.464 INFO:teuthology.orchestra.run.smithi203.stdout: 2024-04-25T10:28:09.464 INFO:teuthology.orchestra.run.smithi203.stdout: # rotate auth tickets quickly to exercise renewal paths 2024-04-25T10:28:09.464 INFO:teuthology.orchestra.run.smithi203.stdout: auth mon ticket ttl = 660 # 11m 2024-04-25T10:28:09.464 INFO:teuthology.orchestra.run.smithi203.stdout: auth service ticket ttl = 240 # 4m 2024-04-25T10:28:09.464 INFO:teuthology.orchestra.run.smithi203.stdout: 2024-04-25T10:28:09.465 INFO:teuthology.orchestra.run.smithi203.stdout: # don't complain about insecure global_id in the test suite 2024-04-25T10:28:09.465 INFO:teuthology.orchestra.run.smithi203.stdout: mon_warn_on_insecure_global_id_reclaim = false 2024-04-25T10:28:09.465 INFO:teuthology.orchestra.run.smithi203.stdout: mon_warn_on_insecure_global_id_reclaim_allowed = false 2024-04-25T10:28:09.465 INFO:teuthology.orchestra.run.smithi203.stdout: 2024-04-25T10:28:09.465 INFO:teuthology.orchestra.run.smithi203.stdout: # 1m isn't quite enough 2024-04-25T10:28:09.465 INFO:teuthology.orchestra.run.smithi203.stdout: mon_down_mkfs_grace = 2m 2024-04-25T10:28:09.465 INFO:teuthology.orchestra.run.smithi203.stdout: 2024-04-25T10:28:09.465 INFO:teuthology.orchestra.run.smithi203.stdout: mon_warn_on_filestore_osds = false 2024-04-25T10:28:09.465 INFO:teuthology.orchestra.run.smithi203.stdout: mon down mkfs grace = 300 2024-04-25T10:28:09.465 INFO:teuthology.orchestra.run.smithi203.stdout: mon op complaint time = 120 2024-04-25T10:28:09.465 INFO:teuthology.orchestra.run.smithi203.stdout: 2024-04-25T10:28:09.465 INFO:teuthology.orchestra.run.smithi203.stdout:[client] 2024-04-25T10:28:09.465 INFO:teuthology.orchestra.run.smithi203.stdout: rgw cache enabled = true 2024-04-25T10:28:09.465 INFO:teuthology.orchestra.run.smithi203.stdout: rgw enable ops log = true 2024-04-25T10:28:09.465 INFO:teuthology.orchestra.run.smithi203.stdout: rgw enable usage log = true 2024-04-25T10:28:09.466 INFO:teuthology.orchestra.run.smithi203.stdout: log file = /var/log/ceph/$cluster-$name.$pid.log 2024-04-25T10:28:09.466 INFO:teuthology.orchestra.run.smithi203.stdout: admin socket = /var/run/ceph/$cluster-$name.$pid.asok 2024-04-25T10:28:09.466 INFO:teuthology.orchestra.run.smithi203.stdout: client mount timeout = 600 2024-04-25T10:28:09.466 INFO:teuthology.orchestra.run.smithi203.stdout: debug client = 20 2024-04-25T10:28:09.466 INFO:teuthology.orchestra.run.smithi203.stdout: debug ms = 1 2024-04-25T10:28:09.466 INFO:teuthology.orchestra.run.smithi203.stdout: pid file = "" 2024-04-25T10:28:09.466 INFO:teuthology.orchestra.run.smithi203.stdout: rados mon op timeout = 900 2024-04-25T10:28:09.466 INFO:teuthology.orchestra.run.smithi203.stdout: rados osd op timeout = 900 2024-04-25T10:28:09.466 INFO:teuthology.orchestra.run.smithi203.stdout:[mon.a] 2024-04-25T10:28:09.466 INFO:teuthology.orchestra.run.smithi203.stdout:[mon.b] 2024-04-25T10:28:09.466 INFO:teuthology.orchestra.run.smithi203.stdout:[mon.c] 2024-04-25T10:28:09.466 INFO:teuthology.orchestra.run.smithi203.stdout:[mds.a] 2024-04-25T10:28:09.466 INFO:teuthology.orchestra.run.smithi203.stdout:[mds.c] 2024-04-25T10:28:09.466 INFO:teuthology.orchestra.run.smithi203.stdout:[mds.b] 2024-04-25T10:28:09.466 INFO:teuthology.orchestra.run.smithi203.stdout:[mds.d] 2024-04-25T10:28:09.467 INFO:teuthology.orchestra.run.smithi203.stdout:[mds] 2024-04-25T10:28:09.467 INFO:teuthology.orchestra.run.smithi203.stdout: debug mds = 20 2024-04-25T10:28:09.467 INFO:teuthology.orchestra.run.smithi203.stdout: debug mds balancer = 20 2024-04-25T10:28:09.467 INFO:teuthology.orchestra.run.smithi203.stdout: debug ms = 1 2024-04-25T10:28:09.467 INFO:teuthology.orchestra.run.smithi203.stdout: mds debug frag = True 2024-04-25T10:28:09.467 INFO:teuthology.orchestra.run.smithi203.stdout: mds debug scatterstat = True 2024-04-25T10:28:09.467 INFO:teuthology.orchestra.run.smithi203.stdout: mds op complaint time = 180 2024-04-25T10:28:09.467 INFO:teuthology.orchestra.run.smithi203.stdout: mds verify scatter = True 2024-04-25T10:28:09.467 INFO:teuthology.orchestra.run.smithi203.stdout: osd op complaint time = 180 2024-04-25T10:28:09.467 INFO:teuthology.orchestra.run.smithi203.stdout: rados mon op timeout = 900 2024-04-25T10:28:09.467 INFO:teuthology.orchestra.run.smithi203.stdout: rados osd op timeout = 900 2024-04-25T10:28:09.469 INFO:tasks.ceph:Creating admin key on mon.a... 2024-04-25T10:28:09.469 DEBUG:teuthology.orchestra.run.smithi037:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool --gen-key --name=client.admin --cap mon 'allow *' --cap osd 'allow *' --cap mds 'allow *' --cap mgr 'allow *' /etc/ceph/ceph.keyring 2024-04-25T10:28:09.572 INFO:tasks.ceph:Copying monmap to all nodes... 2024-04-25T10:28:09.572 DEBUG:teuthology.orchestra.run.smithi037:> set -ex 2024-04-25T10:28:09.572 DEBUG:teuthology.orchestra.run.smithi037:> dd if=/etc/ceph/ceph.keyring of=/dev/stdout 2024-04-25T10:28:09.587 DEBUG:teuthology.orchestra.run.smithi037:> set -ex 2024-04-25T10:28:09.587 DEBUG:teuthology.orchestra.run.smithi037:> dd if=/home/ubuntu/cephtest/ceph.monmap of=/dev/stdout 2024-04-25T10:28:09.642 INFO:tasks.ceph:Sending monmap to node ubuntu@smithi037.front.sepia.ceph.com 2024-04-25T10:28:09.642 DEBUG:teuthology.orchestra.run.smithi037:> set -ex 2024-04-25T10:28:09.642 DEBUG:teuthology.orchestra.run.smithi037:> sudo dd of=/etc/ceph/ceph.keyring 2024-04-25T10:28:09.642 DEBUG:teuthology.orchestra.run.smithi037:> sudo chmod 0644 /etc/ceph/ceph.keyring 2024-04-25T10:28:09.716 DEBUG:teuthology.orchestra.run.smithi037:> set -ex 2024-04-25T10:28:09.716 DEBUG:teuthology.orchestra.run.smithi037:> dd of=/home/ubuntu/cephtest/ceph.monmap 2024-04-25T10:28:09.771 INFO:tasks.ceph:Sending monmap to node ubuntu@smithi203.front.sepia.ceph.com 2024-04-25T10:28:09.771 DEBUG:teuthology.orchestra.run.smithi203:> set -ex 2024-04-25T10:28:09.771 DEBUG:teuthology.orchestra.run.smithi203:> sudo dd of=/etc/ceph/ceph.keyring 2024-04-25T10:28:09.771 DEBUG:teuthology.orchestra.run.smithi203:> sudo chmod 0644 /etc/ceph/ceph.keyring 2024-04-25T10:28:09.805 DEBUG:teuthology.orchestra.run.smithi203:> set -ex 2024-04-25T10:28:09.805 DEBUG:teuthology.orchestra.run.smithi203:> dd of=/home/ubuntu/cephtest/ceph.monmap 2024-04-25T10:28:09.860 INFO:tasks.ceph:Setting up mon nodes... 2024-04-25T10:28:09.860 INFO:tasks.ceph:Setting up mgr nodes... 2024-04-25T10:28:09.860 DEBUG:teuthology.orchestra.run.smithi037:> sudo mkdir -p /var/lib/ceph/mgr/ceph-y && sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool --create-keyring --gen-key --name=mgr.y /var/lib/ceph/mgr/ceph-y/keyring 2024-04-25T10:28:09.922 INFO:teuthology.orchestra.run.smithi037.stdout:creating /var/lib/ceph/mgr/ceph-y/keyring 2024-04-25T10:28:09.923 DEBUG:teuthology.orchestra.run.smithi203:> sudo mkdir -p /var/lib/ceph/mgr/ceph-x && sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool --create-keyring --gen-key --name=mgr.x /var/lib/ceph/mgr/ceph-x/keyring 2024-04-25T10:28:10.001 INFO:teuthology.orchestra.run.smithi203.stdout:creating /var/lib/ceph/mgr/ceph-x/keyring 2024-04-25T10:28:10.005 INFO:tasks.ceph:Setting up mds nodes... 2024-04-25T10:28:10.005 DEBUG:teuthology.orchestra.run.smithi037:> sudo mkdir -p /var/lib/ceph/mds/ceph-a && sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool --create-keyring --gen-key --name=mds.a /var/lib/ceph/mds/ceph-a/keyring 2024-04-25T10:28:10.064 INFO:teuthology.orchestra.run.smithi037.stdout:creating /var/lib/ceph/mds/ceph-a/keyring 2024-04-25T10:28:10.067 DEBUG:teuthology.orchestra.run.smithi037:> sudo chown -R ceph:ceph /var/lib/ceph/mds/ceph-a 2024-04-25T10:28:10.092 DEBUG:teuthology.orchestra.run.smithi037:> sudo mkdir -p /var/lib/ceph/mds/ceph-c && sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool --create-keyring --gen-key --name=mds.c /var/lib/ceph/mds/ceph-c/keyring 2024-04-25T10:28:10.190 INFO:teuthology.orchestra.run.smithi037.stdout:creating /var/lib/ceph/mds/ceph-c/keyring 2024-04-25T10:28:10.193 DEBUG:teuthology.orchestra.run.smithi037:> sudo chown -R ceph:ceph /var/lib/ceph/mds/ceph-c 2024-04-25T10:28:10.217 DEBUG:teuthology.orchestra.run.smithi203:> sudo mkdir -p /var/lib/ceph/mds/ceph-b && sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool --create-keyring --gen-key --name=mds.b /var/lib/ceph/mds/ceph-b/keyring 2024-04-25T10:28:10.274 INFO:teuthology.orchestra.run.smithi203.stdout:creating /var/lib/ceph/mds/ceph-b/keyring 2024-04-25T10:28:10.277 DEBUG:teuthology.orchestra.run.smithi203:> sudo chown -R ceph:ceph /var/lib/ceph/mds/ceph-b 2024-04-25T10:28:10.300 DEBUG:teuthology.orchestra.run.smithi203:> sudo mkdir -p /var/lib/ceph/mds/ceph-d && sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool --create-keyring --gen-key --name=mds.d /var/lib/ceph/mds/ceph-d/keyring 2024-04-25T10:28:10.397 INFO:teuthology.orchestra.run.smithi203.stdout:creating /var/lib/ceph/mds/ceph-d/keyring 2024-04-25T10:28:10.400 DEBUG:teuthology.orchestra.run.smithi203:> sudo chown -R ceph:ceph /var/lib/ceph/mds/ceph-d 2024-04-25T10:28:10.423 INFO:tasks.ceph_client:Setting up client nodes... 2024-04-25T10:28:10.423 DEBUG:teuthology.orchestra.run.smithi037:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool --create-keyring --gen-key --name=client.0 /etc/ceph/ceph.client.0.keyring && sudo chmod 0644 /etc/ceph/ceph.client.0.keyring 2024-04-25T10:28:10.474 INFO:teuthology.orchestra.run.smithi037.stdout:creating /etc/ceph/ceph.client.0.keyring 2024-04-25T10:28:10.487 DEBUG:teuthology.orchestra.run.smithi037:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool --create-keyring --gen-key --name=client.2 /etc/ceph/ceph.client.2.keyring && sudo chmod 0644 /etc/ceph/ceph.client.2.keyring 2024-04-25T10:28:10.536 INFO:teuthology.orchestra.run.smithi037.stdout:creating /etc/ceph/ceph.client.2.keyring 2024-04-25T10:28:10.549 DEBUG:teuthology.orchestra.run.smithi203:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool --create-keyring --gen-key --name=client.1 /etc/ceph/ceph.client.1.keyring && sudo chmod 0644 /etc/ceph/ceph.client.1.keyring 2024-04-25T10:28:10.597 INFO:teuthology.orchestra.run.smithi203.stdout:creating /etc/ceph/ceph.client.1.keyring 2024-04-25T10:28:10.610 DEBUG:teuthology.orchestra.run.smithi203:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool --create-keyring --gen-key --name=client.3 /etc/ceph/ceph.client.3.keyring && sudo chmod 0644 /etc/ceph/ceph.client.3.keyring 2024-04-25T10:28:10.658 INFO:teuthology.orchestra.run.smithi203.stdout:creating /etc/ceph/ceph.client.3.keyring 2024-04-25T10:28:10.670 INFO:tasks.ceph:Running mkfs on osd nodes... 2024-04-25T10:28:10.671 INFO:tasks.ceph:ctx.disk_config.remote_to_roles_to_dev: {Remote(name='ubuntu@smithi037.front.sepia.ceph.com'): {'osd.0': '/dev/vg_nvme/lv_1', 'osd.1': '/dev/vg_nvme/lv_2', 'osd.2': '/dev/vg_nvme/lv_3', 'osd.3': '/dev/vg_nvme/lv_4'}, Remote(name='ubuntu@smithi203.front.sepia.ceph.com'): {'osd.4': '/dev/vg_nvme/lv_1', 'osd.5': '/dev/vg_nvme/lv_2', 'osd.6': '/dev/vg_nvme/lv_3', 'osd.7': '/dev/vg_nvme/lv_4'}} 2024-04-25T10:28:10.671 DEBUG:teuthology.orchestra.run.smithi037:> sudo mkdir -p /var/lib/ceph/osd/ceph-0 2024-04-25T10:28:10.695 INFO:tasks.ceph:roles_to_devs: {'osd.0': '/dev/vg_nvme/lv_1', 'osd.1': '/dev/vg_nvme/lv_2', 'osd.2': '/dev/vg_nvme/lv_3', 'osd.3': '/dev/vg_nvme/lv_4'} 2024-04-25T10:28:10.695 INFO:tasks.ceph:role: osd.0 2024-04-25T10:28:10.695 INFO:tasks.ceph:['mkfs.xfs', '-f', '-i', 'size=2048'] on /dev/vg_nvme/lv_1 on ubuntu@smithi037.front.sepia.ceph.com 2024-04-25T10:28:10.695 DEBUG:teuthology.orchestra.run.smithi037:> yes | sudo mkfs.xfs -f -i size=2048 /dev/vg_nvme/lv_1 2024-04-25T10:28:10.769 INFO:teuthology.orchestra.run.smithi037.stdout:meta-data=/dev/vg_nvme/lv_1 isize=2048 agcount=4, agsize=5859072 blks 2024-04-25T10:28:10.769 INFO:teuthology.orchestra.run.smithi037.stdout: = sectsz=512 attr=2, projid32bit=1 2024-04-25T10:28:10.769 INFO:teuthology.orchestra.run.smithi037.stdout: = crc=1 finobt=1, sparse=1, rmapbt=0 2024-04-25T10:28:10.769 INFO:teuthology.orchestra.run.smithi037.stdout: = reflink=1 bigtime=1 inobtcount=1 nrext64=0 2024-04-25T10:28:10.769 INFO:teuthology.orchestra.run.smithi037.stdout:data = bsize=4096 blocks=23436288, imaxpct=25 2024-04-25T10:28:10.769 INFO:teuthology.orchestra.run.smithi037.stdout: = sunit=0 swidth=0 blks 2024-04-25T10:28:10.769 INFO:teuthology.orchestra.run.smithi037.stdout:naming =version 2 bsize=4096 ascii-ci=0, ftype=1 2024-04-25T10:28:10.769 INFO:teuthology.orchestra.run.smithi037.stdout:log =internal log bsize=4096 blocks=16384, version=2 2024-04-25T10:28:10.770 INFO:teuthology.orchestra.run.smithi037.stdout: = sectsz=512 sunit=0 blks, lazy-count=1 2024-04-25T10:28:10.770 INFO:teuthology.orchestra.run.smithi037.stdout:realtime =none extsz=4096 blocks=0, rtextents=0 2024-04-25T10:28:11.279 INFO:teuthology.orchestra.run.smithi037.stdout:Discarding blocks...Done. 2024-04-25T10:28:11.281 INFO:tasks.ceph:mount /dev/vg_nvme/lv_1 on ubuntu@smithi037.front.sepia.ceph.com -o noatime 2024-04-25T10:28:11.281 DEBUG:teuthology.orchestra.run.smithi037:> sudo mount -t xfs -o noatime /dev/vg_nvme/lv_1 /var/lib/ceph/osd/ceph-0 2024-04-25T10:28:11.408 DEBUG:teuthology.orchestra.run.smithi037:> sudo /sbin/restorecon /var/lib/ceph/osd/ceph-0 2024-04-25T10:28:11.442 DEBUG:teuthology.orchestra.run.smithi037:> sudo mkdir -p /var/lib/ceph/osd/ceph-1 2024-04-25T10:28:11.508 INFO:tasks.ceph:roles_to_devs: {'osd.0': '/dev/vg_nvme/lv_1', 'osd.1': '/dev/vg_nvme/lv_2', 'osd.2': '/dev/vg_nvme/lv_3', 'osd.3': '/dev/vg_nvme/lv_4'} 2024-04-25T10:28:11.509 INFO:tasks.ceph:role: osd.1 2024-04-25T10:28:11.509 INFO:tasks.ceph:['mkfs.xfs', '-f', '-i', 'size=2048'] on /dev/vg_nvme/lv_2 on ubuntu@smithi037.front.sepia.ceph.com 2024-04-25T10:28:11.509 DEBUG:teuthology.orchestra.run.smithi037:> yes | sudo mkfs.xfs -f -i size=2048 /dev/vg_nvme/lv_2 2024-04-25T10:28:11.582 INFO:teuthology.orchestra.run.smithi037.stdout:meta-data=/dev/vg_nvme/lv_2 isize=2048 agcount=4, agsize=5859072 blks 2024-04-25T10:28:11.583 INFO:teuthology.orchestra.run.smithi037.stdout: = sectsz=512 attr=2, projid32bit=1 2024-04-25T10:28:11.583 INFO:teuthology.orchestra.run.smithi037.stdout: = crc=1 finobt=1, sparse=1, rmapbt=0 2024-04-25T10:28:11.583 INFO:teuthology.orchestra.run.smithi037.stdout: = reflink=1 bigtime=1 inobtcount=1 nrext64=0 2024-04-25T10:28:11.583 INFO:teuthology.orchestra.run.smithi037.stdout:data = bsize=4096 blocks=23436288, imaxpct=25 2024-04-25T10:28:11.583 INFO:teuthology.orchestra.run.smithi037.stdout: = sunit=0 swidth=0 blks 2024-04-25T10:28:11.583 INFO:teuthology.orchestra.run.smithi037.stdout:naming =version 2 bsize=4096 ascii-ci=0, ftype=1 2024-04-25T10:28:11.583 INFO:teuthology.orchestra.run.smithi037.stdout:log =internal log bsize=4096 blocks=16384, version=2 2024-04-25T10:28:11.583 INFO:teuthology.orchestra.run.smithi037.stdout: = sectsz=512 sunit=0 blks, lazy-count=1 2024-04-25T10:28:11.583 INFO:teuthology.orchestra.run.smithi037.stdout:realtime =none extsz=4096 blocks=0, rtextents=0 2024-04-25T10:28:12.029 INFO:teuthology.orchestra.run.smithi037.stdout:Discarding blocks...Done. 2024-04-25T10:28:12.031 INFO:tasks.ceph:mount /dev/vg_nvme/lv_2 on ubuntu@smithi037.front.sepia.ceph.com -o noatime 2024-04-25T10:28:12.031 DEBUG:teuthology.orchestra.run.smithi037:> sudo mount -t xfs -o noatime /dev/vg_nvme/lv_2 /var/lib/ceph/osd/ceph-1 2024-04-25T10:28:12.114 DEBUG:teuthology.orchestra.run.smithi037:> sudo /sbin/restorecon /var/lib/ceph/osd/ceph-1 2024-04-25T10:28:12.182 DEBUG:teuthology.orchestra.run.smithi037:> sudo mkdir -p /var/lib/ceph/osd/ceph-2 2024-04-25T10:28:12.247 INFO:tasks.ceph:roles_to_devs: {'osd.0': '/dev/vg_nvme/lv_1', 'osd.1': '/dev/vg_nvme/lv_2', 'osd.2': '/dev/vg_nvme/lv_3', 'osd.3': '/dev/vg_nvme/lv_4'} 2024-04-25T10:28:12.248 INFO:tasks.ceph:role: osd.2 2024-04-25T10:28:12.248 INFO:tasks.ceph:['mkfs.xfs', '-f', '-i', 'size=2048'] on /dev/vg_nvme/lv_3 on ubuntu@smithi037.front.sepia.ceph.com 2024-04-25T10:28:12.248 DEBUG:teuthology.orchestra.run.smithi037:> yes | sudo mkfs.xfs -f -i size=2048 /dev/vg_nvme/lv_3 2024-04-25T10:28:12.319 INFO:teuthology.orchestra.run.smithi037.stdout:meta-data=/dev/vg_nvme/lv_3 isize=2048 agcount=4, agsize=5859072 blks 2024-04-25T10:28:12.320 INFO:teuthology.orchestra.run.smithi037.stdout: = sectsz=512 attr=2, projid32bit=1 2024-04-25T10:28:12.320 INFO:teuthology.orchestra.run.smithi037.stdout: = crc=1 finobt=1, sparse=1, rmapbt=0 2024-04-25T10:28:12.320 INFO:teuthology.orchestra.run.smithi037.stdout: = reflink=1 bigtime=1 inobtcount=1 nrext64=0 2024-04-25T10:28:12.320 INFO:teuthology.orchestra.run.smithi037.stdout:data = bsize=4096 blocks=23436288, imaxpct=25 2024-04-25T10:28:12.320 INFO:teuthology.orchestra.run.smithi037.stdout: = sunit=0 swidth=0 blks 2024-04-25T10:28:12.320 INFO:teuthology.orchestra.run.smithi037.stdout:naming =version 2 bsize=4096 ascii-ci=0, ftype=1 2024-04-25T10:28:12.320 INFO:teuthology.orchestra.run.smithi037.stdout:log =internal log bsize=4096 blocks=16384, version=2 2024-04-25T10:28:12.320 INFO:teuthology.orchestra.run.smithi037.stdout: = sectsz=512 sunit=0 blks, lazy-count=1 2024-04-25T10:28:12.320 INFO:teuthology.orchestra.run.smithi037.stdout:realtime =none extsz=4096 blocks=0, rtextents=0 2024-04-25T10:28:12.801 INFO:teuthology.orchestra.run.smithi037.stdout:Discarding blocks...Done. 2024-04-25T10:28:12.803 INFO:tasks.ceph:mount /dev/vg_nvme/lv_3 on ubuntu@smithi037.front.sepia.ceph.com -o noatime 2024-04-25T10:28:12.803 DEBUG:teuthology.orchestra.run.smithi037:> sudo mount -t xfs -o noatime /dev/vg_nvme/lv_3 /var/lib/ceph/osd/ceph-2 2024-04-25T10:28:12.929 DEBUG:teuthology.orchestra.run.smithi037:> sudo /sbin/restorecon /var/lib/ceph/osd/ceph-2 2024-04-25T10:28:12.960 DEBUG:teuthology.orchestra.run.smithi037:> sudo mkdir -p /var/lib/ceph/osd/ceph-3 2024-04-25T10:28:13.025 INFO:tasks.ceph:roles_to_devs: {'osd.0': '/dev/vg_nvme/lv_1', 'osd.1': '/dev/vg_nvme/lv_2', 'osd.2': '/dev/vg_nvme/lv_3', 'osd.3': '/dev/vg_nvme/lv_4'} 2024-04-25T10:28:13.025 INFO:tasks.ceph:role: osd.3 2024-04-25T10:28:13.025 INFO:tasks.ceph:['mkfs.xfs', '-f', '-i', 'size=2048'] on /dev/vg_nvme/lv_4 on ubuntu@smithi037.front.sepia.ceph.com 2024-04-25T10:28:13.026 DEBUG:teuthology.orchestra.run.smithi037:> yes | sudo mkfs.xfs -f -i size=2048 /dev/vg_nvme/lv_4 2024-04-25T10:28:13.098 INFO:teuthology.orchestra.run.smithi037.stdout:meta-data=/dev/vg_nvme/lv_4 isize=2048 agcount=4, agsize=5859072 blks 2024-04-25T10:28:13.098 INFO:teuthology.orchestra.run.smithi037.stdout: = sectsz=512 attr=2, projid32bit=1 2024-04-25T10:28:13.098 INFO:teuthology.orchestra.run.smithi037.stdout: = crc=1 finobt=1, sparse=1, rmapbt=0 2024-04-25T10:28:13.098 INFO:teuthology.orchestra.run.smithi037.stdout: = reflink=1 bigtime=1 inobtcount=1 nrext64=0 2024-04-25T10:28:13.098 INFO:teuthology.orchestra.run.smithi037.stdout:data = bsize=4096 blocks=23436288, imaxpct=25 2024-04-25T10:28:13.098 INFO:teuthology.orchestra.run.smithi037.stdout: = sunit=0 swidth=0 blks 2024-04-25T10:28:13.098 INFO:teuthology.orchestra.run.smithi037.stdout:naming =version 2 bsize=4096 ascii-ci=0, ftype=1 2024-04-25T10:28:13.098 INFO:teuthology.orchestra.run.smithi037.stdout:log =internal log bsize=4096 blocks=16384, version=2 2024-04-25T10:28:13.099 INFO:teuthology.orchestra.run.smithi037.stdout: = sectsz=512 sunit=0 blks, lazy-count=1 2024-04-25T10:28:13.099 INFO:teuthology.orchestra.run.smithi037.stdout:realtime =none extsz=4096 blocks=0, rtextents=0 2024-04-25T10:28:13.623 INFO:teuthology.orchestra.run.smithi037.stdout:Discarding blocks...Done. 2024-04-25T10:28:13.625 INFO:tasks.ceph:mount /dev/vg_nvme/lv_4 on ubuntu@smithi037.front.sepia.ceph.com -o noatime 2024-04-25T10:28:13.625 DEBUG:teuthology.orchestra.run.smithi037:> sudo mount -t xfs -o noatime /dev/vg_nvme/lv_4 /var/lib/ceph/osd/ceph-3 2024-04-25T10:28:13.774 DEBUG:teuthology.orchestra.run.smithi037:> sudo /sbin/restorecon /var/lib/ceph/osd/ceph-3 2024-04-25T10:28:13.811 DEBUG:teuthology.orchestra.run.smithi037:> sudo MALLOC_CHECK_=3 adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-osd --no-mon-config --cluster ceph --mkfs --mkkey -i 0 --monmap /home/ubuntu/cephtest/ceph.monmap 2024-04-25T10:28:13.919 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:28:13.917+0000 7fd4967bc640 -1 auth: error reading file: /var/lib/ceph/osd/ceph-0/keyring: can't open /var/lib/ceph/osd/ceph-0/keyring: (2) No such file or directory 2024-04-25T10:28:13.919 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:28:13.918+0000 7fd4967bc640 -1 created new key in keyring /var/lib/ceph/osd/ceph-0/keyring 2024-04-25T10:28:13.920 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:28:13.918+0000 7fd4967bc640 -1 bluestore(/var/lib/ceph/osd/ceph-0/block) _read_bdev_label failed to open /var/lib/ceph/osd/ceph-0/block: (2) No such file or directory 2024-04-25T10:28:13.920 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:28:13.918+0000 7fd4967bc640 -1 bluestore(/var/lib/ceph/osd/ceph-0/block) _read_bdev_label failed to open /var/lib/ceph/osd/ceph-0/block: (2) No such file or directory 2024-04-25T10:28:13.920 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:28:13.918+0000 7fd4967bc640 -1 bluestore(/var/lib/ceph/osd/ceph-0/block) _read_bdev_label failed to open /var/lib/ceph/osd/ceph-0/block: (2) No such file or directory 2024-04-25T10:28:13.920 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:28:13.918+0000 7fd4967bc640 -1 bluestore(/var/lib/ceph/osd/ceph-0) _read_fsid unparsable uuid 2024-04-25T10:28:16.779 DEBUG:teuthology.orchestra.run.smithi037:> sudo chown -R ceph:ceph /var/lib/ceph/osd/ceph-0 2024-04-25T10:28:16.809 DEBUG:teuthology.orchestra.run.smithi037:> sudo MALLOC_CHECK_=3 adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-osd --no-mon-config --cluster ceph --mkfs --mkkey -i 1 --monmap /home/ubuntu/cephtest/ceph.monmap 2024-04-25T10:28:16.912 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:28:16.909+0000 7fecb5bbc640 -1 auth: error reading file: /var/lib/ceph/osd/ceph-1/keyring: can't open /var/lib/ceph/osd/ceph-1/keyring: (2) No such file or directory 2024-04-25T10:28:16.912 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:28:16.910+0000 7fecb5bbc640 -1 created new key in keyring /var/lib/ceph/osd/ceph-1/keyring 2024-04-25T10:28:16.912 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:28:16.910+0000 7fecb5bbc640 -1 bluestore(/var/lib/ceph/osd/ceph-1/block) _read_bdev_label failed to open /var/lib/ceph/osd/ceph-1/block: (2) No such file or directory 2024-04-25T10:28:16.912 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:28:16.910+0000 7fecb5bbc640 -1 bluestore(/var/lib/ceph/osd/ceph-1/block) _read_bdev_label failed to open /var/lib/ceph/osd/ceph-1/block: (2) No such file or directory 2024-04-25T10:28:16.912 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:28:16.910+0000 7fecb5bbc640 -1 bluestore(/var/lib/ceph/osd/ceph-1/block) _read_bdev_label failed to open /var/lib/ceph/osd/ceph-1/block: (2) No such file or directory 2024-04-25T10:28:16.913 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:28:16.910+0000 7fecb5bbc640 -1 bluestore(/var/lib/ceph/osd/ceph-1) _read_fsid unparsable uuid 2024-04-25T10:28:19.775 DEBUG:teuthology.orchestra.run.smithi037:> sudo chown -R ceph:ceph /var/lib/ceph/osd/ceph-1 2024-04-25T10:28:19.804 DEBUG:teuthology.orchestra.run.smithi037:> sudo MALLOC_CHECK_=3 adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-osd --no-mon-config --cluster ceph --mkfs --mkkey -i 2 --monmap /home/ubuntu/cephtest/ceph.monmap 2024-04-25T10:28:19.907 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:28:19.905+0000 7fbf08e31640 -1 auth: error reading file: /var/lib/ceph/osd/ceph-2/keyring: can't open /var/lib/ceph/osd/ceph-2/keyring: (2) No such file or directory 2024-04-25T10:28:19.907 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:28:19.905+0000 7fbf08e31640 -1 created new key in keyring /var/lib/ceph/osd/ceph-2/keyring 2024-04-25T10:28:19.908 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:28:19.905+0000 7fbf08e31640 -1 bluestore(/var/lib/ceph/osd/ceph-2/block) _read_bdev_label failed to open /var/lib/ceph/osd/ceph-2/block: (2) No such file or directory 2024-04-25T10:28:19.908 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:28:19.905+0000 7fbf08e31640 -1 bluestore(/var/lib/ceph/osd/ceph-2/block) _read_bdev_label failed to open /var/lib/ceph/osd/ceph-2/block: (2) No such file or directory 2024-04-25T10:28:19.908 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:28:19.905+0000 7fbf08e31640 -1 bluestore(/var/lib/ceph/osd/ceph-2/block) _read_bdev_label failed to open /var/lib/ceph/osd/ceph-2/block: (2) No such file or directory 2024-04-25T10:28:19.908 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:28:19.906+0000 7fbf08e31640 -1 bluestore(/var/lib/ceph/osd/ceph-2) _read_fsid unparsable uuid 2024-04-25T10:28:22.770 DEBUG:teuthology.orchestra.run.smithi037:> sudo chown -R ceph:ceph /var/lib/ceph/osd/ceph-2 2024-04-25T10:28:22.800 DEBUG:teuthology.orchestra.run.smithi037:> sudo MALLOC_CHECK_=3 adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-osd --no-mon-config --cluster ceph --mkfs --mkkey -i 3 --monmap /home/ubuntu/cephtest/ceph.monmap 2024-04-25T10:28:22.907 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:28:22.904+0000 7f8c22484640 -1 auth: error reading file: /var/lib/ceph/osd/ceph-3/keyring: can't open /var/lib/ceph/osd/ceph-3/keyring: (2) No such file or directory 2024-04-25T10:28:22.907 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:28:22.904+0000 7f8c22484640 -1 created new key in keyring /var/lib/ceph/osd/ceph-3/keyring 2024-04-25T10:28:22.907 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:28:22.904+0000 7f8c22484640 -1 bluestore(/var/lib/ceph/osd/ceph-3/block) _read_bdev_label failed to open /var/lib/ceph/osd/ceph-3/block: (2) No such file or directory 2024-04-25T10:28:22.907 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:28:22.904+0000 7f8c22484640 -1 bluestore(/var/lib/ceph/osd/ceph-3/block) _read_bdev_label failed to open /var/lib/ceph/osd/ceph-3/block: (2) No such file or directory 2024-04-25T10:28:22.907 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:28:22.904+0000 7f8c22484640 -1 bluestore(/var/lib/ceph/osd/ceph-3/block) _read_bdev_label failed to open /var/lib/ceph/osd/ceph-3/block: (2) No such file or directory 2024-04-25T10:28:22.907 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:28:22.905+0000 7f8c22484640 -1 bluestore(/var/lib/ceph/osd/ceph-3) _read_fsid unparsable uuid 2024-04-25T10:28:25.761 DEBUG:teuthology.orchestra.run.smithi037:> sudo chown -R ceph:ceph /var/lib/ceph/osd/ceph-3 2024-04-25T10:28:25.790 DEBUG:teuthology.orchestra.run.smithi203:> sudo mkdir -p /var/lib/ceph/osd/ceph-4 2024-04-25T10:28:25.819 INFO:tasks.ceph:roles_to_devs: {'osd.4': '/dev/vg_nvme/lv_1', 'osd.5': '/dev/vg_nvme/lv_2', 'osd.6': '/dev/vg_nvme/lv_3', 'osd.7': '/dev/vg_nvme/lv_4'} 2024-04-25T10:28:25.819 INFO:tasks.ceph:role: osd.4 2024-04-25T10:28:25.819 INFO:tasks.ceph:['mkfs.xfs', '-f', '-i', 'size=2048'] on /dev/vg_nvme/lv_1 on ubuntu@smithi203.front.sepia.ceph.com 2024-04-25T10:28:25.819 DEBUG:teuthology.orchestra.run.smithi203:> yes | sudo mkfs.xfs -f -i size=2048 /dev/vg_nvme/lv_1 2024-04-25T10:28:25.893 INFO:teuthology.orchestra.run.smithi203.stdout:meta-data=/dev/vg_nvme/lv_1 isize=2048 agcount=4, agsize=5859072 blks 2024-04-25T10:28:25.894 INFO:teuthology.orchestra.run.smithi203.stdout: = sectsz=512 attr=2, projid32bit=1 2024-04-25T10:28:25.894 INFO:teuthology.orchestra.run.smithi203.stdout: = crc=1 finobt=1, sparse=1, rmapbt=0 2024-04-25T10:28:25.894 INFO:teuthology.orchestra.run.smithi203.stdout: = reflink=1 bigtime=1 inobtcount=1 nrext64=0 2024-04-25T10:28:25.894 INFO:teuthology.orchestra.run.smithi203.stdout:data = bsize=4096 blocks=23436288, imaxpct=25 2024-04-25T10:28:25.894 INFO:teuthology.orchestra.run.smithi203.stdout: = sunit=0 swidth=0 blks 2024-04-25T10:28:25.894 INFO:teuthology.orchestra.run.smithi203.stdout:naming =version 2 bsize=4096 ascii-ci=0, ftype=1 2024-04-25T10:28:25.894 INFO:teuthology.orchestra.run.smithi203.stdout:log =internal log bsize=4096 blocks=16384, version=2 2024-04-25T10:28:25.894 INFO:teuthology.orchestra.run.smithi203.stdout: = sectsz=512 sunit=0 blks, lazy-count=1 2024-04-25T10:28:25.894 INFO:teuthology.orchestra.run.smithi203.stdout:realtime =none extsz=4096 blocks=0, rtextents=0 2024-04-25T10:28:26.401 INFO:teuthology.orchestra.run.smithi203.stdout:Discarding blocks...Done. 2024-04-25T10:28:26.404 INFO:tasks.ceph:mount /dev/vg_nvme/lv_1 on ubuntu@smithi203.front.sepia.ceph.com -o noatime 2024-04-25T10:28:26.404 DEBUG:teuthology.orchestra.run.smithi203:> sudo mount -t xfs -o noatime /dev/vg_nvme/lv_1 /var/lib/ceph/osd/ceph-4 2024-04-25T10:28:26.527 DEBUG:teuthology.orchestra.run.smithi203:> sudo /sbin/restorecon /var/lib/ceph/osd/ceph-4 2024-04-25T10:28:26.587 DEBUG:teuthology.orchestra.run.smithi203:> sudo mkdir -p /var/lib/ceph/osd/ceph-5 2024-04-25T10:28:26.651 INFO:tasks.ceph:roles_to_devs: {'osd.4': '/dev/vg_nvme/lv_1', 'osd.5': '/dev/vg_nvme/lv_2', 'osd.6': '/dev/vg_nvme/lv_3', 'osd.7': '/dev/vg_nvme/lv_4'} 2024-04-25T10:28:26.651 INFO:tasks.ceph:role: osd.5 2024-04-25T10:28:26.652 INFO:tasks.ceph:['mkfs.xfs', '-f', '-i', 'size=2048'] on /dev/vg_nvme/lv_2 on ubuntu@smithi203.front.sepia.ceph.com 2024-04-25T10:28:26.652 DEBUG:teuthology.orchestra.run.smithi203:> yes | sudo mkfs.xfs -f -i size=2048 /dev/vg_nvme/lv_2 2024-04-25T10:28:26.723 INFO:teuthology.orchestra.run.smithi203.stdout:meta-data=/dev/vg_nvme/lv_2 isize=2048 agcount=4, agsize=5859072 blks 2024-04-25T10:28:26.723 INFO:teuthology.orchestra.run.smithi203.stdout: = sectsz=512 attr=2, projid32bit=1 2024-04-25T10:28:26.723 INFO:teuthology.orchestra.run.smithi203.stdout: = crc=1 finobt=1, sparse=1, rmapbt=0 2024-04-25T10:28:26.723 INFO:teuthology.orchestra.run.smithi203.stdout: = reflink=1 bigtime=1 inobtcount=1 nrext64=0 2024-04-25T10:28:26.723 INFO:teuthology.orchestra.run.smithi203.stdout:data = bsize=4096 blocks=23436288, imaxpct=25 2024-04-25T10:28:26.723 INFO:teuthology.orchestra.run.smithi203.stdout: = sunit=0 swidth=0 blks 2024-04-25T10:28:26.724 INFO:teuthology.orchestra.run.smithi203.stdout:naming =version 2 bsize=4096 ascii-ci=0, ftype=1 2024-04-25T10:28:26.724 INFO:teuthology.orchestra.run.smithi203.stdout:log =internal log bsize=4096 blocks=16384, version=2 2024-04-25T10:28:26.724 INFO:teuthology.orchestra.run.smithi203.stdout: = sectsz=512 sunit=0 blks, lazy-count=1 2024-04-25T10:28:26.724 INFO:teuthology.orchestra.run.smithi203.stdout:realtime =none extsz=4096 blocks=0, rtextents=0 2024-04-25T10:28:27.170 INFO:teuthology.orchestra.run.smithi203.stdout:Discarding blocks...Done. 2024-04-25T10:28:27.172 INFO:tasks.ceph:mount /dev/vg_nvme/lv_2 on ubuntu@smithi203.front.sepia.ceph.com -o noatime 2024-04-25T10:28:27.172 DEBUG:teuthology.orchestra.run.smithi203:> sudo mount -t xfs -o noatime /dev/vg_nvme/lv_2 /var/lib/ceph/osd/ceph-5 2024-04-25T10:28:27.256 DEBUG:teuthology.orchestra.run.smithi203:> sudo /sbin/restorecon /var/lib/ceph/osd/ceph-5 2024-04-25T10:28:27.323 DEBUG:teuthology.orchestra.run.smithi203:> sudo mkdir -p /var/lib/ceph/osd/ceph-6 2024-04-25T10:28:27.387 INFO:tasks.ceph:roles_to_devs: {'osd.4': '/dev/vg_nvme/lv_1', 'osd.5': '/dev/vg_nvme/lv_2', 'osd.6': '/dev/vg_nvme/lv_3', 'osd.7': '/dev/vg_nvme/lv_4'} 2024-04-25T10:28:27.387 INFO:tasks.ceph:role: osd.6 2024-04-25T10:28:27.387 INFO:tasks.ceph:['mkfs.xfs', '-f', '-i', 'size=2048'] on /dev/vg_nvme/lv_3 on ubuntu@smithi203.front.sepia.ceph.com 2024-04-25T10:28:27.387 DEBUG:teuthology.orchestra.run.smithi203:> yes | sudo mkfs.xfs -f -i size=2048 /dev/vg_nvme/lv_3 2024-04-25T10:28:27.457 INFO:teuthology.orchestra.run.smithi203.stdout:meta-data=/dev/vg_nvme/lv_3 isize=2048 agcount=4, agsize=5859072 blks 2024-04-25T10:28:27.457 INFO:teuthology.orchestra.run.smithi203.stdout: = sectsz=512 attr=2, projid32bit=1 2024-04-25T10:28:27.457 INFO:teuthology.orchestra.run.smithi203.stdout: = crc=1 finobt=1, sparse=1, rmapbt=0 2024-04-25T10:28:27.457 INFO:teuthology.orchestra.run.smithi203.stdout: = reflink=1 bigtime=1 inobtcount=1 nrext64=0 2024-04-25T10:28:27.458 INFO:teuthology.orchestra.run.smithi203.stdout:data = bsize=4096 blocks=23436288, imaxpct=25 2024-04-25T10:28:27.458 INFO:teuthology.orchestra.run.smithi203.stdout: = sunit=0 swidth=0 blks 2024-04-25T10:28:27.458 INFO:teuthology.orchestra.run.smithi203.stdout:naming =version 2 bsize=4096 ascii-ci=0, ftype=1 2024-04-25T10:28:27.458 INFO:teuthology.orchestra.run.smithi203.stdout:log =internal log bsize=4096 blocks=16384, version=2 2024-04-25T10:28:27.458 INFO:teuthology.orchestra.run.smithi203.stdout: = sectsz=512 sunit=0 blks, lazy-count=1 2024-04-25T10:28:27.458 INFO:teuthology.orchestra.run.smithi203.stdout:realtime =none extsz=4096 blocks=0, rtextents=0 2024-04-25T10:28:27.943 INFO:teuthology.orchestra.run.smithi203.stdout:Discarding blocks...Done. 2024-04-25T10:28:27.944 INFO:tasks.ceph:mount /dev/vg_nvme/lv_3 on ubuntu@smithi203.front.sepia.ceph.com -o noatime 2024-04-25T10:28:27.945 DEBUG:teuthology.orchestra.run.smithi203:> sudo mount -t xfs -o noatime /dev/vg_nvme/lv_3 /var/lib/ceph/osd/ceph-6 2024-04-25T10:28:28.079 DEBUG:teuthology.orchestra.run.smithi203:> sudo /sbin/restorecon /var/lib/ceph/osd/ceph-6 2024-04-25T10:28:28.108 DEBUG:teuthology.orchestra.run.smithi203:> sudo mkdir -p /var/lib/ceph/osd/ceph-7 2024-04-25T10:28:28.172 INFO:tasks.ceph:roles_to_devs: {'osd.4': '/dev/vg_nvme/lv_1', 'osd.5': '/dev/vg_nvme/lv_2', 'osd.6': '/dev/vg_nvme/lv_3', 'osd.7': '/dev/vg_nvme/lv_4'} 2024-04-25T10:28:28.172 INFO:tasks.ceph:role: osd.7 2024-04-25T10:28:28.172 INFO:tasks.ceph:['mkfs.xfs', '-f', '-i', 'size=2048'] on /dev/vg_nvme/lv_4 on ubuntu@smithi203.front.sepia.ceph.com 2024-04-25T10:28:28.173 DEBUG:teuthology.orchestra.run.smithi203:> yes | sudo mkfs.xfs -f -i size=2048 /dev/vg_nvme/lv_4 2024-04-25T10:28:28.244 INFO:teuthology.orchestra.run.smithi203.stdout:meta-data=/dev/vg_nvme/lv_4 isize=2048 agcount=4, agsize=5859072 blks 2024-04-25T10:28:28.244 INFO:teuthology.orchestra.run.smithi203.stdout: = sectsz=512 attr=2, projid32bit=1 2024-04-25T10:28:28.244 INFO:teuthology.orchestra.run.smithi203.stdout: = crc=1 finobt=1, sparse=1, rmapbt=0 2024-04-25T10:28:28.244 INFO:teuthology.orchestra.run.smithi203.stdout: = reflink=1 bigtime=1 inobtcount=1 nrext64=0 2024-04-25T10:28:28.244 INFO:teuthology.orchestra.run.smithi203.stdout:data = bsize=4096 blocks=23436288, imaxpct=25 2024-04-25T10:28:28.244 INFO:teuthology.orchestra.run.smithi203.stdout: = sunit=0 swidth=0 blks 2024-04-25T10:28:28.244 INFO:teuthology.orchestra.run.smithi203.stdout:naming =version 2 bsize=4096 ascii-ci=0, ftype=1 2024-04-25T10:28:28.244 INFO:teuthology.orchestra.run.smithi203.stdout:log =internal log bsize=4096 blocks=16384, version=2 2024-04-25T10:28:28.245 INFO:teuthology.orchestra.run.smithi203.stdout: = sectsz=512 sunit=0 blks, lazy-count=1 2024-04-25T10:28:28.245 INFO:teuthology.orchestra.run.smithi203.stdout:realtime =none extsz=4096 blocks=0, rtextents=0 2024-04-25T10:28:28.762 INFO:teuthology.orchestra.run.smithi203.stdout:Discarding blocks...Done. 2024-04-25T10:28:28.763 INFO:tasks.ceph:mount /dev/vg_nvme/lv_4 on ubuntu@smithi203.front.sepia.ceph.com -o noatime 2024-04-25T10:28:28.764 DEBUG:teuthology.orchestra.run.smithi203:> sudo mount -t xfs -o noatime /dev/vg_nvme/lv_4 /var/lib/ceph/osd/ceph-7 2024-04-25T10:28:28.914 DEBUG:teuthology.orchestra.run.smithi203:> sudo /sbin/restorecon /var/lib/ceph/osd/ceph-7 2024-04-25T10:28:28.948 DEBUG:teuthology.orchestra.run.smithi203:> sudo MALLOC_CHECK_=3 adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-osd --no-mon-config --cluster ceph --mkfs --mkkey -i 4 --monmap /home/ubuntu/cephtest/ceph.monmap 2024-04-25T10:28:29.053 INFO:teuthology.orchestra.run.smithi203.stderr:2024-04-25T10:28:29.051+0000 7f014225f640 -1 auth: error reading file: /var/lib/ceph/osd/ceph-4/keyring: can't open /var/lib/ceph/osd/ceph-4/keyring: (2) No such file or directory 2024-04-25T10:28:29.053 INFO:teuthology.orchestra.run.smithi203.stderr:2024-04-25T10:28:29.051+0000 7f014225f640 -1 created new key in keyring /var/lib/ceph/osd/ceph-4/keyring 2024-04-25T10:28:29.053 INFO:teuthology.orchestra.run.smithi203.stderr:2024-04-25T10:28:29.051+0000 7f014225f640 -1 bluestore(/var/lib/ceph/osd/ceph-4/block) _read_bdev_label failed to open /var/lib/ceph/osd/ceph-4/block: (2) No such file or directory 2024-04-25T10:28:29.053 INFO:teuthology.orchestra.run.smithi203.stderr:2024-04-25T10:28:29.051+0000 7f014225f640 -1 bluestore(/var/lib/ceph/osd/ceph-4/block) _read_bdev_label failed to open /var/lib/ceph/osd/ceph-4/block: (2) No such file or directory 2024-04-25T10:28:29.054 INFO:teuthology.orchestra.run.smithi203.stderr:2024-04-25T10:28:29.051+0000 7f014225f640 -1 bluestore(/var/lib/ceph/osd/ceph-4/block) _read_bdev_label failed to open /var/lib/ceph/osd/ceph-4/block: (2) No such file or directory 2024-04-25T10:28:29.054 INFO:teuthology.orchestra.run.smithi203.stderr:2024-04-25T10:28:29.052+0000 7f014225f640 -1 bluestore(/var/lib/ceph/osd/ceph-4) _read_fsid unparsable uuid 2024-04-25T10:28:31.907 DEBUG:teuthology.orchestra.run.smithi203:> sudo chown -R ceph:ceph /var/lib/ceph/osd/ceph-4 2024-04-25T10:28:31.934 DEBUG:teuthology.orchestra.run.smithi203:> sudo MALLOC_CHECK_=3 adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-osd --no-mon-config --cluster ceph --mkfs --mkkey -i 5 --monmap /home/ubuntu/cephtest/ceph.monmap 2024-04-25T10:28:32.032 INFO:teuthology.orchestra.run.smithi203.stderr:2024-04-25T10:28:32.029+0000 7f9448c23640 -1 auth: error reading file: /var/lib/ceph/osd/ceph-5/keyring: can't open /var/lib/ceph/osd/ceph-5/keyring: (2) No such file or directory 2024-04-25T10:28:32.032 INFO:teuthology.orchestra.run.smithi203.stderr:2024-04-25T10:28:32.029+0000 7f9448c23640 -1 created new key in keyring /var/lib/ceph/osd/ceph-5/keyring 2024-04-25T10:28:32.032 INFO:teuthology.orchestra.run.smithi203.stderr:2024-04-25T10:28:32.029+0000 7f9448c23640 -1 bluestore(/var/lib/ceph/osd/ceph-5/block) _read_bdev_label failed to open /var/lib/ceph/osd/ceph-5/block: (2) No such file or directory 2024-04-25T10:28:32.032 INFO:teuthology.orchestra.run.smithi203.stderr:2024-04-25T10:28:32.029+0000 7f9448c23640 -1 bluestore(/var/lib/ceph/osd/ceph-5/block) _read_bdev_label failed to open /var/lib/ceph/osd/ceph-5/block: (2) No such file or directory 2024-04-25T10:28:32.032 INFO:teuthology.orchestra.run.smithi203.stderr:2024-04-25T10:28:32.029+0000 7f9448c23640 -1 bluestore(/var/lib/ceph/osd/ceph-5/block) _read_bdev_label failed to open /var/lib/ceph/osd/ceph-5/block: (2) No such file or directory 2024-04-25T10:28:32.032 INFO:teuthology.orchestra.run.smithi203.stderr:2024-04-25T10:28:32.030+0000 7f9448c23640 -1 bluestore(/var/lib/ceph/osd/ceph-5) _read_fsid unparsable uuid 2024-04-25T10:28:34.886 DEBUG:teuthology.orchestra.run.smithi203:> sudo chown -R ceph:ceph /var/lib/ceph/osd/ceph-5 2024-04-25T10:28:34.914 DEBUG:teuthology.orchestra.run.smithi203:> sudo MALLOC_CHECK_=3 adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-osd --no-mon-config --cluster ceph --mkfs --mkkey -i 6 --monmap /home/ubuntu/cephtest/ceph.monmap 2024-04-25T10:28:35.015 INFO:teuthology.orchestra.run.smithi203.stderr:2024-04-25T10:28:35.012+0000 7f59ebc15640 -1 auth: error reading file: /var/lib/ceph/osd/ceph-6/keyring: can't open /var/lib/ceph/osd/ceph-6/keyring: (2) No such file or directory 2024-04-25T10:28:35.015 INFO:teuthology.orchestra.run.smithi203.stderr:2024-04-25T10:28:35.013+0000 7f59ebc15640 -1 created new key in keyring /var/lib/ceph/osd/ceph-6/keyring 2024-04-25T10:28:35.015 INFO:teuthology.orchestra.run.smithi203.stderr:2024-04-25T10:28:35.013+0000 7f59ebc15640 -1 bluestore(/var/lib/ceph/osd/ceph-6/block) _read_bdev_label failed to open /var/lib/ceph/osd/ceph-6/block: (2) No such file or directory 2024-04-25T10:28:35.015 INFO:teuthology.orchestra.run.smithi203.stderr:2024-04-25T10:28:35.013+0000 7f59ebc15640 -1 bluestore(/var/lib/ceph/osd/ceph-6/block) _read_bdev_label failed to open /var/lib/ceph/osd/ceph-6/block: (2) No such file or directory 2024-04-25T10:28:35.015 INFO:teuthology.orchestra.run.smithi203.stderr:2024-04-25T10:28:35.013+0000 7f59ebc15640 -1 bluestore(/var/lib/ceph/osd/ceph-6/block) _read_bdev_label failed to open /var/lib/ceph/osd/ceph-6/block: (2) No such file or directory 2024-04-25T10:28:35.016 INFO:teuthology.orchestra.run.smithi203.stderr:2024-04-25T10:28:35.013+0000 7f59ebc15640 -1 bluestore(/var/lib/ceph/osd/ceph-6) _read_fsid unparsable uuid 2024-04-25T10:28:37.472 DEBUG:teuthology.orchestra.run.smithi037:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-25T10:28:37.475 DEBUG:teuthology.orchestra.run.smithi203:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-25T10:28:37.879 DEBUG:teuthology.orchestra.run.smithi203:> sudo chown -R ceph:ceph /var/lib/ceph/osd/ceph-6 2024-04-25T10:28:37.902 DEBUG:teuthology.orchestra.run.smithi203:> sudo MALLOC_CHECK_=3 adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-osd --no-mon-config --cluster ceph --mkfs --mkkey -i 7 --monmap /home/ubuntu/cephtest/ceph.monmap 2024-04-25T10:28:37.997 INFO:teuthology.orchestra.run.smithi203.stderr:2024-04-25T10:28:37.994+0000 7f10c3b16640 -1 auth: error reading file: /var/lib/ceph/osd/ceph-7/keyring: can't open /var/lib/ceph/osd/ceph-7/keyring: (2) No such file or directory 2024-04-25T10:28:37.997 INFO:teuthology.orchestra.run.smithi203.stderr:2024-04-25T10:28:37.994+0000 7f10c3b16640 -1 created new key in keyring /var/lib/ceph/osd/ceph-7/keyring 2024-04-25T10:28:37.997 INFO:teuthology.orchestra.run.smithi203.stderr:2024-04-25T10:28:37.994+0000 7f10c3b16640 -1 bluestore(/var/lib/ceph/osd/ceph-7/block) _read_bdev_label failed to open /var/lib/ceph/osd/ceph-7/block: (2) No such file or directory 2024-04-25T10:28:37.997 INFO:teuthology.orchestra.run.smithi203.stderr:2024-04-25T10:28:37.995+0000 7f10c3b16640 -1 bluestore(/var/lib/ceph/osd/ceph-7/block) _read_bdev_label failed to open /var/lib/ceph/osd/ceph-7/block: (2) No such file or directory 2024-04-25T10:28:37.997 INFO:teuthology.orchestra.run.smithi203.stderr:2024-04-25T10:28:37.995+0000 7f10c3b16640 -1 bluestore(/var/lib/ceph/osd/ceph-7/block) _read_bdev_label failed to open /var/lib/ceph/osd/ceph-7/block: (2) No such file or directory 2024-04-25T10:28:37.998 INFO:teuthology.orchestra.run.smithi203.stderr:2024-04-25T10:28:37.995+0000 7f10c3b16640 -1 bluestore(/var/lib/ceph/osd/ceph-7) _read_fsid unparsable uuid 2024-04-25T10:28:40.857 DEBUG:teuthology.orchestra.run.smithi203:> sudo chown -R ceph:ceph /var/lib/ceph/osd/ceph-7 2024-04-25T10:28:40.885 INFO:tasks.ceph:Reading keys from all nodes... 2024-04-25T10:28:40.885 DEBUG:teuthology.orchestra.run.smithi037:> set -ex 2024-04-25T10:28:40.886 DEBUG:teuthology.orchestra.run.smithi037:> sudo dd if=/var/lib/ceph/mgr/ceph-y/keyring of=/dev/stdout 2024-04-25T10:28:40.915 DEBUG:teuthology.orchestra.run.smithi037:> set -ex 2024-04-25T10:28:40.915 DEBUG:teuthology.orchestra.run.smithi037:> sudo dd if=/var/lib/ceph/mds/ceph-a/keyring of=/dev/stdout 2024-04-25T10:28:40.984 DEBUG:teuthology.orchestra.run.smithi037:> set -ex 2024-04-25T10:28:40.984 DEBUG:teuthology.orchestra.run.smithi037:> sudo dd if=/var/lib/ceph/mds/ceph-c/keyring of=/dev/stdout 2024-04-25T10:28:41.051 DEBUG:teuthology.orchestra.run.smithi037:> set -ex 2024-04-25T10:28:41.051 DEBUG:teuthology.orchestra.run.smithi037:> sudo dd if=/var/lib/ceph/osd/ceph-0/keyring of=/dev/stdout 2024-04-25T10:28:41.118 DEBUG:teuthology.orchestra.run.smithi037:> set -ex 2024-04-25T10:28:41.119 DEBUG:teuthology.orchestra.run.smithi037:> sudo dd if=/var/lib/ceph/osd/ceph-1/keyring of=/dev/stdout 2024-04-25T10:28:41.185 DEBUG:teuthology.orchestra.run.smithi037:> set -ex 2024-04-25T10:28:41.185 DEBUG:teuthology.orchestra.run.smithi037:> sudo dd if=/var/lib/ceph/osd/ceph-2/keyring of=/dev/stdout 2024-04-25T10:28:41.251 DEBUG:teuthology.orchestra.run.smithi037:> set -ex 2024-04-25T10:28:41.251 DEBUG:teuthology.orchestra.run.smithi037:> sudo dd if=/var/lib/ceph/osd/ceph-3/keyring of=/dev/stdout 2024-04-25T10:28:41.316 DEBUG:teuthology.orchestra.run.smithi203:> set -ex 2024-04-25T10:28:41.317 DEBUG:teuthology.orchestra.run.smithi203:> sudo dd if=/var/lib/ceph/mgr/ceph-x/keyring of=/dev/stdout 2024-04-25T10:28:41.345 DEBUG:teuthology.orchestra.run.smithi203:> set -ex 2024-04-25T10:28:41.345 DEBUG:teuthology.orchestra.run.smithi203:> sudo dd if=/var/lib/ceph/mds/ceph-b/keyring of=/dev/stdout 2024-04-25T10:28:41.412 DEBUG:teuthology.orchestra.run.smithi203:> set -ex 2024-04-25T10:28:41.412 DEBUG:teuthology.orchestra.run.smithi203:> sudo dd if=/var/lib/ceph/mds/ceph-d/keyring of=/dev/stdout 2024-04-25T10:28:41.479 DEBUG:teuthology.orchestra.run.smithi203:> set -ex 2024-04-25T10:28:41.479 DEBUG:teuthology.orchestra.run.smithi203:> sudo dd if=/var/lib/ceph/osd/ceph-4/keyring of=/dev/stdout 2024-04-25T10:28:41.546 DEBUG:teuthology.orchestra.run.smithi203:> set -ex 2024-04-25T10:28:41.546 DEBUG:teuthology.orchestra.run.smithi203:> sudo dd if=/var/lib/ceph/osd/ceph-5/keyring of=/dev/stdout 2024-04-25T10:28:41.612 DEBUG:teuthology.orchestra.run.smithi203:> set -ex 2024-04-25T10:28:41.612 DEBUG:teuthology.orchestra.run.smithi203:> sudo dd if=/var/lib/ceph/osd/ceph-6/keyring of=/dev/stdout 2024-04-25T10:28:41.677 DEBUG:teuthology.orchestra.run.smithi203:> set -ex 2024-04-25T10:28:41.677 DEBUG:teuthology.orchestra.run.smithi203:> sudo dd if=/var/lib/ceph/osd/ceph-7/keyring of=/dev/stdout 2024-04-25T10:28:41.742 DEBUG:teuthology.orchestra.run.smithi037:> set -ex 2024-04-25T10:28:41.742 DEBUG:teuthology.orchestra.run.smithi037:> dd if=/etc/ceph/ceph.client.0.keyring of=/dev/stdout 2024-04-25T10:28:41.760 DEBUG:teuthology.orchestra.run.smithi037:> set -ex 2024-04-25T10:28:41.760 DEBUG:teuthology.orchestra.run.smithi037:> dd if=/etc/ceph/ceph.client.2.keyring of=/dev/stdout 2024-04-25T10:28:41.818 DEBUG:teuthology.orchestra.run.smithi203:> set -ex 2024-04-25T10:28:41.818 DEBUG:teuthology.orchestra.run.smithi203:> dd if=/etc/ceph/ceph.client.1.keyring of=/dev/stdout 2024-04-25T10:28:41.833 DEBUG:teuthology.orchestra.run.smithi203:> set -ex 2024-04-25T10:28:41.833 DEBUG:teuthology.orchestra.run.smithi203:> dd if=/etc/ceph/ceph.client.3.keyring of=/dev/stdout 2024-04-25T10:28:41.888 INFO:tasks.ceph:Adding keys to all mons... 2024-04-25T10:28:41.888 DEBUG:teuthology.orchestra.run.smithi037:> sudo tee -a /etc/ceph/ceph.keyring 2024-04-25T10:28:41.891 DEBUG:teuthology.orchestra.run.smithi203:> sudo tee -a /etc/ceph/ceph.keyring 2024-04-25T10:28:41.931 INFO:teuthology.orchestra.run.smithi037.stdout:[mgr.y] 2024-04-25T10:28:41.931 INFO:teuthology.orchestra.run.smithi037.stdout: key = AQA5MCpmaXHDNhAA4ipqVxc6vQM915k93tOJIQ== 2024-04-25T10:28:41.931 INFO:teuthology.orchestra.run.smithi037.stdout:[mds.a] 2024-04-25T10:28:41.931 INFO:teuthology.orchestra.run.smithi037.stdout: key = AQA6MCpmOqPOAxAAA6+9BBhumKh0ZpFfmoGY9g== 2024-04-25T10:28:41.932 INFO:teuthology.orchestra.run.smithi037.stdout:[mds.c] 2024-04-25T10:28:41.932 INFO:teuthology.orchestra.run.smithi037.stdout: key = AQA6MCpmxhNNCxAA96FDgnsZN+3HKcD7mcXOKQ== 2024-04-25T10:28:41.932 INFO:teuthology.orchestra.run.smithi037.stdout:[osd.0] 2024-04-25T10:28:41.932 INFO:teuthology.orchestra.run.smithi037.stdout: key = AQA9MCpmY2W2NhAA9bGxmGeGHUKB9mkN+aVQeQ== 2024-04-25T10:28:41.932 INFO:teuthology.orchestra.run.smithi037.stdout:[osd.1] 2024-04-25T10:28:41.932 INFO:teuthology.orchestra.run.smithi037.stdout: key = AQBAMCpmVaZMNhAA0f+Rbv1taIO6L4ghaHc0mw== 2024-04-25T10:28:41.932 INFO:teuthology.orchestra.run.smithi037.stdout:[osd.2] 2024-04-25T10:28:41.932 INFO:teuthology.orchestra.run.smithi037.stdout: key = AQBDMCpmjj4FNhAAwOpaIswpIllQpGT1shypQw== 2024-04-25T10:28:41.932 INFO:teuthology.orchestra.run.smithi037.stdout:[osd.3] 2024-04-25T10:28:41.932 INFO:teuthology.orchestra.run.smithi037.stdout: key = AQBGMCpmIFb7NRAA3/p5NiVot6OJ/4zCIgfgBQ== 2024-04-25T10:28:41.932 INFO:teuthology.orchestra.run.smithi037.stdout:[mgr.x] 2024-04-25T10:28:41.932 INFO:teuthology.orchestra.run.smithi037.stdout: key = AQA6MCpmyf8OABAAiqPbP7X6j/lHlk9pzU6ABQ== 2024-04-25T10:28:41.932 INFO:teuthology.orchestra.run.smithi037.stdout:[mds.b] 2024-04-25T10:28:41.932 INFO:teuthology.orchestra.run.smithi037.stdout: key = AQA6MCpm8nFTEBAAHcOblByoU8A86+xc++RP4g== 2024-04-25T10:28:41.933 INFO:teuthology.orchestra.run.smithi037.stdout:[mds.d] 2024-04-25T10:28:41.933 INFO:teuthology.orchestra.run.smithi037.stdout: key = AQA6MCpmkcehFxAAwy8W6aXfuVc30DJQg3ubTQ== 2024-04-25T10:28:41.933 INFO:teuthology.orchestra.run.smithi037.stdout:[osd.4] 2024-04-25T10:28:41.933 INFO:teuthology.orchestra.run.smithi037.stdout: key = AQBNMCpm93QfAxAAEWVT2WCL9FZGei9lOGVklQ== 2024-04-25T10:28:41.933 INFO:teuthology.orchestra.run.smithi037.stdout:[osd.5] 2024-04-25T10:28:41.933 INFO:teuthology.orchestra.run.smithi037.stdout: key = AQBQMCpm1ivWARAAQFZjKw9Fma0rQJB7W/9mdQ== 2024-04-25T10:28:41.933 INFO:teuthology.orchestra.run.smithi037.stdout:[osd.6] 2024-04-25T10:28:41.933 INFO:teuthology.orchestra.run.smithi037.stdout: key = AQBTMCpm5h3YABAAvQ+ACgMvLhelhESFoNkdbQ== 2024-04-25T10:28:41.933 INFO:teuthology.orchestra.run.smithi037.stdout:[osd.7] 2024-04-25T10:28:41.933 INFO:teuthology.orchestra.run.smithi037.stdout: key = AQBVMCpmTjVfOxAAj6Zg4/eeQqNxiVlZS0by4w== 2024-04-25T10:28:41.933 INFO:teuthology.orchestra.run.smithi037.stdout:[client.0] 2024-04-25T10:28:41.933 INFO:teuthology.orchestra.run.smithi037.stdout: key = AQA6MCpmQ308HBAAvMknoENSGUD2vblm1SMnBw== 2024-04-25T10:28:41.933 INFO:teuthology.orchestra.run.smithi037.stdout:[client.2] 2024-04-25T10:28:41.933 INFO:teuthology.orchestra.run.smithi037.stdout: key = AQA6MCpmJuXpHxAAvIdzKkZOqcGEhbcmTQ15FQ== 2024-04-25T10:28:41.934 INFO:teuthology.orchestra.run.smithi037.stdout:[client.1] 2024-04-25T10:28:41.934 INFO:teuthology.orchestra.run.smithi037.stdout: key = AQA6MCpmpPyRIxAAic9ZcD+WF6YRCzQu9u0jCg== 2024-04-25T10:28:41.934 INFO:teuthology.orchestra.run.smithi037.stdout:[client.3] 2024-04-25T10:28:41.934 INFO:teuthology.orchestra.run.smithi037.stdout: key = AQA6MCpmlk0vJxAAi5on23wOapMW7UdNXB71Lw== 2024-04-25T10:28:41.950 INFO:teuthology.orchestra.run.smithi203.stdout:[mgr.y] 2024-04-25T10:28:41.951 INFO:teuthology.orchestra.run.smithi203.stdout: key = AQA5MCpmaXHDNhAA4ipqVxc6vQM915k93tOJIQ== 2024-04-25T10:28:41.951 INFO:teuthology.orchestra.run.smithi203.stdout:[mds.a] 2024-04-25T10:28:41.951 INFO:teuthology.orchestra.run.smithi203.stdout: key = AQA6MCpmOqPOAxAAA6+9BBhumKh0ZpFfmoGY9g== 2024-04-25T10:28:41.951 INFO:teuthology.orchestra.run.smithi203.stdout:[mds.c] 2024-04-25T10:28:41.951 INFO:teuthology.orchestra.run.smithi203.stdout: key = AQA6MCpmxhNNCxAA96FDgnsZN+3HKcD7mcXOKQ== 2024-04-25T10:28:41.951 INFO:teuthology.orchestra.run.smithi203.stdout:[osd.0] 2024-04-25T10:28:41.951 INFO:teuthology.orchestra.run.smithi203.stdout: key = AQA9MCpmY2W2NhAA9bGxmGeGHUKB9mkN+aVQeQ== 2024-04-25T10:28:41.951 INFO:teuthology.orchestra.run.smithi203.stdout:[osd.1] 2024-04-25T10:28:41.951 INFO:teuthology.orchestra.run.smithi203.stdout: key = AQBAMCpmVaZMNhAA0f+Rbv1taIO6L4ghaHc0mw== 2024-04-25T10:28:41.951 INFO:teuthology.orchestra.run.smithi203.stdout:[osd.2] 2024-04-25T10:28:41.951 INFO:teuthology.orchestra.run.smithi203.stdout: key = AQBDMCpmjj4FNhAAwOpaIswpIllQpGT1shypQw== 2024-04-25T10:28:41.951 INFO:teuthology.orchestra.run.smithi203.stdout:[osd.3] 2024-04-25T10:28:41.951 INFO:teuthology.orchestra.run.smithi203.stdout: key = AQBGMCpmIFb7NRAA3/p5NiVot6OJ/4zCIgfgBQ== 2024-04-25T10:28:41.952 INFO:teuthology.orchestra.run.smithi203.stdout:[mgr.x] 2024-04-25T10:28:41.952 INFO:teuthology.orchestra.run.smithi203.stdout: key = AQA6MCpmyf8OABAAiqPbP7X6j/lHlk9pzU6ABQ== 2024-04-25T10:28:41.952 INFO:teuthology.orchestra.run.smithi203.stdout:[mds.b] 2024-04-25T10:28:41.952 INFO:teuthology.orchestra.run.smithi203.stdout: key = AQA6MCpm8nFTEBAAHcOblByoU8A86+xc++RP4g== 2024-04-25T10:28:41.952 INFO:teuthology.orchestra.run.smithi203.stdout:[mds.d] 2024-04-25T10:28:41.952 INFO:teuthology.orchestra.run.smithi203.stdout: key = AQA6MCpmkcehFxAAwy8W6aXfuVc30DJQg3ubTQ== 2024-04-25T10:28:41.952 INFO:teuthology.orchestra.run.smithi203.stdout:[osd.4] 2024-04-25T10:28:41.952 INFO:teuthology.orchestra.run.smithi203.stdout: key = AQBNMCpm93QfAxAAEWVT2WCL9FZGei9lOGVklQ== 2024-04-25T10:28:41.952 INFO:teuthology.orchestra.run.smithi203.stdout:[osd.5] 2024-04-25T10:28:41.952 INFO:teuthology.orchestra.run.smithi203.stdout: key = AQBQMCpm1ivWARAAQFZjKw9Fma0rQJB7W/9mdQ== 2024-04-25T10:28:41.952 INFO:teuthology.orchestra.run.smithi203.stdout:[osd.6] 2024-04-25T10:28:41.952 INFO:teuthology.orchestra.run.smithi203.stdout: key = AQBTMCpm5h3YABAAvQ+ACgMvLhelhESFoNkdbQ== 2024-04-25T10:28:41.952 INFO:teuthology.orchestra.run.smithi203.stdout:[osd.7] 2024-04-25T10:28:41.952 INFO:teuthology.orchestra.run.smithi203.stdout: key = AQBVMCpmTjVfOxAAj6Zg4/eeQqNxiVlZS0by4w== 2024-04-25T10:28:41.952 INFO:teuthology.orchestra.run.smithi203.stdout:[client.0] 2024-04-25T10:28:41.953 INFO:teuthology.orchestra.run.smithi203.stdout: key = AQA6MCpmQ308HBAAvMknoENSGUD2vblm1SMnBw== 2024-04-25T10:28:41.953 INFO:teuthology.orchestra.run.smithi203.stdout:[client.2] 2024-04-25T10:28:41.953 INFO:teuthology.orchestra.run.smithi203.stdout: key = AQA6MCpmJuXpHxAAvIdzKkZOqcGEhbcmTQ15FQ== 2024-04-25T10:28:41.953 INFO:teuthology.orchestra.run.smithi203.stdout:[client.1] 2024-04-25T10:28:41.953 INFO:teuthology.orchestra.run.smithi203.stdout: key = AQA6MCpmpPyRIxAAic9ZcD+WF6YRCzQu9u0jCg== 2024-04-25T10:28:41.953 INFO:teuthology.orchestra.run.smithi203.stdout:[client.3] 2024-04-25T10:28:41.953 INFO:teuthology.orchestra.run.smithi203.stdout: key = AQA6MCpmlk0vJxAAi5on23wOapMW7UdNXB71Lw== 2024-04-25T10:28:41.954 DEBUG:teuthology.orchestra.run.smithi037:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=mgr.y --cap mon 'allow profile mgr' --cap osd 'allow *' --cap mds 'allow *' 2024-04-25T10:28:41.978 DEBUG:teuthology.orchestra.run.smithi203:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=mgr.y --cap mon 'allow profile mgr' --cap osd 'allow *' --cap mds 'allow *' 2024-04-25T10:28:42.055 DEBUG:teuthology.orchestra.run.smithi037:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=mds.a --cap mon 'allow *' --cap mgr 'allow *' --cap osd 'allow *' --cap mds allow 2024-04-25T10:28:42.057 DEBUG:teuthology.orchestra.run.smithi203:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=mds.a --cap mon 'allow *' --cap mgr 'allow *' --cap osd 'allow *' --cap mds allow 2024-04-25T10:28:42.122 DEBUG:teuthology.orchestra.run.smithi037:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=mds.c --cap mon 'allow *' --cap mgr 'allow *' --cap osd 'allow *' --cap mds allow 2024-04-25T10:28:42.124 DEBUG:teuthology.orchestra.run.smithi203:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=mds.c --cap mon 'allow *' --cap mgr 'allow *' --cap osd 'allow *' --cap mds allow 2024-04-25T10:28:42.186 DEBUG:teuthology.orchestra.run.smithi037:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=osd.0 --cap mon 'allow profile osd' --cap mgr 'allow profile osd' --cap osd 'allow *' 2024-04-25T10:28:42.191 DEBUG:teuthology.orchestra.run.smithi203:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=osd.0 --cap mon 'allow profile osd' --cap mgr 'allow profile osd' --cap osd 'allow *' 2024-04-25T10:28:42.250 DEBUG:teuthology.orchestra.run.smithi037:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=osd.1 --cap mon 'allow profile osd' --cap mgr 'allow profile osd' --cap osd 'allow *' 2024-04-25T10:28:42.253 DEBUG:teuthology.orchestra.run.smithi203:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=osd.1 --cap mon 'allow profile osd' --cap mgr 'allow profile osd' --cap osd 'allow *' 2024-04-25T10:28:42.311 DEBUG:teuthology.orchestra.run.smithi037:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=osd.2 --cap mon 'allow profile osd' --cap mgr 'allow profile osd' --cap osd 'allow *' 2024-04-25T10:28:42.314 DEBUG:teuthology.orchestra.run.smithi203:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=osd.2 --cap mon 'allow profile osd' --cap mgr 'allow profile osd' --cap osd 'allow *' 2024-04-25T10:28:42.371 DEBUG:teuthology.orchestra.run.smithi037:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=osd.3 --cap mon 'allow profile osd' --cap mgr 'allow profile osd' --cap osd 'allow *' 2024-04-25T10:28:42.374 DEBUG:teuthology.orchestra.run.smithi203:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=osd.3 --cap mon 'allow profile osd' --cap mgr 'allow profile osd' --cap osd 'allow *' 2024-04-25T10:28:42.432 DEBUG:teuthology.orchestra.run.smithi037:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=mgr.x --cap mon 'allow profile mgr' --cap osd 'allow *' --cap mds 'allow *' 2024-04-25T10:28:42.435 DEBUG:teuthology.orchestra.run.smithi203:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=mgr.x --cap mon 'allow profile mgr' --cap osd 'allow *' --cap mds 'allow *' 2024-04-25T10:28:42.493 DEBUG:teuthology.orchestra.run.smithi037:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=mds.b --cap mon 'allow *' --cap mgr 'allow *' --cap osd 'allow *' --cap mds allow 2024-04-25T10:28:42.496 DEBUG:teuthology.orchestra.run.smithi203:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=mds.b --cap mon 'allow *' --cap mgr 'allow *' --cap osd 'allow *' --cap mds allow 2024-04-25T10:28:42.558 DEBUG:teuthology.orchestra.run.smithi037:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=mds.d --cap mon 'allow *' --cap mgr 'allow *' --cap osd 'allow *' --cap mds allow 2024-04-25T10:28:42.560 DEBUG:teuthology.orchestra.run.smithi203:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=mds.d --cap mon 'allow *' --cap mgr 'allow *' --cap osd 'allow *' --cap mds allow 2024-04-25T10:28:42.622 DEBUG:teuthology.orchestra.run.smithi037:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=osd.4 --cap mon 'allow profile osd' --cap mgr 'allow profile osd' --cap osd 'allow *' 2024-04-25T10:28:42.624 DEBUG:teuthology.orchestra.run.smithi203:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=osd.4 --cap mon 'allow profile osd' --cap mgr 'allow profile osd' --cap osd 'allow *' 2024-04-25T10:28:42.683 DEBUG:teuthology.orchestra.run.smithi037:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=osd.5 --cap mon 'allow profile osd' --cap mgr 'allow profile osd' --cap osd 'allow *' 2024-04-25T10:28:42.685 DEBUG:teuthology.orchestra.run.smithi203:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=osd.5 --cap mon 'allow profile osd' --cap mgr 'allow profile osd' --cap osd 'allow *' 2024-04-25T10:28:42.743 DEBUG:teuthology.orchestra.run.smithi037:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=osd.6 --cap mon 'allow profile osd' --cap mgr 'allow profile osd' --cap osd 'allow *' 2024-04-25T10:28:42.746 DEBUG:teuthology.orchestra.run.smithi203:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=osd.6 --cap mon 'allow profile osd' --cap mgr 'allow profile osd' --cap osd 'allow *' 2024-04-25T10:28:42.805 DEBUG:teuthology.orchestra.run.smithi037:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=osd.7 --cap mon 'allow profile osd' --cap mgr 'allow profile osd' --cap osd 'allow *' 2024-04-25T10:28:42.808 DEBUG:teuthology.orchestra.run.smithi203:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=osd.7 --cap mon 'allow profile osd' --cap mgr 'allow profile osd' --cap osd 'allow *' 2024-04-25T10:28:42.866 DEBUG:teuthology.orchestra.run.smithi037:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=client.0 --cap mon 'allow rw' --cap mgr 'allow r' --cap osd 'allow rwx' --cap mds allow 2024-04-25T10:28:42.868 DEBUG:teuthology.orchestra.run.smithi203:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=client.0 --cap mon 'allow rw' --cap mgr 'allow r' --cap osd 'allow rwx' --cap mds allow 2024-04-25T10:28:42.930 DEBUG:teuthology.orchestra.run.smithi037:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=client.2 --cap mon 'allow rw' --cap mgr 'allow r' --cap osd 'allow rwx' --cap mds allow 2024-04-25T10:28:42.932 DEBUG:teuthology.orchestra.run.smithi203:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=client.2 --cap mon 'allow rw' --cap mgr 'allow r' --cap osd 'allow rwx' --cap mds allow 2024-04-25T10:28:42.994 DEBUG:teuthology.orchestra.run.smithi037:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=client.1 --cap mon 'allow rw' --cap mgr 'allow r' --cap osd 'allow rwx' --cap mds allow 2024-04-25T10:28:42.996 DEBUG:teuthology.orchestra.run.smithi203:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=client.1 --cap mon 'allow rw' --cap mgr 'allow r' --cap osd 'allow rwx' --cap mds allow 2024-04-25T10:28:43.058 DEBUG:teuthology.orchestra.run.smithi037:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=client.3 --cap mon 'allow rw' --cap mgr 'allow r' --cap osd 'allow rwx' --cap mds allow 2024-04-25T10:28:43.060 DEBUG:teuthology.orchestra.run.smithi203:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=client.3 --cap mon 'allow rw' --cap mgr 'allow r' --cap osd 'allow rwx' --cap mds allow 2024-04-25T10:28:43.122 INFO:tasks.ceph:Running mkfs on mon nodes... 2024-04-25T10:28:43.122 DEBUG:teuthology.orchestra.run.smithi037:> sudo mkdir -p /var/lib/ceph/mon/ceph-a 2024-04-25T10:28:43.146 DEBUG:teuthology.orchestra.run.smithi037:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-mon --cluster ceph --mkfs -i a --monmap /home/ubuntu/cephtest/ceph.monmap --keyring /etc/ceph/ceph.keyring 2024-04-25T10:28:43.252 DEBUG:teuthology.orchestra.run.smithi037:> sudo chown -R ceph:ceph /var/lib/ceph/mon/ceph-a 2024-04-25T10:28:43.276 DEBUG:teuthology.orchestra.run.smithi203:> sudo mkdir -p /var/lib/ceph/mon/ceph-b 2024-04-25T10:28:43.300 DEBUG:teuthology.orchestra.run.smithi203:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-mon --cluster ceph --mkfs -i b --monmap /home/ubuntu/cephtest/ceph.monmap --keyring /etc/ceph/ceph.keyring 2024-04-25T10:28:43.405 DEBUG:teuthology.orchestra.run.smithi203:> sudo chown -R ceph:ceph /var/lib/ceph/mon/ceph-b 2024-04-25T10:28:43.428 DEBUG:teuthology.orchestra.run.smithi203:> sudo mkdir -p /var/lib/ceph/mon/ceph-c 2024-04-25T10:28:43.493 DEBUG:teuthology.orchestra.run.smithi203:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-mon --cluster ceph --mkfs -i c --monmap /home/ubuntu/cephtest/ceph.monmap --keyring /etc/ceph/ceph.keyring 2024-04-25T10:28:43.596 DEBUG:teuthology.orchestra.run.smithi203:> sudo chown -R ceph:ceph /var/lib/ceph/mon/ceph-c 2024-04-25T10:28:43.620 DEBUG:teuthology.orchestra.run.smithi037:> rm -- /home/ubuntu/cephtest/ceph.monmap 2024-04-25T10:28:43.622 DEBUG:teuthology.orchestra.run.smithi203:> rm -- /home/ubuntu/cephtest/ceph.monmap 2024-04-25T10:28:43.675 INFO:tasks.ceph:Starting mon daemons in cluster ceph... 2024-04-25T10:28:43.675 INFO:tasks.ceph.mon.a:Restarting daemon 2024-04-25T10:28:43.675 DEBUG:teuthology.orchestra.run.smithi037:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mon -f --cluster ceph -i a 2024-04-25T10:28:43.678 INFO:tasks.ceph.mon.a:Started 2024-04-25T10:28:43.678 INFO:tasks.ceph.mon.b:Restarting daemon 2024-04-25T10:28:43.678 DEBUG:teuthology.orchestra.run.smithi203:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mon -f --cluster ceph -i b 2024-04-25T10:28:43.717 INFO:tasks.ceph.mon.b:Started 2024-04-25T10:28:43.718 INFO:tasks.ceph.mon.c:Restarting daemon 2024-04-25T10:28:43.718 DEBUG:teuthology.orchestra.run.smithi203:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mon -f --cluster ceph -i c 2024-04-25T10:28:43.720 INFO:tasks.ceph.mon.c:Started 2024-04-25T10:28:43.720 INFO:tasks.ceph:Starting mgr daemons in cluster ceph... 2024-04-25T10:28:43.720 INFO:tasks.ceph.mgr.y:Restarting daemon 2024-04-25T10:28:43.720 DEBUG:teuthology.orchestra.run.smithi037:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mgr -f --cluster ceph -i y 2024-04-25T10:28:43.723 INFO:tasks.ceph.mgr.y:Started 2024-04-25T10:28:43.723 INFO:tasks.ceph.mgr.x:Restarting daemon 2024-04-25T10:28:43.723 DEBUG:teuthology.orchestra.run.smithi203:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mgr -f --cluster ceph -i x 2024-04-25T10:28:43.725 INFO:tasks.ceph.mgr.x:Started 2024-04-25T10:28:43.725 INFO:tasks.ceph:Setting crush tunables to default 2024-04-25T10:28:43.726 DEBUG:teuthology.orchestra.run.smithi037:> sudo ceph --cluster ceph osd crush tunables default 2024-04-25T10:28:43.840 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:28:43.838+0000 7fdd3e338640 1 Processor -- start 2024-04-25T10:28:43.840 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:28:43.838+0000 7fdd3e338640 1 -- start start 2024-04-25T10:28:43.842 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:28:43.839+0000 7fdd3e338640 1 --2- >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7fdd380ce360 0x7fdd38161b80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:28:43.842 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:28:43.839+0000 7fdd3e338640 1 --2- >> v2:172.21.15.203:3300/0 conn(0x7fdd380700f0 0x7fdd38070500 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:28:43.842 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:28:43.839+0000 7fdd3e338640 1 -- --> v1:172.21.15.37:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fdd38070a40 con 0x7fdd380715d0 2024-04-25T10:28:43.843 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:28:43.839+0000 7fdd3e338640 1 -- --> v2:172.21.15.203:3300/0 -- mon_getmap magic: 0 -- 0x7fdd380d37d0 con 0x7fdd380700f0 2024-04-25T10:28:43.843 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:28:43.839+0000 7fdd3e338640 1 -- --> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] -- mon_getmap magic: 0 -- 0x7fdd381623d0 con 0x7fdd380ce360 2024-04-25T10:28:43.843 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:28:43.839+0000 7fdd3c8d5640 1 --1- >> v1:172.21.15.37:6789/0 conn(0x7fdd380715d0 0x7fdd380719e0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.37:6789/0 says I am v1:172.21.15.37:40422/0 (socket says 172.21.15.37:40422) 2024-04-25T10:28:43.843 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:28:43.839+0000 7fdd3c8d5640 1 -- 172.21.15.37:0/3616618552 learned_addr learned my addr 172.21.15.37:0/3616618552 (peer_addr_for_me v1:172.21.15.37:0/0) 2024-04-25T10:28:43.843 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:28:43.840+0000 7fdd37fff640 1 --2- 172.21.15.37:0/3616618552 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7fdd380ce360 0x7fdd38161b80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:28:43.843 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:28:43.840+0000 7fdd377fe640 1 --2- 172.21.15.37:0/3616618552 >> v2:172.21.15.203:3300/0 conn(0x7fdd380700f0 0x7fdd38070500 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:28:43.843 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:28:43.840+0000 7fdd37fff640 1 -- 172.21.15.37:0/3616618552 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7fdd380ce360 msgr2=0x7fdd38161b80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_bulk peer close file descriptor 13 2024-04-25T10:28:43.843 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:28:43.840+0000 7fdd37fff640 1 -- 172.21.15.37:0/3616618552 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7fdd380ce360 msgr2=0x7fdd38161b80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_until read failed 2024-04-25T10:28:43.843 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:28:43.840+0000 7fdd37fff640 1 --2- 172.21.15.37:0/3616618552 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7fdd380ce360 0x7fdd38161b80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_read_frame_preamble_main read frame preamble failed r=-1 2024-04-25T10:28:43.843 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:28:43.840+0000 7fdd377fe640 1 -- 172.21.15.37:0/3616618552 >> v2:172.21.15.203:3300/0 conn(0x7fdd380700f0 msgr2=0x7fdd38070500 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_bulk peer close file descriptor 14 2024-04-25T10:28:43.843 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:28:43.840+0000 7fdd377fe640 1 -- 172.21.15.37:0/3616618552 >> v2:172.21.15.203:3300/0 conn(0x7fdd380700f0 msgr2=0x7fdd38070500 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_until read failed 2024-04-25T10:28:43.843 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:28:43.840+0000 7fdd377fe640 1 --2- 172.21.15.37:0/3616618552 >> v2:172.21.15.203:3300/0 conn(0x7fdd380700f0 0x7fdd38070500 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_read_frame_preamble_main read frame preamble failed r=-1 2024-04-25T10:28:43.843 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:28:43.840+0000 7fdd37fff640 1 --2- 172.21.15.37:0/3616618552 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7fdd380ce360 0x7fdd38161b80 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-04-25T10:28:43.844 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:28:43.840+0000 7fdd377fe640 1 --2- 172.21.15.37:0/3616618552 >> v2:172.21.15.203:3300/0 conn(0x7fdd380700f0 0x7fdd38070500 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-04-25T10:28:44.008 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:28:44.000+0000 7fdd36ffd640 1 -- 172.21.15.37:0/3616618552 <== mon.0 v1:172.21.15.37:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 807779963 0 0) 0x7fdd30001490 con 0x7fdd380715d0 2024-04-25T10:28:44.008 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:28:44.000+0000 7fdd36ffd640 1 -- 172.21.15.37:0/3616618552 --> v1:172.21.15.37:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7fdd20003520 con 0x7fdd380715d0 2024-04-25T10:28:44.008 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:28:44.001+0000 7fdd36ffd640 1 -- 172.21.15.37:0/3616618552 <== mon.0 v1:172.21.15.37:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 3256682369 0 0) 0x7fdd20003520 con 0x7fdd380715d0 2024-04-25T10:28:44.008 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:28:44.001+0000 7fdd36ffd640 1 -- 172.21.15.37:0/3616618552 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7fdd380ce360 msgr2=0x7fdd38161b80 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-04-25T10:28:44.009 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:28:44.001+0000 7fdd36ffd640 1 --2- 172.21.15.37:0/3616618552 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7fdd380ce360 0x7fdd38161b80 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:28:44.009 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:28:44.001+0000 7fdd36ffd640 1 -- 172.21.15.37:0/3616618552 >> v2:172.21.15.203:3300/0 conn(0x7fdd380700f0 msgr2=0x7fdd38070500 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-04-25T10:28:44.009 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:28:44.001+0000 7fdd36ffd640 1 --2- 172.21.15.37:0/3616618552 >> v2:172.21.15.203:3300/0 conn(0x7fdd380700f0 0x7fdd38070500 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:28:44.009 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:28:44.001+0000 7fdd36ffd640 1 -- 172.21.15.37:0/3616618552 --> v1:172.21.15.37:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fdd381626b0 con 0x7fdd380715d0 2024-04-25T10:28:44.009 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:28:44.001+0000 7fdd36ffd640 1 -- 172.21.15.37:0/3616618552 <== mon.0 v1:172.21.15.37:6789/0 3 ==== mon_map magic: 0 ==== 413+0+0 (unknown 1577794600 0 0) 0x7fdd30001bc0 con 0x7fdd380715d0 2024-04-25T10:28:44.009 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:28:44.001+0000 7fdd36ffd640 1 -- 172.21.15.37:0/3616618552 >> v1:172.21.15.37:6789/0 conn(0x7fdd380715d0 legacy=0x7fdd380719e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:28:44.009 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:28:44.001+0000 7fdd36ffd640 1 --2- 172.21.15.37:0/3616618552 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fdd200039b0 0x7fdd20003e20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:28:44.010 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:28:44.001+0000 7fdd36ffd640 1 --2- 172.21.15.37:0/3616618552 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7fdd380ce360 0x7fdd20004800 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:28:44.010 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:28:44.001+0000 7fdd36ffd640 1 --2- 172.21.15.37:0/3616618552 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fdd20004d40 0x7fdd20005150 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:28:44.010 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:28:44.001+0000 7fdd36ffd640 1 -- 172.21.15.37:0/3616618552 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 -- 0x7fdd380d37d0 con 0x7fdd20004d40 2024-04-25T10:28:44.010 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:28:44.001+0000 7fdd36ffd640 1 -- 172.21.15.37:0/3616618552 --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_getmap magic: 0 -- 0x7fdd381623d0 con 0x7fdd200039b0 2024-04-25T10:28:44.010 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:28:44.001+0000 7fdd36ffd640 1 -- 172.21.15.37:0/3616618552 --> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] -- mon_getmap magic: 0 -- 0x7fdd2000bb20 con 0x7fdd380ce360 2024-04-25T10:28:44.010 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:28:44.001+0000 7fdd3c8d5640 1 --2- 172.21.15.37:0/3616618552 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fdd20004d40 0x7fdd20005150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:28:44.011 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:28:44.001+0000 7fdd377fe640 1 --2- 172.21.15.37:0/3616618552 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7fdd380ce360 0x7fdd20004800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:28:44.011 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:28:44.001+0000 7fdd37fff640 1 --2- 172.21.15.37:0/3616618552 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fdd200039b0 0x7fdd20003e20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:28:44.011 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:28:44.002+0000 7fdd3c8d5640 1 -- 172.21.15.37:0/3616618552 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7fdd380ce360 msgr2=0x7fdd20004800 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:28:44.011 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:28:44.002+0000 7fdd3c8d5640 1 --2- 172.21.15.37:0/3616618552 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7fdd380ce360 0x7fdd20004800 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:28:44.011 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:28:44.002+0000 7fdd3c8d5640 1 -- 172.21.15.37:0/3616618552 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fdd200039b0 msgr2=0x7fdd20003e20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:28:44.011 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:28:44.002+0000 7fdd3c8d5640 1 --2- 172.21.15.37:0/3616618552 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fdd200039b0 0x7fdd20003e20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:28:44.012 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:28:44.002+0000 7fdd3c8d5640 1 -- 172.21.15.37:0/3616618552 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fdd20003520 con 0x7fdd20004d40 2024-04-25T10:28:44.012 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:28:44.002+0000 7fdd3c8d5640 1 --2- 172.21.15.37:0/3616618552 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fdd20004d40 0x7fdd20005150 secure :-1 s=READY pgs=8 cs=0 l=1 rev1=1 crypto rx=0x7fdd380b9ad0 tx=0x7fdd20005830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=1372b0414f58cd09 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:28:44.012 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:28:44.003+0000 7fdd36ffd640 1 -- 172.21.15.37:0/3616618552 <== mon.0 v2:172.21.15.37:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdd3000a6b0 con 0x7fdd20004d40 2024-04-25T10:28:44.012 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:28:44.003+0000 7fdd36ffd640 1 -- 172.21.15.37:0/3616618552 <== mon.0 v2:172.21.15.37:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fdd3002f070 con 0x7fdd20004d40 2024-04-25T10:28:44.012 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:28:44.003+0000 7fdd36ffd640 1 -- 172.21.15.37:0/3616618552 <== mon.0 v2:172.21.15.37:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdd30034470 con 0x7fdd20004d40 2024-04-25T10:28:44.012 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:28:44.003+0000 7fdd3e338640 1 -- 172.21.15.37:0/3616618552 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fdd20004d40 msgr2=0x7fdd20005150 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:28:44.013 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:28:44.003+0000 7fdd3e338640 1 --2- 172.21.15.37:0/3616618552 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fdd20004d40 0x7fdd20005150 secure :-1 s=READY pgs=8 cs=0 l=1 rev1=1 crypto rx=0x7fdd380b9ad0 tx=0x7fdd20005830 comp rx=0 tx=0).stop 2024-04-25T10:28:44.013 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:28:44.003+0000 7fdd3c8d5640 1 -- 172.21.15.37:0/3616618552 reap_dead start 2024-04-25T10:28:44.013 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:28:44.003+0000 7fdd3e338640 1 -- 172.21.15.37:0/3616618552 shutdown_connections 2024-04-25T10:28:44.013 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:28:44.003+0000 7fdd3e338640 1 -- 172.21.15.37:0/3616618552 >> 172.21.15.37:0/3616618552 conn(0x7fdd380d0460 msgr2=0x7fdd380d0870 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:28:44.013 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:28:44.003+0000 7fdd3e338640 1 -- 172.21.15.37:0/3616618552 shutdown_connections 2024-04-25T10:28:44.013 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:28:44.004+0000 7fdd3e338640 1 -- 172.21.15.37:0/3616618552 wait complete. 2024-04-25T10:28:44.018 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:28:44.006+0000 7fdd3e338640 1 Processor -- start 2024-04-25T10:28:44.019 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:28:44.007+0000 7fdd3e338640 1 -- start start 2024-04-25T10:28:44.019 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:28:44.009+0000 7fdd3e338640 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fdd20004d40 0x7fdd38151750 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:28:44.019 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:28:44.009+0000 7fdd3e338640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fdd38160b20 0x7fdd380cd040 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:28:44.019 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:28:44.009+0000 7fdd3e338640 1 --2- >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7fdd380b7d10 0x7fdd380b8120 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:28:44.019 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:28:44.009+0000 7fdd3e338640 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 -- 0x7fdd380d22a0 con 0x7fdd38160b20 2024-04-25T10:28:44.019 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:28:44.009+0000 7fdd3e338640 1 -- --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_getmap magic: 0 -- 0x7fdd380d1fc0 con 0x7fdd20004d40 2024-04-25T10:28:44.019 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:28:44.009+0000 7fdd3e338640 1 -- --> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] -- mon_getmap magic: 0 -- 0x7fdd3815e350 con 0x7fdd380b7d10 2024-04-25T10:28:44.020 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:28:44.009+0000 7fdd377fe640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fdd38160b20 0x7fdd380cd040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:28:44.020 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:28:44.009+0000 7fdd377fe640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fdd38160b20 0x7fdd380cd040 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.37:3300/0 says I am v2:172.21.15.37:42384/0 (socket says 172.21.15.37:42384) 2024-04-25T10:28:44.020 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:28:44.009+0000 7fdd377fe640 1 -- 172.21.15.37:0/468308738 learned_addr learned my addr 172.21.15.37:0/468308738 (peer_addr_for_me v2:172.21.15.37:0/0) 2024-04-25T10:28:44.020 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:28:44.009+0000 7fdd37fff640 1 --2- 172.21.15.37:0/468308738 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fdd20004d40 0x7fdd38151750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:28:44.020 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:28:44.009+0000 7fdd3c8d5640 1 --2- 172.21.15.37:0/468308738 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7fdd380b7d10 0x7fdd380b8120 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:28:44.020 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:28:44.009+0000 7fdd377fe640 1 -- 172.21.15.37:0/468308738 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7fdd380b7d10 msgr2=0x7fdd380b8120 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:28:44.021 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:28:44.009+0000 7fdd377fe640 1 --2- 172.21.15.37:0/468308738 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7fdd380b7d10 0x7fdd380b8120 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:28:44.021 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:28:44.009+0000 7fdd377fe640 1 -- 172.21.15.37:0/468308738 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fdd20004d40 msgr2=0x7fdd38151750 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:28:44.021 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:28:44.009+0000 7fdd377fe640 1 --2- 172.21.15.37:0/468308738 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fdd20004d40 0x7fdd38151750 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:28:44.021 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:28:44.009+0000 7fdd377fe640 1 -- 172.21.15.37:0/468308738 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fdd380b8660 con 0x7fdd38160b20 2024-04-25T10:28:44.021 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:28:44.009+0000 7fdd377fe640 1 --2- 172.21.15.37:0/468308738 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fdd38160b20 0x7fdd380cd040 secure :-1 s=READY pgs=11 cs=0 l=1 rev1=1 crypto rx=0x7fdd2c010870 tx=0x7fdd2c010d30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:28:44.021 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:28:44.009+0000 7fdd34ff9640 1 -- 172.21.15.37:0/468308738 <== mon.0 v2:172.21.15.37:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdd2c01a020 con 0x7fdd38160b20 2024-04-25T10:28:44.022 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:28:44.009+0000 7fdd34ff9640 1 -- 172.21.15.37:0/468308738 <== mon.0 v2:172.21.15.37:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fdd2c00e9c0 con 0x7fdd38160b20 2024-04-25T10:28:44.022 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:28:44.009+0000 7fdd3e338640 1 -- 172.21.15.37:0/468308738 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fdd380cd580 con 0x7fdd38160b20 2024-04-25T10:28:44.022 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:28:44.009+0000 7fdd34ff9640 1 -- 172.21.15.37:0/468308738 <== mon.0 v2:172.21.15.37:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdd2c00ed90 con 0x7fdd38160b20 2024-04-25T10:28:44.022 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:28:44.009+0000 7fdd3e338640 1 -- 172.21.15.37:0/468308738 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fdd3815fd60 con 0x7fdd38160b20 2024-04-25T10:28:44.022 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:28:44.010+0000 7fdd34ff9640 1 -- 172.21.15.37:0/468308738 <== mon.0 v2:172.21.15.37:3300/0 4 ==== mgrmap(e 1) ==== 807+0+0 (secure 0 0 0) 0x7fdd2c01d020 con 0x7fdd38160b20 2024-04-25T10:28:44.022 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:28:44.010+0000 7fdd34ff9640 1 -- 172.21.15.37:0/468308738 <== mon.0 v2:172.21.15.37:3300/0 5 ==== osd_map(1..1 src has 1..1) ==== 725+0+0 (secure 0 0 0) 0x7fdd2c0148c0 con 0x7fdd38160b20 2024-04-25T10:28:44.023 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:28:44.011+0000 7fdd3e338640 1 -- 172.21.15.37:0/468308738 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fdd3806a140 con 0x7fdd38160b20 2024-04-25T10:28:44.023 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:28:44.013+0000 7fdd34ff9640 1 -- 172.21.15.37:0/468308738 <== mon.0 v2:172.21.15.37:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+73265 (secure 0 0 0) 0x7fdd2c024020 con 0x7fdd38160b20 2024-04-25T10:28:44.082 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:28:44.080+0000 7fdd3e338640 1 -- 172.21.15.37:0/468308738 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_command({"prefix": "osd crush tunables", "profile": "default"} v 0) -- 0x7fdd380d1b70 con 0x7fdd38160b20 2024-04-25T10:28:44.083 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:28:44.081+0000 7fdd34ff9640 1 -- 172.21.15.37:0/468308738 <== mon.0 v2:172.21.15.37:3300/0 7 ==== mon_command_ack([{"prefix": "osd crush tunables", "profile": "default"}]=0 adjusted tunables profile to default v2) ==== 124+0+0 (secure 0 0 0) 0x7fdd380b8660 con 0x7fdd38160b20 2024-04-25T10:28:44.084 INFO:teuthology.orchestra.run.smithi037.stderr:adjusted tunables profile to default 2024-04-25T10:28:44.085 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:28:44.082+0000 7fdd3e338640 1 -- 172.21.15.37:0/468308738 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fdd38160b20 msgr2=0x7fdd380cd040 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:28:44.085 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:28:44.082+0000 7fdd3e338640 1 --2- 172.21.15.37:0/468308738 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fdd38160b20 0x7fdd380cd040 secure :-1 s=READY pgs=11 cs=0 l=1 rev1=1 crypto rx=0x7fdd2c010870 tx=0x7fdd2c010d30 comp rx=0 tx=0).stop 2024-04-25T10:28:44.085 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:28:44.082+0000 7fdd3e338640 1 -- 172.21.15.37:0/468308738 shutdown_connections 2024-04-25T10:28:44.085 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:28:44.082+0000 7fdd3e338640 1 --2- 172.21.15.37:0/468308738 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7fdd380b7d10 0x7fdd380b8120 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:28:44.085 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:28:44.082+0000 7fdd3e338640 1 --2- 172.21.15.37:0/468308738 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fdd38160b20 0x7fdd380cd040 unknown :-1 s=CLOSED pgs=11 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:28:44.085 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:28:44.082+0000 7fdd3e338640 1 --2- 172.21.15.37:0/468308738 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fdd20004d40 0x7fdd38151750 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:28:44.085 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:28:44.082+0000 7fdd3e338640 1 -- 172.21.15.37:0/468308738 >> 172.21.15.37:0/468308738 conn(0x7fdd380d0460 msgr2=0x7fdd3815a690 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:28:44.085 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:28:44.083+0000 7fdd3e338640 1 -- 172.21.15.37:0/468308738 shutdown_connections 2024-04-25T10:28:44.085 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:28:44.083+0000 7fdd3e338640 1 -- 172.21.15.37:0/468308738 wait complete. 2024-04-25T10:28:44.097 INFO:tasks.ceph:check_enable_crimson: False 2024-04-25T10:28:44.097 INFO:tasks.ceph:Starting osd daemons in cluster ceph... 2024-04-25T10:28:44.097 DEBUG:teuthology.orchestra.run.smithi037:> set -ex 2024-04-25T10:28:44.097 DEBUG:teuthology.orchestra.run.smithi037:> sudo dd if=/var/lib/ceph/osd/ceph-0/fsid of=/dev/stdout 2024-04-25T10:28:44.109 INFO:tasks.ceph.mgr.x.smithi203.stderr:2024-04-25T10:28:44.107+0000 7fbc1bc32040 -1 mgr[py] Module balancer has missing NOTIFY_TYPES member 2024-04-25T10:28:44.122 DEBUG:teuthology.orchestra.run.smithi037:> set -ex 2024-04-25T10:28:44.122 DEBUG:teuthology.orchestra.run.smithi037:> sudo dd if=/var/lib/ceph/osd/ceph-1/fsid of=/dev/stdout 2024-04-25T10:28:44.131 INFO:tasks.ceph.mgr.y.smithi037.stderr:2024-04-25T10:28:44.128+0000 7f1c9c0f7040 -1 mgr[py] Module balancer has missing NOTIFY_TYPES member 2024-04-25T10:28:44.155 DEBUG:teuthology.orchestra.run.smithi037:> set -ex 2024-04-25T10:28:44.155 DEBUG:teuthology.orchestra.run.smithi037:> sudo dd if=/var/lib/ceph/osd/ceph-2/fsid of=/dev/stdout 2024-04-25T10:28:44.219 DEBUG:teuthology.orchestra.run.smithi037:> set -ex 2024-04-25T10:28:44.219 DEBUG:teuthology.orchestra.run.smithi037:> sudo dd if=/var/lib/ceph/osd/ceph-3/fsid of=/dev/stdout 2024-04-25T10:28:44.263 INFO:tasks.ceph.mgr.x.smithi203.stderr:2024-04-25T10:28:44.262+0000 7fbc1bc32040 -1 mgr[py] Module orchestrator has missing NOTIFY_TYPES member 2024-04-25T10:28:44.284 DEBUG:teuthology.orchestra.run.smithi203:> set -ex 2024-04-25T10:28:44.284 DEBUG:teuthology.orchestra.run.smithi203:> sudo dd if=/var/lib/ceph/osd/ceph-4/fsid of=/dev/stdout 2024-04-25T10:28:44.311 DEBUG:teuthology.orchestra.run.smithi203:> set -ex 2024-04-25T10:28:44.311 DEBUG:teuthology.orchestra.run.smithi203:> sudo dd if=/var/lib/ceph/osd/ceph-5/fsid of=/dev/stdout 2024-04-25T10:28:44.318 INFO:tasks.ceph.mgr.x.smithi203.stderr:2024-04-25T10:28:44.317+0000 7fbc1bc32040 -1 mgr[py] Module snap_schedule has missing NOTIFY_TYPES member 2024-04-25T10:28:44.341 DEBUG:teuthology.orchestra.run.smithi203:> set -ex 2024-04-25T10:28:44.341 DEBUG:teuthology.orchestra.run.smithi203:> sudo dd if=/var/lib/ceph/osd/ceph-6/fsid of=/dev/stdout 2024-04-25T10:28:44.357 INFO:tasks.ceph.mgr.y.smithi037.stderr:2024-04-25T10:28:44.355+0000 7f1c9c0f7040 -1 mgr[py] Module orchestrator has missing NOTIFY_TYPES member 2024-04-25T10:28:44.405 DEBUG:teuthology.orchestra.run.smithi203:> set -ex 2024-04-25T10:28:44.405 DEBUG:teuthology.orchestra.run.smithi203:> sudo dd if=/var/lib/ceph/osd/ceph-7/fsid of=/dev/stdout 2024-04-25T10:28:44.413 INFO:tasks.ceph.mgr.y.smithi037.stderr:2024-04-25T10:28:44.411+0000 7f1c9c0f7040 -1 mgr[py] Module snap_schedule has missing NOTIFY_TYPES member 2024-04-25T10:28:44.468 DEBUG:teuthology.orchestra.run.smithi203:> sudo ceph --cluster ceph osd new a2f79028-b332-4166-9ebb-a7024a37ea22 0 2024-04-25T10:28:44.571 INFO:tasks.ceph.mgr.x.smithi203.stderr:2024-04-25T10:28:44.569+0000 7fbc1bc32040 -1 mgr[py] Module progress has missing NOTIFY_TYPES member 2024-04-25T10:28:44.618 INFO:teuthology.orchestra.run.smithi203.stderr:2024-04-25T10:28:44.616+0000 7f418ecb6640 1 Processor -- start 2024-04-25T10:28:44.618 INFO:teuthology.orchestra.run.smithi203.stderr:2024-04-25T10:28:44.616+0000 7f418ecb6640 1 -- start start 2024-04-25T10:28:44.620 INFO:teuthology.orchestra.run.smithi203.stderr:2024-04-25T10:28:44.618+0000 7f418ecb6640 1 --2- >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f41880b1d70 0x7f418815d520 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:28:44.620 INFO:teuthology.orchestra.run.smithi203.stderr:2024-04-25T10:28:44.618+0000 7f418ecb6640 1 --2- >> v2:172.21.15.203:3300/0 conn(0x7f41880b2840 0x7f41880b2c50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:28:44.620 INFO:teuthology.orchestra.run.smithi203.stderr:2024-04-25T10:28:44.618+0000 7f418ecb6640 1 -- --> v1:172.21.15.203:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f41880d2700 con 0x7f4188160b10 2024-04-25T10:28:44.620 INFO:teuthology.orchestra.run.smithi203.stderr:2024-04-25T10:28:44.618+0000 7f418ecb6640 1 -- --> v2:172.21.15.203:3300/0 -- mon_getmap magic: 0 -- 0x7f41880b3190 con 0x7f41880b2840 2024-04-25T10:28:44.620 INFO:teuthology.orchestra.run.smithi203.stderr:2024-04-25T10:28:44.618+0000 7f418ecb6640 1 -- --> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] -- mon_getmap magic: 0 -- 0x7f418815aef0 con 0x7f41880b1d70 2024-04-25T10:28:44.620 INFO:teuthology.orchestra.run.smithi203.stderr:2024-04-25T10:28:44.618+0000 7f418ca52640 1 --2- >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f41880b1d70 0x7f418815d520 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:28:44.620 INFO:teuthology.orchestra.run.smithi203.stderr:2024-04-25T10:28:44.618+0000 7f418d253640 1 --2- >> v2:172.21.15.203:3300/0 conn(0x7f41880b2840 0x7f41880b2c50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:28:44.621 INFO:teuthology.orchestra.run.smithi203.stderr:2024-04-25T10:28:44.618+0000 7f418ca52640 1 --2- >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f41880b1d70 0x7f418815d520 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.203:3301/0 says I am v2:172.21.15.203:38856/0 (socket says 172.21.15.203:38856) 2024-04-25T10:28:44.621 INFO:teuthology.orchestra.run.smithi203.stderr:2024-04-25T10:28:44.618+0000 7f418ca52640 1 -- 172.21.15.203:0/776960556 learned_addr learned my addr 172.21.15.203:0/776960556 (peer_addr_for_me v2:172.21.15.203:0/0) 2024-04-25T10:28:44.621 INFO:teuthology.orchestra.run.smithi203.stderr:2024-04-25T10:28:44.618+0000 7f418d253640 1 -- 172.21.15.203:0/776960556 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f41880b1d70 msgr2=0x7f418815d520 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:28:44.621 INFO:teuthology.orchestra.run.smithi203.stderr:2024-04-25T10:28:44.618+0000 7f418d253640 1 --2- 172.21.15.203:0/776960556 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f41880b1d70 0x7f418815d520 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:28:44.621 INFO:teuthology.orchestra.run.smithi203.stderr:2024-04-25T10:28:44.618+0000 7f418d253640 1 -- 172.21.15.203:0/776960556 >> v1:172.21.15.203:6789/0 conn(0x7f4188160b10 legacy=0x7f41880d2e90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:28:44.621 INFO:teuthology.orchestra.run.smithi203.stderr:2024-04-25T10:28:44.618+0000 7f418d253640 1 -- 172.21.15.203:0/776960556 --> v2:172.21.15.203:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f4188161360 con 0x7f41880b2840 2024-04-25T10:28:44.621 INFO:teuthology.orchestra.run.smithi203.stderr:2024-04-25T10:28:44.618+0000 7f418ca52640 1 --2- 172.21.15.203:0/776960556 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f41880b1d70 0x7f418815d520 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-25T10:28:44.621 INFO:tasks.ceph.mgr.x.smithi203.stderr:2024-04-25T10:28:44.619+0000 7fbc1bc32040 -1 mgr[py] Module influx has missing NOTIFY_TYPES member 2024-04-25T10:28:44.622 INFO:teuthology.orchestra.run.smithi203.stderr:2024-04-25T10:28:44.619+0000 7f418d253640 1 --2- 172.21.15.203:0/776960556 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f41880b2840 0x7f41880b2c50 secure :-1 s=READY pgs=6 cs=0 l=1 rev1=1 crypto rx=0x7f4180009d90 tx=0x7f418002a660 comp rx=0 tx=0).ready entity=mon.1 client_cookie=dea7a8dd27e39d68 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:28:44.623 INFO:teuthology.orchestra.run.smithi203.stderr:2024-04-25T10:28:44.620+0000 7f41877fe640 1 -- 172.21.15.203:0/776960556 <== mon.1 v2:172.21.15.203:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4180007740 con 0x7f41880b2840 2024-04-25T10:28:44.623 INFO:teuthology.orchestra.run.smithi203.stderr:2024-04-25T10:28:44.620+0000 7f41877fe640 1 -- 172.21.15.203:0/776960556 <== mon.1 v2:172.21.15.203:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f41800078e0 con 0x7f41880b2840 2024-04-25T10:28:44.623 INFO:teuthology.orchestra.run.smithi203.stderr:2024-04-25T10:28:44.620+0000 7f41877fe640 1 -- 172.21.15.203:0/776960556 <== mon.1 v2:172.21.15.203:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4180007cb0 con 0x7f41880b2840 2024-04-25T10:28:44.623 INFO:teuthology.orchestra.run.smithi203.stderr:2024-04-25T10:28:44.620+0000 7f418ecb6640 1 -- 172.21.15.203:0/776960556 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f41880b2840 msgr2=0x7f41880b2c50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:28:44.623 INFO:teuthology.orchestra.run.smithi203.stderr:2024-04-25T10:28:44.620+0000 7f418ecb6640 1 --2- 172.21.15.203:0/776960556 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f41880b2840 0x7f41880b2c50 secure :-1 s=READY pgs=6 cs=0 l=1 rev1=1 crypto rx=0x7f4180009d90 tx=0x7f418002a660 comp rx=0 tx=0).stop 2024-04-25T10:28:44.623 INFO:teuthology.orchestra.run.smithi203.stderr:2024-04-25T10:28:44.620+0000 7f418ecb6640 1 -- 172.21.15.203:0/776960556 shutdown_connections 2024-04-25T10:28:44.623 INFO:teuthology.orchestra.run.smithi203.stderr:2024-04-25T10:28:44.620+0000 7f418ecb6640 1 --2- 172.21.15.203:0/776960556 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f41880b2840 0x7f41880b2c50 unknown :-1 s=CLOSED pgs=6 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:28:44.623 INFO:teuthology.orchestra.run.smithi203.stderr:2024-04-25T10:28:44.620+0000 7f418ecb6640 1 --2- 172.21.15.203:0/776960556 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f41880b1d70 0x7f418815d520 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:28:44.623 INFO:teuthology.orchestra.run.smithi203.stderr:2024-04-25T10:28:44.620+0000 7f418ecb6640 1 -- 172.21.15.203:0/776960556 >> 172.21.15.203:0/776960556 conn(0x7f41880b7620 msgr2=0x7f41880b7c50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:28:44.623 INFO:teuthology.orchestra.run.smithi203.stderr:2024-04-25T10:28:44.620+0000 7f418ecb6640 1 -- 172.21.15.203:0/776960556 shutdown_connections 2024-04-25T10:28:44.623 INFO:teuthology.orchestra.run.smithi203.stderr:2024-04-25T10:28:44.621+0000 7f418ecb6640 1 -- 172.21.15.203:0/776960556 wait complete. 2024-04-25T10:28:44.624 INFO:teuthology.orchestra.run.smithi203.stderr:2024-04-25T10:28:44.622+0000 7f418ecb6640 1 Processor -- start 2024-04-25T10:28:44.624 INFO:teuthology.orchestra.run.smithi203.stderr:2024-04-25T10:28:44.622+0000 7f418ecb6640 1 -- start start 2024-04-25T10:28:44.627 INFO:teuthology.orchestra.run.smithi203.stderr:2024-04-25T10:28:44.624+0000 7f418ecb6640 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f41880b1d70 0x7f4188155310 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:28:44.627 INFO:teuthology.orchestra.run.smithi203.stderr:2024-04-25T10:28:44.624+0000 7f418ecb6640 1 --2- >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f41880b2840 0x7f4188157f10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:28:44.627 INFO:teuthology.orchestra.run.smithi203.stderr:2024-04-25T10:28:44.624+0000 7f418ecb6640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f4188160b10 0x7f4188158450 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:28:44.627 INFO:teuthology.orchestra.run.smithi203.stderr:2024-04-25T10:28:44.624+0000 7f418ecb6640 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 -- 0x7f4188161920 con 0x7f4188160b10 2024-04-25T10:28:44.627 INFO:teuthology.orchestra.run.smithi203.stderr:2024-04-25T10:28:44.624+0000 7f418ecb6640 1 -- --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_getmap magic: 0 -- 0x7f41880d2970 con 0x7f41880b1d70 2024-04-25T10:28:44.627 INFO:teuthology.orchestra.run.smithi203.stderr:2024-04-25T10:28:44.624+0000 7f418ecb6640 1 -- --> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] -- mon_getmap magic: 0 -- 0x7f41880d35a0 con 0x7f41880b2840 2024-04-25T10:28:44.627 INFO:teuthology.orchestra.run.smithi203.stderr:2024-04-25T10:28:44.624+0000 7f418ca52640 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f41880b1d70 0x7f4188155310 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:28:44.627 INFO:teuthology.orchestra.run.smithi203.stderr:2024-04-25T10:28:44.624+0000 7f4187fff640 1 --2- >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f41880b2840 0x7f4188157f10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:28:44.628 INFO:teuthology.orchestra.run.smithi203.stderr:2024-04-25T10:28:44.624+0000 7f418ca52640 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f41880b1d70 0x7f4188155310 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.203:3300/0 says I am v2:172.21.15.203:44180/0 (socket says 172.21.15.203:44180) 2024-04-25T10:28:44.628 INFO:teuthology.orchestra.run.smithi203.stderr:2024-04-25T10:28:44.624+0000 7f418ca52640 1 -- 172.21.15.203:0/790594567 learned_addr learned my addr 172.21.15.203:0/790594567 (peer_addr_for_me v2:172.21.15.203:0/0) 2024-04-25T10:28:44.628 INFO:teuthology.orchestra.run.smithi203.stderr:2024-04-25T10:28:44.624+0000 7f418d253640 1 --2- 172.21.15.203:0/790594567 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f4188160b10 0x7f4188158450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:28:44.628 INFO:teuthology.orchestra.run.smithi203.stderr:2024-04-25T10:28:44.624+0000 7f418ca52640 1 -- 172.21.15.203:0/790594567 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f41880b2840 msgr2=0x7f4188157f10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:28:44.628 INFO:teuthology.orchestra.run.smithi203.stderr:2024-04-25T10:28:44.624+0000 7f418ca52640 1 --2- 172.21.15.203:0/790594567 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f41880b2840 0x7f4188157f10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:28:44.628 INFO:teuthology.orchestra.run.smithi203.stderr:2024-04-25T10:28:44.624+0000 7f418ca52640 1 -- 172.21.15.203:0/790594567 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f4188160b10 msgr2=0x7f4188158450 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:28:44.628 INFO:teuthology.orchestra.run.smithi203.stderr:2024-04-25T10:28:44.624+0000 7f418ca52640 1 --2- 172.21.15.203:0/790594567 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f4188160b10 0x7f4188158450 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:28:44.628 INFO:teuthology.orchestra.run.smithi203.stderr:2024-04-25T10:28:44.624+0000 7f418ca52640 1 -- 172.21.15.203:0/790594567 --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f4188157120 con 0x7f41880b1d70 2024-04-25T10:28:44.628 INFO:teuthology.orchestra.run.smithi203.stderr:2024-04-25T10:28:44.624+0000 7f4187fff640 1 --2- 172.21.15.203:0/790594567 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f41880b2840 0x7f4188157f10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-25T10:28:44.628 INFO:teuthology.orchestra.run.smithi203.stderr:2024-04-25T10:28:44.624+0000 7f418d253640 1 --2- 172.21.15.203:0/790594567 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f4188160b10 0x7f4188158450 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T10:28:44.628 INFO:teuthology.orchestra.run.smithi203.stderr:2024-04-25T10:28:44.624+0000 7f418ca52640 1 --2- 172.21.15.203:0/790594567 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f41880b1d70 0x7f4188155310 secure :-1 s=READY pgs=7 cs=0 l=1 rev1=1 crypto rx=0x7f417c00ea10 tx=0x7f417c00eed0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:28:44.628 INFO:teuthology.orchestra.run.smithi203.stderr:2024-04-25T10:28:44.624+0000 7f41857fa640 1 -- 172.21.15.203:0/790594567 <== mon.1 v2:172.21.15.203:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f417c017070 con 0x7f41880b1d70 2024-04-25T10:28:44.628 INFO:teuthology.orchestra.run.smithi203.stderr:2024-04-25T10:28:44.625+0000 7f41857fa640 1 -- 172.21.15.203:0/790594567 <== mon.1 v2:172.21.15.203:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f417c00ca00 con 0x7f41880b1d70 2024-04-25T10:28:44.628 INFO:teuthology.orchestra.run.smithi203.stderr:2024-04-25T10:28:44.625+0000 7f418ecb6640 1 -- 172.21.15.203:0/790594567 --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f41881576e0 con 0x7f41880b1d70 2024-04-25T10:28:44.629 INFO:teuthology.orchestra.run.smithi203.stderr:2024-04-25T10:28:44.625+0000 7f41857fa640 1 -- 172.21.15.203:0/790594567 <== mon.1 v2:172.21.15.203:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f417c00cdb0 con 0x7f41880b1d70 2024-04-25T10:28:44.629 INFO:teuthology.orchestra.run.smithi203.stderr:2024-04-25T10:28:44.625+0000 7f418ecb6640 1 -- 172.21.15.203:0/790594567 --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f41880d0610 con 0x7f41880b1d70 2024-04-25T10:28:44.629 INFO:teuthology.orchestra.run.smithi203.stderr:2024-04-25T10:28:44.625+0000 7f41857fa640 1 -- 172.21.15.203:0/790594567 <== mon.1 v2:172.21.15.203:3300/0 4 ==== mgrmap(e 1) ==== 807+0+0 (secure 0 0 0) 0x7f417c01a070 con 0x7f41880b1d70 2024-04-25T10:28:44.629 INFO:teuthology.orchestra.run.smithi203.stderr:2024-04-25T10:28:44.625+0000 7f41857fa640 1 -- 172.21.15.203:0/790594567 <== mon.1 v2:172.21.15.203:3300/0 5 ==== osd_map(2..2 src has 1..2) ==== 725+0+0 (secure 0 0 0) 0x7f417c013470 con 0x7f41880b1d70 2024-04-25T10:28:44.629 INFO:teuthology.orchestra.run.smithi203.stderr:2024-04-25T10:28:44.626+0000 7f418ecb6640 1 -- 172.21.15.203:0/790594567 --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f418806a140 con 0x7f41880b1d70 2024-04-25T10:28:44.629 INFO:teuthology.orchestra.run.smithi203.stderr:2024-04-25T10:28:44.628+0000 7f41857fa640 1 -- 172.21.15.203:0/790594567 <== mon.1 v2:172.21.15.203:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+73265 (secure 0 0 0) 0x7f417c005990 con 0x7f41880b1d70 2024-04-25T10:28:44.676 INFO:tasks.ceph.mgr.y.smithi037.stderr:2024-04-25T10:28:44.674+0000 7f1c9c0f7040 -1 mgr[py] Module progress has missing NOTIFY_TYPES member 2024-04-25T10:28:44.702 INFO:teuthology.orchestra.run.smithi203.stderr:2024-04-25T10:28:44.701+0000 7f418ecb6640 1 -- 172.21.15.203:0/790594567 --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_command({"prefix": "osd new", "uuid": "a2f79028-b332-4166-9ebb-a7024a37ea22", "id": 0} v 0) -- 0x7f41880cea60 con 0x7f41880b1d70 2024-04-25T10:28:44.705 INFO:teuthology.orchestra.run.smithi203.stderr:2024-04-25T10:28:44.704+0000 7f41857fa640 1 -- 172.21.15.203:0/790594567 <== mon.1 v2:172.21.15.203:3300/0 7 ==== mon_command_ack([{"prefix": "osd new", "uuid": "a2f79028-b332-4166-9ebb-a7024a37ea22", "id": 0}]=0 v3) ==== 112+0+2 (secure 0 0 0) 0x7f417c005b70 con 0x7f41880b1d70 2024-04-25T10:28:44.705 INFO:teuthology.orchestra.run.smithi203.stdout:0 2024-04-25T10:28:44.707 INFO:teuthology.orchestra.run.smithi203.stderr:2024-04-25T10:28:44.705+0000 7f418ecb6640 1 -- 172.21.15.203:0/790594567 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f41880b1d70 msgr2=0x7f4188155310 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:28:44.707 INFO:teuthology.orchestra.run.smithi203.stderr:2024-04-25T10:28:44.705+0000 7f418ecb6640 1 --2- 172.21.15.203:0/790594567 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f41880b1d70 0x7f4188155310 secure :-1 s=READY pgs=7 cs=0 l=1 rev1=1 crypto rx=0x7f417c00ea10 tx=0x7f417c00eed0 comp rx=0 tx=0).stop 2024-04-25T10:28:44.707 INFO:teuthology.orchestra.run.smithi203.stderr:2024-04-25T10:28:44.705+0000 7f418ecb6640 1 -- 172.21.15.203:0/790594567 shutdown_connections 2024-04-25T10:28:44.707 INFO:teuthology.orchestra.run.smithi203.stderr:2024-04-25T10:28:44.705+0000 7f418ecb6640 1 --2- 172.21.15.203:0/790594567 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f4188160b10 0x7f4188158450 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:28:44.707 INFO:teuthology.orchestra.run.smithi203.stderr:2024-04-25T10:28:44.705+0000 7f418ecb6640 1 --2- 172.21.15.203:0/790594567 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f41880b2840 0x7f4188157f10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:28:44.707 INFO:teuthology.orchestra.run.smithi203.stderr:2024-04-25T10:28:44.705+0000 7f418ecb6640 1 --2- 172.21.15.203:0/790594567 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f41880b1d70 0x7f4188155310 unknown :-1 s=CLOSED pgs=7 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:28:44.707 INFO:teuthology.orchestra.run.smithi203.stderr:2024-04-25T10:28:44.705+0000 7f418ecb6640 1 -- 172.21.15.203:0/790594567 >> 172.21.15.203:0/790594567 conn(0x7f41880b7620 msgr2=0x7f418815ddc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:28:44.707 INFO:teuthology.orchestra.run.smithi203.stderr:2024-04-25T10:28:44.705+0000 7f418ecb6640 1 -- 172.21.15.203:0/790594567 shutdown_connections 2024-04-25T10:28:44.707 INFO:teuthology.orchestra.run.smithi203.stderr:2024-04-25T10:28:44.705+0000 7f418ecb6640 1 -- 172.21.15.203:0/790594567 wait complete. 2024-04-25T10:28:44.719 DEBUG:teuthology.orchestra.run.smithi203:> sudo ceph --cluster ceph osd new 973159de-c2ea-4083-aecc-69373d1c1132 1 2024-04-25T10:28:44.724 INFO:tasks.ceph.mgr.y.smithi037.stderr:2024-04-25T10:28:44.722+0000 7f1c9c0f7040 -1 mgr[py] Module influx has missing NOTIFY_TYPES member 2024-04-25T10:28:44.827 INFO:teuthology.orchestra.run.smithi203.stderr:2024-04-25T10:28:44.825+0000 7fd1dbf7d640 1 Processor -- start 2024-04-25T10:28:44.827 INFO:teuthology.orchestra.run.smithi203.stderr:2024-04-25T10:28:44.826+0000 7fd1dbf7d640 1 -- start start 2024-04-25T10:28:44.830 INFO:teuthology.orchestra.run.smithi203.stderr:2024-04-25T10:28:44.827+0000 7fd1dbf7d640 1 --2- >> v2:172.21.15.37:3300/0 conn(0x7fd1d40ce280 0x7fd1d40ce690 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:28:44.830 INFO:teuthology.orchestra.run.smithi203.stderr:2024-04-25T10:28:44.827+0000 7fd1dbf7d640 1 --2- >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7fd1d40b26f0 0x7fd1d40b2b00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:28:44.830 INFO:teuthology.orchestra.run.smithi203.stderr:2024-04-25T10:28:44.827+0000 7fd1dbf7d640 1 -- --> v1:172.21.15.37:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fd1d40cf800 con 0x7fd1d415e420 2024-04-25T10:28:44.830 INFO:teuthology.orchestra.run.smithi203.stderr:2024-04-25T10:28:44.827+0000 7fd1dbf7d640 1 -- --> v2:172.21.15.37:3300/0 -- mon_getmap magic: 0 -- 0x7fd1d40cfae0 con 0x7fd1d40ce280 2024-04-25T10:28:44.830 INFO:teuthology.orchestra.run.smithi203.stderr:2024-04-25T10:28:44.827+0000 7fd1dbf7d640 1 -- --> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] -- mon_getmap magic: 0 -- 0x7fd1d40cfdc0 con 0x7fd1d40b26f0 2024-04-25T10:28:44.830 INFO:teuthology.orchestra.run.smithi203.stderr:2024-04-25T10:28:44.827+0000 7fd1da51a640 1 --2- >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7fd1d40b26f0 0x7fd1d40b2b00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:28:44.830 INFO:teuthology.orchestra.run.smithi203.stderr:2024-04-25T10:28:44.827+0000 7fd1da51a640 1 --2- >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7fd1d40b26f0 0x7fd1d40b2b00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.203:3301/0 says I am v2:172.21.15.203:38874/0 (socket says 172.21.15.203:38874) 2024-04-25T10:28:44.830 INFO:teuthology.orchestra.run.smithi203.stderr:2024-04-25T10:28:44.827+0000 7fd1da51a640 1 -- 172.21.15.203:0/4226016222 learned_addr learned my addr 172.21.15.203:0/4226016222 (peer_addr_for_me v2:172.21.15.203:0/0) 2024-04-25T10:28:44.830 INFO:teuthology.orchestra.run.smithi203.stderr:2024-04-25T10:28:44.827+0000 7fd1d9518640 1 --2- 172.21.15.203:0/4226016222 >> v2:172.21.15.37:3300/0 conn(0x7fd1d40ce280 0x7fd1d40ce690 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:28:44.830 INFO:teuthology.orchestra.run.smithi203.stderr:2024-04-25T10:28:44.827+0000 7fd1da51a640 1 -- 172.21.15.203:0/4226016222 >> v2:172.21.15.37:3300/0 conn(0x7fd1d40ce280 msgr2=0x7fd1d40ce690 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:28:44.830 INFO:teuthology.orchestra.run.smithi203.stderr:2024-04-25T10:28:44.827+0000 7fd1da51a640 1 --2- 172.21.15.203:0/4226016222 >> v2:172.21.15.37:3300/0 conn(0x7fd1d40ce280 0x7fd1d40ce690 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:28:44.831 INFO:teuthology.orchestra.run.smithi203.stderr:2024-04-25T10:28:44.827+0000 7fd1da51a640 1 -- 172.21.15.203:0/4226016222 >> v1:172.21.15.37:6789/0 conn(0x7fd1d415e420 legacy=0x7fd1d40cef40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:28:44.831 INFO:teuthology.orchestra.run.smithi203.stderr:2024-04-25T10:28:44.827+0000 7fd1da51a640 1 -- 172.21.15.203:0/4226016222 --> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fd1d40d00a0 con 0x7fd1d40b26f0 2024-04-25T10:28:44.831 INFO:teuthology.orchestra.run.smithi203.stderr:2024-04-25T10:28:44.828+0000 7fd1da51a640 1 --2- 172.21.15.203:0/4226016222 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7fd1d40b26f0 0x7fd1d40b2b00 secure :-1 s=READY pgs=8 cs=0 l=1 rev1=1 crypto rx=0x7fd1cc009e50 tx=0x7fd1cc02a710 comp rx=0 tx=0).ready entity=mon.2 client_cookie=e4cc7004406e29fb server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:28:44.831 INFO:teuthology.orchestra.run.smithi203.stderr:2024-04-25T10:28:44.828+0000 7fd1d8d17640 1 -- 172.21.15.203:0/4226016222 <== mon.2 v2:172.21.15.203:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd1cc036070 con 0x7fd1d40b26f0 2024-04-25T10:28:44.831 INFO:teuthology.orchestra.run.smithi203.stderr:2024-04-25T10:28:44.828+0000 7fd1d8d17640 1 -- 172.21.15.203:0/4226016222 <== mon.2 v2:172.21.15.203:3301/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fd1cc031600 con 0x7fd1d40b26f0 2024-04-25T10:28:44.831 INFO:teuthology.orchestra.run.smithi203.stderr:2024-04-25T10:28:44.828+0000 7fd1d8d17640 1 -- 172.21.15.203:0/4226016222 <== mon.2 v2:172.21.15.203:3301/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd1cc0319d0 con 0x7fd1d40b26f0 2024-04-25T10:28:44.832 INFO:teuthology.orchestra.run.smithi203.stderr:2024-04-25T10:28:44.829+0000 7fd1dbf7d640 1 -- 172.21.15.203:0/4226016222 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7fd1d40b26f0 msgr2=0x7fd1d40b2b00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:28:44.832 INFO:teuthology.orchestra.run.smithi203.stderr:2024-04-25T10:28:44.829+0000 7fd1dbf7d640 1 --2- 172.21.15.203:0/4226016222 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7fd1d40b26f0 0x7fd1d40b2b00 secure :-1 s=READY pgs=8 cs=0 l=1 rev1=1 crypto rx=0x7fd1cc009e50 tx=0x7fd1cc02a710 comp rx=0 tx=0).stop 2024-04-25T10:28:44.832 INFO:teuthology.orchestra.run.smithi203.stderr:2024-04-25T10:28:44.829+0000 7fd1dbf7d640 1 -- 172.21.15.203:0/4226016222 shutdown_connections 2024-04-25T10:28:44.832 INFO:teuthology.orchestra.run.smithi203.stderr:2024-04-25T10:28:44.829+0000 7fd1dbf7d640 1 --2- 172.21.15.203:0/4226016222 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7fd1d40b26f0 0x7fd1d40b2b00 unknown :-1 s=CLOSED pgs=8 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:28:44.832 INFO:teuthology.orchestra.run.smithi203.stderr:2024-04-25T10:28:44.829+0000 7fd1dbf7d640 1 --2- 172.21.15.203:0/4226016222 >> v2:172.21.15.37:3300/0 conn(0x7fd1d40ce280 0x7fd1d40ce690 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:28:44.832 INFO:teuthology.orchestra.run.smithi203.stderr:2024-04-25T10:28:44.829+0000 7fd1dbf7d640 1 -- 172.21.15.203:0/4226016222 >> 172.21.15.203:0/4226016222 conn(0x7fd1d409b700 msgr2=0x7fd1d409bb10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:28:44.832 INFO:teuthology.orchestra.run.smithi203.stderr:2024-04-25T10:28:44.829+0000 7fd1dbf7d640 1 -- 172.21.15.203:0/4226016222 shutdown_connections 2024-04-25T10:28:44.832 INFO:teuthology.orchestra.run.smithi203.stderr:2024-04-25T10:28:44.829+0000 7fd1dbf7d640 1 -- 172.21.15.203:0/4226016222 wait complete. 2024-04-25T10:28:44.833 INFO:teuthology.orchestra.run.smithi203.stderr:2024-04-25T10:28:44.831+0000 7fd1dbf7d640 1 Processor -- start 2024-04-25T10:28:44.833 INFO:teuthology.orchestra.run.smithi203.stderr:2024-04-25T10:28:44.831+0000 7fd1dbf7d640 1 -- start start 2024-04-25T10:28:44.835 INFO:teuthology.orchestra.run.smithi203.stderr:2024-04-25T10:28:44.832+0000 7fd1dbf7d640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fd1d40b26f0 0x7fd1d41f5430 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:28:44.835 INFO:teuthology.orchestra.run.smithi203.stderr:2024-04-25T10:28:44.832+0000 7fd1dbf7d640 1 --2- >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7fd1d40ce280 0x7fd1d4158fa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:28:44.835 INFO:teuthology.orchestra.run.smithi203.stderr:2024-04-25T10:28:44.832+0000 7fd1dbf7d640 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fd1d415e420 0x7fd1d40c89c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:28:44.835 INFO:teuthology.orchestra.run.smithi203.stderr:2024-04-25T10:28:44.832+0000 7fd1dbf7d640 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 -- 0x7fd1d40b3b20 con 0x7fd1d40b26f0 2024-04-25T10:28:44.835 INFO:teuthology.orchestra.run.smithi203.stderr:2024-04-25T10:28:44.832+0000 7fd1dbf7d640 1 -- --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_getmap magic: 0 -- 0x7fd1d40b39a0 con 0x7fd1d415e420 2024-04-25T10:28:44.835 INFO:teuthology.orchestra.run.smithi203.stderr:2024-04-25T10:28:44.832+0000 7fd1dbf7d640 1 -- --> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] -- mon_getmap magic: 0 -- 0x7fd1d415e830 con 0x7fd1d40ce280 2024-04-25T10:28:44.835 INFO:teuthology.orchestra.run.smithi203.stderr:2024-04-25T10:28:44.832+0000 7fd1d9518640 1 --2- >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7fd1d40ce280 0x7fd1d4158fa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:28:44.835 INFO:teuthology.orchestra.run.smithi203.stderr:2024-04-25T10:28:44.832+0000 7fd1da51a640 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fd1d415e420 0x7fd1d40c89c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:28:44.835 INFO:teuthology.orchestra.run.smithi203.stderr:2024-04-25T10:28:44.832+0000 7fd1d9518640 1 --2- >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7fd1d40ce280 0x7fd1d4158fa0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.203:3301/0 says I am v2:172.21.15.203:38882/0 (socket says 172.21.15.203:38882) 2024-04-25T10:28:44.836 INFO:teuthology.orchestra.run.smithi203.stderr:2024-04-25T10:28:44.832+0000 7fd1da51a640 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fd1d415e420 0x7fd1d40c89c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.203:3300/0 says I am v2:172.21.15.203:44188/0 (socket says 172.21.15.203:44188) 2024-04-25T10:28:44.836 INFO:teuthology.orchestra.run.smithi203.stderr:2024-04-25T10:28:44.832+0000 7fd1d9518640 1 -- 172.21.15.203:0/1629792132 learned_addr learned my addr 172.21.15.203:0/1629792132 (peer_addr_for_me v2:172.21.15.203:0/0) 2024-04-25T10:28:44.836 INFO:teuthology.orchestra.run.smithi203.stderr:2024-04-25T10:28:44.832+0000 7fd1d9d19640 1 --2- 172.21.15.203:0/1629792132 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fd1d40b26f0 0x7fd1d41f5430 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:28:44.836 INFO:teuthology.orchestra.run.smithi203.stderr:2024-04-25T10:28:44.833+0000 7fd1d9518640 1 -- 172.21.15.203:0/1629792132 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fd1d415e420 msgr2=0x7fd1d40c89c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:28:44.836 INFO:teuthology.orchestra.run.smithi203.stderr:2024-04-25T10:28:44.833+0000 7fd1d9518640 1 --2- 172.21.15.203:0/1629792132 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fd1d415e420 0x7fd1d40c89c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:28:44.836 INFO:teuthology.orchestra.run.smithi203.stderr:2024-04-25T10:28:44.833+0000 7fd1d9518640 1 -- 172.21.15.203:0/1629792132 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fd1d40b26f0 msgr2=0x7fd1d41f5430 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:28:44.836 INFO:teuthology.orchestra.run.smithi203.stderr:2024-04-25T10:28:44.833+0000 7fd1d9518640 1 --2- 172.21.15.203:0/1629792132 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fd1d40b26f0 0x7fd1d41f5430 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:28:44.836 INFO:teuthology.orchestra.run.smithi203.stderr:2024-04-25T10:28:44.833+0000 7fd1d9518640 1 -- 172.21.15.203:0/1629792132 --> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fd1d40c6cd0 con 0x7fd1d40ce280 2024-04-25T10:28:44.836 INFO:teuthology.orchestra.run.smithi203.stderr:2024-04-25T10:28:44.833+0000 7fd1da51a640 1 --2- 172.21.15.203:0/1629792132 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fd1d415e420 0x7fd1d40c89c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-25T10:28:44.836 INFO:teuthology.orchestra.run.smithi203.stderr:2024-04-25T10:28:44.833+0000 7fd1d9d19640 1 --2- 172.21.15.203:0/1629792132 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fd1d40b26f0 0x7fd1d41f5430 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-25T10:28:44.836 INFO:teuthology.orchestra.run.smithi203.stderr:2024-04-25T10:28:44.833+0000 7fd1d9518640 1 --2- 172.21.15.203:0/1629792132 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7fd1d40ce280 0x7fd1d4158fa0 secure :-1 s=READY pgs=9 cs=0 l=1 rev1=1 crypto rx=0x7fd1d000e870 tx=0x7fd1d000ed30 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:28:44.836 INFO:teuthology.orchestra.run.smithi203.stderr:2024-04-25T10:28:44.833+0000 7fd1c67fc640 1 -- 172.21.15.203:0/1629792132 <== mon.2 v2:172.21.15.203:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd1d0017070 con 0x7fd1d40ce280 2024-04-25T10:28:44.836 INFO:teuthology.orchestra.run.smithi203.stderr:2024-04-25T10:28:44.833+0000 7fd1c67fc640 1 -- 172.21.15.203:0/1629792132 <== mon.2 v2:172.21.15.203:3301/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fd1d000c940 con 0x7fd1d40ce280 2024-04-25T10:28:44.836 INFO:teuthology.orchestra.run.smithi203.stderr:2024-04-25T10:28:44.833+0000 7fd1dbf7d640 1 -- 172.21.15.203:0/1629792132 --> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fd1d40c6fb0 con 0x7fd1d40ce280 2024-04-25T10:28:44.836 INFO:teuthology.orchestra.run.smithi203.stderr:2024-04-25T10:28:44.833+0000 7fd1c67fc640 1 -- 172.21.15.203:0/1629792132 <== mon.2 v2:172.21.15.203:3301/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd1d000cd10 con 0x7fd1d40ce280 2024-04-25T10:28:44.837 INFO:teuthology.orchestra.run.smithi203.stderr:2024-04-25T10:28:44.833+0000 7fd1dbf7d640 1 -- 172.21.15.203:0/1629792132 --> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] -- mon_subscribe({osdmap=0}) -- 0x7fd1d415d320 con 0x7fd1d40ce280 2024-04-25T10:28:44.837 INFO:teuthology.orchestra.run.smithi203.stderr:2024-04-25T10:28:44.833+0000 7fd1c67fc640 1 -- 172.21.15.203:0/1629792132 <== mon.2 v2:172.21.15.203:3301/0 4 ==== mgrmap(e 1) ==== 807+0+0 (secure 0 0 0) 0x7fd1d001a070 con 0x7fd1d40ce280 2024-04-25T10:28:44.837 INFO:teuthology.orchestra.run.smithi203.stderr:2024-04-25T10:28:44.833+0000 7fd1c67fc640 1 -- 172.21.15.203:0/1629792132 <== mon.2 v2:172.21.15.203:3301/0 5 ==== osd_map(3..3 src has 1..3) ==== 840+0+0 (secure 0 0 0) 0x7fd1d0013470 con 0x7fd1d40ce280 2024-04-25T10:28:44.837 INFO:teuthology.orchestra.run.smithi203.stderr:2024-04-25T10:28:44.834+0000 7fd1dbf7d640 1 -- 172.21.15.203:0/1629792132 --> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fd1d406a140 con 0x7fd1d40ce280 2024-04-25T10:28:44.837 INFO:teuthology.orchestra.run.smithi203.stderr:2024-04-25T10:28:44.836+0000 7fd1c67fc640 1 -- 172.21.15.203:0/1629792132 <== mon.2 v2:172.21.15.203:3301/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+73265 (secure 0 0 0) 0x7fd1d0006020 con 0x7fd1d40ce280 2024-04-25T10:28:44.910 INFO:teuthology.orchestra.run.smithi203.stderr:2024-04-25T10:28:44.909+0000 7fd1dbf7d640 1 -- 172.21.15.203:0/1629792132 --> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] -- mon_command({"prefix": "osd new", "uuid": "973159de-c2ea-4083-aecc-69373d1c1132", "id": 1} v 0) -- 0x7fd1d40d3300 con 0x7fd1d40ce280 2024-04-25T10:28:44.914 INFO:teuthology.orchestra.run.smithi203.stderr:2024-04-25T10:28:44.913+0000 7fd1c67fc640 1 -- 172.21.15.203:0/1629792132 <== mon.2 v2:172.21.15.203:3301/0 7 ==== mon_command_ack([{"prefix": "osd new", "uuid": "973159de-c2ea-4083-aecc-69373d1c1132", "id": 1}]=0 v4) ==== 112+0+2 (secure 0 0 0) 0x7fd1d0005990 con 0x7fd1d40ce280 2024-04-25T10:28:44.914 INFO:teuthology.orchestra.run.smithi203.stdout:1 2024-04-25T10:28:44.915 INFO:teuthology.orchestra.run.smithi203.stderr:2024-04-25T10:28:44.914+0000 7fd1dbf7d640 1 -- 172.21.15.203:0/1629792132 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7fd1d40ce280 msgr2=0x7fd1d4158fa0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:28:44.915 INFO:teuthology.orchestra.run.smithi203.stderr:2024-04-25T10:28:44.914+0000 7fd1dbf7d640 1 --2- 172.21.15.203:0/1629792132 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7fd1d40ce280 0x7fd1d4158fa0 secure :-1 s=READY pgs=9 cs=0 l=1 rev1=1 crypto rx=0x7fd1d000e870 tx=0x7fd1d000ed30 comp rx=0 tx=0).stop 2024-04-25T10:28:44.915 INFO:teuthology.orchestra.run.smithi203.stderr:2024-04-25T10:28:44.914+0000 7fd1dbf7d640 1 -- 172.21.15.203:0/1629792132 shutdown_connections 2024-04-25T10:28:44.915 INFO:teuthology.orchestra.run.smithi203.stderr:2024-04-25T10:28:44.914+0000 7fd1dbf7d640 1 --2- 172.21.15.203:0/1629792132 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fd1d415e420 0x7fd1d40c89c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:28:44.916 INFO:teuthology.orchestra.run.smithi203.stderr:2024-04-25T10:28:44.914+0000 7fd1dbf7d640 1 --2- 172.21.15.203:0/1629792132 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7fd1d40ce280 0x7fd1d4158fa0 unknown :-1 s=CLOSED pgs=9 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:28:44.916 INFO:teuthology.orchestra.run.smithi203.stderr:2024-04-25T10:28:44.914+0000 7fd1dbf7d640 1 --2- 172.21.15.203:0/1629792132 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fd1d40b26f0 0x7fd1d41f5430 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:28:44.916 INFO:teuthology.orchestra.run.smithi203.stderr:2024-04-25T10:28:44.914+0000 7fd1dbf7d640 1 -- 172.21.15.203:0/1629792132 >> 172.21.15.203:0/1629792132 conn(0x7fd1d409b700 msgr2=0x7fd1d415ad40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:28:44.916 INFO:teuthology.orchestra.run.smithi203.stderr:2024-04-25T10:28:44.914+0000 7fd1dbf7d640 1 -- 172.21.15.203:0/1629792132 shutdown_connections 2024-04-25T10:28:44.916 INFO:teuthology.orchestra.run.smithi203.stderr:2024-04-25T10:28:44.914+0000 7fd1dbf7d640 1 -- 172.21.15.203:0/1629792132 wait complete. 2024-04-25T10:28:44.926 DEBUG:teuthology.orchestra.run.smithi203:> sudo ceph --cluster ceph osd new 4ae7a0e9-cb3f-484c-a75f-f3766b721c47 2 2024-04-25T10:28:45.038 INFO:teuthology.orchestra.run.smithi203.stderr:2024-04-25T10:28:45.036+0000 7f8f94a0c640 1 Processor -- start 2024-04-25T10:28:45.038 INFO:teuthology.orchestra.run.smithi203.stderr:2024-04-25T10:28:45.036+0000 7f8f94a0c640 1 -- start start 2024-04-25T10:28:45.041 INFO:teuthology.orchestra.run.smithi203.stderr:2024-04-25T10:28:45.038+0000 7f8f94a0c640 1 --2- >> v2:172.21.15.37:3300/0 conn(0x7f8f900d20a0 0x7f8f900d31f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:28:45.041 INFO:teuthology.orchestra.run.smithi203.stderr:2024-04-25T10:28:45.038+0000 7f8f94a0c640 1 --2- >> v2:172.21.15.203:3300/0 conn(0x7f8f9009b700 0x7f8f9009bb10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:28:45.041 INFO:teuthology.orchestra.run.smithi203.stderr:2024-04-25T10:28:45.038+0000 7f8f94a0c640 1 -- --> v1:172.21.15.203:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f8f9009c050 con 0x7f8f9015e200 2024-04-25T10:28:45.041 INFO:teuthology.orchestra.run.smithi203.stderr:2024-04-25T10:28:45.038+0000 7f8f94a0c640 1 -- --> v2:172.21.15.37:3300/0 -- mon_getmap magic: 0 -- 0x7f8f9006f250 con 0x7f8f900d20a0 2024-04-25T10:28:45.041 INFO:teuthology.orchestra.run.smithi203.stderr:2024-04-25T10:28:45.038+0000 7f8f94a0c640 1 -- --> v2:172.21.15.203:3300/0 -- mon_getmap magic: 0 -- 0x7f8f9006f500 con 0x7f8f9009b700 2024-04-25T10:28:45.041 INFO:teuthology.orchestra.run.smithi203.stderr:2024-04-25T10:28:45.038+0000 7f8f8ed9d640 1 --2- >> v2:172.21.15.203:3300/0 conn(0x7f8f9009b700 0x7f8f9009bb10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:28:45.041 INFO:teuthology.orchestra.run.smithi203.stderr:2024-04-25T10:28:45.038+0000 7f8f8ed9d640 1 --2- >> v2:172.21.15.203:3300/0 conn(0x7f8f9009b700 0x7f8f9009bb10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.203:3300/0 says I am v2:172.21.15.203:44204/0 (socket says 172.21.15.203:44204) 2024-04-25T10:28:45.041 INFO:teuthology.orchestra.run.smithi203.stderr:2024-04-25T10:28:45.038+0000 7f8f8ed9d640 1 -- 172.21.15.203:0/1922420815 learned_addr learned my addr 172.21.15.203:0/1922420815 (peer_addr_for_me v2:172.21.15.203:0/0) 2024-04-25T10:28:45.041 INFO:teuthology.orchestra.run.smithi203.stderr:2024-04-25T10:28:45.038+0000 7f8f8dd9b640 1 --1- 172.21.15.203:0/1922420815 >> v1:172.21.15.203:6789/0 conn(0x7f8f9015e200 0x7f8f9006eb40 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.203:6789/0 says I am v1:172.21.15.203:45936/0 (socket says 172.21.15.203:45936) 2024-04-25T10:28:45.041 INFO:teuthology.orchestra.run.smithi203.stderr:2024-04-25T10:28:45.038+0000 7f8f8e59c640 1 --2- 172.21.15.203:0/1922420815 >> v2:172.21.15.37:3300/0 conn(0x7f8f900d20a0 0x7f8f900d31f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:28:45.041 INFO:teuthology.orchestra.run.smithi203.stderr:2024-04-25T10:28:45.038+0000 7f8f8ed9d640 1 -- 172.21.15.203:0/1922420815 >> v2:172.21.15.37:3300/0 conn(0x7f8f900d20a0 msgr2=0x7f8f900d31f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:28:45.041 INFO:teuthology.orchestra.run.smithi203.stderr:2024-04-25T10:28:45.038+0000 7f8f8ed9d640 1 --2- 172.21.15.203:0/1922420815 >> v2:172.21.15.37:3300/0 conn(0x7f8f900d20a0 0x7f8f900d31f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:28:45.042 INFO:teuthology.orchestra.run.smithi203.stderr:2024-04-25T10:28:45.038+0000 7f8f8ed9d640 1 -- 172.21.15.203:0/1922420815 >> v1:172.21.15.203:6789/0 conn(0x7f8f9015e200 legacy=0x7f8f9006eb40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:28:45.042 INFO:teuthology.orchestra.run.smithi203.stderr:2024-04-25T10:28:45.038+0000 7f8f8ed9d640 1 -- 172.21.15.203:0/1922420815 --> v2:172.21.15.203:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f8f9006f7e0 con 0x7f8f9009b700 2024-04-25T10:28:45.042 INFO:teuthology.orchestra.run.smithi203.stderr:2024-04-25T10:28:45.039+0000 7f8f8ed9d640 1 --2- 172.21.15.203:0/1922420815 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f8f9009b700 0x7f8f9009bb10 secure :-1 s=READY pgs=9 cs=0 l=1 rev1=1 crypto rx=0x7f8f9009c450 tx=0x7f8f8802a730 comp rx=0 tx=0).ready entity=mon.1 client_cookie=6bf7a4f76f020d09 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:28:45.042 INFO:teuthology.orchestra.run.smithi203.stderr:2024-04-25T10:28:45.039+0000 7f8f8d59a640 1 -- 172.21.15.203:0/1922420815 <== mon.1 v2:172.21.15.203:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8f88036070 con 0x7f8f9009b700 2024-04-25T10:28:45.042 INFO:teuthology.orchestra.run.smithi203.stderr:2024-04-25T10:28:45.039+0000 7f8f8d59a640 1 -- 172.21.15.203:0/1922420815 <== mon.1 v2:172.21.15.203:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f8f88031600 con 0x7f8f9009b700 2024-04-25T10:28:45.042 INFO:teuthology.orchestra.run.smithi203.stderr:2024-04-25T10:28:45.039+0000 7f8f8d59a640 1 -- 172.21.15.203:0/1922420815 <== mon.1 v2:172.21.15.203:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8f880319d0 con 0x7f8f9009b700 2024-04-25T10:28:45.043 INFO:teuthology.orchestra.run.smithi203.stderr:2024-04-25T10:28:45.040+0000 7f8f94a0c640 1 -- 172.21.15.203:0/1922420815 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f8f9009b700 msgr2=0x7f8f9009bb10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:28:45.043 INFO:teuthology.orchestra.run.smithi203.stderr:2024-04-25T10:28:45.040+0000 7f8f94a0c640 1 --2- 172.21.15.203:0/1922420815 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f8f9009b700 0x7f8f9009bb10 secure :-1 s=READY pgs=9 cs=0 l=1 rev1=1 crypto rx=0x7f8f9009c450 tx=0x7f8f8802a730 comp rx=0 tx=0).stop 2024-04-25T10:28:45.043 INFO:teuthology.orchestra.run.smithi203.stderr:2024-04-25T10:28:45.040+0000 7f8f94a0c640 1 -- 172.21.15.203:0/1922420815 shutdown_connections 2024-04-25T10:28:45.043 INFO:teuthology.orchestra.run.smithi203.stderr:2024-04-25T10:28:45.040+0000 7f8f94a0c640 1 --2- 172.21.15.203:0/1922420815 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f8f9009b700 0x7f8f9009bb10 unknown :-1 s=CLOSED pgs=9 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:28:45.043 INFO:teuthology.orchestra.run.smithi203.stderr:2024-04-25T10:28:45.040+0000 7f8f94a0c640 1 --2- 172.21.15.203:0/1922420815 >> v2:172.21.15.37:3300/0 conn(0x7f8f900d20a0 0x7f8f900d31f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:28:45.043 INFO:teuthology.orchestra.run.smithi203.stderr:2024-04-25T10:28:45.040+0000 7f8f94a0c640 1 -- 172.21.15.203:0/1922420815 >> 172.21.15.203:0/1922420815 conn(0x7f8f900b7620 msgr2=0x7f8f900b7c50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:28:45.043 INFO:teuthology.orchestra.run.smithi203.stderr:2024-04-25T10:28:45.040+0000 7f8f94a0c640 1 -- 172.21.15.203:0/1922420815 shutdown_connections 2024-04-25T10:28:45.043 INFO:teuthology.orchestra.run.smithi203.stderr:2024-04-25T10:28:45.040+0000 7f8f94a0c640 1 -- 172.21.15.203:0/1922420815 wait complete. 2024-04-25T10:28:45.044 INFO:teuthology.orchestra.run.smithi203.stderr:2024-04-25T10:28:45.042+0000 7f8f94a0c640 1 Processor -- start 2024-04-25T10:28:45.044 INFO:teuthology.orchestra.run.smithi203.stderr:2024-04-25T10:28:45.042+0000 7f8f94a0c640 1 -- start start 2024-04-25T10:28:45.046 INFO:teuthology.orchestra.run.smithi203.stderr:2024-04-25T10:28:45.044+0000 7f8f94a0c640 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f8f9009b700 0x7f8f901560e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:28:45.047 INFO:teuthology.orchestra.run.smithi203.stderr:2024-04-25T10:28:45.044+0000 7f8f94a0c640 1 --2- >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f8f900d20a0 0x7f8f901f5c70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:28:45.047 INFO:teuthology.orchestra.run.smithi203.stderr:2024-04-25T10:28:45.044+0000 7f8f94a0c640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f8f9015e200 0x7f8f901f4a60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:28:45.047 INFO:teuthology.orchestra.run.smithi203.stderr:2024-04-25T10:28:45.044+0000 7f8f94a0c640 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 -- 0x7f8f900d2cd0 con 0x7f8f9015e200 2024-04-25T10:28:45.047 INFO:teuthology.orchestra.run.smithi203.stderr:2024-04-25T10:28:45.044+0000 7f8f94a0c640 1 -- --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_getmap magic: 0 -- 0x7f8f900d29f0 con 0x7f8f9009b700 2024-04-25T10:28:45.047 INFO:teuthology.orchestra.run.smithi203.stderr:2024-04-25T10:28:45.044+0000 7f8f94a0c640 1 -- --> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] -- mon_getmap magic: 0 -- 0x7f8f900b28b0 con 0x7f8f900d20a0 2024-04-25T10:28:45.047 INFO:teuthology.orchestra.run.smithi203.stderr:2024-04-25T10:28:45.044+0000 7f8f8e59c640 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f8f9009b700 0x7f8f901560e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:28:45.047 INFO:teuthology.orchestra.run.smithi203.stderr:2024-04-25T10:28:45.044+0000 7f8f8dd9b640 1 --2- >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f8f900d20a0 0x7f8f901f5c70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:28:45.047 INFO:teuthology.orchestra.run.smithi203.stderr:2024-04-25T10:28:45.044+0000 7f8f8dd9b640 1 --2- >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f8f900d20a0 0x7f8f901f5c70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.203:3301/0 says I am v2:172.21.15.203:38896/0 (socket says 172.21.15.203:38896) 2024-04-25T10:28:45.047 INFO:teuthology.orchestra.run.smithi203.stderr:2024-04-25T10:28:45.044+0000 7f8f8e59c640 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f8f9009b700 0x7f8f901560e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.203:3300/0 says I am v2:172.21.15.203:44218/0 (socket says 172.21.15.203:44218) 2024-04-25T10:28:45.047 INFO:teuthology.orchestra.run.smithi203.stderr:2024-04-25T10:28:45.044+0000 7f8f8dd9b640 1 -- 172.21.15.203:0/1672635141 learned_addr learned my addr 172.21.15.203:0/1672635141 (peer_addr_for_me v2:172.21.15.203:0/0) 2024-04-25T10:28:45.047 INFO:teuthology.orchestra.run.smithi203.stderr:2024-04-25T10:28:45.044+0000 7f8f8ed9d640 1 --2- 172.21.15.203:0/1672635141 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f8f9015e200 0x7f8f901f4a60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:28:45.047 INFO:teuthology.orchestra.run.smithi203.stderr:2024-04-25T10:28:45.044+0000 7f8f8e59c640 1 -- 172.21.15.203:0/1672635141 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f8f900d20a0 msgr2=0x7f8f901f5c70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:28:45.047 INFO:teuthology.orchestra.run.smithi203.stderr:2024-04-25T10:28:45.044+0000 7f8f8e59c640 1 --2- 172.21.15.203:0/1672635141 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f8f900d20a0 0x7f8f901f5c70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:28:45.048 INFO:teuthology.orchestra.run.smithi203.stderr:2024-04-25T10:28:45.044+0000 7f8f8e59c640 1 -- 172.21.15.203:0/1672635141 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f8f9015e200 msgr2=0x7f8f901f4a60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:28:45.048 INFO:teuthology.orchestra.run.smithi203.stderr:2024-04-25T10:28:45.044+0000 7f8f8e59c640 1 --2- 172.21.15.203:0/1672635141 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f8f9015e200 0x7f8f901f4a60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:28:45.048 INFO:teuthology.orchestra.run.smithi203.stderr:2024-04-25T10:28:45.044+0000 7f8f8e59c640 1 -- 172.21.15.203:0/1672635141 --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f8f901f5700 con 0x7f8f9009b700 2024-04-25T10:28:45.048 INFO:teuthology.orchestra.run.smithi203.stderr:2024-04-25T10:28:45.044+0000 7f8f8e59c640 1 --2- 172.21.15.203:0/1672635141 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f8f9009b700 0x7f8f901560e0 secure :-1 s=READY pgs=10 cs=0 l=1 rev1=1 crypto rx=0x7f8f8000e870 tx=0x7f8f8000ed30 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:28:45.048 INFO:teuthology.orchestra.run.smithi203.stderr:2024-04-25T10:28:45.044+0000 7f8f7effd640 1 -- 172.21.15.203:0/1672635141 <== mon.1 v2:172.21.15.203:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8f80017070 con 0x7f8f9009b700 2024-04-25T10:28:45.048 INFO:teuthology.orchestra.run.smithi203.stderr:2024-04-25T10:28:45.044+0000 7f8f7effd640 1 -- 172.21.15.203:0/1672635141 <== mon.1 v2:172.21.15.203:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f8f8000ee60 con 0x7f8f9009b700 2024-04-25T10:28:45.048 INFO:teuthology.orchestra.run.smithi203.stderr:2024-04-25T10:28:45.044+0000 7f8f94a0c640 1 -- 172.21.15.203:0/1672635141 --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f8f901f59e0 con 0x7f8f9009b700 2024-04-25T10:28:45.048 INFO:teuthology.orchestra.run.smithi203.stderr:2024-04-25T10:28:45.044+0000 7f8f7effd640 1 -- 172.21.15.203:0/1672635141 <== mon.1 v2:172.21.15.203:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8f8000cb60 con 0x7f8f9009b700 2024-04-25T10:28:45.048 INFO:teuthology.orchestra.run.smithi203.stderr:2024-04-25T10:28:45.044+0000 7f8f94a0c640 1 -- 172.21.15.203:0/1672635141 --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f8f90158750 con 0x7f8f9009b700 2024-04-25T10:28:45.048 INFO:teuthology.orchestra.run.smithi203.stderr:2024-04-25T10:28:45.045+0000 7f8f7effd640 1 -- 172.21.15.203:0/1672635141 <== mon.1 v2:172.21.15.203:3300/0 4 ==== mgrmap(e 1) ==== 807+0+0 (secure 0 0 0) 0x7f8f8001a070 con 0x7f8f9009b700 2024-04-25T10:28:45.048 INFO:teuthology.orchestra.run.smithi203.stderr:2024-04-25T10:28:45.045+0000 7f8f7effd640 1 -- 172.21.15.203:0/1672635141 <== mon.1 v2:172.21.15.203:3300/0 5 ==== osd_map(4..4 src has 1..4) ==== 955+0+0 (secure 0 0 0) 0x7f8f8001c870 con 0x7f8f9009b700 2024-04-25T10:28:45.048 INFO:teuthology.orchestra.run.smithi203.stderr:2024-04-25T10:28:45.045+0000 7f8f94a0c640 1 -- 172.21.15.203:0/1672635141 --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f8f9006a140 con 0x7f8f9009b700 2024-04-25T10:28:45.049 INFO:teuthology.orchestra.run.smithi203.stderr:2024-04-25T10:28:45.048+0000 7f8f7effd640 1 -- 172.21.15.203:0/1672635141 <== mon.1 v2:172.21.15.203:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+73265 (secure 0 0 0) 0x7f8f8000cd00 con 0x7f8f9009b700 2024-04-25T10:28:45.120 INFO:teuthology.orchestra.run.smithi203.stderr:2024-04-25T10:28:45.119+0000 7f8f94a0c640 1 -- 172.21.15.203:0/1672635141 --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_command({"prefix": "osd new", "uuid": "4ae7a0e9-cb3f-484c-a75f-f3766b721c47", "id": 2} v 0) -- 0x7f8f901599c0 con 0x7f8f9009b700 2024-04-25T10:28:45.126 INFO:teuthology.orchestra.run.smithi203.stderr:2024-04-25T10:28:45.125+0000 7f8f7effd640 1 -- 172.21.15.203:0/1672635141 <== mon.1 v2:172.21.15.203:3300/0 7 ==== mon_command_ack([{"prefix": "osd new", "uuid": "4ae7a0e9-cb3f-484c-a75f-f3766b721c47", "id": 2}]=0 v5) ==== 112+0+2 (secure 0 0 0) 0x7f8f8001c520 con 0x7f8f9009b700 2024-04-25T10:28:45.127 INFO:teuthology.orchestra.run.smithi203.stdout:2 2024-04-25T10:28:45.128 INFO:teuthology.orchestra.run.smithi203.stderr:2024-04-25T10:28:45.126+0000 7f8f94a0c640 1 -- 172.21.15.203:0/1672635141 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f8f9009b700 msgr2=0x7f8f901560e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:28:45.128 INFO:teuthology.orchestra.run.smithi203.stderr:2024-04-25T10:28:45.126+0000 7f8f94a0c640 1 --2- 172.21.15.203:0/1672635141 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f8f9009b700 0x7f8f901560e0 secure :-1 s=READY pgs=10 cs=0 l=1 rev1=1 crypto rx=0x7f8f8000e870 tx=0x7f8f8000ed30 comp rx=0 tx=0).stop 2024-04-25T10:28:45.128 INFO:teuthology.orchestra.run.smithi203.stderr:2024-04-25T10:28:45.126+0000 7f8f94a0c640 1 -- 172.21.15.203:0/1672635141 shutdown_connections 2024-04-25T10:28:45.128 INFO:teuthology.orchestra.run.smithi203.stderr:2024-04-25T10:28:45.126+0000 7f8f94a0c640 1 --2- 172.21.15.203:0/1672635141 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f8f9015e200 0x7f8f901f4a60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:28:45.128 INFO:teuthology.orchestra.run.smithi203.stderr:2024-04-25T10:28:45.126+0000 7f8f94a0c640 1 --2- 172.21.15.203:0/1672635141 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f8f900d20a0 0x7f8f901f5c70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:28:45.128 INFO:teuthology.orchestra.run.smithi203.stderr:2024-04-25T10:28:45.126+0000 7f8f94a0c640 1 --2- 172.21.15.203:0/1672635141 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f8f9009b700 0x7f8f901560e0 unknown :-1 s=CLOSED pgs=10 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:28:45.128 INFO:teuthology.orchestra.run.smithi203.stderr:2024-04-25T10:28:45.126+0000 7f8f94a0c640 1 -- 172.21.15.203:0/1672635141 >> 172.21.15.203:0/1672635141 conn(0x7f8f900b7620 msgr2=0x7f8f900b2b50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:28:45.128 INFO:teuthology.orchestra.run.smithi203.stderr:2024-04-25T10:28:45.126+0000 7f8f94a0c640 1 -- 172.21.15.203:0/1672635141 shutdown_connections 2024-04-25T10:28:45.129 INFO:teuthology.orchestra.run.smithi203.stderr:2024-04-25T10:28:45.126+0000 7f8f94a0c640 1 -- 172.21.15.203:0/1672635141 wait complete. 2024-04-25T10:28:45.139 DEBUG:teuthology.orchestra.run.smithi203:> sudo ceph --cluster ceph osd new 1ba4a746-1608-47a7-ace2-960a5cfae934 3 2024-04-25T10:28:45.249 INFO:teuthology.orchestra.run.smithi203.stderr:2024-04-25T10:28:45.248+0000 7f88ed46e640 1 Processor -- start 2024-04-25T10:28:45.249 INFO:teuthology.orchestra.run.smithi203.stderr:2024-04-25T10:28:45.248+0000 7f88ed46e640 1 -- start start 2024-04-25T10:28:45.253 INFO:teuthology.orchestra.run.smithi203.stderr:2024-04-25T10:28:45.249+0000 7f88ed46e640 1 --2- >> v2:172.21.15.37:3300/0 conn(0x7f88e806d5c0 0x7f88e8160ba0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:28:45.254 INFO:teuthology.orchestra.run.smithi203.stderr:2024-04-25T10:28:45.249+0000 7f88ed46e640 1 -- --> v1:172.21.15.37:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f88e81614f0 con 0x7f88e81610e0 2024-04-25T10:28:45.254 INFO:teuthology.orchestra.run.smithi203.stderr:2024-04-25T10:28:45.249+0000 7f88ed46e640 1 -- --> v1:172.21.15.203:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f88e815dc10 con 0x7f88e815d5e0 2024-04-25T10:28:45.254 INFO:teuthology.orchestra.run.smithi203.stderr:2024-04-25T10:28:45.249+0000 7f88ed46e640 1 -- --> v2:172.21.15.37:3300/0 -- mon_getmap magic: 0 -- 0x7f88e815d9f0 con 0x7f88e806d5c0 2024-04-25T10:28:45.254 INFO:teuthology.orchestra.run.smithi203.stderr:2024-04-25T10:28:45.249+0000 7f88e6ffd640 1 --1- >> v1:172.21.15.203:6789/0 conn(0x7f88e815d5e0 0x7f88e81469c0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.203:6789/0 says I am v1:172.21.15.203:45942/0 (socket says 172.21.15.203:45942) 2024-04-25T10:28:45.254 INFO:teuthology.orchestra.run.smithi203.stderr:2024-04-25T10:28:45.249+0000 7f88e6ffd640 1 -- 172.21.15.203:0/2902258935 learned_addr learned my addr 172.21.15.203:0/2902258935 (peer_addr_for_me v1:172.21.15.203:0/0) 2024-04-25T10:28:45.254 INFO:teuthology.orchestra.run.smithi203.stderr:2024-04-25T10:28:45.249+0000 7f88e67fc640 1 --2- 172.21.15.203:0/2902258935 >> v2:172.21.15.37:3300/0 conn(0x7f88e806d5c0 0x7f88e8160ba0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:28:45.254 INFO:teuthology.orchestra.run.smithi203.stderr:2024-04-25T10:28:45.249+0000 7f88e5ffb640 1 -- 172.21.15.203:0/2902258935 <== mon.1 v1:172.21.15.203:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 4149844899 0 0) 0x7f88e815dc10 con 0x7f88e815d5e0 2024-04-25T10:28:45.254 INFO:teuthology.orchestra.run.smithi203.stderr:2024-04-25T10:28:45.249+0000 7f88e5ffb640 1 -- 172.21.15.203:0/2902258935 --> v1:172.21.15.203:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f88d0003520 con 0x7f88e815d5e0 2024-04-25T10:28:45.254 INFO:teuthology.orchestra.run.smithi203.stderr:2024-04-25T10:28:45.250+0000 7f88e5ffb640 1 -- 172.21.15.203:0/2902258935 <== mon.0 v1:172.21.15.37:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 737905163 0 0) 0x7f88e0001490 con 0x7f88e81610e0 2024-04-25T10:28:45.254 INFO:teuthology.orchestra.run.smithi203.stderr:2024-04-25T10:28:45.250+0000 7f88e5ffb640 1 -- 172.21.15.203:0/2902258935 --> v1:172.21.15.37:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f88e815dc10 con 0x7f88e81610e0 2024-04-25T10:28:45.254 INFO:teuthology.orchestra.run.smithi203.stderr:2024-04-25T10:28:45.250+0000 7f88e5ffb640 1 -- 172.21.15.203:0/2902258935 <== mon.1 v1:172.21.15.203:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 3575725114 0 0) 0x7f88d0003520 con 0x7f88e815d5e0 2024-04-25T10:28:45.254 INFO:teuthology.orchestra.run.smithi203.stderr:2024-04-25T10:28:45.250+0000 7f88e5ffb640 1 -- 172.21.15.203:0/2902258935 >> v2:172.21.15.37:3300/0 conn(0x7f88e806d5c0 msgr2=0x7f88e8160ba0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:28:45.254 INFO:teuthology.orchestra.run.smithi203.stderr:2024-04-25T10:28:45.250+0000 7f88e5ffb640 1 --2- 172.21.15.203:0/2902258935 >> v2:172.21.15.37:3300/0 conn(0x7f88e806d5c0 0x7f88e8160ba0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:28:45.255 INFO:teuthology.orchestra.run.smithi203.stderr:2024-04-25T10:28:45.250+0000 7f88e5ffb640 1 -- 172.21.15.203:0/2902258935 >> v1:172.21.15.37:6789/0 conn(0x7f88e81610e0 legacy=0x7f88e815ecb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:28:45.255 INFO:teuthology.orchestra.run.smithi203.stderr:2024-04-25T10:28:45.250+0000 7f88e5ffb640 1 -- 172.21.15.203:0/2902258935 --> v1:172.21.15.203:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f88e815def0 con 0x7f88e815d5e0 2024-04-25T10:28:45.255 INFO:teuthology.orchestra.run.smithi203.stderr:2024-04-25T10:28:45.250+0000 7f88e5ffb640 1 -- 172.21.15.203:0/2902258935 <== mon.1 v1:172.21.15.203:6789/0 3 ==== mon_map magic: 0 ==== 413+0+0 (unknown 1577794600 0 0) 0x7f88d8001670 con 0x7f88e815d5e0 2024-04-25T10:28:45.255 INFO:teuthology.orchestra.run.smithi203.stderr:2024-04-25T10:28:45.250+0000 7f88e5ffb640 1 -- 172.21.15.203:0/2902258935 >> v1:172.21.15.203:6789/0 conn(0x7f88e815d5e0 legacy=0x7f88e81469c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:28:45.255 INFO:teuthology.orchestra.run.smithi203.stderr:2024-04-25T10:28:45.250+0000 7f88e5ffb640 1 --2- 172.21.15.203:0/2902258935 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f88d0003b40 0x7f88d0003f90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:28:45.255 INFO:teuthology.orchestra.run.smithi203.stderr:2024-04-25T10:28:45.250+0000 7f88e5ffb640 1 --2- 172.21.15.203:0/2902258935 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f88d0004800 0x7f88d0004c10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:28:45.255 INFO:teuthology.orchestra.run.smithi203.stderr:2024-04-25T10:28:45.250+0000 7f88e5ffb640 1 --2- 172.21.15.203:0/2902258935 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f88d0005260 0x7f88d000b9c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:28:45.255 INFO:teuthology.orchestra.run.smithi203.stderr:2024-04-25T10:28:45.250+0000 7f88e5ffb640 1 -- 172.21.15.203:0/2902258935 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 -- 0x7f88e815d9f0 con 0x7f88d0004800 2024-04-25T10:28:45.255 INFO:teuthology.orchestra.run.smithi203.stderr:2024-04-25T10:28:45.250+0000 7f88e5ffb640 1 -- 172.21.15.203:0/2902258935 --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_getmap magic: 0 -- 0x7f88d000c1c0 con 0x7f88d0005260 2024-04-25T10:28:45.255 INFO:teuthology.orchestra.run.smithi203.stderr:2024-04-25T10:28:45.250+0000 7f88e5ffb640 1 -- 172.21.15.203:0/2902258935 --> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] -- mon_getmap magic: 0 -- 0x7f88d000c3c0 con 0x7f88d0003b40 2024-04-25T10:28:45.255 INFO:teuthology.orchestra.run.smithi203.stderr:2024-04-25T10:28:45.250+0000 7f88e67fc640 1 --2- 172.21.15.203:0/2902258935 >> v2:172.21.15.37:3300/0 conn(0x7f88e806d5c0 0x7f88e8160ba0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-25T10:28:45.255 INFO:teuthology.orchestra.run.smithi203.stderr:2024-04-25T10:28:45.250+0000 7f88e6ffd640 1 --2- 172.21.15.203:0/2902258935 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f88d0003b40 0x7f88d0003f90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:28:45.255 INFO:teuthology.orchestra.run.smithi203.stderr:2024-04-25T10:28:45.250+0000 7f88e77fe640 1 --2- 172.21.15.203:0/2902258935 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f88d0005260 0x7f88d000b9c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:28:45.255 INFO:teuthology.orchestra.run.smithi203.stderr:2024-04-25T10:28:45.250+0000 7f88e67fc640 1 --2- 172.21.15.203:0/2902258935 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f88d0004800 0x7f88d0004c10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:28:45.255 INFO:teuthology.orchestra.run.smithi203.stderr:2024-04-25T10:28:45.250+0000 7f88e6ffd640 1 -- 172.21.15.203:0/2902258935 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f88d0005260 msgr2=0x7f88d000b9c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:28:45.256 INFO:teuthology.orchestra.run.smithi203.stderr:2024-04-25T10:28:45.250+0000 7f88e6ffd640 1 --2- 172.21.15.203:0/2902258935 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f88d0005260 0x7f88d000b9c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:28:45.256 INFO:teuthology.orchestra.run.smithi203.stderr:2024-04-25T10:28:45.250+0000 7f88e6ffd640 1 -- 172.21.15.203:0/2902258935 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f88d0004800 msgr2=0x7f88d0004c10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:28:45.256 INFO:teuthology.orchestra.run.smithi203.stderr:2024-04-25T10:28:45.250+0000 7f88e6ffd640 1 --2- 172.21.15.203:0/2902258935 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f88d0004800 0x7f88d0004c10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:28:45.256 INFO:teuthology.orchestra.run.smithi203.stderr:2024-04-25T10:28:45.250+0000 7f88e6ffd640 1 -- 172.21.15.203:0/2902258935 --> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f88d0003520 con 0x7f88d0003b40 2024-04-25T10:28:45.256 INFO:teuthology.orchestra.run.smithi203.stderr:2024-04-25T10:28:45.250+0000 7f88e67fc640 1 --2- 172.21.15.203:0/2902258935 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f88d0004800 0x7f88d0004c10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T10:28:45.256 INFO:teuthology.orchestra.run.smithi203.stderr:2024-04-25T10:28:45.250+0000 7f88e77fe640 1 -- 172.21.15.203:0/2902258935 reap_dead start 2024-04-25T10:28:45.256 INFO:teuthology.orchestra.run.smithi203.stderr:2024-04-25T10:28:45.251+0000 7f88e6ffd640 1 --2- 172.21.15.203:0/2902258935 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f88d0003b40 0x7f88d0003f90 secure :-1 s=READY pgs=10 cs=0 l=1 rev1=1 crypto rx=0x7f88d8007fd0 tx=0x7f88d0005830 comp rx=0 tx=0).ready entity=mon.2 client_cookie=dd31d1d6eecb2087 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:28:45.256 INFO:teuthology.orchestra.run.smithi203.stderr:2024-04-25T10:28:45.251+0000 7f88e5ffb640 1 -- 172.21.15.203:0/2902258935 <== mon.2 v2:172.21.15.203:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f88d8007ca0 con 0x7f88d0003b40 2024-04-25T10:28:45.256 INFO:teuthology.orchestra.run.smithi203.stderr:2024-04-25T10:28:45.251+0000 7f88e5ffb640 1 -- 172.21.15.203:0/2902258935 <== mon.2 v2:172.21.15.203:3301/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f88d8005af0 con 0x7f88d0003b40 2024-04-25T10:28:45.256 INFO:teuthology.orchestra.run.smithi203.stderr:2024-04-25T10:28:45.251+0000 7f88e5ffb640 1 -- 172.21.15.203:0/2902258935 <== mon.2 v2:172.21.15.203:3301/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f88d8007070 con 0x7f88d0003b40 2024-04-25T10:28:45.256 INFO:teuthology.orchestra.run.smithi203.stderr:2024-04-25T10:28:45.252+0000 7f88ed46e640 1 -- 172.21.15.203:0/2902258935 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f88d0003b40 msgr2=0x7f88d0003f90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:28:45.256 INFO:teuthology.orchestra.run.smithi203.stderr:2024-04-25T10:28:45.252+0000 7f88ed46e640 1 --2- 172.21.15.203:0/2902258935 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f88d0003b40 0x7f88d0003f90 secure :-1 s=READY pgs=10 cs=0 l=1 rev1=1 crypto rx=0x7f88d8007fd0 tx=0x7f88d0005830 comp rx=0 tx=0).stop 2024-04-25T10:28:45.256 INFO:teuthology.orchestra.run.smithi203.stderr:2024-04-25T10:28:45.252+0000 7f88ed46e640 1 -- 172.21.15.203:0/2902258935 shutdown_connections 2024-04-25T10:28:45.257 INFO:teuthology.orchestra.run.smithi203.stderr:2024-04-25T10:28:45.252+0000 7f88ed46e640 1 --2- 172.21.15.203:0/2902258935 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f88d0003b40 0x7f88d0003f90 unknown :-1 s=CLOSED pgs=10 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:28:45.257 INFO:teuthology.orchestra.run.smithi203.stderr:2024-04-25T10:28:45.252+0000 7f88ed46e640 1 -- 172.21.15.203:0/2902258935 >> 172.21.15.203:0/2902258935 conn(0x7f88e8073b40 msgr2=0x7f88e80b6ff0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:28:45.257 INFO:teuthology.orchestra.run.smithi203.stderr:2024-04-25T10:28:45.252+0000 7f88ed46e640 1 -- 172.21.15.203:0/2902258935 shutdown_connections 2024-04-25T10:28:45.257 INFO:teuthology.orchestra.run.smithi203.stderr:2024-04-25T10:28:45.252+0000 7f88ed46e640 1 -- 172.21.15.203:0/2902258935 wait complete. 2024-04-25T10:28:45.259 INFO:teuthology.orchestra.run.smithi203.stderr:2024-04-25T10:28:45.254+0000 7f88ed46e640 1 Processor -- start 2024-04-25T10:28:45.259 INFO:teuthology.orchestra.run.smithi203.stderr:2024-04-25T10:28:45.254+0000 7f88ed46e640 1 -- start start 2024-04-25T10:28:45.259 INFO:teuthology.orchestra.run.smithi203.stderr:2024-04-25T10:28:45.255+0000 7f88ed46e640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f88d0003b40 0x7f88e8138ea0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:28:45.259 INFO:teuthology.orchestra.run.smithi203.stderr:2024-04-25T10:28:45.255+0000 7f88ed46e640 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f88e8143000 0x7f88e80d2060 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:28:45.259 INFO:teuthology.orchestra.run.smithi203.stderr:2024-04-25T10:28:45.255+0000 7f88ed46e640 1 --2- >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f88e80d25a0 0x7f88e813cb40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:28:45.260 INFO:teuthology.orchestra.run.smithi203.stderr:2024-04-25T10:28:45.255+0000 7f88ed46e640 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 -- 0x7f88e815e4b0 con 0x7f88d0003b40 2024-04-25T10:28:45.260 INFO:teuthology.orchestra.run.smithi203.stderr:2024-04-25T10:28:45.255+0000 7f88ed46e640 1 -- --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_getmap magic: 0 -- 0x7f88e815e820 con 0x7f88e8143000 2024-04-25T10:28:45.260 INFO:teuthology.orchestra.run.smithi203.stderr:2024-04-25T10:28:45.255+0000 7f88ed46e640 1 -- --> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] -- mon_getmap magic: 0 -- 0x7f88e806d9d0 con 0x7f88e80d25a0 2024-04-25T10:28:45.260 INFO:teuthology.orchestra.run.smithi203.stderr:2024-04-25T10:28:45.255+0000 7f88e77fe640 1 --2- >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f88e80d25a0 0x7f88e813cb40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:28:45.260 INFO:teuthology.orchestra.run.smithi203.stderr:2024-04-25T10:28:45.255+0000 7f88e77fe640 1 --2- >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f88e80d25a0 0x7f88e813cb40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.203:3301/0 says I am v2:172.21.15.203:38912/0 (socket says 172.21.15.203:38912) 2024-04-25T10:28:45.260 INFO:teuthology.orchestra.run.smithi203.stderr:2024-04-25T10:28:45.255+0000 7f88e77fe640 1 -- 172.21.15.203:0/2400963553 learned_addr learned my addr 172.21.15.203:0/2400963553 (peer_addr_for_me v2:172.21.15.203:0/0) 2024-04-25T10:28:45.260 INFO:teuthology.orchestra.run.smithi203.stderr:2024-04-25T10:28:45.255+0000 7f88e67fc640 1 --2- 172.21.15.203:0/2400963553 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f88e8143000 0x7f88e80d2060 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:28:45.260 INFO:teuthology.orchestra.run.smithi203.stderr:2024-04-25T10:28:45.255+0000 7f88e6ffd640 1 --2- 172.21.15.203:0/2400963553 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f88d0003b40 0x7f88e8138ea0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:28:45.260 INFO:teuthology.orchestra.run.smithi203.stderr:2024-04-25T10:28:45.256+0000 7f88e77fe640 1 -- 172.21.15.203:0/2400963553 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f88e8143000 msgr2=0x7f88e80d2060 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:28:45.260 INFO:teuthology.orchestra.run.smithi203.stderr:2024-04-25T10:28:45.256+0000 7f88e77fe640 1 --2- 172.21.15.203:0/2400963553 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f88e8143000 0x7f88e80d2060 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:28:45.260 INFO:teuthology.orchestra.run.smithi203.stderr:2024-04-25T10:28:45.256+0000 7f88e77fe640 1 -- 172.21.15.203:0/2400963553 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f88d0003b40 msgr2=0x7f88e8138ea0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:28:45.260 INFO:teuthology.orchestra.run.smithi203.stderr:2024-04-25T10:28:45.256+0000 7f88e77fe640 1 --2- 172.21.15.203:0/2400963553 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f88d0003b40 0x7f88e8138ea0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:28:45.260 INFO:teuthology.orchestra.run.smithi203.stderr:2024-04-25T10:28:45.256+0000 7f88e77fe640 1 -- 172.21.15.203:0/2400963553 --> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f88e80d29b0 con 0x7f88e80d25a0 2024-04-25T10:28:45.260 INFO:teuthology.orchestra.run.smithi203.stderr:2024-04-25T10:28:45.256+0000 7f88e77fe640 1 --2- 172.21.15.203:0/2400963553 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f88e80d25a0 0x7f88e813cb40 secure :-1 s=READY pgs=11 cs=0 l=1 rev1=1 crypto rx=0x7f88e80738a0 tx=0x7f88e000feb0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:28:45.261 INFO:teuthology.orchestra.run.smithi203.stderr:2024-04-25T10:28:45.256+0000 7f88cf7fe640 1 -- 172.21.15.203:0/2400963553 <== mon.2 v2:172.21.15.203:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f88e0019070 con 0x7f88e80d25a0 2024-04-25T10:28:45.261 INFO:teuthology.orchestra.run.smithi203.stderr:2024-04-25T10:28:45.256+0000 7f88cf7fe640 1 -- 172.21.15.203:0/2400963553 <== mon.2 v2:172.21.15.203:3301/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f88e0006d60 con 0x7f88e80d25a0 2024-04-25T10:28:45.261 INFO:teuthology.orchestra.run.smithi203.stderr:2024-04-25T10:28:45.256+0000 7f88ed46e640 1 -- 172.21.15.203:0/2400963553 --> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f88e81f7b30 con 0x7f88e80d25a0 2024-04-25T10:28:45.261 INFO:teuthology.orchestra.run.smithi203.stderr:2024-04-25T10:28:45.256+0000 7f88cf7fe640 1 -- 172.21.15.203:0/2400963553 <== mon.2 v2:172.21.15.203:3301/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f88e0003bb0 con 0x7f88e80d25a0 2024-04-25T10:28:45.261 INFO:teuthology.orchestra.run.smithi203.stderr:2024-04-25T10:28:45.256+0000 7f88ed46e640 1 -- 172.21.15.203:0/2400963553 --> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] -- mon_subscribe({osdmap=0}) -- 0x7f88e81f80f0 con 0x7f88e80d25a0 2024-04-25T10:28:45.261 INFO:teuthology.orchestra.run.smithi203.stderr:2024-04-25T10:28:45.256+0000 7f88cf7fe640 1 -- 172.21.15.203:0/2400963553 <== mon.2 v2:172.21.15.203:3301/0 4 ==== mgrmap(e 1) ==== 807+0+0 (secure 0 0 0) 0x7f88e001c070 con 0x7f88e80d25a0 2024-04-25T10:28:45.261 INFO:teuthology.orchestra.run.smithi203.stderr:2024-04-25T10:28:45.256+0000 7f88cf7fe640 1 -- 172.21.15.203:0/2400963553 <== mon.2 v2:172.21.15.203:3301/0 5 ==== osd_map(5..5 src has 1..5) ==== 1070+0+0 (secure 0 0 0) 0x7f88e000dbb0 con 0x7f88e80d25a0 2024-04-25T10:28:45.261 INFO:teuthology.orchestra.run.smithi203.stderr:2024-04-25T10:28:45.257+0000 7f88ed46e640 1 -- 172.21.15.203:0/2400963553 --> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f88e806a140 con 0x7f88e80d25a0 2024-04-25T10:28:45.261 INFO:teuthology.orchestra.run.smithi203.stderr:2024-04-25T10:28:45.259+0000 7f88cf7fe640 1 -- 172.21.15.203:0/2400963553 <== mon.2 v2:172.21.15.203:3301/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+73265 (secure 0 0 0) 0x7f88e80d29b0 con 0x7f88e80d25a0 2024-04-25T10:28:45.333 INFO:teuthology.orchestra.run.smithi203.stderr:2024-04-25T10:28:45.332+0000 7f88ed46e640 1 -- 172.21.15.203:0/2400963553 --> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] -- mon_command({"prefix": "osd new", "uuid": "1ba4a746-1608-47a7-ace2-960a5cfae934", "id": 3} v 0) -- 0x7f88e8074470 con 0x7f88e80d25a0 2024-04-25T10:28:45.339 INFO:teuthology.orchestra.run.smithi203.stderr:2024-04-25T10:28:45.337+0000 7f88cf7fe640 1 -- 172.21.15.203:0/2400963553 <== mon.2 v2:172.21.15.203:3301/0 7 ==== mon_command_ack([{"prefix": "osd new", "uuid": "1ba4a746-1608-47a7-ace2-960a5cfae934", "id": 3}]=0 v6) ==== 112+0+2 (secure 0 0 0) 0x7f88e81614f0 con 0x7f88e80d25a0 2024-04-25T10:28:45.339 INFO:teuthology.orchestra.run.smithi203.stdout:3 2024-04-25T10:28:45.340 INFO:teuthology.orchestra.run.smithi203.stderr:2024-04-25T10:28:45.338+0000 7f88ed46e640 1 -- 172.21.15.203:0/2400963553 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f88e80d25a0 msgr2=0x7f88e813cb40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:28:45.340 INFO:teuthology.orchestra.run.smithi203.stderr:2024-04-25T10:28:45.338+0000 7f88ed46e640 1 --2- 172.21.15.203:0/2400963553 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f88e80d25a0 0x7f88e813cb40 secure :-1 s=READY pgs=11 cs=0 l=1 rev1=1 crypto rx=0x7f88e80738a0 tx=0x7f88e000feb0 comp rx=0 tx=0).stop 2024-04-25T10:28:45.340 INFO:teuthology.orchestra.run.smithi203.stderr:2024-04-25T10:28:45.338+0000 7f88ed46e640 1 -- 172.21.15.203:0/2400963553 shutdown_connections 2024-04-25T10:28:45.340 INFO:teuthology.orchestra.run.smithi203.stderr:2024-04-25T10:28:45.338+0000 7f88ed46e640 1 --2- 172.21.15.203:0/2400963553 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f88e80d25a0 0x7f88e813cb40 unknown :-1 s=CLOSED pgs=11 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:28:45.340 INFO:teuthology.orchestra.run.smithi203.stderr:2024-04-25T10:28:45.338+0000 7f88ed46e640 1 --2- 172.21.15.203:0/2400963553 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f88e8143000 0x7f88e80d2060 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:28:45.340 INFO:teuthology.orchestra.run.smithi203.stderr:2024-04-25T10:28:45.338+0000 7f88ed46e640 1 --2- 172.21.15.203:0/2400963553 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f88d0003b40 0x7f88e8138ea0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:28:45.340 INFO:teuthology.orchestra.run.smithi203.stderr:2024-04-25T10:28:45.338+0000 7f88ed46e640 1 -- 172.21.15.203:0/2400963553 >> 172.21.15.203:0/2400963553 conn(0x7f88e8073b40 msgr2=0x7f88e8070d80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:28:45.340 INFO:teuthology.orchestra.run.smithi203.stderr:2024-04-25T10:28:45.339+0000 7f88ed46e640 1 -- 172.21.15.203:0/2400963553 shutdown_connections 2024-04-25T10:28:45.340 INFO:teuthology.orchestra.run.smithi203.stderr:2024-04-25T10:28:45.339+0000 7f88ed46e640 1 -- 172.21.15.203:0/2400963553 wait complete. 2024-04-25T10:28:45.351 DEBUG:teuthology.orchestra.run.smithi203:> sudo ceph --cluster ceph osd new d4f59f5d-0413-4b79-87ee-bb212c1fcf62 4 2024-04-25T10:28:45.376 INFO:tasks.ceph.mgr.x.smithi203.stderr:2024-04-25T10:28:45.375+0000 7fbc1bc32040 -1 mgr[py] Module telegraf has missing NOTIFY_TYPES member 2024-04-25T10:28:45.460 INFO:teuthology.orchestra.run.smithi203.stderr:2024-04-25T10:28:45.459+0000 7fd1a1420640 1 Processor -- start 2024-04-25T10:28:45.460 INFO:teuthology.orchestra.run.smithi203.stderr:2024-04-25T10:28:45.459+0000 7fd1a1420640 1 -- start start 2024-04-25T10:28:45.462 INFO:teuthology.orchestra.run.smithi203.stderr:2024-04-25T10:28:45.460+0000 7fd1a1420640 1 --2- >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7fd19c0b6e80 0x7fd19c071cf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:28:45.462 INFO:teuthology.orchestra.run.smithi203.stderr:2024-04-25T10:28:45.460+0000 7fd1a1420640 1 -- --> v1:172.21.15.37:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fd19c0d1320 con 0x7fd19c0718e0 2024-04-25T10:28:45.462 INFO:teuthology.orchestra.run.smithi203.stderr:2024-04-25T10:28:45.460+0000 7fd1a1420640 1 -- --> v1:172.21.15.203:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fd19c0d1600 con 0x7fd19c1603f0 2024-04-25T10:28:45.462 INFO:teuthology.orchestra.run.smithi203.stderr:2024-04-25T10:28:45.460+0000 7fd1a1420640 1 -- --> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] -- mon_getmap magic: 0 -- 0x7fd19c0d18e0 con 0x7fd19c0b6e80 2024-04-25T10:28:45.462 INFO:teuthology.orchestra.run.smithi203.stderr:2024-04-25T10:28:45.460+0000 7fd19affd640 1 --1- >> v1:172.21.15.203:6789/0 conn(0x7fd19c1603f0 0x7fd19c0d20e0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.203:6789/0 says I am v1:172.21.15.203:45952/0 (socket says 172.21.15.203:45952) 2024-04-25T10:28:45.462 INFO:teuthology.orchestra.run.smithi203.stderr:2024-04-25T10:28:45.460+0000 7fd19affd640 1 -- 172.21.15.203:0/853822023 learned_addr learned my addr 172.21.15.203:0/853822023 (peer_addr_for_me v1:172.21.15.203:0/0) 2024-04-25T10:28:45.462 INFO:teuthology.orchestra.run.smithi203.stderr:2024-04-25T10:28:45.460+0000 7fd19b7fe640 1 --2- >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7fd19c0b6e80 0x7fd19c071cf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:28:45.463 INFO:teuthology.orchestra.run.smithi203.stderr:2024-04-25T10:28:45.461+0000 7fd199ffb640 1 -- 172.21.15.203:0/853822023 <== mon.1 v1:172.21.15.203:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 412086513 0 0) 0x7fd19c0d1600 con 0x7fd19c1603f0 2024-04-25T10:28:45.463 INFO:teuthology.orchestra.run.smithi203.stderr:2024-04-25T10:28:45.461+0000 7fd199ffb640 1 -- 172.21.15.203:0/853822023 --> v1:172.21.15.203:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7fd184003520 con 0x7fd19c1603f0 2024-04-25T10:28:45.463 INFO:teuthology.orchestra.run.smithi203.stderr:2024-04-25T10:28:45.461+0000 7fd19b7fe640 1 -- 172.21.15.203:0/853822023 >> v1:172.21.15.203:6789/0 conn(0x7fd19c1603f0 legacy=0x7fd19c0d20e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:28:45.463 INFO:teuthology.orchestra.run.smithi203.stderr:2024-04-25T10:28:45.461+0000 7fd19b7fe640 1 -- 172.21.15.203:0/853822023 >> v1:172.21.15.37:6789/0 conn(0x7fd19c0718e0 legacy=0x7fd19c0b6770 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:28:45.463 INFO:teuthology.orchestra.run.smithi203.stderr:2024-04-25T10:28:45.461+0000 7fd19b7fe640 1 -- 172.21.15.203:0/853822023 --> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fd19c0d1bc0 con 0x7fd19c0b6e80 2024-04-25T10:28:45.463 INFO:teuthology.orchestra.run.smithi203.stderr:2024-04-25T10:28:45.462+0000 7fd19b7fe640 1 --2- 172.21.15.203:0/853822023 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7fd19c0b6e80 0x7fd19c071cf0 secure :-1 s=READY pgs=12 cs=0 l=1 rev1=1 crypto rx=0x7fd194009dc0 tx=0x7fd19402a6d0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=19c0b4f38e4d44f4 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:28:45.464 INFO:teuthology.orchestra.run.smithi203.stderr:2024-04-25T10:28:45.462+0000 7fd199ffb640 1 -- 172.21.15.203:0/853822023 <== mon.2 v2:172.21.15.203:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd194007740 con 0x7fd19c0b6e80 2024-04-25T10:28:45.464 INFO:teuthology.orchestra.run.smithi203.stderr:2024-04-25T10:28:45.462+0000 7fd199ffb640 1 -- 172.21.15.203:0/853822023 <== mon.2 v2:172.21.15.203:3301/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fd1940078e0 con 0x7fd19c0b6e80 2024-04-25T10:28:45.464 INFO:teuthology.orchestra.run.smithi203.stderr:2024-04-25T10:28:45.462+0000 7fd199ffb640 1 -- 172.21.15.203:0/853822023 <== mon.2 v2:172.21.15.203:3301/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd194007cb0 con 0x7fd19c0b6e80 2024-04-25T10:28:45.464 INFO:teuthology.orchestra.run.smithi203.stderr:2024-04-25T10:28:45.463+0000 7fd1a1420640 1 -- 172.21.15.203:0/853822023 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7fd19c0b6e80 msgr2=0x7fd19c071cf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:28:45.465 INFO:teuthology.orchestra.run.smithi203.stderr:2024-04-25T10:28:45.463+0000 7fd1a1420640 1 --2- 172.21.15.203:0/853822023 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7fd19c0b6e80 0x7fd19c071cf0 secure :-1 s=READY pgs=12 cs=0 l=1 rev1=1 crypto rx=0x7fd194009dc0 tx=0x7fd19402a6d0 comp rx=0 tx=0).stop 2024-04-25T10:28:45.465 INFO:teuthology.orchestra.run.smithi203.stderr:2024-04-25T10:28:45.463+0000 7fd1a1420640 1 -- 172.21.15.203:0/853822023 shutdown_connections 2024-04-25T10:28:45.465 INFO:teuthology.orchestra.run.smithi203.stderr:2024-04-25T10:28:45.463+0000 7fd1a1420640 1 --2- 172.21.15.203:0/853822023 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7fd19c0b6e80 0x7fd19c071cf0 unknown :-1 s=CLOSED pgs=12 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:28:45.465 INFO:teuthology.orchestra.run.smithi203.stderr:2024-04-25T10:28:45.463+0000 7fd1a1420640 1 -- 172.21.15.203:0/853822023 >> 172.21.15.203:0/853822023 conn(0x7fd19c0d2850 msgr2=0x7fd19c0d2c60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:28:45.465 INFO:teuthology.orchestra.run.smithi203.stderr:2024-04-25T10:28:45.463+0000 7fd1a1420640 1 -- 172.21.15.203:0/853822023 shutdown_connections 2024-04-25T10:28:45.465 INFO:teuthology.orchestra.run.smithi203.stderr:2024-04-25T10:28:45.463+0000 7fd1a1420640 1 -- 172.21.15.203:0/853822023 wait complete. 2024-04-25T10:28:45.466 INFO:teuthology.orchestra.run.smithi203.stderr:2024-04-25T10:28:45.465+0000 7fd1a1420640 1 Processor -- start 2024-04-25T10:28:45.466 INFO:teuthology.orchestra.run.smithi203.stderr:2024-04-25T10:28:45.465+0000 7fd1a1420640 1 -- start start 2024-04-25T10:28:45.469 INFO:teuthology.orchestra.run.smithi203.stderr:2024-04-25T10:28:45.466+0000 7fd1a1420640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fd19c0718e0 0x7fd19c146170 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:28:45.469 INFO:teuthology.orchestra.run.smithi203.stderr:2024-04-25T10:28:45.466+0000 7fd1a1420640 1 --2- >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7fd19c1603f0 0x7fd19c161f00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:28:45.469 INFO:teuthology.orchestra.run.smithi203.stderr:2024-04-25T10:28:45.466+0000 7fd1a1420640 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fd19c162440 0x7fd19c15f0d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:28:45.469 INFO:teuthology.orchestra.run.smithi203.stderr:2024-04-25T10:28:45.466+0000 7fd1a1420640 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 -- 0x7fd19c06ce60 con 0x7fd19c0718e0 2024-04-25T10:28:45.469 INFO:teuthology.orchestra.run.smithi203.stderr:2024-04-25T10:28:45.466+0000 7fd1a1420640 1 -- --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_getmap magic: 0 -- 0x7fd19c06cce0 con 0x7fd19c162440 2024-04-25T10:28:45.469 INFO:teuthology.orchestra.run.smithi203.stderr:2024-04-25T10:28:45.466+0000 7fd1a1420640 1 -- --> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] -- mon_getmap magic: 0 -- 0x7fd19c06cfe0 con 0x7fd19c1603f0 2024-04-25T10:28:45.469 INFO:teuthology.orchestra.run.smithi203.stderr:2024-04-25T10:28:45.466+0000 7fd19a7fc640 1 --2- >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7fd19c1603f0 0x7fd19c161f00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:28:45.469 INFO:teuthology.orchestra.run.smithi203.stderr:2024-04-25T10:28:45.466+0000 7fd19b7fe640 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fd19c162440 0x7fd19c15f0d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:28:45.469 INFO:teuthology.orchestra.run.smithi203.stderr:2024-04-25T10:28:45.466+0000 7fd19a7fc640 1 --2- >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7fd19c1603f0 0x7fd19c161f00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.203:3301/0 says I am v2:172.21.15.203:38936/0 (socket says 172.21.15.203:38936) 2024-04-25T10:28:45.469 INFO:teuthology.orchestra.run.smithi203.stderr:2024-04-25T10:28:45.466+0000 7fd19a7fc640 1 -- 172.21.15.203:0/1565271885 learned_addr learned my addr 172.21.15.203:0/1565271885 (peer_addr_for_me v2:172.21.15.203:0/0) 2024-04-25T10:28:45.469 INFO:teuthology.orchestra.run.smithi203.stderr:2024-04-25T10:28:45.466+0000 7fd19affd640 1 --2- 172.21.15.203:0/1565271885 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fd19c0718e0 0x7fd19c146170 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:28:45.469 INFO:teuthology.orchestra.run.smithi203.stderr:2024-04-25T10:28:45.466+0000 7fd19a7fc640 1 -- 172.21.15.203:0/1565271885 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fd19c162440 msgr2=0x7fd19c15f0d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:28:45.469 INFO:teuthology.orchestra.run.smithi203.stderr:2024-04-25T10:28:45.466+0000 7fd19a7fc640 1 --2- 172.21.15.203:0/1565271885 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fd19c162440 0x7fd19c15f0d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:28:45.470 INFO:teuthology.orchestra.run.smithi203.stderr:2024-04-25T10:28:45.466+0000 7fd19a7fc640 1 -- 172.21.15.203:0/1565271885 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fd19c0718e0 msgr2=0x7fd19c146170 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:28:45.470 INFO:teuthology.orchestra.run.smithi203.stderr:2024-04-25T10:28:45.466+0000 7fd19a7fc640 1 --2- 172.21.15.203:0/1565271885 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fd19c0718e0 0x7fd19c146170 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:28:45.470 INFO:teuthology.orchestra.run.smithi203.stderr:2024-04-25T10:28:45.466+0000 7fd19a7fc640 1 -- 172.21.15.203:0/1565271885 --> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fd19c150e90 con 0x7fd19c1603f0 2024-04-25T10:28:45.470 INFO:teuthology.orchestra.run.smithi203.stderr:2024-04-25T10:28:45.466+0000 7fd19b7fe640 1 --2- 172.21.15.203:0/1565271885 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fd19c162440 0x7fd19c15f0d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-25T10:28:45.470 INFO:teuthology.orchestra.run.smithi203.stderr:2024-04-25T10:28:45.467+0000 7fd19a7fc640 1 --2- 172.21.15.203:0/1565271885 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7fd19c1603f0 0x7fd19c161f00 secure :-1 s=READY pgs=13 cs=0 l=1 rev1=1 crypto rx=0x7fd19000c3c0 tx=0x7fd19000c880 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:28:45.470 INFO:teuthology.orchestra.run.smithi203.stderr:2024-04-25T10:28:45.467+0000 7fd1837fe640 1 -- 172.21.15.203:0/1565271885 <== mon.2 v2:172.21.15.203:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd1900052d0 con 0x7fd19c1603f0 2024-04-25T10:28:45.470 INFO:teuthology.orchestra.run.smithi203.stderr:2024-04-25T10:28:45.467+0000 7fd1837fe640 1 -- 172.21.15.203:0/1565271885 <== mon.2 v2:172.21.15.203:3301/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fd190005490 con 0x7fd19c1603f0 2024-04-25T10:28:45.470 INFO:teuthology.orchestra.run.smithi203.stderr:2024-04-25T10:28:45.467+0000 7fd1a1420640 1 -- 172.21.15.203:0/1565271885 --> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fd19c162b50 con 0x7fd19c1603f0 2024-04-25T10:28:45.470 INFO:teuthology.orchestra.run.smithi203.stderr:2024-04-25T10:28:45.467+0000 7fd1837fe640 1 -- 172.21.15.203:0/1565271885 <== mon.2 v2:172.21.15.203:3301/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd190007260 con 0x7fd19c1603f0 2024-04-25T10:28:45.470 INFO:teuthology.orchestra.run.smithi203.stderr:2024-04-25T10:28:45.467+0000 7fd1a1420640 1 -- 172.21.15.203:0/1565271885 --> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] -- mon_subscribe({osdmap=0}) -- 0x7fd19c151410 con 0x7fd19c1603f0 2024-04-25T10:28:45.470 INFO:teuthology.orchestra.run.smithi203.stderr:2024-04-25T10:28:45.467+0000 7fd1837fe640 1 -- 172.21.15.203:0/1565271885 <== mon.2 v2:172.21.15.203:3301/0 4 ==== mgrmap(e 1) ==== 807+0+0 (secure 0 0 0) 0x7fd190007400 con 0x7fd19c1603f0 2024-04-25T10:28:45.470 INFO:teuthology.orchestra.run.smithi203.stderr:2024-04-25T10:28:45.467+0000 7fd1837fe640 1 -- 172.21.15.203:0/1565271885 <== mon.2 v2:172.21.15.203:3301/0 5 ==== osd_map(6..6 src has 1..6) ==== 1185+0+0 (secure 0 0 0) 0x7fd190011590 con 0x7fd19c1603f0 2024-04-25T10:28:45.471 INFO:teuthology.orchestra.run.smithi203.stderr:2024-04-25T10:28:45.468+0000 7fd1a1420640 1 -- 172.21.15.203:0/1565271885 --> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fd19c1628c0 con 0x7fd19c1603f0 2024-04-25T10:28:45.471 INFO:tasks.ceph.mgr.x.smithi203.stderr:2024-04-25T10:28:45.470+0000 7fbc1bc32040 -1 mgr[py] Module osd_support has missing NOTIFY_TYPES member 2024-04-25T10:28:45.471 INFO:teuthology.orchestra.run.smithi203.stderr:2024-04-25T10:28:45.470+0000 7fd1837fe640 1 -- 172.21.15.203:0/1565271885 <== mon.2 v2:172.21.15.203:3301/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+73265 (secure 0 0 0) 0x7fd190003a90 con 0x7fd19c1603f0 2024-04-25T10:28:45.515 INFO:tasks.ceph.mgr.x.smithi203.stderr:2024-04-25T10:28:45.514+0000 7fbc1bc32040 -1 mgr[py] Module iostat has missing NOTIFY_TYPES member 2024-04-25T10:28:45.544 INFO:teuthology.orchestra.run.smithi203.stderr:2024-04-25T10:28:45.543+0000 7fd1a1420640 1 -- 172.21.15.203:0/1565271885 --> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] -- mon_command({"prefix": "osd new", "uuid": "d4f59f5d-0413-4b79-87ee-bb212c1fcf62", "id": 4} v 0) -- 0x7fd19c06a140 con 0x7fd19c1603f0 2024-04-25T10:28:45.548 INFO:teuthology.orchestra.run.smithi203.stderr:2024-04-25T10:28:45.547+0000 7fd1837fe640 1 -- 172.21.15.203:0/1565271885 <== mon.2 v2:172.21.15.203:3301/0 7 ==== mon_command_ack([{"prefix": "osd new", "uuid": "d4f59f5d-0413-4b79-87ee-bb212c1fcf62", "id": 4}]=0 v7) ==== 112+0+2 (secure 0 0 0) 0x7fd190003c70 con 0x7fd19c1603f0 2024-04-25T10:28:45.548 INFO:teuthology.orchestra.run.smithi203.stdout:4 2024-04-25T10:28:45.549 INFO:teuthology.orchestra.run.smithi203.stderr:2024-04-25T10:28:45.548+0000 7fd1a1420640 1 -- 172.21.15.203:0/1565271885 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7fd19c1603f0 msgr2=0x7fd19c161f00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:28:45.549 INFO:teuthology.orchestra.run.smithi203.stderr:2024-04-25T10:28:45.548+0000 7fd1a1420640 1 --2- 172.21.15.203:0/1565271885 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7fd19c1603f0 0x7fd19c161f00 secure :-1 s=READY pgs=13 cs=0 l=1 rev1=1 crypto rx=0x7fd19000c3c0 tx=0x7fd19000c880 comp rx=0 tx=0).stop 2024-04-25T10:28:45.550 INFO:teuthology.orchestra.run.smithi203.stderr:2024-04-25T10:28:45.548+0000 7fd1a1420640 1 -- 172.21.15.203:0/1565271885 shutdown_connections 2024-04-25T10:28:45.550 INFO:teuthology.orchestra.run.smithi203.stderr:2024-04-25T10:28:45.548+0000 7fd1a1420640 1 --2- 172.21.15.203:0/1565271885 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fd19c162440 0x7fd19c15f0d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:28:45.550 INFO:teuthology.orchestra.run.smithi203.stderr:2024-04-25T10:28:45.548+0000 7fd1a1420640 1 --2- 172.21.15.203:0/1565271885 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7fd19c1603f0 0x7fd19c161f00 unknown :-1 s=CLOSED pgs=13 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:28:45.550 INFO:teuthology.orchestra.run.smithi203.stderr:2024-04-25T10:28:45.548+0000 7fd1a1420640 1 --2- 172.21.15.203:0/1565271885 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fd19c0718e0 0x7fd19c146170 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:28:45.550 INFO:teuthology.orchestra.run.smithi203.stderr:2024-04-25T10:28:45.548+0000 7fd1a1420640 1 -- 172.21.15.203:0/1565271885 >> 172.21.15.203:0/1565271885 conn(0x7fd19c0d2850 msgr2=0x7fd19c159c90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:28:45.550 INFO:teuthology.orchestra.run.smithi203.stderr:2024-04-25T10:28:45.548+0000 7fd1a1420640 1 -- 172.21.15.203:0/1565271885 shutdown_connections 2024-04-25T10:28:45.550 INFO:teuthology.orchestra.run.smithi203.stderr:2024-04-25T10:28:45.548+0000 7fd1a1420640 1 -- 172.21.15.203:0/1565271885 wait complete. 2024-04-25T10:28:45.560 DEBUG:teuthology.orchestra.run.smithi203:> sudo ceph --cluster ceph osd new 69710a0e-41d8-4e7f-9a10-17ce540b67be 5 2024-04-25T10:28:45.575 INFO:tasks.ceph.mgr.y.smithi037.stderr:2024-04-25T10:28:45.573+0000 7f1c9c0f7040 -1 mgr[py] Module telegraf has missing NOTIFY_TYPES member 2024-04-25T10:28:45.666 INFO:tasks.ceph.mgr.y.smithi037.stderr:2024-04-25T10:28:45.664+0000 7f1c9c0f7040 -1 mgr[py] Module osd_support has missing NOTIFY_TYPES member 2024-04-25T10:28:45.690 INFO:tasks.ceph.mgr.x.smithi203.stderr:2024-04-25T10:28:45.689+0000 7fbc1bc32040 -1 mgr[py] Module alerts has missing NOTIFY_TYPES member 2024-04-25T10:28:45.709 INFO:teuthology.orchestra.run.smithi203.stderr:2024-04-25T10:28:45.707+0000 7f23f7ba8640 1 Processor -- start 2024-04-25T10:28:45.709 INFO:teuthology.orchestra.run.smithi203.stderr:2024-04-25T10:28:45.708+0000 7f23f7ba8640 1 -- start start 2024-04-25T10:28:45.710 INFO:tasks.ceph.mgr.y.smithi037.stderr:2024-04-25T10:28:45.708+0000 7f1c9c0f7040 -1 mgr[py] Module iostat has missing NOTIFY_TYPES member 2024-04-25T10:28:45.711 INFO:teuthology.orchestra.run.smithi203.stderr:2024-04-25T10:28:45.709+0000 7f23f7ba8640 1 --2- >> v2:172.21.15.203:3300/0 conn(0x7f23f006f200 0x7f23f015a8c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:28:45.711 INFO:teuthology.orchestra.run.smithi203.stderr:2024-04-25T10:28:45.709+0000 7f23f7ba8640 1 -- --> v1:172.21.15.37:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f23f0160f40 con 0x7f23f01607d0 2024-04-25T10:28:45.711 INFO:teuthology.orchestra.run.smithi203.stderr:2024-04-25T10:28:45.709+0000 7f23f7ba8640 1 -- --> v1:172.21.15.203:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f23f0161220 con 0x7f23f00d2e80 2024-04-25T10:28:45.711 INFO:teuthology.orchestra.run.smithi203.stderr:2024-04-25T10:28:45.709+0000 7f23f7ba8640 1 -- --> v2:172.21.15.203:3300/0 -- mon_getmap magic: 0 -- 0x7f23f0161500 con 0x7f23f006f200 2024-04-25T10:28:45.711 INFO:teuthology.orchestra.run.smithi203.stderr:2024-04-25T10:28:45.709+0000 7f23f5944640 1 --2- >> v2:172.21.15.203:3300/0 conn(0x7f23f006f200 0x7f23f015a8c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:28:45.712 INFO:teuthology.orchestra.run.smithi203.stderr:2024-04-25T10:28:45.709+0000 7f23f5944640 1 --2- >> v2:172.21.15.203:3300/0 conn(0x7f23f006f200 0x7f23f015a8c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.203:3300/0 says I am v2:172.21.15.203:44248/0 (socket says 172.21.15.203:44248) 2024-04-25T10:28:45.712 INFO:teuthology.orchestra.run.smithi203.stderr:2024-04-25T10:28:45.709+0000 7f23f5944640 1 -- 172.21.15.203:0/2179246832 learned_addr learned my addr 172.21.15.203:0/2179246832 (peer_addr_for_me v2:172.21.15.203:0/0) 2024-04-25T10:28:45.712 INFO:teuthology.orchestra.run.smithi203.stderr:2024-04-25T10:28:45.709+0000 7f23f6145640 1 --1- 172.21.15.203:0/2179246832 >> v1:172.21.15.203:6789/0 conn(0x7f23f00d2e80 0x7f23f009b700 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.203:6789/0 says I am v1:172.21.15.203:45954/0 (socket says 172.21.15.203:45954) 2024-04-25T10:28:45.712 INFO:teuthology.orchestra.run.smithi203.stderr:2024-04-25T10:28:45.710+0000 7f23f5944640 1 -- 172.21.15.203:0/2179246832 >> v1:172.21.15.203:6789/0 conn(0x7f23f00d2e80 legacy=0x7f23f009b700 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:28:45.712 INFO:teuthology.orchestra.run.smithi203.stderr:2024-04-25T10:28:45.710+0000 7f23f5944640 1 -- 172.21.15.203:0/2179246832 >> v1:172.21.15.37:6789/0 conn(0x7f23f01607d0 legacy=0x7f23f00b2530 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:28:45.712 INFO:teuthology.orchestra.run.smithi203.stderr:2024-04-25T10:28:45.710+0000 7f23f5944640 1 -- 172.21.15.203:0/2179246832 --> v2:172.21.15.203:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f23f00d2960 con 0x7f23f006f200 2024-04-25T10:28:45.712 INFO:teuthology.orchestra.run.smithi203.stderr:2024-04-25T10:28:45.710+0000 7f23f5944640 1 --2- 172.21.15.203:0/2179246832 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f23f006f200 0x7f23f015a8c0 secure :-1 s=READY pgs=14 cs=0 l=1 rev1=1 crypto rx=0x7f23e0009870 tx=0x7f23e002a300 comp rx=0 tx=0).ready entity=mon.1 client_cookie=7afb5ff52ec623dc server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:28:45.713 INFO:teuthology.orchestra.run.smithi203.stderr:2024-04-25T10:28:45.710+0000 7f23f4942640 1 -- 172.21.15.203:0/2179246832 <== mon.1 v2:172.21.15.203:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f23e0037070 con 0x7f23f006f200 2024-04-25T10:28:45.713 INFO:teuthology.orchestra.run.smithi203.stderr:2024-04-25T10:28:45.710+0000 7f23f4942640 1 -- 172.21.15.203:0/2179246832 <== mon.1 v2:172.21.15.203:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f23e002ae50 con 0x7f23f006f200 2024-04-25T10:28:45.713 INFO:teuthology.orchestra.run.smithi203.stderr:2024-04-25T10:28:45.710+0000 7f23f4942640 1 -- 172.21.15.203:0/2179246832 <== mon.1 v2:172.21.15.203:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f23e0032860 con 0x7f23f006f200 2024-04-25T10:28:45.713 INFO:teuthology.orchestra.run.smithi203.stderr:2024-04-25T10:28:45.711+0000 7f23f7ba8640 1 -- 172.21.15.203:0/2179246832 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f23f006f200 msgr2=0x7f23f015a8c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:28:45.713 INFO:teuthology.orchestra.run.smithi203.stderr:2024-04-25T10:28:45.711+0000 7f23f7ba8640 1 --2- 172.21.15.203:0/2179246832 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f23f006f200 0x7f23f015a8c0 secure :-1 s=READY pgs=14 cs=0 l=1 rev1=1 crypto rx=0x7f23e0009870 tx=0x7f23e002a300 comp rx=0 tx=0).stop 2024-04-25T10:28:45.713 INFO:teuthology.orchestra.run.smithi203.stderr:2024-04-25T10:28:45.711+0000 7f23f7ba8640 1 -- 172.21.15.203:0/2179246832 shutdown_connections 2024-04-25T10:28:45.713 INFO:teuthology.orchestra.run.smithi203.stderr:2024-04-25T10:28:45.711+0000 7f23f7ba8640 1 --2- 172.21.15.203:0/2179246832 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f23f006f200 0x7f23f015a8c0 unknown :-1 s=CLOSED pgs=14 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:28:45.714 INFO:teuthology.orchestra.run.smithi203.stderr:2024-04-25T10:28:45.711+0000 7f23f7ba8640 1 -- 172.21.15.203:0/2179246832 >> 172.21.15.203:0/2179246832 conn(0x7f23f00b7770 msgr2=0x7f23f00b7da0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:28:45.714 INFO:teuthology.orchestra.run.smithi203.stderr:2024-04-25T10:28:45.711+0000 7f23f7ba8640 1 -- 172.21.15.203:0/2179246832 shutdown_connections 2024-04-25T10:28:45.714 INFO:teuthology.orchestra.run.smithi203.stderr:2024-04-25T10:28:45.711+0000 7f23f7ba8640 1 -- 172.21.15.203:0/2179246832 wait complete. 2024-04-25T10:28:45.714 INFO:teuthology.orchestra.run.smithi203.stderr:2024-04-25T10:28:45.713+0000 7f23f7ba8640 1 Processor -- start 2024-04-25T10:28:45.714 INFO:teuthology.orchestra.run.smithi203.stderr:2024-04-25T10:28:45.713+0000 7f23f7ba8640 1 -- start start 2024-04-25T10:28:45.717 INFO:teuthology.orchestra.run.smithi203.stderr:2024-04-25T10:28:45.714+0000 7f23f7ba8640 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f23f006f200 0x7f23f0145640 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:28:45.717 INFO:teuthology.orchestra.run.smithi203.stderr:2024-04-25T10:28:45.714+0000 7f23f7ba8640 1 --2- >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f23f00d2e80 0x7f23f01ea550 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:28:45.717 INFO:teuthology.orchestra.run.smithi203.stderr:2024-04-25T10:28:45.714+0000 7f23f7ba8640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f23f01607d0 0x7f23f0159530 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:28:45.718 INFO:teuthology.orchestra.run.smithi203.stderr:2024-04-25T10:28:45.714+0000 7f23f7ba8640 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 -- 0x7f23f00b2c40 con 0x7f23f01607d0 2024-04-25T10:28:45.718 INFO:teuthology.orchestra.run.smithi203.stderr:2024-04-25T10:28:45.714+0000 7f23f7ba8640 1 -- --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_getmap magic: 0 -- 0x7f23f009be10 con 0x7f23f006f200 2024-04-25T10:28:45.718 INFO:teuthology.orchestra.run.smithi203.stderr:2024-04-25T10:28:45.714+0000 7f23f7ba8640 1 -- --> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] -- mon_getmap magic: 0 -- 0x7f23f0145b80 con 0x7f23f00d2e80 2024-04-25T10:28:45.718 INFO:teuthology.orchestra.run.smithi203.stderr:2024-04-25T10:28:45.715+0000 7f23f5944640 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f23f006f200 0x7f23f0145640 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:28:45.718 INFO:teuthology.orchestra.run.smithi203.stderr:2024-04-25T10:28:45.715+0000 7f23f5944640 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f23f006f200 0x7f23f0145640 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.203:3300/0 says I am v2:172.21.15.203:44256/0 (socket says 172.21.15.203:44256) 2024-04-25T10:28:45.718 INFO:teuthology.orchestra.run.smithi203.stderr:2024-04-25T10:28:45.715+0000 7f23f5944640 1 -- 172.21.15.203:0/3705611769 learned_addr learned my addr 172.21.15.203:0/3705611769 (peer_addr_for_me v2:172.21.15.203:0/0) 2024-04-25T10:28:45.718 INFO:teuthology.orchestra.run.smithi203.stderr:2024-04-25T10:28:45.715+0000 7f23f5143640 1 --2- 172.21.15.203:0/3705611769 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f23f00d2e80 0x7f23f01ea550 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:28:45.718 INFO:teuthology.orchestra.run.smithi203.stderr:2024-04-25T10:28:45.715+0000 7f23f6145640 1 --2- 172.21.15.203:0/3705611769 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f23f01607d0 0x7f23f0159530 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:28:45.718 INFO:teuthology.orchestra.run.smithi203.stderr:2024-04-25T10:28:45.715+0000 7f23f5944640 1 -- 172.21.15.203:0/3705611769 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f23f00d2e80 msgr2=0x7f23f01ea550 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:28:45.718 INFO:teuthology.orchestra.run.smithi203.stderr:2024-04-25T10:28:45.715+0000 7f23f5944640 1 --2- 172.21.15.203:0/3705611769 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f23f00d2e80 0x7f23f01ea550 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:28:45.718 INFO:teuthology.orchestra.run.smithi203.stderr:2024-04-25T10:28:45.715+0000 7f23f5944640 1 -- 172.21.15.203:0/3705611769 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f23f01607d0 msgr2=0x7f23f0159530 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:28:45.718 INFO:teuthology.orchestra.run.smithi203.stderr:2024-04-25T10:28:45.715+0000 7f23f5944640 1 --2- 172.21.15.203:0/3705611769 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f23f01607d0 0x7f23f0159530 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:28:45.718 INFO:teuthology.orchestra.run.smithi203.stderr:2024-04-25T10:28:45.715+0000 7f23f5944640 1 -- 172.21.15.203:0/3705611769 --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f23f009ddf0 con 0x7f23f006f200 2024-04-25T10:28:45.718 INFO:teuthology.orchestra.run.smithi203.stderr:2024-04-25T10:28:45.715+0000 7f23f5944640 1 --2- 172.21.15.203:0/3705611769 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f23f006f200 0x7f23f0145640 secure :-1 s=READY pgs=15 cs=0 l=1 rev1=1 crypto rx=0x7f23e0000c00 tx=0x7f23e00046c0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:28:45.719 INFO:teuthology.orchestra.run.smithi203.stderr:2024-04-25T10:28:45.715+0000 7f23e67fc640 1 -- 172.21.15.203:0/3705611769 <== mon.1 v2:172.21.15.203:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f23e0041070 con 0x7f23f006f200 2024-04-25T10:28:45.719 INFO:teuthology.orchestra.run.smithi203.stderr:2024-04-25T10:28:45.715+0000 7f23e67fc640 1 -- 172.21.15.203:0/3705611769 <== mon.1 v2:172.21.15.203:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f23e003c890 con 0x7f23f006f200 2024-04-25T10:28:45.719 INFO:teuthology.orchestra.run.smithi203.stderr:2024-04-25T10:28:45.715+0000 7f23e67fc640 1 -- 172.21.15.203:0/3705611769 <== mon.1 v2:172.21.15.203:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f23e0037040 con 0x7f23f006f200 2024-04-25T10:28:45.719 INFO:teuthology.orchestra.run.smithi203.stderr:2024-04-25T10:28:45.715+0000 7f23f7ba8640 1 -- 172.21.15.203:0/3705611769 --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f23f01576d0 con 0x7f23f006f200 2024-04-25T10:28:45.719 INFO:teuthology.orchestra.run.smithi203.stderr:2024-04-25T10:28:45.715+0000 7f23f7ba8640 1 -- 172.21.15.203:0/3705611769 --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f23f009d030 con 0x7f23f006f200 2024-04-25T10:28:45.719 INFO:teuthology.orchestra.run.smithi203.stderr:2024-04-25T10:28:45.716+0000 7f23e67fc640 1 -- 172.21.15.203:0/3705611769 <== mon.1 v2:172.21.15.203:3300/0 4 ==== mgrmap(e 1) ==== 807+0+0 (secure 0 0 0) 0x7f23e0044070 con 0x7f23f006f200 2024-04-25T10:28:45.719 INFO:teuthology.orchestra.run.smithi203.stderr:2024-04-25T10:28:45.716+0000 7f23e67fc640 1 -- 172.21.15.203:0/3705611769 <== mon.1 v2:172.21.15.203:3300/0 5 ==== osd_map(7..7 src has 1..7) ==== 1300+0+0 (secure 0 0 0) 0x7f23e0046b30 con 0x7f23f006f200 2024-04-25T10:28:45.719 INFO:teuthology.orchestra.run.smithi203.stderr:2024-04-25T10:28:45.716+0000 7f23f7ba8640 1 -- 172.21.15.203:0/3705611769 --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f23f006a140 con 0x7f23f006f200 2024-04-25T10:28:45.719 INFO:teuthology.orchestra.run.smithi203.stderr:2024-04-25T10:28:45.718+0000 7f23e67fc640 1 -- 172.21.15.203:0/3705611769 <== mon.1 v2:172.21.15.203:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+73265 (secure 0 0 0) 0x7f23e003cc80 con 0x7f23f006f200 2024-04-25T10:28:45.741 INFO:tasks.ceph.mgr.x.smithi203.stderr:2024-04-25T10:28:45.739+0000 7fbc1bc32040 -1 mgr[py] Module selftest has missing NOTIFY_TYPES member 2024-04-25T10:28:45.793 INFO:teuthology.orchestra.run.smithi203.stderr:2024-04-25T10:28:45.791+0000 7f23f7ba8640 1 -- 172.21.15.203:0/3705611769 --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_command({"prefix": "osd new", "uuid": "69710a0e-41d8-4e7f-9a10-17ce540b67be", "id": 5} v 0) -- 0x7f23f006f900 con 0x7f23f006f200 2024-04-25T10:28:45.795 INFO:tasks.ceph.mgr.x.smithi203.stderr:2024-04-25T10:28:45.794+0000 7fbc1bc32040 -1 mgr[py] Module pg_autoscaler has missing NOTIFY_TYPES member 2024-04-25T10:28:45.799 INFO:teuthology.orchestra.run.smithi203.stderr:2024-04-25T10:28:45.797+0000 7f23e67fc640 1 -- 172.21.15.203:0/3705611769 <== mon.1 v2:172.21.15.203:3300/0 7 ==== mon_command_ack([{"prefix": "osd new", "uuid": "69710a0e-41d8-4e7f-9a10-17ce540b67be", "id": 5}]=0 v8) ==== 112+0+2 (secure 0 0 0) 0x7f23e0046450 con 0x7f23f006f200 2024-04-25T10:28:45.799 INFO:teuthology.orchestra.run.smithi203.stdout:5 2024-04-25T10:28:45.800 INFO:teuthology.orchestra.run.smithi203.stderr:2024-04-25T10:28:45.798+0000 7f23f7ba8640 1 -- 172.21.15.203:0/3705611769 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f23f006f200 msgr2=0x7f23f0145640 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:28:45.800 INFO:teuthology.orchestra.run.smithi203.stderr:2024-04-25T10:28:45.798+0000 7f23f7ba8640 1 --2- 172.21.15.203:0/3705611769 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f23f006f200 0x7f23f0145640 secure :-1 s=READY pgs=15 cs=0 l=1 rev1=1 crypto rx=0x7f23e0000c00 tx=0x7f23e00046c0 comp rx=0 tx=0).stop 2024-04-25T10:28:45.800 INFO:teuthology.orchestra.run.smithi203.stderr:2024-04-25T10:28:45.798+0000 7f23f7ba8640 1 -- 172.21.15.203:0/3705611769 shutdown_connections 2024-04-25T10:28:45.800 INFO:teuthology.orchestra.run.smithi203.stderr:2024-04-25T10:28:45.798+0000 7f23f7ba8640 1 --2- 172.21.15.203:0/3705611769 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f23f01607d0 0x7f23f0159530 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:28:45.800 INFO:teuthology.orchestra.run.smithi203.stderr:2024-04-25T10:28:45.798+0000 7f23f7ba8640 1 --2- 172.21.15.203:0/3705611769 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f23f00d2e80 0x7f23f01ea550 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:28:45.800 INFO:teuthology.orchestra.run.smithi203.stderr:2024-04-25T10:28:45.798+0000 7f23f7ba8640 1 --2- 172.21.15.203:0/3705611769 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f23f006f200 0x7f23f0145640 unknown :-1 s=CLOSED pgs=15 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:28:45.800 INFO:teuthology.orchestra.run.smithi203.stderr:2024-04-25T10:28:45.798+0000 7f23f7ba8640 1 -- 172.21.15.203:0/3705611769 >> 172.21.15.203:0/3705611769 conn(0x7f23f00b7770 msgr2=0x7f23f00708c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:28:45.800 INFO:teuthology.orchestra.run.smithi203.stderr:2024-04-25T10:28:45.798+0000 7f23f7ba8640 1 -- 172.21.15.203:0/3705611769 shutdown_connections 2024-04-25T10:28:45.800 INFO:teuthology.orchestra.run.smithi203.stderr:2024-04-25T10:28:45.798+0000 7f23f7ba8640 1 -- 172.21.15.203:0/3705611769 wait complete. 2024-04-25T10:28:45.811 DEBUG:teuthology.orchestra.run.smithi203:> sudo ceph --cluster ceph osd new f29dd047-3b48-473d-a235-6642a6293f20 6 2024-04-25T10:28:45.884 INFO:tasks.ceph.mgr.y.smithi037.stderr:2024-04-25T10:28:45.881+0000 7f1c9c0f7040 -1 mgr[py] Module alerts has missing NOTIFY_TYPES member 2024-04-25T10:28:45.917 INFO:tasks.ceph.mgr.x.smithi203.stderr:2024-04-25T10:28:45.916+0000 7fbc1bc32040 -1 mgr[py] Module test_orchestrator has missing NOTIFY_TYPES member 2024-04-25T10:28:45.919 INFO:teuthology.orchestra.run.smithi203.stderr:2024-04-25T10:28:45.918+0000 7f3903667640 1 Processor -- start 2024-04-25T10:28:45.920 INFO:teuthology.orchestra.run.smithi203.stderr:2024-04-25T10:28:45.918+0000 7f3903667640 1 -- start start 2024-04-25T10:28:45.922 INFO:teuthology.orchestra.run.smithi203.stderr:2024-04-25T10:28:45.920+0000 7f3903667640 1 --2- >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f38fc15d520 0x7f38fc0d2e90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:28:45.923 INFO:teuthology.orchestra.run.smithi203.stderr:2024-04-25T10:28:45.920+0000 7f3903667640 1 --2- >> v2:172.21.15.37:3300/0 conn(0x7f38fc0d33d0 0x7f38fc0b2840 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:28:45.923 INFO:teuthology.orchestra.run.smithi203.stderr:2024-04-25T10:28:45.920+0000 7f3903667640 1 -- --> v1:172.21.15.37:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f38fc0d2700 con 0x7f38fc0b1d70 2024-04-25T10:28:45.923 INFO:teuthology.orchestra.run.smithi203.stderr:2024-04-25T10:28:45.920+0000 7f3903667640 1 -- --> v2:172.21.15.37:3300/0 -- mon_getmap magic: 0 -- 0x7f38fc15aef0 con 0x7f38fc0d33d0 2024-04-25T10:28:45.923 INFO:teuthology.orchestra.run.smithi203.stderr:2024-04-25T10:28:45.920+0000 7f3903667640 1 -- --> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] -- mon_getmap magic: 0 -- 0x7f38fc161360 con 0x7f38fc15d520 2024-04-25T10:28:45.923 INFO:teuthology.orchestra.run.smithi203.stderr:2024-04-25T10:28:45.920+0000 7f3900c02640 1 --2- >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f38fc15d520 0x7f38fc0d2e90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:28:45.923 INFO:teuthology.orchestra.run.smithi203.stderr:2024-04-25T10:28:45.920+0000 7f3900c02640 1 --2- >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f38fc15d520 0x7f38fc0d2e90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.203:3301/0 says I am v2:172.21.15.203:38950/0 (socket says 172.21.15.203:38950) 2024-04-25T10:28:45.923 INFO:teuthology.orchestra.run.smithi203.stderr:2024-04-25T10:28:45.920+0000 7f3900c02640 1 -- 172.21.15.203:0/2665499716 learned_addr learned my addr 172.21.15.203:0/2665499716 (peer_addr_for_me v2:172.21.15.203:0/0) 2024-04-25T10:28:45.923 INFO:teuthology.orchestra.run.smithi203.stderr:2024-04-25T10:28:45.920+0000 7f3901c04640 1 --2- 172.21.15.203:0/2665499716 >> v2:172.21.15.37:3300/0 conn(0x7f38fc0d33d0 0x7f38fc0b2840 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:28:45.923 INFO:teuthology.orchestra.run.smithi203.stderr:2024-04-25T10:28:45.920+0000 7f3900c02640 1 -- 172.21.15.203:0/2665499716 >> v2:172.21.15.37:3300/0 conn(0x7f38fc0d33d0 msgr2=0x7f38fc0b2840 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:28:45.923 INFO:teuthology.orchestra.run.smithi203.stderr:2024-04-25T10:28:45.920+0000 7f3900c02640 1 --2- 172.21.15.203:0/2665499716 >> v2:172.21.15.37:3300/0 conn(0x7f38fc0d33d0 0x7f38fc0b2840 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:28:45.923 INFO:teuthology.orchestra.run.smithi203.stderr:2024-04-25T10:28:45.920+0000 7f3900c02640 1 -- 172.21.15.203:0/2665499716 >> v1:172.21.15.37:6789/0 conn(0x7f38fc0b1d70 legacy=0x7f38fc160b10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:28:45.923 INFO:teuthology.orchestra.run.smithi203.stderr:2024-04-25T10:28:45.920+0000 7f3900c02640 1 -- 172.21.15.203:0/2665499716 --> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f38fc161640 con 0x7f38fc15d520 2024-04-25T10:28:45.923 INFO:teuthology.orchestra.run.smithi203.stderr:2024-04-25T10:28:45.920+0000 7f3901c04640 1 --2- 172.21.15.203:0/2665499716 >> v2:172.21.15.37:3300/0 conn(0x7f38fc0d33d0 0x7f38fc0b2840 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T10:28:45.924 INFO:teuthology.orchestra.run.smithi203.stderr:2024-04-25T10:28:45.921+0000 7f3900c02640 1 --2- 172.21.15.203:0/2665499716 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f38fc15d520 0x7f38fc0d2e90 secure :-1 s=READY pgs=14 cs=0 l=1 rev1=1 crypto rx=0x7f38f8009870 tx=0x7f38f802a270 comp rx=0 tx=0).ready entity=mon.2 client_cookie=619f0bf51b198033 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:28:45.924 INFO:teuthology.orchestra.run.smithi203.stderr:2024-04-25T10:28:45.921+0000 7f38f3fff640 1 -- 172.21.15.203:0/2665499716 <== mon.2 v2:172.21.15.203:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f38f8037070 con 0x7f38fc15d520 2024-04-25T10:28:45.924 INFO:teuthology.orchestra.run.smithi203.stderr:2024-04-25T10:28:45.921+0000 7f38f3fff640 1 -- 172.21.15.203:0/2665499716 <== mon.2 v2:172.21.15.203:3301/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f38f8032440 con 0x7f38fc15d520 2024-04-25T10:28:45.924 INFO:teuthology.orchestra.run.smithi203.stderr:2024-04-25T10:28:45.921+0000 7f38f3fff640 1 -- 172.21.15.203:0/2665499716 <== mon.2 v2:172.21.15.203:3301/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f38f8032810 con 0x7f38fc15d520 2024-04-25T10:28:45.925 INFO:teuthology.orchestra.run.smithi203.stderr:2024-04-25T10:28:45.922+0000 7f3903667640 1 -- 172.21.15.203:0/2665499716 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f38fc15d520 msgr2=0x7f38fc0d2e90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:28:45.925 INFO:teuthology.orchestra.run.smithi203.stderr:2024-04-25T10:28:45.922+0000 7f3903667640 1 --2- 172.21.15.203:0/2665499716 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f38fc15d520 0x7f38fc0d2e90 secure :-1 s=READY pgs=14 cs=0 l=1 rev1=1 crypto rx=0x7f38f8009870 tx=0x7f38f802a270 comp rx=0 tx=0).stop 2024-04-25T10:28:45.925 INFO:teuthology.orchestra.run.smithi203.stderr:2024-04-25T10:28:45.922+0000 7f3903667640 1 -- 172.21.15.203:0/2665499716 shutdown_connections 2024-04-25T10:28:45.925 INFO:teuthology.orchestra.run.smithi203.stderr:2024-04-25T10:28:45.922+0000 7f3903667640 1 --2- 172.21.15.203:0/2665499716 >> v2:172.21.15.37:3300/0 conn(0x7f38fc0d33d0 0x7f38fc0b2840 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:28:45.925 INFO:teuthology.orchestra.run.smithi203.stderr:2024-04-25T10:28:45.922+0000 7f3903667640 1 --2- 172.21.15.203:0/2665499716 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f38fc15d520 0x7f38fc0d2e90 unknown :-1 s=CLOSED pgs=14 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:28:45.925 INFO:teuthology.orchestra.run.smithi203.stderr:2024-04-25T10:28:45.922+0000 7f3903667640 1 -- 172.21.15.203:0/2665499716 >> 172.21.15.203:0/2665499716 conn(0x7f38fc0b7620 msgr2=0x7f38fc0b7c50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:28:45.925 INFO:teuthology.orchestra.run.smithi203.stderr:2024-04-25T10:28:45.922+0000 7f3903667640 1 -- 172.21.15.203:0/2665499716 shutdown_connections 2024-04-25T10:28:45.925 INFO:teuthology.orchestra.run.smithi203.stderr:2024-04-25T10:28:45.922+0000 7f3903667640 1 -- 172.21.15.203:0/2665499716 wait complete. 2024-04-25T10:28:45.926 INFO:teuthology.orchestra.run.smithi203.stderr:2024-04-25T10:28:45.924+0000 7f3903667640 1 Processor -- start 2024-04-25T10:28:45.926 INFO:teuthology.orchestra.run.smithi203.stderr:2024-04-25T10:28:45.924+0000 7f3903667640 1 -- start start 2024-04-25T10:28:45.928 INFO:teuthology.orchestra.run.smithi203.stderr:2024-04-25T10:28:45.925+0000 7f3903667640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f38fc0b1d70 0x7f38fc1517b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:28:45.928 INFO:teuthology.orchestra.run.smithi203.stderr:2024-04-25T10:28:45.925+0000 7f3903667640 1 --2- >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f38fc0d33d0 0x7f38fc0d1650 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:28:45.928 INFO:teuthology.orchestra.run.smithi203.stderr:2024-04-25T10:28:45.925+0000 7f3903667640 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f38fc15d520 0x7f38fc06cdd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:28:45.929 INFO:teuthology.orchestra.run.smithi203.stderr:2024-04-25T10:28:45.925+0000 7f3903667640 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 -- 0x7f38fc0b2d80 con 0x7f38fc0b1d70 2024-04-25T10:28:45.929 INFO:teuthology.orchestra.run.smithi203.stderr:2024-04-25T10:28:45.925+0000 7f3903667640 1 -- --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_getmap magic: 0 -- 0x7f38fc161920 con 0x7f38fc15d520 2024-04-25T10:28:45.929 INFO:teuthology.orchestra.run.smithi203.stderr:2024-04-25T10:28:45.925+0000 7f3903667640 1 -- --> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] -- mon_getmap magic: 0 -- 0x7f38fc0b2f00 con 0x7f38fc0d33d0 2024-04-25T10:28:45.929 INFO:teuthology.orchestra.run.smithi203.stderr:2024-04-25T10:28:45.926+0000 7f3900c02640 1 --2- >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f38fc0d33d0 0x7f38fc0d1650 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:28:45.929 INFO:teuthology.orchestra.run.smithi203.stderr:2024-04-25T10:28:45.926+0000 7f3901c04640 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f38fc15d520 0x7f38fc06cdd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:28:45.929 INFO:teuthology.orchestra.run.smithi203.stderr:2024-04-25T10:28:45.926+0000 7f3900c02640 1 --2- >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f38fc0d33d0 0x7f38fc0d1650 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.203:3301/0 says I am v2:172.21.15.203:38966/0 (socket says 172.21.15.203:38966) 2024-04-25T10:28:45.929 INFO:teuthology.orchestra.run.smithi203.stderr:2024-04-25T10:28:45.926+0000 7f3900c02640 1 -- 172.21.15.203:0/922800156 learned_addr learned my addr 172.21.15.203:0/922800156 (peer_addr_for_me v2:172.21.15.203:0/0) 2024-04-25T10:28:45.929 INFO:teuthology.orchestra.run.smithi203.stderr:2024-04-25T10:28:45.926+0000 7f3901403640 1 --2- 172.21.15.203:0/922800156 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f38fc0b1d70 0x7f38fc1517b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:28:45.929 INFO:teuthology.orchestra.run.smithi203.stderr:2024-04-25T10:28:45.926+0000 7f3901c04640 1 -- 172.21.15.203:0/922800156 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f38fc0d33d0 msgr2=0x7f38fc0d1650 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:28:45.929 INFO:teuthology.orchestra.run.smithi203.stderr:2024-04-25T10:28:45.926+0000 7f3901c04640 1 --2- 172.21.15.203:0/922800156 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f38fc0d33d0 0x7f38fc0d1650 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:28:45.929 INFO:teuthology.orchestra.run.smithi203.stderr:2024-04-25T10:28:45.926+0000 7f3901c04640 1 -- 172.21.15.203:0/922800156 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f38fc0b1d70 msgr2=0x7f38fc1517b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:28:45.929 INFO:teuthology.orchestra.run.smithi203.stderr:2024-04-25T10:28:45.926+0000 7f3901c04640 1 --2- 172.21.15.203:0/922800156 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f38fc0b1d70 0x7f38fc1517b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:28:45.929 INFO:teuthology.orchestra.run.smithi203.stderr:2024-04-25T10:28:45.926+0000 7f3901c04640 1 -- 172.21.15.203:0/922800156 --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f38fc0cc270 con 0x7f38fc15d520 2024-04-25T10:28:45.929 INFO:teuthology.orchestra.run.smithi203.stderr:2024-04-25T10:28:45.926+0000 7f3900c02640 1 --2- 172.21.15.203:0/922800156 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f38fc0d33d0 0x7f38fc0d1650 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-25T10:28:45.930 INFO:teuthology.orchestra.run.smithi203.stderr:2024-04-25T10:28:45.926+0000 7f3901c04640 1 --2- 172.21.15.203:0/922800156 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f38fc15d520 0x7f38fc06cdd0 secure :-1 s=READY pgs=16 cs=0 l=1 rev1=1 crypto rx=0x7f38f4007d40 tx=0x7f38f400d4a0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:28:45.930 INFO:teuthology.orchestra.run.smithi203.stderr:2024-04-25T10:28:45.926+0000 7f38f1ffb640 1 -- 172.21.15.203:0/922800156 <== mon.1 v2:172.21.15.203:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f38f401a070 con 0x7f38fc15d520 2024-04-25T10:28:45.930 INFO:teuthology.orchestra.run.smithi203.stderr:2024-04-25T10:28:45.926+0000 7f38f1ffb640 1 -- 172.21.15.203:0/922800156 <== mon.1 v2:172.21.15.203:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f38f4003990 con 0x7f38fc15d520 2024-04-25T10:28:45.930 INFO:teuthology.orchestra.run.smithi203.stderr:2024-04-25T10:28:45.926+0000 7f3903667640 1 -- 172.21.15.203:0/922800156 --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f38fc0cc550 con 0x7f38fc15d520 2024-04-25T10:28:45.930 INFO:teuthology.orchestra.run.smithi203.stderr:2024-04-25T10:28:45.926+0000 7f38f1ffb640 1 -- 172.21.15.203:0/922800156 <== mon.1 v2:172.21.15.203:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f38f4003d80 con 0x7f38fc15d520 2024-04-25T10:28:45.930 INFO:teuthology.orchestra.run.smithi203.stderr:2024-04-25T10:28:45.927+0000 7f3903667640 1 -- 172.21.15.203:0/922800156 --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f38fc0ccb10 con 0x7f38fc15d520 2024-04-25T10:28:45.930 INFO:teuthology.orchestra.run.smithi203.stderr:2024-04-25T10:28:45.927+0000 7f38f1ffb640 1 -- 172.21.15.203:0/922800156 <== mon.1 v2:172.21.15.203:3300/0 4 ==== mgrmap(e 1) ==== 807+0+0 (secure 0 0 0) 0x7f38f401d070 con 0x7f38fc15d520 2024-04-25T10:28:45.930 INFO:teuthology.orchestra.run.smithi203.stderr:2024-04-25T10:28:45.927+0000 7f38f1ffb640 1 -- 172.21.15.203:0/922800156 <== mon.1 v2:172.21.15.203:3300/0 5 ==== osd_map(8..8 src has 1..8) ==== 1415+0+0 (secure 0 0 0) 0x7f38f4015a30 con 0x7f38fc15d520 2024-04-25T10:28:45.930 INFO:teuthology.orchestra.run.smithi203.stderr:2024-04-25T10:28:45.928+0000 7f3903667640 1 -- 172.21.15.203:0/922800156 --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f38fc06a140 con 0x7f38fc15d520 2024-04-25T10:28:45.931 INFO:teuthology.orchestra.run.smithi203.stderr:2024-04-25T10:28:45.930+0000 7f38f1ffb640 1 -- 172.21.15.203:0/922800156 <== mon.1 v2:172.21.15.203:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+73265 (secure 0 0 0) 0x7f38f4024020 con 0x7f38fc15d520 2024-04-25T10:28:45.933 INFO:tasks.ceph.mgr.y.smithi037.stderr:2024-04-25T10:28:45.931+0000 7f1c9c0f7040 -1 mgr[py] Module selftest has missing NOTIFY_TYPES member 2024-04-25T10:28:45.988 INFO:tasks.ceph.mgr.y.smithi037.stderr:2024-04-25T10:28:45.986+0000 7f1c9c0f7040 -1 mgr[py] Module pg_autoscaler has missing NOTIFY_TYPES member 2024-04-25T10:28:46.004 INFO:teuthology.orchestra.run.smithi203.stderr:2024-04-25T10:28:46.003+0000 7f3903667640 1 -- 172.21.15.203:0/922800156 --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_command({"prefix": "osd new", "uuid": "f29dd047-3b48-473d-a235-6642a6293f20", "id": 6} v 0) -- 0x7f38fc06e100 con 0x7f38fc15d520 2024-04-25T10:28:46.012 INFO:teuthology.orchestra.run.smithi203.stderr:2024-04-25T10:28:46.010+0000 7f38f1ffb640 1 -- 172.21.15.203:0/922800156 <== mon.1 v2:172.21.15.203:3300/0 7 ==== mon_command_ack([{"prefix": "osd new", "uuid": "f29dd047-3b48-473d-a235-6642a6293f20", "id": 6}]=0 v9) ==== 112+0+2 (secure 0 0 0) 0x7f38f4010070 con 0x7f38fc15d520 2024-04-25T10:28:46.012 INFO:teuthology.orchestra.run.smithi203.stdout:6 2024-04-25T10:28:46.013 INFO:teuthology.orchestra.run.smithi203.stderr:2024-04-25T10:28:46.011+0000 7f3903667640 1 -- 172.21.15.203:0/922800156 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f38fc15d520 msgr2=0x7f38fc06cdd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:28:46.013 INFO:teuthology.orchestra.run.smithi203.stderr:2024-04-25T10:28:46.011+0000 7f3903667640 1 --2- 172.21.15.203:0/922800156 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f38fc15d520 0x7f38fc06cdd0 secure :-1 s=READY pgs=16 cs=0 l=1 rev1=1 crypto rx=0x7f38f4007d40 tx=0x7f38f400d4a0 comp rx=0 tx=0).stop 2024-04-25T10:28:46.013 INFO:teuthology.orchestra.run.smithi203.stderr:2024-04-25T10:28:46.011+0000 7f3903667640 1 -- 172.21.15.203:0/922800156 shutdown_connections 2024-04-25T10:28:46.013 INFO:teuthology.orchestra.run.smithi203.stderr:2024-04-25T10:28:46.011+0000 7f3903667640 1 --2- 172.21.15.203:0/922800156 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f38fc15d520 0x7f38fc06cdd0 unknown :-1 s=CLOSED pgs=16 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:28:46.013 INFO:teuthology.orchestra.run.smithi203.stderr:2024-04-25T10:28:46.011+0000 7f3903667640 1 --2- 172.21.15.203:0/922800156 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f38fc0d33d0 0x7f38fc0d1650 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:28:46.013 INFO:teuthology.orchestra.run.smithi203.stderr:2024-04-25T10:28:46.011+0000 7f3903667640 1 --2- 172.21.15.203:0/922800156 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f38fc0b1d70 0x7f38fc1517b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:28:46.013 INFO:teuthology.orchestra.run.smithi203.stderr:2024-04-25T10:28:46.011+0000 7f3903667640 1 -- 172.21.15.203:0/922800156 >> 172.21.15.203:0/922800156 conn(0x7f38fc0b7620 msgr2=0x7f38fc071180 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:28:46.013 INFO:teuthology.orchestra.run.smithi203.stderr:2024-04-25T10:28:46.011+0000 7f3903667640 1 -- 172.21.15.203:0/922800156 shutdown_connections 2024-04-25T10:28:46.013 INFO:teuthology.orchestra.run.smithi203.stderr:2024-04-25T10:28:46.011+0000 7f3903667640 1 -- 172.21.15.203:0/922800156 wait complete. 2024-04-25T10:28:46.024 DEBUG:teuthology.orchestra.run.smithi203:> sudo ceph --cluster ceph osd new 6e691644-378b-4fdc-8e86-fb5f56e31f29 7 2024-04-25T10:28:46.106 INFO:tasks.ceph.mgr.y.smithi037.stderr:2024-04-25T10:28:46.104+0000 7f1c9c0f7040 -1 mgr[py] Module test_orchestrator has missing NOTIFY_TYPES member 2024-04-25T10:28:46.132 INFO:teuthology.orchestra.run.smithi203.stderr:2024-04-25T10:28:46.130+0000 7f442ba1a640 1 Processor -- start 2024-04-25T10:28:46.132 INFO:teuthology.orchestra.run.smithi203.stderr:2024-04-25T10:28:46.131+0000 7f442ba1a640 1 -- start start 2024-04-25T10:28:46.134 INFO:teuthology.orchestra.run.smithi203.stderr:2024-04-25T10:28:46.132+0000 7f442ba1a640 1 --2- >> v2:172.21.15.203:3300/0 conn(0x7f44240d1c20 0x7f44240d2030 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:28:46.134 INFO:teuthology.orchestra.run.smithi203.stderr:2024-04-25T10:28:46.132+0000 7f442ba1a640 1 --2- >> v2:172.21.15.37:3300/0 conn(0x7f44240b71b0 0x7f44240b75c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:28:46.134 INFO:teuthology.orchestra.run.smithi203.stderr:2024-04-25T10:28:46.132+0000 7f442ba1a640 1 -- --> v1:172.21.15.203:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f44240d2570 con 0x7f442408cc90 2024-04-25T10:28:46.134 INFO:teuthology.orchestra.run.smithi203.stderr:2024-04-25T10:28:46.132+0000 7f442ba1a640 1 -- --> v2:172.21.15.37:3300/0 -- mon_getmap magic: 0 -- 0x7f442409d2d0 con 0x7f44240b71b0 2024-04-25T10:28:46.134 INFO:teuthology.orchestra.run.smithi203.stderr:2024-04-25T10:28:46.132+0000 7f442ba1a640 1 -- --> v2:172.21.15.203:3300/0 -- mon_getmap magic: 0 -- 0x7f44240b8d40 con 0x7f44240d1c20 2024-04-25T10:28:46.134 INFO:teuthology.orchestra.run.smithi203.stderr:2024-04-25T10:28:46.132+0000 7f4428fb5640 1 --2- >> v2:172.21.15.203:3300/0 conn(0x7f44240d1c20 0x7f44240d2030 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:28:46.134 INFO:teuthology.orchestra.run.smithi203.stderr:2024-04-25T10:28:46.132+0000 7f4428fb5640 1 --2- >> v2:172.21.15.203:3300/0 conn(0x7f44240d1c20 0x7f44240d2030 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.203:3300/0 says I am v2:172.21.15.203:44276/0 (socket says 172.21.15.203:44276) 2024-04-25T10:28:46.134 INFO:teuthology.orchestra.run.smithi203.stderr:2024-04-25T10:28:46.132+0000 7f4428fb5640 1 -- 172.21.15.203:0/3751357271 learned_addr learned my addr 172.21.15.203:0/3751357271 (peer_addr_for_me v2:172.21.15.203:0/0) 2024-04-25T10:28:46.134 INFO:teuthology.orchestra.run.smithi203.stderr:2024-04-25T10:28:46.132+0000 7f4429fb7640 1 --2- 172.21.15.203:0/3751357271 >> v2:172.21.15.37:3300/0 conn(0x7f44240b71b0 0x7f44240b75c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:28:46.135 INFO:teuthology.orchestra.run.smithi203.stderr:2024-04-25T10:28:46.133+0000 7f4428fb5640 1 -- 172.21.15.203:0/3751357271 >> v2:172.21.15.37:3300/0 conn(0x7f44240b71b0 msgr2=0x7f44240b75c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:28:46.135 INFO:teuthology.orchestra.run.smithi203.stderr:2024-04-25T10:28:46.133+0000 7f4428fb5640 1 --2- 172.21.15.203:0/3751357271 >> v2:172.21.15.37:3300/0 conn(0x7f44240b71b0 0x7f44240b75c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:28:46.135 INFO:teuthology.orchestra.run.smithi203.stderr:2024-04-25T10:28:46.133+0000 7f4428fb5640 1 -- 172.21.15.203:0/3751357271 >> v1:172.21.15.203:6789/0 conn(0x7f442408cc90 legacy=0x7f442409cbc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:28:46.135 INFO:teuthology.orchestra.run.smithi203.stderr:2024-04-25T10:28:46.133+0000 7f4428fb5640 1 -- 172.21.15.203:0/3751357271 --> v2:172.21.15.203:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f44240d27f0 con 0x7f44240d1c20 2024-04-25T10:28:46.136 INFO:teuthology.orchestra.run.smithi203.stderr:2024-04-25T10:28:46.133+0000 7f4428fb5640 1 --2- 172.21.15.203:0/3751357271 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f44240d1c20 0x7f44240d2030 secure :-1 s=READY pgs=18 cs=0 l=1 rev1=1 crypto rx=0x7f4424069f10 tx=0x7f442002a270 comp rx=0 tx=0).ready entity=mon.1 client_cookie=4a8b1d053c114581 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:28:46.136 INFO:teuthology.orchestra.run.smithi203.stderr:2024-04-25T10:28:46.133+0000 7f441bfff640 1 -- 172.21.15.203:0/3751357271 <== mon.1 v2:172.21.15.203:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4420037070 con 0x7f44240d1c20 2024-04-25T10:28:46.136 INFO:teuthology.orchestra.run.smithi203.stderr:2024-04-25T10:28:46.133+0000 7f441bfff640 1 -- 172.21.15.203:0/3751357271 <== mon.1 v2:172.21.15.203:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f4420032440 con 0x7f44240d1c20 2024-04-25T10:28:46.136 INFO:teuthology.orchestra.run.smithi203.stderr:2024-04-25T10:28:46.133+0000 7f441bfff640 1 -- 172.21.15.203:0/3751357271 <== mon.1 v2:172.21.15.203:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4420032810 con 0x7f44240d1c20 2024-04-25T10:28:46.136 INFO:teuthology.orchestra.run.smithi203.stderr:2024-04-25T10:28:46.134+0000 7f442ba1a640 1 -- 172.21.15.203:0/3751357271 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f44240d1c20 msgr2=0x7f44240d2030 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:28:46.137 INFO:teuthology.orchestra.run.smithi203.stderr:2024-04-25T10:28:46.134+0000 7f442ba1a640 1 --2- 172.21.15.203:0/3751357271 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f44240d1c20 0x7f44240d2030 secure :-1 s=READY pgs=18 cs=0 l=1 rev1=1 crypto rx=0x7f4424069f10 tx=0x7f442002a270 comp rx=0 tx=0).stop 2024-04-25T10:28:46.137 INFO:teuthology.orchestra.run.smithi203.stderr:2024-04-25T10:28:46.134+0000 7f442ba1a640 1 -- 172.21.15.203:0/3751357271 shutdown_connections 2024-04-25T10:28:46.137 INFO:teuthology.orchestra.run.smithi203.stderr:2024-04-25T10:28:46.134+0000 7f442ba1a640 1 --2- 172.21.15.203:0/3751357271 >> v2:172.21.15.37:3300/0 conn(0x7f44240b71b0 0x7f44240b75c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:28:46.137 INFO:teuthology.orchestra.run.smithi203.stderr:2024-04-25T10:28:46.134+0000 7f442ba1a640 1 --2- 172.21.15.203:0/3751357271 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f44240d1c20 0x7f44240d2030 unknown :-1 s=CLOSED pgs=18 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:28:46.137 INFO:teuthology.orchestra.run.smithi203.stderr:2024-04-25T10:28:46.134+0000 7f442ba1a640 1 -- 172.21.15.203:0/3751357271 >> 172.21.15.203:0/3751357271 conn(0x7f44240d03d0 msgr2=0x7f44240d07e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:28:46.137 INFO:teuthology.orchestra.run.smithi203.stderr:2024-04-25T10:28:46.134+0000 7f442ba1a640 1 -- 172.21.15.203:0/3751357271 shutdown_connections 2024-04-25T10:28:46.137 INFO:teuthology.orchestra.run.smithi203.stderr:2024-04-25T10:28:46.134+0000 7f442ba1a640 1 -- 172.21.15.203:0/3751357271 wait complete. 2024-04-25T10:28:46.138 INFO:teuthology.orchestra.run.smithi203.stderr:2024-04-25T10:28:46.137+0000 7f442ba1a640 1 Processor -- start 2024-04-25T10:28:46.139 INFO:teuthology.orchestra.run.smithi203.stderr:2024-04-25T10:28:46.137+0000 7f442ba1a640 1 -- start start 2024-04-25T10:28:46.142 INFO:teuthology.orchestra.run.smithi203.stderr:2024-04-25T10:28:46.140+0000 7f442ba1a640 1 --2- >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f44240b71b0 0x7f44241dd670 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:28:46.143 INFO:teuthology.orchestra.run.smithi203.stderr:2024-04-25T10:28:46.140+0000 7f442ba1a640 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f44240d1c20 0x7f44240a1030 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:28:46.143 INFO:teuthology.orchestra.run.smithi203.stderr:2024-04-25T10:28:46.140+0000 7f442ba1a640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f44240c8ca0 0x7f44240c90b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:28:46.143 INFO:teuthology.orchestra.run.smithi203.stderr:2024-04-25T10:28:46.140+0000 7f442ba1a640 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 -- 0x7f44240b93b0 con 0x7f44240c8ca0 2024-04-25T10:28:46.143 INFO:teuthology.orchestra.run.smithi203.stderr:2024-04-25T10:28:46.140+0000 7f442ba1a640 1 -- --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_getmap magic: 0 -- 0x7f44240b8fd0 con 0x7f44240d1c20 2024-04-25T10:28:46.143 INFO:teuthology.orchestra.run.smithi203.stderr:2024-04-25T10:28:46.140+0000 7f442ba1a640 1 -- --> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] -- mon_getmap magic: 0 -- 0x7f44240b7b00 con 0x7f44240b71b0 2024-04-25T10:28:46.143 INFO:teuthology.orchestra.run.smithi203.stderr:2024-04-25T10:28:46.140+0000 7f4428fb5640 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f44240d1c20 0x7f44240a1030 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:28:46.143 INFO:teuthology.orchestra.run.smithi203.stderr:2024-04-25T10:28:46.140+0000 7f44297b6640 1 --2- >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f44240b71b0 0x7f44241dd670 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:28:46.143 INFO:teuthology.orchestra.run.smithi203.stderr:2024-04-25T10:28:46.140+0000 7f4428fb5640 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f44240d1c20 0x7f44240a1030 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.203:3300/0 says I am v2:172.21.15.203:44280/0 (socket says 172.21.15.203:44280) 2024-04-25T10:28:46.143 INFO:teuthology.orchestra.run.smithi203.stderr:2024-04-25T10:28:46.140+0000 7f44297b6640 1 --2- >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f44240b71b0 0x7f44241dd670 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.203:3301/0 says I am v2:172.21.15.203:38970/0 (socket says 172.21.15.203:38970) 2024-04-25T10:28:46.143 INFO:teuthology.orchestra.run.smithi203.stderr:2024-04-25T10:28:46.140+0000 7f4428fb5640 1 -- 172.21.15.203:0/628028236 learned_addr learned my addr 172.21.15.203:0/628028236 (peer_addr_for_me v2:172.21.15.203:0/0) 2024-04-25T10:28:46.143 INFO:teuthology.orchestra.run.smithi203.stderr:2024-04-25T10:28:46.140+0000 7f4429fb7640 1 --2- 172.21.15.203:0/628028236 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f44240c8ca0 0x7f44240c90b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:28:46.143 INFO:teuthology.orchestra.run.smithi203.stderr:2024-04-25T10:28:46.140+0000 7f4428fb5640 1 -- 172.21.15.203:0/628028236 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f44240b71b0 msgr2=0x7f44241dd670 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:28:46.143 INFO:teuthology.orchestra.run.smithi203.stderr:2024-04-25T10:28:46.140+0000 7f4428fb5640 1 --2- 172.21.15.203:0/628028236 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f44240b71b0 0x7f44241dd670 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:28:46.144 INFO:teuthology.orchestra.run.smithi203.stderr:2024-04-25T10:28:46.140+0000 7f4428fb5640 1 -- 172.21.15.203:0/628028236 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f44240c8ca0 msgr2=0x7f44240c90b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:28:46.144 INFO:teuthology.orchestra.run.smithi203.stderr:2024-04-25T10:28:46.140+0000 7f4428fb5640 1 --2- 172.21.15.203:0/628028236 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f44240c8ca0 0x7f44240c90b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:28:46.144 INFO:teuthology.orchestra.run.smithi203.stderr:2024-04-25T10:28:46.140+0000 7f4428fb5640 1 -- 172.21.15.203:0/628028236 --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f44240a1570 con 0x7f44240d1c20 2024-04-25T10:28:46.144 INFO:teuthology.orchestra.run.smithi203.stderr:2024-04-25T10:28:46.140+0000 7f44297b6640 1 --2- 172.21.15.203:0/628028236 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f44240b71b0 0x7f44241dd670 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-25T10:28:46.144 INFO:teuthology.orchestra.run.smithi203.stderr:2024-04-25T10:28:46.140+0000 7f4428fb5640 1 --2- 172.21.15.203:0/628028236 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f44240d1c20 0x7f44240a1030 secure :-1 s=READY pgs=19 cs=0 l=1 rev1=1 crypto rx=0x7f4420000c00 tx=0x7f44200046c0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:28:46.144 INFO:teuthology.orchestra.run.smithi203.stderr:2024-04-25T10:28:46.140+0000 7f4419ffb640 1 -- 172.21.15.203:0/628028236 <== mon.1 v2:172.21.15.203:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4420041020 con 0x7f44240d1c20 2024-04-25T10:28:46.144 INFO:teuthology.orchestra.run.smithi203.stderr:2024-04-25T10:28:46.141+0000 7f4419ffb640 1 -- 172.21.15.203:0/628028236 <== mon.1 v2:172.21.15.203:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f44200306c0 con 0x7f44240d1c20 2024-04-25T10:28:46.144 INFO:teuthology.orchestra.run.smithi203.stderr:2024-04-25T10:28:46.141+0000 7f442ba1a640 1 -- 172.21.15.203:0/628028236 --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f44240cc3e0 con 0x7f44240d1c20 2024-04-25T10:28:46.144 INFO:teuthology.orchestra.run.smithi203.stderr:2024-04-25T10:28:46.141+0000 7f4419ffb640 1 -- 172.21.15.203:0/628028236 <== mon.1 v2:172.21.15.203:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4420037040 con 0x7f44240d1c20 2024-04-25T10:28:46.144 INFO:teuthology.orchestra.run.smithi203.stderr:2024-04-25T10:28:46.141+0000 7f442ba1a640 1 -- 172.21.15.203:0/628028236 --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f44240cc8e0 con 0x7f44240d1c20 2024-04-25T10:28:46.144 INFO:teuthology.orchestra.run.smithi203.stderr:2024-04-25T10:28:46.141+0000 7f4419ffb640 1 -- 172.21.15.203:0/628028236 <== mon.1 v2:172.21.15.203:3300/0 4 ==== mgrmap(e 1) ==== 807+0+0 (secure 0 0 0) 0x7f4420044020 con 0x7f44240d1c20 2024-04-25T10:28:46.144 INFO:teuthology.orchestra.run.smithi203.stderr:2024-04-25T10:28:46.141+0000 7f4419ffb640 1 -- 172.21.15.203:0/628028236 <== mon.1 v2:172.21.15.203:3300/0 5 ==== osd_map(9..9 src has 1..9) ==== 1530+0+0 (secure 0 0 0) 0x7f442003dd20 con 0x7f44240d1c20 2024-04-25T10:28:46.145 INFO:teuthology.orchestra.run.smithi203.stderr:2024-04-25T10:28:46.142+0000 7f442ba1a640 1 -- 172.21.15.203:0/628028236 --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f442406a140 con 0x7f44240d1c20 2024-04-25T10:28:46.145 INFO:teuthology.orchestra.run.smithi203.stderr:2024-04-25T10:28:46.144+0000 7f4419ffb640 1 -- 172.21.15.203:0/628028236 <== mon.1 v2:172.21.15.203:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+73265 (secure 0 0 0) 0x7f442004b030 con 0x7f44240d1c20 2024-04-25T10:28:46.216 INFO:teuthology.orchestra.run.smithi203.stderr:2024-04-25T10:28:46.215+0000 7f442ba1a640 1 -- 172.21.15.203:0/628028236 --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_command({"prefix": "osd new", "uuid": "6e691644-378b-4fdc-8e86-fb5f56e31f29", "id": 7} v 0) -- 0x7f44240b7870 con 0x7f44240d1c20 2024-04-25T10:28:46.225 INFO:teuthology.orchestra.run.smithi203.stderr:2024-04-25T10:28:46.224+0000 7f4419ffb640 1 -- 172.21.15.203:0/628028236 <== mon.1 v2:172.21.15.203:3300/0 7 ==== mon_command_ack([{"prefix": "osd new", "uuid": "6e691644-378b-4fdc-8e86-fb5f56e31f29", "id": 7}]=0 v10) ==== 112+0+2 (secure 0 0 0) 0x7f44240a1570 con 0x7f44240d1c20 2024-04-25T10:28:46.225 INFO:teuthology.orchestra.run.smithi203.stdout:7 2024-04-25T10:28:46.226 INFO:teuthology.orchestra.run.smithi203.stderr:2024-04-25T10:28:46.225+0000 7f442ba1a640 1 -- 172.21.15.203:0/628028236 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f44240d1c20 msgr2=0x7f44240a1030 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:28:46.226 INFO:teuthology.orchestra.run.smithi203.stderr:2024-04-25T10:28:46.225+0000 7f442ba1a640 1 --2- 172.21.15.203:0/628028236 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f44240d1c20 0x7f44240a1030 secure :-1 s=READY pgs=19 cs=0 l=1 rev1=1 crypto rx=0x7f4420000c00 tx=0x7f44200046c0 comp rx=0 tx=0).stop 2024-04-25T10:28:46.227 INFO:teuthology.orchestra.run.smithi203.stderr:2024-04-25T10:28:46.225+0000 7f442ba1a640 1 -- 172.21.15.203:0/628028236 shutdown_connections 2024-04-25T10:28:46.227 INFO:teuthology.orchestra.run.smithi203.stderr:2024-04-25T10:28:46.225+0000 7f442ba1a640 1 --2- 172.21.15.203:0/628028236 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f44240c8ca0 0x7f44240c90b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:28:46.227 INFO:teuthology.orchestra.run.smithi203.stderr:2024-04-25T10:28:46.225+0000 7f442ba1a640 1 --2- 172.21.15.203:0/628028236 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f44240d1c20 0x7f44240a1030 unknown :-1 s=CLOSED pgs=19 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:28:46.227 INFO:teuthology.orchestra.run.smithi203.stderr:2024-04-25T10:28:46.225+0000 7f442ba1a640 1 --2- 172.21.15.203:0/628028236 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f44240b71b0 0x7f44241dd670 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:28:46.227 INFO:teuthology.orchestra.run.smithi203.stderr:2024-04-25T10:28:46.225+0000 7f442ba1a640 1 -- 172.21.15.203:0/628028236 >> 172.21.15.203:0/628028236 conn(0x7f44240d03d0 msgr2=0x7f442409cb30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:28:46.227 INFO:teuthology.orchestra.run.smithi203.stderr:2024-04-25T10:28:46.225+0000 7f442ba1a640 1 -- 172.21.15.203:0/628028236 shutdown_connections 2024-04-25T10:28:46.227 INFO:teuthology.orchestra.run.smithi203.stderr:2024-04-25T10:28:46.225+0000 7f442ba1a640 1 -- 172.21.15.203:0/628028236 wait complete. 2024-04-25T10:28:46.237 INFO:tasks.ceph.osd.0:Restarting daemon 2024-04-25T10:28:46.237 DEBUG:teuthology.orchestra.run.smithi037:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-osd -f --cluster ceph -i 0 2024-04-25T10:28:46.239 INFO:tasks.ceph.osd.0:Started 2024-04-25T10:28:46.239 INFO:tasks.ceph.osd.1:Restarting daemon 2024-04-25T10:28:46.239 DEBUG:teuthology.orchestra.run.smithi037:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-osd -f --cluster ceph -i 1 2024-04-25T10:28:46.243 INFO:tasks.ceph.osd.1:Started 2024-04-25T10:28:46.243 INFO:tasks.ceph.osd.2:Restarting daemon 2024-04-25T10:28:46.243 DEBUG:teuthology.orchestra.run.smithi037:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-osd -f --cluster ceph -i 2 2024-04-25T10:28:46.245 INFO:tasks.ceph.osd.2:Started 2024-04-25T10:28:46.245 INFO:tasks.ceph.osd.3:Restarting daemon 2024-04-25T10:28:46.246 DEBUG:teuthology.orchestra.run.smithi037:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-osd -f --cluster ceph -i 3 2024-04-25T10:28:46.248 INFO:tasks.ceph.osd.3:Started 2024-04-25T10:28:46.248 INFO:tasks.ceph.osd.4:Restarting daemon 2024-04-25T10:28:46.248 DEBUG:teuthology.orchestra.run.smithi203:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-osd -f --cluster ceph -i 4 2024-04-25T10:28:46.250 INFO:tasks.ceph.osd.4:Started 2024-04-25T10:28:46.250 INFO:tasks.ceph.osd.5:Restarting daemon 2024-04-25T10:28:46.250 DEBUG:teuthology.orchestra.run.smithi203:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-osd -f --cluster ceph -i 5 2024-04-25T10:28:46.252 INFO:tasks.ceph.osd.5:Started 2024-04-25T10:28:46.252 INFO:tasks.ceph.osd.6:Restarting daemon 2024-04-25T10:28:46.253 DEBUG:teuthology.orchestra.run.smithi203:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-osd -f --cluster ceph -i 6 2024-04-25T10:28:46.254 INFO:tasks.ceph.osd.6:Started 2024-04-25T10:28:46.254 INFO:tasks.ceph.osd.7:Restarting daemon 2024-04-25T10:28:46.255 DEBUG:teuthology.orchestra.run.smithi203:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-osd -f --cluster ceph -i 7 2024-04-25T10:28:46.257 INFO:tasks.ceph.osd.7:Started 2024-04-25T10:28:46.257 DEBUG:teuthology.orchestra.run.smithi037:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-04-25T10:28:46.360 INFO:tasks.ceph.mgr.x.smithi203.stderr:2024-04-25T10:28:46.358+0000 7fbc1bc32040 -1 mgr[py] Module rook has missing NOTIFY_TYPES member 2024-04-25T10:28:46.408 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:28:46.406+0000 7fdc08cbd640 1 Processor -- start 2024-04-25T10:28:46.408 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:28:46.406+0000 7fdc08cbd640 1 -- start start 2024-04-25T10:28:46.411 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:28:46.408+0000 7fdc08cbd640 1 --2- >> v2:172.21.15.203:3300/0 conn(0x7fdc040700f0 0x7fdc04070500 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:28:46.411 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:28:46.408+0000 7fdc08cbd640 1 --2- >> v2:172.21.15.37:3300/0 conn(0x7fdc040d27c0 0x7fdc040d2bd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:28:46.411 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:28:46.408+0000 7fdc08cbd640 1 -- --> v1:172.21.15.37:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fdc04070a40 con 0x7fdc040ce360 2024-04-25T10:28:46.411 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:28:46.408+0000 7fdc08cbd640 1 -- --> v2:172.21.15.37:3300/0 -- mon_getmap magic: 0 -- 0x7fdc041623d0 con 0x7fdc040d27c0 2024-04-25T10:28:46.411 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:28:46.408+0000 7fdc08cbd640 1 -- --> v2:172.21.15.203:3300/0 -- mon_getmap magic: 0 -- 0x7fdc041626b0 con 0x7fdc040700f0 2024-04-25T10:28:46.411 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:28:46.408+0000 7fdc02d9d640 1 --2- >> v2:172.21.15.37:3300/0 conn(0x7fdc040d27c0 0x7fdc040d2bd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:28:46.411 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:28:46.408+0000 7fdc02d9d640 1 --2- >> v2:172.21.15.37:3300/0 conn(0x7fdc040d27c0 0x7fdc040d2bd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.37:3300/0 says I am v2:172.21.15.37:42430/0 (socket says 172.21.15.37:42430) 2024-04-25T10:28:46.411 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:28:46.408+0000 7fdc0259c640 1 --1- >> v1:172.21.15.37:6789/0 conn(0x7fdc040ce360 0x7fdc04161b80 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.37:6789/0 says I am v1:172.21.15.37:40436/0 (socket says 172.21.15.37:40436) 2024-04-25T10:28:46.411 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:28:46.408+0000 7fdc02d9d640 1 -- 172.21.15.37:0/3683750836 learned_addr learned my addr 172.21.15.37:0/3683750836 (peer_addr_for_me v2:172.21.15.37:0/0) 2024-04-25T10:28:46.411 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:28:46.408+0000 7fdc01d9b640 1 --2- 172.21.15.37:0/3683750836 >> v2:172.21.15.203:3300/0 conn(0x7fdc040700f0 0x7fdc04070500 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:28:46.411 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:28:46.408+0000 7fdc02d9d640 1 -- 172.21.15.37:0/3683750836 >> v2:172.21.15.203:3300/0 conn(0x7fdc040700f0 msgr2=0x7fdc04070500 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:28:46.411 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:28:46.408+0000 7fdc02d9d640 1 --2- 172.21.15.37:0/3683750836 >> v2:172.21.15.203:3300/0 conn(0x7fdc040700f0 0x7fdc04070500 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:28:46.411 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:28:46.408+0000 7fdc02d9d640 1 -- 172.21.15.37:0/3683750836 >> v1:172.21.15.37:6789/0 conn(0x7fdc040ce360 legacy=0x7fdc04161b80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:28:46.411 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:28:46.408+0000 7fdc02d9d640 1 -- 172.21.15.37:0/3683750836 --> v2:172.21.15.37:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fdc04162990 con 0x7fdc040d27c0 2024-04-25T10:28:46.412 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:28:46.408+0000 7fdc0159a640 1 -- 172.21.15.37:0/3683750836 <== mon.0 v1:172.21.15.37:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 2096594611 0 0) 0x7fdbf4001110 con 0x7fdc040ce360 2024-04-25T10:28:46.413 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:28:46.409+0000 7fdc02d9d640 1 --2- 172.21.15.37:0/3683750836 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fdc040d27c0 0x7fdc040d2bd0 secure :-1 s=READY pgs=22 cs=0 l=1 rev1=1 crypto rx=0x7fdc040d1860 tx=0x7fdbfc02a730 comp rx=0 tx=0).ready entity=mon.0 client_cookie=86413771ae3b7090 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:28:46.413 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:28:46.409+0000 7fdc0159a640 1 -- 172.21.15.37:0/3683750836 <== mon.0 v2:172.21.15.37:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdbfc036070 con 0x7fdc040d27c0 2024-04-25T10:28:46.413 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:28:46.410+0000 7fdc0159a640 1 -- 172.21.15.37:0/3683750836 <== mon.0 v2:172.21.15.37:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fdbfc031600 con 0x7fdc040d27c0 2024-04-25T10:28:46.413 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:28:46.410+0000 7fdc0159a640 1 -- 172.21.15.37:0/3683750836 <== mon.0 v2:172.21.15.37:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdbfc0319d0 con 0x7fdc040d27c0 2024-04-25T10:28:46.413 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:28:46.410+0000 7fdc08cbd640 1 -- 172.21.15.37:0/3683750836 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fdc040d27c0 msgr2=0x7fdc040d2bd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:28:46.413 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:28:46.410+0000 7fdc08cbd640 1 --2- 172.21.15.37:0/3683750836 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fdc040d27c0 0x7fdc040d2bd0 secure :-1 s=READY pgs=22 cs=0 l=1 rev1=1 crypto rx=0x7fdc040d1860 tx=0x7fdbfc02a730 comp rx=0 tx=0).stop 2024-04-25T10:28:46.413 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:28:46.410+0000 7fdc08cbd640 1 -- 172.21.15.37:0/3683750836 shutdown_connections 2024-04-25T10:28:46.413 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:28:46.410+0000 7fdc08cbd640 1 --2- 172.21.15.37:0/3683750836 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fdc040d27c0 0x7fdc040d2bd0 unknown :-1 s=CLOSED pgs=22 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:28:46.413 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:28:46.410+0000 7fdc08cbd640 1 --2- 172.21.15.37:0/3683750836 >> v2:172.21.15.203:3300/0 conn(0x7fdc040700f0 0x7fdc04070500 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:28:46.413 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:28:46.410+0000 7fdc08cbd640 1 -- 172.21.15.37:0/3683750836 >> 172.21.15.37:0/3683750836 conn(0x7fdc040d0460 msgr2=0x7fdc040d0870 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:28:46.414 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:28:46.410+0000 7fdc08cbd640 1 -- 172.21.15.37:0/3683750836 shutdown_connections 2024-04-25T10:28:46.414 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:28:46.410+0000 7fdc08cbd640 1 -- 172.21.15.37:0/3683750836 wait complete. 2024-04-25T10:28:46.414 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:28:46.412+0000 7fdc08cbd640 1 Processor -- start 2024-04-25T10:28:46.414 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:28:46.412+0000 7fdc08cbd640 1 -- start start 2024-04-25T10:28:46.418 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:28:46.414+0000 7fdc08cbd640 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fdc040700f0 0x7fdc04153cc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:28:46.418 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:28:46.414+0000 7fdc08cbd640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fdc040ce360 0x7fdc040cce10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:28:46.418 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:28:46.414+0000 7fdc08cbd640 1 --2- >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7fdc040d27c0 0x7fdc04158e70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:28:46.418 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:28:46.414+0000 7fdc08cbd640 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 -- 0x7fdc0415e350 con 0x7fdc040ce360 2024-04-25T10:28:46.418 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:28:46.414+0000 7fdc08cbd640 1 -- --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_getmap magic: 0 -- 0x7fdc040d22a0 con 0x7fdc040700f0 2024-04-25T10:28:46.418 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:28:46.414+0000 7fdc08cbd640 1 -- --> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] -- mon_getmap magic: 0 -- 0x7fdc0415e4d0 con 0x7fdc040d27c0 2024-04-25T10:28:46.418 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:28:46.414+0000 7fdc01d9b640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fdc040ce360 0x7fdc040cce10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:28:46.418 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:28:46.414+0000 7fdc0259c640 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fdc040700f0 0x7fdc04153cc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:28:46.418 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:28:46.414+0000 7fdc01d9b640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fdc040ce360 0x7fdc040cce10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.37:3300/0 says I am v2:172.21.15.37:42444/0 (socket says 172.21.15.37:42444) 2024-04-25T10:28:46.418 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:28:46.414+0000 7fdc01d9b640 1 -- 172.21.15.37:0/4269032516 learned_addr learned my addr 172.21.15.37:0/4269032516 (peer_addr_for_me v2:172.21.15.37:0/0) 2024-04-25T10:28:46.419 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:28:46.414+0000 7fdc02d9d640 1 --2- 172.21.15.37:0/4269032516 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7fdc040d27c0 0x7fdc04158e70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:28:46.419 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:28:46.414+0000 7fdc01d9b640 1 -- 172.21.15.37:0/4269032516 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7fdc040d27c0 msgr2=0x7fdc04158e70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:28:46.419 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:28:46.414+0000 7fdc01d9b640 1 --2- 172.21.15.37:0/4269032516 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7fdc040d27c0 0x7fdc04158e70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:28:46.419 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:28:46.414+0000 7fdc01d9b640 1 -- 172.21.15.37:0/4269032516 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fdc040700f0 msgr2=0x7fdc04153cc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:28:46.419 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:28:46.414+0000 7fdc01d9b640 1 --2- 172.21.15.37:0/4269032516 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fdc040700f0 0x7fdc04153cc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:28:46.419 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:28:46.414+0000 7fdc01d9b640 1 -- 172.21.15.37:0/4269032516 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fdc0415f970 con 0x7fdc040ce360 2024-04-25T10:28:46.419 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:28:46.414+0000 7fdc0259c640 1 --2- 172.21.15.37:0/4269032516 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fdc040700f0 0x7fdc04153cc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-25T10:28:46.419 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:28:46.415+0000 7fdc01d9b640 1 --2- 172.21.15.37:0/4269032516 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fdc040ce360 0x7fdc040cce10 secure :-1 s=READY pgs=23 cs=0 l=1 rev1=1 crypto rx=0x7fdbec00e870 tx=0x7fdbec00ed30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:28:46.419 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:28:46.415+0000 7fdbfaffd640 1 -- 172.21.15.37:0/4269032516 <== mon.0 v2:172.21.15.37:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdbec017070 con 0x7fdc040ce360 2024-04-25T10:28:46.419 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:28:46.415+0000 7fdc08cbd640 1 -- 172.21.15.37:0/4269032516 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fdc0415fc50 con 0x7fdc040ce360 2024-04-25T10:28:46.419 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:28:46.415+0000 7fdbfaffd640 1 -- 172.21.15.37:0/4269032516 <== mon.0 v2:172.21.15.37:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fdbec00c8e0 con 0x7fdc040ce360 2024-04-25T10:28:46.419 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:28:46.415+0000 7fdbfaffd640 1 -- 172.21.15.37:0/4269032516 <== mon.0 v2:172.21.15.37:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdbec00ccd0 con 0x7fdc040ce360 2024-04-25T10:28:46.419 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:28:46.415+0000 7fdc08cbd640 1 -- 172.21.15.37:0/4269032516 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fdc04160210 con 0x7fdc040ce360 2024-04-25T10:28:46.419 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:28:46.415+0000 7fdbfaffd640 1 -- 172.21.15.37:0/4269032516 <== mon.0 v2:172.21.15.37:3300/0 4 ==== mgrmap(e 1) ==== 807+0+0 (secure 0 0 0) 0x7fdbec01a070 con 0x7fdc040ce360 2024-04-25T10:28:46.419 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:28:46.415+0000 7fdbfaffd640 1 -- 172.21.15.37:0/4269032516 <== mon.0 v2:172.21.15.37:3300/0 5 ==== osd_map(10..10 src has 1..10) ==== 1645+0+0 (secure 0 0 0) 0x7fdbec00ca90 con 0x7fdc040ce360 2024-04-25T10:28:46.420 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:28:46.416+0000 7fdc08cbd640 1 -- 172.21.15.37:0/4269032516 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fdc0406a140 con 0x7fdc040ce360 2024-04-25T10:28:46.420 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:28:46.418+0000 7fdbfaffd640 1 -- 172.21.15.37:0/4269032516 <== mon.0 v2:172.21.15.37:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+73265 (secure 0 0 0) 0x7fdbec01cb70 con 0x7fdc040ce360 2024-04-25T10:28:46.483 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:28:46.481+0000 7fdc08cbd640 1 -- 172.21.15.37:0/4269032516 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7fdc040d1b70 con 0x7fdc040ce360 2024-04-25T10:28:46.483 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:28:46.481+0000 7fdbfaffd640 1 -- 172.21.15.37:0/4269032516 <== mon.0 v2:172.21.15.37:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v10) ==== 74+0+7058 (secure 0 0 0) 0x7fdbec01cb70 con 0x7fdc040ce360 2024-04-25T10:28:46.484 INFO:teuthology.orchestra.run.smithi037.stdout: 2024-04-25T10:28:46.484 INFO:teuthology.orchestra.run.smithi037.stdout:{"epoch":10,"fsid":"f4acda28-be34-492a-b286-994afd36f61d","created":"2024-04-25T10:28:43.985718+0000","modified":"2024-04-25T10:28:46.216973+0000","last_up_change":"0.000000","last_in_change":"2024-04-25T10:28:46.216973+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":2,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":0,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[],"osds":[{"osd":0,"uuid":"a2f79028-b332-4166-9ebb-a7024a37ea22","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":1,"uuid":"973159de-c2ea-4083-aecc-69373d1c1132","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":2,"uuid":"4ae7a0e9-cb3f-484c-a75f-f3766b721c47","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":3,"uuid":"1ba4a746-1608-47a7-ace2-960a5cfae934","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":4,"uuid":"d4f59f5d-0413-4b79-87ee-bb212c1fcf62","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":5,"uuid":"69710a0e-41d8-4e7f-9a10-17ce540b67be","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":6,"uuid":"f29dd047-3b48-473d-a235-6642a6293f20","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":7,"uuid":"6e691644-378b-4fdc-8e86-fb5f56e31f29","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-04-25T10:28:46.485 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:28:46.482+0000 7fdc08cbd640 1 -- 172.21.15.37:0/4269032516 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fdc040ce360 msgr2=0x7fdc040cce10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:28:46.485 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:28:46.482+0000 7fdc08cbd640 1 --2- 172.21.15.37:0/4269032516 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fdc040ce360 0x7fdc040cce10 secure :-1 s=READY pgs=23 cs=0 l=1 rev1=1 crypto rx=0x7fdbec00e870 tx=0x7fdbec00ed30 comp rx=0 tx=0).stop 2024-04-25T10:28:46.485 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:28:46.482+0000 7fdc08cbd640 1 -- 172.21.15.37:0/4269032516 shutdown_connections 2024-04-25T10:28:46.485 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:28:46.482+0000 7fdc08cbd640 1 --2- 172.21.15.37:0/4269032516 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7fdc040d27c0 0x7fdc04158e70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:28:46.485 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:28:46.482+0000 7fdc08cbd640 1 --2- 172.21.15.37:0/4269032516 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fdc040ce360 0x7fdc040cce10 unknown :-1 s=CLOSED pgs=23 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:28:46.485 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:28:46.482+0000 7fdc08cbd640 1 --2- 172.21.15.37:0/4269032516 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fdc040700f0 0x7fdc04153cc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:28:46.485 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:28:46.482+0000 7fdc08cbd640 1 -- 172.21.15.37:0/4269032516 >> 172.21.15.37:0/4269032516 conn(0x7fdc040d0460 msgr2=0x7fdc040d2bd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:28:46.485 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:28:46.482+0000 7fdc08cbd640 1 -- 172.21.15.37:0/4269032516 shutdown_connections 2024-04-25T10:28:46.485 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:28:46.482+0000 7fdc08cbd640 1 -- 172.21.15.37:0/4269032516 wait complete. 2024-04-25T10:28:46.495 INFO:tasks.ceph.ceph_manager.ceph:[] 2024-04-25T10:28:46.495 INFO:tasks.ceph:Waiting for OSDs to come up 2024-04-25T10:28:46.566 INFO:tasks.ceph.mgr.y.smithi037.stderr:2024-04-25T10:28:46.564+0000 7f1c9c0f7040 -1 mgr[py] Module rook has missing NOTIFY_TYPES member 2024-04-25T10:28:46.621 INFO:tasks.ceph.mgr.x.smithi203.stderr:2024-04-25T10:28:46.620+0000 7fbc1bc32040 -1 mgr[py] Module volumes has missing NOTIFY_TYPES member 2024-04-25T10:28:46.674 INFO:tasks.ceph.mgr.x.smithi203.stderr:2024-04-25T10:28:46.673+0000 7fbc1bc32040 -1 mgr[py] Module crash has missing NOTIFY_TYPES member 2024-04-25T10:28:46.789 INFO:tasks.ceph.mgr.x.smithi203.stderr:2024-04-25T10:28:46.788+0000 7fbc1bc32040 -1 mgr[py] Module rgw has missing NOTIFY_TYPES member 2024-04-25T10:28:46.836 INFO:tasks.ceph.mgr.y.smithi037.stderr:2024-04-25T10:28:46.834+0000 7f1c9c0f7040 -1 mgr[py] Module volumes has missing NOTIFY_TYPES member 2024-04-25T10:28:46.867 INFO:tasks.ceph.osd.1.smithi037.stderr:2024-04-25T10:28:46.864+0000 7f98f2b18640 -1 Falling back to public interface 2024-04-25T10:28:46.867 INFO:tasks.ceph.osd.4.smithi203.stderr:2024-04-25T10:28:46.865+0000 7fccbf019640 -1 Falling back to public interface 2024-04-25T10:28:46.870 INFO:tasks.ceph.osd.7.smithi203.stderr:2024-04-25T10:28:46.868+0000 7ff62c00c640 -1 Falling back to public interface 2024-04-25T10:28:46.870 INFO:tasks.ceph.osd.6.smithi203.stderr:2024-04-25T10:28:46.869+0000 7f19a0755640 -1 Falling back to public interface 2024-04-25T10:28:46.874 INFO:tasks.ceph.osd.2.smithi037.stderr:2024-04-25T10:28:46.872+0000 7f6172425640 -1 Falling back to public interface 2024-04-25T10:28:46.880 INFO:tasks.ceph.osd.0.smithi037.stderr:2024-04-25T10:28:46.878+0000 7f10b9e9d640 -1 Falling back to public interface 2024-04-25T10:28:46.886 INFO:tasks.ceph.osd.3.smithi037.stderr:2024-04-25T10:28:46.884+0000 7fa57dc0f640 -1 Falling back to public interface 2024-04-25T10:28:46.891 INFO:tasks.ceph.mgr.y.smithi037.stderr:2024-04-25T10:28:46.889+0000 7f1c9c0f7040 -1 mgr[py] Module crash has missing NOTIFY_TYPES member 2024-04-25T10:28:46.903 INFO:tasks.ceph.osd.5.smithi203.stderr:2024-04-25T10:28:46.902+0000 7fd976b50640 -1 Falling back to public interface 2024-04-25T10:28:47.014 INFO:tasks.ceph.mgr.y.smithi037.stderr:2024-04-25T10:28:47.012+0000 7f1c9c0f7040 -1 mgr[py] Module rgw has missing NOTIFY_TYPES member 2024-04-25T10:28:47.074 INFO:tasks.ceph.mgr.x.smithi203.stderr:2024-04-25T10:28:47.073+0000 7fbc1bc32040 -1 mgr[py] Module prometheus has missing NOTIFY_TYPES member 2024-04-25T10:28:47.121 INFO:tasks.ceph.mgr.x.smithi203.stderr:2024-04-25T10:28:47.120+0000 7fbc1bc32040 -1 mgr[py] Module devicehealth has missing NOTIFY_TYPES member 2024-04-25T10:28:47.278 INFO:tasks.ceph.mgr.x.smithi203.stderr:2024-04-25T10:28:47.276+0000 7fbc1bc32040 -1 mgr[py] Module telemetry has missing NOTIFY_TYPES member 2024-04-25T10:28:47.301 DEBUG:teuthology.orchestra.run.smithi037:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph --cluster ceph osd dump --format=json 2024-04-25T10:28:47.307 INFO:tasks.ceph.mgr.y.smithi037.stderr:2024-04-25T10:28:47.306+0000 7f1c9c0f7040 -1 mgr[py] Module prometheus has missing NOTIFY_TYPES member 2024-04-25T10:28:47.359 INFO:tasks.ceph.mgr.y.smithi037.stderr:2024-04-25T10:28:47.357+0000 7f1c9c0f7040 -1 mgr[py] Module devicehealth has missing NOTIFY_TYPES member 2024-04-25T10:28:47.404 INFO:teuthology.misc.health.smithi037.stderr:2024-04-25T10:28:47.402+0000 7f60094e0640 1 Processor -- start 2024-04-25T10:28:47.405 INFO:teuthology.misc.health.smithi037.stderr:2024-04-25T10:28:47.402+0000 7f60094e0640 1 -- start start 2024-04-25T10:28:47.406 INFO:teuthology.misc.health.smithi037.stderr:2024-04-25T10:28:47.403+0000 7f60094e0640 1 --2- >> v2:172.21.15.203:3300/0 conn(0x7f60040b4cc0 0x7f60040d1320 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:28:47.406 INFO:teuthology.misc.health.smithi037.stderr:2024-04-25T10:28:47.404+0000 7f60094e0640 1 --2- >> v2:172.21.15.37:3300/0 conn(0x7f60040d53c0 0x7f60040b9120 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:28:47.407 INFO:teuthology.misc.health.smithi037.stderr:2024-04-25T10:28:47.404+0000 7f60094e0640 1 -- --> v1:172.21.15.203:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f60040d3f10 con 0x7f60040d4fb0 2024-04-25T10:28:47.407 INFO:teuthology.misc.health.smithi037.stderr:2024-04-25T10:28:47.404+0000 7f60094e0640 1 -- --> v2:172.21.15.37:3300/0 -- mon_getmap magic: 0 -- 0x7f60040d4b20 con 0x7f60040d53c0 2024-04-25T10:28:47.407 INFO:teuthology.misc.health.smithi037.stderr:2024-04-25T10:28:47.404+0000 7f60094e0640 1 -- --> v2:172.21.15.203:3300/0 -- mon_getmap magic: 0 -- 0x7f60040d41f0 con 0x7f60040b4cc0 2024-04-25T10:28:47.407 INFO:teuthology.misc.health.smithi037.stderr:2024-04-25T10:28:47.404+0000 7f60037fe640 1 --2- >> v2:172.21.15.37:3300/0 conn(0x7f60040d53c0 0x7f60040b9120 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:28:47.407 INFO:teuthology.misc.health.smithi037.stderr:2024-04-25T10:28:47.404+0000 7f60037fe640 1 --2- >> v2:172.21.15.37:3300/0 conn(0x7f60040d53c0 0x7f60040b9120 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.37:3300/0 says I am v2:172.21.15.37:42446/0 (socket says 172.21.15.37:42446) 2024-04-25T10:28:47.407 INFO:teuthology.misc.health.smithi037.stderr:2024-04-25T10:28:47.404+0000 7f60037fe640 1 -- 172.21.15.37:0/3645360768 learned_addr learned my addr 172.21.15.37:0/3645360768 (peer_addr_for_me v2:172.21.15.37:0/0) 2024-04-25T10:28:47.407 INFO:teuthology.misc.health.smithi037.stderr:2024-04-25T10:28:47.404+0000 7f6002ffd640 1 --2- 172.21.15.37:0/3645360768 >> v2:172.21.15.203:3300/0 conn(0x7f60040b4cc0 0x7f60040d1320 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:28:47.407 INFO:teuthology.misc.health.smithi037.stderr:2024-04-25T10:28:47.404+0000 7f60037fe640 1 -- 172.21.15.37:0/3645360768 >> v2:172.21.15.203:3300/0 conn(0x7f60040b4cc0 msgr2=0x7f60040d1320 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:28:47.407 INFO:teuthology.misc.health.smithi037.stderr:2024-04-25T10:28:47.404+0000 7f60037fe640 1 --2- 172.21.15.37:0/3645360768 >> v2:172.21.15.203:3300/0 conn(0x7f60040b4cc0 0x7f60040d1320 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:28:47.407 INFO:teuthology.misc.health.smithi037.stderr:2024-04-25T10:28:47.404+0000 7f60037fe640 1 -- 172.21.15.37:0/3645360768 >> v1:172.21.15.203:6789/0 conn(0x7f60040d4fb0 legacy=0x7f60040d23e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:28:47.407 INFO:teuthology.misc.health.smithi037.stderr:2024-04-25T10:28:47.404+0000 7f60037fe640 1 -- 172.21.15.37:0/3645360768 --> v2:172.21.15.37:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f60040d44d0 con 0x7f60040d53c0 2024-04-25T10:28:47.408 INFO:teuthology.misc.health.smithi037.stderr:2024-04-25T10:28:47.405+0000 7f60037fe640 1 --2- 172.21.15.37:0/3645360768 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f60040d53c0 0x7f60040b9120 secure :-1 s=READY pgs=24 cs=0 l=1 rev1=1 crypto rx=0x7f60040d3d00 tx=0x7f5ffc02a6a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=70744b38041d21c8 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:28:47.409 INFO:teuthology.misc.health.smithi037.stderr:2024-04-25T10:28:47.405+0000 7f6001ffb640 1 -- 172.21.15.37:0/3645360768 <== mon.0 v2:172.21.15.37:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5ffc036070 con 0x7f60040d53c0 2024-04-25T10:28:47.409 INFO:teuthology.misc.health.smithi037.stderr:2024-04-25T10:28:47.405+0000 7f6001ffb640 1 -- 172.21.15.37:0/3645360768 <== mon.0 v2:172.21.15.37:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f5ffc031600 con 0x7f60040d53c0 2024-04-25T10:28:47.409 INFO:teuthology.misc.health.smithi037.stderr:2024-04-25T10:28:47.405+0000 7f6001ffb640 1 -- 172.21.15.37:0/3645360768 <== mon.0 v2:172.21.15.37:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5ffc0319d0 con 0x7f60040d53c0 2024-04-25T10:28:47.409 INFO:teuthology.misc.health.smithi037.stderr:2024-04-25T10:28:47.406+0000 7f60094e0640 1 -- 172.21.15.37:0/3645360768 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f60040d53c0 msgr2=0x7f60040b9120 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:28:47.409 INFO:teuthology.misc.health.smithi037.stderr:2024-04-25T10:28:47.406+0000 7f60094e0640 1 --2- 172.21.15.37:0/3645360768 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f60040d53c0 0x7f60040b9120 secure :-1 s=READY pgs=24 cs=0 l=1 rev1=1 crypto rx=0x7f60040d3d00 tx=0x7f5ffc02a6a0 comp rx=0 tx=0).stop 2024-04-25T10:28:47.409 INFO:teuthology.misc.health.smithi037.stderr:2024-04-25T10:28:47.406+0000 7f60094e0640 1 -- 172.21.15.37:0/3645360768 shutdown_connections 2024-04-25T10:28:47.409 INFO:teuthology.misc.health.smithi037.stderr:2024-04-25T10:28:47.406+0000 7f60094e0640 1 --2- 172.21.15.37:0/3645360768 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f60040d53c0 0x7f60040b9120 unknown :-1 s=CLOSED pgs=24 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:28:47.409 INFO:teuthology.misc.health.smithi037.stderr:2024-04-25T10:28:47.406+0000 7f60094e0640 1 --2- 172.21.15.37:0/3645360768 >> v2:172.21.15.203:3300/0 conn(0x7f60040b4cc0 0x7f60040d1320 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:28:47.409 INFO:teuthology.misc.health.smithi037.stderr:2024-04-25T10:28:47.406+0000 7f60094e0640 1 -- 172.21.15.37:0/3645360768 >> 172.21.15.37:0/3645360768 conn(0x7f600406dc90 msgr2=0x7f600406e0a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:28:47.409 INFO:teuthology.misc.health.smithi037.stderr:2024-04-25T10:28:47.406+0000 7f60094e0640 1 -- 172.21.15.37:0/3645360768 shutdown_connections 2024-04-25T10:28:47.409 INFO:teuthology.misc.health.smithi037.stderr:2024-04-25T10:28:47.406+0000 7f60094e0640 1 -- 172.21.15.37:0/3645360768 wait complete. 2024-04-25T10:28:47.410 INFO:teuthology.misc.health.smithi037.stderr:2024-04-25T10:28:47.408+0000 7f60094e0640 1 Processor -- start 2024-04-25T10:28:47.410 INFO:teuthology.misc.health.smithi037.stderr:2024-04-25T10:28:47.408+0000 7f60094e0640 1 -- start start 2024-04-25T10:28:47.413 INFO:teuthology.misc.health.smithi037.stderr:2024-04-25T10:28:47.409+0000 7f60094e0640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f60040b4cc0 0x7f60041565c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:28:47.413 INFO:teuthology.misc.health.smithi037.stderr:2024-04-25T10:28:47.409+0000 7f60094e0640 1 --2- >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f60040d4fb0 0x7f60041583f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:28:47.413 INFO:teuthology.misc.health.smithi037.stderr:2024-04-25T10:28:47.409+0000 7f60094e0640 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f60040d53c0 0x7f6004158930 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:28:47.413 INFO:teuthology.misc.health.smithi037.stderr:2024-04-25T10:28:47.409+0000 7f60094e0640 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 -- 0x7f6004160460 con 0x7f60040b4cc0 2024-04-25T10:28:47.413 INFO:teuthology.misc.health.smithi037.stderr:2024-04-25T10:28:47.409+0000 7f60094e0640 1 -- --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_getmap magic: 0 -- 0x7f60040d2af0 con 0x7f60040d53c0 2024-04-25T10:28:47.413 INFO:teuthology.misc.health.smithi037.stderr:2024-04-25T10:28:47.409+0000 7f60094e0640 1 -- --> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] -- mon_getmap magic: 0 -- 0x7f60041f65a0 con 0x7f60040d4fb0 2024-04-25T10:28:47.413 INFO:teuthology.misc.health.smithi037.stderr:2024-04-25T10:28:47.410+0000 7f6002ffd640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f60040b4cc0 0x7f60041565c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:28:47.413 INFO:teuthology.misc.health.smithi037.stderr:2024-04-25T10:28:47.410+0000 7f6002ffd640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f60040b4cc0 0x7f60041565c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.37:3300/0 says I am v2:172.21.15.37:42452/0 (socket says 172.21.15.37:42452) 2024-04-25T10:28:47.413 INFO:teuthology.misc.health.smithi037.stderr:2024-04-25T10:28:47.410+0000 7f6002ffd640 1 -- 172.21.15.37:0/714116744 learned_addr learned my addr 172.21.15.37:0/714116744 (peer_addr_for_me v2:172.21.15.37:0/0) 2024-04-25T10:28:47.414 INFO:teuthology.misc.health.smithi037.stderr:2024-04-25T10:28:47.410+0000 7f60037fe640 1 --2- 172.21.15.37:0/714116744 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f60040d53c0 0x7f6004158930 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:28:47.414 INFO:teuthology.misc.health.smithi037.stderr:2024-04-25T10:28:47.410+0000 7f60027fc640 1 --2- 172.21.15.37:0/714116744 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f60040d4fb0 0x7f60041583f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:28:47.414 INFO:teuthology.misc.health.smithi037.stderr:2024-04-25T10:28:47.410+0000 7f6002ffd640 1 -- 172.21.15.37:0/714116744 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f60040d4fb0 msgr2=0x7f60041583f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:28:47.414 INFO:teuthology.misc.health.smithi037.stderr:2024-04-25T10:28:47.410+0000 7f6002ffd640 1 --2- 172.21.15.37:0/714116744 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f60040d4fb0 0x7f60041583f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:28:47.414 INFO:teuthology.misc.health.smithi037.stderr:2024-04-25T10:28:47.410+0000 7f6002ffd640 1 -- 172.21.15.37:0/714116744 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f60040d53c0 msgr2=0x7f6004158930 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:28:47.414 INFO:teuthology.misc.health.smithi037.stderr:2024-04-25T10:28:47.410+0000 7f6002ffd640 1 --2- 172.21.15.37:0/714116744 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f60040d53c0 0x7f6004158930 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:28:47.414 INFO:teuthology.misc.health.smithi037.stderr:2024-04-25T10:28:47.410+0000 7f6002ffd640 1 -- 172.21.15.37:0/714116744 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f60041570b0 con 0x7f60040b4cc0 2024-04-25T10:28:47.414 INFO:teuthology.misc.health.smithi037.stderr:2024-04-25T10:28:47.410+0000 7f6002ffd640 1 --2- 172.21.15.37:0/714116744 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f60040b4cc0 0x7f60041565c0 secure :-1 s=READY pgs=25 cs=0 l=1 rev1=1 crypto rx=0x7f5ff800e8a0 tx=0x7f5ff800ed60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:28:47.414 INFO:teuthology.misc.health.smithi037.stderr:2024-04-25T10:28:47.410+0000 7f5feb7fe640 1 -- 172.21.15.37:0/714116744 <== mon.0 v2:172.21.15.37:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5ff8017070 con 0x7f60040b4cc0 2024-04-25T10:28:47.414 INFO:teuthology.misc.health.smithi037.stderr:2024-04-25T10:28:47.410+0000 7f5feb7fe640 1 -- 172.21.15.37:0/714116744 <== mon.0 v2:172.21.15.37:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f5ff800c940 con 0x7f60040b4cc0 2024-04-25T10:28:47.414 INFO:teuthology.misc.health.smithi037.stderr:2024-04-25T10:28:47.410+0000 7f60094e0640 1 -- 172.21.15.37:0/714116744 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f6004162db0 con 0x7f60040b4cc0 2024-04-25T10:28:47.414 INFO:teuthology.misc.health.smithi037.stderr:2024-04-25T10:28:47.410+0000 7f5feb7fe640 1 -- 172.21.15.37:0/714116744 <== mon.0 v2:172.21.15.37:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5ff800cd30 con 0x7f60040b4cc0 2024-04-25T10:28:47.414 INFO:teuthology.misc.health.smithi037.stderr:2024-04-25T10:28:47.411+0000 7f60094e0640 1 -- 172.21.15.37:0/714116744 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f6004157910 con 0x7f60040b4cc0 2024-04-25T10:28:47.415 INFO:teuthology.misc.health.smithi037.stderr:2024-04-25T10:28:47.411+0000 7f5feb7fe640 1 -- 172.21.15.37:0/714116744 <== mon.0 v2:172.21.15.37:3300/0 4 ==== mgrmap(e 1) ==== 807+0+0 (secure 0 0 0) 0x7f5ff801a070 con 0x7f60040b4cc0 2024-04-25T10:28:47.415 INFO:teuthology.misc.health.smithi037.stderr:2024-04-25T10:28:47.411+0000 7f5feb7fe640 1 -- 172.21.15.37:0/714116744 <== mon.0 v2:172.21.15.37:3300/0 5 ==== osd_map(10..10 src has 1..10) ==== 1645+0+0 (secure 0 0 0) 0x7f5ff8006020 con 0x7f60040b4cc0 2024-04-25T10:28:47.415 INFO:tasks.ceph.mgr.x.smithi203.stderr:2024-04-25T10:28:47.413+0000 7fbc1bc32040 -1 mgr[py] Module diskprediction_local has missing NOTIFY_TYPES member 2024-04-25T10:28:47.415 INFO:teuthology.misc.health.smithi037.stderr:2024-04-25T10:28:47.411+0000 7f60094e0640 1 -- 172.21.15.37:0/714116744 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f6004050740 con 0x7f60040b4cc0 2024-04-25T10:28:47.416 INFO:teuthology.misc.health.smithi037.stderr:2024-04-25T10:28:47.414+0000 7f5feb7fe640 1 -- 172.21.15.37:0/714116744 <== mon.0 v2:172.21.15.37:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+73265 (secure 0 0 0) 0x7f60041570b0 con 0x7f60040b4cc0 2024-04-25T10:28:47.466 INFO:tasks.ceph.mgr.x.smithi203.stderr:2024-04-25T10:28:47.465+0000 7fbc1bc32040 -1 mgr[py] Module osd_perf_query has missing NOTIFY_TYPES member 2024-04-25T10:28:47.478 INFO:teuthology.misc.health.smithi037.stderr:2024-04-25T10:28:47.476+0000 7f60094e0640 1 -- 172.21.15.37:0/714116744 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7f600415e810 con 0x7f60040b4cc0 2024-04-25T10:28:47.479 INFO:teuthology.misc.health.smithi037.stderr:2024-04-25T10:28:47.477+0000 7f5feb7fe640 1 -- 172.21.15.37:0/714116744 <== mon.0 v2:172.21.15.37:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v10) ==== 74+0+7058 (secure 0 0 0) 0x7f5ff8006480 con 0x7f60040b4cc0 2024-04-25T10:28:47.479 INFO:teuthology.misc.health.smithi037.stdout: 2024-04-25T10:28:47.479 INFO:teuthology.misc.health.smithi037.stdout:{"epoch":10,"fsid":"f4acda28-be34-492a-b286-994afd36f61d","created":"2024-04-25T10:28:43.985718+0000","modified":"2024-04-25T10:28:46.216973+0000","last_up_change":"0.000000","last_in_change":"2024-04-25T10:28:46.216973+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":2,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":0,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[],"osds":[{"osd":0,"uuid":"a2f79028-b332-4166-9ebb-a7024a37ea22","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":1,"uuid":"973159de-c2ea-4083-aecc-69373d1c1132","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":2,"uuid":"4ae7a0e9-cb3f-484c-a75f-f3766b721c47","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":3,"uuid":"1ba4a746-1608-47a7-ace2-960a5cfae934","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":4,"uuid":"d4f59f5d-0413-4b79-87ee-bb212c1fcf62","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":5,"uuid":"69710a0e-41d8-4e7f-9a10-17ce540b67be","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":6,"uuid":"f29dd047-3b48-473d-a235-6642a6293f20","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":7,"uuid":"6e691644-378b-4fdc-8e86-fb5f56e31f29","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-04-25T10:28:47.480 INFO:teuthology.misc.health.smithi037.stderr:2024-04-25T10:28:47.478+0000 7f60094e0640 1 -- 172.21.15.37:0/714116744 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f60040b4cc0 msgr2=0x7f60041565c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:28:47.480 INFO:teuthology.misc.health.smithi037.stderr:2024-04-25T10:28:47.478+0000 7f60094e0640 1 --2- 172.21.15.37:0/714116744 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f60040b4cc0 0x7f60041565c0 secure :-1 s=READY pgs=25 cs=0 l=1 rev1=1 crypto rx=0x7f5ff800e8a0 tx=0x7f5ff800ed60 comp rx=0 tx=0).stop 2024-04-25T10:28:47.480 INFO:teuthology.misc.health.smithi037.stderr:2024-04-25T10:28:47.478+0000 7f60094e0640 1 -- 172.21.15.37:0/714116744 shutdown_connections 2024-04-25T10:28:47.480 INFO:teuthology.misc.health.smithi037.stderr:2024-04-25T10:28:47.478+0000 7f60094e0640 1 --2- 172.21.15.37:0/714116744 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f60040d53c0 0x7f6004158930 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:28:47.481 INFO:teuthology.misc.health.smithi037.stderr:2024-04-25T10:28:47.478+0000 7f60094e0640 1 --2- 172.21.15.37:0/714116744 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f60040d4fb0 0x7f60041583f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:28:47.481 INFO:teuthology.misc.health.smithi037.stderr:2024-04-25T10:28:47.478+0000 7f60094e0640 1 --2- 172.21.15.37:0/714116744 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f60040b4cc0 0x7f60041565c0 unknown :-1 s=CLOSED pgs=25 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:28:47.481 INFO:teuthology.misc.health.smithi037.stderr:2024-04-25T10:28:47.478+0000 7f60094e0640 1 -- 172.21.15.37:0/714116744 >> 172.21.15.37:0/714116744 conn(0x7f600406dc90 msgr2=0x7f600415ad40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:28:47.481 INFO:teuthology.misc.health.smithi037.stderr:2024-04-25T10:28:47.478+0000 7f60094e0640 1 -- 172.21.15.37:0/714116744 shutdown_connections 2024-04-25T10:28:47.481 INFO:teuthology.misc.health.smithi037.stderr:2024-04-25T10:28:47.478+0000 7f60094e0640 1 -- 172.21.15.37:0/714116744 wait complete. 2024-04-25T10:28:47.490 DEBUG:teuthology.misc:0 of 8 OSDs are up 2024-04-25T10:28:47.529 INFO:tasks.ceph.mgr.y.smithi037.stderr:2024-04-25T10:28:47.526+0000 7f1c9c0f7040 -1 mgr[py] Module telemetry has missing NOTIFY_TYPES member 2024-04-25T10:28:47.534 INFO:tasks.ceph.mgr.x.smithi203.stderr:2024-04-25T10:28:47.533+0000 7fbc1bc32040 -1 mgr[py] Module rbd_support has missing NOTIFY_TYPES member 2024-04-25T10:28:47.579 INFO:tasks.ceph.mgr.x.smithi203.stderr:2024-04-25T10:28:47.578+0000 7fbc1bc32040 -1 mgr[py] Module zabbix has missing NOTIFY_TYPES member 2024-04-25T10:28:47.672 INFO:tasks.ceph.mgr.y.smithi037.stderr:2024-04-25T10:28:47.669+0000 7f1c9c0f7040 -1 mgr[py] Module diskprediction_local has missing NOTIFY_TYPES member 2024-04-25T10:28:47.723 INFO:tasks.ceph.mgr.y.smithi037.stderr:2024-04-25T10:28:47.721+0000 7f1c9c0f7040 -1 mgr[py] Module osd_perf_query has missing NOTIFY_TYPES member 2024-04-25T10:28:47.797 INFO:tasks.ceph.mgr.y.smithi037.stderr:2024-04-25T10:28:47.795+0000 7f1c9c0f7040 -1 mgr[py] Module rbd_support has missing NOTIFY_TYPES member 2024-04-25T10:28:47.844 INFO:tasks.ceph.mgr.y.smithi037.stderr:2024-04-25T10:28:47.842+0000 7f1c9c0f7040 -1 mgr[py] Module zabbix has missing NOTIFY_TYPES member 2024-04-25T10:28:48.058 INFO:tasks.ceph.mgr.x.smithi203.stderr:2024-04-25T10:28:48.057+0000 7fbc1bc32040 -1 mgr[py] Module status has missing NOTIFY_TYPES member 2024-04-25T10:28:48.173 INFO:tasks.ceph.mgr.x.smithi203.stderr:2024-04-25T10:28:48.171+0000 7fbc1bc32040 -1 mgr[py] Module nfs has missing NOTIFY_TYPES member 2024-04-25T10:28:48.347 INFO:tasks.ceph.mgr.y.smithi037.stderr:2024-04-25T10:28:48.345+0000 7f1c9c0f7040 -1 mgr[py] Module status has missing NOTIFY_TYPES member 2024-04-25T10:28:48.468 INFO:tasks.ceph.mgr.y.smithi037.stderr:2024-04-25T10:28:48.466+0000 7f1c9c0f7040 -1 mgr[py] Module nfs has missing NOTIFY_TYPES member 2024-04-25T10:28:49.191 INFO:tasks.ceph.mgr.x.smithi203.stderr:2024-04-25T10:28:49.190+0000 7fbbbb740640 -1 mgr.server handle_report got status from non-daemon mon.c 2024-04-25T10:28:49.192 INFO:tasks.ceph.mgr.x.smithi203.stderr:2024-04-25T10:28:49.191+0000 7fbbbb740640 -1 mgr.server handle_report got status from non-daemon mon.b 2024-04-25T10:28:50.115 INFO:tasks.ceph.osd.4.smithi203.stderr:2024-04-25T10:28:50.113+0000 7fccbf019640 -1 osd.4 0 log_to_monitors true 2024-04-25T10:28:50.335 INFO:tasks.ceph.osd.1.smithi037.stderr:2024-04-25T10:28:50.333+0000 7f98f2b18640 -1 osd.1 0 log_to_monitors true 2024-04-25T10:28:50.382 INFO:tasks.ceph.osd.0.smithi037.stderr:2024-04-25T10:28:50.380+0000 7f10b9e9d640 -1 osd.0 0 log_to_monitors true 2024-04-25T10:28:50.383 INFO:tasks.ceph.osd.2.smithi037.stderr:2024-04-25T10:28:50.381+0000 7f6172425640 -1 osd.2 0 log_to_monitors true 2024-04-25T10:28:50.396 INFO:tasks.ceph.osd.7.smithi203.stderr:2024-04-25T10:28:50.395+0000 7ff62c00c640 -1 osd.7 0 log_to_monitors true 2024-04-25T10:28:50.396 INFO:tasks.ceph.osd.6.smithi203.stderr:2024-04-25T10:28:50.395+0000 7f19a0755640 -1 osd.6 0 log_to_monitors true 2024-04-25T10:28:50.398 INFO:tasks.ceph.osd.3.smithi037.stderr:2024-04-25T10:28:50.396+0000 7fa57dc0f640 -1 osd.3 0 log_to_monitors true 2024-04-25T10:28:50.413 INFO:tasks.ceph.osd.5.smithi203.stderr:2024-04-25T10:28:50.412+0000 7fd976b50640 -1 osd.5 0 log_to_monitors true 2024-04-25T10:28:52.098 INFO:tasks.ceph.osd.4.smithi203.stderr:2024-04-25T10:28:52.096+0000 7fccba92c640 -1 osd.4 0 waiting for initial osdmap 2024-04-25T10:28:52.107 INFO:tasks.ceph.osd.4.smithi203.stderr:2024-04-25T10:28:52.106+0000 7fccb2727640 -1 osd.4 12 set_numa_affinity unable to identify public interface '' numa node: (2) No such file or directory 2024-04-25T10:28:53.899 INFO:tasks.ceph.osd.7.smithi203.stderr:2024-04-25T10:28:53.898+0000 7ff62788c640 -1 osd.7 0 waiting for initial osdmap 2024-04-25T10:28:53.911 INFO:tasks.ceph.osd.7.smithi203.stderr:2024-04-25T10:28:53.910+0000 7ff61f687640 -1 osd.7 14 set_numa_affinity unable to identify public interface '' numa node: (2) No such file or directory 2024-04-25T10:28:53.946 INFO:tasks.ceph.osd.6.smithi203.stderr:2024-04-25T10:28:53.945+0000 7f199c6eb640 -1 osd.6 0 waiting for initial osdmap 2024-04-25T10:28:53.952 INFO:tasks.ceph.osd.5.smithi203.stderr:2024-04-25T10:28:53.951+0000 7fd9732fa640 -1 osd.5 0 waiting for initial osdmap 2024-04-25T10:28:53.958 INFO:tasks.ceph.osd.6.smithi203.stderr:2024-04-25T10:28:53.957+0000 7f19944e6640 -1 osd.6 14 set_numa_affinity unable to identify public interface '' numa node: (2) No such file or directory 2024-04-25T10:28:53.970 INFO:tasks.ceph.osd.5.smithi203.stderr:2024-04-25T10:28:53.969+0000 7fd96a8e1640 -1 osd.5 14 set_numa_affinity unable to identify public interface '' numa node: (2) No such file or directory 2024-04-25T10:28:54.297 DEBUG:teuthology.orchestra.run.smithi037:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph --cluster ceph osd dump --format=json 2024-04-25T10:28:54.394 INFO:tasks.ceph.osd.3.smithi037.stderr:2024-04-25T10:28:54.392+0000 7fa579ca0640 -1 osd.3 0 waiting for initial osdmap 2024-04-25T10:28:54.402 INFO:tasks.ceph.osd.2.smithi037.stderr:2024-04-25T10:28:54.400+0000 7f616de67640 -1 osd.2 0 waiting for initial osdmap 2024-04-25T10:28:54.408 INFO:tasks.ceph.osd.3.smithi037.stderr:2024-04-25T10:28:54.406+0000 7fa571287640 -1 osd.3 14 set_numa_affinity unable to identify public interface '' numa node: (2) No such file or directory 2024-04-25T10:28:54.417 INFO:tasks.ceph.osd.2.smithi037.stderr:2024-04-25T10:28:54.415+0000 7f6165c62640 -1 osd.2 14 set_numa_affinity unable to identify public interface '' numa node: (2) No such file or directory 2024-04-25T10:28:54.443 INFO:teuthology.misc.health.smithi037.stderr:2024-04-25T10:28:54.441+0000 7fcb1f1be640 1 Processor -- start 2024-04-25T10:28:54.443 INFO:teuthology.misc.health.smithi037.stderr:2024-04-25T10:28:54.441+0000 7fcb1f1be640 1 -- start start 2024-04-25T10:28:54.445 INFO:teuthology.misc.health.smithi037.stderr:2024-04-25T10:28:54.442+0000 7fcb1f1be640 1 --2- >> v2:172.21.15.37:3300/0 conn(0x7fcb1815abd0 0x7fcb18161490 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:28:54.445 INFO:teuthology.misc.health.smithi037.stderr:2024-04-25T10:28:54.442+0000 7fcb1f1be640 1 --2- >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7fcb180b42f0 0x7fcb180b4700 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:28:54.445 INFO:teuthology.misc.health.smithi037.stderr:2024-04-25T10:28:54.442+0000 7fcb1f1be640 1 -- --> v1:172.21.15.37:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fcb180b4c40 con 0x7fcb180ccec0 2024-04-25T10:28:54.445 INFO:teuthology.misc.health.smithi037.stderr:2024-04-25T10:28:54.442+0000 7fcb1f1be640 1 -- --> v2:172.21.15.37:3300/0 -- mon_getmap magic: 0 -- 0x7fcb1815afe0 con 0x7fcb1815abd0 2024-04-25T10:28:54.445 INFO:teuthology.misc.health.smithi037.stderr:2024-04-25T10:28:54.442+0000 7fcb1f1be640 1 -- --> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] -- mon_getmap magic: 0 -- 0x7fcb180cf0e0 con 0x7fcb180b42f0 2024-04-25T10:28:54.445 INFO:teuthology.misc.health.smithi037.stderr:2024-04-25T10:28:54.442+0000 7fcb17fff640 1 --2- >> v2:172.21.15.37:3300/0 conn(0x7fcb1815abd0 0x7fcb18161490 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:28:54.445 INFO:teuthology.misc.health.smithi037.stderr:2024-04-25T10:28:54.442+0000 7fcb17fff640 1 --2- >> v2:172.21.15.37:3300/0 conn(0x7fcb1815abd0 0x7fcb18161490 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.37:3300/0 says I am v2:172.21.15.37:58218/0 (socket says 172.21.15.37:58218) 2024-04-25T10:28:54.446 INFO:teuthology.misc.health.smithi037.stderr:2024-04-25T10:28:54.442+0000 7fcb17fff640 1 -- 172.21.15.37:0/1343873474 learned_addr learned my addr 172.21.15.37:0/1343873474 (peer_addr_for_me v2:172.21.15.37:0/0) 2024-04-25T10:28:54.446 INFO:teuthology.misc.health.smithi037.stderr:2024-04-25T10:28:54.443+0000 7fcb17fff640 1 -- 172.21.15.37:0/1343873474 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7fcb180b42f0 msgr2=0x7fcb180b4700 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:28:54.446 INFO:teuthology.misc.health.smithi037.stderr:2024-04-25T10:28:54.443+0000 7fcb17fff640 1 --2- 172.21.15.37:0/1343873474 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7fcb180b42f0 0x7fcb180b4700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:28:54.446 INFO:teuthology.misc.health.smithi037.stderr:2024-04-25T10:28:54.443+0000 7fcb17fff640 1 -- 172.21.15.37:0/1343873474 >> v1:172.21.15.37:6789/0 conn(0x7fcb180ccec0 legacy=0x7fcb1815dfc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:28:54.446 INFO:teuthology.misc.health.smithi037.stderr:2024-04-25T10:28:54.443+0000 7fcb17fff640 1 -- 172.21.15.37:0/1343873474 --> v2:172.21.15.37:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fcb18160cd0 con 0x7fcb1815abd0 2024-04-25T10:28:54.447 INFO:teuthology.misc.health.smithi037.stderr:2024-04-25T10:28:54.444+0000 7fcb17fff640 1 --2- 172.21.15.37:0/1343873474 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fcb1815abd0 0x7fcb18161490 secure :-1 s=READY pgs=30 cs=0 l=1 rev1=1 crypto rx=0x7fcb04009870 tx=0x7fcb0402c480 comp rx=0 tx=0).ready entity=mon.0 client_cookie=cca58af0ddf6df12 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:28:54.447 INFO:teuthology.misc.health.smithi037.stderr:2024-04-25T10:28:54.444+0000 7fcb177fe640 1 -- 172.21.15.37:0/1343873474 <== mon.0 v2:172.21.15.37:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcb04038070 con 0x7fcb1815abd0 2024-04-25T10:28:54.448 INFO:teuthology.misc.health.smithi037.stderr:2024-04-25T10:28:54.444+0000 7fcb177fe640 1 -- 172.21.15.37:0/1343873474 <== mon.0 v2:172.21.15.37:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fcb04031710 con 0x7fcb1815abd0 2024-04-25T10:28:54.448 INFO:teuthology.misc.health.smithi037.stderr:2024-04-25T10:28:54.444+0000 7fcb177fe640 1 -- 172.21.15.37:0/1343873474 <== mon.0 v2:172.21.15.37:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcb04031ae0 con 0x7fcb1815abd0 2024-04-25T10:28:54.448 INFO:teuthology.misc.health.smithi037.stderr:2024-04-25T10:28:54.445+0000 7fcb1f1be640 1 -- 172.21.15.37:0/1343873474 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fcb1815abd0 msgr2=0x7fcb18161490 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:28:54.448 INFO:teuthology.misc.health.smithi037.stderr:2024-04-25T10:28:54.445+0000 7fcb1f1be640 1 --2- 172.21.15.37:0/1343873474 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fcb1815abd0 0x7fcb18161490 secure :-1 s=READY pgs=30 cs=0 l=1 rev1=1 crypto rx=0x7fcb04009870 tx=0x7fcb0402c480 comp rx=0 tx=0).stop 2024-04-25T10:28:54.448 INFO:teuthology.misc.health.smithi037.stderr:2024-04-25T10:28:54.445+0000 7fcb1f1be640 1 -- 172.21.15.37:0/1343873474 shutdown_connections 2024-04-25T10:28:54.448 INFO:teuthology.misc.health.smithi037.stderr:2024-04-25T10:28:54.445+0000 7fcb1f1be640 1 --2- 172.21.15.37:0/1343873474 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7fcb180b42f0 0x7fcb180b4700 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:28:54.448 INFO:teuthology.misc.health.smithi037.stderr:2024-04-25T10:28:54.445+0000 7fcb1f1be640 1 --2- 172.21.15.37:0/1343873474 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fcb1815abd0 0x7fcb18161490 unknown :-1 s=CLOSED pgs=30 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:28:54.448 INFO:teuthology.misc.health.smithi037.stderr:2024-04-25T10:28:54.445+0000 7fcb1f1be640 1 -- 172.21.15.37:0/1343873474 >> 172.21.15.37:0/1343873474 conn(0x7fcb180cf2f0 msgr2=0x7fcb180cf700 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:28:54.448 INFO:teuthology.misc.health.smithi037.stderr:2024-04-25T10:28:54.445+0000 7fcb1f1be640 1 -- 172.21.15.37:0/1343873474 shutdown_connections 2024-04-25T10:28:54.448 INFO:teuthology.misc.health.smithi037.stderr:2024-04-25T10:28:54.445+0000 7fcb1f1be640 1 -- 172.21.15.37:0/1343873474 wait complete. 2024-04-25T10:28:54.449 INFO:teuthology.misc.health.smithi037.stderr:2024-04-25T10:28:54.447+0000 7fcb1f1be640 1 Processor -- start 2024-04-25T10:28:54.449 INFO:teuthology.misc.health.smithi037.stderr:2024-04-25T10:28:54.447+0000 7fcb1f1be640 1 -- start start 2024-04-25T10:28:54.451 INFO:teuthology.misc.health.smithi037.stderr:2024-04-25T10:28:54.448+0000 7fcb1f1be640 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fcb180b42f0 0x7fcb18156200 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:28:54.452 INFO:teuthology.misc.health.smithi037.stderr:2024-04-25T10:28:54.448+0000 7fcb1f1be640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fcb180ccec0 0x7fcb1814ac00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:28:54.452 INFO:teuthology.misc.health.smithi037.stderr:2024-04-25T10:28:54.448+0000 7fcb1f1be640 1 --2- >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7fcb1815abd0 0x7fcb1806c540 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:28:54.452 INFO:teuthology.misc.health.smithi037.stderr:2024-04-25T10:28:54.448+0000 7fcb1f1be640 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 -- 0x7fcb18160f70 con 0x7fcb180ccec0 2024-04-25T10:28:54.452 INFO:teuthology.misc.health.smithi037.stderr:2024-04-25T10:28:54.448+0000 7fcb1f1be640 1 -- --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_getmap magic: 0 -- 0x7fcb180cee00 con 0x7fcb180b42f0 2024-04-25T10:28:54.452 INFO:teuthology.misc.health.smithi037.stderr:2024-04-25T10:28:54.448+0000 7fcb1f1be640 1 -- --> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] -- mon_getmap magic: 0 -- 0x7fcb181619d0 con 0x7fcb1815abd0 2024-04-25T10:28:54.452 INFO:teuthology.misc.health.smithi037.stderr:2024-04-25T10:28:54.448+0000 7fcb17fff640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fcb180ccec0 0x7fcb1814ac00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:28:54.452 INFO:teuthology.misc.health.smithi037.stderr:2024-04-25T10:28:54.448+0000 7fcb17fff640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fcb180ccec0 0x7fcb1814ac00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.37:3300/0 says I am v2:172.21.15.37:58228/0 (socket says 172.21.15.37:58228) 2024-04-25T10:28:54.452 INFO:teuthology.misc.health.smithi037.stderr:2024-04-25T10:28:54.448+0000 7fcb17fff640 1 -- 172.21.15.37:0/2099962412 learned_addr learned my addr 172.21.15.37:0/2099962412 (peer_addr_for_me v2:172.21.15.37:0/0) 2024-04-25T10:28:54.452 INFO:teuthology.misc.health.smithi037.stderr:2024-04-25T10:28:54.448+0000 7fcb1cf5a640 1 --2- 172.21.15.37:0/2099962412 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fcb180b42f0 0x7fcb18156200 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:28:54.452 INFO:teuthology.misc.health.smithi037.stderr:2024-04-25T10:28:54.448+0000 7fcb1d75b640 1 --2- 172.21.15.37:0/2099962412 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7fcb1815abd0 0x7fcb1806c540 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:28:54.452 INFO:teuthology.misc.health.smithi037.stderr:2024-04-25T10:28:54.449+0000 7fcb17fff640 1 -- 172.21.15.37:0/2099962412 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7fcb1815abd0 msgr2=0x7fcb1806c540 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:28:54.452 INFO:teuthology.misc.health.smithi037.stderr:2024-04-25T10:28:54.449+0000 7fcb17fff640 1 --2- 172.21.15.37:0/2099962412 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7fcb1815abd0 0x7fcb1806c540 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:28:54.452 INFO:teuthology.misc.health.smithi037.stderr:2024-04-25T10:28:54.449+0000 7fcb17fff640 1 -- 172.21.15.37:0/2099962412 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fcb180b42f0 msgr2=0x7fcb18156200 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:28:54.452 INFO:teuthology.misc.health.smithi037.stderr:2024-04-25T10:28:54.449+0000 7fcb17fff640 1 --2- 172.21.15.37:0/2099962412 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fcb180b42f0 0x7fcb18156200 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:28:54.453 INFO:teuthology.misc.health.smithi037.stderr:2024-04-25T10:28:54.449+0000 7fcb17fff640 1 -- 172.21.15.37:0/2099962412 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fcb18156a50 con 0x7fcb180ccec0 2024-04-25T10:28:54.453 INFO:teuthology.misc.health.smithi037.stderr:2024-04-25T10:28:54.449+0000 7fcb1cf5a640 1 --2- 172.21.15.37:0/2099962412 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fcb180b42f0 0x7fcb18156200 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-25T10:28:54.453 INFO:teuthology.misc.health.smithi037.stderr:2024-04-25T10:28:54.449+0000 7fcb17fff640 1 --2- 172.21.15.37:0/2099962412 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fcb180ccec0 0x7fcb1814ac00 secure :-1 s=READY pgs=31 cs=0 l=1 rev1=1 crypto rx=0x7fcb04009870 tx=0x7fcb040046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:28:54.453 INFO:teuthology.misc.health.smithi037.stderr:2024-04-25T10:28:54.449+0000 7fcb157fa640 1 -- 172.21.15.37:0/2099962412 <== mon.0 v2:172.21.15.37:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcb04042020 con 0x7fcb180ccec0 2024-04-25T10:28:54.453 INFO:teuthology.misc.health.smithi037.stderr:2024-04-25T10:28:54.449+0000 7fcb157fa640 1 -- 172.21.15.37:0/2099962412 <== mon.0 v2:172.21.15.37:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fcb04033830 con 0x7fcb180ccec0 2024-04-25T10:28:54.453 INFO:teuthology.misc.health.smithi037.stderr:2024-04-25T10:28:54.449+0000 7fcb1f1be640 1 -- 172.21.15.37:0/2099962412 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fcb18156d30 con 0x7fcb180ccec0 2024-04-25T10:28:54.453 INFO:teuthology.misc.health.smithi037.stderr:2024-04-25T10:28:54.449+0000 7fcb157fa640 1 -- 172.21.15.37:0/2099962412 <== mon.0 v2:172.21.15.37:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcb04038040 con 0x7fcb180ccec0 2024-04-25T10:28:54.453 INFO:teuthology.misc.health.smithi037.stderr:2024-04-25T10:28:54.449+0000 7fcb1f1be640 1 -- 172.21.15.37:0/2099962412 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fcb180b70b0 con 0x7fcb180ccec0 2024-04-25T10:28:54.454 INFO:teuthology.misc.health.smithi037.stderr:2024-04-25T10:28:54.451+0000 7fcb1f1be640 1 -- 172.21.15.37:0/2099962412 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fcb181567c0 con 0x7fcb180ccec0 2024-04-25T10:28:54.454 INFO:teuthology.misc.health.smithi037.stderr:2024-04-25T10:28:54.451+0000 7fcb157fa640 1 -- 172.21.15.37:0/2099962412 <== mon.0 v2:172.21.15.37:3300/0 4 ==== mgrmap(e 4) ==== 96726+0+0 (secure 0 0 0) 0x7fcb04045020 con 0x7fcb180ccec0 2024-04-25T10:28:54.454 INFO:teuthology.misc.health.smithi037.stderr:2024-04-25T10:28:54.452+0000 7fcb157fa640 1 --2- 172.21.15.37:0/2099962412 >> [v2:172.21.15.203:6832/2418205237,v1:172.21.15.203:6833/2418205237] conn(0x7fcafc074420 0x7fcafc076930 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:28:54.454 INFO:teuthology.misc.health.smithi037.stderr:2024-04-25T10:28:54.452+0000 7fcb157fa640 1 -- 172.21.15.37:0/2099962412 <== mon.0 v2:172.21.15.37:3300/0 5 ==== osd_map(14..14 src has 1..14) ==== 2511+0+0 (secure 0 0 0) 0x7fcb040b5060 con 0x7fcb180ccec0 2024-04-25T10:28:54.454 INFO:teuthology.misc.health.smithi037.stderr:2024-04-25T10:28:54.452+0000 7fcb1cf5a640 1 --2- 172.21.15.37:0/2099962412 >> [v2:172.21.15.203:6832/2418205237,v1:172.21.15.203:6833/2418205237] conn(0x7fcafc074420 0x7fcafc076930 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:28:54.454 INFO:teuthology.misc.health.smithi037.stderr:2024-04-25T10:28:54.452+0000 7fcb1cf5a640 1 --2- 172.21.15.37:0/2099962412 >> [v2:172.21.15.203:6832/2418205237,v1:172.21.15.203:6833/2418205237] conn(0x7fcafc074420 0x7fcafc076930 secure :-1 s=READY pgs=34 cs=0 l=1 rev1=1 crypto rx=0x7fcb0c000f60 tx=0x7fcb0c009040 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:28:54.457 INFO:tasks.ceph.osd.1.smithi037.stderr:2024-04-25T10:28:54.455+0000 7f98eeaae640 -1 osd.1 0 waiting for initial osdmap 2024-04-25T10:28:54.458 INFO:tasks.ceph.osd.0.smithi037.stderr:2024-04-25T10:28:54.457+0000 7f10b5e33640 -1 osd.0 0 waiting for initial osdmap 2024-04-25T10:28:54.460 INFO:teuthology.misc.health.smithi037.stderr:2024-04-25T10:28:54.458+0000 7fcb157fa640 1 -- 172.21.15.37:0/2099962412 <== mon.0 v2:172.21.15.37:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187246 (secure 0 0 0) 0x7fcb0404c030 con 0x7fcb180ccec0 2024-04-25T10:28:54.474 INFO:tasks.ceph.osd.0.smithi037.stderr:2024-04-25T10:28:54.472+0000 7f10adc2e640 -1 osd.0 14 set_numa_affinity unable to identify public interface '' numa node: (2) No such file or directory 2024-04-25T10:28:54.474 INFO:tasks.ceph.osd.1.smithi037.stderr:2024-04-25T10:28:54.472+0000 7f98e68a9640 -1 osd.1 14 set_numa_affinity unable to identify public interface '' numa node: (2) No such file or directory 2024-04-25T10:28:54.648 INFO:teuthology.misc.health.smithi037.stderr:2024-04-25T10:28:54.646+0000 7fcb1f1be640 1 -- 172.21.15.37:0/2099962412 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7fcb18050740 con 0x7fcb180ccec0 2024-04-25T10:28:54.649 INFO:teuthology.misc.health.smithi037.stderr:2024-04-25T10:28:54.647+0000 7fcb157fa640 1 -- 172.21.15.37:0/2099962412 <== mon.0 v2:172.21.15.37:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v14) ==== 74+0+7590 (secure 0 0 0) 0x7fcb0407ff00 con 0x7fcb180ccec0 2024-04-25T10:28:54.649 INFO:teuthology.misc.health.smithi037.stdout: 2024-04-25T10:28:54.649 INFO:teuthology.misc.health.smithi037.stdout:{"epoch":14,"fsid":"f4acda28-be34-492a-b286-994afd36f61d","created":"2024-04-25T10:28:43.985718+0000","modified":"2024-04-25T10:28:53.314292+0000","last_up_change":"2024-04-25T10:28:52.192314+0000","last_in_change":"2024-04-25T10:28:46.216973+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":0,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[],"osds":[{"osd":0,"uuid":"a2f79028-b332-4166-9ebb-a7024a37ea22","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":1,"uuid":"973159de-c2ea-4083-aecc-69373d1c1132","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":2,"uuid":"4ae7a0e9-cb3f-484c-a75f-f3766b721c47","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":3,"uuid":"1ba4a746-1608-47a7-ace2-960a5cfae934","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":4,"uuid":"d4f59f5d-0413-4b79-87ee-bb212c1fcf62","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6800","nonce":4051532063},{"type":"v1","addr":"172.21.15.203:6801","nonce":4051532063}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6802","nonce":4051532063},{"type":"v1","addr":"172.21.15.203:6803","nonce":4051532063}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6806","nonce":4051532063},{"type":"v1","addr":"172.21.15.203:6807","nonce":4051532063}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6804","nonce":4051532063},{"type":"v1","addr":"172.21.15.203:6805","nonce":4051532063}]},"public_addr":"172.21.15.203:6801/4051532063","cluster_addr":"172.21.15.203:6803/4051532063","heartbeat_back_addr":"172.21.15.203:6807/4051532063","heartbeat_front_addr":"172.21.15.203:6805/4051532063","state":["exists","up"]},{"osd":5,"uuid":"69710a0e-41d8-4e7f-9a10-17ce540b67be","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":6,"uuid":"f29dd047-3b48-473d-a235-6642a6293f20","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":7,"uuid":"6e691644-378b-4fdc-8e86-fb5f56e31f29","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T10:28:51.105812+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-04-25T10:28:54.651 INFO:teuthology.misc.health.smithi037.stderr:2024-04-25T10:28:54.649+0000 7fcb1f1be640 1 -- 172.21.15.37:0/2099962412 >> [v2:172.21.15.203:6832/2418205237,v1:172.21.15.203:6833/2418205237] conn(0x7fcafc074420 msgr2=0x7fcafc076930 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:28:54.652 INFO:teuthology.misc.health.smithi037.stderr:2024-04-25T10:28:54.649+0000 7fcb1f1be640 1 --2- 172.21.15.37:0/2099962412 >> [v2:172.21.15.203:6832/2418205237,v1:172.21.15.203:6833/2418205237] conn(0x7fcafc074420 0x7fcafc076930 secure :-1 s=READY pgs=34 cs=0 l=1 rev1=1 crypto rx=0x7fcb0c000f60 tx=0x7fcb0c009040 comp rx=0 tx=0).stop 2024-04-25T10:28:54.652 INFO:teuthology.misc.health.smithi037.stderr:2024-04-25T10:28:54.649+0000 7fcb1f1be640 1 -- 172.21.15.37:0/2099962412 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fcb180ccec0 msgr2=0x7fcb1814ac00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:28:54.652 INFO:teuthology.misc.health.smithi037.stderr:2024-04-25T10:28:54.649+0000 7fcb1f1be640 1 --2- 172.21.15.37:0/2099962412 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fcb180ccec0 0x7fcb1814ac00 secure :-1 s=READY pgs=31 cs=0 l=1 rev1=1 crypto rx=0x7fcb04009870 tx=0x7fcb040046c0 comp rx=0 tx=0).stop 2024-04-25T10:28:54.652 INFO:teuthology.misc.health.smithi037.stderr:2024-04-25T10:28:54.649+0000 7fcb1f1be640 1 -- 172.21.15.37:0/2099962412 shutdown_connections 2024-04-25T10:28:54.652 INFO:teuthology.misc.health.smithi037.stderr:2024-04-25T10:28:54.649+0000 7fcb1f1be640 1 --2- 172.21.15.37:0/2099962412 >> [v2:172.21.15.203:6832/2418205237,v1:172.21.15.203:6833/2418205237] conn(0x7fcafc074420 0x7fcafc076930 unknown :-1 s=CLOSED pgs=34 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:28:54.652 INFO:teuthology.misc.health.smithi037.stderr:2024-04-25T10:28:54.649+0000 7fcb1f1be640 1 --2- 172.21.15.37:0/2099962412 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7fcb1815abd0 0x7fcb1806c540 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:28:54.652 INFO:teuthology.misc.health.smithi037.stderr:2024-04-25T10:28:54.649+0000 7fcb1f1be640 1 --2- 172.21.15.37:0/2099962412 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fcb180ccec0 0x7fcb1814ac00 unknown :-1 s=CLOSED pgs=31 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:28:54.652 INFO:teuthology.misc.health.smithi037.stderr:2024-04-25T10:28:54.649+0000 7fcb1f1be640 1 --2- 172.21.15.37:0/2099962412 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fcb180b42f0 0x7fcb18156200 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:28:54.652 INFO:teuthology.misc.health.smithi037.stderr:2024-04-25T10:28:54.649+0000 7fcb1f1be640 1 -- 172.21.15.37:0/2099962412 >> 172.21.15.37:0/2099962412 conn(0x7fcb180cf2f0 msgr2=0x7fcb1815fda0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:28:54.652 INFO:teuthology.misc.health.smithi037.stderr:2024-04-25T10:28:54.649+0000 7fcb1f1be640 1 -- 172.21.15.37:0/2099962412 shutdown_connections 2024-04-25T10:28:54.652 INFO:teuthology.misc.health.smithi037.stderr:2024-04-25T10:28:54.649+0000 7fcb1f1be640 1 -- 172.21.15.37:0/2099962412 wait complete. 2024-04-25T10:28:54.661 DEBUG:teuthology.misc:1 of 8 OSDs are up 2024-04-25T10:29:01.469 DEBUG:teuthology.orchestra.run.smithi037:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph --cluster ceph osd dump --format=json 2024-04-25T10:29:01.586 INFO:teuthology.misc.health.smithi037.stderr:2024-04-25T10:29:01.584+0000 7f16078bf640 1 Processor -- start 2024-04-25T10:29:01.586 INFO:teuthology.misc.health.smithi037.stderr:2024-04-25T10:29:01.584+0000 7f16078bf640 1 -- start start 2024-04-25T10:29:01.589 INFO:teuthology.misc.health.smithi037.stderr:2024-04-25T10:29:01.585+0000 7f16078bf640 1 --2- >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f1600164670 0x7f16000d2560 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:29:01.589 INFO:teuthology.misc.health.smithi037.stderr:2024-04-25T10:29:01.585+0000 7f16078bf640 1 --2- >> v2:172.21.15.37:3300/0 conn(0x7f16000d2aa0 0x7f16000d4430 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:29:01.589 INFO:teuthology.misc.health.smithi037.stderr:2024-04-25T10:29:01.585+0000 7f16078bf640 1 --2- >> v2:172.21.15.203:3300/0 conn(0x7f16000d4970 0x7f16000d0800 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:29:01.589 INFO:teuthology.misc.health.smithi037.stderr:2024-04-25T10:29:01.585+0000 7f16078bf640 1 -- --> v2:172.21.15.37:3300/0 -- mon_getmap magic: 0 -- 0x7f16000d2eb0 con 0x7f16000d2aa0 2024-04-25T10:29:01.589 INFO:teuthology.misc.health.smithi037.stderr:2024-04-25T10:29:01.585+0000 7f16078bf640 1 -- --> v2:172.21.15.203:3300/0 -- mon_getmap magic: 0 -- 0x7f1600161170 con 0x7f16000d4970 2024-04-25T10:29:01.589 INFO:teuthology.misc.health.smithi037.stderr:2024-04-25T10:29:01.585+0000 7f16078bf640 1 -- --> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] -- mon_getmap magic: 0 -- 0x7f1600161450 con 0x7f1600164670 2024-04-25T10:29:01.589 INFO:teuthology.misc.health.smithi037.stderr:2024-04-25T10:29:01.585+0000 7f1604e5a640 1 --2- >> v2:172.21.15.37:3300/0 conn(0x7f16000d2aa0 0x7f16000d4430 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:29:01.589 INFO:teuthology.misc.health.smithi037.stderr:2024-04-25T10:29:01.585+0000 7f1604e5a640 1 --2- >> v2:172.21.15.37:3300/0 conn(0x7f16000d2aa0 0x7f16000d4430 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.37:3300/0 says I am v2:172.21.15.37:46220/0 (socket says 172.21.15.37:46220) 2024-04-25T10:29:01.590 INFO:teuthology.misc.health.smithi037.stderr:2024-04-25T10:29:01.586+0000 7f1604e5a640 1 -- 172.21.15.37:0/3772854615 learned_addr learned my addr 172.21.15.37:0/3772854615 (peer_addr_for_me v2:172.21.15.37:0/0) 2024-04-25T10:29:01.590 INFO:teuthology.misc.health.smithi037.stderr:2024-04-25T10:29:01.586+0000 7f1605e5c640 1 --2- 172.21.15.37:0/3772854615 >> v2:172.21.15.203:3300/0 conn(0x7f16000d4970 0x7f16000d0800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:29:01.590 INFO:teuthology.misc.health.smithi037.stderr:2024-04-25T10:29:01.586+0000 7f160565b640 1 --2- 172.21.15.37:0/3772854615 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f1600164670 0x7f16000d2560 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:29:01.590 INFO:teuthology.misc.health.smithi037.stderr:2024-04-25T10:29:01.586+0000 7f1604e5a640 1 -- 172.21.15.37:0/3772854615 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f1600164670 msgr2=0x7f16000d2560 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:29:01.590 INFO:teuthology.misc.health.smithi037.stderr:2024-04-25T10:29:01.586+0000 7f1604e5a640 1 --2- 172.21.15.37:0/3772854615 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f1600164670 0x7f16000d2560 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:29:01.590 INFO:teuthology.misc.health.smithi037.stderr:2024-04-25T10:29:01.586+0000 7f1604e5a640 1 -- 172.21.15.37:0/3772854615 >> v2:172.21.15.203:3300/0 conn(0x7f16000d4970 msgr2=0x7f16000d0800 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:29:01.590 INFO:teuthology.misc.health.smithi037.stderr:2024-04-25T10:29:01.586+0000 7f1604e5a640 1 --2- 172.21.15.37:0/3772854615 >> v2:172.21.15.203:3300/0 conn(0x7f16000d4970 0x7f16000d0800 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:29:01.590 INFO:teuthology.misc.health.smithi037.stderr:2024-04-25T10:29:01.586+0000 7f1604e5a640 1 -- 172.21.15.37:0/3772854615 --> v2:172.21.15.37:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f1600163e70 con 0x7f16000d2aa0 2024-04-25T10:29:01.590 INFO:teuthology.misc.health.smithi037.stderr:2024-04-25T10:29:01.586+0000 7f1605e5c640 1 --2- 172.21.15.37:0/3772854615 >> v2:172.21.15.203:3300/0 conn(0x7f16000d4970 0x7f16000d0800 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T10:29:01.590 INFO:teuthology.misc.health.smithi037.stderr:2024-04-25T10:29:01.587+0000 7f1604e5a640 1 --2- 172.21.15.37:0/3772854615 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f16000d2aa0 0x7f16000d4430 secure :-1 s=READY pgs=33 cs=0 l=1 rev1=1 crypto rx=0x7f16000b8650 tx=0x7f15f002a1e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=6a8dfa52f4248a89 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:29:01.590 INFO:teuthology.misc.health.smithi037.stderr:2024-04-25T10:29:01.587+0000 7f15f7fff640 1 -- 172.21.15.37:0/3772854615 <== mon.0 v2:172.21.15.37:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f15f0037070 con 0x7f16000d2aa0 2024-04-25T10:29:01.590 INFO:teuthology.misc.health.smithi037.stderr:2024-04-25T10:29:01.587+0000 7f15f7fff640 1 -- 172.21.15.37:0/3772854615 <== mon.0 v2:172.21.15.37:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f15f0032440 con 0x7f16000d2aa0 2024-04-25T10:29:01.591 INFO:teuthology.misc.health.smithi037.stderr:2024-04-25T10:29:01.587+0000 7f15f7fff640 1 -- 172.21.15.37:0/3772854615 <== mon.0 v2:172.21.15.37:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f15f0032810 con 0x7f16000d2aa0 2024-04-25T10:29:01.592 INFO:teuthology.misc.health.smithi037.stderr:2024-04-25T10:29:01.587+0000 7f16078bf640 1 -- 172.21.15.37:0/3772854615 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f16000d2aa0 msgr2=0x7f16000d4430 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:29:01.592 INFO:teuthology.misc.health.smithi037.stderr:2024-04-25T10:29:01.587+0000 7f16078bf640 1 --2- 172.21.15.37:0/3772854615 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f16000d2aa0 0x7f16000d4430 secure :-1 s=READY pgs=33 cs=0 l=1 rev1=1 crypto rx=0x7f16000b8650 tx=0x7f15f002a1e0 comp rx=0 tx=0).stop 2024-04-25T10:29:01.592 INFO:teuthology.misc.health.smithi037.stderr:2024-04-25T10:29:01.587+0000 7f16078bf640 1 -- 172.21.15.37:0/3772854615 shutdown_connections 2024-04-25T10:29:01.592 INFO:teuthology.misc.health.smithi037.stderr:2024-04-25T10:29:01.587+0000 7f16078bf640 1 --2- 172.21.15.37:0/3772854615 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f16000d2aa0 0x7f16000d4430 unknown :-1 s=CLOSED pgs=33 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:29:01.592 INFO:teuthology.misc.health.smithi037.stderr:2024-04-25T10:29:01.587+0000 7f16078bf640 1 --2- 172.21.15.37:0/3772854615 >> v2:172.21.15.203:3300/0 conn(0x7f16000d4970 0x7f16000d0800 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:29:01.592 INFO:teuthology.misc.health.smithi037.stderr:2024-04-25T10:29:01.587+0000 7f16078bf640 1 --2- 172.21.15.37:0/3772854615 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f1600164670 0x7f16000d2560 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:29:01.592 INFO:teuthology.misc.health.smithi037.stderr:2024-04-25T10:29:01.587+0000 7f16078bf640 1 -- 172.21.15.37:0/3772854615 >> 172.21.15.37:0/3772854615 conn(0x7f16000747f0 msgr2=0x7f160006dd90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:29:01.592 INFO:teuthology.misc.health.smithi037.stderr:2024-04-25T10:29:01.588+0000 7f16078bf640 1 -- 172.21.15.37:0/3772854615 shutdown_connections 2024-04-25T10:29:01.592 INFO:teuthology.misc.health.smithi037.stderr:2024-04-25T10:29:01.588+0000 7f16078bf640 1 -- 172.21.15.37:0/3772854615 wait complete. 2024-04-25T10:29:01.592 INFO:teuthology.misc.health.smithi037.stderr:2024-04-25T10:29:01.589+0000 7f16078bf640 1 Processor -- start 2024-04-25T10:29:01.592 INFO:teuthology.misc.health.smithi037.stderr:2024-04-25T10:29:01.589+0000 7f16078bf640 1 -- start start 2024-04-25T10:29:01.594 INFO:teuthology.misc.health.smithi037.stderr:2024-04-25T10:29:01.591+0000 7f16078bf640 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f16000d2aa0 0x7f16000ac550 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:29:01.594 INFO:teuthology.misc.health.smithi037.stderr:2024-04-25T10:29:01.591+0000 7f16078bf640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f16000d4970 0x7f16000ce3f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:29:01.594 INFO:teuthology.misc.health.smithi037.stderr:2024-04-25T10:29:01.591+0000 7f16078bf640 1 --2- >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f1600164670 0x7f16000ce930 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:29:01.595 INFO:teuthology.misc.health.smithi037.stderr:2024-04-25T10:29:01.591+0000 7f16078bf640 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 -- 0x7f16000b5080 con 0x7f16000d4970 2024-04-25T10:29:01.595 INFO:teuthology.misc.health.smithi037.stderr:2024-04-25T10:29:01.591+0000 7f16078bf640 1 -- --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_getmap magic: 0 -- 0x7f16000b7800 con 0x7f16000d2aa0 2024-04-25T10:29:01.595 INFO:teuthology.misc.health.smithi037.stderr:2024-04-25T10:29:01.591+0000 7f16078bf640 1 -- --> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] -- mon_getmap magic: 0 -- 0x7f16000b5200 con 0x7f1600164670 2024-04-25T10:29:01.595 INFO:teuthology.misc.health.smithi037.stderr:2024-04-25T10:29:01.591+0000 7f1604e5a640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f16000d4970 0x7f16000ce3f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:29:01.595 INFO:teuthology.misc.health.smithi037.stderr:2024-04-25T10:29:01.591+0000 7f1604e5a640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f16000d4970 0x7f16000ce3f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.37:3300/0 says I am v2:172.21.15.37:46236/0 (socket says 172.21.15.37:46236) 2024-04-25T10:29:01.595 INFO:teuthology.misc.health.smithi037.stderr:2024-04-25T10:29:01.591+0000 7f1604e5a640 1 -- 172.21.15.37:0/1348648109 learned_addr learned my addr 172.21.15.37:0/1348648109 (peer_addr_for_me v2:172.21.15.37:0/0) 2024-04-25T10:29:01.595 INFO:teuthology.misc.health.smithi037.stderr:2024-04-25T10:29:01.591+0000 7f160565b640 1 --2- 172.21.15.37:0/1348648109 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f16000d2aa0 0x7f16000ac550 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:29:01.595 INFO:teuthology.misc.health.smithi037.stderr:2024-04-25T10:29:01.591+0000 7f1605e5c640 1 --2- 172.21.15.37:0/1348648109 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f1600164670 0x7f16000ce930 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:29:01.595 INFO:teuthology.misc.health.smithi037.stderr:2024-04-25T10:29:01.591+0000 7f1604e5a640 1 -- 172.21.15.37:0/1348648109 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f1600164670 msgr2=0x7f16000ce930 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:29:01.595 INFO:teuthology.misc.health.smithi037.stderr:2024-04-25T10:29:01.591+0000 7f1604e5a640 1 --2- 172.21.15.37:0/1348648109 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f1600164670 0x7f16000ce930 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:29:01.595 INFO:teuthology.misc.health.smithi037.stderr:2024-04-25T10:29:01.591+0000 7f1604e5a640 1 -- 172.21.15.37:0/1348648109 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f16000d2aa0 msgr2=0x7f16000ac550 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:29:01.595 INFO:teuthology.misc.health.smithi037.stderr:2024-04-25T10:29:01.591+0000 7f1604e5a640 1 --2- 172.21.15.37:0/1348648109 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f16000d2aa0 0x7f16000ac550 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:29:01.595 INFO:teuthology.misc.health.smithi037.stderr:2024-04-25T10:29:01.591+0000 7f1604e5a640 1 -- 172.21.15.37:0/1348648109 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f16000ace50 con 0x7f16000d4970 2024-04-25T10:29:01.596 INFO:teuthology.misc.health.smithi037.stderr:2024-04-25T10:29:01.591+0000 7f1604e5a640 1 --2- 172.21.15.37:0/1348648109 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f16000d4970 0x7f16000ce3f0 secure :-1 s=READY pgs=34 cs=0 l=1 rev1=1 crypto rx=0x7f15f0000c00 tx=0x7f15f00046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:29:01.596 INFO:teuthology.misc.health.smithi037.stderr:2024-04-25T10:29:01.592+0000 7f15f5ffb640 1 -- 172.21.15.37:0/1348648109 <== mon.0 v2:172.21.15.37:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f15f0037040 con 0x7f16000d4970 2024-04-25T10:29:01.596 INFO:teuthology.misc.health.smithi037.stderr:2024-04-25T10:29:01.592+0000 7f15f5ffb640 1 -- 172.21.15.37:0/1348648109 <== mon.0 v2:172.21.15.37:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f15f0042070 con 0x7f16000d4970 2024-04-25T10:29:01.596 INFO:teuthology.misc.health.smithi037.stderr:2024-04-25T10:29:01.592+0000 7f16078bf640 1 -- 172.21.15.37:0/1348648109 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f16000ad130 con 0x7f16000d4970 2024-04-25T10:29:01.596 INFO:teuthology.misc.health.smithi037.stderr:2024-04-25T10:29:01.592+0000 7f15f5ffb640 1 -- 172.21.15.37:0/1348648109 <== mon.0 v2:172.21.15.37:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f15f0030bb0 con 0x7f16000d4970 2024-04-25T10:29:01.596 INFO:teuthology.misc.health.smithi037.stderr:2024-04-25T10:29:01.592+0000 7f16078bf640 1 -- 172.21.15.37:0/1348648109 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f16000ad6f0 con 0x7f16000d4970 2024-04-25T10:29:01.597 INFO:teuthology.misc.health.smithi037.stderr:2024-04-25T10:29:01.593+0000 7f16078bf640 1 -- 172.21.15.37:0/1348648109 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f16000acbc0 con 0x7f16000d4970 2024-04-25T10:29:01.597 INFO:teuthology.misc.health.smithi037.stderr:2024-04-25T10:29:01.593+0000 7f15f5ffb640 1 -- 172.21.15.37:0/1348648109 <== mon.0 v2:172.21.15.37:3300/0 4 ==== mgrmap(e 5) ==== 96782+0+0 (secure 0 0 0) 0x7f15f0047460 con 0x7f16000d4970 2024-04-25T10:29:01.597 INFO:teuthology.misc.health.smithi037.stderr:2024-04-25T10:29:01.594+0000 7f15f5ffb640 1 --2- 172.21.15.37:0/1348648109 >> [v2:172.21.15.203:6832/2418205237,v1:172.21.15.203:6833/2418205237] conn(0x7f15e40744e0 0x7f15e40769f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:29:01.597 INFO:teuthology.misc.health.smithi037.stderr:2024-04-25T10:29:01.594+0000 7f15f5ffb640 1 -- 172.21.15.37:0/1348648109 <== mon.0 v2:172.21.15.37:3300/0 5 ==== osd_map(19..19 src has 1..19) ==== 4857+0+0 (secure 0 0 0) 0x7f15f00b59f0 con 0x7f16000d4970 2024-04-25T10:29:01.597 INFO:teuthology.misc.health.smithi037.stderr:2024-04-25T10:29:01.594+0000 7f160565b640 1 --2- 172.21.15.37:0/1348648109 >> [v2:172.21.15.203:6832/2418205237,v1:172.21.15.203:6833/2418205237] conn(0x7f15e40744e0 0x7f15e40769f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:29:01.597 INFO:teuthology.misc.health.smithi037.stderr:2024-04-25T10:29:01.595+0000 7f160565b640 1 --2- 172.21.15.37:0/1348648109 >> [v2:172.21.15.203:6832/2418205237,v1:172.21.15.203:6833/2418205237] conn(0x7f15e40744e0 0x7f15e40769f0 secure :-1 s=READY pgs=43 cs=0 l=1 rev1=1 crypto rx=0x7f160008e560 tx=0x7f15fc008040 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:29:01.601 INFO:teuthology.misc.health.smithi037.stderr:2024-04-25T10:29:01.599+0000 7f15f5ffb640 1 -- 172.21.15.37:0/1348648109 <== mon.0 v2:172.21.15.37:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187246 (secure 0 0 0) 0x7f15f007fe80 con 0x7f16000d4970 2024-04-25T10:29:01.786 INFO:teuthology.misc.health.smithi037.stderr:2024-04-25T10:29:01.784+0000 7f16078bf640 1 -- 172.21.15.37:0/1348648109 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7f1600050740 con 0x7f16000d4970 2024-04-25T10:29:01.787 INFO:teuthology.misc.health.smithi037.stderr:2024-04-25T10:29:01.785+0000 7f15f5ffb640 1 -- 172.21.15.37:0/1348648109 <== mon.0 v2:172.21.15.37:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v19) ==== 74+0+12896 (secure 0 0 0) 0x7f15f0084180 con 0x7f16000d4970 2024-04-25T10:29:01.787 INFO:teuthology.misc.health.smithi037.stdout: 2024-04-25T10:29:01.787 INFO:teuthology.misc.health.smithi037.stdout:{"epoch":19,"fsid":"f4acda28-be34-492a-b286-994afd36f61d","created":"2024-04-25T10:28:43.985718+0000","modified":"2024-04-25T10:28:58.970539+0000","last_up_change":"2024-04-25T10:28:54.900654+0000","last_in_change":"2024-04-25T10:28:46.216973+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":1,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-04-25T10:28:56.202153+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"19","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"a2f79028-b332-4166-9ebb-a7024a37ea22","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.37:6816","nonce":2746074720},{"type":"v1","addr":"172.21.15.37:6817","nonce":2746074720}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.37:6818","nonce":2746074720},{"type":"v1","addr":"172.21.15.37:6819","nonce":2746074720}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.37:6822","nonce":2746074720},{"type":"v1","addr":"172.21.15.37:6823","nonce":2746074720}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.37:6820","nonce":2746074720},{"type":"v1","addr":"172.21.15.37:6821","nonce":2746074720}]},"public_addr":"172.21.15.37:6817/2746074720","cluster_addr":"172.21.15.37:6819/2746074720","heartbeat_back_addr":"172.21.15.37:6823/2746074720","heartbeat_front_addr":"172.21.15.37:6821/2746074720","state":["exists","up"]},{"osd":1,"uuid":"973159de-c2ea-4083-aecc-69373d1c1132","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":17,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.37:6800","nonce":1690341662},{"type":"v1","addr":"172.21.15.37:6801","nonce":1690341662}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.37:6802","nonce":1690341662},{"type":"v1","addr":"172.21.15.37:6803","nonce":1690341662}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.37:6806","nonce":1690341662},{"type":"v1","addr":"172.21.15.37:6807","nonce":1690341662}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.37:6804","nonce":1690341662},{"type":"v1","addr":"172.21.15.37:6805","nonce":1690341662}]},"public_addr":"172.21.15.37:6801/1690341662","cluster_addr":"172.21.15.37:6803/1690341662","heartbeat_back_addr":"172.21.15.37:6807/1690341662","heartbeat_front_addr":"172.21.15.37:6805/1690341662","state":["exists","up"]},{"osd":2,"uuid":"4ae7a0e9-cb3f-484c-a75f-f3766b721c47","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.37:6808","nonce":2754669649},{"type":"v1","addr":"172.21.15.37:6809","nonce":2754669649}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.37:6810","nonce":2754669649},{"type":"v1","addr":"172.21.15.37:6811","nonce":2754669649}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.37:6814","nonce":2754669649},{"type":"v1","addr":"172.21.15.37:6815","nonce":2754669649}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.37:6812","nonce":2754669649},{"type":"v1","addr":"172.21.15.37:6813","nonce":2754669649}]},"public_addr":"172.21.15.37:6809/2754669649","cluster_addr":"172.21.15.37:6811/2754669649","heartbeat_back_addr":"172.21.15.37:6815/2754669649","heartbeat_front_addr":"172.21.15.37:6813/2754669649","state":["exists","up"]},{"osd":3,"uuid":"1ba4a746-1608-47a7-ace2-960a5cfae934","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.37:6824","nonce":1815290618},{"type":"v1","addr":"172.21.15.37:6825","nonce":1815290618}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.37:6826","nonce":1815290618},{"type":"v1","addr":"172.21.15.37:6827","nonce":1815290618}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.37:6830","nonce":1815290618},{"type":"v1","addr":"172.21.15.37:6831","nonce":1815290618}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.37:6828","nonce":1815290618},{"type":"v1","addr":"172.21.15.37:6829","nonce":1815290618}]},"public_addr":"172.21.15.37:6825/1815290618","cluster_addr":"172.21.15.37:6827/1815290618","heartbeat_back_addr":"172.21.15.37:6831/1815290618","heartbeat_front_addr":"172.21.15.37:6829/1815290618","state":["exists","up"]},{"osd":4,"uuid":"d4f59f5d-0413-4b79-87ee-bb212c1fcf62","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6800","nonce":4051532063},{"type":"v1","addr":"172.21.15.203:6801","nonce":4051532063}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6802","nonce":4051532063},{"type":"v1","addr":"172.21.15.203:6803","nonce":4051532063}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6806","nonce":4051532063},{"type":"v1","addr":"172.21.15.203:6807","nonce":4051532063}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6804","nonce":4051532063},{"type":"v1","addr":"172.21.15.203:6805","nonce":4051532063}]},"public_addr":"172.21.15.203:6801/4051532063","cluster_addr":"172.21.15.203:6803/4051532063","heartbeat_back_addr":"172.21.15.203:6807/4051532063","heartbeat_front_addr":"172.21.15.203:6805/4051532063","state":["exists","up"]},{"osd":5,"uuid":"69710a0e-41d8-4e7f-9a10-17ce540b67be","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6824","nonce":1203822890},{"type":"v1","addr":"172.21.15.203:6825","nonce":1203822890}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6826","nonce":1203822890},{"type":"v1","addr":"172.21.15.203:6827","nonce":1203822890}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6830","nonce":1203822890},{"type":"v1","addr":"172.21.15.203:6831","nonce":1203822890}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6828","nonce":1203822890},{"type":"v1","addr":"172.21.15.203:6829","nonce":1203822890}]},"public_addr":"172.21.15.203:6825/1203822890","cluster_addr":"172.21.15.203:6827/1203822890","heartbeat_back_addr":"172.21.15.203:6831/1203822890","heartbeat_front_addr":"172.21.15.203:6829/1203822890","state":["exists","up"]},{"osd":6,"uuid":"f29dd047-3b48-473d-a235-6642a6293f20","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6810","nonce":2309615014},{"type":"v1","addr":"172.21.15.203:6812","nonce":2309615014}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6814","nonce":2309615014},{"type":"v1","addr":"172.21.15.203:6816","nonce":2309615014}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6822","nonce":2309615014},{"type":"v1","addr":"172.21.15.203:6823","nonce":2309615014}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6818","nonce":2309615014},{"type":"v1","addr":"172.21.15.203:6820","nonce":2309615014}]},"public_addr":"172.21.15.203:6812/2309615014","cluster_addr":"172.21.15.203:6816/2309615014","heartbeat_back_addr":"172.21.15.203:6823/2309615014","heartbeat_front_addr":"172.21.15.203:6820/2309615014","state":["exists","up"]},{"osd":7,"uuid":"6e691644-378b-4fdc-8e86-fb5f56e31f29","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6808","nonce":3688099014},{"type":"v1","addr":"172.21.15.203:6809","nonce":3688099014}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6811","nonce":3688099014},{"type":"v1","addr":"172.21.15.203:6813","nonce":3688099014}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6819","nonce":3688099014},{"type":"v1","addr":"172.21.15.203:6821","nonce":3688099014}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6815","nonce":3688099014},{"type":"v1","addr":"172.21.15.203:6817","nonce":3688099014}]},"public_addr":"172.21.15.203:6809/3688099014","cluster_addr":"172.21.15.203:6813/3688099014","heartbeat_back_addr":"172.21.15.203:6821/3688099014","heartbeat_front_addr":"172.21.15.203:6817/3688099014","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T10:28:51.419141+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T10:28:51.366448+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T10:28:51.348276+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T10:28:51.380519+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T10:28:51.105812+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T10:28:51.384496+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T10:28:51.388755+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T10:28:51.366958+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-04-25T10:29:01.789 INFO:teuthology.misc.health.smithi037.stderr:2024-04-25T10:29:01.786+0000 7f16078bf640 1 -- 172.21.15.37:0/1348648109 >> [v2:172.21.15.203:6832/2418205237,v1:172.21.15.203:6833/2418205237] conn(0x7f15e40744e0 msgr2=0x7f15e40769f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:29:01.789 INFO:teuthology.misc.health.smithi037.stderr:2024-04-25T10:29:01.787+0000 7f16078bf640 1 --2- 172.21.15.37:0/1348648109 >> [v2:172.21.15.203:6832/2418205237,v1:172.21.15.203:6833/2418205237] conn(0x7f15e40744e0 0x7f15e40769f0 secure :-1 s=READY pgs=43 cs=0 l=1 rev1=1 crypto rx=0x7f160008e560 tx=0x7f15fc008040 comp rx=0 tx=0).stop 2024-04-25T10:29:01.789 INFO:teuthology.misc.health.smithi037.stderr:2024-04-25T10:29:01.787+0000 7f16078bf640 1 -- 172.21.15.37:0/1348648109 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f16000d4970 msgr2=0x7f16000ce3f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:29:01.789 INFO:teuthology.misc.health.smithi037.stderr:2024-04-25T10:29:01.787+0000 7f16078bf640 1 --2- 172.21.15.37:0/1348648109 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f16000d4970 0x7f16000ce3f0 secure :-1 s=READY pgs=34 cs=0 l=1 rev1=1 crypto rx=0x7f15f0000c00 tx=0x7f15f00046c0 comp rx=0 tx=0).stop 2024-04-25T10:29:01.789 INFO:teuthology.misc.health.smithi037.stderr:2024-04-25T10:29:01.787+0000 7f16078bf640 1 -- 172.21.15.37:0/1348648109 shutdown_connections 2024-04-25T10:29:01.789 INFO:teuthology.misc.health.smithi037.stderr:2024-04-25T10:29:01.787+0000 7f16078bf640 1 --2- 172.21.15.37:0/1348648109 >> [v2:172.21.15.203:6832/2418205237,v1:172.21.15.203:6833/2418205237] conn(0x7f15e40744e0 0x7f15e40769f0 unknown :-1 s=CLOSED pgs=43 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:29:01.789 INFO:teuthology.misc.health.smithi037.stderr:2024-04-25T10:29:01.787+0000 7f16078bf640 1 --2- 172.21.15.37:0/1348648109 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f1600164670 0x7f16000ce930 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:29:01.790 INFO:teuthology.misc.health.smithi037.stderr:2024-04-25T10:29:01.787+0000 7f16078bf640 1 --2- 172.21.15.37:0/1348648109 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f16000d4970 0x7f16000ce3f0 unknown :-1 s=CLOSED pgs=34 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:29:01.790 INFO:teuthology.misc.health.smithi037.stderr:2024-04-25T10:29:01.787+0000 7f16078bf640 1 --2- 172.21.15.37:0/1348648109 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f16000d2aa0 0x7f16000ac550 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:29:01.790 INFO:teuthology.misc.health.smithi037.stderr:2024-04-25T10:29:01.787+0000 7f16078bf640 1 -- 172.21.15.37:0/1348648109 >> 172.21.15.37:0/1348648109 conn(0x7f16000747f0 msgr2=0x7f16000d0180 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:29:01.790 INFO:teuthology.misc.health.smithi037.stderr:2024-04-25T10:29:01.787+0000 7f16078bf640 1 -- 172.21.15.37:0/1348648109 shutdown_connections 2024-04-25T10:29:01.790 INFO:teuthology.misc.health.smithi037.stderr:2024-04-25T10:29:01.787+0000 7f16078bf640 1 -- 172.21.15.37:0/1348648109 wait complete. 2024-04-25T10:29:01.799 DEBUG:teuthology.misc:8 of 8 OSDs are up 2024-04-25T10:29:01.799 INFO:tasks.ceph:Starting mds daemons in cluster ceph... 2024-04-25T10:29:01.800 INFO:tasks.ceph.mds.a:Restarting daemon 2024-04-25T10:29:01.800 DEBUG:teuthology.orchestra.run.smithi037:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mds -f --cluster ceph -i a 2024-04-25T10:29:01.803 INFO:tasks.ceph.mds.a:Started 2024-04-25T10:29:01.803 INFO:tasks.ceph.mds.c:Restarting daemon 2024-04-25T10:29:01.803 DEBUG:teuthology.orchestra.run.smithi037:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mds -f --cluster ceph -i c 2024-04-25T10:29:01.805 INFO:tasks.ceph.mds.c:Started 2024-04-25T10:29:01.805 INFO:tasks.ceph.mds.b:Restarting daemon 2024-04-25T10:29:01.805 DEBUG:teuthology.orchestra.run.smithi203:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mds -f --cluster ceph -i b 2024-04-25T10:29:01.808 INFO:tasks.ceph.mds.b:Started 2024-04-25T10:29:01.808 INFO:tasks.ceph.mds.d:Restarting daemon 2024-04-25T10:29:01.808 DEBUG:teuthology.orchestra.run.smithi203:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mds -f --cluster ceph -i d 2024-04-25T10:29:01.810 INFO:tasks.ceph.mds.d:Started 2024-04-25T10:29:01.811 INFO:tasks.ceph:Setting up CephFS filesystem(s)... 2024-04-25T10:29:01.811 DEBUG:teuthology.orchestra.run.smithi037:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-04-25T10:29:01.891 INFO:tasks.ceph.mds.a.smithi037.stdout:starting mds.a at 2024-04-25T10:29:01.891 INFO:tasks.ceph.mds.c.smithi037.stdout:starting mds.c at 2024-04-25T10:29:01.904 INFO:tasks.ceph.mds.b.smithi203.stdout:starting mds.b at 2024-04-25T10:29:01.905 INFO:tasks.ceph.mds.d.smithi203.stdout:starting mds.d at 2024-04-25T10:29:01.930 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:01.928+0000 7f6a4318f640 1 Processor -- start 2024-04-25T10:29:01.930 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:01.928+0000 7f6a4318f640 1 -- start start 2024-04-25T10:29:01.932 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:01.929+0000 7f6a4318f640 1 --2- >> v2:172.21.15.203:3300/0 conn(0x7f6a3c09cf40 0x7f6a3c0d2230 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:29:01.932 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:01.929+0000 7f6a4318f640 1 -- --> v1:172.21.15.37:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f6a3c0d1ae0 con 0x7f6a3c09cb30 2024-04-25T10:29:01.932 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:01.929+0000 7f6a4318f640 1 -- --> v1:172.21.15.203:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f6a3c0cfdc0 con 0x7f6a3c0d2770 2024-04-25T10:29:01.933 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:01.929+0000 7f6a4318f640 1 -- --> v2:172.21.15.203:3300/0 -- mon_getmap magic: 0 -- 0x7f6a3c0b9880 con 0x7f6a3c09cf40 2024-04-25T10:29:01.933 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:01.929+0000 7f6a40f2b640 1 --1- >> v1:172.21.15.37:6789/0 conn(0x7f6a3c09cb30 0x7f6a3c0b3cf0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.37:6789/0 says I am v1:172.21.15.37:56704/0 (socket says 172.21.15.37:56704) 2024-04-25T10:29:01.933 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:01.929+0000 7f6a40f2b640 1 -- 172.21.15.37:0/3366334756 learned_addr learned my addr 172.21.15.37:0/3366334756 (peer_addr_for_me v1:172.21.15.37:0/0) 2024-04-25T10:29:01.933 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:01.930+0000 7f6a3bfff640 1 --2- 172.21.15.37:0/3366334756 >> v2:172.21.15.203:3300/0 conn(0x7f6a3c09cf40 0x7f6a3c0d2230 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:29:01.933 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:01.930+0000 7f6a3b7fe640 1 -- 172.21.15.37:0/3366334756 <== mon.0 v1:172.21.15.37:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 131236123 0 0) 0x7f6a3c0d1ae0 con 0x7f6a3c09cb30 2024-04-25T10:29:01.933 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:01.930+0000 7f6a3b7fe640 1 -- 172.21.15.37:0/3366334756 --> v1:172.21.15.37:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f6a2c003520 con 0x7f6a3c09cb30 2024-04-25T10:29:01.933 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:01.930+0000 7f6a3b7fe640 1 -- 172.21.15.37:0/3366334756 <== mon.1 v1:172.21.15.203:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 4280198163 0 0) 0x7f6a3c0cfdc0 con 0x7f6a3c0d2770 2024-04-25T10:29:01.933 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:01.930+0000 7f6a3b7fe640 1 -- 172.21.15.37:0/3366334756 --> v1:172.21.15.203:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f6a3c0d1ae0 con 0x7f6a3c0d2770 2024-04-25T10:29:01.933 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:01.930+0000 7f6a3bfff640 1 -- 172.21.15.37:0/3366334756 >> v1:172.21.15.203:6789/0 conn(0x7f6a3c0d2770 legacy=0x7f6a3c1686b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:29:01.933 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:01.930+0000 7f6a3bfff640 1 -- 172.21.15.37:0/3366334756 >> v1:172.21.15.37:6789/0 conn(0x7f6a3c09cb30 legacy=0x7f6a3c0b3cf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:29:01.933 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:01.930+0000 7f6a3bfff640 1 -- 172.21.15.37:0/3366334756 --> v2:172.21.15.203:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f6a3c0d1d50 con 0x7f6a3c09cf40 2024-04-25T10:29:01.934 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:01.931+0000 7f6a3bfff640 1 --2- 172.21.15.37:0/3366334756 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f6a3c09cf40 0x7f6a3c0d2230 secure :-1 s=READY pgs=47 cs=0 l=1 rev1=1 crypto rx=0x7f6a280097a0 tx=0x7f6a2802a230 comp rx=0 tx=0).ready entity=mon.1 client_cookie=aa27e15197d691c9 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:29:01.935 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:01.931+0000 7f6a3b7fe640 1 -- 172.21.15.37:0/3366334756 <== mon.1 v2:172.21.15.203:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6a28037070 con 0x7f6a3c09cf40 2024-04-25T10:29:01.935 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:01.931+0000 7f6a3b7fe640 1 -- 172.21.15.37:0/3366334756 <== mon.1 v2:172.21.15.203:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f6a28032440 con 0x7f6a3c09cf40 2024-04-25T10:29:01.935 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:01.931+0000 7f6a3b7fe640 1 -- 172.21.15.37:0/3366334756 <== mon.1 v2:172.21.15.203:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6a280327f0 con 0x7f6a3c09cf40 2024-04-25T10:29:01.935 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:01.932+0000 7f6a4318f640 1 -- 172.21.15.37:0/3366334756 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f6a3c09cf40 msgr2=0x7f6a3c0d2230 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:29:01.935 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:01.932+0000 7f6a4318f640 1 --2- 172.21.15.37:0/3366334756 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f6a3c09cf40 0x7f6a3c0d2230 secure :-1 s=READY pgs=47 cs=0 l=1 rev1=1 crypto rx=0x7f6a280097a0 tx=0x7f6a2802a230 comp rx=0 tx=0).stop 2024-04-25T10:29:01.935 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:01.932+0000 7f6a4318f640 1 -- 172.21.15.37:0/3366334756 shutdown_connections 2024-04-25T10:29:01.935 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:01.932+0000 7f6a4318f640 1 --2- 172.21.15.37:0/3366334756 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f6a3c09cf40 0x7f6a3c0d2230 unknown :-1 s=CLOSED pgs=47 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:29:01.935 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:01.932+0000 7f6a4318f640 1 -- 172.21.15.37:0/3366334756 >> 172.21.15.37:0/3366334756 conn(0x7f6a3c0d0460 msgr2=0x7f6a3c0d0870 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:29:01.935 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:01.932+0000 7f6a4318f640 1 -- 172.21.15.37:0/3366334756 shutdown_connections 2024-04-25T10:29:01.935 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:01.932+0000 7f6a4318f640 1 -- 172.21.15.37:0/3366334756 wait complete. 2024-04-25T10:29:01.936 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:01.933+0000 7f6a4318f640 1 Processor -- start 2024-04-25T10:29:01.936 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:01.933+0000 7f6a4318f640 1 -- start start 2024-04-25T10:29:01.938 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:01.935+0000 7f6a4318f640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f6a3c09cb30 0x7f6a3c1dfd00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:29:01.938 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:01.935+0000 7f6a4318f640 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f6a3c09cf40 0x7f6a3c0ce790 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:29:01.938 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:01.935+0000 7f6a4318f640 1 --2- >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f6a3c0d2770 0x7f6a3c0cecd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:29:01.939 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:01.935+0000 7f6a4318f640 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 -- 0x7f6a3c0b4400 con 0x7f6a3c09cb30 2024-04-25T10:29:01.939 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:01.935+0000 7f6a4318f640 1 -- --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_getmap magic: 0 -- 0x7f6a3c0b9410 con 0x7f6a3c09cf40 2024-04-25T10:29:01.939 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:01.935+0000 7f6a4318f640 1 -- --> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] -- mon_getmap magic: 0 -- 0x7f6a3c0d3630 con 0x7f6a3c0d2770 2024-04-25T10:29:01.939 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:01.935+0000 7f6a40f2b640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f6a3c09cb30 0x7f6a3c1dfd00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:29:01.939 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:01.935+0000 7f6a40f2b640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f6a3c09cb30 0x7f6a3c1dfd00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.37:3300/0 says I am v2:172.21.15.37:46286/0 (socket says 172.21.15.37:46286) 2024-04-25T10:29:01.939 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:01.935+0000 7f6a40f2b640 1 -- 172.21.15.37:0/741405959 learned_addr learned my addr 172.21.15.37:0/741405959 (peer_addr_for_me v2:172.21.15.37:0/0) 2024-04-25T10:29:01.939 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:01.935+0000 7f6a4172c640 1 --2- 172.21.15.37:0/741405959 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f6a3c0d2770 0x7f6a3c0cecd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:29:01.939 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:01.935+0000 7f6a3bfff640 1 --2- 172.21.15.37:0/741405959 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f6a3c09cf40 0x7f6a3c0ce790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:29:01.939 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:01.935+0000 7f6a40f2b640 1 -- 172.21.15.37:0/741405959 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f6a3c0d2770 msgr2=0x7f6a3c0cecd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:29:01.939 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:01.935+0000 7f6a40f2b640 1 --2- 172.21.15.37:0/741405959 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f6a3c0d2770 0x7f6a3c0cecd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:29:01.939 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:01.935+0000 7f6a40f2b640 1 -- 172.21.15.37:0/741405959 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f6a3c09cf40 msgr2=0x7f6a3c0ce790 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:29:01.939 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:01.935+0000 7f6a40f2b640 1 --2- 172.21.15.37:0/741405959 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f6a3c09cf40 0x7f6a3c0ce790 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:29:01.939 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:01.935+0000 7f6a40f2b640 1 -- 172.21.15.37:0/741405959 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f6a3c1e5c60 con 0x7f6a3c09cb30 2024-04-25T10:29:01.939 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:01.935+0000 7f6a3bfff640 1 --2- 172.21.15.37:0/741405959 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f6a3c09cf40 0x7f6a3c0ce790 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T10:29:01.939 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:01.936+0000 7f6a40f2b640 1 --2- 172.21.15.37:0/741405959 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f6a3c09cb30 0x7f6a3c1dfd00 secure :-1 s=READY pgs=40 cs=0 l=1 rev1=1 crypto rx=0x7f6a3000c400 tx=0x7f6a3000c8c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:29:01.940 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:01.936+0000 7f6a397fa640 1 -- 172.21.15.37:0/741405959 <== mon.0 v2:172.21.15.37:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6a30017070 con 0x7f6a3c09cb30 2024-04-25T10:29:01.940 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:01.936+0000 7f6a397fa640 1 -- 172.21.15.37:0/741405959 <== mon.0 v2:172.21.15.37:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f6a300074e0 con 0x7f6a3c09cb30 2024-04-25T10:29:01.940 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:01.936+0000 7f6a4318f640 1 -- 172.21.15.37:0/741405959 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f6a3c1e5f40 con 0x7f6a3c09cb30 2024-04-25T10:29:01.940 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:01.936+0000 7f6a397fa640 1 -- 172.21.15.37:0/741405959 <== mon.0 v2:172.21.15.37:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6a300059c0 con 0x7f6a3c09cb30 2024-04-25T10:29:01.940 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:01.936+0000 7f6a4318f640 1 -- 172.21.15.37:0/741405959 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f6a3c1e6500 con 0x7f6a3c09cb30 2024-04-25T10:29:01.941 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:01.937+0000 7f6a4318f640 1 -- 172.21.15.37:0/741405959 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f6a3c0b5250 con 0x7f6a3c09cb30 2024-04-25T10:29:01.941 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:01.937+0000 7f6a397fa640 1 -- 172.21.15.37:0/741405959 <== mon.0 v2:172.21.15.37:3300/0 4 ==== mgrmap(e 5) ==== 96782+0+0 (secure 0 0 0) 0x7f6a3001a070 con 0x7f6a3c09cb30 2024-04-25T10:29:01.941 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:01.938+0000 7f6a397fa640 1 --2- 172.21.15.37:0/741405959 >> [v2:172.21.15.203:6832/2418205237,v1:172.21.15.203:6833/2418205237] conn(0x7f6a200744e0 0x7f6a200769f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:29:01.941 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:01.938+0000 7f6a397fa640 1 -- 172.21.15.37:0/741405959 <== mon.0 v2:172.21.15.37:3300/0 5 ==== osd_map(19..19 src has 1..19) ==== 4857+0+0 (secure 0 0 0) 0x7f6a30094350 con 0x7f6a3c09cb30 2024-04-25T10:29:01.941 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:01.938+0000 7f6a3bfff640 1 --2- 172.21.15.37:0/741405959 >> [v2:172.21.15.203:6832/2418205237,v1:172.21.15.203:6833/2418205237] conn(0x7f6a200744e0 0x7f6a200769f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:29:01.941 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:01.939+0000 7f6a3bfff640 1 --2- 172.21.15.37:0/741405959 >> [v2:172.21.15.203:6832/2418205237,v1:172.21.15.203:6833/2418205237] conn(0x7f6a200744e0 0x7f6a200769f0 secure :-1 s=READY pgs=45 cs=0 l=1 rev1=1 crypto rx=0x7f6a28034000 tx=0x7f6a2802e000 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:29:01.944 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:01.942+0000 7f6a397fa640 1 -- 172.21.15.37:0/741405959 <== mon.0 v2:172.21.15.37:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187246 (secure 0 0 0) 0x7f6a3c0b5250 con 0x7f6a3c09cb30 2024-04-25T10:29:02.130 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:02.128+0000 7f6a4318f640 1 -- 172.21.15.37:0/741405959 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7f6a3c06a140 con 0x7f6a3c09cb30 2024-04-25T10:29:02.131 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:02.129+0000 7f6a397fa640 1 -- 172.21.15.37:0/741405959 <== mon.0 v2:172.21.15.37:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v19) ==== 74+0+12896 (secure 0 0 0) 0x7f6a3005edb0 con 0x7f6a3c09cb30 2024-04-25T10:29:02.131 INFO:teuthology.orchestra.run.smithi037.stdout: 2024-04-25T10:29:02.131 INFO:teuthology.orchestra.run.smithi037.stdout:{"epoch":19,"fsid":"f4acda28-be34-492a-b286-994afd36f61d","created":"2024-04-25T10:28:43.985718+0000","modified":"2024-04-25T10:28:58.970539+0000","last_up_change":"2024-04-25T10:28:54.900654+0000","last_in_change":"2024-04-25T10:28:46.216973+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":1,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-04-25T10:28:56.202153+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"19","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"a2f79028-b332-4166-9ebb-a7024a37ea22","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.37:6816","nonce":2746074720},{"type":"v1","addr":"172.21.15.37:6817","nonce":2746074720}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.37:6818","nonce":2746074720},{"type":"v1","addr":"172.21.15.37:6819","nonce":2746074720}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.37:6822","nonce":2746074720},{"type":"v1","addr":"172.21.15.37:6823","nonce":2746074720}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.37:6820","nonce":2746074720},{"type":"v1","addr":"172.21.15.37:6821","nonce":2746074720}]},"public_addr":"172.21.15.37:6817/2746074720","cluster_addr":"172.21.15.37:6819/2746074720","heartbeat_back_addr":"172.21.15.37:6823/2746074720","heartbeat_front_addr":"172.21.15.37:6821/2746074720","state":["exists","up"]},{"osd":1,"uuid":"973159de-c2ea-4083-aecc-69373d1c1132","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":17,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.37:6800","nonce":1690341662},{"type":"v1","addr":"172.21.15.37:6801","nonce":1690341662}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.37:6802","nonce":1690341662},{"type":"v1","addr":"172.21.15.37:6803","nonce":1690341662}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.37:6806","nonce":1690341662},{"type":"v1","addr":"172.21.15.37:6807","nonce":1690341662}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.37:6804","nonce":1690341662},{"type":"v1","addr":"172.21.15.37:6805","nonce":1690341662}]},"public_addr":"172.21.15.37:6801/1690341662","cluster_addr":"172.21.15.37:6803/1690341662","heartbeat_back_addr":"172.21.15.37:6807/1690341662","heartbeat_front_addr":"172.21.15.37:6805/1690341662","state":["exists","up"]},{"osd":2,"uuid":"4ae7a0e9-cb3f-484c-a75f-f3766b721c47","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.37:6808","nonce":2754669649},{"type":"v1","addr":"172.21.15.37:6809","nonce":2754669649}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.37:6810","nonce":2754669649},{"type":"v1","addr":"172.21.15.37:6811","nonce":2754669649}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.37:6814","nonce":2754669649},{"type":"v1","addr":"172.21.15.37:6815","nonce":2754669649}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.37:6812","nonce":2754669649},{"type":"v1","addr":"172.21.15.37:6813","nonce":2754669649}]},"public_addr":"172.21.15.37:6809/2754669649","cluster_addr":"172.21.15.37:6811/2754669649","heartbeat_back_addr":"172.21.15.37:6815/2754669649","heartbeat_front_addr":"172.21.15.37:6813/2754669649","state":["exists","up"]},{"osd":3,"uuid":"1ba4a746-1608-47a7-ace2-960a5cfae934","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.37:6824","nonce":1815290618},{"type":"v1","addr":"172.21.15.37:6825","nonce":1815290618}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.37:6826","nonce":1815290618},{"type":"v1","addr":"172.21.15.37:6827","nonce":1815290618}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.37:6830","nonce":1815290618},{"type":"v1","addr":"172.21.15.37:6831","nonce":1815290618}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.37:6828","nonce":1815290618},{"type":"v1","addr":"172.21.15.37:6829","nonce":1815290618}]},"public_addr":"172.21.15.37:6825/1815290618","cluster_addr":"172.21.15.37:6827/1815290618","heartbeat_back_addr":"172.21.15.37:6831/1815290618","heartbeat_front_addr":"172.21.15.37:6829/1815290618","state":["exists","up"]},{"osd":4,"uuid":"d4f59f5d-0413-4b79-87ee-bb212c1fcf62","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6800","nonce":4051532063},{"type":"v1","addr":"172.21.15.203:6801","nonce":4051532063}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6802","nonce":4051532063},{"type":"v1","addr":"172.21.15.203:6803","nonce":4051532063}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6806","nonce":4051532063},{"type":"v1","addr":"172.21.15.203:6807","nonce":4051532063}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6804","nonce":4051532063},{"type":"v1","addr":"172.21.15.203:6805","nonce":4051532063}]},"public_addr":"172.21.15.203:6801/4051532063","cluster_addr":"172.21.15.203:6803/4051532063","heartbeat_back_addr":"172.21.15.203:6807/4051532063","heartbeat_front_addr":"172.21.15.203:6805/4051532063","state":["exists","up"]},{"osd":5,"uuid":"69710a0e-41d8-4e7f-9a10-17ce540b67be","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6824","nonce":1203822890},{"type":"v1","addr":"172.21.15.203:6825","nonce":1203822890}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6826","nonce":1203822890},{"type":"v1","addr":"172.21.15.203:6827","nonce":1203822890}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6830","nonce":1203822890},{"type":"v1","addr":"172.21.15.203:6831","nonce":1203822890}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6828","nonce":1203822890},{"type":"v1","addr":"172.21.15.203:6829","nonce":1203822890}]},"public_addr":"172.21.15.203:6825/1203822890","cluster_addr":"172.21.15.203:6827/1203822890","heartbeat_back_addr":"172.21.15.203:6831/1203822890","heartbeat_front_addr":"172.21.15.203:6829/1203822890","state":["exists","up"]},{"osd":6,"uuid":"f29dd047-3b48-473d-a235-6642a6293f20","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6810","nonce":2309615014},{"type":"v1","addr":"172.21.15.203:6812","nonce":2309615014}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6814","nonce":2309615014},{"type":"v1","addr":"172.21.15.203:6816","nonce":2309615014}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6822","nonce":2309615014},{"type":"v1","addr":"172.21.15.203:6823","nonce":2309615014}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6818","nonce":2309615014},{"type":"v1","addr":"172.21.15.203:6820","nonce":2309615014}]},"public_addr":"172.21.15.203:6812/2309615014","cluster_addr":"172.21.15.203:6816/2309615014","heartbeat_back_addr":"172.21.15.203:6823/2309615014","heartbeat_front_addr":"172.21.15.203:6820/2309615014","state":["exists","up"]},{"osd":7,"uuid":"6e691644-378b-4fdc-8e86-fb5f56e31f29","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6808","nonce":3688099014},{"type":"v1","addr":"172.21.15.203:6809","nonce":3688099014}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6811","nonce":3688099014},{"type":"v1","addr":"172.21.15.203:6813","nonce":3688099014}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6819","nonce":3688099014},{"type":"v1","addr":"172.21.15.203:6821","nonce":3688099014}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6815","nonce":3688099014},{"type":"v1","addr":"172.21.15.203:6817","nonce":3688099014}]},"public_addr":"172.21.15.203:6809/3688099014","cluster_addr":"172.21.15.203:6813/3688099014","heartbeat_back_addr":"172.21.15.203:6821/3688099014","heartbeat_front_addr":"172.21.15.203:6817/3688099014","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T10:28:51.419141+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T10:28:51.366448+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T10:28:51.348276+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T10:28:51.380519+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T10:28:51.105812+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T10:28:51.384496+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T10:28:51.388755+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T10:28:51.366958+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-04-25T10:29:02.133 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:02.131+0000 7f6a4318f640 1 -- 172.21.15.37:0/741405959 >> [v2:172.21.15.203:6832/2418205237,v1:172.21.15.203:6833/2418205237] conn(0x7f6a200744e0 msgr2=0x7f6a200769f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:29:02.134 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:02.131+0000 7f6a4318f640 1 --2- 172.21.15.37:0/741405959 >> [v2:172.21.15.203:6832/2418205237,v1:172.21.15.203:6833/2418205237] conn(0x7f6a200744e0 0x7f6a200769f0 secure :-1 s=READY pgs=45 cs=0 l=1 rev1=1 crypto rx=0x7f6a28034000 tx=0x7f6a2802e000 comp rx=0 tx=0).stop 2024-04-25T10:29:02.134 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:02.131+0000 7f6a4318f640 1 -- 172.21.15.37:0/741405959 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f6a3c09cb30 msgr2=0x7f6a3c1dfd00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:29:02.134 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:02.131+0000 7f6a4318f640 1 --2- 172.21.15.37:0/741405959 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f6a3c09cb30 0x7f6a3c1dfd00 secure :-1 s=READY pgs=40 cs=0 l=1 rev1=1 crypto rx=0x7f6a3000c400 tx=0x7f6a3000c8c0 comp rx=0 tx=0).stop 2024-04-25T10:29:02.134 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:02.131+0000 7f6a4318f640 1 -- 172.21.15.37:0/741405959 shutdown_connections 2024-04-25T10:29:02.134 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:02.131+0000 7f6a4318f640 1 --2- 172.21.15.37:0/741405959 >> [v2:172.21.15.203:6832/2418205237,v1:172.21.15.203:6833/2418205237] conn(0x7f6a200744e0 0x7f6a200769f0 unknown :-1 s=CLOSED pgs=45 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:29:02.134 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:02.131+0000 7f6a4318f640 1 --2- 172.21.15.37:0/741405959 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f6a3c0d2770 0x7f6a3c0cecd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:29:02.134 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:02.131+0000 7f6a4318f640 1 --2- 172.21.15.37:0/741405959 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f6a3c09cf40 0x7f6a3c0ce790 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:29:02.134 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:02.131+0000 7f6a4318f640 1 --2- 172.21.15.37:0/741405959 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f6a3c09cb30 0x7f6a3c1dfd00 unknown :-1 s=CLOSED pgs=40 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:29:02.134 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:02.131+0000 7f6a4318f640 1 -- 172.21.15.37:0/741405959 >> 172.21.15.37:0/741405959 conn(0x7f6a3c0d0460 msgr2=0x7f6a3c072a20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:29:02.134 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:02.131+0000 7f6a4318f640 1 -- 172.21.15.37:0/741405959 shutdown_connections 2024-04-25T10:29:02.134 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:02.131+0000 7f6a4318f640 1 -- 172.21.15.37:0/741405959 wait complete. 2024-04-25T10:29:02.144 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2024-04-25T10:28:56.202153+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '19', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 8, 'score_stable': 8, 'optimal_score': 0.25, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}] 2024-04-25T10:29:02.145 DEBUG:teuthology.orchestra.run.smithi037:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2024-04-25T10:29:02.259 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:02.256+0000 7f10dc8f2640 1 Processor -- start 2024-04-25T10:29:02.259 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:02.257+0000 7f10dc8f2640 1 -- start start 2024-04-25T10:29:02.262 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:02.258+0000 7f10dc8f2640 1 --2- >> v2:172.21.15.203:3300/0 conn(0x7f10d80d5160 0x7f10d80b4e80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:29:02.262 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:02.258+0000 7f10dc8f2640 1 --2- >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f10d80d0990 0x7f10d80d2b20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:29:02.262 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:02.258+0000 7f10dc8f2640 1 -- --> v1:172.21.15.37:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f10d80d1370 con 0x7f10d80d0da0 2024-04-25T10:29:02.262 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:02.258+0000 7f10dc8f2640 1 -- --> v2:172.21.15.203:3300/0 -- mon_getmap magic: 0 -- 0x7f10d80d1650 con 0x7f10d80d5160 2024-04-25T10:29:02.262 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:02.258+0000 7f10dc8f2640 1 -- --> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] -- mon_getmap magic: 0 -- 0x7f10d80d4680 con 0x7f10d80d0990 2024-04-25T10:29:02.262 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:02.258+0000 7f10d6d9d640 1 --1- >> v1:172.21.15.37:6789/0 conn(0x7f10d80d0da0 0x7f10d806db10 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.37:6789/0 says I am v1:172.21.15.37:56712/0 (socket says 172.21.15.37:56712) 2024-04-25T10:29:02.262 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:02.258+0000 7f10d6d9d640 1 -- 172.21.15.37:0/333405639 learned_addr learned my addr 172.21.15.37:0/333405639 (peer_addr_for_me v1:172.21.15.37:0/0) 2024-04-25T10:29:02.262 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:02.258+0000 7f10d659c640 1 --2- 172.21.15.37:0/333405639 >> v2:172.21.15.203:3300/0 conn(0x7f10d80d5160 0x7f10d80b4e80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:29:02.262 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:02.258+0000 7f10cdd9b640 1 --2- 172.21.15.37:0/333405639 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f10d80d0990 0x7f10d80d2b20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:29:02.262 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:02.258+0000 7f10d5d9b640 1 -- 172.21.15.37:0/333405639 <== mon.0 v1:172.21.15.37:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 583025268 0 0) 0x7f10d80d1370 con 0x7f10d80d0da0 2024-04-25T10:29:02.262 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:02.258+0000 7f10d5d9b640 1 -- 172.21.15.37:0/333405639 --> v1:172.21.15.37:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f10c4003520 con 0x7f10d80d0da0 2024-04-25T10:29:02.262 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:02.258+0000 7f10cdd9b640 1 -- 172.21.15.37:0/333405639 >> v2:172.21.15.203:3300/0 conn(0x7f10d80d5160 msgr2=0x7f10d80b4e80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:29:02.263 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:02.258+0000 7f10cdd9b640 1 --2- 172.21.15.37:0/333405639 >> v2:172.21.15.203:3300/0 conn(0x7f10d80d5160 0x7f10d80b4e80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:29:02.263 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:02.258+0000 7f10cdd9b640 1 -- 172.21.15.37:0/333405639 >> v1:172.21.15.37:6789/0 conn(0x7f10d80d0da0 legacy=0x7f10d806db10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:29:02.263 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:02.258+0000 7f10cdd9b640 1 -- 172.21.15.37:0/333405639 --> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f10d80d4960 con 0x7f10d80d0990 2024-04-25T10:29:02.263 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:02.258+0000 7f10d659c640 1 --2- 172.21.15.37:0/333405639 >> v2:172.21.15.203:3300/0 conn(0x7f10d80d5160 0x7f10d80b4e80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-25T10:29:02.263 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:02.259+0000 7f10cdd9b640 1 --2- 172.21.15.37:0/333405639 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f10d80d0990 0x7f10d80d2b20 secure :-1 s=READY pgs=27 cs=0 l=1 rev1=1 crypto rx=0x7f10c00097a0 tx=0x7f10c002a060 comp rx=0 tx=0).ready entity=mon.2 client_cookie=85342accb2fd58ca server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:29:02.263 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:02.259+0000 7f10d5d9b640 1 -- 172.21.15.37:0/333405639 <== mon.2 v2:172.21.15.203:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f10c0037070 con 0x7f10d80d0990 2024-04-25T10:29:02.263 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:02.259+0000 7f10d5d9b640 1 -- 172.21.15.37:0/333405639 <== mon.2 v2:172.21.15.203:3301/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f10c002adb0 con 0x7f10d80d0990 2024-04-25T10:29:02.263 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:02.259+0000 7f10d5d9b640 1 -- 172.21.15.37:0/333405639 <== mon.2 v2:172.21.15.203:3301/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f10c0032640 con 0x7f10d80d0990 2024-04-25T10:29:02.264 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:02.260+0000 7f10dc8f2640 1 -- 172.21.15.37:0/333405639 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f10d80d0990 msgr2=0x7f10d80d2b20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:29:02.264 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:02.260+0000 7f10dc8f2640 1 --2- 172.21.15.37:0/333405639 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f10d80d0990 0x7f10d80d2b20 secure :-1 s=READY pgs=27 cs=0 l=1 rev1=1 crypto rx=0x7f10c00097a0 tx=0x7f10c002a060 comp rx=0 tx=0).stop 2024-04-25T10:29:02.265 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:02.260+0000 7f10dc8f2640 1 -- 172.21.15.37:0/333405639 shutdown_connections 2024-04-25T10:29:02.265 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:02.260+0000 7f10dc8f2640 1 --2- 172.21.15.37:0/333405639 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f10d80d0990 0x7f10d80d2b20 unknown :-1 s=CLOSED pgs=27 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:29:02.265 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:02.260+0000 7f10dc8f2640 1 --2- 172.21.15.37:0/333405639 >> v2:172.21.15.203:3300/0 conn(0x7f10d80d5160 0x7f10d80b4e80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:29:02.265 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:02.260+0000 7f10dc8f2640 1 -- 172.21.15.37:0/333405639 >> 172.21.15.37:0/333405639 conn(0x7f10d80b7a80 msgr2=0x7f10d80b7e90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:29:02.265 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:02.260+0000 7f10dc8f2640 1 -- 172.21.15.37:0/333405639 shutdown_connections 2024-04-25T10:29:02.265 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:02.260+0000 7f10dc8f2640 1 -- 172.21.15.37:0/333405639 wait complete. 2024-04-25T10:29:02.265 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:02.262+0000 7f10dc8f2640 1 Processor -- start 2024-04-25T10:29:02.265 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:02.262+0000 7f10dc8f2640 1 -- start start 2024-04-25T10:29:02.267 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:02.263+0000 7f10dc8f2640 1 --2- >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f10d80d0990 0x7f10d8152ec0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:29:02.267 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:02.264+0000 7f10dc8f2640 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f10d80d0da0 0x7f10d8160470 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:29:02.268 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:02.264+0000 7f10dc8f2640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f10d80d5160 0x7f10d80cedd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:29:02.268 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:02.264+0000 7f10dc8f2640 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 -- 0x7f10d8164c40 con 0x7f10d80d5160 2024-04-25T10:29:02.268 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:02.264+0000 7f10dc8f2640 1 -- --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_getmap magic: 0 -- 0x7f10d8164ac0 con 0x7f10d80d0da0 2024-04-25T10:29:02.268 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:02.264+0000 7f10dc8f2640 1 -- --> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] -- mon_getmap magic: 0 -- 0x7f10d80d3060 con 0x7f10d80d0990 2024-04-25T10:29:02.268 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:02.264+0000 7f10d6d9d640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f10d80d5160 0x7f10d80cedd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:29:02.268 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:02.264+0000 7f10d6d9d640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f10d80d5160 0x7f10d80cedd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.37:3300/0 says I am v2:172.21.15.37:46298/0 (socket says 172.21.15.37:46298) 2024-04-25T10:29:02.268 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:02.264+0000 7f10d6d9d640 1 -- 172.21.15.37:0/3125376301 learned_addr learned my addr 172.21.15.37:0/3125376301 (peer_addr_for_me v2:172.21.15.37:0/0) 2024-04-25T10:29:02.268 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:02.264+0000 7f10d659c640 1 --2- 172.21.15.37:0/3125376301 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f10d80d0990 0x7f10d8152ec0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:29:02.268 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:02.264+0000 7f10cdd9b640 1 --2- 172.21.15.37:0/3125376301 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f10d80d0da0 0x7f10d8160470 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:29:02.268 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:02.264+0000 7f10d6d9d640 1 -- 172.21.15.37:0/3125376301 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f10d80d0990 msgr2=0x7f10d8152ec0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:29:02.268 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:02.264+0000 7f10d6d9d640 1 --2- 172.21.15.37:0/3125376301 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f10d80d0990 0x7f10d8152ec0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:29:02.268 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:02.264+0000 7f10d6d9d640 1 -- 172.21.15.37:0/3125376301 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f10d80d0da0 msgr2=0x7f10d8160470 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:29:02.268 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:02.264+0000 7f10d6d9d640 1 --2- 172.21.15.37:0/3125376301 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f10d80d0da0 0x7f10d8160470 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:29:02.269 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:02.264+0000 7f10d6d9d640 1 -- 172.21.15.37:0/3125376301 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f10d80cac90 con 0x7f10d80d5160 2024-04-25T10:29:02.269 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:02.264+0000 7f10d6d9d640 1 --2- 172.21.15.37:0/3125376301 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f10d80d5160 0x7f10d80cedd0 secure :-1 s=READY pgs=42 cs=0 l=1 rev1=1 crypto rx=0x7f10d000cad0 tx=0x7f10d000cf90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:29:02.269 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:02.264+0000 7f10cf7fe640 1 -- 172.21.15.37:0/3125376301 <== mon.0 v2:172.21.15.37:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f10d0017070 con 0x7f10d80d5160 2024-04-25T10:29:02.269 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:02.264+0000 7f10cf7fe640 1 -- 172.21.15.37:0/3125376301 <== mon.0 v2:172.21.15.37:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f10d000abd0 con 0x7f10d80d5160 2024-04-25T10:29:02.269 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:02.264+0000 7f10dc8f2640 1 -- 172.21.15.37:0/3125376301 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f10d815dcd0 con 0x7f10d80d5160 2024-04-25T10:29:02.269 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:02.264+0000 7f10cf7fe640 1 -- 172.21.15.37:0/3125376301 <== mon.0 v2:172.21.15.37:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f10d0012400 con 0x7f10d80d5160 2024-04-25T10:29:02.269 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:02.264+0000 7f10dc8f2640 1 -- 172.21.15.37:0/3125376301 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f10d80cb1e0 con 0x7f10d80d5160 2024-04-25T10:29:02.269 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:02.265+0000 7f10dc8f2640 1 -- 172.21.15.37:0/3125376301 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f10d806a140 con 0x7f10d80d5160 2024-04-25T10:29:02.270 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:02.266+0000 7f10cf7fe640 1 -- 172.21.15.37:0/3125376301 <== mon.0 v2:172.21.15.37:3300/0 4 ==== mgrmap(e 5) ==== 96782+0+0 (secure 0 0 0) 0x7f10d001a070 con 0x7f10d80d5160 2024-04-25T10:29:02.270 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:02.267+0000 7f10cf7fe640 1 --2- 172.21.15.37:0/3125376301 >> [v2:172.21.15.203:6832/2418205237,v1:172.21.15.203:6833/2418205237] conn(0x7f10b80744b0 0x7f10b80769c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:29:02.270 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:02.267+0000 7f10cf7fe640 1 -- 172.21.15.37:0/3125376301 <== mon.0 v2:172.21.15.37:3300/0 5 ==== osd_map(19..19 src has 1..19) ==== 4857+0+0 (secure 0 0 0) 0x7f10d00949a0 con 0x7f10d80d5160 2024-04-25T10:29:02.270 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:02.267+0000 7f10d659c640 1 --2- 172.21.15.37:0/3125376301 >> [v2:172.21.15.203:6832/2418205237,v1:172.21.15.203:6833/2418205237] conn(0x7f10b80744b0 0x7f10b80769c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:29:02.270 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:02.268+0000 7f10d659c640 1 --2- 172.21.15.37:0/3125376301 >> [v2:172.21.15.203:6832/2418205237,v1:172.21.15.203:6833/2418205237] conn(0x7f10b80744b0 0x7f10b80769c0 secure :-1 s=READY pgs=46 cs=0 l=1 rev1=1 crypto rx=0x7f10c8009560 tx=0x7f10c8009190 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:29:02.273 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:02.272+0000 7f10cf7fe640 1 -- 172.21.15.37:0/3125376301 <== mon.0 v2:172.21.15.37:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187246 (secure 0 0 0) 0x7f10d005eec0 con 0x7f10d80d5160 2024-04-25T10:29:02.458 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:02.456+0000 7f10dc8f2640 1 -- 172.21.15.37:0/3125376301 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) -- 0x7f10d80d3f50 con 0x7f10d80d5160 2024-04-25T10:29:02.459 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:02.456+0000 7f10cf7fe640 1 -- 172.21.15.37:0/3125376301 <== mon.0 v2:172.21.15.37:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v19) ==== 93+0+10 (secure 0 0 0) 0x7f10d00631c0 con 0x7f10d80d5160 2024-04-25T10:29:02.459 INFO:teuthology.orchestra.run.smithi037.stdout:pg_num: 1 2024-04-25T10:29:02.461 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:02.458+0000 7f10dc8f2640 1 -- 172.21.15.37:0/3125376301 >> [v2:172.21.15.203:6832/2418205237,v1:172.21.15.203:6833/2418205237] conn(0x7f10b80744b0 msgr2=0x7f10b80769c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:29:02.461 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:02.458+0000 7f10dc8f2640 1 --2- 172.21.15.37:0/3125376301 >> [v2:172.21.15.203:6832/2418205237,v1:172.21.15.203:6833/2418205237] conn(0x7f10b80744b0 0x7f10b80769c0 secure :-1 s=READY pgs=46 cs=0 l=1 rev1=1 crypto rx=0x7f10c8009560 tx=0x7f10c8009190 comp rx=0 tx=0).stop 2024-04-25T10:29:02.461 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:02.458+0000 7f10dc8f2640 1 -- 172.21.15.37:0/3125376301 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f10d80d5160 msgr2=0x7f10d80cedd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:29:02.461 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:02.458+0000 7f10dc8f2640 1 --2- 172.21.15.37:0/3125376301 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f10d80d5160 0x7f10d80cedd0 secure :-1 s=READY pgs=42 cs=0 l=1 rev1=1 crypto rx=0x7f10d000cad0 tx=0x7f10d000cf90 comp rx=0 tx=0).stop 2024-04-25T10:29:02.461 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:02.459+0000 7f10dc8f2640 1 -- 172.21.15.37:0/3125376301 shutdown_connections 2024-04-25T10:29:02.461 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:02.459+0000 7f10dc8f2640 1 --2- 172.21.15.37:0/3125376301 >> [v2:172.21.15.203:6832/2418205237,v1:172.21.15.203:6833/2418205237] conn(0x7f10b80744b0 0x7f10b80769c0 unknown :-1 s=CLOSED pgs=46 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:29:02.461 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:02.459+0000 7f10dc8f2640 1 --2- 172.21.15.37:0/3125376301 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f10d80d5160 0x7f10d80cedd0 unknown :-1 s=CLOSED pgs=42 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:29:02.461 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:02.459+0000 7f10dc8f2640 1 --2- 172.21.15.37:0/3125376301 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f10d80d0da0 0x7f10d8160470 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:29:02.461 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:02.459+0000 7f10dc8f2640 1 --2- 172.21.15.37:0/3125376301 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f10d80d0990 0x7f10d8152ec0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:29:02.462 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:02.459+0000 7f10dc8f2640 1 -- 172.21.15.37:0/3125376301 >> 172.21.15.37:0/3125376301 conn(0x7f10d80b7a80 msgr2=0x7f10d80ba550 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:29:02.462 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:02.459+0000 7f10dc8f2640 1 -- 172.21.15.37:0/3125376301 shutdown_connections 2024-04-25T10:29:02.462 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:02.459+0000 7f10dc8f2640 1 -- 172.21.15.37:0/3125376301 wait complete. 2024-04-25T10:29:02.472 DEBUG:teuthology.orchestra.run.smithi037:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-04-25T10:29:02.583 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:02.581+0000 7f98ac57d640 1 Processor -- start 2024-04-25T10:29:02.583 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:02.581+0000 7f98ac57d640 1 -- start start 2024-04-25T10:29:02.585 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:02.582+0000 7f98ac57d640 1 --2- >> v2:172.21.15.37:3300/0 conn(0x7f98a40b4e80 0x7f98a40d2b20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:29:02.585 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:02.582+0000 7f98ac57d640 1 --2- >> v2:172.21.15.203:3300/0 conn(0x7f98a40d3060 0x7f98a40b9060 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:29:02.585 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:02.582+0000 7f98ac57d640 1 -- --> v1:172.21.15.37:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f98a40d3470 con 0x7f98a40d5160 2024-04-25T10:29:02.585 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:02.582+0000 7f98ac57d640 1 -- --> v2:172.21.15.37:3300/0 -- mon_getmap magic: 0 -- 0x7f98a40d1650 con 0x7f98a40b4e80 2024-04-25T10:29:02.585 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:02.582+0000 7f98ac57d640 1 -- --> v2:172.21.15.203:3300/0 -- mon_getmap magic: 0 -- 0x7f98a40b5290 con 0x7f98a40d3060 2024-04-25T10:29:02.586 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:02.583+0000 7f98a9b18640 1 --2- >> v2:172.21.15.37:3300/0 conn(0x7f98a40b4e80 0x7f98a40d2b20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:29:02.586 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:02.583+0000 7f98a9b18640 1 --2- >> v2:172.21.15.37:3300/0 conn(0x7f98a40b4e80 0x7f98a40d2b20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.37:3300/0 says I am v2:172.21.15.37:46300/0 (socket says 172.21.15.37:46300) 2024-04-25T10:29:02.586 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:02.583+0000 7f98a9b18640 1 -- 172.21.15.37:0/1531535902 learned_addr learned my addr 172.21.15.37:0/1531535902 (peer_addr_for_me v2:172.21.15.37:0/0) 2024-04-25T10:29:02.586 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:02.583+0000 7f98aab1a640 1 --2- 172.21.15.37:0/1531535902 >> v2:172.21.15.203:3300/0 conn(0x7f98a40d3060 0x7f98a40b9060 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:29:02.586 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:02.583+0000 7f98a9b18640 1 -- 172.21.15.37:0/1531535902 >> v2:172.21.15.203:3300/0 conn(0x7f98a40d3060 msgr2=0x7f98a40b9060 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:29:02.586 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:02.583+0000 7f98a9b18640 1 --2- 172.21.15.37:0/1531535902 >> v2:172.21.15.203:3300/0 conn(0x7f98a40d3060 0x7f98a40b9060 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:29:02.586 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:02.583+0000 7f98a9b18640 1 -- 172.21.15.37:0/1531535902 >> v1:172.21.15.37:6789/0 conn(0x7f98a40d5160 legacy=0x7f98a40d0990 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:29:02.586 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:02.583+0000 7f98a9317640 1 -- 172.21.15.37:0/1531535902 <== mon.0 v1:172.21.15.37:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 2701719889 0 0) 0x7f98a0001110 con 0x7f98a40d5160 2024-04-25T10:29:02.586 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:02.583+0000 7f98a9b18640 1 -- 172.21.15.37:0/1531535902 --> v2:172.21.15.37:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f98a40d10a0 con 0x7f98a40b4e80 2024-04-25T10:29:02.587 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:02.584+0000 7f98a9b18640 1 --2- 172.21.15.37:0/1531535902 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f98a40b4e80 0x7f98a40d2b20 secure :-1 s=READY pgs=44 cs=0 l=1 rev1=1 crypto rx=0x7f98a408f050 tx=0x7f989402a1e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b6c262f3ccd63f30 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:29:02.588 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:02.584+0000 7f98a9317640 1 -- 172.21.15.37:0/1531535902 <== mon.0 v2:172.21.15.37:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9894037070 con 0x7f98a40b4e80 2024-04-25T10:29:02.588 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:02.584+0000 7f98a9317640 1 -- 172.21.15.37:0/1531535902 <== mon.0 v2:172.21.15.37:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f9894032440 con 0x7f98a40b4e80 2024-04-25T10:29:02.588 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:02.584+0000 7f98a9317640 1 -- 172.21.15.37:0/1531535902 <== mon.0 v2:172.21.15.37:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9894032810 con 0x7f98a40b4e80 2024-04-25T10:29:02.588 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:02.584+0000 7f98ac57d640 1 -- 172.21.15.37:0/1531535902 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f98a40b4e80 msgr2=0x7f98a40d2b20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:29:02.588 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:02.584+0000 7f98ac57d640 1 --2- 172.21.15.37:0/1531535902 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f98a40b4e80 0x7f98a40d2b20 secure :-1 s=READY pgs=44 cs=0 l=1 rev1=1 crypto rx=0x7f98a408f050 tx=0x7f989402a1e0 comp rx=0 tx=0).stop 2024-04-25T10:29:02.588 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:02.585+0000 7f98ac57d640 1 -- 172.21.15.37:0/1531535902 shutdown_connections 2024-04-25T10:29:02.588 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:02.585+0000 7f98ac57d640 1 --2- 172.21.15.37:0/1531535902 >> v2:172.21.15.203:3300/0 conn(0x7f98a40d3060 0x7f98a40b9060 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:29:02.588 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:02.585+0000 7f98ac57d640 1 --2- 172.21.15.37:0/1531535902 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f98a40b4e80 0x7f98a40d2b20 unknown :-1 s=CLOSED pgs=44 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:29:02.588 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:02.585+0000 7f98ac57d640 1 -- 172.21.15.37:0/1531535902 >> 172.21.15.37:0/1531535902 conn(0x7f98a40b7a80 msgr2=0x7f98a40b7e90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:29:02.588 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:02.585+0000 7f98ac57d640 1 -- 172.21.15.37:0/1531535902 shutdown_connections 2024-04-25T10:29:02.588 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:02.585+0000 7f98ac57d640 1 -- 172.21.15.37:0/1531535902 wait complete. 2024-04-25T10:29:02.589 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:02.586+0000 7f98ac57d640 1 Processor -- start 2024-04-25T10:29:02.589 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:02.586+0000 7f98ac57d640 1 -- start start 2024-04-25T10:29:02.591 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:02.588+0000 7f98ac57d640 1 --2- >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f98a40b4e80 0x7f98a41f71c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:29:02.591 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:02.588+0000 7f98ac57d640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f98a40d3060 0x7f98a415c010 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:29:02.591 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:02.588+0000 7f98ac57d640 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f98a40d5160 0x7f98a41575c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:29:02.592 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:02.588+0000 7f98ac57d640 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 -- 0x7f98a40d4960 con 0x7f98a40d3060 2024-04-25T10:29:02.592 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:02.588+0000 7f98ac57d640 1 -- --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_getmap magic: 0 -- 0x7f98a40d4680 con 0x7f98a40d5160 2024-04-25T10:29:02.592 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:02.588+0000 7f98ac57d640 1 -- --> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] -- mon_getmap magic: 0 -- 0x7f98a40d4c40 con 0x7f98a40b4e80 2024-04-25T10:29:02.592 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:02.588+0000 7f98a9b18640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f98a40d3060 0x7f98a415c010 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:29:02.592 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:02.588+0000 7f98a9b18640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f98a40d3060 0x7f98a415c010 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.37:3300/0 says I am v2:172.21.15.37:46306/0 (socket says 172.21.15.37:46306) 2024-04-25T10:29:02.592 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:02.588+0000 7f98a9b18640 1 -- 172.21.15.37:0/2812111182 learned_addr learned my addr 172.21.15.37:0/2812111182 (peer_addr_for_me v2:172.21.15.37:0/0) 2024-04-25T10:29:02.592 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:02.588+0000 7f98aa319640 1 --2- 172.21.15.37:0/2812111182 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f98a40b4e80 0x7f98a41f71c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:29:02.592 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:02.588+0000 7f98aab1a640 1 --2- 172.21.15.37:0/2812111182 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f98a40d5160 0x7f98a41575c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:29:02.592 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:02.588+0000 7f98a9b18640 1 -- 172.21.15.37:0/2812111182 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f98a40b4e80 msgr2=0x7f98a41f71c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:29:02.593 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:02.588+0000 7f98a9b18640 1 --2- 172.21.15.37:0/2812111182 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f98a40b4e80 0x7f98a41f71c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:29:02.593 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:02.588+0000 7f98a9b18640 1 -- 172.21.15.37:0/2812111182 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f98a40d5160 msgr2=0x7f98a41575c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:29:02.593 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:02.588+0000 7f98a9b18640 1 --2- 172.21.15.37:0/2812111182 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f98a40d5160 0x7f98a41575c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:29:02.593 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:02.588+0000 7f98a9b18640 1 -- 172.21.15.37:0/2812111182 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f98a4160b30 con 0x7f98a40d3060 2024-04-25T10:29:02.593 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:02.588+0000 7f98a9b18640 1 --2- 172.21.15.37:0/2812111182 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f98a40d3060 0x7f98a415c010 secure :-1 s=READY pgs=45 cs=0 l=1 rev1=1 crypto rx=0x7f98940090b0 tx=0x7f98940046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:29:02.593 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:02.588+0000 7f989affd640 1 -- 172.21.15.37:0/2812111182 <== mon.0 v2:172.21.15.37:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9894037040 con 0x7f98a40d3060 2024-04-25T10:29:02.593 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:02.588+0000 7f989affd640 1 -- 172.21.15.37:0/2812111182 <== mon.0 v2:172.21.15.37:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f9894004230 con 0x7f98a40d3060 2024-04-25T10:29:02.593 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:02.589+0000 7f989affd640 1 -- 172.21.15.37:0/2812111182 <== mon.0 v2:172.21.15.37:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9894003610 con 0x7f98a40d3060 2024-04-25T10:29:02.593 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:02.589+0000 7f98ac57d640 1 -- 172.21.15.37:0/2812111182 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f98a4160e10 con 0x7f98a40d3060 2024-04-25T10:29:02.593 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:02.589+0000 7f98ac57d640 1 -- 172.21.15.37:0/2812111182 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f98a41613d0 con 0x7f98a40d3060 2024-04-25T10:29:02.593 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:02.589+0000 7f98ac57d640 1 -- 172.21.15.37:0/2812111182 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f98a40d5600 con 0x7f98a40d3060 2024-04-25T10:29:02.594 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:02.590+0000 7f989affd640 1 -- 172.21.15.37:0/2812111182 <== mon.0 v2:172.21.15.37:3300/0 4 ==== mgrmap(e 5) ==== 96782+0+0 (secure 0 0 0) 0x7f98940037b0 con 0x7f98a40d3060 2024-04-25T10:29:02.594 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:02.591+0000 7f989affd640 1 --2- 172.21.15.37:0/2812111182 >> [v2:172.21.15.203:6832/2418205237,v1:172.21.15.203:6833/2418205237] conn(0x7f98880744b0 0x7f98880769c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:29:02.594 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:02.591+0000 7f989affd640 1 -- 172.21.15.37:0/2812111182 <== mon.0 v2:172.21.15.37:3300/0 5 ==== osd_map(19..19 src has 1..19) ==== 4857+0+0 (secure 0 0 0) 0x7f98940b5ae0 con 0x7f98a40d3060 2024-04-25T10:29:02.594 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:02.591+0000 7f98aa319640 1 --2- 172.21.15.37:0/2812111182 >> [v2:172.21.15.203:6832/2418205237,v1:172.21.15.203:6833/2418205237] conn(0x7f98880744b0 0x7f98880769c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:29:02.594 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:02.591+0000 7f98aa319640 1 --2- 172.21.15.37:0/2812111182 >> [v2:172.21.15.203:6832/2418205237,v1:172.21.15.203:6833/2418205237] conn(0x7f98880744b0 0x7f98880769c0 secure :-1 s=READY pgs=47 cs=0 l=1 rev1=1 crypto rx=0x7f98a0000f60 tx=0x7f98a0009040 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:29:02.597 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:02.595+0000 7f989affd640 1 -- 172.21.15.37:0/2812111182 <== mon.0 v2:172.21.15.37:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187246 (secure 0 0 0) 0x7f98a40d5600 con 0x7f98a40d3060 2024-04-25T10:29:02.820 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:02.818+0000 7f98ac57d640 1 -- 172.21.15.37:0/2812111182 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f98a41636e0 con 0x7f98a40d3060 2024-04-25T10:29:02.820 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:02.818+0000 7f989affd640 1 -- 172.21.15.37:0/2812111182 <== mon.0 v2:172.21.15.37:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 2 v2) ==== 93+0+1389 (secure 0 0 0) 0x7f9894080000 con 0x7f98a40d3060 2024-04-25T10:29:02.821 INFO:teuthology.orchestra.run.smithi037.stdout: 2024-04-25T10:29:02.821 INFO:teuthology.orchestra.run.smithi037.stdout:{"epoch":2,"default_fscid":-1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":4278,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.37:6834/179723013","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.37:6832","nonce":179723013},{"type":"v1","addr":"172.21.15.37:6834","nonce":179723013}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":2}],"filesystems":[]} 2024-04-25T10:29:02.821 INFO:teuthology.orchestra.run.smithi037.stderr:dumped fsmap epoch 2 2024-04-25T10:29:02.823 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:02.820+0000 7f98ac57d640 1 -- 172.21.15.37:0/2812111182 >> [v2:172.21.15.203:6832/2418205237,v1:172.21.15.203:6833/2418205237] conn(0x7f98880744b0 msgr2=0x7f98880769c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:29:02.823 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:02.820+0000 7f98ac57d640 1 --2- 172.21.15.37:0/2812111182 >> [v2:172.21.15.203:6832/2418205237,v1:172.21.15.203:6833/2418205237] conn(0x7f98880744b0 0x7f98880769c0 secure :-1 s=READY pgs=47 cs=0 l=1 rev1=1 crypto rx=0x7f98a0000f60 tx=0x7f98a0009040 comp rx=0 tx=0).stop 2024-04-25T10:29:02.823 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:02.820+0000 7f98ac57d640 1 -- 172.21.15.37:0/2812111182 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f98a40d3060 msgr2=0x7f98a415c010 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:29:02.823 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:02.820+0000 7f98ac57d640 1 --2- 172.21.15.37:0/2812111182 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f98a40d3060 0x7f98a415c010 secure :-1 s=READY pgs=45 cs=0 l=1 rev1=1 crypto rx=0x7f98940090b0 tx=0x7f98940046c0 comp rx=0 tx=0).stop 2024-04-25T10:29:02.823 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:02.820+0000 7f98ac57d640 1 -- 172.21.15.37:0/2812111182 shutdown_connections 2024-04-25T10:29:02.823 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:02.820+0000 7f98ac57d640 1 --2- 172.21.15.37:0/2812111182 >> [v2:172.21.15.203:6832/2418205237,v1:172.21.15.203:6833/2418205237] conn(0x7f98880744b0 0x7f98880769c0 unknown :-1 s=CLOSED pgs=47 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:29:02.823 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:02.820+0000 7f98ac57d640 1 --2- 172.21.15.37:0/2812111182 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f98a40d5160 0x7f98a41575c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:29:02.823 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:02.820+0000 7f98ac57d640 1 --2- 172.21.15.37:0/2812111182 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f98a40d3060 0x7f98a415c010 unknown :-1 s=CLOSED pgs=45 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:29:02.823 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:02.820+0000 7f98ac57d640 1 --2- 172.21.15.37:0/2812111182 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f98a40b4e80 0x7f98a41f71c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:29:02.823 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:02.820+0000 7f98ac57d640 1 -- 172.21.15.37:0/2812111182 >> 172.21.15.37:0/2812111182 conn(0x7f98a40b7a80 msgr2=0x7f98a40b54d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:29:02.823 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:02.820+0000 7f98ac57d640 1 -- 172.21.15.37:0/2812111182 shutdown_connections 2024-04-25T10:29:02.823 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:02.820+0000 7f98ac57d640 1 -- 172.21.15.37:0/2812111182 wait complete. 2024-04-25T10:29:04.834 DEBUG:teuthology.orchestra.run.smithi037:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-04-25T10:29:04.960 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:04.958+0000 7fa952849640 1 Processor -- start 2024-04-25T10:29:04.960 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:04.958+0000 7fa952849640 1 -- start start 2024-04-25T10:29:04.962 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:04.959+0000 7fa952849640 1 --2- >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7fa94c0b5fd0 0x7fa94c0b2ca0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:29:04.962 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:04.959+0000 7fa952849640 1 --2- >> v2:172.21.15.203:3300/0 conn(0x7fa94c08cc90 0x7fa94c08d0a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:29:04.962 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:04.959+0000 7fa952849640 1 --2- >> v2:172.21.15.37:3300/0 conn(0x7fa94c08d5e0 0x7fa94c0d18c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:29:04.962 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:04.959+0000 7fa952849640 1 -- --> v2:172.21.15.37:3300/0 -- mon_getmap magic: 0 -- 0x7fa94c0cf120 con 0x7fa94c08d5e0 2024-04-25T10:29:04.962 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:04.959+0000 7fa952849640 1 -- --> v2:172.21.15.203:3300/0 -- mon_getmap magic: 0 -- 0x7fa94c0cf400 con 0x7fa94c08cc90 2024-04-25T10:29:04.962 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:04.959+0000 7fa952849640 1 -- --> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] -- mon_getmap magic: 0 -- 0x7fa94c0cf6e0 con 0x7fa94c0b5fd0 2024-04-25T10:29:04.962 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:04.959+0000 7fa950de6640 1 --2- >> v2:172.21.15.37:3300/0 conn(0x7fa94c08d5e0 0x7fa94c0d18c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:29:04.962 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:04.959+0000 7fa950de6640 1 --2- >> v2:172.21.15.37:3300/0 conn(0x7fa94c08d5e0 0x7fa94c0d18c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.37:3300/0 says I am v2:172.21.15.37:46308/0 (socket says 172.21.15.37:46308) 2024-04-25T10:29:04.962 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:04.959+0000 7fa950de6640 1 -- 172.21.15.37:0/1909487425 learned_addr learned my addr 172.21.15.37:0/1909487425 (peer_addr_for_me v2:172.21.15.37:0/0) 2024-04-25T10:29:04.963 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:04.960+0000 7fa94b7fe640 1 --2- 172.21.15.37:0/1909487425 >> v2:172.21.15.203:3300/0 conn(0x7fa94c08cc90 0x7fa94c08d0a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:29:04.963 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:04.960+0000 7fa94bfff640 1 --2- 172.21.15.37:0/1909487425 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7fa94c0b5fd0 0x7fa94c0b2ca0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:29:04.963 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:04.960+0000 7fa950de6640 1 -- 172.21.15.37:0/1909487425 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7fa94c0b5fd0 msgr2=0x7fa94c0b2ca0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:29:04.963 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:04.960+0000 7fa950de6640 1 --2- 172.21.15.37:0/1909487425 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7fa94c0b5fd0 0x7fa94c0b2ca0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:29:04.963 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:04.960+0000 7fa950de6640 1 -- 172.21.15.37:0/1909487425 >> v2:172.21.15.203:3300/0 conn(0x7fa94c08cc90 msgr2=0x7fa94c08d0a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:29:04.963 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:04.960+0000 7fa950de6640 1 --2- 172.21.15.37:0/1909487425 >> v2:172.21.15.203:3300/0 conn(0x7fa94c08cc90 0x7fa94c08d0a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:29:04.963 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:04.960+0000 7fa950de6640 1 -- 172.21.15.37:0/1909487425 --> v2:172.21.15.37:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fa94c0cf9c0 con 0x7fa94c08d5e0 2024-04-25T10:29:04.964 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:04.960+0000 7fa950de6640 1 --2- 172.21.15.37:0/1909487425 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fa94c08d5e0 0x7fa94c0d18c0 secure :-1 s=READY pgs=46 cs=0 l=1 rev1=1 crypto rx=0x7fa94c074790 tx=0x7fa94402a610 comp rx=0 tx=0).ready entity=mon.0 client_cookie=9761dd43d686287b server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:29:04.964 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:04.960+0000 7fa94affd640 1 -- 172.21.15.37:0/1909487425 <== mon.0 v2:172.21.15.37:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa944036070 con 0x7fa94c08d5e0 2024-04-25T10:29:04.964 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:04.961+0000 7fa94affd640 1 -- 172.21.15.37:0/1909487425 <== mon.0 v2:172.21.15.37:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fa944031600 con 0x7fa94c08d5e0 2024-04-25T10:29:04.964 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:04.961+0000 7fa94affd640 1 -- 172.21.15.37:0/1909487425 <== mon.0 v2:172.21.15.37:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa9440319d0 con 0x7fa94c08d5e0 2024-04-25T10:29:04.964 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:04.961+0000 7fa952849640 1 -- 172.21.15.37:0/1909487425 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fa94c08d5e0 msgr2=0x7fa94c0d18c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:29:04.965 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:04.961+0000 7fa952849640 1 --2- 172.21.15.37:0/1909487425 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fa94c08d5e0 0x7fa94c0d18c0 secure :-1 s=READY pgs=46 cs=0 l=1 rev1=1 crypto rx=0x7fa94c074790 tx=0x7fa94402a610 comp rx=0 tx=0).stop 2024-04-25T10:29:04.965 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:04.961+0000 7fa952849640 1 -- 172.21.15.37:0/1909487425 shutdown_connections 2024-04-25T10:29:04.965 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:04.961+0000 7fa952849640 1 --2- 172.21.15.37:0/1909487425 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fa94c08d5e0 0x7fa94c0d18c0 unknown :-1 s=CLOSED pgs=46 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:29:04.965 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:04.961+0000 7fa952849640 1 --2- 172.21.15.37:0/1909487425 >> v2:172.21.15.203:3300/0 conn(0x7fa94c08cc90 0x7fa94c08d0a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:29:04.965 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:04.961+0000 7fa952849640 1 --2- 172.21.15.37:0/1909487425 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7fa94c0b5fd0 0x7fa94c0b2ca0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:29:04.965 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:04.961+0000 7fa952849640 1 -- 172.21.15.37:0/1909487425 >> 172.21.15.37:0/1909487425 conn(0x7fa94c0b5440 msgr2=0x7fa94c0b5850 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:29:04.965 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:04.961+0000 7fa952849640 1 -- 172.21.15.37:0/1909487425 shutdown_connections 2024-04-25T10:29:04.965 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:04.961+0000 7fa952849640 1 -- 172.21.15.37:0/1909487425 wait complete. 2024-04-25T10:29:04.965 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:04.963+0000 7fa952849640 1 Processor -- start 2024-04-25T10:29:04.966 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:04.963+0000 7fa952849640 1 -- start start 2024-04-25T10:29:04.968 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:04.965+0000 7fa952849640 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fa94c08cc90 0x7fa94c158f40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:29:04.968 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:04.965+0000 7fa952849640 1 --2- >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7fa94c08d5e0 0x7fa94c15da90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:29:04.968 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:04.965+0000 7fa952849640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fa94c0b5fd0 0x7fa94c1f56c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:29:04.969 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:04.965+0000 7fa952849640 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 -- 0x7fa94c0d1f80 con 0x7fa94c0b5fd0 2024-04-25T10:29:04.969 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:04.965+0000 7fa952849640 1 -- --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_getmap magic: 0 -- 0x7fa94c0d1e00 con 0x7fa94c08cc90 2024-04-25T10:29:04.969 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:04.965+0000 7fa952849640 1 -- --> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] -- mon_getmap magic: 0 -- 0x7fa94c0d2100 con 0x7fa94c08d5e0 2024-04-25T10:29:04.969 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:04.965+0000 7fa950de6640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fa94c0b5fd0 0x7fa94c1f56c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:29:04.969 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:04.965+0000 7fa950de6640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fa94c0b5fd0 0x7fa94c1f56c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.37:3300/0 says I am v2:172.21.15.37:46324/0 (socket says 172.21.15.37:46324) 2024-04-25T10:29:04.969 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:04.965+0000 7fa950de6640 1 -- 172.21.15.37:0/3744669601 learned_addr learned my addr 172.21.15.37:0/3744669601 (peer_addr_for_me v2:172.21.15.37:0/0) 2024-04-25T10:29:04.969 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:04.965+0000 7fa94bfff640 1 --2- 172.21.15.37:0/3744669601 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fa94c08cc90 0x7fa94c158f40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:29:04.969 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:04.965+0000 7fa94b7fe640 1 --2- 172.21.15.37:0/3744669601 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7fa94c08d5e0 0x7fa94c15da90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:29:04.969 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:04.965+0000 7fa950de6640 1 -- 172.21.15.37:0/3744669601 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7fa94c08d5e0 msgr2=0x7fa94c15da90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:29:04.969 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:04.965+0000 7fa950de6640 1 --2- 172.21.15.37:0/3744669601 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7fa94c08d5e0 0x7fa94c15da90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:29:04.969 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:04.965+0000 7fa950de6640 1 -- 172.21.15.37:0/3744669601 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fa94c08cc90 msgr2=0x7fa94c158f40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:29:04.969 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:04.965+0000 7fa950de6640 1 --2- 172.21.15.37:0/3744669601 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fa94c08cc90 0x7fa94c158f40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:29:04.969 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:04.965+0000 7fa950de6640 1 -- 172.21.15.37:0/3744669601 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fa94c0cae40 con 0x7fa94c0b5fd0 2024-04-25T10:29:04.969 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:04.965+0000 7fa950de6640 1 --2- 172.21.15.37:0/3744669601 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fa94c0b5fd0 0x7fa94c1f56c0 secure :-1 s=READY pgs=47 cs=0 l=1 rev1=1 crypto rx=0x7fa944009f40 tx=0x7fa944004a40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:29:04.970 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:04.966+0000 7fa948ff9640 1 -- 172.21.15.37:0/3744669601 <== mon.0 v2:172.21.15.37:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa944036050 con 0x7fa94c0b5fd0 2024-04-25T10:29:04.970 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:04.966+0000 7fa948ff9640 1 -- 172.21.15.37:0/3744669601 <== mon.0 v2:172.21.15.37:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fa944042070 con 0x7fa94c0b5fd0 2024-04-25T10:29:04.970 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:04.966+0000 7fa952849640 1 -- 172.21.15.37:0/3744669601 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fa94c0cb120 con 0x7fa94c0b5fd0 2024-04-25T10:29:04.970 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:04.966+0000 7fa948ff9640 1 -- 172.21.15.37:0/3744669601 <== mon.0 v2:172.21.15.37:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa944009510 con 0x7fa94c0b5fd0 2024-04-25T10:29:04.970 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:04.966+0000 7fa952849640 1 -- 172.21.15.37:0/3744669601 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fa94c0cb6e0 con 0x7fa94c0b5fd0 2024-04-25T10:29:04.971 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:04.967+0000 7fa952849640 1 -- 172.21.15.37:0/3744669601 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fa94c06a140 con 0x7fa94c0b5fd0 2024-04-25T10:29:04.971 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:04.967+0000 7fa948ff9640 1 -- 172.21.15.37:0/3744669601 <== mon.0 v2:172.21.15.37:3300/0 4 ==== mgrmap(e 5) ==== 96782+0+0 (secure 0 0 0) 0x7fa944007bd0 con 0x7fa94c0b5fd0 2024-04-25T10:29:04.971 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:04.968+0000 7fa948ff9640 1 --2- 172.21.15.37:0/3744669601 >> [v2:172.21.15.203:6832/2418205237,v1:172.21.15.203:6833/2418205237] conn(0x7fa92c0744b0 0x7fa92c0769c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:29:04.971 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:04.968+0000 7fa948ff9640 1 -- 172.21.15.37:0/3744669601 <== mon.0 v2:172.21.15.37:3300/0 5 ==== osd_map(19..19 src has 1..19) ==== 4857+0+0 (secure 0 0 0) 0x7fa9440b5c00 con 0x7fa94c0b5fd0 2024-04-25T10:29:04.971 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:04.968+0000 7fa94bfff640 1 --2- 172.21.15.37:0/3744669601 >> [v2:172.21.15.203:6832/2418205237,v1:172.21.15.203:6833/2418205237] conn(0x7fa92c0744b0 0x7fa92c0769c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:29:04.971 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:04.969+0000 7fa94bfff640 1 --2- 172.21.15.37:0/3744669601 >> [v2:172.21.15.203:6832/2418205237,v1:172.21.15.203:6833/2418205237] conn(0x7fa92c0744b0 0x7fa92c0769c0 secure :-1 s=READY pgs=51 cs=0 l=1 rev1=1 crypto rx=0x7fa94c073c80 tx=0x7fa93c008040 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:29:04.974 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:04.972+0000 7fa948ff9640 1 -- 172.21.15.37:0/3744669601 <== mon.0 v2:172.21.15.37:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187246 (secure 0 0 0) 0x7fa944080080 con 0x7fa94c0b5fd0 2024-04-25T10:29:05.198 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:05.196+0000 7fa952849640 1 -- 172.21.15.37:0/3744669601 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7fa94c15dfd0 con 0x7fa94c0b5fd0 2024-04-25T10:29:05.199 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:05.197+0000 7fa948ff9640 1 -- 172.21.15.37:0/3744669601 <== mon.0 v2:172.21.15.37:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 3 v3) ==== 93+0+3867 (secure 0 0 0) 0x7fa944084380 con 0x7fa94c0b5fd0 2024-04-25T10:29:05.199 INFO:teuthology.orchestra.run.smithi037.stdout: 2024-04-25T10:29:05.199 INFO:teuthology.orchestra.run.smithi037.stdout:{"epoch":3,"default_fscid":-1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":4232,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.37:6835/1911169480","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.37:6833","nonce":1911169480},{"type":"v1","addr":"172.21.15.37:6835","nonce":1911169480}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3},{"gid":4241,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.203:6835/3873243328","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6834","nonce":3873243328},{"type":"v1","addr":"172.21.15.203:6835","nonce":3873243328}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3},{"gid":4278,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.37:6834/179723013","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.37:6832","nonce":179723013},{"type":"v1","addr":"172.21.15.37:6834","nonce":179723013}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":2},{"gid":4279,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.203:6837/3215191422","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6836","nonce":3215191422},{"type":"v1","addr":"172.21.15.203:6837","nonce":3215191422}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3}],"filesystems":[]} 2024-04-25T10:29:05.199 INFO:teuthology.orchestra.run.smithi037.stderr:dumped fsmap epoch 3 2024-04-25T10:29:05.201 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:05.199+0000 7fa952849640 1 -- 172.21.15.37:0/3744669601 >> [v2:172.21.15.203:6832/2418205237,v1:172.21.15.203:6833/2418205237] conn(0x7fa92c0744b0 msgr2=0x7fa92c0769c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:29:05.201 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:05.199+0000 7fa952849640 1 --2- 172.21.15.37:0/3744669601 >> [v2:172.21.15.203:6832/2418205237,v1:172.21.15.203:6833/2418205237] conn(0x7fa92c0744b0 0x7fa92c0769c0 secure :-1 s=READY pgs=51 cs=0 l=1 rev1=1 crypto rx=0x7fa94c073c80 tx=0x7fa93c008040 comp rx=0 tx=0).stop 2024-04-25T10:29:05.201 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:05.199+0000 7fa952849640 1 -- 172.21.15.37:0/3744669601 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fa94c0b5fd0 msgr2=0x7fa94c1f56c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:29:05.201 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:05.199+0000 7fa952849640 1 --2- 172.21.15.37:0/3744669601 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fa94c0b5fd0 0x7fa94c1f56c0 secure :-1 s=READY pgs=47 cs=0 l=1 rev1=1 crypto rx=0x7fa944009f40 tx=0x7fa944004a40 comp rx=0 tx=0).stop 2024-04-25T10:29:05.201 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:05.199+0000 7fa952849640 1 -- 172.21.15.37:0/3744669601 shutdown_connections 2024-04-25T10:29:05.202 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:05.199+0000 7fa952849640 1 --2- 172.21.15.37:0/3744669601 >> [v2:172.21.15.203:6832/2418205237,v1:172.21.15.203:6833/2418205237] conn(0x7fa92c0744b0 0x7fa92c0769c0 unknown :-1 s=CLOSED pgs=51 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:29:05.202 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:05.199+0000 7fa952849640 1 --2- 172.21.15.37:0/3744669601 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fa94c0b5fd0 0x7fa94c1f56c0 unknown :-1 s=CLOSED pgs=47 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:29:05.202 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:05.199+0000 7fa952849640 1 --2- 172.21.15.37:0/3744669601 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7fa94c08d5e0 0x7fa94c15da90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:29:05.202 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:05.199+0000 7fa952849640 1 --2- 172.21.15.37:0/3744669601 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fa94c08cc90 0x7fa94c158f40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:29:05.202 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:05.199+0000 7fa952849640 1 -- 172.21.15.37:0/3744669601 >> 172.21.15.37:0/3744669601 conn(0x7fa94c0b5440 msgr2=0x7fa94c0b8580 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:29:05.202 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:05.199+0000 7fa952849640 1 -- 172.21.15.37:0/3744669601 shutdown_connections 2024-04-25T10:29:05.202 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:05.199+0000 7fa952849640 1 -- 172.21.15.37:0/3744669601 wait complete. 2024-04-25T10:29:05.213 DEBUG:teuthology.orchestra.run.smithi037:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-04-25T10:29:05.323 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:05.321+0000 7f80c08aa640 1 Processor -- start 2024-04-25T10:29:05.323 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:05.321+0000 7f80c08aa640 1 -- start start 2024-04-25T10:29:05.325 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:05.323+0000 7f80c08aa640 1 --2- >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f80bc0b7cd0 0x7f80bc160a00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:29:05.325 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:05.323+0000 7f80c08aa640 1 --2- >> v2:172.21.15.37:3300/0 conn(0x7f80bc160f40 0x7f80bc161880 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:29:05.326 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:05.323+0000 7f80c08aa640 1 -- --> v1:172.21.15.37:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f80bc0d5840 con 0x7f80bc0d5430 2024-04-25T10:29:05.326 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:05.323+0000 7f80c08aa640 1 -- --> v2:172.21.15.37:3300/0 -- mon_getmap magic: 0 -- 0x7f80bc0d4420 con 0x7f80bc160f40 2024-04-25T10:29:05.326 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:05.323+0000 7f80c08aa640 1 -- --> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] -- mon_getmap magic: 0 -- 0x7f80bc0d4670 con 0x7f80bc0b7cd0 2024-04-25T10:29:05.326 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:05.323+0000 7f80bad9d640 1 --2- >> v2:172.21.15.37:3300/0 conn(0x7f80bc160f40 0x7f80bc161880 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:29:05.326 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:05.323+0000 7f80ba59c640 1 --1- >> v1:172.21.15.37:6789/0 conn(0x7f80bc0d5430 0x7f80bc0b75c0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.37:6789/0 says I am v1:172.21.15.37:56722/0 (socket says 172.21.15.37:56722) 2024-04-25T10:29:05.326 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:05.323+0000 7f80bad9d640 1 --2- >> v2:172.21.15.37:3300/0 conn(0x7f80bc160f40 0x7f80bc161880 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.37:3300/0 says I am v2:172.21.15.37:46326/0 (socket says 172.21.15.37:46326) 2024-04-25T10:29:05.326 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:05.323+0000 7f80ba59c640 1 -- 172.21.15.37:0/3528619312 learned_addr learned my addr 172.21.15.37:0/3528619312 (peer_addr_for_me v1:172.21.15.37:0/0) 2024-04-25T10:29:05.326 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:05.323+0000 7f80b9d9b640 1 --2- 172.21.15.37:0/3528619312 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f80bc0b7cd0 0x7f80bc160a00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:29:05.326 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:05.323+0000 7f80bad9d640 1 -- 172.21.15.37:0/3528619312 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f80bc0b7cd0 msgr2=0x7f80bc160a00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:29:05.326 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:05.323+0000 7f80bad9d640 1 --2- 172.21.15.37:0/3528619312 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f80bc0b7cd0 0x7f80bc160a00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:29:05.326 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:05.323+0000 7f80bad9d640 1 -- 172.21.15.37:0/3528619312 >> v1:172.21.15.37:6789/0 conn(0x7f80bc0d5430 legacy=0x7f80bc0b75c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:29:05.326 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:05.323+0000 7f80b959a640 1 -- 172.21.15.37:0/3528619312 <== mon.0 v1:172.21.15.37:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 3802218298 0 0) 0x7f80bc0d5840 con 0x7f80bc0d5430 2024-04-25T10:29:05.326 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:05.323+0000 7f80bad9d640 1 -- 172.21.15.37:0/3528619312 --> v2:172.21.15.37:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f80bc0d4950 con 0x7f80bc160f40 2024-04-25T10:29:05.327 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:05.323+0000 7f80b9d9b640 1 --2- 172.21.15.37:0/3528619312 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f80bc0b7cd0 0x7f80bc160a00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T10:29:05.328 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:05.324+0000 7f80bad9d640 1 --2- 172.21.15.37:0/3528619312 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f80bc160f40 0x7f80bc161880 secure :-1 s=READY pgs=49 cs=0 l=1 rev1=1 crypto rx=0x7f80b40097a0 tx=0x7f80b402a100 comp rx=0 tx=0).ready entity=mon.0 client_cookie=dab54d56feff50fa server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:29:05.328 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:05.324+0000 7f80b959a640 1 -- 172.21.15.37:0/3528619312 <== mon.0 v2:172.21.15.37:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f80b4037070 con 0x7f80bc160f40 2024-04-25T10:29:05.328 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:05.324+0000 7f80b959a640 1 -- 172.21.15.37:0/3528619312 <== mon.0 v2:172.21.15.37:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f80b402ae50 con 0x7f80bc160f40 2024-04-25T10:29:05.328 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:05.324+0000 7f80b959a640 1 -- 172.21.15.37:0/3528619312 <== mon.0 v2:172.21.15.37:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f80b4032660 con 0x7f80bc160f40 2024-04-25T10:29:05.328 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:05.325+0000 7f80c08aa640 1 -- 172.21.15.37:0/3528619312 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f80bc160f40 msgr2=0x7f80bc161880 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:29:05.328 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:05.325+0000 7f80c08aa640 1 --2- 172.21.15.37:0/3528619312 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f80bc160f40 0x7f80bc161880 secure :-1 s=READY pgs=49 cs=0 l=1 rev1=1 crypto rx=0x7f80b40097a0 tx=0x7f80b402a100 comp rx=0 tx=0).stop 2024-04-25T10:29:05.328 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:05.325+0000 7f80c08aa640 1 -- 172.21.15.37:0/3528619312 shutdown_connections 2024-04-25T10:29:05.328 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:05.325+0000 7f80c08aa640 1 --2- 172.21.15.37:0/3528619312 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f80bc160f40 0x7f80bc161880 unknown :-1 s=CLOSED pgs=49 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:29:05.328 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:05.325+0000 7f80c08aa640 1 --2- 172.21.15.37:0/3528619312 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f80bc0b7cd0 0x7f80bc160a00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:29:05.328 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:05.325+0000 7f80c08aa640 1 -- 172.21.15.37:0/3528619312 >> 172.21.15.37:0/3528619312 conn(0x7f80bc0d27a0 msgr2=0x7f80bc0d2bb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:29:05.328 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:05.325+0000 7f80c08aa640 1 -- 172.21.15.37:0/3528619312 shutdown_connections 2024-04-25T10:29:05.328 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:05.325+0000 7f80c08aa640 1 -- 172.21.15.37:0/3528619312 wait complete. 2024-04-25T10:29:05.329 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:05.327+0000 7f80c08aa640 1 Processor -- start 2024-04-25T10:29:05.329 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:05.327+0000 7f80c08aa640 1 -- start start 2024-04-25T10:29:05.332 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:05.328+0000 7f80c08aa640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f80bc0d5430 0x7f80bc1527f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:29:05.332 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:05.328+0000 7f80c08aa640 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f80bc160f40 0x7f80bc0cf8e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:29:05.332 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:05.328+0000 7f80c08aa640 1 --2- >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f80bc0cfe20 0x7f80bc0ba0f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:29:05.332 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:05.328+0000 7f80c08aa640 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 -- 0x7f80bc161350 con 0x7f80bc0d5430 2024-04-25T10:29:05.332 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:05.328+0000 7f80c08aa640 1 -- --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_getmap magic: 0 -- 0x7f80bc0d4f10 con 0x7f80bc160f40 2024-04-25T10:29:05.332 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:05.328+0000 7f80c08aa640 1 -- --> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] -- mon_getmap magic: 0 -- 0x7f80bc1614d0 con 0x7f80bc0cfe20 2024-04-25T10:29:05.332 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:05.328+0000 7f80ba59c640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f80bc0d5430 0x7f80bc1527f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:29:05.332 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:05.328+0000 7f80ba59c640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f80bc0d5430 0x7f80bc1527f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.37:3300/0 says I am v2:172.21.15.37:46342/0 (socket says 172.21.15.37:46342) 2024-04-25T10:29:05.332 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:05.328+0000 7f80ba59c640 1 -- 172.21.15.37:0/1842839946 learned_addr learned my addr 172.21.15.37:0/1842839946 (peer_addr_for_me v2:172.21.15.37:0/0) 2024-04-25T10:29:05.333 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:05.328+0000 7f80bad9d640 1 --2- 172.21.15.37:0/1842839946 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f80bc0cfe20 0x7f80bc0ba0f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:29:05.333 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:05.328+0000 7f80b9d9b640 1 --2- 172.21.15.37:0/1842839946 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f80bc160f40 0x7f80bc0cf8e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:29:05.333 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:05.329+0000 7f80ba59c640 1 -- 172.21.15.37:0/1842839946 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f80bc0cfe20 msgr2=0x7f80bc0ba0f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:29:05.333 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:05.329+0000 7f80ba59c640 1 --2- 172.21.15.37:0/1842839946 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f80bc0cfe20 0x7f80bc0ba0f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:29:05.333 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:05.329+0000 7f80ba59c640 1 -- 172.21.15.37:0/1842839946 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f80bc160f40 msgr2=0x7f80bc0cf8e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:29:05.333 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:05.329+0000 7f80ba59c640 1 --2- 172.21.15.37:0/1842839946 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f80bc160f40 0x7f80bc0cf8e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:29:05.333 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:05.329+0000 7f80ba59c640 1 -- 172.21.15.37:0/1842839946 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f80bc1f9350 con 0x7f80bc0d5430 2024-04-25T10:29:05.333 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:05.329+0000 7f80bad9d640 1 --2- 172.21.15.37:0/1842839946 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f80bc0cfe20 0x7f80bc0ba0f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T10:29:05.333 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:05.329+0000 7f80ba59c640 1 --2- 172.21.15.37:0/1842839946 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f80bc0d5430 0x7f80bc1527f0 secure :-1 s=READY pgs=50 cs=0 l=1 rev1=1 crypto rx=0x7f80ac00c7b0 tx=0x7f80ac00cc70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:29:05.333 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:05.329+0000 7f80b2ffd640 1 -- 172.21.15.37:0/1842839946 <== mon.0 v2:172.21.15.37:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f80ac018070 con 0x7f80bc0d5430 2024-04-25T10:29:05.333 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:05.329+0000 7f80b2ffd640 1 -- 172.21.15.37:0/1842839946 <== mon.0 v2:172.21.15.37:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f80ac00a640 con 0x7f80bc0d5430 2024-04-25T10:29:05.333 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:05.329+0000 7f80c08aa640 1 -- 172.21.15.37:0/1842839946 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f80bc1f9630 con 0x7f80bc0d5430 2024-04-25T10:29:05.333 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:05.329+0000 7f80b2ffd640 1 -- 172.21.15.37:0/1842839946 <== mon.0 v2:172.21.15.37:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f80ac00aa30 con 0x7f80bc0d5430 2024-04-25T10:29:05.333 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:05.329+0000 7f80c08aa640 1 -- 172.21.15.37:0/1842839946 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f80bc1f9bf0 con 0x7f80bc0d5430 2024-04-25T10:29:05.334 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:05.330+0000 7f80c08aa640 1 -- 172.21.15.37:0/1842839946 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f80bc06a140 con 0x7f80bc0d5430 2024-04-25T10:29:05.334 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:05.331+0000 7f80b2ffd640 1 -- 172.21.15.37:0/1842839946 <== mon.0 v2:172.21.15.37:3300/0 4 ==== mgrmap(e 5) ==== 96782+0+0 (secure 0 0 0) 0x7f80ac01b070 con 0x7f80bc0d5430 2024-04-25T10:29:05.334 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:05.331+0000 7f80b2ffd640 1 --2- 172.21.15.37:0/1842839946 >> [v2:172.21.15.203:6832/2418205237,v1:172.21.15.203:6833/2418205237] conn(0x7f809c0744e0 0x7f809c0769f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:29:05.334 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:05.331+0000 7f80b2ffd640 1 -- 172.21.15.37:0/1842839946 <== mon.0 v2:172.21.15.37:3300/0 5 ==== osd_map(19..19 src has 1..19) ==== 4857+0+0 (secure 0 0 0) 0x7f80ac0944e0 con 0x7f80bc0d5430 2024-04-25T10:29:05.335 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:05.332+0000 7f80b9d9b640 1 --2- 172.21.15.37:0/1842839946 >> [v2:172.21.15.203:6832/2418205237,v1:172.21.15.203:6833/2418205237] conn(0x7f809c0744e0 0x7f809c0769f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:29:05.335 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:05.332+0000 7f80b9d9b640 1 --2- 172.21.15.37:0/1842839946 >> [v2:172.21.15.203:6832/2418205237,v1:172.21.15.203:6833/2418205237] conn(0x7f809c0744e0 0x7f809c0769f0 secure :-1 s=READY pgs=52 cs=0 l=1 rev1=1 crypto rx=0x7f80bc0d6b10 tx=0x7f80a4008040 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:29:05.338 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:05.336+0000 7f80b2ffd640 1 -- 172.21.15.37:0/1842839946 <== mon.0 v2:172.21.15.37:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187246 (secure 0 0 0) 0x7f80ac05efc0 con 0x7f80bc0d5430 2024-04-25T10:29:05.523 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:05.520+0000 7f80c08aa640 1 -- 172.21.15.37:0/1842839946 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7f80bc0b78d0 con 0x7f80bc0d5430 2024-04-25T10:29:05.523 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:05.521+0000 7f80b2ffd640 1 -- 172.21.15.37:0/1842839946 <== mon.0 v2:172.21.15.37:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v19) ==== 74+0+12896 (secure 0 0 0) 0x7f80ac05e8c0 con 0x7f80bc0d5430 2024-04-25T10:29:05.524 INFO:teuthology.orchestra.run.smithi037.stdout: 2024-04-25T10:29:05.524 INFO:teuthology.orchestra.run.smithi037.stdout:{"epoch":19,"fsid":"f4acda28-be34-492a-b286-994afd36f61d","created":"2024-04-25T10:28:43.985718+0000","modified":"2024-04-25T10:28:58.970539+0000","last_up_change":"2024-04-25T10:28:54.900654+0000","last_in_change":"2024-04-25T10:28:46.216973+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":1,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-04-25T10:28:56.202153+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"19","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"a2f79028-b332-4166-9ebb-a7024a37ea22","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.37:6816","nonce":2746074720},{"type":"v1","addr":"172.21.15.37:6817","nonce":2746074720}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.37:6818","nonce":2746074720},{"type":"v1","addr":"172.21.15.37:6819","nonce":2746074720}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.37:6822","nonce":2746074720},{"type":"v1","addr":"172.21.15.37:6823","nonce":2746074720}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.37:6820","nonce":2746074720},{"type":"v1","addr":"172.21.15.37:6821","nonce":2746074720}]},"public_addr":"172.21.15.37:6817/2746074720","cluster_addr":"172.21.15.37:6819/2746074720","heartbeat_back_addr":"172.21.15.37:6823/2746074720","heartbeat_front_addr":"172.21.15.37:6821/2746074720","state":["exists","up"]},{"osd":1,"uuid":"973159de-c2ea-4083-aecc-69373d1c1132","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":17,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.37:6800","nonce":1690341662},{"type":"v1","addr":"172.21.15.37:6801","nonce":1690341662}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.37:6802","nonce":1690341662},{"type":"v1","addr":"172.21.15.37:6803","nonce":1690341662}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.37:6806","nonce":1690341662},{"type":"v1","addr":"172.21.15.37:6807","nonce":1690341662}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.37:6804","nonce":1690341662},{"type":"v1","addr":"172.21.15.37:6805","nonce":1690341662}]},"public_addr":"172.21.15.37:6801/1690341662","cluster_addr":"172.21.15.37:6803/1690341662","heartbeat_back_addr":"172.21.15.37:6807/1690341662","heartbeat_front_addr":"172.21.15.37:6805/1690341662","state":["exists","up"]},{"osd":2,"uuid":"4ae7a0e9-cb3f-484c-a75f-f3766b721c47","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.37:6808","nonce":2754669649},{"type":"v1","addr":"172.21.15.37:6809","nonce":2754669649}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.37:6810","nonce":2754669649},{"type":"v1","addr":"172.21.15.37:6811","nonce":2754669649}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.37:6814","nonce":2754669649},{"type":"v1","addr":"172.21.15.37:6815","nonce":2754669649}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.37:6812","nonce":2754669649},{"type":"v1","addr":"172.21.15.37:6813","nonce":2754669649}]},"public_addr":"172.21.15.37:6809/2754669649","cluster_addr":"172.21.15.37:6811/2754669649","heartbeat_back_addr":"172.21.15.37:6815/2754669649","heartbeat_front_addr":"172.21.15.37:6813/2754669649","state":["exists","up"]},{"osd":3,"uuid":"1ba4a746-1608-47a7-ace2-960a5cfae934","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.37:6824","nonce":1815290618},{"type":"v1","addr":"172.21.15.37:6825","nonce":1815290618}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.37:6826","nonce":1815290618},{"type":"v1","addr":"172.21.15.37:6827","nonce":1815290618}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.37:6830","nonce":1815290618},{"type":"v1","addr":"172.21.15.37:6831","nonce":1815290618}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.37:6828","nonce":1815290618},{"type":"v1","addr":"172.21.15.37:6829","nonce":1815290618}]},"public_addr":"172.21.15.37:6825/1815290618","cluster_addr":"172.21.15.37:6827/1815290618","heartbeat_back_addr":"172.21.15.37:6831/1815290618","heartbeat_front_addr":"172.21.15.37:6829/1815290618","state":["exists","up"]},{"osd":4,"uuid":"d4f59f5d-0413-4b79-87ee-bb212c1fcf62","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6800","nonce":4051532063},{"type":"v1","addr":"172.21.15.203:6801","nonce":4051532063}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6802","nonce":4051532063},{"type":"v1","addr":"172.21.15.203:6803","nonce":4051532063}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6806","nonce":4051532063},{"type":"v1","addr":"172.21.15.203:6807","nonce":4051532063}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6804","nonce":4051532063},{"type":"v1","addr":"172.21.15.203:6805","nonce":4051532063}]},"public_addr":"172.21.15.203:6801/4051532063","cluster_addr":"172.21.15.203:6803/4051532063","heartbeat_back_addr":"172.21.15.203:6807/4051532063","heartbeat_front_addr":"172.21.15.203:6805/4051532063","state":["exists","up"]},{"osd":5,"uuid":"69710a0e-41d8-4e7f-9a10-17ce540b67be","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6824","nonce":1203822890},{"type":"v1","addr":"172.21.15.203:6825","nonce":1203822890}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6826","nonce":1203822890},{"type":"v1","addr":"172.21.15.203:6827","nonce":1203822890}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6830","nonce":1203822890},{"type":"v1","addr":"172.21.15.203:6831","nonce":1203822890}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6828","nonce":1203822890},{"type":"v1","addr":"172.21.15.203:6829","nonce":1203822890}]},"public_addr":"172.21.15.203:6825/1203822890","cluster_addr":"172.21.15.203:6827/1203822890","heartbeat_back_addr":"172.21.15.203:6831/1203822890","heartbeat_front_addr":"172.21.15.203:6829/1203822890","state":["exists","up"]},{"osd":6,"uuid":"f29dd047-3b48-473d-a235-6642a6293f20","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6810","nonce":2309615014},{"type":"v1","addr":"172.21.15.203:6812","nonce":2309615014}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6814","nonce":2309615014},{"type":"v1","addr":"172.21.15.203:6816","nonce":2309615014}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6822","nonce":2309615014},{"type":"v1","addr":"172.21.15.203:6823","nonce":2309615014}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6818","nonce":2309615014},{"type":"v1","addr":"172.21.15.203:6820","nonce":2309615014}]},"public_addr":"172.21.15.203:6812/2309615014","cluster_addr":"172.21.15.203:6816/2309615014","heartbeat_back_addr":"172.21.15.203:6823/2309615014","heartbeat_front_addr":"172.21.15.203:6820/2309615014","state":["exists","up"]},{"osd":7,"uuid":"6e691644-378b-4fdc-8e86-fb5f56e31f29","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6808","nonce":3688099014},{"type":"v1","addr":"172.21.15.203:6809","nonce":3688099014}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6811","nonce":3688099014},{"type":"v1","addr":"172.21.15.203:6813","nonce":3688099014}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6819","nonce":3688099014},{"type":"v1","addr":"172.21.15.203:6821","nonce":3688099014}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6815","nonce":3688099014},{"type":"v1","addr":"172.21.15.203:6817","nonce":3688099014}]},"public_addr":"172.21.15.203:6809/3688099014","cluster_addr":"172.21.15.203:6813/3688099014","heartbeat_back_addr":"172.21.15.203:6821/3688099014","heartbeat_front_addr":"172.21.15.203:6817/3688099014","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T10:28:51.419141+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T10:28:51.366448+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T10:28:51.348276+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T10:28:51.380519+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T10:28:51.105812+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T10:28:51.384496+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T10:28:51.388755+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T10:28:51.366958+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-04-25T10:29:05.526 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:05.523+0000 7f80c08aa640 1 -- 172.21.15.37:0/1842839946 >> [v2:172.21.15.203:6832/2418205237,v1:172.21.15.203:6833/2418205237] conn(0x7f809c0744e0 msgr2=0x7f809c0769f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:29:05.526 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:05.523+0000 7f80c08aa640 1 --2- 172.21.15.37:0/1842839946 >> [v2:172.21.15.203:6832/2418205237,v1:172.21.15.203:6833/2418205237] conn(0x7f809c0744e0 0x7f809c0769f0 secure :-1 s=READY pgs=52 cs=0 l=1 rev1=1 crypto rx=0x7f80bc0d6b10 tx=0x7f80a4008040 comp rx=0 tx=0).stop 2024-04-25T10:29:05.526 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:05.523+0000 7f80c08aa640 1 -- 172.21.15.37:0/1842839946 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f80bc0d5430 msgr2=0x7f80bc1527f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:29:05.526 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:05.523+0000 7f80c08aa640 1 --2- 172.21.15.37:0/1842839946 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f80bc0d5430 0x7f80bc1527f0 secure :-1 s=READY pgs=50 cs=0 l=1 rev1=1 crypto rx=0x7f80ac00c7b0 tx=0x7f80ac00cc70 comp rx=0 tx=0).stop 2024-04-25T10:29:05.526 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:05.523+0000 7f80c08aa640 1 -- 172.21.15.37:0/1842839946 shutdown_connections 2024-04-25T10:29:05.526 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:05.523+0000 7f80c08aa640 1 --2- 172.21.15.37:0/1842839946 >> [v2:172.21.15.203:6832/2418205237,v1:172.21.15.203:6833/2418205237] conn(0x7f809c0744e0 0x7f809c0769f0 unknown :-1 s=CLOSED pgs=52 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:29:05.526 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:05.523+0000 7f80c08aa640 1 --2- 172.21.15.37:0/1842839946 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f80bc0cfe20 0x7f80bc0ba0f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:29:05.526 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:05.523+0000 7f80c08aa640 1 --2- 172.21.15.37:0/1842839946 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f80bc160f40 0x7f80bc0cf8e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:29:05.526 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:05.523+0000 7f80c08aa640 1 --2- 172.21.15.37:0/1842839946 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f80bc0d5430 0x7f80bc1527f0 unknown :-1 s=CLOSED pgs=50 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:29:05.527 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:05.523+0000 7f80c08aa640 1 -- 172.21.15.37:0/1842839946 >> 172.21.15.37:0/1842839946 conn(0x7f80bc0d27a0 msgr2=0x7f80bc15f5f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:29:05.527 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:05.523+0000 7f80c08aa640 1 -- 172.21.15.37:0/1842839946 shutdown_connections 2024-04-25T10:29:05.527 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:05.523+0000 7f80c08aa640 1 -- 172.21.15.37:0/1842839946 wait complete. 2024-04-25T10:29:05.537 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2024-04-25T10:28:56.202153+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '19', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 8, 'score_stable': 8, 'optimal_score': 0.25, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}] 2024-04-25T10:29:05.537 DEBUG:teuthology.orchestra.run.smithi037:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2024-04-25T10:29:05.652 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:05.649+0000 7f62b857d640 1 Processor -- start 2024-04-25T10:29:05.652 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:05.650+0000 7f62b857d640 1 -- start start 2024-04-25T10:29:05.654 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:05.651+0000 7f62b857d640 1 --2- >> v2:172.21.15.37:3300/0 conn(0x7f62b009ced0 0x7f62b00b71d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:29:05.654 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:05.651+0000 7f62b857d640 1 --2- >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f62b00b7710 0x7f62b00b7b20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:29:05.654 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:05.651+0000 7f62b857d640 1 -- --> v1:172.21.15.203:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f62b00b8dc0 con 0x7f62b008cc90 2024-04-25T10:29:05.654 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:05.651+0000 7f62b857d640 1 -- --> v2:172.21.15.37:3300/0 -- mon_getmap magic: 0 -- 0x7f62b00d1e10 con 0x7f62b009ced0 2024-04-25T10:29:05.654 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:05.651+0000 7f62b857d640 1 -- --> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] -- mon_getmap magic: 0 -- 0x7f62b009d2e0 con 0x7f62b00b7710 2024-04-25T10:29:05.654 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:05.651+0000 7f62b5b18640 1 --2- >> v2:172.21.15.37:3300/0 conn(0x7f62b009ced0 0x7f62b00b71d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:29:05.654 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:05.651+0000 7f62b5b18640 1 --2- >> v2:172.21.15.37:3300/0 conn(0x7f62b009ced0 0x7f62b00b71d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.37:3300/0 says I am v2:172.21.15.37:46358/0 (socket says 172.21.15.37:46358) 2024-04-25T10:29:05.654 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:05.651+0000 7f62b5b18640 1 -- 172.21.15.37:0/1382952253 learned_addr learned my addr 172.21.15.37:0/1382952253 (peer_addr_for_me v2:172.21.15.37:0/0) 2024-04-25T10:29:05.654 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:05.651+0000 7f62b6b1a640 1 --2- 172.21.15.37:0/1382952253 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f62b00b7710 0x7f62b00b7b20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:29:05.654 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:05.651+0000 7f62b5b18640 1 -- 172.21.15.37:0/1382952253 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f62b00b7710 msgr2=0x7f62b00b7b20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:29:05.654 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:05.651+0000 7f62b5b18640 1 --2- 172.21.15.37:0/1382952253 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f62b00b7710 0x7f62b00b7b20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:29:05.655 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:05.651+0000 7f62b5b18640 1 -- 172.21.15.37:0/1382952253 >> v1:172.21.15.203:6789/0 conn(0x7f62b008cc90 legacy=0x7f62b00d21c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:29:05.655 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:05.651+0000 7f62b5b18640 1 -- 172.21.15.37:0/1382952253 --> v2:172.21.15.37:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f62b00d1820 con 0x7f62b009ced0 2024-04-25T10:29:05.656 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:05.652+0000 7f62b5b18640 1 --2- 172.21.15.37:0/1382952253 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f62b009ced0 0x7f62b00b71d0 secure :-1 s=READY pgs=51 cs=0 l=1 rev1=1 crypto rx=0x7f62b0074230 tx=0x7f62a402a0b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=80a3d85687131c00 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:29:05.656 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:05.652+0000 7f62b5317640 1 -- 172.21.15.37:0/1382952253 <== mon.0 v2:172.21.15.37:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f62a4037070 con 0x7f62b009ced0 2024-04-25T10:29:05.656 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:05.652+0000 7f62b5317640 1 -- 172.21.15.37:0/1382952253 <== mon.0 v2:172.21.15.37:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f62a402adc0 con 0x7f62b009ced0 2024-04-25T10:29:05.656 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:05.652+0000 7f62b5317640 1 -- 172.21.15.37:0/1382952253 <== mon.0 v2:172.21.15.37:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f62a4032660 con 0x7f62b009ced0 2024-04-25T10:29:05.656 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:05.653+0000 7f62b857d640 1 -- 172.21.15.37:0/1382952253 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f62b009ced0 msgr2=0x7f62b00b71d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:29:05.656 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:05.653+0000 7f62b857d640 1 --2- 172.21.15.37:0/1382952253 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f62b009ced0 0x7f62b00b71d0 secure :-1 s=READY pgs=51 cs=0 l=1 rev1=1 crypto rx=0x7f62b0074230 tx=0x7f62a402a0b0 comp rx=0 tx=0).stop 2024-04-25T10:29:05.656 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:05.653+0000 7f62b857d640 1 -- 172.21.15.37:0/1382952253 shutdown_connections 2024-04-25T10:29:05.656 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:05.653+0000 7f62b857d640 1 --2- 172.21.15.37:0/1382952253 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f62b00b7710 0x7f62b00b7b20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:29:05.656 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:05.653+0000 7f62b857d640 1 --2- 172.21.15.37:0/1382952253 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f62b009ced0 0x7f62b00b71d0 unknown :-1 s=CLOSED pgs=51 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:29:05.656 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:05.653+0000 7f62b857d640 1 -- 172.21.15.37:0/1382952253 >> 172.21.15.37:0/1382952253 conn(0x7f62b00d03f0 msgr2=0x7f62b00d0800 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:29:05.656 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:05.653+0000 7f62b857d640 1 -- 172.21.15.37:0/1382952253 shutdown_connections 2024-04-25T10:29:05.657 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:05.653+0000 7f62b857d640 1 -- 172.21.15.37:0/1382952253 wait complete. 2024-04-25T10:29:05.657 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:05.655+0000 7f62b857d640 1 Processor -- start 2024-04-25T10:29:05.657 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:05.655+0000 7f62b857d640 1 -- start start 2024-04-25T10:29:05.660 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:05.656+0000 7f62b857d640 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f62b008cc90 0x7f62b01e8cc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:29:05.660 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:05.656+0000 7f62b857d640 1 --2- >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f62b009ced0 0x7f62b01e2a10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:29:05.660 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:05.656+0000 7f62b857d640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f62b00b7710 0x7f62b01e44a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:29:05.661 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:05.656+0000 7f62b857d640 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 -- 0x7f62b00b9210 con 0x7f62b00b7710 2024-04-25T10:29:05.661 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:05.656+0000 7f62b857d640 1 -- --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_getmap magic: 0 -- 0x7f62b00d1b30 con 0x7f62b008cc90 2024-04-25T10:29:05.661 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:05.656+0000 7f62b857d640 1 -- --> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] -- mon_getmap magic: 0 -- 0x7f62b00d28d0 con 0x7f62b009ced0 2024-04-25T10:29:05.661 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:05.656+0000 7f62b6b1a640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f62b00b7710 0x7f62b01e44a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:29:05.661 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:05.656+0000 7f62b6b1a640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f62b00b7710 0x7f62b01e44a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.37:3300/0 says I am v2:172.21.15.37:46366/0 (socket says 172.21.15.37:46366) 2024-04-25T10:29:05.661 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:05.656+0000 7f62b6b1a640 1 -- 172.21.15.37:0/852021789 learned_addr learned my addr 172.21.15.37:0/852021789 (peer_addr_for_me v2:172.21.15.37:0/0) 2024-04-25T10:29:05.661 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:05.657+0000 7f62b6319640 1 --2- 172.21.15.37:0/852021789 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f62b008cc90 0x7f62b01e8cc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:29:05.661 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:05.657+0000 7f62b5b18640 1 --2- 172.21.15.37:0/852021789 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f62b009ced0 0x7f62b01e2a10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:29:05.661 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:05.657+0000 7f62b6b1a640 1 -- 172.21.15.37:0/852021789 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f62b009ced0 msgr2=0x7f62b01e2a10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:29:05.661 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:05.657+0000 7f62b6b1a640 1 --2- 172.21.15.37:0/852021789 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f62b009ced0 0x7f62b01e2a10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:29:05.661 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:05.657+0000 7f62b6b1a640 1 -- 172.21.15.37:0/852021789 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f62b008cc90 msgr2=0x7f62b01e8cc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:29:05.661 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:05.657+0000 7f62b6b1a640 1 --2- 172.21.15.37:0/852021789 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f62b008cc90 0x7f62b01e8cc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:29:05.661 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:05.657+0000 7f62b6b1a640 1 -- 172.21.15.37:0/852021789 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f62b00cbad0 con 0x7f62b00b7710 2024-04-25T10:29:05.661 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:05.657+0000 7f62b6b1a640 1 --2- 172.21.15.37:0/852021789 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f62b00b7710 0x7f62b01e44a0 secure :-1 s=READY pgs=52 cs=0 l=1 rev1=1 crypto rx=0x7f62a0007d80 tx=0x7f62a000e4b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:29:05.662 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:05.657+0000 7f62aeffd640 1 -- 172.21.15.37:0/852021789 <== mon.0 v2:172.21.15.37:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f62a0014420 con 0x7f62b00b7710 2024-04-25T10:29:05.662 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:05.657+0000 7f62aeffd640 1 -- 172.21.15.37:0/852021789 <== mon.0 v2:172.21.15.37:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f62a0019070 con 0x7f62b00b7710 2024-04-25T10:29:05.662 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:05.657+0000 7f62b857d640 1 -- 172.21.15.37:0/852021789 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f62b00cb230 con 0x7f62b00b7710 2024-04-25T10:29:05.662 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:05.657+0000 7f62aeffd640 1 -- 172.21.15.37:0/852021789 <== mon.0 v2:172.21.15.37:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f62a0014800 con 0x7f62b00b7710 2024-04-25T10:29:05.662 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:05.657+0000 7f62b857d640 1 -- 172.21.15.37:0/852021789 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f62b00cb7f0 con 0x7f62b00b7710 2024-04-25T10:29:05.662 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:05.658+0000 7f62b857d640 1 -- 172.21.15.37:0/852021789 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f62b00d21c0 con 0x7f62b00b7710 2024-04-25T10:29:05.663 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:05.659+0000 7f62aeffd640 1 -- 172.21.15.37:0/852021789 <== mon.0 v2:172.21.15.37:3300/0 4 ==== mgrmap(e 5) ==== 96782+0+0 (secure 0 0 0) 0x7f62a00149a0 con 0x7f62b00b7710 2024-04-25T10:29:05.663 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:05.660+0000 7f62aeffd640 1 --2- 172.21.15.37:0/852021789 >> [v2:172.21.15.203:6832/2418205237,v1:172.21.15.203:6833/2418205237] conn(0x7f629c0744e0 0x7f629c0769f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:29:05.663 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:05.660+0000 7f62aeffd640 1 -- 172.21.15.37:0/852021789 <== mon.0 v2:172.21.15.37:3300/0 5 ==== osd_map(19..19 src has 1..19) ==== 4857+0+0 (secure 0 0 0) 0x7f62a0095570 con 0x7f62b00b7710 2024-04-25T10:29:05.663 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:05.660+0000 7f62b6319640 1 --2- 172.21.15.37:0/852021789 >> [v2:172.21.15.203:6832/2418205237,v1:172.21.15.203:6833/2418205237] conn(0x7f629c0744e0 0x7f629c0769f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:29:05.663 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:05.660+0000 7f62b6319640 1 --2- 172.21.15.37:0/852021789 >> [v2:172.21.15.203:6832/2418205237,v1:172.21.15.203:6833/2418205237] conn(0x7f629c0744e0 0x7f629c0769f0 secure :-1 s=READY pgs=53 cs=0 l=1 rev1=1 crypto rx=0x7f62a80098c0 tx=0x7f62a8009060 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:29:05.666 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:05.664+0000 7f62aeffd640 1 -- 172.21.15.37:0/852021789 <== mon.0 v2:172.21.15.37:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187246 (secure 0 0 0) 0x7f62b00d21c0 con 0x7f62b00b7710 2024-04-25T10:29:05.849 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:05.847+0000 7f62b857d640 1 -- 172.21.15.37:0/852021789 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) -- 0x7f62b006a140 con 0x7f62b00b7710 2024-04-25T10:29:05.849 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:05.847+0000 7f62aeffd640 1 -- 172.21.15.37:0/852021789 <== mon.0 v2:172.21.15.37:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v19) ==== 93+0+10 (secure 0 0 0) 0x7f62a005fa00 con 0x7f62b00b7710 2024-04-25T10:29:05.850 INFO:teuthology.orchestra.run.smithi037.stdout:pg_num: 1 2024-04-25T10:29:05.852 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:05.849+0000 7f62b857d640 1 -- 172.21.15.37:0/852021789 >> [v2:172.21.15.203:6832/2418205237,v1:172.21.15.203:6833/2418205237] conn(0x7f629c0744e0 msgr2=0x7f629c0769f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:29:05.852 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:05.849+0000 7f62b857d640 1 --2- 172.21.15.37:0/852021789 >> [v2:172.21.15.203:6832/2418205237,v1:172.21.15.203:6833/2418205237] conn(0x7f629c0744e0 0x7f629c0769f0 secure :-1 s=READY pgs=53 cs=0 l=1 rev1=1 crypto rx=0x7f62a80098c0 tx=0x7f62a8009060 comp rx=0 tx=0).stop 2024-04-25T10:29:05.852 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:05.849+0000 7f62b857d640 1 -- 172.21.15.37:0/852021789 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f62b00b7710 msgr2=0x7f62b01e44a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:29:05.852 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:05.849+0000 7f62b857d640 1 --2- 172.21.15.37:0/852021789 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f62b00b7710 0x7f62b01e44a0 secure :-1 s=READY pgs=52 cs=0 l=1 rev1=1 crypto rx=0x7f62a0007d80 tx=0x7f62a000e4b0 comp rx=0 tx=0).stop 2024-04-25T10:29:05.852 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:05.849+0000 7f62b857d640 1 -- 172.21.15.37:0/852021789 shutdown_connections 2024-04-25T10:29:05.853 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:05.849+0000 7f62b857d640 1 --2- 172.21.15.37:0/852021789 >> [v2:172.21.15.203:6832/2418205237,v1:172.21.15.203:6833/2418205237] conn(0x7f629c0744e0 0x7f629c0769f0 unknown :-1 s=CLOSED pgs=53 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:29:05.853 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:05.849+0000 7f62b857d640 1 --2- 172.21.15.37:0/852021789 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f62b00b7710 0x7f62b01e44a0 unknown :-1 s=CLOSED pgs=52 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:29:05.853 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:05.849+0000 7f62b857d640 1 --2- 172.21.15.37:0/852021789 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f62b009ced0 0x7f62b01e2a10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:29:05.853 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:05.849+0000 7f62b857d640 1 --2- 172.21.15.37:0/852021789 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f62b008cc90 0x7f62b01e8cc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:29:05.853 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:05.849+0000 7f62b857d640 1 -- 172.21.15.37:0/852021789 >> 172.21.15.37:0/852021789 conn(0x7f62b00d03f0 msgr2=0x7f62b00721b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:29:05.853 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:05.849+0000 7f62b857d640 1 -- 172.21.15.37:0/852021789 shutdown_connections 2024-04-25T10:29:05.853 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:05.849+0000 7f62b857d640 1 -- 172.21.15.37:0/852021789 wait complete. 2024-04-25T10:29:05.863 DEBUG:teuthology.orchestra.run.smithi037:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph --format=json-pretty osd lspools 2024-04-25T10:29:05.974 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:05.971+0000 7f51ec0d1640 1 Processor -- start 2024-04-25T10:29:05.974 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:05.972+0000 7f51ec0d1640 1 -- start start 2024-04-25T10:29:05.976 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:05.973+0000 7f51ec0d1640 1 --2- >> v2:172.21.15.37:3300/0 conn(0x7f51e409b700 0x7f51e409bb10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:29:05.976 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:05.973+0000 7f51ec0d1640 1 --2- >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f51e40b2dd0 0x7f51e40b31e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:29:05.976 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:05.973+0000 7f51ec0d1640 1 -- --> v1:172.21.15.203:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f51e409c050 con 0x7f51e415a940 2024-04-25T10:29:05.976 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:05.973+0000 7f51ec0d1640 1 -- --> v2:172.21.15.37:3300/0 -- mon_getmap magic: 0 -- 0x7f51e40b2940 con 0x7f51e409b700 2024-04-25T10:29:05.976 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:05.973+0000 7f51ec0d1640 1 -- --> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] -- mon_getmap magic: 0 -- 0x7f51e41612e0 con 0x7f51e40b2dd0 2024-04-25T10:29:05.976 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:05.973+0000 7f51e966c640 1 --2- >> v2:172.21.15.37:3300/0 conn(0x7f51e409b700 0x7f51e409bb10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:29:05.976 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:05.973+0000 7f51e966c640 1 --2- >> v2:172.21.15.37:3300/0 conn(0x7f51e409b700 0x7f51e409bb10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.37:3300/0 says I am v2:172.21.15.37:46372/0 (socket says 172.21.15.37:46372) 2024-04-25T10:29:05.976 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:05.973+0000 7f51e966c640 1 -- 172.21.15.37:0/2151598559 learned_addr learned my addr 172.21.15.37:0/2151598559 (peer_addr_for_me v2:172.21.15.37:0/0) 2024-04-25T10:29:05.976 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:05.973+0000 7f51ea66e640 1 --2- 172.21.15.37:0/2151598559 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f51e40b2dd0 0x7f51e40b31e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:29:05.976 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:05.974+0000 7f51e966c640 1 -- 172.21.15.37:0/2151598559 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f51e40b2dd0 msgr2=0x7f51e40b31e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:29:05.976 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:05.974+0000 7f51e966c640 1 --2- 172.21.15.37:0/2151598559 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f51e40b2dd0 0x7f51e40b31e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:29:05.976 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:05.974+0000 7f51e966c640 1 -- 172.21.15.37:0/2151598559 >> v1:172.21.15.203:6789/0 conn(0x7f51e415a940 legacy=0x7f51e41607b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:29:05.977 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:05.974+0000 7f51e966c640 1 -- 172.21.15.37:0/2151598559 --> v2:172.21.15.37:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f51e41615c0 con 0x7f51e409b700 2024-04-25T10:29:05.978 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:05.974+0000 7f51e966c640 1 --2- 172.21.15.37:0/2151598559 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f51e409b700 0x7f51e409bb10 secure :-1 s=READY pgs=53 cs=0 l=1 rev1=1 crypto rx=0x7f51e0009870 tx=0x7f51e002a270 comp rx=0 tx=0).ready entity=mon.0 client_cookie=964680618415665f server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:29:05.978 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:05.974+0000 7f51e8e6b640 1 -- 172.21.15.37:0/2151598559 <== mon.0 v2:172.21.15.37:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f51e0037070 con 0x7f51e409b700 2024-04-25T10:29:05.978 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:05.975+0000 7f51e8e6b640 1 -- 172.21.15.37:0/2151598559 <== mon.0 v2:172.21.15.37:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f51e0032440 con 0x7f51e409b700 2024-04-25T10:29:05.978 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:05.975+0000 7f51e8e6b640 1 -- 172.21.15.37:0/2151598559 <== mon.0 v2:172.21.15.37:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f51e0032810 con 0x7f51e409b700 2024-04-25T10:29:05.978 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:05.975+0000 7f51ec0d1640 1 -- 172.21.15.37:0/2151598559 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f51e409b700 msgr2=0x7f51e409bb10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:29:05.978 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:05.975+0000 7f51ec0d1640 1 --2- 172.21.15.37:0/2151598559 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f51e409b700 0x7f51e409bb10 secure :-1 s=READY pgs=53 cs=0 l=1 rev1=1 crypto rx=0x7f51e0009870 tx=0x7f51e002a270 comp rx=0 tx=0).stop 2024-04-25T10:29:05.978 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:05.975+0000 7f51ec0d1640 1 -- 172.21.15.37:0/2151598559 shutdown_connections 2024-04-25T10:29:05.978 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:05.975+0000 7f51ec0d1640 1 --2- 172.21.15.37:0/2151598559 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f51e40b2dd0 0x7f51e40b31e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:29:05.978 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:05.975+0000 7f51ec0d1640 1 --2- 172.21.15.37:0/2151598559 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f51e409b700 0x7f51e409bb10 unknown :-1 s=CLOSED pgs=53 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:29:05.978 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:05.975+0000 7f51ec0d1640 1 -- 172.21.15.37:0/2151598559 >> 172.21.15.37:0/2151598559 conn(0x7f51e40b7600 msgr2=0x7f51e40b7c30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:29:05.979 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:05.975+0000 7f51ec0d1640 1 -- 172.21.15.37:0/2151598559 shutdown_connections 2024-04-25T10:29:05.979 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:05.975+0000 7f51ec0d1640 1 -- 172.21.15.37:0/2151598559 wait complete. 2024-04-25T10:29:05.979 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:05.977+0000 7f51ec0d1640 1 Processor -- start 2024-04-25T10:29:05.979 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:05.977+0000 7f51ec0d1640 1 -- start start 2024-04-25T10:29:05.982 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:05.978+0000 7f51ec0d1640 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f51e409b700 0x7f51e41eb380 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:29:05.982 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:05.978+0000 7f51ec0d1640 1 --2- >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f51e40b2dd0 0x7f51e41e84c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:29:05.982 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:05.978+0000 7f51ec0d1640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f51e415a940 0x7f51e41e8a00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:29:05.983 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:05.979+0000 7f51ec0d1640 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 -- 0x7f51e406ede0 con 0x7f51e415a940 2024-04-25T10:29:05.983 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:05.979+0000 7f51ec0d1640 1 -- --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_getmap magic: 0 -- 0x7f51e406ec60 con 0x7f51e409b700 2024-04-25T10:29:05.983 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:05.979+0000 7f51ec0d1640 1 -- --> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] -- mon_getmap magic: 0 -- 0x7f51e4160ec0 con 0x7f51e40b2dd0 2024-04-25T10:29:05.983 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:05.979+0000 7f51ea66e640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f51e415a940 0x7f51e41e8a00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:29:05.983 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:05.979+0000 7f51ea66e640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f51e415a940 0x7f51e41e8a00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.37:3300/0 says I am v2:172.21.15.37:46376/0 (socket says 172.21.15.37:46376) 2024-04-25T10:29:05.983 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:05.979+0000 7f51ea66e640 1 -- 172.21.15.37:0/254105206 learned_addr learned my addr 172.21.15.37:0/254105206 (peer_addr_for_me v2:172.21.15.37:0/0) 2024-04-25T10:29:05.983 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:05.979+0000 7f51e966c640 1 --2- 172.21.15.37:0/254105206 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f51e40b2dd0 0x7f51e41e84c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:29:05.983 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:05.979+0000 7f51e9e6d640 1 --2- 172.21.15.37:0/254105206 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f51e409b700 0x7f51e41eb380 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:29:05.983 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:05.979+0000 7f51ea66e640 1 -- 172.21.15.37:0/254105206 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f51e40b2dd0 msgr2=0x7f51e41e84c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:29:05.983 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:05.979+0000 7f51ea66e640 1 --2- 172.21.15.37:0/254105206 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f51e40b2dd0 0x7f51e41e84c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:29:05.983 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:05.979+0000 7f51ea66e640 1 -- 172.21.15.37:0/254105206 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f51e409b700 msgr2=0x7f51e41eb380 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:29:05.983 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:05.979+0000 7f51ea66e640 1 --2- 172.21.15.37:0/254105206 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f51e409b700 0x7f51e41eb380 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:29:05.983 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:05.979+0000 7f51ea66e640 1 -- 172.21.15.37:0/254105206 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f51e41f6790 con 0x7f51e415a940 2024-04-25T10:29:05.984 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:05.979+0000 7f51ea66e640 1 --2- 172.21.15.37:0/254105206 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f51e415a940 0x7f51e41e8a00 secure :-1 s=READY pgs=54 cs=0 l=1 rev1=1 crypto rx=0x7f51dc00ef90 tx=0x7f51dc00c490 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:29:05.984 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:05.979+0000 7f51da7fc640 1 -- 172.21.15.37:0/254105206 <== mon.0 v2:172.21.15.37:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f51dc018020 con 0x7f51e415a940 2024-04-25T10:29:05.984 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:05.979+0000 7f51da7fc640 1 -- 172.21.15.37:0/254105206 <== mon.0 v2:172.21.15.37:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f51dc014480 con 0x7f51e415a940 2024-04-25T10:29:05.984 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:05.979+0000 7f51da7fc640 1 -- 172.21.15.37:0/254105206 <== mon.0 v2:172.21.15.37:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f51dc009e60 con 0x7f51e415a940 2024-04-25T10:29:05.984 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:05.979+0000 7f51ec0d1640 1 -- 172.21.15.37:0/254105206 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f51e4154fa0 con 0x7f51e415a940 2024-04-25T10:29:05.984 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:05.979+0000 7f51ec0d1640 1 -- 172.21.15.37:0/254105206 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f51e41f6a10 con 0x7f51e415a940 2024-04-25T10:29:05.984 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:05.980+0000 7f51ec0d1640 1 -- 172.21.15.37:0/254105206 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f51e41554c0 con 0x7f51e415a940 2024-04-25T10:29:05.985 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:05.981+0000 7f51da7fc640 1 -- 172.21.15.37:0/254105206 <== mon.0 v2:172.21.15.37:3300/0 4 ==== mgrmap(e 5) ==== 96782+0+0 (secure 0 0 0) 0x7f51dc005d10 con 0x7f51e415a940 2024-04-25T10:29:05.985 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:05.982+0000 7f51da7fc640 1 --2- 172.21.15.37:0/254105206 >> [v2:172.21.15.203:6832/2418205237,v1:172.21.15.203:6833/2418205237] conn(0x7f51c80744e0 0x7f51c80769f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:29:05.985 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:05.982+0000 7f51da7fc640 1 -- 172.21.15.37:0/254105206 <== mon.0 v2:172.21.15.37:3300/0 5 ==== osd_map(19..19 src has 1..19) ==== 4857+0+0 (secure 0 0 0) 0x7f51dc095980 con 0x7f51e415a940 2024-04-25T10:29:05.985 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:05.982+0000 7f51e9e6d640 1 --2- 172.21.15.37:0/254105206 >> [v2:172.21.15.203:6832/2418205237,v1:172.21.15.203:6833/2418205237] conn(0x7f51c80744e0 0x7f51c80769f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:29:05.985 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:05.982+0000 7f51e9e6d640 1 --2- 172.21.15.37:0/254105206 >> [v2:172.21.15.203:6832/2418205237,v1:172.21.15.203:6833/2418205237] conn(0x7f51c80744e0 0x7f51c80769f0 secure :-1 s=READY pgs=54 cs=0 l=1 rev1=1 crypto rx=0x7f51e40d38c0 tx=0x7f51d4009040 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:29:05.989 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:05.986+0000 7f51da7fc640 1 -- 172.21.15.37:0/254105206 <== mon.0 v2:172.21.15.37:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187246 (secure 0 0 0) 0x7f51dc022030 con 0x7f51e415a940 2024-04-25T10:29:06.175 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:06.173+0000 7f51ec0d1640 1 -- 172.21.15.37:0/254105206 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_command({"prefix": "osd lspools", "format": "json-pretty"} v 0) -- 0x7f51e406a140 con 0x7f51e415a940 2024-04-25T10:29:06.176 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:06.173+0000 7f51da7fc640 1 -- 172.21.15.37:0/254105206 <== mon.0 v2:172.21.15.37:3300/0 7 ==== mon_command_ack([{"prefix": "osd lspools", "format": "json-pretty"}]=0 v19) ==== 84+0+65 (secure 0 0 0) 0x7f51e41f6790 con 0x7f51e415a940 2024-04-25T10:29:06.176 INFO:teuthology.orchestra.run.smithi037.stdout: 2024-04-25T10:29:06.176 INFO:teuthology.orchestra.run.smithi037.stdout:[ 2024-04-25T10:29:06.176 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-04-25T10:29:06.176 INFO:teuthology.orchestra.run.smithi037.stdout: "poolnum": 1, 2024-04-25T10:29:06.176 INFO:teuthology.orchestra.run.smithi037.stdout: "poolname": ".mgr" 2024-04-25T10:29:06.176 INFO:teuthology.orchestra.run.smithi037.stdout: } 2024-04-25T10:29:06.176 INFO:teuthology.orchestra.run.smithi037.stdout:] 2024-04-25T10:29:06.177 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:06.175+0000 7f51ec0d1640 1 -- 172.21.15.37:0/254105206 >> [v2:172.21.15.203:6832/2418205237,v1:172.21.15.203:6833/2418205237] conn(0x7f51c80744e0 msgr2=0x7f51c80769f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:29:06.178 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:06.175+0000 7f51ec0d1640 1 --2- 172.21.15.37:0/254105206 >> [v2:172.21.15.203:6832/2418205237,v1:172.21.15.203:6833/2418205237] conn(0x7f51c80744e0 0x7f51c80769f0 secure :-1 s=READY pgs=54 cs=0 l=1 rev1=1 crypto rx=0x7f51e40d38c0 tx=0x7f51d4009040 comp rx=0 tx=0).stop 2024-04-25T10:29:06.178 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:06.175+0000 7f51ec0d1640 1 -- 172.21.15.37:0/254105206 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f51e415a940 msgr2=0x7f51e41e8a00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:29:06.178 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:06.175+0000 7f51ec0d1640 1 --2- 172.21.15.37:0/254105206 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f51e415a940 0x7f51e41e8a00 secure :-1 s=READY pgs=54 cs=0 l=1 rev1=1 crypto rx=0x7f51dc00ef90 tx=0x7f51dc00c490 comp rx=0 tx=0).stop 2024-04-25T10:29:06.178 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:06.175+0000 7f51ec0d1640 1 -- 172.21.15.37:0/254105206 shutdown_connections 2024-04-25T10:29:06.178 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:06.175+0000 7f51ec0d1640 1 --2- 172.21.15.37:0/254105206 >> [v2:172.21.15.203:6832/2418205237,v1:172.21.15.203:6833/2418205237] conn(0x7f51c80744e0 0x7f51c80769f0 unknown :-1 s=CLOSED pgs=54 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:29:06.178 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:06.175+0000 7f51ec0d1640 1 --2- 172.21.15.37:0/254105206 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f51e415a940 0x7f51e41e8a00 unknown :-1 s=CLOSED pgs=54 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:29:06.178 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:06.175+0000 7f51ec0d1640 1 --2- 172.21.15.37:0/254105206 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f51e40b2dd0 0x7f51e41e84c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:29:06.178 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:06.175+0000 7f51ec0d1640 1 --2- 172.21.15.37:0/254105206 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f51e409b700 0x7f51e41eb380 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:29:06.178 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:06.175+0000 7f51ec0d1640 1 -- 172.21.15.37:0/254105206 >> 172.21.15.37:0/254105206 conn(0x7f51e40b7600 msgr2=0x7f51e40afa40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:29:06.178 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:06.175+0000 7f51ec0d1640 1 -- 172.21.15.37:0/254105206 shutdown_connections 2024-04-25T10:29:06.178 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:06.175+0000 7f51ec0d1640 1 -- 172.21.15.37:0/254105206 wait complete. 2024-04-25T10:29:06.188 DEBUG:tasks.cephfs.filesystem:Creating filesystem 'cephfs' 2024-04-25T10:29:06.188 DEBUG:teuthology.orchestra.run.smithi037:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool create cephfs_metadata --pg_num_min 64 2024-04-25T10:29:06.302 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:06.300+0000 7f5d69d96640 1 Processor -- start 2024-04-25T10:29:06.303 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:06.300+0000 7f5d69d96640 1 -- start start 2024-04-25T10:29:06.304 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:06.302+0000 7f5d69d96640 1 --2- >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f5d640d0f70 0x7f5d640d15a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:29:06.304 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:06.302+0000 7f5d69d96640 1 --2- >> v2:172.21.15.203:3300/0 conn(0x7f5d640b6f30 0x7f5d640ce9f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:29:06.304 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:06.302+0000 7f5d69d96640 1 -- --> v1:172.21.15.203:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f5d640b90c0 con 0x7f5d6409cf50 2024-04-25T10:29:06.304 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:06.302+0000 7f5d69d96640 1 -- --> v2:172.21.15.203:3300/0 -- mon_getmap magic: 0 -- 0x7f5d640ca8f0 con 0x7f5d640b6f30 2024-04-25T10:29:06.305 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:06.302+0000 7f5d69d96640 1 -- --> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] -- mon_getmap magic: 0 -- 0x7f5d640b2ce0 con 0x7f5d640d0f70 2024-04-25T10:29:06.305 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:06.302+0000 7f5d63fff640 1 --2- >> v2:172.21.15.203:3300/0 conn(0x7f5d640b6f30 0x7f5d640ce9f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:29:06.305 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:06.302+0000 7f5d637fe640 1 --2- >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f5d640d0f70 0x7f5d640d15a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:29:06.305 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:06.302+0000 7f5d637fe640 1 --2- >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f5d640d0f70 0x7f5d640d15a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.203:3301/0 says I am v2:172.21.15.37:51402/0 (socket says 172.21.15.37:51402) 2024-04-25T10:29:06.305 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:06.302+0000 7f5d63fff640 1 --2- >> v2:172.21.15.203:3300/0 conn(0x7f5d640b6f30 0x7f5d640ce9f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.203:3300/0 says I am v2:172.21.15.37:45064/0 (socket says 172.21.15.37:45064) 2024-04-25T10:29:06.305 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:06.302+0000 7f5d637fe640 1 -- 172.21.15.37:0/127188276 learned_addr learned my addr 172.21.15.37:0/127188276 (peer_addr_for_me v2:172.21.15.37:0/0) 2024-04-25T10:29:06.305 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:06.302+0000 7f5d62ffd640 1 --1- >> v1:172.21.15.203:6789/0 conn(0x7f5d6409cf50 0x7f5d640b6820 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.203:6789/0 says I am v1:172.21.15.37:35994/0 (socket says 172.21.15.37:35994) 2024-04-25T10:29:06.305 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:06.302+0000 7f5d637fe640 1 -- 172.21.15.37:0/127188276 >> v2:172.21.15.203:3300/0 conn(0x7f5d640b6f30 msgr2=0x7f5d640ce9f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:29:06.305 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:06.302+0000 7f5d637fe640 1 --2- 172.21.15.37:0/127188276 >> v2:172.21.15.203:3300/0 conn(0x7f5d640b6f30 0x7f5d640ce9f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:29:06.305 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:06.302+0000 7f5d637fe640 1 -- 172.21.15.37:0/127188276 >> v1:172.21.15.203:6789/0 conn(0x7f5d6409cf50 legacy=0x7f5d640b6820 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:29:06.305 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:06.302+0000 7f5d637fe640 1 -- 172.21.15.37:0/127188276 --> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f5d640b93a0 con 0x7f5d640d0f70 2024-04-25T10:29:06.307 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:06.303+0000 7f5d637fe640 1 --2- 172.21.15.37:0/127188276 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f5d640d0f70 0x7f5d640d15a0 secure :-1 s=READY pgs=28 cs=0 l=1 rev1=1 crypto rx=0x7f5d54009870 tx=0x7f5d5402a270 comp rx=0 tx=0).ready entity=mon.2 client_cookie=1f323caf3993bae2 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:29:06.307 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:06.303+0000 7f5d627fc640 1 -- 172.21.15.37:0/127188276 <== mon.2 v2:172.21.15.203:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5d54037070 con 0x7f5d640d0f70 2024-04-25T10:29:06.307 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:06.303+0000 7f5d627fc640 1 -- 172.21.15.37:0/127188276 <== mon.2 v2:172.21.15.203:3301/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f5d54032440 con 0x7f5d640d0f70 2024-04-25T10:29:06.307 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:06.303+0000 7f5d627fc640 1 -- 172.21.15.37:0/127188276 <== mon.2 v2:172.21.15.203:3301/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5d54032810 con 0x7f5d640d0f70 2024-04-25T10:29:06.307 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:06.304+0000 7f5d69d96640 1 -- 172.21.15.37:0/127188276 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f5d640d0f70 msgr2=0x7f5d640d15a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:29:06.307 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:06.304+0000 7f5d69d96640 1 --2- 172.21.15.37:0/127188276 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f5d640d0f70 0x7f5d640d15a0 secure :-1 s=READY pgs=28 cs=0 l=1 rev1=1 crypto rx=0x7f5d54009870 tx=0x7f5d5402a270 comp rx=0 tx=0).stop 2024-04-25T10:29:06.307 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:06.304+0000 7f5d69d96640 1 -- 172.21.15.37:0/127188276 shutdown_connections 2024-04-25T10:29:06.307 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:06.304+0000 7f5d69d96640 1 --2- 172.21.15.37:0/127188276 >> v2:172.21.15.203:3300/0 conn(0x7f5d640b6f30 0x7f5d640ce9f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:29:06.307 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:06.304+0000 7f5d69d96640 1 --2- 172.21.15.37:0/127188276 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f5d640d0f70 0x7f5d640d15a0 unknown :-1 s=CLOSED pgs=28 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:29:06.307 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:06.304+0000 7f5d69d96640 1 -- 172.21.15.37:0/127188276 >> 172.21.15.37:0/127188276 conn(0x7f5d640d0450 msgr2=0x7f5d640d0860 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:29:06.307 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:06.304+0000 7f5d69d96640 1 -- 172.21.15.37:0/127188276 shutdown_connections 2024-04-25T10:29:06.307 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:06.304+0000 7f5d69d96640 1 -- 172.21.15.37:0/127188276 wait complete. 2024-04-25T10:29:06.308 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:06.306+0000 7f5d69d96640 1 Processor -- start 2024-04-25T10:29:06.308 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:06.306+0000 7f5d69d96640 1 -- start start 2024-04-25T10:29:06.310 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:06.307+0000 7f5d69d96640 1 --2- >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f5d6409cf50 0x7f5d641f5310 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:29:06.311 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:06.307+0000 7f5d69d96640 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f5d640b6f30 0x7f5d64161150 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:29:06.311 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:06.307+0000 7f5d69d96640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f5d640d0f70 0x7f5d6414dd10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:29:06.311 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:06.307+0000 7f5d69d96640 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 -- 0x7f5d640d00c0 con 0x7f5d640d0f70 2024-04-25T10:29:06.311 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:06.307+0000 7f5d69d96640 1 -- --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_getmap magic: 0 -- 0x7f5d6409cac0 con 0x7f5d640b6f30 2024-04-25T10:29:06.311 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:06.307+0000 7f5d69d96640 1 -- --> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] -- mon_getmap magic: 0 -- 0x7f5d640d0240 con 0x7f5d6409cf50 2024-04-25T10:29:06.311 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:06.307+0000 7f5d63fff640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f5d640d0f70 0x7f5d6414dd10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:29:06.311 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:06.307+0000 7f5d63fff640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f5d640d0f70 0x7f5d6414dd10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.37:3300/0 says I am v2:172.21.15.37:46388/0 (socket says 172.21.15.37:46388) 2024-04-25T10:29:06.311 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:06.307+0000 7f5d63fff640 1 -- 172.21.15.37:0/2302228401 learned_addr learned my addr 172.21.15.37:0/2302228401 (peer_addr_for_me v2:172.21.15.37:0/0) 2024-04-25T10:29:06.311 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:06.307+0000 7f5d637fe640 1 --2- 172.21.15.37:0/2302228401 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f5d6409cf50 0x7f5d641f5310 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:29:06.311 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:06.308+0000 7f5d62ffd640 1 --2- 172.21.15.37:0/2302228401 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f5d640b6f30 0x7f5d64161150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:29:06.311 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:06.308+0000 7f5d63fff640 1 -- 172.21.15.37:0/2302228401 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f5d6409cf50 msgr2=0x7f5d641f5310 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:29:06.311 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:06.308+0000 7f5d63fff640 1 --2- 172.21.15.37:0/2302228401 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f5d6409cf50 0x7f5d641f5310 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:29:06.311 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:06.308+0000 7f5d63fff640 1 -- 172.21.15.37:0/2302228401 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f5d640b6f30 msgr2=0x7f5d64161150 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:29:06.312 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:06.308+0000 7f5d63fff640 1 --2- 172.21.15.37:0/2302228401 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f5d640b6f30 0x7f5d64161150 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:29:06.312 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:06.308+0000 7f5d63fff640 1 -- 172.21.15.37:0/2302228401 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f5d641f5850 con 0x7f5d640d0f70 2024-04-25T10:29:06.312 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:06.308+0000 7f5d62ffd640 1 --2- 172.21.15.37:0/2302228401 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f5d640b6f30 0x7f5d64161150 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T10:29:06.312 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:06.308+0000 7f5d63fff640 1 --2- 172.21.15.37:0/2302228401 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f5d640d0f70 0x7f5d6414dd10 secure :-1 s=READY pgs=55 cs=0 l=1 rev1=1 crypto rx=0x7f5d5c009cb0 tx=0x7f5d5c00c490 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:29:06.312 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:06.308+0000 7f5d4bfff640 1 -- 172.21.15.37:0/2302228401 <== mon.0 v2:172.21.15.37:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5d5c018020 con 0x7f5d640d0f70 2024-04-25T10:29:06.312 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:06.308+0000 7f5d4bfff640 1 -- 172.21.15.37:0/2302228401 <== mon.0 v2:172.21.15.37:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f5d5c014480 con 0x7f5d640d0f70 2024-04-25T10:29:06.312 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:06.308+0000 7f5d4bfff640 1 -- 172.21.15.37:0/2302228401 <== mon.0 v2:172.21.15.37:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5d5c014870 con 0x7f5d640d0f70 2024-04-25T10:29:06.312 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:06.308+0000 7f5d69d96640 1 -- 172.21.15.37:0/2302228401 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f5d6415e660 con 0x7f5d640d0f70 2024-04-25T10:29:06.312 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:06.308+0000 7f5d69d96640 1 -- 172.21.15.37:0/2302228401 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f5d64160950 con 0x7f5d640d0f70 2024-04-25T10:29:06.313 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:06.309+0000 7f5d69d96640 1 -- 172.21.15.37:0/2302228401 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f5d6406a140 con 0x7f5d640d0f70 2024-04-25T10:29:06.313 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:06.310+0000 7f5d4bfff640 1 -- 172.21.15.37:0/2302228401 <== mon.0 v2:172.21.15.37:3300/0 4 ==== mgrmap(e 5) ==== 96782+0+0 (secure 0 0 0) 0x7f5d5c005d10 con 0x7f5d640d0f70 2024-04-25T10:29:06.313 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:06.310+0000 7f5d4bfff640 1 --2- 172.21.15.37:0/2302228401 >> [v2:172.21.15.203:6832/2418205237,v1:172.21.15.203:6833/2418205237] conn(0x7f5d500744b0 0x7f5d500769c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:29:06.313 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:06.310+0000 7f5d4bfff640 1 -- 172.21.15.37:0/2302228401 <== mon.0 v2:172.21.15.37:3300/0 5 ==== osd_map(19..19 src has 1..19) ==== 4857+0+0 (secure 0 0 0) 0x7f5d5c095b40 con 0x7f5d640d0f70 2024-04-25T10:29:06.314 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:06.311+0000 7f5d637fe640 1 --2- 172.21.15.37:0/2302228401 >> [v2:172.21.15.203:6832/2418205237,v1:172.21.15.203:6833/2418205237] conn(0x7f5d500744b0 0x7f5d500769c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:29:06.314 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:06.311+0000 7f5d637fe640 1 --2- 172.21.15.37:0/2302228401 >> [v2:172.21.15.203:6832/2418205237,v1:172.21.15.203:6833/2418205237] conn(0x7f5d500744b0 0x7f5d500769c0 secure :-1 s=READY pgs=55 cs=0 l=1 rev1=1 crypto rx=0x7f5d54034000 tx=0x7f5d5402e000 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:29:06.317 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:06.315+0000 7f5d4bfff640 1 -- 172.21.15.37:0/2302228401 <== mon.0 v2:172.21.15.37:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187246 (secure 0 0 0) 0x7f5d5c022030 con 0x7f5d640d0f70 2024-04-25T10:29:06.505 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:06.502+0000 7f5d69d96640 1 -- 172.21.15.37:0/2302228401 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_command({"prefix": "osd pool create", "pool": "cephfs_metadata", "pg_num_min": 64} v 0) -- 0x7f5d6415e3f0 con 0x7f5d640d0f70 2024-04-25T10:29:06.996 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:06.993+0000 7f5d4bfff640 1 -- 172.21.15.37:0/2302228401 <== mon.0 v2:172.21.15.37:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool create", "pool": "cephfs_metadata", "pg_num_min": 64}]=0 pool 'cephfs_metadata' created v20) ==== 138+0+0 (secure 0 0 0) 0x7f5d5c010040 con 0x7f5d640d0f70 2024-04-25T10:29:06.996 INFO:teuthology.orchestra.run.smithi037.stderr:pool 'cephfs_metadata' created 2024-04-25T10:29:06.999 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:06.997+0000 7f5d69d96640 1 -- 172.21.15.37:0/2302228401 >> [v2:172.21.15.203:6832/2418205237,v1:172.21.15.203:6833/2418205237] conn(0x7f5d500744b0 msgr2=0x7f5d500769c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:29:06.999 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:06.997+0000 7f5d69d96640 1 --2- 172.21.15.37:0/2302228401 >> [v2:172.21.15.203:6832/2418205237,v1:172.21.15.203:6833/2418205237] conn(0x7f5d500744b0 0x7f5d500769c0 secure :-1 s=READY pgs=55 cs=0 l=1 rev1=1 crypto rx=0x7f5d54034000 tx=0x7f5d5402e000 comp rx=0 tx=0).stop 2024-04-25T10:29:06.999 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:06.997+0000 7f5d69d96640 1 -- 172.21.15.37:0/2302228401 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f5d640d0f70 msgr2=0x7f5d6414dd10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:29:06.999 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:06.997+0000 7f5d69d96640 1 --2- 172.21.15.37:0/2302228401 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f5d640d0f70 0x7f5d6414dd10 secure :-1 s=READY pgs=55 cs=0 l=1 rev1=1 crypto rx=0x7f5d5c009cb0 tx=0x7f5d5c00c490 comp rx=0 tx=0).stop 2024-04-25T10:29:06.999 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:06.997+0000 7f5d69d96640 1 -- 172.21.15.37:0/2302228401 shutdown_connections 2024-04-25T10:29:06.999 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:06.997+0000 7f5d69d96640 1 --2- 172.21.15.37:0/2302228401 >> [v2:172.21.15.203:6832/2418205237,v1:172.21.15.203:6833/2418205237] conn(0x7f5d500744b0 0x7f5d500769c0 unknown :-1 s=CLOSED pgs=55 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:29:07.000 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:06.997+0000 7f5d69d96640 1 --2- 172.21.15.37:0/2302228401 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f5d640d0f70 0x7f5d6414dd10 unknown :-1 s=CLOSED pgs=55 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:29:07.000 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:06.997+0000 7f5d69d96640 1 --2- 172.21.15.37:0/2302228401 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f5d640b6f30 0x7f5d64161150 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:29:07.000 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:06.997+0000 7f5d69d96640 1 --2- 172.21.15.37:0/2302228401 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f5d6409cf50 0x7f5d641f5310 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:29:07.000 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:06.997+0000 7f5d69d96640 1 -- 172.21.15.37:0/2302228401 >> 172.21.15.37:0/2302228401 conn(0x7f5d640d0450 msgr2=0x7f5d64071320 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:29:07.000 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:06.997+0000 7f5d69d96640 1 -- 172.21.15.37:0/2302228401 shutdown_connections 2024-04-25T10:29:07.000 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:06.997+0000 7f5d69d96640 1 -- 172.21.15.37:0/2302228401 wait complete. 2024-04-25T10:29:07.018 DEBUG:teuthology.orchestra.run.smithi037:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool create cephfs_data 64 --pg_num_min 64 --target_size_ratio 0.9 2024-04-25T10:29:07.146 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:07.144+0000 7f3462e0c640 1 Processor -- start 2024-04-25T10:29:07.146 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:07.144+0000 7f3462e0c640 1 -- start start 2024-04-25T10:29:07.147 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:07.145+0000 7f3462e0c640 1 --2- >> v2:172.21.15.37:3300/0 conn(0x7f345c1607e0 0x7f345c0d4560 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:29:07.147 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:07.145+0000 7f3462e0c640 1 -- --> v1:172.21.15.37:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f345c161190 con 0x7f345c0b4bb0 2024-04-25T10:29:07.147 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:07.145+0000 7f3462e0c640 1 -- --> v1:172.21.15.203:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f345c161470 con 0x7f345c0b7600 2024-04-25T10:29:07.147 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:07.145+0000 7f3462e0c640 1 -- --> v2:172.21.15.37:3300/0 -- mon_getmap magic: 0 -- 0x7f345c163eb0 con 0x7f345c1607e0 2024-04-25T10:29:07.148 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:07.145+0000 7f3460ba8640 1 --1- >> v1:172.21.15.37:6789/0 conn(0x7f345c0b4bb0 0x7f345c1646b0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.37:6789/0 says I am v1:172.21.15.37:56732/0 (socket says 172.21.15.37:56732) 2024-04-25T10:29:07.148 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:07.145+0000 7f3460ba8640 1 -- 172.21.15.37:0/4253073674 learned_addr learned my addr 172.21.15.37:0/4253073674 (peer_addr_for_me v1:172.21.15.37:0/0) 2024-04-25T10:29:07.148 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:07.145+0000 7f34613a9640 1 --2- 172.21.15.37:0/4253073674 >> v2:172.21.15.37:3300/0 conn(0x7f345c1607e0 0x7f345c0d4560 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:29:07.149 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:07.146+0000 7f34613a9640 1 -- 172.21.15.37:0/4253073674 >> v1:172.21.15.203:6789/0 conn(0x7f345c0b7600 legacy=0x7f345c0b7a10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:29:07.149 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:07.146+0000 7f34613a9640 1 -- 172.21.15.37:0/4253073674 >> v1:172.21.15.37:6789/0 conn(0x7f345c0b4bb0 legacy=0x7f345c1646b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:29:07.149 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:07.146+0000 7f34613a9640 1 -- 172.21.15.37:0/4253073674 --> v2:172.21.15.37:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f345c164190 con 0x7f345c1607e0 2024-04-25T10:29:07.149 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:07.146+0000 7f345b7fe640 1 -- 172.21.15.37:0/4253073674 <== mon.0 v1:172.21.15.37:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 1801494737 0 0) 0x7f345c161190 con 0x7f345c0b4bb0 2024-04-25T10:29:07.149 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:07.146+0000 7f34613a9640 1 --2- 172.21.15.37:0/4253073674 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f345c1607e0 0x7f345c0d4560 secure :-1 s=READY pgs=57 cs=0 l=1 rev1=1 crypto rx=0x7f344c0097a0 tx=0x7f344c02a1e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=5219e86e7d6f1641 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:29:07.149 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:07.146+0000 7f345b7fe640 1 -- 172.21.15.37:0/4253073674 <== mon.0 v2:172.21.15.37:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f344c037070 con 0x7f345c1607e0 2024-04-25T10:29:07.149 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:07.146+0000 7f345b7fe640 1 -- 172.21.15.37:0/4253073674 <== mon.0 v2:172.21.15.37:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f344c032440 con 0x7f345c1607e0 2024-04-25T10:29:07.149 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:07.146+0000 7f345b7fe640 1 -- 172.21.15.37:0/4253073674 <== mon.0 v2:172.21.15.37:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f344c032810 con 0x7f345c1607e0 2024-04-25T10:29:07.150 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:07.147+0000 7f3462e0c640 1 -- 172.21.15.37:0/4253073674 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f345c1607e0 msgr2=0x7f345c0d4560 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:29:07.150 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:07.147+0000 7f3462e0c640 1 --2- 172.21.15.37:0/4253073674 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f345c1607e0 0x7f345c0d4560 secure :-1 s=READY pgs=57 cs=0 l=1 rev1=1 crypto rx=0x7f344c0097a0 tx=0x7f344c02a1e0 comp rx=0 tx=0).stop 2024-04-25T10:29:07.150 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:07.147+0000 7f3462e0c640 1 -- 172.21.15.37:0/4253073674 shutdown_connections 2024-04-25T10:29:07.150 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:07.147+0000 7f3462e0c640 1 --2- 172.21.15.37:0/4253073674 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f345c1607e0 0x7f345c0d4560 unknown :-1 s=CLOSED pgs=57 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:29:07.150 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:07.147+0000 7f3462e0c640 1 -- 172.21.15.37:0/4253073674 >> 172.21.15.37:0/4253073674 conn(0x7f345c0d27e0 msgr2=0x7f345c0d2bf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:29:07.150 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:07.147+0000 7f3462e0c640 1 -- 172.21.15.37:0/4253073674 shutdown_connections 2024-04-25T10:29:07.150 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:07.147+0000 7f3462e0c640 1 -- 172.21.15.37:0/4253073674 wait complete. 2024-04-25T10:29:07.151 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:07.149+0000 7f3462e0c640 1 Processor -- start 2024-04-25T10:29:07.151 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:07.149+0000 7f3462e0c640 1 -- start start 2024-04-25T10:29:07.154 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:07.150+0000 7f3462e0c640 1 --2- >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f345c0b4bb0 0x7f345c14f740 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:29:07.154 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:07.150+0000 7f3462e0c640 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f345c0b7600 0x7f345c0b3ed0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:29:07.154 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:07.150+0000 7f3462e0c640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f345c1607e0 0x7f345c0d4800 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:29:07.154 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:07.150+0000 7f3462e0c640 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 -- 0x7f345c0d0b60 con 0x7f345c1607e0 2024-04-25T10:29:07.154 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:07.150+0000 7f3462e0c640 1 -- --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_getmap magic: 0 -- 0x7f345c0d09e0 con 0x7f345c0b7600 2024-04-25T10:29:07.155 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:07.150+0000 7f3462e0c640 1 -- --> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] -- mon_getmap magic: 0 -- 0x7f345c160bf0 con 0x7f345c0b4bb0 2024-04-25T10:29:07.155 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:07.151+0000 7f34613a9640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f345c1607e0 0x7f345c0d4800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:29:07.155 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:07.151+0000 7f34613a9640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f345c1607e0 0x7f345c0d4800 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.37:3300/0 says I am v2:172.21.15.37:46408/0 (socket says 172.21.15.37:46408) 2024-04-25T10:29:07.155 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:07.151+0000 7f34613a9640 1 -- 172.21.15.37:0/3114549584 learned_addr learned my addr 172.21.15.37:0/3114549584 (peer_addr_for_me v2:172.21.15.37:0/0) 2024-04-25T10:29:07.155 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:07.151+0000 7f3460ba8640 1 --2- 172.21.15.37:0/3114549584 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f345c0b4bb0 0x7f345c14f740 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:29:07.155 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:07.151+0000 7f345bfff640 1 --2- 172.21.15.37:0/3114549584 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f345c0b7600 0x7f345c0b3ed0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:29:07.155 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:07.151+0000 7f34613a9640 1 -- 172.21.15.37:0/3114549584 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f345c0b4bb0 msgr2=0x7f345c14f740 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:29:07.155 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:07.151+0000 7f34613a9640 1 --2- 172.21.15.37:0/3114549584 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f345c0b4bb0 0x7f345c14f740 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:29:07.155 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:07.151+0000 7f34613a9640 1 -- 172.21.15.37:0/3114549584 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f345c0b7600 msgr2=0x7f345c0b3ed0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:29:07.155 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:07.151+0000 7f34613a9640 1 --2- 172.21.15.37:0/3114549584 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f345c0b7600 0x7f345c0b3ed0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:29:07.155 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:07.151+0000 7f34613a9640 1 -- 172.21.15.37:0/3114549584 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f345c150040 con 0x7f345c1607e0 2024-04-25T10:29:07.155 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:07.151+0000 7f34613a9640 1 --2- 172.21.15.37:0/3114549584 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f345c1607e0 0x7f345c0d4800 secure :-1 s=READY pgs=58 cs=0 l=1 rev1=1 crypto rx=0x7f344c0098c0 tx=0x7f344c0046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:29:07.155 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:07.151+0000 7f34597fa640 1 -- 172.21.15.37:0/3114549584 <== mon.0 v2:172.21.15.37:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f344c041070 con 0x7f345c1607e0 2024-04-25T10:29:07.156 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:07.151+0000 7f34597fa640 1 -- 172.21.15.37:0/3114549584 <== mon.0 v2:172.21.15.37:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f344c03c870 con 0x7f345c1607e0 2024-04-25T10:29:07.156 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:07.151+0000 7f3462e0c640 1 -- 172.21.15.37:0/3114549584 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f345c150320 con 0x7f345c1607e0 2024-04-25T10:29:07.156 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:07.151+0000 7f34597fa640 1 -- 172.21.15.37:0/3114549584 <== mon.0 v2:172.21.15.37:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f344c037040 con 0x7f345c1607e0 2024-04-25T10:29:07.156 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:07.151+0000 7f3462e0c640 1 -- 172.21.15.37:0/3114549584 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f345c0cbdd0 con 0x7f345c1607e0 2024-04-25T10:29:07.157 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:07.152+0000 7f3462e0c640 1 -- 172.21.15.37:0/3114549584 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f345c14fdb0 con 0x7f345c1607e0 2024-04-25T10:29:07.157 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:07.153+0000 7f34597fa640 1 -- 172.21.15.37:0/3114549584 <== mon.0 v2:172.21.15.37:3300/0 4 ==== mgrmap(e 5) ==== 96782+0+0 (secure 0 0 0) 0x7f344c044070 con 0x7f345c1607e0 2024-04-25T10:29:07.157 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:07.153+0000 7f34597fa640 1 --2- 172.21.15.37:0/3114549584 >> [v2:172.21.15.203:6832/2418205237,v1:172.21.15.203:6833/2418205237] conn(0x7f343c0744e0 0x7f343c0769f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:29:07.157 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:07.154+0000 7f34597fa640 1 -- 172.21.15.37:0/3114549584 <== mon.0 v2:172.21.15.37:3300/0 5 ==== osd_map(20..20 src has 1..20) ==== 5227+0+0 (secure 0 0 0) 0x7f344c0b5ae0 con 0x7f345c1607e0 2024-04-25T10:29:07.157 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:07.154+0000 7f3460ba8640 1 --2- 172.21.15.37:0/3114549584 >> [v2:172.21.15.203:6832/2418205237,v1:172.21.15.203:6833/2418205237] conn(0x7f343c0744e0 0x7f343c0769f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:29:07.157 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:07.154+0000 7f3460ba8640 1 --2- 172.21.15.37:0/3114549584 >> [v2:172.21.15.203:6832/2418205237,v1:172.21.15.203:6833/2418205237] conn(0x7f343c0744e0 0x7f343c0769f0 secure :-1 s=READY pgs=56 cs=0 l=1 rev1=1 crypto rx=0x7f34540012e0 tx=0x7f3454001630 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:29:07.160 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:07.158+0000 7f34597fa640 1 -- 172.21.15.37:0/3114549584 <== mon.0 v2:172.21.15.37:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187246 (secure 0 0 0) 0x7f344c07fe90 con 0x7f345c1607e0 2024-04-25T10:29:07.345 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:07.343+0000 7f3462e0c640 1 -- 172.21.15.37:0/3114549584 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_command({"prefix": "osd pool create", "pool": "cephfs_data", "pg_num": 64, "pg_num_min": 64, "target_size_ratio": 0.9} v 0) -- 0x7f345c06a140 con 0x7f345c1607e0 2024-04-25T10:29:07.527 DEBUG:teuthology.orchestra.run.smithi037:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-25T10:29:07.529 DEBUG:teuthology.orchestra.run.smithi203:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-25T10:29:08.002 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:08.000+0000 7f34597fa640 1 -- 172.21.15.37:0/3114549584 <== mon.0 v2:172.21.15.37:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool create", "pool": "cephfs_data", "pg_num": 64, "pg_num_min": 64, "target_size_ratio": 0.9}]=0 pool 'cephfs_data' created v21) ==== 170+0+0 (secure 0 0 0) 0x7f344c084190 con 0x7f345c1607e0 2024-04-25T10:29:08.002 INFO:teuthology.orchestra.run.smithi037.stderr:pool 'cephfs_data' created 2024-04-25T10:29:08.006 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:08.004+0000 7f3462e0c640 1 -- 172.21.15.37:0/3114549584 >> [v2:172.21.15.203:6832/2418205237,v1:172.21.15.203:6833/2418205237] conn(0x7f343c0744e0 msgr2=0x7f343c0769f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:29:08.006 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:08.004+0000 7f3462e0c640 1 --2- 172.21.15.37:0/3114549584 >> [v2:172.21.15.203:6832/2418205237,v1:172.21.15.203:6833/2418205237] conn(0x7f343c0744e0 0x7f343c0769f0 secure :-1 s=READY pgs=56 cs=0 l=1 rev1=1 crypto rx=0x7f34540012e0 tx=0x7f3454001630 comp rx=0 tx=0).stop 2024-04-25T10:29:08.006 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:08.004+0000 7f3462e0c640 1 -- 172.21.15.37:0/3114549584 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f345c1607e0 msgr2=0x7f345c0d4800 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:29:08.007 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:08.004+0000 7f3462e0c640 1 --2- 172.21.15.37:0/3114549584 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f345c1607e0 0x7f345c0d4800 secure :-1 s=READY pgs=58 cs=0 l=1 rev1=1 crypto rx=0x7f344c0098c0 tx=0x7f344c0046c0 comp rx=0 tx=0).stop 2024-04-25T10:29:08.011 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:08.008+0000 7f3462e0c640 1 -- 172.21.15.37:0/3114549584 shutdown_connections 2024-04-25T10:29:08.011 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:08.008+0000 7f3462e0c640 1 --2- 172.21.15.37:0/3114549584 >> [v2:172.21.15.203:6832/2418205237,v1:172.21.15.203:6833/2418205237] conn(0x7f343c0744e0 0x7f343c0769f0 unknown :-1 s=CLOSED pgs=56 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:29:08.012 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:08.008+0000 7f3462e0c640 1 --2- 172.21.15.37:0/3114549584 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f345c1607e0 0x7f345c0d4800 unknown :-1 s=CLOSED pgs=58 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:29:08.012 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:08.008+0000 7f3462e0c640 1 --2- 172.21.15.37:0/3114549584 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f345c0b7600 0x7f345c0b3ed0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:29:08.012 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:08.008+0000 7f3462e0c640 1 --2- 172.21.15.37:0/3114549584 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f345c0b4bb0 0x7f345c14f740 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:29:08.012 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:08.008+0000 7f3462e0c640 1 -- 172.21.15.37:0/3114549584 >> 172.21.15.37:0/3114549584 conn(0x7f345c0d27e0 msgr2=0x7f345c0b54c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:29:08.012 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:08.009+0000 7f3462e0c640 1 -- 172.21.15.37:0/3114549584 shutdown_connections 2024-04-25T10:29:08.013 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:08.011+0000 7f3462e0c640 1 -- 172.21.15.37:0/3114549584 wait complete. 2024-04-25T10:29:08.038 DEBUG:teuthology.orchestra.run.smithi037:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs new cephfs cephfs_metadata cephfs_data 2024-04-25T10:29:08.157 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:08.155+0000 7f410afa8640 1 Processor -- start 2024-04-25T10:29:08.158 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:08.155+0000 7f410afa8640 1 -- start start 2024-04-25T10:29:08.162 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:08.157+0000 7f410afa8640 1 --2- >> v2:172.21.15.203:3300/0 conn(0x7f41040d5160 0x7f41040b4e80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:29:08.162 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:08.157+0000 7f410afa8640 1 --2- >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f41040d3230 0x7f41040b9060 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:29:08.162 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:08.157+0000 7f410afa8640 1 -- --> v1:172.21.15.37:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f41040d1370 con 0x7f41040d0990 2024-04-25T10:29:08.162 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:08.157+0000 7f410afa8640 1 -- --> v2:172.21.15.203:3300/0 -- mon_getmap magic: 0 -- 0x7f41040d1650 con 0x7f41040d5160 2024-04-25T10:29:08.162 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:08.157+0000 7f410afa8640 1 -- --> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] -- mon_getmap magic: 0 -- 0x7f41040d4680 con 0x7f41040d3230 2024-04-25T10:29:08.162 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:08.157+0000 7f4103fff640 1 --1- >> v1:172.21.15.37:6789/0 conn(0x7f41040d0990 0x7f41040d2b20 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.37:6789/0 says I am v1:172.21.15.37:56748/0 (socket says 172.21.15.37:56748) 2024-04-25T10:29:08.162 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:08.157+0000 7f4103fff640 1 -- 172.21.15.37:0/2084645205 learned_addr learned my addr 172.21.15.37:0/2084645205 (peer_addr_for_me v1:172.21.15.37:0/0) 2024-04-25T10:29:08.162 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:08.157+0000 7f4108d44640 1 --2- 172.21.15.37:0/2084645205 >> v2:172.21.15.203:3300/0 conn(0x7f41040d5160 0x7f41040b4e80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:29:08.162 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:08.157+0000 7f4109545640 1 --2- 172.21.15.37:0/2084645205 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f41040d3230 0x7f41040b9060 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:29:08.162 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:08.157+0000 7f41037fe640 1 -- 172.21.15.37:0/2084645205 <== mon.0 v1:172.21.15.37:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 2449810569 0 0) 0x7f41040d1370 con 0x7f41040d0990 2024-04-25T10:29:08.163 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:08.157+0000 7f41037fe640 1 -- 172.21.15.37:0/2084645205 --> v1:172.21.15.37:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f40f0003520 con 0x7f41040d0990 2024-04-25T10:29:08.163 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:08.157+0000 7f41037fe640 1 -- 172.21.15.37:0/2084645205 <== mon.0 v1:172.21.15.37:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 1392406212 0 0) 0x7f40f0003520 con 0x7f41040d0990 2024-04-25T10:29:08.163 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:08.157+0000 7f41037fe640 1 -- 172.21.15.37:0/2084645205 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f41040d3230 msgr2=0x7f41040b9060 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:29:08.163 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:08.157+0000 7f41037fe640 1 --2- 172.21.15.37:0/2084645205 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f41040d3230 0x7f41040b9060 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:29:08.163 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:08.157+0000 7f41037fe640 1 -- 172.21.15.37:0/2084645205 >> v2:172.21.15.203:3300/0 conn(0x7f41040d5160 msgr2=0x7f41040b4e80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:29:08.163 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:08.157+0000 7f41037fe640 1 --2- 172.21.15.37:0/2084645205 >> v2:172.21.15.203:3300/0 conn(0x7f41040d5160 0x7f41040b4e80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:29:08.163 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:08.157+0000 7f41037fe640 1 -- 172.21.15.37:0/2084645205 --> v1:172.21.15.37:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f41040d4960 con 0x7f41040d0990 2024-04-25T10:29:08.163 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:08.157+0000 7f41037fe640 1 -- 172.21.15.37:0/2084645205 <== mon.0 v1:172.21.15.37:6789/0 3 ==== mon_map magic: 0 ==== 413+0+0 (unknown 1577794600 0 0) 0x7f40f4001a10 con 0x7f41040d0990 2024-04-25T10:29:08.163 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:08.157+0000 7f41037fe640 1 -- 172.21.15.37:0/2084645205 >> v1:172.21.15.37:6789/0 conn(0x7f41040d0990 legacy=0x7f41040d2b20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:29:08.163 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:08.157+0000 7f41037fe640 1 --2- 172.21.15.37:0/2084645205 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f40f00039b0 0x7f40f0003e20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:29:08.163 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:08.157+0000 7f41037fe640 1 --2- 172.21.15.37:0/2084645205 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f40f0004800 0x7f40f0004c10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:29:08.163 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:08.157+0000 7f41037fe640 1 --2- 172.21.15.37:0/2084645205 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f40f00051a0 0x7f40f000b9c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:29:08.163 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:08.157+0000 7f41037fe640 1 -- 172.21.15.37:0/2084645205 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 -- 0x7f41040d1650 con 0x7f40f00039b0 2024-04-25T10:29:08.164 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:08.157+0000 7f41037fe640 1 -- 172.21.15.37:0/2084645205 --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_getmap magic: 0 -- 0x7f41040d4680 con 0x7f40f0004800 2024-04-25T10:29:08.164 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:08.157+0000 7f41037fe640 1 -- 172.21.15.37:0/2084645205 --> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] -- mon_getmap magic: 0 -- 0x7f40f000bf80 con 0x7f40f00051a0 2024-04-25T10:29:08.164 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:08.158+0000 7f4108d44640 1 --2- 172.21.15.37:0/2084645205 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f40f00039b0 0x7f40f0003e20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:29:08.164 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:08.158+0000 7f4103fff640 1 --2- 172.21.15.37:0/2084645205 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f40f0004800 0x7f40f0004c10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:29:08.164 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:08.158+0000 7f4109545640 1 --2- 172.21.15.37:0/2084645205 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f40f00051a0 0x7f40f000b9c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:29:08.164 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:08.158+0000 7f4108d44640 1 -- 172.21.15.37:0/2084645205 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f40f00051a0 msgr2=0x7f40f000b9c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:29:08.164 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:08.158+0000 7f4108d44640 1 --2- 172.21.15.37:0/2084645205 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f40f00051a0 0x7f40f000b9c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:29:08.164 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:08.158+0000 7f4108d44640 1 -- 172.21.15.37:0/2084645205 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f40f0004800 msgr2=0x7f40f0004c10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:29:08.164 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:08.158+0000 7f4108d44640 1 --2- 172.21.15.37:0/2084645205 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f40f0004800 0x7f40f0004c10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:29:08.164 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:08.158+0000 7f4108d44640 1 -- 172.21.15.37:0/2084645205 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f40f0003520 con 0x7f40f00039b0 2024-04-25T10:29:08.164 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:08.159+0000 7f4108d44640 1 --2- 172.21.15.37:0/2084645205 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f40f00039b0 0x7f40f0003e20 secure :-1 s=READY pgs=60 cs=0 l=1 rev1=1 crypto rx=0x7f40fc009ac0 tx=0x7f40f0005830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=6078efedc5bd8e5a server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:29:08.164 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:08.159+0000 7f41037fe640 1 -- 172.21.15.37:0/2084645205 <== mon.0 v2:172.21.15.37:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f40fc036070 con 0x7f40f00039b0 2024-04-25T10:29:08.164 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:08.159+0000 7f41037fe640 1 -- 172.21.15.37:0/2084645205 <== mon.0 v2:172.21.15.37:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f40fc031400 con 0x7f40f00039b0 2024-04-25T10:29:08.164 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:08.159+0000 7f41037fe640 1 -- 172.21.15.37:0/2084645205 <== mon.0 v2:172.21.15.37:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f40fc009690 con 0x7f40f00039b0 2024-04-25T10:29:08.165 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:08.159+0000 7f410afa8640 1 -- 172.21.15.37:0/2084645205 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f40f00039b0 msgr2=0x7f40f0003e20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:29:08.165 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:08.159+0000 7f410afa8640 1 --2- 172.21.15.37:0/2084645205 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f40f00039b0 0x7f40f0003e20 secure :-1 s=READY pgs=60 cs=0 l=1 rev1=1 crypto rx=0x7f40fc009ac0 tx=0x7f40f0005830 comp rx=0 tx=0).stop 2024-04-25T10:29:08.165 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:08.159+0000 7f4109545640 1 -- 172.21.15.37:0/2084645205 reap_dead start 2024-04-25T10:29:08.165 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:08.159+0000 7f410afa8640 1 -- 172.21.15.37:0/2084645205 shutdown_connections 2024-04-25T10:29:08.165 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:08.159+0000 7f410afa8640 1 -- 172.21.15.37:0/2084645205 >> 172.21.15.37:0/2084645205 conn(0x7f41040b7a80 msgr2=0x7f41040b7e90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:29:08.165 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:08.159+0000 7f410afa8640 1 -- 172.21.15.37:0/2084645205 shutdown_connections 2024-04-25T10:29:08.165 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:08.159+0000 7f410afa8640 1 -- 172.21.15.37:0/2084645205 wait complete. 2024-04-25T10:29:08.168 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:08.161+0000 7f410afa8640 1 Processor -- start 2024-04-25T10:29:08.168 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:08.161+0000 7f410afa8640 1 -- start start 2024-04-25T10:29:08.168 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:08.163+0000 7f410afa8640 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f40f00039b0 0x7f41040cf360 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:29:08.168 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:08.163+0000 7f410afa8640 1 --2- >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f4104157f60 0x7f41040c9b30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:29:08.168 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:08.163+0000 7f410afa8640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f4104158370 0x7f41040ca070 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:29:08.168 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:08.163+0000 7f410afa8640 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 -- 0x7f4104164c40 con 0x7f4104158370 2024-04-25T10:29:08.168 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:08.163+0000 7f410afa8640 1 -- --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_getmap magic: 0 -- 0x7f4104164ac0 con 0x7f40f00039b0 2024-04-25T10:29:08.168 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:08.163+0000 7f410afa8640 1 -- --> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] -- mon_getmap magic: 0 -- 0x7f41040d3230 con 0x7f4104157f60 2024-04-25T10:29:08.169 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:08.163+0000 7f4109545640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f4104158370 0x7f41040ca070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:29:08.169 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:08.163+0000 7f4109545640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f4104158370 0x7f41040ca070 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.37:3300/0 says I am v2:172.21.15.37:46422/0 (socket says 172.21.15.37:46422) 2024-04-25T10:29:08.169 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:08.163+0000 7f4109545640 1 -- 172.21.15.37:0/2069072372 learned_addr learned my addr 172.21.15.37:0/2069072372 (peer_addr_for_me v2:172.21.15.37:0/0) 2024-04-25T10:29:08.169 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:08.163+0000 7f4108d44640 1 --2- 172.21.15.37:0/2069072372 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f40f00039b0 0x7f41040cf360 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:29:08.169 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:08.163+0000 7f4103fff640 1 --2- 172.21.15.37:0/2069072372 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f4104157f60 0x7f41040c9b30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:29:08.169 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:08.163+0000 7f4109545640 1 -- 172.21.15.37:0/2069072372 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f4104157f60 msgr2=0x7f41040c9b30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:29:08.169 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:08.163+0000 7f4109545640 1 --2- 172.21.15.37:0/2069072372 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f4104157f60 0x7f41040c9b30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:29:08.169 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:08.163+0000 7f4109545640 1 -- 172.21.15.37:0/2069072372 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f40f00039b0 msgr2=0x7f41040cf360 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:29:08.169 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:08.163+0000 7f4109545640 1 --2- 172.21.15.37:0/2069072372 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f40f00039b0 0x7f41040cf360 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:29:08.169 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:08.163+0000 7f4109545640 1 -- 172.21.15.37:0/2069072372 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f41040cf8a0 con 0x7f4104158370 2024-04-25T10:29:08.169 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:08.163+0000 7f4109545640 1 --2- 172.21.15.37:0/2069072372 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f4104158370 0x7f41040ca070 secure :-1 s=READY pgs=61 cs=0 l=1 rev1=1 crypto rx=0x7f410408f070 tx=0x7f40ec011c30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:29:08.169 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:08.163+0000 7f41017fa640 1 -- 172.21.15.37:0/2069072372 <== mon.0 v2:172.21.15.37:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f40ec004030 con 0x7f4104158370 2024-04-25T10:29:08.169 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:08.163+0000 7f41017fa640 1 -- 172.21.15.37:0/2069072372 <== mon.0 v2:172.21.15.37:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f40ec01b070 con 0x7f4104158370 2024-04-25T10:29:08.169 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:08.163+0000 7f410afa8640 1 -- 172.21.15.37:0/2069072372 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f41040cdd20 con 0x7f4104158370 2024-04-25T10:29:08.169 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:08.163+0000 7f41017fa640 1 -- 172.21.15.37:0/2069072372 <== mon.0 v2:172.21.15.37:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f40ec0043f0 con 0x7f4104158370 2024-04-25T10:29:08.170 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:08.164+0000 7f410afa8640 1 -- 172.21.15.37:0/2069072372 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f41040ce2e0 con 0x7f4104158370 2024-04-25T10:29:08.170 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:08.164+0000 7f410afa8640 1 -- 172.21.15.37:0/2069072372 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f410406a140 con 0x7f4104158370 2024-04-25T10:29:08.170 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:08.165+0000 7f41017fa640 1 -- 172.21.15.37:0/2069072372 <== mon.0 v2:172.21.15.37:3300/0 4 ==== mgrmap(e 5) ==== 96782+0+0 (secure 0 0 0) 0x7f40ec012040 con 0x7f4104158370 2024-04-25T10:29:08.170 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:08.166+0000 7f41017fa640 1 --2- 172.21.15.37:0/2069072372 >> [v2:172.21.15.203:6832/2418205237,v1:172.21.15.203:6833/2418205237] conn(0x7f40e40744b0 0x7f40e40769c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:29:08.170 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:08.166+0000 7f41017fa640 1 -- 172.21.15.37:0/2069072372 <== mon.0 v2:172.21.15.37:3300/0 5 ==== osd_map(21..21 src has 1..21) ==== 5609+0+0 (secure 0 0 0) 0x7f40ec0950d0 con 0x7f4104158370 2024-04-25T10:29:08.170 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:08.166+0000 7f4108d44640 1 --2- 172.21.15.37:0/2069072372 >> [v2:172.21.15.203:6832/2418205237,v1:172.21.15.203:6833/2418205237] conn(0x7f40e40744b0 0x7f40e40769c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:29:08.170 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:08.166+0000 7f4108d44640 1 --2- 172.21.15.37:0/2069072372 >> [v2:172.21.15.203:6832/2418205237,v1:172.21.15.203:6833/2418205237] conn(0x7f40e40744b0 0x7f40e40769c0 secure :-1 s=READY pgs=57 cs=0 l=1 rev1=1 crypto rx=0x7f40fc009ac0 tx=0x7f40fc008030 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:29:08.172 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:08.171+0000 7f41017fa640 1 -- 172.21.15.37:0/2069072372 <== mon.0 v2:172.21.15.37:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187246 (secure 0 0 0) 0x7f40ec05f8c0 con 0x7f4104158370 2024-04-25T10:29:08.398 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:08.396+0000 7f410afa8640 1 -- 172.21.15.37:0/2069072372 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_command({"prefix": "fs new", "fs_name": "cephfs", "metadata": "cephfs_metadata", "data": "cephfs_data"} v 0) -- 0x7f41040d3f50 con 0x7f4104158370 2024-04-25T10:29:09.018 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:09.016+0000 7f41017fa640 1 -- 172.21.15.37:0/2069072372 <== mon.0 v2:172.21.15.37:3300/0 7 ==== mon_command_ack([{"prefix": "fs new", "fs_name": "cephfs", "metadata": "cephfs_metadata", "data": "cephfs_data"}]=0 new fs with metadata pool 2 and data pool 3 v4) ==== 172+0+0 (secure 0 0 0) 0x7f40ec05f1c0 con 0x7f4104158370 2024-04-25T10:29:09.018 INFO:teuthology.orchestra.run.smithi037.stderr:new fs with metadata pool 2 and data pool 3 2024-04-25T10:29:09.027 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:09.025+0000 7f410afa8640 1 -- 172.21.15.37:0/2069072372 >> [v2:172.21.15.203:6832/2418205237,v1:172.21.15.203:6833/2418205237] conn(0x7f40e40744b0 msgr2=0x7f40e40769c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:29:09.027 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:09.025+0000 7f410afa8640 1 --2- 172.21.15.37:0/2069072372 >> [v2:172.21.15.203:6832/2418205237,v1:172.21.15.203:6833/2418205237] conn(0x7f40e40744b0 0x7f40e40769c0 secure :-1 s=READY pgs=57 cs=0 l=1 rev1=1 crypto rx=0x7f40fc009ac0 tx=0x7f40fc008030 comp rx=0 tx=0).stop 2024-04-25T10:29:09.027 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:09.025+0000 7f410afa8640 1 -- 172.21.15.37:0/2069072372 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f4104158370 msgr2=0x7f41040ca070 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:29:09.027 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:09.025+0000 7f410afa8640 1 --2- 172.21.15.37:0/2069072372 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f4104158370 0x7f41040ca070 secure :-1 s=READY pgs=61 cs=0 l=1 rev1=1 crypto rx=0x7f410408f070 tx=0x7f40ec011c30 comp rx=0 tx=0).stop 2024-04-25T10:29:09.028 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:09.026+0000 7f410afa8640 1 -- 172.21.15.37:0/2069072372 shutdown_connections 2024-04-25T10:29:09.029 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:09.026+0000 7f410afa8640 1 --2- 172.21.15.37:0/2069072372 >> [v2:172.21.15.203:6832/2418205237,v1:172.21.15.203:6833/2418205237] conn(0x7f40e40744b0 0x7f40e40769c0 unknown :-1 s=CLOSED pgs=57 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:29:09.029 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:09.026+0000 7f410afa8640 1 --2- 172.21.15.37:0/2069072372 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f4104158370 0x7f41040ca070 unknown :-1 s=CLOSED pgs=61 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:29:09.029 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:09.026+0000 7f410afa8640 1 --2- 172.21.15.37:0/2069072372 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f4104157f60 0x7f41040c9b30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:29:09.029 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:09.026+0000 7f410afa8640 1 --2- 172.21.15.37:0/2069072372 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f40f00039b0 0x7f41040cf360 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:29:09.029 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:09.026+0000 7f410afa8640 1 -- 172.21.15.37:0/2069072372 >> 172.21.15.37:0/2069072372 conn(0x7f41040b7a80 msgr2=0x7f4104163130 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:29:09.032 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:09.030+0000 7f410afa8640 1 -- 172.21.15.37:0/2069072372 shutdown_connections 2024-04-25T10:29:09.032 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:09.030+0000 7f410afa8640 1 -- 172.21.15.37:0/2069072372 wait complete. 2024-04-25T10:29:09.053 DEBUG:teuthology.orchestra.run.smithi037:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-04-25T10:29:09.172 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:09.167+0000 7fea463a8640 1 Processor -- start 2024-04-25T10:29:09.172 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:09.167+0000 7fea463a8640 1 -- start start 2024-04-25T10:29:09.172 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:09.168+0000 7fea463a8640 1 --2- >> v2:172.21.15.37:3300/0 conn(0x7fea400d5430 0x7fea400b75c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:29:09.172 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:09.168+0000 7fea463a8640 1 --2- >> v2:172.21.15.203:3300/0 conn(0x7fea40161110 0x7fea40161880 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:29:09.172 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:09.168+0000 7fea463a8640 1 -- --> v1:172.21.15.37:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fea400b7f10 con 0x7fea400b7b00 2024-04-25T10:29:09.172 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:09.168+0000 7fea463a8640 1 -- --> v2:172.21.15.37:3300/0 -- mon_getmap magic: 0 -- 0x7fea400d4420 con 0x7fea400d5430 2024-04-25T10:29:09.172 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:09.168+0000 7fea463a8640 1 -- --> v2:172.21.15.203:3300/0 -- mon_getmap magic: 0 -- 0x7fea40161520 con 0x7fea40161110 2024-04-25T10:29:09.172 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:09.168+0000 7fea3ffff640 1 --2- >> v2:172.21.15.37:3300/0 conn(0x7fea400d5430 0x7fea400b75c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:29:09.172 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:09.168+0000 7fea3f7fe640 1 --1- >> v1:172.21.15.37:6789/0 conn(0x7fea400b7b00 0x7fea40160a00 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.37:6789/0 says I am v1:172.21.15.37:56756/0 (socket says 172.21.15.37:56756) 2024-04-25T10:29:09.172 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:09.168+0000 7fea3f7fe640 1 -- 172.21.15.37:0/1016251345 learned_addr learned my addr 172.21.15.37:0/1016251345 (peer_addr_for_me v1:172.21.15.37:0/0) 2024-04-25T10:29:09.172 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:09.168+0000 7fea44945640 1 --2- 172.21.15.37:0/1016251345 >> v2:172.21.15.203:3300/0 conn(0x7fea40161110 0x7fea40161880 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:29:09.173 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:09.169+0000 7fea3ffff640 1 -- 172.21.15.37:0/1016251345 >> v2:172.21.15.203:3300/0 conn(0x7fea40161110 msgr2=0x7fea40161880 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:29:09.173 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:09.169+0000 7fea3ffff640 1 --2- 172.21.15.37:0/1016251345 >> v2:172.21.15.203:3300/0 conn(0x7fea40161110 0x7fea40161880 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:29:09.173 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:09.169+0000 7fea3ffff640 1 -- 172.21.15.37:0/1016251345 >> v1:172.21.15.37:6789/0 conn(0x7fea400b7b00 legacy=0x7fea40160a00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:29:09.173 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:09.169+0000 7fea3effd640 1 -- 172.21.15.37:0/1016251345 <== mon.0 v1:172.21.15.37:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 3748364325 0 0) 0x7fea34001110 con 0x7fea400b7b00 2024-04-25T10:29:09.173 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:09.169+0000 7fea3ffff640 1 -- 172.21.15.37:0/1016251345 --> v2:172.21.15.37:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fea400d5840 con 0x7fea400d5430 2024-04-25T10:29:09.173 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:09.169+0000 7fea3ffff640 1 --2- 172.21.15.37:0/1016251345 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fea400d5430 0x7fea400b75c0 secure :-1 s=READY pgs=63 cs=0 l=1 rev1=1 crypto rx=0x7fea400d3cb0 tx=0x7fea3002a110 comp rx=0 tx=0).ready entity=mon.0 client_cookie=152425151b6cce68 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:29:09.173 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:09.170+0000 7fea3effd640 1 -- 172.21.15.37:0/1016251345 <== mon.0 v2:172.21.15.37:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fea30037070 con 0x7fea400d5430 2024-04-25T10:29:09.173 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:09.170+0000 7fea3effd640 1 -- 172.21.15.37:0/1016251345 <== mon.0 v2:172.21.15.37:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fea30032440 con 0x7fea400d5430 2024-04-25T10:29:09.173 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:09.170+0000 7fea3effd640 1 -- 172.21.15.37:0/1016251345 <== mon.0 v2:172.21.15.37:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fea3002ae60 con 0x7fea400d5430 2024-04-25T10:29:09.174 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:09.170+0000 7fea463a8640 1 -- 172.21.15.37:0/1016251345 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fea400d5430 msgr2=0x7fea400b75c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:29:09.174 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:09.170+0000 7fea463a8640 1 --2- 172.21.15.37:0/1016251345 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fea400d5430 0x7fea400b75c0 secure :-1 s=READY pgs=63 cs=0 l=1 rev1=1 crypto rx=0x7fea400d3cb0 tx=0x7fea3002a110 comp rx=0 tx=0).stop 2024-04-25T10:29:09.174 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:09.170+0000 7fea463a8640 1 -- 172.21.15.37:0/1016251345 shutdown_connections 2024-04-25T10:29:09.174 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:09.170+0000 7fea463a8640 1 --2- 172.21.15.37:0/1016251345 >> v2:172.21.15.203:3300/0 conn(0x7fea40161110 0x7fea40161880 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:29:09.175 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:09.170+0000 7fea463a8640 1 --2- 172.21.15.37:0/1016251345 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fea400d5430 0x7fea400b75c0 unknown :-1 s=CLOSED pgs=63 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:29:09.175 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:09.170+0000 7fea463a8640 1 -- 172.21.15.37:0/1016251345 >> 172.21.15.37:0/1016251345 conn(0x7fea400d27a0 msgr2=0x7fea400d2bb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:29:09.175 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:09.170+0000 7fea463a8640 1 -- 172.21.15.37:0/1016251345 shutdown_connections 2024-04-25T10:29:09.175 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:09.170+0000 7fea463a8640 1 -- 172.21.15.37:0/1016251345 wait complete. 2024-04-25T10:29:09.175 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:09.172+0000 7fea463a8640 1 Processor -- start 2024-04-25T10:29:09.175 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:09.172+0000 7fea463a8640 1 -- start start 2024-04-25T10:29:09.177 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:09.173+0000 7fea463a8640 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fea400b7b00 0x7fea4015bb20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:29:09.177 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:09.174+0000 7fea463a8640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fea400d5430 0x7fea400ce450 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:29:09.178 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:09.174+0000 7fea463a8640 1 --2- >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7fea40161110 0x7fea400cc980 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:29:09.178 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:09.174+0000 7fea463a8640 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 -- 0x7fea400d4c30 con 0x7fea400d5430 2024-04-25T10:29:09.178 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:09.174+0000 7fea463a8640 1 -- --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_getmap magic: 0 -- 0x7fea400d4950 con 0x7fea400b7b00 2024-04-25T10:29:09.178 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:09.174+0000 7fea463a8640 1 -- --> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] -- mon_getmap magic: 0 -- 0x7fea400d4f10 con 0x7fea40161110 2024-04-25T10:29:09.178 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:09.174+0000 7fea3f7fe640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fea400d5430 0x7fea400ce450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:29:09.178 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:09.174+0000 7fea3f7fe640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fea400d5430 0x7fea400ce450 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.37:3300/0 says I am v2:172.21.15.37:46450/0 (socket says 172.21.15.37:46450) 2024-04-25T10:29:09.178 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:09.174+0000 7fea3f7fe640 1 -- 172.21.15.37:0/2511295240 learned_addr learned my addr 172.21.15.37:0/2511295240 (peer_addr_for_me v2:172.21.15.37:0/0) 2024-04-25T10:29:09.178 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:09.174+0000 7fea3ffff640 1 --2- 172.21.15.37:0/2511295240 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fea400b7b00 0x7fea4015bb20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:29:09.178 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:09.174+0000 7fea44945640 1 --2- 172.21.15.37:0/2511295240 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7fea40161110 0x7fea400cc980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:29:09.178 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:09.174+0000 7fea3f7fe640 1 -- 172.21.15.37:0/2511295240 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7fea40161110 msgr2=0x7fea400cc980 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:29:09.178 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:09.174+0000 7fea3f7fe640 1 --2- 172.21.15.37:0/2511295240 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7fea40161110 0x7fea400cc980 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:29:09.178 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:09.174+0000 7fea3f7fe640 1 -- 172.21.15.37:0/2511295240 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fea400b7b00 msgr2=0x7fea4015bb20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:29:09.178 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:09.174+0000 7fea3f7fe640 1 --2- 172.21.15.37:0/2511295240 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fea400b7b00 0x7fea4015bb20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:29:09.178 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:09.174+0000 7fea3f7fe640 1 -- 172.21.15.37:0/2511295240 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fea400cd3b0 con 0x7fea400d5430 2024-04-25T10:29:09.179 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:09.174+0000 7fea3ffff640 1 --2- 172.21.15.37:0/2511295240 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fea400b7b00 0x7fea4015bb20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T10:29:09.179 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:09.174+0000 7fea3f7fe640 1 --2- 172.21.15.37:0/2511295240 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fea400d5430 0x7fea400ce450 secure :-1 s=READY pgs=64 cs=0 l=1 rev1=1 crypto rx=0x7fea3400c460 tx=0x7fea3400c920 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:29:09.179 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:09.174+0000 7fea3cff9640 1 -- 172.21.15.37:0/2511295240 <== mon.0 v2:172.21.15.37:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fea34018070 con 0x7fea400d5430 2024-04-25T10:29:09.179 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:09.174+0000 7fea3cff9640 1 -- 172.21.15.37:0/2511295240 <== mon.0 v2:172.21.15.37:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fea34007500 con 0x7fea400d5430 2024-04-25T10:29:09.179 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:09.174+0000 7fea463a8640 1 -- 172.21.15.37:0/2511295240 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fea400cd690 con 0x7fea400d5430 2024-04-25T10:29:09.179 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:09.174+0000 7fea3cff9640 1 -- 172.21.15.37:0/2511295240 <== mon.0 v2:172.21.15.37:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fea34004a20 con 0x7fea400d5430 2024-04-25T10:29:09.179 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:09.174+0000 7fea463a8640 1 -- 172.21.15.37:0/2511295240 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fea400cdc50 con 0x7fea400d5430 2024-04-25T10:29:09.179 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:09.175+0000 7fea463a8640 1 -- 172.21.15.37:0/2511295240 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fea4006a140 con 0x7fea400d5430 2024-04-25T10:29:09.180 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:09.176+0000 7fea3cff9640 1 -- 172.21.15.37:0/2511295240 <== mon.0 v2:172.21.15.37:3300/0 4 ==== mgrmap(e 5) ==== 96782+0+0 (secure 0 0 0) 0x7fea3401b070 con 0x7fea400d5430 2024-04-25T10:29:09.180 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:09.176+0000 7fea3cff9640 1 --2- 172.21.15.37:0/2511295240 >> [v2:172.21.15.203:6832/2418205237,v1:172.21.15.203:6833/2418205237] conn(0x7fea2c0744b0 0x7fea2c0769c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:29:09.180 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:09.177+0000 7fea3cff9640 1 -- 172.21.15.37:0/2511295240 <== mon.0 v2:172.21.15.37:3300/0 5 ==== osd_map(22..22 src has 1..22) ==== 5709+0+0 (secure 0 0 0) 0x7fea340943e0 con 0x7fea400d5430 2024-04-25T10:29:09.180 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:09.177+0000 7fea3ffff640 1 --2- 172.21.15.37:0/2511295240 >> [v2:172.21.15.203:6832/2418205237,v1:172.21.15.203:6833/2418205237] conn(0x7fea2c0744b0 0x7fea2c0769c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:29:09.180 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:09.177+0000 7fea3ffff640 1 --2- 172.21.15.37:0/2511295240 >> [v2:172.21.15.203:6832/2418205237,v1:172.21.15.203:6833/2418205237] conn(0x7fea2c0744b0 0x7fea2c0769c0 secure :-1 s=READY pgs=58 cs=0 l=1 rev1=1 crypto rx=0x7fea30034000 tx=0x7fea3002e000 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:29:09.183 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:09.181+0000 7fea3cff9640 1 -- 172.21.15.37:0/2511295240 <== mon.0 v2:172.21.15.37:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187246 (secure 0 0 0) 0x7fea400b7f10 con 0x7fea400d5430 2024-04-25T10:29:09.368 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:09.366+0000 7fea463a8640 1 -- 172.21.15.37:0/2511295240 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7fea40162b20 con 0x7fea400d5430 2024-04-25T10:29:09.369 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:09.367+0000 7fea3cff9640 1 -- 172.21.15.37:0/2511295240 <== mon.0 v2:172.21.15.37:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v22) ==== 74+0+16634 (secure 0 0 0) 0x7fea3405ec90 con 0x7fea400d5430 2024-04-25T10:29:09.369 INFO:teuthology.orchestra.run.smithi037.stdout: 2024-04-25T10:29:09.370 INFO:teuthology.orchestra.run.smithi037.stdout:{"epoch":22,"fsid":"f4acda28-be34-492a-b286-994afd36f61d","created":"2024-04-25T10:28:43.985718+0000","modified":"2024-04-25T10:29:08.397887+0000","last_up_change":"2024-04-25T10:28:54.900654+0000","last_in_change":"2024-04-25T10:28:46.216973+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":3,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-04-25T10:28:56.202153+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"19","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2024-04-25T10:29:06.507244+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"22","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":2.25,"score_stable":2.25,"optimal_score":1,"raw_score_acting":2.25,"raw_score_stable":2.25,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2024-04-25T10:29:07.347844+0000","flags":32769,"flags_names":"hashpspool,creating","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"22","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"a2f79028-b332-4166-9ebb-a7024a37ea22","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":21,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.37:6816","nonce":2746074720},{"type":"v1","addr":"172.21.15.37:6817","nonce":2746074720}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.37:6818","nonce":2746074720},{"type":"v1","addr":"172.21.15.37:6819","nonce":2746074720}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.37:6822","nonce":2746074720},{"type":"v1","addr":"172.21.15.37:6823","nonce":2746074720}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.37:6820","nonce":2746074720},{"type":"v1","addr":"172.21.15.37:6821","nonce":2746074720}]},"public_addr":"172.21.15.37:6817/2746074720","cluster_addr":"172.21.15.37:6819/2746074720","heartbeat_back_addr":"172.21.15.37:6823/2746074720","heartbeat_front_addr":"172.21.15.37:6821/2746074720","state":["exists","up"]},{"osd":1,"uuid":"973159de-c2ea-4083-aecc-69373d1c1132","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":21,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.37:6800","nonce":1690341662},{"type":"v1","addr":"172.21.15.37:6801","nonce":1690341662}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.37:6802","nonce":1690341662},{"type":"v1","addr":"172.21.15.37:6803","nonce":1690341662}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.37:6806","nonce":1690341662},{"type":"v1","addr":"172.21.15.37:6807","nonce":1690341662}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.37:6804","nonce":1690341662},{"type":"v1","addr":"172.21.15.37:6805","nonce":1690341662}]},"public_addr":"172.21.15.37:6801/1690341662","cluster_addr":"172.21.15.37:6803/1690341662","heartbeat_back_addr":"172.21.15.37:6807/1690341662","heartbeat_front_addr":"172.21.15.37:6805/1690341662","state":["exists","up"]},{"osd":2,"uuid":"4ae7a0e9-cb3f-484c-a75f-f3766b721c47","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":21,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.37:6808","nonce":2754669649},{"type":"v1","addr":"172.21.15.37:6809","nonce":2754669649}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.37:6810","nonce":2754669649},{"type":"v1","addr":"172.21.15.37:6811","nonce":2754669649}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.37:6814","nonce":2754669649},{"type":"v1","addr":"172.21.15.37:6815","nonce":2754669649}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.37:6812","nonce":2754669649},{"type":"v1","addr":"172.21.15.37:6813","nonce":2754669649}]},"public_addr":"172.21.15.37:6809/2754669649","cluster_addr":"172.21.15.37:6811/2754669649","heartbeat_back_addr":"172.21.15.37:6815/2754669649","heartbeat_front_addr":"172.21.15.37:6813/2754669649","state":["exists","up"]},{"osd":3,"uuid":"1ba4a746-1608-47a7-ace2-960a5cfae934","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":21,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.37:6824","nonce":1815290618},{"type":"v1","addr":"172.21.15.37:6825","nonce":1815290618}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.37:6826","nonce":1815290618},{"type":"v1","addr":"172.21.15.37:6827","nonce":1815290618}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.37:6830","nonce":1815290618},{"type":"v1","addr":"172.21.15.37:6831","nonce":1815290618}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.37:6828","nonce":1815290618},{"type":"v1","addr":"172.21.15.37:6829","nonce":1815290618}]},"public_addr":"172.21.15.37:6825/1815290618","cluster_addr":"172.21.15.37:6827/1815290618","heartbeat_back_addr":"172.21.15.37:6831/1815290618","heartbeat_front_addr":"172.21.15.37:6829/1815290618","state":["exists","up"]},{"osd":4,"uuid":"d4f59f5d-0413-4b79-87ee-bb212c1fcf62","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":21,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6800","nonce":4051532063},{"type":"v1","addr":"172.21.15.203:6801","nonce":4051532063}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6802","nonce":4051532063},{"type":"v1","addr":"172.21.15.203:6803","nonce":4051532063}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6806","nonce":4051532063},{"type":"v1","addr":"172.21.15.203:6807","nonce":4051532063}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6804","nonce":4051532063},{"type":"v1","addr":"172.21.15.203:6805","nonce":4051532063}]},"public_addr":"172.21.15.203:6801/4051532063","cluster_addr":"172.21.15.203:6803/4051532063","heartbeat_back_addr":"172.21.15.203:6807/4051532063","heartbeat_front_addr":"172.21.15.203:6805/4051532063","state":["exists","up"]},{"osd":5,"uuid":"69710a0e-41d8-4e7f-9a10-17ce540b67be","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":21,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6824","nonce":1203822890},{"type":"v1","addr":"172.21.15.203:6825","nonce":1203822890}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6826","nonce":1203822890},{"type":"v1","addr":"172.21.15.203:6827","nonce":1203822890}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6830","nonce":1203822890},{"type":"v1","addr":"172.21.15.203:6831","nonce":1203822890}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6828","nonce":1203822890},{"type":"v1","addr":"172.21.15.203:6829","nonce":1203822890}]},"public_addr":"172.21.15.203:6825/1203822890","cluster_addr":"172.21.15.203:6827/1203822890","heartbeat_back_addr":"172.21.15.203:6831/1203822890","heartbeat_front_addr":"172.21.15.203:6829/1203822890","state":["exists","up"]},{"osd":6,"uuid":"f29dd047-3b48-473d-a235-6642a6293f20","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":21,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6810","nonce":2309615014},{"type":"v1","addr":"172.21.15.203:6812","nonce":2309615014}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6814","nonce":2309615014},{"type":"v1","addr":"172.21.15.203:6816","nonce":2309615014}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6822","nonce":2309615014},{"type":"v1","addr":"172.21.15.203:6823","nonce":2309615014}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6818","nonce":2309615014},{"type":"v1","addr":"172.21.15.203:6820","nonce":2309615014}]},"public_addr":"172.21.15.203:6812/2309615014","cluster_addr":"172.21.15.203:6816/2309615014","heartbeat_back_addr":"172.21.15.203:6823/2309615014","heartbeat_front_addr":"172.21.15.203:6820/2309615014","state":["exists","up"]},{"osd":7,"uuid":"6e691644-378b-4fdc-8e86-fb5f56e31f29","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":21,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6808","nonce":3688099014},{"type":"v1","addr":"172.21.15.203:6809","nonce":3688099014}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6811","nonce":3688099014},{"type":"v1","addr":"172.21.15.203:6813","nonce":3688099014}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6819","nonce":3688099014},{"type":"v1","addr":"172.21.15.203:6821","nonce":3688099014}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6815","nonce":3688099014},{"type":"v1","addr":"172.21.15.203:6817","nonce":3688099014}]},"public_addr":"172.21.15.203:6809/3688099014","cluster_addr":"172.21.15.203:6813/3688099014","heartbeat_back_addr":"172.21.15.203:6821/3688099014","heartbeat_front_addr":"172.21.15.203:6817/3688099014","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T10:28:51.419141+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T10:28:51.366448+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T10:28:51.348276+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T10:28:51.380519+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T10:28:51.105812+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T10:28:51.384496+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T10:28:51.388755+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T10:28:51.366958+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-04-25T10:29:09.371 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:09.369+0000 7fea463a8640 1 -- 172.21.15.37:0/2511295240 >> [v2:172.21.15.203:6832/2418205237,v1:172.21.15.203:6833/2418205237] conn(0x7fea2c0744b0 msgr2=0x7fea2c0769c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:29:09.371 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:09.369+0000 7fea463a8640 1 --2- 172.21.15.37:0/2511295240 >> [v2:172.21.15.203:6832/2418205237,v1:172.21.15.203:6833/2418205237] conn(0x7fea2c0744b0 0x7fea2c0769c0 secure :-1 s=READY pgs=58 cs=0 l=1 rev1=1 crypto rx=0x7fea30034000 tx=0x7fea3002e000 comp rx=0 tx=0).stop 2024-04-25T10:29:09.372 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:09.369+0000 7fea463a8640 1 -- 172.21.15.37:0/2511295240 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fea400d5430 msgr2=0x7fea400ce450 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:29:09.372 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:09.369+0000 7fea463a8640 1 --2- 172.21.15.37:0/2511295240 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fea400d5430 0x7fea400ce450 secure :-1 s=READY pgs=64 cs=0 l=1 rev1=1 crypto rx=0x7fea3400c460 tx=0x7fea3400c920 comp rx=0 tx=0).stop 2024-04-25T10:29:09.372 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:09.369+0000 7fea463a8640 1 -- 172.21.15.37:0/2511295240 shutdown_connections 2024-04-25T10:29:09.372 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:09.369+0000 7fea463a8640 1 --2- 172.21.15.37:0/2511295240 >> [v2:172.21.15.203:6832/2418205237,v1:172.21.15.203:6833/2418205237] conn(0x7fea2c0744b0 0x7fea2c0769c0 unknown :-1 s=CLOSED pgs=58 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:29:09.372 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:09.369+0000 7fea463a8640 1 --2- 172.21.15.37:0/2511295240 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7fea40161110 0x7fea400cc980 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:29:09.372 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:09.369+0000 7fea463a8640 1 --2- 172.21.15.37:0/2511295240 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fea400d5430 0x7fea400ce450 unknown :-1 s=CLOSED pgs=64 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:29:09.372 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:09.369+0000 7fea463a8640 1 --2- 172.21.15.37:0/2511295240 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fea400b7b00 0x7fea4015bb20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:29:09.372 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:09.369+0000 7fea463a8640 1 -- 172.21.15.37:0/2511295240 >> 172.21.15.37:0/2511295240 conn(0x7fea400d27a0 msgr2=0x7fea40162180 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:29:09.372 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:09.369+0000 7fea463a8640 1 -- 172.21.15.37:0/2511295240 shutdown_connections 2024-04-25T10:29:09.372 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:09.369+0000 7fea463a8640 1 -- 172.21.15.37:0/2511295240 wait complete. 2024-04-25T10:29:09.382 DEBUG:teuthology.orchestra.run.smithi037:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-04-25T10:29:09.493 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:09.491+0000 7f0d23538640 1 Processor -- start 2024-04-25T10:29:09.493 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:09.491+0000 7f0d23538640 1 -- start start 2024-04-25T10:29:09.495 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:09.492+0000 7f0d23538640 1 --2- >> v2:172.21.15.37:3300/0 conn(0x7f0d1c0d5430 0x7f0d1c0b75c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:29:09.495 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:09.492+0000 7f0d23538640 1 --2- >> v2:172.21.15.203:3300/0 conn(0x7f0d1c161110 0x7f0d1c161880 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:29:09.495 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:09.492+0000 7f0d23538640 1 -- --> v1:172.21.15.37:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f0d1c0b7f10 con 0x7f0d1c0b7b00 2024-04-25T10:29:09.495 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:09.492+0000 7f0d23538640 1 -- --> v2:172.21.15.37:3300/0 -- mon_getmap magic: 0 -- 0x7f0d1c0d4420 con 0x7f0d1c0d5430 2024-04-25T10:29:09.495 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:09.492+0000 7f0d23538640 1 -- --> v2:172.21.15.203:3300/0 -- mon_getmap magic: 0 -- 0x7f0d1c161520 con 0x7f0d1c161110 2024-04-25T10:29:09.495 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:09.492+0000 7f0d212d4640 1 --2- >> v2:172.21.15.37:3300/0 conn(0x7f0d1c0d5430 0x7f0d1c0b75c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:29:09.495 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:09.492+0000 7f0d20ad3640 1 --1- >> v1:172.21.15.37:6789/0 conn(0x7f0d1c0b7b00 0x7f0d1c160a00 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.37:6789/0 says I am v1:172.21.15.37:56766/0 (socket says 172.21.15.37:56766) 2024-04-25T10:29:09.495 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:09.492+0000 7f0d212d4640 1 --2- >> v2:172.21.15.37:3300/0 conn(0x7f0d1c0d5430 0x7f0d1c0b75c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.37:3300/0 says I am v2:172.21.15.37:46458/0 (socket says 172.21.15.37:46458) 2024-04-25T10:29:09.495 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:09.492+0000 7f0d20ad3640 1 -- 172.21.15.37:0/1582956261 learned_addr learned my addr 172.21.15.37:0/1582956261 (peer_addr_for_me v1:172.21.15.37:0/0) 2024-04-25T10:29:09.496 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:09.493+0000 7f0d21ad5640 1 --2- 172.21.15.37:0/1582956261 >> v2:172.21.15.203:3300/0 conn(0x7f0d1c161110 0x7f0d1c161880 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:29:09.496 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:09.493+0000 7f0d212d4640 1 -- 172.21.15.37:0/1582956261 >> v2:172.21.15.203:3300/0 conn(0x7f0d1c161110 msgr2=0x7f0d1c161880 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:29:09.496 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:09.493+0000 7f0d212d4640 1 --2- 172.21.15.37:0/1582956261 >> v2:172.21.15.203:3300/0 conn(0x7f0d1c161110 0x7f0d1c161880 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:29:09.496 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:09.493+0000 7f0d212d4640 1 -- 172.21.15.37:0/1582956261 >> v1:172.21.15.37:6789/0 conn(0x7f0d1c0b7b00 legacy=0x7f0d1c160a00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:29:09.496 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:09.493+0000 7f0d212d4640 1 -- 172.21.15.37:0/1582956261 --> v2:172.21.15.37:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f0d1c0d5840 con 0x7f0d1c0d5430 2024-04-25T10:29:09.497 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:09.493+0000 7f0d212d4640 1 --2- 172.21.15.37:0/1582956261 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f0d1c0d5430 0x7f0d1c0b75c0 secure :-1 s=READY pgs=66 cs=0 l=1 rev1=1 crypto rx=0x7f0d1c0d3cb0 tx=0x7f0d1802a110 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a9fcc0cc79bb1c2b server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:29:09.497 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:09.494+0000 7f0d13fff640 1 -- 172.21.15.37:0/1582956261 <== mon.0 v2:172.21.15.37:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0d18037070 con 0x7f0d1c0d5430 2024-04-25T10:29:09.497 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:09.494+0000 7f0d13fff640 1 -- 172.21.15.37:0/1582956261 <== mon.0 v2:172.21.15.37:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f0d18032440 con 0x7f0d1c0d5430 2024-04-25T10:29:09.497 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:09.494+0000 7f0d13fff640 1 -- 172.21.15.37:0/1582956261 <== mon.0 v2:172.21.15.37:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0d1802ae60 con 0x7f0d1c0d5430 2024-04-25T10:29:09.497 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:09.494+0000 7f0d23538640 1 -- 172.21.15.37:0/1582956261 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f0d1c0d5430 msgr2=0x7f0d1c0b75c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:29:09.497 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:09.494+0000 7f0d23538640 1 --2- 172.21.15.37:0/1582956261 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f0d1c0d5430 0x7f0d1c0b75c0 secure :-1 s=READY pgs=66 cs=0 l=1 rev1=1 crypto rx=0x7f0d1c0d3cb0 tx=0x7f0d1802a110 comp rx=0 tx=0).stop 2024-04-25T10:29:09.497 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:09.494+0000 7f0d23538640 1 -- 172.21.15.37:0/1582956261 shutdown_connections 2024-04-25T10:29:09.498 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:09.494+0000 7f0d23538640 1 --2- 172.21.15.37:0/1582956261 >> v2:172.21.15.203:3300/0 conn(0x7f0d1c161110 0x7f0d1c161880 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:29:09.498 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:09.494+0000 7f0d23538640 1 --2- 172.21.15.37:0/1582956261 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f0d1c0d5430 0x7f0d1c0b75c0 unknown :-1 s=CLOSED pgs=66 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:29:09.498 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:09.494+0000 7f0d23538640 1 -- 172.21.15.37:0/1582956261 >> 172.21.15.37:0/1582956261 conn(0x7f0d1c0d27a0 msgr2=0x7f0d1c0d2bb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:29:09.498 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:09.494+0000 7f0d23538640 1 -- 172.21.15.37:0/1582956261 shutdown_connections 2024-04-25T10:29:09.498 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:09.494+0000 7f0d23538640 1 -- 172.21.15.37:0/1582956261 wait complete. 2024-04-25T10:29:09.498 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:09.496+0000 7f0d23538640 1 Processor -- start 2024-04-25T10:29:09.498 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:09.496+0000 7f0d23538640 1 -- start start 2024-04-25T10:29:09.501 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:09.497+0000 7f0d23538640 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f0d1c0b7b00 0x7f0d1c0cd600 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:29:09.501 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:09.497+0000 7f0d23538640 1 --2- >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f0d1c0d5430 0x7f0d1c15b9d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:29:09.501 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:09.497+0000 7f0d23538640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f0d1c161110 0x7f0d1c0cfa90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:29:09.501 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:09.497+0000 7f0d23538640 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 -- 0x7f0d1c0d4c30 con 0x7f0d1c161110 2024-04-25T10:29:09.501 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:09.497+0000 7f0d23538640 1 -- --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_getmap magic: 0 -- 0x7f0d1c0d4950 con 0x7f0d1c0b7b00 2024-04-25T10:29:09.501 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:09.497+0000 7f0d23538640 1 -- --> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] -- mon_getmap magic: 0 -- 0x7f0d1c0d4f10 con 0x7f0d1c0d5430 2024-04-25T10:29:09.502 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:09.497+0000 7f0d21ad5640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f0d1c161110 0x7f0d1c0cfa90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:29:09.502 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:09.498+0000 7f0d21ad5640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f0d1c161110 0x7f0d1c0cfa90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.37:3300/0 says I am v2:172.21.15.37:46460/0 (socket says 172.21.15.37:46460) 2024-04-25T10:29:09.502 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:09.498+0000 7f0d21ad5640 1 -- 172.21.15.37:0/648307204 learned_addr learned my addr 172.21.15.37:0/648307204 (peer_addr_for_me v2:172.21.15.37:0/0) 2024-04-25T10:29:09.502 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:09.498+0000 7f0d20ad3640 1 --2- 172.21.15.37:0/648307204 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f0d1c0d5430 0x7f0d1c15b9d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:29:09.502 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:09.498+0000 7f0d212d4640 1 --2- 172.21.15.37:0/648307204 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f0d1c0b7b00 0x7f0d1c0cd600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:29:09.502 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:09.498+0000 7f0d21ad5640 1 -- 172.21.15.37:0/648307204 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f0d1c0d5430 msgr2=0x7f0d1c15b9d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:29:09.502 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:09.498+0000 7f0d21ad5640 1 --2- 172.21.15.37:0/648307204 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f0d1c0d5430 0x7f0d1c15b9d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:29:09.502 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:09.498+0000 7f0d21ad5640 1 -- 172.21.15.37:0/648307204 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f0d1c0b7b00 msgr2=0x7f0d1c0cd600 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:29:09.502 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:09.498+0000 7f0d21ad5640 1 --2- 172.21.15.37:0/648307204 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f0d1c0b7b00 0x7f0d1c0cd600 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:29:09.502 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:09.498+0000 7f0d21ad5640 1 -- 172.21.15.37:0/648307204 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f0d1c0cde80 con 0x7f0d1c161110 2024-04-25T10:29:09.502 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:09.498+0000 7f0d21ad5640 1 --2- 172.21.15.37:0/648307204 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f0d1c161110 0x7f0d1c0cfa90 secure :-1 s=READY pgs=67 cs=0 l=1 rev1=1 crypto rx=0x7f0d14007e90 tx=0x7f0d1400e4b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:29:09.502 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:09.498+0000 7f0d11ffb640 1 -- 172.21.15.37:0/648307204 <== mon.0 v2:172.21.15.37:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0d14014420 con 0x7f0d1c161110 2024-04-25T10:29:09.502 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:09.498+0000 7f0d11ffb640 1 -- 172.21.15.37:0/648307204 <== mon.0 v2:172.21.15.37:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f0d14019070 con 0x7f0d1c161110 2024-04-25T10:29:09.502 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:09.498+0000 7f0d23538640 1 -- 172.21.15.37:0/648307204 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f0d1c0ce160 con 0x7f0d1c161110 2024-04-25T10:29:09.503 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:09.498+0000 7f0d11ffb640 1 -- 172.21.15.37:0/648307204 <== mon.0 v2:172.21.15.37:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0d14014800 con 0x7f0d1c161110 2024-04-25T10:29:09.503 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:09.498+0000 7f0d23538640 1 -- 172.21.15.37:0/648307204 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f0d1c1ea070 con 0x7f0d1c161110 2024-04-25T10:29:09.503 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:09.499+0000 7f0d23538640 1 -- 172.21.15.37:0/648307204 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f0d1c06a140 con 0x7f0d1c161110 2024-04-25T10:29:09.503 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:09.500+0000 7f0d11ffb640 1 -- 172.21.15.37:0/648307204 <== mon.0 v2:172.21.15.37:3300/0 4 ==== mgrmap(e 5) ==== 96782+0+0 (secure 0 0 0) 0x7f0d140149a0 con 0x7f0d1c161110 2024-04-25T10:29:09.504 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:09.500+0000 7f0d11ffb640 1 --2- 172.21.15.37:0/648307204 >> [v2:172.21.15.203:6832/2418205237,v1:172.21.15.203:6833/2418205237] conn(0x7f0d000744b0 0x7f0d000769c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:29:09.504 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:09.501+0000 7f0d11ffb640 1 -- 172.21.15.37:0/648307204 <== mon.0 v2:172.21.15.37:3300/0 5 ==== osd_map(22..22 src has 1..22) ==== 5709+0+0 (secure 0 0 0) 0x7f0d14095ba0 con 0x7f0d1c161110 2024-04-25T10:29:09.504 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:09.501+0000 7f0d212d4640 1 --2- 172.21.15.37:0/648307204 >> [v2:172.21.15.203:6832/2418205237,v1:172.21.15.203:6833/2418205237] conn(0x7f0d000744b0 0x7f0d000769c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:29:09.504 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:09.501+0000 7f0d212d4640 1 --2- 172.21.15.37:0/648307204 >> [v2:172.21.15.203:6832/2418205237,v1:172.21.15.203:6833/2418205237] conn(0x7f0d000744b0 0x7f0d000769c0 secure :-1 s=READY pgs=59 cs=0 l=1 rev1=1 crypto rx=0x7f0d18034000 tx=0x7f0d1802e000 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:29:09.507 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:09.505+0000 7f0d11ffb640 1 -- 172.21.15.37:0/648307204 <== mon.0 v2:172.21.15.37:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187246 (secure 0 0 0) 0x7f0d1405fc40 con 0x7f0d1c161110 2024-04-25T10:29:09.695 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:09.693+0000 7f0d23538640 1 -- 172.21.15.37:0/648307204 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7f0d1c0cdc10 con 0x7f0d1c161110 2024-04-25T10:29:09.696 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:09.694+0000 7f0d11ffb640 1 -- 172.21.15.37:0/648307204 <== mon.0 v2:172.21.15.37:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v22) ==== 74+0+16634 (secure 0 0 0) 0x7f0d14063f40 con 0x7f0d1c161110 2024-04-25T10:29:09.697 INFO:teuthology.orchestra.run.smithi037.stdout: 2024-04-25T10:29:09.697 INFO:teuthology.orchestra.run.smithi037.stdout:{"epoch":22,"fsid":"f4acda28-be34-492a-b286-994afd36f61d","created":"2024-04-25T10:28:43.985718+0000","modified":"2024-04-25T10:29:08.397887+0000","last_up_change":"2024-04-25T10:28:54.900654+0000","last_in_change":"2024-04-25T10:28:46.216973+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":3,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-04-25T10:28:56.202153+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"19","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2024-04-25T10:29:06.507244+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"22","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":2.25,"score_stable":2.25,"optimal_score":1,"raw_score_acting":2.25,"raw_score_stable":2.25,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2024-04-25T10:29:07.347844+0000","flags":32769,"flags_names":"hashpspool,creating","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"22","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"a2f79028-b332-4166-9ebb-a7024a37ea22","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":21,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.37:6816","nonce":2746074720},{"type":"v1","addr":"172.21.15.37:6817","nonce":2746074720}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.37:6818","nonce":2746074720},{"type":"v1","addr":"172.21.15.37:6819","nonce":2746074720}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.37:6822","nonce":2746074720},{"type":"v1","addr":"172.21.15.37:6823","nonce":2746074720}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.37:6820","nonce":2746074720},{"type":"v1","addr":"172.21.15.37:6821","nonce":2746074720}]},"public_addr":"172.21.15.37:6817/2746074720","cluster_addr":"172.21.15.37:6819/2746074720","heartbeat_back_addr":"172.21.15.37:6823/2746074720","heartbeat_front_addr":"172.21.15.37:6821/2746074720","state":["exists","up"]},{"osd":1,"uuid":"973159de-c2ea-4083-aecc-69373d1c1132","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":21,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.37:6800","nonce":1690341662},{"type":"v1","addr":"172.21.15.37:6801","nonce":1690341662}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.37:6802","nonce":1690341662},{"type":"v1","addr":"172.21.15.37:6803","nonce":1690341662}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.37:6806","nonce":1690341662},{"type":"v1","addr":"172.21.15.37:6807","nonce":1690341662}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.37:6804","nonce":1690341662},{"type":"v1","addr":"172.21.15.37:6805","nonce":1690341662}]},"public_addr":"172.21.15.37:6801/1690341662","cluster_addr":"172.21.15.37:6803/1690341662","heartbeat_back_addr":"172.21.15.37:6807/1690341662","heartbeat_front_addr":"172.21.15.37:6805/1690341662","state":["exists","up"]},{"osd":2,"uuid":"4ae7a0e9-cb3f-484c-a75f-f3766b721c47","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":21,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.37:6808","nonce":2754669649},{"type":"v1","addr":"172.21.15.37:6809","nonce":2754669649}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.37:6810","nonce":2754669649},{"type":"v1","addr":"172.21.15.37:6811","nonce":2754669649}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.37:6814","nonce":2754669649},{"type":"v1","addr":"172.21.15.37:6815","nonce":2754669649}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.37:6812","nonce":2754669649},{"type":"v1","addr":"172.21.15.37:6813","nonce":2754669649}]},"public_addr":"172.21.15.37:6809/2754669649","cluster_addr":"172.21.15.37:6811/2754669649","heartbeat_back_addr":"172.21.15.37:6815/2754669649","heartbeat_front_addr":"172.21.15.37:6813/2754669649","state":["exists","up"]},{"osd":3,"uuid":"1ba4a746-1608-47a7-ace2-960a5cfae934","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":21,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.37:6824","nonce":1815290618},{"type":"v1","addr":"172.21.15.37:6825","nonce":1815290618}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.37:6826","nonce":1815290618},{"type":"v1","addr":"172.21.15.37:6827","nonce":1815290618}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.37:6830","nonce":1815290618},{"type":"v1","addr":"172.21.15.37:6831","nonce":1815290618}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.37:6828","nonce":1815290618},{"type":"v1","addr":"172.21.15.37:6829","nonce":1815290618}]},"public_addr":"172.21.15.37:6825/1815290618","cluster_addr":"172.21.15.37:6827/1815290618","heartbeat_back_addr":"172.21.15.37:6831/1815290618","heartbeat_front_addr":"172.21.15.37:6829/1815290618","state":["exists","up"]},{"osd":4,"uuid":"d4f59f5d-0413-4b79-87ee-bb212c1fcf62","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":21,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6800","nonce":4051532063},{"type":"v1","addr":"172.21.15.203:6801","nonce":4051532063}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6802","nonce":4051532063},{"type":"v1","addr":"172.21.15.203:6803","nonce":4051532063}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6806","nonce":4051532063},{"type":"v1","addr":"172.21.15.203:6807","nonce":4051532063}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6804","nonce":4051532063},{"type":"v1","addr":"172.21.15.203:6805","nonce":4051532063}]},"public_addr":"172.21.15.203:6801/4051532063","cluster_addr":"172.21.15.203:6803/4051532063","heartbeat_back_addr":"172.21.15.203:6807/4051532063","heartbeat_front_addr":"172.21.15.203:6805/4051532063","state":["exists","up"]},{"osd":5,"uuid":"69710a0e-41d8-4e7f-9a10-17ce540b67be","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":21,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6824","nonce":1203822890},{"type":"v1","addr":"172.21.15.203:6825","nonce":1203822890}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6826","nonce":1203822890},{"type":"v1","addr":"172.21.15.203:6827","nonce":1203822890}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6830","nonce":1203822890},{"type":"v1","addr":"172.21.15.203:6831","nonce":1203822890}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6828","nonce":1203822890},{"type":"v1","addr":"172.21.15.203:6829","nonce":1203822890}]},"public_addr":"172.21.15.203:6825/1203822890","cluster_addr":"172.21.15.203:6827/1203822890","heartbeat_back_addr":"172.21.15.203:6831/1203822890","heartbeat_front_addr":"172.21.15.203:6829/1203822890","state":["exists","up"]},{"osd":6,"uuid":"f29dd047-3b48-473d-a235-6642a6293f20","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":21,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6810","nonce":2309615014},{"type":"v1","addr":"172.21.15.203:6812","nonce":2309615014}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6814","nonce":2309615014},{"type":"v1","addr":"172.21.15.203:6816","nonce":2309615014}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6822","nonce":2309615014},{"type":"v1","addr":"172.21.15.203:6823","nonce":2309615014}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6818","nonce":2309615014},{"type":"v1","addr":"172.21.15.203:6820","nonce":2309615014}]},"public_addr":"172.21.15.203:6812/2309615014","cluster_addr":"172.21.15.203:6816/2309615014","heartbeat_back_addr":"172.21.15.203:6823/2309615014","heartbeat_front_addr":"172.21.15.203:6820/2309615014","state":["exists","up"]},{"osd":7,"uuid":"6e691644-378b-4fdc-8e86-fb5f56e31f29","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":21,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6808","nonce":3688099014},{"type":"v1","addr":"172.21.15.203:6809","nonce":3688099014}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6811","nonce":3688099014},{"type":"v1","addr":"172.21.15.203:6813","nonce":3688099014}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6819","nonce":3688099014},{"type":"v1","addr":"172.21.15.203:6821","nonce":3688099014}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6815","nonce":3688099014},{"type":"v1","addr":"172.21.15.203:6817","nonce":3688099014}]},"public_addr":"172.21.15.203:6809/3688099014","cluster_addr":"172.21.15.203:6813/3688099014","heartbeat_back_addr":"172.21.15.203:6821/3688099014","heartbeat_front_addr":"172.21.15.203:6817/3688099014","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T10:28:51.419141+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T10:28:51.366448+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T10:28:51.348276+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T10:28:51.380519+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T10:28:51.105812+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T10:28:51.384496+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T10:28:51.388755+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T10:28:51.366958+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-04-25T10:29:09.699 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:09.696+0000 7f0d23538640 1 -- 172.21.15.37:0/648307204 >> [v2:172.21.15.203:6832/2418205237,v1:172.21.15.203:6833/2418205237] conn(0x7f0d000744b0 msgr2=0x7f0d000769c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:29:09.699 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:09.696+0000 7f0d23538640 1 --2- 172.21.15.37:0/648307204 >> [v2:172.21.15.203:6832/2418205237,v1:172.21.15.203:6833/2418205237] conn(0x7f0d000744b0 0x7f0d000769c0 secure :-1 s=READY pgs=59 cs=0 l=1 rev1=1 crypto rx=0x7f0d18034000 tx=0x7f0d1802e000 comp rx=0 tx=0).stop 2024-04-25T10:29:09.699 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:09.696+0000 7f0d23538640 1 -- 172.21.15.37:0/648307204 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f0d1c161110 msgr2=0x7f0d1c0cfa90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:29:09.699 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:09.696+0000 7f0d23538640 1 --2- 172.21.15.37:0/648307204 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f0d1c161110 0x7f0d1c0cfa90 secure :-1 s=READY pgs=67 cs=0 l=1 rev1=1 crypto rx=0x7f0d14007e90 tx=0x7f0d1400e4b0 comp rx=0 tx=0).stop 2024-04-25T10:29:09.699 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:09.697+0000 7f0d23538640 1 -- 172.21.15.37:0/648307204 shutdown_connections 2024-04-25T10:29:09.699 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:09.697+0000 7f0d23538640 1 --2- 172.21.15.37:0/648307204 >> [v2:172.21.15.203:6832/2418205237,v1:172.21.15.203:6833/2418205237] conn(0x7f0d000744b0 0x7f0d000769c0 unknown :-1 s=CLOSED pgs=59 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:29:09.699 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:09.697+0000 7f0d23538640 1 --2- 172.21.15.37:0/648307204 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f0d1c161110 0x7f0d1c0cfa90 unknown :-1 s=CLOSED pgs=67 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:29:09.699 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:09.697+0000 7f0d23538640 1 --2- 172.21.15.37:0/648307204 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f0d1c0d5430 0x7f0d1c15b9d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:29:09.700 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:09.697+0000 7f0d23538640 1 --2- 172.21.15.37:0/648307204 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f0d1c0b7b00 0x7f0d1c0cd600 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:29:09.700 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:09.697+0000 7f0d23538640 1 -- 172.21.15.37:0/648307204 >> 172.21.15.37:0/648307204 conn(0x7f0d1c0d27a0 msgr2=0x7f0d1c0d5b20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:29:09.700 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:09.697+0000 7f0d23538640 1 -- 172.21.15.37:0/648307204 shutdown_connections 2024-04-25T10:29:09.700 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:09.697+0000 7f0d23538640 1 -- 172.21.15.37:0/648307204 wait complete. 2024-04-25T10:29:09.710 DEBUG:teuthology.orchestra.run.smithi037:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs set cephfs standby_count_wanted 0 2024-04-25T10:29:09.824 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:09.822+0000 7faf33a5a640 1 Processor -- start 2024-04-25T10:29:09.824 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:09.822+0000 7faf33a5a640 1 -- start start 2024-04-25T10:29:09.828 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:09.823+0000 7faf33a5a640 1 --2- >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7faf2c161110 0x7faf2c161880 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:29:09.828 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:09.823+0000 7faf33a5a640 1 -- --> v1:172.21.15.37:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7faf2c161520 con 0x7faf2c0b7cd0 2024-04-25T10:29:09.828 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:09.823+0000 7faf33a5a640 1 -- --> v1:172.21.15.203:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7faf2c0d5840 con 0x7faf2c0d5430 2024-04-25T10:29:09.828 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:09.823+0000 7faf33a5a640 1 -- --> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] -- mon_getmap magic: 0 -- 0x7faf2c0d4420 con 0x7faf2c161110 2024-04-25T10:29:09.828 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:09.824+0000 7faf30ff5640 1 --1- >> v1:172.21.15.37:6789/0 conn(0x7faf2c0b7cd0 0x7faf2c160a00 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.37:6789/0 says I am v1:172.21.15.37:51382/0 (socket says 172.21.15.37:51382) 2024-04-25T10:29:09.828 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:09.824+0000 7faf30ff5640 1 -- 172.21.15.37:0/3031512908 learned_addr learned my addr 172.21.15.37:0/3031512908 (peer_addr_for_me v1:172.21.15.37:0/0) 2024-04-25T10:29:09.828 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:09.824+0000 7faf31ff7640 1 --2- 172.21.15.37:0/3031512908 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7faf2c161110 0x7faf2c161880 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:29:09.828 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:09.824+0000 7faf23fff640 1 -- 172.21.15.37:0/3031512908 <== mon.0 v1:172.21.15.37:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 2315460300 0 0) 0x7faf2c161520 con 0x7faf2c0b7cd0 2024-04-25T10:29:09.828 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:09.824+0000 7faf23fff640 1 -- 172.21.15.37:0/3031512908 --> v1:172.21.15.37:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7faf18003520 con 0x7faf2c0b7cd0 2024-04-25T10:29:09.829 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:09.824+0000 7faf23fff640 1 -- 172.21.15.37:0/3031512908 <== mon.1 v1:172.21.15.203:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 4035835176 0 0) 0x7faf2c0d5840 con 0x7faf2c0d5430 2024-04-25T10:29:09.829 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:09.824+0000 7faf23fff640 1 -- 172.21.15.37:0/3031512908 --> v1:172.21.15.203:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7faf2c161520 con 0x7faf2c0d5430 2024-04-25T10:29:09.829 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:09.824+0000 7faf23fff640 1 -- 172.21.15.37:0/3031512908 <== mon.0 v1:172.21.15.37:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 2320971825 0 0) 0x7faf18003520 con 0x7faf2c0b7cd0 2024-04-25T10:29:09.829 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:09.824+0000 7faf23fff640 1 -- 172.21.15.37:0/3031512908 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7faf2c161110 msgr2=0x7faf2c161880 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:29:09.829 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:09.824+0000 7faf23fff640 1 --2- 172.21.15.37:0/3031512908 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7faf2c161110 0x7faf2c161880 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:29:09.829 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:09.824+0000 7faf23fff640 1 -- 172.21.15.37:0/3031512908 >> v1:172.21.15.203:6789/0 conn(0x7faf2c0d5430 legacy=0x7faf2c0b75c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:29:09.829 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:09.824+0000 7faf23fff640 1 -- 172.21.15.37:0/3031512908 --> v1:172.21.15.37:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7faf2c0d4670 con 0x7faf2c0b7cd0 2024-04-25T10:29:09.829 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:09.824+0000 7faf23fff640 1 -- 172.21.15.37:0/3031512908 <== mon.0 v1:172.21.15.37:6789/0 3 ==== mon_map magic: 0 ==== 413+0+0 (unknown 1577794600 0 0) 0x7faf1c001670 con 0x7faf2c0b7cd0 2024-04-25T10:29:09.829 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:09.824+0000 7faf23fff640 1 -- 172.21.15.37:0/3031512908 >> v1:172.21.15.37:6789/0 conn(0x7faf2c0b7cd0 legacy=0x7faf2c160a00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:29:09.829 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:09.824+0000 7faf23fff640 1 --2- 172.21.15.37:0/3031512908 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7faf18003b70 0x7faf18003fc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:29:09.829 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:09.824+0000 7faf23fff640 1 --2- 172.21.15.37:0/3031512908 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7faf2c161110 0x7faf18004800 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:29:09.829 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:09.824+0000 7faf23fff640 1 --2- 172.21.15.37:0/3031512908 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7faf18004d40 0x7faf18005190 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:29:09.829 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:09.824+0000 7faf23fff640 1 -- 172.21.15.37:0/3031512908 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 -- 0x7faf2c0d4420 con 0x7faf18003b70 2024-04-25T10:29:09.829 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:09.824+0000 7faf23fff640 1 -- 172.21.15.37:0/3031512908 --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_getmap magic: 0 -- 0x7faf1800bbd0 con 0x7faf18004d40 2024-04-25T10:29:09.829 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:09.824+0000 7faf23fff640 1 -- 172.21.15.37:0/3031512908 --> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] -- mon_getmap magic: 0 -- 0x7faf1800bdd0 con 0x7faf2c161110 2024-04-25T10:29:09.830 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:09.824+0000 7faf317f6640 1 --2- 172.21.15.37:0/3031512908 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7faf18003b70 0x7faf18003fc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:29:09.830 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:09.825+0000 7faf30ff5640 1 --2- 172.21.15.37:0/3031512908 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7faf2c161110 0x7faf18004800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:29:09.830 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:09.825+0000 7faf31ff7640 1 --2- 172.21.15.37:0/3031512908 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7faf18004d40 0x7faf18005190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:29:09.830 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:09.825+0000 7faf317f6640 1 -- 172.21.15.37:0/3031512908 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7faf2c161110 msgr2=0x7faf18004800 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:29:09.830 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:09.825+0000 7faf317f6640 1 --2- 172.21.15.37:0/3031512908 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7faf2c161110 0x7faf18004800 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:29:09.830 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:09.825+0000 7faf317f6640 1 -- 172.21.15.37:0/3031512908 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7faf18004d40 msgr2=0x7faf18005190 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:29:09.830 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:09.825+0000 7faf317f6640 1 --2- 172.21.15.37:0/3031512908 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7faf18004d40 0x7faf18005190 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:29:09.830 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:09.825+0000 7faf317f6640 1 -- 172.21.15.37:0/3031512908 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7faf18003520 con 0x7faf18003b70 2024-04-25T10:29:09.830 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:09.825+0000 7faf317f6640 1 --2- 172.21.15.37:0/3031512908 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7faf18003b70 0x7faf18003fc0 secure :-1 s=READY pgs=69 cs=0 l=1 rev1=1 crypto rx=0x7faf2c08e110 tx=0x7faf18005830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c87945bf01c24456 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:29:09.830 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:09.826+0000 7faf23fff640 1 -- 172.21.15.37:0/3031512908 <== mon.0 v2:172.21.15.37:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7faf280014f0 con 0x7faf18003b70 2024-04-25T10:29:09.830 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:09.826+0000 7faf23fff640 1 -- 172.21.15.37:0/3031512908 <== mon.0 v2:172.21.15.37:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7faf280052d0 con 0x7faf18003b70 2024-04-25T10:29:09.830 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:09.826+0000 7faf23fff640 1 -- 172.21.15.37:0/3031512908 <== mon.0 v2:172.21.15.37:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7faf2802ac30 con 0x7faf18003b70 2024-04-25T10:29:09.833 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:09.826+0000 7faf33a5a640 1 -- 172.21.15.37:0/3031512908 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7faf18003b70 msgr2=0x7faf18003fc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:29:09.833 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:09.826+0000 7faf33a5a640 1 --2- 172.21.15.37:0/3031512908 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7faf18003b70 0x7faf18003fc0 secure :-1 s=READY pgs=69 cs=0 l=1 rev1=1 crypto rx=0x7faf2c08e110 tx=0x7faf18005830 comp rx=0 tx=0).stop 2024-04-25T10:29:09.833 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:09.826+0000 7faf31ff7640 1 -- 172.21.15.37:0/3031512908 reap_dead start 2024-04-25T10:29:09.833 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:09.826+0000 7faf33a5a640 1 -- 172.21.15.37:0/3031512908 shutdown_connections 2024-04-25T10:29:09.833 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:09.826+0000 7faf33a5a640 1 -- 172.21.15.37:0/3031512908 >> 172.21.15.37:0/3031512908 conn(0x7faf2c0d27a0 msgr2=0x7faf2c0d2bb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:29:09.833 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:09.826+0000 7faf33a5a640 1 -- 172.21.15.37:0/3031512908 shutdown_connections 2024-04-25T10:29:09.834 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:09.826+0000 7faf33a5a640 1 -- 172.21.15.37:0/3031512908 wait complete. 2024-04-25T10:29:09.834 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:09.828+0000 7faf33a5a640 1 Processor -- start 2024-04-25T10:29:09.834 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:09.828+0000 7faf33a5a640 1 -- start start 2024-04-25T10:29:09.834 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:09.829+0000 7faf33a5a640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7faf18003b70 0x7faf2c0ccbf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:29:09.834 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:09.829+0000 7faf33a5a640 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7faf2c0ac680 0x7faf2c14ff80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:29:09.834 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:09.830+0000 7faf33a5a640 1 --2- >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7faf2c0ba020 0x7faf2c0ba430 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:29:09.834 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:09.830+0000 7faf33a5a640 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 -- 0x7faf2c0d4f10 con 0x7faf18003b70 2024-04-25T10:29:09.834 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:09.830+0000 7faf33a5a640 1 -- --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_getmap magic: 0 -- 0x7faf2c0d4c30 con 0x7faf2c0ac680 2024-04-25T10:29:09.834 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:09.830+0000 7faf33a5a640 1 -- --> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] -- mon_getmap magic: 0 -- 0x7faf2c160380 con 0x7faf2c0ba020 2024-04-25T10:29:09.834 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:09.830+0000 7faf317f6640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7faf18003b70 0x7faf2c0ccbf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:29:09.834 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:09.830+0000 7faf317f6640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7faf18003b70 0x7faf2c0ccbf0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.37:3300/0 says I am v2:172.21.15.37:46242/0 (socket says 172.21.15.37:46242) 2024-04-25T10:29:09.834 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:09.830+0000 7faf317f6640 1 -- 172.21.15.37:0/2836667320 learned_addr learned my addr 172.21.15.37:0/2836667320 (peer_addr_for_me v2:172.21.15.37:0/0) 2024-04-25T10:29:09.834 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:09.830+0000 7faf31ff7640 1 --2- 172.21.15.37:0/2836667320 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7faf2c0ba020 0x7faf2c0ba430 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:29:09.834 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:09.830+0000 7faf30ff5640 1 --2- 172.21.15.37:0/2836667320 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7faf2c0ac680 0x7faf2c14ff80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:29:09.834 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:09.830+0000 7faf317f6640 1 -- 172.21.15.37:0/2836667320 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7faf2c0ba020 msgr2=0x7faf2c0ba430 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:29:09.835 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:09.830+0000 7faf317f6640 1 --2- 172.21.15.37:0/2836667320 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7faf2c0ba020 0x7faf2c0ba430 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:29:09.835 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:09.830+0000 7faf317f6640 1 -- 172.21.15.37:0/2836667320 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7faf2c0ac680 msgr2=0x7faf2c14ff80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:29:09.835 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:09.830+0000 7faf317f6640 1 --2- 172.21.15.37:0/2836667320 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7faf2c0ac680 0x7faf2c14ff80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:29:09.835 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:09.830+0000 7faf317f6640 1 -- 172.21.15.37:0/2836667320 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7faf2c1504c0 con 0x7faf18003b70 2024-04-25T10:29:09.835 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:09.830+0000 7faf317f6640 1 --2- 172.21.15.37:0/2836667320 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7faf18003b70 0x7faf2c0ccbf0 secure :-1 s=READY pgs=70 cs=0 l=1 rev1=1 crypto rx=0x7faf28001990 tx=0x7faf280012d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:29:09.835 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:09.830+0000 7faf21ffb640 1 -- 172.21.15.37:0/2836667320 <== mon.0 v2:172.21.15.37:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7faf2803f020 con 0x7faf18003b70 2024-04-25T10:29:09.835 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:09.830+0000 7faf21ffb640 1 -- 172.21.15.37:0/2836667320 <== mon.0 v2:172.21.15.37:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7faf28006e50 con 0x7faf18003b70 2024-04-25T10:29:09.835 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:09.830+0000 7faf33a5a640 1 -- 172.21.15.37:0/2836667320 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7faf2c0ba970 con 0x7faf18003b70 2024-04-25T10:29:09.835 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:09.830+0000 7faf21ffb640 1 -- 172.21.15.37:0/2836667320 <== mon.0 v2:172.21.15.37:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7faf28037040 con 0x7faf18003b70 2024-04-25T10:29:09.835 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:09.831+0000 7faf33a5a640 1 -- 172.21.15.37:0/2836667320 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7faf2c0cd840 con 0x7faf18003b70 2024-04-25T10:29:09.836 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:09.832+0000 7faf33a5a640 1 -- 172.21.15.37:0/2836667320 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7faf2c06a140 con 0x7faf18003b70 2024-04-25T10:29:09.836 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:09.832+0000 7faf21ffb640 1 -- 172.21.15.37:0/2836667320 <== mon.0 v2:172.21.15.37:3300/0 4 ==== mgrmap(e 5) ==== 96782+0+0 (secure 0 0 0) 0x7faf28042020 con 0x7faf18003b70 2024-04-25T10:29:09.836 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:09.833+0000 7faf21ffb640 1 --2- 172.21.15.37:0/2836667320 >> [v2:172.21.15.203:6832/2418205237,v1:172.21.15.203:6833/2418205237] conn(0x7faf100744e0 0x7faf100769f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:29:09.836 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:09.833+0000 7faf21ffb640 1 -- 172.21.15.37:0/2836667320 <== mon.0 v2:172.21.15.37:3300/0 5 ==== osd_map(22..22 src has 1..22) ==== 5709+0+0 (secure 0 0 0) 0x7faf280b4e40 con 0x7faf18003b70 2024-04-25T10:29:09.836 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:09.833+0000 7faf30ff5640 1 --2- 172.21.15.37:0/2836667320 >> [v2:172.21.15.203:6832/2418205237,v1:172.21.15.203:6833/2418205237] conn(0x7faf100744e0 0x7faf100769f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:29:09.836 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:09.833+0000 7faf30ff5640 1 --2- 172.21.15.37:0/2836667320 >> [v2:172.21.15.203:6832/2418205237,v1:172.21.15.203:6833/2418205237] conn(0x7faf100744e0 0x7faf100769f0 secure :-1 s=READY pgs=60 cs=0 l=1 rev1=1 crypto rx=0x7faf2c1f9510 tx=0x7faf1c001000 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:29:09.839 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:09.837+0000 7faf21ffb640 1 -- 172.21.15.37:0/2836667320 <== mon.0 v2:172.21.15.37:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187246 (secure 0 0 0) 0x7faf28049030 con 0x7faf18003b70 2024-04-25T10:29:10.069 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:10.067+0000 7faf33a5a640 1 -- 172.21.15.37:0/2836667320 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_command({"prefix": "fs set", "fs_name": "cephfs", "var": "standby_count_wanted", "val": "0"} v 0) -- 0x7faf2c076570 con 0x7faf18003b70 2024-04-25T10:29:11.030 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:11.028+0000 7faf21ffb640 1 -- 172.21.15.37:0/2836667320 <== mon.0 v2:172.21.15.37:3300/0 7 ==== mon_command_ack([{"prefix": "fs set", "fs_name": "cephfs", "var": "standby_count_wanted", "val": "0"}]=0 v7) ==== 118+0+0 (secure 0 0 0) 0x7faf2c0ba970 con 0x7faf18003b70 2024-04-25T10:29:11.033 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:11.031+0000 7faf33a5a640 1 -- 172.21.15.37:0/2836667320 >> [v2:172.21.15.203:6832/2418205237,v1:172.21.15.203:6833/2418205237] conn(0x7faf100744e0 msgr2=0x7faf100769f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:29:11.033 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:11.031+0000 7faf33a5a640 1 --2- 172.21.15.37:0/2836667320 >> [v2:172.21.15.203:6832/2418205237,v1:172.21.15.203:6833/2418205237] conn(0x7faf100744e0 0x7faf100769f0 secure :-1 s=READY pgs=60 cs=0 l=1 rev1=1 crypto rx=0x7faf2c1f9510 tx=0x7faf1c001000 comp rx=0 tx=0).stop 2024-04-25T10:29:11.033 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:11.031+0000 7faf33a5a640 1 -- 172.21.15.37:0/2836667320 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7faf18003b70 msgr2=0x7faf2c0ccbf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:29:11.033 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:11.031+0000 7faf33a5a640 1 --2- 172.21.15.37:0/2836667320 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7faf18003b70 0x7faf2c0ccbf0 secure :-1 s=READY pgs=70 cs=0 l=1 rev1=1 crypto rx=0x7faf28001990 tx=0x7faf280012d0 comp rx=0 tx=0).stop 2024-04-25T10:29:11.033 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:11.031+0000 7faf33a5a640 1 -- 172.21.15.37:0/2836667320 shutdown_connections 2024-04-25T10:29:11.033 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:11.031+0000 7faf33a5a640 1 --2- 172.21.15.37:0/2836667320 >> [v2:172.21.15.203:6832/2418205237,v1:172.21.15.203:6833/2418205237] conn(0x7faf100744e0 0x7faf100769f0 unknown :-1 s=CLOSED pgs=60 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:29:11.034 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:11.031+0000 7faf33a5a640 1 --2- 172.21.15.37:0/2836667320 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7faf2c0ba020 0x7faf2c0ba430 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:29:11.034 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:11.031+0000 7faf33a5a640 1 --2- 172.21.15.37:0/2836667320 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7faf2c0ac680 0x7faf2c14ff80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:29:11.034 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:11.031+0000 7faf33a5a640 1 --2- 172.21.15.37:0/2836667320 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7faf18003b70 0x7faf2c0ccbf0 unknown :-1 s=CLOSED pgs=70 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:29:11.034 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:11.031+0000 7faf33a5a640 1 -- 172.21.15.37:0/2836667320 >> 172.21.15.37:0/2836667320 conn(0x7faf2c0d27a0 msgr2=0x7faf2c0d6220 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:29:11.034 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:11.031+0000 7faf33a5a640 1 -- 172.21.15.37:0/2836667320 shutdown_connections 2024-04-25T10:29:11.034 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:11.031+0000 7faf33a5a640 1 -- 172.21.15.37:0/2836667320 wait complete. 2024-04-25T10:29:11.047 DEBUG:tasks.cephfs.filesystem:fs_config: {} 2024-04-25T10:29:11.047 DEBUG:teuthology.orchestra.run.smithi037:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs set cephfs allow_standby_replay false 2024-04-25T10:29:11.167 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:11.165+0000 7f4a05cf2640 1 Processor -- start 2024-04-25T10:29:11.167 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:11.165+0000 7f4a05cf2640 1 -- start start 2024-04-25T10:29:11.169 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:11.166+0000 7f4a05cf2640 1 --2- >> v2:172.21.15.37:3300/0 conn(0x7f4a000d5160 0x7f4a000b4e80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:29:11.169 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:11.167+0000 7f4a05cf2640 1 -- --> v1:172.21.15.37:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f4a000d1370 con 0x7f4a000d0990 2024-04-25T10:29:11.169 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:11.167+0000 7f4a05cf2640 1 -- --> v1:172.21.15.203:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f4a000d4680 con 0x7f4a000d3230 2024-04-25T10:29:11.169 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:11.167+0000 7f4a05cf2640 1 -- --> v2:172.21.15.37:3300/0 -- mon_getmap magic: 0 -- 0x7f4a000d1650 con 0x7f4a000d5160 2024-04-25T10:29:11.169 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:11.167+0000 7f49ff7fe640 1 --2- >> v2:172.21.15.37:3300/0 conn(0x7f4a000d5160 0x7f4a000b4e80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:29:11.169 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:11.167+0000 7f49feffd640 1 --1- >> v1:172.21.15.37:6789/0 conn(0x7f4a000d0990 0x7f4a000d2b20 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.37:6789/0 says I am v1:172.21.15.37:51396/0 (socket says 172.21.15.37:51396) 2024-04-25T10:29:11.169 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:11.167+0000 7f49feffd640 1 -- 172.21.15.37:0/2913254667 learned_addr learned my addr 172.21.15.37:0/2913254667 (peer_addr_for_me v1:172.21.15.37:0/0) 2024-04-25T10:29:11.169 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:11.167+0000 7f49ff7fe640 1 -- 172.21.15.37:0/2913254667 >> v1:172.21.15.203:6789/0 conn(0x7f4a000d3230 legacy=0x7f4a000b9060 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:29:11.169 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:11.167+0000 7f49ff7fe640 1 -- 172.21.15.37:0/2913254667 >> v1:172.21.15.37:6789/0 conn(0x7f4a000d0990 legacy=0x7f4a000d2b20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:29:11.170 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:11.167+0000 7f49ff7fe640 1 -- 172.21.15.37:0/2913254667 --> v2:172.21.15.37:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f4a000d4960 con 0x7f4a000d5160 2024-04-25T10:29:11.171 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:11.168+0000 7f49ff7fe640 1 --2- 172.21.15.37:0/2913254667 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f4a000d5160 0x7f4a000b4e80 secure :-1 s=READY pgs=72 cs=0 l=1 rev1=1 crypto rx=0x7f49f00097a0 tx=0x7f49f002a0b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=bb6681b3200325ce server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:29:11.171 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:11.168+0000 7f49fe7fc640 1 -- 172.21.15.37:0/2913254667 <== mon.0 v2:172.21.15.37:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f49f0037070 con 0x7f4a000d5160 2024-04-25T10:29:11.171 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:11.168+0000 7f49fe7fc640 1 -- 172.21.15.37:0/2913254667 <== mon.0 v2:172.21.15.37:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f49f002adc0 con 0x7f4a000d5160 2024-04-25T10:29:11.171 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:11.168+0000 7f49fe7fc640 1 -- 172.21.15.37:0/2913254667 <== mon.0 v2:172.21.15.37:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f49f0032660 con 0x7f4a000d5160 2024-04-25T10:29:11.171 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:11.169+0000 7f4a05cf2640 1 -- 172.21.15.37:0/2913254667 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f4a000d5160 msgr2=0x7f4a000b4e80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:29:11.171 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:11.169+0000 7f4a05cf2640 1 --2- 172.21.15.37:0/2913254667 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f4a000d5160 0x7f4a000b4e80 secure :-1 s=READY pgs=72 cs=0 l=1 rev1=1 crypto rx=0x7f49f00097a0 tx=0x7f49f002a0b0 comp rx=0 tx=0).stop 2024-04-25T10:29:11.171 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:11.169+0000 7f4a05cf2640 1 -- 172.21.15.37:0/2913254667 shutdown_connections 2024-04-25T10:29:11.171 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:11.169+0000 7f4a05cf2640 1 --2- 172.21.15.37:0/2913254667 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f4a000d5160 0x7f4a000b4e80 unknown :-1 s=CLOSED pgs=72 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:29:11.171 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:11.169+0000 7f4a05cf2640 1 -- 172.21.15.37:0/2913254667 >> 172.21.15.37:0/2913254667 conn(0x7f4a000b7a80 msgr2=0x7f4a000b7e90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:29:11.171 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:11.169+0000 7f4a05cf2640 1 -- 172.21.15.37:0/2913254667 shutdown_connections 2024-04-25T10:29:11.172 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:11.169+0000 7f4a05cf2640 1 -- 172.21.15.37:0/2913254667 wait complete. 2024-04-25T10:29:11.172 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:11.170+0000 7f4a05cf2640 1 Processor -- start 2024-04-25T10:29:11.172 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:11.171+0000 7f4a05cf2640 1 -- start start 2024-04-25T10:29:11.175 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:11.172+0000 7f4a05cf2640 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f4a000d0990 0x7f4a000cea30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:29:11.175 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:11.172+0000 7f4a05cf2640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f4a000d5160 0x7f4a0015a090 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:29:11.175 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:11.172+0000 7f4a05cf2640 1 --2- >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f4a0015a5d0 0x7f4a000ccd00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:29:11.175 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:11.172+0000 7f4a05cf2640 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 -- 0x7f4a00164c40 con 0x7f4a000d5160 2024-04-25T10:29:11.176 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:11.172+0000 7f4a05cf2640 1 -- --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_getmap magic: 0 -- 0x7f4a00164ac0 con 0x7f4a000d0990 2024-04-25T10:29:11.176 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:11.172+0000 7f4a05cf2640 1 -- --> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] -- mon_getmap magic: 0 -- 0x7f4a000d0da0 con 0x7f4a0015a5d0 2024-04-25T10:29:11.176 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:11.172+0000 7f49feffd640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f4a000d5160 0x7f4a0015a090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:29:11.176 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:11.172+0000 7f49feffd640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f4a000d5160 0x7f4a0015a090 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.37:3300/0 says I am v2:172.21.15.37:46252/0 (socket says 172.21.15.37:46252) 2024-04-25T10:29:11.176 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:11.172+0000 7f49feffd640 1 -- 172.21.15.37:0/3781906194 learned_addr learned my addr 172.21.15.37:0/3781906194 (peer_addr_for_me v2:172.21.15.37:0/0) 2024-04-25T10:29:11.176 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:11.172+0000 7f49ff7fe640 1 --2- 172.21.15.37:0/3781906194 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f4a000d0990 0x7f4a000cea30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:29:11.176 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:11.172+0000 7f49fffff640 1 --2- 172.21.15.37:0/3781906194 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f4a0015a5d0 0x7f4a000ccd00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:29:11.176 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:11.172+0000 7f49feffd640 1 -- 172.21.15.37:0/3781906194 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f4a0015a5d0 msgr2=0x7f4a000ccd00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:29:11.176 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:11.172+0000 7f49feffd640 1 --2- 172.21.15.37:0/3781906194 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f4a0015a5d0 0x7f4a000ccd00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:29:11.176 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:11.172+0000 7f49feffd640 1 -- 172.21.15.37:0/3781906194 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f4a000d0990 msgr2=0x7f4a000cea30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:29:11.176 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:11.172+0000 7f49feffd640 1 --2- 172.21.15.37:0/3781906194 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f4a000d0990 0x7f4a000cea30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:29:11.176 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:11.172+0000 7f49feffd640 1 -- 172.21.15.37:0/3781906194 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f4a0015ae30 con 0x7f4a000d5160 2024-04-25T10:29:11.176 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:11.172+0000 7f49feffd640 1 --2- 172.21.15.37:0/3781906194 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f4a000d5160 0x7f4a0015a090 secure :-1 s=READY pgs=73 cs=0 l=1 rev1=1 crypto rx=0x7f49f400c3f0 tx=0x7f49f400c8b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:29:11.176 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:11.173+0000 7f49e7fff640 1 -- 172.21.15.37:0/3781906194 <== mon.0 v2:172.21.15.37:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f49f4017070 con 0x7f4a000d5160 2024-04-25T10:29:11.177 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:11.173+0000 7f49e7fff640 1 -- 172.21.15.37:0/3781906194 <== mon.0 v2:172.21.15.37:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f49f4005ce0 con 0x7f4a000d5160 2024-04-25T10:29:11.177 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:11.173+0000 7f4a05cf2640 1 -- 172.21.15.37:0/3781906194 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f4a0015b110 con 0x7f4a000d5160 2024-04-25T10:29:11.177 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:11.173+0000 7f49e7fff640 1 -- 172.21.15.37:0/3781906194 <== mon.0 v2:172.21.15.37:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f49f40072f0 con 0x7f4a000d5160 2024-04-25T10:29:11.177 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:11.173+0000 7f4a05cf2640 1 -- 172.21.15.37:0/3781906194 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f4a0015b6d0 con 0x7f4a000d5160 2024-04-25T10:29:11.178 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:11.174+0000 7f4a05cf2640 1 -- 172.21.15.37:0/3781906194 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f4a0006a140 con 0x7f4a000d5160 2024-04-25T10:29:11.178 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:11.174+0000 7f49e7fff640 1 -- 172.21.15.37:0/3781906194 <== mon.0 v2:172.21.15.37:3300/0 4 ==== mgrmap(e 5) ==== 96782+0+0 (secure 0 0 0) 0x7f49f401a070 con 0x7f4a000d5160 2024-04-25T10:29:11.178 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:11.175+0000 7f49e7fff640 1 --2- 172.21.15.37:0/3781906194 >> [v2:172.21.15.203:6832/2418205237,v1:172.21.15.203:6833/2418205237] conn(0x7f49ec0744b0 0x7f49ec0769c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:29:11.178 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:11.175+0000 7f49e7fff640 1 -- 172.21.15.37:0/3781906194 <== mon.0 v2:172.21.15.37:3300/0 5 ==== osd_map(23..23 src has 1..23) ==== 5709+0+0 (secure 0 0 0) 0x7f49f4094510 con 0x7f4a000d5160 2024-04-25T10:29:11.178 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:11.175+0000 7f49ff7fe640 1 --2- 172.21.15.37:0/3781906194 >> [v2:172.21.15.203:6832/2418205237,v1:172.21.15.203:6833/2418205237] conn(0x7f49ec0744b0 0x7f49ec0769c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:29:11.178 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:11.176+0000 7f49ff7fe640 1 --2- 172.21.15.37:0/3781906194 >> [v2:172.21.15.203:6832/2418205237,v1:172.21.15.203:6833/2418205237] conn(0x7f49ec0744b0 0x7f49ec0769c0 secure :-1 s=READY pgs=61 cs=0 l=1 rev1=1 crypto rx=0x7f49f0034000 tx=0x7f49f002e000 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:29:11.181 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:11.180+0000 7f49e7fff640 1 -- 172.21.15.37:0/3781906194 <== mon.0 v2:172.21.15.37:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187246 (secure 0 0 0) 0x7f49f405ed30 con 0x7f4a000d5160 2024-04-25T10:29:11.408 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:11.406+0000 7f4a05cf2640 1 -- 172.21.15.37:0/3781906194 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_command({"prefix": "fs set", "fs_name": "cephfs", "var": "allow_standby_replay", "val": "false"} v 0) -- 0x7f4a0015abc0 con 0x7f4a000d5160 2024-04-25T10:29:12.032 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:12.029+0000 7f49e7fff640 1 -- 172.21.15.37:0/3781906194 <== mon.0 v2:172.21.15.37:3300/0 7 ==== mon_command_ack([{"prefix": "fs set", "fs_name": "cephfs", "var": "allow_standby_replay", "val": "false"}]=0 v8) ==== 122+0+0 (secure 0 0 0) 0x7f49f405e630 con 0x7f4a000d5160 2024-04-25T10:29:12.035 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:12.032+0000 7f4a05cf2640 1 -- 172.21.15.37:0/3781906194 >> [v2:172.21.15.203:6832/2418205237,v1:172.21.15.203:6833/2418205237] conn(0x7f49ec0744b0 msgr2=0x7f49ec0769c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:29:12.035 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:12.032+0000 7f4a05cf2640 1 --2- 172.21.15.37:0/3781906194 >> [v2:172.21.15.203:6832/2418205237,v1:172.21.15.203:6833/2418205237] conn(0x7f49ec0744b0 0x7f49ec0769c0 secure :-1 s=READY pgs=61 cs=0 l=1 rev1=1 crypto rx=0x7f49f0034000 tx=0x7f49f002e000 comp rx=0 tx=0).stop 2024-04-25T10:29:12.035 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:12.032+0000 7f4a05cf2640 1 -- 172.21.15.37:0/3781906194 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f4a000d5160 msgr2=0x7f4a0015a090 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:29:12.035 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:12.032+0000 7f4a05cf2640 1 --2- 172.21.15.37:0/3781906194 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f4a000d5160 0x7f4a0015a090 secure :-1 s=READY pgs=73 cs=0 l=1 rev1=1 crypto rx=0x7f49f400c3f0 tx=0x7f49f400c8b0 comp rx=0 tx=0).stop 2024-04-25T10:29:12.035 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:12.032+0000 7f4a05cf2640 1 -- 172.21.15.37:0/3781906194 shutdown_connections 2024-04-25T10:29:12.035 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:12.032+0000 7f4a05cf2640 1 --2- 172.21.15.37:0/3781906194 >> [v2:172.21.15.203:6832/2418205237,v1:172.21.15.203:6833/2418205237] conn(0x7f49ec0744b0 0x7f49ec0769c0 unknown :-1 s=CLOSED pgs=61 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:29:12.035 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:12.032+0000 7f4a05cf2640 1 --2- 172.21.15.37:0/3781906194 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f4a0015a5d0 0x7f4a000ccd00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:29:12.035 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:12.032+0000 7f4a05cf2640 1 --2- 172.21.15.37:0/3781906194 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f4a000d5160 0x7f4a0015a090 unknown :-1 s=CLOSED pgs=73 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:29:12.035 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:12.032+0000 7f4a05cf2640 1 --2- 172.21.15.37:0/3781906194 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f4a000d0990 0x7f4a000cea30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:29:12.035 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:12.032+0000 7f4a05cf2640 1 -- 172.21.15.37:0/3781906194 >> 172.21.15.37:0/3781906194 conn(0x7f4a000b7a80 msgr2=0x7f4a00160890 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:29:12.035 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:12.033+0000 7f4a05cf2640 1 -- 172.21.15.37:0/3781906194 shutdown_connections 2024-04-25T10:29:12.036 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:12.033+0000 7f4a05cf2640 1 -- 172.21.15.37:0/3781906194 wait complete. 2024-04-25T10:29:12.048 DEBUG:tasks.cephfs.filesystem:Not Creating any subvols for filesystem 'cephfs' 2024-04-25T10:29:12.049 DEBUG:teuthology.orchestra.run.smithi037:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-04-25T10:29:12.167 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:12.164+0000 7fd43c06e640 1 Processor -- start 2024-04-25T10:29:12.167 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:12.165+0000 7fd43c06e640 1 -- start start 2024-04-25T10:29:12.169 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:12.166+0000 7fd43c06e640 1 --2- >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7fd43415a9f0 0x7fd43406c540 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:29:12.169 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:12.166+0000 7fd43c06e640 1 --2- >> v2:172.21.15.37:3300/0 conn(0x7fd43406ca80 0x7fd434160910 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:29:12.169 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:12.166+0000 7fd43c06e640 1 -- --> v1:172.21.15.203:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fd43406ce90 con 0x7fd434160e50 2024-04-25T10:29:12.169 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:12.166+0000 7fd43c06e640 1 -- --> v2:172.21.15.37:3300/0 -- mon_getmap magic: 0 -- 0x7fd4340b6870 con 0x7fd43406ca80 2024-04-25T10:29:12.169 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:12.166+0000 7fd43c06e640 1 -- --> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] -- mon_getmap magic: 0 -- 0x7fd4341616d0 con 0x7fd43415a9f0 2024-04-25T10:29:12.169 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:12.166+0000 7fd439609640 1 --2- >> v2:172.21.15.37:3300/0 conn(0x7fd43406ca80 0x7fd434160910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:29:12.169 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:12.166+0000 7fd439609640 1 --2- >> v2:172.21.15.37:3300/0 conn(0x7fd43406ca80 0x7fd434160910 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.37:3300/0 says I am v2:172.21.15.37:46258/0 (socket says 172.21.15.37:46258) 2024-04-25T10:29:12.170 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:12.166+0000 7fd439609640 1 -- 172.21.15.37:0/649915825 learned_addr learned my addr 172.21.15.37:0/649915825 (peer_addr_for_me v2:172.21.15.37:0/0) 2024-04-25T10:29:12.170 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:12.166+0000 7fd439e0a640 1 --2- 172.21.15.37:0/649915825 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7fd43415a9f0 0x7fd43406c540 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:29:12.170 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:12.166+0000 7fd439609640 1 -- 172.21.15.37:0/649915825 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7fd43415a9f0 msgr2=0x7fd43406c540 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:29:12.170 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:12.166+0000 7fd439609640 1 --2- 172.21.15.37:0/649915825 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7fd43415a9f0 0x7fd43406c540 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:29:12.170 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:12.166+0000 7fd439609640 1 -- 172.21.15.37:0/649915825 >> v1:172.21.15.203:6789/0 conn(0x7fd434160e50 legacy=0x7fd434145a30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:29:12.170 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:12.166+0000 7fd439609640 1 -- 172.21.15.37:0/649915825 --> v2:172.21.15.37:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fd434163e00 con 0x7fd43406ca80 2024-04-25T10:29:12.170 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:12.167+0000 7fd439609640 1 --2- 172.21.15.37:0/649915825 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fd43406ca80 0x7fd434160910 secure :-1 s=READY pgs=74 cs=0 l=1 rev1=1 crypto rx=0x7fd428009870 tx=0x7fd42802a210 comp rx=0 tx=0).ready entity=mon.0 client_cookie=84a531c9e03cd459 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:29:12.171 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:12.167+0000 7fd438e08640 1 -- 172.21.15.37:0/649915825 <== mon.0 v2:172.21.15.37:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd428037070 con 0x7fd43406ca80 2024-04-25T10:29:12.171 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:12.167+0000 7fd438e08640 1 -- 172.21.15.37:0/649915825 <== mon.0 v2:172.21.15.37:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fd428032440 con 0x7fd43406ca80 2024-04-25T10:29:12.171 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:12.167+0000 7fd438e08640 1 -- 172.21.15.37:0/649915825 <== mon.0 v2:172.21.15.37:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd428032810 con 0x7fd43406ca80 2024-04-25T10:29:12.171 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:12.168+0000 7fd43c06e640 1 -- 172.21.15.37:0/649915825 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fd43406ca80 msgr2=0x7fd434160910 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:29:12.171 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:12.168+0000 7fd43c06e640 1 --2- 172.21.15.37:0/649915825 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fd43406ca80 0x7fd434160910 secure :-1 s=READY pgs=74 cs=0 l=1 rev1=1 crypto rx=0x7fd428009870 tx=0x7fd42802a210 comp rx=0 tx=0).stop 2024-04-25T10:29:12.171 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:12.168+0000 7fd43c06e640 1 -- 172.21.15.37:0/649915825 shutdown_connections 2024-04-25T10:29:12.172 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:12.168+0000 7fd43c06e640 1 --2- 172.21.15.37:0/649915825 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fd43406ca80 0x7fd434160910 unknown :-1 s=CLOSED pgs=74 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:29:12.172 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:12.168+0000 7fd43c06e640 1 --2- 172.21.15.37:0/649915825 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7fd43415a9f0 0x7fd43406c540 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:29:12.172 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:12.168+0000 7fd43c06e640 1 -- 172.21.15.37:0/649915825 >> 172.21.15.37:0/649915825 conn(0x7fd4340b4560 msgr2=0x7fd4340b4970 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:29:12.172 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:12.168+0000 7fd43c06e640 1 -- 172.21.15.37:0/649915825 shutdown_connections 2024-04-25T10:29:12.172 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:12.168+0000 7fd43c06e640 1 -- 172.21.15.37:0/649915825 wait complete. 2024-04-25T10:29:12.172 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:12.170+0000 7fd43c06e640 1 Processor -- start 2024-04-25T10:29:12.172 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:12.170+0000 7fd43c06e640 1 -- start start 2024-04-25T10:29:12.175 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:12.171+0000 7fd43c06e640 1 --2- >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7fd43406ca80 0x7fd434143410 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:29:12.175 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:12.171+0000 7fd43c06e640 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fd43415a9f0 0x7fd4341f9b80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:29:12.175 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:12.171+0000 7fd43c06e640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fd434160e50 0x7fd434163050 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:29:12.175 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:12.171+0000 7fd43c06e640 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 -- 0x7fd4341646a0 con 0x7fd434160e50 2024-04-25T10:29:12.175 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:12.171+0000 7fd43c06e640 1 -- --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_getmap magic: 0 -- 0x7fd4341643c0 con 0x7fd43415a9f0 2024-04-25T10:29:12.175 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:12.171+0000 7fd43c06e640 1 -- --> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] -- mon_getmap magic: 0 -- 0x7fd434164bc0 con 0x7fd43406ca80 2024-04-25T10:29:12.175 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:12.172+0000 7fd43a60b640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fd434160e50 0x7fd434163050 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:29:12.175 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:12.172+0000 7fd43a60b640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fd434160e50 0x7fd434163050 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.37:3300/0 says I am v2:172.21.15.37:46266/0 (socket says 172.21.15.37:46266) 2024-04-25T10:29:12.175 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:12.172+0000 7fd43a60b640 1 -- 172.21.15.37:0/1225674619 learned_addr learned my addr 172.21.15.37:0/1225674619 (peer_addr_for_me v2:172.21.15.37:0/0) 2024-04-25T10:29:12.175 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:12.172+0000 7fd439609640 1 --2- 172.21.15.37:0/1225674619 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fd43415a9f0 0x7fd4341f9b80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:29:12.175 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:12.172+0000 7fd439e0a640 1 --2- 172.21.15.37:0/1225674619 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7fd43406ca80 0x7fd434143410 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:29:12.175 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:12.172+0000 7fd43a60b640 1 -- 172.21.15.37:0/1225674619 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7fd43406ca80 msgr2=0x7fd434143410 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:29:12.175 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:12.172+0000 7fd43a60b640 1 --2- 172.21.15.37:0/1225674619 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7fd43406ca80 0x7fd434143410 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:29:12.176 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:12.172+0000 7fd43a60b640 1 -- 172.21.15.37:0/1225674619 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fd43415a9f0 msgr2=0x7fd4341f9b80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:29:12.176 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:12.172+0000 7fd43a60b640 1 --2- 172.21.15.37:0/1225674619 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fd43415a9f0 0x7fd4341f9b80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:29:12.176 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:12.172+0000 7fd43a60b640 1 -- 172.21.15.37:0/1225674619 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fd434143950 con 0x7fd434160e50 2024-04-25T10:29:12.176 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:12.172+0000 7fd43a60b640 1 --2- 172.21.15.37:0/1225674619 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fd434160e50 0x7fd434163050 secure :-1 s=READY pgs=75 cs=0 l=1 rev1=1 crypto rx=0x7fd42400b970 tx=0x7fd42400be30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:29:12.176 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:12.172+0000 7fd4327fc640 1 -- 172.21.15.37:0/1225674619 <== mon.0 v2:172.21.15.37:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd424019070 con 0x7fd434160e50 2024-04-25T10:29:12.176 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:12.172+0000 7fd4327fc640 1 -- 172.21.15.37:0/1225674619 <== mon.0 v2:172.21.15.37:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fd42400cb40 con 0x7fd434160e50 2024-04-25T10:29:12.176 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:12.172+0000 7fd43c06e640 1 -- 172.21.15.37:0/1225674619 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fd4341fa0c0 con 0x7fd434160e50 2024-04-25T10:29:12.176 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:12.172+0000 7fd4327fc640 1 -- 172.21.15.37:0/1225674619 <== mon.0 v2:172.21.15.37:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd424014440 con 0x7fd434160e50 2024-04-25T10:29:12.176 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:12.172+0000 7fd43c06e640 1 -- 172.21.15.37:0/1225674619 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fd4341fa8b0 con 0x7fd434160e50 2024-04-25T10:29:12.177 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:12.173+0000 7fd43c06e640 1 -- 172.21.15.37:0/1225674619 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fd4340cfa20 con 0x7fd434160e50 2024-04-25T10:29:12.177 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:12.174+0000 7fd4327fc640 1 -- 172.21.15.37:0/1225674619 <== mon.0 v2:172.21.15.37:3300/0 4 ==== mgrmap(e 5) ==== 96782+0+0 (secure 0 0 0) 0x7fd42401c070 con 0x7fd434160e50 2024-04-25T10:29:12.177 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:12.174+0000 7fd4327fc640 1 --2- 172.21.15.37:0/1225674619 >> [v2:172.21.15.203:6832/2418205237,v1:172.21.15.203:6833/2418205237] conn(0x7fd4200744b0 0x7fd4200769c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:29:12.177 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:12.175+0000 7fd4327fc640 1 -- 172.21.15.37:0/1225674619 <== mon.0 v2:172.21.15.37:3300/0 5 ==== osd_map(23..23 src has 1..23) ==== 5709+0+0 (secure 0 0 0) 0x7fd424094e20 con 0x7fd434160e50 2024-04-25T10:29:12.177 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:12.175+0000 7fd439e0a640 1 --2- 172.21.15.37:0/1225674619 >> [v2:172.21.15.203:6832/2418205237,v1:172.21.15.203:6833/2418205237] conn(0x7fd4200744b0 0x7fd4200769c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:29:12.177 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:12.175+0000 7fd439e0a640 1 --2- 172.21.15.37:0/1225674619 >> [v2:172.21.15.203:6832/2418205237,v1:172.21.15.203:6833/2418205237] conn(0x7fd4200744b0 0x7fd4200769c0 secure :-1 s=READY pgs=62 cs=0 l=1 rev1=1 crypto rx=0x7fd434073140 tx=0x7fd42c008040 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:29:12.181 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:12.179+0000 7fd4327fc640 1 -- 172.21.15.37:0/1225674619 <== mon.0 v2:172.21.15.37:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187246 (secure 0 0 0) 0x7fd4340cfa20 con 0x7fd434160e50 2024-04-25T10:29:12.406 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:12.404+0000 7fd43c06e640 1 -- 172.21.15.37:0/1225674619 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7fd43406a140 con 0x7fd434160e50 2024-04-25T10:29:12.407 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:12.405+0000 7fd4327fc640 1 -- 172.21.15.37:0/1225674619 <== mon.0 v2:172.21.15.37:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 8 v8) ==== 93+0+5178 (secure 0 0 0) 0x7fd434143950 con 0x7fd434160e50 2024-04-25T10:29:12.407 INFO:teuthology.orchestra.run.smithi037.stdout: 2024-04-25T10:29:12.407 INFO:teuthology.orchestra.run.smithi037.stdout:{"epoch":8,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":4232,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.37:6835/1911169480","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.37:6833","nonce":1911169480},{"type":"v1","addr":"172.21.15.37:6835","nonce":1911169480}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3},{"gid":4241,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.203:6835/3873243328","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6834","nonce":3873243328},{"type":"v1","addr":"172.21.15.203:6835","nonce":3873243328}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3},{"gid":4278,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.37:6834/179723013","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.37:6832","nonce":179723013},{"type":"v1","addr":"172.21.15.37:6834","nonce":179723013}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":2}],"filesystems":[{"mdsmap":{"epoch":8,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-04-25T10:29:08.397829+0000","modified":"2024-04-25T10:29:11.407694+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":1,"in":[0],"up":{"mds_0":4279},"failed":[],"damaged":[],"stopped":[],"info":{"gid_4279":{"gid":4279,"name":"d","rank":0,"incarnation":4,"state":"up:active","state_seq":3,"addr":"172.21.15.203:6837/3215191422","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6836","nonce":3215191422},{"type":"v1","addr":"172.21.15.203:6837","nonce":3215191422}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":4279,"qdb_cluster":[4279]},"id":1}]} 2024-04-25T10:29:12.408 INFO:teuthology.orchestra.run.smithi037.stderr:dumped fsmap epoch 8 2024-04-25T10:29:12.409 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:12.407+0000 7fd43c06e640 1 -- 172.21.15.37:0/1225674619 >> [v2:172.21.15.203:6832/2418205237,v1:172.21.15.203:6833/2418205237] conn(0x7fd4200744b0 msgr2=0x7fd4200769c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:29:12.409 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:12.407+0000 7fd43c06e640 1 --2- 172.21.15.37:0/1225674619 >> [v2:172.21.15.203:6832/2418205237,v1:172.21.15.203:6833/2418205237] conn(0x7fd4200744b0 0x7fd4200769c0 secure :-1 s=READY pgs=62 cs=0 l=1 rev1=1 crypto rx=0x7fd434073140 tx=0x7fd42c008040 comp rx=0 tx=0).stop 2024-04-25T10:29:12.409 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:12.407+0000 7fd43c06e640 1 -- 172.21.15.37:0/1225674619 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fd434160e50 msgr2=0x7fd434163050 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:29:12.409 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:12.407+0000 7fd43c06e640 1 --2- 172.21.15.37:0/1225674619 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fd434160e50 0x7fd434163050 secure :-1 s=READY pgs=75 cs=0 l=1 rev1=1 crypto rx=0x7fd42400b970 tx=0x7fd42400be30 comp rx=0 tx=0).stop 2024-04-25T10:29:12.409 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:12.407+0000 7fd43c06e640 1 -- 172.21.15.37:0/1225674619 shutdown_connections 2024-04-25T10:29:12.409 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:12.407+0000 7fd43c06e640 1 --2- 172.21.15.37:0/1225674619 >> [v2:172.21.15.203:6832/2418205237,v1:172.21.15.203:6833/2418205237] conn(0x7fd4200744b0 0x7fd4200769c0 unknown :-1 s=CLOSED pgs=62 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:29:12.410 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:12.407+0000 7fd43c06e640 1 --2- 172.21.15.37:0/1225674619 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fd434160e50 0x7fd434163050 unknown :-1 s=CLOSED pgs=75 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:29:12.410 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:12.407+0000 7fd43c06e640 1 --2- 172.21.15.37:0/1225674619 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fd43415a9f0 0x7fd4341f9b80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:29:12.410 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:12.407+0000 7fd43c06e640 1 --2- 172.21.15.37:0/1225674619 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7fd43406ca80 0x7fd434143410 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:29:12.410 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:12.407+0000 7fd43c06e640 1 -- 172.21.15.37:0/1225674619 >> 172.21.15.37:0/1225674619 conn(0x7fd4340b4560 msgr2=0x7fd4340cf310 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:29:12.410 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:12.407+0000 7fd43c06e640 1 -- 172.21.15.37:0/1225674619 shutdown_connections 2024-04-25T10:29:12.410 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:12.407+0000 7fd43c06e640 1 -- 172.21.15.37:0/1225674619 wait complete. 2024-04-25T10:29:12.420 DEBUG:teuthology.orchestra.run.smithi037:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-04-25T10:29:12.531 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:12.529+0000 7f0770bee640 1 Processor -- start 2024-04-25T10:29:12.531 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:12.529+0000 7f0770bee640 1 -- start start 2024-04-25T10:29:12.533 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:12.530+0000 7f0770bee640 1 --2- >> v2:172.21.15.37:3300/0 conn(0x7f076c08cc90 0x7f076c09cbc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:29:12.534 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:12.530+0000 7f0770bee640 1 --2- >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f076c0b71b0 0x7f076c0b75c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:29:12.534 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:12.530+0000 7f0770bee640 1 -- --> v1:172.21.15.37:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f076c0b8d40 con 0x7f076c0d1c20 2024-04-25T10:29:12.534 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:12.530+0000 7f0770bee640 1 -- --> v2:172.21.15.37:3300/0 -- mon_getmap magic: 0 -- 0x7f076c0d27f0 con 0x7f076c08cc90 2024-04-25T10:29:12.534 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:12.530+0000 7f0770bee640 1 -- --> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] -- mon_getmap magic: 0 -- 0x7f076c0d2ad0 con 0x7f076c0b71b0 2024-04-25T10:29:12.534 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:12.530+0000 7f076a59c640 1 --2- >> v2:172.21.15.37:3300/0 conn(0x7f076c08cc90 0x7f076c09cbc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:29:12.534 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:12.530+0000 7f076a59c640 1 --2- >> v2:172.21.15.37:3300/0 conn(0x7f076c08cc90 0x7f076c09cbc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.37:3300/0 says I am v2:172.21.15.37:46276/0 (socket says 172.21.15.37:46276) 2024-04-25T10:29:12.534 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:12.530+0000 7f076a59c640 1 -- 172.21.15.37:0/3958762063 learned_addr learned my addr 172.21.15.37:0/3958762063 (peer_addr_for_me v2:172.21.15.37:0/0) 2024-04-25T10:29:12.534 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:12.530+0000 7f0763fff640 1 --1- 172.21.15.37:0/3958762063 >> v1:172.21.15.37:6789/0 conn(0x7f076c0d1c20 0x7f076c0d2030 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.37:6789/0 says I am v1:172.21.15.37:51402/0 (socket says 172.21.15.37:51402) 2024-04-25T10:29:12.534 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:12.531+0000 7f076ad9d640 1 --2- 172.21.15.37:0/3958762063 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f076c0b71b0 0x7f076c0b75c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:29:12.534 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:12.531+0000 7f076a59c640 1 -- 172.21.15.37:0/3958762063 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f076c0b71b0 msgr2=0x7f076c0b75c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:29:12.534 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:12.531+0000 7f076a59c640 1 --2- 172.21.15.37:0/3958762063 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f076c0b71b0 0x7f076c0b75c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:29:12.534 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:12.531+0000 7f076a59c640 1 -- 172.21.15.37:0/3958762063 >> v1:172.21.15.37:6789/0 conn(0x7f076c0d1c20 legacy=0x7f076c0d2030 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:29:12.535 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:12.531+0000 7f0769d9b640 1 -- 172.21.15.37:0/3958762063 <== mon.0 v1:172.21.15.37:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 844943848 0 0) 0x7f076c0b8d40 con 0x7f076c0d1c20 2024-04-25T10:29:12.535 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:12.531+0000 7f076a59c640 1 -- 172.21.15.37:0/3958762063 --> v2:172.21.15.37:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f076c0cf7a0 con 0x7f076c08cc90 2024-04-25T10:29:12.535 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:12.531+0000 7f076a59c640 1 --2- 172.21.15.37:0/3958762063 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f076c08cc90 0x7f076c09cbc0 secure :-1 s=READY pgs=77 cs=0 l=1 rev1=1 crypto rx=0x7f075c009870 tx=0x7f075c02a270 comp rx=0 tx=0).ready entity=mon.0 client_cookie=27a2c7c64148bd30 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:29:12.535 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:12.532+0000 7f0769d9b640 1 -- 172.21.15.37:0/3958762063 <== mon.0 v2:172.21.15.37:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f075c037070 con 0x7f076c08cc90 2024-04-25T10:29:12.535 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:12.532+0000 7f0769d9b640 1 -- 172.21.15.37:0/3958762063 <== mon.0 v2:172.21.15.37:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f075c032440 con 0x7f076c08cc90 2024-04-25T10:29:12.535 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:12.532+0000 7f0769d9b640 1 -- 172.21.15.37:0/3958762063 <== mon.0 v2:172.21.15.37:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f075c032810 con 0x7f076c08cc90 2024-04-25T10:29:12.536 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:12.532+0000 7f0770bee640 1 -- 172.21.15.37:0/3958762063 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f076c08cc90 msgr2=0x7f076c09cbc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:29:12.536 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:12.532+0000 7f0770bee640 1 --2- 172.21.15.37:0/3958762063 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f076c08cc90 0x7f076c09cbc0 secure :-1 s=READY pgs=77 cs=0 l=1 rev1=1 crypto rx=0x7f075c009870 tx=0x7f075c02a270 comp rx=0 tx=0).stop 2024-04-25T10:29:12.536 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:12.532+0000 7f0770bee640 1 -- 172.21.15.37:0/3958762063 shutdown_connections 2024-04-25T10:29:12.536 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:12.532+0000 7f0770bee640 1 --2- 172.21.15.37:0/3958762063 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f076c0b71b0 0x7f076c0b75c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:29:12.536 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:12.532+0000 7f0770bee640 1 --2- 172.21.15.37:0/3958762063 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f076c08cc90 0x7f076c09cbc0 unknown :-1 s=CLOSED pgs=77 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:29:12.536 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:12.532+0000 7f0770bee640 1 -- 172.21.15.37:0/3958762063 >> 172.21.15.37:0/3958762063 conn(0x7f076c0d03d0 msgr2=0x7f076c0d07e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:29:12.536 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:12.532+0000 7f0770bee640 1 -- 172.21.15.37:0/3958762063 shutdown_connections 2024-04-25T10:29:12.536 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:12.532+0000 7f0770bee640 1 -- 172.21.15.37:0/3958762063 wait complete. 2024-04-25T10:29:12.537 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:12.534+0000 7f0770bee640 1 Processor -- start 2024-04-25T10:29:12.537 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:12.534+0000 7f0770bee640 1 -- start start 2024-04-25T10:29:12.539 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:12.536+0000 7f0770bee640 1 --2- >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f076c0b71b0 0x7f076c0b0500 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:29:12.539 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:12.536+0000 7f0770bee640 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f076c0d1c20 0x7f076c1e5ab0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:29:12.539 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:12.536+0000 7f0770bee640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f076c0c89b0 0x7f076c0c8dc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:29:12.539 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:12.536+0000 7f0770bee640 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 -- 0x7f076c0b7b00 con 0x7f076c0c89b0 2024-04-25T10:29:12.539 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:12.536+0000 7f0770bee640 1 -- --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_getmap magic: 0 -- 0x7f076c0b93b0 con 0x7f076c0d1c20 2024-04-25T10:29:12.539 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:12.536+0000 7f0770bee640 1 -- --> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] -- mon_getmap magic: 0 -- 0x7f076c0b7c80 con 0x7f076c0b71b0 2024-04-25T10:29:12.539 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:12.536+0000 7f076ad9d640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f076c0c89b0 0x7f076c0c8dc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:29:12.539 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:12.536+0000 7f076ad9d640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f076c0c89b0 0x7f076c0c8dc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.37:3300/0 says I am v2:172.21.15.37:46280/0 (socket says 172.21.15.37:46280) 2024-04-25T10:29:12.539 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:12.536+0000 7f076ad9d640 1 -- 172.21.15.37:0/1278566214 learned_addr learned my addr 172.21.15.37:0/1278566214 (peer_addr_for_me v2:172.21.15.37:0/0) 2024-04-25T10:29:12.539 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:12.536+0000 7f0763fff640 1 --2- 172.21.15.37:0/1278566214 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f076c0d1c20 0x7f076c1e5ab0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:29:12.540 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:12.536+0000 7f076a59c640 1 --2- 172.21.15.37:0/1278566214 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f076c0b71b0 0x7f076c0b0500 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:29:12.540 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:12.536+0000 7f076ad9d640 1 -- 172.21.15.37:0/1278566214 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f076c0b71b0 msgr2=0x7f076c0b0500 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:29:12.540 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:12.536+0000 7f076ad9d640 1 --2- 172.21.15.37:0/1278566214 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f076c0b71b0 0x7f076c0b0500 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:29:12.540 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:12.536+0000 7f076ad9d640 1 -- 172.21.15.37:0/1278566214 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f076c0d1c20 msgr2=0x7f076c1e5ab0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:29:12.540 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:12.536+0000 7f076ad9d640 1 --2- 172.21.15.37:0/1278566214 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f076c0d1c20 0x7f076c1e5ab0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:29:12.540 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:12.536+0000 7f076ad9d640 1 -- 172.21.15.37:0/1278566214 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f076c0c9300 con 0x7f076c0c89b0 2024-04-25T10:29:12.540 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:12.536+0000 7f076ad9d640 1 --2- 172.21.15.37:0/1278566214 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f076c0c89b0 0x7f076c0c8dc0 secure :-1 s=READY pgs=78 cs=0 l=1 rev1=1 crypto rx=0x7f0764007e90 tx=0x7f076400e4b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:29:12.540 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:12.536+0000 7f0762ffd640 1 -- 172.21.15.37:0/1278566214 <== mon.0 v2:172.21.15.37:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0764014420 con 0x7f076c0c89b0 2024-04-25T10:29:12.540 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:12.536+0000 7f0762ffd640 1 -- 172.21.15.37:0/1278566214 <== mon.0 v2:172.21.15.37:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f0764019070 con 0x7f076c0c89b0 2024-04-25T10:29:12.540 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:12.536+0000 7f0762ffd640 1 -- 172.21.15.37:0/1278566214 <== mon.0 v2:172.21.15.37:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0764014800 con 0x7f076c0c89b0 2024-04-25T10:29:12.540 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:12.536+0000 7f0770bee640 1 -- 172.21.15.37:0/1278566214 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f076c0a05a0 con 0x7f076c0c89b0 2024-04-25T10:29:12.540 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:12.536+0000 7f0770bee640 1 -- 172.21.15.37:0/1278566214 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f076c09e6c0 con 0x7f076c0c89b0 2024-04-25T10:29:12.541 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:12.538+0000 7f0770bee640 1 -- 172.21.15.37:0/1278566214 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f076c0b2000 con 0x7f076c0c89b0 2024-04-25T10:29:12.541 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:12.539+0000 7f0762ffd640 1 -- 172.21.15.37:0/1278566214 <== mon.0 v2:172.21.15.37:3300/0 4 ==== mgrmap(e 5) ==== 96782+0+0 (secure 0 0 0) 0x7f07640149a0 con 0x7f076c0c89b0 2024-04-25T10:29:12.541 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:12.539+0000 7f0762ffd640 1 --2- 172.21.15.37:0/1278566214 >> [v2:172.21.15.203:6832/2418205237,v1:172.21.15.203:6833/2418205237] conn(0x7f074c0744b0 0x7f074c0769c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:29:12.541 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:12.539+0000 7f0762ffd640 1 -- 172.21.15.37:0/1278566214 <== mon.0 v2:172.21.15.37:3300/0 5 ==== osd_map(23..23 src has 1..23) ==== 5709+0+0 (secure 0 0 0) 0x7f0764096750 con 0x7f076c0c89b0 2024-04-25T10:29:12.541 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:12.539+0000 7f076a59c640 1 --2- 172.21.15.37:0/1278566214 >> [v2:172.21.15.203:6832/2418205237,v1:172.21.15.203:6833/2418205237] conn(0x7f074c0744b0 0x7f074c0769c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:29:12.542 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:12.540+0000 7f076a59c640 1 --2- 172.21.15.37:0/1278566214 >> [v2:172.21.15.203:6832/2418205237,v1:172.21.15.203:6833/2418205237] conn(0x7f074c0744b0 0x7f074c0769c0 secure :-1 s=READY pgs=63 cs=0 l=1 rev1=1 crypto rx=0x7f075c034000 tx=0x7f075c02e000 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:29:12.545 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:12.543+0000 7f0762ffd640 1 -- 172.21.15.37:0/1278566214 <== mon.0 v2:172.21.15.37:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187246 (secure 0 0 0) 0x7f076c0c9300 con 0x7f076c0c89b0 2024-04-25T10:29:12.732 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:12.729+0000 7f0770bee640 1 -- 172.21.15.37:0/1278566214 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7f076c09e140 con 0x7f076c0c89b0 2024-04-25T10:29:12.860 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:12.731+0000 7f0762ffd640 1 -- 172.21.15.37:0/1278566214 <== mon.0 v2:172.21.15.37:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v23) ==== 74+0+16621 (secure 0 0 0) 0x7f0764060970 con 0x7f076c0c89b0 2024-04-25T10:29:12.861 INFO:teuthology.orchestra.run.smithi037.stdout: 2024-04-25T10:29:12.861 INFO:teuthology.orchestra.run.smithi037.stdout:{"epoch":23,"fsid":"f4acda28-be34-492a-b286-994afd36f61d","created":"2024-04-25T10:28:43.985718+0000","modified":"2024-04-25T10:29:10.016188+0000","last_up_change":"2024-04-25T10:28:54.900654+0000","last_in_change":"2024-04-25T10:28:46.216973+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":3,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-04-25T10:28:56.202153+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"19","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2024-04-25T10:29:06.507244+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"22","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":2.25,"score_stable":2.25,"optimal_score":1,"raw_score_acting":2.25,"raw_score_stable":2.25,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2024-04-25T10:29:07.347844+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"23","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"a2f79028-b332-4166-9ebb-a7024a37ea22","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":21,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.37:6816","nonce":2746074720},{"type":"v1","addr":"172.21.15.37:6817","nonce":2746074720}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.37:6818","nonce":2746074720},{"type":"v1","addr":"172.21.15.37:6819","nonce":2746074720}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.37:6822","nonce":2746074720},{"type":"v1","addr":"172.21.15.37:6823","nonce":2746074720}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.37:6820","nonce":2746074720},{"type":"v1","addr":"172.21.15.37:6821","nonce":2746074720}]},"public_addr":"172.21.15.37:6817/2746074720","cluster_addr":"172.21.15.37:6819/2746074720","heartbeat_back_addr":"172.21.15.37:6823/2746074720","heartbeat_front_addr":"172.21.15.37:6821/2746074720","state":["exists","up"]},{"osd":1,"uuid":"973159de-c2ea-4083-aecc-69373d1c1132","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":21,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.37:6800","nonce":1690341662},{"type":"v1","addr":"172.21.15.37:6801","nonce":1690341662}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.37:6802","nonce":1690341662},{"type":"v1","addr":"172.21.15.37:6803","nonce":1690341662}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.37:6806","nonce":1690341662},{"type":"v1","addr":"172.21.15.37:6807","nonce":1690341662}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.37:6804","nonce":1690341662},{"type":"v1","addr":"172.21.15.37:6805","nonce":1690341662}]},"public_addr":"172.21.15.37:6801/1690341662","cluster_addr":"172.21.15.37:6803/1690341662","heartbeat_back_addr":"172.21.15.37:6807/1690341662","heartbeat_front_addr":"172.21.15.37:6805/1690341662","state":["exists","up"]},{"osd":2,"uuid":"4ae7a0e9-cb3f-484c-a75f-f3766b721c47","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":21,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.37:6808","nonce":2754669649},{"type":"v1","addr":"172.21.15.37:6809","nonce":2754669649}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.37:6810","nonce":2754669649},{"type":"v1","addr":"172.21.15.37:6811","nonce":2754669649}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.37:6814","nonce":2754669649},{"type":"v1","addr":"172.21.15.37:6815","nonce":2754669649}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.37:6812","nonce":2754669649},{"type":"v1","addr":"172.21.15.37:6813","nonce":2754669649}]},"public_addr":"172.21.15.37:6809/2754669649","cluster_addr":"172.21.15.37:6811/2754669649","heartbeat_back_addr":"172.21.15.37:6815/2754669649","heartbeat_front_addr":"172.21.15.37:6813/2754669649","state":["exists","up"]},{"osd":3,"uuid":"1ba4a746-1608-47a7-ace2-960a5cfae934","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":21,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.37:6824","nonce":1815290618},{"type":"v1","addr":"172.21.15.37:6825","nonce":1815290618}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.37:6826","nonce":1815290618},{"type":"v1","addr":"172.21.15.37:6827","nonce":1815290618}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.37:6830","nonce":1815290618},{"type":"v1","addr":"172.21.15.37:6831","nonce":1815290618}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.37:6828","nonce":1815290618},{"type":"v1","addr":"172.21.15.37:6829","nonce":1815290618}]},"public_addr":"172.21.15.37:6825/1815290618","cluster_addr":"172.21.15.37:6827/1815290618","heartbeat_back_addr":"172.21.15.37:6831/1815290618","heartbeat_front_addr":"172.21.15.37:6829/1815290618","state":["exists","up"]},{"osd":4,"uuid":"d4f59f5d-0413-4b79-87ee-bb212c1fcf62","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":21,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6800","nonce":4051532063},{"type":"v1","addr":"172.21.15.203:6801","nonce":4051532063}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6802","nonce":4051532063},{"type":"v1","addr":"172.21.15.203:6803","nonce":4051532063}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6806","nonce":4051532063},{"type":"v1","addr":"172.21.15.203:6807","nonce":4051532063}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6804","nonce":4051532063},{"type":"v1","addr":"172.21.15.203:6805","nonce":4051532063}]},"public_addr":"172.21.15.203:6801/4051532063","cluster_addr":"172.21.15.203:6803/4051532063","heartbeat_back_addr":"172.21.15.203:6807/4051532063","heartbeat_front_addr":"172.21.15.203:6805/4051532063","state":["exists","up"]},{"osd":5,"uuid":"69710a0e-41d8-4e7f-9a10-17ce540b67be","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":21,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6824","nonce":1203822890},{"type":"v1","addr":"172.21.15.203:6825","nonce":1203822890}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6826","nonce":1203822890},{"type":"v1","addr":"172.21.15.203:6827","nonce":1203822890}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6830","nonce":1203822890},{"type":"v1","addr":"172.21.15.203:6831","nonce":1203822890}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6828","nonce":1203822890},{"type":"v1","addr":"172.21.15.203:6829","nonce":1203822890}]},"public_addr":"172.21.15.203:6825/1203822890","cluster_addr":"172.21.15.203:6827/1203822890","heartbeat_back_addr":"172.21.15.203:6831/1203822890","heartbeat_front_addr":"172.21.15.203:6829/1203822890","state":["exists","up"]},{"osd":6,"uuid":"f29dd047-3b48-473d-a235-6642a6293f20","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":21,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6810","nonce":2309615014},{"type":"v1","addr":"172.21.15.203:6812","nonce":2309615014}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6814","nonce":2309615014},{"type":"v1","addr":"172.21.15.203:6816","nonce":2309615014}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6822","nonce":2309615014},{"type":"v1","addr":"172.21.15.203:6823","nonce":2309615014}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6818","nonce":2309615014},{"type":"v1","addr":"172.21.15.203:6820","nonce":2309615014}]},"public_addr":"172.21.15.203:6812/2309615014","cluster_addr":"172.21.15.203:6816/2309615014","heartbeat_back_addr":"172.21.15.203:6823/2309615014","heartbeat_front_addr":"172.21.15.203:6820/2309615014","state":["exists","up"]},{"osd":7,"uuid":"6e691644-378b-4fdc-8e86-fb5f56e31f29","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":21,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6808","nonce":3688099014},{"type":"v1","addr":"172.21.15.203:6809","nonce":3688099014}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6811","nonce":3688099014},{"type":"v1","addr":"172.21.15.203:6813","nonce":3688099014}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6819","nonce":3688099014},{"type":"v1","addr":"172.21.15.203:6821","nonce":3688099014}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6815","nonce":3688099014},{"type":"v1","addr":"172.21.15.203:6817","nonce":3688099014}]},"public_addr":"172.21.15.203:6809/3688099014","cluster_addr":"172.21.15.203:6813/3688099014","heartbeat_back_addr":"172.21.15.203:6821/3688099014","heartbeat_front_addr":"172.21.15.203:6817/3688099014","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T10:28:51.419141+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T10:28:51.366448+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T10:28:51.348276+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T10:28:51.380519+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T10:28:51.105812+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T10:28:51.384496+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T10:28:51.388755+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T10:28:51.366958+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-04-25T10:29:12.862 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:12.733+0000 7f0770bee640 1 -- 172.21.15.37:0/1278566214 >> [v2:172.21.15.203:6832/2418205237,v1:172.21.15.203:6833/2418205237] conn(0x7f074c0744b0 msgr2=0x7f074c0769c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:29:12.863 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:12.733+0000 7f0770bee640 1 --2- 172.21.15.37:0/1278566214 >> [v2:172.21.15.203:6832/2418205237,v1:172.21.15.203:6833/2418205237] conn(0x7f074c0744b0 0x7f074c0769c0 secure :-1 s=READY pgs=63 cs=0 l=1 rev1=1 crypto rx=0x7f075c034000 tx=0x7f075c02e000 comp rx=0 tx=0).stop 2024-04-25T10:29:12.863 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:12.733+0000 7f0770bee640 1 -- 172.21.15.37:0/1278566214 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f076c0c89b0 msgr2=0x7f076c0c8dc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:29:12.863 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:12.733+0000 7f0770bee640 1 --2- 172.21.15.37:0/1278566214 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f076c0c89b0 0x7f076c0c8dc0 secure :-1 s=READY pgs=78 cs=0 l=1 rev1=1 crypto rx=0x7f0764007e90 tx=0x7f076400e4b0 comp rx=0 tx=0).stop 2024-04-25T10:29:12.863 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:12.733+0000 7f0770bee640 1 -- 172.21.15.37:0/1278566214 shutdown_connections 2024-04-25T10:29:12.863 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:12.733+0000 7f0770bee640 1 --2- 172.21.15.37:0/1278566214 >> [v2:172.21.15.203:6832/2418205237,v1:172.21.15.203:6833/2418205237] conn(0x7f074c0744b0 0x7f074c0769c0 unknown :-1 s=CLOSED pgs=63 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:29:12.863 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:12.733+0000 7f0770bee640 1 --2- 172.21.15.37:0/1278566214 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f076c0c89b0 0x7f076c0c8dc0 unknown :-1 s=CLOSED pgs=78 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:29:12.863 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:12.733+0000 7f0770bee640 1 --2- 172.21.15.37:0/1278566214 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f076c0d1c20 0x7f076c1e5ab0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:29:12.863 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:12.733+0000 7f0770bee640 1 --2- 172.21.15.37:0/1278566214 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f076c0b71b0 0x7f076c0b0500 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:29:12.863 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:12.733+0000 7f0770bee640 1 -- 172.21.15.37:0/1278566214 >> 172.21.15.37:0/1278566214 conn(0x7f076c0d03d0 msgr2=0x7f076c0cca10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:29:12.863 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:12.733+0000 7f0770bee640 1 -- 172.21.15.37:0/1278566214 shutdown_connections 2024-04-25T10:29:12.863 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:12.733+0000 7f0770bee640 1 -- 172.21.15.37:0/1278566214 wait complete. 2024-04-25T10:29:12.864 INFO:tasks.cephfs.filesystem.ceph_manager:waiting for clean 2024-04-25T10:29:12.864 DEBUG:teuthology.orchestra.run.smithi037:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-04-25T10:29:12.906 INFO:tasks.ceph.mds.d.smithi203.stderr:2024-04-25T10:29:12.904+0000 7fa45031c640 -1 mds.pinger is_rank_lagging: rank=0 was never sent ping request. 2024-04-25T10:29:12.975 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:12.973+0000 7f09d4e84640 1 Processor -- start 2024-04-25T10:29:12.975 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:12.973+0000 7f09d4e84640 1 -- start start 2024-04-25T10:29:12.977 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:12.974+0000 7f09d4e84640 1 --2- >> v2:172.21.15.37:3300/0 conn(0x7f09d00b7cd0 0x7f09d0160a00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:29:12.977 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:12.974+0000 7f09d4e84640 1 --2- >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f09d0160f40 0x7f09d0161880 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:29:12.977 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:12.974+0000 7f09d4e84640 1 -- --> v1:172.21.15.203:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f09d00d5840 con 0x7f09d00d5430 2024-04-25T10:29:12.977 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:12.974+0000 7f09d4e84640 1 -- --> v2:172.21.15.37:3300/0 -- mon_getmap magic: 0 -- 0x7f09d00d4420 con 0x7f09d00b7cd0 2024-04-25T10:29:12.977 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:12.974+0000 7f09d4e84640 1 -- --> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] -- mon_getmap magic: 0 -- 0x7f09d00d4670 con 0x7f09d0160f40 2024-04-25T10:29:12.977 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:12.975+0000 7f09cdd9b640 1 --2- >> v2:172.21.15.37:3300/0 conn(0x7f09d00b7cd0 0x7f09d0160a00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:29:12.977 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:12.975+0000 7f09cdd9b640 1 --2- >> v2:172.21.15.37:3300/0 conn(0x7f09d00b7cd0 0x7f09d0160a00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.37:3300/0 says I am v2:172.21.15.37:46290/0 (socket says 172.21.15.37:46290) 2024-04-25T10:29:12.977 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:12.975+0000 7f09cdd9b640 1 -- 172.21.15.37:0/3095089495 learned_addr learned my addr 172.21.15.37:0/3095089495 (peer_addr_for_me v2:172.21.15.37:0/0) 2024-04-25T10:29:12.978 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:12.975+0000 7f09ced9d640 1 --2- 172.21.15.37:0/3095089495 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f09d0160f40 0x7f09d0161880 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:29:12.978 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:12.975+0000 7f09cdd9b640 1 -- 172.21.15.37:0/3095089495 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f09d0160f40 msgr2=0x7f09d0161880 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:29:12.978 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:12.975+0000 7f09cdd9b640 1 --2- 172.21.15.37:0/3095089495 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f09d0160f40 0x7f09d0161880 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:29:12.978 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:12.975+0000 7f09cdd9b640 1 -- 172.21.15.37:0/3095089495 >> v1:172.21.15.203:6789/0 conn(0x7f09d00d5430 legacy=0x7f09d00b75c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:29:12.978 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:12.975+0000 7f09cdd9b640 1 -- 172.21.15.37:0/3095089495 --> v2:172.21.15.37:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f09d00d4950 con 0x7f09d00b7cd0 2024-04-25T10:29:12.979 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:12.976+0000 7f09cdd9b640 1 --2- 172.21.15.37:0/3095089495 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f09d00b7cd0 0x7f09d0160a00 secure :-1 s=READY pgs=79 cs=0 l=1 rev1=1 crypto rx=0x7f09b8009870 tx=0x7f09b802a1c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a57f993ea24ffe84 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:29:12.979 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:12.976+0000 7f09cd59a640 1 -- 172.21.15.37:0/3095089495 <== mon.0 v2:172.21.15.37:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f09b8037070 con 0x7f09d00b7cd0 2024-04-25T10:29:12.979 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:12.976+0000 7f09cd59a640 1 -- 172.21.15.37:0/3095089495 <== mon.0 v2:172.21.15.37:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f09b8032440 con 0x7f09d00b7cd0 2024-04-25T10:29:12.979 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:12.976+0000 7f09cd59a640 1 -- 172.21.15.37:0/3095089495 <== mon.0 v2:172.21.15.37:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f09b8032810 con 0x7f09d00b7cd0 2024-04-25T10:29:12.979 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:12.976+0000 7f09d4e84640 1 -- 172.21.15.37:0/3095089495 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f09d00b7cd0 msgr2=0x7f09d0160a00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:29:12.979 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:12.976+0000 7f09d4e84640 1 --2- 172.21.15.37:0/3095089495 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f09d00b7cd0 0x7f09d0160a00 secure :-1 s=READY pgs=79 cs=0 l=1 rev1=1 crypto rx=0x7f09b8009870 tx=0x7f09b802a1c0 comp rx=0 tx=0).stop 2024-04-25T10:29:12.980 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:12.976+0000 7f09d4e84640 1 -- 172.21.15.37:0/3095089495 shutdown_connections 2024-04-25T10:29:12.980 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:12.976+0000 7f09d4e84640 1 --2- 172.21.15.37:0/3095089495 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f09d0160f40 0x7f09d0161880 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:29:12.980 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:12.976+0000 7f09d4e84640 1 --2- 172.21.15.37:0/3095089495 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f09d00b7cd0 0x7f09d0160a00 unknown :-1 s=CLOSED pgs=79 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:29:12.980 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:12.977+0000 7f09d4e84640 1 -- 172.21.15.37:0/3095089495 >> 172.21.15.37:0/3095089495 conn(0x7f09d00d27a0 msgr2=0x7f09d00d2bb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:29:12.980 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:12.977+0000 7f09d4e84640 1 -- 172.21.15.37:0/3095089495 shutdown_connections 2024-04-25T10:29:12.980 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:12.977+0000 7f09d4e84640 1 -- 172.21.15.37:0/3095089495 wait complete. 2024-04-25T10:29:12.980 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:12.978+0000 7f09d4e84640 1 Processor -- start 2024-04-25T10:29:12.980 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:12.978+0000 7f09d4e84640 1 -- start start 2024-04-25T10:29:12.985 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:12.980+0000 7f09d4e84640 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f09d00d5430 0x7f09d01f5dc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:29:12.985 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:12.980+0000 7f09d4e84640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f09d0160f40 0x7f09d015b950 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:29:12.985 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:12.980+0000 7f09d4e84640 1 --2- >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f09d015be90 0x7f09d0146310 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:29:12.985 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:12.980+0000 7f09d4e84640 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 -- 0x7f09d0161350 con 0x7f09d0160f40 2024-04-25T10:29:12.985 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:12.980+0000 7f09d4e84640 1 -- --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_getmap magic: 0 -- 0x7f09d00d4f10 con 0x7f09d00d5430 2024-04-25T10:29:12.985 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:12.980+0000 7f09d4e84640 1 -- --> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] -- mon_getmap magic: 0 -- 0x7f09d01614d0 con 0x7f09d015be90 2024-04-25T10:29:12.985 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:12.980+0000 7f09cdd9b640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f09d0160f40 0x7f09d015b950 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:29:12.986 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:12.980+0000 7f09cdd9b640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f09d0160f40 0x7f09d015b950 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.37:3300/0 says I am v2:172.21.15.37:46296/0 (socket says 172.21.15.37:46296) 2024-04-25T10:29:12.986 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:12.980+0000 7f09cdd9b640 1 -- 172.21.15.37:0/1675636704 learned_addr learned my addr 172.21.15.37:0/1675636704 (peer_addr_for_me v2:172.21.15.37:0/0) 2024-04-25T10:29:12.986 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:12.980+0000 7f09ce59c640 1 --2- 172.21.15.37:0/1675636704 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f09d00d5430 0x7f09d01f5dc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:29:12.986 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:12.980+0000 7f09ced9d640 1 --2- 172.21.15.37:0/1675636704 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f09d015be90 0x7f09d0146310 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:29:12.986 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:12.980+0000 7f09cdd9b640 1 -- 172.21.15.37:0/1675636704 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f09d015be90 msgr2=0x7f09d0146310 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:29:12.986 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:12.980+0000 7f09cdd9b640 1 --2- 172.21.15.37:0/1675636704 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f09d015be90 0x7f09d0146310 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:29:12.986 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:12.980+0000 7f09cdd9b640 1 -- 172.21.15.37:0/1675636704 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f09d00d5430 msgr2=0x7f09d01f5dc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:29:12.986 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:12.980+0000 7f09cdd9b640 1 --2- 172.21.15.37:0/1675636704 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f09d00d5430 0x7f09d01f5dc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:29:12.986 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:12.980+0000 7f09cdd9b640 1 -- 172.21.15.37:0/1675636704 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f09d015c2a0 con 0x7f09d0160f40 2024-04-25T10:29:12.986 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:12.980+0000 7f09cdd9b640 1 --2- 172.21.15.37:0/1675636704 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f09d0160f40 0x7f09d015b950 secure :-1 s=READY pgs=80 cs=0 l=1 rev1=1 crypto rx=0x7f09b8000c00 tx=0x7f09b80046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:29:12.986 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:12.980+0000 7f09c6ffd640 1 -- 172.21.15.37:0/1675636704 <== mon.0 v2:172.21.15.37:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f09b8041020 con 0x7f09d0160f40 2024-04-25T10:29:12.987 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:12.980+0000 7f09c6ffd640 1 -- 172.21.15.37:0/1675636704 <== mon.0 v2:172.21.15.37:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f09b80306c0 con 0x7f09d0160f40 2024-04-25T10:29:12.987 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:12.981+0000 7f09d4e84640 1 -- 172.21.15.37:0/1675636704 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f09d01f6760 con 0x7f09d0160f40 2024-04-25T10:29:12.987 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:12.981+0000 7f09c6ffd640 1 -- 172.21.15.37:0/1675636704 <== mon.0 v2:172.21.15.37:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f09b8037040 con 0x7f09d0160f40 2024-04-25T10:29:12.987 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:12.981+0000 7f09d4e84640 1 -- 172.21.15.37:0/1675636704 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f09d01f6d20 con 0x7f09d0160f40 2024-04-25T10:29:12.987 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:12.982+0000 7f09d4e84640 1 -- 172.21.15.37:0/1675636704 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f09d006a140 con 0x7f09d0160f40 2024-04-25T10:29:12.987 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:12.982+0000 7f09c6ffd640 1 -- 172.21.15.37:0/1675636704 <== mon.0 v2:172.21.15.37:3300/0 4 ==== mgrmap(e 5) ==== 96782+0+0 (secure 0 0 0) 0x7f09b8044020 con 0x7f09d0160f40 2024-04-25T10:29:12.987 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:12.983+0000 7f09c6ffd640 1 --2- 172.21.15.37:0/1675636704 >> [v2:172.21.15.203:6832/2418205237,v1:172.21.15.203:6833/2418205237] conn(0x7f09b00744b0 0x7f09b00769c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:29:12.987 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:12.983+0000 7f09c6ffd640 1 -- 172.21.15.37:0/1675636704 <== mon.0 v2:172.21.15.37:3300/0 5 ==== osd_map(23..23 src has 1..23) ==== 5709+0+0 (secure 0 0 0) 0x7f09b80b59c0 con 0x7f09d0160f40 2024-04-25T10:29:12.987 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:12.983+0000 7f09ce59c640 1 --2- 172.21.15.37:0/1675636704 >> [v2:172.21.15.203:6832/2418205237,v1:172.21.15.203:6833/2418205237] conn(0x7f09b00744b0 0x7f09b00769c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:29:12.988 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:12.983+0000 7f09ce59c640 1 --2- 172.21.15.37:0/1675636704 >> [v2:172.21.15.203:6832/2418205237,v1:172.21.15.203:6833/2418205237] conn(0x7f09b00744b0 0x7f09b00769c0 secure :-1 s=READY pgs=64 cs=0 l=1 rev1=1 crypto rx=0x7f09d008e110 tx=0x7f09c0009040 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:29:12.989 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:12.987+0000 7f09c6ffd640 1 -- 172.21.15.37:0/1675636704 <== mon.0 v2:172.21.15.37:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187246 (secure 0 0 0) 0x7f09b804b030 con 0x7f09d0160f40 2024-04-25T10:29:13.172 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:13.170+0000 7f09d4e84640 1 -- 172.21.15.37:0/1675636704 --> [v2:172.21.15.203:6832/2418205237,v1:172.21.15.203:6833/2418205237] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) -- 0x7f09d01f64f0 con 0x7f09b00744b0 2024-04-25T10:29:13.179 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:13.177+0000 7f09c6ffd640 1 -- 172.21.15.37:0/1675636704 <== mgr.4100 v2:172.21.15.203:6832/2418205237 1 ==== mgr_command_reply(tid 0: 0 dumped all) ==== 18+0+251335 (secure 0 0 0) 0x7f09d01f64f0 con 0x7f09b00744b0 2024-04-25T10:29:13.182 INFO:teuthology.orchestra.run.smithi037.stdout: 2024-04-25T10:29:13.184 INFO:teuthology.orchestra.run.smithi037.stderr:dumped all 2024-04-25T10:29:13.186 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:13.181+0000 7f09d4e84640 1 -- 172.21.15.37:0/1675636704 >> [v2:172.21.15.203:6832/2418205237,v1:172.21.15.203:6833/2418205237] conn(0x7f09b00744b0 msgr2=0x7f09b00769c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:29:13.186 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:13.181+0000 7f09d4e84640 1 --2- 172.21.15.37:0/1675636704 >> [v2:172.21.15.203:6832/2418205237,v1:172.21.15.203:6833/2418205237] conn(0x7f09b00744b0 0x7f09b00769c0 secure :-1 s=READY pgs=64 cs=0 l=1 rev1=1 crypto rx=0x7f09d008e110 tx=0x7f09c0009040 comp rx=0 tx=0).stop 2024-04-25T10:29:13.186 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:13.181+0000 7f09d4e84640 1 -- 172.21.15.37:0/1675636704 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f09d0160f40 msgr2=0x7f09d015b950 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:29:13.186 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:13.181+0000 7f09d4e84640 1 --2- 172.21.15.37:0/1675636704 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f09d0160f40 0x7f09d015b950 secure :-1 s=READY pgs=80 cs=0 l=1 rev1=1 crypto rx=0x7f09b8000c00 tx=0x7f09b80046c0 comp rx=0 tx=0).stop 2024-04-25T10:29:13.186 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:13.181+0000 7f09d4e84640 1 -- 172.21.15.37:0/1675636704 shutdown_connections 2024-04-25T10:29:13.186 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:13.181+0000 7f09d4e84640 1 --2- 172.21.15.37:0/1675636704 >> [v2:172.21.15.203:6832/2418205237,v1:172.21.15.203:6833/2418205237] conn(0x7f09b00744b0 0x7f09b00769c0 unknown :-1 s=CLOSED pgs=64 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:29:13.187 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:13.181+0000 7f09d4e84640 1 --2- 172.21.15.37:0/1675636704 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f09d015be90 0x7f09d0146310 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:29:13.187 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:13.181+0000 7f09d4e84640 1 --2- 172.21.15.37:0/1675636704 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f09d0160f40 0x7f09d015b950 unknown :-1 s=CLOSED pgs=80 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:29:13.187 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:13.181+0000 7f09d4e84640 1 --2- 172.21.15.37:0/1675636704 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f09d00d5430 0x7f09d01f5dc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:29:13.187 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:13.181+0000 7f09d4e84640 1 -- 172.21.15.37:0/1675636704 >> 172.21.15.37:0/1675636704 conn(0x7f09d00d27a0 msgr2=0x7f09d01617c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:29:13.187 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:13.181+0000 7f09d4e84640 1 -- 172.21.15.37:0/1675636704 shutdown_connections 2024-04-25T10:29:13.187 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:13.181+0000 7f09d4e84640 1 -- 172.21.15.37:0/1675636704 wait complete. 2024-04-25T10:29:13.194 INFO:teuthology.orchestra.run.smithi037.stdout:{"pg_ready":true,"pg_map":{"version":26,"stamp":"2024-04-25T10:29:12.186439+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":592196,"num_objects":16,"num_object_clones":0,"num_object_copies":32,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":16,"num_whiteouts":0,"num_read":66,"num_read_kb":55,"num_write":126,"num_write_kb":1641,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":7,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":96,"ondisk_log_size":96,"up":194,"acting":194,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":52,"num_osds":8,"num_per_pool_osds":8,"num_per_pool_omap_osds":8,"kb":754974720,"kb_used":219068,"kb_used_data":2908,"kb_used_omap":12,"kb_used_meta":216051,"kb_avail":754755652,"statfs":{"total":773094113280,"available":772869787648,"internally_reserved":0,"allocated":2977792,"data_stored":1769702,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":12752,"internal_metadata":221236784},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":30,"apply_latency_ms":30,"commit_latency_ns":30000000,"apply_latency_ns":30000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":1828,"num_objects":14,"num_object_clones":0,"num_object_copies":28,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":14,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":25,"num_write_kb":13,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":7,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"5.190009"},"pg_stats":[{"pgid":"3.3f","version":"0'0","reported_seq":12,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-04-25T10:29:10.037967+0000","last_change":"2024-04-25T10:29:09.024146+0000","last_active":"2024-04-25T10:29:10.037967+0000","last_peered":"2024-04-25T10:29:10.037967+0000","last_clean":"2024-04-25T10:29:10.037967+0000","last_became_active":"2024-04-25T10:29:09.023798+0000","last_became_peered":"2024-04-25T10:29:09.023798+0000","last_unstale":"2024-04-25T10:29:10.037967+0000","last_undegraded":"2024-04-25T10:29:10.037967+0000","last_fullsized":"2024-04-25T10:29:10.037967+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:29:07.995096+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:29:07.995096+0000","last_clean_scrub_stamp":"2024-04-25T10:29:07.995096+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T20:37:20.726766+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.3e","version":"0'0","reported_seq":12,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-04-25T10:29:10.189579+0000","last_change":"2024-04-25T10:29:09.028600+0000","last_active":"2024-04-25T10:29:10.189579+0000","last_peered":"2024-04-25T10:29:10.189579+0000","last_clean":"2024-04-25T10:29:10.189579+0000","last_became_active":"2024-04-25T10:29:09.028286+0000","last_became_peered":"2024-04-25T10:29:09.028286+0000","last_unstale":"2024-04-25T10:29:10.189579+0000","last_undegraded":"2024-04-25T10:29:10.189579+0000","last_fullsized":"2024-04-25T10:29:10.189579+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:29:07.995096+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:29:07.995096+0000","last_clean_scrub_stamp":"2024-04-25T10:29:07.995096+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T20:52:02.098872+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6],"acting":[5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.3d","version":"0'0","reported_seq":12,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-04-25T10:29:10.189318+0000","last_change":"2024-04-25T10:29:09.039030+0000","last_active":"2024-04-25T10:29:10.189318+0000","last_peered":"2024-04-25T10:29:10.189318+0000","last_clean":"2024-04-25T10:29:10.189318+0000","last_became_active":"2024-04-25T10:29:09.037692+0000","last_became_peered":"2024-04-25T10:29:09.037692+0000","last_unstale":"2024-04-25T10:29:10.189318+0000","last_undegraded":"2024-04-25T10:29:10.189318+0000","last_fullsized":"2024-04-25T10:29:10.189318+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:29:07.995096+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:29:07.995096+0000","last_clean_scrub_stamp":"2024-04-25T10:29:07.995096+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T15:51:52.989545+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.3c","version":"0'0","reported_seq":12,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-04-25T10:29:10.026169+0000","last_change":"2024-04-25T10:29:09.027520+0000","last_active":"2024-04-25T10:29:10.026169+0000","last_peered":"2024-04-25T10:29:10.026169+0000","last_clean":"2024-04-25T10:29:10.026169+0000","last_became_active":"2024-04-25T10:29:09.027075+0000","last_became_peered":"2024-04-25T10:29:09.027075+0000","last_unstale":"2024-04-25T10:29:10.026169+0000","last_undegraded":"2024-04-25T10:29:10.026169+0000","last_fullsized":"2024-04-25T10:29:10.026169+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:29:07.995096+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:29:07.995096+0000","last_clean_scrub_stamp":"2024-04-25T10:29:07.995096+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T10:55:01.505157+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4],"acting":[2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.3b","version":"0'0","reported_seq":12,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-04-25T10:29:10.037590+0000","last_change":"2024-04-25T10:29:09.040684+0000","last_active":"2024-04-25T10:29:10.037590+0000","last_peered":"2024-04-25T10:29:10.037590+0000","last_clean":"2024-04-25T10:29:10.037590+0000","last_became_active":"2024-04-25T10:29:09.040372+0000","last_became_peered":"2024-04-25T10:29:09.040372+0000","last_unstale":"2024-04-25T10:29:10.037590+0000","last_undegraded":"2024-04-25T10:29:10.037590+0000","last_fullsized":"2024-04-25T10:29:10.037590+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:29:07.995096+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:29:07.995096+0000","last_clean_scrub_stamp":"2024-04-25T10:29:07.995096+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T11:52:24.415332+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.3a","version":"0'0","reported_seq":12,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-04-25T10:29:10.025911+0000","last_change":"2024-04-25T10:29:09.027391+0000","last_active":"2024-04-25T10:29:10.025911+0000","last_peered":"2024-04-25T10:29:10.025911+0000","last_clean":"2024-04-25T10:29:10.025911+0000","last_became_active":"2024-04-25T10:29:09.027029+0000","last_became_peered":"2024-04-25T10:29:09.027029+0000","last_unstale":"2024-04-25T10:29:10.025911+0000","last_undegraded":"2024-04-25T10:29:10.025911+0000","last_fullsized":"2024-04-25T10:29:10.025911+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:29:07.995096+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:29:07.995096+0000","last_clean_scrub_stamp":"2024-04-25T10:29:07.995096+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T10:35:31.361514+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4],"acting":[2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.39","version":"0'0","reported_seq":12,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-04-25T10:29:10.037462+0000","last_change":"2024-04-25T10:29:09.029272+0000","last_active":"2024-04-25T10:29:10.037462+0000","last_peered":"2024-04-25T10:29:10.037462+0000","last_clean":"2024-04-25T10:29:10.037462+0000","last_became_active":"2024-04-25T10:29:09.028302+0000","last_became_peered":"2024-04-25T10:29:09.028302+0000","last_unstale":"2024-04-25T10:29:10.037462+0000","last_undegraded":"2024-04-25T10:29:10.037462+0000","last_fullsized":"2024-04-25T10:29:10.037462+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:29:07.995096+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:29:07.995096+0000","last_clean_scrub_stamp":"2024-04-25T10:29:07.995096+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T11:31:45.283247+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.38","version":"0'0","reported_seq":12,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-04-25T10:29:10.188973+0000","last_change":"2024-04-25T10:29:09.035089+0000","last_active":"2024-04-25T10:29:10.188973+0000","last_peered":"2024-04-25T10:29:10.188973+0000","last_clean":"2024-04-25T10:29:10.188973+0000","last_became_active":"2024-04-25T10:29:09.034324+0000","last_became_peered":"2024-04-25T10:29:09.034324+0000","last_unstale":"2024-04-25T10:29:10.188973+0000","last_undegraded":"2024-04-25T10:29:10.188973+0000","last_fullsized":"2024-04-25T10:29:10.188973+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:29:07.995096+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:29:07.995096+0000","last_clean_scrub_stamp":"2024-04-25T10:29:07.995096+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T21:39:39.743441+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.37","version":"0'0","reported_seq":12,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-04-25T10:29:10.187981+0000","last_change":"2024-04-25T10:29:09.016966+0000","last_active":"2024-04-25T10:29:10.187981+0000","last_peered":"2024-04-25T10:29:10.187981+0000","last_clean":"2024-04-25T10:29:10.187981+0000","last_became_active":"2024-04-25T10:29:09.016566+0000","last_became_peered":"2024-04-25T10:29:09.016566+0000","last_unstale":"2024-04-25T10:29:10.187981+0000","last_undegraded":"2024-04-25T10:29:10.187981+0000","last_fullsized":"2024-04-25T10:29:10.187981+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:29:07.995096+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:29:07.995096+0000","last_clean_scrub_stamp":"2024-04-25T10:29:07.995096+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T10:55:01.505157+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.36","version":"0'0","reported_seq":12,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-04-25T10:29:10.037810+0000","last_change":"2024-04-25T10:29:09.030321+0000","last_active":"2024-04-25T10:29:10.037810+0000","last_peered":"2024-04-25T10:29:10.037810+0000","last_clean":"2024-04-25T10:29:10.037810+0000","last_became_active":"2024-04-25T10:29:09.029976+0000","last_became_peered":"2024-04-25T10:29:09.029976+0000","last_unstale":"2024-04-25T10:29:10.037810+0000","last_undegraded":"2024-04-25T10:29:10.037810+0000","last_fullsized":"2024-04-25T10:29:10.037810+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:29:07.995096+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:29:07.995096+0000","last_clean_scrub_stamp":"2024-04-25T10:29:07.995096+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T12:36:31.191100+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.35","version":"0'0","reported_seq":12,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-04-25T10:29:10.037641+0000","last_change":"2024-04-25T10:29:09.030469+0000","last_active":"2024-04-25T10:29:10.037641+0000","last_peered":"2024-04-25T10:29:10.037641+0000","last_clean":"2024-04-25T10:29:10.037641+0000","last_became_active":"2024-04-25T10:29:09.029350+0000","last_became_peered":"2024-04-25T10:29:09.029350+0000","last_unstale":"2024-04-25T10:29:10.037641+0000","last_undegraded":"2024-04-25T10:29:10.037641+0000","last_fullsized":"2024-04-25T10:29:10.037641+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:29:07.995096+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:29:07.995096+0000","last_clean_scrub_stamp":"2024-04-25T10:29:07.995096+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T15:51:52.989545+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.34","version":"0'0","reported_seq":12,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-04-25T10:29:10.037728+0000","last_change":"2024-04-25T10:29:09.029772+0000","last_active":"2024-04-25T10:29:10.037728+0000","last_peered":"2024-04-25T10:29:10.037728+0000","last_clean":"2024-04-25T10:29:10.037728+0000","last_became_active":"2024-04-25T10:29:09.029350+0000","last_became_peered":"2024-04-25T10:29:09.029350+0000","last_unstale":"2024-04-25T10:29:10.037728+0000","last_undegraded":"2024-04-25T10:29:10.037728+0000","last_fullsized":"2024-04-25T10:29:10.037728+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:29:07.995096+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:29:07.995096+0000","last_clean_scrub_stamp":"2024-04-25T10:29:07.995096+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T19:14:48.737269+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.33","version":"0'0","reported_seq":12,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-04-25T10:29:10.188971+0000","last_change":"2024-04-25T10:29:09.038690+0000","last_active":"2024-04-25T10:29:10.188971+0000","last_peered":"2024-04-25T10:29:10.188971+0000","last_clean":"2024-04-25T10:29:10.188971+0000","last_became_active":"2024-04-25T10:29:09.038336+0000","last_became_peered":"2024-04-25T10:29:09.038336+0000","last_unstale":"2024-04-25T10:29:10.188971+0000","last_undegraded":"2024-04-25T10:29:10.188971+0000","last_fullsized":"2024-04-25T10:29:10.188971+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:29:07.995096+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:29:07.995096+0000","last_clean_scrub_stamp":"2024-04-25T10:29:07.995096+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T19:23:12.589414+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.32","version":"0'0","reported_seq":12,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-04-25T10:29:10.188641+0000","last_change":"2024-04-25T10:29:09.028195+0000","last_active":"2024-04-25T10:29:10.188641+0000","last_peered":"2024-04-25T10:29:10.188641+0000","last_clean":"2024-04-25T10:29:10.188641+0000","last_became_active":"2024-04-25T10:29:09.027893+0000","last_became_peered":"2024-04-25T10:29:09.027893+0000","last_unstale":"2024-04-25T10:29:10.188641+0000","last_undegraded":"2024-04-25T10:29:10.188641+0000","last_fullsized":"2024-04-25T10:29:10.188641+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:29:07.995096+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:29:07.995096+0000","last_clean_scrub_stamp":"2024-04-25T10:29:07.995096+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T11:35:27.466874+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.31","version":"0'0","reported_seq":12,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-04-25T10:29:10.188956+0000","last_change":"2024-04-25T10:29:09.034532+0000","last_active":"2024-04-25T10:29:10.188956+0000","last_peered":"2024-04-25T10:29:10.188956+0000","last_clean":"2024-04-25T10:29:10.188956+0000","last_became_active":"2024-04-25T10:29:09.034021+0000","last_became_peered":"2024-04-25T10:29:09.034021+0000","last_unstale":"2024-04-25T10:29:10.188956+0000","last_undegraded":"2024-04-25T10:29:10.188956+0000","last_fullsized":"2024-04-25T10:29:10.188956+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:29:07.995096+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:29:07.995096+0000","last_clean_scrub_stamp":"2024-04-25T10:29:07.995096+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T12:16:04.880732+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.30","version":"0'0","reported_seq":12,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-04-25T10:29:10.024134+0000","last_change":"2024-04-25T10:29:09.021224+0000","last_active":"2024-04-25T10:29:10.024134+0000","last_peered":"2024-04-25T10:29:10.024134+0000","last_clean":"2024-04-25T10:29:10.024134+0000","last_became_active":"2024-04-25T10:29:09.020964+0000","last_became_peered":"2024-04-25T10:29:09.020964+0000","last_unstale":"2024-04-25T10:29:10.024134+0000","last_undegraded":"2024-04-25T10:29:10.024134+0000","last_fullsized":"2024-04-25T10:29:10.024134+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:29:07.995096+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:29:07.995096+0000","last_clean_scrub_stamp":"2024-04-25T10:29:07.995096+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T18:44:23.610374+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.2f","version":"0'0","reported_seq":12,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-04-25T10:29:10.024944+0000","last_change":"2024-04-25T10:29:09.025638+0000","last_active":"2024-04-25T10:29:10.024944+0000","last_peered":"2024-04-25T10:29:10.024944+0000","last_clean":"2024-04-25T10:29:10.024944+0000","last_became_active":"2024-04-25T10:29:09.024517+0000","last_became_peered":"2024-04-25T10:29:09.024517+0000","last_unstale":"2024-04-25T10:29:10.024944+0000","last_undegraded":"2024-04-25T10:29:10.024944+0000","last_fullsized":"2024-04-25T10:29:10.024944+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:29:07.995096+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:29:07.995096+0000","last_clean_scrub_stamp":"2024-04-25T10:29:07.995096+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T16:29:08.975346+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.2e","version":"0'0","reported_seq":12,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-04-25T10:29:10.189006+0000","last_change":"2024-04-25T10:29:09.023923+0000","last_active":"2024-04-25T10:29:10.189006+0000","last_peered":"2024-04-25T10:29:10.189006+0000","last_clean":"2024-04-25T10:29:10.189006+0000","last_became_active":"2024-04-25T10:29:09.023579+0000","last_became_peered":"2024-04-25T10:29:09.023579+0000","last_unstale":"2024-04-25T10:29:10.189006+0000","last_undegraded":"2024-04-25T10:29:10.189006+0000","last_fullsized":"2024-04-25T10:29:10.189006+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:29:07.995096+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:29:07.995096+0000","last_clean_scrub_stamp":"2024-04-25T10:29:07.995096+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T10:35:31.361514+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.2d","version":"0'0","reported_seq":12,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-04-25T10:29:10.024852+0000","last_change":"2024-04-25T10:29:09.023039+0000","last_active":"2024-04-25T10:29:10.024852+0000","last_peered":"2024-04-25T10:29:10.024852+0000","last_clean":"2024-04-25T10:29:10.024852+0000","last_became_active":"2024-04-25T10:29:09.022706+0000","last_became_peered":"2024-04-25T10:29:09.022706+0000","last_unstale":"2024-04-25T10:29:10.024852+0000","last_undegraded":"2024-04-25T10:29:10.024852+0000","last_fullsized":"2024-04-25T10:29:10.024852+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:29:07.995096+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:29:07.995096+0000","last_clean_scrub_stamp":"2024-04-25T10:29:07.995096+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T16:48:17.352944+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.2c","version":"0'0","reported_seq":12,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-04-25T10:29:10.037891+0000","last_change":"2024-04-25T10:29:09.028979+0000","last_active":"2024-04-25T10:29:10.037891+0000","last_peered":"2024-04-25T10:29:10.037891+0000","last_clean":"2024-04-25T10:29:10.037891+0000","last_became_active":"2024-04-25T10:29:09.028061+0000","last_became_peered":"2024-04-25T10:29:09.028061+0000","last_unstale":"2024-04-25T10:29:10.037891+0000","last_undegraded":"2024-04-25T10:29:10.037891+0000","last_fullsized":"2024-04-25T10:29:10.037891+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:29:07.995096+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:29:07.995096+0000","last_clean_scrub_stamp":"2024-04-25T10:29:07.995096+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T18:44:23.610374+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.2b","version":"0'0","reported_seq":1,"reported_epoch":21,"state":"creating+peering","last_fresh":"2024-04-25T10:29:08.012124+0000","last_change":"2024-04-25T10:29:08.012124+0000","last_active":"2024-04-25T10:29:07.995096+0000","last_peered":"2024-04-25T10:29:07.995096+0000","last_clean":"2024-04-25T10:29:07.995096+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T10:29:08.012124+0000","last_undegraded":"2024-04-25T10:29:08.012124+0000","last_fullsized":"2024-04-25T10:29:08.012124+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:29:07.995096+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:29:07.995096+0000","last_clean_scrub_stamp":"2024-04-25T10:29:07.995096+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.2a","version":"0'0","reported_seq":12,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-04-25T10:29:10.187695+0000","last_change":"2024-04-25T10:29:09.029965+0000","last_active":"2024-04-25T10:29:10.187695+0000","last_peered":"2024-04-25T10:29:10.187695+0000","last_clean":"2024-04-25T10:29:10.187695+0000","last_became_active":"2024-04-25T10:29:09.029616+0000","last_became_peered":"2024-04-25T10:29:09.029616+0000","last_unstale":"2024-04-25T10:29:10.187695+0000","last_undegraded":"2024-04-25T10:29:10.187695+0000","last_fullsized":"2024-04-25T10:29:10.187695+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:29:07.995096+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:29:07.995096+0000","last_clean_scrub_stamp":"2024-04-25T10:29:07.995096+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T15:58:36.848831+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.29","version":"0'0","reported_seq":12,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-04-25T10:29:10.189345+0000","last_change":"2024-04-25T10:29:09.033401+0000","last_active":"2024-04-25T10:29:10.189345+0000","last_peered":"2024-04-25T10:29:10.189345+0000","last_clean":"2024-04-25T10:29:10.189345+0000","last_became_active":"2024-04-25T10:29:09.032764+0000","last_became_peered":"2024-04-25T10:29:09.032764+0000","last_unstale":"2024-04-25T10:29:10.189345+0000","last_undegraded":"2024-04-25T10:29:10.189345+0000","last_fullsized":"2024-04-25T10:29:10.189345+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:29:07.995096+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:29:07.995096+0000","last_clean_scrub_stamp":"2024-04-25T10:29:07.995096+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T17:30:54.093151+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.28","version":"0'0","reported_seq":12,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-04-25T10:29:10.024133+0000","last_change":"2024-04-25T10:29:09.041063+0000","last_active":"2024-04-25T10:29:10.024133+0000","last_peered":"2024-04-25T10:29:10.024133+0000","last_clean":"2024-04-25T10:29:10.024133+0000","last_became_active":"2024-04-25T10:29:09.040734+0000","last_became_peered":"2024-04-25T10:29:09.040734+0000","last_unstale":"2024-04-25T10:29:10.024133+0000","last_undegraded":"2024-04-25T10:29:10.024133+0000","last_fullsized":"2024-04-25T10:29:10.024133+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:29:07.995096+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:29:07.995096+0000","last_clean_scrub_stamp":"2024-04-25T10:29:07.995096+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T13:54:22.968216+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.27","version":"0'0","reported_seq":12,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-04-25T10:29:10.189019+0000","last_change":"2024-04-25T10:29:09.034268+0000","last_active":"2024-04-25T10:29:10.189019+0000","last_peered":"2024-04-25T10:29:10.189019+0000","last_clean":"2024-04-25T10:29:10.189019+0000","last_became_active":"2024-04-25T10:29:09.033199+0000","last_became_peered":"2024-04-25T10:29:09.033199+0000","last_unstale":"2024-04-25T10:29:10.189019+0000","last_undegraded":"2024-04-25T10:29:10.189019+0000","last_fullsized":"2024-04-25T10:29:10.189019+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:29:07.995096+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:29:07.995096+0000","last_clean_scrub_stamp":"2024-04-25T10:29:07.995096+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T17:56:09.999494+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.26","version":"0'0","reported_seq":12,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-04-25T10:29:10.026183+0000","last_change":"2024-04-25T10:29:09.025643+0000","last_active":"2024-04-25T10:29:10.026183+0000","last_peered":"2024-04-25T10:29:10.026183+0000","last_clean":"2024-04-25T10:29:10.026183+0000","last_became_active":"2024-04-25T10:29:09.024894+0000","last_became_peered":"2024-04-25T10:29:09.024894+0000","last_unstale":"2024-04-25T10:29:10.026183+0000","last_undegraded":"2024-04-25T10:29:10.026183+0000","last_fullsized":"2024-04-25T10:29:10.026183+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:29:07.995096+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:29:07.995096+0000","last_clean_scrub_stamp":"2024-04-25T10:29:07.995096+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T22:08:00.367431+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.25","version":"0'0","reported_seq":12,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-04-25T10:29:10.189130+0000","last_change":"2024-04-25T10:29:09.023357+0000","last_active":"2024-04-25T10:29:10.189130+0000","last_peered":"2024-04-25T10:29:10.189130+0000","last_clean":"2024-04-25T10:29:10.189130+0000","last_became_active":"2024-04-25T10:29:09.022988+0000","last_became_peered":"2024-04-25T10:29:09.022988+0000","last_unstale":"2024-04-25T10:29:10.189130+0000","last_undegraded":"2024-04-25T10:29:10.189130+0000","last_fullsized":"2024-04-25T10:29:10.189130+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:29:07.995096+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:29:07.995096+0000","last_clean_scrub_stamp":"2024-04-25T10:29:07.995096+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T21:14:39.661957+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.24","version":"0'0","reported_seq":12,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-04-25T10:29:10.189239+0000","last_change":"2024-04-25T10:29:09.034328+0000","last_active":"2024-04-25T10:29:10.189239+0000","last_peered":"2024-04-25T10:29:10.189239+0000","last_clean":"2024-04-25T10:29:10.189239+0000","last_became_active":"2024-04-25T10:29:09.033978+0000","last_became_peered":"2024-04-25T10:29:09.033978+0000","last_unstale":"2024-04-25T10:29:10.189239+0000","last_undegraded":"2024-04-25T10:29:10.189239+0000","last_fullsized":"2024-04-25T10:29:10.189239+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:29:07.995096+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:29:07.995096+0000","last_clean_scrub_stamp":"2024-04-25T10:29:07.995096+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T21:24:14.497242+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.23","version":"0'0","reported_seq":12,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-04-25T10:29:10.189583+0000","last_change":"2024-04-25T10:29:09.034045+0000","last_active":"2024-04-25T10:29:10.189583+0000","last_peered":"2024-04-25T10:29:10.189583+0000","last_clean":"2024-04-25T10:29:10.189583+0000","last_became_active":"2024-04-25T10:29:09.033730+0000","last_became_peered":"2024-04-25T10:29:09.033730+0000","last_unstale":"2024-04-25T10:29:10.189583+0000","last_undegraded":"2024-04-25T10:29:10.189583+0000","last_fullsized":"2024-04-25T10:29:10.189583+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:29:07.995096+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:29:07.995096+0000","last_clean_scrub_stamp":"2024-04-25T10:29:07.995096+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T12:36:31.191100+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.22","version":"0'0","reported_seq":12,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-04-25T10:29:10.024663+0000","last_change":"2024-04-25T10:29:09.034262+0000","last_active":"2024-04-25T10:29:10.024663+0000","last_peered":"2024-04-25T10:29:10.024663+0000","last_clean":"2024-04-25T10:29:10.024663+0000","last_became_active":"2024-04-25T10:29:09.033905+0000","last_became_peered":"2024-04-25T10:29:09.033905+0000","last_unstale":"2024-04-25T10:29:10.024663+0000","last_undegraded":"2024-04-25T10:29:10.024663+0000","last_fullsized":"2024-04-25T10:29:10.024663+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:29:07.995096+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:29:07.995096+0000","last_clean_scrub_stamp":"2024-04-25T10:29:07.995096+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T16:22:45.608928+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.21","version":"0'0","reported_seq":12,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-04-25T10:29:10.188201+0000","last_change":"2024-04-25T10:29:09.027088+0000","last_active":"2024-04-25T10:29:10.188201+0000","last_peered":"2024-04-25T10:29:10.188201+0000","last_clean":"2024-04-25T10:29:10.188201+0000","last_became_active":"2024-04-25T10:29:09.026651+0000","last_became_peered":"2024-04-25T10:29:09.026651+0000","last_unstale":"2024-04-25T10:29:10.188201+0000","last_undegraded":"2024-04-25T10:29:10.188201+0000","last_fullsized":"2024-04-25T10:29:10.188201+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:29:07.995096+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:29:07.995096+0000","last_clean_scrub_stamp":"2024-04-25T10:29:07.995096+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T15:37:06.723380+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.20","version":"0'0","reported_seq":12,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-04-25T10:29:10.025741+0000","last_change":"2024-04-25T10:29:09.028008+0000","last_active":"2024-04-25T10:29:10.025741+0000","last_peered":"2024-04-25T10:29:10.025741+0000","last_clean":"2024-04-25T10:29:10.025741+0000","last_became_active":"2024-04-25T10:29:09.027679+0000","last_became_peered":"2024-04-25T10:29:09.027679+0000","last_unstale":"2024-04-25T10:29:10.025741+0000","last_undegraded":"2024-04-25T10:29:10.025741+0000","last_fullsized":"2024-04-25T10:29:10.025741+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:29:07.995096+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:29:07.995096+0000","last_clean_scrub_stamp":"2024-04-25T10:29:07.995096+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T15:37:06.723380+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,6],"acting":[2,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.1a","version":"0'0","reported_seq":12,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-04-25T10:29:10.187787+0000","last_change":"2024-04-25T10:29:09.032819+0000","last_active":"2024-04-25T10:29:10.187787+0000","last_peered":"2024-04-25T10:29:10.187787+0000","last_clean":"2024-04-25T10:29:10.187787+0000","last_became_active":"2024-04-25T10:29:09.032493+0000","last_became_peered":"2024-04-25T10:29:09.032493+0000","last_unstale":"2024-04-25T10:29:10.187787+0000","last_undegraded":"2024-04-25T10:29:10.187787+0000","last_fullsized":"2024-04-25T10:29:10.187787+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:29:07.995096+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:29:07.995096+0000","last_clean_scrub_stamp":"2024-04-25T10:29:07.995096+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T15:37:06.723380+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.1b","version":"22'1","reported_seq":15,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-04-25T10:29:10.189463+0000","last_change":"2024-04-25T10:29:08.025272+0000","last_active":"2024-04-25T10:29:10.189463+0000","last_peered":"2024-04-25T10:29:10.189463+0000","last_clean":"2024-04-25T10:29:10.189463+0000","last_became_active":"2024-04-25T10:29:08.024912+0000","last_became_peered":"2024-04-25T10:29:08.024912+0000","last_unstale":"2024-04-25T10:29:10.189463+0000","last_undegraded":"2024-04-25T10:29:10.189463+0000","last_fullsized":"2024-04-25T10:29:10.189463+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:29:06.989570+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:29:06.989570+0000","last_clean_scrub_stamp":"2024-04-25T10:29:06.989570+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T18:11:32.758179+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6],"acting":[5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.1b","version":"0'0","reported_seq":1,"reported_epoch":21,"state":"creating+peering","last_fresh":"2024-04-25T10:29:08.008604+0000","last_change":"2024-04-25T10:29:08.008604+0000","last_active":"2024-04-25T10:29:07.995096+0000","last_peered":"2024-04-25T10:29:07.995096+0000","last_clean":"2024-04-25T10:29:07.995096+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T10:29:08.008604+0000","last_undegraded":"2024-04-25T10:29:08.008604+0000","last_fullsized":"2024-04-25T10:29:08.008604+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:29:07.995096+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:29:07.995096+0000","last_clean_scrub_stamp":"2024-04-25T10:29:07.995096+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.1a","version":"22'1","reported_seq":15,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-04-25T10:29:10.024506+0000","last_change":"2024-04-25T10:29:08.026018+0000","last_active":"2024-04-25T10:29:10.024506+0000","last_peered":"2024-04-25T10:29:10.024506+0000","last_clean":"2024-04-25T10:29:10.024506+0000","last_became_active":"2024-04-25T10:29:08.025668+0000","last_became_peered":"2024-04-25T10:29:08.025668+0000","last_unstale":"2024-04-25T10:29:10.024506+0000","last_undegraded":"2024-04-25T10:29:10.024506+0000","last_fullsized":"2024-04-25T10:29:10.024506+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:29:06.989570+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:29:06.989570+0000","last_clean_scrub_stamp":"2024-04-25T10:29:06.989570+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T11:35:26.461348+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.19","version":"0'0","reported_seq":14,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-04-25T10:29:10.188333+0000","last_change":"2024-04-25T10:29:08.011702+0000","last_active":"2024-04-25T10:29:10.188333+0000","last_peered":"2024-04-25T10:29:10.188333+0000","last_clean":"2024-04-25T10:29:10.188333+0000","last_became_active":"2024-04-25T10:29:08.011353+0000","last_became_peered":"2024-04-25T10:29:08.011353+0000","last_unstale":"2024-04-25T10:29:10.188333+0000","last_undegraded":"2024-04-25T10:29:10.188333+0000","last_fullsized":"2024-04-25T10:29:10.188333+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:29:06.989570+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:29:06.989570+0000","last_clean_scrub_stamp":"2024-04-25T10:29:06.989570+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T16:31:11.685101+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.18","version":"0'0","reported_seq":12,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-04-25T10:29:10.189830+0000","last_change":"2024-04-25T10:29:09.033872+0000","last_active":"2024-04-25T10:29:10.189830+0000","last_peered":"2024-04-25T10:29:10.189830+0000","last_clean":"2024-04-25T10:29:10.189830+0000","last_became_active":"2024-04-25T10:29:09.033544+0000","last_became_peered":"2024-04-25T10:29:09.033544+0000","last_unstale":"2024-04-25T10:29:10.189830+0000","last_undegraded":"2024-04-25T10:29:10.189830+0000","last_fullsized":"2024-04-25T10:29:10.189830+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:29:07.995096+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:29:07.995096+0000","last_clean_scrub_stamp":"2024-04-25T10:29:07.995096+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T13:54:22.968216+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6],"acting":[5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.18","version":"0'0","reported_seq":14,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-04-25T10:29:10.024423+0000","last_change":"2024-04-25T10:29:08.013340+0000","last_active":"2024-04-25T10:29:10.024423+0000","last_peered":"2024-04-25T10:29:10.024423+0000","last_clean":"2024-04-25T10:29:10.024423+0000","last_became_active":"2024-04-25T10:29:08.012940+0000","last_became_peered":"2024-04-25T10:29:08.012940+0000","last_unstale":"2024-04-25T10:29:10.024423+0000","last_undegraded":"2024-04-25T10:29:10.024423+0000","last_fullsized":"2024-04-25T10:29:10.024423+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:29:06.989570+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:29:06.989570+0000","last_clean_scrub_stamp":"2024-04-25T10:29:06.989570+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T10:35:30.355988+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.19","version":"0'0","reported_seq":1,"reported_epoch":21,"state":"creating+peering","last_fresh":"2024-04-25T10:29:08.012731+0000","last_change":"2024-04-25T10:29:08.012731+0000","last_active":"2024-04-25T10:29:07.995096+0000","last_peered":"2024-04-25T10:29:07.995096+0000","last_clean":"2024-04-25T10:29:07.995096+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T10:29:08.012731+0000","last_undegraded":"2024-04-25T10:29:08.012731+0000","last_fullsized":"2024-04-25T10:29:08.012731+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:29:07.995096+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:29:07.995096+0000","last_clean_scrub_stamp":"2024-04-25T10:29:07.995096+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.17","version":"0'0","reported_seq":14,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-04-25T10:29:10.187949+0000","last_change":"2024-04-25T10:29:08.020281+0000","last_active":"2024-04-25T10:29:10.187949+0000","last_peered":"2024-04-25T10:29:10.187949+0000","last_clean":"2024-04-25T10:29:10.187949+0000","last_became_active":"2024-04-25T10:29:08.019921+0000","last_became_peered":"2024-04-25T10:29:08.019921+0000","last_unstale":"2024-04-25T10:29:10.187949+0000","last_undegraded":"2024-04-25T10:29:10.187949+0000","last_fullsized":"2024-04-25T10:29:10.187949+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:29:06.989570+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:29:06.989570+0000","last_clean_scrub_stamp":"2024-04-25T10:29:06.989570+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T19:13:07.068780+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.16","version":"0'0","reported_seq":12,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-04-25T10:29:10.026197+0000","last_change":"2024-04-25T10:29:09.022744+0000","last_active":"2024-04-25T10:29:10.026197+0000","last_peered":"2024-04-25T10:29:10.026197+0000","last_clean":"2024-04-25T10:29:10.026197+0000","last_became_active":"2024-04-25T10:29:09.022392+0000","last_became_peered":"2024-04-25T10:29:09.022392+0000","last_unstale":"2024-04-25T10:29:10.026197+0000","last_undegraded":"2024-04-25T10:29:10.026197+0000","last_fullsized":"2024-04-25T10:29:10.026197+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:29:07.995096+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:29:07.995096+0000","last_clean_scrub_stamp":"2024-04-25T10:29:07.995096+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T11:35:27.466874+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.16","version":"0'0","reported_seq":14,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-04-25T10:29:10.037869+0000","last_change":"2024-04-25T10:29:08.026388+0000","last_active":"2024-04-25T10:29:10.037869+0000","last_peered":"2024-04-25T10:29:10.037869+0000","last_clean":"2024-04-25T10:29:10.037869+0000","last_became_active":"2024-04-25T10:29:08.026039+0000","last_became_peered":"2024-04-25T10:29:08.026039+0000","last_unstale":"2024-04-25T10:29:10.037869+0000","last_undegraded":"2024-04-25T10:29:10.037869+0000","last_fullsized":"2024-04-25T10:29:10.037869+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:29:06.989570+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:29:06.989570+0000","last_clean_scrub_stamp":"2024-04-25T10:29:06.989570+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T18:11:32.758179+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.17","version":"0'0","reported_seq":12,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-04-25T10:29:10.189465+0000","last_change":"2024-04-25T10:29:09.032902+0000","last_active":"2024-04-25T10:29:10.189465+0000","last_peered":"2024-04-25T10:29:10.189465+0000","last_clean":"2024-04-25T10:29:10.189465+0000","last_became_active":"2024-04-25T10:29:09.032596+0000","last_became_peered":"2024-04-25T10:29:09.032596+0000","last_unstale":"2024-04-25T10:29:10.189465+0000","last_undegraded":"2024-04-25T10:29:10.189465+0000","last_fullsized":"2024-04-25T10:29:10.189465+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:29:07.995096+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:29:07.995096+0000","last_clean_scrub_stamp":"2024-04-25T10:29:07.995096+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T19:14:48.737269+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.15","version":"22'3","reported_seq":17,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-04-25T10:29:10.188352+0000","last_change":"2024-04-25T10:29:08.021347+0000","last_active":"2024-04-25T10:29:10.188352+0000","last_peered":"2024-04-25T10:29:10.188352+0000","last_clean":"2024-04-25T10:29:10.188352+0000","last_became_active":"2024-04-25T10:29:08.021039+0000","last_became_peered":"2024-04-25T10:29:08.021039+0000","last_unstale":"2024-04-25T10:29:10.188352+0000","last_undegraded":"2024-04-25T10:29:10.188352+0000","last_fullsized":"2024-04-25T10:29:10.188352+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:29:06.989570+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:29:06.989570+0000","last_clean_scrub_stamp":"2024-04-25T10:29:06.989570+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T20:22:56.380539+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.14","version":"0'0","reported_seq":12,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-04-25T10:29:10.024492+0000","last_change":"2024-04-25T10:29:09.021410+0000","last_active":"2024-04-25T10:29:10.024492+0000","last_peered":"2024-04-25T10:29:10.024492+0000","last_clean":"2024-04-25T10:29:10.024492+0000","last_became_active":"2024-04-25T10:29:09.021051+0000","last_became_peered":"2024-04-25T10:29:09.021051+0000","last_unstale":"2024-04-25T10:29:10.024492+0000","last_undegraded":"2024-04-25T10:29:10.024492+0000","last_fullsized":"2024-04-25T10:29:10.024492+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:29:07.995096+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:29:07.995096+0000","last_clean_scrub_stamp":"2024-04-25T10:29:07.995096+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T13:22:13.080513+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.14","version":"22'6","reported_seq":20,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-04-25T10:29:10.025912+0000","last_change":"2024-04-25T10:29:08.018044+0000","last_active":"2024-04-25T10:29:10.025912+0000","last_peered":"2024-04-25T10:29:10.025912+0000","last_clean":"2024-04-25T10:29:10.025912+0000","last_became_active":"2024-04-25T10:29:08.016984+0000","last_became_peered":"2024-04-25T10:29:08.016984+0000","last_unstale":"2024-04-25T10:29:10.025912+0000","last_undegraded":"2024-04-25T10:29:10.025912+0000","last_fullsized":"2024-04-25T10:29:10.025912+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:29:06.989570+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:29:06.989570+0000","last_clean_scrub_stamp":"2024-04-25T10:29:06.989570+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T18:29:56.525236+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1612,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4],"acting":[2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.15","version":"0'0","reported_seq":12,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-04-25T10:29:10.037668+0000","last_change":"2024-04-25T10:29:09.029034+0000","last_active":"2024-04-25T10:29:10.037668+0000","last_peered":"2024-04-25T10:29:10.037668+0000","last_clean":"2024-04-25T10:29:10.037668+0000","last_became_active":"2024-04-25T10:29:09.028662+0000","last_became_peered":"2024-04-25T10:29:09.028662+0000","last_unstale":"2024-04-25T10:29:10.037668+0000","last_undegraded":"2024-04-25T10:29:10.037668+0000","last_fullsized":"2024-04-25T10:29:10.037668+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:29:07.995096+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:29:07.995096+0000","last_clean_scrub_stamp":"2024-04-25T10:29:07.995096+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T17:30:54.093151+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.13","version":"22'2","reported_seq":16,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-04-25T10:29:10.189430+0000","last_change":"2024-04-25T10:29:08.020476+0000","last_active":"2024-04-25T10:29:10.189430+0000","last_peered":"2024-04-25T10:29:10.189430+0000","last_clean":"2024-04-25T10:29:10.189430+0000","last_became_active":"2024-04-25T10:29:08.020096+0000","last_became_peered":"2024-04-25T10:29:08.020096+0000","last_unstale":"2024-04-25T10:29:10.189430+0000","last_undegraded":"2024-04-25T10:29:10.189430+0000","last_fullsized":"2024-04-25T10:29:10.189430+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:29:06.989570+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:29:06.989570+0000","last_clean_scrub_stamp":"2024-04-25T10:29:06.989570+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T10:32:06.103123+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":34,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":7,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.12","version":"0'0","reported_seq":12,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-04-25T10:29:10.188838+0000","last_change":"2024-04-25T10:29:09.027996+0000","last_active":"2024-04-25T10:29:10.188838+0000","last_peered":"2024-04-25T10:29:10.188838+0000","last_clean":"2024-04-25T10:29:10.188838+0000","last_became_active":"2024-04-25T10:29:09.027538+0000","last_became_peered":"2024-04-25T10:29:09.027538+0000","last_unstale":"2024-04-25T10:29:10.188838+0000","last_undegraded":"2024-04-25T10:29:10.188838+0000","last_fullsized":"2024-04-25T10:29:10.188838+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:29:07.995096+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:29:07.995096+0000","last_clean_scrub_stamp":"2024-04-25T10:29:07.995096+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T12:16:47.983013+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.12","version":"0'0","reported_seq":14,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-04-25T10:29:10.024603+0000","last_change":"2024-04-25T10:29:08.019028+0000","last_active":"2024-04-25T10:29:10.024603+0000","last_peered":"2024-04-25T10:29:10.024603+0000","last_clean":"2024-04-25T10:29:10.024603+0000","last_became_active":"2024-04-25T10:29:08.018582+0000","last_became_peered":"2024-04-25T10:29:08.018582+0000","last_unstale":"2024-04-25T10:29:10.024603+0000","last_undegraded":"2024-04-25T10:29:10.024603+0000","last_fullsized":"2024-04-25T10:29:10.024603+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:29:06.989570+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:29:06.989570+0000","last_clean_scrub_stamp":"2024-04-25T10:29:06.989570+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T19:23:11.583888+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.13","version":"0'0","reported_seq":12,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-04-25T10:29:10.024281+0000","last_change":"2024-04-25T10:29:09.023995+0000","last_active":"2024-04-25T10:29:10.024281+0000","last_peered":"2024-04-25T10:29:10.024281+0000","last_clean":"2024-04-25T10:29:10.024281+0000","last_became_active":"2024-04-25T10:29:09.023690+0000","last_became_peered":"2024-04-25T10:29:09.023690+0000","last_unstale":"2024-04-25T10:29:10.024281+0000","last_undegraded":"2024-04-25T10:29:10.024281+0000","last_fullsized":"2024-04-25T10:29:10.024281+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:29:07.995096+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:29:07.995096+0000","last_clean_scrub_stamp":"2024-04-25T10:29:07.995096+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T15:28:23.689564+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.11","version":"0'0","reported_seq":14,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-04-25T10:29:10.037576+0000","last_change":"2024-04-25T10:29:08.027174+0000","last_active":"2024-04-25T10:29:10.037576+0000","last_peered":"2024-04-25T10:29:10.037576+0000","last_clean":"2024-04-25T10:29:10.037576+0000","last_became_active":"2024-04-25T10:29:08.026809+0000","last_became_peered":"2024-04-25T10:29:08.026809+0000","last_unstale":"2024-04-25T10:29:10.037576+0000","last_undegraded":"2024-04-25T10:29:10.037576+0000","last_fullsized":"2024-04-25T10:29:10.037576+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:29:06.989570+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:29:06.989570+0000","last_clean_scrub_stamp":"2024-04-25T10:29:06.989570+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T10:35:30.355988+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.10","version":"0'0","reported_seq":12,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-04-25T10:29:10.187809+0000","last_change":"2024-04-25T10:29:09.041722+0000","last_active":"2024-04-25T10:29:10.187809+0000","last_peered":"2024-04-25T10:29:10.187809+0000","last_clean":"2024-04-25T10:29:10.187809+0000","last_became_active":"2024-04-25T10:29:09.040971+0000","last_became_peered":"2024-04-25T10:29:09.040971+0000","last_unstale":"2024-04-25T10:29:10.187809+0000","last_undegraded":"2024-04-25T10:29:10.187809+0000","last_fullsized":"2024-04-25T10:29:10.187809+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:29:07.995096+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:29:07.995096+0000","last_clean_scrub_stamp":"2024-04-25T10:29:07.995096+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T10:35:31.361514+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.10","version":"0'0","reported_seq":9,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-04-25T10:29:08.019405+0000","last_change":"2024-04-25T10:29:08.014110+0000","last_active":"2024-04-25T10:29:08.019405+0000","last_peered":"2024-04-25T10:29:08.019405+0000","last_clean":"2024-04-25T10:29:08.019405+0000","last_became_active":"2024-04-25T10:29:08.013543+0000","last_became_peered":"2024-04-25T10:29:08.013543+0000","last_unstale":"2024-04-25T10:29:08.019405+0000","last_undegraded":"2024-04-25T10:29:08.019405+0000","last_fullsized":"2024-04-25T10:29:08.019405+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:29:06.989570+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:29:06.989570+0000","last_clean_scrub_stamp":"2024-04-25T10:29:06.989570+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T21:35:43.931886+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7],"acting":[4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.11","version":"0'0","reported_seq":12,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-04-25T10:29:10.024238+0000","last_change":"2024-04-25T10:29:09.020490+0000","last_active":"2024-04-25T10:29:10.024238+0000","last_peered":"2024-04-25T10:29:10.024238+0000","last_clean":"2024-04-25T10:29:10.024238+0000","last_became_active":"2024-04-25T10:29:09.020178+0000","last_became_peered":"2024-04-25T10:29:09.020178+0000","last_unstale":"2024-04-25T10:29:10.024238+0000","last_undegraded":"2024-04-25T10:29:10.024238+0000","last_fullsized":"2024-04-25T10:29:10.024238+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:29:07.995096+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:29:07.995096+0000","last_clean_scrub_stamp":"2024-04-25T10:29:07.995096+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T20:37:20.726766+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.f","version":"22'1","reported_seq":15,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-04-25T10:29:10.038104+0000","last_change":"2024-04-25T10:29:08.029879+0000","last_active":"2024-04-25T10:29:10.038104+0000","last_peered":"2024-04-25T10:29:10.038104+0000","last_clean":"2024-04-25T10:29:10.038104+0000","last_became_active":"2024-04-25T10:29:08.029549+0000","last_became_peered":"2024-04-25T10:29:08.029549+0000","last_unstale":"2024-04-25T10:29:10.038104+0000","last_undegraded":"2024-04-25T10:29:10.038104+0000","last_fullsized":"2024-04-25T10:29:10.038104+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:29:06.989570+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:29:06.989570+0000","last_clean_scrub_stamp":"2024-04-25T10:29:06.989570+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T15:37:05.717854+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.e","version":"0'0","reported_seq":12,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-04-25T10:29:10.026273+0000","last_change":"2024-04-25T10:29:09.033703+0000","last_active":"2024-04-25T10:29:10.026273+0000","last_peered":"2024-04-25T10:29:10.026273+0000","last_clean":"2024-04-25T10:29:10.026273+0000","last_became_active":"2024-04-25T10:29:09.033318+0000","last_became_peered":"2024-04-25T10:29:09.033318+0000","last_unstale":"2024-04-25T10:29:10.026273+0000","last_undegraded":"2024-04-25T10:29:10.026273+0000","last_fullsized":"2024-04-25T10:29:10.026273+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:29:07.995096+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:29:07.995096+0000","last_clean_scrub_stamp":"2024-04-25T10:29:07.995096+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T12:16:47.983013+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.e","version":"0'0","reported_seq":9,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-04-25T10:29:08.027622+0000","last_change":"2024-04-25T10:29:08.020703+0000","last_active":"2024-04-25T10:29:08.027622+0000","last_peered":"2024-04-25T10:29:08.027622+0000","last_clean":"2024-04-25T10:29:08.027622+0000","last_became_active":"2024-04-25T10:29:08.020321+0000","last_became_peered":"2024-04-25T10:29:08.020321+0000","last_unstale":"2024-04-25T10:29:08.027622+0000","last_undegraded":"2024-04-25T10:29:08.027622+0000","last_fullsized":"2024-04-25T10:29:08.027622+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:29:06.989570+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:29:06.989570+0000","last_clean_scrub_stamp":"2024-04-25T10:29:06.989570+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T15:21:12.948208+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5],"acting":[4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.f","version":"0'0","reported_seq":12,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-04-25T10:29:10.038152+0000","last_change":"2024-04-25T10:29:09.022708+0000","last_active":"2024-04-25T10:29:10.038152+0000","last_peered":"2024-04-25T10:29:10.038152+0000","last_clean":"2024-04-25T10:29:10.038152+0000","last_became_active":"2024-04-25T10:29:09.020856+0000","last_became_peered":"2024-04-25T10:29:09.020856+0000","last_unstale":"2024-04-25T10:29:10.038152+0000","last_undegraded":"2024-04-25T10:29:10.038152+0000","last_fullsized":"2024-04-25T10:29:10.038152+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:29:07.995096+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:29:07.995096+0000","last_clean_scrub_stamp":"2024-04-25T10:29:07.995096+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T15:28:23.689564+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.d","version":"22'1","reported_seq":15,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-04-25T10:29:10.189469+0000","last_change":"2024-04-25T10:29:08.019488+0000","last_active":"2024-04-25T10:29:10.189469+0000","last_peered":"2024-04-25T10:29:10.189469+0000","last_clean":"2024-04-25T10:29:10.189469+0000","last_became_active":"2024-04-25T10:29:08.018971+0000","last_became_peered":"2024-04-25T10:29:08.018971+0000","last_unstale":"2024-04-25T10:29:10.189469+0000","last_undegraded":"2024-04-25T10:29:10.189469+0000","last_fullsized":"2024-04-25T10:29:10.189469+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:29:06.989570+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:29:06.989570+0000","last_clean_scrub_stamp":"2024-04-25T10:29:06.989570+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T20:22:56.380539+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":70,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.c","version":"0'0","reported_seq":12,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-04-25T10:29:10.037902+0000","last_change":"2024-04-25T10:29:09.029666+0000","last_active":"2024-04-25T10:29:10.037902+0000","last_peered":"2024-04-25T10:29:10.037902+0000","last_clean":"2024-04-25T10:29:10.037902+0000","last_became_active":"2024-04-25T10:29:09.029184+0000","last_became_peered":"2024-04-25T10:29:09.029184+0000","last_unstale":"2024-04-25T10:29:10.037902+0000","last_undegraded":"2024-04-25T10:29:10.037902+0000","last_fullsized":"2024-04-25T10:29:10.037902+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:29:07.995096+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:29:07.995096+0000","last_clean_scrub_stamp":"2024-04-25T10:29:07.995096+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T13:54:22.968216+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.c","version":"0'0","reported_seq":14,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-04-25T10:29:10.189541+0000","last_change":"2024-04-25T10:29:08.012124+0000","last_active":"2024-04-25T10:29:10.189541+0000","last_peered":"2024-04-25T10:29:10.189541+0000","last_clean":"2024-04-25T10:29:10.189541+0000","last_became_active":"2024-04-25T10:29:08.011817+0000","last_became_peered":"2024-04-25T10:29:08.011817+0000","last_unstale":"2024-04-25T10:29:10.189541+0000","last_undegraded":"2024-04-25T10:29:10.189541+0000","last_fullsized":"2024-04-25T10:29:10.189541+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:29:06.989570+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:29:06.989570+0000","last_clean_scrub_stamp":"2024-04-25T10:29:06.989570+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T20:22:56.380539+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.d","version":"0'0","reported_seq":12,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-04-25T10:29:10.024815+0000","last_change":"2024-04-25T10:29:09.017227+0000","last_active":"2024-04-25T10:29:10.024815+0000","last_peered":"2024-04-25T10:29:10.024815+0000","last_clean":"2024-04-25T10:29:10.024815+0000","last_became_active":"2024-04-25T10:29:09.016911+0000","last_became_peered":"2024-04-25T10:29:09.016911+0000","last_unstale":"2024-04-25T10:29:10.024815+0000","last_undegraded":"2024-04-25T10:29:10.024815+0000","last_fullsized":"2024-04-25T10:29:10.024815+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:29:07.995096+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:29:07.995096+0000","last_clean_scrub_stamp":"2024-04-25T10:29:07.995096+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T18:51:41.934269+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.0","version":"18'72","reported_seq":110,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-04-25T10:29:10.187517+0000","last_change":"2024-04-25T10:28:57.972696+0000","last_active":"2024-04-25T10:29:10.187517+0000","last_peered":"2024-04-25T10:29:10.187517+0000","last_clean":"2024-04-25T10:29:10.187517+0000","last_became_active":"2024-04-25T10:28:57.972373+0000","last_became_peered":"2024-04-25T10:28:57.972373+0000","last_unstale":"2024-04-25T10:29:10.187517+0000","last_undegraded":"2024-04-25T10:29:10.187517+0000","last_fullsized":"2024-04-25T10:29:10.187517+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:28:56.959439+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:28:56.959439+0000","last_clean_scrub_stamp":"2024-04-25T10:28:56.959439+0000","objects_scrubbed":0,"log_size":72,"log_dups_size":0,"ondisk_log_size":72,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T17:34:34.057408+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":66,"num_read_kb":55,"num_write":101,"num_write_kb":1628,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.3","version":"0'0","reported_seq":14,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-04-25T10:29:10.024562+0000","last_change":"2024-04-25T10:29:08.025080+0000","last_active":"2024-04-25T10:29:10.024562+0000","last_peered":"2024-04-25T10:29:10.024562+0000","last_clean":"2024-04-25T10:29:10.024562+0000","last_became_active":"2024-04-25T10:29:08.024747+0000","last_became_peered":"2024-04-25T10:29:08.024747+0000","last_unstale":"2024-04-25T10:29:10.024562+0000","last_undegraded":"2024-04-25T10:29:10.024562+0000","last_fullsized":"2024-04-25T10:29:10.024562+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:29:06.989570+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:29:06.989570+0000","last_clean_scrub_stamp":"2024-04-25T10:29:06.989570+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T22:07:59.361905+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.2","version":"0'0","reported_seq":1,"reported_epoch":21,"state":"creating+peering","last_fresh":"2024-04-25T10:29:08.008681+0000","last_change":"2024-04-25T10:29:08.008681+0000","last_active":"2024-04-25T10:29:07.995096+0000","last_peered":"2024-04-25T10:29:07.995096+0000","last_clean":"2024-04-25T10:29:07.995096+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T10:29:08.008681+0000","last_undegraded":"2024-04-25T10:29:08.008681+0000","last_fullsized":"2024-04-25T10:29:08.008681+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:29:07.995096+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:29:07.995096+0000","last_clean_scrub_stamp":"2024-04-25T10:29:07.995096+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.0","version":"0'0","reported_seq":14,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-04-25T10:29:10.024309+0000","last_change":"2024-04-25T10:29:08.027479+0000","last_active":"2024-04-25T10:29:10.024309+0000","last_peered":"2024-04-25T10:29:10.024309+0000","last_clean":"2024-04-25T10:29:10.024309+0000","last_became_active":"2024-04-25T10:29:08.027178+0000","last_became_peered":"2024-04-25T10:29:08.027178+0000","last_unstale":"2024-04-25T10:29:10.024309+0000","last_undegraded":"2024-04-25T10:29:10.024309+0000","last_fullsized":"2024-04-25T10:29:10.024309+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:29:06.989570+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:29:06.989570+0000","last_clean_scrub_stamp":"2024-04-25T10:29:06.989570+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T19:54:16.697792+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.1","version":"0'0","reported_seq":12,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-04-25T10:29:10.188195+0000","last_change":"2024-04-25T10:29:09.028777+0000","last_active":"2024-04-25T10:29:10.188195+0000","last_peered":"2024-04-25T10:29:10.188195+0000","last_clean":"2024-04-25T10:29:10.188195+0000","last_became_active":"2024-04-25T10:29:09.027709+0000","last_became_peered":"2024-04-25T10:29:09.027709+0000","last_unstale":"2024-04-25T10:29:10.188195+0000","last_undegraded":"2024-04-25T10:29:10.188195+0000","last_fullsized":"2024-04-25T10:29:10.188195+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:29:07.995096+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:29:07.995096+0000","last_clean_scrub_stamp":"2024-04-25T10:29:07.995096+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T22:08:00.367431+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,2],"acting":[6,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.1","version":"0'0","reported_seq":14,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-04-25T10:29:10.188429+0000","last_change":"2024-04-25T10:29:08.020944+0000","last_active":"2024-04-25T10:29:10.188429+0000","last_peered":"2024-04-25T10:29:10.188429+0000","last_clean":"2024-04-25T10:29:10.188429+0000","last_became_active":"2024-04-25T10:29:08.020259+0000","last_became_peered":"2024-04-25T10:29:08.020259+0000","last_unstale":"2024-04-25T10:29:10.188429+0000","last_undegraded":"2024-04-25T10:29:10.188429+0000","last_fullsized":"2024-04-25T10:29:10.188429+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:29:06.989570+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:29:06.989570+0000","last_clean_scrub_stamp":"2024-04-25T10:29:06.989570+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T16:31:11.685101+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.0","version":"0'0","reported_seq":12,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-04-25T10:29:10.189305+0000","last_change":"2024-04-25T10:29:09.035455+0000","last_active":"2024-04-25T10:29:10.189305+0000","last_peered":"2024-04-25T10:29:10.189305+0000","last_clean":"2024-04-25T10:29:10.189305+0000","last_became_active":"2024-04-25T10:29:09.035130+0000","last_became_peered":"2024-04-25T10:29:09.035130+0000","last_unstale":"2024-04-25T10:29:10.189305+0000","last_undegraded":"2024-04-25T10:29:10.189305+0000","last_fullsized":"2024-04-25T10:29:10.189305+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:29:07.995096+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:29:07.995096+0000","last_clean_scrub_stamp":"2024-04-25T10:29:07.995096+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T11:52:24.415332+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.2","version":"0'0","reported_seq":14,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-04-25T10:29:10.024714+0000","last_change":"2024-04-25T10:29:08.021694+0000","last_active":"2024-04-25T10:29:10.024714+0000","last_peered":"2024-04-25T10:29:10.024714+0000","last_clean":"2024-04-25T10:29:10.024714+0000","last_became_active":"2024-04-25T10:29:08.021052+0000","last_became_peered":"2024-04-25T10:29:08.021052+0000","last_unstale":"2024-04-25T10:29:10.024714+0000","last_undegraded":"2024-04-25T10:29:10.024714+0000","last_fullsized":"2024-04-25T10:29:10.024714+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:29:06.989570+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:29:06.989570+0000","last_clean_scrub_stamp":"2024-04-25T10:29:06.989570+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T21:14:38.656431+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.3","version":"0'0","reported_seq":12,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-04-25T10:29:10.037784+0000","last_change":"2024-04-25T10:29:09.021686+0000","last_active":"2024-04-25T10:29:10.037784+0000","last_peered":"2024-04-25T10:29:10.037784+0000","last_clean":"2024-04-25T10:29:10.037784+0000","last_became_active":"2024-04-25T10:29:09.020672+0000","last_became_peered":"2024-04-25T10:29:09.020672+0000","last_unstale":"2024-04-25T10:29:10.037784+0000","last_undegraded":"2024-04-25T10:29:10.037784+0000","last_fullsized":"2024-04-25T10:29:10.037784+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:29:07.995096+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:29:07.995096+0000","last_clean_scrub_stamp":"2024-04-25T10:29:07.995096+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T20:52:02.098872+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.4","version":"0'0","reported_seq":14,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-04-25T10:29:10.189133+0000","last_change":"2024-04-25T10:29:08.025582+0000","last_active":"2024-04-25T10:29:10.189133+0000","last_peered":"2024-04-25T10:29:10.189133+0000","last_clean":"2024-04-25T10:29:10.189133+0000","last_became_active":"2024-04-25T10:29:08.024961+0000","last_became_peered":"2024-04-25T10:29:08.024961+0000","last_unstale":"2024-04-25T10:29:10.189133+0000","last_undegraded":"2024-04-25T10:29:10.189133+0000","last_fullsized":"2024-04-25T10:29:10.189133+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:29:06.989570+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:29:06.989570+0000","last_clean_scrub_stamp":"2024-04-25T10:29:06.989570+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T10:32:06.103123+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6],"acting":[5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.5","version":"0'0","reported_seq":12,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-04-25T10:29:10.026083+0000","last_change":"2024-04-25T10:29:09.028003+0000","last_active":"2024-04-25T10:29:10.026083+0000","last_peered":"2024-04-25T10:29:10.026083+0000","last_clean":"2024-04-25T10:29:10.026083+0000","last_became_active":"2024-04-25T10:29:09.027654+0000","last_became_peered":"2024-04-25T10:29:09.027654+0000","last_unstale":"2024-04-25T10:29:10.026083+0000","last_undegraded":"2024-04-25T10:29:10.026083+0000","last_fullsized":"2024-04-25T10:29:10.026083+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:29:07.995096+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:29:07.995096+0000","last_clean_scrub_stamp":"2024-04-25T10:29:07.995096+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T19:54:17.703318+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,6],"acting":[2,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.5","version":"0'0","reported_seq":14,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-04-25T10:29:10.024926+0000","last_change":"2024-04-25T10:29:08.026181+0000","last_active":"2024-04-25T10:29:10.024926+0000","last_peered":"2024-04-25T10:29:10.024926+0000","last_clean":"2024-04-25T10:29:10.024926+0000","last_became_active":"2024-04-25T10:29:08.025780+0000","last_became_peered":"2024-04-25T10:29:08.025780+0000","last_unstale":"2024-04-25T10:29:10.024926+0000","last_undegraded":"2024-04-25T10:29:10.024926+0000","last_fullsized":"2024-04-25T10:29:10.024926+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:29:06.989570+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:29:06.989570+0000","last_clean_scrub_stamp":"2024-04-25T10:29:06.989570+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T12:16:46.977487+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.4","version":"0'0","reported_seq":12,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-04-25T10:29:10.189423+0000","last_change":"2024-04-25T10:29:09.037824+0000","last_active":"2024-04-25T10:29:10.189423+0000","last_peered":"2024-04-25T10:29:10.189423+0000","last_clean":"2024-04-25T10:29:10.189423+0000","last_became_active":"2024-04-25T10:29:09.037478+0000","last_became_peered":"2024-04-25T10:29:09.037478+0000","last_unstale":"2024-04-25T10:29:10.189423+0000","last_undegraded":"2024-04-25T10:29:10.189423+0000","last_fullsized":"2024-04-25T10:29:10.189423+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:29:07.995096+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:29:07.995096+0000","last_clean_scrub_stamp":"2024-04-25T10:29:07.995096+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T15:02:17.480054+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.6","version":"0'0","reported_seq":9,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-04-25T10:29:08.027595+0000","last_change":"2024-04-25T10:29:08.020327+0000","last_active":"2024-04-25T10:29:08.027595+0000","last_peered":"2024-04-25T10:29:08.027595+0000","last_clean":"2024-04-25T10:29:08.027595+0000","last_became_active":"2024-04-25T10:29:08.019988+0000","last_became_peered":"2024-04-25T10:29:08.019988+0000","last_unstale":"2024-04-25T10:29:08.027595+0000","last_undegraded":"2024-04-25T10:29:08.027595+0000","last_fullsized":"2024-04-25T10:29:08.027595+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:29:06.989570+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:29:06.989570+0000","last_clean_scrub_stamp":"2024-04-25T10:29:06.989570+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T12:07:01.943484+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.7","version":"0'0","reported_seq":1,"reported_epoch":21,"state":"creating+peering","last_fresh":"2024-04-25T10:29:08.013060+0000","last_change":"2024-04-25T10:29:08.013060+0000","last_active":"2024-04-25T10:29:07.995096+0000","last_peered":"2024-04-25T10:29:07.995096+0000","last_clean":"2024-04-25T10:29:07.995096+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T10:29:08.013060+0000","last_undegraded":"2024-04-25T10:29:08.013060+0000","last_fullsized":"2024-04-25T10:29:08.013060+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:29:07.995096+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:29:07.995096+0000","last_clean_scrub_stamp":"2024-04-25T10:29:07.995096+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.7","version":"22'2","reported_seq":16,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-04-25T10:29:10.024876+0000","last_change":"2024-04-25T10:29:08.026080+0000","last_active":"2024-04-25T10:29:10.024876+0000","last_peered":"2024-04-25T10:29:10.024876+0000","last_clean":"2024-04-25T10:29:10.024876+0000","last_became_active":"2024-04-25T10:29:08.025707+0000","last_became_peered":"2024-04-25T10:29:08.025707+0000","last_unstale":"2024-04-25T10:29:10.024876+0000","last_undegraded":"2024-04-25T10:29:10.024876+0000","last_fullsized":"2024-04-25T10:29:10.024876+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:29:06.989570+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:29:06.989570+0000","last_clean_scrub_stamp":"2024-04-25T10:29:06.989570+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T10:55:00.499631+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.6","version":"0'0","reported_seq":1,"reported_epoch":21,"state":"creating+peering","last_fresh":"2024-04-25T10:29:08.009604+0000","last_change":"2024-04-25T10:29:08.009604+0000","last_active":"2024-04-25T10:29:07.995096+0000","last_peered":"2024-04-25T10:29:07.995096+0000","last_clean":"2024-04-25T10:29:07.995096+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T10:29:08.009604+0000","last_undegraded":"2024-04-25T10:29:08.009604+0000","last_fullsized":"2024-04-25T10:29:08.009604+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:29:07.995096+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:29:07.995096+0000","last_clean_scrub_stamp":"2024-04-25T10:29:07.995096+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7],"acting":[4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.8","version":"0'0","reported_seq":14,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-04-25T10:29:10.037271+0000","last_change":"2024-04-25T10:29:08.024845+0000","last_active":"2024-04-25T10:29:10.037271+0000","last_peered":"2024-04-25T10:29:10.037271+0000","last_clean":"2024-04-25T10:29:10.037271+0000","last_became_active":"2024-04-25T10:29:08.024484+0000","last_became_peered":"2024-04-25T10:29:08.024484+0000","last_unstale":"2024-04-25T10:29:10.037271+0000","last_undegraded":"2024-04-25T10:29:10.037271+0000","last_fullsized":"2024-04-25T10:29:10.037271+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:29:06.989570+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:29:06.989570+0000","last_clean_scrub_stamp":"2024-04-25T10:29:06.989570+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T14:12:57.950482+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.9","version":"0'0","reported_seq":12,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-04-25T10:29:10.024346+0000","last_change":"2024-04-25T10:29:09.022502+0000","last_active":"2024-04-25T10:29:10.024346+0000","last_peered":"2024-04-25T10:29:10.024346+0000","last_clean":"2024-04-25T10:29:10.024346+0000","last_became_active":"2024-04-25T10:29:09.022184+0000","last_became_peered":"2024-04-25T10:29:09.022184+0000","last_unstale":"2024-04-25T10:29:10.024346+0000","last_undegraded":"2024-04-25T10:29:10.024346+0000","last_fullsized":"2024-04-25T10:29:10.024346+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:29:07.995096+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:29:07.995096+0000","last_clean_scrub_stamp":"2024-04-25T10:29:07.995096+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T19:56:27.663748+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.9","version":"0'0","reported_seq":9,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-04-25T10:29:08.027947+0000","last_change":"2024-04-25T10:29:08.024989+0000","last_active":"2024-04-25T10:29:08.027947+0000","last_peered":"2024-04-25T10:29:08.027947+0000","last_clean":"2024-04-25T10:29:08.027947+0000","last_became_active":"2024-04-25T10:29:08.024655+0000","last_became_peered":"2024-04-25T10:29:08.024655+0000","last_unstale":"2024-04-25T10:29:08.027947+0000","last_undegraded":"2024-04-25T10:29:08.027947+0000","last_fullsized":"2024-04-25T10:29:08.027947+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:29:06.989570+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:29:06.989570+0000","last_clean_scrub_stamp":"2024-04-25T10:29:06.989570+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T13:46:20.443676+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.8","version":"0'0","reported_seq":1,"reported_epoch":21,"state":"creating+peering","last_fresh":"2024-04-25T10:29:08.009188+0000","last_change":"2024-04-25T10:29:08.009188+0000","last_active":"2024-04-25T10:29:07.995096+0000","last_peered":"2024-04-25T10:29:07.995096+0000","last_clean":"2024-04-25T10:29:07.995096+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T10:29:08.009188+0000","last_undegraded":"2024-04-25T10:29:08.009188+0000","last_fullsized":"2024-04-25T10:29:08.009188+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:29:07.995096+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:29:07.995096+0000","last_clean_scrub_stamp":"2024-04-25T10:29:07.995096+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7],"acting":[4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.a","version":"0'0","reported_seq":14,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-04-25T10:29:10.188957+0000","last_change":"2024-04-25T10:29:08.020363+0000","last_active":"2024-04-25T10:29:10.188957+0000","last_peered":"2024-04-25T10:29:10.188957+0000","last_clean":"2024-04-25T10:29:10.188957+0000","last_became_active":"2024-04-25T10:29:08.020037+0000","last_became_peered":"2024-04-25T10:29:08.020037+0000","last_unstale":"2024-04-25T10:29:10.188957+0000","last_undegraded":"2024-04-25T10:29:10.188957+0000","last_fullsized":"2024-04-25T10:29:10.188957+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:29:06.989570+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:29:06.989570+0000","last_clean_scrub_stamp":"2024-04-25T10:29:06.989570+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T10:32:06.103123+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.b","version":"0'0","reported_seq":12,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-04-25T10:29:10.189294+0000","last_change":"2024-04-25T10:29:09.028820+0000","last_active":"2024-04-25T10:29:10.189294+0000","last_peered":"2024-04-25T10:29:10.189294+0000","last_clean":"2024-04-25T10:29:10.189294+0000","last_became_active":"2024-04-25T10:29:09.028426+0000","last_became_peered":"2024-04-25T10:29:09.028426+0000","last_unstale":"2024-04-25T10:29:10.189294+0000","last_undegraded":"2024-04-25T10:29:10.189294+0000","last_fullsized":"2024-04-25T10:29:10.189294+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:29:07.995096+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:29:07.995096+0000","last_clean_scrub_stamp":"2024-04-25T10:29:07.995096+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T19:54:17.703318+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.b","version":"22'2","reported_seq":16,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-04-25T10:29:10.024445+0000","last_change":"2024-04-25T10:29:08.025011+0000","last_active":"2024-04-25T10:29:10.024445+0000","last_peered":"2024-04-25T10:29:10.024445+0000","last_clean":"2024-04-25T10:29:10.024445+0000","last_became_active":"2024-04-25T10:29:08.024667+0000","last_became_peered":"2024-04-25T10:29:08.024667+0000","last_unstale":"2024-04-25T10:29:10.024445+0000","last_undegraded":"2024-04-25T10:29:10.024445+0000","last_fullsized":"2024-04-25T10:29:10.024445+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:29:06.989570+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:29:06.989570+0000","last_clean_scrub_stamp":"2024-04-25T10:29:06.989570+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T15:37:05.717854+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.a","version":"0'0","reported_seq":12,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-04-25T10:29:10.024394+0000","last_change":"2024-04-25T10:29:09.017160+0000","last_active":"2024-04-25T10:29:10.024394+0000","last_peered":"2024-04-25T10:29:10.024394+0000","last_clean":"2024-04-25T10:29:10.024394+0000","last_became_active":"2024-04-25T10:29:09.016821+0000","last_became_peered":"2024-04-25T10:29:09.016821+0000","last_unstale":"2024-04-25T10:29:10.024394+0000","last_undegraded":"2024-04-25T10:29:10.024394+0000","last_fullsized":"2024-04-25T10:29:10.024394+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:29:07.995096+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:29:07.995096+0000","last_clean_scrub_stamp":"2024-04-25T10:29:07.995096+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T20:40:19.840339+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.1d","version":"0'0","reported_seq":12,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-04-25T10:29:10.024622+0000","last_change":"2024-04-25T10:29:09.030341+0000","last_active":"2024-04-25T10:29:10.024622+0000","last_peered":"2024-04-25T10:29:10.024622+0000","last_clean":"2024-04-25T10:29:10.024622+0000","last_became_active":"2024-04-25T10:29:09.029988+0000","last_became_peered":"2024-04-25T10:29:09.029988+0000","last_unstale":"2024-04-25T10:29:10.024622+0000","last_undegraded":"2024-04-25T10:29:10.024622+0000","last_fullsized":"2024-04-25T10:29:10.024622+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:29:07.995096+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:29:07.995096+0000","last_clean_scrub_stamp":"2024-04-25T10:29:07.995096+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T11:40:18.624659+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.1c","version":"22'2","reported_seq":16,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-04-25T10:29:10.037969+0000","last_change":"2024-04-25T10:29:08.027197+0000","last_active":"2024-04-25T10:29:10.037969+0000","last_peered":"2024-04-25T10:29:10.037969+0000","last_clean":"2024-04-25T10:29:10.037969+0000","last_became_active":"2024-04-25T10:29:08.026872+0000","last_became_peered":"2024-04-25T10:29:08.026872+0000","last_unstale":"2024-04-25T10:29:10.037969+0000","last_undegraded":"2024-04-25T10:29:10.037969+0000","last_fullsized":"2024-04-25T10:29:10.037969+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:29:06.989570+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:29:06.989570+0000","last_clean_scrub_stamp":"2024-04-25T10:29:06.989570+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T15:58:35.843305+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.1c","version":"0'0","reported_seq":12,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-04-25T10:29:10.187853+0000","last_change":"2024-04-25T10:29:09.032731+0000","last_active":"2024-04-25T10:29:10.187853+0000","last_peered":"2024-04-25T10:29:10.187853+0000","last_clean":"2024-04-25T10:29:10.187853+0000","last_became_active":"2024-04-25T10:29:09.032383+0000","last_became_peered":"2024-04-25T10:29:09.032383+0000","last_unstale":"2024-04-25T10:29:10.187853+0000","last_undegraded":"2024-04-25T10:29:10.187853+0000","last_fullsized":"2024-04-25T10:29:10.187853+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:29:07.995096+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:29:07.995096+0000","last_clean_scrub_stamp":"2024-04-25T10:29:07.995096+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T19:54:17.703318+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.1d","version":"22'2","reported_seq":16,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-04-25T10:29:10.037807+0000","last_change":"2024-04-25T10:29:08.026410+0000","last_active":"2024-04-25T10:29:10.037807+0000","last_peered":"2024-04-25T10:29:10.037807+0000","last_clean":"2024-04-25T10:29:10.037807+0000","last_became_active":"2024-04-25T10:29:08.026071+0000","last_became_peered":"2024-04-25T10:29:08.026071+0000","last_unstale":"2024-04-25T10:29:10.037807+0000","last_undegraded":"2024-04-25T10:29:10.037807+0000","last_fullsized":"2024-04-25T10:29:10.037807+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:29:06.989570+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:29:06.989570+0000","last_clean_scrub_stamp":"2024-04-25T10:29:06.989570+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T19:54:16.697792+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.1f","version":"0'0","reported_seq":12,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-04-25T10:29:10.189240+0000","last_change":"2024-04-25T10:29:09.032735+0000","last_active":"2024-04-25T10:29:10.189240+0000","last_peered":"2024-04-25T10:29:10.189240+0000","last_clean":"2024-04-25T10:29:10.189240+0000","last_became_active":"2024-04-25T10:29:09.032227+0000","last_became_peered":"2024-04-25T10:29:09.032227+0000","last_unstale":"2024-04-25T10:29:10.189240+0000","last_undegraded":"2024-04-25T10:29:10.189240+0000","last_fullsized":"2024-04-25T10:29:10.189240+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:29:07.995096+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:29:07.995096+0000","last_clean_scrub_stamp":"2024-04-25T10:29:07.995096+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T11:31:45.283247+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.1e","version":"22'1","reported_seq":15,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-04-25T10:29:10.189728+0000","last_change":"2024-04-25T10:29:08.017025+0000","last_active":"2024-04-25T10:29:10.189728+0000","last_peered":"2024-04-25T10:29:10.189728+0000","last_clean":"2024-04-25T10:29:10.189728+0000","last_became_active":"2024-04-25T10:29:08.016702+0000","last_became_peered":"2024-04-25T10:29:08.016702+0000","last_unstale":"2024-04-25T10:29:10.189728+0000","last_undegraded":"2024-04-25T10:29:10.189728+0000","last_fullsized":"2024-04-25T10:29:10.189728+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:29:06.989570+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:29:06.989570+0000","last_clean_scrub_stamp":"2024-04-25T10:29:06.989570+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T14:12:57.950482+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.1e","version":"0'0","reported_seq":12,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-04-25T10:29:10.189730+0000","last_change":"2024-04-25T10:29:09.036208+0000","last_active":"2024-04-25T10:29:10.189730+0000","last_peered":"2024-04-25T10:29:10.189730+0000","last_clean":"2024-04-25T10:29:10.189730+0000","last_became_active":"2024-04-25T10:29:09.034890+0000","last_became_peered":"2024-04-25T10:29:09.034890+0000","last_unstale":"2024-04-25T10:29:10.189730+0000","last_undegraded":"2024-04-25T10:29:10.189730+0000","last_fullsized":"2024-04-25T10:29:10.189730+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:29:07.995096+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:29:07.995096+0000","last_clean_scrub_stamp":"2024-04-25T10:29:07.995096+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T18:44:23.610374+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.1f","version":"0'0","reported_seq":9,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-04-25T10:29:08.027976+0000","last_change":"2024-04-25T10:29:08.026202+0000","last_active":"2024-04-25T10:29:08.027976+0000","last_peered":"2024-04-25T10:29:08.027976+0000","last_clean":"2024-04-25T10:29:08.027976+0000","last_became_active":"2024-04-25T10:29:08.025877+0000","last_became_peered":"2024-04-25T10:29:08.025877+0000","last_unstale":"2024-04-25T10:29:08.027976+0000","last_undegraded":"2024-04-25T10:29:08.027976+0000","last_fullsized":"2024-04-25T10:29:08.027976+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:29:06.989570+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:29:06.989570+0000","last_clean_scrub_stamp":"2024-04-25T10:29:06.989570+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T18:56:03.528807+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]}],"pool_stats":[{"poolid":3,"num_pg":64,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":128,"acting":128,"num_store_stats":8},{"poolid":2,"num_pg":32,"stat_sum":{"num_bytes":1828,"num_objects":14,"num_object_clones":0,"num_object_copies":28,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":14,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":25,"num_write_kb":13,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":7,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":45056,"data_stored":2536,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":24,"ondisk_log_size":24,"up":64,"acting":64,"num_store_stats":8},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":66,"num_read_kb":55,"num_write":101,"num_write_kb":1628,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1187840,"data_stored":1180736,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":72,"ondisk_log_size":72,"up":2,"acting":2,"num_store_stats":2}],"osd_stats":[{"osd":7,"up_from":15,"seq":64424509445,"num_pgs":3,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27244,"kb_used_data":224,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344596,"statfs":{"total":96636764160,"available":96608866304,"internally_reserved":0,"allocated":229376,"data_stored":75059,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1590,"internal_metadata":27654602},"hb_peers":[0,1,2,3,4,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[]},{"osd":6,"up_from":15,"seq":64424509446,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27236,"kb_used_data":216,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344604,"statfs":{"total":96636764160,"available":96608874496,"internally_reserved":0,"allocated":221184,"data_stored":74935,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":27654598},"hb_peers":[0,1,2,3,4,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[]},{"osd":5,"up_from":15,"seq":64424509447,"num_pgs":1,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27824,"kb_used_data":804,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344016,"statfs":{"total":96636764160,"available":96608272384,"internally_reserved":0,"allocated":823296,"data_stored":665395,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":6,"apply_latency_ms":6,"commit_latency_ns":6000000,"apply_latency_ns":6000000},"alerts":[]},{"osd":3,"up_from":15,"seq":64424509447,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27240,"kb_used_data":220,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344600,"statfs":{"total":96636764160,"available":96608870400,"internally_reserved":0,"allocated":225280,"data_stored":75517,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1593,"internal_metadata":27654599},"hb_peers":[0,1,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[]},{"osd":2,"up_from":15,"seq":64424509447,"num_pgs":1,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27248,"kb_used_data":228,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344592,"statfs":{"total":96636764160,"available":96608862208,"internally_reserved":0,"allocated":233472,"data_stored":76547,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[]},{"osd":1,"up_from":15,"seq":64424509445,"num_pgs":12,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27820,"kb_used_data":800,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344020,"statfs":{"total":96636764160,"available":96608276480,"internally_reserved":0,"allocated":819200,"data_stored":665373,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[]},{"osd":0,"up_from":15,"seq":64424509446,"num_pgs":35,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27244,"kb_used_data":224,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344596,"statfs":{"total":96636764160,"available":96608866304,"internally_reserved":0,"allocated":229376,"data_stored":74991,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[1,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":5,"apply_latency_ms":5,"commit_latency_ns":5000000,"apply_latency_ns":5000000},"alerts":[]},{"osd":4,"up_from":13,"seq":55834574853,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27212,"kb_used_data":192,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344628,"statfs":{"total":96636764160,"available":96608899072,"internally_reserved":0,"allocated":196608,"data_stored":61885,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":56,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":70,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":1612,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":582,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":92,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":124,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-04-25T10:29:13.198 DEBUG:teuthology.orchestra.run.smithi037:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-04-25T10:29:13.345 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:13.342+0000 7fb54c5e3640 1 Processor -- start 2024-04-25T10:29:13.345 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:13.342+0000 7fb54c5e3640 1 -- start start 2024-04-25T10:29:13.347 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:13.344+0000 7fb54c5e3640 1 --2- >> v2:172.21.15.203:3300/0 conn(0x7fb5440d5430 0x7fb5440b75c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:29:13.347 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:13.344+0000 7fb54c5e3640 1 -- --> v1:172.21.15.37:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fb5440b7f10 con 0x7fb5440b7b00 2024-04-25T10:29:13.347 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:13.344+0000 7fb54c5e3640 1 -- --> v1:172.21.15.203:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fb544161520 con 0x7fb544161110 2024-04-25T10:29:13.347 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:13.344+0000 7fb54c5e3640 1 -- --> v2:172.21.15.203:3300/0 -- mon_getmap magic: 0 -- 0x7fb5440d4420 con 0x7fb5440d5430 2024-04-25T10:29:13.347 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:13.344+0000 7fb549b7e640 1 --1- >> v1:172.21.15.37:6789/0 conn(0x7fb5440b7b00 0x7fb544160a00 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.37:6789/0 says I am v1:172.21.15.37:51418/0 (socket says 172.21.15.37:51418) 2024-04-25T10:29:13.347 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:13.344+0000 7fb549b7e640 1 -- 172.21.15.37:0/1586880745 learned_addr learned my addr 172.21.15.37:0/1586880745 (peer_addr_for_me v1:172.21.15.37:0/0) 2024-04-25T10:29:13.347 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:13.344+0000 7fb54a37f640 1 --2- 172.21.15.37:0/1586880745 >> v2:172.21.15.203:3300/0 conn(0x7fb5440d5430 0x7fb5440b75c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:29:13.347 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:13.344+0000 7fb54937d640 1 -- 172.21.15.37:0/1586880745 <== mon.0 v1:172.21.15.37:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 2761127960 0 0) 0x7fb538001110 con 0x7fb5440b7b00 2024-04-25T10:29:13.348 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:13.344+0000 7fb54937d640 1 -- 172.21.15.37:0/1586880745 --> v1:172.21.15.37:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7fb52c003520 con 0x7fb5440b7b00 2024-04-25T10:29:13.348 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:13.345+0000 7fb54937d640 1 -- 172.21.15.37:0/1586880745 <== mon.1 v1:172.21.15.203:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 2744478708 0 0) 0x7fb544161520 con 0x7fb544161110 2024-04-25T10:29:13.348 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:13.345+0000 7fb54937d640 1 -- 172.21.15.37:0/1586880745 --> v1:172.21.15.203:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7fb538001110 con 0x7fb544161110 2024-04-25T10:29:13.348 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:13.345+0000 7fb54937d640 1 -- 172.21.15.37:0/1586880745 <== mon.0 v1:172.21.15.37:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 987014523 0 0) 0x7fb52c003520 con 0x7fb5440b7b00 2024-04-25T10:29:13.348 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:13.345+0000 7fb54a37f640 1 -- 172.21.15.37:0/1586880745 >> v1:172.21.15.203:6789/0 conn(0x7fb544161110 legacy=0x7fb544161880 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:29:13.348 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:13.345+0000 7fb54a37f640 1 -- 172.21.15.37:0/1586880745 >> v1:172.21.15.37:6789/0 conn(0x7fb5440b7b00 legacy=0x7fb544160a00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:29:13.348 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:13.345+0000 7fb54a37f640 1 -- 172.21.15.37:0/1586880745 --> v2:172.21.15.203:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fb5440d5840 con 0x7fb5440d5430 2024-04-25T10:29:13.349 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:13.345+0000 7fb54a37f640 1 --2- 172.21.15.37:0/1586880745 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fb5440d5430 0x7fb5440b75c0 secure :-1 s=READY pgs=57 cs=0 l=1 rev1=1 crypto rx=0x7fb5441616f0 tx=0x7fb53c02a110 comp rx=0 tx=0).ready entity=mon.1 client_cookie=bc7579b6174060c2 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:29:13.349 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:13.346+0000 7fb54937d640 1 -- 172.21.15.37:0/1586880745 <== mon.1 v2:172.21.15.203:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb53c037070 con 0x7fb5440d5430 2024-04-25T10:29:13.349 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:13.346+0000 7fb54937d640 1 -- 172.21.15.37:0/1586880745 <== mon.1 v2:172.21.15.203:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fb53c02aba0 con 0x7fb5440d5430 2024-04-25T10:29:13.349 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:13.346+0000 7fb54937d640 1 -- 172.21.15.37:0/1586880745 <== mon.1 v2:172.21.15.203:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb538001840 con 0x7fb5440d5430 2024-04-25T10:29:13.349 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:13.346+0000 7fb54c5e3640 1 -- 172.21.15.37:0/1586880745 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fb5440d5430 msgr2=0x7fb5440b75c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:29:13.350 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:13.346+0000 7fb54c5e3640 1 --2- 172.21.15.37:0/1586880745 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fb5440d5430 0x7fb5440b75c0 secure :-1 s=READY pgs=57 cs=0 l=1 rev1=1 crypto rx=0x7fb5441616f0 tx=0x7fb53c02a110 comp rx=0 tx=0).stop 2024-04-25T10:29:13.350 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:13.346+0000 7fb54c5e3640 1 -- 172.21.15.37:0/1586880745 shutdown_connections 2024-04-25T10:29:13.350 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:13.346+0000 7fb54c5e3640 1 --2- 172.21.15.37:0/1586880745 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fb5440d5430 0x7fb5440b75c0 unknown :-1 s=CLOSED pgs=57 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:29:13.350 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:13.346+0000 7fb54c5e3640 1 -- 172.21.15.37:0/1586880745 >> 172.21.15.37:0/1586880745 conn(0x7fb5440d27a0 msgr2=0x7fb5440d2bb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:29:13.350 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:13.346+0000 7fb54c5e3640 1 -- 172.21.15.37:0/1586880745 shutdown_connections 2024-04-25T10:29:13.350 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:13.347+0000 7fb54c5e3640 1 -- 172.21.15.37:0/1586880745 wait complete. 2024-04-25T10:29:13.350 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:13.348+0000 7fb54c5e3640 1 Processor -- start 2024-04-25T10:29:13.350 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:13.348+0000 7fb54c5e3640 1 -- start start 2024-04-25T10:29:13.353 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:13.349+0000 7fb54c5e3640 1 --2- >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7fb5440b7b00 0x7fb5440cf3f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:29:13.353 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:13.349+0000 7fb54c5e3640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fb5440d5430 0x7fb5441e8fa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:29:13.353 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:13.349+0000 7fb54c5e3640 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fb544161110 0x7fb54415c990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:29:13.353 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:13.349+0000 7fb54c5e3640 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 -- 0x7fb5440d4c30 con 0x7fb5440d5430 2024-04-25T10:29:13.353 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:13.350+0000 7fb54c5e3640 1 -- --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_getmap magic: 0 -- 0x7fb5440d4950 con 0x7fb544161110 2024-04-25T10:29:13.353 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:13.350+0000 7fb54c5e3640 1 -- --> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] -- mon_getmap magic: 0 -- 0x7fb5440d4f10 con 0x7fb5440b7b00 2024-04-25T10:29:13.353 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:13.350+0000 7fb549b7e640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fb5440d5430 0x7fb5441e8fa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:29:13.353 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:13.350+0000 7fb549b7e640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fb5440d5430 0x7fb5441e8fa0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.37:3300/0 says I am v2:172.21.15.37:46310/0 (socket says 172.21.15.37:46310) 2024-04-25T10:29:13.353 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:13.350+0000 7fb549b7e640 1 -- 172.21.15.37:0/1692110121 learned_addr learned my addr 172.21.15.37:0/1692110121 (peer_addr_for_me v2:172.21.15.37:0/0) 2024-04-25T10:29:13.354 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:13.350+0000 7fb54a37f640 1 --2- 172.21.15.37:0/1692110121 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7fb5440b7b00 0x7fb5440cf3f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:29:13.354 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:13.350+0000 7fb54ab80640 1 --2- 172.21.15.37:0/1692110121 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fb544161110 0x7fb54415c990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:29:13.354 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:13.350+0000 7fb549b7e640 1 -- 172.21.15.37:0/1692110121 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7fb5440b7b00 msgr2=0x7fb5440cf3f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:29:13.354 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:13.350+0000 7fb549b7e640 1 --2- 172.21.15.37:0/1692110121 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7fb5440b7b00 0x7fb5440cf3f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:29:13.354 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:13.350+0000 7fb549b7e640 1 -- 172.21.15.37:0/1692110121 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fb544161110 msgr2=0x7fb54415c990 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:29:13.354 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:13.350+0000 7fb549b7e640 1 --2- 172.21.15.37:0/1692110121 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fb544161110 0x7fb54415c990 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:29:13.354 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:13.350+0000 7fb549b7e640 1 -- 172.21.15.37:0/1692110121 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fb5441ea9a0 con 0x7fb5440d5430 2024-04-25T10:29:13.354 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:13.350+0000 7fb54ab80640 1 --2- 172.21.15.37:0/1692110121 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fb544161110 0x7fb54415c990 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T10:29:13.354 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:13.350+0000 7fb549b7e640 1 --2- 172.21.15.37:0/1692110121 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fb5440d5430 0x7fb5441e8fa0 secure :-1 s=READY pgs=82 cs=0 l=1 rev1=1 crypto rx=0x7fb5380019b0 tx=0x7fb53800cb90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:29:13.354 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:13.350+0000 7fb542ffd640 1 -- 172.21.15.37:0/1692110121 <== mon.0 v2:172.21.15.37:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb538017070 con 0x7fb5440d5430 2024-04-25T10:29:13.354 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:13.350+0000 7fb542ffd640 1 -- 172.21.15.37:0/1692110121 <== mon.0 v2:172.21.15.37:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fb538005af0 con 0x7fb5440d5430 2024-04-25T10:29:13.354 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:13.350+0000 7fb54c5e3640 1 -- 172.21.15.37:0/1692110121 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fb5440cdba0 con 0x7fb5440d5430 2024-04-25T10:29:13.354 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:13.350+0000 7fb542ffd640 1 -- 172.21.15.37:0/1692110121 <== mon.0 v2:172.21.15.37:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb538007070 con 0x7fb5440d5430 2024-04-25T10:29:13.354 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:13.350+0000 7fb54c5e3640 1 -- 172.21.15.37:0/1692110121 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fb5441f76f0 con 0x7fb5440d5430 2024-04-25T10:29:13.355 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:13.351+0000 7fb54c5e3640 1 -- 172.21.15.37:0/1692110121 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fb5440ce260 con 0x7fb5440d5430 2024-04-25T10:29:13.355 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:13.352+0000 7fb542ffd640 1 -- 172.21.15.37:0/1692110121 <== mon.0 v2:172.21.15.37:3300/0 4 ==== mgrmap(e 5) ==== 96782+0+0 (secure 0 0 0) 0x7fb53801a070 con 0x7fb5440d5430 2024-04-25T10:29:13.355 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:13.353+0000 7fb542ffd640 1 --2- 172.21.15.37:0/1692110121 >> [v2:172.21.15.203:6832/2418205237,v1:172.21.15.203:6833/2418205237] conn(0x7fb5300744b0 0x7fb5300769c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:29:13.355 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:13.353+0000 7fb542ffd640 1 -- 172.21.15.37:0/1692110121 <== mon.0 v2:172.21.15.37:3300/0 5 ==== osd_map(23..23 src has 1..23) ==== 5709+0+0 (secure 0 0 0) 0x7fb538094830 con 0x7fb5440d5430 2024-04-25T10:29:13.355 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:13.353+0000 7fb54a37f640 1 --2- 172.21.15.37:0/1692110121 >> [v2:172.21.15.203:6832/2418205237,v1:172.21.15.203:6833/2418205237] conn(0x7fb5300744b0 0x7fb5300769c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:29:13.356 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:13.353+0000 7fb54a37f640 1 --2- 172.21.15.37:0/1692110121 >> [v2:172.21.15.203:6832/2418205237,v1:172.21.15.203:6833/2418205237] conn(0x7fb5300744b0 0x7fb5300769c0 secure :-1 s=READY pgs=65 cs=0 l=1 rev1=1 crypto rx=0x7fb53c034000 tx=0x7fb53c02e000 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:29:13.359 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:13.357+0000 7fb542ffd640 1 -- 172.21.15.37:0/1692110121 <== mon.0 v2:172.21.15.37:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187246 (secure 0 0 0) 0x7fb5440ce260 con 0x7fb5440d5430 2024-04-25T10:29:13.538 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:13.536+0000 7fb54c5e3640 1 -- 172.21.15.37:0/1692110121 --> [v2:172.21.15.203:6832/2418205237,v1:172.21.15.203:6833/2418205237] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) -- 0x7fb54406a140 con 0x7fb5300744b0 2024-04-25T10:29:13.545 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:13.543+0000 7fb542ffd640 1 -- 172.21.15.37:0/1692110121 <== mgr.4100 v2:172.21.15.203:6832/2418205237 1 ==== mgr_command_reply(tid 0: 0 dumped all) ==== 18+0+251335 (secure 0 0 0) 0x7fb54406a140 con 0x7fb5300744b0 2024-04-25T10:29:13.548 INFO:teuthology.orchestra.run.smithi037.stdout: 2024-04-25T10:29:13.549 INFO:teuthology.orchestra.run.smithi037.stderr:dumped all 2024-04-25T10:29:13.551 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:13.545+0000 7fb54c5e3640 1 -- 172.21.15.37:0/1692110121 >> [v2:172.21.15.203:6832/2418205237,v1:172.21.15.203:6833/2418205237] conn(0x7fb5300744b0 msgr2=0x7fb5300769c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:29:13.551 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:13.545+0000 7fb54c5e3640 1 --2- 172.21.15.37:0/1692110121 >> [v2:172.21.15.203:6832/2418205237,v1:172.21.15.203:6833/2418205237] conn(0x7fb5300744b0 0x7fb5300769c0 secure :-1 s=READY pgs=65 cs=0 l=1 rev1=1 crypto rx=0x7fb53c034000 tx=0x7fb53c02e000 comp rx=0 tx=0).stop 2024-04-25T10:29:13.551 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:13.545+0000 7fb54c5e3640 1 -- 172.21.15.37:0/1692110121 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fb5440d5430 msgr2=0x7fb5441e8fa0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:29:13.551 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:13.545+0000 7fb54c5e3640 1 --2- 172.21.15.37:0/1692110121 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fb5440d5430 0x7fb5441e8fa0 secure :-1 s=READY pgs=82 cs=0 l=1 rev1=1 crypto rx=0x7fb5380019b0 tx=0x7fb53800cb90 comp rx=0 tx=0).stop 2024-04-25T10:29:13.551 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:13.545+0000 7fb54c5e3640 1 -- 172.21.15.37:0/1692110121 shutdown_connections 2024-04-25T10:29:13.551 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:13.545+0000 7fb54c5e3640 1 --2- 172.21.15.37:0/1692110121 >> [v2:172.21.15.203:6832/2418205237,v1:172.21.15.203:6833/2418205237] conn(0x7fb5300744b0 0x7fb5300769c0 unknown :-1 s=CLOSED pgs=65 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:29:13.551 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:13.545+0000 7fb54c5e3640 1 --2- 172.21.15.37:0/1692110121 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fb544161110 0x7fb54415c990 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:29:13.551 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:13.545+0000 7fb54c5e3640 1 --2- 172.21.15.37:0/1692110121 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fb5440d5430 0x7fb5441e8fa0 unknown :-1 s=CLOSED pgs=82 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:29:13.551 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:13.545+0000 7fb54c5e3640 1 --2- 172.21.15.37:0/1692110121 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7fb5440b7b00 0x7fb5440cf3f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:29:13.551 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:13.545+0000 7fb54c5e3640 1 -- 172.21.15.37:0/1692110121 >> 172.21.15.37:0/1692110121 conn(0x7fb5440d27a0 msgr2=0x7fb544161830 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:29:13.551 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:13.546+0000 7fb54c5e3640 1 -- 172.21.15.37:0/1692110121 shutdown_connections 2024-04-25T10:29:13.551 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:13.546+0000 7fb54c5e3640 1 -- 172.21.15.37:0/1692110121 wait complete. 2024-04-25T10:29:13.559 INFO:teuthology.orchestra.run.smithi037.stdout:{"pg_ready":true,"pg_map":{"version":26,"stamp":"2024-04-25T10:29:12.186439+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":592196,"num_objects":16,"num_object_clones":0,"num_object_copies":32,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":16,"num_whiteouts":0,"num_read":66,"num_read_kb":55,"num_write":126,"num_write_kb":1641,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":7,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":96,"ondisk_log_size":96,"up":194,"acting":194,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":52,"num_osds":8,"num_per_pool_osds":8,"num_per_pool_omap_osds":8,"kb":754974720,"kb_used":219068,"kb_used_data":2908,"kb_used_omap":12,"kb_used_meta":216051,"kb_avail":754755652,"statfs":{"total":773094113280,"available":772869787648,"internally_reserved":0,"allocated":2977792,"data_stored":1769702,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":12752,"internal_metadata":221236784},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":30,"apply_latency_ms":30,"commit_latency_ns":30000000,"apply_latency_ns":30000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":1828,"num_objects":14,"num_object_clones":0,"num_object_copies":28,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":14,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":25,"num_write_kb":13,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":7,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"5.190009"},"pg_stats":[{"pgid":"3.3f","version":"0'0","reported_seq":12,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-04-25T10:29:10.037967+0000","last_change":"2024-04-25T10:29:09.024146+0000","last_active":"2024-04-25T10:29:10.037967+0000","last_peered":"2024-04-25T10:29:10.037967+0000","last_clean":"2024-04-25T10:29:10.037967+0000","last_became_active":"2024-04-25T10:29:09.023798+0000","last_became_peered":"2024-04-25T10:29:09.023798+0000","last_unstale":"2024-04-25T10:29:10.037967+0000","last_undegraded":"2024-04-25T10:29:10.037967+0000","last_fullsized":"2024-04-25T10:29:10.037967+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:29:07.995096+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:29:07.995096+0000","last_clean_scrub_stamp":"2024-04-25T10:29:07.995096+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T20:37:20.726766+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.3e","version":"0'0","reported_seq":12,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-04-25T10:29:10.189579+0000","last_change":"2024-04-25T10:29:09.028600+0000","last_active":"2024-04-25T10:29:10.189579+0000","last_peered":"2024-04-25T10:29:10.189579+0000","last_clean":"2024-04-25T10:29:10.189579+0000","last_became_active":"2024-04-25T10:29:09.028286+0000","last_became_peered":"2024-04-25T10:29:09.028286+0000","last_unstale":"2024-04-25T10:29:10.189579+0000","last_undegraded":"2024-04-25T10:29:10.189579+0000","last_fullsized":"2024-04-25T10:29:10.189579+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:29:07.995096+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:29:07.995096+0000","last_clean_scrub_stamp":"2024-04-25T10:29:07.995096+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T20:52:02.098872+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6],"acting":[5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.3d","version":"0'0","reported_seq":12,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-04-25T10:29:10.189318+0000","last_change":"2024-04-25T10:29:09.039030+0000","last_active":"2024-04-25T10:29:10.189318+0000","last_peered":"2024-04-25T10:29:10.189318+0000","last_clean":"2024-04-25T10:29:10.189318+0000","last_became_active":"2024-04-25T10:29:09.037692+0000","last_became_peered":"2024-04-25T10:29:09.037692+0000","last_unstale":"2024-04-25T10:29:10.189318+0000","last_undegraded":"2024-04-25T10:29:10.189318+0000","last_fullsized":"2024-04-25T10:29:10.189318+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:29:07.995096+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:29:07.995096+0000","last_clean_scrub_stamp":"2024-04-25T10:29:07.995096+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T15:51:52.989545+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.3c","version":"0'0","reported_seq":12,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-04-25T10:29:10.026169+0000","last_change":"2024-04-25T10:29:09.027520+0000","last_active":"2024-04-25T10:29:10.026169+0000","last_peered":"2024-04-25T10:29:10.026169+0000","last_clean":"2024-04-25T10:29:10.026169+0000","last_became_active":"2024-04-25T10:29:09.027075+0000","last_became_peered":"2024-04-25T10:29:09.027075+0000","last_unstale":"2024-04-25T10:29:10.026169+0000","last_undegraded":"2024-04-25T10:29:10.026169+0000","last_fullsized":"2024-04-25T10:29:10.026169+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:29:07.995096+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:29:07.995096+0000","last_clean_scrub_stamp":"2024-04-25T10:29:07.995096+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T10:55:01.505157+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4],"acting":[2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.3b","version":"0'0","reported_seq":12,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-04-25T10:29:10.037590+0000","last_change":"2024-04-25T10:29:09.040684+0000","last_active":"2024-04-25T10:29:10.037590+0000","last_peered":"2024-04-25T10:29:10.037590+0000","last_clean":"2024-04-25T10:29:10.037590+0000","last_became_active":"2024-04-25T10:29:09.040372+0000","last_became_peered":"2024-04-25T10:29:09.040372+0000","last_unstale":"2024-04-25T10:29:10.037590+0000","last_undegraded":"2024-04-25T10:29:10.037590+0000","last_fullsized":"2024-04-25T10:29:10.037590+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:29:07.995096+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:29:07.995096+0000","last_clean_scrub_stamp":"2024-04-25T10:29:07.995096+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T11:52:24.415332+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.3a","version":"0'0","reported_seq":12,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-04-25T10:29:10.025911+0000","last_change":"2024-04-25T10:29:09.027391+0000","last_active":"2024-04-25T10:29:10.025911+0000","last_peered":"2024-04-25T10:29:10.025911+0000","last_clean":"2024-04-25T10:29:10.025911+0000","last_became_active":"2024-04-25T10:29:09.027029+0000","last_became_peered":"2024-04-25T10:29:09.027029+0000","last_unstale":"2024-04-25T10:29:10.025911+0000","last_undegraded":"2024-04-25T10:29:10.025911+0000","last_fullsized":"2024-04-25T10:29:10.025911+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:29:07.995096+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:29:07.995096+0000","last_clean_scrub_stamp":"2024-04-25T10:29:07.995096+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T10:35:31.361514+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4],"acting":[2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.39","version":"0'0","reported_seq":12,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-04-25T10:29:10.037462+0000","last_change":"2024-04-25T10:29:09.029272+0000","last_active":"2024-04-25T10:29:10.037462+0000","last_peered":"2024-04-25T10:29:10.037462+0000","last_clean":"2024-04-25T10:29:10.037462+0000","last_became_active":"2024-04-25T10:29:09.028302+0000","last_became_peered":"2024-04-25T10:29:09.028302+0000","last_unstale":"2024-04-25T10:29:10.037462+0000","last_undegraded":"2024-04-25T10:29:10.037462+0000","last_fullsized":"2024-04-25T10:29:10.037462+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:29:07.995096+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:29:07.995096+0000","last_clean_scrub_stamp":"2024-04-25T10:29:07.995096+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T11:31:45.283247+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.38","version":"0'0","reported_seq":12,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-04-25T10:29:10.188973+0000","last_change":"2024-04-25T10:29:09.035089+0000","last_active":"2024-04-25T10:29:10.188973+0000","last_peered":"2024-04-25T10:29:10.188973+0000","last_clean":"2024-04-25T10:29:10.188973+0000","last_became_active":"2024-04-25T10:29:09.034324+0000","last_became_peered":"2024-04-25T10:29:09.034324+0000","last_unstale":"2024-04-25T10:29:10.188973+0000","last_undegraded":"2024-04-25T10:29:10.188973+0000","last_fullsized":"2024-04-25T10:29:10.188973+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:29:07.995096+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:29:07.995096+0000","last_clean_scrub_stamp":"2024-04-25T10:29:07.995096+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T21:39:39.743441+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.37","version":"0'0","reported_seq":12,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-04-25T10:29:10.187981+0000","last_change":"2024-04-25T10:29:09.016966+0000","last_active":"2024-04-25T10:29:10.187981+0000","last_peered":"2024-04-25T10:29:10.187981+0000","last_clean":"2024-04-25T10:29:10.187981+0000","last_became_active":"2024-04-25T10:29:09.016566+0000","last_became_peered":"2024-04-25T10:29:09.016566+0000","last_unstale":"2024-04-25T10:29:10.187981+0000","last_undegraded":"2024-04-25T10:29:10.187981+0000","last_fullsized":"2024-04-25T10:29:10.187981+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:29:07.995096+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:29:07.995096+0000","last_clean_scrub_stamp":"2024-04-25T10:29:07.995096+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T10:55:01.505157+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.36","version":"0'0","reported_seq":12,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-04-25T10:29:10.037810+0000","last_change":"2024-04-25T10:29:09.030321+0000","last_active":"2024-04-25T10:29:10.037810+0000","last_peered":"2024-04-25T10:29:10.037810+0000","last_clean":"2024-04-25T10:29:10.037810+0000","last_became_active":"2024-04-25T10:29:09.029976+0000","last_became_peered":"2024-04-25T10:29:09.029976+0000","last_unstale":"2024-04-25T10:29:10.037810+0000","last_undegraded":"2024-04-25T10:29:10.037810+0000","last_fullsized":"2024-04-25T10:29:10.037810+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:29:07.995096+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:29:07.995096+0000","last_clean_scrub_stamp":"2024-04-25T10:29:07.995096+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T12:36:31.191100+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.35","version":"0'0","reported_seq":12,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-04-25T10:29:10.037641+0000","last_change":"2024-04-25T10:29:09.030469+0000","last_active":"2024-04-25T10:29:10.037641+0000","last_peered":"2024-04-25T10:29:10.037641+0000","last_clean":"2024-04-25T10:29:10.037641+0000","last_became_active":"2024-04-25T10:29:09.029350+0000","last_became_peered":"2024-04-25T10:29:09.029350+0000","last_unstale":"2024-04-25T10:29:10.037641+0000","last_undegraded":"2024-04-25T10:29:10.037641+0000","last_fullsized":"2024-04-25T10:29:10.037641+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:29:07.995096+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:29:07.995096+0000","last_clean_scrub_stamp":"2024-04-25T10:29:07.995096+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T15:51:52.989545+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.34","version":"0'0","reported_seq":12,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-04-25T10:29:10.037728+0000","last_change":"2024-04-25T10:29:09.029772+0000","last_active":"2024-04-25T10:29:10.037728+0000","last_peered":"2024-04-25T10:29:10.037728+0000","last_clean":"2024-04-25T10:29:10.037728+0000","last_became_active":"2024-04-25T10:29:09.029350+0000","last_became_peered":"2024-04-25T10:29:09.029350+0000","last_unstale":"2024-04-25T10:29:10.037728+0000","last_undegraded":"2024-04-25T10:29:10.037728+0000","last_fullsized":"2024-04-25T10:29:10.037728+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:29:07.995096+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:29:07.995096+0000","last_clean_scrub_stamp":"2024-04-25T10:29:07.995096+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T19:14:48.737269+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.33","version":"0'0","reported_seq":12,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-04-25T10:29:10.188971+0000","last_change":"2024-04-25T10:29:09.038690+0000","last_active":"2024-04-25T10:29:10.188971+0000","last_peered":"2024-04-25T10:29:10.188971+0000","last_clean":"2024-04-25T10:29:10.188971+0000","last_became_active":"2024-04-25T10:29:09.038336+0000","last_became_peered":"2024-04-25T10:29:09.038336+0000","last_unstale":"2024-04-25T10:29:10.188971+0000","last_undegraded":"2024-04-25T10:29:10.188971+0000","last_fullsized":"2024-04-25T10:29:10.188971+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:29:07.995096+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:29:07.995096+0000","last_clean_scrub_stamp":"2024-04-25T10:29:07.995096+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T19:23:12.589414+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.32","version":"0'0","reported_seq":12,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-04-25T10:29:10.188641+0000","last_change":"2024-04-25T10:29:09.028195+0000","last_active":"2024-04-25T10:29:10.188641+0000","last_peered":"2024-04-25T10:29:10.188641+0000","last_clean":"2024-04-25T10:29:10.188641+0000","last_became_active":"2024-04-25T10:29:09.027893+0000","last_became_peered":"2024-04-25T10:29:09.027893+0000","last_unstale":"2024-04-25T10:29:10.188641+0000","last_undegraded":"2024-04-25T10:29:10.188641+0000","last_fullsized":"2024-04-25T10:29:10.188641+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:29:07.995096+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:29:07.995096+0000","last_clean_scrub_stamp":"2024-04-25T10:29:07.995096+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T11:35:27.466874+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.31","version":"0'0","reported_seq":12,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-04-25T10:29:10.188956+0000","last_change":"2024-04-25T10:29:09.034532+0000","last_active":"2024-04-25T10:29:10.188956+0000","last_peered":"2024-04-25T10:29:10.188956+0000","last_clean":"2024-04-25T10:29:10.188956+0000","last_became_active":"2024-04-25T10:29:09.034021+0000","last_became_peered":"2024-04-25T10:29:09.034021+0000","last_unstale":"2024-04-25T10:29:10.188956+0000","last_undegraded":"2024-04-25T10:29:10.188956+0000","last_fullsized":"2024-04-25T10:29:10.188956+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:29:07.995096+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:29:07.995096+0000","last_clean_scrub_stamp":"2024-04-25T10:29:07.995096+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T12:16:04.880732+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.30","version":"0'0","reported_seq":12,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-04-25T10:29:10.024134+0000","last_change":"2024-04-25T10:29:09.021224+0000","last_active":"2024-04-25T10:29:10.024134+0000","last_peered":"2024-04-25T10:29:10.024134+0000","last_clean":"2024-04-25T10:29:10.024134+0000","last_became_active":"2024-04-25T10:29:09.020964+0000","last_became_peered":"2024-04-25T10:29:09.020964+0000","last_unstale":"2024-04-25T10:29:10.024134+0000","last_undegraded":"2024-04-25T10:29:10.024134+0000","last_fullsized":"2024-04-25T10:29:10.024134+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:29:07.995096+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:29:07.995096+0000","last_clean_scrub_stamp":"2024-04-25T10:29:07.995096+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T18:44:23.610374+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.2f","version":"0'0","reported_seq":12,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-04-25T10:29:10.024944+0000","last_change":"2024-04-25T10:29:09.025638+0000","last_active":"2024-04-25T10:29:10.024944+0000","last_peered":"2024-04-25T10:29:10.024944+0000","last_clean":"2024-04-25T10:29:10.024944+0000","last_became_active":"2024-04-25T10:29:09.024517+0000","last_became_peered":"2024-04-25T10:29:09.024517+0000","last_unstale":"2024-04-25T10:29:10.024944+0000","last_undegraded":"2024-04-25T10:29:10.024944+0000","last_fullsized":"2024-04-25T10:29:10.024944+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:29:07.995096+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:29:07.995096+0000","last_clean_scrub_stamp":"2024-04-25T10:29:07.995096+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T16:29:08.975346+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.2e","version":"0'0","reported_seq":12,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-04-25T10:29:10.189006+0000","last_change":"2024-04-25T10:29:09.023923+0000","last_active":"2024-04-25T10:29:10.189006+0000","last_peered":"2024-04-25T10:29:10.189006+0000","last_clean":"2024-04-25T10:29:10.189006+0000","last_became_active":"2024-04-25T10:29:09.023579+0000","last_became_peered":"2024-04-25T10:29:09.023579+0000","last_unstale":"2024-04-25T10:29:10.189006+0000","last_undegraded":"2024-04-25T10:29:10.189006+0000","last_fullsized":"2024-04-25T10:29:10.189006+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:29:07.995096+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:29:07.995096+0000","last_clean_scrub_stamp":"2024-04-25T10:29:07.995096+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T10:35:31.361514+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.2d","version":"0'0","reported_seq":12,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-04-25T10:29:10.024852+0000","last_change":"2024-04-25T10:29:09.023039+0000","last_active":"2024-04-25T10:29:10.024852+0000","last_peered":"2024-04-25T10:29:10.024852+0000","last_clean":"2024-04-25T10:29:10.024852+0000","last_became_active":"2024-04-25T10:29:09.022706+0000","last_became_peered":"2024-04-25T10:29:09.022706+0000","last_unstale":"2024-04-25T10:29:10.024852+0000","last_undegraded":"2024-04-25T10:29:10.024852+0000","last_fullsized":"2024-04-25T10:29:10.024852+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:29:07.995096+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:29:07.995096+0000","last_clean_scrub_stamp":"2024-04-25T10:29:07.995096+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T16:48:17.352944+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.2c","version":"0'0","reported_seq":12,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-04-25T10:29:10.037891+0000","last_change":"2024-04-25T10:29:09.028979+0000","last_active":"2024-04-25T10:29:10.037891+0000","last_peered":"2024-04-25T10:29:10.037891+0000","last_clean":"2024-04-25T10:29:10.037891+0000","last_became_active":"2024-04-25T10:29:09.028061+0000","last_became_peered":"2024-04-25T10:29:09.028061+0000","last_unstale":"2024-04-25T10:29:10.037891+0000","last_undegraded":"2024-04-25T10:29:10.037891+0000","last_fullsized":"2024-04-25T10:29:10.037891+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:29:07.995096+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:29:07.995096+0000","last_clean_scrub_stamp":"2024-04-25T10:29:07.995096+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T18:44:23.610374+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.2b","version":"0'0","reported_seq":1,"reported_epoch":21,"state":"creating+peering","last_fresh":"2024-04-25T10:29:08.012124+0000","last_change":"2024-04-25T10:29:08.012124+0000","last_active":"2024-04-25T10:29:07.995096+0000","last_peered":"2024-04-25T10:29:07.995096+0000","last_clean":"2024-04-25T10:29:07.995096+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T10:29:08.012124+0000","last_undegraded":"2024-04-25T10:29:08.012124+0000","last_fullsized":"2024-04-25T10:29:08.012124+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:29:07.995096+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:29:07.995096+0000","last_clean_scrub_stamp":"2024-04-25T10:29:07.995096+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.2a","version":"0'0","reported_seq":12,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-04-25T10:29:10.187695+0000","last_change":"2024-04-25T10:29:09.029965+0000","last_active":"2024-04-25T10:29:10.187695+0000","last_peered":"2024-04-25T10:29:10.187695+0000","last_clean":"2024-04-25T10:29:10.187695+0000","last_became_active":"2024-04-25T10:29:09.029616+0000","last_became_peered":"2024-04-25T10:29:09.029616+0000","last_unstale":"2024-04-25T10:29:10.187695+0000","last_undegraded":"2024-04-25T10:29:10.187695+0000","last_fullsized":"2024-04-25T10:29:10.187695+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:29:07.995096+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:29:07.995096+0000","last_clean_scrub_stamp":"2024-04-25T10:29:07.995096+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T15:58:36.848831+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.29","version":"0'0","reported_seq":12,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-04-25T10:29:10.189345+0000","last_change":"2024-04-25T10:29:09.033401+0000","last_active":"2024-04-25T10:29:10.189345+0000","last_peered":"2024-04-25T10:29:10.189345+0000","last_clean":"2024-04-25T10:29:10.189345+0000","last_became_active":"2024-04-25T10:29:09.032764+0000","last_became_peered":"2024-04-25T10:29:09.032764+0000","last_unstale":"2024-04-25T10:29:10.189345+0000","last_undegraded":"2024-04-25T10:29:10.189345+0000","last_fullsized":"2024-04-25T10:29:10.189345+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:29:07.995096+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:29:07.995096+0000","last_clean_scrub_stamp":"2024-04-25T10:29:07.995096+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T17:30:54.093151+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.28","version":"0'0","reported_seq":12,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-04-25T10:29:10.024133+0000","last_change":"2024-04-25T10:29:09.041063+0000","last_active":"2024-04-25T10:29:10.024133+0000","last_peered":"2024-04-25T10:29:10.024133+0000","last_clean":"2024-04-25T10:29:10.024133+0000","last_became_active":"2024-04-25T10:29:09.040734+0000","last_became_peered":"2024-04-25T10:29:09.040734+0000","last_unstale":"2024-04-25T10:29:10.024133+0000","last_undegraded":"2024-04-25T10:29:10.024133+0000","last_fullsized":"2024-04-25T10:29:10.024133+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:29:07.995096+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:29:07.995096+0000","last_clean_scrub_stamp":"2024-04-25T10:29:07.995096+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T13:54:22.968216+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.27","version":"0'0","reported_seq":12,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-04-25T10:29:10.189019+0000","last_change":"2024-04-25T10:29:09.034268+0000","last_active":"2024-04-25T10:29:10.189019+0000","last_peered":"2024-04-25T10:29:10.189019+0000","last_clean":"2024-04-25T10:29:10.189019+0000","last_became_active":"2024-04-25T10:29:09.033199+0000","last_became_peered":"2024-04-25T10:29:09.033199+0000","last_unstale":"2024-04-25T10:29:10.189019+0000","last_undegraded":"2024-04-25T10:29:10.189019+0000","last_fullsized":"2024-04-25T10:29:10.189019+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:29:07.995096+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:29:07.995096+0000","last_clean_scrub_stamp":"2024-04-25T10:29:07.995096+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T17:56:09.999494+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.26","version":"0'0","reported_seq":12,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-04-25T10:29:10.026183+0000","last_change":"2024-04-25T10:29:09.025643+0000","last_active":"2024-04-25T10:29:10.026183+0000","last_peered":"2024-04-25T10:29:10.026183+0000","last_clean":"2024-04-25T10:29:10.026183+0000","last_became_active":"2024-04-25T10:29:09.024894+0000","last_became_peered":"2024-04-25T10:29:09.024894+0000","last_unstale":"2024-04-25T10:29:10.026183+0000","last_undegraded":"2024-04-25T10:29:10.026183+0000","last_fullsized":"2024-04-25T10:29:10.026183+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:29:07.995096+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:29:07.995096+0000","last_clean_scrub_stamp":"2024-04-25T10:29:07.995096+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T22:08:00.367431+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.25","version":"0'0","reported_seq":12,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-04-25T10:29:10.189130+0000","last_change":"2024-04-25T10:29:09.023357+0000","last_active":"2024-04-25T10:29:10.189130+0000","last_peered":"2024-04-25T10:29:10.189130+0000","last_clean":"2024-04-25T10:29:10.189130+0000","last_became_active":"2024-04-25T10:29:09.022988+0000","last_became_peered":"2024-04-25T10:29:09.022988+0000","last_unstale":"2024-04-25T10:29:10.189130+0000","last_undegraded":"2024-04-25T10:29:10.189130+0000","last_fullsized":"2024-04-25T10:29:10.189130+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:29:07.995096+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:29:07.995096+0000","last_clean_scrub_stamp":"2024-04-25T10:29:07.995096+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T21:14:39.661957+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.24","version":"0'0","reported_seq":12,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-04-25T10:29:10.189239+0000","last_change":"2024-04-25T10:29:09.034328+0000","last_active":"2024-04-25T10:29:10.189239+0000","last_peered":"2024-04-25T10:29:10.189239+0000","last_clean":"2024-04-25T10:29:10.189239+0000","last_became_active":"2024-04-25T10:29:09.033978+0000","last_became_peered":"2024-04-25T10:29:09.033978+0000","last_unstale":"2024-04-25T10:29:10.189239+0000","last_undegraded":"2024-04-25T10:29:10.189239+0000","last_fullsized":"2024-04-25T10:29:10.189239+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:29:07.995096+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:29:07.995096+0000","last_clean_scrub_stamp":"2024-04-25T10:29:07.995096+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T21:24:14.497242+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.23","version":"0'0","reported_seq":12,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-04-25T10:29:10.189583+0000","last_change":"2024-04-25T10:29:09.034045+0000","last_active":"2024-04-25T10:29:10.189583+0000","last_peered":"2024-04-25T10:29:10.189583+0000","last_clean":"2024-04-25T10:29:10.189583+0000","last_became_active":"2024-04-25T10:29:09.033730+0000","last_became_peered":"2024-04-25T10:29:09.033730+0000","last_unstale":"2024-04-25T10:29:10.189583+0000","last_undegraded":"2024-04-25T10:29:10.189583+0000","last_fullsized":"2024-04-25T10:29:10.189583+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:29:07.995096+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:29:07.995096+0000","last_clean_scrub_stamp":"2024-04-25T10:29:07.995096+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T12:36:31.191100+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.22","version":"0'0","reported_seq":12,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-04-25T10:29:10.024663+0000","last_change":"2024-04-25T10:29:09.034262+0000","last_active":"2024-04-25T10:29:10.024663+0000","last_peered":"2024-04-25T10:29:10.024663+0000","last_clean":"2024-04-25T10:29:10.024663+0000","last_became_active":"2024-04-25T10:29:09.033905+0000","last_became_peered":"2024-04-25T10:29:09.033905+0000","last_unstale":"2024-04-25T10:29:10.024663+0000","last_undegraded":"2024-04-25T10:29:10.024663+0000","last_fullsized":"2024-04-25T10:29:10.024663+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:29:07.995096+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:29:07.995096+0000","last_clean_scrub_stamp":"2024-04-25T10:29:07.995096+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T16:22:45.608928+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.21","version":"0'0","reported_seq":12,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-04-25T10:29:10.188201+0000","last_change":"2024-04-25T10:29:09.027088+0000","last_active":"2024-04-25T10:29:10.188201+0000","last_peered":"2024-04-25T10:29:10.188201+0000","last_clean":"2024-04-25T10:29:10.188201+0000","last_became_active":"2024-04-25T10:29:09.026651+0000","last_became_peered":"2024-04-25T10:29:09.026651+0000","last_unstale":"2024-04-25T10:29:10.188201+0000","last_undegraded":"2024-04-25T10:29:10.188201+0000","last_fullsized":"2024-04-25T10:29:10.188201+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:29:07.995096+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:29:07.995096+0000","last_clean_scrub_stamp":"2024-04-25T10:29:07.995096+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T15:37:06.723380+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.20","version":"0'0","reported_seq":12,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-04-25T10:29:10.025741+0000","last_change":"2024-04-25T10:29:09.028008+0000","last_active":"2024-04-25T10:29:10.025741+0000","last_peered":"2024-04-25T10:29:10.025741+0000","last_clean":"2024-04-25T10:29:10.025741+0000","last_became_active":"2024-04-25T10:29:09.027679+0000","last_became_peered":"2024-04-25T10:29:09.027679+0000","last_unstale":"2024-04-25T10:29:10.025741+0000","last_undegraded":"2024-04-25T10:29:10.025741+0000","last_fullsized":"2024-04-25T10:29:10.025741+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:29:07.995096+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:29:07.995096+0000","last_clean_scrub_stamp":"2024-04-25T10:29:07.995096+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T15:37:06.723380+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,6],"acting":[2,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.1a","version":"0'0","reported_seq":12,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-04-25T10:29:10.187787+0000","last_change":"2024-04-25T10:29:09.032819+0000","last_active":"2024-04-25T10:29:10.187787+0000","last_peered":"2024-04-25T10:29:10.187787+0000","last_clean":"2024-04-25T10:29:10.187787+0000","last_became_active":"2024-04-25T10:29:09.032493+0000","last_became_peered":"2024-04-25T10:29:09.032493+0000","last_unstale":"2024-04-25T10:29:10.187787+0000","last_undegraded":"2024-04-25T10:29:10.187787+0000","last_fullsized":"2024-04-25T10:29:10.187787+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:29:07.995096+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:29:07.995096+0000","last_clean_scrub_stamp":"2024-04-25T10:29:07.995096+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T15:37:06.723380+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.1b","version":"22'1","reported_seq":15,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-04-25T10:29:10.189463+0000","last_change":"2024-04-25T10:29:08.025272+0000","last_active":"2024-04-25T10:29:10.189463+0000","last_peered":"2024-04-25T10:29:10.189463+0000","last_clean":"2024-04-25T10:29:10.189463+0000","last_became_active":"2024-04-25T10:29:08.024912+0000","last_became_peered":"2024-04-25T10:29:08.024912+0000","last_unstale":"2024-04-25T10:29:10.189463+0000","last_undegraded":"2024-04-25T10:29:10.189463+0000","last_fullsized":"2024-04-25T10:29:10.189463+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:29:06.989570+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:29:06.989570+0000","last_clean_scrub_stamp":"2024-04-25T10:29:06.989570+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T18:11:32.758179+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6],"acting":[5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.1b","version":"0'0","reported_seq":1,"reported_epoch":21,"state":"creating+peering","last_fresh":"2024-04-25T10:29:08.008604+0000","last_change":"2024-04-25T10:29:08.008604+0000","last_active":"2024-04-25T10:29:07.995096+0000","last_peered":"2024-04-25T10:29:07.995096+0000","last_clean":"2024-04-25T10:29:07.995096+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T10:29:08.008604+0000","last_undegraded":"2024-04-25T10:29:08.008604+0000","last_fullsized":"2024-04-25T10:29:08.008604+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:29:07.995096+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:29:07.995096+0000","last_clean_scrub_stamp":"2024-04-25T10:29:07.995096+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.1a","version":"22'1","reported_seq":15,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-04-25T10:29:10.024506+0000","last_change":"2024-04-25T10:29:08.026018+0000","last_active":"2024-04-25T10:29:10.024506+0000","last_peered":"2024-04-25T10:29:10.024506+0000","last_clean":"2024-04-25T10:29:10.024506+0000","last_became_active":"2024-04-25T10:29:08.025668+0000","last_became_peered":"2024-04-25T10:29:08.025668+0000","last_unstale":"2024-04-25T10:29:10.024506+0000","last_undegraded":"2024-04-25T10:29:10.024506+0000","last_fullsized":"2024-04-25T10:29:10.024506+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:29:06.989570+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:29:06.989570+0000","last_clean_scrub_stamp":"2024-04-25T10:29:06.989570+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T11:35:26.461348+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.19","version":"0'0","reported_seq":14,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-04-25T10:29:10.188333+0000","last_change":"2024-04-25T10:29:08.011702+0000","last_active":"2024-04-25T10:29:10.188333+0000","last_peered":"2024-04-25T10:29:10.188333+0000","last_clean":"2024-04-25T10:29:10.188333+0000","last_became_active":"2024-04-25T10:29:08.011353+0000","last_became_peered":"2024-04-25T10:29:08.011353+0000","last_unstale":"2024-04-25T10:29:10.188333+0000","last_undegraded":"2024-04-25T10:29:10.188333+0000","last_fullsized":"2024-04-25T10:29:10.188333+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:29:06.989570+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:29:06.989570+0000","last_clean_scrub_stamp":"2024-04-25T10:29:06.989570+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T16:31:11.685101+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.18","version":"0'0","reported_seq":12,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-04-25T10:29:10.189830+0000","last_change":"2024-04-25T10:29:09.033872+0000","last_active":"2024-04-25T10:29:10.189830+0000","last_peered":"2024-04-25T10:29:10.189830+0000","last_clean":"2024-04-25T10:29:10.189830+0000","last_became_active":"2024-04-25T10:29:09.033544+0000","last_became_peered":"2024-04-25T10:29:09.033544+0000","last_unstale":"2024-04-25T10:29:10.189830+0000","last_undegraded":"2024-04-25T10:29:10.189830+0000","last_fullsized":"2024-04-25T10:29:10.189830+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:29:07.995096+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:29:07.995096+0000","last_clean_scrub_stamp":"2024-04-25T10:29:07.995096+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T13:54:22.968216+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6],"acting":[5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.18","version":"0'0","reported_seq":14,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-04-25T10:29:10.024423+0000","last_change":"2024-04-25T10:29:08.013340+0000","last_active":"2024-04-25T10:29:10.024423+0000","last_peered":"2024-04-25T10:29:10.024423+0000","last_clean":"2024-04-25T10:29:10.024423+0000","last_became_active":"2024-04-25T10:29:08.012940+0000","last_became_peered":"2024-04-25T10:29:08.012940+0000","last_unstale":"2024-04-25T10:29:10.024423+0000","last_undegraded":"2024-04-25T10:29:10.024423+0000","last_fullsized":"2024-04-25T10:29:10.024423+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:29:06.989570+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:29:06.989570+0000","last_clean_scrub_stamp":"2024-04-25T10:29:06.989570+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T10:35:30.355988+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.19","version":"0'0","reported_seq":1,"reported_epoch":21,"state":"creating+peering","last_fresh":"2024-04-25T10:29:08.012731+0000","last_change":"2024-04-25T10:29:08.012731+0000","last_active":"2024-04-25T10:29:07.995096+0000","last_peered":"2024-04-25T10:29:07.995096+0000","last_clean":"2024-04-25T10:29:07.995096+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T10:29:08.012731+0000","last_undegraded":"2024-04-25T10:29:08.012731+0000","last_fullsized":"2024-04-25T10:29:08.012731+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:29:07.995096+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:29:07.995096+0000","last_clean_scrub_stamp":"2024-04-25T10:29:07.995096+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.17","version":"0'0","reported_seq":14,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-04-25T10:29:10.187949+0000","last_change":"2024-04-25T10:29:08.020281+0000","last_active":"2024-04-25T10:29:10.187949+0000","last_peered":"2024-04-25T10:29:10.187949+0000","last_clean":"2024-04-25T10:29:10.187949+0000","last_became_active":"2024-04-25T10:29:08.019921+0000","last_became_peered":"2024-04-25T10:29:08.019921+0000","last_unstale":"2024-04-25T10:29:10.187949+0000","last_undegraded":"2024-04-25T10:29:10.187949+0000","last_fullsized":"2024-04-25T10:29:10.187949+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:29:06.989570+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:29:06.989570+0000","last_clean_scrub_stamp":"2024-04-25T10:29:06.989570+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T19:13:07.068780+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.16","version":"0'0","reported_seq":12,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-04-25T10:29:10.026197+0000","last_change":"2024-04-25T10:29:09.022744+0000","last_active":"2024-04-25T10:29:10.026197+0000","last_peered":"2024-04-25T10:29:10.026197+0000","last_clean":"2024-04-25T10:29:10.026197+0000","last_became_active":"2024-04-25T10:29:09.022392+0000","last_became_peered":"2024-04-25T10:29:09.022392+0000","last_unstale":"2024-04-25T10:29:10.026197+0000","last_undegraded":"2024-04-25T10:29:10.026197+0000","last_fullsized":"2024-04-25T10:29:10.026197+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:29:07.995096+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:29:07.995096+0000","last_clean_scrub_stamp":"2024-04-25T10:29:07.995096+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T11:35:27.466874+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.16","version":"0'0","reported_seq":14,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-04-25T10:29:10.037869+0000","last_change":"2024-04-25T10:29:08.026388+0000","last_active":"2024-04-25T10:29:10.037869+0000","last_peered":"2024-04-25T10:29:10.037869+0000","last_clean":"2024-04-25T10:29:10.037869+0000","last_became_active":"2024-04-25T10:29:08.026039+0000","last_became_peered":"2024-04-25T10:29:08.026039+0000","last_unstale":"2024-04-25T10:29:10.037869+0000","last_undegraded":"2024-04-25T10:29:10.037869+0000","last_fullsized":"2024-04-25T10:29:10.037869+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:29:06.989570+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:29:06.989570+0000","last_clean_scrub_stamp":"2024-04-25T10:29:06.989570+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T18:11:32.758179+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.17","version":"0'0","reported_seq":12,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-04-25T10:29:10.189465+0000","last_change":"2024-04-25T10:29:09.032902+0000","last_active":"2024-04-25T10:29:10.189465+0000","last_peered":"2024-04-25T10:29:10.189465+0000","last_clean":"2024-04-25T10:29:10.189465+0000","last_became_active":"2024-04-25T10:29:09.032596+0000","last_became_peered":"2024-04-25T10:29:09.032596+0000","last_unstale":"2024-04-25T10:29:10.189465+0000","last_undegraded":"2024-04-25T10:29:10.189465+0000","last_fullsized":"2024-04-25T10:29:10.189465+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:29:07.995096+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:29:07.995096+0000","last_clean_scrub_stamp":"2024-04-25T10:29:07.995096+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T19:14:48.737269+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.15","version":"22'3","reported_seq":17,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-04-25T10:29:10.188352+0000","last_change":"2024-04-25T10:29:08.021347+0000","last_active":"2024-04-25T10:29:10.188352+0000","last_peered":"2024-04-25T10:29:10.188352+0000","last_clean":"2024-04-25T10:29:10.188352+0000","last_became_active":"2024-04-25T10:29:08.021039+0000","last_became_peered":"2024-04-25T10:29:08.021039+0000","last_unstale":"2024-04-25T10:29:10.188352+0000","last_undegraded":"2024-04-25T10:29:10.188352+0000","last_fullsized":"2024-04-25T10:29:10.188352+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:29:06.989570+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:29:06.989570+0000","last_clean_scrub_stamp":"2024-04-25T10:29:06.989570+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T20:22:56.380539+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.14","version":"0'0","reported_seq":12,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-04-25T10:29:10.024492+0000","last_change":"2024-04-25T10:29:09.021410+0000","last_active":"2024-04-25T10:29:10.024492+0000","last_peered":"2024-04-25T10:29:10.024492+0000","last_clean":"2024-04-25T10:29:10.024492+0000","last_became_active":"2024-04-25T10:29:09.021051+0000","last_became_peered":"2024-04-25T10:29:09.021051+0000","last_unstale":"2024-04-25T10:29:10.024492+0000","last_undegraded":"2024-04-25T10:29:10.024492+0000","last_fullsized":"2024-04-25T10:29:10.024492+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:29:07.995096+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:29:07.995096+0000","last_clean_scrub_stamp":"2024-04-25T10:29:07.995096+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T13:22:13.080513+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.14","version":"22'6","reported_seq":20,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-04-25T10:29:10.025912+0000","last_change":"2024-04-25T10:29:08.018044+0000","last_active":"2024-04-25T10:29:10.025912+0000","last_peered":"2024-04-25T10:29:10.025912+0000","last_clean":"2024-04-25T10:29:10.025912+0000","last_became_active":"2024-04-25T10:29:08.016984+0000","last_became_peered":"2024-04-25T10:29:08.016984+0000","last_unstale":"2024-04-25T10:29:10.025912+0000","last_undegraded":"2024-04-25T10:29:10.025912+0000","last_fullsized":"2024-04-25T10:29:10.025912+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:29:06.989570+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:29:06.989570+0000","last_clean_scrub_stamp":"2024-04-25T10:29:06.989570+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T18:29:56.525236+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1612,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4],"acting":[2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.15","version":"0'0","reported_seq":12,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-04-25T10:29:10.037668+0000","last_change":"2024-04-25T10:29:09.029034+0000","last_active":"2024-04-25T10:29:10.037668+0000","last_peered":"2024-04-25T10:29:10.037668+0000","last_clean":"2024-04-25T10:29:10.037668+0000","last_became_active":"2024-04-25T10:29:09.028662+0000","last_became_peered":"2024-04-25T10:29:09.028662+0000","last_unstale":"2024-04-25T10:29:10.037668+0000","last_undegraded":"2024-04-25T10:29:10.037668+0000","last_fullsized":"2024-04-25T10:29:10.037668+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:29:07.995096+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:29:07.995096+0000","last_clean_scrub_stamp":"2024-04-25T10:29:07.995096+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T17:30:54.093151+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.13","version":"22'2","reported_seq":16,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-04-25T10:29:10.189430+0000","last_change":"2024-04-25T10:29:08.020476+0000","last_active":"2024-04-25T10:29:10.189430+0000","last_peered":"2024-04-25T10:29:10.189430+0000","last_clean":"2024-04-25T10:29:10.189430+0000","last_became_active":"2024-04-25T10:29:08.020096+0000","last_became_peered":"2024-04-25T10:29:08.020096+0000","last_unstale":"2024-04-25T10:29:10.189430+0000","last_undegraded":"2024-04-25T10:29:10.189430+0000","last_fullsized":"2024-04-25T10:29:10.189430+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:29:06.989570+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:29:06.989570+0000","last_clean_scrub_stamp":"2024-04-25T10:29:06.989570+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T10:32:06.103123+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":34,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":7,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.12","version":"0'0","reported_seq":12,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-04-25T10:29:10.188838+0000","last_change":"2024-04-25T10:29:09.027996+0000","last_active":"2024-04-25T10:29:10.188838+0000","last_peered":"2024-04-25T10:29:10.188838+0000","last_clean":"2024-04-25T10:29:10.188838+0000","last_became_active":"2024-04-25T10:29:09.027538+0000","last_became_peered":"2024-04-25T10:29:09.027538+0000","last_unstale":"2024-04-25T10:29:10.188838+0000","last_undegraded":"2024-04-25T10:29:10.188838+0000","last_fullsized":"2024-04-25T10:29:10.188838+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:29:07.995096+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:29:07.995096+0000","last_clean_scrub_stamp":"2024-04-25T10:29:07.995096+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T12:16:47.983013+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.12","version":"0'0","reported_seq":14,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-04-25T10:29:10.024603+0000","last_change":"2024-04-25T10:29:08.019028+0000","last_active":"2024-04-25T10:29:10.024603+0000","last_peered":"2024-04-25T10:29:10.024603+0000","last_clean":"2024-04-25T10:29:10.024603+0000","last_became_active":"2024-04-25T10:29:08.018582+0000","last_became_peered":"2024-04-25T10:29:08.018582+0000","last_unstale":"2024-04-25T10:29:10.024603+0000","last_undegraded":"2024-04-25T10:29:10.024603+0000","last_fullsized":"2024-04-25T10:29:10.024603+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:29:06.989570+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:29:06.989570+0000","last_clean_scrub_stamp":"2024-04-25T10:29:06.989570+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T19:23:11.583888+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.13","version":"0'0","reported_seq":12,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-04-25T10:29:10.024281+0000","last_change":"2024-04-25T10:29:09.023995+0000","last_active":"2024-04-25T10:29:10.024281+0000","last_peered":"2024-04-25T10:29:10.024281+0000","last_clean":"2024-04-25T10:29:10.024281+0000","last_became_active":"2024-04-25T10:29:09.023690+0000","last_became_peered":"2024-04-25T10:29:09.023690+0000","last_unstale":"2024-04-25T10:29:10.024281+0000","last_undegraded":"2024-04-25T10:29:10.024281+0000","last_fullsized":"2024-04-25T10:29:10.024281+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:29:07.995096+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:29:07.995096+0000","last_clean_scrub_stamp":"2024-04-25T10:29:07.995096+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T15:28:23.689564+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.11","version":"0'0","reported_seq":14,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-04-25T10:29:10.037576+0000","last_change":"2024-04-25T10:29:08.027174+0000","last_active":"2024-04-25T10:29:10.037576+0000","last_peered":"2024-04-25T10:29:10.037576+0000","last_clean":"2024-04-25T10:29:10.037576+0000","last_became_active":"2024-04-25T10:29:08.026809+0000","last_became_peered":"2024-04-25T10:29:08.026809+0000","last_unstale":"2024-04-25T10:29:10.037576+0000","last_undegraded":"2024-04-25T10:29:10.037576+0000","last_fullsized":"2024-04-25T10:29:10.037576+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:29:06.989570+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:29:06.989570+0000","last_clean_scrub_stamp":"2024-04-25T10:29:06.989570+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T10:35:30.355988+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.10","version":"0'0","reported_seq":12,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-04-25T10:29:10.187809+0000","last_change":"2024-04-25T10:29:09.041722+0000","last_active":"2024-04-25T10:29:10.187809+0000","last_peered":"2024-04-25T10:29:10.187809+0000","last_clean":"2024-04-25T10:29:10.187809+0000","last_became_active":"2024-04-25T10:29:09.040971+0000","last_became_peered":"2024-04-25T10:29:09.040971+0000","last_unstale":"2024-04-25T10:29:10.187809+0000","last_undegraded":"2024-04-25T10:29:10.187809+0000","last_fullsized":"2024-04-25T10:29:10.187809+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:29:07.995096+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:29:07.995096+0000","last_clean_scrub_stamp":"2024-04-25T10:29:07.995096+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T10:35:31.361514+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.10","version":"0'0","reported_seq":9,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-04-25T10:29:08.019405+0000","last_change":"2024-04-25T10:29:08.014110+0000","last_active":"2024-04-25T10:29:08.019405+0000","last_peered":"2024-04-25T10:29:08.019405+0000","last_clean":"2024-04-25T10:29:08.019405+0000","last_became_active":"2024-04-25T10:29:08.013543+0000","last_became_peered":"2024-04-25T10:29:08.013543+0000","last_unstale":"2024-04-25T10:29:08.019405+0000","last_undegraded":"2024-04-25T10:29:08.019405+0000","last_fullsized":"2024-04-25T10:29:08.019405+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:29:06.989570+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:29:06.989570+0000","last_clean_scrub_stamp":"2024-04-25T10:29:06.989570+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T21:35:43.931886+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7],"acting":[4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.11","version":"0'0","reported_seq":12,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-04-25T10:29:10.024238+0000","last_change":"2024-04-25T10:29:09.020490+0000","last_active":"2024-04-25T10:29:10.024238+0000","last_peered":"2024-04-25T10:29:10.024238+0000","last_clean":"2024-04-25T10:29:10.024238+0000","last_became_active":"2024-04-25T10:29:09.020178+0000","last_became_peered":"2024-04-25T10:29:09.020178+0000","last_unstale":"2024-04-25T10:29:10.024238+0000","last_undegraded":"2024-04-25T10:29:10.024238+0000","last_fullsized":"2024-04-25T10:29:10.024238+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:29:07.995096+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:29:07.995096+0000","last_clean_scrub_stamp":"2024-04-25T10:29:07.995096+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T20:37:20.726766+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.f","version":"22'1","reported_seq":15,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-04-25T10:29:10.038104+0000","last_change":"2024-04-25T10:29:08.029879+0000","last_active":"2024-04-25T10:29:10.038104+0000","last_peered":"2024-04-25T10:29:10.038104+0000","last_clean":"2024-04-25T10:29:10.038104+0000","last_became_active":"2024-04-25T10:29:08.029549+0000","last_became_peered":"2024-04-25T10:29:08.029549+0000","last_unstale":"2024-04-25T10:29:10.038104+0000","last_undegraded":"2024-04-25T10:29:10.038104+0000","last_fullsized":"2024-04-25T10:29:10.038104+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:29:06.989570+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:29:06.989570+0000","last_clean_scrub_stamp":"2024-04-25T10:29:06.989570+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T15:37:05.717854+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.e","version":"0'0","reported_seq":12,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-04-25T10:29:10.026273+0000","last_change":"2024-04-25T10:29:09.033703+0000","last_active":"2024-04-25T10:29:10.026273+0000","last_peered":"2024-04-25T10:29:10.026273+0000","last_clean":"2024-04-25T10:29:10.026273+0000","last_became_active":"2024-04-25T10:29:09.033318+0000","last_became_peered":"2024-04-25T10:29:09.033318+0000","last_unstale":"2024-04-25T10:29:10.026273+0000","last_undegraded":"2024-04-25T10:29:10.026273+0000","last_fullsized":"2024-04-25T10:29:10.026273+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:29:07.995096+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:29:07.995096+0000","last_clean_scrub_stamp":"2024-04-25T10:29:07.995096+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T12:16:47.983013+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.e","version":"0'0","reported_seq":9,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-04-25T10:29:08.027622+0000","last_change":"2024-04-25T10:29:08.020703+0000","last_active":"2024-04-25T10:29:08.027622+0000","last_peered":"2024-04-25T10:29:08.027622+0000","last_clean":"2024-04-25T10:29:08.027622+0000","last_became_active":"2024-04-25T10:29:08.020321+0000","last_became_peered":"2024-04-25T10:29:08.020321+0000","last_unstale":"2024-04-25T10:29:08.027622+0000","last_undegraded":"2024-04-25T10:29:08.027622+0000","last_fullsized":"2024-04-25T10:29:08.027622+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:29:06.989570+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:29:06.989570+0000","last_clean_scrub_stamp":"2024-04-25T10:29:06.989570+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T15:21:12.948208+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5],"acting":[4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.f","version":"0'0","reported_seq":12,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-04-25T10:29:10.038152+0000","last_change":"2024-04-25T10:29:09.022708+0000","last_active":"2024-04-25T10:29:10.038152+0000","last_peered":"2024-04-25T10:29:10.038152+0000","last_clean":"2024-04-25T10:29:10.038152+0000","last_became_active":"2024-04-25T10:29:09.020856+0000","last_became_peered":"2024-04-25T10:29:09.020856+0000","last_unstale":"2024-04-25T10:29:10.038152+0000","last_undegraded":"2024-04-25T10:29:10.038152+0000","last_fullsized":"2024-04-25T10:29:10.038152+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:29:07.995096+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:29:07.995096+0000","last_clean_scrub_stamp":"2024-04-25T10:29:07.995096+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T15:28:23.689564+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.d","version":"22'1","reported_seq":15,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-04-25T10:29:10.189469+0000","last_change":"2024-04-25T10:29:08.019488+0000","last_active":"2024-04-25T10:29:10.189469+0000","last_peered":"2024-04-25T10:29:10.189469+0000","last_clean":"2024-04-25T10:29:10.189469+0000","last_became_active":"2024-04-25T10:29:08.018971+0000","last_became_peered":"2024-04-25T10:29:08.018971+0000","last_unstale":"2024-04-25T10:29:10.189469+0000","last_undegraded":"2024-04-25T10:29:10.189469+0000","last_fullsized":"2024-04-25T10:29:10.189469+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:29:06.989570+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:29:06.989570+0000","last_clean_scrub_stamp":"2024-04-25T10:29:06.989570+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T20:22:56.380539+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":70,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.c","version":"0'0","reported_seq":12,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-04-25T10:29:10.037902+0000","last_change":"2024-04-25T10:29:09.029666+0000","last_active":"2024-04-25T10:29:10.037902+0000","last_peered":"2024-04-25T10:29:10.037902+0000","last_clean":"2024-04-25T10:29:10.037902+0000","last_became_active":"2024-04-25T10:29:09.029184+0000","last_became_peered":"2024-04-25T10:29:09.029184+0000","last_unstale":"2024-04-25T10:29:10.037902+0000","last_undegraded":"2024-04-25T10:29:10.037902+0000","last_fullsized":"2024-04-25T10:29:10.037902+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:29:07.995096+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:29:07.995096+0000","last_clean_scrub_stamp":"2024-04-25T10:29:07.995096+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T13:54:22.968216+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.c","version":"0'0","reported_seq":14,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-04-25T10:29:10.189541+0000","last_change":"2024-04-25T10:29:08.012124+0000","last_active":"2024-04-25T10:29:10.189541+0000","last_peered":"2024-04-25T10:29:10.189541+0000","last_clean":"2024-04-25T10:29:10.189541+0000","last_became_active":"2024-04-25T10:29:08.011817+0000","last_became_peered":"2024-04-25T10:29:08.011817+0000","last_unstale":"2024-04-25T10:29:10.189541+0000","last_undegraded":"2024-04-25T10:29:10.189541+0000","last_fullsized":"2024-04-25T10:29:10.189541+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:29:06.989570+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:29:06.989570+0000","last_clean_scrub_stamp":"2024-04-25T10:29:06.989570+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T20:22:56.380539+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.d","version":"0'0","reported_seq":12,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-04-25T10:29:10.024815+0000","last_change":"2024-04-25T10:29:09.017227+0000","last_active":"2024-04-25T10:29:10.024815+0000","last_peered":"2024-04-25T10:29:10.024815+0000","last_clean":"2024-04-25T10:29:10.024815+0000","last_became_active":"2024-04-25T10:29:09.016911+0000","last_became_peered":"2024-04-25T10:29:09.016911+0000","last_unstale":"2024-04-25T10:29:10.024815+0000","last_undegraded":"2024-04-25T10:29:10.024815+0000","last_fullsized":"2024-04-25T10:29:10.024815+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:29:07.995096+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:29:07.995096+0000","last_clean_scrub_stamp":"2024-04-25T10:29:07.995096+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T18:51:41.934269+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.0","version":"18'72","reported_seq":110,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-04-25T10:29:10.187517+0000","last_change":"2024-04-25T10:28:57.972696+0000","last_active":"2024-04-25T10:29:10.187517+0000","last_peered":"2024-04-25T10:29:10.187517+0000","last_clean":"2024-04-25T10:29:10.187517+0000","last_became_active":"2024-04-25T10:28:57.972373+0000","last_became_peered":"2024-04-25T10:28:57.972373+0000","last_unstale":"2024-04-25T10:29:10.187517+0000","last_undegraded":"2024-04-25T10:29:10.187517+0000","last_fullsized":"2024-04-25T10:29:10.187517+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:28:56.959439+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:28:56.959439+0000","last_clean_scrub_stamp":"2024-04-25T10:28:56.959439+0000","objects_scrubbed":0,"log_size":72,"log_dups_size":0,"ondisk_log_size":72,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T17:34:34.057408+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":66,"num_read_kb":55,"num_write":101,"num_write_kb":1628,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.3","version":"0'0","reported_seq":14,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-04-25T10:29:10.024562+0000","last_change":"2024-04-25T10:29:08.025080+0000","last_active":"2024-04-25T10:29:10.024562+0000","last_peered":"2024-04-25T10:29:10.024562+0000","last_clean":"2024-04-25T10:29:10.024562+0000","last_became_active":"2024-04-25T10:29:08.024747+0000","last_became_peered":"2024-04-25T10:29:08.024747+0000","last_unstale":"2024-04-25T10:29:10.024562+0000","last_undegraded":"2024-04-25T10:29:10.024562+0000","last_fullsized":"2024-04-25T10:29:10.024562+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:29:06.989570+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:29:06.989570+0000","last_clean_scrub_stamp":"2024-04-25T10:29:06.989570+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T22:07:59.361905+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.2","version":"0'0","reported_seq":1,"reported_epoch":21,"state":"creating+peering","last_fresh":"2024-04-25T10:29:08.008681+0000","last_change":"2024-04-25T10:29:08.008681+0000","last_active":"2024-04-25T10:29:07.995096+0000","last_peered":"2024-04-25T10:29:07.995096+0000","last_clean":"2024-04-25T10:29:07.995096+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T10:29:08.008681+0000","last_undegraded":"2024-04-25T10:29:08.008681+0000","last_fullsized":"2024-04-25T10:29:08.008681+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:29:07.995096+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:29:07.995096+0000","last_clean_scrub_stamp":"2024-04-25T10:29:07.995096+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.0","version":"0'0","reported_seq":14,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-04-25T10:29:10.024309+0000","last_change":"2024-04-25T10:29:08.027479+0000","last_active":"2024-04-25T10:29:10.024309+0000","last_peered":"2024-04-25T10:29:10.024309+0000","last_clean":"2024-04-25T10:29:10.024309+0000","last_became_active":"2024-04-25T10:29:08.027178+0000","last_became_peered":"2024-04-25T10:29:08.027178+0000","last_unstale":"2024-04-25T10:29:10.024309+0000","last_undegraded":"2024-04-25T10:29:10.024309+0000","last_fullsized":"2024-04-25T10:29:10.024309+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:29:06.989570+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:29:06.989570+0000","last_clean_scrub_stamp":"2024-04-25T10:29:06.989570+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T19:54:16.697792+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.1","version":"0'0","reported_seq":12,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-04-25T10:29:10.188195+0000","last_change":"2024-04-25T10:29:09.028777+0000","last_active":"2024-04-25T10:29:10.188195+0000","last_peered":"2024-04-25T10:29:10.188195+0000","last_clean":"2024-04-25T10:29:10.188195+0000","last_became_active":"2024-04-25T10:29:09.027709+0000","last_became_peered":"2024-04-25T10:29:09.027709+0000","last_unstale":"2024-04-25T10:29:10.188195+0000","last_undegraded":"2024-04-25T10:29:10.188195+0000","last_fullsized":"2024-04-25T10:29:10.188195+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:29:07.995096+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:29:07.995096+0000","last_clean_scrub_stamp":"2024-04-25T10:29:07.995096+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T22:08:00.367431+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,2],"acting":[6,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.1","version":"0'0","reported_seq":14,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-04-25T10:29:10.188429+0000","last_change":"2024-04-25T10:29:08.020944+0000","last_active":"2024-04-25T10:29:10.188429+0000","last_peered":"2024-04-25T10:29:10.188429+0000","last_clean":"2024-04-25T10:29:10.188429+0000","last_became_active":"2024-04-25T10:29:08.020259+0000","last_became_peered":"2024-04-25T10:29:08.020259+0000","last_unstale":"2024-04-25T10:29:10.188429+0000","last_undegraded":"2024-04-25T10:29:10.188429+0000","last_fullsized":"2024-04-25T10:29:10.188429+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:29:06.989570+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:29:06.989570+0000","last_clean_scrub_stamp":"2024-04-25T10:29:06.989570+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T16:31:11.685101+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.0","version":"0'0","reported_seq":12,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-04-25T10:29:10.189305+0000","last_change":"2024-04-25T10:29:09.035455+0000","last_active":"2024-04-25T10:29:10.189305+0000","last_peered":"2024-04-25T10:29:10.189305+0000","last_clean":"2024-04-25T10:29:10.189305+0000","last_became_active":"2024-04-25T10:29:09.035130+0000","last_became_peered":"2024-04-25T10:29:09.035130+0000","last_unstale":"2024-04-25T10:29:10.189305+0000","last_undegraded":"2024-04-25T10:29:10.189305+0000","last_fullsized":"2024-04-25T10:29:10.189305+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:29:07.995096+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:29:07.995096+0000","last_clean_scrub_stamp":"2024-04-25T10:29:07.995096+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T11:52:24.415332+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.2","version":"0'0","reported_seq":14,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-04-25T10:29:10.024714+0000","last_change":"2024-04-25T10:29:08.021694+0000","last_active":"2024-04-25T10:29:10.024714+0000","last_peered":"2024-04-25T10:29:10.024714+0000","last_clean":"2024-04-25T10:29:10.024714+0000","last_became_active":"2024-04-25T10:29:08.021052+0000","last_became_peered":"2024-04-25T10:29:08.021052+0000","last_unstale":"2024-04-25T10:29:10.024714+0000","last_undegraded":"2024-04-25T10:29:10.024714+0000","last_fullsized":"2024-04-25T10:29:10.024714+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:29:06.989570+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:29:06.989570+0000","last_clean_scrub_stamp":"2024-04-25T10:29:06.989570+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T21:14:38.656431+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.3","version":"0'0","reported_seq":12,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-04-25T10:29:10.037784+0000","last_change":"2024-04-25T10:29:09.021686+0000","last_active":"2024-04-25T10:29:10.037784+0000","last_peered":"2024-04-25T10:29:10.037784+0000","last_clean":"2024-04-25T10:29:10.037784+0000","last_became_active":"2024-04-25T10:29:09.020672+0000","last_became_peered":"2024-04-25T10:29:09.020672+0000","last_unstale":"2024-04-25T10:29:10.037784+0000","last_undegraded":"2024-04-25T10:29:10.037784+0000","last_fullsized":"2024-04-25T10:29:10.037784+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:29:07.995096+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:29:07.995096+0000","last_clean_scrub_stamp":"2024-04-25T10:29:07.995096+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T20:52:02.098872+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.4","version":"0'0","reported_seq":14,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-04-25T10:29:10.189133+0000","last_change":"2024-04-25T10:29:08.025582+0000","last_active":"2024-04-25T10:29:10.189133+0000","last_peered":"2024-04-25T10:29:10.189133+0000","last_clean":"2024-04-25T10:29:10.189133+0000","last_became_active":"2024-04-25T10:29:08.024961+0000","last_became_peered":"2024-04-25T10:29:08.024961+0000","last_unstale":"2024-04-25T10:29:10.189133+0000","last_undegraded":"2024-04-25T10:29:10.189133+0000","last_fullsized":"2024-04-25T10:29:10.189133+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:29:06.989570+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:29:06.989570+0000","last_clean_scrub_stamp":"2024-04-25T10:29:06.989570+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T10:32:06.103123+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6],"acting":[5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.5","version":"0'0","reported_seq":12,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-04-25T10:29:10.026083+0000","last_change":"2024-04-25T10:29:09.028003+0000","last_active":"2024-04-25T10:29:10.026083+0000","last_peered":"2024-04-25T10:29:10.026083+0000","last_clean":"2024-04-25T10:29:10.026083+0000","last_became_active":"2024-04-25T10:29:09.027654+0000","last_became_peered":"2024-04-25T10:29:09.027654+0000","last_unstale":"2024-04-25T10:29:10.026083+0000","last_undegraded":"2024-04-25T10:29:10.026083+0000","last_fullsized":"2024-04-25T10:29:10.026083+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:29:07.995096+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:29:07.995096+0000","last_clean_scrub_stamp":"2024-04-25T10:29:07.995096+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T19:54:17.703318+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,6],"acting":[2,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.5","version":"0'0","reported_seq":14,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-04-25T10:29:10.024926+0000","last_change":"2024-04-25T10:29:08.026181+0000","last_active":"2024-04-25T10:29:10.024926+0000","last_peered":"2024-04-25T10:29:10.024926+0000","last_clean":"2024-04-25T10:29:10.024926+0000","last_became_active":"2024-04-25T10:29:08.025780+0000","last_became_peered":"2024-04-25T10:29:08.025780+0000","last_unstale":"2024-04-25T10:29:10.024926+0000","last_undegraded":"2024-04-25T10:29:10.024926+0000","last_fullsized":"2024-04-25T10:29:10.024926+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:29:06.989570+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:29:06.989570+0000","last_clean_scrub_stamp":"2024-04-25T10:29:06.989570+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T12:16:46.977487+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.4","version":"0'0","reported_seq":12,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-04-25T10:29:10.189423+0000","last_change":"2024-04-25T10:29:09.037824+0000","last_active":"2024-04-25T10:29:10.189423+0000","last_peered":"2024-04-25T10:29:10.189423+0000","last_clean":"2024-04-25T10:29:10.189423+0000","last_became_active":"2024-04-25T10:29:09.037478+0000","last_became_peered":"2024-04-25T10:29:09.037478+0000","last_unstale":"2024-04-25T10:29:10.189423+0000","last_undegraded":"2024-04-25T10:29:10.189423+0000","last_fullsized":"2024-04-25T10:29:10.189423+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:29:07.995096+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:29:07.995096+0000","last_clean_scrub_stamp":"2024-04-25T10:29:07.995096+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T15:02:17.480054+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.6","version":"0'0","reported_seq":9,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-04-25T10:29:08.027595+0000","last_change":"2024-04-25T10:29:08.020327+0000","last_active":"2024-04-25T10:29:08.027595+0000","last_peered":"2024-04-25T10:29:08.027595+0000","last_clean":"2024-04-25T10:29:08.027595+0000","last_became_active":"2024-04-25T10:29:08.019988+0000","last_became_peered":"2024-04-25T10:29:08.019988+0000","last_unstale":"2024-04-25T10:29:08.027595+0000","last_undegraded":"2024-04-25T10:29:08.027595+0000","last_fullsized":"2024-04-25T10:29:08.027595+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:29:06.989570+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:29:06.989570+0000","last_clean_scrub_stamp":"2024-04-25T10:29:06.989570+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T12:07:01.943484+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.7","version":"0'0","reported_seq":1,"reported_epoch":21,"state":"creating+peering","last_fresh":"2024-04-25T10:29:08.013060+0000","last_change":"2024-04-25T10:29:08.013060+0000","last_active":"2024-04-25T10:29:07.995096+0000","last_peered":"2024-04-25T10:29:07.995096+0000","last_clean":"2024-04-25T10:29:07.995096+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T10:29:08.013060+0000","last_undegraded":"2024-04-25T10:29:08.013060+0000","last_fullsized":"2024-04-25T10:29:08.013060+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:29:07.995096+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:29:07.995096+0000","last_clean_scrub_stamp":"2024-04-25T10:29:07.995096+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.7","version":"22'2","reported_seq":16,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-04-25T10:29:10.024876+0000","last_change":"2024-04-25T10:29:08.026080+0000","last_active":"2024-04-25T10:29:10.024876+0000","last_peered":"2024-04-25T10:29:10.024876+0000","last_clean":"2024-04-25T10:29:10.024876+0000","last_became_active":"2024-04-25T10:29:08.025707+0000","last_became_peered":"2024-04-25T10:29:08.025707+0000","last_unstale":"2024-04-25T10:29:10.024876+0000","last_undegraded":"2024-04-25T10:29:10.024876+0000","last_fullsized":"2024-04-25T10:29:10.024876+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:29:06.989570+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:29:06.989570+0000","last_clean_scrub_stamp":"2024-04-25T10:29:06.989570+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T10:55:00.499631+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.6","version":"0'0","reported_seq":1,"reported_epoch":21,"state":"creating+peering","last_fresh":"2024-04-25T10:29:08.009604+0000","last_change":"2024-04-25T10:29:08.009604+0000","last_active":"2024-04-25T10:29:07.995096+0000","last_peered":"2024-04-25T10:29:07.995096+0000","last_clean":"2024-04-25T10:29:07.995096+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T10:29:08.009604+0000","last_undegraded":"2024-04-25T10:29:08.009604+0000","last_fullsized":"2024-04-25T10:29:08.009604+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:29:07.995096+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:29:07.995096+0000","last_clean_scrub_stamp":"2024-04-25T10:29:07.995096+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7],"acting":[4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.8","version":"0'0","reported_seq":14,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-04-25T10:29:10.037271+0000","last_change":"2024-04-25T10:29:08.024845+0000","last_active":"2024-04-25T10:29:10.037271+0000","last_peered":"2024-04-25T10:29:10.037271+0000","last_clean":"2024-04-25T10:29:10.037271+0000","last_became_active":"2024-04-25T10:29:08.024484+0000","last_became_peered":"2024-04-25T10:29:08.024484+0000","last_unstale":"2024-04-25T10:29:10.037271+0000","last_undegraded":"2024-04-25T10:29:10.037271+0000","last_fullsized":"2024-04-25T10:29:10.037271+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:29:06.989570+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:29:06.989570+0000","last_clean_scrub_stamp":"2024-04-25T10:29:06.989570+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T14:12:57.950482+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.9","version":"0'0","reported_seq":12,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-04-25T10:29:10.024346+0000","last_change":"2024-04-25T10:29:09.022502+0000","last_active":"2024-04-25T10:29:10.024346+0000","last_peered":"2024-04-25T10:29:10.024346+0000","last_clean":"2024-04-25T10:29:10.024346+0000","last_became_active":"2024-04-25T10:29:09.022184+0000","last_became_peered":"2024-04-25T10:29:09.022184+0000","last_unstale":"2024-04-25T10:29:10.024346+0000","last_undegraded":"2024-04-25T10:29:10.024346+0000","last_fullsized":"2024-04-25T10:29:10.024346+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:29:07.995096+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:29:07.995096+0000","last_clean_scrub_stamp":"2024-04-25T10:29:07.995096+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T19:56:27.663748+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.9","version":"0'0","reported_seq":9,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-04-25T10:29:08.027947+0000","last_change":"2024-04-25T10:29:08.024989+0000","last_active":"2024-04-25T10:29:08.027947+0000","last_peered":"2024-04-25T10:29:08.027947+0000","last_clean":"2024-04-25T10:29:08.027947+0000","last_became_active":"2024-04-25T10:29:08.024655+0000","last_became_peered":"2024-04-25T10:29:08.024655+0000","last_unstale":"2024-04-25T10:29:08.027947+0000","last_undegraded":"2024-04-25T10:29:08.027947+0000","last_fullsized":"2024-04-25T10:29:08.027947+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:29:06.989570+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:29:06.989570+0000","last_clean_scrub_stamp":"2024-04-25T10:29:06.989570+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T13:46:20.443676+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.8","version":"0'0","reported_seq":1,"reported_epoch":21,"state":"creating+peering","last_fresh":"2024-04-25T10:29:08.009188+0000","last_change":"2024-04-25T10:29:08.009188+0000","last_active":"2024-04-25T10:29:07.995096+0000","last_peered":"2024-04-25T10:29:07.995096+0000","last_clean":"2024-04-25T10:29:07.995096+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T10:29:08.009188+0000","last_undegraded":"2024-04-25T10:29:08.009188+0000","last_fullsized":"2024-04-25T10:29:08.009188+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:29:07.995096+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:29:07.995096+0000","last_clean_scrub_stamp":"2024-04-25T10:29:07.995096+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7],"acting":[4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.a","version":"0'0","reported_seq":14,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-04-25T10:29:10.188957+0000","last_change":"2024-04-25T10:29:08.020363+0000","last_active":"2024-04-25T10:29:10.188957+0000","last_peered":"2024-04-25T10:29:10.188957+0000","last_clean":"2024-04-25T10:29:10.188957+0000","last_became_active":"2024-04-25T10:29:08.020037+0000","last_became_peered":"2024-04-25T10:29:08.020037+0000","last_unstale":"2024-04-25T10:29:10.188957+0000","last_undegraded":"2024-04-25T10:29:10.188957+0000","last_fullsized":"2024-04-25T10:29:10.188957+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:29:06.989570+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:29:06.989570+0000","last_clean_scrub_stamp":"2024-04-25T10:29:06.989570+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T10:32:06.103123+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.b","version":"0'0","reported_seq":12,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-04-25T10:29:10.189294+0000","last_change":"2024-04-25T10:29:09.028820+0000","last_active":"2024-04-25T10:29:10.189294+0000","last_peered":"2024-04-25T10:29:10.189294+0000","last_clean":"2024-04-25T10:29:10.189294+0000","last_became_active":"2024-04-25T10:29:09.028426+0000","last_became_peered":"2024-04-25T10:29:09.028426+0000","last_unstale":"2024-04-25T10:29:10.189294+0000","last_undegraded":"2024-04-25T10:29:10.189294+0000","last_fullsized":"2024-04-25T10:29:10.189294+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:29:07.995096+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:29:07.995096+0000","last_clean_scrub_stamp":"2024-04-25T10:29:07.995096+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T19:54:17.703318+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.b","version":"22'2","reported_seq":16,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-04-25T10:29:10.024445+0000","last_change":"2024-04-25T10:29:08.025011+0000","last_active":"2024-04-25T10:29:10.024445+0000","last_peered":"2024-04-25T10:29:10.024445+0000","last_clean":"2024-04-25T10:29:10.024445+0000","last_became_active":"2024-04-25T10:29:08.024667+0000","last_became_peered":"2024-04-25T10:29:08.024667+0000","last_unstale":"2024-04-25T10:29:10.024445+0000","last_undegraded":"2024-04-25T10:29:10.024445+0000","last_fullsized":"2024-04-25T10:29:10.024445+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:29:06.989570+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:29:06.989570+0000","last_clean_scrub_stamp":"2024-04-25T10:29:06.989570+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T15:37:05.717854+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.a","version":"0'0","reported_seq":12,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-04-25T10:29:10.024394+0000","last_change":"2024-04-25T10:29:09.017160+0000","last_active":"2024-04-25T10:29:10.024394+0000","last_peered":"2024-04-25T10:29:10.024394+0000","last_clean":"2024-04-25T10:29:10.024394+0000","last_became_active":"2024-04-25T10:29:09.016821+0000","last_became_peered":"2024-04-25T10:29:09.016821+0000","last_unstale":"2024-04-25T10:29:10.024394+0000","last_undegraded":"2024-04-25T10:29:10.024394+0000","last_fullsized":"2024-04-25T10:29:10.024394+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:29:07.995096+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:29:07.995096+0000","last_clean_scrub_stamp":"2024-04-25T10:29:07.995096+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T20:40:19.840339+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.1d","version":"0'0","reported_seq":12,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-04-25T10:29:10.024622+0000","last_change":"2024-04-25T10:29:09.030341+0000","last_active":"2024-04-25T10:29:10.024622+0000","last_peered":"2024-04-25T10:29:10.024622+0000","last_clean":"2024-04-25T10:29:10.024622+0000","last_became_active":"2024-04-25T10:29:09.029988+0000","last_became_peered":"2024-04-25T10:29:09.029988+0000","last_unstale":"2024-04-25T10:29:10.024622+0000","last_undegraded":"2024-04-25T10:29:10.024622+0000","last_fullsized":"2024-04-25T10:29:10.024622+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:29:07.995096+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:29:07.995096+0000","last_clean_scrub_stamp":"2024-04-25T10:29:07.995096+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T11:40:18.624659+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.1c","version":"22'2","reported_seq":16,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-04-25T10:29:10.037969+0000","last_change":"2024-04-25T10:29:08.027197+0000","last_active":"2024-04-25T10:29:10.037969+0000","last_peered":"2024-04-25T10:29:10.037969+0000","last_clean":"2024-04-25T10:29:10.037969+0000","last_became_active":"2024-04-25T10:29:08.026872+0000","last_became_peered":"2024-04-25T10:29:08.026872+0000","last_unstale":"2024-04-25T10:29:10.037969+0000","last_undegraded":"2024-04-25T10:29:10.037969+0000","last_fullsized":"2024-04-25T10:29:10.037969+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:29:06.989570+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:29:06.989570+0000","last_clean_scrub_stamp":"2024-04-25T10:29:06.989570+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T15:58:35.843305+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.1c","version":"0'0","reported_seq":12,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-04-25T10:29:10.187853+0000","last_change":"2024-04-25T10:29:09.032731+0000","last_active":"2024-04-25T10:29:10.187853+0000","last_peered":"2024-04-25T10:29:10.187853+0000","last_clean":"2024-04-25T10:29:10.187853+0000","last_became_active":"2024-04-25T10:29:09.032383+0000","last_became_peered":"2024-04-25T10:29:09.032383+0000","last_unstale":"2024-04-25T10:29:10.187853+0000","last_undegraded":"2024-04-25T10:29:10.187853+0000","last_fullsized":"2024-04-25T10:29:10.187853+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:29:07.995096+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:29:07.995096+0000","last_clean_scrub_stamp":"2024-04-25T10:29:07.995096+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T19:54:17.703318+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.1d","version":"22'2","reported_seq":16,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-04-25T10:29:10.037807+0000","last_change":"2024-04-25T10:29:08.026410+0000","last_active":"2024-04-25T10:29:10.037807+0000","last_peered":"2024-04-25T10:29:10.037807+0000","last_clean":"2024-04-25T10:29:10.037807+0000","last_became_active":"2024-04-25T10:29:08.026071+0000","last_became_peered":"2024-04-25T10:29:08.026071+0000","last_unstale":"2024-04-25T10:29:10.037807+0000","last_undegraded":"2024-04-25T10:29:10.037807+0000","last_fullsized":"2024-04-25T10:29:10.037807+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:29:06.989570+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:29:06.989570+0000","last_clean_scrub_stamp":"2024-04-25T10:29:06.989570+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T19:54:16.697792+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.1f","version":"0'0","reported_seq":12,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-04-25T10:29:10.189240+0000","last_change":"2024-04-25T10:29:09.032735+0000","last_active":"2024-04-25T10:29:10.189240+0000","last_peered":"2024-04-25T10:29:10.189240+0000","last_clean":"2024-04-25T10:29:10.189240+0000","last_became_active":"2024-04-25T10:29:09.032227+0000","last_became_peered":"2024-04-25T10:29:09.032227+0000","last_unstale":"2024-04-25T10:29:10.189240+0000","last_undegraded":"2024-04-25T10:29:10.189240+0000","last_fullsized":"2024-04-25T10:29:10.189240+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:29:07.995096+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:29:07.995096+0000","last_clean_scrub_stamp":"2024-04-25T10:29:07.995096+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T11:31:45.283247+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.1e","version":"22'1","reported_seq":15,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-04-25T10:29:10.189728+0000","last_change":"2024-04-25T10:29:08.017025+0000","last_active":"2024-04-25T10:29:10.189728+0000","last_peered":"2024-04-25T10:29:10.189728+0000","last_clean":"2024-04-25T10:29:10.189728+0000","last_became_active":"2024-04-25T10:29:08.016702+0000","last_became_peered":"2024-04-25T10:29:08.016702+0000","last_unstale":"2024-04-25T10:29:10.189728+0000","last_undegraded":"2024-04-25T10:29:10.189728+0000","last_fullsized":"2024-04-25T10:29:10.189728+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:29:06.989570+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:29:06.989570+0000","last_clean_scrub_stamp":"2024-04-25T10:29:06.989570+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T14:12:57.950482+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.1e","version":"0'0","reported_seq":12,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-04-25T10:29:10.189730+0000","last_change":"2024-04-25T10:29:09.036208+0000","last_active":"2024-04-25T10:29:10.189730+0000","last_peered":"2024-04-25T10:29:10.189730+0000","last_clean":"2024-04-25T10:29:10.189730+0000","last_became_active":"2024-04-25T10:29:09.034890+0000","last_became_peered":"2024-04-25T10:29:09.034890+0000","last_unstale":"2024-04-25T10:29:10.189730+0000","last_undegraded":"2024-04-25T10:29:10.189730+0000","last_fullsized":"2024-04-25T10:29:10.189730+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:29:07.995096+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:29:07.995096+0000","last_clean_scrub_stamp":"2024-04-25T10:29:07.995096+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T18:44:23.610374+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.1f","version":"0'0","reported_seq":9,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-04-25T10:29:08.027976+0000","last_change":"2024-04-25T10:29:08.026202+0000","last_active":"2024-04-25T10:29:08.027976+0000","last_peered":"2024-04-25T10:29:08.027976+0000","last_clean":"2024-04-25T10:29:08.027976+0000","last_became_active":"2024-04-25T10:29:08.025877+0000","last_became_peered":"2024-04-25T10:29:08.025877+0000","last_unstale":"2024-04-25T10:29:08.027976+0000","last_undegraded":"2024-04-25T10:29:08.027976+0000","last_fullsized":"2024-04-25T10:29:08.027976+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:29:06.989570+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:29:06.989570+0000","last_clean_scrub_stamp":"2024-04-25T10:29:06.989570+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T18:56:03.528807+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]}],"pool_stats":[{"poolid":3,"num_pg":64,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":128,"acting":128,"num_store_stats":8},{"poolid":2,"num_pg":32,"stat_sum":{"num_bytes":1828,"num_objects":14,"num_object_clones":0,"num_object_copies":28,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":14,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":25,"num_write_kb":13,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":7,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":45056,"data_stored":2536,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":24,"ondisk_log_size":24,"up":64,"acting":64,"num_store_stats":8},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":66,"num_read_kb":55,"num_write":101,"num_write_kb":1628,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1187840,"data_stored":1180736,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":72,"ondisk_log_size":72,"up":2,"acting":2,"num_store_stats":2}],"osd_stats":[{"osd":7,"up_from":15,"seq":64424509445,"num_pgs":3,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27244,"kb_used_data":224,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344596,"statfs":{"total":96636764160,"available":96608866304,"internally_reserved":0,"allocated":229376,"data_stored":75059,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1590,"internal_metadata":27654602},"hb_peers":[0,1,2,3,4,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[]},{"osd":6,"up_from":15,"seq":64424509446,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27236,"kb_used_data":216,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344604,"statfs":{"total":96636764160,"available":96608874496,"internally_reserved":0,"allocated":221184,"data_stored":74935,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":27654598},"hb_peers":[0,1,2,3,4,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[]},{"osd":5,"up_from":15,"seq":64424509447,"num_pgs":1,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27824,"kb_used_data":804,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344016,"statfs":{"total":96636764160,"available":96608272384,"internally_reserved":0,"allocated":823296,"data_stored":665395,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":6,"apply_latency_ms":6,"commit_latency_ns":6000000,"apply_latency_ns":6000000},"alerts":[]},{"osd":3,"up_from":15,"seq":64424509447,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27240,"kb_used_data":220,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344600,"statfs":{"total":96636764160,"available":96608870400,"internally_reserved":0,"allocated":225280,"data_stored":75517,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1593,"internal_metadata":27654599},"hb_peers":[0,1,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[]},{"osd":2,"up_from":15,"seq":64424509447,"num_pgs":1,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27248,"kb_used_data":228,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344592,"statfs":{"total":96636764160,"available":96608862208,"internally_reserved":0,"allocated":233472,"data_stored":76547,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[]},{"osd":1,"up_from":15,"seq":64424509445,"num_pgs":12,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27820,"kb_used_data":800,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344020,"statfs":{"total":96636764160,"available":96608276480,"internally_reserved":0,"allocated":819200,"data_stored":665373,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[]},{"osd":0,"up_from":15,"seq":64424509446,"num_pgs":35,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27244,"kb_used_data":224,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344596,"statfs":{"total":96636764160,"available":96608866304,"internally_reserved":0,"allocated":229376,"data_stored":74991,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[1,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":5,"apply_latency_ms":5,"commit_latency_ns":5000000,"apply_latency_ns":5000000},"alerts":[]},{"osd":4,"up_from":13,"seq":55834574853,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27212,"kb_used_data":192,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344628,"statfs":{"total":96636764160,"available":96608899072,"internally_reserved":0,"allocated":196608,"data_stored":61885,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":56,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":70,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":1612,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":582,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":92,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":124,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-04-25T10:29:13.563 DEBUG:teuthology.orchestra.run.smithi037:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-04-25T10:29:13.710 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:13.708+0000 7f57ba851640 1 Processor -- start 2024-04-25T10:29:13.710 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:13.708+0000 7f57ba851640 1 -- start start 2024-04-25T10:29:13.712 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:13.709+0000 7f57ba851640 1 --2- >> v2:172.21.15.203:3300/0 conn(0x7f57b40d1c20 0x7f57b40d2030 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:29:13.712 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:13.709+0000 7f57ba851640 1 -- --> v1:172.21.15.37:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f57b40d2570 con 0x7f57b408cc90 2024-04-25T10:29:13.712 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:13.709+0000 7f57ba851640 1 -- --> v1:172.21.15.203:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f57b40b8d40 con 0x7f57b40b71b0 2024-04-25T10:29:13.712 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:13.709+0000 7f57ba851640 1 -- --> v2:172.21.15.203:3300/0 -- mon_getmap magic: 0 -- 0x7f57b409d2d0 con 0x7f57b40d1c20 2024-04-25T10:29:13.712 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:13.709+0000 7f57b3fff640 1 --1- >> v1:172.21.15.37:6789/0 conn(0x7f57b408cc90 0x7f57b409cbc0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.37:6789/0 says I am v1:172.21.15.37:51428/0 (socket says 172.21.15.37:51428) 2024-04-25T10:29:13.712 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:13.709+0000 7f57b3fff640 1 -- 172.21.15.37:0/3034479861 learned_addr learned my addr 172.21.15.37:0/3034479861 (peer_addr_for_me v1:172.21.15.37:0/0) 2024-04-25T10:29:13.712 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:13.710+0000 7f57b37fe640 1 --2- 172.21.15.37:0/3034479861 >> v2:172.21.15.203:3300/0 conn(0x7f57b40d1c20 0x7f57b40d2030 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:29:13.713 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:13.710+0000 7f57b2ffd640 1 -- 172.21.15.37:0/3034479861 <== mon.0 v1:172.21.15.37:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 3132114170 0 0) 0x7f57a8001110 con 0x7f57b408cc90 2024-04-25T10:29:13.713 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:13.710+0000 7f57b2ffd640 1 -- 172.21.15.37:0/3034479861 --> v1:172.21.15.37:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f579c003520 con 0x7f57b408cc90 2024-04-25T10:29:13.713 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:13.710+0000 7f57b37fe640 1 -- 172.21.15.37:0/3034479861 >> v1:172.21.15.203:6789/0 conn(0x7f57b40b71b0 legacy=0x7f57b40b75c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:29:13.713 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:13.710+0000 7f57b2ffd640 1 -- 172.21.15.37:0/3034479861 <== mon.0 v1:172.21.15.37:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 895485847 0 0) 0x7f579c003520 con 0x7f57b408cc90 2024-04-25T10:29:13.713 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:13.710+0000 7f57b37fe640 1 -- 172.21.15.37:0/3034479861 >> v1:172.21.15.37:6789/0 conn(0x7f57b408cc90 legacy=0x7f57b409cbc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:29:13.713 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:13.710+0000 7f57b37fe640 1 -- 172.21.15.37:0/3034479861 --> v2:172.21.15.203:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f57b40d27f0 con 0x7f57b40d1c20 2024-04-25T10:29:13.714 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:13.711+0000 7f57b37fe640 1 --2- 172.21.15.37:0/3034479861 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f57b40d1c20 0x7f57b40d2030 secure :-1 s=READY pgs=59 cs=0 l=1 rev1=1 crypto rx=0x7f57a40097a0 tx=0x7f57a402a1c0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=f693eb517830891f server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:29:13.714 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:13.711+0000 7f57b2ffd640 1 -- 172.21.15.37:0/3034479861 <== mon.1 v2:172.21.15.203:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f57a4037070 con 0x7f57b40d1c20 2024-04-25T10:29:13.714 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:13.711+0000 7f57b2ffd640 1 -- 172.21.15.37:0/3034479861 <== mon.1 v2:172.21.15.203:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f57a402ac50 con 0x7f57b40d1c20 2024-04-25T10:29:13.714 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:13.711+0000 7f57b2ffd640 1 -- 172.21.15.37:0/3034479861 <== mon.1 v2:172.21.15.203:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f57a8001840 con 0x7f57b40d1c20 2024-04-25T10:29:13.714 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:13.712+0000 7f57ba851640 1 -- 172.21.15.37:0/3034479861 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f57b40d1c20 msgr2=0x7f57b40d2030 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:29:13.714 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:13.712+0000 7f57ba851640 1 --2- 172.21.15.37:0/3034479861 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f57b40d1c20 0x7f57b40d2030 secure :-1 s=READY pgs=59 cs=0 l=1 rev1=1 crypto rx=0x7f57a40097a0 tx=0x7f57a402a1c0 comp rx=0 tx=0).stop 2024-04-25T10:29:13.714 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:13.712+0000 7f57ba851640 1 -- 172.21.15.37:0/3034479861 shutdown_connections 2024-04-25T10:29:13.714 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:13.712+0000 7f57ba851640 1 --2- 172.21.15.37:0/3034479861 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f57b40d1c20 0x7f57b40d2030 unknown :-1 s=CLOSED pgs=59 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:29:13.714 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:13.712+0000 7f57ba851640 1 -- 172.21.15.37:0/3034479861 >> 172.21.15.37:0/3034479861 conn(0x7f57b40d03d0 msgr2=0x7f57b40d07e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:29:13.715 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:13.712+0000 7f57ba851640 1 -- 172.21.15.37:0/3034479861 shutdown_connections 2024-04-25T10:29:13.715 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:13.712+0000 7f57ba851640 1 -- 172.21.15.37:0/3034479861 wait complete. 2024-04-25T10:29:13.715 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:13.713+0000 7f57ba851640 1 Processor -- start 2024-04-25T10:29:13.716 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:13.714+0000 7f57ba851640 1 -- start start 2024-04-25T10:29:13.718 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:13.715+0000 7f57ba851640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f57b40b71b0 0x7f57b41dd700 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:29:13.718 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:13.715+0000 7f57ba851640 1 --2- >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f57b40d1c20 0x7f57b40cbf70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:29:13.718 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:13.715+0000 7f57ba851640 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f57b40cc4b0 0x7f57b406ecf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:29:13.718 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:13.715+0000 7f57ba851640 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 -- 0x7f57b40b8fd0 con 0x7f57b40b71b0 2024-04-25T10:29:13.718 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:13.715+0000 7f57ba851640 1 -- --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_getmap magic: 0 -- 0x7f57b40cf7a0 con 0x7f57b40cc4b0 2024-04-25T10:29:13.719 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:13.715+0000 7f57ba851640 1 -- --> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] -- mon_getmap magic: 0 -- 0x7f57b40b93b0 con 0x7f57b40d1c20 2024-04-25T10:29:13.719 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:13.715+0000 7f57b3fff640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f57b40b71b0 0x7f57b41dd700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:29:13.719 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:13.715+0000 7f57b3fff640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f57b40b71b0 0x7f57b41dd700 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.37:3300/0 says I am v2:172.21.15.37:46320/0 (socket says 172.21.15.37:46320) 2024-04-25T10:29:13.719 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:13.715+0000 7f57b3fff640 1 -- 172.21.15.37:0/1714612535 learned_addr learned my addr 172.21.15.37:0/1714612535 (peer_addr_for_me v2:172.21.15.37:0/0) 2024-04-25T10:29:13.719 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:13.715+0000 7f57b37fe640 1 --2- 172.21.15.37:0/1714612535 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f57b40d1c20 0x7f57b40cbf70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:29:13.719 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:13.715+0000 7f57b8dee640 1 --2- 172.21.15.37:0/1714612535 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f57b40cc4b0 0x7f57b406ecf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:29:13.719 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:13.715+0000 7f57b3fff640 1 -- 172.21.15.37:0/1714612535 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f57b40d1c20 msgr2=0x7f57b40cbf70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:29:13.719 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:13.715+0000 7f57b3fff640 1 --2- 172.21.15.37:0/1714612535 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f57b40d1c20 0x7f57b40cbf70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:29:13.719 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:13.715+0000 7f57b3fff640 1 -- 172.21.15.37:0/1714612535 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f57b40cc4b0 msgr2=0x7f57b406ecf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:29:13.719 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:13.715+0000 7f57b3fff640 1 --2- 172.21.15.37:0/1714612535 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f57b40cc4b0 0x7f57b406ecf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:29:13.719 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:13.715+0000 7f57b3fff640 1 -- 172.21.15.37:0/1714612535 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f57b41f6310 con 0x7f57b40b71b0 2024-04-25T10:29:13.719 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:13.715+0000 7f57b37fe640 1 --2- 172.21.15.37:0/1714612535 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f57b40d1c20 0x7f57b40cbf70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T10:29:13.719 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:13.715+0000 7f57b8dee640 1 --2- 172.21.15.37:0/1714612535 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f57b40cc4b0 0x7f57b406ecf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T10:29:13.719 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:13.716+0000 7f57b3fff640 1 --2- 172.21.15.37:0/1714612535 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f57b40b71b0 0x7f57b41dd700 secure :-1 s=READY pgs=84 cs=0 l=1 rev1=1 crypto rx=0x7f57a800c500 tx=0x7f57a800c9c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:29:13.720 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:13.716+0000 7f57b0ff9640 1 -- 172.21.15.37:0/1714612535 <== mon.0 v2:172.21.15.37:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f57a8017070 con 0x7f57b40b71b0 2024-04-25T10:29:13.720 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:13.716+0000 7f57b0ff9640 1 -- 172.21.15.37:0/1714612535 <== mon.0 v2:172.21.15.37:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f57a8005ce0 con 0x7f57b40b71b0 2024-04-25T10:29:13.720 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:13.716+0000 7f57ba851640 1 -- 172.21.15.37:0/1714612535 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f57b41f65f0 con 0x7f57b40b71b0 2024-04-25T10:29:13.720 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:13.716+0000 7f57b0ff9640 1 -- 172.21.15.37:0/1714612535 <== mon.0 v2:172.21.15.37:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f57a8007310 con 0x7f57b40b71b0 2024-04-25T10:29:13.720 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:13.716+0000 7f57ba851640 1 -- 172.21.15.37:0/1714612535 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f57b41f6bb0 con 0x7f57b40b71b0 2024-04-25T10:29:13.722 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:13.717+0000 7f57ba851640 1 -- 172.21.15.37:0/1714612535 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f57b40cc970 con 0x7f57b40b71b0 2024-04-25T10:29:13.723 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:13.717+0000 7f57b0ff9640 1 -- 172.21.15.37:0/1714612535 <== mon.0 v2:172.21.15.37:3300/0 4 ==== mgrmap(e 5) ==== 96782+0+0 (secure 0 0 0) 0x7f57a801a070 con 0x7f57b40b71b0 2024-04-25T10:29:13.723 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:13.718+0000 7f57b0ff9640 1 --2- 172.21.15.37:0/1714612535 >> [v2:172.21.15.203:6832/2418205237,v1:172.21.15.203:6833/2418205237] conn(0x7f57a00744b0 0x7f57a00769c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:29:13.723 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:13.718+0000 7f57b0ff9640 1 -- 172.21.15.37:0/1714612535 <== mon.0 v2:172.21.15.37:3300/0 5 ==== osd_map(23..23 src has 1..23) ==== 5709+0+0 (secure 0 0 0) 0x7f57a8094560 con 0x7f57b40b71b0 2024-04-25T10:29:13.723 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:13.718+0000 7f57b37fe640 1 --2- 172.21.15.37:0/1714612535 >> [v2:172.21.15.203:6832/2418205237,v1:172.21.15.203:6833/2418205237] conn(0x7f57a00744b0 0x7f57a00769c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:29:13.723 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:13.719+0000 7f57b37fe640 1 --2- 172.21.15.37:0/1714612535 >> [v2:172.21.15.203:6832/2418205237,v1:172.21.15.203:6833/2418205237] conn(0x7f57a00744b0 0x7f57a00769c0 secure :-1 s=READY pgs=66 cs=0 l=1 rev1=1 crypto rx=0x7f57a4034000 tx=0x7f57a402e000 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:29:13.724 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:13.722+0000 7f57b0ff9640 1 -- 172.21.15.37:0/1714612535 <== mon.0 v2:172.21.15.37:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187246 (secure 0 0 0) 0x7f57b40cc970 con 0x7f57b40b71b0 2024-04-25T10:29:13.907 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:13.904+0000 7f57ba851640 1 -- 172.21.15.37:0/1714612535 --> [v2:172.21.15.203:6832/2418205237,v1:172.21.15.203:6833/2418205237] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) -- 0x7f57b406a140 con 0x7f57a00744b0 2024-04-25T10:29:13.913 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:13.912+0000 7f57b0ff9640 1 -- 172.21.15.37:0/1714612535 <== mgr.4100 v2:172.21.15.203:6832/2418205237 1 ==== mgr_command_reply(tid 0: 0 dumped all) ==== 18+0+251335 (secure 0 0 0) 0x7f57b406a140 con 0x7f57a00744b0 2024-04-25T10:29:13.916 INFO:teuthology.orchestra.run.smithi037.stdout: 2024-04-25T10:29:13.918 INFO:teuthology.orchestra.run.smithi037.stderr:dumped all 2024-04-25T10:29:13.919 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:13.915+0000 7f57ba851640 1 -- 172.21.15.37:0/1714612535 >> [v2:172.21.15.203:6832/2418205237,v1:172.21.15.203:6833/2418205237] conn(0x7f57a00744b0 msgr2=0x7f57a00769c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:29:13.919 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:13.915+0000 7f57ba851640 1 --2- 172.21.15.37:0/1714612535 >> [v2:172.21.15.203:6832/2418205237,v1:172.21.15.203:6833/2418205237] conn(0x7f57a00744b0 0x7f57a00769c0 secure :-1 s=READY pgs=66 cs=0 l=1 rev1=1 crypto rx=0x7f57a4034000 tx=0x7f57a402e000 comp rx=0 tx=0).stop 2024-04-25T10:29:13.919 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:13.915+0000 7f57ba851640 1 -- 172.21.15.37:0/1714612535 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f57b40b71b0 msgr2=0x7f57b41dd700 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:29:13.920 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:13.915+0000 7f57ba851640 1 --2- 172.21.15.37:0/1714612535 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f57b40b71b0 0x7f57b41dd700 secure :-1 s=READY pgs=84 cs=0 l=1 rev1=1 crypto rx=0x7f57a800c500 tx=0x7f57a800c9c0 comp rx=0 tx=0).stop 2024-04-25T10:29:13.920 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:13.915+0000 7f57ba851640 1 -- 172.21.15.37:0/1714612535 shutdown_connections 2024-04-25T10:29:13.920 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:13.915+0000 7f57ba851640 1 --2- 172.21.15.37:0/1714612535 >> [v2:172.21.15.203:6832/2418205237,v1:172.21.15.203:6833/2418205237] conn(0x7f57a00744b0 0x7f57a00769c0 unknown :-1 s=CLOSED pgs=66 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:29:13.920 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:13.915+0000 7f57ba851640 1 --2- 172.21.15.37:0/1714612535 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f57b40cc4b0 0x7f57b406ecf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:29:13.920 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:13.915+0000 7f57ba851640 1 --2- 172.21.15.37:0/1714612535 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f57b40d1c20 0x7f57b40cbf70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:29:13.920 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:13.915+0000 7f57ba851640 1 --2- 172.21.15.37:0/1714612535 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f57b40b71b0 0x7f57b41dd700 unknown :-1 s=CLOSED pgs=84 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:29:13.920 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:13.915+0000 7f57ba851640 1 -- 172.21.15.37:0/1714612535 >> 172.21.15.37:0/1714612535 conn(0x7f57b40d03d0 msgr2=0x7f57b409bb20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:29:13.920 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:13.915+0000 7f57ba851640 1 -- 172.21.15.37:0/1714612535 shutdown_connections 2024-04-25T10:29:13.920 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:13.915+0000 7f57ba851640 1 -- 172.21.15.37:0/1714612535 wait complete. 2024-04-25T10:29:13.928 INFO:teuthology.orchestra.run.smithi037.stdout:{"pg_ready":true,"pg_map":{"version":26,"stamp":"2024-04-25T10:29:12.186439+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":592196,"num_objects":16,"num_object_clones":0,"num_object_copies":32,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":16,"num_whiteouts":0,"num_read":66,"num_read_kb":55,"num_write":126,"num_write_kb":1641,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":7,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":96,"ondisk_log_size":96,"up":194,"acting":194,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":52,"num_osds":8,"num_per_pool_osds":8,"num_per_pool_omap_osds":8,"kb":754974720,"kb_used":219068,"kb_used_data":2908,"kb_used_omap":12,"kb_used_meta":216051,"kb_avail":754755652,"statfs":{"total":773094113280,"available":772869787648,"internally_reserved":0,"allocated":2977792,"data_stored":1769702,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":12752,"internal_metadata":221236784},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":30,"apply_latency_ms":30,"commit_latency_ns":30000000,"apply_latency_ns":30000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":1828,"num_objects":14,"num_object_clones":0,"num_object_copies":28,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":14,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":25,"num_write_kb":13,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":7,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"5.190009"},"pg_stats":[{"pgid":"3.3f","version":"0'0","reported_seq":12,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-04-25T10:29:10.037967+0000","last_change":"2024-04-25T10:29:09.024146+0000","last_active":"2024-04-25T10:29:10.037967+0000","last_peered":"2024-04-25T10:29:10.037967+0000","last_clean":"2024-04-25T10:29:10.037967+0000","last_became_active":"2024-04-25T10:29:09.023798+0000","last_became_peered":"2024-04-25T10:29:09.023798+0000","last_unstale":"2024-04-25T10:29:10.037967+0000","last_undegraded":"2024-04-25T10:29:10.037967+0000","last_fullsized":"2024-04-25T10:29:10.037967+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:29:07.995096+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:29:07.995096+0000","last_clean_scrub_stamp":"2024-04-25T10:29:07.995096+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T20:37:20.726766+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.3e","version":"0'0","reported_seq":12,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-04-25T10:29:10.189579+0000","last_change":"2024-04-25T10:29:09.028600+0000","last_active":"2024-04-25T10:29:10.189579+0000","last_peered":"2024-04-25T10:29:10.189579+0000","last_clean":"2024-04-25T10:29:10.189579+0000","last_became_active":"2024-04-25T10:29:09.028286+0000","last_became_peered":"2024-04-25T10:29:09.028286+0000","last_unstale":"2024-04-25T10:29:10.189579+0000","last_undegraded":"2024-04-25T10:29:10.189579+0000","last_fullsized":"2024-04-25T10:29:10.189579+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:29:07.995096+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:29:07.995096+0000","last_clean_scrub_stamp":"2024-04-25T10:29:07.995096+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T20:52:02.098872+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6],"acting":[5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.3d","version":"0'0","reported_seq":12,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-04-25T10:29:10.189318+0000","last_change":"2024-04-25T10:29:09.039030+0000","last_active":"2024-04-25T10:29:10.189318+0000","last_peered":"2024-04-25T10:29:10.189318+0000","last_clean":"2024-04-25T10:29:10.189318+0000","last_became_active":"2024-04-25T10:29:09.037692+0000","last_became_peered":"2024-04-25T10:29:09.037692+0000","last_unstale":"2024-04-25T10:29:10.189318+0000","last_undegraded":"2024-04-25T10:29:10.189318+0000","last_fullsized":"2024-04-25T10:29:10.189318+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:29:07.995096+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:29:07.995096+0000","last_clean_scrub_stamp":"2024-04-25T10:29:07.995096+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T15:51:52.989545+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.3c","version":"0'0","reported_seq":12,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-04-25T10:29:10.026169+0000","last_change":"2024-04-25T10:29:09.027520+0000","last_active":"2024-04-25T10:29:10.026169+0000","last_peered":"2024-04-25T10:29:10.026169+0000","last_clean":"2024-04-25T10:29:10.026169+0000","last_became_active":"2024-04-25T10:29:09.027075+0000","last_became_peered":"2024-04-25T10:29:09.027075+0000","last_unstale":"2024-04-25T10:29:10.026169+0000","last_undegraded":"2024-04-25T10:29:10.026169+0000","last_fullsized":"2024-04-25T10:29:10.026169+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:29:07.995096+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:29:07.995096+0000","last_clean_scrub_stamp":"2024-04-25T10:29:07.995096+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T10:55:01.505157+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4],"acting":[2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.3b","version":"0'0","reported_seq":12,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-04-25T10:29:10.037590+0000","last_change":"2024-04-25T10:29:09.040684+0000","last_active":"2024-04-25T10:29:10.037590+0000","last_peered":"2024-04-25T10:29:10.037590+0000","last_clean":"2024-04-25T10:29:10.037590+0000","last_became_active":"2024-04-25T10:29:09.040372+0000","last_became_peered":"2024-04-25T10:29:09.040372+0000","last_unstale":"2024-04-25T10:29:10.037590+0000","last_undegraded":"2024-04-25T10:29:10.037590+0000","last_fullsized":"2024-04-25T10:29:10.037590+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:29:07.995096+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:29:07.995096+0000","last_clean_scrub_stamp":"2024-04-25T10:29:07.995096+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T11:52:24.415332+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.3a","version":"0'0","reported_seq":12,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-04-25T10:29:10.025911+0000","last_change":"2024-04-25T10:29:09.027391+0000","last_active":"2024-04-25T10:29:10.025911+0000","last_peered":"2024-04-25T10:29:10.025911+0000","last_clean":"2024-04-25T10:29:10.025911+0000","last_became_active":"2024-04-25T10:29:09.027029+0000","last_became_peered":"2024-04-25T10:29:09.027029+0000","last_unstale":"2024-04-25T10:29:10.025911+0000","last_undegraded":"2024-04-25T10:29:10.025911+0000","last_fullsized":"2024-04-25T10:29:10.025911+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:29:07.995096+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:29:07.995096+0000","last_clean_scrub_stamp":"2024-04-25T10:29:07.995096+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T10:35:31.361514+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4],"acting":[2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.39","version":"0'0","reported_seq":12,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-04-25T10:29:10.037462+0000","last_change":"2024-04-25T10:29:09.029272+0000","last_active":"2024-04-25T10:29:10.037462+0000","last_peered":"2024-04-25T10:29:10.037462+0000","last_clean":"2024-04-25T10:29:10.037462+0000","last_became_active":"2024-04-25T10:29:09.028302+0000","last_became_peered":"2024-04-25T10:29:09.028302+0000","last_unstale":"2024-04-25T10:29:10.037462+0000","last_undegraded":"2024-04-25T10:29:10.037462+0000","last_fullsized":"2024-04-25T10:29:10.037462+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:29:07.995096+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:29:07.995096+0000","last_clean_scrub_stamp":"2024-04-25T10:29:07.995096+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T11:31:45.283247+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.38","version":"0'0","reported_seq":12,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-04-25T10:29:10.188973+0000","last_change":"2024-04-25T10:29:09.035089+0000","last_active":"2024-04-25T10:29:10.188973+0000","last_peered":"2024-04-25T10:29:10.188973+0000","last_clean":"2024-04-25T10:29:10.188973+0000","last_became_active":"2024-04-25T10:29:09.034324+0000","last_became_peered":"2024-04-25T10:29:09.034324+0000","last_unstale":"2024-04-25T10:29:10.188973+0000","last_undegraded":"2024-04-25T10:29:10.188973+0000","last_fullsized":"2024-04-25T10:29:10.188973+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:29:07.995096+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:29:07.995096+0000","last_clean_scrub_stamp":"2024-04-25T10:29:07.995096+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T21:39:39.743441+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.37","version":"0'0","reported_seq":12,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-04-25T10:29:10.187981+0000","last_change":"2024-04-25T10:29:09.016966+0000","last_active":"2024-04-25T10:29:10.187981+0000","last_peered":"2024-04-25T10:29:10.187981+0000","last_clean":"2024-04-25T10:29:10.187981+0000","last_became_active":"2024-04-25T10:29:09.016566+0000","last_became_peered":"2024-04-25T10:29:09.016566+0000","last_unstale":"2024-04-25T10:29:10.187981+0000","last_undegraded":"2024-04-25T10:29:10.187981+0000","last_fullsized":"2024-04-25T10:29:10.187981+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:29:07.995096+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:29:07.995096+0000","last_clean_scrub_stamp":"2024-04-25T10:29:07.995096+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T10:55:01.505157+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.36","version":"0'0","reported_seq":12,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-04-25T10:29:10.037810+0000","last_change":"2024-04-25T10:29:09.030321+0000","last_active":"2024-04-25T10:29:10.037810+0000","last_peered":"2024-04-25T10:29:10.037810+0000","last_clean":"2024-04-25T10:29:10.037810+0000","last_became_active":"2024-04-25T10:29:09.029976+0000","last_became_peered":"2024-04-25T10:29:09.029976+0000","last_unstale":"2024-04-25T10:29:10.037810+0000","last_undegraded":"2024-04-25T10:29:10.037810+0000","last_fullsized":"2024-04-25T10:29:10.037810+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:29:07.995096+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:29:07.995096+0000","last_clean_scrub_stamp":"2024-04-25T10:29:07.995096+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T12:36:31.191100+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.35","version":"0'0","reported_seq":12,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-04-25T10:29:10.037641+0000","last_change":"2024-04-25T10:29:09.030469+0000","last_active":"2024-04-25T10:29:10.037641+0000","last_peered":"2024-04-25T10:29:10.037641+0000","last_clean":"2024-04-25T10:29:10.037641+0000","last_became_active":"2024-04-25T10:29:09.029350+0000","last_became_peered":"2024-04-25T10:29:09.029350+0000","last_unstale":"2024-04-25T10:29:10.037641+0000","last_undegraded":"2024-04-25T10:29:10.037641+0000","last_fullsized":"2024-04-25T10:29:10.037641+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:29:07.995096+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:29:07.995096+0000","last_clean_scrub_stamp":"2024-04-25T10:29:07.995096+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T15:51:52.989545+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.34","version":"0'0","reported_seq":12,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-04-25T10:29:10.037728+0000","last_change":"2024-04-25T10:29:09.029772+0000","last_active":"2024-04-25T10:29:10.037728+0000","last_peered":"2024-04-25T10:29:10.037728+0000","last_clean":"2024-04-25T10:29:10.037728+0000","last_became_active":"2024-04-25T10:29:09.029350+0000","last_became_peered":"2024-04-25T10:29:09.029350+0000","last_unstale":"2024-04-25T10:29:10.037728+0000","last_undegraded":"2024-04-25T10:29:10.037728+0000","last_fullsized":"2024-04-25T10:29:10.037728+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:29:07.995096+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:29:07.995096+0000","last_clean_scrub_stamp":"2024-04-25T10:29:07.995096+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T19:14:48.737269+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.33","version":"0'0","reported_seq":12,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-04-25T10:29:10.188971+0000","last_change":"2024-04-25T10:29:09.038690+0000","last_active":"2024-04-25T10:29:10.188971+0000","last_peered":"2024-04-25T10:29:10.188971+0000","last_clean":"2024-04-25T10:29:10.188971+0000","last_became_active":"2024-04-25T10:29:09.038336+0000","last_became_peered":"2024-04-25T10:29:09.038336+0000","last_unstale":"2024-04-25T10:29:10.188971+0000","last_undegraded":"2024-04-25T10:29:10.188971+0000","last_fullsized":"2024-04-25T10:29:10.188971+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:29:07.995096+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:29:07.995096+0000","last_clean_scrub_stamp":"2024-04-25T10:29:07.995096+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T19:23:12.589414+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.32","version":"0'0","reported_seq":12,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-04-25T10:29:10.188641+0000","last_change":"2024-04-25T10:29:09.028195+0000","last_active":"2024-04-25T10:29:10.188641+0000","last_peered":"2024-04-25T10:29:10.188641+0000","last_clean":"2024-04-25T10:29:10.188641+0000","last_became_active":"2024-04-25T10:29:09.027893+0000","last_became_peered":"2024-04-25T10:29:09.027893+0000","last_unstale":"2024-04-25T10:29:10.188641+0000","last_undegraded":"2024-04-25T10:29:10.188641+0000","last_fullsized":"2024-04-25T10:29:10.188641+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:29:07.995096+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:29:07.995096+0000","last_clean_scrub_stamp":"2024-04-25T10:29:07.995096+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T11:35:27.466874+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.31","version":"0'0","reported_seq":12,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-04-25T10:29:10.188956+0000","last_change":"2024-04-25T10:29:09.034532+0000","last_active":"2024-04-25T10:29:10.188956+0000","last_peered":"2024-04-25T10:29:10.188956+0000","last_clean":"2024-04-25T10:29:10.188956+0000","last_became_active":"2024-04-25T10:29:09.034021+0000","last_became_peered":"2024-04-25T10:29:09.034021+0000","last_unstale":"2024-04-25T10:29:10.188956+0000","last_undegraded":"2024-04-25T10:29:10.188956+0000","last_fullsized":"2024-04-25T10:29:10.188956+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:29:07.995096+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:29:07.995096+0000","last_clean_scrub_stamp":"2024-04-25T10:29:07.995096+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T12:16:04.880732+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.30","version":"0'0","reported_seq":12,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-04-25T10:29:10.024134+0000","last_change":"2024-04-25T10:29:09.021224+0000","last_active":"2024-04-25T10:29:10.024134+0000","last_peered":"2024-04-25T10:29:10.024134+0000","last_clean":"2024-04-25T10:29:10.024134+0000","last_became_active":"2024-04-25T10:29:09.020964+0000","last_became_peered":"2024-04-25T10:29:09.020964+0000","last_unstale":"2024-04-25T10:29:10.024134+0000","last_undegraded":"2024-04-25T10:29:10.024134+0000","last_fullsized":"2024-04-25T10:29:10.024134+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:29:07.995096+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:29:07.995096+0000","last_clean_scrub_stamp":"2024-04-25T10:29:07.995096+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T18:44:23.610374+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.2f","version":"0'0","reported_seq":12,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-04-25T10:29:10.024944+0000","last_change":"2024-04-25T10:29:09.025638+0000","last_active":"2024-04-25T10:29:10.024944+0000","last_peered":"2024-04-25T10:29:10.024944+0000","last_clean":"2024-04-25T10:29:10.024944+0000","last_became_active":"2024-04-25T10:29:09.024517+0000","last_became_peered":"2024-04-25T10:29:09.024517+0000","last_unstale":"2024-04-25T10:29:10.024944+0000","last_undegraded":"2024-04-25T10:29:10.024944+0000","last_fullsized":"2024-04-25T10:29:10.024944+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:29:07.995096+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:29:07.995096+0000","last_clean_scrub_stamp":"2024-04-25T10:29:07.995096+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T16:29:08.975346+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.2e","version":"0'0","reported_seq":12,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-04-25T10:29:10.189006+0000","last_change":"2024-04-25T10:29:09.023923+0000","last_active":"2024-04-25T10:29:10.189006+0000","last_peered":"2024-04-25T10:29:10.189006+0000","last_clean":"2024-04-25T10:29:10.189006+0000","last_became_active":"2024-04-25T10:29:09.023579+0000","last_became_peered":"2024-04-25T10:29:09.023579+0000","last_unstale":"2024-04-25T10:29:10.189006+0000","last_undegraded":"2024-04-25T10:29:10.189006+0000","last_fullsized":"2024-04-25T10:29:10.189006+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:29:07.995096+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:29:07.995096+0000","last_clean_scrub_stamp":"2024-04-25T10:29:07.995096+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T10:35:31.361514+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.2d","version":"0'0","reported_seq":12,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-04-25T10:29:10.024852+0000","last_change":"2024-04-25T10:29:09.023039+0000","last_active":"2024-04-25T10:29:10.024852+0000","last_peered":"2024-04-25T10:29:10.024852+0000","last_clean":"2024-04-25T10:29:10.024852+0000","last_became_active":"2024-04-25T10:29:09.022706+0000","last_became_peered":"2024-04-25T10:29:09.022706+0000","last_unstale":"2024-04-25T10:29:10.024852+0000","last_undegraded":"2024-04-25T10:29:10.024852+0000","last_fullsized":"2024-04-25T10:29:10.024852+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:29:07.995096+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:29:07.995096+0000","last_clean_scrub_stamp":"2024-04-25T10:29:07.995096+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T16:48:17.352944+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.2c","version":"0'0","reported_seq":12,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-04-25T10:29:10.037891+0000","last_change":"2024-04-25T10:29:09.028979+0000","last_active":"2024-04-25T10:29:10.037891+0000","last_peered":"2024-04-25T10:29:10.037891+0000","last_clean":"2024-04-25T10:29:10.037891+0000","last_became_active":"2024-04-25T10:29:09.028061+0000","last_became_peered":"2024-04-25T10:29:09.028061+0000","last_unstale":"2024-04-25T10:29:10.037891+0000","last_undegraded":"2024-04-25T10:29:10.037891+0000","last_fullsized":"2024-04-25T10:29:10.037891+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:29:07.995096+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:29:07.995096+0000","last_clean_scrub_stamp":"2024-04-25T10:29:07.995096+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T18:44:23.610374+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.2b","version":"0'0","reported_seq":1,"reported_epoch":21,"state":"creating+peering","last_fresh":"2024-04-25T10:29:08.012124+0000","last_change":"2024-04-25T10:29:08.012124+0000","last_active":"2024-04-25T10:29:07.995096+0000","last_peered":"2024-04-25T10:29:07.995096+0000","last_clean":"2024-04-25T10:29:07.995096+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T10:29:08.012124+0000","last_undegraded":"2024-04-25T10:29:08.012124+0000","last_fullsized":"2024-04-25T10:29:08.012124+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:29:07.995096+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:29:07.995096+0000","last_clean_scrub_stamp":"2024-04-25T10:29:07.995096+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.2a","version":"0'0","reported_seq":12,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-04-25T10:29:10.187695+0000","last_change":"2024-04-25T10:29:09.029965+0000","last_active":"2024-04-25T10:29:10.187695+0000","last_peered":"2024-04-25T10:29:10.187695+0000","last_clean":"2024-04-25T10:29:10.187695+0000","last_became_active":"2024-04-25T10:29:09.029616+0000","last_became_peered":"2024-04-25T10:29:09.029616+0000","last_unstale":"2024-04-25T10:29:10.187695+0000","last_undegraded":"2024-04-25T10:29:10.187695+0000","last_fullsized":"2024-04-25T10:29:10.187695+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:29:07.995096+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:29:07.995096+0000","last_clean_scrub_stamp":"2024-04-25T10:29:07.995096+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T15:58:36.848831+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.29","version":"0'0","reported_seq":12,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-04-25T10:29:10.189345+0000","last_change":"2024-04-25T10:29:09.033401+0000","last_active":"2024-04-25T10:29:10.189345+0000","last_peered":"2024-04-25T10:29:10.189345+0000","last_clean":"2024-04-25T10:29:10.189345+0000","last_became_active":"2024-04-25T10:29:09.032764+0000","last_became_peered":"2024-04-25T10:29:09.032764+0000","last_unstale":"2024-04-25T10:29:10.189345+0000","last_undegraded":"2024-04-25T10:29:10.189345+0000","last_fullsized":"2024-04-25T10:29:10.189345+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:29:07.995096+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:29:07.995096+0000","last_clean_scrub_stamp":"2024-04-25T10:29:07.995096+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T17:30:54.093151+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.28","version":"0'0","reported_seq":12,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-04-25T10:29:10.024133+0000","last_change":"2024-04-25T10:29:09.041063+0000","last_active":"2024-04-25T10:29:10.024133+0000","last_peered":"2024-04-25T10:29:10.024133+0000","last_clean":"2024-04-25T10:29:10.024133+0000","last_became_active":"2024-04-25T10:29:09.040734+0000","last_became_peered":"2024-04-25T10:29:09.040734+0000","last_unstale":"2024-04-25T10:29:10.024133+0000","last_undegraded":"2024-04-25T10:29:10.024133+0000","last_fullsized":"2024-04-25T10:29:10.024133+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:29:07.995096+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:29:07.995096+0000","last_clean_scrub_stamp":"2024-04-25T10:29:07.995096+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T13:54:22.968216+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.27","version":"0'0","reported_seq":12,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-04-25T10:29:10.189019+0000","last_change":"2024-04-25T10:29:09.034268+0000","last_active":"2024-04-25T10:29:10.189019+0000","last_peered":"2024-04-25T10:29:10.189019+0000","last_clean":"2024-04-25T10:29:10.189019+0000","last_became_active":"2024-04-25T10:29:09.033199+0000","last_became_peered":"2024-04-25T10:29:09.033199+0000","last_unstale":"2024-04-25T10:29:10.189019+0000","last_undegraded":"2024-04-25T10:29:10.189019+0000","last_fullsized":"2024-04-25T10:29:10.189019+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:29:07.995096+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:29:07.995096+0000","last_clean_scrub_stamp":"2024-04-25T10:29:07.995096+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T17:56:09.999494+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.26","version":"0'0","reported_seq":12,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-04-25T10:29:10.026183+0000","last_change":"2024-04-25T10:29:09.025643+0000","last_active":"2024-04-25T10:29:10.026183+0000","last_peered":"2024-04-25T10:29:10.026183+0000","last_clean":"2024-04-25T10:29:10.026183+0000","last_became_active":"2024-04-25T10:29:09.024894+0000","last_became_peered":"2024-04-25T10:29:09.024894+0000","last_unstale":"2024-04-25T10:29:10.026183+0000","last_undegraded":"2024-04-25T10:29:10.026183+0000","last_fullsized":"2024-04-25T10:29:10.026183+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:29:07.995096+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:29:07.995096+0000","last_clean_scrub_stamp":"2024-04-25T10:29:07.995096+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T22:08:00.367431+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.25","version":"0'0","reported_seq":12,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-04-25T10:29:10.189130+0000","last_change":"2024-04-25T10:29:09.023357+0000","last_active":"2024-04-25T10:29:10.189130+0000","last_peered":"2024-04-25T10:29:10.189130+0000","last_clean":"2024-04-25T10:29:10.189130+0000","last_became_active":"2024-04-25T10:29:09.022988+0000","last_became_peered":"2024-04-25T10:29:09.022988+0000","last_unstale":"2024-04-25T10:29:10.189130+0000","last_undegraded":"2024-04-25T10:29:10.189130+0000","last_fullsized":"2024-04-25T10:29:10.189130+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:29:07.995096+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:29:07.995096+0000","last_clean_scrub_stamp":"2024-04-25T10:29:07.995096+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T21:14:39.661957+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.24","version":"0'0","reported_seq":12,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-04-25T10:29:10.189239+0000","last_change":"2024-04-25T10:29:09.034328+0000","last_active":"2024-04-25T10:29:10.189239+0000","last_peered":"2024-04-25T10:29:10.189239+0000","last_clean":"2024-04-25T10:29:10.189239+0000","last_became_active":"2024-04-25T10:29:09.033978+0000","last_became_peered":"2024-04-25T10:29:09.033978+0000","last_unstale":"2024-04-25T10:29:10.189239+0000","last_undegraded":"2024-04-25T10:29:10.189239+0000","last_fullsized":"2024-04-25T10:29:10.189239+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:29:07.995096+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:29:07.995096+0000","last_clean_scrub_stamp":"2024-04-25T10:29:07.995096+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T21:24:14.497242+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.23","version":"0'0","reported_seq":12,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-04-25T10:29:10.189583+0000","last_change":"2024-04-25T10:29:09.034045+0000","last_active":"2024-04-25T10:29:10.189583+0000","last_peered":"2024-04-25T10:29:10.189583+0000","last_clean":"2024-04-25T10:29:10.189583+0000","last_became_active":"2024-04-25T10:29:09.033730+0000","last_became_peered":"2024-04-25T10:29:09.033730+0000","last_unstale":"2024-04-25T10:29:10.189583+0000","last_undegraded":"2024-04-25T10:29:10.189583+0000","last_fullsized":"2024-04-25T10:29:10.189583+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:29:07.995096+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:29:07.995096+0000","last_clean_scrub_stamp":"2024-04-25T10:29:07.995096+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T12:36:31.191100+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.22","version":"0'0","reported_seq":12,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-04-25T10:29:10.024663+0000","last_change":"2024-04-25T10:29:09.034262+0000","last_active":"2024-04-25T10:29:10.024663+0000","last_peered":"2024-04-25T10:29:10.024663+0000","last_clean":"2024-04-25T10:29:10.024663+0000","last_became_active":"2024-04-25T10:29:09.033905+0000","last_became_peered":"2024-04-25T10:29:09.033905+0000","last_unstale":"2024-04-25T10:29:10.024663+0000","last_undegraded":"2024-04-25T10:29:10.024663+0000","last_fullsized":"2024-04-25T10:29:10.024663+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:29:07.995096+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:29:07.995096+0000","last_clean_scrub_stamp":"2024-04-25T10:29:07.995096+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T16:22:45.608928+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.21","version":"0'0","reported_seq":12,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-04-25T10:29:10.188201+0000","last_change":"2024-04-25T10:29:09.027088+0000","last_active":"2024-04-25T10:29:10.188201+0000","last_peered":"2024-04-25T10:29:10.188201+0000","last_clean":"2024-04-25T10:29:10.188201+0000","last_became_active":"2024-04-25T10:29:09.026651+0000","last_became_peered":"2024-04-25T10:29:09.026651+0000","last_unstale":"2024-04-25T10:29:10.188201+0000","last_undegraded":"2024-04-25T10:29:10.188201+0000","last_fullsized":"2024-04-25T10:29:10.188201+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:29:07.995096+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:29:07.995096+0000","last_clean_scrub_stamp":"2024-04-25T10:29:07.995096+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T15:37:06.723380+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.20","version":"0'0","reported_seq":12,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-04-25T10:29:10.025741+0000","last_change":"2024-04-25T10:29:09.028008+0000","last_active":"2024-04-25T10:29:10.025741+0000","last_peered":"2024-04-25T10:29:10.025741+0000","last_clean":"2024-04-25T10:29:10.025741+0000","last_became_active":"2024-04-25T10:29:09.027679+0000","last_became_peered":"2024-04-25T10:29:09.027679+0000","last_unstale":"2024-04-25T10:29:10.025741+0000","last_undegraded":"2024-04-25T10:29:10.025741+0000","last_fullsized":"2024-04-25T10:29:10.025741+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:29:07.995096+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:29:07.995096+0000","last_clean_scrub_stamp":"2024-04-25T10:29:07.995096+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T15:37:06.723380+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,6],"acting":[2,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.1a","version":"0'0","reported_seq":12,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-04-25T10:29:10.187787+0000","last_change":"2024-04-25T10:29:09.032819+0000","last_active":"2024-04-25T10:29:10.187787+0000","last_peered":"2024-04-25T10:29:10.187787+0000","last_clean":"2024-04-25T10:29:10.187787+0000","last_became_active":"2024-04-25T10:29:09.032493+0000","last_became_peered":"2024-04-25T10:29:09.032493+0000","last_unstale":"2024-04-25T10:29:10.187787+0000","last_undegraded":"2024-04-25T10:29:10.187787+0000","last_fullsized":"2024-04-25T10:29:10.187787+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:29:07.995096+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:29:07.995096+0000","last_clean_scrub_stamp":"2024-04-25T10:29:07.995096+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T15:37:06.723380+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.1b","version":"22'1","reported_seq":15,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-04-25T10:29:10.189463+0000","last_change":"2024-04-25T10:29:08.025272+0000","last_active":"2024-04-25T10:29:10.189463+0000","last_peered":"2024-04-25T10:29:10.189463+0000","last_clean":"2024-04-25T10:29:10.189463+0000","last_became_active":"2024-04-25T10:29:08.024912+0000","last_became_peered":"2024-04-25T10:29:08.024912+0000","last_unstale":"2024-04-25T10:29:10.189463+0000","last_undegraded":"2024-04-25T10:29:10.189463+0000","last_fullsized":"2024-04-25T10:29:10.189463+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:29:06.989570+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:29:06.989570+0000","last_clean_scrub_stamp":"2024-04-25T10:29:06.989570+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T18:11:32.758179+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6],"acting":[5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.1b","version":"0'0","reported_seq":1,"reported_epoch":21,"state":"creating+peering","last_fresh":"2024-04-25T10:29:08.008604+0000","last_change":"2024-04-25T10:29:08.008604+0000","last_active":"2024-04-25T10:29:07.995096+0000","last_peered":"2024-04-25T10:29:07.995096+0000","last_clean":"2024-04-25T10:29:07.995096+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T10:29:08.008604+0000","last_undegraded":"2024-04-25T10:29:08.008604+0000","last_fullsized":"2024-04-25T10:29:08.008604+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:29:07.995096+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:29:07.995096+0000","last_clean_scrub_stamp":"2024-04-25T10:29:07.995096+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.1a","version":"22'1","reported_seq":15,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-04-25T10:29:10.024506+0000","last_change":"2024-04-25T10:29:08.026018+0000","last_active":"2024-04-25T10:29:10.024506+0000","last_peered":"2024-04-25T10:29:10.024506+0000","last_clean":"2024-04-25T10:29:10.024506+0000","last_became_active":"2024-04-25T10:29:08.025668+0000","last_became_peered":"2024-04-25T10:29:08.025668+0000","last_unstale":"2024-04-25T10:29:10.024506+0000","last_undegraded":"2024-04-25T10:29:10.024506+0000","last_fullsized":"2024-04-25T10:29:10.024506+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:29:06.989570+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:29:06.989570+0000","last_clean_scrub_stamp":"2024-04-25T10:29:06.989570+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T11:35:26.461348+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.19","version":"0'0","reported_seq":14,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-04-25T10:29:10.188333+0000","last_change":"2024-04-25T10:29:08.011702+0000","last_active":"2024-04-25T10:29:10.188333+0000","last_peered":"2024-04-25T10:29:10.188333+0000","last_clean":"2024-04-25T10:29:10.188333+0000","last_became_active":"2024-04-25T10:29:08.011353+0000","last_became_peered":"2024-04-25T10:29:08.011353+0000","last_unstale":"2024-04-25T10:29:10.188333+0000","last_undegraded":"2024-04-25T10:29:10.188333+0000","last_fullsized":"2024-04-25T10:29:10.188333+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:29:06.989570+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:29:06.989570+0000","last_clean_scrub_stamp":"2024-04-25T10:29:06.989570+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T16:31:11.685101+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.18","version":"0'0","reported_seq":12,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-04-25T10:29:10.189830+0000","last_change":"2024-04-25T10:29:09.033872+0000","last_active":"2024-04-25T10:29:10.189830+0000","last_peered":"2024-04-25T10:29:10.189830+0000","last_clean":"2024-04-25T10:29:10.189830+0000","last_became_active":"2024-04-25T10:29:09.033544+0000","last_became_peered":"2024-04-25T10:29:09.033544+0000","last_unstale":"2024-04-25T10:29:10.189830+0000","last_undegraded":"2024-04-25T10:29:10.189830+0000","last_fullsized":"2024-04-25T10:29:10.189830+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:29:07.995096+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:29:07.995096+0000","last_clean_scrub_stamp":"2024-04-25T10:29:07.995096+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T13:54:22.968216+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6],"acting":[5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.18","version":"0'0","reported_seq":14,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-04-25T10:29:10.024423+0000","last_change":"2024-04-25T10:29:08.013340+0000","last_active":"2024-04-25T10:29:10.024423+0000","last_peered":"2024-04-25T10:29:10.024423+0000","last_clean":"2024-04-25T10:29:10.024423+0000","last_became_active":"2024-04-25T10:29:08.012940+0000","last_became_peered":"2024-04-25T10:29:08.012940+0000","last_unstale":"2024-04-25T10:29:10.024423+0000","last_undegraded":"2024-04-25T10:29:10.024423+0000","last_fullsized":"2024-04-25T10:29:10.024423+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:29:06.989570+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:29:06.989570+0000","last_clean_scrub_stamp":"2024-04-25T10:29:06.989570+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T10:35:30.355988+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.19","version":"0'0","reported_seq":1,"reported_epoch":21,"state":"creating+peering","last_fresh":"2024-04-25T10:29:08.012731+0000","last_change":"2024-04-25T10:29:08.012731+0000","last_active":"2024-04-25T10:29:07.995096+0000","last_peered":"2024-04-25T10:29:07.995096+0000","last_clean":"2024-04-25T10:29:07.995096+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T10:29:08.012731+0000","last_undegraded":"2024-04-25T10:29:08.012731+0000","last_fullsized":"2024-04-25T10:29:08.012731+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:29:07.995096+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:29:07.995096+0000","last_clean_scrub_stamp":"2024-04-25T10:29:07.995096+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.17","version":"0'0","reported_seq":14,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-04-25T10:29:10.187949+0000","last_change":"2024-04-25T10:29:08.020281+0000","last_active":"2024-04-25T10:29:10.187949+0000","last_peered":"2024-04-25T10:29:10.187949+0000","last_clean":"2024-04-25T10:29:10.187949+0000","last_became_active":"2024-04-25T10:29:08.019921+0000","last_became_peered":"2024-04-25T10:29:08.019921+0000","last_unstale":"2024-04-25T10:29:10.187949+0000","last_undegraded":"2024-04-25T10:29:10.187949+0000","last_fullsized":"2024-04-25T10:29:10.187949+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:29:06.989570+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:29:06.989570+0000","last_clean_scrub_stamp":"2024-04-25T10:29:06.989570+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T19:13:07.068780+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.16","version":"0'0","reported_seq":12,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-04-25T10:29:10.026197+0000","last_change":"2024-04-25T10:29:09.022744+0000","last_active":"2024-04-25T10:29:10.026197+0000","last_peered":"2024-04-25T10:29:10.026197+0000","last_clean":"2024-04-25T10:29:10.026197+0000","last_became_active":"2024-04-25T10:29:09.022392+0000","last_became_peered":"2024-04-25T10:29:09.022392+0000","last_unstale":"2024-04-25T10:29:10.026197+0000","last_undegraded":"2024-04-25T10:29:10.026197+0000","last_fullsized":"2024-04-25T10:29:10.026197+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:29:07.995096+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:29:07.995096+0000","last_clean_scrub_stamp":"2024-04-25T10:29:07.995096+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T11:35:27.466874+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.16","version":"0'0","reported_seq":14,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-04-25T10:29:10.037869+0000","last_change":"2024-04-25T10:29:08.026388+0000","last_active":"2024-04-25T10:29:10.037869+0000","last_peered":"2024-04-25T10:29:10.037869+0000","last_clean":"2024-04-25T10:29:10.037869+0000","last_became_active":"2024-04-25T10:29:08.026039+0000","last_became_peered":"2024-04-25T10:29:08.026039+0000","last_unstale":"2024-04-25T10:29:10.037869+0000","last_undegraded":"2024-04-25T10:29:10.037869+0000","last_fullsized":"2024-04-25T10:29:10.037869+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:29:06.989570+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:29:06.989570+0000","last_clean_scrub_stamp":"2024-04-25T10:29:06.989570+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T18:11:32.758179+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.17","version":"0'0","reported_seq":12,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-04-25T10:29:10.189465+0000","last_change":"2024-04-25T10:29:09.032902+0000","last_active":"2024-04-25T10:29:10.189465+0000","last_peered":"2024-04-25T10:29:10.189465+0000","last_clean":"2024-04-25T10:29:10.189465+0000","last_became_active":"2024-04-25T10:29:09.032596+0000","last_became_peered":"2024-04-25T10:29:09.032596+0000","last_unstale":"2024-04-25T10:29:10.189465+0000","last_undegraded":"2024-04-25T10:29:10.189465+0000","last_fullsized":"2024-04-25T10:29:10.189465+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:29:07.995096+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:29:07.995096+0000","last_clean_scrub_stamp":"2024-04-25T10:29:07.995096+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T19:14:48.737269+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.15","version":"22'3","reported_seq":17,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-04-25T10:29:10.188352+0000","last_change":"2024-04-25T10:29:08.021347+0000","last_active":"2024-04-25T10:29:10.188352+0000","last_peered":"2024-04-25T10:29:10.188352+0000","last_clean":"2024-04-25T10:29:10.188352+0000","last_became_active":"2024-04-25T10:29:08.021039+0000","last_became_peered":"2024-04-25T10:29:08.021039+0000","last_unstale":"2024-04-25T10:29:10.188352+0000","last_undegraded":"2024-04-25T10:29:10.188352+0000","last_fullsized":"2024-04-25T10:29:10.188352+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:29:06.989570+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:29:06.989570+0000","last_clean_scrub_stamp":"2024-04-25T10:29:06.989570+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T20:22:56.380539+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.14","version":"0'0","reported_seq":12,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-04-25T10:29:10.024492+0000","last_change":"2024-04-25T10:29:09.021410+0000","last_active":"2024-04-25T10:29:10.024492+0000","last_peered":"2024-04-25T10:29:10.024492+0000","last_clean":"2024-04-25T10:29:10.024492+0000","last_became_active":"2024-04-25T10:29:09.021051+0000","last_became_peered":"2024-04-25T10:29:09.021051+0000","last_unstale":"2024-04-25T10:29:10.024492+0000","last_undegraded":"2024-04-25T10:29:10.024492+0000","last_fullsized":"2024-04-25T10:29:10.024492+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:29:07.995096+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:29:07.995096+0000","last_clean_scrub_stamp":"2024-04-25T10:29:07.995096+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T13:22:13.080513+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.14","version":"22'6","reported_seq":20,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-04-25T10:29:10.025912+0000","last_change":"2024-04-25T10:29:08.018044+0000","last_active":"2024-04-25T10:29:10.025912+0000","last_peered":"2024-04-25T10:29:10.025912+0000","last_clean":"2024-04-25T10:29:10.025912+0000","last_became_active":"2024-04-25T10:29:08.016984+0000","last_became_peered":"2024-04-25T10:29:08.016984+0000","last_unstale":"2024-04-25T10:29:10.025912+0000","last_undegraded":"2024-04-25T10:29:10.025912+0000","last_fullsized":"2024-04-25T10:29:10.025912+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:29:06.989570+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:29:06.989570+0000","last_clean_scrub_stamp":"2024-04-25T10:29:06.989570+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T18:29:56.525236+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1612,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4],"acting":[2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.15","version":"0'0","reported_seq":12,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-04-25T10:29:10.037668+0000","last_change":"2024-04-25T10:29:09.029034+0000","last_active":"2024-04-25T10:29:10.037668+0000","last_peered":"2024-04-25T10:29:10.037668+0000","last_clean":"2024-04-25T10:29:10.037668+0000","last_became_active":"2024-04-25T10:29:09.028662+0000","last_became_peered":"2024-04-25T10:29:09.028662+0000","last_unstale":"2024-04-25T10:29:10.037668+0000","last_undegraded":"2024-04-25T10:29:10.037668+0000","last_fullsized":"2024-04-25T10:29:10.037668+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:29:07.995096+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:29:07.995096+0000","last_clean_scrub_stamp":"2024-04-25T10:29:07.995096+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T17:30:54.093151+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.13","version":"22'2","reported_seq":16,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-04-25T10:29:10.189430+0000","last_change":"2024-04-25T10:29:08.020476+0000","last_active":"2024-04-25T10:29:10.189430+0000","last_peered":"2024-04-25T10:29:10.189430+0000","last_clean":"2024-04-25T10:29:10.189430+0000","last_became_active":"2024-04-25T10:29:08.020096+0000","last_became_peered":"2024-04-25T10:29:08.020096+0000","last_unstale":"2024-04-25T10:29:10.189430+0000","last_undegraded":"2024-04-25T10:29:10.189430+0000","last_fullsized":"2024-04-25T10:29:10.189430+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:29:06.989570+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:29:06.989570+0000","last_clean_scrub_stamp":"2024-04-25T10:29:06.989570+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T10:32:06.103123+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":34,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":7,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.12","version":"0'0","reported_seq":12,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-04-25T10:29:10.188838+0000","last_change":"2024-04-25T10:29:09.027996+0000","last_active":"2024-04-25T10:29:10.188838+0000","last_peered":"2024-04-25T10:29:10.188838+0000","last_clean":"2024-04-25T10:29:10.188838+0000","last_became_active":"2024-04-25T10:29:09.027538+0000","last_became_peered":"2024-04-25T10:29:09.027538+0000","last_unstale":"2024-04-25T10:29:10.188838+0000","last_undegraded":"2024-04-25T10:29:10.188838+0000","last_fullsized":"2024-04-25T10:29:10.188838+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:29:07.995096+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:29:07.995096+0000","last_clean_scrub_stamp":"2024-04-25T10:29:07.995096+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T12:16:47.983013+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.12","version":"0'0","reported_seq":14,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-04-25T10:29:10.024603+0000","last_change":"2024-04-25T10:29:08.019028+0000","last_active":"2024-04-25T10:29:10.024603+0000","last_peered":"2024-04-25T10:29:10.024603+0000","last_clean":"2024-04-25T10:29:10.024603+0000","last_became_active":"2024-04-25T10:29:08.018582+0000","last_became_peered":"2024-04-25T10:29:08.018582+0000","last_unstale":"2024-04-25T10:29:10.024603+0000","last_undegraded":"2024-04-25T10:29:10.024603+0000","last_fullsized":"2024-04-25T10:29:10.024603+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:29:06.989570+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:29:06.989570+0000","last_clean_scrub_stamp":"2024-04-25T10:29:06.989570+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T19:23:11.583888+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.13","version":"0'0","reported_seq":12,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-04-25T10:29:10.024281+0000","last_change":"2024-04-25T10:29:09.023995+0000","last_active":"2024-04-25T10:29:10.024281+0000","last_peered":"2024-04-25T10:29:10.024281+0000","last_clean":"2024-04-25T10:29:10.024281+0000","last_became_active":"2024-04-25T10:29:09.023690+0000","last_became_peered":"2024-04-25T10:29:09.023690+0000","last_unstale":"2024-04-25T10:29:10.024281+0000","last_undegraded":"2024-04-25T10:29:10.024281+0000","last_fullsized":"2024-04-25T10:29:10.024281+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:29:07.995096+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:29:07.995096+0000","last_clean_scrub_stamp":"2024-04-25T10:29:07.995096+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T15:28:23.689564+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.11","version":"0'0","reported_seq":14,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-04-25T10:29:10.037576+0000","last_change":"2024-04-25T10:29:08.027174+0000","last_active":"2024-04-25T10:29:10.037576+0000","last_peered":"2024-04-25T10:29:10.037576+0000","last_clean":"2024-04-25T10:29:10.037576+0000","last_became_active":"2024-04-25T10:29:08.026809+0000","last_became_peered":"2024-04-25T10:29:08.026809+0000","last_unstale":"2024-04-25T10:29:10.037576+0000","last_undegraded":"2024-04-25T10:29:10.037576+0000","last_fullsized":"2024-04-25T10:29:10.037576+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:29:06.989570+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:29:06.989570+0000","last_clean_scrub_stamp":"2024-04-25T10:29:06.989570+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T10:35:30.355988+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.10","version":"0'0","reported_seq":12,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-04-25T10:29:10.187809+0000","last_change":"2024-04-25T10:29:09.041722+0000","last_active":"2024-04-25T10:29:10.187809+0000","last_peered":"2024-04-25T10:29:10.187809+0000","last_clean":"2024-04-25T10:29:10.187809+0000","last_became_active":"2024-04-25T10:29:09.040971+0000","last_became_peered":"2024-04-25T10:29:09.040971+0000","last_unstale":"2024-04-25T10:29:10.187809+0000","last_undegraded":"2024-04-25T10:29:10.187809+0000","last_fullsized":"2024-04-25T10:29:10.187809+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:29:07.995096+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:29:07.995096+0000","last_clean_scrub_stamp":"2024-04-25T10:29:07.995096+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T10:35:31.361514+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.10","version":"0'0","reported_seq":9,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-04-25T10:29:08.019405+0000","last_change":"2024-04-25T10:29:08.014110+0000","last_active":"2024-04-25T10:29:08.019405+0000","last_peered":"2024-04-25T10:29:08.019405+0000","last_clean":"2024-04-25T10:29:08.019405+0000","last_became_active":"2024-04-25T10:29:08.013543+0000","last_became_peered":"2024-04-25T10:29:08.013543+0000","last_unstale":"2024-04-25T10:29:08.019405+0000","last_undegraded":"2024-04-25T10:29:08.019405+0000","last_fullsized":"2024-04-25T10:29:08.019405+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:29:06.989570+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:29:06.989570+0000","last_clean_scrub_stamp":"2024-04-25T10:29:06.989570+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T21:35:43.931886+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7],"acting":[4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.11","version":"0'0","reported_seq":12,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-04-25T10:29:10.024238+0000","last_change":"2024-04-25T10:29:09.020490+0000","last_active":"2024-04-25T10:29:10.024238+0000","last_peered":"2024-04-25T10:29:10.024238+0000","last_clean":"2024-04-25T10:29:10.024238+0000","last_became_active":"2024-04-25T10:29:09.020178+0000","last_became_peered":"2024-04-25T10:29:09.020178+0000","last_unstale":"2024-04-25T10:29:10.024238+0000","last_undegraded":"2024-04-25T10:29:10.024238+0000","last_fullsized":"2024-04-25T10:29:10.024238+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:29:07.995096+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:29:07.995096+0000","last_clean_scrub_stamp":"2024-04-25T10:29:07.995096+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T20:37:20.726766+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.f","version":"22'1","reported_seq":15,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-04-25T10:29:10.038104+0000","last_change":"2024-04-25T10:29:08.029879+0000","last_active":"2024-04-25T10:29:10.038104+0000","last_peered":"2024-04-25T10:29:10.038104+0000","last_clean":"2024-04-25T10:29:10.038104+0000","last_became_active":"2024-04-25T10:29:08.029549+0000","last_became_peered":"2024-04-25T10:29:08.029549+0000","last_unstale":"2024-04-25T10:29:10.038104+0000","last_undegraded":"2024-04-25T10:29:10.038104+0000","last_fullsized":"2024-04-25T10:29:10.038104+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:29:06.989570+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:29:06.989570+0000","last_clean_scrub_stamp":"2024-04-25T10:29:06.989570+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T15:37:05.717854+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.e","version":"0'0","reported_seq":12,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-04-25T10:29:10.026273+0000","last_change":"2024-04-25T10:29:09.033703+0000","last_active":"2024-04-25T10:29:10.026273+0000","last_peered":"2024-04-25T10:29:10.026273+0000","last_clean":"2024-04-25T10:29:10.026273+0000","last_became_active":"2024-04-25T10:29:09.033318+0000","last_became_peered":"2024-04-25T10:29:09.033318+0000","last_unstale":"2024-04-25T10:29:10.026273+0000","last_undegraded":"2024-04-25T10:29:10.026273+0000","last_fullsized":"2024-04-25T10:29:10.026273+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:29:07.995096+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:29:07.995096+0000","last_clean_scrub_stamp":"2024-04-25T10:29:07.995096+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T12:16:47.983013+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.e","version":"0'0","reported_seq":9,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-04-25T10:29:08.027622+0000","last_change":"2024-04-25T10:29:08.020703+0000","last_active":"2024-04-25T10:29:08.027622+0000","last_peered":"2024-04-25T10:29:08.027622+0000","last_clean":"2024-04-25T10:29:08.027622+0000","last_became_active":"2024-04-25T10:29:08.020321+0000","last_became_peered":"2024-04-25T10:29:08.020321+0000","last_unstale":"2024-04-25T10:29:08.027622+0000","last_undegraded":"2024-04-25T10:29:08.027622+0000","last_fullsized":"2024-04-25T10:29:08.027622+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:29:06.989570+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:29:06.989570+0000","last_clean_scrub_stamp":"2024-04-25T10:29:06.989570+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T15:21:12.948208+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5],"acting":[4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.f","version":"0'0","reported_seq":12,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-04-25T10:29:10.038152+0000","last_change":"2024-04-25T10:29:09.022708+0000","last_active":"2024-04-25T10:29:10.038152+0000","last_peered":"2024-04-25T10:29:10.038152+0000","last_clean":"2024-04-25T10:29:10.038152+0000","last_became_active":"2024-04-25T10:29:09.020856+0000","last_became_peered":"2024-04-25T10:29:09.020856+0000","last_unstale":"2024-04-25T10:29:10.038152+0000","last_undegraded":"2024-04-25T10:29:10.038152+0000","last_fullsized":"2024-04-25T10:29:10.038152+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:29:07.995096+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:29:07.995096+0000","last_clean_scrub_stamp":"2024-04-25T10:29:07.995096+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T15:28:23.689564+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.d","version":"22'1","reported_seq":15,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-04-25T10:29:10.189469+0000","last_change":"2024-04-25T10:29:08.019488+0000","last_active":"2024-04-25T10:29:10.189469+0000","last_peered":"2024-04-25T10:29:10.189469+0000","last_clean":"2024-04-25T10:29:10.189469+0000","last_became_active":"2024-04-25T10:29:08.018971+0000","last_became_peered":"2024-04-25T10:29:08.018971+0000","last_unstale":"2024-04-25T10:29:10.189469+0000","last_undegraded":"2024-04-25T10:29:10.189469+0000","last_fullsized":"2024-04-25T10:29:10.189469+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:29:06.989570+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:29:06.989570+0000","last_clean_scrub_stamp":"2024-04-25T10:29:06.989570+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T20:22:56.380539+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":70,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.c","version":"0'0","reported_seq":12,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-04-25T10:29:10.037902+0000","last_change":"2024-04-25T10:29:09.029666+0000","last_active":"2024-04-25T10:29:10.037902+0000","last_peered":"2024-04-25T10:29:10.037902+0000","last_clean":"2024-04-25T10:29:10.037902+0000","last_became_active":"2024-04-25T10:29:09.029184+0000","last_became_peered":"2024-04-25T10:29:09.029184+0000","last_unstale":"2024-04-25T10:29:10.037902+0000","last_undegraded":"2024-04-25T10:29:10.037902+0000","last_fullsized":"2024-04-25T10:29:10.037902+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:29:07.995096+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:29:07.995096+0000","last_clean_scrub_stamp":"2024-04-25T10:29:07.995096+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T13:54:22.968216+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.c","version":"0'0","reported_seq":14,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-04-25T10:29:10.189541+0000","last_change":"2024-04-25T10:29:08.012124+0000","last_active":"2024-04-25T10:29:10.189541+0000","last_peered":"2024-04-25T10:29:10.189541+0000","last_clean":"2024-04-25T10:29:10.189541+0000","last_became_active":"2024-04-25T10:29:08.011817+0000","last_became_peered":"2024-04-25T10:29:08.011817+0000","last_unstale":"2024-04-25T10:29:10.189541+0000","last_undegraded":"2024-04-25T10:29:10.189541+0000","last_fullsized":"2024-04-25T10:29:10.189541+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:29:06.989570+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:29:06.989570+0000","last_clean_scrub_stamp":"2024-04-25T10:29:06.989570+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T20:22:56.380539+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.d","version":"0'0","reported_seq":12,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-04-25T10:29:10.024815+0000","last_change":"2024-04-25T10:29:09.017227+0000","last_active":"2024-04-25T10:29:10.024815+0000","last_peered":"2024-04-25T10:29:10.024815+0000","last_clean":"2024-04-25T10:29:10.024815+0000","last_became_active":"2024-04-25T10:29:09.016911+0000","last_became_peered":"2024-04-25T10:29:09.016911+0000","last_unstale":"2024-04-25T10:29:10.024815+0000","last_undegraded":"2024-04-25T10:29:10.024815+0000","last_fullsized":"2024-04-25T10:29:10.024815+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:29:07.995096+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:29:07.995096+0000","last_clean_scrub_stamp":"2024-04-25T10:29:07.995096+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T18:51:41.934269+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.0","version":"18'72","reported_seq":110,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-04-25T10:29:10.187517+0000","last_change":"2024-04-25T10:28:57.972696+0000","last_active":"2024-04-25T10:29:10.187517+0000","last_peered":"2024-04-25T10:29:10.187517+0000","last_clean":"2024-04-25T10:29:10.187517+0000","last_became_active":"2024-04-25T10:28:57.972373+0000","last_became_peered":"2024-04-25T10:28:57.972373+0000","last_unstale":"2024-04-25T10:29:10.187517+0000","last_undegraded":"2024-04-25T10:29:10.187517+0000","last_fullsized":"2024-04-25T10:29:10.187517+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:28:56.959439+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:28:56.959439+0000","last_clean_scrub_stamp":"2024-04-25T10:28:56.959439+0000","objects_scrubbed":0,"log_size":72,"log_dups_size":0,"ondisk_log_size":72,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T17:34:34.057408+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":66,"num_read_kb":55,"num_write":101,"num_write_kb":1628,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.3","version":"0'0","reported_seq":14,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-04-25T10:29:10.024562+0000","last_change":"2024-04-25T10:29:08.025080+0000","last_active":"2024-04-25T10:29:10.024562+0000","last_peered":"2024-04-25T10:29:10.024562+0000","last_clean":"2024-04-25T10:29:10.024562+0000","last_became_active":"2024-04-25T10:29:08.024747+0000","last_became_peered":"2024-04-25T10:29:08.024747+0000","last_unstale":"2024-04-25T10:29:10.024562+0000","last_undegraded":"2024-04-25T10:29:10.024562+0000","last_fullsized":"2024-04-25T10:29:10.024562+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:29:06.989570+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:29:06.989570+0000","last_clean_scrub_stamp":"2024-04-25T10:29:06.989570+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T22:07:59.361905+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.2","version":"0'0","reported_seq":1,"reported_epoch":21,"state":"creating+peering","last_fresh":"2024-04-25T10:29:08.008681+0000","last_change":"2024-04-25T10:29:08.008681+0000","last_active":"2024-04-25T10:29:07.995096+0000","last_peered":"2024-04-25T10:29:07.995096+0000","last_clean":"2024-04-25T10:29:07.995096+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T10:29:08.008681+0000","last_undegraded":"2024-04-25T10:29:08.008681+0000","last_fullsized":"2024-04-25T10:29:08.008681+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:29:07.995096+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:29:07.995096+0000","last_clean_scrub_stamp":"2024-04-25T10:29:07.995096+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.0","version":"0'0","reported_seq":14,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-04-25T10:29:10.024309+0000","last_change":"2024-04-25T10:29:08.027479+0000","last_active":"2024-04-25T10:29:10.024309+0000","last_peered":"2024-04-25T10:29:10.024309+0000","last_clean":"2024-04-25T10:29:10.024309+0000","last_became_active":"2024-04-25T10:29:08.027178+0000","last_became_peered":"2024-04-25T10:29:08.027178+0000","last_unstale":"2024-04-25T10:29:10.024309+0000","last_undegraded":"2024-04-25T10:29:10.024309+0000","last_fullsized":"2024-04-25T10:29:10.024309+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:29:06.989570+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:29:06.989570+0000","last_clean_scrub_stamp":"2024-04-25T10:29:06.989570+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T19:54:16.697792+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.1","version":"0'0","reported_seq":12,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-04-25T10:29:10.188195+0000","last_change":"2024-04-25T10:29:09.028777+0000","last_active":"2024-04-25T10:29:10.188195+0000","last_peered":"2024-04-25T10:29:10.188195+0000","last_clean":"2024-04-25T10:29:10.188195+0000","last_became_active":"2024-04-25T10:29:09.027709+0000","last_became_peered":"2024-04-25T10:29:09.027709+0000","last_unstale":"2024-04-25T10:29:10.188195+0000","last_undegraded":"2024-04-25T10:29:10.188195+0000","last_fullsized":"2024-04-25T10:29:10.188195+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:29:07.995096+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:29:07.995096+0000","last_clean_scrub_stamp":"2024-04-25T10:29:07.995096+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T22:08:00.367431+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,2],"acting":[6,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.1","version":"0'0","reported_seq":14,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-04-25T10:29:10.188429+0000","last_change":"2024-04-25T10:29:08.020944+0000","last_active":"2024-04-25T10:29:10.188429+0000","last_peered":"2024-04-25T10:29:10.188429+0000","last_clean":"2024-04-25T10:29:10.188429+0000","last_became_active":"2024-04-25T10:29:08.020259+0000","last_became_peered":"2024-04-25T10:29:08.020259+0000","last_unstale":"2024-04-25T10:29:10.188429+0000","last_undegraded":"2024-04-25T10:29:10.188429+0000","last_fullsized":"2024-04-25T10:29:10.188429+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:29:06.989570+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:29:06.989570+0000","last_clean_scrub_stamp":"2024-04-25T10:29:06.989570+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T16:31:11.685101+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.0","version":"0'0","reported_seq":12,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-04-25T10:29:10.189305+0000","last_change":"2024-04-25T10:29:09.035455+0000","last_active":"2024-04-25T10:29:10.189305+0000","last_peered":"2024-04-25T10:29:10.189305+0000","last_clean":"2024-04-25T10:29:10.189305+0000","last_became_active":"2024-04-25T10:29:09.035130+0000","last_became_peered":"2024-04-25T10:29:09.035130+0000","last_unstale":"2024-04-25T10:29:10.189305+0000","last_undegraded":"2024-04-25T10:29:10.189305+0000","last_fullsized":"2024-04-25T10:29:10.189305+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:29:07.995096+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:29:07.995096+0000","last_clean_scrub_stamp":"2024-04-25T10:29:07.995096+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T11:52:24.415332+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.2","version":"0'0","reported_seq":14,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-04-25T10:29:10.024714+0000","last_change":"2024-04-25T10:29:08.021694+0000","last_active":"2024-04-25T10:29:10.024714+0000","last_peered":"2024-04-25T10:29:10.024714+0000","last_clean":"2024-04-25T10:29:10.024714+0000","last_became_active":"2024-04-25T10:29:08.021052+0000","last_became_peered":"2024-04-25T10:29:08.021052+0000","last_unstale":"2024-04-25T10:29:10.024714+0000","last_undegraded":"2024-04-25T10:29:10.024714+0000","last_fullsized":"2024-04-25T10:29:10.024714+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:29:06.989570+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:29:06.989570+0000","last_clean_scrub_stamp":"2024-04-25T10:29:06.989570+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T21:14:38.656431+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.3","version":"0'0","reported_seq":12,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-04-25T10:29:10.037784+0000","last_change":"2024-04-25T10:29:09.021686+0000","last_active":"2024-04-25T10:29:10.037784+0000","last_peered":"2024-04-25T10:29:10.037784+0000","last_clean":"2024-04-25T10:29:10.037784+0000","last_became_active":"2024-04-25T10:29:09.020672+0000","last_became_peered":"2024-04-25T10:29:09.020672+0000","last_unstale":"2024-04-25T10:29:10.037784+0000","last_undegraded":"2024-04-25T10:29:10.037784+0000","last_fullsized":"2024-04-25T10:29:10.037784+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:29:07.995096+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:29:07.995096+0000","last_clean_scrub_stamp":"2024-04-25T10:29:07.995096+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T20:52:02.098872+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.4","version":"0'0","reported_seq":14,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-04-25T10:29:10.189133+0000","last_change":"2024-04-25T10:29:08.025582+0000","last_active":"2024-04-25T10:29:10.189133+0000","last_peered":"2024-04-25T10:29:10.189133+0000","last_clean":"2024-04-25T10:29:10.189133+0000","last_became_active":"2024-04-25T10:29:08.024961+0000","last_became_peered":"2024-04-25T10:29:08.024961+0000","last_unstale":"2024-04-25T10:29:10.189133+0000","last_undegraded":"2024-04-25T10:29:10.189133+0000","last_fullsized":"2024-04-25T10:29:10.189133+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:29:06.989570+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:29:06.989570+0000","last_clean_scrub_stamp":"2024-04-25T10:29:06.989570+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T10:32:06.103123+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6],"acting":[5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.5","version":"0'0","reported_seq":12,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-04-25T10:29:10.026083+0000","last_change":"2024-04-25T10:29:09.028003+0000","last_active":"2024-04-25T10:29:10.026083+0000","last_peered":"2024-04-25T10:29:10.026083+0000","last_clean":"2024-04-25T10:29:10.026083+0000","last_became_active":"2024-04-25T10:29:09.027654+0000","last_became_peered":"2024-04-25T10:29:09.027654+0000","last_unstale":"2024-04-25T10:29:10.026083+0000","last_undegraded":"2024-04-25T10:29:10.026083+0000","last_fullsized":"2024-04-25T10:29:10.026083+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:29:07.995096+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:29:07.995096+0000","last_clean_scrub_stamp":"2024-04-25T10:29:07.995096+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T19:54:17.703318+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,6],"acting":[2,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.5","version":"0'0","reported_seq":14,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-04-25T10:29:10.024926+0000","last_change":"2024-04-25T10:29:08.026181+0000","last_active":"2024-04-25T10:29:10.024926+0000","last_peered":"2024-04-25T10:29:10.024926+0000","last_clean":"2024-04-25T10:29:10.024926+0000","last_became_active":"2024-04-25T10:29:08.025780+0000","last_became_peered":"2024-04-25T10:29:08.025780+0000","last_unstale":"2024-04-25T10:29:10.024926+0000","last_undegraded":"2024-04-25T10:29:10.024926+0000","last_fullsized":"2024-04-25T10:29:10.024926+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:29:06.989570+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:29:06.989570+0000","last_clean_scrub_stamp":"2024-04-25T10:29:06.989570+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T12:16:46.977487+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.4","version":"0'0","reported_seq":12,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-04-25T10:29:10.189423+0000","last_change":"2024-04-25T10:29:09.037824+0000","last_active":"2024-04-25T10:29:10.189423+0000","last_peered":"2024-04-25T10:29:10.189423+0000","last_clean":"2024-04-25T10:29:10.189423+0000","last_became_active":"2024-04-25T10:29:09.037478+0000","last_became_peered":"2024-04-25T10:29:09.037478+0000","last_unstale":"2024-04-25T10:29:10.189423+0000","last_undegraded":"2024-04-25T10:29:10.189423+0000","last_fullsized":"2024-04-25T10:29:10.189423+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:29:07.995096+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:29:07.995096+0000","last_clean_scrub_stamp":"2024-04-25T10:29:07.995096+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T15:02:17.480054+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.6","version":"0'0","reported_seq":9,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-04-25T10:29:08.027595+0000","last_change":"2024-04-25T10:29:08.020327+0000","last_active":"2024-04-25T10:29:08.027595+0000","last_peered":"2024-04-25T10:29:08.027595+0000","last_clean":"2024-04-25T10:29:08.027595+0000","last_became_active":"2024-04-25T10:29:08.019988+0000","last_became_peered":"2024-04-25T10:29:08.019988+0000","last_unstale":"2024-04-25T10:29:08.027595+0000","last_undegraded":"2024-04-25T10:29:08.027595+0000","last_fullsized":"2024-04-25T10:29:08.027595+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:29:06.989570+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:29:06.989570+0000","last_clean_scrub_stamp":"2024-04-25T10:29:06.989570+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T12:07:01.943484+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.7","version":"0'0","reported_seq":1,"reported_epoch":21,"state":"creating+peering","last_fresh":"2024-04-25T10:29:08.013060+0000","last_change":"2024-04-25T10:29:08.013060+0000","last_active":"2024-04-25T10:29:07.995096+0000","last_peered":"2024-04-25T10:29:07.995096+0000","last_clean":"2024-04-25T10:29:07.995096+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T10:29:08.013060+0000","last_undegraded":"2024-04-25T10:29:08.013060+0000","last_fullsized":"2024-04-25T10:29:08.013060+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:29:07.995096+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:29:07.995096+0000","last_clean_scrub_stamp":"2024-04-25T10:29:07.995096+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.7","version":"22'2","reported_seq":16,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-04-25T10:29:10.024876+0000","last_change":"2024-04-25T10:29:08.026080+0000","last_active":"2024-04-25T10:29:10.024876+0000","last_peered":"2024-04-25T10:29:10.024876+0000","last_clean":"2024-04-25T10:29:10.024876+0000","last_became_active":"2024-04-25T10:29:08.025707+0000","last_became_peered":"2024-04-25T10:29:08.025707+0000","last_unstale":"2024-04-25T10:29:10.024876+0000","last_undegraded":"2024-04-25T10:29:10.024876+0000","last_fullsized":"2024-04-25T10:29:10.024876+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:29:06.989570+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:29:06.989570+0000","last_clean_scrub_stamp":"2024-04-25T10:29:06.989570+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T10:55:00.499631+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.6","version":"0'0","reported_seq":1,"reported_epoch":21,"state":"creating+peering","last_fresh":"2024-04-25T10:29:08.009604+0000","last_change":"2024-04-25T10:29:08.009604+0000","last_active":"2024-04-25T10:29:07.995096+0000","last_peered":"2024-04-25T10:29:07.995096+0000","last_clean":"2024-04-25T10:29:07.995096+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T10:29:08.009604+0000","last_undegraded":"2024-04-25T10:29:08.009604+0000","last_fullsized":"2024-04-25T10:29:08.009604+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:29:07.995096+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:29:07.995096+0000","last_clean_scrub_stamp":"2024-04-25T10:29:07.995096+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7],"acting":[4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.8","version":"0'0","reported_seq":14,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-04-25T10:29:10.037271+0000","last_change":"2024-04-25T10:29:08.024845+0000","last_active":"2024-04-25T10:29:10.037271+0000","last_peered":"2024-04-25T10:29:10.037271+0000","last_clean":"2024-04-25T10:29:10.037271+0000","last_became_active":"2024-04-25T10:29:08.024484+0000","last_became_peered":"2024-04-25T10:29:08.024484+0000","last_unstale":"2024-04-25T10:29:10.037271+0000","last_undegraded":"2024-04-25T10:29:10.037271+0000","last_fullsized":"2024-04-25T10:29:10.037271+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:29:06.989570+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:29:06.989570+0000","last_clean_scrub_stamp":"2024-04-25T10:29:06.989570+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T14:12:57.950482+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.9","version":"0'0","reported_seq":12,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-04-25T10:29:10.024346+0000","last_change":"2024-04-25T10:29:09.022502+0000","last_active":"2024-04-25T10:29:10.024346+0000","last_peered":"2024-04-25T10:29:10.024346+0000","last_clean":"2024-04-25T10:29:10.024346+0000","last_became_active":"2024-04-25T10:29:09.022184+0000","last_became_peered":"2024-04-25T10:29:09.022184+0000","last_unstale":"2024-04-25T10:29:10.024346+0000","last_undegraded":"2024-04-25T10:29:10.024346+0000","last_fullsized":"2024-04-25T10:29:10.024346+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:29:07.995096+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:29:07.995096+0000","last_clean_scrub_stamp":"2024-04-25T10:29:07.995096+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T19:56:27.663748+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.9","version":"0'0","reported_seq":9,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-04-25T10:29:08.027947+0000","last_change":"2024-04-25T10:29:08.024989+0000","last_active":"2024-04-25T10:29:08.027947+0000","last_peered":"2024-04-25T10:29:08.027947+0000","last_clean":"2024-04-25T10:29:08.027947+0000","last_became_active":"2024-04-25T10:29:08.024655+0000","last_became_peered":"2024-04-25T10:29:08.024655+0000","last_unstale":"2024-04-25T10:29:08.027947+0000","last_undegraded":"2024-04-25T10:29:08.027947+0000","last_fullsized":"2024-04-25T10:29:08.027947+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:29:06.989570+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:29:06.989570+0000","last_clean_scrub_stamp":"2024-04-25T10:29:06.989570+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T13:46:20.443676+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.8","version":"0'0","reported_seq":1,"reported_epoch":21,"state":"creating+peering","last_fresh":"2024-04-25T10:29:08.009188+0000","last_change":"2024-04-25T10:29:08.009188+0000","last_active":"2024-04-25T10:29:07.995096+0000","last_peered":"2024-04-25T10:29:07.995096+0000","last_clean":"2024-04-25T10:29:07.995096+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T10:29:08.009188+0000","last_undegraded":"2024-04-25T10:29:08.009188+0000","last_fullsized":"2024-04-25T10:29:08.009188+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:29:07.995096+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:29:07.995096+0000","last_clean_scrub_stamp":"2024-04-25T10:29:07.995096+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7],"acting":[4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.a","version":"0'0","reported_seq":14,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-04-25T10:29:10.188957+0000","last_change":"2024-04-25T10:29:08.020363+0000","last_active":"2024-04-25T10:29:10.188957+0000","last_peered":"2024-04-25T10:29:10.188957+0000","last_clean":"2024-04-25T10:29:10.188957+0000","last_became_active":"2024-04-25T10:29:08.020037+0000","last_became_peered":"2024-04-25T10:29:08.020037+0000","last_unstale":"2024-04-25T10:29:10.188957+0000","last_undegraded":"2024-04-25T10:29:10.188957+0000","last_fullsized":"2024-04-25T10:29:10.188957+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:29:06.989570+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:29:06.989570+0000","last_clean_scrub_stamp":"2024-04-25T10:29:06.989570+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T10:32:06.103123+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.b","version":"0'0","reported_seq":12,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-04-25T10:29:10.189294+0000","last_change":"2024-04-25T10:29:09.028820+0000","last_active":"2024-04-25T10:29:10.189294+0000","last_peered":"2024-04-25T10:29:10.189294+0000","last_clean":"2024-04-25T10:29:10.189294+0000","last_became_active":"2024-04-25T10:29:09.028426+0000","last_became_peered":"2024-04-25T10:29:09.028426+0000","last_unstale":"2024-04-25T10:29:10.189294+0000","last_undegraded":"2024-04-25T10:29:10.189294+0000","last_fullsized":"2024-04-25T10:29:10.189294+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:29:07.995096+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:29:07.995096+0000","last_clean_scrub_stamp":"2024-04-25T10:29:07.995096+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T19:54:17.703318+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.b","version":"22'2","reported_seq":16,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-04-25T10:29:10.024445+0000","last_change":"2024-04-25T10:29:08.025011+0000","last_active":"2024-04-25T10:29:10.024445+0000","last_peered":"2024-04-25T10:29:10.024445+0000","last_clean":"2024-04-25T10:29:10.024445+0000","last_became_active":"2024-04-25T10:29:08.024667+0000","last_became_peered":"2024-04-25T10:29:08.024667+0000","last_unstale":"2024-04-25T10:29:10.024445+0000","last_undegraded":"2024-04-25T10:29:10.024445+0000","last_fullsized":"2024-04-25T10:29:10.024445+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:29:06.989570+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:29:06.989570+0000","last_clean_scrub_stamp":"2024-04-25T10:29:06.989570+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T15:37:05.717854+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.a","version":"0'0","reported_seq":12,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-04-25T10:29:10.024394+0000","last_change":"2024-04-25T10:29:09.017160+0000","last_active":"2024-04-25T10:29:10.024394+0000","last_peered":"2024-04-25T10:29:10.024394+0000","last_clean":"2024-04-25T10:29:10.024394+0000","last_became_active":"2024-04-25T10:29:09.016821+0000","last_became_peered":"2024-04-25T10:29:09.016821+0000","last_unstale":"2024-04-25T10:29:10.024394+0000","last_undegraded":"2024-04-25T10:29:10.024394+0000","last_fullsized":"2024-04-25T10:29:10.024394+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:29:07.995096+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:29:07.995096+0000","last_clean_scrub_stamp":"2024-04-25T10:29:07.995096+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T20:40:19.840339+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.1d","version":"0'0","reported_seq":12,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-04-25T10:29:10.024622+0000","last_change":"2024-04-25T10:29:09.030341+0000","last_active":"2024-04-25T10:29:10.024622+0000","last_peered":"2024-04-25T10:29:10.024622+0000","last_clean":"2024-04-25T10:29:10.024622+0000","last_became_active":"2024-04-25T10:29:09.029988+0000","last_became_peered":"2024-04-25T10:29:09.029988+0000","last_unstale":"2024-04-25T10:29:10.024622+0000","last_undegraded":"2024-04-25T10:29:10.024622+0000","last_fullsized":"2024-04-25T10:29:10.024622+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:29:07.995096+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:29:07.995096+0000","last_clean_scrub_stamp":"2024-04-25T10:29:07.995096+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T11:40:18.624659+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.1c","version":"22'2","reported_seq":16,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-04-25T10:29:10.037969+0000","last_change":"2024-04-25T10:29:08.027197+0000","last_active":"2024-04-25T10:29:10.037969+0000","last_peered":"2024-04-25T10:29:10.037969+0000","last_clean":"2024-04-25T10:29:10.037969+0000","last_became_active":"2024-04-25T10:29:08.026872+0000","last_became_peered":"2024-04-25T10:29:08.026872+0000","last_unstale":"2024-04-25T10:29:10.037969+0000","last_undegraded":"2024-04-25T10:29:10.037969+0000","last_fullsized":"2024-04-25T10:29:10.037969+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:29:06.989570+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:29:06.989570+0000","last_clean_scrub_stamp":"2024-04-25T10:29:06.989570+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T15:58:35.843305+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.1c","version":"0'0","reported_seq":12,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-04-25T10:29:10.187853+0000","last_change":"2024-04-25T10:29:09.032731+0000","last_active":"2024-04-25T10:29:10.187853+0000","last_peered":"2024-04-25T10:29:10.187853+0000","last_clean":"2024-04-25T10:29:10.187853+0000","last_became_active":"2024-04-25T10:29:09.032383+0000","last_became_peered":"2024-04-25T10:29:09.032383+0000","last_unstale":"2024-04-25T10:29:10.187853+0000","last_undegraded":"2024-04-25T10:29:10.187853+0000","last_fullsized":"2024-04-25T10:29:10.187853+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:29:07.995096+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:29:07.995096+0000","last_clean_scrub_stamp":"2024-04-25T10:29:07.995096+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T19:54:17.703318+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.1d","version":"22'2","reported_seq":16,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-04-25T10:29:10.037807+0000","last_change":"2024-04-25T10:29:08.026410+0000","last_active":"2024-04-25T10:29:10.037807+0000","last_peered":"2024-04-25T10:29:10.037807+0000","last_clean":"2024-04-25T10:29:10.037807+0000","last_became_active":"2024-04-25T10:29:08.026071+0000","last_became_peered":"2024-04-25T10:29:08.026071+0000","last_unstale":"2024-04-25T10:29:10.037807+0000","last_undegraded":"2024-04-25T10:29:10.037807+0000","last_fullsized":"2024-04-25T10:29:10.037807+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:29:06.989570+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:29:06.989570+0000","last_clean_scrub_stamp":"2024-04-25T10:29:06.989570+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T19:54:16.697792+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.1f","version":"0'0","reported_seq":12,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-04-25T10:29:10.189240+0000","last_change":"2024-04-25T10:29:09.032735+0000","last_active":"2024-04-25T10:29:10.189240+0000","last_peered":"2024-04-25T10:29:10.189240+0000","last_clean":"2024-04-25T10:29:10.189240+0000","last_became_active":"2024-04-25T10:29:09.032227+0000","last_became_peered":"2024-04-25T10:29:09.032227+0000","last_unstale":"2024-04-25T10:29:10.189240+0000","last_undegraded":"2024-04-25T10:29:10.189240+0000","last_fullsized":"2024-04-25T10:29:10.189240+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:29:07.995096+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:29:07.995096+0000","last_clean_scrub_stamp":"2024-04-25T10:29:07.995096+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T11:31:45.283247+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.1e","version":"22'1","reported_seq":15,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-04-25T10:29:10.189728+0000","last_change":"2024-04-25T10:29:08.017025+0000","last_active":"2024-04-25T10:29:10.189728+0000","last_peered":"2024-04-25T10:29:10.189728+0000","last_clean":"2024-04-25T10:29:10.189728+0000","last_became_active":"2024-04-25T10:29:08.016702+0000","last_became_peered":"2024-04-25T10:29:08.016702+0000","last_unstale":"2024-04-25T10:29:10.189728+0000","last_undegraded":"2024-04-25T10:29:10.189728+0000","last_fullsized":"2024-04-25T10:29:10.189728+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:29:06.989570+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:29:06.989570+0000","last_clean_scrub_stamp":"2024-04-25T10:29:06.989570+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T14:12:57.950482+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.1e","version":"0'0","reported_seq":12,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-04-25T10:29:10.189730+0000","last_change":"2024-04-25T10:29:09.036208+0000","last_active":"2024-04-25T10:29:10.189730+0000","last_peered":"2024-04-25T10:29:10.189730+0000","last_clean":"2024-04-25T10:29:10.189730+0000","last_became_active":"2024-04-25T10:29:09.034890+0000","last_became_peered":"2024-04-25T10:29:09.034890+0000","last_unstale":"2024-04-25T10:29:10.189730+0000","last_undegraded":"2024-04-25T10:29:10.189730+0000","last_fullsized":"2024-04-25T10:29:10.189730+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:29:07.995096+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:29:07.995096+0000","last_clean_scrub_stamp":"2024-04-25T10:29:07.995096+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T18:44:23.610374+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.1f","version":"0'0","reported_seq":9,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-04-25T10:29:08.027976+0000","last_change":"2024-04-25T10:29:08.026202+0000","last_active":"2024-04-25T10:29:08.027976+0000","last_peered":"2024-04-25T10:29:08.027976+0000","last_clean":"2024-04-25T10:29:08.027976+0000","last_became_active":"2024-04-25T10:29:08.025877+0000","last_became_peered":"2024-04-25T10:29:08.025877+0000","last_unstale":"2024-04-25T10:29:08.027976+0000","last_undegraded":"2024-04-25T10:29:08.027976+0000","last_fullsized":"2024-04-25T10:29:08.027976+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:29:06.989570+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:29:06.989570+0000","last_clean_scrub_stamp":"2024-04-25T10:29:06.989570+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T18:56:03.528807+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]}],"pool_stats":[{"poolid":3,"num_pg":64,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":128,"acting":128,"num_store_stats":8},{"poolid":2,"num_pg":32,"stat_sum":{"num_bytes":1828,"num_objects":14,"num_object_clones":0,"num_object_copies":28,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":14,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":25,"num_write_kb":13,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":7,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":45056,"data_stored":2536,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":24,"ondisk_log_size":24,"up":64,"acting":64,"num_store_stats":8},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":66,"num_read_kb":55,"num_write":101,"num_write_kb":1628,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1187840,"data_stored":1180736,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":72,"ondisk_log_size":72,"up":2,"acting":2,"num_store_stats":2}],"osd_stats":[{"osd":7,"up_from":15,"seq":64424509445,"num_pgs":3,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27244,"kb_used_data":224,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344596,"statfs":{"total":96636764160,"available":96608866304,"internally_reserved":0,"allocated":229376,"data_stored":75059,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1590,"internal_metadata":27654602},"hb_peers":[0,1,2,3,4,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[]},{"osd":6,"up_from":15,"seq":64424509446,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27236,"kb_used_data":216,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344604,"statfs":{"total":96636764160,"available":96608874496,"internally_reserved":0,"allocated":221184,"data_stored":74935,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":27654598},"hb_peers":[0,1,2,3,4,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[]},{"osd":5,"up_from":15,"seq":64424509447,"num_pgs":1,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27824,"kb_used_data":804,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344016,"statfs":{"total":96636764160,"available":96608272384,"internally_reserved":0,"allocated":823296,"data_stored":665395,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":6,"apply_latency_ms":6,"commit_latency_ns":6000000,"apply_latency_ns":6000000},"alerts":[]},{"osd":3,"up_from":15,"seq":64424509447,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27240,"kb_used_data":220,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344600,"statfs":{"total":96636764160,"available":96608870400,"internally_reserved":0,"allocated":225280,"data_stored":75517,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1593,"internal_metadata":27654599},"hb_peers":[0,1,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[]},{"osd":2,"up_from":15,"seq":64424509447,"num_pgs":1,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27248,"kb_used_data":228,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344592,"statfs":{"total":96636764160,"available":96608862208,"internally_reserved":0,"allocated":233472,"data_stored":76547,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[]},{"osd":1,"up_from":15,"seq":64424509445,"num_pgs":12,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27820,"kb_used_data":800,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344020,"statfs":{"total":96636764160,"available":96608276480,"internally_reserved":0,"allocated":819200,"data_stored":665373,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[]},{"osd":0,"up_from":15,"seq":64424509446,"num_pgs":35,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27244,"kb_used_data":224,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344596,"statfs":{"total":96636764160,"available":96608866304,"internally_reserved":0,"allocated":229376,"data_stored":74991,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[1,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":5,"apply_latency_ms":5,"commit_latency_ns":5000000,"apply_latency_ns":5000000},"alerts":[]},{"osd":4,"up_from":13,"seq":55834574853,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27212,"kb_used_data":192,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344628,"statfs":{"total":96636764160,"available":96608899072,"internally_reserved":0,"allocated":196608,"data_stored":61885,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":56,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":70,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":1612,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":582,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":92,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":124,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-04-25T10:29:13.932 INFO:tasks.cephfs.filesystem.ceph_manager:PG 3.2b is not active+clean 2024-04-25T10:29:13.932 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '3.2b', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 21, 'state': 'creating+peering', 'last_fresh': '2024-04-25T10:29:08.012124+0000', 'last_change': '2024-04-25T10:29:08.012124+0000', 'last_active': '2024-04-25T10:29:07.995096+0000', 'last_peered': '2024-04-25T10:29:07.995096+0000', 'last_clean': '2024-04-25T10:29:07.995096+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-04-25T10:29:08.012124+0000', 'last_undegraded': '2024-04-25T10:29:08.012124+0000', 'last_fullsized': '2024-04-25T10:29:08.012124+0000', 'mapping_epoch': 21, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 21, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-04-25T10:29:07.995096+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-04-25T10:29:07.995096+0000', 'last_clean_scrub_stamp': '2024-04-25T10:29:07.995096+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [4, 2], 'acting': [4, 2], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 4, 'acting_primary': 4, 'purged_snaps': []} 2024-04-25T10:29:13.932 INFO:tasks.cephfs.filesystem.ceph_manager:PG 3.1b is not active+clean 2024-04-25T10:29:13.933 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '3.1b', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 21, 'state': 'creating+peering', 'last_fresh': '2024-04-25T10:29:08.008604+0000', 'last_change': '2024-04-25T10:29:08.008604+0000', 'last_active': '2024-04-25T10:29:07.995096+0000', 'last_peered': '2024-04-25T10:29:07.995096+0000', 'last_clean': '2024-04-25T10:29:07.995096+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-04-25T10:29:08.008604+0000', 'last_undegraded': '2024-04-25T10:29:08.008604+0000', 'last_fullsized': '2024-04-25T10:29:08.008604+0000', 'mapping_epoch': 21, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 21, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-04-25T10:29:07.995096+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-04-25T10:29:07.995096+0000', 'last_clean_scrub_stamp': '2024-04-25T10:29:07.995096+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [4, 0], 'acting': [4, 0], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 4, 'acting_primary': 4, 'purged_snaps': []} 2024-04-25T10:29:13.933 INFO:tasks.cephfs.filesystem.ceph_manager:PG 3.19 is not active+clean 2024-04-25T10:29:13.933 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '3.19', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 21, 'state': 'creating+peering', 'last_fresh': '2024-04-25T10:29:08.012731+0000', 'last_change': '2024-04-25T10:29:08.012731+0000', 'last_active': '2024-04-25T10:29:07.995096+0000', 'last_peered': '2024-04-25T10:29:07.995096+0000', 'last_clean': '2024-04-25T10:29:07.995096+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-04-25T10:29:08.012731+0000', 'last_undegraded': '2024-04-25T10:29:08.012731+0000', 'last_fullsized': '2024-04-25T10:29:08.012731+0000', 'mapping_epoch': 21, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 21, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-04-25T10:29:07.995096+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-04-25T10:29:07.995096+0000', 'last_clean_scrub_stamp': '2024-04-25T10:29:07.995096+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [4, 1], 'acting': [4, 1], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 4, 'acting_primary': 4, 'purged_snaps': []} 2024-04-25T10:29:13.933 INFO:tasks.cephfs.filesystem.ceph_manager:PG 3.2 is not active+clean 2024-04-25T10:29:13.933 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '3.2', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 21, 'state': 'creating+peering', 'last_fresh': '2024-04-25T10:29:08.008681+0000', 'last_change': '2024-04-25T10:29:08.008681+0000', 'last_active': '2024-04-25T10:29:07.995096+0000', 'last_peered': '2024-04-25T10:29:07.995096+0000', 'last_clean': '2024-04-25T10:29:07.995096+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-04-25T10:29:08.008681+0000', 'last_undegraded': '2024-04-25T10:29:08.008681+0000', 'last_fullsized': '2024-04-25T10:29:08.008681+0000', 'mapping_epoch': 21, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 21, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-04-25T10:29:07.995096+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-04-25T10:29:07.995096+0000', 'last_clean_scrub_stamp': '2024-04-25T10:29:07.995096+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [4, 2], 'acting': [4, 2], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 4, 'acting_primary': 4, 'purged_snaps': []} 2024-04-25T10:29:13.933 INFO:tasks.cephfs.filesystem.ceph_manager:PG 3.7 is not active+clean 2024-04-25T10:29:13.933 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '3.7', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 21, 'state': 'creating+peering', 'last_fresh': '2024-04-25T10:29:08.013060+0000', 'last_change': '2024-04-25T10:29:08.013060+0000', 'last_active': '2024-04-25T10:29:07.995096+0000', 'last_peered': '2024-04-25T10:29:07.995096+0000', 'last_clean': '2024-04-25T10:29:07.995096+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-04-25T10:29:08.013060+0000', 'last_undegraded': '2024-04-25T10:29:08.013060+0000', 'last_fullsized': '2024-04-25T10:29:08.013060+0000', 'mapping_epoch': 21, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 21, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-04-25T10:29:07.995096+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-04-25T10:29:07.995096+0000', 'last_clean_scrub_stamp': '2024-04-25T10:29:07.995096+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [4, 6], 'acting': [4, 6], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 4, 'acting_primary': 4, 'purged_snaps': []} 2024-04-25T10:29:13.933 INFO:tasks.cephfs.filesystem.ceph_manager:PG 3.6 is not active+clean 2024-04-25T10:29:13.933 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '3.6', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 21, 'state': 'creating+peering', 'last_fresh': '2024-04-25T10:29:08.009604+0000', 'last_change': '2024-04-25T10:29:08.009604+0000', 'last_active': '2024-04-25T10:29:07.995096+0000', 'last_peered': '2024-04-25T10:29:07.995096+0000', 'last_clean': '2024-04-25T10:29:07.995096+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-04-25T10:29:08.009604+0000', 'last_undegraded': '2024-04-25T10:29:08.009604+0000', 'last_fullsized': '2024-04-25T10:29:08.009604+0000', 'mapping_epoch': 21, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 21, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-04-25T10:29:07.995096+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-04-25T10:29:07.995096+0000', 'last_clean_scrub_stamp': '2024-04-25T10:29:07.995096+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [4, 7], 'acting': [4, 7], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 4, 'acting_primary': 4, 'purged_snaps': []} 2024-04-25T10:29:13.933 INFO:tasks.cephfs.filesystem.ceph_manager:PG 3.8 is not active+clean 2024-04-25T10:29:13.933 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '3.8', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 21, 'state': 'creating+peering', 'last_fresh': '2024-04-25T10:29:08.009188+0000', 'last_change': '2024-04-25T10:29:08.009188+0000', 'last_active': '2024-04-25T10:29:07.995096+0000', 'last_peered': '2024-04-25T10:29:07.995096+0000', 'last_clean': '2024-04-25T10:29:07.995096+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-04-25T10:29:08.009188+0000', 'last_undegraded': '2024-04-25T10:29:08.009188+0000', 'last_fullsized': '2024-04-25T10:29:08.009188+0000', 'mapping_epoch': 21, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 21, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-04-25T10:29:07.995096+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-04-25T10:29:07.995096+0000', 'last_clean_scrub_stamp': '2024-04-25T10:29:07.995096+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [4, 7], 'acting': [4, 7], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 4, 'acting_primary': 4, 'purged_snaps': []} 2024-04-25T10:29:13.934 DEBUG:teuthology.orchestra.run.smithi037:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph status --format=json 2024-04-25T10:29:14.079 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:14.077+0000 7f178f97a640 1 Processor -- start 2024-04-25T10:29:14.079 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:14.077+0000 7f178f97a640 1 -- start start 2024-04-25T10:29:14.081 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:14.078+0000 7f178f97a640 1 --2- >> v2:172.21.15.203:3300/0 conn(0x7f17880b50e0 0x7f17880b79d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:29:14.081 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:14.079+0000 7f178f97a640 1 --2- >> v2:172.21.15.37:3300/0 conn(0x7f17880b7f10 0x7f17880d0920 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:29:14.081 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:14.079+0000 7f178f97a640 1 -- --> v1:172.21.15.203:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f17880b8320 con 0x7f17880d0e60 2024-04-25T10:29:14.081 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:14.079+0000 7f178f97a640 1 -- --> v2:172.21.15.37:3300/0 -- mon_getmap magic: 0 -- 0x7f17881646e0 con 0x7f17880b7f10 2024-04-25T10:29:14.082 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:14.079+0000 7f178f97a640 1 -- --> v2:172.21.15.203:3300/0 -- mon_getmap magic: 0 -- 0x7f17880d15e0 con 0x7f17880b50e0 2024-04-25T10:29:14.082 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:14.079+0000 7f178cf15640 1 --2- >> v2:172.21.15.37:3300/0 conn(0x7f17880b7f10 0x7f17880d0920 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:29:14.082 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:14.079+0000 7f178cf15640 1 --2- >> v2:172.21.15.37:3300/0 conn(0x7f17880b7f10 0x7f17880d0920 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.37:3300/0 says I am v2:172.21.15.37:46328/0 (socket says 172.21.15.37:46328) 2024-04-25T10:29:14.082 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:14.079+0000 7f178cf15640 1 -- 172.21.15.37:0/1567802589 learned_addr learned my addr 172.21.15.37:0/1567802589 (peer_addr_for_me v2:172.21.15.37:0/0) 2024-04-25T10:29:14.082 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:14.079+0000 7f178cf15640 1 -- 172.21.15.37:0/1567802589 >> v2:172.21.15.203:3300/0 conn(0x7f17880b50e0 msgr2=0x7f17880b79d0 unknown :-1 s=STATE_CONNECTING_RE l=0).mark_down 2024-04-25T10:29:14.082 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:14.079+0000 7f178cf15640 1 --2- 172.21.15.37:0/1567802589 >> v2:172.21.15.203:3300/0 conn(0x7f17880b50e0 0x7f17880b79d0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:29:14.082 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:14.079+0000 7f178cf15640 1 -- 172.21.15.37:0/1567802589 >> v1:172.21.15.203:6789/0 conn(0x7f17880d0e60 legacy=0x7f178806dc00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:29:14.082 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:14.079+0000 7f178cf15640 1 -- 172.21.15.37:0/1567802589 --> v2:172.21.15.37:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f1788163db0 con 0x7f17880b7f10 2024-04-25T10:29:14.083 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:14.080+0000 7f178cf15640 1 --2- 172.21.15.37:0/1567802589 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f17880b7f10 0x7f17880d0920 secure :-1 s=READY pgs=85 cs=0 l=1 rev1=1 crypto rx=0x7f17880d00e0 tx=0x7f177802a250 comp rx=0 tx=0).ready entity=mon.0 client_cookie=1dceca53ef4fd882 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:29:14.083 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:14.080+0000 7f177ffff640 1 -- 172.21.15.37:0/1567802589 <== mon.0 v2:172.21.15.37:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1778037070 con 0x7f17880b7f10 2024-04-25T10:29:14.084 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:14.080+0000 7f177ffff640 1 -- 172.21.15.37:0/1567802589 <== mon.0 v2:172.21.15.37:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f1778032440 con 0x7f17880b7f10 2024-04-25T10:29:14.084 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:14.080+0000 7f177ffff640 1 -- 172.21.15.37:0/1567802589 <== mon.0 v2:172.21.15.37:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1778032810 con 0x7f17880b7f10 2024-04-25T10:29:14.084 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:14.081+0000 7f178f97a640 1 -- 172.21.15.37:0/1567802589 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f17880b7f10 msgr2=0x7f17880d0920 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:29:14.084 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:14.081+0000 7f178f97a640 1 --2- 172.21.15.37:0/1567802589 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f17880b7f10 0x7f17880d0920 secure :-1 s=READY pgs=85 cs=0 l=1 rev1=1 crypto rx=0x7f17880d00e0 tx=0x7f177802a250 comp rx=0 tx=0).stop 2024-04-25T10:29:14.084 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:14.081+0000 7f178f97a640 1 -- 172.21.15.37:0/1567802589 shutdown_connections 2024-04-25T10:29:14.084 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:14.081+0000 7f178f97a640 1 --2- 172.21.15.37:0/1567802589 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f17880b7f10 0x7f17880d0920 unknown :-1 s=CLOSED pgs=85 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:29:14.084 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:14.081+0000 7f178f97a640 1 --2- 172.21.15.37:0/1567802589 >> v2:172.21.15.203:3300/0 conn(0x7f17880b50e0 0x7f17880b79d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:29:14.084 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:14.081+0000 7f178f97a640 1 -- 172.21.15.37:0/1567802589 >> 172.21.15.37:0/1567802589 conn(0x7f17880d5d50 msgr2=0x7f17880d6160 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:29:14.084 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:14.081+0000 7f178f97a640 1 -- 172.21.15.37:0/1567802589 shutdown_connections 2024-04-25T10:29:14.084 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:14.081+0000 7f178f97a640 1 -- 172.21.15.37:0/1567802589 wait complete. 2024-04-25T10:29:14.085 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:14.082+0000 7f178f97a640 1 Processor -- start 2024-04-25T10:29:14.085 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:14.082+0000 7f178f97a640 1 -- start start 2024-04-25T10:29:14.087 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:14.084+0000 7f178f97a640 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f17880b50e0 0x7f17881f6700 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:29:14.088 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:14.084+0000 7f178f97a640 1 --2- >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f17880b7f10 0x7f17880cbc60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:29:14.088 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:14.084+0000 7f178f97a640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f17880d0e60 0x7f17880cc1a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:29:14.088 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:14.084+0000 7f178f97a640 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 -- 0x7f1788164b70 con 0x7f17880d0e60 2024-04-25T10:29:14.088 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:14.084+0000 7f178f97a640 1 -- --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_getmap magic: 0 -- 0x7f1788164370 con 0x7f17880b50e0 2024-04-25T10:29:14.088 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:14.084+0000 7f178f97a640 1 -- --> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] -- mon_getmap magic: 0 -- 0x7f17880d1270 con 0x7f17880b7f10 2024-04-25T10:29:14.088 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:14.084+0000 7f178df17640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f17880d0e60 0x7f17880cc1a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:29:14.088 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:14.084+0000 7f178df17640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f17880d0e60 0x7f17880cc1a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.37:3300/0 says I am v2:172.21.15.37:46330/0 (socket says 172.21.15.37:46330) 2024-04-25T10:29:14.088 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:14.084+0000 7f178df17640 1 -- 172.21.15.37:0/2432532674 learned_addr learned my addr 172.21.15.37:0/2432532674 (peer_addr_for_me v2:172.21.15.37:0/0) 2024-04-25T10:29:14.088 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:14.084+0000 7f178cf15640 1 --2- 172.21.15.37:0/2432532674 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f17880b7f10 0x7f17880cbc60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:29:14.088 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:14.084+0000 7f178d716640 1 --2- 172.21.15.37:0/2432532674 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f17880b50e0 0x7f17881f6700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:29:14.088 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:14.084+0000 7f178df17640 1 -- 172.21.15.37:0/2432532674 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f17880b7f10 msgr2=0x7f17880cbc60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:29:14.089 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:14.084+0000 7f178df17640 1 --2- 172.21.15.37:0/2432532674 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f17880b7f10 0x7f17880cbc60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:29:14.089 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:14.084+0000 7f178df17640 1 -- 172.21.15.37:0/2432532674 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f17880b50e0 msgr2=0x7f17881f6700 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:29:14.089 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:14.084+0000 7f178df17640 1 --2- 172.21.15.37:0/2432532674 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f17880b50e0 0x7f17881f6700 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:29:14.089 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:14.084+0000 7f178df17640 1 -- 172.21.15.37:0/2432532674 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f17880c9f20 con 0x7f17880d0e60 2024-04-25T10:29:14.089 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:14.084+0000 7f178df17640 1 --2- 172.21.15.37:0/2432532674 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f17880d0e60 0x7f17880cc1a0 secure :-1 s=READY pgs=86 cs=0 l=1 rev1=1 crypto rx=0x7f178000b9e0 tx=0x7f178000bea0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:29:14.089 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:14.085+0000 7f177dffb640 1 -- 172.21.15.37:0/2432532674 <== mon.0 v2:172.21.15.37:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1780003b70 con 0x7f17880d0e60 2024-04-25T10:29:14.089 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:14.085+0000 7f177dffb640 1 -- 172.21.15.37:0/2432532674 <== mon.0 v2:172.21.15.37:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f1780003d10 con 0x7f17880d0e60 2024-04-25T10:29:14.089 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:14.085+0000 7f178f97a640 1 -- 172.21.15.37:0/2432532674 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f17880ca200 con 0x7f17880d0e60 2024-04-25T10:29:14.089 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:14.085+0000 7f177dffb640 1 -- 172.21.15.37:0/2432532674 <== mon.0 v2:172.21.15.37:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f178000cb60 con 0x7f17880d0e60 2024-04-25T10:29:14.090 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:14.085+0000 7f178f97a640 1 -- 172.21.15.37:0/2432532674 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f17880ca7c0 con 0x7f17880d0e60 2024-04-25T10:29:14.091 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:14.086+0000 7f178f97a640 1 -- 172.21.15.37:0/2432532674 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f178806eee0 con 0x7f17880d0e60 2024-04-25T10:29:14.091 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:14.086+0000 7f177dffb640 1 -- 172.21.15.37:0/2432532674 <== mon.0 v2:172.21.15.37:3300/0 4 ==== mgrmap(e 5) ==== 96782+0+0 (secure 0 0 0) 0x7f178000cd00 con 0x7f17880d0e60 2024-04-25T10:29:14.091 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:14.087+0000 7f177dffb640 1 --2- 172.21.15.37:0/2432532674 >> [v2:172.21.15.203:6832/2418205237,v1:172.21.15.203:6833/2418205237] conn(0x7f176c0744b0 0x7f176c0769c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:29:14.091 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:14.087+0000 7f177dffb640 1 -- 172.21.15.37:0/2432532674 <== mon.0 v2:172.21.15.37:3300/0 5 ==== osd_map(23..23 src has 1..23) ==== 5709+0+0 (secure 0 0 0) 0x7f1780095070 con 0x7f17880d0e60 2024-04-25T10:29:14.091 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:14.087+0000 7f178d716640 1 --2- 172.21.15.37:0/2432532674 >> [v2:172.21.15.203:6832/2418205237,v1:172.21.15.203:6833/2418205237] conn(0x7f176c0744b0 0x7f176c0769c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:29:14.091 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:14.088+0000 7f178d716640 1 --2- 172.21.15.37:0/2432532674 >> [v2:172.21.15.203:6832/2418205237,v1:172.21.15.203:6833/2418205237] conn(0x7f176c0744b0 0x7f176c0769c0 secure :-1 s=READY pgs=67 cs=0 l=1 rev1=1 crypto rx=0x7f1784006fd0 tx=0x7f1784009060 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:29:14.094 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:14.092+0000 7f177dffb640 1 -- 172.21.15.37:0/2432532674 <== mon.0 v2:172.21.15.37:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187246 (secure 0 0 0) 0x7f178005f110 con 0x7f17880d0e60 2024-04-25T10:29:14.349 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:14.347+0000 7f178f97a640 1 -- 172.21.15.37:0/2432532674 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_command({"prefix": "status", "format": "json"} v 0) -- 0x7f178806a140 con 0x7f17880d0e60 2024-04-25T10:29:14.349 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:14.347+0000 7f177dffb640 1 -- 172.21.15.37:0/2432532674 <== mon.0 v2:172.21.15.37:3300/0 7 ==== mon_command_ack([{"prefix": "status", "format": "json"}]=0 v0) ==== 72+0+1097 (secure 0 0 0) 0x7f1780063410 con 0x7f17880d0e60 2024-04-25T10:29:14.349 INFO:teuthology.orchestra.run.smithi037.stdout: 2024-04-25T10:29:14.350 INFO:teuthology.orchestra.run.smithi037.stdout:{"fsid":"f4acda28-be34-492a-b286-994afd36f61d","health":{"status":"HEALTH_OK","checks":{},"mutes":[]},"election_epoch":4,"quorum":[0,1,2],"quorum_names":["a","b","c"],"quorum_age":30,"monmap":{"epoch":1,"min_mon_release_name":"squid","num_mons":3},"osdmap":{"epoch":23,"num_osds":8,"num_up_osds":8,"osd_up_since":1714040934,"num_in_osds":8,"osd_in_since":1714040926,"num_remapped_pgs":0},"pgmap":{"pgs_by_state":[{"state_name":"active+clean","count":90},{"state_name":"creating+peering","count":7}],"num_pgs":97,"num_pools":3,"num_objects":16,"data_bytes":592196,"bytes_used":224325632,"bytes_avail":772869787648,"bytes_total":773094113280,"inactive_pgs_ratio":0.072164945304393768,"write_bytes_sec":2564,"read_op_per_sec":0,"write_op_per_sec":4},"fsmap":{"epoch":8,"id":1,"up":1,"in":1,"max":1,"by_rank":[{"filesystem_id":1,"rank":0,"name":"d","status":"up:active","gid":4279}],"up:standby":3},"mgrmap":{"available":true,"num_standbys":1,"modules":["iostat","nfs","restful"],"services":{}},"servicemap":{"epoch":1,"modified":"2024-04-25T10:28:43.987552+0000","services":{}},"progress_events":{}} 2024-04-25T10:29:14.351 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:14.349+0000 7f178f97a640 1 -- 172.21.15.37:0/2432532674 >> [v2:172.21.15.203:6832/2418205237,v1:172.21.15.203:6833/2418205237] conn(0x7f176c0744b0 msgr2=0x7f176c0769c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:29:14.352 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:14.349+0000 7f178f97a640 1 --2- 172.21.15.37:0/2432532674 >> [v2:172.21.15.203:6832/2418205237,v1:172.21.15.203:6833/2418205237] conn(0x7f176c0744b0 0x7f176c0769c0 secure :-1 s=READY pgs=67 cs=0 l=1 rev1=1 crypto rx=0x7f1784006fd0 tx=0x7f1784009060 comp rx=0 tx=0).stop 2024-04-25T10:29:14.352 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:14.349+0000 7f178f97a640 1 -- 172.21.15.37:0/2432532674 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f17880d0e60 msgr2=0x7f17880cc1a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:29:14.352 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:14.349+0000 7f178f97a640 1 --2- 172.21.15.37:0/2432532674 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f17880d0e60 0x7f17880cc1a0 secure :-1 s=READY pgs=86 cs=0 l=1 rev1=1 crypto rx=0x7f178000b9e0 tx=0x7f178000bea0 comp rx=0 tx=0).stop 2024-04-25T10:29:14.352 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:14.349+0000 7f178f97a640 1 -- 172.21.15.37:0/2432532674 shutdown_connections 2024-04-25T10:29:14.352 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:14.349+0000 7f178f97a640 1 --2- 172.21.15.37:0/2432532674 >> [v2:172.21.15.203:6832/2418205237,v1:172.21.15.203:6833/2418205237] conn(0x7f176c0744b0 0x7f176c0769c0 unknown :-1 s=CLOSED pgs=67 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:29:14.352 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:14.349+0000 7f178f97a640 1 --2- 172.21.15.37:0/2432532674 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f17880d0e60 0x7f17880cc1a0 unknown :-1 s=CLOSED pgs=86 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:29:14.352 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:14.349+0000 7f178f97a640 1 --2- 172.21.15.37:0/2432532674 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f17880b7f10 0x7f17880cbc60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:29:14.352 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:14.349+0000 7f178f97a640 1 --2- 172.21.15.37:0/2432532674 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f17880b50e0 0x7f17881f6700 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:29:14.352 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:14.349+0000 7f178f97a640 1 -- 172.21.15.37:0/2432532674 >> 172.21.15.37:0/2432532674 conn(0x7f17880d5d50 msgr2=0x7f178815baa0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:29:14.352 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:14.349+0000 7f178f97a640 1 -- 172.21.15.37:0/2432532674 shutdown_connections 2024-04-25T10:29:14.352 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:14.349+0000 7f178f97a640 1 -- 172.21.15.37:0/2432532674 wait complete. 2024-04-25T10:29:14.362 INFO:tasks.cephfs.filesystem.ceph_manager:no progress seen, keeping timeout for now 2024-04-25T10:29:14.363 DEBUG:teuthology.orchestra.run.smithi037:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-04-25T10:29:14.473 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:14.471+0000 7f4256ea7640 1 Processor -- start 2024-04-25T10:29:14.473 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:14.471+0000 7f4256ea7640 1 -- start start 2024-04-25T10:29:14.476 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:14.472+0000 7f4256ea7640 1 --2- >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f42500d5160 0x7f42500b4e80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:29:14.476 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:14.472+0000 7f4256ea7640 1 --2- >> v2:172.21.15.37:3300/0 conn(0x7f42500d0990 0x7f42500d2b20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:29:14.476 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:14.472+0000 7f4256ea7640 1 -- --> v1:172.21.15.37:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f42500d1370 con 0x7f42500d0da0 2024-04-25T10:29:14.476 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:14.472+0000 7f4256ea7640 1 -- --> v2:172.21.15.37:3300/0 -- mon_getmap magic: 0 -- 0x7f42500d1650 con 0x7f42500d0990 2024-04-25T10:29:14.476 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:14.472+0000 7f4256ea7640 1 -- --> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] -- mon_getmap magic: 0 -- 0x7f42500d4680 con 0x7f42500d5160 2024-04-25T10:29:14.476 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:14.473+0000 7f424ffff640 1 --2- >> v2:172.21.15.37:3300/0 conn(0x7f42500d0990 0x7f42500d2b20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:29:14.477 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:14.473+0000 7f424ffff640 1 --2- >> v2:172.21.15.37:3300/0 conn(0x7f42500d0990 0x7f42500d2b20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.37:3300/0 says I am v2:172.21.15.37:46344/0 (socket says 172.21.15.37:46344) 2024-04-25T10:29:14.477 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:14.473+0000 7f4255444640 1 --1- >> v1:172.21.15.37:6789/0 conn(0x7f42500d0da0 0x7f425006db10 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.37:6789/0 says I am v1:172.21.15.37:51434/0 (socket says 172.21.15.37:51434) 2024-04-25T10:29:14.477 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:14.473+0000 7f424ffff640 1 -- 172.21.15.37:0/103361376 learned_addr learned my addr 172.21.15.37:0/103361376 (peer_addr_for_me v2:172.21.15.37:0/0) 2024-04-25T10:29:14.477 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:14.473+0000 7f4254c43640 1 --2- 172.21.15.37:0/103361376 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f42500d5160 0x7f42500b4e80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:29:14.477 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:14.473+0000 7f424ffff640 1 -- 172.21.15.37:0/103361376 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f42500d5160 msgr2=0x7f42500b4e80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:29:14.477 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:14.473+0000 7f424ffff640 1 --2- 172.21.15.37:0/103361376 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f42500d5160 0x7f42500b4e80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:29:14.477 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:14.473+0000 7f424ffff640 1 -- 172.21.15.37:0/103361376 >> v1:172.21.15.37:6789/0 conn(0x7f42500d0da0 legacy=0x7f425006db10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:29:14.477 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:14.473+0000 7f424f7fe640 1 -- 172.21.15.37:0/103361376 <== mon.0 v1:172.21.15.37:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 1609050635 0 0) 0x7f42500d1370 con 0x7f42500d0da0 2024-04-25T10:29:14.477 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:14.473+0000 7f424ffff640 1 -- 172.21.15.37:0/103361376 --> v2:172.21.15.37:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f42500d4960 con 0x7f42500d0990 2024-04-25T10:29:14.477 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:14.474+0000 7f424ffff640 1 --2- 172.21.15.37:0/103361376 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f42500d0990 0x7f42500d2b20 secure :-1 s=READY pgs=88 cs=0 l=1 rev1=1 crypto rx=0x7f4240009e50 tx=0x7f424002a5f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e924f921502fa7d2 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:29:14.477 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:14.474+0000 7f424f7fe640 1 -- 172.21.15.37:0/103361376 <== mon.0 v2:172.21.15.37:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4240036070 con 0x7f42500d0990 2024-04-25T10:29:14.477 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:14.474+0000 7f424f7fe640 1 -- 172.21.15.37:0/103361376 <== mon.0 v2:172.21.15.37:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f4240031600 con 0x7f42500d0990 2024-04-25T10:29:14.477 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:14.474+0000 7f424f7fe640 1 -- 172.21.15.37:0/103361376 <== mon.0 v2:172.21.15.37:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f42400319d0 con 0x7f42500d0990 2024-04-25T10:29:14.479 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:14.475+0000 7f4256ea7640 1 -- 172.21.15.37:0/103361376 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f42500d0990 msgr2=0x7f42500d2b20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:29:14.479 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:14.475+0000 7f4256ea7640 1 --2- 172.21.15.37:0/103361376 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f42500d0990 0x7f42500d2b20 secure :-1 s=READY pgs=88 cs=0 l=1 rev1=1 crypto rx=0x7f4240009e50 tx=0x7f424002a5f0 comp rx=0 tx=0).stop 2024-04-25T10:29:14.479 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:14.475+0000 7f4256ea7640 1 -- 172.21.15.37:0/103361376 shutdown_connections 2024-04-25T10:29:14.479 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:14.475+0000 7f4256ea7640 1 --2- 172.21.15.37:0/103361376 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f42500d0990 0x7f42500d2b20 unknown :-1 s=CLOSED pgs=88 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:29:14.479 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:14.475+0000 7f4256ea7640 1 --2- 172.21.15.37:0/103361376 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f42500d5160 0x7f42500b4e80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:29:14.479 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:14.475+0000 7f4256ea7640 1 -- 172.21.15.37:0/103361376 >> 172.21.15.37:0/103361376 conn(0x7f42500b7a80 msgr2=0x7f42500b7e90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:29:14.479 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:14.475+0000 7f4256ea7640 1 -- 172.21.15.37:0/103361376 shutdown_connections 2024-04-25T10:29:14.479 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:14.475+0000 7f4256ea7640 1 -- 172.21.15.37:0/103361376 wait complete. 2024-04-25T10:29:14.479 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:14.476+0000 7f4256ea7640 1 Processor -- start 2024-04-25T10:29:14.479 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:14.476+0000 7f4256ea7640 1 -- start start 2024-04-25T10:29:14.481 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:14.478+0000 7f4256ea7640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f42500d0990 0x7f4250159b90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:29:14.482 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:14.478+0000 7f4256ea7640 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f42500d0da0 0x7f42500d00f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:29:14.482 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:14.478+0000 7f4256ea7640 1 --2- >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f42500d5160 0x7f42500ba0d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:29:14.482 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:14.478+0000 7f4256ea7640 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 -- 0x7f4250164c40 con 0x7f42500d0990 2024-04-25T10:29:14.482 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:14.478+0000 7f4256ea7640 1 -- --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_getmap magic: 0 -- 0x7f4250164ac0 con 0x7f42500d0da0 2024-04-25T10:29:14.482 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:14.478+0000 7f4256ea7640 1 -- --> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] -- mon_getmap magic: 0 -- 0x7f42500d3060 con 0x7f42500d5160 2024-04-25T10:29:14.482 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:14.478+0000 7f4254c43640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f42500d0990 0x7f4250159b90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:29:14.482 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:14.478+0000 7f4254c43640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f42500d0990 0x7f4250159b90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.37:3300/0 says I am v2:172.21.15.37:46350/0 (socket says 172.21.15.37:46350) 2024-04-25T10:29:14.482 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:14.478+0000 7f4254c43640 1 -- 172.21.15.37:0/618258760 learned_addr learned my addr 172.21.15.37:0/618258760 (peer_addr_for_me v2:172.21.15.37:0/0) 2024-04-25T10:29:14.482 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:14.478+0000 7f424ffff640 1 --2- 172.21.15.37:0/618258760 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f42500d0da0 0x7f42500d00f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:29:14.482 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:14.478+0000 7f4255444640 1 --2- 172.21.15.37:0/618258760 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f42500d5160 0x7f42500ba0d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:29:14.482 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:14.478+0000 7f4254c43640 1 -- 172.21.15.37:0/618258760 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f42500d5160 msgr2=0x7f42500ba0d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:29:14.483 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:14.478+0000 7f4254c43640 1 --2- 172.21.15.37:0/618258760 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f42500d5160 0x7f42500ba0d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:29:14.483 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:14.478+0000 7f4254c43640 1 -- 172.21.15.37:0/618258760 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f42500d0da0 msgr2=0x7f42500d00f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:29:14.483 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:14.478+0000 7f4254c43640 1 --2- 172.21.15.37:0/618258760 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f42500d0da0 0x7f42500d00f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:29:14.483 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:14.478+0000 7f4254c43640 1 -- 172.21.15.37:0/618258760 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f425015a0d0 con 0x7f42500d0990 2024-04-25T10:29:14.483 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:14.478+0000 7f4254c43640 1 --2- 172.21.15.37:0/618258760 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f42500d0990 0x7f4250159b90 secure :-1 s=READY pgs=89 cs=0 l=1 rev1=1 crypto rx=0x7f424800e870 tx=0x7f424800ed30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:29:14.483 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:14.479+0000 7f424d7fa640 1 -- 172.21.15.37:0/618258760 <== mon.0 v2:172.21.15.37:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4248017070 con 0x7f42500d0990 2024-04-25T10:29:14.483 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:14.479+0000 7f424d7fa640 1 -- 172.21.15.37:0/618258760 <== mon.0 v2:172.21.15.37:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f424800c940 con 0x7f42500d0990 2024-04-25T10:29:14.483 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:14.479+0000 7f4256ea7640 1 -- 172.21.15.37:0/618258760 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f4250157620 con 0x7f42500d0990 2024-04-25T10:29:14.483 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:14.479+0000 7f424d7fa640 1 -- 172.21.15.37:0/618258760 <== mon.0 v2:172.21.15.37:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f424800cd30 con 0x7f42500d0990 2024-04-25T10:29:14.483 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:14.479+0000 7f4256ea7640 1 -- 172.21.15.37:0/618258760 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f4250157be0 con 0x7f42500d0990 2024-04-25T10:29:14.483 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:14.480+0000 7f4256ea7640 1 -- 172.21.15.37:0/618258760 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f42500d1370 con 0x7f42500d0990 2024-04-25T10:29:14.484 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:14.480+0000 7f424d7fa640 1 -- 172.21.15.37:0/618258760 <== mon.0 v2:172.21.15.37:3300/0 4 ==== mgrmap(e 5) ==== 96782+0+0 (secure 0 0 0) 0x7f424801a070 con 0x7f42500d0990 2024-04-25T10:29:14.484 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:14.481+0000 7f424d7fa640 1 --2- 172.21.15.37:0/618258760 >> [v2:172.21.15.203:6832/2418205237,v1:172.21.15.203:6833/2418205237] conn(0x7f42300744b0 0x7f42300769c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:29:14.484 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:14.481+0000 7f424d7fa640 1 -- 172.21.15.37:0/618258760 <== mon.0 v2:172.21.15.37:3300/0 5 ==== osd_map(23..23 src has 1..23) ==== 5709+0+0 (secure 0 0 0) 0x7f4248094fc0 con 0x7f42500d0990 2024-04-25T10:29:14.484 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:14.481+0000 7f424ffff640 1 --2- 172.21.15.37:0/618258760 >> [v2:172.21.15.203:6832/2418205237,v1:172.21.15.203:6833/2418205237] conn(0x7f42300744b0 0x7f42300769c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:29:14.484 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:14.482+0000 7f424ffff640 1 --2- 172.21.15.37:0/618258760 >> [v2:172.21.15.203:6832/2418205237,v1:172.21.15.203:6833/2418205237] conn(0x7f42300744b0 0x7f42300769c0 secure :-1 s=READY pgs=68 cs=0 l=1 rev1=1 crypto rx=0x7f4240033000 tx=0x7f4240009000 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:29:14.487 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:14.485+0000 7f424d7fa640 1 -- 172.21.15.37:0/618258760 <== mon.0 v2:172.21.15.37:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187246 (secure 0 0 0) 0x7f42500d1370 con 0x7f42500d0990 2024-04-25T10:29:14.668 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:14.666+0000 7f4256ea7640 1 -- 172.21.15.37:0/618258760 --> [v2:172.21.15.203:6832/2418205237,v1:172.21.15.203:6833/2418205237] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) -- 0x7f42501573d0 con 0x7f42300744b0 2024-04-25T10:29:14.675 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:14.673+0000 7f424d7fa640 1 -- 172.21.15.37:0/618258760 <== mgr.4100 v2:172.21.15.203:6832/2418205237 1 ==== mgr_command_reply(tid 0: 0 dumped all) ==== 18+0+251927 (secure 0 0 0) 0x7f42501573d0 con 0x7f42300744b0 2024-04-25T10:29:14.678 INFO:teuthology.orchestra.run.smithi037.stdout: 2024-04-25T10:29:14.679 INFO:teuthology.orchestra.run.smithi037.stderr:dumped all 2024-04-25T10:29:14.680 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:14.676+0000 7f4256ea7640 1 -- 172.21.15.37:0/618258760 >> [v2:172.21.15.203:6832/2418205237,v1:172.21.15.203:6833/2418205237] conn(0x7f42300744b0 msgr2=0x7f42300769c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:29:14.680 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:14.676+0000 7f4256ea7640 1 --2- 172.21.15.37:0/618258760 >> [v2:172.21.15.203:6832/2418205237,v1:172.21.15.203:6833/2418205237] conn(0x7f42300744b0 0x7f42300769c0 secure :-1 s=READY pgs=68 cs=0 l=1 rev1=1 crypto rx=0x7f4240033000 tx=0x7f4240009000 comp rx=0 tx=0).stop 2024-04-25T10:29:14.681 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:14.676+0000 7f4256ea7640 1 -- 172.21.15.37:0/618258760 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f42500d0990 msgr2=0x7f4250159b90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:29:14.681 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:14.676+0000 7f4256ea7640 1 --2- 172.21.15.37:0/618258760 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f42500d0990 0x7f4250159b90 secure :-1 s=READY pgs=89 cs=0 l=1 rev1=1 crypto rx=0x7f424800e870 tx=0x7f424800ed30 comp rx=0 tx=0).stop 2024-04-25T10:29:14.681 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:14.676+0000 7f4256ea7640 1 -- 172.21.15.37:0/618258760 shutdown_connections 2024-04-25T10:29:14.681 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:14.676+0000 7f4256ea7640 1 --2- 172.21.15.37:0/618258760 >> [v2:172.21.15.203:6832/2418205237,v1:172.21.15.203:6833/2418205237] conn(0x7f42300744b0 0x7f42300769c0 unknown :-1 s=CLOSED pgs=68 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:29:14.681 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:14.676+0000 7f4256ea7640 1 --2- 172.21.15.37:0/618258760 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f42500d5160 0x7f42500ba0d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:29:14.681 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:14.676+0000 7f4256ea7640 1 --2- 172.21.15.37:0/618258760 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f42500d0da0 0x7f42500d00f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:29:14.681 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:14.676+0000 7f4256ea7640 1 --2- 172.21.15.37:0/618258760 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f42500d0990 0x7f4250159b90 unknown :-1 s=CLOSED pgs=89 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:29:14.681 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:14.676+0000 7f4256ea7640 1 -- 172.21.15.37:0/618258760 >> 172.21.15.37:0/618258760 conn(0x7f42500b7a80 msgr2=0x7f42500b6dc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:29:14.681 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:14.676+0000 7f4256ea7640 1 -- 172.21.15.37:0/618258760 shutdown_connections 2024-04-25T10:29:14.681 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:14.676+0000 7f4256ea7640 1 -- 172.21.15.37:0/618258760 wait complete. 2024-04-25T10:29:14.689 INFO:teuthology.orchestra.run.smithi037.stdout:{"pg_ready":true,"pg_map":{"version":27,"stamp":"2024-04-25T10:29:14.187208+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":592778,"num_objects":24,"num_object_clones":0,"num_object_copies":48,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":24,"num_whiteouts":0,"num_read":66,"num_read_kb":55,"num_write":146,"num_write_kb":1642,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":14,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":111,"ondisk_log_size":111,"up":194,"acting":194,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":74,"num_osds":8,"num_per_pool_osds":8,"num_per_pool_omap_osds":8,"kb":754974720,"kb_used":219132,"kb_used_data":2964,"kb_used_omap":12,"kb_used_meta":216051,"kb_avail":754755588,"statfs":{"total":773094113280,"available":772869722112,"internally_reserved":0,"allocated":3035136,"data_stored":1791336,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":12752,"internal_metadata":221236784},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":31,"apply_latency_ms":31,"commit_latency_ns":31000000,"apply_latency_ns":31000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":2410,"num_objects":22,"num_object_clones":0,"num_object_copies":44,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":22,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":45,"num_write_kb":14,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":14,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"6.185214"},"pg_stats":[{"pgid":"3.3f","version":"0'0","reported_seq":12,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-04-25T10:29:10.037967+0000","last_change":"2024-04-25T10:29:09.024146+0000","last_active":"2024-04-25T10:29:10.037967+0000","last_peered":"2024-04-25T10:29:10.037967+0000","last_clean":"2024-04-25T10:29:10.037967+0000","last_became_active":"2024-04-25T10:29:09.023798+0000","last_became_peered":"2024-04-25T10:29:09.023798+0000","last_unstale":"2024-04-25T10:29:10.037967+0000","last_undegraded":"2024-04-25T10:29:10.037967+0000","last_fullsized":"2024-04-25T10:29:10.037967+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:29:07.995096+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:29:07.995096+0000","last_clean_scrub_stamp":"2024-04-25T10:29:07.995096+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T20:37:20.726766+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.3e","version":"0'0","reported_seq":12,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-04-25T10:29:10.189579+0000","last_change":"2024-04-25T10:29:09.028600+0000","last_active":"2024-04-25T10:29:10.189579+0000","last_peered":"2024-04-25T10:29:10.189579+0000","last_clean":"2024-04-25T10:29:10.189579+0000","last_became_active":"2024-04-25T10:29:09.028286+0000","last_became_peered":"2024-04-25T10:29:09.028286+0000","last_unstale":"2024-04-25T10:29:10.189579+0000","last_undegraded":"2024-04-25T10:29:10.189579+0000","last_fullsized":"2024-04-25T10:29:10.189579+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:29:07.995096+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:29:07.995096+0000","last_clean_scrub_stamp":"2024-04-25T10:29:07.995096+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T20:52:02.098872+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6],"acting":[5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.3d","version":"0'0","reported_seq":12,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-04-25T10:29:10.189318+0000","last_change":"2024-04-25T10:29:09.039030+0000","last_active":"2024-04-25T10:29:10.189318+0000","last_peered":"2024-04-25T10:29:10.189318+0000","last_clean":"2024-04-25T10:29:10.189318+0000","last_became_active":"2024-04-25T10:29:09.037692+0000","last_became_peered":"2024-04-25T10:29:09.037692+0000","last_unstale":"2024-04-25T10:29:10.189318+0000","last_undegraded":"2024-04-25T10:29:10.189318+0000","last_fullsized":"2024-04-25T10:29:10.189318+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:29:07.995096+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:29:07.995096+0000","last_clean_scrub_stamp":"2024-04-25T10:29:07.995096+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T15:51:52.989545+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.3c","version":"0'0","reported_seq":12,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-04-25T10:29:10.026169+0000","last_change":"2024-04-25T10:29:09.027520+0000","last_active":"2024-04-25T10:29:10.026169+0000","last_peered":"2024-04-25T10:29:10.026169+0000","last_clean":"2024-04-25T10:29:10.026169+0000","last_became_active":"2024-04-25T10:29:09.027075+0000","last_became_peered":"2024-04-25T10:29:09.027075+0000","last_unstale":"2024-04-25T10:29:10.026169+0000","last_undegraded":"2024-04-25T10:29:10.026169+0000","last_fullsized":"2024-04-25T10:29:10.026169+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:29:07.995096+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:29:07.995096+0000","last_clean_scrub_stamp":"2024-04-25T10:29:07.995096+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T10:55:01.505157+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4],"acting":[2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.3b","version":"0'0","reported_seq":12,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-04-25T10:29:10.037590+0000","last_change":"2024-04-25T10:29:09.040684+0000","last_active":"2024-04-25T10:29:10.037590+0000","last_peered":"2024-04-25T10:29:10.037590+0000","last_clean":"2024-04-25T10:29:10.037590+0000","last_became_active":"2024-04-25T10:29:09.040372+0000","last_became_peered":"2024-04-25T10:29:09.040372+0000","last_unstale":"2024-04-25T10:29:10.037590+0000","last_undegraded":"2024-04-25T10:29:10.037590+0000","last_fullsized":"2024-04-25T10:29:10.037590+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:29:07.995096+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:29:07.995096+0000","last_clean_scrub_stamp":"2024-04-25T10:29:07.995096+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T11:52:24.415332+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.3a","version":"0'0","reported_seq":12,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-04-25T10:29:10.025911+0000","last_change":"2024-04-25T10:29:09.027391+0000","last_active":"2024-04-25T10:29:10.025911+0000","last_peered":"2024-04-25T10:29:10.025911+0000","last_clean":"2024-04-25T10:29:10.025911+0000","last_became_active":"2024-04-25T10:29:09.027029+0000","last_became_peered":"2024-04-25T10:29:09.027029+0000","last_unstale":"2024-04-25T10:29:10.025911+0000","last_undegraded":"2024-04-25T10:29:10.025911+0000","last_fullsized":"2024-04-25T10:29:10.025911+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:29:07.995096+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:29:07.995096+0000","last_clean_scrub_stamp":"2024-04-25T10:29:07.995096+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T10:35:31.361514+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4],"acting":[2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.39","version":"0'0","reported_seq":12,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-04-25T10:29:10.037462+0000","last_change":"2024-04-25T10:29:09.029272+0000","last_active":"2024-04-25T10:29:10.037462+0000","last_peered":"2024-04-25T10:29:10.037462+0000","last_clean":"2024-04-25T10:29:10.037462+0000","last_became_active":"2024-04-25T10:29:09.028302+0000","last_became_peered":"2024-04-25T10:29:09.028302+0000","last_unstale":"2024-04-25T10:29:10.037462+0000","last_undegraded":"2024-04-25T10:29:10.037462+0000","last_fullsized":"2024-04-25T10:29:10.037462+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:29:07.995096+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:29:07.995096+0000","last_clean_scrub_stamp":"2024-04-25T10:29:07.995096+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T11:31:45.283247+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.38","version":"0'0","reported_seq":12,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-04-25T10:29:10.188973+0000","last_change":"2024-04-25T10:29:09.035089+0000","last_active":"2024-04-25T10:29:10.188973+0000","last_peered":"2024-04-25T10:29:10.188973+0000","last_clean":"2024-04-25T10:29:10.188973+0000","last_became_active":"2024-04-25T10:29:09.034324+0000","last_became_peered":"2024-04-25T10:29:09.034324+0000","last_unstale":"2024-04-25T10:29:10.188973+0000","last_undegraded":"2024-04-25T10:29:10.188973+0000","last_fullsized":"2024-04-25T10:29:10.188973+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:29:07.995096+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:29:07.995096+0000","last_clean_scrub_stamp":"2024-04-25T10:29:07.995096+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T21:39:39.743441+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.37","version":"0'0","reported_seq":12,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-04-25T10:29:10.187981+0000","last_change":"2024-04-25T10:29:09.016966+0000","last_active":"2024-04-25T10:29:10.187981+0000","last_peered":"2024-04-25T10:29:10.187981+0000","last_clean":"2024-04-25T10:29:10.187981+0000","last_became_active":"2024-04-25T10:29:09.016566+0000","last_became_peered":"2024-04-25T10:29:09.016566+0000","last_unstale":"2024-04-25T10:29:10.187981+0000","last_undegraded":"2024-04-25T10:29:10.187981+0000","last_fullsized":"2024-04-25T10:29:10.187981+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:29:07.995096+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:29:07.995096+0000","last_clean_scrub_stamp":"2024-04-25T10:29:07.995096+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T10:55:01.505157+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.36","version":"0'0","reported_seq":12,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-04-25T10:29:10.037810+0000","last_change":"2024-04-25T10:29:09.030321+0000","last_active":"2024-04-25T10:29:10.037810+0000","last_peered":"2024-04-25T10:29:10.037810+0000","last_clean":"2024-04-25T10:29:10.037810+0000","last_became_active":"2024-04-25T10:29:09.029976+0000","last_became_peered":"2024-04-25T10:29:09.029976+0000","last_unstale":"2024-04-25T10:29:10.037810+0000","last_undegraded":"2024-04-25T10:29:10.037810+0000","last_fullsized":"2024-04-25T10:29:10.037810+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:29:07.995096+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:29:07.995096+0000","last_clean_scrub_stamp":"2024-04-25T10:29:07.995096+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T12:36:31.191100+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.35","version":"0'0","reported_seq":12,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-04-25T10:29:10.037641+0000","last_change":"2024-04-25T10:29:09.030469+0000","last_active":"2024-04-25T10:29:10.037641+0000","last_peered":"2024-04-25T10:29:10.037641+0000","last_clean":"2024-04-25T10:29:10.037641+0000","last_became_active":"2024-04-25T10:29:09.029350+0000","last_became_peered":"2024-04-25T10:29:09.029350+0000","last_unstale":"2024-04-25T10:29:10.037641+0000","last_undegraded":"2024-04-25T10:29:10.037641+0000","last_fullsized":"2024-04-25T10:29:10.037641+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:29:07.995096+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:29:07.995096+0000","last_clean_scrub_stamp":"2024-04-25T10:29:07.995096+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T15:51:52.989545+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.34","version":"0'0","reported_seq":12,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-04-25T10:29:10.037728+0000","last_change":"2024-04-25T10:29:09.029772+0000","last_active":"2024-04-25T10:29:10.037728+0000","last_peered":"2024-04-25T10:29:10.037728+0000","last_clean":"2024-04-25T10:29:10.037728+0000","last_became_active":"2024-04-25T10:29:09.029350+0000","last_became_peered":"2024-04-25T10:29:09.029350+0000","last_unstale":"2024-04-25T10:29:10.037728+0000","last_undegraded":"2024-04-25T10:29:10.037728+0000","last_fullsized":"2024-04-25T10:29:10.037728+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:29:07.995096+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:29:07.995096+0000","last_clean_scrub_stamp":"2024-04-25T10:29:07.995096+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T19:14:48.737269+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.33","version":"0'0","reported_seq":12,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-04-25T10:29:10.188971+0000","last_change":"2024-04-25T10:29:09.038690+0000","last_active":"2024-04-25T10:29:10.188971+0000","last_peered":"2024-04-25T10:29:10.188971+0000","last_clean":"2024-04-25T10:29:10.188971+0000","last_became_active":"2024-04-25T10:29:09.038336+0000","last_became_peered":"2024-04-25T10:29:09.038336+0000","last_unstale":"2024-04-25T10:29:10.188971+0000","last_undegraded":"2024-04-25T10:29:10.188971+0000","last_fullsized":"2024-04-25T10:29:10.188971+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:29:07.995096+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:29:07.995096+0000","last_clean_scrub_stamp":"2024-04-25T10:29:07.995096+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T19:23:12.589414+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.32","version":"0'0","reported_seq":12,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-04-25T10:29:10.188641+0000","last_change":"2024-04-25T10:29:09.028195+0000","last_active":"2024-04-25T10:29:10.188641+0000","last_peered":"2024-04-25T10:29:10.188641+0000","last_clean":"2024-04-25T10:29:10.188641+0000","last_became_active":"2024-04-25T10:29:09.027893+0000","last_became_peered":"2024-04-25T10:29:09.027893+0000","last_unstale":"2024-04-25T10:29:10.188641+0000","last_undegraded":"2024-04-25T10:29:10.188641+0000","last_fullsized":"2024-04-25T10:29:10.188641+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:29:07.995096+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:29:07.995096+0000","last_clean_scrub_stamp":"2024-04-25T10:29:07.995096+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T11:35:27.466874+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.31","version":"0'0","reported_seq":12,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-04-25T10:29:10.188956+0000","last_change":"2024-04-25T10:29:09.034532+0000","last_active":"2024-04-25T10:29:10.188956+0000","last_peered":"2024-04-25T10:29:10.188956+0000","last_clean":"2024-04-25T10:29:10.188956+0000","last_became_active":"2024-04-25T10:29:09.034021+0000","last_became_peered":"2024-04-25T10:29:09.034021+0000","last_unstale":"2024-04-25T10:29:10.188956+0000","last_undegraded":"2024-04-25T10:29:10.188956+0000","last_fullsized":"2024-04-25T10:29:10.188956+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:29:07.995096+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:29:07.995096+0000","last_clean_scrub_stamp":"2024-04-25T10:29:07.995096+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T12:16:04.880732+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.30","version":"0'0","reported_seq":12,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-04-25T10:29:10.024134+0000","last_change":"2024-04-25T10:29:09.021224+0000","last_active":"2024-04-25T10:29:10.024134+0000","last_peered":"2024-04-25T10:29:10.024134+0000","last_clean":"2024-04-25T10:29:10.024134+0000","last_became_active":"2024-04-25T10:29:09.020964+0000","last_became_peered":"2024-04-25T10:29:09.020964+0000","last_unstale":"2024-04-25T10:29:10.024134+0000","last_undegraded":"2024-04-25T10:29:10.024134+0000","last_fullsized":"2024-04-25T10:29:10.024134+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:29:07.995096+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:29:07.995096+0000","last_clean_scrub_stamp":"2024-04-25T10:29:07.995096+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T18:44:23.610374+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.2f","version":"0'0","reported_seq":12,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-04-25T10:29:10.024944+0000","last_change":"2024-04-25T10:29:09.025638+0000","last_active":"2024-04-25T10:29:10.024944+0000","last_peered":"2024-04-25T10:29:10.024944+0000","last_clean":"2024-04-25T10:29:10.024944+0000","last_became_active":"2024-04-25T10:29:09.024517+0000","last_became_peered":"2024-04-25T10:29:09.024517+0000","last_unstale":"2024-04-25T10:29:10.024944+0000","last_undegraded":"2024-04-25T10:29:10.024944+0000","last_fullsized":"2024-04-25T10:29:10.024944+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:29:07.995096+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:29:07.995096+0000","last_clean_scrub_stamp":"2024-04-25T10:29:07.995096+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T16:29:08.975346+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.2e","version":"0'0","reported_seq":12,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-04-25T10:29:10.189006+0000","last_change":"2024-04-25T10:29:09.023923+0000","last_active":"2024-04-25T10:29:10.189006+0000","last_peered":"2024-04-25T10:29:10.189006+0000","last_clean":"2024-04-25T10:29:10.189006+0000","last_became_active":"2024-04-25T10:29:09.023579+0000","last_became_peered":"2024-04-25T10:29:09.023579+0000","last_unstale":"2024-04-25T10:29:10.189006+0000","last_undegraded":"2024-04-25T10:29:10.189006+0000","last_fullsized":"2024-04-25T10:29:10.189006+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:29:07.995096+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:29:07.995096+0000","last_clean_scrub_stamp":"2024-04-25T10:29:07.995096+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T10:35:31.361514+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.2d","version":"0'0","reported_seq":12,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-04-25T10:29:10.024852+0000","last_change":"2024-04-25T10:29:09.023039+0000","last_active":"2024-04-25T10:29:10.024852+0000","last_peered":"2024-04-25T10:29:10.024852+0000","last_clean":"2024-04-25T10:29:10.024852+0000","last_became_active":"2024-04-25T10:29:09.022706+0000","last_became_peered":"2024-04-25T10:29:09.022706+0000","last_unstale":"2024-04-25T10:29:10.024852+0000","last_undegraded":"2024-04-25T10:29:10.024852+0000","last_fullsized":"2024-04-25T10:29:10.024852+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:29:07.995096+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:29:07.995096+0000","last_clean_scrub_stamp":"2024-04-25T10:29:07.995096+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T16:48:17.352944+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.2c","version":"0'0","reported_seq":12,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-04-25T10:29:10.037891+0000","last_change":"2024-04-25T10:29:09.028979+0000","last_active":"2024-04-25T10:29:10.037891+0000","last_peered":"2024-04-25T10:29:10.037891+0000","last_clean":"2024-04-25T10:29:10.037891+0000","last_became_active":"2024-04-25T10:29:09.028061+0000","last_became_peered":"2024-04-25T10:29:09.028061+0000","last_unstale":"2024-04-25T10:29:10.037891+0000","last_undegraded":"2024-04-25T10:29:10.037891+0000","last_fullsized":"2024-04-25T10:29:10.037891+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:29:07.995096+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:29:07.995096+0000","last_clean_scrub_stamp":"2024-04-25T10:29:07.995096+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T18:44:23.610374+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.2b","version":"0'0","reported_seq":12,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-04-25T10:29:10.025554+0000","last_change":"2024-04-25T10:29:09.022874+0000","last_active":"2024-04-25T10:29:10.025554+0000","last_peered":"2024-04-25T10:29:10.025554+0000","last_clean":"2024-04-25T10:29:10.025554+0000","last_became_active":"2024-04-25T10:29:09.022360+0000","last_became_peered":"2024-04-25T10:29:09.022360+0000","last_unstale":"2024-04-25T10:29:10.025554+0000","last_undegraded":"2024-04-25T10:29:10.025554+0000","last_fullsized":"2024-04-25T10:29:10.025554+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:29:07.995096+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:29:07.995096+0000","last_clean_scrub_stamp":"2024-04-25T10:29:07.995096+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T19:32:59.816545+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.2a","version":"0'0","reported_seq":12,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-04-25T10:29:10.187695+0000","last_change":"2024-04-25T10:29:09.029965+0000","last_active":"2024-04-25T10:29:10.187695+0000","last_peered":"2024-04-25T10:29:10.187695+0000","last_clean":"2024-04-25T10:29:10.187695+0000","last_became_active":"2024-04-25T10:29:09.029616+0000","last_became_peered":"2024-04-25T10:29:09.029616+0000","last_unstale":"2024-04-25T10:29:10.187695+0000","last_undegraded":"2024-04-25T10:29:10.187695+0000","last_fullsized":"2024-04-25T10:29:10.187695+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:29:07.995096+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:29:07.995096+0000","last_clean_scrub_stamp":"2024-04-25T10:29:07.995096+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T15:58:36.848831+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.29","version":"0'0","reported_seq":12,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-04-25T10:29:10.189345+0000","last_change":"2024-04-25T10:29:09.033401+0000","last_active":"2024-04-25T10:29:10.189345+0000","last_peered":"2024-04-25T10:29:10.189345+0000","last_clean":"2024-04-25T10:29:10.189345+0000","last_became_active":"2024-04-25T10:29:09.032764+0000","last_became_peered":"2024-04-25T10:29:09.032764+0000","last_unstale":"2024-04-25T10:29:10.189345+0000","last_undegraded":"2024-04-25T10:29:10.189345+0000","last_fullsized":"2024-04-25T10:29:10.189345+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:29:07.995096+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:29:07.995096+0000","last_clean_scrub_stamp":"2024-04-25T10:29:07.995096+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T17:30:54.093151+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.28","version":"0'0","reported_seq":12,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-04-25T10:29:10.024133+0000","last_change":"2024-04-25T10:29:09.041063+0000","last_active":"2024-04-25T10:29:10.024133+0000","last_peered":"2024-04-25T10:29:10.024133+0000","last_clean":"2024-04-25T10:29:10.024133+0000","last_became_active":"2024-04-25T10:29:09.040734+0000","last_became_peered":"2024-04-25T10:29:09.040734+0000","last_unstale":"2024-04-25T10:29:10.024133+0000","last_undegraded":"2024-04-25T10:29:10.024133+0000","last_fullsized":"2024-04-25T10:29:10.024133+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:29:07.995096+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:29:07.995096+0000","last_clean_scrub_stamp":"2024-04-25T10:29:07.995096+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T13:54:22.968216+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.27","version":"0'0","reported_seq":12,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-04-25T10:29:10.189019+0000","last_change":"2024-04-25T10:29:09.034268+0000","last_active":"2024-04-25T10:29:10.189019+0000","last_peered":"2024-04-25T10:29:10.189019+0000","last_clean":"2024-04-25T10:29:10.189019+0000","last_became_active":"2024-04-25T10:29:09.033199+0000","last_became_peered":"2024-04-25T10:29:09.033199+0000","last_unstale":"2024-04-25T10:29:10.189019+0000","last_undegraded":"2024-04-25T10:29:10.189019+0000","last_fullsized":"2024-04-25T10:29:10.189019+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:29:07.995096+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:29:07.995096+0000","last_clean_scrub_stamp":"2024-04-25T10:29:07.995096+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T17:56:09.999494+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.26","version":"0'0","reported_seq":12,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-04-25T10:29:10.026183+0000","last_change":"2024-04-25T10:29:09.025643+0000","last_active":"2024-04-25T10:29:10.026183+0000","last_peered":"2024-04-25T10:29:10.026183+0000","last_clean":"2024-04-25T10:29:10.026183+0000","last_became_active":"2024-04-25T10:29:09.024894+0000","last_became_peered":"2024-04-25T10:29:09.024894+0000","last_unstale":"2024-04-25T10:29:10.026183+0000","last_undegraded":"2024-04-25T10:29:10.026183+0000","last_fullsized":"2024-04-25T10:29:10.026183+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:29:07.995096+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:29:07.995096+0000","last_clean_scrub_stamp":"2024-04-25T10:29:07.995096+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T22:08:00.367431+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.25","version":"0'0","reported_seq":12,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-04-25T10:29:10.189130+0000","last_change":"2024-04-25T10:29:09.023357+0000","last_active":"2024-04-25T10:29:10.189130+0000","last_peered":"2024-04-25T10:29:10.189130+0000","last_clean":"2024-04-25T10:29:10.189130+0000","last_became_active":"2024-04-25T10:29:09.022988+0000","last_became_peered":"2024-04-25T10:29:09.022988+0000","last_unstale":"2024-04-25T10:29:10.189130+0000","last_undegraded":"2024-04-25T10:29:10.189130+0000","last_fullsized":"2024-04-25T10:29:10.189130+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:29:07.995096+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:29:07.995096+0000","last_clean_scrub_stamp":"2024-04-25T10:29:07.995096+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T21:14:39.661957+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.24","version":"0'0","reported_seq":12,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-04-25T10:29:10.189239+0000","last_change":"2024-04-25T10:29:09.034328+0000","last_active":"2024-04-25T10:29:10.189239+0000","last_peered":"2024-04-25T10:29:10.189239+0000","last_clean":"2024-04-25T10:29:10.189239+0000","last_became_active":"2024-04-25T10:29:09.033978+0000","last_became_peered":"2024-04-25T10:29:09.033978+0000","last_unstale":"2024-04-25T10:29:10.189239+0000","last_undegraded":"2024-04-25T10:29:10.189239+0000","last_fullsized":"2024-04-25T10:29:10.189239+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:29:07.995096+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:29:07.995096+0000","last_clean_scrub_stamp":"2024-04-25T10:29:07.995096+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T21:24:14.497242+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.23","version":"0'0","reported_seq":12,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-04-25T10:29:10.189583+0000","last_change":"2024-04-25T10:29:09.034045+0000","last_active":"2024-04-25T10:29:10.189583+0000","last_peered":"2024-04-25T10:29:10.189583+0000","last_clean":"2024-04-25T10:29:10.189583+0000","last_became_active":"2024-04-25T10:29:09.033730+0000","last_became_peered":"2024-04-25T10:29:09.033730+0000","last_unstale":"2024-04-25T10:29:10.189583+0000","last_undegraded":"2024-04-25T10:29:10.189583+0000","last_fullsized":"2024-04-25T10:29:10.189583+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:29:07.995096+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:29:07.995096+0000","last_clean_scrub_stamp":"2024-04-25T10:29:07.995096+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T12:36:31.191100+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.22","version":"0'0","reported_seq":12,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-04-25T10:29:10.024663+0000","last_change":"2024-04-25T10:29:09.034262+0000","last_active":"2024-04-25T10:29:10.024663+0000","last_peered":"2024-04-25T10:29:10.024663+0000","last_clean":"2024-04-25T10:29:10.024663+0000","last_became_active":"2024-04-25T10:29:09.033905+0000","last_became_peered":"2024-04-25T10:29:09.033905+0000","last_unstale":"2024-04-25T10:29:10.024663+0000","last_undegraded":"2024-04-25T10:29:10.024663+0000","last_fullsized":"2024-04-25T10:29:10.024663+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:29:07.995096+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:29:07.995096+0000","last_clean_scrub_stamp":"2024-04-25T10:29:07.995096+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T16:22:45.608928+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.21","version":"0'0","reported_seq":12,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-04-25T10:29:10.188201+0000","last_change":"2024-04-25T10:29:09.027088+0000","last_active":"2024-04-25T10:29:10.188201+0000","last_peered":"2024-04-25T10:29:10.188201+0000","last_clean":"2024-04-25T10:29:10.188201+0000","last_became_active":"2024-04-25T10:29:09.026651+0000","last_became_peered":"2024-04-25T10:29:09.026651+0000","last_unstale":"2024-04-25T10:29:10.188201+0000","last_undegraded":"2024-04-25T10:29:10.188201+0000","last_fullsized":"2024-04-25T10:29:10.188201+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:29:07.995096+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:29:07.995096+0000","last_clean_scrub_stamp":"2024-04-25T10:29:07.995096+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T15:37:06.723380+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.20","version":"0'0","reported_seq":12,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-04-25T10:29:10.025741+0000","last_change":"2024-04-25T10:29:09.028008+0000","last_active":"2024-04-25T10:29:10.025741+0000","last_peered":"2024-04-25T10:29:10.025741+0000","last_clean":"2024-04-25T10:29:10.025741+0000","last_became_active":"2024-04-25T10:29:09.027679+0000","last_became_peered":"2024-04-25T10:29:09.027679+0000","last_unstale":"2024-04-25T10:29:10.025741+0000","last_undegraded":"2024-04-25T10:29:10.025741+0000","last_fullsized":"2024-04-25T10:29:10.025741+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:29:07.995096+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:29:07.995096+0000","last_clean_scrub_stamp":"2024-04-25T10:29:07.995096+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T15:37:06.723380+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,6],"acting":[2,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.1a","version":"0'0","reported_seq":12,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-04-25T10:29:10.187787+0000","last_change":"2024-04-25T10:29:09.032819+0000","last_active":"2024-04-25T10:29:10.187787+0000","last_peered":"2024-04-25T10:29:10.187787+0000","last_clean":"2024-04-25T10:29:10.187787+0000","last_became_active":"2024-04-25T10:29:09.032493+0000","last_became_peered":"2024-04-25T10:29:09.032493+0000","last_unstale":"2024-04-25T10:29:10.187787+0000","last_undegraded":"2024-04-25T10:29:10.187787+0000","last_fullsized":"2024-04-25T10:29:10.187787+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:29:07.995096+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:29:07.995096+0000","last_clean_scrub_stamp":"2024-04-25T10:29:07.995096+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T15:37:06.723380+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.1b","version":"22'1","reported_seq":15,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-04-25T10:29:10.189463+0000","last_change":"2024-04-25T10:29:08.025272+0000","last_active":"2024-04-25T10:29:10.189463+0000","last_peered":"2024-04-25T10:29:10.189463+0000","last_clean":"2024-04-25T10:29:10.189463+0000","last_became_active":"2024-04-25T10:29:08.024912+0000","last_became_peered":"2024-04-25T10:29:08.024912+0000","last_unstale":"2024-04-25T10:29:10.189463+0000","last_undegraded":"2024-04-25T10:29:10.189463+0000","last_fullsized":"2024-04-25T10:29:10.189463+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:29:06.989570+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:29:06.989570+0000","last_clean_scrub_stamp":"2024-04-25T10:29:06.989570+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T18:11:32.758179+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6],"acting":[5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.1b","version":"0'0","reported_seq":12,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-04-25T10:29:10.025596+0000","last_change":"2024-04-25T10:29:09.032005+0000","last_active":"2024-04-25T10:29:10.025596+0000","last_peered":"2024-04-25T10:29:10.025596+0000","last_clean":"2024-04-25T10:29:10.025596+0000","last_became_active":"2024-04-25T10:29:09.031572+0000","last_became_peered":"2024-04-25T10:29:09.031572+0000","last_unstale":"2024-04-25T10:29:10.025596+0000","last_undegraded":"2024-04-25T10:29:10.025596+0000","last_fullsized":"2024-04-25T10:29:10.025596+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:29:07.995096+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:29:07.995096+0000","last_clean_scrub_stamp":"2024-04-25T10:29:07.995096+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T12:47:09.477097+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.1a","version":"22'1","reported_seq":15,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-04-25T10:29:10.024506+0000","last_change":"2024-04-25T10:29:08.026018+0000","last_active":"2024-04-25T10:29:10.024506+0000","last_peered":"2024-04-25T10:29:10.024506+0000","last_clean":"2024-04-25T10:29:10.024506+0000","last_became_active":"2024-04-25T10:29:08.025668+0000","last_became_peered":"2024-04-25T10:29:08.025668+0000","last_unstale":"2024-04-25T10:29:10.024506+0000","last_undegraded":"2024-04-25T10:29:10.024506+0000","last_fullsized":"2024-04-25T10:29:10.024506+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:29:06.989570+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:29:06.989570+0000","last_clean_scrub_stamp":"2024-04-25T10:29:06.989570+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T11:35:26.461348+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.19","version":"0'0","reported_seq":14,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-04-25T10:29:10.188333+0000","last_change":"2024-04-25T10:29:08.011702+0000","last_active":"2024-04-25T10:29:10.188333+0000","last_peered":"2024-04-25T10:29:10.188333+0000","last_clean":"2024-04-25T10:29:10.188333+0000","last_became_active":"2024-04-25T10:29:08.011353+0000","last_became_peered":"2024-04-25T10:29:08.011353+0000","last_unstale":"2024-04-25T10:29:10.188333+0000","last_undegraded":"2024-04-25T10:29:10.188333+0000","last_fullsized":"2024-04-25T10:29:10.188333+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:29:06.989570+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:29:06.989570+0000","last_clean_scrub_stamp":"2024-04-25T10:29:06.989570+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T16:31:11.685101+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.18","version":"0'0","reported_seq":12,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-04-25T10:29:10.189830+0000","last_change":"2024-04-25T10:29:09.033872+0000","last_active":"2024-04-25T10:29:10.189830+0000","last_peered":"2024-04-25T10:29:10.189830+0000","last_clean":"2024-04-25T10:29:10.189830+0000","last_became_active":"2024-04-25T10:29:09.033544+0000","last_became_peered":"2024-04-25T10:29:09.033544+0000","last_unstale":"2024-04-25T10:29:10.189830+0000","last_undegraded":"2024-04-25T10:29:10.189830+0000","last_fullsized":"2024-04-25T10:29:10.189830+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:29:07.995096+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:29:07.995096+0000","last_clean_scrub_stamp":"2024-04-25T10:29:07.995096+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T13:54:22.968216+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6],"acting":[5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.18","version":"0'0","reported_seq":14,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-04-25T10:29:10.024423+0000","last_change":"2024-04-25T10:29:08.013340+0000","last_active":"2024-04-25T10:29:10.024423+0000","last_peered":"2024-04-25T10:29:10.024423+0000","last_clean":"2024-04-25T10:29:10.024423+0000","last_became_active":"2024-04-25T10:29:08.012940+0000","last_became_peered":"2024-04-25T10:29:08.012940+0000","last_unstale":"2024-04-25T10:29:10.024423+0000","last_undegraded":"2024-04-25T10:29:10.024423+0000","last_fullsized":"2024-04-25T10:29:10.024423+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:29:06.989570+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:29:06.989570+0000","last_clean_scrub_stamp":"2024-04-25T10:29:06.989570+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T10:35:30.355988+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.19","version":"0'0","reported_seq":12,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-04-25T10:29:10.025603+0000","last_change":"2024-04-25T10:29:09.021718+0000","last_active":"2024-04-25T10:29:10.025603+0000","last_peered":"2024-04-25T10:29:10.025603+0000","last_clean":"2024-04-25T10:29:10.025603+0000","last_became_active":"2024-04-25T10:29:09.021416+0000","last_became_peered":"2024-04-25T10:29:09.021416+0000","last_unstale":"2024-04-25T10:29:10.025603+0000","last_undegraded":"2024-04-25T10:29:10.025603+0000","last_fullsized":"2024-04-25T10:29:10.025603+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:29:07.995096+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:29:07.995096+0000","last_clean_scrub_stamp":"2024-04-25T10:29:07.995096+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T18:55:45.597763+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.17","version":"0'0","reported_seq":14,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-04-25T10:29:10.187949+0000","last_change":"2024-04-25T10:29:08.020281+0000","last_active":"2024-04-25T10:29:10.187949+0000","last_peered":"2024-04-25T10:29:10.187949+0000","last_clean":"2024-04-25T10:29:10.187949+0000","last_became_active":"2024-04-25T10:29:08.019921+0000","last_became_peered":"2024-04-25T10:29:08.019921+0000","last_unstale":"2024-04-25T10:29:10.187949+0000","last_undegraded":"2024-04-25T10:29:10.187949+0000","last_fullsized":"2024-04-25T10:29:10.187949+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:29:06.989570+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:29:06.989570+0000","last_clean_scrub_stamp":"2024-04-25T10:29:06.989570+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T19:13:07.068780+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.16","version":"0'0","reported_seq":12,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-04-25T10:29:10.026197+0000","last_change":"2024-04-25T10:29:09.022744+0000","last_active":"2024-04-25T10:29:10.026197+0000","last_peered":"2024-04-25T10:29:10.026197+0000","last_clean":"2024-04-25T10:29:10.026197+0000","last_became_active":"2024-04-25T10:29:09.022392+0000","last_became_peered":"2024-04-25T10:29:09.022392+0000","last_unstale":"2024-04-25T10:29:10.026197+0000","last_undegraded":"2024-04-25T10:29:10.026197+0000","last_fullsized":"2024-04-25T10:29:10.026197+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:29:07.995096+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:29:07.995096+0000","last_clean_scrub_stamp":"2024-04-25T10:29:07.995096+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T11:35:27.466874+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.16","version":"0'0","reported_seq":14,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-04-25T10:29:10.037869+0000","last_change":"2024-04-25T10:29:08.026388+0000","last_active":"2024-04-25T10:29:10.037869+0000","last_peered":"2024-04-25T10:29:10.037869+0000","last_clean":"2024-04-25T10:29:10.037869+0000","last_became_active":"2024-04-25T10:29:08.026039+0000","last_became_peered":"2024-04-25T10:29:08.026039+0000","last_unstale":"2024-04-25T10:29:10.037869+0000","last_undegraded":"2024-04-25T10:29:10.037869+0000","last_fullsized":"2024-04-25T10:29:10.037869+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:29:06.989570+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:29:06.989570+0000","last_clean_scrub_stamp":"2024-04-25T10:29:06.989570+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T18:11:32.758179+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.17","version":"0'0","reported_seq":12,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-04-25T10:29:10.189465+0000","last_change":"2024-04-25T10:29:09.032902+0000","last_active":"2024-04-25T10:29:10.189465+0000","last_peered":"2024-04-25T10:29:10.189465+0000","last_clean":"2024-04-25T10:29:10.189465+0000","last_became_active":"2024-04-25T10:29:09.032596+0000","last_became_peered":"2024-04-25T10:29:09.032596+0000","last_unstale":"2024-04-25T10:29:10.189465+0000","last_undegraded":"2024-04-25T10:29:10.189465+0000","last_fullsized":"2024-04-25T10:29:10.189465+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:29:07.995096+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:29:07.995096+0000","last_clean_scrub_stamp":"2024-04-25T10:29:07.995096+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T19:14:48.737269+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.15","version":"22'3","reported_seq":17,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-04-25T10:29:10.188352+0000","last_change":"2024-04-25T10:29:08.021347+0000","last_active":"2024-04-25T10:29:10.188352+0000","last_peered":"2024-04-25T10:29:10.188352+0000","last_clean":"2024-04-25T10:29:10.188352+0000","last_became_active":"2024-04-25T10:29:08.021039+0000","last_became_peered":"2024-04-25T10:29:08.021039+0000","last_unstale":"2024-04-25T10:29:10.188352+0000","last_undegraded":"2024-04-25T10:29:10.188352+0000","last_fullsized":"2024-04-25T10:29:10.188352+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:29:06.989570+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:29:06.989570+0000","last_clean_scrub_stamp":"2024-04-25T10:29:06.989570+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T20:22:56.380539+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.14","version":"0'0","reported_seq":12,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-04-25T10:29:10.024492+0000","last_change":"2024-04-25T10:29:09.021410+0000","last_active":"2024-04-25T10:29:10.024492+0000","last_peered":"2024-04-25T10:29:10.024492+0000","last_clean":"2024-04-25T10:29:10.024492+0000","last_became_active":"2024-04-25T10:29:09.021051+0000","last_became_peered":"2024-04-25T10:29:09.021051+0000","last_unstale":"2024-04-25T10:29:10.024492+0000","last_undegraded":"2024-04-25T10:29:10.024492+0000","last_fullsized":"2024-04-25T10:29:10.024492+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:29:07.995096+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:29:07.995096+0000","last_clean_scrub_stamp":"2024-04-25T10:29:07.995096+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T13:22:13.080513+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.14","version":"22'6","reported_seq":20,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-04-25T10:29:10.025912+0000","last_change":"2024-04-25T10:29:08.018044+0000","last_active":"2024-04-25T10:29:10.025912+0000","last_peered":"2024-04-25T10:29:10.025912+0000","last_clean":"2024-04-25T10:29:10.025912+0000","last_became_active":"2024-04-25T10:29:08.016984+0000","last_became_peered":"2024-04-25T10:29:08.016984+0000","last_unstale":"2024-04-25T10:29:10.025912+0000","last_undegraded":"2024-04-25T10:29:10.025912+0000","last_fullsized":"2024-04-25T10:29:10.025912+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:29:06.989570+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:29:06.989570+0000","last_clean_scrub_stamp":"2024-04-25T10:29:06.989570+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T18:29:56.525236+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1612,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4],"acting":[2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.15","version":"0'0","reported_seq":12,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-04-25T10:29:10.037668+0000","last_change":"2024-04-25T10:29:09.029034+0000","last_active":"2024-04-25T10:29:10.037668+0000","last_peered":"2024-04-25T10:29:10.037668+0000","last_clean":"2024-04-25T10:29:10.037668+0000","last_became_active":"2024-04-25T10:29:09.028662+0000","last_became_peered":"2024-04-25T10:29:09.028662+0000","last_unstale":"2024-04-25T10:29:10.037668+0000","last_undegraded":"2024-04-25T10:29:10.037668+0000","last_fullsized":"2024-04-25T10:29:10.037668+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:29:07.995096+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:29:07.995096+0000","last_clean_scrub_stamp":"2024-04-25T10:29:07.995096+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T17:30:54.093151+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.13","version":"22'2","reported_seq":16,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-04-25T10:29:10.189430+0000","last_change":"2024-04-25T10:29:08.020476+0000","last_active":"2024-04-25T10:29:10.189430+0000","last_peered":"2024-04-25T10:29:10.189430+0000","last_clean":"2024-04-25T10:29:10.189430+0000","last_became_active":"2024-04-25T10:29:08.020096+0000","last_became_peered":"2024-04-25T10:29:08.020096+0000","last_unstale":"2024-04-25T10:29:10.189430+0000","last_undegraded":"2024-04-25T10:29:10.189430+0000","last_fullsized":"2024-04-25T10:29:10.189430+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:29:06.989570+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:29:06.989570+0000","last_clean_scrub_stamp":"2024-04-25T10:29:06.989570+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T10:32:06.103123+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":34,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":7,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.12","version":"0'0","reported_seq":12,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-04-25T10:29:10.188838+0000","last_change":"2024-04-25T10:29:09.027996+0000","last_active":"2024-04-25T10:29:10.188838+0000","last_peered":"2024-04-25T10:29:10.188838+0000","last_clean":"2024-04-25T10:29:10.188838+0000","last_became_active":"2024-04-25T10:29:09.027538+0000","last_became_peered":"2024-04-25T10:29:09.027538+0000","last_unstale":"2024-04-25T10:29:10.188838+0000","last_undegraded":"2024-04-25T10:29:10.188838+0000","last_fullsized":"2024-04-25T10:29:10.188838+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:29:07.995096+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:29:07.995096+0000","last_clean_scrub_stamp":"2024-04-25T10:29:07.995096+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T12:16:47.983013+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.12","version":"0'0","reported_seq":14,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-04-25T10:29:10.024603+0000","last_change":"2024-04-25T10:29:08.019028+0000","last_active":"2024-04-25T10:29:10.024603+0000","last_peered":"2024-04-25T10:29:10.024603+0000","last_clean":"2024-04-25T10:29:10.024603+0000","last_became_active":"2024-04-25T10:29:08.018582+0000","last_became_peered":"2024-04-25T10:29:08.018582+0000","last_unstale":"2024-04-25T10:29:10.024603+0000","last_undegraded":"2024-04-25T10:29:10.024603+0000","last_fullsized":"2024-04-25T10:29:10.024603+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:29:06.989570+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:29:06.989570+0000","last_clean_scrub_stamp":"2024-04-25T10:29:06.989570+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T19:23:11.583888+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.13","version":"0'0","reported_seq":12,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-04-25T10:29:10.024281+0000","last_change":"2024-04-25T10:29:09.023995+0000","last_active":"2024-04-25T10:29:10.024281+0000","last_peered":"2024-04-25T10:29:10.024281+0000","last_clean":"2024-04-25T10:29:10.024281+0000","last_became_active":"2024-04-25T10:29:09.023690+0000","last_became_peered":"2024-04-25T10:29:09.023690+0000","last_unstale":"2024-04-25T10:29:10.024281+0000","last_undegraded":"2024-04-25T10:29:10.024281+0000","last_fullsized":"2024-04-25T10:29:10.024281+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:29:07.995096+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:29:07.995096+0000","last_clean_scrub_stamp":"2024-04-25T10:29:07.995096+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T15:28:23.689564+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.11","version":"0'0","reported_seq":14,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-04-25T10:29:10.037576+0000","last_change":"2024-04-25T10:29:08.027174+0000","last_active":"2024-04-25T10:29:10.037576+0000","last_peered":"2024-04-25T10:29:10.037576+0000","last_clean":"2024-04-25T10:29:10.037576+0000","last_became_active":"2024-04-25T10:29:08.026809+0000","last_became_peered":"2024-04-25T10:29:08.026809+0000","last_unstale":"2024-04-25T10:29:10.037576+0000","last_undegraded":"2024-04-25T10:29:10.037576+0000","last_fullsized":"2024-04-25T10:29:10.037576+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:29:06.989570+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:29:06.989570+0000","last_clean_scrub_stamp":"2024-04-25T10:29:06.989570+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T10:35:30.355988+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.10","version":"0'0","reported_seq":12,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-04-25T10:29:10.187809+0000","last_change":"2024-04-25T10:29:09.041722+0000","last_active":"2024-04-25T10:29:10.187809+0000","last_peered":"2024-04-25T10:29:10.187809+0000","last_clean":"2024-04-25T10:29:10.187809+0000","last_became_active":"2024-04-25T10:29:09.040971+0000","last_became_peered":"2024-04-25T10:29:09.040971+0000","last_unstale":"2024-04-25T10:29:10.187809+0000","last_undegraded":"2024-04-25T10:29:10.187809+0000","last_fullsized":"2024-04-25T10:29:10.187809+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:29:07.995096+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:29:07.995096+0000","last_clean_scrub_stamp":"2024-04-25T10:29:07.995096+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T10:35:31.361514+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.10","version":"22'8","reported_seq":22,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-04-25T10:29:10.025277+0000","last_change":"2024-04-25T10:29:08.014110+0000","last_active":"2024-04-25T10:29:10.025277+0000","last_peered":"2024-04-25T10:29:10.025277+0000","last_clean":"2024-04-25T10:29:10.025277+0000","last_became_active":"2024-04-25T10:29:08.013543+0000","last_became_peered":"2024-04-25T10:29:08.013543+0000","last_unstale":"2024-04-25T10:29:10.025277+0000","last_undegraded":"2024-04-25T10:29:10.025277+0000","last_fullsized":"2024-04-25T10:29:10.025277+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:29:06.989570+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:29:06.989570+0000","last_clean_scrub_stamp":"2024-04-25T10:29:06.989570+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T16:44:30.899898+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":12,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7],"acting":[4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.11","version":"0'0","reported_seq":12,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-04-25T10:29:10.024238+0000","last_change":"2024-04-25T10:29:09.020490+0000","last_active":"2024-04-25T10:29:10.024238+0000","last_peered":"2024-04-25T10:29:10.024238+0000","last_clean":"2024-04-25T10:29:10.024238+0000","last_became_active":"2024-04-25T10:29:09.020178+0000","last_became_peered":"2024-04-25T10:29:09.020178+0000","last_unstale":"2024-04-25T10:29:10.024238+0000","last_undegraded":"2024-04-25T10:29:10.024238+0000","last_fullsized":"2024-04-25T10:29:10.024238+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:29:07.995096+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:29:07.995096+0000","last_clean_scrub_stamp":"2024-04-25T10:29:07.995096+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T20:37:20.726766+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.f","version":"22'1","reported_seq":15,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-04-25T10:29:10.038104+0000","last_change":"2024-04-25T10:29:08.029879+0000","last_active":"2024-04-25T10:29:10.038104+0000","last_peered":"2024-04-25T10:29:10.038104+0000","last_clean":"2024-04-25T10:29:10.038104+0000","last_became_active":"2024-04-25T10:29:08.029549+0000","last_became_peered":"2024-04-25T10:29:08.029549+0000","last_unstale":"2024-04-25T10:29:10.038104+0000","last_undegraded":"2024-04-25T10:29:10.038104+0000","last_fullsized":"2024-04-25T10:29:10.038104+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:29:06.989570+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:29:06.989570+0000","last_clean_scrub_stamp":"2024-04-25T10:29:06.989570+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T15:37:05.717854+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.e","version":"0'0","reported_seq":12,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-04-25T10:29:10.026273+0000","last_change":"2024-04-25T10:29:09.033703+0000","last_active":"2024-04-25T10:29:10.026273+0000","last_peered":"2024-04-25T10:29:10.026273+0000","last_clean":"2024-04-25T10:29:10.026273+0000","last_became_active":"2024-04-25T10:29:09.033318+0000","last_became_peered":"2024-04-25T10:29:09.033318+0000","last_unstale":"2024-04-25T10:29:10.026273+0000","last_undegraded":"2024-04-25T10:29:10.026273+0000","last_fullsized":"2024-04-25T10:29:10.026273+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:29:07.995096+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:29:07.995096+0000","last_clean_scrub_stamp":"2024-04-25T10:29:07.995096+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T12:16:47.983013+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.e","version":"22'2","reported_seq":16,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-04-25T10:29:10.026042+0000","last_change":"2024-04-25T10:29:08.020703+0000","last_active":"2024-04-25T10:29:10.026042+0000","last_peered":"2024-04-25T10:29:10.026042+0000","last_clean":"2024-04-25T10:29:10.026042+0000","last_became_active":"2024-04-25T10:29:08.020321+0000","last_became_peered":"2024-04-25T10:29:08.020321+0000","last_unstale":"2024-04-25T10:29:10.026042+0000","last_undegraded":"2024-04-25T10:29:10.026042+0000","last_fullsized":"2024-04-25T10:29:10.026042+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:29:06.989570+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:29:06.989570+0000","last_clean_scrub_stamp":"2024-04-25T10:29:06.989570+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T15:42:07.059913+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5],"acting":[4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.f","version":"0'0","reported_seq":12,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-04-25T10:29:10.038152+0000","last_change":"2024-04-25T10:29:09.022708+0000","last_active":"2024-04-25T10:29:10.038152+0000","last_peered":"2024-04-25T10:29:10.038152+0000","last_clean":"2024-04-25T10:29:10.038152+0000","last_became_active":"2024-04-25T10:29:09.020856+0000","last_became_peered":"2024-04-25T10:29:09.020856+0000","last_unstale":"2024-04-25T10:29:10.038152+0000","last_undegraded":"2024-04-25T10:29:10.038152+0000","last_fullsized":"2024-04-25T10:29:10.038152+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:29:07.995096+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:29:07.995096+0000","last_clean_scrub_stamp":"2024-04-25T10:29:07.995096+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T15:28:23.689564+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.d","version":"22'1","reported_seq":15,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-04-25T10:29:10.189469+0000","last_change":"2024-04-25T10:29:08.019488+0000","last_active":"2024-04-25T10:29:10.189469+0000","last_peered":"2024-04-25T10:29:10.189469+0000","last_clean":"2024-04-25T10:29:10.189469+0000","last_became_active":"2024-04-25T10:29:08.018971+0000","last_became_peered":"2024-04-25T10:29:08.018971+0000","last_unstale":"2024-04-25T10:29:10.189469+0000","last_undegraded":"2024-04-25T10:29:10.189469+0000","last_fullsized":"2024-04-25T10:29:10.189469+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:29:06.989570+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:29:06.989570+0000","last_clean_scrub_stamp":"2024-04-25T10:29:06.989570+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T20:22:56.380539+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":70,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.c","version":"0'0","reported_seq":12,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-04-25T10:29:10.037902+0000","last_change":"2024-04-25T10:29:09.029666+0000","last_active":"2024-04-25T10:29:10.037902+0000","last_peered":"2024-04-25T10:29:10.037902+0000","last_clean":"2024-04-25T10:29:10.037902+0000","last_became_active":"2024-04-25T10:29:09.029184+0000","last_became_peered":"2024-04-25T10:29:09.029184+0000","last_unstale":"2024-04-25T10:29:10.037902+0000","last_undegraded":"2024-04-25T10:29:10.037902+0000","last_fullsized":"2024-04-25T10:29:10.037902+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:29:07.995096+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:29:07.995096+0000","last_clean_scrub_stamp":"2024-04-25T10:29:07.995096+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T13:54:22.968216+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.c","version":"0'0","reported_seq":14,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-04-25T10:29:10.189541+0000","last_change":"2024-04-25T10:29:08.012124+0000","last_active":"2024-04-25T10:29:10.189541+0000","last_peered":"2024-04-25T10:29:10.189541+0000","last_clean":"2024-04-25T10:29:10.189541+0000","last_became_active":"2024-04-25T10:29:08.011817+0000","last_became_peered":"2024-04-25T10:29:08.011817+0000","last_unstale":"2024-04-25T10:29:10.189541+0000","last_undegraded":"2024-04-25T10:29:10.189541+0000","last_fullsized":"2024-04-25T10:29:10.189541+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:29:06.989570+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:29:06.989570+0000","last_clean_scrub_stamp":"2024-04-25T10:29:06.989570+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T20:22:56.380539+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.d","version":"0'0","reported_seq":12,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-04-25T10:29:10.024815+0000","last_change":"2024-04-25T10:29:09.017227+0000","last_active":"2024-04-25T10:29:10.024815+0000","last_peered":"2024-04-25T10:29:10.024815+0000","last_clean":"2024-04-25T10:29:10.024815+0000","last_became_active":"2024-04-25T10:29:09.016911+0000","last_became_peered":"2024-04-25T10:29:09.016911+0000","last_unstale":"2024-04-25T10:29:10.024815+0000","last_undegraded":"2024-04-25T10:29:10.024815+0000","last_fullsized":"2024-04-25T10:29:10.024815+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:29:07.995096+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:29:07.995096+0000","last_clean_scrub_stamp":"2024-04-25T10:29:07.995096+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T18:51:41.934269+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.0","version":"18'72","reported_seq":110,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-04-25T10:29:10.187517+0000","last_change":"2024-04-25T10:28:57.972696+0000","last_active":"2024-04-25T10:29:10.187517+0000","last_peered":"2024-04-25T10:29:10.187517+0000","last_clean":"2024-04-25T10:29:10.187517+0000","last_became_active":"2024-04-25T10:28:57.972373+0000","last_became_peered":"2024-04-25T10:28:57.972373+0000","last_unstale":"2024-04-25T10:29:10.187517+0000","last_undegraded":"2024-04-25T10:29:10.187517+0000","last_fullsized":"2024-04-25T10:29:10.187517+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:28:56.959439+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:28:56.959439+0000","last_clean_scrub_stamp":"2024-04-25T10:28:56.959439+0000","objects_scrubbed":0,"log_size":72,"log_dups_size":0,"ondisk_log_size":72,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T17:34:34.057408+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":66,"num_read_kb":55,"num_write":101,"num_write_kb":1628,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.3","version":"0'0","reported_seq":14,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-04-25T10:29:10.024562+0000","last_change":"2024-04-25T10:29:08.025080+0000","last_active":"2024-04-25T10:29:10.024562+0000","last_peered":"2024-04-25T10:29:10.024562+0000","last_clean":"2024-04-25T10:29:10.024562+0000","last_became_active":"2024-04-25T10:29:08.024747+0000","last_became_peered":"2024-04-25T10:29:08.024747+0000","last_unstale":"2024-04-25T10:29:10.024562+0000","last_undegraded":"2024-04-25T10:29:10.024562+0000","last_fullsized":"2024-04-25T10:29:10.024562+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:29:06.989570+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:29:06.989570+0000","last_clean_scrub_stamp":"2024-04-25T10:29:06.989570+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T22:07:59.361905+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.2","version":"0'0","reported_seq":12,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-04-25T10:29:10.025717+0000","last_change":"2024-04-25T10:29:09.023626+0000","last_active":"2024-04-25T10:29:10.025717+0000","last_peered":"2024-04-25T10:29:10.025717+0000","last_clean":"2024-04-25T10:29:10.025717+0000","last_became_active":"2024-04-25T10:29:09.020335+0000","last_became_peered":"2024-04-25T10:29:09.020335+0000","last_unstale":"2024-04-25T10:29:10.025717+0000","last_undegraded":"2024-04-25T10:29:10.025717+0000","last_fullsized":"2024-04-25T10:29:10.025717+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:29:07.995096+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:29:07.995096+0000","last_clean_scrub_stamp":"2024-04-25T10:29:07.995096+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T15:30:10.628413+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.0","version":"0'0","reported_seq":14,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-04-25T10:29:10.024309+0000","last_change":"2024-04-25T10:29:08.027479+0000","last_active":"2024-04-25T10:29:10.024309+0000","last_peered":"2024-04-25T10:29:10.024309+0000","last_clean":"2024-04-25T10:29:10.024309+0000","last_became_active":"2024-04-25T10:29:08.027178+0000","last_became_peered":"2024-04-25T10:29:08.027178+0000","last_unstale":"2024-04-25T10:29:10.024309+0000","last_undegraded":"2024-04-25T10:29:10.024309+0000","last_fullsized":"2024-04-25T10:29:10.024309+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:29:06.989570+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:29:06.989570+0000","last_clean_scrub_stamp":"2024-04-25T10:29:06.989570+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T19:54:16.697792+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.1","version":"0'0","reported_seq":12,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-04-25T10:29:10.188195+0000","last_change":"2024-04-25T10:29:09.028777+0000","last_active":"2024-04-25T10:29:10.188195+0000","last_peered":"2024-04-25T10:29:10.188195+0000","last_clean":"2024-04-25T10:29:10.188195+0000","last_became_active":"2024-04-25T10:29:09.027709+0000","last_became_peered":"2024-04-25T10:29:09.027709+0000","last_unstale":"2024-04-25T10:29:10.188195+0000","last_undegraded":"2024-04-25T10:29:10.188195+0000","last_fullsized":"2024-04-25T10:29:10.188195+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:29:07.995096+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:29:07.995096+0000","last_clean_scrub_stamp":"2024-04-25T10:29:07.995096+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T22:08:00.367431+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,2],"acting":[6,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.1","version":"0'0","reported_seq":14,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-04-25T10:29:10.188429+0000","last_change":"2024-04-25T10:29:08.020944+0000","last_active":"2024-04-25T10:29:10.188429+0000","last_peered":"2024-04-25T10:29:10.188429+0000","last_clean":"2024-04-25T10:29:10.188429+0000","last_became_active":"2024-04-25T10:29:08.020259+0000","last_became_peered":"2024-04-25T10:29:08.020259+0000","last_unstale":"2024-04-25T10:29:10.188429+0000","last_undegraded":"2024-04-25T10:29:10.188429+0000","last_fullsized":"2024-04-25T10:29:10.188429+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:29:06.989570+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:29:06.989570+0000","last_clean_scrub_stamp":"2024-04-25T10:29:06.989570+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T16:31:11.685101+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.0","version":"0'0","reported_seq":12,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-04-25T10:29:10.189305+0000","last_change":"2024-04-25T10:29:09.035455+0000","last_active":"2024-04-25T10:29:10.189305+0000","last_peered":"2024-04-25T10:29:10.189305+0000","last_clean":"2024-04-25T10:29:10.189305+0000","last_became_active":"2024-04-25T10:29:09.035130+0000","last_became_peered":"2024-04-25T10:29:09.035130+0000","last_unstale":"2024-04-25T10:29:10.189305+0000","last_undegraded":"2024-04-25T10:29:10.189305+0000","last_fullsized":"2024-04-25T10:29:10.189305+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:29:07.995096+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:29:07.995096+0000","last_clean_scrub_stamp":"2024-04-25T10:29:07.995096+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T11:52:24.415332+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.2","version":"0'0","reported_seq":14,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-04-25T10:29:10.024714+0000","last_change":"2024-04-25T10:29:08.021694+0000","last_active":"2024-04-25T10:29:10.024714+0000","last_peered":"2024-04-25T10:29:10.024714+0000","last_clean":"2024-04-25T10:29:10.024714+0000","last_became_active":"2024-04-25T10:29:08.021052+0000","last_became_peered":"2024-04-25T10:29:08.021052+0000","last_unstale":"2024-04-25T10:29:10.024714+0000","last_undegraded":"2024-04-25T10:29:10.024714+0000","last_fullsized":"2024-04-25T10:29:10.024714+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:29:06.989570+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:29:06.989570+0000","last_clean_scrub_stamp":"2024-04-25T10:29:06.989570+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T21:14:38.656431+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.3","version":"0'0","reported_seq":12,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-04-25T10:29:10.037784+0000","last_change":"2024-04-25T10:29:09.021686+0000","last_active":"2024-04-25T10:29:10.037784+0000","last_peered":"2024-04-25T10:29:10.037784+0000","last_clean":"2024-04-25T10:29:10.037784+0000","last_became_active":"2024-04-25T10:29:09.020672+0000","last_became_peered":"2024-04-25T10:29:09.020672+0000","last_unstale":"2024-04-25T10:29:10.037784+0000","last_undegraded":"2024-04-25T10:29:10.037784+0000","last_fullsized":"2024-04-25T10:29:10.037784+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:29:07.995096+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:29:07.995096+0000","last_clean_scrub_stamp":"2024-04-25T10:29:07.995096+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T20:52:02.098872+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.4","version":"0'0","reported_seq":14,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-04-25T10:29:10.189133+0000","last_change":"2024-04-25T10:29:08.025582+0000","last_active":"2024-04-25T10:29:10.189133+0000","last_peered":"2024-04-25T10:29:10.189133+0000","last_clean":"2024-04-25T10:29:10.189133+0000","last_became_active":"2024-04-25T10:29:08.024961+0000","last_became_peered":"2024-04-25T10:29:08.024961+0000","last_unstale":"2024-04-25T10:29:10.189133+0000","last_undegraded":"2024-04-25T10:29:10.189133+0000","last_fullsized":"2024-04-25T10:29:10.189133+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:29:06.989570+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:29:06.989570+0000","last_clean_scrub_stamp":"2024-04-25T10:29:06.989570+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T10:32:06.103123+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6],"acting":[5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.5","version":"0'0","reported_seq":12,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-04-25T10:29:10.026083+0000","last_change":"2024-04-25T10:29:09.028003+0000","last_active":"2024-04-25T10:29:10.026083+0000","last_peered":"2024-04-25T10:29:10.026083+0000","last_clean":"2024-04-25T10:29:10.026083+0000","last_became_active":"2024-04-25T10:29:09.027654+0000","last_became_peered":"2024-04-25T10:29:09.027654+0000","last_unstale":"2024-04-25T10:29:10.026083+0000","last_undegraded":"2024-04-25T10:29:10.026083+0000","last_fullsized":"2024-04-25T10:29:10.026083+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:29:07.995096+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:29:07.995096+0000","last_clean_scrub_stamp":"2024-04-25T10:29:07.995096+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T19:54:17.703318+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,6],"acting":[2,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.5","version":"0'0","reported_seq":14,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-04-25T10:29:10.024926+0000","last_change":"2024-04-25T10:29:08.026181+0000","last_active":"2024-04-25T10:29:10.024926+0000","last_peered":"2024-04-25T10:29:10.024926+0000","last_clean":"2024-04-25T10:29:10.024926+0000","last_became_active":"2024-04-25T10:29:08.025780+0000","last_became_peered":"2024-04-25T10:29:08.025780+0000","last_unstale":"2024-04-25T10:29:10.024926+0000","last_undegraded":"2024-04-25T10:29:10.024926+0000","last_fullsized":"2024-04-25T10:29:10.024926+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:29:06.989570+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:29:06.989570+0000","last_clean_scrub_stamp":"2024-04-25T10:29:06.989570+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T12:16:46.977487+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.4","version":"0'0","reported_seq":12,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-04-25T10:29:10.189423+0000","last_change":"2024-04-25T10:29:09.037824+0000","last_active":"2024-04-25T10:29:10.189423+0000","last_peered":"2024-04-25T10:29:10.189423+0000","last_clean":"2024-04-25T10:29:10.189423+0000","last_became_active":"2024-04-25T10:29:09.037478+0000","last_became_peered":"2024-04-25T10:29:09.037478+0000","last_unstale":"2024-04-25T10:29:10.189423+0000","last_undegraded":"2024-04-25T10:29:10.189423+0000","last_fullsized":"2024-04-25T10:29:10.189423+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:29:07.995096+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:29:07.995096+0000","last_clean_scrub_stamp":"2024-04-25T10:29:07.995096+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T15:02:17.480054+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.6","version":"23'1","reported_seq":15,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-04-25T10:29:10.026003+0000","last_change":"2024-04-25T10:29:08.020327+0000","last_active":"2024-04-25T10:29:10.026003+0000","last_peered":"2024-04-25T10:29:10.026003+0000","last_clean":"2024-04-25T10:29:10.026003+0000","last_became_active":"2024-04-25T10:29:08.019988+0000","last_became_peered":"2024-04-25T10:29:08.019988+0000","last_unstale":"2024-04-25T10:29:10.026003+0000","last_undegraded":"2024-04-25T10:29:10.026003+0000","last_fullsized":"2024-04-25T10:29:10.026003+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:29:06.989570+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:29:06.989570+0000","last_clean_scrub_stamp":"2024-04-25T10:29:06.989570+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T13:17:35.774897+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.7","version":"0'0","reported_seq":12,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-04-25T10:29:10.025861+0000","last_change":"2024-04-25T10:29:09.029850+0000","last_active":"2024-04-25T10:29:10.025861+0000","last_peered":"2024-04-25T10:29:10.025861+0000","last_clean":"2024-04-25T10:29:10.025861+0000","last_became_active":"2024-04-25T10:29:09.029534+0000","last_became_peered":"2024-04-25T10:29:09.029534+0000","last_unstale":"2024-04-25T10:29:10.025861+0000","last_undegraded":"2024-04-25T10:29:10.025861+0000","last_fullsized":"2024-04-25T10:29:10.025861+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:29:07.995096+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:29:07.995096+0000","last_clean_scrub_stamp":"2024-04-25T10:29:07.995096+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T15:03:07.958155+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.7","version":"22'2","reported_seq":16,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-04-25T10:29:10.024876+0000","last_change":"2024-04-25T10:29:08.026080+0000","last_active":"2024-04-25T10:29:10.024876+0000","last_peered":"2024-04-25T10:29:10.024876+0000","last_clean":"2024-04-25T10:29:10.024876+0000","last_became_active":"2024-04-25T10:29:08.025707+0000","last_became_peered":"2024-04-25T10:29:08.025707+0000","last_unstale":"2024-04-25T10:29:10.024876+0000","last_undegraded":"2024-04-25T10:29:10.024876+0000","last_fullsized":"2024-04-25T10:29:10.024876+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:29:06.989570+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:29:06.989570+0000","last_clean_scrub_stamp":"2024-04-25T10:29:06.989570+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T10:55:00.499631+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.6","version":"0'0","reported_seq":12,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-04-25T10:29:10.025829+0000","last_change":"2024-04-25T10:29:09.025814+0000","last_active":"2024-04-25T10:29:10.025829+0000","last_peered":"2024-04-25T10:29:10.025829+0000","last_clean":"2024-04-25T10:29:10.025829+0000","last_became_active":"2024-04-25T10:29:09.025474+0000","last_became_peered":"2024-04-25T10:29:09.025474+0000","last_unstale":"2024-04-25T10:29:10.025829+0000","last_undegraded":"2024-04-25T10:29:10.025829+0000","last_fullsized":"2024-04-25T10:29:10.025829+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:29:07.995096+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:29:07.995096+0000","last_clean_scrub_stamp":"2024-04-25T10:29:07.995096+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T12:35:12.945537+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7],"acting":[4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.8","version":"0'0","reported_seq":14,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-04-25T10:29:10.037271+0000","last_change":"2024-04-25T10:29:08.024845+0000","last_active":"2024-04-25T10:29:10.037271+0000","last_peered":"2024-04-25T10:29:10.037271+0000","last_clean":"2024-04-25T10:29:10.037271+0000","last_became_active":"2024-04-25T10:29:08.024484+0000","last_became_peered":"2024-04-25T10:29:08.024484+0000","last_unstale":"2024-04-25T10:29:10.037271+0000","last_undegraded":"2024-04-25T10:29:10.037271+0000","last_fullsized":"2024-04-25T10:29:10.037271+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:29:06.989570+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:29:06.989570+0000","last_clean_scrub_stamp":"2024-04-25T10:29:06.989570+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T14:12:57.950482+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.9","version":"0'0","reported_seq":12,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-04-25T10:29:10.024346+0000","last_change":"2024-04-25T10:29:09.022502+0000","last_active":"2024-04-25T10:29:10.024346+0000","last_peered":"2024-04-25T10:29:10.024346+0000","last_clean":"2024-04-25T10:29:10.024346+0000","last_became_active":"2024-04-25T10:29:09.022184+0000","last_became_peered":"2024-04-25T10:29:09.022184+0000","last_unstale":"2024-04-25T10:29:10.024346+0000","last_undegraded":"2024-04-25T10:29:10.024346+0000","last_fullsized":"2024-04-25T10:29:10.024346+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:29:07.995096+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:29:07.995096+0000","last_clean_scrub_stamp":"2024-04-25T10:29:07.995096+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T19:56:27.663748+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.9","version":"0'0","reported_seq":14,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-04-25T10:29:10.025738+0000","last_change":"2024-04-25T10:29:08.024989+0000","last_active":"2024-04-25T10:29:10.025738+0000","last_peered":"2024-04-25T10:29:10.025738+0000","last_clean":"2024-04-25T10:29:10.025738+0000","last_became_active":"2024-04-25T10:29:08.024655+0000","last_became_peered":"2024-04-25T10:29:08.024655+0000","last_unstale":"2024-04-25T10:29:10.025738+0000","last_undegraded":"2024-04-25T10:29:10.025738+0000","last_fullsized":"2024-04-25T10:29:10.025738+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:29:06.989570+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:29:06.989570+0000","last_clean_scrub_stamp":"2024-04-25T10:29:06.989570+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T21:05:51.419862+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.8","version":"0'0","reported_seq":12,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-04-25T10:29:10.025393+0000","last_change":"2024-04-25T10:29:09.023330+0000","last_active":"2024-04-25T10:29:10.025393+0000","last_peered":"2024-04-25T10:29:10.025393+0000","last_clean":"2024-04-25T10:29:10.025393+0000","last_became_active":"2024-04-25T10:29:09.020543+0000","last_became_peered":"2024-04-25T10:29:09.020543+0000","last_unstale":"2024-04-25T10:29:10.025393+0000","last_undegraded":"2024-04-25T10:29:10.025393+0000","last_fullsized":"2024-04-25T10:29:10.025393+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:29:07.995096+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:29:07.995096+0000","last_clean_scrub_stamp":"2024-04-25T10:29:07.995096+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T11:12:41.953761+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7],"acting":[4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.a","version":"0'0","reported_seq":14,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-04-25T10:29:10.188957+0000","last_change":"2024-04-25T10:29:08.020363+0000","last_active":"2024-04-25T10:29:10.188957+0000","last_peered":"2024-04-25T10:29:10.188957+0000","last_clean":"2024-04-25T10:29:10.188957+0000","last_became_active":"2024-04-25T10:29:08.020037+0000","last_became_peered":"2024-04-25T10:29:08.020037+0000","last_unstale":"2024-04-25T10:29:10.188957+0000","last_undegraded":"2024-04-25T10:29:10.188957+0000","last_fullsized":"2024-04-25T10:29:10.188957+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:29:06.989570+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:29:06.989570+0000","last_clean_scrub_stamp":"2024-04-25T10:29:06.989570+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T10:32:06.103123+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.b","version":"0'0","reported_seq":12,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-04-25T10:29:10.189294+0000","last_change":"2024-04-25T10:29:09.028820+0000","last_active":"2024-04-25T10:29:10.189294+0000","last_peered":"2024-04-25T10:29:10.189294+0000","last_clean":"2024-04-25T10:29:10.189294+0000","last_became_active":"2024-04-25T10:29:09.028426+0000","last_became_peered":"2024-04-25T10:29:09.028426+0000","last_unstale":"2024-04-25T10:29:10.189294+0000","last_undegraded":"2024-04-25T10:29:10.189294+0000","last_fullsized":"2024-04-25T10:29:10.189294+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:29:07.995096+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:29:07.995096+0000","last_clean_scrub_stamp":"2024-04-25T10:29:07.995096+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T19:54:17.703318+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.b","version":"22'2","reported_seq":16,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-04-25T10:29:10.024445+0000","last_change":"2024-04-25T10:29:08.025011+0000","last_active":"2024-04-25T10:29:10.024445+0000","last_peered":"2024-04-25T10:29:10.024445+0000","last_clean":"2024-04-25T10:29:10.024445+0000","last_became_active":"2024-04-25T10:29:08.024667+0000","last_became_peered":"2024-04-25T10:29:08.024667+0000","last_unstale":"2024-04-25T10:29:10.024445+0000","last_undegraded":"2024-04-25T10:29:10.024445+0000","last_fullsized":"2024-04-25T10:29:10.024445+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:29:06.989570+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:29:06.989570+0000","last_clean_scrub_stamp":"2024-04-25T10:29:06.989570+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T15:37:05.717854+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.a","version":"0'0","reported_seq":12,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-04-25T10:29:10.024394+0000","last_change":"2024-04-25T10:29:09.017160+0000","last_active":"2024-04-25T10:29:10.024394+0000","last_peered":"2024-04-25T10:29:10.024394+0000","last_clean":"2024-04-25T10:29:10.024394+0000","last_became_active":"2024-04-25T10:29:09.016821+0000","last_became_peered":"2024-04-25T10:29:09.016821+0000","last_unstale":"2024-04-25T10:29:10.024394+0000","last_undegraded":"2024-04-25T10:29:10.024394+0000","last_fullsized":"2024-04-25T10:29:10.024394+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:29:07.995096+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:29:07.995096+0000","last_clean_scrub_stamp":"2024-04-25T10:29:07.995096+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T20:40:19.840339+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.1d","version":"0'0","reported_seq":12,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-04-25T10:29:10.024622+0000","last_change":"2024-04-25T10:29:09.030341+0000","last_active":"2024-04-25T10:29:10.024622+0000","last_peered":"2024-04-25T10:29:10.024622+0000","last_clean":"2024-04-25T10:29:10.024622+0000","last_became_active":"2024-04-25T10:29:09.029988+0000","last_became_peered":"2024-04-25T10:29:09.029988+0000","last_unstale":"2024-04-25T10:29:10.024622+0000","last_undegraded":"2024-04-25T10:29:10.024622+0000","last_fullsized":"2024-04-25T10:29:10.024622+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:29:07.995096+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:29:07.995096+0000","last_clean_scrub_stamp":"2024-04-25T10:29:07.995096+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T11:40:18.624659+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.1c","version":"22'2","reported_seq":16,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-04-25T10:29:10.037969+0000","last_change":"2024-04-25T10:29:08.027197+0000","last_active":"2024-04-25T10:29:10.037969+0000","last_peered":"2024-04-25T10:29:10.037969+0000","last_clean":"2024-04-25T10:29:10.037969+0000","last_became_active":"2024-04-25T10:29:08.026872+0000","last_became_peered":"2024-04-25T10:29:08.026872+0000","last_unstale":"2024-04-25T10:29:10.037969+0000","last_undegraded":"2024-04-25T10:29:10.037969+0000","last_fullsized":"2024-04-25T10:29:10.037969+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:29:06.989570+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:29:06.989570+0000","last_clean_scrub_stamp":"2024-04-25T10:29:06.989570+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T15:58:35.843305+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.1c","version":"0'0","reported_seq":12,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-04-25T10:29:10.187853+0000","last_change":"2024-04-25T10:29:09.032731+0000","last_active":"2024-04-25T10:29:10.187853+0000","last_peered":"2024-04-25T10:29:10.187853+0000","last_clean":"2024-04-25T10:29:10.187853+0000","last_became_active":"2024-04-25T10:29:09.032383+0000","last_became_peered":"2024-04-25T10:29:09.032383+0000","last_unstale":"2024-04-25T10:29:10.187853+0000","last_undegraded":"2024-04-25T10:29:10.187853+0000","last_fullsized":"2024-04-25T10:29:10.187853+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:29:07.995096+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:29:07.995096+0000","last_clean_scrub_stamp":"2024-04-25T10:29:07.995096+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T19:54:17.703318+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.1d","version":"22'2","reported_seq":16,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-04-25T10:29:10.037807+0000","last_change":"2024-04-25T10:29:08.026410+0000","last_active":"2024-04-25T10:29:10.037807+0000","last_peered":"2024-04-25T10:29:10.037807+0000","last_clean":"2024-04-25T10:29:10.037807+0000","last_became_active":"2024-04-25T10:29:08.026071+0000","last_became_peered":"2024-04-25T10:29:08.026071+0000","last_unstale":"2024-04-25T10:29:10.037807+0000","last_undegraded":"2024-04-25T10:29:10.037807+0000","last_fullsized":"2024-04-25T10:29:10.037807+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:29:06.989570+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:29:06.989570+0000","last_clean_scrub_stamp":"2024-04-25T10:29:06.989570+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T19:54:16.697792+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.1f","version":"0'0","reported_seq":12,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-04-25T10:29:10.189240+0000","last_change":"2024-04-25T10:29:09.032735+0000","last_active":"2024-04-25T10:29:10.189240+0000","last_peered":"2024-04-25T10:29:10.189240+0000","last_clean":"2024-04-25T10:29:10.189240+0000","last_became_active":"2024-04-25T10:29:09.032227+0000","last_became_peered":"2024-04-25T10:29:09.032227+0000","last_unstale":"2024-04-25T10:29:10.189240+0000","last_undegraded":"2024-04-25T10:29:10.189240+0000","last_fullsized":"2024-04-25T10:29:10.189240+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:29:07.995096+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:29:07.995096+0000","last_clean_scrub_stamp":"2024-04-25T10:29:07.995096+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T11:31:45.283247+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.1e","version":"22'1","reported_seq":15,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-04-25T10:29:10.189728+0000","last_change":"2024-04-25T10:29:08.017025+0000","last_active":"2024-04-25T10:29:10.189728+0000","last_peered":"2024-04-25T10:29:10.189728+0000","last_clean":"2024-04-25T10:29:10.189728+0000","last_became_active":"2024-04-25T10:29:08.016702+0000","last_became_peered":"2024-04-25T10:29:08.016702+0000","last_unstale":"2024-04-25T10:29:10.189728+0000","last_undegraded":"2024-04-25T10:29:10.189728+0000","last_fullsized":"2024-04-25T10:29:10.189728+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:29:06.989570+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:29:06.989570+0000","last_clean_scrub_stamp":"2024-04-25T10:29:06.989570+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T14:12:57.950482+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.1e","version":"0'0","reported_seq":12,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-04-25T10:29:10.189730+0000","last_change":"2024-04-25T10:29:09.036208+0000","last_active":"2024-04-25T10:29:10.189730+0000","last_peered":"2024-04-25T10:29:10.189730+0000","last_clean":"2024-04-25T10:29:10.189730+0000","last_became_active":"2024-04-25T10:29:09.034890+0000","last_became_peered":"2024-04-25T10:29:09.034890+0000","last_unstale":"2024-04-25T10:29:10.189730+0000","last_undegraded":"2024-04-25T10:29:10.189730+0000","last_fullsized":"2024-04-25T10:29:10.189730+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:29:07.995096+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:29:07.995096+0000","last_clean_scrub_stamp":"2024-04-25T10:29:07.995096+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T18:44:23.610374+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.1f","version":"22'4","reported_seq":18,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-04-25T10:29:10.026053+0000","last_change":"2024-04-25T10:29:08.026202+0000","last_active":"2024-04-25T10:29:10.026053+0000","last_peered":"2024-04-25T10:29:10.026053+0000","last_clean":"2024-04-25T10:29:10.026053+0000","last_became_active":"2024-04-25T10:29:08.025877+0000","last_became_peered":"2024-04-25T10:29:08.025877+0000","last_unstale":"2024-04-25T10:29:10.026053+0000","last_undegraded":"2024-04-25T10:29:10.026053+0000","last_fullsized":"2024-04-25T10:29:10.026053+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:29:06.989570+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:29:06.989570+0000","last_clean_scrub_stamp":"2024-04-25T10:29:06.989570+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T20:00:40.198709+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":582,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]}],"pool_stats":[{"poolid":3,"num_pg":64,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":128,"acting":128,"num_store_stats":8},{"poolid":2,"num_pg":32,"stat_sum":{"num_bytes":2410,"num_objects":22,"num_object_clones":0,"num_object_copies":44,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":22,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":45,"num_write_kb":14,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":14,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":65536,"data_stored":4820,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":39,"ondisk_log_size":39,"up":64,"acting":64,"num_store_stats":8},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":66,"num_read_kb":55,"num_write":101,"num_write_kb":1628,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1187840,"data_stored":1180736,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":72,"ondisk_log_size":72,"up":2,"acting":2,"num_store_stats":2}],"osd_stats":[{"osd":7,"up_from":15,"seq":64424509445,"num_pgs":3,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27244,"kb_used_data":224,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344596,"statfs":{"total":96636764160,"available":96608866304,"internally_reserved":0,"allocated":229376,"data_stored":75059,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1590,"internal_metadata":27654602},"hb_peers":[0,1,2,3,4,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[]},{"osd":6,"up_from":15,"seq":64424509446,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27236,"kb_used_data":216,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344604,"statfs":{"total":96636764160,"available":96608874496,"internally_reserved":0,"allocated":221184,"data_stored":74935,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":27654598},"hb_peers":[0,1,2,3,4,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[]},{"osd":5,"up_from":15,"seq":64424509447,"num_pgs":1,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27824,"kb_used_data":804,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344016,"statfs":{"total":96636764160,"available":96608272384,"internally_reserved":0,"allocated":823296,"data_stored":665395,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":6,"apply_latency_ms":6,"commit_latency_ns":6000000,"apply_latency_ns":6000000},"alerts":[]},{"osd":3,"up_from":15,"seq":64424509447,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27240,"kb_used_data":220,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344600,"statfs":{"total":96636764160,"available":96608870400,"internally_reserved":0,"allocated":225280,"data_stored":75517,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1593,"internal_metadata":27654599},"hb_peers":[0,1,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[]},{"osd":2,"up_from":15,"seq":64424509447,"num_pgs":1,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27248,"kb_used_data":228,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344592,"statfs":{"total":96636764160,"available":96608862208,"internally_reserved":0,"allocated":233472,"data_stored":76547,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[]},{"osd":1,"up_from":15,"seq":64424509445,"num_pgs":12,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27820,"kb_used_data":800,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344020,"statfs":{"total":96636764160,"available":96608276480,"internally_reserved":0,"allocated":819200,"data_stored":665373,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[]},{"osd":0,"up_from":15,"seq":64424509446,"num_pgs":35,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27244,"kb_used_data":224,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344596,"statfs":{"total":96636764160,"available":96608866304,"internally_reserved":0,"allocated":229376,"data_stored":74991,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[1,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":5,"apply_latency_ms":5,"commit_latency_ns":5000000,"apply_latency_ns":5000000},"alerts":[]},{"osd":4,"up_from":13,"seq":55834574854,"num_pgs":22,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27276,"kb_used_data":248,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344564,"statfs":{"total":96636764160,"available":96608833536,"internally_reserved":0,"allocated":253952,"data_stored":83519,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":5,"apply_latency_ms":5,"commit_latency_ns":5000000,"apply_latency_ns":5000000},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":56,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":70,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":1612,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":582,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":20480,"data_stored":2284,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":92,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":124,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-04-25T10:29:17.694 DEBUG:teuthology.orchestra.run.smithi037:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-04-25T10:29:17.821 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:17.818+0000 7f378438f640 1 Processor -- start 2024-04-25T10:29:17.822 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:17.818+0000 7f378438f640 1 -- start start 2024-04-25T10:29:17.837 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:17.819+0000 7f378438f640 1 --2- >> v2:172.21.15.203:3300/0 conn(0x7f377c0b5fd0 0x7f377c0b2ca0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:29:17.837 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:17.819+0000 7f378438f640 1 --2- >> v2:172.21.15.37:3300/0 conn(0x7f377c08cc90 0x7f377c08d0a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:29:17.837 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:17.819+0000 7f378438f640 1 -- --> v1:172.21.15.37:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f377c0cf120 con 0x7f377c08d5e0 2024-04-25T10:29:17.837 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:17.819+0000 7f378438f640 1 -- --> v2:172.21.15.37:3300/0 -- mon_getmap magic: 0 -- 0x7f377c0cf400 con 0x7f377c08cc90 2024-04-25T10:29:17.837 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:17.819+0000 7f378438f640 1 -- --> v2:172.21.15.203:3300/0 -- mon_getmap magic: 0 -- 0x7f377c0cf6e0 con 0x7f377c0b5fd0 2024-04-25T10:29:17.837 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:17.819+0000 7f378192a640 1 --2- >> v2:172.21.15.37:3300/0 conn(0x7f377c08cc90 0x7f377c08d0a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:29:17.837 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:17.819+0000 7f378192a640 1 --2- >> v2:172.21.15.37:3300/0 conn(0x7f377c08cc90 0x7f377c08d0a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.37:3300/0 says I am v2:172.21.15.37:46364/0 (socket says 172.21.15.37:46364) 2024-04-25T10:29:17.837 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:17.819+0000 7f378292c640 1 --1- >> v1:172.21.15.37:6789/0 conn(0x7f377c08d5e0 0x7f377c0d18c0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.37:6789/0 says I am v1:172.21.15.37:51446/0 (socket says 172.21.15.37:51446) 2024-04-25T10:29:17.837 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:17.819+0000 7f378192a640 1 -- 172.21.15.37:0/3686600463 learned_addr learned my addr 172.21.15.37:0/3686600463 (peer_addr_for_me v2:172.21.15.37:0/0) 2024-04-25T10:29:17.837 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:17.820+0000 7f378212b640 1 --2- 172.21.15.37:0/3686600463 >> v2:172.21.15.203:3300/0 conn(0x7f377c0b5fd0 0x7f377c0b2ca0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:29:17.837 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:17.820+0000 7f378192a640 1 -- 172.21.15.37:0/3686600463 >> v2:172.21.15.203:3300/0 conn(0x7f377c0b5fd0 msgr2=0x7f377c0b2ca0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:29:17.838 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:17.820+0000 7f378192a640 1 --2- 172.21.15.37:0/3686600463 >> v2:172.21.15.203:3300/0 conn(0x7f377c0b5fd0 0x7f377c0b2ca0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:29:17.838 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:17.820+0000 7f378192a640 1 -- 172.21.15.37:0/3686600463 >> v1:172.21.15.37:6789/0 conn(0x7f377c08d5e0 legacy=0x7f377c0d18c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:29:17.838 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:17.820+0000 7f3781129640 1 -- 172.21.15.37:0/3686600463 <== mon.0 v1:172.21.15.37:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 2598568985 0 0) 0x7f377c0cf120 con 0x7f377c08d5e0 2024-04-25T10:29:17.838 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:17.820+0000 7f378192a640 1 -- 172.21.15.37:0/3686600463 --> v2:172.21.15.37:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f377c0cf9c0 con 0x7f377c08cc90 2024-04-25T10:29:17.838 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:17.820+0000 7f378212b640 1 --2- 172.21.15.37:0/3686600463 >> v2:172.21.15.203:3300/0 conn(0x7f377c0b5fd0 0x7f377c0b2ca0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T10:29:17.838 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:17.820+0000 7f378192a640 1 --2- 172.21.15.37:0/3686600463 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f377c08cc90 0x7f377c08d0a0 secure :-1 s=READY pgs=91 cs=0 l=1 rev1=1 crypto rx=0x7f377c070b40 tx=0x7f376c02a1e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=4a02f545685ec91b server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:29:17.838 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:17.821+0000 7f3781129640 1 -- 172.21.15.37:0/3686600463 <== mon.0 v2:172.21.15.37:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f376c037070 con 0x7f377c08cc90 2024-04-25T10:29:17.838 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:17.821+0000 7f3781129640 1 -- 172.21.15.37:0/3686600463 <== mon.0 v2:172.21.15.37:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f376c032440 con 0x7f377c08cc90 2024-04-25T10:29:17.838 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:17.821+0000 7f3781129640 1 -- 172.21.15.37:0/3686600463 <== mon.0 v2:172.21.15.37:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f376c032810 con 0x7f377c08cc90 2024-04-25T10:29:17.838 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:17.821+0000 7f378438f640 1 -- 172.21.15.37:0/3686600463 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f377c08cc90 msgr2=0x7f377c08d0a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:29:17.838 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:17.821+0000 7f378438f640 1 --2- 172.21.15.37:0/3686600463 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f377c08cc90 0x7f377c08d0a0 secure :-1 s=READY pgs=91 cs=0 l=1 rev1=1 crypto rx=0x7f377c070b40 tx=0x7f376c02a1e0 comp rx=0 tx=0).stop 2024-04-25T10:29:17.838 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:17.821+0000 7f378438f640 1 -- 172.21.15.37:0/3686600463 shutdown_connections 2024-04-25T10:29:17.838 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:17.821+0000 7f378438f640 1 --2- 172.21.15.37:0/3686600463 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f377c08cc90 0x7f377c08d0a0 unknown :-1 s=CLOSED pgs=91 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:29:17.838 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:17.821+0000 7f378438f640 1 --2- 172.21.15.37:0/3686600463 >> v2:172.21.15.203:3300/0 conn(0x7f377c0b5fd0 0x7f377c0b2ca0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:29:17.839 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:17.821+0000 7f378438f640 1 -- 172.21.15.37:0/3686600463 >> 172.21.15.37:0/3686600463 conn(0x7f377c0b5440 msgr2=0x7f377c0b5850 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:29:17.839 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:17.821+0000 7f378438f640 1 -- 172.21.15.37:0/3686600463 shutdown_connections 2024-04-25T10:29:17.839 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:17.821+0000 7f378438f640 1 -- 172.21.15.37:0/3686600463 wait complete. 2024-04-25T10:29:17.839 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:17.823+0000 7f378438f640 1 Processor -- start 2024-04-25T10:29:17.839 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:17.823+0000 7f378438f640 1 -- start start 2024-04-25T10:29:17.839 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:17.825+0000 7f378438f640 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f377c08cc90 0x7f377c157950 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:29:17.839 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:17.825+0000 7f378438f640 1 --2- >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f377c08d5e0 0x7f377c1f72f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:29:17.839 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:17.825+0000 7f378438f640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f377c0b5fd0 0x7f377c1f7830 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:29:17.839 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:17.825+0000 7f378438f640 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 -- 0x7f377c0b4800 con 0x7f377c0b5fd0 2024-04-25T10:29:17.839 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:17.825+0000 7f378438f640 1 -- --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_getmap magic: 0 -- 0x7f377c0b4680 con 0x7f377c08cc90 2024-04-25T10:29:17.839 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:17.825+0000 7f378438f640 1 -- --> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] -- mon_getmap magic: 0 -- 0x7f377c0b4980 con 0x7f377c08d5e0 2024-04-25T10:29:17.839 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:17.825+0000 7f378292c640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f377c0b5fd0 0x7f377c1f7830 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:29:17.839 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:17.825+0000 7f378292c640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f377c0b5fd0 0x7f377c1f7830 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.37:3300/0 says I am v2:172.21.15.37:46380/0 (socket says 172.21.15.37:46380) 2024-04-25T10:29:17.839 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:17.825+0000 7f378292c640 1 -- 172.21.15.37:0/1202136688 learned_addr learned my addr 172.21.15.37:0/1202136688 (peer_addr_for_me v2:172.21.15.37:0/0) 2024-04-25T10:29:17.840 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:17.825+0000 7f378192a640 1 --2- 172.21.15.37:0/1202136688 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f377c08d5e0 0x7f377c1f72f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:29:17.840 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:17.825+0000 7f378292c640 1 -- 172.21.15.37:0/1202136688 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f377c08d5e0 msgr2=0x7f377c1f72f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:29:17.840 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:17.825+0000 7f378292c640 1 --2- 172.21.15.37:0/1202136688 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f377c08d5e0 0x7f377c1f72f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:29:17.840 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:17.825+0000 7f378292c640 1 -- 172.21.15.37:0/1202136688 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f377c08cc90 msgr2=0x7f377c157950 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:29:17.840 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:17.825+0000 7f378212b640 1 --2- 172.21.15.37:0/1202136688 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f377c08cc90 0x7f377c157950 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:29:17.840 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:17.825+0000 7f378292c640 1 --2- 172.21.15.37:0/1202136688 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f377c08cc90 0x7f377c157950 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:29:17.840 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:17.825+0000 7f378292c640 1 -- 172.21.15.37:0/1202136688 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f377c158490 con 0x7f377c0b5fd0 2024-04-25T10:29:17.840 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:17.825+0000 7f378292c640 1 --2- 172.21.15.37:0/1202136688 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f377c0b5fd0 0x7f377c1f7830 secure :-1 s=READY pgs=92 cs=0 l=1 rev1=1 crypto rx=0x7f377400c700 tx=0x7f377400cbc0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:29:17.840 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:17.825+0000 7f3772ffd640 1 -- 172.21.15.37:0/1202136688 <== mon.0 v2:172.21.15.37:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3774018070 con 0x7f377c0b5fd0 2024-04-25T10:29:17.840 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:17.825+0000 7f3772ffd640 1 -- 172.21.15.37:0/1202136688 <== mon.0 v2:172.21.15.37:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f377400ce10 con 0x7f377c0b5fd0 2024-04-25T10:29:17.840 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:17.825+0000 7f378438f640 1 -- 172.21.15.37:0/1202136688 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f377c0b8f40 con 0x7f377c0b5fd0 2024-04-25T10:29:17.840 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:17.825+0000 7f3772ffd640 1 -- 172.21.15.37:0/1202136688 <== mon.0 v2:172.21.15.37:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f377400a880 con 0x7f377c0b5fd0 2024-04-25T10:29:17.840 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:17.825+0000 7f378438f640 1 -- 172.21.15.37:0/1202136688 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f377c0ccbf0 con 0x7f377c0b5fd0 2024-04-25T10:29:17.840 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:17.826+0000 7f378438f640 1 -- 172.21.15.37:0/1202136688 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f377c06a140 con 0x7f377c0b5fd0 2024-04-25T10:29:17.841 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:17.827+0000 7f3772ffd640 1 -- 172.21.15.37:0/1202136688 <== mon.0 v2:172.21.15.37:3300/0 4 ==== mgrmap(e 5) ==== 96782+0+0 (secure 0 0 0) 0x7f377401b070 con 0x7f377c0b5fd0 2024-04-25T10:29:17.841 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:17.828+0000 7f3772ffd640 1 --2- 172.21.15.37:0/1202136688 >> [v2:172.21.15.203:6832/2418205237,v1:172.21.15.203:6833/2418205237] conn(0x7f37600744e0 0x7f37600769f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:29:17.841 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:17.828+0000 7f3772ffd640 1 -- 172.21.15.37:0/1202136688 <== mon.0 v2:172.21.15.37:3300/0 5 ==== osd_map(23..23 src has 1..23) ==== 5709+0+0 (secure 0 0 0) 0x7f37740946c0 con 0x7f377c0b5fd0 2024-04-25T10:29:17.841 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:17.828+0000 7f378212b640 1 --2- 172.21.15.37:0/1202136688 >> [v2:172.21.15.203:6832/2418205237,v1:172.21.15.203:6833/2418205237] conn(0x7f37600744e0 0x7f37600769f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:29:17.841 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:17.828+0000 7f378212b640 1 --2- 172.21.15.37:0/1202136688 >> [v2:172.21.15.203:6832/2418205237,v1:172.21.15.203:6833/2418205237] conn(0x7f37600744e0 0x7f37600769f0 secure :-1 s=READY pgs=69 cs=0 l=1 rev1=1 crypto rx=0x7f377c073c80 tx=0x7f3778008040 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:29:17.841 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:17.832+0000 7f3772ffd640 1 -- 172.21.15.37:0/1202136688 <== mon.0 v2:172.21.15.37:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187246 (secure 0 0 0) 0x7f377405eee0 con 0x7f377c0b5fd0 2024-04-25T10:29:18.016 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:18.014+0000 7f378438f640 1 -- 172.21.15.37:0/1202136688 --> [v2:172.21.15.203:6832/2418205237,v1:172.21.15.203:6833/2418205237] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) -- 0x7f377c0b64f0 con 0x7f37600744e0 2024-04-25T10:29:18.022 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:18.021+0000 7f3772ffd640 1 -- 172.21.15.37:0/1202136688 <== mgr.4100 v2:172.21.15.203:6832/2418205237 1 ==== mgr_command_reply(tid 0: 0 dumped all) ==== 18+0+251845 (secure 0 0 0) 0x7f377c0b64f0 con 0x7f37600744e0 2024-04-25T10:29:18.025 INFO:teuthology.orchestra.run.smithi037.stdout: 2024-04-25T10:29:18.026 INFO:teuthology.orchestra.run.smithi037.stderr:dumped all 2024-04-25T10:29:18.028 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:18.023+0000 7f378438f640 1 -- 172.21.15.37:0/1202136688 >> [v2:172.21.15.203:6832/2418205237,v1:172.21.15.203:6833/2418205237] conn(0x7f37600744e0 msgr2=0x7f37600769f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:29:18.028 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:18.024+0000 7f378438f640 1 --2- 172.21.15.37:0/1202136688 >> [v2:172.21.15.203:6832/2418205237,v1:172.21.15.203:6833/2418205237] conn(0x7f37600744e0 0x7f37600769f0 secure :-1 s=READY pgs=69 cs=0 l=1 rev1=1 crypto rx=0x7f377c073c80 tx=0x7f3778008040 comp rx=0 tx=0).stop 2024-04-25T10:29:18.028 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:18.024+0000 7f378438f640 1 -- 172.21.15.37:0/1202136688 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f377c0b5fd0 msgr2=0x7f377c1f7830 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:29:18.028 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:18.024+0000 7f378438f640 1 --2- 172.21.15.37:0/1202136688 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f377c0b5fd0 0x7f377c1f7830 secure :-1 s=READY pgs=92 cs=0 l=1 rev1=1 crypto rx=0x7f377400c700 tx=0x7f377400cbc0 comp rx=0 tx=0).stop 2024-04-25T10:29:18.028 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:18.024+0000 7f378438f640 1 -- 172.21.15.37:0/1202136688 shutdown_connections 2024-04-25T10:29:18.029 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:18.024+0000 7f378438f640 1 --2- 172.21.15.37:0/1202136688 >> [v2:172.21.15.203:6832/2418205237,v1:172.21.15.203:6833/2418205237] conn(0x7f37600744e0 0x7f37600769f0 unknown :-1 s=CLOSED pgs=69 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:29:18.029 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:18.024+0000 7f378438f640 1 --2- 172.21.15.37:0/1202136688 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f377c0b5fd0 0x7f377c1f7830 unknown :-1 s=CLOSED pgs=92 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:29:18.029 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:18.024+0000 7f378438f640 1 --2- 172.21.15.37:0/1202136688 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f377c08d5e0 0x7f377c1f72f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:29:18.029 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:18.024+0000 7f378438f640 1 --2- 172.21.15.37:0/1202136688 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f377c08cc90 0x7f377c157950 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:29:18.029 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:18.024+0000 7f378438f640 1 -- 172.21.15.37:0/1202136688 >> 172.21.15.37:0/1202136688 conn(0x7f377c0b5440 msgr2=0x7f377c0712b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:29:18.029 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:18.024+0000 7f378438f640 1 -- 172.21.15.37:0/1202136688 shutdown_connections 2024-04-25T10:29:18.029 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:18.024+0000 7f378438f640 1 -- 172.21.15.37:0/1202136688 wait complete. 2024-04-25T10:29:18.037 INFO:teuthology.orchestra.run.smithi037.stdout:{"pg_ready":true,"pg_map":{"version":28,"stamp":"2024-04-25T10:29:16.187872+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":592778,"num_objects":24,"num_object_clones":0,"num_object_copies":48,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":24,"num_whiteouts":0,"num_read":66,"num_read_kb":55,"num_write":146,"num_write_kb":1642,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":14,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":111,"ondisk_log_size":111,"up":194,"acting":194,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":194,"num_osds":8,"num_per_pool_osds":8,"num_per_pool_omap_osds":8,"kb":754974720,"kb_used":219244,"kb_used_data":3048,"kb_used_omap":12,"kb_used_meta":216051,"kb_avail":754755476,"statfs":{"total":773094113280,"available":772869607424,"internally_reserved":0,"allocated":3121152,"data_stored":1835436,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":12752,"internal_metadata":221236784},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":5,"apply_latency_ms":5,"commit_latency_ns":5000000,"apply_latency_ns":5000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":2410,"num_objects":22,"num_object_clones":0,"num_object_copies":44,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":22,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":45,"num_write_kb":14,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":14,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"8.002564"},"pg_stats":[{"pgid":"3.3f","version":"0'0","reported_seq":12,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-04-25T10:29:10.037967+0000","last_change":"2024-04-25T10:29:09.024146+0000","last_active":"2024-04-25T10:29:10.037967+0000","last_peered":"2024-04-25T10:29:10.037967+0000","last_clean":"2024-04-25T10:29:10.037967+0000","last_became_active":"2024-04-25T10:29:09.023798+0000","last_became_peered":"2024-04-25T10:29:09.023798+0000","last_unstale":"2024-04-25T10:29:10.037967+0000","last_undegraded":"2024-04-25T10:29:10.037967+0000","last_fullsized":"2024-04-25T10:29:10.037967+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:29:07.995096+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:29:07.995096+0000","last_clean_scrub_stamp":"2024-04-25T10:29:07.995096+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T20:37:20.726766+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.3e","version":"0'0","reported_seq":12,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-04-25T10:29:10.189579+0000","last_change":"2024-04-25T10:29:09.028600+0000","last_active":"2024-04-25T10:29:10.189579+0000","last_peered":"2024-04-25T10:29:10.189579+0000","last_clean":"2024-04-25T10:29:10.189579+0000","last_became_active":"2024-04-25T10:29:09.028286+0000","last_became_peered":"2024-04-25T10:29:09.028286+0000","last_unstale":"2024-04-25T10:29:10.189579+0000","last_undegraded":"2024-04-25T10:29:10.189579+0000","last_fullsized":"2024-04-25T10:29:10.189579+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:29:07.995096+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:29:07.995096+0000","last_clean_scrub_stamp":"2024-04-25T10:29:07.995096+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T20:52:02.098872+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6],"acting":[5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.3d","version":"0'0","reported_seq":12,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-04-25T10:29:10.189318+0000","last_change":"2024-04-25T10:29:09.039030+0000","last_active":"2024-04-25T10:29:10.189318+0000","last_peered":"2024-04-25T10:29:10.189318+0000","last_clean":"2024-04-25T10:29:10.189318+0000","last_became_active":"2024-04-25T10:29:09.037692+0000","last_became_peered":"2024-04-25T10:29:09.037692+0000","last_unstale":"2024-04-25T10:29:10.189318+0000","last_undegraded":"2024-04-25T10:29:10.189318+0000","last_fullsized":"2024-04-25T10:29:10.189318+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:29:07.995096+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:29:07.995096+0000","last_clean_scrub_stamp":"2024-04-25T10:29:07.995096+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T15:51:52.989545+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.3c","version":"0'0","reported_seq":12,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-04-25T10:29:10.026169+0000","last_change":"2024-04-25T10:29:09.027520+0000","last_active":"2024-04-25T10:29:10.026169+0000","last_peered":"2024-04-25T10:29:10.026169+0000","last_clean":"2024-04-25T10:29:10.026169+0000","last_became_active":"2024-04-25T10:29:09.027075+0000","last_became_peered":"2024-04-25T10:29:09.027075+0000","last_unstale":"2024-04-25T10:29:10.026169+0000","last_undegraded":"2024-04-25T10:29:10.026169+0000","last_fullsized":"2024-04-25T10:29:10.026169+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:29:07.995096+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:29:07.995096+0000","last_clean_scrub_stamp":"2024-04-25T10:29:07.995096+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T10:55:01.505157+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4],"acting":[2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.3b","version":"0'0","reported_seq":12,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-04-25T10:29:10.037590+0000","last_change":"2024-04-25T10:29:09.040684+0000","last_active":"2024-04-25T10:29:10.037590+0000","last_peered":"2024-04-25T10:29:10.037590+0000","last_clean":"2024-04-25T10:29:10.037590+0000","last_became_active":"2024-04-25T10:29:09.040372+0000","last_became_peered":"2024-04-25T10:29:09.040372+0000","last_unstale":"2024-04-25T10:29:10.037590+0000","last_undegraded":"2024-04-25T10:29:10.037590+0000","last_fullsized":"2024-04-25T10:29:10.037590+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:29:07.995096+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:29:07.995096+0000","last_clean_scrub_stamp":"2024-04-25T10:29:07.995096+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T11:52:24.415332+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.3a","version":"0'0","reported_seq":12,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-04-25T10:29:10.025911+0000","last_change":"2024-04-25T10:29:09.027391+0000","last_active":"2024-04-25T10:29:10.025911+0000","last_peered":"2024-04-25T10:29:10.025911+0000","last_clean":"2024-04-25T10:29:10.025911+0000","last_became_active":"2024-04-25T10:29:09.027029+0000","last_became_peered":"2024-04-25T10:29:09.027029+0000","last_unstale":"2024-04-25T10:29:10.025911+0000","last_undegraded":"2024-04-25T10:29:10.025911+0000","last_fullsized":"2024-04-25T10:29:10.025911+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:29:07.995096+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:29:07.995096+0000","last_clean_scrub_stamp":"2024-04-25T10:29:07.995096+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T10:35:31.361514+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4],"acting":[2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.39","version":"0'0","reported_seq":12,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-04-25T10:29:10.037462+0000","last_change":"2024-04-25T10:29:09.029272+0000","last_active":"2024-04-25T10:29:10.037462+0000","last_peered":"2024-04-25T10:29:10.037462+0000","last_clean":"2024-04-25T10:29:10.037462+0000","last_became_active":"2024-04-25T10:29:09.028302+0000","last_became_peered":"2024-04-25T10:29:09.028302+0000","last_unstale":"2024-04-25T10:29:10.037462+0000","last_undegraded":"2024-04-25T10:29:10.037462+0000","last_fullsized":"2024-04-25T10:29:10.037462+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:29:07.995096+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:29:07.995096+0000","last_clean_scrub_stamp":"2024-04-25T10:29:07.995096+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T11:31:45.283247+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.38","version":"0'0","reported_seq":12,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-04-25T10:29:10.188973+0000","last_change":"2024-04-25T10:29:09.035089+0000","last_active":"2024-04-25T10:29:10.188973+0000","last_peered":"2024-04-25T10:29:10.188973+0000","last_clean":"2024-04-25T10:29:10.188973+0000","last_became_active":"2024-04-25T10:29:09.034324+0000","last_became_peered":"2024-04-25T10:29:09.034324+0000","last_unstale":"2024-04-25T10:29:10.188973+0000","last_undegraded":"2024-04-25T10:29:10.188973+0000","last_fullsized":"2024-04-25T10:29:10.188973+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:29:07.995096+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:29:07.995096+0000","last_clean_scrub_stamp":"2024-04-25T10:29:07.995096+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T21:39:39.743441+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.37","version":"0'0","reported_seq":12,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-04-25T10:29:10.187981+0000","last_change":"2024-04-25T10:29:09.016966+0000","last_active":"2024-04-25T10:29:10.187981+0000","last_peered":"2024-04-25T10:29:10.187981+0000","last_clean":"2024-04-25T10:29:10.187981+0000","last_became_active":"2024-04-25T10:29:09.016566+0000","last_became_peered":"2024-04-25T10:29:09.016566+0000","last_unstale":"2024-04-25T10:29:10.187981+0000","last_undegraded":"2024-04-25T10:29:10.187981+0000","last_fullsized":"2024-04-25T10:29:10.187981+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:29:07.995096+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:29:07.995096+0000","last_clean_scrub_stamp":"2024-04-25T10:29:07.995096+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T10:55:01.505157+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.36","version":"0'0","reported_seq":12,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-04-25T10:29:10.037810+0000","last_change":"2024-04-25T10:29:09.030321+0000","last_active":"2024-04-25T10:29:10.037810+0000","last_peered":"2024-04-25T10:29:10.037810+0000","last_clean":"2024-04-25T10:29:10.037810+0000","last_became_active":"2024-04-25T10:29:09.029976+0000","last_became_peered":"2024-04-25T10:29:09.029976+0000","last_unstale":"2024-04-25T10:29:10.037810+0000","last_undegraded":"2024-04-25T10:29:10.037810+0000","last_fullsized":"2024-04-25T10:29:10.037810+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:29:07.995096+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:29:07.995096+0000","last_clean_scrub_stamp":"2024-04-25T10:29:07.995096+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T12:36:31.191100+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.35","version":"0'0","reported_seq":12,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-04-25T10:29:10.037641+0000","last_change":"2024-04-25T10:29:09.030469+0000","last_active":"2024-04-25T10:29:10.037641+0000","last_peered":"2024-04-25T10:29:10.037641+0000","last_clean":"2024-04-25T10:29:10.037641+0000","last_became_active":"2024-04-25T10:29:09.029350+0000","last_became_peered":"2024-04-25T10:29:09.029350+0000","last_unstale":"2024-04-25T10:29:10.037641+0000","last_undegraded":"2024-04-25T10:29:10.037641+0000","last_fullsized":"2024-04-25T10:29:10.037641+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:29:07.995096+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:29:07.995096+0000","last_clean_scrub_stamp":"2024-04-25T10:29:07.995096+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T15:51:52.989545+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.34","version":"0'0","reported_seq":12,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-04-25T10:29:10.037728+0000","last_change":"2024-04-25T10:29:09.029772+0000","last_active":"2024-04-25T10:29:10.037728+0000","last_peered":"2024-04-25T10:29:10.037728+0000","last_clean":"2024-04-25T10:29:10.037728+0000","last_became_active":"2024-04-25T10:29:09.029350+0000","last_became_peered":"2024-04-25T10:29:09.029350+0000","last_unstale":"2024-04-25T10:29:10.037728+0000","last_undegraded":"2024-04-25T10:29:10.037728+0000","last_fullsized":"2024-04-25T10:29:10.037728+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:29:07.995096+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:29:07.995096+0000","last_clean_scrub_stamp":"2024-04-25T10:29:07.995096+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T19:14:48.737269+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.33","version":"0'0","reported_seq":12,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-04-25T10:29:10.188971+0000","last_change":"2024-04-25T10:29:09.038690+0000","last_active":"2024-04-25T10:29:10.188971+0000","last_peered":"2024-04-25T10:29:10.188971+0000","last_clean":"2024-04-25T10:29:10.188971+0000","last_became_active":"2024-04-25T10:29:09.038336+0000","last_became_peered":"2024-04-25T10:29:09.038336+0000","last_unstale":"2024-04-25T10:29:10.188971+0000","last_undegraded":"2024-04-25T10:29:10.188971+0000","last_fullsized":"2024-04-25T10:29:10.188971+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:29:07.995096+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:29:07.995096+0000","last_clean_scrub_stamp":"2024-04-25T10:29:07.995096+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T19:23:12.589414+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.32","version":"0'0","reported_seq":12,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-04-25T10:29:10.188641+0000","last_change":"2024-04-25T10:29:09.028195+0000","last_active":"2024-04-25T10:29:10.188641+0000","last_peered":"2024-04-25T10:29:10.188641+0000","last_clean":"2024-04-25T10:29:10.188641+0000","last_became_active":"2024-04-25T10:29:09.027893+0000","last_became_peered":"2024-04-25T10:29:09.027893+0000","last_unstale":"2024-04-25T10:29:10.188641+0000","last_undegraded":"2024-04-25T10:29:10.188641+0000","last_fullsized":"2024-04-25T10:29:10.188641+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:29:07.995096+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:29:07.995096+0000","last_clean_scrub_stamp":"2024-04-25T10:29:07.995096+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T11:35:27.466874+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.31","version":"0'0","reported_seq":12,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-04-25T10:29:10.188956+0000","last_change":"2024-04-25T10:29:09.034532+0000","last_active":"2024-04-25T10:29:10.188956+0000","last_peered":"2024-04-25T10:29:10.188956+0000","last_clean":"2024-04-25T10:29:10.188956+0000","last_became_active":"2024-04-25T10:29:09.034021+0000","last_became_peered":"2024-04-25T10:29:09.034021+0000","last_unstale":"2024-04-25T10:29:10.188956+0000","last_undegraded":"2024-04-25T10:29:10.188956+0000","last_fullsized":"2024-04-25T10:29:10.188956+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:29:07.995096+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:29:07.995096+0000","last_clean_scrub_stamp":"2024-04-25T10:29:07.995096+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T12:16:04.880732+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.30","version":"0'0","reported_seq":12,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-04-25T10:29:10.024134+0000","last_change":"2024-04-25T10:29:09.021224+0000","last_active":"2024-04-25T10:29:10.024134+0000","last_peered":"2024-04-25T10:29:10.024134+0000","last_clean":"2024-04-25T10:29:10.024134+0000","last_became_active":"2024-04-25T10:29:09.020964+0000","last_became_peered":"2024-04-25T10:29:09.020964+0000","last_unstale":"2024-04-25T10:29:10.024134+0000","last_undegraded":"2024-04-25T10:29:10.024134+0000","last_fullsized":"2024-04-25T10:29:10.024134+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:29:07.995096+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:29:07.995096+0000","last_clean_scrub_stamp":"2024-04-25T10:29:07.995096+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T18:44:23.610374+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.2f","version":"0'0","reported_seq":12,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-04-25T10:29:10.024944+0000","last_change":"2024-04-25T10:29:09.025638+0000","last_active":"2024-04-25T10:29:10.024944+0000","last_peered":"2024-04-25T10:29:10.024944+0000","last_clean":"2024-04-25T10:29:10.024944+0000","last_became_active":"2024-04-25T10:29:09.024517+0000","last_became_peered":"2024-04-25T10:29:09.024517+0000","last_unstale":"2024-04-25T10:29:10.024944+0000","last_undegraded":"2024-04-25T10:29:10.024944+0000","last_fullsized":"2024-04-25T10:29:10.024944+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:29:07.995096+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:29:07.995096+0000","last_clean_scrub_stamp":"2024-04-25T10:29:07.995096+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T16:29:08.975346+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.2e","version":"0'0","reported_seq":12,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-04-25T10:29:10.189006+0000","last_change":"2024-04-25T10:29:09.023923+0000","last_active":"2024-04-25T10:29:10.189006+0000","last_peered":"2024-04-25T10:29:10.189006+0000","last_clean":"2024-04-25T10:29:10.189006+0000","last_became_active":"2024-04-25T10:29:09.023579+0000","last_became_peered":"2024-04-25T10:29:09.023579+0000","last_unstale":"2024-04-25T10:29:10.189006+0000","last_undegraded":"2024-04-25T10:29:10.189006+0000","last_fullsized":"2024-04-25T10:29:10.189006+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:29:07.995096+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:29:07.995096+0000","last_clean_scrub_stamp":"2024-04-25T10:29:07.995096+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T10:35:31.361514+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.2d","version":"0'0","reported_seq":12,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-04-25T10:29:10.024852+0000","last_change":"2024-04-25T10:29:09.023039+0000","last_active":"2024-04-25T10:29:10.024852+0000","last_peered":"2024-04-25T10:29:10.024852+0000","last_clean":"2024-04-25T10:29:10.024852+0000","last_became_active":"2024-04-25T10:29:09.022706+0000","last_became_peered":"2024-04-25T10:29:09.022706+0000","last_unstale":"2024-04-25T10:29:10.024852+0000","last_undegraded":"2024-04-25T10:29:10.024852+0000","last_fullsized":"2024-04-25T10:29:10.024852+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:29:07.995096+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:29:07.995096+0000","last_clean_scrub_stamp":"2024-04-25T10:29:07.995096+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T16:48:17.352944+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.2c","version":"0'0","reported_seq":12,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-04-25T10:29:10.037891+0000","last_change":"2024-04-25T10:29:09.028979+0000","last_active":"2024-04-25T10:29:10.037891+0000","last_peered":"2024-04-25T10:29:10.037891+0000","last_clean":"2024-04-25T10:29:10.037891+0000","last_became_active":"2024-04-25T10:29:09.028061+0000","last_became_peered":"2024-04-25T10:29:09.028061+0000","last_unstale":"2024-04-25T10:29:10.037891+0000","last_undegraded":"2024-04-25T10:29:10.037891+0000","last_fullsized":"2024-04-25T10:29:10.037891+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:29:07.995096+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:29:07.995096+0000","last_clean_scrub_stamp":"2024-04-25T10:29:07.995096+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T18:44:23.610374+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.2b","version":"0'0","reported_seq":12,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-04-25T10:29:10.025554+0000","last_change":"2024-04-25T10:29:09.022874+0000","last_active":"2024-04-25T10:29:10.025554+0000","last_peered":"2024-04-25T10:29:10.025554+0000","last_clean":"2024-04-25T10:29:10.025554+0000","last_became_active":"2024-04-25T10:29:09.022360+0000","last_became_peered":"2024-04-25T10:29:09.022360+0000","last_unstale":"2024-04-25T10:29:10.025554+0000","last_undegraded":"2024-04-25T10:29:10.025554+0000","last_fullsized":"2024-04-25T10:29:10.025554+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:29:07.995096+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:29:07.995096+0000","last_clean_scrub_stamp":"2024-04-25T10:29:07.995096+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T19:32:59.816545+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.2a","version":"0'0","reported_seq":12,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-04-25T10:29:10.187695+0000","last_change":"2024-04-25T10:29:09.029965+0000","last_active":"2024-04-25T10:29:10.187695+0000","last_peered":"2024-04-25T10:29:10.187695+0000","last_clean":"2024-04-25T10:29:10.187695+0000","last_became_active":"2024-04-25T10:29:09.029616+0000","last_became_peered":"2024-04-25T10:29:09.029616+0000","last_unstale":"2024-04-25T10:29:10.187695+0000","last_undegraded":"2024-04-25T10:29:10.187695+0000","last_fullsized":"2024-04-25T10:29:10.187695+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:29:07.995096+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:29:07.995096+0000","last_clean_scrub_stamp":"2024-04-25T10:29:07.995096+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T15:58:36.848831+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.29","version":"0'0","reported_seq":12,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-04-25T10:29:10.189345+0000","last_change":"2024-04-25T10:29:09.033401+0000","last_active":"2024-04-25T10:29:10.189345+0000","last_peered":"2024-04-25T10:29:10.189345+0000","last_clean":"2024-04-25T10:29:10.189345+0000","last_became_active":"2024-04-25T10:29:09.032764+0000","last_became_peered":"2024-04-25T10:29:09.032764+0000","last_unstale":"2024-04-25T10:29:10.189345+0000","last_undegraded":"2024-04-25T10:29:10.189345+0000","last_fullsized":"2024-04-25T10:29:10.189345+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:29:07.995096+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:29:07.995096+0000","last_clean_scrub_stamp":"2024-04-25T10:29:07.995096+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T17:30:54.093151+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.28","version":"0'0","reported_seq":12,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-04-25T10:29:10.024133+0000","last_change":"2024-04-25T10:29:09.041063+0000","last_active":"2024-04-25T10:29:10.024133+0000","last_peered":"2024-04-25T10:29:10.024133+0000","last_clean":"2024-04-25T10:29:10.024133+0000","last_became_active":"2024-04-25T10:29:09.040734+0000","last_became_peered":"2024-04-25T10:29:09.040734+0000","last_unstale":"2024-04-25T10:29:10.024133+0000","last_undegraded":"2024-04-25T10:29:10.024133+0000","last_fullsized":"2024-04-25T10:29:10.024133+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:29:07.995096+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:29:07.995096+0000","last_clean_scrub_stamp":"2024-04-25T10:29:07.995096+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T13:54:22.968216+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.27","version":"0'0","reported_seq":12,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-04-25T10:29:10.189019+0000","last_change":"2024-04-25T10:29:09.034268+0000","last_active":"2024-04-25T10:29:10.189019+0000","last_peered":"2024-04-25T10:29:10.189019+0000","last_clean":"2024-04-25T10:29:10.189019+0000","last_became_active":"2024-04-25T10:29:09.033199+0000","last_became_peered":"2024-04-25T10:29:09.033199+0000","last_unstale":"2024-04-25T10:29:10.189019+0000","last_undegraded":"2024-04-25T10:29:10.189019+0000","last_fullsized":"2024-04-25T10:29:10.189019+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:29:07.995096+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:29:07.995096+0000","last_clean_scrub_stamp":"2024-04-25T10:29:07.995096+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T17:56:09.999494+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.26","version":"0'0","reported_seq":12,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-04-25T10:29:10.026183+0000","last_change":"2024-04-25T10:29:09.025643+0000","last_active":"2024-04-25T10:29:10.026183+0000","last_peered":"2024-04-25T10:29:10.026183+0000","last_clean":"2024-04-25T10:29:10.026183+0000","last_became_active":"2024-04-25T10:29:09.024894+0000","last_became_peered":"2024-04-25T10:29:09.024894+0000","last_unstale":"2024-04-25T10:29:10.026183+0000","last_undegraded":"2024-04-25T10:29:10.026183+0000","last_fullsized":"2024-04-25T10:29:10.026183+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:29:07.995096+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:29:07.995096+0000","last_clean_scrub_stamp":"2024-04-25T10:29:07.995096+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T22:08:00.367431+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.25","version":"0'0","reported_seq":12,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-04-25T10:29:10.189130+0000","last_change":"2024-04-25T10:29:09.023357+0000","last_active":"2024-04-25T10:29:10.189130+0000","last_peered":"2024-04-25T10:29:10.189130+0000","last_clean":"2024-04-25T10:29:10.189130+0000","last_became_active":"2024-04-25T10:29:09.022988+0000","last_became_peered":"2024-04-25T10:29:09.022988+0000","last_unstale":"2024-04-25T10:29:10.189130+0000","last_undegraded":"2024-04-25T10:29:10.189130+0000","last_fullsized":"2024-04-25T10:29:10.189130+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:29:07.995096+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:29:07.995096+0000","last_clean_scrub_stamp":"2024-04-25T10:29:07.995096+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T21:14:39.661957+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.24","version":"0'0","reported_seq":12,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-04-25T10:29:10.189239+0000","last_change":"2024-04-25T10:29:09.034328+0000","last_active":"2024-04-25T10:29:10.189239+0000","last_peered":"2024-04-25T10:29:10.189239+0000","last_clean":"2024-04-25T10:29:10.189239+0000","last_became_active":"2024-04-25T10:29:09.033978+0000","last_became_peered":"2024-04-25T10:29:09.033978+0000","last_unstale":"2024-04-25T10:29:10.189239+0000","last_undegraded":"2024-04-25T10:29:10.189239+0000","last_fullsized":"2024-04-25T10:29:10.189239+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:29:07.995096+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:29:07.995096+0000","last_clean_scrub_stamp":"2024-04-25T10:29:07.995096+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T21:24:14.497242+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.23","version":"0'0","reported_seq":12,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-04-25T10:29:10.189583+0000","last_change":"2024-04-25T10:29:09.034045+0000","last_active":"2024-04-25T10:29:10.189583+0000","last_peered":"2024-04-25T10:29:10.189583+0000","last_clean":"2024-04-25T10:29:10.189583+0000","last_became_active":"2024-04-25T10:29:09.033730+0000","last_became_peered":"2024-04-25T10:29:09.033730+0000","last_unstale":"2024-04-25T10:29:10.189583+0000","last_undegraded":"2024-04-25T10:29:10.189583+0000","last_fullsized":"2024-04-25T10:29:10.189583+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:29:07.995096+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:29:07.995096+0000","last_clean_scrub_stamp":"2024-04-25T10:29:07.995096+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T12:36:31.191100+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.22","version":"0'0","reported_seq":12,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-04-25T10:29:10.024663+0000","last_change":"2024-04-25T10:29:09.034262+0000","last_active":"2024-04-25T10:29:10.024663+0000","last_peered":"2024-04-25T10:29:10.024663+0000","last_clean":"2024-04-25T10:29:10.024663+0000","last_became_active":"2024-04-25T10:29:09.033905+0000","last_became_peered":"2024-04-25T10:29:09.033905+0000","last_unstale":"2024-04-25T10:29:10.024663+0000","last_undegraded":"2024-04-25T10:29:10.024663+0000","last_fullsized":"2024-04-25T10:29:10.024663+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:29:07.995096+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:29:07.995096+0000","last_clean_scrub_stamp":"2024-04-25T10:29:07.995096+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T16:22:45.608928+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.21","version":"0'0","reported_seq":12,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-04-25T10:29:10.188201+0000","last_change":"2024-04-25T10:29:09.027088+0000","last_active":"2024-04-25T10:29:10.188201+0000","last_peered":"2024-04-25T10:29:10.188201+0000","last_clean":"2024-04-25T10:29:10.188201+0000","last_became_active":"2024-04-25T10:29:09.026651+0000","last_became_peered":"2024-04-25T10:29:09.026651+0000","last_unstale":"2024-04-25T10:29:10.188201+0000","last_undegraded":"2024-04-25T10:29:10.188201+0000","last_fullsized":"2024-04-25T10:29:10.188201+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:29:07.995096+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:29:07.995096+0000","last_clean_scrub_stamp":"2024-04-25T10:29:07.995096+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T15:37:06.723380+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.20","version":"0'0","reported_seq":12,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-04-25T10:29:10.025741+0000","last_change":"2024-04-25T10:29:09.028008+0000","last_active":"2024-04-25T10:29:10.025741+0000","last_peered":"2024-04-25T10:29:10.025741+0000","last_clean":"2024-04-25T10:29:10.025741+0000","last_became_active":"2024-04-25T10:29:09.027679+0000","last_became_peered":"2024-04-25T10:29:09.027679+0000","last_unstale":"2024-04-25T10:29:10.025741+0000","last_undegraded":"2024-04-25T10:29:10.025741+0000","last_fullsized":"2024-04-25T10:29:10.025741+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:29:07.995096+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:29:07.995096+0000","last_clean_scrub_stamp":"2024-04-25T10:29:07.995096+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T15:37:06.723380+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,6],"acting":[2,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.1a","version":"0'0","reported_seq":12,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-04-25T10:29:10.187787+0000","last_change":"2024-04-25T10:29:09.032819+0000","last_active":"2024-04-25T10:29:10.187787+0000","last_peered":"2024-04-25T10:29:10.187787+0000","last_clean":"2024-04-25T10:29:10.187787+0000","last_became_active":"2024-04-25T10:29:09.032493+0000","last_became_peered":"2024-04-25T10:29:09.032493+0000","last_unstale":"2024-04-25T10:29:10.187787+0000","last_undegraded":"2024-04-25T10:29:10.187787+0000","last_fullsized":"2024-04-25T10:29:10.187787+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:29:07.995096+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:29:07.995096+0000","last_clean_scrub_stamp":"2024-04-25T10:29:07.995096+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T15:37:06.723380+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.1b","version":"22'1","reported_seq":15,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-04-25T10:29:10.189463+0000","last_change":"2024-04-25T10:29:08.025272+0000","last_active":"2024-04-25T10:29:10.189463+0000","last_peered":"2024-04-25T10:29:10.189463+0000","last_clean":"2024-04-25T10:29:10.189463+0000","last_became_active":"2024-04-25T10:29:08.024912+0000","last_became_peered":"2024-04-25T10:29:08.024912+0000","last_unstale":"2024-04-25T10:29:10.189463+0000","last_undegraded":"2024-04-25T10:29:10.189463+0000","last_fullsized":"2024-04-25T10:29:10.189463+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:29:06.989570+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:29:06.989570+0000","last_clean_scrub_stamp":"2024-04-25T10:29:06.989570+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T18:11:32.758179+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6],"acting":[5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.1b","version":"0'0","reported_seq":12,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-04-25T10:29:10.025596+0000","last_change":"2024-04-25T10:29:09.032005+0000","last_active":"2024-04-25T10:29:10.025596+0000","last_peered":"2024-04-25T10:29:10.025596+0000","last_clean":"2024-04-25T10:29:10.025596+0000","last_became_active":"2024-04-25T10:29:09.031572+0000","last_became_peered":"2024-04-25T10:29:09.031572+0000","last_unstale":"2024-04-25T10:29:10.025596+0000","last_undegraded":"2024-04-25T10:29:10.025596+0000","last_fullsized":"2024-04-25T10:29:10.025596+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:29:07.995096+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:29:07.995096+0000","last_clean_scrub_stamp":"2024-04-25T10:29:07.995096+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T12:47:09.477097+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.1a","version":"22'1","reported_seq":15,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-04-25T10:29:10.024506+0000","last_change":"2024-04-25T10:29:08.026018+0000","last_active":"2024-04-25T10:29:10.024506+0000","last_peered":"2024-04-25T10:29:10.024506+0000","last_clean":"2024-04-25T10:29:10.024506+0000","last_became_active":"2024-04-25T10:29:08.025668+0000","last_became_peered":"2024-04-25T10:29:08.025668+0000","last_unstale":"2024-04-25T10:29:10.024506+0000","last_undegraded":"2024-04-25T10:29:10.024506+0000","last_fullsized":"2024-04-25T10:29:10.024506+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:29:06.989570+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:29:06.989570+0000","last_clean_scrub_stamp":"2024-04-25T10:29:06.989570+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T11:35:26.461348+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.19","version":"0'0","reported_seq":14,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-04-25T10:29:10.188333+0000","last_change":"2024-04-25T10:29:08.011702+0000","last_active":"2024-04-25T10:29:10.188333+0000","last_peered":"2024-04-25T10:29:10.188333+0000","last_clean":"2024-04-25T10:29:10.188333+0000","last_became_active":"2024-04-25T10:29:08.011353+0000","last_became_peered":"2024-04-25T10:29:08.011353+0000","last_unstale":"2024-04-25T10:29:10.188333+0000","last_undegraded":"2024-04-25T10:29:10.188333+0000","last_fullsized":"2024-04-25T10:29:10.188333+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:29:06.989570+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:29:06.989570+0000","last_clean_scrub_stamp":"2024-04-25T10:29:06.989570+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T16:31:11.685101+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.18","version":"0'0","reported_seq":12,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-04-25T10:29:10.189830+0000","last_change":"2024-04-25T10:29:09.033872+0000","last_active":"2024-04-25T10:29:10.189830+0000","last_peered":"2024-04-25T10:29:10.189830+0000","last_clean":"2024-04-25T10:29:10.189830+0000","last_became_active":"2024-04-25T10:29:09.033544+0000","last_became_peered":"2024-04-25T10:29:09.033544+0000","last_unstale":"2024-04-25T10:29:10.189830+0000","last_undegraded":"2024-04-25T10:29:10.189830+0000","last_fullsized":"2024-04-25T10:29:10.189830+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:29:07.995096+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:29:07.995096+0000","last_clean_scrub_stamp":"2024-04-25T10:29:07.995096+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T13:54:22.968216+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6],"acting":[5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.18","version":"0'0","reported_seq":14,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-04-25T10:29:10.024423+0000","last_change":"2024-04-25T10:29:08.013340+0000","last_active":"2024-04-25T10:29:10.024423+0000","last_peered":"2024-04-25T10:29:10.024423+0000","last_clean":"2024-04-25T10:29:10.024423+0000","last_became_active":"2024-04-25T10:29:08.012940+0000","last_became_peered":"2024-04-25T10:29:08.012940+0000","last_unstale":"2024-04-25T10:29:10.024423+0000","last_undegraded":"2024-04-25T10:29:10.024423+0000","last_fullsized":"2024-04-25T10:29:10.024423+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:29:06.989570+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:29:06.989570+0000","last_clean_scrub_stamp":"2024-04-25T10:29:06.989570+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T10:35:30.355988+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.19","version":"0'0","reported_seq":12,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-04-25T10:29:10.025603+0000","last_change":"2024-04-25T10:29:09.021718+0000","last_active":"2024-04-25T10:29:10.025603+0000","last_peered":"2024-04-25T10:29:10.025603+0000","last_clean":"2024-04-25T10:29:10.025603+0000","last_became_active":"2024-04-25T10:29:09.021416+0000","last_became_peered":"2024-04-25T10:29:09.021416+0000","last_unstale":"2024-04-25T10:29:10.025603+0000","last_undegraded":"2024-04-25T10:29:10.025603+0000","last_fullsized":"2024-04-25T10:29:10.025603+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:29:07.995096+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:29:07.995096+0000","last_clean_scrub_stamp":"2024-04-25T10:29:07.995096+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T18:55:45.597763+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.17","version":"0'0","reported_seq":14,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-04-25T10:29:10.187949+0000","last_change":"2024-04-25T10:29:08.020281+0000","last_active":"2024-04-25T10:29:10.187949+0000","last_peered":"2024-04-25T10:29:10.187949+0000","last_clean":"2024-04-25T10:29:10.187949+0000","last_became_active":"2024-04-25T10:29:08.019921+0000","last_became_peered":"2024-04-25T10:29:08.019921+0000","last_unstale":"2024-04-25T10:29:10.187949+0000","last_undegraded":"2024-04-25T10:29:10.187949+0000","last_fullsized":"2024-04-25T10:29:10.187949+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:29:06.989570+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:29:06.989570+0000","last_clean_scrub_stamp":"2024-04-25T10:29:06.989570+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T19:13:07.068780+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.16","version":"0'0","reported_seq":12,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-04-25T10:29:10.026197+0000","last_change":"2024-04-25T10:29:09.022744+0000","last_active":"2024-04-25T10:29:10.026197+0000","last_peered":"2024-04-25T10:29:10.026197+0000","last_clean":"2024-04-25T10:29:10.026197+0000","last_became_active":"2024-04-25T10:29:09.022392+0000","last_became_peered":"2024-04-25T10:29:09.022392+0000","last_unstale":"2024-04-25T10:29:10.026197+0000","last_undegraded":"2024-04-25T10:29:10.026197+0000","last_fullsized":"2024-04-25T10:29:10.026197+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:29:07.995096+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:29:07.995096+0000","last_clean_scrub_stamp":"2024-04-25T10:29:07.995096+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T11:35:27.466874+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.16","version":"0'0","reported_seq":14,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-04-25T10:29:10.037869+0000","last_change":"2024-04-25T10:29:08.026388+0000","last_active":"2024-04-25T10:29:10.037869+0000","last_peered":"2024-04-25T10:29:10.037869+0000","last_clean":"2024-04-25T10:29:10.037869+0000","last_became_active":"2024-04-25T10:29:08.026039+0000","last_became_peered":"2024-04-25T10:29:08.026039+0000","last_unstale":"2024-04-25T10:29:10.037869+0000","last_undegraded":"2024-04-25T10:29:10.037869+0000","last_fullsized":"2024-04-25T10:29:10.037869+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:29:06.989570+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:29:06.989570+0000","last_clean_scrub_stamp":"2024-04-25T10:29:06.989570+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T18:11:32.758179+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.17","version":"0'0","reported_seq":12,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-04-25T10:29:10.189465+0000","last_change":"2024-04-25T10:29:09.032902+0000","last_active":"2024-04-25T10:29:10.189465+0000","last_peered":"2024-04-25T10:29:10.189465+0000","last_clean":"2024-04-25T10:29:10.189465+0000","last_became_active":"2024-04-25T10:29:09.032596+0000","last_became_peered":"2024-04-25T10:29:09.032596+0000","last_unstale":"2024-04-25T10:29:10.189465+0000","last_undegraded":"2024-04-25T10:29:10.189465+0000","last_fullsized":"2024-04-25T10:29:10.189465+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:29:07.995096+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:29:07.995096+0000","last_clean_scrub_stamp":"2024-04-25T10:29:07.995096+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T19:14:48.737269+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.15","version":"22'3","reported_seq":17,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-04-25T10:29:10.188352+0000","last_change":"2024-04-25T10:29:08.021347+0000","last_active":"2024-04-25T10:29:10.188352+0000","last_peered":"2024-04-25T10:29:10.188352+0000","last_clean":"2024-04-25T10:29:10.188352+0000","last_became_active":"2024-04-25T10:29:08.021039+0000","last_became_peered":"2024-04-25T10:29:08.021039+0000","last_unstale":"2024-04-25T10:29:10.188352+0000","last_undegraded":"2024-04-25T10:29:10.188352+0000","last_fullsized":"2024-04-25T10:29:10.188352+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:29:06.989570+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:29:06.989570+0000","last_clean_scrub_stamp":"2024-04-25T10:29:06.989570+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T20:22:56.380539+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.14","version":"0'0","reported_seq":12,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-04-25T10:29:10.024492+0000","last_change":"2024-04-25T10:29:09.021410+0000","last_active":"2024-04-25T10:29:10.024492+0000","last_peered":"2024-04-25T10:29:10.024492+0000","last_clean":"2024-04-25T10:29:10.024492+0000","last_became_active":"2024-04-25T10:29:09.021051+0000","last_became_peered":"2024-04-25T10:29:09.021051+0000","last_unstale":"2024-04-25T10:29:10.024492+0000","last_undegraded":"2024-04-25T10:29:10.024492+0000","last_fullsized":"2024-04-25T10:29:10.024492+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:29:07.995096+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:29:07.995096+0000","last_clean_scrub_stamp":"2024-04-25T10:29:07.995096+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T13:22:13.080513+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.14","version":"22'6","reported_seq":20,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-04-25T10:29:10.025912+0000","last_change":"2024-04-25T10:29:08.018044+0000","last_active":"2024-04-25T10:29:10.025912+0000","last_peered":"2024-04-25T10:29:10.025912+0000","last_clean":"2024-04-25T10:29:10.025912+0000","last_became_active":"2024-04-25T10:29:08.016984+0000","last_became_peered":"2024-04-25T10:29:08.016984+0000","last_unstale":"2024-04-25T10:29:10.025912+0000","last_undegraded":"2024-04-25T10:29:10.025912+0000","last_fullsized":"2024-04-25T10:29:10.025912+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:29:06.989570+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:29:06.989570+0000","last_clean_scrub_stamp":"2024-04-25T10:29:06.989570+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T18:29:56.525236+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1612,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4],"acting":[2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.15","version":"0'0","reported_seq":12,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-04-25T10:29:10.037668+0000","last_change":"2024-04-25T10:29:09.029034+0000","last_active":"2024-04-25T10:29:10.037668+0000","last_peered":"2024-04-25T10:29:10.037668+0000","last_clean":"2024-04-25T10:29:10.037668+0000","last_became_active":"2024-04-25T10:29:09.028662+0000","last_became_peered":"2024-04-25T10:29:09.028662+0000","last_unstale":"2024-04-25T10:29:10.037668+0000","last_undegraded":"2024-04-25T10:29:10.037668+0000","last_fullsized":"2024-04-25T10:29:10.037668+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:29:07.995096+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:29:07.995096+0000","last_clean_scrub_stamp":"2024-04-25T10:29:07.995096+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T17:30:54.093151+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.13","version":"22'2","reported_seq":16,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-04-25T10:29:10.189430+0000","last_change":"2024-04-25T10:29:08.020476+0000","last_active":"2024-04-25T10:29:10.189430+0000","last_peered":"2024-04-25T10:29:10.189430+0000","last_clean":"2024-04-25T10:29:10.189430+0000","last_became_active":"2024-04-25T10:29:08.020096+0000","last_became_peered":"2024-04-25T10:29:08.020096+0000","last_unstale":"2024-04-25T10:29:10.189430+0000","last_undegraded":"2024-04-25T10:29:10.189430+0000","last_fullsized":"2024-04-25T10:29:10.189430+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:29:06.989570+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:29:06.989570+0000","last_clean_scrub_stamp":"2024-04-25T10:29:06.989570+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T10:32:06.103123+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":34,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":7,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.12","version":"0'0","reported_seq":12,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-04-25T10:29:10.188838+0000","last_change":"2024-04-25T10:29:09.027996+0000","last_active":"2024-04-25T10:29:10.188838+0000","last_peered":"2024-04-25T10:29:10.188838+0000","last_clean":"2024-04-25T10:29:10.188838+0000","last_became_active":"2024-04-25T10:29:09.027538+0000","last_became_peered":"2024-04-25T10:29:09.027538+0000","last_unstale":"2024-04-25T10:29:10.188838+0000","last_undegraded":"2024-04-25T10:29:10.188838+0000","last_fullsized":"2024-04-25T10:29:10.188838+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:29:07.995096+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:29:07.995096+0000","last_clean_scrub_stamp":"2024-04-25T10:29:07.995096+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T12:16:47.983013+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.12","version":"0'0","reported_seq":14,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-04-25T10:29:10.024603+0000","last_change":"2024-04-25T10:29:08.019028+0000","last_active":"2024-04-25T10:29:10.024603+0000","last_peered":"2024-04-25T10:29:10.024603+0000","last_clean":"2024-04-25T10:29:10.024603+0000","last_became_active":"2024-04-25T10:29:08.018582+0000","last_became_peered":"2024-04-25T10:29:08.018582+0000","last_unstale":"2024-04-25T10:29:10.024603+0000","last_undegraded":"2024-04-25T10:29:10.024603+0000","last_fullsized":"2024-04-25T10:29:10.024603+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:29:06.989570+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:29:06.989570+0000","last_clean_scrub_stamp":"2024-04-25T10:29:06.989570+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T19:23:11.583888+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.13","version":"0'0","reported_seq":12,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-04-25T10:29:10.024281+0000","last_change":"2024-04-25T10:29:09.023995+0000","last_active":"2024-04-25T10:29:10.024281+0000","last_peered":"2024-04-25T10:29:10.024281+0000","last_clean":"2024-04-25T10:29:10.024281+0000","last_became_active":"2024-04-25T10:29:09.023690+0000","last_became_peered":"2024-04-25T10:29:09.023690+0000","last_unstale":"2024-04-25T10:29:10.024281+0000","last_undegraded":"2024-04-25T10:29:10.024281+0000","last_fullsized":"2024-04-25T10:29:10.024281+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:29:07.995096+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:29:07.995096+0000","last_clean_scrub_stamp":"2024-04-25T10:29:07.995096+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T15:28:23.689564+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.11","version":"0'0","reported_seq":14,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-04-25T10:29:10.037576+0000","last_change":"2024-04-25T10:29:08.027174+0000","last_active":"2024-04-25T10:29:10.037576+0000","last_peered":"2024-04-25T10:29:10.037576+0000","last_clean":"2024-04-25T10:29:10.037576+0000","last_became_active":"2024-04-25T10:29:08.026809+0000","last_became_peered":"2024-04-25T10:29:08.026809+0000","last_unstale":"2024-04-25T10:29:10.037576+0000","last_undegraded":"2024-04-25T10:29:10.037576+0000","last_fullsized":"2024-04-25T10:29:10.037576+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:29:06.989570+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:29:06.989570+0000","last_clean_scrub_stamp":"2024-04-25T10:29:06.989570+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T10:35:30.355988+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.10","version":"0'0","reported_seq":12,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-04-25T10:29:10.187809+0000","last_change":"2024-04-25T10:29:09.041722+0000","last_active":"2024-04-25T10:29:10.187809+0000","last_peered":"2024-04-25T10:29:10.187809+0000","last_clean":"2024-04-25T10:29:10.187809+0000","last_became_active":"2024-04-25T10:29:09.040971+0000","last_became_peered":"2024-04-25T10:29:09.040971+0000","last_unstale":"2024-04-25T10:29:10.187809+0000","last_undegraded":"2024-04-25T10:29:10.187809+0000","last_fullsized":"2024-04-25T10:29:10.187809+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:29:07.995096+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:29:07.995096+0000","last_clean_scrub_stamp":"2024-04-25T10:29:07.995096+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T10:35:31.361514+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.10","version":"22'8","reported_seq":22,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-04-25T10:29:10.025277+0000","last_change":"2024-04-25T10:29:08.014110+0000","last_active":"2024-04-25T10:29:10.025277+0000","last_peered":"2024-04-25T10:29:10.025277+0000","last_clean":"2024-04-25T10:29:10.025277+0000","last_became_active":"2024-04-25T10:29:08.013543+0000","last_became_peered":"2024-04-25T10:29:08.013543+0000","last_unstale":"2024-04-25T10:29:10.025277+0000","last_undegraded":"2024-04-25T10:29:10.025277+0000","last_fullsized":"2024-04-25T10:29:10.025277+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:29:06.989570+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:29:06.989570+0000","last_clean_scrub_stamp":"2024-04-25T10:29:06.989570+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T16:44:30.899898+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":12,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7],"acting":[4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.11","version":"0'0","reported_seq":12,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-04-25T10:29:10.024238+0000","last_change":"2024-04-25T10:29:09.020490+0000","last_active":"2024-04-25T10:29:10.024238+0000","last_peered":"2024-04-25T10:29:10.024238+0000","last_clean":"2024-04-25T10:29:10.024238+0000","last_became_active":"2024-04-25T10:29:09.020178+0000","last_became_peered":"2024-04-25T10:29:09.020178+0000","last_unstale":"2024-04-25T10:29:10.024238+0000","last_undegraded":"2024-04-25T10:29:10.024238+0000","last_fullsized":"2024-04-25T10:29:10.024238+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:29:07.995096+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:29:07.995096+0000","last_clean_scrub_stamp":"2024-04-25T10:29:07.995096+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T20:37:20.726766+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.f","version":"22'1","reported_seq":15,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-04-25T10:29:10.038104+0000","last_change":"2024-04-25T10:29:08.029879+0000","last_active":"2024-04-25T10:29:10.038104+0000","last_peered":"2024-04-25T10:29:10.038104+0000","last_clean":"2024-04-25T10:29:10.038104+0000","last_became_active":"2024-04-25T10:29:08.029549+0000","last_became_peered":"2024-04-25T10:29:08.029549+0000","last_unstale":"2024-04-25T10:29:10.038104+0000","last_undegraded":"2024-04-25T10:29:10.038104+0000","last_fullsized":"2024-04-25T10:29:10.038104+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:29:06.989570+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:29:06.989570+0000","last_clean_scrub_stamp":"2024-04-25T10:29:06.989570+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T15:37:05.717854+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.e","version":"0'0","reported_seq":12,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-04-25T10:29:10.026273+0000","last_change":"2024-04-25T10:29:09.033703+0000","last_active":"2024-04-25T10:29:10.026273+0000","last_peered":"2024-04-25T10:29:10.026273+0000","last_clean":"2024-04-25T10:29:10.026273+0000","last_became_active":"2024-04-25T10:29:09.033318+0000","last_became_peered":"2024-04-25T10:29:09.033318+0000","last_unstale":"2024-04-25T10:29:10.026273+0000","last_undegraded":"2024-04-25T10:29:10.026273+0000","last_fullsized":"2024-04-25T10:29:10.026273+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:29:07.995096+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:29:07.995096+0000","last_clean_scrub_stamp":"2024-04-25T10:29:07.995096+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T12:16:47.983013+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.e","version":"22'2","reported_seq":16,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-04-25T10:29:10.026042+0000","last_change":"2024-04-25T10:29:08.020703+0000","last_active":"2024-04-25T10:29:10.026042+0000","last_peered":"2024-04-25T10:29:10.026042+0000","last_clean":"2024-04-25T10:29:10.026042+0000","last_became_active":"2024-04-25T10:29:08.020321+0000","last_became_peered":"2024-04-25T10:29:08.020321+0000","last_unstale":"2024-04-25T10:29:10.026042+0000","last_undegraded":"2024-04-25T10:29:10.026042+0000","last_fullsized":"2024-04-25T10:29:10.026042+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:29:06.989570+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:29:06.989570+0000","last_clean_scrub_stamp":"2024-04-25T10:29:06.989570+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T15:42:07.059913+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5],"acting":[4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.f","version":"0'0","reported_seq":12,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-04-25T10:29:10.038152+0000","last_change":"2024-04-25T10:29:09.022708+0000","last_active":"2024-04-25T10:29:10.038152+0000","last_peered":"2024-04-25T10:29:10.038152+0000","last_clean":"2024-04-25T10:29:10.038152+0000","last_became_active":"2024-04-25T10:29:09.020856+0000","last_became_peered":"2024-04-25T10:29:09.020856+0000","last_unstale":"2024-04-25T10:29:10.038152+0000","last_undegraded":"2024-04-25T10:29:10.038152+0000","last_fullsized":"2024-04-25T10:29:10.038152+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:29:07.995096+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:29:07.995096+0000","last_clean_scrub_stamp":"2024-04-25T10:29:07.995096+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T15:28:23.689564+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.d","version":"22'1","reported_seq":15,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-04-25T10:29:10.189469+0000","last_change":"2024-04-25T10:29:08.019488+0000","last_active":"2024-04-25T10:29:10.189469+0000","last_peered":"2024-04-25T10:29:10.189469+0000","last_clean":"2024-04-25T10:29:10.189469+0000","last_became_active":"2024-04-25T10:29:08.018971+0000","last_became_peered":"2024-04-25T10:29:08.018971+0000","last_unstale":"2024-04-25T10:29:10.189469+0000","last_undegraded":"2024-04-25T10:29:10.189469+0000","last_fullsized":"2024-04-25T10:29:10.189469+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:29:06.989570+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:29:06.989570+0000","last_clean_scrub_stamp":"2024-04-25T10:29:06.989570+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T20:22:56.380539+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":70,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.c","version":"0'0","reported_seq":12,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-04-25T10:29:10.037902+0000","last_change":"2024-04-25T10:29:09.029666+0000","last_active":"2024-04-25T10:29:10.037902+0000","last_peered":"2024-04-25T10:29:10.037902+0000","last_clean":"2024-04-25T10:29:10.037902+0000","last_became_active":"2024-04-25T10:29:09.029184+0000","last_became_peered":"2024-04-25T10:29:09.029184+0000","last_unstale":"2024-04-25T10:29:10.037902+0000","last_undegraded":"2024-04-25T10:29:10.037902+0000","last_fullsized":"2024-04-25T10:29:10.037902+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:29:07.995096+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:29:07.995096+0000","last_clean_scrub_stamp":"2024-04-25T10:29:07.995096+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T13:54:22.968216+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.c","version":"0'0","reported_seq":14,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-04-25T10:29:10.189541+0000","last_change":"2024-04-25T10:29:08.012124+0000","last_active":"2024-04-25T10:29:10.189541+0000","last_peered":"2024-04-25T10:29:10.189541+0000","last_clean":"2024-04-25T10:29:10.189541+0000","last_became_active":"2024-04-25T10:29:08.011817+0000","last_became_peered":"2024-04-25T10:29:08.011817+0000","last_unstale":"2024-04-25T10:29:10.189541+0000","last_undegraded":"2024-04-25T10:29:10.189541+0000","last_fullsized":"2024-04-25T10:29:10.189541+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:29:06.989570+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:29:06.989570+0000","last_clean_scrub_stamp":"2024-04-25T10:29:06.989570+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T20:22:56.380539+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.d","version":"0'0","reported_seq":12,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-04-25T10:29:10.024815+0000","last_change":"2024-04-25T10:29:09.017227+0000","last_active":"2024-04-25T10:29:10.024815+0000","last_peered":"2024-04-25T10:29:10.024815+0000","last_clean":"2024-04-25T10:29:10.024815+0000","last_became_active":"2024-04-25T10:29:09.016911+0000","last_became_peered":"2024-04-25T10:29:09.016911+0000","last_unstale":"2024-04-25T10:29:10.024815+0000","last_undegraded":"2024-04-25T10:29:10.024815+0000","last_fullsized":"2024-04-25T10:29:10.024815+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:29:07.995096+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:29:07.995096+0000","last_clean_scrub_stamp":"2024-04-25T10:29:07.995096+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T18:51:41.934269+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.0","version":"18'72","reported_seq":110,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-04-25T10:29:10.187517+0000","last_change":"2024-04-25T10:28:57.972696+0000","last_active":"2024-04-25T10:29:10.187517+0000","last_peered":"2024-04-25T10:29:10.187517+0000","last_clean":"2024-04-25T10:29:10.187517+0000","last_became_active":"2024-04-25T10:28:57.972373+0000","last_became_peered":"2024-04-25T10:28:57.972373+0000","last_unstale":"2024-04-25T10:29:10.187517+0000","last_undegraded":"2024-04-25T10:29:10.187517+0000","last_fullsized":"2024-04-25T10:29:10.187517+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:28:56.959439+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:28:56.959439+0000","last_clean_scrub_stamp":"2024-04-25T10:28:56.959439+0000","objects_scrubbed":0,"log_size":72,"log_dups_size":0,"ondisk_log_size":72,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T17:34:34.057408+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":66,"num_read_kb":55,"num_write":101,"num_write_kb":1628,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.3","version":"0'0","reported_seq":14,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-04-25T10:29:10.024562+0000","last_change":"2024-04-25T10:29:08.025080+0000","last_active":"2024-04-25T10:29:10.024562+0000","last_peered":"2024-04-25T10:29:10.024562+0000","last_clean":"2024-04-25T10:29:10.024562+0000","last_became_active":"2024-04-25T10:29:08.024747+0000","last_became_peered":"2024-04-25T10:29:08.024747+0000","last_unstale":"2024-04-25T10:29:10.024562+0000","last_undegraded":"2024-04-25T10:29:10.024562+0000","last_fullsized":"2024-04-25T10:29:10.024562+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:29:06.989570+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:29:06.989570+0000","last_clean_scrub_stamp":"2024-04-25T10:29:06.989570+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T22:07:59.361905+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.2","version":"0'0","reported_seq":12,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-04-25T10:29:10.025717+0000","last_change":"2024-04-25T10:29:09.023626+0000","last_active":"2024-04-25T10:29:10.025717+0000","last_peered":"2024-04-25T10:29:10.025717+0000","last_clean":"2024-04-25T10:29:10.025717+0000","last_became_active":"2024-04-25T10:29:09.020335+0000","last_became_peered":"2024-04-25T10:29:09.020335+0000","last_unstale":"2024-04-25T10:29:10.025717+0000","last_undegraded":"2024-04-25T10:29:10.025717+0000","last_fullsized":"2024-04-25T10:29:10.025717+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:29:07.995096+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:29:07.995096+0000","last_clean_scrub_stamp":"2024-04-25T10:29:07.995096+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T15:30:10.628413+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.0","version":"0'0","reported_seq":14,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-04-25T10:29:10.024309+0000","last_change":"2024-04-25T10:29:08.027479+0000","last_active":"2024-04-25T10:29:10.024309+0000","last_peered":"2024-04-25T10:29:10.024309+0000","last_clean":"2024-04-25T10:29:10.024309+0000","last_became_active":"2024-04-25T10:29:08.027178+0000","last_became_peered":"2024-04-25T10:29:08.027178+0000","last_unstale":"2024-04-25T10:29:10.024309+0000","last_undegraded":"2024-04-25T10:29:10.024309+0000","last_fullsized":"2024-04-25T10:29:10.024309+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:29:06.989570+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:29:06.989570+0000","last_clean_scrub_stamp":"2024-04-25T10:29:06.989570+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T19:54:16.697792+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.1","version":"0'0","reported_seq":12,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-04-25T10:29:10.188195+0000","last_change":"2024-04-25T10:29:09.028777+0000","last_active":"2024-04-25T10:29:10.188195+0000","last_peered":"2024-04-25T10:29:10.188195+0000","last_clean":"2024-04-25T10:29:10.188195+0000","last_became_active":"2024-04-25T10:29:09.027709+0000","last_became_peered":"2024-04-25T10:29:09.027709+0000","last_unstale":"2024-04-25T10:29:10.188195+0000","last_undegraded":"2024-04-25T10:29:10.188195+0000","last_fullsized":"2024-04-25T10:29:10.188195+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:29:07.995096+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:29:07.995096+0000","last_clean_scrub_stamp":"2024-04-25T10:29:07.995096+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T22:08:00.367431+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,2],"acting":[6,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.1","version":"0'0","reported_seq":14,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-04-25T10:29:10.188429+0000","last_change":"2024-04-25T10:29:08.020944+0000","last_active":"2024-04-25T10:29:10.188429+0000","last_peered":"2024-04-25T10:29:10.188429+0000","last_clean":"2024-04-25T10:29:10.188429+0000","last_became_active":"2024-04-25T10:29:08.020259+0000","last_became_peered":"2024-04-25T10:29:08.020259+0000","last_unstale":"2024-04-25T10:29:10.188429+0000","last_undegraded":"2024-04-25T10:29:10.188429+0000","last_fullsized":"2024-04-25T10:29:10.188429+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:29:06.989570+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:29:06.989570+0000","last_clean_scrub_stamp":"2024-04-25T10:29:06.989570+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T16:31:11.685101+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.0","version":"0'0","reported_seq":12,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-04-25T10:29:10.189305+0000","last_change":"2024-04-25T10:29:09.035455+0000","last_active":"2024-04-25T10:29:10.189305+0000","last_peered":"2024-04-25T10:29:10.189305+0000","last_clean":"2024-04-25T10:29:10.189305+0000","last_became_active":"2024-04-25T10:29:09.035130+0000","last_became_peered":"2024-04-25T10:29:09.035130+0000","last_unstale":"2024-04-25T10:29:10.189305+0000","last_undegraded":"2024-04-25T10:29:10.189305+0000","last_fullsized":"2024-04-25T10:29:10.189305+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:29:07.995096+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:29:07.995096+0000","last_clean_scrub_stamp":"2024-04-25T10:29:07.995096+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T11:52:24.415332+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.2","version":"0'0","reported_seq":14,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-04-25T10:29:10.024714+0000","last_change":"2024-04-25T10:29:08.021694+0000","last_active":"2024-04-25T10:29:10.024714+0000","last_peered":"2024-04-25T10:29:10.024714+0000","last_clean":"2024-04-25T10:29:10.024714+0000","last_became_active":"2024-04-25T10:29:08.021052+0000","last_became_peered":"2024-04-25T10:29:08.021052+0000","last_unstale":"2024-04-25T10:29:10.024714+0000","last_undegraded":"2024-04-25T10:29:10.024714+0000","last_fullsized":"2024-04-25T10:29:10.024714+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:29:06.989570+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:29:06.989570+0000","last_clean_scrub_stamp":"2024-04-25T10:29:06.989570+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T21:14:38.656431+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.3","version":"0'0","reported_seq":12,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-04-25T10:29:10.037784+0000","last_change":"2024-04-25T10:29:09.021686+0000","last_active":"2024-04-25T10:29:10.037784+0000","last_peered":"2024-04-25T10:29:10.037784+0000","last_clean":"2024-04-25T10:29:10.037784+0000","last_became_active":"2024-04-25T10:29:09.020672+0000","last_became_peered":"2024-04-25T10:29:09.020672+0000","last_unstale":"2024-04-25T10:29:10.037784+0000","last_undegraded":"2024-04-25T10:29:10.037784+0000","last_fullsized":"2024-04-25T10:29:10.037784+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:29:07.995096+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:29:07.995096+0000","last_clean_scrub_stamp":"2024-04-25T10:29:07.995096+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T20:52:02.098872+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.4","version":"0'0","reported_seq":14,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-04-25T10:29:10.189133+0000","last_change":"2024-04-25T10:29:08.025582+0000","last_active":"2024-04-25T10:29:10.189133+0000","last_peered":"2024-04-25T10:29:10.189133+0000","last_clean":"2024-04-25T10:29:10.189133+0000","last_became_active":"2024-04-25T10:29:08.024961+0000","last_became_peered":"2024-04-25T10:29:08.024961+0000","last_unstale":"2024-04-25T10:29:10.189133+0000","last_undegraded":"2024-04-25T10:29:10.189133+0000","last_fullsized":"2024-04-25T10:29:10.189133+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:29:06.989570+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:29:06.989570+0000","last_clean_scrub_stamp":"2024-04-25T10:29:06.989570+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T10:32:06.103123+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6],"acting":[5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.5","version":"0'0","reported_seq":12,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-04-25T10:29:10.026083+0000","last_change":"2024-04-25T10:29:09.028003+0000","last_active":"2024-04-25T10:29:10.026083+0000","last_peered":"2024-04-25T10:29:10.026083+0000","last_clean":"2024-04-25T10:29:10.026083+0000","last_became_active":"2024-04-25T10:29:09.027654+0000","last_became_peered":"2024-04-25T10:29:09.027654+0000","last_unstale":"2024-04-25T10:29:10.026083+0000","last_undegraded":"2024-04-25T10:29:10.026083+0000","last_fullsized":"2024-04-25T10:29:10.026083+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:29:07.995096+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:29:07.995096+0000","last_clean_scrub_stamp":"2024-04-25T10:29:07.995096+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T19:54:17.703318+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,6],"acting":[2,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.5","version":"0'0","reported_seq":14,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-04-25T10:29:10.024926+0000","last_change":"2024-04-25T10:29:08.026181+0000","last_active":"2024-04-25T10:29:10.024926+0000","last_peered":"2024-04-25T10:29:10.024926+0000","last_clean":"2024-04-25T10:29:10.024926+0000","last_became_active":"2024-04-25T10:29:08.025780+0000","last_became_peered":"2024-04-25T10:29:08.025780+0000","last_unstale":"2024-04-25T10:29:10.024926+0000","last_undegraded":"2024-04-25T10:29:10.024926+0000","last_fullsized":"2024-04-25T10:29:10.024926+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:29:06.989570+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:29:06.989570+0000","last_clean_scrub_stamp":"2024-04-25T10:29:06.989570+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T12:16:46.977487+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.4","version":"0'0","reported_seq":12,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-04-25T10:29:10.189423+0000","last_change":"2024-04-25T10:29:09.037824+0000","last_active":"2024-04-25T10:29:10.189423+0000","last_peered":"2024-04-25T10:29:10.189423+0000","last_clean":"2024-04-25T10:29:10.189423+0000","last_became_active":"2024-04-25T10:29:09.037478+0000","last_became_peered":"2024-04-25T10:29:09.037478+0000","last_unstale":"2024-04-25T10:29:10.189423+0000","last_undegraded":"2024-04-25T10:29:10.189423+0000","last_fullsized":"2024-04-25T10:29:10.189423+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:29:07.995096+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:29:07.995096+0000","last_clean_scrub_stamp":"2024-04-25T10:29:07.995096+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T15:02:17.480054+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.6","version":"23'1","reported_seq":15,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-04-25T10:29:10.026003+0000","last_change":"2024-04-25T10:29:08.020327+0000","last_active":"2024-04-25T10:29:10.026003+0000","last_peered":"2024-04-25T10:29:10.026003+0000","last_clean":"2024-04-25T10:29:10.026003+0000","last_became_active":"2024-04-25T10:29:08.019988+0000","last_became_peered":"2024-04-25T10:29:08.019988+0000","last_unstale":"2024-04-25T10:29:10.026003+0000","last_undegraded":"2024-04-25T10:29:10.026003+0000","last_fullsized":"2024-04-25T10:29:10.026003+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:29:06.989570+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:29:06.989570+0000","last_clean_scrub_stamp":"2024-04-25T10:29:06.989570+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T13:17:35.774897+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.7","version":"0'0","reported_seq":12,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-04-25T10:29:10.025861+0000","last_change":"2024-04-25T10:29:09.029850+0000","last_active":"2024-04-25T10:29:10.025861+0000","last_peered":"2024-04-25T10:29:10.025861+0000","last_clean":"2024-04-25T10:29:10.025861+0000","last_became_active":"2024-04-25T10:29:09.029534+0000","last_became_peered":"2024-04-25T10:29:09.029534+0000","last_unstale":"2024-04-25T10:29:10.025861+0000","last_undegraded":"2024-04-25T10:29:10.025861+0000","last_fullsized":"2024-04-25T10:29:10.025861+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:29:07.995096+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:29:07.995096+0000","last_clean_scrub_stamp":"2024-04-25T10:29:07.995096+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T15:03:07.958155+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.7","version":"22'2","reported_seq":16,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-04-25T10:29:10.024876+0000","last_change":"2024-04-25T10:29:08.026080+0000","last_active":"2024-04-25T10:29:10.024876+0000","last_peered":"2024-04-25T10:29:10.024876+0000","last_clean":"2024-04-25T10:29:10.024876+0000","last_became_active":"2024-04-25T10:29:08.025707+0000","last_became_peered":"2024-04-25T10:29:08.025707+0000","last_unstale":"2024-04-25T10:29:10.024876+0000","last_undegraded":"2024-04-25T10:29:10.024876+0000","last_fullsized":"2024-04-25T10:29:10.024876+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:29:06.989570+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:29:06.989570+0000","last_clean_scrub_stamp":"2024-04-25T10:29:06.989570+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T10:55:00.499631+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.6","version":"0'0","reported_seq":12,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-04-25T10:29:10.025829+0000","last_change":"2024-04-25T10:29:09.025814+0000","last_active":"2024-04-25T10:29:10.025829+0000","last_peered":"2024-04-25T10:29:10.025829+0000","last_clean":"2024-04-25T10:29:10.025829+0000","last_became_active":"2024-04-25T10:29:09.025474+0000","last_became_peered":"2024-04-25T10:29:09.025474+0000","last_unstale":"2024-04-25T10:29:10.025829+0000","last_undegraded":"2024-04-25T10:29:10.025829+0000","last_fullsized":"2024-04-25T10:29:10.025829+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:29:07.995096+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:29:07.995096+0000","last_clean_scrub_stamp":"2024-04-25T10:29:07.995096+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T12:35:12.945537+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7],"acting":[4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.8","version":"0'0","reported_seq":14,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-04-25T10:29:10.037271+0000","last_change":"2024-04-25T10:29:08.024845+0000","last_active":"2024-04-25T10:29:10.037271+0000","last_peered":"2024-04-25T10:29:10.037271+0000","last_clean":"2024-04-25T10:29:10.037271+0000","last_became_active":"2024-04-25T10:29:08.024484+0000","last_became_peered":"2024-04-25T10:29:08.024484+0000","last_unstale":"2024-04-25T10:29:10.037271+0000","last_undegraded":"2024-04-25T10:29:10.037271+0000","last_fullsized":"2024-04-25T10:29:10.037271+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:29:06.989570+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:29:06.989570+0000","last_clean_scrub_stamp":"2024-04-25T10:29:06.989570+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T14:12:57.950482+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.9","version":"0'0","reported_seq":12,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-04-25T10:29:10.024346+0000","last_change":"2024-04-25T10:29:09.022502+0000","last_active":"2024-04-25T10:29:10.024346+0000","last_peered":"2024-04-25T10:29:10.024346+0000","last_clean":"2024-04-25T10:29:10.024346+0000","last_became_active":"2024-04-25T10:29:09.022184+0000","last_became_peered":"2024-04-25T10:29:09.022184+0000","last_unstale":"2024-04-25T10:29:10.024346+0000","last_undegraded":"2024-04-25T10:29:10.024346+0000","last_fullsized":"2024-04-25T10:29:10.024346+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:29:07.995096+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:29:07.995096+0000","last_clean_scrub_stamp":"2024-04-25T10:29:07.995096+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T19:56:27.663748+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.9","version":"0'0","reported_seq":14,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-04-25T10:29:10.025738+0000","last_change":"2024-04-25T10:29:08.024989+0000","last_active":"2024-04-25T10:29:10.025738+0000","last_peered":"2024-04-25T10:29:10.025738+0000","last_clean":"2024-04-25T10:29:10.025738+0000","last_became_active":"2024-04-25T10:29:08.024655+0000","last_became_peered":"2024-04-25T10:29:08.024655+0000","last_unstale":"2024-04-25T10:29:10.025738+0000","last_undegraded":"2024-04-25T10:29:10.025738+0000","last_fullsized":"2024-04-25T10:29:10.025738+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:29:06.989570+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:29:06.989570+0000","last_clean_scrub_stamp":"2024-04-25T10:29:06.989570+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T21:05:51.419862+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.8","version":"0'0","reported_seq":12,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-04-25T10:29:10.025393+0000","last_change":"2024-04-25T10:29:09.023330+0000","last_active":"2024-04-25T10:29:10.025393+0000","last_peered":"2024-04-25T10:29:10.025393+0000","last_clean":"2024-04-25T10:29:10.025393+0000","last_became_active":"2024-04-25T10:29:09.020543+0000","last_became_peered":"2024-04-25T10:29:09.020543+0000","last_unstale":"2024-04-25T10:29:10.025393+0000","last_undegraded":"2024-04-25T10:29:10.025393+0000","last_fullsized":"2024-04-25T10:29:10.025393+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:29:07.995096+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:29:07.995096+0000","last_clean_scrub_stamp":"2024-04-25T10:29:07.995096+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T11:12:41.953761+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7],"acting":[4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.a","version":"0'0","reported_seq":14,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-04-25T10:29:10.188957+0000","last_change":"2024-04-25T10:29:08.020363+0000","last_active":"2024-04-25T10:29:10.188957+0000","last_peered":"2024-04-25T10:29:10.188957+0000","last_clean":"2024-04-25T10:29:10.188957+0000","last_became_active":"2024-04-25T10:29:08.020037+0000","last_became_peered":"2024-04-25T10:29:08.020037+0000","last_unstale":"2024-04-25T10:29:10.188957+0000","last_undegraded":"2024-04-25T10:29:10.188957+0000","last_fullsized":"2024-04-25T10:29:10.188957+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:29:06.989570+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:29:06.989570+0000","last_clean_scrub_stamp":"2024-04-25T10:29:06.989570+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T10:32:06.103123+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.b","version":"0'0","reported_seq":12,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-04-25T10:29:10.189294+0000","last_change":"2024-04-25T10:29:09.028820+0000","last_active":"2024-04-25T10:29:10.189294+0000","last_peered":"2024-04-25T10:29:10.189294+0000","last_clean":"2024-04-25T10:29:10.189294+0000","last_became_active":"2024-04-25T10:29:09.028426+0000","last_became_peered":"2024-04-25T10:29:09.028426+0000","last_unstale":"2024-04-25T10:29:10.189294+0000","last_undegraded":"2024-04-25T10:29:10.189294+0000","last_fullsized":"2024-04-25T10:29:10.189294+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:29:07.995096+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:29:07.995096+0000","last_clean_scrub_stamp":"2024-04-25T10:29:07.995096+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T19:54:17.703318+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.b","version":"22'2","reported_seq":16,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-04-25T10:29:10.024445+0000","last_change":"2024-04-25T10:29:08.025011+0000","last_active":"2024-04-25T10:29:10.024445+0000","last_peered":"2024-04-25T10:29:10.024445+0000","last_clean":"2024-04-25T10:29:10.024445+0000","last_became_active":"2024-04-25T10:29:08.024667+0000","last_became_peered":"2024-04-25T10:29:08.024667+0000","last_unstale":"2024-04-25T10:29:10.024445+0000","last_undegraded":"2024-04-25T10:29:10.024445+0000","last_fullsized":"2024-04-25T10:29:10.024445+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:29:06.989570+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:29:06.989570+0000","last_clean_scrub_stamp":"2024-04-25T10:29:06.989570+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T15:37:05.717854+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.a","version":"0'0","reported_seq":12,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-04-25T10:29:10.024394+0000","last_change":"2024-04-25T10:29:09.017160+0000","last_active":"2024-04-25T10:29:10.024394+0000","last_peered":"2024-04-25T10:29:10.024394+0000","last_clean":"2024-04-25T10:29:10.024394+0000","last_became_active":"2024-04-25T10:29:09.016821+0000","last_became_peered":"2024-04-25T10:29:09.016821+0000","last_unstale":"2024-04-25T10:29:10.024394+0000","last_undegraded":"2024-04-25T10:29:10.024394+0000","last_fullsized":"2024-04-25T10:29:10.024394+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:29:07.995096+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:29:07.995096+0000","last_clean_scrub_stamp":"2024-04-25T10:29:07.995096+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T20:40:19.840339+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.1d","version":"0'0","reported_seq":12,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-04-25T10:29:10.024622+0000","last_change":"2024-04-25T10:29:09.030341+0000","last_active":"2024-04-25T10:29:10.024622+0000","last_peered":"2024-04-25T10:29:10.024622+0000","last_clean":"2024-04-25T10:29:10.024622+0000","last_became_active":"2024-04-25T10:29:09.029988+0000","last_became_peered":"2024-04-25T10:29:09.029988+0000","last_unstale":"2024-04-25T10:29:10.024622+0000","last_undegraded":"2024-04-25T10:29:10.024622+0000","last_fullsized":"2024-04-25T10:29:10.024622+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:29:07.995096+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:29:07.995096+0000","last_clean_scrub_stamp":"2024-04-25T10:29:07.995096+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T11:40:18.624659+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.1c","version":"22'2","reported_seq":16,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-04-25T10:29:10.037969+0000","last_change":"2024-04-25T10:29:08.027197+0000","last_active":"2024-04-25T10:29:10.037969+0000","last_peered":"2024-04-25T10:29:10.037969+0000","last_clean":"2024-04-25T10:29:10.037969+0000","last_became_active":"2024-04-25T10:29:08.026872+0000","last_became_peered":"2024-04-25T10:29:08.026872+0000","last_unstale":"2024-04-25T10:29:10.037969+0000","last_undegraded":"2024-04-25T10:29:10.037969+0000","last_fullsized":"2024-04-25T10:29:10.037969+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:29:06.989570+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:29:06.989570+0000","last_clean_scrub_stamp":"2024-04-25T10:29:06.989570+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T15:58:35.843305+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.1c","version":"0'0","reported_seq":12,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-04-25T10:29:10.187853+0000","last_change":"2024-04-25T10:29:09.032731+0000","last_active":"2024-04-25T10:29:10.187853+0000","last_peered":"2024-04-25T10:29:10.187853+0000","last_clean":"2024-04-25T10:29:10.187853+0000","last_became_active":"2024-04-25T10:29:09.032383+0000","last_became_peered":"2024-04-25T10:29:09.032383+0000","last_unstale":"2024-04-25T10:29:10.187853+0000","last_undegraded":"2024-04-25T10:29:10.187853+0000","last_fullsized":"2024-04-25T10:29:10.187853+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:29:07.995096+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:29:07.995096+0000","last_clean_scrub_stamp":"2024-04-25T10:29:07.995096+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T19:54:17.703318+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.1d","version":"22'2","reported_seq":16,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-04-25T10:29:10.037807+0000","last_change":"2024-04-25T10:29:08.026410+0000","last_active":"2024-04-25T10:29:10.037807+0000","last_peered":"2024-04-25T10:29:10.037807+0000","last_clean":"2024-04-25T10:29:10.037807+0000","last_became_active":"2024-04-25T10:29:08.026071+0000","last_became_peered":"2024-04-25T10:29:08.026071+0000","last_unstale":"2024-04-25T10:29:10.037807+0000","last_undegraded":"2024-04-25T10:29:10.037807+0000","last_fullsized":"2024-04-25T10:29:10.037807+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:29:06.989570+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:29:06.989570+0000","last_clean_scrub_stamp":"2024-04-25T10:29:06.989570+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T19:54:16.697792+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.1f","version":"0'0","reported_seq":12,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-04-25T10:29:10.189240+0000","last_change":"2024-04-25T10:29:09.032735+0000","last_active":"2024-04-25T10:29:10.189240+0000","last_peered":"2024-04-25T10:29:10.189240+0000","last_clean":"2024-04-25T10:29:10.189240+0000","last_became_active":"2024-04-25T10:29:09.032227+0000","last_became_peered":"2024-04-25T10:29:09.032227+0000","last_unstale":"2024-04-25T10:29:10.189240+0000","last_undegraded":"2024-04-25T10:29:10.189240+0000","last_fullsized":"2024-04-25T10:29:10.189240+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:29:07.995096+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:29:07.995096+0000","last_clean_scrub_stamp":"2024-04-25T10:29:07.995096+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T11:31:45.283247+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.1e","version":"22'1","reported_seq":15,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-04-25T10:29:10.189728+0000","last_change":"2024-04-25T10:29:08.017025+0000","last_active":"2024-04-25T10:29:10.189728+0000","last_peered":"2024-04-25T10:29:10.189728+0000","last_clean":"2024-04-25T10:29:10.189728+0000","last_became_active":"2024-04-25T10:29:08.016702+0000","last_became_peered":"2024-04-25T10:29:08.016702+0000","last_unstale":"2024-04-25T10:29:10.189728+0000","last_undegraded":"2024-04-25T10:29:10.189728+0000","last_fullsized":"2024-04-25T10:29:10.189728+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:29:06.989570+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:29:06.989570+0000","last_clean_scrub_stamp":"2024-04-25T10:29:06.989570+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T14:12:57.950482+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.1e","version":"0'0","reported_seq":12,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-04-25T10:29:10.189730+0000","last_change":"2024-04-25T10:29:09.036208+0000","last_active":"2024-04-25T10:29:10.189730+0000","last_peered":"2024-04-25T10:29:10.189730+0000","last_clean":"2024-04-25T10:29:10.189730+0000","last_became_active":"2024-04-25T10:29:09.034890+0000","last_became_peered":"2024-04-25T10:29:09.034890+0000","last_unstale":"2024-04-25T10:29:10.189730+0000","last_undegraded":"2024-04-25T10:29:10.189730+0000","last_fullsized":"2024-04-25T10:29:10.189730+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:29:07.995096+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:29:07.995096+0000","last_clean_scrub_stamp":"2024-04-25T10:29:07.995096+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T18:44:23.610374+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.1f","version":"22'4","reported_seq":18,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-04-25T10:29:10.026053+0000","last_change":"2024-04-25T10:29:08.026202+0000","last_active":"2024-04-25T10:29:10.026053+0000","last_peered":"2024-04-25T10:29:10.026053+0000","last_clean":"2024-04-25T10:29:10.026053+0000","last_became_active":"2024-04-25T10:29:08.025877+0000","last_became_peered":"2024-04-25T10:29:08.025877+0000","last_unstale":"2024-04-25T10:29:10.026053+0000","last_undegraded":"2024-04-25T10:29:10.026053+0000","last_fullsized":"2024-04-25T10:29:10.026053+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:29:06.989570+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:29:06.989570+0000","last_clean_scrub_stamp":"2024-04-25T10:29:06.989570+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T20:00:40.198709+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":582,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]}],"pool_stats":[{"poolid":3,"num_pg":64,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":128,"acting":128,"num_store_stats":8},{"poolid":2,"num_pg":32,"stat_sum":{"num_bytes":2410,"num_objects":22,"num_object_clones":0,"num_object_copies":44,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":22,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":45,"num_write_kb":14,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":14,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":65536,"data_stored":4820,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":39,"ondisk_log_size":39,"up":64,"acting":64,"num_store_stats":8},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":66,"num_read_kb":55,"num_write":101,"num_write_kb":1628,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1187840,"data_stored":1180736,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":72,"ondisk_log_size":72,"up":2,"acting":2,"num_store_stats":2}],"osd_stats":[{"osd":7,"up_from":15,"seq":64424509446,"num_pgs":16,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27260,"kb_used_data":236,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344580,"statfs":{"total":96636764160,"available":96608849920,"internally_reserved":0,"allocated":241664,"data_stored":81359,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1590,"internal_metadata":27654602},"hb_peers":[0,1,2,3,4,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":6,"up_from":15,"seq":64424509447,"num_pgs":20,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27252,"kb_used_data":228,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344588,"statfs":{"total":96636764160,"available":96608858112,"internally_reserved":0,"allocated":233472,"data_stored":81235,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":27654598},"hb_peers":[0,1,2,3,4,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":5,"up_from":15,"seq":64424509448,"num_pgs":28,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27840,"kb_used_data":816,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344000,"statfs":{"total":96636764160,"available":96608256000,"internally_reserved":0,"allocated":835584,"data_stored":671695,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":3,"up_from":15,"seq":64424509448,"num_pgs":30,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27256,"kb_used_data":232,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344584,"statfs":{"total":96636764160,"available":96608854016,"internally_reserved":0,"allocated":237568,"data_stored":81817,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1593,"internal_metadata":27654599},"hb_peers":[0,1,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":2,"up_from":15,"seq":64424509448,"num_pgs":22,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27264,"kb_used_data":240,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344576,"statfs":{"total":96636764160,"available":96608845824,"internally_reserved":0,"allocated":245760,"data_stored":82847,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":1,"up_from":15,"seq":64424509446,"num_pgs":21,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27836,"kb_used_data":812,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344004,"statfs":{"total":96636764160,"available":96608260096,"internally_reserved":0,"allocated":831488,"data_stored":671673,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":0,"up_from":15,"seq":64424509447,"num_pgs":35,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27260,"kb_used_data":236,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344580,"statfs":{"total":96636764160,"available":96608849920,"internally_reserved":0,"allocated":241664,"data_stored":81291,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[1,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":4,"up_from":13,"seq":55834574854,"num_pgs":22,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27276,"kb_used_data":248,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344564,"statfs":{"total":96636764160,"available":96608833536,"internally_reserved":0,"allocated":253952,"data_stored":83519,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":5,"apply_latency_ms":5,"commit_latency_ns":5000000,"apply_latency_ns":5000000},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":56,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":70,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":1612,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":582,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":20480,"data_stored":2284,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":92,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":124,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-04-25T10:29:18.041 INFO:tasks.cephfs.filesystem.ceph_manager:clean! 2024-04-25T10:29:18.041 INFO:tasks.ceph:Waiting until ceph daemons up and pgs clean... 2024-04-25T10:29:18.041 INFO:tasks.ceph.ceph_manager.ceph:waiting for mgr available 2024-04-25T10:29:18.041 DEBUG:teuthology.orchestra.run.smithi037:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph mgr dump --format=json 2024-04-25T10:29:18.042 INFO:tasks.daemonwatchdog.daemon_watchdog:watchdog starting 2024-04-25T10:29:18.188 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:18.186+0000 7fb027972640 1 Processor -- start 2024-04-25T10:29:18.188 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:18.186+0000 7fb027972640 1 -- start start 2024-04-25T10:29:18.190 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:18.187+0000 7fb027972640 1 --2- >> v2:172.21.15.203:3300/0 conn(0x7fb0200d5430 0x7fb0200b75c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:29:18.190 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:18.187+0000 7fb027972640 1 -- --> v1:172.21.15.37:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fb0200b7f10 con 0x7fb020161110 2024-04-25T10:29:18.191 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:18.188+0000 7fb027972640 1 -- --> v1:172.21.15.203:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fb020161520 con 0x7fb0200b7b00 2024-04-25T10:29:18.191 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:18.188+0000 7fb027972640 1 -- --> v2:172.21.15.203:3300/0 -- mon_getmap magic: 0 -- 0x7fb0200d4420 con 0x7fb0200d5430 2024-04-25T10:29:18.191 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:18.188+0000 7fb025f0f640 1 --1- >> v1:172.21.15.37:6789/0 conn(0x7fb020161110 0x7fb020161880 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.37:6789/0 says I am v1:172.21.15.37:51450/0 (socket says 172.21.15.37:51450) 2024-04-25T10:29:18.191 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:18.188+0000 7fb025f0f640 1 -- 172.21.15.37:0/858987384 learned_addr learned my addr 172.21.15.37:0/858987384 (peer_addr_for_me v1:172.21.15.37:0/0) 2024-04-25T10:29:18.191 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:18.188+0000 7fb02570e640 1 --2- 172.21.15.37:0/858987384 >> v2:172.21.15.203:3300/0 conn(0x7fb0200d5430 0x7fb0200b75c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:29:18.191 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:18.188+0000 7fb017fff640 1 -- 172.21.15.37:0/858987384 <== mon.1 v1:172.21.15.203:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 1848713370 0 0) 0x7fb020161520 con 0x7fb0200b7b00 2024-04-25T10:29:18.191 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:18.188+0000 7fb017fff640 1 -- 172.21.15.37:0/858987384 --> v1:172.21.15.203:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7fb00c003520 con 0x7fb0200b7b00 2024-04-25T10:29:18.191 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:18.188+0000 7fb02570e640 1 -- 172.21.15.37:0/858987384 >> v1:172.21.15.203:6789/0 conn(0x7fb0200b7b00 legacy=0x7fb020160a00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:29:18.191 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:18.188+0000 7fb02570e640 1 -- 172.21.15.37:0/858987384 >> v1:172.21.15.37:6789/0 conn(0x7fb020161110 legacy=0x7fb020161880 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:29:18.191 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:18.188+0000 7fb02570e640 1 -- 172.21.15.37:0/858987384 --> v2:172.21.15.203:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fb0200d5840 con 0x7fb0200d5430 2024-04-25T10:29:18.192 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:18.189+0000 7fb02570e640 1 --2- 172.21.15.37:0/858987384 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fb0200d5430 0x7fb0200b75c0 secure :-1 s=READY pgs=62 cs=0 l=1 rev1=1 crypto rx=0x7fb0201616f0 tx=0x7fb01002a180 comp rx=0 tx=0).ready entity=mon.1 client_cookie=f76a715dfccbaf18 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:29:18.192 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:18.189+0000 7fb017fff640 1 -- 172.21.15.37:0/858987384 <== mon.1 v2:172.21.15.203:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb010037070 con 0x7fb0200d5430 2024-04-25T10:29:18.193 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:18.189+0000 7fb017fff640 1 -- 172.21.15.37:0/858987384 <== mon.1 v2:172.21.15.203:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fb010032440 con 0x7fb0200d5430 2024-04-25T10:29:18.193 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:18.189+0000 7fb017fff640 1 -- 172.21.15.37:0/858987384 <== mon.1 v2:172.21.15.203:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb010032810 con 0x7fb0200d5430 2024-04-25T10:29:18.193 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:18.190+0000 7fb027972640 1 -- 172.21.15.37:0/858987384 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fb0200d5430 msgr2=0x7fb0200b75c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:29:18.193 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:18.190+0000 7fb027972640 1 --2- 172.21.15.37:0/858987384 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fb0200d5430 0x7fb0200b75c0 secure :-1 s=READY pgs=62 cs=0 l=1 rev1=1 crypto rx=0x7fb0201616f0 tx=0x7fb01002a180 comp rx=0 tx=0).stop 2024-04-25T10:29:18.193 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:18.190+0000 7fb027972640 1 -- 172.21.15.37:0/858987384 shutdown_connections 2024-04-25T10:29:18.193 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:18.190+0000 7fb027972640 1 --2- 172.21.15.37:0/858987384 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fb0200d5430 0x7fb0200b75c0 unknown :-1 s=CLOSED pgs=62 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:29:18.193 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:18.190+0000 7fb027972640 1 -- 172.21.15.37:0/858987384 >> 172.21.15.37:0/858987384 conn(0x7fb0200d27a0 msgr2=0x7fb0200d2bb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:29:18.193 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:18.190+0000 7fb027972640 1 -- 172.21.15.37:0/858987384 shutdown_connections 2024-04-25T10:29:18.193 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:18.190+0000 7fb027972640 1 -- 172.21.15.37:0/858987384 wait complete. 2024-04-25T10:29:18.194 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:18.192+0000 7fb027972640 1 Processor -- start 2024-04-25T10:29:18.194 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:18.192+0000 7fb027972640 1 -- start start 2024-04-25T10:29:18.196 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:18.193+0000 7fb027972640 1 --2- >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7fb0200b7b00 0x7fb0200cf3e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:29:18.196 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:18.193+0000 7fb027972640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fb0200d5430 0x7fb0201f5c90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:29:18.197 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:18.193+0000 7fb027972640 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fb020161110 0x7fb0201e8a10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:29:18.197 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:18.193+0000 7fb027972640 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 -- 0x7fb0200d4c30 con 0x7fb0200d5430 2024-04-25T10:29:18.197 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:18.193+0000 7fb027972640 1 -- --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_getmap magic: 0 -- 0x7fb0200d4950 con 0x7fb020161110 2024-04-25T10:29:18.197 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:18.193+0000 7fb027972640 1 -- --> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] -- mon_getmap magic: 0 -- 0x7fb0200d4f10 con 0x7fb0200b7b00 2024-04-25T10:29:18.197 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:18.193+0000 7fb024f0d640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fb0200d5430 0x7fb0201f5c90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:29:18.197 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:18.193+0000 7fb024f0d640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fb0200d5430 0x7fb0201f5c90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.37:3300/0 says I am v2:172.21.15.37:46386/0 (socket says 172.21.15.37:46386) 2024-04-25T10:29:18.197 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:18.193+0000 7fb024f0d640 1 -- 172.21.15.37:0/3778608024 learned_addr learned my addr 172.21.15.37:0/3778608024 (peer_addr_for_me v2:172.21.15.37:0/0) 2024-04-25T10:29:18.197 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:18.193+0000 7fb02570e640 1 --2- 172.21.15.37:0/3778608024 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7fb0200b7b00 0x7fb0200cf3e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:29:18.197 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:18.193+0000 7fb025f0f640 1 --2- 172.21.15.37:0/3778608024 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fb020161110 0x7fb0201e8a10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:29:18.197 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:18.193+0000 7fb024f0d640 1 -- 172.21.15.37:0/3778608024 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7fb0200b7b00 msgr2=0x7fb0200cf3e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:29:18.197 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:18.193+0000 7fb024f0d640 1 --2- 172.21.15.37:0/3778608024 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7fb0200b7b00 0x7fb0200cf3e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:29:18.197 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:18.194+0000 7fb024f0d640 1 -- 172.21.15.37:0/3778608024 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fb020161110 msgr2=0x7fb0201e8a10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:29:18.197 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:18.194+0000 7fb024f0d640 1 --2- 172.21.15.37:0/3778608024 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fb020161110 0x7fb0201e8a10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:29:18.197 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:18.194+0000 7fb024f0d640 1 -- 172.21.15.37:0/3778608024 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fb0200ce340 con 0x7fb0200d5430 2024-04-25T10:29:18.198 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:18.194+0000 7fb02570e640 1 --2- 172.21.15.37:0/3778608024 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7fb0200b7b00 0x7fb0200cf3e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T10:29:18.198 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:18.194+0000 7fb024f0d640 1 --2- 172.21.15.37:0/3778608024 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fb0200d5430 0x7fb0201f5c90 secure :-1 s=READY pgs=94 cs=0 l=1 rev1=1 crypto rx=0x7fb01c00c460 tx=0x7fb01c00c920 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:29:18.198 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:18.194+0000 7fb015ffb640 1 -- 172.21.15.37:0/3778608024 <== mon.0 v2:172.21.15.37:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb01c0052d0 con 0x7fb0200d5430 2024-04-25T10:29:18.198 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:18.194+0000 7fb015ffb640 1 -- 172.21.15.37:0/3778608024 <== mon.0 v2:172.21.15.37:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fb01c005490 con 0x7fb0200d5430 2024-04-25T10:29:18.198 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:18.194+0000 7fb027972640 1 -- 172.21.15.37:0/3778608024 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fb0200ce620 con 0x7fb0200d5430 2024-04-25T10:29:18.198 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:18.194+0000 7fb015ffb640 1 -- 172.21.15.37:0/3778608024 <== mon.0 v2:172.21.15.37:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb01c012870 con 0x7fb0200d5430 2024-04-25T10:29:18.198 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:18.194+0000 7fb027972640 1 -- 172.21.15.37:0/3778608024 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fb0200cebe0 con 0x7fb0200d5430 2024-04-25T10:29:18.199 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:18.195+0000 7fb027972640 1 -- 172.21.15.37:0/3778608024 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fb02006a140 con 0x7fb0200d5430 2024-04-25T10:29:18.199 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:18.196+0000 7fb015ffb640 1 -- 172.21.15.37:0/3778608024 <== mon.0 v2:172.21.15.37:3300/0 4 ==== mgrmap(e 5) ==== 96782+0+0 (secure 0 0 0) 0x7fb01c012a10 con 0x7fb0200d5430 2024-04-25T10:29:18.199 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:18.196+0000 7fb015ffb640 1 --2- 172.21.15.37:0/3778608024 >> [v2:172.21.15.203:6832/2418205237,v1:172.21.15.203:6833/2418205237] conn(0x7fb0040744b0 0x7fb0040769c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:29:18.199 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:18.196+0000 7fb015ffb640 1 -- 172.21.15.37:0/3778608024 <== mon.0 v2:172.21.15.37:3300/0 5 ==== osd_map(23..23 src has 1..23) ==== 5709+0+0 (secure 0 0 0) 0x7fb01c094f10 con 0x7fb0200d5430 2024-04-25T10:29:18.199 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:18.197+0000 7fb02570e640 1 --2- 172.21.15.37:0/3778608024 >> [v2:172.21.15.203:6832/2418205237,v1:172.21.15.203:6833/2418205237] conn(0x7fb0040744b0 0x7fb0040769c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:29:18.199 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:18.197+0000 7fb02570e640 1 --2- 172.21.15.37:0/3778608024 >> [v2:172.21.15.203:6832/2418205237,v1:172.21.15.203:6833/2418205237] conn(0x7fb0040744b0 0x7fb0040769c0 secure :-1 s=READY pgs=70 cs=0 l=1 rev1=1 crypto rx=0x7fb010034000 tx=0x7fb01002e000 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:29:18.203 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:18.201+0000 7fb015ffb640 1 -- 172.21.15.37:0/3778608024 <== mon.0 v2:172.21.15.37:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187246 (secure 0 0 0) 0x7fb01c05f600 con 0x7fb0200d5430 2024-04-25T10:29:18.439 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:18.437+0000 7fb027972640 1 -- 172.21.15.37:0/3778608024 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_command({"prefix": "mgr dump", "format": "json"} v 0) -- 0x7fb0200ce0d0 con 0x7fb0200d5430 2024-04-25T10:29:18.442 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:18.440+0000 7fb015ffb640 1 -- 172.21.15.37:0/3778608024 <== mon.0 v2:172.21.15.37:3300/0 7 ==== mon_command_ack([{"prefix": "mgr dump", "format": "json"}]=0 v5) ==== 74+0+186750 (secure 0 0 0) 0x7fb01c05ef00 con 0x7fb0200d5430 2024-04-25T10:29:18.444 INFO:teuthology.orchestra.run.smithi037.stdout: 2024-04-25T10:29:18.446 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:18.443+0000 7fb027972640 1 -- 172.21.15.37:0/3778608024 >> [v2:172.21.15.203:6832/2418205237,v1:172.21.15.203:6833/2418205237] conn(0x7fb0040744b0 msgr2=0x7fb0040769c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:29:18.446 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:18.443+0000 7fb027972640 1 --2- 172.21.15.37:0/3778608024 >> [v2:172.21.15.203:6832/2418205237,v1:172.21.15.203:6833/2418205237] conn(0x7fb0040744b0 0x7fb0040769c0 secure :-1 s=READY pgs=70 cs=0 l=1 rev1=1 crypto rx=0x7fb010034000 tx=0x7fb01002e000 comp rx=0 tx=0).stop 2024-04-25T10:29:18.446 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:18.443+0000 7fb027972640 1 -- 172.21.15.37:0/3778608024 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fb0200d5430 msgr2=0x7fb0201f5c90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:29:18.446 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:18.443+0000 7fb027972640 1 --2- 172.21.15.37:0/3778608024 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fb0200d5430 0x7fb0201f5c90 secure :-1 s=READY pgs=94 cs=0 l=1 rev1=1 crypto rx=0x7fb01c00c460 tx=0x7fb01c00c920 comp rx=0 tx=0).stop 2024-04-25T10:29:18.446 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:18.443+0000 7fb027972640 1 -- 172.21.15.37:0/3778608024 shutdown_connections 2024-04-25T10:29:18.446 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:18.443+0000 7fb027972640 1 --2- 172.21.15.37:0/3778608024 >> [v2:172.21.15.203:6832/2418205237,v1:172.21.15.203:6833/2418205237] conn(0x7fb0040744b0 0x7fb0040769c0 unknown :-1 s=CLOSED pgs=70 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:29:18.446 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:18.443+0000 7fb027972640 1 --2- 172.21.15.37:0/3778608024 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fb020161110 0x7fb0201e8a10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:29:18.447 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:18.443+0000 7fb027972640 1 --2- 172.21.15.37:0/3778608024 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fb0200d5430 0x7fb0201f5c90 unknown :-1 s=CLOSED pgs=94 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:29:18.447 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:18.443+0000 7fb027972640 1 --2- 172.21.15.37:0/3778608024 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7fb0200b7b00 0x7fb0200cf3e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:29:18.447 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:18.443+0000 7fb027972640 1 -- 172.21.15.37:0/3778608024 >> 172.21.15.37:0/3778608024 conn(0x7fb0200d27a0 msgr2=0x7fb02015f610 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:29:18.447 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:18.443+0000 7fb027972640 1 -- 172.21.15.37:0/3778608024 shutdown_connections 2024-04-25T10:29:18.447 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:18.443+0000 7fb027972640 1 -- 172.21.15.37:0/3778608024 wait complete. 2024-04-25T10:29:18.456 INFO:teuthology.orchestra.run.smithi037.stdout:{"epoch":5,"flags":0,"active_gid":4100,"active_name":"x","active_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6832","nonce":2418205237},{"type":"v1","addr":"172.21.15.203:6833","nonce":2418205237}]},"active_addr":"172.21.15.203:6833/2418205237","active_change":"2024-04-25T10:28:48.174228+0000","active_mgr_features":4540701547738038271,"available":true,"standbys":[{"gid":4110,"name":"y","mgr_features":4540701547738038271,"available_modules":[{"name":"alerts","can_run":true,"error_string":"","module_options":{"interval":{"name":"interval","type":"secs","level":"advanced","flags":1,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"How frequently to reexamine health status","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"smtp_destination":{"name":"smtp_destination","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Email address to send alerts to","long_desc":"","tags":[],"see_also":[]},"smtp_from_name":{"name":"smtp_from_name","type":"str","level":"advanced","flags":1,"default_value":"Ceph","min":"","max":"","enum_allowed":[],"desc":"Email From: name","long_desc":"","tags":[],"see_also":[]},"smtp_host":{"name":"smtp_host","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"SMTP server","long_desc":"","tags":[],"see_also":[]},"smtp_password":{"name":"smtp_password","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Password to authenticate with","long_desc":"","tags":[],"see_also":[]},"smtp_port":{"name":"smtp_port","type":"int","level":"advanced","flags":1,"default_value":"465","min":"","max":"","enum_allowed":[],"desc":"SMTP port","long_desc":"","tags":[],"see_also":[]},"smtp_sender":{"name":"smtp_sender","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"SMTP envelope sender","long_desc":"","tags":[],"see_also":[]},"smtp_ssl":{"name":"smtp_ssl","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Use SSL to connect to SMTP server","long_desc":"","tags":[],"see_also":[]},"smtp_user":{"name":"smtp_user","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"User to authenticate as","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"balancer","can_run":true,"error_string":"","module_options":{"active":{"name":"active","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"automatically balance PGs across cluster","long_desc":"","tags":[],"see_also":[]},"begin_time":{"name":"begin_time","type":"str","level":"advanced","flags":1,"default_value":"0000","min":"","max":"","enum_allowed":[],"desc":"beginning time of day to automatically balance","long_desc":"This is a time of day in the format HHMM.","tags":[],"see_also":[]},"begin_weekday":{"name":"begin_weekday","type":"uint","level":"advanced","flags":1,"default_value":"0","min":"0","max":"6","enum_allowed":[],"desc":"Restrict automatic balancing to this day of the week or later","long_desc":"0 = Sunday, 1 = Monday, etc.","tags":[],"see_also":[]},"crush_compat_max_iterations":{"name":"crush_compat_max_iterations","type":"uint","level":"advanced","flags":1,"default_value":"25","min":"1","max":"250","enum_allowed":[],"desc":"maximum number of iterations to attempt optimization","long_desc":"","tags":[],"see_also":[]},"crush_compat_metrics":{"name":"crush_compat_metrics","type":"str","level":"advanced","flags":1,"default_value":"pgs,objects,bytes","min":"","max":"","enum_allowed":[],"desc":"metrics with which to calculate OSD utilization","long_desc":"Value is a list of one or more of \"pgs\", \"objects\", or \"bytes\", and indicates which metrics to use to balance utilization.","tags":[],"see_also":[]},"crush_compat_step":{"name":"crush_compat_step","type":"float","level":"advanced","flags":1,"default_value":"0.5","min":"0.001","max":"0.999","enum_allowed":[],"desc":"aggressiveness of optimization","long_desc":".99 is very aggressive, .01 is less aggressive","tags":[],"see_also":[]},"end_time":{"name":"end_time","type":"str","level":"advanced","flags":1,"default_value":"2359","min":"","max":"","enum_allowed":[],"desc":"ending time of day to automatically balance","long_desc":"This is a time of day in the format HHMM.","tags":[],"see_also":[]},"end_weekday":{"name":"end_weekday","type":"uint","level":"advanced","flags":1,"default_value":"0","min":"0","max":"6","enum_allowed":[],"desc":"Restrict automatic balancing to days of the week earlier than this","long_desc":"0 = Sunday, 1 = Monday, etc.","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"min_score":{"name":"min_score","type":"float","level":"advanced","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"minimum score, below which no optimization is attempted","long_desc":"","tags":[],"see_also":[]},"mode":{"name":"mode","type":"str","level":"advanced","flags":1,"default_value":"upmap","min":"","max":"","enum_allowed":["crush-compat","none","read","upmap","upmap-read"],"desc":"Balancer mode","long_desc":"","tags":[],"see_also":[]},"pool_ids":{"name":"pool_ids","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"pools which the automatic balancing will be limited to","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"secs","level":"advanced","flags":1,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"how frequently to wake up and attempt optimization","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"upmap_max_deviation":{"name":"upmap_max_deviation","type":"int","level":"advanced","flags":1,"default_value":"5","min":"1","max":"","enum_allowed":[],"desc":"deviation below which no optimization is attempted","long_desc":"If the number of PGs are within this count then no optimization is attempted","tags":[],"see_also":[]},"upmap_max_optimizations":{"name":"upmap_max_optimizations","type":"uint","level":"advanced","flags":1,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"maximum upmap optimizations to make per attempt","long_desc":"","tags":[],"see_also":[]}}},{"name":"cephadm","can_run":true,"error_string":"","module_options":{"agent_down_multiplier":{"name":"agent_down_multiplier","type":"float","level":"advanced","flags":0,"default_value":"3.0","min":"","max":"","enum_allowed":[],"desc":"Multiplied by agent refresh rate to calculate how long agent must not report before being marked down","long_desc":"","tags":[],"see_also":[]},"agent_refresh_rate":{"name":"agent_refresh_rate","type":"secs","level":"advanced","flags":0,"default_value":"20","min":"","max":"","enum_allowed":[],"desc":"How often agent on each host will try to gather and send metadata","long_desc":"","tags":[],"see_also":[]},"agent_starting_port":{"name":"agent_starting_port","type":"int","level":"advanced","flags":0,"default_value":"4721","min":"","max":"","enum_allowed":[],"desc":"First port agent will try to bind to (will also try up to next 1000 subsequent ports if blocked)","long_desc":"","tags":[],"see_also":[]},"allow_ptrace":{"name":"allow_ptrace","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"allow SYS_PTRACE capability on ceph containers","long_desc":"The SYS_PTRACE capability is needed to attach to a process with gdb or strace. Enabling this options can allow debugging daemons that encounter problems at runtime.","tags":[],"see_also":[]},"autotune_interval":{"name":"autotune_interval","type":"secs","level":"advanced","flags":0,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"how frequently to autotune daemon memory","long_desc":"","tags":[],"see_also":[]},"autotune_memory_target_ratio":{"name":"autotune_memory_target_ratio","type":"float","level":"advanced","flags":0,"default_value":"0.7","min":"","max":"","enum_allowed":[],"desc":"ratio of total system memory to divide amongst autotuned daemons","long_desc":"","tags":[],"see_also":[]},"cephadm_log_destination":{"name":"cephadm_log_destination","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":["file","file,syslog","syslog"],"desc":"Destination for cephadm command's persistent logging","long_desc":"","tags":[],"see_also":[]},"cgroups_split":{"name":"cgroups_split","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Pass --cgroups=split when cephadm creates containers (currently podman only)","long_desc":"","tags":[],"see_also":[]},"config_checks_enabled":{"name":"config_checks_enabled","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Enable or disable the cephadm configuration analysis","long_desc":"","tags":[],"see_also":[]},"config_dashboard":{"name":"config_dashboard","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"manage configs like API endpoints in Dashboard.","long_desc":"","tags":[],"see_also":[]},"container_image_alertmanager":{"name":"container_image_alertmanager","type":"str","level":"advanced","flags":0,"default_value":"quay.io/prometheus/alertmanager:v0.25.0","min":"","max":"","enum_allowed":[],"desc":"Prometheus container image","long_desc":"","tags":[],"see_also":[]},"container_image_base":{"name":"container_image_base","type":"str","level":"advanced","flags":1,"default_value":"quay.io/ceph/ceph","min":"","max":"","enum_allowed":[],"desc":"Container image name, without the tag","long_desc":"","tags":[],"see_also":[]},"container_image_elasticsearch":{"name":"container_image_elasticsearch","type":"str","level":"advanced","flags":0,"default_value":"quay.io/omrizeneva/elasticsearch:6.8.23","min":"","max":"","enum_allowed":[],"desc":"elasticsearch container image","long_desc":"","tags":[],"see_also":[]},"container_image_grafana":{"name":"container_image_grafana","type":"str","level":"advanced","flags":0,"default_value":"quay.io/ceph/grafana:9.4.12","min":"","max":"","enum_allowed":[],"desc":"Prometheus container image","long_desc":"","tags":[],"see_also":[]},"container_image_haproxy":{"name":"container_image_haproxy","type":"str","level":"advanced","flags":0,"default_value":"quay.io/ceph/haproxy:2.3","min":"","max":"","enum_allowed":[],"desc":"HAproxy container image","long_desc":"","tags":[],"see_also":[]},"container_image_jaeger_agent":{"name":"container_image_jaeger_agent","type":"str","level":"advanced","flags":0,"default_value":"quay.io/jaegertracing/jaeger-agent:1.29","min":"","max":"","enum_allowed":[],"desc":"Jaeger agent container image","long_desc":"","tags":[],"see_also":[]},"container_image_jaeger_collector":{"name":"container_image_jaeger_collector","type":"str","level":"advanced","flags":0,"default_value":"quay.io/jaegertracing/jaeger-collector:1.29","min":"","max":"","enum_allowed":[],"desc":"Jaeger collector container image","long_desc":"","tags":[],"see_also":[]},"container_image_jaeger_query":{"name":"container_image_jaeger_query","type":"str","level":"advanced","flags":0,"default_value":"quay.io/jaegertracing/jaeger-query:1.29","min":"","max":"","enum_allowed":[],"desc":"Jaeger query container image","long_desc":"","tags":[],"see_also":[]},"container_image_keepalived":{"name":"container_image_keepalived","type":"str","level":"advanced","flags":0,"default_value":"quay.io/ceph/keepalived:2.2.4","min":"","max":"","enum_allowed":[],"desc":"Keepalived container image","long_desc":"","tags":[],"see_also":[]},"container_image_loki":{"name":"container_image_loki","type":"str","level":"advanced","flags":0,"default_value":"docker.io/grafana/loki:2.4.0","min":"","max":"","enum_allowed":[],"desc":"Loki container image","long_desc":"","tags":[],"see_also":[]},"container_image_node_exporter":{"name":"container_image_node_exporter","type":"str","level":"advanced","flags":0,"default_value":"quay.io/prometheus/node-exporter:v1.5.0","min":"","max":"","enum_allowed":[],"desc":"Prometheus container image","long_desc":"","tags":[],"see_also":[]},"container_image_nvmeof":{"name":"container_image_nvmeof","type":"str","level":"advanced","flags":0,"default_value":"quay.io/ceph/nvmeof:1.2.1","min":"","max":"","enum_allowed":[],"desc":"Nvme-of container image","long_desc":"","tags":[],"see_also":[]},"container_image_prometheus":{"name":"container_image_prometheus","type":"str","level":"advanced","flags":0,"default_value":"quay.io/prometheus/prometheus:v2.43.0","min":"","max":"","enum_allowed":[],"desc":"Prometheus container image","long_desc":"","tags":[],"see_also":[]},"container_image_promtail":{"name":"container_image_promtail","type":"str","level":"advanced","flags":0,"default_value":"docker.io/grafana/promtail:2.4.0","min":"","max":"","enum_allowed":[],"desc":"Promtail container image","long_desc":"","tags":[],"see_also":[]},"container_image_samba":{"name":"container_image_samba","type":"str","level":"advanced","flags":0,"default_value":"quay.io/samba.org/samba-server:devbuilds-centos-amd64","min":"","max":"","enum_allowed":[],"desc":"Samba/SMB container image","long_desc":"","tags":[],"see_also":[]},"container_image_snmp_gateway":{"name":"container_image_snmp_gateway","type":"str","level":"advanced","flags":0,"default_value":"docker.io/maxwo/snmp-notifier:v1.2.1","min":"","max":"","enum_allowed":[],"desc":"SNMP Gateway container image","long_desc":"","tags":[],"see_also":[]},"container_init":{"name":"container_init","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Run podman/docker with `--init`","long_desc":"","tags":[],"see_also":[]},"daemon_cache_timeout":{"name":"daemon_cache_timeout","type":"secs","level":"advanced","flags":0,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"seconds to cache service (daemon) inventory","long_desc":"","tags":[],"see_also":[]},"default_cephadm_command_timeout":{"name":"default_cephadm_command_timeout","type":"int","level":"advanced","flags":0,"default_value":"900","min":"","max":"","enum_allowed":[],"desc":"Default timeout applied to cephadm commands run directly on the host (in seconds)","long_desc":"","tags":[],"see_also":[]},"default_registry":{"name":"default_registry","type":"str","level":"advanced","flags":0,"default_value":"docker.io","min":"","max":"","enum_allowed":[],"desc":"Search-registry to which we should normalize unqualified image names. This is not the default registry","long_desc":"","tags":[],"see_also":[]},"device_cache_timeout":{"name":"device_cache_timeout","type":"secs","level":"advanced","flags":0,"default_value":"1800","min":"","max":"","enum_allowed":[],"desc":"seconds to cache device inventory","long_desc":"","tags":[],"see_also":[]},"device_enhanced_scan":{"name":"device_enhanced_scan","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Use libstoragemgmt during device scans","long_desc":"","tags":[],"see_also":[]},"facts_cache_timeout":{"name":"facts_cache_timeout","type":"secs","level":"advanced","flags":0,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"seconds to cache host facts data","long_desc":"","tags":[],"see_also":[]},"grafana_dashboards_path":{"name":"grafana_dashboards_path","type":"str","level":"advanced","flags":0,"default_value":"/etc/grafana/dashboards/ceph-dashboard/","min":"","max":"","enum_allowed":[],"desc":"location of dashboards to include in grafana deployments","long_desc":"","tags":[],"see_also":[]},"host_check_interval":{"name":"host_check_interval","type":"secs","level":"advanced","flags":0,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"how frequently to perform a host check","long_desc":"","tags":[],"see_also":[]},"hw_monitoring":{"name":"hw_monitoring","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Deploy hw monitoring daemon on every host.","long_desc":"","tags":[],"see_also":[]},"inventory_list_all":{"name":"inventory_list_all","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Whether ceph-volume inventory should report more devices (mostly mappers (LVs / mpaths), partitions...)","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_refresh_metadata":{"name":"log_refresh_metadata","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Log all refresh metadata. Includes daemon, device, and host info collected regularly. Only has effect if logging at debug level","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"log to the \"cephadm\" cluster log channel\"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"manage_etc_ceph_ceph_conf":{"name":"manage_etc_ceph_ceph_conf","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Manage and own /etc/ceph/ceph.conf on the hosts.","long_desc":"","tags":[],"see_also":[]},"manage_etc_ceph_ceph_conf_hosts":{"name":"manage_etc_ceph_ceph_conf_hosts","type":"str","level":"advanced","flags":0,"default_value":"*","min":"","max":"","enum_allowed":[],"desc":"PlacementSpec describing on which hosts to manage /etc/ceph/ceph.conf","long_desc":"","tags":[],"see_also":[]},"max_count_per_host":{"name":"max_count_per_host","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"max number of daemons per service per host","long_desc":"","tags":[],"see_also":[]},"max_osd_draining_count":{"name":"max_osd_draining_count","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"max number of osds that will be drained simultaneously when osds are removed","long_desc":"","tags":[],"see_also":[]},"migration_current":{"name":"migration_current","type":"int","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"internal - do not modify","long_desc":"","tags":[],"see_also":[]},"mode":{"name":"mode","type":"str","level":"advanced","flags":0,"default_value":"root","min":"","max":"","enum_allowed":["cephadm-package","root"],"desc":"mode for remote execution of cephadm","long_desc":"","tags":[],"see_also":[]},"oob_default_addr":{"name":"oob_default_addr","type":"str","level":"advanced","flags":0,"default_value":"169.254.1.1","min":"","max":"","enum_allowed":[],"desc":"Default address for RedFish API (oob management).","long_desc":"","tags":[],"see_also":[]},"prometheus_alerts_path":{"name":"prometheus_alerts_path","type":"str","level":"advanced","flags":0,"default_value":"/etc/prometheus/ceph/ceph_default_alerts.yml","min":"","max":"","enum_allowed":[],"desc":"location of alerts to include in prometheus deployments","long_desc":"","tags":[],"see_also":[]},"registry_insecure":{"name":"registry_insecure","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Registry is to be considered insecure (no TLS available). Only for development purposes.","long_desc":"","tags":[],"see_also":[]},"registry_password":{"name":"registry_password","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Custom repository password. Only used for logging into a registry.","long_desc":"","tags":[],"see_also":[]},"registry_url":{"name":"registry_url","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Registry url for login purposes. This is not the default registry","long_desc":"","tags":[],"see_also":[]},"registry_username":{"name":"registry_username","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Custom repository username. Only used for logging into a registry.","long_desc":"","tags":[],"see_also":[]},"secure_monitoring_stack":{"name":"secure_monitoring_stack","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Enable TLS security for all the monitoring stack daemons","long_desc":"","tags":[],"see_also":[]},"service_discovery_port":{"name":"service_discovery_port","type":"int","level":"advanced","flags":0,"default_value":"8765","min":"","max":"","enum_allowed":[],"desc":"cephadm service discovery port","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ssh_config_file":{"name":"ssh_config_file","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"customized SSH config file to connect to managed hosts","long_desc":"","tags":[],"see_also":[]},"use_agent":{"name":"use_agent","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Use cephadm agent on each host to gather and send metadata","long_desc":"","tags":[],"see_also":[]},"use_repo_digest":{"name":"use_repo_digest","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Automatically convert image tags to image digest. Make sure all daemons use the same image","long_desc":"","tags":[],"see_also":[]},"warn_on_failed_host_check":{"name":"warn_on_failed_host_check","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"raise a health warning if the host check fails","long_desc":"","tags":[],"see_also":[]},"warn_on_stray_daemons":{"name":"warn_on_stray_daemons","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"raise a health warning if daemons are detected that are not managed by cephadm","long_desc":"","tags":[],"see_also":[]},"warn_on_stray_hosts":{"name":"warn_on_stray_hosts","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"raise a health warning if daemons are detected on a host that is not managed by cephadm","long_desc":"","tags":[],"see_also":[]}}},{"name":"crash","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"retain_interval":{"name":"retain_interval","type":"secs","level":"advanced","flags":1,"default_value":"31536000","min":"","max":"","enum_allowed":[],"desc":"how long to retain crashes before pruning them","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"warn_recent_interval":{"name":"warn_recent_interval","type":"secs","level":"advanced","flags":1,"default_value":"1209600","min":"","max":"","enum_allowed":[],"desc":"time interval in which to warn about recent crashes","long_desc":"","tags":[],"see_also":[]}}},{"name":"dashboard","can_run":true,"error_string":"","module_options":{"ACCOUNT_LOCKOUT_ATTEMPTS":{"name":"ACCOUNT_LOCKOUT_ATTEMPTS","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ALERTMANAGER_API_HOST":{"name":"ALERTMANAGER_API_HOST","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ALERTMANAGER_API_SSL_VERIFY":{"name":"ALERTMANAGER_API_SSL_VERIFY","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"AUDIT_API_ENABLED":{"name":"AUDIT_API_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"AUDIT_API_LOG_PAYLOAD":{"name":"AUDIT_API_LOG_PAYLOAD","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ENABLE_BROWSABLE_API":{"name":"ENABLE_BROWSABLE_API","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_CEPHFS":{"name":"FEATURE_TOGGLE_CEPHFS","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_DASHBOARD":{"name":"FEATURE_TOGGLE_DASHBOARD","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_ISCSI":{"name":"FEATURE_TOGGLE_ISCSI","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_MIRRORING":{"name":"FEATURE_TOGGLE_MIRRORING","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_NFS":{"name":"FEATURE_TOGGLE_NFS","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_RBD":{"name":"FEATURE_TOGGLE_RBD","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_RGW":{"name":"FEATURE_TOGGLE_RGW","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GANESHA_CLUSTERS_RADOS_POOL_NAMESPACE":{"name":"GANESHA_CLUSTERS_RADOS_POOL_NAMESPACE","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_PASSWORD":{"name":"GRAFANA_API_PASSWORD","type":"str","level":"advanced","flags":0,"default_value":"admin","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_SSL_VERIFY":{"name":"GRAFANA_API_SSL_VERIFY","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_URL":{"name":"GRAFANA_API_URL","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_USERNAME":{"name":"GRAFANA_API_USERNAME","type":"str","level":"advanced","flags":0,"default_value":"admin","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_FRONTEND_API_URL":{"name":"GRAFANA_FRONTEND_API_URL","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_UPDATE_DASHBOARDS":{"name":"GRAFANA_UPDATE_DASHBOARDS","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ISCSI_API_SSL_VERIFICATION":{"name":"ISCSI_API_SSL_VERIFICATION","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ISSUE_TRACKER_API_KEY":{"name":"ISSUE_TRACKER_API_KEY","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"MANAGED_BY_CLUSTERS":{"name":"MANAGED_BY_CLUSTERS","type":"str","level":"advanced","flags":0,"default_value":"[]","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"MULTICLUSTER_CONFIG":{"name":"MULTICLUSTER_CONFIG","type":"str","level":"advanced","flags":0,"default_value":"{}","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PROMETHEUS_API_HOST":{"name":"PROMETHEUS_API_HOST","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PROMETHEUS_API_SSL_VERIFY":{"name":"PROMETHEUS_API_SSL_VERIFY","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_COMPLEXITY_ENABLED":{"name":"PWD_POLICY_CHECK_COMPLEXITY_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_EXCLUSION_LIST_ENABLED":{"name":"PWD_POLICY_CHECK_EXCLUSION_LIST_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_LENGTH_ENABLED":{"name":"PWD_POLICY_CHECK_LENGTH_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_OLDPWD_ENABLED":{"name":"PWD_POLICY_CHECK_OLDPWD_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_REPETITIVE_CHARS_ENABLED":{"name":"PWD_POLICY_CHECK_REPETITIVE_CHARS_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_SEQUENTIAL_CHARS_ENABLED":{"name":"PWD_POLICY_CHECK_SEQUENTIAL_CHARS_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_USERNAME_ENABLED":{"name":"PWD_POLICY_CHECK_USERNAME_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_ENABLED":{"name":"PWD_POLICY_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_EXCLUSION_LIST":{"name":"PWD_POLICY_EXCLUSION_LIST","type":"str","level":"advanced","flags":0,"default_value":"osd,host,dashboard,pool,block,nfs,ceph,monitors,gateway,logs,crush,maps","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_MIN_COMPLEXITY":{"name":"PWD_POLICY_MIN_COMPLEXITY","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_MIN_LENGTH":{"name":"PWD_POLICY_MIN_LENGTH","type":"int","level":"advanced","flags":0,"default_value":"8","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"REST_REQUESTS_TIMEOUT":{"name":"REST_REQUESTS_TIMEOUT","type":"int","level":"advanced","flags":0,"default_value":"45","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_ACCESS_KEY":{"name":"RGW_API_ACCESS_KEY","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_ADMIN_RESOURCE":{"name":"RGW_API_ADMIN_RESOURCE","type":"str","level":"advanced","flags":0,"default_value":"admin","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_SECRET_KEY":{"name":"RGW_API_SECRET_KEY","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_SSL_VERIFY":{"name":"RGW_API_SSL_VERIFY","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"UNSAFE_TLS_v1_2":{"name":"UNSAFE_TLS_v1_2","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"USER_PWD_EXPIRATION_SPAN":{"name":"USER_PWD_EXPIRATION_SPAN","type":"int","level":"advanced","flags":0,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"USER_PWD_EXPIRATION_WARNING_1":{"name":"USER_PWD_EXPIRATION_WARNING_1","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"USER_PWD_EXPIRATION_WARNING_2":{"name":"USER_PWD_EXPIRATION_WARNING_2","type":"int","level":"advanced","flags":0,"default_value":"5","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"cross_origin_url":{"name":"cross_origin_url","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"crt_file":{"name":"crt_file","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"debug":{"name":"debug","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Enable/disable debug options","long_desc":"","tags":[],"see_also":[]},"jwt_token_ttl":{"name":"jwt_token_ttl","type":"int","level":"advanced","flags":0,"default_value":"28800","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"key_file":{"name":"key_file","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"motd":{"name":"motd","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"The message of the day","long_desc":"","tags":[],"see_also":[]},"redirect_resolve_ip_addr":{"name":"redirect_resolve_ip_addr","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_addr":{"name":"server_addr","type":"str","level":"advanced","flags":0,"default_value":"::","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_port":{"name":"server_port","type":"int","level":"advanced","flags":0,"default_value":"8080","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ssl":{"name":"ssl","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ssl_server_port":{"name":"ssl_server_port","type":"int","level":"advanced","flags":0,"default_value":"8443","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"standby_behaviour":{"name":"standby_behaviour","type":"str","level":"advanced","flags":0,"default_value":"redirect","min":"","max":"","enum_allowed":["error","redirect"],"desc":"","long_desc":"","tags":[],"see_also":[]},"standby_error_status_code":{"name":"standby_error_status_code","type":"int","level":"advanced","flags":0,"default_value":"500","min":"400","max":"599","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"url_prefix":{"name":"url_prefix","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"devicehealth","can_run":true,"error_string":"","module_options":{"enable_monitoring":{"name":"enable_monitoring","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"monitor device health metrics","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"mark_out_threshold":{"name":"mark_out_threshold","type":"secs","level":"advanced","flags":1,"default_value":"2419200","min":"","max":"","enum_allowed":[],"desc":"automatically mark OSD if it may fail before this long","long_desc":"","tags":[],"see_also":[]},"pool_name":{"name":"pool_name","type":"str","level":"advanced","flags":1,"default_value":"device_health_metrics","min":"","max":"","enum_allowed":[],"desc":"name of pool in which to store device health metrics","long_desc":"","tags":[],"see_also":[]},"retention_period":{"name":"retention_period","type":"secs","level":"advanced","flags":1,"default_value":"15552000","min":"","max":"","enum_allowed":[],"desc":"how long to retain device health metrics","long_desc":"","tags":[],"see_also":[]},"scrape_frequency":{"name":"scrape_frequency","type":"secs","level":"advanced","flags":1,"default_value":"86400","min":"","max":"","enum_allowed":[],"desc":"how frequently to scrape device health metrics","long_desc":"","tags":[],"see_also":[]},"self_heal":{"name":"self_heal","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"preemptively heal cluster around devices that may fail","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"secs","level":"advanced","flags":1,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"how frequently to wake up and check device health","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"warn_threshold":{"name":"warn_threshold","type":"secs","level":"advanced","flags":1,"default_value":"7257600","min":"","max":"","enum_allowed":[],"desc":"raise health warning if OSD may fail before this long","long_desc":"","tags":[],"see_also":[]}}},{"name":"diskprediction_local","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"predict_interval":{"name":"predict_interval","type":"str","level":"advanced","flags":0,"default_value":"86400","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"predictor_model":{"name":"predictor_model","type":"str","level":"advanced","flags":0,"default_value":"prophetstor","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"str","level":"advanced","flags":0,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"influx","can_run":false,"error_string":"influxdb python module not found","module_options":{"batch_size":{"name":"batch_size","type":"int","level":"advanced","flags":0,"default_value":"5000","min":"","max":"","enum_allowed":[],"desc":"How big batches of data points should be when sending to InfluxDB.","long_desc":"","tags":[],"see_also":[]},"database":{"name":"database","type":"str","level":"advanced","flags":0,"default_value":"ceph","min":"","max":"","enum_allowed":[],"desc":"InfluxDB database name. You will need to create this database and grant write privileges to the configured username or the username must have admin privileges to create it.","long_desc":"","tags":[],"see_also":[]},"hostname":{"name":"hostname","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"InfluxDB server hostname","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"secs","level":"advanced","flags":0,"default_value":"30","min":"5","max":"","enum_allowed":[],"desc":"Time between reports to InfluxDB. Default 30 seconds.","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"password":{"name":"password","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"password of InfluxDB server user","long_desc":"","tags":[],"see_also":[]},"port":{"name":"port","type":"int","level":"advanced","flags":0,"default_value":"8086","min":"","max":"","enum_allowed":[],"desc":"InfluxDB server port","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ssl":{"name":"ssl","type":"str","level":"advanced","flags":0,"default_value":"false","min":"","max":"","enum_allowed":[],"desc":"Use https connection for InfluxDB server. Use \"true\" or \"false\".","long_desc":"","tags":[],"see_also":[]},"threads":{"name":"threads","type":"int","level":"advanced","flags":0,"default_value":"5","min":"1","max":"32","enum_allowed":[],"desc":"How many worker threads should be spawned for sending data to InfluxDB.","long_desc":"","tags":[],"see_also":[]},"username":{"name":"username","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"username of InfluxDB server user","long_desc":"","tags":[],"see_also":[]},"verify_ssl":{"name":"verify_ssl","type":"str","level":"advanced","flags":0,"default_value":"true","min":"","max":"","enum_allowed":[],"desc":"Verify https cert for InfluxDB server. Use \"true\" or \"false\".","long_desc":"","tags":[],"see_also":[]}}},{"name":"insights","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"iostat","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"localpool","can_run":true,"error_string":"","module_options":{"failure_domain":{"name":"failure_domain","type":"str","level":"advanced","flags":1,"default_value":"host","min":"","max":"","enum_allowed":[],"desc":"failure domain for any created local pool","long_desc":"what failure domain we should separate data replicas across.","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"min_size":{"name":"min_size","type":"int","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"default min_size for any created local pool","long_desc":"value to set min_size to (unchanged from Ceph's default if this option is not set)","tags":[],"see_also":[]},"num_rep":{"name":"num_rep","type":"int","level":"advanced","flags":1,"default_value":"3","min":"","max":"","enum_allowed":[],"desc":"default replica count for any created local pool","long_desc":"","tags":[],"see_also":[]},"pg_num":{"name":"pg_num","type":"int","level":"advanced","flags":1,"default_value":"128","min":"","max":"","enum_allowed":[],"desc":"default pg_num for any created local pool","long_desc":"","tags":[],"see_also":[]},"prefix":{"name":"prefix","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"name prefix for any created local pool","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"subtree":{"name":"subtree","type":"str","level":"advanced","flags":1,"default_value":"rack","min":"","max":"","enum_allowed":[],"desc":"CRUSH level for which to create a local pool","long_desc":"which CRUSH subtree type the module should create a pool for.","tags":[],"see_also":[]}}},{"name":"mds_autoscaler","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"mirroring","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"nfs","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"orchestrator","can_run":true,"error_string":"","module_options":{"fail_fs":{"name":"fail_fs","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Fail filesystem for rapid multi-rank mds upgrade","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"orchestrator":{"name":"orchestrator","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["cephadm","rook","test_orchestrator"],"desc":"Orchestrator backend","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"osd_perf_query","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"osd_support","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"pg_autoscaler","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"secs","level":"advanced","flags":0,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"threshold":{"name":"threshold","type":"float","level":"advanced","flags":0,"default_value":"3.0","min":"1.0","max":"","enum_allowed":[],"desc":"scaling threshold","long_desc":"The factor by which the `NEW PG_NUM` must vary from the current`PG_NUM` before being accepted. Cannot be less than 1.0","tags":[],"see_also":[]}}},{"name":"progress","can_run":true,"error_string":"","module_options":{"allow_pg_recovery_event":{"name":"allow_pg_recovery_event","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"allow the module to show pg recovery progress","long_desc":"","tags":[],"see_also":[]},"enabled":{"name":"enabled","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"max_completed_events":{"name":"max_completed_events","type":"int","level":"advanced","flags":1,"default_value":"50","min":"","max":"","enum_allowed":[],"desc":"number of past completed events to remember","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"secs","level":"advanced","flags":1,"default_value":"5","min":"","max":"","enum_allowed":[],"desc":"how long the module is going to sleep","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"prometheus","can_run":true,"error_string":"","module_options":{"cache":{"name":"cache","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"exclude_perf_counters":{"name":"exclude_perf_counters","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Do not include perf-counters in the metrics output","long_desc":"Gathering perf-counters from a single Prometheus exporter can degrade ceph-mgr performance, especially in large clusters. Instead, Ceph-exporter daemons are now used by default for perf-counter gathering. This should only be disabled when no ceph-exporters are deployed.","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rbd_stats_pools":{"name":"rbd_stats_pools","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rbd_stats_pools_refresh_interval":{"name":"rbd_stats_pools_refresh_interval","type":"int","level":"advanced","flags":0,"default_value":"300","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"scrape_interval":{"name":"scrape_interval","type":"float","level":"advanced","flags":0,"default_value":"15.0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_addr":{"name":"server_addr","type":"str","level":"advanced","flags":0,"default_value":"::","min":"","max":"","enum_allowed":[],"desc":"the IPv4 or IPv6 address on which the module listens for HTTP requests","long_desc":"","tags":[],"see_also":[]},"server_port":{"name":"server_port","type":"int","level":"advanced","flags":1,"default_value":"9283","min":"","max":"","enum_allowed":[],"desc":"the port on which the module listens for HTTP requests","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"stale_cache_strategy":{"name":"stale_cache_strategy","type":"str","level":"advanced","flags":0,"default_value":"log","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"standby_behaviour":{"name":"standby_behaviour","type":"str","level":"advanced","flags":1,"default_value":"default","min":"","max":"","enum_allowed":["default","error"],"desc":"","long_desc":"","tags":[],"see_also":[]},"standby_error_status_code":{"name":"standby_error_status_code","type":"int","level":"advanced","flags":1,"default_value":"500","min":"400","max":"599","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"rbd_support","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"max_concurrent_snap_create":{"name":"max_concurrent_snap_create","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"mirror_snapshot_schedule":{"name":"mirror_snapshot_schedule","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"trash_purge_schedule":{"name":"trash_purge_schedule","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"restful","can_run":true,"error_string":"","module_options":{"enable_auth":{"name":"enable_auth","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"key_file":{"name":"key_file","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_addr":{"name":"server_addr","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_port":{"name":"server_port","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"rgw","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"rook","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"storage_class":{"name":"storage_class","type":"str","level":"advanced","flags":0,"default_value":"local","min":"","max":"","enum_allowed":[],"desc":"storage class name for LSO-discovered PVs","long_desc":"","tags":[],"see_also":[]}}},{"name":"selftest","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"roption1":{"name":"roption1","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"roption2":{"name":"roption2","type":"str","level":"advanced","flags":0,"default_value":"xyz","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption1":{"name":"rwoption1","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption2":{"name":"rwoption2","type":"int","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption3":{"name":"rwoption3","type":"float","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption4":{"name":"rwoption4","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption5":{"name":"rwoption5","type":"bool","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption6":{"name":"rwoption6","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption7":{"name":"rwoption7","type":"int","level":"advanced","flags":0,"default_value":"","min":"1","max":"42","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"testkey":{"name":"testkey","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"testlkey":{"name":"testlkey","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"testnewline":{"name":"testnewline","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"snap_schedule","can_run":true,"error_string":"","module_options":{"allow_m_granularity":{"name":"allow_m_granularity","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"allow minute scheduled snapshots","long_desc":"","tags":[],"see_also":[]},"dump_on_update":{"name":"dump_on_update","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"dump database to debug log on update","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"stats","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"status","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"telegraf","can_run":true,"error_string":"","module_options":{"address":{"name":"address","type":"str","level":"advanced","flags":0,"default_value":"unixgram:///tmp/telegraf.sock","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"secs","level":"advanced","flags":0,"default_value":"15","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"telemetry","can_run":true,"error_string":"","module_options":{"channel_basic":{"name":"channel_basic","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Share basic cluster information (size, version)","long_desc":"","tags":[],"see_also":[]},"channel_crash":{"name":"channel_crash","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Share metadata about Ceph daemon crashes (version, stack straces, etc)","long_desc":"","tags":[],"see_also":[]},"channel_device":{"name":"channel_device","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Share device health metrics (e.g., SMART data, minus potentially identifying info like serial numbers)","long_desc":"","tags":[],"see_also":[]},"channel_ident":{"name":"channel_ident","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Share a user-provided description and/or contact email for the cluster","long_desc":"","tags":[],"see_also":[]},"channel_perf":{"name":"channel_perf","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Share various performance metrics of a cluster","long_desc":"","tags":[],"see_also":[]},"contact":{"name":"contact","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"description":{"name":"description","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"device_url":{"name":"device_url","type":"str","level":"advanced","flags":0,"default_value":"https://telemetry.ceph.com/device","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"enabled":{"name":"enabled","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"int","level":"advanced","flags":0,"default_value":"24","min":"8","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"last_opt_revision":{"name":"last_opt_revision","type":"int","level":"advanced","flags":0,"default_value":"1","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"leaderboard":{"name":"leaderboard","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"leaderboard_description":{"name":"leaderboard_description","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"organization":{"name":"organization","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"proxy":{"name":"proxy","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"url":{"name":"url","type":"str","level":"advanced","flags":0,"default_value":"https://telemetry.ceph.com/report","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"test_orchestrator","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"volumes","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"max_concurrent_clones":{"name":"max_concurrent_clones","type":"int","level":"advanced","flags":0,"default_value":"4","min":"","max":"","enum_allowed":[],"desc":"Number of asynchronous cloner threads","long_desc":"","tags":[],"see_also":[]},"periodic_async_work":{"name":"periodic_async_work","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Periodically check for async work","long_desc":"","tags":[],"see_also":[]},"snapshot_clone_delay":{"name":"snapshot_clone_delay","type":"int","level":"advanced","flags":0,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"Delay clone begin operation by snapshot_clone_delay seconds","long_desc":"","tags":[],"see_also":[]},"snapshot_clone_no_wait":{"name":"snapshot_clone_no_wait","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Reject subvolume clone request when cloner threads are busy","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"zabbix","can_run":true,"error_string":"","module_options":{"discovery_interval":{"name":"discovery_interval","type":"uint","level":"advanced","flags":0,"default_value":"100","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"identifier":{"name":"identifier","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"secs","level":"advanced","flags":0,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"zabbix_host":{"name":"zabbix_host","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"zabbix_port":{"name":"zabbix_port","type":"int","level":"advanced","flags":0,"default_value":"10051","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"zabbix_sender":{"name":"zabbix_sender","type":"str","level":"advanced","flags":0,"default_value":"/usr/bin/zabbix_sender","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}}]}],"modules":["iostat","nfs","restful"],"available_modules":[{"name":"alerts","can_run":true,"error_string":"","module_options":{"interval":{"name":"interval","type":"secs","level":"advanced","flags":1,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"How frequently to reexamine health status","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"smtp_destination":{"name":"smtp_destination","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Email address to send alerts to","long_desc":"","tags":[],"see_also":[]},"smtp_from_name":{"name":"smtp_from_name","type":"str","level":"advanced","flags":1,"default_value":"Ceph","min":"","max":"","enum_allowed":[],"desc":"Email From: name","long_desc":"","tags":[],"see_also":[]},"smtp_host":{"name":"smtp_host","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"SMTP server","long_desc":"","tags":[],"see_also":[]},"smtp_password":{"name":"smtp_password","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Password to authenticate with","long_desc":"","tags":[],"see_also":[]},"smtp_port":{"name":"smtp_port","type":"int","level":"advanced","flags":1,"default_value":"465","min":"","max":"","enum_allowed":[],"desc":"SMTP port","long_desc":"","tags":[],"see_also":[]},"smtp_sender":{"name":"smtp_sender","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"SMTP envelope sender","long_desc":"","tags":[],"see_also":[]},"smtp_ssl":{"name":"smtp_ssl","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Use SSL to connect to SMTP server","long_desc":"","tags":[],"see_also":[]},"smtp_user":{"name":"smtp_user","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"User to authenticate as","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"balancer","can_run":true,"error_string":"","module_options":{"active":{"name":"active","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"automatically balance PGs across cluster","long_desc":"","tags":[],"see_also":[]},"begin_time":{"name":"begin_time","type":"str","level":"advanced","flags":1,"default_value":"0000","min":"","max":"","enum_allowed":[],"desc":"beginning time of day to automatically balance","long_desc":"This is a time of day in the format HHMM.","tags":[],"see_also":[]},"begin_weekday":{"name":"begin_weekday","type":"uint","level":"advanced","flags":1,"default_value":"0","min":"0","max":"6","enum_allowed":[],"desc":"Restrict automatic balancing to this day of the week or later","long_desc":"0 = Sunday, 1 = Monday, etc.","tags":[],"see_also":[]},"crush_compat_max_iterations":{"name":"crush_compat_max_iterations","type":"uint","level":"advanced","flags":1,"default_value":"25","min":"1","max":"250","enum_allowed":[],"desc":"maximum number of iterations to attempt optimization","long_desc":"","tags":[],"see_also":[]},"crush_compat_metrics":{"name":"crush_compat_metrics","type":"str","level":"advanced","flags":1,"default_value":"pgs,objects,bytes","min":"","max":"","enum_allowed":[],"desc":"metrics with which to calculate OSD utilization","long_desc":"Value is a list of one or more of \"pgs\", \"objects\", or \"bytes\", and indicates which metrics to use to balance utilization.","tags":[],"see_also":[]},"crush_compat_step":{"name":"crush_compat_step","type":"float","level":"advanced","flags":1,"default_value":"0.5","min":"0.001","max":"0.999","enum_allowed":[],"desc":"aggressiveness of optimization","long_desc":".99 is very aggressive, .01 is less aggressive","tags":[],"see_also":[]},"end_time":{"name":"end_time","type":"str","level":"advanced","flags":1,"default_value":"2359","min":"","max":"","enum_allowed":[],"desc":"ending time of day to automatically balance","long_desc":"This is a time of day in the format HHMM.","tags":[],"see_also":[]},"end_weekday":{"name":"end_weekday","type":"uint","level":"advanced","flags":1,"default_value":"0","min":"0","max":"6","enum_allowed":[],"desc":"Restrict automatic balancing to days of the week earlier than this","long_desc":"0 = Sunday, 1 = Monday, etc.","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"min_score":{"name":"min_score","type":"float","level":"advanced","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"minimum score, below which no optimization is attempted","long_desc":"","tags":[],"see_also":[]},"mode":{"name":"mode","type":"str","level":"advanced","flags":1,"default_value":"upmap","min":"","max":"","enum_allowed":["crush-compat","none","read","upmap","upmap-read"],"desc":"Balancer mode","long_desc":"","tags":[],"see_also":[]},"pool_ids":{"name":"pool_ids","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"pools which the automatic balancing will be limited to","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"secs","level":"advanced","flags":1,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"how frequently to wake up and attempt optimization","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"upmap_max_deviation":{"name":"upmap_max_deviation","type":"int","level":"advanced","flags":1,"default_value":"5","min":"1","max":"","enum_allowed":[],"desc":"deviation below which no optimization is attempted","long_desc":"If the number of PGs are within this count then no optimization is attempted","tags":[],"see_also":[]},"upmap_max_optimizations":{"name":"upmap_max_optimizations","type":"uint","level":"advanced","flags":1,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"maximum upmap optimizations to make per attempt","long_desc":"","tags":[],"see_also":[]}}},{"name":"cephadm","can_run":true,"error_string":"","module_options":{"agent_down_multiplier":{"name":"agent_down_multiplier","type":"float","level":"advanced","flags":0,"default_value":"3.0","min":"","max":"","enum_allowed":[],"desc":"Multiplied by agent refresh rate to calculate how long agent must not report before being marked down","long_desc":"","tags":[],"see_also":[]},"agent_refresh_rate":{"name":"agent_refresh_rate","type":"secs","level":"advanced","flags":0,"default_value":"20","min":"","max":"","enum_allowed":[],"desc":"How often agent on each host will try to gather and send metadata","long_desc":"","tags":[],"see_also":[]},"agent_starting_port":{"name":"agent_starting_port","type":"int","level":"advanced","flags":0,"default_value":"4721","min":"","max":"","enum_allowed":[],"desc":"First port agent will try to bind to (will also try up to next 1000 subsequent ports if blocked)","long_desc":"","tags":[],"see_also":[]},"allow_ptrace":{"name":"allow_ptrace","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"allow SYS_PTRACE capability on ceph containers","long_desc":"The SYS_PTRACE capability is needed to attach to a process with gdb or strace. Enabling this options can allow debugging daemons that encounter problems at runtime.","tags":[],"see_also":[]},"autotune_interval":{"name":"autotune_interval","type":"secs","level":"advanced","flags":0,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"how frequently to autotune daemon memory","long_desc":"","tags":[],"see_also":[]},"autotune_memory_target_ratio":{"name":"autotune_memory_target_ratio","type":"float","level":"advanced","flags":0,"default_value":"0.7","min":"","max":"","enum_allowed":[],"desc":"ratio of total system memory to divide amongst autotuned daemons","long_desc":"","tags":[],"see_also":[]},"cephadm_log_destination":{"name":"cephadm_log_destination","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":["file","file,syslog","syslog"],"desc":"Destination for cephadm command's persistent logging","long_desc":"","tags":[],"see_also":[]},"cgroups_split":{"name":"cgroups_split","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Pass --cgroups=split when cephadm creates containers (currently podman only)","long_desc":"","tags":[],"see_also":[]},"config_checks_enabled":{"name":"config_checks_enabled","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Enable or disable the cephadm configuration analysis","long_desc":"","tags":[],"see_also":[]},"config_dashboard":{"name":"config_dashboard","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"manage configs like API endpoints in Dashboard.","long_desc":"","tags":[],"see_also":[]},"container_image_alertmanager":{"name":"container_image_alertmanager","type":"str","level":"advanced","flags":0,"default_value":"quay.io/prometheus/alertmanager:v0.25.0","min":"","max":"","enum_allowed":[],"desc":"Prometheus container image","long_desc":"","tags":[],"see_also":[]},"container_image_base":{"name":"container_image_base","type":"str","level":"advanced","flags":1,"default_value":"quay.io/ceph/ceph","min":"","max":"","enum_allowed":[],"desc":"Container image name, without the tag","long_desc":"","tags":[],"see_also":[]},"container_image_elasticsearch":{"name":"container_image_elasticsearch","type":"str","level":"advanced","flags":0,"default_value":"quay.io/omrizeneva/elasticsearch:6.8.23","min":"","max":"","enum_allowed":[],"desc":"elasticsearch container image","long_desc":"","tags":[],"see_also":[]},"container_image_grafana":{"name":"container_image_grafana","type":"str","level":"advanced","flags":0,"default_value":"quay.io/ceph/grafana:9.4.12","min":"","max":"","enum_allowed":[],"desc":"Prometheus container image","long_desc":"","tags":[],"see_also":[]},"container_image_haproxy":{"name":"container_image_haproxy","type":"str","level":"advanced","flags":0,"default_value":"quay.io/ceph/haproxy:2.3","min":"","max":"","enum_allowed":[],"desc":"HAproxy container image","long_desc":"","tags":[],"see_also":[]},"container_image_jaeger_agent":{"name":"container_image_jaeger_agent","type":"str","level":"advanced","flags":0,"default_value":"quay.io/jaegertracing/jaeger-agent:1.29","min":"","max":"","enum_allowed":[],"desc":"Jaeger agent container image","long_desc":"","tags":[],"see_also":[]},"container_image_jaeger_collector":{"name":"container_image_jaeger_collector","type":"str","level":"advanced","flags":0,"default_value":"quay.io/jaegertracing/jaeger-collector:1.29","min":"","max":"","enum_allowed":[],"desc":"Jaeger collector container image","long_desc":"","tags":[],"see_also":[]},"container_image_jaeger_query":{"name":"container_image_jaeger_query","type":"str","level":"advanced","flags":0,"default_value":"quay.io/jaegertracing/jaeger-query:1.29","min":"","max":"","enum_allowed":[],"desc":"Jaeger query container image","long_desc":"","tags":[],"see_also":[]},"container_image_keepalived":{"name":"container_image_keepalived","type":"str","level":"advanced","flags":0,"default_value":"quay.io/ceph/keepalived:2.2.4","min":"","max":"","enum_allowed":[],"desc":"Keepalived container image","long_desc":"","tags":[],"see_also":[]},"container_image_loki":{"name":"container_image_loki","type":"str","level":"advanced","flags":0,"default_value":"docker.io/grafana/loki:2.4.0","min":"","max":"","enum_allowed":[],"desc":"Loki container image","long_desc":"","tags":[],"see_also":[]},"container_image_node_exporter":{"name":"container_image_node_exporter","type":"str","level":"advanced","flags":0,"default_value":"quay.io/prometheus/node-exporter:v1.5.0","min":"","max":"","enum_allowed":[],"desc":"Prometheus container image","long_desc":"","tags":[],"see_also":[]},"container_image_nvmeof":{"name":"container_image_nvmeof","type":"str","level":"advanced","flags":0,"default_value":"quay.io/ceph/nvmeof:1.2.1","min":"","max":"","enum_allowed":[],"desc":"Nvme-of container image","long_desc":"","tags":[],"see_also":[]},"container_image_prometheus":{"name":"container_image_prometheus","type":"str","level":"advanced","flags":0,"default_value":"quay.io/prometheus/prometheus:v2.43.0","min":"","max":"","enum_allowed":[],"desc":"Prometheus container image","long_desc":"","tags":[],"see_also":[]},"container_image_promtail":{"name":"container_image_promtail","type":"str","level":"advanced","flags":0,"default_value":"docker.io/grafana/promtail:2.4.0","min":"","max":"","enum_allowed":[],"desc":"Promtail container image","long_desc":"","tags":[],"see_also":[]},"container_image_samba":{"name":"container_image_samba","type":"str","level":"advanced","flags":0,"default_value":"quay.io/samba.org/samba-server:devbuilds-centos-amd64","min":"","max":"","enum_allowed":[],"desc":"Samba/SMB container image","long_desc":"","tags":[],"see_also":[]},"container_image_snmp_gateway":{"name":"container_image_snmp_gateway","type":"str","level":"advanced","flags":0,"default_value":"docker.io/maxwo/snmp-notifier:v1.2.1","min":"","max":"","enum_allowed":[],"desc":"SNMP Gateway container image","long_desc":"","tags":[],"see_also":[]},"container_init":{"name":"container_init","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Run podman/docker with `--init`","long_desc":"","tags":[],"see_also":[]},"daemon_cache_timeout":{"name":"daemon_cache_timeout","type":"secs","level":"advanced","flags":0,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"seconds to cache service (daemon) inventory","long_desc":"","tags":[],"see_also":[]},"default_cephadm_command_timeout":{"name":"default_cephadm_command_timeout","type":"int","level":"advanced","flags":0,"default_value":"900","min":"","max":"","enum_allowed":[],"desc":"Default timeout applied to cephadm commands run directly on the host (in seconds)","long_desc":"","tags":[],"see_also":[]},"default_registry":{"name":"default_registry","type":"str","level":"advanced","flags":0,"default_value":"docker.io","min":"","max":"","enum_allowed":[],"desc":"Search-registry to which we should normalize unqualified image names. This is not the default registry","long_desc":"","tags":[],"see_also":[]},"device_cache_timeout":{"name":"device_cache_timeout","type":"secs","level":"advanced","flags":0,"default_value":"1800","min":"","max":"","enum_allowed":[],"desc":"seconds to cache device inventory","long_desc":"","tags":[],"see_also":[]},"device_enhanced_scan":{"name":"device_enhanced_scan","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Use libstoragemgmt during device scans","long_desc":"","tags":[],"see_also":[]},"facts_cache_timeout":{"name":"facts_cache_timeout","type":"secs","level":"advanced","flags":0,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"seconds to cache host facts data","long_desc":"","tags":[],"see_also":[]},"grafana_dashboards_path":{"name":"grafana_dashboards_path","type":"str","level":"advanced","flags":0,"default_value":"/etc/grafana/dashboards/ceph-dashboard/","min":"","max":"","enum_allowed":[],"desc":"location of dashboards to include in grafana deployments","long_desc":"","tags":[],"see_also":[]},"host_check_interval":{"name":"host_check_interval","type":"secs","level":"advanced","flags":0,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"how frequently to perform a host check","long_desc":"","tags":[],"see_also":[]},"hw_monitoring":{"name":"hw_monitoring","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Deploy hw monitoring daemon on every host.","long_desc":"","tags":[],"see_also":[]},"inventory_list_all":{"name":"inventory_list_all","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Whether ceph-volume inventory should report more devices (mostly mappers (LVs / mpaths), partitions...)","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_refresh_metadata":{"name":"log_refresh_metadata","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Log all refresh metadata. Includes daemon, device, and host info collected regularly. Only has effect if logging at debug level","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"log to the \"cephadm\" cluster log channel\"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"manage_etc_ceph_ceph_conf":{"name":"manage_etc_ceph_ceph_conf","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Manage and own /etc/ceph/ceph.conf on the hosts.","long_desc":"","tags":[],"see_also":[]},"manage_etc_ceph_ceph_conf_hosts":{"name":"manage_etc_ceph_ceph_conf_hosts","type":"str","level":"advanced","flags":0,"default_value":"*","min":"","max":"","enum_allowed":[],"desc":"PlacementSpec describing on which hosts to manage /etc/ceph/ceph.conf","long_desc":"","tags":[],"see_also":[]},"max_count_per_host":{"name":"max_count_per_host","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"max number of daemons per service per host","long_desc":"","tags":[],"see_also":[]},"max_osd_draining_count":{"name":"max_osd_draining_count","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"max number of osds that will be drained simultaneously when osds are removed","long_desc":"","tags":[],"see_also":[]},"migration_current":{"name":"migration_current","type":"int","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"internal - do not modify","long_desc":"","tags":[],"see_also":[]},"mode":{"name":"mode","type":"str","level":"advanced","flags":0,"default_value":"root","min":"","max":"","enum_allowed":["cephadm-package","root"],"desc":"mode for remote execution of cephadm","long_desc":"","tags":[],"see_also":[]},"oob_default_addr":{"name":"oob_default_addr","type":"str","level":"advanced","flags":0,"default_value":"169.254.1.1","min":"","max":"","enum_allowed":[],"desc":"Default address for RedFish API (oob management).","long_desc":"","tags":[],"see_also":[]},"prometheus_alerts_path":{"name":"prometheus_alerts_path","type":"str","level":"advanced","flags":0,"default_value":"/etc/prometheus/ceph/ceph_default_alerts.yml","min":"","max":"","enum_allowed":[],"desc":"location of alerts to include in prometheus deployments","long_desc":"","tags":[],"see_also":[]},"registry_insecure":{"name":"registry_insecure","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Registry is to be considered insecure (no TLS available). Only for development purposes.","long_desc":"","tags":[],"see_also":[]},"registry_password":{"name":"registry_password","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Custom repository password. Only used for logging into a registry.","long_desc":"","tags":[],"see_also":[]},"registry_url":{"name":"registry_url","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Registry url for login purposes. This is not the default registry","long_desc":"","tags":[],"see_also":[]},"registry_username":{"name":"registry_username","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Custom repository username. Only used for logging into a registry.","long_desc":"","tags":[],"see_also":[]},"secure_monitoring_stack":{"name":"secure_monitoring_stack","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Enable TLS security for all the monitoring stack daemons","long_desc":"","tags":[],"see_also":[]},"service_discovery_port":{"name":"service_discovery_port","type":"int","level":"advanced","flags":0,"default_value":"8765","min":"","max":"","enum_allowed":[],"desc":"cephadm service discovery port","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ssh_config_file":{"name":"ssh_config_file","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"customized SSH config file to connect to managed hosts","long_desc":"","tags":[],"see_also":[]},"use_agent":{"name":"use_agent","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Use cephadm agent on each host to gather and send metadata","long_desc":"","tags":[],"see_also":[]},"use_repo_digest":{"name":"use_repo_digest","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Automatically convert image tags to image digest. Make sure all daemons use the same image","long_desc":"","tags":[],"see_also":[]},"warn_on_failed_host_check":{"name":"warn_on_failed_host_check","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"raise a health warning if the host check fails","long_desc":"","tags":[],"see_also":[]},"warn_on_stray_daemons":{"name":"warn_on_stray_daemons","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"raise a health warning if daemons are detected that are not managed by cephadm","long_desc":"","tags":[],"see_also":[]},"warn_on_stray_hosts":{"name":"warn_on_stray_hosts","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"raise a health warning if daemons are detected on a host that is not managed by cephadm","long_desc":"","tags":[],"see_also":[]}}},{"name":"crash","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"retain_interval":{"name":"retain_interval","type":"secs","level":"advanced","flags":1,"default_value":"31536000","min":"","max":"","enum_allowed":[],"desc":"how long to retain crashes before pruning them","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"warn_recent_interval":{"name":"warn_recent_interval","type":"secs","level":"advanced","flags":1,"default_value":"1209600","min":"","max":"","enum_allowed":[],"desc":"time interval in which to warn about recent crashes","long_desc":"","tags":[],"see_also":[]}}},{"name":"dashboard","can_run":true,"error_string":"","module_options":{"ACCOUNT_LOCKOUT_ATTEMPTS":{"name":"ACCOUNT_LOCKOUT_ATTEMPTS","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ALERTMANAGER_API_HOST":{"name":"ALERTMANAGER_API_HOST","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ALERTMANAGER_API_SSL_VERIFY":{"name":"ALERTMANAGER_API_SSL_VERIFY","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"AUDIT_API_ENABLED":{"name":"AUDIT_API_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"AUDIT_API_LOG_PAYLOAD":{"name":"AUDIT_API_LOG_PAYLOAD","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ENABLE_BROWSABLE_API":{"name":"ENABLE_BROWSABLE_API","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_CEPHFS":{"name":"FEATURE_TOGGLE_CEPHFS","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_DASHBOARD":{"name":"FEATURE_TOGGLE_DASHBOARD","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_ISCSI":{"name":"FEATURE_TOGGLE_ISCSI","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_MIRRORING":{"name":"FEATURE_TOGGLE_MIRRORING","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_NFS":{"name":"FEATURE_TOGGLE_NFS","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_RBD":{"name":"FEATURE_TOGGLE_RBD","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_RGW":{"name":"FEATURE_TOGGLE_RGW","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GANESHA_CLUSTERS_RADOS_POOL_NAMESPACE":{"name":"GANESHA_CLUSTERS_RADOS_POOL_NAMESPACE","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_PASSWORD":{"name":"GRAFANA_API_PASSWORD","type":"str","level":"advanced","flags":0,"default_value":"admin","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_SSL_VERIFY":{"name":"GRAFANA_API_SSL_VERIFY","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_URL":{"name":"GRAFANA_API_URL","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_USERNAME":{"name":"GRAFANA_API_USERNAME","type":"str","level":"advanced","flags":0,"default_value":"admin","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_FRONTEND_API_URL":{"name":"GRAFANA_FRONTEND_API_URL","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_UPDATE_DASHBOARDS":{"name":"GRAFANA_UPDATE_DASHBOARDS","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ISCSI_API_SSL_VERIFICATION":{"name":"ISCSI_API_SSL_VERIFICATION","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ISSUE_TRACKER_API_KEY":{"name":"ISSUE_TRACKER_API_KEY","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"MANAGED_BY_CLUSTERS":{"name":"MANAGED_BY_CLUSTERS","type":"str","level":"advanced","flags":0,"default_value":"[]","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"MULTICLUSTER_CONFIG":{"name":"MULTICLUSTER_CONFIG","type":"str","level":"advanced","flags":0,"default_value":"{}","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PROMETHEUS_API_HOST":{"name":"PROMETHEUS_API_HOST","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PROMETHEUS_API_SSL_VERIFY":{"name":"PROMETHEUS_API_SSL_VERIFY","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_COMPLEXITY_ENABLED":{"name":"PWD_POLICY_CHECK_COMPLEXITY_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_EXCLUSION_LIST_ENABLED":{"name":"PWD_POLICY_CHECK_EXCLUSION_LIST_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_LENGTH_ENABLED":{"name":"PWD_POLICY_CHECK_LENGTH_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_OLDPWD_ENABLED":{"name":"PWD_POLICY_CHECK_OLDPWD_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_REPETITIVE_CHARS_ENABLED":{"name":"PWD_POLICY_CHECK_REPETITIVE_CHARS_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_SEQUENTIAL_CHARS_ENABLED":{"name":"PWD_POLICY_CHECK_SEQUENTIAL_CHARS_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_USERNAME_ENABLED":{"name":"PWD_POLICY_CHECK_USERNAME_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_ENABLED":{"name":"PWD_POLICY_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_EXCLUSION_LIST":{"name":"PWD_POLICY_EXCLUSION_LIST","type":"str","level":"advanced","flags":0,"default_value":"osd,host,dashboard,pool,block,nfs,ceph,monitors,gateway,logs,crush,maps","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_MIN_COMPLEXITY":{"name":"PWD_POLICY_MIN_COMPLEXITY","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_MIN_LENGTH":{"name":"PWD_POLICY_MIN_LENGTH","type":"int","level":"advanced","flags":0,"default_value":"8","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"REST_REQUESTS_TIMEOUT":{"name":"REST_REQUESTS_TIMEOUT","type":"int","level":"advanced","flags":0,"default_value":"45","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_ACCESS_KEY":{"name":"RGW_API_ACCESS_KEY","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_ADMIN_RESOURCE":{"name":"RGW_API_ADMIN_RESOURCE","type":"str","level":"advanced","flags":0,"default_value":"admin","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_SECRET_KEY":{"name":"RGW_API_SECRET_KEY","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_SSL_VERIFY":{"name":"RGW_API_SSL_VERIFY","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"UNSAFE_TLS_v1_2":{"name":"UNSAFE_TLS_v1_2","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"USER_PWD_EXPIRATION_SPAN":{"name":"USER_PWD_EXPIRATION_SPAN","type":"int","level":"advanced","flags":0,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"USER_PWD_EXPIRATION_WARNING_1":{"name":"USER_PWD_EXPIRATION_WARNING_1","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"USER_PWD_EXPIRATION_WARNING_2":{"name":"USER_PWD_EXPIRATION_WARNING_2","type":"int","level":"advanced","flags":0,"default_value":"5","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"cross_origin_url":{"name":"cross_origin_url","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"crt_file":{"name":"crt_file","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"debug":{"name":"debug","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Enable/disable debug options","long_desc":"","tags":[],"see_also":[]},"jwt_token_ttl":{"name":"jwt_token_ttl","type":"int","level":"advanced","flags":0,"default_value":"28800","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"key_file":{"name":"key_file","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"motd":{"name":"motd","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"The message of the day","long_desc":"","tags":[],"see_also":[]},"redirect_resolve_ip_addr":{"name":"redirect_resolve_ip_addr","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_addr":{"name":"server_addr","type":"str","level":"advanced","flags":0,"default_value":"::","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_port":{"name":"server_port","type":"int","level":"advanced","flags":0,"default_value":"8080","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ssl":{"name":"ssl","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ssl_server_port":{"name":"ssl_server_port","type":"int","level":"advanced","flags":0,"default_value":"8443","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"standby_behaviour":{"name":"standby_behaviour","type":"str","level":"advanced","flags":0,"default_value":"redirect","min":"","max":"","enum_allowed":["error","redirect"],"desc":"","long_desc":"","tags":[],"see_also":[]},"standby_error_status_code":{"name":"standby_error_status_code","type":"int","level":"advanced","flags":0,"default_value":"500","min":"400","max":"599","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"url_prefix":{"name":"url_prefix","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"devicehealth","can_run":true,"error_string":"","module_options":{"enable_monitoring":{"name":"enable_monitoring","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"monitor device health metrics","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"mark_out_threshold":{"name":"mark_out_threshold","type":"secs","level":"advanced","flags":1,"default_value":"2419200","min":"","max":"","enum_allowed":[],"desc":"automatically mark OSD if it may fail before this long","long_desc":"","tags":[],"see_also":[]},"pool_name":{"name":"pool_name","type":"str","level":"advanced","flags":1,"default_value":"device_health_metrics","min":"","max":"","enum_allowed":[],"desc":"name of pool in which to store device health metrics","long_desc":"","tags":[],"see_also":[]},"retention_period":{"name":"retention_period","type":"secs","level":"advanced","flags":1,"default_value":"15552000","min":"","max":"","enum_allowed":[],"desc":"how long to retain device health metrics","long_desc":"","tags":[],"see_also":[]},"scrape_frequency":{"name":"scrape_frequency","type":"secs","level":"advanced","flags":1,"default_value":"86400","min":"","max":"","enum_allowed":[],"desc":"how frequently to scrape device health metrics","long_desc":"","tags":[],"see_also":[]},"self_heal":{"name":"self_heal","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"preemptively heal cluster around devices that may fail","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"secs","level":"advanced","flags":1,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"how frequently to wake up and check device health","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"warn_threshold":{"name":"warn_threshold","type":"secs","level":"advanced","flags":1,"default_value":"7257600","min":"","max":"","enum_allowed":[],"desc":"raise health warning if OSD may fail before this long","long_desc":"","tags":[],"see_also":[]}}},{"name":"diskprediction_local","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"predict_interval":{"name":"predict_interval","type":"str","level":"advanced","flags":0,"default_value":"86400","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"predictor_model":{"name":"predictor_model","type":"str","level":"advanced","flags":0,"default_value":"prophetstor","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"str","level":"advanced","flags":0,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"influx","can_run":false,"error_string":"influxdb python module not found","module_options":{"batch_size":{"name":"batch_size","type":"int","level":"advanced","flags":0,"default_value":"5000","min":"","max":"","enum_allowed":[],"desc":"How big batches of data points should be when sending to InfluxDB.","long_desc":"","tags":[],"see_also":[]},"database":{"name":"database","type":"str","level":"advanced","flags":0,"default_value":"ceph","min":"","max":"","enum_allowed":[],"desc":"InfluxDB database name. You will need to create this database and grant write privileges to the configured username or the username must have admin privileges to create it.","long_desc":"","tags":[],"see_also":[]},"hostname":{"name":"hostname","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"InfluxDB server hostname","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"secs","level":"advanced","flags":0,"default_value":"30","min":"5","max":"","enum_allowed":[],"desc":"Time between reports to InfluxDB. Default 30 seconds.","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"password":{"name":"password","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"password of InfluxDB server user","long_desc":"","tags":[],"see_also":[]},"port":{"name":"port","type":"int","level":"advanced","flags":0,"default_value":"8086","min":"","max":"","enum_allowed":[],"desc":"InfluxDB server port","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ssl":{"name":"ssl","type":"str","level":"advanced","flags":0,"default_value":"false","min":"","max":"","enum_allowed":[],"desc":"Use https connection for InfluxDB server. Use \"true\" or \"false\".","long_desc":"","tags":[],"see_also":[]},"threads":{"name":"threads","type":"int","level":"advanced","flags":0,"default_value":"5","min":"1","max":"32","enum_allowed":[],"desc":"How many worker threads should be spawned for sending data to InfluxDB.","long_desc":"","tags":[],"see_also":[]},"username":{"name":"username","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"username of InfluxDB server user","long_desc":"","tags":[],"see_also":[]},"verify_ssl":{"name":"verify_ssl","type":"str","level":"advanced","flags":0,"default_value":"true","min":"","max":"","enum_allowed":[],"desc":"Verify https cert for InfluxDB server. Use \"true\" or \"false\".","long_desc":"","tags":[],"see_also":[]}}},{"name":"insights","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"iostat","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"localpool","can_run":true,"error_string":"","module_options":{"failure_domain":{"name":"failure_domain","type":"str","level":"advanced","flags":1,"default_value":"host","min":"","max":"","enum_allowed":[],"desc":"failure domain for any created local pool","long_desc":"what failure domain we should separate data replicas across.","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"min_size":{"name":"min_size","type":"int","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"default min_size for any created local pool","long_desc":"value to set min_size to (unchanged from Ceph's default if this option is not set)","tags":[],"see_also":[]},"num_rep":{"name":"num_rep","type":"int","level":"advanced","flags":1,"default_value":"3","min":"","max":"","enum_allowed":[],"desc":"default replica count for any created local pool","long_desc":"","tags":[],"see_also":[]},"pg_num":{"name":"pg_num","type":"int","level":"advanced","flags":1,"default_value":"128","min":"","max":"","enum_allowed":[],"desc":"default pg_num for any created local pool","long_desc":"","tags":[],"see_also":[]},"prefix":{"name":"prefix","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"name prefix for any created local pool","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"subtree":{"name":"subtree","type":"str","level":"advanced","flags":1,"default_value":"rack","min":"","max":"","enum_allowed":[],"desc":"CRUSH level for which to create a local pool","long_desc":"which CRUSH subtree type the module should create a pool for.","tags":[],"see_also":[]}}},{"name":"mds_autoscaler","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"mirroring","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"nfs","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"orchestrator","can_run":true,"error_string":"","module_options":{"fail_fs":{"name":"fail_fs","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Fail filesystem for rapid multi-rank mds upgrade","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"orchestrator":{"name":"orchestrator","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["cephadm","rook","test_orchestrator"],"desc":"Orchestrator backend","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"osd_perf_query","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"osd_support","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"pg_autoscaler","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"secs","level":"advanced","flags":0,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"threshold":{"name":"threshold","type":"float","level":"advanced","flags":0,"default_value":"3.0","min":"1.0","max":"","enum_allowed":[],"desc":"scaling threshold","long_desc":"The factor by which the `NEW PG_NUM` must vary from the current`PG_NUM` before being accepted. Cannot be less than 1.0","tags":[],"see_also":[]}}},{"name":"progress","can_run":true,"error_string":"","module_options":{"allow_pg_recovery_event":{"name":"allow_pg_recovery_event","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"allow the module to show pg recovery progress","long_desc":"","tags":[],"see_also":[]},"enabled":{"name":"enabled","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"max_completed_events":{"name":"max_completed_events","type":"int","level":"advanced","flags":1,"default_value":"50","min":"","max":"","enum_allowed":[],"desc":"number of past completed events to remember","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"secs","level":"advanced","flags":1,"default_value":"5","min":"","max":"","enum_allowed":[],"desc":"how long the module is going to sleep","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"prometheus","can_run":true,"error_string":"","module_options":{"cache":{"name":"cache","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"exclude_perf_counters":{"name":"exclude_perf_counters","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Do not include perf-counters in the metrics output","long_desc":"Gathering perf-counters from a single Prometheus exporter can degrade ceph-mgr performance, especially in large clusters. Instead, Ceph-exporter daemons are now used by default for perf-counter gathering. This should only be disabled when no ceph-exporters are deployed.","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rbd_stats_pools":{"name":"rbd_stats_pools","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rbd_stats_pools_refresh_interval":{"name":"rbd_stats_pools_refresh_interval","type":"int","level":"advanced","flags":0,"default_value":"300","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"scrape_interval":{"name":"scrape_interval","type":"float","level":"advanced","flags":0,"default_value":"15.0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_addr":{"name":"server_addr","type":"str","level":"advanced","flags":0,"default_value":"::","min":"","max":"","enum_allowed":[],"desc":"the IPv4 or IPv6 address on which the module listens for HTTP requests","long_desc":"","tags":[],"see_also":[]},"server_port":{"name":"server_port","type":"int","level":"advanced","flags":1,"default_value":"9283","min":"","max":"","enum_allowed":[],"desc":"the port on which the module listens for HTTP requests","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"stale_cache_strategy":{"name":"stale_cache_strategy","type":"str","level":"advanced","flags":0,"default_value":"log","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"standby_behaviour":{"name":"standby_behaviour","type":"str","level":"advanced","flags":1,"default_value":"default","min":"","max":"","enum_allowed":["default","error"],"desc":"","long_desc":"","tags":[],"see_also":[]},"standby_error_status_code":{"name":"standby_error_status_code","type":"int","level":"advanced","flags":1,"default_value":"500","min":"400","max":"599","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"rbd_support","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"max_concurrent_snap_create":{"name":"max_concurrent_snap_create","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"mirror_snapshot_schedule":{"name":"mirror_snapshot_schedule","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"trash_purge_schedule":{"name":"trash_purge_schedule","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"restful","can_run":true,"error_string":"","module_options":{"enable_auth":{"name":"enable_auth","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"key_file":{"name":"key_file","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_addr":{"name":"server_addr","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_port":{"name":"server_port","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"rgw","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"rook","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"storage_class":{"name":"storage_class","type":"str","level":"advanced","flags":0,"default_value":"local","min":"","max":"","enum_allowed":[],"desc":"storage class name for LSO-discovered PVs","long_desc":"","tags":[],"see_also":[]}}},{"name":"selftest","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"roption1":{"name":"roption1","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"roption2":{"name":"roption2","type":"str","level":"advanced","flags":0,"default_value":"xyz","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption1":{"name":"rwoption1","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption2":{"name":"rwoption2","type":"int","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption3":{"name":"rwoption3","type":"float","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption4":{"name":"rwoption4","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption5":{"name":"rwoption5","type":"bool","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption6":{"name":"rwoption6","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption7":{"name":"rwoption7","type":"int","level":"advanced","flags":0,"default_value":"","min":"1","max":"42","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"testkey":{"name":"testkey","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"testlkey":{"name":"testlkey","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"testnewline":{"name":"testnewline","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"snap_schedule","can_run":true,"error_string":"","module_options":{"allow_m_granularity":{"name":"allow_m_granularity","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"allow minute scheduled snapshots","long_desc":"","tags":[],"see_also":[]},"dump_on_update":{"name":"dump_on_update","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"dump database to debug log on update","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"stats","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"status","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"telegraf","can_run":true,"error_string":"","module_options":{"address":{"name":"address","type":"str","level":"advanced","flags":0,"default_value":"unixgram:///tmp/telegraf.sock","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"secs","level":"advanced","flags":0,"default_value":"15","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"telemetry","can_run":true,"error_string":"","module_options":{"channel_basic":{"name":"channel_basic","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Share basic cluster information (size, version)","long_desc":"","tags":[],"see_also":[]},"channel_crash":{"name":"channel_crash","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Share metadata about Ceph daemon crashes (version, stack straces, etc)","long_desc":"","tags":[],"see_also":[]},"channel_device":{"name":"channel_device","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Share device health metrics (e.g., SMART data, minus potentially identifying info like serial numbers)","long_desc":"","tags":[],"see_also":[]},"channel_ident":{"name":"channel_ident","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Share a user-provided description and/or contact email for the cluster","long_desc":"","tags":[],"see_also":[]},"channel_perf":{"name":"channel_perf","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Share various performance metrics of a cluster","long_desc":"","tags":[],"see_also":[]},"contact":{"name":"contact","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"description":{"name":"description","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"device_url":{"name":"device_url","type":"str","level":"advanced","flags":0,"default_value":"https://telemetry.ceph.com/device","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"enabled":{"name":"enabled","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"int","level":"advanced","flags":0,"default_value":"24","min":"8","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"last_opt_revision":{"name":"last_opt_revision","type":"int","level":"advanced","flags":0,"default_value":"1","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"leaderboard":{"name":"leaderboard","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"leaderboard_description":{"name":"leaderboard_description","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"organization":{"name":"organization","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"proxy":{"name":"proxy","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"url":{"name":"url","type":"str","level":"advanced","flags":0,"default_value":"https://telemetry.ceph.com/report","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"test_orchestrator","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"volumes","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"max_concurrent_clones":{"name":"max_concurrent_clones","type":"int","level":"advanced","flags":0,"default_value":"4","min":"","max":"","enum_allowed":[],"desc":"Number of asynchronous cloner threads","long_desc":"","tags":[],"see_also":[]},"periodic_async_work":{"name":"periodic_async_work","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Periodically check for async work","long_desc":"","tags":[],"see_also":[]},"snapshot_clone_delay":{"name":"snapshot_clone_delay","type":"int","level":"advanced","flags":0,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"Delay clone begin operation by snapshot_clone_delay seconds","long_desc":"","tags":[],"see_also":[]},"snapshot_clone_no_wait":{"name":"snapshot_clone_no_wait","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Reject subvolume clone request when cloner threads are busy","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"zabbix","can_run":true,"error_string":"","module_options":{"discovery_interval":{"name":"discovery_interval","type":"uint","level":"advanced","flags":0,"default_value":"100","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"identifier":{"name":"identifier","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"secs","level":"advanced","flags":0,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"zabbix_host":{"name":"zabbix_host","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"zabbix_port":{"name":"zabbix_port","type":"int","level":"advanced","flags":0,"default_value":"10051","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"zabbix_sender":{"name":"zabbix_sender","type":"str","level":"advanced","flags":0,"default_value":"/usr/bin/zabbix_sender","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}}],"services":{},"always_on_modules":{"octopus":["balancer","crash","devicehealth","orchestrator","pg_autoscaler","progress","rbd_support","status","telemetry","volumes"],"pacific":["balancer","crash","devicehealth","orchestrator","pg_autoscaler","progress","rbd_support","status","telemetry","volumes"],"quincy":["balancer","crash","devicehealth","orchestrator","pg_autoscaler","progress","rbd_support","status","telemetry","volumes"],"reef":["balancer","crash","devicehealth","orchestrator","pg_autoscaler","progress","rbd_support","status","telemetry","volumes"],"squid":["balancer","crash","devicehealth","orchestrator","pg_autoscaler","progress","rbd_support","status","telemetry","volumes"]},"last_failure_osd_epoch":0,"active_clients":[{"name":"devicehealth","addrvec":[{"type":"v2","addr":"172.21.15.203:0","nonce":1066889528}]},{"name":"libcephsqlite","addrvec":[{"type":"v2","addr":"172.21.15.203:0","nonce":1619399744}]},{"name":"rbd_support","addrvec":[{"type":"v2","addr":"172.21.15.203:0","nonce":2151284732}]},{"name":"volumes","addrvec":[{"type":"v2","addr":"172.21.15.203:0","nonce":2501837343}]}]} 2024-04-25T10:29:18.459 INFO:tasks.ceph.ceph_manager.ceph:mgr available! 2024-04-25T10:29:18.459 INFO:tasks.ceph.ceph_manager.ceph:waiting for all up 2024-04-25T10:29:18.460 DEBUG:teuthology.orchestra.run.smithi037:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-04-25T10:29:18.570 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:18.568+0000 7f85dff57640 1 Processor -- start 2024-04-25T10:29:18.570 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:18.568+0000 7f85dff57640 1 -- start start 2024-04-25T10:29:18.572 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:18.570+0000 7f85dff57640 1 --2- >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f85d80b50e0 0x7f85d80b79d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:29:18.572 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:18.570+0000 7f85dff57640 1 --2- >> v2:172.21.15.37:3300/0 conn(0x7f85d80d1030 0x7f85d815f7c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:29:18.572 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:18.570+0000 7f85dff57640 1 -- --> v1:172.21.15.203:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f85d80b8320 con 0x7f85d80b7f10 2024-04-25T10:29:18.573 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:18.570+0000 7f85dff57640 1 -- --> v2:172.21.15.37:3300/0 -- mon_getmap magic: 0 -- 0x7f85d81646e0 con 0x7f85d80d1030 2024-04-25T10:29:18.573 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:18.570+0000 7f85dff57640 1 -- --> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] -- mon_getmap magic: 0 -- 0x7f85d80d15e0 con 0x7f85d80b50e0 2024-04-25T10:29:18.573 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:18.570+0000 7f85de4f4640 1 --2- >> v2:172.21.15.37:3300/0 conn(0x7f85d80d1030 0x7f85d815f7c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:29:18.573 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:18.570+0000 7f85de4f4640 1 --2- >> v2:172.21.15.37:3300/0 conn(0x7f85d80d1030 0x7f85d815f7c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.37:3300/0 says I am v2:172.21.15.37:46394/0 (socket says 172.21.15.37:46394) 2024-04-25T10:29:18.573 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:18.570+0000 7f85de4f4640 1 -- 172.21.15.37:0/279772953 learned_addr learned my addr 172.21.15.37:0/279772953 (peer_addr_for_me v2:172.21.15.37:0/0) 2024-04-25T10:29:18.573 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:18.570+0000 7f85ddcf3640 1 --2- 172.21.15.37:0/279772953 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f85d80b50e0 0x7f85d80b79d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:29:18.573 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:18.570+0000 7f85de4f4640 1 -- 172.21.15.37:0/279772953 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f85d80b50e0 msgr2=0x7f85d80b79d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:29:18.573 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:18.570+0000 7f85de4f4640 1 --2- 172.21.15.37:0/279772953 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f85d80b50e0 0x7f85d80b79d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:29:18.573 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:18.570+0000 7f85de4f4640 1 -- 172.21.15.37:0/279772953 >> v1:172.21.15.203:6789/0 conn(0x7f85d80b7f10 legacy=0x7f85d80d0920 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:29:18.573 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:18.570+0000 7f85de4f4640 1 -- 172.21.15.37:0/279772953 --> v2:172.21.15.37:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f85d8163db0 con 0x7f85d80d1030 2024-04-25T10:29:18.574 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:18.571+0000 7f85de4f4640 1 --2- 172.21.15.37:0/279772953 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f85d80d1030 0x7f85d815f7c0 secure :-1 s=READY pgs=95 cs=0 l=1 rev1=1 crypto rx=0x7f85d0009e80 tx=0x7f85d002a6f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=16464dedea33a94c server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:29:18.575 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:18.571+0000 7f85dccf1640 1 -- 172.21.15.37:0/279772953 <== mon.0 v2:172.21.15.37:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f85d0036070 con 0x7f85d80d1030 2024-04-25T10:29:18.575 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:18.571+0000 7f85dccf1640 1 -- 172.21.15.37:0/279772953 <== mon.0 v2:172.21.15.37:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f85d0031600 con 0x7f85d80d1030 2024-04-25T10:29:18.575 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:18.571+0000 7f85dccf1640 1 -- 172.21.15.37:0/279772953 <== mon.0 v2:172.21.15.37:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f85d00319d0 con 0x7f85d80d1030 2024-04-25T10:29:18.575 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:18.572+0000 7f85dff57640 1 -- 172.21.15.37:0/279772953 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f85d80d1030 msgr2=0x7f85d815f7c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:29:18.575 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:18.572+0000 7f85dff57640 1 --2- 172.21.15.37:0/279772953 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f85d80d1030 0x7f85d815f7c0 secure :-1 s=READY pgs=95 cs=0 l=1 rev1=1 crypto rx=0x7f85d0009e80 tx=0x7f85d002a6f0 comp rx=0 tx=0).stop 2024-04-25T10:29:18.575 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:18.572+0000 7f85dff57640 1 -- 172.21.15.37:0/279772953 shutdown_connections 2024-04-25T10:29:18.575 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:18.572+0000 7f85dff57640 1 --2- 172.21.15.37:0/279772953 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f85d80d1030 0x7f85d815f7c0 unknown :-1 s=CLOSED pgs=95 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:29:18.575 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:18.572+0000 7f85dff57640 1 --2- 172.21.15.37:0/279772953 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f85d80b50e0 0x7f85d80b79d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:29:18.575 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:18.572+0000 7f85dff57640 1 -- 172.21.15.37:0/279772953 >> 172.21.15.37:0/279772953 conn(0x7f85d80d5d50 msgr2=0x7f85d80d6160 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:29:18.575 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:18.572+0000 7f85dff57640 1 -- 172.21.15.37:0/279772953 shutdown_connections 2024-04-25T10:29:18.575 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:18.572+0000 7f85dff57640 1 -- 172.21.15.37:0/279772953 wait complete. 2024-04-25T10:29:18.576 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:18.574+0000 7f85dff57640 1 Processor -- start 2024-04-25T10:29:18.576 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:18.574+0000 7f85dff57640 1 -- start start 2024-04-25T10:29:18.578 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:18.575+0000 7f85dff57640 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f85d80b50e0 0x7f85d8161bb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:29:18.578 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:18.575+0000 7f85dff57640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f85d80b7f10 0x7f85d81f6410 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:29:18.578 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:18.575+0000 7f85dff57640 1 --2- >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f85d80d1030 0x7f85d80ce8d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:29:18.579 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:18.575+0000 7f85dff57640 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 -- 0x7f85d8164b70 con 0x7f85d80b7f10 2024-04-25T10:29:18.579 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:18.575+0000 7f85dff57640 1 -- --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_getmap magic: 0 -- 0x7f85d8164370 con 0x7f85d80b50e0 2024-04-25T10:29:18.579 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:18.575+0000 7f85dff57640 1 -- --> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] -- mon_getmap magic: 0 -- 0x7f85d815fd00 con 0x7f85d80d1030 2024-04-25T10:29:18.579 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:18.575+0000 7f85dd4f2640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f85d80b7f10 0x7f85d81f6410 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:29:18.579 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:18.575+0000 7f85dd4f2640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f85d80b7f10 0x7f85d81f6410 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.37:3300/0 says I am v2:172.21.15.37:46404/0 (socket says 172.21.15.37:46404) 2024-04-25T10:29:18.579 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:18.575+0000 7f85dd4f2640 1 -- 172.21.15.37:0/3157126745 learned_addr learned my addr 172.21.15.37:0/3157126745 (peer_addr_for_me v2:172.21.15.37:0/0) 2024-04-25T10:29:18.579 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:18.575+0000 7f85ddcf3640 1 --2- 172.21.15.37:0/3157126745 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f85d80b50e0 0x7f85d8161bb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:29:18.579 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:18.575+0000 7f85de4f4640 1 --2- 172.21.15.37:0/3157126745 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f85d80d1030 0x7f85d80ce8d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:29:18.579 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:18.575+0000 7f85dd4f2640 1 -- 172.21.15.37:0/3157126745 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f85d80d1030 msgr2=0x7f85d80ce8d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:29:18.579 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:18.575+0000 7f85dd4f2640 1 --2- 172.21.15.37:0/3157126745 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f85d80d1030 0x7f85d80ce8d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:29:18.579 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:18.575+0000 7f85dd4f2640 1 -- 172.21.15.37:0/3157126745 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f85d80b50e0 msgr2=0x7f85d8161bb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:29:18.579 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:18.575+0000 7f85dd4f2640 1 --2- 172.21.15.37:0/3157126745 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f85d80b50e0 0x7f85d8161bb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:29:18.579 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:18.575+0000 7f85dd4f2640 1 -- 172.21.15.37:0/3157126745 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f85d81e9ba0 con 0x7f85d80b7f10 2024-04-25T10:29:18.579 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:18.576+0000 7f85dd4f2640 1 --2- 172.21.15.37:0/3157126745 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f85d80b7f10 0x7f85d81f6410 secure :-1 s=READY pgs=96 cs=0 l=1 rev1=1 crypto rx=0x7f85d80d00e0 tx=0x7f85c800b820 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:29:18.580 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:18.576+0000 7f85ce7fc640 1 -- 172.21.15.37:0/3157126745 <== mon.0 v2:172.21.15.37:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f85c8004230 con 0x7f85d80b7f10 2024-04-25T10:29:18.580 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:18.576+0000 7f85ce7fc640 1 -- 172.21.15.37:0/3157126745 <== mon.0 v2:172.21.15.37:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f85c80043d0 con 0x7f85d80b7f10 2024-04-25T10:29:18.580 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:18.576+0000 7f85dff57640 1 -- 172.21.15.37:0/3157126745 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f85d81e9e80 con 0x7f85d80b7f10 2024-04-25T10:29:18.580 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:18.576+0000 7f85ce7fc640 1 -- 172.21.15.37:0/3157126745 <== mon.0 v2:172.21.15.37:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f85c8011850 con 0x7f85d80b7f10 2024-04-25T10:29:18.580 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:18.576+0000 7f85dff57640 1 -- 172.21.15.37:0/3157126745 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f85d81ea440 con 0x7f85d80b7f10 2024-04-25T10:29:18.581 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:18.577+0000 7f85dff57640 1 -- 172.21.15.37:0/3157126745 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f85d81621e0 con 0x7f85d80b7f10 2024-04-25T10:29:18.581 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:18.577+0000 7f85ce7fc640 1 -- 172.21.15.37:0/3157126745 <== mon.0 v2:172.21.15.37:3300/0 4 ==== mgrmap(e 5) ==== 96782+0+0 (secure 0 0 0) 0x7f85c80119f0 con 0x7f85d80b7f10 2024-04-25T10:29:18.581 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:18.578+0000 7f85ce7fc640 1 --2- 172.21.15.37:0/3157126745 >> [v2:172.21.15.203:6832/2418205237,v1:172.21.15.203:6833/2418205237] conn(0x7f85bc0744b0 0x7f85bc0769c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:29:18.581 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:18.578+0000 7f85ce7fc640 1 -- 172.21.15.37:0/3157126745 <== mon.0 v2:172.21.15.37:3300/0 5 ==== osd_map(23..23 src has 1..23) ==== 5709+0+0 (secure 0 0 0) 0x7f85c8095330 con 0x7f85d80b7f10 2024-04-25T10:29:18.581 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:18.578+0000 7f85ddcf3640 1 --2- 172.21.15.37:0/3157126745 >> [v2:172.21.15.203:6832/2418205237,v1:172.21.15.203:6833/2418205237] conn(0x7f85bc0744b0 0x7f85bc0769c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:29:18.581 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:18.579+0000 7f85ddcf3640 1 --2- 172.21.15.37:0/3157126745 >> [v2:172.21.15.203:6832/2418205237,v1:172.21.15.203:6833/2418205237] conn(0x7f85bc0744b0 0x7f85bc0769c0 secure :-1 s=READY pgs=71 cs=0 l=1 rev1=1 crypto rx=0x7f85d4006fd0 tx=0x7f85d4008040 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:29:18.584 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:18.583+0000 7f85ce7fc640 1 -- 172.21.15.37:0/3157126745 <== mon.0 v2:172.21.15.37:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187246 (secure 0 0 0) 0x7f85c805fac0 con 0x7f85d80b7f10 2024-04-25T10:29:18.770 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:18.768+0000 7f85dff57640 1 -- 172.21.15.37:0/3157126745 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7f85d806a140 con 0x7f85d80b7f10 2024-04-25T10:29:18.772 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:18.769+0000 7f85ce7fc640 1 -- 172.21.15.37:0/3157126745 <== mon.0 v2:172.21.15.37:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v23) ==== 74+0+16621 (secure 0 0 0) 0x7f85c805f3c0 con 0x7f85d80b7f10 2024-04-25T10:29:18.772 INFO:teuthology.orchestra.run.smithi037.stdout: 2024-04-25T10:29:18.772 INFO:teuthology.orchestra.run.smithi037.stdout:{"epoch":23,"fsid":"f4acda28-be34-492a-b286-994afd36f61d","created":"2024-04-25T10:28:43.985718+0000","modified":"2024-04-25T10:29:10.016188+0000","last_up_change":"2024-04-25T10:28:54.900654+0000","last_in_change":"2024-04-25T10:28:46.216973+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":3,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-04-25T10:28:56.202153+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"19","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2024-04-25T10:29:06.507244+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"22","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":2.25,"score_stable":2.25,"optimal_score":1,"raw_score_acting":2.25,"raw_score_stable":2.25,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2024-04-25T10:29:07.347844+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"23","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"a2f79028-b332-4166-9ebb-a7024a37ea22","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":21,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.37:6816","nonce":2746074720},{"type":"v1","addr":"172.21.15.37:6817","nonce":2746074720}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.37:6818","nonce":2746074720},{"type":"v1","addr":"172.21.15.37:6819","nonce":2746074720}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.37:6822","nonce":2746074720},{"type":"v1","addr":"172.21.15.37:6823","nonce":2746074720}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.37:6820","nonce":2746074720},{"type":"v1","addr":"172.21.15.37:6821","nonce":2746074720}]},"public_addr":"172.21.15.37:6817/2746074720","cluster_addr":"172.21.15.37:6819/2746074720","heartbeat_back_addr":"172.21.15.37:6823/2746074720","heartbeat_front_addr":"172.21.15.37:6821/2746074720","state":["exists","up"]},{"osd":1,"uuid":"973159de-c2ea-4083-aecc-69373d1c1132","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":21,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.37:6800","nonce":1690341662},{"type":"v1","addr":"172.21.15.37:6801","nonce":1690341662}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.37:6802","nonce":1690341662},{"type":"v1","addr":"172.21.15.37:6803","nonce":1690341662}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.37:6806","nonce":1690341662},{"type":"v1","addr":"172.21.15.37:6807","nonce":1690341662}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.37:6804","nonce":1690341662},{"type":"v1","addr":"172.21.15.37:6805","nonce":1690341662}]},"public_addr":"172.21.15.37:6801/1690341662","cluster_addr":"172.21.15.37:6803/1690341662","heartbeat_back_addr":"172.21.15.37:6807/1690341662","heartbeat_front_addr":"172.21.15.37:6805/1690341662","state":["exists","up"]},{"osd":2,"uuid":"4ae7a0e9-cb3f-484c-a75f-f3766b721c47","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":21,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.37:6808","nonce":2754669649},{"type":"v1","addr":"172.21.15.37:6809","nonce":2754669649}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.37:6810","nonce":2754669649},{"type":"v1","addr":"172.21.15.37:6811","nonce":2754669649}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.37:6814","nonce":2754669649},{"type":"v1","addr":"172.21.15.37:6815","nonce":2754669649}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.37:6812","nonce":2754669649},{"type":"v1","addr":"172.21.15.37:6813","nonce":2754669649}]},"public_addr":"172.21.15.37:6809/2754669649","cluster_addr":"172.21.15.37:6811/2754669649","heartbeat_back_addr":"172.21.15.37:6815/2754669649","heartbeat_front_addr":"172.21.15.37:6813/2754669649","state":["exists","up"]},{"osd":3,"uuid":"1ba4a746-1608-47a7-ace2-960a5cfae934","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":21,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.37:6824","nonce":1815290618},{"type":"v1","addr":"172.21.15.37:6825","nonce":1815290618}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.37:6826","nonce":1815290618},{"type":"v1","addr":"172.21.15.37:6827","nonce":1815290618}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.37:6830","nonce":1815290618},{"type":"v1","addr":"172.21.15.37:6831","nonce":1815290618}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.37:6828","nonce":1815290618},{"type":"v1","addr":"172.21.15.37:6829","nonce":1815290618}]},"public_addr":"172.21.15.37:6825/1815290618","cluster_addr":"172.21.15.37:6827/1815290618","heartbeat_back_addr":"172.21.15.37:6831/1815290618","heartbeat_front_addr":"172.21.15.37:6829/1815290618","state":["exists","up"]},{"osd":4,"uuid":"d4f59f5d-0413-4b79-87ee-bb212c1fcf62","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":21,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6800","nonce":4051532063},{"type":"v1","addr":"172.21.15.203:6801","nonce":4051532063}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6802","nonce":4051532063},{"type":"v1","addr":"172.21.15.203:6803","nonce":4051532063}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6806","nonce":4051532063},{"type":"v1","addr":"172.21.15.203:6807","nonce":4051532063}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6804","nonce":4051532063},{"type":"v1","addr":"172.21.15.203:6805","nonce":4051532063}]},"public_addr":"172.21.15.203:6801/4051532063","cluster_addr":"172.21.15.203:6803/4051532063","heartbeat_back_addr":"172.21.15.203:6807/4051532063","heartbeat_front_addr":"172.21.15.203:6805/4051532063","state":["exists","up"]},{"osd":5,"uuid":"69710a0e-41d8-4e7f-9a10-17ce540b67be","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":21,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6824","nonce":1203822890},{"type":"v1","addr":"172.21.15.203:6825","nonce":1203822890}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6826","nonce":1203822890},{"type":"v1","addr":"172.21.15.203:6827","nonce":1203822890}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6830","nonce":1203822890},{"type":"v1","addr":"172.21.15.203:6831","nonce":1203822890}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6828","nonce":1203822890},{"type":"v1","addr":"172.21.15.203:6829","nonce":1203822890}]},"public_addr":"172.21.15.203:6825/1203822890","cluster_addr":"172.21.15.203:6827/1203822890","heartbeat_back_addr":"172.21.15.203:6831/1203822890","heartbeat_front_addr":"172.21.15.203:6829/1203822890","state":["exists","up"]},{"osd":6,"uuid":"f29dd047-3b48-473d-a235-6642a6293f20","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":21,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6810","nonce":2309615014},{"type":"v1","addr":"172.21.15.203:6812","nonce":2309615014}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6814","nonce":2309615014},{"type":"v1","addr":"172.21.15.203:6816","nonce":2309615014}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6822","nonce":2309615014},{"type":"v1","addr":"172.21.15.203:6823","nonce":2309615014}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6818","nonce":2309615014},{"type":"v1","addr":"172.21.15.203:6820","nonce":2309615014}]},"public_addr":"172.21.15.203:6812/2309615014","cluster_addr":"172.21.15.203:6816/2309615014","heartbeat_back_addr":"172.21.15.203:6823/2309615014","heartbeat_front_addr":"172.21.15.203:6820/2309615014","state":["exists","up"]},{"osd":7,"uuid":"6e691644-378b-4fdc-8e86-fb5f56e31f29","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":21,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6808","nonce":3688099014},{"type":"v1","addr":"172.21.15.203:6809","nonce":3688099014}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6811","nonce":3688099014},{"type":"v1","addr":"172.21.15.203:6813","nonce":3688099014}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6819","nonce":3688099014},{"type":"v1","addr":"172.21.15.203:6821","nonce":3688099014}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6815","nonce":3688099014},{"type":"v1","addr":"172.21.15.203:6817","nonce":3688099014}]},"public_addr":"172.21.15.203:6809/3688099014","cluster_addr":"172.21.15.203:6813/3688099014","heartbeat_back_addr":"172.21.15.203:6821/3688099014","heartbeat_front_addr":"172.21.15.203:6817/3688099014","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T10:28:51.419141+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T10:28:51.366448+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T10:28:51.348276+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T10:28:51.380519+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T10:28:51.105812+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T10:28:51.384496+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T10:28:51.388755+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T10:28:51.366958+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-04-25T10:29:18.774 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:18.771+0000 7f85dff57640 1 -- 172.21.15.37:0/3157126745 >> [v2:172.21.15.203:6832/2418205237,v1:172.21.15.203:6833/2418205237] conn(0x7f85bc0744b0 msgr2=0x7f85bc0769c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:29:18.774 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:18.771+0000 7f85dff57640 1 --2- 172.21.15.37:0/3157126745 >> [v2:172.21.15.203:6832/2418205237,v1:172.21.15.203:6833/2418205237] conn(0x7f85bc0744b0 0x7f85bc0769c0 secure :-1 s=READY pgs=71 cs=0 l=1 rev1=1 crypto rx=0x7f85d4006fd0 tx=0x7f85d4008040 comp rx=0 tx=0).stop 2024-04-25T10:29:18.774 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:18.771+0000 7f85dff57640 1 -- 172.21.15.37:0/3157126745 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f85d80b7f10 msgr2=0x7f85d81f6410 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:29:18.774 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:18.771+0000 7f85dff57640 1 --2- 172.21.15.37:0/3157126745 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f85d80b7f10 0x7f85d81f6410 secure :-1 s=READY pgs=96 cs=0 l=1 rev1=1 crypto rx=0x7f85d80d00e0 tx=0x7f85c800b820 comp rx=0 tx=0).stop 2024-04-25T10:29:18.774 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:18.771+0000 7f85dff57640 1 -- 172.21.15.37:0/3157126745 shutdown_connections 2024-04-25T10:29:18.775 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:18.771+0000 7f85dff57640 1 --2- 172.21.15.37:0/3157126745 >> [v2:172.21.15.203:6832/2418205237,v1:172.21.15.203:6833/2418205237] conn(0x7f85bc0744b0 0x7f85bc0769c0 unknown :-1 s=CLOSED pgs=71 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:29:18.775 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:18.771+0000 7f85dff57640 1 --2- 172.21.15.37:0/3157126745 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f85d80d1030 0x7f85d80ce8d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:29:18.775 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:18.771+0000 7f85dff57640 1 --2- 172.21.15.37:0/3157126745 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f85d80b7f10 0x7f85d81f6410 unknown :-1 s=CLOSED pgs=96 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:29:18.775 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:18.771+0000 7f85dff57640 1 --2- 172.21.15.37:0/3157126745 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f85d80b50e0 0x7f85d8161bb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:29:18.775 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:18.771+0000 7f85dff57640 1 -- 172.21.15.37:0/3157126745 >> 172.21.15.37:0/3157126745 conn(0x7f85d80d5d50 msgr2=0x7f85d80b6eb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:29:18.775 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:18.772+0000 7f85dff57640 1 -- 172.21.15.37:0/3157126745 shutdown_connections 2024-04-25T10:29:18.775 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:18.772+0000 7f85dff57640 1 -- 172.21.15.37:0/3157126745 wait complete. 2024-04-25T10:29:18.785 INFO:tasks.ceph.ceph_manager.ceph:all up! 2024-04-25T10:29:18.785 DEBUG:teuthology.orchestra.run.smithi037:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-04-25T10:29:18.896 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:18.894+0000 7fc53ca8c640 1 Processor -- start 2024-04-25T10:29:18.896 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:18.894+0000 7fc53ca8c640 1 -- start start 2024-04-25T10:29:18.898 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:18.895+0000 7fc53ca8c640 1 --2- >> v2:172.21.15.37:3300/0 conn(0x7fc53806e7b0 0x7fc5380d1ec0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:29:18.898 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:18.895+0000 7fc53ca8c640 1 --2- >> v2:172.21.15.203:3300/0 conn(0x7fc5380d2400 0x7fc5380d2810 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:29:18.898 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:18.895+0000 7fc53ca8c640 1 -- --> v1:172.21.15.37:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fc53809cd40 con 0x7fc53808cc90 2024-04-25T10:29:18.898 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:18.895+0000 7fc53ca8c640 1 -- --> v2:172.21.15.37:3300/0 -- mon_getmap magic: 0 -- 0x7fc5380d3200 con 0x7fc53806e7b0 2024-04-25T10:29:18.898 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:18.895+0000 7fc53ca8c640 1 -- --> v2:172.21.15.203:3300/0 -- mon_getmap magic: 0 -- 0x7fc5380d1100 con 0x7fc5380d2400 2024-04-25T10:29:18.898 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:18.895+0000 7fc53659c640 1 --1- >> v1:172.21.15.37:6789/0 conn(0x7fc53808cc90 0x7fc53806e0a0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.37:6789/0 says I am v1:172.21.15.37:51454/0 (socket says 172.21.15.37:51454) 2024-04-25T10:29:18.898 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:18.895+0000 7fc53659c640 1 -- 172.21.15.37:0/3223363698 learned_addr learned my addr 172.21.15.37:0/3223363698 (peer_addr_for_me v1:172.21.15.37:0/0) 2024-04-25T10:29:18.898 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:18.895+0000 7fc52ffff640 1 --2- 172.21.15.37:0/3223363698 >> v2:172.21.15.37:3300/0 conn(0x7fc53806e7b0 0x7fc5380d1ec0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:29:18.898 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:18.896+0000 7fc52ffff640 1 -- 172.21.15.37:0/3223363698 >> v2:172.21.15.203:3300/0 conn(0x7fc5380d2400 msgr2=0x7fc5380d2810 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:29:18.898 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:18.896+0000 7fc536d9d640 1 --2- 172.21.15.37:0/3223363698 >> v2:172.21.15.203:3300/0 conn(0x7fc5380d2400 0x7fc5380d2810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:29:18.898 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:18.896+0000 7fc52ffff640 1 --2- 172.21.15.37:0/3223363698 >> v2:172.21.15.203:3300/0 conn(0x7fc5380d2400 0x7fc5380d2810 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:29:18.898 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:18.896+0000 7fc52ffff640 1 -- 172.21.15.37:0/3223363698 >> v1:172.21.15.37:6789/0 conn(0x7fc53808cc90 legacy=0x7fc53806e0a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:29:18.899 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:18.896+0000 7fc52ffff640 1 -- 172.21.15.37:0/3223363698 --> v2:172.21.15.37:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fc53809d000 con 0x7fc53806e7b0 2024-04-25T10:29:18.899 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:18.896+0000 7fc535d9b640 1 -- 172.21.15.37:0/3223363698 <== mon.0 v1:172.21.15.37:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 3396501737 0 0) 0x7fc53809cd40 con 0x7fc53808cc90 2024-04-25T10:29:18.900 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:18.896+0000 7fc52ffff640 1 --2- 172.21.15.37:0/3223363698 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fc53806e7b0 0x7fc5380d1ec0 secure :-1 s=READY pgs=98 cs=0 l=1 rev1=1 crypto rx=0x7fc5380d14b0 tx=0x7fc52002a1a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=8b03992b066da191 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:29:18.900 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:18.896+0000 7fc535d9b640 1 -- 172.21.15.37:0/3223363698 <== mon.0 v2:172.21.15.37:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc520037070 con 0x7fc53806e7b0 2024-04-25T10:29:18.900 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:18.896+0000 7fc535d9b640 1 -- 172.21.15.37:0/3223363698 <== mon.0 v2:172.21.15.37:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fc520032440 con 0x7fc53806e7b0 2024-04-25T10:29:18.900 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:18.896+0000 7fc535d9b640 1 -- 172.21.15.37:0/3223363698 <== mon.0 v2:172.21.15.37:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc520032810 con 0x7fc53806e7b0 2024-04-25T10:29:18.900 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:18.897+0000 7fc53ca8c640 1 -- 172.21.15.37:0/3223363698 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fc53806e7b0 msgr2=0x7fc5380d1ec0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:29:18.900 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:18.897+0000 7fc53ca8c640 1 --2- 172.21.15.37:0/3223363698 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fc53806e7b0 0x7fc5380d1ec0 secure :-1 s=READY pgs=98 cs=0 l=1 rev1=1 crypto rx=0x7fc5380d14b0 tx=0x7fc52002a1a0 comp rx=0 tx=0).stop 2024-04-25T10:29:18.900 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:18.897+0000 7fc53ca8c640 1 -- 172.21.15.37:0/3223363698 shutdown_connections 2024-04-25T10:29:18.900 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:18.897+0000 7fc53ca8c640 1 --2- 172.21.15.37:0/3223363698 >> v2:172.21.15.203:3300/0 conn(0x7fc5380d2400 0x7fc5380d2810 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:29:18.900 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:18.897+0000 7fc53ca8c640 1 --2- 172.21.15.37:0/3223363698 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fc53806e7b0 0x7fc5380d1ec0 unknown :-1 s=CLOSED pgs=98 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:29:18.900 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:18.897+0000 7fc53ca8c640 1 -- 172.21.15.37:0/3223363698 >> 172.21.15.37:0/3223363698 conn(0x7fc5380d03d0 msgr2=0x7fc5380d07e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:29:18.901 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:18.897+0000 7fc53ca8c640 1 -- 172.21.15.37:0/3223363698 shutdown_connections 2024-04-25T10:29:18.901 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:18.897+0000 7fc53ca8c640 1 -- 172.21.15.37:0/3223363698 wait complete. 2024-04-25T10:29:18.901 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:18.899+0000 7fc53ca8c640 1 Processor -- start 2024-04-25T10:29:18.901 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:18.899+0000 7fc53ca8c640 1 -- start start 2024-04-25T10:29:18.904 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:18.900+0000 7fc53ca8c640 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fc53806e7b0 0x7fc5380caa00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:29:18.904 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:18.900+0000 7fc53ca8c640 1 --2- >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7fc5380d2400 0x7fc5380b79d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:29:18.904 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:18.900+0000 7fc53ca8c640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fc5380b7f10 0x7fc53814f9a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:29:18.904 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:18.900+0000 7fc53ca8c640 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 -- 0x7fc5380d2d50 con 0x7fc5380b7f10 2024-04-25T10:29:18.904 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:18.900+0000 7fc53ca8c640 1 -- --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_getmap magic: 0 -- 0x7fc5380d3690 con 0x7fc53806e7b0 2024-04-25T10:29:18.904 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:18.900+0000 7fc53ca8c640 1 -- --> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] -- mon_getmap magic: 0 -- 0x7fc5380d2ed0 con 0x7fc5380d2400 2024-04-25T10:29:18.904 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:18.901+0000 7fc536d9d640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fc5380b7f10 0x7fc53814f9a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:29:18.904 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:18.901+0000 7fc536d9d640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fc5380b7f10 0x7fc53814f9a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.37:3300/0 says I am v2:172.21.15.37:46420/0 (socket says 172.21.15.37:46420) 2024-04-25T10:29:18.905 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:18.901+0000 7fc536d9d640 1 -- 172.21.15.37:0/354177788 learned_addr learned my addr 172.21.15.37:0/354177788 (peer_addr_for_me v2:172.21.15.37:0/0) 2024-04-25T10:29:18.905 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:18.901+0000 7fc53659c640 1 --2- 172.21.15.37:0/354177788 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fc53806e7b0 0x7fc5380caa00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:29:18.905 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:18.901+0000 7fc52ffff640 1 --2- 172.21.15.37:0/354177788 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7fc5380d2400 0x7fc5380b79d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:29:18.905 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:18.901+0000 7fc536d9d640 1 -- 172.21.15.37:0/354177788 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7fc5380d2400 msgr2=0x7fc5380b79d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:29:18.905 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:18.901+0000 7fc536d9d640 1 --2- 172.21.15.37:0/354177788 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7fc5380d2400 0x7fc5380b79d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:29:18.905 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:18.901+0000 7fc536d9d640 1 -- 172.21.15.37:0/354177788 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fc53806e7b0 msgr2=0x7fc5380caa00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:29:18.905 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:18.901+0000 7fc536d9d640 1 --2- 172.21.15.37:0/354177788 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fc53806e7b0 0x7fc5380caa00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:29:18.905 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:18.901+0000 7fc536d9d640 1 -- 172.21.15.37:0/354177788 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fc5380cb4c0 con 0x7fc5380b7f10 2024-04-25T10:29:18.905 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:18.901+0000 7fc536d9d640 1 --2- 172.21.15.37:0/354177788 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fc5380b7f10 0x7fc53814f9a0 secure :-1 s=READY pgs=99 cs=0 l=1 rev1=1 crypto rx=0x7fc53000cc80 tx=0x7fc53000a5c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:29:18.905 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:18.901+0000 7fc52effd640 1 -- 172.21.15.37:0/354177788 <== mon.0 v2:172.21.15.37:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc530003990 con 0x7fc5380b7f10 2024-04-25T10:29:18.905 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:18.901+0000 7fc52effd640 1 -- 172.21.15.37:0/354177788 <== mon.0 v2:172.21.15.37:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fc530003b50 con 0x7fc5380b7f10 2024-04-25T10:29:18.905 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:18.901+0000 7fc52effd640 1 -- 172.21.15.37:0/354177788 <== mon.0 v2:172.21.15.37:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc530004020 con 0x7fc5380b7f10 2024-04-25T10:29:18.905 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:18.901+0000 7fc53ca8c640 1 -- 172.21.15.37:0/354177788 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fc5380cb7a0 con 0x7fc5380b7f10 2024-04-25T10:29:18.905 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:18.901+0000 7fc53ca8c640 1 -- 172.21.15.37:0/354177788 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fc5380cbd60 con 0x7fc5380b7f10 2024-04-25T10:29:18.906 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:18.903+0000 7fc53ca8c640 1 -- 172.21.15.37:0/354177788 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fc5380cb230 con 0x7fc5380b7f10 2024-04-25T10:29:18.906 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:18.903+0000 7fc52effd640 1 -- 172.21.15.37:0/354177788 <== mon.0 v2:172.21.15.37:3300/0 4 ==== mgrmap(e 5) ==== 96782+0+0 (secure 0 0 0) 0x7fc5300041c0 con 0x7fc5380b7f10 2024-04-25T10:29:18.907 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:18.904+0000 7fc52effd640 1 --2- 172.21.15.37:0/354177788 >> [v2:172.21.15.203:6832/2418205237,v1:172.21.15.203:6833/2418205237] conn(0x7fc5180744e0 0x7fc5180769f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:29:18.907 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:18.904+0000 7fc52effd640 1 -- 172.21.15.37:0/354177788 <== mon.0 v2:172.21.15.37:3300/0 5 ==== osd_map(23..23 src has 1..23) ==== 5709+0+0 (secure 0 0 0) 0x7fc5300956d0 con 0x7fc5380b7f10 2024-04-25T10:29:18.907 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:18.904+0000 7fc53659c640 1 --2- 172.21.15.37:0/354177788 >> [v2:172.21.15.203:6832/2418205237,v1:172.21.15.203:6833/2418205237] conn(0x7fc5180744e0 0x7fc5180769f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:29:18.907 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:18.904+0000 7fc53659c640 1 --2- 172.21.15.37:0/354177788 >> [v2:172.21.15.203:6832/2418205237,v1:172.21.15.203:6833/2418205237] conn(0x7fc5180744e0 0x7fc5180769f0 secure :-1 s=READY pgs=72 cs=0 l=1 rev1=1 crypto rx=0x7fc528000f60 tx=0x7fc528009040 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:29:18.910 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:18.908+0000 7fc52effd640 1 -- 172.21.15.37:0/354177788 <== mon.0 v2:172.21.15.37:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187246 (secure 0 0 0) 0x7fc530014030 con 0x7fc5380b7f10 2024-04-25T10:29:19.096 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:19.094+0000 7fc53ca8c640 1 -- 172.21.15.37:0/354177788 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7fc5380d2b30 con 0x7fc5380b7f10 2024-04-25T10:29:19.097 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:19.095+0000 7fc52effd640 1 -- 172.21.15.37:0/354177788 <== mon.0 v2:172.21.15.37:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v23) ==== 74+0+16621 (secure 0 0 0) 0x7fc53005fe60 con 0x7fc5380b7f10 2024-04-25T10:29:19.098 INFO:teuthology.orchestra.run.smithi037.stdout: 2024-04-25T10:29:19.098 INFO:teuthology.orchestra.run.smithi037.stdout:{"epoch":23,"fsid":"f4acda28-be34-492a-b286-994afd36f61d","created":"2024-04-25T10:28:43.985718+0000","modified":"2024-04-25T10:29:10.016188+0000","last_up_change":"2024-04-25T10:28:54.900654+0000","last_in_change":"2024-04-25T10:28:46.216973+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":3,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-04-25T10:28:56.202153+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"19","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2024-04-25T10:29:06.507244+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"22","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":2.25,"score_stable":2.25,"optimal_score":1,"raw_score_acting":2.25,"raw_score_stable":2.25,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2024-04-25T10:29:07.347844+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"23","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"a2f79028-b332-4166-9ebb-a7024a37ea22","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":21,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.37:6816","nonce":2746074720},{"type":"v1","addr":"172.21.15.37:6817","nonce":2746074720}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.37:6818","nonce":2746074720},{"type":"v1","addr":"172.21.15.37:6819","nonce":2746074720}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.37:6822","nonce":2746074720},{"type":"v1","addr":"172.21.15.37:6823","nonce":2746074720}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.37:6820","nonce":2746074720},{"type":"v1","addr":"172.21.15.37:6821","nonce":2746074720}]},"public_addr":"172.21.15.37:6817/2746074720","cluster_addr":"172.21.15.37:6819/2746074720","heartbeat_back_addr":"172.21.15.37:6823/2746074720","heartbeat_front_addr":"172.21.15.37:6821/2746074720","state":["exists","up"]},{"osd":1,"uuid":"973159de-c2ea-4083-aecc-69373d1c1132","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":21,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.37:6800","nonce":1690341662},{"type":"v1","addr":"172.21.15.37:6801","nonce":1690341662}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.37:6802","nonce":1690341662},{"type":"v1","addr":"172.21.15.37:6803","nonce":1690341662}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.37:6806","nonce":1690341662},{"type":"v1","addr":"172.21.15.37:6807","nonce":1690341662}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.37:6804","nonce":1690341662},{"type":"v1","addr":"172.21.15.37:6805","nonce":1690341662}]},"public_addr":"172.21.15.37:6801/1690341662","cluster_addr":"172.21.15.37:6803/1690341662","heartbeat_back_addr":"172.21.15.37:6807/1690341662","heartbeat_front_addr":"172.21.15.37:6805/1690341662","state":["exists","up"]},{"osd":2,"uuid":"4ae7a0e9-cb3f-484c-a75f-f3766b721c47","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":21,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.37:6808","nonce":2754669649},{"type":"v1","addr":"172.21.15.37:6809","nonce":2754669649}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.37:6810","nonce":2754669649},{"type":"v1","addr":"172.21.15.37:6811","nonce":2754669649}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.37:6814","nonce":2754669649},{"type":"v1","addr":"172.21.15.37:6815","nonce":2754669649}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.37:6812","nonce":2754669649},{"type":"v1","addr":"172.21.15.37:6813","nonce":2754669649}]},"public_addr":"172.21.15.37:6809/2754669649","cluster_addr":"172.21.15.37:6811/2754669649","heartbeat_back_addr":"172.21.15.37:6815/2754669649","heartbeat_front_addr":"172.21.15.37:6813/2754669649","state":["exists","up"]},{"osd":3,"uuid":"1ba4a746-1608-47a7-ace2-960a5cfae934","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":21,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.37:6824","nonce":1815290618},{"type":"v1","addr":"172.21.15.37:6825","nonce":1815290618}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.37:6826","nonce":1815290618},{"type":"v1","addr":"172.21.15.37:6827","nonce":1815290618}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.37:6830","nonce":1815290618},{"type":"v1","addr":"172.21.15.37:6831","nonce":1815290618}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.37:6828","nonce":1815290618},{"type":"v1","addr":"172.21.15.37:6829","nonce":1815290618}]},"public_addr":"172.21.15.37:6825/1815290618","cluster_addr":"172.21.15.37:6827/1815290618","heartbeat_back_addr":"172.21.15.37:6831/1815290618","heartbeat_front_addr":"172.21.15.37:6829/1815290618","state":["exists","up"]},{"osd":4,"uuid":"d4f59f5d-0413-4b79-87ee-bb212c1fcf62","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":21,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6800","nonce":4051532063},{"type":"v1","addr":"172.21.15.203:6801","nonce":4051532063}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6802","nonce":4051532063},{"type":"v1","addr":"172.21.15.203:6803","nonce":4051532063}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6806","nonce":4051532063},{"type":"v1","addr":"172.21.15.203:6807","nonce":4051532063}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6804","nonce":4051532063},{"type":"v1","addr":"172.21.15.203:6805","nonce":4051532063}]},"public_addr":"172.21.15.203:6801/4051532063","cluster_addr":"172.21.15.203:6803/4051532063","heartbeat_back_addr":"172.21.15.203:6807/4051532063","heartbeat_front_addr":"172.21.15.203:6805/4051532063","state":["exists","up"]},{"osd":5,"uuid":"69710a0e-41d8-4e7f-9a10-17ce540b67be","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":21,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6824","nonce":1203822890},{"type":"v1","addr":"172.21.15.203:6825","nonce":1203822890}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6826","nonce":1203822890},{"type":"v1","addr":"172.21.15.203:6827","nonce":1203822890}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6830","nonce":1203822890},{"type":"v1","addr":"172.21.15.203:6831","nonce":1203822890}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6828","nonce":1203822890},{"type":"v1","addr":"172.21.15.203:6829","nonce":1203822890}]},"public_addr":"172.21.15.203:6825/1203822890","cluster_addr":"172.21.15.203:6827/1203822890","heartbeat_back_addr":"172.21.15.203:6831/1203822890","heartbeat_front_addr":"172.21.15.203:6829/1203822890","state":["exists","up"]},{"osd":6,"uuid":"f29dd047-3b48-473d-a235-6642a6293f20","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":21,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6810","nonce":2309615014},{"type":"v1","addr":"172.21.15.203:6812","nonce":2309615014}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6814","nonce":2309615014},{"type":"v1","addr":"172.21.15.203:6816","nonce":2309615014}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6822","nonce":2309615014},{"type":"v1","addr":"172.21.15.203:6823","nonce":2309615014}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6818","nonce":2309615014},{"type":"v1","addr":"172.21.15.203:6820","nonce":2309615014}]},"public_addr":"172.21.15.203:6812/2309615014","cluster_addr":"172.21.15.203:6816/2309615014","heartbeat_back_addr":"172.21.15.203:6823/2309615014","heartbeat_front_addr":"172.21.15.203:6820/2309615014","state":["exists","up"]},{"osd":7,"uuid":"6e691644-378b-4fdc-8e86-fb5f56e31f29","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":21,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6808","nonce":3688099014},{"type":"v1","addr":"172.21.15.203:6809","nonce":3688099014}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6811","nonce":3688099014},{"type":"v1","addr":"172.21.15.203:6813","nonce":3688099014}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6819","nonce":3688099014},{"type":"v1","addr":"172.21.15.203:6821","nonce":3688099014}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6815","nonce":3688099014},{"type":"v1","addr":"172.21.15.203:6817","nonce":3688099014}]},"public_addr":"172.21.15.203:6809/3688099014","cluster_addr":"172.21.15.203:6813/3688099014","heartbeat_back_addr":"172.21.15.203:6821/3688099014","heartbeat_front_addr":"172.21.15.203:6817/3688099014","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T10:28:51.419141+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T10:28:51.366448+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T10:28:51.348276+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T10:28:51.380519+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T10:28:51.105812+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T10:28:51.384496+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T10:28:51.388755+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T10:28:51.366958+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-04-25T10:29:19.100 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:19.097+0000 7fc53ca8c640 1 -- 172.21.15.37:0/354177788 >> [v2:172.21.15.203:6832/2418205237,v1:172.21.15.203:6833/2418205237] conn(0x7fc5180744e0 msgr2=0x7fc5180769f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:29:19.100 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:19.097+0000 7fc53ca8c640 1 --2- 172.21.15.37:0/354177788 >> [v2:172.21.15.203:6832/2418205237,v1:172.21.15.203:6833/2418205237] conn(0x7fc5180744e0 0x7fc5180769f0 secure :-1 s=READY pgs=72 cs=0 l=1 rev1=1 crypto rx=0x7fc528000f60 tx=0x7fc528009040 comp rx=0 tx=0).stop 2024-04-25T10:29:19.100 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:19.097+0000 7fc53ca8c640 1 -- 172.21.15.37:0/354177788 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fc5380b7f10 msgr2=0x7fc53814f9a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:29:19.100 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:19.097+0000 7fc53ca8c640 1 --2- 172.21.15.37:0/354177788 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fc5380b7f10 0x7fc53814f9a0 secure :-1 s=READY pgs=99 cs=0 l=1 rev1=1 crypto rx=0x7fc53000cc80 tx=0x7fc53000a5c0 comp rx=0 tx=0).stop 2024-04-25T10:29:19.100 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:19.097+0000 7fc53ca8c640 1 -- 172.21.15.37:0/354177788 shutdown_connections 2024-04-25T10:29:19.100 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:19.097+0000 7fc53ca8c640 1 --2- 172.21.15.37:0/354177788 >> [v2:172.21.15.203:6832/2418205237,v1:172.21.15.203:6833/2418205237] conn(0x7fc5180744e0 0x7fc5180769f0 unknown :-1 s=CLOSED pgs=72 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:29:19.100 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:19.097+0000 7fc53ca8c640 1 --2- 172.21.15.37:0/354177788 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fc5380b7f10 0x7fc53814f9a0 unknown :-1 s=CLOSED pgs=99 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:29:19.100 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:19.097+0000 7fc53ca8c640 1 --2- 172.21.15.37:0/354177788 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7fc5380d2400 0x7fc5380b79d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:29:19.100 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:19.097+0000 7fc53ca8c640 1 --2- 172.21.15.37:0/354177788 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fc53806e7b0 0x7fc5380caa00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:29:19.100 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:19.097+0000 7fc53ca8c640 1 -- 172.21.15.37:0/354177788 >> 172.21.15.37:0/354177788 conn(0x7fc5380d03d0 msgr2=0x7fc53806e0a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:29:19.101 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:19.097+0000 7fc53ca8c640 1 -- 172.21.15.37:0/354177788 shutdown_connections 2024-04-25T10:29:19.101 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:19.097+0000 7fc53ca8c640 1 -- 172.21.15.37:0/354177788 wait complete. 2024-04-25T10:29:19.111 DEBUG:teuthology.orchestra.run.smithi037:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.0 flush_pg_stats 2024-04-25T10:29:19.112 DEBUG:teuthology.orchestra.run.smithi037:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.1 flush_pg_stats 2024-04-25T10:29:19.112 DEBUG:teuthology.orchestra.run.smithi037:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.2 flush_pg_stats 2024-04-25T10:29:19.112 DEBUG:teuthology.orchestra.run.smithi037:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.3 flush_pg_stats 2024-04-25T10:29:19.113 DEBUG:teuthology.orchestra.run.smithi037:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.4 flush_pg_stats 2024-04-25T10:29:19.113 DEBUG:teuthology.orchestra.run.smithi037:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.5 flush_pg_stats 2024-04-25T10:29:19.113 DEBUG:teuthology.orchestra.run.smithi037:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.6 flush_pg_stats 2024-04-25T10:29:19.114 DEBUG:teuthology.orchestra.run.smithi037:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.7 flush_pg_stats 2024-04-25T10:29:19.336 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:19.334+0000 7fd915ca4640 1 Processor -- start 2024-04-25T10:29:19.337 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:19.334+0000 7f8cdcd46640 1 Processor -- start 2024-04-25T10:29:19.337 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:19.334+0000 7fd915ca4640 1 -- start start 2024-04-25T10:29:19.337 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:19.334+0000 7f376e50d640 1 Processor -- start 2024-04-25T10:29:19.338 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:19.334+0000 7f67770b1640 1 Processor -- start 2024-04-25T10:29:19.338 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:19.334+0000 7f67770b1640 1 -- start start 2024-04-25T10:29:19.338 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:19.334+0000 7f8cdcd46640 1 -- start start 2024-04-25T10:29:19.338 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:19.335+0000 7f2d0e1ab640 1 Processor -- start 2024-04-25T10:29:19.339 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:19.334+0000 7f376e50d640 1 -- start start 2024-04-25T10:29:19.339 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:19.336+0000 7efdeb334640 1 Processor -- start 2024-04-25T10:29:19.340 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:19.336+0000 7fd915ca4640 1 --2- >> v2:172.21.15.37:3300/0 conn(0x7fd9080340c0 0x7fd908006aa0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:29:19.340 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:19.336+0000 7fd915ca4640 1 --2- >> v2:172.21.15.203:3300/0 conn(0x7fd9080e08b0 0x7fd90804fa00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:29:19.340 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:19.336+0000 7fd915ca4640 1 -- --> v1:172.21.15.203:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fd9080071d0 con 0x7fd9080dfd90 2024-04-25T10:29:19.340 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:19.336+0000 7fd915ca4640 1 -- --> v2:172.21.15.37:3300/0 -- mon_getmap magic: 0 -- 0x7fd9080344d0 con 0x7fd9080340c0 2024-04-25T10:29:19.340 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:19.336+0000 7fd915ca4640 1 -- --> v2:172.21.15.203:3300/0 -- mon_getmap magic: 0 -- 0x7fd9080074b0 con 0x7fd9080e08b0 2024-04-25T10:29:19.340 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:19.336+0000 7f8cdcd46640 1 --2- >> v2:172.21.15.203:3300/0 conn(0x7f8cd0047db0 0x7f8cd00e7d50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:29:19.340 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:19.336+0000 7efdeb334640 1 -- start start 2024-04-25T10:29:19.341 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:19.337+0000 7fd90ffff640 1 --1- >> v1:172.21.15.203:6789/0 conn(0x7fd9080dfd90 0x7fd9080e01a0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.203:6789/0 says I am v1:172.21.15.37:37458/0 (socket says 172.21.15.37:37458) 2024-04-25T10:29:19.341 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:19.337+0000 7fd90ffff640 1 -- 172.21.15.37:0/1879434666 learned_addr learned my addr 172.21.15.37:0/1879434666 (peer_addr_for_me v1:172.21.15.37:0/0) 2024-04-25T10:29:19.341 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:19.337+0000 7fd9154a3640 1 --2- >> v2:172.21.15.203:3300/0 conn(0x7fd9080e08b0 0x7fd90804fa00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:29:19.342 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:19.337+0000 7f8cd77fe640 1 --2- >> v2:172.21.15.203:3300/0 conn(0x7f8cd0047db0 0x7f8cd00e7d50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:29:19.342 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:19.337+0000 7f8cd77fe640 1 --2- >> v2:172.21.15.203:3300/0 conn(0x7f8cd0047db0 0x7f8cd00e7d50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.203:3300/0 says I am v2:172.21.15.37:37974/0 (socket says 172.21.15.37:37974) 2024-04-25T10:29:19.342 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:19.337+0000 7f8cd6ffd640 1 --1- >> v1:172.21.15.37:6789/0 conn(0x7f8cd0048300 0x7f8cd0033950 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.37:6789/0 says I am v1:172.21.15.37:51464/0 (socket says 172.21.15.37:51464) 2024-04-25T10:29:19.342 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:19.337+0000 7f67770b1640 1 --2- >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f67700b0680 0x7f67700d0d90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:29:19.342 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:19.337+0000 7f67770b1640 1 --2- >> v2:172.21.15.203:3300/0 conn(0x7f67700b4590 0x7f677015bd90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:29:19.343 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:19.337+0000 7f1f1f1e3640 1 Processor -- start 2024-04-25T10:29:19.343 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:19.337+0000 7f8cd77fe640 1 -- 172.21.15.37:0/371701842 learned_addr learned my addr 172.21.15.37:0/371701842 (peer_addr_for_me v2:172.21.15.37:0/0) 2024-04-25T10:29:19.343 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:19.337+0000 7f8cdcd46640 1 --2- 172.21.15.37:0/371701842 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f8cd0034060 0x7f8cd00065e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:29:19.344 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:19.337+0000 7f6774e4d640 1 --2- >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f67700b0680 0x7f67700d0d90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:29:19.344 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:19.337+0000 7f6774e4d640 1 --2- >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f67700b0680 0x7f67700d0d90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.203:3301/0 says I am v2:172.21.15.37:51558/0 (socket says 172.21.15.37:51558) 2024-04-25T10:29:19.344 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:19.337+0000 7f6774e4d640 1 -- 172.21.15.37:0/1477415135 learned_addr learned my addr 172.21.15.37:0/1477415135 (peer_addr_for_me v2:172.21.15.37:0/0) 2024-04-25T10:29:19.344 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:19.337+0000 7f8cdcd46640 1 -- 172.21.15.37:0/371701842 --> v1:172.21.15.37:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f8cd0006130 con 0x7f8cd0048300 2024-04-25T10:29:19.344 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:19.337+0000 7f67770b1640 1 -- 172.21.15.37:0/1477415135 --> v1:172.21.15.203:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f67700b5b70 con 0x7f6770072eb0 2024-04-25T10:29:19.345 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:19.337+0000 7fd90f7fe640 1 -- 172.21.15.37:0/1879434666 <== mon.1 v1:172.21.15.203:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 1226041549 0 0) 0x7fd9080071d0 con 0x7fd9080dfd90 2024-04-25T10:29:19.345 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:19.337+0000 7fd90f7fe640 1 -- 172.21.15.37:0/1879434666 --> v1:172.21.15.203:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7fd900003520 con 0x7fd9080dfd90 2024-04-25T10:29:19.345 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:19.337+0000 7fd90f7fe640 1 -- 172.21.15.37:0/1879434666 <== mon.1 v1:172.21.15.203:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 1561000162 0 0) 0x7fd900003520 con 0x7fd9080dfd90 2024-04-25T10:29:19.345 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:19.338+0000 7fd90f7fe640 1 -- 172.21.15.37:0/1879434666 >> v2:172.21.15.203:3300/0 conn(0x7fd9080e08b0 msgr2=0x7fd90804fa00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:29:19.345 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:19.338+0000 7fd90f7fe640 1 --2- 172.21.15.37:0/1879434666 >> v2:172.21.15.203:3300/0 conn(0x7fd9080e08b0 0x7fd90804fa00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:29:19.345 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:19.338+0000 7fd90f7fe640 1 -- 172.21.15.37:0/1879434666 >> v2:172.21.15.37:3300/0 conn(0x7fd9080340c0 msgr2=0x7fd908006aa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:29:19.346 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:19.338+0000 7f67770b1640 1 -- 172.21.15.37:0/1477415135 --> v2:172.21.15.203:3300/0 -- mon_getmap magic: 0 -- 0x7f67700d12d0 con 0x7f67700b4590 2024-04-25T10:29:19.346 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:19.338+0000 7f67770b1640 1 -- 172.21.15.37:0/1477415135 --> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] -- mon_getmap magic: 0 -- 0x7f67700b5e50 con 0x7f67700b0680 2024-04-25T10:29:19.346 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:19.338+0000 7fd90f7fe640 1 --2- 172.21.15.37:0/1879434666 >> v2:172.21.15.37:3300/0 conn(0x7fd9080340c0 0x7fd908006aa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:29:19.346 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:19.338+0000 7fd90f7fe640 1 -- 172.21.15.37:0/1879434666 --> v1:172.21.15.203:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fd908007790 con 0x7fd9080dfd90 2024-04-25T10:29:19.346 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:19.338+0000 7f8cd67fc640 1 -- 172.21.15.37:0/371701842 <== mon.0 v1:172.21.15.37:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 3904384516 0 0) 0x7f8cd0006130 con 0x7f8cd0048300 2024-04-25T10:29:19.347 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:19.338+0000 7f8cd7fff640 1 --2- 172.21.15.37:0/371701842 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f8cd0034060 0x7f8cd00065e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:29:19.347 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:19.335+0000 7f2d0e1ab640 1 -- start start 2024-04-25T10:29:19.347 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:19.338+0000 7fd90f7fe640 1 -- 172.21.15.37:0/1879434666 <== mon.1 v1:172.21.15.203:6789/0 3 ==== mon_map magic: 0 ==== 413+0+0 (unknown 1577794600 0 0) 0x7fd8fc000ed0 con 0x7fd9080dfd90 2024-04-25T10:29:19.347 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:19.338+0000 7f8cdcd46640 1 -- 172.21.15.37:0/371701842 --> v2:172.21.15.203:3300/0 -- mon_getmap magic: 0 -- 0x7f8cd004f690 con 0x7f8cd0047db0 2024-04-25T10:29:19.347 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:19.338+0000 7f8cdcd46640 1 -- 172.21.15.37:0/371701842 --> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] -- mon_getmap magic: 0 -- 0x7f8cd004ffa0 con 0x7f8cd0034060 2024-04-25T10:29:19.348 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:19.338+0000 7f2d0e1ab640 1 --2- >> v2:172.21.15.37:3300/0 conn(0x7f2d00051810 0x7f2d00035070 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:29:19.348 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:19.338+0000 7f2d0e1ab640 1 -- --> v1:172.21.15.37:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f2d00054c80 con 0x7f2d000512d0 2024-04-25T10:29:19.348 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:19.338+0000 7f2d0e1ab640 1 -- --> v1:172.21.15.203:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f2d00054f60 con 0x7f2d00054780 2024-04-25T10:29:19.348 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:19.338+0000 7f2d0e1ab640 1 -- --> v2:172.21.15.37:3300/0 -- mon_getmap magic: 0 -- 0x7f2d00055890 con 0x7f2d00051810 2024-04-25T10:29:19.348 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:19.338+0000 7fd90f7fe640 1 -- 172.21.15.37:0/1879434666 >> v1:172.21.15.203:6789/0 conn(0x7fd9080dfd90 legacy=0x7fd9080e01a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:29:19.349 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:19.338+0000 7f676f7fe640 1 -- 172.21.15.37:0/1477415135 <== mon.1 v1:172.21.15.203:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 3501717184 0 0) 0x7f67700b5b70 con 0x7f6770072eb0 2024-04-25T10:29:19.349 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:19.338+0000 7f676f7fe640 1 -- 172.21.15.37:0/1477415135 --> v1:172.21.15.203:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f6760003520 con 0x7f6770072eb0 2024-04-25T10:29:19.349 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:19.338+0000 7fd90f7fe640 1 --2- 172.21.15.37:0/1879434666 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fd9000039b0 0x7fd900003e20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:29:19.349 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:19.338+0000 7fd90f7fe640 1 --2- 172.21.15.37:0/1879434666 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7fd900004800 0x7fd900004c10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:29:19.349 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:19.338+0000 7fd90f7fe640 1 --2- 172.21.15.37:0/1879434666 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fd900005330 0x7fd90000dbf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:29:19.349 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:19.338+0000 7fd90f7fe640 1 -- 172.21.15.37:0/1879434666 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 -- 0x7fd9080344d0 con 0x7fd900005330 2024-04-25T10:29:19.350 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:19.338+0000 7fd90f7fe640 1 -- 172.21.15.37:0/1879434666 --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_getmap magic: 0 -- 0x7fd9080074b0 con 0x7fd9000039b0 2024-04-25T10:29:19.350 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:19.338+0000 7fd90f7fe640 1 -- 172.21.15.37:0/1879434666 --> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] -- mon_getmap magic: 0 -- 0x7fd90000e550 con 0x7fd900004800 2024-04-25T10:29:19.350 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:19.338+0000 7f1f1f1e3640 1 -- start start 2024-04-25T10:29:19.350 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:19.338+0000 7f2d06ffd640 1 --1- >> v1:172.21.15.37:6789/0 conn(0x7f2d000512d0 0x7f2d000dfa60 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.37:6789/0 says I am v1:172.21.15.37:51478/0 (socket says 172.21.15.37:51478) 2024-04-25T10:29:19.350 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:19.338+0000 7f2d06ffd640 1 -- 172.21.15.37:0/1513236428 learned_addr learned my addr 172.21.15.37:0/1513236428 (peer_addr_for_me v1:172.21.15.37:0/0) 2024-04-25T10:29:19.351 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:19.338+0000 7f8cd67fc640 1 -- 172.21.15.37:0/371701842 --> v1:172.21.15.37:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f8cc8003520 con 0x7f8cd0048300 2024-04-25T10:29:19.351 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:19.338+0000 7f2d07fff640 1 --2- 172.21.15.37:0/1513236428 >> v2:172.21.15.37:3300/0 conn(0x7f2d00051810 0x7f2d00035070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:29:19.351 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:19.338+0000 7f676ffff640 1 --2- 172.21.15.37:0/1477415135 >> v2:172.21.15.203:3300/0 conn(0x7f67700b4590 0x7f677015bd90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:29:19.351 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:19.338+0000 7efdeb334640 1 --2- >> v2:172.21.15.203:3300/0 conn(0x7efddc0547b0 0x7efddc006d50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:29:19.352 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:19.338+0000 7f8cd77fe640 1 -- 172.21.15.37:0/371701842 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f8cd0034060 msgr2=0x7f8cd00065e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:29:19.352 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:19.338+0000 7efdeb334640 1 --2- >> v2:172.21.15.37:3300/0 conn(0x7efddc007290 0x7efddc03d410 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:29:19.352 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:19.338+0000 7efde90d0640 1 --1- >> v1:172.21.15.203:6789/0 conn(0x7efddc0e3890 0x7efddc0540a0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.203:6789/0 says I am v1:172.21.15.37:37484/0 (socket says 172.21.15.37:37484) 2024-04-25T10:29:19.353 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:19.338+0000 7fd90ffff640 1 --2- 172.21.15.37:0/1879434666 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7fd900004800 0x7fd900004c10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:29:19.353 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:19.339+0000 7fd90ffff640 1 -- 172.21.15.37:0/1879434666 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fd9000039b0 msgr2=0x7fd900003e20 unknown :-1 s=STATE_CONNECTING_RE l=0).mark_down 2024-04-25T10:29:19.353 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:19.339+0000 7fd90ffff640 1 --2- 172.21.15.37:0/1879434666 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fd9000039b0 0x7fd900003e20 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:29:19.353 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:19.339+0000 7fd90ffff640 1 -- 172.21.15.37:0/1879434666 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fd900005330 msgr2=0x7fd90000dbf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:29:19.353 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:19.339+0000 7fd90ffff640 1 --2- 172.21.15.37:0/1879434666 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fd900005330 0x7fd90000dbf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:29:19.353 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:19.339+0000 7fd90ffff640 1 -- 172.21.15.37:0/1879434666 --> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fd908007790 con 0x7fd900004800 2024-04-25T10:29:19.353 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:19.340+0000 7fd9154a3640 1 -- 172.21.15.37:0/1879434666 reap_dead start 2024-04-25T10:29:19.354 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:19.337+0000 7f376e50d640 1 --2- >> v2:172.21.15.203:3300/0 conn(0x7f37680b0d00 0x7f3768164960 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:29:19.354 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:19.337+0000 7f376e50d640 1 -- --> v1:172.21.15.37:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f37681652b0 con 0x7f3768164ea0 2024-04-25T10:29:19.354 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:19.337+0000 7f376e50d640 1 -- --> v1:172.21.15.203:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f3768165540 con 0x7f37680b50e0 2024-04-25T10:29:19.354 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:19.337+0000 7f376e50d640 1 -- --> v2:172.21.15.203:3300/0 -- mon_getmap magic: 0 -- 0x7f3768165820 con 0x7f37680b0d00 2024-04-25T10:29:19.354 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:19.338+0000 7f3767fff640 1 --1- >> v1:172.21.15.203:6789/0 conn(0x7f37680b50e0 0x7f37680b05f0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.203:6789/0 says I am v1:172.21.15.37:37464/0 (socket says 172.21.15.37:37464) 2024-04-25T10:29:19.354 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:19.338+0000 7f3767fff640 1 -- 172.21.15.37:0/3137835675 learned_addr learned my addr 172.21.15.37:0/3137835675 (peer_addr_for_me v1:172.21.15.37:0/0) 2024-04-25T10:29:19.354 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:19.338+0000 7f3766ffd640 1 -- 172.21.15.37:0/3137835675 <== mon.1 v1:172.21.15.203:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 3259522633 0 0) 0x7f3768165540 con 0x7f37680b50e0 2024-04-25T10:29:19.354 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:19.338+0000 7f3766ffd640 1 -- 172.21.15.37:0/3137835675 --> v1:172.21.15.203:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f3750003520 con 0x7f37680b50e0 2024-04-25T10:29:19.354 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:19.339+0000 7f3766ffd640 1 -- 172.21.15.37:0/3137835675 <== mon.1 v1:172.21.15.203:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 3783093397 0 0) 0x7f3750003520 con 0x7f37680b50e0 2024-04-25T10:29:19.354 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:19.339+0000 7f3766ffd640 1 -- 172.21.15.37:0/3137835675 >> v2:172.21.15.203:3300/0 conn(0x7f37680b0d00 msgr2=0x7f3768164960 unknown :-1 s=STATE_CONNECTING_RE l=0).mark_down 2024-04-25T10:29:19.354 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:19.339+0000 7f3766ffd640 1 --2- 172.21.15.37:0/3137835675 >> v2:172.21.15.203:3300/0 conn(0x7f37680b0d00 0x7f3768164960 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:29:19.354 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:19.339+0000 7f3766ffd640 1 -- 172.21.15.37:0/3137835675 >> v1:172.21.15.37:6789/0 conn(0x7f3768164ea0 legacy=0x7f376806ef50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:29:19.354 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:19.339+0000 7f3766ffd640 1 -- 172.21.15.37:0/3137835675 --> v1:172.21.15.203:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f37680b4600 con 0x7f37680b50e0 2024-04-25T10:29:19.355 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:19.339+0000 7f3766ffd640 1 -- 172.21.15.37:0/3137835675 <== mon.1 v1:172.21.15.203:6789/0 3 ==== mon_map magic: 0 ==== 413+0+0 (unknown 1577794600 0 0) 0x7f3758001670 con 0x7f37680b50e0 2024-04-25T10:29:19.355 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:19.339+0000 7f3766ffd640 1 -- 172.21.15.37:0/3137835675 >> v1:172.21.15.203:6789/0 conn(0x7f37680b50e0 legacy=0x7f37680b05f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:29:19.355 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:19.339+0000 7f3766ffd640 1 --2- 172.21.15.37:0/3137835675 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f37500039b0 0x7f3750003e00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:29:19.355 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:19.339+0000 7f3766ffd640 1 --2- 172.21.15.37:0/3137835675 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f3750004800 0x7f3750004c10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:29:19.355 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:19.339+0000 7f3766ffd640 1 --2- 172.21.15.37:0/3137835675 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f37500053f0 0x7f375000ba80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:29:19.355 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:19.339+0000 7f3766ffd640 1 -- 172.21.15.37:0/3137835675 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 -- 0x7f3768165820 con 0x7f37500039b0 2024-04-25T10:29:19.355 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:19.339+0000 7f3766ffd640 1 -- 172.21.15.37:0/3137835675 --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_getmap magic: 0 -- 0x7f375000c360 con 0x7f3750004800 2024-04-25T10:29:19.355 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:19.339+0000 7f3766ffd640 1 -- 172.21.15.37:0/3137835675 --> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] -- mon_getmap magic: 0 -- 0x7f375000c560 con 0x7f37500053f0 2024-04-25T10:29:19.355 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:19.339+0000 7f676f7fe640 1 -- 172.21.15.37:0/1477415135 <== mon.1 v1:172.21.15.203:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 1382060330 0 0) 0x7f6760003520 con 0x7f6770072eb0 2024-04-25T10:29:19.355 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:19.339+0000 7f676f7fe640 1 -- 172.21.15.37:0/1477415135 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f67700b0680 msgr2=0x7f67700d0d90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:29:19.356 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:19.339+0000 7f676f7fe640 1 --2- 172.21.15.37:0/1477415135 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f67700b0680 0x7f67700d0d90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:29:19.356 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:19.339+0000 7f676f7fe640 1 -- 172.21.15.37:0/1477415135 >> v2:172.21.15.203:3300/0 conn(0x7f67700b4590 msgr2=0x7f677015bd90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:29:19.356 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:19.339+0000 7f676f7fe640 1 --2- 172.21.15.37:0/1477415135 >> v2:172.21.15.203:3300/0 conn(0x7f67700b4590 0x7f677015bd90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:29:19.356 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:19.339+0000 7f676f7fe640 1 -- 172.21.15.37:0/1477415135 --> v1:172.21.15.203:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f67700b6130 con 0x7f6770072eb0 2024-04-25T10:29:19.356 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:19.339+0000 7f676f7fe640 1 -- 172.21.15.37:0/1477415135 <== mon.1 v1:172.21.15.203:6789/0 3 ==== mon_map magic: 0 ==== 413+0+0 (unknown 1577794600 0 0) 0x7f6768001a20 con 0x7f6770072eb0 2024-04-25T10:29:19.356 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:19.339+0000 7f676f7fe640 1 -- 172.21.15.37:0/1477415135 >> v1:172.21.15.203:6789/0 conn(0x7f6770072eb0 legacy=0x7f67700732c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:29:19.356 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:19.339+0000 7f676f7fe640 1 --2- 172.21.15.37:0/1477415135 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f67700b0680 0x7f67600039b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:29:19.356 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:19.339+0000 7f676f7fe640 1 --2- 172.21.15.37:0/1477415135 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f6760003f10 0x7f6760004800 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:29:19.356 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:19.339+0000 7f676f7fe640 1 --2- 172.21.15.37:0/1477415135 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f6760004d40 0x7f6760005150 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:29:19.356 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:19.339+0000 7f6774e4d640 1 --2- 172.21.15.37:0/1477415135 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f67700b0680 0x7f67600039b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:29:19.356 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:19.340+0000 7f676f7fe640 1 -- 172.21.15.37:0/1477415135 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 -- 0x7f67700d12d0 con 0x7f6760003f10 2024-04-25T10:29:19.356 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:19.340+0000 7f676f7fe640 1 -- 172.21.15.37:0/1477415135 --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_getmap magic: 0 -- 0x7f67700b5e50 con 0x7f6760004d40 2024-04-25T10:29:19.356 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:19.340+0000 7f676f7fe640 1 -- 172.21.15.37:0/1477415135 --> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] -- mon_getmap magic: 0 -- 0x7f676000bc30 con 0x7f67700b0680 2024-04-25T10:29:19.356 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:19.340+0000 7f677564e640 1 --2- 172.21.15.37:0/1477415135 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f6760004d40 0x7f6760005150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:29:19.357 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:19.338+0000 7f8cd77fe640 1 --2- 172.21.15.37:0/371701842 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f8cd0034060 0x7f8cd00065e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:29:19.357 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:19.338+0000 7f8cd77fe640 1 -- 172.21.15.37:0/371701842 >> v1:172.21.15.37:6789/0 conn(0x7f8cd0048300 legacy=0x7f8cd0033950 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:29:19.357 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:19.339+0000 7f8cd77fe640 1 -- 172.21.15.37:0/371701842 --> v2:172.21.15.203:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f8cd0050280 con 0x7f8cd0047db0 2024-04-25T10:29:19.357 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:19.339+0000 7f2d07fff640 1 -- 172.21.15.37:0/1513236428 >> v1:172.21.15.203:6789/0 conn(0x7f2d00054780 legacy=0x7f2d00050bc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:29:19.357 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:19.339+0000 7f2d07fff640 1 -- 172.21.15.37:0/1513236428 >> v1:172.21.15.37:6789/0 conn(0x7f2d000512d0 legacy=0x7f2d000dfa60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:29:19.357 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:19.339+0000 7f2d07fff640 1 -- 172.21.15.37:0/1513236428 --> v2:172.21.15.37:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f2d00055240 con 0x7f2d00051810 2024-04-25T10:29:19.358 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:19.340+0000 7f2d07fff640 1 --2- 172.21.15.37:0/1513236428 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f2d00051810 0x7f2d00035070 secure :-1 s=READY pgs=103 cs=0 l=1 rev1=1 crypto rx=0x7f2d08052040 tx=0x7f2d080a9440 comp rx=0 tx=0).ready entity=mon.0 client_cookie=bf7a6530fb2c24c3 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:29:19.358 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:19.339+0000 7efde90d0640 1 -- 172.21.15.37:0/3291425638 learned_addr learned my addr 172.21.15.37:0/3291425638 (peer_addr_for_me v1:172.21.15.37:0/0) 2024-04-25T10:29:19.358 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:19.339+0000 7efde88cf640 1 --2- 172.21.15.37:0/3291425638 >> v2:172.21.15.203:3300/0 conn(0x7efddc0547b0 0x7efddc006d50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:29:19.358 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:19.339+0000 7efdeb334640 1 -- 172.21.15.37:0/3291425638 --> v1:172.21.15.203:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7efddc0e3ca0 con 0x7efddc0e3890 2024-04-25T10:29:19.358 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:19.339+0000 7efdeb334640 1 -- 172.21.15.37:0/3291425638 --> v2:172.21.15.37:3300/0 -- mon_getmap magic: 0 -- 0x7efddc007990 con 0x7efddc007290 2024-04-25T10:29:19.358 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:19.339+0000 7efdeb334640 1 -- 172.21.15.37:0/3291425638 --> v2:172.21.15.203:3300/0 -- mon_getmap magic: 0 -- 0x7efddc007c70 con 0x7efddc0547b0 2024-04-25T10:29:19.358 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:19.339+0000 7efddbfff640 1 -- 172.21.15.37:0/3291425638 <== mon.1 v1:172.21.15.203:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 2520389782 0 0) 0x7efddc0e3ca0 con 0x7efddc0e3890 2024-04-25T10:29:19.358 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:19.339+0000 7efddbfff640 1 -- 172.21.15.37:0/3291425638 --> v1:172.21.15.203:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7efdd0003520 con 0x7efddc0e3890 2024-04-25T10:29:19.358 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:19.339+0000 7efddbfff640 1 -- 172.21.15.37:0/3291425638 <== mon.1 v1:172.21.15.203:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 3535639283 0 0) 0x7efdd0003520 con 0x7efddc0e3890 2024-04-25T10:29:19.358 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:19.339+0000 7efddbfff640 1 -- 172.21.15.37:0/3291425638 >> v2:172.21.15.203:3300/0 conn(0x7efddc0547b0 msgr2=0x7efddc006d50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:29:19.358 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:19.339+0000 7efde98d1640 1 --2- 172.21.15.37:0/3291425638 >> v2:172.21.15.37:3300/0 conn(0x7efddc007290 0x7efddc03d410 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:29:19.359 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:19.339+0000 7efddbfff640 1 --2- 172.21.15.37:0/3291425638 >> v2:172.21.15.203:3300/0 conn(0x7efddc0547b0 0x7efddc006d50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:29:19.359 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:19.340+0000 7efddbfff640 1 -- 172.21.15.37:0/3291425638 >> v2:172.21.15.37:3300/0 conn(0x7efddc007290 msgr2=0x7efddc03d410 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:29:19.359 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:19.340+0000 7efddbfff640 1 --2- 172.21.15.37:0/3291425638 >> v2:172.21.15.37:3300/0 conn(0x7efddc007290 0x7efddc03d410 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:29:19.359 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:19.340+0000 7efddbfff640 1 -- 172.21.15.37:0/3291425638 --> v1:172.21.15.203:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7efddc03c950 con 0x7efddc0e3890 2024-04-25T10:29:19.359 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:19.340+0000 7efddbfff640 1 -- 172.21.15.37:0/3291425638 <== mon.1 v1:172.21.15.203:6789/0 3 ==== mon_map magic: 0 ==== 413+0+0 (unknown 1577794600 0 0) 0x7efde0001690 con 0x7efddc0e3890 2024-04-25T10:29:19.359 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:19.340+0000 7efddbfff640 1 -- 172.21.15.37:0/3291425638 >> v1:172.21.15.203:6789/0 conn(0x7efddc0e3890 legacy=0x7efddc0540a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:29:19.359 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:19.340+0000 7efddbfff640 1 --2- 172.21.15.37:0/3291425638 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7efdd0003a00 0x7efdd0003e70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:29:19.359 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:19.340+0000 7efddbfff640 1 --2- 172.21.15.37:0/3291425638 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7efdd0004800 0x7efdd0004c10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:29:19.359 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:19.339+0000 7f9563fff640 1 Processor -- start 2024-04-25T10:29:19.360 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:19.340+0000 7f9563fff640 1 -- start start 2024-04-25T10:29:19.360 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:19.340+0000 7f8cd77fe640 1 --2- 172.21.15.37:0/371701842 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f8cd0047db0 0x7f8cd00e7d50 secure :-1 s=READY pgs=69 cs=0 l=1 rev1=1 crypto rx=0x7f8ccc009870 tx=0x7f8ccc02a270 comp rx=0 tx=0).ready entity=mon.1 client_cookie=cf403a90c68c85fe server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:29:19.360 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:19.340+0000 7efddbfff640 1 --2- 172.21.15.37:0/3291425638 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7efdd0005270 0x7efdd000b9c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:29:19.361 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:19.340+0000 7efddbfff640 1 -- 172.21.15.37:0/3291425638 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 -- 0x7efddc007990 con 0x7efdd0004800 2024-04-25T10:29:19.361 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:19.340+0000 7efddbfff640 1 -- 172.21.15.37:0/3291425638 --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_getmap magic: 0 -- 0x7efddc007c70 con 0x7efdd0003a00 2024-04-25T10:29:19.361 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:19.340+0000 7efddbfff640 1 -- 172.21.15.37:0/3291425638 --> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] -- mon_getmap magic: 0 -- 0x7efdd000c210 con 0x7efdd0005270 2024-04-25T10:29:19.361 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:19.340+0000 7efde88cf640 1 --2- 172.21.15.37:0/3291425638 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7efdd0004800 0x7efdd0004c10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:29:19.361 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:19.340+0000 7efde90d0640 1 --2- 172.21.15.37:0/3291425638 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7efdd0003a00 0x7efdd0003e70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:29:19.361 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:19.340+0000 7f676ffff640 1 --2- 172.21.15.37:0/1477415135 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f6760003f10 0x7f6760004800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:29:19.361 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:19.340+0000 7fd90ffff640 1 --2- 172.21.15.37:0/1879434666 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7fd900004800 0x7fd900004c10 secure :-1 s=READY pgs=29 cs=0 l=1 rev1=1 crypto rx=0x7fd8fc001eb0 tx=0x7fd900005830 comp rx=0 tx=0).ready entity=mon.2 client_cookie=aea2d2f94d0968f2 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:29:19.362 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:19.340+0000 7f3767fff640 1 --2- 172.21.15.37:0/3137835675 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f37500039b0 0x7f3750003e00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:29:19.362 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:19.340+0000 7f376caaa640 1 --2- 172.21.15.37:0/3137835675 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f37500053f0 0x7f375000ba80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:29:19.362 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:19.340+0000 7f37677fe640 1 --2- 172.21.15.37:0/3137835675 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f3750004800 0x7f3750004c10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:29:19.362 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:19.341+0000 7f1f1f1e3640 1 --2- >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f1f100522b0 0x7f1f10009410 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:29:19.362 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:19.341+0000 7f1f1f1e3640 1 --2- >> v2:172.21.15.37:3300/0 conn(0x7f1f100542c0 0x7f1f100546d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:29:19.363 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:19.341+0000 7fd90f7fe640 1 -- 172.21.15.37:0/1879434666 <== mon.2 v2:172.21.15.203:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd8fc001b80 con 0x7fd900004800 2024-04-25T10:29:19.363 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:19.341+0000 7fd90f7fe640 1 -- 172.21.15.37:0/1879434666 <== mon.2 v2:172.21.15.203:3301/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fd8fc02f070 con 0x7fd900004800 2024-04-25T10:29:19.363 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:19.341+0000 7fd90f7fe640 1 -- 172.21.15.37:0/1879434666 <== mon.2 v2:172.21.15.203:3301/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd8fc02bbd0 con 0x7fd900004800 2024-04-25T10:29:19.363 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:19.341+0000 7f1f1d780640 1 --2- >> v2:172.21.15.37:3300/0 conn(0x7f1f100542c0 0x7f1f100546d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:29:19.363 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:19.341+0000 7f1f1d780640 1 --2- >> v2:172.21.15.37:3300/0 conn(0x7f1f100542c0 0x7f1f100546d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.37:3300/0 says I am v2:172.21.15.37:46498/0 (socket says 172.21.15.37:46498) 2024-04-25T10:29:19.364 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:19.341+0000 7f1f1d780640 1 -- 172.21.15.37:0/2161163013 learned_addr learned my addr 172.21.15.37:0/2161163013 (peer_addr_for_me v2:172.21.15.37:0/0) 2024-04-25T10:29:19.364 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:19.341+0000 7f376caaa640 1 -- 172.21.15.37:0/3137835675 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f3750004800 msgr2=0x7f3750004c10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:29:19.364 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:19.341+0000 7f376caaa640 1 --2- 172.21.15.37:0/3137835675 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f3750004800 0x7f3750004c10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:29:19.364 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:19.341+0000 7f376caaa640 1 -- 172.21.15.37:0/3137835675 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f37500039b0 msgr2=0x7f3750003e00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:29:19.364 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:19.341+0000 7f376caaa640 1 --2- 172.21.15.37:0/3137835675 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f37500039b0 0x7f3750003e00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:29:19.364 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:19.341+0000 7f376caaa640 1 -- 172.21.15.37:0/3137835675 --> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f37680b4600 con 0x7f37500053f0 2024-04-25T10:29:19.364 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:19.341+0000 7f676ffff640 1 -- 172.21.15.37:0/1477415135 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f67700b0680 msgr2=0x7f67600039b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:29:19.364 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:19.341+0000 7f676ffff640 1 --2- 172.21.15.37:0/1477415135 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f67700b0680 0x7f67600039b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:29:19.365 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:19.341+0000 7f676ffff640 1 -- 172.21.15.37:0/1477415135 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f6760004d40 msgr2=0x7f6760005150 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:29:19.365 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:19.341+0000 7f676ffff640 1 --2- 172.21.15.37:0/1477415135 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f6760004d40 0x7f6760005150 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:29:19.365 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:19.341+0000 7f676ffff640 1 -- 172.21.15.37:0/1477415135 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f6760003520 con 0x7f6760003f10 2024-04-25T10:29:19.365 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:19.341+0000 7f1f1f1e3640 1 -- 172.21.15.37:0/2161163013 --> v1:172.21.15.37:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f1f10054de0 con 0x7f1f10009000 2024-04-25T10:29:19.365 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:19.341+0000 7f1f1f1e3640 1 -- 172.21.15.37:0/2161163013 --> v2:172.21.15.37:3300/0 -- mon_getmap magic: 0 -- 0x7f1f100550c0 con 0x7f1f100542c0 2024-04-25T10:29:19.365 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:19.341+0000 7f1f1f1e3640 1 -- 172.21.15.37:0/2161163013 --> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] -- mon_getmap magic: 0 -- 0x7f1f100553a0 con 0x7f1f100522b0 2024-04-25T10:29:19.366 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:19.340+0000 7f8cd67fc640 1 -- 172.21.15.37:0/371701842 <== mon.1 v2:172.21.15.203:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8ccc037070 con 0x7f8cd0047db0 2024-04-25T10:29:19.366 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:19.340+0000 7f8cd67fc640 1 -- 172.21.15.37:0/371701842 <== mon.1 v2:172.21.15.203:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f8ccc032440 con 0x7f8cd0047db0 2024-04-25T10:29:19.366 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:19.340+0000 7f8cd67fc640 1 -- 172.21.15.37:0/371701842 <== mon.1 v2:172.21.15.203:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8ccc032810 con 0x7f8cd0047db0 2024-04-25T10:29:19.366 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:19.341+0000 7f8cdcd46640 1 -- 172.21.15.37:0/371701842 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f8cd0047db0 msgr2=0x7f8cd00e7d50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:29:19.366 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:19.341+0000 7f8cdcd46640 1 --2- 172.21.15.37:0/371701842 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f8cd0047db0 0x7f8cd00e7d50 secure :-1 s=READY pgs=69 cs=0 l=1 rev1=1 crypto rx=0x7f8ccc009870 tx=0x7f8ccc02a270 comp rx=0 tx=0).stop 2024-04-25T10:29:19.366 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:19.340+0000 7efde98d1640 1 --2- 172.21.15.37:0/3291425638 >> v2:172.21.15.37:3300/0 conn(0x7efddc007290 0x7efddc03d410 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T10:29:19.366 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:19.340+0000 7efde88cf640 1 -- 172.21.15.37:0/3291425638 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7efdd0005270 msgr2=0x7efdd000b9c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:29:19.366 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:19.341+0000 7efde88cf640 1 --2- 172.21.15.37:0/3291425638 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7efdd0005270 0x7efdd000b9c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:29:19.366 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:19.341+0000 7efde88cf640 1 -- 172.21.15.37:0/3291425638 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7efdd0003a00 msgr2=0x7efdd0003e70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:29:19.366 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:19.341+0000 7efde88cf640 1 --2- 172.21.15.37:0/3291425638 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7efdd0003a00 0x7efdd0003e70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:29:19.367 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:19.341+0000 7efde88cf640 1 -- 172.21.15.37:0/3291425638 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7efdd0003520 con 0x7efdd0004800 2024-04-25T10:29:19.367 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:19.341+0000 7efde98d1640 1 -- 172.21.15.37:0/3291425638 reap_dead start 2024-04-25T10:29:19.367 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:19.342+0000 7f1f1d780640 1 -- 172.21.15.37:0/2161163013 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f1f100522b0 msgr2=0x7f1f10009410 unknown :-1 s=STATE_CONNECTING_RE l=0).mark_down 2024-04-25T10:29:19.367 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:19.342+0000 7f1f1d780640 1 --2- 172.21.15.37:0/2161163013 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f1f100522b0 0x7f1f10009410 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:29:19.367 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:19.340+0000 7f2d067fc640 1 -- 172.21.15.37:0/1513236428 <== mon.0 v2:172.21.15.37:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2d080b5070 con 0x7f2d00051810 2024-04-25T10:29:19.367 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:19.340+0000 7f2d067fc640 1 -- 172.21.15.37:0/1513236428 <== mon.0 v2:172.21.15.37:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f2d080b0600 con 0x7f2d00051810 2024-04-25T10:29:19.367 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:19.340+0000 7f2d067fc640 1 -- 172.21.15.37:0/1513236428 <== mon.0 v2:172.21.15.37:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2d080b0a70 con 0x7f2d00051810 2024-04-25T10:29:19.368 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:19.342+0000 7f1f1d780640 1 -- 172.21.15.37:0/2161163013 >> v1:172.21.15.37:6789/0 conn(0x7f1f10009000 legacy=0x7f1f10053bb0 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-04-25T10:29:19.368 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:19.342+0000 7f1f1d780640 1 -- 172.21.15.37:0/2161163013 --> v2:172.21.15.37:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f1f10050aa0 con 0x7f1f100542c0 2024-04-25T10:29:19.368 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:19.342+0000 7f376caaa640 1 -- 172.21.15.37:0/3137835675 reap_dead start 2024-04-25T10:29:19.368 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:19.342+0000 7f8cdcd46640 1 -- 172.21.15.37:0/371701842 shutdown_connections 2024-04-25T10:29:19.368 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:19.342+0000 7f8cdcd46640 1 --2- 172.21.15.37:0/371701842 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f8cd0034060 0x7f8cd00065e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:29:19.369 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:19.342+0000 7f8cdcd46640 1 --2- 172.21.15.37:0/371701842 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f8cd0047db0 0x7f8cd00e7d50 unknown :-1 s=CLOSED pgs=69 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:29:19.369 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:19.342+0000 7f8cdcd46640 1 -- 172.21.15.37:0/371701842 >> 172.21.15.37:0/371701842 conn(0x7f8cd0054cf0 msgr2=0x7f8cd0055100 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:29:19.369 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:19.342+0000 7efde88cf640 1 --2- 172.21.15.37:0/3291425638 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7efdd0004800 0x7efdd0004c10 secure :-1 s=READY pgs=104 cs=0 l=1 rev1=1 crypto rx=0x7efdd4009630 tx=0x7efdd0005830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d0ef4d62d9c96e6c server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:29:19.369 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:19.342+0000 7fd915ca4640 1 -- 172.21.15.37:0/1879434666 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7fd900004800 msgr2=0x7fd900004c10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:29:19.369 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:19.342+0000 7f376caaa640 1 --2- 172.21.15.37:0/3137835675 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f37500053f0 0x7f375000ba80 secure :-1 s=READY pgs=30 cs=0 l=1 rev1=1 crypto rx=0x7f37600044b0 tx=0x7f3750005830 comp rx=0 tx=0).ready entity=mon.2 client_cookie=55c0f1918564a71e server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:29:19.370 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:19.342+0000 7fd915ca4640 1 --2- 172.21.15.37:0/1879434666 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7fd900004800 0x7fd900004c10 secure :-1 s=READY pgs=29 cs=0 l=1 rev1=1 crypto rx=0x7fd8fc001eb0 tx=0x7fd900005830 comp rx=0 tx=0).stop 2024-04-25T10:29:19.370 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:19.342+0000 7f8cdcd46640 1 -- 172.21.15.37:0/371701842 shutdown_connections 2024-04-25T10:29:19.370 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:19.342+0000 7f8cdcd46640 1 -- 172.21.15.37:0/371701842 wait complete. 2024-04-25T10:29:19.370 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:19.342+0000 7f9563fff640 1 --2- >> v2:172.21.15.37:3300/0 conn(0x7f956406cc50 0x7f95640b0060 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:29:19.371 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:19.342+0000 7f676ffff640 1 --2- 172.21.15.37:0/1477415135 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f6760003f10 0x7f6760004800 secure :-1 s=READY pgs=105 cs=0 l=1 rev1=1 crypto rx=0x7f675c00a9c0 tx=0x7f6760005830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c2ad214c5e18dc25 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:29:19.371 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:19.342+0000 7f3766ffd640 1 -- 172.21.15.37:0/3137835675 <== mon.2 v2:172.21.15.203:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3760037070 con 0x7f37500053f0 2024-04-25T10:29:19.371 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:19.342+0000 7f3766ffd640 1 -- 172.21.15.37:0/3137835675 <== mon.2 v2:172.21.15.203:3301/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f376002ad80 con 0x7f37500053f0 2024-04-25T10:29:19.371 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:19.342+0000 7f3766ffd640 1 -- 172.21.15.37:0/3137835675 <== mon.2 v2:172.21.15.203:3301/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3760004080 con 0x7f37500053f0 2024-04-25T10:29:19.371 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:19.342+0000 7f9563fff640 1 --2- >> v2:172.21.15.203:3300/0 conn(0x7f956406d060 0x7f95640b05a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:29:19.372 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:19.342+0000 7f2d0e1ab640 1 -- 172.21.15.37:0/1513236428 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f2d00051810 msgr2=0x7f2d00035070 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:29:19.372 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:19.342+0000 7f2d0e1ab640 1 --2- 172.21.15.37:0/1513236428 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f2d00051810 0x7f2d00035070 secure :-1 s=READY pgs=103 cs=0 l=1 rev1=1 crypto rx=0x7f2d08052040 tx=0x7f2d080a9440 comp rx=0 tx=0).stop 2024-04-25T10:29:19.372 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:19.343+0000 7f2d0e1ab640 1 -- 172.21.15.37:0/1513236428 shutdown_connections 2024-04-25T10:29:19.372 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:19.343+0000 7f2d0e1ab640 1 --2- 172.21.15.37:0/1513236428 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f2d00051810 0x7f2d00035070 unknown :-1 s=CLOSED pgs=103 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:29:19.372 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:19.343+0000 7f2d0e1ab640 1 -- 172.21.15.37:0/1513236428 >> 172.21.15.37:0/1513236428 conn(0x7f2d00055e10 msgr2=0x7f2d00056220 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:29:19.372 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:19.342+0000 7f9563fff640 1 -- --> v1:172.21.15.203:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f956409bb10 con 0x7f956409b700 2024-04-25T10:29:19.372 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:19.342+0000 7f9563fff640 1 -- --> v2:172.21.15.37:3300/0 -- mon_getmap magic: 0 -- 0x7f9564160bc0 con 0x7f956406cc50 2024-04-25T10:29:19.372 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:19.342+0000 7f9563fff640 1 -- --> v2:172.21.15.203:3300/0 -- mon_getmap magic: 0 -- 0x7f95640afbd0 con 0x7f956406d060 2024-04-25T10:29:19.372 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:19.342+0000 7f95627fc640 1 --2- >> v2:172.21.15.37:3300/0 conn(0x7f956406cc50 0x7f95640b0060 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:29:19.373 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:19.343+0000 7f95627fc640 1 --2- >> v2:172.21.15.37:3300/0 conn(0x7f956406cc50 0x7f95640b0060 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.37:3300/0 says I am v2:172.21.15.37:46504/0 (socket says 172.21.15.37:46504) 2024-04-25T10:29:19.373 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:19.343+0000 7f95627fc640 1 -- 172.21.15.37:0/319090434 learned_addr learned my addr 172.21.15.37:0/319090434 (peer_addr_for_me v2:172.21.15.37:0/0) 2024-04-25T10:29:19.373 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:19.343+0000 7f676f7fe640 1 -- 172.21.15.37:0/1477415135 <== mon.0 v2:172.21.15.37:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f675c00a590 con 0x7f6760003f10 2024-04-25T10:29:19.373 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:19.343+0000 7f95637fe640 1 --2- 172.21.15.37:0/319090434 >> v2:172.21.15.203:3300/0 conn(0x7f956406d060 0x7f95640b05a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:29:19.373 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:19.343+0000 7f2d0e1ab640 1 -- 172.21.15.37:0/1513236428 shutdown_connections 2024-04-25T10:29:19.374 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:19.343+0000 7f676f7fe640 1 -- 172.21.15.37:0/1477415135 <== mon.0 v2:172.21.15.37:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f675c02abf0 con 0x7f6760003f10 2024-04-25T10:29:19.374 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:19.343+0000 7f676f7fe640 1 -- 172.21.15.37:0/1477415135 <== mon.0 v2:172.21.15.37:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f675c004020 con 0x7f6760003f10 2024-04-25T10:29:19.374 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:19.343+0000 7f2d0e1ab640 1 -- 172.21.15.37:0/1513236428 wait complete. 2024-04-25T10:29:19.374 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:19.342+0000 7efddbfff640 1 -- 172.21.15.37:0/3291425638 <== mon.0 v2:172.21.15.37:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7efdd4040070 con 0x7efdd0004800 2024-04-25T10:29:19.374 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:19.342+0000 7efddbfff640 1 -- 172.21.15.37:0/3291425638 <== mon.0 v2:172.21.15.37:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7efdd4033670 con 0x7efdd0004800 2024-04-25T10:29:19.375 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:19.342+0000 7efddbfff640 1 -- 172.21.15.37:0/3291425638 <== mon.0 v2:172.21.15.37:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7efdd4009670 con 0x7efdd0004800 2024-04-25T10:29:19.375 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:19.343+0000 7f95627fc640 1 -- 172.21.15.37:0/319090434 >> v2:172.21.15.203:3300/0 conn(0x7f956406d060 msgr2=0x7f95640b05a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:29:19.375 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:19.343+0000 7f95627fc640 1 --2- 172.21.15.37:0/319090434 >> v2:172.21.15.203:3300/0 conn(0x7f956406d060 0x7f95640b05a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:29:19.375 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:19.343+0000 7f95627fc640 1 -- 172.21.15.37:0/319090434 >> v1:172.21.15.203:6789/0 conn(0x7f956409b700 legacy=0x7f956406c540 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:29:19.375 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:19.343+0000 7f95627fc640 1 -- 172.21.15.37:0/319090434 --> v2:172.21.15.37:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f95640b2840 con 0x7f956406cc50 2024-04-25T10:29:19.376 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:19.343+0000 7fd915ca4640 1 -- 172.21.15.37:0/1879434666 shutdown_connections 2024-04-25T10:29:19.376 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:19.343+0000 7fd915ca4640 1 --2- 172.21.15.37:0/1879434666 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7fd900004800 0x7fd900004c10 unknown :-1 s=CLOSED pgs=29 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:29:19.376 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:19.343+0000 7fd915ca4640 1 -- 172.21.15.37:0/1879434666 >> 172.21.15.37:0/1879434666 conn(0x7fd908054cf0 msgr2=0x7fd908055100 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:29:19.376 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:19.343+0000 7f1f1d780640 1 --2- 172.21.15.37:0/2161163013 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f1f100542c0 0x7f1f100546d0 secure :-1 s=READY pgs=106 cs=0 l=1 rev1=1 crypto rx=0x7f1f180768e0 tx=0x7f1f180a71d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a694cc07cb6d88ba server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:29:19.377 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:19.343+0000 7f1f177fe640 1 -- 172.21.15.37:0/2161163013 <== mon.0 v2:172.21.15.37:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1f180b4070 con 0x7f1f100542c0 2024-04-25T10:29:19.377 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:19.344+0000 7f1f177fe640 1 -- 172.21.15.37:0/2161163013 <== mon.0 v2:172.21.15.37:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f1f180a7dd0 con 0x7f1f100542c0 2024-04-25T10:29:19.377 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:19.344+0000 7f1f177fe640 1 -- 172.21.15.37:0/2161163013 <== mon.0 v2:172.21.15.37:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1f180af640 con 0x7f1f100542c0 2024-04-25T10:29:19.377 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:19.344+0000 7f376e50d640 1 -- 172.21.15.37:0/3137835675 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f37500053f0 msgr2=0x7f375000ba80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:29:19.377 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:19.344+0000 7f376e50d640 1 --2- 172.21.15.37:0/3137835675 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f37500053f0 0x7f375000ba80 secure :-1 s=READY pgs=30 cs=0 l=1 rev1=1 crypto rx=0x7f37600044b0 tx=0x7f3750005830 comp rx=0 tx=0).stop 2024-04-25T10:29:19.378 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:19.344+0000 7fd915ca4640 1 -- 172.21.15.37:0/1879434666 shutdown_connections 2024-04-25T10:29:19.378 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:19.344+0000 7fd915ca4640 1 -- 172.21.15.37:0/1879434666 wait complete. 2024-04-25T10:29:19.378 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:19.344+0000 7f376e50d640 1 -- 172.21.15.37:0/3137835675 shutdown_connections 2024-04-25T10:29:19.378 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:19.344+0000 7efdeb334640 1 -- 172.21.15.37:0/3291425638 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7efdd0004800 msgr2=0x7efdd0004c10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:29:19.378 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:19.344+0000 7efdeb334640 1 --2- 172.21.15.37:0/3291425638 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7efdd0004800 0x7efdd0004c10 secure :-1 s=READY pgs=104 cs=0 l=1 rev1=1 crypto rx=0x7efdd4009630 tx=0x7efdd0005830 comp rx=0 tx=0).stop 2024-04-25T10:29:19.378 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:19.344+0000 7f95627fc640 1 --2- 172.21.15.37:0/319090434 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f956406cc50 0x7f95640b0060 secure :-1 s=READY pgs=107 cs=0 l=1 rev1=1 crypto rx=0x7f95640b4bf0 tx=0x7f955402a1f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b31663dcb995b9c6 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:29:19.379 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:19.344+0000 7f67770b1640 1 -- 172.21.15.37:0/1477415135 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f6760003f10 msgr2=0x7f6760004800 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:29:19.379 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:19.344+0000 7f67770b1640 1 --2- 172.21.15.37:0/1477415135 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f6760003f10 0x7f6760004800 secure :-1 s=READY pgs=105 cs=0 l=1 rev1=1 crypto rx=0x7f675c00a9c0 tx=0x7f6760005830 comp rx=0 tx=0).stop 2024-04-25T10:29:19.379 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:19.344+0000 7f677564e640 1 -- 172.21.15.37:0/1477415135 reap_dead start 2024-04-25T10:29:19.379 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:19.344+0000 7efdeb334640 1 -- 172.21.15.37:0/3291425638 shutdown_connections 2024-04-25T10:29:19.379 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:19.344+0000 7efdeb334640 1 --2- 172.21.15.37:0/3291425638 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7efdd0004800 0x7efdd0004c10 secure :-1 s=CLOSED pgs=104 cs=0 l=1 rev1=1 crypto rx=0x7efdd4009630 tx=0x7efdd0005830 comp rx=0 tx=0).stop 2024-04-25T10:29:19.380 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:19.344+0000 7efdeb334640 1 -- 172.21.15.37:0/3291425638 >> 172.21.15.37:0/3291425638 conn(0x7efddc03ac50 msgr2=0x7efddc03b060 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:29:19.380 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:19.344+0000 7f376e50d640 1 --2- 172.21.15.37:0/3137835675 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f37500053f0 0x7f375000ba80 unknown :-1 s=CLOSED pgs=30 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:29:19.380 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:19.344+0000 7f376e50d640 1 -- 172.21.15.37:0/3137835675 >> 172.21.15.37:0/3137835675 conn(0x7f376806cd90 msgr2=0x7f37680b3300 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:29:19.380 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:19.345+0000 7f67770b1640 1 -- 172.21.15.37:0/1477415135 shutdown_connections 2024-04-25T10:29:19.381 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:19.345+0000 7f67770b1640 1 -- 172.21.15.37:0/1477415135 >> 172.21.15.37:0/1477415135 conn(0x7f677006cba0 msgr2=0x7f67700b3290 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:29:19.381 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:19.345+0000 7f67770b1640 1 -- 172.21.15.37:0/1477415135 shutdown_connections 2024-04-25T10:29:19.381 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:19.345+0000 7f67770b1640 1 -- 172.21.15.37:0/1477415135 wait complete. 2024-04-25T10:29:19.381 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:19.345+0000 7f1f1f1e3640 1 -- 172.21.15.37:0/2161163013 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f1f100542c0 msgr2=0x7f1f100546d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:29:19.381 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:19.345+0000 7f1f1f1e3640 1 --2- 172.21.15.37:0/2161163013 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f1f100542c0 0x7f1f100546d0 secure :-1 s=READY pgs=106 cs=0 l=1 rev1=1 crypto rx=0x7f1f180768e0 tx=0x7f1f180a71d0 comp rx=0 tx=0).stop 2024-04-25T10:29:19.381 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:19.345+0000 7f8cdcd46640 1 Processor -- start 2024-04-25T10:29:19.382 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:19.345+0000 7efdeb334640 1 -- 172.21.15.37:0/3291425638 shutdown_connections 2024-04-25T10:29:19.382 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:19.345+0000 7f9561ffb640 1 -- 172.21.15.37:0/319090434 <== mon.0 v2:172.21.15.37:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9554037070 con 0x7f956406cc50 2024-04-25T10:29:19.382 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:19.345+0000 7f9561ffb640 1 -- 172.21.15.37:0/319090434 <== mon.0 v2:172.21.15.37:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f9554032440 con 0x7f956406cc50 2024-04-25T10:29:19.382 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:19.345+0000 7f9561ffb640 1 -- 172.21.15.37:0/319090434 <== mon.0 v2:172.21.15.37:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9554032810 con 0x7f956406cc50 2024-04-25T10:29:19.383 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:19.345+0000 7f1f1f1e3640 1 -- 172.21.15.37:0/2161163013 shutdown_connections 2024-04-25T10:29:19.383 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:19.345+0000 7f1f1f1e3640 1 --2- 172.21.15.37:0/2161163013 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f1f100542c0 0x7f1f100546d0 unknown :-1 s=CLOSED pgs=106 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:29:19.383 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:19.345+0000 7f1f1f1e3640 1 --2- 172.21.15.37:0/2161163013 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f1f100522b0 0x7f1f10009410 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:29:19.383 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:19.345+0000 7f1f1f1e3640 1 -- 172.21.15.37:0/2161163013 >> 172.21.15.37:0/2161163013 conn(0x7f1f10055e00 msgr2=0x7f1f10056210 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:29:19.383 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:19.345+0000 7efdeb334640 1 -- 172.21.15.37:0/3291425638 wait complete. 2024-04-25T10:29:19.384 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:19.345+0000 7f1f1f1e3640 1 -- 172.21.15.37:0/2161163013 shutdown_connections 2024-04-25T10:29:19.384 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:19.345+0000 7f376e50d640 1 -- 172.21.15.37:0/3137835675 shutdown_connections 2024-04-25T10:29:19.384 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:19.345+0000 7f376e50d640 1 -- 172.21.15.37:0/3137835675 wait complete. 2024-04-25T10:29:19.384 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:19.346+0000 7f8cdcd46640 1 -- start start 2024-04-25T10:29:19.385 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:19.346+0000 7f2d0e1ab640 1 Processor -- start 2024-04-25T10:29:19.385 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:19.346+0000 7f2d0e1ab640 1 -- start start 2024-04-25T10:29:19.385 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:19.346+0000 7f9563fff640 1 -- 172.21.15.37:0/319090434 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f956406cc50 msgr2=0x7f95640b0060 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:29:19.385 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:19.346+0000 7f9563fff640 1 --2- 172.21.15.37:0/319090434 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f956406cc50 0x7f95640b0060 secure :-1 s=READY pgs=107 cs=0 l=1 rev1=1 crypto rx=0x7f95640b4bf0 tx=0x7f955402a1f0 comp rx=0 tx=0).stop 2024-04-25T10:29:19.385 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:19.347+0000 7f9563fff640 1 -- 172.21.15.37:0/319090434 shutdown_connections 2024-04-25T10:29:19.386 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:19.347+0000 7f9563fff640 1 --2- 172.21.15.37:0/319090434 >> v2:172.21.15.203:3300/0 conn(0x7f956406d060 0x7f95640b05a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:29:19.386 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:19.347+0000 7f1f1f1e3640 1 -- 172.21.15.37:0/2161163013 wait complete. 2024-04-25T10:29:19.386 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:19.347+0000 7f9563fff640 1 --2- 172.21.15.37:0/319090434 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f956406cc50 0x7f95640b0060 unknown :-1 s=CLOSED pgs=107 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:29:19.386 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:19.347+0000 7f9563fff640 1 -- 172.21.15.37:0/319090434 >> 172.21.15.37:0/319090434 conn(0x7f9564073960 msgr2=0x7f95640b6e10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:29:19.386 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:19.347+0000 7f9563fff640 1 -- 172.21.15.37:0/319090434 shutdown_connections 2024-04-25T10:29:19.387 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:19.347+0000 7f9563fff640 1 -- 172.21.15.37:0/319090434 wait complete. 2024-04-25T10:29:19.387 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:19.347+0000 7f376e50d640 1 Processor -- start 2024-04-25T10:29:19.387 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:19.348+0000 7efdeb334640 1 Processor -- start 2024-04-25T10:29:19.387 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:19.348+0000 7f8cdcd46640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f8cd0047db0 0x7f8cd00d91d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:29:19.387 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:19.348+0000 7f376e50d640 1 -- start start 2024-04-25T10:29:19.388 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:19.348+0000 7f67770b1640 1 Processor -- start 2024-04-25T10:29:19.388 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:19.348+0000 7f67770b1640 1 -- start start 2024-04-25T10:29:19.389 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:19.348+0000 7f8cd77fe640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f8cd0047db0 0x7f8cd00d91d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:29:19.389 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:19.348+0000 7f8cd77fe640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f8cd0047db0 0x7f8cd00d91d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.37:3300/0 says I am v2:172.21.15.37:46514/0 (socket says 172.21.15.37:46514) 2024-04-25T10:29:19.389 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:19.348+0000 7f8cd77fe640 1 -- 172.21.15.37:0/3904300012 learned_addr learned my addr 172.21.15.37:0/3904300012 (peer_addr_for_me v2:172.21.15.37:0/0) 2024-04-25T10:29:19.389 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:19.349+0000 7f8cdcd46640 1 --2- 172.21.15.37:0/3904300012 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f8cd0048300 0x7f8cd00ea320 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:29:19.389 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:19.349+0000 7f8cdcd46640 1 --2- 172.21.15.37:0/3904300012 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f8cd00ea860 0x7f8cd0035a40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:29:19.389 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:19.349+0000 7f2d0e1ab640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f2d00051810 0x7f2d0004ba10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:29:19.390 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:19.349+0000 7f8cdcd46640 1 -- 172.21.15.37:0/3904300012 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 -- 0x7f8cd00509c0 con 0x7f8cd0047db0 2024-04-25T10:29:19.390 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:19.349+0000 7f8cdcd46640 1 -- 172.21.15.37:0/3904300012 --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_getmap magic: 0 -- 0x7f8cd0050840 con 0x7f8cd00ea860 2024-04-25T10:29:19.390 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:19.349+0000 7f8cdcd46640 1 -- 172.21.15.37:0/3904300012 --> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] -- mon_getmap magic: 0 -- 0x7f8cd0006b20 con 0x7f8cd0048300 2024-04-25T10:29:19.391 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:19.349+0000 7f2d0e1ab640 1 --2- >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f2d00054780 0x7f2d000defd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:29:19.391 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:19.349+0000 7f2d0e1ab640 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f2d0004bf50 0x7f2d000df510 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:29:19.391 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:19.349+0000 7f2d0e1ab640 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 -- 0x7f2d000091b0 con 0x7f2d00051810 2024-04-25T10:29:19.391 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:19.349+0000 7f2d0e1ab640 1 -- --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_getmap magic: 0 -- 0x7f2d00009030 con 0x7f2d0004bf50 2024-04-25T10:29:19.391 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:19.349+0000 7f2d077fe640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f2d00051810 0x7f2d0004ba10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:29:19.391 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:19.349+0000 7f2d0e1ab640 1 -- --> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] -- mon_getmap magic: 0 -- 0x7f2d00009330 con 0x7f2d00054780 2024-04-25T10:29:19.391 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:19.349+0000 7f2d077fe640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f2d00051810 0x7f2d0004ba10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.37:3300/0 says I am v2:172.21.15.37:46530/0 (socket says 172.21.15.37:46530) 2024-04-25T10:29:19.391 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:19.349+0000 7f2d077fe640 1 -- 172.21.15.37:0/3107331413 learned_addr learned my addr 172.21.15.37:0/3107331413 (peer_addr_for_me v2:172.21.15.37:0/0) 2024-04-25T10:29:19.391 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:19.348+0000 7efdeb334640 1 -- start start 2024-04-25T10:29:19.392 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:19.349+0000 7f8cd6ffd640 1 --2- 172.21.15.37:0/3904300012 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f8cd0048300 0x7f8cd00ea320 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:29:19.392 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:19.349+0000 7f2d07fff640 1 --2- 172.21.15.37:0/3107331413 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f2d0004bf50 0x7f2d000df510 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:29:19.393 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:19.347+0000 7fd915ca4640 1 Processor -- start 2024-04-25T10:29:19.393 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:19.347+0000 7fd915ca4640 1 -- start start 2024-04-25T10:29:19.393 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:19.349+0000 7fd915ca4640 1 --2- >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7fd900004800 0x7fd90804cdd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:29:19.393 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:19.350+0000 7fd915ca4640 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fd9080de090 0x7fd9080d1fa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:29:19.393 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:19.349+0000 7f8cd77fe640 1 -- 172.21.15.37:0/3904300012 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f8cd0048300 msgr2=0x7f8cd00ea320 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:29:19.393 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:19.349+0000 7f8cd77fe640 1 --2- 172.21.15.37:0/3904300012 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f8cd0048300 0x7f8cd00ea320 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:29:19.393 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:19.349+0000 7f8cd77fe640 1 -- 172.21.15.37:0/3904300012 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f8cd00ea860 msgr2=0x7f8cd0035a40 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2024-04-25T10:29:19.393 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:19.349+0000 7f8cd77fe640 1 --2- 172.21.15.37:0/3904300012 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f8cd00ea860 0x7f8cd0035a40 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:29:19.394 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:19.350+0000 7f2d06ffd640 1 --2- 172.21.15.37:0/3107331413 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f2d00054780 0x7f2d000defd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:29:19.394 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:19.349+0000 7f8cd77fe640 1 -- 172.21.15.37:0/3904300012 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f8cd00d9710 con 0x7f8cd0047db0 2024-04-25T10:29:19.394 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:19.350+0000 7f8cd6ffd640 1 --2- 172.21.15.37:0/3904300012 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f8cd0048300 0x7f8cd00ea320 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T10:29:19.395 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:19.350+0000 7fd915ca4640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fd908034650 0x7fd908034a60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:29:19.395 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:19.350+0000 7fd915ca4640 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 -- 0x7fd908006aa0 con 0x7fd908034650 2024-04-25T10:29:19.395 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:19.350+0000 7fd915ca4640 1 -- --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_getmap magic: 0 -- 0x7fd90800c4e0 con 0x7fd9080de090 2024-04-25T10:29:19.395 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:19.350+0000 7fd915ca4640 1 -- --> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] -- mon_getmap magic: 0 -- 0x7fd908006c20 con 0x7fd900004800 2024-04-25T10:29:19.395 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:19.350+0000 7fd9154a3640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fd908034650 0x7fd908034a60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:29:19.395 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:19.350+0000 7fd9154a3640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fd908034650 0x7fd908034a60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.37:3300/0 says I am v2:172.21.15.37:46532/0 (socket says 172.21.15.37:46532) 2024-04-25T10:29:19.395 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:19.350+0000 7fd9154a3640 1 -- 172.21.15.37:0/1699500182 learned_addr learned my addr 172.21.15.37:0/1699500182 (peer_addr_for_me v2:172.21.15.37:0/0) 2024-04-25T10:29:19.395 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:19.350+0000 7fd90ffff640 1 --2- 172.21.15.37:0/1699500182 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fd9080de090 0x7fd9080d1fa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:29:19.395 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:19.350+0000 7fd914ca2640 1 --2- 172.21.15.37:0/1699500182 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7fd900004800 0x7fd90804cdd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:29:19.395 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:19.350+0000 7f1f1f1e3640 1 Processor -- start 2024-04-25T10:29:19.396 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:19.350+0000 7efdeb334640 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7efddc052f40 0x7efddc050ca0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:29:19.396 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:19.350+0000 7f1f1f1e3640 1 -- start start 2024-04-25T10:29:19.396 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:19.350+0000 7f2d06ffd640 1 -- 172.21.15.37:0/3107331413 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f2d0004bf50 msgr2=0x7f2d000df510 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:29:19.396 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:19.350+0000 7f2d06ffd640 1 --2- 172.21.15.37:0/3107331413 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f2d0004bf50 0x7f2d000df510 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:29:19.397 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:19.350+0000 7efdeb334640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7efddc0d2410 0x7efddc0d40c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:29:19.397 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:19.350+0000 7efdeb334640 1 --2- >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7efddc0d2820 0x7efddc0dcab0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:29:19.397 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:19.350+0000 7efdeb334640 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 -- 0x7efddc0076a0 con 0x7efddc0d2410 2024-04-25T10:29:19.397 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:19.350+0000 7efdeb334640 1 -- --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_getmap magic: 0 -- 0x7efddc03cf10 con 0x7efddc052f40 2024-04-25T10:29:19.397 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:19.350+0000 7efdeb334640 1 -- --> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] -- mon_getmap magic: 0 -- 0x7efddc006d50 con 0x7efddc0d2820 2024-04-25T10:29:19.398 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:19.350+0000 7f2d06ffd640 1 -- 172.21.15.37:0/3107331413 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f2d00051810 msgr2=0x7f2d0004ba10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:29:19.398 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:19.350+0000 7f2d06ffd640 1 --2- 172.21.15.37:0/3107331413 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f2d00051810 0x7f2d0004ba10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:29:19.398 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:19.350+0000 7fd9154a3640 1 -- 172.21.15.37:0/1699500182 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7fd900004800 msgr2=0x7fd90804cdd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:29:19.398 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:19.350+0000 7fd9154a3640 1 --2- 172.21.15.37:0/1699500182 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7fd900004800 0x7fd90804cdd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:29:19.398 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:19.350+0000 7fd9154a3640 1 -- 172.21.15.37:0/1699500182 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fd9080de090 msgr2=0x7fd9080d1fa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:29:19.398 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:19.350+0000 7fd9154a3640 1 --2- 172.21.15.37:0/1699500182 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fd9080de090 0x7fd9080d1fa0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:29:19.398 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:19.350+0000 7fd9154a3640 1 -- 172.21.15.37:0/1699500182 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fd908034fa0 con 0x7fd908034650 2024-04-25T10:29:19.399 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:19.351+0000 7f2d06ffd640 1 -- 172.21.15.37:0/3107331413 --> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f2d0004c690 con 0x7f2d00054780 2024-04-25T10:29:19.399 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:19.350+0000 7efde88cf640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7efddc0d2410 0x7efddc0d40c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:29:19.399 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:19.351+0000 7efde90d0640 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7efddc052f40 0x7efddc050ca0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:29:19.400 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:19.351+0000 7efde90d0640 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7efddc052f40 0x7efddc050ca0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.203:3300/0 says I am v2:172.21.15.37:38070/0 (socket says 172.21.15.37:38070) 2024-04-25T10:29:19.400 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:19.351+0000 7efde90d0640 1 -- 172.21.15.37:0/2894599583 learned_addr learned my addr 172.21.15.37:0/2894599583 (peer_addr_for_me v2:172.21.15.37:0/0) 2024-04-25T10:29:19.400 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:19.351+0000 7f9563fff640 1 Processor -- start 2024-04-25T10:29:19.400 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:19.351+0000 7f2d077fe640 1 --2- 172.21.15.37:0/3107331413 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f2d00051810 0x7f2d0004ba10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T10:29:19.400 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:19.351+0000 7efde98d1640 1 --2- 172.21.15.37:0/2894599583 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7efddc0d2820 0x7efddc0dcab0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:29:19.401 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:19.351+0000 7f9563fff640 1 -- start start 2024-04-25T10:29:19.401 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:19.351+0000 7f2d06ffd640 1 --2- 172.21.15.37:0/3107331413 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f2d00054780 0x7f2d000defd0 secure :-1 s=READY pgs=31 cs=0 l=1 rev1=1 crypto rx=0x7f2cf400c440 tx=0x7f2cf400c900 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:29:19.401 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:19.351+0000 7f8cd77fe640 1 --2- 172.21.15.37:0/3904300012 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f8cd0047db0 0x7f8cd00d91d0 secure :-1 s=READY pgs=108 cs=0 l=1 rev1=1 crypto rx=0x7f8ccc000c00 tx=0x7f8ccc0046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:29:19.402 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:19.351+0000 7fd9154a3640 1 --2- 172.21.15.37:0/1699500182 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fd908034650 0x7fd908034a60 secure :-1 s=READY pgs=109 cs=0 l=1 rev1=1 crypto rx=0x7fd910066010 tx=0x7fd91006da90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:29:19.402 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:19.351+0000 7f67770b1640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f6760003f10 0x7f67701534e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:29:19.402 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:19.351+0000 7f67770b1640 1 --2- >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f67701f2100 0x7f6770140e00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:29:19.402 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:19.351+0000 7f67770b1640 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f6770151390 0x7f67701517a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:29:19.402 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:19.351+0000 7f67770b1640 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 -- 0x7f677015c450 con 0x7f6760003f10 2024-04-25T10:29:19.402 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:19.351+0000 7f67770b1640 1 -- --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_getmap magic: 0 -- 0x7f677015c2d0 con 0x7f6770151390 2024-04-25T10:29:19.402 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:19.351+0000 7f67770b1640 1 -- --> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] -- mon_getmap magic: 0 -- 0x7f67700b49a0 con 0x7f67701f2100 2024-04-25T10:29:19.403 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:19.351+0000 7f8cc3fff640 1 -- 172.21.15.37:0/3904300012 <== mon.0 v2:172.21.15.37:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8ccc037040 con 0x7f8cd0047db0 2024-04-25T10:29:19.403 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:19.351+0000 7f2cf3fff640 1 -- 172.21.15.37:0/3107331413 <== mon.2 v2:172.21.15.203:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2cf4017070 con 0x7f2d00054780 2024-04-25T10:29:19.403 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:19.351+0000 7efde90d0640 1 -- 172.21.15.37:0/2894599583 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7efddc0d2820 msgr2=0x7efddc0dcab0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:29:19.403 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:19.351+0000 7efde90d0640 1 --2- 172.21.15.37:0/2894599583 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7efddc0d2820 0x7efddc0dcab0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:29:19.403 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:19.351+0000 7efde90d0640 1 -- 172.21.15.37:0/2894599583 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7efddc0d2410 msgr2=0x7efddc0d40c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:29:19.403 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:19.351+0000 7efde90d0640 1 --2- 172.21.15.37:0/2894599583 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7efddc0d2410 0x7efddc0d40c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:29:19.403 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:19.351+0000 7efde90d0640 1 -- 172.21.15.37:0/2894599583 --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7efddc0515a0 con 0x7efddc052f40 2024-04-25T10:29:19.404 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:19.352+0000 7fd90d7fa640 1 -- 172.21.15.37:0/1699500182 <== mon.0 v2:172.21.15.37:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd910076110 con 0x7fd908034650 2024-04-25T10:29:19.404 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:19.352+0000 7fd90d7fa640 1 -- 172.21.15.37:0/1699500182 <== mon.0 v2:172.21.15.37:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fd9100762b0 con 0x7fd908034650 2024-04-25T10:29:19.404 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:19.352+0000 7fd915ca4640 1 -- 172.21.15.37:0/1699500182 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fd9080dc4d0 con 0x7fd908034650 2024-04-25T10:29:19.404 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:19.352+0000 7fd90d7fa640 1 -- 172.21.15.37:0/1699500182 <== mon.0 v2:172.21.15.37:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd910074150 con 0x7fd908034650 2024-04-25T10:29:19.404 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:19.352+0000 7fd915ca4640 1 -- 172.21.15.37:0/1699500182 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fd90804c010 con 0x7fd908034650 2024-04-25T10:29:19.404 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:19.352+0000 7f376e50d640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f3768147850 0x7f37681edfe0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:29:19.404 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:19.352+0000 7f376e50d640 1 --2- >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f37681ee520 0x7f37681676e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:29:19.404 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:19.352+0000 7f376e50d640 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f3768167c20 0x7f376815b6b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:29:19.405 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:19.352+0000 7f376e50d640 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 -- 0x7f37680b5df0 con 0x7f3768147850 2024-04-25T10:29:19.405 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:19.352+0000 7f376e50d640 1 -- --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_getmap magic: 0 -- 0x7f37680b4bc0 con 0x7f3768167c20 2024-04-25T10:29:19.405 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:19.352+0000 7f376e50d640 1 -- --> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] -- mon_getmap magic: 0 -- 0x7f37680b54f0 con 0x7f37681ee520 2024-04-25T10:29:19.405 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:19.352+0000 7f8cdcd46640 1 -- 172.21.15.37:0/3904300012 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f8cd0048960 con 0x7f8cd0047db0 2024-04-25T10:29:19.405 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:19.352+0000 7f8cc3fff640 1 -- 172.21.15.37:0/3904300012 <== mon.0 v2:172.21.15.37:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f8ccc042070 con 0x7f8cd0047db0 2024-04-25T10:29:19.405 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:19.352+0000 7f8cdcd46640 1 -- 172.21.15.37:0/3904300012 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f8cd0048f20 con 0x7f8cd0047db0 2024-04-25T10:29:19.405 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:19.352+0000 7efde90d0640 1 --2- 172.21.15.37:0/2894599583 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7efddc052f40 0x7efddc050ca0 secure :-1 s=READY pgs=71 cs=0 l=1 rev1=1 crypto rx=0x7efde0007310 tx=0x7efde00077d0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:29:19.406 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:19.352+0000 7f6774e4d640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f6760003f10 0x7f67701534e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:29:19.406 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:19.352+0000 7f6774e4d640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f6760003f10 0x7f67701534e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.37:3300/0 says I am v2:172.21.15.37:46544/0 (socket says 172.21.15.37:46544) 2024-04-25T10:29:19.406 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:19.352+0000 7f6774e4d640 1 -- 172.21.15.37:0/1129867855 learned_addr learned my addr 172.21.15.37:0/1129867855 (peer_addr_for_me v2:172.21.15.37:0/0) 2024-04-25T10:29:19.406 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:19.352+0000 7f677564e640 1 --2- 172.21.15.37:0/1129867855 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f6770151390 0x7f67701517a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:29:19.406 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:19.352+0000 7f6774e4d640 1 -- 172.21.15.37:0/1129867855 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f67701f2100 msgr2=0x7f6770140e00 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2024-04-25T10:29:19.406 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:19.352+0000 7f6774e4d640 1 --2- 172.21.15.37:0/1129867855 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f67701f2100 0x7f6770140e00 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:29:19.406 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:19.352+0000 7f6774e4d640 1 -- 172.21.15.37:0/1129867855 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f6770151390 msgr2=0x7f67701517a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:29:19.406 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:19.352+0000 7f6774e4d640 1 --2- 172.21.15.37:0/1129867855 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f6770151390 0x7f67701517a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:29:19.406 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:19.352+0000 7f6774e4d640 1 -- 172.21.15.37:0/1129867855 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f6770151ce0 con 0x7f6760003f10 2024-04-25T10:29:19.407 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:19.352+0000 7f2d0e1ab640 1 -- 172.21.15.37:0/3107331413 --> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f2d0004c970 con 0x7f2d00054780 2024-04-25T10:29:19.407 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:19.352+0000 7f2d0e1ab640 1 -- 172.21.15.37:0/3107331413 --> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] -- mon_subscribe({osdmap=0}) -- 0x7f2d0004cf30 con 0x7f2d00054780 2024-04-25T10:29:19.407 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:19.353+0000 7f6774e4d640 1 --2- 172.21.15.37:0/1129867855 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f6760003f10 0x7f67701534e0 secure :-1 s=READY pgs=110 cs=0 l=1 rev1=1 crypto rx=0x7f6764011890 tx=0x7f6764011c90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:29:19.407 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:19.353+0000 7efdd9ffb640 1 -- 172.21.15.37:0/2894599583 <== mon.1 v2:172.21.15.203:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7efde000f960 con 0x7efddc052f40 2024-04-25T10:29:19.408 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:19.353+0000 7f9563fff640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f956406cc50 0x7f9564151130 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:29:19.408 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:19.353+0000 7f1f1f1e3640 1 --2- >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f1f10009000 0x7f1f10031460 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:29:19.408 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:19.353+0000 7efdeb334640 1 -- 172.21.15.37:0/2894599583 --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7efddc0d6690 con 0x7efddc052f40 2024-04-25T10:29:19.408 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:19.353+0000 7f9562ffd640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f956406cc50 0x7f9564151130 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:29:19.409 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:19.353+0000 7f1f1f1e3640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f1f100522b0 0x7f1f1004e500 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:29:19.409 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:19.353+0000 7efdeb334640 1 -- 172.21.15.37:0/2894599583 --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7efddc0d6c50 con 0x7efddc052f40 2024-04-25T10:29:19.409 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:19.353+0000 7f9562ffd640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f956406cc50 0x7f9564151130 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.37:3300/0 says I am v2:172.21.15.37:46568/0 (socket says 172.21.15.37:46568) 2024-04-25T10:29:19.409 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:19.353+0000 7f9562ffd640 1 -- 172.21.15.37:0/3561152926 learned_addr learned my addr 172.21.15.37:0/3561152926 (peer_addr_for_me v2:172.21.15.37:0/0) 2024-04-25T10:29:19.410 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:19.353+0000 7f376caaa640 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f3768167c20 0x7f376815b6b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:29:19.410 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:19.353+0000 7f376caaa640 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f3768167c20 0x7f376815b6b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.203:3300/0 says I am v2:172.21.15.37:38086/0 (socket says 172.21.15.37:38086) 2024-04-25T10:29:19.410 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:19.353+0000 7f376caaa640 1 -- 172.21.15.37:0/2391780133 learned_addr learned my addr 172.21.15.37:0/2391780133 (peer_addr_for_me v2:172.21.15.37:0/0) 2024-04-25T10:29:19.410 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:19.353+0000 7f9563fff640 1 --2- 172.21.15.37:0/3561152926 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f956406d060 0x7f9564148100 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:29:19.410 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:19.353+0000 7f1f1f1e3640 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f1f100542c0 0x7f1f100cfff0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:29:19.411 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:19.353+0000 7efdd9ffb640 1 -- 172.21.15.37:0/2894599583 <== mon.1 v2:172.21.15.203:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7efde0018070 con 0x7efddc052f40 2024-04-25T10:29:19.411 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:19.353+0000 7efdd9ffb640 1 -- 172.21.15.37:0/2894599583 <== mon.1 v2:172.21.15.203:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7efde000fd80 con 0x7efddc052f40 2024-04-25T10:29:19.411 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:19.353+0000 7f9563fff640 1 --2- 172.21.15.37:0/3561152926 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f956409b700 0x7f956416a540 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:29:19.411 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:19.353+0000 7f1f1cf7f640 1 --2- >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f1f10009000 0x7f1f10031460 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:29:19.412 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:19.353+0000 7f95627fc640 1 --2- 172.21.15.37:0/3561152926 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f956406d060 0x7f9564148100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:29:19.412 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:19.353+0000 7f37677fe640 1 --2- 172.21.15.37:0/2391780133 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f37681ee520 0x7f37681676e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:29:19.412 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:19.354+0000 7f1f1cf7f640 1 --2- >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f1f10009000 0x7f1f10031460 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.203:3301/0 says I am v2:172.21.15.37:51634/0 (socket says 172.21.15.37:51634) 2024-04-25T10:29:19.412 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:19.354+0000 7f1f1f1e3640 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 -- 0x7f1f100ed4e0 con 0x7f1f100522b0 2024-04-25T10:29:19.412 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:19.354+0000 7f1f1f1e3640 1 -- --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_getmap magic: 0 -- 0x7f1f100ed360 con 0x7f1f100542c0 2024-04-25T10:29:19.413 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:19.353+0000 7f2cf3fff640 1 -- 172.21.15.37:0/3107331413 <== mon.2 v2:172.21.15.203:3301/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f2cf4005ce0 con 0x7f2d00054780 2024-04-25T10:29:19.413 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:19.353+0000 7f2d0e1ab640 1 -- 172.21.15.37:0/3107331413 --> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] -- mon_get_version(what=osdmap handle=1) -- 0x7f2d000059c0 con 0x7f2d00054780 2024-04-25T10:29:19.413 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:19.354+0000 7f9563fff640 1 -- 172.21.15.37:0/3561152926 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 -- 0x7f9564160f40 con 0x7f956406cc50 2024-04-25T10:29:19.413 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:19.354+0000 7f1f1f1e3640 1 -- --> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] -- mon_getmap magic: 0 -- 0x7f1f100ed660 con 0x7f1f10009000 2024-04-25T10:29:19.413 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:19.354+0000 7f95637fe640 1 --2- 172.21.15.37:0/3561152926 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f956409b700 0x7f956416a540 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:29:19.414 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:19.354+0000 7f1f1cf7f640 1 -- 172.21.15.37:0/997756408 learned_addr learned my addr 172.21.15.37:0/997756408 (peer_addr_for_me v2:172.21.15.37:0/0) 2024-04-25T10:29:19.414 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:19.354+0000 7f9563fff640 1 -- 172.21.15.37:0/3561152926 --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_getmap magic: 0 -- 0x7f95640b2e00 con 0x7f956406d060 2024-04-25T10:29:19.414 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:19.354+0000 7f9563fff640 1 -- 172.21.15.37:0/3561152926 --> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] -- mon_getmap magic: 0 -- 0x7f95640af7f0 con 0x7f956409b700 2024-04-25T10:29:19.414 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:19.353+0000 7f8cc3fff640 1 -- 172.21.15.37:0/3904300012 <== mon.0 v2:172.21.15.37:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8ccc030c00 con 0x7f8cd0047db0 2024-04-25T10:29:19.415 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:19.353+0000 7f8cc1ffb640 1 -- 172.21.15.37:0/3904300012 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_get_version(what=osdmap handle=1) -- 0x7f8cb8001030 con 0x7f8cd0047db0 2024-04-25T10:29:19.415 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:19.354+0000 7f1f1d780640 1 --2- 172.21.15.37:0/997756408 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f1f100542c0 0x7f1f100cfff0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:29:19.415 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:19.354+0000 7fd915ca4640 1 -- 172.21.15.37:0/1699500182 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_get_version(what=osdmap handle=1) -- 0x7fd90804c9b0 con 0x7fd908034650 2024-04-25T10:29:19.415 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:19.354+0000 7f376caaa640 1 -- 172.21.15.37:0/2391780133 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f37681ee520 msgr2=0x7f37681676e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:29:19.415 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:19.354+0000 7f376caaa640 1 --2- 172.21.15.37:0/2391780133 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f37681ee520 0x7f37681676e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:29:19.416 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:19.354+0000 7f376caaa640 1 -- 172.21.15.37:0/2391780133 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f3768147850 msgr2=0x7f37681edfe0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:29:19.416 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:19.354+0000 7f376caaa640 1 --2- 172.21.15.37:0/2391780133 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f3768147850 0x7f37681edfe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:29:19.416 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:19.354+0000 7f376caaa640 1 -- 172.21.15.37:0/2391780133 --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f3768168030 con 0x7f3768167c20 2024-04-25T10:29:19.416 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:19.354+0000 7f2cf3fff640 1 -- 172.21.15.37:0/3107331413 <== mon.2 v2:172.21.15.203:3301/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2cf4017070 con 0x7f2d00054780 2024-04-25T10:29:19.416 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:19.355+0000 7f95627fc640 1 -- 172.21.15.37:0/3561152926 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f956409b700 msgr2=0x7f956416a540 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:29:19.416 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:19.355+0000 7f95627fc640 1 --2- 172.21.15.37:0/3561152926 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f956409b700 0x7f956416a540 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:29:19.416 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:19.355+0000 7f95627fc640 1 -- 172.21.15.37:0/3561152926 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f956406cc50 msgr2=0x7f9564151130 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:29:19.417 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:19.355+0000 7f95627fc640 1 --2- 172.21.15.37:0/3561152926 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f956406cc50 0x7f9564151130 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:29:19.417 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:19.355+0000 7f95627fc640 1 -- 172.21.15.37:0/3561152926 --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f95640cbd60 con 0x7f956406d060 2024-04-25T10:29:19.417 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:19.355+0000 7efdeb334640 1 -- 172.21.15.37:0/2894599583 --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_get_version(what=osdmap handle=1) -- 0x7efdc4001030 con 0x7efddc052f40 2024-04-25T10:29:19.417 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:19.355+0000 7f376caaa640 1 --2- 172.21.15.37:0/2391780133 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f3768167c20 0x7f376815b6b0 secure :-1 s=READY pgs=72 cs=0 l=1 rev1=1 crypto rx=0x7f37600045d0 tx=0x7f3760003a70 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:29:19.418 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:19.355+0000 7f1f17fff640 1 --2- 172.21.15.37:0/997756408 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f1f100522b0 0x7f1f1004e500 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:29:19.418 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:19.355+0000 7f1f1d780640 1 -- 172.21.15.37:0/997756408 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f1f10009000 msgr2=0x7f1f10031460 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:29:19.418 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:19.355+0000 7f1f1d780640 1 --2- 172.21.15.37:0/997756408 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f1f10009000 0x7f1f10031460 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:29:19.418 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:19.355+0000 7f1f1d780640 1 -- 172.21.15.37:0/997756408 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f1f100522b0 msgr2=0x7f1f1004e500 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:29:19.418 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:19.355+0000 7f1f1d780640 1 --2- 172.21.15.37:0/997756408 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f1f100522b0 0x7f1f1004e500 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:29:19.418 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:19.355+0000 7f1f1d780640 1 -- 172.21.15.37:0/997756408 --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f1f10031d20 con 0x7f1f100542c0 2024-04-25T10:29:19.418 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:19.355+0000 7f1f17fff640 1 --2- 172.21.15.37:0/997756408 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f1f100522b0 0x7f1f1004e500 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T10:29:19.419 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:19.355+0000 7f3764ff9640 1 -- 172.21.15.37:0/2391780133 <== mon.1 v2:172.21.15.203:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3760044070 con 0x7f3768167c20 2024-04-25T10:29:19.419 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:19.355+0000 7f1f1cf7f640 1 --2- 172.21.15.37:0/997756408 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f1f10009000 0x7f1f10031460 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-25T10:29:19.419 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:19.355+0000 7f95627fc640 1 --2- 172.21.15.37:0/3561152926 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f956406d060 0x7f9564148100 secure :-1 s=READY pgs=73 cs=0 l=1 rev1=1 crypto rx=0x7f9564072de0 tx=0x7f95540046c0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:29:19.419 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:19.355+0000 7f1f1d780640 1 --2- 172.21.15.37:0/997756408 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f1f100542c0 0x7f1f100cfff0 secure :-1 s=READY pgs=74 cs=0 l=1 rev1=1 crypto rx=0x7f1f1806af30 tx=0x7f1f1806a5a0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:29:19.420 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:19.355+0000 7f376e50d640 1 -- 172.21.15.37:0/2391780133 --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f3768165f80 con 0x7f3768167c20 2024-04-25T10:29:19.420 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:19.355+0000 7f3764ff9640 1 -- 172.21.15.37:0/2391780133 <== mon.1 v2:172.21.15.203:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f3760038040 con 0x7f3768167c20 2024-04-25T10:29:19.420 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:19.355+0000 7f376e50d640 1 -- 172.21.15.37:0/2391780133 --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f3768166540 con 0x7f3768167c20 2024-04-25T10:29:19.420 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:19.355+0000 7f954b7fe640 1 -- 172.21.15.37:0/3561152926 <== mon.1 v2:172.21.15.203:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9554041030 con 0x7f956406d060 2024-04-25T10:29:19.420 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:19.356+0000 7f3764ff9640 1 -- 172.21.15.37:0/2391780133 <== mon.1 v2:172.21.15.203:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3760037050 con 0x7f3768167c20 2024-04-25T10:29:19.421 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:19.356+0000 7f954b7fe640 1 -- 172.21.15.37:0/3561152926 <== mon.1 v2:172.21.15.203:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f9554030500 con 0x7f956406d060 2024-04-25T10:29:19.421 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:19.356+0000 7f954b7fe640 1 -- 172.21.15.37:0/3561152926 <== mon.1 v2:172.21.15.203:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9554041030 con 0x7f956406d060 2024-04-25T10:29:19.421 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:19.355+0000 7f676d7fa640 1 -- 172.21.15.37:0/1129867855 <== mon.0 v2:172.21.15.37:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6764004440 con 0x7f6760003f10 2024-04-25T10:29:19.421 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:19.355+0000 7f676d7fa640 1 -- 172.21.15.37:0/1129867855 <== mon.0 v2:172.21.15.37:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f676400f720 con 0x7f6760003f10 2024-04-25T10:29:19.421 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:19.355+0000 7f676d7fa640 1 -- 172.21.15.37:0/1129867855 <== mon.0 v2:172.21.15.37:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f676400fad0 con 0x7f6760003f10 2024-04-25T10:29:19.422 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:19.356+0000 7f67770b1640 1 -- 172.21.15.37:0/1129867855 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f6770154680 con 0x7f6760003f10 2024-04-25T10:29:19.422 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:19.356+0000 7f67770b1640 1 -- 172.21.15.37:0/1129867855 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f67701540c0 con 0x7f6760003f10 2024-04-25T10:29:19.422 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:19.356+0000 7f1f157fa640 1 -- 172.21.15.37:0/997756408 <== mon.1 v2:172.21.15.203:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1f180b4050 con 0x7f1f100542c0 2024-04-25T10:29:19.422 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:19.356+0000 7f1f1f1e3640 1 -- 172.21.15.37:0/997756408 --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f1f10033420 con 0x7f1f100542c0 2024-04-25T10:29:19.422 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:19.356+0000 7fd90d7fa640 1 -- 172.21.15.37:0/1699500182 <== mon.0 v2:172.21.15.37:3300/0 4 ==== mgrmap(e 5) ==== 96782+0+0 (secure 0 0 0) 0x7fd9100742f0 con 0x7fd908034650 2024-04-25T10:29:19.423 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:19.356+0000 7f1f1f1e3640 1 -- 172.21.15.37:0/997756408 --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f1f100339e0 con 0x7f1f100542c0 2024-04-25T10:29:19.423 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:19.356+0000 7f8cc3fff640 1 -- 172.21.15.37:0/3904300012 <== mon.0 v2:172.21.15.37:3300/0 4 ==== mgrmap(e 5) ==== 96782+0+0 (secure 0 0 0) 0x7f8ccc047460 con 0x7f8cd0047db0 2024-04-25T10:29:19.423 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:19.357+0000 7f1f157fa640 1 -- 172.21.15.37:0/997756408 <== mon.1 v2:172.21.15.203:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f1f180b5040 con 0x7f1f100542c0 2024-04-25T10:29:19.424 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:19.357+0000 7f67770b1640 1 -- 172.21.15.37:0/1129867855 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_get_version(what=osdmap handle=1) -- 0x7f6750001030 con 0x7f6760003f10 2024-04-25T10:29:19.424 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:19.357+0000 7fd90d7fa640 1 --2- 172.21.15.37:0/1699500182 >> [v2:172.21.15.203:6832/2418205237,v1:172.21.15.203:6833/2418205237] conn(0x7fd8f4074580 0x7fd8f4076a90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:29:19.424 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:19.357+0000 7f8cc3fff640 1 --2- 172.21.15.37:0/3904300012 >> [v2:172.21.15.203:6832/2418205237,v1:172.21.15.203:6833/2418205237] conn(0x7f8cbc074580 0x7f8cbc076a90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:29:19.424 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:19.357+0000 7fd914ca2640 1 --2- 172.21.15.37:0/1699500182 >> [v2:172.21.15.203:6832/2418205237,v1:172.21.15.203:6833/2418205237] conn(0x7fd8f4074580 0x7fd8f4076a90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:29:19.425 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:19.357+0000 7f8cc3fff640 1 -- 172.21.15.37:0/3904300012 <== mon.0 v2:172.21.15.37:3300/0 5 ==== osd_map(23..23 src has 1..23) ==== 5709+0+0 (secure 0 0 0) 0x7f8ccc0b6830 con 0x7f8cd0047db0 2024-04-25T10:29:19.425 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:19.357+0000 7f8cd6ffd640 1 --2- 172.21.15.37:0/3904300012 >> [v2:172.21.15.203:6832/2418205237,v1:172.21.15.203:6833/2418205237] conn(0x7f8cbc074580 0x7f8cbc076a90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:29:19.425 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:19.357+0000 7f2cf3fff640 1 -- 172.21.15.37:0/3107331413 <== mon.2 v2:172.21.15.203:3301/0 4 ==== mgrmap(e 5) ==== 96782+0+0 (secure 0 0 0) 0x7f2cf401a070 con 0x7f2d00054780 2024-04-25T10:29:19.425 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:19.358+0000 7f2cf3fff640 1 --2- 172.21.15.37:0/3107331413 >> [v2:172.21.15.203:6832/2418205237,v1:172.21.15.203:6833/2418205237] conn(0x7f2cec0748b0 0x7f2cec076dc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:29:19.426 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:19.357+0000 7efdd9ffb640 1 -- 172.21.15.37:0/2894599583 <== mon.1 v2:172.21.15.203:3300/0 4 ==== mgrmap(e 5) ==== 96782+0+0 (secure 0 0 0) 0x7efde001d460 con 0x7efddc052f40 2024-04-25T10:29:19.426 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:19.358+0000 7efdd9ffb640 1 --2- 172.21.15.37:0/2894599583 >> [v2:172.21.15.203:6832/2418205237,v1:172.21.15.203:6833/2418205237] conn(0x7efdc8074580 0x7efdc8076a90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:29:19.426 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:19.358+0000 7efdd9ffb640 1 -- 172.21.15.37:0/2894599583 <== mon.1 v2:172.21.15.203:3300/0 5 ==== osd_map(23..23 src has 1..23) ==== 5709+0+0 (secure 0 0 0) 0x7efde00957a0 con 0x7efddc052f40 2024-04-25T10:29:19.426 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:19.358+0000 7efdd9ffb640 1 --2- 172.21.15.37:0/2894599583 >> [v2:172.21.15.203:6810/2309615014,v1:172.21.15.203:6812/2309615014] conn(0x7efdc807c7f0 0x7efdc807ec40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:29:19.426 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:19.358+0000 7efdd9ffb640 1 -- 172.21.15.37:0/2894599583 --> [v2:172.21.15.203:6810/2309615014,v1:172.21.15.203:6812/2309615014] -- command(tid 1: {"prefix": "get_command_descriptions"}) -- 0x7efde0014e90 con 0x7efdc807c7f0 2024-04-25T10:29:19.426 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:19.358+0000 7efdd9ffb640 1 -- 172.21.15.37:0/2894599583 <== mon.1 v2:172.21.15.203:3300/0 6 ==== mon_get_version_reply(handle=1 version=23) ==== 24+0+0 (secure 0 0 0) 0x7efde0095b90 con 0x7efddc052f40 2024-04-25T10:29:19.426 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:19.358+0000 7f1f157fa640 1 -- 172.21.15.37:0/997756408 <== mon.1 v2:172.21.15.203:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1f180b9810 con 0x7f1f100542c0 2024-04-25T10:29:19.426 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:19.358+0000 7f1f1f1e3640 1 -- 172.21.15.37:0/997756408 --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_get_version(what=osdmap handle=1) -- 0x7f1f100059c0 con 0x7f1f100542c0 2024-04-25T10:29:19.427 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:19.358+0000 7fd90d7fa640 1 -- 172.21.15.37:0/1699500182 <== mon.0 v2:172.21.15.37:3300/0 5 ==== osd_map(23..23 src has 1..23) ==== 5709+0+0 (secure 0 0 0) 0x7fd910110820 con 0x7fd908034650 2024-04-25T10:29:19.427 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:19.358+0000 7fd90d7fa640 1 --2- 172.21.15.37:0/1699500182 >> [v2:172.21.15.37:6816/2746074720,v1:172.21.15.37:6817/2746074720] conn(0x7fd8f407c850 0x7fd8f407eca0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:29:19.427 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:19.358+0000 7f2cf3fff640 1 -- 172.21.15.37:0/3107331413 <== mon.2 v2:172.21.15.203:3301/0 5 ==== osd_map(23..23 src has 1..23) ==== 5709+0+0 (secure 0 0 0) 0x7f2cf4005ae0 con 0x7f2d00054780 2024-04-25T10:29:19.427 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:19.358+0000 7f2cf3fff640 1 --2- 172.21.15.37:0/3107331413 >> [v2:172.21.15.203:6824/1203822890,v1:172.21.15.203:6825/1203822890] conn(0x7f2cec07d630 0x7f2cec07fb20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:29:19.427 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:19.358+0000 7f2cf3fff640 1 -- 172.21.15.37:0/3107331413 --> [v2:172.21.15.203:6824/1203822890,v1:172.21.15.203:6825/1203822890] -- command(tid 1: {"prefix": "get_command_descriptions"}) -- 0x7f2cec0801f0 con 0x7f2cec07d630 2024-04-25T10:29:19.428 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:19.358+0000 7f3764ff9640 1 -- 172.21.15.37:0/2391780133 <== mon.1 v2:172.21.15.203:3300/0 4 ==== mgrmap(e 5) ==== 96782+0+0 (secure 0 0 0) 0x7f376003c610 con 0x7f3768167c20 2024-04-25T10:29:19.428 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:19.358+0000 7f2cf3fff640 1 -- 172.21.15.37:0/3107331413 <== mon.2 v2:172.21.15.203:3301/0 6 ==== mon_get_version_reply(handle=1 version=23) ==== 24+0+0 (secure 0 0 0) 0x7f2cf405af70 con 0x7f2d00054780 2024-04-25T10:29:19.428 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:19.357+0000 7f9563fff640 1 -- 172.21.15.37:0/3561152926 --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f95640cc040 con 0x7f956406d060 2024-04-25T10:29:19.428 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:19.357+0000 7f9563fff640 1 -- 172.21.15.37:0/3561152926 --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f95640cc600 con 0x7f956406d060 2024-04-25T10:29:19.429 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:19.357+0000 7f8cc3fff640 1 --2- 172.21.15.37:0/3904300012 >> [v2:172.21.15.203:6800/4051532063,v1:172.21.15.203:6801/4051532063] conn(0x7f8cbc07c7f0 0x7f8cbc07ec40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:29:19.429 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:19.357+0000 7f8cc3fff640 1 -- 172.21.15.37:0/3904300012 --> [v2:172.21.15.203:6800/4051532063,v1:172.21.15.203:6801/4051532063] -- command(tid 1: {"prefix": "get_command_descriptions"}) -- 0x7f8ccc001d30 con 0x7f8cbc07c7f0 2024-04-25T10:29:19.429 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:19.358+0000 7f8cc3fff640 1 -- 172.21.15.37:0/3904300012 <== mon.0 v2:172.21.15.37:3300/0 6 ==== mon_get_version_reply(handle=1 version=23) ==== 24+0+0 (secure 0 0 0) 0x7f8ccc0b6c20 con 0x7f8cd0047db0 2024-04-25T10:29:19.429 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:19.359+0000 7fd90d7fa640 1 -- 172.21.15.37:0/1699500182 --> [v2:172.21.15.37:6816/2746074720,v1:172.21.15.37:6817/2746074720] -- command(tid 1: {"prefix": "get_command_descriptions"}) -- 0x7fd910099e50 con 0x7fd8f407c850 2024-04-25T10:29:19.429 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:19.359+0000 7f8cd7fff640 1 --2- 172.21.15.37:0/3904300012 >> [v2:172.21.15.203:6800/4051532063,v1:172.21.15.203:6801/4051532063] conn(0x7f8cbc07c7f0 0x7f8cbc07ec40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:29:19.430 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:19.359+0000 7fd90d7fa640 1 -- 172.21.15.37:0/1699500182 <== mon.0 v2:172.21.15.37:3300/0 6 ==== mon_get_version_reply(handle=1 version=23) ==== 24+0+0 (secure 0 0 0) 0x7fd910110c10 con 0x7fd908034650 2024-04-25T10:29:19.430 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:19.359+0000 7fd914ca2640 1 --2- 172.21.15.37:0/1699500182 >> [v2:172.21.15.203:6832/2418205237,v1:172.21.15.203:6833/2418205237] conn(0x7fd8f4074580 0x7fd8f4076a90 secure :-1 s=READY pgs=73 cs=0 l=1 rev1=1 crypto rx=0x7fd908003fc0 tx=0x7fd904009110 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:29:19.430 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:19.359+0000 7efde88cf640 1 --2- 172.21.15.37:0/2894599583 >> [v2:172.21.15.203:6832/2418205237,v1:172.21.15.203:6833/2418205237] conn(0x7efdc8074580 0x7efdc8076a90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:29:19.430 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:19.359+0000 7f9563fff640 1 -- 172.21.15.37:0/3561152926 --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_get_version(what=osdmap handle=1) -- 0x7f9544001030 con 0x7f956406d060 2024-04-25T10:29:19.431 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:19.359+0000 7f376e50d640 1 -- 172.21.15.37:0/2391780133 --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_get_version(what=osdmap handle=1) -- 0x7f3768076bc0 con 0x7f3768167c20 2024-04-25T10:29:19.431 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:19.359+0000 7fd90ffff640 1 --2- 172.21.15.37:0/1699500182 >> [v2:172.21.15.37:6816/2746074720,v1:172.21.15.37:6817/2746074720] conn(0x7fd8f407c850 0x7fd8f407eca0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:29:19.431 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:19.359+0000 7f8cd6ffd640 1 --2- 172.21.15.37:0/3904300012 >> [v2:172.21.15.203:6832/2418205237,v1:172.21.15.203:6833/2418205237] conn(0x7f8cbc074580 0x7f8cbc076a90 secure :-1 s=READY pgs=74 cs=0 l=1 rev1=1 crypto rx=0x7f8cd00358b0 tx=0x7f8cc4009040 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:29:19.431 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:19.359+0000 7f2d077fe640 1 --2- 172.21.15.37:0/3107331413 >> [v2:172.21.15.203:6832/2418205237,v1:172.21.15.203:6833/2418205237] conn(0x7f2cec0748b0 0x7f2cec076dc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:29:19.432 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:19.359+0000 7f8cd7fff640 1 --2- 172.21.15.37:0/3904300012 >> [v2:172.21.15.203:6800/4051532063,v1:172.21.15.203:6801/4051532063] conn(0x7f8cbc07c7f0 0x7f8cbc07ec40 crc :-1 s=READY pgs=9 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.4 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:29:19.432 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:19.359+0000 7f2d07fff640 1 --2- 172.21.15.37:0/3107331413 >> [v2:172.21.15.203:6824/1203822890,v1:172.21.15.203:6825/1203822890] conn(0x7f2cec07d630 0x7f2cec07fb20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:29:19.432 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:19.359+0000 7f1f157fa640 1 -- 172.21.15.37:0/997756408 <== mon.1 v2:172.21.15.203:3300/0 4 ==== mgrmap(e 5) ==== 96782+0+0 (secure 0 0 0) 0x7f1f180c2020 con 0x7f1f100542c0 2024-04-25T10:29:19.433 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:19.360+0000 7efde98d1640 1 --2- 172.21.15.37:0/2894599583 >> [v2:172.21.15.203:6810/2309615014,v1:172.21.15.203:6812/2309615014] conn(0x7efdc807c7f0 0x7efdc807ec40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:29:19.433 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:19.360+0000 7f2d07fff640 1 --2- 172.21.15.37:0/3107331413 >> [v2:172.21.15.203:6824/1203822890,v1:172.21.15.203:6825/1203822890] conn(0x7f2cec07d630 0x7f2cec07fb20 crc :-1 s=READY pgs=10 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.5 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:29:19.433 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:19.360+0000 7f3764ff9640 1 --2- 172.21.15.37:0/2391780133 >> [v2:172.21.15.203:6832/2418205237,v1:172.21.15.203:6833/2418205237] conn(0x7f37540744b0 0x7f37540769c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:29:19.433 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:19.360+0000 7f1f157fa640 1 --2- 172.21.15.37:0/997756408 >> [v2:172.21.15.203:6832/2418205237,v1:172.21.15.203:6833/2418205237] conn(0x7f1efc0744b0 0x7f1efc0769c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:29:19.434 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:19.360+0000 7f2d077fe640 1 --2- 172.21.15.37:0/3107331413 >> [v2:172.21.15.203:6832/2418205237,v1:172.21.15.203:6833/2418205237] conn(0x7f2cec0748b0 0x7f2cec076dc0 secure :-1 s=READY pgs=75 cs=0 l=1 rev1=1 crypto rx=0x7f2cfc0016e0 tx=0x7f2cfc009040 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:29:19.434 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:19.360+0000 7f3764ff9640 1 -- 172.21.15.37:0/2391780133 <== mon.1 v2:172.21.15.203:3300/0 5 ==== osd_map(23..23 src has 1..23) ==== 5709+0+0 (secure 0 0 0) 0x7f37600b4f50 con 0x7f3768167c20 2024-04-25T10:29:19.434 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:19.360+0000 7fd90ffff640 1 --2- 172.21.15.37:0/1699500182 >> [v2:172.21.15.37:6816/2746074720,v1:172.21.15.37:6817/2746074720] conn(0x7fd8f407c850 0x7fd8f407eca0 crc :-1 s=READY pgs=10 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:29:19.434 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:19.360+0000 7f1f157fa640 1 -- 172.21.15.37:0/997756408 <== mon.1 v2:172.21.15.203:3300/0 5 ==== osd_map(23..23 src has 1..23) ==== 5709+0+0 (secure 0 0 0) 0x7f1f181325c0 con 0x7f1f100542c0 2024-04-25T10:29:19.435 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:19.360+0000 7f3767fff640 1 --2- 172.21.15.37:0/2391780133 >> [v2:172.21.15.203:6832/2418205237,v1:172.21.15.203:6833/2418205237] conn(0x7f37540744b0 0x7f37540769c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:29:19.435 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:19.361+0000 7f3764ff9640 1 --2- 172.21.15.37:0/2391780133 >> [v2:172.21.15.37:6800/1690341662,v1:172.21.15.37:6801/1690341662] conn(0x7f375407c6a0 0x7f375407eaf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:29:19.435 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:19.361+0000 7f3764ff9640 1 -- 172.21.15.37:0/2391780133 --> [v2:172.21.15.37:6800/1690341662,v1:172.21.15.37:6801/1690341662] -- command(tid 1: {"prefix": "get_command_descriptions"}) -- 0x7f375407f1c0 con 0x7f375407c6a0 2024-04-25T10:29:19.435 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:19.361+0000 7f1f157fa640 1 --2- 172.21.15.37:0/997756408 >> [v2:172.21.15.37:6824/1815290618,v1:172.21.15.37:6825/1815290618] conn(0x7f1efc07c780 0x7f1efc07ebd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:29:19.436 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:19.361+0000 7f3764ff9640 1 -- 172.21.15.37:0/2391780133 <== mon.1 v2:172.21.15.203:3300/0 6 ==== mon_get_version_reply(handle=1 version=23) ==== 24+0+0 (secure 0 0 0) 0x7f376007b760 con 0x7f3768167c20 2024-04-25T10:29:19.436 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:19.361+0000 7f8cc3fff640 1 -- 172.21.15.37:0/3904300012 <== osd.4 v2:172.21.15.203:6800/4051532063 1 ==== command_reply(tid 1: 0 ) ==== 8+0+27193 (crc 0 0 0) 0x7f8ccc001d30 con 0x7f8cbc07c7f0 2024-04-25T10:29:19.436 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:19.361+0000 7f1f1cf7f640 1 --2- 172.21.15.37:0/997756408 >> [v2:172.21.15.203:6832/2418205237,v1:172.21.15.203:6833/2418205237] conn(0x7f1efc0744b0 0x7f1efc0769c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:29:19.437 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:19.361+0000 7f1f157fa640 1 -- 172.21.15.37:0/997756408 --> [v2:172.21.15.37:6824/1815290618,v1:172.21.15.37:6825/1815290618] -- command(tid 1: {"prefix": "get_command_descriptions"}) -- 0x7f1f1806e520 con 0x7f1efc07c780 2024-04-25T10:29:19.437 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:19.361+0000 7f1f157fa640 1 -- 172.21.15.37:0/997756408 <== mon.1 v2:172.21.15.203:3300/0 6 ==== mon_get_version_reply(handle=1 version=23) ==== 24+0+0 (secure 0 0 0) 0x7f1f181329b0 con 0x7f1f100542c0 2024-04-25T10:29:19.437 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:19.361+0000 7f1f17fff640 1 --2- 172.21.15.37:0/997756408 >> [v2:172.21.15.37:6824/1815290618,v1:172.21.15.37:6825/1815290618] conn(0x7f1efc07c780 0x7f1efc07ebd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:29:19.437 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:19.361+0000 7f954b7fe640 1 -- 172.21.15.37:0/3561152926 <== mon.1 v2:172.21.15.203:3300/0 4 ==== mgrmap(e 5) ==== 96782+0+0 (secure 0 0 0) 0x7f9554044070 con 0x7f956406d060 2024-04-25T10:29:19.437 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:19.361+0000 7f37677fe640 1 --2- 172.21.15.37:0/2391780133 >> [v2:172.21.15.37:6800/1690341662,v1:172.21.15.37:6801/1690341662] conn(0x7f375407c6a0 0x7f375407eaf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:29:19.437 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:19.361+0000 7f2cf3fff640 1 -- 172.21.15.37:0/3107331413 <== osd.5 v2:172.21.15.203:6824/1203822890 1 ==== command_reply(tid 1: 0 ) ==== 8+0+27193 (crc 0 0 0) 0x7f2cec0801f0 con 0x7f2cec07d630 2024-04-25T10:29:19.438 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:19.361+0000 7f3767fff640 1 --2- 172.21.15.37:0/2391780133 >> [v2:172.21.15.203:6832/2418205237,v1:172.21.15.203:6833/2418205237] conn(0x7f37540744b0 0x7f37540769c0 secure :-1 s=READY pgs=76 cs=0 l=1 rev1=1 crypto rx=0x7f376815aa40 tx=0x7f3758000f50 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:29:19.438 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:19.361+0000 7f1f1cf7f640 1 --2- 172.21.15.37:0/997756408 >> [v2:172.21.15.203:6832/2418205237,v1:172.21.15.203:6833/2418205237] conn(0x7f1efc0744b0 0x7f1efc0769c0 secure :-1 s=READY pgs=77 cs=0 l=1 rev1=1 crypto rx=0x7f1f04009dc0 tx=0x7f1f040090e0 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:29:19.438 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:19.361+0000 7f1f17fff640 1 --2- 172.21.15.37:0/997756408 >> [v2:172.21.15.37:6824/1815290618,v1:172.21.15.37:6825/1815290618] conn(0x7f1efc07c780 0x7f1efc07ebd0 crc :-1 s=READY pgs=10 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.3 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:29:19.438 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:19.362+0000 7f37677fe640 1 --2- 172.21.15.37:0/2391780133 >> [v2:172.21.15.37:6800/1690341662,v1:172.21.15.37:6801/1690341662] conn(0x7f375407c6a0 0x7f375407eaf0 crc :-1 s=READY pgs=11 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:29:19.439 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:19.362+0000 7efde98d1640 1 --2- 172.21.15.37:0/2894599583 >> [v2:172.21.15.203:6810/2309615014,v1:172.21.15.203:6812/2309615014] conn(0x7efdc807c7f0 0x7efdc807ec40 crc :-1 s=READY pgs=9 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.6 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:29:19.439 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:19.362+0000 7fd90d7fa640 1 -- 172.21.15.37:0/1699500182 <== osd.0 v2:172.21.15.37:6816/2746074720 1 ==== command_reply(tid 1: 0 ) ==== 8+0+27193 (crc 0 0 0) 0x7fd910099e50 con 0x7fd8f407c850 2024-04-25T10:29:19.439 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:19.362+0000 7efde88cf640 1 --2- 172.21.15.37:0/2894599583 >> [v2:172.21.15.203:6832/2418205237,v1:172.21.15.203:6833/2418205237] conn(0x7efdc8074580 0x7efdc8076a90 secure :-1 s=READY pgs=78 cs=0 l=1 rev1=1 crypto rx=0x7efdd403d000 tx=0x7efdd4037000 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:29:19.440 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:19.361+0000 7f676d7fa640 1 -- 172.21.15.37:0/1129867855 <== mon.0 v2:172.21.15.37:3300/0 4 ==== mgrmap(e 5) ==== 96782+0+0 (secure 0 0 0) 0x7f676400fc70 con 0x7f6760003f10 2024-04-25T10:29:19.440 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:19.362+0000 7f676d7fa640 1 --2- 172.21.15.37:0/1129867855 >> [v2:172.21.15.203:6832/2418205237,v1:172.21.15.203:6833/2418205237] conn(0x7f6754074580 0x7f6754076a90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:29:19.440 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:19.362+0000 7f676d7fa640 1 -- 172.21.15.37:0/1129867855 <== mon.0 v2:172.21.15.37:3300/0 5 ==== osd_map(23..23 src has 1..23) ==== 5709+0+0 (secure 0 0 0) 0x7f6764095ce0 con 0x7f6760003f10 2024-04-25T10:29:19.440 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:19.362+0000 7f676d7fa640 1 --2- 172.21.15.37:0/1129867855 >> [v2:172.21.15.37:6808/2754669649,v1:172.21.15.37:6809/2754669649] conn(0x7f675407c850 0x7f675407eca0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:29:19.440 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:19.362+0000 7f676d7fa640 1 -- 172.21.15.37:0/1129867855 --> [v2:172.21.15.37:6808/2754669649,v1:172.21.15.37:6809/2754669649] -- command(tid 1: {"prefix": "get_command_descriptions"}) -- 0x7f6764001d30 con 0x7f675407c850 2024-04-25T10:29:19.440 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:19.362+0000 7f676d7fa640 1 -- 172.21.15.37:0/1129867855 <== mon.0 v2:172.21.15.37:3300/0 6 ==== mon_get_version_reply(handle=1 version=23) ==== 24+0+0 (secure 0 0 0) 0x7f67640960d0 con 0x7f6760003f10 2024-04-25T10:29:19.440 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:19.362+0000 7f1f157fa640 1 -- 172.21.15.37:0/997756408 <== osd.3 v2:172.21.15.37:6824/1815290618 1 ==== command_reply(tid 1: 0 ) ==== 8+0+27193 (crc 0 0 0) 0x7f1f1806e520 con 0x7f1efc07c780 2024-04-25T10:29:19.441 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:19.363+0000 7f676ffff640 1 --2- 172.21.15.37:0/1129867855 >> [v2:172.21.15.203:6832/2418205237,v1:172.21.15.203:6833/2418205237] conn(0x7f6754074580 0x7f6754076a90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:29:19.441 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:19.363+0000 7f677564e640 1 --2- 172.21.15.37:0/1129867855 >> [v2:172.21.15.37:6808/2754669649,v1:172.21.15.37:6809/2754669649] conn(0x7f675407c850 0x7f675407eca0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:29:19.441 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:19.363+0000 7f3764ff9640 1 -- 172.21.15.37:0/2391780133 <== osd.1 v2:172.21.15.37:6800/1690341662 1 ==== command_reply(tid 1: 0 ) ==== 8+0+27193 (crc 0 0 0) 0x7f375407f1c0 con 0x7f375407c6a0 2024-04-25T10:29:19.442 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:19.362+0000 7f954b7fe640 1 --2- 172.21.15.37:0/3561152926 >> [v2:172.21.15.203:6832/2418205237,v1:172.21.15.203:6833/2418205237] conn(0x7f95500748b0 0x7f9550076dc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:29:19.442 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:19.363+0000 7f954b7fe640 1 -- 172.21.15.37:0/3561152926 <== mon.1 v2:172.21.15.203:3300/0 5 ==== osd_map(23..23 src has 1..23) ==== 5709+0+0 (secure 0 0 0) 0x7f9554042070 con 0x7f956406d060 2024-04-25T10:29:19.442 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:19.363+0000 7f9562ffd640 1 --2- 172.21.15.37:0/3561152926 >> [v2:172.21.15.203:6832/2418205237,v1:172.21.15.203:6833/2418205237] conn(0x7f95500748b0 0x7f9550076dc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:29:19.442 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:19.363+0000 7f954b7fe640 1 --2- 172.21.15.37:0/3561152926 >> [v2:172.21.15.203:6808/3688099014,v1:172.21.15.203:6809/3688099014] conn(0x7f955007cb00 0x7f955007ef50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:29:19.442 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:19.363+0000 7f954b7fe640 1 -- 172.21.15.37:0/3561152926 --> [v2:172.21.15.203:6808/3688099014,v1:172.21.15.203:6809/3688099014] -- command(tid 1: {"prefix": "get_command_descriptions"}) -- 0x7f9554049e80 con 0x7f955007cb00 2024-04-25T10:29:19.442 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:19.363+0000 7f954b7fe640 1 -- 172.21.15.37:0/3561152926 <== mon.1 v2:172.21.15.203:3300/0 6 ==== mon_get_version_reply(handle=1 version=23) ==== 24+0+0 (secure 0 0 0) 0x7f95540b5c10 con 0x7f956406d060 2024-04-25T10:29:19.442 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:19.363+0000 7efdd9ffb640 1 -- 172.21.15.37:0/2894599583 <== osd.6 v2:172.21.15.203:6810/2309615014 1 ==== command_reply(tid 1: 0 ) ==== 8+0+27193 (crc 0 0 0) 0x7efde0014e90 con 0x7efdc807c7f0 2024-04-25T10:29:19.443 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:19.363+0000 7f95637fe640 1 --2- 172.21.15.37:0/3561152926 >> [v2:172.21.15.203:6808/3688099014,v1:172.21.15.203:6809/3688099014] conn(0x7f955007cb00 0x7f955007ef50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:29:19.443 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:19.363+0000 7f677564e640 1 --2- 172.21.15.37:0/1129867855 >> [v2:172.21.15.37:6808/2754669649,v1:172.21.15.37:6809/2754669649] conn(0x7f675407c850 0x7f675407eca0 crc :-1 s=READY pgs=10 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:29:19.443 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:19.364+0000 7f676ffff640 1 --2- 172.21.15.37:0/1129867855 >> [v2:172.21.15.203:6832/2418205237,v1:172.21.15.203:6833/2418205237] conn(0x7f6754074580 0x7f6754076a90 secure :-1 s=READY pgs=79 cs=0 l=1 rev1=1 crypto rx=0x7f675c033000 tx=0x7f675c02e000 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:29:19.443 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:19.364+0000 7f676d7fa640 1 -- 172.21.15.37:0/1129867855 <== osd.2 v2:172.21.15.37:6808/2754669649 1 ==== command_reply(tid 1: 0 ) ==== 8+0+27193 (crc 0 0 0) 0x7f6764001d30 con 0x7f675407c850 2024-04-25T10:29:19.444 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:19.365+0000 7f9562ffd640 1 --2- 172.21.15.37:0/3561152926 >> [v2:172.21.15.203:6832/2418205237,v1:172.21.15.203:6833/2418205237] conn(0x7f95500748b0 0x7f9550076dc0 secure :-1 s=READY pgs=80 cs=0 l=1 rev1=1 crypto rx=0x7f9558001600 tx=0x7f9558009040 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:29:19.444 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:19.365+0000 7f95637fe640 1 --2- 172.21.15.37:0/3561152926 >> [v2:172.21.15.203:6808/3688099014,v1:172.21.15.203:6809/3688099014] conn(0x7f955007cb00 0x7f955007ef50 crc :-1 s=READY pgs=10 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.7 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:29:19.444 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:19.366+0000 7f954b7fe640 1 -- 172.21.15.37:0/3561152926 <== osd.7 v2:172.21.15.203:6808/3688099014 1 ==== command_reply(tid 1: 0 ) ==== 8+0+27193 (crc 0 0 0) 0x7f9554049e80 con 0x7f955007cb00 2024-04-25T10:29:19.444 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:19.394+0000 7f8cc1ffb640 1 -- 172.21.15.37:0/3904300012 --> [v2:172.21.15.203:6800/4051532063,v1:172.21.15.203:6801/4051532063] -- command(tid 2: {"prefix": "flush_pg_stats"}) -- 0x7f8cb8002dd0 con 0x7f8cbc07c7f0 2024-04-25T10:29:19.444 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:19.395+0000 7f2d0e1ab640 1 -- 172.21.15.37:0/3107331413 --> [v2:172.21.15.203:6824/1203822890,v1:172.21.15.203:6825/1203822890] -- command(tid 2: {"prefix": "flush_pg_stats"}) -- 0x7f2d0004c440 con 0x7f2cec07d630 2024-04-25T10:29:19.445 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:19.395+0000 7f8cc3fff640 1 -- 172.21.15.37:0/3904300012 <== osd.4 v2:172.21.15.203:6800/4051532063 2 ==== command_reply(tid 2: 0 ) ==== 8+0+11 (crc 0 0 0) 0x7f8cb8002dd0 con 0x7f8cbc07c7f0 2024-04-25T10:29:19.445 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:19.395+0000 7fd915ca4640 1 -- 172.21.15.37:0/1699500182 --> [v2:172.21.15.37:6816/2746074720,v1:172.21.15.37:6817/2746074720] -- command(tid 2: {"prefix": "flush_pg_stats"}) -- 0x7fd90804bae0 con 0x7fd8f407c850 2024-04-25T10:29:19.445 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:19.395+0000 7f2cf3fff640 1 -- 172.21.15.37:0/3107331413 <== osd.5 v2:172.21.15.203:6824/1203822890 2 ==== command_reply(tid 2: 0 ) ==== 8+0+11 (crc 0 0 0) 0x7f2d0004c440 con 0x7f2cec07d630 2024-04-25T10:29:19.446 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:19.395+0000 7f8cc1ffb640 1 -- 172.21.15.37:0/3904300012 >> [v2:172.21.15.203:6800/4051532063,v1:172.21.15.203:6801/4051532063] conn(0x7f8cbc07c7f0 msgr2=0x7f8cbc07ec40 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:29:19.446 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:19.395+0000 7f8cc1ffb640 1 --2- 172.21.15.37:0/3904300012 >> [v2:172.21.15.203:6800/4051532063,v1:172.21.15.203:6801/4051532063] conn(0x7f8cbc07c7f0 0x7f8cbc07ec40 crc :-1 s=READY pgs=9 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:29:19.446 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:19.395+0000 7f2d0e1ab640 1 -- 172.21.15.37:0/3107331413 >> [v2:172.21.15.203:6824/1203822890,v1:172.21.15.203:6825/1203822890] conn(0x7f2cec07d630 msgr2=0x7f2cec07fb20 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:29:19.447 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:19.396+0000 7f8cc1ffb640 1 -- 172.21.15.37:0/3904300012 >> [v2:172.21.15.203:6832/2418205237,v1:172.21.15.203:6833/2418205237] conn(0x7f8cbc074580 msgr2=0x7f8cbc076a90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:29:19.447 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:19.396+0000 7f8cc1ffb640 1 --2- 172.21.15.37:0/3904300012 >> [v2:172.21.15.203:6832/2418205237,v1:172.21.15.203:6833/2418205237] conn(0x7f8cbc074580 0x7f8cbc076a90 secure :-1 s=READY pgs=74 cs=0 l=1 rev1=1 crypto rx=0x7f8cd00358b0 tx=0x7f8cc4009040 comp rx=0 tx=0).stop 2024-04-25T10:29:19.447 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:19.396+0000 7fd90d7fa640 1 -- 172.21.15.37:0/1699500182 <== osd.0 v2:172.21.15.37:6816/2746074720 2 ==== command_reply(tid 2: 0 ) ==== 8+0+11 (crc 0 0 0) 0x7fd90804bae0 con 0x7fd8f407c850 2024-04-25T10:29:19.448 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:19.396+0000 7f8cc1ffb640 1 -- 172.21.15.37:0/3904300012 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f8cd0047db0 msgr2=0x7f8cd00d91d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:29:19.448 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:19.396+0000 7f8cc1ffb640 1 --2- 172.21.15.37:0/3904300012 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f8cd0047db0 0x7f8cd00d91d0 secure :-1 s=READY pgs=108 cs=0 l=1 rev1=1 crypto rx=0x7f8ccc000c00 tx=0x7f8ccc0046c0 comp rx=0 tx=0).stop 2024-04-25T10:29:19.448 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:19.396+0000 7f8cd7fff640 1 -- 172.21.15.37:0/3904300012 reap_dead start 2024-04-25T10:29:19.448 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:19.396+0000 7f2d0e1ab640 1 --2- 172.21.15.37:0/3107331413 >> [v2:172.21.15.203:6824/1203822890,v1:172.21.15.203:6825/1203822890] conn(0x7f2cec07d630 0x7f2cec07fb20 crc :-1 s=READY pgs=10 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:29:19.449 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:19.396+0000 7f2d0e1ab640 1 -- 172.21.15.37:0/3107331413 >> [v2:172.21.15.203:6832/2418205237,v1:172.21.15.203:6833/2418205237] conn(0x7f2cec0748b0 msgr2=0x7f2cec076dc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:29:19.449 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:19.396+0000 7f2d0e1ab640 1 --2- 172.21.15.37:0/3107331413 >> [v2:172.21.15.203:6832/2418205237,v1:172.21.15.203:6833/2418205237] conn(0x7f2cec0748b0 0x7f2cec076dc0 secure :-1 s=READY pgs=75 cs=0 l=1 rev1=1 crypto rx=0x7f2cfc0016e0 tx=0x7f2cfc009040 comp rx=0 tx=0).stop 2024-04-25T10:29:19.449 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:19.396+0000 7f2d0e1ab640 1 -- 172.21.15.37:0/3107331413 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f2d00054780 msgr2=0x7f2d000defd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:29:19.449 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:19.396+0000 7f2d0e1ab640 1 --2- 172.21.15.37:0/3107331413 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f2d00054780 0x7f2d000defd0 secure :-1 s=READY pgs=31 cs=0 l=1 rev1=1 crypto rx=0x7f2cf400c440 tx=0x7f2cf400c900 comp rx=0 tx=0).stop 2024-04-25T10:29:19.449 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:19.396+0000 7f2d07fff640 1 -- 172.21.15.37:0/3107331413 reap_dead start 2024-04-25T10:29:19.449 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:19.396+0000 7f8cc1ffb640 1 -- 172.21.15.37:0/3904300012 shutdown_connections 2024-04-25T10:29:19.449 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:19.396+0000 7f8cc1ffb640 1 -- 172.21.15.37:0/3904300012 >> 172.21.15.37:0/3904300012 conn(0x7f8cd0054cf0 msgr2=0x7f8cd00367c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:29:19.450 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:19.396+0000 7f2d0e1ab640 1 -- 172.21.15.37:0/3107331413 shutdown_connections 2024-04-25T10:29:19.450 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:19.396+0000 7f2d0e1ab640 1 -- 172.21.15.37:0/3107331413 >> 172.21.15.37:0/3107331413 conn(0x7f2d00055e10 msgr2=0x7f2d00050ec0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:29:19.450 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:19.396+0000 7f1f1f1e3640 1 -- 172.21.15.37:0/997756408 --> [v2:172.21.15.37:6824/1815290618,v1:172.21.15.37:6825/1815290618] -- command(tid 2: {"prefix": "flush_pg_stats"}) -- 0x7f1f10031ad0 con 0x7f1efc07c780 2024-04-25T10:29:19.450 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:19.396+0000 7f2d0e1ab640 1 -- 172.21.15.37:0/3107331413 shutdown_connections 2024-04-25T10:29:19.450 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:19.396+0000 7f8cc1ffb640 1 -- 172.21.15.37:0/3904300012 shutdown_connections 2024-04-25T10:29:19.451 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:19.396+0000 7f2d0e1ab640 1 -- 172.21.15.37:0/3107331413 wait complete. 2024-04-25T10:29:19.451 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:19.396+0000 7f8cc1ffb640 1 -- 172.21.15.37:0/3904300012 wait complete. 2024-04-25T10:29:19.451 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:19.397+0000 7efdeb334640 1 -- 172.21.15.37:0/2894599583 --> [v2:172.21.15.203:6810/2309615014,v1:172.21.15.203:6812/2309615014] -- command(tid 2: {"prefix": "flush_pg_stats"}) -- 0x7efdc4002dd0 con 0x7efdc807c7f0 2024-04-25T10:29:19.452 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:19.396+0000 7fd915ca4640 1 -- 172.21.15.37:0/1699500182 >> [v2:172.21.15.37:6816/2746074720,v1:172.21.15.37:6817/2746074720] conn(0x7fd8f407c850 msgr2=0x7fd8f407eca0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:29:19.452 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:19.396+0000 7fd915ca4640 1 --2- 172.21.15.37:0/1699500182 >> [v2:172.21.15.37:6816/2746074720,v1:172.21.15.37:6817/2746074720] conn(0x7fd8f407c850 0x7fd8f407eca0 crc :-1 s=READY pgs=10 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:29:19.452 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:19.397+0000 7fd915ca4640 1 -- 172.21.15.37:0/1699500182 >> [v2:172.21.15.203:6832/2418205237,v1:172.21.15.203:6833/2418205237] conn(0x7fd8f4074580 msgr2=0x7fd8f4076a90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:29:19.452 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:19.397+0000 7fd915ca4640 1 --2- 172.21.15.37:0/1699500182 >> [v2:172.21.15.203:6832/2418205237,v1:172.21.15.203:6833/2418205237] conn(0x7fd8f4074580 0x7fd8f4076a90 secure :-1 s=READY pgs=73 cs=0 l=1 rev1=1 crypto rx=0x7fd908003fc0 tx=0x7fd904009110 comp rx=0 tx=0).stop 2024-04-25T10:29:19.452 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:19.397+0000 7fd915ca4640 1 -- 172.21.15.37:0/1699500182 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fd908034650 msgr2=0x7fd908034a60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:29:19.452 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:19.397+0000 7fd915ca4640 1 --2- 172.21.15.37:0/1699500182 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fd908034650 0x7fd908034a60 secure :-1 s=READY pgs=109 cs=0 l=1 rev1=1 crypto rx=0x7fd910066010 tx=0x7fd91006da90 comp rx=0 tx=0).stop 2024-04-25T10:29:19.452 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:19.397+0000 7fd9154a3640 1 -- 172.21.15.37:0/1699500182 reap_dead start 2024-04-25T10:29:19.452 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:19.397+0000 7f1f157fa640 1 -- 172.21.15.37:0/997756408 <== osd.3 v2:172.21.15.37:6824/1815290618 2 ==== command_reply(tid 2: 0 ) ==== 8+0+11 (crc 0 0 0) 0x7f1f10031ad0 con 0x7f1efc07c780 2024-04-25T10:29:19.453 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:19.397+0000 7f67770b1640 1 -- 172.21.15.37:0/1129867855 --> [v2:172.21.15.37:6808/2754669649,v1:172.21.15.37:6809/2754669649] -- command(tid 2: {"prefix": "flush_pg_stats"}) -- 0x7f6750002dd0 con 0x7f675407c850 2024-04-25T10:29:19.453 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:19.397+0000 7fd915ca4640 1 -- 172.21.15.37:0/1699500182 shutdown_connections 2024-04-25T10:29:19.453 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:19.397+0000 7fd915ca4640 1 -- 172.21.15.37:0/1699500182 >> 172.21.15.37:0/1699500182 conn(0x7fd908054cf0 msgr2=0x7fd908047a50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:29:19.453 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:19.397+0000 7efdd9ffb640 1 -- 172.21.15.37:0/2894599583 <== osd.6 v2:172.21.15.203:6810/2309615014 2 ==== command_reply(tid 2: 0 ) ==== 8+0+11 (crc 0 0 0) 0x7efdc4002dd0 con 0x7efdc807c7f0 2024-04-25T10:29:19.454 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:19.397+0000 7f376e50d640 1 -- 172.21.15.37:0/2391780133 --> [v2:172.21.15.37:6800/1690341662,v1:172.21.15.37:6801/1690341662] -- command(tid 2: {"prefix": "flush_pg_stats"}) -- 0x7f3768073b80 con 0x7f375407c6a0 2024-04-25T10:29:19.454 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:19.398+0000 7efdeb334640 1 -- 172.21.15.37:0/2894599583 >> [v2:172.21.15.203:6810/2309615014,v1:172.21.15.203:6812/2309615014] conn(0x7efdc807c7f0 msgr2=0x7efdc807ec40 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:29:19.455 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:19.398+0000 7f3764ff9640 1 -- 172.21.15.37:0/2391780133 <== osd.1 v2:172.21.15.37:6800/1690341662 2 ==== command_reply(tid 2: 0 ) ==== 8+0+11 (crc 0 0 0) 0x7f3768073b80 con 0x7f375407c6a0 2024-04-25T10:29:19.455 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:19.398+0000 7efdeb334640 1 --2- 172.21.15.37:0/2894599583 >> [v2:172.21.15.203:6810/2309615014,v1:172.21.15.203:6812/2309615014] conn(0x7efdc807c7f0 0x7efdc807ec40 crc :-1 s=READY pgs=9 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:29:19.455 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:19.398+0000 7f676d7fa640 1 -- 172.21.15.37:0/1129867855 <== osd.2 v2:172.21.15.37:6808/2754669649 2 ==== command_reply(tid 2: 0 ) ==== 8+0+11 (crc 0 0 0) 0x7f6750002dd0 con 0x7f675407c850 2024-04-25T10:29:19.456 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:19.398+0000 7f1f1f1e3640 1 -- 172.21.15.37:0/997756408 >> [v2:172.21.15.37:6824/1815290618,v1:172.21.15.37:6825/1815290618] conn(0x7f1efc07c780 msgr2=0x7f1efc07ebd0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:29:19.456 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:19.398+0000 7f1f1f1e3640 1 --2- 172.21.15.37:0/997756408 >> [v2:172.21.15.37:6824/1815290618,v1:172.21.15.37:6825/1815290618] conn(0x7f1efc07c780 0x7f1efc07ebd0 crc :-1 s=READY pgs=10 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:29:19.456 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:19.398+0000 7fd915ca4640 1 -- 172.21.15.37:0/1699500182 shutdown_connections 2024-04-25T10:29:19.456 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:19.398+0000 7fd915ca4640 1 -- 172.21.15.37:0/1699500182 wait complete. 2024-04-25T10:29:19.456 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:19.398+0000 7f1f1f1e3640 1 -- 172.21.15.37:0/997756408 >> [v2:172.21.15.203:6832/2418205237,v1:172.21.15.203:6833/2418205237] conn(0x7f1efc0744b0 msgr2=0x7f1efc0769c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:29:19.457 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:19.398+0000 7f1f1f1e3640 1 --2- 172.21.15.37:0/997756408 >> [v2:172.21.15.203:6832/2418205237,v1:172.21.15.203:6833/2418205237] conn(0x7f1efc0744b0 0x7f1efc0769c0 secure :-1 s=READY pgs=77 cs=0 l=1 rev1=1 crypto rx=0x7f1f04009dc0 tx=0x7f1f040090e0 comp rx=0 tx=0).stop 2024-04-25T10:29:19.457 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:19.398+0000 7f1f1f1e3640 1 -- 172.21.15.37:0/997756408 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f1f100542c0 msgr2=0x7f1f100cfff0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:29:19.457 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:19.398+0000 7f1f1f1e3640 1 --2- 172.21.15.37:0/997756408 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f1f100542c0 0x7f1f100cfff0 secure :-1 s=READY pgs=74 cs=0 l=1 rev1=1 crypto rx=0x7f1f1806af30 tx=0x7f1f1806a5a0 comp rx=0 tx=0).stop 2024-04-25T10:29:19.457 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:19.398+0000 7efdeb334640 1 -- 172.21.15.37:0/2894599583 >> [v2:172.21.15.203:6832/2418205237,v1:172.21.15.203:6833/2418205237] conn(0x7efdc8074580 msgr2=0x7efdc8076a90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:29:19.457 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:19.398+0000 7efdeb334640 1 --2- 172.21.15.37:0/2894599583 >> [v2:172.21.15.203:6832/2418205237,v1:172.21.15.203:6833/2418205237] conn(0x7efdc8074580 0x7efdc8076a90 secure :-1 s=READY pgs=78 cs=0 l=1 rev1=1 crypto rx=0x7efdd403d000 tx=0x7efdd4037000 comp rx=0 tx=0).stop 2024-04-25T10:29:19.458 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:19.398+0000 7efdeb334640 1 -- 172.21.15.37:0/2894599583 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7efddc052f40 msgr2=0x7efddc050ca0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:29:19.458 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:19.398+0000 7efdeb334640 1 --2- 172.21.15.37:0/2894599583 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7efddc052f40 0x7efddc050ca0 secure :-1 s=READY pgs=71 cs=0 l=1 rev1=1 crypto rx=0x7efde0007310 tx=0x7efde00077d0 comp rx=0 tx=0).stop 2024-04-25T10:29:19.458 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:19.398+0000 7efde98d1640 1 -- 172.21.15.37:0/2894599583 reap_dead start 2024-04-25T10:29:19.458 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:19.398+0000 7f1f1d780640 1 -- 172.21.15.37:0/997756408 reap_dead start 2024-04-25T10:29:19.459 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:19.398+0000 7f1f1f1e3640 1 -- 172.21.15.37:0/997756408 shutdown_connections 2024-04-25T10:29:19.459 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:19.398+0000 7f1f1f1e3640 1 -- 172.21.15.37:0/997756408 >> 172.21.15.37:0/997756408 conn(0x7f1f10055e00 msgr2=0x7f1f10036370 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:29:19.459 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:19.398+0000 7efdeb334640 1 -- 172.21.15.37:0/2894599583 shutdown_connections 2024-04-25T10:29:19.459 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:19.398+0000 7efdeb334640 1 -- 172.21.15.37:0/2894599583 >> 172.21.15.37:0/2894599583 conn(0x7efddc03ac50 msgr2=0x7efddc0543d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:29:19.459 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:19.398+0000 7f1f1f1e3640 1 -- 172.21.15.37:0/997756408 shutdown_connections 2024-04-25T10:29:19.459 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:19.398+0000 7f1f1f1e3640 1 -- 172.21.15.37:0/997756408 wait complete. 2024-04-25T10:29:19.460 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:19.398+0000 7efdeb334640 1 -- 172.21.15.37:0/2894599583 shutdown_connections 2024-04-25T10:29:19.460 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:19.399+0000 7efdeb334640 1 -- 172.21.15.37:0/2894599583 wait complete. 2024-04-25T10:29:19.460 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:19.399+0000 7f67770b1640 1 -- 172.21.15.37:0/1129867855 >> [v2:172.21.15.37:6808/2754669649,v1:172.21.15.37:6809/2754669649] conn(0x7f675407c850 msgr2=0x7f675407eca0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:29:19.460 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:19.399+0000 7f67770b1640 1 --2- 172.21.15.37:0/1129867855 >> [v2:172.21.15.37:6808/2754669649,v1:172.21.15.37:6809/2754669649] conn(0x7f675407c850 0x7f675407eca0 crc :-1 s=READY pgs=10 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:29:19.461 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:19.399+0000 7f376e50d640 1 -- 172.21.15.37:0/2391780133 >> [v2:172.21.15.37:6800/1690341662,v1:172.21.15.37:6801/1690341662] conn(0x7f375407c6a0 msgr2=0x7f375407eaf0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:29:19.461 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:19.399+0000 7f376e50d640 1 --2- 172.21.15.37:0/2391780133 >> [v2:172.21.15.37:6800/1690341662,v1:172.21.15.37:6801/1690341662] conn(0x7f375407c6a0 0x7f375407eaf0 crc :-1 s=READY pgs=11 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:29:19.461 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:19.399+0000 7f376e50d640 1 -- 172.21.15.37:0/2391780133 >> [v2:172.21.15.203:6832/2418205237,v1:172.21.15.203:6833/2418205237] conn(0x7f37540744b0 msgr2=0x7f37540769c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:29:19.461 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:19.399+0000 7f67770b1640 1 -- 172.21.15.37:0/1129867855 >> [v2:172.21.15.203:6832/2418205237,v1:172.21.15.203:6833/2418205237] conn(0x7f6754074580 msgr2=0x7f6754076a90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:29:19.461 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:19.399+0000 7f67770b1640 1 --2- 172.21.15.37:0/1129867855 >> [v2:172.21.15.203:6832/2418205237,v1:172.21.15.203:6833/2418205237] conn(0x7f6754074580 0x7f6754076a90 secure :-1 s=READY pgs=79 cs=0 l=1 rev1=1 crypto rx=0x7f675c033000 tx=0x7f675c02e000 comp rx=0 tx=0).stop 2024-04-25T10:29:19.461 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:19.399+0000 7f67770b1640 1 -- 172.21.15.37:0/1129867855 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f6760003f10 msgr2=0x7f67701534e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:29:19.462 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:19.399+0000 7f67770b1640 1 --2- 172.21.15.37:0/1129867855 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f6760003f10 0x7f67701534e0 secure :-1 s=READY pgs=110 cs=0 l=1 rev1=1 crypto rx=0x7f6764011890 tx=0x7f6764011c90 comp rx=0 tx=0).stop 2024-04-25T10:29:19.462 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:19.399+0000 7f376e50d640 1 --2- 172.21.15.37:0/2391780133 >> [v2:172.21.15.203:6832/2418205237,v1:172.21.15.203:6833/2418205237] conn(0x7f37540744b0 0x7f37540769c0 secure :-1 s=READY pgs=76 cs=0 l=1 rev1=1 crypto rx=0x7f376815aa40 tx=0x7f3758000f50 comp rx=0 tx=0).stop 2024-04-25T10:29:19.462 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:19.400+0000 7f376e50d640 1 -- 172.21.15.37:0/2391780133 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f3768167c20 msgr2=0x7f376815b6b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:29:19.462 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:19.400+0000 7f677564e640 1 -- 172.21.15.37:0/1129867855 reap_dead start 2024-04-25T10:29:19.462 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:19.400+0000 7f67770b1640 1 -- 172.21.15.37:0/1129867855 shutdown_connections 2024-04-25T10:29:19.463 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:19.400+0000 7f376e50d640 1 --2- 172.21.15.37:0/2391780133 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f3768167c20 0x7f376815b6b0 secure :-1 s=READY pgs=72 cs=0 l=1 rev1=1 crypto rx=0x7f37600045d0 tx=0x7f3760003a70 comp rx=0 tx=0).stop 2024-04-25T10:29:19.463 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:19.400+0000 7f67770b1640 1 -- 172.21.15.37:0/1129867855 >> 172.21.15.37:0/1129867855 conn(0x7f677006cba0 msgr2=0x7f677015dba0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:29:19.463 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:19.400+0000 7f67770b1640 1 -- 172.21.15.37:0/1129867855 shutdown_connections 2024-04-25T10:29:19.463 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:19.400+0000 7f67770b1640 1 -- 172.21.15.37:0/1129867855 wait complete. 2024-04-25T10:29:19.464 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:19.400+0000 7f9563fff640 1 -- 172.21.15.37:0/3561152926 --> [v2:172.21.15.203:6808/3688099014,v1:172.21.15.203:6809/3688099014] -- command(tid 2: {"prefix": "flush_pg_stats"}) -- 0x7f9544002dd0 con 0x7f955007cb00 2024-04-25T10:29:19.464 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:19.400+0000 7f376caaa640 1 -- 172.21.15.37:0/2391780133 reap_dead start 2024-04-25T10:29:19.464 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:19.400+0000 7f376e50d640 1 -- 172.21.15.37:0/2391780133 shutdown_connections 2024-04-25T10:29:19.464 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:19.400+0000 7f376e50d640 1 -- 172.21.15.37:0/2391780133 >> 172.21.15.37:0/2391780133 conn(0x7f376806cd90 msgr2=0x7f3768070920 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:29:19.464 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:19.401+0000 7f376e50d640 1 -- 172.21.15.37:0/2391780133 shutdown_connections 2024-04-25T10:29:19.464 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:19.401+0000 7f376e50d640 1 -- 172.21.15.37:0/2391780133 wait complete. 2024-04-25T10:29:19.465 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:19.401+0000 7f954b7fe640 1 -- 172.21.15.37:0/3561152926 <== osd.7 v2:172.21.15.203:6808/3688099014 2 ==== command_reply(tid 2: 0 ) ==== 8+0+11 (crc 0 0 0) 0x7f9544002dd0 con 0x7f955007cb00 2024-04-25T10:29:19.466 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:19.402+0000 7f9563fff640 1 -- 172.21.15.37:0/3561152926 >> [v2:172.21.15.203:6808/3688099014,v1:172.21.15.203:6809/3688099014] conn(0x7f955007cb00 msgr2=0x7f955007ef50 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:29:19.466 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:19.402+0000 7f9563fff640 1 --2- 172.21.15.37:0/3561152926 >> [v2:172.21.15.203:6808/3688099014,v1:172.21.15.203:6809/3688099014] conn(0x7f955007cb00 0x7f955007ef50 crc :-1 s=READY pgs=10 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:29:19.466 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:19.404+0000 7f9563fff640 1 -- 172.21.15.37:0/3561152926 >> [v2:172.21.15.203:6832/2418205237,v1:172.21.15.203:6833/2418205237] conn(0x7f95500748b0 msgr2=0x7f9550076dc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:29:19.466 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:19.404+0000 7f9563fff640 1 --2- 172.21.15.37:0/3561152926 >> [v2:172.21.15.203:6832/2418205237,v1:172.21.15.203:6833/2418205237] conn(0x7f95500748b0 0x7f9550076dc0 secure :-1 s=READY pgs=80 cs=0 l=1 rev1=1 crypto rx=0x7f9558001600 tx=0x7f9558009040 comp rx=0 tx=0).stop 2024-04-25T10:29:19.466 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:19.404+0000 7f9563fff640 1 -- 172.21.15.37:0/3561152926 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f956406d060 msgr2=0x7f9564148100 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:29:19.466 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:19.404+0000 7f9563fff640 1 --2- 172.21.15.37:0/3561152926 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f956406d060 0x7f9564148100 secure :-1 s=READY pgs=73 cs=0 l=1 rev1=1 crypto rx=0x7f9564072de0 tx=0x7f95540046c0 comp rx=0 tx=0).stop 2024-04-25T10:29:19.466 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:19.404+0000 7f95637fe640 1 -- 172.21.15.37:0/3561152926 reap_dead start 2024-04-25T10:29:19.466 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:19.406+0000 7f9563fff640 1 -- 172.21.15.37:0/3561152926 shutdown_connections 2024-04-25T10:29:19.466 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:19.406+0000 7f9563fff640 1 -- 172.21.15.37:0/3561152926 >> 172.21.15.37:0/3561152926 conn(0x7f9564073960 msgr2=0x7f9564154a40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:29:19.466 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:19.407+0000 7f9563fff640 1 -- 172.21.15.37:0/3561152926 shutdown_connections 2024-04-25T10:29:19.466 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:19.407+0000 7f9563fff640 1 -- 172.21.15.37:0/3561152926 wait complete. 2024-04-25T10:29:19.467 INFO:teuthology.orchestra.run.smithi037.stdout:64424509449 2024-04-25T10:29:19.467 DEBUG:teuthology.orchestra.run.smithi037:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.5 2024-04-25T10:29:19.468 INFO:teuthology.orchestra.run.smithi037.stdout:55834574856 2024-04-25T10:29:19.469 DEBUG:teuthology.orchestra.run.smithi037:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.4 2024-04-25T10:29:19.469 INFO:teuthology.orchestra.run.smithi037.stdout:64424509448 2024-04-25T10:29:19.470 DEBUG:teuthology.orchestra.run.smithi037:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.0 2024-04-25T10:29:19.470 INFO:teuthology.orchestra.run.smithi037.stdout:64424509449 2024-04-25T10:29:19.471 DEBUG:teuthology.orchestra.run.smithi037:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.2 2024-04-25T10:29:19.471 INFO:teuthology.orchestra.run.smithi037.stdout:64424509448 2024-04-25T10:29:19.472 DEBUG:teuthology.orchestra.run.smithi037:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.6 2024-04-25T10:29:19.472 INFO:teuthology.orchestra.run.smithi037.stdout:64424509449 2024-04-25T10:29:19.473 DEBUG:teuthology.orchestra.run.smithi037:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.3 2024-04-25T10:29:19.473 INFO:teuthology.orchestra.run.smithi037.stdout:64424509447 2024-04-25T10:29:19.474 DEBUG:teuthology.orchestra.run.smithi037:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.1 2024-04-25T10:29:19.475 INFO:teuthology.orchestra.run.smithi037.stdout:64424509447 2024-04-25T10:29:19.475 DEBUG:teuthology.orchestra.run.smithi037:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.7 2024-04-25T10:29:19.733 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:19.730+0000 7fb8e86d8640 1 Processor -- start 2024-04-25T10:29:19.734 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:19.732+0000 7f4dd31c5640 1 Processor -- start 2024-04-25T10:29:19.734 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:19.732+0000 7f4dd31c5640 1 -- start start 2024-04-25T10:29:19.734 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:19.731+0000 7fb8e86d8640 1 -- start start 2024-04-25T10:29:19.735 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:19.732+0000 7f60743b2640 1 Processor -- start 2024-04-25T10:29:19.735 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:19.732+0000 7f60743b2640 1 -- start start 2024-04-25T10:29:19.736 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:19.733+0000 7fb8e86d8640 1 --2- >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7fb8d8053e10 0x7fb8d80e3d10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:29:19.736 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:19.733+0000 7fb8e86d8640 1 --2- >> v2:172.21.15.37:3300/0 conn(0x7fb8d8034090 0x7fb8d80e0380 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:29:19.736 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:19.733+0000 7fb8e86d8640 1 -- --> v1:172.21.15.203:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fb8d8050490 con 0x7fb8d8055840 2024-04-25T10:29:19.736 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:19.733+0000 7fb8e86d8640 1 -- --> v2:172.21.15.37:3300/0 -- mon_getmap magic: 0 -- 0x7fb8d80e3880 con 0x7fb8d8034090 2024-04-25T10:29:19.736 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:19.733+0000 7fb8e86d8640 1 -- --> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] -- mon_getmap magic: 0 -- 0x7fb8d80e3550 con 0x7fb8d8053e10 2024-04-25T10:29:19.736 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:19.734+0000 7fb8e6c75640 1 --2- >> v2:172.21.15.37:3300/0 conn(0x7fb8d8034090 0x7fb8d80e0380 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:29:19.736 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:19.734+0000 7f4dd31c5640 1 --2- >> v2:172.21.15.203:3300/0 conn(0x7f4dcc0b7750 0x7f4dcc0b7b60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:29:19.737 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:19.734+0000 7f4dd31c5640 1 --2- >> v2:172.21.15.37:3300/0 conn(0x7f4dcc0d0d10 0x7f4dcc0d1120 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:29:19.737 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:19.734+0000 7f3affd72640 1 Processor -- start 2024-04-25T10:29:19.737 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:19.734+0000 7f4dd31c5640 1 -- --> v1:172.21.15.203:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f4dcc0b80a0 con 0x7f4dcc15aa20 2024-04-25T10:29:19.737 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:19.734+0000 7f4dd31c5640 1 -- --> v2:172.21.15.37:3300/0 -- mon_getmap magic: 0 -- 0x7f4dcc0d4b30 con 0x7f4dcc0d0d10 2024-04-25T10:29:19.738 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:19.734+0000 7fb8e6c75640 1 --2- >> v2:172.21.15.37:3300/0 conn(0x7fb8d8034090 0x7fb8d80e0380 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.37:3300/0 says I am v2:172.21.15.37:34882/0 (socket says 172.21.15.37:34882) 2024-04-25T10:29:19.738 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:19.734+0000 7f4dd31c5640 1 -- --> v2:172.21.15.203:3300/0 -- mon_getmap magic: 0 -- 0x7f4dcc0d4240 con 0x7f4dcc0b7750 2024-04-25T10:29:19.739 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:19.734+0000 7fb8e6c75640 1 -- 172.21.15.37:0/278573253 learned_addr learned my addr 172.21.15.37:0/278573253 (peer_addr_for_me v2:172.21.15.37:0/0) 2024-04-25T10:29:19.739 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:19.734+0000 7fb8e6474640 1 --2- >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7fb8d8053e10 0x7fb8d80e3d10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:29:19.739 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:19.734+0000 7fb8e5c73640 1 --1- >> v1:172.21.15.203:6789/0 conn(0x7fb8d8055840 0x7fb8d8033980 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.203:6789/0 says I am v1:172.21.15.37:50874/0 (socket says 172.21.15.37:50874) 2024-04-25T10:29:19.739 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:19.734+0000 7f4dd0f61640 1 --1- >> v1:172.21.15.203:6789/0 conn(0x7f4dcc15aa20 0x7f4dcc161a80 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.203:6789/0 says I am v1:172.21.15.37:50882/0 (socket says 172.21.15.37:50882) 2024-04-25T10:29:19.739 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:19.734+0000 7f4dd0f61640 1 -- 172.21.15.37:0/4105505245 learned_addr learned my addr 172.21.15.37:0/4105505245 (peer_addr_for_me v1:172.21.15.37:0/0) 2024-04-25T10:29:19.739 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:19.734+0000 7f9658984640 1 Processor -- start 2024-04-25T10:29:19.740 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:19.735+0000 7f4af0bb7640 1 Processor -- start 2024-04-25T10:29:19.740 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:19.735+0000 7f4dcbfff640 1 --2- 172.21.15.37:0/4105505245 >> v2:172.21.15.203:3300/0 conn(0x7f4dcc0b7750 0x7f4dcc0b7b60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:29:19.740 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:19.735+0000 7f4dd1762640 1 --2- 172.21.15.37:0/4105505245 >> v2:172.21.15.37:3300/0 conn(0x7f4dcc0d0d10 0x7f4dcc0d1120 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:29:19.741 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:19.735+0000 7f6939dc5640 1 Processor -- start 2024-04-25T10:29:19.741 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:19.736+0000 7f6939dc5640 1 -- start start 2024-04-25T10:29:19.741 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:19.735+0000 7f4dd1762640 1 -- 172.21.15.37:0/4105505245 >> v2:172.21.15.203:3300/0 conn(0x7f4dcc0b7750 msgr2=0x7f4dcc0b7b60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:29:19.741 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:19.735+0000 7f4dd1762640 1 --2- 172.21.15.37:0/4105505245 >> v2:172.21.15.203:3300/0 conn(0x7f4dcc0b7750 0x7f4dcc0b7b60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:29:19.742 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:19.735+0000 7f4dd1762640 1 -- 172.21.15.37:0/4105505245 >> v1:172.21.15.203:6789/0 conn(0x7f4dcc15aa20 legacy=0x7f4dcc161a80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:29:19.742 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:19.735+0000 7f4dcb7fe640 1 -- 172.21.15.37:0/4105505245 <== mon.1 v1:172.21.15.203:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 3858334392 0 0) 0x7f4dbc001110 con 0x7f4dcc15aa20 2024-04-25T10:29:19.742 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:19.735+0000 7f4dd1762640 1 -- 172.21.15.37:0/4105505245 --> v2:172.21.15.37:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f4dcc0d1a00 con 0x7f4dcc0d0d10 2024-04-25T10:29:19.742 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:19.736+0000 7f60743b2640 1 --2- >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f60640540c0 0x7f6064050540 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:29:19.742 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:19.735+0000 7f4af0bb7640 1 -- start start 2024-04-25T10:29:19.743 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:19.735+0000 7f3affd72640 1 -- start start 2024-04-25T10:29:19.743 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:19.736+0000 7f607194d640 1 --2- >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f60640540c0 0x7f6064050540 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:29:19.743 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:19.736+0000 7f607194d640 1 --2- >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f60640540c0 0x7f6064050540 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.203:3301/0 says I am v2:172.21.15.37:36700/0 (socket says 172.21.15.37:36700) 2024-04-25T10:29:19.744 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:19.736+0000 7f6073150640 1 --1- >> v1:172.21.15.37:6789/0 conn(0x7f60640500f0 0x7f6064100b30 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.37:6789/0 says I am v1:172.21.15.37:59726/0 (socket says 172.21.15.37:59726) 2024-04-25T10:29:19.744 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:19.736+0000 7f607194d640 1 -- 172.21.15.37:0/649036103 learned_addr learned my addr 172.21.15.37:0/649036103 (peer_addr_for_me v2:172.21.15.37:0/0) 2024-04-25T10:29:19.744 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:19.736+0000 7f60743b2640 1 -- 172.21.15.37:0/649036103 --> v1:172.21.15.37:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f6064102950 con 0x7f60640500f0 2024-04-25T10:29:19.744 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:19.736+0000 7f60743b2640 1 -- 172.21.15.37:0/649036103 --> v1:172.21.15.203:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f606404fa50 con 0x7f60640545c0 2024-04-25T10:29:19.744 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:19.736+0000 7f60743b2640 1 -- 172.21.15.37:0/649036103 --> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] -- mon_getmap magic: 0 -- 0x7f6064102420 con 0x7f60640540c0 2024-04-25T10:29:19.744 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:19.735+0000 7f9658984640 1 -- start start 2024-04-25T10:29:19.745 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:19.737+0000 7f4dd1762640 1 --2- 172.21.15.37:0/4105505245 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f4dcc0d0d10 0x7f4dcc0d1120 secure :-1 s=READY pgs=113 cs=0 l=1 rev1=1 crypto rx=0x7f4dcc08e370 tx=0x7f4dc0032c50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=1c58287b4e3344ed server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:29:19.745 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:19.736+0000 7f3e3b2ec640 1 Processor -- start 2024-04-25T10:29:19.745 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:19.737+0000 7f3e3b2ec640 1 -- start start 2024-04-25T10:29:19.745 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:19.737+0000 7f9658984640 1 --2- >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f96540c6f10 0x7f96540b86a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:29:19.745 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:19.737+0000 7f9658984640 1 -- --> v1:172.21.15.37:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f965406d650 con 0x7f96540b2a70 2024-04-25T10:29:19.745 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:19.737+0000 7f9658984640 1 -- --> v1:172.21.15.203:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f96540d1bf0 con 0x7f96540d1610 2024-04-25T10:29:19.745 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:19.737+0000 7f9658984640 1 -- --> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] -- mon_getmap magic: 0 -- 0x7f96540d1ed0 con 0x7f96540c6f10 2024-04-25T10:29:19.746 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:19.735+0000 7fb8e5472640 1 -- 172.21.15.37:0/278573253 <== mon.1 v1:172.21.15.203:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 2683927478 0 0) 0x7fb8d8050490 con 0x7fb8d8055840 2024-04-25T10:29:19.746 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:19.735+0000 7fb8e6c75640 1 -- 172.21.15.37:0/278573253 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7fb8d8053e10 msgr2=0x7fb8d80e3d10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:29:19.746 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:19.735+0000 7fb8e6c75640 1 --2- 172.21.15.37:0/278573253 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7fb8d8053e10 0x7fb8d80e3d10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:29:19.746 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:19.735+0000 7fb8e6c75640 1 -- 172.21.15.37:0/278573253 >> v1:172.21.15.203:6789/0 conn(0x7fb8d8055840 legacy=0x7fb8d8033980 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:29:19.746 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:19.735+0000 7fb8e6c75640 1 -- 172.21.15.37:0/278573253 --> v2:172.21.15.37:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fb8d8050730 con 0x7fb8d8034090 2024-04-25T10:29:19.746 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:19.736+0000 7fb8e6c75640 1 --2- 172.21.15.37:0/278573253 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fb8d8034090 0x7fb8d80e0380 secure :-1 s=READY pgs=111 cs=0 l=1 rev1=1 crypto rx=0x7fb8e002e0e0 tx=0x7fb8e00a71b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b37e6abb932b9201 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:29:19.746 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:19.737+0000 7fb8e5472640 1 -- 172.21.15.37:0/278573253 <== mon.0 v2:172.21.15.37:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb8e00b4070 con 0x7fb8d8034090 2024-04-25T10:29:19.746 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:19.737+0000 7fb8e5472640 1 -- 172.21.15.37:0/278573253 <== mon.0 v2:172.21.15.37:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fb8e00a7db0 con 0x7fb8d8034090 2024-04-25T10:29:19.746 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:19.737+0000 7fb8e5472640 1 -- 172.21.15.37:0/278573253 <== mon.0 v2:172.21.15.37:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb8e00b4070 con 0x7fb8d8034090 2024-04-25T10:29:19.747 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:19.737+0000 7f9651d9b640 1 --1- >> v1:172.21.15.37:6789/0 conn(0x7f96540b2a70 0x7f96540c6800 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.37:6789/0 says I am v1:172.21.15.37:59728/0 (socket says 172.21.15.37:59728) 2024-04-25T10:29:19.747 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:19.737+0000 7f9651d9b640 1 -- 172.21.15.37:0/3532182670 learned_addr learned my addr 172.21.15.37:0/3532182670 (peer_addr_for_me v1:172.21.15.37:0/0) 2024-04-25T10:29:19.747 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:19.737+0000 7f4af0bb7640 1 --2- >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f4aec14d8c0 0x7f4aec14ab00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:29:19.748 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:19.737+0000 7f607194d640 1 -- 172.21.15.37:0/649036103 >> v1:172.21.15.203:6789/0 conn(0x7f60640545c0 legacy=0x7f60640539b0 unknown :-1 s=STATE_CONNECTING_RE l=0).mark_down 2024-04-25T10:29:19.748 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:19.737+0000 7f607194d640 1 -- 172.21.15.37:0/649036103 >> v1:172.21.15.37:6789/0 conn(0x7f60640500f0 legacy=0x7f6064100b30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:29:19.748 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:19.737+0000 7f607194d640 1 -- 172.21.15.37:0/649036103 --> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f606404fd30 con 0x7f60640540c0 2024-04-25T10:29:19.748 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:19.737+0000 7f3affd72640 1 --2- >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f3af80b0ba0 0x7f3af806d7e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:29:19.748 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:19.738+0000 7f3affd72640 1 --2- >> v2:172.21.15.203:3300/0 conn(0x7f3af80d1db0 0x7f3af80d21c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:29:19.748 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:19.738+0000 7f3affd72640 1 -- --> v1:172.21.15.203:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f3af806dd20 con 0x7f3af809b700 2024-04-25T10:29:19.748 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:19.738+0000 7f3affd72640 1 -- --> v2:172.21.15.203:3300/0 -- mon_getmap magic: 0 -- 0x7f3af80b0710 con 0x7f3af80d1db0 2024-04-25T10:29:19.748 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:19.738+0000 7f3affd72640 1 -- --> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] -- mon_getmap magic: 0 -- 0x7f3af80d2700 con 0x7f3af80b0ba0 2024-04-25T10:29:19.749 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:19.738+0000 7f9652d9d640 1 --2- 172.21.15.37:0/3532182670 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f96540c6f10 0x7f96540b86a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:29:19.749 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:19.738+0000 7f6939dc5640 1 --2- >> v2:172.21.15.203:3300/0 conn(0x7f69340b3b40 0x7f69340b7e00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:29:19.749 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:19.738+0000 7f6939dc5640 1 --2- >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f69340ce6a0 0x7f69340b5930 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:29:19.749 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:19.738+0000 7f6939dc5640 1 -- --> v1:172.21.15.37:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f69340b4d40 con 0x7f69340b5520 2024-04-25T10:29:19.750 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:19.738+0000 7f6939dc5640 1 -- --> v2:172.21.15.203:3300/0 -- mon_getmap magic: 0 -- 0x7f69340b5020 con 0x7f69340b3b40 2024-04-25T10:29:19.750 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:19.738+0000 7f6939dc5640 1 -- --> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] -- mon_getmap magic: 0 -- 0x7f69340d12c0 con 0x7f69340ce6a0 2024-04-25T10:29:19.750 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:19.738+0000 7f965159a640 1 -- 172.21.15.37:0/3532182670 <== mon.0 v1:172.21.15.37:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 2234382160 0 0) 0x7f965406d650 con 0x7f96540b2a70 2024-04-25T10:29:19.750 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:19.738+0000 7f965159a640 1 -- 172.21.15.37:0/3532182670 --> v1:172.21.15.37:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f963c003520 con 0x7f96540b2a70 2024-04-25T10:29:19.750 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:19.738+0000 7f965159a640 1 -- 172.21.15.37:0/3532182670 <== mon.1 v1:172.21.15.203:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 1901364179 0 0) 0x7f96540d1bf0 con 0x7f96540d1610 2024-04-25T10:29:19.751 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:19.737+0000 7f4dcb7fe640 1 -- 172.21.15.37:0/4105505245 <== mon.0 v2:172.21.15.37:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4dc0040070 con 0x7f4dcc0d0d10 2024-04-25T10:29:19.751 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:19.737+0000 7f4dcb7fe640 1 -- 172.21.15.37:0/4105505245 <== mon.0 v2:172.21.15.37:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f4dc0033790 con 0x7f4dcc0d0d10 2024-04-25T10:29:19.751 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:19.738+0000 7f4dd31c5640 1 -- 172.21.15.37:0/4105505245 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f4dcc0d0d10 msgr2=0x7f4dcc0d1120 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:29:19.751 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:19.738+0000 7f4dd31c5640 1 --2- 172.21.15.37:0/4105505245 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f4dcc0d0d10 0x7f4dcc0d1120 secure :-1 s=READY pgs=113 cs=0 l=1 rev1=1 crypto rx=0x7f4dcc08e370 tx=0x7f4dc0032c50 comp rx=0 tx=0).stop 2024-04-25T10:29:19.751 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:19.738+0000 7f3afdb0e640 1 --2- >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f3af80b0ba0 0x7f3af806d7e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:29:19.751 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:19.738+0000 7f3afdb0e640 1 --2- >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f3af80b0ba0 0x7f3af806d7e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.203:3301/0 says I am v2:172.21.15.37:36742/0 (socket says 172.21.15.37:36742) 2024-04-25T10:29:19.751 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:19.738+0000 7f3afdb0e640 1 -- 172.21.15.37:0/1018670515 learned_addr learned my addr 172.21.15.37:0/1018670515 (peer_addr_for_me v2:172.21.15.37:0/0) 2024-04-25T10:29:19.752 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:19.737+0000 7f4af0bb7640 1 --2- >> v2:172.21.15.203:3300/0 conn(0x7f4aec0b4fd0 0x7f4aec0b53e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:29:19.752 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:19.737+0000 7f4af0bb7640 1 -- --> v1:172.21.15.203:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f4aec06f100 con 0x7f4aec0701a0 2024-04-25T10:29:19.752 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:19.737+0000 7f4af0bb7640 1 -- --> v2:172.21.15.203:3300/0 -- mon_getmap magic: 0 -- 0x7f4aec06f3e0 con 0x7f4aec0b4fd0 2024-04-25T10:29:19.752 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:19.737+0000 7f4af0bb7640 1 -- --> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] -- mon_getmap magic: 0 -- 0x7f4aec06f6c0 con 0x7f4aec14d8c0 2024-04-25T10:29:19.752 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:19.738+0000 7f6932ffd640 1 --1- >> v1:172.21.15.37:6789/0 conn(0x7f69340b5520 0x7f69340cdf90 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.37:6789/0 says I am v1:172.21.15.37:59744/0 (socket says 172.21.15.37:59744) 2024-04-25T10:29:19.752 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:19.738+0000 7f69337fe640 1 --2- >> v2:172.21.15.203:3300/0 conn(0x7f69340b3b40 0x7f69340b7e00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:29:19.753 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:19.738+0000 7f69337fe640 1 --2- >> v2:172.21.15.203:3300/0 conn(0x7f69340b3b40 0x7f69340b7e00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.203:3300/0 says I am v2:172.21.15.37:49442/0 (socket says 172.21.15.37:49442) 2024-04-25T10:29:19.753 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:19.738+0000 7f69337fe640 1 -- 172.21.15.37:0/1585103278 learned_addr learned my addr 172.21.15.37:0/1585103278 (peer_addr_for_me v2:172.21.15.37:0/0) 2024-04-25T10:29:19.753 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:19.738+0000 7f4aea59c640 1 --2- >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f4aec14d8c0 0x7f4aec14ab00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:29:19.753 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:19.738+0000 7f4aea59c640 1 --2- >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f4aec14d8c0 0x7f4aec14ab00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.203:3301/0 says I am v2:172.21.15.37:36730/0 (socket says 172.21.15.37:36730) 2024-04-25T10:29:19.753 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:19.738+0000 7f4aea59c640 1 -- 172.21.15.37:0/3407252501 learned_addr learned my addr 172.21.15.37:0/3407252501 (peer_addr_for_me v2:172.21.15.37:0/0) 2024-04-25T10:29:19.753 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:19.738+0000 7f3afe30f640 1 --2- 172.21.15.37:0/1018670515 >> v2:172.21.15.203:3300/0 conn(0x7f3af80d1db0 0x7f3af80d21c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:29:19.754 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:19.738+0000 7f6933fff640 1 --2- 172.21.15.37:0/1585103278 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f69340ce6a0 0x7f69340b5930 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:29:19.754 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:19.738+0000 7f4dd31c5640 1 -- 172.21.15.37:0/4105505245 shutdown_connections 2024-04-25T10:29:19.754 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:19.738+0000 7f4dd31c5640 1 --2- 172.21.15.37:0/4105505245 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f4dcc0d0d10 0x7f4dcc0d1120 unknown :-1 s=CLOSED pgs=113 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:29:19.754 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:19.738+0000 7f4dd31c5640 1 --2- 172.21.15.37:0/4105505245 >> v2:172.21.15.203:3300/0 conn(0x7f4dcc0b7750 0x7f4dcc0b7b60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:29:19.754 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:19.738+0000 7f4dd31c5640 1 -- 172.21.15.37:0/4105505245 >> 172.21.15.37:0/4105505245 conn(0x7f4dcc0d2750 msgr2=0x7f4dcc0d2b60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:29:19.755 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:19.738+0000 7f965159a640 1 -- 172.21.15.37:0/3532182670 --> v1:172.21.15.203:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f965406d650 con 0x7f96540d1610 2024-04-25T10:29:19.755 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:19.739+0000 7f4dd31c5640 1 -- 172.21.15.37:0/4105505245 shutdown_connections 2024-04-25T10:29:19.755 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:19.739+0000 7f4dd31c5640 1 -- 172.21.15.37:0/4105505245 wait complete. 2024-04-25T10:29:19.756 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:19.738+0000 7f965159a640 1 -- 172.21.15.37:0/3532182670 <== mon.0 v1:172.21.15.37:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 725886342 0 0) 0x7f963c003520 con 0x7f96540b2a70 2024-04-25T10:29:19.756 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:19.738+0000 7f965159a640 1 -- 172.21.15.37:0/3532182670 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f96540c6f10 msgr2=0x7f96540b86a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:29:19.756 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:19.739+0000 7f69337fe640 1 -- 172.21.15.37:0/1585103278 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f69340ce6a0 msgr2=0x7f69340b5930 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:29:19.756 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:19.739+0000 7f69337fe640 1 --2- 172.21.15.37:0/1585103278 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f69340ce6a0 0x7f69340b5930 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:29:19.756 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:19.739+0000 7f4ae959a640 1 -- 172.21.15.37:0/3407252501 <== mon.1 v1:172.21.15.203:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 2114994406 0 0) 0x7f4aec06f100 con 0x7f4aec0701a0 2024-04-25T10:29:19.757 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:19.739+0000 7f69337fe640 1 -- 172.21.15.37:0/1585103278 >> v1:172.21.15.37:6789/0 conn(0x7f69340b5520 legacy=0x7f69340cdf90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:29:19.757 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:19.738+0000 7f965159a640 1 --2- 172.21.15.37:0/3532182670 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f96540c6f10 0x7f96540b86a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:29:19.757 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:19.738+0000 7f965159a640 1 -- 172.21.15.37:0/3532182670 >> v1:172.21.15.203:6789/0 conn(0x7f96540d1610 legacy=0x7f96540b4bb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:29:19.757 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:19.738+0000 7f965159a640 1 -- 172.21.15.37:0/3532182670 --> v1:172.21.15.37:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f965406d910 con 0x7f96540b2a70 2024-04-25T10:29:19.757 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:19.738+0000 7f965159a640 1 -- 172.21.15.37:0/3532182670 <== mon.0 v1:172.21.15.37:6789/0 3 ==== mon_map magic: 0 ==== 413+0+0 (unknown 1577794600 0 0) 0x7f9648001670 con 0x7f96540b2a70 2024-04-25T10:29:19.757 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:19.739+0000 7fb8e86d8640 1 -- 172.21.15.37:0/278573253 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fb8d8034090 msgr2=0x7fb8d80e0380 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:29:19.757 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:19.739+0000 7fb8e86d8640 1 --2- 172.21.15.37:0/278573253 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fb8d8034090 0x7fb8d80e0380 secure :-1 s=READY pgs=111 cs=0 l=1 rev1=1 crypto rx=0x7fb8e002e0e0 tx=0x7fb8e00a71b0 comp rx=0 tx=0).stop 2024-04-25T10:29:19.758 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:19.738+0000 7f965159a640 1 -- 172.21.15.37:0/3532182670 >> v1:172.21.15.37:6789/0 conn(0x7f96540b2a70 legacy=0x7f96540c6800 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:29:19.758 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:19.739+0000 7f965159a640 1 --2- 172.21.15.37:0/3532182670 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f963c003b70 0x7f963c003fa0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:29:19.758 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:19.739+0000 7f965159a640 1 --2- 172.21.15.37:0/3532182670 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f963c004800 0x7f963c004c10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:29:19.758 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:19.739+0000 7f965159a640 1 --2- 172.21.15.37:0/3532182670 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f963c005150 0x7f963c00b9c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:29:19.758 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:19.739+0000 7f965159a640 1 -- 172.21.15.37:0/3532182670 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 -- 0x7f96540d1ed0 con 0x7f963c004800 2024-04-25T10:29:19.759 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:19.739+0000 7f4ae959a640 1 -- 172.21.15.37:0/3407252501 --> v1:172.21.15.203:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f4ad4003520 con 0x7f4aec0701a0 2024-04-25T10:29:19.759 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:19.739+0000 7f965159a640 1 -- 172.21.15.37:0/3532182670 --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_getmap magic: 0 -- 0x7f963c00c030 con 0x7f963c003b70 2024-04-25T10:29:19.759 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:19.739+0000 7f965159a640 1 -- 172.21.15.37:0/3532182670 --> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] -- mon_getmap magic: 0 -- 0x7f963c00c230 con 0x7f963c005150 2024-04-25T10:29:19.759 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:19.739+0000 7f9652d9d640 1 --2- 172.21.15.37:0/3532182670 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f963c005150 0x7f963c00b9c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:29:19.759 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:19.739+0000 7f9651d9b640 1 --2- 172.21.15.37:0/3532182670 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f963c004800 0x7f963c004c10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:29:19.760 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:19.739+0000 7f607194d640 1 --2- 172.21.15.37:0/649036103 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f60640540c0 0x7f6064050540 secure :-1 s=READY pgs=32 cs=0 l=1 rev1=1 crypto rx=0x7f605c0098a0 tx=0x7f605c02c260 comp rx=0 tx=0).ready entity=mon.2 client_cookie=234d807add267a6b server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:29:19.760 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:19.739+0000 7f3afdb0e640 1 -- 172.21.15.37:0/1018670515 >> v2:172.21.15.203:3300/0 conn(0x7f3af80d1db0 msgr2=0x7f3af80d21c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:29:19.760 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:19.739+0000 7f3afdb0e640 1 --2- 172.21.15.37:0/1018670515 >> v2:172.21.15.203:3300/0 conn(0x7f3af80d1db0 0x7f3af80d21c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:29:19.760 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:19.739+0000 7f3afdb0e640 1 -- 172.21.15.37:0/1018670515 >> v1:172.21.15.203:6789/0 conn(0x7f3af809b700 legacy=0x7f3af815c720 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:29:19.761 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:19.739+0000 7f3afdb0e640 1 -- 172.21.15.37:0/1018670515 --> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f3af80afde0 con 0x7f3af80b0ba0 2024-04-25T10:29:19.761 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:19.739+0000 7f3e3b2ec640 1 --2- >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f3e34164c40 0x7f3e34167060 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:29:19.761 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:19.739+0000 7fb8e86d8640 1 -- 172.21.15.37:0/278573253 shutdown_connections 2024-04-25T10:29:19.762 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:19.739+0000 7fb8e86d8640 1 --2- 172.21.15.37:0/278573253 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fb8d8034090 0x7fb8d80e0380 unknown :-1 s=CLOSED pgs=111 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:29:19.762 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:19.739+0000 7f965259c640 1 --2- 172.21.15.37:0/3532182670 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f963c003b70 0x7f963c003fa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:29:19.763 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:19.739+0000 7fb8e86d8640 1 --2- 172.21.15.37:0/278573253 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7fb8d8053e10 0x7fb8d80e3d10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:29:19.763 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:19.739+0000 7fb8e86d8640 1 -- 172.21.15.37:0/278573253 >> 172.21.15.37:0/278573253 conn(0x7fb8d8054d20 msgr2=0x7fb8d8055130 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:29:19.764 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:19.739+0000 7f3e39ae9640 1 --1- >> v1:172.21.15.37:6789/0 conn(0x7f3e34072ee0 0x7f3e340732f0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.37:6789/0 says I am v1:172.21.15.37:59760/0 (socket says 172.21.15.37:59760) 2024-04-25T10:29:19.765 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:19.739+0000 7f9651d9b640 1 -- 172.21.15.37:0/3532182670 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f963c005150 msgr2=0x7f963c00b9c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:29:19.765 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:19.739+0000 7f9651d9b640 1 --2- 172.21.15.37:0/3532182670 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f963c005150 0x7f963c00b9c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:29:19.765 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:19.740+0000 7f9651d9b640 1 -- 172.21.15.37:0/3532182670 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f963c003b70 msgr2=0x7f963c003fa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:29:19.765 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:19.739+0000 7f3e3b2ec640 1 -- --> v1:172.21.15.37:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f3e34070a00 con 0x7f3e34072ee0 2024-04-25T10:29:19.766 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:19.740+0000 7f3e3b2ec640 1 -- 172.21.15.37:0/2601886273 --> v1:172.21.15.203:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f3e340b53f0 con 0x7f3e340705f0 2024-04-25T10:29:19.766 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:19.740+0000 7f9651d9b640 1 --2- 172.21.15.37:0/3532182670 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f963c003b70 0x7f963c003fa0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:29:19.766 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:19.740+0000 7f3e3b2ec640 1 -- 172.21.15.37:0/2601886273 --> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] -- mon_getmap magic: 0 -- 0x7f3e3406ff20 con 0x7f3e34164c40 2024-04-25T10:29:19.767 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:19.740+0000 7f3e39ae9640 1 -- 172.21.15.37:0/2601886273 learned_addr learned my addr 172.21.15.37:0/2601886273 (peer_addr_for_me v1:172.21.15.37:0/0) 2024-04-25T10:29:19.767 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:19.740+0000 7f607114c640 1 -- 172.21.15.37:0/649036103 <== mon.2 v2:172.21.15.203:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f605c039070 con 0x7f60640540c0 2024-04-25T10:29:19.768 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:19.740+0000 7f9651d9b640 1 -- 172.21.15.37:0/3532182670 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f965406d910 con 0x7f963c004800 2024-04-25T10:29:19.768 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:19.740+0000 7f607114c640 1 -- 172.21.15.37:0/649036103 <== mon.2 v2:172.21.15.203:3301/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f605c034440 con 0x7f60640540c0 2024-04-25T10:29:19.768 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:19.740+0000 7f607114c640 1 -- 172.21.15.37:0/649036103 <== mon.2 v2:172.21.15.203:3301/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f605c034810 con 0x7f60640540c0 2024-04-25T10:29:19.769 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:19.739+0000 7f4aea59c640 1 -- 172.21.15.37:0/3407252501 >> v2:172.21.15.203:3300/0 conn(0x7f4aec0b4fd0 msgr2=0x7f4aec0b53e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:29:19.769 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:19.739+0000 7f4aea59c640 1 --2- 172.21.15.37:0/3407252501 >> v2:172.21.15.203:3300/0 conn(0x7f4aec0b4fd0 0x7f4aec0b53e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:29:19.769 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:19.739+0000 7f4aea59c640 1 -- 172.21.15.37:0/3407252501 >> v1:172.21.15.203:6789/0 conn(0x7f4aec0701a0 legacy=0x7f4aec168110 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:29:19.769 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:19.739+0000 7f4aea59c640 1 -- 172.21.15.37:0/3407252501 --> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f4aec06f9a0 con 0x7f4aec14d8c0 2024-04-25T10:29:19.770 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:19.740+0000 7f3e3aaeb640 1 --2- 172.21.15.37:0/2601886273 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f3e34164c40 0x7f3e34167060 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:29:19.770 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:19.740+0000 7f69337fe640 1 -- 172.21.15.37:0/1585103278 --> v2:172.21.15.203:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f69340d1630 con 0x7f69340b3b40 2024-04-25T10:29:19.771 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:19.740+0000 7fb8e86d8640 1 -- 172.21.15.37:0/278573253 shutdown_connections 2024-04-25T10:29:19.771 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:19.740+0000 7f3e3aaeb640 1 -- 172.21.15.37:0/2601886273 >> v1:172.21.15.203:6789/0 conn(0x7f3e340705f0 legacy=0x7f3e340d0e40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:29:19.771 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:19.740+0000 7f4aea59c640 1 --2- 172.21.15.37:0/3407252501 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f4aec14d8c0 0x7f4aec14ab00 secure :-1 s=READY pgs=33 cs=0 l=1 rev1=1 crypto rx=0x7f4ae0009870 tx=0x7f4ae002a210 comp rx=0 tx=0).ready entity=mon.2 client_cookie=de5d44cb012d3842 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:29:19.772 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:19.740+0000 7f3e3aaeb640 1 -- 172.21.15.37:0/2601886273 >> v1:172.21.15.37:6789/0 conn(0x7f3e34072ee0 legacy=0x7f3e340732f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:29:19.772 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:19.740+0000 7f3e392e8640 1 -- 172.21.15.37:0/2601886273 <== mon.0 v1:172.21.15.37:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 3907213731 0 0) 0x7f3e30001110 con 0x7f3e34072ee0 2024-04-25T10:29:19.772 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:19.740+0000 7f4ae959a640 1 -- 172.21.15.37:0/3407252501 <== mon.2 v2:172.21.15.203:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4ae0037070 con 0x7f4aec14d8c0 2024-04-25T10:29:19.772 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:19.741+0000 7fb8e86d8640 1 -- 172.21.15.37:0/278573253 wait complete. 2024-04-25T10:29:19.773 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:19.741+0000 7f9651d9b640 1 --2- 172.21.15.37:0/3532182670 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f963c004800 0x7f963c004c10 secure :-1 s=READY pgs=117 cs=0 l=1 rev1=1 crypto rx=0x7f964800c210 tx=0x7f963c005830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=7959e5f314a995f6 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:29:19.773 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:19.741+0000 7f69337fe640 1 --2- 172.21.15.37:0/1585103278 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f69340b3b40 0x7f69340b7e00 secure :-1 s=READY pgs=81 cs=0 l=1 rev1=1 crypto rx=0x7f6924009870 tx=0x7f692402a180 comp rx=0 tx=0).ready entity=mon.1 client_cookie=9ac394ecd1c0bdcd server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:29:19.773 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:19.740+0000 7f3e3aaeb640 1 -- 172.21.15.37:0/2601886273 --> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f3e34070c80 con 0x7f3e34164c40 2024-04-25T10:29:19.773 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:19.741+0000 7f60743b2640 1 -- 172.21.15.37:0/649036103 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f60640540c0 msgr2=0x7f6064050540 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:29:19.773 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:19.741+0000 7f60743b2640 1 --2- 172.21.15.37:0/649036103 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f60640540c0 0x7f6064050540 secure :-1 s=READY pgs=32 cs=0 l=1 rev1=1 crypto rx=0x7f605c0098a0 tx=0x7f605c02c260 comp rx=0 tx=0).stop 2024-04-25T10:29:19.774 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:19.741+0000 7f69327fc640 1 -- 172.21.15.37:0/1585103278 <== mon.1 v2:172.21.15.203:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6924037070 con 0x7f69340b3b40 2024-04-25T10:29:19.774 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:19.741+0000 7f69327fc640 1 -- 172.21.15.37:0/1585103278 <== mon.1 v2:172.21.15.203:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f6924032440 con 0x7f69340b3b40 2024-04-25T10:29:19.774 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:19.741+0000 7f69327fc640 1 -- 172.21.15.37:0/1585103278 <== mon.1 v2:172.21.15.203:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6924032810 con 0x7f69340b3b40 2024-04-25T10:29:19.774 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:19.741+0000 7f965159a640 1 -- 172.21.15.37:0/3532182670 <== mon.0 v2:172.21.15.37:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9648007d30 con 0x7f963c004800 2024-04-25T10:29:19.774 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:19.741+0000 7f965159a640 1 -- 172.21.15.37:0/3532182670 <== mon.0 v2:172.21.15.37:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f9648005ad0 con 0x7f963c004800 2024-04-25T10:29:19.774 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:19.741+0000 7f965159a640 1 -- 172.21.15.37:0/3532182670 <== mon.0 v2:172.21.15.37:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9648007070 con 0x7f963c004800 2024-04-25T10:29:19.775 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:19.741+0000 7f4ae959a640 1 -- 172.21.15.37:0/3407252501 <== mon.2 v2:172.21.15.203:3301/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f4ae0032440 con 0x7f4aec14d8c0 2024-04-25T10:29:19.775 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:19.741+0000 7f4ae959a640 1 -- 172.21.15.37:0/3407252501 <== mon.2 v2:172.21.15.203:3301/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4ae0032850 con 0x7f4aec14d8c0 2024-04-25T10:29:19.775 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:19.741+0000 7f3afdb0e640 1 --2- 172.21.15.37:0/1018670515 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f3af80b0ba0 0x7f3af806d7e0 secure :-1 s=READY pgs=34 cs=0 l=1 rev1=1 crypto rx=0x7f3af8074c50 tx=0x7f3af402c670 comp rx=0 tx=0).ready entity=mon.2 client_cookie=e40181682dab5c13 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:29:19.775 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:19.741+0000 7f3afcb0c640 1 -- 172.21.15.37:0/1018670515 <== mon.2 v2:172.21.15.203:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3af403a070 con 0x7f3af80b0ba0 2024-04-25T10:29:19.776 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:19.742+0000 7f3e3aaeb640 1 --2- 172.21.15.37:0/2601886273 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f3e34164c40 0x7f3e34167060 secure :-1 s=READY pgs=35 cs=0 l=1 rev1=1 crypto rx=0x7f3e2c00bfd0 tx=0x7f3e2c02a6d0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=5995a3bf780db658 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:29:19.776 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:19.742+0000 7f3afcb0c640 1 -- 172.21.15.37:0/1018670515 <== mon.2 v2:172.21.15.203:3301/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f3af402ce50 con 0x7f3af80b0ba0 2024-04-25T10:29:19.776 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:19.742+0000 7f4af0bb7640 1 -- 172.21.15.37:0/3407252501 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f4aec14d8c0 msgr2=0x7f4aec14ab00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:29:19.776 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:19.742+0000 7f4af0bb7640 1 --2- 172.21.15.37:0/3407252501 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f4aec14d8c0 0x7f4aec14ab00 secure :-1 s=READY pgs=33 cs=0 l=1 rev1=1 crypto rx=0x7f4ae0009870 tx=0x7f4ae002a210 comp rx=0 tx=0).stop 2024-04-25T10:29:19.776 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:19.742+0000 7f4af0bb7640 1 -- 172.21.15.37:0/3407252501 shutdown_connections 2024-04-25T10:29:19.777 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:19.742+0000 7f4af0bb7640 1 --2- 172.21.15.37:0/3407252501 >> v2:172.21.15.203:3300/0 conn(0x7f4aec0b4fd0 0x7f4aec0b53e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:29:19.777 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:19.742+0000 7f4af0bb7640 1 --2- 172.21.15.37:0/3407252501 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f4aec14d8c0 0x7f4aec14ab00 unknown :-1 s=CLOSED pgs=33 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:29:19.777 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:19.742+0000 7f4af0bb7640 1 -- 172.21.15.37:0/3407252501 >> 172.21.15.37:0/3407252501 conn(0x7f4aec0b36b0 msgr2=0x7f4aec0b3ac0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:29:19.777 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:19.742+0000 7f9658984640 1 -- 172.21.15.37:0/3532182670 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f963c004800 msgr2=0x7f963c004c10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:29:19.777 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:19.742+0000 7f6939dc5640 1 -- 172.21.15.37:0/1585103278 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f69340b3b40 msgr2=0x7f69340b7e00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:29:19.777 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:19.742+0000 7f6939dc5640 1 --2- 172.21.15.37:0/1585103278 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f69340b3b40 0x7f69340b7e00 secure :-1 s=READY pgs=81 cs=0 l=1 rev1=1 crypto rx=0x7f6924009870 tx=0x7f692402a180 comp rx=0 tx=0).stop 2024-04-25T10:29:19.778 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:19.742+0000 7f4af0bb7640 1 -- 172.21.15.37:0/3407252501 shutdown_connections 2024-04-25T10:29:19.778 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:19.742+0000 7f9658984640 1 --2- 172.21.15.37:0/3532182670 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f963c004800 0x7f963c004c10 secure :-1 s=READY pgs=117 cs=0 l=1 rev1=1 crypto rx=0x7f964800c210 tx=0x7f963c005830 comp rx=0 tx=0).stop 2024-04-25T10:29:19.778 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:19.742+0000 7f4af0bb7640 1 -- 172.21.15.37:0/3407252501 wait complete. 2024-04-25T10:29:19.779 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:19.742+0000 7f9652d9d640 1 -- 172.21.15.37:0/3532182670 reap_dead start 2024-04-25T10:29:19.779 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:19.742+0000 7f6939dc5640 1 -- 172.21.15.37:0/1585103278 shutdown_connections 2024-04-25T10:29:19.779 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:19.742+0000 7f6939dc5640 1 --2- 172.21.15.37:0/1585103278 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f69340ce6a0 0x7f69340b5930 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:29:19.779 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:19.742+0000 7f6939dc5640 1 --2- 172.21.15.37:0/1585103278 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f69340b3b40 0x7f69340b7e00 unknown :-1 s=CLOSED pgs=81 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:29:19.779 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:19.742+0000 7f6939dc5640 1 -- 172.21.15.37:0/1585103278 >> 172.21.15.37:0/1585103278 conn(0x7f69340b4220 msgr2=0x7f69340b4630 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:29:19.779 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:19.742+0000 7f6939dc5640 1 -- 172.21.15.37:0/1585103278 shutdown_connections 2024-04-25T10:29:19.780 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:19.743+0000 7f9658984640 1 -- 172.21.15.37:0/3532182670 shutdown_connections 2024-04-25T10:29:19.780 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:19.743+0000 7f9658984640 1 -- 172.21.15.37:0/3532182670 >> 172.21.15.37:0/3532182670 conn(0x7f96540d02b0 msgr2=0x7f96540d06c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:29:19.780 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:19.743+0000 7f3e392e8640 1 -- 172.21.15.37:0/2601886273 <== mon.2 v2:172.21.15.203:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3e2c038070 con 0x7f3e34164c40 2024-04-25T10:29:19.780 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:19.743+0000 7f3e392e8640 1 -- 172.21.15.37:0/2601886273 <== mon.2 v2:172.21.15.203:3301/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f3e2c030480 con 0x7f3e34164c40 2024-04-25T10:29:19.781 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:19.743+0000 7f9658984640 1 -- 172.21.15.37:0/3532182670 shutdown_connections 2024-04-25T10:29:19.781 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:19.743+0000 7f3e392e8640 1 -- 172.21.15.37:0/2601886273 <== mon.2 v2:172.21.15.203:3301/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3e2c030850 con 0x7f3e34164c40 2024-04-25T10:29:19.781 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:19.743+0000 7f60743b2640 1 -- 172.21.15.37:0/649036103 shutdown_connections 2024-04-25T10:29:19.781 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:19.743+0000 7f60743b2640 1 --2- 172.21.15.37:0/649036103 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f60640540c0 0x7f6064050540 unknown :-1 s=CLOSED pgs=32 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:29:19.781 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:19.743+0000 7f60743b2640 1 -- 172.21.15.37:0/649036103 >> 172.21.15.37:0/649036103 conn(0x7f6064054ed0 msgr2=0x7f60640552e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:29:19.782 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:19.743+0000 7f4dd31c5640 1 Processor -- start 2024-04-25T10:29:19.782 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:19.743+0000 7f3afcb0c640 1 -- 172.21.15.37:0/1018670515 <== mon.2 v2:172.21.15.203:3301/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3af40326a0 con 0x7f3af80b0ba0 2024-04-25T10:29:19.782 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:19.743+0000 7f3affd72640 1 -- 172.21.15.37:0/1018670515 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f3af80b0ba0 msgr2=0x7f3af806d7e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:29:19.782 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:19.743+0000 7f3affd72640 1 --2- 172.21.15.37:0/1018670515 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f3af80b0ba0 0x7f3af806d7e0 secure :-1 s=READY pgs=34 cs=0 l=1 rev1=1 crypto rx=0x7f3af8074c50 tx=0x7f3af402c670 comp rx=0 tx=0).stop 2024-04-25T10:29:19.782 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:19.743+0000 7f3affd72640 1 -- 172.21.15.37:0/1018670515 shutdown_connections 2024-04-25T10:29:19.782 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:19.743+0000 7f3affd72640 1 --2- 172.21.15.37:0/1018670515 >> v2:172.21.15.203:3300/0 conn(0x7f3af80d1db0 0x7f3af80d21c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:29:19.783 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:19.743+0000 7f3affd72640 1 --2- 172.21.15.37:0/1018670515 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f3af80b0ba0 0x7f3af806d7e0 unknown :-1 s=CLOSED pgs=34 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:29:19.783 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:19.743+0000 7f3affd72640 1 -- 172.21.15.37:0/1018670515 >> 172.21.15.37:0/1018670515 conn(0x7f3af8073ba0 msgr2=0x7f3af80b7530 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:29:19.783 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:19.743+0000 7fb8e86d8640 1 Processor -- start 2024-04-25T10:29:19.783 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:19.743+0000 7f6939dc5640 1 -- 172.21.15.37:0/1585103278 wait complete. 2024-04-25T10:29:19.783 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:19.743+0000 7f60743b2640 1 -- 172.21.15.37:0/649036103 shutdown_connections 2024-04-25T10:29:19.784 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:19.743+0000 7f60743b2640 1 -- 172.21.15.37:0/649036103 wait complete. 2024-04-25T10:29:19.784 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:19.743+0000 7f3affd72640 1 -- 172.21.15.37:0/1018670515 shutdown_connections 2024-04-25T10:29:19.784 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:19.743+0000 7f3affd72640 1 -- 172.21.15.37:0/1018670515 wait complete. 2024-04-25T10:29:19.784 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:19.743+0000 7fb8e86d8640 1 -- start start 2024-04-25T10:29:19.784 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:19.743+0000 7f9658984640 1 -- 172.21.15.37:0/3532182670 wait complete. 2024-04-25T10:29:19.785 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:19.744+0000 7f3e3b2ec640 1 -- 172.21.15.37:0/2601886273 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f3e34164c40 msgr2=0x7f3e34167060 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:29:19.785 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:19.744+0000 7f3e3b2ec640 1 --2- 172.21.15.37:0/2601886273 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f3e34164c40 0x7f3e34167060 secure :-1 s=READY pgs=35 cs=0 l=1 rev1=1 crypto rx=0x7f3e2c00bfd0 tx=0x7f3e2c02a6d0 comp rx=0 tx=0).stop 2024-04-25T10:29:19.785 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:19.744+0000 7f4dd31c5640 1 -- start start 2024-04-25T10:29:19.786 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:19.745+0000 7f4af0bb7640 1 Processor -- start 2024-04-25T10:29:19.786 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:19.745+0000 7f4af0bb7640 1 -- start start 2024-04-25T10:29:19.786 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:19.746+0000 7fb8e86d8640 1 --2- >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7fb8d8053e10 0x7fb8d8033dc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:29:19.786 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:19.746+0000 7fb8e86d8640 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fb8d8055840 0x7fb8d80d5370 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:29:19.786 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:19.746+0000 7f6939dc5640 1 Processor -- start 2024-04-25T10:29:19.787 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:19.746+0000 7fb8e86d8640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fb8d80d58b0 0x7fb8d80399f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:29:19.787 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:19.746+0000 7f9658984640 1 Processor -- start 2024-04-25T10:29:19.787 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:19.746+0000 7fb8e6c75640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fb8d80d58b0 0x7fb8d80399f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:29:19.787 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:19.746+0000 7fb8e6c75640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fb8d80d58b0 0x7fb8d80399f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.37:3300/0 says I am v2:172.21.15.37:34900/0 (socket says 172.21.15.37:34900) 2024-04-25T10:29:19.788 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:19.746+0000 7fb8e5c73640 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fb8d8055840 0x7fb8d80d5370 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:29:19.788 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:19.746+0000 7fb8e6c75640 1 -- 172.21.15.37:0/756336885 learned_addr learned my addr 172.21.15.37:0/756336885 (peer_addr_for_me v2:172.21.15.37:0/0) 2024-04-25T10:29:19.788 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:19.746+0000 7f3affd72640 1 Processor -- start 2024-04-25T10:29:19.788 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:19.746+0000 7f3affd72640 1 -- start start 2024-04-25T10:29:19.788 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:19.746+0000 7fb8e86d8640 1 -- 172.21.15.37:0/756336885 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 -- 0x7fb8d8050cf0 con 0x7fb8d80d58b0 2024-04-25T10:29:19.788 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:19.746+0000 7fb8e86d8640 1 -- 172.21.15.37:0/756336885 --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_getmap magic: 0 -- 0x7fb8d80e4250 con 0x7fb8d8055840 2024-04-25T10:29:19.789 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:19.746+0000 7fb8e86d8640 1 -- 172.21.15.37:0/756336885 --> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] -- mon_getmap magic: 0 -- 0x7fb8d80e08c0 con 0x7fb8d8053e10 2024-04-25T10:29:19.789 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:19.746+0000 7f9658984640 1 -- start start 2024-04-25T10:29:19.789 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:19.747+0000 7f60743b2640 1 Processor -- start 2024-04-25T10:29:19.789 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:19.747+0000 7f60743b2640 1 -- start start 2024-04-25T10:29:19.790 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:19.747+0000 7f4dd31c5640 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f4dcc0b7750 0x7f4dcc154d10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:29:19.790 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:19.747+0000 7f4dd31c5640 1 --2- >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f4dcc0d0d10 0x7f4dcc0ce9f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:29:19.790 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:19.747+0000 7f4af0bb7640 1 --2- >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f4aec0701a0 0x7f4aec1474b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:29:19.790 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:19.747+0000 7f4dd0f61640 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f4dcc0b7750 0x7f4dcc154d10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:29:19.791 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:19.747+0000 7fb8e6c75640 1 -- 172.21.15.37:0/756336885 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7fb8d8053e10 msgr2=0x7fb8d8033dc0 unknown :-1 s=STATE_CONNECTING_RE l=1).mark_down 2024-04-25T10:29:19.791 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:19.747+0000 7fb8e6c75640 1 --2- 172.21.15.37:0/756336885 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7fb8d8053e10 0x7fb8d8033dc0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:29:19.791 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:19.747+0000 7fb8e6c75640 1 -- 172.21.15.37:0/756336885 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fb8d8055840 msgr2=0x7fb8d80d5370 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:29:19.791 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:19.747+0000 7fb8e6c75640 1 --2- 172.21.15.37:0/756336885 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fb8d8055840 0x7fb8d80d5370 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:29:19.791 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:19.747+0000 7fb8e6c75640 1 -- 172.21.15.37:0/756336885 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fb8d80d60e0 con 0x7fb8d80d58b0 2024-04-25T10:29:19.791 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:19.747+0000 7f4af0bb7640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f4aec0b4fd0 0x7f4aec161f90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:29:19.791 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:19.747+0000 7f4dcbfff640 1 --2- >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f4dcc0d0d10 0x7f4dcc0ce9f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:29:19.792 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:19.748+0000 7f4af0bb7640 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f4aec14d8c0 0x7f4aec14bb50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:29:19.792 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:19.747+0000 7f4dcbfff640 1 --2- >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f4dcc0d0d10 0x7f4dcc0ce9f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.203:3301/0 says I am v2:172.21.15.37:36772/0 (socket says 172.21.15.37:36772) 2024-04-25T10:29:19.792 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:19.748+0000 7f4ae9d9b640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f4aec0b4fd0 0x7f4aec161f90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:29:19.792 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:19.748+0000 7f4dd0f61640 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f4dcc0b7750 0x7f4dcc154d10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.203:3300/0 says I am v2:172.21.15.37:49482/0 (socket says 172.21.15.37:49482) 2024-04-25T10:29:19.793 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:19.748+0000 7f4dcbfff640 1 -- 172.21.15.37:0/2431781660 learned_addr learned my addr 172.21.15.37:0/2431781660 (peer_addr_for_me v2:172.21.15.37:0/0) 2024-04-25T10:29:19.793 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:19.745+0000 7f3e3b2ec640 1 -- 172.21.15.37:0/2601886273 shutdown_connections 2024-04-25T10:29:19.793 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:19.745+0000 7f3e3b2ec640 1 --2- 172.21.15.37:0/2601886273 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f3e34164c40 0x7f3e34167060 unknown :-1 s=CLOSED pgs=35 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:29:19.793 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:19.745+0000 7f3e3b2ec640 1 -- 172.21.15.37:0/2601886273 >> 172.21.15.37:0/2601886273 conn(0x7f3e3406cb70 msgr2=0x7f3e340b3290 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:29:19.793 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:19.745+0000 7f3e3b2ec640 1 -- 172.21.15.37:0/2601886273 shutdown_connections 2024-04-25T10:29:19.793 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:19.745+0000 7f3e3b2ec640 1 -- 172.21.15.37:0/2601886273 wait complete. 2024-04-25T10:29:19.793 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:19.748+0000 7f4af0bb7640 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 -- 0x7f4aec0705b0 con 0x7f4aec0b4fd0 2024-04-25T10:29:19.793 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:19.748+0000 7f4af0bb7640 1 -- --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_getmap magic: 0 -- 0x7f4aec14e0a0 con 0x7f4aec14d8c0 2024-04-25T10:29:19.794 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:19.748+0000 7f4af0bb7640 1 -- --> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] -- mon_getmap magic: 0 -- 0x7f4aec070730 con 0x7f4aec0701a0 2024-04-25T10:29:19.794 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:19.748+0000 7f4ae9d9b640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f4aec0b4fd0 0x7f4aec161f90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.37:3300/0 says I am v2:172.21.15.37:34904/0 (socket says 172.21.15.37:34904) 2024-04-25T10:29:19.794 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:19.748+0000 7f4ae9d9b640 1 -- 172.21.15.37:0/3186560673 learned_addr learned my addr 172.21.15.37:0/3186560673 (peer_addr_for_me v2:172.21.15.37:0/0) 2024-04-25T10:29:19.794 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:19.748+0000 7f4dd31c5640 1 --2- 172.21.15.37:0/2431781660 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f4dcc15aa20 0x7f4dcc0cef30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:29:19.794 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:19.748+0000 7fb8e6c75640 1 --2- 172.21.15.37:0/756336885 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fb8d80d58b0 0x7fb8d80399f0 secure :-1 s=READY pgs=118 cs=0 l=1 rev1=1 crypto rx=0x7fb8e006b610 tx=0x7fb8e00759d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:29:19.795 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:19.748+0000 7f4dd1762640 1 --2- 172.21.15.37:0/2431781660 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f4dcc15aa20 0x7f4dcc0cef30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:29:19.795 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:19.748+0000 7f3e3b2ec640 1 Processor -- start 2024-04-25T10:29:19.795 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:19.748+0000 7f4dd31c5640 1 -- 172.21.15.37:0/2431781660 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 -- 0x7f4dcc0d1660 con 0x7f4dcc15aa20 2024-04-25T10:29:19.796 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:19.748+0000 7fb8d6ffd640 1 -- 172.21.15.37:0/756336885 <== mon.0 v2:172.21.15.37:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb8e00b4070 con 0x7fb8d80d58b0 2024-04-25T10:29:19.796 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:19.748+0000 7fb8d6ffd640 1 -- 172.21.15.37:0/756336885 <== mon.0 v2:172.21.15.37:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fb8e0068e40 con 0x7fb8d80d58b0 2024-04-25T10:29:19.796 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:19.748+0000 7fb8d6ffd640 1 -- 172.21.15.37:0/756336885 <== mon.0 v2:172.21.15.37:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb8e00af640 con 0x7fb8d80d58b0 2024-04-25T10:29:19.796 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:19.748+0000 7f3e3b2ec640 1 -- start start 2024-04-25T10:29:19.796 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:19.748+0000 7f4dd31c5640 1 -- 172.21.15.37:0/2431781660 --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_getmap magic: 0 -- 0x7f4dcc0d47c0 con 0x7f4dcc0b7750 2024-04-25T10:29:19.796 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:19.748+0000 7f4dd31c5640 1 -- 172.21.15.37:0/2431781660 --> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] -- mon_getmap magic: 0 -- 0x7f4dcc0d17e0 con 0x7f4dcc0d0d10 2024-04-25T10:29:19.797 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:19.748+0000 7fb8e86d8640 1 -- 172.21.15.37:0/756336885 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fb8d80d63c0 con 0x7fb8d80d58b0 2024-04-25T10:29:19.797 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:19.748+0000 7fb8e86d8640 1 -- 172.21.15.37:0/756336885 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fb8d80d6980 con 0x7fb8d80d58b0 2024-04-25T10:29:19.797 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:19.748+0000 7f4ae9d9b640 1 -- 172.21.15.37:0/3186560673 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f4aec0701a0 msgr2=0x7f4aec1474b0 unknown :-1 s=STATE_CONNECTING_RE l=1).mark_down 2024-04-25T10:29:19.797 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:19.748+0000 7f4ae9d9b640 1 --2- 172.21.15.37:0/3186560673 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f4aec0701a0 0x7f4aec1474b0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:29:19.797 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:19.748+0000 7f4ae9d9b640 1 -- 172.21.15.37:0/3186560673 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f4aec14d8c0 msgr2=0x7f4aec14bb50 unknown :-1 s=STATE_CONNECTING_RE l=1).mark_down 2024-04-25T10:29:19.798 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:19.748+0000 7f4ae9d9b640 1 --2- 172.21.15.37:0/3186560673 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f4aec14d8c0 0x7f4aec14bb50 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:29:19.798 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:19.748+0000 7f4ae9d9b640 1 -- 172.21.15.37:0/3186560673 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f4aec1624d0 con 0x7f4aec0b4fd0 2024-04-25T10:29:19.798 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:19.749+0000 7f4ae9d9b640 1 --2- 172.21.15.37:0/3186560673 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f4aec0b4fd0 0x7f4aec161f90 secure :-1 s=READY pgs=119 cs=0 l=1 rev1=1 crypto rx=0x7f4adc00b440 tx=0x7f4adc009f90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:29:19.798 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:19.746+0000 7f6939dc5640 1 -- start start 2024-04-25T10:29:19.798 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:19.749+0000 7f9658984640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f963c004800 0x7f96540b1d30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:29:19.799 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:19.749+0000 7f3affd72640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f3af809b700 0x7f3af8149370 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:29:19.799 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:19.749+0000 7f9658984640 1 --2- >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f96540c7ae0 0x7f96540c9940 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:29:19.799 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:19.749+0000 7f9658984640 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f96540cb500 0x7f96540cb910 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:29:19.799 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:19.749+0000 7f60743b2640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f60640500f0 0x7f606404d760 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:29:19.800 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:19.749+0000 7f9658984640 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 -- 0x7f96540c6f10 con 0x7f963c004800 2024-04-25T10:29:19.800 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:19.749+0000 7f60743b2640 1 --2- >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f60640545c0 0x7f6064199280 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:29:19.800 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:19.749+0000 7f9652d9d640 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f96540cb500 0x7f96540cb910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:29:19.801 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:19.749+0000 7f9658984640 1 -- --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_getmap magic: 0 -- 0x7f96540b2e80 con 0x7f96540cb500 2024-04-25T10:29:19.801 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:19.749+0000 7f607214e640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f60640500f0 0x7f606404d760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:29:19.802 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:19.750+0000 7f9652d9d640 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f96540cb500 0x7f96540cb910 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.203:3300/0 says I am v2:172.21.15.37:49490/0 (socket says 172.21.15.37:49490) 2024-04-25T10:29:19.802 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:19.750+0000 7f9652d9d640 1 -- 172.21.15.37:0/322652459 learned_addr learned my addr 172.21.15.37:0/322652459 (peer_addr_for_me v2:172.21.15.37:0/0) 2024-04-25T10:29:19.802 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:19.750+0000 7f9658984640 1 -- 172.21.15.37:0/322652459 --> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] -- mon_getmap magic: 0 -- 0x7f96540c7090 con 0x7f96540c7ae0 2024-04-25T10:29:19.802 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:19.749+0000 7f4dd1762640 1 -- 172.21.15.37:0/2431781660 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f4dcc0d0d10 msgr2=0x7f4dcc0ce9f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:29:19.802 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:19.749+0000 7f4dd1762640 1 --2- 172.21.15.37:0/2431781660 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f4dcc0d0d10 0x7f4dcc0ce9f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:29:19.803 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:19.750+0000 7f607214e640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f60640500f0 0x7f606404d760 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.37:3300/0 says I am v2:172.21.15.37:34924/0 (socket says 172.21.15.37:34924) 2024-04-25T10:29:19.803 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:19.749+0000 7f4dd1762640 1 -- 172.21.15.37:0/2431781660 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f4dcc0b7750 msgr2=0x7f4dcc154d10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:29:19.803 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:19.750+0000 7f607214e640 1 -- 172.21.15.37:0/92992236 learned_addr learned my addr 172.21.15.37:0/92992236 (peer_addr_for_me v2:172.21.15.37:0/0) 2024-04-25T10:29:19.803 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:19.750+0000 7f607194d640 1 --2- 172.21.15.37:0/92992236 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f60640545c0 0x7f6064199280 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:29:19.804 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:19.750+0000 7f9651d9b640 1 --2- 172.21.15.37:0/322652459 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f96540c7ae0 0x7f96540c9940 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:29:19.804 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:19.750+0000 7f60743b2640 1 --2- 172.21.15.37:0/92992236 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f606419be20 0x7f6064103b30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:29:19.804 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:19.750+0000 7f6073150640 1 --2- 172.21.15.37:0/92992236 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f606419be20 0x7f6064103b30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:29:19.805 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:19.750+0000 7f9652d9d640 1 -- 172.21.15.37:0/322652459 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f96540c7ae0 msgr2=0x7f96540c9940 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:29:19.805 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:19.750+0000 7f60743b2640 1 -- 172.21.15.37:0/92992236 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 -- 0x7f6064034510 con 0x7f60640500f0 2024-04-25T10:29:19.805 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:19.750+0000 7f9652d9d640 1 --2- 172.21.15.37:0/322652459 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f96540c7ae0 0x7f96540c9940 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:29:19.805 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:19.750+0000 7f9652d9d640 1 -- 172.21.15.37:0/322652459 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f963c004800 msgr2=0x7f96540b1d30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:29:19.805 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:19.750+0000 7f9652d9d640 1 --2- 172.21.15.37:0/322652459 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f963c004800 0x7f96540b1d30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:29:19.805 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:19.750+0000 7f9652d9d640 1 -- 172.21.15.37:0/322652459 --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f96540c7300 con 0x7f96540cb500 2024-04-25T10:29:19.806 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:19.750+0000 7f60743b2640 1 -- 172.21.15.37:0/92992236 --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_getmap magic: 0 -- 0x7f60640063a0 con 0x7f606419be20 2024-04-25T10:29:19.806 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:19.750+0000 7f60743b2640 1 -- 172.21.15.37:0/92992236 --> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] -- mon_getmap magic: 0 -- 0x7f6064101240 con 0x7f60640545c0 2024-04-25T10:29:19.806 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:19.749+0000 7f6939dc5640 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f69340b3b40 0x7f69340cc230 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:29:19.807 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:19.750+0000 7f3e3b2ec640 1 --2- >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f3e340705f0 0x7f3e342060e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:29:19.807 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:19.750+0000 7f3e3b2ec640 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f3e34072ee0 0x7f3e34162bd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:29:19.807 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:19.750+0000 7f9652d9d640 1 --2- 172.21.15.37:0/322652459 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f96540cb500 0x7f96540cb910 secure :-1 s=READY pgs=82 cs=0 l=1 rev1=1 crypto rx=0x7f96540738f0 tx=0x7f964c00f580 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:29:19.807 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:19.751+0000 7f3e3b2ec640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f3e34164c40 0x7f3e340aa160 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:29:19.808 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:19.749+0000 7f4dd1762640 1 --2- 172.21.15.37:0/2431781660 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f4dcc0b7750 0x7f4dcc154d10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:29:19.808 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:19.749+0000 7f4dd1762640 1 -- 172.21.15.37:0/2431781660 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f4dcc0cb110 con 0x7f4dcc15aa20 2024-04-25T10:29:19.808 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:19.750+0000 7f4dd0f61640 1 --2- 172.21.15.37:0/2431781660 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f4dcc0b7750 0x7f4dcc154d10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T10:29:19.809 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:19.751+0000 7f3affd72640 1 --2- >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f3af80b0ba0 0x7f3af809e940 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:29:19.809 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:19.751+0000 7f3affd72640 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f3af80d1db0 0x7f3af815e0d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:29:19.809 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:19.751+0000 7f3affd72640 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 -- 0x7f3af80afa00 con 0x7f3af809b700 2024-04-25T10:29:19.809 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:19.751+0000 7f9642ffd640 1 -- 172.21.15.37:0/322652459 <== mon.1 v2:172.21.15.203:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f964c00fde0 con 0x7f96540cb500 2024-04-25T10:29:19.809 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:19.751+0000 7fb8e86d8640 1 -- 172.21.15.37:0/756336885 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fb8c8005290 con 0x7fb8d80d58b0 2024-04-25T10:29:19.810 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:19.751+0000 7f607194d640 1 -- 172.21.15.37:0/92992236 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f606419be20 msgr2=0x7f6064103b30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:29:19.810 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:19.751+0000 7f607194d640 1 --2- 172.21.15.37:0/92992236 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f606419be20 0x7f6064103b30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:29:19.810 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:19.751+0000 7f607194d640 1 -- 172.21.15.37:0/92992236 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f60640500f0 msgr2=0x7f606404d760 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:29:19.810 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:19.751+0000 7f607194d640 1 --2- 172.21.15.37:0/92992236 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f60640500f0 0x7f606404d760 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:29:19.810 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:19.751+0000 7f607194d640 1 -- 172.21.15.37:0/92992236 --> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f606419c5d0 con 0x7f60640545c0 2024-04-25T10:29:19.810 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:19.751+0000 7f9642ffd640 1 -- 172.21.15.37:0/322652459 <== mon.1 v2:172.21.15.203:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f964c01c070 con 0x7f96540cb500 2024-04-25T10:29:19.811 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:19.751+0000 7f3e3a2ea640 1 --2- >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f3e340705f0 0x7f3e342060e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:29:19.811 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:19.751+0000 7f3e3a2ea640 1 --2- >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f3e340705f0 0x7f3e342060e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.203:3301/0 says I am v2:172.21.15.37:36818/0 (socket says 172.21.15.37:36818) 2024-04-25T10:29:19.811 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:19.751+0000 7f3e39ae9640 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f3e34072ee0 0x7f3e34162bd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:29:19.811 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:19.751+0000 7f3e39ae9640 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f3e34072ee0 0x7f3e34162bd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.203:3300/0 says I am v2:172.21.15.37:49522/0 (socket says 172.21.15.37:49522) 2024-04-25T10:29:19.811 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:19.751+0000 7f3e3a2ea640 1 -- 172.21.15.37:0/624744560 learned_addr learned my addr 172.21.15.37:0/624744560 (peer_addr_for_me v2:172.21.15.37:0/0) 2024-04-25T10:29:19.811 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:19.751+0000 7f3e3b2ec640 1 -- 172.21.15.37:0/624744560 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 -- 0x7f3e340b4910 con 0x7f3e34164c40 2024-04-25T10:29:19.811 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:19.751+0000 7f3e3b2ec640 1 -- 172.21.15.37:0/624744560 --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_getmap magic: 0 -- 0x7f3e340cf060 con 0x7f3e34072ee0 2024-04-25T10:29:19.811 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:19.751+0000 7f3e3b2ec640 1 -- 172.21.15.37:0/624744560 --> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] -- mon_getmap magic: 0 -- 0x7f3e340b4bf0 con 0x7f3e340705f0 2024-04-25T10:29:19.811 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:19.751+0000 7f3e3aaeb640 1 --2- 172.21.15.37:0/624744560 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f3e34164c40 0x7f3e340aa160 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:29:19.812 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:19.751+0000 7f4adaffd640 1 -- 172.21.15.37:0/3186560673 <== mon.0 v2:172.21.15.37:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4adc015020 con 0x7f4aec0b4fd0 2024-04-25T10:29:19.812 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:19.751+0000 7f4adaffd640 1 -- 172.21.15.37:0/3186560673 <== mon.0 v2:172.21.15.37:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f4adc011830 con 0x7f4aec0b4fd0 2024-04-25T10:29:19.812 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:19.751+0000 7f3affd72640 1 -- --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_getmap magic: 0 -- 0x7f3af80b03a0 con 0x7f3af80d1db0 2024-04-25T10:29:19.813 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:19.751+0000 7f9658984640 1 -- 172.21.15.37:0/322652459 --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f96540c7800 con 0x7f96540cb500 2024-04-25T10:29:19.813 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:19.751+0000 7f9658984640 1 -- 172.21.15.37:0/322652459 --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f96540b1550 con 0x7f96540cb500 2024-04-25T10:29:19.813 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:19.751+0000 7f4dd1762640 1 --2- 172.21.15.37:0/2431781660 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f4dcc15aa20 0x7f4dcc0cef30 secure :-1 s=READY pgs=121 cs=0 l=1 rev1=1 crypto rx=0x7f4dc003a000 tx=0x7f4dc00046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:29:19.813 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:19.752+0000 7f4dc97fa640 1 -- 172.21.15.37:0/2431781660 <== mon.0 v2:172.21.15.37:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4dc0040040 con 0x7f4dcc15aa20 2024-04-25T10:29:19.813 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:19.752+0000 7f4dc97fa640 1 -- 172.21.15.37:0/2431781660 <== mon.0 v2:172.21.15.37:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f4dc004b070 con 0x7f4dcc15aa20 2024-04-25T10:29:19.813 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:19.752+0000 7f4dd31c5640 1 -- 172.21.15.37:0/2431781660 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f4dcc0ca370 con 0x7f4dcc15aa20 2024-04-25T10:29:19.813 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:19.751+0000 7f3e39ae9640 1 -- 172.21.15.37:0/624744560 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f3e340705f0 msgr2=0x7f3e342060e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:29:19.814 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:19.751+0000 7f3e39ae9640 1 --2- 172.21.15.37:0/624744560 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f3e340705f0 0x7f3e342060e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:29:19.814 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:19.751+0000 7f3e39ae9640 1 -- 172.21.15.37:0/624744560 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f3e34164c40 msgr2=0x7f3e340aa160 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:29:19.814 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:19.751+0000 7f3e39ae9640 1 --2- 172.21.15.37:0/624744560 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f3e34164c40 0x7f3e340aa160 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:29:19.814 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:19.751+0000 7f3e39ae9640 1 -- 172.21.15.37:0/624744560 --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f3e34163790 con 0x7f3e34072ee0 2024-04-25T10:29:19.814 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:19.751+0000 7f3e3a2ea640 1 --2- 172.21.15.37:0/624744560 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f3e340705f0 0x7f3e342060e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-25T10:29:19.814 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:19.751+0000 7f607194d640 1 --2- 172.21.15.37:0/92992236 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f60640545c0 0x7f6064199280 secure :-1 s=READY pgs=36 cs=0 l=1 rev1=1 crypto rx=0x7f605c033000 tx=0x7f605c0046c0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:29:19.815 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:19.751+0000 7f3affd72640 1 -- --> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] -- mon_getmap magic: 0 -- 0x7f3af80afb80 con 0x7f3af80b0ba0 2024-04-25T10:29:19.815 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:19.751+0000 7f3afd30d640 1 --2- >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f3af80b0ba0 0x7f3af809e940 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:29:19.815 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:19.751+0000 7f3afd30d640 1 --2- >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f3af80b0ba0 0x7f3af809e940 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.203:3301/0 says I am v2:172.21.15.37:36824/0 (socket says 172.21.15.37:36824) 2024-04-25T10:29:19.815 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:19.751+0000 7f3afd30d640 1 -- 172.21.15.37:0/2546797036 learned_addr learned my addr 172.21.15.37:0/2546797036 (peer_addr_for_me v2:172.21.15.37:0/0) 2024-04-25T10:29:19.815 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:19.752+0000 7f3afdb0e640 1 --2- 172.21.15.37:0/2546797036 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f3af809b700 0x7f3af8149370 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:29:19.815 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:19.752+0000 7f3afd30d640 1 -- 172.21.15.37:0/2546797036 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f3af80d1db0 msgr2=0x7f3af815e0d0 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2024-04-25T10:29:19.815 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:19.752+0000 7f3afd30d640 1 --2- 172.21.15.37:0/2546797036 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f3af80d1db0 0x7f3af815e0d0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:29:19.815 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:19.752+0000 7f3afd30d640 1 -- 172.21.15.37:0/2546797036 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f3af809b700 msgr2=0x7f3af8149370 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:29:19.815 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:19.752+0000 7f3afd30d640 1 --2- 172.21.15.37:0/2546797036 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f3af809b700 0x7f3af8149370 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:29:19.815 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:19.752+0000 7f3afd30d640 1 -- 172.21.15.37:0/2546797036 --> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f3af8157960 con 0x7f3af80b0ba0 2024-04-25T10:29:19.815 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:19.749+0000 7f6939dc5640 1 --2- >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f69340b5520 0x7f6934154db0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:29:19.816 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:19.749+0000 7f6939dc5640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f69340ce6a0 0x7f69341538d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:29:19.816 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:19.749+0000 7f6939dc5640 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 -- 0x7f69340d0810 con 0x7f69340ce6a0 2024-04-25T10:29:19.816 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:19.749+0000 7f6939dc5640 1 -- --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_getmap magic: 0 -- 0x7f69340d0690 con 0x7f69340b3b40 2024-04-25T10:29:19.816 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:19.749+0000 7f6939dc5640 1 -- --> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] -- mon_getmap magic: 0 -- 0x7f69340d0990 con 0x7f69340b5520 2024-04-25T10:29:19.816 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:19.750+0000 7f6933fff640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f69340ce6a0 0x7f69341538d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:29:19.816 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:19.750+0000 7f6933fff640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f69340ce6a0 0x7f69341538d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.37:3300/0 says I am v2:172.21.15.37:34934/0 (socket says 172.21.15.37:34934) 2024-04-25T10:29:19.816 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:19.750+0000 7f6933fff640 1 -- 172.21.15.37:0/3380961062 learned_addr learned my addr 172.21.15.37:0/3380961062 (peer_addr_for_me v2:172.21.15.37:0/0) 2024-04-25T10:29:19.816 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:19.750+0000 7f6933fff640 1 -- 172.21.15.37:0/3380961062 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f69340b5520 msgr2=0x7f6934154db0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:29:19.816 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:19.750+0000 7f6933fff640 1 --2- 172.21.15.37:0/3380961062 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f69340b5520 0x7f6934154db0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:29:19.816 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:19.750+0000 7f6933fff640 1 -- 172.21.15.37:0/3380961062 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f69340b3b40 msgr2=0x7f69340cc230 unknown :-1 s=STATE_CONNECTING_RE l=1).mark_down 2024-04-25T10:29:19.816 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:19.750+0000 7f6933fff640 1 --2- 172.21.15.37:0/3380961062 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f69340b3b40 0x7f69340cc230 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:29:19.816 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:19.750+0000 7f6933fff640 1 -- 172.21.15.37:0/3380961062 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f6934152200 con 0x7f69340ce6a0 2024-04-25T10:29:19.816 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:19.751+0000 7f6933fff640 1 --2- 172.21.15.37:0/3380961062 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f69340ce6a0 0x7f69341538d0 secure :-1 s=READY pgs=120 cs=0 l=1 rev1=1 crypto rx=0x7f692c00efc0 tx=0x7f692c00c490 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:29:19.817 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:19.751+0000 7f9642ffd640 1 -- 172.21.15.37:0/322652459 <== mon.1 v2:172.21.15.203:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f964c003bd0 con 0x7f96540cb500 2024-04-25T10:29:19.817 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:19.751+0000 7f4adaffd640 1 -- 172.21.15.37:0/3186560673 <== mon.0 v2:172.21.15.37:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4adc011be0 con 0x7f4aec0b4fd0 2024-04-25T10:29:19.817 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:19.751+0000 7f4af0bb7640 1 -- 172.21.15.37:0/3186560673 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f4aec145740 con 0x7f4aec0b4fd0 2024-04-25T10:29:19.817 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:19.751+0000 7f4af0bb7640 1 -- 172.21.15.37:0/3186560673 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f4aec145d00 con 0x7f4aec0b4fd0 2024-04-25T10:29:19.817 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:19.752+0000 7f4dd31c5640 1 -- 172.21.15.37:0/2431781660 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f4dcc0ca910 con 0x7f4dcc15aa20 2024-04-25T10:29:19.818 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:19.752+0000 7f6062ffd640 1 -- 172.21.15.37:0/92992236 <== mon.2 v2:172.21.15.203:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f605c039040 con 0x7f60640545c0 2024-04-25T10:29:19.818 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:19.752+0000 7f3e39ae9640 1 --2- 172.21.15.37:0/624744560 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f3e34072ee0 0x7f3e34162bd0 secure :-1 s=READY pgs=83 cs=0 l=1 rev1=1 crypto rx=0x7f3e3000c410 tx=0x7f3e3000c8d0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:29:19.818 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:19.752+0000 7f3e2affd640 1 -- 172.21.15.37:0/624744560 <== mon.1 v2:172.21.15.203:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3e300074e0 con 0x7f3e34072ee0 2024-04-25T10:29:19.818 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:19.752+0000 7f3e2affd640 1 -- 172.21.15.37:0/624744560 <== mon.1 v2:172.21.15.203:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f3e300047e0 con 0x7f3e34072ee0 2024-04-25T10:29:19.818 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:19.752+0000 7f3e2affd640 1 -- 172.21.15.37:0/624744560 <== mon.1 v2:172.21.15.203:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3e300074e0 con 0x7f3e34072ee0 2024-04-25T10:29:19.818 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:19.752+0000 7f3e3b2ec640 1 -- 172.21.15.37:0/624744560 --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f3e34163a70 con 0x7f3e34072ee0 2024-04-25T10:29:19.818 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:19.752+0000 7f3e3b2ec640 1 -- 172.21.15.37:0/624744560 --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f3e3415d050 con 0x7f3e34072ee0 2024-04-25T10:29:19.819 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:19.752+0000 7f60743b2640 1 -- 172.21.15.37:0/92992236 --> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f606419c8b0 con 0x7f60640545c0 2024-04-25T10:29:19.819 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:19.753+0000 7f60743b2640 1 -- 172.21.15.37:0/92992236 --> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] -- mon_subscribe({osdmap=0}) -- 0x7f606404c0d0 con 0x7f60640545c0 2024-04-25T10:29:19.819 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:19.753+0000 7f6062ffd640 1 -- 172.21.15.37:0/92992236 <== mon.2 v2:172.21.15.203:3301/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f605c004230 con 0x7f60640545c0 2024-04-25T10:29:19.819 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:19.752+0000 7f691bfff640 1 -- 172.21.15.37:0/3380961062 <== mon.0 v2:172.21.15.37:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f692c028070 con 0x7f69340ce6a0 2024-04-25T10:29:19.819 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:19.753+0000 7f691bfff640 1 -- 172.21.15.37:0/3380961062 <== mon.0 v2:172.21.15.37:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f692c0243f0 con 0x7f69340ce6a0 2024-04-25T10:29:19.819 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:19.753+0000 7f6939dc5640 1 -- 172.21.15.37:0/3380961062 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f69341524e0 con 0x7f69340ce6a0 2024-04-25T10:29:19.819 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:19.753+0000 7f6939dc5640 1 -- 172.21.15.37:0/3380961062 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f69340b5fb0 con 0x7f69340ce6a0 2024-04-25T10:29:19.820 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:19.753+0000 7f9658984640 1 -- 172.21.15.37:0/322652459 --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f9638005290 con 0x7f96540cb500 2024-04-25T10:29:19.820 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:19.753+0000 7f3afd30d640 1 --2- 172.21.15.37:0/2546797036 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f3af80b0ba0 0x7f3af809e940 secure :-1 s=READY pgs=37 cs=0 l=1 rev1=1 crypto rx=0x7f3ae800b3f0 tx=0x7f3ae800b8b0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:29:19.820 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:19.753+0000 7f4ad8ff9640 1 -- 172.21.15.37:0/3186560673 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f4ac4005290 con 0x7f4aec0b4fd0 2024-04-25T10:29:19.820 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:19.753+0000 7f4dc97fa640 1 -- 172.21.15.37:0/2431781660 <== mon.0 v2:172.21.15.37:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4dc0003be0 con 0x7f4dcc15aa20 2024-04-25T10:29:19.821 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:19.753+0000 7f691bfff640 1 -- 172.21.15.37:0/3380961062 <== mon.0 v2:172.21.15.37:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f692c0247c0 con 0x7f69340ce6a0 2024-04-25T10:29:19.821 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:19.753+0000 7fb8d6ffd640 1 -- 172.21.15.37:0/756336885 <== mon.0 v2:172.21.15.37:3300/0 4 ==== mgrmap(e 5) ==== 96782+0+0 (secure 0 0 0) 0x7fb8e00af7e0 con 0x7fb8d80d58b0 2024-04-25T10:29:19.821 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:19.753+0000 7fb8d6ffd640 1 --2- 172.21.15.37:0/756336885 >> [v2:172.21.15.203:6832/2418205237,v1:172.21.15.203:6833/2418205237] conn(0x7fb8c4074580 0x7fb8c4076a90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:29:19.821 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:19.754+0000 7fb8d6ffd640 1 -- 172.21.15.37:0/756336885 <== mon.0 v2:172.21.15.37:3300/0 5 ==== osd_map(23..23 src has 1..23) ==== 5709+0+0 (secure 0 0 0) 0x7fb8e0132450 con 0x7fb8d80d58b0 2024-04-25T10:29:19.822 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:19.753+0000 7f6062ffd640 1 -- 172.21.15.37:0/92992236 <== mon.2 v2:172.21.15.203:3301/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f605c032750 con 0x7f60640545c0 2024-04-25T10:29:19.822 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:19.755+0000 7f3aee7fc640 1 -- 172.21.15.37:0/2546797036 <== mon.2 v2:172.21.15.203:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3ae8015020 con 0x7f3af80b0ba0 2024-04-25T10:29:19.822 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:19.755+0000 7f3aee7fc640 1 -- 172.21.15.37:0/2546797036 <== mon.2 v2:172.21.15.203:3301/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f3ae8011830 con 0x7f3af80b0ba0 2024-04-25T10:29:19.822 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:19.755+0000 7f3aee7fc640 1 -- 172.21.15.37:0/2546797036 <== mon.2 v2:172.21.15.203:3301/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3ae8011c20 con 0x7f3af80b0ba0 2024-04-25T10:29:19.823 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:19.755+0000 7f6060ff9640 1 -- 172.21.15.37:0/92992236 --> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f6064005750 con 0x7f60640545c0 2024-04-25T10:29:19.823 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:19.756+0000 7f6939dc5640 1 -- 172.21.15.37:0/3380961062 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f6914005290 con 0x7f69340ce6a0 2024-04-25T10:29:19.823 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:19.756+0000 7f6062ffd640 1 -- 172.21.15.37:0/92992236 <== mon.2 v2:172.21.15.203:3301/0 4 ==== mgrmap(e 5) ==== 96782+0+0 (secure 0 0 0) 0x7f605c0328f0 con 0x7f60640545c0 2024-04-25T10:29:19.823 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:19.755+0000 7fb8e6474640 1 --2- 172.21.15.37:0/756336885 >> [v2:172.21.15.203:6832/2418205237,v1:172.21.15.203:6833/2418205237] conn(0x7fb8c4074580 0x7fb8c4076a90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:29:19.824 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:19.755+0000 7f3affd72640 1 -- 172.21.15.37:0/2546797036 --> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f3af809f4b0 con 0x7f3af80b0ba0 2024-04-25T10:29:19.824 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:19.755+0000 7f3affd72640 1 -- 172.21.15.37:0/2546797036 --> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] -- mon_subscribe({osdmap=0}) -- 0x7f3af809fa70 con 0x7f3af80b0ba0 2024-04-25T10:29:19.824 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:19.756+0000 7fb8e6474640 1 --2- 172.21.15.37:0/756336885 >> [v2:172.21.15.203:6832/2418205237,v1:172.21.15.203:6833/2418205237] conn(0x7fb8c4074580 0x7fb8c4076a90 secure :-1 s=READY pgs=81 cs=0 l=1 rev1=1 crypto rx=0x7fb8d800d700 tx=0x7fb8d0009190 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:29:19.825 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:19.755+0000 7f3e2affd640 1 -- 172.21.15.37:0/624744560 <== mon.1 v2:172.21.15.203:3300/0 4 ==== mgrmap(e 5) ==== 96782+0+0 (secure 0 0 0) 0x7f3e30011070 con 0x7f3e34072ee0 2024-04-25T10:29:19.825 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:19.755+0000 7f3e2affd640 1 --2- 172.21.15.37:0/624744560 >> [v2:172.21.15.203:6832/2418205237,v1:172.21.15.203:6833/2418205237] conn(0x7f3e180748e0 0x7f3e18076df0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:29:19.825 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:19.756+0000 7f3e2affd640 1 -- 172.21.15.37:0/624744560 <== mon.1 v2:172.21.15.203:3300/0 5 ==== osd_map(23..23 src has 1..23) ==== 5709+0+0 (secure 0 0 0) 0x7f3e300072e0 con 0x7f3e34072ee0 2024-04-25T10:29:19.825 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:19.756+0000 7f3e3a2ea640 1 --2- 172.21.15.37:0/624744560 >> [v2:172.21.15.203:6832/2418205237,v1:172.21.15.203:6833/2418205237] conn(0x7f3e180748e0 0x7f3e18076df0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:29:19.825 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:19.756+0000 7f3e3b2ec640 1 -- 172.21.15.37:0/624744560 --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f3e1c005290 con 0x7f3e34072ee0 2024-04-25T10:29:19.825 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:19.756+0000 7f9642ffd640 1 -- 172.21.15.37:0/322652459 <== mon.1 v2:172.21.15.203:3300/0 4 ==== mgrmap(e 5) ==== 96782+0+0 (secure 0 0 0) 0x7f964c004030 con 0x7f96540cb500 2024-04-25T10:29:19.826 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:19.756+0000 7f4db2ffd640 1 -- 172.21.15.37:0/2431781660 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f4dac005290 con 0x7f4dcc15aa20 2024-04-25T10:29:19.826 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:19.757+0000 7f6062ffd640 1 --2- 172.21.15.37:0/92992236 >> [v2:172.21.15.203:6832/2418205237,v1:172.21.15.203:6833/2418205237] conn(0x7f6050074580 0x7f6050076a90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:29:19.826 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:19.757+0000 7f6062ffd640 1 -- 172.21.15.37:0/92992236 <== mon.2 v2:172.21.15.203:3301/0 5 ==== osd_map(23..23 src has 1..23) ==== 5709+0+0 (secure 0 0 0) 0x7f605c0b5c70 con 0x7f60640545c0 2024-04-25T10:29:19.827 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:19.757+0000 7f4dc97fa640 1 -- 172.21.15.37:0/2431781660 <== mon.0 v2:172.21.15.37:3300/0 4 ==== mgrmap(e 5) ==== 96782+0+0 (secure 0 0 0) 0x7f4dc0050460 con 0x7f4dcc15aa20 2024-04-25T10:29:19.827 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:19.757+0000 7f607214e640 1 --2- 172.21.15.37:0/92992236 >> [v2:172.21.15.203:6832/2418205237,v1:172.21.15.203:6833/2418205237] conn(0x7f6050074580 0x7f6050076a90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:29:19.827 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:19.757+0000 7f3e3a2ea640 1 --2- 172.21.15.37:0/624744560 >> [v2:172.21.15.203:6832/2418205237,v1:172.21.15.203:6833/2418205237] conn(0x7f3e180748e0 0x7f3e18076df0 secure :-1 s=READY pgs=82 cs=0 l=1 rev1=1 crypto rx=0x7f3e24001bf0 tx=0x7f3e24009040 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:29:19.828 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:19.757+0000 7f4dc97fa640 1 --2- 172.21.15.37:0/2431781660 >> [v2:172.21.15.203:6832/2418205237,v1:172.21.15.203:6833/2418205237] conn(0x7f4db8074500 0x7f4db8076a10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:29:19.828 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:19.757+0000 7f9642ffd640 1 --2- 172.21.15.37:0/322652459 >> [v2:172.21.15.203:6832/2418205237,v1:172.21.15.203:6833/2418205237] conn(0x7f96340745b0 0x7f9634076ac0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:29:19.828 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:19.757+0000 7f965259c640 1 --2- 172.21.15.37:0/322652459 >> [v2:172.21.15.203:6832/2418205237,v1:172.21.15.203:6833/2418205237] conn(0x7f96340745b0 0x7f9634076ac0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:29:19.828 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:19.757+0000 7f9642ffd640 1 -- 172.21.15.37:0/322652459 <== mon.1 v2:172.21.15.203:3300/0 5 ==== osd_map(23..23 src has 1..23) ==== 5709+0+0 (secure 0 0 0) 0x7f964c095b30 con 0x7f96540cb500 2024-04-25T10:29:19.829 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:19.758+0000 7f3affd72640 1 -- 172.21.15.37:0/2546797036 --> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f3af806a140 con 0x7f3af80b0ba0 2024-04-25T10:29:19.829 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:19.758+0000 7f607214e640 1 --2- 172.21.15.37:0/92992236 >> [v2:172.21.15.203:6832/2418205237,v1:172.21.15.203:6833/2418205237] conn(0x7f6050074580 0x7f6050076a90 secure :-1 s=READY pgs=83 cs=0 l=1 rev1=1 crypto rx=0x7f6068009870 tx=0x7f6068008040 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:29:19.829 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:19.758+0000 7f4dd0f61640 1 --2- 172.21.15.37:0/2431781660 >> [v2:172.21.15.203:6832/2418205237,v1:172.21.15.203:6833/2418205237] conn(0x7f4db8074500 0x7f4db8076a10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:29:19.829 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:19.758+0000 7f3aee7fc640 1 -- 172.21.15.37:0/2546797036 <== mon.2 v2:172.21.15.203:3301/0 4 ==== mgrmap(e 5) ==== 96782+0+0 (secure 0 0 0) 0x7f3ae8018020 con 0x7f3af80b0ba0 2024-04-25T10:29:19.830 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:19.758+0000 7f965259c640 1 --2- 172.21.15.37:0/322652459 >> [v2:172.21.15.203:6832/2418205237,v1:172.21.15.203:6833/2418205237] conn(0x7f96340745b0 0x7f9634076ac0 secure :-1 s=READY pgs=84 cs=0 l=1 rev1=1 crypto rx=0x7f96440010a0 tx=0x7f9644009040 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:29:19.830 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:19.758+0000 7f4dd0f61640 1 --2- 172.21.15.37:0/2431781660 >> [v2:172.21.15.203:6832/2418205237,v1:172.21.15.203:6833/2418205237] conn(0x7f4db8074500 0x7f4db8076a10 secure :-1 s=READY pgs=85 cs=0 l=1 rev1=1 crypto rx=0x7f4dbc007fd0 tx=0x7f4dbc007dc0 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:29:19.831 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:19.758+0000 7f3aee7fc640 1 --2- 172.21.15.37:0/2546797036 >> [v2:172.21.15.203:6832/2418205237,v1:172.21.15.203:6833/2418205237] conn(0x7f3adc074530 0x7f3adc076a40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:29:19.831 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:19.759+0000 7f3afdb0e640 1 --2- 172.21.15.37:0/2546797036 >> [v2:172.21.15.203:6832/2418205237,v1:172.21.15.203:6833/2418205237] conn(0x7f3adc074530 0x7f3adc076a40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:29:19.831 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:19.759+0000 7f3afdb0e640 1 --2- 172.21.15.37:0/2546797036 >> [v2:172.21.15.203:6832/2418205237,v1:172.21.15.203:6833/2418205237] conn(0x7f3adc074530 0x7f3adc076a40 secure :-1 s=READY pgs=86 cs=0 l=1 rev1=1 crypto rx=0x7f3af4037000 tx=0x7f3af4030000 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:29:19.831 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:19.760+0000 7fb8d6ffd640 1 -- 172.21.15.37:0/756336885 <== mon.0 v2:172.21.15.37:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187246 (secure 0 0 0) 0x7fb8e00bc030 con 0x7fb8d80d58b0 2024-04-25T10:29:19.831 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:19.760+0000 7f4dc97fa640 1 -- 172.21.15.37:0/2431781660 <== mon.0 v2:172.21.15.37:3300/0 5 ==== osd_map(23..23 src has 1..23) ==== 5709+0+0 (secure 0 0 0) 0x7f4dc0003d80 con 0x7f4dcc15aa20 2024-04-25T10:29:19.832 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:19.761+0000 7f4adaffd640 1 -- 172.21.15.37:0/3186560673 <== mon.0 v2:172.21.15.37:3300/0 4 ==== mgrmap(e 5) ==== 96782+0+0 (secure 0 0 0) 0x7f4adc018020 con 0x7f4aec0b4fd0 2024-04-25T10:29:19.832 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:19.762+0000 7f691bfff640 1 -- 172.21.15.37:0/3380961062 <== mon.0 v2:172.21.15.37:3300/0 4 ==== mgrmap(e 5) ==== 96782+0+0 (secure 0 0 0) 0x7f692c005d10 con 0x7f69340ce6a0 2024-04-25T10:29:19.832 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:19.762+0000 7f4adaffd640 1 --2- 172.21.15.37:0/3186560673 >> [v2:172.21.15.203:6832/2418205237,v1:172.21.15.203:6833/2418205237] conn(0x7f4acc074500 0x7f4acc076a10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:29:19.832 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:19.762+0000 7f4adaffd640 1 -- 172.21.15.37:0/3186560673 <== mon.0 v2:172.21.15.37:3300/0 5 ==== osd_map(23..23 src has 1..23) ==== 5709+0+0 (secure 0 0 0) 0x7f4adc0956f0 con 0x7f4aec0b4fd0 2024-04-25T10:29:19.832 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:19.762+0000 7f3aee7fc640 1 -- 172.21.15.37:0/2546797036 <== mon.2 v2:172.21.15.203:3301/0 5 ==== osd_map(23..23 src has 1..23) ==== 5709+0+0 (secure 0 0 0) 0x7f3ae8011dc0 con 0x7f3af80b0ba0 2024-04-25T10:29:19.833 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:19.762+0000 7f6062ffd640 1 -- 172.21.15.37:0/92992236 <== mon.2 v2:172.21.15.203:3301/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187246 (secure 0 0 0) 0x7f605c0803e0 con 0x7f60640545c0 2024-04-25T10:29:19.833 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:19.762+0000 7f691bfff640 1 --2- 172.21.15.37:0/3380961062 >> [v2:172.21.15.203:6832/2418205237,v1:172.21.15.203:6833/2418205237] conn(0x7f6920074500 0x7f6920076a10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:29:19.833 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:19.762+0000 7f69337fe640 1 --2- 172.21.15.37:0/3380961062 >> [v2:172.21.15.203:6832/2418205237,v1:172.21.15.203:6833/2418205237] conn(0x7f6920074500 0x7f6920076a10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:29:19.834 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:19.762+0000 7f9642ffd640 1 -- 172.21.15.37:0/322652459 <== mon.1 v2:172.21.15.203:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187246 (secure 0 0 0) 0x7f964c010040 con 0x7f96540cb500 2024-04-25T10:29:19.834 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:19.763+0000 7f4aea59c640 1 --2- 172.21.15.37:0/3186560673 >> [v2:172.21.15.203:6832/2418205237,v1:172.21.15.203:6833/2418205237] conn(0x7f4acc074500 0x7f4acc076a10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:29:19.834 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:19.763+0000 7f69337fe640 1 --2- 172.21.15.37:0/3380961062 >> [v2:172.21.15.203:6832/2418205237,v1:172.21.15.203:6833/2418205237] conn(0x7f6920074500 0x7f6920076a10 secure :-1 s=READY pgs=87 cs=0 l=1 rev1=1 crypto rx=0x7f6924034000 tx=0x7f692402e000 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:29:19.835 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:19.763+0000 7f4aea59c640 1 --2- 172.21.15.37:0/3186560673 >> [v2:172.21.15.203:6832/2418205237,v1:172.21.15.203:6833/2418205237] conn(0x7f4acc074500 0x7f4acc076a10 secure :-1 s=READY pgs=88 cs=0 l=1 rev1=1 crypto rx=0x7f4ae0034000 tx=0x7f4ae002e000 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:29:19.835 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:19.767+0000 7f691bfff640 1 -- 172.21.15.37:0/3380961062 <== mon.0 v2:172.21.15.37:3300/0 5 ==== osd_map(23..23 src has 1..23) ==== 5709+0+0 (secure 0 0 0) 0x7f692c06d100 con 0x7f69340ce6a0 2024-04-25T10:29:19.835 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:19.767+0000 7f4dc97fa640 1 -- 172.21.15.37:0/2431781660 <== mon.0 v2:172.21.15.37:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187246 (secure 0 0 0) 0x7f4dc00044a0 con 0x7f4dcc15aa20 2024-04-25T10:29:19.835 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:19.767+0000 7f3aee7fc640 1 -- 172.21.15.37:0/2546797036 <== mon.2 v2:172.21.15.203:3301/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187246 (secure 0 0 0) 0x7f3ae801f070 con 0x7f3af80b0ba0 2024-04-25T10:29:19.836 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:19.767+0000 7f3e2affd640 1 -- 172.21.15.37:0/624744560 <== mon.1 v2:172.21.15.203:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187246 (secure 0 0 0) 0x7f3e34070a00 con 0x7f3e34072ee0 2024-04-25T10:29:19.836 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:19.772+0000 7f4adaffd640 1 -- 172.21.15.37:0/3186560673 <== mon.0 v2:172.21.15.37:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187246 (secure 0 0 0) 0x7f4adc01f020 con 0x7f4aec0b4fd0 2024-04-25T10:29:19.836 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:19.779+0000 7f691bfff640 1 -- 172.21.15.37:0/3380961062 <== mon.0 v2:172.21.15.37:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187246 (secure 0 0 0) 0x7f692c071570 con 0x7f69340ce6a0 2024-04-25T10:29:20.192 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:20.190+0000 7fb8e86d8640 1 -- 172.21.15.37:0/756336885 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 6} v 0) -- 0x7fb8c8005580 con 0x7fb8d80d58b0 2024-04-25T10:29:20.192 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:20.190+0000 7fb8d6ffd640 1 -- 172.21.15.37:0/756336885 <== mon.0 v2:172.21.15.37:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 6}]=0 v0) ==== 74+0+12 (secure 0 0 0) 0x7fb8e00fcbc0 con 0x7fb8d80d58b0 2024-04-25T10:29:20.193 INFO:teuthology.orchestra.run.smithi037.stdout:64424509447 2024-04-25T10:29:20.197 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:20.194+0000 7fb8d4ff9640 1 -- 172.21.15.37:0/756336885 >> [v2:172.21.15.203:6832/2418205237,v1:172.21.15.203:6833/2418205237] conn(0x7fb8c4074580 msgr2=0x7fb8c4076a90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:29:20.197 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:20.194+0000 7fb8d4ff9640 1 --2- 172.21.15.37:0/756336885 >> [v2:172.21.15.203:6832/2418205237,v1:172.21.15.203:6833/2418205237] conn(0x7fb8c4074580 0x7fb8c4076a90 secure :-1 s=READY pgs=81 cs=0 l=1 rev1=1 crypto rx=0x7fb8d800d700 tx=0x7fb8d0009190 comp rx=0 tx=0).stop 2024-04-25T10:29:20.197 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:20.194+0000 7fb8d4ff9640 1 -- 172.21.15.37:0/756336885 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fb8d80d58b0 msgr2=0x7fb8d80399f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:29:20.197 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:20.194+0000 7fb8d4ff9640 1 --2- 172.21.15.37:0/756336885 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fb8d80d58b0 0x7fb8d80399f0 secure :-1 s=READY pgs=118 cs=0 l=1 rev1=1 crypto rx=0x7fb8e006b610 tx=0x7fb8e00759d0 comp rx=0 tx=0).stop 2024-04-25T10:29:20.197 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:20.194+0000 7fb8d4ff9640 1 -- 172.21.15.37:0/756336885 shutdown_connections 2024-04-25T10:29:20.197 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:20.194+0000 7fb8d4ff9640 1 --2- 172.21.15.37:0/756336885 >> [v2:172.21.15.203:6832/2418205237,v1:172.21.15.203:6833/2418205237] conn(0x7fb8c4074580 0x7fb8c4076a90 secure :-1 s=CLOSED pgs=81 cs=0 l=1 rev1=1 crypto rx=0x7fb8d800d700 tx=0x7fb8d0009190 comp rx=0 tx=0).stop 2024-04-25T10:29:20.197 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:20.195+0000 7fb8d4ff9640 1 --2- 172.21.15.37:0/756336885 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fb8d80d58b0 0x7fb8d80399f0 unknown :-1 s=CLOSED pgs=118 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:29:20.197 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:20.195+0000 7fb8d4ff9640 1 --2- 172.21.15.37:0/756336885 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fb8d8055840 0x7fb8d80d5370 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:29:20.197 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:20.195+0000 7fb8d4ff9640 1 --2- 172.21.15.37:0/756336885 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7fb8d8053e10 0x7fb8d8033dc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:29:20.197 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:20.195+0000 7fb8d4ff9640 1 -- 172.21.15.37:0/756336885 >> 172.21.15.37:0/756336885 conn(0x7fb8d8054d20 msgr2=0x7fb8d80d6db0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:29:20.198 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:20.196+0000 7fb8d4ff9640 1 -- 172.21.15.37:0/756336885 shutdown_connections 2024-04-25T10:29:20.198 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:20.196+0000 7f6060ff9640 1 -- 172.21.15.37:0/92992236 --> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 0} v 0) -- 0x7f6064005750 con 0x7f60640545c0 2024-04-25T10:29:20.199 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:20.196+0000 7fb8d4ff9640 1 -- 172.21.15.37:0/756336885 wait complete. 2024-04-25T10:29:20.199 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:20.197+0000 7f6062ffd640 1 -- 172.21.15.37:0/92992236 <== mon.2 v2:172.21.15.203:3301/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 0}]=0 v0) ==== 74+0+12 (secure 0 0 0) 0x7f605c07fce0 con 0x7f60640545c0 2024-04-25T10:29:20.199 INFO:teuthology.orchestra.run.smithi037.stdout:64424509447 2024-04-25T10:29:20.202 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:20.200+0000 7f6060ff9640 1 -- 172.21.15.37:0/92992236 >> [v2:172.21.15.203:6832/2418205237,v1:172.21.15.203:6833/2418205237] conn(0x7f6050074580 msgr2=0x7f6050076a90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:29:20.202 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:20.200+0000 7f6060ff9640 1 --2- 172.21.15.37:0/92992236 >> [v2:172.21.15.203:6832/2418205237,v1:172.21.15.203:6833/2418205237] conn(0x7f6050074580 0x7f6050076a90 secure :-1 s=READY pgs=83 cs=0 l=1 rev1=1 crypto rx=0x7f6068009870 tx=0x7f6068008040 comp rx=0 tx=0).stop 2024-04-25T10:29:20.203 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:20.200+0000 7f6060ff9640 1 -- 172.21.15.37:0/92992236 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f60640545c0 msgr2=0x7f6064199280 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:29:20.203 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:20.200+0000 7f6060ff9640 1 --2- 172.21.15.37:0/92992236 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f60640545c0 0x7f6064199280 secure :-1 s=READY pgs=36 cs=0 l=1 rev1=1 crypto rx=0x7f605c033000 tx=0x7f605c0046c0 comp rx=0 tx=0).stop 2024-04-25T10:29:20.203 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:20.201+0000 7f9658984640 1 -- 172.21.15.37:0/322652459 --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 1} v 0) -- 0x7f9638005580 con 0x7f96540cb500 2024-04-25T10:29:20.204 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:20.201+0000 7f6060ff9640 1 -- 172.21.15.37:0/92992236 shutdown_connections 2024-04-25T10:29:20.204 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:20.201+0000 7f6060ff9640 1 --2- 172.21.15.37:0/92992236 >> [v2:172.21.15.203:6832/2418205237,v1:172.21.15.203:6833/2418205237] conn(0x7f6050074580 0x7f6050076a90 unknown :-1 s=CLOSED pgs=83 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:29:20.204 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:20.201+0000 7f6060ff9640 1 --2- 172.21.15.37:0/92992236 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f606419be20 0x7f6064103b30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:29:20.204 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:20.201+0000 7f6060ff9640 1 --2- 172.21.15.37:0/92992236 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f60640545c0 0x7f6064199280 unknown :-1 s=CLOSED pgs=36 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:29:20.204 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:20.201+0000 7f6060ff9640 1 --2- 172.21.15.37:0/92992236 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f60640500f0 0x7f606404d760 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:29:20.204 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:20.201+0000 7f6060ff9640 1 -- 172.21.15.37:0/92992236 >> 172.21.15.37:0/92992236 conn(0x7f6064054ed0 msgr2=0x7f6064038ce0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:29:20.204 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:20.201+0000 7f6060ff9640 1 -- 172.21.15.37:0/92992236 shutdown_connections 2024-04-25T10:29:20.204 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:20.201+0000 7f6060ff9640 1 -- 172.21.15.37:0/92992236 wait complete. 2024-04-25T10:29:20.205 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:20.201+0000 7f9642ffd640 1 -- 172.21.15.37:0/322652459 <== mon.1 v2:172.21.15.203:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 1}]=0 v0) ==== 74+0+12 (secure 0 0 0) 0x7f964c060330 con 0x7f96540cb500 2024-04-25T10:29:20.205 INFO:teuthology.orchestra.run.smithi037.stdout:64424509446 2024-04-25T10:29:20.206 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:20.204+0000 7f3affd72640 1 -- 172.21.15.37:0/2546797036 --> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 2} v 0) -- 0x7f3af809f240 con 0x7f3af80b0ba0 2024-04-25T10:29:20.206 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:20.204+0000 7f3aee7fc640 1 -- 172.21.15.37:0/2546797036 <== mon.2 v2:172.21.15.203:3301/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 2}]=0 v0) ==== 74+0+12 (secure 0 0 0) 0x7f3ae805f1f0 con 0x7f3af80b0ba0 2024-04-25T10:29:20.207 INFO:teuthology.orchestra.run.smithi037.stdout:64424509448 2024-04-25T10:29:20.209 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:20.205+0000 7f9640ff9640 1 -- 172.21.15.37:0/322652459 >> [v2:172.21.15.203:6832/2418205237,v1:172.21.15.203:6833/2418205237] conn(0x7f96340745b0 msgr2=0x7f9634076ac0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:29:20.209 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:20.205+0000 7f9640ff9640 1 --2- 172.21.15.37:0/322652459 >> [v2:172.21.15.203:6832/2418205237,v1:172.21.15.203:6833/2418205237] conn(0x7f96340745b0 0x7f9634076ac0 secure :-1 s=READY pgs=84 cs=0 l=1 rev1=1 crypto rx=0x7f96440010a0 tx=0x7f9644009040 comp rx=0 tx=0).stop 2024-04-25T10:29:20.210 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:20.207+0000 7f4db2ffd640 1 -- 172.21.15.37:0/2431781660 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 4} v 0) -- 0x7f4dac005580 con 0x7f4dcc15aa20 2024-04-25T10:29:20.210 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:20.205+0000 7f9640ff9640 1 -- 172.21.15.37:0/322652459 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f96540cb500 msgr2=0x7f96540cb910 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:29:20.210 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:20.205+0000 7f9640ff9640 1 --2- 172.21.15.37:0/322652459 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f96540cb500 0x7f96540cb910 secure :-1 s=READY pgs=82 cs=0 l=1 rev1=1 crypto rx=0x7f96540738f0 tx=0x7f964c00f580 comp rx=0 tx=0).stop 2024-04-25T10:29:20.210 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:20.207+0000 7f9640ff9640 1 -- 172.21.15.37:0/322652459 shutdown_connections 2024-04-25T10:29:20.210 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:20.207+0000 7f9640ff9640 1 --2- 172.21.15.37:0/322652459 >> [v2:172.21.15.203:6832/2418205237,v1:172.21.15.203:6833/2418205237] conn(0x7f96340745b0 0x7f9634076ac0 unknown :-1 s=CLOSED pgs=84 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:29:20.210 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:20.207+0000 7f9640ff9640 1 --2- 172.21.15.37:0/322652459 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f96540cb500 0x7f96540cb910 unknown :-1 s=CLOSED pgs=82 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:29:20.211 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:20.207+0000 7f9640ff9640 1 --2- 172.21.15.37:0/322652459 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f96540c7ae0 0x7f96540c9940 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:29:20.211 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:20.207+0000 7f9640ff9640 1 --2- 172.21.15.37:0/322652459 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f963c004800 0x7f96540b1d30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:29:20.211 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:20.207+0000 7f9640ff9640 1 -- 172.21.15.37:0/322652459 >> 172.21.15.37:0/322652459 conn(0x7f96540d02b0 msgr2=0x7f96540717d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:29:20.211 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:20.208+0000 7f4dc97fa640 1 -- 172.21.15.37:0/2431781660 <== mon.0 v2:172.21.15.37:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 4}]=0 v0) ==== 74+0+12 (secure 0 0 0) 0x7f4dc00889b0 con 0x7f4dcc15aa20 2024-04-25T10:29:20.211 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:20.208+0000 7f3e3b2ec640 1 -- 172.21.15.37:0/624744560 --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 5} v 0) -- 0x7f3e1c005580 con 0x7f3e34072ee0 2024-04-25T10:29:20.212 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:20.208+0000 7f3affd72640 1 -- 172.21.15.37:0/2546797036 >> [v2:172.21.15.203:6832/2418205237,v1:172.21.15.203:6833/2418205237] conn(0x7f3adc074530 msgr2=0x7f3adc076a40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:29:20.212 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:20.208+0000 7f3affd72640 1 --2- 172.21.15.37:0/2546797036 >> [v2:172.21.15.203:6832/2418205237,v1:172.21.15.203:6833/2418205237] conn(0x7f3adc074530 0x7f3adc076a40 secure :-1 s=READY pgs=86 cs=0 l=1 rev1=1 crypto rx=0x7f3af4037000 tx=0x7f3af4030000 comp rx=0 tx=0).stop 2024-04-25T10:29:20.212 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:20.208+0000 7f9640ff9640 1 -- 172.21.15.37:0/322652459 shutdown_connections 2024-04-25T10:29:20.212 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:20.208+0000 7f3affd72640 1 -- 172.21.15.37:0/2546797036 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f3af80b0ba0 msgr2=0x7f3af809e940 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:29:20.212 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:20.208+0000 7f9640ff9640 1 -- 172.21.15.37:0/322652459 wait complete. 2024-04-25T10:29:20.213 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:20.208+0000 7f3affd72640 1 --2- 172.21.15.37:0/2546797036 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f3af80b0ba0 0x7f3af809e940 secure :-1 s=READY pgs=37 cs=0 l=1 rev1=1 crypto rx=0x7f3ae800b3f0 tx=0x7f3ae800b8b0 comp rx=0 tx=0).stop 2024-04-25T10:29:20.213 INFO:teuthology.orchestra.run.smithi037.stdout:55834574854 2024-04-25T10:29:20.213 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:20.208+0000 7f3affd72640 1 -- 172.21.15.37:0/2546797036 shutdown_connections 2024-04-25T10:29:20.213 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:20.208+0000 7f3affd72640 1 --2- 172.21.15.37:0/2546797036 >> [v2:172.21.15.203:6832/2418205237,v1:172.21.15.203:6833/2418205237] conn(0x7f3adc074530 0x7f3adc076a40 unknown :-1 s=CLOSED pgs=86 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:29:20.213 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:20.208+0000 7f3affd72640 1 --2- 172.21.15.37:0/2546797036 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f3af80d1db0 0x7f3af815e0d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:29:20.214 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:20.208+0000 7f3affd72640 1 --2- 172.21.15.37:0/2546797036 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f3af80b0ba0 0x7f3af809e940 unknown :-1 s=CLOSED pgs=37 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:29:20.214 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:20.208+0000 7f3affd72640 1 --2- 172.21.15.37:0/2546797036 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f3af809b700 0x7f3af8149370 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:29:20.214 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:20.208+0000 7f3affd72640 1 -- 172.21.15.37:0/2546797036 >> 172.21.15.37:0/2546797036 conn(0x7f3af8073ba0 msgr2=0x7f3af815ba00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:29:20.214 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:20.208+0000 7f3e2affd640 1 -- 172.21.15.37:0/624744560 <== mon.1 v2:172.21.15.203:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 5}]=0 v0) ==== 74+0+12 (secure 0 0 0) 0x7f3e34070a00 con 0x7f3e34072ee0 2024-04-25T10:29:20.214 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:20.208+0000 7f3affd72640 1 -- 172.21.15.37:0/2546797036 shutdown_connections 2024-04-25T10:29:20.214 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:20.208+0000 7f3affd72640 1 -- 172.21.15.37:0/2546797036 wait complete. 2024-04-25T10:29:20.215 INFO:teuthology.orchestra.run.smithi037.stdout:64424509448 2024-04-25T10:29:20.216 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:20.211+0000 7f4db2ffd640 1 -- 172.21.15.37:0/2431781660 >> [v2:172.21.15.203:6832/2418205237,v1:172.21.15.203:6833/2418205237] conn(0x7f4db8074500 msgr2=0x7f4db8076a10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:29:20.216 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:20.211+0000 7f4db2ffd640 1 --2- 172.21.15.37:0/2431781660 >> [v2:172.21.15.203:6832/2418205237,v1:172.21.15.203:6833/2418205237] conn(0x7f4db8074500 0x7f4db8076a10 secure :-1 s=READY pgs=85 cs=0 l=1 rev1=1 crypto rx=0x7f4dbc007fd0 tx=0x7f4dbc007dc0 comp rx=0 tx=0).stop 2024-04-25T10:29:20.216 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:20.211+0000 7f4db2ffd640 1 -- 172.21.15.37:0/2431781660 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f4dcc15aa20 msgr2=0x7f4dcc0cef30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:29:20.216 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:20.211+0000 7f4db2ffd640 1 --2- 172.21.15.37:0/2431781660 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f4dcc15aa20 0x7f4dcc0cef30 secure :-1 s=READY pgs=121 cs=0 l=1 rev1=1 crypto rx=0x7f4dc003a000 tx=0x7f4dc00046c0 comp rx=0 tx=0).stop 2024-04-25T10:29:20.216 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:20.211+0000 7f4db2ffd640 1 -- 172.21.15.37:0/2431781660 shutdown_connections 2024-04-25T10:29:20.216 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:20.212+0000 7f4db2ffd640 1 --2- 172.21.15.37:0/2431781660 >> [v2:172.21.15.203:6832/2418205237,v1:172.21.15.203:6833/2418205237] conn(0x7f4db8074500 0x7f4db8076a10 unknown :-1 s=CLOSED pgs=85 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:29:20.216 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:20.212+0000 7f4db2ffd640 1 --2- 172.21.15.37:0/2431781660 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f4dcc15aa20 0x7f4dcc0cef30 unknown :-1 s=CLOSED pgs=121 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:29:20.216 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:20.212+0000 7f4db2ffd640 1 --2- 172.21.15.37:0/2431781660 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f4dcc0d0d10 0x7f4dcc0ce9f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:29:20.217 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:20.212+0000 7f4db2ffd640 1 --2- 172.21.15.37:0/2431781660 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f4dcc0b7750 0x7f4dcc154d10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:29:20.217 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:20.212+0000 7f4db2ffd640 1 -- 172.21.15.37:0/2431781660 >> 172.21.15.37:0/2431781660 conn(0x7f4dcc0d2750 msgr2=0x7f4dcc157100 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:29:20.217 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:20.212+0000 7f3e3b2ec640 1 -- 172.21.15.37:0/624744560 >> [v2:172.21.15.203:6832/2418205237,v1:172.21.15.203:6833/2418205237] conn(0x7f3e180748e0 msgr2=0x7f3e18076df0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:29:20.217 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:20.212+0000 7f4db2ffd640 1 -- 172.21.15.37:0/2431781660 shutdown_connections 2024-04-25T10:29:20.217 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:20.212+0000 7f4db2ffd640 1 -- 172.21.15.37:0/2431781660 wait complete. 2024-04-25T10:29:20.219 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:20.212+0000 7f3e3b2ec640 1 --2- 172.21.15.37:0/624744560 >> [v2:172.21.15.203:6832/2418205237,v1:172.21.15.203:6833/2418205237] conn(0x7f3e180748e0 0x7f3e18076df0 secure :-1 s=READY pgs=82 cs=0 l=1 rev1=1 crypto rx=0x7f3e24001bf0 tx=0x7f3e24009040 comp rx=0 tx=0).stop 2024-04-25T10:29:20.219 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:20.212+0000 7f3e3b2ec640 1 -- 172.21.15.37:0/624744560 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f3e34072ee0 msgr2=0x7f3e34162bd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:29:20.219 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:20.212+0000 7f3e3b2ec640 1 --2- 172.21.15.37:0/624744560 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f3e34072ee0 0x7f3e34162bd0 secure :-1 s=READY pgs=83 cs=0 l=1 rev1=1 crypto rx=0x7f3e3000c410 tx=0x7f3e3000c8d0 comp rx=0 tx=0).stop 2024-04-25T10:29:20.219 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:20.212+0000 7f3e3b2ec640 1 -- 172.21.15.37:0/624744560 shutdown_connections 2024-04-25T10:29:20.219 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:20.212+0000 7f3e3b2ec640 1 --2- 172.21.15.37:0/624744560 >> [v2:172.21.15.203:6832/2418205237,v1:172.21.15.203:6833/2418205237] conn(0x7f3e180748e0 0x7f3e18076df0 secure :-1 s=CLOSED pgs=82 cs=0 l=1 rev1=1 crypto rx=0x7f3e24001bf0 tx=0x7f3e24009040 comp rx=0 tx=0).stop 2024-04-25T10:29:20.219 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:20.212+0000 7f3e3b2ec640 1 --2- 172.21.15.37:0/624744560 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f3e34164c40 0x7f3e340aa160 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:29:20.219 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:20.213+0000 7f3e3b2ec640 1 --2- 172.21.15.37:0/624744560 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f3e34072ee0 0x7f3e34162bd0 unknown :-1 s=CLOSED pgs=83 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:29:20.219 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:20.213+0000 7f3e3b2ec640 1 --2- 172.21.15.37:0/624744560 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f3e340705f0 0x7f3e342060e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:29:20.219 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:20.213+0000 7f3e3b2ec640 1 -- 172.21.15.37:0/624744560 >> 172.21.15.37:0/624744560 conn(0x7f3e3406cb70 msgr2=0x7f3e34160050 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:29:20.219 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:20.213+0000 7f3e3b2ec640 1 -- 172.21.15.37:0/624744560 shutdown_connections 2024-04-25T10:29:20.219 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:20.213+0000 7f3e3b2ec640 1 -- 172.21.15.37:0/624744560 wait complete. 2024-04-25T10:29:20.220 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:20.214+0000 7f4ad8ff9640 1 -- 172.21.15.37:0/3186560673 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 7} v 0) -- 0x7f4ac4005850 con 0x7f4aec0b4fd0 2024-04-25T10:29:20.220 INFO:teuthology.orchestra.run.smithi037.stdout:64424509446 2024-04-25T10:29:20.220 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:20.215+0000 7f4adaffd640 1 -- 172.21.15.37:0/3186560673 <== mon.0 v2:172.21.15.37:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 7}]=0 v0) ==== 74+0+12 (secure 0 0 0) 0x7f4adc05ff60 con 0x7f4aec0b4fd0 2024-04-25T10:29:20.221 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:20.215+0000 7f6939dc5640 1 -- 172.21.15.37:0/3380961062 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 3} v 0) -- 0x7f6914005850 con 0x7f69340ce6a0 2024-04-25T10:29:20.221 INFO:tasks.ceph.ceph_manager.ceph:need seq 64424509448 got 64424509447 for osd.6 2024-04-25T10:29:20.222 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:20.216+0000 7f691bfff640 1 -- 172.21.15.37:0/3380961062 <== mon.0 v2:172.21.15.37:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 3}]=0 v0) ==== 74+0+12 (secure 0 0 0) 0x7f692c070e70 con 0x7f69340ce6a0 2024-04-25T10:29:20.222 INFO:teuthology.orchestra.run.smithi037.stdout:64424509448 2024-04-25T10:29:20.223 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:20.218+0000 7f4ad8ff9640 1 -- 172.21.15.37:0/3186560673 >> [v2:172.21.15.203:6832/2418205237,v1:172.21.15.203:6833/2418205237] conn(0x7f4acc074500 msgr2=0x7f4acc076a10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:29:20.224 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:20.218+0000 7f4ad8ff9640 1 --2- 172.21.15.37:0/3186560673 >> [v2:172.21.15.203:6832/2418205237,v1:172.21.15.203:6833/2418205237] conn(0x7f4acc074500 0x7f4acc076a10 secure :-1 s=READY pgs=88 cs=0 l=1 rev1=1 crypto rx=0x7f4ae0034000 tx=0x7f4ae002e000 comp rx=0 tx=0).stop 2024-04-25T10:29:20.224 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:20.218+0000 7f4ad8ff9640 1 -- 172.21.15.37:0/3186560673 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f4aec0b4fd0 msgr2=0x7f4aec161f90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:29:20.224 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:20.218+0000 7f4ad8ff9640 1 --2- 172.21.15.37:0/3186560673 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f4aec0b4fd0 0x7f4aec161f90 secure :-1 s=READY pgs=119 cs=0 l=1 rev1=1 crypto rx=0x7f4adc00b440 tx=0x7f4adc009f90 comp rx=0 tx=0).stop 2024-04-25T10:29:20.224 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:20.219+0000 7f4ad8ff9640 1 -- 172.21.15.37:0/3186560673 shutdown_connections 2024-04-25T10:29:20.224 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:20.219+0000 7f4ad8ff9640 1 --2- 172.21.15.37:0/3186560673 >> [v2:172.21.15.203:6832/2418205237,v1:172.21.15.203:6833/2418205237] conn(0x7f4acc074500 0x7f4acc076a10 unknown :-1 s=CLOSED pgs=88 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:29:20.224 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:20.219+0000 7f4ad8ff9640 1 --2- 172.21.15.37:0/3186560673 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f4aec14d8c0 0x7f4aec14bb50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:29:20.224 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:20.219+0000 7f4ad8ff9640 1 --2- 172.21.15.37:0/3186560673 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f4aec0b4fd0 0x7f4aec161f90 unknown :-1 s=CLOSED pgs=119 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:29:20.224 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:20.219+0000 7f4ad8ff9640 1 --2- 172.21.15.37:0/3186560673 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f4aec0701a0 0x7f4aec1474b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:29:20.224 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:20.219+0000 7f4ad8ff9640 1 -- 172.21.15.37:0/3186560673 >> 172.21.15.37:0/3186560673 conn(0x7f4aec0b36b0 msgr2=0x7f4aec1680a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:29:20.224 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:20.219+0000 7f4ad8ff9640 1 -- 172.21.15.37:0/3186560673 shutdown_connections 2024-04-25T10:29:20.224 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:20.219+0000 7f4ad8ff9640 1 -- 172.21.15.37:0/3186560673 wait complete. 2024-04-25T10:29:20.225 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:20.220+0000 7f6939dc5640 1 -- 172.21.15.37:0/3380961062 >> [v2:172.21.15.203:6832/2418205237,v1:172.21.15.203:6833/2418205237] conn(0x7f6920074500 msgr2=0x7f6920076a10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:29:20.226 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:20.220+0000 7f6939dc5640 1 --2- 172.21.15.37:0/3380961062 >> [v2:172.21.15.203:6832/2418205237,v1:172.21.15.203:6833/2418205237] conn(0x7f6920074500 0x7f6920076a10 secure :-1 s=READY pgs=87 cs=0 l=1 rev1=1 crypto rx=0x7f6924034000 tx=0x7f692402e000 comp rx=0 tx=0).stop 2024-04-25T10:29:20.226 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:20.220+0000 7f6939dc5640 1 -- 172.21.15.37:0/3380961062 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f69340ce6a0 msgr2=0x7f69341538d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:29:20.226 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:20.220+0000 7f6939dc5640 1 --2- 172.21.15.37:0/3380961062 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f69340ce6a0 0x7f69341538d0 secure :-1 s=READY pgs=120 cs=0 l=1 rev1=1 crypto rx=0x7f692c00efc0 tx=0x7f692c00c490 comp rx=0 tx=0).stop 2024-04-25T10:29:20.226 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:20.220+0000 7f6939dc5640 1 -- 172.21.15.37:0/3380961062 shutdown_connections 2024-04-25T10:29:20.226 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:20.220+0000 7f6939dc5640 1 --2- 172.21.15.37:0/3380961062 >> [v2:172.21.15.203:6832/2418205237,v1:172.21.15.203:6833/2418205237] conn(0x7f6920074500 0x7f6920076a10 unknown :-1 s=CLOSED pgs=87 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:29:20.226 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:20.220+0000 7f6939dc5640 1 --2- 172.21.15.37:0/3380961062 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f69340ce6a0 0x7f69341538d0 unknown :-1 s=CLOSED pgs=120 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:29:20.226 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:20.220+0000 7f6939dc5640 1 --2- 172.21.15.37:0/3380961062 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f69340b5520 0x7f6934154db0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:29:20.226 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:20.220+0000 7f6939dc5640 1 --2- 172.21.15.37:0/3380961062 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f69340b3b40 0x7f69340cc230 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:29:20.226 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:20.220+0000 7f6939dc5640 1 -- 172.21.15.37:0/3380961062 >> 172.21.15.37:0/3380961062 conn(0x7f69340b4220 msgr2=0x7f693406c540 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:29:20.226 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:20.220+0000 7f6939dc5640 1 -- 172.21.15.37:0/3380961062 shutdown_connections 2024-04-25T10:29:20.226 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:20.220+0000 7f6939dc5640 1 -- 172.21.15.37:0/3380961062 wait complete. 2024-04-25T10:29:20.228 INFO:tasks.ceph.ceph_manager.ceph:need seq 64424509448 got 64424509447 for osd.0 2024-04-25T10:29:20.235 INFO:tasks.ceph.ceph_manager.ceph:need seq 64424509447 got 64424509446 for osd.1 2024-04-25T10:29:20.236 INFO:tasks.ceph.ceph_manager.ceph:need seq 64424509449 got 64424509448 for osd.2 2024-04-25T10:29:20.237 INFO:tasks.ceph.ceph_manager.ceph:need seq 64424509449 got 64424509448 for osd.5 2024-04-25T10:29:20.238 INFO:tasks.ceph.ceph_manager.ceph:need seq 55834574856 got 55834574854 for osd.4 2024-04-25T10:29:20.239 INFO:tasks.ceph.ceph_manager.ceph:need seq 64424509449 got 64424509448 for osd.3 2024-04-25T10:29:20.241 INFO:tasks.ceph.ceph_manager.ceph:need seq 64424509447 got 64424509446 for osd.7 2024-04-25T10:29:21.223 DEBUG:teuthology.orchestra.run.smithi037:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.6 2024-04-25T10:29:21.229 DEBUG:teuthology.orchestra.run.smithi037:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.0 2024-04-25T10:29:21.235 DEBUG:teuthology.orchestra.run.smithi037:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.1 2024-04-25T10:29:21.237 DEBUG:teuthology.orchestra.run.smithi037:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.2 2024-04-25T10:29:21.237 DEBUG:teuthology.orchestra.run.smithi037:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.5 2024-04-25T10:29:21.239 DEBUG:teuthology.orchestra.run.smithi037:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.4 2024-04-25T10:29:21.240 DEBUG:teuthology.orchestra.run.smithi037:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.3 2024-04-25T10:29:21.242 DEBUG:teuthology.orchestra.run.smithi037:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.7 2024-04-25T10:29:21.421 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:21.418+0000 7fdc44d4e640 1 Processor -- start 2024-04-25T10:29:21.422 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:21.419+0000 7fdc44d4e640 1 -- start start 2024-04-25T10:29:21.425 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:21.423+0000 7fdc44d4e640 1 --2- >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7fdc4014ab00 0x7fdc400701a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:29:21.426 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:21.423+0000 7fdc44d4e640 1 --2- >> v2:172.21.15.37:3300/0 conn(0x7fdc400706e0 0x7fdc400d03a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:29:21.426 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:21.423+0000 7fdc44d4e640 1 -- --> v1:172.21.15.203:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fdc4014af10 con 0x7fdc4014d8c0 2024-04-25T10:29:21.426 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:21.423+0000 7fdc44d4e640 1 -- --> v2:172.21.15.37:3300/0 -- mon_getmap magic: 0 -- 0x7fdc40070af0 con 0x7fdc400706e0 2024-04-25T10:29:21.426 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:21.423+0000 7fdc44d4e640 1 -- --> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] -- mon_getmap magic: 0 -- 0x7fdc4006f100 con 0x7fdc4014ab00 2024-04-25T10:29:21.427 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:21.424+0000 7fdc3e59c640 1 --1- >> v1:172.21.15.203:6789/0 conn(0x7fdc4014d8c0 0x7fdc400b4fd0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.203:6789/0 says I am v1:172.21.15.37:50944/0 (socket says 172.21.15.37:50944) 2024-04-25T10:29:21.427 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:21.424+0000 7fdc3e59c640 1 -- 172.21.15.37:0/2338976759 learned_addr learned my addr 172.21.15.37:0/2338976759 (peer_addr_for_me v1:172.21.15.37:0/0) 2024-04-25T10:29:21.427 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:21.425+0000 7fdc3ed9d640 1 --2- 172.21.15.37:0/2338976759 >> v2:172.21.15.37:3300/0 conn(0x7fdc400706e0 0x7fdc400d03a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:29:21.428 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:21.426+0000 7fdc3dd9b640 1 --2- 172.21.15.37:0/2338976759 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7fdc4014ab00 0x7fdc400701a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:29:21.430 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:21.426+0000 7fdc3ed9d640 1 -- 172.21.15.37:0/2338976759 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7fdc4014ab00 msgr2=0x7fdc400701a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:29:21.430 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:21.426+0000 7fdc3ed9d640 1 --2- 172.21.15.37:0/2338976759 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7fdc4014ab00 0x7fdc400701a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:29:21.430 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:21.426+0000 7fdc3ed9d640 1 -- 172.21.15.37:0/2338976759 >> v1:172.21.15.203:6789/0 conn(0x7fdc4014d8c0 legacy=0x7fdc400b4fd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:29:21.430 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:21.427+0000 7fdc3ed9d640 1 -- 172.21.15.37:0/2338976759 --> v2:172.21.15.37:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fdc4006f3e0 con 0x7fdc400706e0 2024-04-25T10:29:21.431 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:21.429+0000 7fdc3ed9d640 1 --2- 172.21.15.37:0/2338976759 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fdc400706e0 0x7fdc400d03a0 secure :-1 s=READY pgs=122 cs=0 l=1 rev1=1 crypto rx=0x7fdc340097a0 tx=0x7fdc3402a0a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=650190177f315d25 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:29:21.431 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:21.429+0000 7fdc3d59a640 1 -- 172.21.15.37:0/2338976759 <== mon.0 v2:172.21.15.37:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdc34037070 con 0x7fdc400706e0 2024-04-25T10:29:21.432 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:21.430+0000 7fdc3d59a640 1 -- 172.21.15.37:0/2338976759 <== mon.0 v2:172.21.15.37:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fdc3402ad30 con 0x7fdc400706e0 2024-04-25T10:29:21.432 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:21.430+0000 7fdc3d59a640 1 -- 172.21.15.37:0/2338976759 <== mon.0 v2:172.21.15.37:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdc34032660 con 0x7fdc400706e0 2024-04-25T10:29:21.437 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:21.435+0000 7fdc44d4e640 1 -- 172.21.15.37:0/2338976759 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fdc400706e0 msgr2=0x7fdc400d03a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:29:21.437 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:21.435+0000 7fdc44d4e640 1 --2- 172.21.15.37:0/2338976759 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fdc400706e0 0x7fdc400d03a0 secure :-1 s=READY pgs=122 cs=0 l=1 rev1=1 crypto rx=0x7fdc340097a0 tx=0x7fdc3402a0a0 comp rx=0 tx=0).stop 2024-04-25T10:29:21.438 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:21.435+0000 7fdc44d4e640 1 -- 172.21.15.37:0/2338976759 shutdown_connections 2024-04-25T10:29:21.438 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:21.436+0000 7fdc44d4e640 1 --2- 172.21.15.37:0/2338976759 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fdc400706e0 0x7fdc400d03a0 unknown :-1 s=CLOSED pgs=122 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:29:21.438 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:21.436+0000 7fdc44d4e640 1 --2- 172.21.15.37:0/2338976759 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7fdc4014ab00 0x7fdc400701a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:29:21.438 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:21.436+0000 7fdc44d4e640 1 -- 172.21.15.37:0/2338976759 >> 172.21.15.37:0/2338976759 conn(0x7fdc400b36b0 msgr2=0x7fdc400b3ac0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:29:21.438 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:21.436+0000 7fdc44d4e640 1 -- 172.21.15.37:0/2338976759 shutdown_connections 2024-04-25T10:29:21.438 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:21.436+0000 7fdc44d4e640 1 -- 172.21.15.37:0/2338976759 wait complete. 2024-04-25T10:29:21.442 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:21.440+0000 7fdc44d4e640 1 Processor -- start 2024-04-25T10:29:21.442 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:21.440+0000 7fdc44d4e640 1 -- start start 2024-04-25T10:29:21.445 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:21.443+0000 7fdc44d4e640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fdc400706e0 0x7fdc4014cb80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:29:21.446 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:21.443+0000 7fdc3e59c640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fdc400706e0 0x7fdc4014cb80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:29:21.446 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:21.443+0000 7fdc3e59c640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fdc400706e0 0x7fdc4014cb80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.37:3300/0 says I am v2:172.21.15.37:34976/0 (socket says 172.21.15.37:34976) 2024-04-25T10:29:21.446 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:21.443+0000 7fdc3e59c640 1 -- 172.21.15.37:0/2010102121 learned_addr learned my addr 172.21.15.37:0/2010102121 (peer_addr_for_me v2:172.21.15.37:0/0) 2024-04-25T10:29:21.446 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:21.443+0000 7fdc44d4e640 1 --2- 172.21.15.37:0/2010102121 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7fdc4014ab00 0x7fdc401479d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:29:21.446 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:21.444+0000 7fdc44d4e640 1 --2- 172.21.15.37:0/2010102121 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fdc4014d8c0 0x7fdc40161da0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:29:21.446 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:21.444+0000 7fdc44d4e640 1 -- 172.21.15.37:0/2010102121 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 -- 0x7fdc4006fc80 con 0x7fdc400706e0 2024-04-25T10:29:21.446 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:21.444+0000 7fdc3dd9b640 1 --2- 172.21.15.37:0/2010102121 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7fdc4014ab00 0x7fdc401479d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:29:21.446 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:21.444+0000 7fdc44d4e640 1 -- 172.21.15.37:0/2010102121 --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_getmap magic: 0 -- 0x7fdc4006f9a0 con 0x7fdc4014d8c0 2024-04-25T10:29:21.446 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:21.444+0000 7fdc44d4e640 1 -- 172.21.15.37:0/2010102121 --> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] -- mon_getmap magic: 0 -- 0x7fdc4014e0a0 con 0x7fdc4014ab00 2024-04-25T10:29:21.447 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:21.445+0000 7fdc3e59c640 1 -- 172.21.15.37:0/2010102121 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7fdc4014ab00 msgr2=0x7fdc401479d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:29:21.447 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:21.445+0000 7fdc3e59c640 1 --2- 172.21.15.37:0/2010102121 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7fdc4014ab00 0x7fdc401479d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:29:21.447 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:21.445+0000 7fdc3e59c640 1 -- 172.21.15.37:0/2010102121 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fdc4014d8c0 msgr2=0x7fdc40161da0 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2024-04-25T10:29:21.448 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:21.445+0000 7fdc3e59c640 1 --2- 172.21.15.37:0/2010102121 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fdc4014d8c0 0x7fdc40161da0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:29:21.448 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:21.445+0000 7fdc3e59c640 1 -- 172.21.15.37:0/2010102121 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fdc401ef0b0 con 0x7fdc400706e0 2024-04-25T10:29:21.448 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:21.446+0000 7fdc3e59c640 1 --2- 172.21.15.37:0/2010102121 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fdc400706e0 0x7fdc4014cb80 secure :-1 s=READY pgs=123 cs=0 l=1 rev1=1 crypto rx=0x7fdc3000c780 tx=0x7fdc3000cc40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:29:21.448 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:21.446+0000 7fdc2effd640 1 -- 172.21.15.37:0/2010102121 <== mon.0 v2:172.21.15.37:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdc30018070 con 0x7fdc400706e0 2024-04-25T10:29:21.450 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:21.446+0000 7fdc44d4e640 1 -- 172.21.15.37:0/2010102121 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fdc401ef390 con 0x7fdc400706e0 2024-04-25T10:29:21.450 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:21.446+0000 7fdc44d4e640 1 -- 172.21.15.37:0/2010102121 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fdc401ef950 con 0x7fdc400706e0 2024-04-25T10:29:21.451 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:21.449+0000 7fdc2effd640 1 -- 172.21.15.37:0/2010102121 <== mon.0 v2:172.21.15.37:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fdc300072d0 con 0x7fdc400706e0 2024-04-25T10:29:21.452 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:21.450+0000 7fdc2effd640 1 -- 172.21.15.37:0/2010102121 <== mon.0 v2:172.21.15.37:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdc300047e0 con 0x7fdc400706e0 2024-04-25T10:29:21.453 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:21.451+0000 7fdc2effd640 1 -- 172.21.15.37:0/2010102121 <== mon.0 v2:172.21.15.37:3300/0 4 ==== mgrmap(e 5) ==== 96782+0+0 (secure 0 0 0) 0x7fdc3001b070 con 0x7fdc400706e0 2024-04-25T10:29:21.454 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:21.452+0000 7fdc2effd640 1 --2- 172.21.15.37:0/2010102121 >> [v2:172.21.15.203:6832/2418205237,v1:172.21.15.203:6833/2418205237] conn(0x7fdc200745b0 0x7fdc20076ac0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:29:21.454 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:21.452+0000 7fdc44d4e640 1 -- 172.21.15.37:0/2010102121 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fdc24005290 con 0x7fdc400706e0 2024-04-25T10:29:21.464 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:21.454+0000 7fdc2effd640 1 -- 172.21.15.37:0/2010102121 <== mon.0 v2:172.21.15.37:3300/0 5 ==== osd_map(23..23 src has 1..23) ==== 5709+0+0 (secure 0 0 0) 0x7fdc30094730 con 0x7fdc400706e0 2024-04-25T10:29:21.464 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:21.462+0000 7fdc3dd9b640 1 --2- 172.21.15.37:0/2010102121 >> [v2:172.21.15.203:6832/2418205237,v1:172.21.15.203:6833/2418205237] conn(0x7fdc200745b0 0x7fdc20076ac0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:29:21.465 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:21.463+0000 7fdc2effd640 1 -- 172.21.15.37:0/2010102121 <== mon.0 v2:172.21.15.37:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187246 (secure 0 0 0) 0x7fdc3005e850 con 0x7fdc400706e0 2024-04-25T10:29:21.467 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:21.465+0000 7fdc3dd9b640 1 --2- 172.21.15.37:0/2010102121 >> [v2:172.21.15.203:6832/2418205237,v1:172.21.15.203:6833/2418205237] conn(0x7fdc200745b0 0x7fdc20076ac0 secure :-1 s=READY pgs=89 cs=0 l=1 rev1=1 crypto rx=0x7fdc40164390 tx=0x7fdc3800b040 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:29:21.492 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:21.487+0000 7f98dc89f640 1 Processor -- start 2024-04-25T10:29:21.492 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:21.487+0000 7f98dc89f640 1 -- start start 2024-04-25T10:29:21.492 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:21.489+0000 7f98dc89f640 1 --2- >> v2:172.21.15.37:3300/0 conn(0x7f98d806cc50 0x7f98d80b0060 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:29:21.492 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:21.489+0000 7f98dc89f640 1 -- --> v1:172.21.15.37:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f98d809bb10 con 0x7f98d806d060 2024-04-25T10:29:21.492 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:21.489+0000 7f98dc89f640 1 -- --> v1:172.21.15.203:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f98d80b2840 con 0x7f98d809b700 2024-04-25T10:29:21.492 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:21.489+0000 7f98dc89f640 1 -- --> v2:172.21.15.37:3300/0 -- mon_getmap magic: 0 -- 0x7f98d8160bc0 con 0x7f98d806cc50 2024-04-25T10:29:21.493 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:21.490+0000 7f98d6ffd640 1 --2- >> v2:172.21.15.37:3300/0 conn(0x7f98d806cc50 0x7f98d80b0060 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:29:21.493 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:21.490+0000 7f98d6ffd640 1 --2- >> v2:172.21.15.37:3300/0 conn(0x7f98d806cc50 0x7f98d80b0060 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.37:3300/0 says I am v2:172.21.15.37:34992/0 (socket says 172.21.15.37:34992) 2024-04-25T10:29:21.493 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:21.490+0000 7f98d6ffd640 1 -- 172.21.15.37:0/3318005116 learned_addr learned my addr 172.21.15.37:0/3318005116 (peer_addr_for_me v2:172.21.15.37:0/0) 2024-04-25T10:29:21.493 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:21.490+0000 7f98d6ffd640 1 -- 172.21.15.37:0/3318005116 >> v1:172.21.15.203:6789/0 conn(0x7f98d809b700 legacy=0x7f98d806c540 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:29:21.493 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:21.490+0000 7f98d6ffd640 1 -- 172.21.15.37:0/3318005116 >> v1:172.21.15.37:6789/0 conn(0x7f98d806d060 legacy=0x7f98d80b05a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:29:21.493 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:21.490+0000 7f98d6ffd640 1 -- 172.21.15.37:0/3318005116 --> v2:172.21.15.37:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f98d80b2b20 con 0x7f98d806cc50 2024-04-25T10:29:21.494 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:21.491+0000 7fc43a897640 1 Processor -- start 2024-04-25T10:29:21.494 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:21.491+0000 7f98d6ffd640 1 --2- 172.21.15.37:0/3318005116 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f98d806cc50 0x7f98d80b0060 secure :-1 s=READY pgs=124 cs=0 l=1 rev1=1 crypto rx=0x7f98d80b4f00 tx=0x7f98cc02c6d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b98513f4b9b06c1 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:29:21.494 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:21.492+0000 7f98d67fc640 1 -- 172.21.15.37:0/3318005116 <== mon.0 v2:172.21.15.37:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f98cc03a070 con 0x7f98d806cc50 2024-04-25T10:29:21.494 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:21.491+0000 7fc43a897640 1 -- start start 2024-04-25T10:29:21.495 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:21.492+0000 7f98d67fc640 1 -- 172.21.15.37:0/3318005116 <== mon.0 v2:172.21.15.37:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f98cc032480 con 0x7f98d806cc50 2024-04-25T10:29:21.495 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:21.492+0000 7f98d67fc640 1 -- 172.21.15.37:0/3318005116 <== mon.0 v2:172.21.15.37:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f98cc032830 con 0x7f98d806cc50 2024-04-25T10:29:21.495 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:21.493+0000 7f772591c640 1 Processor -- start 2024-04-25T10:29:21.495 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:21.492+0000 7f03c6d3b640 1 Processor -- start 2024-04-25T10:29:21.496 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:21.493+0000 7f772591c640 1 -- start start 2024-04-25T10:29:21.496 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:21.493+0000 7f98dc89f640 1 -- 172.21.15.37:0/3318005116 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f98d806cc50 msgr2=0x7f98d80b0060 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:29:21.496 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:21.493+0000 7f98dc89f640 1 --2- 172.21.15.37:0/3318005116 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f98d806cc50 0x7f98d80b0060 secure :-1 s=READY pgs=124 cs=0 l=1 rev1=1 crypto rx=0x7f98d80b4f00 tx=0x7f98cc02c6d0 comp rx=0 tx=0).stop 2024-04-25T10:29:21.497 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:21.494+0000 7f98dc89f640 1 -- 172.21.15.37:0/3318005116 shutdown_connections 2024-04-25T10:29:21.497 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:21.494+0000 7f98dc89f640 1 --2- 172.21.15.37:0/3318005116 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f98d806cc50 0x7f98d80b0060 unknown :-1 s=CLOSED pgs=124 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:29:21.497 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:21.493+0000 7f03c6d3b640 1 -- start start 2024-04-25T10:29:21.497 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:21.494+0000 7f98dc89f640 1 -- 172.21.15.37:0/3318005116 >> 172.21.15.37:0/3318005116 conn(0x7f98d8073960 msgr2=0x7f98d80b6e10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:29:21.498 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:21.494+0000 7f31d6955640 1 Processor -- start 2024-04-25T10:29:21.498 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:21.494+0000 7f98dc89f640 1 -- 172.21.15.37:0/3318005116 shutdown_connections 2024-04-25T10:29:21.498 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:21.494+0000 7f31d6955640 1 -- start start 2024-04-25T10:29:21.499 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:21.494+0000 7fc43a897640 1 --2- >> v2:172.21.15.37:3300/0 conn(0x7fc434161da0 0x7fc4341652d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:29:21.499 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:21.494+0000 7fc43a897640 1 --2- >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7fc434165810 0x7fc434072ee0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:29:21.499 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:21.494+0000 7fc43a897640 1 -- --> v1:172.21.15.37:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fc4341688e0 con 0x7fc4340b0bc0 2024-04-25T10:29:21.499 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:21.494+0000 7fc43a897640 1 -- --> v2:172.21.15.37:3300/0 -- mon_getmap magic: 0 -- 0x7fc434168bc0 con 0x7fc434161da0 2024-04-25T10:29:21.499 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:21.494+0000 7fc43a897640 1 -- --> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] -- mon_getmap magic: 0 -- 0x7fc434168ea0 con 0x7fc434165810 2024-04-25T10:29:21.500 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:21.495+0000 7f03c6d3b640 1 --2- >> v2:172.21.15.203:3300/0 conn(0x7f03b8047db0 0x7f03b80e7d50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:29:21.500 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:21.495+0000 7fc439094640 1 --2- >> v2:172.21.15.37:3300/0 conn(0x7fc434161da0 0x7fc4341652d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:29:21.500 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:21.495+0000 7fc439895640 1 --1- >> v1:172.21.15.37:6789/0 conn(0x7fc4340b0bc0 0x7fc434168090 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.37:6789/0 says I am v1:172.21.15.37:59774/0 (socket says 172.21.15.37:59774) 2024-04-25T10:29:21.500 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:21.495+0000 7fc439895640 1 -- 172.21.15.37:0/1606251726 learned_addr learned my addr 172.21.15.37:0/1606251726 (peer_addr_for_me v1:172.21.15.37:0/0) 2024-04-25T10:29:21.500 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:21.495+0000 7fc43a096640 1 --2- 172.21.15.37:0/1606251726 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7fc434165810 0x7fc434072ee0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:29:21.501 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:21.495+0000 7f03c6d3b640 1 --2- >> v2:172.21.15.37:3300/0 conn(0x7f03b8034060 0x7f03b80065e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:29:21.501 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:21.496+0000 7f03c5d39640 1 --2- >> v2:172.21.15.203:3300/0 conn(0x7f03b8047db0 0x7f03b80e7d50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:29:21.501 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:21.496+0000 7fb658134640 1 Processor -- start 2024-04-25T10:29:21.502 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:21.496+0000 7f772591c640 1 --2- >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f7718038b70 0x7f7718006450 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:29:21.502 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:21.496+0000 7f772591c640 1 --2- >> v2:172.21.15.203:3300/0 conn(0x7f7718038f80 0x7f7718007bb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:29:21.502 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:21.496+0000 7f772591c640 1 -- --> v1:172.21.15.203:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f77180084e0 con 0x7f7718006d50 2024-04-25T10:29:21.502 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:21.496+0000 7f772591c640 1 -- --> v2:172.21.15.203:3300/0 -- mon_getmap magic: 0 -- 0x7f77180541f0 con 0x7f7718038f80 2024-04-25T10:29:21.502 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:21.496+0000 7f772591c640 1 -- --> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] -- mon_getmap magic: 0 -- 0x7f77180538c0 con 0x7f7718038b70 2024-04-25T10:29:21.502 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:21.494+0000 7f98dc89f640 1 -- 172.21.15.37:0/3318005116 wait complete. 2024-04-25T10:29:21.503 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:21.496+0000 7f771effd640 1 --1- >> v1:172.21.15.203:6789/0 conn(0x7f7718006d50 0x7f77180dfbd0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.203:6789/0 says I am v1:172.21.15.37:50964/0 (socket says 172.21.15.37:50964) 2024-04-25T10:29:21.503 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:21.496+0000 7f771effd640 1 -- 172.21.15.37:0/3609844798 learned_addr learned my addr 172.21.15.37:0/3609844798 (peer_addr_for_me v1:172.21.15.37:0/0) 2024-04-25T10:29:21.503 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:21.496+0000 7f6a6522a640 1 Processor -- start 2024-04-25T10:29:21.503 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:21.496+0000 7fb658134640 1 -- start start 2024-04-25T10:29:21.504 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:21.496+0000 7f03c5d39640 1 --2- >> v2:172.21.15.203:3300/0 conn(0x7f03b8047db0 0x7f03b80e7d50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.203:3300/0 says I am v2:172.21.15.37:49528/0 (socket says 172.21.15.37:49528) 2024-04-25T10:29:21.504 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:21.496+0000 7f771dffb640 1 -- 172.21.15.37:0/3609844798 <== mon.1 v1:172.21.15.203:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 1168663220 0 0) 0x7f77180084e0 con 0x7f7718006d50 2024-04-25T10:29:21.504 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:21.496+0000 7f771dffb640 1 -- 172.21.15.37:0/3609844798 --> v1:172.21.15.203:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f7708003520 con 0x7f7718006d50 2024-04-25T10:29:21.505 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:21.496+0000 7f03c5538640 1 --1- >> v1:172.21.15.37:6789/0 conn(0x7f03b8048300 0x7f03b8033950 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.37:6789/0 says I am v1:172.21.15.37:59790/0 (socket says 172.21.15.37:59790) 2024-04-25T10:29:21.505 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:21.496+0000 7f03c5d39640 1 -- 172.21.15.37:0/2400483252 learned_addr learned my addr 172.21.15.37:0/2400483252 (peer_addr_for_me v2:172.21.15.37:0/0) 2024-04-25T10:29:21.505 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:21.496+0000 7f03c6d3b640 1 -- 172.21.15.37:0/2400483252 --> v1:172.21.15.37:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f03b8006130 con 0x7f03b8048300 2024-04-25T10:29:21.505 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:21.496+0000 7f03c6d3b640 1 -- 172.21.15.37:0/2400483252 --> v2:172.21.15.37:3300/0 -- mon_getmap magic: 0 -- 0x7f03b804f690 con 0x7f03b8034060 2024-04-25T10:29:21.505 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:21.496+0000 7f31d6955640 1 --2- >> v2:172.21.15.37:3300/0 conn(0x7f31d015aa20 0x7f31d0161a80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:29:21.505 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:21.496+0000 7f31d6955640 1 --2- >> v2:172.21.15.203:3300/0 conn(0x7f31d00b7750 0x7f31d00b7b60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:29:21.505 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:21.496+0000 7f31d6955640 1 --2- >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f31d00d0d10 0x7f31d00d1120 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:29:21.505 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:21.496+0000 7f31d6955640 1 -- --> v2:172.21.15.37:3300/0 -- mon_getmap magic: 0 -- 0x7f31d00d4b30 con 0x7f31d015aa20 2024-04-25T10:29:21.506 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:21.496+0000 7f31d6955640 1 -- --> v2:172.21.15.203:3300/0 -- mon_getmap magic: 0 -- 0x7f31d00b80a0 con 0x7f31d00b7750 2024-04-25T10:29:21.506 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:21.496+0000 7f31d6955640 1 -- --> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] -- mon_getmap magic: 0 -- 0x7f31d00d4240 con 0x7f31d00d0d10 2024-04-25T10:29:21.506 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:21.497+0000 7f771dffb640 1 -- 172.21.15.37:0/3609844798 <== mon.1 v1:172.21.15.203:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 3161214483 0 0) 0x7f7708003520 con 0x7f7718006d50 2024-04-25T10:29:21.506 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:21.497+0000 7f771dffb640 1 -- 172.21.15.37:0/3609844798 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f7718038b70 msgr2=0x7f7718006450 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-04-25T10:29:21.506 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:21.497+0000 7f771dffb640 1 --2- 172.21.15.37:0/3609844798 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f7718038b70 0x7f7718006450 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:29:21.506 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:21.497+0000 7f771dffb640 1 -- 172.21.15.37:0/3609844798 >> v2:172.21.15.203:3300/0 conn(0x7f7718038f80 msgr2=0x7f7718007bb0 unknown :-1 s=STATE_CONNECTING_RE l=0).mark_down 2024-04-25T10:29:21.506 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:21.497+0000 7f771dffb640 1 --2- 172.21.15.37:0/3609844798 >> v2:172.21.15.203:3300/0 conn(0x7f7718038f80 0x7f7718007bb0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:29:21.506 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:21.497+0000 7f771dffb640 1 -- 172.21.15.37:0/3609844798 --> v1:172.21.15.203:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f7718053ba0 con 0x7f7718006d50 2024-04-25T10:29:21.506 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:21.497+0000 7f771dffb640 1 -- 172.21.15.37:0/3609844798 <== mon.1 v1:172.21.15.203:6789/0 3 ==== mon_map magic: 0 ==== 413+0+0 (unknown 1577794600 0 0) 0x7f7710001670 con 0x7f7718006d50 2024-04-25T10:29:21.507 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:21.497+0000 7f771dffb640 1 -- 172.21.15.37:0/3609844798 >> v1:172.21.15.203:6789/0 conn(0x7f7718006d50 legacy=0x7f77180dfbd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:29:21.507 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:21.497+0000 7f771dffb640 1 --2- 172.21.15.37:0/3609844798 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f7718038b70 0x7f77080039d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:29:21.507 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:21.497+0000 7f771dffb640 1 --2- 172.21.15.37:0/3609844798 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f77080040e0 0x7f7708004800 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:29:21.507 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:21.497+0000 7f771dffb640 1 --2- 172.21.15.37:0/3609844798 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f7708004e60 0x7f77080052d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:29:21.507 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:21.497+0000 7f771dffb640 1 -- 172.21.15.37:0/3609844798 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 -- 0x7f77180541f0 con 0x7f7708004e60 2024-04-25T10:29:21.507 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:21.497+0000 7f771dffb640 1 -- 172.21.15.37:0/3609844798 --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_getmap magic: 0 -- 0x7f77180538c0 con 0x7f77080040e0 2024-04-25T10:29:21.507 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:21.497+0000 7f771dffb640 1 -- 172.21.15.37:0/3609844798 --> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] -- mon_getmap magic: 0 -- 0x7f7708014550 con 0x7f7718038b70 2024-04-25T10:29:21.507 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:21.497+0000 7f31d4ef2640 1 --2- >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f31d00d0d10 0x7f31d00d1120 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:29:21.507 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:21.497+0000 7f31d4ef2640 1 --2- >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f31d00d0d10 0x7f31d00d1120 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.203:3301/0 says I am v2:172.21.15.37:36844/0 (socket says 172.21.15.37:36844) 2024-04-25T10:29:21.508 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:21.497+0000 7f31d4ef2640 1 -- 172.21.15.37:0/799342495 learned_addr learned my addr 172.21.15.37:0/799342495 (peer_addr_for_me v2:172.21.15.37:0/0) 2024-04-25T10:29:21.508 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:21.497+0000 7f98dc89f640 1 Processor -- start 2024-04-25T10:29:21.508 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:21.496+0000 7f03c6d3b640 1 -- 172.21.15.37:0/2400483252 --> v2:172.21.15.203:3300/0 -- mon_getmap magic: 0 -- 0x7f03b804ffa0 con 0x7f03b8047db0 2024-04-25T10:29:21.508 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:21.497+0000 7f03c653a640 1 --2- 172.21.15.37:0/2400483252 >> v2:172.21.15.37:3300/0 conn(0x7f03b8034060 0x7f03b80065e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:29:21.509 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:21.497+0000 7f771e7fc640 1 --2- 172.21.15.37:0/3609844798 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f77080040e0 0x7f7708004800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:29:21.509 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:21.497+0000 7f98dc89f640 1 -- start start 2024-04-25T10:29:21.509 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:21.498+0000 7f771effd640 1 --2- 172.21.15.37:0/3609844798 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f7718038b70 0x7f77080039d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:29:21.510 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:21.497+0000 7f31cffff640 1 --2- 172.21.15.37:0/799342495 >> v2:172.21.15.37:3300/0 conn(0x7f31d015aa20 0x7f31d0161a80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:29:21.510 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:21.498+0000 7f771e7fc640 1 -- 172.21.15.37:0/3609844798 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f7718038b70 msgr2=0x7f77080039d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:29:21.510 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:21.498+0000 7f03c4d37640 1 -- 172.21.15.37:0/2400483252 <== mon.0 v1:172.21.15.37:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 336677813 0 0) 0x7f03b8006130 con 0x7f03b8048300 2024-04-25T10:29:21.511 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:21.498+0000 7f03c4d37640 1 -- 172.21.15.37:0/2400483252 --> v1:172.21.15.37:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f03ac003520 con 0x7f03b8048300 2024-04-25T10:29:21.511 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:21.498+0000 7f771e7fc640 1 --2- 172.21.15.37:0/3609844798 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f7718038b70 0x7f77080039d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:29:21.511 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:21.498+0000 7f771e7fc640 1 -- 172.21.15.37:0/3609844798 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f7708004e60 msgr2=0x7f77080052d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:29:21.511 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:21.498+0000 7f771e7fc640 1 --2- 172.21.15.37:0/3609844798 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f7708004e60 0x7f77080052d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:29:21.512 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:21.497+0000 7fc43a096640 1 -- 172.21.15.37:0/1606251726 >> v2:172.21.15.37:3300/0 conn(0x7fc434161da0 msgr2=0x7fc4341652d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:29:21.512 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:21.497+0000 7fc43a096640 1 --2- 172.21.15.37:0/1606251726 >> v2:172.21.15.37:3300/0 conn(0x7fc434161da0 0x7fc4341652d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:29:21.512 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:21.497+0000 7fc438893640 1 -- 172.21.15.37:0/1606251726 <== mon.0 v1:172.21.15.37:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 2596966790 0 0) 0x7fc4341688e0 con 0x7fc4340b0bc0 2024-04-25T10:29:21.512 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:21.497+0000 7fc43a096640 1 -- 172.21.15.37:0/1606251726 >> v1:172.21.15.37:6789/0 conn(0x7fc4340b0bc0 legacy=0x7fc434168090 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:29:21.512 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:21.497+0000 7fc43a096640 1 -- 172.21.15.37:0/1606251726 --> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fc434164db0 con 0x7fc434165810 2024-04-25T10:29:21.512 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:21.498+0000 7f771e7fc640 1 -- 172.21.15.37:0/3609844798 --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f7718053ba0 con 0x7f77080040e0 2024-04-25T10:29:21.513 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:21.496+0000 7f6a6522a640 1 -- start start 2024-04-25T10:29:21.513 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:21.498+0000 7fb658134640 1 --2- >> v2:172.21.15.203:3300/0 conn(0x7fb6500d07f0 0x7fb65015dc30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:29:21.513 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:21.498+0000 7f31cf7fe640 1 --2- 172.21.15.37:0/799342495 >> v2:172.21.15.203:3300/0 conn(0x7f31d00b7750 0x7f31d00b7b60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:29:21.514 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:21.498+0000 7fc43a096640 1 --2- 172.21.15.37:0/1606251726 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7fc434165810 0x7fc434072ee0 secure :-1 s=READY pgs=38 cs=0 l=1 rev1=1 crypto rx=0x7fc42c009d90 tx=0x7fc42c02a680 comp rx=0 tx=0).ready entity=mon.2 client_cookie=cb77076bb93788c4 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:29:21.514 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:21.498+0000 7f6a6522a640 1 --2- >> v2:172.21.15.37:3300/0 conn(0x7f6a600d13c0 0x7f6a600b7b50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:29:21.514 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:21.498+0000 7f6a6522a640 1 --2- >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f6a600cf650 0x7f6a600b1650 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:29:21.515 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:21.498+0000 7f6a6522a640 1 -- --> v1:172.21.15.203:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f6a600b3030 con 0x7f6a600ce3c0 2024-04-25T10:29:21.515 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:21.498+0000 7f6a6522a640 1 -- --> v2:172.21.15.37:3300/0 -- mon_getmap magic: 0 -- 0x7f6a600cfd20 con 0x7f6a600d13c0 2024-04-25T10:29:21.515 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:21.498+0000 7f6a6522a640 1 -- --> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] -- mon_getmap magic: 0 -- 0x7f6a600d0c50 con 0x7f6a600cf650 2024-04-25T10:29:21.515 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:21.499+0000 7fb658134640 1 --2- >> v2:172.21.15.37:3300/0 conn(0x7fb6500b75e0 0x7fb6500b79f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:29:21.515 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:21.498+0000 7f03c5d39640 1 -- 172.21.15.37:0/2400483252 >> v2:172.21.15.37:3300/0 conn(0x7f03b8034060 msgr2=0x7f03b80065e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:29:21.515 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:21.498+0000 7f03c5d39640 1 --2- 172.21.15.37:0/2400483252 >> v2:172.21.15.37:3300/0 conn(0x7f03b8034060 0x7f03b80065e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:29:21.515 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:21.498+0000 7f03c5d39640 1 -- 172.21.15.37:0/2400483252 >> v1:172.21.15.37:6789/0 conn(0x7f03b8048300 legacy=0x7f03b8033950 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:29:21.515 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:21.499+0000 7f03c5d39640 1 -- 172.21.15.37:0/2400483252 --> v2:172.21.15.203:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f03b8050280 con 0x7f03b8047db0 2024-04-25T10:29:21.516 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:21.499+0000 7f6a64a29640 1 --2- >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f6a600cf650 0x7f6a600b1650 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:29:21.516 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:21.499+0000 7f6a64a29640 1 --2- >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f6a600cf650 0x7f6a600b1650 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.203:3301/0 says I am v2:172.21.15.37:36858/0 (socket says 172.21.15.37:36858) 2024-04-25T10:29:21.516 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:21.499+0000 7f771e7fc640 1 --2- 172.21.15.37:0/3609844798 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f77080040e0 0x7f7708004800 secure :-1 s=READY pgs=87 cs=0 l=1 rev1=1 crypto rx=0x7f772006bde0 tx=0x7f7708005830 comp rx=0 tx=0).ready entity=mon.1 client_cookie=3cccf1ab72e95fbd server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:29:21.517 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:21.499+0000 7f6a64a29640 1 -- 172.21.15.37:0/987503817 learned_addr learned my addr 172.21.15.37:0/987503817 (peer_addr_for_me v2:172.21.15.37:0/0) 2024-04-25T10:29:21.517 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:21.499+0000 7f6a5f7fe640 1 --2- 172.21.15.37:0/987503817 >> v2:172.21.15.37:3300/0 conn(0x7f6a600d13c0 0x7f6a600b7b50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:29:21.517 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:21.499+0000 7fb658134640 1 -- --> v1:172.21.15.203:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fb6500b7f30 con 0x7fb650161090 2024-04-25T10:29:21.518 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:21.499+0000 7fb658134640 1 -- --> v2:172.21.15.37:3300/0 -- mon_getmap magic: 0 -- 0x7fb6500d4fc0 con 0x7fb6500b75e0 2024-04-25T10:29:21.518 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:21.499+0000 7fb6556cf640 1 --1- >> v1:172.21.15.203:6789/0 conn(0x7fb650161090 0x7fb6500d46c0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.203:6789/0 says I am v1:172.21.15.37:50974/0 (socket says 172.21.15.37:50974) 2024-04-25T10:29:21.518 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:21.499+0000 7fb6556cf640 1 -- 172.21.15.37:0/1323333005 learned_addr learned my addr 172.21.15.37:0/1323333005 (peer_addr_for_me v1:172.21.15.37:0/0) 2024-04-25T10:29:21.518 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:21.499+0000 7fb658134640 1 -- 172.21.15.37:0/1323333005 --> v2:172.21.15.203:3300/0 -- mon_getmap magic: 0 -- 0x7fb6500d52a0 con 0x7fb6500d07f0 2024-04-25T10:29:21.518 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:21.499+0000 7f771dffb640 1 -- 172.21.15.37:0/3609844798 <== mon.1 v2:172.21.15.203:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f77200b4070 con 0x7f77080040e0 2024-04-25T10:29:21.518 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:21.499+0000 7fb655ed0640 1 --2- 172.21.15.37:0/1323333005 >> v2:172.21.15.203:3300/0 conn(0x7fb6500d07f0 0x7fb65015dc30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:29:21.519 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:21.499+0000 7f6a5f7fe640 1 -- 172.21.15.37:0/987503817 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f6a600cf650 msgr2=0x7f6a600b1650 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:29:21.519 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:21.499+0000 7fc438893640 1 -- 172.21.15.37:0/1606251726 <== mon.2 v2:172.21.15.203:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc42c007740 con 0x7fc434165810 2024-04-25T10:29:21.519 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:21.499+0000 7f771dffb640 1 -- 172.21.15.37:0/3609844798 <== mon.1 v2:172.21.15.203:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f77200a7950 con 0x7f77080040e0 2024-04-25T10:29:21.519 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:21.499+0000 7f771dffb640 1 -- 172.21.15.37:0/3609844798 <== mon.1 v2:172.21.15.203:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f77200b4070 con 0x7f77080040e0 2024-04-25T10:29:21.520 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:21.500+0000 7fc438893640 1 -- 172.21.15.37:0/1606251726 <== mon.2 v2:172.21.15.203:3301/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fc42c0078e0 con 0x7fc434165810 2024-04-25T10:29:21.520 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:21.499+0000 7f6a5effd640 1 -- 172.21.15.37:0/987503817 <== mon.1 v1:172.21.15.203:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 4261887378 0 0) 0x7f6a600b3030 con 0x7f6a600ce3c0 2024-04-25T10:29:21.520 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:21.500+0000 7f6a5f7fe640 1 --2- 172.21.15.37:0/987503817 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f6a600cf650 0x7f6a600b1650 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:29:21.520 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:21.500+0000 7f6a5f7fe640 1 -- 172.21.15.37:0/987503817 >> v1:172.21.15.203:6789/0 conn(0x7f6a600ce3c0 legacy=0x7f6a600cef40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:29:21.520 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:21.500+0000 7f6a5f7fe640 1 -- 172.21.15.37:0/987503817 --> v2:172.21.15.37:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f6a600cfa60 con 0x7f6a600d13c0 2024-04-25T10:29:21.521 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:21.500+0000 7f98dc89f640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f98d806cc50 0x7f98d8154fe0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:29:21.521 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:21.500+0000 7fb6566d1640 1 --2- 172.21.15.37:0/1323333005 >> v2:172.21.15.37:3300/0 conn(0x7fb6500b75e0 0x7fb6500b79f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:29:21.521 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:21.500+0000 7fb6566d1640 1 -- 172.21.15.37:0/1323333005 >> v2:172.21.15.203:3300/0 conn(0x7fb6500d07f0 msgr2=0x7fb65015dc30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:29:21.521 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:21.500+0000 7fb6566d1640 1 --2- 172.21.15.37:0/1323333005 >> v2:172.21.15.203:3300/0 conn(0x7fb6500d07f0 0x7fb65015dc30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:29:21.521 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:21.500+0000 7fb6566d1640 1 -- 172.21.15.37:0/1323333005 >> v1:172.21.15.203:6789/0 conn(0x7fb650161090 legacy=0x7fb6500d46c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:29:21.521 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:21.500+0000 7fb6566d1640 1 -- 172.21.15.37:0/1323333005 --> v2:172.21.15.37:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fb650160890 con 0x7fb6500b75e0 2024-04-25T10:29:21.521 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:21.499+0000 7f31cffff640 1 -- 172.21.15.37:0/799342495 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f31d00d0d10 msgr2=0x7f31d00d1120 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:29:21.522 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:21.500+0000 7f98d77fe640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f98d806cc50 0x7f98d8154fe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:29:21.522 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:21.500+0000 7f98d77fe640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f98d806cc50 0x7f98d8154fe0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.37:3300/0 says I am v2:172.21.15.37:35048/0 (socket says 172.21.15.37:35048) 2024-04-25T10:29:21.522 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:21.500+0000 7f98d77fe640 1 -- 172.21.15.37:0/2976130512 learned_addr learned my addr 172.21.15.37:0/2976130512 (peer_addr_for_me v2:172.21.15.37:0/0) 2024-04-25T10:29:21.522 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:21.500+0000 7f98dc89f640 1 --2- 172.21.15.37:0/2976130512 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f98d806d060 0x7f98d81502b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:29:21.523 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:21.499+0000 7f31cffff640 1 --2- 172.21.15.37:0/799342495 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f31d00d0d10 0x7f31d00d1120 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:29:21.523 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:21.499+0000 7f31cffff640 1 -- 172.21.15.37:0/799342495 >> v2:172.21.15.203:3300/0 conn(0x7f31d00b7750 msgr2=0x7f31d00b7b60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:29:21.523 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:21.499+0000 7f31cffff640 1 --2- 172.21.15.37:0/799342495 >> v2:172.21.15.203:3300/0 conn(0x7f31d00b7750 0x7f31d00b7b60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:29:21.523 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:21.499+0000 7f31cffff640 1 -- 172.21.15.37:0/799342495 --> v2:172.21.15.37:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f31d00d1a00 con 0x7f31d015aa20 2024-04-25T10:29:21.523 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:21.501+0000 7f98dc89f640 1 --2- 172.21.15.37:0/2976130512 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f98d809b700 0x7f98d815a340 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:29:21.523 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:21.500+0000 7f31cffff640 1 --2- 172.21.15.37:0/799342495 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f31d015aa20 0x7f31d0161a80 secure :-1 s=READY pgs=127 cs=0 l=1 rev1=1 crypto rx=0x7f31d008e370 tx=0x7f31c002a140 comp rx=0 tx=0).ready entity=mon.0 client_cookie=94356352a94450ea server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:29:21.524 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:21.501+0000 7f98d6ffd640 1 --2- 172.21.15.37:0/2976130512 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f98d806d060 0x7f98d81502b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:29:21.524 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:21.501+0000 7fc43a897640 1 -- 172.21.15.37:0/1606251726 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7fc434165810 msgr2=0x7fc434072ee0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:29:21.524 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:21.501+0000 7fc43a897640 1 --2- 172.21.15.37:0/1606251726 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7fc434165810 0x7fc434072ee0 secure :-1 s=READY pgs=38 cs=0 l=1 rev1=1 crypto rx=0x7fc42c009d90 tx=0x7fc42c02a680 comp rx=0 tx=0).stop 2024-04-25T10:29:21.524 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:21.501+0000 7f772591c640 1 -- 172.21.15.37:0/3609844798 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f77080040e0 msgr2=0x7f7708004800 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:29:21.525 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:21.501+0000 7f31ceffd640 1 -- 172.21.15.37:0/799342495 <== mon.0 v2:172.21.15.37:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f31c0037070 con 0x7f31d015aa20 2024-04-25T10:29:21.525 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:21.501+0000 7f772591c640 1 --2- 172.21.15.37:0/3609844798 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f77080040e0 0x7f7708004800 secure :-1 s=READY pgs=87 cs=0 l=1 rev1=1 crypto rx=0x7f772006bde0 tx=0x7f7708005830 comp rx=0 tx=0).stop 2024-04-25T10:29:21.525 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:21.501+0000 7f98dc89f640 1 -- 172.21.15.37:0/2976130512 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 -- 0x7f98d8160f40 con 0x7f98d806cc50 2024-04-25T10:29:21.525 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:21.501+0000 7f03c5d39640 1 --2- 172.21.15.37:0/2400483252 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f03b8047db0 0x7f03b80e7d50 secure :-1 s=READY pgs=90 cs=0 l=1 rev1=1 crypto rx=0x7f03b800d9b0 tx=0x7f03bc02a280 comp rx=0 tx=0).ready entity=mon.1 client_cookie=666971f98fcf4fea server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:29:21.526 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:21.501+0000 7fc43a897640 1 -- 172.21.15.37:0/1606251726 shutdown_connections 2024-04-25T10:29:21.526 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:21.501+0000 7fc43a897640 1 --2- 172.21.15.37:0/1606251726 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7fc434165810 0x7fc434072ee0 unknown :-1 s=CLOSED pgs=38 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:29:21.526 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:21.501+0000 7fc43a897640 1 --2- 172.21.15.37:0/1606251726 >> v2:172.21.15.37:3300/0 conn(0x7fc434161da0 0x7fc4341652d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:29:21.526 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:21.501+0000 7f771f7fe640 1 -- 172.21.15.37:0/3609844798 reap_dead start 2024-04-25T10:29:21.526 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:21.501+0000 7f31ceffd640 1 -- 172.21.15.37:0/799342495 <== mon.0 v2:172.21.15.37:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f31c0032440 con 0x7f31d015aa20 2024-04-25T10:29:21.527 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:21.501+0000 7fc43a897640 1 -- 172.21.15.37:0/1606251726 >> 172.21.15.37:0/1606251726 conn(0x7fc43406cb70 msgr2=0x7fc4340b3290 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:29:21.527 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:21.501+0000 7f772591c640 1 -- 172.21.15.37:0/3609844798 shutdown_connections 2024-04-25T10:29:21.527 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:21.501+0000 7f772591c640 1 -- 172.21.15.37:0/3609844798 >> 172.21.15.37:0/3609844798 conn(0x7f771800aaa0 msgr2=0x7f771800aeb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:29:21.527 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:21.501+0000 7fc43a897640 1 -- 172.21.15.37:0/1606251726 shutdown_connections 2024-04-25T10:29:21.528 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:21.501+0000 7f98d7fff640 1 --2- 172.21.15.37:0/2976130512 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f98d809b700 0x7f98d815a340 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:29:21.528 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:21.501+0000 7f6a64a29640 1 --2- 172.21.15.37:0/987503817 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f6a600cf650 0x7f6a600b1650 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T10:29:21.528 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:21.501+0000 7f6a5f7fe640 1 --2- 172.21.15.37:0/987503817 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f6a600d13c0 0x7f6a600b7b50 secure :-1 s=READY pgs=128 cs=0 l=1 rev1=1 crypto rx=0x7f6a58009d90 tx=0x7f6a5802a5f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e5d2470c09276b58 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:29:21.528 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:21.501+0000 7f772591c640 1 -- 172.21.15.37:0/3609844798 shutdown_connections 2024-04-25T10:29:21.529 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:21.501+0000 7f03c4d37640 1 -- 172.21.15.37:0/2400483252 <== mon.1 v2:172.21.15.203:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f03bc037070 con 0x7f03b8047db0 2024-04-25T10:29:21.529 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:21.501+0000 7f03c4d37640 1 -- 172.21.15.37:0/2400483252 <== mon.1 v2:172.21.15.203:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f03bc032440 con 0x7f03b8047db0 2024-04-25T10:29:21.529 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:21.501+0000 7f03c4d37640 1 -- 172.21.15.37:0/2400483252 <== mon.1 v2:172.21.15.203:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f03bc032810 con 0x7f03b8047db0 2024-04-25T10:29:21.529 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:21.501+0000 7fc43a897640 1 -- 172.21.15.37:0/1606251726 wait complete. 2024-04-25T10:29:21.529 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:21.501+0000 7f772591c640 1 -- 172.21.15.37:0/3609844798 wait complete. 2024-04-25T10:29:21.530 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:21.501+0000 7f98dc89f640 1 -- 172.21.15.37:0/2976130512 --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_getmap magic: 0 -- 0x7f98d80afbd0 con 0x7f98d809b700 2024-04-25T10:29:21.530 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:21.501+0000 7f98dc89f640 1 -- 172.21.15.37:0/2976130512 --> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] -- mon_getmap magic: 0 -- 0x7f98d80af7f0 con 0x7f98d806d060 2024-04-25T10:29:21.530 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:21.502+0000 7fb6566d1640 1 --2- 172.21.15.37:0/1323333005 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fb6500b75e0 0x7fb6500b79f0 secure :-1 s=READY pgs=129 cs=0 l=1 rev1=1 crypto rx=0x7fb65008e8a0 tx=0x7fb64802a680 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c44455cfb3af21e server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:29:21.531 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:21.502+0000 7f31d6955640 1 -- 172.21.15.37:0/799342495 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f31d015aa20 msgr2=0x7f31d0161a80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:29:21.531 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:21.502+0000 7f31d6955640 1 --2- 172.21.15.37:0/799342495 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f31d015aa20 0x7f31d0161a80 secure :-1 s=READY pgs=127 cs=0 l=1 rev1=1 crypto rx=0x7f31d008e370 tx=0x7f31c002a140 comp rx=0 tx=0).stop 2024-04-25T10:29:21.531 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:21.502+0000 7f98d6ffd640 1 -- 172.21.15.37:0/2976130512 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f98d809b700 msgr2=0x7f98d815a340 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:29:21.531 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:21.502+0000 7fb654ece640 1 -- 172.21.15.37:0/1323333005 <== mon.0 v2:172.21.15.37:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb648007740 con 0x7fb6500b75e0 2024-04-25T10:29:21.532 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:21.502+0000 7fb654ece640 1 -- 172.21.15.37:0/1323333005 <== mon.0 v2:172.21.15.37:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fb6480078e0 con 0x7fb6500b75e0 2024-04-25T10:29:21.532 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:21.502+0000 7fb654ece640 1 -- 172.21.15.37:0/1323333005 <== mon.0 v2:172.21.15.37:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb648007cb0 con 0x7fb6500b75e0 2024-04-25T10:29:21.532 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:21.502+0000 7f6a5effd640 1 -- 172.21.15.37:0/987503817 <== mon.0 v2:172.21.15.37:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6a58007740 con 0x7f6a600d13c0 2024-04-25T10:29:21.532 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:21.502+0000 7f6a5effd640 1 -- 172.21.15.37:0/987503817 <== mon.0 v2:172.21.15.37:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f6a580078e0 con 0x7f6a600d13c0 2024-04-25T10:29:21.532 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:21.502+0000 7f6a5effd640 1 -- 172.21.15.37:0/987503817 <== mon.0 v2:172.21.15.37:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6a58007c90 con 0x7f6a600d13c0 2024-04-25T10:29:21.532 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:21.502+0000 7f31d6955640 1 -- 172.21.15.37:0/799342495 shutdown_connections 2024-04-25T10:29:21.533 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:21.502+0000 7f31d6955640 1 --2- 172.21.15.37:0/799342495 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f31d00d0d10 0x7f31d00d1120 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:29:21.533 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:21.503+0000 7f31d6955640 1 --2- 172.21.15.37:0/799342495 >> v2:172.21.15.203:3300/0 conn(0x7f31d00b7750 0x7f31d00b7b60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:29:21.533 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:21.503+0000 7f31d6955640 1 --2- 172.21.15.37:0/799342495 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f31d015aa20 0x7f31d0161a80 unknown :-1 s=CLOSED pgs=127 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:29:21.533 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:21.503+0000 7f31d6955640 1 -- 172.21.15.37:0/799342495 >> 172.21.15.37:0/799342495 conn(0x7f31d00d2750 msgr2=0x7f31d00d2b60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:29:21.533 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:21.502+0000 7f03c6d3b640 1 -- 172.21.15.37:0/2400483252 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f03b8047db0 msgr2=0x7f03b80e7d50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:29:21.533 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:21.502+0000 7f03c6d3b640 1 --2- 172.21.15.37:0/2400483252 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f03b8047db0 0x7f03b80e7d50 secure :-1 s=READY pgs=90 cs=0 l=1 rev1=1 crypto rx=0x7f03b800d9b0 tx=0x7f03bc02a280 comp rx=0 tx=0).stop 2024-04-25T10:29:21.533 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:21.503+0000 7f03c6d3b640 1 -- 172.21.15.37:0/2400483252 shutdown_connections 2024-04-25T10:29:21.533 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:21.503+0000 7f03c6d3b640 1 --2- 172.21.15.37:0/2400483252 >> v2:172.21.15.37:3300/0 conn(0x7f03b8034060 0x7f03b80065e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:29:21.534 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:21.503+0000 7f03c6d3b640 1 --2- 172.21.15.37:0/2400483252 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f03b8047db0 0x7f03b80e7d50 unknown :-1 s=CLOSED pgs=90 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:29:21.534 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:21.503+0000 7f03c6d3b640 1 -- 172.21.15.37:0/2400483252 >> 172.21.15.37:0/2400483252 conn(0x7f03b8054cf0 msgr2=0x7f03b8055100 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:29:21.534 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:21.503+0000 7f31d6955640 1 -- 172.21.15.37:0/799342495 shutdown_connections 2024-04-25T10:29:21.534 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:21.503+0000 7f03c6d3b640 1 -- 172.21.15.37:0/2400483252 shutdown_connections 2024-04-25T10:29:21.535 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:21.502+0000 7f98d6ffd640 1 --2- 172.21.15.37:0/2976130512 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f98d809b700 0x7f98d815a340 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:29:21.535 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:21.502+0000 7f98d6ffd640 1 -- 172.21.15.37:0/2976130512 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f98d806cc50 msgr2=0x7f98d8154fe0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:29:21.535 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:21.503+0000 7f31d6955640 1 -- 172.21.15.37:0/799342495 wait complete. 2024-04-25T10:29:21.535 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:21.503+0000 7f98d6ffd640 1 --2- 172.21.15.37:0/2976130512 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f98d806cc50 0x7f98d8154fe0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:29:21.536 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:21.503+0000 7f98d6ffd640 1 -- 172.21.15.37:0/2976130512 --> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f98d8156960 con 0x7f98d806d060 2024-04-25T10:29:21.536 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:21.504+0000 7fb658134640 1 -- 172.21.15.37:0/1323333005 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fb6500b75e0 msgr2=0x7fb6500b79f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:29:21.536 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:21.504+0000 7fb658134640 1 --2- 172.21.15.37:0/1323333005 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fb6500b75e0 0x7fb6500b79f0 secure :-1 s=READY pgs=129 cs=0 l=1 rev1=1 crypto rx=0x7fb65008e8a0 tx=0x7fb64802a680 comp rx=0 tx=0).stop 2024-04-25T10:29:21.536 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:21.504+0000 7f6a6522a640 1 -- 172.21.15.37:0/987503817 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f6a600d13c0 msgr2=0x7f6a600b7b50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:29:21.536 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:21.504+0000 7f6a6522a640 1 --2- 172.21.15.37:0/987503817 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f6a600d13c0 0x7f6a600b7b50 secure :-1 s=READY pgs=128 cs=0 l=1 rev1=1 crypto rx=0x7f6a58009d90 tx=0x7f6a5802a5f0 comp rx=0 tx=0).stop 2024-04-25T10:29:21.537 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:21.504+0000 7f6a6522a640 1 -- 172.21.15.37:0/987503817 shutdown_connections 2024-04-25T10:29:21.537 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:21.504+0000 7fc43a897640 1 Processor -- start 2024-04-25T10:29:21.537 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:21.503+0000 7f98d77fe640 1 --2- 172.21.15.37:0/2976130512 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f98d806cc50 0x7f98d8154fe0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-25T10:29:21.537 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:21.503+0000 7f98d6ffd640 1 --2- 172.21.15.37:0/2976130512 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f98d806d060 0x7f98d81502b0 secure :-1 s=READY pgs=39 cs=0 l=1 rev1=1 crypto rx=0x7f98d809bd00 tx=0x7f98cc0046c0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:29:21.537 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:21.504+0000 7f6a6522a640 1 --2- 172.21.15.37:0/987503817 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f6a600cf650 0x7f6a600b1650 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:29:21.538 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:21.504+0000 7fb658134640 1 -- 172.21.15.37:0/1323333005 shutdown_connections 2024-04-25T10:29:21.538 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:21.504+0000 7f98bffff640 1 -- 172.21.15.37:0/2976130512 <== mon.2 v2:172.21.15.203:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f98cc03a040 con 0x7f98d806d060 2024-04-25T10:29:21.538 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:21.504+0000 7fb658134640 1 --2- 172.21.15.37:0/1323333005 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fb6500b75e0 0x7fb6500b79f0 unknown :-1 s=CLOSED pgs=129 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:29:21.538 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:21.504+0000 7fb658134640 1 --2- 172.21.15.37:0/1323333005 >> v2:172.21.15.203:3300/0 conn(0x7fb6500d07f0 0x7fb65015dc30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:29:21.539 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:21.504+0000 7f772591c640 1 Processor -- start 2024-04-25T10:29:21.539 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:21.504+0000 7f772591c640 1 -- start start 2024-04-25T10:29:21.539 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:21.504+0000 7fb658134640 1 -- 172.21.15.37:0/1323333005 >> 172.21.15.37:0/1323333005 conn(0x7fb6500d27c0 msgr2=0x7fb6500d2bd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:29:21.540 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:21.504+0000 7f98bffff640 1 -- 172.21.15.37:0/2976130512 <== mon.2 v2:172.21.15.203:3301/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f98cc032d60 con 0x7f98d806d060 2024-04-25T10:29:21.540 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:21.504+0000 7f98bffff640 1 -- 172.21.15.37:0/2976130512 <== mon.2 v2:172.21.15.203:3301/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f98cc003830 con 0x7f98d806d060 2024-04-25T10:29:21.540 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:21.504+0000 7f6a6522a640 1 --2- 172.21.15.37:0/987503817 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f6a600d13c0 0x7f6a600b7b50 secure :-1 s=CLOSED pgs=128 cs=0 l=1 rev1=1 crypto rx=0x7f6a58009d90 tx=0x7f6a5802a5f0 comp rx=0 tx=0).stop 2024-04-25T10:29:21.540 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:21.504+0000 7f98dc89f640 1 -- 172.21.15.37:0/2976130512 --> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f98d81558c0 con 0x7f98d806d060 2024-04-25T10:29:21.541 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:21.504+0000 7f6a6522a640 1 -- 172.21.15.37:0/987503817 >> 172.21.15.37:0/987503817 conn(0x7f6a600d2670 msgr2=0x7f6a600d2a80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:29:21.541 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:21.504+0000 7fc43a897640 1 -- start start 2024-04-25T10:29:21.541 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:21.504+0000 7f98dc89f640 1 -- 172.21.15.37:0/2976130512 --> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] -- mon_subscribe({osdmap=0}) -- 0x7f98d8155e80 con 0x7f98d806d060 2024-04-25T10:29:21.541 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:21.504+0000 7fb658134640 1 -- 172.21.15.37:0/1323333005 shutdown_connections 2024-04-25T10:29:21.542 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:21.503+0000 7f03c6d3b640 1 -- 172.21.15.37:0/2400483252 wait complete. 2024-04-25T10:29:21.542 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:21.505+0000 7f6a6522a640 1 -- 172.21.15.37:0/987503817 shutdown_connections 2024-04-25T10:29:21.542 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:21.506+0000 7fc43a897640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fc4340b0bc0 0x7fc434165be0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:29:21.542 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:21.507+0000 7fc43a897640 1 --2- >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7fc434161da0 0x7fc4340afa80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:29:21.542 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:21.507+0000 7fc43a897640 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fc43415b740 0x7fc4341637e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:29:21.542 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:21.507+0000 7fc43a897640 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 -- 0x7fc434162330 con 0x7fc4340b0bc0 2024-04-25T10:29:21.543 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:21.507+0000 7fc439094640 1 --2- >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7fc434161da0 0x7fc4340afa80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:29:21.543 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:21.507+0000 7fc439094640 1 --2- >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7fc434161da0 0x7fc4340afa80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.203:3301/0 says I am v2:172.21.15.37:36890/0 (socket says 172.21.15.37:36890) 2024-04-25T10:29:21.543 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:21.507+0000 7fc439094640 1 -- 172.21.15.37:0/3090975409 learned_addr learned my addr 172.21.15.37:0/3090975409 (peer_addr_for_me v2:172.21.15.37:0/0) 2024-04-25T10:29:21.543 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:21.507+0000 7fc439895640 1 --2- 172.21.15.37:0/3090975409 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fc4340b0bc0 0x7fc434165be0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:29:21.543 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:21.508+0000 7fc43a897640 1 -- 172.21.15.37:0/3090975409 --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_getmap magic: 0 -- 0x7fc4341621b0 con 0x7fc43415b740 2024-04-25T10:29:21.543 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:21.505+0000 7fb658134640 1 -- 172.21.15.37:0/1323333005 wait complete. 2024-04-25T10:29:21.543 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:21.506+0000 7f772591c640 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f77080040e0 0x7f771804b160 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:29:21.544 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:21.506+0000 7f772591c640 1 --2- >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f77180de210 0x7f77180de620 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:29:21.544 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:21.506+0000 7f772591c640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f7718037080 0x7f7718037490 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:29:21.544 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:21.506+0000 7f772591c640 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 -- 0x7f7718008270 con 0x7f7718037080 2024-04-25T10:29:21.544 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:21.506+0000 7f772591c640 1 -- --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_getmap magic: 0 -- 0x7f77180080f0 con 0x7f77080040e0 2024-04-25T10:29:21.544 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:21.506+0000 7f772591c640 1 -- --> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] -- mon_getmap magic: 0 -- 0x7f7718006450 con 0x7f77180de210 2024-04-25T10:29:21.544 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:21.506+0000 7f771f7fe640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f7718037080 0x7f7718037490 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:29:21.544 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:21.507+0000 7f771effd640 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f77080040e0 0x7f771804b160 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:29:21.544 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:21.507+0000 7f771f7fe640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f7718037080 0x7f7718037490 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.37:3300/0 says I am v2:172.21.15.37:35062/0 (socket says 172.21.15.37:35062) 2024-04-25T10:29:21.544 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:21.507+0000 7f771f7fe640 1 -- 172.21.15.37:0/859076513 learned_addr learned my addr 172.21.15.37:0/859076513 (peer_addr_for_me v2:172.21.15.37:0/0) 2024-04-25T10:29:21.544 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:21.507+0000 7f771e7fc640 1 --2- 172.21.15.37:0/859076513 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f77180de210 0x7f77180de620 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:29:21.544 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:21.507+0000 7f771effd640 1 -- 172.21.15.37:0/859076513 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f77180de210 msgr2=0x7f77180de620 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:29:21.544 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:21.507+0000 7f771effd640 1 --2- 172.21.15.37:0/859076513 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f77180de210 0x7f77180de620 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:29:21.544 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:21.507+0000 7f771effd640 1 -- 172.21.15.37:0/859076513 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f7718037080 msgr2=0x7f7718037490 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:29:21.544 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:21.507+0000 7f771effd640 1 --2- 172.21.15.37:0/859076513 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f7718037080 0x7f7718037490 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:29:21.545 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:21.507+0000 7f771effd640 1 -- 172.21.15.37:0/859076513 --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f771804b6a0 con 0x7f77080040e0 2024-04-25T10:29:21.545 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:21.507+0000 7f98bffff640 1 -- 172.21.15.37:0/2976130512 <== mon.2 v2:172.21.15.203:3301/0 4 ==== mgrmap(e 5) ==== 96782+0+0 (secure 0 0 0) 0x7f98cc0039d0 con 0x7f98d806d060 2024-04-25T10:29:21.545 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:21.505+0000 7f6a6522a640 1 -- 172.21.15.37:0/987503817 wait complete. 2024-04-25T10:29:21.546 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:21.507+0000 7f31d6955640 1 Processor -- start 2024-04-25T10:29:21.546 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:21.506+0000 7f03c6d3b640 1 Processor -- start 2024-04-25T10:29:21.546 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:21.506+0000 7f03c6d3b640 1 -- start start 2024-04-25T10:29:21.547 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:21.508+0000 7f98bffff640 1 --2- 172.21.15.37:0/2976130512 >> [v2:172.21.15.203:6832/2418205237,v1:172.21.15.203:6833/2418205237] conn(0x7f98c40744e0 0x7f98c40769f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:29:21.547 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:21.508+0000 7f31d6955640 1 -- start start 2024-04-25T10:29:21.547 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:21.508+0000 7f6a6522a640 1 Processor -- start 2024-04-25T10:29:21.547 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:21.508+0000 7f771effd640 1 --2- 172.21.15.37:0/859076513 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f77080040e0 0x7f771804b160 secure :-1 s=READY pgs=91 cs=0 l=1 rev1=1 crypto rx=0x7f771000e840 tx=0x7f771000ed00 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:29:21.548 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:21.508+0000 7f98bffff640 1 -- 172.21.15.37:0/2976130512 <== mon.2 v2:172.21.15.203:3301/0 5 ==== osd_map(23..23 src has 1..23) ==== 5709+0+0 (secure 0 0 0) 0x7f98cc0b6020 con 0x7f98d806d060 2024-04-25T10:29:21.548 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:21.508+0000 7f7707fff640 1 -- 172.21.15.37:0/859076513 <== mon.1 v2:172.21.15.203:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f771000c870 con 0x7f77080040e0 2024-04-25T10:29:21.548 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:21.508+0000 7f6a6522a640 1 -- start start 2024-04-25T10:29:21.549 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:21.508+0000 7fc43a897640 1 -- 172.21.15.37:0/3090975409 --> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] -- mon_getmap magic: 0 -- 0x7fc4341624b0 con 0x7fc434161da0 2024-04-25T10:29:21.549 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:21.508+0000 7fc43a096640 1 --2- 172.21.15.37:0/3090975409 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fc43415b740 0x7fc4341637e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:29:21.549 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:21.508+0000 7f7707fff640 1 -- 172.21.15.37:0/859076513 <== mon.1 v2:172.21.15.203:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f7710018070 con 0x7f77080040e0 2024-04-25T10:29:21.550 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:21.508+0000 7f772591c640 1 -- 172.21.15.37:0/859076513 --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f77180379d0 con 0x7f77080040e0 2024-04-25T10:29:21.550 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:21.508+0000 7f7707fff640 1 -- 172.21.15.37:0/859076513 <== mon.1 v2:172.21.15.203:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f771000cc50 con 0x7f77080040e0 2024-04-25T10:29:21.550 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:21.509+0000 7f772591c640 1 -- 172.21.15.37:0/859076513 --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f77180dd1e0 con 0x7f77080040e0 2024-04-25T10:29:21.550 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:21.509+0000 7f03c6d3b640 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f03b8047db0 0x7f03b80e1fa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:29:21.551 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:21.509+0000 7f03c6d3b640 1 --2- >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f03b8048300 0x7f03b80e3eb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:29:21.551 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:21.509+0000 7f03c5d39640 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f03b8047db0 0x7f03b80e1fa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:29:21.551 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:21.509+0000 7fc43a096640 1 -- 172.21.15.37:0/3090975409 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7fc434161da0 msgr2=0x7fc4340afa80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:29:21.551 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:21.509+0000 7fc43a096640 1 --2- 172.21.15.37:0/3090975409 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7fc434161da0 0x7fc4340afa80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:29:21.551 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:21.509+0000 7fc43a096640 1 -- 172.21.15.37:0/3090975409 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fc4340b0bc0 msgr2=0x7fc434165be0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:29:21.551 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:21.509+0000 7fc43a096640 1 --2- 172.21.15.37:0/3090975409 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fc4340b0bc0 0x7fc434165be0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:29:21.551 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:21.509+0000 7fc43a096640 1 -- 172.21.15.37:0/3090975409 --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fc434206870 con 0x7fc43415b740 2024-04-25T10:29:21.551 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:21.509+0000 7fc43a096640 1 --2- 172.21.15.37:0/3090975409 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fc43415b740 0x7fc4341637e0 secure :-1 s=READY pgs=92 cs=0 l=1 rev1=1 crypto rx=0x7fc42c031000 tx=0x7fc42c004a40 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:29:21.552 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:21.509+0000 7f03c5d39640 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f03b8047db0 0x7f03b80e1fa0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.203:3300/0 says I am v2:172.21.15.37:49616/0 (socket says 172.21.15.37:49616) 2024-04-25T10:29:21.552 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:21.510+0000 7fc42a7fc640 1 -- 172.21.15.37:0/3090975409 <== mon.1 v2:172.21.15.203:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc42c03f070 con 0x7fc43415b740 2024-04-25T10:29:21.552 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:21.510+0000 7fc42a7fc640 1 -- 172.21.15.37:0/3090975409 <== mon.1 v2:172.21.15.203:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fc42c044460 con 0x7fc43415b740 2024-04-25T10:29:21.552 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:21.510+0000 7f98dc89f640 1 -- 172.21.15.37:0/2976130512 --> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f98d8076bc0 con 0x7f98d806d060 2024-04-25T10:29:21.553 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:21.510+0000 7fc43a897640 1 -- 172.21.15.37:0/3090975409 --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fc43415a230 con 0x7fc43415b740 2024-04-25T10:29:21.553 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:21.510+0000 7fc43a897640 1 -- 172.21.15.37:0/3090975409 --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fc434167900 con 0x7fc43415b740 2024-04-25T10:29:21.553 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:21.508+0000 7fb658134640 1 Processor -- start 2024-04-25T10:29:21.553 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:21.508+0000 7fb658134640 1 -- start start 2024-04-25T10:29:21.553 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:21.510+0000 7fb658134640 1 --2- >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7fb6500b75e0 0x7fb650151ae0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:29:21.554 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:21.510+0000 7f31d6955640 1 --2- >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f31d00b7750 0x7f31d014bb30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:29:21.554 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:21.510+0000 7f31d6955640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f31d00d0d10 0x7f31d014c070 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:29:21.554 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:21.510+0000 7f31d6955640 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f31d015aa20 0x7f31d00cddd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:29:21.554 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:21.511+0000 7f31d6955640 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 -- 0x7f31d0161fc0 con 0x7f31d00d0d10 2024-04-25T10:29:21.554 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:21.511+0000 7f31d6955640 1 -- --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_getmap magic: 0 -- 0x7f31d00d47c0 con 0x7f31d015aa20 2024-04-25T10:29:21.554 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:21.511+0000 7f31d6955640 1 -- --> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] -- mon_getmap magic: 0 -- 0x7f31d00d1660 con 0x7f31d00b7750 2024-04-25T10:29:21.554 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:21.511+0000 7f31cffff640 1 --2- >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f31d00b7750 0x7f31d014bb30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:29:21.555 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:21.511+0000 7f31cffff640 1 --2- >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f31d00b7750 0x7f31d014bb30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.203:3301/0 says I am v2:172.21.15.37:36908/0 (socket says 172.21.15.37:36908) 2024-04-25T10:29:21.555 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:21.511+0000 7f31cffff640 1 -- 172.21.15.37:0/310476549 learned_addr learned my addr 172.21.15.37:0/310476549 (peer_addr_for_me v2:172.21.15.37:0/0) 2024-04-25T10:29:21.555 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:21.511+0000 7f31cf7fe640 1 --2- 172.21.15.37:0/310476549 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f31d00d0d10 0x7f31d014c070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:29:21.555 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:21.511+0000 7f31d4ef2640 1 --2- 172.21.15.37:0/310476549 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f31d015aa20 0x7f31d00cddd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:29:21.555 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:21.511+0000 7fc42a7fc640 1 -- 172.21.15.37:0/3090975409 <== mon.1 v2:172.21.15.203:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc42c031e60 con 0x7fc43415b740 2024-04-25T10:29:21.556 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:21.510+0000 7fb658134640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fb6500d07f0 0x7fb6501555f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:29:21.556 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:21.510+0000 7fb658134640 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fb650161090 0x7fb650156780 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:29:21.556 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:21.510+0000 7fb658134640 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 -- 0x7fb6501614a0 con 0x7fb6500d07f0 2024-04-25T10:29:21.556 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:21.510+0000 7fb658134640 1 -- --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_getmap magic: 0 -- 0x7fb650163c70 con 0x7fb650161090 2024-04-25T10:29:21.556 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:21.510+0000 7fb658134640 1 -- --> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] -- mon_getmap magic: 0 -- 0x7fb65015ad30 con 0x7fb6500b75e0 2024-04-25T10:29:21.556 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:21.510+0000 7fb6556cf640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fb6500d07f0 0x7fb6501555f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:29:21.556 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:21.510+0000 7fb6556cf640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fb6500d07f0 0x7fb6501555f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.37:3300/0 says I am v2:172.21.15.37:35076/0 (socket says 172.21.15.37:35076) 2024-04-25T10:29:21.556 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:21.510+0000 7fb6556cf640 1 -- 172.21.15.37:0/125183910 learned_addr learned my addr 172.21.15.37:0/125183910 (peer_addr_for_me v2:172.21.15.37:0/0) 2024-04-25T10:29:21.556 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:21.511+0000 7fb6556cf640 1 -- 172.21.15.37:0/125183910 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7fb6500b75e0 msgr2=0x7fb650151ae0 unknown :-1 s=STATE_CONNECTING_RE l=1).mark_down 2024-04-25T10:29:21.557 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:21.511+0000 7fb6556cf640 1 --2- 172.21.15.37:0/125183910 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7fb6500b75e0 0x7fb650151ae0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:29:21.557 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:21.511+0000 7fb6556cf640 1 -- 172.21.15.37:0/125183910 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fb650161090 msgr2=0x7fb650156780 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:29:21.557 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:21.511+0000 7fb6556cf640 1 --2- 172.21.15.37:0/125183910 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fb650161090 0x7fb650156780 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:29:21.557 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:21.511+0000 7fb6556cf640 1 -- 172.21.15.37:0/125183910 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fb65015af80 con 0x7fb6500d07f0 2024-04-25T10:29:21.557 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:21.509+0000 7f03c5d39640 1 -- 172.21.15.37:0/1015258610 learned_addr learned my addr 172.21.15.37:0/1015258610 (peer_addr_for_me v2:172.21.15.37:0/0) 2024-04-25T10:29:21.557 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:21.509+0000 7f03c5538640 1 --2- 172.21.15.37:0/1015258610 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f03b8048300 0x7f03b80e3eb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:29:21.557 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:21.510+0000 7f03c6d3b640 1 --2- 172.21.15.37:0/1015258610 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f03b8048710 0x7f03b80e43f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:29:21.558 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:21.510+0000 7f03c6d3b640 1 -- 172.21.15.37:0/1015258610 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 -- 0x7f03b80509c0 con 0x7f03b8048710 2024-04-25T10:29:21.558 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:21.510+0000 7f03c6d3b640 1 -- 172.21.15.37:0/1015258610 --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_getmap magic: 0 -- 0x7f03b8050840 con 0x7f03b8047db0 2024-04-25T10:29:21.558 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:21.510+0000 7f03c6d3b640 1 -- 172.21.15.37:0/1015258610 --> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] -- mon_getmap magic: 0 -- 0x7f03b8006b20 con 0x7f03b8048300 2024-04-25T10:29:21.558 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:21.510+0000 7f03c653a640 1 --2- 172.21.15.37:0/1015258610 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f03b8048710 0x7f03b80e43f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:29:21.558 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:21.511+0000 7f03c653a640 1 -- 172.21.15.37:0/1015258610 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f03b8048300 msgr2=0x7f03b80e3eb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:29:21.558 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:21.511+0000 7f03c653a640 1 --2- 172.21.15.37:0/1015258610 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f03b8048300 0x7f03b80e3eb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:29:21.558 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:21.511+0000 7f03c653a640 1 -- 172.21.15.37:0/1015258610 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f03b8047db0 msgr2=0x7f03b80e1fa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:29:21.558 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:21.511+0000 7f03c653a640 1 --2- 172.21.15.37:0/1015258610 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f03b8047db0 0x7f03b80e1fa0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:29:21.558 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:21.511+0000 7f03c653a640 1 -- 172.21.15.37:0/1015258610 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f03b804ecf0 con 0x7f03b8048710 2024-04-25T10:29:21.558 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:21.511+0000 7f03c653a640 1 --2- 172.21.15.37:0/1015258610 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f03b8048710 0x7f03b80e43f0 secure :-1 s=READY pgs=130 cs=0 l=1 rev1=1 crypto rx=0x7f03c006b610 tx=0x7f03c006eaa0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:29:21.559 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:21.511+0000 7f7707fff640 1 -- 172.21.15.37:0/859076513 <== mon.1 v2:172.21.15.203:3300/0 4 ==== mgrmap(e 5) ==== 96782+0+0 (secure 0 0 0) 0x7f771001d460 con 0x7f77080040e0 2024-04-25T10:29:21.559 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:21.512+0000 7fb6556cf640 1 --2- 172.21.15.37:0/125183910 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fb6500d07f0 0x7fb6501555f0 secure :-1 s=READY pgs=131 cs=0 l=1 rev1=1 crypto rx=0x7fb64000c3f0 tx=0x7fb64000c8b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:29:21.559 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:21.511+0000 7f6a6522a640 1 --2- >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f6a600ce3c0 0x7f6a6015a120 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:29:21.559 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:21.511+0000 7f6a6522a640 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f6a600cf650 0x7f6a6006c540 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:29:21.559 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:21.511+0000 7f6a6522a640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f6a60160300 0x7f6a6006ca80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:29:21.559 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:21.511+0000 7f6a6522a640 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 -- 0x7f6a600d0ea0 con 0x7f6a60160300 2024-04-25T10:29:21.560 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:21.512+0000 7f6a5f7fe640 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f6a600cf650 0x7f6a6006c540 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:29:21.560 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:21.512+0000 7f6a5f7fe640 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f6a600cf650 0x7f6a6006c540 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.203:3300/0 says I am v2:172.21.15.37:49634/0 (socket says 172.21.15.37:49634) 2024-04-25T10:29:21.560 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:21.512+0000 7f6a5f7fe640 1 -- 172.21.15.37:0/2222824938 learned_addr learned my addr 172.21.15.37:0/2222824938 (peer_addr_for_me v2:172.21.15.37:0/0) 2024-04-25T10:29:21.560 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:21.512+0000 7f6a6522a640 1 -- 172.21.15.37:0/2222824938 --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_getmap magic: 0 -- 0x7f6a600b3d20 con 0x7f6a600cf650 2024-04-25T10:29:21.560 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:21.512+0000 7f6a6522a640 1 -- 172.21.15.37:0/2222824938 --> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] -- mon_getmap magic: 0 -- 0x7f6a600b1b90 con 0x7f6a600ce3c0 2024-04-25T10:29:21.560 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:21.512+0000 7f7707fff640 1 --2- 172.21.15.37:0/859076513 >> [v2:172.21.15.203:6832/2418205237,v1:172.21.15.203:6833/2418205237] conn(0x7f770c0744b0 0x7f770c0769c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:29:21.560 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:21.512+0000 7f7707fff640 1 -- 172.21.15.37:0/859076513 <== mon.1 v2:172.21.15.203:3300/0 5 ==== osd_map(23..23 src has 1..23) ==== 5709+0+0 (secure 0 0 0) 0x7f7710095190 con 0x7f77080040e0 2024-04-25T10:29:21.561 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:21.512+0000 7f6a64a29640 1 --2- 172.21.15.37:0/2222824938 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f6a60160300 0x7f6a6006ca80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:29:21.561 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:21.512+0000 7f771e7fc640 1 --2- 172.21.15.37:0/859076513 >> [v2:172.21.15.203:6832/2418205237,v1:172.21.15.203:6833/2418205237] conn(0x7f770c0744b0 0x7f770c0769c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:29:21.561 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:21.512+0000 7f6a5f7fe640 1 -- 172.21.15.37:0/2222824938 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f6a600ce3c0 msgr2=0x7f6a6015a120 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2024-04-25T10:29:21.561 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:21.512+0000 7f6a5f7fe640 1 --2- 172.21.15.37:0/2222824938 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f6a600ce3c0 0x7f6a6015a120 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:29:21.562 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:21.512+0000 7f98d77fe640 1 --2- 172.21.15.37:0/2976130512 >> [v2:172.21.15.203:6832/2418205237,v1:172.21.15.203:6833/2418205237] conn(0x7f98c40744e0 0x7f98c40769f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:29:21.562 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:21.512+0000 7f772591c640 1 -- 172.21.15.37:0/859076513 --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f77180059f0 con 0x7f77080040e0 2024-04-25T10:29:21.562 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:21.512+0000 7f31d4ef2640 1 -- 172.21.15.37:0/310476549 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f31d00b7750 msgr2=0x7f31d014bb30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:29:21.562 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:21.512+0000 7f31d4ef2640 1 --2- 172.21.15.37:0/310476549 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f31d00b7750 0x7f31d014bb30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:29:21.562 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:21.512+0000 7f31d4ef2640 1 -- 172.21.15.37:0/310476549 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f31d00d0d10 msgr2=0x7f31d014c070 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:29:21.562 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:21.512+0000 7f31d4ef2640 1 --2- 172.21.15.37:0/310476549 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f31d00d0d10 0x7f31d014c070 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:29:21.563 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:21.512+0000 7f31d4ef2640 1 -- 172.21.15.37:0/310476549 --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f31d0149a60 con 0x7f31d015aa20 2024-04-25T10:29:21.563 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:21.513+0000 7f771e7fc640 1 --2- 172.21.15.37:0/859076513 >> [v2:172.21.15.203:6832/2418205237,v1:172.21.15.203:6833/2418205237] conn(0x7f770c0744b0 0x7f770c0769c0 secure :-1 s=READY pgs=90 cs=0 l=1 rev1=1 crypto rx=0x7f7708014520 tx=0x7f77200b1040 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:29:21.563 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:21.512+0000 7f6a5f7fe640 1 -- 172.21.15.37:0/2222824938 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f6a60160300 msgr2=0x7f6a6006ca80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:29:21.563 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:21.513+0000 7f6a5f7fe640 1 --2- 172.21.15.37:0/2222824938 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f6a60160300 0x7f6a6006ca80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:29:21.563 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:21.513+0000 7f6a5f7fe640 1 -- 172.21.15.37:0/2222824938 --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f6a6015f540 con 0x7f6a600cf650 2024-04-25T10:29:21.564 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:21.513+0000 7f31d4ef2640 1 --2- 172.21.15.37:0/310476549 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f31d015aa20 0x7f31d00cddd0 secure :-1 s=READY pgs=93 cs=0 l=1 rev1=1 crypto rx=0x7f31c8007a90 tx=0x7f31c8007f50 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:29:21.564 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:21.512+0000 7f03b67fc640 1 -- 172.21.15.37:0/1015258610 <== mon.0 v2:172.21.15.37:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f03c00ab070 con 0x7f03b8048710 2024-04-25T10:29:21.564 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:21.512+0000 7f03b67fc640 1 -- 172.21.15.37:0/1015258610 <== mon.0 v2:172.21.15.37:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f03c0075f00 con 0x7f03b8048710 2024-04-25T10:29:21.564 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:21.512+0000 7f03b67fc640 1 -- 172.21.15.37:0/1015258610 <== mon.0 v2:172.21.15.37:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f03c00762d0 con 0x7f03b8048710 2024-04-25T10:29:21.564 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:21.513+0000 7f03c6d3b640 1 -- 172.21.15.37:0/1015258610 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f03b80e24e0 con 0x7f03b8048710 2024-04-25T10:29:21.564 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:21.513+0000 7f03c6d3b640 1 -- 172.21.15.37:0/1015258610 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f03b8047530 con 0x7f03b8048710 2024-04-25T10:29:21.564 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:21.513+0000 7fb6467fc640 1 -- 172.21.15.37:0/125183910 <== mon.0 v2:172.21.15.37:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb640017070 con 0x7fb6500d07f0 2024-04-25T10:29:21.565 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:21.513+0000 7fb6467fc640 1 -- 172.21.15.37:0/125183910 <== mon.0 v2:172.21.15.37:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fb640005ce0 con 0x7fb6500d07f0 2024-04-25T10:29:21.565 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:21.513+0000 7fb6467fc640 1 -- 172.21.15.37:0/125183910 <== mon.0 v2:172.21.15.37:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb640017070 con 0x7fb6500d07f0 2024-04-25T10:29:21.565 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:21.513+0000 7fb658134640 1 -- 172.21.15.37:0/125183910 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fb65015b260 con 0x7fb6500d07f0 2024-04-25T10:29:21.565 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:21.513+0000 7fb658134640 1 -- 172.21.15.37:0/125183910 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fb650158600 con 0x7fb6500d07f0 2024-04-25T10:29:21.565 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:21.513+0000 7f98d77fe640 1 --2- 172.21.15.37:0/2976130512 >> [v2:172.21.15.203:6832/2418205237,v1:172.21.15.203:6833/2418205237] conn(0x7f98c40744e0 0x7f98c40769f0 secure :-1 s=READY pgs=91 cs=0 l=1 rev1=1 crypto rx=0x7f98c8001bb0 tx=0x7f98c8009040 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:29:21.566 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:21.513+0000 7f6a64a29640 1 --2- 172.21.15.37:0/2222824938 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f6a60160300 0x7f6a6006ca80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T10:29:21.566 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:21.513+0000 7f6a5f7fe640 1 --2- 172.21.15.37:0/2222824938 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f6a600cf650 0x7f6a6006c540 secure :-1 s=READY pgs=94 cs=0 l=1 rev1=1 crypto rx=0x7f6a5802a630 tx=0x7f6a58004a40 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:29:21.566 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:21.514+0000 7fc43a897640 1 -- 172.21.15.37:0/3090975409 --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fc41c005290 con 0x7fc43415b740 2024-04-25T10:29:21.566 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:21.514+0000 7fc42a7fc640 1 -- 172.21.15.37:0/3090975409 <== mon.1 v2:172.21.15.203:3300/0 4 ==== mgrmap(e 5) ==== 96782+0+0 (secure 0 0 0) 0x7fc42c044850 con 0x7fc43415b740 2024-04-25T10:29:21.566 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:21.514+0000 7f03c6d3b640 1 -- 172.21.15.37:0/1015258610 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f03b8005570 con 0x7f03b8048710 2024-04-25T10:29:21.567 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:21.514+0000 7fc42a7fc640 1 --2- 172.21.15.37:0/3090975409 >> [v2:172.21.15.203:6832/2418205237,v1:172.21.15.203:6833/2418205237] conn(0x7fc418074580 0x7fc418076a90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:29:21.567 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:21.514+0000 7fc42a7fc640 1 -- 172.21.15.37:0/3090975409 <== mon.1 v2:172.21.15.203:3300/0 5 ==== osd_map(23..23 src has 1..23) ==== 5709+0+0 (secure 0 0 0) 0x7fc42c0b6910 con 0x7fc43415b740 2024-04-25T10:29:21.567 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:21.515+0000 7fc439895640 1 --2- 172.21.15.37:0/3090975409 >> [v2:172.21.15.203:6832/2418205237,v1:172.21.15.203:6833/2418205237] conn(0x7fc418074580 0x7fc418076a90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:29:21.567 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:21.515+0000 7fb658134640 1 -- 172.21.15.37:0/125183910 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fb6501f6860 con 0x7fb6500d07f0 2024-04-25T10:29:21.568 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:21.515+0000 7f03b67fc640 1 -- 172.21.15.37:0/1015258610 <== mon.0 v2:172.21.15.37:3300/0 4 ==== mgrmap(e 5) ==== 96782+0+0 (secure 0 0 0) 0x7f03c00ae070 con 0x7f03b8048710 2024-04-25T10:29:21.568 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:21.515+0000 7fc439895640 1 --2- 172.21.15.37:0/3090975409 >> [v2:172.21.15.203:6832/2418205237,v1:172.21.15.203:6833/2418205237] conn(0x7fc418074580 0x7fc418076a90 secure :-1 s=READY pgs=92 cs=0 l=1 rev1=1 crypto rx=0x7fc424000f60 tx=0x7fc424007e40 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:29:21.568 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:21.516+0000 7f03b67fc640 1 --2- 172.21.15.37:0/1015258610 >> [v2:172.21.15.203:6832/2418205237,v1:172.21.15.203:6833/2418205237] conn(0x7f03a4074530 0x7f03a4076a40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:29:21.568 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:21.516+0000 7f98bffff640 1 -- 172.21.15.37:0/2976130512 <== mon.2 v2:172.21.15.203:3301/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187246 (secure 0 0 0) 0x7f98cc00b040 con 0x7f98d806d060 2024-04-25T10:29:21.569 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:21.516+0000 7f03b67fc640 1 -- 172.21.15.37:0/1015258610 <== mon.0 v2:172.21.15.37:3300/0 5 ==== osd_map(23..23 src has 1..23) ==== 5709+0+0 (secure 0 0 0) 0x7f03c0122800 con 0x7f03b8048710 2024-04-25T10:29:21.569 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:21.516+0000 7f03c5d39640 1 --2- 172.21.15.37:0/1015258610 >> [v2:172.21.15.203:6832/2418205237,v1:172.21.15.203:6833/2418205237] conn(0x7f03a4074530 0x7f03a4076a40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:29:21.569 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:21.516+0000 7fb6467fc640 1 -- 172.21.15.37:0/125183910 <== mon.0 v2:172.21.15.37:3300/0 4 ==== mgrmap(e 5) ==== 96782+0+0 (secure 0 0 0) 0x7fb64001a070 con 0x7fb6500d07f0 2024-04-25T10:29:21.569 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:21.516+0000 7f03c5d39640 1 --2- 172.21.15.37:0/1015258610 >> [v2:172.21.15.203:6832/2418205237,v1:172.21.15.203:6833/2418205237] conn(0x7f03a4074530 0x7f03a4076a40 secure :-1 s=READY pgs=93 cs=0 l=1 rev1=1 crypto rx=0x7f03bc009630 tx=0x7f03bc02e000 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:29:21.570 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:21.517+0000 7fb6467fc640 1 --2- 172.21.15.37:0/125183910 >> [v2:172.21.15.203:6832/2418205237,v1:172.21.15.203:6833/2418205237] conn(0x7fb6340748e0 0x7fb634076df0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:29:21.570 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:21.517+0000 7fb655ed0640 1 --2- 172.21.15.37:0/125183910 >> [v2:172.21.15.203:6832/2418205237,v1:172.21.15.203:6833/2418205237] conn(0x7fb6340748e0 0x7fb634076df0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:29:21.570 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:21.518+0000 7fb655ed0640 1 --2- 172.21.15.37:0/125183910 >> [v2:172.21.15.203:6832/2418205237,v1:172.21.15.203:6833/2418205237] conn(0x7fb6340748e0 0x7fb634076df0 secure :-1 s=READY pgs=94 cs=0 l=1 rev1=1 crypto rx=0x7fb6500c91e0 tx=0x7fb64c009190 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:29:21.571 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:21.518+0000 7f6a5cff9640 1 -- 172.21.15.37:0/2222824938 <== mon.1 v2:172.21.15.203:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6a58040070 con 0x7f6a600cf650 2024-04-25T10:29:21.571 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:21.519+0000 7f6a6522a640 1 -- 172.21.15.37:0/2222824938 --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f6a6015f820 con 0x7f6a600cf650 2024-04-25T10:29:21.571 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:21.519+0000 7f7707fff640 1 -- 172.21.15.37:0/859076513 <== mon.1 v2:172.21.15.203:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187246 (secure 0 0 0) 0x7f77180379d0 con 0x7f77080040e0 2024-04-25T10:29:21.571 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:21.519+0000 7f6a6522a640 1 -- 172.21.15.37:0/2222824938 --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f6a600b5e80 con 0x7f6a600cf650 2024-04-25T10:29:21.572 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:21.519+0000 7f31ccff9640 1 -- 172.21.15.37:0/310476549 <== mon.1 v2:172.21.15.203:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f31c8003990 con 0x7f31d015aa20 2024-04-25T10:29:21.572 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:21.519+0000 7f31d6955640 1 -- 172.21.15.37:0/310476549 --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f31d0149d40 con 0x7f31d015aa20 2024-04-25T10:29:21.572 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:21.519+0000 7f31d6955640 1 -- 172.21.15.37:0/310476549 --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f31d00c9c40 con 0x7f31d015aa20 2024-04-25T10:29:21.572 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:21.520+0000 7f6a6522a640 1 -- 172.21.15.37:0/2222824938 --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f6a60076bc0 con 0x7f6a600cf650 2024-04-25T10:29:21.573 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:21.520+0000 7fb6467fc640 1 -- 172.21.15.37:0/125183910 <== mon.0 v2:172.21.15.37:3300/0 5 ==== osd_map(23..23 src has 1..23) ==== 5709+0+0 (secure 0 0 0) 0x7fb640005ae0 con 0x7fb6500d07f0 2024-04-25T10:29:21.573 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:21.520+0000 7f03b67fc640 1 -- 172.21.15.37:0/1015258610 <== mon.0 v2:172.21.15.37:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187246 (secure 0 0 0) 0x7f03c00b6020 con 0x7f03b8048710 2024-04-25T10:29:21.573 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:21.521+0000 7f31d6955640 1 -- 172.21.15.37:0/310476549 --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f31d006a140 con 0x7f31d015aa20 2024-04-25T10:29:21.574 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:21.523+0000 7fc42a7fc640 1 -- 172.21.15.37:0/3090975409 <== mon.1 v2:172.21.15.203:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187246 (secure 0 0 0) 0x7fc42c081110 con 0x7fc43415b740 2024-04-25T10:29:21.574 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:21.523+0000 7f6a5cff9640 1 -- 172.21.15.37:0/2222824938 <== mon.1 v2:172.21.15.203:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f6a580078e0 con 0x7f6a600cf650 2024-04-25T10:29:21.574 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:21.523+0000 7f6a5cff9640 1 -- 172.21.15.37:0/2222824938 <== mon.1 v2:172.21.15.203:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6a5804a060 con 0x7f6a600cf650 2024-04-25T10:29:21.574 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:21.523+0000 7f31ccff9640 1 -- 172.21.15.37:0/310476549 <== mon.1 v2:172.21.15.203:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f31c800d960 con 0x7f31d015aa20 2024-04-25T10:29:21.575 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:21.523+0000 7f31ccff9640 1 -- 172.21.15.37:0/310476549 <== mon.1 v2:172.21.15.203:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f31c800dd10 con 0x7f31d015aa20 2024-04-25T10:29:21.575 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:21.526+0000 7fb6467fc640 1 -- 172.21.15.37:0/125183910 <== mon.0 v2:172.21.15.37:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187246 (secure 0 0 0) 0x7fb640021020 con 0x7fb6500d07f0 2024-04-25T10:29:21.575 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:21.526+0000 7f31ccff9640 1 -- 172.21.15.37:0/310476549 <== mon.1 v2:172.21.15.203:3300/0 4 ==== mgrmap(e 5) ==== 96782+0+0 (secure 0 0 0) 0x7f31c801a020 con 0x7f31d015aa20 2024-04-25T10:29:21.576 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:21.526+0000 7f31ccff9640 1 --2- 172.21.15.37:0/310476549 >> [v2:172.21.15.203:6832/2418205237,v1:172.21.15.203:6833/2418205237] conn(0x7f31bc0744b0 0x7f31bc0769c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:29:21.576 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:21.527+0000 7f31ccff9640 1 -- 172.21.15.37:0/310476549 <== mon.1 v2:172.21.15.203:3300/0 5 ==== osd_map(23..23 src has 1..23) ==== 5709+0+0 (secure 0 0 0) 0x7f31c80957c0 con 0x7f31d015aa20 2024-04-25T10:29:21.576 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:21.527+0000 7f31cffff640 1 --2- 172.21.15.37:0/310476549 >> [v2:172.21.15.203:6832/2418205237,v1:172.21.15.203:6833/2418205237] conn(0x7f31bc0744b0 0x7f31bc0769c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:29:21.576 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:21.528+0000 7f31cffff640 1 --2- 172.21.15.37:0/310476549 >> [v2:172.21.15.203:6832/2418205237,v1:172.21.15.203:6833/2418205237] conn(0x7f31bc0744b0 0x7f31bc0769c0 secure :-1 s=READY pgs=95 cs=0 l=1 rev1=1 crypto rx=0x7f31c002a180 tx=0x7f31c002e000 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:29:21.576 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:21.528+0000 7f6a5cff9640 1 -- 172.21.15.37:0/2222824938 <== mon.1 v2:172.21.15.203:3300/0 4 ==== mgrmap(e 5) ==== 96782+0+0 (secure 0 0 0) 0x7f6a58043070 con 0x7f6a600cf650 2024-04-25T10:29:21.576 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:21.528+0000 7f6a5cff9640 1 --2- 172.21.15.37:0/2222824938 >> [v2:172.21.15.203:6832/2418205237,v1:172.21.15.203:6833/2418205237] conn(0x7f6a4c0744e0 0x7f6a4c0769f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:29:21.576 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:21.528+0000 7f6a5cff9640 1 -- 172.21.15.37:0/2222824938 <== mon.1 v2:172.21.15.203:3300/0 5 ==== osd_map(23..23 src has 1..23) ==== 5709+0+0 (secure 0 0 0) 0x7f6a580b6bf0 con 0x7f6a600cf650 2024-04-25T10:29:21.576 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:21.529+0000 7f31ccff9640 1 -- 172.21.15.37:0/310476549 <== mon.1 v2:172.21.15.203:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187246 (secure 0 0 0) 0x7f31c8010070 con 0x7f31d015aa20 2024-04-25T10:29:21.577 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:21.529+0000 7f6a5ffff640 1 --2- 172.21.15.37:0/2222824938 >> [v2:172.21.15.203:6832/2418205237,v1:172.21.15.203:6833/2418205237] conn(0x7f6a4c0744e0 0x7f6a4c0769f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:29:21.577 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:21.530+0000 7f6a5ffff640 1 --2- 172.21.15.37:0/2222824938 >> [v2:172.21.15.203:6832/2418205237,v1:172.21.15.203:6833/2418205237] conn(0x7f6a4c0744e0 0x7f6a4c0769f0 secure :-1 s=READY pgs=96 cs=0 l=1 rev1=1 crypto rx=0x7f6a600b51d0 tx=0x7f6a50007930 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:29:21.577 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:21.534+0000 7f6a5cff9640 1 -- 172.21.15.37:0/2222824938 <== mon.1 v2:172.21.15.203:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187246 (secure 0 0 0) 0x7f6a580813d0 con 0x7f6a600cf650 2024-04-25T10:29:21.879 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:21.877+0000 7fdc44d4e640 1 -- 172.21.15.37:0/2010102121 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 6} v 0) -- 0x7fdc24005580 con 0x7fdc400706e0 2024-04-25T10:29:21.880 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:21.878+0000 7fdc2effd640 1 -- 172.21.15.37:0/2010102121 <== mon.0 v2:172.21.15.37:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 6}]=0 v0) ==== 74+0+12 (secure 0 0 0) 0x7fdc30062b50 con 0x7fdc400706e0 2024-04-25T10:29:21.881 INFO:teuthology.orchestra.run.smithi037.stdout:64424509448 2024-04-25T10:29:21.884 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:21.882+0000 7fdc2cff9640 1 -- 172.21.15.37:0/2010102121 >> [v2:172.21.15.203:6832/2418205237,v1:172.21.15.203:6833/2418205237] conn(0x7fdc200745b0 msgr2=0x7fdc20076ac0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:29:21.884 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:21.882+0000 7fdc2cff9640 1 --2- 172.21.15.37:0/2010102121 >> [v2:172.21.15.203:6832/2418205237,v1:172.21.15.203:6833/2418205237] conn(0x7fdc200745b0 0x7fdc20076ac0 secure :-1 s=READY pgs=89 cs=0 l=1 rev1=1 crypto rx=0x7fdc40164390 tx=0x7fdc3800b040 comp rx=0 tx=0).stop 2024-04-25T10:29:21.885 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:21.882+0000 7fdc2cff9640 1 -- 172.21.15.37:0/2010102121 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fdc400706e0 msgr2=0x7fdc4014cb80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:29:21.885 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:21.882+0000 7fdc2cff9640 1 --2- 172.21.15.37:0/2010102121 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fdc400706e0 0x7fdc4014cb80 secure :-1 s=READY pgs=123 cs=0 l=1 rev1=1 crypto rx=0x7fdc3000c780 tx=0x7fdc3000cc40 comp rx=0 tx=0).stop 2024-04-25T10:29:21.886 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:21.883+0000 7fdc2cff9640 1 -- 172.21.15.37:0/2010102121 shutdown_connections 2024-04-25T10:29:21.886 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:21.883+0000 7fdc2cff9640 1 --2- 172.21.15.37:0/2010102121 >> [v2:172.21.15.203:6832/2418205237,v1:172.21.15.203:6833/2418205237] conn(0x7fdc200745b0 0x7fdc20076ac0 unknown :-1 s=CLOSED pgs=89 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:29:21.886 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:21.883+0000 7fdc2cff9640 1 --2- 172.21.15.37:0/2010102121 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fdc4014d8c0 0x7fdc40161da0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:29:21.886 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:21.883+0000 7fdc2cff9640 1 --2- 172.21.15.37:0/2010102121 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7fdc4014ab00 0x7fdc401479d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:29:21.886 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:21.883+0000 7fdc2cff9640 1 --2- 172.21.15.37:0/2010102121 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fdc400706e0 0x7fdc4014cb80 unknown :-1 s=CLOSED pgs=123 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:29:21.886 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:21.883+0000 7fdc2cff9640 1 -- 172.21.15.37:0/2010102121 >> 172.21.15.37:0/2010102121 conn(0x7fdc400b36b0 msgr2=0x7fdc400b5140 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:29:21.886 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:21.883+0000 7fdc2cff9640 1 -- 172.21.15.37:0/2010102121 shutdown_connections 2024-04-25T10:29:21.886 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:21.883+0000 7fdc2cff9640 1 -- 172.21.15.37:0/2010102121 wait complete. 2024-04-25T10:29:21.903 INFO:tasks.ceph.ceph_manager.ceph:need seq 64424509448 got 64424509448 for osd.6 2024-04-25T10:29:21.904 DEBUG:teuthology.parallel:result is None 2024-04-25T10:29:21.927 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:21.924+0000 7fb658134640 1 -- 172.21.15.37:0/125183910 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 1} v 0) -- 0x7fb650076bc0 con 0x7fb6500d07f0 2024-04-25T10:29:21.928 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:21.925+0000 7fb6467fc640 1 -- 172.21.15.37:0/125183910 <== mon.0 v2:172.21.15.37:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 1}]=0 v0) ==== 74+0+12 (secure 0 0 0) 0x7fb6500b7f30 con 0x7fb6500d07f0 2024-04-25T10:29:21.928 INFO:teuthology.orchestra.run.smithi037.stdout:64424509447 2024-04-25T10:29:21.929 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:21.927+0000 7fb658134640 1 -- 172.21.15.37:0/125183910 >> [v2:172.21.15.203:6832/2418205237,v1:172.21.15.203:6833/2418205237] conn(0x7fb6340748e0 msgr2=0x7fb634076df0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:29:21.929 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:21.927+0000 7fb658134640 1 --2- 172.21.15.37:0/125183910 >> [v2:172.21.15.203:6832/2418205237,v1:172.21.15.203:6833/2418205237] conn(0x7fb6340748e0 0x7fb634076df0 secure :-1 s=READY pgs=94 cs=0 l=1 rev1=1 crypto rx=0x7fb6500c91e0 tx=0x7fb64c009190 comp rx=0 tx=0).stop 2024-04-25T10:29:21.929 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:21.927+0000 7fb658134640 1 -- 172.21.15.37:0/125183910 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fb6500d07f0 msgr2=0x7fb6501555f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:29:21.929 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:21.927+0000 7fb658134640 1 --2- 172.21.15.37:0/125183910 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fb6500d07f0 0x7fb6501555f0 secure :-1 s=READY pgs=131 cs=0 l=1 rev1=1 crypto rx=0x7fb64000c3f0 tx=0x7fb64000c8b0 comp rx=0 tx=0).stop 2024-04-25T10:29:21.930 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:21.927+0000 7fb658134640 1 -- 172.21.15.37:0/125183910 shutdown_connections 2024-04-25T10:29:21.930 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:21.927+0000 7fb658134640 1 --2- 172.21.15.37:0/125183910 >> [v2:172.21.15.203:6832/2418205237,v1:172.21.15.203:6833/2418205237] conn(0x7fb6340748e0 0x7fb634076df0 unknown :-1 s=CLOSED pgs=94 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:29:21.930 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:21.927+0000 7fb658134640 1 --2- 172.21.15.37:0/125183910 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fb650161090 0x7fb650156780 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:29:21.930 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:21.928+0000 7fb658134640 1 --2- 172.21.15.37:0/125183910 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fb6500d07f0 0x7fb6501555f0 unknown :-1 s=CLOSED pgs=131 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:29:21.930 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:21.928+0000 7fb658134640 1 --2- 172.21.15.37:0/125183910 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7fb6500b75e0 0x7fb650151ae0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:29:21.930 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:21.928+0000 7fb658134640 1 -- 172.21.15.37:0/125183910 >> 172.21.15.37:0/125183910 conn(0x7fb6500d27c0 msgr2=0x7fb650164040 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:29:21.931 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:21.928+0000 7fb658134640 1 -- 172.21.15.37:0/125183910 shutdown_connections 2024-04-25T10:29:21.931 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:21.928+0000 7fb658134640 1 -- 172.21.15.37:0/125183910 wait complete. 2024-04-25T10:29:21.941 INFO:tasks.ceph.ceph_manager.ceph:need seq 64424509447 got 64424509447 for osd.1 2024-04-25T10:29:21.941 DEBUG:teuthology.parallel:result is None 2024-04-25T10:29:21.951 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:21.949+0000 7f98dc89f640 1 -- 172.21.15.37:0/2976130512 --> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 5} v 0) -- 0x7f98d8076bc0 con 0x7f98d806d060 2024-04-25T10:29:21.951 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:21.950+0000 7f98bffff640 1 -- 172.21.15.37:0/2976130512 <== mon.2 v2:172.21.15.203:3301/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 5}]=0 v0) ==== 74+0+12 (secure 0 0 0) 0x7f98cc080210 con 0x7f98d806d060 2024-04-25T10:29:21.952 INFO:teuthology.orchestra.run.smithi037.stdout:64424509449 2024-04-25T10:29:21.953 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:21.951+0000 7f772591c640 1 -- 172.21.15.37:0/859076513 --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 2} v 0) -- 0x7f77180da490 con 0x7f77080040e0 2024-04-25T10:29:21.954 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:21.952+0000 7f7707fff640 1 -- 172.21.15.37:0/859076513 <== mon.1 v2:172.21.15.203:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 2}]=0 v0) ==== 74+0+12 (secure 0 0 0) 0x7f771005f880 con 0x7f77080040e0 2024-04-25T10:29:21.954 INFO:teuthology.orchestra.run.smithi037.stdout:64424509449 2024-04-25T10:29:21.956 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:21.953+0000 7f98dc89f640 1 -- 172.21.15.37:0/2976130512 >> [v2:172.21.15.203:6832/2418205237,v1:172.21.15.203:6833/2418205237] conn(0x7f98c40744e0 msgr2=0x7f98c40769f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:29:21.956 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:21.953+0000 7f98dc89f640 1 --2- 172.21.15.37:0/2976130512 >> [v2:172.21.15.203:6832/2418205237,v1:172.21.15.203:6833/2418205237] conn(0x7f98c40744e0 0x7f98c40769f0 secure :-1 s=READY pgs=91 cs=0 l=1 rev1=1 crypto rx=0x7f98c8001bb0 tx=0x7f98c8009040 comp rx=0 tx=0).stop 2024-04-25T10:29:21.956 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:21.953+0000 7f98dc89f640 1 -- 172.21.15.37:0/2976130512 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f98d806d060 msgr2=0x7f98d81502b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:29:21.956 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:21.953+0000 7f98dc89f640 1 --2- 172.21.15.37:0/2976130512 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f98d806d060 0x7f98d81502b0 secure :-1 s=READY pgs=39 cs=0 l=1 rev1=1 crypto rx=0x7f98d809bd00 tx=0x7f98cc0046c0 comp rx=0 tx=0).stop 2024-04-25T10:29:21.956 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:21.953+0000 7f98dc89f640 1 -- 172.21.15.37:0/2976130512 shutdown_connections 2024-04-25T10:29:21.956 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:21.953+0000 7f98dc89f640 1 --2- 172.21.15.37:0/2976130512 >> [v2:172.21.15.203:6832/2418205237,v1:172.21.15.203:6833/2418205237] conn(0x7f98c40744e0 0x7f98c40769f0 unknown :-1 s=CLOSED pgs=91 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:29:21.956 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:21.953+0000 7f98dc89f640 1 --2- 172.21.15.37:0/2976130512 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f98d809b700 0x7f98d815a340 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:29:21.956 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:21.953+0000 7f98dc89f640 1 --2- 172.21.15.37:0/2976130512 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f98d806d060 0x7f98d81502b0 unknown :-1 s=CLOSED pgs=39 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:29:21.956 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:21.953+0000 7f98dc89f640 1 --2- 172.21.15.37:0/2976130512 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f98d806cc50 0x7f98d8154fe0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:29:21.956 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:21.953+0000 7f98dc89f640 1 -- 172.21.15.37:0/2976130512 >> 172.21.15.37:0/2976130512 conn(0x7f98d8073960 msgr2=0x7f98d8154520 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:29:21.956 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:21.953+0000 7f98dc89f640 1 -- 172.21.15.37:0/2976130512 shutdown_connections 2024-04-25T10:29:21.956 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:21.953+0000 7f98dc89f640 1 -- 172.21.15.37:0/2976130512 wait complete. 2024-04-25T10:29:21.957 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:21.954+0000 7f03c6d3b640 1 -- 172.21.15.37:0/1015258610 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 7} v 0) -- 0x7f03b8035360 con 0x7f03b8048710 2024-04-25T10:29:21.958 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:21.954+0000 7f772591c640 1 -- 172.21.15.37:0/859076513 >> [v2:172.21.15.203:6832/2418205237,v1:172.21.15.203:6833/2418205237] conn(0x7f770c0744b0 msgr2=0x7f770c0769c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:29:21.958 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:21.954+0000 7f772591c640 1 --2- 172.21.15.37:0/859076513 >> [v2:172.21.15.203:6832/2418205237,v1:172.21.15.203:6833/2418205237] conn(0x7f770c0744b0 0x7f770c0769c0 secure :-1 s=READY pgs=90 cs=0 l=1 rev1=1 crypto rx=0x7f7708014520 tx=0x7f77200b1040 comp rx=0 tx=0).stop 2024-04-25T10:29:21.958 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:21.954+0000 7f772591c640 1 -- 172.21.15.37:0/859076513 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f77080040e0 msgr2=0x7f771804b160 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:29:21.958 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:21.954+0000 7f772591c640 1 --2- 172.21.15.37:0/859076513 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f77080040e0 0x7f771804b160 secure :-1 s=READY pgs=91 cs=0 l=1 rev1=1 crypto rx=0x7f771000e840 tx=0x7f771000ed00 comp rx=0 tx=0).stop 2024-04-25T10:29:21.958 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:21.955+0000 7f772591c640 1 -- 172.21.15.37:0/859076513 shutdown_connections 2024-04-25T10:29:21.958 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:21.955+0000 7f772591c640 1 --2- 172.21.15.37:0/859076513 >> [v2:172.21.15.203:6832/2418205237,v1:172.21.15.203:6833/2418205237] conn(0x7f770c0744b0 0x7f770c0769c0 unknown :-1 s=CLOSED pgs=90 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:29:21.958 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:21.955+0000 7f772591c640 1 --2- 172.21.15.37:0/859076513 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f7718037080 0x7f7718037490 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:29:21.958 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:21.955+0000 7f772591c640 1 --2- 172.21.15.37:0/859076513 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f77180de210 0x7f77180de620 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:29:21.958 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:21.955+0000 7f772591c640 1 --2- 172.21.15.37:0/859076513 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f77080040e0 0x7f771804b160 unknown :-1 s=CLOSED pgs=91 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:29:21.958 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:21.955+0000 7f772591c640 1 -- 172.21.15.37:0/859076513 >> 172.21.15.37:0/859076513 conn(0x7f771800aaa0 msgr2=0x7f77180d96f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:29:21.958 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:21.955+0000 7f772591c640 1 -- 172.21.15.37:0/859076513 shutdown_connections 2024-04-25T10:29:21.959 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:21.955+0000 7f772591c640 1 -- 172.21.15.37:0/859076513 wait complete. 2024-04-25T10:29:21.959 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:21.955+0000 7f03b67fc640 1 -- 172.21.15.37:0/1015258610 <== mon.0 v2:172.21.15.37:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 7}]=0 v0) ==== 74+0+12 (secure 0 0 0) 0x7f03c0120e20 con 0x7f03b8048710 2024-04-25T10:29:21.959 INFO:teuthology.orchestra.run.smithi037.stdout:64424509447 2024-04-25T10:29:21.960 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:21.957+0000 7f03c6d3b640 1 -- 172.21.15.37:0/1015258610 >> [v2:172.21.15.203:6832/2418205237,v1:172.21.15.203:6833/2418205237] conn(0x7f03a4074530 msgr2=0x7f03a4076a40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:29:21.960 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:21.957+0000 7f03c6d3b640 1 --2- 172.21.15.37:0/1015258610 >> [v2:172.21.15.203:6832/2418205237,v1:172.21.15.203:6833/2418205237] conn(0x7f03a4074530 0x7f03a4076a40 secure :-1 s=READY pgs=93 cs=0 l=1 rev1=1 crypto rx=0x7f03bc009630 tx=0x7f03bc02e000 comp rx=0 tx=0).stop 2024-04-25T10:29:21.960 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:21.957+0000 7f03c6d3b640 1 -- 172.21.15.37:0/1015258610 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f03b8048710 msgr2=0x7f03b80e43f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:29:21.960 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:21.957+0000 7f03c6d3b640 1 --2- 172.21.15.37:0/1015258610 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f03b8048710 0x7f03b80e43f0 secure :-1 s=READY pgs=130 cs=0 l=1 rev1=1 crypto rx=0x7f03c006b610 tx=0x7f03c006eaa0 comp rx=0 tx=0).stop 2024-04-25T10:29:21.960 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:21.958+0000 7f03c6d3b640 1 -- 172.21.15.37:0/1015258610 shutdown_connections 2024-04-25T10:29:21.960 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:21.958+0000 7f03c6d3b640 1 --2- 172.21.15.37:0/1015258610 >> [v2:172.21.15.203:6832/2418205237,v1:172.21.15.203:6833/2418205237] conn(0x7f03a4074530 0x7f03a4076a40 unknown :-1 s=CLOSED pgs=93 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:29:21.960 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:21.958+0000 7f03c6d3b640 1 --2- 172.21.15.37:0/1015258610 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f03b8048710 0x7f03b80e43f0 unknown :-1 s=CLOSED pgs=130 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:29:21.960 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:21.958+0000 7f03c6d3b640 1 --2- 172.21.15.37:0/1015258610 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f03b8048300 0x7f03b80e3eb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:29:21.960 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:21.958+0000 7f03c6d3b640 1 --2- 172.21.15.37:0/1015258610 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f03b8047db0 0x7f03b80e1fa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:29:21.960 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:21.958+0000 7f03c6d3b640 1 -- 172.21.15.37:0/1015258610 >> 172.21.15.37:0/1015258610 conn(0x7f03b8054cf0 msgr2=0x7f03b80367c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:29:21.961 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:21.958+0000 7f03c6d3b640 1 -- 172.21.15.37:0/1015258610 shutdown_connections 2024-04-25T10:29:21.961 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:21.958+0000 7f03c6d3b640 1 -- 172.21.15.37:0/1015258610 wait complete. 2024-04-25T10:29:21.963 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:21.961+0000 7f31d6955640 1 -- 172.21.15.37:0/310476549 --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 3} v 0) -- 0x7f31d015b2f0 con 0x7f31d015aa20 2024-04-25T10:29:21.964 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:21.962+0000 7fc43a897640 1 -- 172.21.15.37:0/3090975409 --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 0} v 0) -- 0x7fc41c005580 con 0x7fc43415b740 2024-04-25T10:29:21.964 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:21.962+0000 7f31ccff9640 1 -- 172.21.15.37:0/310476549 <== mon.1 v2:172.21.15.203:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 3}]=0 v0) ==== 74+0+12 (secure 0 0 0) 0x7f31c805ffc0 con 0x7f31d015aa20 2024-04-25T10:29:21.964 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:21.962+0000 7fc42a7fc640 1 -- 172.21.15.37:0/3090975409 <== mon.1 v2:172.21.15.203:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 0}]=0 v0) ==== 74+0+12 (secure 0 0 0) 0x7fc42c080a10 con 0x7fc43415b740 2024-04-25T10:29:21.964 INFO:teuthology.orchestra.run.smithi037.stdout:64424509449 2024-04-25T10:29:21.965 INFO:teuthology.orchestra.run.smithi037.stdout:64424509448 2024-04-25T10:29:21.967 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:21.965+0000 7f6a6522a640 1 -- 172.21.15.37:0/2222824938 --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 4} v 0) -- 0x7f6a600cf270 con 0x7f6a600cf650 2024-04-25T10:29:21.968 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:21.965+0000 7f31d6955640 1 -- 172.21.15.37:0/310476549 >> [v2:172.21.15.203:6832/2418205237,v1:172.21.15.203:6833/2418205237] conn(0x7f31bc0744b0 msgr2=0x7f31bc0769c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:29:21.968 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:21.965+0000 7f31d6955640 1 --2- 172.21.15.37:0/310476549 >> [v2:172.21.15.203:6832/2418205237,v1:172.21.15.203:6833/2418205237] conn(0x7f31bc0744b0 0x7f31bc0769c0 secure :-1 s=READY pgs=95 cs=0 l=1 rev1=1 crypto rx=0x7f31c002a180 tx=0x7f31c002e000 comp rx=0 tx=0).stop 2024-04-25T10:29:21.968 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:21.965+0000 7f31d6955640 1 -- 172.21.15.37:0/310476549 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f31d015aa20 msgr2=0x7f31d00cddd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:29:21.968 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:21.965+0000 7f31d6955640 1 --2- 172.21.15.37:0/310476549 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f31d015aa20 0x7f31d00cddd0 secure :-1 s=READY pgs=93 cs=0 l=1 rev1=1 crypto rx=0x7f31c8007a90 tx=0x7f31c8007f50 comp rx=0 tx=0).stop 2024-04-25T10:29:21.968 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:21.965+0000 7f31d6955640 1 -- 172.21.15.37:0/310476549 shutdown_connections 2024-04-25T10:29:21.968 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:21.965+0000 7f31d6955640 1 --2- 172.21.15.37:0/310476549 >> [v2:172.21.15.203:6832/2418205237,v1:172.21.15.203:6833/2418205237] conn(0x7f31bc0744b0 0x7f31bc0769c0 unknown :-1 s=CLOSED pgs=95 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:29:21.968 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:21.965+0000 7f31d6955640 1 --2- 172.21.15.37:0/310476549 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f31d015aa20 0x7f31d00cddd0 unknown :-1 s=CLOSED pgs=93 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:29:21.968 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:21.965+0000 7f31d6955640 1 --2- 172.21.15.37:0/310476549 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f31d00d0d10 0x7f31d014c070 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:29:21.969 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:21.965+0000 7f31d6955640 1 --2- 172.21.15.37:0/310476549 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f31d00b7750 0x7f31d014bb30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:29:21.969 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:21.966+0000 7f31d6955640 1 -- 172.21.15.37:0/310476549 >> 172.21.15.37:0/310476549 conn(0x7f31d00d2750 msgr2=0x7f31d00d2b60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:29:21.969 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:21.966+0000 7fc43a897640 1 -- 172.21.15.37:0/3090975409 >> [v2:172.21.15.203:6832/2418205237,v1:172.21.15.203:6833/2418205237] conn(0x7fc418074580 msgr2=0x7fc418076a90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:29:21.969 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:21.966+0000 7fc43a897640 1 --2- 172.21.15.37:0/3090975409 >> [v2:172.21.15.203:6832/2418205237,v1:172.21.15.203:6833/2418205237] conn(0x7fc418074580 0x7fc418076a90 secure :-1 s=READY pgs=92 cs=0 l=1 rev1=1 crypto rx=0x7fc424000f60 tx=0x7fc424007e40 comp rx=0 tx=0).stop 2024-04-25T10:29:21.969 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:21.966+0000 7fc43a897640 1 -- 172.21.15.37:0/3090975409 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fc43415b740 msgr2=0x7fc4341637e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:29:21.969 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:21.966+0000 7fc43a897640 1 --2- 172.21.15.37:0/3090975409 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fc43415b740 0x7fc4341637e0 secure :-1 s=READY pgs=92 cs=0 l=1 rev1=1 crypto rx=0x7fc42c031000 tx=0x7fc42c004a40 comp rx=0 tx=0).stop 2024-04-25T10:29:21.969 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:21.966+0000 7f31d6955640 1 -- 172.21.15.37:0/310476549 shutdown_connections 2024-04-25T10:29:21.969 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:21.966+0000 7f31d6955640 1 -- 172.21.15.37:0/310476549 wait complete. 2024-04-25T10:29:21.970 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:21.966+0000 7fc43a897640 1 -- 172.21.15.37:0/3090975409 shutdown_connections 2024-04-25T10:29:21.970 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:21.966+0000 7fc43a897640 1 --2- 172.21.15.37:0/3090975409 >> [v2:172.21.15.203:6832/2418205237,v1:172.21.15.203:6833/2418205237] conn(0x7fc418074580 0x7fc418076a90 unknown :-1 s=CLOSED pgs=92 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:29:21.970 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:21.966+0000 7fc43a897640 1 --2- 172.21.15.37:0/3090975409 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fc43415b740 0x7fc4341637e0 unknown :-1 s=CLOSED pgs=92 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:29:21.970 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:21.966+0000 7fc43a897640 1 --2- 172.21.15.37:0/3090975409 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7fc434161da0 0x7fc4340afa80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:29:21.970 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:21.966+0000 7fc43a897640 1 --2- 172.21.15.37:0/3090975409 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fc4340b0bc0 0x7fc434165be0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:29:21.970 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:21.966+0000 7fc43a897640 1 -- 172.21.15.37:0/3090975409 >> 172.21.15.37:0/3090975409 conn(0x7fc43406cb70 msgr2=0x7fc4341605f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:29:21.970 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:21.966+0000 7f6a5cff9640 1 -- 172.21.15.37:0/2222824938 <== mon.1 v2:172.21.15.203:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 4}]=0 v0) ==== 74+0+12 (secure 0 0 0) 0x7f6a580893c0 con 0x7f6a600cf650 2024-04-25T10:29:21.971 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:21.966+0000 7fc43a897640 1 -- 172.21.15.37:0/3090975409 shutdown_connections 2024-04-25T10:29:21.971 INFO:teuthology.orchestra.run.smithi037.stdout:55834574856 2024-04-25T10:29:21.971 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:21.966+0000 7fc43a897640 1 -- 172.21.15.37:0/3090975409 wait complete. 2024-04-25T10:29:21.972 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:21.970+0000 7f6a6522a640 1 -- 172.21.15.37:0/2222824938 >> [v2:172.21.15.203:6832/2418205237,v1:172.21.15.203:6833/2418205237] conn(0x7f6a4c0744e0 msgr2=0x7f6a4c0769f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:29:21.972 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:21.970+0000 7f6a6522a640 1 --2- 172.21.15.37:0/2222824938 >> [v2:172.21.15.203:6832/2418205237,v1:172.21.15.203:6833/2418205237] conn(0x7f6a4c0744e0 0x7f6a4c0769f0 secure :-1 s=READY pgs=96 cs=0 l=1 rev1=1 crypto rx=0x7f6a600b51d0 tx=0x7f6a50007930 comp rx=0 tx=0).stop 2024-04-25T10:29:21.972 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:21.970+0000 7f6a6522a640 1 -- 172.21.15.37:0/2222824938 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f6a600cf650 msgr2=0x7f6a6006c540 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:29:21.972 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:21.970+0000 7f6a6522a640 1 --2- 172.21.15.37:0/2222824938 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f6a600cf650 0x7f6a6006c540 secure :-1 s=READY pgs=94 cs=0 l=1 rev1=1 crypto rx=0x7f6a5802a630 tx=0x7f6a58004a40 comp rx=0 tx=0).stop 2024-04-25T10:29:21.972 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:21.970+0000 7f6a6522a640 1 -- 172.21.15.37:0/2222824938 shutdown_connections 2024-04-25T10:29:21.972 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:21.970+0000 7f6a6522a640 1 --2- 172.21.15.37:0/2222824938 >> [v2:172.21.15.203:6832/2418205237,v1:172.21.15.203:6833/2418205237] conn(0x7f6a4c0744e0 0x7f6a4c0769f0 unknown :-1 s=CLOSED pgs=96 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:29:21.972 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:21.970+0000 7f6a6522a640 1 --2- 172.21.15.37:0/2222824938 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f6a60160300 0x7f6a6006ca80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:29:21.973 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:21.970+0000 7f6a6522a640 1 --2- 172.21.15.37:0/2222824938 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f6a600cf650 0x7f6a6006c540 unknown :-1 s=CLOSED pgs=94 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:29:21.973 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:21.970+0000 7f6a6522a640 1 --2- 172.21.15.37:0/2222824938 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f6a600ce3c0 0x7f6a6015a120 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:29:21.973 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:21.970+0000 7f6a6522a640 1 -- 172.21.15.37:0/2222824938 >> 172.21.15.37:0/2222824938 conn(0x7f6a600d2670 msgr2=0x7f6a60070a50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:29:21.973 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:21.970+0000 7f6a6522a640 1 -- 172.21.15.37:0/2222824938 shutdown_connections 2024-04-25T10:29:21.973 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:21.970+0000 7f6a6522a640 1 -- 172.21.15.37:0/2222824938 wait complete. 2024-04-25T10:29:21.974 INFO:tasks.ceph.ceph_manager.ceph:need seq 64424509449 got 64424509449 for osd.2 2024-04-25T10:29:21.974 DEBUG:teuthology.parallel:result is None 2024-04-25T10:29:21.976 INFO:tasks.ceph.ceph_manager.ceph:need seq 64424509449 got 64424509449 for osd.5 2024-04-25T10:29:21.976 DEBUG:teuthology.parallel:result is None 2024-04-25T10:29:21.977 INFO:tasks.ceph.ceph_manager.ceph:need seq 64424509447 got 64424509447 for osd.7 2024-04-25T10:29:21.977 DEBUG:teuthology.parallel:result is None 2024-04-25T10:29:21.984 INFO:tasks.ceph.ceph_manager.ceph:need seq 64424509449 got 64424509449 for osd.3 2024-04-25T10:29:21.984 DEBUG:teuthology.parallel:result is None 2024-04-25T10:29:21.984 INFO:tasks.ceph.ceph_manager.ceph:need seq 64424509448 got 64424509448 for osd.0 2024-04-25T10:29:21.985 DEBUG:teuthology.parallel:result is None 2024-04-25T10:29:21.987 INFO:tasks.ceph.ceph_manager.ceph:need seq 55834574856 got 55834574856 for osd.4 2024-04-25T10:29:21.987 DEBUG:teuthology.parallel:result is None 2024-04-25T10:29:21.987 INFO:tasks.ceph.ceph_manager.ceph:waiting for clean 2024-04-25T10:29:21.987 DEBUG:teuthology.orchestra.run.smithi037:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-04-25T10:29:22.139 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:22.136+0000 7f4f52f7a640 1 Processor -- start 2024-04-25T10:29:22.139 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:22.136+0000 7f4f52f7a640 1 -- start start 2024-04-25T10:29:22.140 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:22.138+0000 7f4f52f7a640 1 --2- >> v2:172.21.15.37:3300/0 conn(0x7f4f4c0d5430 0x7f4f4c0b75c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:29:22.140 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:22.138+0000 7f4f52f7a640 1 --2- >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f4f4c161110 0x7f4f4c161880 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:29:22.141 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:22.138+0000 7f4f52f7a640 1 -- --> v1:172.21.15.37:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f4f4c0b7f10 con 0x7f4f4c0b7b00 2024-04-25T10:29:22.141 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:22.138+0000 7f4f52f7a640 1 -- --> v2:172.21.15.37:3300/0 -- mon_getmap magic: 0 -- 0x7f4f4c0d4420 con 0x7f4f4c0d5430 2024-04-25T10:29:22.141 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:22.138+0000 7f4f52f7a640 1 -- --> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] -- mon_getmap magic: 0 -- 0x7f4f4c161520 con 0x7f4f4c161110 2024-04-25T10:29:22.141 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:22.138+0000 7f4f50d16640 1 --2- >> v2:172.21.15.37:3300/0 conn(0x7f4f4c0d5430 0x7f4f4c0b75c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:29:22.141 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:22.138+0000 7f4f4bfff640 1 --1- >> v1:172.21.15.37:6789/0 conn(0x7f4f4c0b7b00 0x7f4f4c160a00 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.37:6789/0 says I am v1:172.21.15.37:59804/0 (socket says 172.21.15.37:59804) 2024-04-25T10:29:22.141 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:22.138+0000 7f4f50d16640 1 --2- >> v2:172.21.15.37:3300/0 conn(0x7f4f4c0d5430 0x7f4f4c0b75c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.37:3300/0 says I am v2:172.21.15.37:35098/0 (socket says 172.21.15.37:35098) 2024-04-25T10:29:22.141 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:22.138+0000 7f4f4bfff640 1 -- 172.21.15.37:0/1097278198 learned_addr learned my addr 172.21.15.37:0/1097278198 (peer_addr_for_me v1:172.21.15.37:0/0) 2024-04-25T10:29:22.141 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:22.138+0000 7f4f51517640 1 --2- 172.21.15.37:0/1097278198 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f4f4c161110 0x7f4f4c161880 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:29:22.141 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:22.138+0000 7f4f50d16640 1 -- 172.21.15.37:0/1097278198 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f4f4c161110 msgr2=0x7f4f4c161880 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:29:22.141 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:22.138+0000 7f4f50d16640 1 --2- 172.21.15.37:0/1097278198 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f4f4c161110 0x7f4f4c161880 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:29:22.141 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:22.138+0000 7f4f50d16640 1 -- 172.21.15.37:0/1097278198 >> v1:172.21.15.37:6789/0 conn(0x7f4f4c0b7b00 legacy=0x7f4f4c160a00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:29:22.141 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:22.138+0000 7f4f50d16640 1 -- 172.21.15.37:0/1097278198 --> v2:172.21.15.37:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f4f4c0d5840 con 0x7f4f4c0d5430 2024-04-25T10:29:22.143 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:22.139+0000 7f4f50d16640 1 --2- 172.21.15.37:0/1097278198 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f4f4c0d5430 0x7f4f4c0b75c0 secure :-1 s=READY pgs=133 cs=0 l=1 rev1=1 crypto rx=0x7f4f3c0097a0 tx=0x7f4f3c02a100 comp rx=0 tx=0).ready entity=mon.0 client_cookie=52d5b2f8cc5c828b server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:29:22.143 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:22.139+0000 7f4f4b7fe640 1 -- 172.21.15.37:0/1097278198 <== mon.0 v2:172.21.15.37:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4f3c037070 con 0x7f4f4c0d5430 2024-04-25T10:29:22.143 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:22.139+0000 7f4f4b7fe640 1 -- 172.21.15.37:0/1097278198 <== mon.0 v2:172.21.15.37:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f4f3c02ae50 con 0x7f4f4c0d5430 2024-04-25T10:29:22.144 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:22.139+0000 7f4f4b7fe640 1 -- 172.21.15.37:0/1097278198 <== mon.0 v2:172.21.15.37:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4f3c032660 con 0x7f4f4c0d5430 2024-04-25T10:29:22.144 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:22.140+0000 7f4f52f7a640 1 -- 172.21.15.37:0/1097278198 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f4f4c0d5430 msgr2=0x7f4f4c0b75c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:29:22.144 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:22.140+0000 7f4f52f7a640 1 --2- 172.21.15.37:0/1097278198 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f4f4c0d5430 0x7f4f4c0b75c0 secure :-1 s=READY pgs=133 cs=0 l=1 rev1=1 crypto rx=0x7f4f3c0097a0 tx=0x7f4f3c02a100 comp rx=0 tx=0).stop 2024-04-25T10:29:22.144 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:22.140+0000 7f4f52f7a640 1 -- 172.21.15.37:0/1097278198 shutdown_connections 2024-04-25T10:29:22.144 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:22.140+0000 7f4f52f7a640 1 --2- 172.21.15.37:0/1097278198 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f4f4c161110 0x7f4f4c161880 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:29:22.144 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:22.140+0000 7f4f52f7a640 1 --2- 172.21.15.37:0/1097278198 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f4f4c0d5430 0x7f4f4c0b75c0 unknown :-1 s=CLOSED pgs=133 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:29:22.144 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:22.140+0000 7f4f52f7a640 1 -- 172.21.15.37:0/1097278198 >> 172.21.15.37:0/1097278198 conn(0x7f4f4c0d27a0 msgr2=0x7f4f4c0d2bb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:29:22.144 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:22.140+0000 7f4f52f7a640 1 -- 172.21.15.37:0/1097278198 shutdown_connections 2024-04-25T10:29:22.144 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:22.140+0000 7f4f52f7a640 1 -- 172.21.15.37:0/1097278198 wait complete. 2024-04-25T10:29:22.148 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:22.141+0000 7f4f52f7a640 1 Processor -- start 2024-04-25T10:29:22.148 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:22.142+0000 7f4f52f7a640 1 -- start start 2024-04-25T10:29:22.148 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:22.143+0000 7f4f52f7a640 1 --2- >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f4f4c0b7b00 0x7f4f4c1e92d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:29:22.149 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:22.143+0000 7f4f52f7a640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f4f4c0d5430 0x7f4f4c0b9560 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:29:22.149 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:22.143+0000 7f4f52f7a640 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f4f4c161110 0x7f4f4c0b9aa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:29:22.149 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:22.143+0000 7f4f52f7a640 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 -- 0x7f4f4c0d4c30 con 0x7f4f4c0d5430 2024-04-25T10:29:22.149 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:22.143+0000 7f4f52f7a640 1 -- --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_getmap magic: 0 -- 0x7f4f4c0d4950 con 0x7f4f4c161110 2024-04-25T10:29:22.149 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:22.143+0000 7f4f52f7a640 1 -- --> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] -- mon_getmap magic: 0 -- 0x7f4f4c0d4f10 con 0x7f4f4c0b7b00 2024-04-25T10:29:22.149 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:22.143+0000 7f4f4bfff640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f4f4c0d5430 0x7f4f4c0b9560 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:29:22.149 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:22.143+0000 7f4f4bfff640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f4f4c0d5430 0x7f4f4c0b9560 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.37:3300/0 says I am v2:172.21.15.37:35102/0 (socket says 172.21.15.37:35102) 2024-04-25T10:29:22.149 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:22.143+0000 7f4f4bfff640 1 -- 172.21.15.37:0/3921659094 learned_addr learned my addr 172.21.15.37:0/3921659094 (peer_addr_for_me v2:172.21.15.37:0/0) 2024-04-25T10:29:22.149 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:22.143+0000 7f4f50d16640 1 --2- 172.21.15.37:0/3921659094 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f4f4c0b7b00 0x7f4f4c1e92d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:29:22.149 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:22.143+0000 7f4f51517640 1 --2- 172.21.15.37:0/3921659094 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f4f4c161110 0x7f4f4c0b9aa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:29:22.149 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:22.143+0000 7f4f4bfff640 1 -- 172.21.15.37:0/3921659094 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f4f4c0b7b00 msgr2=0x7f4f4c1e92d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:29:22.149 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:22.143+0000 7f4f4bfff640 1 --2- 172.21.15.37:0/3921659094 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f4f4c0b7b00 0x7f4f4c1e92d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:29:22.149 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:22.143+0000 7f4f4bfff640 1 -- 172.21.15.37:0/3921659094 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f4f4c161110 msgr2=0x7f4f4c0b9aa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:29:22.150 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:22.143+0000 7f4f4bfff640 1 --2- 172.21.15.37:0/3921659094 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f4f4c161110 0x7f4f4c0b9aa0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:29:22.150 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:22.143+0000 7f4f4bfff640 1 -- 172.21.15.37:0/3921659094 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f4f4c0cba10 con 0x7f4f4c0d5430 2024-04-25T10:29:22.150 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:22.143+0000 7f4f51517640 1 --2- 172.21.15.37:0/3921659094 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f4f4c161110 0x7f4f4c0b9aa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T10:29:22.150 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:22.144+0000 7f4f4bfff640 1 --2- 172.21.15.37:0/3921659094 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f4f4c0d5430 0x7f4f4c0b9560 secure :-1 s=READY pgs=134 cs=0 l=1 rev1=1 crypto rx=0x7f4f3400c3c0 tx=0x7f4f3400c880 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:29:22.150 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:22.144+0000 7f4f497fa640 1 -- 172.21.15.37:0/3921659094 <== mon.0 v2:172.21.15.37:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4f34017070 con 0x7f4f4c0d5430 2024-04-25T10:29:22.150 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:22.144+0000 7f4f497fa640 1 -- 172.21.15.37:0/3921659094 <== mon.0 v2:172.21.15.37:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f4f34005ce0 con 0x7f4f4c0d5430 2024-04-25T10:29:22.150 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:22.144+0000 7f4f52f7a640 1 -- 172.21.15.37:0/3921659094 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f4f4c1e8d10 con 0x7f4f4c0d5430 2024-04-25T10:29:22.150 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:22.144+0000 7f4f497fa640 1 -- 172.21.15.37:0/3921659094 <== mon.0 v2:172.21.15.37:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4f340072f0 con 0x7f4f4c0d5430 2024-04-25T10:29:22.150 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:22.144+0000 7f4f52f7a640 1 -- 172.21.15.37:0/3921659094 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f4f4c0cdb50 con 0x7f4f4c0d5430 2024-04-25T10:29:22.150 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:22.145+0000 7f4f52f7a640 1 -- 172.21.15.37:0/3921659094 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f4f4c06a140 con 0x7f4f4c0d5430 2024-04-25T10:29:22.150 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:22.145+0000 7f4f497fa640 1 -- 172.21.15.37:0/3921659094 <== mon.0 v2:172.21.15.37:3300/0 4 ==== mgrmap(e 5) ==== 96782+0+0 (secure 0 0 0) 0x7f4f3401a070 con 0x7f4f4c0d5430 2024-04-25T10:29:22.150 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:22.146+0000 7f4f497fa640 1 --2- 172.21.15.37:0/3921659094 >> [v2:172.21.15.203:6832/2418205237,v1:172.21.15.203:6833/2418205237] conn(0x7f4f2c0744b0 0x7f4f2c0769c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:29:22.150 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:22.146+0000 7f4f497fa640 1 -- 172.21.15.37:0/3921659094 <== mon.0 v2:172.21.15.37:3300/0 5 ==== osd_map(23..23 src has 1..23) ==== 5709+0+0 (secure 0 0 0) 0x7f4f34094580 con 0x7f4f4c0d5430 2024-04-25T10:29:22.151 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:22.146+0000 7f4f50d16640 1 --2- 172.21.15.37:0/3921659094 >> [v2:172.21.15.203:6832/2418205237,v1:172.21.15.203:6833/2418205237] conn(0x7f4f2c0744b0 0x7f4f2c0769c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:29:22.151 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:22.147+0000 7f4f50d16640 1 --2- 172.21.15.37:0/3921659094 >> [v2:172.21.15.203:6832/2418205237,v1:172.21.15.203:6833/2418205237] conn(0x7f4f2c0744b0 0x7f4f2c0769c0 secure :-1 s=READY pgs=97 cs=0 l=1 rev1=1 crypto rx=0x7f4f3c034000 tx=0x7f4f3c02e000 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:29:22.153 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:22.151+0000 7f4f497fa640 1 -- 172.21.15.37:0/3921659094 <== mon.0 v2:172.21.15.37:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187246 (secure 0 0 0) 0x7f4f4c0b7f10 con 0x7f4f4c0d5430 2024-04-25T10:29:22.334 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:22.332+0000 7f4f52f7a640 1 -- 172.21.15.37:0/3921659094 --> [v2:172.21.15.203:6832/2418205237,v1:172.21.15.203:6833/2418205237] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) -- 0x7f4f4c15ed90 con 0x7f4f2c0744b0 2024-04-25T10:29:22.341 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:22.339+0000 7f4f497fa640 1 -- 172.21.15.37:0/3921659094 <== mgr.4100 v2:172.21.15.203:6832/2418205237 1 ==== mgr_command_reply(tid 0: 0 dumped all) ==== 18+0+251822 (secure 0 0 0) 0x7f4f3c07f7c0 con 0x7f4f2c0744b0 2024-04-25T10:29:22.346 INFO:teuthology.orchestra.run.smithi037.stdout: 2024-04-25T10:29:22.348 INFO:teuthology.orchestra.run.smithi037.stderr:dumped all 2024-04-25T10:29:22.350 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:22.342+0000 7f4f52f7a640 1 -- 172.21.15.37:0/3921659094 >> [v2:172.21.15.203:6832/2418205237,v1:172.21.15.203:6833/2418205237] conn(0x7f4f2c0744b0 msgr2=0x7f4f2c0769c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:29:22.351 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:22.342+0000 7f4f52f7a640 1 --2- 172.21.15.37:0/3921659094 >> [v2:172.21.15.203:6832/2418205237,v1:172.21.15.203:6833/2418205237] conn(0x7f4f2c0744b0 0x7f4f2c0769c0 secure :-1 s=READY pgs=97 cs=0 l=1 rev1=1 crypto rx=0x7f4f3c034000 tx=0x7f4f3c02e000 comp rx=0 tx=0).stop 2024-04-25T10:29:22.351 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:22.342+0000 7f4f52f7a640 1 -- 172.21.15.37:0/3921659094 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f4f4c0d5430 msgr2=0x7f4f4c0b9560 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:29:22.351 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:22.342+0000 7f4f52f7a640 1 --2- 172.21.15.37:0/3921659094 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f4f4c0d5430 0x7f4f4c0b9560 secure :-1 s=READY pgs=134 cs=0 l=1 rev1=1 crypto rx=0x7f4f3400c3c0 tx=0x7f4f3400c880 comp rx=0 tx=0).stop 2024-04-25T10:29:22.351 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:22.342+0000 7f4f52f7a640 1 -- 172.21.15.37:0/3921659094 shutdown_connections 2024-04-25T10:29:22.351 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:22.342+0000 7f4f52f7a640 1 --2- 172.21.15.37:0/3921659094 >> [v2:172.21.15.203:6832/2418205237,v1:172.21.15.203:6833/2418205237] conn(0x7f4f2c0744b0 0x7f4f2c0769c0 unknown :-1 s=CLOSED pgs=97 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:29:22.352 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:22.342+0000 7f4f52f7a640 1 --2- 172.21.15.37:0/3921659094 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f4f4c161110 0x7f4f4c0b9aa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:29:22.352 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:22.342+0000 7f4f52f7a640 1 --2- 172.21.15.37:0/3921659094 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f4f4c0d5430 0x7f4f4c0b9560 unknown :-1 s=CLOSED pgs=134 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:29:22.352 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:22.342+0000 7f4f52f7a640 1 --2- 172.21.15.37:0/3921659094 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f4f4c0b7b00 0x7f4f4c1e92d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:29:22.352 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:22.342+0000 7f4f52f7a640 1 -- 172.21.15.37:0/3921659094 >> 172.21.15.37:0/3921659094 conn(0x7f4f4c0d27a0 msgr2=0x7f4f4c15f3f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:29:22.352 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:22.342+0000 7f4f52f7a640 1 -- 172.21.15.37:0/3921659094 shutdown_connections 2024-04-25T10:29:22.352 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:22.342+0000 7f4f52f7a640 1 -- 172.21.15.37:0/3921659094 wait complete. 2024-04-25T10:29:22.355 INFO:teuthology.orchestra.run.smithi037.stdout:{"pg_ready":true,"pg_map":{"version":31,"stamp":"2024-04-25T10:29:22.190081+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":592778,"num_objects":24,"num_object_clones":0,"num_object_copies":48,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":24,"num_whiteouts":0,"num_read":66,"num_read_kb":55,"num_write":146,"num_write_kb":1642,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":14,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":111,"ondisk_log_size":111,"up":194,"acting":194,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":194,"num_osds":8,"num_per_pool_osds":8,"num_per_pool_omap_osds":8,"kb":754974720,"kb_used":219244,"kb_used_data":3048,"kb_used_omap":12,"kb_used_meta":216051,"kb_avail":754755476,"statfs":{"total":773094113280,"available":772869607424,"internally_reserved":0,"allocated":3121152,"data_stored":1835436,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":12752,"internal_metadata":221236784},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":2410,"num_objects":22,"num_object_clones":0,"num_object_copies":44,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":22,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":45,"num_write_kb":14,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":14,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"12.004319"},"pg_stats":[{"pgid":"3.3f","version":"0'0","reported_seq":12,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-04-25T10:29:10.037967+0000","last_change":"2024-04-25T10:29:09.024146+0000","last_active":"2024-04-25T10:29:10.037967+0000","last_peered":"2024-04-25T10:29:10.037967+0000","last_clean":"2024-04-25T10:29:10.037967+0000","last_became_active":"2024-04-25T10:29:09.023798+0000","last_became_peered":"2024-04-25T10:29:09.023798+0000","last_unstale":"2024-04-25T10:29:10.037967+0000","last_undegraded":"2024-04-25T10:29:10.037967+0000","last_fullsized":"2024-04-25T10:29:10.037967+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:29:07.995096+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:29:07.995096+0000","last_clean_scrub_stamp":"2024-04-25T10:29:07.995096+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T20:37:20.726766+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.3e","version":"0'0","reported_seq":12,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-04-25T10:29:10.189579+0000","last_change":"2024-04-25T10:29:09.028600+0000","last_active":"2024-04-25T10:29:10.189579+0000","last_peered":"2024-04-25T10:29:10.189579+0000","last_clean":"2024-04-25T10:29:10.189579+0000","last_became_active":"2024-04-25T10:29:09.028286+0000","last_became_peered":"2024-04-25T10:29:09.028286+0000","last_unstale":"2024-04-25T10:29:10.189579+0000","last_undegraded":"2024-04-25T10:29:10.189579+0000","last_fullsized":"2024-04-25T10:29:10.189579+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:29:07.995096+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:29:07.995096+0000","last_clean_scrub_stamp":"2024-04-25T10:29:07.995096+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T20:52:02.098872+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6],"acting":[5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.3d","version":"0'0","reported_seq":12,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-04-25T10:29:10.189318+0000","last_change":"2024-04-25T10:29:09.039030+0000","last_active":"2024-04-25T10:29:10.189318+0000","last_peered":"2024-04-25T10:29:10.189318+0000","last_clean":"2024-04-25T10:29:10.189318+0000","last_became_active":"2024-04-25T10:29:09.037692+0000","last_became_peered":"2024-04-25T10:29:09.037692+0000","last_unstale":"2024-04-25T10:29:10.189318+0000","last_undegraded":"2024-04-25T10:29:10.189318+0000","last_fullsized":"2024-04-25T10:29:10.189318+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:29:07.995096+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:29:07.995096+0000","last_clean_scrub_stamp":"2024-04-25T10:29:07.995096+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T15:51:52.989545+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.3c","version":"0'0","reported_seq":12,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-04-25T10:29:10.026169+0000","last_change":"2024-04-25T10:29:09.027520+0000","last_active":"2024-04-25T10:29:10.026169+0000","last_peered":"2024-04-25T10:29:10.026169+0000","last_clean":"2024-04-25T10:29:10.026169+0000","last_became_active":"2024-04-25T10:29:09.027075+0000","last_became_peered":"2024-04-25T10:29:09.027075+0000","last_unstale":"2024-04-25T10:29:10.026169+0000","last_undegraded":"2024-04-25T10:29:10.026169+0000","last_fullsized":"2024-04-25T10:29:10.026169+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:29:07.995096+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:29:07.995096+0000","last_clean_scrub_stamp":"2024-04-25T10:29:07.995096+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T10:55:01.505157+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4],"acting":[2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.3b","version":"0'0","reported_seq":12,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-04-25T10:29:10.037590+0000","last_change":"2024-04-25T10:29:09.040684+0000","last_active":"2024-04-25T10:29:10.037590+0000","last_peered":"2024-04-25T10:29:10.037590+0000","last_clean":"2024-04-25T10:29:10.037590+0000","last_became_active":"2024-04-25T10:29:09.040372+0000","last_became_peered":"2024-04-25T10:29:09.040372+0000","last_unstale":"2024-04-25T10:29:10.037590+0000","last_undegraded":"2024-04-25T10:29:10.037590+0000","last_fullsized":"2024-04-25T10:29:10.037590+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:29:07.995096+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:29:07.995096+0000","last_clean_scrub_stamp":"2024-04-25T10:29:07.995096+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T11:52:24.415332+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.3a","version":"0'0","reported_seq":12,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-04-25T10:29:10.025911+0000","last_change":"2024-04-25T10:29:09.027391+0000","last_active":"2024-04-25T10:29:10.025911+0000","last_peered":"2024-04-25T10:29:10.025911+0000","last_clean":"2024-04-25T10:29:10.025911+0000","last_became_active":"2024-04-25T10:29:09.027029+0000","last_became_peered":"2024-04-25T10:29:09.027029+0000","last_unstale":"2024-04-25T10:29:10.025911+0000","last_undegraded":"2024-04-25T10:29:10.025911+0000","last_fullsized":"2024-04-25T10:29:10.025911+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:29:07.995096+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:29:07.995096+0000","last_clean_scrub_stamp":"2024-04-25T10:29:07.995096+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T10:35:31.361514+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4],"acting":[2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.39","version":"0'0","reported_seq":12,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-04-25T10:29:10.037462+0000","last_change":"2024-04-25T10:29:09.029272+0000","last_active":"2024-04-25T10:29:10.037462+0000","last_peered":"2024-04-25T10:29:10.037462+0000","last_clean":"2024-04-25T10:29:10.037462+0000","last_became_active":"2024-04-25T10:29:09.028302+0000","last_became_peered":"2024-04-25T10:29:09.028302+0000","last_unstale":"2024-04-25T10:29:10.037462+0000","last_undegraded":"2024-04-25T10:29:10.037462+0000","last_fullsized":"2024-04-25T10:29:10.037462+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:29:07.995096+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:29:07.995096+0000","last_clean_scrub_stamp":"2024-04-25T10:29:07.995096+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T11:31:45.283247+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.38","version":"0'0","reported_seq":12,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-04-25T10:29:10.188973+0000","last_change":"2024-04-25T10:29:09.035089+0000","last_active":"2024-04-25T10:29:10.188973+0000","last_peered":"2024-04-25T10:29:10.188973+0000","last_clean":"2024-04-25T10:29:10.188973+0000","last_became_active":"2024-04-25T10:29:09.034324+0000","last_became_peered":"2024-04-25T10:29:09.034324+0000","last_unstale":"2024-04-25T10:29:10.188973+0000","last_undegraded":"2024-04-25T10:29:10.188973+0000","last_fullsized":"2024-04-25T10:29:10.188973+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:29:07.995096+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:29:07.995096+0000","last_clean_scrub_stamp":"2024-04-25T10:29:07.995096+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T21:39:39.743441+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.37","version":"0'0","reported_seq":12,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-04-25T10:29:10.187981+0000","last_change":"2024-04-25T10:29:09.016966+0000","last_active":"2024-04-25T10:29:10.187981+0000","last_peered":"2024-04-25T10:29:10.187981+0000","last_clean":"2024-04-25T10:29:10.187981+0000","last_became_active":"2024-04-25T10:29:09.016566+0000","last_became_peered":"2024-04-25T10:29:09.016566+0000","last_unstale":"2024-04-25T10:29:10.187981+0000","last_undegraded":"2024-04-25T10:29:10.187981+0000","last_fullsized":"2024-04-25T10:29:10.187981+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:29:07.995096+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:29:07.995096+0000","last_clean_scrub_stamp":"2024-04-25T10:29:07.995096+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T10:55:01.505157+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.36","version":"0'0","reported_seq":12,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-04-25T10:29:10.037810+0000","last_change":"2024-04-25T10:29:09.030321+0000","last_active":"2024-04-25T10:29:10.037810+0000","last_peered":"2024-04-25T10:29:10.037810+0000","last_clean":"2024-04-25T10:29:10.037810+0000","last_became_active":"2024-04-25T10:29:09.029976+0000","last_became_peered":"2024-04-25T10:29:09.029976+0000","last_unstale":"2024-04-25T10:29:10.037810+0000","last_undegraded":"2024-04-25T10:29:10.037810+0000","last_fullsized":"2024-04-25T10:29:10.037810+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:29:07.995096+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:29:07.995096+0000","last_clean_scrub_stamp":"2024-04-25T10:29:07.995096+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T12:36:31.191100+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.35","version":"0'0","reported_seq":12,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-04-25T10:29:10.037641+0000","last_change":"2024-04-25T10:29:09.030469+0000","last_active":"2024-04-25T10:29:10.037641+0000","last_peered":"2024-04-25T10:29:10.037641+0000","last_clean":"2024-04-25T10:29:10.037641+0000","last_became_active":"2024-04-25T10:29:09.029350+0000","last_became_peered":"2024-04-25T10:29:09.029350+0000","last_unstale":"2024-04-25T10:29:10.037641+0000","last_undegraded":"2024-04-25T10:29:10.037641+0000","last_fullsized":"2024-04-25T10:29:10.037641+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:29:07.995096+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:29:07.995096+0000","last_clean_scrub_stamp":"2024-04-25T10:29:07.995096+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T15:51:52.989545+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.34","version":"0'0","reported_seq":12,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-04-25T10:29:10.037728+0000","last_change":"2024-04-25T10:29:09.029772+0000","last_active":"2024-04-25T10:29:10.037728+0000","last_peered":"2024-04-25T10:29:10.037728+0000","last_clean":"2024-04-25T10:29:10.037728+0000","last_became_active":"2024-04-25T10:29:09.029350+0000","last_became_peered":"2024-04-25T10:29:09.029350+0000","last_unstale":"2024-04-25T10:29:10.037728+0000","last_undegraded":"2024-04-25T10:29:10.037728+0000","last_fullsized":"2024-04-25T10:29:10.037728+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:29:07.995096+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:29:07.995096+0000","last_clean_scrub_stamp":"2024-04-25T10:29:07.995096+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T19:14:48.737269+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.33","version":"0'0","reported_seq":12,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-04-25T10:29:10.188971+0000","last_change":"2024-04-25T10:29:09.038690+0000","last_active":"2024-04-25T10:29:10.188971+0000","last_peered":"2024-04-25T10:29:10.188971+0000","last_clean":"2024-04-25T10:29:10.188971+0000","last_became_active":"2024-04-25T10:29:09.038336+0000","last_became_peered":"2024-04-25T10:29:09.038336+0000","last_unstale":"2024-04-25T10:29:10.188971+0000","last_undegraded":"2024-04-25T10:29:10.188971+0000","last_fullsized":"2024-04-25T10:29:10.188971+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:29:07.995096+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:29:07.995096+0000","last_clean_scrub_stamp":"2024-04-25T10:29:07.995096+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T19:23:12.589414+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.32","version":"0'0","reported_seq":12,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-04-25T10:29:10.188641+0000","last_change":"2024-04-25T10:29:09.028195+0000","last_active":"2024-04-25T10:29:10.188641+0000","last_peered":"2024-04-25T10:29:10.188641+0000","last_clean":"2024-04-25T10:29:10.188641+0000","last_became_active":"2024-04-25T10:29:09.027893+0000","last_became_peered":"2024-04-25T10:29:09.027893+0000","last_unstale":"2024-04-25T10:29:10.188641+0000","last_undegraded":"2024-04-25T10:29:10.188641+0000","last_fullsized":"2024-04-25T10:29:10.188641+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:29:07.995096+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:29:07.995096+0000","last_clean_scrub_stamp":"2024-04-25T10:29:07.995096+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T11:35:27.466874+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.31","version":"0'0","reported_seq":12,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-04-25T10:29:10.188956+0000","last_change":"2024-04-25T10:29:09.034532+0000","last_active":"2024-04-25T10:29:10.188956+0000","last_peered":"2024-04-25T10:29:10.188956+0000","last_clean":"2024-04-25T10:29:10.188956+0000","last_became_active":"2024-04-25T10:29:09.034021+0000","last_became_peered":"2024-04-25T10:29:09.034021+0000","last_unstale":"2024-04-25T10:29:10.188956+0000","last_undegraded":"2024-04-25T10:29:10.188956+0000","last_fullsized":"2024-04-25T10:29:10.188956+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:29:07.995096+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:29:07.995096+0000","last_clean_scrub_stamp":"2024-04-25T10:29:07.995096+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T12:16:04.880732+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.30","version":"0'0","reported_seq":12,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-04-25T10:29:10.024134+0000","last_change":"2024-04-25T10:29:09.021224+0000","last_active":"2024-04-25T10:29:10.024134+0000","last_peered":"2024-04-25T10:29:10.024134+0000","last_clean":"2024-04-25T10:29:10.024134+0000","last_became_active":"2024-04-25T10:29:09.020964+0000","last_became_peered":"2024-04-25T10:29:09.020964+0000","last_unstale":"2024-04-25T10:29:10.024134+0000","last_undegraded":"2024-04-25T10:29:10.024134+0000","last_fullsized":"2024-04-25T10:29:10.024134+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:29:07.995096+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:29:07.995096+0000","last_clean_scrub_stamp":"2024-04-25T10:29:07.995096+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T18:44:23.610374+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.2f","version":"0'0","reported_seq":12,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-04-25T10:29:10.024944+0000","last_change":"2024-04-25T10:29:09.025638+0000","last_active":"2024-04-25T10:29:10.024944+0000","last_peered":"2024-04-25T10:29:10.024944+0000","last_clean":"2024-04-25T10:29:10.024944+0000","last_became_active":"2024-04-25T10:29:09.024517+0000","last_became_peered":"2024-04-25T10:29:09.024517+0000","last_unstale":"2024-04-25T10:29:10.024944+0000","last_undegraded":"2024-04-25T10:29:10.024944+0000","last_fullsized":"2024-04-25T10:29:10.024944+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:29:07.995096+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:29:07.995096+0000","last_clean_scrub_stamp":"2024-04-25T10:29:07.995096+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T16:29:08.975346+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.2e","version":"0'0","reported_seq":12,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-04-25T10:29:10.189006+0000","last_change":"2024-04-25T10:29:09.023923+0000","last_active":"2024-04-25T10:29:10.189006+0000","last_peered":"2024-04-25T10:29:10.189006+0000","last_clean":"2024-04-25T10:29:10.189006+0000","last_became_active":"2024-04-25T10:29:09.023579+0000","last_became_peered":"2024-04-25T10:29:09.023579+0000","last_unstale":"2024-04-25T10:29:10.189006+0000","last_undegraded":"2024-04-25T10:29:10.189006+0000","last_fullsized":"2024-04-25T10:29:10.189006+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:29:07.995096+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:29:07.995096+0000","last_clean_scrub_stamp":"2024-04-25T10:29:07.995096+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T10:35:31.361514+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.2d","version":"0'0","reported_seq":12,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-04-25T10:29:10.024852+0000","last_change":"2024-04-25T10:29:09.023039+0000","last_active":"2024-04-25T10:29:10.024852+0000","last_peered":"2024-04-25T10:29:10.024852+0000","last_clean":"2024-04-25T10:29:10.024852+0000","last_became_active":"2024-04-25T10:29:09.022706+0000","last_became_peered":"2024-04-25T10:29:09.022706+0000","last_unstale":"2024-04-25T10:29:10.024852+0000","last_undegraded":"2024-04-25T10:29:10.024852+0000","last_fullsized":"2024-04-25T10:29:10.024852+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:29:07.995096+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:29:07.995096+0000","last_clean_scrub_stamp":"2024-04-25T10:29:07.995096+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T16:48:17.352944+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.2c","version":"0'0","reported_seq":12,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-04-25T10:29:10.037891+0000","last_change":"2024-04-25T10:29:09.028979+0000","last_active":"2024-04-25T10:29:10.037891+0000","last_peered":"2024-04-25T10:29:10.037891+0000","last_clean":"2024-04-25T10:29:10.037891+0000","last_became_active":"2024-04-25T10:29:09.028061+0000","last_became_peered":"2024-04-25T10:29:09.028061+0000","last_unstale":"2024-04-25T10:29:10.037891+0000","last_undegraded":"2024-04-25T10:29:10.037891+0000","last_fullsized":"2024-04-25T10:29:10.037891+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:29:07.995096+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:29:07.995096+0000","last_clean_scrub_stamp":"2024-04-25T10:29:07.995096+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T18:44:23.610374+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.2b","version":"0'0","reported_seq":12,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-04-25T10:29:10.025554+0000","last_change":"2024-04-25T10:29:09.022874+0000","last_active":"2024-04-25T10:29:10.025554+0000","last_peered":"2024-04-25T10:29:10.025554+0000","last_clean":"2024-04-25T10:29:10.025554+0000","last_became_active":"2024-04-25T10:29:09.022360+0000","last_became_peered":"2024-04-25T10:29:09.022360+0000","last_unstale":"2024-04-25T10:29:10.025554+0000","last_undegraded":"2024-04-25T10:29:10.025554+0000","last_fullsized":"2024-04-25T10:29:10.025554+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:29:07.995096+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:29:07.995096+0000","last_clean_scrub_stamp":"2024-04-25T10:29:07.995096+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T19:32:59.816545+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.2a","version":"0'0","reported_seq":12,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-04-25T10:29:10.187695+0000","last_change":"2024-04-25T10:29:09.029965+0000","last_active":"2024-04-25T10:29:10.187695+0000","last_peered":"2024-04-25T10:29:10.187695+0000","last_clean":"2024-04-25T10:29:10.187695+0000","last_became_active":"2024-04-25T10:29:09.029616+0000","last_became_peered":"2024-04-25T10:29:09.029616+0000","last_unstale":"2024-04-25T10:29:10.187695+0000","last_undegraded":"2024-04-25T10:29:10.187695+0000","last_fullsized":"2024-04-25T10:29:10.187695+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:29:07.995096+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:29:07.995096+0000","last_clean_scrub_stamp":"2024-04-25T10:29:07.995096+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T15:58:36.848831+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.29","version":"0'0","reported_seq":12,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-04-25T10:29:10.189345+0000","last_change":"2024-04-25T10:29:09.033401+0000","last_active":"2024-04-25T10:29:10.189345+0000","last_peered":"2024-04-25T10:29:10.189345+0000","last_clean":"2024-04-25T10:29:10.189345+0000","last_became_active":"2024-04-25T10:29:09.032764+0000","last_became_peered":"2024-04-25T10:29:09.032764+0000","last_unstale":"2024-04-25T10:29:10.189345+0000","last_undegraded":"2024-04-25T10:29:10.189345+0000","last_fullsized":"2024-04-25T10:29:10.189345+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:29:07.995096+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:29:07.995096+0000","last_clean_scrub_stamp":"2024-04-25T10:29:07.995096+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T17:30:54.093151+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.28","version":"0'0","reported_seq":12,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-04-25T10:29:10.024133+0000","last_change":"2024-04-25T10:29:09.041063+0000","last_active":"2024-04-25T10:29:10.024133+0000","last_peered":"2024-04-25T10:29:10.024133+0000","last_clean":"2024-04-25T10:29:10.024133+0000","last_became_active":"2024-04-25T10:29:09.040734+0000","last_became_peered":"2024-04-25T10:29:09.040734+0000","last_unstale":"2024-04-25T10:29:10.024133+0000","last_undegraded":"2024-04-25T10:29:10.024133+0000","last_fullsized":"2024-04-25T10:29:10.024133+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:29:07.995096+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:29:07.995096+0000","last_clean_scrub_stamp":"2024-04-25T10:29:07.995096+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T13:54:22.968216+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.27","version":"0'0","reported_seq":12,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-04-25T10:29:10.189019+0000","last_change":"2024-04-25T10:29:09.034268+0000","last_active":"2024-04-25T10:29:10.189019+0000","last_peered":"2024-04-25T10:29:10.189019+0000","last_clean":"2024-04-25T10:29:10.189019+0000","last_became_active":"2024-04-25T10:29:09.033199+0000","last_became_peered":"2024-04-25T10:29:09.033199+0000","last_unstale":"2024-04-25T10:29:10.189019+0000","last_undegraded":"2024-04-25T10:29:10.189019+0000","last_fullsized":"2024-04-25T10:29:10.189019+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:29:07.995096+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:29:07.995096+0000","last_clean_scrub_stamp":"2024-04-25T10:29:07.995096+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T17:56:09.999494+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.26","version":"0'0","reported_seq":12,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-04-25T10:29:10.026183+0000","last_change":"2024-04-25T10:29:09.025643+0000","last_active":"2024-04-25T10:29:10.026183+0000","last_peered":"2024-04-25T10:29:10.026183+0000","last_clean":"2024-04-25T10:29:10.026183+0000","last_became_active":"2024-04-25T10:29:09.024894+0000","last_became_peered":"2024-04-25T10:29:09.024894+0000","last_unstale":"2024-04-25T10:29:10.026183+0000","last_undegraded":"2024-04-25T10:29:10.026183+0000","last_fullsized":"2024-04-25T10:29:10.026183+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:29:07.995096+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:29:07.995096+0000","last_clean_scrub_stamp":"2024-04-25T10:29:07.995096+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T22:08:00.367431+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.25","version":"0'0","reported_seq":12,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-04-25T10:29:10.189130+0000","last_change":"2024-04-25T10:29:09.023357+0000","last_active":"2024-04-25T10:29:10.189130+0000","last_peered":"2024-04-25T10:29:10.189130+0000","last_clean":"2024-04-25T10:29:10.189130+0000","last_became_active":"2024-04-25T10:29:09.022988+0000","last_became_peered":"2024-04-25T10:29:09.022988+0000","last_unstale":"2024-04-25T10:29:10.189130+0000","last_undegraded":"2024-04-25T10:29:10.189130+0000","last_fullsized":"2024-04-25T10:29:10.189130+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:29:07.995096+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:29:07.995096+0000","last_clean_scrub_stamp":"2024-04-25T10:29:07.995096+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T21:14:39.661957+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.24","version":"0'0","reported_seq":12,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-04-25T10:29:10.189239+0000","last_change":"2024-04-25T10:29:09.034328+0000","last_active":"2024-04-25T10:29:10.189239+0000","last_peered":"2024-04-25T10:29:10.189239+0000","last_clean":"2024-04-25T10:29:10.189239+0000","last_became_active":"2024-04-25T10:29:09.033978+0000","last_became_peered":"2024-04-25T10:29:09.033978+0000","last_unstale":"2024-04-25T10:29:10.189239+0000","last_undegraded":"2024-04-25T10:29:10.189239+0000","last_fullsized":"2024-04-25T10:29:10.189239+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:29:07.995096+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:29:07.995096+0000","last_clean_scrub_stamp":"2024-04-25T10:29:07.995096+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T21:24:14.497242+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.23","version":"0'0","reported_seq":12,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-04-25T10:29:10.189583+0000","last_change":"2024-04-25T10:29:09.034045+0000","last_active":"2024-04-25T10:29:10.189583+0000","last_peered":"2024-04-25T10:29:10.189583+0000","last_clean":"2024-04-25T10:29:10.189583+0000","last_became_active":"2024-04-25T10:29:09.033730+0000","last_became_peered":"2024-04-25T10:29:09.033730+0000","last_unstale":"2024-04-25T10:29:10.189583+0000","last_undegraded":"2024-04-25T10:29:10.189583+0000","last_fullsized":"2024-04-25T10:29:10.189583+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:29:07.995096+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:29:07.995096+0000","last_clean_scrub_stamp":"2024-04-25T10:29:07.995096+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T12:36:31.191100+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.22","version":"0'0","reported_seq":12,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-04-25T10:29:10.024663+0000","last_change":"2024-04-25T10:29:09.034262+0000","last_active":"2024-04-25T10:29:10.024663+0000","last_peered":"2024-04-25T10:29:10.024663+0000","last_clean":"2024-04-25T10:29:10.024663+0000","last_became_active":"2024-04-25T10:29:09.033905+0000","last_became_peered":"2024-04-25T10:29:09.033905+0000","last_unstale":"2024-04-25T10:29:10.024663+0000","last_undegraded":"2024-04-25T10:29:10.024663+0000","last_fullsized":"2024-04-25T10:29:10.024663+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:29:07.995096+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:29:07.995096+0000","last_clean_scrub_stamp":"2024-04-25T10:29:07.995096+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T16:22:45.608928+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.21","version":"0'0","reported_seq":12,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-04-25T10:29:10.188201+0000","last_change":"2024-04-25T10:29:09.027088+0000","last_active":"2024-04-25T10:29:10.188201+0000","last_peered":"2024-04-25T10:29:10.188201+0000","last_clean":"2024-04-25T10:29:10.188201+0000","last_became_active":"2024-04-25T10:29:09.026651+0000","last_became_peered":"2024-04-25T10:29:09.026651+0000","last_unstale":"2024-04-25T10:29:10.188201+0000","last_undegraded":"2024-04-25T10:29:10.188201+0000","last_fullsized":"2024-04-25T10:29:10.188201+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:29:07.995096+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:29:07.995096+0000","last_clean_scrub_stamp":"2024-04-25T10:29:07.995096+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T15:37:06.723380+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.20","version":"0'0","reported_seq":12,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-04-25T10:29:10.025741+0000","last_change":"2024-04-25T10:29:09.028008+0000","last_active":"2024-04-25T10:29:10.025741+0000","last_peered":"2024-04-25T10:29:10.025741+0000","last_clean":"2024-04-25T10:29:10.025741+0000","last_became_active":"2024-04-25T10:29:09.027679+0000","last_became_peered":"2024-04-25T10:29:09.027679+0000","last_unstale":"2024-04-25T10:29:10.025741+0000","last_undegraded":"2024-04-25T10:29:10.025741+0000","last_fullsized":"2024-04-25T10:29:10.025741+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:29:07.995096+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:29:07.995096+0000","last_clean_scrub_stamp":"2024-04-25T10:29:07.995096+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T15:37:06.723380+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,6],"acting":[2,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.1a","version":"0'0","reported_seq":12,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-04-25T10:29:10.187787+0000","last_change":"2024-04-25T10:29:09.032819+0000","last_active":"2024-04-25T10:29:10.187787+0000","last_peered":"2024-04-25T10:29:10.187787+0000","last_clean":"2024-04-25T10:29:10.187787+0000","last_became_active":"2024-04-25T10:29:09.032493+0000","last_became_peered":"2024-04-25T10:29:09.032493+0000","last_unstale":"2024-04-25T10:29:10.187787+0000","last_undegraded":"2024-04-25T10:29:10.187787+0000","last_fullsized":"2024-04-25T10:29:10.187787+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:29:07.995096+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:29:07.995096+0000","last_clean_scrub_stamp":"2024-04-25T10:29:07.995096+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T15:37:06.723380+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.1b","version":"22'1","reported_seq":15,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-04-25T10:29:10.189463+0000","last_change":"2024-04-25T10:29:08.025272+0000","last_active":"2024-04-25T10:29:10.189463+0000","last_peered":"2024-04-25T10:29:10.189463+0000","last_clean":"2024-04-25T10:29:10.189463+0000","last_became_active":"2024-04-25T10:29:08.024912+0000","last_became_peered":"2024-04-25T10:29:08.024912+0000","last_unstale":"2024-04-25T10:29:10.189463+0000","last_undegraded":"2024-04-25T10:29:10.189463+0000","last_fullsized":"2024-04-25T10:29:10.189463+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:29:06.989570+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:29:06.989570+0000","last_clean_scrub_stamp":"2024-04-25T10:29:06.989570+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T18:11:32.758179+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6],"acting":[5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.1b","version":"0'0","reported_seq":12,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-04-25T10:29:10.025596+0000","last_change":"2024-04-25T10:29:09.032005+0000","last_active":"2024-04-25T10:29:10.025596+0000","last_peered":"2024-04-25T10:29:10.025596+0000","last_clean":"2024-04-25T10:29:10.025596+0000","last_became_active":"2024-04-25T10:29:09.031572+0000","last_became_peered":"2024-04-25T10:29:09.031572+0000","last_unstale":"2024-04-25T10:29:10.025596+0000","last_undegraded":"2024-04-25T10:29:10.025596+0000","last_fullsized":"2024-04-25T10:29:10.025596+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:29:07.995096+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:29:07.995096+0000","last_clean_scrub_stamp":"2024-04-25T10:29:07.995096+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T12:47:09.477097+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.1a","version":"22'1","reported_seq":15,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-04-25T10:29:10.024506+0000","last_change":"2024-04-25T10:29:08.026018+0000","last_active":"2024-04-25T10:29:10.024506+0000","last_peered":"2024-04-25T10:29:10.024506+0000","last_clean":"2024-04-25T10:29:10.024506+0000","last_became_active":"2024-04-25T10:29:08.025668+0000","last_became_peered":"2024-04-25T10:29:08.025668+0000","last_unstale":"2024-04-25T10:29:10.024506+0000","last_undegraded":"2024-04-25T10:29:10.024506+0000","last_fullsized":"2024-04-25T10:29:10.024506+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:29:06.989570+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:29:06.989570+0000","last_clean_scrub_stamp":"2024-04-25T10:29:06.989570+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T11:35:26.461348+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.19","version":"0'0","reported_seq":14,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-04-25T10:29:10.188333+0000","last_change":"2024-04-25T10:29:08.011702+0000","last_active":"2024-04-25T10:29:10.188333+0000","last_peered":"2024-04-25T10:29:10.188333+0000","last_clean":"2024-04-25T10:29:10.188333+0000","last_became_active":"2024-04-25T10:29:08.011353+0000","last_became_peered":"2024-04-25T10:29:08.011353+0000","last_unstale":"2024-04-25T10:29:10.188333+0000","last_undegraded":"2024-04-25T10:29:10.188333+0000","last_fullsized":"2024-04-25T10:29:10.188333+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:29:06.989570+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:29:06.989570+0000","last_clean_scrub_stamp":"2024-04-25T10:29:06.989570+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T16:31:11.685101+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.18","version":"0'0","reported_seq":12,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-04-25T10:29:10.189830+0000","last_change":"2024-04-25T10:29:09.033872+0000","last_active":"2024-04-25T10:29:10.189830+0000","last_peered":"2024-04-25T10:29:10.189830+0000","last_clean":"2024-04-25T10:29:10.189830+0000","last_became_active":"2024-04-25T10:29:09.033544+0000","last_became_peered":"2024-04-25T10:29:09.033544+0000","last_unstale":"2024-04-25T10:29:10.189830+0000","last_undegraded":"2024-04-25T10:29:10.189830+0000","last_fullsized":"2024-04-25T10:29:10.189830+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:29:07.995096+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:29:07.995096+0000","last_clean_scrub_stamp":"2024-04-25T10:29:07.995096+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T13:54:22.968216+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6],"acting":[5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.18","version":"0'0","reported_seq":14,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-04-25T10:29:10.024423+0000","last_change":"2024-04-25T10:29:08.013340+0000","last_active":"2024-04-25T10:29:10.024423+0000","last_peered":"2024-04-25T10:29:10.024423+0000","last_clean":"2024-04-25T10:29:10.024423+0000","last_became_active":"2024-04-25T10:29:08.012940+0000","last_became_peered":"2024-04-25T10:29:08.012940+0000","last_unstale":"2024-04-25T10:29:10.024423+0000","last_undegraded":"2024-04-25T10:29:10.024423+0000","last_fullsized":"2024-04-25T10:29:10.024423+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:29:06.989570+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:29:06.989570+0000","last_clean_scrub_stamp":"2024-04-25T10:29:06.989570+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T10:35:30.355988+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.19","version":"0'0","reported_seq":12,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-04-25T10:29:10.025603+0000","last_change":"2024-04-25T10:29:09.021718+0000","last_active":"2024-04-25T10:29:10.025603+0000","last_peered":"2024-04-25T10:29:10.025603+0000","last_clean":"2024-04-25T10:29:10.025603+0000","last_became_active":"2024-04-25T10:29:09.021416+0000","last_became_peered":"2024-04-25T10:29:09.021416+0000","last_unstale":"2024-04-25T10:29:10.025603+0000","last_undegraded":"2024-04-25T10:29:10.025603+0000","last_fullsized":"2024-04-25T10:29:10.025603+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:29:07.995096+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:29:07.995096+0000","last_clean_scrub_stamp":"2024-04-25T10:29:07.995096+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T18:55:45.597763+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.17","version":"0'0","reported_seq":14,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-04-25T10:29:10.187949+0000","last_change":"2024-04-25T10:29:08.020281+0000","last_active":"2024-04-25T10:29:10.187949+0000","last_peered":"2024-04-25T10:29:10.187949+0000","last_clean":"2024-04-25T10:29:10.187949+0000","last_became_active":"2024-04-25T10:29:08.019921+0000","last_became_peered":"2024-04-25T10:29:08.019921+0000","last_unstale":"2024-04-25T10:29:10.187949+0000","last_undegraded":"2024-04-25T10:29:10.187949+0000","last_fullsized":"2024-04-25T10:29:10.187949+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:29:06.989570+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:29:06.989570+0000","last_clean_scrub_stamp":"2024-04-25T10:29:06.989570+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T19:13:07.068780+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.16","version":"0'0","reported_seq":12,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-04-25T10:29:10.026197+0000","last_change":"2024-04-25T10:29:09.022744+0000","last_active":"2024-04-25T10:29:10.026197+0000","last_peered":"2024-04-25T10:29:10.026197+0000","last_clean":"2024-04-25T10:29:10.026197+0000","last_became_active":"2024-04-25T10:29:09.022392+0000","last_became_peered":"2024-04-25T10:29:09.022392+0000","last_unstale":"2024-04-25T10:29:10.026197+0000","last_undegraded":"2024-04-25T10:29:10.026197+0000","last_fullsized":"2024-04-25T10:29:10.026197+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:29:07.995096+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:29:07.995096+0000","last_clean_scrub_stamp":"2024-04-25T10:29:07.995096+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T11:35:27.466874+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.16","version":"0'0","reported_seq":14,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-04-25T10:29:10.037869+0000","last_change":"2024-04-25T10:29:08.026388+0000","last_active":"2024-04-25T10:29:10.037869+0000","last_peered":"2024-04-25T10:29:10.037869+0000","last_clean":"2024-04-25T10:29:10.037869+0000","last_became_active":"2024-04-25T10:29:08.026039+0000","last_became_peered":"2024-04-25T10:29:08.026039+0000","last_unstale":"2024-04-25T10:29:10.037869+0000","last_undegraded":"2024-04-25T10:29:10.037869+0000","last_fullsized":"2024-04-25T10:29:10.037869+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:29:06.989570+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:29:06.989570+0000","last_clean_scrub_stamp":"2024-04-25T10:29:06.989570+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T18:11:32.758179+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.17","version":"0'0","reported_seq":12,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-04-25T10:29:10.189465+0000","last_change":"2024-04-25T10:29:09.032902+0000","last_active":"2024-04-25T10:29:10.189465+0000","last_peered":"2024-04-25T10:29:10.189465+0000","last_clean":"2024-04-25T10:29:10.189465+0000","last_became_active":"2024-04-25T10:29:09.032596+0000","last_became_peered":"2024-04-25T10:29:09.032596+0000","last_unstale":"2024-04-25T10:29:10.189465+0000","last_undegraded":"2024-04-25T10:29:10.189465+0000","last_fullsized":"2024-04-25T10:29:10.189465+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:29:07.995096+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:29:07.995096+0000","last_clean_scrub_stamp":"2024-04-25T10:29:07.995096+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T19:14:48.737269+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.15","version":"22'3","reported_seq":17,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-04-25T10:29:10.188352+0000","last_change":"2024-04-25T10:29:08.021347+0000","last_active":"2024-04-25T10:29:10.188352+0000","last_peered":"2024-04-25T10:29:10.188352+0000","last_clean":"2024-04-25T10:29:10.188352+0000","last_became_active":"2024-04-25T10:29:08.021039+0000","last_became_peered":"2024-04-25T10:29:08.021039+0000","last_unstale":"2024-04-25T10:29:10.188352+0000","last_undegraded":"2024-04-25T10:29:10.188352+0000","last_fullsized":"2024-04-25T10:29:10.188352+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:29:06.989570+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:29:06.989570+0000","last_clean_scrub_stamp":"2024-04-25T10:29:06.989570+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T20:22:56.380539+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.14","version":"0'0","reported_seq":12,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-04-25T10:29:10.024492+0000","last_change":"2024-04-25T10:29:09.021410+0000","last_active":"2024-04-25T10:29:10.024492+0000","last_peered":"2024-04-25T10:29:10.024492+0000","last_clean":"2024-04-25T10:29:10.024492+0000","last_became_active":"2024-04-25T10:29:09.021051+0000","last_became_peered":"2024-04-25T10:29:09.021051+0000","last_unstale":"2024-04-25T10:29:10.024492+0000","last_undegraded":"2024-04-25T10:29:10.024492+0000","last_fullsized":"2024-04-25T10:29:10.024492+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:29:07.995096+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:29:07.995096+0000","last_clean_scrub_stamp":"2024-04-25T10:29:07.995096+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T13:22:13.080513+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.14","version":"22'6","reported_seq":20,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-04-25T10:29:10.025912+0000","last_change":"2024-04-25T10:29:08.018044+0000","last_active":"2024-04-25T10:29:10.025912+0000","last_peered":"2024-04-25T10:29:10.025912+0000","last_clean":"2024-04-25T10:29:10.025912+0000","last_became_active":"2024-04-25T10:29:08.016984+0000","last_became_peered":"2024-04-25T10:29:08.016984+0000","last_unstale":"2024-04-25T10:29:10.025912+0000","last_undegraded":"2024-04-25T10:29:10.025912+0000","last_fullsized":"2024-04-25T10:29:10.025912+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:29:06.989570+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:29:06.989570+0000","last_clean_scrub_stamp":"2024-04-25T10:29:06.989570+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T18:29:56.525236+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1612,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4],"acting":[2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.15","version":"0'0","reported_seq":12,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-04-25T10:29:10.037668+0000","last_change":"2024-04-25T10:29:09.029034+0000","last_active":"2024-04-25T10:29:10.037668+0000","last_peered":"2024-04-25T10:29:10.037668+0000","last_clean":"2024-04-25T10:29:10.037668+0000","last_became_active":"2024-04-25T10:29:09.028662+0000","last_became_peered":"2024-04-25T10:29:09.028662+0000","last_unstale":"2024-04-25T10:29:10.037668+0000","last_undegraded":"2024-04-25T10:29:10.037668+0000","last_fullsized":"2024-04-25T10:29:10.037668+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:29:07.995096+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:29:07.995096+0000","last_clean_scrub_stamp":"2024-04-25T10:29:07.995096+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T17:30:54.093151+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.13","version":"22'2","reported_seq":16,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-04-25T10:29:10.189430+0000","last_change":"2024-04-25T10:29:08.020476+0000","last_active":"2024-04-25T10:29:10.189430+0000","last_peered":"2024-04-25T10:29:10.189430+0000","last_clean":"2024-04-25T10:29:10.189430+0000","last_became_active":"2024-04-25T10:29:08.020096+0000","last_became_peered":"2024-04-25T10:29:08.020096+0000","last_unstale":"2024-04-25T10:29:10.189430+0000","last_undegraded":"2024-04-25T10:29:10.189430+0000","last_fullsized":"2024-04-25T10:29:10.189430+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:29:06.989570+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:29:06.989570+0000","last_clean_scrub_stamp":"2024-04-25T10:29:06.989570+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T10:32:06.103123+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":34,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":7,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.12","version":"0'0","reported_seq":12,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-04-25T10:29:10.188838+0000","last_change":"2024-04-25T10:29:09.027996+0000","last_active":"2024-04-25T10:29:10.188838+0000","last_peered":"2024-04-25T10:29:10.188838+0000","last_clean":"2024-04-25T10:29:10.188838+0000","last_became_active":"2024-04-25T10:29:09.027538+0000","last_became_peered":"2024-04-25T10:29:09.027538+0000","last_unstale":"2024-04-25T10:29:10.188838+0000","last_undegraded":"2024-04-25T10:29:10.188838+0000","last_fullsized":"2024-04-25T10:29:10.188838+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:29:07.995096+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:29:07.995096+0000","last_clean_scrub_stamp":"2024-04-25T10:29:07.995096+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T12:16:47.983013+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.12","version":"0'0","reported_seq":14,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-04-25T10:29:10.024603+0000","last_change":"2024-04-25T10:29:08.019028+0000","last_active":"2024-04-25T10:29:10.024603+0000","last_peered":"2024-04-25T10:29:10.024603+0000","last_clean":"2024-04-25T10:29:10.024603+0000","last_became_active":"2024-04-25T10:29:08.018582+0000","last_became_peered":"2024-04-25T10:29:08.018582+0000","last_unstale":"2024-04-25T10:29:10.024603+0000","last_undegraded":"2024-04-25T10:29:10.024603+0000","last_fullsized":"2024-04-25T10:29:10.024603+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:29:06.989570+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:29:06.989570+0000","last_clean_scrub_stamp":"2024-04-25T10:29:06.989570+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T19:23:11.583888+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.13","version":"0'0","reported_seq":12,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-04-25T10:29:10.024281+0000","last_change":"2024-04-25T10:29:09.023995+0000","last_active":"2024-04-25T10:29:10.024281+0000","last_peered":"2024-04-25T10:29:10.024281+0000","last_clean":"2024-04-25T10:29:10.024281+0000","last_became_active":"2024-04-25T10:29:09.023690+0000","last_became_peered":"2024-04-25T10:29:09.023690+0000","last_unstale":"2024-04-25T10:29:10.024281+0000","last_undegraded":"2024-04-25T10:29:10.024281+0000","last_fullsized":"2024-04-25T10:29:10.024281+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:29:07.995096+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:29:07.995096+0000","last_clean_scrub_stamp":"2024-04-25T10:29:07.995096+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T15:28:23.689564+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.11","version":"0'0","reported_seq":14,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-04-25T10:29:10.037576+0000","last_change":"2024-04-25T10:29:08.027174+0000","last_active":"2024-04-25T10:29:10.037576+0000","last_peered":"2024-04-25T10:29:10.037576+0000","last_clean":"2024-04-25T10:29:10.037576+0000","last_became_active":"2024-04-25T10:29:08.026809+0000","last_became_peered":"2024-04-25T10:29:08.026809+0000","last_unstale":"2024-04-25T10:29:10.037576+0000","last_undegraded":"2024-04-25T10:29:10.037576+0000","last_fullsized":"2024-04-25T10:29:10.037576+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:29:06.989570+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:29:06.989570+0000","last_clean_scrub_stamp":"2024-04-25T10:29:06.989570+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T10:35:30.355988+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.10","version":"0'0","reported_seq":12,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-04-25T10:29:10.187809+0000","last_change":"2024-04-25T10:29:09.041722+0000","last_active":"2024-04-25T10:29:10.187809+0000","last_peered":"2024-04-25T10:29:10.187809+0000","last_clean":"2024-04-25T10:29:10.187809+0000","last_became_active":"2024-04-25T10:29:09.040971+0000","last_became_peered":"2024-04-25T10:29:09.040971+0000","last_unstale":"2024-04-25T10:29:10.187809+0000","last_undegraded":"2024-04-25T10:29:10.187809+0000","last_fullsized":"2024-04-25T10:29:10.187809+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:29:07.995096+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:29:07.995096+0000","last_clean_scrub_stamp":"2024-04-25T10:29:07.995096+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T10:35:31.361514+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.10","version":"22'8","reported_seq":22,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-04-25T10:29:10.025277+0000","last_change":"2024-04-25T10:29:08.014110+0000","last_active":"2024-04-25T10:29:10.025277+0000","last_peered":"2024-04-25T10:29:10.025277+0000","last_clean":"2024-04-25T10:29:10.025277+0000","last_became_active":"2024-04-25T10:29:08.013543+0000","last_became_peered":"2024-04-25T10:29:08.013543+0000","last_unstale":"2024-04-25T10:29:10.025277+0000","last_undegraded":"2024-04-25T10:29:10.025277+0000","last_fullsized":"2024-04-25T10:29:10.025277+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:29:06.989570+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:29:06.989570+0000","last_clean_scrub_stamp":"2024-04-25T10:29:06.989570+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T16:44:30.899898+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":12,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7],"acting":[4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.11","version":"0'0","reported_seq":12,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-04-25T10:29:10.024238+0000","last_change":"2024-04-25T10:29:09.020490+0000","last_active":"2024-04-25T10:29:10.024238+0000","last_peered":"2024-04-25T10:29:10.024238+0000","last_clean":"2024-04-25T10:29:10.024238+0000","last_became_active":"2024-04-25T10:29:09.020178+0000","last_became_peered":"2024-04-25T10:29:09.020178+0000","last_unstale":"2024-04-25T10:29:10.024238+0000","last_undegraded":"2024-04-25T10:29:10.024238+0000","last_fullsized":"2024-04-25T10:29:10.024238+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:29:07.995096+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:29:07.995096+0000","last_clean_scrub_stamp":"2024-04-25T10:29:07.995096+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T20:37:20.726766+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.f","version":"22'1","reported_seq":15,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-04-25T10:29:10.038104+0000","last_change":"2024-04-25T10:29:08.029879+0000","last_active":"2024-04-25T10:29:10.038104+0000","last_peered":"2024-04-25T10:29:10.038104+0000","last_clean":"2024-04-25T10:29:10.038104+0000","last_became_active":"2024-04-25T10:29:08.029549+0000","last_became_peered":"2024-04-25T10:29:08.029549+0000","last_unstale":"2024-04-25T10:29:10.038104+0000","last_undegraded":"2024-04-25T10:29:10.038104+0000","last_fullsized":"2024-04-25T10:29:10.038104+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:29:06.989570+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:29:06.989570+0000","last_clean_scrub_stamp":"2024-04-25T10:29:06.989570+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T15:37:05.717854+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.e","version":"0'0","reported_seq":12,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-04-25T10:29:10.026273+0000","last_change":"2024-04-25T10:29:09.033703+0000","last_active":"2024-04-25T10:29:10.026273+0000","last_peered":"2024-04-25T10:29:10.026273+0000","last_clean":"2024-04-25T10:29:10.026273+0000","last_became_active":"2024-04-25T10:29:09.033318+0000","last_became_peered":"2024-04-25T10:29:09.033318+0000","last_unstale":"2024-04-25T10:29:10.026273+0000","last_undegraded":"2024-04-25T10:29:10.026273+0000","last_fullsized":"2024-04-25T10:29:10.026273+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:29:07.995096+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:29:07.995096+0000","last_clean_scrub_stamp":"2024-04-25T10:29:07.995096+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T12:16:47.983013+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.e","version":"22'2","reported_seq":16,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-04-25T10:29:10.026042+0000","last_change":"2024-04-25T10:29:08.020703+0000","last_active":"2024-04-25T10:29:10.026042+0000","last_peered":"2024-04-25T10:29:10.026042+0000","last_clean":"2024-04-25T10:29:10.026042+0000","last_became_active":"2024-04-25T10:29:08.020321+0000","last_became_peered":"2024-04-25T10:29:08.020321+0000","last_unstale":"2024-04-25T10:29:10.026042+0000","last_undegraded":"2024-04-25T10:29:10.026042+0000","last_fullsized":"2024-04-25T10:29:10.026042+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:29:06.989570+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:29:06.989570+0000","last_clean_scrub_stamp":"2024-04-25T10:29:06.989570+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T15:42:07.059913+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5],"acting":[4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.f","version":"0'0","reported_seq":12,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-04-25T10:29:10.038152+0000","last_change":"2024-04-25T10:29:09.022708+0000","last_active":"2024-04-25T10:29:10.038152+0000","last_peered":"2024-04-25T10:29:10.038152+0000","last_clean":"2024-04-25T10:29:10.038152+0000","last_became_active":"2024-04-25T10:29:09.020856+0000","last_became_peered":"2024-04-25T10:29:09.020856+0000","last_unstale":"2024-04-25T10:29:10.038152+0000","last_undegraded":"2024-04-25T10:29:10.038152+0000","last_fullsized":"2024-04-25T10:29:10.038152+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:29:07.995096+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:29:07.995096+0000","last_clean_scrub_stamp":"2024-04-25T10:29:07.995096+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T15:28:23.689564+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.d","version":"22'1","reported_seq":15,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-04-25T10:29:10.189469+0000","last_change":"2024-04-25T10:29:08.019488+0000","last_active":"2024-04-25T10:29:10.189469+0000","last_peered":"2024-04-25T10:29:10.189469+0000","last_clean":"2024-04-25T10:29:10.189469+0000","last_became_active":"2024-04-25T10:29:08.018971+0000","last_became_peered":"2024-04-25T10:29:08.018971+0000","last_unstale":"2024-04-25T10:29:10.189469+0000","last_undegraded":"2024-04-25T10:29:10.189469+0000","last_fullsized":"2024-04-25T10:29:10.189469+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:29:06.989570+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:29:06.989570+0000","last_clean_scrub_stamp":"2024-04-25T10:29:06.989570+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T20:22:56.380539+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":70,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.c","version":"0'0","reported_seq":12,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-04-25T10:29:10.037902+0000","last_change":"2024-04-25T10:29:09.029666+0000","last_active":"2024-04-25T10:29:10.037902+0000","last_peered":"2024-04-25T10:29:10.037902+0000","last_clean":"2024-04-25T10:29:10.037902+0000","last_became_active":"2024-04-25T10:29:09.029184+0000","last_became_peered":"2024-04-25T10:29:09.029184+0000","last_unstale":"2024-04-25T10:29:10.037902+0000","last_undegraded":"2024-04-25T10:29:10.037902+0000","last_fullsized":"2024-04-25T10:29:10.037902+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:29:07.995096+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:29:07.995096+0000","last_clean_scrub_stamp":"2024-04-25T10:29:07.995096+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T13:54:22.968216+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.c","version":"0'0","reported_seq":14,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-04-25T10:29:10.189541+0000","last_change":"2024-04-25T10:29:08.012124+0000","last_active":"2024-04-25T10:29:10.189541+0000","last_peered":"2024-04-25T10:29:10.189541+0000","last_clean":"2024-04-25T10:29:10.189541+0000","last_became_active":"2024-04-25T10:29:08.011817+0000","last_became_peered":"2024-04-25T10:29:08.011817+0000","last_unstale":"2024-04-25T10:29:10.189541+0000","last_undegraded":"2024-04-25T10:29:10.189541+0000","last_fullsized":"2024-04-25T10:29:10.189541+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:29:06.989570+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:29:06.989570+0000","last_clean_scrub_stamp":"2024-04-25T10:29:06.989570+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T20:22:56.380539+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.d","version":"0'0","reported_seq":12,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-04-25T10:29:10.024815+0000","last_change":"2024-04-25T10:29:09.017227+0000","last_active":"2024-04-25T10:29:10.024815+0000","last_peered":"2024-04-25T10:29:10.024815+0000","last_clean":"2024-04-25T10:29:10.024815+0000","last_became_active":"2024-04-25T10:29:09.016911+0000","last_became_peered":"2024-04-25T10:29:09.016911+0000","last_unstale":"2024-04-25T10:29:10.024815+0000","last_undegraded":"2024-04-25T10:29:10.024815+0000","last_fullsized":"2024-04-25T10:29:10.024815+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:29:07.995096+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:29:07.995096+0000","last_clean_scrub_stamp":"2024-04-25T10:29:07.995096+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T18:51:41.934269+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.0","version":"18'72","reported_seq":110,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-04-25T10:29:10.187517+0000","last_change":"2024-04-25T10:28:57.972696+0000","last_active":"2024-04-25T10:29:10.187517+0000","last_peered":"2024-04-25T10:29:10.187517+0000","last_clean":"2024-04-25T10:29:10.187517+0000","last_became_active":"2024-04-25T10:28:57.972373+0000","last_became_peered":"2024-04-25T10:28:57.972373+0000","last_unstale":"2024-04-25T10:29:10.187517+0000","last_undegraded":"2024-04-25T10:29:10.187517+0000","last_fullsized":"2024-04-25T10:29:10.187517+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:28:56.959439+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:28:56.959439+0000","last_clean_scrub_stamp":"2024-04-25T10:28:56.959439+0000","objects_scrubbed":0,"log_size":72,"log_dups_size":0,"ondisk_log_size":72,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T17:34:34.057408+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":66,"num_read_kb":55,"num_write":101,"num_write_kb":1628,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.3","version":"0'0","reported_seq":14,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-04-25T10:29:10.024562+0000","last_change":"2024-04-25T10:29:08.025080+0000","last_active":"2024-04-25T10:29:10.024562+0000","last_peered":"2024-04-25T10:29:10.024562+0000","last_clean":"2024-04-25T10:29:10.024562+0000","last_became_active":"2024-04-25T10:29:08.024747+0000","last_became_peered":"2024-04-25T10:29:08.024747+0000","last_unstale":"2024-04-25T10:29:10.024562+0000","last_undegraded":"2024-04-25T10:29:10.024562+0000","last_fullsized":"2024-04-25T10:29:10.024562+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:29:06.989570+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:29:06.989570+0000","last_clean_scrub_stamp":"2024-04-25T10:29:06.989570+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T22:07:59.361905+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.2","version":"0'0","reported_seq":12,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-04-25T10:29:10.025717+0000","last_change":"2024-04-25T10:29:09.023626+0000","last_active":"2024-04-25T10:29:10.025717+0000","last_peered":"2024-04-25T10:29:10.025717+0000","last_clean":"2024-04-25T10:29:10.025717+0000","last_became_active":"2024-04-25T10:29:09.020335+0000","last_became_peered":"2024-04-25T10:29:09.020335+0000","last_unstale":"2024-04-25T10:29:10.025717+0000","last_undegraded":"2024-04-25T10:29:10.025717+0000","last_fullsized":"2024-04-25T10:29:10.025717+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:29:07.995096+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:29:07.995096+0000","last_clean_scrub_stamp":"2024-04-25T10:29:07.995096+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T15:30:10.628413+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.0","version":"0'0","reported_seq":14,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-04-25T10:29:10.024309+0000","last_change":"2024-04-25T10:29:08.027479+0000","last_active":"2024-04-25T10:29:10.024309+0000","last_peered":"2024-04-25T10:29:10.024309+0000","last_clean":"2024-04-25T10:29:10.024309+0000","last_became_active":"2024-04-25T10:29:08.027178+0000","last_became_peered":"2024-04-25T10:29:08.027178+0000","last_unstale":"2024-04-25T10:29:10.024309+0000","last_undegraded":"2024-04-25T10:29:10.024309+0000","last_fullsized":"2024-04-25T10:29:10.024309+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:29:06.989570+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:29:06.989570+0000","last_clean_scrub_stamp":"2024-04-25T10:29:06.989570+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T19:54:16.697792+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.1","version":"0'0","reported_seq":12,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-04-25T10:29:10.188195+0000","last_change":"2024-04-25T10:29:09.028777+0000","last_active":"2024-04-25T10:29:10.188195+0000","last_peered":"2024-04-25T10:29:10.188195+0000","last_clean":"2024-04-25T10:29:10.188195+0000","last_became_active":"2024-04-25T10:29:09.027709+0000","last_became_peered":"2024-04-25T10:29:09.027709+0000","last_unstale":"2024-04-25T10:29:10.188195+0000","last_undegraded":"2024-04-25T10:29:10.188195+0000","last_fullsized":"2024-04-25T10:29:10.188195+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:29:07.995096+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:29:07.995096+0000","last_clean_scrub_stamp":"2024-04-25T10:29:07.995096+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T22:08:00.367431+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,2],"acting":[6,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.1","version":"0'0","reported_seq":14,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-04-25T10:29:10.188429+0000","last_change":"2024-04-25T10:29:08.020944+0000","last_active":"2024-04-25T10:29:10.188429+0000","last_peered":"2024-04-25T10:29:10.188429+0000","last_clean":"2024-04-25T10:29:10.188429+0000","last_became_active":"2024-04-25T10:29:08.020259+0000","last_became_peered":"2024-04-25T10:29:08.020259+0000","last_unstale":"2024-04-25T10:29:10.188429+0000","last_undegraded":"2024-04-25T10:29:10.188429+0000","last_fullsized":"2024-04-25T10:29:10.188429+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:29:06.989570+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:29:06.989570+0000","last_clean_scrub_stamp":"2024-04-25T10:29:06.989570+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T16:31:11.685101+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.0","version":"0'0","reported_seq":12,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-04-25T10:29:10.189305+0000","last_change":"2024-04-25T10:29:09.035455+0000","last_active":"2024-04-25T10:29:10.189305+0000","last_peered":"2024-04-25T10:29:10.189305+0000","last_clean":"2024-04-25T10:29:10.189305+0000","last_became_active":"2024-04-25T10:29:09.035130+0000","last_became_peered":"2024-04-25T10:29:09.035130+0000","last_unstale":"2024-04-25T10:29:10.189305+0000","last_undegraded":"2024-04-25T10:29:10.189305+0000","last_fullsized":"2024-04-25T10:29:10.189305+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:29:07.995096+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:29:07.995096+0000","last_clean_scrub_stamp":"2024-04-25T10:29:07.995096+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T11:52:24.415332+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.2","version":"0'0","reported_seq":14,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-04-25T10:29:10.024714+0000","last_change":"2024-04-25T10:29:08.021694+0000","last_active":"2024-04-25T10:29:10.024714+0000","last_peered":"2024-04-25T10:29:10.024714+0000","last_clean":"2024-04-25T10:29:10.024714+0000","last_became_active":"2024-04-25T10:29:08.021052+0000","last_became_peered":"2024-04-25T10:29:08.021052+0000","last_unstale":"2024-04-25T10:29:10.024714+0000","last_undegraded":"2024-04-25T10:29:10.024714+0000","last_fullsized":"2024-04-25T10:29:10.024714+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:29:06.989570+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:29:06.989570+0000","last_clean_scrub_stamp":"2024-04-25T10:29:06.989570+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T21:14:38.656431+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.3","version":"0'0","reported_seq":12,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-04-25T10:29:10.037784+0000","last_change":"2024-04-25T10:29:09.021686+0000","last_active":"2024-04-25T10:29:10.037784+0000","last_peered":"2024-04-25T10:29:10.037784+0000","last_clean":"2024-04-25T10:29:10.037784+0000","last_became_active":"2024-04-25T10:29:09.020672+0000","last_became_peered":"2024-04-25T10:29:09.020672+0000","last_unstale":"2024-04-25T10:29:10.037784+0000","last_undegraded":"2024-04-25T10:29:10.037784+0000","last_fullsized":"2024-04-25T10:29:10.037784+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:29:07.995096+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:29:07.995096+0000","last_clean_scrub_stamp":"2024-04-25T10:29:07.995096+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T20:52:02.098872+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.4","version":"0'0","reported_seq":14,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-04-25T10:29:10.189133+0000","last_change":"2024-04-25T10:29:08.025582+0000","last_active":"2024-04-25T10:29:10.189133+0000","last_peered":"2024-04-25T10:29:10.189133+0000","last_clean":"2024-04-25T10:29:10.189133+0000","last_became_active":"2024-04-25T10:29:08.024961+0000","last_became_peered":"2024-04-25T10:29:08.024961+0000","last_unstale":"2024-04-25T10:29:10.189133+0000","last_undegraded":"2024-04-25T10:29:10.189133+0000","last_fullsized":"2024-04-25T10:29:10.189133+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:29:06.989570+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:29:06.989570+0000","last_clean_scrub_stamp":"2024-04-25T10:29:06.989570+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T10:32:06.103123+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6],"acting":[5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.5","version":"0'0","reported_seq":12,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-04-25T10:29:10.026083+0000","last_change":"2024-04-25T10:29:09.028003+0000","last_active":"2024-04-25T10:29:10.026083+0000","last_peered":"2024-04-25T10:29:10.026083+0000","last_clean":"2024-04-25T10:29:10.026083+0000","last_became_active":"2024-04-25T10:29:09.027654+0000","last_became_peered":"2024-04-25T10:29:09.027654+0000","last_unstale":"2024-04-25T10:29:10.026083+0000","last_undegraded":"2024-04-25T10:29:10.026083+0000","last_fullsized":"2024-04-25T10:29:10.026083+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:29:07.995096+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:29:07.995096+0000","last_clean_scrub_stamp":"2024-04-25T10:29:07.995096+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T19:54:17.703318+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,6],"acting":[2,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.5","version":"0'0","reported_seq":14,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-04-25T10:29:10.024926+0000","last_change":"2024-04-25T10:29:08.026181+0000","last_active":"2024-04-25T10:29:10.024926+0000","last_peered":"2024-04-25T10:29:10.024926+0000","last_clean":"2024-04-25T10:29:10.024926+0000","last_became_active":"2024-04-25T10:29:08.025780+0000","last_became_peered":"2024-04-25T10:29:08.025780+0000","last_unstale":"2024-04-25T10:29:10.024926+0000","last_undegraded":"2024-04-25T10:29:10.024926+0000","last_fullsized":"2024-04-25T10:29:10.024926+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:29:06.989570+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:29:06.989570+0000","last_clean_scrub_stamp":"2024-04-25T10:29:06.989570+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T12:16:46.977487+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.4","version":"0'0","reported_seq":12,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-04-25T10:29:10.189423+0000","last_change":"2024-04-25T10:29:09.037824+0000","last_active":"2024-04-25T10:29:10.189423+0000","last_peered":"2024-04-25T10:29:10.189423+0000","last_clean":"2024-04-25T10:29:10.189423+0000","last_became_active":"2024-04-25T10:29:09.037478+0000","last_became_peered":"2024-04-25T10:29:09.037478+0000","last_unstale":"2024-04-25T10:29:10.189423+0000","last_undegraded":"2024-04-25T10:29:10.189423+0000","last_fullsized":"2024-04-25T10:29:10.189423+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:29:07.995096+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:29:07.995096+0000","last_clean_scrub_stamp":"2024-04-25T10:29:07.995096+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T15:02:17.480054+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.6","version":"23'1","reported_seq":15,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-04-25T10:29:10.026003+0000","last_change":"2024-04-25T10:29:08.020327+0000","last_active":"2024-04-25T10:29:10.026003+0000","last_peered":"2024-04-25T10:29:10.026003+0000","last_clean":"2024-04-25T10:29:10.026003+0000","last_became_active":"2024-04-25T10:29:08.019988+0000","last_became_peered":"2024-04-25T10:29:08.019988+0000","last_unstale":"2024-04-25T10:29:10.026003+0000","last_undegraded":"2024-04-25T10:29:10.026003+0000","last_fullsized":"2024-04-25T10:29:10.026003+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:29:06.989570+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:29:06.989570+0000","last_clean_scrub_stamp":"2024-04-25T10:29:06.989570+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T13:17:35.774897+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.7","version":"0'0","reported_seq":12,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-04-25T10:29:10.025861+0000","last_change":"2024-04-25T10:29:09.029850+0000","last_active":"2024-04-25T10:29:10.025861+0000","last_peered":"2024-04-25T10:29:10.025861+0000","last_clean":"2024-04-25T10:29:10.025861+0000","last_became_active":"2024-04-25T10:29:09.029534+0000","last_became_peered":"2024-04-25T10:29:09.029534+0000","last_unstale":"2024-04-25T10:29:10.025861+0000","last_undegraded":"2024-04-25T10:29:10.025861+0000","last_fullsized":"2024-04-25T10:29:10.025861+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:29:07.995096+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:29:07.995096+0000","last_clean_scrub_stamp":"2024-04-25T10:29:07.995096+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T15:03:07.958155+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.7","version":"22'2","reported_seq":16,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-04-25T10:29:10.024876+0000","last_change":"2024-04-25T10:29:08.026080+0000","last_active":"2024-04-25T10:29:10.024876+0000","last_peered":"2024-04-25T10:29:10.024876+0000","last_clean":"2024-04-25T10:29:10.024876+0000","last_became_active":"2024-04-25T10:29:08.025707+0000","last_became_peered":"2024-04-25T10:29:08.025707+0000","last_unstale":"2024-04-25T10:29:10.024876+0000","last_undegraded":"2024-04-25T10:29:10.024876+0000","last_fullsized":"2024-04-25T10:29:10.024876+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:29:06.989570+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:29:06.989570+0000","last_clean_scrub_stamp":"2024-04-25T10:29:06.989570+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T10:55:00.499631+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.6","version":"0'0","reported_seq":12,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-04-25T10:29:10.025829+0000","last_change":"2024-04-25T10:29:09.025814+0000","last_active":"2024-04-25T10:29:10.025829+0000","last_peered":"2024-04-25T10:29:10.025829+0000","last_clean":"2024-04-25T10:29:10.025829+0000","last_became_active":"2024-04-25T10:29:09.025474+0000","last_became_peered":"2024-04-25T10:29:09.025474+0000","last_unstale":"2024-04-25T10:29:10.025829+0000","last_undegraded":"2024-04-25T10:29:10.025829+0000","last_fullsized":"2024-04-25T10:29:10.025829+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:29:07.995096+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:29:07.995096+0000","last_clean_scrub_stamp":"2024-04-25T10:29:07.995096+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T12:35:12.945537+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7],"acting":[4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.8","version":"0'0","reported_seq":14,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-04-25T10:29:10.037271+0000","last_change":"2024-04-25T10:29:08.024845+0000","last_active":"2024-04-25T10:29:10.037271+0000","last_peered":"2024-04-25T10:29:10.037271+0000","last_clean":"2024-04-25T10:29:10.037271+0000","last_became_active":"2024-04-25T10:29:08.024484+0000","last_became_peered":"2024-04-25T10:29:08.024484+0000","last_unstale":"2024-04-25T10:29:10.037271+0000","last_undegraded":"2024-04-25T10:29:10.037271+0000","last_fullsized":"2024-04-25T10:29:10.037271+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:29:06.989570+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:29:06.989570+0000","last_clean_scrub_stamp":"2024-04-25T10:29:06.989570+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T14:12:57.950482+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.9","version":"0'0","reported_seq":12,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-04-25T10:29:10.024346+0000","last_change":"2024-04-25T10:29:09.022502+0000","last_active":"2024-04-25T10:29:10.024346+0000","last_peered":"2024-04-25T10:29:10.024346+0000","last_clean":"2024-04-25T10:29:10.024346+0000","last_became_active":"2024-04-25T10:29:09.022184+0000","last_became_peered":"2024-04-25T10:29:09.022184+0000","last_unstale":"2024-04-25T10:29:10.024346+0000","last_undegraded":"2024-04-25T10:29:10.024346+0000","last_fullsized":"2024-04-25T10:29:10.024346+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:29:07.995096+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:29:07.995096+0000","last_clean_scrub_stamp":"2024-04-25T10:29:07.995096+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T19:56:27.663748+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.9","version":"0'0","reported_seq":14,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-04-25T10:29:10.025738+0000","last_change":"2024-04-25T10:29:08.024989+0000","last_active":"2024-04-25T10:29:10.025738+0000","last_peered":"2024-04-25T10:29:10.025738+0000","last_clean":"2024-04-25T10:29:10.025738+0000","last_became_active":"2024-04-25T10:29:08.024655+0000","last_became_peered":"2024-04-25T10:29:08.024655+0000","last_unstale":"2024-04-25T10:29:10.025738+0000","last_undegraded":"2024-04-25T10:29:10.025738+0000","last_fullsized":"2024-04-25T10:29:10.025738+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:29:06.989570+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:29:06.989570+0000","last_clean_scrub_stamp":"2024-04-25T10:29:06.989570+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T21:05:51.419862+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.8","version":"0'0","reported_seq":12,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-04-25T10:29:10.025393+0000","last_change":"2024-04-25T10:29:09.023330+0000","last_active":"2024-04-25T10:29:10.025393+0000","last_peered":"2024-04-25T10:29:10.025393+0000","last_clean":"2024-04-25T10:29:10.025393+0000","last_became_active":"2024-04-25T10:29:09.020543+0000","last_became_peered":"2024-04-25T10:29:09.020543+0000","last_unstale":"2024-04-25T10:29:10.025393+0000","last_undegraded":"2024-04-25T10:29:10.025393+0000","last_fullsized":"2024-04-25T10:29:10.025393+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:29:07.995096+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:29:07.995096+0000","last_clean_scrub_stamp":"2024-04-25T10:29:07.995096+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T11:12:41.953761+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7],"acting":[4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.a","version":"0'0","reported_seq":14,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-04-25T10:29:10.188957+0000","last_change":"2024-04-25T10:29:08.020363+0000","last_active":"2024-04-25T10:29:10.188957+0000","last_peered":"2024-04-25T10:29:10.188957+0000","last_clean":"2024-04-25T10:29:10.188957+0000","last_became_active":"2024-04-25T10:29:08.020037+0000","last_became_peered":"2024-04-25T10:29:08.020037+0000","last_unstale":"2024-04-25T10:29:10.188957+0000","last_undegraded":"2024-04-25T10:29:10.188957+0000","last_fullsized":"2024-04-25T10:29:10.188957+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:29:06.989570+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:29:06.989570+0000","last_clean_scrub_stamp":"2024-04-25T10:29:06.989570+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T10:32:06.103123+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.b","version":"0'0","reported_seq":12,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-04-25T10:29:10.189294+0000","last_change":"2024-04-25T10:29:09.028820+0000","last_active":"2024-04-25T10:29:10.189294+0000","last_peered":"2024-04-25T10:29:10.189294+0000","last_clean":"2024-04-25T10:29:10.189294+0000","last_became_active":"2024-04-25T10:29:09.028426+0000","last_became_peered":"2024-04-25T10:29:09.028426+0000","last_unstale":"2024-04-25T10:29:10.189294+0000","last_undegraded":"2024-04-25T10:29:10.189294+0000","last_fullsized":"2024-04-25T10:29:10.189294+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:29:07.995096+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:29:07.995096+0000","last_clean_scrub_stamp":"2024-04-25T10:29:07.995096+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T19:54:17.703318+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.b","version":"22'2","reported_seq":16,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-04-25T10:29:10.024445+0000","last_change":"2024-04-25T10:29:08.025011+0000","last_active":"2024-04-25T10:29:10.024445+0000","last_peered":"2024-04-25T10:29:10.024445+0000","last_clean":"2024-04-25T10:29:10.024445+0000","last_became_active":"2024-04-25T10:29:08.024667+0000","last_became_peered":"2024-04-25T10:29:08.024667+0000","last_unstale":"2024-04-25T10:29:10.024445+0000","last_undegraded":"2024-04-25T10:29:10.024445+0000","last_fullsized":"2024-04-25T10:29:10.024445+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:29:06.989570+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:29:06.989570+0000","last_clean_scrub_stamp":"2024-04-25T10:29:06.989570+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T15:37:05.717854+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.a","version":"0'0","reported_seq":12,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-04-25T10:29:10.024394+0000","last_change":"2024-04-25T10:29:09.017160+0000","last_active":"2024-04-25T10:29:10.024394+0000","last_peered":"2024-04-25T10:29:10.024394+0000","last_clean":"2024-04-25T10:29:10.024394+0000","last_became_active":"2024-04-25T10:29:09.016821+0000","last_became_peered":"2024-04-25T10:29:09.016821+0000","last_unstale":"2024-04-25T10:29:10.024394+0000","last_undegraded":"2024-04-25T10:29:10.024394+0000","last_fullsized":"2024-04-25T10:29:10.024394+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:29:07.995096+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:29:07.995096+0000","last_clean_scrub_stamp":"2024-04-25T10:29:07.995096+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T20:40:19.840339+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.1d","version":"0'0","reported_seq":12,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-04-25T10:29:10.024622+0000","last_change":"2024-04-25T10:29:09.030341+0000","last_active":"2024-04-25T10:29:10.024622+0000","last_peered":"2024-04-25T10:29:10.024622+0000","last_clean":"2024-04-25T10:29:10.024622+0000","last_became_active":"2024-04-25T10:29:09.029988+0000","last_became_peered":"2024-04-25T10:29:09.029988+0000","last_unstale":"2024-04-25T10:29:10.024622+0000","last_undegraded":"2024-04-25T10:29:10.024622+0000","last_fullsized":"2024-04-25T10:29:10.024622+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:29:07.995096+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:29:07.995096+0000","last_clean_scrub_stamp":"2024-04-25T10:29:07.995096+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T11:40:18.624659+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.1c","version":"22'2","reported_seq":16,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-04-25T10:29:10.037969+0000","last_change":"2024-04-25T10:29:08.027197+0000","last_active":"2024-04-25T10:29:10.037969+0000","last_peered":"2024-04-25T10:29:10.037969+0000","last_clean":"2024-04-25T10:29:10.037969+0000","last_became_active":"2024-04-25T10:29:08.026872+0000","last_became_peered":"2024-04-25T10:29:08.026872+0000","last_unstale":"2024-04-25T10:29:10.037969+0000","last_undegraded":"2024-04-25T10:29:10.037969+0000","last_fullsized":"2024-04-25T10:29:10.037969+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:29:06.989570+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:29:06.989570+0000","last_clean_scrub_stamp":"2024-04-25T10:29:06.989570+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T15:58:35.843305+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.1c","version":"0'0","reported_seq":12,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-04-25T10:29:10.187853+0000","last_change":"2024-04-25T10:29:09.032731+0000","last_active":"2024-04-25T10:29:10.187853+0000","last_peered":"2024-04-25T10:29:10.187853+0000","last_clean":"2024-04-25T10:29:10.187853+0000","last_became_active":"2024-04-25T10:29:09.032383+0000","last_became_peered":"2024-04-25T10:29:09.032383+0000","last_unstale":"2024-04-25T10:29:10.187853+0000","last_undegraded":"2024-04-25T10:29:10.187853+0000","last_fullsized":"2024-04-25T10:29:10.187853+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:29:07.995096+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:29:07.995096+0000","last_clean_scrub_stamp":"2024-04-25T10:29:07.995096+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T19:54:17.703318+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.1d","version":"22'2","reported_seq":16,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-04-25T10:29:10.037807+0000","last_change":"2024-04-25T10:29:08.026410+0000","last_active":"2024-04-25T10:29:10.037807+0000","last_peered":"2024-04-25T10:29:10.037807+0000","last_clean":"2024-04-25T10:29:10.037807+0000","last_became_active":"2024-04-25T10:29:08.026071+0000","last_became_peered":"2024-04-25T10:29:08.026071+0000","last_unstale":"2024-04-25T10:29:10.037807+0000","last_undegraded":"2024-04-25T10:29:10.037807+0000","last_fullsized":"2024-04-25T10:29:10.037807+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:29:06.989570+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:29:06.989570+0000","last_clean_scrub_stamp":"2024-04-25T10:29:06.989570+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T19:54:16.697792+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.1f","version":"0'0","reported_seq":12,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-04-25T10:29:10.189240+0000","last_change":"2024-04-25T10:29:09.032735+0000","last_active":"2024-04-25T10:29:10.189240+0000","last_peered":"2024-04-25T10:29:10.189240+0000","last_clean":"2024-04-25T10:29:10.189240+0000","last_became_active":"2024-04-25T10:29:09.032227+0000","last_became_peered":"2024-04-25T10:29:09.032227+0000","last_unstale":"2024-04-25T10:29:10.189240+0000","last_undegraded":"2024-04-25T10:29:10.189240+0000","last_fullsized":"2024-04-25T10:29:10.189240+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:29:07.995096+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:29:07.995096+0000","last_clean_scrub_stamp":"2024-04-25T10:29:07.995096+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T11:31:45.283247+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.1e","version":"22'1","reported_seq":15,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-04-25T10:29:10.189728+0000","last_change":"2024-04-25T10:29:08.017025+0000","last_active":"2024-04-25T10:29:10.189728+0000","last_peered":"2024-04-25T10:29:10.189728+0000","last_clean":"2024-04-25T10:29:10.189728+0000","last_became_active":"2024-04-25T10:29:08.016702+0000","last_became_peered":"2024-04-25T10:29:08.016702+0000","last_unstale":"2024-04-25T10:29:10.189728+0000","last_undegraded":"2024-04-25T10:29:10.189728+0000","last_fullsized":"2024-04-25T10:29:10.189728+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:29:06.989570+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:29:06.989570+0000","last_clean_scrub_stamp":"2024-04-25T10:29:06.989570+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T14:12:57.950482+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.1e","version":"0'0","reported_seq":12,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-04-25T10:29:10.189730+0000","last_change":"2024-04-25T10:29:09.036208+0000","last_active":"2024-04-25T10:29:10.189730+0000","last_peered":"2024-04-25T10:29:10.189730+0000","last_clean":"2024-04-25T10:29:10.189730+0000","last_became_active":"2024-04-25T10:29:09.034890+0000","last_became_peered":"2024-04-25T10:29:09.034890+0000","last_unstale":"2024-04-25T10:29:10.189730+0000","last_undegraded":"2024-04-25T10:29:10.189730+0000","last_fullsized":"2024-04-25T10:29:10.189730+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:29:07.995096+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:29:07.995096+0000","last_clean_scrub_stamp":"2024-04-25T10:29:07.995096+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T18:44:23.610374+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.1f","version":"22'4","reported_seq":18,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-04-25T10:29:10.026053+0000","last_change":"2024-04-25T10:29:08.026202+0000","last_active":"2024-04-25T10:29:10.026053+0000","last_peered":"2024-04-25T10:29:10.026053+0000","last_clean":"2024-04-25T10:29:10.026053+0000","last_became_active":"2024-04-25T10:29:08.025877+0000","last_became_peered":"2024-04-25T10:29:08.025877+0000","last_unstale":"2024-04-25T10:29:10.026053+0000","last_undegraded":"2024-04-25T10:29:10.026053+0000","last_fullsized":"2024-04-25T10:29:10.026053+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:29:06.989570+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:29:06.989570+0000","last_clean_scrub_stamp":"2024-04-25T10:29:06.989570+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T20:00:40.198709+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":582,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]}],"pool_stats":[{"poolid":3,"num_pg":64,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":128,"acting":128,"num_store_stats":8},{"poolid":2,"num_pg":32,"stat_sum":{"num_bytes":2410,"num_objects":22,"num_object_clones":0,"num_object_copies":44,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":22,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":45,"num_write_kb":14,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":14,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":65536,"data_stored":4820,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":39,"ondisk_log_size":39,"up":64,"acting":64,"num_store_stats":8},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":66,"num_read_kb":55,"num_write":101,"num_write_kb":1628,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1187840,"data_stored":1180736,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":72,"ondisk_log_size":72,"up":2,"acting":2,"num_store_stats":2}],"osd_stats":[{"osd":7,"up_from":15,"seq":64424509448,"num_pgs":16,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27260,"kb_used_data":236,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344580,"statfs":{"total":96636764160,"available":96608849920,"internally_reserved":0,"allocated":241664,"data_stored":81359,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1590,"internal_metadata":27654602},"hb_peers":[0,1,2,3,4,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":6,"up_from":15,"seq":64424509449,"num_pgs":20,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27252,"kb_used_data":228,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344588,"statfs":{"total":96636764160,"available":96608858112,"internally_reserved":0,"allocated":233472,"data_stored":81235,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":27654598},"hb_peers":[0,1,2,3,4,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":5,"up_from":15,"seq":64424509450,"num_pgs":28,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27840,"kb_used_data":816,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344000,"statfs":{"total":96636764160,"available":96608256000,"internally_reserved":0,"allocated":835584,"data_stored":671695,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":3,"up_from":15,"seq":64424509450,"num_pgs":30,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27256,"kb_used_data":232,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344584,"statfs":{"total":96636764160,"available":96608854016,"internally_reserved":0,"allocated":237568,"data_stored":81817,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1593,"internal_metadata":27654599},"hb_peers":[0,1,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":2,"up_from":15,"seq":64424509450,"num_pgs":22,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27264,"kb_used_data":240,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344576,"statfs":{"total":96636764160,"available":96608845824,"internally_reserved":0,"allocated":245760,"data_stored":82847,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":1,"up_from":15,"seq":64424509448,"num_pgs":21,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27836,"kb_used_data":812,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344004,"statfs":{"total":96636764160,"available":96608260096,"internally_reserved":0,"allocated":831488,"data_stored":671673,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":0,"up_from":15,"seq":64424509449,"num_pgs":35,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27260,"kb_used_data":236,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344580,"statfs":{"total":96636764160,"available":96608849920,"internally_reserved":0,"allocated":241664,"data_stored":81291,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[1,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":4,"up_from":13,"seq":55834574856,"num_pgs":22,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27276,"kb_used_data":248,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344564,"statfs":{"total":96636764160,"available":96608833536,"internally_reserved":0,"allocated":253952,"data_stored":83519,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":56,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":70,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":1612,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":582,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":20480,"data_stored":2284,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":92,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":124,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-04-25T10:29:22.362 DEBUG:teuthology.orchestra.run.smithi037:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-04-25T10:29:22.507 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:22.505+0000 7f0c15e92640 1 Processor -- start 2024-04-25T10:29:22.508 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:22.505+0000 7f0c15e92640 1 -- start start 2024-04-25T10:29:22.509 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:22.507+0000 7f0c15e92640 1 --2- >> v2:172.21.15.37:3300/0 conn(0x7f0c100d0980 0x7f0c100d0d90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:29:22.510 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:22.507+0000 7f0c15e92640 1 --2- >> v2:172.21.15.203:3300/0 conn(0x7f0c100b75c0 0x7f0c100b79d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:29:22.510 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:22.507+0000 7f0c15e92640 1 -- --> v1:172.21.15.37:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f0c100b7f10 con 0x7f0c100d5110 2024-04-25T10:29:22.510 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:22.507+0000 7f0c15e92640 1 -- --> v2:172.21.15.37:3300/0 -- mon_getmap magic: 0 -- 0x7f0c100d5520 con 0x7f0c100d0980 2024-04-25T10:29:22.510 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:22.507+0000 7f0c15e92640 1 -- --> v2:172.21.15.203:3300/0 -- mon_getmap magic: 0 -- 0x7f0c100d1370 con 0x7f0c100b75c0 2024-04-25T10:29:22.510 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:22.507+0000 7f0c0effd640 1 --2- >> v2:172.21.15.37:3300/0 conn(0x7f0c100d0980 0x7f0c100d0d90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:29:22.510 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:22.507+0000 7f0c0effd640 1 --2- >> v2:172.21.15.37:3300/0 conn(0x7f0c100d0980 0x7f0c100d0d90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.37:3300/0 says I am v2:172.21.15.37:35110/0 (socket says 172.21.15.37:35110) 2024-04-25T10:29:22.510 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:22.507+0000 7f0c0effd640 1 -- 172.21.15.37:0/3262855141 learned_addr learned my addr 172.21.15.37:0/3262855141 (peer_addr_for_me v2:172.21.15.37:0/0) 2024-04-25T10:29:22.510 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:22.507+0000 7f0c0f7fe640 1 --1- 172.21.15.37:0/3262855141 >> v1:172.21.15.37:6789/0 conn(0x7f0c100d5110 0x7f0c100b4d20 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.37:6789/0 says I am v1:172.21.15.37:59812/0 (socket says 172.21.15.37:59812) 2024-04-25T10:29:22.510 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:22.507+0000 7f0c0effd640 1 -- 172.21.15.37:0/3262855141 >> v2:172.21.15.203:3300/0 conn(0x7f0c100b75c0 msgr2=0x7f0c100b79d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:29:22.510 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:22.507+0000 7f0c0ffff640 1 --2- 172.21.15.37:0/3262855141 >> v2:172.21.15.203:3300/0 conn(0x7f0c100b75c0 0x7f0c100b79d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:29:22.510 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:22.507+0000 7f0c0e7fc640 1 -- 172.21.15.37:0/3262855141 <== mon.0 v1:172.21.15.37:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 3343284042 0 0) 0x7f0c00001110 con 0x7f0c100d5110 2024-04-25T10:29:22.510 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:22.507+0000 7f0c0effd640 1 --2- 172.21.15.37:0/3262855141 >> v2:172.21.15.203:3300/0 conn(0x7f0c100b75c0 0x7f0c100b79d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:29:22.510 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:22.507+0000 7f0c0effd640 1 -- 172.21.15.37:0/3262855141 >> v1:172.21.15.37:6789/0 conn(0x7f0c100d5110 legacy=0x7f0c100b4d20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:29:22.510 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:22.507+0000 7f0c0effd640 1 -- 172.21.15.37:0/3262855141 --> v2:172.21.15.37:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f0c100d4630 con 0x7f0c100d0980 2024-04-25T10:29:22.511 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:22.507+0000 7f0c0ffff640 1 --2- 172.21.15.37:0/3262855141 >> v2:172.21.15.203:3300/0 conn(0x7f0c100b75c0 0x7f0c100b79d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T10:29:22.512 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:22.508+0000 7f0c0effd640 1 --2- 172.21.15.37:0/3262855141 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f0c100d0980 0x7f0c100d0d90 secure :-1 s=READY pgs=136 cs=0 l=1 rev1=1 crypto rx=0x7f0c10160810 tx=0x7f0c0402a270 comp rx=0 tx=0).ready entity=mon.0 client_cookie=5d39be5dff1c8765 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:29:22.512 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:22.508+0000 7f0c0e7fc640 1 -- 172.21.15.37:0/3262855141 <== mon.0 v2:172.21.15.37:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0c04037070 con 0x7f0c100d0980 2024-04-25T10:29:22.512 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:22.508+0000 7f0c0e7fc640 1 -- 172.21.15.37:0/3262855141 <== mon.0 v2:172.21.15.37:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f0c04032440 con 0x7f0c100d0980 2024-04-25T10:29:22.512 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:22.508+0000 7f0c0e7fc640 1 -- 172.21.15.37:0/3262855141 <== mon.0 v2:172.21.15.37:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0c04032810 con 0x7f0c100d0980 2024-04-25T10:29:22.512 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:22.509+0000 7f0c15e92640 1 -- 172.21.15.37:0/3262855141 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f0c100d0980 msgr2=0x7f0c100d0d90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:29:22.512 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:22.509+0000 7f0c15e92640 1 --2- 172.21.15.37:0/3262855141 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f0c100d0980 0x7f0c100d0d90 secure :-1 s=READY pgs=136 cs=0 l=1 rev1=1 crypto rx=0x7f0c10160810 tx=0x7f0c0402a270 comp rx=0 tx=0).stop 2024-04-25T10:29:22.512 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:22.509+0000 7f0c15e92640 1 -- 172.21.15.37:0/3262855141 shutdown_connections 2024-04-25T10:29:22.512 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:22.509+0000 7f0c15e92640 1 --2- 172.21.15.37:0/3262855141 >> v2:172.21.15.203:3300/0 conn(0x7f0c100b75c0 0x7f0c100b79d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:29:22.512 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:22.509+0000 7f0c15e92640 1 --2- 172.21.15.37:0/3262855141 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f0c100d0980 0x7f0c100d0d90 unknown :-1 s=CLOSED pgs=136 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:29:22.512 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:22.509+0000 7f0c15e92640 1 -- 172.21.15.37:0/3262855141 >> 172.21.15.37:0/3262855141 conn(0x7f0c100d27a0 msgr2=0x7f0c100d2bb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:29:22.512 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:22.509+0000 7f0c15e92640 1 -- 172.21.15.37:0/3262855141 shutdown_connections 2024-04-25T10:29:22.513 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:22.509+0000 7f0c15e92640 1 -- 172.21.15.37:0/3262855141 wait complete. 2024-04-25T10:29:22.513 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:22.510+0000 7f0c15e92640 1 Processor -- start 2024-04-25T10:29:22.513 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:22.511+0000 7f0c15e92640 1 -- start start 2024-04-25T10:29:22.515 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:22.512+0000 7f0c15e92640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f0c100b75c0 0x7f0c1015c2d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:29:22.516 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:22.512+0000 7f0c15e92640 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f0c100d0980 0x7f0c101570f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:29:22.516 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:22.512+0000 7f0c15e92640 1 --2- >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f0c100d5110 0x7f0c101ebd70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:29:22.516 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:22.512+0000 7f0c15e92640 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 -- 0x7f0c100d1650 con 0x7f0c100b75c0 2024-04-25T10:29:22.516 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:22.512+0000 7f0c15e92640 1 -- --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_getmap magic: 0 -- 0x7f0c100d4c30 con 0x7f0c100d0980 2024-04-25T10:29:22.516 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:22.512+0000 7f0c15e92640 1 -- --> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] -- mon_getmap magic: 0 -- 0x7f0c10164ae0 con 0x7f0c100d5110 2024-04-25T10:29:22.516 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:22.512+0000 7f0c0f7fe640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f0c100b75c0 0x7f0c1015c2d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:29:22.516 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:22.512+0000 7f0c0f7fe640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f0c100b75c0 0x7f0c1015c2d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.37:3300/0 says I am v2:172.21.15.37:35124/0 (socket says 172.21.15.37:35124) 2024-04-25T10:29:22.516 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:22.512+0000 7f0c0f7fe640 1 -- 172.21.15.37:0/1721827300 learned_addr learned my addr 172.21.15.37:0/1721827300 (peer_addr_for_me v2:172.21.15.37:0/0) 2024-04-25T10:29:22.516 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:22.512+0000 7f0c0effd640 1 --2- 172.21.15.37:0/1721827300 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f0c100d0980 0x7f0c101570f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:29:22.516 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:22.512+0000 7f0c0ffff640 1 --2- 172.21.15.37:0/1721827300 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f0c100d5110 0x7f0c101ebd70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:29:22.516 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:22.512+0000 7f0c0f7fe640 1 -- 172.21.15.37:0/1721827300 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f0c100d5110 msgr2=0x7f0c101ebd70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:29:22.516 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:22.512+0000 7f0c0f7fe640 1 --2- 172.21.15.37:0/1721827300 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f0c100d5110 0x7f0c101ebd70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:29:22.516 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:22.512+0000 7f0c0f7fe640 1 -- 172.21.15.37:0/1721827300 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f0c100d0980 msgr2=0x7f0c101570f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:29:22.516 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:22.512+0000 7f0c0f7fe640 1 --2- 172.21.15.37:0/1721827300 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f0c100d0980 0x7f0c101570f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:29:22.517 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:22.512+0000 7f0c0f7fe640 1 -- 172.21.15.37:0/1721827300 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f0c10159290 con 0x7f0c100b75c0 2024-04-25T10:29:22.517 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:22.512+0000 7f0c0effd640 1 --2- 172.21.15.37:0/1721827300 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f0c100d0980 0x7f0c101570f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T10:29:22.517 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:22.513+0000 7f0c0f7fe640 1 --2- 172.21.15.37:0/1721827300 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f0c100b75c0 0x7f0c1015c2d0 secure :-1 s=READY pgs=137 cs=0 l=1 rev1=1 crypto rx=0x7f0c0000c430 tx=0x7f0c0000c8f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:29:22.517 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:22.513+0000 7f0bf7fff640 1 -- 172.21.15.37:0/1721827300 <== mon.0 v2:172.21.15.37:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0c000074e0 con 0x7f0c100b75c0 2024-04-25T10:29:22.517 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:22.513+0000 7f0bf7fff640 1 -- 172.21.15.37:0/1721827300 <== mon.0 v2:172.21.15.37:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f0c000047e0 con 0x7f0c100b75c0 2024-04-25T10:29:22.517 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:22.513+0000 7f0c15e92640 1 -- 172.21.15.37:0/1721827300 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f0c101584e0 con 0x7f0c100b75c0 2024-04-25T10:29:22.517 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:22.513+0000 7f0bf7fff640 1 -- 172.21.15.37:0/1721827300 <== mon.0 v2:172.21.15.37:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0c00004bd0 con 0x7f0c100b75c0 2024-04-25T10:29:22.517 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:22.513+0000 7f0c15e92640 1 -- 172.21.15.37:0/1721827300 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f0c1015e0c0 con 0x7f0c100b75c0 2024-04-25T10:29:22.517 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:22.514+0000 7f0c15e92640 1 -- 172.21.15.37:0/1721827300 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f0c1006a140 con 0x7f0c100b75c0 2024-04-25T10:29:22.518 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:22.514+0000 7f0bf7fff640 1 -- 172.21.15.37:0/1721827300 <== mon.0 v2:172.21.15.37:3300/0 4 ==== mgrmap(e 5) ==== 96782+0+0 (secure 0 0 0) 0x7f0c00011070 con 0x7f0c100b75c0 2024-04-25T10:29:22.518 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:22.515+0000 7f0bf7fff640 1 --2- 172.21.15.37:0/1721827300 >> [v2:172.21.15.203:6832/2418205237,v1:172.21.15.203:6833/2418205237] conn(0x7f0bfc0744b0 0x7f0bfc0769c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:29:22.518 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:22.515+0000 7f0bf7fff640 1 -- 172.21.15.37:0/1721827300 <== mon.0 v2:172.21.15.37:3300/0 5 ==== osd_map(23..23 src has 1..23) ==== 5709+0+0 (secure 0 0 0) 0x7f0c00094d80 con 0x7f0c100b75c0 2024-04-25T10:29:22.518 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:22.515+0000 7f0c0effd640 1 --2- 172.21.15.37:0/1721827300 >> [v2:172.21.15.203:6832/2418205237,v1:172.21.15.203:6833/2418205237] conn(0x7f0bfc0744b0 0x7f0bfc0769c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:29:22.518 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:22.516+0000 7f0c0effd640 1 --2- 172.21.15.37:0/1721827300 >> [v2:172.21.15.203:6832/2418205237,v1:172.21.15.203:6833/2418205237] conn(0x7f0bfc0744b0 0x7f0bfc0769c0 secure :-1 s=READY pgs=98 cs=0 l=1 rev1=1 crypto rx=0x7f0c04034000 tx=0x7f0c0402e000 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:29:22.521 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:22.519+0000 7f0bf7fff640 1 -- 172.21.15.37:0/1721827300 <== mon.0 v2:172.21.15.37:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187246 (secure 0 0 0) 0x7f0c100b7f10 con 0x7f0c100b75c0 2024-04-25T10:29:22.704 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:22.702+0000 7f0c15e92640 1 -- 172.21.15.37:0/1721827300 --> [v2:172.21.15.203:6832/2418205237,v1:172.21.15.203:6833/2418205237] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) -- 0x7f0c1015e360 con 0x7f0bfc0744b0 2024-04-25T10:29:22.710 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:22.709+0000 7f0bf7fff640 1 -- 172.21.15.37:0/1721827300 <== mgr.4100 v2:172.21.15.203:6832/2418205237 1 ==== mgr_command_reply(tid 0: 0 dumped all) ==== 18+0+251822 (secure 0 0 0) 0x7f0c1015e360 con 0x7f0bfc0744b0 2024-04-25T10:29:22.713 INFO:teuthology.orchestra.run.smithi037.stdout: 2024-04-25T10:29:22.715 INFO:teuthology.orchestra.run.smithi037.stderr:dumped all 2024-04-25T10:29:22.716 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:22.711+0000 7f0c15e92640 1 -- 172.21.15.37:0/1721827300 >> [v2:172.21.15.203:6832/2418205237,v1:172.21.15.203:6833/2418205237] conn(0x7f0bfc0744b0 msgr2=0x7f0bfc0769c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:29:22.716 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:22.711+0000 7f0c15e92640 1 --2- 172.21.15.37:0/1721827300 >> [v2:172.21.15.203:6832/2418205237,v1:172.21.15.203:6833/2418205237] conn(0x7f0bfc0744b0 0x7f0bfc0769c0 secure :-1 s=READY pgs=98 cs=0 l=1 rev1=1 crypto rx=0x7f0c04034000 tx=0x7f0c0402e000 comp rx=0 tx=0).stop 2024-04-25T10:29:22.716 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:22.711+0000 7f0c15e92640 1 -- 172.21.15.37:0/1721827300 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f0c100b75c0 msgr2=0x7f0c1015c2d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:29:22.716 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:22.711+0000 7f0c15e92640 1 --2- 172.21.15.37:0/1721827300 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f0c100b75c0 0x7f0c1015c2d0 secure :-1 s=READY pgs=137 cs=0 l=1 rev1=1 crypto rx=0x7f0c0000c430 tx=0x7f0c0000c8f0 comp rx=0 tx=0).stop 2024-04-25T10:29:22.716 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:22.712+0000 7f0c15e92640 1 -- 172.21.15.37:0/1721827300 shutdown_connections 2024-04-25T10:29:22.716 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:22.712+0000 7f0c15e92640 1 --2- 172.21.15.37:0/1721827300 >> [v2:172.21.15.203:6832/2418205237,v1:172.21.15.203:6833/2418205237] conn(0x7f0bfc0744b0 0x7f0bfc0769c0 unknown :-1 s=CLOSED pgs=98 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:29:22.716 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:22.712+0000 7f0c15e92640 1 --2- 172.21.15.37:0/1721827300 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f0c100d5110 0x7f0c101ebd70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:29:22.717 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:22.712+0000 7f0c15e92640 1 --2- 172.21.15.37:0/1721827300 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f0c100d0980 0x7f0c101570f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:29:22.717 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:22.712+0000 7f0c15e92640 1 --2- 172.21.15.37:0/1721827300 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f0c100b75c0 0x7f0c1015c2d0 unknown :-1 s=CLOSED pgs=137 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:29:22.717 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:22.712+0000 7f0c15e92640 1 -- 172.21.15.37:0/1721827300 >> 172.21.15.37:0/1721827300 conn(0x7f0c100d27a0 msgr2=0x7f0c100d2bb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:29:22.717 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:22.712+0000 7f0c15e92640 1 -- 172.21.15.37:0/1721827300 shutdown_connections 2024-04-25T10:29:22.717 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:22.712+0000 7f0c15e92640 1 -- 172.21.15.37:0/1721827300 wait complete. 2024-04-25T10:29:22.725 INFO:teuthology.orchestra.run.smithi037.stdout:{"pg_ready":true,"pg_map":{"version":31,"stamp":"2024-04-25T10:29:22.190081+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":592778,"num_objects":24,"num_object_clones":0,"num_object_copies":48,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":24,"num_whiteouts":0,"num_read":66,"num_read_kb":55,"num_write":146,"num_write_kb":1642,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":14,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":111,"ondisk_log_size":111,"up":194,"acting":194,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":194,"num_osds":8,"num_per_pool_osds":8,"num_per_pool_omap_osds":8,"kb":754974720,"kb_used":219244,"kb_used_data":3048,"kb_used_omap":12,"kb_used_meta":216051,"kb_avail":754755476,"statfs":{"total":773094113280,"available":772869607424,"internally_reserved":0,"allocated":3121152,"data_stored":1835436,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":12752,"internal_metadata":221236784},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":2410,"num_objects":22,"num_object_clones":0,"num_object_copies":44,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":22,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":45,"num_write_kb":14,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":14,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"12.004319"},"pg_stats":[{"pgid":"3.3f","version":"0'0","reported_seq":12,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-04-25T10:29:10.037967+0000","last_change":"2024-04-25T10:29:09.024146+0000","last_active":"2024-04-25T10:29:10.037967+0000","last_peered":"2024-04-25T10:29:10.037967+0000","last_clean":"2024-04-25T10:29:10.037967+0000","last_became_active":"2024-04-25T10:29:09.023798+0000","last_became_peered":"2024-04-25T10:29:09.023798+0000","last_unstale":"2024-04-25T10:29:10.037967+0000","last_undegraded":"2024-04-25T10:29:10.037967+0000","last_fullsized":"2024-04-25T10:29:10.037967+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:29:07.995096+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:29:07.995096+0000","last_clean_scrub_stamp":"2024-04-25T10:29:07.995096+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T20:37:20.726766+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.3e","version":"0'0","reported_seq":12,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-04-25T10:29:10.189579+0000","last_change":"2024-04-25T10:29:09.028600+0000","last_active":"2024-04-25T10:29:10.189579+0000","last_peered":"2024-04-25T10:29:10.189579+0000","last_clean":"2024-04-25T10:29:10.189579+0000","last_became_active":"2024-04-25T10:29:09.028286+0000","last_became_peered":"2024-04-25T10:29:09.028286+0000","last_unstale":"2024-04-25T10:29:10.189579+0000","last_undegraded":"2024-04-25T10:29:10.189579+0000","last_fullsized":"2024-04-25T10:29:10.189579+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:29:07.995096+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:29:07.995096+0000","last_clean_scrub_stamp":"2024-04-25T10:29:07.995096+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T20:52:02.098872+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6],"acting":[5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.3d","version":"0'0","reported_seq":12,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-04-25T10:29:10.189318+0000","last_change":"2024-04-25T10:29:09.039030+0000","last_active":"2024-04-25T10:29:10.189318+0000","last_peered":"2024-04-25T10:29:10.189318+0000","last_clean":"2024-04-25T10:29:10.189318+0000","last_became_active":"2024-04-25T10:29:09.037692+0000","last_became_peered":"2024-04-25T10:29:09.037692+0000","last_unstale":"2024-04-25T10:29:10.189318+0000","last_undegraded":"2024-04-25T10:29:10.189318+0000","last_fullsized":"2024-04-25T10:29:10.189318+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:29:07.995096+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:29:07.995096+0000","last_clean_scrub_stamp":"2024-04-25T10:29:07.995096+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T15:51:52.989545+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.3c","version":"0'0","reported_seq":12,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-04-25T10:29:10.026169+0000","last_change":"2024-04-25T10:29:09.027520+0000","last_active":"2024-04-25T10:29:10.026169+0000","last_peered":"2024-04-25T10:29:10.026169+0000","last_clean":"2024-04-25T10:29:10.026169+0000","last_became_active":"2024-04-25T10:29:09.027075+0000","last_became_peered":"2024-04-25T10:29:09.027075+0000","last_unstale":"2024-04-25T10:29:10.026169+0000","last_undegraded":"2024-04-25T10:29:10.026169+0000","last_fullsized":"2024-04-25T10:29:10.026169+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:29:07.995096+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:29:07.995096+0000","last_clean_scrub_stamp":"2024-04-25T10:29:07.995096+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T10:55:01.505157+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4],"acting":[2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.3b","version":"0'0","reported_seq":12,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-04-25T10:29:10.037590+0000","last_change":"2024-04-25T10:29:09.040684+0000","last_active":"2024-04-25T10:29:10.037590+0000","last_peered":"2024-04-25T10:29:10.037590+0000","last_clean":"2024-04-25T10:29:10.037590+0000","last_became_active":"2024-04-25T10:29:09.040372+0000","last_became_peered":"2024-04-25T10:29:09.040372+0000","last_unstale":"2024-04-25T10:29:10.037590+0000","last_undegraded":"2024-04-25T10:29:10.037590+0000","last_fullsized":"2024-04-25T10:29:10.037590+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:29:07.995096+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:29:07.995096+0000","last_clean_scrub_stamp":"2024-04-25T10:29:07.995096+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T11:52:24.415332+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.3a","version":"0'0","reported_seq":12,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-04-25T10:29:10.025911+0000","last_change":"2024-04-25T10:29:09.027391+0000","last_active":"2024-04-25T10:29:10.025911+0000","last_peered":"2024-04-25T10:29:10.025911+0000","last_clean":"2024-04-25T10:29:10.025911+0000","last_became_active":"2024-04-25T10:29:09.027029+0000","last_became_peered":"2024-04-25T10:29:09.027029+0000","last_unstale":"2024-04-25T10:29:10.025911+0000","last_undegraded":"2024-04-25T10:29:10.025911+0000","last_fullsized":"2024-04-25T10:29:10.025911+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:29:07.995096+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:29:07.995096+0000","last_clean_scrub_stamp":"2024-04-25T10:29:07.995096+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T10:35:31.361514+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4],"acting":[2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.39","version":"0'0","reported_seq":12,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-04-25T10:29:10.037462+0000","last_change":"2024-04-25T10:29:09.029272+0000","last_active":"2024-04-25T10:29:10.037462+0000","last_peered":"2024-04-25T10:29:10.037462+0000","last_clean":"2024-04-25T10:29:10.037462+0000","last_became_active":"2024-04-25T10:29:09.028302+0000","last_became_peered":"2024-04-25T10:29:09.028302+0000","last_unstale":"2024-04-25T10:29:10.037462+0000","last_undegraded":"2024-04-25T10:29:10.037462+0000","last_fullsized":"2024-04-25T10:29:10.037462+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:29:07.995096+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:29:07.995096+0000","last_clean_scrub_stamp":"2024-04-25T10:29:07.995096+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T11:31:45.283247+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.38","version":"0'0","reported_seq":12,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-04-25T10:29:10.188973+0000","last_change":"2024-04-25T10:29:09.035089+0000","last_active":"2024-04-25T10:29:10.188973+0000","last_peered":"2024-04-25T10:29:10.188973+0000","last_clean":"2024-04-25T10:29:10.188973+0000","last_became_active":"2024-04-25T10:29:09.034324+0000","last_became_peered":"2024-04-25T10:29:09.034324+0000","last_unstale":"2024-04-25T10:29:10.188973+0000","last_undegraded":"2024-04-25T10:29:10.188973+0000","last_fullsized":"2024-04-25T10:29:10.188973+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:29:07.995096+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:29:07.995096+0000","last_clean_scrub_stamp":"2024-04-25T10:29:07.995096+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T21:39:39.743441+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.37","version":"0'0","reported_seq":12,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-04-25T10:29:10.187981+0000","last_change":"2024-04-25T10:29:09.016966+0000","last_active":"2024-04-25T10:29:10.187981+0000","last_peered":"2024-04-25T10:29:10.187981+0000","last_clean":"2024-04-25T10:29:10.187981+0000","last_became_active":"2024-04-25T10:29:09.016566+0000","last_became_peered":"2024-04-25T10:29:09.016566+0000","last_unstale":"2024-04-25T10:29:10.187981+0000","last_undegraded":"2024-04-25T10:29:10.187981+0000","last_fullsized":"2024-04-25T10:29:10.187981+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:29:07.995096+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:29:07.995096+0000","last_clean_scrub_stamp":"2024-04-25T10:29:07.995096+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T10:55:01.505157+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.36","version":"0'0","reported_seq":12,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-04-25T10:29:10.037810+0000","last_change":"2024-04-25T10:29:09.030321+0000","last_active":"2024-04-25T10:29:10.037810+0000","last_peered":"2024-04-25T10:29:10.037810+0000","last_clean":"2024-04-25T10:29:10.037810+0000","last_became_active":"2024-04-25T10:29:09.029976+0000","last_became_peered":"2024-04-25T10:29:09.029976+0000","last_unstale":"2024-04-25T10:29:10.037810+0000","last_undegraded":"2024-04-25T10:29:10.037810+0000","last_fullsized":"2024-04-25T10:29:10.037810+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:29:07.995096+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:29:07.995096+0000","last_clean_scrub_stamp":"2024-04-25T10:29:07.995096+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T12:36:31.191100+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.35","version":"0'0","reported_seq":12,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-04-25T10:29:10.037641+0000","last_change":"2024-04-25T10:29:09.030469+0000","last_active":"2024-04-25T10:29:10.037641+0000","last_peered":"2024-04-25T10:29:10.037641+0000","last_clean":"2024-04-25T10:29:10.037641+0000","last_became_active":"2024-04-25T10:29:09.029350+0000","last_became_peered":"2024-04-25T10:29:09.029350+0000","last_unstale":"2024-04-25T10:29:10.037641+0000","last_undegraded":"2024-04-25T10:29:10.037641+0000","last_fullsized":"2024-04-25T10:29:10.037641+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:29:07.995096+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:29:07.995096+0000","last_clean_scrub_stamp":"2024-04-25T10:29:07.995096+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T15:51:52.989545+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.34","version":"0'0","reported_seq":12,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-04-25T10:29:10.037728+0000","last_change":"2024-04-25T10:29:09.029772+0000","last_active":"2024-04-25T10:29:10.037728+0000","last_peered":"2024-04-25T10:29:10.037728+0000","last_clean":"2024-04-25T10:29:10.037728+0000","last_became_active":"2024-04-25T10:29:09.029350+0000","last_became_peered":"2024-04-25T10:29:09.029350+0000","last_unstale":"2024-04-25T10:29:10.037728+0000","last_undegraded":"2024-04-25T10:29:10.037728+0000","last_fullsized":"2024-04-25T10:29:10.037728+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:29:07.995096+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:29:07.995096+0000","last_clean_scrub_stamp":"2024-04-25T10:29:07.995096+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T19:14:48.737269+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.33","version":"0'0","reported_seq":12,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-04-25T10:29:10.188971+0000","last_change":"2024-04-25T10:29:09.038690+0000","last_active":"2024-04-25T10:29:10.188971+0000","last_peered":"2024-04-25T10:29:10.188971+0000","last_clean":"2024-04-25T10:29:10.188971+0000","last_became_active":"2024-04-25T10:29:09.038336+0000","last_became_peered":"2024-04-25T10:29:09.038336+0000","last_unstale":"2024-04-25T10:29:10.188971+0000","last_undegraded":"2024-04-25T10:29:10.188971+0000","last_fullsized":"2024-04-25T10:29:10.188971+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:29:07.995096+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:29:07.995096+0000","last_clean_scrub_stamp":"2024-04-25T10:29:07.995096+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T19:23:12.589414+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.32","version":"0'0","reported_seq":12,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-04-25T10:29:10.188641+0000","last_change":"2024-04-25T10:29:09.028195+0000","last_active":"2024-04-25T10:29:10.188641+0000","last_peered":"2024-04-25T10:29:10.188641+0000","last_clean":"2024-04-25T10:29:10.188641+0000","last_became_active":"2024-04-25T10:29:09.027893+0000","last_became_peered":"2024-04-25T10:29:09.027893+0000","last_unstale":"2024-04-25T10:29:10.188641+0000","last_undegraded":"2024-04-25T10:29:10.188641+0000","last_fullsized":"2024-04-25T10:29:10.188641+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:29:07.995096+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:29:07.995096+0000","last_clean_scrub_stamp":"2024-04-25T10:29:07.995096+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T11:35:27.466874+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.31","version":"0'0","reported_seq":12,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-04-25T10:29:10.188956+0000","last_change":"2024-04-25T10:29:09.034532+0000","last_active":"2024-04-25T10:29:10.188956+0000","last_peered":"2024-04-25T10:29:10.188956+0000","last_clean":"2024-04-25T10:29:10.188956+0000","last_became_active":"2024-04-25T10:29:09.034021+0000","last_became_peered":"2024-04-25T10:29:09.034021+0000","last_unstale":"2024-04-25T10:29:10.188956+0000","last_undegraded":"2024-04-25T10:29:10.188956+0000","last_fullsized":"2024-04-25T10:29:10.188956+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:29:07.995096+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:29:07.995096+0000","last_clean_scrub_stamp":"2024-04-25T10:29:07.995096+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T12:16:04.880732+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.30","version":"0'0","reported_seq":12,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-04-25T10:29:10.024134+0000","last_change":"2024-04-25T10:29:09.021224+0000","last_active":"2024-04-25T10:29:10.024134+0000","last_peered":"2024-04-25T10:29:10.024134+0000","last_clean":"2024-04-25T10:29:10.024134+0000","last_became_active":"2024-04-25T10:29:09.020964+0000","last_became_peered":"2024-04-25T10:29:09.020964+0000","last_unstale":"2024-04-25T10:29:10.024134+0000","last_undegraded":"2024-04-25T10:29:10.024134+0000","last_fullsized":"2024-04-25T10:29:10.024134+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:29:07.995096+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:29:07.995096+0000","last_clean_scrub_stamp":"2024-04-25T10:29:07.995096+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T18:44:23.610374+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.2f","version":"0'0","reported_seq":12,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-04-25T10:29:10.024944+0000","last_change":"2024-04-25T10:29:09.025638+0000","last_active":"2024-04-25T10:29:10.024944+0000","last_peered":"2024-04-25T10:29:10.024944+0000","last_clean":"2024-04-25T10:29:10.024944+0000","last_became_active":"2024-04-25T10:29:09.024517+0000","last_became_peered":"2024-04-25T10:29:09.024517+0000","last_unstale":"2024-04-25T10:29:10.024944+0000","last_undegraded":"2024-04-25T10:29:10.024944+0000","last_fullsized":"2024-04-25T10:29:10.024944+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:29:07.995096+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:29:07.995096+0000","last_clean_scrub_stamp":"2024-04-25T10:29:07.995096+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T16:29:08.975346+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.2e","version":"0'0","reported_seq":12,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-04-25T10:29:10.189006+0000","last_change":"2024-04-25T10:29:09.023923+0000","last_active":"2024-04-25T10:29:10.189006+0000","last_peered":"2024-04-25T10:29:10.189006+0000","last_clean":"2024-04-25T10:29:10.189006+0000","last_became_active":"2024-04-25T10:29:09.023579+0000","last_became_peered":"2024-04-25T10:29:09.023579+0000","last_unstale":"2024-04-25T10:29:10.189006+0000","last_undegraded":"2024-04-25T10:29:10.189006+0000","last_fullsized":"2024-04-25T10:29:10.189006+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:29:07.995096+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:29:07.995096+0000","last_clean_scrub_stamp":"2024-04-25T10:29:07.995096+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T10:35:31.361514+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.2d","version":"0'0","reported_seq":12,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-04-25T10:29:10.024852+0000","last_change":"2024-04-25T10:29:09.023039+0000","last_active":"2024-04-25T10:29:10.024852+0000","last_peered":"2024-04-25T10:29:10.024852+0000","last_clean":"2024-04-25T10:29:10.024852+0000","last_became_active":"2024-04-25T10:29:09.022706+0000","last_became_peered":"2024-04-25T10:29:09.022706+0000","last_unstale":"2024-04-25T10:29:10.024852+0000","last_undegraded":"2024-04-25T10:29:10.024852+0000","last_fullsized":"2024-04-25T10:29:10.024852+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:29:07.995096+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:29:07.995096+0000","last_clean_scrub_stamp":"2024-04-25T10:29:07.995096+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T16:48:17.352944+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.2c","version":"0'0","reported_seq":12,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-04-25T10:29:10.037891+0000","last_change":"2024-04-25T10:29:09.028979+0000","last_active":"2024-04-25T10:29:10.037891+0000","last_peered":"2024-04-25T10:29:10.037891+0000","last_clean":"2024-04-25T10:29:10.037891+0000","last_became_active":"2024-04-25T10:29:09.028061+0000","last_became_peered":"2024-04-25T10:29:09.028061+0000","last_unstale":"2024-04-25T10:29:10.037891+0000","last_undegraded":"2024-04-25T10:29:10.037891+0000","last_fullsized":"2024-04-25T10:29:10.037891+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:29:07.995096+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:29:07.995096+0000","last_clean_scrub_stamp":"2024-04-25T10:29:07.995096+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T18:44:23.610374+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.2b","version":"0'0","reported_seq":12,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-04-25T10:29:10.025554+0000","last_change":"2024-04-25T10:29:09.022874+0000","last_active":"2024-04-25T10:29:10.025554+0000","last_peered":"2024-04-25T10:29:10.025554+0000","last_clean":"2024-04-25T10:29:10.025554+0000","last_became_active":"2024-04-25T10:29:09.022360+0000","last_became_peered":"2024-04-25T10:29:09.022360+0000","last_unstale":"2024-04-25T10:29:10.025554+0000","last_undegraded":"2024-04-25T10:29:10.025554+0000","last_fullsized":"2024-04-25T10:29:10.025554+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:29:07.995096+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:29:07.995096+0000","last_clean_scrub_stamp":"2024-04-25T10:29:07.995096+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T19:32:59.816545+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.2a","version":"0'0","reported_seq":12,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-04-25T10:29:10.187695+0000","last_change":"2024-04-25T10:29:09.029965+0000","last_active":"2024-04-25T10:29:10.187695+0000","last_peered":"2024-04-25T10:29:10.187695+0000","last_clean":"2024-04-25T10:29:10.187695+0000","last_became_active":"2024-04-25T10:29:09.029616+0000","last_became_peered":"2024-04-25T10:29:09.029616+0000","last_unstale":"2024-04-25T10:29:10.187695+0000","last_undegraded":"2024-04-25T10:29:10.187695+0000","last_fullsized":"2024-04-25T10:29:10.187695+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:29:07.995096+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:29:07.995096+0000","last_clean_scrub_stamp":"2024-04-25T10:29:07.995096+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T15:58:36.848831+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.29","version":"0'0","reported_seq":12,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-04-25T10:29:10.189345+0000","last_change":"2024-04-25T10:29:09.033401+0000","last_active":"2024-04-25T10:29:10.189345+0000","last_peered":"2024-04-25T10:29:10.189345+0000","last_clean":"2024-04-25T10:29:10.189345+0000","last_became_active":"2024-04-25T10:29:09.032764+0000","last_became_peered":"2024-04-25T10:29:09.032764+0000","last_unstale":"2024-04-25T10:29:10.189345+0000","last_undegraded":"2024-04-25T10:29:10.189345+0000","last_fullsized":"2024-04-25T10:29:10.189345+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:29:07.995096+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:29:07.995096+0000","last_clean_scrub_stamp":"2024-04-25T10:29:07.995096+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T17:30:54.093151+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.28","version":"0'0","reported_seq":12,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-04-25T10:29:10.024133+0000","last_change":"2024-04-25T10:29:09.041063+0000","last_active":"2024-04-25T10:29:10.024133+0000","last_peered":"2024-04-25T10:29:10.024133+0000","last_clean":"2024-04-25T10:29:10.024133+0000","last_became_active":"2024-04-25T10:29:09.040734+0000","last_became_peered":"2024-04-25T10:29:09.040734+0000","last_unstale":"2024-04-25T10:29:10.024133+0000","last_undegraded":"2024-04-25T10:29:10.024133+0000","last_fullsized":"2024-04-25T10:29:10.024133+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:29:07.995096+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:29:07.995096+0000","last_clean_scrub_stamp":"2024-04-25T10:29:07.995096+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T13:54:22.968216+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.27","version":"0'0","reported_seq":12,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-04-25T10:29:10.189019+0000","last_change":"2024-04-25T10:29:09.034268+0000","last_active":"2024-04-25T10:29:10.189019+0000","last_peered":"2024-04-25T10:29:10.189019+0000","last_clean":"2024-04-25T10:29:10.189019+0000","last_became_active":"2024-04-25T10:29:09.033199+0000","last_became_peered":"2024-04-25T10:29:09.033199+0000","last_unstale":"2024-04-25T10:29:10.189019+0000","last_undegraded":"2024-04-25T10:29:10.189019+0000","last_fullsized":"2024-04-25T10:29:10.189019+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:29:07.995096+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:29:07.995096+0000","last_clean_scrub_stamp":"2024-04-25T10:29:07.995096+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T17:56:09.999494+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.26","version":"0'0","reported_seq":12,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-04-25T10:29:10.026183+0000","last_change":"2024-04-25T10:29:09.025643+0000","last_active":"2024-04-25T10:29:10.026183+0000","last_peered":"2024-04-25T10:29:10.026183+0000","last_clean":"2024-04-25T10:29:10.026183+0000","last_became_active":"2024-04-25T10:29:09.024894+0000","last_became_peered":"2024-04-25T10:29:09.024894+0000","last_unstale":"2024-04-25T10:29:10.026183+0000","last_undegraded":"2024-04-25T10:29:10.026183+0000","last_fullsized":"2024-04-25T10:29:10.026183+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:29:07.995096+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:29:07.995096+0000","last_clean_scrub_stamp":"2024-04-25T10:29:07.995096+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T22:08:00.367431+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.25","version":"0'0","reported_seq":12,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-04-25T10:29:10.189130+0000","last_change":"2024-04-25T10:29:09.023357+0000","last_active":"2024-04-25T10:29:10.189130+0000","last_peered":"2024-04-25T10:29:10.189130+0000","last_clean":"2024-04-25T10:29:10.189130+0000","last_became_active":"2024-04-25T10:29:09.022988+0000","last_became_peered":"2024-04-25T10:29:09.022988+0000","last_unstale":"2024-04-25T10:29:10.189130+0000","last_undegraded":"2024-04-25T10:29:10.189130+0000","last_fullsized":"2024-04-25T10:29:10.189130+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:29:07.995096+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:29:07.995096+0000","last_clean_scrub_stamp":"2024-04-25T10:29:07.995096+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T21:14:39.661957+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.24","version":"0'0","reported_seq":12,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-04-25T10:29:10.189239+0000","last_change":"2024-04-25T10:29:09.034328+0000","last_active":"2024-04-25T10:29:10.189239+0000","last_peered":"2024-04-25T10:29:10.189239+0000","last_clean":"2024-04-25T10:29:10.189239+0000","last_became_active":"2024-04-25T10:29:09.033978+0000","last_became_peered":"2024-04-25T10:29:09.033978+0000","last_unstale":"2024-04-25T10:29:10.189239+0000","last_undegraded":"2024-04-25T10:29:10.189239+0000","last_fullsized":"2024-04-25T10:29:10.189239+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:29:07.995096+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:29:07.995096+0000","last_clean_scrub_stamp":"2024-04-25T10:29:07.995096+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T21:24:14.497242+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.23","version":"0'0","reported_seq":12,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-04-25T10:29:10.189583+0000","last_change":"2024-04-25T10:29:09.034045+0000","last_active":"2024-04-25T10:29:10.189583+0000","last_peered":"2024-04-25T10:29:10.189583+0000","last_clean":"2024-04-25T10:29:10.189583+0000","last_became_active":"2024-04-25T10:29:09.033730+0000","last_became_peered":"2024-04-25T10:29:09.033730+0000","last_unstale":"2024-04-25T10:29:10.189583+0000","last_undegraded":"2024-04-25T10:29:10.189583+0000","last_fullsized":"2024-04-25T10:29:10.189583+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:29:07.995096+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:29:07.995096+0000","last_clean_scrub_stamp":"2024-04-25T10:29:07.995096+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T12:36:31.191100+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.22","version":"0'0","reported_seq":12,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-04-25T10:29:10.024663+0000","last_change":"2024-04-25T10:29:09.034262+0000","last_active":"2024-04-25T10:29:10.024663+0000","last_peered":"2024-04-25T10:29:10.024663+0000","last_clean":"2024-04-25T10:29:10.024663+0000","last_became_active":"2024-04-25T10:29:09.033905+0000","last_became_peered":"2024-04-25T10:29:09.033905+0000","last_unstale":"2024-04-25T10:29:10.024663+0000","last_undegraded":"2024-04-25T10:29:10.024663+0000","last_fullsized":"2024-04-25T10:29:10.024663+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:29:07.995096+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:29:07.995096+0000","last_clean_scrub_stamp":"2024-04-25T10:29:07.995096+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T16:22:45.608928+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.21","version":"0'0","reported_seq":12,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-04-25T10:29:10.188201+0000","last_change":"2024-04-25T10:29:09.027088+0000","last_active":"2024-04-25T10:29:10.188201+0000","last_peered":"2024-04-25T10:29:10.188201+0000","last_clean":"2024-04-25T10:29:10.188201+0000","last_became_active":"2024-04-25T10:29:09.026651+0000","last_became_peered":"2024-04-25T10:29:09.026651+0000","last_unstale":"2024-04-25T10:29:10.188201+0000","last_undegraded":"2024-04-25T10:29:10.188201+0000","last_fullsized":"2024-04-25T10:29:10.188201+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:29:07.995096+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:29:07.995096+0000","last_clean_scrub_stamp":"2024-04-25T10:29:07.995096+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T15:37:06.723380+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.20","version":"0'0","reported_seq":12,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-04-25T10:29:10.025741+0000","last_change":"2024-04-25T10:29:09.028008+0000","last_active":"2024-04-25T10:29:10.025741+0000","last_peered":"2024-04-25T10:29:10.025741+0000","last_clean":"2024-04-25T10:29:10.025741+0000","last_became_active":"2024-04-25T10:29:09.027679+0000","last_became_peered":"2024-04-25T10:29:09.027679+0000","last_unstale":"2024-04-25T10:29:10.025741+0000","last_undegraded":"2024-04-25T10:29:10.025741+0000","last_fullsized":"2024-04-25T10:29:10.025741+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:29:07.995096+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:29:07.995096+0000","last_clean_scrub_stamp":"2024-04-25T10:29:07.995096+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T15:37:06.723380+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,6],"acting":[2,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.1a","version":"0'0","reported_seq":12,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-04-25T10:29:10.187787+0000","last_change":"2024-04-25T10:29:09.032819+0000","last_active":"2024-04-25T10:29:10.187787+0000","last_peered":"2024-04-25T10:29:10.187787+0000","last_clean":"2024-04-25T10:29:10.187787+0000","last_became_active":"2024-04-25T10:29:09.032493+0000","last_became_peered":"2024-04-25T10:29:09.032493+0000","last_unstale":"2024-04-25T10:29:10.187787+0000","last_undegraded":"2024-04-25T10:29:10.187787+0000","last_fullsized":"2024-04-25T10:29:10.187787+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:29:07.995096+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:29:07.995096+0000","last_clean_scrub_stamp":"2024-04-25T10:29:07.995096+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T15:37:06.723380+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.1b","version":"22'1","reported_seq":15,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-04-25T10:29:10.189463+0000","last_change":"2024-04-25T10:29:08.025272+0000","last_active":"2024-04-25T10:29:10.189463+0000","last_peered":"2024-04-25T10:29:10.189463+0000","last_clean":"2024-04-25T10:29:10.189463+0000","last_became_active":"2024-04-25T10:29:08.024912+0000","last_became_peered":"2024-04-25T10:29:08.024912+0000","last_unstale":"2024-04-25T10:29:10.189463+0000","last_undegraded":"2024-04-25T10:29:10.189463+0000","last_fullsized":"2024-04-25T10:29:10.189463+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:29:06.989570+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:29:06.989570+0000","last_clean_scrub_stamp":"2024-04-25T10:29:06.989570+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T18:11:32.758179+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6],"acting":[5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.1b","version":"0'0","reported_seq":12,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-04-25T10:29:10.025596+0000","last_change":"2024-04-25T10:29:09.032005+0000","last_active":"2024-04-25T10:29:10.025596+0000","last_peered":"2024-04-25T10:29:10.025596+0000","last_clean":"2024-04-25T10:29:10.025596+0000","last_became_active":"2024-04-25T10:29:09.031572+0000","last_became_peered":"2024-04-25T10:29:09.031572+0000","last_unstale":"2024-04-25T10:29:10.025596+0000","last_undegraded":"2024-04-25T10:29:10.025596+0000","last_fullsized":"2024-04-25T10:29:10.025596+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:29:07.995096+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:29:07.995096+0000","last_clean_scrub_stamp":"2024-04-25T10:29:07.995096+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T12:47:09.477097+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.1a","version":"22'1","reported_seq":15,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-04-25T10:29:10.024506+0000","last_change":"2024-04-25T10:29:08.026018+0000","last_active":"2024-04-25T10:29:10.024506+0000","last_peered":"2024-04-25T10:29:10.024506+0000","last_clean":"2024-04-25T10:29:10.024506+0000","last_became_active":"2024-04-25T10:29:08.025668+0000","last_became_peered":"2024-04-25T10:29:08.025668+0000","last_unstale":"2024-04-25T10:29:10.024506+0000","last_undegraded":"2024-04-25T10:29:10.024506+0000","last_fullsized":"2024-04-25T10:29:10.024506+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:29:06.989570+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:29:06.989570+0000","last_clean_scrub_stamp":"2024-04-25T10:29:06.989570+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T11:35:26.461348+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.19","version":"0'0","reported_seq":14,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-04-25T10:29:10.188333+0000","last_change":"2024-04-25T10:29:08.011702+0000","last_active":"2024-04-25T10:29:10.188333+0000","last_peered":"2024-04-25T10:29:10.188333+0000","last_clean":"2024-04-25T10:29:10.188333+0000","last_became_active":"2024-04-25T10:29:08.011353+0000","last_became_peered":"2024-04-25T10:29:08.011353+0000","last_unstale":"2024-04-25T10:29:10.188333+0000","last_undegraded":"2024-04-25T10:29:10.188333+0000","last_fullsized":"2024-04-25T10:29:10.188333+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:29:06.989570+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:29:06.989570+0000","last_clean_scrub_stamp":"2024-04-25T10:29:06.989570+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T16:31:11.685101+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.18","version":"0'0","reported_seq":12,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-04-25T10:29:10.189830+0000","last_change":"2024-04-25T10:29:09.033872+0000","last_active":"2024-04-25T10:29:10.189830+0000","last_peered":"2024-04-25T10:29:10.189830+0000","last_clean":"2024-04-25T10:29:10.189830+0000","last_became_active":"2024-04-25T10:29:09.033544+0000","last_became_peered":"2024-04-25T10:29:09.033544+0000","last_unstale":"2024-04-25T10:29:10.189830+0000","last_undegraded":"2024-04-25T10:29:10.189830+0000","last_fullsized":"2024-04-25T10:29:10.189830+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:29:07.995096+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:29:07.995096+0000","last_clean_scrub_stamp":"2024-04-25T10:29:07.995096+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T13:54:22.968216+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6],"acting":[5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.18","version":"0'0","reported_seq":14,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-04-25T10:29:10.024423+0000","last_change":"2024-04-25T10:29:08.013340+0000","last_active":"2024-04-25T10:29:10.024423+0000","last_peered":"2024-04-25T10:29:10.024423+0000","last_clean":"2024-04-25T10:29:10.024423+0000","last_became_active":"2024-04-25T10:29:08.012940+0000","last_became_peered":"2024-04-25T10:29:08.012940+0000","last_unstale":"2024-04-25T10:29:10.024423+0000","last_undegraded":"2024-04-25T10:29:10.024423+0000","last_fullsized":"2024-04-25T10:29:10.024423+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:29:06.989570+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:29:06.989570+0000","last_clean_scrub_stamp":"2024-04-25T10:29:06.989570+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T10:35:30.355988+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.19","version":"0'0","reported_seq":12,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-04-25T10:29:10.025603+0000","last_change":"2024-04-25T10:29:09.021718+0000","last_active":"2024-04-25T10:29:10.025603+0000","last_peered":"2024-04-25T10:29:10.025603+0000","last_clean":"2024-04-25T10:29:10.025603+0000","last_became_active":"2024-04-25T10:29:09.021416+0000","last_became_peered":"2024-04-25T10:29:09.021416+0000","last_unstale":"2024-04-25T10:29:10.025603+0000","last_undegraded":"2024-04-25T10:29:10.025603+0000","last_fullsized":"2024-04-25T10:29:10.025603+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:29:07.995096+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:29:07.995096+0000","last_clean_scrub_stamp":"2024-04-25T10:29:07.995096+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T18:55:45.597763+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.17","version":"0'0","reported_seq":14,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-04-25T10:29:10.187949+0000","last_change":"2024-04-25T10:29:08.020281+0000","last_active":"2024-04-25T10:29:10.187949+0000","last_peered":"2024-04-25T10:29:10.187949+0000","last_clean":"2024-04-25T10:29:10.187949+0000","last_became_active":"2024-04-25T10:29:08.019921+0000","last_became_peered":"2024-04-25T10:29:08.019921+0000","last_unstale":"2024-04-25T10:29:10.187949+0000","last_undegraded":"2024-04-25T10:29:10.187949+0000","last_fullsized":"2024-04-25T10:29:10.187949+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:29:06.989570+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:29:06.989570+0000","last_clean_scrub_stamp":"2024-04-25T10:29:06.989570+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T19:13:07.068780+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.16","version":"0'0","reported_seq":12,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-04-25T10:29:10.026197+0000","last_change":"2024-04-25T10:29:09.022744+0000","last_active":"2024-04-25T10:29:10.026197+0000","last_peered":"2024-04-25T10:29:10.026197+0000","last_clean":"2024-04-25T10:29:10.026197+0000","last_became_active":"2024-04-25T10:29:09.022392+0000","last_became_peered":"2024-04-25T10:29:09.022392+0000","last_unstale":"2024-04-25T10:29:10.026197+0000","last_undegraded":"2024-04-25T10:29:10.026197+0000","last_fullsized":"2024-04-25T10:29:10.026197+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:29:07.995096+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:29:07.995096+0000","last_clean_scrub_stamp":"2024-04-25T10:29:07.995096+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T11:35:27.466874+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.16","version":"0'0","reported_seq":14,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-04-25T10:29:10.037869+0000","last_change":"2024-04-25T10:29:08.026388+0000","last_active":"2024-04-25T10:29:10.037869+0000","last_peered":"2024-04-25T10:29:10.037869+0000","last_clean":"2024-04-25T10:29:10.037869+0000","last_became_active":"2024-04-25T10:29:08.026039+0000","last_became_peered":"2024-04-25T10:29:08.026039+0000","last_unstale":"2024-04-25T10:29:10.037869+0000","last_undegraded":"2024-04-25T10:29:10.037869+0000","last_fullsized":"2024-04-25T10:29:10.037869+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:29:06.989570+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:29:06.989570+0000","last_clean_scrub_stamp":"2024-04-25T10:29:06.989570+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T18:11:32.758179+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.17","version":"0'0","reported_seq":12,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-04-25T10:29:10.189465+0000","last_change":"2024-04-25T10:29:09.032902+0000","last_active":"2024-04-25T10:29:10.189465+0000","last_peered":"2024-04-25T10:29:10.189465+0000","last_clean":"2024-04-25T10:29:10.189465+0000","last_became_active":"2024-04-25T10:29:09.032596+0000","last_became_peered":"2024-04-25T10:29:09.032596+0000","last_unstale":"2024-04-25T10:29:10.189465+0000","last_undegraded":"2024-04-25T10:29:10.189465+0000","last_fullsized":"2024-04-25T10:29:10.189465+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:29:07.995096+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:29:07.995096+0000","last_clean_scrub_stamp":"2024-04-25T10:29:07.995096+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T19:14:48.737269+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.15","version":"22'3","reported_seq":17,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-04-25T10:29:10.188352+0000","last_change":"2024-04-25T10:29:08.021347+0000","last_active":"2024-04-25T10:29:10.188352+0000","last_peered":"2024-04-25T10:29:10.188352+0000","last_clean":"2024-04-25T10:29:10.188352+0000","last_became_active":"2024-04-25T10:29:08.021039+0000","last_became_peered":"2024-04-25T10:29:08.021039+0000","last_unstale":"2024-04-25T10:29:10.188352+0000","last_undegraded":"2024-04-25T10:29:10.188352+0000","last_fullsized":"2024-04-25T10:29:10.188352+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:29:06.989570+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:29:06.989570+0000","last_clean_scrub_stamp":"2024-04-25T10:29:06.989570+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T20:22:56.380539+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.14","version":"0'0","reported_seq":12,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-04-25T10:29:10.024492+0000","last_change":"2024-04-25T10:29:09.021410+0000","last_active":"2024-04-25T10:29:10.024492+0000","last_peered":"2024-04-25T10:29:10.024492+0000","last_clean":"2024-04-25T10:29:10.024492+0000","last_became_active":"2024-04-25T10:29:09.021051+0000","last_became_peered":"2024-04-25T10:29:09.021051+0000","last_unstale":"2024-04-25T10:29:10.024492+0000","last_undegraded":"2024-04-25T10:29:10.024492+0000","last_fullsized":"2024-04-25T10:29:10.024492+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:29:07.995096+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:29:07.995096+0000","last_clean_scrub_stamp":"2024-04-25T10:29:07.995096+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T13:22:13.080513+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.14","version":"22'6","reported_seq":20,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-04-25T10:29:10.025912+0000","last_change":"2024-04-25T10:29:08.018044+0000","last_active":"2024-04-25T10:29:10.025912+0000","last_peered":"2024-04-25T10:29:10.025912+0000","last_clean":"2024-04-25T10:29:10.025912+0000","last_became_active":"2024-04-25T10:29:08.016984+0000","last_became_peered":"2024-04-25T10:29:08.016984+0000","last_unstale":"2024-04-25T10:29:10.025912+0000","last_undegraded":"2024-04-25T10:29:10.025912+0000","last_fullsized":"2024-04-25T10:29:10.025912+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:29:06.989570+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:29:06.989570+0000","last_clean_scrub_stamp":"2024-04-25T10:29:06.989570+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T18:29:56.525236+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1612,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4],"acting":[2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.15","version":"0'0","reported_seq":12,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-04-25T10:29:10.037668+0000","last_change":"2024-04-25T10:29:09.029034+0000","last_active":"2024-04-25T10:29:10.037668+0000","last_peered":"2024-04-25T10:29:10.037668+0000","last_clean":"2024-04-25T10:29:10.037668+0000","last_became_active":"2024-04-25T10:29:09.028662+0000","last_became_peered":"2024-04-25T10:29:09.028662+0000","last_unstale":"2024-04-25T10:29:10.037668+0000","last_undegraded":"2024-04-25T10:29:10.037668+0000","last_fullsized":"2024-04-25T10:29:10.037668+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:29:07.995096+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:29:07.995096+0000","last_clean_scrub_stamp":"2024-04-25T10:29:07.995096+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T17:30:54.093151+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.13","version":"22'2","reported_seq":16,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-04-25T10:29:10.189430+0000","last_change":"2024-04-25T10:29:08.020476+0000","last_active":"2024-04-25T10:29:10.189430+0000","last_peered":"2024-04-25T10:29:10.189430+0000","last_clean":"2024-04-25T10:29:10.189430+0000","last_became_active":"2024-04-25T10:29:08.020096+0000","last_became_peered":"2024-04-25T10:29:08.020096+0000","last_unstale":"2024-04-25T10:29:10.189430+0000","last_undegraded":"2024-04-25T10:29:10.189430+0000","last_fullsized":"2024-04-25T10:29:10.189430+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:29:06.989570+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:29:06.989570+0000","last_clean_scrub_stamp":"2024-04-25T10:29:06.989570+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T10:32:06.103123+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":34,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":7,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.12","version":"0'0","reported_seq":12,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-04-25T10:29:10.188838+0000","last_change":"2024-04-25T10:29:09.027996+0000","last_active":"2024-04-25T10:29:10.188838+0000","last_peered":"2024-04-25T10:29:10.188838+0000","last_clean":"2024-04-25T10:29:10.188838+0000","last_became_active":"2024-04-25T10:29:09.027538+0000","last_became_peered":"2024-04-25T10:29:09.027538+0000","last_unstale":"2024-04-25T10:29:10.188838+0000","last_undegraded":"2024-04-25T10:29:10.188838+0000","last_fullsized":"2024-04-25T10:29:10.188838+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:29:07.995096+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:29:07.995096+0000","last_clean_scrub_stamp":"2024-04-25T10:29:07.995096+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T12:16:47.983013+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.12","version":"0'0","reported_seq":14,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-04-25T10:29:10.024603+0000","last_change":"2024-04-25T10:29:08.019028+0000","last_active":"2024-04-25T10:29:10.024603+0000","last_peered":"2024-04-25T10:29:10.024603+0000","last_clean":"2024-04-25T10:29:10.024603+0000","last_became_active":"2024-04-25T10:29:08.018582+0000","last_became_peered":"2024-04-25T10:29:08.018582+0000","last_unstale":"2024-04-25T10:29:10.024603+0000","last_undegraded":"2024-04-25T10:29:10.024603+0000","last_fullsized":"2024-04-25T10:29:10.024603+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:29:06.989570+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:29:06.989570+0000","last_clean_scrub_stamp":"2024-04-25T10:29:06.989570+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T19:23:11.583888+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.13","version":"0'0","reported_seq":12,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-04-25T10:29:10.024281+0000","last_change":"2024-04-25T10:29:09.023995+0000","last_active":"2024-04-25T10:29:10.024281+0000","last_peered":"2024-04-25T10:29:10.024281+0000","last_clean":"2024-04-25T10:29:10.024281+0000","last_became_active":"2024-04-25T10:29:09.023690+0000","last_became_peered":"2024-04-25T10:29:09.023690+0000","last_unstale":"2024-04-25T10:29:10.024281+0000","last_undegraded":"2024-04-25T10:29:10.024281+0000","last_fullsized":"2024-04-25T10:29:10.024281+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:29:07.995096+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:29:07.995096+0000","last_clean_scrub_stamp":"2024-04-25T10:29:07.995096+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T15:28:23.689564+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.11","version":"0'0","reported_seq":14,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-04-25T10:29:10.037576+0000","last_change":"2024-04-25T10:29:08.027174+0000","last_active":"2024-04-25T10:29:10.037576+0000","last_peered":"2024-04-25T10:29:10.037576+0000","last_clean":"2024-04-25T10:29:10.037576+0000","last_became_active":"2024-04-25T10:29:08.026809+0000","last_became_peered":"2024-04-25T10:29:08.026809+0000","last_unstale":"2024-04-25T10:29:10.037576+0000","last_undegraded":"2024-04-25T10:29:10.037576+0000","last_fullsized":"2024-04-25T10:29:10.037576+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:29:06.989570+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:29:06.989570+0000","last_clean_scrub_stamp":"2024-04-25T10:29:06.989570+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T10:35:30.355988+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.10","version":"0'0","reported_seq":12,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-04-25T10:29:10.187809+0000","last_change":"2024-04-25T10:29:09.041722+0000","last_active":"2024-04-25T10:29:10.187809+0000","last_peered":"2024-04-25T10:29:10.187809+0000","last_clean":"2024-04-25T10:29:10.187809+0000","last_became_active":"2024-04-25T10:29:09.040971+0000","last_became_peered":"2024-04-25T10:29:09.040971+0000","last_unstale":"2024-04-25T10:29:10.187809+0000","last_undegraded":"2024-04-25T10:29:10.187809+0000","last_fullsized":"2024-04-25T10:29:10.187809+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:29:07.995096+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:29:07.995096+0000","last_clean_scrub_stamp":"2024-04-25T10:29:07.995096+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T10:35:31.361514+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.10","version":"22'8","reported_seq":22,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-04-25T10:29:10.025277+0000","last_change":"2024-04-25T10:29:08.014110+0000","last_active":"2024-04-25T10:29:10.025277+0000","last_peered":"2024-04-25T10:29:10.025277+0000","last_clean":"2024-04-25T10:29:10.025277+0000","last_became_active":"2024-04-25T10:29:08.013543+0000","last_became_peered":"2024-04-25T10:29:08.013543+0000","last_unstale":"2024-04-25T10:29:10.025277+0000","last_undegraded":"2024-04-25T10:29:10.025277+0000","last_fullsized":"2024-04-25T10:29:10.025277+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:29:06.989570+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:29:06.989570+0000","last_clean_scrub_stamp":"2024-04-25T10:29:06.989570+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T16:44:30.899898+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":12,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7],"acting":[4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.11","version":"0'0","reported_seq":12,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-04-25T10:29:10.024238+0000","last_change":"2024-04-25T10:29:09.020490+0000","last_active":"2024-04-25T10:29:10.024238+0000","last_peered":"2024-04-25T10:29:10.024238+0000","last_clean":"2024-04-25T10:29:10.024238+0000","last_became_active":"2024-04-25T10:29:09.020178+0000","last_became_peered":"2024-04-25T10:29:09.020178+0000","last_unstale":"2024-04-25T10:29:10.024238+0000","last_undegraded":"2024-04-25T10:29:10.024238+0000","last_fullsized":"2024-04-25T10:29:10.024238+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:29:07.995096+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:29:07.995096+0000","last_clean_scrub_stamp":"2024-04-25T10:29:07.995096+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T20:37:20.726766+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.f","version":"22'1","reported_seq":15,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-04-25T10:29:10.038104+0000","last_change":"2024-04-25T10:29:08.029879+0000","last_active":"2024-04-25T10:29:10.038104+0000","last_peered":"2024-04-25T10:29:10.038104+0000","last_clean":"2024-04-25T10:29:10.038104+0000","last_became_active":"2024-04-25T10:29:08.029549+0000","last_became_peered":"2024-04-25T10:29:08.029549+0000","last_unstale":"2024-04-25T10:29:10.038104+0000","last_undegraded":"2024-04-25T10:29:10.038104+0000","last_fullsized":"2024-04-25T10:29:10.038104+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:29:06.989570+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:29:06.989570+0000","last_clean_scrub_stamp":"2024-04-25T10:29:06.989570+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T15:37:05.717854+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.e","version":"0'0","reported_seq":12,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-04-25T10:29:10.026273+0000","last_change":"2024-04-25T10:29:09.033703+0000","last_active":"2024-04-25T10:29:10.026273+0000","last_peered":"2024-04-25T10:29:10.026273+0000","last_clean":"2024-04-25T10:29:10.026273+0000","last_became_active":"2024-04-25T10:29:09.033318+0000","last_became_peered":"2024-04-25T10:29:09.033318+0000","last_unstale":"2024-04-25T10:29:10.026273+0000","last_undegraded":"2024-04-25T10:29:10.026273+0000","last_fullsized":"2024-04-25T10:29:10.026273+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:29:07.995096+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:29:07.995096+0000","last_clean_scrub_stamp":"2024-04-25T10:29:07.995096+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T12:16:47.983013+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.e","version":"22'2","reported_seq":16,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-04-25T10:29:10.026042+0000","last_change":"2024-04-25T10:29:08.020703+0000","last_active":"2024-04-25T10:29:10.026042+0000","last_peered":"2024-04-25T10:29:10.026042+0000","last_clean":"2024-04-25T10:29:10.026042+0000","last_became_active":"2024-04-25T10:29:08.020321+0000","last_became_peered":"2024-04-25T10:29:08.020321+0000","last_unstale":"2024-04-25T10:29:10.026042+0000","last_undegraded":"2024-04-25T10:29:10.026042+0000","last_fullsized":"2024-04-25T10:29:10.026042+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:29:06.989570+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:29:06.989570+0000","last_clean_scrub_stamp":"2024-04-25T10:29:06.989570+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T15:42:07.059913+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5],"acting":[4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.f","version":"0'0","reported_seq":12,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-04-25T10:29:10.038152+0000","last_change":"2024-04-25T10:29:09.022708+0000","last_active":"2024-04-25T10:29:10.038152+0000","last_peered":"2024-04-25T10:29:10.038152+0000","last_clean":"2024-04-25T10:29:10.038152+0000","last_became_active":"2024-04-25T10:29:09.020856+0000","last_became_peered":"2024-04-25T10:29:09.020856+0000","last_unstale":"2024-04-25T10:29:10.038152+0000","last_undegraded":"2024-04-25T10:29:10.038152+0000","last_fullsized":"2024-04-25T10:29:10.038152+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:29:07.995096+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:29:07.995096+0000","last_clean_scrub_stamp":"2024-04-25T10:29:07.995096+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T15:28:23.689564+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.d","version":"22'1","reported_seq":15,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-04-25T10:29:10.189469+0000","last_change":"2024-04-25T10:29:08.019488+0000","last_active":"2024-04-25T10:29:10.189469+0000","last_peered":"2024-04-25T10:29:10.189469+0000","last_clean":"2024-04-25T10:29:10.189469+0000","last_became_active":"2024-04-25T10:29:08.018971+0000","last_became_peered":"2024-04-25T10:29:08.018971+0000","last_unstale":"2024-04-25T10:29:10.189469+0000","last_undegraded":"2024-04-25T10:29:10.189469+0000","last_fullsized":"2024-04-25T10:29:10.189469+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:29:06.989570+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:29:06.989570+0000","last_clean_scrub_stamp":"2024-04-25T10:29:06.989570+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T20:22:56.380539+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":70,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.c","version":"0'0","reported_seq":12,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-04-25T10:29:10.037902+0000","last_change":"2024-04-25T10:29:09.029666+0000","last_active":"2024-04-25T10:29:10.037902+0000","last_peered":"2024-04-25T10:29:10.037902+0000","last_clean":"2024-04-25T10:29:10.037902+0000","last_became_active":"2024-04-25T10:29:09.029184+0000","last_became_peered":"2024-04-25T10:29:09.029184+0000","last_unstale":"2024-04-25T10:29:10.037902+0000","last_undegraded":"2024-04-25T10:29:10.037902+0000","last_fullsized":"2024-04-25T10:29:10.037902+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:29:07.995096+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:29:07.995096+0000","last_clean_scrub_stamp":"2024-04-25T10:29:07.995096+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T13:54:22.968216+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.c","version":"0'0","reported_seq":14,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-04-25T10:29:10.189541+0000","last_change":"2024-04-25T10:29:08.012124+0000","last_active":"2024-04-25T10:29:10.189541+0000","last_peered":"2024-04-25T10:29:10.189541+0000","last_clean":"2024-04-25T10:29:10.189541+0000","last_became_active":"2024-04-25T10:29:08.011817+0000","last_became_peered":"2024-04-25T10:29:08.011817+0000","last_unstale":"2024-04-25T10:29:10.189541+0000","last_undegraded":"2024-04-25T10:29:10.189541+0000","last_fullsized":"2024-04-25T10:29:10.189541+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:29:06.989570+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:29:06.989570+0000","last_clean_scrub_stamp":"2024-04-25T10:29:06.989570+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T20:22:56.380539+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.d","version":"0'0","reported_seq":12,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-04-25T10:29:10.024815+0000","last_change":"2024-04-25T10:29:09.017227+0000","last_active":"2024-04-25T10:29:10.024815+0000","last_peered":"2024-04-25T10:29:10.024815+0000","last_clean":"2024-04-25T10:29:10.024815+0000","last_became_active":"2024-04-25T10:29:09.016911+0000","last_became_peered":"2024-04-25T10:29:09.016911+0000","last_unstale":"2024-04-25T10:29:10.024815+0000","last_undegraded":"2024-04-25T10:29:10.024815+0000","last_fullsized":"2024-04-25T10:29:10.024815+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:29:07.995096+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:29:07.995096+0000","last_clean_scrub_stamp":"2024-04-25T10:29:07.995096+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T18:51:41.934269+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.0","version":"18'72","reported_seq":110,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-04-25T10:29:10.187517+0000","last_change":"2024-04-25T10:28:57.972696+0000","last_active":"2024-04-25T10:29:10.187517+0000","last_peered":"2024-04-25T10:29:10.187517+0000","last_clean":"2024-04-25T10:29:10.187517+0000","last_became_active":"2024-04-25T10:28:57.972373+0000","last_became_peered":"2024-04-25T10:28:57.972373+0000","last_unstale":"2024-04-25T10:29:10.187517+0000","last_undegraded":"2024-04-25T10:29:10.187517+0000","last_fullsized":"2024-04-25T10:29:10.187517+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:28:56.959439+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:28:56.959439+0000","last_clean_scrub_stamp":"2024-04-25T10:28:56.959439+0000","objects_scrubbed":0,"log_size":72,"log_dups_size":0,"ondisk_log_size":72,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T17:34:34.057408+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":66,"num_read_kb":55,"num_write":101,"num_write_kb":1628,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.3","version":"0'0","reported_seq":14,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-04-25T10:29:10.024562+0000","last_change":"2024-04-25T10:29:08.025080+0000","last_active":"2024-04-25T10:29:10.024562+0000","last_peered":"2024-04-25T10:29:10.024562+0000","last_clean":"2024-04-25T10:29:10.024562+0000","last_became_active":"2024-04-25T10:29:08.024747+0000","last_became_peered":"2024-04-25T10:29:08.024747+0000","last_unstale":"2024-04-25T10:29:10.024562+0000","last_undegraded":"2024-04-25T10:29:10.024562+0000","last_fullsized":"2024-04-25T10:29:10.024562+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:29:06.989570+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:29:06.989570+0000","last_clean_scrub_stamp":"2024-04-25T10:29:06.989570+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T22:07:59.361905+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.2","version":"0'0","reported_seq":12,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-04-25T10:29:10.025717+0000","last_change":"2024-04-25T10:29:09.023626+0000","last_active":"2024-04-25T10:29:10.025717+0000","last_peered":"2024-04-25T10:29:10.025717+0000","last_clean":"2024-04-25T10:29:10.025717+0000","last_became_active":"2024-04-25T10:29:09.020335+0000","last_became_peered":"2024-04-25T10:29:09.020335+0000","last_unstale":"2024-04-25T10:29:10.025717+0000","last_undegraded":"2024-04-25T10:29:10.025717+0000","last_fullsized":"2024-04-25T10:29:10.025717+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:29:07.995096+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:29:07.995096+0000","last_clean_scrub_stamp":"2024-04-25T10:29:07.995096+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T15:30:10.628413+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.0","version":"0'0","reported_seq":14,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-04-25T10:29:10.024309+0000","last_change":"2024-04-25T10:29:08.027479+0000","last_active":"2024-04-25T10:29:10.024309+0000","last_peered":"2024-04-25T10:29:10.024309+0000","last_clean":"2024-04-25T10:29:10.024309+0000","last_became_active":"2024-04-25T10:29:08.027178+0000","last_became_peered":"2024-04-25T10:29:08.027178+0000","last_unstale":"2024-04-25T10:29:10.024309+0000","last_undegraded":"2024-04-25T10:29:10.024309+0000","last_fullsized":"2024-04-25T10:29:10.024309+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:29:06.989570+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:29:06.989570+0000","last_clean_scrub_stamp":"2024-04-25T10:29:06.989570+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T19:54:16.697792+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.1","version":"0'0","reported_seq":12,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-04-25T10:29:10.188195+0000","last_change":"2024-04-25T10:29:09.028777+0000","last_active":"2024-04-25T10:29:10.188195+0000","last_peered":"2024-04-25T10:29:10.188195+0000","last_clean":"2024-04-25T10:29:10.188195+0000","last_became_active":"2024-04-25T10:29:09.027709+0000","last_became_peered":"2024-04-25T10:29:09.027709+0000","last_unstale":"2024-04-25T10:29:10.188195+0000","last_undegraded":"2024-04-25T10:29:10.188195+0000","last_fullsized":"2024-04-25T10:29:10.188195+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:29:07.995096+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:29:07.995096+0000","last_clean_scrub_stamp":"2024-04-25T10:29:07.995096+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T22:08:00.367431+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,2],"acting":[6,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.1","version":"0'0","reported_seq":14,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-04-25T10:29:10.188429+0000","last_change":"2024-04-25T10:29:08.020944+0000","last_active":"2024-04-25T10:29:10.188429+0000","last_peered":"2024-04-25T10:29:10.188429+0000","last_clean":"2024-04-25T10:29:10.188429+0000","last_became_active":"2024-04-25T10:29:08.020259+0000","last_became_peered":"2024-04-25T10:29:08.020259+0000","last_unstale":"2024-04-25T10:29:10.188429+0000","last_undegraded":"2024-04-25T10:29:10.188429+0000","last_fullsized":"2024-04-25T10:29:10.188429+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:29:06.989570+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:29:06.989570+0000","last_clean_scrub_stamp":"2024-04-25T10:29:06.989570+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T16:31:11.685101+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.0","version":"0'0","reported_seq":12,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-04-25T10:29:10.189305+0000","last_change":"2024-04-25T10:29:09.035455+0000","last_active":"2024-04-25T10:29:10.189305+0000","last_peered":"2024-04-25T10:29:10.189305+0000","last_clean":"2024-04-25T10:29:10.189305+0000","last_became_active":"2024-04-25T10:29:09.035130+0000","last_became_peered":"2024-04-25T10:29:09.035130+0000","last_unstale":"2024-04-25T10:29:10.189305+0000","last_undegraded":"2024-04-25T10:29:10.189305+0000","last_fullsized":"2024-04-25T10:29:10.189305+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:29:07.995096+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:29:07.995096+0000","last_clean_scrub_stamp":"2024-04-25T10:29:07.995096+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T11:52:24.415332+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.2","version":"0'0","reported_seq":14,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-04-25T10:29:10.024714+0000","last_change":"2024-04-25T10:29:08.021694+0000","last_active":"2024-04-25T10:29:10.024714+0000","last_peered":"2024-04-25T10:29:10.024714+0000","last_clean":"2024-04-25T10:29:10.024714+0000","last_became_active":"2024-04-25T10:29:08.021052+0000","last_became_peered":"2024-04-25T10:29:08.021052+0000","last_unstale":"2024-04-25T10:29:10.024714+0000","last_undegraded":"2024-04-25T10:29:10.024714+0000","last_fullsized":"2024-04-25T10:29:10.024714+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:29:06.989570+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:29:06.989570+0000","last_clean_scrub_stamp":"2024-04-25T10:29:06.989570+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T21:14:38.656431+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.3","version":"0'0","reported_seq":12,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-04-25T10:29:10.037784+0000","last_change":"2024-04-25T10:29:09.021686+0000","last_active":"2024-04-25T10:29:10.037784+0000","last_peered":"2024-04-25T10:29:10.037784+0000","last_clean":"2024-04-25T10:29:10.037784+0000","last_became_active":"2024-04-25T10:29:09.020672+0000","last_became_peered":"2024-04-25T10:29:09.020672+0000","last_unstale":"2024-04-25T10:29:10.037784+0000","last_undegraded":"2024-04-25T10:29:10.037784+0000","last_fullsized":"2024-04-25T10:29:10.037784+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:29:07.995096+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:29:07.995096+0000","last_clean_scrub_stamp":"2024-04-25T10:29:07.995096+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T20:52:02.098872+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.4","version":"0'0","reported_seq":14,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-04-25T10:29:10.189133+0000","last_change":"2024-04-25T10:29:08.025582+0000","last_active":"2024-04-25T10:29:10.189133+0000","last_peered":"2024-04-25T10:29:10.189133+0000","last_clean":"2024-04-25T10:29:10.189133+0000","last_became_active":"2024-04-25T10:29:08.024961+0000","last_became_peered":"2024-04-25T10:29:08.024961+0000","last_unstale":"2024-04-25T10:29:10.189133+0000","last_undegraded":"2024-04-25T10:29:10.189133+0000","last_fullsized":"2024-04-25T10:29:10.189133+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:29:06.989570+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:29:06.989570+0000","last_clean_scrub_stamp":"2024-04-25T10:29:06.989570+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T10:32:06.103123+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6],"acting":[5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.5","version":"0'0","reported_seq":12,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-04-25T10:29:10.026083+0000","last_change":"2024-04-25T10:29:09.028003+0000","last_active":"2024-04-25T10:29:10.026083+0000","last_peered":"2024-04-25T10:29:10.026083+0000","last_clean":"2024-04-25T10:29:10.026083+0000","last_became_active":"2024-04-25T10:29:09.027654+0000","last_became_peered":"2024-04-25T10:29:09.027654+0000","last_unstale":"2024-04-25T10:29:10.026083+0000","last_undegraded":"2024-04-25T10:29:10.026083+0000","last_fullsized":"2024-04-25T10:29:10.026083+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:29:07.995096+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:29:07.995096+0000","last_clean_scrub_stamp":"2024-04-25T10:29:07.995096+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T19:54:17.703318+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,6],"acting":[2,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.5","version":"0'0","reported_seq":14,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-04-25T10:29:10.024926+0000","last_change":"2024-04-25T10:29:08.026181+0000","last_active":"2024-04-25T10:29:10.024926+0000","last_peered":"2024-04-25T10:29:10.024926+0000","last_clean":"2024-04-25T10:29:10.024926+0000","last_became_active":"2024-04-25T10:29:08.025780+0000","last_became_peered":"2024-04-25T10:29:08.025780+0000","last_unstale":"2024-04-25T10:29:10.024926+0000","last_undegraded":"2024-04-25T10:29:10.024926+0000","last_fullsized":"2024-04-25T10:29:10.024926+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:29:06.989570+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:29:06.989570+0000","last_clean_scrub_stamp":"2024-04-25T10:29:06.989570+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T12:16:46.977487+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.4","version":"0'0","reported_seq":12,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-04-25T10:29:10.189423+0000","last_change":"2024-04-25T10:29:09.037824+0000","last_active":"2024-04-25T10:29:10.189423+0000","last_peered":"2024-04-25T10:29:10.189423+0000","last_clean":"2024-04-25T10:29:10.189423+0000","last_became_active":"2024-04-25T10:29:09.037478+0000","last_became_peered":"2024-04-25T10:29:09.037478+0000","last_unstale":"2024-04-25T10:29:10.189423+0000","last_undegraded":"2024-04-25T10:29:10.189423+0000","last_fullsized":"2024-04-25T10:29:10.189423+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:29:07.995096+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:29:07.995096+0000","last_clean_scrub_stamp":"2024-04-25T10:29:07.995096+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T15:02:17.480054+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.6","version":"23'1","reported_seq":15,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-04-25T10:29:10.026003+0000","last_change":"2024-04-25T10:29:08.020327+0000","last_active":"2024-04-25T10:29:10.026003+0000","last_peered":"2024-04-25T10:29:10.026003+0000","last_clean":"2024-04-25T10:29:10.026003+0000","last_became_active":"2024-04-25T10:29:08.019988+0000","last_became_peered":"2024-04-25T10:29:08.019988+0000","last_unstale":"2024-04-25T10:29:10.026003+0000","last_undegraded":"2024-04-25T10:29:10.026003+0000","last_fullsized":"2024-04-25T10:29:10.026003+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:29:06.989570+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:29:06.989570+0000","last_clean_scrub_stamp":"2024-04-25T10:29:06.989570+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T13:17:35.774897+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.7","version":"0'0","reported_seq":12,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-04-25T10:29:10.025861+0000","last_change":"2024-04-25T10:29:09.029850+0000","last_active":"2024-04-25T10:29:10.025861+0000","last_peered":"2024-04-25T10:29:10.025861+0000","last_clean":"2024-04-25T10:29:10.025861+0000","last_became_active":"2024-04-25T10:29:09.029534+0000","last_became_peered":"2024-04-25T10:29:09.029534+0000","last_unstale":"2024-04-25T10:29:10.025861+0000","last_undegraded":"2024-04-25T10:29:10.025861+0000","last_fullsized":"2024-04-25T10:29:10.025861+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:29:07.995096+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:29:07.995096+0000","last_clean_scrub_stamp":"2024-04-25T10:29:07.995096+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T15:03:07.958155+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.7","version":"22'2","reported_seq":16,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-04-25T10:29:10.024876+0000","last_change":"2024-04-25T10:29:08.026080+0000","last_active":"2024-04-25T10:29:10.024876+0000","last_peered":"2024-04-25T10:29:10.024876+0000","last_clean":"2024-04-25T10:29:10.024876+0000","last_became_active":"2024-04-25T10:29:08.025707+0000","last_became_peered":"2024-04-25T10:29:08.025707+0000","last_unstale":"2024-04-25T10:29:10.024876+0000","last_undegraded":"2024-04-25T10:29:10.024876+0000","last_fullsized":"2024-04-25T10:29:10.024876+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:29:06.989570+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:29:06.989570+0000","last_clean_scrub_stamp":"2024-04-25T10:29:06.989570+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T10:55:00.499631+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.6","version":"0'0","reported_seq":12,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-04-25T10:29:10.025829+0000","last_change":"2024-04-25T10:29:09.025814+0000","last_active":"2024-04-25T10:29:10.025829+0000","last_peered":"2024-04-25T10:29:10.025829+0000","last_clean":"2024-04-25T10:29:10.025829+0000","last_became_active":"2024-04-25T10:29:09.025474+0000","last_became_peered":"2024-04-25T10:29:09.025474+0000","last_unstale":"2024-04-25T10:29:10.025829+0000","last_undegraded":"2024-04-25T10:29:10.025829+0000","last_fullsized":"2024-04-25T10:29:10.025829+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:29:07.995096+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:29:07.995096+0000","last_clean_scrub_stamp":"2024-04-25T10:29:07.995096+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T12:35:12.945537+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7],"acting":[4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.8","version":"0'0","reported_seq":14,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-04-25T10:29:10.037271+0000","last_change":"2024-04-25T10:29:08.024845+0000","last_active":"2024-04-25T10:29:10.037271+0000","last_peered":"2024-04-25T10:29:10.037271+0000","last_clean":"2024-04-25T10:29:10.037271+0000","last_became_active":"2024-04-25T10:29:08.024484+0000","last_became_peered":"2024-04-25T10:29:08.024484+0000","last_unstale":"2024-04-25T10:29:10.037271+0000","last_undegraded":"2024-04-25T10:29:10.037271+0000","last_fullsized":"2024-04-25T10:29:10.037271+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:29:06.989570+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:29:06.989570+0000","last_clean_scrub_stamp":"2024-04-25T10:29:06.989570+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T14:12:57.950482+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.9","version":"0'0","reported_seq":12,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-04-25T10:29:10.024346+0000","last_change":"2024-04-25T10:29:09.022502+0000","last_active":"2024-04-25T10:29:10.024346+0000","last_peered":"2024-04-25T10:29:10.024346+0000","last_clean":"2024-04-25T10:29:10.024346+0000","last_became_active":"2024-04-25T10:29:09.022184+0000","last_became_peered":"2024-04-25T10:29:09.022184+0000","last_unstale":"2024-04-25T10:29:10.024346+0000","last_undegraded":"2024-04-25T10:29:10.024346+0000","last_fullsized":"2024-04-25T10:29:10.024346+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:29:07.995096+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:29:07.995096+0000","last_clean_scrub_stamp":"2024-04-25T10:29:07.995096+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T19:56:27.663748+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.9","version":"0'0","reported_seq":14,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-04-25T10:29:10.025738+0000","last_change":"2024-04-25T10:29:08.024989+0000","last_active":"2024-04-25T10:29:10.025738+0000","last_peered":"2024-04-25T10:29:10.025738+0000","last_clean":"2024-04-25T10:29:10.025738+0000","last_became_active":"2024-04-25T10:29:08.024655+0000","last_became_peered":"2024-04-25T10:29:08.024655+0000","last_unstale":"2024-04-25T10:29:10.025738+0000","last_undegraded":"2024-04-25T10:29:10.025738+0000","last_fullsized":"2024-04-25T10:29:10.025738+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:29:06.989570+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:29:06.989570+0000","last_clean_scrub_stamp":"2024-04-25T10:29:06.989570+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T21:05:51.419862+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.8","version":"0'0","reported_seq":12,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-04-25T10:29:10.025393+0000","last_change":"2024-04-25T10:29:09.023330+0000","last_active":"2024-04-25T10:29:10.025393+0000","last_peered":"2024-04-25T10:29:10.025393+0000","last_clean":"2024-04-25T10:29:10.025393+0000","last_became_active":"2024-04-25T10:29:09.020543+0000","last_became_peered":"2024-04-25T10:29:09.020543+0000","last_unstale":"2024-04-25T10:29:10.025393+0000","last_undegraded":"2024-04-25T10:29:10.025393+0000","last_fullsized":"2024-04-25T10:29:10.025393+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:29:07.995096+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:29:07.995096+0000","last_clean_scrub_stamp":"2024-04-25T10:29:07.995096+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T11:12:41.953761+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7],"acting":[4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.a","version":"0'0","reported_seq":14,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-04-25T10:29:10.188957+0000","last_change":"2024-04-25T10:29:08.020363+0000","last_active":"2024-04-25T10:29:10.188957+0000","last_peered":"2024-04-25T10:29:10.188957+0000","last_clean":"2024-04-25T10:29:10.188957+0000","last_became_active":"2024-04-25T10:29:08.020037+0000","last_became_peered":"2024-04-25T10:29:08.020037+0000","last_unstale":"2024-04-25T10:29:10.188957+0000","last_undegraded":"2024-04-25T10:29:10.188957+0000","last_fullsized":"2024-04-25T10:29:10.188957+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:29:06.989570+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:29:06.989570+0000","last_clean_scrub_stamp":"2024-04-25T10:29:06.989570+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T10:32:06.103123+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.b","version":"0'0","reported_seq":12,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-04-25T10:29:10.189294+0000","last_change":"2024-04-25T10:29:09.028820+0000","last_active":"2024-04-25T10:29:10.189294+0000","last_peered":"2024-04-25T10:29:10.189294+0000","last_clean":"2024-04-25T10:29:10.189294+0000","last_became_active":"2024-04-25T10:29:09.028426+0000","last_became_peered":"2024-04-25T10:29:09.028426+0000","last_unstale":"2024-04-25T10:29:10.189294+0000","last_undegraded":"2024-04-25T10:29:10.189294+0000","last_fullsized":"2024-04-25T10:29:10.189294+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:29:07.995096+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:29:07.995096+0000","last_clean_scrub_stamp":"2024-04-25T10:29:07.995096+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T19:54:17.703318+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.b","version":"22'2","reported_seq":16,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-04-25T10:29:10.024445+0000","last_change":"2024-04-25T10:29:08.025011+0000","last_active":"2024-04-25T10:29:10.024445+0000","last_peered":"2024-04-25T10:29:10.024445+0000","last_clean":"2024-04-25T10:29:10.024445+0000","last_became_active":"2024-04-25T10:29:08.024667+0000","last_became_peered":"2024-04-25T10:29:08.024667+0000","last_unstale":"2024-04-25T10:29:10.024445+0000","last_undegraded":"2024-04-25T10:29:10.024445+0000","last_fullsized":"2024-04-25T10:29:10.024445+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:29:06.989570+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:29:06.989570+0000","last_clean_scrub_stamp":"2024-04-25T10:29:06.989570+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T15:37:05.717854+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.a","version":"0'0","reported_seq":12,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-04-25T10:29:10.024394+0000","last_change":"2024-04-25T10:29:09.017160+0000","last_active":"2024-04-25T10:29:10.024394+0000","last_peered":"2024-04-25T10:29:10.024394+0000","last_clean":"2024-04-25T10:29:10.024394+0000","last_became_active":"2024-04-25T10:29:09.016821+0000","last_became_peered":"2024-04-25T10:29:09.016821+0000","last_unstale":"2024-04-25T10:29:10.024394+0000","last_undegraded":"2024-04-25T10:29:10.024394+0000","last_fullsized":"2024-04-25T10:29:10.024394+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:29:07.995096+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:29:07.995096+0000","last_clean_scrub_stamp":"2024-04-25T10:29:07.995096+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T20:40:19.840339+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.1d","version":"0'0","reported_seq":12,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-04-25T10:29:10.024622+0000","last_change":"2024-04-25T10:29:09.030341+0000","last_active":"2024-04-25T10:29:10.024622+0000","last_peered":"2024-04-25T10:29:10.024622+0000","last_clean":"2024-04-25T10:29:10.024622+0000","last_became_active":"2024-04-25T10:29:09.029988+0000","last_became_peered":"2024-04-25T10:29:09.029988+0000","last_unstale":"2024-04-25T10:29:10.024622+0000","last_undegraded":"2024-04-25T10:29:10.024622+0000","last_fullsized":"2024-04-25T10:29:10.024622+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:29:07.995096+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:29:07.995096+0000","last_clean_scrub_stamp":"2024-04-25T10:29:07.995096+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T11:40:18.624659+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.1c","version":"22'2","reported_seq":16,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-04-25T10:29:10.037969+0000","last_change":"2024-04-25T10:29:08.027197+0000","last_active":"2024-04-25T10:29:10.037969+0000","last_peered":"2024-04-25T10:29:10.037969+0000","last_clean":"2024-04-25T10:29:10.037969+0000","last_became_active":"2024-04-25T10:29:08.026872+0000","last_became_peered":"2024-04-25T10:29:08.026872+0000","last_unstale":"2024-04-25T10:29:10.037969+0000","last_undegraded":"2024-04-25T10:29:10.037969+0000","last_fullsized":"2024-04-25T10:29:10.037969+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:29:06.989570+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:29:06.989570+0000","last_clean_scrub_stamp":"2024-04-25T10:29:06.989570+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T15:58:35.843305+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.1c","version":"0'0","reported_seq":12,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-04-25T10:29:10.187853+0000","last_change":"2024-04-25T10:29:09.032731+0000","last_active":"2024-04-25T10:29:10.187853+0000","last_peered":"2024-04-25T10:29:10.187853+0000","last_clean":"2024-04-25T10:29:10.187853+0000","last_became_active":"2024-04-25T10:29:09.032383+0000","last_became_peered":"2024-04-25T10:29:09.032383+0000","last_unstale":"2024-04-25T10:29:10.187853+0000","last_undegraded":"2024-04-25T10:29:10.187853+0000","last_fullsized":"2024-04-25T10:29:10.187853+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:29:07.995096+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:29:07.995096+0000","last_clean_scrub_stamp":"2024-04-25T10:29:07.995096+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T19:54:17.703318+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.1d","version":"22'2","reported_seq":16,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-04-25T10:29:10.037807+0000","last_change":"2024-04-25T10:29:08.026410+0000","last_active":"2024-04-25T10:29:10.037807+0000","last_peered":"2024-04-25T10:29:10.037807+0000","last_clean":"2024-04-25T10:29:10.037807+0000","last_became_active":"2024-04-25T10:29:08.026071+0000","last_became_peered":"2024-04-25T10:29:08.026071+0000","last_unstale":"2024-04-25T10:29:10.037807+0000","last_undegraded":"2024-04-25T10:29:10.037807+0000","last_fullsized":"2024-04-25T10:29:10.037807+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:29:06.989570+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:29:06.989570+0000","last_clean_scrub_stamp":"2024-04-25T10:29:06.989570+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T19:54:16.697792+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.1f","version":"0'0","reported_seq":12,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-04-25T10:29:10.189240+0000","last_change":"2024-04-25T10:29:09.032735+0000","last_active":"2024-04-25T10:29:10.189240+0000","last_peered":"2024-04-25T10:29:10.189240+0000","last_clean":"2024-04-25T10:29:10.189240+0000","last_became_active":"2024-04-25T10:29:09.032227+0000","last_became_peered":"2024-04-25T10:29:09.032227+0000","last_unstale":"2024-04-25T10:29:10.189240+0000","last_undegraded":"2024-04-25T10:29:10.189240+0000","last_fullsized":"2024-04-25T10:29:10.189240+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:29:07.995096+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:29:07.995096+0000","last_clean_scrub_stamp":"2024-04-25T10:29:07.995096+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T11:31:45.283247+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.1e","version":"22'1","reported_seq":15,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-04-25T10:29:10.189728+0000","last_change":"2024-04-25T10:29:08.017025+0000","last_active":"2024-04-25T10:29:10.189728+0000","last_peered":"2024-04-25T10:29:10.189728+0000","last_clean":"2024-04-25T10:29:10.189728+0000","last_became_active":"2024-04-25T10:29:08.016702+0000","last_became_peered":"2024-04-25T10:29:08.016702+0000","last_unstale":"2024-04-25T10:29:10.189728+0000","last_undegraded":"2024-04-25T10:29:10.189728+0000","last_fullsized":"2024-04-25T10:29:10.189728+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:29:06.989570+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:29:06.989570+0000","last_clean_scrub_stamp":"2024-04-25T10:29:06.989570+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T14:12:57.950482+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.1e","version":"0'0","reported_seq":12,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-04-25T10:29:10.189730+0000","last_change":"2024-04-25T10:29:09.036208+0000","last_active":"2024-04-25T10:29:10.189730+0000","last_peered":"2024-04-25T10:29:10.189730+0000","last_clean":"2024-04-25T10:29:10.189730+0000","last_became_active":"2024-04-25T10:29:09.034890+0000","last_became_peered":"2024-04-25T10:29:09.034890+0000","last_unstale":"2024-04-25T10:29:10.189730+0000","last_undegraded":"2024-04-25T10:29:10.189730+0000","last_fullsized":"2024-04-25T10:29:10.189730+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:29:07.995096+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:29:07.995096+0000","last_clean_scrub_stamp":"2024-04-25T10:29:07.995096+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T18:44:23.610374+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.1f","version":"22'4","reported_seq":18,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-04-25T10:29:10.026053+0000","last_change":"2024-04-25T10:29:08.026202+0000","last_active":"2024-04-25T10:29:10.026053+0000","last_peered":"2024-04-25T10:29:10.026053+0000","last_clean":"2024-04-25T10:29:10.026053+0000","last_became_active":"2024-04-25T10:29:08.025877+0000","last_became_peered":"2024-04-25T10:29:08.025877+0000","last_unstale":"2024-04-25T10:29:10.026053+0000","last_undegraded":"2024-04-25T10:29:10.026053+0000","last_fullsized":"2024-04-25T10:29:10.026053+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:29:06.989570+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:29:06.989570+0000","last_clean_scrub_stamp":"2024-04-25T10:29:06.989570+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T20:00:40.198709+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":582,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]}],"pool_stats":[{"poolid":3,"num_pg":64,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":128,"acting":128,"num_store_stats":8},{"poolid":2,"num_pg":32,"stat_sum":{"num_bytes":2410,"num_objects":22,"num_object_clones":0,"num_object_copies":44,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":22,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":45,"num_write_kb":14,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":14,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":65536,"data_stored":4820,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":39,"ondisk_log_size":39,"up":64,"acting":64,"num_store_stats":8},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":66,"num_read_kb":55,"num_write":101,"num_write_kb":1628,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1187840,"data_stored":1180736,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":72,"ondisk_log_size":72,"up":2,"acting":2,"num_store_stats":2}],"osd_stats":[{"osd":7,"up_from":15,"seq":64424509448,"num_pgs":16,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27260,"kb_used_data":236,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344580,"statfs":{"total":96636764160,"available":96608849920,"internally_reserved":0,"allocated":241664,"data_stored":81359,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1590,"internal_metadata":27654602},"hb_peers":[0,1,2,3,4,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":6,"up_from":15,"seq":64424509449,"num_pgs":20,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27252,"kb_used_data":228,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344588,"statfs":{"total":96636764160,"available":96608858112,"internally_reserved":0,"allocated":233472,"data_stored":81235,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":27654598},"hb_peers":[0,1,2,3,4,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":5,"up_from":15,"seq":64424509450,"num_pgs":28,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27840,"kb_used_data":816,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344000,"statfs":{"total":96636764160,"available":96608256000,"internally_reserved":0,"allocated":835584,"data_stored":671695,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":3,"up_from":15,"seq":64424509450,"num_pgs":30,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27256,"kb_used_data":232,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344584,"statfs":{"total":96636764160,"available":96608854016,"internally_reserved":0,"allocated":237568,"data_stored":81817,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1593,"internal_metadata":27654599},"hb_peers":[0,1,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":2,"up_from":15,"seq":64424509450,"num_pgs":22,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27264,"kb_used_data":240,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344576,"statfs":{"total":96636764160,"available":96608845824,"internally_reserved":0,"allocated":245760,"data_stored":82847,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":1,"up_from":15,"seq":64424509448,"num_pgs":21,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27836,"kb_used_data":812,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344004,"statfs":{"total":96636764160,"available":96608260096,"internally_reserved":0,"allocated":831488,"data_stored":671673,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":0,"up_from":15,"seq":64424509449,"num_pgs":35,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27260,"kb_used_data":236,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344580,"statfs":{"total":96636764160,"available":96608849920,"internally_reserved":0,"allocated":241664,"data_stored":81291,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[1,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":4,"up_from":13,"seq":55834574856,"num_pgs":22,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27276,"kb_used_data":248,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344564,"statfs":{"total":96636764160,"available":96608833536,"internally_reserved":0,"allocated":253952,"data_stored":83519,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":56,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":70,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":1612,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":582,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":20480,"data_stored":2284,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":92,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":124,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-04-25T10:29:22.729 INFO:tasks.ceph.ceph_manager.ceph:clean! 2024-04-25T10:29:22.729 INFO:tasks.ceph:Waiting until ceph cluster ceph is healthy... 2024-04-25T10:29:22.729 INFO:tasks.ceph.ceph_manager.ceph:wait_until_healthy 2024-04-25T10:29:22.729 DEBUG:teuthology.orchestra.run.smithi037:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph health --format=json 2024-04-25T10:29:22.875 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:22.873+0000 7f15078a1640 1 Processor -- start 2024-04-25T10:29:22.875 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:22.873+0000 7f15078a1640 1 -- start start 2024-04-25T10:29:22.878 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:22.874+0000 7f15078a1640 1 --2- >> v2:172.21.15.37:3300/0 conn(0x7f15000d5160 0x7f15000b4e80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:29:22.878 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:22.875+0000 7f15078a1640 1 --2- >> v2:172.21.15.203:3300/0 conn(0x7f15000d0990 0x7f15000d2b20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:29:22.878 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:22.875+0000 7f15078a1640 1 -- --> v1:172.21.15.37:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f15000d1370 con 0x7f15000d0da0 2024-04-25T10:29:22.878 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:22.875+0000 7f15078a1640 1 -- --> v2:172.21.15.37:3300/0 -- mon_getmap magic: 0 -- 0x7f15000d1650 con 0x7f15000d5160 2024-04-25T10:29:22.878 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:22.875+0000 7f15078a1640 1 -- --> v2:172.21.15.203:3300/0 -- mon_getmap magic: 0 -- 0x7f15000d4680 con 0x7f15000d0990 2024-04-25T10:29:22.878 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:22.875+0000 7f150563d640 1 --2- >> v2:172.21.15.37:3300/0 conn(0x7f15000d5160 0x7f15000b4e80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:29:22.878 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:22.875+0000 7f150563d640 1 --2- >> v2:172.21.15.37:3300/0 conn(0x7f15000d5160 0x7f15000b4e80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.37:3300/0 says I am v2:172.21.15.37:35130/0 (socket says 172.21.15.37:35130) 2024-04-25T10:29:22.878 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:22.875+0000 7f150563d640 1 -- 172.21.15.37:0/3109721827 learned_addr learned my addr 172.21.15.37:0/3109721827 (peer_addr_for_me v2:172.21.15.37:0/0) 2024-04-25T10:29:22.878 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:22.875+0000 7f1505e3e640 1 --1- 172.21.15.37:0/3109721827 >> v1:172.21.15.37:6789/0 conn(0x7f15000d0da0 0x7f150006db10 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.37:6789/0 says I am v1:172.21.15.37:59822/0 (socket says 172.21.15.37:59822) 2024-04-25T10:29:22.879 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:22.875+0000 7f1504e3c640 1 --2- 172.21.15.37:0/3109721827 >> v2:172.21.15.203:3300/0 conn(0x7f15000d0990 0x7f15000d2b20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:29:22.879 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:22.875+0000 7f150563d640 1 -- 172.21.15.37:0/3109721827 >> v2:172.21.15.203:3300/0 conn(0x7f15000d0990 msgr2=0x7f15000d2b20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:29:22.879 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:22.875+0000 7f150563d640 1 --2- 172.21.15.37:0/3109721827 >> v2:172.21.15.203:3300/0 conn(0x7f15000d0990 0x7f15000d2b20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:29:22.879 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:22.875+0000 7f14f7fff640 1 -- 172.21.15.37:0/3109721827 <== mon.0 v1:172.21.15.37:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 4128662735 0 0) 0x7f15000d1370 con 0x7f15000d0da0 2024-04-25T10:29:22.879 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:22.875+0000 7f150563d640 1 -- 172.21.15.37:0/3109721827 >> v1:172.21.15.37:6789/0 conn(0x7f15000d0da0 legacy=0x7f150006db10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:29:22.879 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:22.875+0000 7f150563d640 1 -- 172.21.15.37:0/3109721827 --> v2:172.21.15.37:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f15000d4960 con 0x7f15000d5160 2024-04-25T10:29:22.879 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:22.876+0000 7f150563d640 1 --2- 172.21.15.37:0/3109721827 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f15000d5160 0x7f15000b4e80 secure :-1 s=READY pgs=139 cs=0 l=1 rev1=1 crypto rx=0x7f15000d6b80 tx=0x7f14fc02a150 comp rx=0 tx=0).ready entity=mon.0 client_cookie=4cfc6eb16d9e2960 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:29:22.879 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:22.876+0000 7f14f7fff640 1 -- 172.21.15.37:0/3109721827 <== mon.0 v2:172.21.15.37:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f14fc037070 con 0x7f15000d5160 2024-04-25T10:29:22.879 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:22.876+0000 7f14f7fff640 1 -- 172.21.15.37:0/3109721827 <== mon.0 v2:172.21.15.37:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f14fc032440 con 0x7f15000d5160 2024-04-25T10:29:22.879 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:22.876+0000 7f14f7fff640 1 -- 172.21.15.37:0/3109721827 <== mon.0 v2:172.21.15.37:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f14fc032810 con 0x7f15000d5160 2024-04-25T10:29:22.880 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:22.877+0000 7f15078a1640 1 -- 172.21.15.37:0/3109721827 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f15000d5160 msgr2=0x7f15000b4e80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:29:22.881 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:22.877+0000 7f15078a1640 1 --2- 172.21.15.37:0/3109721827 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f15000d5160 0x7f15000b4e80 secure :-1 s=READY pgs=139 cs=0 l=1 rev1=1 crypto rx=0x7f15000d6b80 tx=0x7f14fc02a150 comp rx=0 tx=0).stop 2024-04-25T10:29:22.881 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:22.877+0000 7f15078a1640 1 -- 172.21.15.37:0/3109721827 shutdown_connections 2024-04-25T10:29:22.881 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:22.877+0000 7f15078a1640 1 --2- 172.21.15.37:0/3109721827 >> v2:172.21.15.203:3300/0 conn(0x7f15000d0990 0x7f15000d2b20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:29:22.881 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:22.877+0000 7f15078a1640 1 --2- 172.21.15.37:0/3109721827 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f15000d5160 0x7f15000b4e80 unknown :-1 s=CLOSED pgs=139 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:29:22.881 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:22.877+0000 7f15078a1640 1 -- 172.21.15.37:0/3109721827 >> 172.21.15.37:0/3109721827 conn(0x7f15000b7a80 msgr2=0x7f15000b7e90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:29:22.881 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:22.877+0000 7f15078a1640 1 -- 172.21.15.37:0/3109721827 shutdown_connections 2024-04-25T10:29:22.881 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:22.877+0000 7f15078a1640 1 -- 172.21.15.37:0/3109721827 wait complete. 2024-04-25T10:29:22.881 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:22.878+0000 7f15078a1640 1 Processor -- start 2024-04-25T10:29:22.881 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:22.878+0000 7f15078a1640 1 -- start start 2024-04-25T10:29:22.883 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:22.880+0000 7f15078a1640 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f15000d0990 0x7f1500159d90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:29:22.883 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:22.880+0000 7f15078a1640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f15000d0da0 0x7f150015a2d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:29:22.883 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:22.880+0000 7f15078a1640 1 --2- >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f15000d5160 0x7f150015af20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:29:22.883 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:22.880+0000 7f15078a1640 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 -- 0x7f1500164c40 con 0x7f15000d0da0 2024-04-25T10:29:22.883 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:22.880+0000 7f15078a1640 1 -- --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_getmap magic: 0 -- 0x7f1500164ac0 con 0x7f15000d0990 2024-04-25T10:29:22.884 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:22.880+0000 7f15078a1640 1 -- --> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] -- mon_getmap magic: 0 -- 0x7f15000d3060 con 0x7f15000d5160 2024-04-25T10:29:22.884 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:22.880+0000 7f1504e3c640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f15000d0da0 0x7f150015a2d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:29:22.884 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:22.880+0000 7f1504e3c640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f15000d0da0 0x7f150015a2d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.37:3300/0 says I am v2:172.21.15.37:35132/0 (socket says 172.21.15.37:35132) 2024-04-25T10:29:22.884 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:22.880+0000 7f1504e3c640 1 -- 172.21.15.37:0/3255696311 learned_addr learned my addr 172.21.15.37:0/3255696311 (peer_addr_for_me v2:172.21.15.37:0/0) 2024-04-25T10:29:22.884 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:22.880+0000 7f1505e3e640 1 --2- 172.21.15.37:0/3255696311 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f15000d5160 0x7f150015af20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:29:22.884 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:22.880+0000 7f150563d640 1 --2- 172.21.15.37:0/3255696311 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f15000d0990 0x7f1500159d90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:29:22.884 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:22.880+0000 7f1504e3c640 1 -- 172.21.15.37:0/3255696311 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f15000d5160 msgr2=0x7f150015af20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:29:22.884 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:22.880+0000 7f1504e3c640 1 --2- 172.21.15.37:0/3255696311 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f15000d5160 0x7f150015af20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:29:22.884 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:22.880+0000 7f1504e3c640 1 -- 172.21.15.37:0/3255696311 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f15000d0990 msgr2=0x7f1500159d90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:29:22.884 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:22.880+0000 7f1504e3c640 1 --2- 172.21.15.37:0/3255696311 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f15000d0990 0x7f1500159d90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:29:22.884 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:22.880+0000 7f1504e3c640 1 -- 172.21.15.37:0/3255696311 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f15000acb80 con 0x7f15000d0da0 2024-04-25T10:29:22.884 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:22.880+0000 7f1504e3c640 1 --2- 172.21.15.37:0/3255696311 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f15000d0da0 0x7f150015a2d0 secure :-1 s=READY pgs=140 cs=0 l=1 rev1=1 crypto rx=0x7f14f000d5f0 tx=0x7f14f000dab0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:29:22.884 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:22.880+0000 7f14f5ffb640 1 -- 172.21.15.37:0/3255696311 <== mon.0 v2:172.21.15.37:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f14f00125f0 con 0x7f15000d0da0 2024-04-25T10:29:22.884 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:22.880+0000 7f14f5ffb640 1 -- 172.21.15.37:0/3255696311 <== mon.0 v2:172.21.15.37:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f14f0019070 con 0x7f15000d0da0 2024-04-25T10:29:22.885 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:22.880+0000 7f15078a1640 1 -- 172.21.15.37:0/3255696311 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f15000ace60 con 0x7f15000d0da0 2024-04-25T10:29:22.885 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:22.880+0000 7f14f5ffb640 1 -- 172.21.15.37:0/3255696311 <== mon.0 v2:172.21.15.37:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f14f00129d0 con 0x7f15000d0da0 2024-04-25T10:29:22.885 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:22.881+0000 7f15078a1640 1 -- 172.21.15.37:0/3255696311 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f15000ad420 con 0x7f15000d0da0 2024-04-25T10:29:22.886 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:22.882+0000 7f15078a1640 1 -- 172.21.15.37:0/3255696311 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f150006a140 con 0x7f15000d0da0 2024-04-25T10:29:22.886 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:22.882+0000 7f14f5ffb640 1 -- 172.21.15.37:0/3255696311 <== mon.0 v2:172.21.15.37:3300/0 4 ==== mgrmap(e 5) ==== 96782+0+0 (secure 0 0 0) 0x7f14f0012b70 con 0x7f15000d0da0 2024-04-25T10:29:22.886 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:22.883+0000 7f14f5ffb640 1 --2- 172.21.15.37:0/3255696311 >> [v2:172.21.15.203:6832/2418205237,v1:172.21.15.203:6833/2418205237] conn(0x7f14e40744e0 0x7f14e40769f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:29:22.886 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:22.883+0000 7f14f5ffb640 1 -- 172.21.15.37:0/3255696311 <== mon.0 v2:172.21.15.37:3300/0 5 ==== osd_map(23..23 src has 1..23) ==== 5709+0+0 (secure 0 0 0) 0x7f14f00956f0 con 0x7f15000d0da0 2024-04-25T10:29:22.886 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:22.883+0000 7f150563d640 1 --2- 172.21.15.37:0/3255696311 >> [v2:172.21.15.203:6832/2418205237,v1:172.21.15.203:6833/2418205237] conn(0x7f14e40744e0 0x7f14e40769f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:29:22.886 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:22.884+0000 7f150563d640 1 --2- 172.21.15.37:0/3255696311 >> [v2:172.21.15.203:6832/2418205237,v1:172.21.15.203:6833/2418205237] conn(0x7f14e40744e0 0x7f14e40769f0 secure :-1 s=READY pgs=99 cs=0 l=1 rev1=1 crypto rx=0x7f14fc034000 tx=0x7f14fc02e000 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:29:22.890 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:22.888+0000 7f14f5ffb640 1 -- 172.21.15.37:0/3255696311 <== mon.0 v2:172.21.15.37:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187246 (secure 0 0 0) 0x7f14f005fde0 con 0x7f15000d0da0 2024-04-25T10:29:23.118 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:23.116+0000 7f15078a1640 1 -- 172.21.15.37:0/3255696311 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_command({"prefix": "health", "format": "json"} v 0) -- 0x7f15000d3f50 con 0x7f15000d0da0 2024-04-25T10:29:23.118 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:23.116+0000 7f14f5ffb640 1 -- 172.21.15.37:0/3255696311 <== mon.0 v2:172.21.15.37:3300/0 7 ==== mon_command_ack([{"prefix": "health", "format": "json"}]=0 v0) ==== 72+0+46 (secure 0 0 0) 0x7f14f005f6e0 con 0x7f15000d0da0 2024-04-25T10:29:23.119 INFO:teuthology.orchestra.run.smithi037.stdout: 2024-04-25T10:29:23.119 INFO:teuthology.orchestra.run.smithi037.stdout:{"status":"HEALTH_OK","checks":{},"mutes":[]} 2024-04-25T10:29:23.121 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:23.119+0000 7f15078a1640 1 -- 172.21.15.37:0/3255696311 >> [v2:172.21.15.203:6832/2418205237,v1:172.21.15.203:6833/2418205237] conn(0x7f14e40744e0 msgr2=0x7f14e40769f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:29:23.121 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:23.119+0000 7f15078a1640 1 --2- 172.21.15.37:0/3255696311 >> [v2:172.21.15.203:6832/2418205237,v1:172.21.15.203:6833/2418205237] conn(0x7f14e40744e0 0x7f14e40769f0 secure :-1 s=READY pgs=99 cs=0 l=1 rev1=1 crypto rx=0x7f14fc034000 tx=0x7f14fc02e000 comp rx=0 tx=0).stop 2024-04-25T10:29:23.121 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:23.119+0000 7f15078a1640 1 -- 172.21.15.37:0/3255696311 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f15000d0da0 msgr2=0x7f150015a2d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:29:23.121 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:23.119+0000 7f15078a1640 1 --2- 172.21.15.37:0/3255696311 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f15000d0da0 0x7f150015a2d0 secure :-1 s=READY pgs=140 cs=0 l=1 rev1=1 crypto rx=0x7f14f000d5f0 tx=0x7f14f000dab0 comp rx=0 tx=0).stop 2024-04-25T10:29:23.121 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:23.119+0000 7f15078a1640 1 -- 172.21.15.37:0/3255696311 shutdown_connections 2024-04-25T10:29:23.121 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:23.119+0000 7f15078a1640 1 --2- 172.21.15.37:0/3255696311 >> [v2:172.21.15.203:6832/2418205237,v1:172.21.15.203:6833/2418205237] conn(0x7f14e40744e0 0x7f14e40769f0 unknown :-1 s=CLOSED pgs=99 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:29:23.122 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:23.119+0000 7f15078a1640 1 --2- 172.21.15.37:0/3255696311 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f15000d5160 0x7f150015af20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:29:23.122 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:23.119+0000 7f15078a1640 1 --2- 172.21.15.37:0/3255696311 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f15000d0da0 0x7f150015a2d0 unknown :-1 s=CLOSED pgs=140 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:29:23.122 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:23.119+0000 7f15078a1640 1 --2- 172.21.15.37:0/3255696311 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f15000d0990 0x7f1500159d90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:29:23.122 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:23.119+0000 7f15078a1640 1 -- 172.21.15.37:0/3255696311 >> 172.21.15.37:0/3255696311 conn(0x7f15000b7a80 msgr2=0x7f15000b54d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:29:23.122 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:23.119+0000 7f15078a1640 1 -- 172.21.15.37:0/3255696311 shutdown_connections 2024-04-25T10:29:23.122 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:23.119+0000 7f15078a1640 1 -- 172.21.15.37:0/3255696311 wait complete. 2024-04-25T10:29:23.132 INFO:tasks.ceph.ceph_manager.ceph:wait_until_healthy done 2024-04-25T10:29:23.132 DEBUG:teuthology.orchestra.run.smithi037:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-04-25T10:29:23.243 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:23.241+0000 7f3b3aef9640 1 Processor -- start 2024-04-25T10:29:23.243 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:23.241+0000 7f3b3aef9640 1 -- start start 2024-04-25T10:29:23.245 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:23.242+0000 7f3b3aef9640 1 --2- >> v2:172.21.15.37:3300/0 conn(0x7f3b340d5160 0x7f3b340b4e80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:29:23.245 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:23.242+0000 7f3b3aef9640 1 --2- >> v2:172.21.15.203:3300/0 conn(0x7f3b340d3230 0x7f3b340b9060 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:29:23.245 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:23.242+0000 7f3b3aef9640 1 -- --> v1:172.21.15.37:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f3b340d1370 con 0x7f3b340d0990 2024-04-25T10:29:23.245 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:23.242+0000 7f3b3aef9640 1 -- --> v2:172.21.15.37:3300/0 -- mon_getmap magic: 0 -- 0x7f3b340d1650 con 0x7f3b340d5160 2024-04-25T10:29:23.245 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:23.242+0000 7f3b3aef9640 1 -- --> v2:172.21.15.203:3300/0 -- mon_getmap magic: 0 -- 0x7f3b340d4680 con 0x7f3b340d3230 2024-04-25T10:29:23.245 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:23.242+0000 7f3b38c95640 1 --2- >> v2:172.21.15.37:3300/0 conn(0x7f3b340d5160 0x7f3b340b4e80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:29:23.245 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:23.242+0000 7f3b38c95640 1 --2- >> v2:172.21.15.37:3300/0 conn(0x7f3b340d5160 0x7f3b340b4e80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.37:3300/0 says I am v2:172.21.15.37:35138/0 (socket says 172.21.15.37:35138) 2024-04-25T10:29:23.245 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:23.242+0000 7f3b33fff640 1 --1- >> v1:172.21.15.37:6789/0 conn(0x7f3b340d0990 0x7f3b340d2b20 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.37:6789/0 says I am v1:172.21.15.37:59838/0 (socket says 172.21.15.37:59838) 2024-04-25T10:29:23.245 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:23.242+0000 7f3b38c95640 1 -- 172.21.15.37:0/3898846935 learned_addr learned my addr 172.21.15.37:0/3898846935 (peer_addr_for_me v2:172.21.15.37:0/0) 2024-04-25T10:29:23.245 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:23.243+0000 7f3b39496640 1 --2- 172.21.15.37:0/3898846935 >> v2:172.21.15.203:3300/0 conn(0x7f3b340d3230 0x7f3b340b9060 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:29:23.246 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:23.243+0000 7f3b38c95640 1 -- 172.21.15.37:0/3898846935 >> v2:172.21.15.203:3300/0 conn(0x7f3b340d3230 msgr2=0x7f3b340b9060 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:29:23.246 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:23.243+0000 7f3b38c95640 1 --2- 172.21.15.37:0/3898846935 >> v2:172.21.15.203:3300/0 conn(0x7f3b340d3230 0x7f3b340b9060 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:29:23.246 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:23.243+0000 7f3b38c95640 1 -- 172.21.15.37:0/3898846935 >> v1:172.21.15.37:6789/0 conn(0x7f3b340d0990 legacy=0x7f3b340d2b20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:29:23.246 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:23.243+0000 7f3b38c95640 1 -- 172.21.15.37:0/3898846935 --> v2:172.21.15.37:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f3b340d4960 con 0x7f3b340d5160 2024-04-25T10:29:23.248 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:23.243+0000 7f3b38c95640 1 --2- 172.21.15.37:0/3898846935 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f3b340d5160 0x7f3b340b4e80 secure :-1 s=READY pgs=142 cs=0 l=1 rev1=1 crypto rx=0x7f3b3408f050 tx=0x7f3b2802a150 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d1912bfbdd7c124c server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:29:23.248 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:23.244+0000 7f3b337fe640 1 -- 172.21.15.37:0/3898846935 <== mon.0 v2:172.21.15.37:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3b28037070 con 0x7f3b340d5160 2024-04-25T10:29:23.249 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:23.244+0000 7f3b337fe640 1 -- 172.21.15.37:0/3898846935 <== mon.0 v2:172.21.15.37:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f3b28032440 con 0x7f3b340d5160 2024-04-25T10:29:23.249 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:23.244+0000 7f3b337fe640 1 -- 172.21.15.37:0/3898846935 <== mon.0 v2:172.21.15.37:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3b28032810 con 0x7f3b340d5160 2024-04-25T10:29:23.249 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:23.244+0000 7f3b3aef9640 1 -- 172.21.15.37:0/3898846935 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f3b340d5160 msgr2=0x7f3b340b4e80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:29:23.249 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:23.244+0000 7f3b3aef9640 1 --2- 172.21.15.37:0/3898846935 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f3b340d5160 0x7f3b340b4e80 secure :-1 s=READY pgs=142 cs=0 l=1 rev1=1 crypto rx=0x7f3b3408f050 tx=0x7f3b2802a150 comp rx=0 tx=0).stop 2024-04-25T10:29:23.249 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:23.244+0000 7f3b3aef9640 1 -- 172.21.15.37:0/3898846935 shutdown_connections 2024-04-25T10:29:23.249 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:23.244+0000 7f3b3aef9640 1 --2- 172.21.15.37:0/3898846935 >> v2:172.21.15.203:3300/0 conn(0x7f3b340d3230 0x7f3b340b9060 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:29:23.250 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:23.244+0000 7f3b3aef9640 1 --2- 172.21.15.37:0/3898846935 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f3b340d5160 0x7f3b340b4e80 unknown :-1 s=CLOSED pgs=142 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:29:23.250 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:23.244+0000 7f3b3aef9640 1 -- 172.21.15.37:0/3898846935 >> 172.21.15.37:0/3898846935 conn(0x7f3b340b7a80 msgr2=0x7f3b340b7e90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:29:23.250 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:23.244+0000 7f3b3aef9640 1 -- 172.21.15.37:0/3898846935 shutdown_connections 2024-04-25T10:29:23.250 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:23.244+0000 7f3b3aef9640 1 -- 172.21.15.37:0/3898846935 wait complete. 2024-04-25T10:29:23.250 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:23.246+0000 7f3b3aef9640 1 Processor -- start 2024-04-25T10:29:23.250 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:23.246+0000 7f3b3aef9640 1 -- start start 2024-04-25T10:29:23.253 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:23.247+0000 7f3b3aef9640 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f3b340d0990 0x7f3b34155f10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:29:23.254 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:23.248+0000 7f3b3aef9640 1 --2- >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f3b340d5160 0x7f3b341570c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:29:23.254 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:23.248+0000 7f3b3aef9640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f3b3415b980 0x7f3b34157600 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:29:23.254 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:23.248+0000 7f3b3aef9640 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 -- 0x7f3b34164c40 con 0x7f3b3415b980 2024-04-25T10:29:23.254 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:23.248+0000 7f3b3aef9640 1 -- --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_getmap magic: 0 -- 0x7f3b34164ac0 con 0x7f3b340d0990 2024-04-25T10:29:23.254 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:23.248+0000 7f3b3aef9640 1 -- --> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] -- mon_getmap magic: 0 -- 0x7f3b340d0da0 con 0x7f3b340d5160 2024-04-25T10:29:23.254 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:23.248+0000 7f3b39496640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f3b3415b980 0x7f3b34157600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:29:23.254 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:23.248+0000 7f3b39496640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f3b3415b980 0x7f3b34157600 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.37:3300/0 says I am v2:172.21.15.37:35144/0 (socket says 172.21.15.37:35144) 2024-04-25T10:29:23.254 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:23.248+0000 7f3b39496640 1 -- 172.21.15.37:0/950323587 learned_addr learned my addr 172.21.15.37:0/950323587 (peer_addr_for_me v2:172.21.15.37:0/0) 2024-04-25T10:29:23.254 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:23.248+0000 7f3b33fff640 1 --2- 172.21.15.37:0/950323587 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f3b340d5160 0x7f3b341570c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:29:23.254 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:23.248+0000 7f3b38c95640 1 --2- 172.21.15.37:0/950323587 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f3b340d0990 0x7f3b34155f10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:29:23.254 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:23.248+0000 7f3b39496640 1 -- 172.21.15.37:0/950323587 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f3b340d5160 msgr2=0x7f3b341570c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:29:23.254 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:23.248+0000 7f3b39496640 1 --2- 172.21.15.37:0/950323587 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f3b340d5160 0x7f3b341570c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:29:23.255 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:23.248+0000 7f3b39496640 1 -- 172.21.15.37:0/950323587 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f3b340d0990 msgr2=0x7f3b34155f10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:29:23.255 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:23.248+0000 7f3b39496640 1 --2- 172.21.15.37:0/950323587 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f3b340d0990 0x7f3b34155f10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:29:23.255 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:23.248+0000 7f3b39496640 1 -- 172.21.15.37:0/950323587 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f3b341f7470 con 0x7f3b3415b980 2024-04-25T10:29:23.255 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:23.248+0000 7f3b38c95640 1 --2- 172.21.15.37:0/950323587 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f3b340d0990 0x7f3b34155f10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T10:29:23.255 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:23.248+0000 7f3b33fff640 1 --2- 172.21.15.37:0/950323587 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f3b340d5160 0x7f3b341570c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T10:29:23.255 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:23.248+0000 7f3b39496640 1 --2- 172.21.15.37:0/950323587 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f3b3415b980 0x7f3b34157600 secure :-1 s=READY pgs=143 cs=0 l=1 rev1=1 crypto rx=0x7f3b2c007e90 tx=0x7f3b2c00e4b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:29:23.255 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:23.248+0000 7f3b317fa640 1 -- 172.21.15.37:0/950323587 <== mon.0 v2:172.21.15.37:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3b2c014420 con 0x7f3b3415b980 2024-04-25T10:29:23.255 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:23.248+0000 7f3b317fa640 1 -- 172.21.15.37:0/950323587 <== mon.0 v2:172.21.15.37:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f3b2c019070 con 0x7f3b3415b980 2024-04-25T10:29:23.256 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:23.248+0000 7f3b3aef9640 1 -- 172.21.15.37:0/950323587 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f3b341f7750 con 0x7f3b3415b980 2024-04-25T10:29:23.256 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:23.248+0000 7f3b317fa640 1 -- 172.21.15.37:0/950323587 <== mon.0 v2:172.21.15.37:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3b2c014820 con 0x7f3b3415b980 2024-04-25T10:29:23.256 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:23.248+0000 7f3b3aef9640 1 -- 172.21.15.37:0/950323587 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f3b34156450 con 0x7f3b3415b980 2024-04-25T10:29:23.256 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:23.249+0000 7f3b3aef9640 1 -- 172.21.15.37:0/950323587 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f3b3406a140 con 0x7f3b3415b980 2024-04-25T10:29:23.256 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:23.250+0000 7f3b317fa640 1 -- 172.21.15.37:0/950323587 <== mon.0 v2:172.21.15.37:3300/0 4 ==== mgrmap(e 5) ==== 96782+0+0 (secure 0 0 0) 0x7f3b2c0149c0 con 0x7f3b3415b980 2024-04-25T10:29:23.256 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:23.250+0000 7f3b317fa640 1 --2- 172.21.15.37:0/950323587 >> [v2:172.21.15.203:6832/2418205237,v1:172.21.15.203:6833/2418205237] conn(0x7f3b180744e0 0x7f3b180769f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:29:23.256 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:23.250+0000 7f3b317fa640 1 -- 172.21.15.37:0/950323587 <== mon.0 v2:172.21.15.37:3300/0 5 ==== osd_map(23..23 src has 1..23) ==== 5709+0+0 (secure 0 0 0) 0x7f3b2c095840 con 0x7f3b3415b980 2024-04-25T10:29:23.256 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:23.251+0000 7f3b38c95640 1 --2- 172.21.15.37:0/950323587 >> [v2:172.21.15.203:6832/2418205237,v1:172.21.15.203:6833/2418205237] conn(0x7f3b180744e0 0x7f3b180769f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:29:23.256 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:23.251+0000 7f3b38c95640 1 --2- 172.21.15.37:0/950323587 >> [v2:172.21.15.203:6832/2418205237,v1:172.21.15.203:6833/2418205237] conn(0x7f3b180744e0 0x7f3b180769f0 secure :-1 s=READY pgs=100 cs=0 l=1 rev1=1 crypto rx=0x7f3b28034000 tx=0x7f3b2802e000 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:29:23.257 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:23.255+0000 7f3b317fa640 1 -- 172.21.15.37:0/950323587 <== mon.0 v2:172.21.15.37:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187246 (secure 0 0 0) 0x7f3b2c0600f0 con 0x7f3b3415b980 2024-04-25T10:29:23.443 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:23.441+0000 7f3b3aef9640 1 -- 172.21.15.37:0/950323587 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7f3b340d3f50 con 0x7f3b3415b980 2024-04-25T10:29:23.444 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:23.442+0000 7f3b317fa640 1 -- 172.21.15.37:0/950323587 <== mon.0 v2:172.21.15.37:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v23) ==== 74+0+16621 (secure 0 0 0) 0x7f3b2c05f9f0 con 0x7f3b3415b980 2024-04-25T10:29:23.444 INFO:teuthology.orchestra.run.smithi037.stdout: 2024-04-25T10:29:23.444 INFO:teuthology.orchestra.run.smithi037.stdout:{"epoch":23,"fsid":"f4acda28-be34-492a-b286-994afd36f61d","created":"2024-04-25T10:28:43.985718+0000","modified":"2024-04-25T10:29:10.016188+0000","last_up_change":"2024-04-25T10:28:54.900654+0000","last_in_change":"2024-04-25T10:28:46.216973+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":3,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-04-25T10:28:56.202153+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"19","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2024-04-25T10:29:06.507244+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"22","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":2.25,"score_stable":2.25,"optimal_score":1,"raw_score_acting":2.25,"raw_score_stable":2.25,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2024-04-25T10:29:07.347844+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"23","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"a2f79028-b332-4166-9ebb-a7024a37ea22","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":21,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.37:6816","nonce":2746074720},{"type":"v1","addr":"172.21.15.37:6817","nonce":2746074720}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.37:6818","nonce":2746074720},{"type":"v1","addr":"172.21.15.37:6819","nonce":2746074720}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.37:6822","nonce":2746074720},{"type":"v1","addr":"172.21.15.37:6823","nonce":2746074720}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.37:6820","nonce":2746074720},{"type":"v1","addr":"172.21.15.37:6821","nonce":2746074720}]},"public_addr":"172.21.15.37:6817/2746074720","cluster_addr":"172.21.15.37:6819/2746074720","heartbeat_back_addr":"172.21.15.37:6823/2746074720","heartbeat_front_addr":"172.21.15.37:6821/2746074720","state":["exists","up"]},{"osd":1,"uuid":"973159de-c2ea-4083-aecc-69373d1c1132","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":21,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.37:6800","nonce":1690341662},{"type":"v1","addr":"172.21.15.37:6801","nonce":1690341662}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.37:6802","nonce":1690341662},{"type":"v1","addr":"172.21.15.37:6803","nonce":1690341662}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.37:6806","nonce":1690341662},{"type":"v1","addr":"172.21.15.37:6807","nonce":1690341662}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.37:6804","nonce":1690341662},{"type":"v1","addr":"172.21.15.37:6805","nonce":1690341662}]},"public_addr":"172.21.15.37:6801/1690341662","cluster_addr":"172.21.15.37:6803/1690341662","heartbeat_back_addr":"172.21.15.37:6807/1690341662","heartbeat_front_addr":"172.21.15.37:6805/1690341662","state":["exists","up"]},{"osd":2,"uuid":"4ae7a0e9-cb3f-484c-a75f-f3766b721c47","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":21,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.37:6808","nonce":2754669649},{"type":"v1","addr":"172.21.15.37:6809","nonce":2754669649}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.37:6810","nonce":2754669649},{"type":"v1","addr":"172.21.15.37:6811","nonce":2754669649}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.37:6814","nonce":2754669649},{"type":"v1","addr":"172.21.15.37:6815","nonce":2754669649}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.37:6812","nonce":2754669649},{"type":"v1","addr":"172.21.15.37:6813","nonce":2754669649}]},"public_addr":"172.21.15.37:6809/2754669649","cluster_addr":"172.21.15.37:6811/2754669649","heartbeat_back_addr":"172.21.15.37:6815/2754669649","heartbeat_front_addr":"172.21.15.37:6813/2754669649","state":["exists","up"]},{"osd":3,"uuid":"1ba4a746-1608-47a7-ace2-960a5cfae934","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":21,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.37:6824","nonce":1815290618},{"type":"v1","addr":"172.21.15.37:6825","nonce":1815290618}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.37:6826","nonce":1815290618},{"type":"v1","addr":"172.21.15.37:6827","nonce":1815290618}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.37:6830","nonce":1815290618},{"type":"v1","addr":"172.21.15.37:6831","nonce":1815290618}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.37:6828","nonce":1815290618},{"type":"v1","addr":"172.21.15.37:6829","nonce":1815290618}]},"public_addr":"172.21.15.37:6825/1815290618","cluster_addr":"172.21.15.37:6827/1815290618","heartbeat_back_addr":"172.21.15.37:6831/1815290618","heartbeat_front_addr":"172.21.15.37:6829/1815290618","state":["exists","up"]},{"osd":4,"uuid":"d4f59f5d-0413-4b79-87ee-bb212c1fcf62","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":21,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6800","nonce":4051532063},{"type":"v1","addr":"172.21.15.203:6801","nonce":4051532063}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6802","nonce":4051532063},{"type":"v1","addr":"172.21.15.203:6803","nonce":4051532063}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6806","nonce":4051532063},{"type":"v1","addr":"172.21.15.203:6807","nonce":4051532063}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6804","nonce":4051532063},{"type":"v1","addr":"172.21.15.203:6805","nonce":4051532063}]},"public_addr":"172.21.15.203:6801/4051532063","cluster_addr":"172.21.15.203:6803/4051532063","heartbeat_back_addr":"172.21.15.203:6807/4051532063","heartbeat_front_addr":"172.21.15.203:6805/4051532063","state":["exists","up"]},{"osd":5,"uuid":"69710a0e-41d8-4e7f-9a10-17ce540b67be","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":21,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6824","nonce":1203822890},{"type":"v1","addr":"172.21.15.203:6825","nonce":1203822890}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6826","nonce":1203822890},{"type":"v1","addr":"172.21.15.203:6827","nonce":1203822890}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6830","nonce":1203822890},{"type":"v1","addr":"172.21.15.203:6831","nonce":1203822890}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6828","nonce":1203822890},{"type":"v1","addr":"172.21.15.203:6829","nonce":1203822890}]},"public_addr":"172.21.15.203:6825/1203822890","cluster_addr":"172.21.15.203:6827/1203822890","heartbeat_back_addr":"172.21.15.203:6831/1203822890","heartbeat_front_addr":"172.21.15.203:6829/1203822890","state":["exists","up"]},{"osd":6,"uuid":"f29dd047-3b48-473d-a235-6642a6293f20","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":21,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6810","nonce":2309615014},{"type":"v1","addr":"172.21.15.203:6812","nonce":2309615014}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6814","nonce":2309615014},{"type":"v1","addr":"172.21.15.203:6816","nonce":2309615014}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6822","nonce":2309615014},{"type":"v1","addr":"172.21.15.203:6823","nonce":2309615014}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6818","nonce":2309615014},{"type":"v1","addr":"172.21.15.203:6820","nonce":2309615014}]},"public_addr":"172.21.15.203:6812/2309615014","cluster_addr":"172.21.15.203:6816/2309615014","heartbeat_back_addr":"172.21.15.203:6823/2309615014","heartbeat_front_addr":"172.21.15.203:6820/2309615014","state":["exists","up"]},{"osd":7,"uuid":"6e691644-378b-4fdc-8e86-fb5f56e31f29","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":21,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6808","nonce":3688099014},{"type":"v1","addr":"172.21.15.203:6809","nonce":3688099014}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6811","nonce":3688099014},{"type":"v1","addr":"172.21.15.203:6813","nonce":3688099014}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6819","nonce":3688099014},{"type":"v1","addr":"172.21.15.203:6821","nonce":3688099014}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6815","nonce":3688099014},{"type":"v1","addr":"172.21.15.203:6817","nonce":3688099014}]},"public_addr":"172.21.15.203:6809/3688099014","cluster_addr":"172.21.15.203:6813/3688099014","heartbeat_back_addr":"172.21.15.203:6821/3688099014","heartbeat_front_addr":"172.21.15.203:6817/3688099014","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T10:28:51.419141+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T10:28:51.366448+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T10:28:51.348276+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T10:28:51.380519+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T10:28:51.105812+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T10:28:51.384496+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T10:28:51.388755+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T10:28:51.366958+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-04-25T10:29:23.446 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:23.444+0000 7f3b3aef9640 1 -- 172.21.15.37:0/950323587 >> [v2:172.21.15.203:6832/2418205237,v1:172.21.15.203:6833/2418205237] conn(0x7f3b180744e0 msgr2=0x7f3b180769f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:29:23.446 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:23.444+0000 7f3b3aef9640 1 --2- 172.21.15.37:0/950323587 >> [v2:172.21.15.203:6832/2418205237,v1:172.21.15.203:6833/2418205237] conn(0x7f3b180744e0 0x7f3b180769f0 secure :-1 s=READY pgs=100 cs=0 l=1 rev1=1 crypto rx=0x7f3b28034000 tx=0x7f3b2802e000 comp rx=0 tx=0).stop 2024-04-25T10:29:23.446 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:23.444+0000 7f3b3aef9640 1 -- 172.21.15.37:0/950323587 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f3b3415b980 msgr2=0x7f3b34157600 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:29:23.446 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:23.444+0000 7f3b3aef9640 1 --2- 172.21.15.37:0/950323587 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f3b3415b980 0x7f3b34157600 secure :-1 s=READY pgs=143 cs=0 l=1 rev1=1 crypto rx=0x7f3b2c007e90 tx=0x7f3b2c00e4b0 comp rx=0 tx=0).stop 2024-04-25T10:29:23.447 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:23.444+0000 7f3b3aef9640 1 -- 172.21.15.37:0/950323587 shutdown_connections 2024-04-25T10:29:23.447 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:23.444+0000 7f3b3aef9640 1 --2- 172.21.15.37:0/950323587 >> [v2:172.21.15.203:6832/2418205237,v1:172.21.15.203:6833/2418205237] conn(0x7f3b180744e0 0x7f3b180769f0 unknown :-1 s=CLOSED pgs=100 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:29:23.447 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:23.444+0000 7f3b3aef9640 1 --2- 172.21.15.37:0/950323587 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f3b3415b980 0x7f3b34157600 unknown :-1 s=CLOSED pgs=143 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:29:23.447 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:23.444+0000 7f3b3aef9640 1 --2- 172.21.15.37:0/950323587 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f3b340d5160 0x7f3b341570c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:29:23.447 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:23.444+0000 7f3b3aef9640 1 --2- 172.21.15.37:0/950323587 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f3b340d0990 0x7f3b34155f10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:29:23.447 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:23.444+0000 7f3b3aef9640 1 -- 172.21.15.37:0/950323587 >> 172.21.15.37:0/950323587 conn(0x7f3b340b7a80 msgr2=0x7f3b34160890 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:29:23.447 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:23.444+0000 7f3b3aef9640 1 -- 172.21.15.37:0/950323587 shutdown_connections 2024-04-25T10:29:23.447 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:23.444+0000 7f3b3aef9640 1 -- 172.21.15.37:0/950323587 wait complete. 2024-04-25T10:29:23.457 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2024-04-25T10:28:56.202153+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '19', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 8, 'score_stable': 8, 'optimal_score': 0.25, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 2, 'pool_name': 'cephfs_metadata', 'create_time': '2024-04-25T10:29:06.507244+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '22', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_autoscale_bias': 4, 'pg_num_min': 16, 'recovery_priority': 5}, 'application_metadata': {'cephfs': {'metadata': 'cephfs'}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 2.25, 'score_stable': 2.25, 'optimal_score': 1, 'raw_score_acting': 2.25, 'raw_score_stable': 2.25, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 3, 'pool_name': 'cephfs_data', 'create_time': '2024-04-25T10:29:07.347844+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 64, 'pg_placement_num': 64, 'pg_placement_num_target': 64, 'pg_num_target': 64, 'pg_num_pending': 64, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '23', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.8999999761581421}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 1.5, 'score_stable': 1.5, 'optimal_score': 1, 'raw_score_acting': 1.5, 'raw_score_stable': 1.5, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}] 2024-04-25T10:29:23.458 DEBUG:teuthology.orchestra.run.smithi037:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2024-04-25T10:29:23.572 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:23.570+0000 7f29d19b2640 1 Processor -- start 2024-04-25T10:29:23.572 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:23.570+0000 7f29d19b2640 1 -- start start 2024-04-25T10:29:23.574 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:23.571+0000 7f29d19b2640 1 --2- >> v2:172.21.15.203:3300/0 conn(0x7f29cc08cc90 0x7f29cc06e0a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:29:23.574 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:23.571+0000 7f29d19b2640 1 -- --> v1:172.21.15.37:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f29cc165610 con 0x7f29cc06e9f0 2024-04-25T10:29:23.574 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:23.571+0000 7f29d19b2640 1 -- --> v1:172.21.15.203:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f29cc0d2ed0 con 0x7f29cc06e5e0 2024-04-25T10:29:23.574 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:23.571+0000 7f29d19b2640 1 -- --> v2:172.21.15.203:3300/0 -- mon_getmap magic: 0 -- 0x7f29cc06ee00 con 0x7f29cc08cc90 2024-04-25T10:29:23.574 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:23.571+0000 7f29cb7fe640 1 --1- >> v1:172.21.15.37:6789/0 conn(0x7f29cc06e9f0 0x7f29cc164f00 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.37:6789/0 says I am v1:172.21.15.37:59852/0 (socket says 172.21.15.37:59852) 2024-04-25T10:29:23.574 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:23.571+0000 7f29cb7fe640 1 -- 172.21.15.37:0/3727479700 learned_addr learned my addr 172.21.15.37:0/3727479700 (peer_addr_for_me v1:172.21.15.37:0/0) 2024-04-25T10:29:23.574 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:23.571+0000 7f29caffd640 1 --2- 172.21.15.37:0/3727479700 >> v2:172.21.15.203:3300/0 conn(0x7f29cc08cc90 0x7f29cc06e0a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:29:23.574 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:23.571+0000 7f29c9ffb640 1 -- 172.21.15.37:0/3727479700 <== mon.0 v1:172.21.15.37:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 1310149476 0 0) 0x7f29cc165610 con 0x7f29cc06e9f0 2024-04-25T10:29:23.574 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:23.571+0000 7f29c9ffb640 1 -- 172.21.15.37:0/3727479700 --> v1:172.21.15.37:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f29b8003520 con 0x7f29cc06e9f0 2024-04-25T10:29:23.574 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:23.572+0000 7f29caffd640 1 -- 172.21.15.37:0/3727479700 >> v1:172.21.15.203:6789/0 conn(0x7f29cc06e5e0 legacy=0x7f29cc1647f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:29:23.575 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:23.572+0000 7f29caffd640 1 -- 172.21.15.37:0/3727479700 >> v1:172.21.15.37:6789/0 conn(0x7f29cc06e9f0 legacy=0x7f29cc164f00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:29:23.575 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:23.572+0000 7f29caffd640 1 -- 172.21.15.37:0/3727479700 --> v2:172.21.15.203:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f29cc09cd40 con 0x7f29cc08cc90 2024-04-25T10:29:23.576 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:23.572+0000 7f29caffd640 1 --2- 172.21.15.37:0/3727479700 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f29cc08cc90 0x7f29cc06e0a0 secure :-1 s=READY pgs=96 cs=0 l=1 rev1=1 crypto rx=0x7f29c40097a0 tx=0x7f29c402a230 comp rx=0 tx=0).ready entity=mon.1 client_cookie=3c05b77b468bfd11 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:29:23.576 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:23.573+0000 7f29c9ffb640 1 -- 172.21.15.37:0/3727479700 <== mon.1 v2:172.21.15.203:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f29c4037070 con 0x7f29cc08cc90 2024-04-25T10:29:23.576 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:23.573+0000 7f29c9ffb640 1 -- 172.21.15.37:0/3727479700 <== mon.1 v2:172.21.15.203:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f29c4032440 con 0x7f29cc08cc90 2024-04-25T10:29:23.576 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:23.573+0000 7f29c9ffb640 1 -- 172.21.15.37:0/3727479700 <== mon.1 v2:172.21.15.203:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f29c40327f0 con 0x7f29cc08cc90 2024-04-25T10:29:23.576 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:23.573+0000 7f29d19b2640 1 -- 172.21.15.37:0/3727479700 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f29cc08cc90 msgr2=0x7f29cc06e0a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:29:23.576 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:23.573+0000 7f29d19b2640 1 --2- 172.21.15.37:0/3727479700 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f29cc08cc90 0x7f29cc06e0a0 secure :-1 s=READY pgs=96 cs=0 l=1 rev1=1 crypto rx=0x7f29c40097a0 tx=0x7f29c402a230 comp rx=0 tx=0).stop 2024-04-25T10:29:23.576 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:23.573+0000 7f29d19b2640 1 -- 172.21.15.37:0/3727479700 shutdown_connections 2024-04-25T10:29:23.576 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:23.573+0000 7f29d19b2640 1 --2- 172.21.15.37:0/3727479700 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f29cc08cc90 0x7f29cc06e0a0 unknown :-1 s=CLOSED pgs=96 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:29:23.576 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:23.573+0000 7f29d19b2640 1 -- 172.21.15.37:0/3727479700 >> 172.21.15.37:0/3727479700 conn(0x7f29cc0d03d0 msgr2=0x7f29cc0d07e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:29:23.576 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:23.573+0000 7f29d19b2640 1 -- 172.21.15.37:0/3727479700 shutdown_connections 2024-04-25T10:29:23.577 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:23.573+0000 7f29d19b2640 1 -- 172.21.15.37:0/3727479700 wait complete. 2024-04-25T10:29:23.577 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:23.575+0000 7f29d19b2640 1 Processor -- start 2024-04-25T10:29:23.577 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:23.575+0000 7f29d19b2640 1 -- start start 2024-04-25T10:29:23.580 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:23.577+0000 7f29d19b2640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f29cc06e5e0 0x7f29cc1638b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:29:23.581 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:23.577+0000 7f29d19b2640 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f29cc06e9f0 0x7f29cc152ce0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:29:23.581 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:23.577+0000 7f29d19b2640 1 --2- >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f29cc071f80 0x7f29cc072390 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:29:23.581 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:23.577+0000 7f29d19b2640 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 -- 0x7f29cc09d000 con 0x7f29cc06e5e0 2024-04-25T10:29:23.581 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:23.577+0000 7f29d19b2640 1 -- --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_getmap magic: 0 -- 0x7f29cc0d3200 con 0x7f29cc06e9f0 2024-04-25T10:29:23.581 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:23.577+0000 7f29d19b2640 1 -- --> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] -- mon_getmap magic: 0 -- 0x7f29cc0cf970 con 0x7f29cc071f80 2024-04-25T10:29:23.581 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:23.577+0000 7f29caffd640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f29cc06e5e0 0x7f29cc1638b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:29:23.581 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:23.577+0000 7f29caffd640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f29cc06e5e0 0x7f29cc1638b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.37:3300/0 says I am v2:172.21.15.37:35156/0 (socket says 172.21.15.37:35156) 2024-04-25T10:29:23.581 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:23.577+0000 7f29caffd640 1 -- 172.21.15.37:0/2920289928 learned_addr learned my addr 172.21.15.37:0/2920289928 (peer_addr_for_me v2:172.21.15.37:0/0) 2024-04-25T10:29:23.581 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:23.577+0000 7f29cb7fe640 1 --2- 172.21.15.37:0/2920289928 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f29cc071f80 0x7f29cc072390 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:29:23.581 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:23.577+0000 7f29ca7fc640 1 --2- 172.21.15.37:0/2920289928 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f29cc06e9f0 0x7f29cc152ce0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:29:23.581 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:23.577+0000 7f29caffd640 1 -- 172.21.15.37:0/2920289928 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f29cc071f80 msgr2=0x7f29cc072390 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:29:23.581 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:23.577+0000 7f29caffd640 1 --2- 172.21.15.37:0/2920289928 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f29cc071f80 0x7f29cc072390 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:29:23.581 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:23.577+0000 7f29caffd640 1 -- 172.21.15.37:0/2920289928 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f29cc06e9f0 msgr2=0x7f29cc152ce0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:29:23.581 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:23.577+0000 7f29caffd640 1 --2- 172.21.15.37:0/2920289928 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f29cc06e9f0 0x7f29cc152ce0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:29:23.582 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:23.577+0000 7f29caffd640 1 -- 172.21.15.37:0/2920289928 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f29cc0728d0 con 0x7f29cc06e5e0 2024-04-25T10:29:23.582 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:23.577+0000 7f29caffd640 1 --2- 172.21.15.37:0/2920289928 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f29cc06e5e0 0x7f29cc1638b0 secure :-1 s=READY pgs=145 cs=0 l=1 rev1=1 crypto rx=0x7f29c40097a0 tx=0x7f29c40046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:29:23.582 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:23.577+0000 7f29c3fff640 1 -- 172.21.15.37:0/2920289928 <== mon.0 v2:172.21.15.37:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f29c4037050 con 0x7f29cc06e5e0 2024-04-25T10:29:23.582 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:23.577+0000 7f29c3fff640 1 -- 172.21.15.37:0/2920289928 <== mon.0 v2:172.21.15.37:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f29c4043070 con 0x7f29cc06e5e0 2024-04-25T10:29:23.582 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:23.577+0000 7f29d19b2640 1 -- 172.21.15.37:0/2920289928 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f29cc0c93b0 con 0x7f29cc06e5e0 2024-04-25T10:29:23.582 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:23.577+0000 7f29c3fff640 1 -- 172.21.15.37:0/2920289928 <== mon.0 v2:172.21.15.37:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f29c4032810 con 0x7f29cc06e5e0 2024-04-25T10:29:23.582 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:23.578+0000 7f29d19b2640 1 -- 172.21.15.37:0/2920289928 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f29cc0c9970 con 0x7f29cc06e5e0 2024-04-25T10:29:23.582 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:23.578+0000 7f29d19b2640 1 -- 172.21.15.37:0/2920289928 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f29cc06a140 con 0x7f29cc06e5e0 2024-04-25T10:29:23.583 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:23.579+0000 7f29c3fff640 1 -- 172.21.15.37:0/2920289928 <== mon.0 v2:172.21.15.37:3300/0 4 ==== mgrmap(e 5) ==== 96782+0+0 (secure 0 0 0) 0x7f29c40329b0 con 0x7f29cc06e5e0 2024-04-25T10:29:23.583 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:23.579+0000 7f29c3fff640 1 --2- 172.21.15.37:0/2920289928 >> [v2:172.21.15.203:6832/2418205237,v1:172.21.15.203:6833/2418205237] conn(0x7f29ac0744b0 0x7f29ac0769c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:29:23.583 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:23.580+0000 7f29c3fff640 1 -- 172.21.15.37:0/2920289928 <== mon.0 v2:172.21.15.37:3300/0 5 ==== osd_map(23..23 src has 1..23) ==== 5709+0+0 (secure 0 0 0) 0x7f29c40b5bc0 con 0x7f29cc06e5e0 2024-04-25T10:29:23.583 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:23.580+0000 7f29ca7fc640 1 --2- 172.21.15.37:0/2920289928 >> [v2:172.21.15.203:6832/2418205237,v1:172.21.15.203:6833/2418205237] conn(0x7f29ac0744b0 0x7f29ac0769c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:29:23.583 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:23.580+0000 7f29ca7fc640 1 --2- 172.21.15.37:0/2920289928 >> [v2:172.21.15.203:6832/2418205237,v1:172.21.15.203:6833/2418205237] conn(0x7f29ac0744b0 0x7f29ac0769c0 secure :-1 s=READY pgs=101 cs=0 l=1 rev1=1 crypto rx=0x7f29cc076110 tx=0x7f29b4009040 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:29:23.586 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:23.584+0000 7f29c3fff640 1 -- 172.21.15.37:0/2920289928 <== mon.0 v2:172.21.15.37:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187246 (secure 0 0 0) 0x7f29cc0728d0 con 0x7f29cc06e5e0 2024-04-25T10:29:23.771 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:23.768+0000 7f29d19b2640 1 -- 172.21.15.37:0/2920289928 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) -- 0x7f29cc0b9480 con 0x7f29cc06e5e0 2024-04-25T10:29:23.771 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:23.769+0000 7f29c3fff640 1 -- 172.21.15.37:0/2920289928 <== mon.0 v2:172.21.15.37:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v23) ==== 93+0+10 (secure 0 0 0) 0x7f29c4080340 con 0x7f29cc06e5e0 2024-04-25T10:29:23.771 INFO:teuthology.orchestra.run.smithi037.stdout:pg_num: 1 2024-04-25T10:29:23.773 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:23.771+0000 7f29d19b2640 1 -- 172.21.15.37:0/2920289928 >> [v2:172.21.15.203:6832/2418205237,v1:172.21.15.203:6833/2418205237] conn(0x7f29ac0744b0 msgr2=0x7f29ac0769c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:29:23.773 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:23.771+0000 7f29d19b2640 1 --2- 172.21.15.37:0/2920289928 >> [v2:172.21.15.203:6832/2418205237,v1:172.21.15.203:6833/2418205237] conn(0x7f29ac0744b0 0x7f29ac0769c0 secure :-1 s=READY pgs=101 cs=0 l=1 rev1=1 crypto rx=0x7f29cc076110 tx=0x7f29b4009040 comp rx=0 tx=0).stop 2024-04-25T10:29:23.773 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:23.771+0000 7f29d19b2640 1 -- 172.21.15.37:0/2920289928 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f29cc06e5e0 msgr2=0x7f29cc1638b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:29:23.774 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:23.771+0000 7f29d19b2640 1 --2- 172.21.15.37:0/2920289928 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f29cc06e5e0 0x7f29cc1638b0 secure :-1 s=READY pgs=145 cs=0 l=1 rev1=1 crypto rx=0x7f29c40097a0 tx=0x7f29c40046c0 comp rx=0 tx=0).stop 2024-04-25T10:29:23.774 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:23.771+0000 7f29d19b2640 1 -- 172.21.15.37:0/2920289928 shutdown_connections 2024-04-25T10:29:23.774 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:23.771+0000 7f29d19b2640 1 --2- 172.21.15.37:0/2920289928 >> [v2:172.21.15.203:6832/2418205237,v1:172.21.15.203:6833/2418205237] conn(0x7f29ac0744b0 0x7f29ac0769c0 unknown :-1 s=CLOSED pgs=101 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:29:23.774 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:23.771+0000 7f29d19b2640 1 --2- 172.21.15.37:0/2920289928 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f29cc071f80 0x7f29cc072390 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:29:23.774 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:23.771+0000 7f29d19b2640 1 --2- 172.21.15.37:0/2920289928 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f29cc06e9f0 0x7f29cc152ce0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:29:23.774 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:23.771+0000 7f29d19b2640 1 --2- 172.21.15.37:0/2920289928 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f29cc06e5e0 0x7f29cc1638b0 unknown :-1 s=CLOSED pgs=145 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:29:23.774 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:23.771+0000 7f29d19b2640 1 -- 172.21.15.37:0/2920289928 >> 172.21.15.37:0/2920289928 conn(0x7f29cc0d03d0 msgr2=0x7f29cc1640b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:29:23.774 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:23.771+0000 7f29d19b2640 1 -- 172.21.15.37:0/2920289928 shutdown_connections 2024-04-25T10:29:23.774 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:23.771+0000 7f29d19b2640 1 -- 172.21.15.37:0/2920289928 wait complete. 2024-04-25T10:29:23.784 DEBUG:teuthology.orchestra.run.smithi037:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_metadata pg_num 2024-04-25T10:29:23.898 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:23.896+0000 7f2fd9e84640 1 Processor -- start 2024-04-25T10:29:23.898 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:23.896+0000 7f2fd9e84640 1 -- start start 2024-04-25T10:29:23.900 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:23.897+0000 7f2fd9e84640 1 --2- >> v2:172.21.15.37:3300/0 conn(0x7f2fd40b4bb0 0x7f2fd41646b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:29:23.900 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:23.897+0000 7f2fd9e84640 1 --2- >> v2:172.21.15.203:3300/0 conn(0x7f2fd40b7600 0x7f2fd40b7a10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:29:23.900 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:23.897+0000 7f2fd9e84640 1 --2- >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f2fd40d4560 0x7f2fd40d4970 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:29:23.900 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:23.897+0000 7f2fd9e84640 1 -- --> v2:172.21.15.37:3300/0 -- mon_getmap magic: 0 -- 0x7f2fd40b7f50 con 0x7f2fd40b4bb0 2024-04-25T10:29:23.900 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:23.897+0000 7f2fd9e84640 1 -- --> v2:172.21.15.203:3300/0 -- mon_getmap magic: 0 -- 0x7f2fd4161190 con 0x7f2fd40b7600 2024-04-25T10:29:23.900 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:23.897+0000 7f2fd9e84640 1 -- --> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] -- mon_getmap magic: 0 -- 0x7f2fd4161470 con 0x7f2fd40d4560 2024-04-25T10:29:23.900 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:23.897+0000 7f2fd37fe640 1 --2- >> v2:172.21.15.37:3300/0 conn(0x7f2fd40b4bb0 0x7f2fd41646b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:29:23.901 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:23.897+0000 7f2fd37fe640 1 --2- >> v2:172.21.15.37:3300/0 conn(0x7f2fd40b4bb0 0x7f2fd41646b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.37:3300/0 says I am v2:172.21.15.37:35166/0 (socket says 172.21.15.37:35166) 2024-04-25T10:29:23.901 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:23.897+0000 7f2fd37fe640 1 -- 172.21.15.37:0/2490201482 learned_addr learned my addr 172.21.15.37:0/2490201482 (peer_addr_for_me v2:172.21.15.37:0/0) 2024-04-25T10:29:23.901 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:23.898+0000 7f2fd3fff640 1 --2- 172.21.15.37:0/2490201482 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f2fd40d4560 0x7f2fd40d4970 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:29:23.901 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:23.898+0000 7f2fd2ffd640 1 --2- 172.21.15.37:0/2490201482 >> v2:172.21.15.203:3300/0 conn(0x7f2fd40b7600 0x7f2fd40b7a10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:29:23.901 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:23.898+0000 7f2fd37fe640 1 -- 172.21.15.37:0/2490201482 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f2fd40d4560 msgr2=0x7f2fd40d4970 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:29:23.901 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:23.898+0000 7f2fd37fe640 1 --2- 172.21.15.37:0/2490201482 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f2fd40d4560 0x7f2fd40d4970 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:29:23.901 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:23.898+0000 7f2fd37fe640 1 -- 172.21.15.37:0/2490201482 >> v2:172.21.15.203:3300/0 conn(0x7f2fd40b7600 msgr2=0x7f2fd40b7a10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:29:23.901 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:23.898+0000 7f2fd37fe640 1 --2- 172.21.15.37:0/2490201482 >> v2:172.21.15.203:3300/0 conn(0x7f2fd40b7600 0x7f2fd40b7a10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:29:23.901 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:23.898+0000 7f2fd37fe640 1 -- 172.21.15.37:0/2490201482 --> v2:172.21.15.37:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f2fd4163eb0 con 0x7f2fd40b4bb0 2024-04-25T10:29:23.901 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:23.898+0000 7f2fd3fff640 1 --2- 172.21.15.37:0/2490201482 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f2fd40d4560 0x7f2fd40d4970 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T10:29:23.901 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:23.898+0000 7f2fd37fe640 1 --2- 172.21.15.37:0/2490201482 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f2fd40b4bb0 0x7f2fd41646b0 secure :-1 s=READY pgs=146 cs=0 l=1 rev1=1 crypto rx=0x7f2fd40ccbe0 tx=0x7f2fc402a1e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=4ada6f4b6a1921b4 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:29:23.902 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:23.899+0000 7f2fd27fc640 1 -- 172.21.15.37:0/2490201482 <== mon.0 v2:172.21.15.37:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2fc4037070 con 0x7f2fd40b4bb0 2024-04-25T10:29:23.903 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:23.899+0000 7f2fd27fc640 1 -- 172.21.15.37:0/2490201482 <== mon.0 v2:172.21.15.37:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f2fc4032440 con 0x7f2fd40b4bb0 2024-04-25T10:29:23.903 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:23.899+0000 7f2fd27fc640 1 -- 172.21.15.37:0/2490201482 <== mon.0 v2:172.21.15.37:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2fc4032810 con 0x7f2fd40b4bb0 2024-04-25T10:29:23.903 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:23.899+0000 7f2fd9e84640 1 -- 172.21.15.37:0/2490201482 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f2fd40b4bb0 msgr2=0x7f2fd41646b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:29:23.903 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:23.899+0000 7f2fd9e84640 1 --2- 172.21.15.37:0/2490201482 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f2fd40b4bb0 0x7f2fd41646b0 secure :-1 s=READY pgs=146 cs=0 l=1 rev1=1 crypto rx=0x7f2fd40ccbe0 tx=0x7f2fc402a1e0 comp rx=0 tx=0).stop 2024-04-25T10:29:23.903 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:23.899+0000 7f2fd9e84640 1 -- 172.21.15.37:0/2490201482 shutdown_connections 2024-04-25T10:29:23.903 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:23.899+0000 7f2fd9e84640 1 --2- 172.21.15.37:0/2490201482 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f2fd40d4560 0x7f2fd40d4970 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:29:23.903 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:23.899+0000 7f2fd9e84640 1 --2- 172.21.15.37:0/2490201482 >> v2:172.21.15.203:3300/0 conn(0x7f2fd40b7600 0x7f2fd40b7a10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:29:23.903 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:23.899+0000 7f2fd9e84640 1 --2- 172.21.15.37:0/2490201482 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f2fd40b4bb0 0x7f2fd41646b0 unknown :-1 s=CLOSED pgs=146 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:29:23.903 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:23.899+0000 7f2fd9e84640 1 -- 172.21.15.37:0/2490201482 >> 172.21.15.37:0/2490201482 conn(0x7f2fd40d27e0 msgr2=0x7f2fd40d2bf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:29:23.903 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:23.899+0000 7f2fd9e84640 1 -- 172.21.15.37:0/2490201482 shutdown_connections 2024-04-25T10:29:23.903 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:23.899+0000 7f2fd9e84640 1 -- 172.21.15.37:0/2490201482 wait complete. 2024-04-25T10:29:23.904 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:23.901+0000 7f2fd9e84640 1 Processor -- start 2024-04-25T10:29:23.904 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:23.901+0000 7f2fd9e84640 1 -- start start 2024-04-25T10:29:23.906 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:23.902+0000 7f2fd9e84640 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f2fd40b4bb0 0x7f2fd4159a20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:29:23.906 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:23.902+0000 7f2fd9e84640 1 --2- >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f2fd40b7600 0x7f2fd40c81d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:29:23.906 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:23.903+0000 7f2fd9e84640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f2fd40d4560 0x7f2fd40ba0e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:29:23.906 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:23.903+0000 7f2fd9e84640 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 -- 0x7f2fd4164bf0 con 0x7f2fd40d4560 2024-04-25T10:29:23.907 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:23.903+0000 7f2fd9e84640 1 -- --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_getmap magic: 0 -- 0x7f2fd40b4fc0 con 0x7f2fd40b4bb0 2024-04-25T10:29:23.907 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:23.903+0000 7f2fd9e84640 1 -- --> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] -- mon_getmap magic: 0 -- 0x7f2fd40d09e0 con 0x7f2fd40b7600 2024-04-25T10:29:23.907 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:23.903+0000 7f2fd3fff640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f2fd40d4560 0x7f2fd40ba0e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:29:23.907 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:23.903+0000 7f2fd3fff640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f2fd40d4560 0x7f2fd40ba0e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.37:3300/0 says I am v2:172.21.15.37:35180/0 (socket says 172.21.15.37:35180) 2024-04-25T10:29:23.907 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:23.903+0000 7f2fd3fff640 1 -- 172.21.15.37:0/4016531596 learned_addr learned my addr 172.21.15.37:0/4016531596 (peer_addr_for_me v2:172.21.15.37:0/0) 2024-04-25T10:29:23.907 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:23.903+0000 7f2fd2ffd640 1 --2- 172.21.15.37:0/4016531596 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f2fd40b7600 0x7f2fd40c81d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:29:23.907 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:23.903+0000 7f2fd37fe640 1 --2- 172.21.15.37:0/4016531596 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f2fd40b4bb0 0x7f2fd4159a20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:29:23.907 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:23.903+0000 7f2fd3fff640 1 -- 172.21.15.37:0/4016531596 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f2fd40b7600 msgr2=0x7f2fd40c81d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:29:23.907 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:23.903+0000 7f2fd3fff640 1 --2- 172.21.15.37:0/4016531596 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f2fd40b7600 0x7f2fd40c81d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:29:23.907 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:23.903+0000 7f2fd3fff640 1 -- 172.21.15.37:0/4016531596 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f2fd40b4bb0 msgr2=0x7f2fd4159a20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:29:23.907 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:23.903+0000 7f2fd3fff640 1 --2- 172.21.15.37:0/4016531596 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f2fd40b4bb0 0x7f2fd4159a20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:29:23.907 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:23.903+0000 7f2fd3fff640 1 -- 172.21.15.37:0/4016531596 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f2fd41e9d20 con 0x7f2fd40d4560 2024-04-25T10:29:23.907 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:23.903+0000 7f2fd3fff640 1 --2- 172.21.15.37:0/4016531596 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f2fd40d4560 0x7f2fd40ba0e0 secure :-1 s=READY pgs=147 cs=0 l=1 rev1=1 crypto rx=0x7f2fcc007df0 tx=0x7f2fcc00d4a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:29:23.907 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:23.903+0000 7f2fcbfff640 1 -- 172.21.15.37:0/4016531596 <== mon.0 v2:172.21.15.37:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2fcc01a070 con 0x7f2fd40d4560 2024-04-25T10:29:23.907 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:23.903+0000 7f2fcbfff640 1 -- 172.21.15.37:0/4016531596 <== mon.0 v2:172.21.15.37:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f2fcc003990 con 0x7f2fd40d4560 2024-04-25T10:29:23.907 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:23.903+0000 7f2fd9e84640 1 -- 172.21.15.37:0/4016531596 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f2fd41ea000 con 0x7f2fd40d4560 2024-04-25T10:29:23.908 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:23.903+0000 7f2fcbfff640 1 -- 172.21.15.37:0/4016531596 <== mon.0 v2:172.21.15.37:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2fcc003d80 con 0x7f2fd40d4560 2024-04-25T10:29:23.908 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:23.903+0000 7f2fd9e84640 1 -- 172.21.15.37:0/4016531596 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f2fd41eaae0 con 0x7f2fd40d4560 2024-04-25T10:29:23.908 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:23.904+0000 7f2fd9e84640 1 -- 172.21.15.37:0/4016531596 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f2fd406a140 con 0x7f2fd40d4560 2024-04-25T10:29:23.908 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:23.905+0000 7f2fcbfff640 1 -- 172.21.15.37:0/4016531596 <== mon.0 v2:172.21.15.37:3300/0 4 ==== mgrmap(e 5) ==== 96782+0+0 (secure 0 0 0) 0x7f2fcc01d070 con 0x7f2fd40d4560 2024-04-25T10:29:23.908 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:23.906+0000 7f2fcbfff640 1 --2- 172.21.15.37:0/4016531596 >> [v2:172.21.15.203:6832/2418205237,v1:172.21.15.203:6833/2418205237] conn(0x7f2fb40744b0 0x7f2fb40769c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:29:23.909 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:23.906+0000 7f2fcbfff640 1 -- 172.21.15.37:0/4016531596 <== mon.0 v2:172.21.15.37:3300/0 5 ==== osd_map(23..23 src has 1..23) ==== 5709+0+0 (secure 0 0 0) 0x7f2fcc0954a0 con 0x7f2fd40d4560 2024-04-25T10:29:23.909 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:23.906+0000 7f2fd37fe640 1 --2- 172.21.15.37:0/4016531596 >> [v2:172.21.15.203:6832/2418205237,v1:172.21.15.203:6833/2418205237] conn(0x7f2fb40744b0 0x7f2fb40769c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:29:23.909 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:23.907+0000 7f2fd37fe640 1 --2- 172.21.15.37:0/4016531596 >> [v2:172.21.15.203:6832/2418205237,v1:172.21.15.203:6833/2418205237] conn(0x7f2fb40744b0 0x7f2fb40769c0 secure :-1 s=READY pgs=102 cs=0 l=1 rev1=1 crypto rx=0x7f2fc4034000 tx=0x7f2fc402e000 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:29:23.912 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:23.910+0000 7f2fcbfff640 1 -- 172.21.15.37:0/4016531596 <== mon.0 v2:172.21.15.37:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187246 (secure 0 0 0) 0x7f2fcc010070 con 0x7f2fd40d4560 2024-04-25T10:29:24.097 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:24.095+0000 7f2fd9e84640 1 -- 172.21.15.37:0/4016531596 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"} v 0) -- 0x7f2fd40d3ff0 con 0x7f2fd40d4560 2024-04-25T10:29:24.098 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:24.096+0000 7f2fcbfff640 1 -- 172.21.15.37:0/4016531596 <== mon.0 v2:172.21.15.37:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]=0 v23) ==== 104+0+11 (secure 0 0 0) 0x7f2fcc05f540 con 0x7f2fd40d4560 2024-04-25T10:29:24.098 INFO:teuthology.orchestra.run.smithi037.stdout:pg_num: 32 2024-04-25T10:29:24.100 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:24.097+0000 7f2fd9e84640 1 -- 172.21.15.37:0/4016531596 >> [v2:172.21.15.203:6832/2418205237,v1:172.21.15.203:6833/2418205237] conn(0x7f2fb40744b0 msgr2=0x7f2fb40769c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:29:24.100 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:24.097+0000 7f2fd9e84640 1 --2- 172.21.15.37:0/4016531596 >> [v2:172.21.15.203:6832/2418205237,v1:172.21.15.203:6833/2418205237] conn(0x7f2fb40744b0 0x7f2fb40769c0 secure :-1 s=READY pgs=102 cs=0 l=1 rev1=1 crypto rx=0x7f2fc4034000 tx=0x7f2fc402e000 comp rx=0 tx=0).stop 2024-04-25T10:29:24.100 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:24.097+0000 7f2fd9e84640 1 -- 172.21.15.37:0/4016531596 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f2fd40d4560 msgr2=0x7f2fd40ba0e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:29:24.100 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:24.097+0000 7f2fd9e84640 1 --2- 172.21.15.37:0/4016531596 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f2fd40d4560 0x7f2fd40ba0e0 secure :-1 s=READY pgs=147 cs=0 l=1 rev1=1 crypto rx=0x7f2fcc007df0 tx=0x7f2fcc00d4a0 comp rx=0 tx=0).stop 2024-04-25T10:29:24.100 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:24.097+0000 7f2fd9e84640 1 -- 172.21.15.37:0/4016531596 shutdown_connections 2024-04-25T10:29:24.100 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:24.097+0000 7f2fd9e84640 1 --2- 172.21.15.37:0/4016531596 >> [v2:172.21.15.203:6832/2418205237,v1:172.21.15.203:6833/2418205237] conn(0x7f2fb40744b0 0x7f2fb40769c0 unknown :-1 s=CLOSED pgs=102 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:29:24.100 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:24.097+0000 7f2fd9e84640 1 --2- 172.21.15.37:0/4016531596 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f2fd40d4560 0x7f2fd40ba0e0 unknown :-1 s=CLOSED pgs=147 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:29:24.100 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:24.097+0000 7f2fd9e84640 1 --2- 172.21.15.37:0/4016531596 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f2fd40b7600 0x7f2fd40c81d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:29:24.100 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:24.097+0000 7f2fd9e84640 1 --2- 172.21.15.37:0/4016531596 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f2fd40b4bb0 0x7f2fd4159a20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:29:24.100 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:24.097+0000 7f2fd9e84640 1 -- 172.21.15.37:0/4016531596 >> 172.21.15.37:0/4016531596 conn(0x7f2fd40d27e0 msgr2=0x7f2fd40b54c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:29:24.100 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:24.098+0000 7f2fd9e84640 1 -- 172.21.15.37:0/4016531596 shutdown_connections 2024-04-25T10:29:24.100 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:24.098+0000 7f2fd9e84640 1 -- 172.21.15.37:0/4016531596 wait complete. 2024-04-25T10:29:24.111 DEBUG:teuthology.orchestra.run.smithi037:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data pg_num 2024-04-25T10:29:24.225 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:24.223+0000 7f9edd36c640 1 Processor -- start 2024-04-25T10:29:24.225 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:24.223+0000 7f9edd36c640 1 -- start start 2024-04-25T10:29:24.226 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:24.225+0000 7f9edd36c640 1 --2- >> v2:172.21.15.37:3300/0 conn(0x7f9ed80b50e0 0x7f9ed80b79d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:29:24.226 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:24.225+0000 7f9edd36c640 1 -- --> v1:172.21.15.37:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f9ed80b8320 con 0x7f9ed80b7f10 2024-04-25T10:29:24.227 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:24.225+0000 7f9edd36c640 1 -- --> v1:172.21.15.203:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f9ed80d15e0 con 0x7f9ed80d1030 2024-04-25T10:29:24.227 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:24.225+0000 7f9edd36c640 1 -- --> v2:172.21.15.37:3300/0 -- mon_getmap magic: 0 -- 0x7f9ed81646e0 con 0x7f9ed80b50e0 2024-04-25T10:29:24.227 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:24.225+0000 7f9ed6ffd640 1 --2- >> v2:172.21.15.37:3300/0 conn(0x7f9ed80b50e0 0x7f9ed80b79d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:29:24.227 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:24.225+0000 7f9ed67fc640 1 --1- >> v1:172.21.15.37:6789/0 conn(0x7f9ed80b7f10 0x7f9ed80d0920 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.37:6789/0 says I am v1:172.21.15.37:59856/0 (socket says 172.21.15.37:59856) 2024-04-25T10:29:24.227 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:24.225+0000 7f9ed67fc640 1 -- 172.21.15.37:0/76386386 learned_addr learned my addr 172.21.15.37:0/76386386 (peer_addr_for_me v1:172.21.15.37:0/0) 2024-04-25T10:29:24.227 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:24.225+0000 7f9ed6ffd640 1 -- 172.21.15.37:0/76386386 >> v1:172.21.15.203:6789/0 conn(0x7f9ed80d1030 legacy=0x7f9ed815f7c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:29:24.227 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:24.225+0000 7f9ed6ffd640 1 -- 172.21.15.37:0/76386386 >> v1:172.21.15.37:6789/0 conn(0x7f9ed80b7f10 legacy=0x7f9ed80d0920 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:29:24.227 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:24.225+0000 7f9ed6ffd640 1 -- 172.21.15.37:0/76386386 --> v2:172.21.15.37:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f9ed8163db0 con 0x7f9ed80b50e0 2024-04-25T10:29:24.228 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:24.225+0000 7f9ed6ffd640 1 --2- 172.21.15.37:0/76386386 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f9ed80b50e0 0x7f9ed80b79d0 secure :-1 s=READY pgs=149 cs=0 l=1 rev1=1 crypto rx=0x7f9ed00097d0 tx=0x7f9ed002a270 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c9d60eaae1651df4 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:29:24.228 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:24.225+0000 7f9ed5ffb640 1 -- 172.21.15.37:0/76386386 <== mon.0 v2:172.21.15.37:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9ed0037070 con 0x7f9ed80b50e0 2024-04-25T10:29:24.228 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:24.225+0000 7f9ed5ffb640 1 -- 172.21.15.37:0/76386386 <== mon.0 v2:172.21.15.37:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f9ed0032440 con 0x7f9ed80b50e0 2024-04-25T10:29:24.229 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:24.225+0000 7f9ed5ffb640 1 -- 172.21.15.37:0/76386386 <== mon.0 v2:172.21.15.37:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9ed0032810 con 0x7f9ed80b50e0 2024-04-25T10:29:24.229 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:24.226+0000 7f9edd36c640 1 -- 172.21.15.37:0/76386386 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f9ed80b50e0 msgr2=0x7f9ed80b79d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:29:24.229 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:24.226+0000 7f9edd36c640 1 --2- 172.21.15.37:0/76386386 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f9ed80b50e0 0x7f9ed80b79d0 secure :-1 s=READY pgs=149 cs=0 l=1 rev1=1 crypto rx=0x7f9ed00097d0 tx=0x7f9ed002a270 comp rx=0 tx=0).stop 2024-04-25T10:29:24.229 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:24.226+0000 7f9edd36c640 1 -- 172.21.15.37:0/76386386 shutdown_connections 2024-04-25T10:29:24.229 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:24.226+0000 7f9edd36c640 1 --2- 172.21.15.37:0/76386386 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f9ed80b50e0 0x7f9ed80b79d0 unknown :-1 s=CLOSED pgs=149 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:29:24.229 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:24.226+0000 7f9edd36c640 1 -- 172.21.15.37:0/76386386 >> 172.21.15.37:0/76386386 conn(0x7f9ed80d5d50 msgr2=0x7f9ed80d6160 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:29:24.229 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:24.226+0000 7f9edd36c640 1 -- 172.21.15.37:0/76386386 shutdown_connections 2024-04-25T10:29:24.229 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:24.226+0000 7f9edd36c640 1 -- 172.21.15.37:0/76386386 wait complete. 2024-04-25T10:29:24.230 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:24.228+0000 7f9edd36c640 1 Processor -- start 2024-04-25T10:29:24.230 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:24.228+0000 7f9edd36c640 1 -- start start 2024-04-25T10:29:24.235 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:24.229+0000 7f9edd36c640 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f9ed80b50e0 0x7f9ed81f6060 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:29:24.235 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:24.229+0000 7f9edd36c640 1 --2- >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f9ed80b7f10 0x7f9ed815b390 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:29:24.235 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:24.229+0000 7f9edd36c640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f9ed80d1030 0x7f9ed81591d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:29:24.235 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:24.229+0000 7f9edd36c640 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 -- 0x7f9ed8164b70 con 0x7f9ed80d1030 2024-04-25T10:29:24.235 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:24.229+0000 7f9edd36c640 1 -- --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_getmap magic: 0 -- 0x7f9ed8164370 con 0x7f9ed80b50e0 2024-04-25T10:29:24.236 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:24.229+0000 7f9edd36c640 1 -- --> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] -- mon_getmap magic: 0 -- 0x7f9ed80ba070 con 0x7f9ed80b7f10 2024-04-25T10:29:24.236 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:24.229+0000 7f9ed77fe640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f9ed80d1030 0x7f9ed81591d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:29:24.236 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:24.229+0000 7f9ed77fe640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f9ed80d1030 0x7f9ed81591d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.37:3300/0 says I am v2:172.21.15.37:35190/0 (socket says 172.21.15.37:35190) 2024-04-25T10:29:24.236 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:24.229+0000 7f9ed77fe640 1 -- 172.21.15.37:0/4121333620 learned_addr learned my addr 172.21.15.37:0/4121333620 (peer_addr_for_me v2:172.21.15.37:0/0) 2024-04-25T10:29:24.236 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:24.229+0000 7f9ed67fc640 1 --2- 172.21.15.37:0/4121333620 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f9ed80b7f10 0x7f9ed815b390 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:29:24.236 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:24.229+0000 7f9ed77fe640 1 -- 172.21.15.37:0/4121333620 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f9ed80b7f10 msgr2=0x7f9ed815b390 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:29:24.236 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:24.229+0000 7f9ed77fe640 1 --2- 172.21.15.37:0/4121333620 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f9ed80b7f10 0x7f9ed815b390 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:29:24.236 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:24.229+0000 7f9ed77fe640 1 -- 172.21.15.37:0/4121333620 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f9ed80b50e0 msgr2=0x7f9ed81f6060 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:29:24.236 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:24.229+0000 7f9ed77fe640 1 --2- 172.21.15.37:0/4121333620 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f9ed80b50e0 0x7f9ed81f6060 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:29:24.236 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:24.229+0000 7f9ed77fe640 1 -- 172.21.15.37:0/4121333620 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f9ed814bd00 con 0x7f9ed80d1030 2024-04-25T10:29:24.236 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:24.229+0000 7f9ed67fc640 1 --2- 172.21.15.37:0/4121333620 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f9ed80b7f10 0x7f9ed815b390 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T10:29:24.236 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:24.230+0000 7f9ed77fe640 1 --2- 172.21.15.37:0/4121333620 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f9ed80d1030 0x7f9ed81591d0 secure :-1 s=READY pgs=150 cs=0 l=1 rev1=1 crypto rx=0x7f9ec800c680 tx=0x7f9ec800cb40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:29:24.236 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:24.230+0000 7f9ecf7fe640 1 -- 172.21.15.37:0/4121333620 <== mon.0 v2:172.21.15.37:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9ec8003b70 con 0x7f9ed80d1030 2024-04-25T10:29:24.236 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:24.230+0000 7f9ecf7fe640 1 -- 172.21.15.37:0/4121333620 <== mon.0 v2:172.21.15.37:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f9ec8003d10 con 0x7f9ed80d1030 2024-04-25T10:29:24.237 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:24.230+0000 7f9edd36c640 1 -- 172.21.15.37:0/4121333620 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f9ed815c100 con 0x7f9ed80d1030 2024-04-25T10:29:24.237 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:24.230+0000 7f9ecf7fe640 1 -- 172.21.15.37:0/4121333620 <== mon.0 v2:172.21.15.37:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9ec8005310 con 0x7f9ed80d1030 2024-04-25T10:29:24.237 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:24.230+0000 7f9edd36c640 1 -- 172.21.15.37:0/4121333620 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f9ed80cde50 con 0x7f9ed80d1030 2024-04-25T10:29:24.237 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:24.231+0000 7f9edd36c640 1 -- 172.21.15.37:0/4121333620 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f9ed815be70 con 0x7f9ed80d1030 2024-04-25T10:29:24.237 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:24.231+0000 7f9ecf7fe640 1 -- 172.21.15.37:0/4121333620 <== mon.0 v2:172.21.15.37:3300/0 4 ==== mgrmap(e 5) ==== 96782+0+0 (secure 0 0 0) 0x7f9ec80054b0 con 0x7f9ed80d1030 2024-04-25T10:29:24.237 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:24.233+0000 7f9ecf7fe640 1 --2- 172.21.15.37:0/4121333620 >> [v2:172.21.15.203:6832/2418205237,v1:172.21.15.203:6833/2418205237] conn(0x7f9eb80744e0 0x7f9eb80769f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:29:24.237 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:24.233+0000 7f9ecf7fe640 1 -- 172.21.15.37:0/4121333620 <== mon.0 v2:172.21.15.37:3300/0 5 ==== osd_map(23..23 src has 1..23) ==== 5709+0+0 (secure 0 0 0) 0x7f9ec80950a0 con 0x7f9ed80d1030 2024-04-25T10:29:24.237 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:24.233+0000 7f9ed6ffd640 1 --2- 172.21.15.37:0/4121333620 >> [v2:172.21.15.203:6832/2418205237,v1:172.21.15.203:6833/2418205237] conn(0x7f9eb80744e0 0x7f9eb80769f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:29:24.237 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:24.233+0000 7f9ed6ffd640 1 --2- 172.21.15.37:0/4121333620 >> [v2:172.21.15.203:6832/2418205237,v1:172.21.15.203:6833/2418205237] conn(0x7f9eb80744e0 0x7f9eb80769f0 secure :-1 s=READY pgs=103 cs=0 l=1 rev1=1 crypto rx=0x7f9ed0034000 tx=0x7f9ed002c000 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:29:24.239 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:24.237+0000 7f9ecf7fe640 1 -- 172.21.15.37:0/4121333620 <== mon.0 v2:172.21.15.37:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187246 (secure 0 0 0) 0x7f9ec805f830 con 0x7f9ed80d1030 2024-04-25T10:29:24.426 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:24.424+0000 7f9edd36c640 1 -- 172.21.15.37:0/4121333620 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"} v 0) -- 0x7f9ed806a140 con 0x7f9ed80d1030 2024-04-25T10:29:24.426 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:24.424+0000 7f9ecf7fe640 1 -- 172.21.15.37:0/4121333620 <== mon.0 v2:172.21.15.37:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]=0 v23) ==== 100+0+11 (secure 0 0 0) 0x7f9ec805f130 con 0x7f9ed80d1030 2024-04-25T10:29:24.427 INFO:teuthology.orchestra.run.smithi037.stdout:pg_num: 64 2024-04-25T10:29:24.428 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:24.426+0000 7f9edd36c640 1 -- 172.21.15.37:0/4121333620 >> [v2:172.21.15.203:6832/2418205237,v1:172.21.15.203:6833/2418205237] conn(0x7f9eb80744e0 msgr2=0x7f9eb80769f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:29:24.429 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:24.426+0000 7f9edd36c640 1 --2- 172.21.15.37:0/4121333620 >> [v2:172.21.15.203:6832/2418205237,v1:172.21.15.203:6833/2418205237] conn(0x7f9eb80744e0 0x7f9eb80769f0 secure :-1 s=READY pgs=103 cs=0 l=1 rev1=1 crypto rx=0x7f9ed0034000 tx=0x7f9ed002c000 comp rx=0 tx=0).stop 2024-04-25T10:29:24.429 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:24.426+0000 7f9edd36c640 1 -- 172.21.15.37:0/4121333620 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f9ed80d1030 msgr2=0x7f9ed81591d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:29:24.429 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:24.426+0000 7f9edd36c640 1 --2- 172.21.15.37:0/4121333620 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f9ed80d1030 0x7f9ed81591d0 secure :-1 s=READY pgs=150 cs=0 l=1 rev1=1 crypto rx=0x7f9ec800c680 tx=0x7f9ec800cb40 comp rx=0 tx=0).stop 2024-04-25T10:29:24.429 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:24.426+0000 7f9edd36c640 1 -- 172.21.15.37:0/4121333620 shutdown_connections 2024-04-25T10:29:24.429 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:24.426+0000 7f9edd36c640 1 --2- 172.21.15.37:0/4121333620 >> [v2:172.21.15.203:6832/2418205237,v1:172.21.15.203:6833/2418205237] conn(0x7f9eb80744e0 0x7f9eb80769f0 unknown :-1 s=CLOSED pgs=103 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:29:24.429 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:24.426+0000 7f9edd36c640 1 --2- 172.21.15.37:0/4121333620 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f9ed80d1030 0x7f9ed81591d0 unknown :-1 s=CLOSED pgs=150 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:29:24.429 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:24.426+0000 7f9edd36c640 1 --2- 172.21.15.37:0/4121333620 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f9ed80b7f10 0x7f9ed815b390 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:29:24.429 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:24.426+0000 7f9edd36c640 1 --2- 172.21.15.37:0/4121333620 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f9ed80b50e0 0x7f9ed81f6060 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:29:24.429 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:24.426+0000 7f9edd36c640 1 -- 172.21.15.37:0/4121333620 >> 172.21.15.37:0/4121333620 conn(0x7f9ed80d5d50 msgr2=0x7f9ed8162290 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:29:24.429 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:24.426+0000 7f9edd36c640 1 -- 172.21.15.37:0/4121333620 shutdown_connections 2024-04-25T10:29:24.429 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:24.426+0000 7f9edd36c640 1 -- 172.21.15.37:0/4121333620 wait complete. 2024-04-25T10:29:24.440 DEBUG:teuthology.orchestra.run.smithi037:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-04-25T10:29:24.551 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:24.549+0000 7fd5d859d640 1 Processor -- start 2024-04-25T10:29:24.551 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:24.549+0000 7fd5d859d640 1 -- start start 2024-04-25T10:29:24.555 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:24.550+0000 7fd5d859d640 1 --2- >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7fd5d00b2dc0 0x7fd5d00d2860 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:29:24.555 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:24.550+0000 7fd5d859d640 1 -- --> v1:172.21.15.37:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fd5d00d2e40 con 0x7fd5d006e610 2024-04-25T10:29:24.555 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:24.550+0000 7fd5d859d640 1 -- --> v1:172.21.15.203:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fd5d00d3120 con 0x7fd5d006daf0 2024-04-25T10:29:24.555 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:24.550+0000 7fd5d859d640 1 -- --> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] -- mon_getmap magic: 0 -- 0x7fd5d00b92c0 con 0x7fd5d00b2dc0 2024-04-25T10:29:24.555 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:24.550+0000 7fd5d6b3a640 1 --1- >> v1:172.21.15.37:6789/0 conn(0x7fd5d006e610 0x7fd5d00ce560 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.37:6789/0 says I am v1:172.21.15.37:59866/0 (socket says 172.21.15.37:59866) 2024-04-25T10:29:24.555 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:24.550+0000 7fd5d6b3a640 1 -- 172.21.15.37:0/652756206 learned_addr learned my addr 172.21.15.37:0/652756206 (peer_addr_for_me v1:172.21.15.37:0/0) 2024-04-25T10:29:24.556 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:24.551+0000 7fd5d5337640 1 -- 172.21.15.37:0/652756206 <== mon.0 v1:172.21.15.37:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 1179128169 0 0) 0x7fd5d00d2e40 con 0x7fd5d006e610 2024-04-25T10:29:24.556 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:24.551+0000 7fd5d5337640 1 -- 172.21.15.37:0/652756206 --> v1:172.21.15.37:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7fd5bc003520 con 0x7fd5d006e610 2024-04-25T10:29:24.556 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:24.551+0000 7fd5d6339640 1 --2- 172.21.15.37:0/652756206 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7fd5d00b2dc0 0x7fd5d00d2860 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:29:24.556 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:24.551+0000 7fd5d5337640 1 -- 172.21.15.37:0/652756206 <== mon.0 v1:172.21.15.37:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 2254534389 0 0) 0x7fd5bc003520 con 0x7fd5d006e610 2024-04-25T10:29:24.556 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:24.551+0000 7fd5d5337640 1 -- 172.21.15.37:0/652756206 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7fd5d00b2dc0 msgr2=0x7fd5d00d2860 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:29:24.556 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:24.551+0000 7fd5d5337640 1 --2- 172.21.15.37:0/652756206 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7fd5d00b2dc0 0x7fd5d00d2860 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:29:24.556 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:24.551+0000 7fd5d5337640 1 -- 172.21.15.37:0/652756206 >> v1:172.21.15.203:6789/0 conn(0x7fd5d006daf0 legacy=0x7fd5d006df00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:29:24.556 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:24.551+0000 7fd5d5337640 1 -- 172.21.15.37:0/652756206 --> v1:172.21.15.37:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fd5d00d3400 con 0x7fd5d006e610 2024-04-25T10:29:24.556 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:24.551+0000 7fd5d5337640 1 -- 172.21.15.37:0/652756206 <== mon.0 v1:172.21.15.37:6789/0 3 ==== mon_map magic: 0 ==== 413+0+0 (unknown 1577794600 0 0) 0x7fd5c80019f0 con 0x7fd5d006e610 2024-04-25T10:29:24.556 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:24.551+0000 7fd5d5337640 1 -- 172.21.15.37:0/652756206 >> v1:172.21.15.37:6789/0 conn(0x7fd5d006e610 legacy=0x7fd5d00ce560 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:29:24.556 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:24.551+0000 7fd5d5337640 1 --2- 172.21.15.37:0/652756206 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fd5bc0039e0 0x7fd5bc003e50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:29:24.556 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:24.551+0000 7fd5d5337640 1 --2- 172.21.15.37:0/652756206 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fd5bc004800 0x7fd5bc004c10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:29:24.556 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:24.551+0000 7fd5d5337640 1 --2- 172.21.15.37:0/652756206 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7fd5d00b2dc0 0x7fd5bc0052d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:29:24.556 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:24.551+0000 7fd5d5337640 1 -- 172.21.15.37:0/652756206 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 -- 0x7fd5bc00bb70 con 0x7fd5bc0039e0 2024-04-25T10:29:24.557 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:24.551+0000 7fd5d5337640 1 -- 172.21.15.37:0/652756206 --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_getmap magic: 0 -- 0x7fd5bc00bd20 con 0x7fd5bc004800 2024-04-25T10:29:24.557 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:24.551+0000 7fd5d5337640 1 -- 172.21.15.37:0/652756206 --> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] -- mon_getmap magic: 0 -- 0x7fd5bc00bf20 con 0x7fd5d00b2dc0 2024-04-25T10:29:24.557 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:24.551+0000 7fd5d6339640 1 --2- 172.21.15.37:0/652756206 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fd5bc0039e0 0x7fd5bc003e50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:29:24.557 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:24.551+0000 7fd5d5b38640 1 --2- 172.21.15.37:0/652756206 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fd5bc004800 0x7fd5bc004c10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:29:24.557 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:24.551+0000 7fd5d6b3a640 1 --2- 172.21.15.37:0/652756206 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7fd5d00b2dc0 0x7fd5bc0052d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:29:24.557 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:24.551+0000 7fd5d6339640 1 -- 172.21.15.37:0/652756206 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7fd5d00b2dc0 msgr2=0x7fd5bc0052d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:29:24.557 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:24.551+0000 7fd5d6339640 1 --2- 172.21.15.37:0/652756206 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7fd5d00b2dc0 0x7fd5bc0052d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:29:24.557 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:24.551+0000 7fd5d6339640 1 -- 172.21.15.37:0/652756206 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fd5bc004800 msgr2=0x7fd5bc004c10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:29:24.557 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:24.552+0000 7fd5d6339640 1 --2- 172.21.15.37:0/652756206 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fd5bc004800 0x7fd5bc004c10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:29:24.557 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:24.552+0000 7fd5d6339640 1 -- 172.21.15.37:0/652756206 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fd5bc003520 con 0x7fd5bc0039e0 2024-04-25T10:29:24.557 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:24.552+0000 7fd5d6339640 1 --2- 172.21.15.37:0/652756206 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fd5bc0039e0 0x7fd5bc003e50 secure :-1 s=READY pgs=152 cs=0 l=1 rev1=1 crypto rx=0x7fd5cc00b9f0 tx=0x7fd5bc005830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=acaeb376d05883b8 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:29:24.557 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:24.552+0000 7fd5d5337640 1 -- 172.21.15.37:0/652756206 <== mon.0 v2:172.21.15.37:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd5cc030070 con 0x7fd5bc0039e0 2024-04-25T10:29:24.557 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:24.552+0000 7fd5d5337640 1 -- 172.21.15.37:0/652756206 <== mon.0 v2:172.21.15.37:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fd5cc02adb0 con 0x7fd5bc0039e0 2024-04-25T10:29:24.557 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:24.552+0000 7fd5d5337640 1 -- 172.21.15.37:0/652756206 <== mon.0 v2:172.21.15.37:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd5cc00b5f0 con 0x7fd5bc0039e0 2024-04-25T10:29:24.557 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:24.553+0000 7fd5d859d640 1 -- 172.21.15.37:0/652756206 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fd5bc0039e0 msgr2=0x7fd5bc003e50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:29:24.558 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:24.553+0000 7fd5d859d640 1 --2- 172.21.15.37:0/652756206 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fd5bc0039e0 0x7fd5bc003e50 secure :-1 s=READY pgs=152 cs=0 l=1 rev1=1 crypto rx=0x7fd5cc00b9f0 tx=0x7fd5bc005830 comp rx=0 tx=0).stop 2024-04-25T10:29:24.558 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:24.553+0000 7fd5d6b3a640 1 -- 172.21.15.37:0/652756206 reap_dead start 2024-04-25T10:29:24.558 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:24.553+0000 7fd5d859d640 1 -- 172.21.15.37:0/652756206 shutdown_connections 2024-04-25T10:29:24.558 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:24.553+0000 7fd5d859d640 1 -- 172.21.15.37:0/652756206 >> 172.21.15.37:0/652756206 conn(0x7fd5d00b5440 msgr2=0x7fd5d00b5850 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:29:24.558 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:24.553+0000 7fd5d859d640 1 -- 172.21.15.37:0/652756206 shutdown_connections 2024-04-25T10:29:24.558 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:24.553+0000 7fd5d859d640 1 -- 172.21.15.37:0/652756206 wait complete. 2024-04-25T10:29:24.560 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:24.556+0000 7fd5d859d640 1 Processor -- start 2024-04-25T10:29:24.560 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:24.556+0000 7fd5d859d640 1 -- start start 2024-04-25T10:29:24.560 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:24.557+0000 7fd5d859d640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fd5bc0039e0 0x7fd5d0155680 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:29:24.560 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:24.557+0000 7fd5d859d640 1 --2- >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7fd5d00b6ec0 0x7fd5d00b5850 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:29:24.561 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:24.557+0000 7fd5d859d640 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fd5d0155ec0 0x7fd5d00b5d90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:29:24.561 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:24.557+0000 7fd5d859d640 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 -- 0x7fd5d00d2860 con 0x7fd5bc0039e0 2024-04-25T10:29:24.561 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:24.557+0000 7fd5d859d640 1 -- --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_getmap magic: 0 -- 0x7fd5d00ce0d0 con 0x7fd5d0155ec0 2024-04-25T10:29:24.561 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:24.557+0000 7fd5d859d640 1 -- --> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] -- mon_getmap magic: 0 -- 0x7fd5d00d29e0 con 0x7fd5d00b6ec0 2024-04-25T10:29:24.561 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:24.557+0000 7fd5d6339640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fd5bc0039e0 0x7fd5d0155680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:29:24.561 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:24.557+0000 7fd5d6339640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fd5bc0039e0 0x7fd5d0155680 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.37:3300/0 says I am v2:172.21.15.37:35194/0 (socket says 172.21.15.37:35194) 2024-04-25T10:29:24.561 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:24.557+0000 7fd5d6339640 1 -- 172.21.15.37:0/2095873458 learned_addr learned my addr 172.21.15.37:0/2095873458 (peer_addr_for_me v2:172.21.15.37:0/0) 2024-04-25T10:29:24.561 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:24.557+0000 7fd5d6b3a640 1 --2- 172.21.15.37:0/2095873458 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fd5d0155ec0 0x7fd5d00b5d90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:29:24.561 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:24.557+0000 7fd5d5b38640 1 --2- 172.21.15.37:0/2095873458 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7fd5d00b6ec0 0x7fd5d00b5850 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:29:24.561 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:24.557+0000 7fd5d6339640 1 -- 172.21.15.37:0/2095873458 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7fd5d00b6ec0 msgr2=0x7fd5d00b5850 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:29:24.561 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:24.557+0000 7fd5d6339640 1 --2- 172.21.15.37:0/2095873458 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7fd5d00b6ec0 0x7fd5d00b5850 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:29:24.561 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:24.557+0000 7fd5d6339640 1 -- 172.21.15.37:0/2095873458 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fd5d0155ec0 msgr2=0x7fd5d00b5d90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:29:24.561 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:24.557+0000 7fd5d6339640 1 --2- 172.21.15.37:0/2095873458 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fd5d0155ec0 0x7fd5d00b5d90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:29:24.561 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:24.557+0000 7fd5d6339640 1 -- 172.21.15.37:0/2095873458 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fd5d00b7630 con 0x7fd5bc0039e0 2024-04-25T10:29:24.561 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:24.557+0000 7fd5d6339640 1 --2- 172.21.15.37:0/2095873458 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fd5bc0039e0 0x7fd5d0155680 secure :-1 s=READY pgs=153 cs=0 l=1 rev1=1 crypto rx=0x7fd5cc00b9f0 tx=0x7fd5cc004730 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:29:24.562 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:24.557+0000 7fd5c6ffd640 1 -- 172.21.15.37:0/2095873458 <== mon.0 v2:172.21.15.37:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd5cc043020 con 0x7fd5bc0039e0 2024-04-25T10:29:24.562 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:24.557+0000 7fd5c6ffd640 1 -- 172.21.15.37:0/2095873458 <== mon.0 v2:172.21.15.37:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fd5cc03f680 con 0x7fd5bc0039e0 2024-04-25T10:29:24.562 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:24.557+0000 7fd5d859d640 1 -- 172.21.15.37:0/2095873458 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fd5d00b7910 con 0x7fd5bc0039e0 2024-04-25T10:29:24.562 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:24.557+0000 7fd5c6ffd640 1 -- 172.21.15.37:0/2095873458 <== mon.0 v2:172.21.15.37:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd5cc00b5f0 con 0x7fd5bc0039e0 2024-04-25T10:29:24.562 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:24.557+0000 7fd5d859d640 1 -- 172.21.15.37:0/2095873458 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fd5d0154c90 con 0x7fd5bc0039e0 2024-04-25T10:29:24.562 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:24.558+0000 7fd5d859d640 1 -- 172.21.15.37:0/2095873458 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fd5d00b9270 con 0x7fd5bc0039e0 2024-04-25T10:29:24.563 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:24.559+0000 7fd5c6ffd640 1 -- 172.21.15.37:0/2095873458 <== mon.0 v2:172.21.15.37:3300/0 4 ==== mgrmap(e 5) ==== 96782+0+0 (secure 0 0 0) 0x7fd5cc046020 con 0x7fd5bc0039e0 2024-04-25T10:29:24.563 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:24.559+0000 7fd5c6ffd640 1 --2- 172.21.15.37:0/2095873458 >> [v2:172.21.15.203:6832/2418205237,v1:172.21.15.203:6833/2418205237] conn(0x7fd5b40744b0 0x7fd5b40769c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:29:24.563 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:24.559+0000 7fd5c6ffd640 1 -- 172.21.15.37:0/2095873458 <== mon.0 v2:172.21.15.37:3300/0 5 ==== osd_map(23..23 src has 1..23) ==== 5709+0+0 (secure 0 0 0) 0x7fd5cc0b5d20 con 0x7fd5bc0039e0 2024-04-25T10:29:24.563 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:24.560+0000 7fd5d5b38640 1 --2- 172.21.15.37:0/2095873458 >> [v2:172.21.15.203:6832/2418205237,v1:172.21.15.203:6833/2418205237] conn(0x7fd5b40744b0 0x7fd5b40769c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:29:24.563 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:24.560+0000 7fd5d5b38640 1 --2- 172.21.15.37:0/2095873458 >> [v2:172.21.15.203:6832/2418205237,v1:172.21.15.203:6833/2418205237] conn(0x7fd5b40744b0 0x7fd5b40769c0 secure :-1 s=READY pgs=104 cs=0 l=1 rev1=1 crypto rx=0x7fd5d015fcc0 tx=0x7fd5c0009040 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:29:24.566 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:24.564+0000 7fd5c6ffd640 1 -- 172.21.15.37:0/2095873458 <== mon.0 v2:172.21.15.37:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187246 (secure 0 0 0) 0x7fd5d00b9270 con 0x7fd5bc0039e0 2024-04-25T10:29:24.795 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:24.793+0000 7fd5d859d640 1 -- 172.21.15.37:0/2095873458 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7fd5d006a140 con 0x7fd5bc0039e0 2024-04-25T10:29:24.795 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:24.793+0000 7fd5c6ffd640 1 -- 172.21.15.37:0/2095873458 <== mon.0 v2:172.21.15.37:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 8 v8) ==== 93+0+5178 (secure 0 0 0) 0x7fd5cc04d030 con 0x7fd5bc0039e0 2024-04-25T10:29:24.796 INFO:teuthology.orchestra.run.smithi037.stdout: 2024-04-25T10:29:24.796 INFO:teuthology.orchestra.run.smithi037.stdout:{"epoch":8,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":4232,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.37:6835/1911169480","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.37:6833","nonce":1911169480},{"type":"v1","addr":"172.21.15.37:6835","nonce":1911169480}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3},{"gid":4241,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.203:6835/3873243328","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6834","nonce":3873243328},{"type":"v1","addr":"172.21.15.203:6835","nonce":3873243328}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3},{"gid":4278,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.37:6834/179723013","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.37:6832","nonce":179723013},{"type":"v1","addr":"172.21.15.37:6834","nonce":179723013}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":2}],"filesystems":[{"mdsmap":{"epoch":8,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-04-25T10:29:08.397829+0000","modified":"2024-04-25T10:29:11.407694+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":1,"in":[0],"up":{"mds_0":4279},"failed":[],"damaged":[],"stopped":[],"info":{"gid_4279":{"gid":4279,"name":"d","rank":0,"incarnation":4,"state":"up:active","state_seq":3,"addr":"172.21.15.203:6837/3215191422","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6836","nonce":3215191422},{"type":"v1","addr":"172.21.15.203:6837","nonce":3215191422}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":4279,"qdb_cluster":[4279]},"id":1}]} 2024-04-25T10:29:24.796 INFO:teuthology.orchestra.run.smithi037.stderr:dumped fsmap epoch 8 2024-04-25T10:29:24.798 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:24.795+0000 7fd5d859d640 1 -- 172.21.15.37:0/2095873458 >> [v2:172.21.15.203:6832/2418205237,v1:172.21.15.203:6833/2418205237] conn(0x7fd5b40744b0 msgr2=0x7fd5b40769c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:29:24.798 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:24.795+0000 7fd5d859d640 1 --2- 172.21.15.37:0/2095873458 >> [v2:172.21.15.203:6832/2418205237,v1:172.21.15.203:6833/2418205237] conn(0x7fd5b40744b0 0x7fd5b40769c0 secure :-1 s=READY pgs=104 cs=0 l=1 rev1=1 crypto rx=0x7fd5d015fcc0 tx=0x7fd5c0009040 comp rx=0 tx=0).stop 2024-04-25T10:29:24.798 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:24.795+0000 7fd5d859d640 1 -- 172.21.15.37:0/2095873458 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fd5bc0039e0 msgr2=0x7fd5d0155680 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:29:24.798 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:24.795+0000 7fd5d859d640 1 --2- 172.21.15.37:0/2095873458 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fd5bc0039e0 0x7fd5d0155680 secure :-1 s=READY pgs=153 cs=0 l=1 rev1=1 crypto rx=0x7fd5cc00b9f0 tx=0x7fd5cc004730 comp rx=0 tx=0).stop 2024-04-25T10:29:24.798 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:24.795+0000 7fd5d859d640 1 -- 172.21.15.37:0/2095873458 shutdown_connections 2024-04-25T10:29:24.798 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:24.795+0000 7fd5d859d640 1 --2- 172.21.15.37:0/2095873458 >> [v2:172.21.15.203:6832/2418205237,v1:172.21.15.203:6833/2418205237] conn(0x7fd5b40744b0 0x7fd5b40769c0 unknown :-1 s=CLOSED pgs=104 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:29:24.798 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:24.795+0000 7fd5d859d640 1 --2- 172.21.15.37:0/2095873458 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fd5d0155ec0 0x7fd5d00b5d90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:29:24.798 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:24.795+0000 7fd5d859d640 1 --2- 172.21.15.37:0/2095873458 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7fd5d00b6ec0 0x7fd5d00b5850 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:29:24.798 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:24.795+0000 7fd5d859d640 1 --2- 172.21.15.37:0/2095873458 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fd5bc0039e0 0x7fd5d0155680 unknown :-1 s=CLOSED pgs=153 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:29:24.798 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:24.795+0000 7fd5d859d640 1 -- 172.21.15.37:0/2095873458 >> 172.21.15.37:0/2095873458 conn(0x7fd5d00b5440 msgr2=0x7fd5d00b3670 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:29:24.798 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:24.795+0000 7fd5d859d640 1 -- 172.21.15.37:0/2095873458 shutdown_connections 2024-04-25T10:29:24.799 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:24.795+0000 7fd5d859d640 1 -- 172.21.15.37:0/2095873458 wait complete. 2024-04-25T10:29:24.809 DEBUG:teuthology.orchestra.run.smithi037:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-04-25T10:29:24.919 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:24.918+0000 7f5db8946640 1 Processor -- start 2024-04-25T10:29:24.920 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:24.918+0000 7f5db8946640 1 -- start start 2024-04-25T10:29:24.922 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:24.920+0000 7f5db8946640 1 --2- >> v2:172.21.15.37:3300/0 conn(0x7f5db40b2dc0 0x7f5db40d2860 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:29:24.922 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:24.920+0000 7f5db8946640 1 --2- >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f5db406e610 0x7f5db40ce560 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:29:24.922 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:24.920+0000 7f5db8946640 1 -- --> v1:172.21.15.203:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f5db40d2e40 con 0x7f5db406daf0 2024-04-25T10:29:24.922 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:24.920+0000 7f5db8946640 1 -- --> v2:172.21.15.37:3300/0 -- mon_getmap magic: 0 -- 0x7f5db40b92c0 con 0x7f5db40b2dc0 2024-04-25T10:29:24.922 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:24.920+0000 7f5db8946640 1 -- --> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] -- mon_getmap magic: 0 -- 0x7f5db40ce0d0 con 0x7f5db406e610 2024-04-25T10:29:24.922 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:24.920+0000 7f5db259c640 1 --2- >> v2:172.21.15.37:3300/0 conn(0x7f5db40b2dc0 0x7f5db40d2860 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:29:24.922 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:24.920+0000 7f5db259c640 1 --2- >> v2:172.21.15.37:3300/0 conn(0x7f5db40b2dc0 0x7f5db40d2860 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.37:3300/0 says I am v2:172.21.15.37:35198/0 (socket says 172.21.15.37:35198) 2024-04-25T10:29:24.923 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:24.920+0000 7f5db259c640 1 -- 172.21.15.37:0/2926397228 learned_addr learned my addr 172.21.15.37:0/2926397228 (peer_addr_for_me v2:172.21.15.37:0/0) 2024-04-25T10:29:24.923 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:24.920+0000 7f5db2d9d640 1 --2- 172.21.15.37:0/2926397228 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f5db406e610 0x7f5db40ce560 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:29:24.923 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:24.920+0000 7f5db259c640 1 -- 172.21.15.37:0/2926397228 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f5db406e610 msgr2=0x7f5db40ce560 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:29:24.923 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:24.920+0000 7f5db259c640 1 --2- 172.21.15.37:0/2926397228 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f5db406e610 0x7f5db40ce560 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:29:24.923 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:24.920+0000 7f5db259c640 1 -- 172.21.15.37:0/2926397228 >> v1:172.21.15.203:6789/0 conn(0x7f5db406daf0 legacy=0x7f5db406df00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:29:24.923 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:24.920+0000 7f5db259c640 1 -- 172.21.15.37:0/2926397228 --> v2:172.21.15.37:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f5db40d3120 con 0x7f5db40b2dc0 2024-04-25T10:29:24.923 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:24.920+0000 7f5db2d9d640 1 --2- 172.21.15.37:0/2926397228 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f5db406e610 0x7f5db40ce560 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T10:29:24.923 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:24.920+0000 7f5db259c640 1 --2- 172.21.15.37:0/2926397228 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f5db40b2dc0 0x7f5db40d2860 secure :-1 s=READY pgs=154 cs=0 l=1 rev1=1 crypto rx=0x7f5dac009850 tx=0x7f5dac02a1e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c2a3773f69490d59 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:29:24.923 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:24.920+0000 7f5db159a640 1 -- 172.21.15.37:0/2926397228 <== mon.0 v2:172.21.15.37:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5dac004030 con 0x7f5db40b2dc0 2024-04-25T10:29:24.923 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:24.920+0000 7f5db159a640 1 -- 172.21.15.37:0/2926397228 <== mon.0 v2:172.21.15.37:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f5dac0041d0 con 0x7f5db40b2dc0 2024-04-25T10:29:24.923 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:24.920+0000 7f5db159a640 1 -- 172.21.15.37:0/2926397228 <== mon.0 v2:172.21.15.37:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5dac003610 con 0x7f5db40b2dc0 2024-04-25T10:29:24.924 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:24.921+0000 7f5db8946640 1 -- 172.21.15.37:0/2926397228 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f5db40b2dc0 msgr2=0x7f5db40d2860 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:29:24.924 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:24.921+0000 7f5db8946640 1 --2- 172.21.15.37:0/2926397228 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f5db40b2dc0 0x7f5db40d2860 secure :-1 s=READY pgs=154 cs=0 l=1 rev1=1 crypto rx=0x7f5dac009850 tx=0x7f5dac02a1e0 comp rx=0 tx=0).stop 2024-04-25T10:29:24.924 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:24.921+0000 7f5db8946640 1 -- 172.21.15.37:0/2926397228 shutdown_connections 2024-04-25T10:29:24.924 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:24.921+0000 7f5db8946640 1 --2- 172.21.15.37:0/2926397228 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f5db406e610 0x7f5db40ce560 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:29:24.924 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:24.921+0000 7f5db8946640 1 --2- 172.21.15.37:0/2926397228 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f5db40b2dc0 0x7f5db40d2860 unknown :-1 s=CLOSED pgs=154 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:29:24.924 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:24.921+0000 7f5db8946640 1 -- 172.21.15.37:0/2926397228 >> 172.21.15.37:0/2926397228 conn(0x7f5db40b5440 msgr2=0x7f5db40b5850 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:29:24.924 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:24.921+0000 7f5db8946640 1 -- 172.21.15.37:0/2926397228 shutdown_connections 2024-04-25T10:29:24.925 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:24.921+0000 7f5db8946640 1 -- 172.21.15.37:0/2926397228 wait complete. 2024-04-25T10:29:24.925 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:24.923+0000 7f5db8946640 1 Processor -- start 2024-04-25T10:29:24.925 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:24.923+0000 7f5db8946640 1 -- start start 2024-04-25T10:29:24.928 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:24.924+0000 7f5db8946640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f5db406daf0 0x7f5db40cd080 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:29:24.928 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:24.924+0000 7f5db8946640 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f5db406e610 0x7f5db40cb9b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:29:24.928 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:24.924+0000 7f5db8946640 1 --2- >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f5db40b2dc0 0x7f5db40cbef0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:29:24.928 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:24.924+0000 7f5db8946640 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 -- 0x7f5db40ceaa0 con 0x7f5db406daf0 2024-04-25T10:29:24.928 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:24.924+0000 7f5db8946640 1 -- --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_getmap magic: 0 -- 0x7f5db40d36e0 con 0x7f5db406e610 2024-04-25T10:29:24.928 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:24.924+0000 7f5db8946640 1 -- --> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] -- mon_getmap magic: 0 -- 0x7f5db40cec20 con 0x7f5db40b2dc0 2024-04-25T10:29:24.928 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:24.924+0000 7f5db259c640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f5db406daf0 0x7f5db40cd080 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:29:24.928 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:24.924+0000 7f5db259c640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f5db406daf0 0x7f5db40cd080 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.37:3300/0 says I am v2:172.21.15.37:35210/0 (socket says 172.21.15.37:35210) 2024-04-25T10:29:24.928 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:24.924+0000 7f5db259c640 1 -- 172.21.15.37:0/668234259 learned_addr learned my addr 172.21.15.37:0/668234259 (peer_addr_for_me v2:172.21.15.37:0/0) 2024-04-25T10:29:24.928 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:24.924+0000 7f5db1d9b640 1 --2- 172.21.15.37:0/668234259 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f5db406e610 0x7f5db40cb9b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:29:24.928 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:24.924+0000 7f5db2d9d640 1 --2- 172.21.15.37:0/668234259 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f5db40b2dc0 0x7f5db40cbef0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:29:24.928 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:24.924+0000 7f5db259c640 1 -- 172.21.15.37:0/668234259 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f5db40b2dc0 msgr2=0x7f5db40cbef0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:29:24.929 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:24.924+0000 7f5db259c640 1 --2- 172.21.15.37:0/668234259 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f5db40b2dc0 0x7f5db40cbef0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:29:24.929 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:24.924+0000 7f5db259c640 1 -- 172.21.15.37:0/668234259 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f5db406e610 msgr2=0x7f5db40cb9b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:29:24.929 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:24.924+0000 7f5db259c640 1 --2- 172.21.15.37:0/668234259 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f5db406e610 0x7f5db40cb9b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:29:24.929 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:24.924+0000 7f5db259c640 1 -- 172.21.15.37:0/668234259 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f5db40ca910 con 0x7f5db406daf0 2024-04-25T10:29:24.929 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:24.924+0000 7f5db1d9b640 1 --2- 172.21.15.37:0/668234259 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f5db406e610 0x7f5db40cb9b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T10:29:24.929 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:24.925+0000 7f5db259c640 1 --2- 172.21.15.37:0/668234259 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f5db406daf0 0x7f5db40cd080 secure :-1 s=READY pgs=155 cs=0 l=1 rev1=1 crypto rx=0x7f5dac000c00 tx=0x7f5dac0046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:29:24.929 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:24.925+0000 7f5daaffd640 1 -- 172.21.15.37:0/668234259 <== mon.0 v2:172.21.15.37:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5dac041020 con 0x7f5db406daf0 2024-04-25T10:29:24.929 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:24.925+0000 7f5daaffd640 1 -- 172.21.15.37:0/668234259 <== mon.0 v2:172.21.15.37:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f5dac003c50 con 0x7f5db406daf0 2024-04-25T10:29:24.929 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:24.925+0000 7f5db8946640 1 -- 172.21.15.37:0/668234259 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f5db40cabf0 con 0x7f5db406daf0 2024-04-25T10:29:24.929 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:24.925+0000 7f5daaffd640 1 -- 172.21.15.37:0/668234259 <== mon.0 v2:172.21.15.37:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5dac004030 con 0x7f5db406daf0 2024-04-25T10:29:24.929 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:24.925+0000 7f5db8946640 1 -- 172.21.15.37:0/668234259 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f5db40cb1b0 con 0x7f5db406daf0 2024-04-25T10:29:24.929 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:24.926+0000 7f5db8946640 1 -- 172.21.15.37:0/668234259 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f5db406a140 con 0x7f5db406daf0 2024-04-25T10:29:24.930 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:24.927+0000 7f5daaffd640 1 -- 172.21.15.37:0/668234259 <== mon.0 v2:172.21.15.37:3300/0 4 ==== mgrmap(e 5) ==== 96782+0+0 (secure 0 0 0) 0x7f5dac044020 con 0x7f5db406daf0 2024-04-25T10:29:24.930 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:24.927+0000 7f5daaffd640 1 --2- 172.21.15.37:0/668234259 >> [v2:172.21.15.203:6832/2418205237,v1:172.21.15.203:6833/2418205237] conn(0x7f5d940744b0 0x7f5d940769c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:29:24.930 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:24.927+0000 7f5daaffd640 1 -- 172.21.15.37:0/668234259 <== mon.0 v2:172.21.15.37:3300/0 5 ==== osd_map(23..23 src has 1..23) ==== 5709+0+0 (secure 0 0 0) 0x7f5dac0b5ef0 con 0x7f5db406daf0 2024-04-25T10:29:24.930 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:24.927+0000 7f5db1d9b640 1 --2- 172.21.15.37:0/668234259 >> [v2:172.21.15.203:6832/2418205237,v1:172.21.15.203:6833/2418205237] conn(0x7f5d940744b0 0x7f5d940769c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:29:24.930 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:24.928+0000 7f5db1d9b640 1 --2- 172.21.15.37:0/668234259 >> [v2:172.21.15.203:6832/2418205237,v1:172.21.15.203:6833/2418205237] conn(0x7f5d940744b0 0x7f5d940769c0 secure :-1 s=READY pgs=105 cs=0 l=1 rev1=1 crypto rx=0x7f5db40b8400 tx=0x7f5d9c008040 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:29:24.933 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:24.932+0000 7f5daaffd640 1 -- 172.21.15.37:0/668234259 <== mon.0 v2:172.21.15.37:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187246 (secure 0 0 0) 0x7f5dac04b030 con 0x7f5db406daf0 2024-04-25T10:29:25.122 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:25.120+0000 7f5db8946640 1 -- 172.21.15.37:0/668234259 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7f5db40b2000 con 0x7f5db406daf0 2024-04-25T10:29:25.123 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:25.121+0000 7f5daaffd640 1 -- 172.21.15.37:0/668234259 <== mon.0 v2:172.21.15.37:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v23) ==== 74+0+16621 (secure 0 0 0) 0x7f5dac080680 con 0x7f5db406daf0 2024-04-25T10:29:25.123 INFO:teuthology.orchestra.run.smithi037.stdout: 2024-04-25T10:29:25.124 INFO:teuthology.orchestra.run.smithi037.stdout:{"epoch":23,"fsid":"f4acda28-be34-492a-b286-994afd36f61d","created":"2024-04-25T10:28:43.985718+0000","modified":"2024-04-25T10:29:10.016188+0000","last_up_change":"2024-04-25T10:28:54.900654+0000","last_in_change":"2024-04-25T10:28:46.216973+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":3,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-04-25T10:28:56.202153+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"19","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2024-04-25T10:29:06.507244+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"22","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":2.25,"score_stable":2.25,"optimal_score":1,"raw_score_acting":2.25,"raw_score_stable":2.25,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2024-04-25T10:29:07.347844+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"23","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"a2f79028-b332-4166-9ebb-a7024a37ea22","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":21,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.37:6816","nonce":2746074720},{"type":"v1","addr":"172.21.15.37:6817","nonce":2746074720}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.37:6818","nonce":2746074720},{"type":"v1","addr":"172.21.15.37:6819","nonce":2746074720}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.37:6822","nonce":2746074720},{"type":"v1","addr":"172.21.15.37:6823","nonce":2746074720}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.37:6820","nonce":2746074720},{"type":"v1","addr":"172.21.15.37:6821","nonce":2746074720}]},"public_addr":"172.21.15.37:6817/2746074720","cluster_addr":"172.21.15.37:6819/2746074720","heartbeat_back_addr":"172.21.15.37:6823/2746074720","heartbeat_front_addr":"172.21.15.37:6821/2746074720","state":["exists","up"]},{"osd":1,"uuid":"973159de-c2ea-4083-aecc-69373d1c1132","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":21,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.37:6800","nonce":1690341662},{"type":"v1","addr":"172.21.15.37:6801","nonce":1690341662}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.37:6802","nonce":1690341662},{"type":"v1","addr":"172.21.15.37:6803","nonce":1690341662}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.37:6806","nonce":1690341662},{"type":"v1","addr":"172.21.15.37:6807","nonce":1690341662}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.37:6804","nonce":1690341662},{"type":"v1","addr":"172.21.15.37:6805","nonce":1690341662}]},"public_addr":"172.21.15.37:6801/1690341662","cluster_addr":"172.21.15.37:6803/1690341662","heartbeat_back_addr":"172.21.15.37:6807/1690341662","heartbeat_front_addr":"172.21.15.37:6805/1690341662","state":["exists","up"]},{"osd":2,"uuid":"4ae7a0e9-cb3f-484c-a75f-f3766b721c47","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":21,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.37:6808","nonce":2754669649},{"type":"v1","addr":"172.21.15.37:6809","nonce":2754669649}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.37:6810","nonce":2754669649},{"type":"v1","addr":"172.21.15.37:6811","nonce":2754669649}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.37:6814","nonce":2754669649},{"type":"v1","addr":"172.21.15.37:6815","nonce":2754669649}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.37:6812","nonce":2754669649},{"type":"v1","addr":"172.21.15.37:6813","nonce":2754669649}]},"public_addr":"172.21.15.37:6809/2754669649","cluster_addr":"172.21.15.37:6811/2754669649","heartbeat_back_addr":"172.21.15.37:6815/2754669649","heartbeat_front_addr":"172.21.15.37:6813/2754669649","state":["exists","up"]},{"osd":3,"uuid":"1ba4a746-1608-47a7-ace2-960a5cfae934","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":21,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.37:6824","nonce":1815290618},{"type":"v1","addr":"172.21.15.37:6825","nonce":1815290618}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.37:6826","nonce":1815290618},{"type":"v1","addr":"172.21.15.37:6827","nonce":1815290618}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.37:6830","nonce":1815290618},{"type":"v1","addr":"172.21.15.37:6831","nonce":1815290618}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.37:6828","nonce":1815290618},{"type":"v1","addr":"172.21.15.37:6829","nonce":1815290618}]},"public_addr":"172.21.15.37:6825/1815290618","cluster_addr":"172.21.15.37:6827/1815290618","heartbeat_back_addr":"172.21.15.37:6831/1815290618","heartbeat_front_addr":"172.21.15.37:6829/1815290618","state":["exists","up"]},{"osd":4,"uuid":"d4f59f5d-0413-4b79-87ee-bb212c1fcf62","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":21,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6800","nonce":4051532063},{"type":"v1","addr":"172.21.15.203:6801","nonce":4051532063}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6802","nonce":4051532063},{"type":"v1","addr":"172.21.15.203:6803","nonce":4051532063}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6806","nonce":4051532063},{"type":"v1","addr":"172.21.15.203:6807","nonce":4051532063}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6804","nonce":4051532063},{"type":"v1","addr":"172.21.15.203:6805","nonce":4051532063}]},"public_addr":"172.21.15.203:6801/4051532063","cluster_addr":"172.21.15.203:6803/4051532063","heartbeat_back_addr":"172.21.15.203:6807/4051532063","heartbeat_front_addr":"172.21.15.203:6805/4051532063","state":["exists","up"]},{"osd":5,"uuid":"69710a0e-41d8-4e7f-9a10-17ce540b67be","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":21,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6824","nonce":1203822890},{"type":"v1","addr":"172.21.15.203:6825","nonce":1203822890}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6826","nonce":1203822890},{"type":"v1","addr":"172.21.15.203:6827","nonce":1203822890}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6830","nonce":1203822890},{"type":"v1","addr":"172.21.15.203:6831","nonce":1203822890}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6828","nonce":1203822890},{"type":"v1","addr":"172.21.15.203:6829","nonce":1203822890}]},"public_addr":"172.21.15.203:6825/1203822890","cluster_addr":"172.21.15.203:6827/1203822890","heartbeat_back_addr":"172.21.15.203:6831/1203822890","heartbeat_front_addr":"172.21.15.203:6829/1203822890","state":["exists","up"]},{"osd":6,"uuid":"f29dd047-3b48-473d-a235-6642a6293f20","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":21,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6810","nonce":2309615014},{"type":"v1","addr":"172.21.15.203:6812","nonce":2309615014}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6814","nonce":2309615014},{"type":"v1","addr":"172.21.15.203:6816","nonce":2309615014}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6822","nonce":2309615014},{"type":"v1","addr":"172.21.15.203:6823","nonce":2309615014}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6818","nonce":2309615014},{"type":"v1","addr":"172.21.15.203:6820","nonce":2309615014}]},"public_addr":"172.21.15.203:6812/2309615014","cluster_addr":"172.21.15.203:6816/2309615014","heartbeat_back_addr":"172.21.15.203:6823/2309615014","heartbeat_front_addr":"172.21.15.203:6820/2309615014","state":["exists","up"]},{"osd":7,"uuid":"6e691644-378b-4fdc-8e86-fb5f56e31f29","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":21,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6808","nonce":3688099014},{"type":"v1","addr":"172.21.15.203:6809","nonce":3688099014}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6811","nonce":3688099014},{"type":"v1","addr":"172.21.15.203:6813","nonce":3688099014}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6819","nonce":3688099014},{"type":"v1","addr":"172.21.15.203:6821","nonce":3688099014}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6815","nonce":3688099014},{"type":"v1","addr":"172.21.15.203:6817","nonce":3688099014}]},"public_addr":"172.21.15.203:6809/3688099014","cluster_addr":"172.21.15.203:6813/3688099014","heartbeat_back_addr":"172.21.15.203:6821/3688099014","heartbeat_front_addr":"172.21.15.203:6817/3688099014","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T10:28:51.419141+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T10:28:51.366448+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T10:28:51.348276+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T10:28:51.380519+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T10:28:51.105812+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T10:28:51.384496+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T10:28:51.388755+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T10:28:51.366958+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-04-25T10:29:25.125 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:25.123+0000 7f5db8946640 1 -- 172.21.15.37:0/668234259 >> [v2:172.21.15.203:6832/2418205237,v1:172.21.15.203:6833/2418205237] conn(0x7f5d940744b0 msgr2=0x7f5d940769c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:29:25.125 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:25.123+0000 7f5db8946640 1 --2- 172.21.15.37:0/668234259 >> [v2:172.21.15.203:6832/2418205237,v1:172.21.15.203:6833/2418205237] conn(0x7f5d940744b0 0x7f5d940769c0 secure :-1 s=READY pgs=105 cs=0 l=1 rev1=1 crypto rx=0x7f5db40b8400 tx=0x7f5d9c008040 comp rx=0 tx=0).stop 2024-04-25T10:29:25.126 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:25.123+0000 7f5db8946640 1 -- 172.21.15.37:0/668234259 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f5db406daf0 msgr2=0x7f5db40cd080 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:29:25.126 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:25.123+0000 7f5db8946640 1 --2- 172.21.15.37:0/668234259 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f5db406daf0 0x7f5db40cd080 secure :-1 s=READY pgs=155 cs=0 l=1 rev1=1 crypto rx=0x7f5dac000c00 tx=0x7f5dac0046c0 comp rx=0 tx=0).stop 2024-04-25T10:29:25.126 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:25.123+0000 7f5db8946640 1 -- 172.21.15.37:0/668234259 shutdown_connections 2024-04-25T10:29:25.126 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:25.123+0000 7f5db8946640 1 --2- 172.21.15.37:0/668234259 >> [v2:172.21.15.203:6832/2418205237,v1:172.21.15.203:6833/2418205237] conn(0x7f5d940744b0 0x7f5d940769c0 unknown :-1 s=CLOSED pgs=105 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:29:25.126 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:25.123+0000 7f5db8946640 1 --2- 172.21.15.37:0/668234259 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f5db40b2dc0 0x7f5db40cbef0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:29:25.126 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:25.123+0000 7f5db8946640 1 --2- 172.21.15.37:0/668234259 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f5db406e610 0x7f5db40cb9b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:29:25.126 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:25.123+0000 7f5db8946640 1 --2- 172.21.15.37:0/668234259 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f5db406daf0 0x7f5db40cd080 unknown :-1 s=CLOSED pgs=155 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:29:25.126 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:25.123+0000 7f5db8946640 1 -- 172.21.15.37:0/668234259 >> 172.21.15.37:0/668234259 conn(0x7f5db40b5440 msgr2=0x7f5db40b8a90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:29:25.126 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:25.123+0000 7f5db8946640 1 -- 172.21.15.37:0/668234259 shutdown_connections 2024-04-25T10:29:25.126 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:25.123+0000 7f5db8946640 1 -- 172.21.15.37:0/668234259 wait complete. 2024-04-25T10:29:25.136 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2024-04-25T10:28:56.202153+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '19', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 8, 'score_stable': 8, 'optimal_score': 0.25, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 2, 'pool_name': 'cephfs_metadata', 'create_time': '2024-04-25T10:29:06.507244+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '22', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_autoscale_bias': 4, 'pg_num_min': 16, 'recovery_priority': 5}, 'application_metadata': {'cephfs': {'metadata': 'cephfs'}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 2.25, 'score_stable': 2.25, 'optimal_score': 1, 'raw_score_acting': 2.25, 'raw_score_stable': 2.25, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 3, 'pool_name': 'cephfs_data', 'create_time': '2024-04-25T10:29:07.347844+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 64, 'pg_placement_num': 64, 'pg_placement_num_target': 64, 'pg_num_target': 64, 'pg_num_pending': 64, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '23', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.8999999761581421}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 1.5, 'score_stable': 1.5, 'optimal_score': 1, 'raw_score_acting': 1.5, 'raw_score_stable': 1.5, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}] 2024-04-25T10:29:25.137 DEBUG:teuthology.orchestra.run.smithi037:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2024-04-25T10:29:25.251 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:25.250+0000 7f983f492640 1 Processor -- start 2024-04-25T10:29:25.251 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:25.250+0000 7f983f492640 1 -- start start 2024-04-25T10:29:25.252 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:25.250+0000 7f983f492640 1 --2- >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f98380b7cd0 0x7f9838160a00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:29:25.252 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:25.250+0000 7f983f492640 1 --2- >> v2:172.21.15.203:3300/0 conn(0x7f9838160f40 0x7f9838161880 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:29:25.252 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:25.250+0000 7f983f492640 1 -- --> v1:172.21.15.203:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f98380d5840 con 0x7f98380d5430 2024-04-25T10:29:25.252 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:25.250+0000 7f983f492640 1 -- --> v2:172.21.15.203:3300/0 -- mon_getmap magic: 0 -- 0x7f98380d4420 con 0x7f9838160f40 2024-04-25T10:29:25.253 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:25.250+0000 7f983f492640 1 -- --> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] -- mon_getmap magic: 0 -- 0x7f98380d4670 con 0x7f98380b7cd0 2024-04-25T10:29:25.254 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:25.251+0000 7f983d22e640 1 --1- >> v1:172.21.15.203:6789/0 conn(0x7f98380d5430 0x7f98380b75c0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.203:6789/0 says I am v1:172.21.15.37:51042/0 (socket says 172.21.15.37:51042) 2024-04-25T10:29:25.254 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:25.251+0000 7f983d22e640 1 -- 172.21.15.37:0/2794050001 learned_addr learned my addr 172.21.15.37:0/2794050001 (peer_addr_for_me v1:172.21.15.37:0/0) 2024-04-25T10:29:25.254 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:25.251+0000 7f983da2f640 1 --2- 172.21.15.37:0/2794050001 >> v2:172.21.15.203:3300/0 conn(0x7f9838160f40 0x7f9838161880 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:29:25.254 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:25.251+0000 7f983ca2d640 1 --2- 172.21.15.37:0/2794050001 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f98380b7cd0 0x7f9838160a00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:29:25.254 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:25.251+0000 7f982ffff640 1 -- 172.21.15.37:0/2794050001 <== mon.1 v1:172.21.15.203:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 1980566519 0 0) 0x7f98380d5840 con 0x7f98380d5430 2024-04-25T10:29:25.254 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:25.251+0000 7f982ffff640 1 -- 172.21.15.37:0/2794050001 --> v1:172.21.15.203:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f9824003520 con 0x7f98380d5430 2024-04-25T10:29:25.254 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:25.251+0000 7f983ca2d640 1 -- 172.21.15.37:0/2794050001 >> v2:172.21.15.203:3300/0 conn(0x7f9838160f40 msgr2=0x7f9838161880 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:29:25.254 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:25.251+0000 7f983ca2d640 1 --2- 172.21.15.37:0/2794050001 >> v2:172.21.15.203:3300/0 conn(0x7f9838160f40 0x7f9838161880 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:29:25.254 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:25.251+0000 7f983ca2d640 1 -- 172.21.15.37:0/2794050001 >> v1:172.21.15.203:6789/0 conn(0x7f98380d5430 legacy=0x7f98380b75c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:29:25.254 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:25.251+0000 7f983ca2d640 1 -- 172.21.15.37:0/2794050001 --> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f98380d4950 con 0x7f98380b7cd0 2024-04-25T10:29:25.254 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:25.251+0000 7f983da2f640 1 --2- 172.21.15.37:0/2794050001 >> v2:172.21.15.203:3300/0 conn(0x7f9838160f40 0x7f9838161880 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-25T10:29:25.256 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:25.252+0000 7f983ca2d640 1 --2- 172.21.15.37:0/2794050001 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f98380b7cd0 0x7f9838160a00 secure :-1 s=READY pgs=40 cs=0 l=1 rev1=1 crypto rx=0x7f98280097a0 tx=0x7f982802a100 comp rx=0 tx=0).ready entity=mon.2 client_cookie=c6ddf40f900fd775 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:29:25.256 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:25.252+0000 7f982ffff640 1 -- 172.21.15.37:0/2794050001 <== mon.2 v2:172.21.15.203:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9828037070 con 0x7f98380b7cd0 2024-04-25T10:29:25.256 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:25.252+0000 7f982ffff640 1 -- 172.21.15.37:0/2794050001 <== mon.2 v2:172.21.15.203:3301/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f982802ae50 con 0x7f98380b7cd0 2024-04-25T10:29:25.256 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:25.252+0000 7f982ffff640 1 -- 172.21.15.37:0/2794050001 <== mon.2 v2:172.21.15.203:3301/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9828032640 con 0x7f98380b7cd0 2024-04-25T10:29:25.256 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:25.253+0000 7f983f492640 1 -- 172.21.15.37:0/2794050001 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f98380b7cd0 msgr2=0x7f9838160a00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:29:25.256 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:25.253+0000 7f983f492640 1 --2- 172.21.15.37:0/2794050001 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f98380b7cd0 0x7f9838160a00 secure :-1 s=READY pgs=40 cs=0 l=1 rev1=1 crypto rx=0x7f98280097a0 tx=0x7f982802a100 comp rx=0 tx=0).stop 2024-04-25T10:29:25.256 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:25.253+0000 7f983f492640 1 -- 172.21.15.37:0/2794050001 shutdown_connections 2024-04-25T10:29:25.256 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:25.253+0000 7f983f492640 1 --2- 172.21.15.37:0/2794050001 >> v2:172.21.15.203:3300/0 conn(0x7f9838160f40 0x7f9838161880 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:29:25.256 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:25.253+0000 7f983f492640 1 --2- 172.21.15.37:0/2794050001 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f98380b7cd0 0x7f9838160a00 unknown :-1 s=CLOSED pgs=40 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:29:25.256 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:25.253+0000 7f983f492640 1 -- 172.21.15.37:0/2794050001 >> 172.21.15.37:0/2794050001 conn(0x7f98380d27a0 msgr2=0x7f98380d2bb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:29:25.256 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:25.253+0000 7f983f492640 1 -- 172.21.15.37:0/2794050001 shutdown_connections 2024-04-25T10:29:25.257 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:25.253+0000 7f983f492640 1 -- 172.21.15.37:0/2794050001 wait complete. 2024-04-25T10:29:25.257 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:25.256+0000 7f983f492640 1 Processor -- start 2024-04-25T10:29:25.257 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:25.256+0000 7f983f492640 1 -- start start 2024-04-25T10:29:25.260 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:25.257+0000 7f983f492640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f98380d5430 0x7f983814fe60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:29:25.260 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:25.257+0000 7f983f492640 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f9838160f40 0x7f98380cf3c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:29:25.260 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:25.257+0000 7f983f492640 1 --2- >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f98380cd470 0x7f98380cd880 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:29:25.260 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:25.257+0000 7f983f492640 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 -- 0x7f9838161350 con 0x7f98380d5430 2024-04-25T10:29:25.260 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:25.257+0000 7f983f492640 1 -- --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_getmap magic: 0 -- 0x7f98380d4f10 con 0x7f9838160f40 2024-04-25T10:29:25.260 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:25.257+0000 7f983f492640 1 -- --> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] -- mon_getmap magic: 0 -- 0x7f98381614d0 con 0x7f98380cd470 2024-04-25T10:29:25.260 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:25.257+0000 7f983d22e640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f98380d5430 0x7f983814fe60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:29:25.260 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:25.257+0000 7f983d22e640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f98380d5430 0x7f983814fe60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.37:3300/0 says I am v2:172.21.15.37:35218/0 (socket says 172.21.15.37:35218) 2024-04-25T10:29:25.260 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:25.257+0000 7f983d22e640 1 -- 172.21.15.37:0/3951360747 learned_addr learned my addr 172.21.15.37:0/3951360747 (peer_addr_for_me v2:172.21.15.37:0/0) 2024-04-25T10:29:25.261 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:25.257+0000 7f983ca2d640 1 --2- 172.21.15.37:0/3951360747 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f9838160f40 0x7f98380cf3c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:29:25.261 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:25.257+0000 7f983da2f640 1 --2- 172.21.15.37:0/3951360747 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f98380cd470 0x7f98380cd880 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:29:25.261 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:25.257+0000 7f983d22e640 1 -- 172.21.15.37:0/3951360747 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f98380cd470 msgr2=0x7f98380cd880 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:29:25.261 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:25.257+0000 7f983d22e640 1 --2- 172.21.15.37:0/3951360747 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f98380cd470 0x7f98380cd880 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:29:25.261 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:25.257+0000 7f983d22e640 1 -- 172.21.15.37:0/3951360747 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f9838160f40 msgr2=0x7f98380cf3c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:29:25.261 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:25.257+0000 7f983d22e640 1 --2- 172.21.15.37:0/3951360747 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f9838160f40 0x7f98380cf3c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:29:25.261 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:25.257+0000 7f983d22e640 1 -- 172.21.15.37:0/3951360747 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f98381f5f10 con 0x7f98380d5430 2024-04-25T10:29:25.261 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:25.257+0000 7f983d22e640 1 --2- 172.21.15.37:0/3951360747 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f98380d5430 0x7f983814fe60 secure :-1 s=READY pgs=156 cs=0 l=1 rev1=1 crypto rx=0x7f983400c390 tx=0x7f983400c850 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:29:25.261 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:25.257+0000 7f982dffb640 1 -- 172.21.15.37:0/3951360747 <== mon.0 v2:172.21.15.37:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9834017070 con 0x7f98380d5430 2024-04-25T10:29:25.261 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:25.257+0000 7f982dffb640 1 -- 172.21.15.37:0/3951360747 <== mon.0 v2:172.21.15.37:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f9834005d00 con 0x7f98380d5430 2024-04-25T10:29:25.261 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:25.257+0000 7f982dffb640 1 -- 172.21.15.37:0/3951360747 <== mon.0 v2:172.21.15.37:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9834007310 con 0x7f98380d5430 2024-04-25T10:29:25.261 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:25.257+0000 7f983f492640 1 -- 172.21.15.37:0/3951360747 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f98381503a0 con 0x7f98380d5430 2024-04-25T10:29:25.261 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:25.257+0000 7f983f492640 1 -- 172.21.15.37:0/3951360747 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f98380b9cf0 con 0x7f98380d5430 2024-04-25T10:29:25.263 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:25.259+0000 7f983f492640 1 -- 172.21.15.37:0/3951360747 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f983806a140 con 0x7f98380d5430 2024-04-25T10:29:25.264 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:25.260+0000 7f982dffb640 1 -- 172.21.15.37:0/3951360747 <== mon.0 v2:172.21.15.37:3300/0 4 ==== mgrmap(e 5) ==== 96782+0+0 (secure 0 0 0) 0x7f983401a070 con 0x7f98380d5430 2024-04-25T10:29:25.264 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:25.260+0000 7f982dffb640 1 --2- 172.21.15.37:0/3951360747 >> [v2:172.21.15.203:6832/2418205237,v1:172.21.15.203:6833/2418205237] conn(0x7f981c0744b0 0x7f981c0769c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:29:25.264 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:25.260+0000 7f982dffb640 1 -- 172.21.15.37:0/3951360747 <== mon.0 v2:172.21.15.37:3300/0 5 ==== osd_map(23..23 src has 1..23) ==== 5709+0+0 (secure 0 0 0) 0x7f9834094490 con 0x7f98380d5430 2024-04-25T10:29:25.264 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:25.260+0000 7f983ca2d640 1 --2- 172.21.15.37:0/3951360747 >> [v2:172.21.15.203:6832/2418205237,v1:172.21.15.203:6833/2418205237] conn(0x7f981c0744b0 0x7f981c0769c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:29:25.264 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:25.261+0000 7f983ca2d640 1 --2- 172.21.15.37:0/3951360747 >> [v2:172.21.15.203:6832/2418205237,v1:172.21.15.203:6833/2418205237] conn(0x7f981c0744b0 0x7f981c0769c0 secure :-1 s=READY pgs=106 cs=0 l=1 rev1=1 crypto rx=0x7f9828034000 tx=0x7f982802e000 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:29:25.266 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:25.265+0000 7f982dffb640 1 -- 172.21.15.37:0/3951360747 <== mon.0 v2:172.21.15.37:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187246 (secure 0 0 0) 0x7f983405e660 con 0x7f98380d5430 2024-04-25T10:29:25.453 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:25.452+0000 7f983f492640 1 -- 172.21.15.37:0/3951360747 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) -- 0x7f98380cf900 con 0x7f98380d5430 2024-04-25T10:29:25.453 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:25.452+0000 7f982dffb640 1 -- 172.21.15.37:0/3951360747 <== mon.0 v2:172.21.15.37:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v23) ==== 93+0+10 (secure 0 0 0) 0x7f9834062960 con 0x7f98380d5430 2024-04-25T10:29:25.454 INFO:teuthology.orchestra.run.smithi037.stdout:pg_num: 1 2024-04-25T10:29:25.456 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:25.454+0000 7f983f492640 1 -- 172.21.15.37:0/3951360747 >> [v2:172.21.15.203:6832/2418205237,v1:172.21.15.203:6833/2418205237] conn(0x7f981c0744b0 msgr2=0x7f981c0769c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:29:25.456 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:25.454+0000 7f983f492640 1 --2- 172.21.15.37:0/3951360747 >> [v2:172.21.15.203:6832/2418205237,v1:172.21.15.203:6833/2418205237] conn(0x7f981c0744b0 0x7f981c0769c0 secure :-1 s=READY pgs=106 cs=0 l=1 rev1=1 crypto rx=0x7f9828034000 tx=0x7f982802e000 comp rx=0 tx=0).stop 2024-04-25T10:29:25.456 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:25.454+0000 7f983f492640 1 -- 172.21.15.37:0/3951360747 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f98380d5430 msgr2=0x7f983814fe60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:29:25.456 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:25.454+0000 7f983f492640 1 --2- 172.21.15.37:0/3951360747 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f98380d5430 0x7f983814fe60 secure :-1 s=READY pgs=156 cs=0 l=1 rev1=1 crypto rx=0x7f983400c390 tx=0x7f983400c850 comp rx=0 tx=0).stop 2024-04-25T10:29:25.456 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:25.454+0000 7f983f492640 1 -- 172.21.15.37:0/3951360747 shutdown_connections 2024-04-25T10:29:25.456 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:25.454+0000 7f983f492640 1 --2- 172.21.15.37:0/3951360747 >> [v2:172.21.15.203:6832/2418205237,v1:172.21.15.203:6833/2418205237] conn(0x7f981c0744b0 0x7f981c0769c0 unknown :-1 s=CLOSED pgs=106 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:29:25.456 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:25.454+0000 7f983f492640 1 --2- 172.21.15.37:0/3951360747 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f98380cd470 0x7f98380cd880 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:29:25.456 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:25.454+0000 7f983f492640 1 --2- 172.21.15.37:0/3951360747 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f9838160f40 0x7f98380cf3c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:29:25.456 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:25.454+0000 7f983f492640 1 --2- 172.21.15.37:0/3951360747 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f98380d5430 0x7f983814fe60 unknown :-1 s=CLOSED pgs=156 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:29:25.456 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:25.454+0000 7f983f492640 1 -- 172.21.15.37:0/3951360747 >> 172.21.15.37:0/3951360747 conn(0x7f98380d27a0 msgr2=0x7f983815f440 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:29:25.457 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:25.454+0000 7f983f492640 1 -- 172.21.15.37:0/3951360747 shutdown_connections 2024-04-25T10:29:25.457 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:25.454+0000 7f983f492640 1 -- 172.21.15.37:0/3951360747 wait complete. 2024-04-25T10:29:25.467 DEBUG:teuthology.orchestra.run.smithi037:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_metadata pg_num 2024-04-25T10:29:25.582 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:25.580+0000 7f96fbfb4640 1 Processor -- start 2024-04-25T10:29:25.582 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:25.581+0000 7f96fbfb4640 1 -- start start 2024-04-25T10:29:25.584 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:25.582+0000 7f96fbfb4640 1 --2- >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f96f40b4bb0 0x7f96f41646b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:29:25.584 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:25.582+0000 7f96fbfb4640 1 --2- >> v2:172.21.15.37:3300/0 conn(0x7f96f40b7600 0x7f96f40b7a10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:29:25.585 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:25.582+0000 7f96fbfb4640 1 -- --> v1:172.21.15.203:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f96f40b7f50 con 0x7f96f40d4560 2024-04-25T10:29:25.585 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:25.582+0000 7f96fbfb4640 1 -- --> v2:172.21.15.37:3300/0 -- mon_getmap magic: 0 -- 0x7f96f40d5080 con 0x7f96f40b7600 2024-04-25T10:29:25.585 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:25.582+0000 7f96fbfb4640 1 -- --> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] -- mon_getmap magic: 0 -- 0x7f96f4161190 con 0x7f96f40b4bb0 2024-04-25T10:29:25.585 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:25.582+0000 7f96f954f640 1 --2- >> v2:172.21.15.37:3300/0 conn(0x7f96f40b7600 0x7f96f40b7a10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:29:25.585 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:25.582+0000 7f96f954f640 1 --2- >> v2:172.21.15.37:3300/0 conn(0x7f96f40b7600 0x7f96f40b7a10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.37:3300/0 says I am v2:172.21.15.37:35230/0 (socket says 172.21.15.37:35230) 2024-04-25T10:29:25.585 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:25.582+0000 7f96f954f640 1 -- 172.21.15.37:0/1244722972 learned_addr learned my addr 172.21.15.37:0/1244722972 (peer_addr_for_me v2:172.21.15.37:0/0) 2024-04-25T10:29:25.585 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:25.582+0000 7f96f954f640 1 -- 172.21.15.37:0/1244722972 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f96f40b4bb0 msgr2=0x7f96f41646b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:29:25.585 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:25.582+0000 7f96f9d50640 1 --2- 172.21.15.37:0/1244722972 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f96f40b4bb0 0x7f96f41646b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:29:25.585 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:25.583+0000 7f96f954f640 1 --2- 172.21.15.37:0/1244722972 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f96f40b4bb0 0x7f96f41646b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:29:25.585 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:25.583+0000 7f96f954f640 1 -- 172.21.15.37:0/1244722972 >> v1:172.21.15.203:6789/0 conn(0x7f96f40d4560 legacy=0x7f96f40d4970 unknown :-1 s=STATE_CONNECTING_RE l=0).mark_down 2024-04-25T10:29:25.585 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:25.583+0000 7f96f954f640 1 -- 172.21.15.37:0/1244722972 --> v2:172.21.15.37:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f96f4161470 con 0x7f96f40b7600 2024-04-25T10:29:25.585 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:25.583+0000 7f96f9d50640 1 --2- 172.21.15.37:0/1244722972 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f96f40b4bb0 0x7f96f41646b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T10:29:25.585 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:25.583+0000 7f96f954f640 1 --2- 172.21.15.37:0/1244722972 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f96f40b7600 0x7f96f40b7a10 secure :-1 s=READY pgs=157 cs=0 l=1 rev1=1 crypto rx=0x7f96e8009870 tx=0x7f96e802a210 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d3ce807ce57484e2 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:29:25.585 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:25.583+0000 7f96f8d4e640 1 -- 172.21.15.37:0/1244722972 <== mon.0 v2:172.21.15.37:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f96e8037070 con 0x7f96f40b7600 2024-04-25T10:29:25.586 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:25.583+0000 7f96f8d4e640 1 -- 172.21.15.37:0/1244722972 <== mon.0 v2:172.21.15.37:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f96e8032440 con 0x7f96f40b7600 2024-04-25T10:29:25.586 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:25.583+0000 7f96f8d4e640 1 -- 172.21.15.37:0/1244722972 <== mon.0 v2:172.21.15.37:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f96e8032810 con 0x7f96f40b7600 2024-04-25T10:29:25.586 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:25.584+0000 7f96fbfb4640 1 -- 172.21.15.37:0/1244722972 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f96f40b7600 msgr2=0x7f96f40b7a10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:29:25.587 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:25.584+0000 7f96fbfb4640 1 --2- 172.21.15.37:0/1244722972 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f96f40b7600 0x7f96f40b7a10 secure :-1 s=READY pgs=157 cs=0 l=1 rev1=1 crypto rx=0x7f96e8009870 tx=0x7f96e802a210 comp rx=0 tx=0).stop 2024-04-25T10:29:25.587 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:25.584+0000 7f96fbfb4640 1 -- 172.21.15.37:0/1244722972 shutdown_connections 2024-04-25T10:29:25.587 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:25.584+0000 7f96fbfb4640 1 --2- 172.21.15.37:0/1244722972 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f96f40b7600 0x7f96f40b7a10 unknown :-1 s=CLOSED pgs=157 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:29:25.587 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:25.584+0000 7f96fbfb4640 1 --2- 172.21.15.37:0/1244722972 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f96f40b4bb0 0x7f96f41646b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:29:25.587 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:25.584+0000 7f96fbfb4640 1 -- 172.21.15.37:0/1244722972 >> 172.21.15.37:0/1244722972 conn(0x7f96f40d27e0 msgr2=0x7f96f40d2bf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:29:25.587 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:25.584+0000 7f96fbfb4640 1 -- 172.21.15.37:0/1244722972 shutdown_connections 2024-04-25T10:29:25.587 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:25.584+0000 7f96fbfb4640 1 -- 172.21.15.37:0/1244722972 wait complete. 2024-04-25T10:29:25.588 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:25.586+0000 7f96fbfb4640 1 Processor -- start 2024-04-25T10:29:25.588 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:25.586+0000 7f96fbfb4640 1 -- start start 2024-04-25T10:29:25.590 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:25.587+0000 7f96fbfb4640 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f96f40b4bb0 0x7f96f40cf830 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:29:25.590 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:25.587+0000 7f96fbfb4640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f96f40b7600 0x7f96f41f0410 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:29:25.590 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:25.587+0000 7f96fbfb4640 1 --2- >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f96f40d4560 0x7f96f41f9aa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:29:25.590 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:25.587+0000 7f96fbfb4640 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 -- 0x7f96f40b4fc0 con 0x7f96f40b7600 2024-04-25T10:29:25.590 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:25.587+0000 7f96fbfb4640 1 -- --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_getmap magic: 0 -- 0x7f96f4164190 con 0x7f96f40b4bb0 2024-04-25T10:29:25.590 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:25.587+0000 7f96fbfb4640 1 -- --> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] -- mon_getmap magic: 0 -- 0x7f96f4164bf0 con 0x7f96f40d4560 2024-04-25T10:29:25.591 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:25.587+0000 7f96f954f640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f96f40b7600 0x7f96f41f0410 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:29:25.591 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:25.587+0000 7f96f954f640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f96f40b7600 0x7f96f41f0410 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.37:3300/0 says I am v2:172.21.15.37:35236/0 (socket says 172.21.15.37:35236) 2024-04-25T10:29:25.591 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:25.587+0000 7f96f954f640 1 -- 172.21.15.37:0/3796724018 learned_addr learned my addr 172.21.15.37:0/3796724018 (peer_addr_for_me v2:172.21.15.37:0/0) 2024-04-25T10:29:25.591 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:25.588+0000 7f96fa551640 1 --2- 172.21.15.37:0/3796724018 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f96f40d4560 0x7f96f41f9aa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:29:25.591 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:25.588+0000 7f96f954f640 1 -- 172.21.15.37:0/3796724018 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f96f40d4560 msgr2=0x7f96f41f9aa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:29:25.591 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:25.588+0000 7f96f954f640 1 --2- 172.21.15.37:0/3796724018 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f96f40d4560 0x7f96f41f9aa0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:29:25.591 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:25.588+0000 7f96f954f640 1 -- 172.21.15.37:0/3796724018 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f96f40b4bb0 msgr2=0x7f96f40cf830 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:29:25.591 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:25.588+0000 7f96f954f640 1 --2- 172.21.15.37:0/3796724018 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f96f40b4bb0 0x7f96f40cf830 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:29:25.591 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:25.588+0000 7f96f954f640 1 -- 172.21.15.37:0/3796724018 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f96f40cde20 con 0x7f96f40b7600 2024-04-25T10:29:25.591 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:25.588+0000 7f96f954f640 1 --2- 172.21.15.37:0/3796724018 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f96f40b7600 0x7f96f41f0410 secure :-1 s=READY pgs=158 cs=0 l=1 rev1=1 crypto rx=0x7f96e8000c00 tx=0x7f96e80046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:29:25.591 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:25.588+0000 7f96f27fc640 1 -- 172.21.15.37:0/3796724018 <== mon.0 v2:172.21.15.37:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f96e8037040 con 0x7f96f40b7600 2024-04-25T10:29:25.591 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:25.588+0000 7f96f27fc640 1 -- 172.21.15.37:0/3796724018 <== mon.0 v2:172.21.15.37:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f96e8042070 con 0x7f96f40b7600 2024-04-25T10:29:25.591 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:25.588+0000 7f96fbfb4640 1 -- 172.21.15.37:0/3796724018 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f96f41f0950 con 0x7f96f40b7600 2024-04-25T10:29:25.592 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:25.588+0000 7f96f27fc640 1 -- 172.21.15.37:0/3796724018 <== mon.0 v2:172.21.15.37:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f96e8030bb0 con 0x7f96f40b7600 2024-04-25T10:29:25.592 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:25.588+0000 7f96fbfb4640 1 -- 172.21.15.37:0/3796724018 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f96f40cafc0 con 0x7f96f40b7600 2024-04-25T10:29:25.592 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:25.589+0000 7f96fbfb4640 1 -- 172.21.15.37:0/3796724018 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f96f40cdb90 con 0x7f96f40b7600 2024-04-25T10:29:25.593 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:25.590+0000 7f96f27fc640 1 -- 172.21.15.37:0/3796724018 <== mon.0 v2:172.21.15.37:3300/0 4 ==== mgrmap(e 5) ==== 96782+0+0 (secure 0 0 0) 0x7f96e8047460 con 0x7f96f40b7600 2024-04-25T10:29:25.593 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:25.590+0000 7f96f27fc640 1 --2- 172.21.15.37:0/3796724018 >> [v2:172.21.15.203:6832/2418205237,v1:172.21.15.203:6833/2418205237] conn(0x7f96e00744e0 0x7f96e00769f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:29:25.593 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:25.591+0000 7f96f9d50640 1 --2- 172.21.15.37:0/3796724018 >> [v2:172.21.15.203:6832/2418205237,v1:172.21.15.203:6833/2418205237] conn(0x7f96e00744e0 0x7f96e00769f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:29:25.593 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:25.591+0000 7f96f27fc640 1 -- 172.21.15.37:0/3796724018 <== mon.0 v2:172.21.15.37:3300/0 5 ==== osd_map(23..23 src has 1..23) ==== 5709+0+0 (secure 0 0 0) 0x7f96e80b5f60 con 0x7f96f40b7600 2024-04-25T10:29:25.593 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:25.591+0000 7f96f9d50640 1 --2- 172.21.15.37:0/3796724018 >> [v2:172.21.15.203:6832/2418205237,v1:172.21.15.203:6833/2418205237] conn(0x7f96e00744e0 0x7f96e00769f0 secure :-1 s=READY pgs=107 cs=0 l=1 rev1=1 crypto rx=0x7f96ec006fd0 tx=0x7f96ec008040 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:29:25.596 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:25.595+0000 7f96f27fc640 1 -- 172.21.15.37:0/3796724018 <== mon.0 v2:172.21.15.37:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187246 (secure 0 0 0) 0x7f96f40b7f50 con 0x7f96f40b7600 2024-04-25T10:29:25.782 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:25.781+0000 7f96fbfb4640 1 -- 172.21.15.37:0/3796724018 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"} v 0) -- 0x7f96f406a140 con 0x7f96f40b7600 2024-04-25T10:29:25.782 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:25.781+0000 7f96f27fc640 1 -- 172.21.15.37:0/3796724018 <== mon.0 v2:172.21.15.37:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]=0 v23) ==== 104+0+11 (secure 0 0 0) 0x7f96e80806e0 con 0x7f96f40b7600 2024-04-25T10:29:25.782 INFO:teuthology.orchestra.run.smithi037.stdout:pg_num: 32 2024-04-25T10:29:25.784 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:25.783+0000 7f96fbfb4640 1 -- 172.21.15.37:0/3796724018 >> [v2:172.21.15.203:6832/2418205237,v1:172.21.15.203:6833/2418205237] conn(0x7f96e00744e0 msgr2=0x7f96e00769f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:29:25.785 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:25.783+0000 7f96fbfb4640 1 --2- 172.21.15.37:0/3796724018 >> [v2:172.21.15.203:6832/2418205237,v1:172.21.15.203:6833/2418205237] conn(0x7f96e00744e0 0x7f96e00769f0 secure :-1 s=READY pgs=107 cs=0 l=1 rev1=1 crypto rx=0x7f96ec006fd0 tx=0x7f96ec008040 comp rx=0 tx=0).stop 2024-04-25T10:29:25.785 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:25.783+0000 7f96fbfb4640 1 -- 172.21.15.37:0/3796724018 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f96f40b7600 msgr2=0x7f96f41f0410 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:29:25.785 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:25.783+0000 7f96fbfb4640 1 --2- 172.21.15.37:0/3796724018 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f96f40b7600 0x7f96f41f0410 secure :-1 s=READY pgs=158 cs=0 l=1 rev1=1 crypto rx=0x7f96e8000c00 tx=0x7f96e80046c0 comp rx=0 tx=0).stop 2024-04-25T10:29:25.785 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:25.783+0000 7f96fbfb4640 1 -- 172.21.15.37:0/3796724018 shutdown_connections 2024-04-25T10:29:25.785 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:25.783+0000 7f96fbfb4640 1 --2- 172.21.15.37:0/3796724018 >> [v2:172.21.15.203:6832/2418205237,v1:172.21.15.203:6833/2418205237] conn(0x7f96e00744e0 0x7f96e00769f0 unknown :-1 s=CLOSED pgs=107 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:29:25.785 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:25.783+0000 7f96fbfb4640 1 --2- 172.21.15.37:0/3796724018 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f96f40d4560 0x7f96f41f9aa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:29:25.785 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:25.783+0000 7f96fbfb4640 1 --2- 172.21.15.37:0/3796724018 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f96f40b7600 0x7f96f41f0410 unknown :-1 s=CLOSED pgs=158 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:29:25.785 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:25.783+0000 7f96fbfb4640 1 --2- 172.21.15.37:0/3796724018 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f96f40b4bb0 0x7f96f40cf830 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:29:25.785 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:25.783+0000 7f96fbfb4640 1 -- 172.21.15.37:0/3796724018 >> 172.21.15.37:0/3796724018 conn(0x7f96f40d27e0 msgr2=0x7f96f40b6e10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:29:25.785 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:25.783+0000 7f96fbfb4640 1 -- 172.21.15.37:0/3796724018 shutdown_connections 2024-04-25T10:29:25.785 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:25.783+0000 7f96fbfb4640 1 -- 172.21.15.37:0/3796724018 wait complete. 2024-04-25T10:29:25.795 DEBUG:teuthology.orchestra.run.smithi037:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data pg_num 2024-04-25T10:29:25.909 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:25.908+0000 7f8e6b282640 1 Processor -- start 2024-04-25T10:29:25.910 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:25.908+0000 7f8e6b282640 1 -- start start 2024-04-25T10:29:25.911 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:25.910+0000 7f8e6b282640 1 --2- >> v2:172.21.15.37:3300/0 conn(0x7f8e640d5430 0x7f8e640b75c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:29:25.912 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:25.910+0000 7f8e6b282640 1 -- --> v1:172.21.15.37:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f8e640b7f10 con 0x7f8e640b7b00 2024-04-25T10:29:25.912 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:25.910+0000 7f8e6b282640 1 -- --> v1:172.21.15.203:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f8e64161520 con 0x7f8e64161110 2024-04-25T10:29:25.912 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:25.910+0000 7f8e6b282640 1 -- --> v2:172.21.15.37:3300/0 -- mon_getmap magic: 0 -- 0x7f8e640d4420 con 0x7f8e640d5430 2024-04-25T10:29:25.912 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:25.910+0000 7f8e6901e640 1 --2- >> v2:172.21.15.37:3300/0 conn(0x7f8e640d5430 0x7f8e640b75c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:29:25.912 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:25.910+0000 7f8e6881d640 1 --1- >> v1:172.21.15.37:6789/0 conn(0x7f8e640b7b00 0x7f8e64160a00 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.37:6789/0 says I am v1:172.21.15.37:59878/0 (socket says 172.21.15.37:59878) 2024-04-25T10:29:25.912 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:25.910+0000 7f8e6881d640 1 -- 172.21.15.37:0/2027282817 learned_addr learned my addr 172.21.15.37:0/2027282817 (peer_addr_for_me v1:172.21.15.37:0/0) 2024-04-25T10:29:25.912 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:25.910+0000 7f8e6901e640 1 --2- >> v2:172.21.15.37:3300/0 conn(0x7f8e640d5430 0x7f8e640b75c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.37:3300/0 says I am v2:172.21.15.37:35240/0 (socket says 172.21.15.37:35240) 2024-04-25T10:29:25.912 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:25.910+0000 7f8e6901e640 1 -- 172.21.15.37:0/2027282817 >> v1:172.21.15.203:6789/0 conn(0x7f8e64161110 legacy=0x7f8e64161880 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:29:25.912 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:25.910+0000 7f8e6901e640 1 -- 172.21.15.37:0/2027282817 >> v1:172.21.15.37:6789/0 conn(0x7f8e640b7b00 legacy=0x7f8e64160a00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:29:25.912 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:25.910+0000 7f8e63fff640 1 -- 172.21.15.37:0/2027282817 <== mon.0 v1:172.21.15.37:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 4069331004 0 0) 0x7f8e58001110 con 0x7f8e640b7b00 2024-04-25T10:29:25.912 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:25.910+0000 7f8e6901e640 1 -- 172.21.15.37:0/2027282817 --> v2:172.21.15.37:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f8e640d5840 con 0x7f8e640d5430 2024-04-25T10:29:25.914 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:25.911+0000 7f8e6901e640 1 --2- 172.21.15.37:0/2027282817 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f8e640d5430 0x7f8e640b75c0 secure :-1 s=READY pgs=160 cs=0 l=1 rev1=1 crypto rx=0x7f8e641616f0 tx=0x7f8e5c02a130 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ef2a807c620ca0f3 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:29:25.914 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:25.911+0000 7f8e63fff640 1 -- 172.21.15.37:0/2027282817 <== mon.0 v2:172.21.15.37:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8e5c037070 con 0x7f8e640d5430 2024-04-25T10:29:25.914 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:25.911+0000 7f8e63fff640 1 -- 172.21.15.37:0/2027282817 <== mon.0 v2:172.21.15.37:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f8e5c02ae40 con 0x7f8e640d5430 2024-04-25T10:29:25.914 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:25.911+0000 7f8e63fff640 1 -- 172.21.15.37:0/2027282817 <== mon.0 v2:172.21.15.37:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8e5c032660 con 0x7f8e640d5430 2024-04-25T10:29:25.914 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:25.912+0000 7f8e6b282640 1 -- 172.21.15.37:0/2027282817 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f8e640d5430 msgr2=0x7f8e640b75c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:29:25.914 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:25.912+0000 7f8e6b282640 1 --2- 172.21.15.37:0/2027282817 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f8e640d5430 0x7f8e640b75c0 secure :-1 s=READY pgs=160 cs=0 l=1 rev1=1 crypto rx=0x7f8e641616f0 tx=0x7f8e5c02a130 comp rx=0 tx=0).stop 2024-04-25T10:29:25.915 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:25.912+0000 7f8e6b282640 1 -- 172.21.15.37:0/2027282817 shutdown_connections 2024-04-25T10:29:25.915 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:25.912+0000 7f8e6b282640 1 --2- 172.21.15.37:0/2027282817 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f8e640d5430 0x7f8e640b75c0 unknown :-1 s=CLOSED pgs=160 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:29:25.915 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:25.912+0000 7f8e6b282640 1 -- 172.21.15.37:0/2027282817 >> 172.21.15.37:0/2027282817 conn(0x7f8e640d27a0 msgr2=0x7f8e640d2bb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:29:25.915 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:25.912+0000 7f8e6b282640 1 -- 172.21.15.37:0/2027282817 shutdown_connections 2024-04-25T10:29:25.915 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:25.912+0000 7f8e6b282640 1 -- 172.21.15.37:0/2027282817 wait complete. 2024-04-25T10:29:25.915 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:25.914+0000 7f8e6b282640 1 Processor -- start 2024-04-25T10:29:25.915 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:25.914+0000 7f8e6b282640 1 -- start start 2024-04-25T10:29:25.918 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:25.915+0000 7f8e6b282640 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f8e640b7b00 0x7f8e6414cc20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:29:25.918 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:25.915+0000 7f8e6b282640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f8e640d5430 0x7f8e641e9340 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:29:25.918 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:25.915+0000 7f8e6b282640 1 --2- >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f8e64161110 0x7f8e640cb370 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:29:25.918 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:25.915+0000 7f8e6b282640 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 -- 0x7f8e640d4c30 con 0x7f8e640d5430 2024-04-25T10:29:25.918 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:25.915+0000 7f8e6b282640 1 -- --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_getmap magic: 0 -- 0x7f8e640d4950 con 0x7f8e640b7b00 2024-04-25T10:29:25.919 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:25.915+0000 7f8e6b282640 1 -- --> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] -- mon_getmap magic: 0 -- 0x7f8e640d4f10 con 0x7f8e64161110 2024-04-25T10:29:25.919 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:25.915+0000 7f8e6881d640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f8e640d5430 0x7f8e641e9340 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:29:25.919 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:25.915+0000 7f8e6881d640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f8e640d5430 0x7f8e641e9340 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.37:3300/0 says I am v2:172.21.15.37:35248/0 (socket says 172.21.15.37:35248) 2024-04-25T10:29:25.919 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:25.915+0000 7f8e6881d640 1 -- 172.21.15.37:0/3882824477 learned_addr learned my addr 172.21.15.37:0/3882824477 (peer_addr_for_me v2:172.21.15.37:0/0) 2024-04-25T10:29:25.919 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:25.915+0000 7f8e6901e640 1 --2- 172.21.15.37:0/3882824477 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f8e640b7b00 0x7f8e6414cc20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:29:25.919 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:25.915+0000 7f8e6981f640 1 --2- 172.21.15.37:0/3882824477 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f8e64161110 0x7f8e640cb370 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:29:25.919 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:25.915+0000 7f8e6881d640 1 -- 172.21.15.37:0/3882824477 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f8e64161110 msgr2=0x7f8e640cb370 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:29:25.919 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:25.915+0000 7f8e6881d640 1 --2- 172.21.15.37:0/3882824477 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f8e64161110 0x7f8e640cb370 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:29:25.919 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:25.915+0000 7f8e6881d640 1 -- 172.21.15.37:0/3882824477 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f8e640b7b00 msgr2=0x7f8e6414cc20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:29:25.919 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:25.915+0000 7f8e6881d640 1 --2- 172.21.15.37:0/3882824477 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f8e640b7b00 0x7f8e6414cc20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:29:25.919 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:25.915+0000 7f8e6881d640 1 -- 172.21.15.37:0/3882824477 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f8e641498d0 con 0x7f8e640d5430 2024-04-25T10:29:25.919 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:25.915+0000 7f8e6901e640 1 --2- 172.21.15.37:0/3882824477 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f8e640b7b00 0x7f8e6414cc20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T10:29:25.919 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:25.915+0000 7f8e6981f640 1 --2- 172.21.15.37:0/3882824477 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f8e64161110 0x7f8e640cb370 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T10:29:25.919 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:25.916+0000 7f8e6881d640 1 --2- 172.21.15.37:0/3882824477 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f8e640d5430 0x7f8e641e9340 secure :-1 s=READY pgs=161 cs=0 l=1 rev1=1 crypto rx=0x7f8e5800c430 tx=0x7f8e5800c8f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:29:25.920 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:25.916+0000 7f8e61ffb640 1 -- 172.21.15.37:0/3882824477 <== mon.0 v2:172.21.15.37:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8e580074e0 con 0x7f8e640d5430 2024-04-25T10:29:25.920 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:25.916+0000 7f8e61ffb640 1 -- 172.21.15.37:0/3882824477 <== mon.0 v2:172.21.15.37:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f8e580047e0 con 0x7f8e640d5430 2024-04-25T10:29:25.920 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:25.916+0000 7f8e6b282640 1 -- 172.21.15.37:0/3882824477 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f8e64149bb0 con 0x7f8e640d5430 2024-04-25T10:29:25.920 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:25.916+0000 7f8e61ffb640 1 -- 172.21.15.37:0/3882824477 <== mon.0 v2:172.21.15.37:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8e58004bd0 con 0x7f8e640d5430 2024-04-25T10:29:25.920 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:25.916+0000 7f8e6b282640 1 -- 172.21.15.37:0/3882824477 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f8e6414a170 con 0x7f8e640d5430 2024-04-25T10:29:25.920 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:25.917+0000 7f8e6b282640 1 -- 172.21.15.37:0/3882824477 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f8e6406a140 con 0x7f8e640d5430 2024-04-25T10:29:25.921 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:25.918+0000 7f8e61ffb640 1 -- 172.21.15.37:0/3882824477 <== mon.0 v2:172.21.15.37:3300/0 4 ==== mgrmap(e 5) ==== 96782+0+0 (secure 0 0 0) 0x7f8e58011070 con 0x7f8e640d5430 2024-04-25T10:29:25.921 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:25.918+0000 7f8e61ffb640 1 --2- 172.21.15.37:0/3882824477 >> [v2:172.21.15.203:6832/2418205237,v1:172.21.15.203:6833/2418205237] conn(0x7f8e500744b0 0x7f8e500769c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:29:25.921 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:25.918+0000 7f8e61ffb640 1 -- 172.21.15.37:0/3882824477 <== mon.0 v2:172.21.15.37:3300/0 5 ==== osd_map(23..23 src has 1..23) ==== 5709+0+0 (secure 0 0 0) 0x7f8e58094ed0 con 0x7f8e640d5430 2024-04-25T10:29:25.921 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:25.919+0000 7f8e6901e640 1 --2- 172.21.15.37:0/3882824477 >> [v2:172.21.15.203:6832/2418205237,v1:172.21.15.203:6833/2418205237] conn(0x7f8e500744b0 0x7f8e500769c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:29:25.921 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:25.919+0000 7f8e6901e640 1 --2- 172.21.15.37:0/3882824477 >> [v2:172.21.15.203:6832/2418205237,v1:172.21.15.203:6833/2418205237] conn(0x7f8e500744b0 0x7f8e500769c0 secure :-1 s=READY pgs=108 cs=0 l=1 rev1=1 crypto rx=0x7f8e5c034000 tx=0x7f8e5c02e000 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:29:25.924 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:25.923+0000 7f8e61ffb640 1 -- 172.21.15.37:0/3882824477 <== mon.0 v2:172.21.15.37:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187246 (secure 0 0 0) 0x7f8e640b7f10 con 0x7f8e640d5430 2024-04-25T10:29:26.111 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:26.110+0000 7f8e6b282640 1 -- 172.21.15.37:0/3882824477 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"} v 0) -- 0x7f8e641e9880 con 0x7f8e640d5430 2024-04-25T10:29:26.111 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:26.110+0000 7f8e61ffb640 1 -- 172.21.15.37:0/3882824477 <== mon.0 v2:172.21.15.37:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]=0 v23) ==== 100+0+11 (secure 0 0 0) 0x7f8e5805f660 con 0x7f8e640d5430 2024-04-25T10:29:26.112 INFO:teuthology.orchestra.run.smithi037.stdout:pg_num: 64 2024-04-25T10:29:26.114 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:26.112+0000 7f8e6b282640 1 -- 172.21.15.37:0/3882824477 >> [v2:172.21.15.203:6832/2418205237,v1:172.21.15.203:6833/2418205237] conn(0x7f8e500744b0 msgr2=0x7f8e500769c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:29:26.114 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:26.112+0000 7f8e6b282640 1 --2- 172.21.15.37:0/3882824477 >> [v2:172.21.15.203:6832/2418205237,v1:172.21.15.203:6833/2418205237] conn(0x7f8e500744b0 0x7f8e500769c0 secure :-1 s=READY pgs=108 cs=0 l=1 rev1=1 crypto rx=0x7f8e5c034000 tx=0x7f8e5c02e000 comp rx=0 tx=0).stop 2024-04-25T10:29:26.114 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:26.112+0000 7f8e6b282640 1 -- 172.21.15.37:0/3882824477 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f8e640d5430 msgr2=0x7f8e641e9340 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:29:26.114 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:26.113+0000 7f8e6b282640 1 --2- 172.21.15.37:0/3882824477 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f8e640d5430 0x7f8e641e9340 secure :-1 s=READY pgs=161 cs=0 l=1 rev1=1 crypto rx=0x7f8e5800c430 tx=0x7f8e5800c8f0 comp rx=0 tx=0).stop 2024-04-25T10:29:26.114 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:26.113+0000 7f8e6b282640 1 -- 172.21.15.37:0/3882824477 shutdown_connections 2024-04-25T10:29:26.115 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:26.113+0000 7f8e6b282640 1 --2- 172.21.15.37:0/3882824477 >> [v2:172.21.15.203:6832/2418205237,v1:172.21.15.203:6833/2418205237] conn(0x7f8e500744b0 0x7f8e500769c0 unknown :-1 s=CLOSED pgs=108 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:29:26.115 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:26.113+0000 7f8e6b282640 1 --2- 172.21.15.37:0/3882824477 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f8e64161110 0x7f8e640cb370 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:29:26.115 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:26.113+0000 7f8e6b282640 1 --2- 172.21.15.37:0/3882824477 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f8e640d5430 0x7f8e641e9340 unknown :-1 s=CLOSED pgs=161 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:29:26.115 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:26.113+0000 7f8e6b282640 1 --2- 172.21.15.37:0/3882824477 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f8e640b7b00 0x7f8e6414cc20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:29:26.115 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:26.113+0000 7f8e6b282640 1 -- 172.21.15.37:0/3882824477 >> 172.21.15.37:0/3882824477 conn(0x7f8e640d27a0 msgr2=0x7f8e6415ea10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:29:26.115 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:26.113+0000 7f8e6b282640 1 -- 172.21.15.37:0/3882824477 shutdown_connections 2024-04-25T10:29:26.115 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:26.113+0000 7f8e6b282640 1 -- 172.21.15.37:0/3882824477 wait complete. 2024-04-25T10:29:26.125 DEBUG:teuthology.orchestra.run.smithi037:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-04-25T10:29:26.236 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:26.234+0000 7fcec7269640 1 Processor -- start 2024-04-25T10:29:26.236 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:26.235+0000 7fcec7269640 1 -- start start 2024-04-25T10:29:26.238 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:26.236+0000 7fcec7269640 1 --2- >> v2:172.21.15.203:3300/0 conn(0x7fcec00b7cd0 0x7fcec0160a00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:29:26.238 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:26.236+0000 7fcec7269640 1 --2- >> v2:172.21.15.37:3300/0 conn(0x7fcec0160f40 0x7fcec0161880 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:29:26.238 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:26.236+0000 7fcec7269640 1 -- --> v1:172.21.15.37:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fcec00d5840 con 0x7fcec00d5430 2024-04-25T10:29:26.238 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:26.236+0000 7fcec7269640 1 -- --> v2:172.21.15.37:3300/0 -- mon_getmap magic: 0 -- 0x7fcec00d4420 con 0x7fcec0160f40 2024-04-25T10:29:26.238 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:26.236+0000 7fcec7269640 1 -- --> v2:172.21.15.203:3300/0 -- mon_getmap magic: 0 -- 0x7fcec00d4670 con 0x7fcec00b7cd0 2024-04-25T10:29:26.238 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:26.236+0000 7fcec5806640 1 --2- >> v2:172.21.15.37:3300/0 conn(0x7fcec0160f40 0x7fcec0161880 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:29:26.238 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:26.236+0000 7fcec5005640 1 --1- >> v1:172.21.15.37:6789/0 conn(0x7fcec00d5430 0x7fcec00b75c0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.37:6789/0 says I am v1:172.21.15.37:59890/0 (socket says 172.21.15.37:59890) 2024-04-25T10:29:26.238 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:26.236+0000 7fcec5005640 1 -- 172.21.15.37:0/719962494 learned_addr learned my addr 172.21.15.37:0/719962494 (peer_addr_for_me v1:172.21.15.37:0/0) 2024-04-25T10:29:26.239 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:26.236+0000 7fcec4804640 1 --2- 172.21.15.37:0/719962494 >> v2:172.21.15.203:3300/0 conn(0x7fcec00b7cd0 0x7fcec0160a00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:29:26.239 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:26.237+0000 7fcec5806640 1 -- 172.21.15.37:0/719962494 >> v2:172.21.15.203:3300/0 conn(0x7fcec00b7cd0 msgr2=0x7fcec0160a00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:29:26.239 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:26.237+0000 7fcec5806640 1 --2- 172.21.15.37:0/719962494 >> v2:172.21.15.203:3300/0 conn(0x7fcec00b7cd0 0x7fcec0160a00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:29:26.239 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:26.237+0000 7fcec5806640 1 -- 172.21.15.37:0/719962494 >> v1:172.21.15.37:6789/0 conn(0x7fcec00d5430 legacy=0x7fcec00b75c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:29:26.239 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:26.237+0000 7fcec5806640 1 -- 172.21.15.37:0/719962494 --> v2:172.21.15.37:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fcec00d4950 con 0x7fcec0160f40 2024-04-25T10:29:26.239 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:26.237+0000 7fcebffff640 1 -- 172.21.15.37:0/719962494 <== mon.0 v1:172.21.15.37:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 3704754765 0 0) 0x7fcec00d5840 con 0x7fcec00d5430 2024-04-25T10:29:26.241 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:26.237+0000 7fcec5806640 1 --2- 172.21.15.37:0/719962494 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fcec0160f40 0x7fcec0161880 secure :-1 s=READY pgs=163 cs=0 l=1 rev1=1 crypto rx=0x7fcec00bba40 tx=0x7fceb002a5f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=3304cc2d33dec3a3 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:29:26.241 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:26.237+0000 7fcebffff640 1 -- 172.21.15.37:0/719962494 <== mon.0 v2:172.21.15.37:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fceb0007740 con 0x7fcec0160f40 2024-04-25T10:29:26.241 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:26.238+0000 7fcebffff640 1 -- 172.21.15.37:0/719962494 <== mon.0 v2:172.21.15.37:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fceb00078e0 con 0x7fcec0160f40 2024-04-25T10:29:26.241 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:26.238+0000 7fcebffff640 1 -- 172.21.15.37:0/719962494 <== mon.0 v2:172.21.15.37:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fceb0007cb0 con 0x7fcec0160f40 2024-04-25T10:29:26.241 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:26.238+0000 7fcec7269640 1 -- 172.21.15.37:0/719962494 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fcec0160f40 msgr2=0x7fcec0161880 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:29:26.241 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:26.238+0000 7fcec7269640 1 --2- 172.21.15.37:0/719962494 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fcec0160f40 0x7fcec0161880 secure :-1 s=READY pgs=163 cs=0 l=1 rev1=1 crypto rx=0x7fcec00bba40 tx=0x7fceb002a5f0 comp rx=0 tx=0).stop 2024-04-25T10:29:26.241 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:26.238+0000 7fcec7269640 1 -- 172.21.15.37:0/719962494 shutdown_connections 2024-04-25T10:29:26.241 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:26.238+0000 7fcec7269640 1 --2- 172.21.15.37:0/719962494 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fcec0160f40 0x7fcec0161880 unknown :-1 s=CLOSED pgs=163 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:29:26.241 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:26.238+0000 7fcec7269640 1 --2- 172.21.15.37:0/719962494 >> v2:172.21.15.203:3300/0 conn(0x7fcec00b7cd0 0x7fcec0160a00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:29:26.242 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:26.238+0000 7fcec7269640 1 -- 172.21.15.37:0/719962494 >> 172.21.15.37:0/719962494 conn(0x7fcec00d27a0 msgr2=0x7fcec00d2bb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:29:26.242 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:26.238+0000 7fcec7269640 1 -- 172.21.15.37:0/719962494 shutdown_connections 2024-04-25T10:29:26.242 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:26.238+0000 7fcec7269640 1 -- 172.21.15.37:0/719962494 wait complete. 2024-04-25T10:29:26.242 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:26.240+0000 7fcec7269640 1 Processor -- start 2024-04-25T10:29:26.242 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:26.240+0000 7fcec7269640 1 -- start start 2024-04-25T10:29:26.244 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:26.241+0000 7fcec7269640 1 --2- >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7fcec00d5430 0x7fcec00cdb80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:29:26.244 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:26.241+0000 7fcec7269640 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fcec0160f40 0x7fcec00cb820 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:29:26.244 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:26.241+0000 7fcec7269640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fcec00b6a70 0x7fcec00b6e80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:29:26.244 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:26.241+0000 7fcec7269640 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 -- 0x7fcec0161350 con 0x7fcec00b6a70 2024-04-25T10:29:26.244 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:26.241+0000 7fcec7269640 1 -- --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_getmap magic: 0 -- 0x7fcec00d4f10 con 0x7fcec0160f40 2024-04-25T10:29:26.244 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:26.241+0000 7fcec7269640 1 -- --> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] -- mon_getmap magic: 0 -- 0x7fcec01614d0 con 0x7fcec00d5430 2024-04-25T10:29:26.244 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:26.241+0000 7fcec5806640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fcec00b6a70 0x7fcec00b6e80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:29:26.244 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:26.242+0000 7fcec5806640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fcec00b6a70 0x7fcec00b6e80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.37:3300/0 says I am v2:172.21.15.37:35276/0 (socket says 172.21.15.37:35276) 2024-04-25T10:29:26.245 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:26.242+0000 7fcec5806640 1 -- 172.21.15.37:0/1580545892 learned_addr learned my addr 172.21.15.37:0/1580545892 (peer_addr_for_me v2:172.21.15.37:0/0) 2024-04-25T10:29:26.245 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:26.242+0000 7fcec4804640 1 --2- 172.21.15.37:0/1580545892 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fcec0160f40 0x7fcec00cb820 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:29:26.245 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:26.242+0000 7fcec5005640 1 --2- 172.21.15.37:0/1580545892 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7fcec00d5430 0x7fcec00cdb80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:29:26.245 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:26.242+0000 7fcec5806640 1 -- 172.21.15.37:0/1580545892 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7fcec00d5430 msgr2=0x7fcec00cdb80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:29:26.245 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:26.242+0000 7fcec5806640 1 --2- 172.21.15.37:0/1580545892 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7fcec00d5430 0x7fcec00cdb80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:29:26.245 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:26.242+0000 7fcec5806640 1 -- 172.21.15.37:0/1580545892 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fcec0160f40 msgr2=0x7fcec00cb820 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:29:26.245 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:26.242+0000 7fcec5806640 1 --2- 172.21.15.37:0/1580545892 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fcec0160f40 0x7fcec00cb820 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:29:26.245 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:26.242+0000 7fcec5806640 1 -- 172.21.15.37:0/1580545892 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fcec00b73c0 con 0x7fcec00b6a70 2024-04-25T10:29:26.245 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:26.242+0000 7fcec5806640 1 --2- 172.21.15.37:0/1580545892 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fcec00b6a70 0x7fcec00b6e80 secure :-1 s=READY pgs=164 cs=0 l=1 rev1=1 crypto rx=0x7fceb0009e50 tx=0x7fceb0004a40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:29:26.245 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:26.242+0000 7fcebdffb640 1 -- 172.21.15.37:0/1580545892 <== mon.0 v2:172.21.15.37:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fceb003f070 con 0x7fcec00b6a70 2024-04-25T10:29:26.245 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:26.242+0000 7fcebdffb640 1 -- 172.21.15.37:0/1580545892 <== mon.0 v2:172.21.15.37:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fceb0031bf0 con 0x7fcec00b6a70 2024-04-25T10:29:26.246 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:26.242+0000 7fcebdffb640 1 -- 172.21.15.37:0/1580545892 <== mon.0 v2:172.21.15.37:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fceb0044460 con 0x7fcec00b6a70 2024-04-25T10:29:26.246 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:26.242+0000 7fcec7269640 1 -- 172.21.15.37:0/1580545892 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fcec0157a60 con 0x7fcec00b6a70 2024-04-25T10:29:26.246 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:26.243+0000 7fcec7269640 1 -- 172.21.15.37:0/1580545892 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fcec01571c0 con 0x7fcec00b6a70 2024-04-25T10:29:26.247 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:26.243+0000 7fcec7269640 1 -- 172.21.15.37:0/1580545892 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fcec0160380 con 0x7fcec00b6a70 2024-04-25T10:29:26.247 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:26.244+0000 7fcebdffb640 1 -- 172.21.15.37:0/1580545892 <== mon.0 v2:172.21.15.37:3300/0 4 ==== mgrmap(e 5) ==== 96782+0+0 (secure 0 0 0) 0x7fceb003b410 con 0x7fcec00b6a70 2024-04-25T10:29:26.247 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:26.244+0000 7fcebdffb640 1 --2- 172.21.15.37:0/1580545892 >> [v2:172.21.15.203:6832/2418205237,v1:172.21.15.203:6833/2418205237] conn(0x7fceac0744b0 0x7fceac0769c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:29:26.247 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:26.245+0000 7fcebdffb640 1 -- 172.21.15.37:0/1580545892 <== mon.0 v2:172.21.15.37:3300/0 5 ==== osd_map(23..23 src has 1..23) ==== 5709+0+0 (secure 0 0 0) 0x7fceb00b6a40 con 0x7fcec00b6a70 2024-04-25T10:29:26.247 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:26.245+0000 7fcec5005640 1 --2- 172.21.15.37:0/1580545892 >> [v2:172.21.15.203:6832/2418205237,v1:172.21.15.203:6833/2418205237] conn(0x7fceac0744b0 0x7fceac0769c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:29:26.247 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:26.245+0000 7fcec5005640 1 --2- 172.21.15.37:0/1580545892 >> [v2:172.21.15.203:6832/2418205237,v1:172.21.15.203:6833/2418205237] conn(0x7fceac0744b0 0x7fceac0769c0 secure :-1 s=READY pgs=109 cs=0 l=1 rev1=1 crypto rx=0x7fceb8007fd0 tx=0x7fceb8007de0 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:29:26.250 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:26.249+0000 7fcebdffb640 1 -- 172.21.15.37:0/1580545892 <== mon.0 v2:172.21.15.37:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187246 (secure 0 0 0) 0x7fcec0160380 con 0x7fcec00b6a70 2024-04-25T10:29:26.475 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:26.474+0000 7fcec7269640 1 -- 172.21.15.37:0/1580545892 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7fcec015eab0 con 0x7fcec00b6a70 2024-04-25T10:29:26.476 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:26.475+0000 7fcebdffb640 1 -- 172.21.15.37:0/1580545892 <== mon.0 v2:172.21.15.37:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 8 v8) ==== 93+0+5178 (secure 0 0 0) 0x7fcec00b73c0 con 0x7fcec00b6a70 2024-04-25T10:29:26.476 INFO:teuthology.orchestra.run.smithi037.stdout: 2024-04-25T10:29:26.476 INFO:teuthology.orchestra.run.smithi037.stdout:{"epoch":8,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":4232,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.37:6835/1911169480","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.37:6833","nonce":1911169480},{"type":"v1","addr":"172.21.15.37:6835","nonce":1911169480}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3},{"gid":4241,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.203:6835/3873243328","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6834","nonce":3873243328},{"type":"v1","addr":"172.21.15.203:6835","nonce":3873243328}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3},{"gid":4278,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.37:6834/179723013","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.37:6832","nonce":179723013},{"type":"v1","addr":"172.21.15.37:6834","nonce":179723013}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":2}],"filesystems":[{"mdsmap":{"epoch":8,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-04-25T10:29:08.397829+0000","modified":"2024-04-25T10:29:11.407694+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":1,"in":[0],"up":{"mds_0":4279},"failed":[],"damaged":[],"stopped":[],"info":{"gid_4279":{"gid":4279,"name":"d","rank":0,"incarnation":4,"state":"up:active","state_seq":3,"addr":"172.21.15.203:6837/3215191422","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6836","nonce":3215191422},{"type":"v1","addr":"172.21.15.203:6837","nonce":3215191422}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":4279,"qdb_cluster":[4279]},"id":1}]} 2024-04-25T10:29:26.476 INFO:teuthology.orchestra.run.smithi037.stderr:dumped fsmap epoch 8 2024-04-25T10:29:26.478 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:26.477+0000 7fcec7269640 1 -- 172.21.15.37:0/1580545892 >> [v2:172.21.15.203:6832/2418205237,v1:172.21.15.203:6833/2418205237] conn(0x7fceac0744b0 msgr2=0x7fceac0769c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:29:26.478 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:26.477+0000 7fcec7269640 1 --2- 172.21.15.37:0/1580545892 >> [v2:172.21.15.203:6832/2418205237,v1:172.21.15.203:6833/2418205237] conn(0x7fceac0744b0 0x7fceac0769c0 secure :-1 s=READY pgs=109 cs=0 l=1 rev1=1 crypto rx=0x7fceb8007fd0 tx=0x7fceb8007de0 comp rx=0 tx=0).stop 2024-04-25T10:29:26.478 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:26.477+0000 7fcec7269640 1 -- 172.21.15.37:0/1580545892 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fcec00b6a70 msgr2=0x7fcec00b6e80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:29:26.479 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:26.477+0000 7fcec7269640 1 --2- 172.21.15.37:0/1580545892 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fcec00b6a70 0x7fcec00b6e80 secure :-1 s=READY pgs=164 cs=0 l=1 rev1=1 crypto rx=0x7fceb0009e50 tx=0x7fceb0004a40 comp rx=0 tx=0).stop 2024-04-25T10:29:26.479 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:26.477+0000 7fcec7269640 1 -- 172.21.15.37:0/1580545892 shutdown_connections 2024-04-25T10:29:26.479 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:26.477+0000 7fcec7269640 1 --2- 172.21.15.37:0/1580545892 >> [v2:172.21.15.203:6832/2418205237,v1:172.21.15.203:6833/2418205237] conn(0x7fceac0744b0 0x7fceac0769c0 unknown :-1 s=CLOSED pgs=109 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:29:26.479 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:26.477+0000 7fcec7269640 1 --2- 172.21.15.37:0/1580545892 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fcec00b6a70 0x7fcec00b6e80 unknown :-1 s=CLOSED pgs=164 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:29:26.479 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:26.477+0000 7fcec7269640 1 --2- 172.21.15.37:0/1580545892 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fcec0160f40 0x7fcec00cb820 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:29:26.479 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:26.477+0000 7fcec7269640 1 --2- 172.21.15.37:0/1580545892 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7fcec00d5430 0x7fcec00cdb80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:29:26.479 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:26.477+0000 7fcec7269640 1 -- 172.21.15.37:0/1580545892 >> 172.21.15.37:0/1580545892 conn(0x7fcec00d27a0 msgr2=0x7fcec015d7e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:29:26.479 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:26.477+0000 7fcec7269640 1 -- 172.21.15.37:0/1580545892 shutdown_connections 2024-04-25T10:29:26.479 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:26.477+0000 7fcec7269640 1 -- 172.21.15.37:0/1580545892 wait complete. 2024-04-25T10:29:26.490 DEBUG:teuthology.orchestra.run.smithi037:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-04-25T10:29:26.601 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:26.600+0000 7f675f0bf640 1 Processor -- start 2024-04-25T10:29:26.601 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:26.600+0000 7f675f0bf640 1 -- start start 2024-04-25T10:29:26.603 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:26.601+0000 7f675f0bf640 1 --2- >> v2:172.21.15.37:3300/0 conn(0x7f6758161020 0x7f675816b190 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:29:26.603 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:26.601+0000 7f675f0bf640 1 -- --> v1:172.21.15.37:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f6758161430 con 0x7f675806cc50 2024-04-25T10:29:26.603 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:26.601+0000 7f675f0bf640 1 -- --> v1:172.21.15.203:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f67581616d0 con 0x7f675815a9f0 2024-04-25T10:29:26.603 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:26.601+0000 7f675f0bf640 1 -- --> v2:172.21.15.37:3300/0 -- mon_getmap magic: 0 -- 0x7f67580b6870 con 0x7f6758161020 2024-04-25T10:29:26.603 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:26.601+0000 7f674ffff640 1 --1- >> v1:172.21.15.37:6789/0 conn(0x7f675806cc50 0x7f6758160910 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.37:6789/0 says I am v1:172.21.15.37:59898/0 (socket says 172.21.15.37:59898) 2024-04-25T10:29:26.603 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:26.601+0000 7f675d65c640 1 --2- >> v2:172.21.15.37:3300/0 conn(0x7f6758161020 0x7f675816b190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:29:26.603 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:26.601+0000 7f674ffff640 1 -- 172.21.15.37:0/24651889 learned_addr learned my addr 172.21.15.37:0/24651889 (peer_addr_for_me v1:172.21.15.37:0/0) 2024-04-25T10:29:26.603 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:26.602+0000 7f675d65c640 1 -- 172.21.15.37:0/24651889 >> v1:172.21.15.203:6789/0 conn(0x7f675815a9f0 legacy=0x7f675806c540 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:29:26.603 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:26.602+0000 7f675d65c640 1 -- 172.21.15.37:0/24651889 >> v1:172.21.15.37:6789/0 conn(0x7f675806cc50 legacy=0x7f6758160910 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:29:26.603 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:26.602+0000 7f674f7fe640 1 -- 172.21.15.37:0/24651889 <== mon.0 v1:172.21.15.37:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 2345342327 0 0) 0x7f6758161430 con 0x7f675806cc50 2024-04-25T10:29:26.604 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:26.602+0000 7f675d65c640 1 -- 172.21.15.37:0/24651889 --> v2:172.21.15.37:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f6758163e00 con 0x7f6758161020 2024-04-25T10:29:26.604 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:26.603+0000 7f675d65c640 1 --2- 172.21.15.37:0/24651889 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f6758161020 0x7f675816b190 secure :-1 s=READY pgs=166 cs=0 l=1 rev1=1 crypto rx=0x7f6750009d90 tx=0x7f675002a610 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d0865b47878f5f8d server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:29:26.604 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:26.603+0000 7f674f7fe640 1 -- 172.21.15.37:0/24651889 <== mon.0 v2:172.21.15.37:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6750031600 con 0x7f6758161020 2024-04-25T10:29:26.604 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:26.603+0000 7f674f7fe640 1 -- 172.21.15.37:0/24651889 <== mon.0 v2:172.21.15.37:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f67500317a0 con 0x7f6758161020 2024-04-25T10:29:26.604 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:26.603+0000 7f674f7fe640 1 -- 172.21.15.37:0/24651889 <== mon.0 v2:172.21.15.37:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6750031b70 con 0x7f6758161020 2024-04-25T10:29:26.605 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:26.604+0000 7f675f0bf640 1 -- 172.21.15.37:0/24651889 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f6758161020 msgr2=0x7f675816b190 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:29:26.605 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:26.604+0000 7f675f0bf640 1 --2- 172.21.15.37:0/24651889 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f6758161020 0x7f675816b190 secure :-1 s=READY pgs=166 cs=0 l=1 rev1=1 crypto rx=0x7f6750009d90 tx=0x7f675002a610 comp rx=0 tx=0).stop 2024-04-25T10:29:26.605 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:26.604+0000 7f675f0bf640 1 -- 172.21.15.37:0/24651889 shutdown_connections 2024-04-25T10:29:26.606 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:26.604+0000 7f675f0bf640 1 --2- 172.21.15.37:0/24651889 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f6758161020 0x7f675816b190 unknown :-1 s=CLOSED pgs=166 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:29:26.606 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:26.604+0000 7f675f0bf640 1 -- 172.21.15.37:0/24651889 >> 172.21.15.37:0/24651889 conn(0x7f67580b4560 msgr2=0x7f67580b4970 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:29:26.606 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:26.604+0000 7f675f0bf640 1 -- 172.21.15.37:0/24651889 shutdown_connections 2024-04-25T10:29:26.606 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:26.604+0000 7f675f0bf640 1 -- 172.21.15.37:0/24651889 wait complete. 2024-04-25T10:29:26.606 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:26.605+0000 7f675f0bf640 1 Processor -- start 2024-04-25T10:29:26.606 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:26.605+0000 7f675f0bf640 1 -- start start 2024-04-25T10:29:26.610 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:26.607+0000 7f675f0bf640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f675806cc50 0x7f67581427c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:29:26.610 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:26.607+0000 7f675f0bf640 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f675815a9f0 0x7f675815eef0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:29:26.610 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:26.607+0000 7f675f0bf640 1 --2- >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f6758161020 0x7f67580ccda0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:29:26.610 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:26.607+0000 7f675f0bf640 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 -- 0x7f67581646a0 con 0x7f675806cc50 2024-04-25T10:29:26.610 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:26.607+0000 7f675f0bf640 1 -- --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_getmap magic: 0 -- 0x7f67581643c0 con 0x7f675815a9f0 2024-04-25T10:29:26.610 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:26.607+0000 7f675f0bf640 1 -- --> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] -- mon_getmap magic: 0 -- 0x7f6758164bc0 con 0x7f6758161020 2024-04-25T10:29:26.610 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:26.607+0000 7f675ce5b640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f675806cc50 0x7f67581427c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:29:26.610 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:26.607+0000 7f675ce5b640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f675806cc50 0x7f67581427c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.37:3300/0 says I am v2:172.21.15.37:35294/0 (socket says 172.21.15.37:35294) 2024-04-25T10:29:26.610 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:26.607+0000 7f675ce5b640 1 -- 172.21.15.37:0/2140889298 learned_addr learned my addr 172.21.15.37:0/2140889298 (peer_addr_for_me v2:172.21.15.37:0/0) 2024-04-25T10:29:26.610 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:26.607+0000 7f674ffff640 1 --2- 172.21.15.37:0/2140889298 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f675815a9f0 0x7f675815eef0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:29:26.610 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:26.607+0000 7f675d65c640 1 --2- 172.21.15.37:0/2140889298 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f6758161020 0x7f67580ccda0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:29:26.611 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:26.607+0000 7f675ce5b640 1 -- 172.21.15.37:0/2140889298 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f6758161020 msgr2=0x7f67580ccda0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:29:26.611 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:26.607+0000 7f675ce5b640 1 --2- 172.21.15.37:0/2140889298 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f6758161020 0x7f67580ccda0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:29:26.611 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:26.607+0000 7f675ce5b640 1 -- 172.21.15.37:0/2140889298 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f675815a9f0 msgr2=0x7f675815eef0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:29:26.611 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:26.607+0000 7f675ce5b640 1 --2- 172.21.15.37:0/2140889298 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f675815a9f0 0x7f675815eef0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:29:26.611 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:26.607+0000 7f675ce5b640 1 -- 172.21.15.37:0/2140889298 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f675815de50 con 0x7f675806cc50 2024-04-25T10:29:26.611 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:26.607+0000 7f674ffff640 1 --2- 172.21.15.37:0/2140889298 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f675815a9f0 0x7f675815eef0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T10:29:26.611 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:26.607+0000 7f675d65c640 1 --2- 172.21.15.37:0/2140889298 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f6758161020 0x7f67580ccda0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T10:29:26.611 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:26.608+0000 7f675ce5b640 1 --2- 172.21.15.37:0/2140889298 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f675806cc50 0x7f67581427c0 secure :-1 s=READY pgs=167 cs=0 l=1 rev1=1 crypto rx=0x7f675400b3f0 tx=0x7f675400b8b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:29:26.611 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:26.608+0000 7f674d7fa640 1 -- 172.21.15.37:0/2140889298 <== mon.0 v2:172.21.15.37:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6754015020 con 0x7f675806cc50 2024-04-25T10:29:26.611 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:26.608+0000 7f674d7fa640 1 -- 172.21.15.37:0/2140889298 <== mon.0 v2:172.21.15.37:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f6754011830 con 0x7f675806cc50 2024-04-25T10:29:26.611 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:26.608+0000 7f675f0bf640 1 -- 172.21.15.37:0/2140889298 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f675815e130 con 0x7f675806cc50 2024-04-25T10:29:26.611 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:26.608+0000 7f674d7fa640 1 -- 172.21.15.37:0/2140889298 <== mon.0 v2:172.21.15.37:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6754011c20 con 0x7f675806cc50 2024-04-25T10:29:26.611 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:26.608+0000 7f675f0bf640 1 -- 172.21.15.37:0/2140889298 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f675815e6f0 con 0x7f675806cc50 2024-04-25T10:29:26.611 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:26.609+0000 7f675f0bf640 1 -- 172.21.15.37:0/2140889298 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f675806a140 con 0x7f675806cc50 2024-04-25T10:29:26.612 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:26.609+0000 7f674d7fa640 1 -- 172.21.15.37:0/2140889298 <== mon.0 v2:172.21.15.37:3300/0 4 ==== mgrmap(e 5) ==== 96782+0+0 (secure 0 0 0) 0x7f6754018020 con 0x7f675806cc50 2024-04-25T10:29:26.612 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:26.610+0000 7f674d7fa640 1 --2- 172.21.15.37:0/2140889298 >> [v2:172.21.15.203:6832/2418205237,v1:172.21.15.203:6833/2418205237] conn(0x7f67440744b0 0x7f67440769c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:29:26.613 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:26.610+0000 7f674d7fa640 1 -- 172.21.15.37:0/2140889298 <== mon.0 v2:172.21.15.37:3300/0 5 ==== osd_map(23..23 src has 1..23) ==== 5709+0+0 (secure 0 0 0) 0x7f6754094f50 con 0x7f675806cc50 2024-04-25T10:29:26.613 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:26.610+0000 7f674ffff640 1 --2- 172.21.15.37:0/2140889298 >> [v2:172.21.15.203:6832/2418205237,v1:172.21.15.203:6833/2418205237] conn(0x7f67440744b0 0x7f67440769c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:29:26.613 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:26.610+0000 7f674ffff640 1 --2- 172.21.15.37:0/2140889298 >> [v2:172.21.15.203:6832/2418205237,v1:172.21.15.203:6833/2418205237] conn(0x7f67440744b0 0x7f67440769c0 secure :-1 s=READY pgs=110 cs=0 l=1 rev1=1 crypto rx=0x7f67581f9730 tx=0x7f6748009040 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:29:26.615 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:26.614+0000 7f674d7fa640 1 -- 172.21.15.37:0/2140889298 <== mon.0 v2:172.21.15.37:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187246 (secure 0 0 0) 0x7f675401f030 con 0x7f675806cc50 2024-04-25T10:29:26.799 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:26.797+0000 7f675f0bf640 1 -- 172.21.15.37:0/2140889298 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7f6758071d10 con 0x7f675806cc50 2024-04-25T10:29:26.800 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:26.799+0000 7f674d7fa640 1 -- 172.21.15.37:0/2140889298 <== mon.0 v2:172.21.15.37:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v23) ==== 74+0+16621 (secure 0 0 0) 0x7f675405f640 con 0x7f675806cc50 2024-04-25T10:29:26.800 INFO:teuthology.orchestra.run.smithi037.stdout: 2024-04-25T10:29:26.800 INFO:teuthology.orchestra.run.smithi037.stdout:{"epoch":23,"fsid":"f4acda28-be34-492a-b286-994afd36f61d","created":"2024-04-25T10:28:43.985718+0000","modified":"2024-04-25T10:29:10.016188+0000","last_up_change":"2024-04-25T10:28:54.900654+0000","last_in_change":"2024-04-25T10:28:46.216973+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":3,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-04-25T10:28:56.202153+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"19","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2024-04-25T10:29:06.507244+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"22","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":2.25,"score_stable":2.25,"optimal_score":1,"raw_score_acting":2.25,"raw_score_stable":2.25,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2024-04-25T10:29:07.347844+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"23","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"a2f79028-b332-4166-9ebb-a7024a37ea22","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":21,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.37:6816","nonce":2746074720},{"type":"v1","addr":"172.21.15.37:6817","nonce":2746074720}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.37:6818","nonce":2746074720},{"type":"v1","addr":"172.21.15.37:6819","nonce":2746074720}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.37:6822","nonce":2746074720},{"type":"v1","addr":"172.21.15.37:6823","nonce":2746074720}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.37:6820","nonce":2746074720},{"type":"v1","addr":"172.21.15.37:6821","nonce":2746074720}]},"public_addr":"172.21.15.37:6817/2746074720","cluster_addr":"172.21.15.37:6819/2746074720","heartbeat_back_addr":"172.21.15.37:6823/2746074720","heartbeat_front_addr":"172.21.15.37:6821/2746074720","state":["exists","up"]},{"osd":1,"uuid":"973159de-c2ea-4083-aecc-69373d1c1132","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":21,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.37:6800","nonce":1690341662},{"type":"v1","addr":"172.21.15.37:6801","nonce":1690341662}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.37:6802","nonce":1690341662},{"type":"v1","addr":"172.21.15.37:6803","nonce":1690341662}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.37:6806","nonce":1690341662},{"type":"v1","addr":"172.21.15.37:6807","nonce":1690341662}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.37:6804","nonce":1690341662},{"type":"v1","addr":"172.21.15.37:6805","nonce":1690341662}]},"public_addr":"172.21.15.37:6801/1690341662","cluster_addr":"172.21.15.37:6803/1690341662","heartbeat_back_addr":"172.21.15.37:6807/1690341662","heartbeat_front_addr":"172.21.15.37:6805/1690341662","state":["exists","up"]},{"osd":2,"uuid":"4ae7a0e9-cb3f-484c-a75f-f3766b721c47","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":21,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.37:6808","nonce":2754669649},{"type":"v1","addr":"172.21.15.37:6809","nonce":2754669649}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.37:6810","nonce":2754669649},{"type":"v1","addr":"172.21.15.37:6811","nonce":2754669649}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.37:6814","nonce":2754669649},{"type":"v1","addr":"172.21.15.37:6815","nonce":2754669649}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.37:6812","nonce":2754669649},{"type":"v1","addr":"172.21.15.37:6813","nonce":2754669649}]},"public_addr":"172.21.15.37:6809/2754669649","cluster_addr":"172.21.15.37:6811/2754669649","heartbeat_back_addr":"172.21.15.37:6815/2754669649","heartbeat_front_addr":"172.21.15.37:6813/2754669649","state":["exists","up"]},{"osd":3,"uuid":"1ba4a746-1608-47a7-ace2-960a5cfae934","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":21,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.37:6824","nonce":1815290618},{"type":"v1","addr":"172.21.15.37:6825","nonce":1815290618}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.37:6826","nonce":1815290618},{"type":"v1","addr":"172.21.15.37:6827","nonce":1815290618}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.37:6830","nonce":1815290618},{"type":"v1","addr":"172.21.15.37:6831","nonce":1815290618}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.37:6828","nonce":1815290618},{"type":"v1","addr":"172.21.15.37:6829","nonce":1815290618}]},"public_addr":"172.21.15.37:6825/1815290618","cluster_addr":"172.21.15.37:6827/1815290618","heartbeat_back_addr":"172.21.15.37:6831/1815290618","heartbeat_front_addr":"172.21.15.37:6829/1815290618","state":["exists","up"]},{"osd":4,"uuid":"d4f59f5d-0413-4b79-87ee-bb212c1fcf62","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":21,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6800","nonce":4051532063},{"type":"v1","addr":"172.21.15.203:6801","nonce":4051532063}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6802","nonce":4051532063},{"type":"v1","addr":"172.21.15.203:6803","nonce":4051532063}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6806","nonce":4051532063},{"type":"v1","addr":"172.21.15.203:6807","nonce":4051532063}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6804","nonce":4051532063},{"type":"v1","addr":"172.21.15.203:6805","nonce":4051532063}]},"public_addr":"172.21.15.203:6801/4051532063","cluster_addr":"172.21.15.203:6803/4051532063","heartbeat_back_addr":"172.21.15.203:6807/4051532063","heartbeat_front_addr":"172.21.15.203:6805/4051532063","state":["exists","up"]},{"osd":5,"uuid":"69710a0e-41d8-4e7f-9a10-17ce540b67be","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":21,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6824","nonce":1203822890},{"type":"v1","addr":"172.21.15.203:6825","nonce":1203822890}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6826","nonce":1203822890},{"type":"v1","addr":"172.21.15.203:6827","nonce":1203822890}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6830","nonce":1203822890},{"type":"v1","addr":"172.21.15.203:6831","nonce":1203822890}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6828","nonce":1203822890},{"type":"v1","addr":"172.21.15.203:6829","nonce":1203822890}]},"public_addr":"172.21.15.203:6825/1203822890","cluster_addr":"172.21.15.203:6827/1203822890","heartbeat_back_addr":"172.21.15.203:6831/1203822890","heartbeat_front_addr":"172.21.15.203:6829/1203822890","state":["exists","up"]},{"osd":6,"uuid":"f29dd047-3b48-473d-a235-6642a6293f20","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":21,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6810","nonce":2309615014},{"type":"v1","addr":"172.21.15.203:6812","nonce":2309615014}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6814","nonce":2309615014},{"type":"v1","addr":"172.21.15.203:6816","nonce":2309615014}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6822","nonce":2309615014},{"type":"v1","addr":"172.21.15.203:6823","nonce":2309615014}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6818","nonce":2309615014},{"type":"v1","addr":"172.21.15.203:6820","nonce":2309615014}]},"public_addr":"172.21.15.203:6812/2309615014","cluster_addr":"172.21.15.203:6816/2309615014","heartbeat_back_addr":"172.21.15.203:6823/2309615014","heartbeat_front_addr":"172.21.15.203:6820/2309615014","state":["exists","up"]},{"osd":7,"uuid":"6e691644-378b-4fdc-8e86-fb5f56e31f29","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":21,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6808","nonce":3688099014},{"type":"v1","addr":"172.21.15.203:6809","nonce":3688099014}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6811","nonce":3688099014},{"type":"v1","addr":"172.21.15.203:6813","nonce":3688099014}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6819","nonce":3688099014},{"type":"v1","addr":"172.21.15.203:6821","nonce":3688099014}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6815","nonce":3688099014},{"type":"v1","addr":"172.21.15.203:6817","nonce":3688099014}]},"public_addr":"172.21.15.203:6809/3688099014","cluster_addr":"172.21.15.203:6813/3688099014","heartbeat_back_addr":"172.21.15.203:6821/3688099014","heartbeat_front_addr":"172.21.15.203:6817/3688099014","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T10:28:51.419141+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T10:28:51.366448+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T10:28:51.348276+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T10:28:51.380519+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T10:28:51.105812+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T10:28:51.384496+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T10:28:51.388755+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T10:28:51.366958+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-04-25T10:29:26.802 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:26.800+0000 7f675f0bf640 1 -- 172.21.15.37:0/2140889298 >> [v2:172.21.15.203:6832/2418205237,v1:172.21.15.203:6833/2418205237] conn(0x7f67440744b0 msgr2=0x7f67440769c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:29:26.802 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:26.800+0000 7f675f0bf640 1 --2- 172.21.15.37:0/2140889298 >> [v2:172.21.15.203:6832/2418205237,v1:172.21.15.203:6833/2418205237] conn(0x7f67440744b0 0x7f67440769c0 secure :-1 s=READY pgs=110 cs=0 l=1 rev1=1 crypto rx=0x7f67581f9730 tx=0x7f6748009040 comp rx=0 tx=0).stop 2024-04-25T10:29:26.802 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:26.801+0000 7f675f0bf640 1 -- 172.21.15.37:0/2140889298 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f675806cc50 msgr2=0x7f67581427c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:29:26.802 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:26.801+0000 7f675f0bf640 1 --2- 172.21.15.37:0/2140889298 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f675806cc50 0x7f67581427c0 secure :-1 s=READY pgs=167 cs=0 l=1 rev1=1 crypto rx=0x7f675400b3f0 tx=0x7f675400b8b0 comp rx=0 tx=0).stop 2024-04-25T10:29:26.802 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:26.801+0000 7f675f0bf640 1 -- 172.21.15.37:0/2140889298 shutdown_connections 2024-04-25T10:29:26.802 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:26.801+0000 7f675f0bf640 1 --2- 172.21.15.37:0/2140889298 >> [v2:172.21.15.203:6832/2418205237,v1:172.21.15.203:6833/2418205237] conn(0x7f67440744b0 0x7f67440769c0 unknown :-1 s=CLOSED pgs=110 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:29:26.803 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:26.801+0000 7f675f0bf640 1 --2- 172.21.15.37:0/2140889298 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f6758161020 0x7f67580ccda0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:29:26.803 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:26.801+0000 7f675f0bf640 1 --2- 172.21.15.37:0/2140889298 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f675815a9f0 0x7f675815eef0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:29:26.803 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:26.801+0000 7f675f0bf640 1 --2- 172.21.15.37:0/2140889298 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f675806cc50 0x7f67581427c0 unknown :-1 s=CLOSED pgs=167 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:29:26.803 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:26.801+0000 7f675f0bf640 1 -- 172.21.15.37:0/2140889298 >> 172.21.15.37:0/2140889298 conn(0x7f67580b4560 msgr2=0x7f675806c540 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:29:26.803 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:26.801+0000 7f675f0bf640 1 -- 172.21.15.37:0/2140889298 shutdown_connections 2024-04-25T10:29:26.803 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:26.801+0000 7f675f0bf640 1 -- 172.21.15.37:0/2140889298 wait complete. 2024-04-25T10:29:26.813 DEBUG:teuthology.orchestra.run.smithi037:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-04-25T10:29:26.925 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:26.923+0000 7f75006aa640 1 Processor -- start 2024-04-25T10:29:26.925 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:26.923+0000 7f75006aa640 1 -- start start 2024-04-25T10:29:26.927 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:26.925+0000 7f75006aa640 1 --2- >> v2:172.21.15.37:3300/0 conn(0x7f74f80b7cd0 0x7f74f8160a00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:29:26.927 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:26.925+0000 7f75006aa640 1 --2- >> v2:172.21.15.203:3300/0 conn(0x7f74f8160f40 0x7f74f8161880 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:29:26.927 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:26.925+0000 7f75006aa640 1 -- --> v1:172.21.15.203:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f74f80d5840 con 0x7f74f80d5430 2024-04-25T10:29:26.927 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:26.925+0000 7f75006aa640 1 -- --> v2:172.21.15.37:3300/0 -- mon_getmap magic: 0 -- 0x7f74f80d4420 con 0x7f74f80b7cd0 2024-04-25T10:29:26.927 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:26.925+0000 7f75006aa640 1 -- --> v2:172.21.15.203:3300/0 -- mon_getmap magic: 0 -- 0x7f74f80d4670 con 0x7f74f8160f40 2024-04-25T10:29:26.927 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:26.925+0000 7f74fdc45640 1 --2- >> v2:172.21.15.37:3300/0 conn(0x7f74f80b7cd0 0x7f74f8160a00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:29:26.927 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:26.925+0000 7f74fdc45640 1 --2- >> v2:172.21.15.37:3300/0 conn(0x7f74f80b7cd0 0x7f74f8160a00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.37:3300/0 says I am v2:172.21.15.37:35298/0 (socket says 172.21.15.37:35298) 2024-04-25T10:29:26.927 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:26.925+0000 7f74fdc45640 1 -- 172.21.15.37:0/1582125301 learned_addr learned my addr 172.21.15.37:0/1582125301 (peer_addr_for_me v2:172.21.15.37:0/0) 2024-04-25T10:29:26.927 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:26.925+0000 7f74fec47640 1 --2- 172.21.15.37:0/1582125301 >> v2:172.21.15.203:3300/0 conn(0x7f74f8160f40 0x7f74f8161880 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:29:26.927 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:26.925+0000 7f74fdc45640 1 -- 172.21.15.37:0/1582125301 >> v2:172.21.15.203:3300/0 conn(0x7f74f8160f40 msgr2=0x7f74f8161880 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:29:26.927 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:26.925+0000 7f74fdc45640 1 --2- 172.21.15.37:0/1582125301 >> v2:172.21.15.203:3300/0 conn(0x7f74f8160f40 0x7f74f8161880 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:29:26.927 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:26.925+0000 7f74fdc45640 1 -- 172.21.15.37:0/1582125301 >> v1:172.21.15.203:6789/0 conn(0x7f74f80d5430 legacy=0x7f74f80b75c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:29:26.927 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:26.925+0000 7f74fdc45640 1 -- 172.21.15.37:0/1582125301 --> v2:172.21.15.37:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f74f80d4950 con 0x7f74f80b7cd0 2024-04-25T10:29:26.929 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:26.926+0000 7f74fdc45640 1 --2- 172.21.15.37:0/1582125301 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f74f80b7cd0 0x7f74f8160a00 secure :-1 s=READY pgs=168 cs=0 l=1 rev1=1 crypto rx=0x7f74f80d3cb0 tx=0x7f74ec02a1e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=742023c48399a3a9 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:29:26.929 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:26.926+0000 7f74fd444640 1 -- 172.21.15.37:0/1582125301 <== mon.0 v2:172.21.15.37:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f74ec037070 con 0x7f74f80b7cd0 2024-04-25T10:29:26.929 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:26.926+0000 7f74fd444640 1 -- 172.21.15.37:0/1582125301 <== mon.0 v2:172.21.15.37:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f74ec032440 con 0x7f74f80b7cd0 2024-04-25T10:29:26.929 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:26.926+0000 7f74fd444640 1 -- 172.21.15.37:0/1582125301 <== mon.0 v2:172.21.15.37:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f74ec032810 con 0x7f74f80b7cd0 2024-04-25T10:29:26.929 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:26.927+0000 7f75006aa640 1 -- 172.21.15.37:0/1582125301 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f74f80b7cd0 msgr2=0x7f74f8160a00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:29:26.929 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:26.927+0000 7f75006aa640 1 --2- 172.21.15.37:0/1582125301 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f74f80b7cd0 0x7f74f8160a00 secure :-1 s=READY pgs=168 cs=0 l=1 rev1=1 crypto rx=0x7f74f80d3cb0 tx=0x7f74ec02a1e0 comp rx=0 tx=0).stop 2024-04-25T10:29:26.929 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:26.927+0000 7f75006aa640 1 -- 172.21.15.37:0/1582125301 shutdown_connections 2024-04-25T10:29:26.929 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:26.927+0000 7f75006aa640 1 --2- 172.21.15.37:0/1582125301 >> v2:172.21.15.203:3300/0 conn(0x7f74f8160f40 0x7f74f8161880 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:29:26.929 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:26.927+0000 7f75006aa640 1 --2- 172.21.15.37:0/1582125301 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f74f80b7cd0 0x7f74f8160a00 unknown :-1 s=CLOSED pgs=168 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:29:26.929 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:26.927+0000 7f75006aa640 1 -- 172.21.15.37:0/1582125301 >> 172.21.15.37:0/1582125301 conn(0x7f74f80d27a0 msgr2=0x7f74f80d2bb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:29:26.929 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:26.927+0000 7f75006aa640 1 -- 172.21.15.37:0/1582125301 shutdown_connections 2024-04-25T10:29:26.929 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:26.927+0000 7f75006aa640 1 -- 172.21.15.37:0/1582125301 wait complete. 2024-04-25T10:29:26.930 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:26.929+0000 7f75006aa640 1 Processor -- start 2024-04-25T10:29:26.930 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:26.929+0000 7f75006aa640 1 -- start start 2024-04-25T10:29:26.932 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:26.930+0000 7f75006aa640 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f74f80d5430 0x7f74f815a2a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:29:26.933 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:26.930+0000 7f75006aa640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f74f8160f40 0x7f74f80ba590 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:29:26.933 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:26.930+0000 7f75006aa640 1 --2- >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f74f815bfe0 0x7f74f815c3f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:29:26.933 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:26.930+0000 7f75006aa640 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 -- 0x7f74f8161350 con 0x7f74f8160f40 2024-04-25T10:29:26.933 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:26.930+0000 7f75006aa640 1 -- --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_getmap magic: 0 -- 0x7f74f80d4f10 con 0x7f74f80d5430 2024-04-25T10:29:26.933 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:26.930+0000 7f75006aa640 1 -- --> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] -- mon_getmap magic: 0 -- 0x7f74f81614d0 con 0x7f74f815bfe0 2024-04-25T10:29:26.933 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:26.930+0000 7f74fdc45640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f74f8160f40 0x7f74f80ba590 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:29:26.933 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:26.930+0000 7f74fdc45640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f74f8160f40 0x7f74f80ba590 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.37:3300/0 says I am v2:172.21.15.37:35306/0 (socket says 172.21.15.37:35306) 2024-04-25T10:29:26.933 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:26.930+0000 7f74fdc45640 1 -- 172.21.15.37:0/2799906965 learned_addr learned my addr 172.21.15.37:0/2799906965 (peer_addr_for_me v2:172.21.15.37:0/0) 2024-04-25T10:29:26.933 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:26.930+0000 7f74fec47640 1 --2- 172.21.15.37:0/2799906965 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f74f815bfe0 0x7f74f815c3f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:29:26.933 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:26.930+0000 7f74fe446640 1 --2- 172.21.15.37:0/2799906965 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f74f80d5430 0x7f74f815a2a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:29:26.933 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:26.930+0000 7f74fdc45640 1 -- 172.21.15.37:0/2799906965 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f74f815bfe0 msgr2=0x7f74f815c3f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:29:26.933 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:26.930+0000 7f74fdc45640 1 --2- 172.21.15.37:0/2799906965 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f74f815bfe0 0x7f74f815c3f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:29:26.933 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:26.930+0000 7f74fdc45640 1 -- 172.21.15.37:0/2799906965 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f74f80d5430 msgr2=0x7f74f815a2a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:29:26.934 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:26.930+0000 7f74fdc45640 1 --2- 172.21.15.37:0/2799906965 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f74f80d5430 0x7f74f815a2a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:29:26.934 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:26.931+0000 7f74fdc45640 1 -- 172.21.15.37:0/2799906965 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f74f815c930 con 0x7f74f8160f40 2024-04-25T10:29:26.934 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:26.931+0000 7f74fdc45640 1 --2- 172.21.15.37:0/2799906965 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f74f8160f40 0x7f74f80ba590 secure :-1 s=READY pgs=169 cs=0 l=1 rev1=1 crypto rx=0x7f74ec02a220 tx=0x7f74ec0046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:29:26.934 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:26.931+0000 7f74f6ffd640 1 -- 172.21.15.37:0/2799906965 <== mon.0 v2:172.21.15.37:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f74ec037040 con 0x7f74f8160f40 2024-04-25T10:29:26.934 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:26.931+0000 7f74f6ffd640 1 -- 172.21.15.37:0/2799906965 <== mon.0 v2:172.21.15.37:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f74ec042070 con 0x7f74f8160f40 2024-04-25T10:29:26.934 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:26.931+0000 7f75006aa640 1 -- 172.21.15.37:0/2799906965 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f74f80b96a0 con 0x7f74f8160f40 2024-04-25T10:29:26.934 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:26.931+0000 7f74f6ffd640 1 -- 172.21.15.37:0/2799906965 <== mon.0 v2:172.21.15.37:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f74ec030880 con 0x7f74f8160f40 2024-04-25T10:29:26.934 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:26.931+0000 7f75006aa640 1 -- 172.21.15.37:0/2799906965 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f74f80b9c60 con 0x7f74f8160f40 2024-04-25T10:29:26.935 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:26.932+0000 7f75006aa640 1 -- 172.21.15.37:0/2799906965 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f74f80b93f0 con 0x7f74f8160f40 2024-04-25T10:29:26.935 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:26.932+0000 7f74f6ffd640 1 -- 172.21.15.37:0/2799906965 <== mon.0 v2:172.21.15.37:3300/0 4 ==== mgrmap(e 5) ==== 96782+0+0 (secure 0 0 0) 0x7f74ec030a20 con 0x7f74f8160f40 2024-04-25T10:29:26.935 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:26.933+0000 7f74f6ffd640 1 --2- 172.21.15.37:0/2799906965 >> [v2:172.21.15.203:6832/2418205237,v1:172.21.15.203:6833/2418205237] conn(0x7f74e40744e0 0x7f74e40769f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:29:26.935 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:26.933+0000 7f74f6ffd640 1 -- 172.21.15.37:0/2799906965 <== mon.0 v2:172.21.15.37:3300/0 5 ==== osd_map(23..23 src has 1..23) ==== 5709+0+0 (secure 0 0 0) 0x7f74ec0b5b00 con 0x7f74f8160f40 2024-04-25T10:29:26.935 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:26.933+0000 7f74fe446640 1 --2- 172.21.15.37:0/2799906965 >> [v2:172.21.15.203:6832/2418205237,v1:172.21.15.203:6833/2418205237] conn(0x7f74e40744e0 0x7f74e40769f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:29:26.935 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:26.934+0000 7f74fe446640 1 --2- 172.21.15.37:0/2799906965 >> [v2:172.21.15.203:6832/2418205237,v1:172.21.15.203:6833/2418205237] conn(0x7f74e40744e0 0x7f74e40769f0 secure :-1 s=READY pgs=111 cs=0 l=1 rev1=1 crypto rx=0x7f74f0007fd0 tx=0x7f74f0009040 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:29:26.938 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:26.938+0000 7f74f6ffd640 1 -- 172.21.15.37:0/2799906965 <== mon.0 v2:172.21.15.37:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187246 (secure 0 0 0) 0x7f74f815c930 con 0x7f74f8160f40 2024-04-25T10:29:27.160 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:27.158+0000 7f75006aa640 1 -- 172.21.15.37:0/2799906965 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f74f806a140 con 0x7f74f8160f40 2024-04-25T10:29:27.160 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:27.159+0000 7f74f6ffd640 1 -- 172.21.15.37:0/2799906965 <== mon.0 v2:172.21.15.37:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 8 v8) ==== 93+0+5178 (secure 0 0 0) 0x7f74ec080290 con 0x7f74f8160f40 2024-04-25T10:29:27.160 INFO:teuthology.orchestra.run.smithi037.stdout: 2024-04-25T10:29:27.161 INFO:teuthology.orchestra.run.smithi037.stdout:{"epoch":8,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":4232,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.37:6835/1911169480","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.37:6833","nonce":1911169480},{"type":"v1","addr":"172.21.15.37:6835","nonce":1911169480}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3},{"gid":4241,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.203:6835/3873243328","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6834","nonce":3873243328},{"type":"v1","addr":"172.21.15.203:6835","nonce":3873243328}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3},{"gid":4278,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.37:6834/179723013","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.37:6832","nonce":179723013},{"type":"v1","addr":"172.21.15.37:6834","nonce":179723013}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":2}],"filesystems":[{"mdsmap":{"epoch":8,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-04-25T10:29:08.397829+0000","modified":"2024-04-25T10:29:11.407694+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":1,"in":[0],"up":{"mds_0":4279},"failed":[],"damaged":[],"stopped":[],"info":{"gid_4279":{"gid":4279,"name":"d","rank":0,"incarnation":4,"state":"up:active","state_seq":3,"addr":"172.21.15.203:6837/3215191422","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6836","nonce":3215191422},{"type":"v1","addr":"172.21.15.203:6837","nonce":3215191422}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":4279,"qdb_cluster":[4279]},"id":1}]} 2024-04-25T10:29:27.161 INFO:teuthology.orchestra.run.smithi037.stderr:dumped fsmap epoch 8 2024-04-25T10:29:27.163 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:27.161+0000 7f75006aa640 1 -- 172.21.15.37:0/2799906965 >> [v2:172.21.15.203:6832/2418205237,v1:172.21.15.203:6833/2418205237] conn(0x7f74e40744e0 msgr2=0x7f74e40769f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:29:27.163 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:27.161+0000 7f75006aa640 1 --2- 172.21.15.37:0/2799906965 >> [v2:172.21.15.203:6832/2418205237,v1:172.21.15.203:6833/2418205237] conn(0x7f74e40744e0 0x7f74e40769f0 secure :-1 s=READY pgs=111 cs=0 l=1 rev1=1 crypto rx=0x7f74f0007fd0 tx=0x7f74f0009040 comp rx=0 tx=0).stop 2024-04-25T10:29:27.163 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:27.161+0000 7f75006aa640 1 -- 172.21.15.37:0/2799906965 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f74f8160f40 msgr2=0x7f74f80ba590 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:29:27.163 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:27.161+0000 7f75006aa640 1 --2- 172.21.15.37:0/2799906965 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f74f8160f40 0x7f74f80ba590 secure :-1 s=READY pgs=169 cs=0 l=1 rev1=1 crypto rx=0x7f74ec02a220 tx=0x7f74ec0046c0 comp rx=0 tx=0).stop 2024-04-25T10:29:27.163 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:27.161+0000 7f75006aa640 1 -- 172.21.15.37:0/2799906965 shutdown_connections 2024-04-25T10:29:27.163 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:27.161+0000 7f75006aa640 1 --2- 172.21.15.37:0/2799906965 >> [v2:172.21.15.203:6832/2418205237,v1:172.21.15.203:6833/2418205237] conn(0x7f74e40744e0 0x7f74e40769f0 unknown :-1 s=CLOSED pgs=111 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:29:27.163 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:27.161+0000 7f75006aa640 1 --2- 172.21.15.37:0/2799906965 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f74f815bfe0 0x7f74f815c3f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:29:27.163 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:27.161+0000 7f75006aa640 1 --2- 172.21.15.37:0/2799906965 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f74f8160f40 0x7f74f80ba590 unknown :-1 s=CLOSED pgs=169 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:29:27.163 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:27.161+0000 7f75006aa640 1 --2- 172.21.15.37:0/2799906965 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f74f80d5430 0x7f74f815a2a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:29:27.163 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:27.161+0000 7f75006aa640 1 -- 172.21.15.37:0/2799906965 >> 172.21.15.37:0/2799906965 conn(0x7f74f80d27a0 msgr2=0x7f74f815cfd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:29:27.164 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:27.161+0000 7f75006aa640 1 -- 172.21.15.37:0/2799906965 shutdown_connections 2024-04-25T10:29:27.164 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:27.161+0000 7f75006aa640 1 -- 172.21.15.37:0/2799906965 wait complete. 2024-04-25T10:29:27.577 DEBUG:tasks.cephfs.filesystem:are_daemons_healthy: mds map: {'epoch': 8, 'flags': 18, 'flags_state': {'joinable': True, 'allow_snaps': True, 'allow_multimds_snaps': True, 'allow_standby_replay': False, 'refuse_client_session': False, 'refuse_standby_for_another_fs': False, 'balance_automate': False}, 'ever_allowed_features': 0, 'explicitly_allowed_features': 0, 'created': '2024-04-25T10:29:08.397829+0000', 'modified': '2024-04-25T10:29:11.407694+0000', 'tableserver': 0, 'root': 0, 'session_timeout': 60, 'session_autoclose': 300, 'required_client_features': {}, 'max_file_size': 1099511627776, 'max_xattr_size': 65536, 'last_failure': 0, 'last_failure_osd_epoch': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments', 'feature_12': 'quiesce subvolumes'}}, 'max_mds': 1, 'in': [0], 'up': {'mds_0': 4279}, 'failed': [], 'damaged': [], 'stopped': [], 'info': {'gid_4279': {'gid': 4279, 'name': 'd', 'rank': 0, 'incarnation': 4, 'state': 'up:active', 'state_seq': 3, 'addr': '172.21.15.203:6837/3215191422', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.203:6836', 'nonce': 3215191422}, {'type': 'v1', 'addr': '172.21.15.203:6837', 'nonce': 3215191422}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540701547738038271, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments', 'feature_12': 'quiesce subvolumes'}}}}, 'data_pools': [3], 'metadata_pool': 2, 'enabled': True, 'fs_name': 'cephfs', 'balancer': '', 'bal_rank_mask': '-1', 'standby_count_wanted': 0, 'qdb_leader': 4279, 'qdb_cluster': [4279]} 2024-04-25T10:29:27.577 DEBUG:tasks.cephfs.filesystem:are_daemons_healthy: 1/1 2024-04-25T10:29:27.577 DEBUG:teuthology.orchestra.run.smithi037:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.d status 2024-04-25T10:29:27.697 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:27.696+0000 7f46e5106640 1 Processor -- start 2024-04-25T10:29:27.697 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:27.696+0000 7f46e5106640 1 -- start start 2024-04-25T10:29:27.699 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:27.697+0000 7f46e5106640 1 --2- >> v2:172.21.15.37:3300/0 conn(0x7f46e015a9f0 0x7f46e006c540 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:29:27.699 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:27.697+0000 7f46e5106640 1 --2- >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f46e0161020 0x7f46e00b5860 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:29:27.699 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:27.697+0000 7f46e5106640 1 -- --> v1:172.21.15.37:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f46e0161430 con 0x7f46e006ca80 2024-04-25T10:29:27.699 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:27.697+0000 7f46e5106640 1 -- --> v2:172.21.15.37:3300/0 -- mon_getmap magic: 0 -- 0x7f46e006ce90 con 0x7f46e015a9f0 2024-04-25T10:29:27.700 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:27.697+0000 7f46e5106640 1 -- --> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] -- mon_getmap magic: 0 -- 0x7f46e01616d0 con 0x7f46e0161020 2024-04-25T10:29:27.700 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:27.698+0000 7f46ded9d640 1 --2- >> v2:172.21.15.37:3300/0 conn(0x7f46e015a9f0 0x7f46e006c540 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:29:27.700 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:27.698+0000 7f46d7fff640 1 --1- >> v1:172.21.15.37:6789/0 conn(0x7f46e006ca80 0x7f46e0160910 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.37:6789/0 says I am v1:172.21.15.37:59914/0 (socket says 172.21.15.37:59914) 2024-04-25T10:29:27.700 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:27.698+0000 7f46ded9d640 1 --2- >> v2:172.21.15.37:3300/0 conn(0x7f46e015a9f0 0x7f46e006c540 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.37:3300/0 says I am v2:172.21.15.37:35308/0 (socket says 172.21.15.37:35308) 2024-04-25T10:29:27.700 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:27.698+0000 7f46d7fff640 1 -- 172.21.15.37:0/1959948282 learned_addr learned my addr 172.21.15.37:0/1959948282 (peer_addr_for_me v1:172.21.15.37:0/0) 2024-04-25T10:29:27.700 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:27.698+0000 7f46ded9d640 1 -- 172.21.15.37:0/1959948282 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f46e0161020 msgr2=0x7f46e00b5860 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:29:27.700 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:27.698+0000 7f46ded9d640 1 --2- 172.21.15.37:0/1959948282 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f46e0161020 0x7f46e00b5860 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:29:27.700 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:27.698+0000 7f46ded9d640 1 -- 172.21.15.37:0/1959948282 >> v1:172.21.15.37:6789/0 conn(0x7f46e006ca80 legacy=0x7f46e0160910 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:29:27.700 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:27.698+0000 7f46ded9d640 1 -- 172.21.15.37:0/1959948282 --> v2:172.21.15.37:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f46e0163e00 con 0x7f46e015a9f0 2024-04-25T10:29:27.700 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:27.698+0000 7f46de59c640 1 -- 172.21.15.37:0/1959948282 <== mon.0 v1:172.21.15.37:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 563739531 0 0) 0x7f46e0161430 con 0x7f46e006ca80 2024-04-25T10:29:27.701 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:27.699+0000 7f46ded9d640 1 --2- 172.21.15.37:0/1959948282 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f46e015a9f0 0x7f46e006c540 secure :-1 s=READY pgs=171 cs=0 l=1 rev1=1 crypto rx=0x7f46d00097a0 tx=0x7f46d002a180 comp rx=0 tx=0).ready entity=mon.0 client_cookie=33e25444d08c6377 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:29:27.701 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:27.699+0000 7f46de59c640 1 -- 172.21.15.37:0/1959948282 <== mon.0 v2:172.21.15.37:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f46d0037070 con 0x7f46e015a9f0 2024-04-25T10:29:27.702 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:27.699+0000 7f46de59c640 1 -- 172.21.15.37:0/1959948282 <== mon.0 v2:172.21.15.37:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f46d0032440 con 0x7f46e015a9f0 2024-04-25T10:29:27.702 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:27.699+0000 7f46de59c640 1 -- 172.21.15.37:0/1959948282 <== mon.0 v2:172.21.15.37:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f46d0032810 con 0x7f46e015a9f0 2024-04-25T10:29:27.702 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:27.699+0000 7f46e5106640 1 -- 172.21.15.37:0/1959948282 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f46e015a9f0 msgr2=0x7f46e006c540 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:29:27.702 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:27.699+0000 7f46e5106640 1 --2- 172.21.15.37:0/1959948282 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f46e015a9f0 0x7f46e006c540 secure :-1 s=READY pgs=171 cs=0 l=1 rev1=1 crypto rx=0x7f46d00097a0 tx=0x7f46d002a180 comp rx=0 tx=0).stop 2024-04-25T10:29:27.702 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:27.700+0000 7f46e5106640 1 -- 172.21.15.37:0/1959948282 shutdown_connections 2024-04-25T10:29:27.702 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:27.700+0000 7f46e5106640 1 --2- 172.21.15.37:0/1959948282 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f46e0161020 0x7f46e00b5860 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:29:27.702 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:27.700+0000 7f46e5106640 1 --2- 172.21.15.37:0/1959948282 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f46e015a9f0 0x7f46e006c540 unknown :-1 s=CLOSED pgs=171 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:29:27.702 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:27.700+0000 7f46e5106640 1 -- 172.21.15.37:0/1959948282 >> 172.21.15.37:0/1959948282 conn(0x7f46e00b4560 msgr2=0x7f46e00b4970 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:29:27.702 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:27.700+0000 7f46e5106640 1 -- 172.21.15.37:0/1959948282 shutdown_connections 2024-04-25T10:29:27.702 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:27.700+0000 7f46e5106640 1 -- 172.21.15.37:0/1959948282 wait complete. 2024-04-25T10:29:27.703 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:27.702+0000 7f46e5106640 1 Processor -- start 2024-04-25T10:29:27.703 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:27.702+0000 7f46e5106640 1 -- start start 2024-04-25T10:29:27.705 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:27.703+0000 7f46e5106640 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f46e006ca80 0x7f46e00cbbe0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:29:27.706 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:27.703+0000 7f46e5106640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f46e015a9f0 0x7f46e01f5ea0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:29:27.706 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:27.703+0000 7f46e5106640 1 --2- >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f46e0161020 0x7f46e00d2520 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:29:27.706 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:27.703+0000 7f46e5106640 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 -- 0x7f46e01646a0 con 0x7f46e015a9f0 2024-04-25T10:29:27.706 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:27.703+0000 7f46e5106640 1 -- --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_getmap magic: 0 -- 0x7f46e01643c0 con 0x7f46e006ca80 2024-04-25T10:29:27.706 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:27.703+0000 7f46e5106640 1 -- --> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] -- mon_getmap magic: 0 -- 0x7f46e0164bc0 con 0x7f46e0161020 2024-04-25T10:29:27.706 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:27.703+0000 7f46d7fff640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f46e015a9f0 0x7f46e01f5ea0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:29:27.706 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:27.703+0000 7f46d7fff640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f46e015a9f0 0x7f46e01f5ea0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.37:3300/0 says I am v2:172.21.15.37:35320/0 (socket says 172.21.15.37:35320) 2024-04-25T10:29:27.706 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:27.703+0000 7f46d7fff640 1 -- 172.21.15.37:0/3901923221 learned_addr learned my addr 172.21.15.37:0/3901923221 (peer_addr_for_me v2:172.21.15.37:0/0) 2024-04-25T10:29:27.706 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:27.703+0000 7f46ded9d640 1 --2- 172.21.15.37:0/3901923221 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f46e006ca80 0x7f46e00cbbe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:29:27.706 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:27.703+0000 7f46df59e640 1 --2- 172.21.15.37:0/3901923221 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f46e0161020 0x7f46e00d2520 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:29:27.706 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:27.703+0000 7f46d7fff640 1 -- 172.21.15.37:0/3901923221 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f46e0161020 msgr2=0x7f46e00d2520 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:29:27.706 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:27.703+0000 7f46d7fff640 1 --2- 172.21.15.37:0/3901923221 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f46e0161020 0x7f46e00d2520 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:29:27.707 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:27.703+0000 7f46d7fff640 1 -- 172.21.15.37:0/3901923221 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f46e006ca80 msgr2=0x7f46e00cbbe0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:29:27.707 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:27.703+0000 7f46d7fff640 1 --2- 172.21.15.37:0/3901923221 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f46e006ca80 0x7f46e00cbbe0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:29:27.707 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:27.703+0000 7f46d7fff640 1 -- 172.21.15.37:0/3901923221 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f46e00cc4e0 con 0x7f46e015a9f0 2024-04-25T10:29:27.707 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:27.703+0000 7f46df59e640 1 --2- 172.21.15.37:0/3901923221 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f46e0161020 0x7f46e00d2520 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T10:29:27.707 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:27.704+0000 7f46d7fff640 1 --2- 172.21.15.37:0/3901923221 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f46e015a9f0 0x7f46e01f5ea0 secure :-1 s=READY pgs=172 cs=0 l=1 rev1=1 crypto rx=0x7f46c800c3e0 tx=0x7f46c800c8a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:29:27.707 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:27.704+0000 7f46d77fe640 1 -- 172.21.15.37:0/3901923221 <== mon.0 v2:172.21.15.37:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f46c8017070 con 0x7f46e015a9f0 2024-04-25T10:29:27.707 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:27.704+0000 7f46d77fe640 1 -- 172.21.15.37:0/3901923221 <== mon.0 v2:172.21.15.37:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f46c8005ce0 con 0x7f46e015a9f0 2024-04-25T10:29:27.707 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:27.704+0000 7f46e5106640 1 -- 172.21.15.37:0/3901923221 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f46e00cc7c0 con 0x7f46e015a9f0 2024-04-25T10:29:27.707 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:27.704+0000 7f46d77fe640 1 -- 172.21.15.37:0/3901923221 <== mon.0 v2:172.21.15.37:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f46c80072f0 con 0x7f46e015a9f0 2024-04-25T10:29:27.707 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:27.704+0000 7f46e5106640 1 -- 172.21.15.37:0/3901923221 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f46e00ccd80 con 0x7f46e015a9f0 2024-04-25T10:29:27.708 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:27.705+0000 7f46d77fe640 1 -- 172.21.15.37:0/3901923221 <== mon.0 v2:172.21.15.37:3300/0 4 ==== mgrmap(e 5) ==== 96782+0+0 (secure 0 0 0) 0x7f46c801a070 con 0x7f46e015a9f0 2024-04-25T10:29:27.708 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:27.706+0000 7f46d77fe640 1 --2- 172.21.15.37:0/3901923221 >> [v2:172.21.15.203:6832/2418205237,v1:172.21.15.203:6833/2418205237] conn(0x7f46c00744e0 0x7f46c00769f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:29:27.709 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:27.706+0000 7f46d77fe640 1 -- 172.21.15.37:0/3901923221 <== mon.0 v2:172.21.15.37:3300/0 5 ==== osd_map(23..23 src has 1..23) ==== 5709+0+0 (secure 0 0 0) 0x7f46c8094470 con 0x7f46e015a9f0 2024-04-25T10:29:27.709 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:27.706+0000 7f46ded9d640 1 --2- 172.21.15.37:0/3901923221 >> [v2:172.21.15.203:6832/2418205237,v1:172.21.15.203:6833/2418205237] conn(0x7f46c00744e0 0x7f46c00769f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:29:27.709 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:27.707+0000 7f46ded9d640 1 --2- 172.21.15.37:0/3901923221 >> [v2:172.21.15.203:6832/2418205237,v1:172.21.15.203:6833/2418205237] conn(0x7f46c00744e0 0x7f46c00769f0 secure :-1 s=READY pgs=112 cs=0 l=1 rev1=1 crypto rx=0x7f46d0034000 tx=0x7f46d002e000 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:29:27.709 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:27.707+0000 7f46ebfb9740 1 Processor -- start 2024-04-25T10:29:27.709 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:27.707+0000 7f46ebfb9740 1 -- start start 2024-04-25T10:29:27.711 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:27.708+0000 7f46ebfb9740 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x56017c0c1270 0x56017c48d600 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:29:27.711 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:27.708+0000 7f46ebfb9740 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x56017c0983b0 0x56017c489fa0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:29:27.711 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:27.708+0000 7f46ebfb9740 1 --2- >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x56017c0a0590 0x56017c463260 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:29:27.711 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:27.709+0000 7f46ebfb9740 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 -- 0x56017c3c7740 con 0x56017c0c1270 2024-04-25T10:29:27.711 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:27.709+0000 7f46ebfb9740 1 -- --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_getmap magic: 0 -- 0x56017c3633a0 con 0x56017c0983b0 2024-04-25T10:29:27.711 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:27.709+0000 7f46ebfb9740 1 -- --> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] -- mon_getmap magic: 0 -- 0x56017c3c3700 con 0x56017c0a0590 2024-04-25T10:29:27.711 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:27.709+0000 7f46ded9d640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x56017c0c1270 0x56017c48d600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:29:27.711 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:27.709+0000 7f46ded9d640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x56017c0c1270 0x56017c48d600 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.37:3300/0 says I am v2:172.21.15.37:35334/0 (socket says 172.21.15.37:35334) 2024-04-25T10:29:27.711 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:27.709+0000 7f46ded9d640 1 -- 172.21.15.37:0/2385377940 learned_addr learned my addr 172.21.15.37:0/2385377940 (peer_addr_for_me v2:172.21.15.37:0/0) 2024-04-25T10:29:27.711 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:27.709+0000 7f46ded9d640 1 -- 172.21.15.37:0/2385377940 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x56017c0a0590 msgr2=0x56017c463260 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:29:27.711 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:27.709+0000 7f46ded9d640 1 --2- 172.21.15.37:0/2385377940 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x56017c0a0590 0x56017c463260 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:29:27.712 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:27.709+0000 7f46ded9d640 1 -- 172.21.15.37:0/2385377940 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x56017c0983b0 msgr2=0x56017c489fa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:29:27.712 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:27.709+0000 7f46ded9d640 1 --2- 172.21.15.37:0/2385377940 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x56017c0983b0 0x56017c489fa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:29:27.712 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:27.709+0000 7f46ded9d640 1 -- 172.21.15.37:0/2385377940 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x56017c37f750 con 0x56017c0c1270 2024-04-25T10:29:27.712 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:27.709+0000 7f46ded9d640 1 --2- 172.21.15.37:0/2385377940 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x56017c0c1270 0x56017c48d600 secure :-1 s=READY pgs=173 cs=0 l=1 rev1=1 crypto rx=0x7f46d002a770 tx=0x7f46d002c000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a855668edc2d4a8c server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:29:27.712 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:27.709+0000 7f46d4ff9640 1 -- 172.21.15.37:0/2385377940 <== mon.0 v2:172.21.15.37:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f46d0037040 con 0x56017c0c1270 2024-04-25T10:29:27.712 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:27.709+0000 7f46d4ff9640 1 -- 172.21.15.37:0/2385377940 <== mon.0 v2:172.21.15.37:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f46d004e070 con 0x56017c0c1270 2024-04-25T10:29:27.712 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:27.709+0000 7f46d4ff9640 1 -- 172.21.15.37:0/2385377940 <== mon.0 v2:172.21.15.37:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f46d003c680 con 0x56017c0c1270 2024-04-25T10:29:27.713 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:27.710+0000 7f46ebfb9740 1 -- 172.21.15.37:0/2385377940 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x56017c0c1270 msgr2=0x56017c48d600 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:29:27.713 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:27.710+0000 7f46ebfb9740 1 --2- 172.21.15.37:0/2385377940 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x56017c0c1270 0x56017c48d600 secure :-1 s=READY pgs=173 cs=0 l=1 rev1=1 crypto rx=0x7f46d002a770 tx=0x7f46d002c000 comp rx=0 tx=0).stop 2024-04-25T10:29:27.713 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:27.710+0000 7f46ebfb9740 1 -- 172.21.15.37:0/2385377940 shutdown_connections 2024-04-25T10:29:27.713 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:27.710+0000 7f46ebfb9740 1 --2- 172.21.15.37:0/2385377940 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x56017c0a0590 0x56017c463260 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:29:27.713 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:27.710+0000 7f46ebfb9740 1 --2- 172.21.15.37:0/2385377940 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x56017c0983b0 0x56017c489fa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:29:27.713 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:27.710+0000 7f46ebfb9740 1 --2- 172.21.15.37:0/2385377940 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x56017c0c1270 0x56017c48d600 unknown :-1 s=CLOSED pgs=173 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:29:27.714 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:27.710+0000 7f46ebfb9740 1 -- 172.21.15.37:0/2385377940 >> 172.21.15.37:0/2385377940 conn(0x56017c0b53c0 msgr2=0x56017c4629f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:29:27.714 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:27.710+0000 7f46ebfb9740 1 -- 172.21.15.37:0/2385377940 shutdown_connections 2024-04-25T10:29:27.714 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:27.710+0000 7f46ebfb9740 1 -- 172.21.15.37:0/2385377940 wait complete. 2024-04-25T10:29:27.714 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:27.711+0000 7f46ebfb9740 1 -- start start 2024-04-25T10:29:27.714 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:27.711+0000 7f46ebfb9740 1 Processor -- start 2024-04-25T10:29:27.718 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:27.712+0000 7f46ebfb9740 1 --2- >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x56017c0983b0 0x56017c4a8500 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:29:27.718 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:27.713+0000 7f46ebfb9740 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x56017c0a0590 0x56017c4851d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:29:27.718 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:27.713+0000 7f46ebfb9740 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x56017c0c1270 0x56017c48bf50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:29:27.718 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:27.713+0000 7f46ebfb9740 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 -- 0x56017c34efe0 con 0x56017c0c1270 2024-04-25T10:29:27.718 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:27.713+0000 7f46ebfb9740 1 -- --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_getmap magic: 0 -- 0x56017c485c30 con 0x56017c0a0590 2024-04-25T10:29:27.718 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:27.713+0000 7f46ebfb9740 1 -- --> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] -- mon_getmap magic: 0 -- 0x56017c4a78c0 con 0x56017c0983b0 2024-04-25T10:29:27.718 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:27.713+0000 7f46ebfb9740 10 client.0 _ll_register_callbacks cb 0x56017c365150 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-25T10:29:27.718 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:27.713+0000 7f46df59e640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x56017c0c1270 0x56017c48bf50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:29:27.719 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:27.713+0000 7f46df59e640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x56017c0c1270 0x56017c48bf50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.37:3300/0 says I am v2:172.21.15.37:35346/0 (socket says 172.21.15.37:35346) 2024-04-25T10:29:27.719 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:27.713+0000 7f46df59e640 1 -- 172.21.15.37:0/421574098 learned_addr learned my addr 172.21.15.37:0/421574098 (peer_addr_for_me v2:172.21.15.37:0/0) 2024-04-25T10:29:27.719 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:27.713+0000 7f46d7fff640 1 --2- 172.21.15.37:0/421574098 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x56017c0a0590 0x56017c4851d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:29:27.719 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:27.713+0000 7f46ded9d640 1 --2- 172.21.15.37:0/421574098 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x56017c0983b0 0x56017c4a8500 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:29:27.719 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:27.713+0000 7f46df59e640 1 -- 172.21.15.37:0/421574098 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x56017c0983b0 msgr2=0x56017c4a8500 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:29:27.719 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:27.713+0000 7f46df59e640 1 --2- 172.21.15.37:0/421574098 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x56017c0983b0 0x56017c4a8500 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:29:27.719 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:27.713+0000 7f46df59e640 1 -- 172.21.15.37:0/421574098 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x56017c0a0590 msgr2=0x56017c4851d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:29:27.719 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:27.713+0000 7f46df59e640 1 --2- 172.21.15.37:0/421574098 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x56017c0a0590 0x56017c4851d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:29:27.719 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:27.713+0000 7f46df59e640 1 -- 172.21.15.37:0/421574098 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({osdmap=0}) -- 0x56017c0d92d0 con 0x56017c0c1270 2024-04-25T10:29:27.719 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:27.713+0000 7f46df59e640 1 --2- 172.21.15.37:0/421574098 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x56017c0c1270 0x56017c48bf50 secure :-1 s=READY pgs=174 cs=0 l=1 rev1=1 crypto rx=0x7f46d800ca00 tx=0x7f46d800cec0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=79bcc1a247a7e597 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:29:27.719 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:27.713+0000 7f46bdffb640 10 client.0 ms_handle_connect on v2:172.21.15.37:3300/0 2024-04-25T10:29:27.719 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:27.713+0000 7f46bdffb640 1 -- 172.21.15.37:0/421574098 <== mon.0 v2:172.21.15.37:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f46d801a060 con 0x56017c0c1270 2024-04-25T10:29:27.719 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:27.713+0000 7f46bdffb640 1 -- 172.21.15.37:0/421574098 <== mon.0 v2:172.21.15.37:3300/0 2 ==== osd_map(23..23 src has 1..23) ==== 5709+0+0 (secure 0 0 0) 0x7f46d80079b0 con 0x56017c0c1270 2024-04-25T10:29:27.720 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:27.713+0000 7f46ebfb9740 1 -- 172.21.15.37:0/421574098 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f46d004e070 con 0x56017c0c1270 2024-04-25T10:29:27.720 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:27.713+0000 7f46bdffb640 1 -- 172.21.15.37:0/421574098 <== mon.0 v2:172.21.15.37:3300/0 3 ==== mon_get_version_reply(handle=1 version=8) ==== 24+0+0 (secure 0 0 0) 0x7f46d8020050 con 0x56017c0c1270 2024-04-25T10:29:27.720 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:27.713+0000 7f46ebfb9740 10 client.4707 fetch_fsmap learned FSMap version 8 2024-04-25T10:29:27.720 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:27.713+0000 7f46ebfb9740 1 -- 172.21.15.37:0/421574098 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({config=0+,fsmap=8,monmap=2+}) -- 0x56017c1a36b0 con 0x56017c0c1270 2024-04-25T10:29:27.720 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:27.714+0000 7f46bdffb640 1 -- 172.21.15.37:0/421574098 <== mon.0 v2:172.21.15.37:3300/0 4 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f46d004e070 con 0x56017c0c1270 2024-04-25T10:29:27.720 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:27.714+0000 7f46bdffb640 1 -- 172.21.15.37:0/421574098 <== mon.0 v2:172.21.15.37:3300/0 5 ==== fsmap(e 8) ==== 3494+0+0 (secure 0 0 0) 0x7f46d8023020 con 0x56017c0c1270 2024-04-25T10:29:27.720 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:27.714+0000 7f46ebfb9740 10 client.4707 fetch_fsmap finished waiting for FSMap version 8 2024-04-25T10:29:27.720 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:27.714+0000 7f46ebfb9740 10 client.4707 resolve_mds: resolved name 'd' to daemon mds.d 2024-04-25T10:29:27.720 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:27.714+0000 7f46ebfb9740 20 client.4707 populate_metadata read hostname 'smithi037' 2024-04-25T10:29:27.720 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:27.714+0000 7f46ebfb9740 1 --2- 172.21.15.37:0/421574098 >> [v2:172.21.15.203:6836/3215191422,v1:172.21.15.203:6837/3215191422] conn(0x56017c07fe40 0x56017c48ef40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:29:27.720 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:27.714+0000 7f46ebfb9740 4 client.4707 mds_command: new command op to 4279 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-25T10:29:27.720 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:27.714+0000 7f46ebfb9740 1 -- 172.21.15.37:0/421574098 --> [v2:172.21.15.203:6836/3215191422,v1:172.21.15.203:6837/3215191422] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x56017c3d5350 con 0x56017c07fe40 2024-04-25T10:29:27.720 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:27.714+0000 7f46ded9d640 1 --2- 172.21.15.37:0/421574098 >> [v2:172.21.15.203:6836/3215191422,v1:172.21.15.203:6837/3215191422] conn(0x56017c07fe40 0x56017c48ef40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:29:27.720 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:27.715+0000 7f46ded9d640 1 --2- 172.21.15.37:0/421574098 >> [v2:172.21.15.203:6836/3215191422,v1:172.21.15.203:6837/3215191422] conn(0x56017c07fe40 0x56017c48ef40 crc :-1 s=READY pgs=11 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=53397a837850107a server_cookie=dc263d5834757764 in_seq=0 out_seq=0 2024-04-25T10:29:27.721 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:27.715+0000 7f46bdffb640 10 client.4707 ms_handle_connect on v2:172.21.15.203:6836/3215191422 2024-04-25T10:29:27.721 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:27.715+0000 7f46bdffb640 1 -- 172.21.15.37:0/421574098 <== mds.0 v2:172.21.15.203:6836/3215191422 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23648 (crc 0 0 0) 0x56017c3d5350 con 0x56017c07fe40 2024-04-25T10:29:27.721 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:27.715+0000 7f46bdffb640 10 client.4707 handle_command_reply: tid=0 2024-04-25T10:29:27.721 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:27.715+0000 7f46ebfb9740 1 client.4707 shutdown 2024-04-25T10:29:27.721 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:27.716+0000 7f46ebfb9740 1 -- 172.21.15.37:0/421574098 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x56017c0c1270 msgr2=0x56017c48bf50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:29:27.721 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:27.716+0000 7f46ebfb9740 1 --2- 172.21.15.37:0/421574098 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x56017c0c1270 0x56017c48bf50 secure :-1 s=READY pgs=174 cs=0 l=1 rev1=1 crypto rx=0x7f46d800ca00 tx=0x7f46d800cec0 comp rx=0 tx=0).stop 2024-04-25T10:29:27.721 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:27.716+0000 7f46ebfb9740 1 -- 172.21.15.37:0/421574098 shutdown_connections 2024-04-25T10:29:27.721 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:27.716+0000 7f46ebfb9740 1 --2- 172.21.15.37:0/421574098 >> [v2:172.21.15.203:6836/3215191422,v1:172.21.15.203:6837/3215191422] conn(0x56017c07fe40 0x56017c48ef40 crc :-1 s=READY pgs=11 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:29:27.721 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:27.716+0000 7f46ebfb9740 1 --2- 172.21.15.37:0/421574098 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x56017c0c1270 0x56017c48bf50 unknown :-1 s=CLOSED pgs=174 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:29:27.721 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:27.716+0000 7f46ebfb9740 1 --2- 172.21.15.37:0/421574098 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x56017c0a0590 0x56017c4851d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:29:27.721 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:27.716+0000 7f46bdffb640 0 client.4707 ms_handle_reset on v2:172.21.15.203:6836/3215191422 2024-04-25T10:29:27.721 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:27.716+0000 7f46ebfb9740 1 --2- 172.21.15.37:0/421574098 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x56017c0983b0 0x56017c4a8500 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:29:27.721 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:27.716+0000 7f46ebfb9740 1 -- 172.21.15.37:0/421574098 >> 172.21.15.37:0/421574098 conn(0x56017c0b53c0 msgr2=0x56017c4629f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:29:27.721 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:27.716+0000 7f46ebfb9740 1 -- 172.21.15.37:0/421574098 shutdown_connections 2024-04-25T10:29:27.722 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:27.716+0000 7f46ebfb9740 1 -- 172.21.15.37:0/421574098 wait complete. 2024-04-25T10:29:27.722 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:27.716+0000 7f46ebfb9740 20 client.4707 trim_cache size 0 max 16384 2024-04-25T10:29:27.736 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:27.735+0000 7f46ebfb9740 1 Processor -- start 2024-04-25T10:29:27.736 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:27.735+0000 7f46ebfb9740 1 -- start start 2024-04-25T10:29:27.739 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:27.736+0000 7f46ebfb9740 1 --2- >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x56017c0983b0 0x56017c45ca50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:29:27.739 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:27.736+0000 7f46ebfb9740 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x56017c0a0590 0x56017c485dc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:29:27.739 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:27.736+0000 7f46ebfb9740 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x56017c0c1270 0x56017c482fc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:29:27.739 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:27.736+0000 7f46ebfb9740 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 -- 0x7f46e00d10c0 con 0x56017c0c1270 2024-04-25T10:29:27.739 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:27.736+0000 7f46ebfb9740 1 -- --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_getmap magic: 0 -- 0x7f46e01faba0 con 0x56017c0a0590 2024-04-25T10:29:27.739 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:27.736+0000 7f46ebfb9740 1 -- --> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] -- mon_getmap magic: 0 -- 0x7f46e00d0a70 con 0x56017c0983b0 2024-04-25T10:29:27.739 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:27.736+0000 7f46df59e640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x56017c0c1270 0x56017c482fc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:29:27.739 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:27.736+0000 7f46df59e640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x56017c0c1270 0x56017c482fc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.37:3300/0 says I am v2:172.21.15.37:35354/0 (socket says 172.21.15.37:35354) 2024-04-25T10:29:27.740 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:27.736+0000 7f46df59e640 1 -- 172.21.15.37:0/4063363132 learned_addr learned my addr 172.21.15.37:0/4063363132 (peer_addr_for_me v2:172.21.15.37:0/0) 2024-04-25T10:29:27.740 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:27.736+0000 7f46ded9d640 1 --2- 172.21.15.37:0/4063363132 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x56017c0983b0 0x56017c45ca50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:29:27.740 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:27.736+0000 7f46d7fff640 1 --2- 172.21.15.37:0/4063363132 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x56017c0a0590 0x56017c485dc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:29:27.740 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:27.736+0000 7f46df59e640 1 -- 172.21.15.37:0/4063363132 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x56017c0983b0 msgr2=0x56017c45ca50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:29:27.740 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:27.736+0000 7f46df59e640 1 --2- 172.21.15.37:0/4063363132 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x56017c0983b0 0x56017c45ca50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:29:27.740 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:27.736+0000 7f46df59e640 1 -- 172.21.15.37:0/4063363132 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x56017c0a0590 msgr2=0x56017c485dc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:29:27.740 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:27.736+0000 7f46df59e640 1 --2- 172.21.15.37:0/4063363132 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x56017c0a0590 0x56017c485dc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:29:27.740 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:27.736+0000 7f46df59e640 1 -- 172.21.15.37:0/4063363132 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x56017c4856b0 con 0x56017c0c1270 2024-04-25T10:29:27.740 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:27.737+0000 7f46df59e640 1 --2- 172.21.15.37:0/4063363132 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x56017c0c1270 0x56017c482fc0 secure :-1 s=READY pgs=175 cs=0 l=1 rev1=1 crypto rx=0x7f46d8002230 tx=0x7f46d8011000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=88e39a2c89b1ae52 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:29:27.740 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:27.737+0000 7f46bffff640 1 -- 172.21.15.37:0/4063363132 <== mon.0 v2:172.21.15.37:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f46d802a020 con 0x56017c0c1270 2024-04-25T10:29:27.741 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:27.737+0000 7f46bffff640 1 -- 172.21.15.37:0/4063363132 <== mon.0 v2:172.21.15.37:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f46d8007bf0 con 0x56017c0c1270 2024-04-25T10:29:27.741 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:27.737+0000 7f46bffff640 1 -- 172.21.15.37:0/4063363132 <== mon.0 v2:172.21.15.37:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f46d8016680 con 0x56017c0c1270 2024-04-25T10:29:27.741 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:27.737+0000 7f46ebfb9740 1 -- 172.21.15.37:0/4063363132 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x56017c0c1270 msgr2=0x56017c482fc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:29:27.741 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:27.737+0000 7f46ebfb9740 1 --2- 172.21.15.37:0/4063363132 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x56017c0c1270 0x56017c482fc0 secure :-1 s=READY pgs=175 cs=0 l=1 rev1=1 crypto rx=0x7f46d8002230 tx=0x7f46d8011000 comp rx=0 tx=0).stop 2024-04-25T10:29:27.741 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:27.737+0000 7f46ebfb9740 1 -- 172.21.15.37:0/4063363132 shutdown_connections 2024-04-25T10:29:27.741 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:27.737+0000 7f46ebfb9740 1 --2- 172.21.15.37:0/4063363132 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x56017c0c1270 0x56017c482fc0 unknown :-1 s=CLOSED pgs=175 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:29:27.741 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:27.737+0000 7f46ebfb9740 1 --2- 172.21.15.37:0/4063363132 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x56017c0a0590 0x56017c485dc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:29:27.741 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:27.737+0000 7f46ebfb9740 1 --2- 172.21.15.37:0/4063363132 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x56017c0983b0 0x56017c45ca50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:29:27.741 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:27.737+0000 7f46ebfb9740 1 -- 172.21.15.37:0/4063363132 >> 172.21.15.37:0/4063363132 conn(0x56017c0b53c0 msgr2=0x56017c48cbf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:29:27.742 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:27.737+0000 7f46ebfb9740 1 -- 172.21.15.37:0/4063363132 shutdown_connections 2024-04-25T10:29:27.742 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:27.737+0000 7f46ebfb9740 1 -- 172.21.15.37:0/4063363132 wait complete. 2024-04-25T10:29:27.746 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:27.738+0000 7f46ebfb9740 1 -- start start 2024-04-25T10:29:27.746 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:27.738+0000 7f46ebfb9740 1 Processor -- start 2024-04-25T10:29:27.746 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:27.740+0000 7f46ebfb9740 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x56017c0983b0 0x56017c45c840 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:29:27.746 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:27.740+0000 7f46ebfb9740 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x56017c0a0590 0x56017c524a60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:29:27.746 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:27.740+0000 7f46ebfb9740 1 --2- >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x56017c0c1270 0x56017c4ca1d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:29:27.746 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:27.740+0000 7f46ebfb9740 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 -- 0x56017c4a6710 con 0x56017c0983b0 2024-04-25T10:29:27.746 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:27.740+0000 7f46ebfb9740 1 -- --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_getmap magic: 0 -- 0x56017c484db0 con 0x56017c0a0590 2024-04-25T10:29:27.746 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:27.740+0000 7f46ebfb9740 1 -- --> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] -- mon_getmap magic: 0 -- 0x56017c485930 con 0x56017c0c1270 2024-04-25T10:29:27.746 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:27.740+0000 7f46ebfb9740 10 client.0 _ll_register_callbacks cb 0x7f46d8009690 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-25T10:29:27.747 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:27.740+0000 7f46ded9d640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x56017c0983b0 0x56017c45c840 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:29:27.747 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:27.740+0000 7f46ded9d640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x56017c0983b0 0x56017c45c840 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.37:3300/0 says I am v2:172.21.15.37:35364/0 (socket says 172.21.15.37:35364) 2024-04-25T10:29:27.747 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:27.740+0000 7f46ded9d640 1 -- 172.21.15.37:0/2756216585 learned_addr learned my addr 172.21.15.37:0/2756216585 (peer_addr_for_me v2:172.21.15.37:0/0) 2024-04-25T10:29:27.747 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:27.740+0000 7f46d7fff640 1 --2- 172.21.15.37:0/2756216585 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x56017c0a0590 0x56017c524a60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:29:27.747 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:27.740+0000 7f46df59e640 1 --2- 172.21.15.37:0/2756216585 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x56017c0c1270 0x56017c4ca1d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:29:27.747 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:27.740+0000 7f46ded9d640 1 -- 172.21.15.37:0/2756216585 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x56017c0c1270 msgr2=0x56017c4ca1d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:29:27.747 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:27.740+0000 7f46ded9d640 1 --2- 172.21.15.37:0/2756216585 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x56017c0c1270 0x56017c4ca1d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:29:27.747 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:27.740+0000 7f46ded9d640 1 -- 172.21.15.37:0/2756216585 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x56017c0a0590 msgr2=0x56017c524a60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:29:27.747 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:27.740+0000 7f46ded9d640 1 --2- 172.21.15.37:0/2756216585 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x56017c0a0590 0x56017c524a60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:29:27.747 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:27.740+0000 7f46ded9d640 1 -- 172.21.15.37:0/2756216585 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({osdmap=0}) -- 0x56017c45cd80 con 0x56017c0983b0 2024-04-25T10:29:27.747 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:27.740+0000 7f46d7fff640 1 --2- 172.21.15.37:0/2756216585 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x56017c0a0590 0x56017c524a60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T10:29:27.747 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:27.740+0000 7f46ded9d640 1 --2- 172.21.15.37:0/2756216585 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x56017c0983b0 0x56017c45c840 secure :-1 s=READY pgs=176 cs=0 l=1 rev1=1 crypto rx=0x7f46d00098c0 tx=0x7f46d002c070 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c2976f04bd98bce4 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:29:27.747 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:27.740+0000 7f46bdffb640 10 client.0 ms_handle_connect on v2:172.21.15.37:3300/0 2024-04-25T10:29:27.748 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:27.740+0000 7f46bdffb640 1 -- 172.21.15.37:0/2756216585 <== mon.0 v2:172.21.15.37:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f46d0062060 con 0x56017c0983b0 2024-04-25T10:29:27.748 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:27.740+0000 7f46bdffb640 1 -- 172.21.15.37:0/2756216585 <== mon.0 v2:172.21.15.37:3300/0 2 ==== osd_map(23..23 src has 1..23) ==== 5709+0+0 (secure 0 0 0) 0x7f46d005d5b0 con 0x56017c0983b0 2024-04-25T10:29:27.748 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:27.740+0000 7f46ebfb9740 1 -- 172.21.15.37:0/2756216585 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f46d8007bf0 con 0x56017c0983b0 2024-04-25T10:29:27.748 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:27.740+0000 7f46bdffb640 1 -- 172.21.15.37:0/2756216585 <== mon.0 v2:172.21.15.37:3300/0 3 ==== mon_get_version_reply(handle=1 version=8) ==== 24+0+0 (secure 0 0 0) 0x7f46d005b050 con 0x56017c0983b0 2024-04-25T10:29:27.748 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:27.740+0000 7f46ebfb9740 10 client.4713 fetch_fsmap learned FSMap version 8 2024-04-25T10:29:27.748 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:27.740+0000 7f46ebfb9740 1 -- 172.21.15.37:0/2756216585 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({config=0+,fsmap=8,monmap=2+}) -- 0x56017c4cac30 con 0x56017c0983b0 2024-04-25T10:29:27.748 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:27.741+0000 7f46bdffb640 1 -- 172.21.15.37:0/2756216585 <== mon.0 v2:172.21.15.37:3300/0 4 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f46d8007bf0 con 0x56017c0983b0 2024-04-25T10:29:27.748 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:27.741+0000 7f46bdffb640 1 -- 172.21.15.37:0/2756216585 <== mon.0 v2:172.21.15.37:3300/0 5 ==== fsmap(e 8) ==== 3494+0+0 (secure 0 0 0) 0x7f46d005d7b0 con 0x56017c0983b0 2024-04-25T10:29:27.748 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:27.741+0000 7f46ebfb9740 10 client.4713 fetch_fsmap finished waiting for FSMap version 8 2024-04-25T10:29:27.748 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:27.741+0000 7f46ebfb9740 10 client.4713 resolve_mds: resolved name 'd' to daemon mds.d 2024-04-25T10:29:27.748 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:27.741+0000 7f46ebfb9740 20 client.4713 populate_metadata read hostname 'smithi037' 2024-04-25T10:29:27.749 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:27.741+0000 7f46ebfb9740 1 --2- 172.21.15.37:0/2756216585 >> [v2:172.21.15.203:6836/3215191422,v1:172.21.15.203:6837/3215191422] conn(0x56017c4c17c0 0x56017c528680 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:29:27.749 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:27.741+0000 7f46ebfb9740 4 client.4713 mds_command: new command op to 4279 tid=0 multi_id=0 [{"prefix": "status"}] 2024-04-25T10:29:27.749 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:27.741+0000 7f46ebfb9740 1 -- 172.21.15.37:0/2756216585 --> [v2:172.21.15.203:6836/3215191422,v1:172.21.15.203:6837/3215191422] -- command(tid 0: {"prefix": "status"}) -- 0x56017c3cf1c0 con 0x56017c4c17c0 2024-04-25T10:29:27.749 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:27.741+0000 7f46d7fff640 1 --2- 172.21.15.37:0/2756216585 >> [v2:172.21.15.203:6836/3215191422,v1:172.21.15.203:6837/3215191422] conn(0x56017c4c17c0 0x56017c528680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:29:27.749 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:27.741+0000 7f46bdffb640 10 client.4713 ms_handle_connect on v2:172.21.15.203:6836/3215191422 2024-04-25T10:29:27.749 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:27.741+0000 7f46d7fff640 1 --2- 172.21.15.37:0/2756216585 >> [v2:172.21.15.203:6836/3215191422,v1:172.21.15.203:6837/3215191422] conn(0x56017c4c17c0 0x56017c528680 crc :-1 s=READY pgs=12 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=476ecbd65856bdba server_cookie=e1d3ec363925a363 in_seq=0 out_seq=0 2024-04-25T10:29:27.749 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:27.741+0000 7f46bdffb640 1 -- 172.21.15.37:0/2756216585 <== mds.0 v2:172.21.15.203:6836/3215191422 1 ==== command_reply(tid 0: 0 ) ==== 8+0+330 (crc 0 0 0) 0x56017c3cf1c0 con 0x56017c4c17c0 2024-04-25T10:29:27.749 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:27.741+0000 7f46bdffb640 10 client.4713 handle_command_reply: tid=0 2024-04-25T10:29:27.749 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:27.742+0000 7f46ebfb9740 1 client.4713 shutdown 2024-04-25T10:29:27.749 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:27.742+0000 7f46ebfb9740 1 -- 172.21.15.37:0/2756216585 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x56017c0983b0 msgr2=0x56017c45c840 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:29:27.749 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:27.742+0000 7f46ebfb9740 1 --2- 172.21.15.37:0/2756216585 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x56017c0983b0 0x56017c45c840 secure :-1 s=READY pgs=176 cs=0 l=1 rev1=1 crypto rx=0x7f46d00098c0 tx=0x7f46d002c070 comp rx=0 tx=0).stop 2024-04-25T10:29:27.749 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:27.742+0000 7f46ebfb9740 1 -- 172.21.15.37:0/2756216585 shutdown_connections 2024-04-25T10:29:27.750 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:27.742+0000 7f46ebfb9740 1 --2- 172.21.15.37:0/2756216585 >> [v2:172.21.15.203:6836/3215191422,v1:172.21.15.203:6837/3215191422] conn(0x56017c4c17c0 0x56017c528680 crc :-1 s=READY pgs=12 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:29:27.750 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:27.742+0000 7f46ebfb9740 1 --2- 172.21.15.37:0/2756216585 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x56017c0c1270 0x56017c4ca1d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:29:27.750 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:27.742+0000 7f46ebfb9740 1 --2- 172.21.15.37:0/2756216585 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x56017c0a0590 0x56017c524a60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:29:27.750 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:27.742+0000 7f46ebfb9740 1 --2- 172.21.15.37:0/2756216585 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x56017c0983b0 0x56017c45c840 unknown :-1 s=CLOSED pgs=176 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:29:27.750 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:27.742+0000 7f46bdffb640 0 client.4713 ms_handle_reset on v2:172.21.15.203:6836/3215191422 2024-04-25T10:29:27.750 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:27.742+0000 7f46ebfb9740 1 -- 172.21.15.37:0/2756216585 >> 172.21.15.37:0/2756216585 conn(0x56017c0b53c0 msgr2=0x56017c461b70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:29:27.750 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:27.742+0000 7f46ebfb9740 1 -- 172.21.15.37:0/2756216585 shutdown_connections 2024-04-25T10:29:27.750 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:27.742+0000 7f46ebfb9740 1 -- 172.21.15.37:0/2756216585 wait complete. 2024-04-25T10:29:27.750 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:27.742+0000 7f46ebfb9740 20 client.4713 trim_cache size 0 max 16384 2024-04-25T10:29:27.751 INFO:teuthology.orchestra.run.smithi037.stdout:{ 2024-04-25T10:29:27.751 INFO:teuthology.orchestra.run.smithi037.stdout: "cluster_fsid": "f4acda28-be34-492a-b286-994afd36f61d", 2024-04-25T10:29:27.751 INFO:teuthology.orchestra.run.smithi037.stdout: "whoami": 0, 2024-04-25T10:29:27.751 INFO:teuthology.orchestra.run.smithi037.stdout: "id": 4279, 2024-04-25T10:29:27.751 INFO:teuthology.orchestra.run.smithi037.stdout: "want_state": "up:active", 2024-04-25T10:29:27.751 INFO:teuthology.orchestra.run.smithi037.stdout: "state": "up:active", 2024-04-25T10:29:27.751 INFO:teuthology.orchestra.run.smithi037.stdout: "fs_name": "cephfs", 2024-04-25T10:29:27.751 INFO:teuthology.orchestra.run.smithi037.stdout: "rank_uptime": 18.719768855000002, 2024-04-25T10:29:27.751 INFO:teuthology.orchestra.run.smithi037.stdout: "mdsmap_epoch": 8, 2024-04-25T10:29:27.751 INFO:teuthology.orchestra.run.smithi037.stdout: "osdmap_epoch": 23, 2024-04-25T10:29:27.751 INFO:teuthology.orchestra.run.smithi037.stdout: "osdmap_epoch_barrier": 23, 2024-04-25T10:29:27.752 INFO:teuthology.orchestra.run.smithi037.stdout: "uptime": 25.835524073999999 2024-04-25T10:29:27.752 INFO:teuthology.orchestra.run.smithi037.stdout:} 2024-04-25T10:29:27.753 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:27.742+0000 7f46e5106640 1 -- 172.21.15.37:0/3901923221 >> [v2:172.21.15.203:6832/2418205237,v1:172.21.15.203:6833/2418205237] conn(0x7f46c00744e0 msgr2=0x7f46c00769f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:29:27.753 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:27.742+0000 7f46e5106640 1 --2- 172.21.15.37:0/3901923221 >> [v2:172.21.15.203:6832/2418205237,v1:172.21.15.203:6833/2418205237] conn(0x7f46c00744e0 0x7f46c00769f0 secure :-1 s=READY pgs=112 cs=0 l=1 rev1=1 crypto rx=0x7f46d0034000 tx=0x7f46d002e000 comp rx=0 tx=0).stop 2024-04-25T10:29:27.753 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:27.742+0000 7f46e5106640 1 -- 172.21.15.37:0/3901923221 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f46e015a9f0 msgr2=0x7f46e01f5ea0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:29:27.753 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:27.742+0000 7f46e5106640 1 --2- 172.21.15.37:0/3901923221 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f46e015a9f0 0x7f46e01f5ea0 secure :-1 s=READY pgs=172 cs=0 l=1 rev1=1 crypto rx=0x7f46c800c3e0 tx=0x7f46c800c8a0 comp rx=0 tx=0).stop 2024-04-25T10:29:27.753 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:27.742+0000 7f46e5106640 1 -- 172.21.15.37:0/3901923221 shutdown_connections 2024-04-25T10:29:27.753 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:27.742+0000 7f46e5106640 1 --2- 172.21.15.37:0/3901923221 >> [v2:172.21.15.203:6832/2418205237,v1:172.21.15.203:6833/2418205237] conn(0x7f46c00744e0 0x7f46c00769f0 unknown :-1 s=CLOSED pgs=112 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:29:27.753 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:27.742+0000 7f46e5106640 1 --2- 172.21.15.37:0/3901923221 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f46e0161020 0x7f46e00d2520 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:29:27.753 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:27.742+0000 7f46e5106640 1 --2- 172.21.15.37:0/3901923221 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f46e015a9f0 0x7f46e01f5ea0 unknown :-1 s=CLOSED pgs=172 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:29:27.753 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:27.742+0000 7f46e5106640 1 --2- 172.21.15.37:0/3901923221 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f46e006ca80 0x7f46e00cbbe0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:29:27.753 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:27.742+0000 7f46e5106640 1 -- 172.21.15.37:0/3901923221 >> 172.21.15.37:0/3901923221 conn(0x7f46e00b4560 msgr2=0x7f46e00b5c20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:29:27.754 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:27.742+0000 7f46e5106640 1 -- 172.21.15.37:0/3901923221 shutdown_connections 2024-04-25T10:29:27.754 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:27.742+0000 7f46e5106640 1 -- 172.21.15.37:0/3901923221 wait complete. 2024-04-25T10:29:27.755 INFO:teuthology.run_tasks:Running task sequential... 2024-04-25T10:29:27.766 INFO:teuthology.task.sequential:In sequential, running task sequential... 2024-04-25T10:29:27.766 INFO:teuthology.task.sequential:In sequential, running task print... 2024-04-25T10:29:27.767 INFO:teuthology.task.print:Enabling mgr modules 2024-04-25T10:29:27.767 INFO:teuthology.run_tasks:Running task ceph-fuse... 2024-04-25T10:29:27.793 INFO:tasks.ceph_fuse:Running ceph_fuse task... 2024-04-25T10:29:27.794 INFO:tasks.ceph_fuse:config is {'client.0': {}, 'client.2': {}, 'client.1': {}, 'client.3': {}} 2024-04-25T10:29:27.794 INFO:tasks.ceph_fuse:client.0 config is {} 2024-04-25T10:29:27.794 INFO:tasks.cephfs.mount:cephfs_mntpt = None 2024-04-25T10:29:27.794 INFO:tasks.cephfs.mount:hostfs_mntpt = /home/ubuntu/cephtest/mnt.0 2024-04-25T10:29:27.794 INFO:tasks.ceph_fuse:client.2 config is {} 2024-04-25T10:29:27.794 INFO:tasks.cephfs.mount:cephfs_mntpt = None 2024-04-25T10:29:27.794 INFO:tasks.cephfs.mount:hostfs_mntpt = /home/ubuntu/cephtest/mnt.2 2024-04-25T10:29:27.795 INFO:tasks.ceph_fuse:client.1 config is {} 2024-04-25T10:29:27.795 INFO:tasks.cephfs.mount:cephfs_mntpt = None 2024-04-25T10:29:27.795 INFO:tasks.cephfs.mount:hostfs_mntpt = /home/ubuntu/cephtest/mnt.1 2024-04-25T10:29:27.795 INFO:tasks.ceph_fuse:client.3 config is {} 2024-04-25T10:29:27.795 INFO:tasks.cephfs.mount:cephfs_mntpt = None 2024-04-25T10:29:27.795 INFO:tasks.cephfs.mount:hostfs_mntpt = /home/ubuntu/cephtest/mnt.3 2024-04-25T10:29:27.795 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T10:29:27.795 DEBUG:teuthology.orchestra.run.smithi037:> ip netns list 2024-04-25T10:29:27.811 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T10:29:27.811 DEBUG:teuthology.orchestra.run.smithi037:> sudo ip link delete ceph-brx 2024-04-25T10:29:27.878 INFO:teuthology.orchestra.run.smithi037.stderr:Cannot find device "ceph-brx" 2024-04-25T10:29:27.879 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-04-25T10:29:27.879 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T10:29:27.880 DEBUG:teuthology.orchestra.run.smithi203:> ip netns list 2024-04-25T10:29:27.898 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T10:29:27.898 DEBUG:teuthology.orchestra.run.smithi203:> sudo ip link delete ceph-brx 2024-04-25T10:29:27.969 INFO:teuthology.orchestra.run.smithi203.stderr:Cannot find device "ceph-brx" 2024-04-25T10:29:27.970 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-04-25T10:29:27.970 INFO:tasks.ceph_fuse:Mounting ceph-fuse clients... 2024-04-25T10:29:27.971 DEBUG:teuthology.orchestra.run.smithi037:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs ls 2024-04-25T10:29:28.082 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:28.081+0000 7f0a31834640 1 Processor -- start 2024-04-25T10:29:28.083 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:28.081+0000 7f0a31834640 1 -- start start 2024-04-25T10:29:28.084 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:28.083+0000 7f0a31834640 1 --2- >> v2:172.21.15.203:3300/0 conn(0x7f0a2c0b4bb0 0x7f0a2c1646b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:29:28.084 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:28.083+0000 7f0a31834640 1 --2- >> v2:172.21.15.37:3300/0 conn(0x7f0a2c0d4560 0x7f0a2c0d4970 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:29:28.085 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:28.083+0000 7f0a31834640 1 -- --> v1:172.21.15.37:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f0a2c161190 con 0x7f0a2c0b7600 2024-04-25T10:29:28.085 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:28.083+0000 7f0a31834640 1 -- --> v2:172.21.15.37:3300/0 -- mon_getmap magic: 0 -- 0x7f0a2c161470 con 0x7f0a2c0d4560 2024-04-25T10:29:28.085 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:28.083+0000 7f0a31834640 1 -- --> v2:172.21.15.203:3300/0 -- mon_getmap magic: 0 -- 0x7f0a2c163eb0 con 0x7f0a2c0b4bb0 2024-04-25T10:29:28.085 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:28.083+0000 7f0a2b7fe640 1 --2- >> v2:172.21.15.37:3300/0 conn(0x7f0a2c0d4560 0x7f0a2c0d4970 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:29:28.085 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:28.083+0000 7f0a2b7fe640 1 --2- >> v2:172.21.15.37:3300/0 conn(0x7f0a2c0d4560 0x7f0a2c0d4970 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.37:3300/0 says I am v2:172.21.15.37:35372/0 (socket says 172.21.15.37:35372) 2024-04-25T10:29:28.085 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:28.083+0000 7f0a2a7fc640 1 --1- >> v1:172.21.15.37:6789/0 conn(0x7f0a2c0b7600 0x7f0a2c0b7a10 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.37:6789/0 says I am v1:172.21.15.37:59928/0 (socket says 172.21.15.37:59928) 2024-04-25T10:29:28.085 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:28.083+0000 7f0a2b7fe640 1 -- 172.21.15.37:0/2666477026 learned_addr learned my addr 172.21.15.37:0/2666477026 (peer_addr_for_me v2:172.21.15.37:0/0) 2024-04-25T10:29:28.085 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:28.083+0000 7f0a2affd640 1 --2- 172.21.15.37:0/2666477026 >> v2:172.21.15.203:3300/0 conn(0x7f0a2c0b4bb0 0x7f0a2c1646b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:29:28.085 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:28.083+0000 7f0a2b7fe640 1 -- 172.21.15.37:0/2666477026 >> v2:172.21.15.203:3300/0 conn(0x7f0a2c0b4bb0 msgr2=0x7f0a2c1646b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:29:28.085 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:28.083+0000 7f0a2b7fe640 1 --2- 172.21.15.37:0/2666477026 >> v2:172.21.15.203:3300/0 conn(0x7f0a2c0b4bb0 0x7f0a2c1646b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:29:28.086 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:28.083+0000 7f0a2b7fe640 1 -- 172.21.15.37:0/2666477026 >> v1:172.21.15.37:6789/0 conn(0x7f0a2c0b7600 legacy=0x7f0a2c0b7a10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:29:28.086 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:28.083+0000 7f0a2b7fe640 1 -- 172.21.15.37:0/2666477026 --> v2:172.21.15.37:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f0a2c164190 con 0x7f0a2c0d4560 2024-04-25T10:29:28.086 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:28.083+0000 7f0a29ffb640 1 -- 172.21.15.37:0/2666477026 <== mon.0 v1:172.21.15.37:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 865751030 0 0) 0x7f0a2c161190 con 0x7f0a2c0b7600 2024-04-25T10:29:28.087 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:28.084+0000 7f0a2b7fe640 1 --2- 172.21.15.37:0/2666477026 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f0a2c0d4560 0x7f0a2c0d4970 secure :-1 s=READY pgs=178 cs=0 l=1 rev1=1 crypto rx=0x7f0a2c08e640 tx=0x7f0a1c02a730 comp rx=0 tx=0).ready entity=mon.0 client_cookie=565afaf204b7bf21 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:29:28.087 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:28.084+0000 7f0a29ffb640 1 -- 172.21.15.37:0/2666477026 <== mon.0 v2:172.21.15.37:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0a1c036070 con 0x7f0a2c0d4560 2024-04-25T10:29:28.088 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:28.084+0000 7f0a29ffb640 1 -- 172.21.15.37:0/2666477026 <== mon.0 v2:172.21.15.37:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f0a1c031600 con 0x7f0a2c0d4560 2024-04-25T10:29:28.088 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:28.084+0000 7f0a29ffb640 1 -- 172.21.15.37:0/2666477026 <== mon.0 v2:172.21.15.37:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0a1c0319d0 con 0x7f0a2c0d4560 2024-04-25T10:29:28.088 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:28.085+0000 7f0a31834640 1 -- 172.21.15.37:0/2666477026 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f0a2c0d4560 msgr2=0x7f0a2c0d4970 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:29:28.088 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:28.085+0000 7f0a31834640 1 --2- 172.21.15.37:0/2666477026 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f0a2c0d4560 0x7f0a2c0d4970 secure :-1 s=READY pgs=178 cs=0 l=1 rev1=1 crypto rx=0x7f0a2c08e640 tx=0x7f0a1c02a730 comp rx=0 tx=0).stop 2024-04-25T10:29:28.088 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:28.085+0000 7f0a31834640 1 -- 172.21.15.37:0/2666477026 shutdown_connections 2024-04-25T10:29:28.088 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:28.085+0000 7f0a31834640 1 --2- 172.21.15.37:0/2666477026 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f0a2c0d4560 0x7f0a2c0d4970 unknown :-1 s=CLOSED pgs=178 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:29:28.088 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:28.085+0000 7f0a31834640 1 --2- 172.21.15.37:0/2666477026 >> v2:172.21.15.203:3300/0 conn(0x7f0a2c0b4bb0 0x7f0a2c1646b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:29:28.088 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:28.085+0000 7f0a31834640 1 -- 172.21.15.37:0/2666477026 >> 172.21.15.37:0/2666477026 conn(0x7f0a2c0d27e0 msgr2=0x7f0a2c0d2bf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:29:28.088 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:28.085+0000 7f0a31834640 1 -- 172.21.15.37:0/2666477026 shutdown_connections 2024-04-25T10:29:28.088 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:28.085+0000 7f0a31834640 1 -- 172.21.15.37:0/2666477026 wait complete. 2024-04-25T10:29:28.089 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:28.086+0000 7f0a31834640 1 Processor -- start 2024-04-25T10:29:28.089 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:28.087+0000 7f0a31834640 1 -- start start 2024-04-25T10:29:28.091 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:28.088+0000 7f0a31834640 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f0a2c0b4bb0 0x7f0a2c0ccc10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:29:28.091 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:28.088+0000 7f0a31834640 1 --2- >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f0a2c0b7600 0x7f0a2c14cf50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:29:28.091 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:28.088+0000 7f0a31834640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f0a2c0d4560 0x7f0a2c15fb90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:29:28.091 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:28.088+0000 7f0a31834640 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 -- 0x7f0a2c0d09e0 con 0x7f0a2c0d4560 2024-04-25T10:29:28.091 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:28.088+0000 7f0a31834640 1 -- --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_getmap magic: 0 -- 0x7f0a2c164bf0 con 0x7f0a2c0b4bb0 2024-04-25T10:29:28.091 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:28.088+0000 7f0a31834640 1 -- --> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] -- mon_getmap magic: 0 -- 0x7f0a2c0d0b60 con 0x7f0a2c0b7600 2024-04-25T10:29:28.091 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:28.088+0000 7f0a2b7fe640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f0a2c0d4560 0x7f0a2c15fb90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:29:28.092 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:28.088+0000 7f0a2b7fe640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f0a2c0d4560 0x7f0a2c15fb90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.37:3300/0 says I am v2:172.21.15.37:35374/0 (socket says 172.21.15.37:35374) 2024-04-25T10:29:28.092 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:28.088+0000 7f0a2b7fe640 1 -- 172.21.15.37:0/3962479604 learned_addr learned my addr 172.21.15.37:0/3962479604 (peer_addr_for_me v2:172.21.15.37:0/0) 2024-04-25T10:29:28.092 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:28.088+0000 7f0a2affd640 1 --2- 172.21.15.37:0/3962479604 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f0a2c0b4bb0 0x7f0a2c0ccc10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:29:28.092 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:28.088+0000 7f0a2a7fc640 1 --2- 172.21.15.37:0/3962479604 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f0a2c0b7600 0x7f0a2c14cf50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:29:28.092 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:28.088+0000 7f0a2b7fe640 1 -- 172.21.15.37:0/3962479604 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f0a2c0b7600 msgr2=0x7f0a2c14cf50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:29:28.092 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:28.088+0000 7f0a2b7fe640 1 --2- 172.21.15.37:0/3962479604 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f0a2c0b7600 0x7f0a2c14cf50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:29:28.092 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:28.088+0000 7f0a2b7fe640 1 -- 172.21.15.37:0/3962479604 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f0a2c0b4bb0 msgr2=0x7f0a2c0ccc10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:29:28.092 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:28.088+0000 7f0a2b7fe640 1 --2- 172.21.15.37:0/3962479604 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f0a2c0b4bb0 0x7f0a2c0ccc10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:29:28.092 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:28.088+0000 7f0a2b7fe640 1 -- 172.21.15.37:0/3962479604 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f0a2c0ce700 con 0x7f0a2c0d4560 2024-04-25T10:29:28.092 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:28.089+0000 7f0a2b7fe640 1 --2- 172.21.15.37:0/3962479604 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f0a2c0d4560 0x7f0a2c15fb90 secure :-1 s=READY pgs=179 cs=0 l=1 rev1=1 crypto rx=0x7f0a1c034040 tx=0x7f0a1c004a40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:29:28.093 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:28.089+0000 7f0a13fff640 1 -- 172.21.15.37:0/3962479604 <== mon.0 v2:172.21.15.37:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0a1c041020 con 0x7f0a2c0d4560 2024-04-25T10:29:28.093 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:28.089+0000 7f0a13fff640 1 -- 172.21.15.37:0/3962479604 <== mon.0 v2:172.21.15.37:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f0a1c037040 con 0x7f0a2c0d4560 2024-04-25T10:29:28.093 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:28.089+0000 7f0a31834640 1 -- 172.21.15.37:0/3962479604 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f0a2c0cd150 con 0x7f0a2c0d4560 2024-04-25T10:29:28.093 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:28.089+0000 7f0a13fff640 1 -- 172.21.15.37:0/3962479604 <== mon.0 v2:172.21.15.37:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0a1c036050 con 0x7f0a2c0d4560 2024-04-25T10:29:28.093 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:28.089+0000 7f0a31834640 1 -- 172.21.15.37:0/3962479604 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f0a2c0cd940 con 0x7f0a2c0d4560 2024-04-25T10:29:28.093 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:28.090+0000 7f0a31834640 1 -- 172.21.15.37:0/3962479604 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f0a2c0cd3d0 con 0x7f0a2c0d4560 2024-04-25T10:29:28.094 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:28.091+0000 7f0a13fff640 1 -- 172.21.15.37:0/3962479604 <== mon.0 v2:172.21.15.37:3300/0 4 ==== mgrmap(e 5) ==== 96782+0+0 (secure 0 0 0) 0x7f0a1c007c50 con 0x7f0a2c0d4560 2024-04-25T10:29:28.094 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:28.091+0000 7f0a13fff640 1 --2- 172.21.15.37:0/3962479604 >> [v2:172.21.15.203:6832/2418205237,v1:172.21.15.203:6833/2418205237] conn(0x7f0a180744b0 0x7f0a180769c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:29:28.094 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:28.091+0000 7f0a13fff640 1 -- 172.21.15.37:0/3962479604 <== mon.0 v2:172.21.15.37:3300/0 5 ==== osd_map(23..23 src has 1..23) ==== 5709+0+0 (secure 0 0 0) 0x7f0a1c0b5940 con 0x7f0a2c0d4560 2024-04-25T10:29:28.094 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:28.091+0000 7f0a2affd640 1 --2- 172.21.15.37:0/3962479604 >> [v2:172.21.15.203:6832/2418205237,v1:172.21.15.203:6833/2418205237] conn(0x7f0a180744b0 0x7f0a180769c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:29:28.094 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:28.092+0000 7f0a2affd640 1 --2- 172.21.15.37:0/3962479604 >> [v2:172.21.15.203:6832/2418205237,v1:172.21.15.203:6833/2418205237] conn(0x7f0a180744b0 0x7f0a180769c0 secure :-1 s=READY pgs=113 cs=0 l=1 rev1=1 crypto rx=0x7f0a2c0d6c00 tx=0x7f0a24008040 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:29:28.097 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:28.096+0000 7f0a13fff640 1 -- 172.21.15.37:0/3962479604 <== mon.0 v2:172.21.15.37:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187246 (secure 0 0 0) 0x7f0a1c04b060 con 0x7f0a2c0d4560 2024-04-25T10:29:28.332 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:28.330+0000 7f0a31834640 1 -- 172.21.15.37:0/3962479604 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_command({"prefix": "fs ls"} v 0) -- 0x7f0a2c06a140 con 0x7f0a2c0d4560 2024-04-25T10:29:28.332 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:28.331+0000 7f0a13fff640 1 -- 172.21.15.37:0/3962479604 <== mon.0 v2:172.21.15.37:3300/0 7 ==== mon_command_ack([{"prefix": "fs ls"}]=0 v8) ==== 53+0+73 (secure 0 0 0) 0x7f0a1c07fba0 con 0x7f0a2c0d4560 2024-04-25T10:29:28.332 INFO:teuthology.orchestra.run.smithi037.stdout:name: cephfs, metadata pool: cephfs_metadata, data pools: [cephfs_data ] 2024-04-25T10:29:28.334 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:28.333+0000 7f0a31834640 1 -- 172.21.15.37:0/3962479604 >> [v2:172.21.15.203:6832/2418205237,v1:172.21.15.203:6833/2418205237] conn(0x7f0a180744b0 msgr2=0x7f0a180769c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:29:28.335 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:28.333+0000 7f0a31834640 1 --2- 172.21.15.37:0/3962479604 >> [v2:172.21.15.203:6832/2418205237,v1:172.21.15.203:6833/2418205237] conn(0x7f0a180744b0 0x7f0a180769c0 secure :-1 s=READY pgs=113 cs=0 l=1 rev1=1 crypto rx=0x7f0a2c0d6c00 tx=0x7f0a24008040 comp rx=0 tx=0).stop 2024-04-25T10:29:28.335 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:28.333+0000 7f0a31834640 1 -- 172.21.15.37:0/3962479604 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f0a2c0d4560 msgr2=0x7f0a2c15fb90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:29:28.335 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:28.333+0000 7f0a31834640 1 --2- 172.21.15.37:0/3962479604 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f0a2c0d4560 0x7f0a2c15fb90 secure :-1 s=READY pgs=179 cs=0 l=1 rev1=1 crypto rx=0x7f0a1c034040 tx=0x7f0a1c004a40 comp rx=0 tx=0).stop 2024-04-25T10:29:28.335 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:28.333+0000 7f0a31834640 1 -- 172.21.15.37:0/3962479604 shutdown_connections 2024-04-25T10:29:28.335 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:28.333+0000 7f0a31834640 1 --2- 172.21.15.37:0/3962479604 >> [v2:172.21.15.203:6832/2418205237,v1:172.21.15.203:6833/2418205237] conn(0x7f0a180744b0 0x7f0a180769c0 unknown :-1 s=CLOSED pgs=113 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:29:28.335 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:28.333+0000 7f0a31834640 1 --2- 172.21.15.37:0/3962479604 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f0a2c0d4560 0x7f0a2c15fb90 unknown :-1 s=CLOSED pgs=179 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:29:28.335 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:28.333+0000 7f0a31834640 1 --2- 172.21.15.37:0/3962479604 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f0a2c0b7600 0x7f0a2c14cf50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:29:28.335 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:28.333+0000 7f0a31834640 1 --2- 172.21.15.37:0/3962479604 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f0a2c0b4bb0 0x7f0a2c0ccc10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:29:28.335 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:28.333+0000 7f0a31834640 1 -- 172.21.15.37:0/3962479604 >> 172.21.15.37:0/3962479604 conn(0x7f0a2c0d27e0 msgr2=0x7f0a2c0d4970 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:29:28.335 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:28.333+0000 7f0a31834640 1 -- 172.21.15.37:0/3962479604 shutdown_connections 2024-04-25T10:29:28.335 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:28.333+0000 7f0a31834640 1 -- 172.21.15.37:0/3962479604 wait complete. 2024-04-25T10:29:28.345 INFO:tasks.cephfs.mount:Mounting default Ceph FS; just confirmed its presence on cluster 2024-04-25T10:29:28.345 INFO:tasks.cephfs.mount:Mounting Ceph FS. Following are details of mount; remember "None" represents Python type None - 2024-04-25T10:29:28.345 INFO:tasks.cephfs.mount:self.client_remote.hostname = smithi037.front.sepia.ceph.com 2024-04-25T10:29:28.345 INFO:tasks.cephfs.mount:self.client.name = client.0 2024-04-25T10:29:28.346 INFO:tasks.cephfs.mount:self.hostfs_mntpt = /home/ubuntu/cephtest/mnt.0 2024-04-25T10:29:28.346 INFO:tasks.cephfs.mount:self.cephfs_name = None 2024-04-25T10:29:28.346 INFO:tasks.cephfs.mount:self.cephfs_mntpt = None 2024-04-25T10:29:28.346 INFO:tasks.cephfs.mount:self.client_keyring_path = None 2024-04-25T10:29:28.346 INFO:tasks.cephfs.mount:Setting the 'None' netns for '/home/ubuntu/cephtest/mnt.0' 2024-04-25T10:29:28.346 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T10:29:28.346 DEBUG:teuthology.orchestra.run.smithi037:> ip addr 2024-04-25T10:29:28.361 INFO:teuthology.orchestra.run.smithi037.stdout:1: lo: mtu 65536 qdisc noqueue state UNKNOWN group default qlen 1000 2024-04-25T10:29:28.361 INFO:teuthology.orchestra.run.smithi037.stdout: link/loopback 00:00:00:00:00:00 brd 00:00:00:00:00:00 2024-04-25T10:29:28.361 INFO:teuthology.orchestra.run.smithi037.stdout: inet 127.0.0.1/8 scope host lo 2024-04-25T10:29:28.361 INFO:teuthology.orchestra.run.smithi037.stdout: valid_lft forever preferred_lft forever 2024-04-25T10:29:28.362 INFO:teuthology.orchestra.run.smithi037.stdout: inet6 ::1/128 scope host 2024-04-25T10:29:28.362 INFO:teuthology.orchestra.run.smithi037.stdout: valid_lft forever preferred_lft forever 2024-04-25T10:29:28.362 INFO:teuthology.orchestra.run.smithi037.stdout:2: eno1: mtu 1500 qdisc mq state DOWN group default qlen 1000 2024-04-25T10:29:28.362 INFO:teuthology.orchestra.run.smithi037.stdout: link/ether 0c:c4:7a:6b:fd:18 brd ff:ff:ff:ff:ff:ff 2024-04-25T10:29:28.362 INFO:teuthology.orchestra.run.smithi037.stdout: altname enp7s0f0 2024-04-25T10:29:28.362 INFO:teuthology.orchestra.run.smithi037.stdout:3: eno2: mtu 1500 qdisc mq state DOWN group default qlen 1000 2024-04-25T10:29:28.362 INFO:teuthology.orchestra.run.smithi037.stdout: link/ether 0c:c4:7a:6b:fd:19 brd ff:ff:ff:ff:ff:ff 2024-04-25T10:29:28.362 INFO:teuthology.orchestra.run.smithi037.stdout: altname enp7s0f1 2024-04-25T10:29:28.362 INFO:teuthology.orchestra.run.smithi037.stdout:4: ens1f0: mtu 1500 qdisc mq state UP group default qlen 1000 2024-04-25T10:29:28.362 INFO:teuthology.orchestra.run.smithi037.stdout: link/ether 0c:c4:7a:88:77:94 brd ff:ff:ff:ff:ff:ff 2024-04-25T10:29:28.362 INFO:teuthology.orchestra.run.smithi037.stdout: altname enp1s0f0 2024-04-25T10:29:28.362 INFO:teuthology.orchestra.run.smithi037.stdout: inet 172.21.15.37/20 brd 172.21.15.255 scope global dynamic noprefixroute ens1f0 2024-04-25T10:29:28.362 INFO:teuthology.orchestra.run.smithi037.stdout: valid_lft 42700sec preferred_lft 42700sec 2024-04-25T10:29:28.363 INFO:teuthology.orchestra.run.smithi037.stdout: inet6 fe80::ec4:7aff:fe88:7794/64 scope link noprefixroute 2024-04-25T10:29:28.363 INFO:teuthology.orchestra.run.smithi037.stdout: valid_lft forever preferred_lft forever 2024-04-25T10:29:28.363 INFO:teuthology.orchestra.run.smithi037.stdout:5: ens1f1: mtu 1500 qdisc mq state DOWN group default qlen 1000 2024-04-25T10:29:28.363 INFO:teuthology.orchestra.run.smithi037.stdout: link/ether 0c:c4:7a:88:77:95 brd ff:ff:ff:ff:ff:ff 2024-04-25T10:29:28.363 INFO:teuthology.orchestra.run.smithi037.stdout: altname enp1s0f1 2024-04-25T10:29:28.363 INFO:tasks.cephfs.mount:Setuping the 'ceph-brx' with 192.168.255.254/16 2024-04-25T10:29:28.363 DEBUG:teuthology.orchestra.run.smithi037:> (cd / && exec stdin-killer --timeout=300 -- bash -c ' 2024-04-25T10:29:28.363 DEBUG:teuthology.orchestra.run.smithi037:> set -e 2024-04-25T10:29:28.363 DEBUG:teuthology.orchestra.run.smithi037:> sudo ip link add name ceph-brx type bridge 2024-04-25T10:29:28.364 DEBUG:teuthology.orchestra.run.smithi037:> sudo ip addr flush dev ceph-brx 2024-04-25T10:29:28.364 DEBUG:teuthology.orchestra.run.smithi037:> sudo ip link set ceph-brx up 2024-04-25T10:29:28.364 DEBUG:teuthology.orchestra.run.smithi037:> sudo ip addr add 192.168.255.254/16 brd 192.168.255.255 dev ceph-brx 2024-04-25T10:29:28.364 DEBUG:teuthology.orchestra.run.smithi037:> ') 2024-04-25T10:29:28.451 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:28 stdin-killer INFO: expiration expected; waiting 300 seconds for command to complete 2024-04-25T10:29:28.536 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:28 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-04-25T10:29:28.543 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T10:29:28.543 DEBUG:teuthology.orchestra.run.smithi037:> echo 1 | sudo tee /proc/sys/net/ipv4/ip_forward 2024-04-25T10:29:28.568 INFO:teuthology.orchestra.run.smithi037.stdout:1 2024-04-25T10:29:28.570 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T10:29:28.570 DEBUG:teuthology.orchestra.run.smithi037:> route 2024-04-25T10:29:28.632 INFO:teuthology.orchestra.run.smithi037.stdout:Kernel IP routing table 2024-04-25T10:29:28.632 INFO:teuthology.orchestra.run.smithi037.stdout:Destination Gateway Genmask Flags Metric Ref Use Iface 2024-04-25T10:29:28.632 INFO:teuthology.orchestra.run.smithi037.stdout:default _gateway 0.0.0.0 UG 100 0 0 ens1f0 2024-04-25T10:29:28.632 INFO:teuthology.orchestra.run.smithi037.stdout:172.21.0.0 0.0.0.0 255.255.240.0 U 100 0 0 ens1f0 2024-04-25T10:29:28.632 INFO:teuthology.orchestra.run.smithi037.stdout:192.168.0.0 0.0.0.0 255.255.0.0 U 0 0 0 ceph-brx 2024-04-25T10:29:28.632 DEBUG:teuthology.orchestra.run.smithi037:> (cd / && exec stdin-killer --timeout=300 -- bash -c ' 2024-04-25T10:29:28.633 DEBUG:teuthology.orchestra.run.smithi037:> set -e 2024-04-25T10:29:28.633 DEBUG:teuthology.orchestra.run.smithi037:> sudo iptables -A FORWARD -o ens1f0 -i ceph-brx -j ACCEPT 2024-04-25T10:29:28.633 DEBUG:teuthology.orchestra.run.smithi037:> sudo iptables -A FORWARD -i ens1f0 -o ceph-brx -j ACCEPT 2024-04-25T10:29:28.633 DEBUG:teuthology.orchestra.run.smithi037:> sudo iptables -t nat -A POSTROUTING -s 192.168.255.254/16 -o ens1f0 -j MASQUERADE 2024-04-25T10:29:28.633 DEBUG:teuthology.orchestra.run.smithi037:> ') 2024-04-25T10:29:28.722 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:28 stdin-killer INFO: expiration expected; waiting 300 seconds for command to complete 2024-04-25T10:29:28.893 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:28 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-04-25T10:29:28.902 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T10:29:28.902 DEBUG:teuthology.orchestra.run.smithi037:> ip netns list 2024-04-25T10:29:28.918 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T10:29:28.918 DEBUG:teuthology.orchestra.run.smithi037:> ip netns list-id 2024-04-25T10:29:28.973 DEBUG:teuthology.orchestra.run.smithi037:> (cd / && exec stdin-killer --timeout=300 -- bash -c ' 2024-04-25T10:29:28.974 DEBUG:teuthology.orchestra.run.smithi037:> set -e 2024-04-25T10:29:28.974 DEBUG:teuthology.orchestra.run.smithi037:> sudo ip netns add ceph-ns--home-ubuntu-cephtest-mnt.0 2024-04-25T10:29:28.974 DEBUG:teuthology.orchestra.run.smithi037:> sudo ip netns set ceph-ns--home-ubuntu-cephtest-mnt.0 0 2024-04-25T10:29:28.974 DEBUG:teuthology.orchestra.run.smithi037:> ') 2024-04-25T10:29:29.065 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:29 stdin-killer INFO: expiration expected; waiting 300 seconds for command to complete 2024-04-25T10:29:29.088 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:29 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-04-25T10:29:29.095 INFO:tasks.cephfs.mount:Setuping the netns 'ceph-ns--home-ubuntu-cephtest-mnt.0' with 192.168.0.1/16 2024-04-25T10:29:29.095 DEBUG:teuthology.orchestra.run.smithi037:> (cd / && exec stdin-killer --timeout=300 -- bash -c ' 2024-04-25T10:29:29.095 DEBUG:teuthology.orchestra.run.smithi037:> set -e 2024-04-25T10:29:29.095 DEBUG:teuthology.orchestra.run.smithi037:> sudo ip link add veth0 netns ceph-ns--home-ubuntu-cephtest-mnt.0 type veth peer name brx.0 2024-04-25T10:29:29.095 DEBUG:teuthology.orchestra.run.smithi037:> sudo ip netns exec ceph-ns--home-ubuntu-cephtest-mnt.0 ip addr add 192.168.0.1/16 brd 192.168.255.255 dev veth0 2024-04-25T10:29:29.095 DEBUG:teuthology.orchestra.run.smithi037:> sudo ip netns exec ceph-ns--home-ubuntu-cephtest-mnt.0 ip link set veth0 up 2024-04-25T10:29:29.095 DEBUG:teuthology.orchestra.run.smithi037:> sudo ip netns exec ceph-ns--home-ubuntu-cephtest-mnt.0 ip link set lo up 2024-04-25T10:29:29.095 DEBUG:teuthology.orchestra.run.smithi037:> sudo ip netns exec ceph-ns--home-ubuntu-cephtest-mnt.0 ip route add default via 192.168.255.254 2024-04-25T10:29:29.096 DEBUG:teuthology.orchestra.run.smithi037:> ') 2024-04-25T10:29:29.145 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:29 stdin-killer INFO: expiration expected; waiting 300 seconds for command to complete 2024-04-25T10:29:29.217 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:29 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-04-25T10:29:29.224 DEBUG:teuthology.orchestra.run.smithi037:> (cd / && exec stdin-killer --timeout=300 -- bash -c ' 2024-04-25T10:29:29.224 DEBUG:teuthology.orchestra.run.smithi037:> set -e 2024-04-25T10:29:29.224 DEBUG:teuthology.orchestra.run.smithi037:> sudo ip link set brx.0 up 2024-04-25T10:29:29.224 DEBUG:teuthology.orchestra.run.smithi037:> sudo ip link set dev brx.0 master ceph-brx 2024-04-25T10:29:29.224 DEBUG:teuthology.orchestra.run.smithi037:> ') 2024-04-25T10:29:29.273 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:29 stdin-killer INFO: expiration expected; waiting 300 seconds for command to complete 2024-04-25T10:29:29.295 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:29 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-04-25T10:29:29.302 INFO:tasks.cephfs.fuse_mount:Client client.0 config is {} 2024-04-25T10:29:29.303 INFO:teuthology.orchestra.run:Running command with timeout 60 2024-04-25T10:29:29.303 DEBUG:teuthology.orchestra.run.smithi037:> mkdir -p -v /home/ubuntu/cephtest/mnt.0 2024-04-25T10:29:29.318 INFO:teuthology.orchestra.run.smithi037.stdout:mkdir: created directory '/home/ubuntu/cephtest/mnt.0' 2024-04-25T10:29:29.318 INFO:teuthology.orchestra.run:Running command with timeout 60 2024-04-25T10:29:29.318 DEBUG:teuthology.orchestra.run.smithi037:> chmod 0000 /home/ubuntu/cephtest/mnt.0 2024-04-25T10:29:29.373 DEBUG:teuthology.orchestra.run.smithi037:> sudo modprobe fuse 2024-04-25T10:29:29.438 DEBUG:teuthology.orchestra.run.smithi037:> cat /proc/self/mounts | awk '{print $2}' 2024-04-25T10:29:29.494 INFO:teuthology.orchestra.run.smithi037.stdout:/proc 2024-04-25T10:29:29.494 INFO:teuthology.orchestra.run.smithi037.stdout:/sys 2024-04-25T10:29:29.494 INFO:teuthology.orchestra.run.smithi037.stdout:/dev 2024-04-25T10:29:29.494 INFO:teuthology.orchestra.run.smithi037.stdout:/sys/kernel/security 2024-04-25T10:29:29.494 INFO:teuthology.orchestra.run.smithi037.stdout:/dev/shm 2024-04-25T10:29:29.494 INFO:teuthology.orchestra.run.smithi037.stdout:/dev/pts 2024-04-25T10:29:29.494 INFO:teuthology.orchestra.run.smithi037.stdout:/run 2024-04-25T10:29:29.495 INFO:teuthology.orchestra.run.smithi037.stdout:/sys/fs/cgroup 2024-04-25T10:29:29.495 INFO:teuthology.orchestra.run.smithi037.stdout:/sys/fs/pstore 2024-04-25T10:29:29.495 INFO:teuthology.orchestra.run.smithi037.stdout:/sys/fs/bpf 2024-04-25T10:29:29.495 INFO:teuthology.orchestra.run.smithi037.stdout:/ 2024-04-25T10:29:29.495 INFO:teuthology.orchestra.run.smithi037.stdout:/sys/fs/selinux 2024-04-25T10:29:29.495 INFO:teuthology.orchestra.run.smithi037.stdout:/proc/sys/fs/binfmt_misc 2024-04-25T10:29:29.495 INFO:teuthology.orchestra.run.smithi037.stdout:/dev/hugepages 2024-04-25T10:29:29.495 INFO:teuthology.orchestra.run.smithi037.stdout:/dev/mqueue 2024-04-25T10:29:29.495 INFO:teuthology.orchestra.run.smithi037.stdout:/sys/kernel/debug 2024-04-25T10:29:29.495 INFO:teuthology.orchestra.run.smithi037.stdout:/sys/kernel/tracing 2024-04-25T10:29:29.495 INFO:teuthology.orchestra.run.smithi037.stdout:/sys/fs/fuse/connections 2024-04-25T10:29:29.495 INFO:teuthology.orchestra.run.smithi037.stdout:/sys/kernel/config 2024-04-25T10:29:29.495 INFO:teuthology.orchestra.run.smithi037.stdout:/run/credentials/systemd-sysusers.service 2024-04-25T10:29:29.496 INFO:teuthology.orchestra.run.smithi037.stdout:/run/credentials/systemd-sysctl.service 2024-04-25T10:29:29.496 INFO:teuthology.orchestra.run.smithi037.stdout:/run/credentials/systemd-tmpfiles-setup-dev.service 2024-04-25T10:29:29.496 INFO:teuthology.orchestra.run.smithi037.stdout:/run/credentials/systemd-tmpfiles-setup.service 2024-04-25T10:29:29.496 INFO:teuthology.orchestra.run.smithi037.stdout:/var/lib/nfs/rpc_pipefs 2024-04-25T10:29:29.496 INFO:teuthology.orchestra.run.smithi037.stdout:/run/user/1000 2024-04-25T10:29:29.496 INFO:teuthology.orchestra.run.smithi037.stdout:/var/lib/ceph 2024-04-25T10:29:29.497 INFO:teuthology.orchestra.run.smithi037.stdout:/var/lib/ceph/osd/ceph-0 2024-04-25T10:29:29.497 INFO:teuthology.orchestra.run.smithi037.stdout:/var/lib/ceph/osd/ceph-1 2024-04-25T10:29:29.497 INFO:teuthology.orchestra.run.smithi037.stdout:/var/lib/ceph/osd/ceph-2 2024-04-25T10:29:29.497 INFO:teuthology.orchestra.run.smithi037.stdout:/var/lib/ceph/osd/ceph-3 2024-04-25T10:29:29.497 INFO:teuthology.orchestra.run.smithi037.stdout:/run/netns 2024-04-25T10:29:29.497 INFO:teuthology.orchestra.run.smithi037.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 2024-04-25T10:29:29.497 INFO:teuthology.orchestra.run.smithi037.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 2024-04-25T10:29:29.497 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T10:29:29.498 DEBUG:teuthology.orchestra.run.smithi037:> ls /sys/fs/fuse/connections 2024-04-25T10:29:29.550 INFO:tasks.cephfs.fuse_mount:Pre-mount connections: [] 2024-04-25T10:29:29.550 DEBUG:teuthology.orchestra.run.smithi037:> (cd /home/ubuntu/cephtest && exec sudo nsenter --net=/var/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-fuse -f --admin-socket '/var/run/ceph/$cluster-$name.$pid.asok' /home/ubuntu/cephtest/mnt.0 --id 0) 2024-04-25T10:29:29.592 DEBUG:teuthology.orchestra.run.smithi037:> sudo modprobe fuse 2024-04-25T10:29:29.616 DEBUG:teuthology.orchestra.run.smithi037:> cat /proc/self/mounts | awk '{print $2}' 2024-04-25T10:29:29.673 INFO:teuthology.orchestra.run.smithi037.stdout:/proc 2024-04-25T10:29:29.673 INFO:teuthology.orchestra.run.smithi037.stdout:/sys 2024-04-25T10:29:29.674 INFO:teuthology.orchestra.run.smithi037.stdout:/dev 2024-04-25T10:29:29.674 INFO:teuthology.orchestra.run.smithi037.stdout:/sys/kernel/security 2024-04-25T10:29:29.674 INFO:teuthology.orchestra.run.smithi037.stdout:/dev/shm 2024-04-25T10:29:29.674 INFO:teuthology.orchestra.run.smithi037.stdout:/dev/pts 2024-04-25T10:29:29.674 INFO:teuthology.orchestra.run.smithi037.stdout:/run 2024-04-25T10:29:29.674 INFO:teuthology.orchestra.run.smithi037.stdout:/sys/fs/cgroup 2024-04-25T10:29:29.674 INFO:teuthology.orchestra.run.smithi037.stdout:/sys/fs/pstore 2024-04-25T10:29:29.674 INFO:teuthology.orchestra.run.smithi037.stdout:/sys/fs/bpf 2024-04-25T10:29:29.674 INFO:teuthology.orchestra.run.smithi037.stdout:/ 2024-04-25T10:29:29.674 INFO:teuthology.orchestra.run.smithi037.stdout:/sys/fs/selinux 2024-04-25T10:29:29.674 INFO:teuthology.orchestra.run.smithi037.stdout:/proc/sys/fs/binfmt_misc 2024-04-25T10:29:29.674 INFO:teuthology.orchestra.run.smithi037.stdout:/dev/hugepages 2024-04-25T10:29:29.674 INFO:teuthology.orchestra.run.smithi037.stdout:/dev/mqueue 2024-04-25T10:29:29.675 INFO:teuthology.orchestra.run.smithi037.stdout:/sys/kernel/debug 2024-04-25T10:29:29.675 INFO:teuthology.orchestra.run.smithi037.stdout:/sys/kernel/tracing 2024-04-25T10:29:29.675 INFO:teuthology.orchestra.run.smithi037.stdout:/sys/fs/fuse/connections 2024-04-25T10:29:29.675 INFO:teuthology.orchestra.run.smithi037.stdout:/sys/kernel/config 2024-04-25T10:29:29.675 INFO:teuthology.orchestra.run.smithi037.stdout:/run/credentials/systemd-sysusers.service 2024-04-25T10:29:29.675 INFO:teuthology.orchestra.run.smithi037.stdout:/run/credentials/systemd-sysctl.service 2024-04-25T10:29:29.675 INFO:teuthology.orchestra.run.smithi037.stdout:/run/credentials/systemd-tmpfiles-setup-dev.service 2024-04-25T10:29:29.675 INFO:teuthology.orchestra.run.smithi037.stdout:/run/credentials/systemd-tmpfiles-setup.service 2024-04-25T10:29:29.675 INFO:teuthology.orchestra.run.smithi037.stdout:/var/lib/nfs/rpc_pipefs 2024-04-25T10:29:29.675 INFO:teuthology.orchestra.run.smithi037.stdout:/run/user/1000 2024-04-25T10:29:29.675 INFO:teuthology.orchestra.run.smithi037.stdout:/var/lib/ceph 2024-04-25T10:29:29.675 INFO:teuthology.orchestra.run.smithi037.stdout:/var/lib/ceph/osd/ceph-0 2024-04-25T10:29:29.675 INFO:teuthology.orchestra.run.smithi037.stdout:/var/lib/ceph/osd/ceph-1 2024-04-25T10:29:29.675 INFO:teuthology.orchestra.run.smithi037.stdout:/var/lib/ceph/osd/ceph-2 2024-04-25T10:29:29.676 INFO:teuthology.orchestra.run.smithi037.stdout:/var/lib/ceph/osd/ceph-3 2024-04-25T10:29:29.676 INFO:teuthology.orchestra.run.smithi037.stdout:/run/netns 2024-04-25T10:29:29.676 INFO:teuthology.orchestra.run.smithi037.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 2024-04-25T10:29:29.676 INFO:teuthology.orchestra.run.smithi037.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 2024-04-25T10:29:29.677 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T10:29:29.677 DEBUG:teuthology.orchestra.run.smithi037:> ls /sys/fs/fuse/connections 2024-04-25T10:29:29.689 INFO:tasks.cephfs.fuse_mount.ceph-fuse.0.smithi037.stderr:2024-04-25T10:29:29.688+0000 7f557073f480 -1 init, newargv = 0x7f55600057c0 newargc=15 2024-04-25T10:29:29.690 INFO:tasks.cephfs.fuse_mount.ceph-fuse.0.smithi037.stderr:ceph-fuse[43223]: starting ceph client 2024-04-25T10:29:29.695 INFO:tasks.cephfs.fuse_mount.ceph-fuse.0.smithi037.stderr:ceph-fuse[43223]: starting fuse 2024-04-25T10:29:29.706 INFO:teuthology.orchestra.run.smithi037.stdout:43 2024-04-25T10:29:29.706 INFO:tasks.cephfs.fuse_mount:Post-mount connections: [43] 2024-04-25T10:29:29.706 DEBUG:teuthology.orchestra.run.smithi037:> sudo stdin-killer -- python3 -c ' 2024-04-25T10:29:29.706 DEBUG:teuthology.orchestra.run.smithi037:> import glob 2024-04-25T10:29:29.706 DEBUG:teuthology.orchestra.run.smithi037:> import re 2024-04-25T10:29:29.707 DEBUG:teuthology.orchestra.run.smithi037:> import os 2024-04-25T10:29:29.707 DEBUG:teuthology.orchestra.run.smithi037:> import subprocess 2024-04-25T10:29:29.707 DEBUG:teuthology.orchestra.run.smithi037:> 2024-04-25T10:29:29.707 DEBUG:teuthology.orchestra.run.smithi037:> def _find_admin_socket(client_name): 2024-04-25T10:29:29.707 DEBUG:teuthology.orchestra.run.smithi037:> asok_path = "/var/run/ceph/ceph-client.0.*.asok" 2024-04-25T10:29:29.707 DEBUG:teuthology.orchestra.run.smithi037:> files = glob.glob(asok_path) 2024-04-25T10:29:29.707 DEBUG:teuthology.orchestra.run.smithi037:> mountpoint = "/home/ubuntu/cephtest/mnt.0" 2024-04-25T10:29:29.707 DEBUG:teuthology.orchestra.run.smithi037:> 2024-04-25T10:29:29.707 DEBUG:teuthology.orchestra.run.smithi037:> # Given a non-glob path, it better be there 2024-04-25T10:29:29.707 DEBUG:teuthology.orchestra.run.smithi037:> if "*" not in asok_path: 2024-04-25T10:29:29.707 DEBUG:teuthology.orchestra.run.smithi037:> assert(len(files) == 1) 2024-04-25T10:29:29.707 DEBUG:teuthology.orchestra.run.smithi037:> return files[0] 2024-04-25T10:29:29.707 DEBUG:teuthology.orchestra.run.smithi037:> 2024-04-25T10:29:29.707 DEBUG:teuthology.orchestra.run.smithi037:> for f in files: 2024-04-25T10:29:29.707 DEBUG:teuthology.orchestra.run.smithi037:> pid = re.match(".*\.(\d+)\.asok$", f).group(1) 2024-04-25T10:29:29.707 DEBUG:teuthology.orchestra.run.smithi037:> if os.path.exists("/proc/{0}".format(pid)): 2024-04-25T10:29:29.708 DEBUG:teuthology.orchestra.run.smithi037:> with open("/proc/{0}/cmdline".format(pid), '"'"'r'"'"') as proc_f: 2024-04-25T10:29:29.708 DEBUG:teuthology.orchestra.run.smithi037:> contents = proc_f.read() 2024-04-25T10:29:29.708 DEBUG:teuthology.orchestra.run.smithi037:> if mountpoint in contents: 2024-04-25T10:29:29.708 DEBUG:teuthology.orchestra.run.smithi037:> return f 2024-04-25T10:29:29.708 DEBUG:teuthology.orchestra.run.smithi037:> raise RuntimeError("Client socket {0} not found".format(client_name)) 2024-04-25T10:29:29.708 DEBUG:teuthology.orchestra.run.smithi037:> 2024-04-25T10:29:29.708 DEBUG:teuthology.orchestra.run.smithi037:> print(_find_admin_socket("client.0")) 2024-04-25T10:29:29.708 DEBUG:teuthology.orchestra.run.smithi037:> ' 2024-04-25T10:29:29.825 INFO:teuthology.orchestra.run.smithi037.stdout:/var/run/ceph/ceph-client.0.43223.asok 2024-04-25T10:29:29.828 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:29 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-04-25T10:29:29.836 INFO:tasks.cephfs.fuse_mount:Found client admin socket at /var/run/ceph/ceph-client.0.43223.asok 2024-04-25T10:29:29.836 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T10:29:29.836 DEBUG:teuthology.orchestra.run.smithi037:> sudo ceph --admin-daemon /var/run/ceph/ceph-client.0.43223.asok status 2024-04-25T10:29:29.982 INFO:teuthology.orchestra.run.smithi037.stdout:{ 2024-04-25T10:29:29.982 INFO:teuthology.orchestra.run.smithi037.stdout: "metadata": { 2024-04-25T10:29:29.982 INFO:teuthology.orchestra.run.smithi037.stdout: "ceph_sha1": "994e2f20102a20fa9a27966c14b625cc01884df8", 2024-04-25T10:29:29.982 INFO:teuthology.orchestra.run.smithi037.stdout: "ceph_version": "ceph version 19.0.0-3291-g994e2f20 (994e2f20102a20fa9a27966c14b625cc01884df8) squid (dev)", 2024-04-25T10:29:29.983 INFO:teuthology.orchestra.run.smithi037.stdout: "entity_id": "0", 2024-04-25T10:29:29.983 INFO:teuthology.orchestra.run.smithi037.stdout: "hostname": "smithi037", 2024-04-25T10:29:29.983 INFO:teuthology.orchestra.run.smithi037.stdout: "mount_point": "/home/ubuntu/cephtest/mnt.0", 2024-04-25T10:29:29.983 INFO:teuthology.orchestra.run.smithi037.stdout: "pid": "43223", 2024-04-25T10:29:29.983 INFO:teuthology.orchestra.run.smithi037.stdout: "root": "/" 2024-04-25T10:29:29.983 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-04-25T10:29:29.983 INFO:teuthology.orchestra.run.smithi037.stdout: "dentry_count": 0, 2024-04-25T10:29:29.983 INFO:teuthology.orchestra.run.smithi037.stdout: "dentry_pinned_count": 0, 2024-04-25T10:29:29.983 INFO:teuthology.orchestra.run.smithi037.stdout: "id": 4731, 2024-04-25T10:29:29.983 INFO:teuthology.orchestra.run.smithi037.stdout: "inst": { 2024-04-25T10:29:29.983 INFO:teuthology.orchestra.run.smithi037.stdout: "name": { 2024-04-25T10:29:29.983 INFO:teuthology.orchestra.run.smithi037.stdout: "type": "client", 2024-04-25T10:29:29.983 INFO:teuthology.orchestra.run.smithi037.stdout: "num": 4731 2024-04-25T10:29:29.984 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-04-25T10:29:29.984 INFO:teuthology.orchestra.run.smithi037.stdout: "addr": { 2024-04-25T10:29:29.984 INFO:teuthology.orchestra.run.smithi037.stdout: "type": "v1", 2024-04-25T10:29:29.984 INFO:teuthology.orchestra.run.smithi037.stdout: "addr": "192.168.0.1:0", 2024-04-25T10:29:29.984 INFO:teuthology.orchestra.run.smithi037.stdout: "nonce": 3486912814 2024-04-25T10:29:29.984 INFO:teuthology.orchestra.run.smithi037.stdout: } 2024-04-25T10:29:29.984 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-04-25T10:29:29.984 INFO:teuthology.orchestra.run.smithi037.stdout: "addr": { 2024-04-25T10:29:29.984 INFO:teuthology.orchestra.run.smithi037.stdout: "type": "v1", 2024-04-25T10:29:29.984 INFO:teuthology.orchestra.run.smithi037.stdout: "addr": "192.168.0.1:0", 2024-04-25T10:29:29.984 INFO:teuthology.orchestra.run.smithi037.stdout: "nonce": 3486912814 2024-04-25T10:29:29.984 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-04-25T10:29:29.984 INFO:teuthology.orchestra.run.smithi037.stdout: "inst_str": "client.4731 192.168.0.1:0/3486912814", 2024-04-25T10:29:29.984 INFO:teuthology.orchestra.run.smithi037.stdout: "addr_str": "192.168.0.1:0/3486912814", 2024-04-25T10:29:29.985 INFO:teuthology.orchestra.run.smithi037.stdout: "inode_count": 1, 2024-04-25T10:29:29.985 INFO:teuthology.orchestra.run.smithi037.stdout: "mds_epoch": 8, 2024-04-25T10:29:29.985 INFO:teuthology.orchestra.run.smithi037.stdout: "osd_epoch": 23, 2024-04-25T10:29:29.985 INFO:teuthology.orchestra.run.smithi037.stdout: "osd_epoch_barrier": 0, 2024-04-25T10:29:29.985 INFO:teuthology.orchestra.run.smithi037.stdout: "blocklisted": false, 2024-04-25T10:29:29.985 INFO:teuthology.orchestra.run.smithi037.stdout: "fs_name": "cephfs" 2024-04-25T10:29:29.985 INFO:teuthology.orchestra.run.smithi037.stdout:} 2024-04-25T10:29:29.991 DEBUG:teuthology.orchestra.run.smithi037:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs ls 2024-04-25T10:29:30.102 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:30.100+0000 7f400218f640 1 Processor -- start 2024-04-25T10:29:30.102 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:30.101+0000 7f400218f640 1 -- start start 2024-04-25T10:29:30.103 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:30.102+0000 7f400218f640 1 --2- >> v2:172.21.15.37:3300/0 conn(0x7f3ffc0d1ec0 0x7f3ffc0d22d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:29:30.103 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:30.102+0000 7f400218f640 1 -- --> v1:172.21.15.37:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f3ffc09ce70 con 0x7f3ffc06e260 2024-04-25T10:29:30.103 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:30.102+0000 7f400218f640 1 -- --> v1:172.21.15.203:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f3ffc09d150 con 0x7f3ffc0d2810 2024-04-25T10:29:30.103 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:30.102+0000 7f400218f640 1 -- --> v2:172.21.15.37:3300/0 -- mon_getmap magic: 0 -- 0x7f3ffc0d3370 con 0x7f3ffc0d1ec0 2024-04-25T10:29:30.104 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:30.102+0000 7f3ffb7fe640 1 --1- >> v1:172.21.15.37:6789/0 conn(0x7f3ffc06e260 0x7f3ffc06e670 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.37:6789/0 says I am v1:172.21.15.37:46502/0 (socket says 172.21.15.37:46502) 2024-04-25T10:29:30.104 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:30.102+0000 7f3ffb7fe640 1 -- 172.21.15.37:0/3932603500 learned_addr learned my addr 172.21.15.37:0/3932603500 (peer_addr_for_me v1:172.21.15.37:0/0) 2024-04-25T10:29:30.104 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:30.102+0000 7f3ffaffd640 1 --2- 172.21.15.37:0/3932603500 >> v2:172.21.15.37:3300/0 conn(0x7f3ffc0d1ec0 0x7f3ffc0d22d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:29:30.104 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:30.102+0000 7f3ffaffd640 1 -- 172.21.15.37:0/3932603500 >> v1:172.21.15.203:6789/0 conn(0x7f3ffc0d2810 legacy=0x7f3ffc0b3c80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:29:30.104 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:30.102+0000 7f3ffaffd640 1 -- 172.21.15.37:0/3932603500 >> v1:172.21.15.37:6789/0 conn(0x7f3ffc06e260 legacy=0x7f3ffc06e670 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:29:30.104 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:30.102+0000 7f3ffaffd640 1 -- 172.21.15.37:0/3932603500 --> v2:172.21.15.37:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f3ffc06ed80 con 0x7f3ffc0d1ec0 2024-04-25T10:29:30.104 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:30.102+0000 7f3ffa7fc640 1 -- 172.21.15.37:0/3932603500 <== mon.0 v1:172.21.15.37:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 3118202714 0 0) 0x7f3ffc09ce70 con 0x7f3ffc06e260 2024-04-25T10:29:30.105 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:30.103+0000 7f3ffaffd640 1 --2- 172.21.15.37:0/3932603500 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f3ffc0d1ec0 0x7f3ffc0d22d0 secure :-1 s=READY pgs=184 cs=0 l=1 rev1=1 crypto rx=0x7f3ff00097a0 tx=0x7f3ff002a1a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ca9a2d46f5bfcbd5 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:29:30.105 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:30.103+0000 7f3ffa7fc640 1 -- 172.21.15.37:0/3932603500 <== mon.0 v2:172.21.15.37:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3ff0037070 con 0x7f3ffc0d1ec0 2024-04-25T10:29:30.105 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:30.103+0000 7f3ffa7fc640 1 -- 172.21.15.37:0/3932603500 <== mon.0 v2:172.21.15.37:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f3ff0032440 con 0x7f3ffc0d1ec0 2024-04-25T10:29:30.105 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:30.103+0000 7f3ffa7fc640 1 -- 172.21.15.37:0/3932603500 <== mon.0 v2:172.21.15.37:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3ff0032810 con 0x7f3ffc0d1ec0 2024-04-25T10:29:30.106 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:30.104+0000 7f400218f640 1 -- 172.21.15.37:0/3932603500 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f3ffc0d1ec0 msgr2=0x7f3ffc0d22d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:29:30.106 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:30.104+0000 7f400218f640 1 --2- 172.21.15.37:0/3932603500 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f3ffc0d1ec0 0x7f3ffc0d22d0 secure :-1 s=READY pgs=184 cs=0 l=1 rev1=1 crypto rx=0x7f3ff00097a0 tx=0x7f3ff002a1a0 comp rx=0 tx=0).stop 2024-04-25T10:29:30.106 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:30.104+0000 7f400218f640 1 -- 172.21.15.37:0/3932603500 shutdown_connections 2024-04-25T10:29:30.106 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:30.104+0000 7f400218f640 1 --2- 172.21.15.37:0/3932603500 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f3ffc0d1ec0 0x7f3ffc0d22d0 unknown :-1 s=CLOSED pgs=184 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:29:30.106 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:30.104+0000 7f400218f640 1 -- 172.21.15.37:0/3932603500 >> 172.21.15.37:0/3932603500 conn(0x7f3ffc0d0390 msgr2=0x7f3ffc0d07a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:29:30.106 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:30.104+0000 7f400218f640 1 -- 172.21.15.37:0/3932603500 shutdown_connections 2024-04-25T10:29:30.106 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:30.104+0000 7f400218f640 1 -- 172.21.15.37:0/3932603500 wait complete. 2024-04-25T10:29:30.107 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:30.106+0000 7f400218f640 1 Processor -- start 2024-04-25T10:29:30.107 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:30.106+0000 7f400218f640 1 -- start start 2024-04-25T10:29:30.110 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:30.107+0000 7f400218f640 1 --2- >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f3ffc06e260 0x7f3ffc1de960 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:29:30.110 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:30.107+0000 7f400218f640 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f3ffc0d1ec0 0x7f3ffc0c7c90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:29:30.110 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:30.107+0000 7f400218f640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f3ffc0d2810 0x7f3ffc0b44e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:29:30.110 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:30.107+0000 7f400218f640 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 -- 0x7f3ffc06d830 con 0x7f3ffc0d2810 2024-04-25T10:29:30.110 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:30.107+0000 7f400218f640 1 -- --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_getmap magic: 0 -- 0x7f3ffc0d3090 con 0x7f3ffc0d1ec0 2024-04-25T10:29:30.110 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:30.107+0000 7f400218f640 1 -- --> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] -- mon_getmap magic: 0 -- 0x7f3ffc0d2c20 con 0x7f3ffc06e260 2024-04-25T10:29:30.110 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:30.107+0000 7f3ffbfff640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f3ffc0d2810 0x7f3ffc0b44e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:29:30.110 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:30.108+0000 7f3ffbfff640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f3ffc0d2810 0x7f3ffc0b44e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.37:3300/0 says I am v2:172.21.15.37:36738/0 (socket says 172.21.15.37:36738) 2024-04-25T10:29:30.110 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:30.108+0000 7f3ffbfff640 1 -- 172.21.15.37:0/3565669101 learned_addr learned my addr 172.21.15.37:0/3565669101 (peer_addr_for_me v2:172.21.15.37:0/0) 2024-04-25T10:29:30.111 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:30.108+0000 7f3ffaffd640 1 --2- 172.21.15.37:0/3565669101 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f3ffc0d1ec0 0x7f3ffc0c7c90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:29:30.111 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:30.108+0000 7f3ffb7fe640 1 --2- 172.21.15.37:0/3565669101 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f3ffc06e260 0x7f3ffc1de960 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:29:30.111 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:30.108+0000 7f3ffbfff640 1 -- 172.21.15.37:0/3565669101 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f3ffc06e260 msgr2=0x7f3ffc1de960 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:29:30.111 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:30.108+0000 7f3ffbfff640 1 --2- 172.21.15.37:0/3565669101 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f3ffc06e260 0x7f3ffc1de960 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:29:30.111 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:30.108+0000 7f3ffbfff640 1 -- 172.21.15.37:0/3565669101 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f3ffc0d1ec0 msgr2=0x7f3ffc0c7c90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:29:30.111 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:30.108+0000 7f3ffbfff640 1 --2- 172.21.15.37:0/3565669101 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f3ffc0d1ec0 0x7f3ffc0c7c90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:29:30.111 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:30.108+0000 7f3ffbfff640 1 -- 172.21.15.37:0/3565669101 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f3ffc0a0390 con 0x7f3ffc0d2810 2024-04-25T10:29:30.111 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:30.108+0000 7f3ffbfff640 1 --2- 172.21.15.37:0/3565669101 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f3ffc0d2810 0x7f3ffc0b44e0 secure :-1 s=READY pgs=185 cs=0 l=1 rev1=1 crypto rx=0x7f3ff400c680 tx=0x7f3ff400cb40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:29:30.111 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:30.108+0000 7f3fe3fff640 1 -- 172.21.15.37:0/3565669101 <== mon.0 v2:172.21.15.37:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3ff4003b70 con 0x7f3ffc0d2810 2024-04-25T10:29:30.111 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:30.108+0000 7f3fe3fff640 1 -- 172.21.15.37:0/3565669101 <== mon.0 v2:172.21.15.37:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f3ff4003d10 con 0x7f3ffc0d2810 2024-04-25T10:29:30.111 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:30.108+0000 7f3fe3fff640 1 -- 172.21.15.37:0/3565669101 <== mon.0 v2:172.21.15.37:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3ff4005310 con 0x7f3ffc0d2810 2024-04-25T10:29:30.111 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:30.108+0000 7f400218f640 1 -- 172.21.15.37:0/3565669101 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f3ffc0a0670 con 0x7f3ffc0d2810 2024-04-25T10:29:30.111 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:30.108+0000 7f400218f640 1 -- 172.21.15.37:0/3565669101 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f3ffc0a0c30 con 0x7f3ffc0d2810 2024-04-25T10:29:30.112 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:30.109+0000 7f400218f640 1 -- 172.21.15.37:0/3565669101 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f3ffc06a140 con 0x7f3ffc0d2810 2024-04-25T10:29:30.112 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:30.110+0000 7f3fe3fff640 1 -- 172.21.15.37:0/3565669101 <== mon.0 v2:172.21.15.37:3300/0 4 ==== mgrmap(e 5) ==== 96782+0+0 (secure 0 0 0) 0x7f3ff40054b0 con 0x7f3ffc0d2810 2024-04-25T10:29:30.113 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:30.110+0000 7f3fe3fff640 1 --2- 172.21.15.37:0/3565669101 >> [v2:172.21.15.203:6832/2418205237,v1:172.21.15.203:6833/2418205237] conn(0x7f3fe80744b0 0x7f3fe80769c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:29:30.113 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:30.110+0000 7f3fe3fff640 1 -- 172.21.15.37:0/3565669101 <== mon.0 v2:172.21.15.37:3300/0 5 ==== osd_map(23..23 src has 1..23) ==== 5709+0+0 (secure 0 0 0) 0x7f3ff4095070 con 0x7f3ffc0d2810 2024-04-25T10:29:30.113 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:30.111+0000 7f3ffb7fe640 1 --2- 172.21.15.37:0/3565669101 >> [v2:172.21.15.203:6832/2418205237,v1:172.21.15.203:6833/2418205237] conn(0x7f3fe80744b0 0x7f3fe80769c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:29:30.113 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:30.111+0000 7f3ffb7fe640 1 --2- 172.21.15.37:0/3565669101 >> [v2:172.21.15.203:6832/2418205237,v1:172.21.15.203:6833/2418205237] conn(0x7f3fe80744b0 0x7f3fe80769c0 secure :-1 s=READY pgs=114 cs=0 l=1 rev1=1 crypto rx=0x7f3fec000f60 tx=0x7f3fec007d00 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:29:30.116 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:30.115+0000 7f3fe3fff640 1 -- 172.21.15.37:0/3565669101 <== mon.0 v2:172.21.15.37:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187246 (secure 0 0 0) 0x7f3ff405f890 con 0x7f3ffc0d2810 2024-04-25T10:29:30.339 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:30.337+0000 7f400218f640 1 -- 172.21.15.37:0/3565669101 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_command({"prefix": "fs ls"} v 0) -- 0x7f3ffc09cae0 con 0x7f3ffc0d2810 2024-04-25T10:29:30.339 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:30.338+0000 7f3fe3fff640 1 -- 172.21.15.37:0/3565669101 <== mon.0 v2:172.21.15.37:3300/0 7 ==== mon_command_ack([{"prefix": "fs ls"}]=0 v8) ==== 53+0+73 (secure 0 0 0) 0x7f3ff405f190 con 0x7f3ffc0d2810 2024-04-25T10:29:30.339 INFO:teuthology.orchestra.run.smithi037.stdout:name: cephfs, metadata pool: cephfs_metadata, data pools: [cephfs_data ] 2024-04-25T10:29:30.341 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:30.340+0000 7f400218f640 1 -- 172.21.15.37:0/3565669101 >> [v2:172.21.15.203:6832/2418205237,v1:172.21.15.203:6833/2418205237] conn(0x7f3fe80744b0 msgr2=0x7f3fe80769c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:29:30.341 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:30.340+0000 7f400218f640 1 --2- 172.21.15.37:0/3565669101 >> [v2:172.21.15.203:6832/2418205237,v1:172.21.15.203:6833/2418205237] conn(0x7f3fe80744b0 0x7f3fe80769c0 secure :-1 s=READY pgs=114 cs=0 l=1 rev1=1 crypto rx=0x7f3fec000f60 tx=0x7f3fec007d00 comp rx=0 tx=0).stop 2024-04-25T10:29:30.341 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:30.340+0000 7f400218f640 1 -- 172.21.15.37:0/3565669101 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f3ffc0d2810 msgr2=0x7f3ffc0b44e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:29:30.341 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:30.340+0000 7f400218f640 1 --2- 172.21.15.37:0/3565669101 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f3ffc0d2810 0x7f3ffc0b44e0 secure :-1 s=READY pgs=185 cs=0 l=1 rev1=1 crypto rx=0x7f3ff400c680 tx=0x7f3ff400cb40 comp rx=0 tx=0).stop 2024-04-25T10:29:30.342 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:30.340+0000 7f400218f640 1 -- 172.21.15.37:0/3565669101 shutdown_connections 2024-04-25T10:29:30.342 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:30.340+0000 7f400218f640 1 --2- 172.21.15.37:0/3565669101 >> [v2:172.21.15.203:6832/2418205237,v1:172.21.15.203:6833/2418205237] conn(0x7f3fe80744b0 0x7f3fe80769c0 unknown :-1 s=CLOSED pgs=114 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:29:30.342 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:30.340+0000 7f400218f640 1 --2- 172.21.15.37:0/3565669101 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f3ffc0d2810 0x7f3ffc0b44e0 unknown :-1 s=CLOSED pgs=185 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:29:30.342 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:30.340+0000 7f400218f640 1 --2- 172.21.15.37:0/3565669101 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f3ffc0d1ec0 0x7f3ffc0c7c90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:29:30.342 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:30.340+0000 7f400218f640 1 --2- 172.21.15.37:0/3565669101 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f3ffc06e260 0x7f3ffc1de960 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:29:30.342 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:30.340+0000 7f400218f640 1 -- 172.21.15.37:0/3565669101 >> 172.21.15.37:0/3565669101 conn(0x7f3ffc0d0390 msgr2=0x7f3ffc0b8770 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:29:30.342 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:30.340+0000 7f400218f640 1 -- 172.21.15.37:0/3565669101 shutdown_connections 2024-04-25T10:29:30.342 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:30.340+0000 7f400218f640 1 -- 172.21.15.37:0/3565669101 wait complete. 2024-04-25T10:29:30.352 INFO:tasks.cephfs.mount:Mounting default Ceph FS; just confirmed its presence on cluster 2024-04-25T10:29:30.352 INFO:tasks.cephfs.mount:Mounting Ceph FS. Following are details of mount; remember "None" represents Python type None - 2024-04-25T10:29:30.352 INFO:tasks.cephfs.mount:self.client_remote.hostname = smithi037.front.sepia.ceph.com 2024-04-25T10:29:30.352 INFO:tasks.cephfs.mount:self.client.name = client.2 2024-04-25T10:29:30.352 INFO:tasks.cephfs.mount:self.hostfs_mntpt = /home/ubuntu/cephtest/mnt.2 2024-04-25T10:29:30.352 INFO:tasks.cephfs.mount:self.cephfs_name = None 2024-04-25T10:29:30.352 INFO:tasks.cephfs.mount:self.cephfs_mntpt = None 2024-04-25T10:29:30.352 INFO:tasks.cephfs.mount:self.client_keyring_path = None 2024-04-25T10:29:30.352 INFO:tasks.cephfs.mount:Setting the 'None' netns for '/home/ubuntu/cephtest/mnt.2' 2024-04-25T10:29:30.353 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T10:29:30.353 DEBUG:teuthology.orchestra.run.smithi037:> ip addr 2024-04-25T10:29:30.369 INFO:teuthology.orchestra.run.smithi037.stdout:1: lo: mtu 65536 qdisc noqueue state UNKNOWN group default qlen 1000 2024-04-25T10:29:30.369 INFO:teuthology.orchestra.run.smithi037.stdout: link/loopback 00:00:00:00:00:00 brd 00:00:00:00:00:00 2024-04-25T10:29:30.369 INFO:teuthology.orchestra.run.smithi037.stdout: inet 127.0.0.1/8 scope host lo 2024-04-25T10:29:30.370 INFO:teuthology.orchestra.run.smithi037.stdout: valid_lft forever preferred_lft forever 2024-04-25T10:29:30.370 INFO:teuthology.orchestra.run.smithi037.stdout: inet6 ::1/128 scope host 2024-04-25T10:29:30.370 INFO:teuthology.orchestra.run.smithi037.stdout: valid_lft forever preferred_lft forever 2024-04-25T10:29:30.370 INFO:teuthology.orchestra.run.smithi037.stdout:2: eno1: mtu 1500 qdisc mq state DOWN group default qlen 1000 2024-04-25T10:29:30.370 INFO:teuthology.orchestra.run.smithi037.stdout: link/ether 0c:c4:7a:6b:fd:18 brd ff:ff:ff:ff:ff:ff 2024-04-25T10:29:30.370 INFO:teuthology.orchestra.run.smithi037.stdout: altname enp7s0f0 2024-04-25T10:29:30.370 INFO:teuthology.orchestra.run.smithi037.stdout:3: eno2: mtu 1500 qdisc mq state DOWN group default qlen 1000 2024-04-25T10:29:30.370 INFO:teuthology.orchestra.run.smithi037.stdout: link/ether 0c:c4:7a:6b:fd:19 brd ff:ff:ff:ff:ff:ff 2024-04-25T10:29:30.370 INFO:teuthology.orchestra.run.smithi037.stdout: altname enp7s0f1 2024-04-25T10:29:30.370 INFO:teuthology.orchestra.run.smithi037.stdout:4: ens1f0: mtu 1500 qdisc mq state UP group default qlen 1000 2024-04-25T10:29:30.370 INFO:teuthology.orchestra.run.smithi037.stdout: link/ether 0c:c4:7a:88:77:94 brd ff:ff:ff:ff:ff:ff 2024-04-25T10:29:30.370 INFO:teuthology.orchestra.run.smithi037.stdout: altname enp1s0f0 2024-04-25T10:29:30.371 INFO:teuthology.orchestra.run.smithi037.stdout: inet 172.21.15.37/20 brd 172.21.15.255 scope global dynamic noprefixroute ens1f0 2024-04-25T10:29:30.371 INFO:teuthology.orchestra.run.smithi037.stdout: valid_lft 42698sec preferred_lft 42698sec 2024-04-25T10:29:30.371 INFO:teuthology.orchestra.run.smithi037.stdout: inet6 fe80::ec4:7aff:fe88:7794/64 scope link noprefixroute 2024-04-25T10:29:30.371 INFO:teuthology.orchestra.run.smithi037.stdout: valid_lft forever preferred_lft forever 2024-04-25T10:29:30.371 INFO:teuthology.orchestra.run.smithi037.stdout:5: ens1f1: mtu 1500 qdisc mq state DOWN group default qlen 1000 2024-04-25T10:29:30.371 INFO:teuthology.orchestra.run.smithi037.stdout: link/ether 0c:c4:7a:88:77:95 brd ff:ff:ff:ff:ff:ff 2024-04-25T10:29:30.371 INFO:teuthology.orchestra.run.smithi037.stdout: altname enp1s0f1 2024-04-25T10:29:30.371 INFO:teuthology.orchestra.run.smithi037.stdout:6: ceph-brx: mtu 1500 qdisc noqueue state UP group default qlen 1000 2024-04-25T10:29:30.371 INFO:teuthology.orchestra.run.smithi037.stdout: link/ether ce:87:f0:58:c7:15 brd ff:ff:ff:ff:ff:ff 2024-04-25T10:29:30.371 INFO:teuthology.orchestra.run.smithi037.stdout: inet 192.168.255.254/16 brd 192.168.255.255 scope global ceph-brx 2024-04-25T10:29:30.371 INFO:teuthology.orchestra.run.smithi037.stdout: valid_lft forever preferred_lft forever 2024-04-25T10:29:30.371 INFO:teuthology.orchestra.run.smithi037.stdout: inet6 fe80::9827:f7ff:febf:df95/64 scope link 2024-04-25T10:29:30.371 INFO:teuthology.orchestra.run.smithi037.stdout: valid_lft forever preferred_lft forever 2024-04-25T10:29:30.371 INFO:teuthology.orchestra.run.smithi037.stdout:7: brx.0@if2: mtu 1500 qdisc noqueue master ceph-brx state UP group default qlen 1000 2024-04-25T10:29:30.372 INFO:teuthology.orchestra.run.smithi037.stdout: link/ether ce:87:f0:58:c7:15 brd ff:ff:ff:ff:ff:ff link-netns ceph-ns--home-ubuntu-cephtest-mnt.0 2024-04-25T10:29:30.372 INFO:teuthology.orchestra.run.smithi037.stdout: inet6 fe80::cc87:f0ff:fe58:c715/64 scope link tentative 2024-04-25T10:29:30.372 INFO:teuthology.orchestra.run.smithi037.stdout: valid_lft forever preferred_lft forever 2024-04-25T10:29:30.372 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T10:29:30.372 DEBUG:teuthology.orchestra.run.smithi037:> echo 1 | sudo tee /proc/sys/net/ipv4/ip_forward 2024-04-25T10:29:30.433 INFO:teuthology.orchestra.run.smithi037.stdout:1 2024-04-25T10:29:30.435 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T10:29:30.435 DEBUG:teuthology.orchestra.run.smithi037:> route 2024-04-25T10:29:30.492 INFO:teuthology.orchestra.run.smithi037.stdout:Kernel IP routing table 2024-04-25T10:29:30.492 INFO:teuthology.orchestra.run.smithi037.stdout:Destination Gateway Genmask Flags Metric Ref Use Iface 2024-04-25T10:29:30.492 INFO:teuthology.orchestra.run.smithi037.stdout:default _gateway 0.0.0.0 UG 100 0 0 ens1f0 2024-04-25T10:29:30.492 INFO:teuthology.orchestra.run.smithi037.stdout:172.21.0.0 0.0.0.0 255.255.240.0 U 100 0 0 ens1f0 2024-04-25T10:29:30.492 INFO:teuthology.orchestra.run.smithi037.stdout:192.168.0.0 0.0.0.0 255.255.0.0 U 0 0 0 ceph-brx 2024-04-25T10:29:30.492 DEBUG:teuthology.orchestra.run.smithi037:> (cd / && exec stdin-killer --timeout=300 -- bash -c ' 2024-04-25T10:29:30.492 DEBUG:teuthology.orchestra.run.smithi037:> set -e 2024-04-25T10:29:30.492 DEBUG:teuthology.orchestra.run.smithi037:> sudo iptables -A FORWARD -o ens1f0 -i ceph-brx -j ACCEPT 2024-04-25T10:29:30.492 DEBUG:teuthology.orchestra.run.smithi037:> sudo iptables -A FORWARD -i ens1f0 -o ceph-brx -j ACCEPT 2024-04-25T10:29:30.493 DEBUG:teuthology.orchestra.run.smithi037:> sudo iptables -t nat -A POSTROUTING -s 192.168.255.254/16 -o ens1f0 -j MASQUERADE 2024-04-25T10:29:30.493 DEBUG:teuthology.orchestra.run.smithi037:> ') 2024-04-25T10:29:30.581 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:30 stdin-killer INFO: expiration expected; waiting 300 seconds for command to complete 2024-04-25T10:29:30.612 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:30 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-04-25T10:29:30.618 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T10:29:30.618 DEBUG:teuthology.orchestra.run.smithi037:> ip netns list 2024-04-25T10:29:30.634 INFO:teuthology.orchestra.run.smithi037.stdout:ceph-ns--home-ubuntu-cephtest-mnt.0 (id: 0) 2024-04-25T10:29:30.635 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T10:29:30.635 DEBUG:teuthology.orchestra.run.smithi037:> ip netns list-id 2024-04-25T10:29:30.690 INFO:teuthology.orchestra.run.smithi037.stdout:nsid 0 (iproute2 netns name: ceph-ns--home-ubuntu-cephtest-mnt.0) 2024-04-25T10:29:30.691 DEBUG:teuthology.orchestra.run.smithi037:> (cd / && exec stdin-killer --timeout=300 -- bash -c ' 2024-04-25T10:29:30.691 DEBUG:teuthology.orchestra.run.smithi037:> set -e 2024-04-25T10:29:30.691 DEBUG:teuthology.orchestra.run.smithi037:> sudo ip netns add ceph-ns--home-ubuntu-cephtest-mnt.2 2024-04-25T10:29:30.691 DEBUG:teuthology.orchestra.run.smithi037:> sudo ip netns set ceph-ns--home-ubuntu-cephtest-mnt.2 1 2024-04-25T10:29:30.691 DEBUG:teuthology.orchestra.run.smithi037:> ') 2024-04-25T10:29:30.780 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:30 stdin-killer INFO: expiration expected; waiting 300 seconds for command to complete 2024-04-25T10:29:30.802 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:30 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-04-25T10:29:30.808 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T10:29:30.808 DEBUG:teuthology.orchestra.run.smithi037:> sudo ip netns exec ceph-ns--home-ubuntu-cephtest-mnt.0 ip addr 2024-04-25T10:29:30.832 INFO:teuthology.orchestra.run.smithi037.stdout:1: lo: mtu 65536 qdisc noqueue state UNKNOWN group default qlen 1000 2024-04-25T10:29:30.832 INFO:teuthology.orchestra.run.smithi037.stdout: link/loopback 00:00:00:00:00:00 brd 00:00:00:00:00:00 2024-04-25T10:29:30.832 INFO:teuthology.orchestra.run.smithi037.stdout: inet 127.0.0.1/8 scope host lo 2024-04-25T10:29:30.832 INFO:teuthology.orchestra.run.smithi037.stdout: valid_lft forever preferred_lft forever 2024-04-25T10:29:30.833 INFO:teuthology.orchestra.run.smithi037.stdout: inet6 ::1/128 scope host 2024-04-25T10:29:30.833 INFO:teuthology.orchestra.run.smithi037.stdout: valid_lft forever preferred_lft forever 2024-04-25T10:29:30.833 INFO:teuthology.orchestra.run.smithi037.stdout:2: veth0@if7: mtu 1500 qdisc noqueue state UP group default qlen 1000 2024-04-25T10:29:30.833 INFO:teuthology.orchestra.run.smithi037.stdout: link/ether be:59:cf:7f:f8:22 brd ff:ff:ff:ff:ff:ff link-netnsid 0 2024-04-25T10:29:30.833 INFO:teuthology.orchestra.run.smithi037.stdout: inet 192.168.0.1/16 brd 192.168.255.255 scope global veth0 2024-04-25T10:29:30.834 INFO:teuthology.orchestra.run.smithi037.stdout: valid_lft forever preferred_lft forever 2024-04-25T10:29:30.834 INFO:teuthology.orchestra.run.smithi037.stdout: inet6 fe80::bc59:cfff:fe7f:f822/64 scope link tentative 2024-04-25T10:29:30.834 INFO:teuthology.orchestra.run.smithi037.stdout: valid_lft forever preferred_lft forever 2024-04-25T10:29:30.836 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T10:29:30.836 DEBUG:teuthology.orchestra.run.smithi037:> sudo ip netns exec ceph-ns--home-ubuntu-cephtest-mnt.0 ip addr 2024-04-25T10:29:30.900 INFO:teuthology.orchestra.run.smithi037.stdout:1: lo: mtu 65536 qdisc noqueue state UNKNOWN group default qlen 1000 2024-04-25T10:29:30.900 INFO:teuthology.orchestra.run.smithi037.stdout: link/loopback 00:00:00:00:00:00 brd 00:00:00:00:00:00 2024-04-25T10:29:30.901 INFO:teuthology.orchestra.run.smithi037.stdout: inet 127.0.0.1/8 scope host lo 2024-04-25T10:29:30.901 INFO:teuthology.orchestra.run.smithi037.stdout: valid_lft forever preferred_lft forever 2024-04-25T10:29:30.901 INFO:teuthology.orchestra.run.smithi037.stdout: inet6 ::1/128 scope host 2024-04-25T10:29:30.901 INFO:teuthology.orchestra.run.smithi037.stdout: valid_lft forever preferred_lft forever 2024-04-25T10:29:30.901 INFO:teuthology.orchestra.run.smithi037.stdout:2: veth0@if7: mtu 1500 qdisc noqueue state UP group default qlen 1000 2024-04-25T10:29:30.901 INFO:teuthology.orchestra.run.smithi037.stdout: link/ether be:59:cf:7f:f8:22 brd ff:ff:ff:ff:ff:ff link-netnsid 0 2024-04-25T10:29:30.901 INFO:teuthology.orchestra.run.smithi037.stdout: inet 192.168.0.1/16 brd 192.168.255.255 scope global veth0 2024-04-25T10:29:30.901 INFO:teuthology.orchestra.run.smithi037.stdout: valid_lft forever preferred_lft forever 2024-04-25T10:29:30.901 INFO:teuthology.orchestra.run.smithi037.stdout: inet6 fe80::bc59:cfff:fe7f:f822/64 scope link tentative 2024-04-25T10:29:30.902 INFO:teuthology.orchestra.run.smithi037.stdout: valid_lft forever preferred_lft forever 2024-04-25T10:29:30.903 INFO:tasks.cephfs.mount:Setuping the netns 'ceph-ns--home-ubuntu-cephtest-mnt.2' with 192.168.0.2/16 2024-04-25T10:29:30.903 DEBUG:teuthology.orchestra.run.smithi037:> (cd / && exec stdin-killer --timeout=300 -- bash -c ' 2024-04-25T10:29:30.903 DEBUG:teuthology.orchestra.run.smithi037:> set -e 2024-04-25T10:29:30.903 DEBUG:teuthology.orchestra.run.smithi037:> sudo ip link add veth0 netns ceph-ns--home-ubuntu-cephtest-mnt.2 type veth peer name brx.1 2024-04-25T10:29:30.903 DEBUG:teuthology.orchestra.run.smithi037:> sudo ip netns exec ceph-ns--home-ubuntu-cephtest-mnt.2 ip addr add 192.168.0.2/16 brd 192.168.255.255 dev veth0 2024-04-25T10:29:30.903 DEBUG:teuthology.orchestra.run.smithi037:> sudo ip netns exec ceph-ns--home-ubuntu-cephtest-mnt.2 ip link set veth0 up 2024-04-25T10:29:30.903 DEBUG:teuthology.orchestra.run.smithi037:> sudo ip netns exec ceph-ns--home-ubuntu-cephtest-mnt.2 ip link set lo up 2024-04-25T10:29:30.903 DEBUG:teuthology.orchestra.run.smithi037:> sudo ip netns exec ceph-ns--home-ubuntu-cephtest-mnt.2 ip route add default via 192.168.255.254 2024-04-25T10:29:30.903 DEBUG:teuthology.orchestra.run.smithi037:> ') 2024-04-25T10:29:30.992 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:30 stdin-killer INFO: expiration expected; waiting 300 seconds for command to complete 2024-04-25T10:29:31.048 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:31 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-04-25T10:29:31.054 DEBUG:teuthology.orchestra.run.smithi037:> (cd / && exec stdin-killer --timeout=300 -- bash -c ' 2024-04-25T10:29:31.054 DEBUG:teuthology.orchestra.run.smithi037:> set -e 2024-04-25T10:29:31.054 DEBUG:teuthology.orchestra.run.smithi037:> sudo ip link set brx.1 up 2024-04-25T10:29:31.055 DEBUG:teuthology.orchestra.run.smithi037:> sudo ip link set dev brx.1 master ceph-brx 2024-04-25T10:29:31.055 DEBUG:teuthology.orchestra.run.smithi037:> ') 2024-04-25T10:29:31.104 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:31 stdin-killer INFO: expiration expected; waiting 300 seconds for command to complete 2024-04-25T10:29:31.125 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:31 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-04-25T10:29:31.131 INFO:tasks.cephfs.fuse_mount:Client client.2 config is {} 2024-04-25T10:29:31.131 INFO:teuthology.orchestra.run:Running command with timeout 60 2024-04-25T10:29:31.131 DEBUG:teuthology.orchestra.run.smithi037:> mkdir -p -v /home/ubuntu/cephtest/mnt.2 2024-04-25T10:29:31.146 INFO:teuthology.orchestra.run.smithi037.stdout:mkdir: created directory '/home/ubuntu/cephtest/mnt.2' 2024-04-25T10:29:31.146 INFO:teuthology.orchestra.run:Running command with timeout 60 2024-04-25T10:29:31.146 DEBUG:teuthology.orchestra.run.smithi037:> chmod 0000 /home/ubuntu/cephtest/mnt.2 2024-04-25T10:29:31.202 DEBUG:teuthology.orchestra.run.smithi037:> sudo modprobe fuse 2024-04-25T10:29:31.268 DEBUG:teuthology.orchestra.run.smithi037:> cat /proc/self/mounts | awk '{print $2}' 2024-04-25T10:29:31.326 INFO:teuthology.orchestra.run.smithi037.stdout:/proc 2024-04-25T10:29:31.326 INFO:teuthology.orchestra.run.smithi037.stdout:/sys 2024-04-25T10:29:31.326 INFO:teuthology.orchestra.run.smithi037.stdout:/dev 2024-04-25T10:29:31.326 INFO:teuthology.orchestra.run.smithi037.stdout:/sys/kernel/security 2024-04-25T10:29:31.326 INFO:teuthology.orchestra.run.smithi037.stdout:/dev/shm 2024-04-25T10:29:31.326 INFO:teuthology.orchestra.run.smithi037.stdout:/dev/pts 2024-04-25T10:29:31.326 INFO:teuthology.orchestra.run.smithi037.stdout:/run 2024-04-25T10:29:31.326 INFO:teuthology.orchestra.run.smithi037.stdout:/sys/fs/cgroup 2024-04-25T10:29:31.327 INFO:teuthology.orchestra.run.smithi037.stdout:/sys/fs/pstore 2024-04-25T10:29:31.327 INFO:teuthology.orchestra.run.smithi037.stdout:/sys/fs/bpf 2024-04-25T10:29:31.327 INFO:teuthology.orchestra.run.smithi037.stdout:/ 2024-04-25T10:29:31.327 INFO:teuthology.orchestra.run.smithi037.stdout:/sys/fs/selinux 2024-04-25T10:29:31.327 INFO:teuthology.orchestra.run.smithi037.stdout:/proc/sys/fs/binfmt_misc 2024-04-25T10:29:31.327 INFO:teuthology.orchestra.run.smithi037.stdout:/dev/hugepages 2024-04-25T10:29:31.327 INFO:teuthology.orchestra.run.smithi037.stdout:/dev/mqueue 2024-04-25T10:29:31.327 INFO:teuthology.orchestra.run.smithi037.stdout:/sys/kernel/debug 2024-04-25T10:29:31.327 INFO:teuthology.orchestra.run.smithi037.stdout:/sys/kernel/tracing 2024-04-25T10:29:31.327 INFO:teuthology.orchestra.run.smithi037.stdout:/sys/fs/fuse/connections 2024-04-25T10:29:31.327 INFO:teuthology.orchestra.run.smithi037.stdout:/sys/kernel/config 2024-04-25T10:29:31.327 INFO:teuthology.orchestra.run.smithi037.stdout:/run/credentials/systemd-sysusers.service 2024-04-25T10:29:31.327 INFO:teuthology.orchestra.run.smithi037.stdout:/run/credentials/systemd-sysctl.service 2024-04-25T10:29:31.327 INFO:teuthology.orchestra.run.smithi037.stdout:/run/credentials/systemd-tmpfiles-setup-dev.service 2024-04-25T10:29:31.328 INFO:teuthology.orchestra.run.smithi037.stdout:/run/credentials/systemd-tmpfiles-setup.service 2024-04-25T10:29:31.328 INFO:teuthology.orchestra.run.smithi037.stdout:/var/lib/nfs/rpc_pipefs 2024-04-25T10:29:31.328 INFO:teuthology.orchestra.run.smithi037.stdout:/run/user/1000 2024-04-25T10:29:31.328 INFO:teuthology.orchestra.run.smithi037.stdout:/var/lib/ceph 2024-04-25T10:29:31.328 INFO:teuthology.orchestra.run.smithi037.stdout:/var/lib/ceph/osd/ceph-0 2024-04-25T10:29:31.328 INFO:teuthology.orchestra.run.smithi037.stdout:/var/lib/ceph/osd/ceph-1 2024-04-25T10:29:31.328 INFO:teuthology.orchestra.run.smithi037.stdout:/var/lib/ceph/osd/ceph-2 2024-04-25T10:29:31.328 INFO:teuthology.orchestra.run.smithi037.stdout:/var/lib/ceph/osd/ceph-3 2024-04-25T10:29:31.328 INFO:teuthology.orchestra.run.smithi037.stdout:/run/netns 2024-04-25T10:29:31.328 INFO:teuthology.orchestra.run.smithi037.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 2024-04-25T10:29:31.328 INFO:teuthology.orchestra.run.smithi037.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 2024-04-25T10:29:31.328 INFO:teuthology.orchestra.run.smithi037.stdout:/home/ubuntu/cephtest/mnt.0 2024-04-25T10:29:31.328 INFO:teuthology.orchestra.run.smithi037.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 2024-04-25T10:29:31.328 INFO:teuthology.orchestra.run.smithi037.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 2024-04-25T10:29:31.329 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T10:29:31.329 DEBUG:teuthology.orchestra.run.smithi037:> ls /sys/fs/fuse/connections 2024-04-25T10:29:31.381 INFO:teuthology.orchestra.run.smithi037.stdout:43 2024-04-25T10:29:31.381 INFO:tasks.cephfs.fuse_mount:Pre-mount connections: [43] 2024-04-25T10:29:31.381 DEBUG:teuthology.orchestra.run.smithi037:> (cd /home/ubuntu/cephtest && exec sudo nsenter --net=/var/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-fuse -f --admin-socket '/var/run/ceph/$cluster-$name.$pid.asok' /home/ubuntu/cephtest/mnt.2 --id 2) 2024-04-25T10:29:31.423 DEBUG:teuthology.orchestra.run.smithi037:> sudo modprobe fuse 2024-04-25T10:29:31.448 DEBUG:teuthology.orchestra.run.smithi037:> cat /proc/self/mounts | awk '{print $2}' 2024-04-25T10:29:31.506 INFO:teuthology.orchestra.run.smithi037.stdout:/proc 2024-04-25T10:29:31.506 INFO:teuthology.orchestra.run.smithi037.stdout:/sys 2024-04-25T10:29:31.506 INFO:teuthology.orchestra.run.smithi037.stdout:/dev 2024-04-25T10:29:31.506 INFO:teuthology.orchestra.run.smithi037.stdout:/sys/kernel/security 2024-04-25T10:29:31.506 INFO:teuthology.orchestra.run.smithi037.stdout:/dev/shm 2024-04-25T10:29:31.506 INFO:teuthology.orchestra.run.smithi037.stdout:/dev/pts 2024-04-25T10:29:31.506 INFO:teuthology.orchestra.run.smithi037.stdout:/run 2024-04-25T10:29:31.506 INFO:teuthology.orchestra.run.smithi037.stdout:/sys/fs/cgroup 2024-04-25T10:29:31.506 INFO:teuthology.orchestra.run.smithi037.stdout:/sys/fs/pstore 2024-04-25T10:29:31.506 INFO:teuthology.orchestra.run.smithi037.stdout:/sys/fs/bpf 2024-04-25T10:29:31.507 INFO:teuthology.orchestra.run.smithi037.stdout:/ 2024-04-25T10:29:31.507 INFO:teuthology.orchestra.run.smithi037.stdout:/sys/fs/selinux 2024-04-25T10:29:31.507 INFO:teuthology.orchestra.run.smithi037.stdout:/proc/sys/fs/binfmt_misc 2024-04-25T10:29:31.507 INFO:teuthology.orchestra.run.smithi037.stdout:/dev/hugepages 2024-04-25T10:29:31.507 INFO:teuthology.orchestra.run.smithi037.stdout:/dev/mqueue 2024-04-25T10:29:31.507 INFO:teuthology.orchestra.run.smithi037.stdout:/sys/kernel/debug 2024-04-25T10:29:31.507 INFO:teuthology.orchestra.run.smithi037.stdout:/sys/kernel/tracing 2024-04-25T10:29:31.507 INFO:teuthology.orchestra.run.smithi037.stdout:/sys/fs/fuse/connections 2024-04-25T10:29:31.507 INFO:teuthology.orchestra.run.smithi037.stdout:/sys/kernel/config 2024-04-25T10:29:31.507 INFO:teuthology.orchestra.run.smithi037.stdout:/run/credentials/systemd-sysusers.service 2024-04-25T10:29:31.507 INFO:teuthology.orchestra.run.smithi037.stdout:/run/credentials/systemd-sysctl.service 2024-04-25T10:29:31.507 INFO:teuthology.orchestra.run.smithi037.stdout:/run/credentials/systemd-tmpfiles-setup-dev.service 2024-04-25T10:29:31.507 INFO:teuthology.orchestra.run.smithi037.stdout:/run/credentials/systemd-tmpfiles-setup.service 2024-04-25T10:29:31.507 INFO:teuthology.orchestra.run.smithi037.stdout:/var/lib/nfs/rpc_pipefs 2024-04-25T10:29:31.508 INFO:teuthology.orchestra.run.smithi037.stdout:/run/user/1000 2024-04-25T10:29:31.508 INFO:teuthology.orchestra.run.smithi037.stdout:/var/lib/ceph 2024-04-25T10:29:31.508 INFO:teuthology.orchestra.run.smithi037.stdout:/var/lib/ceph/osd/ceph-0 2024-04-25T10:29:31.508 INFO:teuthology.orchestra.run.smithi037.stdout:/var/lib/ceph/osd/ceph-1 2024-04-25T10:29:31.508 INFO:teuthology.orchestra.run.smithi037.stdout:/var/lib/ceph/osd/ceph-2 2024-04-25T10:29:31.508 INFO:teuthology.orchestra.run.smithi037.stdout:/var/lib/ceph/osd/ceph-3 2024-04-25T10:29:31.508 INFO:teuthology.orchestra.run.smithi037.stdout:/run/netns 2024-04-25T10:29:31.508 INFO:teuthology.orchestra.run.smithi037.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 2024-04-25T10:29:31.508 INFO:teuthology.orchestra.run.smithi037.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 2024-04-25T10:29:31.508 INFO:teuthology.orchestra.run.smithi037.stdout:/home/ubuntu/cephtest/mnt.0 2024-04-25T10:29:31.508 INFO:teuthology.orchestra.run.smithi037.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 2024-04-25T10:29:31.508 INFO:teuthology.orchestra.run.smithi037.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 2024-04-25T10:29:31.508 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T10:29:31.509 DEBUG:teuthology.orchestra.run.smithi037:> ls /sys/fs/fuse/connections 2024-04-25T10:29:31.515 INFO:tasks.cephfs.fuse_mount.ceph-fuse.2.smithi037.stderr:2024-04-25T10:29:31.514+0000 7fce38327480 -1 init, newargv = 0x7fce1c005d20 newargc=15 2024-04-25T10:29:31.515 INFO:tasks.cephfs.fuse_mount.ceph-fuse.2.smithi037.stderr:ceph-fuse[43722]: starting ceph client 2024-04-25T10:29:31.521 INFO:tasks.cephfs.fuse_mount.ceph-fuse.2.smithi037.stderr:ceph-fuse[43722]: starting fuse 2024-04-25T10:29:31.530 INFO:teuthology.orchestra.run.smithi037.stdout:43 2024-04-25T10:29:31.530 INFO:teuthology.orchestra.run.smithi037.stdout:44 2024-04-25T10:29:31.530 INFO:tasks.cephfs.fuse_mount:Post-mount connections: [43, 44] 2024-04-25T10:29:31.531 DEBUG:teuthology.orchestra.run.smithi037:> sudo stdin-killer -- python3 -c ' 2024-04-25T10:29:31.531 DEBUG:teuthology.orchestra.run.smithi037:> import glob 2024-04-25T10:29:31.531 DEBUG:teuthology.orchestra.run.smithi037:> import re 2024-04-25T10:29:31.531 DEBUG:teuthology.orchestra.run.smithi037:> import os 2024-04-25T10:29:31.531 DEBUG:teuthology.orchestra.run.smithi037:> import subprocess 2024-04-25T10:29:31.531 DEBUG:teuthology.orchestra.run.smithi037:> 2024-04-25T10:29:31.531 DEBUG:teuthology.orchestra.run.smithi037:> def _find_admin_socket(client_name): 2024-04-25T10:29:31.531 DEBUG:teuthology.orchestra.run.smithi037:> asok_path = "/var/run/ceph/ceph-client.2.*.asok" 2024-04-25T10:29:31.531 DEBUG:teuthology.orchestra.run.smithi037:> files = glob.glob(asok_path) 2024-04-25T10:29:31.531 DEBUG:teuthology.orchestra.run.smithi037:> mountpoint = "/home/ubuntu/cephtest/mnt.2" 2024-04-25T10:29:31.531 DEBUG:teuthology.orchestra.run.smithi037:> 2024-04-25T10:29:31.531 DEBUG:teuthology.orchestra.run.smithi037:> # Given a non-glob path, it better be there 2024-04-25T10:29:31.531 DEBUG:teuthology.orchestra.run.smithi037:> if "*" not in asok_path: 2024-04-25T10:29:31.532 DEBUG:teuthology.orchestra.run.smithi037:> assert(len(files) == 1) 2024-04-25T10:29:31.532 DEBUG:teuthology.orchestra.run.smithi037:> return files[0] 2024-04-25T10:29:31.532 DEBUG:teuthology.orchestra.run.smithi037:> 2024-04-25T10:29:31.532 DEBUG:teuthology.orchestra.run.smithi037:> for f in files: 2024-04-25T10:29:31.532 DEBUG:teuthology.orchestra.run.smithi037:> pid = re.match(".*\.(\d+)\.asok$", f).group(1) 2024-04-25T10:29:31.532 DEBUG:teuthology.orchestra.run.smithi037:> if os.path.exists("/proc/{0}".format(pid)): 2024-04-25T10:29:31.532 DEBUG:teuthology.orchestra.run.smithi037:> with open("/proc/{0}/cmdline".format(pid), '"'"'r'"'"') as proc_f: 2024-04-25T10:29:31.532 DEBUG:teuthology.orchestra.run.smithi037:> contents = proc_f.read() 2024-04-25T10:29:31.532 DEBUG:teuthology.orchestra.run.smithi037:> if mountpoint in contents: 2024-04-25T10:29:31.532 DEBUG:teuthology.orchestra.run.smithi037:> return f 2024-04-25T10:29:31.532 DEBUG:teuthology.orchestra.run.smithi037:> raise RuntimeError("Client socket {0} not found".format(client_name)) 2024-04-25T10:29:31.532 DEBUG:teuthology.orchestra.run.smithi037:> 2024-04-25T10:29:31.532 DEBUG:teuthology.orchestra.run.smithi037:> print(_find_admin_socket("client.2")) 2024-04-25T10:29:31.533 DEBUG:teuthology.orchestra.run.smithi037:> ' 2024-04-25T10:29:31.650 INFO:teuthology.orchestra.run.smithi037.stdout:/var/run/ceph/ceph-client.2.43722.asok 2024-04-25T10:29:31.653 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:31 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-04-25T10:29:31.661 INFO:tasks.cephfs.fuse_mount:Found client admin socket at /var/run/ceph/ceph-client.2.43722.asok 2024-04-25T10:29:31.661 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T10:29:31.661 DEBUG:teuthology.orchestra.run.smithi037:> sudo ceph --admin-daemon /var/run/ceph/ceph-client.2.43722.asok status 2024-04-25T10:29:31.806 INFO:teuthology.orchestra.run.smithi037.stdout:{ 2024-04-25T10:29:31.806 INFO:teuthology.orchestra.run.smithi037.stdout: "metadata": { 2024-04-25T10:29:31.806 INFO:teuthology.orchestra.run.smithi037.stdout: "ceph_sha1": "994e2f20102a20fa9a27966c14b625cc01884df8", 2024-04-25T10:29:31.806 INFO:teuthology.orchestra.run.smithi037.stdout: "ceph_version": "ceph version 19.0.0-3291-g994e2f20 (994e2f20102a20fa9a27966c14b625cc01884df8) squid (dev)", 2024-04-25T10:29:31.806 INFO:teuthology.orchestra.run.smithi037.stdout: "entity_id": "2", 2024-04-25T10:29:31.807 INFO:teuthology.orchestra.run.smithi037.stdout: "hostname": "smithi037", 2024-04-25T10:29:31.807 INFO:teuthology.orchestra.run.smithi037.stdout: "mount_point": "/home/ubuntu/cephtest/mnt.2", 2024-04-25T10:29:31.807 INFO:teuthology.orchestra.run.smithi037.stdout: "pid": "43722", 2024-04-25T10:29:31.807 INFO:teuthology.orchestra.run.smithi037.stdout: "root": "/" 2024-04-25T10:29:31.807 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-04-25T10:29:31.807 INFO:teuthology.orchestra.run.smithi037.stdout: "dentry_count": 0, 2024-04-25T10:29:31.807 INFO:teuthology.orchestra.run.smithi037.stdout: "dentry_pinned_count": 0, 2024-04-25T10:29:31.807 INFO:teuthology.orchestra.run.smithi037.stdout: "id": 4749, 2024-04-25T10:29:31.807 INFO:teuthology.orchestra.run.smithi037.stdout: "inst": { 2024-04-25T10:29:31.807 INFO:teuthology.orchestra.run.smithi037.stdout: "name": { 2024-04-25T10:29:31.807 INFO:teuthology.orchestra.run.smithi037.stdout: "type": "client", 2024-04-25T10:29:31.807 INFO:teuthology.orchestra.run.smithi037.stdout: "num": 4749 2024-04-25T10:29:31.807 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-04-25T10:29:31.807 INFO:teuthology.orchestra.run.smithi037.stdout: "addr": { 2024-04-25T10:29:31.808 INFO:teuthology.orchestra.run.smithi037.stdout: "type": "v1", 2024-04-25T10:29:31.808 INFO:teuthology.orchestra.run.smithi037.stdout: "addr": "192.168.0.2:0", 2024-04-25T10:29:31.808 INFO:teuthology.orchestra.run.smithi037.stdout: "nonce": 1490986163 2024-04-25T10:29:31.808 INFO:teuthology.orchestra.run.smithi037.stdout: } 2024-04-25T10:29:31.808 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-04-25T10:29:31.808 INFO:teuthology.orchestra.run.smithi037.stdout: "addr": { 2024-04-25T10:29:31.808 INFO:teuthology.orchestra.run.smithi037.stdout: "type": "v1", 2024-04-25T10:29:31.808 INFO:teuthology.orchestra.run.smithi037.stdout: "addr": "192.168.0.2:0", 2024-04-25T10:29:31.808 INFO:teuthology.orchestra.run.smithi037.stdout: "nonce": 1490986163 2024-04-25T10:29:31.808 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-04-25T10:29:31.808 INFO:teuthology.orchestra.run.smithi037.stdout: "inst_str": "client.4749 192.168.0.2:0/1490986163", 2024-04-25T10:29:31.808 INFO:teuthology.orchestra.run.smithi037.stdout: "addr_str": "192.168.0.2:0/1490986163", 2024-04-25T10:29:31.808 INFO:teuthology.orchestra.run.smithi037.stdout: "inode_count": 1, 2024-04-25T10:29:31.808 INFO:teuthology.orchestra.run.smithi037.stdout: "mds_epoch": 8, 2024-04-25T10:29:31.808 INFO:teuthology.orchestra.run.smithi037.stdout: "osd_epoch": 23, 2024-04-25T10:29:31.809 INFO:teuthology.orchestra.run.smithi037.stdout: "osd_epoch_barrier": 0, 2024-04-25T10:29:31.809 INFO:teuthology.orchestra.run.smithi037.stdout: "blocklisted": false, 2024-04-25T10:29:31.809 INFO:teuthology.orchestra.run.smithi037.stdout: "fs_name": "cephfs" 2024-04-25T10:29:31.809 INFO:teuthology.orchestra.run.smithi037.stdout:} 2024-04-25T10:29:31.815 DEBUG:teuthology.orchestra.run.smithi037:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs ls 2024-04-25T10:29:31.926 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:31.925+0000 7fbdcf1c7640 1 Processor -- start 2024-04-25T10:29:31.927 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:31.925+0000 7fbdcf1c7640 1 -- start start 2024-04-25T10:29:31.930 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:31.926+0000 7fbdcf1c7640 1 --2- >> v2:172.21.15.37:3300/0 conn(0x7fbdc8143500 0x7fbdc806f100 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:29:31.930 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:31.926+0000 7fbdcf1c7640 1 -- --> v1:172.21.15.37:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fbdc81614c0 con 0x7fbdc81460e0 2024-04-25T10:29:31.930 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:31.926+0000 7fbdcf1c7640 1 -- --> v1:172.21.15.203:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fbdc8161730 con 0x7fbdc81610b0 2024-04-25T10:29:31.930 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:31.926+0000 7fbdcf1c7640 1 -- --> v2:172.21.15.37:3300/0 -- mon_getmap magic: 0 -- 0x7fbdc80b5b60 con 0x7fbdc8143500 2024-04-25T10:29:31.930 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:31.926+0000 7fbdccf63640 1 --2- >> v2:172.21.15.37:3300/0 conn(0x7fbdc8143500 0x7fbdc806f100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:29:31.931 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:31.926+0000 7fbdc7fff640 1 --1- >> v1:172.21.15.37:6789/0 conn(0x7fbdc81460e0 0x7fbdc81609a0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.37:6789/0 says I am v1:172.21.15.37:46506/0 (socket says 172.21.15.37:46506) 2024-04-25T10:29:31.931 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:31.926+0000 7fbdc7fff640 1 -- 172.21.15.37:0/614862702 learned_addr learned my addr 172.21.15.37:0/614862702 (peer_addr_for_me v1:172.21.15.37:0/0) 2024-04-25T10:29:31.931 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:31.927+0000 7fbdccf63640 1 -- 172.21.15.37:0/614862702 >> v1:172.21.15.203:6789/0 conn(0x7fbdc81610b0 legacy=0x7fbdc8143b30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:29:31.931 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:31.927+0000 7fbdccf63640 1 -- 172.21.15.37:0/614862702 >> v1:172.21.15.37:6789/0 conn(0x7fbdc81460e0 legacy=0x7fbdc81609a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:29:31.931 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:31.927+0000 7fbdccf63640 1 -- 172.21.15.37:0/614862702 --> v2:172.21.15.37:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fbdc80b5e40 con 0x7fbdc8143500 2024-04-25T10:29:31.931 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:31.927+0000 7fbdccf63640 1 --2- 172.21.15.37:0/614862702 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fbdc8143500 0x7fbdc806f100 secure :-1 s=READY pgs=190 cs=0 l=1 rev1=1 crypto rx=0x7fbdbc0097a0 tx=0x7fbdbc02a230 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e5659a244d42b7d8 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:29:31.932 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:31.928+0000 7fbdc77fe640 1 -- 172.21.15.37:0/614862702 <== mon.0 v2:172.21.15.37:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbdbc037070 con 0x7fbdc8143500 2024-04-25T10:29:31.932 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:31.928+0000 7fbdc77fe640 1 -- 172.21.15.37:0/614862702 <== mon.0 v2:172.21.15.37:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fbdbc032440 con 0x7fbdc8143500 2024-04-25T10:29:31.932 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:31.928+0000 7fbdc77fe640 1 -- 172.21.15.37:0/614862702 <== mon.0 v2:172.21.15.37:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbdbc032810 con 0x7fbdc8143500 2024-04-25T10:29:31.932 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:31.928+0000 7fbdcf1c7640 1 -- 172.21.15.37:0/614862702 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fbdc8143500 msgr2=0x7fbdc806f100 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:29:31.932 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:31.928+0000 7fbdcf1c7640 1 --2- 172.21.15.37:0/614862702 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fbdc8143500 0x7fbdc806f100 secure :-1 s=READY pgs=190 cs=0 l=1 rev1=1 crypto rx=0x7fbdbc0097a0 tx=0x7fbdbc02a230 comp rx=0 tx=0).stop 2024-04-25T10:29:31.932 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:31.928+0000 7fbdcf1c7640 1 -- 172.21.15.37:0/614862702 shutdown_connections 2024-04-25T10:29:31.932 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:31.928+0000 7fbdcf1c7640 1 --2- 172.21.15.37:0/614862702 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fbdc8143500 0x7fbdc806f100 unknown :-1 s=CLOSED pgs=190 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:29:31.932 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:31.928+0000 7fbdcf1c7640 1 -- 172.21.15.37:0/614862702 >> 172.21.15.37:0/614862702 conn(0x7fbdc80b36b0 msgr2=0x7fbdc80b3ac0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:29:31.933 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:31.928+0000 7fbdcf1c7640 1 -- 172.21.15.37:0/614862702 shutdown_connections 2024-04-25T10:29:31.933 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:31.928+0000 7fbdcf1c7640 1 -- 172.21.15.37:0/614862702 wait complete. 2024-04-25T10:29:31.938 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:31.930+0000 7fbdcf1c7640 1 Processor -- start 2024-04-25T10:29:31.938 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:31.930+0000 7fbdcf1c7640 1 -- start start 2024-04-25T10:29:31.938 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:31.932+0000 7fbdcf1c7640 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fbdc81460e0 0x7fbdc80adce0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:29:31.939 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:31.932+0000 7fbdcf1c7640 1 --2- >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7fbdc81610b0 0x7fbdc81f0d80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:29:31.939 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:31.932+0000 7fbdcf1c7640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fbdc806f990 0x7fbdc806fda0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:29:31.939 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:31.932+0000 7fbdcf1c7640 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 -- 0x7fbdc80b67c0 con 0x7fbdc806f990 2024-04-25T10:29:31.939 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:31.932+0000 7fbdcf1c7640 1 -- --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_getmap magic: 0 -- 0x7fbdc80b6640 con 0x7fbdc81460e0 2024-04-25T10:29:31.939 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:31.932+0000 7fbdcf1c7640 1 -- --> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] -- mon_getmap magic: 0 -- 0x7fbdc80b6940 con 0x7fbdc81610b0 2024-04-25T10:29:31.939 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:31.932+0000 7fbdcd764640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fbdc806f990 0x7fbdc806fda0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:29:31.939 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:31.932+0000 7fbdcd764640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fbdc806f990 0x7fbdc806fda0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.37:3300/0 says I am v2:172.21.15.37:36758/0 (socket says 172.21.15.37:36758) 2024-04-25T10:29:31.939 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:31.932+0000 7fbdcd764640 1 -- 172.21.15.37:0/2732441221 learned_addr learned my addr 172.21.15.37:0/2732441221 (peer_addr_for_me v2:172.21.15.37:0/0) 2024-04-25T10:29:31.940 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:31.932+0000 7fbdccf63640 1 --2- 172.21.15.37:0/2732441221 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fbdc81460e0 0x7fbdc80adce0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:29:31.940 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:31.932+0000 7fbdc7fff640 1 --2- 172.21.15.37:0/2732441221 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7fbdc81610b0 0x7fbdc81f0d80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:29:31.940 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:31.932+0000 7fbdcd764640 1 -- 172.21.15.37:0/2732441221 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7fbdc81610b0 msgr2=0x7fbdc81f0d80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:29:31.940 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:31.932+0000 7fbdcd764640 1 --2- 172.21.15.37:0/2732441221 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7fbdc81610b0 0x7fbdc81f0d80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:29:31.940 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:31.932+0000 7fbdcd764640 1 -- 172.21.15.37:0/2732441221 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fbdc81460e0 msgr2=0x7fbdc80adce0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:29:31.940 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:31.932+0000 7fbdcd764640 1 --2- 172.21.15.37:0/2732441221 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fbdc81460e0 0x7fbdc80adce0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:29:31.940 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:31.932+0000 7fbdcd764640 1 -- 172.21.15.37:0/2732441221 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fbdc80702e0 con 0x7fbdc806f990 2024-04-25T10:29:31.940 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:31.932+0000 7fbdc7fff640 1 --2- 172.21.15.37:0/2732441221 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7fbdc81610b0 0x7fbdc81f0d80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T10:29:31.941 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:31.932+0000 7fbdcd764640 1 --2- 172.21.15.37:0/2732441221 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fbdc806f990 0x7fbdc806fda0 secure :-1 s=READY pgs=191 cs=0 l=1 rev1=1 crypto rx=0x7fbdc000c6b0 tx=0x7fbdc000cb70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:29:31.941 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:31.932+0000 7fbdc57fa640 1 -- 172.21.15.37:0/2732441221 <== mon.0 v2:172.21.15.37:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbdc0003b70 con 0x7fbdc806f990 2024-04-25T10:29:31.941 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:31.932+0000 7fbdc57fa640 1 -- 172.21.15.37:0/2732441221 <== mon.0 v2:172.21.15.37:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fbdc0003d30 con 0x7fbdc806f990 2024-04-25T10:29:31.941 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:31.932+0000 7fbdc57fa640 1 -- 172.21.15.37:0/2732441221 <== mon.0 v2:172.21.15.37:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbdc0005310 con 0x7fbdc806f990 2024-04-25T10:29:31.941 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:31.932+0000 7fbdcf1c7640 1 -- 172.21.15.37:0/2732441221 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fbdc81f3a00 con 0x7fbdc806f990 2024-04-25T10:29:31.941 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:31.932+0000 7fbdcf1c7640 1 -- 172.21.15.37:0/2732441221 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fbdc81f3cc0 con 0x7fbdc806f990 2024-04-25T10:29:31.941 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:31.933+0000 7fbdcf1c7640 1 -- 172.21.15.37:0/2732441221 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fbdc806a140 con 0x7fbdc806f990 2024-04-25T10:29:31.942 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:31.934+0000 7fbdc57fa640 1 -- 172.21.15.37:0/2732441221 <== mon.0 v2:172.21.15.37:3300/0 4 ==== mgrmap(e 5) ==== 96782+0+0 (secure 0 0 0) 0x7fbdc00054b0 con 0x7fbdc806f990 2024-04-25T10:29:31.942 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:31.934+0000 7fbdc57fa640 1 --2- 172.21.15.37:0/2732441221 >> [v2:172.21.15.203:6832/2418205237,v1:172.21.15.203:6833/2418205237] conn(0x7fbdac0744b0 0x7fbdac0769c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:29:31.942 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:31.935+0000 7fbdc57fa640 1 -- 172.21.15.37:0/2732441221 <== mon.0 v2:172.21.15.37:3300/0 5 ==== osd_map(23..23 src has 1..23) ==== 5709+0+0 (secure 0 0 0) 0x7fbdc0094f70 con 0x7fbdc806f990 2024-04-25T10:29:31.942 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:31.935+0000 7fbdccf63640 1 --2- 172.21.15.37:0/2732441221 >> [v2:172.21.15.203:6832/2418205237,v1:172.21.15.203:6833/2418205237] conn(0x7fbdac0744b0 0x7fbdac0769c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:29:31.942 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:31.935+0000 7fbdccf63640 1 --2- 172.21.15.37:0/2732441221 >> [v2:172.21.15.203:6832/2418205237,v1:172.21.15.203:6833/2418205237] conn(0x7fbdac0744b0 0x7fbdac0769c0 secure :-1 s=READY pgs=115 cs=0 l=1 rev1=1 crypto rx=0x7fbdbc034000 tx=0x7fbdbc02e000 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:29:31.943 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:31.939+0000 7fbdc57fa640 1 -- 172.21.15.37:0/2732441221 <== mon.0 v2:172.21.15.37:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187246 (secure 0 0 0) 0x7fbdc80702e0 con 0x7fbdc806f990 2024-04-25T10:29:32.165 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:32.164+0000 7fbdcf1c7640 1 -- 172.21.15.37:0/2732441221 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_command({"prefix": "fs ls"} v 0) -- 0x7fbdc8160880 con 0x7fbdc806f990 2024-04-25T10:29:32.165 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:32.164+0000 7fbdc57fa640 1 -- 172.21.15.37:0/2732441221 <== mon.0 v2:172.21.15.37:3300/0 7 ==== mon_command_ack([{"prefix": "fs ls"}]=0 v8) ==== 53+0+73 (secure 0 0 0) 0x7fbdc005f790 con 0x7fbdc806f990 2024-04-25T10:29:32.166 INFO:teuthology.orchestra.run.smithi037.stdout:name: cephfs, metadata pool: cephfs_metadata, data pools: [cephfs_data ] 2024-04-25T10:29:32.168 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:32.166+0000 7fbdcf1c7640 1 -- 172.21.15.37:0/2732441221 >> [v2:172.21.15.203:6832/2418205237,v1:172.21.15.203:6833/2418205237] conn(0x7fbdac0744b0 msgr2=0x7fbdac0769c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:29:32.168 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:32.166+0000 7fbdcf1c7640 1 --2- 172.21.15.37:0/2732441221 >> [v2:172.21.15.203:6832/2418205237,v1:172.21.15.203:6833/2418205237] conn(0x7fbdac0744b0 0x7fbdac0769c0 secure :-1 s=READY pgs=115 cs=0 l=1 rev1=1 crypto rx=0x7fbdbc034000 tx=0x7fbdbc02e000 comp rx=0 tx=0).stop 2024-04-25T10:29:32.168 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:32.166+0000 7fbdcf1c7640 1 -- 172.21.15.37:0/2732441221 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fbdc806f990 msgr2=0x7fbdc806fda0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:29:32.168 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:32.166+0000 7fbdcf1c7640 1 --2- 172.21.15.37:0/2732441221 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fbdc806f990 0x7fbdc806fda0 secure :-1 s=READY pgs=191 cs=0 l=1 rev1=1 crypto rx=0x7fbdc000c6b0 tx=0x7fbdc000cb70 comp rx=0 tx=0).stop 2024-04-25T10:29:32.168 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:32.166+0000 7fbdcf1c7640 1 -- 172.21.15.37:0/2732441221 shutdown_connections 2024-04-25T10:29:32.168 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:32.166+0000 7fbdcf1c7640 1 --2- 172.21.15.37:0/2732441221 >> [v2:172.21.15.203:6832/2418205237,v1:172.21.15.203:6833/2418205237] conn(0x7fbdac0744b0 0x7fbdac0769c0 unknown :-1 s=CLOSED pgs=115 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:29:32.168 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:32.166+0000 7fbdcf1c7640 1 --2- 172.21.15.37:0/2732441221 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fbdc806f990 0x7fbdc806fda0 unknown :-1 s=CLOSED pgs=191 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:29:32.168 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:32.166+0000 7fbdcf1c7640 1 --2- 172.21.15.37:0/2732441221 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7fbdc81610b0 0x7fbdc81f0d80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:29:32.168 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:32.166+0000 7fbdcf1c7640 1 --2- 172.21.15.37:0/2732441221 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fbdc81460e0 0x7fbdc80adce0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:29:32.168 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:32.166+0000 7fbdcf1c7640 1 -- 172.21.15.37:0/2732441221 >> 172.21.15.37:0/2732441221 conn(0x7fbdc80b36b0 msgr2=0x7fbdc80b3ac0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:29:32.169 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:32.166+0000 7fbdcf1c7640 1 -- 172.21.15.37:0/2732441221 shutdown_connections 2024-04-25T10:29:32.169 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:32.167+0000 7fbdcf1c7640 1 -- 172.21.15.37:0/2732441221 wait complete. 2024-04-25T10:29:32.179 INFO:tasks.cephfs.mount:Mounting default Ceph FS; just confirmed its presence on cluster 2024-04-25T10:29:32.179 INFO:tasks.cephfs.mount:Mounting Ceph FS. Following are details of mount; remember "None" represents Python type None - 2024-04-25T10:29:32.179 INFO:tasks.cephfs.mount:self.client_remote.hostname = smithi203.front.sepia.ceph.com 2024-04-25T10:29:32.179 INFO:tasks.cephfs.mount:self.client.name = client.1 2024-04-25T10:29:32.179 INFO:tasks.cephfs.mount:self.hostfs_mntpt = /home/ubuntu/cephtest/mnt.1 2024-04-25T10:29:32.179 INFO:tasks.cephfs.mount:self.cephfs_name = None 2024-04-25T10:29:32.179 INFO:tasks.cephfs.mount:self.cephfs_mntpt = None 2024-04-25T10:29:32.179 INFO:tasks.cephfs.mount:self.client_keyring_path = None 2024-04-25T10:29:32.179 INFO:tasks.cephfs.mount:Setting the 'None' netns for '/home/ubuntu/cephtest/mnt.1' 2024-04-25T10:29:32.179 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T10:29:32.179 DEBUG:teuthology.orchestra.run.smithi203:> ip addr 2024-04-25T10:29:32.198 INFO:teuthology.orchestra.run.smithi203.stdout:1: lo: mtu 65536 qdisc noqueue state UNKNOWN group default qlen 1000 2024-04-25T10:29:32.199 INFO:teuthology.orchestra.run.smithi203.stdout: link/loopback 00:00:00:00:00:00 brd 00:00:00:00:00:00 2024-04-25T10:29:32.199 INFO:teuthology.orchestra.run.smithi203.stdout: inet 127.0.0.1/8 scope host lo 2024-04-25T10:29:32.199 INFO:teuthology.orchestra.run.smithi203.stdout: valid_lft forever preferred_lft forever 2024-04-25T10:29:32.199 INFO:teuthology.orchestra.run.smithi203.stdout: inet6 ::1/128 scope host 2024-04-25T10:29:32.199 INFO:teuthology.orchestra.run.smithi203.stdout: valid_lft forever preferred_lft forever 2024-04-25T10:29:32.199 INFO:teuthology.orchestra.run.smithi203.stdout:2: eno1: mtu 1500 qdisc mq state DOWN group default qlen 1000 2024-04-25T10:29:32.199 INFO:teuthology.orchestra.run.smithi203.stdout: link/ether 0c:c4:7a:d9:19:44 brd ff:ff:ff:ff:ff:ff 2024-04-25T10:29:32.199 INFO:teuthology.orchestra.run.smithi203.stdout: altname enp7s0f0 2024-04-25T10:29:32.199 INFO:teuthology.orchestra.run.smithi203.stdout:3: eno2: mtu 1500 qdisc mq state DOWN group default qlen 1000 2024-04-25T10:29:32.199 INFO:teuthology.orchestra.run.smithi203.stdout: link/ether 0c:c4:7a:d9:19:45 brd ff:ff:ff:ff:ff:ff 2024-04-25T10:29:32.199 INFO:teuthology.orchestra.run.smithi203.stdout: altname enp7s0f1 2024-04-25T10:29:32.199 INFO:teuthology.orchestra.run.smithi203.stdout:4: enp3s0f0: mtu 1500 qdisc mq state DOWN group default qlen 1000 2024-04-25T10:29:32.199 INFO:teuthology.orchestra.run.smithi203.stdout: link/ether 0c:c4:7a:8f:4a:22 brd ff:ff:ff:ff:ff:ff 2024-04-25T10:29:32.199 INFO:teuthology.orchestra.run.smithi203.stdout:5: enp3s0f1: mtu 1500 qdisc mq state UP group default qlen 1000 2024-04-25T10:29:32.199 INFO:teuthology.orchestra.run.smithi203.stdout: link/ether 0c:c4:7a:8f:4a:23 brd ff:ff:ff:ff:ff:ff 2024-04-25T10:29:32.200 INFO:teuthology.orchestra.run.smithi203.stdout: inet 172.21.15.203/20 brd 172.21.15.255 scope global dynamic noprefixroute enp3s0f1 2024-04-25T10:29:32.200 INFO:teuthology.orchestra.run.smithi203.stdout: valid_lft 42738sec preferred_lft 42738sec 2024-04-25T10:29:32.200 INFO:teuthology.orchestra.run.smithi203.stdout: inet6 fe80::8054:ab59:b94d:aa24/64 scope link noprefixroute 2024-04-25T10:29:32.200 INFO:teuthology.orchestra.run.smithi203.stdout: valid_lft forever preferred_lft forever 2024-04-25T10:29:32.200 INFO:tasks.cephfs.mount:Setuping the 'ceph-brx' with 192.168.255.254/16 2024-04-25T10:29:32.200 DEBUG:teuthology.orchestra.run.smithi203:> (cd / && exec stdin-killer --timeout=300 -- bash -c ' 2024-04-25T10:29:32.200 DEBUG:teuthology.orchestra.run.smithi203:> set -e 2024-04-25T10:29:32.200 DEBUG:teuthology.orchestra.run.smithi203:> sudo ip link add name ceph-brx type bridge 2024-04-25T10:29:32.200 DEBUG:teuthology.orchestra.run.smithi203:> sudo ip addr flush dev ceph-brx 2024-04-25T10:29:32.200 DEBUG:teuthology.orchestra.run.smithi203:> sudo ip link set ceph-brx up 2024-04-25T10:29:32.200 DEBUG:teuthology.orchestra.run.smithi203:> sudo ip addr add 192.168.255.254/16 brd 192.168.255.255 dev ceph-brx 2024-04-25T10:29:32.201 DEBUG:teuthology.orchestra.run.smithi203:> ') 2024-04-25T10:29:32.295 INFO:teuthology.orchestra.run.smithi203.stderr:2024-04-25T10:29:32 stdin-killer INFO: expiration expected; waiting 300 seconds for command to complete 2024-04-25T10:29:32.397 INFO:teuthology.orchestra.run.smithi203.stderr:2024-04-25T10:29:32 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-04-25T10:29:32.404 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T10:29:32.404 DEBUG:teuthology.orchestra.run.smithi203:> echo 1 | sudo tee /proc/sys/net/ipv4/ip_forward 2024-04-25T10:29:32.428 INFO:teuthology.orchestra.run.smithi203.stdout:1 2024-04-25T10:29:32.429 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T10:29:32.429 DEBUG:teuthology.orchestra.run.smithi203:> route 2024-04-25T10:29:32.507 INFO:teuthology.orchestra.run.smithi203.stdout:Kernel IP routing table 2024-04-25T10:29:32.507 INFO:teuthology.orchestra.run.smithi203.stdout:Destination Gateway Genmask Flags Metric Ref Use Iface 2024-04-25T10:29:32.507 INFO:teuthology.orchestra.run.smithi203.stdout:default _gateway 0.0.0.0 UG 100 0 0 enp3s0f1 2024-04-25T10:29:32.507 INFO:teuthology.orchestra.run.smithi203.stdout:172.21.0.0 0.0.0.0 255.255.240.0 U 100 0 0 enp3s0f1 2024-04-25T10:29:32.507 INFO:teuthology.orchestra.run.smithi203.stdout:192.168.0.0 0.0.0.0 255.255.0.0 U 0 0 0 ceph-brx 2024-04-25T10:29:32.507 DEBUG:teuthology.orchestra.run.smithi203:> (cd / && exec stdin-killer --timeout=300 -- bash -c ' 2024-04-25T10:29:32.508 DEBUG:teuthology.orchestra.run.smithi203:> set -e 2024-04-25T10:29:32.508 DEBUG:teuthology.orchestra.run.smithi203:> sudo iptables -A FORWARD -o enp3s0f1 -i ceph-brx -j ACCEPT 2024-04-25T10:29:32.508 DEBUG:teuthology.orchestra.run.smithi203:> sudo iptables -A FORWARD -i enp3s0f1 -o ceph-brx -j ACCEPT 2024-04-25T10:29:32.508 DEBUG:teuthology.orchestra.run.smithi203:> sudo iptables -t nat -A POSTROUTING -s 192.168.255.254/16 -o enp3s0f1 -j MASQUERADE 2024-04-25T10:29:32.508 DEBUG:teuthology.orchestra.run.smithi203:> ') 2024-04-25T10:29:32.594 INFO:teuthology.orchestra.run.smithi203.stderr:2024-04-25T10:29:32 stdin-killer INFO: expiration expected; waiting 300 seconds for command to complete 2024-04-25T10:29:32.747 INFO:teuthology.orchestra.run.smithi203.stderr:2024-04-25T10:29:32 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-04-25T10:29:32.753 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T10:29:32.754 DEBUG:teuthology.orchestra.run.smithi203:> ip netns list 2024-04-25T10:29:32.769 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T10:29:32.769 DEBUG:teuthology.orchestra.run.smithi203:> ip netns list-id 2024-04-25T10:29:32.824 DEBUG:teuthology.orchestra.run.smithi203:> (cd / && exec stdin-killer --timeout=300 -- bash -c ' 2024-04-25T10:29:32.824 DEBUG:teuthology.orchestra.run.smithi203:> set -e 2024-04-25T10:29:32.824 DEBUG:teuthology.orchestra.run.smithi203:> sudo ip netns add ceph-ns--home-ubuntu-cephtest-mnt.1 2024-04-25T10:29:32.824 DEBUG:teuthology.orchestra.run.smithi203:> sudo ip netns set ceph-ns--home-ubuntu-cephtest-mnt.1 0 2024-04-25T10:29:32.824 DEBUG:teuthology.orchestra.run.smithi203:> ') 2024-04-25T10:29:32.912 INFO:teuthology.orchestra.run.smithi203.stderr:2024-04-25T10:29:32 stdin-killer INFO: expiration expected; waiting 300 seconds for command to complete 2024-04-25T10:29:32.933 INFO:teuthology.orchestra.run.smithi203.stderr:2024-04-25T10:29:32 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-04-25T10:29:32.940 INFO:tasks.cephfs.mount:Setuping the netns 'ceph-ns--home-ubuntu-cephtest-mnt.1' with 192.168.0.1/16 2024-04-25T10:29:32.940 DEBUG:teuthology.orchestra.run.smithi203:> (cd / && exec stdin-killer --timeout=300 -- bash -c ' 2024-04-25T10:29:32.940 DEBUG:teuthology.orchestra.run.smithi203:> set -e 2024-04-25T10:29:32.940 DEBUG:teuthology.orchestra.run.smithi203:> sudo ip link add veth0 netns ceph-ns--home-ubuntu-cephtest-mnt.1 type veth peer name brx.0 2024-04-25T10:29:32.940 DEBUG:teuthology.orchestra.run.smithi203:> sudo ip netns exec ceph-ns--home-ubuntu-cephtest-mnt.1 ip addr add 192.168.0.1/16 brd 192.168.255.255 dev veth0 2024-04-25T10:29:32.940 DEBUG:teuthology.orchestra.run.smithi203:> sudo ip netns exec ceph-ns--home-ubuntu-cephtest-mnt.1 ip link set veth0 up 2024-04-25T10:29:32.940 DEBUG:teuthology.orchestra.run.smithi203:> sudo ip netns exec ceph-ns--home-ubuntu-cephtest-mnt.1 ip link set lo up 2024-04-25T10:29:32.940 DEBUG:teuthology.orchestra.run.smithi203:> sudo ip netns exec ceph-ns--home-ubuntu-cephtest-mnt.1 ip route add default via 192.168.255.254 2024-04-25T10:29:32.940 DEBUG:teuthology.orchestra.run.smithi203:> ') 2024-04-25T10:29:32.988 INFO:teuthology.orchestra.run.smithi203.stderr:2024-04-25T10:29:32 stdin-killer INFO: expiration expected; waiting 300 seconds for command to complete 2024-04-25T10:29:33.060 INFO:teuthology.orchestra.run.smithi203.stderr:2024-04-25T10:29:33 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-04-25T10:29:33.066 DEBUG:teuthology.orchestra.run.smithi203:> (cd / && exec stdin-killer --timeout=300 -- bash -c ' 2024-04-25T10:29:33.067 DEBUG:teuthology.orchestra.run.smithi203:> set -e 2024-04-25T10:29:33.067 DEBUG:teuthology.orchestra.run.smithi203:> sudo ip link set brx.0 up 2024-04-25T10:29:33.067 DEBUG:teuthology.orchestra.run.smithi203:> sudo ip link set dev brx.0 master ceph-brx 2024-04-25T10:29:33.067 DEBUG:teuthology.orchestra.run.smithi203:> ') 2024-04-25T10:29:33.114 INFO:teuthology.orchestra.run.smithi203.stderr:2024-04-25T10:29:33 stdin-killer INFO: expiration expected; waiting 300 seconds for command to complete 2024-04-25T10:29:33.134 INFO:teuthology.orchestra.run.smithi203.stderr:2024-04-25T10:29:33 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-04-25T10:29:33.140 INFO:tasks.cephfs.fuse_mount:Client client.1 config is {} 2024-04-25T10:29:33.141 INFO:teuthology.orchestra.run:Running command with timeout 60 2024-04-25T10:29:33.141 DEBUG:teuthology.orchestra.run.smithi203:> mkdir -p -v /home/ubuntu/cephtest/mnt.1 2024-04-25T10:29:33.155 INFO:teuthology.orchestra.run.smithi203.stdout:mkdir: created directory '/home/ubuntu/cephtest/mnt.1' 2024-04-25T10:29:33.155 INFO:teuthology.orchestra.run:Running command with timeout 60 2024-04-25T10:29:33.155 DEBUG:teuthology.orchestra.run.smithi203:> chmod 0000 /home/ubuntu/cephtest/mnt.1 2024-04-25T10:29:33.206 DEBUG:teuthology.orchestra.run.smithi203:> sudo modprobe fuse 2024-04-25T10:29:33.270 DEBUG:teuthology.orchestra.run.smithi203:> cat /proc/self/mounts | awk '{print $2}' 2024-04-25T10:29:33.327 INFO:teuthology.orchestra.run.smithi203.stdout:/proc 2024-04-25T10:29:33.327 INFO:teuthology.orchestra.run.smithi203.stdout:/sys 2024-04-25T10:29:33.327 INFO:teuthology.orchestra.run.smithi203.stdout:/dev 2024-04-25T10:29:33.327 INFO:teuthology.orchestra.run.smithi203.stdout:/sys/kernel/security 2024-04-25T10:29:33.327 INFO:teuthology.orchestra.run.smithi203.stdout:/dev/shm 2024-04-25T10:29:33.327 INFO:teuthology.orchestra.run.smithi203.stdout:/dev/pts 2024-04-25T10:29:33.327 INFO:teuthology.orchestra.run.smithi203.stdout:/run 2024-04-25T10:29:33.327 INFO:teuthology.orchestra.run.smithi203.stdout:/sys/fs/cgroup 2024-04-25T10:29:33.328 INFO:teuthology.orchestra.run.smithi203.stdout:/sys/fs/pstore 2024-04-25T10:29:33.328 INFO:teuthology.orchestra.run.smithi203.stdout:/sys/fs/bpf 2024-04-25T10:29:33.328 INFO:teuthology.orchestra.run.smithi203.stdout:/ 2024-04-25T10:29:33.328 INFO:teuthology.orchestra.run.smithi203.stdout:/sys/fs/selinux 2024-04-25T10:29:33.328 INFO:teuthology.orchestra.run.smithi203.stdout:/proc/sys/fs/binfmt_misc 2024-04-25T10:29:33.328 INFO:teuthology.orchestra.run.smithi203.stdout:/dev/hugepages 2024-04-25T10:29:33.328 INFO:teuthology.orchestra.run.smithi203.stdout:/dev/mqueue 2024-04-25T10:29:33.328 INFO:teuthology.orchestra.run.smithi203.stdout:/sys/kernel/debug 2024-04-25T10:29:33.328 INFO:teuthology.orchestra.run.smithi203.stdout:/sys/kernel/tracing 2024-04-25T10:29:33.328 INFO:teuthology.orchestra.run.smithi203.stdout:/sys/kernel/config 2024-04-25T10:29:33.328 INFO:teuthology.orchestra.run.smithi203.stdout:/run/credentials/systemd-sysusers.service 2024-04-25T10:29:33.328 INFO:teuthology.orchestra.run.smithi203.stdout:/sys/fs/fuse/connections 2024-04-25T10:29:33.329 INFO:teuthology.orchestra.run.smithi203.stdout:/run/credentials/systemd-sysctl.service 2024-04-25T10:29:33.329 INFO:teuthology.orchestra.run.smithi203.stdout:/run/credentials/systemd-tmpfiles-setup-dev.service 2024-04-25T10:29:33.329 INFO:teuthology.orchestra.run.smithi203.stdout:/run/credentials/systemd-tmpfiles-setup.service 2024-04-25T10:29:33.329 INFO:teuthology.orchestra.run.smithi203.stdout:/var/lib/nfs/rpc_pipefs 2024-04-25T10:29:33.329 INFO:teuthology.orchestra.run.smithi203.stdout:/run/user/1000 2024-04-25T10:29:33.329 INFO:teuthology.orchestra.run.smithi203.stdout:/var/lib/ceph 2024-04-25T10:29:33.329 INFO:teuthology.orchestra.run.smithi203.stdout:/var/lib/ceph/osd/ceph-4 2024-04-25T10:29:33.329 INFO:teuthology.orchestra.run.smithi203.stdout:/var/lib/ceph/osd/ceph-5 2024-04-25T10:29:33.329 INFO:teuthology.orchestra.run.smithi203.stdout:/var/lib/ceph/osd/ceph-6 2024-04-25T10:29:33.329 INFO:teuthology.orchestra.run.smithi203.stdout:/var/lib/ceph/osd/ceph-7 2024-04-25T10:29:33.329 INFO:teuthology.orchestra.run.smithi203.stdout:/run/netns 2024-04-25T10:29:33.330 INFO:teuthology.orchestra.run.smithi203.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 2024-04-25T10:29:33.330 INFO:teuthology.orchestra.run.smithi203.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 2024-04-25T10:29:33.330 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T10:29:33.330 DEBUG:teuthology.orchestra.run.smithi203:> ls /sys/fs/fuse/connections 2024-04-25T10:29:33.382 INFO:tasks.cephfs.fuse_mount:Pre-mount connections: [] 2024-04-25T10:29:33.382 DEBUG:teuthology.orchestra.run.smithi203:> (cd /home/ubuntu/cephtest && exec sudo nsenter --net=/var/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-fuse -f --admin-socket '/var/run/ceph/$cluster-$name.$pid.asok' /home/ubuntu/cephtest/mnt.1 --id 1) 2024-04-25T10:29:33.424 DEBUG:teuthology.orchestra.run.smithi203:> sudo modprobe fuse 2024-04-25T10:29:33.448 DEBUG:teuthology.orchestra.run.smithi203:> cat /proc/self/mounts | awk '{print $2}' 2024-04-25T10:29:33.503 INFO:teuthology.orchestra.run.smithi203.stdout:/proc 2024-04-25T10:29:33.504 INFO:teuthology.orchestra.run.smithi203.stdout:/sys 2024-04-25T10:29:33.504 INFO:teuthology.orchestra.run.smithi203.stdout:/dev 2024-04-25T10:29:33.504 INFO:teuthology.orchestra.run.smithi203.stdout:/sys/kernel/security 2024-04-25T10:29:33.504 INFO:teuthology.orchestra.run.smithi203.stdout:/dev/shm 2024-04-25T10:29:33.504 INFO:teuthology.orchestra.run.smithi203.stdout:/dev/pts 2024-04-25T10:29:33.504 INFO:teuthology.orchestra.run.smithi203.stdout:/run 2024-04-25T10:29:33.504 INFO:teuthology.orchestra.run.smithi203.stdout:/sys/fs/cgroup 2024-04-25T10:29:33.504 INFO:teuthology.orchestra.run.smithi203.stdout:/sys/fs/pstore 2024-04-25T10:29:33.504 INFO:teuthology.orchestra.run.smithi203.stdout:/sys/fs/bpf 2024-04-25T10:29:33.504 INFO:teuthology.orchestra.run.smithi203.stdout:/ 2024-04-25T10:29:33.504 INFO:teuthology.orchestra.run.smithi203.stdout:/sys/fs/selinux 2024-04-25T10:29:33.504 INFO:teuthology.orchestra.run.smithi203.stdout:/proc/sys/fs/binfmt_misc 2024-04-25T10:29:33.504 INFO:teuthology.orchestra.run.smithi203.stdout:/dev/hugepages 2024-04-25T10:29:33.505 INFO:teuthology.orchestra.run.smithi203.stdout:/dev/mqueue 2024-04-25T10:29:33.505 INFO:teuthology.orchestra.run.smithi203.stdout:/sys/kernel/debug 2024-04-25T10:29:33.505 INFO:teuthology.orchestra.run.smithi203.stdout:/sys/kernel/tracing 2024-04-25T10:29:33.505 INFO:teuthology.orchestra.run.smithi203.stdout:/sys/kernel/config 2024-04-25T10:29:33.505 INFO:teuthology.orchestra.run.smithi203.stdout:/run/credentials/systemd-sysusers.service 2024-04-25T10:29:33.505 INFO:teuthology.orchestra.run.smithi203.stdout:/sys/fs/fuse/connections 2024-04-25T10:29:33.505 INFO:teuthology.orchestra.run.smithi203.stdout:/run/credentials/systemd-sysctl.service 2024-04-25T10:29:33.505 INFO:teuthology.orchestra.run.smithi203.stdout:/run/credentials/systemd-tmpfiles-setup-dev.service 2024-04-25T10:29:33.505 INFO:teuthology.orchestra.run.smithi203.stdout:/run/credentials/systemd-tmpfiles-setup.service 2024-04-25T10:29:33.505 INFO:teuthology.orchestra.run.smithi203.stdout:/var/lib/nfs/rpc_pipefs 2024-04-25T10:29:33.505 INFO:teuthology.orchestra.run.smithi203.stdout:/run/user/1000 2024-04-25T10:29:33.505 INFO:teuthology.orchestra.run.smithi203.stdout:/var/lib/ceph 2024-04-25T10:29:33.505 INFO:teuthology.orchestra.run.smithi203.stdout:/var/lib/ceph/osd/ceph-4 2024-04-25T10:29:33.505 INFO:teuthology.orchestra.run.smithi203.stdout:/var/lib/ceph/osd/ceph-5 2024-04-25T10:29:33.505 INFO:teuthology.orchestra.run.smithi203.stdout:/var/lib/ceph/osd/ceph-6 2024-04-25T10:29:33.506 INFO:teuthology.orchestra.run.smithi203.stdout:/var/lib/ceph/osd/ceph-7 2024-04-25T10:29:33.506 INFO:teuthology.orchestra.run.smithi203.stdout:/run/netns 2024-04-25T10:29:33.506 INFO:teuthology.orchestra.run.smithi203.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 2024-04-25T10:29:33.506 INFO:teuthology.orchestra.run.smithi203.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 2024-04-25T10:29:33.506 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T10:29:33.506 DEBUG:teuthology.orchestra.run.smithi203:> ls /sys/fs/fuse/connections 2024-04-25T10:29:33.527 INFO:tasks.cephfs.fuse_mount.ceph-fuse.1.smithi203.stderr:2024-04-25T10:29:33.525+0000 7fd1b43e5480 -1 init, newargv = 0x7fd1a400a100 newargc=15 2024-04-25T10:29:33.527 INFO:tasks.cephfs.fuse_mount.ceph-fuse.1.smithi203.stderr:ceph-fuse[40791]: starting ceph client 2024-04-25T10:29:33.532 INFO:tasks.cephfs.fuse_mount.ceph-fuse.1.smithi203.stderr:ceph-fuse[40791]: starting fuse 2024-04-25T10:29:33.542 INFO:teuthology.orchestra.run.smithi203.stdout:43 2024-04-25T10:29:33.542 INFO:tasks.cephfs.fuse_mount:Post-mount connections: [43] 2024-04-25T10:29:33.543 DEBUG:teuthology.orchestra.run.smithi203:> sudo stdin-killer -- python3 -c ' 2024-04-25T10:29:33.543 DEBUG:teuthology.orchestra.run.smithi203:> import glob 2024-04-25T10:29:33.543 DEBUG:teuthology.orchestra.run.smithi203:> import re 2024-04-25T10:29:33.543 DEBUG:teuthology.orchestra.run.smithi203:> import os 2024-04-25T10:29:33.543 DEBUG:teuthology.orchestra.run.smithi203:> import subprocess 2024-04-25T10:29:33.543 DEBUG:teuthology.orchestra.run.smithi203:> 2024-04-25T10:29:33.543 DEBUG:teuthology.orchestra.run.smithi203:> def _find_admin_socket(client_name): 2024-04-25T10:29:33.543 DEBUG:teuthology.orchestra.run.smithi203:> asok_path = "/var/run/ceph/ceph-client.1.*.asok" 2024-04-25T10:29:33.543 DEBUG:teuthology.orchestra.run.smithi203:> files = glob.glob(asok_path) 2024-04-25T10:29:33.544 DEBUG:teuthology.orchestra.run.smithi203:> mountpoint = "/home/ubuntu/cephtest/mnt.1" 2024-04-25T10:29:33.544 DEBUG:teuthology.orchestra.run.smithi203:> 2024-04-25T10:29:33.544 DEBUG:teuthology.orchestra.run.smithi203:> # Given a non-glob path, it better be there 2024-04-25T10:29:33.544 DEBUG:teuthology.orchestra.run.smithi203:> if "*" not in asok_path: 2024-04-25T10:29:33.544 DEBUG:teuthology.orchestra.run.smithi203:> assert(len(files) == 1) 2024-04-25T10:29:33.544 DEBUG:teuthology.orchestra.run.smithi203:> return files[0] 2024-04-25T10:29:33.544 DEBUG:teuthology.orchestra.run.smithi203:> 2024-04-25T10:29:33.544 DEBUG:teuthology.orchestra.run.smithi203:> for f in files: 2024-04-25T10:29:33.544 DEBUG:teuthology.orchestra.run.smithi203:> pid = re.match(".*\.(\d+)\.asok$", f).group(1) 2024-04-25T10:29:33.544 DEBUG:teuthology.orchestra.run.smithi203:> if os.path.exists("/proc/{0}".format(pid)): 2024-04-25T10:29:33.544 DEBUG:teuthology.orchestra.run.smithi203:> with open("/proc/{0}/cmdline".format(pid), '"'"'r'"'"') as proc_f: 2024-04-25T10:29:33.544 DEBUG:teuthology.orchestra.run.smithi203:> contents = proc_f.read() 2024-04-25T10:29:33.544 DEBUG:teuthology.orchestra.run.smithi203:> if mountpoint in contents: 2024-04-25T10:29:33.544 DEBUG:teuthology.orchestra.run.smithi203:> return f 2024-04-25T10:29:33.544 DEBUG:teuthology.orchestra.run.smithi203:> raise RuntimeError("Client socket {0} not found".format(client_name)) 2024-04-25T10:29:33.544 DEBUG:teuthology.orchestra.run.smithi203:> 2024-04-25T10:29:33.545 DEBUG:teuthology.orchestra.run.smithi203:> print(_find_admin_socket("client.1")) 2024-04-25T10:29:33.545 DEBUG:teuthology.orchestra.run.smithi203:> ' 2024-04-25T10:29:33.658 INFO:teuthology.orchestra.run.smithi203.stdout:/var/run/ceph/ceph-client.1.40791.asok 2024-04-25T10:29:33.661 INFO:teuthology.orchestra.run.smithi203.stderr:2024-04-25T10:29:33 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-04-25T10:29:33.668 INFO:tasks.cephfs.fuse_mount:Found client admin socket at /var/run/ceph/ceph-client.1.40791.asok 2024-04-25T10:29:33.669 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T10:29:33.669 DEBUG:teuthology.orchestra.run.smithi203:> sudo ceph --admin-daemon /var/run/ceph/ceph-client.1.40791.asok status 2024-04-25T10:29:33.810 INFO:teuthology.orchestra.run.smithi203.stdout:{ 2024-04-25T10:29:33.810 INFO:teuthology.orchestra.run.smithi203.stdout: "metadata": { 2024-04-25T10:29:33.811 INFO:teuthology.orchestra.run.smithi203.stdout: "ceph_sha1": "994e2f20102a20fa9a27966c14b625cc01884df8", 2024-04-25T10:29:33.811 INFO:teuthology.orchestra.run.smithi203.stdout: "ceph_version": "ceph version 19.0.0-3291-g994e2f20 (994e2f20102a20fa9a27966c14b625cc01884df8) squid (dev)", 2024-04-25T10:29:33.811 INFO:teuthology.orchestra.run.smithi203.stdout: "entity_id": "1", 2024-04-25T10:29:33.811 INFO:teuthology.orchestra.run.smithi203.stdout: "hostname": "smithi203", 2024-04-25T10:29:33.811 INFO:teuthology.orchestra.run.smithi203.stdout: "mount_point": "/home/ubuntu/cephtest/mnt.1", 2024-04-25T10:29:33.811 INFO:teuthology.orchestra.run.smithi203.stdout: "pid": "40791", 2024-04-25T10:29:33.811 INFO:teuthology.orchestra.run.smithi203.stdout: "root": "/" 2024-04-25T10:29:33.811 INFO:teuthology.orchestra.run.smithi203.stdout: }, 2024-04-25T10:29:33.811 INFO:teuthology.orchestra.run.smithi203.stdout: "dentry_count": 0, 2024-04-25T10:29:33.811 INFO:teuthology.orchestra.run.smithi203.stdout: "dentry_pinned_count": 0, 2024-04-25T10:29:33.811 INFO:teuthology.orchestra.run.smithi203.stdout: "id": 4514, 2024-04-25T10:29:33.811 INFO:teuthology.orchestra.run.smithi203.stdout: "inst": { 2024-04-25T10:29:33.811 INFO:teuthology.orchestra.run.smithi203.stdout: "name": { 2024-04-25T10:29:33.812 INFO:teuthology.orchestra.run.smithi203.stdout: "type": "client", 2024-04-25T10:29:33.812 INFO:teuthology.orchestra.run.smithi203.stdout: "num": 4514 2024-04-25T10:29:33.812 INFO:teuthology.orchestra.run.smithi203.stdout: }, 2024-04-25T10:29:33.812 INFO:teuthology.orchestra.run.smithi203.stdout: "addr": { 2024-04-25T10:29:33.812 INFO:teuthology.orchestra.run.smithi203.stdout: "type": "v1", 2024-04-25T10:29:33.812 INFO:teuthology.orchestra.run.smithi203.stdout: "addr": "192.168.0.1:0", 2024-04-25T10:29:33.812 INFO:teuthology.orchestra.run.smithi203.stdout: "nonce": 4134133139 2024-04-25T10:29:33.812 INFO:teuthology.orchestra.run.smithi203.stdout: } 2024-04-25T10:29:33.812 INFO:teuthology.orchestra.run.smithi203.stdout: }, 2024-04-25T10:29:33.812 INFO:teuthology.orchestra.run.smithi203.stdout: "addr": { 2024-04-25T10:29:33.812 INFO:teuthology.orchestra.run.smithi203.stdout: "type": "v1", 2024-04-25T10:29:33.812 INFO:teuthology.orchestra.run.smithi203.stdout: "addr": "192.168.0.1:0", 2024-04-25T10:29:33.812 INFO:teuthology.orchestra.run.smithi203.stdout: "nonce": 4134133139 2024-04-25T10:29:33.812 INFO:teuthology.orchestra.run.smithi203.stdout: }, 2024-04-25T10:29:33.813 INFO:teuthology.orchestra.run.smithi203.stdout: "inst_str": "client.4514 192.168.0.1:0/4134133139", 2024-04-25T10:29:33.813 INFO:teuthology.orchestra.run.smithi203.stdout: "addr_str": "192.168.0.1:0/4134133139", 2024-04-25T10:29:33.813 INFO:teuthology.orchestra.run.smithi203.stdout: "inode_count": 1, 2024-04-25T10:29:33.813 INFO:teuthology.orchestra.run.smithi203.stdout: "mds_epoch": 8, 2024-04-25T10:29:33.813 INFO:teuthology.orchestra.run.smithi203.stdout: "osd_epoch": 23, 2024-04-25T10:29:33.813 INFO:teuthology.orchestra.run.smithi203.stdout: "osd_epoch_barrier": 0, 2024-04-25T10:29:33.813 INFO:teuthology.orchestra.run.smithi203.stdout: "blocklisted": false, 2024-04-25T10:29:33.813 INFO:teuthology.orchestra.run.smithi203.stdout: "fs_name": "cephfs" 2024-04-25T10:29:33.813 INFO:teuthology.orchestra.run.smithi203.stdout:} 2024-04-25T10:29:33.819 DEBUG:teuthology.orchestra.run.smithi037:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs ls 2024-04-25T10:29:33.946 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:33.943+0000 7f42f135f640 1 Processor -- start 2024-04-25T10:29:33.946 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:33.943+0000 7f42f135f640 1 -- start start 2024-04-25T10:29:33.948 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:33.945+0000 7f42f135f640 1 --2- >> v2:172.21.15.203:3300/0 conn(0x7f42ec0d5160 0x7f42ec0b4e80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:29:33.948 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:33.945+0000 7f42f135f640 1 --2- >> v2:172.21.15.37:3300/0 conn(0x7f42ec0d0990 0x7f42ec0d2b20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:29:33.948 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:33.945+0000 7f42f135f640 1 -- --> v1:172.21.15.203:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f42ec0d1370 con 0x7f42ec0d0da0 2024-04-25T10:29:33.948 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:33.945+0000 7f42f135f640 1 -- --> v2:172.21.15.37:3300/0 -- mon_getmap magic: 0 -- 0x7f42ec0d1650 con 0x7f42ec0d0990 2024-04-25T10:29:33.948 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:33.945+0000 7f42f135f640 1 -- --> v2:172.21.15.203:3300/0 -- mon_getmap magic: 0 -- 0x7f42ec0d4680 con 0x7f42ec0d5160 2024-04-25T10:29:33.948 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:33.945+0000 7f42ea7fc640 1 --2- >> v2:172.21.15.37:3300/0 conn(0x7f42ec0d0990 0x7f42ec0d2b20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:29:33.948 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:33.945+0000 7f42ea7fc640 1 --2- >> v2:172.21.15.37:3300/0 conn(0x7f42ec0d0990 0x7f42ec0d2b20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.37:3300/0 says I am v2:172.21.15.37:36764/0 (socket says 172.21.15.37:36764) 2024-04-25T10:29:33.948 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:33.945+0000 7f42ea7fc640 1 -- 172.21.15.37:0/504145158 learned_addr learned my addr 172.21.15.37:0/504145158 (peer_addr_for_me v2:172.21.15.37:0/0) 2024-04-25T10:29:33.948 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:33.945+0000 7f42eaffd640 1 --2- 172.21.15.37:0/504145158 >> v2:172.21.15.203:3300/0 conn(0x7f42ec0d5160 0x7f42ec0b4e80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:29:33.948 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:33.945+0000 7f42ea7fc640 1 -- 172.21.15.37:0/504145158 >> v2:172.21.15.203:3300/0 conn(0x7f42ec0d5160 msgr2=0x7f42ec0b4e80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:29:33.948 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:33.945+0000 7f42ea7fc640 1 --2- 172.21.15.37:0/504145158 >> v2:172.21.15.203:3300/0 conn(0x7f42ec0d5160 0x7f42ec0b4e80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:29:33.948 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:33.945+0000 7f42ea7fc640 1 -- 172.21.15.37:0/504145158 >> v1:172.21.15.203:6789/0 conn(0x7f42ec0d0da0 legacy=0x7f42ec06db10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:29:33.948 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:33.945+0000 7f42ea7fc640 1 -- 172.21.15.37:0/504145158 --> v2:172.21.15.37:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f42ec0d4960 con 0x7f42ec0d0990 2024-04-25T10:29:33.949 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:33.945+0000 7f42eaffd640 1 --2- 172.21.15.37:0/504145158 >> v2:172.21.15.203:3300/0 conn(0x7f42ec0d5160 0x7f42ec0b4e80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T10:29:33.950 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:33.946+0000 7f42ea7fc640 1 --2- 172.21.15.37:0/504145158 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f42ec0d0990 0x7f42ec0d2b20 secure :-1 s=READY pgs=192 cs=0 l=1 rev1=1 crypto rx=0x7f42ec0d6b80 tx=0x7f42e002a0d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=fa0b8de4b0f705d1 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:29:33.950 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:33.946+0000 7f42e9ffb640 1 -- 172.21.15.37:0/504145158 <== mon.0 v2:172.21.15.37:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f42e0037070 con 0x7f42ec0d0990 2024-04-25T10:29:33.950 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:33.946+0000 7f42e9ffb640 1 -- 172.21.15.37:0/504145158 <== mon.0 v2:172.21.15.37:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f42e002ade0 con 0x7f42ec0d0990 2024-04-25T10:29:33.950 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:33.946+0000 7f42e9ffb640 1 -- 172.21.15.37:0/504145158 <== mon.0 v2:172.21.15.37:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f42e0032660 con 0x7f42ec0d0990 2024-04-25T10:29:33.950 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:33.947+0000 7f42f135f640 1 -- 172.21.15.37:0/504145158 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f42ec0d0990 msgr2=0x7f42ec0d2b20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:29:33.950 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:33.947+0000 7f42f135f640 1 --2- 172.21.15.37:0/504145158 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f42ec0d0990 0x7f42ec0d2b20 secure :-1 s=READY pgs=192 cs=0 l=1 rev1=1 crypto rx=0x7f42ec0d6b80 tx=0x7f42e002a0d0 comp rx=0 tx=0).stop 2024-04-25T10:29:33.950 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:33.947+0000 7f42f135f640 1 -- 172.21.15.37:0/504145158 shutdown_connections 2024-04-25T10:29:33.950 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:33.947+0000 7f42f135f640 1 --2- 172.21.15.37:0/504145158 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f42ec0d0990 0x7f42ec0d2b20 unknown :-1 s=CLOSED pgs=192 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:29:33.951 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:33.947+0000 7f42f135f640 1 --2- 172.21.15.37:0/504145158 >> v2:172.21.15.203:3300/0 conn(0x7f42ec0d5160 0x7f42ec0b4e80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:29:33.951 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:33.947+0000 7f42f135f640 1 -- 172.21.15.37:0/504145158 >> 172.21.15.37:0/504145158 conn(0x7f42ec0b7a80 msgr2=0x7f42ec0b7e90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:29:33.951 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:33.947+0000 7f42f135f640 1 -- 172.21.15.37:0/504145158 shutdown_connections 2024-04-25T10:29:33.951 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:33.947+0000 7f42f135f640 1 -- 172.21.15.37:0/504145158 wait complete. 2024-04-25T10:29:33.951 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:33.949+0000 7f42f135f640 1 Processor -- start 2024-04-25T10:29:33.951 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:33.949+0000 7f42f135f640 1 -- start start 2024-04-25T10:29:33.954 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:33.950+0000 7f42f135f640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f42ec0d0990 0x7f42ec0cc240 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:29:33.954 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:33.950+0000 7f42f135f640 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f42ec0d0da0 0x7f42ec146370 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:29:33.954 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:33.950+0000 7f42f135f640 1 --2- >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f42ec0d5160 0x7f42ec162500 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:29:33.954 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:33.950+0000 7f42f135f640 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 -- 0x7f42ec164c40 con 0x7f42ec0d0990 2024-04-25T10:29:33.954 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:33.950+0000 7f42f135f640 1 -- --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_getmap magic: 0 -- 0x7f42ec164ac0 con 0x7f42ec0d0da0 2024-04-25T10:29:33.954 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:33.950+0000 7f42f135f640 1 -- --> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] -- mon_getmap magic: 0 -- 0x7f42ec0d3060 con 0x7f42ec0d5160 2024-04-25T10:29:33.954 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:33.950+0000 7f42eaffd640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f42ec0d0990 0x7f42ec0cc240 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:29:33.954 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:33.950+0000 7f42eaffd640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f42ec0d0990 0x7f42ec0cc240 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.37:3300/0 says I am v2:172.21.15.37:36776/0 (socket says 172.21.15.37:36776) 2024-04-25T10:29:33.954 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:33.950+0000 7f42eaffd640 1 -- 172.21.15.37:0/2693843611 learned_addr learned my addr 172.21.15.37:0/2693843611 (peer_addr_for_me v2:172.21.15.37:0/0) 2024-04-25T10:29:33.954 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:33.951+0000 7f42eb7fe640 1 --2- 172.21.15.37:0/2693843611 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f42ec0d5160 0x7f42ec162500 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:29:33.954 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:33.951+0000 7f42ea7fc640 1 --2- 172.21.15.37:0/2693843611 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f42ec0d0da0 0x7f42ec146370 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:29:33.954 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:33.951+0000 7f42eaffd640 1 -- 172.21.15.37:0/2693843611 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f42ec0d5160 msgr2=0x7f42ec162500 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:29:33.954 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:33.951+0000 7f42eaffd640 1 --2- 172.21.15.37:0/2693843611 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f42ec0d5160 0x7f42ec162500 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:29:33.955 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:33.951+0000 7f42eaffd640 1 -- 172.21.15.37:0/2693843611 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f42ec0d0da0 msgr2=0x7f42ec146370 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:29:33.955 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:33.951+0000 7f42eaffd640 1 --2- 172.21.15.37:0/2693843611 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f42ec0d0da0 0x7f42ec146370 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:29:33.955 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:33.951+0000 7f42eaffd640 1 -- 172.21.15.37:0/2693843611 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f42ec1468b0 con 0x7f42ec0d0990 2024-04-25T10:29:33.955 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:33.951+0000 7f42eaffd640 1 --2- 172.21.15.37:0/2693843611 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f42ec0d0990 0x7f42ec0cc240 secure :-1 s=READY pgs=193 cs=0 l=1 rev1=1 crypto rx=0x7f42dc00ce30 tx=0x7f42dc00a650 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:29:33.955 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:33.951+0000 7f42d37fe640 1 -- 172.21.15.37:0/2693843611 <== mon.0 v2:172.21.15.37:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f42dc00f890 con 0x7f42ec0d0990 2024-04-25T10:29:33.955 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:33.951+0000 7f42d37fe640 1 -- 172.21.15.37:0/2693843611 <== mon.0 v2:172.21.15.37:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f42dc00fa30 con 0x7f42ec0d0990 2024-04-25T10:29:33.955 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:33.951+0000 7f42f135f640 1 -- 172.21.15.37:0/2693843611 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f42ec155e80 con 0x7f42ec0d0990 2024-04-25T10:29:33.955 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:33.951+0000 7f42d37fe640 1 -- 172.21.15.37:0/2693843611 <== mon.0 v2:172.21.15.37:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f42dc00fe00 con 0x7f42ec0d0990 2024-04-25T10:29:33.955 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:33.951+0000 7f42f135f640 1 -- 172.21.15.37:0/2693843611 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f42ec156440 con 0x7f42ec0d0990 2024-04-25T10:29:33.956 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:33.952+0000 7f42f135f640 1 -- 172.21.15.37:0/2693843611 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f42ec06a140 con 0x7f42ec0d0990 2024-04-25T10:29:33.956 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:33.953+0000 7f42d37fe640 1 -- 172.21.15.37:0/2693843611 <== mon.0 v2:172.21.15.37:3300/0 4 ==== mgrmap(e 5) ==== 96782+0+0 (secure 0 0 0) 0x7f42dc018400 con 0x7f42ec0d0990 2024-04-25T10:29:33.956 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:33.954+0000 7f42d37fe640 1 --2- 172.21.15.37:0/2693843611 >> [v2:172.21.15.203:6832/2418205237,v1:172.21.15.203:6833/2418205237] conn(0x7f42d80744b0 0x7f42d80769c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:29:33.956 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:33.954+0000 7f42d37fe640 1 -- 172.21.15.37:0/2693843611 <== mon.0 v2:172.21.15.37:3300/0 5 ==== osd_map(23..23 src has 1..23) ==== 5709+0+0 (secure 0 0 0) 0x7f42dc095db0 con 0x7f42ec0d0990 2024-04-25T10:29:33.956 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:33.954+0000 7f42ea7fc640 1 --2- 172.21.15.37:0/2693843611 >> [v2:172.21.15.203:6832/2418205237,v1:172.21.15.203:6833/2418205237] conn(0x7f42d80744b0 0x7f42d80769c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:29:33.957 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:33.954+0000 7f42ea7fc640 1 --2- 172.21.15.37:0/2693843611 >> [v2:172.21.15.203:6832/2418205237,v1:172.21.15.203:6833/2418205237] conn(0x7f42d80744b0 0x7f42d80769c0 secure :-1 s=READY pgs=116 cs=0 l=1 rev1=1 crypto rx=0x7f42e0034000 tx=0x7f42e002e000 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:29:33.961 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:33.959+0000 7f42d37fe640 1 -- 172.21.15.37:0/2693843611 <== mon.0 v2:172.21.15.37:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187246 (secure 0 0 0) 0x7f42dc014030 con 0x7f42ec0d0990 2024-04-25T10:29:34.186 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:34.184+0000 7f42f135f640 1 -- 172.21.15.37:0/2693843611 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_command({"prefix": "fs ls"} v 0) -- 0x7f42ec0d3f50 con 0x7f42ec0d0990 2024-04-25T10:29:34.186 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:34.184+0000 7f42d37fe640 1 -- 172.21.15.37:0/2693843611 <== mon.0 v2:172.21.15.37:3300/0 7 ==== mon_command_ack([{"prefix": "fs ls"}]=0 v8) ==== 53+0+73 (secure 0 0 0) 0x7f42dc00ae20 con 0x7f42ec0d0990 2024-04-25T10:29:34.187 INFO:teuthology.orchestra.run.smithi037.stdout:name: cephfs, metadata pool: cephfs_metadata, data pools: [cephfs_data ] 2024-04-25T10:29:34.188 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:34.186+0000 7f42f135f640 1 -- 172.21.15.37:0/2693843611 >> [v2:172.21.15.203:6832/2418205237,v1:172.21.15.203:6833/2418205237] conn(0x7f42d80744b0 msgr2=0x7f42d80769c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:29:34.189 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:34.186+0000 7f42f135f640 1 --2- 172.21.15.37:0/2693843611 >> [v2:172.21.15.203:6832/2418205237,v1:172.21.15.203:6833/2418205237] conn(0x7f42d80744b0 0x7f42d80769c0 secure :-1 s=READY pgs=116 cs=0 l=1 rev1=1 crypto rx=0x7f42e0034000 tx=0x7f42e002e000 comp rx=0 tx=0).stop 2024-04-25T10:29:34.189 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:34.186+0000 7f42f135f640 1 -- 172.21.15.37:0/2693843611 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f42ec0d0990 msgr2=0x7f42ec0cc240 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:29:34.189 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:34.186+0000 7f42f135f640 1 --2- 172.21.15.37:0/2693843611 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f42ec0d0990 0x7f42ec0cc240 secure :-1 s=READY pgs=193 cs=0 l=1 rev1=1 crypto rx=0x7f42dc00ce30 tx=0x7f42dc00a650 comp rx=0 tx=0).stop 2024-04-25T10:29:34.189 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:34.186+0000 7f42f135f640 1 -- 172.21.15.37:0/2693843611 shutdown_connections 2024-04-25T10:29:34.189 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:34.186+0000 7f42f135f640 1 --2- 172.21.15.37:0/2693843611 >> [v2:172.21.15.203:6832/2418205237,v1:172.21.15.203:6833/2418205237] conn(0x7f42d80744b0 0x7f42d80769c0 unknown :-1 s=CLOSED pgs=116 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:29:34.189 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:34.186+0000 7f42f135f640 1 --2- 172.21.15.37:0/2693843611 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f42ec0d5160 0x7f42ec162500 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:29:34.189 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:34.186+0000 7f42f135f640 1 --2- 172.21.15.37:0/2693843611 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f42ec0d0da0 0x7f42ec146370 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:29:34.189 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:34.186+0000 7f42f135f640 1 --2- 172.21.15.37:0/2693843611 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f42ec0d0990 0x7f42ec0cc240 unknown :-1 s=CLOSED pgs=193 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:29:34.189 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:34.186+0000 7f42f135f640 1 -- 172.21.15.37:0/2693843611 >> 172.21.15.37:0/2693843611 conn(0x7f42ec0b7a80 msgr2=0x7f42ec0b5800 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:29:34.189 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:34.186+0000 7f42f135f640 1 -- 172.21.15.37:0/2693843611 shutdown_connections 2024-04-25T10:29:34.189 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:34.186+0000 7f42f135f640 1 -- 172.21.15.37:0/2693843611 wait complete. 2024-04-25T10:29:34.200 INFO:tasks.cephfs.mount:Mounting default Ceph FS; just confirmed its presence on cluster 2024-04-25T10:29:34.200 INFO:tasks.cephfs.mount:Mounting Ceph FS. Following are details of mount; remember "None" represents Python type None - 2024-04-25T10:29:34.200 INFO:tasks.cephfs.mount:self.client_remote.hostname = smithi203.front.sepia.ceph.com 2024-04-25T10:29:34.200 INFO:tasks.cephfs.mount:self.client.name = client.3 2024-04-25T10:29:34.200 INFO:tasks.cephfs.mount:self.hostfs_mntpt = /home/ubuntu/cephtest/mnt.3 2024-04-25T10:29:34.200 INFO:tasks.cephfs.mount:self.cephfs_name = None 2024-04-25T10:29:34.200 INFO:tasks.cephfs.mount:self.cephfs_mntpt = None 2024-04-25T10:29:34.200 INFO:tasks.cephfs.mount:self.client_keyring_path = None 2024-04-25T10:29:34.200 INFO:tasks.cephfs.mount:Setting the 'None' netns for '/home/ubuntu/cephtest/mnt.3' 2024-04-25T10:29:34.200 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T10:29:34.201 DEBUG:teuthology.orchestra.run.smithi203:> ip addr 2024-04-25T10:29:34.216 INFO:teuthology.orchestra.run.smithi203.stdout:1: lo: mtu 65536 qdisc noqueue state UNKNOWN group default qlen 1000 2024-04-25T10:29:34.216 INFO:teuthology.orchestra.run.smithi203.stdout: link/loopback 00:00:00:00:00:00 brd 00:00:00:00:00:00 2024-04-25T10:29:34.217 INFO:teuthology.orchestra.run.smithi203.stdout: inet 127.0.0.1/8 scope host lo 2024-04-25T10:29:34.217 INFO:teuthology.orchestra.run.smithi203.stdout: valid_lft forever preferred_lft forever 2024-04-25T10:29:34.217 INFO:teuthology.orchestra.run.smithi203.stdout: inet6 ::1/128 scope host 2024-04-25T10:29:34.217 INFO:teuthology.orchestra.run.smithi203.stdout: valid_lft forever preferred_lft forever 2024-04-25T10:29:34.217 INFO:teuthology.orchestra.run.smithi203.stdout:2: eno1: mtu 1500 qdisc mq state DOWN group default qlen 1000 2024-04-25T10:29:34.217 INFO:teuthology.orchestra.run.smithi203.stdout: link/ether 0c:c4:7a:d9:19:44 brd ff:ff:ff:ff:ff:ff 2024-04-25T10:29:34.217 INFO:teuthology.orchestra.run.smithi203.stdout: altname enp7s0f0 2024-04-25T10:29:34.217 INFO:teuthology.orchestra.run.smithi203.stdout:3: eno2: mtu 1500 qdisc mq state DOWN group default qlen 1000 2024-04-25T10:29:34.217 INFO:teuthology.orchestra.run.smithi203.stdout: link/ether 0c:c4:7a:d9:19:45 brd ff:ff:ff:ff:ff:ff 2024-04-25T10:29:34.217 INFO:teuthology.orchestra.run.smithi203.stdout: altname enp7s0f1 2024-04-25T10:29:34.217 INFO:teuthology.orchestra.run.smithi203.stdout:4: enp3s0f0: mtu 1500 qdisc mq state DOWN group default qlen 1000 2024-04-25T10:29:34.217 INFO:teuthology.orchestra.run.smithi203.stdout: link/ether 0c:c4:7a:8f:4a:22 brd ff:ff:ff:ff:ff:ff 2024-04-25T10:29:34.218 INFO:teuthology.orchestra.run.smithi203.stdout:5: enp3s0f1: mtu 1500 qdisc mq state UP group default qlen 1000 2024-04-25T10:29:34.218 INFO:teuthology.orchestra.run.smithi203.stdout: link/ether 0c:c4:7a:8f:4a:23 brd ff:ff:ff:ff:ff:ff 2024-04-25T10:29:34.218 INFO:teuthology.orchestra.run.smithi203.stdout: inet 172.21.15.203/20 brd 172.21.15.255 scope global dynamic noprefixroute enp3s0f1 2024-04-25T10:29:34.218 INFO:teuthology.orchestra.run.smithi203.stdout: valid_lft 42736sec preferred_lft 42736sec 2024-04-25T10:29:34.218 INFO:teuthology.orchestra.run.smithi203.stdout: inet6 fe80::8054:ab59:b94d:aa24/64 scope link noprefixroute 2024-04-25T10:29:34.218 INFO:teuthology.orchestra.run.smithi203.stdout: valid_lft forever preferred_lft forever 2024-04-25T10:29:34.218 INFO:teuthology.orchestra.run.smithi203.stdout:6: ceph-brx: mtu 1500 qdisc noqueue state UP group default qlen 1000 2024-04-25T10:29:34.218 INFO:teuthology.orchestra.run.smithi203.stdout: link/ether aa:26:9e:0a:99:78 brd ff:ff:ff:ff:ff:ff 2024-04-25T10:29:34.218 INFO:teuthology.orchestra.run.smithi203.stdout: inet 192.168.255.254/16 brd 192.168.255.255 scope global ceph-brx 2024-04-25T10:29:34.218 INFO:teuthology.orchestra.run.smithi203.stdout: valid_lft forever preferred_lft forever 2024-04-25T10:29:34.218 INFO:teuthology.orchestra.run.smithi203.stdout: inet6 fe80::4469:22ff:fec1:4973/64 scope link tentative 2024-04-25T10:29:34.218 INFO:teuthology.orchestra.run.smithi203.stdout: valid_lft forever preferred_lft forever 2024-04-25T10:29:34.218 INFO:teuthology.orchestra.run.smithi203.stdout:7: brx.0@if2: mtu 1500 qdisc noqueue master ceph-brx state UP group default qlen 1000 2024-04-25T10:29:34.219 INFO:teuthology.orchestra.run.smithi203.stdout: link/ether aa:26:9e:0a:99:78 brd ff:ff:ff:ff:ff:ff link-netns ceph-ns--home-ubuntu-cephtest-mnt.1 2024-04-25T10:29:34.219 INFO:teuthology.orchestra.run.smithi203.stdout: inet6 fe80::a826:9eff:fe0a:9978/64 scope link tentative 2024-04-25T10:29:34.219 INFO:teuthology.orchestra.run.smithi203.stdout: valid_lft forever preferred_lft forever 2024-04-25T10:29:34.219 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T10:29:34.219 DEBUG:teuthology.orchestra.run.smithi203:> echo 1 | sudo tee /proc/sys/net/ipv4/ip_forward 2024-04-25T10:29:34.279 INFO:teuthology.orchestra.run.smithi203.stdout:1 2024-04-25T10:29:34.281 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T10:29:34.281 DEBUG:teuthology.orchestra.run.smithi203:> route 2024-04-25T10:29:34.337 INFO:teuthology.orchestra.run.smithi203.stdout:Kernel IP routing table 2024-04-25T10:29:34.337 INFO:teuthology.orchestra.run.smithi203.stdout:Destination Gateway Genmask Flags Metric Ref Use Iface 2024-04-25T10:29:34.337 INFO:teuthology.orchestra.run.smithi203.stdout:default _gateway 0.0.0.0 UG 100 0 0 enp3s0f1 2024-04-25T10:29:34.337 INFO:teuthology.orchestra.run.smithi203.stdout:172.21.0.0 0.0.0.0 255.255.240.0 U 100 0 0 enp3s0f1 2024-04-25T10:29:34.337 INFO:teuthology.orchestra.run.smithi203.stdout:192.168.0.0 0.0.0.0 255.255.0.0 U 0 0 0 ceph-brx 2024-04-25T10:29:34.337 DEBUG:teuthology.orchestra.run.smithi203:> (cd / && exec stdin-killer --timeout=300 -- bash -c ' 2024-04-25T10:29:34.337 DEBUG:teuthology.orchestra.run.smithi203:> set -e 2024-04-25T10:29:34.338 DEBUG:teuthology.orchestra.run.smithi203:> sudo iptables -A FORWARD -o enp3s0f1 -i ceph-brx -j ACCEPT 2024-04-25T10:29:34.338 DEBUG:teuthology.orchestra.run.smithi203:> sudo iptables -A FORWARD -i enp3s0f1 -o ceph-brx -j ACCEPT 2024-04-25T10:29:34.338 DEBUG:teuthology.orchestra.run.smithi203:> sudo iptables -t nat -A POSTROUTING -s 192.168.255.254/16 -o enp3s0f1 -j MASQUERADE 2024-04-25T10:29:34.338 DEBUG:teuthology.orchestra.run.smithi203:> ') 2024-04-25T10:29:34.425 INFO:teuthology.orchestra.run.smithi203.stderr:2024-04-25T10:29:34 stdin-killer INFO: expiration expected; waiting 300 seconds for command to complete 2024-04-25T10:29:34.454 INFO:teuthology.orchestra.run.smithi203.stderr:2024-04-25T10:29:34 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-04-25T10:29:34.460 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T10:29:34.461 DEBUG:teuthology.orchestra.run.smithi203:> ip netns list 2024-04-25T10:29:34.475 INFO:teuthology.orchestra.run.smithi203.stdout:ceph-ns--home-ubuntu-cephtest-mnt.1 (id: 0) 2024-04-25T10:29:34.475 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T10:29:34.476 DEBUG:teuthology.orchestra.run.smithi203:> ip netns list-id 2024-04-25T10:29:34.529 INFO:teuthology.orchestra.run.smithi203.stdout:nsid 0 (iproute2 netns name: ceph-ns--home-ubuntu-cephtest-mnt.1) 2024-04-25T10:29:34.530 DEBUG:teuthology.orchestra.run.smithi203:> (cd / && exec stdin-killer --timeout=300 -- bash -c ' 2024-04-25T10:29:34.530 DEBUG:teuthology.orchestra.run.smithi203:> set -e 2024-04-25T10:29:34.530 DEBUG:teuthology.orchestra.run.smithi203:> sudo ip netns add ceph-ns--home-ubuntu-cephtest-mnt.3 2024-04-25T10:29:34.530 DEBUG:teuthology.orchestra.run.smithi203:> sudo ip netns set ceph-ns--home-ubuntu-cephtest-mnt.3 1 2024-04-25T10:29:34.530 DEBUG:teuthology.orchestra.run.smithi203:> ') 2024-04-25T10:29:34.617 INFO:teuthology.orchestra.run.smithi203.stderr:2024-04-25T10:29:34 stdin-killer INFO: expiration expected; waiting 300 seconds for command to complete 2024-04-25T10:29:34.638 INFO:teuthology.orchestra.run.smithi203.stderr:2024-04-25T10:29:34 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-04-25T10:29:34.644 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T10:29:34.644 DEBUG:teuthology.orchestra.run.smithi203:> sudo ip netns exec ceph-ns--home-ubuntu-cephtest-mnt.1 ip addr 2024-04-25T10:29:34.667 INFO:teuthology.orchestra.run.smithi203.stdout:1: lo: mtu 65536 qdisc noqueue state UNKNOWN group default qlen 1000 2024-04-25T10:29:34.668 INFO:teuthology.orchestra.run.smithi203.stdout: link/loopback 00:00:00:00:00:00 brd 00:00:00:00:00:00 2024-04-25T10:29:34.668 INFO:teuthology.orchestra.run.smithi203.stdout: inet 127.0.0.1/8 scope host lo 2024-04-25T10:29:34.668 INFO:teuthology.orchestra.run.smithi203.stdout: valid_lft forever preferred_lft forever 2024-04-25T10:29:34.668 INFO:teuthology.orchestra.run.smithi203.stdout: inet6 ::1/128 scope host 2024-04-25T10:29:34.668 INFO:teuthology.orchestra.run.smithi203.stdout: valid_lft forever preferred_lft forever 2024-04-25T10:29:34.668 INFO:teuthology.orchestra.run.smithi203.stdout:2: veth0@if7: mtu 1500 qdisc noqueue state UP group default qlen 1000 2024-04-25T10:29:34.668 INFO:teuthology.orchestra.run.smithi203.stdout: link/ether 0a:64:79:99:19:46 brd ff:ff:ff:ff:ff:ff link-netnsid 0 2024-04-25T10:29:34.668 INFO:teuthology.orchestra.run.smithi203.stdout: inet 192.168.0.1/16 brd 192.168.255.255 scope global veth0 2024-04-25T10:29:34.668 INFO:teuthology.orchestra.run.smithi203.stdout: valid_lft forever preferred_lft forever 2024-04-25T10:29:34.668 INFO:teuthology.orchestra.run.smithi203.stdout: inet6 fe80::864:79ff:fe99:1946/64 scope link tentative 2024-04-25T10:29:34.668 INFO:teuthology.orchestra.run.smithi203.stdout: valid_lft forever preferred_lft forever 2024-04-25T10:29:34.669 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T10:29:34.670 DEBUG:teuthology.orchestra.run.smithi203:> sudo ip netns exec ceph-ns--home-ubuntu-cephtest-mnt.1 ip addr 2024-04-25T10:29:34.734 INFO:teuthology.orchestra.run.smithi203.stdout:1: lo: mtu 65536 qdisc noqueue state UNKNOWN group default qlen 1000 2024-04-25T10:29:34.734 INFO:teuthology.orchestra.run.smithi203.stdout: link/loopback 00:00:00:00:00:00 brd 00:00:00:00:00:00 2024-04-25T10:29:34.734 INFO:teuthology.orchestra.run.smithi203.stdout: inet 127.0.0.1/8 scope host lo 2024-04-25T10:29:34.734 INFO:teuthology.orchestra.run.smithi203.stdout: valid_lft forever preferred_lft forever 2024-04-25T10:29:34.734 INFO:teuthology.orchestra.run.smithi203.stdout: inet6 ::1/128 scope host 2024-04-25T10:29:34.735 INFO:teuthology.orchestra.run.smithi203.stdout: valid_lft forever preferred_lft forever 2024-04-25T10:29:34.735 INFO:teuthology.orchestra.run.smithi203.stdout:2: veth0@if7: mtu 1500 qdisc noqueue state UP group default qlen 1000 2024-04-25T10:29:34.735 INFO:teuthology.orchestra.run.smithi203.stdout: link/ether 0a:64:79:99:19:46 brd ff:ff:ff:ff:ff:ff link-netnsid 0 2024-04-25T10:29:34.735 INFO:teuthology.orchestra.run.smithi203.stdout: inet 192.168.0.1/16 brd 192.168.255.255 scope global veth0 2024-04-25T10:29:34.735 INFO:teuthology.orchestra.run.smithi203.stdout: valid_lft forever preferred_lft forever 2024-04-25T10:29:34.735 INFO:teuthology.orchestra.run.smithi203.stdout: inet6 fe80::864:79ff:fe99:1946/64 scope link 2024-04-25T10:29:34.735 INFO:teuthology.orchestra.run.smithi203.stdout: valid_lft forever preferred_lft forever 2024-04-25T10:29:34.736 INFO:tasks.cephfs.mount:Setuping the netns 'ceph-ns--home-ubuntu-cephtest-mnt.3' with 192.168.0.2/16 2024-04-25T10:29:34.736 DEBUG:teuthology.orchestra.run.smithi203:> (cd / && exec stdin-killer --timeout=300 -- bash -c ' 2024-04-25T10:29:34.736 DEBUG:teuthology.orchestra.run.smithi203:> set -e 2024-04-25T10:29:34.736 DEBUG:teuthology.orchestra.run.smithi203:> sudo ip link add veth0 netns ceph-ns--home-ubuntu-cephtest-mnt.3 type veth peer name brx.1 2024-04-25T10:29:34.737 DEBUG:teuthology.orchestra.run.smithi203:> sudo ip netns exec ceph-ns--home-ubuntu-cephtest-mnt.3 ip addr add 192.168.0.2/16 brd 192.168.255.255 dev veth0 2024-04-25T10:29:34.737 DEBUG:teuthology.orchestra.run.smithi203:> sudo ip netns exec ceph-ns--home-ubuntu-cephtest-mnt.3 ip link set veth0 up 2024-04-25T10:29:34.737 DEBUG:teuthology.orchestra.run.smithi203:> sudo ip netns exec ceph-ns--home-ubuntu-cephtest-mnt.3 ip link set lo up 2024-04-25T10:29:34.737 DEBUG:teuthology.orchestra.run.smithi203:> sudo ip netns exec ceph-ns--home-ubuntu-cephtest-mnt.3 ip route add default via 192.168.255.254 2024-04-25T10:29:34.737 DEBUG:teuthology.orchestra.run.smithi203:> ') 2024-04-25T10:29:34.824 INFO:teuthology.orchestra.run.smithi203.stderr:2024-04-25T10:29:34 stdin-killer INFO: expiration expected; waiting 300 seconds for command to complete 2024-04-25T10:29:34.877 INFO:teuthology.orchestra.run.smithi203.stderr:2024-04-25T10:29:34 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-04-25T10:29:34.884 DEBUG:teuthology.orchestra.run.smithi203:> (cd / && exec stdin-killer --timeout=300 -- bash -c ' 2024-04-25T10:29:34.884 DEBUG:teuthology.orchestra.run.smithi203:> set -e 2024-04-25T10:29:34.884 DEBUG:teuthology.orchestra.run.smithi203:> sudo ip link set brx.1 up 2024-04-25T10:29:34.884 DEBUG:teuthology.orchestra.run.smithi203:> sudo ip link set dev brx.1 master ceph-brx 2024-04-25T10:29:34.884 DEBUG:teuthology.orchestra.run.smithi203:> ') 2024-04-25T10:29:34.932 INFO:teuthology.orchestra.run.smithi203.stderr:2024-04-25T10:29:34 stdin-killer INFO: expiration expected; waiting 300 seconds for command to complete 2024-04-25T10:29:34.952 INFO:teuthology.orchestra.run.smithi203.stderr:2024-04-25T10:29:34 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-04-25T10:29:34.958 INFO:tasks.cephfs.fuse_mount:Client client.3 config is {} 2024-04-25T10:29:34.958 INFO:teuthology.orchestra.run:Running command with timeout 60 2024-04-25T10:29:34.959 DEBUG:teuthology.orchestra.run.smithi203:> mkdir -p -v /home/ubuntu/cephtest/mnt.3 2024-04-25T10:29:34.973 INFO:teuthology.orchestra.run.smithi203.stdout:mkdir: created directory '/home/ubuntu/cephtest/mnt.3' 2024-04-25T10:29:34.973 INFO:teuthology.orchestra.run:Running command with timeout 60 2024-04-25T10:29:34.973 DEBUG:teuthology.orchestra.run.smithi203:> chmod 0000 /home/ubuntu/cephtest/mnt.3 2024-04-25T10:29:35.027 DEBUG:teuthology.orchestra.run.smithi203:> sudo modprobe fuse 2024-04-25T10:29:35.091 DEBUG:teuthology.orchestra.run.smithi203:> cat /proc/self/mounts | awk '{print $2}' 2024-04-25T10:29:35.146 INFO:teuthology.orchestra.run.smithi203.stdout:/proc 2024-04-25T10:29:35.147 INFO:teuthology.orchestra.run.smithi203.stdout:/sys 2024-04-25T10:29:35.147 INFO:teuthology.orchestra.run.smithi203.stdout:/dev 2024-04-25T10:29:35.147 INFO:teuthology.orchestra.run.smithi203.stdout:/sys/kernel/security 2024-04-25T10:29:35.147 INFO:teuthology.orchestra.run.smithi203.stdout:/dev/shm 2024-04-25T10:29:35.147 INFO:teuthology.orchestra.run.smithi203.stdout:/dev/pts 2024-04-25T10:29:35.147 INFO:teuthology.orchestra.run.smithi203.stdout:/run 2024-04-25T10:29:35.147 INFO:teuthology.orchestra.run.smithi203.stdout:/sys/fs/cgroup 2024-04-25T10:29:35.147 INFO:teuthology.orchestra.run.smithi203.stdout:/sys/fs/pstore 2024-04-25T10:29:35.147 INFO:teuthology.orchestra.run.smithi203.stdout:/sys/fs/bpf 2024-04-25T10:29:35.147 INFO:teuthology.orchestra.run.smithi203.stdout:/ 2024-04-25T10:29:35.147 INFO:teuthology.orchestra.run.smithi203.stdout:/sys/fs/selinux 2024-04-25T10:29:35.147 INFO:teuthology.orchestra.run.smithi203.stdout:/proc/sys/fs/binfmt_misc 2024-04-25T10:29:35.147 INFO:teuthology.orchestra.run.smithi203.stdout:/dev/hugepages 2024-04-25T10:29:35.148 INFO:teuthology.orchestra.run.smithi203.stdout:/dev/mqueue 2024-04-25T10:29:35.148 INFO:teuthology.orchestra.run.smithi203.stdout:/sys/kernel/debug 2024-04-25T10:29:35.148 INFO:teuthology.orchestra.run.smithi203.stdout:/sys/kernel/tracing 2024-04-25T10:29:35.148 INFO:teuthology.orchestra.run.smithi203.stdout:/sys/kernel/config 2024-04-25T10:29:35.148 INFO:teuthology.orchestra.run.smithi203.stdout:/run/credentials/systemd-sysusers.service 2024-04-25T10:29:35.148 INFO:teuthology.orchestra.run.smithi203.stdout:/sys/fs/fuse/connections 2024-04-25T10:29:35.148 INFO:teuthology.orchestra.run.smithi203.stdout:/run/credentials/systemd-sysctl.service 2024-04-25T10:29:35.148 INFO:teuthology.orchestra.run.smithi203.stdout:/run/credentials/systemd-tmpfiles-setup-dev.service 2024-04-25T10:29:35.148 INFO:teuthology.orchestra.run.smithi203.stdout:/run/credentials/systemd-tmpfiles-setup.service 2024-04-25T10:29:35.148 INFO:teuthology.orchestra.run.smithi203.stdout:/var/lib/nfs/rpc_pipefs 2024-04-25T10:29:35.148 INFO:teuthology.orchestra.run.smithi203.stdout:/run/user/1000 2024-04-25T10:29:35.148 INFO:teuthology.orchestra.run.smithi203.stdout:/var/lib/ceph 2024-04-25T10:29:35.148 INFO:teuthology.orchestra.run.smithi203.stdout:/var/lib/ceph/osd/ceph-4 2024-04-25T10:29:35.148 INFO:teuthology.orchestra.run.smithi203.stdout:/var/lib/ceph/osd/ceph-5 2024-04-25T10:29:35.149 INFO:teuthology.orchestra.run.smithi203.stdout:/var/lib/ceph/osd/ceph-6 2024-04-25T10:29:35.149 INFO:teuthology.orchestra.run.smithi203.stdout:/var/lib/ceph/osd/ceph-7 2024-04-25T10:29:35.149 INFO:teuthology.orchestra.run.smithi203.stdout:/run/netns 2024-04-25T10:29:35.149 INFO:teuthology.orchestra.run.smithi203.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 2024-04-25T10:29:35.149 INFO:teuthology.orchestra.run.smithi203.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 2024-04-25T10:29:35.149 INFO:teuthology.orchestra.run.smithi203.stdout:/home/ubuntu/cephtest/mnt.1 2024-04-25T10:29:35.149 INFO:teuthology.orchestra.run.smithi203.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 2024-04-25T10:29:35.149 INFO:teuthology.orchestra.run.smithi203.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 2024-04-25T10:29:35.149 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T10:29:35.149 DEBUG:teuthology.orchestra.run.smithi203:> ls /sys/fs/fuse/connections 2024-04-25T10:29:35.202 INFO:teuthology.orchestra.run.smithi203.stdout:43 2024-04-25T10:29:35.202 INFO:tasks.cephfs.fuse_mount:Pre-mount connections: [43] 2024-04-25T10:29:35.202 DEBUG:teuthology.orchestra.run.smithi203:> (cd /home/ubuntu/cephtest && exec sudo nsenter --net=/var/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-fuse -f --admin-socket '/var/run/ceph/$cluster-$name.$pid.asok' /home/ubuntu/cephtest/mnt.3 --id 3) 2024-04-25T10:29:35.245 DEBUG:teuthology.orchestra.run.smithi203:> sudo modprobe fuse 2024-04-25T10:29:35.268 DEBUG:teuthology.orchestra.run.smithi203:> cat /proc/self/mounts | awk '{print $2}' 2024-04-25T10:29:35.325 INFO:teuthology.orchestra.run.smithi203.stdout:/proc 2024-04-25T10:29:35.325 INFO:teuthology.orchestra.run.smithi203.stdout:/sys 2024-04-25T10:29:35.325 INFO:teuthology.orchestra.run.smithi203.stdout:/dev 2024-04-25T10:29:35.325 INFO:teuthology.orchestra.run.smithi203.stdout:/sys/kernel/security 2024-04-25T10:29:35.325 INFO:teuthology.orchestra.run.smithi203.stdout:/dev/shm 2024-04-25T10:29:35.325 INFO:teuthology.orchestra.run.smithi203.stdout:/dev/pts 2024-04-25T10:29:35.325 INFO:teuthology.orchestra.run.smithi203.stdout:/run 2024-04-25T10:29:35.325 INFO:teuthology.orchestra.run.smithi203.stdout:/sys/fs/cgroup 2024-04-25T10:29:35.325 INFO:teuthology.orchestra.run.smithi203.stdout:/sys/fs/pstore 2024-04-25T10:29:35.325 INFO:teuthology.orchestra.run.smithi203.stdout:/sys/fs/bpf 2024-04-25T10:29:35.326 INFO:teuthology.orchestra.run.smithi203.stdout:/ 2024-04-25T10:29:35.326 INFO:teuthology.orchestra.run.smithi203.stdout:/sys/fs/selinux 2024-04-25T10:29:35.326 INFO:teuthology.orchestra.run.smithi203.stdout:/proc/sys/fs/binfmt_misc 2024-04-25T10:29:35.326 INFO:teuthology.orchestra.run.smithi203.stdout:/dev/hugepages 2024-04-25T10:29:35.326 INFO:teuthology.orchestra.run.smithi203.stdout:/dev/mqueue 2024-04-25T10:29:35.326 INFO:teuthology.orchestra.run.smithi203.stdout:/sys/kernel/debug 2024-04-25T10:29:35.326 INFO:teuthology.orchestra.run.smithi203.stdout:/sys/kernel/tracing 2024-04-25T10:29:35.326 INFO:teuthology.orchestra.run.smithi203.stdout:/sys/kernel/config 2024-04-25T10:29:35.326 INFO:teuthology.orchestra.run.smithi203.stdout:/run/credentials/systemd-sysusers.service 2024-04-25T10:29:35.326 INFO:teuthology.orchestra.run.smithi203.stdout:/sys/fs/fuse/connections 2024-04-25T10:29:35.326 INFO:teuthology.orchestra.run.smithi203.stdout:/run/credentials/systemd-sysctl.service 2024-04-25T10:29:35.326 INFO:teuthology.orchestra.run.smithi203.stdout:/run/credentials/systemd-tmpfiles-setup-dev.service 2024-04-25T10:29:35.326 INFO:teuthology.orchestra.run.smithi203.stdout:/run/credentials/systemd-tmpfiles-setup.service 2024-04-25T10:29:35.326 INFO:teuthology.orchestra.run.smithi203.stdout:/var/lib/nfs/rpc_pipefs 2024-04-25T10:29:35.327 INFO:teuthology.orchestra.run.smithi203.stdout:/run/user/1000 2024-04-25T10:29:35.327 INFO:teuthology.orchestra.run.smithi203.stdout:/var/lib/ceph 2024-04-25T10:29:35.327 INFO:teuthology.orchestra.run.smithi203.stdout:/var/lib/ceph/osd/ceph-4 2024-04-25T10:29:35.327 INFO:teuthology.orchestra.run.smithi203.stdout:/var/lib/ceph/osd/ceph-5 2024-04-25T10:29:35.327 INFO:teuthology.orchestra.run.smithi203.stdout:/var/lib/ceph/osd/ceph-6 2024-04-25T10:29:35.327 INFO:teuthology.orchestra.run.smithi203.stdout:/var/lib/ceph/osd/ceph-7 2024-04-25T10:29:35.327 INFO:teuthology.orchestra.run.smithi203.stdout:/run/netns 2024-04-25T10:29:35.327 INFO:teuthology.orchestra.run.smithi203.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 2024-04-25T10:29:35.327 INFO:teuthology.orchestra.run.smithi203.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 2024-04-25T10:29:35.327 INFO:teuthology.orchestra.run.smithi203.stdout:/home/ubuntu/cephtest/mnt.1 2024-04-25T10:29:35.327 INFO:teuthology.orchestra.run.smithi203.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 2024-04-25T10:29:35.327 INFO:teuthology.orchestra.run.smithi203.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 2024-04-25T10:29:35.327 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T10:29:35.328 DEBUG:teuthology.orchestra.run.smithi203:> ls /sys/fs/fuse/connections 2024-04-25T10:29:35.337 INFO:tasks.cephfs.fuse_mount.ceph-fuse.3.smithi203.stderr:2024-04-25T10:29:35.335+0000 7f1fbc11f480 -1 init, newargv = 0x7f1fa0000ed0 newargc=15 2024-04-25T10:29:35.337 INFO:tasks.cephfs.fuse_mount.ceph-fuse.3.smithi203.stderr:ceph-fuse[41240]: starting ceph client 2024-04-25T10:29:35.343 INFO:tasks.cephfs.fuse_mount.ceph-fuse.3.smithi203.stderr:ceph-fuse[41240]: starting fuse 2024-04-25T10:29:35.351 INFO:teuthology.orchestra.run.smithi203.stdout:43 2024-04-25T10:29:35.351 INFO:teuthology.orchestra.run.smithi203.stdout:44 2024-04-25T10:29:35.351 INFO:tasks.cephfs.fuse_mount:Post-mount connections: [43, 44] 2024-04-25T10:29:35.351 DEBUG:teuthology.orchestra.run.smithi203:> sudo stdin-killer -- python3 -c ' 2024-04-25T10:29:35.351 DEBUG:teuthology.orchestra.run.smithi203:> import glob 2024-04-25T10:29:35.351 DEBUG:teuthology.orchestra.run.smithi203:> import re 2024-04-25T10:29:35.351 DEBUG:teuthology.orchestra.run.smithi203:> import os 2024-04-25T10:29:35.351 DEBUG:teuthology.orchestra.run.smithi203:> import subprocess 2024-04-25T10:29:35.352 DEBUG:teuthology.orchestra.run.smithi203:> 2024-04-25T10:29:35.352 DEBUG:teuthology.orchestra.run.smithi203:> def _find_admin_socket(client_name): 2024-04-25T10:29:35.352 DEBUG:teuthology.orchestra.run.smithi203:> asok_path = "/var/run/ceph/ceph-client.3.*.asok" 2024-04-25T10:29:35.352 DEBUG:teuthology.orchestra.run.smithi203:> files = glob.glob(asok_path) 2024-04-25T10:29:35.352 DEBUG:teuthology.orchestra.run.smithi203:> mountpoint = "/home/ubuntu/cephtest/mnt.3" 2024-04-25T10:29:35.352 DEBUG:teuthology.orchestra.run.smithi203:> 2024-04-25T10:29:35.352 DEBUG:teuthology.orchestra.run.smithi203:> # Given a non-glob path, it better be there 2024-04-25T10:29:35.352 DEBUG:teuthology.orchestra.run.smithi203:> if "*" not in asok_path: 2024-04-25T10:29:35.352 DEBUG:teuthology.orchestra.run.smithi203:> assert(len(files) == 1) 2024-04-25T10:29:35.352 DEBUG:teuthology.orchestra.run.smithi203:> return files[0] 2024-04-25T10:29:35.352 DEBUG:teuthology.orchestra.run.smithi203:> 2024-04-25T10:29:35.352 DEBUG:teuthology.orchestra.run.smithi203:> for f in files: 2024-04-25T10:29:35.352 DEBUG:teuthology.orchestra.run.smithi203:> pid = re.match(".*\.(\d+)\.asok$", f).group(1) 2024-04-25T10:29:35.352 DEBUG:teuthology.orchestra.run.smithi203:> if os.path.exists("/proc/{0}".format(pid)): 2024-04-25T10:29:35.352 DEBUG:teuthology.orchestra.run.smithi203:> with open("/proc/{0}/cmdline".format(pid), '"'"'r'"'"') as proc_f: 2024-04-25T10:29:35.352 DEBUG:teuthology.orchestra.run.smithi203:> contents = proc_f.read() 2024-04-25T10:29:35.353 DEBUG:teuthology.orchestra.run.smithi203:> if mountpoint in contents: 2024-04-25T10:29:35.353 DEBUG:teuthology.orchestra.run.smithi203:> return f 2024-04-25T10:29:35.353 DEBUG:teuthology.orchestra.run.smithi203:> raise RuntimeError("Client socket {0} not found".format(client_name)) 2024-04-25T10:29:35.353 DEBUG:teuthology.orchestra.run.smithi203:> 2024-04-25T10:29:35.353 DEBUG:teuthology.orchestra.run.smithi203:> print(_find_admin_socket("client.3")) 2024-04-25T10:29:35.353 DEBUG:teuthology.orchestra.run.smithi203:> ' 2024-04-25T10:29:35.468 INFO:teuthology.orchestra.run.smithi203.stdout:/var/run/ceph/ceph-client.3.41240.asok 2024-04-25T10:29:35.471 INFO:teuthology.orchestra.run.smithi203.stderr:2024-04-25T10:29:35 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-04-25T10:29:35.479 INFO:tasks.cephfs.fuse_mount:Found client admin socket at /var/run/ceph/ceph-client.3.41240.asok 2024-04-25T10:29:35.479 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T10:29:35.479 DEBUG:teuthology.orchestra.run.smithi203:> sudo ceph --admin-daemon /var/run/ceph/ceph-client.3.41240.asok status 2024-04-25T10:29:35.620 INFO:teuthology.orchestra.run.smithi203.stdout:{ 2024-04-25T10:29:35.620 INFO:teuthology.orchestra.run.smithi203.stdout: "metadata": { 2024-04-25T10:29:35.620 INFO:teuthology.orchestra.run.smithi203.stdout: "ceph_sha1": "994e2f20102a20fa9a27966c14b625cc01884df8", 2024-04-25T10:29:35.621 INFO:teuthology.orchestra.run.smithi203.stdout: "ceph_version": "ceph version 19.0.0-3291-g994e2f20 (994e2f20102a20fa9a27966c14b625cc01884df8) squid (dev)", 2024-04-25T10:29:35.621 INFO:teuthology.orchestra.run.smithi203.stdout: "entity_id": "3", 2024-04-25T10:29:35.621 INFO:teuthology.orchestra.run.smithi203.stdout: "hostname": "smithi203", 2024-04-25T10:29:35.621 INFO:teuthology.orchestra.run.smithi203.stdout: "mount_point": "/home/ubuntu/cephtest/mnt.3", 2024-04-25T10:29:35.621 INFO:teuthology.orchestra.run.smithi203.stdout: "pid": "41240", 2024-04-25T10:29:35.621 INFO:teuthology.orchestra.run.smithi203.stdout: "root": "/" 2024-04-25T10:29:35.621 INFO:teuthology.orchestra.run.smithi203.stdout: }, 2024-04-25T10:29:35.621 INFO:teuthology.orchestra.run.smithi203.stdout: "dentry_count": 0, 2024-04-25T10:29:35.621 INFO:teuthology.orchestra.run.smithi203.stdout: "dentry_pinned_count": 0, 2024-04-25T10:29:35.621 INFO:teuthology.orchestra.run.smithi203.stdout: "id": 4520, 2024-04-25T10:29:35.621 INFO:teuthology.orchestra.run.smithi203.stdout: "inst": { 2024-04-25T10:29:35.621 INFO:teuthology.orchestra.run.smithi203.stdout: "name": { 2024-04-25T10:29:35.621 INFO:teuthology.orchestra.run.smithi203.stdout: "type": "client", 2024-04-25T10:29:35.621 INFO:teuthology.orchestra.run.smithi203.stdout: "num": 4520 2024-04-25T10:29:35.622 INFO:teuthology.orchestra.run.smithi203.stdout: }, 2024-04-25T10:29:35.622 INFO:teuthology.orchestra.run.smithi203.stdout: "addr": { 2024-04-25T10:29:35.622 INFO:teuthology.orchestra.run.smithi203.stdout: "type": "v1", 2024-04-25T10:29:35.622 INFO:teuthology.orchestra.run.smithi203.stdout: "addr": "192.168.0.2:0", 2024-04-25T10:29:35.622 INFO:teuthology.orchestra.run.smithi203.stdout: "nonce": 1721863056 2024-04-25T10:29:35.622 INFO:teuthology.orchestra.run.smithi203.stdout: } 2024-04-25T10:29:35.622 INFO:teuthology.orchestra.run.smithi203.stdout: }, 2024-04-25T10:29:35.622 INFO:teuthology.orchestra.run.smithi203.stdout: "addr": { 2024-04-25T10:29:35.622 INFO:teuthology.orchestra.run.smithi203.stdout: "type": "v1", 2024-04-25T10:29:35.622 INFO:teuthology.orchestra.run.smithi203.stdout: "addr": "192.168.0.2:0", 2024-04-25T10:29:35.622 INFO:teuthology.orchestra.run.smithi203.stdout: "nonce": 1721863056 2024-04-25T10:29:35.622 INFO:teuthology.orchestra.run.smithi203.stdout: }, 2024-04-25T10:29:35.622 INFO:teuthology.orchestra.run.smithi203.stdout: "inst_str": "client.4520 192.168.0.2:0/1721863056", 2024-04-25T10:29:35.623 INFO:teuthology.orchestra.run.smithi203.stdout: "addr_str": "192.168.0.2:0/1721863056", 2024-04-25T10:29:35.623 INFO:teuthology.orchestra.run.smithi203.stdout: "inode_count": 1, 2024-04-25T10:29:35.623 INFO:teuthology.orchestra.run.smithi203.stdout: "mds_epoch": 8, 2024-04-25T10:29:35.623 INFO:teuthology.orchestra.run.smithi203.stdout: "osd_epoch": 23, 2024-04-25T10:29:35.623 INFO:teuthology.orchestra.run.smithi203.stdout: "osd_epoch_barrier": 0, 2024-04-25T10:29:35.623 INFO:teuthology.orchestra.run.smithi203.stdout: "blocklisted": false, 2024-04-25T10:29:35.623 INFO:teuthology.orchestra.run.smithi203.stdout: "fs_name": "cephfs" 2024-04-25T10:29:35.623 INFO:teuthology.orchestra.run.smithi203.stdout:} 2024-04-25T10:29:35.629 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T10:29:35.629 DEBUG:teuthology.orchestra.run.smithi037:> stat --file-system '--printf=%T 2024-04-25T10:29:35.629 DEBUG:teuthology.orchestra.run.smithi037:> ' -- /home/ubuntu/cephtest/mnt.0 2024-04-25T10:29:35.649 INFO:teuthology.orchestra.run.smithi037.stdout:fuseblk 2024-04-25T10:29:35.650 INFO:tasks.cephfs.fuse_mount:ceph-fuse is mounted on /home/ubuntu/cephtest/mnt.0 2024-04-25T10:29:35.650 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T10:29:35.650 DEBUG:teuthology.orchestra.run.smithi037:> sudo chmod 1777 /home/ubuntu/cephtest/mnt.0 2024-04-25T10:29:35.722 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T10:29:35.722 DEBUG:teuthology.orchestra.run.smithi037:> stat --file-system '--printf=%T 2024-04-25T10:29:35.723 DEBUG:teuthology.orchestra.run.smithi037:> ' -- /home/ubuntu/cephtest/mnt.2 2024-04-25T10:29:35.782 INFO:teuthology.orchestra.run.smithi037.stdout:fuseblk 2024-04-25T10:29:35.782 INFO:tasks.cephfs.fuse_mount:ceph-fuse is mounted on /home/ubuntu/cephtest/mnt.2 2024-04-25T10:29:35.782 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T10:29:35.782 DEBUG:teuthology.orchestra.run.smithi037:> sudo chmod 1777 /home/ubuntu/cephtest/mnt.2 2024-04-25T10:29:35.855 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T10:29:35.855 DEBUG:teuthology.orchestra.run.smithi203:> stat --file-system '--printf=%T 2024-04-25T10:29:35.855 DEBUG:teuthology.orchestra.run.smithi203:> ' -- /home/ubuntu/cephtest/mnt.1 2024-04-25T10:29:35.871 INFO:teuthology.orchestra.run.smithi203.stdout:fuseblk 2024-04-25T10:29:35.871 INFO:tasks.cephfs.fuse_mount:ceph-fuse is mounted on /home/ubuntu/cephtest/mnt.1 2024-04-25T10:29:35.871 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T10:29:35.871 DEBUG:teuthology.orchestra.run.smithi203:> sudo chmod 1777 /home/ubuntu/cephtest/mnt.1 2024-04-25T10:29:35.935 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T10:29:35.935 DEBUG:teuthology.orchestra.run.smithi203:> stat --file-system '--printf=%T 2024-04-25T10:29:35.935 DEBUG:teuthology.orchestra.run.smithi203:> ' -- /home/ubuntu/cephtest/mnt.3 2024-04-25T10:29:35.990 INFO:teuthology.orchestra.run.smithi203.stdout:fuseblk 2024-04-25T10:29:35.991 INFO:tasks.cephfs.fuse_mount:ceph-fuse is mounted on /home/ubuntu/cephtest/mnt.3 2024-04-25T10:29:35.991 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T10:29:35.991 DEBUG:teuthology.orchestra.run.smithi203:> sudo chmod 1777 /home/ubuntu/cephtest/mnt.3 2024-04-25T10:29:36.055 INFO:teuthology.run_tasks:Running task cephfs_test_runner... 2024-04-25T10:29:36.072 DEBUG:teuthology.orchestra.run.smithi037:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-04-25T10:29:36.197 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:36.195+0000 7f134f72e640 1 Processor -- start 2024-04-25T10:29:36.197 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:36.195+0000 7f134f72e640 1 -- start start 2024-04-25T10:29:36.199 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:36.197+0000 7f134f72e640 1 --2- >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f134806c8f0 0x7f134809b700 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:29:36.200 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:36.197+0000 7f134f72e640 1 --2- >> v2:172.21.15.37:3300/0 conn(0x7f134806cd00 0x7f134809bc40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:29:36.200 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:36.197+0000 7f134f72e640 1 -- --> v1:172.21.15.203:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f13480b4240 con 0x7f134806e390 2024-04-25T10:29:36.200 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:36.197+0000 7f134f72e640 1 -- --> v2:172.21.15.37:3300/0 -- mon_getmap magic: 0 -- 0x7f134806eb00 con 0x7f134806cd00 2024-04-25T10:29:36.200 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:36.197+0000 7f134f72e640 1 -- --> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] -- mon_getmap magic: 0 -- 0x7f134806ede0 con 0x7f134806c8f0 2024-04-25T10:29:36.200 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:36.197+0000 7f134dccb640 1 --2- >> v2:172.21.15.37:3300/0 conn(0x7f134806cd00 0x7f134809bc40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:29:36.200 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:36.197+0000 7f134dccb640 1 --2- >> v2:172.21.15.37:3300/0 conn(0x7f134806cd00 0x7f134809bc40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.37:3300/0 says I am v2:172.21.15.37:36790/0 (socket says 172.21.15.37:36790) 2024-04-25T10:29:36.200 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:36.197+0000 7f134dccb640 1 -- 172.21.15.37:0/2557958180 learned_addr learned my addr 172.21.15.37:0/2557958180 (peer_addr_for_me v2:172.21.15.37:0/0) 2024-04-25T10:29:36.200 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:36.197+0000 7f134ccc9640 1 --2- 172.21.15.37:0/2557958180 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f134806c8f0 0x7f134809b700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:29:36.200 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:36.197+0000 7f134dccb640 1 -- 172.21.15.37:0/2557958180 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f134806c8f0 msgr2=0x7f134809b700 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:29:36.200 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:36.197+0000 7f134dccb640 1 --2- 172.21.15.37:0/2557958180 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f134806c8f0 0x7f134809b700 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:29:36.200 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:36.197+0000 7f134dccb640 1 -- 172.21.15.37:0/2557958180 >> v1:172.21.15.203:6789/0 conn(0x7f134806e390 legacy=0x7f134815eec0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:29:36.200 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:36.197+0000 7f134dccb640 1 -- 172.21.15.37:0/2557958180 --> v2:172.21.15.37:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f134815e6c0 con 0x7f134806cd00 2024-04-25T10:29:36.202 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:36.198+0000 7f134dccb640 1 --2- 172.21.15.37:0/2557958180 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f134806cd00 0x7f134809bc40 secure :-1 s=READY pgs=194 cs=0 l=1 rev1=1 crypto rx=0x7f1340009e50 tx=0x7f134002a710 comp rx=0 tx=0).ready entity=mon.0 client_cookie=1b8707350db89235 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:29:36.202 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:36.198+0000 7f133ffff640 1 -- 172.21.15.37:0/2557958180 <== mon.0 v2:172.21.15.37:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1340036070 con 0x7f134806cd00 2024-04-25T10:29:36.202 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:36.198+0000 7f133ffff640 1 -- 172.21.15.37:0/2557958180 <== mon.0 v2:172.21.15.37:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f1340031600 con 0x7f134806cd00 2024-04-25T10:29:36.202 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:36.198+0000 7f133ffff640 1 -- 172.21.15.37:0/2557958180 <== mon.0 v2:172.21.15.37:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f13400319d0 con 0x7f134806cd00 2024-04-25T10:29:36.202 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:36.199+0000 7f134f72e640 1 -- 172.21.15.37:0/2557958180 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f134806cd00 msgr2=0x7f134809bc40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:29:36.202 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:36.199+0000 7f134f72e640 1 --2- 172.21.15.37:0/2557958180 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f134806cd00 0x7f134809bc40 secure :-1 s=READY pgs=194 cs=0 l=1 rev1=1 crypto rx=0x7f1340009e50 tx=0x7f134002a710 comp rx=0 tx=0).stop 2024-04-25T10:29:36.202 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:36.199+0000 7f134f72e640 1 -- 172.21.15.37:0/2557958180 shutdown_connections 2024-04-25T10:29:36.202 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:36.199+0000 7f134f72e640 1 --2- 172.21.15.37:0/2557958180 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f134806cd00 0x7f134809bc40 unknown :-1 s=CLOSED pgs=194 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:29:36.202 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:36.199+0000 7f134f72e640 1 --2- 172.21.15.37:0/2557958180 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f134806c8f0 0x7f134809b700 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:29:36.202 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:36.199+0000 7f134f72e640 1 -- 172.21.15.37:0/2557958180 >> 172.21.15.37:0/2557958180 conn(0x7f13480b7600 msgr2=0x7f13480b7c30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:29:36.202 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:36.199+0000 7f134f72e640 1 -- 172.21.15.37:0/2557958180 shutdown_connections 2024-04-25T10:29:36.202 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:36.199+0000 7f134f72e640 1 -- 172.21.15.37:0/2557958180 wait complete. 2024-04-25T10:29:36.203 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:36.201+0000 7f134f72e640 1 Processor -- start 2024-04-25T10:29:36.203 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:36.201+0000 7f134f72e640 1 -- start start 2024-04-25T10:29:36.205 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:36.202+0000 7f134f72e640 1 --2- >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f134806c8f0 0x7f13481f3f40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:29:36.205 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:36.202+0000 7f134f72e640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f134806cd00 0x7f13480a00f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:29:36.206 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:36.202+0000 7f134f72e640 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f134806e390 0x7f13481606a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:29:36.206 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:36.202+0000 7f134f72e640 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 -- 0x7f134815bd00 con 0x7f134806cd00 2024-04-25T10:29:36.206 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:36.202+0000 7f134f72e640 1 -- --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_getmap magic: 0 -- 0x7f134806e7a0 con 0x7f134806e390 2024-04-25T10:29:36.206 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:36.202+0000 7f134f72e640 1 -- --> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] -- mon_getmap magic: 0 -- 0x7f134815be80 con 0x7f134806c8f0 2024-04-25T10:29:36.206 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:36.202+0000 7f134ccc9640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f134806cd00 0x7f13480a00f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:29:36.206 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:36.202+0000 7f134ccc9640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f134806cd00 0x7f13480a00f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.37:3300/0 says I am v2:172.21.15.37:36794/0 (socket says 172.21.15.37:36794) 2024-04-25T10:29:36.206 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:36.202+0000 7f134ccc9640 1 -- 172.21.15.37:0/534959984 learned_addr learned my addr 172.21.15.37:0/534959984 (peer_addr_for_me v2:172.21.15.37:0/0) 2024-04-25T10:29:36.206 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:36.202+0000 7f134d4ca640 1 --2- 172.21.15.37:0/534959984 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f134806c8f0 0x7f13481f3f40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:29:36.206 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:36.202+0000 7f134dccb640 1 --2- 172.21.15.37:0/534959984 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f134806e390 0x7f13481606a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:29:36.206 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:36.203+0000 7f134ccc9640 1 -- 172.21.15.37:0/534959984 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f134806c8f0 msgr2=0x7f13481f3f40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:29:36.206 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:36.203+0000 7f134ccc9640 1 --2- 172.21.15.37:0/534959984 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f134806c8f0 0x7f13481f3f40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:29:36.206 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:36.203+0000 7f134ccc9640 1 -- 172.21.15.37:0/534959984 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f134806e390 msgr2=0x7f13481606a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:29:36.206 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:36.203+0000 7f134ccc9640 1 --2- 172.21.15.37:0/534959984 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f134806e390 0x7f13481606a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:29:36.207 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:36.203+0000 7f134ccc9640 1 -- 172.21.15.37:0/534959984 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f13481f7d50 con 0x7f134806cd00 2024-04-25T10:29:36.207 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:36.203+0000 7f134ccc9640 1 --2- 172.21.15.37:0/534959984 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f134806cd00 0x7f13480a00f0 secure :-1 s=READY pgs=195 cs=0 l=1 rev1=1 crypto rx=0x7f133800d5f0 tx=0x7f133800dab0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:29:36.207 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:36.203+0000 7f133dffb640 1 -- 172.21.15.37:0/534959984 <== mon.0 v2:172.21.15.37:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f13380125f0 con 0x7f134806cd00 2024-04-25T10:29:36.207 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:36.203+0000 7f133dffb640 1 -- 172.21.15.37:0/534959984 <== mon.0 v2:172.21.15.37:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f1338019070 con 0x7f134806cd00 2024-04-25T10:29:36.207 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:36.203+0000 7f134f72e640 1 -- 172.21.15.37:0/534959984 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f13480d0f00 con 0x7f134806cd00 2024-04-25T10:29:36.207 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:36.203+0000 7f133dffb640 1 -- 172.21.15.37:0/534959984 <== mon.0 v2:172.21.15.37:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f13380129d0 con 0x7f134806cd00 2024-04-25T10:29:36.207 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:36.203+0000 7f134f72e640 1 -- 172.21.15.37:0/534959984 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f13480d14c0 con 0x7f134806cd00 2024-04-25T10:29:36.208 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:36.204+0000 7f134f72e640 1 -- 172.21.15.37:0/534959984 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f134806a140 con 0x7f134806cd00 2024-04-25T10:29:36.208 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:36.205+0000 7f133dffb640 1 -- 172.21.15.37:0/534959984 <== mon.0 v2:172.21.15.37:3300/0 4 ==== mgrmap(e 5) ==== 96782+0+0 (secure 0 0 0) 0x7f1338012b70 con 0x7f134806cd00 2024-04-25T10:29:36.208 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:36.205+0000 7f133dffb640 1 --2- 172.21.15.37:0/534959984 >> [v2:172.21.15.203:6832/2418205237,v1:172.21.15.203:6833/2418205237] conn(0x7f132c0744e0 0x7f132c0769f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:29:36.208 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:36.206+0000 7f133dffb640 1 -- 172.21.15.37:0/534959984 <== mon.0 v2:172.21.15.37:3300/0 5 ==== osd_map(23..23 src has 1..23) ==== 5709+0+0 (secure 0 0 0) 0x7f1338095500 con 0x7f134806cd00 2024-04-25T10:29:36.208 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:36.206+0000 7f134d4ca640 1 --2- 172.21.15.37:0/534959984 >> [v2:172.21.15.203:6832/2418205237,v1:172.21.15.203:6833/2418205237] conn(0x7f132c0744e0 0x7f132c0769f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:29:36.208 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:36.206+0000 7f134d4ca640 1 --2- 172.21.15.37:0/534959984 >> [v2:172.21.15.203:6832/2418205237,v1:172.21.15.203:6833/2418205237] conn(0x7f132c0744e0 0x7f132c0769f0 secure :-1 s=READY pgs=117 cs=0 l=1 rev1=1 crypto rx=0x7f13480741f0 tx=0x7f1344009040 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:29:36.212 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:36.210+0000 7f133dffb640 1 -- 172.21.15.37:0/534959984 <== mon.0 v2:172.21.15.37:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187246 (secure 0 0 0) 0x7f133805f640 con 0x7f134806cd00 2024-04-25T10:29:36.398 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:36.396+0000 7f134f72e640 1 -- 172.21.15.37:0/534959984 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7f13480b1b10 con 0x7f134806cd00 2024-04-25T10:29:36.400 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:36.397+0000 7f133dffb640 1 -- 172.21.15.37:0/534959984 <== mon.0 v2:172.21.15.37:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v23) ==== 74+0+16621 (secure 0 0 0) 0x7f1338063940 con 0x7f134806cd00 2024-04-25T10:29:36.400 INFO:teuthology.orchestra.run.smithi037.stdout: 2024-04-25T10:29:36.400 INFO:teuthology.orchestra.run.smithi037.stdout:{"epoch":23,"fsid":"f4acda28-be34-492a-b286-994afd36f61d","created":"2024-04-25T10:28:43.985718+0000","modified":"2024-04-25T10:29:10.016188+0000","last_up_change":"2024-04-25T10:28:54.900654+0000","last_in_change":"2024-04-25T10:28:46.216973+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":3,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-04-25T10:28:56.202153+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"19","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2024-04-25T10:29:06.507244+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"22","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":2.25,"score_stable":2.25,"optimal_score":1,"raw_score_acting":2.25,"raw_score_stable":2.25,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2024-04-25T10:29:07.347844+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"23","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"a2f79028-b332-4166-9ebb-a7024a37ea22","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":21,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.37:6816","nonce":2746074720},{"type":"v1","addr":"172.21.15.37:6817","nonce":2746074720}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.37:6818","nonce":2746074720},{"type":"v1","addr":"172.21.15.37:6819","nonce":2746074720}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.37:6822","nonce":2746074720},{"type":"v1","addr":"172.21.15.37:6823","nonce":2746074720}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.37:6820","nonce":2746074720},{"type":"v1","addr":"172.21.15.37:6821","nonce":2746074720}]},"public_addr":"172.21.15.37:6817/2746074720","cluster_addr":"172.21.15.37:6819/2746074720","heartbeat_back_addr":"172.21.15.37:6823/2746074720","heartbeat_front_addr":"172.21.15.37:6821/2746074720","state":["exists","up"]},{"osd":1,"uuid":"973159de-c2ea-4083-aecc-69373d1c1132","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":21,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.37:6800","nonce":1690341662},{"type":"v1","addr":"172.21.15.37:6801","nonce":1690341662}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.37:6802","nonce":1690341662},{"type":"v1","addr":"172.21.15.37:6803","nonce":1690341662}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.37:6806","nonce":1690341662},{"type":"v1","addr":"172.21.15.37:6807","nonce":1690341662}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.37:6804","nonce":1690341662},{"type":"v1","addr":"172.21.15.37:6805","nonce":1690341662}]},"public_addr":"172.21.15.37:6801/1690341662","cluster_addr":"172.21.15.37:6803/1690341662","heartbeat_back_addr":"172.21.15.37:6807/1690341662","heartbeat_front_addr":"172.21.15.37:6805/1690341662","state":["exists","up"]},{"osd":2,"uuid":"4ae7a0e9-cb3f-484c-a75f-f3766b721c47","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":21,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.37:6808","nonce":2754669649},{"type":"v1","addr":"172.21.15.37:6809","nonce":2754669649}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.37:6810","nonce":2754669649},{"type":"v1","addr":"172.21.15.37:6811","nonce":2754669649}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.37:6814","nonce":2754669649},{"type":"v1","addr":"172.21.15.37:6815","nonce":2754669649}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.37:6812","nonce":2754669649},{"type":"v1","addr":"172.21.15.37:6813","nonce":2754669649}]},"public_addr":"172.21.15.37:6809/2754669649","cluster_addr":"172.21.15.37:6811/2754669649","heartbeat_back_addr":"172.21.15.37:6815/2754669649","heartbeat_front_addr":"172.21.15.37:6813/2754669649","state":["exists","up"]},{"osd":3,"uuid":"1ba4a746-1608-47a7-ace2-960a5cfae934","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":21,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.37:6824","nonce":1815290618},{"type":"v1","addr":"172.21.15.37:6825","nonce":1815290618}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.37:6826","nonce":1815290618},{"type":"v1","addr":"172.21.15.37:6827","nonce":1815290618}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.37:6830","nonce":1815290618},{"type":"v1","addr":"172.21.15.37:6831","nonce":1815290618}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.37:6828","nonce":1815290618},{"type":"v1","addr":"172.21.15.37:6829","nonce":1815290618}]},"public_addr":"172.21.15.37:6825/1815290618","cluster_addr":"172.21.15.37:6827/1815290618","heartbeat_back_addr":"172.21.15.37:6831/1815290618","heartbeat_front_addr":"172.21.15.37:6829/1815290618","state":["exists","up"]},{"osd":4,"uuid":"d4f59f5d-0413-4b79-87ee-bb212c1fcf62","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":21,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6800","nonce":4051532063},{"type":"v1","addr":"172.21.15.203:6801","nonce":4051532063}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6802","nonce":4051532063},{"type":"v1","addr":"172.21.15.203:6803","nonce":4051532063}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6806","nonce":4051532063},{"type":"v1","addr":"172.21.15.203:6807","nonce":4051532063}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6804","nonce":4051532063},{"type":"v1","addr":"172.21.15.203:6805","nonce":4051532063}]},"public_addr":"172.21.15.203:6801/4051532063","cluster_addr":"172.21.15.203:6803/4051532063","heartbeat_back_addr":"172.21.15.203:6807/4051532063","heartbeat_front_addr":"172.21.15.203:6805/4051532063","state":["exists","up"]},{"osd":5,"uuid":"69710a0e-41d8-4e7f-9a10-17ce540b67be","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":21,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6824","nonce":1203822890},{"type":"v1","addr":"172.21.15.203:6825","nonce":1203822890}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6826","nonce":1203822890},{"type":"v1","addr":"172.21.15.203:6827","nonce":1203822890}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6830","nonce":1203822890},{"type":"v1","addr":"172.21.15.203:6831","nonce":1203822890}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6828","nonce":1203822890},{"type":"v1","addr":"172.21.15.203:6829","nonce":1203822890}]},"public_addr":"172.21.15.203:6825/1203822890","cluster_addr":"172.21.15.203:6827/1203822890","heartbeat_back_addr":"172.21.15.203:6831/1203822890","heartbeat_front_addr":"172.21.15.203:6829/1203822890","state":["exists","up"]},{"osd":6,"uuid":"f29dd047-3b48-473d-a235-6642a6293f20","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":21,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6810","nonce":2309615014},{"type":"v1","addr":"172.21.15.203:6812","nonce":2309615014}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6814","nonce":2309615014},{"type":"v1","addr":"172.21.15.203:6816","nonce":2309615014}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6822","nonce":2309615014},{"type":"v1","addr":"172.21.15.203:6823","nonce":2309615014}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6818","nonce":2309615014},{"type":"v1","addr":"172.21.15.203:6820","nonce":2309615014}]},"public_addr":"172.21.15.203:6812/2309615014","cluster_addr":"172.21.15.203:6816/2309615014","heartbeat_back_addr":"172.21.15.203:6823/2309615014","heartbeat_front_addr":"172.21.15.203:6820/2309615014","state":["exists","up"]},{"osd":7,"uuid":"6e691644-378b-4fdc-8e86-fb5f56e31f29","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":21,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6808","nonce":3688099014},{"type":"v1","addr":"172.21.15.203:6809","nonce":3688099014}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6811","nonce":3688099014},{"type":"v1","addr":"172.21.15.203:6813","nonce":3688099014}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6819","nonce":3688099014},{"type":"v1","addr":"172.21.15.203:6821","nonce":3688099014}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6815","nonce":3688099014},{"type":"v1","addr":"172.21.15.203:6817","nonce":3688099014}]},"public_addr":"172.21.15.203:6809/3688099014","cluster_addr":"172.21.15.203:6813/3688099014","heartbeat_back_addr":"172.21.15.203:6821/3688099014","heartbeat_front_addr":"172.21.15.203:6817/3688099014","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T10:28:51.419141+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T10:28:51.366448+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T10:28:51.348276+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T10:28:51.380519+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T10:28:51.105812+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T10:28:51.384496+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T10:28:51.388755+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T10:28:51.366958+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-04-25T10:29:36.403 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:36.399+0000 7f134f72e640 1 -- 172.21.15.37:0/534959984 >> [v2:172.21.15.203:6832/2418205237,v1:172.21.15.203:6833/2418205237] conn(0x7f132c0744e0 msgr2=0x7f132c0769f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:29:36.403 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:36.399+0000 7f134f72e640 1 --2- 172.21.15.37:0/534959984 >> [v2:172.21.15.203:6832/2418205237,v1:172.21.15.203:6833/2418205237] conn(0x7f132c0744e0 0x7f132c0769f0 secure :-1 s=READY pgs=117 cs=0 l=1 rev1=1 crypto rx=0x7f13480741f0 tx=0x7f1344009040 comp rx=0 tx=0).stop 2024-04-25T10:29:36.403 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:36.399+0000 7f134f72e640 1 -- 172.21.15.37:0/534959984 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f134806cd00 msgr2=0x7f13480a00f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:29:36.403 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:36.399+0000 7f134f72e640 1 --2- 172.21.15.37:0/534959984 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f134806cd00 0x7f13480a00f0 secure :-1 s=READY pgs=195 cs=0 l=1 rev1=1 crypto rx=0x7f133800d5f0 tx=0x7f133800dab0 comp rx=0 tx=0).stop 2024-04-25T10:29:36.403 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:36.400+0000 7f134f72e640 1 -- 172.21.15.37:0/534959984 shutdown_connections 2024-04-25T10:29:36.404 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:36.400+0000 7f134f72e640 1 --2- 172.21.15.37:0/534959984 >> [v2:172.21.15.203:6832/2418205237,v1:172.21.15.203:6833/2418205237] conn(0x7f132c0744e0 0x7f132c0769f0 unknown :-1 s=CLOSED pgs=117 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:29:36.404 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:36.400+0000 7f134f72e640 1 --2- 172.21.15.37:0/534959984 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f134806e390 0x7f13481606a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:29:36.404 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:36.400+0000 7f134f72e640 1 --2- 172.21.15.37:0/534959984 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f134806cd00 0x7f13480a00f0 unknown :-1 s=CLOSED pgs=195 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:29:36.404 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:36.400+0000 7f134f72e640 1 --2- 172.21.15.37:0/534959984 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f134806c8f0 0x7f13481f3f40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:29:36.404 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:36.400+0000 7f134f72e640 1 -- 172.21.15.37:0/534959984 >> 172.21.15.37:0/534959984 conn(0x7f13480b7600 msgr2=0x7f134815b2f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:29:36.404 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:36.400+0000 7f134f72e640 1 -- 172.21.15.37:0/534959984 shutdown_connections 2024-04-25T10:29:36.405 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:36.400+0000 7f134f72e640 1 -- 172.21.15.37:0/534959984 wait complete. 2024-04-25T10:29:36.414 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2024-04-25T10:28:56.202153+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '19', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 8, 'score_stable': 8, 'optimal_score': 0.25, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 2, 'pool_name': 'cephfs_metadata', 'create_time': '2024-04-25T10:29:06.507244+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '22', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_autoscale_bias': 4, 'pg_num_min': 16, 'recovery_priority': 5}, 'application_metadata': {'cephfs': {'metadata': 'cephfs'}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 2.25, 'score_stable': 2.25, 'optimal_score': 1, 'raw_score_acting': 2.25, 'raw_score_stable': 2.25, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 3, 'pool_name': 'cephfs_data', 'create_time': '2024-04-25T10:29:07.347844+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 64, 'pg_placement_num': 64, 'pg_placement_num_target': 64, 'pg_num_target': 64, 'pg_num_pending': 64, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '23', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.8999999761581421}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 1.5, 'score_stable': 1.5, 'optimal_score': 1, 'raw_score_acting': 1.5, 'raw_score_stable': 1.5, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}] 2024-04-25T10:29:36.414 DEBUG:teuthology.orchestra.run.smithi037:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2024-04-25T10:29:36.529 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:36.527+0000 7f4e00669640 1 Processor -- start 2024-04-25T10:29:36.530 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:36.527+0000 7f4e00669640 1 -- start start 2024-04-25T10:29:36.531 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:36.528+0000 7f4e00669640 1 --2- >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f4df80d5160 0x7f4df80b4e80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:29:36.532 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:36.529+0000 7f4e00669640 1 -- --> v1:172.21.15.37:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f4df80d1370 con 0x7f4df80d3230 2024-04-25T10:29:36.532 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:36.529+0000 7f4e00669640 1 -- --> v1:172.21.15.203:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f4df80d4680 con 0x7f4df80d0990 2024-04-25T10:29:36.532 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:36.529+0000 7f4e00669640 1 -- --> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] -- mon_getmap magic: 0 -- 0x7f4df80d1650 con 0x7f4df80d5160 2024-04-25T10:29:36.532 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:36.529+0000 7f4dfec06640 1 --1- >> v1:172.21.15.37:6789/0 conn(0x7f4df80d3230 0x7f4df80b9060 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.37:6789/0 says I am v1:172.21.15.37:46516/0 (socket says 172.21.15.37:46516) 2024-04-25T10:29:36.532 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:36.529+0000 7f4dfec06640 1 -- 172.21.15.37:0/61619634 learned_addr learned my addr 172.21.15.37:0/61619634 (peer_addr_for_me v1:172.21.15.37:0/0) 2024-04-25T10:29:36.532 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:36.529+0000 7f4dfe405640 1 --2- 172.21.15.37:0/61619634 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f4df80d5160 0x7f4df80b4e80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:29:36.532 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:36.529+0000 7f4dfd403640 1 -- 172.21.15.37:0/61619634 <== mon.0 v1:172.21.15.37:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 817428778 0 0) 0x7f4df80d1370 con 0x7f4df80d3230 2024-04-25T10:29:36.532 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:36.529+0000 7f4dfd403640 1 -- 172.21.15.37:0/61619634 --> v1:172.21.15.37:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f4de0003520 con 0x7f4df80d3230 2024-04-25T10:29:36.532 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:36.529+0000 7f4dfd403640 1 -- 172.21.15.37:0/61619634 <== mon.1 v1:172.21.15.203:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 836764766 0 0) 0x7f4df80d4680 con 0x7f4df80d0990 2024-04-25T10:29:36.532 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:36.529+0000 7f4dfd403640 1 -- 172.21.15.37:0/61619634 --> v1:172.21.15.203:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f4df80d1370 con 0x7f4df80d0990 2024-04-25T10:29:36.532 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:36.529+0000 7f4dfd403640 1 -- 172.21.15.37:0/61619634 <== mon.0 v1:172.21.15.37:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 1199779775 0 0) 0x7f4de0003520 con 0x7f4df80d3230 2024-04-25T10:29:36.532 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:36.529+0000 7f4dfe405640 1 -- 172.21.15.37:0/61619634 >> v1:172.21.15.203:6789/0 conn(0x7f4df80d0990 legacy=0x7f4df80d2b20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:29:36.532 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:36.529+0000 7f4dfe405640 1 -- 172.21.15.37:0/61619634 >> v1:172.21.15.37:6789/0 conn(0x7f4df80d3230 legacy=0x7f4df80b9060 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:29:36.532 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:36.529+0000 7f4dfe405640 1 -- 172.21.15.37:0/61619634 --> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f4df80d4960 con 0x7f4df80d5160 2024-04-25T10:29:36.534 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:36.530+0000 7f4dfe405640 1 --2- 172.21.15.37:0/61619634 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f4df80d5160 0x7f4df80b4e80 secure :-1 s=READY pgs=43 cs=0 l=1 rev1=1 crypto rx=0x7f4df00097d0 tx=0x7f4df002a0a0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=fa0ee1cde5dd822e server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:29:36.534 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:36.530+0000 7f4dfd403640 1 -- 172.21.15.37:0/61619634 <== mon.2 v2:172.21.15.203:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4df0037070 con 0x7f4df80d5160 2024-04-25T10:29:36.534 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:36.530+0000 7f4dfd403640 1 -- 172.21.15.37:0/61619634 <== mon.2 v2:172.21.15.203:3301/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f4df002ab30 con 0x7f4df80d5160 2024-04-25T10:29:36.534 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:36.530+0000 7f4dfd403640 1 -- 172.21.15.37:0/61619634 <== mon.2 v2:172.21.15.203:3301/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4de80019f0 con 0x7f4df80d5160 2024-04-25T10:29:36.534 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:36.531+0000 7f4e00669640 1 -- 172.21.15.37:0/61619634 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f4df80d5160 msgr2=0x7f4df80b4e80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:29:36.534 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:36.531+0000 7f4e00669640 1 --2- 172.21.15.37:0/61619634 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f4df80d5160 0x7f4df80b4e80 secure :-1 s=READY pgs=43 cs=0 l=1 rev1=1 crypto rx=0x7f4df00097d0 tx=0x7f4df002a0a0 comp rx=0 tx=0).stop 2024-04-25T10:29:36.534 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:36.531+0000 7f4e00669640 1 -- 172.21.15.37:0/61619634 shutdown_connections 2024-04-25T10:29:36.534 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:36.531+0000 7f4e00669640 1 --2- 172.21.15.37:0/61619634 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f4df80d5160 0x7f4df80b4e80 unknown :-1 s=CLOSED pgs=43 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:29:36.534 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:36.531+0000 7f4e00669640 1 -- 172.21.15.37:0/61619634 >> 172.21.15.37:0/61619634 conn(0x7f4df80b7a80 msgr2=0x7f4df80b7e90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:29:36.534 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:36.531+0000 7f4e00669640 1 -- 172.21.15.37:0/61619634 shutdown_connections 2024-04-25T10:29:36.534 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:36.531+0000 7f4e00669640 1 -- 172.21.15.37:0/61619634 wait complete. 2024-04-25T10:29:36.535 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:36.533+0000 7f4e00669640 1 Processor -- start 2024-04-25T10:29:36.535 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:36.533+0000 7f4e00669640 1 -- start start 2024-04-25T10:29:36.538 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:36.534+0000 7f4e00669640 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f4df80d0990 0x7f4df81625a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:29:36.538 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:36.534+0000 7f4e00669640 1 --2- >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f4df80d5160 0x7f4df8157cf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:29:36.538 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:36.535+0000 7f4e00669640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f4df80ce090 0x7f4df80b6650 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:29:36.538 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:36.535+0000 7f4e00669640 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 -- 0x7f4df8164c40 con 0x7f4df80ce090 2024-04-25T10:29:36.538 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:36.535+0000 7f4e00669640 1 -- --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_getmap magic: 0 -- 0x7f4df8164ac0 con 0x7f4df80d0990 2024-04-25T10:29:36.538 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:36.535+0000 7f4e00669640 1 -- --> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] -- mon_getmap magic: 0 -- 0x7f4df80d0da0 con 0x7f4df80d5160 2024-04-25T10:29:36.538 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:36.535+0000 7f4dfec06640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f4df80ce090 0x7f4df80b6650 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:29:36.539 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:36.535+0000 7f4dfec06640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f4df80ce090 0x7f4df80b6650 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.37:3300/0 says I am v2:172.21.15.37:36806/0 (socket says 172.21.15.37:36806) 2024-04-25T10:29:36.539 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:36.535+0000 7f4dfec06640 1 -- 172.21.15.37:0/3439543228 learned_addr learned my addr 172.21.15.37:0/3439543228 (peer_addr_for_me v2:172.21.15.37:0/0) 2024-04-25T10:29:36.539 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:36.535+0000 7f4dfe405640 1 --2- 172.21.15.37:0/3439543228 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f4df80d0990 0x7f4df81625a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:29:36.539 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:36.535+0000 7f4dfdc04640 1 --2- 172.21.15.37:0/3439543228 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f4df80d5160 0x7f4df8157cf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:29:36.539 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:36.535+0000 7f4dfec06640 1 -- 172.21.15.37:0/3439543228 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f4df80d5160 msgr2=0x7f4df8157cf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:29:36.539 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:36.535+0000 7f4dfec06640 1 --2- 172.21.15.37:0/3439543228 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f4df80d5160 0x7f4df8157cf0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:29:36.539 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:36.535+0000 7f4dfec06640 1 -- 172.21.15.37:0/3439543228 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f4df80d0990 msgr2=0x7f4df81625a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:29:36.539 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:36.535+0000 7f4dfec06640 1 --2- 172.21.15.37:0/3439543228 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f4df80d0990 0x7f4df81625a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:29:36.539 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:36.535+0000 7f4dfec06640 1 -- 172.21.15.37:0/3439543228 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f4df80cf940 con 0x7f4df80ce090 2024-04-25T10:29:36.539 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:36.535+0000 7f4dfe405640 1 --2- 172.21.15.37:0/3439543228 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f4df80d0990 0x7f4df81625a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T10:29:36.539 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:36.535+0000 7f4dfec06640 1 --2- 172.21.15.37:0/3439543228 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f4df80ce090 0x7f4df80b6650 secure :-1 s=READY pgs=197 cs=0 l=1 rev1=1 crypto rx=0x7f4de8001350 tx=0x7f4de800a430 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:29:36.539 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:36.535+0000 7f4df6ffd640 1 -- 172.21.15.37:0/3439543228 <== mon.0 v2:172.21.15.37:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4de8017070 con 0x7f4df80ce090 2024-04-25T10:29:36.539 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:36.535+0000 7f4df6ffd640 1 -- 172.21.15.37:0/3439543228 <== mon.0 v2:172.21.15.37:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f4de800ace0 con 0x7f4df80ce090 2024-04-25T10:29:36.539 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:36.535+0000 7f4df6ffd640 1 -- 172.21.15.37:0/3439543228 <== mon.0 v2:172.21.15.37:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4de8012640 con 0x7f4df80ce090 2024-04-25T10:29:36.540 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:36.535+0000 7f4e00669640 1 -- 172.21.15.37:0/3439543228 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f4df80ce8a0 con 0x7f4df80ce090 2024-04-25T10:29:36.540 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:36.535+0000 7f4e00669640 1 -- 172.21.15.37:0/3439543228 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f4df80cee60 con 0x7f4df80ce090 2024-04-25T10:29:36.540 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:36.537+0000 7f4e00669640 1 -- 172.21.15.37:0/3439543228 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f4df806a140 con 0x7f4df80ce090 2024-04-25T10:29:36.541 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:36.537+0000 7f4df6ffd640 1 -- 172.21.15.37:0/3439543228 <== mon.0 v2:172.21.15.37:3300/0 4 ==== mgrmap(e 5) ==== 96782+0+0 (secure 0 0 0) 0x7f4de801a070 con 0x7f4df80ce090 2024-04-25T10:29:36.541 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:36.537+0000 7f4df6ffd640 1 --2- 172.21.15.37:0/3439543228 >> [v2:172.21.15.203:6832/2418205237,v1:172.21.15.203:6833/2418205237] conn(0x7f4de40744e0 0x7f4de40769f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:29:36.541 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:36.538+0000 7f4df6ffd640 1 -- 172.21.15.37:0/3439543228 <== mon.0 v2:172.21.15.37:3300/0 5 ==== osd_map(23..23 src has 1..23) ==== 5709+0+0 (secure 0 0 0) 0x7f4de80950f0 con 0x7f4df80ce090 2024-04-25T10:29:36.541 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:36.538+0000 7f4dfe405640 1 --2- 172.21.15.37:0/3439543228 >> [v2:172.21.15.203:6832/2418205237,v1:172.21.15.203:6833/2418205237] conn(0x7f4de40744e0 0x7f4de40769f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:29:36.542 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:36.538+0000 7f4dfe405640 1 --2- 172.21.15.37:0/3439543228 >> [v2:172.21.15.203:6832/2418205237,v1:172.21.15.203:6833/2418205237] conn(0x7f4de40744e0 0x7f4de40769f0 secure :-1 s=READY pgs=118 cs=0 l=1 rev1=1 crypto rx=0x7f4df0034000 tx=0x7f4df002e000 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:29:36.544 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:36.542+0000 7f4df6ffd640 1 -- 172.21.15.37:0/3439543228 <== mon.0 v2:172.21.15.37:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187246 (secure 0 0 0) 0x7f4de805f880 con 0x7f4df80ce090 2024-04-25T10:29:36.732 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:36.729+0000 7f4e00669640 1 -- 172.21.15.37:0/3439543228 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) -- 0x7f4df80d3f50 con 0x7f4df80ce090 2024-04-25T10:29:36.732 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:36.730+0000 7f4df6ffd640 1 -- 172.21.15.37:0/3439543228 <== mon.0 v2:172.21.15.37:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v23) ==== 93+0+10 (secure 0 0 0) 0x7f4de805f180 con 0x7f4df80ce090 2024-04-25T10:29:36.732 INFO:teuthology.orchestra.run.smithi037.stdout:pg_num: 1 2024-04-25T10:29:36.734 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:36.732+0000 7f4e00669640 1 -- 172.21.15.37:0/3439543228 >> [v2:172.21.15.203:6832/2418205237,v1:172.21.15.203:6833/2418205237] conn(0x7f4de40744e0 msgr2=0x7f4de40769f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:29:36.734 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:36.732+0000 7f4e00669640 1 --2- 172.21.15.37:0/3439543228 >> [v2:172.21.15.203:6832/2418205237,v1:172.21.15.203:6833/2418205237] conn(0x7f4de40744e0 0x7f4de40769f0 secure :-1 s=READY pgs=118 cs=0 l=1 rev1=1 crypto rx=0x7f4df0034000 tx=0x7f4df002e000 comp rx=0 tx=0).stop 2024-04-25T10:29:36.734 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:36.732+0000 7f4e00669640 1 -- 172.21.15.37:0/3439543228 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f4df80ce090 msgr2=0x7f4df80b6650 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:29:36.734 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:36.732+0000 7f4e00669640 1 --2- 172.21.15.37:0/3439543228 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f4df80ce090 0x7f4df80b6650 secure :-1 s=READY pgs=197 cs=0 l=1 rev1=1 crypto rx=0x7f4de8001350 tx=0x7f4de800a430 comp rx=0 tx=0).stop 2024-04-25T10:29:36.734 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:36.732+0000 7f4e00669640 1 -- 172.21.15.37:0/3439543228 shutdown_connections 2024-04-25T10:29:36.735 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:36.732+0000 7f4e00669640 1 --2- 172.21.15.37:0/3439543228 >> [v2:172.21.15.203:6832/2418205237,v1:172.21.15.203:6833/2418205237] conn(0x7f4de40744e0 0x7f4de40769f0 unknown :-1 s=CLOSED pgs=118 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:29:36.735 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:36.732+0000 7f4e00669640 1 --2- 172.21.15.37:0/3439543228 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f4df80ce090 0x7f4df80b6650 unknown :-1 s=CLOSED pgs=197 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:29:36.735 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:36.732+0000 7f4e00669640 1 --2- 172.21.15.37:0/3439543228 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f4df80d5160 0x7f4df8157cf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:29:36.735 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:36.732+0000 7f4e00669640 1 --2- 172.21.15.37:0/3439543228 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f4df80d0990 0x7f4df81625a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:29:36.735 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:36.732+0000 7f4e00669640 1 -- 172.21.15.37:0/3439543228 >> 172.21.15.37:0/3439543228 conn(0x7f4df80b7a80 msgr2=0x7f4df80b9fc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:29:36.735 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:36.732+0000 7f4e00669640 1 -- 172.21.15.37:0/3439543228 shutdown_connections 2024-04-25T10:29:36.735 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:36.732+0000 7f4e00669640 1 -- 172.21.15.37:0/3439543228 wait complete. 2024-04-25T10:29:36.745 DEBUG:teuthology.orchestra.run.smithi037:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_metadata pg_num 2024-04-25T10:29:36.860 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:36.857+0000 7f5b8a5d7640 1 Processor -- start 2024-04-25T10:29:36.860 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:36.857+0000 7f5b8a5d7640 1 -- start start 2024-04-25T10:29:36.862 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:36.859+0000 7f5b8a5d7640 1 --2- >> v2:172.21.15.203:3300/0 conn(0x7f5b840b4e80 0x7f5b840d2b20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:29:36.862 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:36.859+0000 7f5b8a5d7640 1 --2- >> v2:172.21.15.37:3300/0 conn(0x7f5b840d3060 0x7f5b840b9060 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:29:36.862 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:36.859+0000 7f5b8a5d7640 1 -- --> v1:172.21.15.37:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f5b840d3470 con 0x7f5b840d5160 2024-04-25T10:29:36.862 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:36.859+0000 7f5b8a5d7640 1 -- --> v2:172.21.15.37:3300/0 -- mon_getmap magic: 0 -- 0x7f5b840d1650 con 0x7f5b840d3060 2024-04-25T10:29:36.862 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:36.859+0000 7f5b8a5d7640 1 -- --> v2:172.21.15.203:3300/0 -- mon_getmap magic: 0 -- 0x7f5b840b5290 con 0x7f5b840b4e80 2024-04-25T10:29:36.863 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:36.859+0000 7f5b83fff640 1 --1- >> v1:172.21.15.37:6789/0 conn(0x7f5b840d5160 0x7f5b840d0990 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.37:6789/0 says I am v1:172.21.15.37:46522/0 (socket says 172.21.15.37:46522) 2024-04-25T10:29:36.863 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:36.859+0000 7f5b83fff640 1 -- 172.21.15.37:0/2542654767 learned_addr learned my addr 172.21.15.37:0/2542654767 (peer_addr_for_me v1:172.21.15.37:0/0) 2024-04-25T10:29:36.863 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:36.859+0000 7f5b88b74640 1 --2- 172.21.15.37:0/2542654767 >> v2:172.21.15.37:3300/0 conn(0x7f5b840d3060 0x7f5b840b9060 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:29:36.863 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:36.859+0000 7f5b837fe640 1 --2- 172.21.15.37:0/2542654767 >> v2:172.21.15.203:3300/0 conn(0x7f5b840b4e80 0x7f5b840d2b20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:29:36.863 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:36.859+0000 7f5b82ffd640 1 -- 172.21.15.37:0/2542654767 <== mon.0 v1:172.21.15.37:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 233848040 0 0) 0x7f5b6c001110 con 0x7f5b840d5160 2024-04-25T10:29:36.863 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:36.859+0000 7f5b82ffd640 1 -- 172.21.15.37:0/2542654767 --> v1:172.21.15.37:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f5b70003520 con 0x7f5b840d5160 2024-04-25T10:29:36.863 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:36.859+0000 7f5b88b74640 1 -- 172.21.15.37:0/2542654767 >> v2:172.21.15.203:3300/0 conn(0x7f5b840b4e80 msgr2=0x7f5b840d2b20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:29:36.863 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:36.859+0000 7f5b88b74640 1 --2- 172.21.15.37:0/2542654767 >> v2:172.21.15.203:3300/0 conn(0x7f5b840b4e80 0x7f5b840d2b20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:29:36.863 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:36.859+0000 7f5b88b74640 1 -- 172.21.15.37:0/2542654767 >> v1:172.21.15.37:6789/0 conn(0x7f5b840d5160 legacy=0x7f5b840d0990 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:29:36.863 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:36.859+0000 7f5b88b74640 1 -- 172.21.15.37:0/2542654767 --> v2:172.21.15.37:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f5b840d10a0 con 0x7f5b840d3060 2024-04-25T10:29:36.863 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:36.860+0000 7f5b88b74640 1 --2- 172.21.15.37:0/2542654767 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f5b840d3060 0x7f5b840b9060 secure :-1 s=READY pgs=199 cs=0 l=1 rev1=1 crypto rx=0x7f5b840d6b80 tx=0x7f5b7c02a5f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=25c8b3311fb68938 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:29:36.863 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:36.860+0000 7f5b82ffd640 1 -- 172.21.15.37:0/2542654767 <== mon.0 v2:172.21.15.37:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5b7c007740 con 0x7f5b840d3060 2024-04-25T10:29:36.863 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:36.860+0000 7f5b82ffd640 1 -- 172.21.15.37:0/2542654767 <== mon.0 v2:172.21.15.37:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f5b7c0078e0 con 0x7f5b840d3060 2024-04-25T10:29:36.863 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:36.860+0000 7f5b82ffd640 1 -- 172.21.15.37:0/2542654767 <== mon.0 v2:172.21.15.37:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5b7c007cb0 con 0x7f5b840d3060 2024-04-25T10:29:36.864 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:36.861+0000 7f5b8a5d7640 1 -- 172.21.15.37:0/2542654767 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f5b840d3060 msgr2=0x7f5b840b9060 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:29:36.865 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:36.861+0000 7f5b8a5d7640 1 --2- 172.21.15.37:0/2542654767 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f5b840d3060 0x7f5b840b9060 secure :-1 s=READY pgs=199 cs=0 l=1 rev1=1 crypto rx=0x7f5b840d6b80 tx=0x7f5b7c02a5f0 comp rx=0 tx=0).stop 2024-04-25T10:29:36.865 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:36.861+0000 7f5b8a5d7640 1 -- 172.21.15.37:0/2542654767 shutdown_connections 2024-04-25T10:29:36.865 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:36.861+0000 7f5b8a5d7640 1 --2- 172.21.15.37:0/2542654767 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f5b840d3060 0x7f5b840b9060 unknown :-1 s=CLOSED pgs=199 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:29:36.865 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:36.861+0000 7f5b8a5d7640 1 --2- 172.21.15.37:0/2542654767 >> v2:172.21.15.203:3300/0 conn(0x7f5b840b4e80 0x7f5b840d2b20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:29:36.865 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:36.861+0000 7f5b8a5d7640 1 -- 172.21.15.37:0/2542654767 >> 172.21.15.37:0/2542654767 conn(0x7f5b840b7a80 msgr2=0x7f5b840b7e90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:29:36.865 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:36.861+0000 7f5b8a5d7640 1 -- 172.21.15.37:0/2542654767 shutdown_connections 2024-04-25T10:29:36.865 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:36.861+0000 7f5b8a5d7640 1 -- 172.21.15.37:0/2542654767 wait complete. 2024-04-25T10:29:36.865 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:36.863+0000 7f5b8a5d7640 1 Processor -- start 2024-04-25T10:29:36.865 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:36.863+0000 7f5b8a5d7640 1 -- start start 2024-04-25T10:29:36.868 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:36.864+0000 7f5b8a5d7640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f5b840b4e80 0x7f5b841eb3e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:29:36.868 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:36.864+0000 7f5b8a5d7640 1 --2- >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f5b840d3060 0x7f5b841fa1d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:29:36.868 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:36.864+0000 7f5b8a5d7640 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f5b840d5160 0x7f5b841fa710 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:29:36.868 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:36.864+0000 7f5b8a5d7640 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 -- 0x7f5b840d4960 con 0x7f5b840b4e80 2024-04-25T10:29:36.868 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:36.864+0000 7f5b8a5d7640 1 -- --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_getmap magic: 0 -- 0x7f5b840d4680 con 0x7f5b840d5160 2024-04-25T10:29:36.868 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:36.864+0000 7f5b8a5d7640 1 -- --> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] -- mon_getmap magic: 0 -- 0x7f5b840d4c40 con 0x7f5b840d3060 2024-04-25T10:29:36.868 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:36.864+0000 7f5b83fff640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f5b840b4e80 0x7f5b841eb3e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:29:36.868 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:36.864+0000 7f5b83fff640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f5b840b4e80 0x7f5b841eb3e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.37:3300/0 says I am v2:172.21.15.37:36824/0 (socket says 172.21.15.37:36824) 2024-04-25T10:29:36.868 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:36.864+0000 7f5b83fff640 1 -- 172.21.15.37:0/2492229687 learned_addr learned my addr 172.21.15.37:0/2492229687 (peer_addr_for_me v2:172.21.15.37:0/0) 2024-04-25T10:29:36.868 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:36.865+0000 7f5b83fff640 1 -- 172.21.15.37:0/2492229687 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f5b840d3060 msgr2=0x7f5b841fa1d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:29:36.868 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:36.865+0000 7f5b83fff640 1 --2- 172.21.15.37:0/2492229687 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f5b840d3060 0x7f5b841fa1d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:29:36.868 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:36.865+0000 7f5b83fff640 1 -- 172.21.15.37:0/2492229687 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f5b840d5160 msgr2=0x7f5b841fa710 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:29:36.868 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:36.865+0000 7f5b83fff640 1 --2- 172.21.15.37:0/2492229687 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f5b840d5160 0x7f5b841fa710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:29:36.869 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:36.865+0000 7f5b83fff640 1 -- 172.21.15.37:0/2492229687 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f5b841eb920 con 0x7f5b840b4e80 2024-04-25T10:29:36.869 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:36.865+0000 7f5b83fff640 1 --2- 172.21.15.37:0/2492229687 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f5b840b4e80 0x7f5b841eb3e0 secure :-1 s=READY pgs=200 cs=0 l=1 rev1=1 crypto rx=0x7f5b6c00c3d0 tx=0x7f5b6c00c890 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:29:36.869 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:36.865+0000 7f5b80ff9640 1 -- 172.21.15.37:0/2492229687 <== mon.0 v2:172.21.15.37:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5b6c017070 con 0x7f5b840b4e80 2024-04-25T10:29:36.869 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:36.865+0000 7f5b80ff9640 1 -- 172.21.15.37:0/2492229687 <== mon.0 v2:172.21.15.37:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f5b6c005ce0 con 0x7f5b840b4e80 2024-04-25T10:29:36.869 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:36.865+0000 7f5b8a5d7640 1 -- 172.21.15.37:0/2492229687 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f5b841e9f30 con 0x7f5b840b4e80 2024-04-25T10:29:36.869 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:36.865+0000 7f5b80ff9640 1 -- 172.21.15.37:0/2492229687 <== mon.0 v2:172.21.15.37:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5b6c007310 con 0x7f5b840b4e80 2024-04-25T10:29:36.869 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:36.865+0000 7f5b8a5d7640 1 -- 172.21.15.37:0/2492229687 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f5b841ea4f0 con 0x7f5b840b4e80 2024-04-25T10:29:36.870 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:36.866+0000 7f5b8a5d7640 1 -- 172.21.15.37:0/2492229687 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f5b8406a140 con 0x7f5b840b4e80 2024-04-25T10:29:36.870 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:36.867+0000 7f5b80ff9640 1 -- 172.21.15.37:0/2492229687 <== mon.0 v2:172.21.15.37:3300/0 4 ==== mgrmap(e 5) ==== 96782+0+0 (secure 0 0 0) 0x7f5b6c01a070 con 0x7f5b840b4e80 2024-04-25T10:29:36.871 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:36.867+0000 7f5b80ff9640 1 --2- 172.21.15.37:0/2492229687 >> [v2:172.21.15.203:6832/2418205237,v1:172.21.15.203:6833/2418205237] conn(0x7f5b640744e0 0x7f5b640769f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:29:36.871 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:36.867+0000 7f5b80ff9640 1 -- 172.21.15.37:0/2492229687 <== mon.0 v2:172.21.15.37:3300/0 5 ==== osd_map(23..23 src has 1..23) ==== 5709+0+0 (secure 0 0 0) 0x7f5b6c094620 con 0x7f5b840b4e80 2024-04-25T10:29:36.871 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:36.867+0000 7f5b837fe640 1 --2- 172.21.15.37:0/2492229687 >> [v2:172.21.15.203:6832/2418205237,v1:172.21.15.203:6833/2418205237] conn(0x7f5b640744e0 0x7f5b640769f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:29:36.871 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:36.868+0000 7f5b837fe640 1 --2- 172.21.15.37:0/2492229687 >> [v2:172.21.15.203:6832/2418205237,v1:172.21.15.203:6833/2418205237] conn(0x7f5b640744e0 0x7f5b640769f0 secure :-1 s=READY pgs=119 cs=0 l=1 rev1=1 crypto rx=0x7f5b74006fd0 tx=0x7f5b74008040 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:29:36.874 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:36.872+0000 7f5b80ff9640 1 -- 172.21.15.37:0/2492229687 <== mon.0 v2:172.21.15.37:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187246 (secure 0 0 0) 0x7f5b840d3470 con 0x7f5b840b4e80 2024-04-25T10:29:37.058 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:37.055+0000 7f5b8a5d7640 1 -- 172.21.15.37:0/2492229687 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"} v 0) -- 0x7f5b841ebb50 con 0x7f5b840b4e80 2024-04-25T10:29:37.058 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:37.056+0000 7f5b80ff9640 1 -- 172.21.15.37:0/2492229687 <== mon.0 v2:172.21.15.37:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]=0 v23) ==== 104+0+11 (secure 0 0 0) 0x7f5b6c05edb0 con 0x7f5b840b4e80 2024-04-25T10:29:37.058 INFO:teuthology.orchestra.run.smithi037.stdout:pg_num: 32 2024-04-25T10:29:37.060 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:37.058+0000 7f5b8a5d7640 1 -- 172.21.15.37:0/2492229687 >> [v2:172.21.15.203:6832/2418205237,v1:172.21.15.203:6833/2418205237] conn(0x7f5b640744e0 msgr2=0x7f5b640769f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:29:37.060 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:37.058+0000 7f5b8a5d7640 1 --2- 172.21.15.37:0/2492229687 >> [v2:172.21.15.203:6832/2418205237,v1:172.21.15.203:6833/2418205237] conn(0x7f5b640744e0 0x7f5b640769f0 secure :-1 s=READY pgs=119 cs=0 l=1 rev1=1 crypto rx=0x7f5b74006fd0 tx=0x7f5b74008040 comp rx=0 tx=0).stop 2024-04-25T10:29:37.060 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:37.058+0000 7f5b8a5d7640 1 -- 172.21.15.37:0/2492229687 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f5b840b4e80 msgr2=0x7f5b841eb3e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:29:37.060 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:37.058+0000 7f5b8a5d7640 1 --2- 172.21.15.37:0/2492229687 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f5b840b4e80 0x7f5b841eb3e0 secure :-1 s=READY pgs=200 cs=0 l=1 rev1=1 crypto rx=0x7f5b6c00c3d0 tx=0x7f5b6c00c890 comp rx=0 tx=0).stop 2024-04-25T10:29:37.061 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:37.058+0000 7f5b8a5d7640 1 -- 172.21.15.37:0/2492229687 shutdown_connections 2024-04-25T10:29:37.061 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:37.058+0000 7f5b8a5d7640 1 --2- 172.21.15.37:0/2492229687 >> [v2:172.21.15.203:6832/2418205237,v1:172.21.15.203:6833/2418205237] conn(0x7f5b640744e0 0x7f5b640769f0 unknown :-1 s=CLOSED pgs=119 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:29:37.061 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:37.058+0000 7f5b8a5d7640 1 --2- 172.21.15.37:0/2492229687 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f5b840d5160 0x7f5b841fa710 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:29:37.061 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:37.058+0000 7f5b8a5d7640 1 --2- 172.21.15.37:0/2492229687 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f5b840d3060 0x7f5b841fa1d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:29:37.061 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:37.058+0000 7f5b8a5d7640 1 --2- 172.21.15.37:0/2492229687 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f5b840b4e80 0x7f5b841eb3e0 unknown :-1 s=CLOSED pgs=200 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:29:37.061 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:37.058+0000 7f5b8a5d7640 1 -- 172.21.15.37:0/2492229687 >> 172.21.15.37:0/2492229687 conn(0x7f5b840b7a80 msgr2=0x7f5b840b54d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:29:37.061 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:37.058+0000 7f5b8a5d7640 1 -- 172.21.15.37:0/2492229687 shutdown_connections 2024-04-25T10:29:37.061 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:37.058+0000 7f5b8a5d7640 1 -- 172.21.15.37:0/2492229687 wait complete. 2024-04-25T10:29:37.071 DEBUG:teuthology.orchestra.run.smithi037:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data pg_num 2024-04-25T10:29:37.185 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:37.183+0000 7f70c9e51640 1 Processor -- start 2024-04-25T10:29:37.185 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:37.183+0000 7f70c9e51640 1 -- start start 2024-04-25T10:29:37.187 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:37.184+0000 7f70c9e51640 1 --2- >> v2:172.21.15.37:3300/0 conn(0x7f70c40b4bb0 0x7f70c41646b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:29:37.187 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:37.184+0000 7f70c9e51640 1 -- --> v1:172.21.15.37:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f70c40d5080 con 0x7f70c40d4560 2024-04-25T10:29:37.187 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:37.184+0000 7f70c9e51640 1 -- --> v1:172.21.15.203:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f70c4161190 con 0x7f70c40b7600 2024-04-25T10:29:37.187 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:37.184+0000 7f70c9e51640 1 -- --> v2:172.21.15.37:3300/0 -- mon_getmap magic: 0 -- 0x7f70c4161470 con 0x7f70c40b4bb0 2024-04-25T10:29:37.187 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:37.184+0000 7f70c37fe640 1 --2- >> v2:172.21.15.37:3300/0 conn(0x7f70c40b4bb0 0x7f70c41646b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:29:37.187 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:37.184+0000 7f70c3fff640 1 --1- >> v1:172.21.15.37:6789/0 conn(0x7f70c40d4560 0x7f70c40d4970 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.37:6789/0 says I am v1:172.21.15.37:46532/0 (socket says 172.21.15.37:46532) 2024-04-25T10:29:37.187 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:37.184+0000 7f70c3fff640 1 -- 172.21.15.37:0/2553206482 learned_addr learned my addr 172.21.15.37:0/2553206482 (peer_addr_for_me v1:172.21.15.37:0/0) 2024-04-25T10:29:37.187 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:37.184+0000 7f70c37fe640 1 --2- >> v2:172.21.15.37:3300/0 conn(0x7f70c40b4bb0 0x7f70c41646b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.37:3300/0 says I am v2:172.21.15.37:36830/0 (socket says 172.21.15.37:36830) 2024-04-25T10:29:37.188 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:37.185+0000 7f70c37fe640 1 -- 172.21.15.37:0/2553206482 >> v1:172.21.15.203:6789/0 conn(0x7f70c40b7600 legacy=0x7f70c40b7a10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:29:37.188 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:37.185+0000 7f70c37fe640 1 -- 172.21.15.37:0/2553206482 >> v1:172.21.15.37:6789/0 conn(0x7f70c40d4560 legacy=0x7f70c40d4970 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:29:37.188 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:37.185+0000 7f70c27fc640 1 -- 172.21.15.37:0/2553206482 <== mon.0 v1:172.21.15.37:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 1736358789 0 0) 0x7f70c40d5080 con 0x7f70c40d4560 2024-04-25T10:29:37.188 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:37.185+0000 7f70c37fe640 1 -- 172.21.15.37:0/2553206482 --> v2:172.21.15.37:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f70c4163eb0 con 0x7f70c40b4bb0 2024-04-25T10:29:37.189 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:37.185+0000 7f70c37fe640 1 --2- 172.21.15.37:0/2553206482 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f70c40b4bb0 0x7f70c41646b0 secure :-1 s=READY pgs=202 cs=0 l=1 rev1=1 crypto rx=0x7f70b40097a0 tx=0x7f70b402a1e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d6df8db28224fbb server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:29:37.189 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:37.186+0000 7f70c27fc640 1 -- 172.21.15.37:0/2553206482 <== mon.0 v2:172.21.15.37:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f70b4037070 con 0x7f70c40b4bb0 2024-04-25T10:29:37.189 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:37.186+0000 7f70c27fc640 1 -- 172.21.15.37:0/2553206482 <== mon.0 v2:172.21.15.37:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f70b4032440 con 0x7f70c40b4bb0 2024-04-25T10:29:37.189 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:37.186+0000 7f70c27fc640 1 -- 172.21.15.37:0/2553206482 <== mon.0 v2:172.21.15.37:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f70b4032810 con 0x7f70c40b4bb0 2024-04-25T10:29:37.189 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:37.186+0000 7f70c9e51640 1 -- 172.21.15.37:0/2553206482 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f70c40b4bb0 msgr2=0x7f70c41646b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:29:37.189 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:37.186+0000 7f70c9e51640 1 --2- 172.21.15.37:0/2553206482 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f70c40b4bb0 0x7f70c41646b0 secure :-1 s=READY pgs=202 cs=0 l=1 rev1=1 crypto rx=0x7f70b40097a0 tx=0x7f70b402a1e0 comp rx=0 tx=0).stop 2024-04-25T10:29:37.189 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:37.186+0000 7f70c9e51640 1 -- 172.21.15.37:0/2553206482 shutdown_connections 2024-04-25T10:29:37.189 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:37.186+0000 7f70c9e51640 1 --2- 172.21.15.37:0/2553206482 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f70c40b4bb0 0x7f70c41646b0 unknown :-1 s=CLOSED pgs=202 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:29:37.189 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:37.186+0000 7f70c9e51640 1 -- 172.21.15.37:0/2553206482 >> 172.21.15.37:0/2553206482 conn(0x7f70c40d27e0 msgr2=0x7f70c40d2bf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:29:37.189 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:37.186+0000 7f70c9e51640 1 -- 172.21.15.37:0/2553206482 shutdown_connections 2024-04-25T10:29:37.190 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:37.186+0000 7f70c9e51640 1 -- 172.21.15.37:0/2553206482 wait complete. 2024-04-25T10:29:37.190 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:37.188+0000 7f70c9e51640 1 Processor -- start 2024-04-25T10:29:37.190 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:37.188+0000 7f70c9e51640 1 -- start start 2024-04-25T10:29:37.193 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:37.189+0000 7f70c9e51640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f70c40b4bb0 0x7f70c41f9e70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:29:37.193 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:37.189+0000 7f70c9e51640 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f70c40b7600 0x7f70c40cb070 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:29:37.193 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:37.190+0000 7f70c9e51640 1 --2- >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f70c40d4560 0x7f70c415f3a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:29:37.193 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:37.190+0000 7f70c9e51640 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 -- 0x7f70c4164bf0 con 0x7f70c40b4bb0 2024-04-25T10:29:37.193 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:37.190+0000 7f70c9e51640 1 -- --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_getmap magic: 0 -- 0x7f70c40b4fc0 con 0x7f70c40b7600 2024-04-25T10:29:37.193 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:37.190+0000 7f70c9e51640 1 -- --> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] -- mon_getmap magic: 0 -- 0x7f70c40d09e0 con 0x7f70c40d4560 2024-04-25T10:29:37.193 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:37.190+0000 7f70c37fe640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f70c40b4bb0 0x7f70c41f9e70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:29:37.194 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:37.190+0000 7f70c37fe640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f70c40b4bb0 0x7f70c41f9e70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.37:3300/0 says I am v2:172.21.15.37:36842/0 (socket says 172.21.15.37:36842) 2024-04-25T10:29:37.194 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:37.190+0000 7f70c37fe640 1 -- 172.21.15.37:0/2947146757 learned_addr learned my addr 172.21.15.37:0/2947146757 (peer_addr_for_me v2:172.21.15.37:0/0) 2024-04-25T10:29:37.194 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:37.190+0000 7f70c3fff640 1 --2- 172.21.15.37:0/2947146757 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f70c40d4560 0x7f70c415f3a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:29:37.194 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:37.190+0000 7f70c2ffd640 1 --2- 172.21.15.37:0/2947146757 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f70c40b7600 0x7f70c40cb070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:29:37.194 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:37.190+0000 7f70c37fe640 1 -- 172.21.15.37:0/2947146757 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f70c40d4560 msgr2=0x7f70c415f3a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:29:37.194 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:37.190+0000 7f70c37fe640 1 --2- 172.21.15.37:0/2947146757 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f70c40d4560 0x7f70c415f3a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:29:37.194 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:37.190+0000 7f70c37fe640 1 -- 172.21.15.37:0/2947146757 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f70c40b7600 msgr2=0x7f70c40cb070 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:29:37.194 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:37.190+0000 7f70c37fe640 1 --2- 172.21.15.37:0/2947146757 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f70c40b7600 0x7f70c40cb070 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:29:37.194 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:37.190+0000 7f70c37fe640 1 -- 172.21.15.37:0/2947146757 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f70c41e9df0 con 0x7f70c40b4bb0 2024-04-25T10:29:37.194 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:37.190+0000 7f70c37fe640 1 --2- 172.21.15.37:0/2947146757 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f70c40b4bb0 0x7f70c41f9e70 secure :-1 s=READY pgs=203 cs=0 l=1 rev1=1 crypto rx=0x7f70b40097a0 tx=0x7f70b40046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:29:37.194 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:37.190+0000 7f70abfff640 1 -- 172.21.15.37:0/2947146757 <== mon.0 v2:172.21.15.37:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f70b4041070 con 0x7f70c40b4bb0 2024-04-25T10:29:37.194 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:37.190+0000 7f70abfff640 1 -- 172.21.15.37:0/2947146757 <== mon.0 v2:172.21.15.37:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f70b403c870 con 0x7f70c40b4bb0 2024-04-25T10:29:37.194 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:37.190+0000 7f70c9e51640 1 -- 172.21.15.37:0/2947146757 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f70c41ea0d0 con 0x7f70c40b4bb0 2024-04-25T10:29:37.195 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:37.190+0000 7f70abfff640 1 -- 172.21.15.37:0/2947146757 <== mon.0 v2:172.21.15.37:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f70b4037040 con 0x7f70c40b4bb0 2024-04-25T10:29:37.195 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:37.190+0000 7f70c9e51640 1 -- 172.21.15.37:0/2947146757 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f70c41ea690 con 0x7f70c40b4bb0 2024-04-25T10:29:37.196 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:37.191+0000 7f70c9e51640 1 -- 172.21.15.37:0/2947146757 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f70c406a140 con 0x7f70c40b4bb0 2024-04-25T10:29:37.196 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:37.192+0000 7f70abfff640 1 -- 172.21.15.37:0/2947146757 <== mon.0 v2:172.21.15.37:3300/0 4 ==== mgrmap(e 5) ==== 96782+0+0 (secure 0 0 0) 0x7f70b4044070 con 0x7f70c40b4bb0 2024-04-25T10:29:37.196 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:37.193+0000 7f70abfff640 1 --2- 172.21.15.37:0/2947146757 >> [v2:172.21.15.203:6832/2418205237,v1:172.21.15.203:6833/2418205237] conn(0x7f70b00744b0 0x7f70b00769c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:29:37.196 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:37.193+0000 7f70abfff640 1 -- 172.21.15.37:0/2947146757 <== mon.0 v2:172.21.15.37:3300/0 5 ==== osd_map(23..23 src has 1..23) ==== 5709+0+0 (secure 0 0 0) 0x7f70b40b5e30 con 0x7f70c40b4bb0 2024-04-25T10:29:37.196 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:37.193+0000 7f70c2ffd640 1 --2- 172.21.15.37:0/2947146757 >> [v2:172.21.15.203:6832/2418205237,v1:172.21.15.203:6833/2418205237] conn(0x7f70b00744b0 0x7f70b00769c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:29:37.196 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:37.193+0000 7f70c2ffd640 1 --2- 172.21.15.37:0/2947146757 >> [v2:172.21.15.203:6832/2418205237,v1:172.21.15.203:6833/2418205237] conn(0x7f70b00744b0 0x7f70b00769c0 secure :-1 s=READY pgs=120 cs=0 l=1 rev1=1 crypto rx=0x7f70b8006fd0 tx=0x7f70b8008040 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:29:37.200 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:37.198+0000 7f70abfff640 1 -- 172.21.15.37:0/2947146757 <== mon.0 v2:172.21.15.37:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187246 (secure 0 0 0) 0x7f70b40805c0 con 0x7f70c40b4bb0 2024-04-25T10:29:37.385 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:37.383+0000 7f70c9e51640 1 -- 172.21.15.37:0/2947146757 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"} v 0) -- 0x7f70c40cc850 con 0x7f70c40b4bb0 2024-04-25T10:29:37.386 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:37.383+0000 7f70abfff640 1 -- 172.21.15.37:0/2947146757 <== mon.0 v2:172.21.15.37:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]=0 v23) ==== 100+0+11 (secure 0 0 0) 0x7f70b407fec0 con 0x7f70c40b4bb0 2024-04-25T10:29:37.386 INFO:teuthology.orchestra.run.smithi037.stdout:pg_num: 64 2024-04-25T10:29:37.388 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:37.385+0000 7f70c9e51640 1 -- 172.21.15.37:0/2947146757 >> [v2:172.21.15.203:6832/2418205237,v1:172.21.15.203:6833/2418205237] conn(0x7f70b00744b0 msgr2=0x7f70b00769c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:29:37.388 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:37.385+0000 7f70c9e51640 1 --2- 172.21.15.37:0/2947146757 >> [v2:172.21.15.203:6832/2418205237,v1:172.21.15.203:6833/2418205237] conn(0x7f70b00744b0 0x7f70b00769c0 secure :-1 s=READY pgs=120 cs=0 l=1 rev1=1 crypto rx=0x7f70b8006fd0 tx=0x7f70b8008040 comp rx=0 tx=0).stop 2024-04-25T10:29:37.388 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:37.385+0000 7f70c9e51640 1 -- 172.21.15.37:0/2947146757 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f70c40b4bb0 msgr2=0x7f70c41f9e70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:29:37.389 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:37.385+0000 7f70c9e51640 1 --2- 172.21.15.37:0/2947146757 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f70c40b4bb0 0x7f70c41f9e70 secure :-1 s=READY pgs=203 cs=0 l=1 rev1=1 crypto rx=0x7f70b40097a0 tx=0x7f70b40046c0 comp rx=0 tx=0).stop 2024-04-25T10:29:37.389 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:37.385+0000 7f70c9e51640 1 -- 172.21.15.37:0/2947146757 shutdown_connections 2024-04-25T10:29:37.389 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:37.385+0000 7f70c9e51640 1 --2- 172.21.15.37:0/2947146757 >> [v2:172.21.15.203:6832/2418205237,v1:172.21.15.203:6833/2418205237] conn(0x7f70b00744b0 0x7f70b00769c0 unknown :-1 s=CLOSED pgs=120 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:29:37.389 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:37.385+0000 7f70c9e51640 1 --2- 172.21.15.37:0/2947146757 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f70c40d4560 0x7f70c415f3a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:29:37.389 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:37.385+0000 7f70c9e51640 1 --2- 172.21.15.37:0/2947146757 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f70c40b7600 0x7f70c40cb070 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:29:37.389 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:37.385+0000 7f70c9e51640 1 --2- 172.21.15.37:0/2947146757 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f70c40b4bb0 0x7f70c41f9e70 unknown :-1 s=CLOSED pgs=203 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:29:37.389 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:37.385+0000 7f70c9e51640 1 -- 172.21.15.37:0/2947146757 >> 172.21.15.37:0/2947146757 conn(0x7f70c40d27e0 msgr2=0x7f70c40b8270 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:29:37.389 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:37.386+0000 7f70c9e51640 1 -- 172.21.15.37:0/2947146757 shutdown_connections 2024-04-25T10:29:37.389 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:37.386+0000 7f70c9e51640 1 -- 172.21.15.37:0/2947146757 wait complete. 2024-04-25T10:29:37.399 DEBUG:teuthology.orchestra.run.smithi037:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-04-25T10:29:37.510 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:37.507+0000 7fe8a1da8640 1 Processor -- start 2024-04-25T10:29:37.510 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:37.508+0000 7fe8a1da8640 1 -- start start 2024-04-25T10:29:37.513 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:37.509+0000 7fe8a1da8640 1 --2- >> v2:172.21.15.203:3300/0 conn(0x7fe89c0d5160 0x7fe89c0b4e80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:29:37.513 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:37.509+0000 7fe8a1da8640 1 --2- >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7fe89c0d0990 0x7fe89c0d2b20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:29:37.513 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:37.509+0000 7fe8a1da8640 1 -- --> v1:172.21.15.37:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fe89c0d1370 con 0x7fe89c0d0da0 2024-04-25T10:29:37.514 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:37.509+0000 7fe8a1da8640 1 -- --> v2:172.21.15.203:3300/0 -- mon_getmap magic: 0 -- 0x7fe89c0d1650 con 0x7fe89c0d5160 2024-04-25T10:29:37.514 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:37.509+0000 7fe8a1da8640 1 -- --> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] -- mon_getmap magic: 0 -- 0x7fe89c0d4680 con 0x7fe89c0d0990 2024-04-25T10:29:37.514 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:37.509+0000 7fe89bfff640 1 --1- >> v1:172.21.15.37:6789/0 conn(0x7fe89c0d0da0 0x7fe89c06db10 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.37:6789/0 says I am v1:172.21.15.37:46536/0 (socket says 172.21.15.37:46536) 2024-04-25T10:29:37.514 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:37.509+0000 7fe89bfff640 1 -- 172.21.15.37:0/2187644944 learned_addr learned my addr 172.21.15.37:0/2187644944 (peer_addr_for_me v1:172.21.15.37:0/0) 2024-04-25T10:29:37.514 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:37.509+0000 7fe89affd640 1 --2- 172.21.15.37:0/2187644944 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7fe89c0d0990 0x7fe89c0d2b20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:29:37.514 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:37.509+0000 7fe89b7fe640 1 --2- 172.21.15.37:0/2187644944 >> v2:172.21.15.203:3300/0 conn(0x7fe89c0d5160 0x7fe89c0b4e80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:29:37.514 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:37.509+0000 7fe89a7fc640 1 -- 172.21.15.37:0/2187644944 <== mon.0 v1:172.21.15.37:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 1271089602 0 0) 0x7fe89c0d1370 con 0x7fe89c0d0da0 2024-04-25T10:29:37.514 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:37.509+0000 7fe89a7fc640 1 -- 172.21.15.37:0/2187644944 --> v1:172.21.15.37:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7fe884003520 con 0x7fe89c0d0da0 2024-04-25T10:29:37.514 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:37.510+0000 7fe89a7fc640 1 -- 172.21.15.37:0/2187644944 <== mon.0 v1:172.21.15.37:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 760959110 0 0) 0x7fe884003520 con 0x7fe89c0d0da0 2024-04-25T10:29:37.514 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:37.510+0000 7fe89a7fc640 1 -- 172.21.15.37:0/2187644944 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7fe89c0d0990 msgr2=0x7fe89c0d2b20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:29:37.514 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:37.510+0000 7fe89a7fc640 1 --2- 172.21.15.37:0/2187644944 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7fe89c0d0990 0x7fe89c0d2b20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:29:37.514 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:37.510+0000 7fe89a7fc640 1 -- 172.21.15.37:0/2187644944 >> v2:172.21.15.203:3300/0 conn(0x7fe89c0d5160 msgr2=0x7fe89c0b4e80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:29:37.514 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:37.510+0000 7fe89a7fc640 1 --2- 172.21.15.37:0/2187644944 >> v2:172.21.15.203:3300/0 conn(0x7fe89c0d5160 0x7fe89c0b4e80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:29:37.514 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:37.510+0000 7fe89a7fc640 1 -- 172.21.15.37:0/2187644944 --> v1:172.21.15.37:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fe89c0d4960 con 0x7fe89c0d0da0 2024-04-25T10:29:37.515 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:37.510+0000 7fe89a7fc640 1 -- 172.21.15.37:0/2187644944 <== mon.0 v1:172.21.15.37:6789/0 3 ==== mon_map magic: 0 ==== 413+0+0 (unknown 1577794600 0 0) 0x7fe894001a10 con 0x7fe89c0d0da0 2024-04-25T10:29:37.515 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:37.510+0000 7fe89a7fc640 1 -- 172.21.15.37:0/2187644944 >> v1:172.21.15.37:6789/0 conn(0x7fe89c0d0da0 legacy=0x7fe89c06db10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:29:37.515 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:37.510+0000 7fe89a7fc640 1 --2- 172.21.15.37:0/2187644944 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fe8840039b0 0x7fe884003e20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:29:37.515 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:37.510+0000 7fe89a7fc640 1 --2- 172.21.15.37:0/2187644944 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7fe89c0d0990 0x7fe884004800 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:29:37.515 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:37.510+0000 7fe89a7fc640 1 --2- 172.21.15.37:0/2187644944 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fe884004d40 0x7fe884005150 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:29:37.515 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:37.510+0000 7fe89a7fc640 1 -- 172.21.15.37:0/2187644944 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 -- 0x7fe89c0d1650 con 0x7fe884004d40 2024-04-25T10:29:37.515 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:37.510+0000 7fe89a7fc640 1 -- 172.21.15.37:0/2187644944 --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_getmap magic: 0 -- 0x7fe89c0d4680 con 0x7fe8840039b0 2024-04-25T10:29:37.515 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:37.510+0000 7fe89a7fc640 1 -- 172.21.15.37:0/2187644944 --> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] -- mon_getmap magic: 0 -- 0x7fe88400baf0 con 0x7fe89c0d0990 2024-04-25T10:29:37.515 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:37.510+0000 7fe89bfff640 1 --2- 172.21.15.37:0/2187644944 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fe884004d40 0x7fe884005150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:29:37.515 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:37.510+0000 7fe89affd640 1 --2- 172.21.15.37:0/2187644944 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7fe89c0d0990 0x7fe884004800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:29:37.515 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:37.510+0000 7fe89b7fe640 1 --2- 172.21.15.37:0/2187644944 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fe8840039b0 0x7fe884003e20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:29:37.515 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:37.510+0000 7fe89bfff640 1 -- 172.21.15.37:0/2187644944 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7fe89c0d0990 msgr2=0x7fe884004800 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:29:37.515 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:37.510+0000 7fe89bfff640 1 --2- 172.21.15.37:0/2187644944 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7fe89c0d0990 0x7fe884004800 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:29:37.515 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:37.510+0000 7fe89bfff640 1 -- 172.21.15.37:0/2187644944 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fe8840039b0 msgr2=0x7fe884003e20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:29:37.516 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:37.510+0000 7fe89bfff640 1 --2- 172.21.15.37:0/2187644944 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fe8840039b0 0x7fe884003e20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:29:37.516 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:37.510+0000 7fe89bfff640 1 -- 172.21.15.37:0/2187644944 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fe884003520 con 0x7fe884004d40 2024-04-25T10:29:37.516 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:37.511+0000 7fe89bfff640 1 --2- 172.21.15.37:0/2187644944 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fe884004d40 0x7fe884005150 secure :-1 s=READY pgs=205 cs=0 l=1 rev1=1 crypto rx=0x7fe89c0d5990 tx=0x7fe884005830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=6206d178532aa17b server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:29:37.516 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:37.511+0000 7fe89a7fc640 1 -- 172.21.15.37:0/2187644944 <== mon.0 v2:172.21.15.37:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe89400a4e0 con 0x7fe884004d40 2024-04-25T10:29:37.516 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:37.511+0000 7fe89a7fc640 1 -- 172.21.15.37:0/2187644944 <== mon.0 v2:172.21.15.37:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fe89402f070 con 0x7fe884004d40 2024-04-25T10:29:37.516 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:37.511+0000 7fe89a7fc640 1 -- 172.21.15.37:0/2187644944 <== mon.0 v2:172.21.15.37:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe89402bc10 con 0x7fe884004d40 2024-04-25T10:29:37.519 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:37.512+0000 7fe8a1da8640 1 -- 172.21.15.37:0/2187644944 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fe884004d40 msgr2=0x7fe884005150 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:29:37.519 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:37.512+0000 7fe8a1da8640 1 --2- 172.21.15.37:0/2187644944 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fe884004d40 0x7fe884005150 secure :-1 s=READY pgs=205 cs=0 l=1 rev1=1 crypto rx=0x7fe89c0d5990 tx=0x7fe884005830 comp rx=0 tx=0).stop 2024-04-25T10:29:37.519 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:37.512+0000 7fe89bfff640 1 -- 172.21.15.37:0/2187644944 reap_dead start 2024-04-25T10:29:37.519 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:37.512+0000 7fe8a1da8640 1 -- 172.21.15.37:0/2187644944 shutdown_connections 2024-04-25T10:29:37.519 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:37.512+0000 7fe8a1da8640 1 -- 172.21.15.37:0/2187644944 >> 172.21.15.37:0/2187644944 conn(0x7fe89c0b7a80 msgr2=0x7fe89c0b7e90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:29:37.519 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:37.512+0000 7fe8a1da8640 1 -- 172.21.15.37:0/2187644944 shutdown_connections 2024-04-25T10:29:37.519 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:37.512+0000 7fe8a1da8640 1 -- 172.21.15.37:0/2187644944 wait complete. 2024-04-25T10:29:37.519 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:37.514+0000 7fe8a1da8640 1 Processor -- start 2024-04-25T10:29:37.519 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:37.514+0000 7fe8a1da8640 1 -- start start 2024-04-25T10:29:37.519 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:37.515+0000 7fe8a1da8640 1 --2- >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7fe89c1510a0 0x7fe89c0cd8d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:29:37.519 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:37.515+0000 7fe8a1da8640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fe89c0b67c0 0x7fe89c0b6bd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:29:37.520 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:37.515+0000 7fe8a1da8640 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fe89c1f69b0 0x7fe89c1f6dc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:29:37.520 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:37.515+0000 7fe8a1da8640 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 -- 0x7fe89c164c40 con 0x7fe89c0b67c0 2024-04-25T10:29:37.520 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:37.515+0000 7fe8a1da8640 1 -- --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_getmap magic: 0 -- 0x7fe89c164ac0 con 0x7fe89c1f69b0 2024-04-25T10:29:37.520 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:37.515+0000 7fe8a1da8640 1 -- --> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] -- mon_getmap magic: 0 -- 0x7fe89c06e220 con 0x7fe89c1510a0 2024-04-25T10:29:37.520 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:37.515+0000 7fe89affd640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fe89c0b67c0 0x7fe89c0b6bd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:29:37.520 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:37.515+0000 7fe89affd640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fe89c0b67c0 0x7fe89c0b6bd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.37:3300/0 says I am v2:172.21.15.37:36856/0 (socket says 172.21.15.37:36856) 2024-04-25T10:29:37.520 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:37.515+0000 7fe89affd640 1 -- 172.21.15.37:0/122355504 learned_addr learned my addr 172.21.15.37:0/122355504 (peer_addr_for_me v2:172.21.15.37:0/0) 2024-04-25T10:29:37.520 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:37.515+0000 7fe89b7fe640 1 --2- 172.21.15.37:0/122355504 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7fe89c1510a0 0x7fe89c0cd8d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:29:37.520 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:37.516+0000 7fe89bfff640 1 --2- 172.21.15.37:0/122355504 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fe89c1f69b0 0x7fe89c1f6dc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:29:37.520 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:37.516+0000 7fe89affd640 1 -- 172.21.15.37:0/122355504 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7fe89c1510a0 msgr2=0x7fe89c0cd8d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:29:37.520 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:37.516+0000 7fe89affd640 1 --2- 172.21.15.37:0/122355504 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7fe89c1510a0 0x7fe89c0cd8d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:29:37.520 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:37.516+0000 7fe89affd640 1 -- 172.21.15.37:0/122355504 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fe89c1f69b0 msgr2=0x7fe89c1f6dc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:29:37.520 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:37.516+0000 7fe89affd640 1 --2- 172.21.15.37:0/122355504 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fe89c1f69b0 0x7fe89c1f6dc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:29:37.520 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:37.516+0000 7fe89affd640 1 -- 172.21.15.37:0/122355504 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fe89c0b7110 con 0x7fe89c0b67c0 2024-04-25T10:29:37.521 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:37.516+0000 7fe89bfff640 1 --2- 172.21.15.37:0/122355504 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fe89c1f69b0 0x7fe89c1f6dc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T10:29:37.521 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:37.516+0000 7fe89b7fe640 1 --2- 172.21.15.37:0/122355504 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7fe89c1510a0 0x7fe89c0cd8d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T10:29:37.521 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:37.516+0000 7fe89affd640 1 --2- 172.21.15.37:0/122355504 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fe89c0b67c0 0x7fe89c0b6bd0 secure :-1 s=READY pgs=206 cs=0 l=1 rev1=1 crypto rx=0x7fe8900119a0 tx=0x7fe890011e60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:29:37.521 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:37.516+0000 7fe883fff640 1 -- 172.21.15.37:0/122355504 <== mon.0 v2:172.21.15.37:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe89000faf0 con 0x7fe89c0b67c0 2024-04-25T10:29:37.521 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:37.516+0000 7fe883fff640 1 -- 172.21.15.37:0/122355504 <== mon.0 v2:172.21.15.37:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fe89000fc90 con 0x7fe89c0b67c0 2024-04-25T10:29:37.521 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:37.516+0000 7fe883fff640 1 -- 172.21.15.37:0/122355504 <== mon.0 v2:172.21.15.37:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe890014440 con 0x7fe89c0b67c0 2024-04-25T10:29:37.521 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:37.516+0000 7fe8a1da8640 1 -- 172.21.15.37:0/122355504 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fe89c1f61d0 con 0x7fe89c0b67c0 2024-04-25T10:29:37.521 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:37.516+0000 7fe8a1da8640 1 -- 172.21.15.37:0/122355504 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fe89c0cc150 con 0x7fe89c0b67c0 2024-04-25T10:29:37.522 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:37.517+0000 7fe8a1da8640 1 -- 172.21.15.37:0/122355504 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fe89c06a140 con 0x7fe89c0b67c0 2024-04-25T10:29:37.522 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:37.518+0000 7fe883fff640 1 -- 172.21.15.37:0/122355504 <== mon.0 v2:172.21.15.37:3300/0 4 ==== mgrmap(e 5) ==== 96782+0+0 (secure 0 0 0) 0x7fe890012040 con 0x7fe89c0b67c0 2024-04-25T10:29:37.522 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:37.518+0000 7fe883fff640 1 --2- 172.21.15.37:0/122355504 >> [v2:172.21.15.203:6832/2418205237,v1:172.21.15.203:6833/2418205237] conn(0x7fe8880744e0 0x7fe8880769f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:29:37.522 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:37.518+0000 7fe883fff640 1 -- 172.21.15.37:0/122355504 <== mon.0 v2:172.21.15.37:3300/0 5 ==== osd_map(23..23 src has 1..23) ==== 5709+0+0 (secure 0 0 0) 0x7fe890095850 con 0x7fe89c0b67c0 2024-04-25T10:29:37.522 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:37.518+0000 7fe89b7fe640 1 --2- 172.21.15.37:0/122355504 >> [v2:172.21.15.203:6832/2418205237,v1:172.21.15.203:6833/2418205237] conn(0x7fe8880744e0 0x7fe8880769f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:29:37.522 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:37.519+0000 7fe89b7fe640 1 --2- 172.21.15.37:0/122355504 >> [v2:172.21.15.203:6832/2418205237,v1:172.21.15.203:6833/2418205237] conn(0x7fe8880744e0 0x7fe8880769f0 secure :-1 s=READY pgs=121 cs=0 l=1 rev1=1 crypto rx=0x7fe89c0d0430 tx=0x7fe88c008040 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:29:37.525 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:37.523+0000 7fe883fff640 1 -- 172.21.15.37:0/122355504 <== mon.0 v2:172.21.15.37:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187246 (secure 0 0 0) 0x7fe89c0b7110 con 0x7fe89c0b67c0 2024-04-25T10:29:37.594 DEBUG:teuthology.orchestra.run.smithi037:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-25T10:29:37.596 DEBUG:teuthology.orchestra.run.smithi203:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-25T10:29:37.712 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:37.710+0000 7fe8a1da8640 1 -- 172.21.15.37:0/122355504 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7fe89c0cffa0 con 0x7fe89c0b67c0 2024-04-25T10:29:37.713 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:37.711+0000 7fe883fff640 1 -- 172.21.15.37:0/122355504 <== mon.0 v2:172.21.15.37:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v23) ==== 74+0+16621 (secure 0 0 0) 0x7fe890019030 con 0x7fe89c0b67c0 2024-04-25T10:29:37.713 INFO:teuthology.orchestra.run.smithi037.stdout: 2024-04-25T10:29:37.713 INFO:teuthology.orchestra.run.smithi037.stdout:{"epoch":23,"fsid":"f4acda28-be34-492a-b286-994afd36f61d","created":"2024-04-25T10:28:43.985718+0000","modified":"2024-04-25T10:29:10.016188+0000","last_up_change":"2024-04-25T10:28:54.900654+0000","last_in_change":"2024-04-25T10:28:46.216973+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":3,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-04-25T10:28:56.202153+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"19","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2024-04-25T10:29:06.507244+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"22","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":2.25,"score_stable":2.25,"optimal_score":1,"raw_score_acting":2.25,"raw_score_stable":2.25,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2024-04-25T10:29:07.347844+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"23","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"a2f79028-b332-4166-9ebb-a7024a37ea22","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":21,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.37:6816","nonce":2746074720},{"type":"v1","addr":"172.21.15.37:6817","nonce":2746074720}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.37:6818","nonce":2746074720},{"type":"v1","addr":"172.21.15.37:6819","nonce":2746074720}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.37:6822","nonce":2746074720},{"type":"v1","addr":"172.21.15.37:6823","nonce":2746074720}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.37:6820","nonce":2746074720},{"type":"v1","addr":"172.21.15.37:6821","nonce":2746074720}]},"public_addr":"172.21.15.37:6817/2746074720","cluster_addr":"172.21.15.37:6819/2746074720","heartbeat_back_addr":"172.21.15.37:6823/2746074720","heartbeat_front_addr":"172.21.15.37:6821/2746074720","state":["exists","up"]},{"osd":1,"uuid":"973159de-c2ea-4083-aecc-69373d1c1132","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":21,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.37:6800","nonce":1690341662},{"type":"v1","addr":"172.21.15.37:6801","nonce":1690341662}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.37:6802","nonce":1690341662},{"type":"v1","addr":"172.21.15.37:6803","nonce":1690341662}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.37:6806","nonce":1690341662},{"type":"v1","addr":"172.21.15.37:6807","nonce":1690341662}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.37:6804","nonce":1690341662},{"type":"v1","addr":"172.21.15.37:6805","nonce":1690341662}]},"public_addr":"172.21.15.37:6801/1690341662","cluster_addr":"172.21.15.37:6803/1690341662","heartbeat_back_addr":"172.21.15.37:6807/1690341662","heartbeat_front_addr":"172.21.15.37:6805/1690341662","state":["exists","up"]},{"osd":2,"uuid":"4ae7a0e9-cb3f-484c-a75f-f3766b721c47","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":21,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.37:6808","nonce":2754669649},{"type":"v1","addr":"172.21.15.37:6809","nonce":2754669649}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.37:6810","nonce":2754669649},{"type":"v1","addr":"172.21.15.37:6811","nonce":2754669649}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.37:6814","nonce":2754669649},{"type":"v1","addr":"172.21.15.37:6815","nonce":2754669649}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.37:6812","nonce":2754669649},{"type":"v1","addr":"172.21.15.37:6813","nonce":2754669649}]},"public_addr":"172.21.15.37:6809/2754669649","cluster_addr":"172.21.15.37:6811/2754669649","heartbeat_back_addr":"172.21.15.37:6815/2754669649","heartbeat_front_addr":"172.21.15.37:6813/2754669649","state":["exists","up"]},{"osd":3,"uuid":"1ba4a746-1608-47a7-ace2-960a5cfae934","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":21,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.37:6824","nonce":1815290618},{"type":"v1","addr":"172.21.15.37:6825","nonce":1815290618}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.37:6826","nonce":1815290618},{"type":"v1","addr":"172.21.15.37:6827","nonce":1815290618}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.37:6830","nonce":1815290618},{"type":"v1","addr":"172.21.15.37:6831","nonce":1815290618}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.37:6828","nonce":1815290618},{"type":"v1","addr":"172.21.15.37:6829","nonce":1815290618}]},"public_addr":"172.21.15.37:6825/1815290618","cluster_addr":"172.21.15.37:6827/1815290618","heartbeat_back_addr":"172.21.15.37:6831/1815290618","heartbeat_front_addr":"172.21.15.37:6829/1815290618","state":["exists","up"]},{"osd":4,"uuid":"d4f59f5d-0413-4b79-87ee-bb212c1fcf62","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":21,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6800","nonce":4051532063},{"type":"v1","addr":"172.21.15.203:6801","nonce":4051532063}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6802","nonce":4051532063},{"type":"v1","addr":"172.21.15.203:6803","nonce":4051532063}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6806","nonce":4051532063},{"type":"v1","addr":"172.21.15.203:6807","nonce":4051532063}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6804","nonce":4051532063},{"type":"v1","addr":"172.21.15.203:6805","nonce":4051532063}]},"public_addr":"172.21.15.203:6801/4051532063","cluster_addr":"172.21.15.203:6803/4051532063","heartbeat_back_addr":"172.21.15.203:6807/4051532063","heartbeat_front_addr":"172.21.15.203:6805/4051532063","state":["exists","up"]},{"osd":5,"uuid":"69710a0e-41d8-4e7f-9a10-17ce540b67be","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":21,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6824","nonce":1203822890},{"type":"v1","addr":"172.21.15.203:6825","nonce":1203822890}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6826","nonce":1203822890},{"type":"v1","addr":"172.21.15.203:6827","nonce":1203822890}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6830","nonce":1203822890},{"type":"v1","addr":"172.21.15.203:6831","nonce":1203822890}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6828","nonce":1203822890},{"type":"v1","addr":"172.21.15.203:6829","nonce":1203822890}]},"public_addr":"172.21.15.203:6825/1203822890","cluster_addr":"172.21.15.203:6827/1203822890","heartbeat_back_addr":"172.21.15.203:6831/1203822890","heartbeat_front_addr":"172.21.15.203:6829/1203822890","state":["exists","up"]},{"osd":6,"uuid":"f29dd047-3b48-473d-a235-6642a6293f20","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":21,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6810","nonce":2309615014},{"type":"v1","addr":"172.21.15.203:6812","nonce":2309615014}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6814","nonce":2309615014},{"type":"v1","addr":"172.21.15.203:6816","nonce":2309615014}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6822","nonce":2309615014},{"type":"v1","addr":"172.21.15.203:6823","nonce":2309615014}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6818","nonce":2309615014},{"type":"v1","addr":"172.21.15.203:6820","nonce":2309615014}]},"public_addr":"172.21.15.203:6812/2309615014","cluster_addr":"172.21.15.203:6816/2309615014","heartbeat_back_addr":"172.21.15.203:6823/2309615014","heartbeat_front_addr":"172.21.15.203:6820/2309615014","state":["exists","up"]},{"osd":7,"uuid":"6e691644-378b-4fdc-8e86-fb5f56e31f29","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":21,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6808","nonce":3688099014},{"type":"v1","addr":"172.21.15.203:6809","nonce":3688099014}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6811","nonce":3688099014},{"type":"v1","addr":"172.21.15.203:6813","nonce":3688099014}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6819","nonce":3688099014},{"type":"v1","addr":"172.21.15.203:6821","nonce":3688099014}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6815","nonce":3688099014},{"type":"v1","addr":"172.21.15.203:6817","nonce":3688099014}]},"public_addr":"172.21.15.203:6809/3688099014","cluster_addr":"172.21.15.203:6813/3688099014","heartbeat_back_addr":"172.21.15.203:6821/3688099014","heartbeat_front_addr":"172.21.15.203:6817/3688099014","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T10:28:51.419141+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T10:28:51.366448+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T10:28:51.348276+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T10:28:51.380519+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T10:28:51.105812+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T10:28:51.384496+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T10:28:51.388755+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T10:28:51.366958+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-04-25T10:29:37.715 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:37.713+0000 7fe8a1da8640 1 -- 172.21.15.37:0/122355504 >> [v2:172.21.15.203:6832/2418205237,v1:172.21.15.203:6833/2418205237] conn(0x7fe8880744e0 msgr2=0x7fe8880769f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:29:37.715 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:37.713+0000 7fe8a1da8640 1 --2- 172.21.15.37:0/122355504 >> [v2:172.21.15.203:6832/2418205237,v1:172.21.15.203:6833/2418205237] conn(0x7fe8880744e0 0x7fe8880769f0 secure :-1 s=READY pgs=121 cs=0 l=1 rev1=1 crypto rx=0x7fe89c0d0430 tx=0x7fe88c008040 comp rx=0 tx=0).stop 2024-04-25T10:29:37.716 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:37.713+0000 7fe8a1da8640 1 -- 172.21.15.37:0/122355504 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fe89c0b67c0 msgr2=0x7fe89c0b6bd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:29:37.716 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:37.713+0000 7fe8a1da8640 1 --2- 172.21.15.37:0/122355504 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fe89c0b67c0 0x7fe89c0b6bd0 secure :-1 s=READY pgs=206 cs=0 l=1 rev1=1 crypto rx=0x7fe8900119a0 tx=0x7fe890011e60 comp rx=0 tx=0).stop 2024-04-25T10:29:37.716 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:37.713+0000 7fe8a1da8640 1 -- 172.21.15.37:0/122355504 shutdown_connections 2024-04-25T10:29:37.716 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:37.713+0000 7fe8a1da8640 1 --2- 172.21.15.37:0/122355504 >> [v2:172.21.15.203:6832/2418205237,v1:172.21.15.203:6833/2418205237] conn(0x7fe8880744e0 0x7fe8880769f0 unknown :-1 s=CLOSED pgs=121 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:29:37.716 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:37.713+0000 7fe8a1da8640 1 --2- 172.21.15.37:0/122355504 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fe89c1f69b0 0x7fe89c1f6dc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:29:37.716 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:37.713+0000 7fe8a1da8640 1 --2- 172.21.15.37:0/122355504 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fe89c0b67c0 0x7fe89c0b6bd0 unknown :-1 s=CLOSED pgs=206 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:29:37.716 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:37.713+0000 7fe8a1da8640 1 --2- 172.21.15.37:0/122355504 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7fe89c1510a0 0x7fe89c0cd8d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:29:37.716 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:37.713+0000 7fe8a1da8640 1 -- 172.21.15.37:0/122355504 >> 172.21.15.37:0/122355504 conn(0x7fe89c0b7a80 msgr2=0x7fe89c163140 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:29:37.716 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:37.713+0000 7fe8a1da8640 1 -- 172.21.15.37:0/122355504 shutdown_connections 2024-04-25T10:29:37.716 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:37.713+0000 7fe8a1da8640 1 -- 172.21.15.37:0/122355504 wait complete. 2024-04-25T10:29:37.726 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2024-04-25T10:28:56.202153+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '19', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 8, 'score_stable': 8, 'optimal_score': 0.25, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 2, 'pool_name': 'cephfs_metadata', 'create_time': '2024-04-25T10:29:06.507244+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '22', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_autoscale_bias': 4, 'pg_num_min': 16, 'recovery_priority': 5}, 'application_metadata': {'cephfs': {'metadata': 'cephfs'}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 2.25, 'score_stable': 2.25, 'optimal_score': 1, 'raw_score_acting': 2.25, 'raw_score_stable': 2.25, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 3, 'pool_name': 'cephfs_data', 'create_time': '2024-04-25T10:29:07.347844+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 64, 'pg_placement_num': 64, 'pg_placement_num_target': 64, 'pg_num_target': 64, 'pg_num_pending': 64, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '23', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.8999999761581421}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 1.5, 'score_stable': 1.5, 'optimal_score': 1, 'raw_score_acting': 1.5, 'raw_score_stable': 1.5, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}] 2024-04-25T10:29:37.727 DEBUG:teuthology.orchestra.run.smithi037:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2024-04-25T10:29:37.840 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:37.838+0000 7f6950ae0640 1 Processor -- start 2024-04-25T10:29:37.841 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:37.838+0000 7f6950ae0640 1 -- start start 2024-04-25T10:29:37.843 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:37.840+0000 7f6950ae0640 1 --2- >> v2:172.21.15.37:3300/0 conn(0x7f694c0b2ca0 0x7f694c08d3a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:29:37.843 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:37.840+0000 7f6950ae0640 1 --2- >> v2:172.21.15.203:3300/0 conn(0x7f694c0b3670 0x7f694c0b3a80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:29:37.843 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:37.840+0000 7f6950ae0640 1 -- --> v1:172.21.15.37:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f694c0cf120 con 0x7f694c0b5fd0 2024-04-25T10:29:37.843 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:37.840+0000 7f6950ae0640 1 -- --> v2:172.21.15.37:3300/0 -- mon_getmap magic: 0 -- 0x7f694c0b30b0 con 0x7f694c0b2ca0 2024-04-25T10:29:37.843 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:37.840+0000 7f6950ae0640 1 -- --> v2:172.21.15.203:3300/0 -- mon_getmap magic: 0 -- 0x7f694c0cf400 con 0x7f694c0b3670 2024-04-25T10:29:37.843 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:37.840+0000 7f694a59c640 1 --1- >> v1:172.21.15.37:6789/0 conn(0x7f694c0b5fd0 0x7f694c08cc90 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.37:6789/0 says I am v1:172.21.15.37:46540/0 (socket says 172.21.15.37:46540) 2024-04-25T10:29:37.843 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:37.840+0000 7f694a59c640 1 -- 172.21.15.37:0/2512431052 learned_addr learned my addr 172.21.15.37:0/2512431052 (peer_addr_for_me v1:172.21.15.37:0/0) 2024-04-25T10:29:37.843 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:37.840+0000 7f6949d9b640 1 --2- 172.21.15.37:0/2512431052 >> v2:172.21.15.37:3300/0 conn(0x7f694c0b2ca0 0x7f694c08d3a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:29:37.843 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:37.840+0000 7f694ad9d640 1 --2- 172.21.15.37:0/2512431052 >> v2:172.21.15.203:3300/0 conn(0x7f694c0b3670 0x7f694c0b3a80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:29:37.843 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:37.840+0000 7f694959a640 1 -- 172.21.15.37:0/2512431052 <== mon.0 v1:172.21.15.37:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 1412075171 0 0) 0x7f694c0cf120 con 0x7f694c0b5fd0 2024-04-25T10:29:37.843 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:37.840+0000 7f694959a640 1 -- 172.21.15.37:0/2512431052 --> v1:172.21.15.37:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f6934003520 con 0x7f694c0b5fd0 2024-04-25T10:29:37.843 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:37.840+0000 7f6949d9b640 1 -- 172.21.15.37:0/2512431052 >> v2:172.21.15.203:3300/0 conn(0x7f694c0b3670 msgr2=0x7f694c0b3a80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:29:37.844 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:37.840+0000 7f6949d9b640 1 --2- 172.21.15.37:0/2512431052 >> v2:172.21.15.203:3300/0 conn(0x7f694c0b3670 0x7f694c0b3a80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:29:37.844 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:37.840+0000 7f6949d9b640 1 -- 172.21.15.37:0/2512431052 >> v1:172.21.15.37:6789/0 conn(0x7f694c0b5fd0 legacy=0x7f694c08cc90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:29:37.844 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:37.840+0000 7f6949d9b640 1 -- 172.21.15.37:0/2512431052 --> v2:172.21.15.37:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f694c0cf6e0 con 0x7f694c0b2ca0 2024-04-25T10:29:37.845 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:37.841+0000 7f6949d9b640 1 --2- 172.21.15.37:0/2512431052 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f694c0b2ca0 0x7f694c08d3a0 secure :-1 s=READY pgs=208 cs=0 l=1 rev1=1 crypto rx=0x7f694c0b9650 tx=0x7f694002a1a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b412cc972c232e8c server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:29:37.845 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:37.841+0000 7f694959a640 1 -- 172.21.15.37:0/2512431052 <== mon.0 v2:172.21.15.37:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6940037070 con 0x7f694c0b2ca0 2024-04-25T10:29:37.845 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:37.841+0000 7f694959a640 1 -- 172.21.15.37:0/2512431052 <== mon.0 v2:172.21.15.37:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f6940032440 con 0x7f694c0b2ca0 2024-04-25T10:29:37.845 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:37.841+0000 7f694959a640 1 -- 172.21.15.37:0/2512431052 <== mon.0 v2:172.21.15.37:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6940032810 con 0x7f694c0b2ca0 2024-04-25T10:29:37.845 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:37.842+0000 7f6950ae0640 1 -- 172.21.15.37:0/2512431052 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f694c0b2ca0 msgr2=0x7f694c08d3a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:29:37.845 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:37.842+0000 7f6950ae0640 1 --2- 172.21.15.37:0/2512431052 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f694c0b2ca0 0x7f694c08d3a0 secure :-1 s=READY pgs=208 cs=0 l=1 rev1=1 crypto rx=0x7f694c0b9650 tx=0x7f694002a1a0 comp rx=0 tx=0).stop 2024-04-25T10:29:37.845 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:37.842+0000 7f6950ae0640 1 -- 172.21.15.37:0/2512431052 shutdown_connections 2024-04-25T10:29:37.846 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:37.842+0000 7f6950ae0640 1 --2- 172.21.15.37:0/2512431052 >> v2:172.21.15.203:3300/0 conn(0x7f694c0b3670 0x7f694c0b3a80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:29:37.846 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:37.842+0000 7f6950ae0640 1 --2- 172.21.15.37:0/2512431052 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f694c0b2ca0 0x7f694c08d3a0 unknown :-1 s=CLOSED pgs=208 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:29:37.846 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:37.842+0000 7f6950ae0640 1 -- 172.21.15.37:0/2512431052 >> 172.21.15.37:0/2512431052 conn(0x7f694c0b5440 msgr2=0x7f694c0b5850 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:29:37.846 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:37.842+0000 7f6950ae0640 1 -- 172.21.15.37:0/2512431052 shutdown_connections 2024-04-25T10:29:37.846 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:37.842+0000 7f6950ae0640 1 -- 172.21.15.37:0/2512431052 wait complete. 2024-04-25T10:29:37.846 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:37.844+0000 7f6950ae0640 1 Processor -- start 2024-04-25T10:29:37.847 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:37.844+0000 7f6950ae0640 1 -- start start 2024-04-25T10:29:37.848 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:37.845+0000 7f6950ae0640 1 --2- >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f694c0b2ca0 0x7f694c09ded0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:29:37.849 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:37.845+0000 7f6950ae0640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f694c0b3670 0x7f694c1f7a60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:29:37.849 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:37.845+0000 7f6950ae0640 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f694c0b5fd0 0x7f694c15c7a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:29:37.849 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:37.845+0000 7f6950ae0640 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 -- 0x7f694c08d8e0 con 0x7f694c0b3670 2024-04-25T10:29:37.849 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:37.845+0000 7f6950ae0640 1 -- --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_getmap magic: 0 -- 0x7f694c0d28d0 con 0x7f694c0b5fd0 2024-04-25T10:29:37.849 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:37.845+0000 7f6950ae0640 1 -- --> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] -- mon_getmap magic: 0 -- 0x7f694c08da60 con 0x7f694c0b2ca0 2024-04-25T10:29:37.849 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:37.845+0000 7f6949d9b640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f694c0b3670 0x7f694c1f7a60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:29:37.849 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:37.845+0000 7f6949d9b640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f694c0b3670 0x7f694c1f7a60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.37:3300/0 says I am v2:172.21.15.37:36872/0 (socket says 172.21.15.37:36872) 2024-04-25T10:29:37.849 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:37.845+0000 7f6949d9b640 1 -- 172.21.15.37:0/1013617471 learned_addr learned my addr 172.21.15.37:0/1013617471 (peer_addr_for_me v2:172.21.15.37:0/0) 2024-04-25T10:29:37.849 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:37.846+0000 7f694a59c640 1 --2- 172.21.15.37:0/1013617471 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f694c0b2ca0 0x7f694c09ded0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:29:37.849 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:37.846+0000 7f694ad9d640 1 --2- 172.21.15.37:0/1013617471 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f694c0b5fd0 0x7f694c15c7a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:29:37.849 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:37.846+0000 7f6949d9b640 1 -- 172.21.15.37:0/1013617471 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f694c0b2ca0 msgr2=0x7f694c09ded0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:29:37.849 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:37.846+0000 7f6949d9b640 1 --2- 172.21.15.37:0/1013617471 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f694c0b2ca0 0x7f694c09ded0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:29:37.849 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:37.846+0000 7f6949d9b640 1 -- 172.21.15.37:0/1013617471 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f694c0b5fd0 msgr2=0x7f694c15c7a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:29:37.849 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:37.846+0000 7f6949d9b640 1 --2- 172.21.15.37:0/1013617471 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f694c0b5fd0 0x7f694c15c7a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:29:37.850 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:37.846+0000 7f6949d9b640 1 -- 172.21.15.37:0/1013617471 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f694c09fa10 con 0x7f694c0b3670 2024-04-25T10:29:37.850 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:37.846+0000 7f6949d9b640 1 --2- 172.21.15.37:0/1013617471 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f694c0b3670 0x7f694c1f7a60 secure :-1 s=READY pgs=209 cs=0 l=1 rev1=1 crypto rx=0x7f6940009890 tx=0x7f69400046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:29:37.850 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:37.846+0000 7f693affd640 1 -- 172.21.15.37:0/1013617471 <== mon.0 v2:172.21.15.37:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6940041020 con 0x7f694c0b3670 2024-04-25T10:29:37.850 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:37.846+0000 7f693affd640 1 -- 172.21.15.37:0/1013617471 <== mon.0 v2:172.21.15.37:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f694003d640 con 0x7f694c0b3670 2024-04-25T10:29:37.850 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:37.846+0000 7f693affd640 1 -- 172.21.15.37:0/1013617471 <== mon.0 v2:172.21.15.37:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6940037040 con 0x7f694c0b3670 2024-04-25T10:29:37.850 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:37.846+0000 7f6950ae0640 1 -- 172.21.15.37:0/1013617471 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f694c0ca050 con 0x7f694c0b3670 2024-04-25T10:29:37.850 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:37.846+0000 7f6950ae0640 1 -- 172.21.15.37:0/1013617471 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f694c09fc90 con 0x7f694c0b3670 2024-04-25T10:29:37.851 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:37.847+0000 7f6950ae0640 1 -- 172.21.15.37:0/1013617471 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f694c06a140 con 0x7f694c0b3670 2024-04-25T10:29:37.851 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:37.848+0000 7f693affd640 1 -- 172.21.15.37:0/1013617471 <== mon.0 v2:172.21.15.37:3300/0 4 ==== mgrmap(e 5) ==== 96782+0+0 (secure 0 0 0) 0x7f6940044020 con 0x7f694c0b3670 2024-04-25T10:29:37.851 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:37.848+0000 7f693affd640 1 --2- 172.21.15.37:0/1013617471 >> [v2:172.21.15.203:6832/2418205237,v1:172.21.15.203:6833/2418205237] conn(0x7f692c0744e0 0x7f692c0769f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:29:37.851 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:37.848+0000 7f693affd640 1 -- 172.21.15.37:0/1013617471 <== mon.0 v2:172.21.15.37:3300/0 5 ==== osd_map(23..23 src has 1..23) ==== 5709+0+0 (secure 0 0 0) 0x7f69400b58c0 con 0x7f694c0b3670 2024-04-25T10:29:37.851 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:37.848+0000 7f694a59c640 1 --2- 172.21.15.37:0/1013617471 >> [v2:172.21.15.203:6832/2418205237,v1:172.21.15.203:6833/2418205237] conn(0x7f692c0744e0 0x7f692c0769f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:29:37.851 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:37.849+0000 7f694a59c640 1 --2- 172.21.15.37:0/1013617471 >> [v2:172.21.15.203:6832/2418205237,v1:172.21.15.203:6833/2418205237] conn(0x7f692c0744e0 0x7f692c0769f0 secure :-1 s=READY pgs=122 cs=0 l=1 rev1=1 crypto rx=0x7f6944001360 tx=0x7f6944009040 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:29:37.855 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:37.853+0000 7f693affd640 1 -- 172.21.15.37:0/1013617471 <== mon.0 v2:172.21.15.37:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187246 (secure 0 0 0) 0x7f694004b030 con 0x7f694c0b3670 2024-04-25T10:29:38.039 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:38.037+0000 7f6950ae0640 1 -- 172.21.15.37:0/1013617471 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) -- 0x7f694c0b0540 con 0x7f694c0b3670 2024-04-25T10:29:38.039 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:38.037+0000 7f693affd640 1 -- 172.21.15.37:0/1013617471 <== mon.0 v2:172.21.15.37:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v23) ==== 93+0+10 (secure 0 0 0) 0x7f694c09fa10 con 0x7f694c0b3670 2024-04-25T10:29:38.039 INFO:teuthology.orchestra.run.smithi037.stdout:pg_num: 1 2024-04-25T10:29:38.041 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:38.039+0000 7f6950ae0640 1 -- 172.21.15.37:0/1013617471 >> [v2:172.21.15.203:6832/2418205237,v1:172.21.15.203:6833/2418205237] conn(0x7f692c0744e0 msgr2=0x7f692c0769f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:29:38.041 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:38.039+0000 7f6950ae0640 1 --2- 172.21.15.37:0/1013617471 >> [v2:172.21.15.203:6832/2418205237,v1:172.21.15.203:6833/2418205237] conn(0x7f692c0744e0 0x7f692c0769f0 secure :-1 s=READY pgs=122 cs=0 l=1 rev1=1 crypto rx=0x7f6944001360 tx=0x7f6944009040 comp rx=0 tx=0).stop 2024-04-25T10:29:38.041 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:38.039+0000 7f6950ae0640 1 -- 172.21.15.37:0/1013617471 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f694c0b3670 msgr2=0x7f694c1f7a60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:29:38.042 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:38.039+0000 7f6950ae0640 1 --2- 172.21.15.37:0/1013617471 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f694c0b3670 0x7f694c1f7a60 secure :-1 s=READY pgs=209 cs=0 l=1 rev1=1 crypto rx=0x7f6940009890 tx=0x7f69400046c0 comp rx=0 tx=0).stop 2024-04-25T10:29:38.042 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:38.039+0000 7f6950ae0640 1 -- 172.21.15.37:0/1013617471 shutdown_connections 2024-04-25T10:29:38.042 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:38.039+0000 7f6950ae0640 1 --2- 172.21.15.37:0/1013617471 >> [v2:172.21.15.203:6832/2418205237,v1:172.21.15.203:6833/2418205237] conn(0x7f692c0744e0 0x7f692c0769f0 unknown :-1 s=CLOSED pgs=122 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:29:38.042 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:38.039+0000 7f6950ae0640 1 --2- 172.21.15.37:0/1013617471 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f694c0b5fd0 0x7f694c15c7a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:29:38.042 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:38.039+0000 7f6950ae0640 1 --2- 172.21.15.37:0/1013617471 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f694c0b3670 0x7f694c1f7a60 unknown :-1 s=CLOSED pgs=209 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:29:38.042 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:38.039+0000 7f6950ae0640 1 --2- 172.21.15.37:0/1013617471 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f694c0b2ca0 0x7f694c09ded0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:29:38.042 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:38.039+0000 7f6950ae0640 1 -- 172.21.15.37:0/1013617471 >> 172.21.15.37:0/1013617471 conn(0x7f694c0b5440 msgr2=0x7f694c0cffa0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:29:38.042 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:38.039+0000 7f6950ae0640 1 -- 172.21.15.37:0/1013617471 shutdown_connections 2024-04-25T10:29:38.042 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:38.039+0000 7f6950ae0640 1 -- 172.21.15.37:0/1013617471 wait complete. 2024-04-25T10:29:38.052 DEBUG:teuthology.orchestra.run.smithi037:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_metadata pg_num 2024-04-25T10:29:38.167 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:38.164+0000 7f79d0a92640 1 Processor -- start 2024-04-25T10:29:38.167 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:38.164+0000 7f79d0a92640 1 -- start start 2024-04-25T10:29:38.168 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:38.166+0000 7f79d0a92640 1 --2- >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f79cc15a9f0 0x7f79cc06c540 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:29:38.169 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:38.166+0000 7f79d0a92640 1 --2- >> v2:172.21.15.37:3300/0 conn(0x7f79cc06ca80 0x7f79cc160910 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:29:38.169 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:38.166+0000 7f79d0a92640 1 -- --> v1:172.21.15.37:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f79cc06ce90 con 0x7f79cc160e50 2024-04-25T10:29:38.169 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:38.166+0000 7f79d0a92640 1 -- --> v2:172.21.15.37:3300/0 -- mon_getmap magic: 0 -- 0x7f79cc0b6870 con 0x7f79cc06ca80 2024-04-25T10:29:38.169 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:38.166+0000 7f79d0a92640 1 -- --> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] -- mon_getmap magic: 0 -- 0x7f79cc1616d0 con 0x7f79cc15a9f0 2024-04-25T10:29:38.169 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:38.166+0000 7f79c9d9b640 1 --2- >> v2:172.21.15.37:3300/0 conn(0x7f79cc06ca80 0x7f79cc160910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:29:38.169 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:38.166+0000 7f79cad9d640 1 --1- >> v1:172.21.15.37:6789/0 conn(0x7f79cc160e50 0x7f79cc145a30 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.37:6789/0 says I am v1:172.21.15.37:46544/0 (socket says 172.21.15.37:46544) 2024-04-25T10:29:38.170 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:38.166+0000 7f79c9d9b640 1 --2- 172.21.15.37:0/2298101451 >> v2:172.21.15.37:3300/0 conn(0x7f79cc06ca80 0x7f79cc160910 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.37:3300/0 says I am v2:172.21.15.37:36888/0 (socket says 172.21.15.37:36888) 2024-04-25T10:29:38.170 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:38.166+0000 7f79cad9d640 1 -- 172.21.15.37:0/2298101451 learned_addr learned my addr 172.21.15.37:0/2298101451 (peer_addr_for_me v1:172.21.15.37:0/0) 2024-04-25T10:29:38.170 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:38.166+0000 7f79ca59c640 1 --2- 172.21.15.37:0/2298101451 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f79cc15a9f0 0x7f79cc06c540 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:29:38.170 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:38.166+0000 7f79c9d9b640 1 -- 172.21.15.37:0/2298101451 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f79cc15a9f0 msgr2=0x7f79cc06c540 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:29:38.170 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:38.166+0000 7f79c9d9b640 1 --2- 172.21.15.37:0/2298101451 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f79cc15a9f0 0x7f79cc06c540 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:29:38.170 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:38.166+0000 7f79c9d9b640 1 -- 172.21.15.37:0/2298101451 >> v1:172.21.15.37:6789/0 conn(0x7f79cc160e50 legacy=0x7f79cc145a30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:29:38.170 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:38.166+0000 7f79c9d9b640 1 -- 172.21.15.37:0/2298101451 --> v2:172.21.15.37:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f79cc163e00 con 0x7f79cc06ca80 2024-04-25T10:29:38.170 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:38.166+0000 7f79c959a640 1 -- 172.21.15.37:0/2298101451 <== mon.0 v1:172.21.15.37:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 796687338 0 0) 0x7f79cc06ce90 con 0x7f79cc160e50 2024-04-25T10:29:38.171 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:38.167+0000 7f79c9d9b640 1 --2- 172.21.15.37:0/2298101451 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f79cc06ca80 0x7f79cc160910 secure :-1 s=READY pgs=211 cs=0 l=1 rev1=1 crypto rx=0x7f79b40097a0 tx=0x7f79b402a180 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d81a5dd975a8db87 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:29:38.171 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:38.167+0000 7f79c959a640 1 -- 172.21.15.37:0/2298101451 <== mon.0 v2:172.21.15.37:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f79b4037070 con 0x7f79cc06ca80 2024-04-25T10:29:38.171 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:38.167+0000 7f79c959a640 1 -- 172.21.15.37:0/2298101451 <== mon.0 v2:172.21.15.37:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f79b4032440 con 0x7f79cc06ca80 2024-04-25T10:29:38.172 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:38.167+0000 7f79c959a640 1 -- 172.21.15.37:0/2298101451 <== mon.0 v2:172.21.15.37:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f79b4032810 con 0x7f79cc06ca80 2024-04-25T10:29:38.172 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:38.168+0000 7f79d0a92640 1 -- 172.21.15.37:0/2298101451 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f79cc06ca80 msgr2=0x7f79cc160910 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:29:38.172 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:38.168+0000 7f79d0a92640 1 --2- 172.21.15.37:0/2298101451 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f79cc06ca80 0x7f79cc160910 secure :-1 s=READY pgs=211 cs=0 l=1 rev1=1 crypto rx=0x7f79b40097a0 tx=0x7f79b402a180 comp rx=0 tx=0).stop 2024-04-25T10:29:38.172 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:38.168+0000 7f79d0a92640 1 -- 172.21.15.37:0/2298101451 shutdown_connections 2024-04-25T10:29:38.172 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:38.168+0000 7f79d0a92640 1 --2- 172.21.15.37:0/2298101451 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f79cc06ca80 0x7f79cc160910 unknown :-1 s=CLOSED pgs=211 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:29:38.172 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:38.168+0000 7f79d0a92640 1 --2- 172.21.15.37:0/2298101451 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f79cc15a9f0 0x7f79cc06c540 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:29:38.172 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:38.168+0000 7f79d0a92640 1 -- 172.21.15.37:0/2298101451 >> 172.21.15.37:0/2298101451 conn(0x7f79cc0b4560 msgr2=0x7f79cc0b4970 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:29:38.172 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:38.168+0000 7f79d0a92640 1 -- 172.21.15.37:0/2298101451 shutdown_connections 2024-04-25T10:29:38.172 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:38.168+0000 7f79d0a92640 1 -- 172.21.15.37:0/2298101451 wait complete. 2024-04-25T10:29:38.173 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:38.170+0000 7f79d0a92640 1 Processor -- start 2024-04-25T10:29:38.173 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:38.170+0000 7f79d0a92640 1 -- start start 2024-04-25T10:29:38.175 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:38.171+0000 7f79d0a92640 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f79cc06ca80 0x7f79cc0cc800 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:29:38.175 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:38.171+0000 7f79d0a92640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f79cc15a9f0 0x7f79cc15aed0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:29:38.175 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:38.171+0000 7f79d0a92640 1 --2- >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f79cc160e50 0x7f79cc15b410 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:29:38.175 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:38.171+0000 7f79d0a92640 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 -- 0x7f79cc1646a0 con 0x7f79cc15a9f0 2024-04-25T10:29:38.175 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:38.171+0000 7f79d0a92640 1 -- --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_getmap magic: 0 -- 0x7f79cc1643c0 con 0x7f79cc06ca80 2024-04-25T10:29:38.175 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:38.171+0000 7f79d0a92640 1 -- --> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] -- mon_getmap magic: 0 -- 0x7f79cc164bc0 con 0x7f79cc160e50 2024-04-25T10:29:38.175 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:38.171+0000 7f79c9d9b640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f79cc15a9f0 0x7f79cc15aed0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:29:38.175 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:38.171+0000 7f79c9d9b640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f79cc15a9f0 0x7f79cc15aed0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.37:3300/0 says I am v2:172.21.15.37:36892/0 (socket says 172.21.15.37:36892) 2024-04-25T10:29:38.176 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:38.171+0000 7f79c9d9b640 1 -- 172.21.15.37:0/2212962819 learned_addr learned my addr 172.21.15.37:0/2212962819 (peer_addr_for_me v2:172.21.15.37:0/0) 2024-04-25T10:29:38.176 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:38.172+0000 7f79cad9d640 1 --2- 172.21.15.37:0/2212962819 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f79cc160e50 0x7f79cc15b410 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:29:38.176 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:38.172+0000 7f79c9d9b640 1 -- 172.21.15.37:0/2212962819 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f79cc160e50 msgr2=0x7f79cc15b410 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:29:38.176 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:38.172+0000 7f79ca59c640 1 --2- 172.21.15.37:0/2212962819 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f79cc06ca80 0x7f79cc0cc800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:29:38.176 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:38.172+0000 7f79c9d9b640 1 --2- 172.21.15.37:0/2212962819 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f79cc160e50 0x7f79cc15b410 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:29:38.176 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:38.172+0000 7f79c9d9b640 1 -- 172.21.15.37:0/2212962819 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f79cc06ca80 msgr2=0x7f79cc0cc800 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:29:38.176 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:38.172+0000 7f79c9d9b640 1 --2- 172.21.15.37:0/2212962819 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f79cc06ca80 0x7f79cc0cc800 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:29:38.176 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:38.172+0000 7f79c9d9b640 1 -- 172.21.15.37:0/2212962819 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f79cc0cd2a0 con 0x7f79cc15a9f0 2024-04-25T10:29:38.176 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:38.172+0000 7f79cad9d640 1 --2- 172.21.15.37:0/2212962819 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f79cc160e50 0x7f79cc15b410 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T10:29:38.176 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:38.172+0000 7f79ca59c640 1 --2- 172.21.15.37:0/2212962819 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f79cc06ca80 0x7f79cc0cc800 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T10:29:38.176 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:38.172+0000 7f79c9d9b640 1 --2- 172.21.15.37:0/2212962819 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f79cc15a9f0 0x7f79cc15aed0 secure :-1 s=READY pgs=212 cs=0 l=1 rev1=1 crypto rx=0x7f79b4035040 tx=0x7f79b40046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:29:38.176 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:38.172+0000 7f79c2ffd640 1 -- 172.21.15.37:0/2212962819 <== mon.0 v2:172.21.15.37:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f79b4037040 con 0x7f79cc15a9f0 2024-04-25T10:29:38.176 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:38.172+0000 7f79c2ffd640 1 -- 172.21.15.37:0/2212962819 <== mon.0 v2:172.21.15.37:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f79b4030500 con 0x7f79cc15a9f0 2024-04-25T10:29:38.177 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:38.172+0000 7f79c2ffd640 1 -- 172.21.15.37:0/2212962819 <== mon.0 v2:172.21.15.37:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f79b4030910 con 0x7f79cc15a9f0 2024-04-25T10:29:38.177 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:38.172+0000 7f79d0a92640 1 -- 172.21.15.37:0/2212962819 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f79cc0cd580 con 0x7f79cc15a9f0 2024-04-25T10:29:38.177 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:38.172+0000 7f79d0a92640 1 -- 172.21.15.37:0/2212962819 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f79cc0cdb40 con 0x7f79cc15a9f0 2024-04-25T10:29:38.177 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:38.173+0000 7f79d0a92640 1 -- 172.21.15.37:0/2212962819 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f79cc06a140 con 0x7f79cc15a9f0 2024-04-25T10:29:38.177 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:38.174+0000 7f79c2ffd640 1 -- 172.21.15.37:0/2212962819 <== mon.0 v2:172.21.15.37:3300/0 4 ==== mgrmap(e 5) ==== 96782+0+0 (secure 0 0 0) 0x7f79b4044020 con 0x7f79cc15a9f0 2024-04-25T10:29:38.178 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:38.174+0000 7f79c2ffd640 1 --2- 172.21.15.37:0/2212962819 >> [v2:172.21.15.203:6832/2418205237,v1:172.21.15.203:6833/2418205237] conn(0x7f79ac0744b0 0x7f79ac0769c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:29:38.178 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:38.174+0000 7f79c2ffd640 1 -- 172.21.15.37:0/2212962819 <== mon.0 v2:172.21.15.37:3300/0 5 ==== osd_map(23..23 src has 1..23) ==== 5709+0+0 (secure 0 0 0) 0x7f79b40b5480 con 0x7f79cc15a9f0 2024-04-25T10:29:38.178 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:38.175+0000 7f79ca59c640 1 --2- 172.21.15.37:0/2212962819 >> [v2:172.21.15.203:6832/2418205237,v1:172.21.15.203:6833/2418205237] conn(0x7f79ac0744b0 0x7f79ac0769c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:29:38.178 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:38.175+0000 7f79ca59c640 1 --2- 172.21.15.37:0/2212962819 >> [v2:172.21.15.203:6832/2418205237,v1:172.21.15.203:6833/2418205237] conn(0x7f79ac0744b0 0x7f79ac0769c0 secure :-1 s=READY pgs=123 cs=0 l=1 rev1=1 crypto rx=0x7f79cc073140 tx=0x7f79c4008040 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:29:38.181 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:38.179+0000 7f79c2ffd640 1 -- 172.21.15.37:0/2212962819 <== mon.0 v2:172.21.15.37:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187246 (secure 0 0 0) 0x7f79b407fca0 con 0x7f79cc15a9f0 2024-04-25T10:29:38.366 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:38.363+0000 7f79d0a92640 1 -- 172.21.15.37:0/2212962819 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"} v 0) -- 0x7f79cc071d10 con 0x7f79cc15a9f0 2024-04-25T10:29:38.366 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:38.364+0000 7f79c2ffd640 1 -- 172.21.15.37:0/2212962819 <== mon.0 v2:172.21.15.37:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]=0 v23) ==== 104+0+11 (secure 0 0 0) 0x7f79b407f5a0 con 0x7f79cc15a9f0 2024-04-25T10:29:38.366 INFO:teuthology.orchestra.run.smithi037.stdout:pg_num: 32 2024-04-25T10:29:38.368 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:38.366+0000 7f79d0a92640 1 -- 172.21.15.37:0/2212962819 >> [v2:172.21.15.203:6832/2418205237,v1:172.21.15.203:6833/2418205237] conn(0x7f79ac0744b0 msgr2=0x7f79ac0769c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:29:38.368 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:38.366+0000 7f79d0a92640 1 --2- 172.21.15.37:0/2212962819 >> [v2:172.21.15.203:6832/2418205237,v1:172.21.15.203:6833/2418205237] conn(0x7f79ac0744b0 0x7f79ac0769c0 secure :-1 s=READY pgs=123 cs=0 l=1 rev1=1 crypto rx=0x7f79cc073140 tx=0x7f79c4008040 comp rx=0 tx=0).stop 2024-04-25T10:29:38.368 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:38.366+0000 7f79d0a92640 1 -- 172.21.15.37:0/2212962819 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f79cc15a9f0 msgr2=0x7f79cc15aed0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:29:38.369 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:38.366+0000 7f79d0a92640 1 --2- 172.21.15.37:0/2212962819 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f79cc15a9f0 0x7f79cc15aed0 secure :-1 s=READY pgs=212 cs=0 l=1 rev1=1 crypto rx=0x7f79b4035040 tx=0x7f79b40046c0 comp rx=0 tx=0).stop 2024-04-25T10:29:38.369 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:38.366+0000 7f79d0a92640 1 -- 172.21.15.37:0/2212962819 shutdown_connections 2024-04-25T10:29:38.369 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:38.366+0000 7f79d0a92640 1 --2- 172.21.15.37:0/2212962819 >> [v2:172.21.15.203:6832/2418205237,v1:172.21.15.203:6833/2418205237] conn(0x7f79ac0744b0 0x7f79ac0769c0 unknown :-1 s=CLOSED pgs=123 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:29:38.369 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:38.366+0000 7f79d0a92640 1 --2- 172.21.15.37:0/2212962819 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f79cc160e50 0x7f79cc15b410 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:29:38.369 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:38.366+0000 7f79d0a92640 1 --2- 172.21.15.37:0/2212962819 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f79cc15a9f0 0x7f79cc15aed0 unknown :-1 s=CLOSED pgs=212 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:29:38.369 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:38.366+0000 7f79d0a92640 1 --2- 172.21.15.37:0/2212962819 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f79cc06ca80 0x7f79cc0cc800 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:29:38.369 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:38.366+0000 7f79d0a92640 1 -- 172.21.15.37:0/2212962819 >> 172.21.15.37:0/2212962819 conn(0x7f79cc0b4560 msgr2=0x7f79cc163560 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:29:38.369 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:38.366+0000 7f79d0a92640 1 -- 172.21.15.37:0/2212962819 shutdown_connections 2024-04-25T10:29:38.369 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:38.366+0000 7f79d0a92640 1 -- 172.21.15.37:0/2212962819 wait complete. 2024-04-25T10:29:38.379 DEBUG:teuthology.orchestra.run.smithi037:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data pg_num 2024-04-25T10:29:38.494 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:38.492+0000 7fa722f3f640 1 Processor -- start 2024-04-25T10:29:38.494 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:38.492+0000 7fa722f3f640 1 -- start start 2024-04-25T10:29:38.497 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:38.493+0000 7fa722f3f640 1 --2- >> v2:172.21.15.203:3300/0 conn(0x7fa71c06e260 0x7fa71c06e670 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:29:38.497 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:38.493+0000 7fa722f3f640 1 --2- >> v2:172.21.15.37:3300/0 conn(0x7fa71c06ebb0 0x7fa71c1645d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:29:38.498 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:38.493+0000 7fa722f3f640 1 -- --> v1:172.21.15.203:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fa71c09ce70 con 0x7fa71c164b10 2024-04-25T10:29:38.498 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:38.493+0000 7fa722f3f640 1 -- --> v2:172.21.15.37:3300/0 -- mon_getmap magic: 0 -- 0x7fa71c0d3370 con 0x7fa71c06ebb0 2024-04-25T10:29:38.498 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:38.493+0000 7fa722f3f640 1 -- --> v2:172.21.15.203:3300/0 -- mon_getmap magic: 0 -- 0x7fa71c0d3090 con 0x7fa71c06e260 2024-04-25T10:29:38.498 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:38.493+0000 7fa71bfff640 1 --2- >> v2:172.21.15.37:3300/0 conn(0x7fa71c06ebb0 0x7fa71c1645d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:29:38.498 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:38.493+0000 7fa71bfff640 1 --2- >> v2:172.21.15.37:3300/0 conn(0x7fa71c06ebb0 0x7fa71c1645d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.37:3300/0 says I am v2:172.21.15.37:36898/0 (socket says 172.21.15.37:36898) 2024-04-25T10:29:38.498 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:38.493+0000 7fa71bfff640 1 -- 172.21.15.37:0/2306960279 learned_addr learned my addr 172.21.15.37:0/2306960279 (peer_addr_for_me v2:172.21.15.37:0/0) 2024-04-25T10:29:38.498 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:38.493+0000 7fa720cdb640 1 --2- 172.21.15.37:0/2306960279 >> v2:172.21.15.203:3300/0 conn(0x7fa71c06e260 0x7fa71c06e670 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:29:38.498 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:38.493+0000 7fa71bfff640 1 -- 172.21.15.37:0/2306960279 >> v2:172.21.15.203:3300/0 conn(0x7fa71c06e260 msgr2=0x7fa71c06e670 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:29:38.498 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:38.493+0000 7fa71bfff640 1 --2- 172.21.15.37:0/2306960279 >> v2:172.21.15.203:3300/0 conn(0x7fa71c06e260 0x7fa71c06e670 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:29:38.498 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:38.493+0000 7fa71bfff640 1 -- 172.21.15.37:0/2306960279 >> v1:172.21.15.203:6789/0 conn(0x7fa71c164b10 legacy=0x7fa71c0d1ec0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:29:38.498 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:38.493+0000 7fa71bfff640 1 -- 172.21.15.37:0/2306960279 --> v2:172.21.15.37:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fa71c09d150 con 0x7fa71c06ebb0 2024-04-25T10:29:38.498 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:38.494+0000 7fa71bfff640 1 --2- 172.21.15.37:0/2306960279 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fa71c06ebb0 0x7fa71c1645d0 secure :-1 s=READY pgs=213 cs=0 l=1 rev1=1 crypto rx=0x7fa71c074460 tx=0x7fa70c02a6a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=81118bb949945e69 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:29:38.498 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:38.494+0000 7fa71b7fe640 1 -- 172.21.15.37:0/2306960279 <== mon.0 v2:172.21.15.37:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa70c036070 con 0x7fa71c06ebb0 2024-04-25T10:29:38.499 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:38.494+0000 7fa71b7fe640 1 -- 172.21.15.37:0/2306960279 <== mon.0 v2:172.21.15.37:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fa70c031600 con 0x7fa71c06ebb0 2024-04-25T10:29:38.499 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:38.494+0000 7fa71b7fe640 1 -- 172.21.15.37:0/2306960279 <== mon.0 v2:172.21.15.37:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa70c0319d0 con 0x7fa71c06ebb0 2024-04-25T10:29:38.499 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:38.495+0000 7fa722f3f640 1 -- 172.21.15.37:0/2306960279 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fa71c06ebb0 msgr2=0x7fa71c1645d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:29:38.499 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:38.495+0000 7fa722f3f640 1 --2- 172.21.15.37:0/2306960279 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fa71c06ebb0 0x7fa71c1645d0 secure :-1 s=READY pgs=213 cs=0 l=1 rev1=1 crypto rx=0x7fa71c074460 tx=0x7fa70c02a6a0 comp rx=0 tx=0).stop 2024-04-25T10:29:38.499 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:38.495+0000 7fa722f3f640 1 -- 172.21.15.37:0/2306960279 shutdown_connections 2024-04-25T10:29:38.499 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:38.495+0000 7fa722f3f640 1 --2- 172.21.15.37:0/2306960279 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fa71c06ebb0 0x7fa71c1645d0 unknown :-1 s=CLOSED pgs=213 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:29:38.499 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:38.495+0000 7fa722f3f640 1 --2- 172.21.15.37:0/2306960279 >> v2:172.21.15.203:3300/0 conn(0x7fa71c06e260 0x7fa71c06e670 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:29:38.499 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:38.495+0000 7fa722f3f640 1 -- 172.21.15.37:0/2306960279 >> 172.21.15.37:0/2306960279 conn(0x7fa71c0d0390 msgr2=0x7fa71c0d07a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:29:38.499 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:38.495+0000 7fa722f3f640 1 -- 172.21.15.37:0/2306960279 shutdown_connections 2024-04-25T10:29:38.499 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:38.495+0000 7fa722f3f640 1 -- 172.21.15.37:0/2306960279 wait complete. 2024-04-25T10:29:38.500 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:38.497+0000 7fa722f3f640 1 Processor -- start 2024-04-25T10:29:38.500 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:38.497+0000 7fa722f3f640 1 -- start start 2024-04-25T10:29:38.502 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:38.498+0000 7fa722f3f640 1 --2- >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7fa71c06e260 0x7fa71c153f40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:29:38.502 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:38.498+0000 7fa722f3f640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fa71c06ebb0 0x7fa71c1f65a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:29:38.502 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:38.498+0000 7fa722f3f640 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fa71c164b10 0x7fa71c1f6ae0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:29:38.502 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:38.498+0000 7fa722f3f640 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 -- 0x7fa71c06db30 con 0x7fa71c06ebb0 2024-04-25T10:29:38.502 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:38.498+0000 7fa722f3f640 1 -- --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_getmap magic: 0 -- 0x7fa71c06d830 con 0x7fa71c164b10 2024-04-25T10:29:38.502 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:38.498+0000 7fa722f3f640 1 -- --> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] -- mon_getmap magic: 0 -- 0x7fa71c06dcb0 con 0x7fa71c06e260 2024-04-25T10:29:38.502 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:38.498+0000 7fa71bfff640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fa71c06ebb0 0x7fa71c1f65a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:29:38.502 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:38.498+0000 7fa71bfff640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fa71c06ebb0 0x7fa71c1f65a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.37:3300/0 says I am v2:172.21.15.37:36912/0 (socket says 172.21.15.37:36912) 2024-04-25T10:29:38.502 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:38.498+0000 7fa71bfff640 1 -- 172.21.15.37:0/3517841889 learned_addr learned my addr 172.21.15.37:0/3517841889 (peer_addr_for_me v2:172.21.15.37:0/0) 2024-04-25T10:29:38.502 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:38.499+0000 7fa720cdb640 1 --2- 172.21.15.37:0/3517841889 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7fa71c06e260 0x7fa71c153f40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:29:38.502 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:38.499+0000 7fa7214dc640 1 --2- 172.21.15.37:0/3517841889 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fa71c164b10 0x7fa71c1f6ae0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:29:38.503 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:38.499+0000 7fa71bfff640 1 -- 172.21.15.37:0/3517841889 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7fa71c06e260 msgr2=0x7fa71c153f40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:29:38.503 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:38.499+0000 7fa71bfff640 1 --2- 172.21.15.37:0/3517841889 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7fa71c06e260 0x7fa71c153f40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:29:38.503 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:38.499+0000 7fa71bfff640 1 -- 172.21.15.37:0/3517841889 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fa71c164b10 msgr2=0x7fa71c1f6ae0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:29:38.503 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:38.499+0000 7fa71bfff640 1 --2- 172.21.15.37:0/3517841889 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fa71c164b10 0x7fa71c1f6ae0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:29:38.503 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:38.499+0000 7fa71bfff640 1 -- 172.21.15.37:0/3517841889 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fa71c1e9cd0 con 0x7fa71c06ebb0 2024-04-25T10:29:38.503 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:38.499+0000 7fa71bfff640 1 --2- 172.21.15.37:0/3517841889 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fa71c06ebb0 0x7fa71c1f65a0 secure :-1 s=READY pgs=214 cs=0 l=1 rev1=1 crypto rx=0x7fa70c000f80 tx=0x7fa70c004a40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:29:38.503 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:38.499+0000 7fa7197fa640 1 -- 172.21.15.37:0/3517841889 <== mon.0 v2:172.21.15.37:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa70c036050 con 0x7fa71c06ebb0 2024-04-25T10:29:38.503 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:38.499+0000 7fa7197fa640 1 -- 172.21.15.37:0/3517841889 <== mon.0 v2:172.21.15.37:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fa70c042070 con 0x7fa71c06ebb0 2024-04-25T10:29:38.503 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:38.499+0000 7fa7197fa640 1 -- 172.21.15.37:0/3517841889 <== mon.0 v2:172.21.15.37:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa70c009510 con 0x7fa71c06ebb0 2024-04-25T10:29:38.503 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:38.499+0000 7fa722f3f640 1 -- 172.21.15.37:0/3517841889 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fa71c1e9f40 con 0x7fa71c06ebb0 2024-04-25T10:29:38.503 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:38.499+0000 7fa722f3f640 1 -- 172.21.15.37:0/3517841889 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fa71c1f7d70 con 0x7fa71c06ebb0 2024-04-25T10:29:38.504 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:38.500+0000 7fa722f3f640 1 -- 172.21.15.37:0/3517841889 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fa71c06a140 con 0x7fa71c06ebb0 2024-04-25T10:29:38.504 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:38.501+0000 7fa7197fa640 1 -- 172.21.15.37:0/3517841889 <== mon.0 v2:172.21.15.37:3300/0 4 ==== mgrmap(e 5) ==== 96782+0+0 (secure 0 0 0) 0x7fa70c007c70 con 0x7fa71c06ebb0 2024-04-25T10:29:38.504 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:38.501+0000 7fa7197fa640 1 --2- 172.21.15.37:0/3517841889 >> [v2:172.21.15.203:6832/2418205237,v1:172.21.15.203:6833/2418205237] conn(0x7fa6fc0744b0 0x7fa6fc0769c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:29:38.504 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:38.502+0000 7fa7197fa640 1 -- 172.21.15.37:0/3517841889 <== mon.0 v2:172.21.15.37:3300/0 5 ==== osd_map(23..23 src has 1..23) ==== 5709+0+0 (secure 0 0 0) 0x7fa70c0b5f20 con 0x7fa71c06ebb0 2024-04-25T10:29:38.504 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:38.502+0000 7fa720cdb640 1 --2- 172.21.15.37:0/3517841889 >> [v2:172.21.15.203:6832/2418205237,v1:172.21.15.203:6833/2418205237] conn(0x7fa6fc0744b0 0x7fa6fc0769c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:29:38.505 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:38.502+0000 7fa720cdb640 1 --2- 172.21.15.37:0/3517841889 >> [v2:172.21.15.203:6832/2418205237,v1:172.21.15.203:6833/2418205237] conn(0x7fa6fc0744b0 0x7fa6fc0769c0 secure :-1 s=READY pgs=124 cs=0 l=1 rev1=1 crypto rx=0x7fa71c0d1490 tx=0x7fa714009190 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:29:38.508 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:38.506+0000 7fa7197fa640 1 -- 172.21.15.37:0/3517841889 <== mon.0 v2:172.21.15.37:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187246 (secure 0 0 0) 0x7fa70c080060 con 0x7fa71c06ebb0 2024-04-25T10:29:38.692 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:38.690+0000 7fa722f3f640 1 -- 172.21.15.37:0/3517841889 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"} v 0) -- 0x7fa71c0d11a0 con 0x7fa71c06ebb0 2024-04-25T10:29:38.692 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:38.690+0000 7fa7197fa640 1 -- 172.21.15.37:0/3517841889 <== mon.0 v2:172.21.15.37:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]=0 v23) ==== 100+0+11 (secure 0 0 0) 0x7fa70c084360 con 0x7fa71c06ebb0 2024-04-25T10:29:38.693 INFO:teuthology.orchestra.run.smithi037.stdout:pg_num: 64 2024-04-25T10:29:38.695 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:38.692+0000 7fa722f3f640 1 -- 172.21.15.37:0/3517841889 >> [v2:172.21.15.203:6832/2418205237,v1:172.21.15.203:6833/2418205237] conn(0x7fa6fc0744b0 msgr2=0x7fa6fc0769c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:29:38.695 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:38.692+0000 7fa722f3f640 1 --2- 172.21.15.37:0/3517841889 >> [v2:172.21.15.203:6832/2418205237,v1:172.21.15.203:6833/2418205237] conn(0x7fa6fc0744b0 0x7fa6fc0769c0 secure :-1 s=READY pgs=124 cs=0 l=1 rev1=1 crypto rx=0x7fa71c0d1490 tx=0x7fa714009190 comp rx=0 tx=0).stop 2024-04-25T10:29:38.695 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:38.692+0000 7fa722f3f640 1 -- 172.21.15.37:0/3517841889 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fa71c06ebb0 msgr2=0x7fa71c1f65a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:29:38.695 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:38.692+0000 7fa722f3f640 1 --2- 172.21.15.37:0/3517841889 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fa71c06ebb0 0x7fa71c1f65a0 secure :-1 s=READY pgs=214 cs=0 l=1 rev1=1 crypto rx=0x7fa70c000f80 tx=0x7fa70c004a40 comp rx=0 tx=0).stop 2024-04-25T10:29:38.695 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:38.692+0000 7fa722f3f640 1 -- 172.21.15.37:0/3517841889 shutdown_connections 2024-04-25T10:29:38.695 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:38.692+0000 7fa722f3f640 1 --2- 172.21.15.37:0/3517841889 >> [v2:172.21.15.203:6832/2418205237,v1:172.21.15.203:6833/2418205237] conn(0x7fa6fc0744b0 0x7fa6fc0769c0 unknown :-1 s=CLOSED pgs=124 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:29:38.695 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:38.692+0000 7fa722f3f640 1 --2- 172.21.15.37:0/3517841889 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fa71c164b10 0x7fa71c1f6ae0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:29:38.695 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:38.692+0000 7fa722f3f640 1 --2- 172.21.15.37:0/3517841889 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fa71c06ebb0 0x7fa71c1f65a0 unknown :-1 s=CLOSED pgs=214 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:29:38.695 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:38.692+0000 7fa722f3f640 1 --2- 172.21.15.37:0/3517841889 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7fa71c06e260 0x7fa71c153f40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:29:38.695 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:38.692+0000 7fa722f3f640 1 -- 172.21.15.37:0/3517841889 >> 172.21.15.37:0/3517841889 conn(0x7fa71c0d0390 msgr2=0x7fa71c09bab0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:29:38.695 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:38.692+0000 7fa722f3f640 1 -- 172.21.15.37:0/3517841889 shutdown_connections 2024-04-25T10:29:38.695 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:38.692+0000 7fa722f3f640 1 -- 172.21.15.37:0/3517841889 wait complete. 2024-04-25T10:29:38.706 DEBUG:teuthology.orchestra.run.smithi037:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-04-25T10:29:38.818 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:38.816+0000 7f15a7d38640 1 Processor -- start 2024-04-25T10:29:38.818 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:38.816+0000 7f15a7d38640 1 -- start start 2024-04-25T10:29:38.820 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:38.817+0000 7f15a7d38640 1 --2- >> v2:172.21.15.203:3300/0 conn(0x7f15a008cc90 0x7f15a009cbc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:29:38.820 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:38.817+0000 7f15a7d38640 1 --2- >> v2:172.21.15.37:3300/0 conn(0x7f15a00b71b0 0x7f15a00b75c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:29:38.820 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:38.817+0000 7f15a7d38640 1 -- --> v1:172.21.15.203:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f15a00b8d40 con 0x7f15a00d1c20 2024-04-25T10:29:38.820 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:38.817+0000 7f15a7d38640 1 -- --> v2:172.21.15.37:3300/0 -- mon_getmap magic: 0 -- 0x7f15a00d27f0 con 0x7f15a00b71b0 2024-04-25T10:29:38.820 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:38.817+0000 7f15a7d38640 1 -- --> v2:172.21.15.203:3300/0 -- mon_getmap magic: 0 -- 0x7f15a00d2ad0 con 0x7f15a008cc90 2024-04-25T10:29:38.820 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:38.817+0000 7f15a62d5640 1 --2- >> v2:172.21.15.37:3300/0 conn(0x7f15a00b71b0 0x7f15a00b75c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:29:38.820 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:38.817+0000 7f15a62d5640 1 --2- >> v2:172.21.15.37:3300/0 conn(0x7f15a00b71b0 0x7f15a00b75c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.37:3300/0 says I am v2:172.21.15.37:36920/0 (socket says 172.21.15.37:36920) 2024-04-25T10:29:38.820 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:38.817+0000 7f15a62d5640 1 -- 172.21.15.37:0/425500658 learned_addr learned my addr 172.21.15.37:0/425500658 (peer_addr_for_me v2:172.21.15.37:0/0) 2024-04-25T10:29:38.820 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:38.818+0000 7f15a62d5640 1 -- 172.21.15.37:0/425500658 >> v2:172.21.15.203:3300/0 conn(0x7f15a008cc90 msgr2=0x7f15a009cbc0 unknown :-1 s=STATE_CONNECTING_RE l=0).mark_down 2024-04-25T10:29:38.820 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:38.818+0000 7f15a62d5640 1 --2- 172.21.15.37:0/425500658 >> v2:172.21.15.203:3300/0 conn(0x7f15a008cc90 0x7f15a009cbc0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:29:38.821 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:38.818+0000 7f15a62d5640 1 -- 172.21.15.37:0/425500658 >> v1:172.21.15.203:6789/0 conn(0x7f15a00d1c20 legacy=0x7f15a00d2030 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:29:38.821 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:38.818+0000 7f15a62d5640 1 -- 172.21.15.37:0/425500658 --> v2:172.21.15.37:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f15a00cf7a0 con 0x7f15a00b71b0 2024-04-25T10:29:38.822 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:38.818+0000 7f15a62d5640 1 --2- 172.21.15.37:0/425500658 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f15a00b71b0 0x7f15a00b75c0 secure :-1 s=READY pgs=215 cs=0 l=1 rev1=1 crypto rx=0x7f15a0069f80 tx=0x7f159802a710 comp rx=0 tx=0).ready entity=mon.0 client_cookie=2fb1f41cd95c1b9b server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:29:38.822 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:38.819+0000 7f15a4ad2640 1 -- 172.21.15.37:0/425500658 <== mon.0 v2:172.21.15.37:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1598036070 con 0x7f15a00b71b0 2024-04-25T10:29:38.822 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:38.819+0000 7f15a4ad2640 1 -- 172.21.15.37:0/425500658 <== mon.0 v2:172.21.15.37:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f1598031600 con 0x7f15a00b71b0 2024-04-25T10:29:38.822 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:38.819+0000 7f15a4ad2640 1 -- 172.21.15.37:0/425500658 <== mon.0 v2:172.21.15.37:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f15980319d0 con 0x7f15a00b71b0 2024-04-25T10:29:38.822 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:38.819+0000 7f15a7d38640 1 -- 172.21.15.37:0/425500658 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f15a00b71b0 msgr2=0x7f15a00b75c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:29:38.822 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:38.819+0000 7f15a7d38640 1 --2- 172.21.15.37:0/425500658 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f15a00b71b0 0x7f15a00b75c0 secure :-1 s=READY pgs=215 cs=0 l=1 rev1=1 crypto rx=0x7f15a0069f80 tx=0x7f159802a710 comp rx=0 tx=0).stop 2024-04-25T10:29:38.822 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:38.819+0000 7f15a7d38640 1 -- 172.21.15.37:0/425500658 shutdown_connections 2024-04-25T10:29:38.822 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:38.819+0000 7f15a7d38640 1 --2- 172.21.15.37:0/425500658 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f15a00b71b0 0x7f15a00b75c0 unknown :-1 s=CLOSED pgs=215 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:29:38.822 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:38.819+0000 7f15a7d38640 1 --2- 172.21.15.37:0/425500658 >> v2:172.21.15.203:3300/0 conn(0x7f15a008cc90 0x7f15a009cbc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:29:38.822 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:38.819+0000 7f15a7d38640 1 -- 172.21.15.37:0/425500658 >> 172.21.15.37:0/425500658 conn(0x7f15a00d03d0 msgr2=0x7f15a00d07e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:29:38.822 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:38.819+0000 7f15a7d38640 1 -- 172.21.15.37:0/425500658 shutdown_connections 2024-04-25T10:29:38.822 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:38.819+0000 7f15a7d38640 1 -- 172.21.15.37:0/425500658 wait complete. 2024-04-25T10:29:38.823 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:38.821+0000 7f15a7d38640 1 Processor -- start 2024-04-25T10:29:38.823 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:38.821+0000 7f15a7d38640 1 -- start start 2024-04-25T10:29:38.826 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:38.822+0000 7f15a7d38640 1 --2- >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f15a00b71b0 0x7f15a01db5e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:29:38.826 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:38.822+0000 7f15a7d38640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f15a00d1c20 0x7f15a01d9f60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:29:38.826 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:38.822+0000 7f15a7d38640 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f15a0071390 0x7f15a00717a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:29:38.826 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:38.822+0000 7f15a7d38640 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 -- 0x7f15a00b7b00 con 0x7f15a00d1c20 2024-04-25T10:29:38.826 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:38.822+0000 7f15a7d38640 1 -- --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_getmap magic: 0 -- 0x7f15a00b93b0 con 0x7f15a0071390 2024-04-25T10:29:38.826 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:38.822+0000 7f15a7d38640 1 -- --> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] -- mon_getmap magic: 0 -- 0x7f15a00b7c80 con 0x7f15a00b71b0 2024-04-25T10:29:38.826 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:38.823+0000 7f15a52d3640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f15a00d1c20 0x7f15a01d9f60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:29:38.826 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:38.823+0000 7f15a52d3640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f15a00d1c20 0x7f15a01d9f60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.37:3300/0 says I am v2:172.21.15.37:36928/0 (socket says 172.21.15.37:36928) 2024-04-25T10:29:38.826 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:38.823+0000 7f15a52d3640 1 -- 172.21.15.37:0/437687525 learned_addr learned my addr 172.21.15.37:0/437687525 (peer_addr_for_me v2:172.21.15.37:0/0) 2024-04-25T10:29:38.827 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:38.823+0000 7f15a62d5640 1 --2- 172.21.15.37:0/437687525 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f15a0071390 0x7f15a00717a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:29:38.827 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:38.823+0000 7f15a5ad4640 1 --2- 172.21.15.37:0/437687525 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f15a00b71b0 0x7f15a01db5e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:29:38.827 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:38.823+0000 7f15a52d3640 1 -- 172.21.15.37:0/437687525 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f15a00b71b0 msgr2=0x7f15a01db5e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:29:38.827 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:38.823+0000 7f15a52d3640 1 --2- 172.21.15.37:0/437687525 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f15a00b71b0 0x7f15a01db5e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:29:38.827 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:38.823+0000 7f15a52d3640 1 -- 172.21.15.37:0/437687525 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f15a0071390 msgr2=0x7f15a00717a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:29:38.827 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:38.823+0000 7f15a52d3640 1 --2- 172.21.15.37:0/437687525 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f15a0071390 0x7f15a00717a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:29:38.827 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:38.823+0000 7f15a52d3640 1 -- 172.21.15.37:0/437687525 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f15a0071ce0 con 0x7f15a00d1c20 2024-04-25T10:29:38.827 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:38.823+0000 7f15a5ad4640 1 --2- 172.21.15.37:0/437687525 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f15a00b71b0 0x7f15a01db5e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T10:29:38.827 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:38.823+0000 7f15a52d3640 1 --2- 172.21.15.37:0/437687525 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f15a00d1c20 0x7f15a01d9f60 secure :-1 s=READY pgs=216 cs=0 l=1 rev1=1 crypto rx=0x7f159000b3e0 tx=0x7f159000b8a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:29:38.827 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:38.823+0000 7f15967fc640 1 -- 172.21.15.37:0/437687525 <== mon.0 v2:172.21.15.37:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1590015020 con 0x7f15a00d1c20 2024-04-25T10:29:38.827 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:38.823+0000 7f15967fc640 1 -- 172.21.15.37:0/437687525 <== mon.0 v2:172.21.15.37:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f1590011830 con 0x7f15a00d1c20 2024-04-25T10:29:38.827 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:38.823+0000 7f15a7d38640 1 -- 172.21.15.37:0/437687525 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f15a01dc0a0 con 0x7f15a00d1c20 2024-04-25T10:29:38.827 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:38.823+0000 7f15967fc640 1 -- 172.21.15.37:0/437687525 <== mon.0 v2:172.21.15.37:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1590011c20 con 0x7f15a00d1c20 2024-04-25T10:29:38.827 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:38.823+0000 7f15a7d38640 1 -- 172.21.15.37:0/437687525 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f15a01dc660 con 0x7f15a00d1c20 2024-04-25T10:29:38.828 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:38.824+0000 7f15a7d38640 1 -- 172.21.15.37:0/437687525 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f15a006a140 con 0x7f15a00d1c20 2024-04-25T10:29:38.828 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:38.825+0000 7f15967fc640 1 -- 172.21.15.37:0/437687525 <== mon.0 v2:172.21.15.37:3300/0 4 ==== mgrmap(e 5) ==== 96782+0+0 (secure 0 0 0) 0x7f1590018020 con 0x7f15a00d1c20 2024-04-25T10:29:38.828 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:38.826+0000 7f15967fc640 1 --2- 172.21.15.37:0/437687525 >> [v2:172.21.15.203:6832/2418205237,v1:172.21.15.203:6833/2418205237] conn(0x7f15840744b0 0x7f15840769c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:29:38.829 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:38.826+0000 7f15967fc640 1 -- 172.21.15.37:0/437687525 <== mon.0 v2:172.21.15.37:3300/0 5 ==== osd_map(23..23 src has 1..23) ==== 5709+0+0 (secure 0 0 0) 0x7f1590095140 con 0x7f15a00d1c20 2024-04-25T10:29:38.829 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:38.826+0000 7f15a5ad4640 1 --2- 172.21.15.37:0/437687525 >> [v2:172.21.15.203:6832/2418205237,v1:172.21.15.203:6833/2418205237] conn(0x7f15840744b0 0x7f15840769c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:29:38.829 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:38.827+0000 7f15a5ad4640 1 --2- 172.21.15.37:0/437687525 >> [v2:172.21.15.203:6832/2418205237,v1:172.21.15.203:6833/2418205237] conn(0x7f15840744b0 0x7f15840769c0 secure :-1 s=READY pgs=125 cs=0 l=1 rev1=1 crypto rx=0x7f159c006fd0 tx=0x7f159c008040 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:29:38.832 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:38.830+0000 7f15967fc640 1 -- 172.21.15.37:0/437687525 <== mon.0 v2:172.21.15.37:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187246 (secure 0 0 0) 0x7f159001f020 con 0x7f15a00d1c20 2024-04-25T10:29:39.018 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:39.016+0000 7f15a7d38640 1 -- 172.21.15.37:0/437687525 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7f15a0071ff0 con 0x7f15a00d1c20 2024-04-25T10:29:39.019 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:39.017+0000 7f15967fc640 1 -- 172.21.15.37:0/437687525 <== mon.0 v2:172.21.15.37:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v23) ==== 74+0+16621 (secure 0 0 0) 0x7f15a0071ce0 con 0x7f15a00d1c20 2024-04-25T10:29:39.020 INFO:teuthology.orchestra.run.smithi037.stdout: 2024-04-25T10:29:39.020 INFO:teuthology.orchestra.run.smithi037.stdout:{"epoch":23,"fsid":"f4acda28-be34-492a-b286-994afd36f61d","created":"2024-04-25T10:28:43.985718+0000","modified":"2024-04-25T10:29:10.016188+0000","last_up_change":"2024-04-25T10:28:54.900654+0000","last_in_change":"2024-04-25T10:28:46.216973+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":3,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-04-25T10:28:56.202153+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"19","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2024-04-25T10:29:06.507244+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"22","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":2.25,"score_stable":2.25,"optimal_score":1,"raw_score_acting":2.25,"raw_score_stable":2.25,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2024-04-25T10:29:07.347844+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"23","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"a2f79028-b332-4166-9ebb-a7024a37ea22","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":21,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.37:6816","nonce":2746074720},{"type":"v1","addr":"172.21.15.37:6817","nonce":2746074720}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.37:6818","nonce":2746074720},{"type":"v1","addr":"172.21.15.37:6819","nonce":2746074720}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.37:6822","nonce":2746074720},{"type":"v1","addr":"172.21.15.37:6823","nonce":2746074720}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.37:6820","nonce":2746074720},{"type":"v1","addr":"172.21.15.37:6821","nonce":2746074720}]},"public_addr":"172.21.15.37:6817/2746074720","cluster_addr":"172.21.15.37:6819/2746074720","heartbeat_back_addr":"172.21.15.37:6823/2746074720","heartbeat_front_addr":"172.21.15.37:6821/2746074720","state":["exists","up"]},{"osd":1,"uuid":"973159de-c2ea-4083-aecc-69373d1c1132","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":21,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.37:6800","nonce":1690341662},{"type":"v1","addr":"172.21.15.37:6801","nonce":1690341662}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.37:6802","nonce":1690341662},{"type":"v1","addr":"172.21.15.37:6803","nonce":1690341662}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.37:6806","nonce":1690341662},{"type":"v1","addr":"172.21.15.37:6807","nonce":1690341662}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.37:6804","nonce":1690341662},{"type":"v1","addr":"172.21.15.37:6805","nonce":1690341662}]},"public_addr":"172.21.15.37:6801/1690341662","cluster_addr":"172.21.15.37:6803/1690341662","heartbeat_back_addr":"172.21.15.37:6807/1690341662","heartbeat_front_addr":"172.21.15.37:6805/1690341662","state":["exists","up"]},{"osd":2,"uuid":"4ae7a0e9-cb3f-484c-a75f-f3766b721c47","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":21,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.37:6808","nonce":2754669649},{"type":"v1","addr":"172.21.15.37:6809","nonce":2754669649}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.37:6810","nonce":2754669649},{"type":"v1","addr":"172.21.15.37:6811","nonce":2754669649}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.37:6814","nonce":2754669649},{"type":"v1","addr":"172.21.15.37:6815","nonce":2754669649}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.37:6812","nonce":2754669649},{"type":"v1","addr":"172.21.15.37:6813","nonce":2754669649}]},"public_addr":"172.21.15.37:6809/2754669649","cluster_addr":"172.21.15.37:6811/2754669649","heartbeat_back_addr":"172.21.15.37:6815/2754669649","heartbeat_front_addr":"172.21.15.37:6813/2754669649","state":["exists","up"]},{"osd":3,"uuid":"1ba4a746-1608-47a7-ace2-960a5cfae934","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":21,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.37:6824","nonce":1815290618},{"type":"v1","addr":"172.21.15.37:6825","nonce":1815290618}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.37:6826","nonce":1815290618},{"type":"v1","addr":"172.21.15.37:6827","nonce":1815290618}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.37:6830","nonce":1815290618},{"type":"v1","addr":"172.21.15.37:6831","nonce":1815290618}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.37:6828","nonce":1815290618},{"type":"v1","addr":"172.21.15.37:6829","nonce":1815290618}]},"public_addr":"172.21.15.37:6825/1815290618","cluster_addr":"172.21.15.37:6827/1815290618","heartbeat_back_addr":"172.21.15.37:6831/1815290618","heartbeat_front_addr":"172.21.15.37:6829/1815290618","state":["exists","up"]},{"osd":4,"uuid":"d4f59f5d-0413-4b79-87ee-bb212c1fcf62","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":21,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6800","nonce":4051532063},{"type":"v1","addr":"172.21.15.203:6801","nonce":4051532063}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6802","nonce":4051532063},{"type":"v1","addr":"172.21.15.203:6803","nonce":4051532063}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6806","nonce":4051532063},{"type":"v1","addr":"172.21.15.203:6807","nonce":4051532063}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6804","nonce":4051532063},{"type":"v1","addr":"172.21.15.203:6805","nonce":4051532063}]},"public_addr":"172.21.15.203:6801/4051532063","cluster_addr":"172.21.15.203:6803/4051532063","heartbeat_back_addr":"172.21.15.203:6807/4051532063","heartbeat_front_addr":"172.21.15.203:6805/4051532063","state":["exists","up"]},{"osd":5,"uuid":"69710a0e-41d8-4e7f-9a10-17ce540b67be","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":21,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6824","nonce":1203822890},{"type":"v1","addr":"172.21.15.203:6825","nonce":1203822890}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6826","nonce":1203822890},{"type":"v1","addr":"172.21.15.203:6827","nonce":1203822890}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6830","nonce":1203822890},{"type":"v1","addr":"172.21.15.203:6831","nonce":1203822890}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6828","nonce":1203822890},{"type":"v1","addr":"172.21.15.203:6829","nonce":1203822890}]},"public_addr":"172.21.15.203:6825/1203822890","cluster_addr":"172.21.15.203:6827/1203822890","heartbeat_back_addr":"172.21.15.203:6831/1203822890","heartbeat_front_addr":"172.21.15.203:6829/1203822890","state":["exists","up"]},{"osd":6,"uuid":"f29dd047-3b48-473d-a235-6642a6293f20","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":21,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6810","nonce":2309615014},{"type":"v1","addr":"172.21.15.203:6812","nonce":2309615014}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6814","nonce":2309615014},{"type":"v1","addr":"172.21.15.203:6816","nonce":2309615014}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6822","nonce":2309615014},{"type":"v1","addr":"172.21.15.203:6823","nonce":2309615014}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6818","nonce":2309615014},{"type":"v1","addr":"172.21.15.203:6820","nonce":2309615014}]},"public_addr":"172.21.15.203:6812/2309615014","cluster_addr":"172.21.15.203:6816/2309615014","heartbeat_back_addr":"172.21.15.203:6823/2309615014","heartbeat_front_addr":"172.21.15.203:6820/2309615014","state":["exists","up"]},{"osd":7,"uuid":"6e691644-378b-4fdc-8e86-fb5f56e31f29","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":21,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6808","nonce":3688099014},{"type":"v1","addr":"172.21.15.203:6809","nonce":3688099014}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6811","nonce":3688099014},{"type":"v1","addr":"172.21.15.203:6813","nonce":3688099014}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6819","nonce":3688099014},{"type":"v1","addr":"172.21.15.203:6821","nonce":3688099014}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6815","nonce":3688099014},{"type":"v1","addr":"172.21.15.203:6817","nonce":3688099014}]},"public_addr":"172.21.15.203:6809/3688099014","cluster_addr":"172.21.15.203:6813/3688099014","heartbeat_back_addr":"172.21.15.203:6821/3688099014","heartbeat_front_addr":"172.21.15.203:6817/3688099014","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T10:28:51.419141+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T10:28:51.366448+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T10:28:51.348276+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T10:28:51.380519+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T10:28:51.105812+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T10:28:51.384496+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T10:28:51.388755+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T10:28:51.366958+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-04-25T10:29:39.022 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:39.019+0000 7f15a7d38640 1 -- 172.21.15.37:0/437687525 >> [v2:172.21.15.203:6832/2418205237,v1:172.21.15.203:6833/2418205237] conn(0x7f15840744b0 msgr2=0x7f15840769c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:29:39.022 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:39.019+0000 7f15a7d38640 1 --2- 172.21.15.37:0/437687525 >> [v2:172.21.15.203:6832/2418205237,v1:172.21.15.203:6833/2418205237] conn(0x7f15840744b0 0x7f15840769c0 secure :-1 s=READY pgs=125 cs=0 l=1 rev1=1 crypto rx=0x7f159c006fd0 tx=0x7f159c008040 comp rx=0 tx=0).stop 2024-04-25T10:29:39.022 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:39.019+0000 7f15a7d38640 1 -- 172.21.15.37:0/437687525 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f15a00d1c20 msgr2=0x7f15a01d9f60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:29:39.022 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:39.019+0000 7f15a7d38640 1 --2- 172.21.15.37:0/437687525 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f15a00d1c20 0x7f15a01d9f60 secure :-1 s=READY pgs=216 cs=0 l=1 rev1=1 crypto rx=0x7f159000b3e0 tx=0x7f159000b8a0 comp rx=0 tx=0).stop 2024-04-25T10:29:39.022 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:39.019+0000 7f15a7d38640 1 -- 172.21.15.37:0/437687525 shutdown_connections 2024-04-25T10:29:39.022 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:39.019+0000 7f15a7d38640 1 --2- 172.21.15.37:0/437687525 >> [v2:172.21.15.203:6832/2418205237,v1:172.21.15.203:6833/2418205237] conn(0x7f15840744b0 0x7f15840769c0 unknown :-1 s=CLOSED pgs=125 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:29:39.022 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:39.019+0000 7f15a7d38640 1 --2- 172.21.15.37:0/437687525 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f15a0071390 0x7f15a00717a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:29:39.022 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:39.019+0000 7f15a7d38640 1 --2- 172.21.15.37:0/437687525 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f15a00d1c20 0x7f15a01d9f60 unknown :-1 s=CLOSED pgs=216 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:29:39.022 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:39.019+0000 7f15a7d38640 1 --2- 172.21.15.37:0/437687525 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f15a00b71b0 0x7f15a01db5e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:29:39.023 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:39.019+0000 7f15a7d38640 1 -- 172.21.15.37:0/437687525 >> 172.21.15.37:0/437687525 conn(0x7f15a00d03d0 msgr2=0x7f15a00d07e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:29:39.023 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:39.019+0000 7f15a7d38640 1 -- 172.21.15.37:0/437687525 shutdown_connections 2024-04-25T10:29:39.023 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:39.019+0000 7f15a7d38640 1 -- 172.21.15.37:0/437687525 wait complete. 2024-04-25T10:29:39.033 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2024-04-25T10:28:56.202153+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '19', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 8, 'score_stable': 8, 'optimal_score': 0.25, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 2, 'pool_name': 'cephfs_metadata', 'create_time': '2024-04-25T10:29:06.507244+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '22', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_autoscale_bias': 4, 'pg_num_min': 16, 'recovery_priority': 5}, 'application_metadata': {'cephfs': {'metadata': 'cephfs'}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 2.25, 'score_stable': 2.25, 'optimal_score': 1, 'raw_score_acting': 2.25, 'raw_score_stable': 2.25, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 3, 'pool_name': 'cephfs_data', 'create_time': '2024-04-25T10:29:07.347844+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 64, 'pg_placement_num': 64, 'pg_placement_num_target': 64, 'pg_num_target': 64, 'pg_num_pending': 64, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '23', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.8999999761581421}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 1.5, 'score_stable': 1.5, 'optimal_score': 1, 'raw_score_acting': 1.5, 'raw_score_stable': 1.5, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}] 2024-04-25T10:29:39.033 DEBUG:teuthology.orchestra.run.smithi037:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2024-04-25T10:29:39.147 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:39.145+0000 7ff68c40c640 1 Processor -- start 2024-04-25T10:29:39.148 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:39.145+0000 7ff68c40c640 1 -- start start 2024-04-25T10:29:39.150 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:39.147+0000 7ff68c40c640 1 --2- >> v2:172.21.15.203:3300/0 conn(0x7ff6840b2dc0 0x7ff6840d2860 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:29:39.150 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:39.147+0000 7ff68c40c640 1 --2- >> v2:172.21.15.37:3300/0 conn(0x7ff68406e610 0x7ff6840ce560 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:29:39.151 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:39.147+0000 7ff68c40c640 1 -- --> v1:172.21.15.37:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7ff6840d2e40 con 0x7ff68406daf0 2024-04-25T10:29:39.151 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:39.147+0000 7ff68c40c640 1 -- --> v2:172.21.15.37:3300/0 -- mon_getmap magic: 0 -- 0x7ff6840b92c0 con 0x7ff68406e610 2024-04-25T10:29:39.151 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:39.147+0000 7ff68c40c640 1 -- --> v2:172.21.15.203:3300/0 -- mon_getmap magic: 0 -- 0x7ff6840ce0d0 con 0x7ff6840b2dc0 2024-04-25T10:29:39.151 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:39.147+0000 7ff68a9a9640 1 --2- >> v2:172.21.15.37:3300/0 conn(0x7ff68406e610 0x7ff6840ce560 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:29:39.151 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:39.147+0000 7ff6899a7640 1 --1- >> v1:172.21.15.37:6789/0 conn(0x7ff68406daf0 0x7ff68406df00 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.37:6789/0 says I am v1:172.21.15.37:46554/0 (socket says 172.21.15.37:46554) 2024-04-25T10:29:39.151 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:39.147+0000 7ff6899a7640 1 -- 172.21.15.37:0/839006231 learned_addr learned my addr 172.21.15.37:0/839006231 (peer_addr_for_me v1:172.21.15.37:0/0) 2024-04-25T10:29:39.151 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:39.147+0000 7ff68a1a8640 1 --2- 172.21.15.37:0/839006231 >> v2:172.21.15.203:3300/0 conn(0x7ff6840b2dc0 0x7ff6840d2860 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:29:39.151 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:39.147+0000 7ff68a9a9640 1 -- 172.21.15.37:0/839006231 >> v2:172.21.15.203:3300/0 conn(0x7ff6840b2dc0 msgr2=0x7ff6840d2860 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:29:39.151 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:39.147+0000 7ff68a9a9640 1 --2- 172.21.15.37:0/839006231 >> v2:172.21.15.203:3300/0 conn(0x7ff6840b2dc0 0x7ff6840d2860 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:29:39.151 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:39.147+0000 7ff68a9a9640 1 -- 172.21.15.37:0/839006231 >> v1:172.21.15.37:6789/0 conn(0x7ff68406daf0 legacy=0x7ff68406df00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:29:39.151 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:39.147+0000 7ff68a9a9640 1 -- 172.21.15.37:0/839006231 --> v2:172.21.15.37:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ff6840d3120 con 0x7ff68406e610 2024-04-25T10:29:39.151 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:39.147+0000 7ff6891a6640 1 -- 172.21.15.37:0/839006231 <== mon.0 v1:172.21.15.37:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 1429990984 0 0) 0x7ff6840d2e40 con 0x7ff68406daf0 2024-04-25T10:29:39.152 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:39.148+0000 7ff68a9a9640 1 --2- 172.21.15.37:0/839006231 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7ff68406e610 0x7ff6840ce560 secure :-1 s=READY pgs=218 cs=0 l=1 rev1=1 crypto rx=0x7ff684070b40 tx=0x7ff67402a680 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d9ee6663fd51b1b server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:29:39.152 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:39.148+0000 7ff6891a6640 1 -- 172.21.15.37:0/839006231 <== mon.0 v2:172.21.15.37:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff674007740 con 0x7ff68406e610 2024-04-25T10:29:39.152 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:39.148+0000 7ff6891a6640 1 -- 172.21.15.37:0/839006231 <== mon.0 v2:172.21.15.37:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7ff6740078e0 con 0x7ff68406e610 2024-04-25T10:29:39.152 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:39.148+0000 7ff6891a6640 1 -- 172.21.15.37:0/839006231 <== mon.0 v2:172.21.15.37:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff674007cb0 con 0x7ff68406e610 2024-04-25T10:29:39.153 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:39.149+0000 7ff68c40c640 1 -- 172.21.15.37:0/839006231 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7ff68406e610 msgr2=0x7ff6840ce560 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:29:39.153 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:39.149+0000 7ff68c40c640 1 --2- 172.21.15.37:0/839006231 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7ff68406e610 0x7ff6840ce560 secure :-1 s=READY pgs=218 cs=0 l=1 rev1=1 crypto rx=0x7ff684070b40 tx=0x7ff67402a680 comp rx=0 tx=0).stop 2024-04-25T10:29:39.153 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:39.149+0000 7ff68c40c640 1 -- 172.21.15.37:0/839006231 shutdown_connections 2024-04-25T10:29:39.153 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:39.149+0000 7ff68c40c640 1 --2- 172.21.15.37:0/839006231 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7ff68406e610 0x7ff6840ce560 unknown :-1 s=CLOSED pgs=218 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:29:39.153 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:39.149+0000 7ff68c40c640 1 --2- 172.21.15.37:0/839006231 >> v2:172.21.15.203:3300/0 conn(0x7ff6840b2dc0 0x7ff6840d2860 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:29:39.153 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:39.149+0000 7ff68c40c640 1 -- 172.21.15.37:0/839006231 >> 172.21.15.37:0/839006231 conn(0x7ff6840b5440 msgr2=0x7ff6840b5850 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:29:39.153 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:39.149+0000 7ff68c40c640 1 -- 172.21.15.37:0/839006231 shutdown_connections 2024-04-25T10:29:39.153 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:39.149+0000 7ff68c40c640 1 -- 172.21.15.37:0/839006231 wait complete. 2024-04-25T10:29:39.153 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:39.151+0000 7ff68c40c640 1 Processor -- start 2024-04-25T10:29:39.154 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:39.151+0000 7ff68c40c640 1 -- start start 2024-04-25T10:29:39.156 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:39.152+0000 7ff68c40c640 1 --2- >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7ff68406daf0 0x7ff684163040 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:29:39.156 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:39.152+0000 7ff68c40c640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7ff68406e610 0x7ff6840c9620 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:29:39.156 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:39.152+0000 7ff68c40c640 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7ff6840b2dc0 0x7ff6841f98c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:29:39.156 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:39.152+0000 7ff68c40c640 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 -- 0x7ff6840ceaa0 con 0x7ff68406e610 2024-04-25T10:29:39.156 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:39.152+0000 7ff68c40c640 1 -- --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_getmap magic: 0 -- 0x7ff6840d36e0 con 0x7ff6840b2dc0 2024-04-25T10:29:39.156 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:39.152+0000 7ff68c40c640 1 -- --> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] -- mon_getmap magic: 0 -- 0x7ff6840cec20 con 0x7ff68406daf0 2024-04-25T10:29:39.156 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:39.152+0000 7ff6899a7640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7ff68406e610 0x7ff6840c9620 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:29:39.156 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:39.152+0000 7ff6899a7640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7ff68406e610 0x7ff6840c9620 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.37:3300/0 says I am v2:172.21.15.37:36944/0 (socket says 172.21.15.37:36944) 2024-04-25T10:29:39.156 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:39.152+0000 7ff6899a7640 1 -- 172.21.15.37:0/3077963087 learned_addr learned my addr 172.21.15.37:0/3077963087 (peer_addr_for_me v2:172.21.15.37:0/0) 2024-04-25T10:29:39.156 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:39.152+0000 7ff68a1a8640 1 --2- 172.21.15.37:0/3077963087 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7ff68406daf0 0x7ff684163040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:29:39.156 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:39.152+0000 7ff68a9a9640 1 --2- 172.21.15.37:0/3077963087 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7ff6840b2dc0 0x7ff6841f98c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:29:39.157 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:39.153+0000 7ff6899a7640 1 -- 172.21.15.37:0/3077963087 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7ff68406daf0 msgr2=0x7ff684163040 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:29:39.157 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:39.153+0000 7ff6899a7640 1 --2- 172.21.15.37:0/3077963087 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7ff68406daf0 0x7ff684163040 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:29:39.157 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:39.153+0000 7ff6899a7640 1 -- 172.21.15.37:0/3077963087 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7ff6840b2dc0 msgr2=0x7ff6841f98c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:29:39.157 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:39.153+0000 7ff6899a7640 1 --2- 172.21.15.37:0/3077963087 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7ff6840b2dc0 0x7ff6841f98c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:29:39.157 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:39.153+0000 7ff6899a7640 1 -- 172.21.15.37:0/3077963087 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ff6840cb960 con 0x7ff68406e610 2024-04-25T10:29:39.157 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:39.153+0000 7ff68a1a8640 1 --2- 172.21.15.37:0/3077963087 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7ff68406daf0 0x7ff684163040 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T10:29:39.157 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:39.153+0000 7ff68a9a9640 1 --2- 172.21.15.37:0/3077963087 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7ff6840b2dc0 0x7ff6841f98c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T10:29:39.157 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:39.153+0000 7ff6899a7640 1 --2- 172.21.15.37:0/3077963087 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7ff68406e610 0x7ff6840c9620 secure :-1 s=READY pgs=219 cs=0 l=1 rev1=1 crypto rx=0x7ff67800c440 tx=0x7ff67800c900 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:29:39.157 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:39.153+0000 7ff682ffd640 1 -- 172.21.15.37:0/3077963087 <== mon.0 v2:172.21.15.37:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff678018070 con 0x7ff68406e610 2024-04-25T10:29:39.157 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:39.153+0000 7ff682ffd640 1 -- 172.21.15.37:0/3077963087 <== mon.0 v2:172.21.15.37:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7ff6780074e0 con 0x7ff68406e610 2024-04-25T10:29:39.157 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:39.153+0000 7ff68c40c640 1 -- 172.21.15.37:0/3077963087 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7ff6840cbbd0 con 0x7ff68406e610 2024-04-25T10:29:39.157 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:39.153+0000 7ff682ffd640 1 -- 172.21.15.37:0/3077963087 <== mon.0 v2:172.21.15.37:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff678004a20 con 0x7ff68406e610 2024-04-25T10:29:39.157 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:39.153+0000 7ff68c40c640 1 -- 172.21.15.37:0/3077963087 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7ff68414be30 con 0x7ff68406e610 2024-04-25T10:29:39.157 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:39.154+0000 7ff68c40c640 1 -- 172.21.15.37:0/3077963087 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7ff68406a140 con 0x7ff68406e610 2024-04-25T10:29:39.158 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:39.155+0000 7ff682ffd640 1 -- 172.21.15.37:0/3077963087 <== mon.0 v2:172.21.15.37:3300/0 4 ==== mgrmap(e 5) ==== 96782+0+0 (secure 0 0 0) 0x7ff678004bc0 con 0x7ff68406e610 2024-04-25T10:29:39.158 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:39.155+0000 7ff682ffd640 1 --2- 172.21.15.37:0/3077963087 >> [v2:172.21.15.203:6832/2418205237,v1:172.21.15.203:6833/2418205237] conn(0x7ff6700744b0 0x7ff6700769c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:29:39.159 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:39.155+0000 7ff682ffd640 1 -- 172.21.15.37:0/3077963087 <== mon.0 v2:172.21.15.37:3300/0 5 ==== osd_map(23..23 src has 1..23) ==== 5709+0+0 (secure 0 0 0) 0x7ff678094590 con 0x7ff68406e610 2024-04-25T10:29:39.159 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:39.156+0000 7ff68a1a8640 1 --2- 172.21.15.37:0/3077963087 >> [v2:172.21.15.203:6832/2418205237,v1:172.21.15.203:6833/2418205237] conn(0x7ff6700744b0 0x7ff6700769c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:29:39.159 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:39.156+0000 7ff68a1a8640 1 --2- 172.21.15.37:0/3077963087 >> [v2:172.21.15.203:6832/2418205237,v1:172.21.15.203:6833/2418205237] conn(0x7ff6700744b0 0x7ff6700769c0 secure :-1 s=READY pgs=126 cs=0 l=1 rev1=1 crypto rx=0x7ff684074cc0 tx=0x7ff67c008040 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:29:39.162 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:39.160+0000 7ff682ffd640 1 -- 172.21.15.37:0/3077963087 <== mon.0 v2:172.21.15.37:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187246 (secure 0 0 0) 0x7ff67805ed30 con 0x7ff68406e610 2024-04-25T10:29:39.348 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:39.345+0000 7ff68c40c640 1 -- 172.21.15.37:0/3077963087 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) -- 0x7ff6840ca030 con 0x7ff68406e610 2024-04-25T10:29:39.348 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:39.346+0000 7ff682ffd640 1 -- 172.21.15.37:0/3077963087 <== mon.0 v2:172.21.15.37:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v23) ==== 93+0+10 (secure 0 0 0) 0x7ff67805e630 con 0x7ff68406e610 2024-04-25T10:29:39.349 INFO:teuthology.orchestra.run.smithi037.stdout:pg_num: 1 2024-04-25T10:29:39.351 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:39.348+0000 7ff68c40c640 1 -- 172.21.15.37:0/3077963087 >> [v2:172.21.15.203:6832/2418205237,v1:172.21.15.203:6833/2418205237] conn(0x7ff6700744b0 msgr2=0x7ff6700769c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:29:39.351 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:39.348+0000 7ff68c40c640 1 --2- 172.21.15.37:0/3077963087 >> [v2:172.21.15.203:6832/2418205237,v1:172.21.15.203:6833/2418205237] conn(0x7ff6700744b0 0x7ff6700769c0 secure :-1 s=READY pgs=126 cs=0 l=1 rev1=1 crypto rx=0x7ff684074cc0 tx=0x7ff67c008040 comp rx=0 tx=0).stop 2024-04-25T10:29:39.351 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:39.348+0000 7ff68c40c640 1 -- 172.21.15.37:0/3077963087 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7ff68406e610 msgr2=0x7ff6840c9620 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:29:39.351 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:39.348+0000 7ff68c40c640 1 --2- 172.21.15.37:0/3077963087 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7ff68406e610 0x7ff6840c9620 secure :-1 s=READY pgs=219 cs=0 l=1 rev1=1 crypto rx=0x7ff67800c440 tx=0x7ff67800c900 comp rx=0 tx=0).stop 2024-04-25T10:29:39.351 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:39.348+0000 7ff68c40c640 1 -- 172.21.15.37:0/3077963087 shutdown_connections 2024-04-25T10:29:39.351 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:39.348+0000 7ff68c40c640 1 --2- 172.21.15.37:0/3077963087 >> [v2:172.21.15.203:6832/2418205237,v1:172.21.15.203:6833/2418205237] conn(0x7ff6700744b0 0x7ff6700769c0 unknown :-1 s=CLOSED pgs=126 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:29:39.351 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:39.348+0000 7ff68c40c640 1 --2- 172.21.15.37:0/3077963087 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7ff6840b2dc0 0x7ff6841f98c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:29:39.351 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:39.348+0000 7ff68c40c640 1 --2- 172.21.15.37:0/3077963087 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7ff68406e610 0x7ff6840c9620 unknown :-1 s=CLOSED pgs=219 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:29:39.352 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:39.348+0000 7ff68c40c640 1 --2- 172.21.15.37:0/3077963087 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7ff68406daf0 0x7ff684163040 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:29:39.352 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:39.348+0000 7ff68c40c640 1 -- 172.21.15.37:0/3077963087 >> 172.21.15.37:0/3077963087 conn(0x7ff6840b5440 msgr2=0x7ff6840b0740 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:29:39.352 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:39.348+0000 7ff68c40c640 1 -- 172.21.15.37:0/3077963087 shutdown_connections 2024-04-25T10:29:39.352 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:39.348+0000 7ff68c40c640 1 -- 172.21.15.37:0/3077963087 wait complete. 2024-04-25T10:29:39.363 DEBUG:teuthology.orchestra.run.smithi037:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_metadata pg_num 2024-04-25T10:29:39.477 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:39.475+0000 7f6cd0f1f640 1 Processor -- start 2024-04-25T10:29:39.477 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:39.475+0000 7f6cd0f1f640 1 -- start start 2024-04-25T10:29:39.479 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:39.476+0000 7f6cd0f1f640 1 --2- >> v2:172.21.15.37:3300/0 conn(0x7f6ccc144490 0x7f6ccc15f760 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:29:39.479 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:39.476+0000 7f6cd0f1f640 1 --2- >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f6ccc15e570 0x7f6ccc0d0780 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:29:39.479 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:39.477+0000 7f6cd0f1f640 1 -- --> v1:172.21.15.37:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f6ccc15eb80 con 0x7f6ccc06c540 2024-04-25T10:29:39.479 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:39.477+0000 7f6cd0f1f640 1 -- --> v2:172.21.15.37:3300/0 -- mon_getmap magic: 0 -- 0x7f6ccc15ee60 con 0x7f6ccc144490 2024-04-25T10:29:39.479 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:39.477+0000 7f6cd0f1f640 1 -- --> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] -- mon_getmap magic: 0 -- 0x7f6ccc15f140 con 0x7f6ccc15e570 2024-04-25T10:29:39.480 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:39.477+0000 7f6cca59c640 1 --2- >> v2:172.21.15.37:3300/0 conn(0x7f6ccc144490 0x7f6ccc15f760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:29:39.480 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:39.477+0000 7f6ccad9d640 1 --1- >> v1:172.21.15.37:6789/0 conn(0x7f6ccc06c540 0x7f6ccc06c950 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.37:6789/0 says I am v1:172.21.15.37:46564/0 (socket says 172.21.15.37:46564) 2024-04-25T10:29:39.480 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:39.477+0000 7f6ccad9d640 1 -- 172.21.15.37:0/410783367 learned_addr learned my addr 172.21.15.37:0/410783367 (peer_addr_for_me v1:172.21.15.37:0/0) 2024-04-25T10:29:39.480 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:39.477+0000 7f6cc9d9b640 1 --2- 172.21.15.37:0/410783367 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f6ccc15e570 0x7f6ccc0d0780 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:29:39.480 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:39.477+0000 7f6cca59c640 1 -- 172.21.15.37:0/410783367 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f6ccc15e570 msgr2=0x7f6ccc0d0780 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:29:39.480 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:39.477+0000 7f6cca59c640 1 --2- 172.21.15.37:0/410783367 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f6ccc15e570 0x7f6ccc0d0780 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:29:39.480 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:39.477+0000 7f6cca59c640 1 -- 172.21.15.37:0/410783367 >> v1:172.21.15.37:6789/0 conn(0x7f6ccc06c540 legacy=0x7f6ccc06c950 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:29:39.480 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:39.477+0000 7f6cca59c640 1 -- 172.21.15.37:0/410783367 --> v2:172.21.15.37:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f6ccc15f420 con 0x7f6ccc144490 2024-04-25T10:29:39.480 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:39.477+0000 7f6cc959a640 1 -- 172.21.15.37:0/410783367 <== mon.0 v1:172.21.15.37:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 1322516516 0 0) 0x7f6ccc15eb80 con 0x7f6ccc06c540 2024-04-25T10:29:39.481 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:39.478+0000 7f6cca59c640 1 --2- 172.21.15.37:0/410783367 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f6ccc144490 0x7f6ccc15f760 secure :-1 s=READY pgs=221 cs=0 l=1 rev1=1 crypto rx=0x7f6cc40097a0 tx=0x7f6cc402a180 comp rx=0 tx=0).ready entity=mon.0 client_cookie=3dc7e1d5a4588c88 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:29:39.481 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:39.478+0000 7f6cc959a640 1 -- 172.21.15.37:0/410783367 <== mon.0 v2:172.21.15.37:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6cc4037070 con 0x7f6ccc144490 2024-04-25T10:29:39.482 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:39.478+0000 7f6cc959a640 1 -- 172.21.15.37:0/410783367 <== mon.0 v2:172.21.15.37:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f6cc4032440 con 0x7f6ccc144490 2024-04-25T10:29:39.482 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:39.478+0000 7f6cc959a640 1 -- 172.21.15.37:0/410783367 <== mon.0 v2:172.21.15.37:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6cc4032810 con 0x7f6ccc144490 2024-04-25T10:29:39.482 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:39.479+0000 7f6cd0f1f640 1 -- 172.21.15.37:0/410783367 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f6ccc144490 msgr2=0x7f6ccc15f760 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:29:39.482 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:39.479+0000 7f6cd0f1f640 1 --2- 172.21.15.37:0/410783367 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f6ccc144490 0x7f6ccc15f760 secure :-1 s=READY pgs=221 cs=0 l=1 rev1=1 crypto rx=0x7f6cc40097a0 tx=0x7f6cc402a180 comp rx=0 tx=0).stop 2024-04-25T10:29:39.482 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:39.479+0000 7f6cd0f1f640 1 -- 172.21.15.37:0/410783367 shutdown_connections 2024-04-25T10:29:39.482 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:39.479+0000 7f6cd0f1f640 1 --2- 172.21.15.37:0/410783367 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f6ccc15e570 0x7f6ccc0d0780 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:29:39.482 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:39.479+0000 7f6cd0f1f640 1 --2- 172.21.15.37:0/410783367 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f6ccc144490 0x7f6ccc15f760 unknown :-1 s=CLOSED pgs=221 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:29:39.482 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:39.479+0000 7f6cd0f1f640 1 -- 172.21.15.37:0/410783367 >> 172.21.15.37:0/410783367 conn(0x7f6ccc0b4560 msgr2=0x7f6ccc0b4970 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:29:39.482 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:39.479+0000 7f6cd0f1f640 1 -- 172.21.15.37:0/410783367 shutdown_connections 2024-04-25T10:29:39.482 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:39.479+0000 7f6cd0f1f640 1 -- 172.21.15.37:0/410783367 wait complete. 2024-04-25T10:29:39.483 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:39.481+0000 7f6cd0f1f640 1 Processor -- start 2024-04-25T10:29:39.483 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:39.481+0000 7f6cd0f1f640 1 -- start start 2024-04-25T10:29:39.485 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:39.482+0000 7f6cd0f1f640 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f6ccc06c540 0x7f6ccc158bc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:29:39.486 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:39.482+0000 7f6cd0f1f640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f6ccc144490 0x7f6ccc0c9820 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:29:39.486 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:39.482+0000 7f6cd0f1f640 1 --2- >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f6ccc15e570 0x7f6ccc0c9d60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:29:39.486 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:39.482+0000 7f6cd0f1f640 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 -- 0x7f6ccc0d0e40 con 0x7f6ccc144490 2024-04-25T10:29:39.486 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:39.482+0000 7f6cd0f1f640 1 -- --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_getmap magic: 0 -- 0x7f6ccc0d0cc0 con 0x7f6ccc06c540 2024-04-25T10:29:39.486 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:39.482+0000 7f6cd0f1f640 1 -- --> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] -- mon_getmap magic: 0 -- 0x7f6ccc0d28d0 con 0x7f6ccc15e570 2024-04-25T10:29:39.486 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:39.482+0000 7f6cc9d9b640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f6ccc144490 0x7f6ccc0c9820 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:29:39.486 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:39.482+0000 7f6cc9d9b640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f6ccc144490 0x7f6ccc0c9820 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.37:3300/0 says I am v2:172.21.15.37:36960/0 (socket says 172.21.15.37:36960) 2024-04-25T10:29:39.486 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:39.482+0000 7f6cc9d9b640 1 -- 172.21.15.37:0/1805526195 learned_addr learned my addr 172.21.15.37:0/1805526195 (peer_addr_for_me v2:172.21.15.37:0/0) 2024-04-25T10:29:39.486 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:39.482+0000 7f6ccad9d640 1 --2- 172.21.15.37:0/1805526195 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f6ccc15e570 0x7f6ccc0c9d60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:29:39.486 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:39.482+0000 7f6cca59c640 1 --2- 172.21.15.37:0/1805526195 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f6ccc06c540 0x7f6ccc158bc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:29:39.486 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:39.482+0000 7f6cc9d9b640 1 -- 172.21.15.37:0/1805526195 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f6ccc15e570 msgr2=0x7f6ccc0c9d60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:29:39.486 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:39.482+0000 7f6cc9d9b640 1 --2- 172.21.15.37:0/1805526195 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f6ccc15e570 0x7f6ccc0c9d60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:29:39.487 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:39.482+0000 7f6cc9d9b640 1 -- 172.21.15.37:0/1805526195 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f6ccc06c540 msgr2=0x7f6ccc158bc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:29:39.487 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:39.482+0000 7f6cc9d9b640 1 --2- 172.21.15.37:0/1805526195 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f6ccc06c540 0x7f6ccc158bc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:29:39.487 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:39.482+0000 7f6cc9d9b640 1 -- 172.21.15.37:0/1805526195 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f6ccc1f4010 con 0x7f6ccc144490 2024-04-25T10:29:39.487 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:39.482+0000 7f6ccad9d640 1 --2- 172.21.15.37:0/1805526195 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f6ccc15e570 0x7f6ccc0c9d60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T10:29:39.487 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:39.482+0000 7f6cca59c640 1 --2- 172.21.15.37:0/1805526195 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f6ccc06c540 0x7f6ccc158bc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T10:29:39.487 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:39.483+0000 7f6cc9d9b640 1 --2- 172.21.15.37:0/1805526195 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f6ccc144490 0x7f6ccc0c9820 secure :-1 s=READY pgs=222 cs=0 l=1 rev1=1 crypto rx=0x7f6cb400d5f0 tx=0x7f6cb400dab0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:29:39.487 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:39.483+0000 7f6cc2ffd640 1 -- 172.21.15.37:0/1805526195 <== mon.0 v2:172.21.15.37:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6cb40125f0 con 0x7f6ccc144490 2024-04-25T10:29:39.487 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:39.483+0000 7f6cc2ffd640 1 -- 172.21.15.37:0/1805526195 <== mon.0 v2:172.21.15.37:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f6cb4019070 con 0x7f6ccc144490 2024-04-25T10:29:39.487 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:39.483+0000 7f6cd0f1f640 1 -- 172.21.15.37:0/1805526195 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f6ccc0caf60 con 0x7f6ccc144490 2024-04-25T10:29:39.487 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:39.483+0000 7f6cc2ffd640 1 -- 172.21.15.37:0/1805526195 <== mon.0 v2:172.21.15.37:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6cb40129d0 con 0x7f6ccc144490 2024-04-25T10:29:39.487 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:39.483+0000 7f6cd0f1f640 1 -- 172.21.15.37:0/1805526195 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f6ccc0cb520 con 0x7f6ccc144490 2024-04-25T10:29:39.488 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:39.484+0000 7f6cd0f1f640 1 -- 172.21.15.37:0/1805526195 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f6ccc06a140 con 0x7f6ccc144490 2024-04-25T10:29:39.488 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:39.484+0000 7f6cc2ffd640 1 -- 172.21.15.37:0/1805526195 <== mon.0 v2:172.21.15.37:3300/0 4 ==== mgrmap(e 5) ==== 96782+0+0 (secure 0 0 0) 0x7f6cb4012b70 con 0x7f6ccc144490 2024-04-25T10:29:39.488 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:39.485+0000 7f6cc2ffd640 1 --2- 172.21.15.37:0/1805526195 >> [v2:172.21.15.203:6832/2418205237,v1:172.21.15.203:6833/2418205237] conn(0x7f6cac0744b0 0x7f6cac0769c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:29:39.488 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:39.485+0000 7f6cc2ffd640 1 -- 172.21.15.37:0/1805526195 <== mon.0 v2:172.21.15.37:3300/0 5 ==== osd_map(23..23 src has 1..23) ==== 5709+0+0 (secure 0 0 0) 0x7f6cb4095540 con 0x7f6ccc144490 2024-04-25T10:29:39.488 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:39.485+0000 7f6cca59c640 1 --2- 172.21.15.37:0/1805526195 >> [v2:172.21.15.203:6832/2418205237,v1:172.21.15.203:6833/2418205237] conn(0x7f6cac0744b0 0x7f6cac0769c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:29:39.489 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:39.486+0000 7f6cca59c640 1 --2- 172.21.15.37:0/1805526195 >> [v2:172.21.15.203:6832/2418205237,v1:172.21.15.203:6833/2418205237] conn(0x7f6cac0744b0 0x7f6cac0769c0 secure :-1 s=READY pgs=127 cs=0 l=1 rev1=1 crypto rx=0x7f6cc4034000 tx=0x7f6cc402e000 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:29:39.491 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:39.489+0000 7f6cc2ffd640 1 -- 172.21.15.37:0/1805526195 <== mon.0 v2:172.21.15.37:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187246 (secure 0 0 0) 0x7f6cb405fdf0 con 0x7f6ccc144490 2024-04-25T10:29:39.674 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:39.672+0000 7f6cd0f1f640 1 -- 172.21.15.37:0/1805526195 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"} v 0) -- 0x7f6ccc0ca300 con 0x7f6ccc144490 2024-04-25T10:29:39.675 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:39.672+0000 7f6cc2ffd640 1 -- 172.21.15.37:0/1805526195 <== mon.0 v2:172.21.15.37:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]=0 v23) ==== 104+0+11 (secure 0 0 0) 0x7f6cb405f6f0 con 0x7f6ccc144490 2024-04-25T10:29:39.675 INFO:teuthology.orchestra.run.smithi037.stdout:pg_num: 32 2024-04-25T10:29:39.677 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:39.674+0000 7f6cd0f1f640 1 -- 172.21.15.37:0/1805526195 >> [v2:172.21.15.203:6832/2418205237,v1:172.21.15.203:6833/2418205237] conn(0x7f6cac0744b0 msgr2=0x7f6cac0769c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:29:39.677 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:39.674+0000 7f6cd0f1f640 1 --2- 172.21.15.37:0/1805526195 >> [v2:172.21.15.203:6832/2418205237,v1:172.21.15.203:6833/2418205237] conn(0x7f6cac0744b0 0x7f6cac0769c0 secure :-1 s=READY pgs=127 cs=0 l=1 rev1=1 crypto rx=0x7f6cc4034000 tx=0x7f6cc402e000 comp rx=0 tx=0).stop 2024-04-25T10:29:39.677 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:39.674+0000 7f6cd0f1f640 1 -- 172.21.15.37:0/1805526195 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f6ccc144490 msgr2=0x7f6ccc0c9820 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:29:39.677 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:39.674+0000 7f6cd0f1f640 1 --2- 172.21.15.37:0/1805526195 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f6ccc144490 0x7f6ccc0c9820 secure :-1 s=READY pgs=222 cs=0 l=1 rev1=1 crypto rx=0x7f6cb400d5f0 tx=0x7f6cb400dab0 comp rx=0 tx=0).stop 2024-04-25T10:29:39.677 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:39.674+0000 7f6cd0f1f640 1 -- 172.21.15.37:0/1805526195 shutdown_connections 2024-04-25T10:29:39.677 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:39.674+0000 7f6cd0f1f640 1 --2- 172.21.15.37:0/1805526195 >> [v2:172.21.15.203:6832/2418205237,v1:172.21.15.203:6833/2418205237] conn(0x7f6cac0744b0 0x7f6cac0769c0 unknown :-1 s=CLOSED pgs=127 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:29:39.677 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:39.674+0000 7f6cd0f1f640 1 --2- 172.21.15.37:0/1805526195 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f6ccc15e570 0x7f6ccc0c9d60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:29:39.677 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:39.674+0000 7f6cd0f1f640 1 --2- 172.21.15.37:0/1805526195 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f6ccc144490 0x7f6ccc0c9820 unknown :-1 s=CLOSED pgs=222 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:29:39.677 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:39.674+0000 7f6cd0f1f640 1 --2- 172.21.15.37:0/1805526195 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f6ccc06c540 0x7f6ccc158bc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:29:39.677 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:39.674+0000 7f6cd0f1f640 1 -- 172.21.15.37:0/1805526195 >> 172.21.15.37:0/1805526195 conn(0x7f6ccc0b4560 msgr2=0x7f6ccc15d980 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:29:39.678 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:39.674+0000 7f6cd0f1f640 1 -- 172.21.15.37:0/1805526195 shutdown_connections 2024-04-25T10:29:39.678 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:39.674+0000 7f6cd0f1f640 1 -- 172.21.15.37:0/1805526195 wait complete. 2024-04-25T10:29:39.688 DEBUG:teuthology.orchestra.run.smithi037:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data pg_num 2024-04-25T10:29:39.802 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:39.800+0000 7f447276c640 1 Processor -- start 2024-04-25T10:29:39.803 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:39.800+0000 7f447276c640 1 -- start start 2024-04-25T10:29:39.804 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:39.801+0000 7f447276c640 1 --2- >> v2:172.21.15.203:3300/0 conn(0x7f446c0d5160 0x7f446c0b4e80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:29:39.804 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:39.801+0000 7f447276c640 1 --2- >> v2:172.21.15.37:3300/0 conn(0x7f446c0d0990 0x7f446c0d2b20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:29:39.804 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:39.801+0000 7f447276c640 1 -- --> v1:172.21.15.203:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f446c0d1370 con 0x7f446c0d0da0 2024-04-25T10:29:39.804 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:39.801+0000 7f447276c640 1 -- --> v2:172.21.15.37:3300/0 -- mon_getmap magic: 0 -- 0x7f446c0d1650 con 0x7f446c0d0990 2024-04-25T10:29:39.804 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:39.801+0000 7f447276c640 1 -- --> v2:172.21.15.203:3300/0 -- mon_getmap magic: 0 -- 0x7f446c0d4680 con 0x7f446c0d5160 2024-04-25T10:29:39.804 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:39.802+0000 7f446b7fe640 1 --2- >> v2:172.21.15.37:3300/0 conn(0x7f446c0d0990 0x7f446c0d2b20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:29:39.804 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:39.802+0000 7f446b7fe640 1 --2- >> v2:172.21.15.37:3300/0 conn(0x7f446c0d0990 0x7f446c0d2b20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.37:3300/0 says I am v2:172.21.15.37:59272/0 (socket says 172.21.15.37:59272) 2024-04-25T10:29:39.804 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:39.802+0000 7f446b7fe640 1 -- 172.21.15.37:0/745988475 learned_addr learned my addr 172.21.15.37:0/745988475 (peer_addr_for_me v2:172.21.15.37:0/0) 2024-04-25T10:29:39.804 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:39.802+0000 7f446b7fe640 1 -- 172.21.15.37:0/745988475 >> v2:172.21.15.203:3300/0 conn(0x7f446c0d5160 msgr2=0x7f446c0b4e80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:29:39.805 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:39.802+0000 7f446bfff640 1 --2- 172.21.15.37:0/745988475 >> v2:172.21.15.203:3300/0 conn(0x7f446c0d5160 0x7f446c0b4e80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:29:39.805 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:39.802+0000 7f446b7fe640 1 --2- 172.21.15.37:0/745988475 >> v2:172.21.15.203:3300/0 conn(0x7f446c0d5160 0x7f446c0b4e80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:29:39.805 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:39.802+0000 7f446b7fe640 1 -- 172.21.15.37:0/745988475 >> v1:172.21.15.203:6789/0 conn(0x7f446c0d0da0 legacy=0x7f446c06db10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:29:39.805 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:39.802+0000 7f446b7fe640 1 -- 172.21.15.37:0/745988475 --> v2:172.21.15.37:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f446c0d4960 con 0x7f446c0d0990 2024-04-25T10:29:39.806 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:39.803+0000 7f446b7fe640 1 --2- 172.21.15.37:0/745988475 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f446c0d0990 0x7f446c0d2b20 secure :-1 s=READY pgs=223 cs=0 l=1 rev1=1 crypto rx=0x7f446c0d6b80 tx=0x7f445402a100 comp rx=0 tx=0).ready entity=mon.0 client_cookie=777617d276555d21 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:29:39.806 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:39.803+0000 7f446affd640 1 -- 172.21.15.37:0/745988475 <== mon.0 v2:172.21.15.37:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4454037070 con 0x7f446c0d0990 2024-04-25T10:29:39.806 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:39.803+0000 7f446affd640 1 -- 172.21.15.37:0/745988475 <== mon.0 v2:172.21.15.37:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f445402ae10 con 0x7f446c0d0990 2024-04-25T10:29:39.806 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:39.803+0000 7f446affd640 1 -- 172.21.15.37:0/745988475 <== mon.0 v2:172.21.15.37:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4454032660 con 0x7f446c0d0990 2024-04-25T10:29:39.806 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:39.803+0000 7f447276c640 1 -- 172.21.15.37:0/745988475 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f446c0d0990 msgr2=0x7f446c0d2b20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:29:39.806 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:39.803+0000 7f447276c640 1 --2- 172.21.15.37:0/745988475 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f446c0d0990 0x7f446c0d2b20 secure :-1 s=READY pgs=223 cs=0 l=1 rev1=1 crypto rx=0x7f446c0d6b80 tx=0x7f445402a100 comp rx=0 tx=0).stop 2024-04-25T10:29:39.806 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:39.804+0000 7f447276c640 1 -- 172.21.15.37:0/745988475 shutdown_connections 2024-04-25T10:29:39.807 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:39.804+0000 7f447276c640 1 --2- 172.21.15.37:0/745988475 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f446c0d0990 0x7f446c0d2b20 unknown :-1 s=CLOSED pgs=223 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:29:39.807 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:39.804+0000 7f447276c640 1 --2- 172.21.15.37:0/745988475 >> v2:172.21.15.203:3300/0 conn(0x7f446c0d5160 0x7f446c0b4e80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:29:39.807 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:39.804+0000 7f447276c640 1 -- 172.21.15.37:0/745988475 >> 172.21.15.37:0/745988475 conn(0x7f446c0b7a80 msgr2=0x7f446c0b7e90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:29:39.807 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:39.804+0000 7f447276c640 1 -- 172.21.15.37:0/745988475 shutdown_connections 2024-04-25T10:29:39.807 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:39.804+0000 7f447276c640 1 -- 172.21.15.37:0/745988475 wait complete. 2024-04-25T10:29:39.807 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:39.805+0000 7f447276c640 1 Processor -- start 2024-04-25T10:29:39.808 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:39.806+0000 7f447276c640 1 -- start start 2024-04-25T10:29:39.810 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:39.807+0000 7f447276c640 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f446c0d0990 0x7f446c158aa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:29:39.810 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:39.807+0000 7f447276c640 1 --2- >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f446c0d0da0 0x7f446c1f9da0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:29:39.810 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:39.807+0000 7f447276c640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f446c0d5160 0x7f446c154db0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:29:39.810 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:39.807+0000 7f447276c640 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 -- 0x7f446c164c40 con 0x7f446c0d5160 2024-04-25T10:29:39.810 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:39.807+0000 7f447276c640 1 -- --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_getmap magic: 0 -- 0x7f446c164ac0 con 0x7f446c0d0990 2024-04-25T10:29:39.811 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:39.807+0000 7f447276c640 1 -- --> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] -- mon_getmap magic: 0 -- 0x7f446c0d3060 con 0x7f446c0d0da0 2024-04-25T10:29:39.811 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:39.807+0000 7f4470d09640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f446c0d5160 0x7f446c154db0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:29:39.811 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:39.807+0000 7f4470d09640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f446c0d5160 0x7f446c154db0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.37:3300/0 says I am v2:172.21.15.37:59282/0 (socket says 172.21.15.37:59282) 2024-04-25T10:29:39.811 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:39.807+0000 7f4470d09640 1 -- 172.21.15.37:0/3114073070 learned_addr learned my addr 172.21.15.37:0/3114073070 (peer_addr_for_me v2:172.21.15.37:0/0) 2024-04-25T10:29:39.811 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:39.807+0000 7f446b7fe640 1 --2- 172.21.15.37:0/3114073070 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f446c0d0da0 0x7f446c1f9da0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:29:39.811 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:39.807+0000 7f446bfff640 1 --2- 172.21.15.37:0/3114073070 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f446c0d0990 0x7f446c158aa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:29:39.811 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:39.807+0000 7f4470d09640 1 -- 172.21.15.37:0/3114073070 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f446c0d0da0 msgr2=0x7f446c1f9da0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:29:39.811 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:39.807+0000 7f4470d09640 1 --2- 172.21.15.37:0/3114073070 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f446c0d0da0 0x7f446c1f9da0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:29:39.811 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:39.807+0000 7f4470d09640 1 -- 172.21.15.37:0/3114073070 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f446c0d0990 msgr2=0x7f446c158aa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:29:39.811 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:39.807+0000 7f4470d09640 1 --2- 172.21.15.37:0/3114073070 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f446c0d0990 0x7f446c158aa0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:29:39.811 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:39.807+0000 7f4470d09640 1 -- 172.21.15.37:0/3114073070 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f446c156b50 con 0x7f446c0d5160 2024-04-25T10:29:39.811 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:39.808+0000 7f4470d09640 1 --2- 172.21.15.37:0/3114073070 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f446c0d5160 0x7f446c154db0 secure :-1 s=READY pgs=224 cs=0 l=1 rev1=1 crypto rx=0x7f446400b990 tx=0x7f446400be50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:29:39.811 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:39.808+0000 7f4468ff9640 1 -- 172.21.15.37:0/3114073070 <== mon.0 v2:172.21.15.37:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4464019070 con 0x7f446c0d5160 2024-04-25T10:29:39.812 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:39.808+0000 7f4468ff9640 1 -- 172.21.15.37:0/3114073070 <== mon.0 v2:172.21.15.37:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f446400cba0 con 0x7f446c0d5160 2024-04-25T10:29:39.812 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:39.808+0000 7f4468ff9640 1 -- 172.21.15.37:0/3114073070 <== mon.0 v2:172.21.15.37:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4464014440 con 0x7f446c0d5160 2024-04-25T10:29:39.812 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:39.808+0000 7f447276c640 1 -- 172.21.15.37:0/3114073070 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f446c156590 con 0x7f446c0d5160 2024-04-25T10:29:39.812 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:39.808+0000 7f447276c640 1 -- 172.21.15.37:0/3114073070 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f446c1f69c0 con 0x7f446c0d5160 2024-04-25T10:29:39.813 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:39.809+0000 7f447276c640 1 -- 172.21.15.37:0/3114073070 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f446c1f67b0 con 0x7f446c0d5160 2024-04-25T10:29:39.813 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:39.810+0000 7f4468ff9640 1 -- 172.21.15.37:0/3114073070 <== mon.0 v2:172.21.15.37:3300/0 4 ==== mgrmap(e 5) ==== 96782+0+0 (secure 0 0 0) 0x7f446401c070 con 0x7f446c0d5160 2024-04-25T10:29:39.813 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:39.810+0000 7f4468ff9640 1 --2- 172.21.15.37:0/3114073070 >> [v2:172.21.15.203:6832/2418205237,v1:172.21.15.203:6833/2418205237] conn(0x7f444c0744b0 0x7f444c0769c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:29:39.813 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:39.810+0000 7f4468ff9640 1 -- 172.21.15.37:0/3114073070 <== mon.0 v2:172.21.15.37:3300/0 5 ==== osd_map(23..23 src has 1..23) ==== 5709+0+0 (secure 0 0 0) 0x7f4464095020 con 0x7f446c0d5160 2024-04-25T10:29:39.813 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:39.811+0000 7f446bfff640 1 --2- 172.21.15.37:0/3114073070 >> [v2:172.21.15.203:6832/2418205237,v1:172.21.15.203:6833/2418205237] conn(0x7f444c0744b0 0x7f444c0769c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:29:39.813 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:39.811+0000 7f446bfff640 1 --2- 172.21.15.37:0/3114073070 >> [v2:172.21.15.203:6832/2418205237,v1:172.21.15.203:6833/2418205237] conn(0x7f444c0744b0 0x7f444c0769c0 secure :-1 s=READY pgs=128 cs=0 l=1 rev1=1 crypto rx=0x7f445c006fd0 tx=0x7f445c008040 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:29:39.817 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:39.815+0000 7f4468ff9640 1 -- 172.21.15.37:0/3114073070 <== mon.0 v2:172.21.15.37:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187246 (secure 0 0 0) 0x7f446c156b50 con 0x7f446c0d5160 2024-04-25T10:29:40.003 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:40.000+0000 7f447276c640 1 -- 172.21.15.37:0/3114073070 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"} v 0) -- 0x7f446c0d1ba0 con 0x7f446c0d5160 2024-04-25T10:29:40.003 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:40.001+0000 7f4468ff9640 1 -- 172.21.15.37:0/3114073070 <== mon.0 v2:172.21.15.37:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]=0 v23) ==== 100+0+11 (secure 0 0 0) 0x7f446405f7b0 con 0x7f446c0d5160 2024-04-25T10:29:40.003 INFO:teuthology.orchestra.run.smithi037.stdout:pg_num: 64 2024-04-25T10:29:40.005 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:40.003+0000 7f447276c640 1 -- 172.21.15.37:0/3114073070 >> [v2:172.21.15.203:6832/2418205237,v1:172.21.15.203:6833/2418205237] conn(0x7f444c0744b0 msgr2=0x7f444c0769c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:29:40.005 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:40.003+0000 7f447276c640 1 --2- 172.21.15.37:0/3114073070 >> [v2:172.21.15.203:6832/2418205237,v1:172.21.15.203:6833/2418205237] conn(0x7f444c0744b0 0x7f444c0769c0 secure :-1 s=READY pgs=128 cs=0 l=1 rev1=1 crypto rx=0x7f445c006fd0 tx=0x7f445c008040 comp rx=0 tx=0).stop 2024-04-25T10:29:40.006 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:40.003+0000 7f447276c640 1 -- 172.21.15.37:0/3114073070 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f446c0d5160 msgr2=0x7f446c154db0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:29:40.006 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:40.003+0000 7f447276c640 1 --2- 172.21.15.37:0/3114073070 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f446c0d5160 0x7f446c154db0 secure :-1 s=READY pgs=224 cs=0 l=1 rev1=1 crypto rx=0x7f446400b990 tx=0x7f446400be50 comp rx=0 tx=0).stop 2024-04-25T10:29:40.006 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:40.003+0000 7f447276c640 1 -- 172.21.15.37:0/3114073070 shutdown_connections 2024-04-25T10:29:40.006 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:40.003+0000 7f447276c640 1 --2- 172.21.15.37:0/3114073070 >> [v2:172.21.15.203:6832/2418205237,v1:172.21.15.203:6833/2418205237] conn(0x7f444c0744b0 0x7f444c0769c0 unknown :-1 s=CLOSED pgs=128 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:29:40.006 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:40.003+0000 7f447276c640 1 --2- 172.21.15.37:0/3114073070 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f446c0d5160 0x7f446c154db0 unknown :-1 s=CLOSED pgs=224 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:29:40.006 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:40.003+0000 7f447276c640 1 --2- 172.21.15.37:0/3114073070 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f446c0d0da0 0x7f446c1f9da0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:29:40.006 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:40.003+0000 7f447276c640 1 --2- 172.21.15.37:0/3114073070 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f446c0d0990 0x7f446c158aa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:29:40.006 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:40.003+0000 7f447276c640 1 -- 172.21.15.37:0/3114073070 >> 172.21.15.37:0/3114073070 conn(0x7f446c0b7a80 msgr2=0x7f446c0ba0a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:29:40.006 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:40.003+0000 7f447276c640 1 -- 172.21.15.37:0/3114073070 shutdown_connections 2024-04-25T10:29:40.006 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:40.003+0000 7f447276c640 1 -- 172.21.15.37:0/3114073070 wait complete. 2024-04-25T10:29:40.019 DEBUG:teuthology.orchestra.run.smithi037:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-04-25T10:29:40.130 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:40.128+0000 7fd161427640 1 Processor -- start 2024-04-25T10:29:40.131 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:40.128+0000 7fd161427640 1 -- start start 2024-04-25T10:29:40.134 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:40.130+0000 7fd161427640 1 --2- >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7fd15c0d1ff0 0x7fd15c0d2400 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:29:40.134 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:40.130+0000 7fd161427640 1 --2- >> v2:172.21.15.203:3300/0 conn(0x7fd15c0ce750 0x7fd15c0ceb60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:29:40.134 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:40.130+0000 7fd161427640 1 -- --> v1:172.21.15.37:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fd15c0b8d10 con 0x7fd15c0b3c80 2024-04-25T10:29:40.134 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:40.130+0000 7fd161427640 1 -- --> v2:172.21.15.203:3300/0 -- mon_getmap magic: 0 -- 0x7fd15c08ceb0 con 0x7fd15c0ce750 2024-04-25T10:29:40.134 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:40.130+0000 7fd161427640 1 -- --> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] -- mon_getmap magic: 0 -- 0x7fd15c0b9300 con 0x7fd15c0d1ff0 2024-04-25T10:29:40.134 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:40.130+0000 7fd15affd640 1 --1- >> v1:172.21.15.37:6789/0 conn(0x7fd15c0b3c80 0x7fd15c0b4090 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.37:6789/0 says I am v1:172.21.15.37:49788/0 (socket says 172.21.15.37:49788) 2024-04-25T10:29:40.134 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:40.130+0000 7fd15affd640 1 -- 172.21.15.37:0/1205958848 learned_addr learned my addr 172.21.15.37:0/1205958848 (peer_addr_for_me v1:172.21.15.37:0/0) 2024-04-25T10:29:40.134 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:40.130+0000 7fd15b7fe640 1 --2- 172.21.15.37:0/1205958848 >> v2:172.21.15.203:3300/0 conn(0x7fd15c0ce750 0x7fd15c0ceb60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:29:40.134 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:40.130+0000 7fd159ffb640 1 -- 172.21.15.37:0/1205958848 <== mon.0 v1:172.21.15.37:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 2424443991 0 0) 0x7fd15c0b8d10 con 0x7fd15c0b3c80 2024-04-25T10:29:40.135 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:40.130+0000 7fd159ffb640 1 -- 172.21.15.37:0/1205958848 --> v1:172.21.15.37:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7fd148003520 con 0x7fd15c0b3c80 2024-04-25T10:29:40.135 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:40.130+0000 7fd159ffb640 1 -- 172.21.15.37:0/1205958848 <== mon.0 v1:172.21.15.37:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 3064456075 0 0) 0x7fd148003520 con 0x7fd15c0b3c80 2024-04-25T10:29:40.135 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:40.130+0000 7fd159ffb640 1 -- 172.21.15.37:0/1205958848 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7fd15c0d1ff0 msgr2=0x7fd15c0d2400 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:29:40.135 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:40.130+0000 7fd159ffb640 1 --2- 172.21.15.37:0/1205958848 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7fd15c0d1ff0 0x7fd15c0d2400 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:29:40.135 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:40.130+0000 7fd159ffb640 1 -- 172.21.15.37:0/1205958848 >> v2:172.21.15.203:3300/0 conn(0x7fd15c0ce750 msgr2=0x7fd15c0ceb60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:29:40.135 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:40.130+0000 7fd159ffb640 1 --2- 172.21.15.37:0/1205958848 >> v2:172.21.15.203:3300/0 conn(0x7fd15c0ce750 0x7fd15c0ceb60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:29:40.135 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:40.130+0000 7fd159ffb640 1 -- 172.21.15.37:0/1205958848 --> v1:172.21.15.37:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fd15c0b8a30 con 0x7fd15c0b3c80 2024-04-25T10:29:40.135 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:40.130+0000 7fd159ffb640 1 -- 172.21.15.37:0/1205958848 <== mon.0 v1:172.21.15.37:6789/0 3 ==== mon_map magic: 0 ==== 413+0+0 (unknown 1577794600 0 0) 0x7fd154001670 con 0x7fd15c0b3c80 2024-04-25T10:29:40.135 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:40.130+0000 7fd159ffb640 1 -- 172.21.15.37:0/1205958848 >> v1:172.21.15.37:6789/0 conn(0x7fd15c0b3c80 legacy=0x7fd15c0b4090 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:29:40.135 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:40.130+0000 7fd159ffb640 1 --2- 172.21.15.37:0/1205958848 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fd1480039b0 0x7fd148003e20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:29:40.135 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:40.130+0000 7fd159ffb640 1 --2- 172.21.15.37:0/1205958848 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7fd15c0d1ff0 0x7fd148004800 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:29:40.135 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:40.130+0000 7fd159ffb640 1 --2- 172.21.15.37:0/1205958848 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fd148004d40 0x7fd148005150 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:29:40.135 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:40.130+0000 7fd159ffb640 1 -- 172.21.15.37:0/1205958848 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 -- 0x7fd15c0b9300 con 0x7fd1480039b0 2024-04-25T10:29:40.135 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:40.130+0000 7fd159ffb640 1 -- 172.21.15.37:0/1205958848 --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_getmap magic: 0 -- 0x7fd14800bb50 con 0x7fd148004d40 2024-04-25T10:29:40.136 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:40.130+0000 7fd159ffb640 1 -- 172.21.15.37:0/1205958848 --> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] -- mon_getmap magic: 0 -- 0x7fd14800bd50 con 0x7fd15c0d1ff0 2024-04-25T10:29:40.136 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:40.131+0000 7fd15affd640 1 --2- 172.21.15.37:0/1205958848 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fd1480039b0 0x7fd148003e20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:29:40.136 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:40.131+0000 7fd15affd640 1 -- 172.21.15.37:0/1205958848 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7fd15c0d1ff0 msgr2=0x7fd148004800 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:29:40.136 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:40.131+0000 7fd15affd640 1 --2- 172.21.15.37:0/1205958848 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7fd15c0d1ff0 0x7fd148004800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:29:40.136 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:40.131+0000 7fd15affd640 1 -- 172.21.15.37:0/1205958848 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fd148004d40 msgr2=0x7fd148005150 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:29:40.136 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:40.131+0000 7fd15affd640 1 --2- 172.21.15.37:0/1205958848 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fd148004d40 0x7fd148005150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:29:40.136 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:40.131+0000 7fd15affd640 1 -- 172.21.15.37:0/1205958848 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fd148003520 con 0x7fd1480039b0 2024-04-25T10:29:40.136 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:40.132+0000 7fd15affd640 1 --2- 172.21.15.37:0/1205958848 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fd1480039b0 0x7fd148003e20 secure :-1 s=READY pgs=226 cs=0 l=1 rev1=1 crypto rx=0x7fd15400c210 tx=0x7fd148005830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=da8bf5234ae9776d server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:29:40.136 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:40.132+0000 7fd159ffb640 1 -- 172.21.15.37:0/1205958848 <== mon.0 v2:172.21.15.37:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd154007d50 con 0x7fd1480039b0 2024-04-25T10:29:40.136 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:40.132+0000 7fd159ffb640 1 -- 172.21.15.37:0/1205958848 <== mon.0 v2:172.21.15.37:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fd154005ad0 con 0x7fd1480039b0 2024-04-25T10:29:40.136 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:40.132+0000 7fd159ffb640 1 -- 172.21.15.37:0/1205958848 <== mon.0 v2:172.21.15.37:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd154007070 con 0x7fd1480039b0 2024-04-25T10:29:40.139 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:40.132+0000 7fd161427640 1 -- 172.21.15.37:0/1205958848 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fd1480039b0 msgr2=0x7fd148003e20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:29:40.139 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:40.132+0000 7fd161427640 1 --2- 172.21.15.37:0/1205958848 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fd1480039b0 0x7fd148003e20 secure :-1 s=READY pgs=226 cs=0 l=1 rev1=1 crypto rx=0x7fd15400c210 tx=0x7fd148005830 comp rx=0 tx=0).stop 2024-04-25T10:29:40.139 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:40.132+0000 7fd15b7fe640 1 -- 172.21.15.37:0/1205958848 reap_dead start 2024-04-25T10:29:40.139 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:40.132+0000 7fd161427640 1 -- 172.21.15.37:0/1205958848 shutdown_connections 2024-04-25T10:29:40.139 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:40.132+0000 7fd161427640 1 -- 172.21.15.37:0/1205958848 >> 172.21.15.37:0/1205958848 conn(0x7fd15c0d0390 msgr2=0x7fd15c0d07a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:29:40.139 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:40.132+0000 7fd161427640 1 -- 172.21.15.37:0/1205958848 shutdown_connections 2024-04-25T10:29:40.139 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:40.132+0000 7fd161427640 1 -- 172.21.15.37:0/1205958848 wait complete. 2024-04-25T10:29:40.139 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:40.134+0000 7fd161427640 1 Processor -- start 2024-04-25T10:29:40.140 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:40.134+0000 7fd161427640 1 -- start start 2024-04-25T10:29:40.140 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:40.136+0000 7fd161427640 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fd1480039b0 0x7fd15c0cf500 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:29:40.140 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:40.136+0000 7fd161427640 1 --2- >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7fd15c0cfa40 0x7fd15c1e5e90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:29:40.140 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:40.136+0000 7fd161427640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fd15c09d610 0x7fd15c1e63d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:29:40.140 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:40.136+0000 7fd161427640 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 -- 0x7fd15c0d18c0 con 0x7fd15c09d610 2024-04-25T10:29:40.140 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:40.136+0000 7fd161427640 1 -- --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_getmap magic: 0 -- 0x7fd15c09ce90 con 0x7fd1480039b0 2024-04-25T10:29:40.140 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:40.136+0000 7fd161427640 1 -- --> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] -- mon_getmap magic: 0 -- 0x7fd15c0d1a40 con 0x7fd15c0cfa40 2024-04-25T10:29:40.140 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:40.136+0000 7fd15b7fe640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fd15c09d610 0x7fd15c1e63d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:29:40.140 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:40.136+0000 7fd15b7fe640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fd15c09d610 0x7fd15c1e63d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.37:3300/0 says I am v2:172.21.15.37:59296/0 (socket says 172.21.15.37:59296) 2024-04-25T10:29:40.140 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:40.136+0000 7fd15b7fe640 1 -- 172.21.15.37:0/1305141460 learned_addr learned my addr 172.21.15.37:0/1305141460 (peer_addr_for_me v2:172.21.15.37:0/0) 2024-04-25T10:29:40.140 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:40.136+0000 7fd15a7fc640 1 --2- 172.21.15.37:0/1305141460 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7fd15c0cfa40 0x7fd15c1e5e90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:29:40.140 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:40.136+0000 7fd15affd640 1 --2- 172.21.15.37:0/1305141460 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fd1480039b0 0x7fd15c0cf500 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:29:40.141 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:40.136+0000 7fd15b7fe640 1 -- 172.21.15.37:0/1305141460 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7fd15c0cfa40 msgr2=0x7fd15c1e5e90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:29:40.141 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:40.136+0000 7fd15b7fe640 1 --2- 172.21.15.37:0/1305141460 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7fd15c0cfa40 0x7fd15c1e5e90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:29:40.141 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:40.136+0000 7fd15b7fe640 1 -- 172.21.15.37:0/1305141460 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fd1480039b0 msgr2=0x7fd15c0cf500 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:29:40.141 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:40.136+0000 7fd15b7fe640 1 --2- 172.21.15.37:0/1305141460 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fd1480039b0 0x7fd15c0cf500 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:29:40.141 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:40.136+0000 7fd15b7fe640 1 -- 172.21.15.37:0/1305141460 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fd15c09fe20 con 0x7fd15c09d610 2024-04-25T10:29:40.141 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:40.136+0000 7fd15affd640 1 --2- 172.21.15.37:0/1305141460 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fd1480039b0 0x7fd15c0cf500 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T10:29:40.141 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:40.136+0000 7fd15b7fe640 1 --2- 172.21.15.37:0/1305141460 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fd15c09d610 0x7fd15c1e63d0 secure :-1 s=READY pgs=227 cs=0 l=1 rev1=1 crypto rx=0x7fd15c074c40 tx=0x7fd14c00b480 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:29:40.141 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:40.136+0000 7fd1537fe640 1 -- 172.21.15.37:0/1305141460 <== mon.0 v2:172.21.15.37:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd14c00bae0 con 0x7fd15c09d610 2024-04-25T10:29:40.141 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:40.137+0000 7fd1537fe640 1 -- 172.21.15.37:0/1305141460 <== mon.0 v2:172.21.15.37:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fd14c01a070 con 0x7fd15c09d610 2024-04-25T10:29:40.141 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:40.137+0000 7fd161427640 1 -- 172.21.15.37:0/1305141460 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fd15c0a0bc0 con 0x7fd15c09d610 2024-04-25T10:29:40.141 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:40.137+0000 7fd1537fe640 1 -- 172.21.15.37:0/1305141460 <== mon.0 v2:172.21.15.37:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd14c014400 con 0x7fd15c09d610 2024-04-25T10:29:40.141 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:40.137+0000 7fd161427640 1 -- 172.21.15.37:0/1305141460 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fd15c0a00e0 con 0x7fd15c09d610 2024-04-25T10:29:40.142 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:40.138+0000 7fd161427640 1 -- 172.21.15.37:0/1305141460 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fd15c06a140 con 0x7fd15c09d610 2024-04-25T10:29:40.142 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:40.138+0000 7fd1537fe640 1 -- 172.21.15.37:0/1305141460 <== mon.0 v2:172.21.15.37:3300/0 4 ==== mgrmap(e 5) ==== 96782+0+0 (secure 0 0 0) 0x7fd14c005d10 con 0x7fd15c09d610 2024-04-25T10:29:40.142 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:40.139+0000 7fd1537fe640 1 --2- 172.21.15.37:0/1305141460 >> [v2:172.21.15.203:6832/2418205237,v1:172.21.15.203:6833/2418205237] conn(0x7fd13c0744b0 0x7fd13c0769c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:29:40.142 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:40.139+0000 7fd1537fe640 1 -- 172.21.15.37:0/1305141460 <== mon.0 v2:172.21.15.37:3300/0 5 ==== osd_map(23..23 src has 1..23) ==== 5709+0+0 (secure 0 0 0) 0x7fd14c0951a0 con 0x7fd15c09d610 2024-04-25T10:29:40.142 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:40.140+0000 7fd15affd640 1 --2- 172.21.15.37:0/1305141460 >> [v2:172.21.15.203:6832/2418205237,v1:172.21.15.203:6833/2418205237] conn(0x7fd13c0744b0 0x7fd13c0769c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:29:40.145 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:40.144+0000 7fd1537fe640 1 -- 172.21.15.37:0/1305141460 <== mon.0 v2:172.21.15.37:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187246 (secure 0 0 0) 0x7fd14c05f930 con 0x7fd15c09d610 2024-04-25T10:29:40.147 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:40.145+0000 7fd15affd640 1 --2- 172.21.15.37:0/1305141460 >> [v2:172.21.15.203:6832/2418205237,v1:172.21.15.203:6833/2418205237] conn(0x7fd13c0744b0 0x7fd13c0769c0 secure :-1 s=READY pgs=129 cs=0 l=1 rev1=1 crypto rx=0x7fd154033000 tx=0x7fd154009030 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:29:40.334 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:40.332+0000 7fd161427640 1 -- 172.21.15.37:0/1305141460 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7fd15c09da80 con 0x7fd15c09d610 2024-04-25T10:29:40.335 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:40.333+0000 7fd1537fe640 1 -- 172.21.15.37:0/1305141460 <== mon.0 v2:172.21.15.37:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v23) ==== 74+0+16621 (secure 0 0 0) 0x7fd14c05f230 con 0x7fd15c09d610 2024-04-25T10:29:40.335 INFO:teuthology.orchestra.run.smithi037.stdout: 2024-04-25T10:29:40.335 INFO:teuthology.orchestra.run.smithi037.stdout:{"epoch":23,"fsid":"f4acda28-be34-492a-b286-994afd36f61d","created":"2024-04-25T10:28:43.985718+0000","modified":"2024-04-25T10:29:10.016188+0000","last_up_change":"2024-04-25T10:28:54.900654+0000","last_in_change":"2024-04-25T10:28:46.216973+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":3,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-04-25T10:28:56.202153+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"19","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2024-04-25T10:29:06.507244+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"22","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":2.25,"score_stable":2.25,"optimal_score":1,"raw_score_acting":2.25,"raw_score_stable":2.25,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2024-04-25T10:29:07.347844+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"23","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"a2f79028-b332-4166-9ebb-a7024a37ea22","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":21,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.37:6816","nonce":2746074720},{"type":"v1","addr":"172.21.15.37:6817","nonce":2746074720}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.37:6818","nonce":2746074720},{"type":"v1","addr":"172.21.15.37:6819","nonce":2746074720}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.37:6822","nonce":2746074720},{"type":"v1","addr":"172.21.15.37:6823","nonce":2746074720}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.37:6820","nonce":2746074720},{"type":"v1","addr":"172.21.15.37:6821","nonce":2746074720}]},"public_addr":"172.21.15.37:6817/2746074720","cluster_addr":"172.21.15.37:6819/2746074720","heartbeat_back_addr":"172.21.15.37:6823/2746074720","heartbeat_front_addr":"172.21.15.37:6821/2746074720","state":["exists","up"]},{"osd":1,"uuid":"973159de-c2ea-4083-aecc-69373d1c1132","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":21,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.37:6800","nonce":1690341662},{"type":"v1","addr":"172.21.15.37:6801","nonce":1690341662}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.37:6802","nonce":1690341662},{"type":"v1","addr":"172.21.15.37:6803","nonce":1690341662}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.37:6806","nonce":1690341662},{"type":"v1","addr":"172.21.15.37:6807","nonce":1690341662}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.37:6804","nonce":1690341662},{"type":"v1","addr":"172.21.15.37:6805","nonce":1690341662}]},"public_addr":"172.21.15.37:6801/1690341662","cluster_addr":"172.21.15.37:6803/1690341662","heartbeat_back_addr":"172.21.15.37:6807/1690341662","heartbeat_front_addr":"172.21.15.37:6805/1690341662","state":["exists","up"]},{"osd":2,"uuid":"4ae7a0e9-cb3f-484c-a75f-f3766b721c47","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":21,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.37:6808","nonce":2754669649},{"type":"v1","addr":"172.21.15.37:6809","nonce":2754669649}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.37:6810","nonce":2754669649},{"type":"v1","addr":"172.21.15.37:6811","nonce":2754669649}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.37:6814","nonce":2754669649},{"type":"v1","addr":"172.21.15.37:6815","nonce":2754669649}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.37:6812","nonce":2754669649},{"type":"v1","addr":"172.21.15.37:6813","nonce":2754669649}]},"public_addr":"172.21.15.37:6809/2754669649","cluster_addr":"172.21.15.37:6811/2754669649","heartbeat_back_addr":"172.21.15.37:6815/2754669649","heartbeat_front_addr":"172.21.15.37:6813/2754669649","state":["exists","up"]},{"osd":3,"uuid":"1ba4a746-1608-47a7-ace2-960a5cfae934","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":21,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.37:6824","nonce":1815290618},{"type":"v1","addr":"172.21.15.37:6825","nonce":1815290618}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.37:6826","nonce":1815290618},{"type":"v1","addr":"172.21.15.37:6827","nonce":1815290618}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.37:6830","nonce":1815290618},{"type":"v1","addr":"172.21.15.37:6831","nonce":1815290618}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.37:6828","nonce":1815290618},{"type":"v1","addr":"172.21.15.37:6829","nonce":1815290618}]},"public_addr":"172.21.15.37:6825/1815290618","cluster_addr":"172.21.15.37:6827/1815290618","heartbeat_back_addr":"172.21.15.37:6831/1815290618","heartbeat_front_addr":"172.21.15.37:6829/1815290618","state":["exists","up"]},{"osd":4,"uuid":"d4f59f5d-0413-4b79-87ee-bb212c1fcf62","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":21,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6800","nonce":4051532063},{"type":"v1","addr":"172.21.15.203:6801","nonce":4051532063}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6802","nonce":4051532063},{"type":"v1","addr":"172.21.15.203:6803","nonce":4051532063}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6806","nonce":4051532063},{"type":"v1","addr":"172.21.15.203:6807","nonce":4051532063}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6804","nonce":4051532063},{"type":"v1","addr":"172.21.15.203:6805","nonce":4051532063}]},"public_addr":"172.21.15.203:6801/4051532063","cluster_addr":"172.21.15.203:6803/4051532063","heartbeat_back_addr":"172.21.15.203:6807/4051532063","heartbeat_front_addr":"172.21.15.203:6805/4051532063","state":["exists","up"]},{"osd":5,"uuid":"69710a0e-41d8-4e7f-9a10-17ce540b67be","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":21,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6824","nonce":1203822890},{"type":"v1","addr":"172.21.15.203:6825","nonce":1203822890}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6826","nonce":1203822890},{"type":"v1","addr":"172.21.15.203:6827","nonce":1203822890}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6830","nonce":1203822890},{"type":"v1","addr":"172.21.15.203:6831","nonce":1203822890}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6828","nonce":1203822890},{"type":"v1","addr":"172.21.15.203:6829","nonce":1203822890}]},"public_addr":"172.21.15.203:6825/1203822890","cluster_addr":"172.21.15.203:6827/1203822890","heartbeat_back_addr":"172.21.15.203:6831/1203822890","heartbeat_front_addr":"172.21.15.203:6829/1203822890","state":["exists","up"]},{"osd":6,"uuid":"f29dd047-3b48-473d-a235-6642a6293f20","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":21,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6810","nonce":2309615014},{"type":"v1","addr":"172.21.15.203:6812","nonce":2309615014}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6814","nonce":2309615014},{"type":"v1","addr":"172.21.15.203:6816","nonce":2309615014}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6822","nonce":2309615014},{"type":"v1","addr":"172.21.15.203:6823","nonce":2309615014}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6818","nonce":2309615014},{"type":"v1","addr":"172.21.15.203:6820","nonce":2309615014}]},"public_addr":"172.21.15.203:6812/2309615014","cluster_addr":"172.21.15.203:6816/2309615014","heartbeat_back_addr":"172.21.15.203:6823/2309615014","heartbeat_front_addr":"172.21.15.203:6820/2309615014","state":["exists","up"]},{"osd":7,"uuid":"6e691644-378b-4fdc-8e86-fb5f56e31f29","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":21,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6808","nonce":3688099014},{"type":"v1","addr":"172.21.15.203:6809","nonce":3688099014}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6811","nonce":3688099014},{"type":"v1","addr":"172.21.15.203:6813","nonce":3688099014}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6819","nonce":3688099014},{"type":"v1","addr":"172.21.15.203:6821","nonce":3688099014}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6815","nonce":3688099014},{"type":"v1","addr":"172.21.15.203:6817","nonce":3688099014}]},"public_addr":"172.21.15.203:6809/3688099014","cluster_addr":"172.21.15.203:6813/3688099014","heartbeat_back_addr":"172.21.15.203:6821/3688099014","heartbeat_front_addr":"172.21.15.203:6817/3688099014","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T10:28:51.419141+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T10:28:51.366448+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T10:28:51.348276+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T10:28:51.380519+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T10:28:51.105812+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T10:28:51.384496+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T10:28:51.388755+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T10:28:51.366958+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-04-25T10:29:40.338 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:40.335+0000 7fd161427640 1 -- 172.21.15.37:0/1305141460 >> [v2:172.21.15.203:6832/2418205237,v1:172.21.15.203:6833/2418205237] conn(0x7fd13c0744b0 msgr2=0x7fd13c0769c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:29:40.338 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:40.335+0000 7fd161427640 1 --2- 172.21.15.37:0/1305141460 >> [v2:172.21.15.203:6832/2418205237,v1:172.21.15.203:6833/2418205237] conn(0x7fd13c0744b0 0x7fd13c0769c0 secure :-1 s=READY pgs=129 cs=0 l=1 rev1=1 crypto rx=0x7fd154033000 tx=0x7fd154009030 comp rx=0 tx=0).stop 2024-04-25T10:29:40.338 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:40.335+0000 7fd161427640 1 -- 172.21.15.37:0/1305141460 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fd15c09d610 msgr2=0x7fd15c1e63d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:29:40.338 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:40.335+0000 7fd161427640 1 --2- 172.21.15.37:0/1305141460 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fd15c09d610 0x7fd15c1e63d0 secure :-1 s=READY pgs=227 cs=0 l=1 rev1=1 crypto rx=0x7fd15c074c40 tx=0x7fd14c00b480 comp rx=0 tx=0).stop 2024-04-25T10:29:40.338 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:40.335+0000 7fd161427640 1 -- 172.21.15.37:0/1305141460 shutdown_connections 2024-04-25T10:29:40.338 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:40.335+0000 7fd161427640 1 --2- 172.21.15.37:0/1305141460 >> [v2:172.21.15.203:6832/2418205237,v1:172.21.15.203:6833/2418205237] conn(0x7fd13c0744b0 0x7fd13c0769c0 unknown :-1 s=CLOSED pgs=129 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:29:40.338 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:40.335+0000 7fd161427640 1 --2- 172.21.15.37:0/1305141460 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fd15c09d610 0x7fd15c1e63d0 unknown :-1 s=CLOSED pgs=227 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:29:40.338 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:40.335+0000 7fd161427640 1 --2- 172.21.15.37:0/1305141460 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7fd15c0cfa40 0x7fd15c1e5e90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:29:40.338 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:40.335+0000 7fd161427640 1 --2- 172.21.15.37:0/1305141460 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fd1480039b0 0x7fd15c0cf500 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:29:40.338 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:40.335+0000 7fd161427640 1 -- 172.21.15.37:0/1305141460 >> 172.21.15.37:0/1305141460 conn(0x7fd15c0d0390 msgr2=0x7fd15c0712f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:29:40.338 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:40.335+0000 7fd161427640 1 -- 172.21.15.37:0/1305141460 shutdown_connections 2024-04-25T10:29:40.339 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:40.335+0000 7fd161427640 1 -- 172.21.15.37:0/1305141460 wait complete. 2024-04-25T10:29:40.349 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2024-04-25T10:28:56.202153+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '19', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 8, 'score_stable': 8, 'optimal_score': 0.25, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 2, 'pool_name': 'cephfs_metadata', 'create_time': '2024-04-25T10:29:06.507244+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '22', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_autoscale_bias': 4, 'pg_num_min': 16, 'recovery_priority': 5}, 'application_metadata': {'cephfs': {'metadata': 'cephfs'}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 2.25, 'score_stable': 2.25, 'optimal_score': 1, 'raw_score_acting': 2.25, 'raw_score_stable': 2.25, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 3, 'pool_name': 'cephfs_data', 'create_time': '2024-04-25T10:29:07.347844+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 64, 'pg_placement_num': 64, 'pg_placement_num_target': 64, 'pg_num_target': 64, 'pg_num_pending': 64, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '23', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.8999999761581421}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 1.5, 'score_stable': 1.5, 'optimal_score': 1, 'raw_score_acting': 1.5, 'raw_score_stable': 1.5, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}] 2024-04-25T10:29:40.349 DEBUG:teuthology.orchestra.run.smithi037:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2024-04-25T10:29:40.463 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:40.461+0000 7f79be72e640 1 Processor -- start 2024-04-25T10:29:40.463 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:40.461+0000 7f79be72e640 1 -- start start 2024-04-25T10:29:40.465 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:40.462+0000 7f79be72e640 1 --2- >> v2:172.21.15.37:3300/0 conn(0x7f79b809b700 0x7f79b809bb10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:29:40.465 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:40.462+0000 7f79be72e640 1 -- --> v1:172.21.15.37:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f79b809c050 con 0x7f79b80b2dd0 2024-04-25T10:29:40.465 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:40.462+0000 7f79be72e640 1 -- --> v1:172.21.15.203:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f79b81612e0 con 0x7f79b815a940 2024-04-25T10:29:40.465 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:40.462+0000 7f79be72e640 1 -- --> v2:172.21.15.37:3300/0 -- mon_getmap magic: 0 -- 0x7f79b80b2940 con 0x7f79b809b700 2024-04-25T10:29:40.465 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:40.462+0000 7f79b77fe640 1 --2- >> v2:172.21.15.37:3300/0 conn(0x7f79b809b700 0x7f79b809bb10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:29:40.465 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:40.462+0000 7f79bcccb640 1 --1- >> v1:172.21.15.37:6789/0 conn(0x7f79b80b2dd0 0x7f79b80b31e0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.37:6789/0 says I am v1:172.21.15.37:49798/0 (socket says 172.21.15.37:49798) 2024-04-25T10:29:40.465 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:40.462+0000 7f79bcccb640 1 -- 172.21.15.37:0/2934131219 learned_addr learned my addr 172.21.15.37:0/2934131219 (peer_addr_for_me v1:172.21.15.37:0/0) 2024-04-25T10:29:40.465 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:40.463+0000 7f79b77fe640 1 -- 172.21.15.37:0/2934131219 >> v1:172.21.15.203:6789/0 conn(0x7f79b815a940 legacy=0x7f79b81607b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:29:40.465 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:40.463+0000 7f79b77fe640 1 -- 172.21.15.37:0/2934131219 >> v1:172.21.15.37:6789/0 conn(0x7f79b80b2dd0 legacy=0x7f79b80b31e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:29:40.466 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:40.463+0000 7f79b77fe640 1 -- 172.21.15.37:0/2934131219 --> v2:172.21.15.37:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f79b81615c0 con 0x7f79b809b700 2024-04-25T10:29:40.466 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:40.463+0000 7f79b6ffd640 1 -- 172.21.15.37:0/2934131219 <== mon.0 v1:172.21.15.37:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 1332670919 0 0) 0x7f79b0001490 con 0x7f79b80b2dd0 2024-04-25T10:29:40.467 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:40.463+0000 7f79b77fe640 1 --2- 172.21.15.37:0/2934131219 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f79b809b700 0x7f79b809bb10 secure :-1 s=READY pgs=229 cs=0 l=1 rev1=1 crypto rx=0x7f79ac0097a0 tx=0x7f79ac02a1a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=2d59031d28536f32 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:29:40.467 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:40.464+0000 7f79b6ffd640 1 -- 172.21.15.37:0/2934131219 <== mon.0 v2:172.21.15.37:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f79ac037070 con 0x7f79b809b700 2024-04-25T10:29:40.467 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:40.464+0000 7f79b6ffd640 1 -- 172.21.15.37:0/2934131219 <== mon.0 v2:172.21.15.37:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f79ac032440 con 0x7f79b809b700 2024-04-25T10:29:40.468 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:40.464+0000 7f79b6ffd640 1 -- 172.21.15.37:0/2934131219 <== mon.0 v2:172.21.15.37:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f79ac032810 con 0x7f79b809b700 2024-04-25T10:29:40.468 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:40.464+0000 7f79be72e640 1 -- 172.21.15.37:0/2934131219 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f79b809b700 msgr2=0x7f79b809bb10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:29:40.468 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:40.464+0000 7f79be72e640 1 --2- 172.21.15.37:0/2934131219 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f79b809b700 0x7f79b809bb10 secure :-1 s=READY pgs=229 cs=0 l=1 rev1=1 crypto rx=0x7f79ac0097a0 tx=0x7f79ac02a1a0 comp rx=0 tx=0).stop 2024-04-25T10:29:40.468 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:40.464+0000 7f79be72e640 1 -- 172.21.15.37:0/2934131219 shutdown_connections 2024-04-25T10:29:40.468 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:40.464+0000 7f79be72e640 1 --2- 172.21.15.37:0/2934131219 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f79b809b700 0x7f79b809bb10 unknown :-1 s=CLOSED pgs=229 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:29:40.468 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:40.464+0000 7f79be72e640 1 -- 172.21.15.37:0/2934131219 >> 172.21.15.37:0/2934131219 conn(0x7f79b80b7600 msgr2=0x7f79b80b7c30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:29:40.468 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:40.464+0000 7f79be72e640 1 -- 172.21.15.37:0/2934131219 shutdown_connections 2024-04-25T10:29:40.468 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:40.464+0000 7f79be72e640 1 -- 172.21.15.37:0/2934131219 wait complete. 2024-04-25T10:29:40.469 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:40.466+0000 7f79be72e640 1 Processor -- start 2024-04-25T10:29:40.469 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:40.466+0000 7f79be72e640 1 -- start start 2024-04-25T10:29:40.471 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:40.467+0000 7f79be72e640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f79b809b700 0x7f79b81e6660 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:29:40.471 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:40.468+0000 7f79be72e640 1 --2- >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f79b80b2dd0 0x7f79b81534f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:29:40.472 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:40.468+0000 7f79be72e640 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f79b815a940 0x7f79b81f6870 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:29:40.472 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:40.468+0000 7f79be72e640 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 -- 0x7f79b806ede0 con 0x7f79b809b700 2024-04-25T10:29:40.472 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:40.468+0000 7f79be72e640 1 -- --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_getmap magic: 0 -- 0x7f79b806ec60 con 0x7f79b815a940 2024-04-25T10:29:40.472 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:40.468+0000 7f79be72e640 1 -- --> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] -- mon_getmap magic: 0 -- 0x7f79b8160ec0 con 0x7f79b80b2dd0 2024-04-25T10:29:40.472 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:40.468+0000 7f79b7fff640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f79b809b700 0x7f79b81e6660 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:29:40.472 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:40.468+0000 7f79b7fff640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f79b809b700 0x7f79b81e6660 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.37:3300/0 says I am v2:172.21.15.37:59316/0 (socket says 172.21.15.37:59316) 2024-04-25T10:29:40.472 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:40.468+0000 7f79b7fff640 1 -- 172.21.15.37:0/1205530040 learned_addr learned my addr 172.21.15.37:0/1205530040 (peer_addr_for_me v2:172.21.15.37:0/0) 2024-04-25T10:29:40.472 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:40.468+0000 7f79b77fe640 1 --2- 172.21.15.37:0/1205530040 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f79b80b2dd0 0x7f79b81534f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:29:40.472 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:40.468+0000 7f79bcccb640 1 --2- 172.21.15.37:0/1205530040 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f79b815a940 0x7f79b81f6870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:29:40.472 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:40.468+0000 7f79b7fff640 1 -- 172.21.15.37:0/1205530040 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f79b80b2dd0 msgr2=0x7f79b81534f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:29:40.472 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:40.468+0000 7f79b7fff640 1 --2- 172.21.15.37:0/1205530040 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f79b80b2dd0 0x7f79b81534f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:29:40.472 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:40.468+0000 7f79b7fff640 1 -- 172.21.15.37:0/1205530040 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f79b815a940 msgr2=0x7f79b81f6870 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:29:40.473 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:40.468+0000 7f79b7fff640 1 --2- 172.21.15.37:0/1205530040 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f79b815a940 0x7f79b81f6870 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:29:40.473 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:40.468+0000 7f79b7fff640 1 -- 172.21.15.37:0/1205530040 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f79b81e6f60 con 0x7f79b809b700 2024-04-25T10:29:40.473 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:40.468+0000 7f79b7fff640 1 --2- 172.21.15.37:0/1205530040 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f79b809b700 0x7f79b81e6660 secure :-1 s=READY pgs=230 cs=0 l=1 rev1=1 crypto rx=0x7f79a800c3f0 tx=0x7f79a800c8b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:29:40.473 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:40.468+0000 7f79b4ff9640 1 -- 172.21.15.37:0/1205530040 <== mon.0 v2:172.21.15.37:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f79a8017070 con 0x7f79b809b700 2024-04-25T10:29:40.473 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:40.468+0000 7f79b4ff9640 1 -- 172.21.15.37:0/1205530040 <== mon.0 v2:172.21.15.37:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f79a80052f0 con 0x7f79b809b700 2024-04-25T10:29:40.473 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:40.468+0000 7f79b4ff9640 1 -- 172.21.15.37:0/1205530040 <== mon.0 v2:172.21.15.37:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f79a8003830 con 0x7f79b809b700 2024-04-25T10:29:40.473 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:40.468+0000 7f79be72e640 1 -- 172.21.15.37:0/1205530040 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f79b81e7240 con 0x7f79b809b700 2024-04-25T10:29:40.473 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:40.468+0000 7f79be72e640 1 -- 172.21.15.37:0/1205530040 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f79b81e8420 con 0x7f79b809b700 2024-04-25T10:29:40.473 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:40.469+0000 7f79be72e640 1 -- 172.21.15.37:0/1205530040 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f79b806a140 con 0x7f79b809b700 2024-04-25T10:29:40.474 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:40.470+0000 7f79b4ff9640 1 -- 172.21.15.37:0/1205530040 <== mon.0 v2:172.21.15.37:3300/0 4 ==== mgrmap(e 5) ==== 96782+0+0 (secure 0 0 0) 0x7f79a801a070 con 0x7f79b809b700 2024-04-25T10:29:40.474 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:40.470+0000 7f79b4ff9640 1 --2- 172.21.15.37:0/1205530040 >> [v2:172.21.15.203:6832/2418205237,v1:172.21.15.203:6833/2418205237] conn(0x7f79a40744b0 0x7f79a40769c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:29:40.474 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:40.471+0000 7f79b4ff9640 1 -- 172.21.15.37:0/1205530040 <== mon.0 v2:172.21.15.37:3300/0 5 ==== osd_map(23..23 src has 1..23) ==== 5709+0+0 (secure 0 0 0) 0x7f79a80945d0 con 0x7f79b809b700 2024-04-25T10:29:40.474 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:40.471+0000 7f79b77fe640 1 --2- 172.21.15.37:0/1205530040 >> [v2:172.21.15.203:6832/2418205237,v1:172.21.15.203:6833/2418205237] conn(0x7f79a40744b0 0x7f79a40769c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:29:40.475 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:40.471+0000 7f79b77fe640 1 --2- 172.21.15.37:0/1205530040 >> [v2:172.21.15.203:6832/2418205237,v1:172.21.15.203:6833/2418205237] conn(0x7f79a40744b0 0x7f79a40769c0 secure :-1 s=READY pgs=130 cs=0 l=1 rev1=1 crypto rx=0x7f79ac034000 tx=0x7f79ac02e000 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:29:40.477 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:40.475+0000 7f79b4ff9640 1 -- 172.21.15.37:0/1205530040 <== mon.0 v2:172.21.15.37:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187246 (secure 0 0 0) 0x7f79a805ecc0 con 0x7f79b809b700 2024-04-25T10:29:40.662 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:40.660+0000 7f79be72e640 1 -- 172.21.15.37:0/1205530040 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) -- 0x7f79b81601d0 con 0x7f79b809b700 2024-04-25T10:29:40.662 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:40.660+0000 7f79b4ff9640 1 -- 172.21.15.37:0/1205530040 <== mon.0 v2:172.21.15.37:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v23) ==== 93+0+10 (secure 0 0 0) 0x7f79a805e5c0 con 0x7f79b809b700 2024-04-25T10:29:40.663 INFO:teuthology.orchestra.run.smithi037.stdout:pg_num: 1 2024-04-25T10:29:40.664 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:40.662+0000 7f79be72e640 1 -- 172.21.15.37:0/1205530040 >> [v2:172.21.15.203:6832/2418205237,v1:172.21.15.203:6833/2418205237] conn(0x7f79a40744b0 msgr2=0x7f79a40769c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:29:40.665 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:40.662+0000 7f79be72e640 1 --2- 172.21.15.37:0/1205530040 >> [v2:172.21.15.203:6832/2418205237,v1:172.21.15.203:6833/2418205237] conn(0x7f79a40744b0 0x7f79a40769c0 secure :-1 s=READY pgs=130 cs=0 l=1 rev1=1 crypto rx=0x7f79ac034000 tx=0x7f79ac02e000 comp rx=0 tx=0).stop 2024-04-25T10:29:40.665 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:40.662+0000 7f79be72e640 1 -- 172.21.15.37:0/1205530040 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f79b809b700 msgr2=0x7f79b81e6660 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:29:40.665 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:40.662+0000 7f79be72e640 1 --2- 172.21.15.37:0/1205530040 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f79b809b700 0x7f79b81e6660 secure :-1 s=READY pgs=230 cs=0 l=1 rev1=1 crypto rx=0x7f79a800c3f0 tx=0x7f79a800c8b0 comp rx=0 tx=0).stop 2024-04-25T10:29:40.665 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:40.662+0000 7f79be72e640 1 -- 172.21.15.37:0/1205530040 shutdown_connections 2024-04-25T10:29:40.665 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:40.662+0000 7f79be72e640 1 --2- 172.21.15.37:0/1205530040 >> [v2:172.21.15.203:6832/2418205237,v1:172.21.15.203:6833/2418205237] conn(0x7f79a40744b0 0x7f79a40769c0 unknown :-1 s=CLOSED pgs=130 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:29:40.665 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:40.662+0000 7f79be72e640 1 --2- 172.21.15.37:0/1205530040 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f79b815a940 0x7f79b81f6870 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:29:40.665 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:40.662+0000 7f79be72e640 1 --2- 172.21.15.37:0/1205530040 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f79b80b2dd0 0x7f79b81534f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:29:40.665 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:40.662+0000 7f79be72e640 1 --2- 172.21.15.37:0/1205530040 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f79b809b700 0x7f79b81e6660 unknown :-1 s=CLOSED pgs=230 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:29:40.665 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:40.662+0000 7f79be72e640 1 -- 172.21.15.37:0/1205530040 >> 172.21.15.37:0/1205530040 conn(0x7f79b80b7600 msgr2=0x7f79b80b43a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:29:40.665 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:40.662+0000 7f79be72e640 1 -- 172.21.15.37:0/1205530040 shutdown_connections 2024-04-25T10:29:40.665 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:40.662+0000 7f79be72e640 1 -- 172.21.15.37:0/1205530040 wait complete. 2024-04-25T10:29:40.675 DEBUG:teuthology.orchestra.run.smithi037:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_metadata pg_num 2024-04-25T10:29:40.790 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:40.788+0000 7ff2b8020640 1 Processor -- start 2024-04-25T10:29:40.790 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:40.788+0000 7ff2b8020640 1 -- start start 2024-04-25T10:29:40.792 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:40.789+0000 7ff2b8020640 1 --2- >> v2:172.21.15.203:3300/0 conn(0x7ff2b00b7600 0x7ff2b00b7a10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:29:40.792 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:40.789+0000 7ff2b8020640 1 --2- >> v2:172.21.15.37:3300/0 conn(0x7ff2b00d4560 0x7ff2b00d4970 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:29:40.792 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:40.789+0000 7ff2b8020640 1 -- --> v1:172.21.15.203:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7ff2b00b7f50 con 0x7ff2b00b4bb0 2024-04-25T10:29:40.792 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:40.789+0000 7ff2b8020640 1 -- --> v2:172.21.15.37:3300/0 -- mon_getmap magic: 0 -- 0x7ff2b0161190 con 0x7ff2b00d4560 2024-04-25T10:29:40.792 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:40.789+0000 7ff2b8020640 1 -- --> v2:172.21.15.203:3300/0 -- mon_getmap magic: 0 -- 0x7ff2b0161470 con 0x7ff2b00b7600 2024-04-25T10:29:40.792 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:40.789+0000 7ff2b65bd640 1 --2- >> v2:172.21.15.37:3300/0 conn(0x7ff2b00d4560 0x7ff2b00d4970 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:29:40.792 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:40.789+0000 7ff2b65bd640 1 --2- >> v2:172.21.15.37:3300/0 conn(0x7ff2b00d4560 0x7ff2b00d4970 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.37:3300/0 says I am v2:172.21.15.37:59330/0 (socket says 172.21.15.37:59330) 2024-04-25T10:29:40.792 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:40.789+0000 7ff2b65bd640 1 -- 172.21.15.37:0/2097751271 learned_addr learned my addr 172.21.15.37:0/2097751271 (peer_addr_for_me v2:172.21.15.37:0/0) 2024-04-25T10:29:40.792 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:40.790+0000 7ff2b55bb640 1 --2- 172.21.15.37:0/2097751271 >> v2:172.21.15.203:3300/0 conn(0x7ff2b00b7600 0x7ff2b00b7a10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:29:40.793 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:40.790+0000 7ff2b65bd640 1 -- 172.21.15.37:0/2097751271 >> v2:172.21.15.203:3300/0 conn(0x7ff2b00b7600 msgr2=0x7ff2b00b7a10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:29:40.793 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:40.790+0000 7ff2b65bd640 1 --2- 172.21.15.37:0/2097751271 >> v2:172.21.15.203:3300/0 conn(0x7ff2b00b7600 0x7ff2b00b7a10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:29:40.793 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:40.790+0000 7ff2b65bd640 1 -- 172.21.15.37:0/2097751271 >> v1:172.21.15.203:6789/0 conn(0x7ff2b00b4bb0 legacy=0x7ff2b01646b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:29:40.793 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:40.790+0000 7ff2b65bd640 1 -- 172.21.15.37:0/2097751271 --> v2:172.21.15.37:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ff2b0163eb0 con 0x7ff2b00d4560 2024-04-25T10:29:40.794 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:40.790+0000 7ff2b65bd640 1 --2- 172.21.15.37:0/2097751271 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7ff2b00d4560 0x7ff2b00d4970 secure :-1 s=READY pgs=231 cs=0 l=1 rev1=1 crypto rx=0x7ff2b00ccbe0 tx=0x7ff2a802a730 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f462c4c2af40762f server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:29:40.794 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:40.791+0000 7ff2b4dba640 1 -- 172.21.15.37:0/2097751271 <== mon.0 v2:172.21.15.37:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff2a8036070 con 0x7ff2b00d4560 2024-04-25T10:29:40.794 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:40.791+0000 7ff2b4dba640 1 -- 172.21.15.37:0/2097751271 <== mon.0 v2:172.21.15.37:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7ff2a8031600 con 0x7ff2b00d4560 2024-04-25T10:29:40.794 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:40.791+0000 7ff2b4dba640 1 -- 172.21.15.37:0/2097751271 <== mon.0 v2:172.21.15.37:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff2a80319d0 con 0x7ff2b00d4560 2024-04-25T10:29:40.794 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:40.791+0000 7ff2b8020640 1 -- 172.21.15.37:0/2097751271 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7ff2b00d4560 msgr2=0x7ff2b00d4970 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:29:40.794 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:40.791+0000 7ff2b8020640 1 --2- 172.21.15.37:0/2097751271 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7ff2b00d4560 0x7ff2b00d4970 secure :-1 s=READY pgs=231 cs=0 l=1 rev1=1 crypto rx=0x7ff2b00ccbe0 tx=0x7ff2a802a730 comp rx=0 tx=0).stop 2024-04-25T10:29:40.795 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:40.791+0000 7ff2b8020640 1 -- 172.21.15.37:0/2097751271 shutdown_connections 2024-04-25T10:29:40.795 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:40.791+0000 7ff2b8020640 1 --2- 172.21.15.37:0/2097751271 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7ff2b00d4560 0x7ff2b00d4970 unknown :-1 s=CLOSED pgs=231 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:29:40.795 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:40.791+0000 7ff2b8020640 1 --2- 172.21.15.37:0/2097751271 >> v2:172.21.15.203:3300/0 conn(0x7ff2b00b7600 0x7ff2b00b7a10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:29:40.795 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:40.791+0000 7ff2b8020640 1 -- 172.21.15.37:0/2097751271 >> 172.21.15.37:0/2097751271 conn(0x7ff2b00d27e0 msgr2=0x7ff2b00d2bf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:29:40.795 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:40.791+0000 7ff2b8020640 1 -- 172.21.15.37:0/2097751271 shutdown_connections 2024-04-25T10:29:40.795 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:40.791+0000 7ff2b8020640 1 -- 172.21.15.37:0/2097751271 wait complete. 2024-04-25T10:29:40.795 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:40.793+0000 7ff2b8020640 1 Processor -- start 2024-04-25T10:29:40.795 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:40.793+0000 7ff2b8020640 1 -- start start 2024-04-25T10:29:40.798 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:40.795+0000 7ff2b8020640 1 --2- >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7ff2b00b4bb0 0x7ff2b015cfa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:29:40.798 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:40.795+0000 7ff2b8020640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7ff2b00b7600 0x7ff2b01f9900 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:29:40.798 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:40.795+0000 7ff2b8020640 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7ff2b00d4560 0x7ff2b00b9810 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:29:40.798 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:40.795+0000 7ff2b8020640 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 -- 0x7ff2b00d09e0 con 0x7ff2b00b7600 2024-04-25T10:29:40.798 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:40.795+0000 7ff2b8020640 1 -- --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_getmap magic: 0 -- 0x7ff2b00b4fc0 con 0x7ff2b00d4560 2024-04-25T10:29:40.799 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:40.795+0000 7ff2b8020640 1 -- --> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] -- mon_getmap magic: 0 -- 0x7ff2b00d0b60 con 0x7ff2b00b4bb0 2024-04-25T10:29:40.799 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:40.795+0000 7ff2b55bb640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7ff2b00b7600 0x7ff2b01f9900 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:29:40.799 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:40.795+0000 7ff2b55bb640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7ff2b00b7600 0x7ff2b01f9900 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.37:3300/0 says I am v2:172.21.15.37:59340/0 (socket says 172.21.15.37:59340) 2024-04-25T10:29:40.799 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:40.795+0000 7ff2b55bb640 1 -- 172.21.15.37:0/1501447711 learned_addr learned my addr 172.21.15.37:0/1501447711 (peer_addr_for_me v2:172.21.15.37:0/0) 2024-04-25T10:29:40.799 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:40.795+0000 7ff2b5dbc640 1 --2- 172.21.15.37:0/1501447711 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7ff2b00b4bb0 0x7ff2b015cfa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:29:40.799 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:40.795+0000 7ff2b65bd640 1 --2- 172.21.15.37:0/1501447711 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7ff2b00d4560 0x7ff2b00b9810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:29:40.799 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:40.795+0000 7ff2b55bb640 1 -- 172.21.15.37:0/1501447711 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7ff2b00b4bb0 msgr2=0x7ff2b015cfa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:29:40.799 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:40.795+0000 7ff2b55bb640 1 --2- 172.21.15.37:0/1501447711 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7ff2b00b4bb0 0x7ff2b015cfa0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:29:40.799 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:40.795+0000 7ff2b55bb640 1 -- 172.21.15.37:0/1501447711 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7ff2b00d4560 msgr2=0x7ff2b00b9810 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:29:40.799 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:40.795+0000 7ff2b55bb640 1 --2- 172.21.15.37:0/1501447711 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7ff2b00d4560 0x7ff2b00b9810 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:29:40.799 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:40.795+0000 7ff2b55bb640 1 -- 172.21.15.37:0/1501447711 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ff2b01fae40 con 0x7ff2b00b7600 2024-04-25T10:29:40.799 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:40.795+0000 7ff2b5dbc640 1 --2- 172.21.15.37:0/1501447711 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7ff2b00b4bb0 0x7ff2b015cfa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T10:29:40.799 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:40.795+0000 7ff2b55bb640 1 --2- 172.21.15.37:0/1501447711 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7ff2b00b7600 0x7ff2b01f9900 secure :-1 s=READY pgs=232 cs=0 l=1 rev1=1 crypto rx=0x7ff2a000d5f0 tx=0x7ff2a000dab0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:29:40.800 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:40.795+0000 7ff2a67fc640 1 -- 172.21.15.37:0/1501447711 <== mon.0 v2:172.21.15.37:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff2a00125f0 con 0x7ff2b00b7600 2024-04-25T10:29:40.800 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:40.795+0000 7ff2a67fc640 1 -- 172.21.15.37:0/1501447711 <== mon.0 v2:172.21.15.37:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7ff2a0019070 con 0x7ff2b00b7600 2024-04-25T10:29:40.800 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:40.795+0000 7ff2b8020640 1 -- 172.21.15.37:0/1501447711 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7ff2b015cbe0 con 0x7ff2b00b7600 2024-04-25T10:29:40.800 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:40.795+0000 7ff2a67fc640 1 -- 172.21.15.37:0/1501447711 <== mon.0 v2:172.21.15.37:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff2a00129d0 con 0x7ff2b00b7600 2024-04-25T10:29:40.800 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:40.795+0000 7ff2b8020640 1 -- 172.21.15.37:0/1501447711 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7ff2b01fa5a0 con 0x7ff2b00b7600 2024-04-25T10:29:40.800 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:40.796+0000 7ff2b8020640 1 -- 172.21.15.37:0/1501447711 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7ff2b006a140 con 0x7ff2b00b7600 2024-04-25T10:29:40.801 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:40.797+0000 7ff2a67fc640 1 -- 172.21.15.37:0/1501447711 <== mon.0 v2:172.21.15.37:3300/0 4 ==== mgrmap(e 5) ==== 96782+0+0 (secure 0 0 0) 0x7ff2a0012b70 con 0x7ff2b00b7600 2024-04-25T10:29:40.801 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:40.797+0000 7ff2a67fc640 1 --2- 172.21.15.37:0/1501447711 >> [v2:172.21.15.203:6832/2418205237,v1:172.21.15.203:6833/2418205237] conn(0x7ff2940744b0 0x7ff2940769c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:29:40.801 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:40.798+0000 7ff2a67fc640 1 -- 172.21.15.37:0/1501447711 <== mon.0 v2:172.21.15.37:3300/0 5 ==== osd_map(23..23 src has 1..23) ==== 5709+0+0 (secure 0 0 0) 0x7ff2a0095750 con 0x7ff2b00b7600 2024-04-25T10:29:40.801 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:40.798+0000 7ff2b5dbc640 1 --2- 172.21.15.37:0/1501447711 >> [v2:172.21.15.203:6832/2418205237,v1:172.21.15.203:6833/2418205237] conn(0x7ff2940744b0 0x7ff2940769c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:29:40.801 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:40.798+0000 7ff2b5dbc640 1 --2- 172.21.15.37:0/1501447711 >> [v2:172.21.15.203:6832/2418205237,v1:172.21.15.203:6833/2418205237] conn(0x7ff2940744b0 0x7ff2940769c0 secure :-1 s=READY pgs=131 cs=0 l=1 rev1=1 crypto rx=0x7ff2b00d6c00 tx=0x7ff2ac009040 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:29:40.806 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:40.802+0000 7ff2a67fc640 1 -- 172.21.15.37:0/1501447711 <== mon.0 v2:172.21.15.37:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187246 (secure 0 0 0) 0x7ff2b015cbe0 con 0x7ff2b00b7600 2024-04-25T10:29:40.992 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:40.990+0000 7ff2b8020640 1 -- 172.21.15.37:0/1501447711 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"} v 0) -- 0x7ff2b00d3ff0 con 0x7ff2b00b7600 2024-04-25T10:29:40.992 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:40.990+0000 7ff2a67fc640 1 -- 172.21.15.37:0/1501447711 <== mon.0 v2:172.21.15.37:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]=0 v23) ==== 104+0+11 (secure 0 0 0) 0x7ff2a005fe50 con 0x7ff2b00b7600 2024-04-25T10:29:40.993 INFO:teuthology.orchestra.run.smithi037.stdout:pg_num: 32 2024-04-25T10:29:40.995 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:40.992+0000 7ff2b8020640 1 -- 172.21.15.37:0/1501447711 >> [v2:172.21.15.203:6832/2418205237,v1:172.21.15.203:6833/2418205237] conn(0x7ff2940744b0 msgr2=0x7ff2940769c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:29:40.995 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:40.992+0000 7ff2b8020640 1 --2- 172.21.15.37:0/1501447711 >> [v2:172.21.15.203:6832/2418205237,v1:172.21.15.203:6833/2418205237] conn(0x7ff2940744b0 0x7ff2940769c0 secure :-1 s=READY pgs=131 cs=0 l=1 rev1=1 crypto rx=0x7ff2b00d6c00 tx=0x7ff2ac009040 comp rx=0 tx=0).stop 2024-04-25T10:29:40.995 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:40.992+0000 7ff2b8020640 1 -- 172.21.15.37:0/1501447711 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7ff2b00b7600 msgr2=0x7ff2b01f9900 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:29:40.995 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:40.992+0000 7ff2b8020640 1 --2- 172.21.15.37:0/1501447711 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7ff2b00b7600 0x7ff2b01f9900 secure :-1 s=READY pgs=232 cs=0 l=1 rev1=1 crypto rx=0x7ff2a000d5f0 tx=0x7ff2a000dab0 comp rx=0 tx=0).stop 2024-04-25T10:29:40.995 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:40.992+0000 7ff2b8020640 1 -- 172.21.15.37:0/1501447711 shutdown_connections 2024-04-25T10:29:40.995 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:40.992+0000 7ff2b8020640 1 --2- 172.21.15.37:0/1501447711 >> [v2:172.21.15.203:6832/2418205237,v1:172.21.15.203:6833/2418205237] conn(0x7ff2940744b0 0x7ff2940769c0 unknown :-1 s=CLOSED pgs=131 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:29:40.995 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:40.992+0000 7ff2b8020640 1 --2- 172.21.15.37:0/1501447711 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7ff2b00d4560 0x7ff2b00b9810 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:29:40.995 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:40.992+0000 7ff2b8020640 1 --2- 172.21.15.37:0/1501447711 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7ff2b00b7600 0x7ff2b01f9900 unknown :-1 s=CLOSED pgs=232 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:29:40.995 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:40.992+0000 7ff2b8020640 1 --2- 172.21.15.37:0/1501447711 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7ff2b00b4bb0 0x7ff2b015cfa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:29:40.996 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:40.992+0000 7ff2b8020640 1 -- 172.21.15.37:0/1501447711 >> 172.21.15.37:0/1501447711 conn(0x7ff2b00d27e0 msgr2=0x7ff2b00b54c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:29:40.996 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:40.992+0000 7ff2b8020640 1 -- 172.21.15.37:0/1501447711 shutdown_connections 2024-04-25T10:29:40.996 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:40.992+0000 7ff2b8020640 1 -- 172.21.15.37:0/1501447711 wait complete. 2024-04-25T10:29:41.005 DEBUG:teuthology.orchestra.run.smithi037:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data pg_num 2024-04-25T10:29:41.122 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:41.119+0000 7f5727b0d640 1 Processor -- start 2024-04-25T10:29:41.122 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:41.119+0000 7f5727b0d640 1 -- start start 2024-04-25T10:29:41.124 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:41.121+0000 7f5727b0d640 1 --2- >> v2:172.21.15.203:3300/0 conn(0x7f572006e260 0x7f572006e670 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:29:41.125 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:41.121+0000 7f5727b0d640 1 --2- >> v2:172.21.15.37:3300/0 conn(0x7f572006ebb0 0x7f57201645d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:29:41.125 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:41.121+0000 7f5727b0d640 1 -- --> v1:172.21.15.37:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f572009ce70 con 0x7f5720164b10 2024-04-25T10:29:41.125 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:41.121+0000 7f5727b0d640 1 -- --> v2:172.21.15.37:3300/0 -- mon_getmap magic: 0 -- 0x7f57200d3370 con 0x7f572006ebb0 2024-04-25T10:29:41.125 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:41.121+0000 7f5727b0d640 1 -- --> v2:172.21.15.203:3300/0 -- mon_getmap magic: 0 -- 0x7f57200d3090 con 0x7f572006e260 2024-04-25T10:29:41.125 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:41.121+0000 7f57250a8640 1 --2- >> v2:172.21.15.37:3300/0 conn(0x7f572006ebb0 0x7f57201645d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:29:41.125 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:41.121+0000 7f57260aa640 1 --1- >> v1:172.21.15.37:6789/0 conn(0x7f5720164b10 0x7f57200d1ec0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.37:6789/0 says I am v1:172.21.15.37:49806/0 (socket says 172.21.15.37:49806) 2024-04-25T10:29:41.126 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:41.121+0000 7f57260aa640 1 -- 172.21.15.37:0/548036307 learned_addr learned my addr 172.21.15.37:0/548036307 (peer_addr_for_me v1:172.21.15.37:0/0) 2024-04-25T10:29:41.126 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:41.121+0000 7f57250a8640 1 -- 172.21.15.37:0/548036307 >> v2:172.21.15.203:3300/0 conn(0x7f572006e260 msgr2=0x7f572006e670 unknown :-1 s=STATE_CONNECTING_RE l=0).mark_down 2024-04-25T10:29:41.126 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:41.121+0000 7f57250a8640 1 --2- 172.21.15.37:0/548036307 >> v2:172.21.15.203:3300/0 conn(0x7f572006e260 0x7f572006e670 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:29:41.126 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:41.121+0000 7f57250a8640 1 -- 172.21.15.37:0/548036307 >> v1:172.21.15.37:6789/0 conn(0x7f5720164b10 legacy=0x7f57200d1ec0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:29:41.126 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:41.121+0000 7f57250a8640 1 -- 172.21.15.37:0/548036307 --> v2:172.21.15.37:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f572009d150 con 0x7f572006ebb0 2024-04-25T10:29:41.127 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:41.121+0000 7f57248a7640 1 -- 172.21.15.37:0/548036307 <== mon.0 v1:172.21.15.37:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 1129632071 0 0) 0x7f572009ce70 con 0x7f5720164b10 2024-04-25T10:29:41.127 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:41.122+0000 7f57250a8640 1 --2- 172.21.15.37:0/548036307 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f572006ebb0 0x7f57201645d0 secure :-1 s=READY pgs=234 cs=0 l=1 rev1=1 crypto rx=0x7f5720074460 tx=0x7f571402a130 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d8db8a0c071d7a66 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:29:41.127 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:41.122+0000 7f57248a7640 1 -- 172.21.15.37:0/548036307 <== mon.0 v2:172.21.15.37:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5714037070 con 0x7f572006ebb0 2024-04-25T10:29:41.127 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:41.122+0000 7f57248a7640 1 -- 172.21.15.37:0/548036307 <== mon.0 v2:172.21.15.37:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f5714032440 con 0x7f572006ebb0 2024-04-25T10:29:41.127 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:41.122+0000 7f57248a7640 1 -- 172.21.15.37:0/548036307 <== mon.0 v2:172.21.15.37:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5714032810 con 0x7f572006ebb0 2024-04-25T10:29:41.132 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:41.123+0000 7f5727b0d640 1 -- 172.21.15.37:0/548036307 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f572006ebb0 msgr2=0x7f57201645d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:29:41.132 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:41.123+0000 7f5727b0d640 1 --2- 172.21.15.37:0/548036307 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f572006ebb0 0x7f57201645d0 secure :-1 s=READY pgs=234 cs=0 l=1 rev1=1 crypto rx=0x7f5720074460 tx=0x7f571402a130 comp rx=0 tx=0).stop 2024-04-25T10:29:41.132 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:41.123+0000 7f5727b0d640 1 -- 172.21.15.37:0/548036307 shutdown_connections 2024-04-25T10:29:41.132 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:41.123+0000 7f5727b0d640 1 --2- 172.21.15.37:0/548036307 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f572006ebb0 0x7f57201645d0 unknown :-1 s=CLOSED pgs=234 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:29:41.132 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:41.123+0000 7f5727b0d640 1 --2- 172.21.15.37:0/548036307 >> v2:172.21.15.203:3300/0 conn(0x7f572006e260 0x7f572006e670 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:29:41.132 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:41.123+0000 7f5727b0d640 1 -- 172.21.15.37:0/548036307 >> 172.21.15.37:0/548036307 conn(0x7f57200d0390 msgr2=0x7f57200d07a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:29:41.132 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:41.123+0000 7f5727b0d640 1 -- 172.21.15.37:0/548036307 shutdown_connections 2024-04-25T10:29:41.132 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:41.123+0000 7f5727b0d640 1 -- 172.21.15.37:0/548036307 wait complete. 2024-04-25T10:29:41.133 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:41.125+0000 7f5727b0d640 1 Processor -- start 2024-04-25T10:29:41.133 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:41.125+0000 7f5727b0d640 1 -- start start 2024-04-25T10:29:41.133 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:41.126+0000 7f5727b0d640 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f572006e260 0x7f57201f6270 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:29:41.133 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:41.126+0000 7f5727b0d640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f572006ebb0 0x7f5720159d80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:29:41.133 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:41.126+0000 7f5727b0d640 1 --2- >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f5720164b10 0x7f57200a0d60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:29:41.133 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:41.126+0000 7f5727b0d640 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 -- 0x7f572006db30 con 0x7f572006ebb0 2024-04-25T10:29:41.133 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:41.126+0000 7f5727b0d640 1 -- --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_getmap magic: 0 -- 0x7f572006d830 con 0x7f572006e260 2024-04-25T10:29:41.133 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:41.126+0000 7f5727b0d640 1 -- --> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] -- mon_getmap magic: 0 -- 0x7f572006dcb0 con 0x7f5720164b10 2024-04-25T10:29:41.133 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:41.126+0000 7f57250a8640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f572006ebb0 0x7f5720159d80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:29:41.133 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:41.126+0000 7f57250a8640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f572006ebb0 0x7f5720159d80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.37:3300/0 says I am v2:172.21.15.37:59356/0 (socket says 172.21.15.37:59356) 2024-04-25T10:29:41.133 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:41.126+0000 7f57250a8640 1 -- 172.21.15.37:0/1651968814 learned_addr learned my addr 172.21.15.37:0/1651968814 (peer_addr_for_me v2:172.21.15.37:0/0) 2024-04-25T10:29:41.133 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:41.126+0000 7f57260aa640 1 --2- 172.21.15.37:0/1651968814 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f5720164b10 0x7f57200a0d60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:29:41.133 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:41.126+0000 7f57258a9640 1 --2- 172.21.15.37:0/1651968814 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f572006e260 0x7f57201f6270 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:29:41.133 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:41.126+0000 7f57250a8640 1 -- 172.21.15.37:0/1651968814 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f5720164b10 msgr2=0x7f57200a0d60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:29:41.134 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:41.126+0000 7f57250a8640 1 --2- 172.21.15.37:0/1651968814 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f5720164b10 0x7f57200a0d60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:29:41.134 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:41.126+0000 7f57250a8640 1 -- 172.21.15.37:0/1651968814 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f572006e260 msgr2=0x7f57201f6270 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:29:41.134 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:41.126+0000 7f57250a8640 1 --2- 172.21.15.37:0/1651968814 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f572006e260 0x7f57201f6270 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:29:41.134 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:41.126+0000 7f57250a8640 1 -- 172.21.15.37:0/1651968814 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f57201f67b0 con 0x7f572006ebb0 2024-04-25T10:29:41.134 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:41.127+0000 7f57250a8640 1 --2- 172.21.15.37:0/1651968814 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f572006ebb0 0x7f5720159d80 secure :-1 s=READY pgs=235 cs=0 l=1 rev1=1 crypto rx=0x7f5714035040 tx=0x7f57140046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:29:41.134 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:41.127+0000 7f571e7fc640 1 -- 172.21.15.37:0/1651968814 <== mon.0 v2:172.21.15.37:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5714037040 con 0x7f572006ebb0 2024-04-25T10:29:41.134 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:41.127+0000 7f571e7fc640 1 -- 172.21.15.37:0/1651968814 <== mon.0 v2:172.21.15.37:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f5714004230 con 0x7f572006ebb0 2024-04-25T10:29:41.134 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:41.127+0000 7f5727b0d640 1 -- 172.21.15.37:0/1651968814 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f572009f7c0 con 0x7f572006ebb0 2024-04-25T10:29:41.134 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:41.127+0000 7f571e7fc640 1 -- 172.21.15.37:0/1651968814 <== mon.0 v2:172.21.15.37:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5714032e20 con 0x7f572006ebb0 2024-04-25T10:29:41.134 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:41.127+0000 7f5727b0d640 1 -- 172.21.15.37:0/1651968814 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f572009ffa0 con 0x7f572006ebb0 2024-04-25T10:29:41.134 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:41.128+0000 7f5727b0d640 1 -- 172.21.15.37:0/1651968814 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f572009fa30 con 0x7f572006ebb0 2024-04-25T10:29:41.134 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:41.128+0000 7f571e7fc640 1 -- 172.21.15.37:0/1651968814 <== mon.0 v2:172.21.15.37:3300/0 4 ==== mgrmap(e 5) ==== 96782+0+0 (secure 0 0 0) 0x7f5714030770 con 0x7f572006ebb0 2024-04-25T10:29:41.134 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:41.129+0000 7f571e7fc640 1 --2- 172.21.15.37:0/1651968814 >> [v2:172.21.15.203:6832/2418205237,v1:172.21.15.203:6833/2418205237] conn(0x7f570c0744e0 0x7f570c0769f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:29:41.134 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:41.129+0000 7f571e7fc640 1 -- 172.21.15.37:0/1651968814 <== mon.0 v2:172.21.15.37:3300/0 5 ==== osd_map(23..23 src has 1..23) ==== 5709+0+0 (secure 0 0 0) 0x7f57140b6410 con 0x7f572006ebb0 2024-04-25T10:29:41.135 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:41.130+0000 7f57258a9640 1 --2- 172.21.15.37:0/1651968814 >> [v2:172.21.15.203:6832/2418205237,v1:172.21.15.203:6833/2418205237] conn(0x7f570c0744e0 0x7f570c0769f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:29:41.135 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:41.130+0000 7f57258a9640 1 --2- 172.21.15.37:0/1651968814 >> [v2:172.21.15.203:6832/2418205237,v1:172.21.15.203:6833/2418205237] conn(0x7f570c0744e0 0x7f570c0769f0 secure :-1 s=READY pgs=132 cs=0 l=1 rev1=1 crypto rx=0x7f5718006fd0 tx=0x7f5718008040 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:29:41.136 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:41.134+0000 7f571e7fc640 1 -- 172.21.15.37:0/1651968814 <== mon.0 v2:172.21.15.37:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187246 (secure 0 0 0) 0x7f5714080a80 con 0x7f572006ebb0 2024-04-25T10:29:41.324 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:41.321+0000 7f5727b0d640 1 -- 172.21.15.37:0/1651968814 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"} v 0) -- 0x7f572006a140 con 0x7f572006ebb0 2024-04-25T10:29:41.324 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:41.322+0000 7f571e7fc640 1 -- 172.21.15.37:0/1651968814 <== mon.0 v2:172.21.15.37:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]=0 v23) ==== 100+0+11 (secure 0 0 0) 0x7f5714080380 con 0x7f572006ebb0 2024-04-25T10:29:41.324 INFO:teuthology.orchestra.run.smithi037.stdout:pg_num: 64 2024-04-25T10:29:41.326 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:41.324+0000 7f5727b0d640 1 -- 172.21.15.37:0/1651968814 >> [v2:172.21.15.203:6832/2418205237,v1:172.21.15.203:6833/2418205237] conn(0x7f570c0744e0 msgr2=0x7f570c0769f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:29:41.326 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:41.324+0000 7f5727b0d640 1 --2- 172.21.15.37:0/1651968814 >> [v2:172.21.15.203:6832/2418205237,v1:172.21.15.203:6833/2418205237] conn(0x7f570c0744e0 0x7f570c0769f0 secure :-1 s=READY pgs=132 cs=0 l=1 rev1=1 crypto rx=0x7f5718006fd0 tx=0x7f5718008040 comp rx=0 tx=0).stop 2024-04-25T10:29:41.326 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:41.324+0000 7f5727b0d640 1 -- 172.21.15.37:0/1651968814 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f572006ebb0 msgr2=0x7f5720159d80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:29:41.326 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:41.324+0000 7f5727b0d640 1 --2- 172.21.15.37:0/1651968814 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f572006ebb0 0x7f5720159d80 secure :-1 s=READY pgs=235 cs=0 l=1 rev1=1 crypto rx=0x7f5714035040 tx=0x7f57140046c0 comp rx=0 tx=0).stop 2024-04-25T10:29:41.327 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:41.324+0000 7f5727b0d640 1 -- 172.21.15.37:0/1651968814 shutdown_connections 2024-04-25T10:29:41.327 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:41.324+0000 7f5727b0d640 1 --2- 172.21.15.37:0/1651968814 >> [v2:172.21.15.203:6832/2418205237,v1:172.21.15.203:6833/2418205237] conn(0x7f570c0744e0 0x7f570c0769f0 unknown :-1 s=CLOSED pgs=132 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:29:41.327 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:41.324+0000 7f5727b0d640 1 --2- 172.21.15.37:0/1651968814 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f5720164b10 0x7f57200a0d60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:29:41.327 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:41.324+0000 7f5727b0d640 1 --2- 172.21.15.37:0/1651968814 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f572006ebb0 0x7f5720159d80 unknown :-1 s=CLOSED pgs=235 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:29:41.327 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:41.324+0000 7f5727b0d640 1 --2- 172.21.15.37:0/1651968814 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f572006e260 0x7f57201f6270 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:29:41.327 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:41.324+0000 7f5727b0d640 1 -- 172.21.15.37:0/1651968814 >> 172.21.15.37:0/1651968814 conn(0x7f57200d0390 msgr2=0x7f5720071d20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:29:41.327 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:41.324+0000 7f5727b0d640 1 -- 172.21.15.37:0/1651968814 shutdown_connections 2024-04-25T10:29:41.327 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:41.324+0000 7f5727b0d640 1 -- 172.21.15.37:0/1651968814 wait complete. 2024-04-25T10:29:41.343 INFO:tasks.cephfs_test_runner:Starting test: test_rebuild_simple (tasks.cephfs.test_recovery_pool.TestRecoveryPool) 2024-04-25T10:29:41.344 DEBUG:teuthology.orchestra.run.smithi037:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-04-25T10:29:41.456 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:41.453+0000 7f6e65e9f640 1 Processor -- start 2024-04-25T10:29:41.456 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:41.453+0000 7f6e65e9f640 1 -- start start 2024-04-25T10:29:41.457 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:41.455+0000 7f6e65e9f640 1 --2- >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f6e600d5430 0x7f6e600b75c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:29:41.457 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:41.455+0000 7f6e65e9f640 1 --2- >> v2:172.21.15.37:3300/0 conn(0x7f6e600b7b00 0x7f6e60160a00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:29:41.457 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:41.455+0000 7f6e65e9f640 1 --2- >> v2:172.21.15.203:3300/0 conn(0x7f6e60160f40 0x7f6e60161880 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:29:41.457 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:41.455+0000 7f6e65e9f640 1 -- --> v2:172.21.15.37:3300/0 -- mon_getmap magic: 0 -- 0x7f6e600d4420 con 0x7f6e600b7b00 2024-04-25T10:29:41.457 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:41.455+0000 7f6e65e9f640 1 -- --> v2:172.21.15.203:3300/0 -- mon_getmap magic: 0 -- 0x7f6e600b7f10 con 0x7f6e60160f40 2024-04-25T10:29:41.458 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:41.455+0000 7f6e65e9f640 1 -- --> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] -- mon_getmap magic: 0 -- 0x7f6e600d5840 con 0x7f6e600d5430 2024-04-25T10:29:41.458 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:41.455+0000 7f6e5effd640 1 --2- >> v2:172.21.15.37:3300/0 conn(0x7f6e600b7b00 0x7f6e60160a00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:29:41.458 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:41.455+0000 7f6e5effd640 1 --2- >> v2:172.21.15.37:3300/0 conn(0x7f6e600b7b00 0x7f6e60160a00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.37:3300/0 says I am v2:172.21.15.37:59366/0 (socket says 172.21.15.37:59366) 2024-04-25T10:29:41.458 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:41.455+0000 7f6e5effd640 1 -- 172.21.15.37:0/1867262502 learned_addr learned my addr 172.21.15.37:0/1867262502 (peer_addr_for_me v2:172.21.15.37:0/0) 2024-04-25T10:29:41.458 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:41.455+0000 7f6e5effd640 1 -- 172.21.15.37:0/1867262502 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f6e600d5430 msgr2=0x7f6e600b75c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:29:41.458 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:41.455+0000 7f6e5effd640 1 --2- 172.21.15.37:0/1867262502 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f6e600d5430 0x7f6e600b75c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:29:41.458 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:41.455+0000 7f6e5effd640 1 -- 172.21.15.37:0/1867262502 >> v2:172.21.15.203:3300/0 conn(0x7f6e60160f40 msgr2=0x7f6e60161880 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:29:41.458 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:41.455+0000 7f6e5effd640 1 --2- 172.21.15.37:0/1867262502 >> v2:172.21.15.203:3300/0 conn(0x7f6e60160f40 0x7f6e60161880 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:29:41.458 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:41.455+0000 7f6e5effd640 1 -- 172.21.15.37:0/1867262502 --> v2:172.21.15.37:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f6e600d4670 con 0x7f6e600b7b00 2024-04-25T10:29:41.459 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:41.456+0000 7f6e5effd640 1 --2- 172.21.15.37:0/1867262502 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f6e600b7b00 0x7f6e60160a00 secure :-1 s=READY pgs=236 cs=0 l=1 rev1=1 crypto rx=0x7f6e6006ec40 tx=0x7f6e5402a140 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ef4e814658dc397c server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:29:41.459 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:41.456+0000 7f6e5e7fc640 1 -- 172.21.15.37:0/1867262502 <== mon.0 v2:172.21.15.37:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6e54037070 con 0x7f6e600b7b00 2024-04-25T10:29:41.459 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:41.456+0000 7f6e5e7fc640 1 -- 172.21.15.37:0/1867262502 <== mon.0 v2:172.21.15.37:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f6e54032440 con 0x7f6e600b7b00 2024-04-25T10:29:41.460 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:41.456+0000 7f6e5e7fc640 1 -- 172.21.15.37:0/1867262502 <== mon.0 v2:172.21.15.37:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6e54032810 con 0x7f6e600b7b00 2024-04-25T10:29:41.460 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:41.457+0000 7f6e65e9f640 1 -- 172.21.15.37:0/1867262502 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f6e600b7b00 msgr2=0x7f6e60160a00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:29:41.460 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:41.457+0000 7f6e65e9f640 1 --2- 172.21.15.37:0/1867262502 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f6e600b7b00 0x7f6e60160a00 secure :-1 s=READY pgs=236 cs=0 l=1 rev1=1 crypto rx=0x7f6e6006ec40 tx=0x7f6e5402a140 comp rx=0 tx=0).stop 2024-04-25T10:29:41.460 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:41.457+0000 7f6e65e9f640 1 -- 172.21.15.37:0/1867262502 shutdown_connections 2024-04-25T10:29:41.460 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:41.457+0000 7f6e65e9f640 1 --2- 172.21.15.37:0/1867262502 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f6e600b7b00 0x7f6e60160a00 unknown :-1 s=CLOSED pgs=236 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:29:41.460 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:41.457+0000 7f6e65e9f640 1 --2- 172.21.15.37:0/1867262502 >> v2:172.21.15.203:3300/0 conn(0x7f6e60160f40 0x7f6e60161880 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:29:41.460 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:41.457+0000 7f6e65e9f640 1 --2- 172.21.15.37:0/1867262502 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f6e600d5430 0x7f6e600b75c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:29:41.460 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:41.457+0000 7f6e65e9f640 1 -- 172.21.15.37:0/1867262502 >> 172.21.15.37:0/1867262502 conn(0x7f6e600d27a0 msgr2=0x7f6e600d2bb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:29:41.460 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:41.457+0000 7f6e65e9f640 1 -- 172.21.15.37:0/1867262502 shutdown_connections 2024-04-25T10:29:41.460 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:41.457+0000 7f6e65e9f640 1 -- 172.21.15.37:0/1867262502 wait complete. 2024-04-25T10:29:41.461 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:41.459+0000 7f6e65e9f640 1 Processor -- start 2024-04-25T10:29:41.461 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:41.459+0000 7f6e65e9f640 1 -- start start 2024-04-25T10:29:41.464 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:41.460+0000 7f6e65e9f640 1 --2- >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f6e600b7b00 0x7f6e600cc010 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:29:41.464 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:41.460+0000 7f6e65e9f640 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f6e600d5430 0x7f6e601f97a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:29:41.464 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:41.460+0000 7f6e65e9f640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f6e60160f40 0x7f6e600cd910 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:29:41.464 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:41.460+0000 7f6e65e9f640 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 -- 0x7f6e600d4f10 con 0x7f6e60160f40 2024-04-25T10:29:41.464 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:41.460+0000 7f6e65e9f640 1 -- --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_getmap magic: 0 -- 0x7f6e600d4c30 con 0x7f6e600d5430 2024-04-25T10:29:41.464 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:41.460+0000 7f6e65e9f640 1 -- --> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] -- mon_getmap magic: 0 -- 0x7f6e60161350 con 0x7f6e600b7b00 2024-04-25T10:29:41.464 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:41.460+0000 7f6e5ffff640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f6e60160f40 0x7f6e600cd910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:29:41.464 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:41.460+0000 7f6e5ffff640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f6e60160f40 0x7f6e600cd910 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.37:3300/0 says I am v2:172.21.15.37:59376/0 (socket says 172.21.15.37:59376) 2024-04-25T10:29:41.464 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:41.460+0000 7f6e5ffff640 1 -- 172.21.15.37:0/4036484719 learned_addr learned my addr 172.21.15.37:0/4036484719 (peer_addr_for_me v2:172.21.15.37:0/0) 2024-04-25T10:29:41.464 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:41.460+0000 7f6e5f7fe640 1 --2- 172.21.15.37:0/4036484719 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f6e600b7b00 0x7f6e600cc010 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:29:41.464 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:41.460+0000 7f6e5effd640 1 --2- 172.21.15.37:0/4036484719 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f6e600d5430 0x7f6e601f97a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:29:41.464 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:41.461+0000 7f6e5ffff640 1 -- 172.21.15.37:0/4036484719 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f6e600b7b00 msgr2=0x7f6e600cc010 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:29:41.465 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:41.461+0000 7f6e5ffff640 1 --2- 172.21.15.37:0/4036484719 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f6e600b7b00 0x7f6e600cc010 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:29:41.465 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:41.461+0000 7f6e5ffff640 1 -- 172.21.15.37:0/4036484719 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f6e600d5430 msgr2=0x7f6e601f97a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:29:41.465 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:41.461+0000 7f6e5ffff640 1 --2- 172.21.15.37:0/4036484719 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f6e600d5430 0x7f6e601f97a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:29:41.465 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:41.461+0000 7f6e5ffff640 1 -- 172.21.15.37:0/4036484719 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f6e600cce70 con 0x7f6e60160f40 2024-04-25T10:29:41.465 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:41.461+0000 7f6e5f7fe640 1 --2- 172.21.15.37:0/4036484719 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f6e600b7b00 0x7f6e600cc010 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T10:29:41.465 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:41.461+0000 7f6e5effd640 1 --2- 172.21.15.37:0/4036484719 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f6e600d5430 0x7f6e601f97a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T10:29:41.465 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:41.461+0000 7f6e5ffff640 1 --2- 172.21.15.37:0/4036484719 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f6e60160f40 0x7f6e600cd910 secure :-1 s=READY pgs=237 cs=0 l=1 rev1=1 crypto rx=0x7f6e5800ba10 tx=0x7f6e5800bed0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:29:41.465 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:41.461+0000 7f6e47fff640 1 -- 172.21.15.37:0/4036484719 <== mon.0 v2:172.21.15.37:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6e58019070 con 0x7f6e60160f40 2024-04-25T10:29:41.465 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:41.461+0000 7f6e47fff640 1 -- 172.21.15.37:0/4036484719 <== mon.0 v2:172.21.15.37:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f6e5800cb00 con 0x7f6e60160f40 2024-04-25T10:29:41.465 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:41.461+0000 7f6e65e9f640 1 -- 172.21.15.37:0/4036484719 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f6e600cc550 con 0x7f6e60160f40 2024-04-25T10:29:41.465 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:41.461+0000 7f6e47fff640 1 -- 172.21.15.37:0/4036484719 <== mon.0 v2:172.21.15.37:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6e58014440 con 0x7f6e60160f40 2024-04-25T10:29:41.465 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:41.461+0000 7f6e65e9f640 1 -- 172.21.15.37:0/4036484719 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f6e601fa540 con 0x7f6e60160f40 2024-04-25T10:29:41.466 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:41.462+0000 7f6e65e9f640 1 -- 172.21.15.37:0/4036484719 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f6e6006a140 con 0x7f6e60160f40 2024-04-25T10:29:41.466 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:41.463+0000 7f6e47fff640 1 -- 172.21.15.37:0/4036484719 <== mon.0 v2:172.21.15.37:3300/0 4 ==== mgrmap(e 5) ==== 96782+0+0 (secure 0 0 0) 0x7f6e5801c070 con 0x7f6e60160f40 2024-04-25T10:29:41.466 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:41.463+0000 7f6e47fff640 1 --2- 172.21.15.37:0/4036484719 >> [v2:172.21.15.203:6832/2418205237,v1:172.21.15.203:6833/2418205237] conn(0x7f6e4c0744e0 0x7f6e4c0769f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:29:41.466 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:41.463+0000 7f6e47fff640 1 -- 172.21.15.37:0/4036484719 <== mon.0 v2:172.21.15.37:3300/0 5 ==== osd_map(23..23 src has 1..23) ==== 5709+0+0 (secure 0 0 0) 0x7f6e58094ec0 con 0x7f6e60160f40 2024-04-25T10:29:41.467 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:41.464+0000 7f6e5f7fe640 1 --2- 172.21.15.37:0/4036484719 >> [v2:172.21.15.203:6832/2418205237,v1:172.21.15.203:6833/2418205237] conn(0x7f6e4c0744e0 0x7f6e4c0769f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:29:41.467 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:41.464+0000 7f6e5f7fe640 1 --2- 172.21.15.37:0/4036484719 >> [v2:172.21.15.203:6832/2418205237,v1:172.21.15.203:6833/2418205237] conn(0x7f6e4c0744e0 0x7f6e4c0769f0 secure :-1 s=READY pgs=133 cs=0 l=1 rev1=1 crypto rx=0x7f6e6008e110 tx=0x7f6e50008040 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:29:41.470 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:41.468+0000 7f6e47fff640 1 -- 172.21.15.37:0/4036484719 <== mon.0 v2:172.21.15.37:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187246 (secure 0 0 0) 0x7f6e5805f660 con 0x7f6e60160f40 2024-04-25T10:29:41.654 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:41.652+0000 7f6e65e9f640 1 -- 172.21.15.37:0/4036484719 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7f6e601604f0 con 0x7f6e60160f40 2024-04-25T10:29:41.656 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:41.653+0000 7f6e47fff640 1 -- 172.21.15.37:0/4036484719 <== mon.0 v2:172.21.15.37:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v23) ==== 74+0+16621 (secure 0 0 0) 0x7f6e5805ef60 con 0x7f6e60160f40 2024-04-25T10:29:41.656 INFO:teuthology.orchestra.run.smithi037.stdout: 2024-04-25T10:29:41.656 INFO:teuthology.orchestra.run.smithi037.stdout:{"epoch":23,"fsid":"f4acda28-be34-492a-b286-994afd36f61d","created":"2024-04-25T10:28:43.985718+0000","modified":"2024-04-25T10:29:10.016188+0000","last_up_change":"2024-04-25T10:28:54.900654+0000","last_in_change":"2024-04-25T10:28:46.216973+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":3,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-04-25T10:28:56.202153+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"19","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2024-04-25T10:29:06.507244+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"22","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":2.25,"score_stable":2.25,"optimal_score":1,"raw_score_acting":2.25,"raw_score_stable":2.25,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2024-04-25T10:29:07.347844+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"23","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"a2f79028-b332-4166-9ebb-a7024a37ea22","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":21,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.37:6816","nonce":2746074720},{"type":"v1","addr":"172.21.15.37:6817","nonce":2746074720}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.37:6818","nonce":2746074720},{"type":"v1","addr":"172.21.15.37:6819","nonce":2746074720}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.37:6822","nonce":2746074720},{"type":"v1","addr":"172.21.15.37:6823","nonce":2746074720}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.37:6820","nonce":2746074720},{"type":"v1","addr":"172.21.15.37:6821","nonce":2746074720}]},"public_addr":"172.21.15.37:6817/2746074720","cluster_addr":"172.21.15.37:6819/2746074720","heartbeat_back_addr":"172.21.15.37:6823/2746074720","heartbeat_front_addr":"172.21.15.37:6821/2746074720","state":["exists","up"]},{"osd":1,"uuid":"973159de-c2ea-4083-aecc-69373d1c1132","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":21,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.37:6800","nonce":1690341662},{"type":"v1","addr":"172.21.15.37:6801","nonce":1690341662}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.37:6802","nonce":1690341662},{"type":"v1","addr":"172.21.15.37:6803","nonce":1690341662}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.37:6806","nonce":1690341662},{"type":"v1","addr":"172.21.15.37:6807","nonce":1690341662}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.37:6804","nonce":1690341662},{"type":"v1","addr":"172.21.15.37:6805","nonce":1690341662}]},"public_addr":"172.21.15.37:6801/1690341662","cluster_addr":"172.21.15.37:6803/1690341662","heartbeat_back_addr":"172.21.15.37:6807/1690341662","heartbeat_front_addr":"172.21.15.37:6805/1690341662","state":["exists","up"]},{"osd":2,"uuid":"4ae7a0e9-cb3f-484c-a75f-f3766b721c47","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":21,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.37:6808","nonce":2754669649},{"type":"v1","addr":"172.21.15.37:6809","nonce":2754669649}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.37:6810","nonce":2754669649},{"type":"v1","addr":"172.21.15.37:6811","nonce":2754669649}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.37:6814","nonce":2754669649},{"type":"v1","addr":"172.21.15.37:6815","nonce":2754669649}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.37:6812","nonce":2754669649},{"type":"v1","addr":"172.21.15.37:6813","nonce":2754669649}]},"public_addr":"172.21.15.37:6809/2754669649","cluster_addr":"172.21.15.37:6811/2754669649","heartbeat_back_addr":"172.21.15.37:6815/2754669649","heartbeat_front_addr":"172.21.15.37:6813/2754669649","state":["exists","up"]},{"osd":3,"uuid":"1ba4a746-1608-47a7-ace2-960a5cfae934","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":21,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.37:6824","nonce":1815290618},{"type":"v1","addr":"172.21.15.37:6825","nonce":1815290618}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.37:6826","nonce":1815290618},{"type":"v1","addr":"172.21.15.37:6827","nonce":1815290618}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.37:6830","nonce":1815290618},{"type":"v1","addr":"172.21.15.37:6831","nonce":1815290618}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.37:6828","nonce":1815290618},{"type":"v1","addr":"172.21.15.37:6829","nonce":1815290618}]},"public_addr":"172.21.15.37:6825/1815290618","cluster_addr":"172.21.15.37:6827/1815290618","heartbeat_back_addr":"172.21.15.37:6831/1815290618","heartbeat_front_addr":"172.21.15.37:6829/1815290618","state":["exists","up"]},{"osd":4,"uuid":"d4f59f5d-0413-4b79-87ee-bb212c1fcf62","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":21,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6800","nonce":4051532063},{"type":"v1","addr":"172.21.15.203:6801","nonce":4051532063}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6802","nonce":4051532063},{"type":"v1","addr":"172.21.15.203:6803","nonce":4051532063}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6806","nonce":4051532063},{"type":"v1","addr":"172.21.15.203:6807","nonce":4051532063}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6804","nonce":4051532063},{"type":"v1","addr":"172.21.15.203:6805","nonce":4051532063}]},"public_addr":"172.21.15.203:6801/4051532063","cluster_addr":"172.21.15.203:6803/4051532063","heartbeat_back_addr":"172.21.15.203:6807/4051532063","heartbeat_front_addr":"172.21.15.203:6805/4051532063","state":["exists","up"]},{"osd":5,"uuid":"69710a0e-41d8-4e7f-9a10-17ce540b67be","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":21,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6824","nonce":1203822890},{"type":"v1","addr":"172.21.15.203:6825","nonce":1203822890}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6826","nonce":1203822890},{"type":"v1","addr":"172.21.15.203:6827","nonce":1203822890}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6830","nonce":1203822890},{"type":"v1","addr":"172.21.15.203:6831","nonce":1203822890}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6828","nonce":1203822890},{"type":"v1","addr":"172.21.15.203:6829","nonce":1203822890}]},"public_addr":"172.21.15.203:6825/1203822890","cluster_addr":"172.21.15.203:6827/1203822890","heartbeat_back_addr":"172.21.15.203:6831/1203822890","heartbeat_front_addr":"172.21.15.203:6829/1203822890","state":["exists","up"]},{"osd":6,"uuid":"f29dd047-3b48-473d-a235-6642a6293f20","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":21,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6810","nonce":2309615014},{"type":"v1","addr":"172.21.15.203:6812","nonce":2309615014}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6814","nonce":2309615014},{"type":"v1","addr":"172.21.15.203:6816","nonce":2309615014}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6822","nonce":2309615014},{"type":"v1","addr":"172.21.15.203:6823","nonce":2309615014}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6818","nonce":2309615014},{"type":"v1","addr":"172.21.15.203:6820","nonce":2309615014}]},"public_addr":"172.21.15.203:6812/2309615014","cluster_addr":"172.21.15.203:6816/2309615014","heartbeat_back_addr":"172.21.15.203:6823/2309615014","heartbeat_front_addr":"172.21.15.203:6820/2309615014","state":["exists","up"]},{"osd":7,"uuid":"6e691644-378b-4fdc-8e86-fb5f56e31f29","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":21,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6808","nonce":3688099014},{"type":"v1","addr":"172.21.15.203:6809","nonce":3688099014}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6811","nonce":3688099014},{"type":"v1","addr":"172.21.15.203:6813","nonce":3688099014}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6819","nonce":3688099014},{"type":"v1","addr":"172.21.15.203:6821","nonce":3688099014}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6815","nonce":3688099014},{"type":"v1","addr":"172.21.15.203:6817","nonce":3688099014}]},"public_addr":"172.21.15.203:6809/3688099014","cluster_addr":"172.21.15.203:6813/3688099014","heartbeat_back_addr":"172.21.15.203:6821/3688099014","heartbeat_front_addr":"172.21.15.203:6817/3688099014","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T10:28:51.419141+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T10:28:51.366448+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T10:28:51.348276+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T10:28:51.380519+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T10:28:51.105812+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T10:28:51.384496+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T10:28:51.388755+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T10:28:51.366958+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-04-25T10:29:41.658 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:41.655+0000 7f6e65e9f640 1 -- 172.21.15.37:0/4036484719 >> [v2:172.21.15.203:6832/2418205237,v1:172.21.15.203:6833/2418205237] conn(0x7f6e4c0744e0 msgr2=0x7f6e4c0769f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:29:41.658 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:41.655+0000 7f6e65e9f640 1 --2- 172.21.15.37:0/4036484719 >> [v2:172.21.15.203:6832/2418205237,v1:172.21.15.203:6833/2418205237] conn(0x7f6e4c0744e0 0x7f6e4c0769f0 secure :-1 s=READY pgs=133 cs=0 l=1 rev1=1 crypto rx=0x7f6e6008e110 tx=0x7f6e50008040 comp rx=0 tx=0).stop 2024-04-25T10:29:41.658 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:41.655+0000 7f6e65e9f640 1 -- 172.21.15.37:0/4036484719 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f6e60160f40 msgr2=0x7f6e600cd910 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:29:41.658 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:41.655+0000 7f6e65e9f640 1 --2- 172.21.15.37:0/4036484719 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f6e60160f40 0x7f6e600cd910 secure :-1 s=READY pgs=237 cs=0 l=1 rev1=1 crypto rx=0x7f6e5800ba10 tx=0x7f6e5800bed0 comp rx=0 tx=0).stop 2024-04-25T10:29:41.658 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:41.655+0000 7f6e65e9f640 1 -- 172.21.15.37:0/4036484719 shutdown_connections 2024-04-25T10:29:41.658 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:41.655+0000 7f6e65e9f640 1 --2- 172.21.15.37:0/4036484719 >> [v2:172.21.15.203:6832/2418205237,v1:172.21.15.203:6833/2418205237] conn(0x7f6e4c0744e0 0x7f6e4c0769f0 unknown :-1 s=CLOSED pgs=133 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:29:41.658 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:41.655+0000 7f6e65e9f640 1 --2- 172.21.15.37:0/4036484719 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f6e60160f40 0x7f6e600cd910 unknown :-1 s=CLOSED pgs=237 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:29:41.658 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:41.655+0000 7f6e65e9f640 1 --2- 172.21.15.37:0/4036484719 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f6e600d5430 0x7f6e601f97a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:29:41.658 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:41.655+0000 7f6e65e9f640 1 --2- 172.21.15.37:0/4036484719 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f6e600b7b00 0x7f6e600cc010 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:29:41.659 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:41.655+0000 7f6e65e9f640 1 -- 172.21.15.37:0/4036484719 >> 172.21.15.37:0/4036484719 conn(0x7f6e600d27a0 msgr2=0x7f6e600d1060 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:29:41.659 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:41.656+0000 7f6e65e9f640 1 -- 172.21.15.37:0/4036484719 shutdown_connections 2024-04-25T10:29:41.659 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:41.656+0000 7f6e65e9f640 1 -- 172.21.15.37:0/4036484719 wait complete. 2024-04-25T10:29:41.669 INFO:tasks.ceph_test_case.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2024-04-25T10:28:56.202153+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '19', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 8, 'score_stable': 8, 'optimal_score': 0.25, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 2, 'pool_name': 'cephfs_metadata', 'create_time': '2024-04-25T10:29:06.507244+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '22', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_autoscale_bias': 4, 'pg_num_min': 16, 'recovery_priority': 5}, 'application_metadata': {'cephfs': {'metadata': 'cephfs'}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 2.25, 'score_stable': 2.25, 'optimal_score': 1, 'raw_score_acting': 2.25, 'raw_score_stable': 2.25, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 3, 'pool_name': 'cephfs_data', 'create_time': '2024-04-25T10:29:07.347844+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 64, 'pg_placement_num': 64, 'pg_placement_num_target': 64, 'pg_num_target': 64, 'pg_num_pending': 64, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '23', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.8999999761581421}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 1.5, 'score_stable': 1.5, 'optimal_score': 1, 'raw_score_acting': 1.5, 'raw_score_stable': 1.5, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}] 2024-04-25T10:29:41.670 DEBUG:teuthology.orchestra.run.smithi037:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2024-04-25T10:29:41.784 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:41.781+0000 7f0dbd36c640 1 Processor -- start 2024-04-25T10:29:41.784 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:41.781+0000 7f0dbd36c640 1 -- start start 2024-04-25T10:29:41.786 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:41.783+0000 7f0dbd36c640 1 --2- >> v2:172.21.15.203:3300/0 conn(0x7f0db80d5160 0x7f0db80b4e80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:29:41.786 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:41.783+0000 7f0dbd36c640 1 --2- >> v2:172.21.15.37:3300/0 conn(0x7f0db80d0990 0x7f0db80d2b20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:29:41.786 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:41.783+0000 7f0dbd36c640 1 --2- >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f0db80d0da0 0x7f0db80d3060 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:29:41.786 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:41.783+0000 7f0dbd36c640 1 -- --> v2:172.21.15.37:3300/0 -- mon_getmap magic: 0 -- 0x7f0db80d1650 con 0x7f0db80d0990 2024-04-25T10:29:41.786 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:41.783+0000 7f0dbd36c640 1 -- --> v2:172.21.15.203:3300/0 -- mon_getmap magic: 0 -- 0x7f0db80d1370 con 0x7f0db80d5160 2024-04-25T10:29:41.787 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:41.783+0000 7f0dbd36c640 1 -- --> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] -- mon_getmap magic: 0 -- 0x7f0db80d4680 con 0x7f0db80d0da0 2024-04-25T10:29:41.787 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:41.783+0000 7f0db67fc640 1 --2- >> v2:172.21.15.37:3300/0 conn(0x7f0db80d0990 0x7f0db80d2b20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:29:41.787 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:41.783+0000 7f0db67fc640 1 --2- >> v2:172.21.15.37:3300/0 conn(0x7f0db80d0990 0x7f0db80d2b20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.37:3300/0 says I am v2:172.21.15.37:59386/0 (socket says 172.21.15.37:59386) 2024-04-25T10:29:41.787 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:41.783+0000 7f0db67fc640 1 -- 172.21.15.37:0/4270566971 learned_addr learned my addr 172.21.15.37:0/4270566971 (peer_addr_for_me v2:172.21.15.37:0/0) 2024-04-25T10:29:41.787 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:41.783+0000 7f0db77fe640 1 --2- 172.21.15.37:0/4270566971 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f0db80d0da0 0x7f0db80d3060 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:29:41.787 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:41.783+0000 7f0db6ffd640 1 --2- 172.21.15.37:0/4270566971 >> v2:172.21.15.203:3300/0 conn(0x7f0db80d5160 0x7f0db80b4e80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:29:41.787 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:41.783+0000 7f0db67fc640 1 -- 172.21.15.37:0/4270566971 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f0db80d0da0 msgr2=0x7f0db80d3060 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:29:41.787 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:41.783+0000 7f0db67fc640 1 --2- 172.21.15.37:0/4270566971 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f0db80d0da0 0x7f0db80d3060 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:29:41.787 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:41.783+0000 7f0db67fc640 1 -- 172.21.15.37:0/4270566971 >> v2:172.21.15.203:3300/0 conn(0x7f0db80d5160 msgr2=0x7f0db80b4e80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:29:41.787 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:41.783+0000 7f0db67fc640 1 --2- 172.21.15.37:0/4270566971 >> v2:172.21.15.203:3300/0 conn(0x7f0db80d5160 0x7f0db80b4e80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:29:41.787 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:41.783+0000 7f0db67fc640 1 -- 172.21.15.37:0/4270566971 --> v2:172.21.15.37:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f0db80d4960 con 0x7f0db80d0990 2024-04-25T10:29:41.787 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:41.784+0000 7f0db67fc640 1 --2- 172.21.15.37:0/4270566971 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f0db80d0990 0x7f0db80d2b20 secure :-1 s=READY pgs=238 cs=0 l=1 rev1=1 crypto rx=0x7f0db80b86b0 tx=0x7f0da002a0b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d667678463e02637 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:29:41.787 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:41.784+0000 7f0db5ffb640 1 -- 172.21.15.37:0/4270566971 <== mon.0 v2:172.21.15.37:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0da0037070 con 0x7f0db80d0990 2024-04-25T10:29:41.787 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:41.784+0000 7f0db5ffb640 1 -- 172.21.15.37:0/4270566971 <== mon.0 v2:172.21.15.37:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f0da002ae00 con 0x7f0db80d0990 2024-04-25T10:29:41.788 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:41.784+0000 7f0db5ffb640 1 -- 172.21.15.37:0/4270566971 <== mon.0 v2:172.21.15.37:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0da0032660 con 0x7f0db80d0990 2024-04-25T10:29:41.788 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:41.785+0000 7f0dbd36c640 1 -- 172.21.15.37:0/4270566971 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f0db80d0990 msgr2=0x7f0db80d2b20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:29:41.788 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:41.785+0000 7f0dbd36c640 1 --2- 172.21.15.37:0/4270566971 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f0db80d0990 0x7f0db80d2b20 secure :-1 s=READY pgs=238 cs=0 l=1 rev1=1 crypto rx=0x7f0db80b86b0 tx=0x7f0da002a0b0 comp rx=0 tx=0).stop 2024-04-25T10:29:41.789 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:41.785+0000 7f0dbd36c640 1 -- 172.21.15.37:0/4270566971 shutdown_connections 2024-04-25T10:29:41.789 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:41.785+0000 7f0dbd36c640 1 --2- 172.21.15.37:0/4270566971 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f0db80d0990 0x7f0db80d2b20 unknown :-1 s=CLOSED pgs=238 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:29:41.789 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:41.785+0000 7f0dbd36c640 1 --2- 172.21.15.37:0/4270566971 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f0db80d0da0 0x7f0db80d3060 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:29:41.789 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:41.785+0000 7f0dbd36c640 1 --2- 172.21.15.37:0/4270566971 >> v2:172.21.15.203:3300/0 conn(0x7f0db80d5160 0x7f0db80b4e80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:29:41.789 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:41.785+0000 7f0dbd36c640 1 -- 172.21.15.37:0/4270566971 >> 172.21.15.37:0/4270566971 conn(0x7f0db80b7a80 msgr2=0x7f0db80b7e90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:29:41.789 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:41.785+0000 7f0dbd36c640 1 -- 172.21.15.37:0/4270566971 shutdown_connections 2024-04-25T10:29:41.789 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:41.785+0000 7f0dbd36c640 1 -- 172.21.15.37:0/4270566971 wait complete. 2024-04-25T10:29:41.789 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:41.787+0000 7f0dbd36c640 1 Processor -- start 2024-04-25T10:29:41.790 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:41.787+0000 7f0dbd36c640 1 -- start start 2024-04-25T10:29:41.791 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:41.788+0000 7f0dbd36c640 1 --2- >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f0db80d0990 0x7f0db8154b00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:29:41.791 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:41.788+0000 7f0dbd36c640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f0db80d0da0 0x7f0db815bb30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:29:41.792 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:41.788+0000 7f0dbd36c640 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f0db80d5160 0x7f0db815d870 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:29:41.792 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:41.788+0000 7f0dbd36c640 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 -- 0x7f0db8164c40 con 0x7f0db80d0da0 2024-04-25T10:29:41.792 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:41.788+0000 7f0dbd36c640 1 -- --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_getmap magic: 0 -- 0x7f0db8164ac0 con 0x7f0db80d5160 2024-04-25T10:29:41.792 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:41.788+0000 7f0dbd36c640 1 -- --> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] -- mon_getmap magic: 0 -- 0x7f0db806db10 con 0x7f0db80d0990 2024-04-25T10:29:41.792 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:41.788+0000 7f0db67fc640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f0db80d0da0 0x7f0db815bb30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:29:41.792 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:41.788+0000 7f0db67fc640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f0db80d0da0 0x7f0db815bb30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.37:3300/0 says I am v2:172.21.15.37:59398/0 (socket says 172.21.15.37:59398) 2024-04-25T10:29:41.792 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:41.788+0000 7f0db67fc640 1 -- 172.21.15.37:0/4253686305 learned_addr learned my addr 172.21.15.37:0/4253686305 (peer_addr_for_me v2:172.21.15.37:0/0) 2024-04-25T10:29:41.792 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:41.788+0000 7f0db77fe640 1 --2- 172.21.15.37:0/4253686305 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f0db80d5160 0x7f0db815d870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:29:41.792 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:41.788+0000 7f0db6ffd640 1 --2- 172.21.15.37:0/4253686305 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f0db80d0990 0x7f0db8154b00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:29:41.792 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:41.789+0000 7f0db67fc640 1 -- 172.21.15.37:0/4253686305 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f0db80d0990 msgr2=0x7f0db8154b00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:29:41.792 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:41.789+0000 7f0db67fc640 1 --2- 172.21.15.37:0/4253686305 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f0db80d0990 0x7f0db8154b00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:29:41.792 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:41.789+0000 7f0db67fc640 1 -- 172.21.15.37:0/4253686305 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f0db80d5160 msgr2=0x7f0db815d870 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:29:41.792 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:41.789+0000 7f0db67fc640 1 --2- 172.21.15.37:0/4253686305 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f0db80d5160 0x7f0db815d870 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:29:41.793 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:41.789+0000 7f0db67fc640 1 -- 172.21.15.37:0/4253686305 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f0db80d0480 con 0x7f0db80d0da0 2024-04-25T10:29:41.793 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:41.789+0000 7f0db77fe640 1 --2- 172.21.15.37:0/4253686305 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f0db80d5160 0x7f0db815d870 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T10:29:41.793 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:41.789+0000 7f0db6ffd640 1 --2- 172.21.15.37:0/4253686305 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f0db80d0990 0x7f0db8154b00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T10:29:41.793 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:41.789+0000 7f0db67fc640 1 --2- 172.21.15.37:0/4253686305 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f0db80d0da0 0x7f0db815bb30 secure :-1 s=READY pgs=239 cs=0 l=1 rev1=1 crypto rx=0x7f0da0000c00 tx=0x7f0da00046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:29:41.793 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:41.789+0000 7f0daf7fe640 1 -- 172.21.15.37:0/4253686305 <== mon.0 v2:172.21.15.37:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0da0041030 con 0x7f0db80d0da0 2024-04-25T10:29:41.793 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:41.789+0000 7f0daf7fe640 1 -- 172.21.15.37:0/4253686305 <== mon.0 v2:172.21.15.37:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f0da002ac00 con 0x7f0db80d0da0 2024-04-25T10:29:41.793 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:41.789+0000 7f0dbd36c640 1 -- 172.21.15.37:0/4253686305 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f0db80d0760 con 0x7f0db80d0da0 2024-04-25T10:29:41.793 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:41.789+0000 7f0daf7fe640 1 -- 172.21.15.37:0/4253686305 <== mon.0 v2:172.21.15.37:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0da0037040 con 0x7f0db80d0da0 2024-04-25T10:29:41.793 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:41.789+0000 7f0dbd36c640 1 -- 172.21.15.37:0/4253686305 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f0db8160410 con 0x7f0db80d0da0 2024-04-25T10:29:41.794 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:41.790+0000 7f0dbd36c640 1 -- 172.21.15.37:0/4253686305 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f0db806a140 con 0x7f0db80d0da0 2024-04-25T10:29:41.794 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:41.791+0000 7f0daf7fe640 1 -- 172.21.15.37:0/4253686305 <== mon.0 v2:172.21.15.37:3300/0 4 ==== mgrmap(e 5) ==== 96782+0+0 (secure 0 0 0) 0x7f0da0044070 con 0x7f0db80d0da0 2024-04-25T10:29:41.794 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:41.791+0000 7f0daf7fe640 1 --2- 172.21.15.37:0/4253686305 >> [v2:172.21.15.203:6832/2418205237,v1:172.21.15.203:6833/2418205237] conn(0x7f0d980744b0 0x7f0d980769c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:29:41.794 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:41.791+0000 7f0daf7fe640 1 -- 172.21.15.37:0/4253686305 <== mon.0 v2:172.21.15.37:3300/0 5 ==== osd_map(23..23 src has 1..23) ==== 5709+0+0 (secure 0 0 0) 0x7f0da00b5630 con 0x7f0db80d0da0 2024-04-25T10:29:41.794 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:41.791+0000 7f0db6ffd640 1 --2- 172.21.15.37:0/4253686305 >> [v2:172.21.15.203:6832/2418205237,v1:172.21.15.203:6833/2418205237] conn(0x7f0d980744b0 0x7f0d980769c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:29:41.794 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:41.792+0000 7f0db6ffd640 1 --2- 172.21.15.37:0/4253686305 >> [v2:172.21.15.203:6832/2418205237,v1:172.21.15.203:6833/2418205237] conn(0x7f0d980744b0 0x7f0d980769c0 secure :-1 s=READY pgs=134 cs=0 l=1 rev1=1 crypto rx=0x7f0db0006fd0 tx=0x7f0db0008040 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:29:41.798 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:41.796+0000 7f0daf7fe640 1 -- 172.21.15.37:0/4253686305 <== mon.0 v2:172.21.15.37:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187246 (secure 0 0 0) 0x7f0da007fdc0 con 0x7f0db80d0da0 2024-04-25T10:29:41.982 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:41.980+0000 7f0dbd36c640 1 -- 172.21.15.37:0/4253686305 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) -- 0x7f0db80d3f50 con 0x7f0db80d0da0 2024-04-25T10:29:41.982 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:41.980+0000 7f0daf7fe640 1 -- 172.21.15.37:0/4253686305 <== mon.0 v2:172.21.15.37:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v23) ==== 93+0+10 (secure 0 0 0) 0x7f0da007f6c0 con 0x7f0db80d0da0 2024-04-25T10:29:41.983 INFO:teuthology.orchestra.run.smithi037.stdout:pg_num: 1 2024-04-25T10:29:41.984 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:41.982+0000 7f0dbd36c640 1 -- 172.21.15.37:0/4253686305 >> [v2:172.21.15.203:6832/2418205237,v1:172.21.15.203:6833/2418205237] conn(0x7f0d980744b0 msgr2=0x7f0d980769c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:29:41.985 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:41.982+0000 7f0dbd36c640 1 --2- 172.21.15.37:0/4253686305 >> [v2:172.21.15.203:6832/2418205237,v1:172.21.15.203:6833/2418205237] conn(0x7f0d980744b0 0x7f0d980769c0 secure :-1 s=READY pgs=134 cs=0 l=1 rev1=1 crypto rx=0x7f0db0006fd0 tx=0x7f0db0008040 comp rx=0 tx=0).stop 2024-04-25T10:29:41.985 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:41.982+0000 7f0dbd36c640 1 -- 172.21.15.37:0/4253686305 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f0db80d0da0 msgr2=0x7f0db815bb30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:29:41.985 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:41.982+0000 7f0dbd36c640 1 --2- 172.21.15.37:0/4253686305 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f0db80d0da0 0x7f0db815bb30 secure :-1 s=READY pgs=239 cs=0 l=1 rev1=1 crypto rx=0x7f0da0000c00 tx=0x7f0da00046c0 comp rx=0 tx=0).stop 2024-04-25T10:29:41.985 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:41.982+0000 7f0dbd36c640 1 -- 172.21.15.37:0/4253686305 shutdown_connections 2024-04-25T10:29:41.985 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:41.982+0000 7f0dbd36c640 1 --2- 172.21.15.37:0/4253686305 >> [v2:172.21.15.203:6832/2418205237,v1:172.21.15.203:6833/2418205237] conn(0x7f0d980744b0 0x7f0d980769c0 unknown :-1 s=CLOSED pgs=134 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:29:41.985 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:41.982+0000 7f0dbd36c640 1 --2- 172.21.15.37:0/4253686305 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f0db80d5160 0x7f0db815d870 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:29:41.985 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:41.982+0000 7f0dbd36c640 1 --2- 172.21.15.37:0/4253686305 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f0db80d0da0 0x7f0db815bb30 unknown :-1 s=CLOSED pgs=239 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:29:41.985 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:41.982+0000 7f0dbd36c640 1 --2- 172.21.15.37:0/4253686305 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f0db80d0990 0x7f0db8154b00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:29:41.985 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:41.982+0000 7f0dbd36c640 1 -- 172.21.15.37:0/4253686305 >> 172.21.15.37:0/4253686305 conn(0x7f0db80b7a80 msgr2=0x7f0db80b54d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:29:41.985 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:41.982+0000 7f0dbd36c640 1 -- 172.21.15.37:0/4253686305 shutdown_connections 2024-04-25T10:29:41.985 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:41.982+0000 7f0dbd36c640 1 -- 172.21.15.37:0/4253686305 wait complete. 2024-04-25T10:29:41.996 DEBUG:teuthology.orchestra.run.smithi037:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_metadata pg_num 2024-04-25T10:29:42.110 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:42.107+0000 7ff33cfab640 1 Processor -- start 2024-04-25T10:29:42.110 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:42.107+0000 7ff33cfab640 1 -- start start 2024-04-25T10:29:42.112 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:42.109+0000 7ff33cfab640 1 --2- >> v2:172.21.15.203:3300/0 conn(0x7ff3380b80e0 0x7ff3380d0920 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:29:42.112 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:42.109+0000 7ff33cfab640 1 --2- >> v2:172.21.15.37:3300/0 conn(0x7ff3380d0e60 0x7ff33815f7c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:29:42.112 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:42.109+0000 7ff33cfab640 1 -- --> v1:172.21.15.203:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7ff3380d15e0 con 0x7ff3380b50e0 2024-04-25T10:29:42.112 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:42.109+0000 7ff33cfab640 1 -- --> v2:172.21.15.37:3300/0 -- mon_getmap magic: 0 -- 0x7ff3381646e0 con 0x7ff3380d0e60 2024-04-25T10:29:42.112 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:42.109+0000 7ff33cfab640 1 -- --> v2:172.21.15.203:3300/0 -- mon_getmap magic: 0 -- 0x7ff338163db0 con 0x7ff3380b80e0 2024-04-25T10:29:42.112 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:42.109+0000 7ff336d9d640 1 --2- >> v2:172.21.15.37:3300/0 conn(0x7ff3380d0e60 0x7ff33815f7c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:29:42.112 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:42.109+0000 7ff336d9d640 1 --2- >> v2:172.21.15.37:3300/0 conn(0x7ff3380d0e60 0x7ff33815f7c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.37:3300/0 says I am v2:172.21.15.37:59408/0 (socket says 172.21.15.37:59408) 2024-04-25T10:29:42.112 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:42.109+0000 7ff336d9d640 1 -- 172.21.15.37:0/284996621 learned_addr learned my addr 172.21.15.37:0/284996621 (peer_addr_for_me v2:172.21.15.37:0/0) 2024-04-25T10:29:42.112 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:42.109+0000 7ff335d9b640 1 --2- 172.21.15.37:0/284996621 >> v2:172.21.15.203:3300/0 conn(0x7ff3380b80e0 0x7ff3380d0920 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:29:42.112 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:42.109+0000 7ff336d9d640 1 -- 172.21.15.37:0/284996621 >> v2:172.21.15.203:3300/0 conn(0x7ff3380b80e0 msgr2=0x7ff3380d0920 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:29:42.112 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:42.109+0000 7ff336d9d640 1 --2- 172.21.15.37:0/284996621 >> v2:172.21.15.203:3300/0 conn(0x7ff3380b80e0 0x7ff3380d0920 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:29:42.113 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:42.109+0000 7ff336d9d640 1 -- 172.21.15.37:0/284996621 >> v1:172.21.15.203:6789/0 conn(0x7ff3380b50e0 legacy=0x7ff3380b79d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:29:42.113 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:42.109+0000 7ff336d9d640 1 -- 172.21.15.37:0/284996621 --> v2:172.21.15.37:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ff338164090 con 0x7ff3380d0e60 2024-04-25T10:29:42.114 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:42.110+0000 7ff336d9d640 1 --2- 172.21.15.37:0/284996621 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7ff3380d0e60 0x7ff33815f7c0 secure :-1 s=READY pgs=240 cs=0 l=1 rev1=1 crypto rx=0x7ff338164860 tx=0x7ff32802a290 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e0a218816168d0d7 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:29:42.114 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:42.110+0000 7ff33559a640 1 -- 172.21.15.37:0/284996621 <== mon.0 v2:172.21.15.37:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff328037070 con 0x7ff3380d0e60 2024-04-25T10:29:42.114 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:42.110+0000 7ff33559a640 1 -- 172.21.15.37:0/284996621 <== mon.0 v2:172.21.15.37:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7ff328032440 con 0x7ff3380d0e60 2024-04-25T10:29:42.114 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:42.110+0000 7ff33559a640 1 -- 172.21.15.37:0/284996621 <== mon.0 v2:172.21.15.37:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff328032810 con 0x7ff3380d0e60 2024-04-25T10:29:42.114 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:42.111+0000 7ff33cfab640 1 -- 172.21.15.37:0/284996621 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7ff3380d0e60 msgr2=0x7ff33815f7c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:29:42.114 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:42.111+0000 7ff33cfab640 1 --2- 172.21.15.37:0/284996621 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7ff3380d0e60 0x7ff33815f7c0 secure :-1 s=READY pgs=240 cs=0 l=1 rev1=1 crypto rx=0x7ff338164860 tx=0x7ff32802a290 comp rx=0 tx=0).stop 2024-04-25T10:29:42.114 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:42.111+0000 7ff33cfab640 1 -- 172.21.15.37:0/284996621 shutdown_connections 2024-04-25T10:29:42.114 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:42.111+0000 7ff33cfab640 1 --2- 172.21.15.37:0/284996621 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7ff3380d0e60 0x7ff33815f7c0 unknown :-1 s=CLOSED pgs=240 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:29:42.114 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:42.111+0000 7ff33cfab640 1 --2- 172.21.15.37:0/284996621 >> v2:172.21.15.203:3300/0 conn(0x7ff3380b80e0 0x7ff3380d0920 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:29:42.115 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:42.111+0000 7ff33cfab640 1 -- 172.21.15.37:0/284996621 >> 172.21.15.37:0/284996621 conn(0x7ff3380d5d50 msgr2=0x7ff3380d6160 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:29:42.115 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:42.111+0000 7ff33cfab640 1 -- 172.21.15.37:0/284996621 shutdown_connections 2024-04-25T10:29:42.115 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:42.111+0000 7ff33cfab640 1 -- 172.21.15.37:0/284996621 wait complete. 2024-04-25T10:29:42.115 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:42.113+0000 7ff33cfab640 1 Processor -- start 2024-04-25T10:29:42.115 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:42.113+0000 7ff33cfab640 1 -- start start 2024-04-25T10:29:42.118 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:42.114+0000 7ff33cfab640 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7ff3380b50e0 0x7ff33814feb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:29:42.118 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:42.114+0000 7ff33cfab640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7ff3380d0e60 0x7ff3380ce350 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:29:42.118 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:42.114+0000 7ff33cfab640 1 --2- >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7ff3380c95f0 0x7ff3380ce890 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:29:42.119 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:42.115+0000 7ff33cfab640 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 -- 0x7ff3380d1270 con 0x7ff3380d0e60 2024-04-25T10:29:42.119 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:42.115+0000 7ff33cfab640 1 -- --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_getmap magic: 0 -- 0x7ff338164b70 con 0x7ff3380b50e0 2024-04-25T10:29:42.119 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:42.115+0000 7ff33cfab640 1 -- --> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] -- mon_getmap magic: 0 -- 0x7ff33815fd00 con 0x7ff3380c95f0 2024-04-25T10:29:42.119 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:42.115+0000 7ff335d9b640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7ff3380d0e60 0x7ff3380ce350 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:29:42.119 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:42.115+0000 7ff335d9b640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7ff3380d0e60 0x7ff3380ce350 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.37:3300/0 says I am v2:172.21.15.37:59422/0 (socket says 172.21.15.37:59422) 2024-04-25T10:29:42.119 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:42.115+0000 7ff335d9b640 1 -- 172.21.15.37:0/3988251146 learned_addr learned my addr 172.21.15.37:0/3988251146 (peer_addr_for_me v2:172.21.15.37:0/0) 2024-04-25T10:29:42.119 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:42.115+0000 7ff336d9d640 1 --2- 172.21.15.37:0/3988251146 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7ff3380c95f0 0x7ff3380ce890 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:29:42.119 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:42.115+0000 7ff33659c640 1 --2- 172.21.15.37:0/3988251146 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7ff3380b50e0 0x7ff33814feb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:29:42.119 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:42.115+0000 7ff335d9b640 1 -- 172.21.15.37:0/3988251146 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7ff3380c95f0 msgr2=0x7ff3380ce890 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:29:42.119 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:42.115+0000 7ff335d9b640 1 --2- 172.21.15.37:0/3988251146 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7ff3380c95f0 0x7ff3380ce890 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:29:42.119 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:42.115+0000 7ff335d9b640 1 -- 172.21.15.37:0/3988251146 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7ff3380b50e0 msgr2=0x7ff33814feb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:29:42.119 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:42.115+0000 7ff335d9b640 1 --2- 172.21.15.37:0/3988251146 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7ff3380b50e0 0x7ff33814feb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:29:42.119 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:42.115+0000 7ff335d9b640 1 -- 172.21.15.37:0/3988251146 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ff3380cc4c0 con 0x7ff3380d0e60 2024-04-25T10:29:42.119 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:42.115+0000 7ff33659c640 1 --2- 172.21.15.37:0/3988251146 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7ff3380b50e0 0x7ff33814feb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T10:29:42.120 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:42.115+0000 7ff335d9b640 1 --2- 172.21.15.37:0/3988251146 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7ff3380d0e60 0x7ff3380ce350 secure :-1 s=READY pgs=241 cs=0 l=1 rev1=1 crypto rx=0x7ff32000d5c0 tx=0x7ff32000da80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:29:42.120 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:42.115+0000 7ff32effd640 1 -- 172.21.15.37:0/3988251146 <== mon.0 v2:172.21.15.37:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff3200125f0 con 0x7ff3380d0e60 2024-04-25T10:29:42.120 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:42.115+0000 7ff32effd640 1 -- 172.21.15.37:0/3988251146 <== mon.0 v2:172.21.15.37:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7ff320019070 con 0x7ff3380d0e60 2024-04-25T10:29:42.120 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:42.115+0000 7ff33cfab640 1 -- 172.21.15.37:0/3988251146 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7ff338161b90 con 0x7ff3380d0e60 2024-04-25T10:29:42.120 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:42.115+0000 7ff32effd640 1 -- 172.21.15.37:0/3988251146 <== mon.0 v2:172.21.15.37:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff3200129d0 con 0x7ff3380d0e60 2024-04-25T10:29:42.120 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:42.116+0000 7ff33cfab640 1 -- 172.21.15.37:0/3988251146 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7ff338162150 con 0x7ff3380d0e60 2024-04-25T10:29:42.120 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:42.116+0000 7ff33cfab640 1 -- 172.21.15.37:0/3988251146 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7ff338161900 con 0x7ff3380d0e60 2024-04-25T10:29:42.121 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:42.117+0000 7ff32effd640 1 -- 172.21.15.37:0/3988251146 <== mon.0 v2:172.21.15.37:3300/0 4 ==== mgrmap(e 5) ==== 96782+0+0 (secure 0 0 0) 0x7ff320012b70 con 0x7ff3380d0e60 2024-04-25T10:29:42.121 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:42.118+0000 7ff32effd640 1 --2- 172.21.15.37:0/3988251146 >> [v2:172.21.15.203:6832/2418205237,v1:172.21.15.203:6833/2418205237] conn(0x7ff3180744e0 0x7ff3180769f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:29:42.121 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:42.118+0000 7ff32effd640 1 -- 172.21.15.37:0/3988251146 <== mon.0 v2:172.21.15.37:3300/0 5 ==== osd_map(23..23 src has 1..23) ==== 5709+0+0 (secure 0 0 0) 0x7ff3200955a0 con 0x7ff3380d0e60 2024-04-25T10:29:42.121 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:42.118+0000 7ff33659c640 1 --2- 172.21.15.37:0/3988251146 >> [v2:172.21.15.203:6832/2418205237,v1:172.21.15.203:6833/2418205237] conn(0x7ff3180744e0 0x7ff3180769f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:29:42.121 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:42.118+0000 7ff33659c640 1 --2- 172.21.15.37:0/3988251146 >> [v2:172.21.15.203:6832/2418205237,v1:172.21.15.203:6833/2418205237] conn(0x7ff3180744e0 0x7ff3180769f0 secure :-1 s=READY pgs=135 cs=0 l=1 rev1=1 crypto rx=0x7ff3380d25c0 tx=0x7ff330009040 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:29:42.124 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:42.122+0000 7ff32effd640 1 -- 172.21.15.37:0/3988251146 <== mon.0 v2:172.21.15.37:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187246 (secure 0 0 0) 0x7ff32005fdd0 con 0x7ff3380d0e60 2024-04-25T10:29:42.310 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:42.308+0000 7ff33cfab640 1 -- 172.21.15.37:0/3988251146 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"} v 0) -- 0x7ff33806a140 con 0x7ff3380d0e60 2024-04-25T10:29:42.313 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:42.311+0000 7ff32effd640 1 -- 172.21.15.37:0/3988251146 <== mon.0 v2:172.21.15.37:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]=0 v23) ==== 104+0+11 (secure 0 0 0) 0x7ff32005f6d0 con 0x7ff3380d0e60 2024-04-25T10:29:42.313 INFO:teuthology.orchestra.run.smithi037.stdout:pg_num: 32 2024-04-25T10:29:42.316 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:42.313+0000 7ff33cfab640 1 -- 172.21.15.37:0/3988251146 >> [v2:172.21.15.203:6832/2418205237,v1:172.21.15.203:6833/2418205237] conn(0x7ff3180744e0 msgr2=0x7ff3180769f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:29:42.316 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:42.313+0000 7ff33cfab640 1 --2- 172.21.15.37:0/3988251146 >> [v2:172.21.15.203:6832/2418205237,v1:172.21.15.203:6833/2418205237] conn(0x7ff3180744e0 0x7ff3180769f0 secure :-1 s=READY pgs=135 cs=0 l=1 rev1=1 crypto rx=0x7ff3380d25c0 tx=0x7ff330009040 comp rx=0 tx=0).stop 2024-04-25T10:29:42.316 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:42.313+0000 7ff33cfab640 1 -- 172.21.15.37:0/3988251146 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7ff3380d0e60 msgr2=0x7ff3380ce350 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:29:42.316 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:42.313+0000 7ff33cfab640 1 --2- 172.21.15.37:0/3988251146 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7ff3380d0e60 0x7ff3380ce350 secure :-1 s=READY pgs=241 cs=0 l=1 rev1=1 crypto rx=0x7ff32000d5c0 tx=0x7ff32000da80 comp rx=0 tx=0).stop 2024-04-25T10:29:42.317 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:42.313+0000 7ff33cfab640 1 -- 172.21.15.37:0/3988251146 shutdown_connections 2024-04-25T10:29:42.317 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:42.313+0000 7ff33cfab640 1 --2- 172.21.15.37:0/3988251146 >> [v2:172.21.15.203:6832/2418205237,v1:172.21.15.203:6833/2418205237] conn(0x7ff3180744e0 0x7ff3180769f0 unknown :-1 s=CLOSED pgs=135 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:29:42.317 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:42.313+0000 7ff33cfab640 1 --2- 172.21.15.37:0/3988251146 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7ff3380c95f0 0x7ff3380ce890 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:29:42.317 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:42.313+0000 7ff33cfab640 1 --2- 172.21.15.37:0/3988251146 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7ff3380d0e60 0x7ff3380ce350 unknown :-1 s=CLOSED pgs=241 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:29:42.317 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:42.313+0000 7ff33cfab640 1 --2- 172.21.15.37:0/3988251146 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7ff3380b50e0 0x7ff33814feb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:29:42.317 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:42.313+0000 7ff33cfab640 1 -- 172.21.15.37:0/3988251146 >> 172.21.15.37:0/3988251146 conn(0x7ff3380d5d50 msgr2=0x7ff3380b78e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:29:42.318 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:42.313+0000 7ff33cfab640 1 -- 172.21.15.37:0/3988251146 shutdown_connections 2024-04-25T10:29:42.318 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:42.313+0000 7ff33cfab640 1 -- 172.21.15.37:0/3988251146 wait complete. 2024-04-25T10:29:42.327 DEBUG:teuthology.orchestra.run.smithi037:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data pg_num 2024-04-25T10:29:42.443 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:42.440+0000 7f83da6d8640 1 Processor -- start 2024-04-25T10:29:42.443 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:42.440+0000 7f83da6d8640 1 -- start start 2024-04-25T10:29:42.445 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:42.442+0000 7f83da6d8640 1 --2- >> v2:172.21.15.37:3300/0 conn(0x7f83d4161110 0x7f83d4161880 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:29:42.445 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:42.442+0000 7f83da6d8640 1 -- --> v1:172.21.15.37:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f83d4161520 con 0x7f83d40b7cd0 2024-04-25T10:29:42.445 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:42.442+0000 7f83da6d8640 1 -- --> v1:172.21.15.203:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f83d40d5840 con 0x7f83d40d5430 2024-04-25T10:29:42.445 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:42.442+0000 7f83da6d8640 1 -- --> v2:172.21.15.37:3300/0 -- mon_getmap magic: 0 -- 0x7f83d40d4420 con 0x7f83d4161110 2024-04-25T10:29:42.445 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:42.442+0000 7f83d8c75640 1 --2- >> v2:172.21.15.37:3300/0 conn(0x7f83d4161110 0x7f83d4161880 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:29:42.445 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:42.442+0000 7f83d37fe640 1 --1- >> v1:172.21.15.37:6789/0 conn(0x7f83d40b7cd0 0x7f83d4160a00 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.37:6789/0 says I am v1:172.21.15.37:49816/0 (socket says 172.21.15.37:49816) 2024-04-25T10:29:42.445 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:42.442+0000 7f83d37fe640 1 -- 172.21.15.37:0/1802790749 learned_addr learned my addr 172.21.15.37:0/1802790749 (peer_addr_for_me v1:172.21.15.37:0/0) 2024-04-25T10:29:42.445 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:42.442+0000 7f83d8c75640 1 -- 172.21.15.37:0/1802790749 >> v1:172.21.15.203:6789/0 conn(0x7f83d40d5430 legacy=0x7f83d40b75c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:29:42.445 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:42.442+0000 7f83d8c75640 1 -- 172.21.15.37:0/1802790749 >> v1:172.21.15.37:6789/0 conn(0x7f83d40b7cd0 legacy=0x7f83d4160a00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:29:42.445 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:42.442+0000 7f83d8c75640 1 -- 172.21.15.37:0/1802790749 --> v2:172.21.15.37:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f83d40d4670 con 0x7f83d4161110 2024-04-25T10:29:42.446 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:42.442+0000 7f83d2ffd640 1 -- 172.21.15.37:0/1802790749 <== mon.0 v1:172.21.15.37:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 4069376056 0 0) 0x7f83d4161520 con 0x7f83d40b7cd0 2024-04-25T10:29:42.446 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:42.443+0000 7f83d8c75640 1 --2- 172.21.15.37:0/1802790749 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f83d4161110 0x7f83d4161880 secure :-1 s=READY pgs=243 cs=0 l=1 rev1=1 crypto rx=0x7f83cc009d90 tx=0x7f83cc02a660 comp rx=0 tx=0).ready entity=mon.0 client_cookie=37f753ec3d710492 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:29:42.446 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:42.443+0000 7f83d2ffd640 1 -- 172.21.15.37:0/1802790749 <== mon.0 v2:172.21.15.37:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f83cc007740 con 0x7f83d4161110 2024-04-25T10:29:42.446 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:42.443+0000 7f83d2ffd640 1 -- 172.21.15.37:0/1802790749 <== mon.0 v2:172.21.15.37:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f83cc0078e0 con 0x7f83d4161110 2024-04-25T10:29:42.446 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:42.443+0000 7f83d2ffd640 1 -- 172.21.15.37:0/1802790749 <== mon.0 v2:172.21.15.37:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f83cc007cb0 con 0x7f83d4161110 2024-04-25T10:29:42.447 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:42.444+0000 7f83da6d8640 1 -- 172.21.15.37:0/1802790749 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f83d4161110 msgr2=0x7f83d4161880 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:29:42.447 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:42.444+0000 7f83da6d8640 1 --2- 172.21.15.37:0/1802790749 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f83d4161110 0x7f83d4161880 secure :-1 s=READY pgs=243 cs=0 l=1 rev1=1 crypto rx=0x7f83cc009d90 tx=0x7f83cc02a660 comp rx=0 tx=0).stop 2024-04-25T10:29:42.447 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:42.444+0000 7f83da6d8640 1 -- 172.21.15.37:0/1802790749 shutdown_connections 2024-04-25T10:29:42.447 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:42.444+0000 7f83da6d8640 1 --2- 172.21.15.37:0/1802790749 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f83d4161110 0x7f83d4161880 unknown :-1 s=CLOSED pgs=243 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:29:42.447 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:42.444+0000 7f83da6d8640 1 -- 172.21.15.37:0/1802790749 >> 172.21.15.37:0/1802790749 conn(0x7f83d40d27a0 msgr2=0x7f83d40d2bb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:29:42.447 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:42.444+0000 7f83da6d8640 1 -- 172.21.15.37:0/1802790749 shutdown_connections 2024-04-25T10:29:42.447 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:42.444+0000 7f83da6d8640 1 -- 172.21.15.37:0/1802790749 wait complete. 2024-04-25T10:29:42.448 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:42.445+0000 7f83da6d8640 1 Processor -- start 2024-04-25T10:29:42.448 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:42.446+0000 7f83da6d8640 1 -- start start 2024-04-25T10:29:42.452 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:42.447+0000 7f83da6d8640 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f83d40d5430 0x7f83d40ce6d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:29:42.452 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:42.447+0000 7f83da6d8640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f83d4161110 0x7f83d40cd8f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:29:42.452 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:42.447+0000 7f83da6d8640 1 --2- >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f83d415df40 0x7f83d41e9af0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:29:42.452 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:42.447+0000 7f83da6d8640 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 -- 0x7f83d40d4f10 con 0x7f83d4161110 2024-04-25T10:29:42.452 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:42.447+0000 7f83da6d8640 1 -- --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_getmap magic: 0 -- 0x7f83d40d4c30 con 0x7f83d40d5430 2024-04-25T10:29:42.452 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:42.447+0000 7f83da6d8640 1 -- --> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] -- mon_getmap magic: 0 -- 0x7f83d4160380 con 0x7f83d415df40 2024-04-25T10:29:42.452 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:42.447+0000 7f83d37fe640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f83d4161110 0x7f83d40cd8f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:29:42.452 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:42.447+0000 7f83d37fe640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f83d4161110 0x7f83d40cd8f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.37:3300/0 says I am v2:172.21.15.37:59442/0 (socket says 172.21.15.37:59442) 2024-04-25T10:29:42.452 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:42.447+0000 7f83d37fe640 1 -- 172.21.15.37:0/1034130014 learned_addr learned my addr 172.21.15.37:0/1034130014 (peer_addr_for_me v2:172.21.15.37:0/0) 2024-04-25T10:29:42.452 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:42.447+0000 7f83d8c75640 1 --2- 172.21.15.37:0/1034130014 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f83d415df40 0x7f83d41e9af0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:29:42.452 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:42.447+0000 7f83d37fe640 1 -- 172.21.15.37:0/1034130014 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f83d415df40 msgr2=0x7f83d41e9af0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:29:42.452 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:42.447+0000 7f83d37fe640 1 --2- 172.21.15.37:0/1034130014 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f83d415df40 0x7f83d41e9af0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:29:42.452 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:42.447+0000 7f83d37fe640 1 -- 172.21.15.37:0/1034130014 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f83d40d5430 msgr2=0x7f83d40ce6d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:29:42.453 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:42.447+0000 7f83d3fff640 1 --2- 172.21.15.37:0/1034130014 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f83d40d5430 0x7f83d40ce6d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:29:42.453 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:42.447+0000 7f83d37fe640 1 --2- 172.21.15.37:0/1034130014 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f83d40d5430 0x7f83d40ce6d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:29:42.453 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:42.447+0000 7f83d37fe640 1 -- 172.21.15.37:0/1034130014 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f83d40ced90 con 0x7f83d4161110 2024-04-25T10:29:42.453 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:42.447+0000 7f83d3fff640 1 --2- 172.21.15.37:0/1034130014 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f83d40d5430 0x7f83d40ce6d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T10:29:42.453 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:42.448+0000 7f83d37fe640 1 --2- 172.21.15.37:0/1034130014 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f83d4161110 0x7f83d40cd8f0 secure :-1 s=READY pgs=244 cs=0 l=1 rev1=1 crypto rx=0x7f83c400c3c0 tx=0x7f83c400c880 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:29:42.453 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:42.448+0000 7f83d0ff9640 1 -- 172.21.15.37:0/1034130014 <== mon.0 v2:172.21.15.37:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f83c4017070 con 0x7f83d4161110 2024-04-25T10:29:42.453 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:42.448+0000 7f83d0ff9640 1 -- 172.21.15.37:0/1034130014 <== mon.0 v2:172.21.15.37:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f83c4005ce0 con 0x7f83d4161110 2024-04-25T10:29:42.453 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:42.448+0000 7f83d0ff9640 1 -- 172.21.15.37:0/1034130014 <== mon.0 v2:172.21.15.37:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f83c4007310 con 0x7f83d4161110 2024-04-25T10:29:42.453 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:42.448+0000 7f83da6d8640 1 -- 172.21.15.37:0/1034130014 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f83d40cf070 con 0x7f83d4161110 2024-04-25T10:29:42.453 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:42.448+0000 7f83da6d8640 1 -- 172.21.15.37:0/1034130014 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f83d40cf630 con 0x7f83d4161110 2024-04-25T10:29:42.453 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:42.449+0000 7f83da6d8640 1 -- 172.21.15.37:0/1034130014 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f83d406a140 con 0x7f83d4161110 2024-04-25T10:29:42.453 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:42.449+0000 7f83d0ff9640 1 -- 172.21.15.37:0/1034130014 <== mon.0 v2:172.21.15.37:3300/0 4 ==== mgrmap(e 5) ==== 96782+0+0 (secure 0 0 0) 0x7f83c401a070 con 0x7f83d4161110 2024-04-25T10:29:42.454 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:42.450+0000 7f83d0ff9640 1 --2- 172.21.15.37:0/1034130014 >> [v2:172.21.15.203:6832/2418205237,v1:172.21.15.203:6833/2418205237] conn(0x7f83b40744b0 0x7f83b40769c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:29:42.454 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:42.450+0000 7f83d0ff9640 1 -- 172.21.15.37:0/1034130014 <== mon.0 v2:172.21.15.37:3300/0 5 ==== osd_map(23..23 src has 1..23) ==== 5709+0+0 (secure 0 0 0) 0x7f83c4094560 con 0x7f83d4161110 2024-04-25T10:29:42.454 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:42.450+0000 7f83d3fff640 1 --2- 172.21.15.37:0/1034130014 >> [v2:172.21.15.203:6832/2418205237,v1:172.21.15.203:6833/2418205237] conn(0x7f83b40744b0 0x7f83b40769c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:29:42.454 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:42.451+0000 7f83d3fff640 1 --2- 172.21.15.37:0/1034130014 >> [v2:172.21.15.203:6832/2418205237,v1:172.21.15.203:6833/2418205237] conn(0x7f83b40744b0 0x7f83b40769c0 secure :-1 s=READY pgs=136 cs=0 l=1 rev1=1 crypto rx=0x7f83bc000f60 tx=0x7f83bc009040 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:29:42.457 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:42.455+0000 7f83d0ff9640 1 -- 172.21.15.37:0/1034130014 <== mon.0 v2:172.21.15.37:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187246 (secure 0 0 0) 0x7f83c405ed80 con 0x7f83d4161110 2024-04-25T10:29:42.641 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:42.639+0000 7f83da6d8640 1 -- 172.21.15.37:0/1034130014 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"} v 0) -- 0x7f83d415f000 con 0x7f83d4161110 2024-04-25T10:29:42.641 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:42.639+0000 7f83d0ff9640 1 -- 172.21.15.37:0/1034130014 <== mon.0 v2:172.21.15.37:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]=0 v23) ==== 100+0+11 (secure 0 0 0) 0x7f83c405e680 con 0x7f83d4161110 2024-04-25T10:29:42.641 INFO:teuthology.orchestra.run.smithi037.stdout:pg_num: 64 2024-04-25T10:29:42.643 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:42.641+0000 7f83da6d8640 1 -- 172.21.15.37:0/1034130014 >> [v2:172.21.15.203:6832/2418205237,v1:172.21.15.203:6833/2418205237] conn(0x7f83b40744b0 msgr2=0x7f83b40769c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:29:42.643 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:42.641+0000 7f83da6d8640 1 --2- 172.21.15.37:0/1034130014 >> [v2:172.21.15.203:6832/2418205237,v1:172.21.15.203:6833/2418205237] conn(0x7f83b40744b0 0x7f83b40769c0 secure :-1 s=READY pgs=136 cs=0 l=1 rev1=1 crypto rx=0x7f83bc000f60 tx=0x7f83bc009040 comp rx=0 tx=0).stop 2024-04-25T10:29:42.644 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:42.641+0000 7f83da6d8640 1 -- 172.21.15.37:0/1034130014 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f83d4161110 msgr2=0x7f83d40cd8f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:29:42.644 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:42.641+0000 7f83da6d8640 1 --2- 172.21.15.37:0/1034130014 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f83d4161110 0x7f83d40cd8f0 secure :-1 s=READY pgs=244 cs=0 l=1 rev1=1 crypto rx=0x7f83c400c3c0 tx=0x7f83c400c880 comp rx=0 tx=0).stop 2024-04-25T10:29:42.644 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:42.641+0000 7f83da6d8640 1 -- 172.21.15.37:0/1034130014 shutdown_connections 2024-04-25T10:29:42.644 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:42.641+0000 7f83da6d8640 1 --2- 172.21.15.37:0/1034130014 >> [v2:172.21.15.203:6832/2418205237,v1:172.21.15.203:6833/2418205237] conn(0x7f83b40744b0 0x7f83b40769c0 unknown :-1 s=CLOSED pgs=136 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:29:42.644 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:42.641+0000 7f83da6d8640 1 --2- 172.21.15.37:0/1034130014 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f83d415df40 0x7f83d41e9af0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:29:42.644 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:42.641+0000 7f83da6d8640 1 --2- 172.21.15.37:0/1034130014 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f83d4161110 0x7f83d40cd8f0 unknown :-1 s=CLOSED pgs=244 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:29:42.644 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:42.641+0000 7f83da6d8640 1 --2- 172.21.15.37:0/1034130014 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f83d40d5430 0x7f83d40ce6d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:29:42.644 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:42.641+0000 7f83da6d8640 1 -- 172.21.15.37:0/1034130014 >> 172.21.15.37:0/1034130014 conn(0x7f83d40d27a0 msgr2=0x7f83d40b77d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:29:42.644 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:42.641+0000 7f83da6d8640 1 -- 172.21.15.37:0/1034130014 shutdown_connections 2024-04-25T10:29:42.644 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:42.641+0000 7f83da6d8640 1 -- 172.21.15.37:0/1034130014 wait complete. 2024-04-25T10:29:42.656 DEBUG:teuthology.orchestra.run.smithi037:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph log 'Starting test tasks.cephfs.test_recovery_pool.TestRecoveryPool.test_rebuild_simple' 2024-04-25T10:29:42.768 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:42.765+0000 7f00c936b640 1 Processor -- start 2024-04-25T10:29:42.768 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:42.765+0000 7f00c936b640 1 -- start start 2024-04-25T10:29:42.769 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:42.767+0000 7f00c936b640 1 --2- >> v2:172.21.15.37:3300/0 conn(0x7f00c40d2ce0 0x7f00c40d0e10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:29:42.770 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:42.767+0000 7f00c936b640 1 -- --> v1:172.21.15.37:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f00c40d14c0 con 0x7f00c40d5220 2024-04-25T10:29:42.770 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:42.767+0000 7f00c936b640 1 -- --> v1:172.21.15.203:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f00c40d4740 con 0x7f00c40d0a00 2024-04-25T10:29:42.770 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:42.767+0000 7f00c936b640 1 -- --> v2:172.21.15.37:3300/0 -- mon_getmap magic: 0 -- 0x7f00c40d5630 con 0x7f00c40d2ce0 2024-04-25T10:29:42.770 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:42.767+0000 7f00c37fe640 1 --2- >> v2:172.21.15.37:3300/0 conn(0x7f00c40d2ce0 0x7f00c40d0e10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:29:42.770 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:42.767+0000 7f00c2ffd640 1 --1- >> v1:172.21.15.37:6789/0 conn(0x7f00c40d5220 0x7f00c4164580 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.37:6789/0 says I am v1:172.21.15.37:49830/0 (socket says 172.21.15.37:49830) 2024-04-25T10:29:42.770 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:42.767+0000 7f00c2ffd640 1 -- 172.21.15.37:0/2644222322 learned_addr learned my addr 172.21.15.37:0/2644222322 (peer_addr_for_me v1:172.21.15.37:0/0) 2024-04-25T10:29:42.770 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:42.767+0000 7f00c37fe640 1 -- 172.21.15.37:0/2644222322 >> v1:172.21.15.203:6789/0 conn(0x7f00c40d0a00 legacy=0x7f00c40d25d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:29:42.770 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:42.767+0000 7f00c37fe640 1 -- 172.21.15.37:0/2644222322 >> v1:172.21.15.37:6789/0 conn(0x7f00c40d5220 legacy=0x7f00c4164580 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:29:42.770 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:42.767+0000 7f00c37fe640 1 -- 172.21.15.37:0/2644222322 --> v2:172.21.15.37:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f00c40d4a20 con 0x7f00c40d2ce0 2024-04-25T10:29:42.770 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:42.767+0000 7f00c1ffb640 1 -- 172.21.15.37:0/2644222322 <== mon.0 v1:172.21.15.37:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 1509942061 0 0) 0x7f00c40d14c0 con 0x7f00c40d5220 2024-04-25T10:29:42.771 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:42.768+0000 7f00c37fe640 1 --2- 172.21.15.37:0/2644222322 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f00c40d2ce0 0x7f00c40d0e10 secure :-1 s=READY pgs=246 cs=0 l=1 rev1=1 crypto rx=0x7f00b4009d90 tx=0x7f00b402a610 comp rx=0 tx=0).ready entity=mon.0 client_cookie=743ed4a840fd71b1 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:29:42.772 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:42.768+0000 7f00c1ffb640 1 -- 172.21.15.37:0/2644222322 <== mon.0 v2:172.21.15.37:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f00b4031600 con 0x7f00c40d2ce0 2024-04-25T10:29:42.772 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:42.768+0000 7f00c1ffb640 1 -- 172.21.15.37:0/2644222322 <== mon.0 v2:172.21.15.37:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f00b40317a0 con 0x7f00c40d2ce0 2024-04-25T10:29:42.772 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:42.768+0000 7f00c1ffb640 1 -- 172.21.15.37:0/2644222322 <== mon.0 v2:172.21.15.37:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f00b4031b70 con 0x7f00c40d2ce0 2024-04-25T10:29:42.772 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:42.769+0000 7f00c936b640 1 -- 172.21.15.37:0/2644222322 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f00c40d2ce0 msgr2=0x7f00c40d0e10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:29:42.772 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:42.769+0000 7f00c936b640 1 --2- 172.21.15.37:0/2644222322 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f00c40d2ce0 0x7f00c40d0e10 secure :-1 s=READY pgs=246 cs=0 l=1 rev1=1 crypto rx=0x7f00b4009d90 tx=0x7f00b402a610 comp rx=0 tx=0).stop 2024-04-25T10:29:42.772 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:42.769+0000 7f00c936b640 1 -- 172.21.15.37:0/2644222322 shutdown_connections 2024-04-25T10:29:42.772 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:42.769+0000 7f00c936b640 1 --2- 172.21.15.37:0/2644222322 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f00c40d2ce0 0x7f00c40d0e10 unknown :-1 s=CLOSED pgs=246 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:29:42.772 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:42.769+0000 7f00c936b640 1 -- 172.21.15.37:0/2644222322 >> 172.21.15.37:0/2644222322 conn(0x7f00c406e160 msgr2=0x7f00c406e570 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:29:42.772 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:42.769+0000 7f00c936b640 1 -- 172.21.15.37:0/2644222322 shutdown_connections 2024-04-25T10:29:42.772 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:42.769+0000 7f00c936b640 1 -- 172.21.15.37:0/2644222322 wait complete. 2024-04-25T10:29:42.773 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:42.771+0000 7f00c936b640 1 Processor -- start 2024-04-25T10:29:42.773 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:42.771+0000 7f00c936b640 1 -- start start 2024-04-25T10:29:42.775 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:42.772+0000 7f00c936b640 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f00c40d0a00 0x7f00c415a940 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:29:42.776 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:42.772+0000 7f00c936b640 1 --2- >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f00c40d5220 0x7f00c40b6180 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:29:42.776 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:42.772+0000 7f00c936b640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f00c40cf390 0x7f00c40b66c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:29:42.776 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:42.772+0000 7f00c936b640 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 -- 0x7f00c416b1b0 con 0x7f00c40cf390 2024-04-25T10:29:42.776 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:42.772+0000 7f00c936b640 1 -- --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_getmap magic: 0 -- 0x7f00c40d17a0 con 0x7f00c40d0a00 2024-04-25T10:29:42.776 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:42.772+0000 7f00c936b640 1 -- --> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] -- mon_getmap magic: 0 -- 0x7f00c416b330 con 0x7f00c40d5220 2024-04-25T10:29:42.776 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:42.772+0000 7f00c37fe640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f00c40cf390 0x7f00c40b66c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:29:42.776 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:42.772+0000 7f00c37fe640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f00c40cf390 0x7f00c40b66c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.37:3300/0 says I am v2:172.21.15.37:59450/0 (socket says 172.21.15.37:59450) 2024-04-25T10:29:42.776 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:42.772+0000 7f00c37fe640 1 -- 172.21.15.37:0/2225817344 learned_addr learned my addr 172.21.15.37:0/2225817344 (peer_addr_for_me v2:172.21.15.37:0/0) 2024-04-25T10:29:42.776 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:42.772+0000 7f00c2ffd640 1 --2- 172.21.15.37:0/2225817344 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f00c40d0a00 0x7f00c415a940 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:29:42.776 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:42.772+0000 7f00c27fc640 1 --2- 172.21.15.37:0/2225817344 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f00c40d5220 0x7f00c40b6180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:29:42.776 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:42.773+0000 7f00c37fe640 1 -- 172.21.15.37:0/2225817344 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f00c40d5220 msgr2=0x7f00c40b6180 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:29:42.776 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:42.773+0000 7f00c37fe640 1 --2- 172.21.15.37:0/2225817344 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f00c40d5220 0x7f00c40b6180 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:29:42.776 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:42.773+0000 7f00c37fe640 1 -- 172.21.15.37:0/2225817344 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f00c40d0a00 msgr2=0x7f00c415a940 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:29:42.776 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:42.773+0000 7f00c37fe640 1 --2- 172.21.15.37:0/2225817344 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f00c40d0a00 0x7f00c415a940 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:29:42.777 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:42.773+0000 7f00c37fe640 1 -- 172.21.15.37:0/2225817344 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f00c40b8340 con 0x7f00c40cf390 2024-04-25T10:29:42.777 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:42.773+0000 7f00c37fe640 1 --2- 172.21.15.37:0/2225817344 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f00c40cf390 0x7f00c40b66c0 secure :-1 s=READY pgs=247 cs=0 l=1 rev1=1 crypto rx=0x7f00b402ac00 tx=0x7f00b4004a40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:29:42.777 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:42.773+0000 7f00bb7fe640 1 -- 172.21.15.37:0/2225817344 <== mon.0 v2:172.21.15.37:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f00b4040070 con 0x7f00c40cf390 2024-04-25T10:29:42.777 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:42.773+0000 7f00bb7fe640 1 -- 172.21.15.37:0/2225817344 <== mon.0 v2:172.21.15.37:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f00b4030b60 con 0x7f00c40cf390 2024-04-25T10:29:42.777 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:42.773+0000 7f00c936b640 1 -- 172.21.15.37:0/2225817344 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f00c40ce8b0 con 0x7f00c40cf390 2024-04-25T10:29:42.777 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:42.773+0000 7f00bb7fe640 1 -- 172.21.15.37:0/2225817344 <== mon.0 v2:172.21.15.37:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f00b4045460 con 0x7f00c40cf390 2024-04-25T10:29:42.777 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:42.773+0000 7f00c936b640 1 -- 172.21.15.37:0/2225817344 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f00c40cee70 con 0x7f00c40cf390 2024-04-25T10:29:42.778 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:42.774+0000 7f00c936b640 1 -- 172.21.15.37:0/2225817344 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f00c40ce6a0 con 0x7f00c40cf390 2024-04-25T10:29:42.778 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:42.775+0000 7f00bb7fe640 1 -- 172.21.15.37:0/2225817344 <== mon.0 v2:172.21.15.37:3300/0 4 ==== mgrmap(e 5) ==== 96782+0+0 (secure 0 0 0) 0x7f00b4039040 con 0x7f00c40cf390 2024-04-25T10:29:42.778 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:42.775+0000 7f00bb7fe640 1 --2- 172.21.15.37:0/2225817344 >> [v2:172.21.15.203:6832/2418205237,v1:172.21.15.203:6833/2418205237] conn(0x7f00a40744b0 0x7f00a40769c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:29:42.778 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:42.775+0000 7f00bb7fe640 1 -- 172.21.15.37:0/2225817344 <== mon.0 v2:172.21.15.37:3300/0 5 ==== osd_map(23..23 src has 1..23) ==== 5709+0+0 (secure 0 0 0) 0x7f00b40b69d0 con 0x7f00c40cf390 2024-04-25T10:29:42.778 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:42.775+0000 7f00c2ffd640 1 --2- 172.21.15.37:0/2225817344 >> [v2:172.21.15.203:6832/2418205237,v1:172.21.15.203:6833/2418205237] conn(0x7f00a40744b0 0x7f00a40769c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:29:42.778 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:42.776+0000 7f00c2ffd640 1 --2- 172.21.15.37:0/2225817344 >> [v2:172.21.15.203:6832/2418205237,v1:172.21.15.203:6833/2418205237] conn(0x7f00a40744b0 0x7f00a40769c0 secure :-1 s=READY pgs=137 cs=0 l=1 rev1=1 crypto rx=0x7f00bc000f60 tx=0x7f00bc007c00 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:29:42.782 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:42.780+0000 7f00bb7fe640 1 -- 172.21.15.37:0/2225817344 <== mon.0 v2:172.21.15.37:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187246 (secure 0 0 0) 0x7f00b40812b0 con 0x7f00c40cf390 2024-04-25T10:29:43.037 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:43.035+0000 7f00c936b640 1 -- 172.21.15.37:0/2225817344 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_command({"prefix": "log", "logtext": ["Starting test tasks.cephfs.test_recovery_pool.TestRecoveryPool.test_rebuild_simple"]} v 0) -- 0x7f00c406a140 con 0x7f00c40cf390 2024-04-25T10:29:43.312 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:43.310+0000 7f00bb7fe640 1 -- 172.21.15.37:0/2225817344 <== mon.0 v2:172.21.15.37:3300/0 7 ==== mon_command_ack([{"prefix": "log", "logtext": ["Starting test tasks.cephfs.test_recovery_pool.TestRecoveryPool.test_rebuild_simple"]}]=0 v53) ==== 150+0+0 (secure 0 0 0) 0x7f00b4080bb0 con 0x7f00c40cf390 2024-04-25T10:29:43.314 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:43.312+0000 7f00c936b640 1 -- 172.21.15.37:0/2225817344 >> [v2:172.21.15.203:6832/2418205237,v1:172.21.15.203:6833/2418205237] conn(0x7f00a40744b0 msgr2=0x7f00a40769c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:29:43.315 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:43.312+0000 7f00c936b640 1 --2- 172.21.15.37:0/2225817344 >> [v2:172.21.15.203:6832/2418205237,v1:172.21.15.203:6833/2418205237] conn(0x7f00a40744b0 0x7f00a40769c0 secure :-1 s=READY pgs=137 cs=0 l=1 rev1=1 crypto rx=0x7f00bc000f60 tx=0x7f00bc007c00 comp rx=0 tx=0).stop 2024-04-25T10:29:43.315 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:43.312+0000 7f00c936b640 1 -- 172.21.15.37:0/2225817344 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f00c40cf390 msgr2=0x7f00c40b66c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:29:43.315 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:43.312+0000 7f00c936b640 1 --2- 172.21.15.37:0/2225817344 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f00c40cf390 0x7f00c40b66c0 secure :-1 s=READY pgs=247 cs=0 l=1 rev1=1 crypto rx=0x7f00b402ac00 tx=0x7f00b4004a40 comp rx=0 tx=0).stop 2024-04-25T10:29:43.315 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:43.312+0000 7f00c936b640 1 -- 172.21.15.37:0/2225817344 shutdown_connections 2024-04-25T10:29:43.315 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:43.312+0000 7f00c936b640 1 --2- 172.21.15.37:0/2225817344 >> [v2:172.21.15.203:6832/2418205237,v1:172.21.15.203:6833/2418205237] conn(0x7f00a40744b0 0x7f00a40769c0 unknown :-1 s=CLOSED pgs=137 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:29:43.315 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:43.312+0000 7f00c936b640 1 --2- 172.21.15.37:0/2225817344 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f00c40cf390 0x7f00c40b66c0 unknown :-1 s=CLOSED pgs=247 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:29:43.315 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:43.312+0000 7f00c936b640 1 --2- 172.21.15.37:0/2225817344 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f00c40d5220 0x7f00c40b6180 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:29:43.315 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:43.312+0000 7f00c936b640 1 --2- 172.21.15.37:0/2225817344 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f00c40d0a00 0x7f00c415a940 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:29:43.315 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:43.312+0000 7f00c936b640 1 -- 172.21.15.37:0/2225817344 >> 172.21.15.37:0/2225817344 conn(0x7f00c406e160 msgr2=0x7f00c4164580 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:29:43.315 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:43.312+0000 7f00c936b640 1 -- 172.21.15.37:0/2225817344 shutdown_connections 2024-04-25T10:29:43.315 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:43.312+0000 7f00c936b640 1 -- 172.21.15.37:0/2225817344 wait complete. 2024-04-25T10:29:43.328 DEBUG:teuthology.orchestra.run.smithi037:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph config set mon mon_allow_pool_delete True 2024-04-25T10:29:43.487 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:43.484+0000 7f4e584f9640 1 Processor -- start 2024-04-25T10:29:43.487 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:43.484+0000 7f4e584f9640 1 -- start start 2024-04-25T10:29:43.489 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:43.486+0000 7f4e584f9640 1 --2- >> v2:172.21.15.37:3300/0 conn(0x7f4e5015aa20 0x7f4e50161a80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:29:43.489 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:43.486+0000 7f4e584f9640 1 --2- >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f4e500b7750 0x7f4e500b7b60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:29:43.489 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:43.486+0000 7f4e584f9640 1 -- --> v1:172.21.15.203:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f4e500b80a0 con 0x7f4e500d0d10 2024-04-25T10:29:43.490 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:43.486+0000 7f4e584f9640 1 -- --> v2:172.21.15.37:3300/0 -- mon_getmap magic: 0 -- 0x7f4e500d4b30 con 0x7f4e5015aa20 2024-04-25T10:29:43.490 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:43.486+0000 7f4e584f9640 1 -- --> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] -- mon_getmap magic: 0 -- 0x7f4e500d4240 con 0x7f4e500b7750 2024-04-25T10:29:43.490 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:43.486+0000 7f4e56295640 1 --2- >> v2:172.21.15.37:3300/0 conn(0x7f4e5015aa20 0x7f4e50161a80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:29:43.490 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:43.486+0000 7f4e56295640 1 --2- >> v2:172.21.15.37:3300/0 conn(0x7f4e5015aa20 0x7f4e50161a80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.37:3300/0 says I am v2:172.21.15.37:59456/0 (socket says 172.21.15.37:59456) 2024-04-25T10:29:43.490 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:43.486+0000 7f4e56295640 1 -- 172.21.15.37:0/1021694037 learned_addr learned my addr 172.21.15.37:0/1021694037 (peer_addr_for_me v2:172.21.15.37:0/0) 2024-04-25T10:29:43.490 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:43.486+0000 7f4e55a94640 1 --2- 172.21.15.37:0/1021694037 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f4e500b7750 0x7f4e500b7b60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:29:43.490 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:43.486+0000 7f4e56295640 1 -- 172.21.15.37:0/1021694037 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f4e500b7750 msgr2=0x7f4e500b7b60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:29:43.490 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:43.486+0000 7f4e56295640 1 --2- 172.21.15.37:0/1021694037 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f4e500b7750 0x7f4e500b7b60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:29:43.490 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:43.486+0000 7f4e56295640 1 -- 172.21.15.37:0/1021694037 >> v1:172.21.15.203:6789/0 conn(0x7f4e500d0d10 legacy=0x7f4e500d1120 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:29:43.490 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:43.487+0000 7f4e56295640 1 -- 172.21.15.37:0/1021694037 --> v2:172.21.15.37:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f4e500d1a00 con 0x7f4e5015aa20 2024-04-25T10:29:43.490 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:43.487+0000 7f4e55a94640 1 --2- 172.21.15.37:0/1021694037 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f4e500b7750 0x7f4e500b7b60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T10:29:43.490 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:43.487+0000 7f4e56295640 1 --2- 172.21.15.37:0/1021694037 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f4e5015aa20 0x7f4e50161a80 secure :-1 s=READY pgs=248 cs=0 l=1 rev1=1 crypto rx=0x7f4e4c009e50 tx=0x7f4e4c02a6c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=395790d0e26b8d88 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:29:43.490 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:43.487+0000 7f4e55293640 1 -- 172.21.15.37:0/1021694037 <== mon.0 v2:172.21.15.37:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4e4c036070 con 0x7f4e5015aa20 2024-04-25T10:29:43.490 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:43.487+0000 7f4e55293640 1 -- 172.21.15.37:0/1021694037 <== mon.0 v2:172.21.15.37:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f4e4c031600 con 0x7f4e5015aa20 2024-04-25T10:29:43.491 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:43.487+0000 7f4e55293640 1 -- 172.21.15.37:0/1021694037 <== mon.0 v2:172.21.15.37:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4e4c0319d0 con 0x7f4e5015aa20 2024-04-25T10:29:43.492 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:43.488+0000 7f4e584f9640 1 -- 172.21.15.37:0/1021694037 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f4e5015aa20 msgr2=0x7f4e50161a80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:29:43.492 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:43.488+0000 7f4e584f9640 1 --2- 172.21.15.37:0/1021694037 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f4e5015aa20 0x7f4e50161a80 secure :-1 s=READY pgs=248 cs=0 l=1 rev1=1 crypto rx=0x7f4e4c009e50 tx=0x7f4e4c02a6c0 comp rx=0 tx=0).stop 2024-04-25T10:29:43.492 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:43.488+0000 7f4e584f9640 1 -- 172.21.15.37:0/1021694037 shutdown_connections 2024-04-25T10:29:43.492 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:43.488+0000 7f4e584f9640 1 --2- 172.21.15.37:0/1021694037 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f4e500b7750 0x7f4e500b7b60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:29:43.492 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:43.488+0000 7f4e584f9640 1 --2- 172.21.15.37:0/1021694037 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f4e5015aa20 0x7f4e50161a80 unknown :-1 s=CLOSED pgs=248 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:29:43.492 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:43.488+0000 7f4e584f9640 1 -- 172.21.15.37:0/1021694037 >> 172.21.15.37:0/1021694037 conn(0x7f4e500d2750 msgr2=0x7f4e500d2b60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:29:43.492 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:43.488+0000 7f4e584f9640 1 -- 172.21.15.37:0/1021694037 shutdown_connections 2024-04-25T10:29:43.492 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:43.488+0000 7f4e584f9640 1 -- 172.21.15.37:0/1021694037 wait complete. 2024-04-25T10:29:43.492 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:43.490+0000 7f4e584f9640 1 Processor -- start 2024-04-25T10:29:43.493 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:43.490+0000 7f4e584f9640 1 -- start start 2024-04-25T10:29:43.495 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:43.492+0000 7f4e584f9640 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f4e500b7750 0x7f4e50154030 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:29:43.495 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:43.492+0000 7f4e584f9640 1 --2- >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f4e500d0d10 0x7f4e500ca1f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:29:43.495 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:43.492+0000 7f4e584f9640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f4e5015aa20 0x7f4e500d5ec0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:29:43.495 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:43.492+0000 7f4e584f9640 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 -- 0x7f4e50161fc0 con 0x7f4e5015aa20 2024-04-25T10:29:43.495 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:43.492+0000 7f4e584f9640 1 -- --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_getmap magic: 0 -- 0x7f4e500d47c0 con 0x7f4e500b7750 2024-04-25T10:29:43.495 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:43.492+0000 7f4e584f9640 1 -- --> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] -- mon_getmap magic: 0 -- 0x7f4e500d4fc0 con 0x7f4e500d0d10 2024-04-25T10:29:43.495 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:43.492+0000 7f4e56a96640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f4e5015aa20 0x7f4e500d5ec0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:29:43.495 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:43.492+0000 7f4e56a96640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f4e5015aa20 0x7f4e500d5ec0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.37:3300/0 says I am v2:172.21.15.37:59466/0 (socket says 172.21.15.37:59466) 2024-04-25T10:29:43.495 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:43.492+0000 7f4e56a96640 1 -- 172.21.15.37:0/3626346072 learned_addr learned my addr 172.21.15.37:0/3626346072 (peer_addr_for_me v2:172.21.15.37:0/0) 2024-04-25T10:29:43.496 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:43.492+0000 7f4e56295640 1 --2- 172.21.15.37:0/3626346072 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f4e500b7750 0x7f4e50154030 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:29:43.496 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:43.492+0000 7f4e55a94640 1 --2- 172.21.15.37:0/3626346072 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f4e500d0d10 0x7f4e500ca1f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:29:43.496 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:43.492+0000 7f4e56a96640 1 -- 172.21.15.37:0/3626346072 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f4e500d0d10 msgr2=0x7f4e500ca1f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:29:43.496 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:43.492+0000 7f4e56a96640 1 --2- 172.21.15.37:0/3626346072 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f4e500d0d10 0x7f4e500ca1f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:29:43.496 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:43.492+0000 7f4e56a96640 1 -- 172.21.15.37:0/3626346072 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f4e500b7750 msgr2=0x7f4e50154030 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:29:43.496 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:43.492+0000 7f4e56a96640 1 --2- 172.21.15.37:0/3626346072 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f4e500b7750 0x7f4e50154030 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:29:43.496 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:43.492+0000 7f4e56a96640 1 -- 172.21.15.37:0/3626346072 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f4e500ca730 con 0x7f4e5015aa20 2024-04-25T10:29:43.496 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:43.492+0000 7f4e56a96640 1 --2- 172.21.15.37:0/3626346072 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f4e5015aa20 0x7f4e500d5ec0 secure :-1 s=READY pgs=249 cs=0 l=1 rev1=1 crypto rx=0x7f4e4800b410 tx=0x7f4e4800b8d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:29:43.496 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:43.493+0000 7f4e46ffd640 1 -- 172.21.15.37:0/3626346072 <== mon.0 v2:172.21.15.37:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4e48015030 con 0x7f4e5015aa20 2024-04-25T10:29:43.496 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:43.493+0000 7f4e46ffd640 1 -- 172.21.15.37:0/3626346072 <== mon.0 v2:172.21.15.37:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f4e48004020 con 0x7f4e5015aa20 2024-04-25T10:29:43.496 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:43.493+0000 7f4e584f9640 1 -- 172.21.15.37:0/3626346072 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f4e500ce4b0 con 0x7f4e5015aa20 2024-04-25T10:29:43.496 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:43.493+0000 7f4e46ffd640 1 -- 172.21.15.37:0/3626346072 <== mon.0 v2:172.21.15.37:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4e48004450 con 0x7f4e5015aa20 2024-04-25T10:29:43.496 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:43.493+0000 7f4e584f9640 1 -- 172.21.15.37:0/3626346072 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f4e500cb420 con 0x7f4e5015aa20 2024-04-25T10:29:43.497 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:43.494+0000 7f4e584f9640 1 -- 172.21.15.37:0/3626346072 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f4e5006a140 con 0x7f4e5015aa20 2024-04-25T10:29:43.497 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:43.494+0000 7f4e46ffd640 1 -- 172.21.15.37:0/3626346072 <== mon.0 v2:172.21.15.37:3300/0 4 ==== mgrmap(e 5) ==== 96782+0+0 (secure 0 0 0) 0x7f4e48018070 con 0x7f4e5015aa20 2024-04-25T10:29:43.497 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:43.495+0000 7f4e46ffd640 1 --2- 172.21.15.37:0/3626346072 >> [v2:172.21.15.203:6832/2418205237,v1:172.21.15.203:6833/2418205237] conn(0x7f4e340744b0 0x7f4e340769c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:29:43.497 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:43.495+0000 7f4e46ffd640 1 -- 172.21.15.37:0/3626346072 <== mon.0 v2:172.21.15.37:3300/0 5 ==== osd_map(23..23 src has 1..23) ==== 5709+0+0 (secure 0 0 0) 0x7f4e48094a50 con 0x7f4e5015aa20 2024-04-25T10:29:43.498 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:43.495+0000 7f4e56295640 1 --2- 172.21.15.37:0/3626346072 >> [v2:172.21.15.203:6832/2418205237,v1:172.21.15.203:6833/2418205237] conn(0x7f4e340744b0 0x7f4e340769c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:29:43.498 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:43.496+0000 7f4e56295640 1 --2- 172.21.15.37:0/3626346072 >> [v2:172.21.15.203:6832/2418205237,v1:172.21.15.203:6833/2418205237] conn(0x7f4e340744b0 0x7f4e340769c0 secure :-1 s=READY pgs=138 cs=0 l=1 rev1=1 crypto rx=0x7f4e4c033000 tx=0x7f4e4c009000 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:29:43.502 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:43.500+0000 7f4e46ffd640 1 -- 172.21.15.37:0/3626346072 <== mon.0 v2:172.21.15.37:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187246 (secure 0 0 0) 0x7f4e4805ecb0 con 0x7f4e5015aa20 2024-04-25T10:29:43.689 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:43.686+0000 7f4e584f9640 1 -- 172.21.15.37:0/3626346072 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_command([{prefix=config set, name=mon_allow_pool_delete}] v 0) -- 0x7f4e500ba1e0 con 0x7f4e5015aa20 2024-04-25T10:29:43.693 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:43.690+0000 7f4e46ffd640 1 -- 172.21.15.37:0/3626346072 <== mon.0 v2:172.21.15.37:3300/0 7 ==== mon_command_ack([{prefix=config set, name=mon_allow_pool_delete}]=0 v10)=0 v10) ==== 122+0+0 (secure 0 0 0) 0x7f4e48062fb0 con 0x7f4e5015aa20 2024-04-25T10:29:43.695 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:43.692+0000 7f4e584f9640 1 -- 172.21.15.37:0/3626346072 >> [v2:172.21.15.203:6832/2418205237,v1:172.21.15.203:6833/2418205237] conn(0x7f4e340744b0 msgr2=0x7f4e340769c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:29:43.695 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:43.692+0000 7f4e584f9640 1 --2- 172.21.15.37:0/3626346072 >> [v2:172.21.15.203:6832/2418205237,v1:172.21.15.203:6833/2418205237] conn(0x7f4e340744b0 0x7f4e340769c0 secure :-1 s=READY pgs=138 cs=0 l=1 rev1=1 crypto rx=0x7f4e4c033000 tx=0x7f4e4c009000 comp rx=0 tx=0).stop 2024-04-25T10:29:43.696 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:43.692+0000 7f4e584f9640 1 -- 172.21.15.37:0/3626346072 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f4e5015aa20 msgr2=0x7f4e500d5ec0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:29:43.696 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:43.692+0000 7f4e584f9640 1 --2- 172.21.15.37:0/3626346072 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f4e5015aa20 0x7f4e500d5ec0 secure :-1 s=READY pgs=249 cs=0 l=1 rev1=1 crypto rx=0x7f4e4800b410 tx=0x7f4e4800b8d0 comp rx=0 tx=0).stop 2024-04-25T10:29:43.696 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:43.692+0000 7f4e584f9640 1 -- 172.21.15.37:0/3626346072 shutdown_connections 2024-04-25T10:29:43.696 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:43.692+0000 7f4e584f9640 1 --2- 172.21.15.37:0/3626346072 >> [v2:172.21.15.203:6832/2418205237,v1:172.21.15.203:6833/2418205237] conn(0x7f4e340744b0 0x7f4e340769c0 unknown :-1 s=CLOSED pgs=138 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:29:43.696 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:43.692+0000 7f4e584f9640 1 --2- 172.21.15.37:0/3626346072 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f4e5015aa20 0x7f4e500d5ec0 unknown :-1 s=CLOSED pgs=249 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:29:43.696 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:43.692+0000 7f4e584f9640 1 --2- 172.21.15.37:0/3626346072 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f4e500d0d10 0x7f4e500ca1f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:29:43.697 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:43.692+0000 7f4e584f9640 1 --2- 172.21.15.37:0/3626346072 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f4e500b7750 0x7f4e50154030 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:29:43.697 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:43.692+0000 7f4e584f9640 1 -- 172.21.15.37:0/3626346072 >> 172.21.15.37:0/3626346072 conn(0x7f4e500d2750 msgr2=0x7f4e500b63a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:29:43.697 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:43.692+0000 7f4e584f9640 1 -- 172.21.15.37:0/3626346072 shutdown_connections 2024-04-25T10:29:43.697 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:43.692+0000 7f4e584f9640 1 -- 172.21.15.37:0/3626346072 wait complete. 2024-04-25T10:29:43.706 INFO:teuthology.nuke.actions:Clearing teuthology firewall rules... 2024-04-25T10:29:43.706 DEBUG:teuthology.orchestra.run.smithi037:> sudo sh -c 'iptables-save | grep -v teuthology | iptables-restore' 2024-04-25T10:29:43.754 DEBUG:teuthology.orchestra.run.smithi203:> sudo sh -c 'iptables-save | grep -v teuthology | iptables-restore' 2024-04-25T10:29:43.794 INFO:teuthology.nuke.actions:Cleared teuthology firewall rules. 2024-04-25T10:29:43.794 DEBUG:teuthology.orchestra.run.smithi037:> set -ex 2024-04-25T10:29:43.794 DEBUG:teuthology.orchestra.run.smithi037:> dd if=/proc/self/mounts of=/dev/stdout 2024-04-25T10:29:43.810 DEBUG:teuthology.orchestra.run.smithi037:> set -ex 2024-04-25T10:29:43.810 DEBUG:teuthology.orchestra.run.smithi037:> dd if=/proc/self/mounts of=/dev/stdout 2024-04-25T10:29:43.866 DEBUG:teuthology.orchestra.run.smithi037:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-04-25T10:29:44.020 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:44.018+0000 7f7a3dd1c640 1 Processor -- start 2024-04-25T10:29:44.020 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:44.018+0000 7f7a3dd1c640 1 -- start start 2024-04-25T10:29:44.023 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:44.019+0000 7f7a3dd1c640 1 --2- >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f7a3808cc90 0x7f7a3809cbc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:29:44.023 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:44.019+0000 7f7a3dd1c640 1 --2- >> v2:172.21.15.37:3300/0 conn(0x7f7a380b71b0 0x7f7a380b75c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:29:44.023 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:44.019+0000 7f7a3dd1c640 1 -- --> v1:172.21.15.203:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f7a380b8d40 con 0x7f7a380d1c20 2024-04-25T10:29:44.023 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:44.019+0000 7f7a3dd1c640 1 -- --> v2:172.21.15.37:3300/0 -- mon_getmap magic: 0 -- 0x7f7a380d27f0 con 0x7f7a380b71b0 2024-04-25T10:29:44.023 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:44.019+0000 7f7a3dd1c640 1 -- --> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] -- mon_getmap magic: 0 -- 0x7f7a380d2ad0 con 0x7f7a3808cc90 2024-04-25T10:29:44.023 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:44.019+0000 7f7a37fff640 1 --2- >> v2:172.21.15.37:3300/0 conn(0x7f7a380b71b0 0x7f7a380b75c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:29:44.023 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:44.019+0000 7f7a37fff640 1 --2- >> v2:172.21.15.37:3300/0 conn(0x7f7a380b71b0 0x7f7a380b75c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.37:3300/0 says I am v2:172.21.15.37:59478/0 (socket says 172.21.15.37:59478) 2024-04-25T10:29:44.023 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:44.019+0000 7f7a37fff640 1 -- 172.21.15.37:0/1409992292 learned_addr learned my addr 172.21.15.37:0/1409992292 (peer_addr_for_me v2:172.21.15.37:0/0) 2024-04-25T10:29:44.023 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:44.019+0000 7f7a377fe640 1 --2- 172.21.15.37:0/1409992292 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f7a3808cc90 0x7f7a3809cbc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:29:44.023 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:44.020+0000 7f7a37fff640 1 -- 172.21.15.37:0/1409992292 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f7a3808cc90 msgr2=0x7f7a3809cbc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:29:44.023 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:44.020+0000 7f7a37fff640 1 --2- 172.21.15.37:0/1409992292 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f7a3808cc90 0x7f7a3809cbc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:29:44.023 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:44.020+0000 7f7a37fff640 1 -- 172.21.15.37:0/1409992292 >> v1:172.21.15.203:6789/0 conn(0x7f7a380d1c20 legacy=0x7f7a380d2030 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:29:44.023 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:44.020+0000 7f7a37fff640 1 -- 172.21.15.37:0/1409992292 --> v2:172.21.15.37:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f7a380cf7a0 con 0x7f7a380b71b0 2024-04-25T10:29:44.023 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:44.020+0000 7f7a377fe640 1 --2- 172.21.15.37:0/1409992292 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f7a3808cc90 0x7f7a3809cbc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T10:29:44.024 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:44.020+0000 7f7a37fff640 1 --2- 172.21.15.37:0/1409992292 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f7a380b71b0 0x7f7a380b75c0 secure :-1 s=READY pgs=250 cs=0 l=1 rev1=1 crypto rx=0x7f7a30009e50 tx=0x7f7a3002a6c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=50cb656f6fbc2b server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:29:44.024 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:44.020+0000 7f7a367fc640 1 -- 172.21.15.37:0/1409992292 <== mon.0 v2:172.21.15.37:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7a30036070 con 0x7f7a380b71b0 2024-04-25T10:29:44.024 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:44.020+0000 7f7a367fc640 1 -- 172.21.15.37:0/1409992292 <== mon.0 v2:172.21.15.37:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f7a30031600 con 0x7f7a380b71b0 2024-04-25T10:29:44.024 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:44.021+0000 7f7a367fc640 1 -- 172.21.15.37:0/1409992292 <== mon.0 v2:172.21.15.37:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7a300319d0 con 0x7f7a380b71b0 2024-04-25T10:29:44.024 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:44.021+0000 7f7a3dd1c640 1 -- 172.21.15.37:0/1409992292 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f7a380b71b0 msgr2=0x7f7a380b75c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:29:44.025 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:44.021+0000 7f7a3dd1c640 1 --2- 172.21.15.37:0/1409992292 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f7a380b71b0 0x7f7a380b75c0 secure :-1 s=READY pgs=250 cs=0 l=1 rev1=1 crypto rx=0x7f7a30009e50 tx=0x7f7a3002a6c0 comp rx=0 tx=0).stop 2024-04-25T10:29:44.025 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:44.021+0000 7f7a3dd1c640 1 -- 172.21.15.37:0/1409992292 shutdown_connections 2024-04-25T10:29:44.025 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:44.021+0000 7f7a3dd1c640 1 --2- 172.21.15.37:0/1409992292 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f7a380b71b0 0x7f7a380b75c0 unknown :-1 s=CLOSED pgs=250 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:29:44.025 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:44.021+0000 7f7a3dd1c640 1 --2- 172.21.15.37:0/1409992292 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f7a3808cc90 0x7f7a3809cbc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:29:44.025 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:44.021+0000 7f7a3dd1c640 1 -- 172.21.15.37:0/1409992292 >> 172.21.15.37:0/1409992292 conn(0x7f7a380d03d0 msgr2=0x7f7a380d07e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:29:44.025 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:44.021+0000 7f7a3dd1c640 1 -- 172.21.15.37:0/1409992292 shutdown_connections 2024-04-25T10:29:44.025 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:44.021+0000 7f7a3dd1c640 1 -- 172.21.15.37:0/1409992292 wait complete. 2024-04-25T10:29:44.025 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:44.023+0000 7f7a3dd1c640 1 Processor -- start 2024-04-25T10:29:44.026 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:44.023+0000 7f7a3dd1c640 1 -- start start 2024-04-25T10:29:44.028 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:44.024+0000 7f7a3dd1c640 1 --2- >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f7a380b71b0 0x7f7a380b1b10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:29:44.028 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:44.024+0000 7f7a3dd1c640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f7a380d1c20 0x7f7a381d5230 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:29:44.028 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:44.024+0000 7f7a3dd1c640 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f7a381d3a60 0x7f7a3809f850 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:29:44.028 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:44.024+0000 7f7a3dd1c640 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 -- 0x7f7a380b7b00 con 0x7f7a380d1c20 2024-04-25T10:29:44.028 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:44.024+0000 7f7a3dd1c640 1 -- --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_getmap magic: 0 -- 0x7f7a380b93b0 con 0x7f7a381d3a60 2024-04-25T10:29:44.028 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:44.024+0000 7f7a3dd1c640 1 -- --> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] -- mon_getmap magic: 0 -- 0x7f7a380b7c80 con 0x7f7a380b71b0 2024-04-25T10:29:44.028 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:44.025+0000 7f7a36ffd640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f7a380d1c20 0x7f7a381d5230 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:29:44.028 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:44.025+0000 7f7a36ffd640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f7a380d1c20 0x7f7a381d5230 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.37:3300/0 says I am v2:172.21.15.37:59488/0 (socket says 172.21.15.37:59488) 2024-04-25T10:29:44.028 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:44.025+0000 7f7a36ffd640 1 -- 172.21.15.37:0/548292398 learned_addr learned my addr 172.21.15.37:0/548292398 (peer_addr_for_me v2:172.21.15.37:0/0) 2024-04-25T10:29:44.028 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:44.025+0000 7f7a377fe640 1 --2- 172.21.15.37:0/548292398 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f7a380b71b0 0x7f7a380b1b10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:29:44.028 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:44.025+0000 7f7a37fff640 1 --2- 172.21.15.37:0/548292398 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f7a381d3a60 0x7f7a3809f850 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:29:44.029 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:44.025+0000 7f7a36ffd640 1 -- 172.21.15.37:0/548292398 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f7a380b71b0 msgr2=0x7f7a380b1b10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:29:44.029 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:44.025+0000 7f7a36ffd640 1 --2- 172.21.15.37:0/548292398 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f7a380b71b0 0x7f7a380b1b10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:29:44.029 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:44.025+0000 7f7a36ffd640 1 -- 172.21.15.37:0/548292398 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f7a381d3a60 msgr2=0x7f7a3809f850 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:29:44.029 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:44.025+0000 7f7a36ffd640 1 --2- 172.21.15.37:0/548292398 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f7a381d3a60 0x7f7a3809f850 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:29:44.029 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:44.025+0000 7f7a36ffd640 1 -- 172.21.15.37:0/548292398 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f7a381d4190 con 0x7f7a380d1c20 2024-04-25T10:29:44.029 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:44.025+0000 7f7a377fe640 1 --2- 172.21.15.37:0/548292398 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f7a380b71b0 0x7f7a380b1b10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T10:29:44.029 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:44.025+0000 7f7a36ffd640 1 --2- 172.21.15.37:0/548292398 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f7a380d1c20 0x7f7a381d5230 secure :-1 s=READY pgs=251 cs=0 l=1 rev1=1 crypto rx=0x7f7a2800b410 tx=0x7f7a2800b8d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:29:44.029 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:44.025+0000 7f7a2ffff640 1 -- 172.21.15.37:0/548292398 <== mon.0 v2:172.21.15.37:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7a28015020 con 0x7f7a380d1c20 2024-04-25T10:29:44.029 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:44.025+0000 7f7a2ffff640 1 -- 172.21.15.37:0/548292398 <== mon.0 v2:172.21.15.37:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f7a28011830 con 0x7f7a380d1c20 2024-04-25T10:29:44.029 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:44.025+0000 7f7a3dd1c640 1 -- 172.21.15.37:0/548292398 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f7a381d4470 con 0x7f7a380d1c20 2024-04-25T10:29:44.029 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:44.025+0000 7f7a2ffff640 1 -- 172.21.15.37:0/548292398 <== mon.0 v2:172.21.15.37:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7a28011c00 con 0x7f7a380d1c20 2024-04-25T10:29:44.029 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:44.026+0000 7f7a3dd1c640 1 -- 172.21.15.37:0/548292398 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f7a381d4a30 con 0x7f7a380d1c20 2024-04-25T10:29:44.030 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:44.026+0000 7f7a3dd1c640 1 -- 172.21.15.37:0/548292398 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f7a3806ca10 con 0x7f7a380d1c20 2024-04-25T10:29:44.030 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:44.027+0000 7f7a2ffff640 1 -- 172.21.15.37:0/548292398 <== mon.0 v2:172.21.15.37:3300/0 4 ==== mgrmap(e 5) ==== 96782+0+0 (secure 0 0 0) 0x7f7a28018020 con 0x7f7a380d1c20 2024-04-25T10:29:44.030 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:44.028+0000 7f7a2ffff640 1 --2- 172.21.15.37:0/548292398 >> [v2:172.21.15.203:6832/2418205237,v1:172.21.15.203:6833/2418205237] conn(0x7f7a180744b0 0x7f7a180769c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:29:44.030 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:44.028+0000 7f7a2ffff640 1 -- 172.21.15.37:0/548292398 <== mon.0 v2:172.21.15.37:3300/0 5 ==== osd_map(23..23 src has 1..23) ==== 5709+0+0 (secure 0 0 0) 0x7f7a28095100 con 0x7f7a380d1c20 2024-04-25T10:29:44.030 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:44.028+0000 7f7a377fe640 1 --2- 172.21.15.37:0/548292398 >> [v2:172.21.15.203:6832/2418205237,v1:172.21.15.203:6833/2418205237] conn(0x7f7a180744b0 0x7f7a180769c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:29:44.031 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:44.029+0000 7f7a377fe640 1 --2- 172.21.15.37:0/548292398 >> [v2:172.21.15.203:6832/2418205237,v1:172.21.15.203:6833/2418205237] conn(0x7f7a180744b0 0x7f7a180769c0 secure :-1 s=READY pgs=139 cs=0 l=1 rev1=1 crypto rx=0x7f7a38070e30 tx=0x7f7a20008040 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:29:44.039 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:44.037+0000 7f7a2ffff640 1 -- 172.21.15.37:0/548292398 <== mon.0 v2:172.21.15.37:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187246 (secure 0 0 0) 0x7f7a3806ca10 con 0x7f7a380d1c20 2024-04-25T10:29:44.223 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:44.221+0000 7f7a3dd1c640 1 -- 172.21.15.37:0/548292398 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7f7a381d3780 con 0x7f7a380d1c20 2024-04-25T10:29:44.225 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:44.223+0000 7f7a2ffff640 1 -- 172.21.15.37:0/548292398 <== mon.0 v2:172.21.15.37:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v23) ==== 74+0+16621 (secure 0 0 0) 0x7f7a381d3780 con 0x7f7a380d1c20 2024-04-25T10:29:44.226 INFO:teuthology.orchestra.run.smithi037.stdout: 2024-04-25T10:29:44.226 INFO:teuthology.orchestra.run.smithi037.stdout:{"epoch":23,"fsid":"f4acda28-be34-492a-b286-994afd36f61d","created":"2024-04-25T10:28:43.985718+0000","modified":"2024-04-25T10:29:10.016188+0000","last_up_change":"2024-04-25T10:28:54.900654+0000","last_in_change":"2024-04-25T10:28:46.216973+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":3,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-04-25T10:28:56.202153+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"19","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2024-04-25T10:29:06.507244+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"22","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":2.25,"score_stable":2.25,"optimal_score":1,"raw_score_acting":2.25,"raw_score_stable":2.25,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2024-04-25T10:29:07.347844+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"23","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"a2f79028-b332-4166-9ebb-a7024a37ea22","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":21,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.37:6816","nonce":2746074720},{"type":"v1","addr":"172.21.15.37:6817","nonce":2746074720}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.37:6818","nonce":2746074720},{"type":"v1","addr":"172.21.15.37:6819","nonce":2746074720}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.37:6822","nonce":2746074720},{"type":"v1","addr":"172.21.15.37:6823","nonce":2746074720}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.37:6820","nonce":2746074720},{"type":"v1","addr":"172.21.15.37:6821","nonce":2746074720}]},"public_addr":"172.21.15.37:6817/2746074720","cluster_addr":"172.21.15.37:6819/2746074720","heartbeat_back_addr":"172.21.15.37:6823/2746074720","heartbeat_front_addr":"172.21.15.37:6821/2746074720","state":["exists","up"]},{"osd":1,"uuid":"973159de-c2ea-4083-aecc-69373d1c1132","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":21,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.37:6800","nonce":1690341662},{"type":"v1","addr":"172.21.15.37:6801","nonce":1690341662}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.37:6802","nonce":1690341662},{"type":"v1","addr":"172.21.15.37:6803","nonce":1690341662}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.37:6806","nonce":1690341662},{"type":"v1","addr":"172.21.15.37:6807","nonce":1690341662}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.37:6804","nonce":1690341662},{"type":"v1","addr":"172.21.15.37:6805","nonce":1690341662}]},"public_addr":"172.21.15.37:6801/1690341662","cluster_addr":"172.21.15.37:6803/1690341662","heartbeat_back_addr":"172.21.15.37:6807/1690341662","heartbeat_front_addr":"172.21.15.37:6805/1690341662","state":["exists","up"]},{"osd":2,"uuid":"4ae7a0e9-cb3f-484c-a75f-f3766b721c47","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":21,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.37:6808","nonce":2754669649},{"type":"v1","addr":"172.21.15.37:6809","nonce":2754669649}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.37:6810","nonce":2754669649},{"type":"v1","addr":"172.21.15.37:6811","nonce":2754669649}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.37:6814","nonce":2754669649},{"type":"v1","addr":"172.21.15.37:6815","nonce":2754669649}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.37:6812","nonce":2754669649},{"type":"v1","addr":"172.21.15.37:6813","nonce":2754669649}]},"public_addr":"172.21.15.37:6809/2754669649","cluster_addr":"172.21.15.37:6811/2754669649","heartbeat_back_addr":"172.21.15.37:6815/2754669649","heartbeat_front_addr":"172.21.15.37:6813/2754669649","state":["exists","up"]},{"osd":3,"uuid":"1ba4a746-1608-47a7-ace2-960a5cfae934","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":21,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.37:6824","nonce":1815290618},{"type":"v1","addr":"172.21.15.37:6825","nonce":1815290618}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.37:6826","nonce":1815290618},{"type":"v1","addr":"172.21.15.37:6827","nonce":1815290618}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.37:6830","nonce":1815290618},{"type":"v1","addr":"172.21.15.37:6831","nonce":1815290618}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.37:6828","nonce":1815290618},{"type":"v1","addr":"172.21.15.37:6829","nonce":1815290618}]},"public_addr":"172.21.15.37:6825/1815290618","cluster_addr":"172.21.15.37:6827/1815290618","heartbeat_back_addr":"172.21.15.37:6831/1815290618","heartbeat_front_addr":"172.21.15.37:6829/1815290618","state":["exists","up"]},{"osd":4,"uuid":"d4f59f5d-0413-4b79-87ee-bb212c1fcf62","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":21,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6800","nonce":4051532063},{"type":"v1","addr":"172.21.15.203:6801","nonce":4051532063}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6802","nonce":4051532063},{"type":"v1","addr":"172.21.15.203:6803","nonce":4051532063}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6806","nonce":4051532063},{"type":"v1","addr":"172.21.15.203:6807","nonce":4051532063}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6804","nonce":4051532063},{"type":"v1","addr":"172.21.15.203:6805","nonce":4051532063}]},"public_addr":"172.21.15.203:6801/4051532063","cluster_addr":"172.21.15.203:6803/4051532063","heartbeat_back_addr":"172.21.15.203:6807/4051532063","heartbeat_front_addr":"172.21.15.203:6805/4051532063","state":["exists","up"]},{"osd":5,"uuid":"69710a0e-41d8-4e7f-9a10-17ce540b67be","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":21,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6824","nonce":1203822890},{"type":"v1","addr":"172.21.15.203:6825","nonce":1203822890}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6826","nonce":1203822890},{"type":"v1","addr":"172.21.15.203:6827","nonce":1203822890}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6830","nonce":1203822890},{"type":"v1","addr":"172.21.15.203:6831","nonce":1203822890}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6828","nonce":1203822890},{"type":"v1","addr":"172.21.15.203:6829","nonce":1203822890}]},"public_addr":"172.21.15.203:6825/1203822890","cluster_addr":"172.21.15.203:6827/1203822890","heartbeat_back_addr":"172.21.15.203:6831/1203822890","heartbeat_front_addr":"172.21.15.203:6829/1203822890","state":["exists","up"]},{"osd":6,"uuid":"f29dd047-3b48-473d-a235-6642a6293f20","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":21,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6810","nonce":2309615014},{"type":"v1","addr":"172.21.15.203:6812","nonce":2309615014}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6814","nonce":2309615014},{"type":"v1","addr":"172.21.15.203:6816","nonce":2309615014}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6822","nonce":2309615014},{"type":"v1","addr":"172.21.15.203:6823","nonce":2309615014}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6818","nonce":2309615014},{"type":"v1","addr":"172.21.15.203:6820","nonce":2309615014}]},"public_addr":"172.21.15.203:6812/2309615014","cluster_addr":"172.21.15.203:6816/2309615014","heartbeat_back_addr":"172.21.15.203:6823/2309615014","heartbeat_front_addr":"172.21.15.203:6820/2309615014","state":["exists","up"]},{"osd":7,"uuid":"6e691644-378b-4fdc-8e86-fb5f56e31f29","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":21,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6808","nonce":3688099014},{"type":"v1","addr":"172.21.15.203:6809","nonce":3688099014}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6811","nonce":3688099014},{"type":"v1","addr":"172.21.15.203:6813","nonce":3688099014}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6819","nonce":3688099014},{"type":"v1","addr":"172.21.15.203:6821","nonce":3688099014}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6815","nonce":3688099014},{"type":"v1","addr":"172.21.15.203:6817","nonce":3688099014}]},"public_addr":"172.21.15.203:6809/3688099014","cluster_addr":"172.21.15.203:6813/3688099014","heartbeat_back_addr":"172.21.15.203:6821/3688099014","heartbeat_front_addr":"172.21.15.203:6817/3688099014","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T10:28:51.419141+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T10:28:51.366448+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T10:28:51.348276+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T10:28:51.380519+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T10:28:51.105812+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T10:28:51.384496+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T10:28:51.388755+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T10:28:51.366958+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-04-25T10:29:44.228 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:44.225+0000 7f7a3dd1c640 1 -- 172.21.15.37:0/548292398 >> [v2:172.21.15.203:6832/2418205237,v1:172.21.15.203:6833/2418205237] conn(0x7f7a180744b0 msgr2=0x7f7a180769c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:29:44.228 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:44.225+0000 7f7a3dd1c640 1 --2- 172.21.15.37:0/548292398 >> [v2:172.21.15.203:6832/2418205237,v1:172.21.15.203:6833/2418205237] conn(0x7f7a180744b0 0x7f7a180769c0 secure :-1 s=READY pgs=139 cs=0 l=1 rev1=1 crypto rx=0x7f7a38070e30 tx=0x7f7a20008040 comp rx=0 tx=0).stop 2024-04-25T10:29:44.228 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:44.225+0000 7f7a3dd1c640 1 -- 172.21.15.37:0/548292398 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f7a380d1c20 msgr2=0x7f7a381d5230 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:29:44.228 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:44.225+0000 7f7a3dd1c640 1 --2- 172.21.15.37:0/548292398 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f7a380d1c20 0x7f7a381d5230 secure :-1 s=READY pgs=251 cs=0 l=1 rev1=1 crypto rx=0x7f7a2800b410 tx=0x7f7a2800b8d0 comp rx=0 tx=0).stop 2024-04-25T10:29:44.228 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:44.225+0000 7f7a3dd1c640 1 -- 172.21.15.37:0/548292398 shutdown_connections 2024-04-25T10:29:44.228 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:44.225+0000 7f7a3dd1c640 1 --2- 172.21.15.37:0/548292398 >> [v2:172.21.15.203:6832/2418205237,v1:172.21.15.203:6833/2418205237] conn(0x7f7a180744b0 0x7f7a180769c0 unknown :-1 s=CLOSED pgs=139 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:29:44.228 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:44.225+0000 7f7a3dd1c640 1 --2- 172.21.15.37:0/548292398 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f7a381d3a60 0x7f7a3809f850 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:29:44.228 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:44.225+0000 7f7a3dd1c640 1 --2- 172.21.15.37:0/548292398 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f7a380d1c20 0x7f7a381d5230 unknown :-1 s=CLOSED pgs=251 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:29:44.228 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:44.225+0000 7f7a3dd1c640 1 --2- 172.21.15.37:0/548292398 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f7a380b71b0 0x7f7a380b1b10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:29:44.228 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:44.225+0000 7f7a3dd1c640 1 -- 172.21.15.37:0/548292398 >> 172.21.15.37:0/548292398 conn(0x7f7a380d03d0 msgr2=0x7f7a380723f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:29:44.228 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:44.225+0000 7f7a3dd1c640 1 -- 172.21.15.37:0/548292398 shutdown_connections 2024-04-25T10:29:44.228 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:44.226+0000 7f7a3dd1c640 1 -- 172.21.15.37:0/548292398 wait complete. 2024-04-25T10:29:44.239 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2024-04-25T10:28:56.202153+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '19', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 8, 'score_stable': 8, 'optimal_score': 0.25, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 2, 'pool_name': 'cephfs_metadata', 'create_time': '2024-04-25T10:29:06.507244+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '22', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_autoscale_bias': 4, 'pg_num_min': 16, 'recovery_priority': 5}, 'application_metadata': {'cephfs': {'metadata': 'cephfs'}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 2.25, 'score_stable': 2.25, 'optimal_score': 1, 'raw_score_acting': 2.25, 'raw_score_stable': 2.25, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 3, 'pool_name': 'cephfs_data', 'create_time': '2024-04-25T10:29:07.347844+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 64, 'pg_placement_num': 64, 'pg_placement_num_target': 64, 'pg_num_target': 64, 'pg_num_pending': 64, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '23', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.8999999761581421}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 1.5, 'score_stable': 1.5, 'optimal_score': 1, 'raw_score_acting': 1.5, 'raw_score_stable': 1.5, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}] 2024-04-25T10:29:44.239 DEBUG:teuthology.orchestra.run.smithi037:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2024-04-25T10:29:44.354 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:44.352+0000 7f1175a4c640 1 Processor -- start 2024-04-25T10:29:44.354 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:44.352+0000 7f1175a4c640 1 -- start start 2024-04-25T10:29:44.358 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:44.353+0000 7f1175a4c640 1 --2- >> v2:172.21.15.203:3300/0 conn(0x7f117015e200 0x7f11700d31f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:29:44.358 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:44.353+0000 7f1175a4c640 1 -- --> v1:172.21.15.37:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f117006f250 con 0x7f117009b700 2024-04-25T10:29:44.358 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:44.353+0000 7f1175a4c640 1 -- --> v1:172.21.15.203:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f117006f500 con 0x7f11700d20a0 2024-04-25T10:29:44.358 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:44.353+0000 7f1175a4c640 1 -- --> v2:172.21.15.203:3300/0 -- mon_getmap magic: 0 -- 0x7f117006f7e0 con 0x7f117015e200 2024-04-25T10:29:44.358 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:44.354+0000 7f116f7fe640 1 --1- >> v1:172.21.15.37:6789/0 conn(0x7f117009b700 0x7f117009bb10 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.37:6789/0 says I am v1:172.21.15.37:49842/0 (socket says 172.21.15.37:49842) 2024-04-25T10:29:44.358 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:44.354+0000 7f116f7fe640 1 -- 172.21.15.37:0/3555294610 learned_addr learned my addr 172.21.15.37:0/3555294610 (peer_addr_for_me v1:172.21.15.37:0/0) 2024-04-25T10:29:44.358 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:44.354+0000 7f1167fff640 1 --2- 172.21.15.37:0/3555294610 >> v2:172.21.15.203:3300/0 conn(0x7f117015e200 0x7f11700d31f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:29:44.358 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:44.354+0000 7f116e7fc640 1 -- 172.21.15.37:0/3555294610 <== mon.0 v1:172.21.15.37:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 4182170670 0 0) 0x7f117006f250 con 0x7f117009b700 2024-04-25T10:29:44.358 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:44.354+0000 7f116e7fc640 1 -- 172.21.15.37:0/3555294610 --> v1:172.21.15.37:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f115c003520 con 0x7f117009b700 2024-04-25T10:29:44.358 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:44.354+0000 7f116e7fc640 1 -- 172.21.15.37:0/3555294610 <== mon.0 v1:172.21.15.37:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 2104020153 0 0) 0x7f115c003520 con 0x7f117009b700 2024-04-25T10:29:44.359 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:44.354+0000 7f116e7fc640 1 -- 172.21.15.37:0/3555294610 >> v2:172.21.15.203:3300/0 conn(0x7f117015e200 msgr2=0x7f11700d31f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:29:44.359 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:44.354+0000 7f116e7fc640 1 --2- 172.21.15.37:0/3555294610 >> v2:172.21.15.203:3300/0 conn(0x7f117015e200 0x7f11700d31f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:29:44.359 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:44.354+0000 7f116e7fc640 1 -- 172.21.15.37:0/3555294610 >> v1:172.21.15.203:6789/0 conn(0x7f11700d20a0 legacy=0x7f117006eb40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:29:44.359 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:44.354+0000 7f116e7fc640 1 -- 172.21.15.37:0/3555294610 --> v1:172.21.15.37:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f11700d2710 con 0x7f117009b700 2024-04-25T10:29:44.359 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:44.354+0000 7f116e7fc640 1 -- 172.21.15.37:0/3555294610 <== mon.0 v1:172.21.15.37:6789/0 3 ==== mon_map magic: 0 ==== 413+0+0 (unknown 1577794600 0 0) 0x7f11680019f0 con 0x7f117009b700 2024-04-25T10:29:44.359 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:44.354+0000 7f116e7fc640 1 -- 172.21.15.37:0/3555294610 >> v1:172.21.15.37:6789/0 conn(0x7f117009b700 legacy=0x7f117009bb10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:29:44.359 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:44.354+0000 7f116e7fc640 1 --2- 172.21.15.37:0/3555294610 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f115c0039b0 0x7f115c003e20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:29:44.359 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:44.354+0000 7f116e7fc640 1 --2- 172.21.15.37:0/3555294610 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f115c004800 0x7f115c004c10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:29:44.359 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:44.354+0000 7f116e7fc640 1 --2- 172.21.15.37:0/3555294610 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f115c0051c0 0x7f115c00b9c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:29:44.359 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:44.354+0000 7f116e7fc640 1 -- 172.21.15.37:0/3555294610 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 -- 0x7f117006f7e0 con 0x7f115c0051c0 2024-04-25T10:29:44.359 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:44.354+0000 7f116e7fc640 1 -- 172.21.15.37:0/3555294610 --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_getmap magic: 0 -- 0x7f115c00c160 con 0x7f115c004800 2024-04-25T10:29:44.359 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:44.354+0000 7f116e7fc640 1 -- 172.21.15.37:0/3555294610 --> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] -- mon_getmap magic: 0 -- 0x7f115c00c360 con 0x7f115c0039b0 2024-04-25T10:29:44.359 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:44.354+0000 7f116f7fe640 1 --2- 172.21.15.37:0/3555294610 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f115c0051c0 0x7f115c00b9c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:29:44.359 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:44.355+0000 7f1167fff640 1 --2- 172.21.15.37:0/3555294610 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f115c004800 0x7f115c004c10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:29:44.359 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:44.355+0000 7f116effd640 1 --2- 172.21.15.37:0/3555294610 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f115c0039b0 0x7f115c003e20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:29:44.360 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:44.355+0000 7f116f7fe640 1 -- 172.21.15.37:0/3555294610 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f115c0039b0 msgr2=0x7f115c003e20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:29:44.360 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:44.355+0000 7f116f7fe640 1 --2- 172.21.15.37:0/3555294610 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f115c0039b0 0x7f115c003e20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:29:44.360 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:44.355+0000 7f116f7fe640 1 -- 172.21.15.37:0/3555294610 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f115c004800 msgr2=0x7f115c004c10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:29:44.360 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:44.355+0000 7f116f7fe640 1 --2- 172.21.15.37:0/3555294610 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f115c004800 0x7f115c004c10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:29:44.360 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:44.355+0000 7f116f7fe640 1 -- 172.21.15.37:0/3555294610 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f115c003520 con 0x7f115c0051c0 2024-04-25T10:29:44.360 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:44.355+0000 7f116f7fe640 1 -- 172.21.15.37:0/3555294610 reap_dead start 2024-04-25T10:29:44.360 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:44.356+0000 7f116f7fe640 1 --2- 172.21.15.37:0/3555294610 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f115c0051c0 0x7f115c00b9c0 secure :-1 s=READY pgs=253 cs=0 l=1 rev1=1 crypto rx=0x7f11700701f0 tx=0x7f115c005830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=4e9c56c7b99a8366 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:29:44.360 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:44.356+0000 7f116e7fc640 1 -- 172.21.15.37:0/3555294610 <== mon.0 v2:172.21.15.37:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f116800a4a0 con 0x7f115c0051c0 2024-04-25T10:29:44.360 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:44.356+0000 7f116e7fc640 1 -- 172.21.15.37:0/3555294610 <== mon.0 v2:172.21.15.37:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f116802f070 con 0x7f115c0051c0 2024-04-25T10:29:44.360 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:44.356+0000 7f116e7fc640 1 -- 172.21.15.37:0/3555294610 <== mon.0 v2:172.21.15.37:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f116802bbe0 con 0x7f115c0051c0 2024-04-25T10:29:44.361 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:44.357+0000 7f1175a4c640 1 -- 172.21.15.37:0/3555294610 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f115c0051c0 msgr2=0x7f115c00b9c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:29:44.361 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:44.357+0000 7f1175a4c640 1 --2- 172.21.15.37:0/3555294610 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f115c0051c0 0x7f115c00b9c0 secure :-1 s=READY pgs=253 cs=0 l=1 rev1=1 crypto rx=0x7f11700701f0 tx=0x7f115c005830 comp rx=0 tx=0).stop 2024-04-25T10:29:44.361 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:44.357+0000 7f1175a4c640 1 -- 172.21.15.37:0/3555294610 shutdown_connections 2024-04-25T10:29:44.361 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:44.357+0000 7f1175a4c640 1 --2- 172.21.15.37:0/3555294610 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f115c0051c0 0x7f115c00b9c0 unknown :-1 s=CLOSED pgs=253 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:29:44.362 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:44.357+0000 7f1175a4c640 1 -- 172.21.15.37:0/3555294610 >> 172.21.15.37:0/3555294610 conn(0x7f11700b7620 msgr2=0x7f11700b7c50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:29:44.362 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:44.357+0000 7f1175a4c640 1 -- 172.21.15.37:0/3555294610 shutdown_connections 2024-04-25T10:29:44.362 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:44.357+0000 7f1175a4c640 1 -- 172.21.15.37:0/3555294610 wait complete. 2024-04-25T10:29:44.362 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:44.358+0000 7f1175a4c640 1 Processor -- start 2024-04-25T10:29:44.362 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:44.359+0000 7f1175a4c640 1 -- start start 2024-04-25T10:29:44.364 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:44.360+0000 7f1175a4c640 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f115c0051c0 0x7f11701e87e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:29:44.364 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:44.360+0000 7f1175a4c640 1 --2- >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f1170152e70 0x7f117013fb60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:29:44.364 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:44.360+0000 7f1175a4c640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f1170154720 0x7f1170140290 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:29:44.364 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:44.360+0000 7f1175a4c640 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 -- 0x7f11700d31f0 con 0x7f1170154720 2024-04-25T10:29:44.364 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:44.360+0000 7f1175a4c640 1 -- --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_getmap magic: 0 -- 0x7f11700d2cd0 con 0x7f115c0051c0 2024-04-25T10:29:44.364 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:44.360+0000 7f1175a4c640 1 -- --> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] -- mon_getmap magic: 0 -- 0x7f11700d3370 con 0x7f1170152e70 2024-04-25T10:29:44.364 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:44.360+0000 7f116f7fe640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f1170154720 0x7f1170140290 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:29:44.364 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:44.360+0000 7f116f7fe640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f1170154720 0x7f1170140290 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.37:3300/0 says I am v2:172.21.15.37:59516/0 (socket says 172.21.15.37:59516) 2024-04-25T10:29:44.364 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:44.360+0000 7f116f7fe640 1 -- 172.21.15.37:0/4242689937 learned_addr learned my addr 172.21.15.37:0/4242689937 (peer_addr_for_me v2:172.21.15.37:0/0) 2024-04-25T10:29:44.364 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:44.360+0000 7f116effd640 1 --2- 172.21.15.37:0/4242689937 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f115c0051c0 0x7f11701e87e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:29:44.364 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:44.360+0000 7f1167fff640 1 --2- 172.21.15.37:0/4242689937 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f1170152e70 0x7f117013fb60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:29:44.364 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:44.360+0000 7f116f7fe640 1 -- 172.21.15.37:0/4242689937 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f1170152e70 msgr2=0x7f117013fb60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:29:44.364 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:44.360+0000 7f116f7fe640 1 --2- 172.21.15.37:0/4242689937 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f1170152e70 0x7f117013fb60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:29:44.364 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:44.360+0000 7f116f7fe640 1 -- 172.21.15.37:0/4242689937 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f115c0051c0 msgr2=0x7f11701e87e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:29:44.365 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:44.360+0000 7f116f7fe640 1 --2- 172.21.15.37:0/4242689937 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f115c0051c0 0x7f11701e87e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:29:44.365 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:44.360+0000 7f116f7fe640 1 -- 172.21.15.37:0/4242689937 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f1170153680 con 0x7f1170154720 2024-04-25T10:29:44.365 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:44.361+0000 7f116f7fe640 1 --2- 172.21.15.37:0/4242689937 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f1170154720 0x7f1170140290 secure :-1 s=READY pgs=254 cs=0 l=1 rev1=1 crypto rx=0x7f116800a880 tx=0x7f116802d070 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:29:44.365 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:44.361+0000 7f116cff9640 1 -- 172.21.15.37:0/4242689937 <== mon.0 v2:172.21.15.37:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1168045070 con 0x7f1170154720 2024-04-25T10:29:44.365 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:44.361+0000 7f116cff9640 1 -- 172.21.15.37:0/4242689937 <== mon.0 v2:172.21.15.37:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f116800a480 con 0x7f1170154720 2024-04-25T10:29:44.365 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:44.361+0000 7f1175a4c640 1 -- 172.21.15.37:0/4242689937 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f1170153960 con 0x7f1170154720 2024-04-25T10:29:44.365 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:44.361+0000 7f116cff9640 1 -- 172.21.15.37:0/4242689937 <== mon.0 v2:172.21.15.37:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f11680406c0 con 0x7f1170154720 2024-04-25T10:29:44.365 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:44.361+0000 7f1175a4c640 1 -- 172.21.15.37:0/4242689937 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f1170153f20 con 0x7f1170154720 2024-04-25T10:29:44.366 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:44.362+0000 7f1175a4c640 1 -- 172.21.15.37:0/4242689937 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f117006a140 con 0x7f1170154720 2024-04-25T10:29:44.366 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:44.362+0000 7f116cff9640 1 -- 172.21.15.37:0/4242689937 <== mon.0 v2:172.21.15.37:3300/0 4 ==== mgrmap(e 5) ==== 96782+0+0 (secure 0 0 0) 0x7f1168048070 con 0x7f1170154720 2024-04-25T10:29:44.366 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:44.363+0000 7f116cff9640 1 --2- 172.21.15.37:0/4242689937 >> [v2:172.21.15.203:6832/2418205237,v1:172.21.15.203:6833/2418205237] conn(0x7f11500744e0 0x7f11500769f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:29:44.366 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:44.363+0000 7f116cff9640 1 -- 172.21.15.37:0/4242689937 <== mon.0 v2:172.21.15.37:3300/0 5 ==== osd_map(23..23 src has 1..23) ==== 5709+0+0 (secure 0 0 0) 0x7f11680b5440 con 0x7f1170154720 2024-04-25T10:29:44.366 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:44.363+0000 7f116effd640 1 --2- 172.21.15.37:0/4242689937 >> [v2:172.21.15.203:6832/2418205237,v1:172.21.15.203:6833/2418205237] conn(0x7f11500744e0 0x7f11500769f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:29:44.366 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:44.364+0000 7f116effd640 1 --2- 172.21.15.37:0/4242689937 >> [v2:172.21.15.203:6832/2418205237,v1:172.21.15.203:6833/2418205237] conn(0x7f11500744e0 0x7f11500769f0 secure :-1 s=READY pgs=140 cs=0 l=1 rev1=1 crypto rx=0x7f1160000c00 tx=0x7f1160009040 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:29:44.374 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:44.372+0000 7f116cff9640 1 -- 172.21.15.37:0/4242689937 <== mon.0 v2:172.21.15.37:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187246 (secure 0 0 0) 0x7f116807fd20 con 0x7f1170154720 2024-04-25T10:29:44.559 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:44.557+0000 7f1175a4c640 1 -- 172.21.15.37:0/4242689937 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) -- 0x7f117006f1c0 con 0x7f1170154720 2024-04-25T10:29:44.559 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:44.557+0000 7f116cff9640 1 -- 172.21.15.37:0/4242689937 <== mon.0 v2:172.21.15.37:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v23) ==== 93+0+10 (secure 0 0 0) 0x7f116807f620 con 0x7f1170154720 2024-04-25T10:29:44.560 INFO:teuthology.orchestra.run.smithi037.stdout:pg_num: 1 2024-04-25T10:29:44.562 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:44.559+0000 7f1175a4c640 1 -- 172.21.15.37:0/4242689937 >> [v2:172.21.15.203:6832/2418205237,v1:172.21.15.203:6833/2418205237] conn(0x7f11500744e0 msgr2=0x7f11500769f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:29:44.562 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:44.559+0000 7f1175a4c640 1 --2- 172.21.15.37:0/4242689937 >> [v2:172.21.15.203:6832/2418205237,v1:172.21.15.203:6833/2418205237] conn(0x7f11500744e0 0x7f11500769f0 secure :-1 s=READY pgs=140 cs=0 l=1 rev1=1 crypto rx=0x7f1160000c00 tx=0x7f1160009040 comp rx=0 tx=0).stop 2024-04-25T10:29:44.562 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:44.559+0000 7f1175a4c640 1 -- 172.21.15.37:0/4242689937 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f1170154720 msgr2=0x7f1170140290 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:29:44.562 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:44.559+0000 7f1175a4c640 1 --2- 172.21.15.37:0/4242689937 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f1170154720 0x7f1170140290 secure :-1 s=READY pgs=254 cs=0 l=1 rev1=1 crypto rx=0x7f116800a880 tx=0x7f116802d070 comp rx=0 tx=0).stop 2024-04-25T10:29:44.562 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:44.559+0000 7f1175a4c640 1 -- 172.21.15.37:0/4242689937 shutdown_connections 2024-04-25T10:29:44.562 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:44.559+0000 7f1175a4c640 1 --2- 172.21.15.37:0/4242689937 >> [v2:172.21.15.203:6832/2418205237,v1:172.21.15.203:6833/2418205237] conn(0x7f11500744e0 0x7f11500769f0 unknown :-1 s=CLOSED pgs=140 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:29:44.562 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:44.559+0000 7f1175a4c640 1 --2- 172.21.15.37:0/4242689937 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f1170154720 0x7f1170140290 unknown :-1 s=CLOSED pgs=254 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:29:44.562 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:44.559+0000 7f1175a4c640 1 --2- 172.21.15.37:0/4242689937 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f1170152e70 0x7f117013fb60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:29:44.562 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:44.559+0000 7f1175a4c640 1 --2- 172.21.15.37:0/4242689937 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f115c0051c0 0x7f11701e87e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:29:44.562 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:44.559+0000 7f1175a4c640 1 -- 172.21.15.37:0/4242689937 >> 172.21.15.37:0/4242689937 conn(0x7f11700b7620 msgr2=0x7f1170075d00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:29:44.563 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:44.559+0000 7f1175a4c640 1 -- 172.21.15.37:0/4242689937 shutdown_connections 2024-04-25T10:29:44.563 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:44.559+0000 7f1175a4c640 1 -- 172.21.15.37:0/4242689937 wait complete. 2024-04-25T10:29:44.573 DEBUG:teuthology.orchestra.run.smithi037:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_metadata pg_num 2024-04-25T10:29:44.687 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:44.685+0000 7f4f40ae0640 1 Processor -- start 2024-04-25T10:29:44.687 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:44.685+0000 7f4f40ae0640 1 -- start start 2024-04-25T10:29:44.689 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:44.686+0000 7f4f40ae0640 1 --2- >> v2:172.21.15.37:3300/0 conn(0x7f4f3c0d5160 0x7f4f3c0b4e80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:29:44.689 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:44.686+0000 7f4f40ae0640 1 --2- >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f4f3c0d3230 0x7f4f3c0b9060 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:29:44.689 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:44.686+0000 7f4f40ae0640 1 -- --> v1:172.21.15.37:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f4f3c0d1370 con 0x7f4f3c0d0990 2024-04-25T10:29:44.689 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:44.686+0000 7f4f40ae0640 1 -- --> v2:172.21.15.37:3300/0 -- mon_getmap magic: 0 -- 0x7f4f3c0d1650 con 0x7f4f3c0d5160 2024-04-25T10:29:44.689 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:44.686+0000 7f4f40ae0640 1 -- --> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] -- mon_getmap magic: 0 -- 0x7f4f3c0d4680 con 0x7f4f3c0d3230 2024-04-25T10:29:44.689 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:44.686+0000 7f4f3a59c640 1 --2- >> v2:172.21.15.37:3300/0 conn(0x7f4f3c0d5160 0x7f4f3c0b4e80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:29:44.690 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:44.686+0000 7f4f31d9b640 1 --1- >> v1:172.21.15.37:6789/0 conn(0x7f4f3c0d0990 0x7f4f3c0d2b20 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.37:6789/0 says I am v1:172.21.15.37:49852/0 (socket says 172.21.15.37:49852) 2024-04-25T10:29:44.690 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:44.686+0000 7f4f31d9b640 1 -- 172.21.15.37:0/276732126 learned_addr learned my addr 172.21.15.37:0/276732126 (peer_addr_for_me v1:172.21.15.37:0/0) 2024-04-25T10:29:44.690 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:44.687+0000 7f4f3ad9d640 1 --2- 172.21.15.37:0/276732126 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f4f3c0d3230 0x7f4f3c0b9060 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:29:44.690 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:44.687+0000 7f4f3a59c640 1 -- 172.21.15.37:0/276732126 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f4f3c0d3230 msgr2=0x7f4f3c0b9060 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:29:44.690 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:44.687+0000 7f4f3a59c640 1 --2- 172.21.15.37:0/276732126 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f4f3c0d3230 0x7f4f3c0b9060 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:29:44.690 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:44.687+0000 7f4f3a59c640 1 -- 172.21.15.37:0/276732126 >> v1:172.21.15.37:6789/0 conn(0x7f4f3c0d0990 legacy=0x7f4f3c0d2b20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:29:44.690 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:44.687+0000 7f4f39d9b640 1 -- 172.21.15.37:0/276732126 <== mon.0 v1:172.21.15.37:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 3631411838 0 0) 0x7f4f3c0d1370 con 0x7f4f3c0d0990 2024-04-25T10:29:44.690 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:44.687+0000 7f4f3a59c640 1 -- 172.21.15.37:0/276732126 --> v2:172.21.15.37:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f4f3c0d4960 con 0x7f4f3c0d5160 2024-04-25T10:29:44.691 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:44.688+0000 7f4f3a59c640 1 --2- 172.21.15.37:0/276732126 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f4f3c0d5160 0x7f4f3c0b4e80 secure :-1 s=READY pgs=256 cs=0 l=1 rev1=1 crypto rx=0x7f4f2c0097a0 tx=0x7f4f2c02a060 comp rx=0 tx=0).ready entity=mon.0 client_cookie=adcf18d0b7929d3e server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:29:44.691 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:44.688+0000 7f4f39d9b640 1 -- 172.21.15.37:0/276732126 <== mon.0 v2:172.21.15.37:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4f2c037070 con 0x7f4f3c0d5160 2024-04-25T10:29:44.691 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:44.688+0000 7f4f39d9b640 1 -- 172.21.15.37:0/276732126 <== mon.0 v2:172.21.15.37:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f4f2c02adb0 con 0x7f4f3c0d5160 2024-04-25T10:29:44.691 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:44.688+0000 7f4f39d9b640 1 -- 172.21.15.37:0/276732126 <== mon.0 v2:172.21.15.37:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4f2c032660 con 0x7f4f3c0d5160 2024-04-25T10:29:44.692 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:44.688+0000 7f4f40ae0640 1 -- 172.21.15.37:0/276732126 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f4f3c0d5160 msgr2=0x7f4f3c0b4e80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:29:44.692 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:44.688+0000 7f4f40ae0640 1 --2- 172.21.15.37:0/276732126 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f4f3c0d5160 0x7f4f3c0b4e80 secure :-1 s=READY pgs=256 cs=0 l=1 rev1=1 crypto rx=0x7f4f2c0097a0 tx=0x7f4f2c02a060 comp rx=0 tx=0).stop 2024-04-25T10:29:44.692 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:44.688+0000 7f4f40ae0640 1 -- 172.21.15.37:0/276732126 shutdown_connections 2024-04-25T10:29:44.692 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:44.688+0000 7f4f40ae0640 1 --2- 172.21.15.37:0/276732126 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f4f3c0d3230 0x7f4f3c0b9060 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:29:44.692 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:44.688+0000 7f4f40ae0640 1 --2- 172.21.15.37:0/276732126 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f4f3c0d5160 0x7f4f3c0b4e80 unknown :-1 s=CLOSED pgs=256 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:29:44.692 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:44.688+0000 7f4f40ae0640 1 -- 172.21.15.37:0/276732126 >> 172.21.15.37:0/276732126 conn(0x7f4f3c0b7a80 msgr2=0x7f4f3c0b7e90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:29:44.692 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:44.688+0000 7f4f40ae0640 1 -- 172.21.15.37:0/276732126 shutdown_connections 2024-04-25T10:29:44.692 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:44.688+0000 7f4f40ae0640 1 -- 172.21.15.37:0/276732126 wait complete. 2024-04-25T10:29:44.693 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:44.691+0000 7f4f40ae0640 1 Processor -- start 2024-04-25T10:29:44.693 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:44.691+0000 7f4f40ae0640 1 -- start start 2024-04-25T10:29:44.699 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:44.692+0000 7f4f40ae0640 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f4f3c0d0990 0x7f4f3c153e10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:29:44.699 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:44.692+0000 7f4f40ae0640 1 --2- >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f4f3c0d5160 0x7f4f3c0cf130 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:29:44.699 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:44.692+0000 7f4f40ae0640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f4f3c1f99c0 0x7f4f3c0cf670 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:29:44.699 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:44.692+0000 7f4f40ae0640 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 -- 0x7f4f3c164c40 con 0x7f4f3c1f99c0 2024-04-25T10:29:44.699 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:44.692+0000 7f4f40ae0640 1 -- --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_getmap magic: 0 -- 0x7f4f3c164ac0 con 0x7f4f3c0d0990 2024-04-25T10:29:44.699 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:44.692+0000 7f4f40ae0640 1 -- --> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] -- mon_getmap magic: 0 -- 0x7f4f3c0d0da0 con 0x7f4f3c0d5160 2024-04-25T10:29:44.699 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:44.692+0000 7f4f3ad9d640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f4f3c1f99c0 0x7f4f3c0cf670 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:29:44.699 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:44.692+0000 7f4f3ad9d640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f4f3c1f99c0 0x7f4f3c0cf670 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.37:3300/0 says I am v2:172.21.15.37:59532/0 (socket says 172.21.15.37:59532) 2024-04-25T10:29:44.699 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:44.692+0000 7f4f3ad9d640 1 -- 172.21.15.37:0/1306486404 learned_addr learned my addr 172.21.15.37:0/1306486404 (peer_addr_for_me v2:172.21.15.37:0/0) 2024-04-25T10:29:44.699 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:44.692+0000 7f4f31d9b640 1 --2- 172.21.15.37:0/1306486404 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f4f3c0d5160 0x7f4f3c0cf130 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:29:44.699 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:44.692+0000 7f4f3a59c640 1 --2- 172.21.15.37:0/1306486404 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f4f3c0d0990 0x7f4f3c153e10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:29:44.699 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:44.693+0000 7f4f3ad9d640 1 -- 172.21.15.37:0/1306486404 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f4f3c0d5160 msgr2=0x7f4f3c0cf130 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:29:44.699 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:44.693+0000 7f4f3ad9d640 1 --2- 172.21.15.37:0/1306486404 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f4f3c0d5160 0x7f4f3c0cf130 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:29:44.699 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:44.693+0000 7f4f3ad9d640 1 -- 172.21.15.37:0/1306486404 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f4f3c0d0990 msgr2=0x7f4f3c153e10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:29:44.700 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:44.693+0000 7f4f3ad9d640 1 --2- 172.21.15.37:0/1306486404 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f4f3c0d0990 0x7f4f3c153e10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:29:44.700 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:44.693+0000 7f4f3ad9d640 1 -- 172.21.15.37:0/1306486404 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f4f3c1fa130 con 0x7f4f3c1f99c0 2024-04-25T10:29:44.700 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:44.693+0000 7f4f3ad9d640 1 --2- 172.21.15.37:0/1306486404 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f4f3c1f99c0 0x7f4f3c0cf670 secure :-1 s=READY pgs=257 cs=0 l=1 rev1=1 crypto rx=0x7f4f34007d70 tx=0x7f4f3400e4b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:29:44.700 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:44.693+0000 7f4f337fe640 1 -- 172.21.15.37:0/1306486404 <== mon.0 v2:172.21.15.37:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4f34014420 con 0x7f4f3c1f99c0 2024-04-25T10:29:44.700 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:44.693+0000 7f4f337fe640 1 -- 172.21.15.37:0/1306486404 <== mon.0 v2:172.21.15.37:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f4f34019070 con 0x7f4f3c1f99c0 2024-04-25T10:29:44.700 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:44.693+0000 7f4f40ae0640 1 -- 172.21.15.37:0/1306486404 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f4f3c1fa410 con 0x7f4f3c1f99c0 2024-04-25T10:29:44.700 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:44.693+0000 7f4f337fe640 1 -- 172.21.15.37:0/1306486404 <== mon.0 v2:172.21.15.37:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4f34014820 con 0x7f4f3c1f99c0 2024-04-25T10:29:44.700 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:44.693+0000 7f4f40ae0640 1 -- 172.21.15.37:0/1306486404 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f4f3c0ccde0 con 0x7f4f3c1f99c0 2024-04-25T10:29:44.700 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:44.694+0000 7f4f40ae0640 1 -- 172.21.15.37:0/1306486404 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f4f3c0ccc00 con 0x7f4f3c1f99c0 2024-04-25T10:29:44.700 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:44.695+0000 7f4f337fe640 1 -- 172.21.15.37:0/1306486404 <== mon.0 v2:172.21.15.37:3300/0 4 ==== mgrmap(e 5) ==== 96782+0+0 (secure 0 0 0) 0x7f4f340149c0 con 0x7f4f3c1f99c0 2024-04-25T10:29:44.700 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:44.695+0000 7f4f337fe640 1 --2- 172.21.15.37:0/1306486404 >> [v2:172.21.15.203:6832/2418205237,v1:172.21.15.203:6833/2418205237] conn(0x7f4f1c0744b0 0x7f4f1c0769c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:29:44.700 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:44.695+0000 7f4f337fe640 1 -- 172.21.15.37:0/1306486404 <== mon.0 v2:172.21.15.37:3300/0 5 ==== osd_map(23..23 src has 1..23) ==== 5709+0+0 (secure 0 0 0) 0x7f4f340958b0 con 0x7f4f3c1f99c0 2024-04-25T10:29:44.700 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:44.695+0000 7f4f3a59c640 1 --2- 172.21.15.37:0/1306486404 >> [v2:172.21.15.203:6832/2418205237,v1:172.21.15.203:6833/2418205237] conn(0x7f4f1c0744b0 0x7f4f1c0769c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:29:44.700 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:44.696+0000 7f4f3a59c640 1 --2- 172.21.15.37:0/1306486404 >> [v2:172.21.15.203:6832/2418205237,v1:172.21.15.203:6833/2418205237] conn(0x7f4f1c0744b0 0x7f4f1c0769c0 secure :-1 s=READY pgs=141 cs=0 l=1 rev1=1 crypto rx=0x7f4f2c034000 tx=0x7f4f2c02e000 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:29:44.702 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:44.700+0000 7f4f337fe640 1 -- 172.21.15.37:0/1306486404 <== mon.0 v2:172.21.15.37:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187246 (secure 0 0 0) 0x7f4f3c0ccc00 con 0x7f4f3c1f99c0 2024-04-25T10:29:44.884 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:44.882+0000 7f4f40ae0640 1 -- 172.21.15.37:0/1306486404 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"} v 0) -- 0x7f4f3c1f9ec0 con 0x7f4f3c1f99c0 2024-04-25T10:29:44.885 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:44.882+0000 7f4f337fe640 1 -- 172.21.15.37:0/1306486404 <== mon.0 v2:172.21.15.37:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]=0 v23) ==== 104+0+11 (secure 0 0 0) 0x7f4f3405f9e0 con 0x7f4f3c1f99c0 2024-04-25T10:29:44.885 INFO:teuthology.orchestra.run.smithi037.stdout:pg_num: 32 2024-04-25T10:29:44.887 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:44.884+0000 7f4f40ae0640 1 -- 172.21.15.37:0/1306486404 >> [v2:172.21.15.203:6832/2418205237,v1:172.21.15.203:6833/2418205237] conn(0x7f4f1c0744b0 msgr2=0x7f4f1c0769c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:29:44.887 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:44.884+0000 7f4f40ae0640 1 --2- 172.21.15.37:0/1306486404 >> [v2:172.21.15.203:6832/2418205237,v1:172.21.15.203:6833/2418205237] conn(0x7f4f1c0744b0 0x7f4f1c0769c0 secure :-1 s=READY pgs=141 cs=0 l=1 rev1=1 crypto rx=0x7f4f2c034000 tx=0x7f4f2c02e000 comp rx=0 tx=0).stop 2024-04-25T10:29:44.887 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:44.884+0000 7f4f40ae0640 1 -- 172.21.15.37:0/1306486404 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f4f3c1f99c0 msgr2=0x7f4f3c0cf670 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:29:44.887 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:44.884+0000 7f4f40ae0640 1 --2- 172.21.15.37:0/1306486404 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f4f3c1f99c0 0x7f4f3c0cf670 secure :-1 s=READY pgs=257 cs=0 l=1 rev1=1 crypto rx=0x7f4f34007d70 tx=0x7f4f3400e4b0 comp rx=0 tx=0).stop 2024-04-25T10:29:44.887 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:44.884+0000 7f4f40ae0640 1 -- 172.21.15.37:0/1306486404 shutdown_connections 2024-04-25T10:29:44.887 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:44.884+0000 7f4f40ae0640 1 --2- 172.21.15.37:0/1306486404 >> [v2:172.21.15.203:6832/2418205237,v1:172.21.15.203:6833/2418205237] conn(0x7f4f1c0744b0 0x7f4f1c0769c0 unknown :-1 s=CLOSED pgs=141 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:29:44.887 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:44.884+0000 7f4f40ae0640 1 --2- 172.21.15.37:0/1306486404 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f4f3c1f99c0 0x7f4f3c0cf670 unknown :-1 s=CLOSED pgs=257 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:29:44.887 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:44.884+0000 7f4f40ae0640 1 --2- 172.21.15.37:0/1306486404 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f4f3c0d5160 0x7f4f3c0cf130 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:29:44.887 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:44.884+0000 7f4f40ae0640 1 --2- 172.21.15.37:0/1306486404 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f4f3c0d0990 0x7f4f3c153e10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:29:44.888 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:44.884+0000 7f4f40ae0640 1 -- 172.21.15.37:0/1306486404 >> 172.21.15.37:0/1306486404 conn(0x7f4f3c0b7a80 msgr2=0x7f4f3c0d00d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:29:44.888 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:44.884+0000 7f4f40ae0640 1 -- 172.21.15.37:0/1306486404 shutdown_connections 2024-04-25T10:29:44.888 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:44.884+0000 7f4f40ae0640 1 -- 172.21.15.37:0/1306486404 wait complete. 2024-04-25T10:29:44.898 DEBUG:teuthology.orchestra.run.smithi037:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data pg_num 2024-04-25T10:29:45.011 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:45.009+0000 7f2658e51640 1 Processor -- start 2024-04-25T10:29:45.012 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:45.009+0000 7f2658e51640 1 -- start start 2024-04-25T10:29:45.014 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:45.011+0000 7f2658e51640 1 --2- >> v2:172.21.15.203:3300/0 conn(0x7f265415a9f0 0x7f265406c540 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:29:45.014 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:45.011+0000 7f2658e51640 1 --2- >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f265406ca80 0x7f2654160910 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:29:45.014 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:45.011+0000 7f2658e51640 1 -- --> v1:172.21.15.37:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f265406ce90 con 0x7f2654160e50 2024-04-25T10:29:45.014 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:45.011+0000 7f2658e51640 1 -- --> v2:172.21.15.203:3300/0 -- mon_getmap magic: 0 -- 0x7f26540b6870 con 0x7f265415a9f0 2024-04-25T10:29:45.014 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:45.011+0000 7f2658e51640 1 -- --> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] -- mon_getmap magic: 0 -- 0x7f26541616d0 con 0x7f265406ca80 2024-04-25T10:29:45.014 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:45.011+0000 7f2652d9d640 1 --1- >> v1:172.21.15.37:6789/0 conn(0x7f2654160e50 0x7f2654145a30 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.37:6789/0 says I am v1:172.21.15.37:49858/0 (socket says 172.21.15.37:49858) 2024-04-25T10:29:45.014 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:45.011+0000 7f2652d9d640 1 -- 172.21.15.37:0/972152807 learned_addr learned my addr 172.21.15.37:0/972152807 (peer_addr_for_me v1:172.21.15.37:0/0) 2024-04-25T10:29:45.014 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:45.011+0000 7f2651d9b640 1 --2- 172.21.15.37:0/972152807 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f265406ca80 0x7f2654160910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:29:45.014 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:45.011+0000 7f265259c640 1 --2- 172.21.15.37:0/972152807 >> v2:172.21.15.203:3300/0 conn(0x7f265415a9f0 0x7f265406c540 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:29:45.014 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:45.011+0000 7f265159a640 1 -- 172.21.15.37:0/972152807 <== mon.0 v1:172.21.15.37:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 3712579052 0 0) 0x7f265406ce90 con 0x7f2654160e50 2024-04-25T10:29:45.014 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:45.011+0000 7f265159a640 1 -- 172.21.15.37:0/972152807 --> v1:172.21.15.37:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f2640003520 con 0x7f2654160e50 2024-04-25T10:29:45.014 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:45.011+0000 7f265159a640 1 -- 172.21.15.37:0/972152807 <== mon.0 v1:172.21.15.37:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 2247034732 0 0) 0x7f2640003520 con 0x7f2654160e50 2024-04-25T10:29:45.015 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:45.011+0000 7f2651d9b640 1 -- 172.21.15.37:0/972152807 >> v2:172.21.15.203:3300/0 conn(0x7f265415a9f0 msgr2=0x7f265406c540 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:29:45.015 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:45.011+0000 7f2651d9b640 1 --2- 172.21.15.37:0/972152807 >> v2:172.21.15.203:3300/0 conn(0x7f265415a9f0 0x7f265406c540 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:29:45.015 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:45.011+0000 7f2651d9b640 1 -- 172.21.15.37:0/972152807 >> v1:172.21.15.37:6789/0 conn(0x7f2654160e50 legacy=0x7f2654145a30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:29:45.015 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:45.011+0000 7f2651d9b640 1 -- 172.21.15.37:0/972152807 --> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f2654163e00 con 0x7f265406ca80 2024-04-25T10:29:45.016 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:45.012+0000 7f2651d9b640 1 --2- 172.21.15.37:0/972152807 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f265406ca80 0x7f2654160910 secure :-1 s=READY pgs=44 cs=0 l=1 rev1=1 crypto rx=0x7f263c0097a0 tx=0x7f263c02a110 comp rx=0 tx=0).ready entity=mon.2 client_cookie=23c3dc7990708f9b server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:29:45.016 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:45.012+0000 7f265159a640 1 -- 172.21.15.37:0/972152807 <== mon.2 v2:172.21.15.203:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f263c037070 con 0x7f265406ca80 2024-04-25T10:29:45.016 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:45.013+0000 7f265159a640 1 -- 172.21.15.37:0/972152807 <== mon.2 v2:172.21.15.203:3301/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f263c02aba0 con 0x7f265406ca80 2024-04-25T10:29:45.016 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:45.013+0000 7f265159a640 1 -- 172.21.15.37:0/972152807 <== mon.2 v2:172.21.15.203:3301/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f26440019f0 con 0x7f265406ca80 2024-04-25T10:29:45.016 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:45.013+0000 7f2658e51640 1 -- 172.21.15.37:0/972152807 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f265406ca80 msgr2=0x7f2654160910 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:29:45.016 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:45.013+0000 7f2658e51640 1 --2- 172.21.15.37:0/972152807 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f265406ca80 0x7f2654160910 secure :-1 s=READY pgs=44 cs=0 l=1 rev1=1 crypto rx=0x7f263c0097a0 tx=0x7f263c02a110 comp rx=0 tx=0).stop 2024-04-25T10:29:45.016 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:45.013+0000 7f2658e51640 1 -- 172.21.15.37:0/972152807 shutdown_connections 2024-04-25T10:29:45.017 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:45.013+0000 7f2658e51640 1 --2- 172.21.15.37:0/972152807 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f265406ca80 0x7f2654160910 unknown :-1 s=CLOSED pgs=44 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:29:45.017 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:45.013+0000 7f2658e51640 1 --2- 172.21.15.37:0/972152807 >> v2:172.21.15.203:3300/0 conn(0x7f265415a9f0 0x7f265406c540 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:29:45.017 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:45.013+0000 7f2658e51640 1 -- 172.21.15.37:0/972152807 >> 172.21.15.37:0/972152807 conn(0x7f26540b4560 msgr2=0x7f26540b4970 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:29:45.017 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:45.013+0000 7f2658e51640 1 -- 172.21.15.37:0/972152807 shutdown_connections 2024-04-25T10:29:45.017 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:45.013+0000 7f2658e51640 1 -- 172.21.15.37:0/972152807 wait complete. 2024-04-25T10:29:45.017 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:45.015+0000 7f2658e51640 1 Processor -- start 2024-04-25T10:29:45.017 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:45.015+0000 7f2658e51640 1 -- start start 2024-04-25T10:29:45.020 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:45.016+0000 7f2658e51640 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f265406ca80 0x7f26540cd1d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:29:45.020 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:45.017+0000 7f2658e51640 1 --2- >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f265415a9f0 0x7f265415f410 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:29:45.020 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:45.017+0000 7f2658e51640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f2654160e50 0x7f265415f950 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:29:45.020 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:45.017+0000 7f2658e51640 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 -- 0x7f26541646a0 con 0x7f2654160e50 2024-04-25T10:29:45.020 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:45.017+0000 7f2658e51640 1 -- --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_getmap magic: 0 -- 0x7f26541643c0 con 0x7f265406ca80 2024-04-25T10:29:45.020 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:45.017+0000 7f2658e51640 1 -- --> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] -- mon_getmap magic: 0 -- 0x7f2654164bc0 con 0x7f265415a9f0 2024-04-25T10:29:45.020 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:45.017+0000 7f2652d9d640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f2654160e50 0x7f265415f950 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:29:45.021 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:45.017+0000 7f2652d9d640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f2654160e50 0x7f265415f950 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.37:3300/0 says I am v2:172.21.15.37:59534/0 (socket says 172.21.15.37:59534) 2024-04-25T10:29:45.021 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:45.017+0000 7f2652d9d640 1 -- 172.21.15.37:0/907116465 learned_addr learned my addr 172.21.15.37:0/907116465 (peer_addr_for_me v2:172.21.15.37:0/0) 2024-04-25T10:29:45.021 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:45.017+0000 7f265259c640 1 --2- 172.21.15.37:0/907116465 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f265406ca80 0x7f26540cd1d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:29:45.021 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:45.017+0000 7f2651d9b640 1 --2- 172.21.15.37:0/907116465 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f265415a9f0 0x7f265415f410 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:29:45.021 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:45.017+0000 7f2652d9d640 1 -- 172.21.15.37:0/907116465 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f265415a9f0 msgr2=0x7f265415f410 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:29:45.021 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:45.017+0000 7f2652d9d640 1 --2- 172.21.15.37:0/907116465 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f265415a9f0 0x7f265415f410 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:29:45.021 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:45.017+0000 7f2652d9d640 1 -- 172.21.15.37:0/907116465 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f265406ca80 msgr2=0x7f26540cd1d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:29:45.021 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:45.017+0000 7f2652d9d640 1 --2- 172.21.15.37:0/907116465 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f265406ca80 0x7f26540cd1d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:29:45.021 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:45.017+0000 7f2652d9d640 1 -- 172.21.15.37:0/907116465 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f26540cdb20 con 0x7f2654160e50 2024-04-25T10:29:45.021 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:45.017+0000 7f2651d9b640 1 --2- 172.21.15.37:0/907116465 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f265415a9f0 0x7f265415f410 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T10:29:45.021 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:45.017+0000 7f2652d9d640 1 --2- 172.21.15.37:0/907116465 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f2654160e50 0x7f265415f950 secure :-1 s=READY pgs=259 cs=0 l=1 rev1=1 crypto rx=0x7f264400cc00 tx=0x7f264400a430 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:29:45.021 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:45.017+0000 7f264affd640 1 -- 172.21.15.37:0/907116465 <== mon.0 v2:172.21.15.37:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2644017070 con 0x7f2654160e50 2024-04-25T10:29:45.021 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:45.017+0000 7f264affd640 1 -- 172.21.15.37:0/907116465 <== mon.0 v2:172.21.15.37:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f264400ce10 con 0x7f2654160e50 2024-04-25T10:29:45.022 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:45.017+0000 7f2658e51640 1 -- 172.21.15.37:0/907116465 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f26540cde00 con 0x7f2654160e50 2024-04-25T10:29:45.022 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:45.017+0000 7f264affd640 1 -- 172.21.15.37:0/907116465 <== mon.0 v2:172.21.15.37:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2644012640 con 0x7f2654160e50 2024-04-25T10:29:45.022 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:45.017+0000 7f2658e51640 1 -- 172.21.15.37:0/907116465 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f26540ce3c0 con 0x7f2654160e50 2024-04-25T10:29:45.023 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:45.019+0000 7f2658e51640 1 -- 172.21.15.37:0/907116465 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f265406a140 con 0x7f2654160e50 2024-04-25T10:29:45.023 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:45.019+0000 7f264affd640 1 -- 172.21.15.37:0/907116465 <== mon.0 v2:172.21.15.37:3300/0 4 ==== mgrmap(e 5) ==== 96782+0+0 (secure 0 0 0) 0x7f264401a070 con 0x7f2654160e50 2024-04-25T10:29:45.023 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:45.020+0000 7f264affd640 1 --2- 172.21.15.37:0/907116465 >> [v2:172.21.15.203:6832/2418205237,v1:172.21.15.203:6833/2418205237] conn(0x7f26340744b0 0x7f26340769c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:29:45.023 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:45.020+0000 7f264affd640 1 -- 172.21.15.37:0/907116465 <== mon.0 v2:172.21.15.37:3300/0 5 ==== osd_map(23..23 src has 1..23) ==== 5709+0+0 (secure 0 0 0) 0x7f26440950b0 con 0x7f2654160e50 2024-04-25T10:29:45.023 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:45.020+0000 7f265259c640 1 --2- 172.21.15.37:0/907116465 >> [v2:172.21.15.203:6832/2418205237,v1:172.21.15.203:6833/2418205237] conn(0x7f26340744b0 0x7f26340769c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:29:45.023 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:45.020+0000 7f265259c640 1 --2- 172.21.15.37:0/907116465 >> [v2:172.21.15.203:6832/2418205237,v1:172.21.15.203:6833/2418205237] conn(0x7f26340744b0 0x7f26340769c0 secure :-1 s=READY pgs=142 cs=0 l=1 rev1=1 crypto rx=0x7f2654073140 tx=0x7f264c008040 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:29:45.026 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:45.024+0000 7f264affd640 1 -- 172.21.15.37:0/907116465 <== mon.0 v2:172.21.15.37:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187246 (secure 0 0 0) 0x7f264405f8d0 con 0x7f2654160e50 2024-04-25T10:29:45.211 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:45.209+0000 7f2658e51640 1 -- 172.21.15.37:0/907116465 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"} v 0) -- 0x7f2654071d10 con 0x7f2654160e50 2024-04-25T10:29:45.211 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:45.209+0000 7f264affd640 1 -- 172.21.15.37:0/907116465 <== mon.0 v2:172.21.15.37:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]=0 v23) ==== 100+0+11 (secure 0 0 0) 0x7f264405f1d0 con 0x7f2654160e50 2024-04-25T10:29:45.212 INFO:teuthology.orchestra.run.smithi037.stdout:pg_num: 64 2024-04-25T10:29:45.214 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:45.211+0000 7f2658e51640 1 -- 172.21.15.37:0/907116465 >> [v2:172.21.15.203:6832/2418205237,v1:172.21.15.203:6833/2418205237] conn(0x7f26340744b0 msgr2=0x7f26340769c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:29:45.214 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:45.211+0000 7f2658e51640 1 --2- 172.21.15.37:0/907116465 >> [v2:172.21.15.203:6832/2418205237,v1:172.21.15.203:6833/2418205237] conn(0x7f26340744b0 0x7f26340769c0 secure :-1 s=READY pgs=142 cs=0 l=1 rev1=1 crypto rx=0x7f2654073140 tx=0x7f264c008040 comp rx=0 tx=0).stop 2024-04-25T10:29:45.214 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:45.211+0000 7f2658e51640 1 -- 172.21.15.37:0/907116465 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f2654160e50 msgr2=0x7f265415f950 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:29:45.214 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:45.211+0000 7f2658e51640 1 --2- 172.21.15.37:0/907116465 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f2654160e50 0x7f265415f950 secure :-1 s=READY pgs=259 cs=0 l=1 rev1=1 crypto rx=0x7f264400cc00 tx=0x7f264400a430 comp rx=0 tx=0).stop 2024-04-25T10:29:45.214 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:45.211+0000 7f2658e51640 1 -- 172.21.15.37:0/907116465 shutdown_connections 2024-04-25T10:29:45.214 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:45.211+0000 7f2658e51640 1 --2- 172.21.15.37:0/907116465 >> [v2:172.21.15.203:6832/2418205237,v1:172.21.15.203:6833/2418205237] conn(0x7f26340744b0 0x7f26340769c0 unknown :-1 s=CLOSED pgs=142 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:29:45.214 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:45.211+0000 7f2658e51640 1 --2- 172.21.15.37:0/907116465 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f2654160e50 0x7f265415f950 unknown :-1 s=CLOSED pgs=259 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:29:45.214 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:45.211+0000 7f2658e51640 1 --2- 172.21.15.37:0/907116465 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f265415a9f0 0x7f265415f410 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:29:45.214 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:45.211+0000 7f2658e51640 1 --2- 172.21.15.37:0/907116465 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f265406ca80 0x7f26540cd1d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:29:45.214 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:45.211+0000 7f2658e51640 1 -- 172.21.15.37:0/907116465 >> 172.21.15.37:0/907116465 conn(0x7f26540b4560 msgr2=0x7f26540cf410 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:29:45.215 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:45.212+0000 7f2658e51640 1 -- 172.21.15.37:0/907116465 shutdown_connections 2024-04-25T10:29:45.215 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:45.212+0000 7f2658e51640 1 -- 172.21.15.37:0/907116465 wait complete. 2024-04-25T10:29:45.225 DEBUG:teuthology.orchestra.run.smithi037:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd blocklist ls 2024-04-25T10:29:45.338 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:45.336+0000 7f56251d5640 1 Processor -- start 2024-04-25T10:29:45.338 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:45.336+0000 7f56251d5640 1 -- start start 2024-04-25T10:29:45.340 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:45.337+0000 7f56251d5640 1 --2- >> v2:172.21.15.37:3300/0 conn(0x7f56200d5430 0x7f56200b75c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:29:45.340 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:45.337+0000 7f56251d5640 1 --2- >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f5620161110 0x7f5620161880 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:29:45.340 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:45.337+0000 7f56251d5640 1 -- --> v1:172.21.15.37:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f56200b7f10 con 0x7f56200b7b00 2024-04-25T10:29:45.340 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:45.337+0000 7f56251d5640 1 -- --> v2:172.21.15.37:3300/0 -- mon_getmap magic: 0 -- 0x7f56200d4420 con 0x7f56200d5430 2024-04-25T10:29:45.340 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:45.337+0000 7f56251d5640 1 -- --> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] -- mon_getmap magic: 0 -- 0x7f5620161520 con 0x7f5620161110 2024-04-25T10:29:45.340 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:45.337+0000 7f561ed9d640 1 --2- >> v2:172.21.15.37:3300/0 conn(0x7f56200d5430 0x7f56200b75c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:29:45.341 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:45.338+0000 7f5617fff640 1 --1- >> v1:172.21.15.37:6789/0 conn(0x7f56200b7b00 0x7f5620160a00 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.37:6789/0 says I am v1:172.21.15.37:49868/0 (socket says 172.21.15.37:49868) 2024-04-25T10:29:45.341 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:45.338+0000 7f561ed9d640 1 --2- >> v2:172.21.15.37:3300/0 conn(0x7f56200d5430 0x7f56200b75c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.37:3300/0 says I am v2:172.21.15.37:59546/0 (socket says 172.21.15.37:59546) 2024-04-25T10:29:45.341 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:45.338+0000 7f5617fff640 1 -- 172.21.15.37:0/2515484257 learned_addr learned my addr 172.21.15.37:0/2515484257 (peer_addr_for_me v1:172.21.15.37:0/0) 2024-04-25T10:29:45.341 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:45.338+0000 7f561f59e640 1 --2- 172.21.15.37:0/2515484257 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f5620161110 0x7f5620161880 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:29:45.341 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:45.338+0000 7f561ed9d640 1 -- 172.21.15.37:0/2515484257 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f5620161110 msgr2=0x7f5620161880 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:29:45.341 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:45.338+0000 7f561ed9d640 1 --2- 172.21.15.37:0/2515484257 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f5620161110 0x7f5620161880 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:29:45.341 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:45.338+0000 7f561ed9d640 1 -- 172.21.15.37:0/2515484257 >> v1:172.21.15.37:6789/0 conn(0x7f56200b7b00 legacy=0x7f5620160a00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:29:45.341 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:45.338+0000 7f561e59c640 1 -- 172.21.15.37:0/2515484257 <== mon.0 v1:172.21.15.37:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 3084726086 0 0) 0x7f5608001110 con 0x7f56200b7b00 2024-04-25T10:29:45.341 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:45.338+0000 7f561ed9d640 1 -- 172.21.15.37:0/2515484257 --> v2:172.21.15.37:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f56200d5840 con 0x7f56200d5430 2024-04-25T10:29:45.342 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:45.339+0000 7f561ed9d640 1 --2- 172.21.15.37:0/2515484257 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f56200d5430 0x7f56200b75c0 secure :-1 s=READY pgs=261 cs=0 l=1 rev1=1 crypto rx=0x7f56100097a0 tx=0x7f561002a100 comp rx=0 tx=0).ready entity=mon.0 client_cookie=25f1c5c2b0165bf1 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:29:45.342 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:45.339+0000 7f561e59c640 1 -- 172.21.15.37:0/2515484257 <== mon.0 v2:172.21.15.37:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5610037070 con 0x7f56200d5430 2024-04-25T10:29:45.343 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:45.339+0000 7f561e59c640 1 -- 172.21.15.37:0/2515484257 <== mon.0 v2:172.21.15.37:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f561002ae50 con 0x7f56200d5430 2024-04-25T10:29:45.343 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:45.339+0000 7f561e59c640 1 -- 172.21.15.37:0/2515484257 <== mon.0 v2:172.21.15.37:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5610032660 con 0x7f56200d5430 2024-04-25T10:29:45.343 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:45.339+0000 7f56251d5640 1 -- 172.21.15.37:0/2515484257 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f56200d5430 msgr2=0x7f56200b75c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:29:45.343 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:45.339+0000 7f56251d5640 1 --2- 172.21.15.37:0/2515484257 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f56200d5430 0x7f56200b75c0 secure :-1 s=READY pgs=261 cs=0 l=1 rev1=1 crypto rx=0x7f56100097a0 tx=0x7f561002a100 comp rx=0 tx=0).stop 2024-04-25T10:29:45.343 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:45.339+0000 7f56251d5640 1 -- 172.21.15.37:0/2515484257 shutdown_connections 2024-04-25T10:29:45.343 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:45.339+0000 7f56251d5640 1 --2- 172.21.15.37:0/2515484257 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f5620161110 0x7f5620161880 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:29:45.343 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:45.339+0000 7f56251d5640 1 --2- 172.21.15.37:0/2515484257 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f56200d5430 0x7f56200b75c0 unknown :-1 s=CLOSED pgs=261 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:29:45.343 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:45.339+0000 7f56251d5640 1 -- 172.21.15.37:0/2515484257 >> 172.21.15.37:0/2515484257 conn(0x7f56200d27a0 msgr2=0x7f56200d2bb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:29:45.343 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:45.339+0000 7f56251d5640 1 -- 172.21.15.37:0/2515484257 shutdown_connections 2024-04-25T10:29:45.343 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:45.339+0000 7f56251d5640 1 -- 172.21.15.37:0/2515484257 wait complete. 2024-04-25T10:29:45.344 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:45.341+0000 7f56251d5640 1 Processor -- start 2024-04-25T10:29:45.344 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:45.341+0000 7f56251d5640 1 -- start start 2024-04-25T10:29:45.346 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:45.343+0000 7f56251d5640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f56200b7b00 0x7f56200ce040 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:29:45.347 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:45.343+0000 7f56251d5640 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f56200d5430 0x7f562014fdf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:29:45.347 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:45.343+0000 7f56251d5640 1 --2- >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f5620161110 0x7f56200ba020 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:29:45.347 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:45.343+0000 7f56251d5640 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 -- 0x7f56200d4c30 con 0x7f56200b7b00 2024-04-25T10:29:45.347 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:45.343+0000 7f56251d5640 1 -- --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_getmap magic: 0 -- 0x7f56200d4950 con 0x7f56200d5430 2024-04-25T10:29:45.347 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:45.343+0000 7f56251d5640 1 -- --> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] -- mon_getmap magic: 0 -- 0x7f56200d4f10 con 0x7f5620161110 2024-04-25T10:29:45.347 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:45.343+0000 7f561ed9d640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f56200b7b00 0x7f56200ce040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:29:45.347 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:45.343+0000 7f561ed9d640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f56200b7b00 0x7f56200ce040 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.37:3300/0 says I am v2:172.21.15.37:59558/0 (socket says 172.21.15.37:59558) 2024-04-25T10:29:45.347 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:45.343+0000 7f561ed9d640 1 -- 172.21.15.37:0/4146716934 learned_addr learned my addr 172.21.15.37:0/4146716934 (peer_addr_for_me v2:172.21.15.37:0/0) 2024-04-25T10:29:45.347 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:45.343+0000 7f561f59e640 1 --2- 172.21.15.37:0/4146716934 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f5620161110 0x7f56200ba020 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:29:45.347 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:45.343+0000 7f5617fff640 1 --2- 172.21.15.37:0/4146716934 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f56200d5430 0x7f562014fdf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:29:45.347 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:45.343+0000 7f561ed9d640 1 -- 172.21.15.37:0/4146716934 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f5620161110 msgr2=0x7f56200ba020 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:29:45.347 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:45.343+0000 7f561ed9d640 1 --2- 172.21.15.37:0/4146716934 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f5620161110 0x7f56200ba020 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:29:45.347 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:45.343+0000 7f561ed9d640 1 -- 172.21.15.37:0/4146716934 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f56200d5430 msgr2=0x7f562014fdf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:29:45.347 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:45.343+0000 7f561ed9d640 1 --2- 172.21.15.37:0/4146716934 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f56200d5430 0x7f562014fdf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:29:45.348 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:45.343+0000 7f561ed9d640 1 -- 172.21.15.37:0/4146716934 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f56201f9860 con 0x7f56200b7b00 2024-04-25T10:29:45.348 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:45.343+0000 7f5617fff640 1 --2- 172.21.15.37:0/4146716934 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f56200d5430 0x7f562014fdf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T10:29:45.348 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:45.343+0000 7f561f59e640 1 --2- 172.21.15.37:0/4146716934 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f5620161110 0x7f56200ba020 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T10:29:45.348 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:45.343+0000 7f561ed9d640 1 --2- 172.21.15.37:0/4146716934 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f56200b7b00 0x7f56200ce040 secure :-1 s=READY pgs=262 cs=0 l=1 rev1=1 crypto rx=0x7f5610000c00 tx=0x7f56100046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:29:45.348 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:45.343+0000 7f56177fe640 1 -- 172.21.15.37:0/4146716934 <== mon.0 v2:172.21.15.37:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5610037040 con 0x7f56200b7b00 2024-04-25T10:29:45.348 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:45.343+0000 7f56177fe640 1 -- 172.21.15.37:0/4146716934 <== mon.0 v2:172.21.15.37:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f5610004030 con 0x7f56200b7b00 2024-04-25T10:29:45.348 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:45.343+0000 7f56251d5640 1 -- 172.21.15.37:0/4146716934 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f56201f9b40 con 0x7f56200b7b00 2024-04-25T10:29:45.348 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:45.343+0000 7f56177fe640 1 -- 172.21.15.37:0/4146716934 <== mon.0 v2:172.21.15.37:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5610004400 con 0x7f56200b7b00 2024-04-25T10:29:45.348 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:45.344+0000 7f56251d5640 1 -- 172.21.15.37:0/4146716934 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f56201fa100 con 0x7f56200b7b00 2024-04-25T10:29:45.348 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:45.344+0000 7f56251d5640 1 -- 172.21.15.37:0/4146716934 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f562006a140 con 0x7f56200b7b00 2024-04-25T10:29:45.349 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:45.345+0000 7f56177fe640 1 -- 172.21.15.37:0/4146716934 <== mon.0 v2:172.21.15.37:3300/0 4 ==== mgrmap(e 5) ==== 96782+0+0 (secure 0 0 0) 0x7f5610003610 con 0x7f56200b7b00 2024-04-25T10:29:45.349 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:45.345+0000 7f56177fe640 1 --2- 172.21.15.37:0/4146716934 >> [v2:172.21.15.203:6832/2418205237,v1:172.21.15.203:6833/2418205237] conn(0x7f56000744b0 0x7f56000769c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:29:45.349 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:45.346+0000 7f56177fe640 1 -- 172.21.15.37:0/4146716934 <== mon.0 v2:172.21.15.37:3300/0 5 ==== osd_map(23..23 src has 1..23) ==== 5709+0+0 (secure 0 0 0) 0x7f56100b5f10 con 0x7f56200b7b00 2024-04-25T10:29:45.349 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:45.346+0000 7f5617fff640 1 --2- 172.21.15.37:0/4146716934 >> [v2:172.21.15.203:6832/2418205237,v1:172.21.15.203:6833/2418205237] conn(0x7f56000744b0 0x7f56000769c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:29:45.349 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:45.346+0000 7f5617fff640 1 --2- 172.21.15.37:0/4146716934 >> [v2:172.21.15.203:6832/2418205237,v1:172.21.15.203:6833/2418205237] conn(0x7f56000744b0 0x7f56000769c0 secure :-1 s=READY pgs=143 cs=0 l=1 rev1=1 crypto rx=0x7f56201f96b0 tx=0x7f5608009040 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:29:45.352 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:45.350+0000 7f56177fe640 1 -- 172.21.15.37:0/4146716934 <== mon.0 v2:172.21.15.37:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187246 (secure 0 0 0) 0x7f56100807c0 con 0x7f56200b7b00 2024-04-25T10:29:45.548 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:45.546+0000 7f56251d5640 1 -- 172.21.15.37:0/4146716934 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_command({"prefix": "osd blocklist ls"} v 0) -- 0x7f562015def0 con 0x7f56200b7b00 2024-04-25T10:29:45.548 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:45.546+0000 7f56177fe640 1 -- 172.21.15.37:0/4146716934 <== mon.0 v2:172.21.15.37:3300/0 7 ==== mon_command_ack([{"prefix": "osd blocklist ls"}]=0 listed 0 entries v23) ==== 80+0+0 (secure 0 0 0) 0x7f56100800c0 con 0x7f56200b7b00 2024-04-25T10:29:45.548 INFO:teuthology.orchestra.run.smithi037.stderr:listed 0 entries 2024-04-25T10:29:45.551 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:45.548+0000 7f56251d5640 1 -- 172.21.15.37:0/4146716934 >> [v2:172.21.15.203:6832/2418205237,v1:172.21.15.203:6833/2418205237] conn(0x7f56000744b0 msgr2=0x7f56000769c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:29:45.551 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:45.548+0000 7f56251d5640 1 --2- 172.21.15.37:0/4146716934 >> [v2:172.21.15.203:6832/2418205237,v1:172.21.15.203:6833/2418205237] conn(0x7f56000744b0 0x7f56000769c0 secure :-1 s=READY pgs=143 cs=0 l=1 rev1=1 crypto rx=0x7f56201f96b0 tx=0x7f5608009040 comp rx=0 tx=0).stop 2024-04-25T10:29:45.551 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:45.548+0000 7f56251d5640 1 -- 172.21.15.37:0/4146716934 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f56200b7b00 msgr2=0x7f56200ce040 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:29:45.551 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:45.548+0000 7f56251d5640 1 --2- 172.21.15.37:0/4146716934 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f56200b7b00 0x7f56200ce040 secure :-1 s=READY pgs=262 cs=0 l=1 rev1=1 crypto rx=0x7f5610000c00 tx=0x7f56100046c0 comp rx=0 tx=0).stop 2024-04-25T10:29:45.551 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:45.548+0000 7f56251d5640 1 -- 172.21.15.37:0/4146716934 shutdown_connections 2024-04-25T10:29:45.551 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:45.548+0000 7f56251d5640 1 --2- 172.21.15.37:0/4146716934 >> [v2:172.21.15.203:6832/2418205237,v1:172.21.15.203:6833/2418205237] conn(0x7f56000744b0 0x7f56000769c0 unknown :-1 s=CLOSED pgs=143 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:29:45.551 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:45.548+0000 7f56251d5640 1 --2- 172.21.15.37:0/4146716934 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f5620161110 0x7f56200ba020 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:29:45.551 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:45.548+0000 7f56251d5640 1 --2- 172.21.15.37:0/4146716934 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f56200d5430 0x7f562014fdf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:29:45.551 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:45.548+0000 7f56251d5640 1 --2- 172.21.15.37:0/4146716934 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f56200b7b00 0x7f56200ce040 unknown :-1 s=CLOSED pgs=262 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:29:45.552 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:45.548+0000 7f56251d5640 1 -- 172.21.15.37:0/4146716934 >> 172.21.15.37:0/4146716934 conn(0x7f56200d27a0 msgr2=0x7f562015f4f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:29:45.552 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:45.548+0000 7f56251d5640 1 -- 172.21.15.37:0/4146716934 shutdown_connections 2024-04-25T10:29:45.552 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:45.548+0000 7f56251d5640 1 -- 172.21.15.37:0/4146716934 wait complete. 2024-04-25T10:29:45.562 DEBUG:teuthology.orchestra.run.smithi037:> set -ex 2024-04-25T10:29:45.562 DEBUG:teuthology.orchestra.run.smithi037:> dd if=/proc/self/mounts of=/dev/stdout 2024-04-25T10:29:45.578 DEBUG:teuthology.orchestra.run.smithi037:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-04-25T10:29:45.729 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:45.727+0000 7f413139f640 1 Processor -- start 2024-04-25T10:29:45.729 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:45.727+0000 7f413139f640 1 -- start start 2024-04-25T10:29:45.733 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:45.728+0000 7f413139f640 1 --2- >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f412c0d2870 0x7f412c164570 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:29:45.733 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:45.728+0000 7f413139f640 1 -- --> v1:172.21.15.37:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f412c160f90 con 0x7f412c0b4d30 2024-04-25T10:29:45.733 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:45.728+0000 7f413139f640 1 -- --> v1:172.21.15.203:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f412c161270 con 0x7f412c0d4260 2024-04-25T10:29:45.734 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:45.728+0000 7f413139f640 1 -- --> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] -- mon_getmap magic: 0 -- 0x7f412c0b5140 con 0x7f412c0d2870 2024-04-25T10:29:45.734 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:45.728+0000 7f412bfff640 1 --1- >> v1:172.21.15.37:6789/0 conn(0x7f412c0b4d30 0x7f412c0d2160 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.37:6789/0 says I am v1:172.21.15.37:49880/0 (socket says 172.21.15.37:49880) 2024-04-25T10:29:45.734 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:45.728+0000 7f412bfff640 1 -- 172.21.15.37:0/644130559 learned_addr learned my addr 172.21.15.37:0/644130559 (peer_addr_for_me v1:172.21.15.37:0/0) 2024-04-25T10:29:45.734 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:45.728+0000 7f412b7fe640 1 --2- 172.21.15.37:0/644130559 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f412c0d2870 0x7f412c164570 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:29:45.734 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:45.729+0000 7f412affd640 1 -- 172.21.15.37:0/644130559 <== mon.0 v1:172.21.15.37:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 2183860260 0 0) 0x7f412c160f90 con 0x7f412c0b4d30 2024-04-25T10:29:45.734 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:45.729+0000 7f412affd640 1 -- 172.21.15.37:0/644130559 --> v1:172.21.15.37:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f4114003520 con 0x7f412c0b4d30 2024-04-25T10:29:45.734 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:45.729+0000 7f412affd640 1 -- 172.21.15.37:0/644130559 <== mon.1 v1:172.21.15.203:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 1921833130 0 0) 0x7f412c161270 con 0x7f412c0d4260 2024-04-25T10:29:45.734 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:45.729+0000 7f412affd640 1 -- 172.21.15.37:0/644130559 --> v1:172.21.15.203:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f412c160f90 con 0x7f412c0d4260 2024-04-25T10:29:45.734 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:45.729+0000 7f412affd640 1 -- 172.21.15.37:0/644130559 <== mon.0 v1:172.21.15.37:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 3520069062 0 0) 0x7f4114003520 con 0x7f412c0b4d30 2024-04-25T10:29:45.734 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:45.729+0000 7f412affd640 1 -- 172.21.15.37:0/644130559 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f412c0d2870 msgr2=0x7f412c164570 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:29:45.734 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:45.729+0000 7f412affd640 1 --2- 172.21.15.37:0/644130559 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f412c0d2870 0x7f412c164570 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:29:45.734 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:45.729+0000 7f412affd640 1 -- 172.21.15.37:0/644130559 >> v1:172.21.15.203:6789/0 conn(0x7f412c0d4260 legacy=0x7f412c0d4670 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:29:45.734 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:45.729+0000 7f412affd640 1 -- 172.21.15.37:0/644130559 --> v1:172.21.15.37:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f412c163a90 con 0x7f412c0b4d30 2024-04-25T10:29:45.735 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:45.729+0000 7f412affd640 1 -- 172.21.15.37:0/644130559 <== mon.0 v1:172.21.15.37:6789/0 3 ==== mon_map magic: 0 ==== 413+0+0 (unknown 1577794600 0 0) 0x7f4120001670 con 0x7f412c0b4d30 2024-04-25T10:29:45.735 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:45.729+0000 7f412b7fe640 1 --2- 172.21.15.37:0/644130559 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f412c0d2870 0x7f412c164570 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-25T10:29:45.735 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:45.729+0000 7f412affd640 1 -- 172.21.15.37:0/644130559 >> v1:172.21.15.37:6789/0 conn(0x7f412c0b4d30 legacy=0x7f412c0d2160 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:29:45.735 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:45.729+0000 7f412affd640 1 --2- 172.21.15.37:0/644130559 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f412c0d2870 0x7f4114003b70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:29:45.735 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:45.729+0000 7f412affd640 1 --2- 172.21.15.37:0/644130559 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f4114004120 0x7f4114004800 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:29:45.735 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:45.729+0000 7f412affd640 1 --2- 172.21.15.37:0/644130559 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f4114004d40 0x7f4114005190 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:29:45.735 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:45.729+0000 7f412affd640 1 -- 172.21.15.37:0/644130559 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 -- 0x7f412c0b5140 con 0x7f4114004120 2024-04-25T10:29:45.735 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:45.729+0000 7f412affd640 1 -- 172.21.15.37:0/644130559 --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_getmap magic: 0 -- 0x7f411400bb80 con 0x7f4114004d40 2024-04-25T10:29:45.735 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:45.729+0000 7f412affd640 1 -- 172.21.15.37:0/644130559 --> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] -- mon_getmap magic: 0 -- 0x7f411400bd80 con 0x7f412c0d2870 2024-04-25T10:29:45.735 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:45.729+0000 7f412b7fe640 1 --2- 172.21.15.37:0/644130559 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f4114004120 0x7f4114004800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:29:45.735 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:45.729+0000 7f412bfff640 1 --2- 172.21.15.37:0/644130559 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f412c0d2870 0x7f4114003b70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:29:45.735 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:45.729+0000 7f4130b9e640 1 --2- 172.21.15.37:0/644130559 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f4114004d40 0x7f4114005190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:29:45.735 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:45.729+0000 7f412b7fe640 1 -- 172.21.15.37:0/644130559 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f412c0d2870 msgr2=0x7f4114003b70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:29:45.735 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:45.729+0000 7f412b7fe640 1 --2- 172.21.15.37:0/644130559 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f412c0d2870 0x7f4114003b70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:29:45.736 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:45.729+0000 7f412b7fe640 1 -- 172.21.15.37:0/644130559 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f4114004d40 msgr2=0x7f4114005190 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:29:45.736 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:45.729+0000 7f412b7fe640 1 --2- 172.21.15.37:0/644130559 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f4114004d40 0x7f4114005190 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:29:45.736 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:45.729+0000 7f412b7fe640 1 -- 172.21.15.37:0/644130559 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f4114003520 con 0x7f4114004120 2024-04-25T10:29:45.736 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:45.730+0000 7f412b7fe640 1 --2- 172.21.15.37:0/644130559 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f4114004120 0x7f4114004800 secure :-1 s=READY pgs=264 cs=0 l=1 rev1=1 crypto rx=0x7f411c009c70 tx=0x7f4114005830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=aaf9886f1cf78a3 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:29:45.736 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:45.730+0000 7f412affd640 1 -- 172.21.15.37:0/644130559 <== mon.0 v2:172.21.15.37:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f411c036070 con 0x7f4114004120 2024-04-25T10:29:45.736 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:45.730+0000 7f412affd640 1 -- 172.21.15.37:0/644130559 <== mon.0 v2:172.21.15.37:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f411c031400 con 0x7f4114004120 2024-04-25T10:29:45.736 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:45.730+0000 7f412affd640 1 -- 172.21.15.37:0/644130559 <== mon.0 v2:172.21.15.37:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f411c009840 con 0x7f4114004120 2024-04-25T10:29:45.736 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:45.731+0000 7f413139f640 1 -- 172.21.15.37:0/644130559 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f4114004120 msgr2=0x7f4114004800 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:29:45.736 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:45.731+0000 7f413139f640 1 --2- 172.21.15.37:0/644130559 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f4114004120 0x7f4114004800 secure :-1 s=READY pgs=264 cs=0 l=1 rev1=1 crypto rx=0x7f411c009c70 tx=0x7f4114005830 comp rx=0 tx=0).stop 2024-04-25T10:29:45.736 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:45.731+0000 7f4130b9e640 1 -- 172.21.15.37:0/644130559 reap_dead start 2024-04-25T10:29:45.736 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:45.731+0000 7f413139f640 1 -- 172.21.15.37:0/644130559 shutdown_connections 2024-04-25T10:29:45.736 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:45.731+0000 7f413139f640 1 -- 172.21.15.37:0/644130559 >> 172.21.15.37:0/644130559 conn(0x7f412c074840 msgr2=0x7f412c06dd20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:29:45.736 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:45.731+0000 7f413139f640 1 -- 172.21.15.37:0/644130559 shutdown_connections 2024-04-25T10:29:45.736 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:45.731+0000 7f413139f640 1 -- 172.21.15.37:0/644130559 wait complete. 2024-04-25T10:29:45.739 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:45.733+0000 7f413139f640 1 Processor -- start 2024-04-25T10:29:45.740 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:45.733+0000 7f413139f640 1 -- start start 2024-04-25T10:29:45.740 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:45.734+0000 7f413139f640 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f412c0cf550 0x7f412c159a20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:29:45.740 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:45.734+0000 7f413139f640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f412c159f60 0x7f412c0b6030 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:29:45.740 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:45.734+0000 7f413139f640 1 --2- >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f412c0b6570 0x7f412c1e98b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:29:45.740 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:45.734+0000 7f413139f640 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 -- 0x7f412c0b7630 con 0x7f412c159f60 2024-04-25T10:29:45.740 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:45.734+0000 7f413139f640 1 -- --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_getmap magic: 0 -- 0x7f412c164050 con 0x7f412c0cf550 2024-04-25T10:29:45.740 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:45.734+0000 7f413139f640 1 -- --> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] -- mon_getmap magic: 0 -- 0x7f412c0b77b0 con 0x7f412c0b6570 2024-04-25T10:29:45.740 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:45.734+0000 7f412b7fe640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f412c159f60 0x7f412c0b6030 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:29:45.740 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:45.734+0000 7f412b7fe640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f412c159f60 0x7f412c0b6030 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.37:3300/0 says I am v2:172.21.15.37:59584/0 (socket says 172.21.15.37:59584) 2024-04-25T10:29:45.740 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:45.735+0000 7f412b7fe640 1 -- 172.21.15.37:0/3797640062 learned_addr learned my addr 172.21.15.37:0/3797640062 (peer_addr_for_me v2:172.21.15.37:0/0) 2024-04-25T10:29:45.740 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:45.735+0000 7f412bfff640 1 --2- 172.21.15.37:0/3797640062 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f412c0cf550 0x7f412c159a20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:29:45.740 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:45.735+0000 7f4130b9e640 1 --2- 172.21.15.37:0/3797640062 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f412c0b6570 0x7f412c1e98b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:29:45.741 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:45.735+0000 7f412b7fe640 1 -- 172.21.15.37:0/3797640062 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f412c0b6570 msgr2=0x7f412c1e98b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:29:45.741 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:45.735+0000 7f412b7fe640 1 --2- 172.21.15.37:0/3797640062 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f412c0b6570 0x7f412c1e98b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:29:45.741 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:45.735+0000 7f412b7fe640 1 -- 172.21.15.37:0/3797640062 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f412c0cf550 msgr2=0x7f412c159a20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:29:45.741 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:45.735+0000 7f412b7fe640 1 --2- 172.21.15.37:0/3797640062 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f412c0cf550 0x7f412c159a20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:29:45.741 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:45.735+0000 7f412b7fe640 1 -- 172.21.15.37:0/3797640062 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f412c0b6980 con 0x7f412c159f60 2024-04-25T10:29:45.741 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:45.735+0000 7f412bfff640 1 --2- 172.21.15.37:0/3797640062 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f412c0cf550 0x7f412c159a20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T10:29:45.741 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:45.735+0000 7f412b7fe640 1 --2- 172.21.15.37:0/3797640062 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f412c159f60 0x7f412c0b6030 secure :-1 s=READY pgs=265 cs=0 l=1 rev1=1 crypto rx=0x7f411c034040 tx=0x7f411c004730 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:29:45.741 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:45.735+0000 7f4128ff9640 1 -- 172.21.15.37:0/3797640062 <== mon.0 v2:172.21.15.37:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f411c040020 con 0x7f412c159f60 2024-04-25T10:29:45.741 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:45.735+0000 7f4128ff9640 1 -- 172.21.15.37:0/3797640062 <== mon.0 v2:172.21.15.37:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f411c0093e0 con 0x7f412c159f60 2024-04-25T10:29:45.741 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:45.735+0000 7f4128ff9640 1 -- 172.21.15.37:0/3797640062 <== mon.0 v2:172.21.15.37:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f411c036040 con 0x7f412c159f60 2024-04-25T10:29:45.741 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:45.735+0000 7f413139f640 1 -- 172.21.15.37:0/3797640062 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f412c15c9e0 con 0x7f412c159f60 2024-04-25T10:29:45.741 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:45.735+0000 7f413139f640 1 -- 172.21.15.37:0/3797640062 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f412c14aaf0 con 0x7f412c159f60 2024-04-25T10:29:45.741 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:45.736+0000 7f413139f640 1 -- 172.21.15.37:0/3797640062 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f412c076bc0 con 0x7f412c159f60 2024-04-25T10:29:45.742 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:45.737+0000 7f4128ff9640 1 -- 172.21.15.37:0/3797640062 <== mon.0 v2:172.21.15.37:3300/0 4 ==== mgrmap(e 5) ==== 96782+0+0 (secure 0 0 0) 0x7f411c043020 con 0x7f412c159f60 2024-04-25T10:29:45.742 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:45.737+0000 7f4128ff9640 1 --2- 172.21.15.37:0/3797640062 >> [v2:172.21.15.203:6832/2418205237,v1:172.21.15.203:6833/2418205237] conn(0x7f41180744e0 0x7f41180769f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:29:45.742 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:45.737+0000 7f4128ff9640 1 -- 172.21.15.37:0/3797640062 <== mon.0 v2:172.21.15.37:3300/0 5 ==== osd_map(23..23 src has 1..23) ==== 5709+0+0 (secure 0 0 0) 0x7f411c0b57e0 con 0x7f412c159f60 2024-04-25T10:29:45.742 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:45.738+0000 7f412bfff640 1 --2- 172.21.15.37:0/3797640062 >> [v2:172.21.15.203:6832/2418205237,v1:172.21.15.203:6833/2418205237] conn(0x7f41180744e0 0x7f41180769f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:29:45.742 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:45.738+0000 7f412bfff640 1 --2- 172.21.15.37:0/3797640062 >> [v2:172.21.15.203:6832/2418205237,v1:172.21.15.203:6833/2418205237] conn(0x7f41180744e0 0x7f41180769f0 secure :-1 s=READY pgs=144 cs=0 l=1 rev1=1 crypto rx=0x7f4120001200 tx=0x7f4120001150 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:29:45.744 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:45.742+0000 7f4128ff9640 1 -- 172.21.15.37:0/3797640062 <== mon.0 v2:172.21.15.37:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187246 (secure 0 0 0) 0x7f411c04a030 con 0x7f412c159f60 2024-04-25T10:29:45.932 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:45.930+0000 7f413139f640 1 -- 172.21.15.37:0/3797640062 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7f412c0d0490 con 0x7f412c159f60 2024-04-25T10:29:45.933 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:45.931+0000 7f4128ff9640 1 -- 172.21.15.37:0/3797640062 <== mon.0 v2:172.21.15.37:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v23) ==== 74+0+16621 (secure 0 0 0) 0x7f412c0b6980 con 0x7f412c159f60 2024-04-25T10:29:45.933 INFO:teuthology.orchestra.run.smithi037.stdout: 2024-04-25T10:29:45.933 INFO:teuthology.orchestra.run.smithi037.stdout:{"epoch":23,"fsid":"f4acda28-be34-492a-b286-994afd36f61d","created":"2024-04-25T10:28:43.985718+0000","modified":"2024-04-25T10:29:10.016188+0000","last_up_change":"2024-04-25T10:28:54.900654+0000","last_in_change":"2024-04-25T10:28:46.216973+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":3,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-04-25T10:28:56.202153+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"19","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2024-04-25T10:29:06.507244+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"22","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":2.25,"score_stable":2.25,"optimal_score":1,"raw_score_acting":2.25,"raw_score_stable":2.25,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2024-04-25T10:29:07.347844+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"23","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"a2f79028-b332-4166-9ebb-a7024a37ea22","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":21,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.37:6816","nonce":2746074720},{"type":"v1","addr":"172.21.15.37:6817","nonce":2746074720}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.37:6818","nonce":2746074720},{"type":"v1","addr":"172.21.15.37:6819","nonce":2746074720}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.37:6822","nonce":2746074720},{"type":"v1","addr":"172.21.15.37:6823","nonce":2746074720}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.37:6820","nonce":2746074720},{"type":"v1","addr":"172.21.15.37:6821","nonce":2746074720}]},"public_addr":"172.21.15.37:6817/2746074720","cluster_addr":"172.21.15.37:6819/2746074720","heartbeat_back_addr":"172.21.15.37:6823/2746074720","heartbeat_front_addr":"172.21.15.37:6821/2746074720","state":["exists","up"]},{"osd":1,"uuid":"973159de-c2ea-4083-aecc-69373d1c1132","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":21,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.37:6800","nonce":1690341662},{"type":"v1","addr":"172.21.15.37:6801","nonce":1690341662}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.37:6802","nonce":1690341662},{"type":"v1","addr":"172.21.15.37:6803","nonce":1690341662}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.37:6806","nonce":1690341662},{"type":"v1","addr":"172.21.15.37:6807","nonce":1690341662}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.37:6804","nonce":1690341662},{"type":"v1","addr":"172.21.15.37:6805","nonce":1690341662}]},"public_addr":"172.21.15.37:6801/1690341662","cluster_addr":"172.21.15.37:6803/1690341662","heartbeat_back_addr":"172.21.15.37:6807/1690341662","heartbeat_front_addr":"172.21.15.37:6805/1690341662","state":["exists","up"]},{"osd":2,"uuid":"4ae7a0e9-cb3f-484c-a75f-f3766b721c47","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":21,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.37:6808","nonce":2754669649},{"type":"v1","addr":"172.21.15.37:6809","nonce":2754669649}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.37:6810","nonce":2754669649},{"type":"v1","addr":"172.21.15.37:6811","nonce":2754669649}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.37:6814","nonce":2754669649},{"type":"v1","addr":"172.21.15.37:6815","nonce":2754669649}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.37:6812","nonce":2754669649},{"type":"v1","addr":"172.21.15.37:6813","nonce":2754669649}]},"public_addr":"172.21.15.37:6809/2754669649","cluster_addr":"172.21.15.37:6811/2754669649","heartbeat_back_addr":"172.21.15.37:6815/2754669649","heartbeat_front_addr":"172.21.15.37:6813/2754669649","state":["exists","up"]},{"osd":3,"uuid":"1ba4a746-1608-47a7-ace2-960a5cfae934","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":21,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.37:6824","nonce":1815290618},{"type":"v1","addr":"172.21.15.37:6825","nonce":1815290618}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.37:6826","nonce":1815290618},{"type":"v1","addr":"172.21.15.37:6827","nonce":1815290618}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.37:6830","nonce":1815290618},{"type":"v1","addr":"172.21.15.37:6831","nonce":1815290618}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.37:6828","nonce":1815290618},{"type":"v1","addr":"172.21.15.37:6829","nonce":1815290618}]},"public_addr":"172.21.15.37:6825/1815290618","cluster_addr":"172.21.15.37:6827/1815290618","heartbeat_back_addr":"172.21.15.37:6831/1815290618","heartbeat_front_addr":"172.21.15.37:6829/1815290618","state":["exists","up"]},{"osd":4,"uuid":"d4f59f5d-0413-4b79-87ee-bb212c1fcf62","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":21,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6800","nonce":4051532063},{"type":"v1","addr":"172.21.15.203:6801","nonce":4051532063}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6802","nonce":4051532063},{"type":"v1","addr":"172.21.15.203:6803","nonce":4051532063}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6806","nonce":4051532063},{"type":"v1","addr":"172.21.15.203:6807","nonce":4051532063}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6804","nonce":4051532063},{"type":"v1","addr":"172.21.15.203:6805","nonce":4051532063}]},"public_addr":"172.21.15.203:6801/4051532063","cluster_addr":"172.21.15.203:6803/4051532063","heartbeat_back_addr":"172.21.15.203:6807/4051532063","heartbeat_front_addr":"172.21.15.203:6805/4051532063","state":["exists","up"]},{"osd":5,"uuid":"69710a0e-41d8-4e7f-9a10-17ce540b67be","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":21,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6824","nonce":1203822890},{"type":"v1","addr":"172.21.15.203:6825","nonce":1203822890}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6826","nonce":1203822890},{"type":"v1","addr":"172.21.15.203:6827","nonce":1203822890}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6830","nonce":1203822890},{"type":"v1","addr":"172.21.15.203:6831","nonce":1203822890}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6828","nonce":1203822890},{"type":"v1","addr":"172.21.15.203:6829","nonce":1203822890}]},"public_addr":"172.21.15.203:6825/1203822890","cluster_addr":"172.21.15.203:6827/1203822890","heartbeat_back_addr":"172.21.15.203:6831/1203822890","heartbeat_front_addr":"172.21.15.203:6829/1203822890","state":["exists","up"]},{"osd":6,"uuid":"f29dd047-3b48-473d-a235-6642a6293f20","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":21,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6810","nonce":2309615014},{"type":"v1","addr":"172.21.15.203:6812","nonce":2309615014}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6814","nonce":2309615014},{"type":"v1","addr":"172.21.15.203:6816","nonce":2309615014}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6822","nonce":2309615014},{"type":"v1","addr":"172.21.15.203:6823","nonce":2309615014}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6818","nonce":2309615014},{"type":"v1","addr":"172.21.15.203:6820","nonce":2309615014}]},"public_addr":"172.21.15.203:6812/2309615014","cluster_addr":"172.21.15.203:6816/2309615014","heartbeat_back_addr":"172.21.15.203:6823/2309615014","heartbeat_front_addr":"172.21.15.203:6820/2309615014","state":["exists","up"]},{"osd":7,"uuid":"6e691644-378b-4fdc-8e86-fb5f56e31f29","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":21,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6808","nonce":3688099014},{"type":"v1","addr":"172.21.15.203:6809","nonce":3688099014}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6811","nonce":3688099014},{"type":"v1","addr":"172.21.15.203:6813","nonce":3688099014}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6819","nonce":3688099014},{"type":"v1","addr":"172.21.15.203:6821","nonce":3688099014}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6815","nonce":3688099014},{"type":"v1","addr":"172.21.15.203:6817","nonce":3688099014}]},"public_addr":"172.21.15.203:6809/3688099014","cluster_addr":"172.21.15.203:6813/3688099014","heartbeat_back_addr":"172.21.15.203:6821/3688099014","heartbeat_front_addr":"172.21.15.203:6817/3688099014","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T10:28:51.419141+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T10:28:51.366448+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T10:28:51.348276+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T10:28:51.380519+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T10:28:51.105812+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T10:28:51.384496+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T10:28:51.388755+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T10:28:51.366958+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-04-25T10:29:45.935 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:45.933+0000 7f413139f640 1 -- 172.21.15.37:0/3797640062 >> [v2:172.21.15.203:6832/2418205237,v1:172.21.15.203:6833/2418205237] conn(0x7f41180744e0 msgr2=0x7f41180769f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:29:45.935 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:45.933+0000 7f413139f640 1 --2- 172.21.15.37:0/3797640062 >> [v2:172.21.15.203:6832/2418205237,v1:172.21.15.203:6833/2418205237] conn(0x7f41180744e0 0x7f41180769f0 secure :-1 s=READY pgs=144 cs=0 l=1 rev1=1 crypto rx=0x7f4120001200 tx=0x7f4120001150 comp rx=0 tx=0).stop 2024-04-25T10:29:45.935 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:45.933+0000 7f413139f640 1 -- 172.21.15.37:0/3797640062 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f412c159f60 msgr2=0x7f412c0b6030 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:29:45.936 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:45.933+0000 7f413139f640 1 --2- 172.21.15.37:0/3797640062 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f412c159f60 0x7f412c0b6030 secure :-1 s=READY pgs=265 cs=0 l=1 rev1=1 crypto rx=0x7f411c034040 tx=0x7f411c004730 comp rx=0 tx=0).stop 2024-04-25T10:29:45.936 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:45.933+0000 7f413139f640 1 -- 172.21.15.37:0/3797640062 shutdown_connections 2024-04-25T10:29:45.936 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:45.933+0000 7f413139f640 1 --2- 172.21.15.37:0/3797640062 >> [v2:172.21.15.203:6832/2418205237,v1:172.21.15.203:6833/2418205237] conn(0x7f41180744e0 0x7f41180769f0 unknown :-1 s=CLOSED pgs=144 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:29:45.936 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:45.933+0000 7f413139f640 1 --2- 172.21.15.37:0/3797640062 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f412c0b6570 0x7f412c1e98b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:29:45.936 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:45.933+0000 7f413139f640 1 --2- 172.21.15.37:0/3797640062 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f412c159f60 0x7f412c0b6030 unknown :-1 s=CLOSED pgs=265 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:29:45.936 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:45.933+0000 7f413139f640 1 --2- 172.21.15.37:0/3797640062 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f412c0cf550 0x7f412c159a20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:29:45.936 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:45.933+0000 7f413139f640 1 -- 172.21.15.37:0/3797640062 >> 172.21.15.37:0/3797640062 conn(0x7f412c074840 msgr2=0x7f412c0b8ed0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:29:45.936 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:45.933+0000 7f413139f640 1 -- 172.21.15.37:0/3797640062 shutdown_connections 2024-04-25T10:29:45.936 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:45.933+0000 7f413139f640 1 -- 172.21.15.37:0/3797640062 wait complete. 2024-04-25T10:29:45.947 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2024-04-25T10:28:56.202153+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '19', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 8, 'score_stable': 8, 'optimal_score': 0.25, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 2, 'pool_name': 'cephfs_metadata', 'create_time': '2024-04-25T10:29:06.507244+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '22', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_autoscale_bias': 4, 'pg_num_min': 16, 'recovery_priority': 5}, 'application_metadata': {'cephfs': {'metadata': 'cephfs'}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 2.25, 'score_stable': 2.25, 'optimal_score': 1, 'raw_score_acting': 2.25, 'raw_score_stable': 2.25, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 3, 'pool_name': 'cephfs_data', 'create_time': '2024-04-25T10:29:07.347844+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 64, 'pg_placement_num': 64, 'pg_placement_num_target': 64, 'pg_num_target': 64, 'pg_num_pending': 64, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '23', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.8999999761581421}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 1.5, 'score_stable': 1.5, 'optimal_score': 1, 'raw_score_acting': 1.5, 'raw_score_stable': 1.5, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}] 2024-04-25T10:29:45.947 DEBUG:teuthology.orchestra.run.smithi037:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2024-04-25T10:29:46.062 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:46.059+0000 7f9cc05b2640 1 Processor -- start 2024-04-25T10:29:46.062 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:46.059+0000 7f9cc05b2640 1 -- start start 2024-04-25T10:29:46.064 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:46.061+0000 7f9cc05b2640 1 --2- >> v2:172.21.15.203:3300/0 conn(0x7f9cb81441d0 0x7f9cb80d0550 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:29:46.064 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:46.061+0000 7f9cc05b2640 1 -- --> v1:172.21.15.37:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f9cb80d0ea0 con 0x7f9cb80d0a90 2024-04-25T10:29:46.064 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:46.061+0000 7f9cc05b2640 1 -- --> v1:172.21.15.203:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f9cb80cf600 con 0x7f9cb80cefe0 2024-04-25T10:29:46.064 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:46.061+0000 7f9cc05b2640 1 -- --> v2:172.21.15.203:3300/0 -- mon_getmap magic: 0 -- 0x7f9cb80cf8e0 con 0x7f9cb81441d0 2024-04-25T10:29:46.064 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:46.061+0000 7f9cbdb4d640 1 --1- >> v1:172.21.15.37:6789/0 conn(0x7f9cb80d0a90 0x7f9cb80ce8d0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.37:6789/0 says I am v1:172.21.15.37:49884/0 (socket says 172.21.15.37:49884) 2024-04-25T10:29:46.064 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:46.061+0000 7f9cbdb4d640 1 -- 172.21.15.37:0/1857803320 learned_addr learned my addr 172.21.15.37:0/1857803320 (peer_addr_for_me v1:172.21.15.37:0/0) 2024-04-25T10:29:46.064 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:46.061+0000 7f9cbe34e640 1 --2- 172.21.15.37:0/1857803320 >> v2:172.21.15.203:3300/0 conn(0x7f9cb81441d0 0x7f9cb80d0550 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:29:46.064 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:46.061+0000 7f9cbd34c640 1 -- 172.21.15.37:0/1857803320 <== mon.0 v1:172.21.15.37:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 747665315 0 0) 0x7f9cac001110 con 0x7f9cb80d0a90 2024-04-25T10:29:46.064 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:46.061+0000 7f9cbd34c640 1 -- 172.21.15.37:0/1857803320 --> v1:172.21.15.37:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f9ca0003520 con 0x7f9cb80d0a90 2024-04-25T10:29:46.065 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:46.062+0000 7f9cbd34c640 1 -- 172.21.15.37:0/1857803320 <== mon.1 v1:172.21.15.203:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 3806536320 0 0) 0x7f9cb80cf600 con 0x7f9cb80cefe0 2024-04-25T10:29:46.065 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:46.062+0000 7f9cbd34c640 1 -- 172.21.15.37:0/1857803320 --> v1:172.21.15.203:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f9cac001110 con 0x7f9cb80cefe0 2024-04-25T10:29:46.065 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:46.062+0000 7f9cbd34c640 1 -- 172.21.15.37:0/1857803320 <== mon.0 v1:172.21.15.37:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 1087595143 0 0) 0x7f9ca0003520 con 0x7f9cb80d0a90 2024-04-25T10:29:46.065 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:46.062+0000 7f9cbe34e640 1 -- 172.21.15.37:0/1857803320 >> v1:172.21.15.203:6789/0 conn(0x7f9cb80cefe0 legacy=0x7f9cb806f3a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:29:46.065 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:46.062+0000 7f9cbe34e640 1 -- 172.21.15.37:0/1857803320 >> v1:172.21.15.37:6789/0 conn(0x7f9cb80d0a90 legacy=0x7f9cb80ce8d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:29:46.065 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:46.062+0000 7f9cbe34e640 1 -- 172.21.15.37:0/1857803320 --> v2:172.21.15.203:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f9cb80cfbc0 con 0x7f9cb81441d0 2024-04-25T10:29:46.066 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:46.062+0000 7f9cbe34e640 1 --2- 172.21.15.37:0/1857803320 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f9cb81441d0 0x7f9cb80d0550 secure :-1 s=READY pgs=126 cs=0 l=1 rev1=1 crypto rx=0x7f9ca80097a0 tx=0x7f9ca802a1c0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=7be3bd23599f9e16 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:29:46.066 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:46.063+0000 7f9cbd34c640 1 -- 172.21.15.37:0/1857803320 <== mon.1 v2:172.21.15.203:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9ca8037070 con 0x7f9cb81441d0 2024-04-25T10:29:46.066 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:46.063+0000 7f9cbd34c640 1 -- 172.21.15.37:0/1857803320 <== mon.1 v2:172.21.15.203:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f9ca802ac50 con 0x7f9cb81441d0 2024-04-25T10:29:46.067 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:46.063+0000 7f9cbd34c640 1 -- 172.21.15.37:0/1857803320 <== mon.1 v2:172.21.15.203:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9cac001840 con 0x7f9cb81441d0 2024-04-25T10:29:46.067 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:46.063+0000 7f9cc05b2640 1 -- 172.21.15.37:0/1857803320 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f9cb81441d0 msgr2=0x7f9cb80d0550 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:29:46.067 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:46.063+0000 7f9cc05b2640 1 --2- 172.21.15.37:0/1857803320 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f9cb81441d0 0x7f9cb80d0550 secure :-1 s=READY pgs=126 cs=0 l=1 rev1=1 crypto rx=0x7f9ca80097a0 tx=0x7f9ca802a1c0 comp rx=0 tx=0).stop 2024-04-25T10:29:46.067 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:46.063+0000 7f9cc05b2640 1 -- 172.21.15.37:0/1857803320 shutdown_connections 2024-04-25T10:29:46.067 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:46.063+0000 7f9cc05b2640 1 --2- 172.21.15.37:0/1857803320 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f9cb81441d0 0x7f9cb80d0550 unknown :-1 s=CLOSED pgs=126 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:29:46.067 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:46.063+0000 7f9cc05b2640 1 -- 172.21.15.37:0/1857803320 >> 172.21.15.37:0/1857803320 conn(0x7f9cb80b36b0 msgr2=0x7f9cb80b3ac0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:29:46.067 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:46.063+0000 7f9cc05b2640 1 -- 172.21.15.37:0/1857803320 shutdown_connections 2024-04-25T10:29:46.067 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:46.063+0000 7f9cc05b2640 1 -- 172.21.15.37:0/1857803320 wait complete. 2024-04-25T10:29:46.068 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:46.065+0000 7f9cc05b2640 1 Processor -- start 2024-04-25T10:29:46.068 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:46.065+0000 7f9cc05b2640 1 -- start start 2024-04-25T10:29:46.070 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:46.067+0000 7f9cc05b2640 1 --2- >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f9cb80cefe0 0x7f9cb815b330 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:29:46.070 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:46.067+0000 7f9cc05b2640 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f9cb80d0a90 0x7f9cb81e6a80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:29:46.070 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:46.067+0000 7f9cc05b2640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f9cb81441d0 0x7f9cb8160ae0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:29:46.070 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:46.067+0000 7f9cc05b2640 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 -- 0x7f9cb81445e0 con 0x7f9cb81441d0 2024-04-25T10:29:46.070 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:46.067+0000 7f9cc05b2640 1 -- --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_getmap magic: 0 -- 0x7f9cb80d0180 con 0x7f9cb80d0a90 2024-04-25T10:29:46.070 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:46.067+0000 7f9cc05b2640 1 -- --> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] -- mon_getmap magic: 0 -- 0x7f9cb8144760 con 0x7f9cb80cefe0 2024-04-25T10:29:46.070 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:46.067+0000 7f9cbeb4f640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f9cb81441d0 0x7f9cb8160ae0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:29:46.070 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:46.067+0000 7f9cbeb4f640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f9cb81441d0 0x7f9cb8160ae0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.37:3300/0 says I am v2:172.21.15.37:59586/0 (socket says 172.21.15.37:59586) 2024-04-25T10:29:46.070 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:46.067+0000 7f9cbeb4f640 1 -- 172.21.15.37:0/4116267569 learned_addr learned my addr 172.21.15.37:0/4116267569 (peer_addr_for_me v2:172.21.15.37:0/0) 2024-04-25T10:29:46.071 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:46.067+0000 7f9cbdb4d640 1 --2- 172.21.15.37:0/4116267569 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f9cb80d0a90 0x7f9cb81e6a80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:29:46.071 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:46.067+0000 7f9cbe34e640 1 --2- 172.21.15.37:0/4116267569 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f9cb80cefe0 0x7f9cb815b330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:29:46.071 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:46.067+0000 7f9cbeb4f640 1 -- 172.21.15.37:0/4116267569 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f9cb80cefe0 msgr2=0x7f9cb815b330 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:29:46.071 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:46.067+0000 7f9cbeb4f640 1 --2- 172.21.15.37:0/4116267569 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f9cb80cefe0 0x7f9cb815b330 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:29:46.071 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:46.067+0000 7f9cbeb4f640 1 -- 172.21.15.37:0/4116267569 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f9cb80d0a90 msgr2=0x7f9cb81e6a80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:29:46.071 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:46.067+0000 7f9cbeb4f640 1 --2- 172.21.15.37:0/4116267569 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f9cb80d0a90 0x7f9cb81e6a80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:29:46.071 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:46.067+0000 7f9cbeb4f640 1 -- 172.21.15.37:0/4116267569 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f9cb81e7770 con 0x7f9cb81441d0 2024-04-25T10:29:46.071 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:46.067+0000 7f9cbe34e640 1 --2- 172.21.15.37:0/4116267569 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f9cb80cefe0 0x7f9cb815b330 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T10:29:46.071 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:46.067+0000 7f9cbeb4f640 1 --2- 172.21.15.37:0/4116267569 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f9cb81441d0 0x7f9cb8160ae0 secure :-1 s=READY pgs=267 cs=0 l=1 rev1=1 crypto rx=0x7f9cb000c6e0 tx=0x7f9cb000cba0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:29:46.071 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:46.067+0000 7f9cb6ffd640 1 -- 172.21.15.37:0/4116267569 <== mon.0 v2:172.21.15.37:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9cb0003b70 con 0x7f9cb81441d0 2024-04-25T10:29:46.071 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:46.068+0000 7f9cb6ffd640 1 -- 172.21.15.37:0/4116267569 <== mon.0 v2:172.21.15.37:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f9cb0003d10 con 0x7f9cb81441d0 2024-04-25T10:29:46.071 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:46.068+0000 7f9cc05b2640 1 -- 172.21.15.37:0/4116267569 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f9cb81e7a50 con 0x7f9cb81441d0 2024-04-25T10:29:46.071 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:46.068+0000 7f9cb6ffd640 1 -- 172.21.15.37:0/4116267569 <== mon.0 v2:172.21.15.37:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9cb00052d0 con 0x7f9cb81441d0 2024-04-25T10:29:46.072 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:46.068+0000 7f9cc05b2640 1 -- 172.21.15.37:0/4116267569 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f9cb80b52d0 con 0x7f9cb81441d0 2024-04-25T10:29:46.072 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:46.069+0000 7f9cc05b2640 1 -- 172.21.15.37:0/4116267569 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f9cb806a140 con 0x7f9cb81441d0 2024-04-25T10:29:46.073 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:46.069+0000 7f9cb6ffd640 1 -- 172.21.15.37:0/4116267569 <== mon.0 v2:172.21.15.37:3300/0 4 ==== mgrmap(e 5) ==== 96782+0+0 (secure 0 0 0) 0x7f9cb0005470 con 0x7f9cb81441d0 2024-04-25T10:29:46.073 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:46.070+0000 7f9cb6ffd640 1 --2- 172.21.15.37:0/4116267569 >> [v2:172.21.15.203:6832/2418205237,v1:172.21.15.203:6833/2418205237] conn(0x7f9ca40744b0 0x7f9ca40769c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:29:46.073 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:46.070+0000 7f9cb6ffd640 1 -- 172.21.15.37:0/4116267569 <== mon.0 v2:172.21.15.37:3300/0 5 ==== osd_map(23..23 src has 1..23) ==== 5709+0+0 (secure 0 0 0) 0x7f9cb0094fd0 con 0x7f9cb81441d0 2024-04-25T10:29:46.073 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:46.070+0000 7f9cbe34e640 1 --2- 172.21.15.37:0/4116267569 >> [v2:172.21.15.203:6832/2418205237,v1:172.21.15.203:6833/2418205237] conn(0x7f9ca40744b0 0x7f9ca40769c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:29:46.073 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:46.071+0000 7f9cbe34e640 1 --2- 172.21.15.37:0/4116267569 >> [v2:172.21.15.203:6832/2418205237,v1:172.21.15.203:6833/2418205237] conn(0x7f9ca40744b0 0x7f9ca40769c0 secure :-1 s=READY pgs=145 cs=0 l=1 rev1=1 crypto rx=0x7f9ca8034000 tx=0x7f9ca802e000 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:29:46.076 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:46.074+0000 7f9cb6ffd640 1 -- 172.21.15.37:0/4116267569 <== mon.0 v2:172.21.15.37:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187246 (secure 0 0 0) 0x7f9cb005f7f0 con 0x7f9cb81441d0 2024-04-25T10:29:46.260 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:46.258+0000 7f9cc05b2640 1 -- 172.21.15.37:0/4116267569 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) -- 0x7f9cb81e7c20 con 0x7f9cb81441d0 2024-04-25T10:29:46.260 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:46.258+0000 7f9cb6ffd640 1 -- 172.21.15.37:0/4116267569 <== mon.0 v2:172.21.15.37:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v23) ==== 93+0+10 (secure 0 0 0) 0x7f9cb005f0f0 con 0x7f9cb81441d0 2024-04-25T10:29:46.261 INFO:teuthology.orchestra.run.smithi037.stdout:pg_num: 1 2024-04-25T10:29:46.263 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:46.260+0000 7f9cc05b2640 1 -- 172.21.15.37:0/4116267569 >> [v2:172.21.15.203:6832/2418205237,v1:172.21.15.203:6833/2418205237] conn(0x7f9ca40744b0 msgr2=0x7f9ca40769c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:29:46.263 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:46.260+0000 7f9cc05b2640 1 --2- 172.21.15.37:0/4116267569 >> [v2:172.21.15.203:6832/2418205237,v1:172.21.15.203:6833/2418205237] conn(0x7f9ca40744b0 0x7f9ca40769c0 secure :-1 s=READY pgs=145 cs=0 l=1 rev1=1 crypto rx=0x7f9ca8034000 tx=0x7f9ca802e000 comp rx=0 tx=0).stop 2024-04-25T10:29:46.263 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:46.260+0000 7f9cc05b2640 1 -- 172.21.15.37:0/4116267569 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f9cb81441d0 msgr2=0x7f9cb8160ae0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:29:46.263 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:46.260+0000 7f9cc05b2640 1 --2- 172.21.15.37:0/4116267569 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f9cb81441d0 0x7f9cb8160ae0 secure :-1 s=READY pgs=267 cs=0 l=1 rev1=1 crypto rx=0x7f9cb000c6e0 tx=0x7f9cb000cba0 comp rx=0 tx=0).stop 2024-04-25T10:29:46.263 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:46.260+0000 7f9cc05b2640 1 -- 172.21.15.37:0/4116267569 shutdown_connections 2024-04-25T10:29:46.263 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:46.260+0000 7f9cc05b2640 1 --2- 172.21.15.37:0/4116267569 >> [v2:172.21.15.203:6832/2418205237,v1:172.21.15.203:6833/2418205237] conn(0x7f9ca40744b0 0x7f9ca40769c0 unknown :-1 s=CLOSED pgs=145 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:29:46.263 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:46.260+0000 7f9cc05b2640 1 --2- 172.21.15.37:0/4116267569 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f9cb81441d0 0x7f9cb8160ae0 unknown :-1 s=CLOSED pgs=267 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:29:46.263 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:46.260+0000 7f9cc05b2640 1 --2- 172.21.15.37:0/4116267569 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f9cb80d0a90 0x7f9cb81e6a80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:29:46.263 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:46.260+0000 7f9cc05b2640 1 --2- 172.21.15.37:0/4116267569 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f9cb80cefe0 0x7f9cb815b330 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:29:46.263 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:46.260+0000 7f9cc05b2640 1 -- 172.21.15.37:0/4116267569 >> 172.21.15.37:0/4116267569 conn(0x7f9cb80b36b0 msgr2=0x7f9cb815c3e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:29:46.263 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:46.260+0000 7f9cc05b2640 1 -- 172.21.15.37:0/4116267569 shutdown_connections 2024-04-25T10:29:46.264 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:46.260+0000 7f9cc05b2640 1 -- 172.21.15.37:0/4116267569 wait complete. 2024-04-25T10:29:46.274 DEBUG:teuthology.orchestra.run.smithi037:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_metadata pg_num 2024-04-25T10:29:46.389 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:46.386+0000 7fc6593c5640 1 Processor -- start 2024-04-25T10:29:46.389 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:46.387+0000 7fc6593c5640 1 -- start start 2024-04-25T10:29:46.391 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:46.388+0000 7fc6593c5640 1 --2- >> v2:172.21.15.37:3300/0 conn(0x7fc6540b80e0 0x7fc6540d0920 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:29:46.391 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:46.388+0000 7fc6593c5640 1 --2- >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7fc6540d0e60 0x7fc65415f7c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:29:46.391 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:46.388+0000 7fc6593c5640 1 -- --> v1:172.21.15.37:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fc6540d15e0 con 0x7fc6540b50e0 2024-04-25T10:29:46.391 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:46.388+0000 7fc6593c5640 1 -- --> v2:172.21.15.37:3300/0 -- mon_getmap magic: 0 -- 0x7fc6541646e0 con 0x7fc6540b80e0 2024-04-25T10:29:46.391 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:46.388+0000 7fc6593c5640 1 -- --> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] -- mon_getmap magic: 0 -- 0x7fc654163db0 con 0x7fc6540d0e60 2024-04-25T10:29:46.391 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:46.388+0000 7fc6527fc640 1 --2- >> v2:172.21.15.37:3300/0 conn(0x7fc6540b80e0 0x7fc6540d0920 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:29:46.391 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:46.388+0000 7fc652ffd640 1 --1- >> v1:172.21.15.37:6789/0 conn(0x7fc6540b50e0 0x7fc6540b79d0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.37:6789/0 says I am v1:172.21.15.37:49886/0 (socket says 172.21.15.37:49886) 2024-04-25T10:29:46.391 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:46.388+0000 7fc652ffd640 1 -- 172.21.15.37:0/2385736511 learned_addr learned my addr 172.21.15.37:0/2385736511 (peer_addr_for_me v1:172.21.15.37:0/0) 2024-04-25T10:29:46.391 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:46.388+0000 7fc6537fe640 1 --2- 172.21.15.37:0/2385736511 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7fc6540d0e60 0x7fc65415f7c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:29:46.391 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:46.388+0000 7fc6527fc640 1 -- 172.21.15.37:0/2385736511 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7fc6540d0e60 msgr2=0x7fc65415f7c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:29:46.391 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:46.388+0000 7fc6527fc640 1 --2- 172.21.15.37:0/2385736511 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7fc6540d0e60 0x7fc65415f7c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:29:46.391 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:46.388+0000 7fc6527fc640 1 -- 172.21.15.37:0/2385736511 >> v1:172.21.15.37:6789/0 conn(0x7fc6540b50e0 legacy=0x7fc6540b79d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:29:46.392 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:46.388+0000 7fc6527fc640 1 -- 172.21.15.37:0/2385736511 --> v2:172.21.15.37:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fc654164090 con 0x7fc6540b80e0 2024-04-25T10:29:46.392 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:46.388+0000 7fc651ffb640 1 -- 172.21.15.37:0/2385736511 <== mon.0 v1:172.21.15.37:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 3061932351 0 0) 0x7fc6540d15e0 con 0x7fc6540b50e0 2024-04-25T10:29:46.393 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:46.389+0000 7fc6527fc640 1 --2- 172.21.15.37:0/2385736511 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fc6540b80e0 0x7fc6540d0920 secure :-1 s=READY pgs=269 cs=0 l=1 rev1=1 crypto rx=0x7fc654164860 tx=0x7fc63c02a180 comp rx=0 tx=0).ready entity=mon.0 client_cookie=21261dc796425778 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:29:46.393 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:46.389+0000 7fc651ffb640 1 -- 172.21.15.37:0/2385736511 <== mon.0 v2:172.21.15.37:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc63c037070 con 0x7fc6540b80e0 2024-04-25T10:29:46.393 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:46.389+0000 7fc651ffb640 1 -- 172.21.15.37:0/2385736511 <== mon.0 v2:172.21.15.37:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fc63c032440 con 0x7fc6540b80e0 2024-04-25T10:29:46.393 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:46.389+0000 7fc651ffb640 1 -- 172.21.15.37:0/2385736511 <== mon.0 v2:172.21.15.37:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc63c032810 con 0x7fc6540b80e0 2024-04-25T10:29:46.393 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:46.390+0000 7fc6593c5640 1 -- 172.21.15.37:0/2385736511 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fc6540b80e0 msgr2=0x7fc6540d0920 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:29:46.393 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:46.390+0000 7fc6593c5640 1 --2- 172.21.15.37:0/2385736511 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fc6540b80e0 0x7fc6540d0920 secure :-1 s=READY pgs=269 cs=0 l=1 rev1=1 crypto rx=0x7fc654164860 tx=0x7fc63c02a180 comp rx=0 tx=0).stop 2024-04-25T10:29:46.393 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:46.390+0000 7fc6593c5640 1 -- 172.21.15.37:0/2385736511 shutdown_connections 2024-04-25T10:29:46.393 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:46.390+0000 7fc6593c5640 1 --2- 172.21.15.37:0/2385736511 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7fc6540d0e60 0x7fc65415f7c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:29:46.394 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:46.390+0000 7fc6593c5640 1 --2- 172.21.15.37:0/2385736511 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fc6540b80e0 0x7fc6540d0920 unknown :-1 s=CLOSED pgs=269 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:29:46.394 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:46.390+0000 7fc6593c5640 1 -- 172.21.15.37:0/2385736511 >> 172.21.15.37:0/2385736511 conn(0x7fc6540d5d50 msgr2=0x7fc6540d6160 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:29:46.394 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:46.390+0000 7fc6593c5640 1 -- 172.21.15.37:0/2385736511 shutdown_connections 2024-04-25T10:29:46.394 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:46.390+0000 7fc6593c5640 1 -- 172.21.15.37:0/2385736511 wait complete. 2024-04-25T10:29:46.394 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:46.392+0000 7fc6593c5640 1 Processor -- start 2024-04-25T10:29:46.394 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:46.392+0000 7fc6593c5640 1 -- start start 2024-04-25T10:29:46.397 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:46.393+0000 7fc6593c5640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fc6540b50e0 0x7fc654160b10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:29:46.397 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:46.393+0000 7fc6593c5640 1 --2- >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7fc6540d0e60 0x7fc6540cee40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:29:46.397 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:46.393+0000 7fc6593c5640 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fc6541540f0 0x7fc6540b54f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:29:46.397 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:46.393+0000 7fc6593c5640 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 -- 0x7fc6540d1270 con 0x7fc6540b50e0 2024-04-25T10:29:46.397 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:46.393+0000 7fc6593c5640 1 -- --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_getmap magic: 0 -- 0x7fc654164b70 con 0x7fc6541540f0 2024-04-25T10:29:46.397 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:46.393+0000 7fc6593c5640 1 -- --> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] -- mon_getmap magic: 0 -- 0x7fc65415fd00 con 0x7fc6540d0e60 2024-04-25T10:29:46.398 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:46.393+0000 7fc652ffd640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fc6540b50e0 0x7fc654160b10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:29:46.398 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:46.393+0000 7fc652ffd640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fc6540b50e0 0x7fc654160b10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.37:3300/0 says I am v2:172.21.15.37:59604/0 (socket says 172.21.15.37:59604) 2024-04-25T10:29:46.398 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:46.393+0000 7fc652ffd640 1 -- 172.21.15.37:0/1717543400 learned_addr learned my addr 172.21.15.37:0/1717543400 (peer_addr_for_me v2:172.21.15.37:0/0) 2024-04-25T10:29:46.398 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:46.394+0000 7fc6527fc640 1 --2- 172.21.15.37:0/1717543400 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7fc6540d0e60 0x7fc6540cee40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:29:46.398 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:46.394+0000 7fc6537fe640 1 --2- 172.21.15.37:0/1717543400 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fc6541540f0 0x7fc6540b54f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:29:46.398 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:46.394+0000 7fc652ffd640 1 -- 172.21.15.37:0/1717543400 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7fc6540d0e60 msgr2=0x7fc6540cee40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:29:46.398 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:46.394+0000 7fc652ffd640 1 --2- 172.21.15.37:0/1717543400 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7fc6540d0e60 0x7fc6540cee40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:29:46.398 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:46.394+0000 7fc652ffd640 1 -- 172.21.15.37:0/1717543400 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fc6541540f0 msgr2=0x7fc6540b54f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:29:46.398 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:46.394+0000 7fc652ffd640 1 --2- 172.21.15.37:0/1717543400 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fc6541540f0 0x7fc6540b54f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:29:46.398 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:46.394+0000 7fc652ffd640 1 -- 172.21.15.37:0/1717543400 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fc6541612d0 con 0x7fc6540b50e0 2024-04-25T10:29:46.398 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:46.394+0000 7fc6537fe640 1 --2- 172.21.15.37:0/1717543400 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fc6541540f0 0x7fc6540b54f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T10:29:46.398 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:46.394+0000 7fc6527fc640 1 --2- 172.21.15.37:0/1717543400 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7fc6540d0e60 0x7fc6540cee40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T10:29:46.398 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:46.394+0000 7fc652ffd640 1 --2- 172.21.15.37:0/1717543400 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fc6540b50e0 0x7fc654160b10 secure :-1 s=READY pgs=270 cs=0 l=1 rev1=1 crypto rx=0x7fc64c00c440 tx=0x7fc64c00c900 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:29:46.398 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:46.394+0000 7fc64b7fe640 1 -- 172.21.15.37:0/1717543400 <== mon.0 v2:172.21.15.37:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc64c018070 con 0x7fc6540b50e0 2024-04-25T10:29:46.399 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:46.394+0000 7fc64b7fe640 1 -- 172.21.15.37:0/1717543400 <== mon.0 v2:172.21.15.37:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fc64c0074e0 con 0x7fc6540b50e0 2024-04-25T10:29:46.399 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:46.394+0000 7fc6593c5640 1 -- 172.21.15.37:0/1717543400 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fc6541615b0 con 0x7fc6540b50e0 2024-04-25T10:29:46.399 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:46.394+0000 7fc64b7fe640 1 -- 172.21.15.37:0/1717543400 <== mon.0 v2:172.21.15.37:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc64c004a20 con 0x7fc6540b50e0 2024-04-25T10:29:46.399 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:46.394+0000 7fc6593c5640 1 -- 172.21.15.37:0/1717543400 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fc654161b70 con 0x7fc6540b50e0 2024-04-25T10:29:46.399 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:46.395+0000 7fc6593c5640 1 -- 172.21.15.37:0/1717543400 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fc65406a140 con 0x7fc6540b50e0 2024-04-25T10:29:46.400 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:46.396+0000 7fc64b7fe640 1 -- 172.21.15.37:0/1717543400 <== mon.0 v2:172.21.15.37:3300/0 4 ==== mgrmap(e 5) ==== 96782+0+0 (secure 0 0 0) 0x7fc64c01b070 con 0x7fc6540b50e0 2024-04-25T10:29:46.400 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:46.396+0000 7fc64b7fe640 1 --2- 172.21.15.37:0/1717543400 >> [v2:172.21.15.203:6832/2418205237,v1:172.21.15.203:6833/2418205237] conn(0x7fc6340744b0 0x7fc6340769c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:29:46.400 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:46.397+0000 7fc64b7fe640 1 -- 172.21.15.37:0/1717543400 <== mon.0 v2:172.21.15.37:3300/0 5 ==== osd_map(23..23 src has 1..23) ==== 5709+0+0 (secure 0 0 0) 0x7fc64c0944f0 con 0x7fc6540b50e0 2024-04-25T10:29:46.400 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:46.397+0000 7fc6527fc640 1 --2- 172.21.15.37:0/1717543400 >> [v2:172.21.15.203:6832/2418205237,v1:172.21.15.203:6833/2418205237] conn(0x7fc6340744b0 0x7fc6340769c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:29:46.400 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:46.397+0000 7fc6527fc640 1 --2- 172.21.15.37:0/1717543400 >> [v2:172.21.15.203:6832/2418205237,v1:172.21.15.203:6833/2418205237] conn(0x7fc6340744b0 0x7fc6340769c0 secure :-1 s=READY pgs=146 cs=0 l=1 rev1=1 crypto rx=0x7fc63c034000 tx=0x7fc63c02e000 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:29:46.403 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:46.401+0000 7fc64b7fe640 1 -- 172.21.15.37:0/1717543400 <== mon.0 v2:172.21.15.37:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187246 (secure 0 0 0) 0x7fc64c05ec70 con 0x7fc6540b50e0 2024-04-25T10:29:46.588 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:46.586+0000 7fc6593c5640 1 -- 172.21.15.37:0/1717543400 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"} v 0) -- 0x7fc6540d4000 con 0x7fc6540b50e0 2024-04-25T10:29:46.589 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:46.586+0000 7fc64b7fe640 1 -- 172.21.15.37:0/1717543400 <== mon.0 v2:172.21.15.37:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]=0 v23) ==== 104+0+11 (secure 0 0 0) 0x7fc64c05e570 con 0x7fc6540b50e0 2024-04-25T10:29:46.589 INFO:teuthology.orchestra.run.smithi037.stdout:pg_num: 32 2024-04-25T10:29:46.591 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:46.588+0000 7fc6593c5640 1 -- 172.21.15.37:0/1717543400 >> [v2:172.21.15.203:6832/2418205237,v1:172.21.15.203:6833/2418205237] conn(0x7fc6340744b0 msgr2=0x7fc6340769c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:29:46.591 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:46.588+0000 7fc6593c5640 1 --2- 172.21.15.37:0/1717543400 >> [v2:172.21.15.203:6832/2418205237,v1:172.21.15.203:6833/2418205237] conn(0x7fc6340744b0 0x7fc6340769c0 secure :-1 s=READY pgs=146 cs=0 l=1 rev1=1 crypto rx=0x7fc63c034000 tx=0x7fc63c02e000 comp rx=0 tx=0).stop 2024-04-25T10:29:46.591 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:46.588+0000 7fc6593c5640 1 -- 172.21.15.37:0/1717543400 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fc6540b50e0 msgr2=0x7fc654160b10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:29:46.591 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:46.588+0000 7fc6593c5640 1 --2- 172.21.15.37:0/1717543400 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fc6540b50e0 0x7fc654160b10 secure :-1 s=READY pgs=270 cs=0 l=1 rev1=1 crypto rx=0x7fc64c00c440 tx=0x7fc64c00c900 comp rx=0 tx=0).stop 2024-04-25T10:29:46.591 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:46.588+0000 7fc6593c5640 1 -- 172.21.15.37:0/1717543400 shutdown_connections 2024-04-25T10:29:46.591 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:46.588+0000 7fc6593c5640 1 --2- 172.21.15.37:0/1717543400 >> [v2:172.21.15.203:6832/2418205237,v1:172.21.15.203:6833/2418205237] conn(0x7fc6340744b0 0x7fc6340769c0 unknown :-1 s=CLOSED pgs=146 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:29:46.591 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:46.588+0000 7fc6593c5640 1 --2- 172.21.15.37:0/1717543400 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fc6541540f0 0x7fc6540b54f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:29:46.591 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:46.588+0000 7fc6593c5640 1 --2- 172.21.15.37:0/1717543400 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7fc6540d0e60 0x7fc6540cee40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:29:46.591 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:46.588+0000 7fc6593c5640 1 --2- 172.21.15.37:0/1717543400 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fc6540b50e0 0x7fc654160b10 unknown :-1 s=CLOSED pgs=270 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:29:46.591 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:46.588+0000 7fc6593c5640 1 -- 172.21.15.37:0/1717543400 >> 172.21.15.37:0/1717543400 conn(0x7fc6540d5d50 msgr2=0x7fc6540b6750 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:29:46.591 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:46.588+0000 7fc6593c5640 1 -- 172.21.15.37:0/1717543400 shutdown_connections 2024-04-25T10:29:46.591 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:46.588+0000 7fc6593c5640 1 -- 172.21.15.37:0/1717543400 wait complete. 2024-04-25T10:29:46.601 DEBUG:teuthology.orchestra.run.smithi037:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data pg_num 2024-04-25T10:29:46.716 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:46.714+0000 7fcb34a6e640 1 Processor -- start 2024-04-25T10:29:46.717 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:46.714+0000 7fcb34a6e640 1 -- start start 2024-04-25T10:29:46.718 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:46.716+0000 7fcb34a6e640 1 --2- >> v2:172.21.15.203:3300/0 conn(0x7fcb300d0c60 0x7fcb300ce8d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:29:46.718 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:46.716+0000 7fcb34a6e640 1 --2- >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7fcb300cee10 0x7fcb3006f3a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:29:46.719 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:46.716+0000 7fcb34a6e640 1 -- --> v1:172.21.15.37:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fcb300cf600 con 0x7fcb301441d0 2024-04-25T10:29:46.719 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:46.716+0000 7fcb34a6e640 1 -- --> v2:172.21.15.203:3300/0 -- mon_getmap magic: 0 -- 0x7fcb300cf8e0 con 0x7fcb300d0c60 2024-04-25T10:29:46.719 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:46.716+0000 7fcb34a6e640 1 -- --> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] -- mon_getmap magic: 0 -- 0x7fcb300cfbc0 con 0x7fcb300cee10 2024-04-25T10:29:46.719 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:46.716+0000 7fcb2e59c640 1 --1- >> v1:172.21.15.37:6789/0 conn(0x7fcb301441d0 0x7fcb300d0550 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.37:6789/0 says I am v1:172.21.15.37:49894/0 (socket says 172.21.15.37:49894) 2024-04-25T10:29:46.719 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:46.716+0000 7fcb2e59c640 1 -- 172.21.15.37:0/2857933834 learned_addr learned my addr 172.21.15.37:0/2857933834 (peer_addr_for_me v1:172.21.15.37:0/0) 2024-04-25T10:29:46.719 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:46.716+0000 7fcb2dd9b640 1 --2- 172.21.15.37:0/2857933834 >> v2:172.21.15.203:3300/0 conn(0x7fcb300d0c60 0x7fcb300ce8d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:29:46.719 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:46.716+0000 7fcb2ed9d640 1 --2- 172.21.15.37:0/2857933834 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7fcb300cee10 0x7fcb3006f3a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:29:46.719 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:46.716+0000 7fcb2d59a640 1 -- 172.21.15.37:0/2857933834 <== mon.0 v1:172.21.15.37:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 4253562431 0 0) 0x7fcb300cf600 con 0x7fcb301441d0 2024-04-25T10:29:46.719 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:46.716+0000 7fcb2d59a640 1 -- 172.21.15.37:0/2857933834 --> v1:172.21.15.37:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7fcb1c003520 con 0x7fcb301441d0 2024-04-25T10:29:46.719 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:46.716+0000 7fcb2d59a640 1 -- 172.21.15.37:0/2857933834 <== mon.0 v1:172.21.15.37:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 2994373066 0 0) 0x7fcb1c003520 con 0x7fcb301441d0 2024-04-25T10:29:46.719 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:46.716+0000 7fcb2dd9b640 1 -- 172.21.15.37:0/2857933834 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7fcb300cee10 msgr2=0x7fcb3006f3a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:29:46.719 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:46.716+0000 7fcb2dd9b640 1 --2- 172.21.15.37:0/2857933834 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7fcb300cee10 0x7fcb3006f3a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:29:46.719 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:46.716+0000 7fcb2dd9b640 1 -- 172.21.15.37:0/2857933834 >> v1:172.21.15.37:6789/0 conn(0x7fcb301441d0 legacy=0x7fcb300d0550 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:29:46.719 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:46.716+0000 7fcb2dd9b640 1 -- 172.21.15.37:0/2857933834 --> v2:172.21.15.203:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fcb300cfea0 con 0x7fcb300d0c60 2024-04-25T10:29:46.720 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:46.716+0000 7fcb2ed9d640 1 --2- 172.21.15.37:0/2857933834 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7fcb300cee10 0x7fcb3006f3a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-25T10:29:46.721 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:46.717+0000 7fcb2dd9b640 1 --2- 172.21.15.37:0/2857933834 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fcb300d0c60 0x7fcb300ce8d0 secure :-1 s=READY pgs=127 cs=0 l=1 rev1=1 crypto rx=0x7fcb180097a0 tx=0x7fcb1802a110 comp rx=0 tx=0).ready entity=mon.1 client_cookie=ccb7fa58d24d4b35 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:29:46.721 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:46.717+0000 7fcb2d59a640 1 -- 172.21.15.37:0/2857933834 <== mon.1 v2:172.21.15.203:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcb18037070 con 0x7fcb300d0c60 2024-04-25T10:29:46.721 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:46.717+0000 7fcb2d59a640 1 -- 172.21.15.37:0/2857933834 <== mon.1 v2:172.21.15.203:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fcb1802aba0 con 0x7fcb300d0c60 2024-04-25T10:29:46.721 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:46.717+0000 7fcb2d59a640 1 -- 172.21.15.37:0/2857933834 <== mon.1 v2:172.21.15.203:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcb28001670 con 0x7fcb300d0c60 2024-04-25T10:29:46.721 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:46.718+0000 7fcb34a6e640 1 -- 172.21.15.37:0/2857933834 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fcb300d0c60 msgr2=0x7fcb300ce8d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:29:46.721 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:46.718+0000 7fcb34a6e640 1 --2- 172.21.15.37:0/2857933834 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fcb300d0c60 0x7fcb300ce8d0 secure :-1 s=READY pgs=127 cs=0 l=1 rev1=1 crypto rx=0x7fcb180097a0 tx=0x7fcb1802a110 comp rx=0 tx=0).stop 2024-04-25T10:29:46.721 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:46.718+0000 7fcb34a6e640 1 -- 172.21.15.37:0/2857933834 shutdown_connections 2024-04-25T10:29:46.721 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:46.718+0000 7fcb34a6e640 1 --2- 172.21.15.37:0/2857933834 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7fcb300cee10 0x7fcb3006f3a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:29:46.721 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:46.718+0000 7fcb34a6e640 1 --2- 172.21.15.37:0/2857933834 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fcb300d0c60 0x7fcb300ce8d0 unknown :-1 s=CLOSED pgs=127 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:29:46.721 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:46.718+0000 7fcb34a6e640 1 -- 172.21.15.37:0/2857933834 >> 172.21.15.37:0/2857933834 conn(0x7fcb300b36b0 msgr2=0x7fcb300b3ac0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:29:46.721 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:46.718+0000 7fcb34a6e640 1 -- 172.21.15.37:0/2857933834 shutdown_connections 2024-04-25T10:29:46.721 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:46.718+0000 7fcb34a6e640 1 -- 172.21.15.37:0/2857933834 wait complete. 2024-04-25T10:29:46.722 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:46.720+0000 7fcb34a6e640 1 Processor -- start 2024-04-25T10:29:46.722 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:46.720+0000 7fcb34a6e640 1 -- start start 2024-04-25T10:29:46.725 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:46.721+0000 7fcb34a6e640 1 --2- >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7fcb300cee10 0x7fcb30160f80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:29:46.725 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:46.721+0000 7fcb34a6e640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fcb301441d0 0x7fcb301475a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:29:46.725 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:46.721+0000 7fcb34a6e640 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fcb301f3960 0x7fcb301f3d70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:29:46.725 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:46.721+0000 7fcb34a6e640 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 -- 0x7fcb300cf3a0 con 0x7fcb301441d0 2024-04-25T10:29:46.725 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:46.721+0000 7fcb34a6e640 1 -- --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_getmap magic: 0 -- 0x7fcb300cf220 con 0x7fcb301f3960 2024-04-25T10:29:46.725 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:46.722+0000 7fcb34a6e640 1 -- --> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] -- mon_getmap magic: 0 -- 0x7fcb301445e0 con 0x7fcb300cee10 2024-04-25T10:29:46.725 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:46.722+0000 7fcb2dd9b640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fcb301441d0 0x7fcb301475a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:29:46.725 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:46.722+0000 7fcb2dd9b640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fcb301441d0 0x7fcb301475a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.37:3300/0 says I am v2:172.21.15.37:59612/0 (socket says 172.21.15.37:59612) 2024-04-25T10:29:46.726 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:46.722+0000 7fcb2dd9b640 1 -- 172.21.15.37:0/2448127211 learned_addr learned my addr 172.21.15.37:0/2448127211 (peer_addr_for_me v2:172.21.15.37:0/0) 2024-04-25T10:29:46.726 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:46.722+0000 7fcb2e59c640 1 --2- 172.21.15.37:0/2448127211 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7fcb300cee10 0x7fcb30160f80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:29:46.726 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:46.722+0000 7fcb2ed9d640 1 --2- 172.21.15.37:0/2448127211 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fcb301f3960 0x7fcb301f3d70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:29:46.726 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:46.722+0000 7fcb2dd9b640 1 -- 172.21.15.37:0/2448127211 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7fcb300cee10 msgr2=0x7fcb30160f80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:29:46.726 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:46.722+0000 7fcb2dd9b640 1 --2- 172.21.15.37:0/2448127211 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7fcb300cee10 0x7fcb30160f80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:29:46.726 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:46.722+0000 7fcb2dd9b640 1 -- 172.21.15.37:0/2448127211 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fcb301f3960 msgr2=0x7fcb301f3d70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:29:46.726 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:46.722+0000 7fcb2dd9b640 1 --2- 172.21.15.37:0/2448127211 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fcb301f3960 0x7fcb301f3d70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:29:46.726 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:46.722+0000 7fcb2dd9b640 1 -- 172.21.15.37:0/2448127211 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fcb301604e0 con 0x7fcb301441d0 2024-04-25T10:29:46.726 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:46.722+0000 7fcb2ed9d640 1 --2- 172.21.15.37:0/2448127211 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fcb301f3960 0x7fcb301f3d70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T10:29:46.726 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:46.722+0000 7fcb2dd9b640 1 --2- 172.21.15.37:0/2448127211 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fcb301441d0 0x7fcb301475a0 secure :-1 s=READY pgs=272 cs=0 l=1 rev1=1 crypto rx=0x7fcb180098c0 tx=0x7fcb180046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:29:46.726 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:46.722+0000 7fcb26ffd640 1 -- 172.21.15.37:0/2448127211 <== mon.0 v2:172.21.15.37:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcb18041070 con 0x7fcb301441d0 2024-04-25T10:29:46.726 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:46.722+0000 7fcb26ffd640 1 -- 172.21.15.37:0/2448127211 <== mon.0 v2:172.21.15.37:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fcb1803c660 con 0x7fcb301441d0 2024-04-25T10:29:46.726 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:46.722+0000 7fcb34a6e640 1 -- 172.21.15.37:0/2448127211 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fcb301e70b0 con 0x7fcb301441d0 2024-04-25T10:29:46.726 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:46.722+0000 7fcb26ffd640 1 -- 172.21.15.37:0/2448127211 <== mon.0 v2:172.21.15.37:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcb18037040 con 0x7fcb301441d0 2024-04-25T10:29:46.726 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:46.722+0000 7fcb34a6e640 1 -- 172.21.15.37:0/2448127211 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fcb30160780 con 0x7fcb301441d0 2024-04-25T10:29:46.727 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:46.724+0000 7fcb34a6e640 1 -- 172.21.15.37:0/2448127211 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fcb3006a140 con 0x7fcb301441d0 2024-04-25T10:29:46.727 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:46.724+0000 7fcb26ffd640 1 -- 172.21.15.37:0/2448127211 <== mon.0 v2:172.21.15.37:3300/0 4 ==== mgrmap(e 5) ==== 96782+0+0 (secure 0 0 0) 0x7fcb18044070 con 0x7fcb301441d0 2024-04-25T10:29:46.727 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:46.725+0000 7fcb26ffd640 1 --2- 172.21.15.37:0/2448127211 >> [v2:172.21.15.203:6832/2418205237,v1:172.21.15.203:6833/2418205237] conn(0x7fcb100744b0 0x7fcb100769c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:29:46.728 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:46.725+0000 7fcb26ffd640 1 -- 172.21.15.37:0/2448127211 <== mon.0 v2:172.21.15.37:3300/0 5 ==== osd_map(23..23 src has 1..23) ==== 5709+0+0 (secure 0 0 0) 0x7fcb180b59d0 con 0x7fcb301441d0 2024-04-25T10:29:46.728 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:46.725+0000 7fcb2e59c640 1 --2- 172.21.15.37:0/2448127211 >> [v2:172.21.15.203:6832/2418205237,v1:172.21.15.203:6833/2418205237] conn(0x7fcb100744b0 0x7fcb100769c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:29:46.728 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:46.725+0000 7fcb2e59c640 1 --2- 172.21.15.37:0/2448127211 >> [v2:172.21.15.203:6832/2418205237,v1:172.21.15.203:6833/2418205237] conn(0x7fcb100744b0 0x7fcb100769c0 secure :-1 s=READY pgs=147 cs=0 l=1 rev1=1 crypto rx=0x7fcb28005880 tx=0x7fcb28009040 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:29:46.731 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:46.729+0000 7fcb26ffd640 1 -- 172.21.15.37:0/2448127211 <== mon.0 v2:172.21.15.37:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187246 (secure 0 0 0) 0x7fcb18080160 con 0x7fcb301441d0 2024-04-25T10:29:46.915 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:46.913+0000 7fcb34a6e640 1 -- 172.21.15.37:0/2448127211 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"} v 0) -- 0x7fcb300b4580 con 0x7fcb301441d0 2024-04-25T10:29:46.915 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:46.913+0000 7fcb26ffd640 1 -- 172.21.15.37:0/2448127211 <== mon.0 v2:172.21.15.37:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]=0 v23) ==== 100+0+11 (secure 0 0 0) 0x7fcb1807fa60 con 0x7fcb301441d0 2024-04-25T10:29:46.915 INFO:teuthology.orchestra.run.smithi037.stdout:pg_num: 64 2024-04-25T10:29:46.918 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:46.915+0000 7fcb34a6e640 1 -- 172.21.15.37:0/2448127211 >> [v2:172.21.15.203:6832/2418205237,v1:172.21.15.203:6833/2418205237] conn(0x7fcb100744b0 msgr2=0x7fcb100769c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:29:46.918 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:46.915+0000 7fcb34a6e640 1 --2- 172.21.15.37:0/2448127211 >> [v2:172.21.15.203:6832/2418205237,v1:172.21.15.203:6833/2418205237] conn(0x7fcb100744b0 0x7fcb100769c0 secure :-1 s=READY pgs=147 cs=0 l=1 rev1=1 crypto rx=0x7fcb28005880 tx=0x7fcb28009040 comp rx=0 tx=0).stop 2024-04-25T10:29:46.918 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:46.915+0000 7fcb34a6e640 1 -- 172.21.15.37:0/2448127211 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fcb301441d0 msgr2=0x7fcb301475a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:29:46.918 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:46.915+0000 7fcb34a6e640 1 --2- 172.21.15.37:0/2448127211 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fcb301441d0 0x7fcb301475a0 secure :-1 s=READY pgs=272 cs=0 l=1 rev1=1 crypto rx=0x7fcb180098c0 tx=0x7fcb180046c0 comp rx=0 tx=0).stop 2024-04-25T10:29:46.918 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:46.915+0000 7fcb34a6e640 1 -- 172.21.15.37:0/2448127211 shutdown_connections 2024-04-25T10:29:46.918 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:46.915+0000 7fcb34a6e640 1 --2- 172.21.15.37:0/2448127211 >> [v2:172.21.15.203:6832/2418205237,v1:172.21.15.203:6833/2418205237] conn(0x7fcb100744b0 0x7fcb100769c0 unknown :-1 s=CLOSED pgs=147 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:29:46.918 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:46.915+0000 7fcb34a6e640 1 --2- 172.21.15.37:0/2448127211 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fcb301f3960 0x7fcb301f3d70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:29:46.918 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:46.915+0000 7fcb34a6e640 1 --2- 172.21.15.37:0/2448127211 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fcb301441d0 0x7fcb301475a0 unknown :-1 s=CLOSED pgs=272 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:29:46.918 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:46.915+0000 7fcb34a6e640 1 --2- 172.21.15.37:0/2448127211 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7fcb300cee10 0x7fcb30160f80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:29:46.919 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:46.915+0000 7fcb34a6e640 1 -- 172.21.15.37:0/2448127211 >> 172.21.15.37:0/2448127211 conn(0x7fcb300b36b0 msgr2=0x7fcb300b26a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:29:46.919 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:46.915+0000 7fcb34a6e640 1 -- 172.21.15.37:0/2448127211 shutdown_connections 2024-04-25T10:29:46.919 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:46.915+0000 7fcb34a6e640 1 -- 172.21.15.37:0/2448127211 wait complete. 2024-04-25T10:29:46.929 DEBUG:teuthology.orchestra.run.smithi037:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd blocklist ls 2024-04-25T10:29:47.042 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:47.039+0000 7f6f4495e640 1 Processor -- start 2024-04-25T10:29:47.042 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:47.039+0000 7f6f4495e640 1 -- start start 2024-04-25T10:29:47.044 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:47.041+0000 7f6f4495e640 1 --2- >> v2:172.21.15.37:3300/0 conn(0x7f6f400d5160 0x7f6f400b4e80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:29:47.044 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:47.041+0000 7f6f4495e640 1 -- --> v1:172.21.15.37:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f6f400d1370 con 0x7f6f400d3230 2024-04-25T10:29:47.044 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:47.041+0000 7f6f4495e640 1 -- --> v1:172.21.15.203:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f6f400d4680 con 0x7f6f400d0990 2024-04-25T10:29:47.044 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:47.041+0000 7f6f4495e640 1 -- --> v2:172.21.15.37:3300/0 -- mon_getmap magic: 0 -- 0x7f6f400d1650 con 0x7f6f400d5160 2024-04-25T10:29:47.044 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:47.041+0000 7f6f3e59c640 1 --2- >> v2:172.21.15.37:3300/0 conn(0x7f6f400d5160 0x7f6f400b4e80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:29:47.044 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:47.041+0000 7f6f3ed9d640 1 --1- >> v1:172.21.15.37:6789/0 conn(0x7f6f400d3230 0x7f6f400b9060 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.37:6789/0 says I am v1:172.21.15.37:49900/0 (socket says 172.21.15.37:49900) 2024-04-25T10:29:47.044 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:47.041+0000 7f6f3ed9d640 1 -- 172.21.15.37:0/4222938072 learned_addr learned my addr 172.21.15.37:0/4222938072 (peer_addr_for_me v1:172.21.15.37:0/0) 2024-04-25T10:29:47.044 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:47.041+0000 7f6f3e59c640 1 -- 172.21.15.37:0/4222938072 >> v1:172.21.15.203:6789/0 conn(0x7f6f400d0990 legacy=0x7f6f400d2b20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:29:47.044 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:47.041+0000 7f6f3e59c640 1 -- 172.21.15.37:0/4222938072 >> v1:172.21.15.37:6789/0 conn(0x7f6f400d3230 legacy=0x7f6f400b9060 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:29:47.044 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:47.041+0000 7f6f3d59a640 1 -- 172.21.15.37:0/4222938072 <== mon.0 v1:172.21.15.37:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 4185445404 0 0) 0x7f6f400d1370 con 0x7f6f400d3230 2024-04-25T10:29:47.044 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:47.041+0000 7f6f3e59c640 1 -- 172.21.15.37:0/4222938072 --> v2:172.21.15.37:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f6f400d4960 con 0x7f6f400d5160 2024-04-25T10:29:47.045 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:47.042+0000 7f6f3e59c640 1 --2- 172.21.15.37:0/4222938072 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f6f400d5160 0x7f6f400b4e80 secure :-1 s=READY pgs=274 cs=0 l=1 rev1=1 crypto rx=0x7f6f380097a0 tx=0x7f6f3802a0b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=6af10e0943de5f8a server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:29:47.045 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:47.042+0000 7f6f3d59a640 1 -- 172.21.15.37:0/4222938072 <== mon.0 v2:172.21.15.37:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6f38037070 con 0x7f6f400d5160 2024-04-25T10:29:47.046 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:47.042+0000 7f6f3d59a640 1 -- 172.21.15.37:0/4222938072 <== mon.0 v2:172.21.15.37:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f6f3802adc0 con 0x7f6f400d5160 2024-04-25T10:29:47.046 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:47.042+0000 7f6f3d59a640 1 -- 172.21.15.37:0/4222938072 <== mon.0 v2:172.21.15.37:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6f38032660 con 0x7f6f400d5160 2024-04-25T10:29:47.046 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:47.043+0000 7f6f4495e640 1 -- 172.21.15.37:0/4222938072 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f6f400d5160 msgr2=0x7f6f400b4e80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:29:47.046 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:47.043+0000 7f6f4495e640 1 --2- 172.21.15.37:0/4222938072 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f6f400d5160 0x7f6f400b4e80 secure :-1 s=READY pgs=274 cs=0 l=1 rev1=1 crypto rx=0x7f6f380097a0 tx=0x7f6f3802a0b0 comp rx=0 tx=0).stop 2024-04-25T10:29:47.046 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:47.043+0000 7f6f4495e640 1 -- 172.21.15.37:0/4222938072 shutdown_connections 2024-04-25T10:29:47.046 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:47.043+0000 7f6f4495e640 1 --2- 172.21.15.37:0/4222938072 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f6f400d5160 0x7f6f400b4e80 unknown :-1 s=CLOSED pgs=274 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:29:47.046 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:47.043+0000 7f6f4495e640 1 -- 172.21.15.37:0/4222938072 >> 172.21.15.37:0/4222938072 conn(0x7f6f400b7a80 msgr2=0x7f6f400b7e90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:29:47.046 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:47.043+0000 7f6f4495e640 1 -- 172.21.15.37:0/4222938072 shutdown_connections 2024-04-25T10:29:47.046 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:47.043+0000 7f6f4495e640 1 -- 172.21.15.37:0/4222938072 wait complete. 2024-04-25T10:29:47.047 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:47.045+0000 7f6f4495e640 1 Processor -- start 2024-04-25T10:29:47.047 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:47.045+0000 7f6f4495e640 1 -- start start 2024-04-25T10:29:47.050 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:47.046+0000 7f6f4495e640 1 --2- >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f6f400d0990 0x7f6f40154820 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:29:47.050 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:47.046+0000 7f6f4495e640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f6f400d5160 0x7f6f40159ba0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:29:47.050 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:47.046+0000 7f6f4495e640 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f6f401f73d0 0x7f6f4015b280 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:29:47.050 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:47.046+0000 7f6f4495e640 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 -- 0x7f6f40164c40 con 0x7f6f400d5160 2024-04-25T10:29:47.050 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:47.046+0000 7f6f4495e640 1 -- --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_getmap magic: 0 -- 0x7f6f40164ac0 con 0x7f6f401f73d0 2024-04-25T10:29:47.050 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:47.046+0000 7f6f4495e640 1 -- --> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] -- mon_getmap magic: 0 -- 0x7f6f400d0da0 con 0x7f6f400d0990 2024-04-25T10:29:47.050 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:47.046+0000 7f6f3dd9b640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f6f400d5160 0x7f6f40159ba0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:29:47.050 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:47.046+0000 7f6f3dd9b640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f6f400d5160 0x7f6f40159ba0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.37:3300/0 says I am v2:172.21.15.37:59632/0 (socket says 172.21.15.37:59632) 2024-04-25T10:29:47.050 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:47.046+0000 7f6f3dd9b640 1 -- 172.21.15.37:0/2553600169 learned_addr learned my addr 172.21.15.37:0/2553600169 (peer_addr_for_me v2:172.21.15.37:0/0) 2024-04-25T10:29:47.051 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:47.047+0000 7f6f3e59c640 1 --2- 172.21.15.37:0/2553600169 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f6f400d0990 0x7f6f40154820 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:29:47.051 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:47.047+0000 7f6f3ed9d640 1 --2- 172.21.15.37:0/2553600169 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f6f401f73d0 0x7f6f4015b280 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:29:47.051 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:47.047+0000 7f6f3dd9b640 1 -- 172.21.15.37:0/2553600169 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f6f400d0990 msgr2=0x7f6f40154820 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:29:47.051 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:47.047+0000 7f6f3dd9b640 1 --2- 172.21.15.37:0/2553600169 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f6f400d0990 0x7f6f40154820 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:29:47.051 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:47.047+0000 7f6f3dd9b640 1 -- 172.21.15.37:0/2553600169 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f6f401f73d0 msgr2=0x7f6f4015b280 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:29:47.051 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:47.047+0000 7f6f3dd9b640 1 --2- 172.21.15.37:0/2553600169 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f6f401f73d0 0x7f6f4015b280 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:29:47.051 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:47.047+0000 7f6f3dd9b640 1 -- 172.21.15.37:0/2553600169 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f6f40158fb0 con 0x7f6f400d5160 2024-04-25T10:29:47.051 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:47.047+0000 7f6f3e59c640 1 --2- 172.21.15.37:0/2553600169 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f6f400d0990 0x7f6f40154820 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T10:29:47.051 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:47.047+0000 7f6f3ed9d640 1 --2- 172.21.15.37:0/2553600169 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f6f401f73d0 0x7f6f4015b280 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T10:29:47.051 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:47.047+0000 7f6f3dd9b640 1 --2- 172.21.15.37:0/2553600169 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f6f400d5160 0x7f6f40159ba0 secure :-1 s=READY pgs=275 cs=0 l=1 rev1=1 crypto rx=0x7f6f2800b410 tx=0x7f6f2800b8d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:29:47.051 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:47.047+0000 7f6f36ffd640 1 -- 172.21.15.37:0/2553600169 <== mon.0 v2:172.21.15.37:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6f28015020 con 0x7f6f400d5160 2024-04-25T10:29:47.051 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:47.047+0000 7f6f36ffd640 1 -- 172.21.15.37:0/2553600169 <== mon.0 v2:172.21.15.37:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f6f28011830 con 0x7f6f400d5160 2024-04-25T10:29:47.051 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:47.047+0000 7f6f4495e640 1 -- 172.21.15.37:0/2553600169 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f6f40156010 con 0x7f6f400d5160 2024-04-25T10:29:47.051 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:47.047+0000 7f6f36ffd640 1 -- 172.21.15.37:0/2553600169 <== mon.0 v2:172.21.15.37:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6f28011c20 con 0x7f6f400d5160 2024-04-25T10:29:47.052 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:47.047+0000 7f6f4495e640 1 -- 172.21.15.37:0/2553600169 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f6f401565d0 con 0x7f6f400d5160 2024-04-25T10:29:47.052 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:47.048+0000 7f6f4495e640 1 -- 172.21.15.37:0/2553600169 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f6f4006a140 con 0x7f6f400d5160 2024-04-25T10:29:47.053 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:47.049+0000 7f6f36ffd640 1 -- 172.21.15.37:0/2553600169 <== mon.0 v2:172.21.15.37:3300/0 4 ==== mgrmap(e 5) ==== 96782+0+0 (secure 0 0 0) 0x7f6f28018020 con 0x7f6f400d5160 2024-04-25T10:29:47.053 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:47.049+0000 7f6f36ffd640 1 --2- 172.21.15.37:0/2553600169 >> [v2:172.21.15.203:6832/2418205237,v1:172.21.15.203:6833/2418205237] conn(0x7f6f200744e0 0x7f6f200769f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:29:47.053 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:47.050+0000 7f6f36ffd640 1 -- 172.21.15.37:0/2553600169 <== mon.0 v2:172.21.15.37:3300/0 5 ==== osd_map(23..23 src has 1..23) ==== 5709+0+0 (secure 0 0 0) 0x7f6f28094c90 con 0x7f6f400d5160 2024-04-25T10:29:47.053 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:47.050+0000 7f6f3e59c640 1 --2- 172.21.15.37:0/2553600169 >> [v2:172.21.15.203:6832/2418205237,v1:172.21.15.203:6833/2418205237] conn(0x7f6f200744e0 0x7f6f200769f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:29:47.053 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:47.050+0000 7f6f3e59c640 1 --2- 172.21.15.37:0/2553600169 >> [v2:172.21.15.203:6832/2418205237,v1:172.21.15.203:6833/2418205237] conn(0x7f6f200744e0 0x7f6f200769f0 secure :-1 s=READY pgs=148 cs=0 l=1 rev1=1 crypto rx=0x7f6f38034000 tx=0x7f6f3802e000 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:29:47.056 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:47.054+0000 7f6f36ffd640 1 -- 172.21.15.37:0/2553600169 <== mon.0 v2:172.21.15.37:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187246 (secure 0 0 0) 0x7f6f2801f070 con 0x7f6f400d5160 2024-04-25T10:29:47.252 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:47.250+0000 7f6f4495e640 1 -- 172.21.15.37:0/2553600169 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_command({"prefix": "osd blocklist ls"} v 0) -- 0x7f6f400d3f50 con 0x7f6f400d5160 2024-04-25T10:29:47.253 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:47.250+0000 7f6f36ffd640 1 -- 172.21.15.37:0/2553600169 <== mon.0 v2:172.21.15.37:3300/0 7 ==== mon_command_ack([{"prefix": "osd blocklist ls"}]=0 listed 0 entries v23) ==== 80+0+0 (secure 0 0 0) 0x7f6f40158fb0 con 0x7f6f400d5160 2024-04-25T10:29:47.253 INFO:teuthology.orchestra.run.smithi037.stderr:listed 0 entries 2024-04-25T10:29:47.255 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:47.252+0000 7f6f4495e640 1 -- 172.21.15.37:0/2553600169 >> [v2:172.21.15.203:6832/2418205237,v1:172.21.15.203:6833/2418205237] conn(0x7f6f200744e0 msgr2=0x7f6f200769f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:29:47.255 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:47.252+0000 7f6f4495e640 1 --2- 172.21.15.37:0/2553600169 >> [v2:172.21.15.203:6832/2418205237,v1:172.21.15.203:6833/2418205237] conn(0x7f6f200744e0 0x7f6f200769f0 secure :-1 s=READY pgs=148 cs=0 l=1 rev1=1 crypto rx=0x7f6f38034000 tx=0x7f6f3802e000 comp rx=0 tx=0).stop 2024-04-25T10:29:47.255 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:47.252+0000 7f6f4495e640 1 -- 172.21.15.37:0/2553600169 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f6f400d5160 msgr2=0x7f6f40159ba0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:29:47.255 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:47.252+0000 7f6f4495e640 1 --2- 172.21.15.37:0/2553600169 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f6f400d5160 0x7f6f40159ba0 secure :-1 s=READY pgs=275 cs=0 l=1 rev1=1 crypto rx=0x7f6f2800b410 tx=0x7f6f2800b8d0 comp rx=0 tx=0).stop 2024-04-25T10:29:47.255 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:47.252+0000 7f6f4495e640 1 -- 172.21.15.37:0/2553600169 shutdown_connections 2024-04-25T10:29:47.255 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:47.252+0000 7f6f4495e640 1 --2- 172.21.15.37:0/2553600169 >> [v2:172.21.15.203:6832/2418205237,v1:172.21.15.203:6833/2418205237] conn(0x7f6f200744e0 0x7f6f200769f0 unknown :-1 s=CLOSED pgs=148 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:29:47.255 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:47.252+0000 7f6f4495e640 1 --2- 172.21.15.37:0/2553600169 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f6f401f73d0 0x7f6f4015b280 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:29:47.255 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:47.252+0000 7f6f4495e640 1 --2- 172.21.15.37:0/2553600169 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f6f400d5160 0x7f6f40159ba0 unknown :-1 s=CLOSED pgs=275 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:29:47.255 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:47.252+0000 7f6f4495e640 1 --2- 172.21.15.37:0/2553600169 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f6f400d0990 0x7f6f40154820 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:29:47.256 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:47.252+0000 7f6f4495e640 1 -- 172.21.15.37:0/2553600169 >> 172.21.15.37:0/2553600169 conn(0x7f6f400b7a80 msgr2=0x7f6f400b7e90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:29:47.256 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:47.252+0000 7f6f4495e640 1 -- 172.21.15.37:0/2553600169 shutdown_connections 2024-04-25T10:29:47.256 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:47.253+0000 7f6f4495e640 1 -- 172.21.15.37:0/2553600169 wait complete. 2024-04-25T10:29:47.266 INFO:tasks.cephfs.fuse_mount:Running fusermount -u on ubuntu@smithi037.front.sepia.ceph.com... 2024-04-25T10:29:47.266 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T10:29:47.266 DEBUG:teuthology.orchestra.run.smithi037:> sudo fusermount -u /home/ubuntu/cephtest/mnt.0 2024-04-25T10:29:47.291 INFO:teuthology.orchestra.run:waiting for 300 2024-04-25T10:29:47.291 INFO:tasks.cephfs.mount:Cleaning up mount ubuntu@smithi037.front.sepia.ceph.com 2024-04-25T10:29:47.291 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T10:29:47.291 DEBUG:teuthology.orchestra.run.smithi037:> (cd /home/ubuntu/cephtest && exec rmdir -- /home/ubuntu/cephtest/mnt.0) 2024-04-25T10:29:47.354 DEBUG:teuthology.orchestra.run.smithi203:> set -ex 2024-04-25T10:29:47.354 DEBUG:teuthology.orchestra.run.smithi203:> dd if=/proc/self/mounts of=/dev/stdout 2024-04-25T10:29:47.373 DEBUG:teuthology.orchestra.run.smithi203:> set -ex 2024-04-25T10:29:47.374 DEBUG:teuthology.orchestra.run.smithi203:> dd if=/proc/self/mounts of=/dev/stdout 2024-04-25T10:29:47.432 DEBUG:teuthology.orchestra.run.smithi037:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-04-25T10:29:47.545 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:47.542+0000 7fae8220c640 1 Processor -- start 2024-04-25T10:29:47.545 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:47.543+0000 7fae8220c640 1 -- start start 2024-04-25T10:29:47.558 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:47.544+0000 7fae8220c640 1 --2- >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7fae7c1460e0 0x7fae7c1610b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:29:47.558 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:47.544+0000 7fae8220c640 1 -- --> v1:172.21.15.37:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fae7c1616a0 con 0x7fae7c143500 2024-04-25T10:29:47.559 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:47.544+0000 7fae8220c640 1 -- --> v1:172.21.15.203:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fae7c0b5b60 con 0x7fae7c06f100 2024-04-25T10:29:47.559 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:47.544+0000 7fae8220c640 1 -- --> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] -- mon_getmap magic: 0 -- 0x7fae7c06f510 con 0x7fae7c1460e0 2024-04-25T10:29:47.559 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:47.544+0000 7fae7b7fe640 1 --1- >> v1:172.21.15.37:6789/0 conn(0x7fae7c143500 0x7fae7c1609a0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.37:6789/0 says I am v1:172.21.15.37:49910/0 (socket says 172.21.15.37:49910) 2024-04-25T10:29:47.559 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:47.544+0000 7fae7b7fe640 1 -- 172.21.15.37:0/1305990954 learned_addr learned my addr 172.21.15.37:0/1305990954 (peer_addr_for_me v1:172.21.15.37:0/0) 2024-04-25T10:29:47.559 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:47.544+0000 7fae7a7fc640 1 -- 172.21.15.37:0/1305990954 <== mon.0 v1:172.21.15.37:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 901496774 0 0) 0x7fae7c1616a0 con 0x7fae7c143500 2024-04-25T10:29:47.559 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:47.544+0000 7fae7a7fc640 1 -- 172.21.15.37:0/1305990954 --> v1:172.21.15.37:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7fae64003520 con 0x7fae7c143500 2024-04-25T10:29:47.560 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:47.544+0000 7fae7affd640 1 --2- 172.21.15.37:0/1305990954 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7fae7c1460e0 0x7fae7c1610b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:29:47.560 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:47.545+0000 7fae7a7fc640 1 -- 172.21.15.37:0/1305990954 <== mon.0 v1:172.21.15.37:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 2160025969 0 0) 0x7fae64003520 con 0x7fae7c143500 2024-04-25T10:29:47.560 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:47.545+0000 7fae7a7fc640 1 -- 172.21.15.37:0/1305990954 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7fae7c1460e0 msgr2=0x7fae7c1610b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:29:47.560 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:47.545+0000 7fae7a7fc640 1 --2- 172.21.15.37:0/1305990954 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7fae7c1460e0 0x7fae7c1610b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:29:47.560 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:47.545+0000 7fae7a7fc640 1 -- 172.21.15.37:0/1305990954 >> v1:172.21.15.203:6789/0 conn(0x7fae7c06f100 legacy=0x7fae7c143b30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:29:47.561 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:47.545+0000 7fae7a7fc640 1 -- 172.21.15.37:0/1305990954 --> v1:172.21.15.37:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fae7c0b5e40 con 0x7fae7c143500 2024-04-25T10:29:47.561 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:47.545+0000 7fae7a7fc640 1 -- 172.21.15.37:0/1305990954 <== mon.0 v1:172.21.15.37:6789/0 3 ==== mon_map magic: 0 ==== 413+0+0 (unknown 1577794600 0 0) 0x7fae6c001670 con 0x7fae7c143500 2024-04-25T10:29:47.561 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:47.545+0000 7fae7affd640 1 --2- 172.21.15.37:0/1305990954 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7fae7c1460e0 0x7fae7c1610b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T10:29:47.561 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:47.545+0000 7fae7a7fc640 1 -- 172.21.15.37:0/1305990954 >> v1:172.21.15.37:6789/0 conn(0x7fae7c143500 legacy=0x7fae7c1609a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:29:47.561 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:47.545+0000 7fae7a7fc640 1 --2- 172.21.15.37:0/1305990954 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7fae7c1460e0 0x7fae640039e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:29:47.561 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:47.545+0000 7fae7a7fc640 1 --2- 172.21.15.37:0/1305990954 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fae64003ff0 0x7fae64004800 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:29:47.561 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:47.545+0000 7fae7a7fc640 1 --2- 172.21.15.37:0/1305990954 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fae64004d40 0x7fae64005210 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:29:47.562 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:47.545+0000 7fae7a7fc640 1 -- 172.21.15.37:0/1305990954 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 -- 0x7fae7c06f510 con 0x7fae64003ff0 2024-04-25T10:29:47.562 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:47.545+0000 7fae7a7fc640 1 -- 172.21.15.37:0/1305990954 --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_getmap magic: 0 -- 0x7fae6400bbd0 con 0x7fae64004d40 2024-04-25T10:29:47.562 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:47.545+0000 7fae7a7fc640 1 -- 172.21.15.37:0/1305990954 --> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] -- mon_getmap magic: 0 -- 0x7fae6400dfe0 con 0x7fae7c1460e0 2024-04-25T10:29:47.562 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:47.545+0000 7fae7affd640 1 --2- 172.21.15.37:0/1305990954 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fae64003ff0 0x7fae64004800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:29:47.562 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:47.545+0000 7fae7b7fe640 1 --2- 172.21.15.37:0/1305990954 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7fae7c1460e0 0x7fae640039e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:29:47.563 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:47.545+0000 7fae7bfff640 1 --2- 172.21.15.37:0/1305990954 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fae64004d40 0x7fae64005210 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:29:47.563 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:47.545+0000 7fae7affd640 1 -- 172.21.15.37:0/1305990954 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7fae7c1460e0 msgr2=0x7fae640039e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:29:47.563 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:47.545+0000 7fae7affd640 1 --2- 172.21.15.37:0/1305990954 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7fae7c1460e0 0x7fae640039e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:29:47.563 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:47.545+0000 7fae7affd640 1 -- 172.21.15.37:0/1305990954 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fae64004d40 msgr2=0x7fae64005210 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:29:47.563 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:47.545+0000 7fae7affd640 1 --2- 172.21.15.37:0/1305990954 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fae64004d40 0x7fae64005210 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:29:47.563 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:47.545+0000 7fae7affd640 1 -- 172.21.15.37:0/1305990954 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fae64003520 con 0x7fae64003ff0 2024-04-25T10:29:47.564 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:47.546+0000 7fae7affd640 1 --2- 172.21.15.37:0/1305990954 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fae64003ff0 0x7fae64004800 secure :-1 s=READY pgs=277 cs=0 l=1 rev1=1 crypto rx=0x7fae700098a0 tx=0x7fae64005830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=9212377e0babe5cc server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:29:47.564 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:47.546+0000 7fae7a7fc640 1 -- 172.21.15.37:0/1305990954 <== mon.0 v2:172.21.15.37:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fae7000f040 con 0x7fae64003ff0 2024-04-25T10:29:47.564 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:47.546+0000 7fae7a7fc640 1 -- 172.21.15.37:0/1305990954 <== mon.0 v2:172.21.15.37:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fae7002ad00 con 0x7fae64003ff0 2024-04-25T10:29:47.564 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:47.546+0000 7fae7a7fc640 1 -- 172.21.15.37:0/1305990954 <== mon.0 v2:172.21.15.37:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fae70004250 con 0x7fae64003ff0 2024-04-25T10:29:47.564 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:47.547+0000 7fae8220c640 1 -- 172.21.15.37:0/1305990954 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fae64003ff0 msgr2=0x7fae64004800 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:29:47.565 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:47.547+0000 7fae8220c640 1 --2- 172.21.15.37:0/1305990954 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fae64003ff0 0x7fae64004800 secure :-1 s=READY pgs=277 cs=0 l=1 rev1=1 crypto rx=0x7fae700098a0 tx=0x7fae64005830 comp rx=0 tx=0).stop 2024-04-25T10:29:47.565 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:47.547+0000 7fae7bfff640 1 -- 172.21.15.37:0/1305990954 reap_dead start 2024-04-25T10:29:47.565 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:47.547+0000 7fae8220c640 1 -- 172.21.15.37:0/1305990954 shutdown_connections 2024-04-25T10:29:47.565 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:47.547+0000 7fae8220c640 1 -- 172.21.15.37:0/1305990954 >> 172.21.15.37:0/1305990954 conn(0x7fae7c0b36b0 msgr2=0x7fae7c0b3ac0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:29:47.565 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:47.547+0000 7fae8220c640 1 -- 172.21.15.37:0/1305990954 shutdown_connections 2024-04-25T10:29:47.565 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:47.547+0000 7fae8220c640 1 -- 172.21.15.37:0/1305990954 wait complete. 2024-04-25T10:29:47.565 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:47.549+0000 7fae8220c640 1 Processor -- start 2024-04-25T10:29:47.565 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:47.549+0000 7fae8220c640 1 -- start start 2024-04-25T10:29:47.565 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:47.550+0000 7fae8220c640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fae64003ff0 0x7fae7c15fb60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:29:47.566 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:47.550+0000 7fae8220c640 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fae7c13fdf0 0x7fae7c13f5d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:29:47.566 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:47.550+0000 7fae8220c640 1 --2- >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7fae7c1f1840 0x7fae7c1f1c50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:29:47.566 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:47.550+0000 7fae8220c640 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 -- 0x7fae7c161230 con 0x7fae64003ff0 2024-04-25T10:29:47.566 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:47.550+0000 7fae8220c640 1 -- --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_getmap magic: 0 -- 0x7fae7c1610b0 con 0x7fae7c13fdf0 2024-04-25T10:29:47.566 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:47.550+0000 7fae8220c640 1 -- --> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] -- mon_getmap magic: 0 -- 0x7fae7c1613b0 con 0x7fae7c1f1840 2024-04-25T10:29:47.566 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:47.550+0000 7fae7b7fe640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fae64003ff0 0x7fae7c15fb60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:29:47.566 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:47.550+0000 7fae7b7fe640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fae64003ff0 0x7fae7c15fb60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.37:3300/0 says I am v2:172.21.15.37:59660/0 (socket says 172.21.15.37:59660) 2024-04-25T10:29:47.567 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:47.550+0000 7fae7b7fe640 1 -- 172.21.15.37:0/2835088589 learned_addr learned my addr 172.21.15.37:0/2835088589 (peer_addr_for_me v2:172.21.15.37:0/0) 2024-04-25T10:29:47.567 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:47.550+0000 7fae7affd640 1 --2- 172.21.15.37:0/2835088589 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fae7c13fdf0 0x7fae7c13f5d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:29:47.567 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:47.550+0000 7fae7bfff640 1 --2- 172.21.15.37:0/2835088589 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7fae7c1f1840 0x7fae7c1f1c50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:29:47.567 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:47.551+0000 7fae7b7fe640 1 -- 172.21.15.37:0/2835088589 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7fae7c1f1840 msgr2=0x7fae7c1f1c50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:29:47.567 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:47.551+0000 7fae7b7fe640 1 --2- 172.21.15.37:0/2835088589 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7fae7c1f1840 0x7fae7c1f1c50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:29:47.567 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:47.551+0000 7fae7b7fe640 1 -- 172.21.15.37:0/2835088589 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fae7c13fdf0 msgr2=0x7fae7c13f5d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:29:47.567 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:47.551+0000 7fae7b7fe640 1 --2- 172.21.15.37:0/2835088589 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fae7c13fdf0 0x7fae7c13f5d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:29:47.567 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:47.551+0000 7fae7b7fe640 1 -- 172.21.15.37:0/2835088589 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fae7c15eac0 con 0x7fae64003ff0 2024-04-25T10:29:47.568 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:47.551+0000 7fae7bfff640 1 --2- 172.21.15.37:0/2835088589 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7fae7c1f1840 0x7fae7c1f1c50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T10:29:47.568 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:47.551+0000 7fae7b7fe640 1 --2- 172.21.15.37:0/2835088589 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fae64003ff0 0x7fae7c15fb60 secure :-1 s=READY pgs=278 cs=0 l=1 rev1=1 crypto rx=0x7fae6c004fa0 tx=0x7fae6c0074f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:29:47.568 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:47.551+0000 7fae63fff640 1 -- 172.21.15.37:0/2835088589 <== mon.0 v2:172.21.15.37:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fae6c01a070 con 0x7fae64003ff0 2024-04-25T10:29:47.568 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:47.551+0000 7fae63fff640 1 -- 172.21.15.37:0/2835088589 <== mon.0 v2:172.21.15.37:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fae6c007740 con 0x7fae64003ff0 2024-04-25T10:29:47.568 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:47.551+0000 7fae8220c640 1 -- 172.21.15.37:0/2835088589 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fae7c15eda0 con 0x7fae64003ff0 2024-04-25T10:29:47.568 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:47.551+0000 7fae63fff640 1 -- 172.21.15.37:0/2835088589 <== mon.0 v2:172.21.15.37:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fae6c00fbd0 con 0x7fae64003ff0 2024-04-25T10:29:47.568 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:47.551+0000 7fae8220c640 1 -- 172.21.15.37:0/2835088589 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fae7c15f360 con 0x7fae64003ff0 2024-04-25T10:29:47.568 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:47.552+0000 7fae8220c640 1 -- 172.21.15.37:0/2835088589 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fae7c06a140 con 0x7fae64003ff0 2024-04-25T10:29:47.569 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:47.553+0000 7fae63fff640 1 -- 172.21.15.37:0/2835088589 <== mon.0 v2:172.21.15.37:3300/0 4 ==== mgrmap(e 5) ==== 96782+0+0 (secure 0 0 0) 0x7fae6c01d360 con 0x7fae64003ff0 2024-04-25T10:29:47.569 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:47.553+0000 7fae63fff640 1 --2- 172.21.15.37:0/2835088589 >> [v2:172.21.15.203:6832/2418205237,v1:172.21.15.203:6833/2418205237] conn(0x7fae680744e0 0x7fae680769f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:29:47.569 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:47.553+0000 7fae63fff640 1 -- 172.21.15.37:0/2835088589 <== mon.0 v2:172.21.15.37:3300/0 5 ==== osd_map(23..23 src has 1..23) ==== 5709+0+0 (secure 0 0 0) 0x7fae6c094ae0 con 0x7fae64003ff0 2024-04-25T10:29:47.569 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:47.553+0000 7fae7affd640 1 --2- 172.21.15.37:0/2835088589 >> [v2:172.21.15.203:6832/2418205237,v1:172.21.15.203:6833/2418205237] conn(0x7fae680744e0 0x7fae680769f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:29:47.569 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:47.554+0000 7fae7affd640 1 --2- 172.21.15.37:0/2835088589 >> [v2:172.21.15.203:6832/2418205237,v1:172.21.15.203:6833/2418205237] conn(0x7fae680744e0 0x7fae680769f0 secure :-1 s=READY pgs=149 cs=0 l=1 rev1=1 crypto rx=0x7fae70033000 tx=0x7fae7002e000 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:29:47.570 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:47.558+0000 7fae63fff640 1 -- 172.21.15.37:0/2835088589 <== mon.0 v2:172.21.15.37:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187246 (secure 0 0 0) 0x7fae6c05f280 con 0x7fae64003ff0 2024-04-25T10:29:47.745 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:47.743+0000 7fae8220c640 1 -- 172.21.15.37:0/2835088589 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7fae7c0b12f0 con 0x7fae64003ff0 2024-04-25T10:29:47.746 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:47.744+0000 7fae63fff640 1 -- 172.21.15.37:0/2835088589 <== mon.0 v2:172.21.15.37:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v23) ==== 74+0+16621 (secure 0 0 0) 0x7fae6c067270 con 0x7fae64003ff0 2024-04-25T10:29:47.746 INFO:teuthology.orchestra.run.smithi037.stdout: 2024-04-25T10:29:47.746 INFO:teuthology.orchestra.run.smithi037.stdout:{"epoch":23,"fsid":"f4acda28-be34-492a-b286-994afd36f61d","created":"2024-04-25T10:28:43.985718+0000","modified":"2024-04-25T10:29:10.016188+0000","last_up_change":"2024-04-25T10:28:54.900654+0000","last_in_change":"2024-04-25T10:28:46.216973+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":3,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-04-25T10:28:56.202153+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"19","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2024-04-25T10:29:06.507244+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"22","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":2.25,"score_stable":2.25,"optimal_score":1,"raw_score_acting":2.25,"raw_score_stable":2.25,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2024-04-25T10:29:07.347844+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"23","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"a2f79028-b332-4166-9ebb-a7024a37ea22","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":21,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.37:6816","nonce":2746074720},{"type":"v1","addr":"172.21.15.37:6817","nonce":2746074720}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.37:6818","nonce":2746074720},{"type":"v1","addr":"172.21.15.37:6819","nonce":2746074720}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.37:6822","nonce":2746074720},{"type":"v1","addr":"172.21.15.37:6823","nonce":2746074720}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.37:6820","nonce":2746074720},{"type":"v1","addr":"172.21.15.37:6821","nonce":2746074720}]},"public_addr":"172.21.15.37:6817/2746074720","cluster_addr":"172.21.15.37:6819/2746074720","heartbeat_back_addr":"172.21.15.37:6823/2746074720","heartbeat_front_addr":"172.21.15.37:6821/2746074720","state":["exists","up"]},{"osd":1,"uuid":"973159de-c2ea-4083-aecc-69373d1c1132","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":21,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.37:6800","nonce":1690341662},{"type":"v1","addr":"172.21.15.37:6801","nonce":1690341662}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.37:6802","nonce":1690341662},{"type":"v1","addr":"172.21.15.37:6803","nonce":1690341662}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.37:6806","nonce":1690341662},{"type":"v1","addr":"172.21.15.37:6807","nonce":1690341662}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.37:6804","nonce":1690341662},{"type":"v1","addr":"172.21.15.37:6805","nonce":1690341662}]},"public_addr":"172.21.15.37:6801/1690341662","cluster_addr":"172.21.15.37:6803/1690341662","heartbeat_back_addr":"172.21.15.37:6807/1690341662","heartbeat_front_addr":"172.21.15.37:6805/1690341662","state":["exists","up"]},{"osd":2,"uuid":"4ae7a0e9-cb3f-484c-a75f-f3766b721c47","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":21,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.37:6808","nonce":2754669649},{"type":"v1","addr":"172.21.15.37:6809","nonce":2754669649}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.37:6810","nonce":2754669649},{"type":"v1","addr":"172.21.15.37:6811","nonce":2754669649}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.37:6814","nonce":2754669649},{"type":"v1","addr":"172.21.15.37:6815","nonce":2754669649}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.37:6812","nonce":2754669649},{"type":"v1","addr":"172.21.15.37:6813","nonce":2754669649}]},"public_addr":"172.21.15.37:6809/2754669649","cluster_addr":"172.21.15.37:6811/2754669649","heartbeat_back_addr":"172.21.15.37:6815/2754669649","heartbeat_front_addr":"172.21.15.37:6813/2754669649","state":["exists","up"]},{"osd":3,"uuid":"1ba4a746-1608-47a7-ace2-960a5cfae934","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":21,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.37:6824","nonce":1815290618},{"type":"v1","addr":"172.21.15.37:6825","nonce":1815290618}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.37:6826","nonce":1815290618},{"type":"v1","addr":"172.21.15.37:6827","nonce":1815290618}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.37:6830","nonce":1815290618},{"type":"v1","addr":"172.21.15.37:6831","nonce":1815290618}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.37:6828","nonce":1815290618},{"type":"v1","addr":"172.21.15.37:6829","nonce":1815290618}]},"public_addr":"172.21.15.37:6825/1815290618","cluster_addr":"172.21.15.37:6827/1815290618","heartbeat_back_addr":"172.21.15.37:6831/1815290618","heartbeat_front_addr":"172.21.15.37:6829/1815290618","state":["exists","up"]},{"osd":4,"uuid":"d4f59f5d-0413-4b79-87ee-bb212c1fcf62","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":21,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6800","nonce":4051532063},{"type":"v1","addr":"172.21.15.203:6801","nonce":4051532063}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6802","nonce":4051532063},{"type":"v1","addr":"172.21.15.203:6803","nonce":4051532063}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6806","nonce":4051532063},{"type":"v1","addr":"172.21.15.203:6807","nonce":4051532063}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6804","nonce":4051532063},{"type":"v1","addr":"172.21.15.203:6805","nonce":4051532063}]},"public_addr":"172.21.15.203:6801/4051532063","cluster_addr":"172.21.15.203:6803/4051532063","heartbeat_back_addr":"172.21.15.203:6807/4051532063","heartbeat_front_addr":"172.21.15.203:6805/4051532063","state":["exists","up"]},{"osd":5,"uuid":"69710a0e-41d8-4e7f-9a10-17ce540b67be","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":21,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6824","nonce":1203822890},{"type":"v1","addr":"172.21.15.203:6825","nonce":1203822890}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6826","nonce":1203822890},{"type":"v1","addr":"172.21.15.203:6827","nonce":1203822890}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6830","nonce":1203822890},{"type":"v1","addr":"172.21.15.203:6831","nonce":1203822890}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6828","nonce":1203822890},{"type":"v1","addr":"172.21.15.203:6829","nonce":1203822890}]},"public_addr":"172.21.15.203:6825/1203822890","cluster_addr":"172.21.15.203:6827/1203822890","heartbeat_back_addr":"172.21.15.203:6831/1203822890","heartbeat_front_addr":"172.21.15.203:6829/1203822890","state":["exists","up"]},{"osd":6,"uuid":"f29dd047-3b48-473d-a235-6642a6293f20","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":21,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6810","nonce":2309615014},{"type":"v1","addr":"172.21.15.203:6812","nonce":2309615014}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6814","nonce":2309615014},{"type":"v1","addr":"172.21.15.203:6816","nonce":2309615014}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6822","nonce":2309615014},{"type":"v1","addr":"172.21.15.203:6823","nonce":2309615014}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6818","nonce":2309615014},{"type":"v1","addr":"172.21.15.203:6820","nonce":2309615014}]},"public_addr":"172.21.15.203:6812/2309615014","cluster_addr":"172.21.15.203:6816/2309615014","heartbeat_back_addr":"172.21.15.203:6823/2309615014","heartbeat_front_addr":"172.21.15.203:6820/2309615014","state":["exists","up"]},{"osd":7,"uuid":"6e691644-378b-4fdc-8e86-fb5f56e31f29","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":21,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6808","nonce":3688099014},{"type":"v1","addr":"172.21.15.203:6809","nonce":3688099014}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6811","nonce":3688099014},{"type":"v1","addr":"172.21.15.203:6813","nonce":3688099014}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6819","nonce":3688099014},{"type":"v1","addr":"172.21.15.203:6821","nonce":3688099014}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6815","nonce":3688099014},{"type":"v1","addr":"172.21.15.203:6817","nonce":3688099014}]},"public_addr":"172.21.15.203:6809/3688099014","cluster_addr":"172.21.15.203:6813/3688099014","heartbeat_back_addr":"172.21.15.203:6821/3688099014","heartbeat_front_addr":"172.21.15.203:6817/3688099014","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T10:28:51.419141+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T10:28:51.366448+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T10:28:51.348276+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T10:28:51.380519+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T10:28:51.105812+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T10:28:51.384496+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T10:28:51.388755+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T10:28:51.366958+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-04-25T10:29:47.748 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:47.746+0000 7fae8220c640 1 -- 172.21.15.37:0/2835088589 >> [v2:172.21.15.203:6832/2418205237,v1:172.21.15.203:6833/2418205237] conn(0x7fae680744e0 msgr2=0x7fae680769f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:29:47.748 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:47.746+0000 7fae8220c640 1 --2- 172.21.15.37:0/2835088589 >> [v2:172.21.15.203:6832/2418205237,v1:172.21.15.203:6833/2418205237] conn(0x7fae680744e0 0x7fae680769f0 secure :-1 s=READY pgs=149 cs=0 l=1 rev1=1 crypto rx=0x7fae70033000 tx=0x7fae7002e000 comp rx=0 tx=0).stop 2024-04-25T10:29:47.748 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:47.746+0000 7fae8220c640 1 -- 172.21.15.37:0/2835088589 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fae64003ff0 msgr2=0x7fae7c15fb60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:29:47.749 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:47.746+0000 7fae8220c640 1 --2- 172.21.15.37:0/2835088589 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fae64003ff0 0x7fae7c15fb60 secure :-1 s=READY pgs=278 cs=0 l=1 rev1=1 crypto rx=0x7fae6c004fa0 tx=0x7fae6c0074f0 comp rx=0 tx=0).stop 2024-04-25T10:29:47.749 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:47.746+0000 7fae8220c640 1 -- 172.21.15.37:0/2835088589 shutdown_connections 2024-04-25T10:29:47.749 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:47.746+0000 7fae8220c640 1 --2- 172.21.15.37:0/2835088589 >> [v2:172.21.15.203:6832/2418205237,v1:172.21.15.203:6833/2418205237] conn(0x7fae680744e0 0x7fae680769f0 unknown :-1 s=CLOSED pgs=149 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:29:47.749 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:47.746+0000 7fae8220c640 1 --2- 172.21.15.37:0/2835088589 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7fae7c1f1840 0x7fae7c1f1c50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:29:47.749 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:47.746+0000 7fae8220c640 1 --2- 172.21.15.37:0/2835088589 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fae7c13fdf0 0x7fae7c13f5d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:29:47.749 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:47.746+0000 7fae8220c640 1 --2- 172.21.15.37:0/2835088589 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fae64003ff0 0x7fae7c15fb60 unknown :-1 s=CLOSED pgs=278 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:29:47.749 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:47.746+0000 7fae8220c640 1 -- 172.21.15.37:0/2835088589 >> 172.21.15.37:0/2835088589 conn(0x7fae7c0b36b0 msgr2=0x7fae7c075d00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:29:47.749 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:47.746+0000 7fae8220c640 1 -- 172.21.15.37:0/2835088589 shutdown_connections 2024-04-25T10:29:47.749 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:47.746+0000 7fae8220c640 1 -- 172.21.15.37:0/2835088589 wait complete. 2024-04-25T10:29:47.762 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2024-04-25T10:28:56.202153+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '19', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 8, 'score_stable': 8, 'optimal_score': 0.25, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 2, 'pool_name': 'cephfs_metadata', 'create_time': '2024-04-25T10:29:06.507244+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '22', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_autoscale_bias': 4, 'pg_num_min': 16, 'recovery_priority': 5}, 'application_metadata': {'cephfs': {'metadata': 'cephfs'}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 2.25, 'score_stable': 2.25, 'optimal_score': 1, 'raw_score_acting': 2.25, 'raw_score_stable': 2.25, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 3, 'pool_name': 'cephfs_data', 'create_time': '2024-04-25T10:29:07.347844+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 64, 'pg_placement_num': 64, 'pg_placement_num_target': 64, 'pg_num_target': 64, 'pg_num_pending': 64, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '23', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.8999999761581421}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 1.5, 'score_stable': 1.5, 'optimal_score': 1, 'raw_score_acting': 1.5, 'raw_score_stable': 1.5, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}] 2024-04-25T10:29:47.763 DEBUG:teuthology.orchestra.run.smithi037:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2024-04-25T10:29:47.877 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:47.875+0000 7ff966a34640 1 Processor -- start 2024-04-25T10:29:47.877 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:47.875+0000 7ff966a34640 1 -- start start 2024-04-25T10:29:47.879 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:47.876+0000 7ff966a34640 1 --2- >> v2:172.21.15.37:3300/0 conn(0x7ff96015aa20 0x7ff960161a80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:29:47.879 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:47.876+0000 7ff966a34640 1 -- --> v1:172.21.15.37:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7ff9600d4240 con 0x7ff9600b7750 2024-04-25T10:29:47.879 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:47.876+0000 7ff966a34640 1 -- --> v1:172.21.15.203:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7ff9600d1a00 con 0x7ff9600d0d10 2024-04-25T10:29:47.879 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:47.876+0000 7ff966a34640 1 -- --> v2:172.21.15.37:3300/0 -- mon_getmap magic: 0 -- 0x7ff9600d4b30 con 0x7ff96015aa20 2024-04-25T10:29:47.879 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:47.876+0000 7ff95ffff640 1 --2- >> v2:172.21.15.37:3300/0 conn(0x7ff96015aa20 0x7ff960161a80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:29:47.879 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:47.876+0000 7ff95f7fe640 1 --1- >> v1:172.21.15.37:6789/0 conn(0x7ff9600b7750 0x7ff9600b7b60 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.37:6789/0 says I am v1:172.21.15.37:49918/0 (socket says 172.21.15.37:49918) 2024-04-25T10:29:47.879 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:47.876+0000 7ff95f7fe640 1 -- 172.21.15.37:0/3620588311 learned_addr learned my addr 172.21.15.37:0/3620588311 (peer_addr_for_me v1:172.21.15.37:0/0) 2024-04-25T10:29:47.879 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:47.877+0000 7ff95ffff640 1 -- 172.21.15.37:0/3620588311 >> v1:172.21.15.203:6789/0 conn(0x7ff9600d0d10 legacy=0x7ff9600d1120 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:29:47.879 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:47.877+0000 7ff95ffff640 1 -- 172.21.15.37:0/3620588311 >> v1:172.21.15.37:6789/0 conn(0x7ff9600b7750 legacy=0x7ff9600b7b60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:29:47.879 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:47.877+0000 7ff95ffff640 1 -- 172.21.15.37:0/3620588311 --> v2:172.21.15.37:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ff9600d44e0 con 0x7ff96015aa20 2024-04-25T10:29:47.879 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:47.877+0000 7ff95effd640 1 -- 172.21.15.37:0/3620588311 <== mon.0 v1:172.21.15.37:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 1546729528 0 0) 0x7ff9600d4240 con 0x7ff9600b7750 2024-04-25T10:29:47.880 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:47.877+0000 7ff95ffff640 1 --2- 172.21.15.37:0/3620588311 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7ff96015aa20 0x7ff960161a80 secure :-1 s=READY pgs=280 cs=0 l=1 rev1=1 crypto rx=0x7ff9500097a0 tx=0x7ff95002a230 comp rx=0 tx=0).ready entity=mon.0 client_cookie=fa7a604086a4a735 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:29:47.880 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:47.877+0000 7ff95effd640 1 -- 172.21.15.37:0/3620588311 <== mon.0 v2:172.21.15.37:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff950037070 con 0x7ff96015aa20 2024-04-25T10:29:47.881 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:47.877+0000 7ff95effd640 1 -- 172.21.15.37:0/3620588311 <== mon.0 v2:172.21.15.37:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7ff950032440 con 0x7ff96015aa20 2024-04-25T10:29:47.881 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:47.877+0000 7ff95effd640 1 -- 172.21.15.37:0/3620588311 <== mon.0 v2:172.21.15.37:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff950032810 con 0x7ff96015aa20 2024-04-25T10:29:47.881 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:47.878+0000 7ff966a34640 1 -- 172.21.15.37:0/3620588311 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7ff96015aa20 msgr2=0x7ff960161a80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:29:47.881 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:47.878+0000 7ff966a34640 1 --2- 172.21.15.37:0/3620588311 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7ff96015aa20 0x7ff960161a80 secure :-1 s=READY pgs=280 cs=0 l=1 rev1=1 crypto rx=0x7ff9500097a0 tx=0x7ff95002a230 comp rx=0 tx=0).stop 2024-04-25T10:29:47.881 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:47.878+0000 7ff966a34640 1 -- 172.21.15.37:0/3620588311 shutdown_connections 2024-04-25T10:29:47.881 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:47.878+0000 7ff966a34640 1 --2- 172.21.15.37:0/3620588311 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7ff96015aa20 0x7ff960161a80 unknown :-1 s=CLOSED pgs=280 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:29:47.881 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:47.878+0000 7ff966a34640 1 -- 172.21.15.37:0/3620588311 >> 172.21.15.37:0/3620588311 conn(0x7ff9600d2750 msgr2=0x7ff9600d2b60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:29:47.881 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:47.878+0000 7ff966a34640 1 -- 172.21.15.37:0/3620588311 shutdown_connections 2024-04-25T10:29:47.881 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:47.878+0000 7ff966a34640 1 -- 172.21.15.37:0/3620588311 wait complete. 2024-04-25T10:29:47.882 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:47.880+0000 7ff966a34640 1 Processor -- start 2024-04-25T10:29:47.882 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:47.880+0000 7ff966a34640 1 -- start start 2024-04-25T10:29:47.885 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:47.881+0000 7ff966a34640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7ff9600b7750 0x7ff9600cd920 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:29:47.885 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:47.881+0000 7ff966a34640 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7ff9600d0d10 0x7ff9600ca950 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:29:47.885 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:47.882+0000 7ff966a34640 1 --2- >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7ff96015aa20 0x7ff9600cae90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:29:47.885 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:47.882+0000 7ff966a34640 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 -- 0x7ff9600d4fc0 con 0x7ff9600b7750 2024-04-25T10:29:47.885 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:47.882+0000 7ff966a34640 1 -- --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_getmap magic: 0 -- 0x7ff960161fc0 con 0x7ff9600d0d10 2024-04-25T10:29:47.885 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:47.882+0000 7ff966a34640 1 -- --> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] -- mon_getmap magic: 0 -- 0x7ff9600d5140 con 0x7ff96015aa20 2024-04-25T10:29:47.885 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:47.882+0000 7ff95ffff640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7ff9600b7750 0x7ff9600cd920 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:29:47.885 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:47.882+0000 7ff95ffff640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7ff9600b7750 0x7ff9600cd920 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.37:3300/0 says I am v2:172.21.15.37:59678/0 (socket says 172.21.15.37:59678) 2024-04-25T10:29:47.886 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:47.882+0000 7ff95ffff640 1 -- 172.21.15.37:0/4102132 learned_addr learned my addr 172.21.15.37:0/4102132 (peer_addr_for_me v2:172.21.15.37:0/0) 2024-04-25T10:29:47.886 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:47.882+0000 7ff95f7fe640 1 --2- 172.21.15.37:0/4102132 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7ff9600d0d10 0x7ff9600ca950 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:29:47.886 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:47.882+0000 7ff95ffff640 1 -- 172.21.15.37:0/4102132 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7ff96015aa20 msgr2=0x7ff9600cae90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:29:47.886 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:47.882+0000 7ff964fd1640 1 --2- 172.21.15.37:0/4102132 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7ff96015aa20 0x7ff9600cae90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:29:47.886 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:47.882+0000 7ff95ffff640 1 --2- 172.21.15.37:0/4102132 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7ff96015aa20 0x7ff9600cae90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:29:47.886 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:47.882+0000 7ff95ffff640 1 -- 172.21.15.37:0/4102132 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7ff9600d0d10 msgr2=0x7ff9600ca950 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:29:47.886 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:47.882+0000 7ff95ffff640 1 --2- 172.21.15.37:0/4102132 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7ff9600d0d10 0x7ff9600ca950 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:29:47.886 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:47.882+0000 7ff95ffff640 1 -- 172.21.15.37:0/4102132 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ff9600c9bf0 con 0x7ff9600b7750 2024-04-25T10:29:47.886 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:47.882+0000 7ff95f7fe640 1 --2- 172.21.15.37:0/4102132 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7ff9600d0d10 0x7ff9600ca950 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T10:29:47.886 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:47.882+0000 7ff964fd1640 1 --2- 172.21.15.37:0/4102132 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7ff96015aa20 0x7ff9600cae90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T10:29:47.886 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:47.882+0000 7ff95ffff640 1 --2- 172.21.15.37:0/4102132 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7ff9600b7750 0x7ff9600cd920 secure :-1 s=READY pgs=281 cs=0 l=1 rev1=1 crypto rx=0x7ff9500098c0 tx=0x7ff9500046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:29:47.886 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:47.882+0000 7ff95cff9640 1 -- 172.21.15.37:0/4102132 <== mon.0 v2:172.21.15.37:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff950037040 con 0x7ff9600b7750 2024-04-25T10:29:47.886 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:47.882+0000 7ff95cff9640 1 -- 172.21.15.37:0/4102132 <== mon.0 v2:172.21.15.37:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7ff950042070 con 0x7ff9600b7750 2024-04-25T10:29:47.886 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:47.882+0000 7ff966a34640 1 -- 172.21.15.37:0/4102132 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7ff9600ce610 con 0x7ff9600b7750 2024-04-25T10:29:47.887 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:47.882+0000 7ff95cff9640 1 -- 172.21.15.37:0/4102132 <== mon.0 v2:172.21.15.37:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff950030a70 con 0x7ff9600b7750 2024-04-25T10:29:47.887 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:47.883+0000 7ff966a34640 1 -- 172.21.15.37:0/4102132 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7ff9600ca150 con 0x7ff9600b7750 2024-04-25T10:29:47.887 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:47.883+0000 7ff966a34640 1 -- 172.21.15.37:0/4102132 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7ff960161db0 con 0x7ff9600b7750 2024-04-25T10:29:47.888 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:47.884+0000 7ff95cff9640 1 -- 172.21.15.37:0/4102132 <== mon.0 v2:172.21.15.37:3300/0 4 ==== mgrmap(e 5) ==== 96782+0+0 (secure 0 0 0) 0x7ff950030c10 con 0x7ff9600b7750 2024-04-25T10:29:47.888 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:47.885+0000 7ff95cff9640 1 --2- 172.21.15.37:0/4102132 >> [v2:172.21.15.203:6832/2418205237,v1:172.21.15.203:6833/2418205237] conn(0x7ff94c0744b0 0x7ff94c0769c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:29:47.888 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:47.885+0000 7ff95cff9640 1 -- 172.21.15.37:0/4102132 <== mon.0 v2:172.21.15.37:3300/0 5 ==== osd_map(23..23 src has 1..23) ==== 5709+0+0 (secure 0 0 0) 0x7ff9500b5b00 con 0x7ff9600b7750 2024-04-25T10:29:47.888 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:47.885+0000 7ff95f7fe640 1 --2- 172.21.15.37:0/4102132 >> [v2:172.21.15.203:6832/2418205237,v1:172.21.15.203:6833/2418205237] conn(0x7ff94c0744b0 0x7ff94c0769c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:29:47.888 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:47.886+0000 7ff95f7fe640 1 --2- 172.21.15.37:0/4102132 >> [v2:172.21.15.203:6832/2418205237,v1:172.21.15.203:6833/2418205237] conn(0x7ff94c0744b0 0x7ff94c0769c0 secure :-1 s=READY pgs=150 cs=0 l=1 rev1=1 crypto rx=0x7ff96015aee0 tx=0x7ff954009040 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:29:47.891 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:47.889+0000 7ff95cff9640 1 -- 172.21.15.37:0/4102132 <== mon.0 v2:172.21.15.37:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187246 (secure 0 0 0) 0x7ff960161db0 con 0x7ff9600b7750 2024-04-25T10:29:48.078 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:48.076+0000 7ff966a34640 1 -- 172.21.15.37:0/4102132 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) -- 0x7ff96006a140 con 0x7ff9600b7750 2024-04-25T10:29:48.078 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:48.076+0000 7ff95cff9640 1 -- 172.21.15.37:0/4102132 <== mon.0 v2:172.21.15.37:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v23) ==== 93+0+10 (secure 0 0 0) 0x7ff95004c020 con 0x7ff9600b7750 2024-04-25T10:29:48.078 INFO:teuthology.orchestra.run.smithi037.stdout:pg_num: 1 2024-04-25T10:29:48.080 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:48.078+0000 7ff966a34640 1 -- 172.21.15.37:0/4102132 >> [v2:172.21.15.203:6832/2418205237,v1:172.21.15.203:6833/2418205237] conn(0x7ff94c0744b0 msgr2=0x7ff94c0769c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:29:48.080 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:48.078+0000 7ff966a34640 1 --2- 172.21.15.37:0/4102132 >> [v2:172.21.15.203:6832/2418205237,v1:172.21.15.203:6833/2418205237] conn(0x7ff94c0744b0 0x7ff94c0769c0 secure :-1 s=READY pgs=150 cs=0 l=1 rev1=1 crypto rx=0x7ff96015aee0 tx=0x7ff954009040 comp rx=0 tx=0).stop 2024-04-25T10:29:48.080 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:48.078+0000 7ff966a34640 1 -- 172.21.15.37:0/4102132 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7ff9600b7750 msgr2=0x7ff9600cd920 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:29:48.080 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:48.078+0000 7ff966a34640 1 --2- 172.21.15.37:0/4102132 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7ff9600b7750 0x7ff9600cd920 secure :-1 s=READY pgs=281 cs=0 l=1 rev1=1 crypto rx=0x7ff9500098c0 tx=0x7ff9500046c0 comp rx=0 tx=0).stop 2024-04-25T10:29:48.080 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:48.078+0000 7ff966a34640 1 -- 172.21.15.37:0/4102132 shutdown_connections 2024-04-25T10:29:48.081 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:48.078+0000 7ff966a34640 1 --2- 172.21.15.37:0/4102132 >> [v2:172.21.15.203:6832/2418205237,v1:172.21.15.203:6833/2418205237] conn(0x7ff94c0744b0 0x7ff94c0769c0 unknown :-1 s=CLOSED pgs=150 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:29:48.081 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:48.078+0000 7ff966a34640 1 --2- 172.21.15.37:0/4102132 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7ff96015aa20 0x7ff9600cae90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:29:48.081 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:48.078+0000 7ff966a34640 1 --2- 172.21.15.37:0/4102132 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7ff9600d0d10 0x7ff9600ca950 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:29:48.081 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:48.078+0000 7ff966a34640 1 --2- 172.21.15.37:0/4102132 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7ff9600b7750 0x7ff9600cd920 unknown :-1 s=CLOSED pgs=281 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:29:48.081 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:48.078+0000 7ff966a34640 1 -- 172.21.15.37:0/4102132 >> 172.21.15.37:0/4102132 conn(0x7ff9600d2750 msgr2=0x7ff960163120 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:29:48.081 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:48.078+0000 7ff966a34640 1 -- 172.21.15.37:0/4102132 shutdown_connections 2024-04-25T10:29:48.081 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:48.078+0000 7ff966a34640 1 -- 172.21.15.37:0/4102132 wait complete. 2024-04-25T10:29:48.091 DEBUG:teuthology.orchestra.run.smithi037:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_metadata pg_num 2024-04-25T10:29:48.206 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:48.204+0000 7f7129c51640 1 Processor -- start 2024-04-25T10:29:48.206 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:48.204+0000 7f7129c51640 1 -- start start 2024-04-25T10:29:48.208 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:48.205+0000 7f7129c51640 1 --2- >> v2:172.21.15.37:3300/0 conn(0x7f712406c8f0 0x7f712409b700 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:29:48.208 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:48.205+0000 7f7129c51640 1 -- --> v1:172.21.15.37:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f71240b4240 con 0x7f712406e390 2024-04-25T10:29:48.208 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:48.205+0000 7f7129c51640 1 -- --> v1:172.21.15.203:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f712406eb00 con 0x7f712406cd00 2024-04-25T10:29:48.208 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:48.205+0000 7f7129c51640 1 -- --> v2:172.21.15.37:3300/0 -- mon_getmap magic: 0 -- 0x7f712406ede0 con 0x7f712406c8f0 2024-04-25T10:29:48.208 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:48.205+0000 7f7122ffd640 1 --2- >> v2:172.21.15.37:3300/0 conn(0x7f712406c8f0 0x7f712409b700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:29:48.208 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:48.205+0000 7f71237fe640 1 --1- >> v1:172.21.15.37:6789/0 conn(0x7f712406e390 0x7f712415eec0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.37:6789/0 says I am v1:172.21.15.37:49928/0 (socket says 172.21.15.37:49928) 2024-04-25T10:29:48.208 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:48.206+0000 7f71237fe640 1 -- 172.21.15.37:0/3288714369 learned_addr learned my addr 172.21.15.37:0/3288714369 (peer_addr_for_me v1:172.21.15.37:0/0) 2024-04-25T10:29:48.208 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:48.206+0000 7f7122ffd640 1 -- 172.21.15.37:0/3288714369 >> v1:172.21.15.203:6789/0 conn(0x7f712406cd00 legacy=0x7f71240d2450 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:29:48.209 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:48.206+0000 7f7122ffd640 1 -- 172.21.15.37:0/3288714369 >> v1:172.21.15.37:6789/0 conn(0x7f712406e390 legacy=0x7f712415eec0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:29:48.209 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:48.206+0000 7f71227fc640 1 -- 172.21.15.37:0/3288714369 <== mon.0 v1:172.21.15.37:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 3636455648 0 0) 0x7f71240b4240 con 0x7f712406e390 2024-04-25T10:29:48.209 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:48.206+0000 7f7122ffd640 1 -- 172.21.15.37:0/3288714369 --> v2:172.21.15.37:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f712415e6c0 con 0x7f712406c8f0 2024-04-25T10:29:48.210 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:48.207+0000 7f7122ffd640 1 --2- 172.21.15.37:0/3288714369 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f712406c8f0 0x7f712409b700 secure :-1 s=READY pgs=283 cs=0 l=1 rev1=1 crypto rx=0x7f71180097a0 tx=0x7f711802a230 comp rx=0 tx=0).ready entity=mon.0 client_cookie=dafa30a5d33a7317 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:29:48.210 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:48.207+0000 7f71227fc640 1 -- 172.21.15.37:0/3288714369 <== mon.0 v2:172.21.15.37:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7118037070 con 0x7f712406c8f0 2024-04-25T10:29:48.210 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:48.207+0000 7f71227fc640 1 -- 172.21.15.37:0/3288714369 <== mon.0 v2:172.21.15.37:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f7118032440 con 0x7f712406c8f0 2024-04-25T10:29:48.210 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:48.207+0000 7f71227fc640 1 -- 172.21.15.37:0/3288714369 <== mon.0 v2:172.21.15.37:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7118032810 con 0x7f712406c8f0 2024-04-25T10:29:48.210 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:48.207+0000 7f7129c51640 1 -- 172.21.15.37:0/3288714369 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f712406c8f0 msgr2=0x7f712409b700 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:29:48.210 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:48.207+0000 7f7129c51640 1 --2- 172.21.15.37:0/3288714369 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f712406c8f0 0x7f712409b700 secure :-1 s=READY pgs=283 cs=0 l=1 rev1=1 crypto rx=0x7f71180097a0 tx=0x7f711802a230 comp rx=0 tx=0).stop 2024-04-25T10:29:48.210 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:48.207+0000 7f7129c51640 1 -- 172.21.15.37:0/3288714369 shutdown_connections 2024-04-25T10:29:48.210 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:48.207+0000 7f7129c51640 1 --2- 172.21.15.37:0/3288714369 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f712406c8f0 0x7f712409b700 unknown :-1 s=CLOSED pgs=283 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:29:48.210 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:48.208+0000 7f7129c51640 1 -- 172.21.15.37:0/3288714369 >> 172.21.15.37:0/3288714369 conn(0x7f71240b7600 msgr2=0x7f71240b7c30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:29:48.210 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:48.208+0000 7f7129c51640 1 -- 172.21.15.37:0/3288714369 shutdown_connections 2024-04-25T10:29:48.210 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:48.208+0000 7f7129c51640 1 -- 172.21.15.37:0/3288714369 wait complete. 2024-04-25T10:29:48.211 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:48.209+0000 7f7129c51640 1 Processor -- start 2024-04-25T10:29:48.211 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:48.209+0000 7f7129c51640 1 -- start start 2024-04-25T10:29:48.214 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:48.211+0000 7f7129c51640 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f712406c8f0 0x7f7124159450 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:29:48.215 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:48.211+0000 7f7129c51640 1 --2- >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f712406cd00 0x7f71241f6f70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:29:48.215 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:48.211+0000 7f7129c51640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f712406e390 0x7f712415cef0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:29:48.215 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:48.211+0000 7f7129c51640 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 -- 0x7f712415bd00 con 0x7f712406e390 2024-04-25T10:29:48.215 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:48.211+0000 7f7129c51640 1 -- --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_getmap magic: 0 -- 0x7f712406e7a0 con 0x7f712406c8f0 2024-04-25T10:29:48.215 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:48.211+0000 7f7129c51640 1 -- --> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] -- mon_getmap magic: 0 -- 0x7f712415be80 con 0x7f712406cd00 2024-04-25T10:29:48.215 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:48.211+0000 7f7123fff640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f712406e390 0x7f712415cef0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:29:48.215 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:48.211+0000 7f7123fff640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f712406e390 0x7f712415cef0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.37:3300/0 says I am v2:172.21.15.37:59686/0 (socket says 172.21.15.37:59686) 2024-04-25T10:29:48.215 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:48.211+0000 7f7123fff640 1 -- 172.21.15.37:0/3893885789 learned_addr learned my addr 172.21.15.37:0/3893885789 (peer_addr_for_me v2:172.21.15.37:0/0) 2024-04-25T10:29:48.215 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:48.211+0000 7f71237fe640 1 --2- 172.21.15.37:0/3893885789 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f712406c8f0 0x7f7124159450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:29:48.215 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:48.211+0000 7f7122ffd640 1 --2- 172.21.15.37:0/3893885789 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f712406cd00 0x7f71241f6f70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:29:48.215 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:48.211+0000 7f7123fff640 1 -- 172.21.15.37:0/3893885789 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f712406cd00 msgr2=0x7f71241f6f70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:29:48.215 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:48.211+0000 7f7123fff640 1 --2- 172.21.15.37:0/3893885789 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f712406cd00 0x7f71241f6f70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:29:48.215 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:48.211+0000 7f7123fff640 1 -- 172.21.15.37:0/3893885789 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f712406c8f0 msgr2=0x7f7124159450 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:29:48.215 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:48.211+0000 7f7123fff640 1 --2- 172.21.15.37:0/3893885789 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f712406c8f0 0x7f7124159450 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:29:48.216 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:48.211+0000 7f7123fff640 1 -- 172.21.15.37:0/3893885789 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f71240cf1f0 con 0x7f712406e390 2024-04-25T10:29:48.216 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:48.211+0000 7f7123fff640 1 --2- 172.21.15.37:0/3893885789 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f712406e390 0x7f712415cef0 secure :-1 s=READY pgs=284 cs=0 l=1 rev1=1 crypto rx=0x7f711c00c6e0 tx=0x7f711c00cba0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:29:48.216 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:48.212+0000 7f710bfff640 1 -- 172.21.15.37:0/3893885789 <== mon.0 v2:172.21.15.37:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f711c003b70 con 0x7f712406e390 2024-04-25T10:29:48.216 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:48.212+0000 7f710bfff640 1 -- 172.21.15.37:0/3893885789 <== mon.0 v2:172.21.15.37:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f711c003d30 con 0x7f712406e390 2024-04-25T10:29:48.216 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:48.212+0000 7f7129c51640 1 -- 172.21.15.37:0/3893885789 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f71240cf4d0 con 0x7f712406e390 2024-04-25T10:29:48.216 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:48.212+0000 7f710bfff640 1 -- 172.21.15.37:0/3893885789 <== mon.0 v2:172.21.15.37:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f711c005310 con 0x7f712406e390 2024-04-25T10:29:48.216 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:48.212+0000 7f7129c51640 1 -- 172.21.15.37:0/3893885789 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f71240cfa90 con 0x7f712406e390 2024-04-25T10:29:48.217 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:48.213+0000 7f7129c51640 1 -- 172.21.15.37:0/3893885789 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f712406a140 con 0x7f712406e390 2024-04-25T10:29:48.217 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:48.213+0000 7f710bfff640 1 -- 172.21.15.37:0/3893885789 <== mon.0 v2:172.21.15.37:3300/0 4 ==== mgrmap(e 5) ==== 96782+0+0 (secure 0 0 0) 0x7f711c0054b0 con 0x7f712406e390 2024-04-25T10:29:48.217 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:48.214+0000 7f710bfff640 1 --2- 172.21.15.37:0/3893885789 >> [v2:172.21.15.203:6832/2418205237,v1:172.21.15.203:6833/2418205237] conn(0x7f71100744b0 0x7f71100769c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:29:48.217 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:48.214+0000 7f710bfff640 1 -- 172.21.15.37:0/3893885789 <== mon.0 v2:172.21.15.37:3300/0 5 ==== osd_map(23..23 src has 1..23) ==== 5709+0+0 (secure 0 0 0) 0x7f711c094ee0 con 0x7f712406e390 2024-04-25T10:29:48.217 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:48.214+0000 7f71237fe640 1 --2- 172.21.15.37:0/3893885789 >> [v2:172.21.15.203:6832/2418205237,v1:172.21.15.203:6833/2418205237] conn(0x7f71100744b0 0x7f71100769c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:29:48.217 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:48.215+0000 7f71237fe640 1 --2- 172.21.15.37:0/3893885789 >> [v2:172.21.15.203:6832/2418205237,v1:172.21.15.203:6833/2418205237] conn(0x7f71100744b0 0x7f71100769c0 secure :-1 s=READY pgs=151 cs=0 l=1 rev1=1 crypto rx=0x7f7114000f60 tx=0x7f7114007e40 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:29:48.220 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:48.219+0000 7f710bfff640 1 -- 172.21.15.37:0/3893885789 <== mon.0 v2:172.21.15.37:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187246 (secure 0 0 0) 0x7f711c05f140 con 0x7f712406e390 2024-04-25T10:29:48.409 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:48.407+0000 7f7129c51640 1 -- 172.21.15.37:0/3893885789 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"} v 0) -- 0x7f712415a3f0 con 0x7f712406e390 2024-04-25T10:29:48.409 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:48.407+0000 7f710bfff640 1 -- 172.21.15.37:0/3893885789 <== mon.0 v2:172.21.15.37:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]=0 v24) ==== 104+0+11 (secure 0 0 0) 0x7f711c063440 con 0x7f712406e390 2024-04-25T10:29:48.410 INFO:teuthology.orchestra.run.smithi037.stdout:pg_num: 32 2024-04-25T10:29:48.412 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:48.409+0000 7f7129c51640 1 -- 172.21.15.37:0/3893885789 >> [v2:172.21.15.203:6832/2418205237,v1:172.21.15.203:6833/2418205237] conn(0x7f71100744b0 msgr2=0x7f71100769c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:29:48.412 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:48.409+0000 7f7129c51640 1 --2- 172.21.15.37:0/3893885789 >> [v2:172.21.15.203:6832/2418205237,v1:172.21.15.203:6833/2418205237] conn(0x7f71100744b0 0x7f71100769c0 secure :-1 s=READY pgs=151 cs=0 l=1 rev1=1 crypto rx=0x7f7114000f60 tx=0x7f7114007e40 comp rx=0 tx=0).stop 2024-04-25T10:29:48.412 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:48.409+0000 7f7129c51640 1 -- 172.21.15.37:0/3893885789 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f712406e390 msgr2=0x7f712415cef0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:29:48.412 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:48.409+0000 7f7129c51640 1 --2- 172.21.15.37:0/3893885789 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f712406e390 0x7f712415cef0 secure :-1 s=READY pgs=284 cs=0 l=1 rev1=1 crypto rx=0x7f711c00c6e0 tx=0x7f711c00cba0 comp rx=0 tx=0).stop 2024-04-25T10:29:48.412 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:48.410+0000 7f7129c51640 1 -- 172.21.15.37:0/3893885789 shutdown_connections 2024-04-25T10:29:48.412 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:48.410+0000 7f7129c51640 1 --2- 172.21.15.37:0/3893885789 >> [v2:172.21.15.203:6832/2418205237,v1:172.21.15.203:6833/2418205237] conn(0x7f71100744b0 0x7f71100769c0 unknown :-1 s=CLOSED pgs=151 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:29:48.412 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:48.410+0000 7f7129c51640 1 --2- 172.21.15.37:0/3893885789 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f712406e390 0x7f712415cef0 unknown :-1 s=CLOSED pgs=284 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:29:48.413 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:48.410+0000 7f7129c51640 1 --2- 172.21.15.37:0/3893885789 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f712406cd00 0x7f71241f6f70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:29:48.413 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:48.410+0000 7f7129c51640 1 --2- 172.21.15.37:0/3893885789 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f712406c8f0 0x7f7124159450 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:29:48.413 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:48.410+0000 7f7129c51640 1 -- 172.21.15.37:0/3893885789 >> 172.21.15.37:0/3893885789 conn(0x7f71240b7600 msgr2=0x7f7124075fa0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:29:48.413 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:48.410+0000 7f7129c51640 1 -- 172.21.15.37:0/3893885789 shutdown_connections 2024-04-25T10:29:48.413 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:48.410+0000 7f7129c51640 1 -- 172.21.15.37:0/3893885789 wait complete. 2024-04-25T10:29:48.423 DEBUG:teuthology.orchestra.run.smithi037:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data pg_num 2024-04-25T10:29:48.537 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:48.535+0000 7ff24cb4d640 1 Processor -- start 2024-04-25T10:29:48.537 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:48.535+0000 7ff24cb4d640 1 -- start start 2024-04-25T10:29:48.541 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:48.537+0000 7ff24cb4d640 1 --2- >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7ff2480b4e80 0x7ff2480d2b20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:29:48.541 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:48.537+0000 7ff24cb4d640 1 --2- >> v2:172.21.15.203:3300/0 conn(0x7ff2480d3060 0x7ff2480b9060 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:29:48.541 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:48.537+0000 7ff24cb4d640 1 -- --> v1:172.21.15.37:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7ff2480d3470 con 0x7ff2480d5160 2024-04-25T10:29:48.541 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:48.537+0000 7ff24cb4d640 1 -- --> v2:172.21.15.203:3300/0 -- mon_getmap magic: 0 -- 0x7ff2480d1650 con 0x7ff2480d3060 2024-04-25T10:29:48.541 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:48.537+0000 7ff24cb4d640 1 -- --> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] -- mon_getmap magic: 0 -- 0x7ff2480b5290 con 0x7ff2480b4e80 2024-04-25T10:29:48.541 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:48.537+0000 7ff24659c640 1 --1- >> v1:172.21.15.37:6789/0 conn(0x7ff2480d5160 0x7ff2480d0990 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.37:6789/0 says I am v1:172.21.15.37:49930/0 (socket says 172.21.15.37:49930) 2024-04-25T10:29:48.541 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:48.537+0000 7ff24659c640 1 -- 172.21.15.37:0/3551611237 learned_addr learned my addr 172.21.15.37:0/3551611237 (peer_addr_for_me v1:172.21.15.37:0/0) 2024-04-25T10:29:48.541 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:48.537+0000 7ff246d9d640 1 --2- 172.21.15.37:0/3551611237 >> v2:172.21.15.203:3300/0 conn(0x7ff2480d3060 0x7ff2480b9060 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:29:48.541 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:48.537+0000 7ff245d9b640 1 --2- 172.21.15.37:0/3551611237 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7ff2480b4e80 0x7ff2480d2b20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:29:48.541 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:48.537+0000 7ff24559a640 1 -- 172.21.15.37:0/3551611237 <== mon.0 v1:172.21.15.37:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 2810690883 0 0) 0x7ff240001110 con 0x7ff2480d5160 2024-04-25T10:29:48.542 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:48.537+0000 7ff24559a640 1 -- 172.21.15.37:0/3551611237 --> v1:172.21.15.37:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7ff234003520 con 0x7ff2480d5160 2024-04-25T10:29:48.542 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:48.537+0000 7ff24559a640 1 -- 172.21.15.37:0/3551611237 <== mon.0 v1:172.21.15.37:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 659851137 0 0) 0x7ff234003520 con 0x7ff2480d5160 2024-04-25T10:29:48.542 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:48.537+0000 7ff24559a640 1 -- 172.21.15.37:0/3551611237 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7ff2480b4e80 msgr2=0x7ff2480d2b20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:29:48.542 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:48.537+0000 7ff24559a640 1 --2- 172.21.15.37:0/3551611237 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7ff2480b4e80 0x7ff2480d2b20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:29:48.542 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:48.537+0000 7ff24559a640 1 -- 172.21.15.37:0/3551611237 >> v2:172.21.15.203:3300/0 conn(0x7ff2480d3060 msgr2=0x7ff2480b9060 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:29:48.542 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:48.537+0000 7ff24559a640 1 --2- 172.21.15.37:0/3551611237 >> v2:172.21.15.203:3300/0 conn(0x7ff2480d3060 0x7ff2480b9060 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:29:48.542 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:48.537+0000 7ff24559a640 1 -- 172.21.15.37:0/3551611237 --> v1:172.21.15.37:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ff2480d10a0 con 0x7ff2480d5160 2024-04-25T10:29:48.542 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:48.537+0000 7ff24559a640 1 -- 172.21.15.37:0/3551611237 <== mon.0 v1:172.21.15.37:6789/0 3 ==== mon_map magic: 0 ==== 413+0+0 (unknown 1577794600 0 0) 0x7ff240001860 con 0x7ff2480d5160 2024-04-25T10:29:48.542 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:48.537+0000 7ff246d9d640 1 --2- 172.21.15.37:0/3551611237 >> v2:172.21.15.203:3300/0 conn(0x7ff2480d3060 0x7ff2480b9060 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-25T10:29:48.542 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:48.537+0000 7ff24559a640 1 -- 172.21.15.37:0/3551611237 >> v1:172.21.15.37:6789/0 conn(0x7ff2480d5160 legacy=0x7ff2480d0990 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:29:48.542 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:48.537+0000 7ff24559a640 1 --2- 172.21.15.37:0/3551611237 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7ff2340039b0 0x7ff234003e20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:29:48.542 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:48.537+0000 7ff24559a640 1 --2- 172.21.15.37:0/3551611237 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7ff2480b4e80 0x7ff234004800 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:29:48.542 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:48.537+0000 7ff24559a640 1 --2- 172.21.15.37:0/3551611237 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7ff234004d40 0x7ff234005150 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:29:48.542 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:48.537+0000 7ff24559a640 1 -- 172.21.15.37:0/3551611237 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 -- 0x7ff2480d1650 con 0x7ff234004d40 2024-04-25T10:29:48.543 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:48.537+0000 7ff24559a640 1 -- 172.21.15.37:0/3551611237 --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_getmap magic: 0 -- 0x7ff2480b5290 con 0x7ff2340039b0 2024-04-25T10:29:48.543 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:48.537+0000 7ff24559a640 1 -- 172.21.15.37:0/3551611237 --> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] -- mon_getmap magic: 0 -- 0x7ff23400ba40 con 0x7ff2480b4e80 2024-04-25T10:29:48.543 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:48.538+0000 7ff246d9d640 1 --2- 172.21.15.37:0/3551611237 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7ff234004d40 0x7ff234005150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:29:48.543 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:48.538+0000 7ff24659c640 1 --2- 172.21.15.37:0/3551611237 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7ff2340039b0 0x7ff234003e20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:29:48.543 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:48.538+0000 7ff245d9b640 1 --2- 172.21.15.37:0/3551611237 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7ff2480b4e80 0x7ff234004800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:29:48.543 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:48.538+0000 7ff246d9d640 1 -- 172.21.15.37:0/3551611237 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7ff2480b4e80 msgr2=0x7ff234004800 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:29:48.543 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:48.538+0000 7ff246d9d640 1 --2- 172.21.15.37:0/3551611237 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7ff2480b4e80 0x7ff234004800 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:29:48.543 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:48.538+0000 7ff246d9d640 1 -- 172.21.15.37:0/3551611237 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7ff2340039b0 msgr2=0x7ff234003e20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:29:48.543 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:48.538+0000 7ff246d9d640 1 --2- 172.21.15.37:0/3551611237 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7ff2340039b0 0x7ff234003e20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:29:48.543 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:48.538+0000 7ff246d9d640 1 -- 172.21.15.37:0/3551611237 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ff234003520 con 0x7ff234004d40 2024-04-25T10:29:48.543 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:48.538+0000 7ff245d9b640 1 --2- 172.21.15.37:0/3551611237 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7ff2480b4e80 0x7ff234004800 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T10:29:48.543 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:48.539+0000 7ff246d9d640 1 --2- 172.21.15.37:0/3551611237 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7ff234004d40 0x7ff234005150 secure :-1 s=READY pgs=286 cs=0 l=1 rev1=1 crypto rx=0x7ff2480d5990 tx=0x7ff234005830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=95533383bd6bd4df server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:29:48.543 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:48.539+0000 7ff24559a640 1 -- 172.21.15.37:0/3551611237 <== mon.0 v2:172.21.15.37:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff238033070 con 0x7ff234004d40 2024-04-25T10:29:48.543 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:48.539+0000 7ff24559a640 1 -- 172.21.15.37:0/3551611237 <== mon.0 v2:172.21.15.37:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7ff238003b90 con 0x7ff234004d40 2024-04-25T10:29:48.544 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:48.539+0000 7ff24559a640 1 -- 172.21.15.37:0/3551611237 <== mon.0 v2:172.21.15.37:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff238009de0 con 0x7ff234004d40 2024-04-25T10:29:48.546 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:48.539+0000 7ff24cb4d640 1 -- 172.21.15.37:0/3551611237 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7ff234004d40 msgr2=0x7ff234005150 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:29:48.546 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:48.539+0000 7ff24cb4d640 1 --2- 172.21.15.37:0/3551611237 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7ff234004d40 0x7ff234005150 secure :-1 s=READY pgs=286 cs=0 l=1 rev1=1 crypto rx=0x7ff2480d5990 tx=0x7ff234005830 comp rx=0 tx=0).stop 2024-04-25T10:29:48.546 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:48.539+0000 7ff246d9d640 1 -- 172.21.15.37:0/3551611237 reap_dead start 2024-04-25T10:29:48.546 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:48.540+0000 7ff24cb4d640 1 -- 172.21.15.37:0/3551611237 shutdown_connections 2024-04-25T10:29:48.546 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:48.540+0000 7ff24cb4d640 1 -- 172.21.15.37:0/3551611237 >> 172.21.15.37:0/3551611237 conn(0x7ff2480b7a80 msgr2=0x7ff2480b7e90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:29:48.547 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:48.540+0000 7ff24cb4d640 1 -- 172.21.15.37:0/3551611237 shutdown_connections 2024-04-25T10:29:48.547 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:48.540+0000 7ff24cb4d640 1 -- 172.21.15.37:0/3551611237 wait complete. 2024-04-25T10:29:48.547 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:48.542+0000 7ff24cb4d640 1 Processor -- start 2024-04-25T10:29:48.547 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:48.542+0000 7ff24cb4d640 1 -- start start 2024-04-25T10:29:48.547 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:48.543+0000 7ff24cb4d640 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7ff234004d40 0x7ff248155700 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:29:48.547 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:48.543+0000 7ff24cb4d640 1 --2- >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7ff248158370 0x7ff248161c00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:29:48.547 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:48.543+0000 7ff24cb4d640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7ff248162140 0x7ff2480cb6a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:29:48.547 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:48.543+0000 7ff24cb4d640 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 -- 0x7ff2480d4960 con 0x7ff248162140 2024-04-25T10:29:48.547 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:48.543+0000 7ff24cb4d640 1 -- --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_getmap magic: 0 -- 0x7ff2480d4680 con 0x7ff234004d40 2024-04-25T10:29:48.547 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:48.543+0000 7ff24cb4d640 1 -- --> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] -- mon_getmap magic: 0 -- 0x7ff2480d4c40 con 0x7ff248158370 2024-04-25T10:29:48.547 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:48.543+0000 7ff246d9d640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7ff248162140 0x7ff2480cb6a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:29:48.547 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:48.543+0000 7ff246d9d640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7ff248162140 0x7ff2480cb6a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.37:3300/0 says I am v2:172.21.15.37:59702/0 (socket says 172.21.15.37:59702) 2024-04-25T10:29:48.547 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:48.543+0000 7ff246d9d640 1 -- 172.21.15.37:0/2051277163 learned_addr learned my addr 172.21.15.37:0/2051277163 (peer_addr_for_me v2:172.21.15.37:0/0) 2024-04-25T10:29:48.547 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:48.543+0000 7ff245d9b640 1 --2- 172.21.15.37:0/2051277163 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7ff248158370 0x7ff248161c00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:29:48.548 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:48.543+0000 7ff24659c640 1 --2- 172.21.15.37:0/2051277163 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7ff234004d40 0x7ff248155700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:29:48.548 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:48.543+0000 7ff246d9d640 1 -- 172.21.15.37:0/2051277163 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7ff248158370 msgr2=0x7ff248161c00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:29:48.548 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:48.543+0000 7ff246d9d640 1 --2- 172.21.15.37:0/2051277163 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7ff248158370 0x7ff248161c00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:29:48.548 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:48.543+0000 7ff246d9d640 1 -- 172.21.15.37:0/2051277163 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7ff234004d40 msgr2=0x7ff248155700 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:29:48.548 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:48.543+0000 7ff246d9d640 1 --2- 172.21.15.37:0/2051277163 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7ff234004d40 0x7ff248155700 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:29:48.548 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:48.543+0000 7ff246d9d640 1 -- 172.21.15.37:0/2051277163 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ff248162550 con 0x7ff248162140 2024-04-25T10:29:48.548 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:48.544+0000 7ff246d9d640 1 --2- 172.21.15.37:0/2051277163 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7ff248162140 0x7ff2480cb6a0 secure :-1 s=READY pgs=287 cs=0 l=1 rev1=1 crypto rx=0x7ff23800e3e0 tx=0x7ff238009750 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:29:48.548 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:48.544+0000 7ff23effd640 1 -- 172.21.15.37:0/2051277163 <== mon.0 v2:172.21.15.37:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff238033040 con 0x7ff248162140 2024-04-25T10:29:48.548 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:48.544+0000 7ff23effd640 1 -- 172.21.15.37:0/2051277163 <== mon.0 v2:172.21.15.37:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7ff238003990 con 0x7ff248162140 2024-04-25T10:29:48.548 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:48.544+0000 7ff24cb4d640 1 -- 172.21.15.37:0/2051277163 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7ff2480d0750 con 0x7ff248162140 2024-04-25T10:29:48.548 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:48.544+0000 7ff23effd640 1 -- 172.21.15.37:0/2051277163 <== mon.0 v2:172.21.15.37:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff238009de0 con 0x7ff248162140 2024-04-25T10:29:48.548 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:48.544+0000 7ff24cb4d640 1 -- 172.21.15.37:0/2051277163 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7ff2481597e0 con 0x7ff248162140 2024-04-25T10:29:48.549 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:48.545+0000 7ff24cb4d640 1 -- 172.21.15.37:0/2051277163 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7ff2480d09a0 con 0x7ff248162140 2024-04-25T10:29:48.549 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:48.546+0000 7ff23effd640 1 -- 172.21.15.37:0/2051277163 <== mon.0 v2:172.21.15.37:3300/0 4 ==== mgrmap(e 5) ==== 96782+0+0 (secure 0 0 0) 0x7ff238037400 con 0x7ff248162140 2024-04-25T10:29:48.549 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:48.546+0000 7ff23effd640 1 --2- 172.21.15.37:0/2051277163 >> [v2:172.21.15.203:6832/2418205237,v1:172.21.15.203:6833/2418205237] conn(0x7ff2280744e0 0x7ff2280769f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:29:48.549 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:48.546+0000 7ff23effd640 1 -- 172.21.15.37:0/2051277163 <== mon.0 v2:172.21.15.37:3300/0 5 ==== osd_map(24..24 src has 1..24) ==== 5738+0+0 (secure 0 0 0) 0x7ff2380b60b0 con 0x7ff248162140 2024-04-25T10:29:48.550 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:48.546+0000 7ff24659c640 1 --2- 172.21.15.37:0/2051277163 >> [v2:172.21.15.203:6832/2418205237,v1:172.21.15.203:6833/2418205237] conn(0x7ff2280744e0 0x7ff2280769f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:29:48.550 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:48.547+0000 7ff24659c640 1 --2- 172.21.15.37:0/2051277163 >> [v2:172.21.15.203:6832/2418205237,v1:172.21.15.203:6833/2418205237] conn(0x7ff2280744e0 0x7ff2280769f0 secure :-1 s=READY pgs=152 cs=0 l=1 rev1=1 crypto rx=0x7ff2480c9650 tx=0x7ff240009040 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:29:48.552 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:48.551+0000 7ff23effd640 1 -- 172.21.15.37:0/2051277163 <== mon.0 v2:172.21.15.37:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187246 (secure 0 0 0) 0x7ff248162550 con 0x7ff248162140 2024-04-25T10:29:48.738 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:48.736+0000 7ff24cb4d640 1 -- 172.21.15.37:0/2051277163 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"} v 0) -- 0x7ff24806a140 con 0x7ff248162140 2024-04-25T10:29:48.738 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:48.736+0000 7ff23effd640 1 -- 172.21.15.37:0/2051277163 <== mon.0 v2:172.21.15.37:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]=0 v24) ==== 100+0+11 (secure 0 0 0) 0x7ff2380807a0 con 0x7ff248162140 2024-04-25T10:29:48.738 INFO:teuthology.orchestra.run.smithi037.stdout:pg_num: 64 2024-04-25T10:29:48.740 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:48.738+0000 7ff24cb4d640 1 -- 172.21.15.37:0/2051277163 >> [v2:172.21.15.203:6832/2418205237,v1:172.21.15.203:6833/2418205237] conn(0x7ff2280744e0 msgr2=0x7ff2280769f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:29:48.740 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:48.738+0000 7ff24cb4d640 1 --2- 172.21.15.37:0/2051277163 >> [v2:172.21.15.203:6832/2418205237,v1:172.21.15.203:6833/2418205237] conn(0x7ff2280744e0 0x7ff2280769f0 secure :-1 s=READY pgs=152 cs=0 l=1 rev1=1 crypto rx=0x7ff2480c9650 tx=0x7ff240009040 comp rx=0 tx=0).stop 2024-04-25T10:29:48.741 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:48.738+0000 7ff24cb4d640 1 -- 172.21.15.37:0/2051277163 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7ff248162140 msgr2=0x7ff2480cb6a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:29:48.741 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:48.738+0000 7ff24cb4d640 1 --2- 172.21.15.37:0/2051277163 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7ff248162140 0x7ff2480cb6a0 secure :-1 s=READY pgs=287 cs=0 l=1 rev1=1 crypto rx=0x7ff23800e3e0 tx=0x7ff238009750 comp rx=0 tx=0).stop 2024-04-25T10:29:48.741 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:48.738+0000 7ff24cb4d640 1 -- 172.21.15.37:0/2051277163 shutdown_connections 2024-04-25T10:29:48.741 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:48.738+0000 7ff24cb4d640 1 --2- 172.21.15.37:0/2051277163 >> [v2:172.21.15.203:6832/2418205237,v1:172.21.15.203:6833/2418205237] conn(0x7ff2280744e0 0x7ff2280769f0 unknown :-1 s=CLOSED pgs=152 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:29:48.741 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:48.738+0000 7ff24cb4d640 1 --2- 172.21.15.37:0/2051277163 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7ff248162140 0x7ff2480cb6a0 unknown :-1 s=CLOSED pgs=287 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:29:48.741 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:48.738+0000 7ff24cb4d640 1 --2- 172.21.15.37:0/2051277163 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7ff248158370 0x7ff248161c00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:29:48.741 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:48.738+0000 7ff24cb4d640 1 --2- 172.21.15.37:0/2051277163 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7ff234004d40 0x7ff248155700 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:29:48.741 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:48.738+0000 7ff24cb4d640 1 -- 172.21.15.37:0/2051277163 >> 172.21.15.37:0/2051277163 conn(0x7ff2480b7a80 msgr2=0x7ff2481632a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:29:48.741 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:48.738+0000 7ff24cb4d640 1 -- 172.21.15.37:0/2051277163 shutdown_connections 2024-04-25T10:29:48.741 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:48.738+0000 7ff24cb4d640 1 -- 172.21.15.37:0/2051277163 wait complete. 2024-04-25T10:29:48.751 DEBUG:teuthology.orchestra.run.smithi037:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd blocklist ls 2024-04-25T10:29:48.864 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:48.861+0000 7fc9be1c5640 1 Processor -- start 2024-04-25T10:29:48.864 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:48.862+0000 7fc9be1c5640 1 -- start start 2024-04-25T10:29:48.870 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:48.863+0000 7fc9be1c5640 1 --2- >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7fc9b80b4e80 0x7fc9b80d2b20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:29:48.870 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:48.863+0000 7fc9be1c5640 1 -- --> v1:172.21.15.37:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fc9b80d3470 con 0x7fc9b80d5160 2024-04-25T10:29:48.870 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:48.863+0000 7fc9be1c5640 1 -- --> v1:172.21.15.203:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fc9b80d10a0 con 0x7fc9b80d3060 2024-04-25T10:29:48.870 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:48.863+0000 7fc9be1c5640 1 -- --> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] -- mon_getmap magic: 0 -- 0x7fc9b80d1650 con 0x7fc9b80b4e80 2024-04-25T10:29:48.870 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:48.863+0000 7fc9b77fe640 1 --1- >> v1:172.21.15.37:6789/0 conn(0x7fc9b80d5160 0x7fc9b80d0990 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.37:6789/0 says I am v1:172.21.15.37:49944/0 (socket says 172.21.15.37:49944) 2024-04-25T10:29:48.870 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:48.863+0000 7fc9b77fe640 1 -- 172.21.15.37:0/1456627291 learned_addr learned my addr 172.21.15.37:0/1456627291 (peer_addr_for_me v1:172.21.15.37:0/0) 2024-04-25T10:29:48.870 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:48.863+0000 7fc9b67fc640 1 -- 172.21.15.37:0/1456627291 <== mon.0 v1:172.21.15.37:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 1260491651 0 0) 0x7fc9a8001110 con 0x7fc9b80d5160 2024-04-25T10:29:48.870 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:48.863+0000 7fc9b6ffd640 1 --2- 172.21.15.37:0/1456627291 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7fc9b80b4e80 0x7fc9b80d2b20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:29:48.870 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:48.863+0000 7fc9b67fc640 1 -- 172.21.15.37:0/1456627291 --> v1:172.21.15.37:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7fc9a0003520 con 0x7fc9b80d5160 2024-04-25T10:29:48.870 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:48.864+0000 7fc9b67fc640 1 -- 172.21.15.37:0/1456627291 <== mon.0 v1:172.21.15.37:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 1837532469 0 0) 0x7fc9a0003520 con 0x7fc9b80d5160 2024-04-25T10:29:48.870 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:48.864+0000 7fc9b67fc640 1 -- 172.21.15.37:0/1456627291 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7fc9b80b4e80 msgr2=0x7fc9b80d2b20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:29:48.870 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:48.864+0000 7fc9b67fc640 1 --2- 172.21.15.37:0/1456627291 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7fc9b80b4e80 0x7fc9b80d2b20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:29:48.871 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:48.864+0000 7fc9b67fc640 1 -- 172.21.15.37:0/1456627291 >> v1:172.21.15.203:6789/0 conn(0x7fc9b80d3060 legacy=0x7fc9b80b9060 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:29:48.871 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:48.864+0000 7fc9b67fc640 1 -- 172.21.15.37:0/1456627291 --> v1:172.21.15.37:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fc9b80d1370 con 0x7fc9b80d5160 2024-04-25T10:29:48.871 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:48.864+0000 7fc9b67fc640 1 -- 172.21.15.37:0/1456627291 <== mon.0 v1:172.21.15.37:6789/0 3 ==== mon_map magic: 0 ==== 413+0+0 (unknown 1577794600 0 0) 0x7fc9a8001860 con 0x7fc9b80d5160 2024-04-25T10:29:48.871 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:48.864+0000 7fc9b67fc640 1 -- 172.21.15.37:0/1456627291 >> v1:172.21.15.37:6789/0 conn(0x7fc9b80d5160 legacy=0x7fc9b80d0990 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:29:48.871 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:48.864+0000 7fc9b67fc640 1 --2- 172.21.15.37:0/1456627291 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7fc9b80b4e80 0x7fc9a00039e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:29:48.871 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:48.864+0000 7fc9b67fc640 1 --2- 172.21.15.37:0/1456627291 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fc9a0003f60 0x7fc9a0004800 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:29:48.871 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:48.864+0000 7fc9b67fc640 1 --2- 172.21.15.37:0/1456627291 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fc9a0004d40 0x7fc9a0005150 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:29:48.871 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:48.864+0000 7fc9b67fc640 1 -- 172.21.15.37:0/1456627291 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 -- 0x7fc9b80d1650 con 0x7fc9a0004d40 2024-04-25T10:29:48.871 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:48.864+0000 7fc9b67fc640 1 -- 172.21.15.37:0/1456627291 --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_getmap magic: 0 -- 0x7fc9a000bc00 con 0x7fc9a0003f60 2024-04-25T10:29:48.871 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:48.864+0000 7fc9b67fc640 1 -- 172.21.15.37:0/1456627291 --> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] -- mon_getmap magic: 0 -- 0x7fc9a000be00 con 0x7fc9b80b4e80 2024-04-25T10:29:48.871 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:48.864+0000 7fc9b7fff640 1 --2- 172.21.15.37:0/1456627291 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fc9a0004d40 0x7fc9a0005150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:29:48.871 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:48.864+0000 7fc9b77fe640 1 --2- 172.21.15.37:0/1456627291 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7fc9b80b4e80 0x7fc9a00039e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:29:48.871 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:48.864+0000 7fc9b6ffd640 1 --2- 172.21.15.37:0/1456627291 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fc9a0003f60 0x7fc9a0004800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:29:48.871 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:48.864+0000 7fc9b7fff640 1 -- 172.21.15.37:0/1456627291 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7fc9b80b4e80 msgr2=0x7fc9a00039e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:29:48.871 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:48.864+0000 7fc9b7fff640 1 --2- 172.21.15.37:0/1456627291 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7fc9b80b4e80 0x7fc9a00039e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:29:48.872 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:48.864+0000 7fc9b7fff640 1 -- 172.21.15.37:0/1456627291 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fc9a0003f60 msgr2=0x7fc9a0004800 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:29:48.872 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:48.864+0000 7fc9b7fff640 1 --2- 172.21.15.37:0/1456627291 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fc9a0003f60 0x7fc9a0004800 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:29:48.872 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:48.864+0000 7fc9b7fff640 1 -- 172.21.15.37:0/1456627291 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fc9a0003520 con 0x7fc9a0004d40 2024-04-25T10:29:48.872 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:48.865+0000 7fc9b7fff640 1 --2- 172.21.15.37:0/1456627291 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fc9a0004d40 0x7fc9a0005150 secure :-1 s=READY pgs=289 cs=0 l=1 rev1=1 crypto rx=0x7fc9b80d5990 tx=0x7fc9a0005830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=98fdb7f88e7dbfce server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:29:48.872 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:48.865+0000 7fc9b67fc640 1 -- 172.21.15.37:0/1456627291 <== mon.0 v2:172.21.15.37:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc9b0001a70 con 0x7fc9a0004d40 2024-04-25T10:29:48.872 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:48.865+0000 7fc9b67fc640 1 -- 172.21.15.37:0/1456627291 <== mon.0 v2:172.21.15.37:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fc9b002add0 con 0x7fc9a0004d40 2024-04-25T10:29:48.872 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:48.865+0000 7fc9b67fc640 1 -- 172.21.15.37:0/1456627291 <== mon.0 v2:172.21.15.37:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc9b0003bb0 con 0x7fc9a0004d40 2024-04-25T10:29:48.872 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:48.866+0000 7fc9be1c5640 1 -- 172.21.15.37:0/1456627291 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fc9a0004d40 msgr2=0x7fc9a0005150 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:29:48.872 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:48.866+0000 7fc9be1c5640 1 --2- 172.21.15.37:0/1456627291 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fc9a0004d40 0x7fc9a0005150 secure :-1 s=READY pgs=289 cs=0 l=1 rev1=1 crypto rx=0x7fc9b80d5990 tx=0x7fc9a0005830 comp rx=0 tx=0).stop 2024-04-25T10:29:48.872 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:48.866+0000 7fc9b7fff640 1 -- 172.21.15.37:0/1456627291 reap_dead start 2024-04-25T10:29:48.872 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:48.866+0000 7fc9be1c5640 1 -- 172.21.15.37:0/1456627291 shutdown_connections 2024-04-25T10:29:48.872 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:48.866+0000 7fc9be1c5640 1 -- 172.21.15.37:0/1456627291 >> 172.21.15.37:0/1456627291 conn(0x7fc9b80b7a80 msgr2=0x7fc9b80b7e90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:29:48.872 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:48.866+0000 7fc9be1c5640 1 -- 172.21.15.37:0/1456627291 shutdown_connections 2024-04-25T10:29:48.872 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:48.866+0000 7fc9be1c5640 1 -- 172.21.15.37:0/1456627291 wait complete. 2024-04-25T10:29:48.873 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:48.868+0000 7fc9be1c5640 1 Processor -- start 2024-04-25T10:29:48.873 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:48.868+0000 7fc9be1c5640 1 -- start start 2024-04-25T10:29:48.875 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:48.869+0000 7fc9be1c5640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fc9a0004d40 0x7fc9b81e99e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:29:48.875 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:48.869+0000 7fc9be1c5640 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fc9b81f97c0 0x7fc9b815b570 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:29:48.875 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:48.869+0000 7fc9be1c5640 1 --2- >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7fc9b8161970 0x7fc9b8161d80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:29:48.875 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:48.869+0000 7fc9be1c5640 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 -- 0x7fc9b80d4960 con 0x7fc9a0004d40 2024-04-25T10:29:48.875 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:48.869+0000 7fc9be1c5640 1 -- --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_getmap magic: 0 -- 0x7fc9b80b5290 con 0x7fc9b81f97c0 2024-04-25T10:29:48.875 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:48.869+0000 7fc9be1c5640 1 -- --> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] -- mon_getmap magic: 0 -- 0x7fc9b80d4c40 con 0x7fc9b8161970 2024-04-25T10:29:48.876 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:48.869+0000 7fc9b77fe640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fc9a0004d40 0x7fc9b81e99e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:29:48.876 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:48.869+0000 7fc9b77fe640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fc9a0004d40 0x7fc9b81e99e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.37:3300/0 says I am v2:172.21.15.37:59724/0 (socket says 172.21.15.37:59724) 2024-04-25T10:29:48.876 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:48.869+0000 7fc9b77fe640 1 -- 172.21.15.37:0/4247894900 learned_addr learned my addr 172.21.15.37:0/4247894900 (peer_addr_for_me v2:172.21.15.37:0/0) 2024-04-25T10:29:48.876 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:48.869+0000 7fc9b6ffd640 1 --2- 172.21.15.37:0/4247894900 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fc9b81f97c0 0x7fc9b815b570 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:29:48.876 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:48.869+0000 7fc9b7fff640 1 --2- 172.21.15.37:0/4247894900 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7fc9b8161970 0x7fc9b8161d80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:29:48.876 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:48.869+0000 7fc9b77fe640 1 -- 172.21.15.37:0/4247894900 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7fc9b8161970 msgr2=0x7fc9b8161d80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:29:48.876 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:48.870+0000 7fc9b77fe640 1 --2- 172.21.15.37:0/4247894900 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7fc9b8161970 0x7fc9b8161d80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:29:48.876 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:48.870+0000 7fc9b77fe640 1 -- 172.21.15.37:0/4247894900 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fc9b81f97c0 msgr2=0x7fc9b815b570 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:29:48.876 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:48.870+0000 7fc9b77fe640 1 --2- 172.21.15.37:0/4247894900 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fc9b81f97c0 0x7fc9b815b570 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:29:48.876 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:48.870+0000 7fc9b77fe640 1 -- 172.21.15.37:0/4247894900 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fc9b81622c0 con 0x7fc9a0004d40 2024-04-25T10:29:48.876 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:48.870+0000 7fc9b7fff640 1 --2- 172.21.15.37:0/4247894900 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7fc9b8161970 0x7fc9b8161d80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T10:29:48.876 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:48.870+0000 7fc9b77fe640 1 --2- 172.21.15.37:0/4247894900 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fc9a0004d40 0x7fc9b81e99e0 secure :-1 s=READY pgs=290 cs=0 l=1 rev1=1 crypto rx=0x7fc9a8007290 tx=0x7fc9a8007690 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:29:48.877 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:48.870+0000 7fc99ffff640 1 -- 172.21.15.37:0/4247894900 <== mon.0 v2:172.21.15.37:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc9a800f960 con 0x7fc9a0004d40 2024-04-25T10:29:48.877 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:48.870+0000 7fc99ffff640 1 -- 172.21.15.37:0/4247894900 <== mon.0 v2:172.21.15.37:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fc9a8018070 con 0x7fc9a0004d40 2024-04-25T10:29:48.877 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:48.870+0000 7fc9be1c5640 1 -- 172.21.15.37:0/4247894900 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fc9b81f9fa0 con 0x7fc9a0004d40 2024-04-25T10:29:48.877 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:48.870+0000 7fc99ffff640 1 -- 172.21.15.37:0/4247894900 <== mon.0 v2:172.21.15.37:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc9a800fd80 con 0x7fc9a0004d40 2024-04-25T10:29:48.877 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:48.870+0000 7fc9be1c5640 1 -- 172.21.15.37:0/4247894900 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fc9b81fa560 con 0x7fc9a0004d40 2024-04-25T10:29:48.877 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:48.871+0000 7fc9be1c5640 1 -- 172.21.15.37:0/4247894900 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fc9b806a140 con 0x7fc9a0004d40 2024-04-25T10:29:48.877 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:48.872+0000 7fc99ffff640 1 -- 172.21.15.37:0/4247894900 <== mon.0 v2:172.21.15.37:3300/0 4 ==== mgrmap(e 5) ==== 96782+0+0 (secure 0 0 0) 0x7fc9a801d460 con 0x7fc9a0004d40 2024-04-25T10:29:48.877 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:48.872+0000 7fc99ffff640 1 --2- 172.21.15.37:0/4247894900 >> [v2:172.21.15.203:6832/2418205237,v1:172.21.15.203:6833/2418205237] conn(0x7fc9a40744b0 0x7fc9a40769c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:29:48.877 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:48.872+0000 7fc99ffff640 1 -- 172.21.15.37:0/4247894900 <== mon.0 v2:172.21.15.37:3300/0 5 ==== osd_map(24..24 src has 1..24) ==== 5738+0+0 (secure 0 0 0) 0x7fc9a8095300 con 0x7fc9a0004d40 2024-04-25T10:29:48.878 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:48.872+0000 7fc9b6ffd640 1 --2- 172.21.15.37:0/4247894900 >> [v2:172.21.15.203:6832/2418205237,v1:172.21.15.203:6833/2418205237] conn(0x7fc9a40744b0 0x7fc9a40769c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:29:48.878 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:48.873+0000 7fc9b6ffd640 1 --2- 172.21.15.37:0/4247894900 >> [v2:172.21.15.203:6832/2418205237,v1:172.21.15.203:6833/2418205237] conn(0x7fc9a40744b0 0x7fc9a40769c0 secure :-1 s=READY pgs=153 cs=0 l=1 rev1=1 crypto rx=0x7fc9b80c9620 tx=0x7fc9ac009190 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:29:48.879 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:48.877+0000 7fc99ffff640 1 -- 172.21.15.37:0/4247894900 <== mon.0 v2:172.21.15.37:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187246 (secure 0 0 0) 0x7fc9b81622c0 con 0x7fc9a0004d40 2024-04-25T10:29:49.083 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:49.080+0000 7fc9be1c5640 1 -- 172.21.15.37:0/4247894900 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_command({"prefix": "osd blocklist ls"} v 0) -- 0x7fc9b81632c0 con 0x7fc9a0004d40 2024-04-25T10:29:49.083 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:49.081+0000 7fc99ffff640 1 -- 172.21.15.37:0/4247894900 <== mon.0 v2:172.21.15.37:3300/0 7 ==== mon_command_ack([{"prefix": "osd blocklist ls"}]=0 listed 0 entries v24) ==== 80+0+0 (secure 0 0 0) 0x7fc9b80d3470 con 0x7fc9a0004d40 2024-04-25T10:29:49.083 INFO:teuthology.orchestra.run.smithi037.stderr:listed 0 entries 2024-04-25T10:29:49.085 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:49.082+0000 7fc9be1c5640 1 -- 172.21.15.37:0/4247894900 >> [v2:172.21.15.203:6832/2418205237,v1:172.21.15.203:6833/2418205237] conn(0x7fc9a40744b0 msgr2=0x7fc9a40769c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:29:49.085 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:49.082+0000 7fc9be1c5640 1 --2- 172.21.15.37:0/4247894900 >> [v2:172.21.15.203:6832/2418205237,v1:172.21.15.203:6833/2418205237] conn(0x7fc9a40744b0 0x7fc9a40769c0 secure :-1 s=READY pgs=153 cs=0 l=1 rev1=1 crypto rx=0x7fc9b80c9620 tx=0x7fc9ac009190 comp rx=0 tx=0).stop 2024-04-25T10:29:49.085 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:49.082+0000 7fc9be1c5640 1 -- 172.21.15.37:0/4247894900 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fc9a0004d40 msgr2=0x7fc9b81e99e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:29:49.085 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:49.082+0000 7fc9be1c5640 1 --2- 172.21.15.37:0/4247894900 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fc9a0004d40 0x7fc9b81e99e0 secure :-1 s=READY pgs=290 cs=0 l=1 rev1=1 crypto rx=0x7fc9a8007290 tx=0x7fc9a8007690 comp rx=0 tx=0).stop 2024-04-25T10:29:49.085 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:49.083+0000 7fc9be1c5640 1 -- 172.21.15.37:0/4247894900 shutdown_connections 2024-04-25T10:29:49.085 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:49.083+0000 7fc9be1c5640 1 --2- 172.21.15.37:0/4247894900 >> [v2:172.21.15.203:6832/2418205237,v1:172.21.15.203:6833/2418205237] conn(0x7fc9a40744b0 0x7fc9a40769c0 unknown :-1 s=CLOSED pgs=153 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:29:49.086 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:49.083+0000 7fc9be1c5640 1 --2- 172.21.15.37:0/4247894900 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7fc9b8161970 0x7fc9b8161d80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:29:49.086 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:49.083+0000 7fc9be1c5640 1 --2- 172.21.15.37:0/4247894900 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fc9b81f97c0 0x7fc9b815b570 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:29:49.086 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:49.083+0000 7fc9be1c5640 1 --2- 172.21.15.37:0/4247894900 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fc9a0004d40 0x7fc9b81e99e0 unknown :-1 s=CLOSED pgs=290 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:29:49.086 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:49.083+0000 7fc9be1c5640 1 -- 172.21.15.37:0/4247894900 >> 172.21.15.37:0/4247894900 conn(0x7fc9b80b7a80 msgr2=0x7fc9b815ece0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:29:49.086 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:49.083+0000 7fc9be1c5640 1 -- 172.21.15.37:0/4247894900 shutdown_connections 2024-04-25T10:29:49.086 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:49.083+0000 7fc9be1c5640 1 -- 172.21.15.37:0/4247894900 wait complete. 2024-04-25T10:29:49.096 DEBUG:teuthology.orchestra.run.smithi203:> set -ex 2024-04-25T10:29:49.096 DEBUG:teuthology.orchestra.run.smithi203:> dd if=/proc/self/mounts of=/dev/stdout 2024-04-25T10:29:49.115 DEBUG:teuthology.orchestra.run.smithi037:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-04-25T10:29:49.226 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:49.224+0000 7fad535f8640 1 Processor -- start 2024-04-25T10:29:49.226 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:49.224+0000 7fad535f8640 1 -- start start 2024-04-25T10:29:49.229 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:49.225+0000 7fad535f8640 1 --2- >> v2:172.21.15.37:3300/0 conn(0x7fad4c0b7cd0 0x7fad4c160a00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:29:49.229 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:49.225+0000 7fad535f8640 1 --2- >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7fad4c160f40 0x7fad4c161880 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:29:49.229 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:49.225+0000 7fad535f8640 1 -- --> v1:172.21.15.203:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fad4c0d5840 con 0x7fad4c0d5430 2024-04-25T10:29:49.229 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:49.225+0000 7fad535f8640 1 -- --> v2:172.21.15.37:3300/0 -- mon_getmap magic: 0 -- 0x7fad4c0d4420 con 0x7fad4c0b7cd0 2024-04-25T10:29:49.229 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:49.225+0000 7fad535f8640 1 -- --> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] -- mon_getmap magic: 0 -- 0x7fad4c0d4670 con 0x7fad4c160f40 2024-04-25T10:29:49.229 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:49.225+0000 7fad50b93640 1 --2- >> v2:172.21.15.37:3300/0 conn(0x7fad4c0b7cd0 0x7fad4c160a00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:29:49.229 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:49.226+0000 7fad50b93640 1 --2- >> v2:172.21.15.37:3300/0 conn(0x7fad4c0b7cd0 0x7fad4c160a00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.37:3300/0 says I am v2:172.21.15.37:59730/0 (socket says 172.21.15.37:59730) 2024-04-25T10:29:49.229 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:49.226+0000 7fad50b93640 1 -- 172.21.15.37:0/4249415053 learned_addr learned my addr 172.21.15.37:0/4249415053 (peer_addr_for_me v2:172.21.15.37:0/0) 2024-04-25T10:29:49.229 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:49.226+0000 7fad51b95640 1 --2- 172.21.15.37:0/4249415053 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7fad4c160f40 0x7fad4c161880 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:29:49.229 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:49.226+0000 7fad50b93640 1 -- 172.21.15.37:0/4249415053 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7fad4c160f40 msgr2=0x7fad4c161880 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:29:49.229 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:49.226+0000 7fad50b93640 1 --2- 172.21.15.37:0/4249415053 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7fad4c160f40 0x7fad4c161880 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:29:49.230 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:49.226+0000 7fad50b93640 1 -- 172.21.15.37:0/4249415053 >> v1:172.21.15.203:6789/0 conn(0x7fad4c0d5430 legacy=0x7fad4c0b75c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:29:49.230 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:49.226+0000 7fad50b93640 1 -- 172.21.15.37:0/4249415053 --> v2:172.21.15.37:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fad4c0d4950 con 0x7fad4c0b7cd0 2024-04-25T10:29:49.230 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:49.226+0000 7fad51b95640 1 --2- 172.21.15.37:0/4249415053 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7fad4c160f40 0x7fad4c161880 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T10:29:49.230 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:49.227+0000 7fad50b93640 1 --2- 172.21.15.37:0/4249415053 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fad4c0b7cd0 0x7fad4c160a00 secure :-1 s=READY pgs=291 cs=0 l=1 rev1=1 crypto rx=0x7fad3c009870 tx=0x7fad3c02a1c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=9fc2c4ead9139c59 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:29:49.230 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:49.227+0000 7fad43fff640 1 -- 172.21.15.37:0/4249415053 <== mon.0 v2:172.21.15.37:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fad3c037070 con 0x7fad4c0b7cd0 2024-04-25T10:29:49.230 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:49.227+0000 7fad43fff640 1 -- 172.21.15.37:0/4249415053 <== mon.0 v2:172.21.15.37:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fad3c032440 con 0x7fad4c0b7cd0 2024-04-25T10:29:49.230 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:49.227+0000 7fad43fff640 1 -- 172.21.15.37:0/4249415053 <== mon.0 v2:172.21.15.37:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fad3c032810 con 0x7fad4c0b7cd0 2024-04-25T10:29:49.231 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:49.227+0000 7fad535f8640 1 -- 172.21.15.37:0/4249415053 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fad4c0b7cd0 msgr2=0x7fad4c160a00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:29:49.231 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:49.227+0000 7fad535f8640 1 --2- 172.21.15.37:0/4249415053 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fad4c0b7cd0 0x7fad4c160a00 secure :-1 s=READY pgs=291 cs=0 l=1 rev1=1 crypto rx=0x7fad3c009870 tx=0x7fad3c02a1c0 comp rx=0 tx=0).stop 2024-04-25T10:29:49.231 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:49.228+0000 7fad535f8640 1 -- 172.21.15.37:0/4249415053 shutdown_connections 2024-04-25T10:29:49.231 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:49.228+0000 7fad535f8640 1 --2- 172.21.15.37:0/4249415053 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7fad4c160f40 0x7fad4c161880 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:29:49.231 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:49.228+0000 7fad535f8640 1 --2- 172.21.15.37:0/4249415053 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fad4c0b7cd0 0x7fad4c160a00 unknown :-1 s=CLOSED pgs=291 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:29:49.231 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:49.228+0000 7fad535f8640 1 -- 172.21.15.37:0/4249415053 >> 172.21.15.37:0/4249415053 conn(0x7fad4c0d27a0 msgr2=0x7fad4c0d2bb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:29:49.231 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:49.228+0000 7fad535f8640 1 -- 172.21.15.37:0/4249415053 shutdown_connections 2024-04-25T10:29:49.231 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:49.228+0000 7fad535f8640 1 -- 172.21.15.37:0/4249415053 wait complete. 2024-04-25T10:29:49.232 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:49.229+0000 7fad535f8640 1 Processor -- start 2024-04-25T10:29:49.232 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:49.230+0000 7fad535f8640 1 -- start start 2024-04-25T10:29:49.234 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:49.231+0000 7fad535f8640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fad4c0d5430 0x7fad4c0cb320 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:29:49.235 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:49.231+0000 7fad535f8640 1 --2- >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7fad4c160f40 0x7fad4c14dba0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:29:49.235 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:49.231+0000 7fad535f8640 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fad4c14e0e0 0x7fad4c0ba320 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:29:49.235 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:49.231+0000 7fad535f8640 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 -- 0x7fad4c161350 con 0x7fad4c0d5430 2024-04-25T10:29:49.235 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:49.231+0000 7fad535f8640 1 -- --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_getmap magic: 0 -- 0x7fad4c0d4f10 con 0x7fad4c14e0e0 2024-04-25T10:29:49.235 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:49.231+0000 7fad535f8640 1 -- --> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] -- mon_getmap magic: 0 -- 0x7fad4c1614d0 con 0x7fad4c160f40 2024-04-25T10:29:49.235 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:49.231+0000 7fad51394640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fad4c0d5430 0x7fad4c0cb320 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:29:49.235 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:49.231+0000 7fad51394640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fad4c0d5430 0x7fad4c0cb320 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.37:3300/0 says I am v2:172.21.15.37:59736/0 (socket says 172.21.15.37:59736) 2024-04-25T10:29:49.235 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:49.231+0000 7fad51394640 1 -- 172.21.15.37:0/2138191996 learned_addr learned my addr 172.21.15.37:0/2138191996 (peer_addr_for_me v2:172.21.15.37:0/0) 2024-04-25T10:29:49.235 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:49.231+0000 7fad50b93640 1 --2- 172.21.15.37:0/2138191996 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7fad4c160f40 0x7fad4c14dba0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:29:49.235 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:49.231+0000 7fad51b95640 1 --2- 172.21.15.37:0/2138191996 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fad4c14e0e0 0x7fad4c0ba320 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:29:49.235 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:49.231+0000 7fad51394640 1 -- 172.21.15.37:0/2138191996 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7fad4c160f40 msgr2=0x7fad4c14dba0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:29:49.235 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:49.231+0000 7fad51394640 1 --2- 172.21.15.37:0/2138191996 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7fad4c160f40 0x7fad4c14dba0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:29:49.235 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:49.231+0000 7fad51394640 1 -- 172.21.15.37:0/2138191996 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fad4c14e0e0 msgr2=0x7fad4c0ba320 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:29:49.236 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:49.231+0000 7fad51394640 1 --2- 172.21.15.37:0/2138191996 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fad4c14e0e0 0x7fad4c0ba320 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:29:49.236 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:49.231+0000 7fad51394640 1 -- 172.21.15.37:0/2138191996 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fad4c14e910 con 0x7fad4c0d5430 2024-04-25T10:29:49.236 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:49.231+0000 7fad50b93640 1 --2- 172.21.15.37:0/2138191996 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7fad4c160f40 0x7fad4c14dba0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T10:29:49.236 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:49.231+0000 7fad51394640 1 --2- 172.21.15.37:0/2138191996 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fad4c0d5430 0x7fad4c0cb320 secure :-1 s=READY pgs=292 cs=0 l=1 rev1=1 crypto rx=0x7fad4800c440 tx=0x7fad4800c900 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:29:49.236 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:49.232+0000 7fad41ffb640 1 -- 172.21.15.37:0/2138191996 <== mon.0 v2:172.21.15.37:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fad480052d0 con 0x7fad4c0d5430 2024-04-25T10:29:49.236 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:49.232+0000 7fad41ffb640 1 -- 172.21.15.37:0/2138191996 <== mon.0 v2:172.21.15.37:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fad48005470 con 0x7fad4c0d5430 2024-04-25T10:29:49.236 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:49.232+0000 7fad535f8640 1 -- 172.21.15.37:0/2138191996 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fad4c14bcf0 con 0x7fad4c0d5430 2024-04-25T10:29:49.236 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:49.232+0000 7fad41ffb640 1 -- 172.21.15.37:0/2138191996 <== mon.0 v2:172.21.15.37:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fad48007260 con 0x7fad4c0d5430 2024-04-25T10:29:49.236 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:49.232+0000 7fad535f8640 1 -- 172.21.15.37:0/2138191996 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fad4c14c2b0 con 0x7fad4c0d5430 2024-04-25T10:29:49.237 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:49.233+0000 7fad535f8640 1 -- 172.21.15.37:0/2138191996 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fad4c06a140 con 0x7fad4c0d5430 2024-04-25T10:29:49.237 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:49.233+0000 7fad41ffb640 1 -- 172.21.15.37:0/2138191996 <== mon.0 v2:172.21.15.37:3300/0 4 ==== mgrmap(e 5) ==== 96782+0+0 (secure 0 0 0) 0x7fad48007400 con 0x7fad4c0d5430 2024-04-25T10:29:49.237 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:49.234+0000 7fad41ffb640 1 --2- 172.21.15.37:0/2138191996 >> [v2:172.21.15.203:6832/2418205237,v1:172.21.15.203:6833/2418205237] conn(0x7fad300744e0 0x7fad300769f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:29:49.238 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:49.234+0000 7fad41ffb640 1 -- 172.21.15.37:0/2138191996 <== mon.0 v2:172.21.15.37:3300/0 5 ==== osd_map(24..24 src has 1..24) ==== 5738+0+0 (secure 0 0 0) 0x7fad480946d0 con 0x7fad4c0d5430 2024-04-25T10:29:49.238 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:49.234+0000 7fad50b93640 1 --2- 172.21.15.37:0/2138191996 >> [v2:172.21.15.203:6832/2418205237,v1:172.21.15.203:6833/2418205237] conn(0x7fad300744e0 0x7fad300769f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:29:49.238 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:49.235+0000 7fad50b93640 1 --2- 172.21.15.37:0/2138191996 >> [v2:172.21.15.203:6832/2418205237,v1:172.21.15.203:6833/2418205237] conn(0x7fad300744e0 0x7fad300769f0 secure :-1 s=READY pgs=154 cs=0 l=1 rev1=1 crypto rx=0x7fad3c034000 tx=0x7fad3c02e000 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:29:49.240 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:49.238+0000 7fad41ffb640 1 -- 172.21.15.37:0/2138191996 <== mon.0 v2:172.21.15.37:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187246 (secure 0 0 0) 0x7fad4805ee40 con 0x7fad4c0d5430 2024-04-25T10:29:49.429 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:49.427+0000 7fad535f8640 1 -- 172.21.15.37:0/2138191996 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7fad4c1630f0 con 0x7fad4c0d5430 2024-04-25T10:29:49.431 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:49.429+0000 7fad41ffb640 1 -- 172.21.15.37:0/2138191996 <== mon.0 v2:172.21.15.37:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v25) ==== 74+0+16670 (secure 0 0 0) 0x7fad4805e740 con 0x7fad4c0d5430 2024-04-25T10:29:49.431 INFO:teuthology.orchestra.run.smithi037.stdout: 2024-04-25T10:29:49.431 INFO:teuthology.orchestra.run.smithi037.stdout:{"epoch":25,"fsid":"f4acda28-be34-492a-b286-994afd36f61d","created":"2024-04-25T10:28:43.985718+0000","modified":"2024-04-25T10:29:49.325719+0000","last_up_change":"2024-04-25T10:28:54.900654+0000","last_in_change":"2024-04-25T10:28:46.216973+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":3,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"luminous","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-04-25T10:28:56.202153+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"19","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2024-04-25T10:29:06.507244+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"22","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":2.25,"score_stable":2.25,"optimal_score":1,"raw_score_acting":2.25,"raw_score_stable":2.25,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2024-04-25T10:29:07.347844+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"23","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"a2f79028-b332-4166-9ebb-a7024a37ea22","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":21,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.37:6816","nonce":2746074720},{"type":"v1","addr":"172.21.15.37:6817","nonce":2746074720}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.37:6818","nonce":2746074720},{"type":"v1","addr":"172.21.15.37:6819","nonce":2746074720}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.37:6822","nonce":2746074720},{"type":"v1","addr":"172.21.15.37:6823","nonce":2746074720}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.37:6820","nonce":2746074720},{"type":"v1","addr":"172.21.15.37:6821","nonce":2746074720}]},"public_addr":"172.21.15.37:6817/2746074720","cluster_addr":"172.21.15.37:6819/2746074720","heartbeat_back_addr":"172.21.15.37:6823/2746074720","heartbeat_front_addr":"172.21.15.37:6821/2746074720","state":["exists","up"]},{"osd":1,"uuid":"973159de-c2ea-4083-aecc-69373d1c1132","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":21,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.37:6800","nonce":1690341662},{"type":"v1","addr":"172.21.15.37:6801","nonce":1690341662}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.37:6802","nonce":1690341662},{"type":"v1","addr":"172.21.15.37:6803","nonce":1690341662}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.37:6806","nonce":1690341662},{"type":"v1","addr":"172.21.15.37:6807","nonce":1690341662}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.37:6804","nonce":1690341662},{"type":"v1","addr":"172.21.15.37:6805","nonce":1690341662}]},"public_addr":"172.21.15.37:6801/1690341662","cluster_addr":"172.21.15.37:6803/1690341662","heartbeat_back_addr":"172.21.15.37:6807/1690341662","heartbeat_front_addr":"172.21.15.37:6805/1690341662","state":["exists","up"]},{"osd":2,"uuid":"4ae7a0e9-cb3f-484c-a75f-f3766b721c47","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":21,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.37:6808","nonce":2754669649},{"type":"v1","addr":"172.21.15.37:6809","nonce":2754669649}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.37:6810","nonce":2754669649},{"type":"v1","addr":"172.21.15.37:6811","nonce":2754669649}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.37:6814","nonce":2754669649},{"type":"v1","addr":"172.21.15.37:6815","nonce":2754669649}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.37:6812","nonce":2754669649},{"type":"v1","addr":"172.21.15.37:6813","nonce":2754669649}]},"public_addr":"172.21.15.37:6809/2754669649","cluster_addr":"172.21.15.37:6811/2754669649","heartbeat_back_addr":"172.21.15.37:6815/2754669649","heartbeat_front_addr":"172.21.15.37:6813/2754669649","state":["exists","up"]},{"osd":3,"uuid":"1ba4a746-1608-47a7-ace2-960a5cfae934","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":21,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.37:6824","nonce":1815290618},{"type":"v1","addr":"172.21.15.37:6825","nonce":1815290618}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.37:6826","nonce":1815290618},{"type":"v1","addr":"172.21.15.37:6827","nonce":1815290618}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.37:6830","nonce":1815290618},{"type":"v1","addr":"172.21.15.37:6831","nonce":1815290618}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.37:6828","nonce":1815290618},{"type":"v1","addr":"172.21.15.37:6829","nonce":1815290618}]},"public_addr":"172.21.15.37:6825/1815290618","cluster_addr":"172.21.15.37:6827/1815290618","heartbeat_back_addr":"172.21.15.37:6831/1815290618","heartbeat_front_addr":"172.21.15.37:6829/1815290618","state":["exists","up"]},{"osd":4,"uuid":"d4f59f5d-0413-4b79-87ee-bb212c1fcf62","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":21,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6800","nonce":4051532063},{"type":"v1","addr":"172.21.15.203:6801","nonce":4051532063}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6802","nonce":4051532063},{"type":"v1","addr":"172.21.15.203:6803","nonce":4051532063}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6806","nonce":4051532063},{"type":"v1","addr":"172.21.15.203:6807","nonce":4051532063}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6804","nonce":4051532063},{"type":"v1","addr":"172.21.15.203:6805","nonce":4051532063}]},"public_addr":"172.21.15.203:6801/4051532063","cluster_addr":"172.21.15.203:6803/4051532063","heartbeat_back_addr":"172.21.15.203:6807/4051532063","heartbeat_front_addr":"172.21.15.203:6805/4051532063","state":["exists","up"]},{"osd":5,"uuid":"69710a0e-41d8-4e7f-9a10-17ce540b67be","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":24,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6824","nonce":1203822890},{"type":"v1","addr":"172.21.15.203:6825","nonce":1203822890}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6826","nonce":1203822890},{"type":"v1","addr":"172.21.15.203:6827","nonce":1203822890}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6830","nonce":1203822890},{"type":"v1","addr":"172.21.15.203:6831","nonce":1203822890}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6828","nonce":1203822890},{"type":"v1","addr":"172.21.15.203:6829","nonce":1203822890}]},"public_addr":"172.21.15.203:6825/1203822890","cluster_addr":"172.21.15.203:6827/1203822890","heartbeat_back_addr":"172.21.15.203:6831/1203822890","heartbeat_front_addr":"172.21.15.203:6829/1203822890","state":["exists","up"]},{"osd":6,"uuid":"f29dd047-3b48-473d-a235-6642a6293f20","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":21,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6810","nonce":2309615014},{"type":"v1","addr":"172.21.15.203:6812","nonce":2309615014}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6814","nonce":2309615014},{"type":"v1","addr":"172.21.15.203:6816","nonce":2309615014}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6822","nonce":2309615014},{"type":"v1","addr":"172.21.15.203:6823","nonce":2309615014}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6818","nonce":2309615014},{"type":"v1","addr":"172.21.15.203:6820","nonce":2309615014}]},"public_addr":"172.21.15.203:6812/2309615014","cluster_addr":"172.21.15.203:6816/2309615014","heartbeat_back_addr":"172.21.15.203:6823/2309615014","heartbeat_front_addr":"172.21.15.203:6820/2309615014","state":["exists","up"]},{"osd":7,"uuid":"6e691644-378b-4fdc-8e86-fb5f56e31f29","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":21,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6808","nonce":3688099014},{"type":"v1","addr":"172.21.15.203:6809","nonce":3688099014}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6811","nonce":3688099014},{"type":"v1","addr":"172.21.15.203:6813","nonce":3688099014}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6819","nonce":3688099014},{"type":"v1","addr":"172.21.15.203:6821","nonce":3688099014}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6815","nonce":3688099014},{"type":"v1","addr":"172.21.15.203:6817","nonce":3688099014}]},"public_addr":"172.21.15.203:6809/3688099014","cluster_addr":"172.21.15.203:6813/3688099014","heartbeat_back_addr":"172.21.15.203:6821/3688099014","heartbeat_front_addr":"172.21.15.203:6817/3688099014","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T10:28:51.419141+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T10:28:51.366448+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T10:28:51.348276+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T10:28:51.380519+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T10:28:51.105812+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T10:28:51.384496+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T10:28:51.388755+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T10:28:51.366958+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[{"pgid":"3.31","mappings":[{"from":0,"to":7}]}],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-04-25T10:29:49.433 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:49.431+0000 7fad535f8640 1 -- 172.21.15.37:0/2138191996 >> [v2:172.21.15.203:6832/2418205237,v1:172.21.15.203:6833/2418205237] conn(0x7fad300744e0 msgr2=0x7fad300769f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:29:49.433 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:49.431+0000 7fad535f8640 1 --2- 172.21.15.37:0/2138191996 >> [v2:172.21.15.203:6832/2418205237,v1:172.21.15.203:6833/2418205237] conn(0x7fad300744e0 0x7fad300769f0 secure :-1 s=READY pgs=154 cs=0 l=1 rev1=1 crypto rx=0x7fad3c034000 tx=0x7fad3c02e000 comp rx=0 tx=0).stop 2024-04-25T10:29:49.433 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:49.431+0000 7fad535f8640 1 -- 172.21.15.37:0/2138191996 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fad4c0d5430 msgr2=0x7fad4c0cb320 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:29:49.434 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:49.431+0000 7fad535f8640 1 --2- 172.21.15.37:0/2138191996 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fad4c0d5430 0x7fad4c0cb320 secure :-1 s=READY pgs=292 cs=0 l=1 rev1=1 crypto rx=0x7fad4800c440 tx=0x7fad4800c900 comp rx=0 tx=0).stop 2024-04-25T10:29:49.434 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:49.431+0000 7fad535f8640 1 -- 172.21.15.37:0/2138191996 shutdown_connections 2024-04-25T10:29:49.434 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:49.431+0000 7fad535f8640 1 --2- 172.21.15.37:0/2138191996 >> [v2:172.21.15.203:6832/2418205237,v1:172.21.15.203:6833/2418205237] conn(0x7fad300744e0 0x7fad300769f0 unknown :-1 s=CLOSED pgs=154 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:29:49.434 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:49.431+0000 7fad535f8640 1 --2- 172.21.15.37:0/2138191996 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fad4c14e0e0 0x7fad4c0ba320 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:29:49.434 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:49.431+0000 7fad535f8640 1 --2- 172.21.15.37:0/2138191996 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7fad4c160f40 0x7fad4c14dba0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:29:49.434 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:49.431+0000 7fad535f8640 1 --2- 172.21.15.37:0/2138191996 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fad4c0d5430 0x7fad4c0cb320 unknown :-1 s=CLOSED pgs=292 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:29:49.434 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:49.431+0000 7fad535f8640 1 -- 172.21.15.37:0/2138191996 >> 172.21.15.37:0/2138191996 conn(0x7fad4c0d27a0 msgr2=0x7fad4c1622a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:29:49.434 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:49.431+0000 7fad535f8640 1 -- 172.21.15.37:0/2138191996 shutdown_connections 2024-04-25T10:29:49.434 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:49.431+0000 7fad535f8640 1 -- 172.21.15.37:0/2138191996 wait complete. 2024-04-25T10:29:49.444 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2024-04-25T10:28:56.202153+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '19', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 8, 'score_stable': 8, 'optimal_score': 0.25, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 2, 'pool_name': 'cephfs_metadata', 'create_time': '2024-04-25T10:29:06.507244+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '22', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_autoscale_bias': 4, 'pg_num_min': 16, 'recovery_priority': 5}, 'application_metadata': {'cephfs': {'metadata': 'cephfs'}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 2.25, 'score_stable': 2.25, 'optimal_score': 1, 'raw_score_acting': 2.25, 'raw_score_stable': 2.25, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 3, 'pool_name': 'cephfs_data', 'create_time': '2024-04-25T10:29:07.347844+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 64, 'pg_placement_num': 64, 'pg_placement_num_target': 64, 'pg_num_target': 64, 'pg_num_pending': 64, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '23', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.8999999761581421}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 1.5, 'score_stable': 1.5, 'optimal_score': 1, 'raw_score_acting': 1.5, 'raw_score_stable': 1.5, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}] 2024-04-25T10:29:49.445 DEBUG:teuthology.orchestra.run.smithi037:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2024-04-25T10:29:49.559 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:49.557+0000 7fc12df72640 1 Processor -- start 2024-04-25T10:29:49.559 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:49.557+0000 7fc12df72640 1 -- start start 2024-04-25T10:29:49.562 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:49.558+0000 7fc12df72640 1 --2- >> v2:172.21.15.203:3300/0 conn(0x7fc1280d5160 0x7fc1280b4e80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:29:49.562 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:49.558+0000 7fc12df72640 1 --2- >> v2:172.21.15.37:3300/0 conn(0x7fc1280d0990 0x7fc1280d2b20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:29:49.562 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:49.558+0000 7fc12df72640 1 -- --> v1:172.21.15.203:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fc1280d1370 con 0x7fc1280d0da0 2024-04-25T10:29:49.562 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:49.558+0000 7fc12df72640 1 -- --> v2:172.21.15.37:3300/0 -- mon_getmap magic: 0 -- 0x7fc1280d1650 con 0x7fc1280d0990 2024-04-25T10:29:49.562 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:49.558+0000 7fc12df72640 1 -- --> v2:172.21.15.203:3300/0 -- mon_getmap magic: 0 -- 0x7fc1280d4680 con 0x7fc1280d5160 2024-04-25T10:29:49.562 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:49.559+0000 7fc126ffd640 1 --2- >> v2:172.21.15.37:3300/0 conn(0x7fc1280d0990 0x7fc1280d2b20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:29:49.562 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:49.559+0000 7fc126ffd640 1 --2- >> v2:172.21.15.37:3300/0 conn(0x7fc1280d0990 0x7fc1280d2b20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.37:3300/0 says I am v2:172.21.15.37:59752/0 (socket says 172.21.15.37:59752) 2024-04-25T10:29:49.562 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:49.559+0000 7fc126ffd640 1 -- 172.21.15.37:0/2825434201 learned_addr learned my addr 172.21.15.37:0/2825434201 (peer_addr_for_me v2:172.21.15.37:0/0) 2024-04-25T10:29:49.562 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:49.559+0000 7fc1277fe640 1 --2- 172.21.15.37:0/2825434201 >> v2:172.21.15.203:3300/0 conn(0x7fc1280d5160 0x7fc1280b4e80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:29:49.562 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:49.559+0000 7fc126ffd640 1 -- 172.21.15.37:0/2825434201 >> v2:172.21.15.203:3300/0 conn(0x7fc1280d5160 msgr2=0x7fc1280b4e80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:29:49.563 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:49.559+0000 7fc126ffd640 1 --2- 172.21.15.37:0/2825434201 >> v2:172.21.15.203:3300/0 conn(0x7fc1280d5160 0x7fc1280b4e80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:29:49.563 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:49.559+0000 7fc126ffd640 1 -- 172.21.15.37:0/2825434201 >> v1:172.21.15.203:6789/0 conn(0x7fc1280d0da0 legacy=0x7fc12806db10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:29:49.563 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:49.559+0000 7fc126ffd640 1 -- 172.21.15.37:0/2825434201 --> v2:172.21.15.37:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fc1280d4960 con 0x7fc1280d0990 2024-04-25T10:29:49.563 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:49.560+0000 7fc126ffd640 1 --2- 172.21.15.37:0/2825434201 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fc1280d0990 0x7fc1280d2b20 secure :-1 s=READY pgs=293 cs=0 l=1 rev1=1 crypto rx=0x7fc1280b86b0 tx=0x7fc11c02a150 comp rx=0 tx=0).ready entity=mon.0 client_cookie=6d6b0ff3ea1e3103 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:29:49.563 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:49.560+0000 7fc1267fc640 1 -- 172.21.15.37:0/2825434201 <== mon.0 v2:172.21.15.37:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc11c037070 con 0x7fc1280d0990 2024-04-25T10:29:49.563 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:49.560+0000 7fc1267fc640 1 -- 172.21.15.37:0/2825434201 <== mon.0 v2:172.21.15.37:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fc11c032440 con 0x7fc1280d0990 2024-04-25T10:29:49.563 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:49.560+0000 7fc1267fc640 1 -- 172.21.15.37:0/2825434201 <== mon.0 v2:172.21.15.37:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc11c032810 con 0x7fc1280d0990 2024-04-25T10:29:49.564 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:49.560+0000 7fc12df72640 1 -- 172.21.15.37:0/2825434201 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fc1280d0990 msgr2=0x7fc1280d2b20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:29:49.564 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:49.560+0000 7fc12df72640 1 --2- 172.21.15.37:0/2825434201 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fc1280d0990 0x7fc1280d2b20 secure :-1 s=READY pgs=293 cs=0 l=1 rev1=1 crypto rx=0x7fc1280b86b0 tx=0x7fc11c02a150 comp rx=0 tx=0).stop 2024-04-25T10:29:49.564 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:49.560+0000 7fc12df72640 1 -- 172.21.15.37:0/2825434201 shutdown_connections 2024-04-25T10:29:49.564 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:49.560+0000 7fc12df72640 1 --2- 172.21.15.37:0/2825434201 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fc1280d0990 0x7fc1280d2b20 unknown :-1 s=CLOSED pgs=293 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:29:49.564 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:49.560+0000 7fc12df72640 1 --2- 172.21.15.37:0/2825434201 >> v2:172.21.15.203:3300/0 conn(0x7fc1280d5160 0x7fc1280b4e80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:29:49.564 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:49.560+0000 7fc12df72640 1 -- 172.21.15.37:0/2825434201 >> 172.21.15.37:0/2825434201 conn(0x7fc1280b7a80 msgr2=0x7fc1280b7e90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:29:49.564 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:49.561+0000 7fc12df72640 1 -- 172.21.15.37:0/2825434201 shutdown_connections 2024-04-25T10:29:49.564 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:49.561+0000 7fc12df72640 1 -- 172.21.15.37:0/2825434201 wait complete. 2024-04-25T10:29:49.565 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:49.562+0000 7fc12df72640 1 Processor -- start 2024-04-25T10:29:49.565 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:49.563+0000 7fc12df72640 1 -- start start 2024-04-25T10:29:49.567 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:49.564+0000 7fc12df72640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fc1280d0990 0x7fc12815dae0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:29:49.567 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:49.564+0000 7fc12df72640 1 --2- >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7fc1280d0da0 0x7fc1280ac5a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:29:49.567 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:49.564+0000 7fc12df72640 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fc1280d5160 0x7fc1280acae0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:29:49.567 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:49.564+0000 7fc12df72640 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 -- 0x7fc128164c40 con 0x7fc1280d0990 2024-04-25T10:29:49.567 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:49.564+0000 7fc12df72640 1 -- --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_getmap magic: 0 -- 0x7fc128164ac0 con 0x7fc1280d5160 2024-04-25T10:29:49.568 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:49.564+0000 7fc12df72640 1 -- --> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] -- mon_getmap magic: 0 -- 0x7fc1280d3060 con 0x7fc1280d0da0 2024-04-25T10:29:49.568 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:49.564+0000 7fc1277fe640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fc1280d0990 0x7fc12815dae0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:29:49.568 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:49.564+0000 7fc1277fe640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fc1280d0990 0x7fc12815dae0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.37:3300/0 says I am v2:172.21.15.37:59766/0 (socket says 172.21.15.37:59766) 2024-04-25T10:29:49.568 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:49.564+0000 7fc1277fe640 1 -- 172.21.15.37:0/635689389 learned_addr learned my addr 172.21.15.37:0/635689389 (peer_addr_for_me v2:172.21.15.37:0/0) 2024-04-25T10:29:49.568 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:49.564+0000 7fc126ffd640 1 --2- 172.21.15.37:0/635689389 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7fc1280d0da0 0x7fc1280ac5a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:29:49.568 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:49.564+0000 7fc127fff640 1 --2- 172.21.15.37:0/635689389 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fc1280d5160 0x7fc1280acae0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:29:49.568 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:49.564+0000 7fc1277fe640 1 -- 172.21.15.37:0/635689389 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7fc1280d0da0 msgr2=0x7fc1280ac5a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:29:49.568 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:49.564+0000 7fc1277fe640 1 --2- 172.21.15.37:0/635689389 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7fc1280d0da0 0x7fc1280ac5a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:29:49.568 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:49.564+0000 7fc1277fe640 1 -- 172.21.15.37:0/635689389 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fc1280d5160 msgr2=0x7fc1280acae0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:29:49.568 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:49.564+0000 7fc1277fe640 1 --2- 172.21.15.37:0/635689389 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fc1280d5160 0x7fc1280acae0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:29:49.568 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:49.564+0000 7fc1277fe640 1 -- 172.21.15.37:0/635689389 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fc12815e020 con 0x7fc1280d0990 2024-04-25T10:29:49.568 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:49.564+0000 7fc1277fe640 1 --2- 172.21.15.37:0/635689389 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fc1280d0990 0x7fc12815dae0 secure :-1 s=READY pgs=294 cs=0 l=1 rev1=1 crypto rx=0x7fc11800d5d0 tx=0x7fc11800da90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:29:49.568 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:49.565+0000 7fc10ffff640 1 -- 172.21.15.37:0/635689389 <== mon.0 v2:172.21.15.37:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc1180125f0 con 0x7fc1280d0990 2024-04-25T10:29:49.569 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:49.565+0000 7fc10ffff640 1 -- 172.21.15.37:0/635689389 <== mon.0 v2:172.21.15.37:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fc118019070 con 0x7fc1280d0990 2024-04-25T10:29:49.569 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:49.565+0000 7fc12df72640 1 -- 172.21.15.37:0/635689389 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fc1281f8270 con 0x7fc1280d0990 2024-04-25T10:29:49.569 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:49.565+0000 7fc10ffff640 1 -- 172.21.15.37:0/635689389 <== mon.0 v2:172.21.15.37:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc1180129f0 con 0x7fc1280d0990 2024-04-25T10:29:49.569 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:49.565+0000 7fc12df72640 1 -- 172.21.15.37:0/635689389 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fc1281f8830 con 0x7fc1280d0990 2024-04-25T10:29:49.570 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:49.566+0000 7fc12df72640 1 -- 172.21.15.37:0/635689389 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fc12806a140 con 0x7fc1280d0990 2024-04-25T10:29:49.570 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:49.566+0000 7fc10ffff640 1 -- 172.21.15.37:0/635689389 <== mon.0 v2:172.21.15.37:3300/0 4 ==== mgrmap(e 5) ==== 96782+0+0 (secure 0 0 0) 0x7fc118012b90 con 0x7fc1280d0990 2024-04-25T10:29:49.570 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:49.567+0000 7fc10ffff640 1 --2- 172.21.15.37:0/635689389 >> [v2:172.21.15.203:6832/2418205237,v1:172.21.15.203:6833/2418205237] conn(0x7fc1140744b0 0x7fc1140769c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:29:49.570 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:49.567+0000 7fc10ffff640 1 -- 172.21.15.37:0/635689389 <== mon.0 v2:172.21.15.37:3300/0 5 ==== osd_map(25..25 src has 1..25) ==== 5738+0+0 (secure 0 0 0) 0x7fc118095600 con 0x7fc1280d0990 2024-04-25T10:29:49.570 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:49.567+0000 7fc126ffd640 1 --2- 172.21.15.37:0/635689389 >> [v2:172.21.15.203:6832/2418205237,v1:172.21.15.203:6833/2418205237] conn(0x7fc1140744b0 0x7fc1140769c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:29:49.570 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:49.568+0000 7fc126ffd640 1 --2- 172.21.15.37:0/635689389 >> [v2:172.21.15.203:6832/2418205237,v1:172.21.15.203:6833/2418205237] conn(0x7fc1140744b0 0x7fc1140769c0 secure :-1 s=READY pgs=155 cs=0 l=1 rev1=1 crypto rx=0x7fc11c034000 tx=0x7fc11c02e000 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:29:49.574 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:49.572+0000 7fc10ffff640 1 -- 172.21.15.37:0/635689389 <== mon.0 v2:172.21.15.37:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187246 (secure 0 0 0) 0x7fc11805fe90 con 0x7fc1280d0990 2024-04-25T10:29:49.759 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:49.757+0000 7fc12df72640 1 -- 172.21.15.37:0/635689389 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) -- 0x7fc1280d3f50 con 0x7fc1280d0990 2024-04-25T10:29:49.760 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:49.757+0000 7fc10ffff640 1 -- 172.21.15.37:0/635689389 <== mon.0 v2:172.21.15.37:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v25) ==== 93+0+10 (secure 0 0 0) 0x7fc11805f790 con 0x7fc1280d0990 2024-04-25T10:29:49.760 INFO:teuthology.orchestra.run.smithi037.stdout:pg_num: 1 2024-04-25T10:29:49.762 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:49.759+0000 7fc12df72640 1 -- 172.21.15.37:0/635689389 >> [v2:172.21.15.203:6832/2418205237,v1:172.21.15.203:6833/2418205237] conn(0x7fc1140744b0 msgr2=0x7fc1140769c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:29:49.762 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:49.759+0000 7fc12df72640 1 --2- 172.21.15.37:0/635689389 >> [v2:172.21.15.203:6832/2418205237,v1:172.21.15.203:6833/2418205237] conn(0x7fc1140744b0 0x7fc1140769c0 secure :-1 s=READY pgs=155 cs=0 l=1 rev1=1 crypto rx=0x7fc11c034000 tx=0x7fc11c02e000 comp rx=0 tx=0).stop 2024-04-25T10:29:49.762 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:49.759+0000 7fc12df72640 1 -- 172.21.15.37:0/635689389 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fc1280d0990 msgr2=0x7fc12815dae0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:29:49.762 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:49.759+0000 7fc12df72640 1 --2- 172.21.15.37:0/635689389 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fc1280d0990 0x7fc12815dae0 secure :-1 s=READY pgs=294 cs=0 l=1 rev1=1 crypto rx=0x7fc11800d5d0 tx=0x7fc11800da90 comp rx=0 tx=0).stop 2024-04-25T10:29:49.762 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:49.759+0000 7fc12df72640 1 -- 172.21.15.37:0/635689389 shutdown_connections 2024-04-25T10:29:49.762 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:49.759+0000 7fc12df72640 1 --2- 172.21.15.37:0/635689389 >> [v2:172.21.15.203:6832/2418205237,v1:172.21.15.203:6833/2418205237] conn(0x7fc1140744b0 0x7fc1140769c0 unknown :-1 s=CLOSED pgs=155 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:29:49.762 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:49.759+0000 7fc12df72640 1 --2- 172.21.15.37:0/635689389 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fc1280d5160 0x7fc1280acae0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:29:49.762 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:49.759+0000 7fc12df72640 1 --2- 172.21.15.37:0/635689389 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7fc1280d0da0 0x7fc1280ac5a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:29:49.762 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:49.759+0000 7fc12df72640 1 --2- 172.21.15.37:0/635689389 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fc1280d0990 0x7fc12815dae0 unknown :-1 s=CLOSED pgs=294 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:29:49.762 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:49.759+0000 7fc12df72640 1 -- 172.21.15.37:0/635689389 >> 172.21.15.37:0/635689389 conn(0x7fc1280b7a80 msgr2=0x7fc1281600d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:29:49.763 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:49.759+0000 7fc12df72640 1 -- 172.21.15.37:0/635689389 shutdown_connections 2024-04-25T10:29:49.763 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:49.759+0000 7fc12df72640 1 -- 172.21.15.37:0/635689389 wait complete. 2024-04-25T10:29:49.773 DEBUG:teuthology.orchestra.run.smithi037:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_metadata pg_num 2024-04-25T10:29:49.887 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:49.884+0000 7fb5753f3640 1 Processor -- start 2024-04-25T10:29:49.887 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:49.885+0000 7fb5753f3640 1 -- start start 2024-04-25T10:29:49.890 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:49.886+0000 7fb5753f3640 1 --2- >> v2:172.21.15.203:3300/0 conn(0x7fb57008cc90 0x7fb57009cbc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:29:49.890 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:49.886+0000 7fb5753f3640 1 --2- >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7fb5700b71b0 0x7fb5700b75c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:29:49.890 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:49.886+0000 7fb5753f3640 1 -- --> v1:172.21.15.37:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fb5700b8d40 con 0x7fb5700d1c20 2024-04-25T10:29:49.890 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:49.886+0000 7fb5753f3640 1 -- --> v2:172.21.15.203:3300/0 -- mon_getmap magic: 0 -- 0x7fb5700d27f0 con 0x7fb57008cc90 2024-04-25T10:29:49.891 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:49.886+0000 7fb5753f3640 1 -- --> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] -- mon_getmap magic: 0 -- 0x7fb5700d2ad0 con 0x7fb5700b71b0 2024-04-25T10:29:49.891 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:49.886+0000 7fb56e7fc640 1 --1- >> v1:172.21.15.37:6789/0 conn(0x7fb5700d1c20 0x7fb5700d2030 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.37:6789/0 says I am v1:172.21.15.37:56746/0 (socket says 172.21.15.37:56746) 2024-04-25T10:29:49.891 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:49.886+0000 7fb56e7fc640 1 -- 172.21.15.37:0/2902596076 learned_addr learned my addr 172.21.15.37:0/2902596076 (peer_addr_for_me v1:172.21.15.37:0/0) 2024-04-25T10:29:49.891 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:49.886+0000 7fb56effd640 1 --2- 172.21.15.37:0/2902596076 >> v2:172.21.15.203:3300/0 conn(0x7fb57008cc90 0x7fb57009cbc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:29:49.891 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:49.886+0000 7fb56f7fe640 1 --2- 172.21.15.37:0/2902596076 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7fb5700b71b0 0x7fb5700b75c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:29:49.891 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:49.886+0000 7fb56dffb640 1 -- 172.21.15.37:0/2902596076 <== mon.0 v1:172.21.15.37:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 3238530971 0 0) 0x7fb5700b8d40 con 0x7fb5700d1c20 2024-04-25T10:29:49.891 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:49.886+0000 7fb56dffb640 1 -- 172.21.15.37:0/2902596076 --> v1:172.21.15.37:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7fb558003520 con 0x7fb5700d1c20 2024-04-25T10:29:49.891 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:49.887+0000 7fb56dffb640 1 -- 172.21.15.37:0/2902596076 <== mon.0 v1:172.21.15.37:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 4206963387 0 0) 0x7fb558003520 con 0x7fb5700d1c20 2024-04-25T10:29:49.891 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:49.887+0000 7fb56dffb640 1 -- 172.21.15.37:0/2902596076 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7fb5700b71b0 msgr2=0x7fb5700b75c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:29:49.891 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:49.887+0000 7fb56dffb640 1 --2- 172.21.15.37:0/2902596076 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7fb5700b71b0 0x7fb5700b75c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:29:49.891 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:49.887+0000 7fb56dffb640 1 -- 172.21.15.37:0/2902596076 >> v2:172.21.15.203:3300/0 conn(0x7fb57008cc90 msgr2=0x7fb57009cbc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:29:49.891 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:49.887+0000 7fb56dffb640 1 --2- 172.21.15.37:0/2902596076 >> v2:172.21.15.203:3300/0 conn(0x7fb57008cc90 0x7fb57009cbc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:29:49.891 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:49.887+0000 7fb56dffb640 1 -- 172.21.15.37:0/2902596076 --> v1:172.21.15.37:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fb5700cf7a0 con 0x7fb5700d1c20 2024-04-25T10:29:49.891 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:49.887+0000 7fb56dffb640 1 -- 172.21.15.37:0/2902596076 <== mon.0 v1:172.21.15.37:6789/0 3 ==== mon_map magic: 0 ==== 413+0+0 (unknown 1577794600 0 0) 0x7fb564001670 con 0x7fb5700d1c20 2024-04-25T10:29:49.892 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:49.887+0000 7fb56effd640 1 --2- 172.21.15.37:0/2902596076 >> v2:172.21.15.203:3300/0 conn(0x7fb57008cc90 0x7fb57009cbc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-25T10:29:49.892 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:49.887+0000 7fb56dffb640 1 -- 172.21.15.37:0/2902596076 >> v1:172.21.15.37:6789/0 conn(0x7fb5700d1c20 legacy=0x7fb5700d2030 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:29:49.892 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:49.887+0000 7fb56dffb640 1 --2- 172.21.15.37:0/2902596076 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fb5580039b0 0x7fb558003e20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:29:49.892 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:49.887+0000 7fb56dffb640 1 --2- 172.21.15.37:0/2902596076 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7fb5700b71b0 0x7fb558004800 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:29:49.892 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:49.887+0000 7fb56dffb640 1 --2- 172.21.15.37:0/2902596076 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fb558004d40 0x7fb558005150 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:29:49.892 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:49.887+0000 7fb56dffb640 1 -- 172.21.15.37:0/2902596076 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 -- 0x7fb5700d27f0 con 0x7fb5580039b0 2024-04-25T10:29:49.892 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:49.887+0000 7fb56dffb640 1 -- 172.21.15.37:0/2902596076 --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_getmap magic: 0 -- 0x7fb5700d2ad0 con 0x7fb558004d40 2024-04-25T10:29:49.892 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:49.887+0000 7fb56dffb640 1 -- 172.21.15.37:0/2902596076 --> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] -- mon_getmap magic: 0 -- 0x7fb55800b9c0 con 0x7fb5700b71b0 2024-04-25T10:29:49.892 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:49.887+0000 7fb56effd640 1 --2- 172.21.15.37:0/2902596076 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fb5580039b0 0x7fb558003e20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:29:49.892 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:49.887+0000 7fb56e7fc640 1 --2- 172.21.15.37:0/2902596076 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7fb5700b71b0 0x7fb558004800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:29:49.892 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:49.887+0000 7fb56f7fe640 1 --2- 172.21.15.37:0/2902596076 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fb558004d40 0x7fb558005150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:29:49.892 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:49.887+0000 7fb56effd640 1 -- 172.21.15.37:0/2902596076 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7fb5700b71b0 msgr2=0x7fb558004800 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:29:49.892 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:49.887+0000 7fb56effd640 1 --2- 172.21.15.37:0/2902596076 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7fb5700b71b0 0x7fb558004800 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:29:49.892 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:49.887+0000 7fb56effd640 1 -- 172.21.15.37:0/2902596076 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fb558004d40 msgr2=0x7fb558005150 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:29:49.893 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:49.887+0000 7fb56effd640 1 --2- 172.21.15.37:0/2902596076 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fb558004d40 0x7fb558005150 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:29:49.893 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:49.887+0000 7fb56effd640 1 -- 172.21.15.37:0/2902596076 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fb558003520 con 0x7fb5580039b0 2024-04-25T10:29:49.893 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:49.887+0000 7fb56f7fe640 1 --2- 172.21.15.37:0/2902596076 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fb558004d40 0x7fb558005150 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T10:29:49.893 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:49.888+0000 7fb56effd640 1 --2- 172.21.15.37:0/2902596076 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fb5580039b0 0x7fb558003e20 secure :-1 s=READY pgs=296 cs=0 l=1 rev1=1 crypto rx=0x7fb57006d180 tx=0x7fb558005830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=6c89ef6ef566baaa server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:29:49.893 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:49.888+0000 7fb56dffb640 1 -- 172.21.15.37:0/2902596076 <== mon.0 v2:172.21.15.37:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb568009860 con 0x7fb5580039b0 2024-04-25T10:29:49.893 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:49.888+0000 7fb56dffb640 1 -- 172.21.15.37:0/2902596076 <== mon.0 v2:172.21.15.37:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fb568004020 con 0x7fb5580039b0 2024-04-25T10:29:49.893 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:49.888+0000 7fb56dffb640 1 -- 172.21.15.37:0/2902596076 <== mon.0 v2:172.21.15.37:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb5680043f0 con 0x7fb5580039b0 2024-04-25T10:29:49.896 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:49.889+0000 7fb5753f3640 1 -- 172.21.15.37:0/2902596076 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fb5580039b0 msgr2=0x7fb558003e20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:29:49.896 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:49.889+0000 7fb5753f3640 1 --2- 172.21.15.37:0/2902596076 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fb5580039b0 0x7fb558003e20 secure :-1 s=READY pgs=296 cs=0 l=1 rev1=1 crypto rx=0x7fb57006d180 tx=0x7fb558005830 comp rx=0 tx=0).stop 2024-04-25T10:29:49.896 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:49.889+0000 7fb56f7fe640 1 -- 172.21.15.37:0/2902596076 reap_dead start 2024-04-25T10:29:49.896 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:49.889+0000 7fb5753f3640 1 -- 172.21.15.37:0/2902596076 shutdown_connections 2024-04-25T10:29:49.896 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:49.889+0000 7fb5753f3640 1 -- 172.21.15.37:0/2902596076 >> 172.21.15.37:0/2902596076 conn(0x7fb5700d03d0 msgr2=0x7fb5700d07e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:29:49.896 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:49.889+0000 7fb5753f3640 1 -- 172.21.15.37:0/2902596076 shutdown_connections 2024-04-25T10:29:49.896 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:49.889+0000 7fb5753f3640 1 -- 172.21.15.37:0/2902596076 wait complete. 2024-04-25T10:29:49.896 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:49.891+0000 7fb5753f3640 1 Processor -- start 2024-04-25T10:29:49.896 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:49.891+0000 7fb5753f3640 1 -- start start 2024-04-25T10:29:49.896 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:49.892+0000 7fb5753f3640 1 --2- >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7fb5580039b0 0x7fb5701d65e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:29:49.896 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:49.892+0000 7fb5753f3640 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fb5701d9090 0x7fb5701d5bd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:29:49.896 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:49.892+0000 7fb5753f3640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fb5701f3c40 0x7fb5701f4050 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:29:49.896 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:49.892+0000 7fb5753f3640 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 -- 0x7fb57009d100 con 0x7fb5701f3c40 2024-04-25T10:29:49.896 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:49.892+0000 7fb5753f3640 1 -- --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_getmap magic: 0 -- 0x7fb5700b93b0 con 0x7fb5701d9090 2024-04-25T10:29:49.897 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:49.892+0000 7fb5753f3640 1 -- --> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] -- mon_getmap magic: 0 -- 0x7fb57009d280 con 0x7fb5580039b0 2024-04-25T10:29:49.897 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:49.892+0000 7fb56f7fe640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fb5701f3c40 0x7fb5701f4050 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:29:49.897 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:49.892+0000 7fb56f7fe640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fb5701f3c40 0x7fb5701f4050 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.37:3300/0 says I am v2:172.21.15.37:36216/0 (socket says 172.21.15.37:36216) 2024-04-25T10:29:49.897 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:49.892+0000 7fb56f7fe640 1 -- 172.21.15.37:0/4120552760 learned_addr learned my addr 172.21.15.37:0/4120552760 (peer_addr_for_me v2:172.21.15.37:0/0) 2024-04-25T10:29:49.897 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:49.893+0000 7fb56effd640 1 --2- 172.21.15.37:0/4120552760 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7fb5580039b0 0x7fb5701d65e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:29:49.897 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:49.893+0000 7fb56e7fc640 1 --2- 172.21.15.37:0/4120552760 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fb5701d9090 0x7fb5701d5bd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:29:49.897 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:49.893+0000 7fb56f7fe640 1 -- 172.21.15.37:0/4120552760 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7fb5580039b0 msgr2=0x7fb5701d65e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:29:49.897 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:49.893+0000 7fb56f7fe640 1 --2- 172.21.15.37:0/4120552760 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7fb5580039b0 0x7fb5701d65e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:29:49.897 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:49.893+0000 7fb56f7fe640 1 -- 172.21.15.37:0/4120552760 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fb5701d9090 msgr2=0x7fb5701d5bd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:29:49.897 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:49.893+0000 7fb56f7fe640 1 --2- 172.21.15.37:0/4120552760 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fb5701d9090 0x7fb5701d5bd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:29:49.897 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:49.893+0000 7fb56f7fe640 1 -- 172.21.15.37:0/4120552760 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fb5701f7200 con 0x7fb5701f3c40 2024-04-25T10:29:49.897 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:49.893+0000 7fb56f7fe640 1 --2- 172.21.15.37:0/4120552760 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fb5701f3c40 0x7fb5701f4050 secure :-1 s=READY pgs=297 cs=0 l=1 rev1=1 crypto rx=0x7fb5700742d0 tx=0x7fb560010590 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:29:49.897 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:49.893+0000 7fb5577fe640 1 -- 172.21.15.37:0/4120552760 <== mon.0 v2:172.21.15.37:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb560010d80 con 0x7fb5701f3c40 2024-04-25T10:29:49.897 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:49.893+0000 7fb5577fe640 1 -- 172.21.15.37:0/4120552760 <== mon.0 v2:172.21.15.37:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fb560003990 con 0x7fb5701f3c40 2024-04-25T10:29:49.898 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:49.893+0000 7fb5577fe640 1 -- 172.21.15.37:0/4120552760 <== mon.0 v2:172.21.15.37:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb560003dc0 con 0x7fb5701f3c40 2024-04-25T10:29:49.898 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:49.893+0000 7fb5753f3640 1 -- 172.21.15.37:0/4120552760 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fb5701f74e0 con 0x7fb5701f3c40 2024-04-25T10:29:49.898 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:49.893+0000 7fb5753f3640 1 -- 172.21.15.37:0/4120552760 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fb5701d6c20 con 0x7fb5701f3c40 2024-04-25T10:29:49.899 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:49.894+0000 7fb5753f3640 1 -- 172.21.15.37:0/4120552760 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fb5701f6f70 con 0x7fb5701f3c40 2024-04-25T10:29:49.899 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:49.895+0000 7fb5577fe640 1 -- 172.21.15.37:0/4120552760 <== mon.0 v2:172.21.15.37:3300/0 4 ==== mgrmap(e 5) ==== 96782+0+0 (secure 0 0 0) 0x7fb560005d10 con 0x7fb5701f3c40 2024-04-25T10:29:49.899 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:49.895+0000 7fb5577fe640 1 --2- 172.21.15.37:0/4120552760 >> [v2:172.21.15.203:6832/2418205237,v1:172.21.15.203:6833/2418205237] conn(0x7fb55c0744b0 0x7fb55c0769c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:29:49.899 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:49.895+0000 7fb5577fe640 1 -- 172.21.15.37:0/4120552760 <== mon.0 v2:172.21.15.37:3300/0 5 ==== osd_map(25..25 src has 1..25) ==== 5738+0+0 (secure 0 0 0) 0x7fb560095900 con 0x7fb5701f3c40 2024-04-25T10:29:49.899 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:49.896+0000 7fb56effd640 1 --2- 172.21.15.37:0/4120552760 >> [v2:172.21.15.203:6832/2418205237,v1:172.21.15.203:6833/2418205237] conn(0x7fb55c0744b0 0x7fb55c0769c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:29:49.899 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:49.896+0000 7fb56effd640 1 --2- 172.21.15.37:0/4120552760 >> [v2:172.21.15.203:6832/2418205237,v1:172.21.15.203:6833/2418205237] conn(0x7fb55c0744b0 0x7fb55c0769c0 secure :-1 s=READY pgs=156 cs=0 l=1 rev1=1 crypto rx=0x7fb568032000 tx=0x7fb568008030 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:29:49.902 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:49.900+0000 7fb5577fe640 1 -- 172.21.15.37:0/4120552760 <== mon.0 v2:172.21.15.37:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187246 (secure 0 0 0) 0x7fb560025020 con 0x7fb5701f3c40 2024-04-25T10:29:50.087 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:50.084+0000 7fb5753f3640 1 -- 172.21.15.37:0/4120552760 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"} v 0) -- 0x7fb5701d6e80 con 0x7fb5701f3c40 2024-04-25T10:29:50.087 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:50.085+0000 7fb5577fe640 1 -- 172.21.15.37:0/4120552760 <== mon.0 v2:172.21.15.37:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]=0 v25) ==== 104+0+11 (secure 0 0 0) 0x7fb560060100 con 0x7fb5701f3c40 2024-04-25T10:29:50.087 INFO:teuthology.orchestra.run.smithi037.stdout:pg_num: 32 2024-04-25T10:29:50.089 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:50.087+0000 7fb5753f3640 1 -- 172.21.15.37:0/4120552760 >> [v2:172.21.15.203:6832/2418205237,v1:172.21.15.203:6833/2418205237] conn(0x7fb55c0744b0 msgr2=0x7fb55c0769c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:29:50.089 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:50.087+0000 7fb5753f3640 1 --2- 172.21.15.37:0/4120552760 >> [v2:172.21.15.203:6832/2418205237,v1:172.21.15.203:6833/2418205237] conn(0x7fb55c0744b0 0x7fb55c0769c0 secure :-1 s=READY pgs=156 cs=0 l=1 rev1=1 crypto rx=0x7fb568032000 tx=0x7fb568008030 comp rx=0 tx=0).stop 2024-04-25T10:29:50.089 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:50.087+0000 7fb5753f3640 1 -- 172.21.15.37:0/4120552760 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fb5701f3c40 msgr2=0x7fb5701f4050 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:29:50.090 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:50.087+0000 7fb5753f3640 1 --2- 172.21.15.37:0/4120552760 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fb5701f3c40 0x7fb5701f4050 secure :-1 s=READY pgs=297 cs=0 l=1 rev1=1 crypto rx=0x7fb5700742d0 tx=0x7fb560010590 comp rx=0 tx=0).stop 2024-04-25T10:29:50.090 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:50.087+0000 7fb5753f3640 1 -- 172.21.15.37:0/4120552760 shutdown_connections 2024-04-25T10:29:50.090 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:50.087+0000 7fb5753f3640 1 --2- 172.21.15.37:0/4120552760 >> [v2:172.21.15.203:6832/2418205237,v1:172.21.15.203:6833/2418205237] conn(0x7fb55c0744b0 0x7fb55c0769c0 unknown :-1 s=CLOSED pgs=156 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:29:50.090 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:50.087+0000 7fb5753f3640 1 --2- 172.21.15.37:0/4120552760 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fb5701f3c40 0x7fb5701f4050 unknown :-1 s=CLOSED pgs=297 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:29:50.090 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:50.087+0000 7fb5753f3640 1 --2- 172.21.15.37:0/4120552760 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fb5701d9090 0x7fb5701d5bd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:29:50.090 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:50.087+0000 7fb5753f3640 1 --2- 172.21.15.37:0/4120552760 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7fb5580039b0 0x7fb5701d65e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:29:50.090 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:50.087+0000 7fb5753f3640 1 -- 172.21.15.37:0/4120552760 >> 172.21.15.37:0/4120552760 conn(0x7fb5700d03d0 msgr2=0x7fb5700d2030 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:29:50.090 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:50.087+0000 7fb5753f3640 1 -- 172.21.15.37:0/4120552760 shutdown_connections 2024-04-25T10:29:50.090 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:50.087+0000 7fb5753f3640 1 -- 172.21.15.37:0/4120552760 wait complete. 2024-04-25T10:29:50.100 DEBUG:teuthology.orchestra.run.smithi037:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data pg_num 2024-04-25T10:29:50.214 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:50.212+0000 7fe31764a640 1 Processor -- start 2024-04-25T10:29:50.214 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:50.212+0000 7fe31764a640 1 -- start start 2024-04-25T10:29:50.216 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:50.213+0000 7fe31764a640 1 --2- >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7fe31008cc90 0x7fe31009cbc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:29:50.216 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:50.213+0000 7fe31764a640 1 --2- >> v2:172.21.15.203:3300/0 conn(0x7fe3100b71b0 0x7fe3100b75c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:29:50.216 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:50.213+0000 7fe31764a640 1 -- --> v1:172.21.15.203:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fe3100b8d40 con 0x7fe3100d1c20 2024-04-25T10:29:50.216 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:50.213+0000 7fe31764a640 1 -- --> v2:172.21.15.203:3300/0 -- mon_getmap magic: 0 -- 0x7fe3100d27f0 con 0x7fe3100b71b0 2024-04-25T10:29:50.216 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:50.213+0000 7fe31764a640 1 -- --> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] -- mon_getmap magic: 0 -- 0x7fe3100d2ad0 con 0x7fe31008cc90 2024-04-25T10:29:50.216 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:50.214+0000 7fe315be7640 1 --2- >> v2:172.21.15.203:3300/0 conn(0x7fe3100b71b0 0x7fe3100b75c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:29:50.217 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:50.214+0000 7fe315be7640 1 --2- >> v2:172.21.15.203:3300/0 conn(0x7fe3100b71b0 0x7fe3100b75c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.203:3300/0 says I am v2:172.21.15.37:33338/0 (socket says 172.21.15.37:33338) 2024-04-25T10:29:50.217 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:50.214+0000 7fe3153e6640 1 --2- >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7fe31008cc90 0x7fe31009cbc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:29:50.217 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:50.214+0000 7fe314be5640 1 --1- >> v1:172.21.15.203:6789/0 conn(0x7fe3100d1c20 0x7fe3100d2030 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.203:6789/0 says I am v1:172.21.15.37:52616/0 (socket says 172.21.15.37:52616) 2024-04-25T10:29:50.217 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:50.214+0000 7fe315be7640 1 -- 172.21.15.37:0/3431406068 learned_addr learned my addr 172.21.15.37:0/3431406068 (peer_addr_for_me v2:172.21.15.37:0/0) 2024-04-25T10:29:50.217 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:50.214+0000 7fe315be7640 1 -- 172.21.15.37:0/3431406068 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7fe31008cc90 msgr2=0x7fe31009cbc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:29:50.217 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:50.214+0000 7fe315be7640 1 --2- 172.21.15.37:0/3431406068 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7fe31008cc90 0x7fe31009cbc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:29:50.217 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:50.214+0000 7fe315be7640 1 -- 172.21.15.37:0/3431406068 >> v1:172.21.15.203:6789/0 conn(0x7fe3100d1c20 legacy=0x7fe3100d2030 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:29:50.217 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:50.214+0000 7fe315be7640 1 -- 172.21.15.37:0/3431406068 --> v2:172.21.15.203:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fe3100cf7a0 con 0x7fe3100b71b0 2024-04-25T10:29:50.217 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:50.214+0000 7fe307fff640 1 -- 172.21.15.37:0/3431406068 <== mon.1 v1:172.21.15.203:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 4074121412 0 0) 0x7fe3100b8d40 con 0x7fe3100d1c20 2024-04-25T10:29:50.218 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:50.215+0000 7fe315be7640 1 --2- 172.21.15.37:0/3431406068 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fe3100b71b0 0x7fe3100b75c0 secure :-1 s=READY pgs=135 cs=0 l=1 rev1=1 crypto rx=0x7fe308009e50 tx=0x7fe30802a710 comp rx=0 tx=0).ready entity=mon.1 client_cookie=74f2c367082bf0fa server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:29:50.219 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:50.215+0000 7fe307fff640 1 -- 172.21.15.37:0/3431406068 <== mon.1 v2:172.21.15.203:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe308036070 con 0x7fe3100b71b0 2024-04-25T10:29:50.219 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:50.215+0000 7fe307fff640 1 -- 172.21.15.37:0/3431406068 <== mon.1 v2:172.21.15.203:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fe308031600 con 0x7fe3100b71b0 2024-04-25T10:29:50.219 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:50.215+0000 7fe307fff640 1 -- 172.21.15.37:0/3431406068 <== mon.1 v2:172.21.15.203:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe3080319d0 con 0x7fe3100b71b0 2024-04-25T10:29:50.219 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:50.216+0000 7fe31764a640 1 -- 172.21.15.37:0/3431406068 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fe3100b71b0 msgr2=0x7fe3100b75c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:29:50.219 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:50.216+0000 7fe31764a640 1 --2- 172.21.15.37:0/3431406068 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fe3100b71b0 0x7fe3100b75c0 secure :-1 s=READY pgs=135 cs=0 l=1 rev1=1 crypto rx=0x7fe308009e50 tx=0x7fe30802a710 comp rx=0 tx=0).stop 2024-04-25T10:29:50.219 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:50.216+0000 7fe31764a640 1 -- 172.21.15.37:0/3431406068 shutdown_connections 2024-04-25T10:29:50.219 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:50.216+0000 7fe31764a640 1 --2- 172.21.15.37:0/3431406068 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fe3100b71b0 0x7fe3100b75c0 unknown :-1 s=CLOSED pgs=135 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:29:50.219 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:50.216+0000 7fe31764a640 1 --2- 172.21.15.37:0/3431406068 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7fe31008cc90 0x7fe31009cbc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:29:50.219 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:50.216+0000 7fe31764a640 1 -- 172.21.15.37:0/3431406068 >> 172.21.15.37:0/3431406068 conn(0x7fe3100d03d0 msgr2=0x7fe3100d07e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:29:50.219 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:50.216+0000 7fe31764a640 1 -- 172.21.15.37:0/3431406068 shutdown_connections 2024-04-25T10:29:50.219 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:50.216+0000 7fe31764a640 1 -- 172.21.15.37:0/3431406068 wait complete. 2024-04-25T10:29:50.220 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:50.218+0000 7fe31764a640 1 Processor -- start 2024-04-25T10:29:50.220 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:50.218+0000 7fe31764a640 1 -- start start 2024-04-25T10:29:50.223 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:50.219+0000 7fe31764a640 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fe3100b71b0 0x7fe310146b40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:29:50.224 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:50.219+0000 7fe31764a640 1 --2- >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7fe3100d1c20 0x7fe310144ab0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:29:50.224 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:50.219+0000 7fe31764a640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fe3101f6660 0x7fe310161ad0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:29:50.224 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:50.219+0000 7fe31764a640 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 -- 0x7fe3100b7b00 con 0x7fe3101f6660 2024-04-25T10:29:50.224 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:50.219+0000 7fe31764a640 1 -- --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_getmap magic: 0 -- 0x7fe3100b93b0 con 0x7fe3100b71b0 2024-04-25T10:29:50.224 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:50.219+0000 7fe31764a640 1 -- --> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] -- mon_getmap magic: 0 -- 0x7fe3100b7c80 con 0x7fe3100d1c20 2024-04-25T10:29:50.224 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:50.219+0000 7fe315be7640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fe3101f6660 0x7fe310161ad0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:29:50.224 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:50.219+0000 7fe315be7640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fe3101f6660 0x7fe310161ad0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.37:3300/0 says I am v2:172.21.15.37:36222/0 (socket says 172.21.15.37:36222) 2024-04-25T10:29:50.224 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:50.219+0000 7fe315be7640 1 -- 172.21.15.37:0/195658645 learned_addr learned my addr 172.21.15.37:0/195658645 (peer_addr_for_me v2:172.21.15.37:0/0) 2024-04-25T10:29:50.224 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:50.219+0000 7fe3153e6640 1 --2- 172.21.15.37:0/195658645 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fe3100b71b0 0x7fe310146b40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:29:50.224 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:50.219+0000 7fe314be5640 1 --2- 172.21.15.37:0/195658645 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7fe3100d1c20 0x7fe310144ab0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:29:50.224 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:50.220+0000 7fe315be7640 1 -- 172.21.15.37:0/195658645 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7fe3100d1c20 msgr2=0x7fe310144ab0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:29:50.224 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:50.220+0000 7fe315be7640 1 --2- 172.21.15.37:0/195658645 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7fe3100d1c20 0x7fe310144ab0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:29:50.225 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:50.220+0000 7fe315be7640 1 -- 172.21.15.37:0/195658645 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fe3100b71b0 msgr2=0x7fe310146b40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:29:50.225 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:50.220+0000 7fe315be7640 1 --2- 172.21.15.37:0/195658645 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fe3100b71b0 0x7fe310146b40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:29:50.225 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:50.220+0000 7fe315be7640 1 -- 172.21.15.37:0/195658645 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fe310147400 con 0x7fe3101f6660 2024-04-25T10:29:50.225 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:50.220+0000 7fe315be7640 1 --2- 172.21.15.37:0/195658645 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fe3101f6660 0x7fe310161ad0 secure :-1 s=READY pgs=298 cs=0 l=1 rev1=1 crypto rx=0x7fe308009e50 tx=0x7fe308004a40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:29:50.225 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:50.220+0000 7fe305ffb640 1 -- 172.21.15.37:0/195658645 <== mon.0 v2:172.21.15.37:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe308041020 con 0x7fe3101f6660 2024-04-25T10:29:50.225 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:50.220+0000 7fe305ffb640 1 -- 172.21.15.37:0/195658645 <== mon.0 v2:172.21.15.37:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fe3080077a0 con 0x7fe3101f6660 2024-04-25T10:29:50.225 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:50.220+0000 7fe31764a640 1 -- 172.21.15.37:0/195658645 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fe3101476e0 con 0x7fe3101f6660 2024-04-25T10:29:50.225 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:50.220+0000 7fe305ffb640 1 -- 172.21.15.37:0/195658645 <== mon.0 v2:172.21.15.37:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe308036040 con 0x7fe3101f6660 2024-04-25T10:29:50.225 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:50.220+0000 7fe31764a640 1 -- 172.21.15.37:0/195658645 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fe3101f5e60 con 0x7fe3101f6660 2024-04-25T10:29:50.225 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:50.221+0000 7fe31764a640 1 -- 172.21.15.37:0/195658645 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fe31006a140 con 0x7fe3101f6660 2024-04-25T10:29:50.226 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:50.222+0000 7fe305ffb640 1 -- 172.21.15.37:0/195658645 <== mon.0 v2:172.21.15.37:3300/0 4 ==== mgrmap(e 5) ==== 96782+0+0 (secure 0 0 0) 0x7fe308044020 con 0x7fe3101f6660 2024-04-25T10:29:50.226 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:50.222+0000 7fe305ffb640 1 --2- 172.21.15.37:0/195658645 >> [v2:172.21.15.203:6832/2418205237,v1:172.21.15.203:6833/2418205237] conn(0x7fe2f40744b0 0x7fe2f40769c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:29:50.226 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:50.223+0000 7fe305ffb640 1 -- 172.21.15.37:0/195658645 <== mon.0 v2:172.21.15.37:3300/0 5 ==== osd_map(25..25 src has 1..25) ==== 5738+0+0 (secure 0 0 0) 0x7fe3080b5e70 con 0x7fe3101f6660 2024-04-25T10:29:50.226 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:50.223+0000 7fe3153e6640 1 --2- 172.21.15.37:0/195658645 >> [v2:172.21.15.203:6832/2418205237,v1:172.21.15.203:6833/2418205237] conn(0x7fe2f40744b0 0x7fe2f40769c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:29:50.227 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:50.223+0000 7fe3153e6640 1 --2- 172.21.15.37:0/195658645 >> [v2:172.21.15.203:6832/2418205237,v1:172.21.15.203:6833/2418205237] conn(0x7fe2f40744b0 0x7fe2f40769c0 secure :-1 s=READY pgs=157 cs=0 l=1 rev1=1 crypto rx=0x7fe310070e30 tx=0x7fe30c009190 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:29:50.229 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:50.227+0000 7fe305ffb640 1 -- 172.21.15.37:0/195658645 <== mon.0 v2:172.21.15.37:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187246 (secure 0 0 0) 0x7fe30804b030 con 0x7fe3101f6660 2024-04-25T10:29:50.413 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:50.411+0000 7fe31764a640 1 -- 172.21.15.37:0/195658645 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"} v 0) -- 0x7fe310147150 con 0x7fe3101f6660 2024-04-25T10:29:50.414 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:50.412+0000 7fe305ffb640 1 -- 172.21.15.37:0/195658645 <== mon.0 v2:172.21.15.37:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]=0 v25) ==== 100+0+11 (secure 0 0 0) 0x7fe30807fe70 con 0x7fe3101f6660 2024-04-25T10:29:50.414 INFO:teuthology.orchestra.run.smithi037.stdout:pg_num: 64 2024-04-25T10:29:50.416 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:50.413+0000 7fe31764a640 1 -- 172.21.15.37:0/195658645 >> [v2:172.21.15.203:6832/2418205237,v1:172.21.15.203:6833/2418205237] conn(0x7fe2f40744b0 msgr2=0x7fe2f40769c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:29:50.416 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:50.413+0000 7fe31764a640 1 --2- 172.21.15.37:0/195658645 >> [v2:172.21.15.203:6832/2418205237,v1:172.21.15.203:6833/2418205237] conn(0x7fe2f40744b0 0x7fe2f40769c0 secure :-1 s=READY pgs=157 cs=0 l=1 rev1=1 crypto rx=0x7fe310070e30 tx=0x7fe30c009190 comp rx=0 tx=0).stop 2024-04-25T10:29:50.416 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:50.414+0000 7fe31764a640 1 -- 172.21.15.37:0/195658645 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fe3101f6660 msgr2=0x7fe310161ad0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:29:50.416 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:50.414+0000 7fe31764a640 1 --2- 172.21.15.37:0/195658645 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fe3101f6660 0x7fe310161ad0 secure :-1 s=READY pgs=298 cs=0 l=1 rev1=1 crypto rx=0x7fe308009e50 tx=0x7fe308004a40 comp rx=0 tx=0).stop 2024-04-25T10:29:50.416 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:50.414+0000 7fe31764a640 1 -- 172.21.15.37:0/195658645 shutdown_connections 2024-04-25T10:29:50.416 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:50.414+0000 7fe31764a640 1 --2- 172.21.15.37:0/195658645 >> [v2:172.21.15.203:6832/2418205237,v1:172.21.15.203:6833/2418205237] conn(0x7fe2f40744b0 0x7fe2f40769c0 unknown :-1 s=CLOSED pgs=157 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:29:50.417 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:50.414+0000 7fe31764a640 1 --2- 172.21.15.37:0/195658645 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fe3101f6660 0x7fe310161ad0 unknown :-1 s=CLOSED pgs=298 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:29:50.417 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:50.414+0000 7fe31764a640 1 --2- 172.21.15.37:0/195658645 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7fe3100d1c20 0x7fe310144ab0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:29:50.417 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:50.414+0000 7fe31764a640 1 --2- 172.21.15.37:0/195658645 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fe3100b71b0 0x7fe310146b40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:29:50.417 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:50.414+0000 7fe31764a640 1 -- 172.21.15.37:0/195658645 >> 172.21.15.37:0/195658645 conn(0x7fe3100d03d0 msgr2=0x7fe31009cb30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:29:50.417 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:50.414+0000 7fe31764a640 1 -- 172.21.15.37:0/195658645 shutdown_connections 2024-04-25T10:29:50.417 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:50.414+0000 7fe31764a640 1 -- 172.21.15.37:0/195658645 wait complete. 2024-04-25T10:29:50.427 DEBUG:teuthology.orchestra.run.smithi037:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd blocklist ls 2024-04-25T10:29:50.540 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:50.537+0000 7f3f199b2640 1 Processor -- start 2024-04-25T10:29:50.540 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:50.537+0000 7f3f199b2640 1 -- start start 2024-04-25T10:29:50.546 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:50.539+0000 7f3f199b2640 1 --2- >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f3f14144490 0x7f3f1415f760 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:29:50.546 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:50.539+0000 7f3f199b2640 1 -- --> v1:172.21.15.37:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f3f140d0e90 con 0x7f3f1406c540 2024-04-25T10:29:50.546 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:50.539+0000 7f3f199b2640 1 -- --> v1:172.21.15.203:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f3f1415eb80 con 0x7f3f1415e570 2024-04-25T10:29:50.546 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:50.539+0000 7f3f199b2640 1 -- --> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] -- mon_getmap magic: 0 -- 0x7f3f1415ee60 con 0x7f3f14144490 2024-04-25T10:29:50.546 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:50.539+0000 7f3f137fe640 1 --1- >> v1:172.21.15.37:6789/0 conn(0x7f3f1406c540 0x7f3f1406c950 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.37:6789/0 says I am v1:172.21.15.37:56762/0 (socket says 172.21.15.37:56762) 2024-04-25T10:29:50.547 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:50.539+0000 7f3f137fe640 1 -- 172.21.15.37:0/3689971852 learned_addr learned my addr 172.21.15.37:0/3689971852 (peer_addr_for_me v1:172.21.15.37:0/0) 2024-04-25T10:29:50.547 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:50.539+0000 7f3f12ffd640 1 --2- 172.21.15.37:0/3689971852 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f3f14144490 0x7f3f1415f760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:29:50.547 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:50.539+0000 7f3f11ffb640 1 -- 172.21.15.37:0/3689971852 <== mon.0 v1:172.21.15.37:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 3919679680 0 0) 0x7f3f0c001110 con 0x7f3f1406c540 2024-04-25T10:29:50.547 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:50.539+0000 7f3f11ffb640 1 -- 172.21.15.37:0/3689971852 --> v1:172.21.15.37:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f3f00003520 con 0x7f3f1406c540 2024-04-25T10:29:50.547 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:50.539+0000 7f3f11ffb640 1 -- 172.21.15.37:0/3689971852 <== mon.1 v1:172.21.15.203:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 343852259 0 0) 0x7f3f1415eb80 con 0x7f3f1415e570 2024-04-25T10:29:50.547 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:50.539+0000 7f3f11ffb640 1 -- 172.21.15.37:0/3689971852 --> v1:172.21.15.203:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f3f0c001110 con 0x7f3f1415e570 2024-04-25T10:29:50.548 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:50.539+0000 7f3f12ffd640 1 -- 172.21.15.37:0/3689971852 >> v1:172.21.15.203:6789/0 conn(0x7f3f1415e570 legacy=0x7f3f140d0780 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:29:50.548 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:50.539+0000 7f3f12ffd640 1 -- 172.21.15.37:0/3689971852 >> v1:172.21.15.37:6789/0 conn(0x7f3f1406c540 legacy=0x7f3f1406c950 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:29:50.548 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:50.539+0000 7f3f12ffd640 1 -- 172.21.15.37:0/3689971852 --> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f3f1415f140 con 0x7f3f14144490 2024-04-25T10:29:50.548 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:50.540+0000 7f3f12ffd640 1 --2- 172.21.15.37:0/3689971852 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f3f14144490 0x7f3f1415f760 secure :-1 s=READY pgs=45 cs=0 l=1 rev1=1 crypto rx=0x7f3f04009dc0 tx=0x7f3f0402a6d0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=32fd7d5af46c84e8 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:29:50.548 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:50.540+0000 7f3f11ffb640 1 -- 172.21.15.37:0/3689971852 <== mon.2 v2:172.21.15.203:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3f04007740 con 0x7f3f14144490 2024-04-25T10:29:50.548 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:50.540+0000 7f3f11ffb640 1 -- 172.21.15.37:0/3689971852 <== mon.2 v2:172.21.15.203:3301/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f3f040078e0 con 0x7f3f14144490 2024-04-25T10:29:50.549 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:50.540+0000 7f3f11ffb640 1 -- 172.21.15.37:0/3689971852 <== mon.2 v2:172.21.15.203:3301/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3f04007c90 con 0x7f3f14144490 2024-04-25T10:29:50.549 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:50.541+0000 7f3f199b2640 1 -- 172.21.15.37:0/3689971852 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f3f14144490 msgr2=0x7f3f1415f760 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:29:50.549 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:50.541+0000 7f3f199b2640 1 --2- 172.21.15.37:0/3689971852 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f3f14144490 0x7f3f1415f760 secure :-1 s=READY pgs=45 cs=0 l=1 rev1=1 crypto rx=0x7f3f04009dc0 tx=0x7f3f0402a6d0 comp rx=0 tx=0).stop 2024-04-25T10:29:50.549 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:50.541+0000 7f3f199b2640 1 -- 172.21.15.37:0/3689971852 shutdown_connections 2024-04-25T10:29:50.549 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:50.541+0000 7f3f199b2640 1 --2- 172.21.15.37:0/3689971852 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f3f14144490 0x7f3f1415f760 unknown :-1 s=CLOSED pgs=45 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:29:50.550 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:50.541+0000 7f3f199b2640 1 -- 172.21.15.37:0/3689971852 >> 172.21.15.37:0/3689971852 conn(0x7f3f140b4560 msgr2=0x7f3f140b4970 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:29:50.550 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:50.541+0000 7f3f199b2640 1 -- 172.21.15.37:0/3689971852 shutdown_connections 2024-04-25T10:29:50.550 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:50.541+0000 7f3f199b2640 1 -- 172.21.15.37:0/3689971852 wait complete. 2024-04-25T10:29:50.550 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:50.543+0000 7f3f199b2640 1 Processor -- start 2024-04-25T10:29:50.550 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:50.543+0000 7f3f199b2640 1 -- start start 2024-04-25T10:29:50.555 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:50.544+0000 7f3f199b2640 1 --2- >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f3f1406c540 0x7f3f1413f160 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:29:50.555 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:50.544+0000 7f3f199b2640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f3f14144490 0x7f3f141e85d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:29:50.556 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:50.544+0000 7f3f199b2640 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f3f1415e570 0x7f3f141e8b10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:29:50.556 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:50.544+0000 7f3f199b2640 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 -- 0x7f3f140d2470 con 0x7f3f14144490 2024-04-25T10:29:50.556 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:50.544+0000 7f3f199b2640 1 -- --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_getmap magic: 0 -- 0x7f3f140d22f0 con 0x7f3f1415e570 2024-04-25T10:29:50.556 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:50.544+0000 7f3f199b2640 1 -- --> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] -- mon_getmap magic: 0 -- 0x7f3f140d28d0 con 0x7f3f1406c540 2024-04-25T10:29:50.556 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:50.545+0000 7f3f127fc640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f3f14144490 0x7f3f141e85d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:29:50.557 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:50.545+0000 7f3f127fc640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f3f14144490 0x7f3f141e85d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.37:3300/0 says I am v2:172.21.15.37:36236/0 (socket says 172.21.15.37:36236) 2024-04-25T10:29:50.557 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:50.545+0000 7f3f127fc640 1 -- 172.21.15.37:0/2846487213 learned_addr learned my addr 172.21.15.37:0/2846487213 (peer_addr_for_me v2:172.21.15.37:0/0) 2024-04-25T10:29:50.557 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:50.545+0000 7f3f137fe640 1 --2- 172.21.15.37:0/2846487213 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f3f1415e570 0x7f3f141e8b10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:29:50.557 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:50.545+0000 7f3f12ffd640 1 --2- 172.21.15.37:0/2846487213 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f3f1406c540 0x7f3f1413f160 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:29:50.557 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:50.545+0000 7f3f127fc640 1 -- 172.21.15.37:0/2846487213 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f3f1406c540 msgr2=0x7f3f1413f160 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:29:50.557 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:50.545+0000 7f3f127fc640 1 --2- 172.21.15.37:0/2846487213 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f3f1406c540 0x7f3f1413f160 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:29:50.558 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:50.545+0000 7f3f127fc640 1 -- 172.21.15.37:0/2846487213 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f3f1415e570 msgr2=0x7f3f141e8b10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:29:50.558 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:50.545+0000 7f3f127fc640 1 --2- 172.21.15.37:0/2846487213 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f3f1415e570 0x7f3f141e8b10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:29:50.558 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:50.545+0000 7f3f127fc640 1 -- 172.21.15.37:0/2846487213 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f3f141e7a90 con 0x7f3f14144490 2024-04-25T10:29:50.558 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:50.545+0000 7f3f137fe640 1 --2- 172.21.15.37:0/2846487213 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f3f1415e570 0x7f3f141e8b10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T10:29:50.558 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:50.545+0000 7f3f127fc640 1 --2- 172.21.15.37:0/2846487213 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f3f14144490 0x7f3f141e85d0 secure :-1 s=READY pgs=300 cs=0 l=1 rev1=1 crypto rx=0x7f3efc00c3f0 tx=0x7f3efc001e70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:29:50.558 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:50.545+0000 7f3f0bfff640 1 -- 172.21.15.37:0/2846487213 <== mon.0 v2:172.21.15.37:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3efc017070 con 0x7f3f14144490 2024-04-25T10:29:50.558 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:50.545+0000 7f3f199b2640 1 -- 172.21.15.37:0/2846487213 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f3f1413e680 con 0x7f3f14144490 2024-04-25T10:29:50.558 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:50.545+0000 7f3f0bfff640 1 -- 172.21.15.37:0/2846487213 <== mon.0 v2:172.21.15.37:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f3efc0052f0 con 0x7f3f14144490 2024-04-25T10:29:50.559 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:50.545+0000 7f3f0bfff640 1 -- 172.21.15.37:0/2846487213 <== mon.0 v2:172.21.15.37:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3efc003830 con 0x7f3f14144490 2024-04-25T10:29:50.559 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:50.545+0000 7f3f199b2640 1 -- 172.21.15.37:0/2846487213 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f3f141f55c0 con 0x7f3f14144490 2024-04-25T10:29:50.559 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:50.546+0000 7f3f199b2640 1 -- 172.21.15.37:0/2846487213 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f3f1413e480 con 0x7f3f14144490 2024-04-25T10:29:50.559 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:50.547+0000 7f3f0bfff640 1 -- 172.21.15.37:0/2846487213 <== mon.0 v2:172.21.15.37:3300/0 4 ==== mgrmap(e 5) ==== 96782+0+0 (secure 0 0 0) 0x7f3efc01a070 con 0x7f3f14144490 2024-04-25T10:29:50.559 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:50.548+0000 7f3f0bfff640 1 --2- 172.21.15.37:0/2846487213 >> [v2:172.21.15.203:6832/2418205237,v1:172.21.15.203:6833/2418205237] conn(0x7f3ef40744b0 0x7f3ef40769c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:29:50.559 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:50.548+0000 7f3f0bfff640 1 -- 172.21.15.37:0/2846487213 <== mon.0 v2:172.21.15.37:3300/0 5 ==== osd_map(25..25 src has 1..25) ==== 5738+0+0 (secure 0 0 0) 0x7f3efc0946c0 con 0x7f3f14144490 2024-04-25T10:29:50.560 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:50.548+0000 7f3f12ffd640 1 --2- 172.21.15.37:0/2846487213 >> [v2:172.21.15.203:6832/2418205237,v1:172.21.15.203:6833/2418205237] conn(0x7f3ef40744b0 0x7f3ef40769c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:29:50.560 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:50.549+0000 7f3f12ffd640 1 --2- 172.21.15.37:0/2846487213 >> [v2:172.21.15.203:6832/2418205237,v1:172.21.15.203:6833/2418205237] conn(0x7f3ef40744b0 0x7f3ef40769c0 secure :-1 s=READY pgs=158 cs=0 l=1 rev1=1 crypto rx=0x7f3f04034000 tx=0x7f3f0402f000 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:29:50.560 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:50.552+0000 7f3f0bfff640 1 -- 172.21.15.37:0/2846487213 <== mon.0 v2:172.21.15.37:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187246 (secure 0 0 0) 0x7f3efc05ed10 con 0x7f3f14144490 2024-04-25T10:29:50.751 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:50.748+0000 7f3f199b2640 1 -- 172.21.15.37:0/2846487213 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_command({"prefix": "osd blocklist ls"} v 0) -- 0x7f3f1406a140 con 0x7f3f14144490 2024-04-25T10:29:50.751 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:50.749+0000 7f3f0bfff640 1 -- 172.21.15.37:0/2846487213 <== mon.0 v2:172.21.15.37:3300/0 7 ==== mon_command_ack([{"prefix": "osd blocklist ls"}]=0 listed 0 entries v25) ==== 80+0+0 (secure 0 0 0) 0x7f3efc05e610 con 0x7f3f14144490 2024-04-25T10:29:50.751 INFO:teuthology.orchestra.run.smithi037.stderr:listed 0 entries 2024-04-25T10:29:50.753 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:50.751+0000 7f3f199b2640 1 -- 172.21.15.37:0/2846487213 >> [v2:172.21.15.203:6832/2418205237,v1:172.21.15.203:6833/2418205237] conn(0x7f3ef40744b0 msgr2=0x7f3ef40769c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:29:50.753 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:50.751+0000 7f3f199b2640 1 --2- 172.21.15.37:0/2846487213 >> [v2:172.21.15.203:6832/2418205237,v1:172.21.15.203:6833/2418205237] conn(0x7f3ef40744b0 0x7f3ef40769c0 secure :-1 s=READY pgs=158 cs=0 l=1 rev1=1 crypto rx=0x7f3f04034000 tx=0x7f3f0402f000 comp rx=0 tx=0).stop 2024-04-25T10:29:50.753 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:50.751+0000 7f3f199b2640 1 -- 172.21.15.37:0/2846487213 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f3f14144490 msgr2=0x7f3f141e85d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:29:50.753 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:50.751+0000 7f3f199b2640 1 --2- 172.21.15.37:0/2846487213 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f3f14144490 0x7f3f141e85d0 secure :-1 s=READY pgs=300 cs=0 l=1 rev1=1 crypto rx=0x7f3efc00c3f0 tx=0x7f3efc001e70 comp rx=0 tx=0).stop 2024-04-25T10:29:50.754 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:50.751+0000 7f3f199b2640 1 -- 172.21.15.37:0/2846487213 shutdown_connections 2024-04-25T10:29:50.754 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:50.751+0000 7f3f199b2640 1 --2- 172.21.15.37:0/2846487213 >> [v2:172.21.15.203:6832/2418205237,v1:172.21.15.203:6833/2418205237] conn(0x7f3ef40744b0 0x7f3ef40769c0 unknown :-1 s=CLOSED pgs=158 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:29:50.754 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:50.751+0000 7f3f199b2640 1 --2- 172.21.15.37:0/2846487213 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f3f1415e570 0x7f3f141e8b10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:29:50.754 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:50.751+0000 7f3f199b2640 1 --2- 172.21.15.37:0/2846487213 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f3f14144490 0x7f3f141e85d0 unknown :-1 s=CLOSED pgs=300 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:29:50.754 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:50.751+0000 7f3f199b2640 1 --2- 172.21.15.37:0/2846487213 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f3f1406c540 0x7f3f1413f160 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:29:50.754 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:50.751+0000 7f3f199b2640 1 -- 172.21.15.37:0/2846487213 >> 172.21.15.37:0/2846487213 conn(0x7f3f140b4560 msgr2=0x7f3f1415f760 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:29:50.754 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:50.751+0000 7f3f199b2640 1 -- 172.21.15.37:0/2846487213 shutdown_connections 2024-04-25T10:29:50.754 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:50.751+0000 7f3f199b2640 1 -- 172.21.15.37:0/2846487213 wait complete. 2024-04-25T10:29:50.764 INFO:tasks.cephfs.fuse_mount:Running fusermount -u on ubuntu@smithi203.front.sepia.ceph.com... 2024-04-25T10:29:50.764 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T10:29:50.764 DEBUG:teuthology.orchestra.run.smithi203:> sudo fusermount -u /home/ubuntu/cephtest/mnt.1 2024-04-25T10:29:50.794 INFO:teuthology.orchestra.run:waiting for 300 2024-04-25T10:29:50.794 INFO:tasks.cephfs.mount:Cleaning up mount ubuntu@smithi203.front.sepia.ceph.com 2024-04-25T10:29:50.794 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T10:29:50.795 DEBUG:teuthology.orchestra.run.smithi203:> (cd /home/ubuntu/cephtest && exec rmdir -- /home/ubuntu/cephtest/mnt.1) 2024-04-25T10:29:50.860 DEBUG:teuthology.orchestra.run.smithi037:> set -ex 2024-04-25T10:29:50.860 DEBUG:teuthology.orchestra.run.smithi037:> dd if=/proc/self/mounts of=/dev/stdout 2024-04-25T10:29:50.876 DEBUG:teuthology.orchestra.run.smithi037:> set -ex 2024-04-25T10:29:50.876 DEBUG:teuthology.orchestra.run.smithi037:> dd if=/proc/self/mounts of=/dev/stdout 2024-04-25T10:29:50.932 DEBUG:teuthology.orchestra.run.smithi037:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-04-25T10:29:51.084 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:51.082+0000 7f34a9572640 1 Processor -- start 2024-04-25T10:29:51.084 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:51.082+0000 7f34a9572640 1 -- start start 2024-04-25T10:29:51.086 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:51.083+0000 7f34a9572640 1 --2- >> v2:172.21.15.37:3300/0 conn(0x7f34a408cc90 0x7f34a409cbc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:29:51.086 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:51.083+0000 7f34a9572640 1 -- --> v1:172.21.15.37:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f34a40b8d40 con 0x7f34a40b71b0 2024-04-25T10:29:51.086 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:51.083+0000 7f34a9572640 1 -- --> v1:172.21.15.203:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f34a40d27f0 con 0x7f34a40d1c20 2024-04-25T10:29:51.086 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:51.083+0000 7f34a9572640 1 -- --> v2:172.21.15.37:3300/0 -- mon_getmap magic: 0 -- 0x7f34a40d2ad0 con 0x7f34a408cc90 2024-04-25T10:29:51.086 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:51.083+0000 7f34a2ffd640 1 --2- >> v2:172.21.15.37:3300/0 conn(0x7f34a408cc90 0x7f34a409cbc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:29:51.086 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:51.083+0000 7f34a37fe640 1 --1- >> v1:172.21.15.37:6789/0 conn(0x7f34a40b71b0 0x7f34a40b75c0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.37:6789/0 says I am v1:172.21.15.37:56776/0 (socket says 172.21.15.37:56776) 2024-04-25T10:29:51.086 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:51.083+0000 7f34a37fe640 1 -- 172.21.15.37:0/4281415775 learned_addr learned my addr 172.21.15.37:0/4281415775 (peer_addr_for_me v1:172.21.15.37:0/0) 2024-04-25T10:29:51.086 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:51.083+0000 7f34a2ffd640 1 -- 172.21.15.37:0/4281415775 >> v1:172.21.15.203:6789/0 conn(0x7f34a40d1c20 legacy=0x7f34a40d2030 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:29:51.086 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:51.083+0000 7f34a2ffd640 1 -- 172.21.15.37:0/4281415775 >> v1:172.21.15.37:6789/0 conn(0x7f34a40b71b0 legacy=0x7f34a40b75c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:29:51.086 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:51.083+0000 7f34a2ffd640 1 -- 172.21.15.37:0/4281415775 --> v2:172.21.15.37:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f34a40cf7a0 con 0x7f34a408cc90 2024-04-25T10:29:51.088 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:51.084+0000 7f34a2ffd640 1 --2- 172.21.15.37:0/4281415775 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f34a408cc90 0x7f34a409cbc0 secure :-1 s=READY pgs=302 cs=0 l=1 rev1=1 crypto rx=0x7f349c0097a0 tx=0x7f349c02a0d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d07c54afe349f138 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:29:51.088 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:51.084+0000 7f34a1ffb640 1 -- 172.21.15.37:0/4281415775 <== mon.0 v2:172.21.15.37:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f349c037070 con 0x7f34a408cc90 2024-04-25T10:29:51.088 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:51.084+0000 7f34a1ffb640 1 -- 172.21.15.37:0/4281415775 <== mon.0 v2:172.21.15.37:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f349c02ad60 con 0x7f34a408cc90 2024-04-25T10:29:51.088 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:51.084+0000 7f34a1ffb640 1 -- 172.21.15.37:0/4281415775 <== mon.0 v2:172.21.15.37:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f349c032660 con 0x7f34a408cc90 2024-04-25T10:29:51.088 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:51.085+0000 7f34a9572640 1 -- 172.21.15.37:0/4281415775 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f34a408cc90 msgr2=0x7f34a409cbc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:29:51.088 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:51.085+0000 7f34a9572640 1 --2- 172.21.15.37:0/4281415775 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f34a408cc90 0x7f34a409cbc0 secure :-1 s=READY pgs=302 cs=0 l=1 rev1=1 crypto rx=0x7f349c0097a0 tx=0x7f349c02a0d0 comp rx=0 tx=0).stop 2024-04-25T10:29:51.088 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:51.085+0000 7f34a9572640 1 -- 172.21.15.37:0/4281415775 shutdown_connections 2024-04-25T10:29:51.088 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:51.085+0000 7f34a9572640 1 --2- 172.21.15.37:0/4281415775 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f34a408cc90 0x7f34a409cbc0 unknown :-1 s=CLOSED pgs=302 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:29:51.088 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:51.085+0000 7f34a9572640 1 -- 172.21.15.37:0/4281415775 >> 172.21.15.37:0/4281415775 conn(0x7f34a40d03d0 msgr2=0x7f34a40d07e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:29:51.088 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:51.085+0000 7f34a9572640 1 -- 172.21.15.37:0/4281415775 shutdown_connections 2024-04-25T10:29:51.088 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:51.085+0000 7f34a9572640 1 -- 172.21.15.37:0/4281415775 wait complete. 2024-04-25T10:29:51.089 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:51.087+0000 7f34a9572640 1 Processor -- start 2024-04-25T10:29:51.089 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:51.087+0000 7f34a9572640 1 -- start start 2024-04-25T10:29:51.092 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:51.088+0000 7f34a9572640 1 --2- >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f34a40b71b0 0x7f34a41d6d50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:29:51.092 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:51.088+0000 7f34a9572640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f34a40d1c20 0x7f34a41e7b30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:29:51.092 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:51.088+0000 7f34a9572640 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f34a41e8070 0x7f34a40a0bd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:29:51.092 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:51.088+0000 7f34a9572640 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 -- 0x7f34a409d100 con 0x7f34a40d1c20 2024-04-25T10:29:51.092 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:51.088+0000 7f34a9572640 1 -- --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_getmap magic: 0 -- 0x7f34a40b93b0 con 0x7f34a41e8070 2024-04-25T10:29:51.092 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:51.088+0000 7f34a9572640 1 -- --> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] -- mon_getmap magic: 0 -- 0x7f34a409d280 con 0x7f34a40b71b0 2024-04-25T10:29:51.092 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:51.088+0000 7f34a27fc640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f34a40d1c20 0x7f34a41e7b30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:29:51.092 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:51.088+0000 7f34a27fc640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f34a40d1c20 0x7f34a41e7b30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.37:3300/0 says I am v2:172.21.15.37:36260/0 (socket says 172.21.15.37:36260) 2024-04-25T10:29:51.093 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:51.089+0000 7f34a27fc640 1 -- 172.21.15.37:0/2719626196 learned_addr learned my addr 172.21.15.37:0/2719626196 (peer_addr_for_me v2:172.21.15.37:0/0) 2024-04-25T10:29:51.093 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:51.089+0000 7f34a37fe640 1 --2- 172.21.15.37:0/2719626196 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f34a41e8070 0x7f34a40a0bd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:29:51.093 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:51.089+0000 7f34a27fc640 1 -- 172.21.15.37:0/2719626196 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f34a40b71b0 msgr2=0x7f34a41d6d50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:29:51.093 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:51.089+0000 7f34a27fc640 1 --2- 172.21.15.37:0/2719626196 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f34a40b71b0 0x7f34a41d6d50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:29:51.093 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:51.089+0000 7f34a27fc640 1 -- 172.21.15.37:0/2719626196 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f34a41e8070 msgr2=0x7f34a40a0bd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:29:51.093 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:51.089+0000 7f34a27fc640 1 --2- 172.21.15.37:0/2719626196 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f34a41e8070 0x7f34a40a0bd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:29:51.093 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:51.089+0000 7f34a27fc640 1 -- 172.21.15.37:0/2719626196 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f34a409d680 con 0x7f34a40d1c20 2024-04-25T10:29:51.093 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:51.089+0000 7f34a27fc640 1 --2- 172.21.15.37:0/2719626196 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f34a40d1c20 0x7f34a41e7b30 secure :-1 s=READY pgs=303 cs=0 l=1 rev1=1 crypto rx=0x7f348c00b3e0 tx=0x7f348c00b8a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:29:51.093 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:51.089+0000 7f349b7fe640 1 -- 172.21.15.37:0/2719626196 <== mon.0 v2:172.21.15.37:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f348c015020 con 0x7f34a40d1c20 2024-04-25T10:29:51.093 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:51.089+0000 7f349b7fe640 1 -- 172.21.15.37:0/2719626196 <== mon.0 v2:172.21.15.37:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f348c011830 con 0x7f34a40d1c20 2024-04-25T10:29:51.093 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:51.089+0000 7f34a9572640 1 -- 172.21.15.37:0/2719626196 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f34a41d7a20 con 0x7f34a40d1c20 2024-04-25T10:29:51.093 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:51.089+0000 7f349b7fe640 1 -- 172.21.15.37:0/2719626196 <== mon.0 v2:172.21.15.37:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f348c011c20 con 0x7f34a40d1c20 2024-04-25T10:29:51.093 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:51.089+0000 7f34a9572640 1 -- 172.21.15.37:0/2719626196 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f34a41d7fe0 con 0x7f34a40d1c20 2024-04-25T10:29:51.093 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:51.090+0000 7f34a9572640 1 -- 172.21.15.37:0/2719626196 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f34a406a140 con 0x7f34a40d1c20 2024-04-25T10:29:51.094 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:51.091+0000 7f349b7fe640 1 -- 172.21.15.37:0/2719626196 <== mon.0 v2:172.21.15.37:3300/0 4 ==== mgrmap(e 5) ==== 96782+0+0 (secure 0 0 0) 0x7f348c018020 con 0x7f34a40d1c20 2024-04-25T10:29:51.094 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:51.091+0000 7f349b7fe640 1 --2- 172.21.15.37:0/2719626196 >> [v2:172.21.15.203:6832/2418205237,v1:172.21.15.203:6833/2418205237] conn(0x7f34840744b0 0x7f34840769c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:29:51.094 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:51.092+0000 7f349b7fe640 1 -- 172.21.15.37:0/2719626196 <== mon.0 v2:172.21.15.37:3300/0 5 ==== osd_map(25..25 src has 1..25) ==== 5738+0+0 (secure 0 0 0) 0x7f348c095280 con 0x7f34a40d1c20 2024-04-25T10:29:51.094 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:51.092+0000 7f34a2ffd640 1 --2- 172.21.15.37:0/2719626196 >> [v2:172.21.15.203:6832/2418205237,v1:172.21.15.203:6833/2418205237] conn(0x7f34840744b0 0x7f34840769c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:29:51.094 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:51.092+0000 7f34a2ffd640 1 --2- 172.21.15.37:0/2719626196 >> [v2:172.21.15.203:6832/2418205237,v1:172.21.15.203:6833/2418205237] conn(0x7f34840744b0 0x7f34840769c0 secure :-1 s=READY pgs=159 cs=0 l=1 rev1=1 crypto rx=0x7f349c034000 tx=0x7f349c02c000 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:29:51.098 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:51.096+0000 7f349b7fe640 1 -- 172.21.15.37:0/2719626196 <== mon.0 v2:172.21.15.37:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187246 (secure 0 0 0) 0x7f348c01f020 con 0x7f34a40d1c20 2024-04-25T10:29:51.285 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:51.283+0000 7f34a9572640 1 -- 172.21.15.37:0/2719626196 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7f34a40b86b0 con 0x7f34a40d1c20 2024-04-25T10:29:51.287 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:51.284+0000 7f349b7fe640 1 -- 172.21.15.37:0/2719626196 <== mon.0 v2:172.21.15.37:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v25) ==== 74+0+16670 (secure 0 0 0) 0x7f348c05f8d0 con 0x7f34a40d1c20 2024-04-25T10:29:51.287 INFO:teuthology.orchestra.run.smithi037.stdout: 2024-04-25T10:29:51.287 INFO:teuthology.orchestra.run.smithi037.stdout:{"epoch":25,"fsid":"f4acda28-be34-492a-b286-994afd36f61d","created":"2024-04-25T10:28:43.985718+0000","modified":"2024-04-25T10:29:49.325719+0000","last_up_change":"2024-04-25T10:28:54.900654+0000","last_in_change":"2024-04-25T10:28:46.216973+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":3,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"luminous","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-04-25T10:28:56.202153+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"19","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2024-04-25T10:29:06.507244+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"22","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":2.25,"score_stable":2.25,"optimal_score":1,"raw_score_acting":2.25,"raw_score_stable":2.25,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2024-04-25T10:29:07.347844+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"23","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"a2f79028-b332-4166-9ebb-a7024a37ea22","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":21,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.37:6816","nonce":2746074720},{"type":"v1","addr":"172.21.15.37:6817","nonce":2746074720}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.37:6818","nonce":2746074720},{"type":"v1","addr":"172.21.15.37:6819","nonce":2746074720}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.37:6822","nonce":2746074720},{"type":"v1","addr":"172.21.15.37:6823","nonce":2746074720}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.37:6820","nonce":2746074720},{"type":"v1","addr":"172.21.15.37:6821","nonce":2746074720}]},"public_addr":"172.21.15.37:6817/2746074720","cluster_addr":"172.21.15.37:6819/2746074720","heartbeat_back_addr":"172.21.15.37:6823/2746074720","heartbeat_front_addr":"172.21.15.37:6821/2746074720","state":["exists","up"]},{"osd":1,"uuid":"973159de-c2ea-4083-aecc-69373d1c1132","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":21,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.37:6800","nonce":1690341662},{"type":"v1","addr":"172.21.15.37:6801","nonce":1690341662}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.37:6802","nonce":1690341662},{"type":"v1","addr":"172.21.15.37:6803","nonce":1690341662}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.37:6806","nonce":1690341662},{"type":"v1","addr":"172.21.15.37:6807","nonce":1690341662}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.37:6804","nonce":1690341662},{"type":"v1","addr":"172.21.15.37:6805","nonce":1690341662}]},"public_addr":"172.21.15.37:6801/1690341662","cluster_addr":"172.21.15.37:6803/1690341662","heartbeat_back_addr":"172.21.15.37:6807/1690341662","heartbeat_front_addr":"172.21.15.37:6805/1690341662","state":["exists","up"]},{"osd":2,"uuid":"4ae7a0e9-cb3f-484c-a75f-f3766b721c47","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":21,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.37:6808","nonce":2754669649},{"type":"v1","addr":"172.21.15.37:6809","nonce":2754669649}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.37:6810","nonce":2754669649},{"type":"v1","addr":"172.21.15.37:6811","nonce":2754669649}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.37:6814","nonce":2754669649},{"type":"v1","addr":"172.21.15.37:6815","nonce":2754669649}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.37:6812","nonce":2754669649},{"type":"v1","addr":"172.21.15.37:6813","nonce":2754669649}]},"public_addr":"172.21.15.37:6809/2754669649","cluster_addr":"172.21.15.37:6811/2754669649","heartbeat_back_addr":"172.21.15.37:6815/2754669649","heartbeat_front_addr":"172.21.15.37:6813/2754669649","state":["exists","up"]},{"osd":3,"uuid":"1ba4a746-1608-47a7-ace2-960a5cfae934","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":21,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.37:6824","nonce":1815290618},{"type":"v1","addr":"172.21.15.37:6825","nonce":1815290618}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.37:6826","nonce":1815290618},{"type":"v1","addr":"172.21.15.37:6827","nonce":1815290618}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.37:6830","nonce":1815290618},{"type":"v1","addr":"172.21.15.37:6831","nonce":1815290618}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.37:6828","nonce":1815290618},{"type":"v1","addr":"172.21.15.37:6829","nonce":1815290618}]},"public_addr":"172.21.15.37:6825/1815290618","cluster_addr":"172.21.15.37:6827/1815290618","heartbeat_back_addr":"172.21.15.37:6831/1815290618","heartbeat_front_addr":"172.21.15.37:6829/1815290618","state":["exists","up"]},{"osd":4,"uuid":"d4f59f5d-0413-4b79-87ee-bb212c1fcf62","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":21,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6800","nonce":4051532063},{"type":"v1","addr":"172.21.15.203:6801","nonce":4051532063}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6802","nonce":4051532063},{"type":"v1","addr":"172.21.15.203:6803","nonce":4051532063}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6806","nonce":4051532063},{"type":"v1","addr":"172.21.15.203:6807","nonce":4051532063}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6804","nonce":4051532063},{"type":"v1","addr":"172.21.15.203:6805","nonce":4051532063}]},"public_addr":"172.21.15.203:6801/4051532063","cluster_addr":"172.21.15.203:6803/4051532063","heartbeat_back_addr":"172.21.15.203:6807/4051532063","heartbeat_front_addr":"172.21.15.203:6805/4051532063","state":["exists","up"]},{"osd":5,"uuid":"69710a0e-41d8-4e7f-9a10-17ce540b67be","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":24,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6824","nonce":1203822890},{"type":"v1","addr":"172.21.15.203:6825","nonce":1203822890}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6826","nonce":1203822890},{"type":"v1","addr":"172.21.15.203:6827","nonce":1203822890}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6830","nonce":1203822890},{"type":"v1","addr":"172.21.15.203:6831","nonce":1203822890}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6828","nonce":1203822890},{"type":"v1","addr":"172.21.15.203:6829","nonce":1203822890}]},"public_addr":"172.21.15.203:6825/1203822890","cluster_addr":"172.21.15.203:6827/1203822890","heartbeat_back_addr":"172.21.15.203:6831/1203822890","heartbeat_front_addr":"172.21.15.203:6829/1203822890","state":["exists","up"]},{"osd":6,"uuid":"f29dd047-3b48-473d-a235-6642a6293f20","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":21,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6810","nonce":2309615014},{"type":"v1","addr":"172.21.15.203:6812","nonce":2309615014}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6814","nonce":2309615014},{"type":"v1","addr":"172.21.15.203:6816","nonce":2309615014}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6822","nonce":2309615014},{"type":"v1","addr":"172.21.15.203:6823","nonce":2309615014}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6818","nonce":2309615014},{"type":"v1","addr":"172.21.15.203:6820","nonce":2309615014}]},"public_addr":"172.21.15.203:6812/2309615014","cluster_addr":"172.21.15.203:6816/2309615014","heartbeat_back_addr":"172.21.15.203:6823/2309615014","heartbeat_front_addr":"172.21.15.203:6820/2309615014","state":["exists","up"]},{"osd":7,"uuid":"6e691644-378b-4fdc-8e86-fb5f56e31f29","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":21,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6808","nonce":3688099014},{"type":"v1","addr":"172.21.15.203:6809","nonce":3688099014}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6811","nonce":3688099014},{"type":"v1","addr":"172.21.15.203:6813","nonce":3688099014}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6819","nonce":3688099014},{"type":"v1","addr":"172.21.15.203:6821","nonce":3688099014}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6815","nonce":3688099014},{"type":"v1","addr":"172.21.15.203:6817","nonce":3688099014}]},"public_addr":"172.21.15.203:6809/3688099014","cluster_addr":"172.21.15.203:6813/3688099014","heartbeat_back_addr":"172.21.15.203:6821/3688099014","heartbeat_front_addr":"172.21.15.203:6817/3688099014","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T10:28:51.419141+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T10:28:51.366448+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T10:28:51.348276+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T10:28:51.380519+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T10:28:51.105812+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T10:28:51.384496+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T10:28:51.388755+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T10:28:51.366958+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[{"pgid":"3.31","mappings":[{"from":0,"to":7}]}],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-04-25T10:29:51.289 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:51.286+0000 7f34a9572640 1 -- 172.21.15.37:0/2719626196 >> [v2:172.21.15.203:6832/2418205237,v1:172.21.15.203:6833/2418205237] conn(0x7f34840744b0 msgr2=0x7f34840769c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:29:51.289 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:51.286+0000 7f34a9572640 1 --2- 172.21.15.37:0/2719626196 >> [v2:172.21.15.203:6832/2418205237,v1:172.21.15.203:6833/2418205237] conn(0x7f34840744b0 0x7f34840769c0 secure :-1 s=READY pgs=159 cs=0 l=1 rev1=1 crypto rx=0x7f349c034000 tx=0x7f349c02c000 comp rx=0 tx=0).stop 2024-04-25T10:29:51.289 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:51.286+0000 7f34a9572640 1 -- 172.21.15.37:0/2719626196 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f34a40d1c20 msgr2=0x7f34a41e7b30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:29:51.289 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:51.286+0000 7f34a9572640 1 --2- 172.21.15.37:0/2719626196 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f34a40d1c20 0x7f34a41e7b30 secure :-1 s=READY pgs=303 cs=0 l=1 rev1=1 crypto rx=0x7f348c00b3e0 tx=0x7f348c00b8a0 comp rx=0 tx=0).stop 2024-04-25T10:29:51.289 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:51.286+0000 7f34a9572640 1 -- 172.21.15.37:0/2719626196 shutdown_connections 2024-04-25T10:29:51.289 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:51.286+0000 7f34a9572640 1 --2- 172.21.15.37:0/2719626196 >> [v2:172.21.15.203:6832/2418205237,v1:172.21.15.203:6833/2418205237] conn(0x7f34840744b0 0x7f34840769c0 unknown :-1 s=CLOSED pgs=159 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:29:51.289 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:51.286+0000 7f34a9572640 1 --2- 172.21.15.37:0/2719626196 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f34a41e8070 0x7f34a40a0bd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:29:51.289 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:51.286+0000 7f34a9572640 1 --2- 172.21.15.37:0/2719626196 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f34a40d1c20 0x7f34a41e7b30 unknown :-1 s=CLOSED pgs=303 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:29:51.289 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:51.286+0000 7f34a9572640 1 --2- 172.21.15.37:0/2719626196 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f34a40b71b0 0x7f34a41d6d50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:29:51.289 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:51.286+0000 7f34a9572640 1 -- 172.21.15.37:0/2719626196 >> 172.21.15.37:0/2719626196 conn(0x7f34a40d03d0 msgr2=0x7f34a4072090 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:29:51.289 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:51.286+0000 7f34a9572640 1 -- 172.21.15.37:0/2719626196 shutdown_connections 2024-04-25T10:29:51.289 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:51.286+0000 7f34a9572640 1 -- 172.21.15.37:0/2719626196 wait complete. 2024-04-25T10:29:51.300 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2024-04-25T10:28:56.202153+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '19', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 8, 'score_stable': 8, 'optimal_score': 0.25, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 2, 'pool_name': 'cephfs_metadata', 'create_time': '2024-04-25T10:29:06.507244+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '22', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_autoscale_bias': 4, 'pg_num_min': 16, 'recovery_priority': 5}, 'application_metadata': {'cephfs': {'metadata': 'cephfs'}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 2.25, 'score_stable': 2.25, 'optimal_score': 1, 'raw_score_acting': 2.25, 'raw_score_stable': 2.25, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 3, 'pool_name': 'cephfs_data', 'create_time': '2024-04-25T10:29:07.347844+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 64, 'pg_placement_num': 64, 'pg_placement_num_target': 64, 'pg_num_target': 64, 'pg_num_pending': 64, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '23', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.8999999761581421}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 1.5, 'score_stable': 1.5, 'optimal_score': 1, 'raw_score_acting': 1.5, 'raw_score_stable': 1.5, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}] 2024-04-25T10:29:51.301 DEBUG:teuthology.orchestra.run.smithi037:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2024-04-25T10:29:51.416 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:51.413+0000 7fea1f70d640 1 Processor -- start 2024-04-25T10:29:51.416 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:51.413+0000 7fea1f70d640 1 -- start start 2024-04-25T10:29:51.418 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:51.415+0000 7fea1f70d640 1 --2- >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7fea18156c30 0x7fea180cec80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:29:51.418 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:51.415+0000 7fea1f70d640 1 --2- >> v2:172.21.15.203:3300/0 conn(0x7fea180cf1c0 0x7fea1806f990 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:29:51.418 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:51.415+0000 7fea1f70d640 1 -- --> v1:172.21.15.203:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fea180cf5d0 con 0x7fea1806fed0 2024-04-25T10:29:51.418 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:51.415+0000 7fea1f70d640 1 -- --> v2:172.21.15.203:3300/0 -- mon_getmap magic: 0 -- 0x7fea180b6f00 con 0x7fea180cf1c0 2024-04-25T10:29:51.418 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:51.415+0000 7fea1f70d640 1 -- --> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] -- mon_getmap magic: 0 -- 0x7fea1815d850 con 0x7fea18156c30 2024-04-25T10:29:51.418 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:51.415+0000 7fea1d4a9640 1 --2- >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7fea18156c30 0x7fea180cec80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:29:51.418 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:51.415+0000 7fea1cca8640 1 --2- >> v2:172.21.15.203:3300/0 conn(0x7fea180cf1c0 0x7fea1806f990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:29:51.418 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:51.415+0000 7fea1dcaa640 1 --1- >> v1:172.21.15.203:6789/0 conn(0x7fea1806fed0 0x7fea181601c0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.203:6789/0 says I am v1:172.21.15.37:52640/0 (socket says 172.21.15.37:52640) 2024-04-25T10:29:51.419 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:51.415+0000 7fea1cca8640 1 --2- >> v2:172.21.15.203:3300/0 conn(0x7fea180cf1c0 0x7fea1806f990 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.203:3300/0 says I am v2:172.21.15.37:33376/0 (socket says 172.21.15.37:33376) 2024-04-25T10:29:51.419 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:51.415+0000 7fea1dcaa640 1 -- 172.21.15.37:0/2728902912 learned_addr learned my addr 172.21.15.37:0/2728902912 (peer_addr_for_me v1:172.21.15.37:0/0) 2024-04-25T10:29:51.419 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:51.416+0000 7fea1cca8640 1 -- 172.21.15.37:0/2728902912 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7fea18156c30 msgr2=0x7fea180cec80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:29:51.419 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:51.416+0000 7fea1cca8640 1 --2- 172.21.15.37:0/2728902912 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7fea18156c30 0x7fea180cec80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:29:51.419 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:51.416+0000 7fea1cca8640 1 -- 172.21.15.37:0/2728902912 >> v1:172.21.15.203:6789/0 conn(0x7fea1806fed0 legacy=0x7fea181601c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:29:51.419 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:51.416+0000 7fea1cca8640 1 -- 172.21.15.37:0/2728902912 --> v2:172.21.15.203:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fea180702e0 con 0x7fea180cf1c0 2024-04-25T10:29:51.419 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:51.416+0000 7fea17fff640 1 -- 172.21.15.37:0/2728902912 <== mon.1 v1:172.21.15.203:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 747230117 0 0) 0x7fea180cf5d0 con 0x7fea1806fed0 2024-04-25T10:29:51.419 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:51.416+0000 7fea1d4a9640 1 --2- 172.21.15.37:0/2728902912 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7fea18156c30 0x7fea180cec80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-25T10:29:51.420 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:51.416+0000 7fea1cca8640 1 --2- 172.21.15.37:0/2728902912 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fea180cf1c0 0x7fea1806f990 secure :-1 s=READY pgs=139 cs=0 l=1 rev1=1 crypto rx=0x7fea0c0097a0 tx=0x7fea0c02a180 comp rx=0 tx=0).ready entity=mon.1 client_cookie=bc3179343755b4e8 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:29:51.420 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:51.417+0000 7fea17fff640 1 -- 172.21.15.37:0/2728902912 <== mon.1 v2:172.21.15.203:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fea0c037070 con 0x7fea180cf1c0 2024-04-25T10:29:51.420 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:51.417+0000 7fea17fff640 1 -- 172.21.15.37:0/2728902912 <== mon.1 v2:172.21.15.203:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fea0c032440 con 0x7fea180cf1c0 2024-04-25T10:29:51.420 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:51.417+0000 7fea17fff640 1 -- 172.21.15.37:0/2728902912 <== mon.1 v2:172.21.15.203:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fea0c032810 con 0x7fea180cf1c0 2024-04-25T10:29:51.420 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:51.417+0000 7fea1f70d640 1 -- 172.21.15.37:0/2728902912 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fea180cf1c0 msgr2=0x7fea1806f990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:29:51.420 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:51.417+0000 7fea1f70d640 1 --2- 172.21.15.37:0/2728902912 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fea180cf1c0 0x7fea1806f990 secure :-1 s=READY pgs=139 cs=0 l=1 rev1=1 crypto rx=0x7fea0c0097a0 tx=0x7fea0c02a180 comp rx=0 tx=0).stop 2024-04-25T10:29:51.420 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:51.417+0000 7fea1f70d640 1 -- 172.21.15.37:0/2728902912 shutdown_connections 2024-04-25T10:29:51.421 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:51.417+0000 7fea1f70d640 1 --2- 172.21.15.37:0/2728902912 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fea180cf1c0 0x7fea1806f990 unknown :-1 s=CLOSED pgs=139 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:29:51.421 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:51.417+0000 7fea1f70d640 1 --2- 172.21.15.37:0/2728902912 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7fea18156c30 0x7fea180cec80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:29:51.421 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:51.417+0000 7fea1f70d640 1 -- 172.21.15.37:0/2728902912 >> 172.21.15.37:0/2728902912 conn(0x7fea180b36b0 msgr2=0x7fea180b3ac0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:29:51.421 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:51.417+0000 7fea1f70d640 1 -- 172.21.15.37:0/2728902912 shutdown_connections 2024-04-25T10:29:51.421 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:51.417+0000 7fea1f70d640 1 -- 172.21.15.37:0/2728902912 wait complete. 2024-04-25T10:29:51.422 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:51.419+0000 7fea1f70d640 1 Processor -- start 2024-04-25T10:29:51.422 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:51.419+0000 7fea1f70d640 1 -- start start 2024-04-25T10:29:51.424 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:51.420+0000 7fea1f70d640 1 --2- >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7fea1806fed0 0x7fea18150100 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:29:51.424 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:51.420+0000 7fea1f70d640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fea180cf1c0 0x7fea1814ede0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:29:51.424 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:51.420+0000 7fea1f70d640 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fea18156c30 0x7fea180b5280 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:29:51.424 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:51.420+0000 7fea1f70d640 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 -- 0x7fea1815d4e0 con 0x7fea180cf1c0 2024-04-25T10:29:51.424 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:51.420+0000 7fea1f70d640 1 -- --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_getmap magic: 0 -- 0x7fea1815d200 con 0x7fea18156c30 2024-04-25T10:29:51.424 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:51.420+0000 7fea1f70d640 1 -- --> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] -- mon_getmap magic: 0 -- 0x7fea18157040 con 0x7fea1806fed0 2024-04-25T10:29:51.424 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:51.421+0000 7fea1cca8640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fea180cf1c0 0x7fea1814ede0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:29:51.424 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:51.421+0000 7fea1cca8640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fea180cf1c0 0x7fea1814ede0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.37:3300/0 says I am v2:172.21.15.37:36266/0 (socket says 172.21.15.37:36266) 2024-04-25T10:29:51.424 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:51.421+0000 7fea1cca8640 1 -- 172.21.15.37:0/3437595980 learned_addr learned my addr 172.21.15.37:0/3437595980 (peer_addr_for_me v2:172.21.15.37:0/0) 2024-04-25T10:29:51.424 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:51.421+0000 7fea1d4a9640 1 --2- 172.21.15.37:0/3437595980 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7fea1806fed0 0x7fea18150100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:29:51.424 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:51.421+0000 7fea1dcaa640 1 --2- 172.21.15.37:0/3437595980 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fea18156c30 0x7fea180b5280 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:29:51.424 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:51.421+0000 7fea1cca8640 1 -- 172.21.15.37:0/3437595980 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7fea1806fed0 msgr2=0x7fea18150100 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:29:51.425 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:51.421+0000 7fea1cca8640 1 --2- 172.21.15.37:0/3437595980 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7fea1806fed0 0x7fea18150100 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:29:51.425 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:51.421+0000 7fea1cca8640 1 -- 172.21.15.37:0/3437595980 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fea18156c30 msgr2=0x7fea180b5280 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:29:51.425 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:51.421+0000 7fea1cca8640 1 --2- 172.21.15.37:0/3437595980 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fea18156c30 0x7fea180b5280 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:29:51.425 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:51.421+0000 7fea1cca8640 1 -- 172.21.15.37:0/3437595980 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fea18153880 con 0x7fea180cf1c0 2024-04-25T10:29:51.425 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:51.421+0000 7fea1cca8640 1 --2- 172.21.15.37:0/3437595980 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fea180cf1c0 0x7fea1814ede0 secure :-1 s=READY pgs=304 cs=0 l=1 rev1=1 crypto rx=0x7fea0c000c00 tx=0x7fea0c0046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:29:51.425 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:51.421+0000 7fea15ffb640 1 -- 172.21.15.37:0/3437595980 <== mon.0 v2:172.21.15.37:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fea0c037040 con 0x7fea180cf1c0 2024-04-25T10:29:51.425 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:51.421+0000 7fea15ffb640 1 -- 172.21.15.37:0/3437595980 <== mon.0 v2:172.21.15.37:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fea0c042070 con 0x7fea180cf1c0 2024-04-25T10:29:51.425 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:51.421+0000 7fea1f70d640 1 -- 172.21.15.37:0/3437595980 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fea18153b60 con 0x7fea180cf1c0 2024-04-25T10:29:51.425 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:51.421+0000 7fea15ffb640 1 -- 172.21.15.37:0/3437595980 <== mon.0 v2:172.21.15.37:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fea0c030940 con 0x7fea180cf1c0 2024-04-25T10:29:51.425 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:51.421+0000 7fea1f70d640 1 -- 172.21.15.37:0/3437595980 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fea180b4a80 con 0x7fea180cf1c0 2024-04-25T10:29:51.426 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:51.423+0000 7fea1f70d640 1 -- 172.21.15.37:0/3437595980 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fea18073f80 con 0x7fea180cf1c0 2024-04-25T10:29:51.426 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:51.423+0000 7fea15ffb640 1 -- 172.21.15.37:0/3437595980 <== mon.0 v2:172.21.15.37:3300/0 4 ==== mgrmap(e 5) ==== 96782+0+0 (secure 0 0 0) 0x7fea0c030ae0 con 0x7fea180cf1c0 2024-04-25T10:29:51.426 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:51.424+0000 7fea15ffb640 1 --2- 172.21.15.37:0/3437595980 >> [v2:172.21.15.203:6832/2418205237,v1:172.21.15.203:6833/2418205237] conn(0x7fea040744b0 0x7fea040769c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:29:51.426 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:51.424+0000 7fea15ffb640 1 -- 172.21.15.37:0/3437595980 <== mon.0 v2:172.21.15.37:3300/0 5 ==== osd_map(25..25 src has 1..25) ==== 5738+0+0 (secure 0 0 0) 0x7fea0c0b5c20 con 0x7fea180cf1c0 2024-04-25T10:29:51.426 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:51.424+0000 7fea1d4a9640 1 --2- 172.21.15.37:0/3437595980 >> [v2:172.21.15.203:6832/2418205237,v1:172.21.15.203:6833/2418205237] conn(0x7fea040744b0 0x7fea040769c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:29:51.427 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:51.425+0000 7fea1d4a9640 1 --2- 172.21.15.37:0/3437595980 >> [v2:172.21.15.203:6832/2418205237,v1:172.21.15.203:6833/2418205237] conn(0x7fea040744b0 0x7fea040769c0 secure :-1 s=READY pgs=160 cs=0 l=1 rev1=1 crypto rx=0x7fea18071210 tx=0x7fea10008040 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:29:51.430 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:51.429+0000 7fea15ffb640 1 -- 172.21.15.37:0/3437595980 <== mon.0 v2:172.21.15.37:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187246 (secure 0 0 0) 0x7fea18073f80 con 0x7fea180cf1c0 2024-04-25T10:29:51.615 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:51.612+0000 7fea1f70d640 1 -- 172.21.15.37:0/3437595980 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) -- 0x7fea18153610 con 0x7fea180cf1c0 2024-04-25T10:29:51.615 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:51.612+0000 7fea15ffb640 1 -- 172.21.15.37:0/3437595980 <== mon.0 v2:172.21.15.37:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v25) ==== 93+0+10 (secure 0 0 0) 0x7fea180702e0 con 0x7fea180cf1c0 2024-04-25T10:29:51.615 INFO:teuthology.orchestra.run.smithi037.stdout:pg_num: 1 2024-04-25T10:29:51.618 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:51.615+0000 7fea1f70d640 1 -- 172.21.15.37:0/3437595980 >> [v2:172.21.15.203:6832/2418205237,v1:172.21.15.203:6833/2418205237] conn(0x7fea040744b0 msgr2=0x7fea040769c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:29:51.618 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:51.615+0000 7fea1f70d640 1 --2- 172.21.15.37:0/3437595980 >> [v2:172.21.15.203:6832/2418205237,v1:172.21.15.203:6833/2418205237] conn(0x7fea040744b0 0x7fea040769c0 secure :-1 s=READY pgs=160 cs=0 l=1 rev1=1 crypto rx=0x7fea18071210 tx=0x7fea10008040 comp rx=0 tx=0).stop 2024-04-25T10:29:51.618 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:51.615+0000 7fea1f70d640 1 -- 172.21.15.37:0/3437595980 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fea180cf1c0 msgr2=0x7fea1814ede0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:29:51.618 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:51.615+0000 7fea1f70d640 1 --2- 172.21.15.37:0/3437595980 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fea180cf1c0 0x7fea1814ede0 secure :-1 s=READY pgs=304 cs=0 l=1 rev1=1 crypto rx=0x7fea0c000c00 tx=0x7fea0c0046c0 comp rx=0 tx=0).stop 2024-04-25T10:29:51.618 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:51.615+0000 7fea1f70d640 1 -- 172.21.15.37:0/3437595980 shutdown_connections 2024-04-25T10:29:51.618 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:51.615+0000 7fea1f70d640 1 --2- 172.21.15.37:0/3437595980 >> [v2:172.21.15.203:6832/2418205237,v1:172.21.15.203:6833/2418205237] conn(0x7fea040744b0 0x7fea040769c0 unknown :-1 s=CLOSED pgs=160 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:29:51.618 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:51.615+0000 7fea1f70d640 1 --2- 172.21.15.37:0/3437595980 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fea18156c30 0x7fea180b5280 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:29:51.618 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:51.615+0000 7fea1f70d640 1 --2- 172.21.15.37:0/3437595980 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fea180cf1c0 0x7fea1814ede0 unknown :-1 s=CLOSED pgs=304 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:29:51.618 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:51.615+0000 7fea1f70d640 1 --2- 172.21.15.37:0/3437595980 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7fea1806fed0 0x7fea18150100 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:29:51.619 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:51.615+0000 7fea1f70d640 1 -- 172.21.15.37:0/3437595980 >> 172.21.15.37:0/3437595980 conn(0x7fea180b36b0 msgr2=0x7fea180d0c60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:29:51.619 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:51.615+0000 7fea1f70d640 1 -- 172.21.15.37:0/3437595980 shutdown_connections 2024-04-25T10:29:51.619 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:51.615+0000 7fea1f70d640 1 -- 172.21.15.37:0/3437595980 wait complete. 2024-04-25T10:29:51.628 DEBUG:teuthology.orchestra.run.smithi037:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_metadata pg_num 2024-04-25T10:29:51.742 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:51.740+0000 7f20d0e20640 1 Processor -- start 2024-04-25T10:29:51.743 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:51.740+0000 7f20d0e20640 1 -- start start 2024-04-25T10:29:51.744 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:51.741+0000 7f20d0e20640 1 --2- >> v2:172.21.15.37:3300/0 conn(0x7f20cc0cf570 0x7f20cc16ad90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:29:51.744 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:51.741+0000 7f20d0e20640 1 -- --> v1:172.21.15.37:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f20cc06e910 con 0x7f20cc0d2860 2024-04-25T10:29:51.744 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:51.741+0000 7f20d0e20640 1 -- --> v1:172.21.15.203:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f20cc0d2e40 con 0x7f20cc0b2dc0 2024-04-25T10:29:51.744 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:51.741+0000 7f20d0e20640 1 -- --> v2:172.21.15.37:3300/0 -- mon_getmap magic: 0 -- 0x7f20cc0b92c0 con 0x7f20cc0cf570 2024-04-25T10:29:51.744 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:51.742+0000 7f20cad9d640 1 --2- >> v2:172.21.15.37:3300/0 conn(0x7f20cc0cf570 0x7f20cc16ad90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:29:51.744 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:51.742+0000 7f20c3fff640 1 --1- >> v1:172.21.15.37:6789/0 conn(0x7f20cc0d2860 0x7f20cc06e200 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.37:6789/0 says I am v1:172.21.15.37:56788/0 (socket says 172.21.15.37:56788) 2024-04-25T10:29:51.744 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:51.742+0000 7f20c3fff640 1 -- 172.21.15.37:0/2219876580 learned_addr learned my addr 172.21.15.37:0/2219876580 (peer_addr_for_me v1:172.21.15.37:0/0) 2024-04-25T10:29:51.744 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:51.742+0000 7f20cad9d640 1 --2- >> v2:172.21.15.37:3300/0 conn(0x7f20cc0cf570 0x7f20cc16ad90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.37:3300/0 says I am v2:172.21.15.37:36268/0 (socket says 172.21.15.37:36268) 2024-04-25T10:29:51.745 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:51.742+0000 7f20cad9d640 1 -- 172.21.15.37:0/2219876580 >> v1:172.21.15.203:6789/0 conn(0x7f20cc0b2dc0 legacy=0x7f20cc06daf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:29:51.745 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:51.742+0000 7f20cad9d640 1 -- 172.21.15.37:0/2219876580 >> v1:172.21.15.37:6789/0 conn(0x7f20cc0d2860 legacy=0x7f20cc06e200 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:29:51.745 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:51.742+0000 7f20c9d9b640 1 -- 172.21.15.37:0/2219876580 <== mon.0 v1:172.21.15.37:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 846485520 0 0) 0x7f20cc06e910 con 0x7f20cc0d2860 2024-04-25T10:29:51.745 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:51.742+0000 7f20cad9d640 1 -- 172.21.15.37:0/2219876580 --> v2:172.21.15.37:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f20cc0d3120 con 0x7f20cc0cf570 2024-04-25T10:29:51.745 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:51.743+0000 7f20cad9d640 1 --2- 172.21.15.37:0/2219876580 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f20cc0cf570 0x7f20cc16ad90 secure :-1 s=READY pgs=306 cs=0 l=1 rev1=1 crypto rx=0x7f20c4009d90 tx=0x7f20c402a6a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d4f921198bf708d1 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:29:51.745 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:51.743+0000 7f20c9d9b640 1 -- 172.21.15.37:0/2219876580 <== mon.0 v2:172.21.15.37:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f20c4007740 con 0x7f20cc0cf570 2024-04-25T10:29:51.745 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:51.743+0000 7f20c9d9b640 1 -- 172.21.15.37:0/2219876580 <== mon.0 v2:172.21.15.37:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f20c40078e0 con 0x7f20cc0cf570 2024-04-25T10:29:51.745 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:51.743+0000 7f20c9d9b640 1 -- 172.21.15.37:0/2219876580 <== mon.0 v2:172.21.15.37:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f20c4007cb0 con 0x7f20cc0cf570 2024-04-25T10:29:51.746 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:51.743+0000 7f20d0e20640 1 -- 172.21.15.37:0/2219876580 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f20cc0cf570 msgr2=0x7f20cc16ad90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:29:51.746 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:51.743+0000 7f20d0e20640 1 --2- 172.21.15.37:0/2219876580 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f20cc0cf570 0x7f20cc16ad90 secure :-1 s=READY pgs=306 cs=0 l=1 rev1=1 crypto rx=0x7f20c4009d90 tx=0x7f20c402a6a0 comp rx=0 tx=0).stop 2024-04-25T10:29:51.747 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:51.743+0000 7f20d0e20640 1 -- 172.21.15.37:0/2219876580 shutdown_connections 2024-04-25T10:29:51.747 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:51.744+0000 7f20d0e20640 1 --2- 172.21.15.37:0/2219876580 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f20cc0cf570 0x7f20cc16ad90 unknown :-1 s=CLOSED pgs=306 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:29:51.747 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:51.744+0000 7f20d0e20640 1 -- 172.21.15.37:0/2219876580 >> 172.21.15.37:0/2219876580 conn(0x7f20cc0b5440 msgr2=0x7f20cc0b5850 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:29:51.747 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:51.744+0000 7f20d0e20640 1 -- 172.21.15.37:0/2219876580 shutdown_connections 2024-04-25T10:29:51.747 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:51.744+0000 7f20d0e20640 1 -- 172.21.15.37:0/2219876580 wait complete. 2024-04-25T10:29:51.747 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:51.745+0000 7f20d0e20640 1 Processor -- start 2024-04-25T10:29:51.748 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:51.746+0000 7f20d0e20640 1 -- start start 2024-04-25T10:29:51.751 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:51.747+0000 7f20d0e20640 1 --2- >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f20cc0b2dc0 0x7f20cc0cac10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:29:51.751 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:51.747+0000 7f20d0e20640 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f20cc0cf570 0x7f20cc15ff90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:29:51.751 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:51.747+0000 7f20d0e20640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f20cc0d2860 0x7f20cc158680 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:29:51.751 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:51.747+0000 7f20d0e20640 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 -- 0x7f20cc0d36e0 con 0x7f20cc0d2860 2024-04-25T10:29:51.751 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:51.747+0000 7f20d0e20640 1 -- --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_getmap magic: 0 -- 0x7f20cc0ce0d0 con 0x7f20cc0cf570 2024-04-25T10:29:51.751 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:51.747+0000 7f20d0e20640 1 -- --> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] -- mon_getmap magic: 0 -- 0x7f20cc16b2d0 con 0x7f20cc0b2dc0 2024-04-25T10:29:51.751 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:51.747+0000 7f20cad9d640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f20cc0d2860 0x7f20cc158680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:29:51.751 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:51.747+0000 7f20cad9d640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f20cc0d2860 0x7f20cc158680 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.37:3300/0 says I am v2:172.21.15.37:36282/0 (socket says 172.21.15.37:36282) 2024-04-25T10:29:51.751 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:51.747+0000 7f20cad9d640 1 -- 172.21.15.37:0/1840478933 learned_addr learned my addr 172.21.15.37:0/1840478933 (peer_addr_for_me v2:172.21.15.37:0/0) 2024-04-25T10:29:51.751 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:51.747+0000 7f20c3fff640 1 --2- 172.21.15.37:0/1840478933 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f20cc0cf570 0x7f20cc15ff90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:29:51.751 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:51.747+0000 7f20ca59c640 1 --2- 172.21.15.37:0/1840478933 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f20cc0b2dc0 0x7f20cc0cac10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:29:51.751 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:51.747+0000 7f20cad9d640 1 -- 172.21.15.37:0/1840478933 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f20cc0b2dc0 msgr2=0x7f20cc0cac10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:29:51.751 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:51.747+0000 7f20cad9d640 1 --2- 172.21.15.37:0/1840478933 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f20cc0b2dc0 0x7f20cc0cac10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:29:51.751 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:51.747+0000 7f20cad9d640 1 -- 172.21.15.37:0/1840478933 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f20cc0cf570 msgr2=0x7f20cc15ff90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:29:51.752 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:51.747+0000 7f20cad9d640 1 --2- 172.21.15.37:0/1840478933 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f20cc0cf570 0x7f20cc15ff90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:29:51.752 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:51.747+0000 7f20cad9d640 1 -- 172.21.15.37:0/1840478933 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f20cc1604d0 con 0x7f20cc0d2860 2024-04-25T10:29:51.752 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:51.747+0000 7f20cad9d640 1 --2- 172.21.15.37:0/1840478933 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f20cc0d2860 0x7f20cc158680 secure :-1 s=READY pgs=307 cs=0 l=1 rev1=1 crypto rx=0x7f20c402ac90 tx=0x7f20c4004a40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:29:51.752 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:51.748+0000 7f20c2ffd640 1 -- 172.21.15.37:0/1840478933 <== mon.0 v2:172.21.15.37:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f20c4040070 con 0x7f20cc0d2860 2024-04-25T10:29:51.752 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:51.748+0000 7f20c2ffd640 1 -- 172.21.15.37:0/1840478933 <== mon.0 v2:172.21.15.37:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f20c4031c90 con 0x7f20cc0d2860 2024-04-25T10:29:51.752 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:51.748+0000 7f20d0e20640 1 -- 172.21.15.37:0/1840478933 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f20cc159d50 con 0x7f20cc0d2860 2024-04-25T10:29:51.752 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:51.748+0000 7f20c2ffd640 1 -- 172.21.15.37:0/1840478933 <== mon.0 v2:172.21.15.37:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f20c4045680 con 0x7f20cc0d2860 2024-04-25T10:29:51.752 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:51.748+0000 7f20d0e20640 1 -- 172.21.15.37:0/1840478933 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f20cc159ff0 con 0x7f20cc0d2860 2024-04-25T10:29:51.752 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:51.749+0000 7f20d0e20640 1 -- 172.21.15.37:0/1840478933 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f20cc06a140 con 0x7f20cc0d2860 2024-04-25T10:29:51.753 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:51.749+0000 7f20c2ffd640 1 -- 172.21.15.37:0/1840478933 <== mon.0 v2:172.21.15.37:3300/0 4 ==== mgrmap(e 5) ==== 96782+0+0 (secure 0 0 0) 0x7f20c4039040 con 0x7f20cc0d2860 2024-04-25T10:29:51.753 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:51.750+0000 7f20c2ffd640 1 --2- 172.21.15.37:0/1840478933 >> [v2:172.21.15.203:6832/2418205237,v1:172.21.15.203:6833/2418205237] conn(0x7f20ac0744e0 0x7f20ac0769f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:29:51.753 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:51.750+0000 7f20c2ffd640 1 -- 172.21.15.37:0/1840478933 <== mon.0 v2:172.21.15.37:3300/0 5 ==== osd_map(25..25 src has 1..25) ==== 5738+0+0 (secure 0 0 0) 0x7f20c40b6a60 con 0x7f20cc0d2860 2024-04-25T10:29:51.753 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:51.750+0000 7f20ca59c640 1 --2- 172.21.15.37:0/1840478933 >> [v2:172.21.15.203:6832/2418205237,v1:172.21.15.203:6833/2418205237] conn(0x7f20ac0744e0 0x7f20ac0769f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:29:51.753 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:51.751+0000 7f20ca59c640 1 --2- 172.21.15.37:0/1840478933 >> [v2:172.21.15.203:6832/2418205237,v1:172.21.15.203:6833/2418205237] conn(0x7f20ac0744e0 0x7f20ac0769f0 secure :-1 s=READY pgs=161 cs=0 l=1 rev1=1 crypto rx=0x7f20cc074cc0 tx=0x7f20bc009060 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:29:51.756 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:51.755+0000 7f20c2ffd640 1 -- 172.21.15.37:0/1840478933 <== mon.0 v2:172.21.15.37:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187246 (secure 0 0 0) 0x7f20cc1604d0 con 0x7f20cc0d2860 2024-04-25T10:29:51.942 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:51.940+0000 7f20d0e20640 1 -- 172.21.15.37:0/1840478933 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"} v 0) -- 0x7f20cc0cb310 con 0x7f20cc0d2860 2024-04-25T10:29:51.942 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:51.940+0000 7f20c2ffd640 1 -- 172.21.15.37:0/1840478933 <== mon.0 v2:172.21.15.37:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]=0 v25) ==== 104+0+11 (secure 0 0 0) 0x7f20c4080cd0 con 0x7f20cc0d2860 2024-04-25T10:29:51.943 INFO:teuthology.orchestra.run.smithi037.stdout:pg_num: 32 2024-04-25T10:29:51.945 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:51.942+0000 7f20d0e20640 1 -- 172.21.15.37:0/1840478933 >> [v2:172.21.15.203:6832/2418205237,v1:172.21.15.203:6833/2418205237] conn(0x7f20ac0744e0 msgr2=0x7f20ac0769f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:29:51.945 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:51.942+0000 7f20d0e20640 1 --2- 172.21.15.37:0/1840478933 >> [v2:172.21.15.203:6832/2418205237,v1:172.21.15.203:6833/2418205237] conn(0x7f20ac0744e0 0x7f20ac0769f0 secure :-1 s=READY pgs=161 cs=0 l=1 rev1=1 crypto rx=0x7f20cc074cc0 tx=0x7f20bc009060 comp rx=0 tx=0).stop 2024-04-25T10:29:51.945 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:51.942+0000 7f20d0e20640 1 -- 172.21.15.37:0/1840478933 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f20cc0d2860 msgr2=0x7f20cc158680 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:29:51.945 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:51.942+0000 7f20d0e20640 1 --2- 172.21.15.37:0/1840478933 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f20cc0d2860 0x7f20cc158680 secure :-1 s=READY pgs=307 cs=0 l=1 rev1=1 crypto rx=0x7f20c402ac90 tx=0x7f20c4004a40 comp rx=0 tx=0).stop 2024-04-25T10:29:51.945 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:51.942+0000 7f20d0e20640 1 -- 172.21.15.37:0/1840478933 shutdown_connections 2024-04-25T10:29:51.945 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:51.942+0000 7f20d0e20640 1 --2- 172.21.15.37:0/1840478933 >> [v2:172.21.15.203:6832/2418205237,v1:172.21.15.203:6833/2418205237] conn(0x7f20ac0744e0 0x7f20ac0769f0 unknown :-1 s=CLOSED pgs=161 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:29:51.945 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:51.942+0000 7f20d0e20640 1 --2- 172.21.15.37:0/1840478933 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f20cc0d2860 0x7f20cc158680 unknown :-1 s=CLOSED pgs=307 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:29:51.945 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:51.942+0000 7f20d0e20640 1 --2- 172.21.15.37:0/1840478933 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f20cc0cf570 0x7f20cc15ff90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:29:51.945 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:51.942+0000 7f20d0e20640 1 --2- 172.21.15.37:0/1840478933 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f20cc0b2dc0 0x7f20cc0cac10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:29:51.945 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:51.942+0000 7f20d0e20640 1 -- 172.21.15.37:0/1840478933 >> 172.21.15.37:0/1840478933 conn(0x7f20cc0b5440 msgr2=0x7f20cc0d0a40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:29:51.945 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:51.942+0000 7f20d0e20640 1 -- 172.21.15.37:0/1840478933 shutdown_connections 2024-04-25T10:29:51.945 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:51.942+0000 7f20d0e20640 1 -- 172.21.15.37:0/1840478933 wait complete. 2024-04-25T10:29:51.956 DEBUG:teuthology.orchestra.run.smithi037:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data pg_num 2024-04-25T10:29:52.069 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:52.067+0000 7f28def6c640 1 Processor -- start 2024-04-25T10:29:52.070 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:52.067+0000 7f28def6c640 1 -- start start 2024-04-25T10:29:52.071 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:52.069+0000 7f28def6c640 1 --2- >> v2:172.21.15.37:3300/0 conn(0x7f28d80d5160 0x7f28d80b4e80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:29:52.071 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:52.069+0000 7f28def6c640 1 --2- >> v2:172.21.15.203:3300/0 conn(0x7f28d80d3230 0x7f28d80b9060 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:29:52.071 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:52.069+0000 7f28def6c640 1 -- --> v1:172.21.15.37:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f28d80d1370 con 0x7f28d80d0990 2024-04-25T10:29:52.071 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:52.069+0000 7f28def6c640 1 -- --> v2:172.21.15.37:3300/0 -- mon_getmap magic: 0 -- 0x7f28d80d1650 con 0x7f28d80d5160 2024-04-25T10:29:52.071 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:52.069+0000 7f28def6c640 1 -- --> v2:172.21.15.203:3300/0 -- mon_getmap magic: 0 -- 0x7f28d80d4680 con 0x7f28d80d3230 2024-04-25T10:29:52.072 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:52.069+0000 7f28dcd08640 1 --2- >> v2:172.21.15.37:3300/0 conn(0x7f28d80d5160 0x7f28d80b4e80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:29:52.072 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:52.069+0000 7f28d7fff640 1 --1- >> v1:172.21.15.37:6789/0 conn(0x7f28d80d0990 0x7f28d80d2b20 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.37:6789/0 says I am v1:172.21.15.37:56794/0 (socket says 172.21.15.37:56794) 2024-04-25T10:29:52.072 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:52.069+0000 7f28d7fff640 1 -- 172.21.15.37:0/3078200221 learned_addr learned my addr 172.21.15.37:0/3078200221 (peer_addr_for_me v1:172.21.15.37:0/0) 2024-04-25T10:29:52.072 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:52.069+0000 7f28dd509640 1 --2- 172.21.15.37:0/3078200221 >> v2:172.21.15.203:3300/0 conn(0x7f28d80d3230 0x7f28d80b9060 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:29:52.072 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:52.069+0000 7f28dcd08640 1 -- 172.21.15.37:0/3078200221 >> v2:172.21.15.203:3300/0 conn(0x7f28d80d3230 msgr2=0x7f28d80b9060 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:29:52.072 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:52.069+0000 7f28dcd08640 1 --2- 172.21.15.37:0/3078200221 >> v2:172.21.15.203:3300/0 conn(0x7f28d80d3230 0x7f28d80b9060 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:29:52.072 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:52.069+0000 7f28dcd08640 1 -- 172.21.15.37:0/3078200221 >> v1:172.21.15.37:6789/0 conn(0x7f28d80d0990 legacy=0x7f28d80d2b20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:29:52.072 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:52.069+0000 7f28dcd08640 1 -- 172.21.15.37:0/3078200221 --> v2:172.21.15.37:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f28d80d4960 con 0x7f28d80d5160 2024-04-25T10:29:52.073 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:52.070+0000 7f28dcd08640 1 --2- 172.21.15.37:0/3078200221 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f28d80d5160 0x7f28d80b4e80 secure :-1 s=READY pgs=309 cs=0 l=1 rev1=1 crypto rx=0x7f28d808f050 tx=0x7f28d002a080 comp rx=0 tx=0).ready entity=mon.0 client_cookie=3b7b4f8ecb5beca6 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:29:52.073 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:52.070+0000 7f28d77fe640 1 -- 172.21.15.37:0/3078200221 <== mon.0 v2:172.21.15.37:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f28d0037070 con 0x7f28d80d5160 2024-04-25T10:29:52.073 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:52.070+0000 7f28d77fe640 1 -- 172.21.15.37:0/3078200221 <== mon.0 v2:172.21.15.37:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f28d002add0 con 0x7f28d80d5160 2024-04-25T10:29:52.074 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:52.070+0000 7f28d77fe640 1 -- 172.21.15.37:0/3078200221 <== mon.0 v2:172.21.15.37:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f28d0032660 con 0x7f28d80d5160 2024-04-25T10:29:52.074 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:52.071+0000 7f28def6c640 1 -- 172.21.15.37:0/3078200221 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f28d80d5160 msgr2=0x7f28d80b4e80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:29:52.074 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:52.071+0000 7f28def6c640 1 --2- 172.21.15.37:0/3078200221 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f28d80d5160 0x7f28d80b4e80 secure :-1 s=READY pgs=309 cs=0 l=1 rev1=1 crypto rx=0x7f28d808f050 tx=0x7f28d002a080 comp rx=0 tx=0).stop 2024-04-25T10:29:52.074 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:52.071+0000 7f28def6c640 1 -- 172.21.15.37:0/3078200221 shutdown_connections 2024-04-25T10:29:52.074 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:52.071+0000 7f28def6c640 1 --2- 172.21.15.37:0/3078200221 >> v2:172.21.15.203:3300/0 conn(0x7f28d80d3230 0x7f28d80b9060 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:29:52.074 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:52.071+0000 7f28def6c640 1 --2- 172.21.15.37:0/3078200221 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f28d80d5160 0x7f28d80b4e80 unknown :-1 s=CLOSED pgs=309 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:29:52.074 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:52.071+0000 7f28def6c640 1 -- 172.21.15.37:0/3078200221 >> 172.21.15.37:0/3078200221 conn(0x7f28d80b7a80 msgr2=0x7f28d80b7e90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:29:52.074 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:52.071+0000 7f28def6c640 1 -- 172.21.15.37:0/3078200221 shutdown_connections 2024-04-25T10:29:52.074 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:52.071+0000 7f28def6c640 1 -- 172.21.15.37:0/3078200221 wait complete. 2024-04-25T10:29:52.075 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:52.073+0000 7f28def6c640 1 Processor -- start 2024-04-25T10:29:52.075 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:52.073+0000 7f28def6c640 1 -- start start 2024-04-25T10:29:52.077 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:52.074+0000 7f28def6c640 1 --2- >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f28d80d0990 0x7f28d80cf7b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:29:52.077 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:52.074+0000 7f28def6c640 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f28d80d5160 0x7f28d8153350 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:29:52.078 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:52.074+0000 7f28def6c640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f28d80b44d0 0x7f28d8153890 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:29:52.078 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:52.074+0000 7f28def6c640 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 -- 0x7f28d8164c40 con 0x7f28d80b44d0 2024-04-25T10:29:52.078 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:52.074+0000 7f28def6c640 1 -- --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_getmap magic: 0 -- 0x7f28d8164ac0 con 0x7f28d80d5160 2024-04-25T10:29:52.078 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:52.074+0000 7f28def6c640 1 -- --> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] -- mon_getmap magic: 0 -- 0x7f28d80d0da0 con 0x7f28d80d0990 2024-04-25T10:29:52.078 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:52.074+0000 7f28dd509640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f28d80b44d0 0x7f28d8153890 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:29:52.078 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:52.074+0000 7f28dd509640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f28d80b44d0 0x7f28d8153890 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.37:3300/0 says I am v2:172.21.15.37:36300/0 (socket says 172.21.15.37:36300) 2024-04-25T10:29:52.078 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:52.074+0000 7f28dd509640 1 -- 172.21.15.37:0/1604398350 learned_addr learned my addr 172.21.15.37:0/1604398350 (peer_addr_for_me v2:172.21.15.37:0/0) 2024-04-25T10:29:52.078 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:52.074+0000 7f28dcd08640 1 --2- 172.21.15.37:0/1604398350 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f28d80d0990 0x7f28d80cf7b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:29:52.078 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:52.074+0000 7f28d7fff640 1 --2- 172.21.15.37:0/1604398350 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f28d80d5160 0x7f28d8153350 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:29:52.078 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:52.075+0000 7f28dd509640 1 -- 172.21.15.37:0/1604398350 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f28d80d0990 msgr2=0x7f28d80cf7b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:29:52.078 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:52.075+0000 7f28dd509640 1 --2- 172.21.15.37:0/1604398350 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f28d80d0990 0x7f28d80cf7b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:29:52.078 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:52.075+0000 7f28dd509640 1 -- 172.21.15.37:0/1604398350 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f28d80d5160 msgr2=0x7f28d8153350 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:29:52.078 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:52.075+0000 7f28dd509640 1 --2- 172.21.15.37:0/1604398350 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f28d80d5160 0x7f28d8153350 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:29:52.079 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:52.075+0000 7f28dd509640 1 -- 172.21.15.37:0/1604398350 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f28d8156c40 con 0x7f28d80b44d0 2024-04-25T10:29:52.079 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:52.075+0000 7f28dd509640 1 --2- 172.21.15.37:0/1604398350 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f28d80b44d0 0x7f28d8153890 secure :-1 s=READY pgs=310 cs=0 l=1 rev1=1 crypto rx=0x7f28c000d5d0 tx=0x7f28c000da90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:29:52.079 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:52.075+0000 7f28d57fa640 1 -- 172.21.15.37:0/1604398350 <== mon.0 v2:172.21.15.37:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f28c00125f0 con 0x7f28d80b44d0 2024-04-25T10:29:52.079 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:52.075+0000 7f28d57fa640 1 -- 172.21.15.37:0/1604398350 <== mon.0 v2:172.21.15.37:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f28c0019070 con 0x7f28d80b44d0 2024-04-25T10:29:52.079 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:52.075+0000 7f28def6c640 1 -- 172.21.15.37:0/1604398350 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f28d8159090 con 0x7f28d80b44d0 2024-04-25T10:29:52.079 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:52.075+0000 7f28d57fa640 1 -- 172.21.15.37:0/1604398350 <== mon.0 v2:172.21.15.37:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f28c00129f0 con 0x7f28d80b44d0 2024-04-25T10:29:52.079 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:52.075+0000 7f28def6c640 1 -- 172.21.15.37:0/1604398350 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f28d81597c0 con 0x7f28d80b44d0 2024-04-25T10:29:52.080 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:52.076+0000 7f28def6c640 1 -- 172.21.15.37:0/1604398350 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f28d806a140 con 0x7f28d80b44d0 2024-04-25T10:29:52.080 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:52.077+0000 7f28d57fa640 1 -- 172.21.15.37:0/1604398350 <== mon.0 v2:172.21.15.37:3300/0 4 ==== mgrmap(e 5) ==== 96782+0+0 (secure 0 0 0) 0x7f28c0012b90 con 0x7f28d80b44d0 2024-04-25T10:29:52.080 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:52.077+0000 7f28d57fa640 1 --2- 172.21.15.37:0/1604398350 >> [v2:172.21.15.203:6832/2418205237,v1:172.21.15.203:6833/2418205237] conn(0x7f28b80744b0 0x7f28b80769c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:29:52.080 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:52.077+0000 7f28d57fa640 1 -- 172.21.15.37:0/1604398350 <== mon.0 v2:172.21.15.37:3300/0 5 ==== osd_map(25..25 src has 1..25) ==== 5738+0+0 (secure 0 0 0) 0x7f28c0095440 con 0x7f28d80b44d0 2024-04-25T10:29:52.080 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:52.077+0000 7f28dcd08640 1 --2- 172.21.15.37:0/1604398350 >> [v2:172.21.15.203:6832/2418205237,v1:172.21.15.203:6833/2418205237] conn(0x7f28b80744b0 0x7f28b80769c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:29:52.080 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:52.078+0000 7f28dcd08640 1 --2- 172.21.15.37:0/1604398350 >> [v2:172.21.15.203:6832/2418205237,v1:172.21.15.203:6833/2418205237] conn(0x7f28b80744b0 0x7f28b80769c0 secure :-1 s=READY pgs=162 cs=0 l=1 rev1=1 crypto rx=0x7f28d0034000 tx=0x7f28d002e000 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:29:52.083 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:52.082+0000 7f28d57fa640 1 -- 172.21.15.37:0/1604398350 <== mon.0 v2:172.21.15.37:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187246 (secure 0 0 0) 0x7f28c005f600 con 0x7f28d80b44d0 2024-04-25T10:29:52.272 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:52.270+0000 7f28def6c640 1 -- 172.21.15.37:0/1604398350 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"} v 0) -- 0x7f28d80b5140 con 0x7f28d80b44d0 2024-04-25T10:29:52.273 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:52.270+0000 7f28d57fa640 1 -- 172.21.15.37:0/1604398350 <== mon.0 v2:172.21.15.37:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]=0 v25) ==== 100+0+11 (secure 0 0 0) 0x7f28c0063900 con 0x7f28d80b44d0 2024-04-25T10:29:52.273 INFO:teuthology.orchestra.run.smithi037.stdout:pg_num: 64 2024-04-25T10:29:52.275 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:52.273+0000 7f28def6c640 1 -- 172.21.15.37:0/1604398350 >> [v2:172.21.15.203:6832/2418205237,v1:172.21.15.203:6833/2418205237] conn(0x7f28b80744b0 msgr2=0x7f28b80769c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:29:52.275 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:52.273+0000 7f28def6c640 1 --2- 172.21.15.37:0/1604398350 >> [v2:172.21.15.203:6832/2418205237,v1:172.21.15.203:6833/2418205237] conn(0x7f28b80744b0 0x7f28b80769c0 secure :-1 s=READY pgs=162 cs=0 l=1 rev1=1 crypto rx=0x7f28d0034000 tx=0x7f28d002e000 comp rx=0 tx=0).stop 2024-04-25T10:29:52.275 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:52.273+0000 7f28def6c640 1 -- 172.21.15.37:0/1604398350 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f28d80b44d0 msgr2=0x7f28d8153890 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:29:52.275 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:52.273+0000 7f28def6c640 1 --2- 172.21.15.37:0/1604398350 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f28d80b44d0 0x7f28d8153890 secure :-1 s=READY pgs=310 cs=0 l=1 rev1=1 crypto rx=0x7f28c000d5d0 tx=0x7f28c000da90 comp rx=0 tx=0).stop 2024-04-25T10:29:52.275 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:52.273+0000 7f28def6c640 1 -- 172.21.15.37:0/1604398350 shutdown_connections 2024-04-25T10:29:52.275 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:52.273+0000 7f28def6c640 1 --2- 172.21.15.37:0/1604398350 >> [v2:172.21.15.203:6832/2418205237,v1:172.21.15.203:6833/2418205237] conn(0x7f28b80744b0 0x7f28b80769c0 unknown :-1 s=CLOSED pgs=162 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:29:52.276 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:52.273+0000 7f28def6c640 1 --2- 172.21.15.37:0/1604398350 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f28d80b44d0 0x7f28d8153890 unknown :-1 s=CLOSED pgs=310 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:29:52.276 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:52.273+0000 7f28def6c640 1 --2- 172.21.15.37:0/1604398350 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f28d80d5160 0x7f28d8153350 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:29:52.276 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:52.273+0000 7f28def6c640 1 --2- 172.21.15.37:0/1604398350 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f28d80d0990 0x7f28d80cf7b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:29:52.276 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:52.273+0000 7f28def6c640 1 -- 172.21.15.37:0/1604398350 >> 172.21.15.37:0/1604398350 conn(0x7f28d80b7a80 msgr2=0x7f28d80b6e30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:29:52.276 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:52.273+0000 7f28def6c640 1 -- 172.21.15.37:0/1604398350 shutdown_connections 2024-04-25T10:29:52.276 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:52.273+0000 7f28def6c640 1 -- 172.21.15.37:0/1604398350 wait complete. 2024-04-25T10:29:52.286 DEBUG:teuthology.orchestra.run.smithi037:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd blocklist ls 2024-04-25T10:29:52.399 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:52.397+0000 7f522e7a8640 1 Processor -- start 2024-04-25T10:29:52.399 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:52.397+0000 7f522e7a8640 1 -- start start 2024-04-25T10:29:52.403 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:52.398+0000 7f522e7a8640 1 --2- >> v2:172.21.15.203:3300/0 conn(0x7f522806c8f0 0x7f522809b700 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:29:52.404 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:52.398+0000 7f522e7a8640 1 --2- >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f522806cd00 0x7f522809bc40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:29:52.404 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:52.398+0000 7f522e7a8640 1 -- --> v1:172.21.15.37:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f52280b4240 con 0x7f522806e390 2024-04-25T10:29:52.404 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:52.398+0000 7f522e7a8640 1 -- --> v2:172.21.15.203:3300/0 -- mon_getmap magic: 0 -- 0x7f522806eb00 con 0x7f522806c8f0 2024-04-25T10:29:52.404 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:52.398+0000 7f522e7a8640 1 -- --> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] -- mon_getmap magic: 0 -- 0x7f522806ede0 con 0x7f522806cd00 2024-04-25T10:29:52.404 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:52.399+0000 7f5227fff640 1 --1- >> v1:172.21.15.37:6789/0 conn(0x7f522806e390 0x7f522815eec0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.37:6789/0 says I am v1:172.21.15.37:56798/0 (socket says 172.21.15.37:56798) 2024-04-25T10:29:52.404 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:52.399+0000 7f5227fff640 1 -- 172.21.15.37:0/981749822 learned_addr learned my addr 172.21.15.37:0/981749822 (peer_addr_for_me v1:172.21.15.37:0/0) 2024-04-25T10:29:52.404 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:52.399+0000 7f52277fe640 1 --2- 172.21.15.37:0/981749822 >> v2:172.21.15.203:3300/0 conn(0x7f522806c8f0 0x7f522809b700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:29:52.404 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:52.399+0000 7f522cd45640 1 --2- 172.21.15.37:0/981749822 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f522806cd00 0x7f522809bc40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:29:52.404 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:52.399+0000 7f5226ffd640 1 -- 172.21.15.37:0/981749822 <== mon.0 v1:172.21.15.37:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 2019012680 0 0) 0x7f52280b4240 con 0x7f522806e390 2024-04-25T10:29:52.404 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:52.399+0000 7f5226ffd640 1 -- 172.21.15.37:0/981749822 --> v1:172.21.15.37:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f5210003520 con 0x7f522806e390 2024-04-25T10:29:52.404 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:52.399+0000 7f5226ffd640 1 -- 172.21.15.37:0/981749822 <== mon.0 v1:172.21.15.37:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 2661648489 0 0) 0x7f5210003520 con 0x7f522806e390 2024-04-25T10:29:52.404 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:52.399+0000 7f5226ffd640 1 -- 172.21.15.37:0/981749822 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f522806cd00 msgr2=0x7f522809bc40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:29:52.404 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:52.399+0000 7f5226ffd640 1 --2- 172.21.15.37:0/981749822 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f522806cd00 0x7f522809bc40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:29:52.404 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:52.399+0000 7f5226ffd640 1 -- 172.21.15.37:0/981749822 >> v2:172.21.15.203:3300/0 conn(0x7f522806c8f0 msgr2=0x7f522809b700 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:29:52.405 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:52.399+0000 7f5226ffd640 1 --2- 172.21.15.37:0/981749822 >> v2:172.21.15.203:3300/0 conn(0x7f522806c8f0 0x7f522809b700 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:29:52.405 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:52.399+0000 7f5226ffd640 1 -- 172.21.15.37:0/981749822 --> v1:172.21.15.37:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f522815e6c0 con 0x7f522806e390 2024-04-25T10:29:52.405 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:52.399+0000 7f5226ffd640 1 -- 172.21.15.37:0/981749822 <== mon.0 v1:172.21.15.37:6789/0 3 ==== mon_map magic: 0 ==== 413+0+0 (unknown 1577794600 0 0) 0x7f5218001670 con 0x7f522806e390 2024-04-25T10:29:52.405 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:52.399+0000 7f5226ffd640 1 -- 172.21.15.37:0/981749822 >> v1:172.21.15.37:6789/0 conn(0x7f522806e390 legacy=0x7f522815eec0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:29:52.405 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:52.399+0000 7f5226ffd640 1 --2- 172.21.15.37:0/981749822 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f52100039b0 0x7f5210003e20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:29:52.405 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:52.399+0000 7f5226ffd640 1 --2- 172.21.15.37:0/981749822 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f5210004800 0x7f5210004c10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:29:52.405 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:52.399+0000 7f5226ffd640 1 --2- 172.21.15.37:0/981749822 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f522806cd00 0x7f5210005150 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:29:52.405 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:52.399+0000 7f5226ffd640 1 -- 172.21.15.37:0/981749822 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 -- 0x7f522806eb00 con 0x7f52100039b0 2024-04-25T10:29:52.405 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:52.399+0000 7f5226ffd640 1 -- 172.21.15.37:0/981749822 --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_getmap magic: 0 -- 0x7f522806ede0 con 0x7f5210004800 2024-04-25T10:29:52.405 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:52.399+0000 7f5226ffd640 1 -- 172.21.15.37:0/981749822 --> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] -- mon_getmap magic: 0 -- 0x7f521000b9c0 con 0x7f522806cd00 2024-04-25T10:29:52.405 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:52.399+0000 7f5227fff640 1 --2- 172.21.15.37:0/981749822 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f52100039b0 0x7f5210003e20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:29:52.405 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:52.400+0000 7f522cd45640 1 --2- 172.21.15.37:0/981749822 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f522806cd00 0x7f5210005150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:29:52.405 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:52.399+0000 7f52277fe640 1 --2- 172.21.15.37:0/981749822 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f5210004800 0x7f5210004c10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:29:52.405 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:52.400+0000 7f5227fff640 1 -- 172.21.15.37:0/981749822 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f522806cd00 msgr2=0x7f5210005150 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:29:52.406 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:52.400+0000 7f5227fff640 1 --2- 172.21.15.37:0/981749822 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f522806cd00 0x7f5210005150 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:29:52.406 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:52.400+0000 7f5227fff640 1 -- 172.21.15.37:0/981749822 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f5210004800 msgr2=0x7f5210004c10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:29:52.406 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:52.400+0000 7f5227fff640 1 --2- 172.21.15.37:0/981749822 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f5210004800 0x7f5210004c10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:29:52.406 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:52.400+0000 7f5227fff640 1 -- 172.21.15.37:0/981749822 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f5210003520 con 0x7f52100039b0 2024-04-25T10:29:52.406 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:52.400+0000 7f5227fff640 1 --2- 172.21.15.37:0/981749822 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f52100039b0 0x7f5210003e20 secure :-1 s=READY pgs=312 cs=0 l=1 rev1=1 crypto rx=0x7f5218007fd0 tx=0x7f5210005830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=96732cdaed416e2c server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:29:52.406 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:52.401+0000 7f5226ffd640 1 -- 172.21.15.37:0/981749822 <== mon.0 v2:172.21.15.37:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5218007d50 con 0x7f52100039b0 2024-04-25T10:29:52.406 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:52.401+0000 7f5226ffd640 1 -- 172.21.15.37:0/981749822 <== mon.0 v2:172.21.15.37:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f5218005af0 con 0x7f52100039b0 2024-04-25T10:29:52.406 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:52.401+0000 7f5226ffd640 1 -- 172.21.15.37:0/981749822 <== mon.0 v2:172.21.15.37:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5218007070 con 0x7f52100039b0 2024-04-25T10:29:52.406 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:52.401+0000 7f522e7a8640 1 -- 172.21.15.37:0/981749822 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f52100039b0 msgr2=0x7f5210003e20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:29:52.406 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:52.401+0000 7f522e7a8640 1 --2- 172.21.15.37:0/981749822 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f52100039b0 0x7f5210003e20 secure :-1 s=READY pgs=312 cs=0 l=1 rev1=1 crypto rx=0x7f5218007fd0 tx=0x7f5210005830 comp rx=0 tx=0).stop 2024-04-25T10:29:52.406 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:52.401+0000 7f522cd45640 1 -- 172.21.15.37:0/981749822 reap_dead start 2024-04-25T10:29:52.406 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:52.401+0000 7f522e7a8640 1 -- 172.21.15.37:0/981749822 shutdown_connections 2024-04-25T10:29:52.406 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:52.401+0000 7f522e7a8640 1 -- 172.21.15.37:0/981749822 >> 172.21.15.37:0/981749822 conn(0x7f52280b7600 msgr2=0x7f52280b7c30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:29:52.406 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:52.401+0000 7f522e7a8640 1 -- 172.21.15.37:0/981749822 shutdown_connections 2024-04-25T10:29:52.407 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:52.401+0000 7f522e7a8640 1 -- 172.21.15.37:0/981749822 wait complete. 2024-04-25T10:29:52.409 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:52.403+0000 7f522e7a8640 1 Processor -- start 2024-04-25T10:29:52.409 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:52.403+0000 7f522e7a8640 1 -- start start 2024-04-25T10:29:52.409 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:52.405+0000 7f522e7a8640 1 --2- >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f52100039b0 0x7f52280d03d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:29:52.409 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:52.405+0000 7f522e7a8640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f5228159d80 0x7f52281f3fe0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:29:52.409 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:52.405+0000 7f522e7a8640 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f522809ccd0 0x7f522809d0e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:29:52.409 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:52.405+0000 7f522e7a8640 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 -- 0x7f522815bd00 con 0x7f5228159d80 2024-04-25T10:29:52.409 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:52.405+0000 7f522e7a8640 1 -- --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_getmap magic: 0 -- 0x7f522806e7a0 con 0x7f522809ccd0 2024-04-25T10:29:52.409 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:52.405+0000 7f522e7a8640 1 -- --> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] -- mon_getmap magic: 0 -- 0x7f522815be80 con 0x7f52100039b0 2024-04-25T10:29:52.409 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:52.405+0000 7f52277fe640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f5228159d80 0x7f52281f3fe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:29:52.409 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:52.405+0000 7f52277fe640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f5228159d80 0x7f52281f3fe0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.37:3300/0 says I am v2:172.21.15.37:36322/0 (socket says 172.21.15.37:36322) 2024-04-25T10:29:52.410 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:52.405+0000 7f52277fe640 1 -- 172.21.15.37:0/548828062 learned_addr learned my addr 172.21.15.37:0/548828062 (peer_addr_for_me v2:172.21.15.37:0/0) 2024-04-25T10:29:52.410 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:52.405+0000 7f5227fff640 1 --2- 172.21.15.37:0/548828062 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f52100039b0 0x7f52280d03d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:29:52.410 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:52.405+0000 7f522cd45640 1 --2- 172.21.15.37:0/548828062 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f522809ccd0 0x7f522809d0e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:29:52.410 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:52.405+0000 7f52277fe640 1 -- 172.21.15.37:0/548828062 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f52100039b0 msgr2=0x7f52280d03d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:29:52.410 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:52.405+0000 7f52277fe640 1 --2- 172.21.15.37:0/548828062 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f52100039b0 0x7f52280d03d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:29:52.410 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:52.405+0000 7f52277fe640 1 -- 172.21.15.37:0/548828062 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f522809ccd0 msgr2=0x7f522809d0e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:29:52.410 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:52.405+0000 7f52277fe640 1 --2- 172.21.15.37:0/548828062 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f522809ccd0 0x7f522809d0e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:29:52.410 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:52.405+0000 7f52277fe640 1 -- 172.21.15.37:0/548828062 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f5228150880 con 0x7f5228159d80 2024-04-25T10:29:52.410 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:52.405+0000 7f5227fff640 1 --2- 172.21.15.37:0/548828062 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f52100039b0 0x7f52280d03d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T10:29:52.410 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:52.405+0000 7f52277fe640 1 --2- 172.21.15.37:0/548828062 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f5228159d80 0x7f52281f3fe0 secure :-1 s=READY pgs=313 cs=0 l=1 rev1=1 crypto rx=0x7f521c011990 tx=0x7f521c011e50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:29:52.410 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:52.405+0000 7f5224ff9640 1 -- 172.21.15.37:0/548828062 <== mon.0 v2:172.21.15.37:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f521c019070 con 0x7f5228159d80 2024-04-25T10:29:52.410 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:52.405+0000 7f5224ff9640 1 -- 172.21.15.37:0/548828062 <== mon.0 v2:172.21.15.37:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f521c004240 con 0x7f5228159d80 2024-04-25T10:29:52.410 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:52.405+0000 7f522e7a8640 1 -- 172.21.15.37:0/548828062 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f52280d0910 con 0x7f5228159d80 2024-04-25T10:29:52.410 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:52.405+0000 7f5224ff9640 1 -- 172.21.15.37:0/548828062 <== mon.0 v2:172.21.15.37:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f521c00f860 con 0x7f5228159d80 2024-04-25T10:29:52.411 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:52.405+0000 7f522e7a8640 1 -- 172.21.15.37:0/548828062 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f522809ddc0 con 0x7f5228159d80 2024-04-25T10:29:52.411 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:52.406+0000 7f522e7a8640 1 -- 172.21.15.37:0/548828062 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f522806a140 con 0x7f5228159d80 2024-04-25T10:29:52.411 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:52.407+0000 7f5224ff9640 1 -- 172.21.15.37:0/548828062 <== mon.0 v2:172.21.15.37:3300/0 4 ==== mgrmap(e 5) ==== 96782+0+0 (secure 0 0 0) 0x7f521c01c070 con 0x7f5228159d80 2024-04-25T10:29:52.411 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:52.407+0000 7f5224ff9640 1 --2- 172.21.15.37:0/548828062 >> [v2:172.21.15.203:6832/2418205237,v1:172.21.15.203:6833/2418205237] conn(0x7f52140744b0 0x7f52140769c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:29:52.411 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:52.408+0000 7f5224ff9640 1 -- 172.21.15.37:0/548828062 <== mon.0 v2:172.21.15.37:3300/0 5 ==== osd_map(25..25 src has 1..25) ==== 5738+0+0 (secure 0 0 0) 0x7f521c095580 con 0x7f5228159d80 2024-04-25T10:29:52.412 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:52.408+0000 7f5227fff640 1 --2- 172.21.15.37:0/548828062 >> [v2:172.21.15.203:6832/2418205237,v1:172.21.15.203:6833/2418205237] conn(0x7f52140744b0 0x7f52140769c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:29:52.412 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:52.409+0000 7f5227fff640 1 --2- 172.21.15.37:0/548828062 >> [v2:172.21.15.203:6832/2418205237,v1:172.21.15.203:6833/2418205237] conn(0x7f52140744b0 0x7f52140769c0 secure :-1 s=READY pgs=163 cs=0 l=1 rev1=1 crypto rx=0x7f5218033000 tx=0x7f5218009030 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:29:52.414 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:52.412+0000 7f5224ff9640 1 -- 172.21.15.37:0/548828062 <== mon.0 v2:172.21.15.37:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187246 (secure 0 0 0) 0x7f521c05fd80 con 0x7f5228159d80 2024-04-25T10:29:52.610 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:52.608+0000 7f522e7a8640 1 -- 172.21.15.37:0/548828062 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_command({"prefix": "osd blocklist ls"} v 0) -- 0x7f52280cfff0 con 0x7f5228159d80 2024-04-25T10:29:52.611 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:52.608+0000 7f5224ff9640 1 -- 172.21.15.37:0/548828062 <== mon.0 v2:172.21.15.37:3300/0 7 ==== mon_command_ack([{"prefix": "osd blocklist ls"}]=0 listed 0 entries v25) ==== 80+0+0 (secure 0 0 0) 0x7f521c05f680 con 0x7f5228159d80 2024-04-25T10:29:52.611 INFO:teuthology.orchestra.run.smithi037.stderr:listed 0 entries 2024-04-25T10:29:52.613 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:52.611+0000 7f522e7a8640 1 -- 172.21.15.37:0/548828062 >> [v2:172.21.15.203:6832/2418205237,v1:172.21.15.203:6833/2418205237] conn(0x7f52140744b0 msgr2=0x7f52140769c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:29:52.613 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:52.611+0000 7f522e7a8640 1 --2- 172.21.15.37:0/548828062 >> [v2:172.21.15.203:6832/2418205237,v1:172.21.15.203:6833/2418205237] conn(0x7f52140744b0 0x7f52140769c0 secure :-1 s=READY pgs=163 cs=0 l=1 rev1=1 crypto rx=0x7f5218033000 tx=0x7f5218009030 comp rx=0 tx=0).stop 2024-04-25T10:29:52.613 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:52.611+0000 7f522e7a8640 1 -- 172.21.15.37:0/548828062 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f5228159d80 msgr2=0x7f52281f3fe0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:29:52.614 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:52.611+0000 7f522e7a8640 1 --2- 172.21.15.37:0/548828062 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f5228159d80 0x7f52281f3fe0 secure :-1 s=READY pgs=313 cs=0 l=1 rev1=1 crypto rx=0x7f521c011990 tx=0x7f521c011e50 comp rx=0 tx=0).stop 2024-04-25T10:29:52.614 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:52.611+0000 7f522e7a8640 1 -- 172.21.15.37:0/548828062 shutdown_connections 2024-04-25T10:29:52.614 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:52.611+0000 7f522e7a8640 1 --2- 172.21.15.37:0/548828062 >> [v2:172.21.15.203:6832/2418205237,v1:172.21.15.203:6833/2418205237] conn(0x7f52140744b0 0x7f52140769c0 unknown :-1 s=CLOSED pgs=163 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:29:52.614 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:52.611+0000 7f522e7a8640 1 --2- 172.21.15.37:0/548828062 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f522809ccd0 0x7f522809d0e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:29:52.614 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:52.611+0000 7f522e7a8640 1 --2- 172.21.15.37:0/548828062 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f5228159d80 0x7f52281f3fe0 unknown :-1 s=CLOSED pgs=313 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:29:52.614 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:52.611+0000 7f522e7a8640 1 --2- 172.21.15.37:0/548828062 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f52100039b0 0x7f52280d03d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:29:52.614 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:52.611+0000 7f522e7a8640 1 -- 172.21.15.37:0/548828062 >> 172.21.15.37:0/548828062 conn(0x7f52280b7600 msgr2=0x7f52280b1ab0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:29:52.614 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:52.611+0000 7f522e7a8640 1 -- 172.21.15.37:0/548828062 shutdown_connections 2024-04-25T10:29:52.614 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:52.611+0000 7f522e7a8640 1 -- 172.21.15.37:0/548828062 wait complete. 2024-04-25T10:29:52.624 DEBUG:teuthology.orchestra.run.smithi037:> set -ex 2024-04-25T10:29:52.624 DEBUG:teuthology.orchestra.run.smithi037:> dd if=/proc/self/mounts of=/dev/stdout 2024-04-25T10:29:52.640 DEBUG:teuthology.orchestra.run.smithi037:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-04-25T10:29:52.791 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:52.788+0000 7fb19a4e7640 1 Processor -- start 2024-04-25T10:29:52.791 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:52.788+0000 7fb19a4e7640 1 -- start start 2024-04-25T10:29:52.793 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:52.790+0000 7fb19a4e7640 1 --2- >> v2:172.21.15.203:3300/0 conn(0x7fb1940b2dc0 0x7fb1940d2860 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:29:52.793 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:52.790+0000 7fb19a4e7640 1 --2- >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7fb19406daf0 0x7fb19406df00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:29:52.793 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:52.790+0000 7fb19a4e7640 1 -- --> v1:172.21.15.203:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fb1940d2e40 con 0x7fb19406e440 2024-04-25T10:29:52.793 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:52.790+0000 7fb19a4e7640 1 -- --> v2:172.21.15.203:3300/0 -- mon_getmap magic: 0 -- 0x7fb1940b92c0 con 0x7fb1940b2dc0 2024-04-25T10:29:52.793 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:52.790+0000 7fb19a4e7640 1 -- --> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] -- mon_getmap magic: 0 -- 0x7fb1940ce0d0 con 0x7fb19406daf0 2024-04-25T10:29:52.793 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:52.790+0000 7fb193fff640 1 --2- >> v2:172.21.15.203:3300/0 conn(0x7fb1940b2dc0 0x7fb1940d2860 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:29:52.793 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:52.790+0000 7fb1937fe640 1 --2- >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7fb19406daf0 0x7fb19406df00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:29:52.793 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:52.790+0000 7fb198a84640 1 --1- >> v1:172.21.15.203:6789/0 conn(0x7fb19406e440 0x7fb19408cc90 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.203:6789/0 says I am v1:172.21.15.37:52662/0 (socket says 172.21.15.37:52662) 2024-04-25T10:29:52.793 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:52.790+0000 7fb193fff640 1 --2- >> v2:172.21.15.203:3300/0 conn(0x7fb1940b2dc0 0x7fb1940d2860 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.203:3300/0 says I am v2:172.21.15.37:33464/0 (socket says 172.21.15.37:33464) 2024-04-25T10:29:52.793 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:52.790+0000 7fb193fff640 1 -- 172.21.15.37:0/4044674282 learned_addr learned my addr 172.21.15.37:0/4044674282 (peer_addr_for_me v2:172.21.15.37:0/0) 2024-04-25T10:29:52.793 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:52.791+0000 7fb193fff640 1 -- 172.21.15.37:0/4044674282 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7fb19406daf0 msgr2=0x7fb19406df00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:29:52.793 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:52.791+0000 7fb193fff640 1 --2- 172.21.15.37:0/4044674282 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7fb19406daf0 0x7fb19406df00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:29:52.794 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:52.791+0000 7fb193fff640 1 -- 172.21.15.37:0/4044674282 >> v1:172.21.15.203:6789/0 conn(0x7fb19406e440 legacy=0x7fb19408cc90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:29:52.794 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:52.791+0000 7fb193fff640 1 -- 172.21.15.37:0/4044674282 --> v2:172.21.15.203:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fb1940d3120 con 0x7fb1940b2dc0 2024-04-25T10:29:52.794 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:52.791+0000 7fb192ffd640 1 -- 172.21.15.37:0/4044674282 <== mon.1 v1:172.21.15.203:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 2858638386 0 0) 0x7fb1940d2e40 con 0x7fb19406e440 2024-04-25T10:29:52.794 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:52.791+0000 7fb1937fe640 1 --2- 172.21.15.37:0/4044674282 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7fb19406daf0 0x7fb19406df00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-25T10:29:52.795 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:52.791+0000 7fb193fff640 1 --2- 172.21.15.37:0/4044674282 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fb1940b2dc0 0x7fb1940d2860 secure :-1 s=READY pgs=142 cs=0 l=1 rev1=1 crypto rx=0x7fb184009850 tx=0x7fb18402a250 comp rx=0 tx=0).ready entity=mon.1 client_cookie=f4842bc4d09e42e5 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:29:52.795 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:52.792+0000 7fb192ffd640 1 -- 172.21.15.37:0/4044674282 <== mon.1 v2:172.21.15.203:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb184004030 con 0x7fb1940b2dc0 2024-04-25T10:29:52.795 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:52.792+0000 7fb192ffd640 1 -- 172.21.15.37:0/4044674282 <== mon.1 v2:172.21.15.203:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fb1840041d0 con 0x7fb1940b2dc0 2024-04-25T10:29:52.795 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:52.792+0000 7fb192ffd640 1 -- 172.21.15.37:0/4044674282 <== mon.1 v2:172.21.15.203:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb184003610 con 0x7fb1940b2dc0 2024-04-25T10:29:52.795 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:52.792+0000 7fb19a4e7640 1 -- 172.21.15.37:0/4044674282 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fb1940b2dc0 msgr2=0x7fb1940d2860 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:29:52.795 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:52.792+0000 7fb19a4e7640 1 --2- 172.21.15.37:0/4044674282 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fb1940b2dc0 0x7fb1940d2860 secure :-1 s=READY pgs=142 cs=0 l=1 rev1=1 crypto rx=0x7fb184009850 tx=0x7fb18402a250 comp rx=0 tx=0).stop 2024-04-25T10:29:52.795 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:52.792+0000 7fb19a4e7640 1 -- 172.21.15.37:0/4044674282 shutdown_connections 2024-04-25T10:29:52.795 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:52.792+0000 7fb19a4e7640 1 --2- 172.21.15.37:0/4044674282 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7fb19406daf0 0x7fb19406df00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:29:52.796 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:52.792+0000 7fb19a4e7640 1 --2- 172.21.15.37:0/4044674282 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fb1940b2dc0 0x7fb1940d2860 unknown :-1 s=CLOSED pgs=142 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:29:52.796 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:52.792+0000 7fb19a4e7640 1 -- 172.21.15.37:0/4044674282 >> 172.21.15.37:0/4044674282 conn(0x7fb1940b5440 msgr2=0x7fb1940b5850 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:29:52.796 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:52.792+0000 7fb19a4e7640 1 -- 172.21.15.37:0/4044674282 shutdown_connections 2024-04-25T10:29:52.796 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:52.792+0000 7fb19a4e7640 1 -- 172.21.15.37:0/4044674282 wait complete. 2024-04-25T10:29:52.796 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:52.794+0000 7fb19a4e7640 1 Processor -- start 2024-04-25T10:29:52.796 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:52.794+0000 7fb19a4e7640 1 -- start start 2024-04-25T10:29:52.799 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:52.796+0000 7fb19a4e7640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fb19406daf0 0x7fb194162410 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:29:52.799 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:52.796+0000 7fb19a4e7640 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fb19406e440 0x7fb194160b20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:29:52.799 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:52.796+0000 7fb19a4e7640 1 --2- >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7fb1940b2dc0 0x7fb194161060 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:29:52.799 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:52.796+0000 7fb19a4e7640 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 -- 0x7fb19406e850 con 0x7fb19406daf0 2024-04-25T10:29:52.799 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:52.796+0000 7fb19a4e7640 1 -- --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_getmap magic: 0 -- 0x7fb1940d36e0 con 0x7fb19406e440 2024-04-25T10:29:52.799 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:52.796+0000 7fb19a4e7640 1 -- --> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] -- mon_getmap magic: 0 -- 0x7fb1940cf570 con 0x7fb1940b2dc0 2024-04-25T10:29:52.799 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:52.796+0000 7fb193fff640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fb19406daf0 0x7fb194162410 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:29:52.800 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:52.796+0000 7fb193fff640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fb19406daf0 0x7fb194162410 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.37:3300/0 says I am v2:172.21.15.37:36334/0 (socket says 172.21.15.37:36334) 2024-04-25T10:29:52.800 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:52.796+0000 7fb193fff640 1 -- 172.21.15.37:0/4624382 learned_addr learned my addr 172.21.15.37:0/4624382 (peer_addr_for_me v2:172.21.15.37:0/0) 2024-04-25T10:29:52.800 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:52.796+0000 7fb1937fe640 1 --2- 172.21.15.37:0/4624382 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fb19406e440 0x7fb194160b20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:29:52.800 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:52.796+0000 7fb198a84640 1 --2- 172.21.15.37:0/4624382 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7fb1940b2dc0 0x7fb194161060 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:29:52.800 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:52.796+0000 7fb193fff640 1 -- 172.21.15.37:0/4624382 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7fb1940b2dc0 msgr2=0x7fb194161060 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:29:52.800 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:52.796+0000 7fb193fff640 1 --2- 172.21.15.37:0/4624382 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7fb1940b2dc0 0x7fb194161060 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:29:52.800 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:52.796+0000 7fb193fff640 1 -- 172.21.15.37:0/4624382 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fb19406e440 msgr2=0x7fb194160b20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:29:52.800 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:52.796+0000 7fb193fff640 1 --2- 172.21.15.37:0/4624382 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fb19406e440 0x7fb194160b20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:29:52.800 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:52.796+0000 7fb193fff640 1 -- 172.21.15.37:0/4624382 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fb1940c8850 con 0x7fb19406daf0 2024-04-25T10:29:52.800 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:52.796+0000 7fb193fff640 1 --2- 172.21.15.37:0/4624382 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fb19406daf0 0x7fb194162410 secure :-1 s=READY pgs=314 cs=0 l=1 rev1=1 crypto rx=0x7fb184000c00 tx=0x7fb1840046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:29:52.800 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:52.796+0000 7fb190ff9640 1 -- 172.21.15.37:0/4624382 <== mon.0 v2:172.21.15.37:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb184041020 con 0x7fb19406daf0 2024-04-25T10:29:52.800 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:52.796+0000 7fb190ff9640 1 -- 172.21.15.37:0/4624382 <== mon.0 v2:172.21.15.37:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fb184003c50 con 0x7fb19406daf0 2024-04-25T10:29:52.800 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:52.797+0000 7fb19a4e7640 1 -- 172.21.15.37:0/4624382 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fb1940c8b30 con 0x7fb19406daf0 2024-04-25T10:29:52.800 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:52.797+0000 7fb190ff9640 1 -- 172.21.15.37:0/4624382 <== mon.0 v2:172.21.15.37:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb184004030 con 0x7fb19406daf0 2024-04-25T10:29:52.801 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:52.797+0000 7fb19a4e7640 1 -- 172.21.15.37:0/4624382 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fb194161ba0 con 0x7fb19406daf0 2024-04-25T10:29:52.801 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:52.798+0000 7fb19a4e7640 1 -- 172.21.15.37:0/4624382 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fb19406a140 con 0x7fb19406daf0 2024-04-25T10:29:52.801 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:52.798+0000 7fb190ff9640 1 -- 172.21.15.37:0/4624382 <== mon.0 v2:172.21.15.37:3300/0 4 ==== mgrmap(e 5) ==== 96782+0+0 (secure 0 0 0) 0x7fb184044020 con 0x7fb19406daf0 2024-04-25T10:29:52.802 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:52.799+0000 7fb190ff9640 1 --2- 172.21.15.37:0/4624382 >> [v2:172.21.15.203:6832/2418205237,v1:172.21.15.203:6833/2418205237] conn(0x7fb1800744e0 0x7fb1800769f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:29:52.802 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:52.799+0000 7fb190ff9640 1 -- 172.21.15.37:0/4624382 <== mon.0 v2:172.21.15.37:3300/0 5 ==== osd_map(25..25 src has 1..25) ==== 5738+0+0 (secure 0 0 0) 0x7fb1840b5e10 con 0x7fb19406daf0 2024-04-25T10:29:52.802 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:52.799+0000 7fb1937fe640 1 --2- 172.21.15.37:0/4624382 >> [v2:172.21.15.203:6832/2418205237,v1:172.21.15.203:6833/2418205237] conn(0x7fb1800744e0 0x7fb1800769f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:29:52.802 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:52.799+0000 7fb1937fe640 1 --2- 172.21.15.37:0/4624382 >> [v2:172.21.15.203:6832/2418205237,v1:172.21.15.203:6833/2418205237] conn(0x7fb1800744e0 0x7fb1800769f0 secure :-1 s=READY pgs=164 cs=0 l=1 rev1=1 crypto rx=0x7fb19408d730 tx=0x7fb188008040 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:29:52.805 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:52.803+0000 7fb190ff9640 1 -- 172.21.15.37:0/4624382 <== mon.0 v2:172.21.15.37:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187246 (secure 0 0 0) 0x7fb1840b9030 con 0x7fb19406daf0 2024-04-25T10:29:52.997 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:52.995+0000 7fb19a4e7640 1 -- 172.21.15.37:0/4624382 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7fb194072d70 con 0x7fb19406daf0 2024-04-25T10:29:52.999 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:52.996+0000 7fb190ff9640 1 -- 172.21.15.37:0/4624382 <== mon.0 v2:172.21.15.37:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v25) ==== 74+0+16670 (secure 0 0 0) 0x7fb18404b030 con 0x7fb19406daf0 2024-04-25T10:29:52.999 INFO:teuthology.orchestra.run.smithi037.stdout: 2024-04-25T10:29:52.999 INFO:teuthology.orchestra.run.smithi037.stdout:{"epoch":25,"fsid":"f4acda28-be34-492a-b286-994afd36f61d","created":"2024-04-25T10:28:43.985718+0000","modified":"2024-04-25T10:29:49.325719+0000","last_up_change":"2024-04-25T10:28:54.900654+0000","last_in_change":"2024-04-25T10:28:46.216973+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":3,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"luminous","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-04-25T10:28:56.202153+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"19","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2024-04-25T10:29:06.507244+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"22","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":2.25,"score_stable":2.25,"optimal_score":1,"raw_score_acting":2.25,"raw_score_stable":2.25,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2024-04-25T10:29:07.347844+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"23","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"a2f79028-b332-4166-9ebb-a7024a37ea22","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":21,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.37:6816","nonce":2746074720},{"type":"v1","addr":"172.21.15.37:6817","nonce":2746074720}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.37:6818","nonce":2746074720},{"type":"v1","addr":"172.21.15.37:6819","nonce":2746074720}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.37:6822","nonce":2746074720},{"type":"v1","addr":"172.21.15.37:6823","nonce":2746074720}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.37:6820","nonce":2746074720},{"type":"v1","addr":"172.21.15.37:6821","nonce":2746074720}]},"public_addr":"172.21.15.37:6817/2746074720","cluster_addr":"172.21.15.37:6819/2746074720","heartbeat_back_addr":"172.21.15.37:6823/2746074720","heartbeat_front_addr":"172.21.15.37:6821/2746074720","state":["exists","up"]},{"osd":1,"uuid":"973159de-c2ea-4083-aecc-69373d1c1132","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":21,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.37:6800","nonce":1690341662},{"type":"v1","addr":"172.21.15.37:6801","nonce":1690341662}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.37:6802","nonce":1690341662},{"type":"v1","addr":"172.21.15.37:6803","nonce":1690341662}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.37:6806","nonce":1690341662},{"type":"v1","addr":"172.21.15.37:6807","nonce":1690341662}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.37:6804","nonce":1690341662},{"type":"v1","addr":"172.21.15.37:6805","nonce":1690341662}]},"public_addr":"172.21.15.37:6801/1690341662","cluster_addr":"172.21.15.37:6803/1690341662","heartbeat_back_addr":"172.21.15.37:6807/1690341662","heartbeat_front_addr":"172.21.15.37:6805/1690341662","state":["exists","up"]},{"osd":2,"uuid":"4ae7a0e9-cb3f-484c-a75f-f3766b721c47","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":21,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.37:6808","nonce":2754669649},{"type":"v1","addr":"172.21.15.37:6809","nonce":2754669649}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.37:6810","nonce":2754669649},{"type":"v1","addr":"172.21.15.37:6811","nonce":2754669649}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.37:6814","nonce":2754669649},{"type":"v1","addr":"172.21.15.37:6815","nonce":2754669649}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.37:6812","nonce":2754669649},{"type":"v1","addr":"172.21.15.37:6813","nonce":2754669649}]},"public_addr":"172.21.15.37:6809/2754669649","cluster_addr":"172.21.15.37:6811/2754669649","heartbeat_back_addr":"172.21.15.37:6815/2754669649","heartbeat_front_addr":"172.21.15.37:6813/2754669649","state":["exists","up"]},{"osd":3,"uuid":"1ba4a746-1608-47a7-ace2-960a5cfae934","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":21,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.37:6824","nonce":1815290618},{"type":"v1","addr":"172.21.15.37:6825","nonce":1815290618}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.37:6826","nonce":1815290618},{"type":"v1","addr":"172.21.15.37:6827","nonce":1815290618}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.37:6830","nonce":1815290618},{"type":"v1","addr":"172.21.15.37:6831","nonce":1815290618}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.37:6828","nonce":1815290618},{"type":"v1","addr":"172.21.15.37:6829","nonce":1815290618}]},"public_addr":"172.21.15.37:6825/1815290618","cluster_addr":"172.21.15.37:6827/1815290618","heartbeat_back_addr":"172.21.15.37:6831/1815290618","heartbeat_front_addr":"172.21.15.37:6829/1815290618","state":["exists","up"]},{"osd":4,"uuid":"d4f59f5d-0413-4b79-87ee-bb212c1fcf62","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":21,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6800","nonce":4051532063},{"type":"v1","addr":"172.21.15.203:6801","nonce":4051532063}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6802","nonce":4051532063},{"type":"v1","addr":"172.21.15.203:6803","nonce":4051532063}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6806","nonce":4051532063},{"type":"v1","addr":"172.21.15.203:6807","nonce":4051532063}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6804","nonce":4051532063},{"type":"v1","addr":"172.21.15.203:6805","nonce":4051532063}]},"public_addr":"172.21.15.203:6801/4051532063","cluster_addr":"172.21.15.203:6803/4051532063","heartbeat_back_addr":"172.21.15.203:6807/4051532063","heartbeat_front_addr":"172.21.15.203:6805/4051532063","state":["exists","up"]},{"osd":5,"uuid":"69710a0e-41d8-4e7f-9a10-17ce540b67be","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":24,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6824","nonce":1203822890},{"type":"v1","addr":"172.21.15.203:6825","nonce":1203822890}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6826","nonce":1203822890},{"type":"v1","addr":"172.21.15.203:6827","nonce":1203822890}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6830","nonce":1203822890},{"type":"v1","addr":"172.21.15.203:6831","nonce":1203822890}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6828","nonce":1203822890},{"type":"v1","addr":"172.21.15.203:6829","nonce":1203822890}]},"public_addr":"172.21.15.203:6825/1203822890","cluster_addr":"172.21.15.203:6827/1203822890","heartbeat_back_addr":"172.21.15.203:6831/1203822890","heartbeat_front_addr":"172.21.15.203:6829/1203822890","state":["exists","up"]},{"osd":6,"uuid":"f29dd047-3b48-473d-a235-6642a6293f20","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":21,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6810","nonce":2309615014},{"type":"v1","addr":"172.21.15.203:6812","nonce":2309615014}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6814","nonce":2309615014},{"type":"v1","addr":"172.21.15.203:6816","nonce":2309615014}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6822","nonce":2309615014},{"type":"v1","addr":"172.21.15.203:6823","nonce":2309615014}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6818","nonce":2309615014},{"type":"v1","addr":"172.21.15.203:6820","nonce":2309615014}]},"public_addr":"172.21.15.203:6812/2309615014","cluster_addr":"172.21.15.203:6816/2309615014","heartbeat_back_addr":"172.21.15.203:6823/2309615014","heartbeat_front_addr":"172.21.15.203:6820/2309615014","state":["exists","up"]},{"osd":7,"uuid":"6e691644-378b-4fdc-8e86-fb5f56e31f29","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":21,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6808","nonce":3688099014},{"type":"v1","addr":"172.21.15.203:6809","nonce":3688099014}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6811","nonce":3688099014},{"type":"v1","addr":"172.21.15.203:6813","nonce":3688099014}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6819","nonce":3688099014},{"type":"v1","addr":"172.21.15.203:6821","nonce":3688099014}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6815","nonce":3688099014},{"type":"v1","addr":"172.21.15.203:6817","nonce":3688099014}]},"public_addr":"172.21.15.203:6809/3688099014","cluster_addr":"172.21.15.203:6813/3688099014","heartbeat_back_addr":"172.21.15.203:6821/3688099014","heartbeat_front_addr":"172.21.15.203:6817/3688099014","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T10:28:51.419141+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T10:28:51.366448+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T10:28:51.348276+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T10:28:51.380519+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T10:28:51.105812+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T10:28:51.384496+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T10:28:51.388755+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T10:28:51.366958+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[{"pgid":"3.31","mappings":[{"from":0,"to":7}]}],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-04-25T10:29:53.001 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:52.998+0000 7fb19a4e7640 1 -- 172.21.15.37:0/4624382 >> [v2:172.21.15.203:6832/2418205237,v1:172.21.15.203:6833/2418205237] conn(0x7fb1800744e0 msgr2=0x7fb1800769f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:29:53.001 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:52.998+0000 7fb19a4e7640 1 --2- 172.21.15.37:0/4624382 >> [v2:172.21.15.203:6832/2418205237,v1:172.21.15.203:6833/2418205237] conn(0x7fb1800744e0 0x7fb1800769f0 secure :-1 s=READY pgs=164 cs=0 l=1 rev1=1 crypto rx=0x7fb19408d730 tx=0x7fb188008040 comp rx=0 tx=0).stop 2024-04-25T10:29:53.001 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:52.998+0000 7fb19a4e7640 1 -- 172.21.15.37:0/4624382 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fb19406daf0 msgr2=0x7fb194162410 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:29:53.001 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:52.998+0000 7fb19a4e7640 1 --2- 172.21.15.37:0/4624382 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fb19406daf0 0x7fb194162410 secure :-1 s=READY pgs=314 cs=0 l=1 rev1=1 crypto rx=0x7fb184000c00 tx=0x7fb1840046c0 comp rx=0 tx=0).stop 2024-04-25T10:29:53.001 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:52.998+0000 7fb19a4e7640 1 -- 172.21.15.37:0/4624382 shutdown_connections 2024-04-25T10:29:53.001 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:52.998+0000 7fb19a4e7640 1 --2- 172.21.15.37:0/4624382 >> [v2:172.21.15.203:6832/2418205237,v1:172.21.15.203:6833/2418205237] conn(0x7fb1800744e0 0x7fb1800769f0 unknown :-1 s=CLOSED pgs=164 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:29:53.001 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:52.998+0000 7fb19a4e7640 1 --2- 172.21.15.37:0/4624382 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7fb1940b2dc0 0x7fb194161060 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:29:53.001 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:52.998+0000 7fb19a4e7640 1 --2- 172.21.15.37:0/4624382 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fb19406e440 0x7fb194160b20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:29:53.001 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:52.998+0000 7fb19a4e7640 1 --2- 172.21.15.37:0/4624382 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fb19406daf0 0x7fb194162410 unknown :-1 s=CLOSED pgs=314 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:29:53.001 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:52.998+0000 7fb19a4e7640 1 -- 172.21.15.37:0/4624382 >> 172.21.15.37:0/4624382 conn(0x7fb1940b5440 msgr2=0x7fb1940b11c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:29:53.001 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:52.999+0000 7fb19a4e7640 1 -- 172.21.15.37:0/4624382 shutdown_connections 2024-04-25T10:29:53.001 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:52.999+0000 7fb19a4e7640 1 -- 172.21.15.37:0/4624382 wait complete. 2024-04-25T10:29:53.012 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2024-04-25T10:28:56.202153+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '19', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 8, 'score_stable': 8, 'optimal_score': 0.25, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 2, 'pool_name': 'cephfs_metadata', 'create_time': '2024-04-25T10:29:06.507244+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '22', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_autoscale_bias': 4, 'pg_num_min': 16, 'recovery_priority': 5}, 'application_metadata': {'cephfs': {'metadata': 'cephfs'}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 2.25, 'score_stable': 2.25, 'optimal_score': 1, 'raw_score_acting': 2.25, 'raw_score_stable': 2.25, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 3, 'pool_name': 'cephfs_data', 'create_time': '2024-04-25T10:29:07.347844+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 64, 'pg_placement_num': 64, 'pg_placement_num_target': 64, 'pg_num_target': 64, 'pg_num_pending': 64, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '23', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.8999999761581421}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 1.5, 'score_stable': 1.5, 'optimal_score': 1, 'raw_score_acting': 1.5, 'raw_score_stable': 1.5, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}] 2024-04-25T10:29:53.013 DEBUG:teuthology.orchestra.run.smithi037:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2024-04-25T10:29:53.127 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:53.125+0000 7fb525bce640 1 Processor -- start 2024-04-25T10:29:53.128 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:53.125+0000 7fb525bce640 1 -- start start 2024-04-25T10:29:53.129 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:53.126+0000 7fb525bce640 1 --2- >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7fb5200d5160 0x7fb5200b4e80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:29:53.129 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:53.126+0000 7fb525bce640 1 --2- >> v2:172.21.15.203:3300/0 conn(0x7fb5200d3230 0x7fb5200b9060 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:29:53.129 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:53.126+0000 7fb525bce640 1 -- --> v1:172.21.15.203:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fb5200d1370 con 0x7fb5200d0990 2024-04-25T10:29:53.129 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:53.126+0000 7fb525bce640 1 -- --> v2:172.21.15.203:3300/0 -- mon_getmap magic: 0 -- 0x7fb5200d1650 con 0x7fb5200d3230 2024-04-25T10:29:53.129 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:53.126+0000 7fb525bce640 1 -- --> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] -- mon_getmap magic: 0 -- 0x7fb5200d4680 con 0x7fb5200d5160 2024-04-25T10:29:53.129 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:53.127+0000 7fb51f7fe640 1 --2- >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7fb5200d5160 0x7fb5200b4e80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:29:53.129 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:53.127+0000 7fb51ffff640 1 --2- >> v2:172.21.15.203:3300/0 conn(0x7fb5200d3230 0x7fb5200b9060 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:29:53.129 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:53.127+0000 7fb51f7fe640 1 --2- >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7fb5200d5160 0x7fb5200b4e80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.203:3301/0 says I am v2:172.21.15.37:45998/0 (socket says 172.21.15.37:45998) 2024-04-25T10:29:53.130 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:53.127+0000 7fb51f7fe640 1 -- 172.21.15.37:0/98216032 learned_addr learned my addr 172.21.15.37:0/98216032 (peer_addr_for_me v2:172.21.15.37:0/0) 2024-04-25T10:29:53.130 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:53.127+0000 7fb51effd640 1 --1- 172.21.15.37:0/98216032 >> v1:172.21.15.203:6789/0 conn(0x7fb5200d0990 0x7fb5200d2b20 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.203:6789/0 says I am v1:172.21.15.37:52678/0 (socket says 172.21.15.37:52678) 2024-04-25T10:29:53.130 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:53.127+0000 7fb51f7fe640 1 -- 172.21.15.37:0/98216032 >> v2:172.21.15.203:3300/0 conn(0x7fb5200d3230 msgr2=0x7fb5200b9060 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:29:53.130 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:53.127+0000 7fb51f7fe640 1 --2- 172.21.15.37:0/98216032 >> v2:172.21.15.203:3300/0 conn(0x7fb5200d3230 0x7fb5200b9060 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:29:53.130 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:53.127+0000 7fb51f7fe640 1 -- 172.21.15.37:0/98216032 >> v1:172.21.15.203:6789/0 conn(0x7fb5200d0990 legacy=0x7fb5200d2b20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:29:53.130 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:53.127+0000 7fb51f7fe640 1 -- 172.21.15.37:0/98216032 --> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fb5200d4960 con 0x7fb5200d5160 2024-04-25T10:29:53.131 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:53.128+0000 7fb51f7fe640 1 --2- 172.21.15.37:0/98216032 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7fb5200d5160 0x7fb5200b4e80 secure :-1 s=READY pgs=46 cs=0 l=1 rev1=1 crypto rx=0x7fb514009870 tx=0x7fb51402a150 comp rx=0 tx=0).ready entity=mon.2 client_cookie=26f3f1ae1730918d server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:29:53.131 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:53.128+0000 7fb51e7fc640 1 -- 172.21.15.37:0/98216032 <== mon.2 v2:172.21.15.203:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb514037070 con 0x7fb5200d5160 2024-04-25T10:29:53.131 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:53.128+0000 7fb51e7fc640 1 -- 172.21.15.37:0/98216032 <== mon.2 v2:172.21.15.203:3301/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fb514032440 con 0x7fb5200d5160 2024-04-25T10:29:53.131 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:53.128+0000 7fb51e7fc640 1 -- 172.21.15.37:0/98216032 <== mon.2 v2:172.21.15.203:3301/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb514032810 con 0x7fb5200d5160 2024-04-25T10:29:53.131 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:53.128+0000 7fb525bce640 1 -- 172.21.15.37:0/98216032 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7fb5200d5160 msgr2=0x7fb5200b4e80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:29:53.131 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:53.129+0000 7fb525bce640 1 --2- 172.21.15.37:0/98216032 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7fb5200d5160 0x7fb5200b4e80 secure :-1 s=READY pgs=46 cs=0 l=1 rev1=1 crypto rx=0x7fb514009870 tx=0x7fb51402a150 comp rx=0 tx=0).stop 2024-04-25T10:29:53.131 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:53.129+0000 7fb525bce640 1 -- 172.21.15.37:0/98216032 shutdown_connections 2024-04-25T10:29:53.132 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:53.129+0000 7fb525bce640 1 --2- 172.21.15.37:0/98216032 >> v2:172.21.15.203:3300/0 conn(0x7fb5200d3230 0x7fb5200b9060 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:29:53.132 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:53.129+0000 7fb525bce640 1 --2- 172.21.15.37:0/98216032 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7fb5200d5160 0x7fb5200b4e80 unknown :-1 s=CLOSED pgs=46 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:29:53.132 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:53.129+0000 7fb525bce640 1 -- 172.21.15.37:0/98216032 >> 172.21.15.37:0/98216032 conn(0x7fb5200b7a80 msgr2=0x7fb5200b7e90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:29:53.132 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:53.129+0000 7fb525bce640 1 -- 172.21.15.37:0/98216032 shutdown_connections 2024-04-25T10:29:53.132 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:53.129+0000 7fb525bce640 1 -- 172.21.15.37:0/98216032 wait complete. 2024-04-25T10:29:53.132 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:53.130+0000 7fb525bce640 1 Processor -- start 2024-04-25T10:29:53.132 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:53.131+0000 7fb525bce640 1 -- start start 2024-04-25T10:29:53.135 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:53.132+0000 7fb525bce640 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fb5200d0990 0x7fb5200ccc00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:29:53.136 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:53.132+0000 7fb525bce640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fb5200d5160 0x7fb520161ac0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:29:53.136 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:53.132+0000 7fb525bce640 1 --2- >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7fb520162000 0x7fb5200cc090 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:29:53.136 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:53.132+0000 7fb525bce640 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 -- 0x7fb520164c40 con 0x7fb5200d5160 2024-04-25T10:29:53.136 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:53.132+0000 7fb525bce640 1 -- --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_getmap magic: 0 -- 0x7fb520164ac0 con 0x7fb5200d0990 2024-04-25T10:29:53.136 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:53.132+0000 7fb525bce640 1 -- --> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] -- mon_getmap magic: 0 -- 0x7fb5200d0da0 con 0x7fb520162000 2024-04-25T10:29:53.136 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:53.132+0000 7fb51effd640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fb5200d5160 0x7fb520161ac0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:29:53.136 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:53.132+0000 7fb51f7fe640 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fb5200d0990 0x7fb5200ccc00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:29:53.136 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:53.132+0000 7fb51effd640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fb5200d5160 0x7fb520161ac0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.37:3300/0 says I am v2:172.21.15.37:36344/0 (socket says 172.21.15.37:36344) 2024-04-25T10:29:53.136 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:53.132+0000 7fb51effd640 1 -- 172.21.15.37:0/2084016412 learned_addr learned my addr 172.21.15.37:0/2084016412 (peer_addr_for_me v2:172.21.15.37:0/0) 2024-04-25T10:29:53.136 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:53.132+0000 7fb51ffff640 1 --2- 172.21.15.37:0/2084016412 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7fb520162000 0x7fb5200cc090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:29:53.136 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:53.132+0000 7fb51effd640 1 -- 172.21.15.37:0/2084016412 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7fb520162000 msgr2=0x7fb5200cc090 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:29:53.136 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:53.132+0000 7fb51effd640 1 --2- 172.21.15.37:0/2084016412 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7fb520162000 0x7fb5200cc090 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:29:53.136 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:53.132+0000 7fb51effd640 1 -- 172.21.15.37:0/2084016412 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fb5200d0990 msgr2=0x7fb5200ccc00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:29:53.136 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:53.132+0000 7fb51effd640 1 --2- 172.21.15.37:0/2084016412 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fb5200d0990 0x7fb5200ccc00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:29:53.136 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:53.132+0000 7fb51effd640 1 -- 172.21.15.37:0/2084016412 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fb520158670 con 0x7fb5200d5160 2024-04-25T10:29:53.137 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:53.132+0000 7fb51f7fe640 1 --2- 172.21.15.37:0/2084016412 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fb5200d0990 0x7fb5200ccc00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-25T10:29:53.137 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:53.133+0000 7fb51effd640 1 --2- 172.21.15.37:0/2084016412 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fb5200d5160 0x7fb520161ac0 secure :-1 s=READY pgs=315 cs=0 l=1 rev1=1 crypto rx=0x7fb51000c390 tx=0x7fb51000c850 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:29:53.137 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:53.133+0000 7fb507fff640 1 -- 172.21.15.37:0/2084016412 <== mon.0 v2:172.21.15.37:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb5100074e0 con 0x7fb5200d5160 2024-04-25T10:29:53.137 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:53.133+0000 7fb507fff640 1 -- 172.21.15.37:0/2084016412 <== mon.0 v2:172.21.15.37:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fb510007680 con 0x7fb5200d5160 2024-04-25T10:29:53.137 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:53.133+0000 7fb525bce640 1 -- 172.21.15.37:0/2084016412 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fb52014c210 con 0x7fb5200d5160 2024-04-25T10:29:53.137 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:53.133+0000 7fb507fff640 1 -- 172.21.15.37:0/2084016412 <== mon.0 v2:172.21.15.37:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb5100049e0 con 0x7fb5200d5160 2024-04-25T10:29:53.137 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:53.133+0000 7fb525bce640 1 -- 172.21.15.37:0/2084016412 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fb52014c7d0 con 0x7fb5200d5160 2024-04-25T10:29:53.138 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:53.134+0000 7fb525bce640 1 -- 172.21.15.37:0/2084016412 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fb52006a140 con 0x7fb5200d5160 2024-04-25T10:29:53.138 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:53.135+0000 7fb507fff640 1 -- 172.21.15.37:0/2084016412 <== mon.0 v2:172.21.15.37:3300/0 4 ==== mgrmap(e 5) ==== 96782+0+0 (secure 0 0 0) 0x7fb51001a020 con 0x7fb5200d5160 2024-04-25T10:29:53.138 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:53.135+0000 7fb507fff640 1 --2- 172.21.15.37:0/2084016412 >> [v2:172.21.15.203:6832/2418205237,v1:172.21.15.203:6833/2418205237] conn(0x7fb50c0744b0 0x7fb50c0769c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:29:53.138 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:53.135+0000 7fb507fff640 1 -- 172.21.15.37:0/2084016412 <== mon.0 v2:172.21.15.37:3300/0 5 ==== osd_map(25..25 src has 1..25) ==== 5738+0+0 (secure 0 0 0) 0x7fb5100948c0 con 0x7fb5200d5160 2024-04-25T10:29:53.138 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:53.135+0000 7fb51f7fe640 1 --2- 172.21.15.37:0/2084016412 >> [v2:172.21.15.203:6832/2418205237,v1:172.21.15.203:6833/2418205237] conn(0x7fb50c0744b0 0x7fb50c0769c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:29:53.139 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:53.136+0000 7fb51f7fe640 1 --2- 172.21.15.37:0/2084016412 >> [v2:172.21.15.203:6832/2418205237,v1:172.21.15.203:6833/2418205237] conn(0x7fb50c0744b0 0x7fb50c0769c0 secure :-1 s=READY pgs=165 cs=0 l=1 rev1=1 crypto rx=0x7fb514034000 tx=0x7fb51402e000 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:29:53.142 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:53.140+0000 7fb507fff640 1 -- 172.21.15.37:0/2084016412 <== mon.0 v2:172.21.15.37:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187246 (secure 0 0 0) 0x7fb520158670 con 0x7fb5200d5160 2024-04-25T10:29:53.325 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:53.322+0000 7fb525bce640 1 -- 172.21.15.37:0/2084016412 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) -- 0x7fb5200d3f50 con 0x7fb5200d5160 2024-04-25T10:29:53.325 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:53.323+0000 7fb507fff640 1 -- 172.21.15.37:0/2084016412 <== mon.0 v2:172.21.15.37:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v25) ==== 93+0+10 (secure 0 0 0) 0x7fb51005e940 con 0x7fb5200d5160 2024-04-25T10:29:53.325 INFO:teuthology.orchestra.run.smithi037.stdout:pg_num: 1 2024-04-25T10:29:53.327 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:53.325+0000 7fb525bce640 1 -- 172.21.15.37:0/2084016412 >> [v2:172.21.15.203:6832/2418205237,v1:172.21.15.203:6833/2418205237] conn(0x7fb50c0744b0 msgr2=0x7fb50c0769c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:29:53.327 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:53.325+0000 7fb525bce640 1 --2- 172.21.15.37:0/2084016412 >> [v2:172.21.15.203:6832/2418205237,v1:172.21.15.203:6833/2418205237] conn(0x7fb50c0744b0 0x7fb50c0769c0 secure :-1 s=READY pgs=165 cs=0 l=1 rev1=1 crypto rx=0x7fb514034000 tx=0x7fb51402e000 comp rx=0 tx=0).stop 2024-04-25T10:29:53.328 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:53.325+0000 7fb525bce640 1 -- 172.21.15.37:0/2084016412 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fb5200d5160 msgr2=0x7fb520161ac0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:29:53.328 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:53.325+0000 7fb525bce640 1 --2- 172.21.15.37:0/2084016412 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fb5200d5160 0x7fb520161ac0 secure :-1 s=READY pgs=315 cs=0 l=1 rev1=1 crypto rx=0x7fb51000c390 tx=0x7fb51000c850 comp rx=0 tx=0).stop 2024-04-25T10:29:53.328 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:53.325+0000 7fb525bce640 1 -- 172.21.15.37:0/2084016412 shutdown_connections 2024-04-25T10:29:53.328 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:53.325+0000 7fb525bce640 1 --2- 172.21.15.37:0/2084016412 >> [v2:172.21.15.203:6832/2418205237,v1:172.21.15.203:6833/2418205237] conn(0x7fb50c0744b0 0x7fb50c0769c0 unknown :-1 s=CLOSED pgs=165 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:29:53.328 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:53.325+0000 7fb525bce640 1 --2- 172.21.15.37:0/2084016412 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7fb520162000 0x7fb5200cc090 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:29:53.328 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:53.325+0000 7fb525bce640 1 --2- 172.21.15.37:0/2084016412 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fb5200d5160 0x7fb520161ac0 unknown :-1 s=CLOSED pgs=315 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:29:53.328 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:53.325+0000 7fb525bce640 1 --2- 172.21.15.37:0/2084016412 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fb5200d0990 0x7fb5200ccc00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:29:53.328 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:53.325+0000 7fb525bce640 1 -- 172.21.15.37:0/2084016412 >> 172.21.15.37:0/2084016412 conn(0x7fb5200b7a80 msgr2=0x7fb5200b9700 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:29:53.328 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:53.325+0000 7fb525bce640 1 -- 172.21.15.37:0/2084016412 shutdown_connections 2024-04-25T10:29:53.328 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:53.325+0000 7fb525bce640 1 -- 172.21.15.37:0/2084016412 wait complete. 2024-04-25T10:29:53.339 DEBUG:teuthology.orchestra.run.smithi037:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_metadata pg_num 2024-04-25T10:29:53.454 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:53.451+0000 7fdeed56c640 1 Processor -- start 2024-04-25T10:29:53.454 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:53.451+0000 7fdeed56c640 1 -- start start 2024-04-25T10:29:53.455 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:53.453+0000 7fdeed56c640 1 --2- >> v2:172.21.15.37:3300/0 conn(0x7fdee80d1ec0 0x7fdee80d22d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:29:53.456 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:53.453+0000 7fdeed56c640 1 --2- >> v2:172.21.15.203:3300/0 conn(0x7fdee80d2810 0x7fdee80b3c80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:29:53.456 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:53.453+0000 7fdeed56c640 1 -- --> v1:172.21.15.37:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fdee809ce70 con 0x7fdee806e260 2024-04-25T10:29:53.456 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:53.453+0000 7fdeed56c640 1 -- --> v2:172.21.15.37:3300/0 -- mon_getmap magic: 0 -- 0x7fdee80d3370 con 0x7fdee80d1ec0 2024-04-25T10:29:53.456 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:53.453+0000 7fdeed56c640 1 -- --> v2:172.21.15.203:3300/0 -- mon_getmap magic: 0 -- 0x7fdee80d3090 con 0x7fdee80d2810 2024-04-25T10:29:53.456 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:53.453+0000 7fdee67fc640 1 --2- >> v2:172.21.15.37:3300/0 conn(0x7fdee80d1ec0 0x7fdee80d22d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:29:53.456 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:53.453+0000 7fdee67fc640 1 --2- >> v2:172.21.15.37:3300/0 conn(0x7fdee80d1ec0 0x7fdee80d22d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.37:3300/0 says I am v2:172.21.15.37:36358/0 (socket says 172.21.15.37:36358) 2024-04-25T10:29:53.456 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:53.453+0000 7fdee6ffd640 1 --1- >> v1:172.21.15.37:6789/0 conn(0x7fdee806e260 0x7fdee806e670 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.37:6789/0 says I am v1:172.21.15.37:56806/0 (socket says 172.21.15.37:56806) 2024-04-25T10:29:53.456 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:53.453+0000 7fdee67fc640 1 -- 172.21.15.37:0/2873730369 learned_addr learned my addr 172.21.15.37:0/2873730369 (peer_addr_for_me v2:172.21.15.37:0/0) 2024-04-25T10:29:53.456 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:53.453+0000 7fdee67fc640 1 -- 172.21.15.37:0/2873730369 >> v2:172.21.15.203:3300/0 conn(0x7fdee80d2810 msgr2=0x7fdee80b3c80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:29:53.456 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:53.453+0000 7fdee67fc640 1 --2- 172.21.15.37:0/2873730369 >> v2:172.21.15.203:3300/0 conn(0x7fdee80d2810 0x7fdee80b3c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:29:53.456 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:53.453+0000 7fdee67fc640 1 -- 172.21.15.37:0/2873730369 >> v1:172.21.15.37:6789/0 conn(0x7fdee806e260 legacy=0x7fdee806e670 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:29:53.456 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:53.453+0000 7fdee67fc640 1 -- 172.21.15.37:0/2873730369 --> v2:172.21.15.37:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fdee809d150 con 0x7fdee80d1ec0 2024-04-25T10:29:53.456 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:53.453+0000 7fdee5ffb640 1 -- 172.21.15.37:0/2873730369 <== mon.0 v1:172.21.15.37:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 4212351694 0 0) 0x7fdee809ce70 con 0x7fdee806e260 2024-04-25T10:29:53.457 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:53.454+0000 7fdee67fc640 1 --2- 172.21.15.37:0/2873730369 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fdee80d1ec0 0x7fdee80d22d0 secure :-1 s=READY pgs=317 cs=0 l=1 rev1=1 crypto rx=0x7fdee808cfc0 tx=0x7fded002a1e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=34648d5e1968b036 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:29:53.458 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:53.454+0000 7fdee5ffb640 1 -- 172.21.15.37:0/2873730369 <== mon.0 v2:172.21.15.37:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fded0037070 con 0x7fdee80d1ec0 2024-04-25T10:29:53.458 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:53.454+0000 7fdee5ffb640 1 -- 172.21.15.37:0/2873730369 <== mon.0 v2:172.21.15.37:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fded0032440 con 0x7fdee80d1ec0 2024-04-25T10:29:53.458 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:53.454+0000 7fdee5ffb640 1 -- 172.21.15.37:0/2873730369 <== mon.0 v2:172.21.15.37:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fded0032810 con 0x7fdee80d1ec0 2024-04-25T10:29:53.458 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:53.455+0000 7fdeed56c640 1 -- 172.21.15.37:0/2873730369 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fdee80d1ec0 msgr2=0x7fdee80d22d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:29:53.458 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:53.455+0000 7fdeed56c640 1 --2- 172.21.15.37:0/2873730369 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fdee80d1ec0 0x7fdee80d22d0 secure :-1 s=READY pgs=317 cs=0 l=1 rev1=1 crypto rx=0x7fdee808cfc0 tx=0x7fded002a1e0 comp rx=0 tx=0).stop 2024-04-25T10:29:53.458 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:53.455+0000 7fdeed56c640 1 -- 172.21.15.37:0/2873730369 shutdown_connections 2024-04-25T10:29:53.458 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:53.455+0000 7fdeed56c640 1 --2- 172.21.15.37:0/2873730369 >> v2:172.21.15.203:3300/0 conn(0x7fdee80d2810 0x7fdee80b3c80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:29:53.458 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:53.455+0000 7fdeed56c640 1 --2- 172.21.15.37:0/2873730369 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fdee80d1ec0 0x7fdee80d22d0 unknown :-1 s=CLOSED pgs=317 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:29:53.458 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:53.455+0000 7fdeed56c640 1 -- 172.21.15.37:0/2873730369 >> 172.21.15.37:0/2873730369 conn(0x7fdee80d0390 msgr2=0x7fdee80d07a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:29:53.458 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:53.455+0000 7fdeed56c640 1 -- 172.21.15.37:0/2873730369 shutdown_connections 2024-04-25T10:29:53.458 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:53.455+0000 7fdeed56c640 1 -- 172.21.15.37:0/2873730369 wait complete. 2024-04-25T10:29:53.459 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:53.457+0000 7fdeed56c640 1 Processor -- start 2024-04-25T10:29:53.459 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:53.457+0000 7fdeed56c640 1 -- start start 2024-04-25T10:29:53.461 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:53.458+0000 7fdeed56c640 1 --2- >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7fdee806e260 0x7fdee809e340 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:29:53.462 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:53.458+0000 7fdeed56c640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fdee80d1ec0 0x7fdee809e880 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:29:53.462 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:53.458+0000 7fdeed56c640 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fdee80d2810 0x7fdee81f9790 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:29:53.462 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:53.458+0000 7fdeed56c640 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 -- 0x7fdee80b9000 con 0x7fdee80d1ec0 2024-04-25T10:29:53.462 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:53.458+0000 7fdeed56c640 1 -- --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_getmap magic: 0 -- 0x7fdee806d830 con 0x7fdee80d2810 2024-04-25T10:29:53.462 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:53.458+0000 7fdeed56c640 1 -- --> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] -- mon_getmap magic: 0 -- 0x7fdee80b9180 con 0x7fdee806e260 2024-04-25T10:29:53.462 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:53.458+0000 7fdee67fc640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fdee80d1ec0 0x7fdee809e880 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:29:53.462 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:53.458+0000 7fdee67fc640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fdee80d1ec0 0x7fdee809e880 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.37:3300/0 says I am v2:172.21.15.37:36370/0 (socket says 172.21.15.37:36370) 2024-04-25T10:29:53.462 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:53.458+0000 7fdee67fc640 1 -- 172.21.15.37:0/1844014325 learned_addr learned my addr 172.21.15.37:0/1844014325 (peer_addr_for_me v2:172.21.15.37:0/0) 2024-04-25T10:29:53.462 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:53.459+0000 7fdee77fe640 1 --2- 172.21.15.37:0/1844014325 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fdee80d2810 0x7fdee81f9790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:29:53.462 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:53.459+0000 7fdee6ffd640 1 --2- 172.21.15.37:0/1844014325 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7fdee806e260 0x7fdee809e340 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:29:53.462 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:53.459+0000 7fdee67fc640 1 -- 172.21.15.37:0/1844014325 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7fdee806e260 msgr2=0x7fdee809e340 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:29:53.462 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:53.459+0000 7fdee67fc640 1 --2- 172.21.15.37:0/1844014325 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7fdee806e260 0x7fdee809e340 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:29:53.462 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:53.459+0000 7fdee67fc640 1 -- 172.21.15.37:0/1844014325 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fdee80d2810 msgr2=0x7fdee81f9790 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:29:53.463 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:53.459+0000 7fdee67fc640 1 --2- 172.21.15.37:0/1844014325 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fdee80d2810 0x7fdee81f9790 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:29:53.463 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:53.459+0000 7fdee67fc640 1 -- 172.21.15.37:0/1844014325 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fdee809fa50 con 0x7fdee80d1ec0 2024-04-25T10:29:53.463 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:53.459+0000 7fdee67fc640 1 --2- 172.21.15.37:0/1844014325 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fdee80d1ec0 0x7fdee809e880 secure :-1 s=READY pgs=318 cs=0 l=1 rev1=1 crypto rx=0x7fded0000c00 tx=0x7fded00046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:29:53.463 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:53.459+0000 7fdedf7fe640 1 -- 172.21.15.37:0/1844014325 <== mon.0 v2:172.21.15.37:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fded0041070 con 0x7fdee80d1ec0 2024-04-25T10:29:53.463 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:53.459+0000 7fdedf7fe640 1 -- 172.21.15.37:0/1844014325 <== mon.0 v2:172.21.15.37:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fded003c870 con 0x7fdee80d1ec0 2024-04-25T10:29:53.463 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:53.459+0000 7fdeed56c640 1 -- 172.21.15.37:0/1844014325 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fdee80c86e0 con 0x7fdee80d1ec0 2024-04-25T10:29:53.463 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:53.459+0000 7fdedf7fe640 1 -- 172.21.15.37:0/1844014325 <== mon.0 v2:172.21.15.37:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fded0037040 con 0x7fdee80d1ec0 2024-04-25T10:29:53.463 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:53.459+0000 7fdeed56c640 1 -- 172.21.15.37:0/1844014325 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fdee809db40 con 0x7fdee80d1ec0 2024-04-25T10:29:53.464 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:53.460+0000 7fdeed56c640 1 -- 172.21.15.37:0/1844014325 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fdee806a140 con 0x7fdee80d1ec0 2024-04-25T10:29:53.464 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:53.461+0000 7fdedf7fe640 1 -- 172.21.15.37:0/1844014325 <== mon.0 v2:172.21.15.37:3300/0 4 ==== mgrmap(e 5) ==== 96782+0+0 (secure 0 0 0) 0x7fded0044070 con 0x7fdee80d1ec0 2024-04-25T10:29:53.464 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:53.461+0000 7fdedf7fe640 1 --2- 172.21.15.37:0/1844014325 >> [v2:172.21.15.203:6832/2418205237,v1:172.21.15.203:6833/2418205237] conn(0x7fdec80744b0 0x7fdec80769c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:29:53.464 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:53.461+0000 7fdedf7fe640 1 -- 172.21.15.37:0/1844014325 <== mon.0 v2:172.21.15.37:3300/0 5 ==== osd_map(25..25 src has 1..25) ==== 5738+0+0 (secure 0 0 0) 0x7fded00b5e20 con 0x7fdee80d1ec0 2024-04-25T10:29:53.464 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:53.461+0000 7fdee6ffd640 1 --2- 172.21.15.37:0/1844014325 >> [v2:172.21.15.203:6832/2418205237,v1:172.21.15.203:6833/2418205237] conn(0x7fdec80744b0 0x7fdec80769c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:29:53.464 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:53.462+0000 7fdee6ffd640 1 --2- 172.21.15.37:0/1844014325 >> [v2:172.21.15.203:6832/2418205237,v1:172.21.15.203:6833/2418205237] conn(0x7fdec80744b0 0x7fdec80769c0 secure :-1 s=READY pgs=166 cs=0 l=1 rev1=1 crypto rx=0x7fdee0000f60 tx=0x7fdee0007c00 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:29:53.468 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:53.466+0000 7fdedf7fe640 1 -- 172.21.15.37:0/1844014325 <== mon.0 v2:172.21.15.37:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187246 (secure 0 0 0) 0x7fded0080620 con 0x7fdee80d1ec0 2024-04-25T10:29:53.653 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:53.651+0000 7fdeed56c640 1 -- 172.21.15.37:0/1844014325 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"} v 0) -- 0x7fdee80c8400 con 0x7fdee80d1ec0 2024-04-25T10:29:53.653 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:53.651+0000 7fdedf7fe640 1 -- 172.21.15.37:0/1844014325 <== mon.0 v2:172.21.15.37:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]=0 v25) ==== 104+0+11 (secure 0 0 0) 0x7fded007ff20 con 0x7fdee80d1ec0 2024-04-25T10:29:53.654 INFO:teuthology.orchestra.run.smithi037.stdout:pg_num: 32 2024-04-25T10:29:53.656 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:53.653+0000 7fdeed56c640 1 -- 172.21.15.37:0/1844014325 >> [v2:172.21.15.203:6832/2418205237,v1:172.21.15.203:6833/2418205237] conn(0x7fdec80744b0 msgr2=0x7fdec80769c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:29:53.656 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:53.653+0000 7fdeed56c640 1 --2- 172.21.15.37:0/1844014325 >> [v2:172.21.15.203:6832/2418205237,v1:172.21.15.203:6833/2418205237] conn(0x7fdec80744b0 0x7fdec80769c0 secure :-1 s=READY pgs=166 cs=0 l=1 rev1=1 crypto rx=0x7fdee0000f60 tx=0x7fdee0007c00 comp rx=0 tx=0).stop 2024-04-25T10:29:53.656 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:53.653+0000 7fdeed56c640 1 -- 172.21.15.37:0/1844014325 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fdee80d1ec0 msgr2=0x7fdee809e880 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:29:53.656 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:53.653+0000 7fdeed56c640 1 --2- 172.21.15.37:0/1844014325 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fdee80d1ec0 0x7fdee809e880 secure :-1 s=READY pgs=318 cs=0 l=1 rev1=1 crypto rx=0x7fded0000c00 tx=0x7fded00046c0 comp rx=0 tx=0).stop 2024-04-25T10:29:53.656 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:53.653+0000 7fdeed56c640 1 -- 172.21.15.37:0/1844014325 shutdown_connections 2024-04-25T10:29:53.657 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:53.653+0000 7fdeed56c640 1 --2- 172.21.15.37:0/1844014325 >> [v2:172.21.15.203:6832/2418205237,v1:172.21.15.203:6833/2418205237] conn(0x7fdec80744b0 0x7fdec80769c0 unknown :-1 s=CLOSED pgs=166 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:29:53.657 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:53.653+0000 7fdeed56c640 1 --2- 172.21.15.37:0/1844014325 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fdee80d2810 0x7fdee81f9790 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:29:53.657 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:53.653+0000 7fdeed56c640 1 --2- 172.21.15.37:0/1844014325 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fdee80d1ec0 0x7fdee809e880 unknown :-1 s=CLOSED pgs=318 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:29:53.657 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:53.653+0000 7fdeed56c640 1 --2- 172.21.15.37:0/1844014325 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7fdee806e260 0x7fdee809e340 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:29:53.657 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:53.653+0000 7fdeed56c640 1 -- 172.21.15.37:0/1844014325 >> 172.21.15.37:0/1844014325 conn(0x7fdee80d0390 msgr2=0x7fdee8072650 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:29:53.657 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:53.653+0000 7fdeed56c640 1 -- 172.21.15.37:0/1844014325 shutdown_connections 2024-04-25T10:29:53.657 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:53.653+0000 7fdeed56c640 1 -- 172.21.15.37:0/1844014325 wait complete. 2024-04-25T10:29:53.667 DEBUG:teuthology.orchestra.run.smithi037:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data pg_num 2024-04-25T10:29:53.782 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:53.779+0000 7fa4d7ac6640 1 Processor -- start 2024-04-25T10:29:53.782 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:53.779+0000 7fa4d7ac6640 1 -- start start 2024-04-25T10:29:53.786 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:53.780+0000 7fa4d7ac6640 1 --2- >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7fa4d008cc90 0x7fa4d009cbc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:29:53.786 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:53.780+0000 7fa4d7ac6640 1 --2- >> v2:172.21.15.203:3300/0 conn(0x7fa4d00d1c20 0x7fa4d00d2030 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:29:53.786 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:53.780+0000 7fa4d7ac6640 1 -- --> v1:172.21.15.37:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fa4d00d2570 con 0x7fa4d00b71b0 2024-04-25T10:29:53.786 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:53.780+0000 7fa4d7ac6640 1 -- --> v2:172.21.15.203:3300/0 -- mon_getmap magic: 0 -- 0x7fa4d00b8d40 con 0x7fa4d00d1c20 2024-04-25T10:29:53.786 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:53.780+0000 7fa4d7ac6640 1 -- --> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] -- mon_getmap magic: 0 -- 0x7fa4d00d27f0 con 0x7fa4d008cc90 2024-04-25T10:29:53.786 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:53.780+0000 7fa4d6063640 1 --1- >> v1:172.21.15.37:6789/0 conn(0x7fa4d00b71b0 0x7fa4d00b75c0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.37:6789/0 says I am v1:172.21.15.37:56816/0 (socket says 172.21.15.37:56816) 2024-04-25T10:29:53.786 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:53.780+0000 7fa4d6063640 1 -- 172.21.15.37:0/1120945457 learned_addr learned my addr 172.21.15.37:0/1120945457 (peer_addr_for_me v1:172.21.15.37:0/0) 2024-04-25T10:29:53.786 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:53.781+0000 7fa4d4860640 1 -- 172.21.15.37:0/1120945457 <== mon.0 v1:172.21.15.37:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 742151149 0 0) 0x7fa4c0001490 con 0x7fa4d00b71b0 2024-04-25T10:29:53.786 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:53.781+0000 7fa4d5061640 1 --2- 172.21.15.37:0/1120945457 >> v2:172.21.15.203:3300/0 conn(0x7fa4d00d1c20 0x7fa4d00d2030 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:29:53.787 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:53.781+0000 7fa4d4860640 1 -- 172.21.15.37:0/1120945457 --> v1:172.21.15.37:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7fa4b8003520 con 0x7fa4d00b71b0 2024-04-25T10:29:53.787 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:53.781+0000 7fa4d5862640 1 --2- 172.21.15.37:0/1120945457 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7fa4d008cc90 0x7fa4d009cbc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:29:53.787 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:53.781+0000 7fa4d4860640 1 -- 172.21.15.37:0/1120945457 <== mon.0 v1:172.21.15.37:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 3119294371 0 0) 0x7fa4b8003520 con 0x7fa4d00b71b0 2024-04-25T10:29:53.787 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:53.781+0000 7fa4d4860640 1 -- 172.21.15.37:0/1120945457 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7fa4d008cc90 msgr2=0x7fa4d009cbc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:29:53.787 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:53.781+0000 7fa4d4860640 1 --2- 172.21.15.37:0/1120945457 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7fa4d008cc90 0x7fa4d009cbc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:29:53.787 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:53.781+0000 7fa4d4860640 1 -- 172.21.15.37:0/1120945457 >> v2:172.21.15.203:3300/0 conn(0x7fa4d00d1c20 msgr2=0x7fa4d00d2030 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:29:53.787 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:53.781+0000 7fa4d4860640 1 --2- 172.21.15.37:0/1120945457 >> v2:172.21.15.203:3300/0 conn(0x7fa4d00d1c20 0x7fa4d00d2030 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:29:53.787 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:53.781+0000 7fa4d4860640 1 -- 172.21.15.37:0/1120945457 --> v1:172.21.15.37:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fa4d00d2ad0 con 0x7fa4d00b71b0 2024-04-25T10:29:53.787 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:53.781+0000 7fa4d4860640 1 -- 172.21.15.37:0/1120945457 <== mon.0 v1:172.21.15.37:6789/0 3 ==== mon_map magic: 0 ==== 413+0+0 (unknown 1577794600 0 0) 0x7fa4c0001bc0 con 0x7fa4d00b71b0 2024-04-25T10:29:53.787 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:53.781+0000 7fa4d4860640 1 -- 172.21.15.37:0/1120945457 >> v1:172.21.15.37:6789/0 conn(0x7fa4d00b71b0 legacy=0x7fa4d00b75c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:29:53.787 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:53.781+0000 7fa4d4860640 1 --2- 172.21.15.37:0/1120945457 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fa4b80039b0 0x7fa4b8003e20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:29:53.787 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:53.781+0000 7fa4d4860640 1 --2- 172.21.15.37:0/1120945457 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7fa4b8004800 0x7fa4b8004c10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:29:53.788 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:53.781+0000 7fa4d4860640 1 --2- 172.21.15.37:0/1120945457 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fa4b8005150 0x7fa4b800b9c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:29:53.788 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:53.781+0000 7fa4d4860640 1 -- 172.21.15.37:0/1120945457 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 -- 0x7fa4d00b8d40 con 0x7fa4b8005150 2024-04-25T10:29:53.788 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:53.781+0000 7fa4d4860640 1 -- 172.21.15.37:0/1120945457 --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_getmap magic: 0 -- 0x7fa4d00d27f0 con 0x7fa4b80039b0 2024-04-25T10:29:53.788 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:53.781+0000 7fa4d4860640 1 -- 172.21.15.37:0/1120945457 --> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] -- mon_getmap magic: 0 -- 0x7fa4b800bf00 con 0x7fa4b8004800 2024-04-25T10:29:53.788 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:53.781+0000 7fa4d6063640 1 --2- 172.21.15.37:0/1120945457 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fa4b8005150 0x7fa4b800b9c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:29:53.788 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:53.781+0000 7fa4d5862640 1 --2- 172.21.15.37:0/1120945457 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fa4b80039b0 0x7fa4b8003e20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:29:53.788 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:53.781+0000 7fa4d5061640 1 --2- 172.21.15.37:0/1120945457 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7fa4b8004800 0x7fa4b8004c10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:29:53.788 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:53.781+0000 7fa4d6063640 1 -- 172.21.15.37:0/1120945457 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7fa4b8004800 msgr2=0x7fa4b8004c10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:29:53.788 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:53.781+0000 7fa4d6063640 1 --2- 172.21.15.37:0/1120945457 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7fa4b8004800 0x7fa4b8004c10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:29:53.788 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:53.781+0000 7fa4d6063640 1 -- 172.21.15.37:0/1120945457 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fa4b80039b0 msgr2=0x7fa4b8003e20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:29:53.788 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:53.781+0000 7fa4d6063640 1 --2- 172.21.15.37:0/1120945457 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fa4b80039b0 0x7fa4b8003e20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:29:53.788 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:53.781+0000 7fa4d6063640 1 -- 172.21.15.37:0/1120945457 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fa4b8003520 con 0x7fa4b8005150 2024-04-25T10:29:53.788 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:53.782+0000 7fa4d6063640 1 --2- 172.21.15.37:0/1120945457 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fa4b8005150 0x7fa4b800b9c0 secure :-1 s=READY pgs=320 cs=0 l=1 rev1=1 crypto rx=0x7fa4c000aa40 tx=0x7fa4b8005830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=931243fcdc148089 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:29:53.788 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:53.782+0000 7fa4d4860640 1 -- 172.21.15.37:0/1120945457 <== mon.0 v2:172.21.15.37:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa4c000a690 con 0x7fa4b8005150 2024-04-25T10:29:53.789 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:53.782+0000 7fa4d4860640 1 -- 172.21.15.37:0/1120945457 <== mon.0 v2:172.21.15.37:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fa4c002f070 con 0x7fa4b8005150 2024-04-25T10:29:53.789 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:53.782+0000 7fa4d4860640 1 -- 172.21.15.37:0/1120945457 <== mon.0 v2:172.21.15.37:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa4c0034470 con 0x7fa4b8005150 2024-04-25T10:29:53.789 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:53.783+0000 7fa4d7ac6640 1 -- 172.21.15.37:0/1120945457 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fa4b8005150 msgr2=0x7fa4b800b9c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:29:53.789 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:53.783+0000 7fa4d7ac6640 1 --2- 172.21.15.37:0/1120945457 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fa4b8005150 0x7fa4b800b9c0 secure :-1 s=READY pgs=320 cs=0 l=1 rev1=1 crypto rx=0x7fa4c000aa40 tx=0x7fa4b8005830 comp rx=0 tx=0).stop 2024-04-25T10:29:53.789 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:53.783+0000 7fa4d6063640 1 -- 172.21.15.37:0/1120945457 reap_dead start 2024-04-25T10:29:53.789 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:53.783+0000 7fa4d7ac6640 1 -- 172.21.15.37:0/1120945457 shutdown_connections 2024-04-25T10:29:53.789 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:53.783+0000 7fa4d7ac6640 1 -- 172.21.15.37:0/1120945457 >> 172.21.15.37:0/1120945457 conn(0x7fa4d00d03d0 msgr2=0x7fa4d00d07e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:29:53.789 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:53.783+0000 7fa4d7ac6640 1 -- 172.21.15.37:0/1120945457 shutdown_connections 2024-04-25T10:29:53.789 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:53.783+0000 7fa4d7ac6640 1 -- 172.21.15.37:0/1120945457 wait complete. 2024-04-25T10:29:53.794 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:53.785+0000 7fa4d7ac6640 1 Processor -- start 2024-04-25T10:29:53.794 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:53.785+0000 7fa4d7ac6640 1 -- start start 2024-04-25T10:29:53.794 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:53.786+0000 7fa4d7ac6640 1 --2- >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7fa4d00a08b0 0x7fa4d01dbb30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:29:53.795 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:53.786+0000 7fa4d7ac6640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fa4d00cb500 0x7fa4d00cb910 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:29:53.795 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:53.786+0000 7fa4d7ac6640 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fa4d00cbe50 0x7fa4d006d650 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:29:53.795 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:53.786+0000 7fa4d7ac6640 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 -- 0x7fa4d00d1100 con 0x7fa4d00cb500 2024-04-25T10:29:53.795 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:53.786+0000 7fa4d7ac6640 1 -- --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_getmap magic: 0 -- 0x7fa4d00b93b0 con 0x7fa4d00cbe50 2024-04-25T10:29:53.795 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:53.786+0000 7fa4d7ac6640 1 -- --> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] -- mon_getmap magic: 0 -- 0x7fa4d00d1280 con 0x7fa4d00a08b0 2024-04-25T10:29:53.795 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:53.786+0000 7fa4d5061640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fa4d00cb500 0x7fa4d00cb910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:29:53.795 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:53.786+0000 7fa4d5061640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fa4d00cb500 0x7fa4d00cb910 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.37:3300/0 says I am v2:172.21.15.37:36390/0 (socket says 172.21.15.37:36390) 2024-04-25T10:29:53.796 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:53.786+0000 7fa4d5061640 1 -- 172.21.15.37:0/3052684856 learned_addr learned my addr 172.21.15.37:0/3052684856 (peer_addr_for_me v2:172.21.15.37:0/0) 2024-04-25T10:29:53.796 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:53.787+0000 7fa4d5862640 1 --2- 172.21.15.37:0/3052684856 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7fa4d00a08b0 0x7fa4d01dbb30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:29:53.796 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:53.787+0000 7fa4d5061640 1 -- 172.21.15.37:0/3052684856 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7fa4d00a08b0 msgr2=0x7fa4d01dbb30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:29:53.796 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:53.787+0000 7fa4d5061640 1 --2- 172.21.15.37:0/3052684856 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7fa4d00a08b0 0x7fa4d01dbb30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:29:53.796 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:53.787+0000 7fa4d5061640 1 -- 172.21.15.37:0/3052684856 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fa4d00cbe50 msgr2=0x7fa4d006d650 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:29:53.796 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:53.787+0000 7fa4d6063640 1 --2- 172.21.15.37:0/3052684856 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fa4d00cbe50 0x7fa4d006d650 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:29:53.796 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:53.787+0000 7fa4d5061640 1 --2- 172.21.15.37:0/3052684856 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fa4d00cbe50 0x7fa4d006d650 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:29:53.797 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:53.787+0000 7fa4d5061640 1 -- 172.21.15.37:0/3052684856 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fa4d01dc070 con 0x7fa4d00cb500 2024-04-25T10:29:53.797 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:53.787+0000 7fa4d5061640 1 --2- 172.21.15.37:0/3052684856 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fa4d00cb500 0x7fa4d00cb910 secure :-1 s=READY pgs=321 cs=0 l=1 rev1=1 crypto rx=0x7fa4c4011960 tx=0x7fa4c4011e20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:29:53.797 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:53.787+0000 7fa4ce7fc640 1 -- 172.21.15.37:0/3052684856 <== mon.0 v2:172.21.15.37:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa4c400faf0 con 0x7fa4d00cb500 2024-04-25T10:29:53.797 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:53.787+0000 7fa4ce7fc640 1 -- 172.21.15.37:0/3052684856 <== mon.0 v2:172.21.15.37:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fa4c400fc90 con 0x7fa4d00cb500 2024-04-25T10:29:53.797 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:53.787+0000 7fa4ce7fc640 1 -- 172.21.15.37:0/3052684856 <== mon.0 v2:172.21.15.37:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa4c4014440 con 0x7fa4d00cb500 2024-04-25T10:29:53.797 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:53.787+0000 7fa4d7ac6640 1 -- 172.21.15.37:0/3052684856 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fa4d006e9d0 con 0x7fa4d00cb500 2024-04-25T10:29:53.797 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:53.787+0000 7fa4d7ac6640 1 -- 172.21.15.37:0/3052684856 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fa4d006ef90 con 0x7fa4d00cb500 2024-04-25T10:29:53.798 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:53.788+0000 7fa4d7ac6640 1 -- 172.21.15.37:0/3052684856 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fa4d006a140 con 0x7fa4d00cb500 2024-04-25T10:29:53.798 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:53.789+0000 7fa4ce7fc640 1 -- 172.21.15.37:0/3052684856 <== mon.0 v2:172.21.15.37:3300/0 4 ==== mgrmap(e 5) ==== 96782+0+0 (secure 0 0 0) 0x7fa4c4012040 con 0x7fa4d00cb500 2024-04-25T10:29:53.798 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:53.789+0000 7fa4ce7fc640 1 --2- 172.21.15.37:0/3052684856 >> [v2:172.21.15.203:6832/2418205237,v1:172.21.15.203:6833/2418205237] conn(0x7fa4bc0744b0 0x7fa4bc0769c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:29:53.798 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:53.790+0000 7fa4ce7fc640 1 -- 172.21.15.37:0/3052684856 <== mon.0 v2:172.21.15.37:3300/0 5 ==== osd_map(25..25 src has 1..25) ==== 5738+0+0 (secure 0 0 0) 0x7fa4c4095970 con 0x7fa4d00cb500 2024-04-25T10:29:53.798 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:53.790+0000 7fa4d5862640 1 --2- 172.21.15.37:0/3052684856 >> [v2:172.21.15.203:6832/2418205237,v1:172.21.15.203:6833/2418205237] conn(0x7fa4bc0744b0 0x7fa4bc0769c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:29:53.799 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:53.790+0000 7fa4d5862640 1 --2- 172.21.15.37:0/3052684856 >> [v2:172.21.15.203:6832/2418205237,v1:172.21.15.203:6833/2418205237] conn(0x7fa4bc0744b0 0x7fa4bc0769c0 secure :-1 s=READY pgs=167 cs=0 l=1 rev1=1 crypto rx=0x7fa4d006c9e0 tx=0x7fa4c8008040 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:29:53.799 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:53.794+0000 7fa4ce7fc640 1 -- 172.21.15.37:0/3052684856 <== mon.0 v2:172.21.15.37:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187246 (secure 0 0 0) 0x7fa4d01dc070 con 0x7fa4d00cb500 2024-04-25T10:29:53.981 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:53.979+0000 7fa4d7ac6640 1 -- 172.21.15.37:0/3052684856 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"} v 0) -- 0x7fa4d0069060 con 0x7fa4d00cb500 2024-04-25T10:29:53.981 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:53.979+0000 7fa4ce7fc640 1 -- 172.21.15.37:0/3052684856 <== mon.0 v2:172.21.15.37:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]=0 v25) ==== 100+0+11 (secure 0 0 0) 0x7fa4c4019030 con 0x7fa4d00cb500 2024-04-25T10:29:53.982 INFO:teuthology.orchestra.run.smithi037.stdout:pg_num: 64 2024-04-25T10:29:53.984 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:53.981+0000 7fa4d7ac6640 1 -- 172.21.15.37:0/3052684856 >> [v2:172.21.15.203:6832/2418205237,v1:172.21.15.203:6833/2418205237] conn(0x7fa4bc0744b0 msgr2=0x7fa4bc0769c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:29:53.984 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:53.981+0000 7fa4d7ac6640 1 --2- 172.21.15.37:0/3052684856 >> [v2:172.21.15.203:6832/2418205237,v1:172.21.15.203:6833/2418205237] conn(0x7fa4bc0744b0 0x7fa4bc0769c0 secure :-1 s=READY pgs=167 cs=0 l=1 rev1=1 crypto rx=0x7fa4d006c9e0 tx=0x7fa4c8008040 comp rx=0 tx=0).stop 2024-04-25T10:29:53.984 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:53.982+0000 7fa4d7ac6640 1 -- 172.21.15.37:0/3052684856 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fa4d00cb500 msgr2=0x7fa4d00cb910 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:29:53.984 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:53.982+0000 7fa4d7ac6640 1 --2- 172.21.15.37:0/3052684856 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fa4d00cb500 0x7fa4d00cb910 secure :-1 s=READY pgs=321 cs=0 l=1 rev1=1 crypto rx=0x7fa4c4011960 tx=0x7fa4c4011e20 comp rx=0 tx=0).stop 2024-04-25T10:29:53.984 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:53.982+0000 7fa4d7ac6640 1 -- 172.21.15.37:0/3052684856 shutdown_connections 2024-04-25T10:29:53.984 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:53.982+0000 7fa4d7ac6640 1 --2- 172.21.15.37:0/3052684856 >> [v2:172.21.15.203:6832/2418205237,v1:172.21.15.203:6833/2418205237] conn(0x7fa4bc0744b0 0x7fa4bc0769c0 unknown :-1 s=CLOSED pgs=167 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:29:53.985 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:53.982+0000 7fa4d7ac6640 1 --2- 172.21.15.37:0/3052684856 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fa4d00cbe50 0x7fa4d006d650 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:29:53.985 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:53.982+0000 7fa4d7ac6640 1 --2- 172.21.15.37:0/3052684856 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fa4d00cb500 0x7fa4d00cb910 unknown :-1 s=CLOSED pgs=321 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:29:53.985 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:53.982+0000 7fa4d7ac6640 1 --2- 172.21.15.37:0/3052684856 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7fa4d00a08b0 0x7fa4d01dbb30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:29:53.985 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:53.982+0000 7fa4d7ac6640 1 -- 172.21.15.37:0/3052684856 >> 172.21.15.37:0/3052684856 conn(0x7fa4d00d03d0 msgr2=0x7fa4d0076000 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:29:53.985 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:53.982+0000 7fa4d7ac6640 1 -- 172.21.15.37:0/3052684856 shutdown_connections 2024-04-25T10:29:53.985 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:53.982+0000 7fa4d7ac6640 1 -- 172.21.15.37:0/3052684856 wait complete. 2024-04-25T10:29:53.995 DEBUG:teuthology.orchestra.run.smithi037:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd blocklist ls 2024-04-25T10:29:54.107 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:54.105+0000 7f3527d57640 1 Processor -- start 2024-04-25T10:29:54.107 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:54.105+0000 7f3527d57640 1 -- start start 2024-04-25T10:29:54.109 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:54.106+0000 7f3527d57640 1 --2- >> v2:172.21.15.203:3300/0 conn(0x7f35200b4bb0 0x7f35201646b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:29:54.109 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:54.106+0000 7f3527d57640 1 --2- >> v2:172.21.15.37:3300/0 conn(0x7f35200b7600 0x7f35200b7a10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:29:54.109 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:54.106+0000 7f3527d57640 1 -- --> v1:172.21.15.37:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f35200b7f50 con 0x7f35200d4560 2024-04-25T10:29:54.109 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:54.106+0000 7f3527d57640 1 -- --> v2:172.21.15.37:3300/0 -- mon_getmap magic: 0 -- 0x7f35200d5080 con 0x7f35200b7600 2024-04-25T10:29:54.109 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:54.106+0000 7f3527d57640 1 -- --> v2:172.21.15.203:3300/0 -- mon_getmap magic: 0 -- 0x7f3520161190 con 0x7f35200b4bb0 2024-04-25T10:29:54.109 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:54.106+0000 7f35252f2640 1 --2- >> v2:172.21.15.37:3300/0 conn(0x7f35200b7600 0x7f35200b7a10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:29:54.109 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:54.106+0000 7f35252f2640 1 --2- >> v2:172.21.15.37:3300/0 conn(0x7f35200b7600 0x7f35200b7a10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.37:3300/0 says I am v2:172.21.15.37:36404/0 (socket says 172.21.15.37:36404) 2024-04-25T10:29:54.110 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:54.106+0000 7f35262f4640 1 --1- >> v1:172.21.15.37:6789/0 conn(0x7f35200d4560 0x7f35200d4970 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.37:6789/0 says I am v1:172.21.15.37:56822/0 (socket says 172.21.15.37:56822) 2024-04-25T10:29:54.110 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:54.106+0000 7f35252f2640 1 -- 172.21.15.37:0/800625465 learned_addr learned my addr 172.21.15.37:0/800625465 (peer_addr_for_me v2:172.21.15.37:0/0) 2024-04-25T10:29:54.110 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:54.107+0000 7f3525af3640 1 --2- 172.21.15.37:0/800625465 >> v2:172.21.15.203:3300/0 conn(0x7f35200b4bb0 0x7f35201646b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:29:54.110 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:54.107+0000 7f35252f2640 1 -- 172.21.15.37:0/800625465 >> v2:172.21.15.203:3300/0 conn(0x7f35200b4bb0 msgr2=0x7f35201646b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:29:54.110 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:54.107+0000 7f35252f2640 1 --2- 172.21.15.37:0/800625465 >> v2:172.21.15.203:3300/0 conn(0x7f35200b4bb0 0x7f35201646b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:29:54.110 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:54.107+0000 7f35252f2640 1 -- 172.21.15.37:0/800625465 >> v1:172.21.15.37:6789/0 conn(0x7f35200d4560 legacy=0x7f35200d4970 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:29:54.110 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:54.107+0000 7f3524af1640 1 -- 172.21.15.37:0/800625465 <== mon.0 v1:172.21.15.37:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 498518085 0 0) 0x7f3518001490 con 0x7f35200d4560 2024-04-25T10:29:54.110 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:54.107+0000 7f35252f2640 1 -- 172.21.15.37:0/800625465 --> v2:172.21.15.37:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f3520161470 con 0x7f35200b7600 2024-04-25T10:29:54.111 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:54.107+0000 7f35252f2640 1 --2- 172.21.15.37:0/800625465 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f35200b7600 0x7f35200b7a10 secure :-1 s=READY pgs=323 cs=0 l=1 rev1=1 crypto rx=0x7f352008e640 tx=0x7f351002a270 comp rx=0 tx=0).ready entity=mon.0 client_cookie=384abe4b84fc615d server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:29:54.111 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:54.108+0000 7f3524af1640 1 -- 172.21.15.37:0/800625465 <== mon.0 v2:172.21.15.37:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3510037070 con 0x7f35200b7600 2024-04-25T10:29:54.111 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:54.108+0000 7f3524af1640 1 -- 172.21.15.37:0/800625465 <== mon.0 v2:172.21.15.37:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f3510032440 con 0x7f35200b7600 2024-04-25T10:29:54.111 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:54.108+0000 7f3524af1640 1 -- 172.21.15.37:0/800625465 <== mon.0 v2:172.21.15.37:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3510032810 con 0x7f35200b7600 2024-04-25T10:29:54.111 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:54.108+0000 7f3527d57640 1 -- 172.21.15.37:0/800625465 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f35200b7600 msgr2=0x7f35200b7a10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:29:54.112 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:54.108+0000 7f3527d57640 1 --2- 172.21.15.37:0/800625465 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f35200b7600 0x7f35200b7a10 secure :-1 s=READY pgs=323 cs=0 l=1 rev1=1 crypto rx=0x7f352008e640 tx=0x7f351002a270 comp rx=0 tx=0).stop 2024-04-25T10:29:54.112 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:54.108+0000 7f3527d57640 1 -- 172.21.15.37:0/800625465 shutdown_connections 2024-04-25T10:29:54.112 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:54.108+0000 7f3527d57640 1 --2- 172.21.15.37:0/800625465 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f35200b7600 0x7f35200b7a10 unknown :-1 s=CLOSED pgs=323 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:29:54.112 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:54.108+0000 7f3527d57640 1 --2- 172.21.15.37:0/800625465 >> v2:172.21.15.203:3300/0 conn(0x7f35200b4bb0 0x7f35201646b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:29:54.112 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:54.108+0000 7f3527d57640 1 -- 172.21.15.37:0/800625465 >> 172.21.15.37:0/800625465 conn(0x7f35200d27e0 msgr2=0x7f35200d2bf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:29:54.112 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:54.108+0000 7f3527d57640 1 -- 172.21.15.37:0/800625465 shutdown_connections 2024-04-25T10:29:54.112 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:54.108+0000 7f3527d57640 1 -- 172.21.15.37:0/800625465 wait complete. 2024-04-25T10:29:54.112 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:54.110+0000 7f3527d57640 1 Processor -- start 2024-04-25T10:29:54.113 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:54.110+0000 7f3527d57640 1 -- start start 2024-04-25T10:29:54.115 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:54.111+0000 7f3527d57640 1 --2- >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f35200b4bb0 0x7f35200cc230 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:29:54.115 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:54.111+0000 7f3527d57640 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f35200b7600 0x7f352015f5e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:29:54.115 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:54.111+0000 7f3527d57640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f35200d4560 0x7f35200ce6f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:29:54.115 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:54.111+0000 7f3527d57640 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 -- 0x7f35200b4fc0 con 0x7f35200d4560 2024-04-25T10:29:54.115 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:54.111+0000 7f3527d57640 1 -- --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_getmap magic: 0 -- 0x7f3520164190 con 0x7f35200b7600 2024-04-25T10:29:54.115 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:54.111+0000 7f3527d57640 1 -- --> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] -- mon_getmap magic: 0 -- 0x7f3520164bf0 con 0x7f35200b4bb0 2024-04-25T10:29:54.115 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:54.112+0000 7f35262f4640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f35200d4560 0x7f35200ce6f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:29:54.115 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:54.112+0000 7f35262f4640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f35200d4560 0x7f35200ce6f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.37:3300/0 says I am v2:172.21.15.37:36414/0 (socket says 172.21.15.37:36414) 2024-04-25T10:29:54.115 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:54.112+0000 7f35262f4640 1 -- 172.21.15.37:0/3717599261 learned_addr learned my addr 172.21.15.37:0/3717599261 (peer_addr_for_me v2:172.21.15.37:0/0) 2024-04-25T10:29:54.115 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:54.112+0000 7f35252f2640 1 --2- 172.21.15.37:0/3717599261 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f35200b7600 0x7f352015f5e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:29:54.115 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:54.112+0000 7f3525af3640 1 --2- 172.21.15.37:0/3717599261 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f35200b4bb0 0x7f35200cc230 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:29:54.116 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:54.112+0000 7f35262f4640 1 -- 172.21.15.37:0/3717599261 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f35200b4bb0 msgr2=0x7f35200cc230 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:29:54.116 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:54.112+0000 7f35262f4640 1 --2- 172.21.15.37:0/3717599261 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f35200b4bb0 0x7f35200cc230 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:29:54.116 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:54.112+0000 7f35262f4640 1 -- 172.21.15.37:0/3717599261 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f35200b7600 msgr2=0x7f352015f5e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:29:54.116 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:54.112+0000 7f35262f4640 1 --2- 172.21.15.37:0/3717599261 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f35200b7600 0x7f352015f5e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:29:54.116 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:54.112+0000 7f35262f4640 1 -- 172.21.15.37:0/3717599261 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f35201f6240 con 0x7f35200d4560 2024-04-25T10:29:54.116 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:54.112+0000 7f35262f4640 1 --2- 172.21.15.37:0/3717599261 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f35200d4560 0x7f35200ce6f0 secure :-1 s=READY pgs=324 cs=0 l=1 rev1=1 crypto rx=0x7f351800c6d0 tx=0x7f351800cb90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:29:54.116 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:54.112+0000 7f35167fc640 1 -- 172.21.15.37:0/3717599261 <== mon.0 v2:172.21.15.37:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3518018070 con 0x7f35200d4560 2024-04-25T10:29:54.116 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:54.112+0000 7f35167fc640 1 -- 172.21.15.37:0/3717599261 <== mon.0 v2:172.21.15.37:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f3518004d60 con 0x7f35200d4560 2024-04-25T10:29:54.116 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:54.112+0000 7f3527d57640 1 -- 172.21.15.37:0/3717599261 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f35201f6500 con 0x7f35200d4560 2024-04-25T10:29:54.116 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:54.112+0000 7f35167fc640 1 -- 172.21.15.37:0/3717599261 <== mon.0 v2:172.21.15.37:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f351800a880 con 0x7f35200d4560 2024-04-25T10:29:54.116 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:54.112+0000 7f3527d57640 1 -- 172.21.15.37:0/3717599261 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f35201f6ac0 con 0x7f35200d4560 2024-04-25T10:29:54.117 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:54.114+0000 7f3527d57640 1 -- 172.21.15.37:0/3717599261 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f352006a140 con 0x7f35200d4560 2024-04-25T10:29:54.117 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:54.114+0000 7f35167fc640 1 -- 172.21.15.37:0/3717599261 <== mon.0 v2:172.21.15.37:3300/0 4 ==== mgrmap(e 5) ==== 96782+0+0 (secure 0 0 0) 0x7f351801b070 con 0x7f35200d4560 2024-04-25T10:29:54.117 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:54.115+0000 7f35167fc640 1 --2- 172.21.15.37:0/3717599261 >> [v2:172.21.15.203:6832/2418205237,v1:172.21.15.203:6833/2418205237] conn(0x7f35040744b0 0x7f35040769c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:29:54.117 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:54.115+0000 7f35167fc640 1 -- 172.21.15.37:0/3717599261 <== mon.0 v2:172.21.15.37:3300/0 5 ==== osd_map(25..25 src has 1..25) ==== 5738+0+0 (secure 0 0 0) 0x7f35180949d0 con 0x7f35200d4560 2024-04-25T10:29:54.117 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:54.115+0000 7f3525af3640 1 --2- 172.21.15.37:0/3717599261 >> [v2:172.21.15.203:6832/2418205237,v1:172.21.15.203:6833/2418205237] conn(0x7f35040744b0 0x7f35040769c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:29:54.118 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:54.115+0000 7f3525af3640 1 --2- 172.21.15.37:0/3717599261 >> [v2:172.21.15.203:6832/2418205237,v1:172.21.15.203:6833/2418205237] conn(0x7f35040744b0 0x7f35040769c0 secure :-1 s=READY pgs=168 cs=0 l=1 rev1=1 crypto rx=0x7f35200d6c00 tx=0x7f351c008040 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:29:54.121 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:54.119+0000 7f35167fc640 1 -- 172.21.15.37:0/3717599261 <== mon.0 v2:172.21.15.37:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187246 (secure 0 0 0) 0x7f35200b7f50 con 0x7f35200d4560 2024-04-25T10:29:54.320 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:54.317+0000 7f3527d57640 1 -- 172.21.15.37:0/3717599261 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_command({"prefix": "osd blocklist ls"} v 0) -- 0x7f35200d3ff0 con 0x7f35200d4560 2024-04-25T10:29:54.320 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:54.318+0000 7f35167fc640 1 -- 172.21.15.37:0/3717599261 <== mon.0 v2:172.21.15.37:3300/0 7 ==== mon_command_ack([{"prefix": "osd blocklist ls"}]=0 listed 0 entries v25) ==== 80+0+0 (secure 0 0 0) 0x7f351805ea50 con 0x7f35200d4560 2024-04-25T10:29:54.320 INFO:teuthology.orchestra.run.smithi037.stderr:listed 0 entries 2024-04-25T10:29:54.322 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:54.320+0000 7f3527d57640 1 -- 172.21.15.37:0/3717599261 >> [v2:172.21.15.203:6832/2418205237,v1:172.21.15.203:6833/2418205237] conn(0x7f35040744b0 msgr2=0x7f35040769c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:29:54.322 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:54.320+0000 7f3527d57640 1 --2- 172.21.15.37:0/3717599261 >> [v2:172.21.15.203:6832/2418205237,v1:172.21.15.203:6833/2418205237] conn(0x7f35040744b0 0x7f35040769c0 secure :-1 s=READY pgs=168 cs=0 l=1 rev1=1 crypto rx=0x7f35200d6c00 tx=0x7f351c008040 comp rx=0 tx=0).stop 2024-04-25T10:29:54.322 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:54.320+0000 7f3527d57640 1 -- 172.21.15.37:0/3717599261 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f35200d4560 msgr2=0x7f35200ce6f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:29:54.322 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:54.320+0000 7f3527d57640 1 --2- 172.21.15.37:0/3717599261 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f35200d4560 0x7f35200ce6f0 secure :-1 s=READY pgs=324 cs=0 l=1 rev1=1 crypto rx=0x7f351800c6d0 tx=0x7f351800cb90 comp rx=0 tx=0).stop 2024-04-25T10:29:54.322 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:54.320+0000 7f3527d57640 1 -- 172.21.15.37:0/3717599261 shutdown_connections 2024-04-25T10:29:54.322 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:54.320+0000 7f3527d57640 1 --2- 172.21.15.37:0/3717599261 >> [v2:172.21.15.203:6832/2418205237,v1:172.21.15.203:6833/2418205237] conn(0x7f35040744b0 0x7f35040769c0 unknown :-1 s=CLOSED pgs=168 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:29:54.323 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:54.320+0000 7f3527d57640 1 --2- 172.21.15.37:0/3717599261 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f35200d4560 0x7f35200ce6f0 unknown :-1 s=CLOSED pgs=324 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:29:54.323 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:54.320+0000 7f3527d57640 1 --2- 172.21.15.37:0/3717599261 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f35200b7600 0x7f352015f5e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:29:54.323 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:54.320+0000 7f3527d57640 1 --2- 172.21.15.37:0/3717599261 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f35200b4bb0 0x7f35200cc230 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:29:54.323 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:54.320+0000 7f3527d57640 1 -- 172.21.15.37:0/3717599261 >> 172.21.15.37:0/3717599261 conn(0x7f35200d27e0 msgr2=0x7f35200d0600 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:29:54.323 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:54.320+0000 7f3527d57640 1 -- 172.21.15.37:0/3717599261 shutdown_connections 2024-04-25T10:29:54.323 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:54.320+0000 7f3527d57640 1 -- 172.21.15.37:0/3717599261 wait complete. 2024-04-25T10:29:54.333 INFO:tasks.cephfs.fuse_mount:Running fusermount -u on ubuntu@smithi037.front.sepia.ceph.com... 2024-04-25T10:29:54.333 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T10:29:54.333 DEBUG:teuthology.orchestra.run.smithi037:> sudo fusermount -u /home/ubuntu/cephtest/mnt.2 2024-04-25T10:29:54.358 INFO:teuthology.orchestra.run:waiting for 300 2024-04-25T10:29:54.358 INFO:tasks.cephfs.mount:Cleaning up mount ubuntu@smithi037.front.sepia.ceph.com 2024-04-25T10:29:54.358 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T10:29:54.359 DEBUG:teuthology.orchestra.run.smithi037:> (cd /home/ubuntu/cephtest && exec rmdir -- /home/ubuntu/cephtest/mnt.2) 2024-04-25T10:29:54.415 DEBUG:teuthology.orchestra.run.smithi203:> set -ex 2024-04-25T10:29:54.415 DEBUG:teuthology.orchestra.run.smithi203:> dd if=/proc/self/mounts of=/dev/stdout 2024-04-25T10:29:54.434 DEBUG:teuthology.orchestra.run.smithi203:> set -ex 2024-04-25T10:29:54.434 DEBUG:teuthology.orchestra.run.smithi203:> dd if=/proc/self/mounts of=/dev/stdout 2024-04-25T10:29:54.493 DEBUG:teuthology.orchestra.run.smithi037:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-04-25T10:29:54.604 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:54.602+0000 7f833c17d640 1 Processor -- start 2024-04-25T10:29:54.605 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:54.603+0000 7f833c17d640 1 -- start start 2024-04-25T10:29:54.607 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:54.604+0000 7f833c17d640 1 --2- >> v2:172.21.15.203:3300/0 conn(0x7f83340d2860 0x7f833406e200 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:29:54.607 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:54.604+0000 7f833c17d640 1 --2- >> v2:172.21.15.37:3300/0 conn(0x7f83340ce560 0x7f83340ce970 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:29:54.607 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:54.604+0000 7f833c17d640 1 -- --> v1:172.21.15.37:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f83340d2e40 con 0x7f83340b2dc0 2024-04-25T10:29:54.607 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:54.604+0000 7f833c17d640 1 -- --> v2:172.21.15.37:3300/0 -- mon_getmap magic: 0 -- 0x7f83340b92c0 con 0x7f83340ce560 2024-04-25T10:29:54.608 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:54.604+0000 7f833c17d640 1 -- --> v2:172.21.15.203:3300/0 -- mon_getmap magic: 0 -- 0x7f83340ce0d0 con 0x7f83340d2860 2024-04-25T10:29:54.608 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:54.604+0000 7f833a71a640 1 --2- >> v2:172.21.15.37:3300/0 conn(0x7f83340ce560 0x7f83340ce970 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:29:54.608 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:54.604+0000 7f833a71a640 1 --2- >> v2:172.21.15.37:3300/0 conn(0x7f83340ce560 0x7f83340ce970 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.37:3300/0 says I am v2:172.21.15.37:36416/0 (socket says 172.21.15.37:36416) 2024-04-25T10:29:54.608 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:54.604+0000 7f8339f19640 1 --1- >> v1:172.21.15.37:6789/0 conn(0x7f83340b2dc0 0x7f833406daf0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.37:6789/0 says I am v1:172.21.15.37:56832/0 (socket says 172.21.15.37:56832) 2024-04-25T10:29:54.608 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:54.604+0000 7f833a71a640 1 -- 172.21.15.37:0/793186092 learned_addr learned my addr 172.21.15.37:0/793186092 (peer_addr_for_me v2:172.21.15.37:0/0) 2024-04-25T10:29:54.608 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:54.604+0000 7f8339718640 1 --2- 172.21.15.37:0/793186092 >> v2:172.21.15.203:3300/0 conn(0x7f83340d2860 0x7f833406e200 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:29:54.608 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:54.604+0000 7f833a71a640 1 -- 172.21.15.37:0/793186092 >> v2:172.21.15.203:3300/0 conn(0x7f83340d2860 msgr2=0x7f833406e200 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:29:54.608 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:54.604+0000 7f833a71a640 1 --2- 172.21.15.37:0/793186092 >> v2:172.21.15.203:3300/0 conn(0x7f83340d2860 0x7f833406e200 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:29:54.608 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:54.604+0000 7f833a71a640 1 -- 172.21.15.37:0/793186092 >> v1:172.21.15.37:6789/0 conn(0x7f83340b2dc0 legacy=0x7f833406daf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:29:54.608 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:54.604+0000 7f833a71a640 1 -- 172.21.15.37:0/793186092 --> v2:172.21.15.37:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f83340d3120 con 0x7f83340ce560 2024-04-25T10:29:54.608 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:54.605+0000 7f833a71a640 1 --2- 172.21.15.37:0/793186092 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f83340ce560 0x7f83340ce970 secure :-1 s=READY pgs=326 cs=0 l=1 rev1=1 crypto rx=0x7f8334070ab0 tx=0x7f832402a730 comp rx=0 tx=0).ready entity=mon.0 client_cookie=3d7366c650f40b36 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:29:54.608 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:54.605+0000 7f8338f17640 1 -- 172.21.15.37:0/793186092 <== mon.0 v2:172.21.15.37:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8324036070 con 0x7f83340ce560 2024-04-25T10:29:54.609 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:54.605+0000 7f8338f17640 1 -- 172.21.15.37:0/793186092 <== mon.0 v2:172.21.15.37:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f8324031600 con 0x7f83340ce560 2024-04-25T10:29:54.609 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:54.605+0000 7f8338f17640 1 -- 172.21.15.37:0/793186092 <== mon.0 v2:172.21.15.37:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f83240319d0 con 0x7f83340ce560 2024-04-25T10:29:54.610 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:54.606+0000 7f833c17d640 1 -- 172.21.15.37:0/793186092 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f83340ce560 msgr2=0x7f83340ce970 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:29:54.610 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:54.606+0000 7f833c17d640 1 --2- 172.21.15.37:0/793186092 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f83340ce560 0x7f83340ce970 secure :-1 s=READY pgs=326 cs=0 l=1 rev1=1 crypto rx=0x7f8334070ab0 tx=0x7f832402a730 comp rx=0 tx=0).stop 2024-04-25T10:29:54.610 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:54.606+0000 7f833c17d640 1 -- 172.21.15.37:0/793186092 shutdown_connections 2024-04-25T10:29:54.610 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:54.606+0000 7f833c17d640 1 --2- 172.21.15.37:0/793186092 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f83340ce560 0x7f83340ce970 unknown :-1 s=CLOSED pgs=326 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:29:54.610 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:54.606+0000 7f833c17d640 1 --2- 172.21.15.37:0/793186092 >> v2:172.21.15.203:3300/0 conn(0x7f83340d2860 0x7f833406e200 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:29:54.610 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:54.606+0000 7f833c17d640 1 -- 172.21.15.37:0/793186092 >> 172.21.15.37:0/793186092 conn(0x7f83340b5440 msgr2=0x7f83340b5850 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:29:54.610 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:54.606+0000 7f833c17d640 1 -- 172.21.15.37:0/793186092 shutdown_connections 2024-04-25T10:29:54.610 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:54.606+0000 7f833c17d640 1 -- 172.21.15.37:0/793186092 wait complete. 2024-04-25T10:29:54.610 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:54.608+0000 7f833c17d640 1 Processor -- start 2024-04-25T10:29:54.611 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:54.608+0000 7f833c17d640 1 -- start start 2024-04-25T10:29:54.613 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:54.609+0000 7f833c17d640 1 --2- >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f83340b2dc0 0x7f8334156260 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:29:54.613 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:54.609+0000 7f833c17d640 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f83340ce560 0x7f83341f9b40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:29:54.613 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:54.609+0000 7f833c17d640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f83340d2860 0x7f83341fa080 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:29:54.613 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:54.609+0000 7f833c17d640 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 -- 0x7f833406e740 con 0x7f83340d2860 2024-04-25T10:29:54.613 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:54.609+0000 7f833c17d640 1 -- --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_getmap magic: 0 -- 0x7f83340d36e0 con 0x7f83340ce560 2024-04-25T10:29:54.613 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:54.609+0000 7f833c17d640 1 -- --> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] -- mon_getmap magic: 0 -- 0x7f833406e8c0 con 0x7f83340b2dc0 2024-04-25T10:29:54.613 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:54.609+0000 7f833a71a640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f83340d2860 0x7f83341fa080 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:29:54.613 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:54.609+0000 7f833a71a640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f83340d2860 0x7f83341fa080 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.37:3300/0 says I am v2:172.21.15.37:36428/0 (socket says 172.21.15.37:36428) 2024-04-25T10:29:54.614 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:54.609+0000 7f833a71a640 1 -- 172.21.15.37:0/1700788318 learned_addr learned my addr 172.21.15.37:0/1700788318 (peer_addr_for_me v2:172.21.15.37:0/0) 2024-04-25T10:29:54.614 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:54.609+0000 7f8339718640 1 --2- 172.21.15.37:0/1700788318 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f83340ce560 0x7f83341f9b40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:29:54.614 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:54.609+0000 7f8339f19640 1 --2- 172.21.15.37:0/1700788318 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f83340b2dc0 0x7f8334156260 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:29:54.614 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:54.610+0000 7f833a71a640 1 -- 172.21.15.37:0/1700788318 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f83340b2dc0 msgr2=0x7f8334156260 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:29:54.614 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:54.610+0000 7f833a71a640 1 --2- 172.21.15.37:0/1700788318 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f83340b2dc0 0x7f8334156260 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:29:54.614 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:54.610+0000 7f833a71a640 1 -- 172.21.15.37:0/1700788318 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f83340ce560 msgr2=0x7f83341f9b40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:29:54.614 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:54.610+0000 7f833a71a640 1 --2- 172.21.15.37:0/1700788318 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f83340ce560 0x7f83341f9b40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:29:54.614 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:54.610+0000 7f833a71a640 1 -- 172.21.15.37:0/1700788318 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f8334156f20 con 0x7f83340d2860 2024-04-25T10:29:54.614 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:54.610+0000 7f8339f19640 1 --2- 172.21.15.37:0/1700788318 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f83340b2dc0 0x7f8334156260 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T10:29:54.614 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:54.610+0000 7f833a71a640 1 --2- 172.21.15.37:0/1700788318 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f83340d2860 0x7f83341fa080 secure :-1 s=READY pgs=327 cs=0 l=1 rev1=1 crypto rx=0x7f8324009f40 tx=0x7f8324004a40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:29:54.614 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:54.610+0000 7f83327fc640 1 -- 172.21.15.37:0/1700788318 <== mon.0 v2:172.21.15.37:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8324036050 con 0x7f83340d2860 2024-04-25T10:29:54.615 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:54.610+0000 7f83327fc640 1 -- 172.21.15.37:0/1700788318 <== mon.0 v2:172.21.15.37:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f8324042070 con 0x7f83340d2860 2024-04-25T10:29:54.615 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:54.610+0000 7f833c17d640 1 -- 172.21.15.37:0/1700788318 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f8334157200 con 0x7f83340d2860 2024-04-25T10:29:54.615 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:54.610+0000 7f83327fc640 1 -- 172.21.15.37:0/1700788318 <== mon.0 v2:172.21.15.37:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8324009510 con 0x7f83340d2860 2024-04-25T10:29:54.615 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:54.610+0000 7f833c17d640 1 -- 172.21.15.37:0/1700788318 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f83340cb960 con 0x7f83340d2860 2024-04-25T10:29:54.615 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:54.611+0000 7f833c17d640 1 -- 172.21.15.37:0/1700788318 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f8334156c90 con 0x7f83340d2860 2024-04-25T10:29:54.616 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:54.612+0000 7f83327fc640 1 -- 172.21.15.37:0/1700788318 <== mon.0 v2:172.21.15.37:3300/0 4 ==== mgrmap(e 5) ==== 96782+0+0 (secure 0 0 0) 0x7f8324007bf0 con 0x7f83340d2860 2024-04-25T10:29:54.616 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:54.612+0000 7f83327fc640 1 --2- 172.21.15.37:0/1700788318 >> [v2:172.21.15.203:6832/2418205237,v1:172.21.15.203:6833/2418205237] conn(0x7f83200744b0 0x7f83200769c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:29:54.616 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:54.612+0000 7f83327fc640 1 -- 172.21.15.37:0/1700788318 <== mon.0 v2:172.21.15.37:3300/0 5 ==== osd_map(25..25 src has 1..25) ==== 5738+0+0 (secure 0 0 0) 0x7f83240b5fa0 con 0x7f83340d2860 2024-04-25T10:29:54.616 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:54.612+0000 7f8339f19640 1 --2- 172.21.15.37:0/1700788318 >> [v2:172.21.15.203:6832/2418205237,v1:172.21.15.203:6833/2418205237] conn(0x7f83200744b0 0x7f83200769c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:29:54.616 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:54.613+0000 7f8339f19640 1 --2- 172.21.15.37:0/1700788318 >> [v2:172.21.15.203:6832/2418205237,v1:172.21.15.203:6833/2418205237] conn(0x7f83200744b0 0x7f83200769c0 secure :-1 s=READY pgs=169 cs=0 l=1 rev1=1 crypto rx=0x7f832c007fd0 tx=0x7f832c009040 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:29:54.618 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:54.617+0000 7f83327fc640 1 -- 172.21.15.37:0/1700788318 <== mon.0 v2:172.21.15.37:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187246 (secure 0 0 0) 0x7f8324080710 con 0x7f83340d2860 2024-04-25T10:29:54.803 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:54.801+0000 7f833c17d640 1 -- 172.21.15.37:0/1700788318 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7f83340722e0 con 0x7f83340d2860 2024-04-25T10:29:54.804 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:54.802+0000 7f83327fc640 1 -- 172.21.15.37:0/1700788318 <== mon.0 v2:172.21.15.37:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v25) ==== 74+0+16670 (secure 0 0 0) 0x7f8324080010 con 0x7f83340d2860 2024-04-25T10:29:54.805 INFO:teuthology.orchestra.run.smithi037.stdout: 2024-04-25T10:29:54.805 INFO:teuthology.orchestra.run.smithi037.stdout:{"epoch":25,"fsid":"f4acda28-be34-492a-b286-994afd36f61d","created":"2024-04-25T10:28:43.985718+0000","modified":"2024-04-25T10:29:49.325719+0000","last_up_change":"2024-04-25T10:28:54.900654+0000","last_in_change":"2024-04-25T10:28:46.216973+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":3,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"luminous","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-04-25T10:28:56.202153+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"19","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2024-04-25T10:29:06.507244+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"22","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":2.25,"score_stable":2.25,"optimal_score":1,"raw_score_acting":2.25,"raw_score_stable":2.25,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2024-04-25T10:29:07.347844+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"23","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"a2f79028-b332-4166-9ebb-a7024a37ea22","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":21,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.37:6816","nonce":2746074720},{"type":"v1","addr":"172.21.15.37:6817","nonce":2746074720}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.37:6818","nonce":2746074720},{"type":"v1","addr":"172.21.15.37:6819","nonce":2746074720}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.37:6822","nonce":2746074720},{"type":"v1","addr":"172.21.15.37:6823","nonce":2746074720}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.37:6820","nonce":2746074720},{"type":"v1","addr":"172.21.15.37:6821","nonce":2746074720}]},"public_addr":"172.21.15.37:6817/2746074720","cluster_addr":"172.21.15.37:6819/2746074720","heartbeat_back_addr":"172.21.15.37:6823/2746074720","heartbeat_front_addr":"172.21.15.37:6821/2746074720","state":["exists","up"]},{"osd":1,"uuid":"973159de-c2ea-4083-aecc-69373d1c1132","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":21,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.37:6800","nonce":1690341662},{"type":"v1","addr":"172.21.15.37:6801","nonce":1690341662}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.37:6802","nonce":1690341662},{"type":"v1","addr":"172.21.15.37:6803","nonce":1690341662}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.37:6806","nonce":1690341662},{"type":"v1","addr":"172.21.15.37:6807","nonce":1690341662}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.37:6804","nonce":1690341662},{"type":"v1","addr":"172.21.15.37:6805","nonce":1690341662}]},"public_addr":"172.21.15.37:6801/1690341662","cluster_addr":"172.21.15.37:6803/1690341662","heartbeat_back_addr":"172.21.15.37:6807/1690341662","heartbeat_front_addr":"172.21.15.37:6805/1690341662","state":["exists","up"]},{"osd":2,"uuid":"4ae7a0e9-cb3f-484c-a75f-f3766b721c47","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":21,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.37:6808","nonce":2754669649},{"type":"v1","addr":"172.21.15.37:6809","nonce":2754669649}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.37:6810","nonce":2754669649},{"type":"v1","addr":"172.21.15.37:6811","nonce":2754669649}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.37:6814","nonce":2754669649},{"type":"v1","addr":"172.21.15.37:6815","nonce":2754669649}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.37:6812","nonce":2754669649},{"type":"v1","addr":"172.21.15.37:6813","nonce":2754669649}]},"public_addr":"172.21.15.37:6809/2754669649","cluster_addr":"172.21.15.37:6811/2754669649","heartbeat_back_addr":"172.21.15.37:6815/2754669649","heartbeat_front_addr":"172.21.15.37:6813/2754669649","state":["exists","up"]},{"osd":3,"uuid":"1ba4a746-1608-47a7-ace2-960a5cfae934","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":21,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.37:6824","nonce":1815290618},{"type":"v1","addr":"172.21.15.37:6825","nonce":1815290618}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.37:6826","nonce":1815290618},{"type":"v1","addr":"172.21.15.37:6827","nonce":1815290618}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.37:6830","nonce":1815290618},{"type":"v1","addr":"172.21.15.37:6831","nonce":1815290618}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.37:6828","nonce":1815290618},{"type":"v1","addr":"172.21.15.37:6829","nonce":1815290618}]},"public_addr":"172.21.15.37:6825/1815290618","cluster_addr":"172.21.15.37:6827/1815290618","heartbeat_back_addr":"172.21.15.37:6831/1815290618","heartbeat_front_addr":"172.21.15.37:6829/1815290618","state":["exists","up"]},{"osd":4,"uuid":"d4f59f5d-0413-4b79-87ee-bb212c1fcf62","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":21,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6800","nonce":4051532063},{"type":"v1","addr":"172.21.15.203:6801","nonce":4051532063}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6802","nonce":4051532063},{"type":"v1","addr":"172.21.15.203:6803","nonce":4051532063}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6806","nonce":4051532063},{"type":"v1","addr":"172.21.15.203:6807","nonce":4051532063}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6804","nonce":4051532063},{"type":"v1","addr":"172.21.15.203:6805","nonce":4051532063}]},"public_addr":"172.21.15.203:6801/4051532063","cluster_addr":"172.21.15.203:6803/4051532063","heartbeat_back_addr":"172.21.15.203:6807/4051532063","heartbeat_front_addr":"172.21.15.203:6805/4051532063","state":["exists","up"]},{"osd":5,"uuid":"69710a0e-41d8-4e7f-9a10-17ce540b67be","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":24,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6824","nonce":1203822890},{"type":"v1","addr":"172.21.15.203:6825","nonce":1203822890}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6826","nonce":1203822890},{"type":"v1","addr":"172.21.15.203:6827","nonce":1203822890}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6830","nonce":1203822890},{"type":"v1","addr":"172.21.15.203:6831","nonce":1203822890}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6828","nonce":1203822890},{"type":"v1","addr":"172.21.15.203:6829","nonce":1203822890}]},"public_addr":"172.21.15.203:6825/1203822890","cluster_addr":"172.21.15.203:6827/1203822890","heartbeat_back_addr":"172.21.15.203:6831/1203822890","heartbeat_front_addr":"172.21.15.203:6829/1203822890","state":["exists","up"]},{"osd":6,"uuid":"f29dd047-3b48-473d-a235-6642a6293f20","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":21,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6810","nonce":2309615014},{"type":"v1","addr":"172.21.15.203:6812","nonce":2309615014}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6814","nonce":2309615014},{"type":"v1","addr":"172.21.15.203:6816","nonce":2309615014}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6822","nonce":2309615014},{"type":"v1","addr":"172.21.15.203:6823","nonce":2309615014}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6818","nonce":2309615014},{"type":"v1","addr":"172.21.15.203:6820","nonce":2309615014}]},"public_addr":"172.21.15.203:6812/2309615014","cluster_addr":"172.21.15.203:6816/2309615014","heartbeat_back_addr":"172.21.15.203:6823/2309615014","heartbeat_front_addr":"172.21.15.203:6820/2309615014","state":["exists","up"]},{"osd":7,"uuid":"6e691644-378b-4fdc-8e86-fb5f56e31f29","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":21,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6808","nonce":3688099014},{"type":"v1","addr":"172.21.15.203:6809","nonce":3688099014}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6811","nonce":3688099014},{"type":"v1","addr":"172.21.15.203:6813","nonce":3688099014}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6819","nonce":3688099014},{"type":"v1","addr":"172.21.15.203:6821","nonce":3688099014}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6815","nonce":3688099014},{"type":"v1","addr":"172.21.15.203:6817","nonce":3688099014}]},"public_addr":"172.21.15.203:6809/3688099014","cluster_addr":"172.21.15.203:6813/3688099014","heartbeat_back_addr":"172.21.15.203:6821/3688099014","heartbeat_front_addr":"172.21.15.203:6817/3688099014","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T10:28:51.419141+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T10:28:51.366448+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T10:28:51.348276+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T10:28:51.380519+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T10:28:51.105812+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T10:28:51.384496+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T10:28:51.388755+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T10:28:51.366958+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[{"pgid":"3.31","mappings":[{"from":0,"to":7}]}],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-04-25T10:29:54.807 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:54.804+0000 7f833c17d640 1 -- 172.21.15.37:0/1700788318 >> [v2:172.21.15.203:6832/2418205237,v1:172.21.15.203:6833/2418205237] conn(0x7f83200744b0 msgr2=0x7f83200769c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:29:54.807 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:54.804+0000 7f833c17d640 1 --2- 172.21.15.37:0/1700788318 >> [v2:172.21.15.203:6832/2418205237,v1:172.21.15.203:6833/2418205237] conn(0x7f83200744b0 0x7f83200769c0 secure :-1 s=READY pgs=169 cs=0 l=1 rev1=1 crypto rx=0x7f832c007fd0 tx=0x7f832c009040 comp rx=0 tx=0).stop 2024-04-25T10:29:54.807 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:54.804+0000 7f833c17d640 1 -- 172.21.15.37:0/1700788318 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f83340d2860 msgr2=0x7f83341fa080 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:29:54.807 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:54.804+0000 7f833c17d640 1 --2- 172.21.15.37:0/1700788318 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f83340d2860 0x7f83341fa080 secure :-1 s=READY pgs=327 cs=0 l=1 rev1=1 crypto rx=0x7f8324009f40 tx=0x7f8324004a40 comp rx=0 tx=0).stop 2024-04-25T10:29:54.807 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:54.805+0000 7f833c17d640 1 -- 172.21.15.37:0/1700788318 shutdown_connections 2024-04-25T10:29:54.807 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:54.805+0000 7f833c17d640 1 --2- 172.21.15.37:0/1700788318 >> [v2:172.21.15.203:6832/2418205237,v1:172.21.15.203:6833/2418205237] conn(0x7f83200744b0 0x7f83200769c0 unknown :-1 s=CLOSED pgs=169 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:29:54.807 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:54.805+0000 7f833c17d640 1 --2- 172.21.15.37:0/1700788318 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f83340d2860 0x7f83341fa080 unknown :-1 s=CLOSED pgs=327 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:29:54.808 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:54.805+0000 7f833c17d640 1 --2- 172.21.15.37:0/1700788318 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f83340ce560 0x7f83341f9b40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:29:54.808 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:54.805+0000 7f833c17d640 1 --2- 172.21.15.37:0/1700788318 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f83340b2dc0 0x7f8334156260 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:29:54.808 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:54.805+0000 7f833c17d640 1 -- 172.21.15.37:0/1700788318 >> 172.21.15.37:0/1700788318 conn(0x7f83340b5440 msgr2=0x7f83340712b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:29:54.808 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:54.805+0000 7f833c17d640 1 -- 172.21.15.37:0/1700788318 shutdown_connections 2024-04-25T10:29:54.808 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:54.805+0000 7f833c17d640 1 -- 172.21.15.37:0/1700788318 wait complete. 2024-04-25T10:29:54.818 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2024-04-25T10:28:56.202153+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '19', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 8, 'score_stable': 8, 'optimal_score': 0.25, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 2, 'pool_name': 'cephfs_metadata', 'create_time': '2024-04-25T10:29:06.507244+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '22', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_autoscale_bias': 4, 'pg_num_min': 16, 'recovery_priority': 5}, 'application_metadata': {'cephfs': {'metadata': 'cephfs'}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 2.25, 'score_stable': 2.25, 'optimal_score': 1, 'raw_score_acting': 2.25, 'raw_score_stable': 2.25, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 3, 'pool_name': 'cephfs_data', 'create_time': '2024-04-25T10:29:07.347844+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 64, 'pg_placement_num': 64, 'pg_placement_num_target': 64, 'pg_num_target': 64, 'pg_num_pending': 64, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '23', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.8999999761581421}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 1.5, 'score_stable': 1.5, 'optimal_score': 1, 'raw_score_acting': 1.5, 'raw_score_stable': 1.5, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}] 2024-04-25T10:29:54.819 DEBUG:teuthology.orchestra.run.smithi037:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2024-04-25T10:29:54.934 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:54.931+0000 7f9dafbeb640 1 Processor -- start 2024-04-25T10:29:54.934 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:54.932+0000 7f9dafbeb640 1 -- start start 2024-04-25T10:29:54.937 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:54.933+0000 7f9dafbeb640 1 --2- >> v2:172.21.15.203:3300/0 conn(0x7f9da80b7600 0x7f9da80b7a10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:29:54.937 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:54.933+0000 7f9dafbeb640 1 -- --> v1:172.21.15.37:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f9da80b7f50 con 0x7f9da80b4bb0 2024-04-25T10:29:54.937 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:54.933+0000 7f9dafbeb640 1 -- --> v1:172.21.15.203:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f9da80d5080 con 0x7f9da80d4560 2024-04-25T10:29:54.938 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:54.933+0000 7f9dafbeb640 1 -- --> v2:172.21.15.203:3300/0 -- mon_getmap magic: 0 -- 0x7f9da8161190 con 0x7f9da80b7600 2024-04-25T10:29:54.938 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:54.933+0000 7f9dad987640 1 --1- >> v1:172.21.15.37:6789/0 conn(0x7f9da80b4bb0 0x7f9da81646b0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.37:6789/0 says I am v1:172.21.15.37:56842/0 (socket says 172.21.15.37:56842) 2024-04-25T10:29:54.938 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:54.933+0000 7f9dad987640 1 -- 172.21.15.37:0/760065657 learned_addr learned my addr 172.21.15.37:0/760065657 (peer_addr_for_me v1:172.21.15.37:0/0) 2024-04-25T10:29:54.938 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:54.933+0000 7f9dac985640 1 -- 172.21.15.37:0/760065657 <== mon.0 v1:172.21.15.37:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 1734966032 0 0) 0x7f9da0001110 con 0x7f9da80b4bb0 2024-04-25T10:29:54.938 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:54.933+0000 7f9dac985640 1 -- 172.21.15.37:0/760065657 --> v1:172.21.15.37:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f9d90003520 con 0x7f9da80b4bb0 2024-04-25T10:29:54.938 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:54.933+0000 7f9dad186640 1 --2- 172.21.15.37:0/760065657 >> v2:172.21.15.203:3300/0 conn(0x7f9da80b7600 0x7f9da80b7a10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:29:54.938 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:54.934+0000 7f9dac985640 1 -- 172.21.15.37:0/760065657 <== mon.0 v1:172.21.15.37:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 1568933995 0 0) 0x7f9d90003520 con 0x7f9da80b4bb0 2024-04-25T10:29:54.938 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:54.934+0000 7f9dac985640 1 -- 172.21.15.37:0/760065657 >> v2:172.21.15.203:3300/0 conn(0x7f9da80b7600 msgr2=0x7f9da80b7a10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:29:54.938 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:54.934+0000 7f9dac985640 1 --2- 172.21.15.37:0/760065657 >> v2:172.21.15.203:3300/0 conn(0x7f9da80b7600 0x7f9da80b7a10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:29:54.938 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:54.934+0000 7f9dac985640 1 -- 172.21.15.37:0/760065657 >> v1:172.21.15.203:6789/0 conn(0x7f9da80d4560 legacy=0x7f9da80d4970 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:29:54.938 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:54.934+0000 7f9dac985640 1 -- 172.21.15.37:0/760065657 --> v1:172.21.15.37:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f9da8161470 con 0x7f9da80b4bb0 2024-04-25T10:29:54.938 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:54.934+0000 7f9dac985640 1 -- 172.21.15.37:0/760065657 <== mon.0 v1:172.21.15.37:6789/0 3 ==== mon_map magic: 0 ==== 413+0+0 (unknown 1577794600 0 0) 0x7f9da0001840 con 0x7f9da80b4bb0 2024-04-25T10:29:54.938 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:54.934+0000 7f9dac985640 1 -- 172.21.15.37:0/760065657 >> v1:172.21.15.37:6789/0 conn(0x7f9da80b4bb0 legacy=0x7f9da81646b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:29:54.938 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:54.934+0000 7f9dac985640 1 --2- 172.21.15.37:0/760065657 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f9d900039b0 0x7f9d90003e20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:29:54.939 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:54.934+0000 7f9dac985640 1 --2- 172.21.15.37:0/760065657 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f9d90004800 0x7f9d90004c10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:29:54.939 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:54.934+0000 7f9dac985640 1 --2- 172.21.15.37:0/760065657 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f9d900051c0 0x7f9d9000b9c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:29:54.939 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:54.934+0000 7f9dac985640 1 -- 172.21.15.37:0/760065657 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 -- 0x7f9da8161190 con 0x7f9d90004800 2024-04-25T10:29:54.939 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:54.934+0000 7f9dac985640 1 -- 172.21.15.37:0/760065657 --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_getmap magic: 0 -- 0x7f9d9000c160 con 0x7f9d900051c0 2024-04-25T10:29:54.939 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:54.934+0000 7f9dac985640 1 -- 172.21.15.37:0/760065657 --> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] -- mon_getmap magic: 0 -- 0x7f9d9000c360 con 0x7f9d900039b0 2024-04-25T10:29:54.939 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:54.934+0000 7f9dad186640 1 --2- 172.21.15.37:0/760065657 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f9d90004800 0x7f9d90004c10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:29:54.939 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:54.934+0000 7f9dad987640 1 --2- 172.21.15.37:0/760065657 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f9d900039b0 0x7f9d90003e20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:29:54.939 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:54.934+0000 7f9dae188640 1 --2- 172.21.15.37:0/760065657 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f9d900051c0 0x7f9d9000b9c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:29:54.939 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:54.934+0000 7f9dad186640 1 -- 172.21.15.37:0/760065657 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f9d900039b0 msgr2=0x7f9d90003e20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:29:54.939 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:54.934+0000 7f9dad186640 1 --2- 172.21.15.37:0/760065657 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f9d900039b0 0x7f9d90003e20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:29:54.939 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:54.934+0000 7f9dad186640 1 -- 172.21.15.37:0/760065657 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f9d900051c0 msgr2=0x7f9d9000b9c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:29:54.940 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:54.934+0000 7f9dad186640 1 --2- 172.21.15.37:0/760065657 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f9d900051c0 0x7f9d9000b9c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:29:54.940 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:54.934+0000 7f9dae188640 1 -- 172.21.15.37:0/760065657 reap_dead start 2024-04-25T10:29:54.940 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:54.934+0000 7f9dad186640 1 -- 172.21.15.37:0/760065657 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f9d90003520 con 0x7f9d90004800 2024-04-25T10:29:54.940 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:54.935+0000 7f9dad186640 1 --2- 172.21.15.37:0/760065657 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f9d90004800 0x7f9d90004c10 secure :-1 s=READY pgs=329 cs=0 l=1 rev1=1 crypto rx=0x7f9d9c009ac0 tx=0x7f9d90005830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=163814459e95c508 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:29:54.940 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:54.935+0000 7f9dac985640 1 -- 172.21.15.37:0/760065657 <== mon.0 v2:172.21.15.37:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9d9c036070 con 0x7f9d90004800 2024-04-25T10:29:54.940 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:54.935+0000 7f9dac985640 1 -- 172.21.15.37:0/760065657 <== mon.0 v2:172.21.15.37:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f9d9c031400 con 0x7f9d90004800 2024-04-25T10:29:54.940 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:54.935+0000 7f9dac985640 1 -- 172.21.15.37:0/760065657 <== mon.0 v2:172.21.15.37:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9d9c009690 con 0x7f9d90004800 2024-04-25T10:29:54.943 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:54.936+0000 7f9dafbeb640 1 -- 172.21.15.37:0/760065657 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f9d90004800 msgr2=0x7f9d90004c10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:29:54.943 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:54.936+0000 7f9dafbeb640 1 --2- 172.21.15.37:0/760065657 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f9d90004800 0x7f9d90004c10 secure :-1 s=READY pgs=329 cs=0 l=1 rev1=1 crypto rx=0x7f9d9c009ac0 tx=0x7f9d90005830 comp rx=0 tx=0).stop 2024-04-25T10:29:54.943 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:54.936+0000 7f9dafbeb640 1 -- 172.21.15.37:0/760065657 shutdown_connections 2024-04-25T10:29:54.943 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:54.936+0000 7f9dafbeb640 1 --2- 172.21.15.37:0/760065657 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f9d90004800 0x7f9d90004c10 unknown :-1 s=CLOSED pgs=329 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:29:54.943 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:54.936+0000 7f9dafbeb640 1 -- 172.21.15.37:0/760065657 >> 172.21.15.37:0/760065657 conn(0x7f9da80d27e0 msgr2=0x7f9da80d2bf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:29:54.943 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:54.936+0000 7f9dafbeb640 1 -- 172.21.15.37:0/760065657 shutdown_connections 2024-04-25T10:29:54.943 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:54.936+0000 7f9dafbeb640 1 -- 172.21.15.37:0/760065657 wait complete. 2024-04-25T10:29:54.943 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:54.938+0000 7f9dafbeb640 1 Processor -- start 2024-04-25T10:29:54.943 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:54.938+0000 7f9dafbeb640 1 -- start start 2024-04-25T10:29:54.943 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:54.939+0000 7f9dafbeb640 1 --2- >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f9d90004800 0x7f9da80ae610 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:29:54.943 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:54.939+0000 7f9dafbeb640 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f9da80cba20 0x7f9da80cbe30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:29:54.944 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:54.939+0000 7f9dafbeb640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f9da80ccc30 0x7f9da80cd040 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:29:54.944 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:54.939+0000 7f9dafbeb640 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 -- 0x7f9da80b4fc0 con 0x7f9da80ccc30 2024-04-25T10:29:54.944 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:54.939+0000 7f9dafbeb640 1 -- --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_getmap magic: 0 -- 0x7f9da8164190 con 0x7f9da80cba20 2024-04-25T10:29:54.944 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:54.939+0000 7f9dafbeb640 1 -- --> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] -- mon_getmap magic: 0 -- 0x7f9da80d09e0 con 0x7f9d90004800 2024-04-25T10:29:54.944 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:54.939+0000 7f9dae188640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f9da80ccc30 0x7f9da80cd040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:29:54.944 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:54.939+0000 7f9dae188640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f9da80ccc30 0x7f9da80cd040 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.37:3300/0 says I am v2:172.21.15.37:36446/0 (socket says 172.21.15.37:36446) 2024-04-25T10:29:54.944 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:54.939+0000 7f9dae188640 1 -- 172.21.15.37:0/146866626 learned_addr learned my addr 172.21.15.37:0/146866626 (peer_addr_for_me v2:172.21.15.37:0/0) 2024-04-25T10:29:54.944 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:54.939+0000 7f9dad186640 1 --2- 172.21.15.37:0/146866626 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f9da80cba20 0x7f9da80cbe30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:29:54.944 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:54.939+0000 7f9dad987640 1 --2- 172.21.15.37:0/146866626 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f9d90004800 0x7f9da80ae610 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:29:54.944 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:54.940+0000 7f9dae188640 1 -- 172.21.15.37:0/146866626 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f9d90004800 msgr2=0x7f9da80ae610 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:29:54.944 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:54.940+0000 7f9dae188640 1 --2- 172.21.15.37:0/146866626 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f9d90004800 0x7f9da80ae610 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:29:54.945 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:54.940+0000 7f9dae188640 1 -- 172.21.15.37:0/146866626 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f9da80cba20 msgr2=0x7f9da80cbe30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:29:54.945 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:54.940+0000 7f9dae188640 1 --2- 172.21.15.37:0/146866626 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f9da80cba20 0x7f9da80cbe30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:29:54.945 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:54.940+0000 7f9dae188640 1 -- 172.21.15.37:0/146866626 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f9da80cd960 con 0x7f9da80ccc30 2024-04-25T10:29:54.945 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:54.940+0000 7f9dad186640 1 --2- 172.21.15.37:0/146866626 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f9da80cba20 0x7f9da80cbe30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T10:29:54.945 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:54.940+0000 7f9dae188640 1 --2- 172.21.15.37:0/146866626 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f9da80ccc30 0x7f9da80cd040 secure :-1 s=READY pgs=330 cs=0 l=1 rev1=1 crypto rx=0x7f9da808f020 tx=0x7f9d9800e980 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:29:54.945 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:54.940+0000 7f9da67fc640 1 -- 172.21.15.37:0/146866626 <== mon.0 v2:172.21.15.37:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9d98019070 con 0x7f9da80ccc30 2024-04-25T10:29:54.945 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:54.940+0000 7f9da67fc640 1 -- 172.21.15.37:0/146866626 <== mon.0 v2:172.21.15.37:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f9d98006d60 con 0x7f9da80ccc30 2024-04-25T10:29:54.945 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:54.940+0000 7f9dafbeb640 1 -- 172.21.15.37:0/146866626 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f9da80cdc40 con 0x7f9da80ccc30 2024-04-25T10:29:54.945 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:54.940+0000 7f9da67fc640 1 -- 172.21.15.37:0/146866626 <== mon.0 v2:172.21.15.37:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9d98003b90 con 0x7f9da80ccc30 2024-04-25T10:29:54.945 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:54.940+0000 7f9dafbeb640 1 -- 172.21.15.37:0/146866626 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f9da80ce200 con 0x7f9da80ccc30 2024-04-25T10:29:54.946 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:54.941+0000 7f9dafbeb640 1 -- 172.21.15.37:0/146866626 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f9da80cd6d0 con 0x7f9da80ccc30 2024-04-25T10:29:54.946 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:54.942+0000 7f9da67fc640 1 -- 172.21.15.37:0/146866626 <== mon.0 v2:172.21.15.37:3300/0 4 ==== mgrmap(e 5) ==== 96782+0+0 (secure 0 0 0) 0x7f9d9801c070 con 0x7f9da80ccc30 2024-04-25T10:29:54.946 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:54.942+0000 7f9da67fc640 1 --2- 172.21.15.37:0/146866626 >> [v2:172.21.15.203:6832/2418205237,v1:172.21.15.203:6833/2418205237] conn(0x7f9d940744b0 0x7f9d940769c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:29:54.946 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:54.942+0000 7f9da67fc640 1 -- 172.21.15.37:0/146866626 <== mon.0 v2:172.21.15.37:3300/0 5 ==== osd_map(25..25 src has 1..25) ==== 5738+0+0 (secure 0 0 0) 0x7f9d980947c0 con 0x7f9da80ccc30 2024-04-25T10:29:54.946 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:54.942+0000 7f9dad987640 1 --2- 172.21.15.37:0/146866626 >> [v2:172.21.15.203:6832/2418205237,v1:172.21.15.203:6833/2418205237] conn(0x7f9d940744b0 0x7f9d940769c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:29:54.947 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:54.943+0000 7f9dad987640 1 --2- 172.21.15.37:0/146866626 >> [v2:172.21.15.203:6832/2418205237,v1:172.21.15.203:6833/2418205237] conn(0x7f9d940744b0 0x7f9d940769c0 secure :-1 s=READY pgs=170 cs=0 l=1 rev1=1 crypto rx=0x7f9da80d6c00 tx=0x7f9da0009040 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:29:54.949 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:54.947+0000 7f9da67fc640 1 -- 172.21.15.37:0/146866626 <== mon.0 v2:172.21.15.37:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187246 (secure 0 0 0) 0x7f9d9805ef30 con 0x7f9da80ccc30 2024-04-25T10:29:55.133 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:55.131+0000 7f9dafbeb640 1 -- 172.21.15.37:0/146866626 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) -- 0x7f9da80cff90 con 0x7f9da80ccc30 2024-04-25T10:29:55.134 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:55.132+0000 7f9da67fc640 1 -- 172.21.15.37:0/146866626 <== mon.0 v2:172.21.15.37:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v25) ==== 93+0+10 (secure 0 0 0) 0x7f9d9805e830 con 0x7f9da80ccc30 2024-04-25T10:29:55.134 INFO:teuthology.orchestra.run.smithi037.stdout:pg_num: 1 2024-04-25T10:29:55.136 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:55.134+0000 7f9dafbeb640 1 -- 172.21.15.37:0/146866626 >> [v2:172.21.15.203:6832/2418205237,v1:172.21.15.203:6833/2418205237] conn(0x7f9d940744b0 msgr2=0x7f9d940769c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:29:55.136 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:55.134+0000 7f9dafbeb640 1 --2- 172.21.15.37:0/146866626 >> [v2:172.21.15.203:6832/2418205237,v1:172.21.15.203:6833/2418205237] conn(0x7f9d940744b0 0x7f9d940769c0 secure :-1 s=READY pgs=170 cs=0 l=1 rev1=1 crypto rx=0x7f9da80d6c00 tx=0x7f9da0009040 comp rx=0 tx=0).stop 2024-04-25T10:29:55.136 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:55.134+0000 7f9dafbeb640 1 -- 172.21.15.37:0/146866626 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f9da80ccc30 msgr2=0x7f9da80cd040 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:29:55.136 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:55.134+0000 7f9dafbeb640 1 --2- 172.21.15.37:0/146866626 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f9da80ccc30 0x7f9da80cd040 secure :-1 s=READY pgs=330 cs=0 l=1 rev1=1 crypto rx=0x7f9da808f020 tx=0x7f9d9800e980 comp rx=0 tx=0).stop 2024-04-25T10:29:55.136 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:55.134+0000 7f9dafbeb640 1 -- 172.21.15.37:0/146866626 shutdown_connections 2024-04-25T10:29:55.137 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:55.134+0000 7f9dafbeb640 1 --2- 172.21.15.37:0/146866626 >> [v2:172.21.15.203:6832/2418205237,v1:172.21.15.203:6833/2418205237] conn(0x7f9d940744b0 0x7f9d940769c0 unknown :-1 s=CLOSED pgs=170 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:29:55.137 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:55.134+0000 7f9dafbeb640 1 --2- 172.21.15.37:0/146866626 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f9da80ccc30 0x7f9da80cd040 unknown :-1 s=CLOSED pgs=330 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:29:55.137 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:55.134+0000 7f9dafbeb640 1 --2- 172.21.15.37:0/146866626 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f9da80cba20 0x7f9da80cbe30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:29:55.137 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:55.134+0000 7f9dafbeb640 1 --2- 172.21.15.37:0/146866626 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f9d90004800 0x7f9da80ae610 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:29:55.137 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:55.134+0000 7f9dafbeb640 1 -- 172.21.15.37:0/146866626 >> 172.21.15.37:0/146866626 conn(0x7f9da80d27e0 msgr2=0x7f9da8162850 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:29:55.137 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:55.134+0000 7f9dafbeb640 1 -- 172.21.15.37:0/146866626 shutdown_connections 2024-04-25T10:29:55.137 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:55.134+0000 7f9dafbeb640 1 -- 172.21.15.37:0/146866626 wait complete. 2024-04-25T10:29:55.147 DEBUG:teuthology.orchestra.run.smithi037:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_metadata pg_num 2024-04-25T10:29:55.261 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:55.259+0000 7fb34ba92640 1 Processor -- start 2024-04-25T10:29:55.261 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:55.259+0000 7fb34ba92640 1 -- start start 2024-04-25T10:29:55.264 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:55.261+0000 7fb34ba92640 1 --2- >> v2:172.21.15.203:3300/0 conn(0x7fb3440b4c90 0x7fb3440d29e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:29:55.264 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:55.261+0000 7fb34ba92640 1 -- --> v1:172.21.15.37:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fb34409ce70 con 0x7fb3440d1ec0 2024-04-25T10:29:55.264 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:55.261+0000 7fb34ba92640 1 -- --> v1:172.21.15.203:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fb34409d150 con 0x7fb34406e260 2024-04-25T10:29:55.264 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:55.261+0000 7fb34ba92640 1 -- --> v2:172.21.15.203:3300/0 -- mon_getmap magic: 0 -- 0x7fb3440d3370 con 0x7fb3440b4c90 2024-04-25T10:29:55.264 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:55.261+0000 7fb34902d640 1 --1- >> v1:172.21.15.37:6789/0 conn(0x7fb3440d1ec0 0x7fb3440d22d0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.37:6789/0 says I am v1:172.21.15.37:56848/0 (socket says 172.21.15.37:56848) 2024-04-25T10:29:55.264 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:55.261+0000 7fb34902d640 1 -- 172.21.15.37:0/1027705017 learned_addr learned my addr 172.21.15.37:0/1027705017 (peer_addr_for_me v1:172.21.15.37:0/0) 2024-04-25T10:29:55.264 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:55.261+0000 7fb34a02f640 1 --2- 172.21.15.37:0/1027705017 >> v2:172.21.15.203:3300/0 conn(0x7fb3440b4c90 0x7fb3440d29e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:29:55.264 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:55.261+0000 7fb34882c640 1 -- 172.21.15.37:0/1027705017 <== mon.0 v1:172.21.15.37:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 1870874700 0 0) 0x7fb34409ce70 con 0x7fb3440d1ec0 2024-04-25T10:29:55.264 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:55.261+0000 7fb34882c640 1 -- 172.21.15.37:0/1027705017 --> v1:172.21.15.37:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7fb330003520 con 0x7fb3440d1ec0 2024-04-25T10:29:55.264 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:55.261+0000 7fb34882c640 1 -- 172.21.15.37:0/1027705017 <== mon.0 v1:172.21.15.37:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 2913096923 0 0) 0x7fb330003520 con 0x7fb3440d1ec0 2024-04-25T10:29:55.264 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:55.261+0000 7fb34a02f640 1 -- 172.21.15.37:0/1027705017 >> v1:172.21.15.203:6789/0 conn(0x7fb34406e260 legacy=0x7fb34406e670 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:29:55.264 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:55.261+0000 7fb34a02f640 1 -- 172.21.15.37:0/1027705017 >> v1:172.21.15.37:6789/0 conn(0x7fb3440d1ec0 legacy=0x7fb3440d22d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:29:55.264 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:55.261+0000 7fb34a02f640 1 -- 172.21.15.37:0/1027705017 --> v2:172.21.15.203:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fb34406ed80 con 0x7fb3440b4c90 2024-04-25T10:29:55.266 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:55.262+0000 7fb34a02f640 1 --2- 172.21.15.37:0/1027705017 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fb3440b4c90 0x7fb3440d29e0 secure :-1 s=READY pgs=146 cs=0 l=1 rev1=1 crypto rx=0x7fb33c009d90 tx=0x7fb33c02a5f0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=7748d79cc1656cfc server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:29:55.266 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:55.262+0000 7fb34882c640 1 -- 172.21.15.37:0/1027705017 <== mon.1 v2:172.21.15.203:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb334001670 con 0x7fb3440b4c90 2024-04-25T10:29:55.266 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:55.262+0000 7fb34882c640 1 -- 172.21.15.37:0/1027705017 <== mon.1 v2:172.21.15.203:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fb33c007540 con 0x7fb3440b4c90 2024-04-25T10:29:55.266 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:55.262+0000 7fb34882c640 1 -- 172.21.15.37:0/1027705017 <== mon.1 v2:172.21.15.203:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb33c0078f0 con 0x7fb3440b4c90 2024-04-25T10:29:55.266 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:55.263+0000 7fb34ba92640 1 -- 172.21.15.37:0/1027705017 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fb3440b4c90 msgr2=0x7fb3440d29e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:29:55.266 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:55.263+0000 7fb34ba92640 1 --2- 172.21.15.37:0/1027705017 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fb3440b4c90 0x7fb3440d29e0 secure :-1 s=READY pgs=146 cs=0 l=1 rev1=1 crypto rx=0x7fb33c009d90 tx=0x7fb33c02a5f0 comp rx=0 tx=0).stop 2024-04-25T10:29:55.266 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:55.263+0000 7fb34ba92640 1 -- 172.21.15.37:0/1027705017 shutdown_connections 2024-04-25T10:29:55.266 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:55.263+0000 7fb34ba92640 1 --2- 172.21.15.37:0/1027705017 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fb3440b4c90 0x7fb3440d29e0 unknown :-1 s=CLOSED pgs=146 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:29:55.266 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:55.263+0000 7fb34ba92640 1 -- 172.21.15.37:0/1027705017 >> 172.21.15.37:0/1027705017 conn(0x7fb3440d0390 msgr2=0x7fb3440d07a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:29:55.266 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:55.263+0000 7fb34ba92640 1 -- 172.21.15.37:0/1027705017 shutdown_connections 2024-04-25T10:29:55.266 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:55.263+0000 7fb34ba92640 1 -- 172.21.15.37:0/1027705017 wait complete. 2024-04-25T10:29:55.267 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:55.265+0000 7fb34ba92640 1 Processor -- start 2024-04-25T10:29:55.267 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:55.265+0000 7fb34ba92640 1 -- start start 2024-04-25T10:29:55.270 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:55.266+0000 7fb34ba92640 1 --2- >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7fb34406e260 0x7fb3440a1100 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:29:55.270 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:55.266+0000 7fb34ba92640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fb3440b4c90 0x7fb3441d5cb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:29:55.270 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:55.266+0000 7fb34ba92640 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fb3440d1ec0 0x7fb3441d8550 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:29:55.270 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:55.266+0000 7fb34ba92640 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 -- 0x7fb34406d830 con 0x7fb3440b4c90 2024-04-25T10:29:55.270 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:55.266+0000 7fb34ba92640 1 -- --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_getmap magic: 0 -- 0x7fb3440d3090 con 0x7fb3440d1ec0 2024-04-25T10:29:55.270 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:55.266+0000 7fb34ba92640 1 -- --> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] -- mon_getmap magic: 0 -- 0x7fb34406db30 con 0x7fb34406e260 2024-04-25T10:29:55.270 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:55.266+0000 7fb34902d640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fb3440b4c90 0x7fb3441d5cb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:29:55.270 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:55.266+0000 7fb34902d640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fb3440b4c90 0x7fb3441d5cb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.37:3300/0 says I am v2:172.21.15.37:36450/0 (socket says 172.21.15.37:36450) 2024-04-25T10:29:55.270 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:55.266+0000 7fb34902d640 1 -- 172.21.15.37:0/3680165266 learned_addr learned my addr 172.21.15.37:0/3680165266 (peer_addr_for_me v2:172.21.15.37:0/0) 2024-04-25T10:29:55.270 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:55.266+0000 7fb34a02f640 1 --2- 172.21.15.37:0/3680165266 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fb3440d1ec0 0x7fb3441d8550 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:29:55.270 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:55.267+0000 7fb34982e640 1 --2- 172.21.15.37:0/3680165266 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7fb34406e260 0x7fb3440a1100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:29:55.271 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:55.267+0000 7fb34902d640 1 -- 172.21.15.37:0/3680165266 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7fb34406e260 msgr2=0x7fb3440a1100 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:29:55.271 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:55.267+0000 7fb34902d640 1 --2- 172.21.15.37:0/3680165266 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7fb34406e260 0x7fb3440a1100 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:29:55.271 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:55.267+0000 7fb34902d640 1 -- 172.21.15.37:0/3680165266 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fb3440d1ec0 msgr2=0x7fb3441d8550 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:29:55.271 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:55.267+0000 7fb34902d640 1 --2- 172.21.15.37:0/3680165266 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fb3440d1ec0 0x7fb3441d8550 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:29:55.271 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:55.267+0000 7fb34902d640 1 -- 172.21.15.37:0/3680165266 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fb3441d61f0 con 0x7fb3440b4c90 2024-04-25T10:29:55.271 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:55.267+0000 7fb34982e640 1 --2- 172.21.15.37:0/3680165266 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7fb34406e260 0x7fb3440a1100 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T10:29:55.271 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:55.267+0000 7fb34902d640 1 --2- 172.21.15.37:0/3680165266 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fb3440b4c90 0x7fb3441d5cb0 secure :-1 s=READY pgs=332 cs=0 l=1 rev1=1 crypto rx=0x7fb33400c580 tx=0x7fb33400ca40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:29:55.271 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:55.267+0000 7fb33a7fc640 1 -- 172.21.15.37:0/3680165266 <== mon.0 v2:172.21.15.37:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb334017070 con 0x7fb3440b4c90 2024-04-25T10:29:55.271 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:55.267+0000 7fb33a7fc640 1 -- 172.21.15.37:0/3680165266 <== mon.0 v2:172.21.15.37:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fb334005ce0 con 0x7fb3440b4c90 2024-04-25T10:29:55.271 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:55.267+0000 7fb34ba92640 1 -- 172.21.15.37:0/3680165266 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fb3441e8e30 con 0x7fb3440b4c90 2024-04-25T10:29:55.271 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:55.267+0000 7fb33a7fc640 1 -- 172.21.15.37:0/3680165266 <== mon.0 v2:172.21.15.37:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb334007330 con 0x7fb3440b4c90 2024-04-25T10:29:55.271 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:55.267+0000 7fb34ba92640 1 -- 172.21.15.37:0/3680165266 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fb34409ee70 con 0x7fb3440b4c90 2024-04-25T10:29:55.272 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:55.268+0000 7fb34ba92640 1 -- 172.21.15.37:0/3680165266 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fb34406a140 con 0x7fb3440b4c90 2024-04-25T10:29:55.272 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:55.269+0000 7fb33a7fc640 1 -- 172.21.15.37:0/3680165266 <== mon.0 v2:172.21.15.37:3300/0 4 ==== mgrmap(e 5) ==== 96782+0+0 (secure 0 0 0) 0x7fb33401a070 con 0x7fb3440b4c90 2024-04-25T10:29:55.272 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:55.269+0000 7fb33a7fc640 1 --2- 172.21.15.37:0/3680165266 >> [v2:172.21.15.203:6832/2418205237,v1:172.21.15.203:6833/2418205237] conn(0x7fb3280744b0 0x7fb3280769c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:29:55.273 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:55.269+0000 7fb33a7fc640 1 -- 172.21.15.37:0/3680165266 <== mon.0 v2:172.21.15.37:3300/0 5 ==== osd_map(25..25 src has 1..25) ==== 5738+0+0 (secure 0 0 0) 0x7fb334094640 con 0x7fb3440b4c90 2024-04-25T10:29:55.273 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:55.269+0000 7fb34982e640 1 --2- 172.21.15.37:0/3680165266 >> [v2:172.21.15.203:6832/2418205237,v1:172.21.15.203:6833/2418205237] conn(0x7fb3280744b0 0x7fb3280769c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:29:55.273 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:55.270+0000 7fb34982e640 1 --2- 172.21.15.37:0/3680165266 >> [v2:172.21.15.203:6832/2418205237,v1:172.21.15.203:6833/2418205237] conn(0x7fb3280744b0 0x7fb3280769c0 secure :-1 s=READY pgs=171 cs=0 l=1 rev1=1 crypto rx=0x7fb340007fd0 tx=0x7fb340009040 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:29:55.276 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:55.274+0000 7fb33a7fc640 1 -- 172.21.15.37:0/3680165266 <== mon.0 v2:172.21.15.37:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187246 (secure 0 0 0) 0x7fb3441e8e30 con 0x7fb3440b4c90 2024-04-25T10:29:55.458 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:55.456+0000 7fb34ba92640 1 -- 172.21.15.37:0/3680165266 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"} v 0) -- 0x7fb34409e920 con 0x7fb3440b4c90 2024-04-25T10:29:55.459 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:55.457+0000 7fb33a7fc640 1 -- 172.21.15.37:0/3680165266 <== mon.0 v2:172.21.15.37:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]=0 v25) ==== 104+0+11 (secure 0 0 0) 0x7fb33405e800 con 0x7fb3440b4c90 2024-04-25T10:29:55.459 INFO:teuthology.orchestra.run.smithi037.stdout:pg_num: 32 2024-04-25T10:29:55.461 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:55.458+0000 7fb34ba92640 1 -- 172.21.15.37:0/3680165266 >> [v2:172.21.15.203:6832/2418205237,v1:172.21.15.203:6833/2418205237] conn(0x7fb3280744b0 msgr2=0x7fb3280769c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:29:55.461 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:55.458+0000 7fb34ba92640 1 --2- 172.21.15.37:0/3680165266 >> [v2:172.21.15.203:6832/2418205237,v1:172.21.15.203:6833/2418205237] conn(0x7fb3280744b0 0x7fb3280769c0 secure :-1 s=READY pgs=171 cs=0 l=1 rev1=1 crypto rx=0x7fb340007fd0 tx=0x7fb340009040 comp rx=0 tx=0).stop 2024-04-25T10:29:55.461 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:55.458+0000 7fb34ba92640 1 -- 172.21.15.37:0/3680165266 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fb3440b4c90 msgr2=0x7fb3441d5cb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:29:55.461 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:55.458+0000 7fb34ba92640 1 --2- 172.21.15.37:0/3680165266 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fb3440b4c90 0x7fb3441d5cb0 secure :-1 s=READY pgs=332 cs=0 l=1 rev1=1 crypto rx=0x7fb33400c580 tx=0x7fb33400ca40 comp rx=0 tx=0).stop 2024-04-25T10:29:55.461 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:55.458+0000 7fb34ba92640 1 -- 172.21.15.37:0/3680165266 shutdown_connections 2024-04-25T10:29:55.461 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:55.458+0000 7fb34ba92640 1 --2- 172.21.15.37:0/3680165266 >> [v2:172.21.15.203:6832/2418205237,v1:172.21.15.203:6833/2418205237] conn(0x7fb3280744b0 0x7fb3280769c0 unknown :-1 s=CLOSED pgs=171 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:29:55.461 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:55.458+0000 7fb34ba92640 1 --2- 172.21.15.37:0/3680165266 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fb3440d1ec0 0x7fb3441d8550 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:29:55.461 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:55.458+0000 7fb34ba92640 1 --2- 172.21.15.37:0/3680165266 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fb3440b4c90 0x7fb3441d5cb0 unknown :-1 s=CLOSED pgs=332 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:29:55.462 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:55.458+0000 7fb34ba92640 1 --2- 172.21.15.37:0/3680165266 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7fb34406e260 0x7fb3440a1100 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:29:55.462 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:55.459+0000 7fb34ba92640 1 -- 172.21.15.37:0/3680165266 >> 172.21.15.37:0/3680165266 conn(0x7fb3440d0390 msgr2=0x7fb3440ce320 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:29:55.462 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:55.459+0000 7fb34ba92640 1 -- 172.21.15.37:0/3680165266 shutdown_connections 2024-04-25T10:29:55.462 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:55.459+0000 7fb34ba92640 1 -- 172.21.15.37:0/3680165266 wait complete. 2024-04-25T10:29:55.472 DEBUG:teuthology.orchestra.run.smithi037:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data pg_num 2024-04-25T10:29:55.586 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:55.583+0000 7fb7d0f72640 1 Processor -- start 2024-04-25T10:29:55.586 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:55.583+0000 7fb7d0f72640 1 -- start start 2024-04-25T10:29:55.588 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:55.585+0000 7fb7d0f72640 1 --2- >> v2:172.21.15.203:3300/0 conn(0x7fb7cc06e260 0x7fb7cc06e670 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:29:55.588 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:55.585+0000 7fb7d0f72640 1 --2- >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7fb7cc164ce0 0x7fb7cc0b3c80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:29:55.588 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:55.585+0000 7fb7d0f72640 1 -- --> v1:172.21.15.203:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fb7cc09ce70 con 0x7fb7cc06ebb0 2024-04-25T10:29:55.588 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:55.585+0000 7fb7d0f72640 1 -- --> v2:172.21.15.203:3300/0 -- mon_getmap magic: 0 -- 0x7fb7cc0d3370 con 0x7fb7cc06e260 2024-04-25T10:29:55.588 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:55.585+0000 7fb7d0f72640 1 -- --> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] -- mon_getmap magic: 0 -- 0x7fb7cc0d3090 con 0x7fb7cc164ce0 2024-04-25T10:29:55.588 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:55.585+0000 7fb7ca59c640 1 --2- >> v2:172.21.15.203:3300/0 conn(0x7fb7cc06e260 0x7fb7cc06e670 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:29:55.588 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:55.585+0000 7fb7cad9d640 1 --2- >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7fb7cc164ce0 0x7fb7cc0b3c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:29:55.588 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:55.585+0000 7fb7ca59c640 1 --2- >> v2:172.21.15.203:3300/0 conn(0x7fb7cc06e260 0x7fb7cc06e670 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.203:3300/0 says I am v2:172.21.15.37:33596/0 (socket says 172.21.15.37:33596) 2024-04-25T10:29:55.588 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:55.585+0000 7fb7ca59c640 1 -- 172.21.15.37:0/2875658090 learned_addr learned my addr 172.21.15.37:0/2875658090 (peer_addr_for_me v2:172.21.15.37:0/0) 2024-04-25T10:29:55.589 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:55.586+0000 7fb7cad9d640 1 -- 172.21.15.37:0/2875658090 >> v2:172.21.15.203:3300/0 conn(0x7fb7cc06e260 msgr2=0x7fb7cc06e670 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:29:55.589 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:55.586+0000 7fb7cad9d640 1 --2- 172.21.15.37:0/2875658090 >> v2:172.21.15.203:3300/0 conn(0x7fb7cc06e260 0x7fb7cc06e670 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:29:55.589 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:55.586+0000 7fb7cad9d640 1 -- 172.21.15.37:0/2875658090 >> v1:172.21.15.203:6789/0 conn(0x7fb7cc06ebb0 legacy=0x7fb7cc1645d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:29:55.589 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:55.586+0000 7fb7cad9d640 1 -- 172.21.15.37:0/2875658090 --> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fb7cc09d150 con 0x7fb7cc164ce0 2024-04-25T10:29:55.589 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:55.586+0000 7fb7c959a640 1 -- 172.21.15.37:0/2875658090 <== mon.1 v1:172.21.15.203:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 3042354027 0 0) 0x7fb7cc09ce70 con 0x7fb7cc06ebb0 2024-04-25T10:29:55.589 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:55.586+0000 7fb7ca59c640 1 --2- 172.21.15.37:0/2875658090 >> v2:172.21.15.203:3300/0 conn(0x7fb7cc06e260 0x7fb7cc06e670 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-25T10:29:55.590 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:55.587+0000 7fb7cad9d640 1 --2- 172.21.15.37:0/2875658090 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7fb7cc164ce0 0x7fb7cc0b3c80 secure :-1 s=READY pgs=47 cs=0 l=1 rev1=1 crypto rx=0x7fb7bc0097a0 tx=0x7fb7bc02a100 comp rx=0 tx=0).ready entity=mon.2 client_cookie=c27fda57a5b33bca server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:29:55.590 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:55.587+0000 7fb7c959a640 1 -- 172.21.15.37:0/2875658090 <== mon.2 v2:172.21.15.203:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb7bc037070 con 0x7fb7cc164ce0 2024-04-25T10:29:55.590 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:55.587+0000 7fb7c959a640 1 -- 172.21.15.37:0/2875658090 <== mon.2 v2:172.21.15.203:3301/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fb7bc02ae50 con 0x7fb7cc164ce0 2024-04-25T10:29:55.590 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:55.587+0000 7fb7c959a640 1 -- 172.21.15.37:0/2875658090 <== mon.2 v2:172.21.15.203:3301/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb7bc032660 con 0x7fb7cc164ce0 2024-04-25T10:29:55.590 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:55.587+0000 7fb7d0f72640 1 -- 172.21.15.37:0/2875658090 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7fb7cc164ce0 msgr2=0x7fb7cc0b3c80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:29:55.591 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:55.587+0000 7fb7d0f72640 1 --2- 172.21.15.37:0/2875658090 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7fb7cc164ce0 0x7fb7cc0b3c80 secure :-1 s=READY pgs=47 cs=0 l=1 rev1=1 crypto rx=0x7fb7bc0097a0 tx=0x7fb7bc02a100 comp rx=0 tx=0).stop 2024-04-25T10:29:55.591 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:55.588+0000 7fb7d0f72640 1 -- 172.21.15.37:0/2875658090 shutdown_connections 2024-04-25T10:29:55.591 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:55.588+0000 7fb7d0f72640 1 --2- 172.21.15.37:0/2875658090 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7fb7cc164ce0 0x7fb7cc0b3c80 unknown :-1 s=CLOSED pgs=47 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:29:55.591 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:55.588+0000 7fb7d0f72640 1 --2- 172.21.15.37:0/2875658090 >> v2:172.21.15.203:3300/0 conn(0x7fb7cc06e260 0x7fb7cc06e670 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:29:55.591 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:55.588+0000 7fb7d0f72640 1 -- 172.21.15.37:0/2875658090 >> 172.21.15.37:0/2875658090 conn(0x7fb7cc0d0390 msgr2=0x7fb7cc0d07a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:29:55.591 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:55.588+0000 7fb7d0f72640 1 -- 172.21.15.37:0/2875658090 shutdown_connections 2024-04-25T10:29:55.591 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:55.588+0000 7fb7d0f72640 1 -- 172.21.15.37:0/2875658090 wait complete. 2024-04-25T10:29:55.592 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:55.590+0000 7fb7d0f72640 1 Processor -- start 2024-04-25T10:29:55.592 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:55.590+0000 7fb7d0f72640 1 -- start start 2024-04-25T10:29:55.594 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:55.591+0000 7fb7d0f72640 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fb7cc06e260 0x7fb7cc156e30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:29:55.594 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:55.591+0000 7fb7d0f72640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fb7cc06ebb0 0x7fb7cc0cef90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:29:55.595 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:55.591+0000 7fb7d0f72640 1 --2- >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7fb7cc164ce0 0x7fb7cc09ee30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:29:55.595 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:55.591+0000 7fb7d0f72640 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 -- 0x7fb7cc06db30 con 0x7fb7cc06ebb0 2024-04-25T10:29:55.595 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:55.591+0000 7fb7d0f72640 1 -- --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_getmap magic: 0 -- 0x7fb7cc06d830 con 0x7fb7cc06e260 2024-04-25T10:29:55.595 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:55.591+0000 7fb7d0f72640 1 -- --> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] -- mon_getmap magic: 0 -- 0x7fb7cc06dcb0 con 0x7fb7cc164ce0 2024-04-25T10:29:55.595 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:55.591+0000 7fb7c9d9b640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fb7cc06ebb0 0x7fb7cc0cef90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:29:55.595 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:55.591+0000 7fb7c9d9b640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fb7cc06ebb0 0x7fb7cc0cef90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.37:3300/0 says I am v2:172.21.15.37:36466/0 (socket says 172.21.15.37:36466) 2024-04-25T10:29:55.595 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:55.591+0000 7fb7c9d9b640 1 -- 172.21.15.37:0/1229136492 learned_addr learned my addr 172.21.15.37:0/1229136492 (peer_addr_for_me v2:172.21.15.37:0/0) 2024-04-25T10:29:55.595 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:55.591+0000 7fb7ca59c640 1 --2- 172.21.15.37:0/1229136492 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fb7cc06e260 0x7fb7cc156e30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:29:55.595 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:55.591+0000 7fb7cad9d640 1 --2- 172.21.15.37:0/1229136492 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7fb7cc164ce0 0x7fb7cc09ee30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:29:55.595 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:55.591+0000 7fb7c9d9b640 1 -- 172.21.15.37:0/1229136492 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7fb7cc164ce0 msgr2=0x7fb7cc09ee30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:29:55.595 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:55.591+0000 7fb7c9d9b640 1 --2- 172.21.15.37:0/1229136492 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7fb7cc164ce0 0x7fb7cc09ee30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:29:55.595 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:55.591+0000 7fb7c9d9b640 1 -- 172.21.15.37:0/1229136492 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fb7cc06e260 msgr2=0x7fb7cc156e30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:29:55.595 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:55.591+0000 7fb7c9d9b640 1 --2- 172.21.15.37:0/1229136492 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fb7cc06e260 0x7fb7cc156e30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:29:55.596 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:55.591+0000 7fb7c9d9b640 1 -- 172.21.15.37:0/1229136492 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fb7cc09d7b0 con 0x7fb7cc06ebb0 2024-04-25T10:29:55.596 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:55.592+0000 7fb7c9d9b640 1 --2- 172.21.15.37:0/1229136492 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fb7cc06ebb0 0x7fb7cc0cef90 secure :-1 s=READY pgs=333 cs=0 l=1 rev1=1 crypto rx=0x7fb7b400c3f0 tx=0x7fb7b400c8b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:29:55.596 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:55.592+0000 7fb7c2ffd640 1 -- 172.21.15.37:0/1229136492 <== mon.0 v2:172.21.15.37:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb7b4017070 con 0x7fb7cc06ebb0 2024-04-25T10:29:55.596 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:55.592+0000 7fb7c2ffd640 1 -- 172.21.15.37:0/1229136492 <== mon.0 v2:172.21.15.37:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fb7b40035f0 con 0x7fb7cc06ebb0 2024-04-25T10:29:55.596 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:55.592+0000 7fb7d0f72640 1 -- 172.21.15.37:0/1229136492 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fb7cc09da90 con 0x7fb7cc06ebb0 2024-04-25T10:29:55.596 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:55.592+0000 7fb7c2ffd640 1 -- 172.21.15.37:0/1229136492 <== mon.0 v2:172.21.15.37:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb7b40039e0 con 0x7fb7cc06ebb0 2024-04-25T10:29:55.596 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:55.592+0000 7fb7d0f72640 1 -- 172.21.15.37:0/1229136492 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fb7cc1631b0 con 0x7fb7cc06ebb0 2024-04-25T10:29:55.597 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:55.593+0000 7fb7d0f72640 1 -- 172.21.15.37:0/1229136492 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fb7cc06a140 con 0x7fb7cc06ebb0 2024-04-25T10:29:55.597 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:55.594+0000 7fb7c2ffd640 1 -- 172.21.15.37:0/1229136492 <== mon.0 v2:172.21.15.37:3300/0 4 ==== mgrmap(e 5) ==== 96782+0+0 (secure 0 0 0) 0x7fb7b401a070 con 0x7fb7cc06ebb0 2024-04-25T10:29:55.597 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:55.594+0000 7fb7c2ffd640 1 --2- 172.21.15.37:0/1229136492 >> [v2:172.21.15.203:6832/2418205237,v1:172.21.15.203:6833/2418205237] conn(0x7fb7ac0744b0 0x7fb7ac0769c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:29:55.597 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:55.594+0000 7fb7c2ffd640 1 -- 172.21.15.37:0/1229136492 <== mon.0 v2:172.21.15.37:3300/0 5 ==== osd_map(25..25 src has 1..25) ==== 5738+0+0 (secure 0 0 0) 0x7fb7b4094600 con 0x7fb7cc06ebb0 2024-04-25T10:29:55.597 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:55.594+0000 7fb7ca59c640 1 --2- 172.21.15.37:0/1229136492 >> [v2:172.21.15.203:6832/2418205237,v1:172.21.15.203:6833/2418205237] conn(0x7fb7ac0744b0 0x7fb7ac0769c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:29:55.597 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:55.595+0000 7fb7ca59c640 1 --2- 172.21.15.37:0/1229136492 >> [v2:172.21.15.203:6832/2418205237,v1:172.21.15.203:6833/2418205237] conn(0x7fb7ac0744b0 0x7fb7ac0769c0 secure :-1 s=READY pgs=172 cs=0 l=1 rev1=1 crypto rx=0x7fb7cc0d1490 tx=0x7fb7c400b040 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:29:55.600 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:55.599+0000 7fb7c2ffd640 1 -- 172.21.15.37:0/1229136492 <== mon.0 v2:172.21.15.37:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187246 (secure 0 0 0) 0x7fb7b405ed70 con 0x7fb7cc06ebb0 2024-04-25T10:29:55.783 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:55.781+0000 7fb7d0f72640 1 -- 172.21.15.37:0/1229136492 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"} v 0) -- 0x7fb7cc0b0980 con 0x7fb7cc06ebb0 2024-04-25T10:29:55.784 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:55.782+0000 7fb7c2ffd640 1 -- 172.21.15.37:0/1229136492 <== mon.0 v2:172.21.15.37:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]=0 v25) ==== 100+0+11 (secure 0 0 0) 0x7fb7b405e670 con 0x7fb7cc06ebb0 2024-04-25T10:29:55.784 INFO:teuthology.orchestra.run.smithi037.stdout:pg_num: 64 2024-04-25T10:29:55.787 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:55.784+0000 7fb7d0f72640 1 -- 172.21.15.37:0/1229136492 >> [v2:172.21.15.203:6832/2418205237,v1:172.21.15.203:6833/2418205237] conn(0x7fb7ac0744b0 msgr2=0x7fb7ac0769c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:29:55.787 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:55.784+0000 7fb7d0f72640 1 --2- 172.21.15.37:0/1229136492 >> [v2:172.21.15.203:6832/2418205237,v1:172.21.15.203:6833/2418205237] conn(0x7fb7ac0744b0 0x7fb7ac0769c0 secure :-1 s=READY pgs=172 cs=0 l=1 rev1=1 crypto rx=0x7fb7cc0d1490 tx=0x7fb7c400b040 comp rx=0 tx=0).stop 2024-04-25T10:29:55.787 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:55.784+0000 7fb7d0f72640 1 -- 172.21.15.37:0/1229136492 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fb7cc06ebb0 msgr2=0x7fb7cc0cef90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:29:55.787 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:55.784+0000 7fb7d0f72640 1 --2- 172.21.15.37:0/1229136492 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fb7cc06ebb0 0x7fb7cc0cef90 secure :-1 s=READY pgs=333 cs=0 l=1 rev1=1 crypto rx=0x7fb7b400c3f0 tx=0x7fb7b400c8b0 comp rx=0 tx=0).stop 2024-04-25T10:29:55.787 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:55.784+0000 7fb7d0f72640 1 -- 172.21.15.37:0/1229136492 shutdown_connections 2024-04-25T10:29:55.787 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:55.784+0000 7fb7d0f72640 1 --2- 172.21.15.37:0/1229136492 >> [v2:172.21.15.203:6832/2418205237,v1:172.21.15.203:6833/2418205237] conn(0x7fb7ac0744b0 0x7fb7ac0769c0 unknown :-1 s=CLOSED pgs=172 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:29:55.787 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:55.784+0000 7fb7d0f72640 1 --2- 172.21.15.37:0/1229136492 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7fb7cc164ce0 0x7fb7cc09ee30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:29:55.787 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:55.784+0000 7fb7d0f72640 1 --2- 172.21.15.37:0/1229136492 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fb7cc06ebb0 0x7fb7cc0cef90 unknown :-1 s=CLOSED pgs=333 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:29:55.787 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:55.784+0000 7fb7d0f72640 1 --2- 172.21.15.37:0/1229136492 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fb7cc06e260 0x7fb7cc156e30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:29:55.787 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:55.784+0000 7fb7d0f72640 1 -- 172.21.15.37:0/1229136492 >> 172.21.15.37:0/1229136492 conn(0x7fb7cc0d0390 msgr2=0x7fb7cc0712f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:29:55.787 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:55.784+0000 7fb7d0f72640 1 -- 172.21.15.37:0/1229136492 shutdown_connections 2024-04-25T10:29:55.787 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:55.784+0000 7fb7d0f72640 1 -- 172.21.15.37:0/1229136492 wait complete. 2024-04-25T10:29:55.798 DEBUG:teuthology.orchestra.run.smithi037:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd blocklist ls 2024-04-25T10:29:55.910 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:55.908+0000 7f5dec5d5640 1 Processor -- start 2024-04-25T10:29:55.910 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:55.908+0000 7f5dec5d5640 1 -- start start 2024-04-25T10:29:55.914 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:55.909+0000 7f5dec5d5640 1 --2- >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f5de40d5160 0x7f5de40b4e80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:29:55.914 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:55.909+0000 7f5dec5d5640 1 --2- >> v2:172.21.15.203:3300/0 conn(0x7f5de40d0990 0x7f5de40d2b20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:29:55.914 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:55.909+0000 7f5dec5d5640 1 --2- >> v2:172.21.15.37:3300/0 conn(0x7f5de40d0da0 0x7f5de40d3060 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:29:55.914 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:55.909+0000 7f5dec5d5640 1 -- --> v2:172.21.15.37:3300/0 -- mon_getmap magic: 0 -- 0x7f5de40d1650 con 0x7f5de40d0da0 2024-04-25T10:29:55.914 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:55.909+0000 7f5dec5d5640 1 -- --> v2:172.21.15.203:3300/0 -- mon_getmap magic: 0 -- 0x7f5de40d1370 con 0x7f5de40d0990 2024-04-25T10:29:55.914 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:55.909+0000 7f5dec5d5640 1 -- --> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] -- mon_getmap magic: 0 -- 0x7f5de40d4680 con 0x7f5de40d5160 2024-04-25T10:29:55.914 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:55.909+0000 7f5deab72640 1 --2- >> v2:172.21.15.37:3300/0 conn(0x7f5de40d0da0 0x7f5de40d3060 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:29:55.914 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:55.909+0000 7f5deab72640 1 --2- >> v2:172.21.15.37:3300/0 conn(0x7f5de40d0da0 0x7f5de40d3060 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.37:3300/0 says I am v2:172.21.15.37:36476/0 (socket says 172.21.15.37:36476) 2024-04-25T10:29:55.915 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:55.910+0000 7f5deab72640 1 -- 172.21.15.37:0/3354796507 learned_addr learned my addr 172.21.15.37:0/3354796507 (peer_addr_for_me v2:172.21.15.37:0/0) 2024-04-25T10:29:55.915 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:55.910+0000 7f5de9b70640 1 --2- 172.21.15.37:0/3354796507 >> v2:172.21.15.203:3300/0 conn(0x7f5de40d0990 0x7f5de40d2b20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:29:55.915 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:55.910+0000 7f5dea371640 1 --2- 172.21.15.37:0/3354796507 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f5de40d5160 0x7f5de40b4e80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:29:55.915 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:55.910+0000 7f5deab72640 1 -- 172.21.15.37:0/3354796507 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f5de40d5160 msgr2=0x7f5de40b4e80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:29:55.915 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:55.910+0000 7f5deab72640 1 --2- 172.21.15.37:0/3354796507 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f5de40d5160 0x7f5de40b4e80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:29:55.915 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:55.910+0000 7f5deab72640 1 -- 172.21.15.37:0/3354796507 >> v2:172.21.15.203:3300/0 conn(0x7f5de40d0990 msgr2=0x7f5de40d2b20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:29:55.915 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:55.910+0000 7f5deab72640 1 --2- 172.21.15.37:0/3354796507 >> v2:172.21.15.203:3300/0 conn(0x7f5de40d0990 0x7f5de40d2b20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:29:55.915 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:55.910+0000 7f5deab72640 1 -- 172.21.15.37:0/3354796507 --> v2:172.21.15.37:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f5de40d4960 con 0x7f5de40d0da0 2024-04-25T10:29:55.915 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:55.910+0000 7f5de9b70640 1 --2- 172.21.15.37:0/3354796507 >> v2:172.21.15.203:3300/0 conn(0x7f5de40d0990 0x7f5de40d2b20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T10:29:55.915 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:55.910+0000 7f5deab72640 1 --2- 172.21.15.37:0/3354796507 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f5de40d0da0 0x7f5de40d3060 secure :-1 s=READY pgs=334 cs=0 l=1 rev1=1 crypto rx=0x7f5de40d4890 tx=0x7f5ddc02a560 comp rx=0 tx=0).ready entity=mon.0 client_cookie=270a4eb90f4417d2 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:29:55.915 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:55.911+0000 7f5de936f640 1 -- 172.21.15.37:0/3354796507 <== mon.0 v2:172.21.15.37:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5ddc036070 con 0x7f5de40d0da0 2024-04-25T10:29:55.915 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:55.911+0000 7f5de936f640 1 -- 172.21.15.37:0/3354796507 <== mon.0 v2:172.21.15.37:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f5ddc031600 con 0x7f5de40d0da0 2024-04-25T10:29:55.915 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:55.911+0000 7f5de936f640 1 -- 172.21.15.37:0/3354796507 <== mon.0 v2:172.21.15.37:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5ddc0319d0 con 0x7f5de40d0da0 2024-04-25T10:29:55.915 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:55.911+0000 7f5dec5d5640 1 -- 172.21.15.37:0/3354796507 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f5de40d0da0 msgr2=0x7f5de40d3060 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:29:55.916 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:55.911+0000 7f5dec5d5640 1 --2- 172.21.15.37:0/3354796507 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f5de40d0da0 0x7f5de40d3060 secure :-1 s=READY pgs=334 cs=0 l=1 rev1=1 crypto rx=0x7f5de40d4890 tx=0x7f5ddc02a560 comp rx=0 tx=0).stop 2024-04-25T10:29:55.916 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:55.911+0000 7f5dec5d5640 1 -- 172.21.15.37:0/3354796507 shutdown_connections 2024-04-25T10:29:55.916 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:55.911+0000 7f5dec5d5640 1 --2- 172.21.15.37:0/3354796507 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f5de40d0da0 0x7f5de40d3060 unknown :-1 s=CLOSED pgs=334 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:29:55.916 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:55.911+0000 7f5dec5d5640 1 --2- 172.21.15.37:0/3354796507 >> v2:172.21.15.203:3300/0 conn(0x7f5de40d0990 0x7f5de40d2b20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:29:55.916 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:55.911+0000 7f5dec5d5640 1 --2- 172.21.15.37:0/3354796507 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f5de40d5160 0x7f5de40b4e80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:29:55.916 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:55.911+0000 7f5dec5d5640 1 -- 172.21.15.37:0/3354796507 >> 172.21.15.37:0/3354796507 conn(0x7f5de40b7a80 msgr2=0x7f5de40b7e90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:29:55.916 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:55.912+0000 7f5dec5d5640 1 -- 172.21.15.37:0/3354796507 shutdown_connections 2024-04-25T10:29:55.916 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:55.912+0000 7f5dec5d5640 1 -- 172.21.15.37:0/3354796507 wait complete. 2024-04-25T10:29:55.918 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:55.913+0000 7f5dec5d5640 1 Processor -- start 2024-04-25T10:29:55.919 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:55.913+0000 7f5dec5d5640 1 -- start start 2024-04-25T10:29:55.919 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:55.914+0000 7f5dec5d5640 1 --2- >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f5de40d0990 0x7f5de4157980 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:29:55.919 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:55.915+0000 7f5dec5d5640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f5de40d0da0 0x7f5de4159ca0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:29:55.919 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:55.915+0000 7f5dec5d5640 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f5de40d5160 0x7f5de415b890 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:29:55.919 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:55.915+0000 7f5dec5d5640 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 -- 0x7f5de4164c40 con 0x7f5de40d0da0 2024-04-25T10:29:55.919 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:55.915+0000 7f5dec5d5640 1 -- --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_getmap magic: 0 -- 0x7f5de4164ac0 con 0x7f5de40d5160 2024-04-25T10:29:55.919 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:55.915+0000 7f5dec5d5640 1 -- --> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] -- mon_getmap magic: 0 -- 0x7f5de406db10 con 0x7f5de40d0990 2024-04-25T10:29:55.919 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:55.915+0000 7f5de9b70640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f5de40d0da0 0x7f5de4159ca0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:29:55.919 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:55.915+0000 7f5de9b70640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f5de40d0da0 0x7f5de4159ca0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.37:3300/0 says I am v2:172.21.15.37:36478/0 (socket says 172.21.15.37:36478) 2024-04-25T10:29:55.919 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:55.915+0000 7f5de9b70640 1 -- 172.21.15.37:0/2655207223 learned_addr learned my addr 172.21.15.37:0/2655207223 (peer_addr_for_me v2:172.21.15.37:0/0) 2024-04-25T10:29:55.919 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:55.915+0000 7f5deab72640 1 --2- 172.21.15.37:0/2655207223 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f5de40d5160 0x7f5de415b890 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:29:55.919 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:55.915+0000 7f5dea371640 1 --2- 172.21.15.37:0/2655207223 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f5de40d0990 0x7f5de4157980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:29:55.920 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:55.915+0000 7f5de9b70640 1 -- 172.21.15.37:0/2655207223 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f5de40d0990 msgr2=0x7f5de4157980 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:29:55.920 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:55.915+0000 7f5de9b70640 1 --2- 172.21.15.37:0/2655207223 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f5de40d0990 0x7f5de4157980 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:29:55.920 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:55.915+0000 7f5de9b70640 1 -- 172.21.15.37:0/2655207223 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f5de40d5160 msgr2=0x7f5de415b890 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:29:55.920 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:55.915+0000 7f5de9b70640 1 --2- 172.21.15.37:0/2655207223 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f5de40d5160 0x7f5de415b890 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:29:55.920 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:55.915+0000 7f5de9b70640 1 -- 172.21.15.37:0/2655207223 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f5de4157ec0 con 0x7f5de40d0da0 2024-04-25T10:29:55.920 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:55.915+0000 7f5de9b70640 1 --2- 172.21.15.37:0/2655207223 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f5de40d0da0 0x7f5de4159ca0 secure :-1 s=READY pgs=335 cs=0 l=1 rev1=1 crypto rx=0x7f5dd400d520 tx=0x7f5dd400d9e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:29:55.920 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:55.915+0000 7f5ddaffd640 1 -- 172.21.15.37:0/2655207223 <== mon.0 v2:172.21.15.37:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5dd4004030 con 0x7f5de40d0da0 2024-04-25T10:29:55.920 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:55.915+0000 7f5ddaffd640 1 -- 172.21.15.37:0/2655207223 <== mon.0 v2:172.21.15.37:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f5dd40041d0 con 0x7f5de40d0da0 2024-04-25T10:29:55.920 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:55.915+0000 7f5dec5d5640 1 -- 172.21.15.37:0/2655207223 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f5de415a7a0 con 0x7f5de40d0da0 2024-04-25T10:29:55.920 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:55.915+0000 7f5ddaffd640 1 -- 172.21.15.37:0/2655207223 <== mon.0 v2:172.21.15.37:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5dd4014620 con 0x7f5de40d0da0 2024-04-25T10:29:55.920 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:55.915+0000 7f5dec5d5640 1 -- 172.21.15.37:0/2655207223 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f5de415ad60 con 0x7f5de40d0da0 2024-04-25T10:29:55.920 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:55.917+0000 7f5dec5d5640 1 -- 172.21.15.37:0/2655207223 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f5de415a4f0 con 0x7f5de40d0da0 2024-04-25T10:29:55.921 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:55.917+0000 7f5ddaffd640 1 -- 172.21.15.37:0/2655207223 <== mon.0 v2:172.21.15.37:3300/0 4 ==== mgrmap(e 5) ==== 96782+0+0 (secure 0 0 0) 0x7f5dd40147c0 con 0x7f5de40d0da0 2024-04-25T10:29:55.921 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:55.918+0000 7f5ddaffd640 1 --2- 172.21.15.37:0/2655207223 >> [v2:172.21.15.203:6832/2418205237,v1:172.21.15.203:6833/2418205237] conn(0x7f5dc80744e0 0x7f5dc80769f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:29:55.922 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:55.918+0000 7f5ddaffd640 1 -- 172.21.15.37:0/2655207223 <== mon.0 v2:172.21.15.37:3300/0 5 ==== osd_map(25..25 src has 1..25) ==== 5738+0+0 (secure 0 0 0) 0x7f5dd40958c0 con 0x7f5de40d0da0 2024-04-25T10:29:55.922 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:55.918+0000 7f5dea371640 1 --2- 172.21.15.37:0/2655207223 >> [v2:172.21.15.203:6832/2418205237,v1:172.21.15.203:6833/2418205237] conn(0x7f5dc80744e0 0x7f5dc80769f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:29:55.922 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:55.918+0000 7f5dea371640 1 --2- 172.21.15.37:0/2655207223 >> [v2:172.21.15.203:6832/2418205237,v1:172.21.15.203:6833/2418205237] conn(0x7f5dc80744e0 0x7f5dc80769f0 secure :-1 s=READY pgs=173 cs=0 l=1 rev1=1 crypto rx=0x7f5de0006fd0 tx=0x7f5de0008040 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:29:55.924 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:55.922+0000 7f5ddaffd640 1 -- 172.21.15.37:0/2655207223 <== mon.0 v2:172.21.15.37:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187246 (secure 0 0 0) 0x7f5de4157ec0 con 0x7f5de40d0da0 2024-04-25T10:29:56.127 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:56.124+0000 7f5dec5d5640 1 -- 172.21.15.37:0/2655207223 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_command({"prefix": "osd blocklist ls"} v 0) -- 0x7f5de406a140 con 0x7f5de40d0da0 2024-04-25T10:29:56.127 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:56.125+0000 7f5ddaffd640 1 -- 172.21.15.37:0/2655207223 <== mon.0 v2:172.21.15.37:3300/0 7 ==== mon_command_ack([{"prefix": "osd blocklist ls"}]=0 listed 0 entries v25) ==== 80+0+0 (secure 0 0 0) 0x7f5dd4060040 con 0x7f5de40d0da0 2024-04-25T10:29:56.127 INFO:teuthology.orchestra.run.smithi037.stderr:listed 0 entries 2024-04-25T10:29:56.129 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:56.127+0000 7f5dec5d5640 1 -- 172.21.15.37:0/2655207223 >> [v2:172.21.15.203:6832/2418205237,v1:172.21.15.203:6833/2418205237] conn(0x7f5dc80744e0 msgr2=0x7f5dc80769f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:29:56.129 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:56.127+0000 7f5dec5d5640 1 --2- 172.21.15.37:0/2655207223 >> [v2:172.21.15.203:6832/2418205237,v1:172.21.15.203:6833/2418205237] conn(0x7f5dc80744e0 0x7f5dc80769f0 secure :-1 s=READY pgs=173 cs=0 l=1 rev1=1 crypto rx=0x7f5de0006fd0 tx=0x7f5de0008040 comp rx=0 tx=0).stop 2024-04-25T10:29:56.130 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:56.127+0000 7f5dec5d5640 1 -- 172.21.15.37:0/2655207223 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f5de40d0da0 msgr2=0x7f5de4159ca0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:29:56.130 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:56.127+0000 7f5dec5d5640 1 --2- 172.21.15.37:0/2655207223 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f5de40d0da0 0x7f5de4159ca0 secure :-1 s=READY pgs=335 cs=0 l=1 rev1=1 crypto rx=0x7f5dd400d520 tx=0x7f5dd400d9e0 comp rx=0 tx=0).stop 2024-04-25T10:29:56.130 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:56.127+0000 7f5dec5d5640 1 -- 172.21.15.37:0/2655207223 shutdown_connections 2024-04-25T10:29:56.130 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:56.127+0000 7f5dec5d5640 1 --2- 172.21.15.37:0/2655207223 >> [v2:172.21.15.203:6832/2418205237,v1:172.21.15.203:6833/2418205237] conn(0x7f5dc80744e0 0x7f5dc80769f0 unknown :-1 s=CLOSED pgs=173 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:29:56.130 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:56.127+0000 7f5dec5d5640 1 --2- 172.21.15.37:0/2655207223 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f5de40d5160 0x7f5de415b890 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:29:56.130 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:56.127+0000 7f5dec5d5640 1 --2- 172.21.15.37:0/2655207223 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f5de40d0da0 0x7f5de4159ca0 unknown :-1 s=CLOSED pgs=335 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:29:56.130 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:56.127+0000 7f5dec5d5640 1 --2- 172.21.15.37:0/2655207223 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f5de40d0990 0x7f5de4157980 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:29:56.130 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:56.127+0000 7f5dec5d5640 1 -- 172.21.15.37:0/2655207223 >> 172.21.15.37:0/2655207223 conn(0x7f5de40b7a80 msgr2=0x7f5de40b9020 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:29:56.130 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:56.127+0000 7f5dec5d5640 1 -- 172.21.15.37:0/2655207223 shutdown_connections 2024-04-25T10:29:56.130 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:56.127+0000 7f5dec5d5640 1 -- 172.21.15.37:0/2655207223 wait complete. 2024-04-25T10:29:56.140 DEBUG:teuthology.orchestra.run.smithi203:> set -ex 2024-04-25T10:29:56.141 DEBUG:teuthology.orchestra.run.smithi203:> dd if=/proc/self/mounts of=/dev/stdout 2024-04-25T10:29:56.159 DEBUG:teuthology.orchestra.run.smithi037:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-04-25T10:29:56.270 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:56.268+0000 7f51640a1640 1 Processor -- start 2024-04-25T10:29:56.271 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:56.269+0000 7f51640a1640 1 -- start start 2024-04-25T10:29:56.273 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:56.270+0000 7f51640a1640 1 --2- >> v2:172.21.15.203:3300/0 conn(0x7f515c0b2dc0 0x7f515c0d2860 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:29:56.273 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:56.270+0000 7f51640a1640 1 --2- >> v2:172.21.15.37:3300/0 conn(0x7f515c06daf0 0x7f515c06df00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:29:56.273 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:56.270+0000 7f51640a1640 1 -- --> v1:172.21.15.203:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f515c0d2e40 con 0x7f515c06e440 2024-04-25T10:29:56.273 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:56.270+0000 7f51640a1640 1 -- --> v2:172.21.15.37:3300/0 -- mon_getmap magic: 0 -- 0x7f515c0b92c0 con 0x7f515c06daf0 2024-04-25T10:29:56.273 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:56.270+0000 7f51640a1640 1 -- --> v2:172.21.15.203:3300/0 -- mon_getmap magic: 0 -- 0x7f515c0ce0d0 con 0x7f515c0b2dc0 2024-04-25T10:29:56.273 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:56.270+0000 7f516163c640 1 --2- >> v2:172.21.15.37:3300/0 conn(0x7f515c06daf0 0x7f515c06df00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:29:56.273 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:56.270+0000 7f516163c640 1 --2- >> v2:172.21.15.37:3300/0 conn(0x7f515c06daf0 0x7f515c06df00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.37:3300/0 says I am v2:172.21.15.37:36484/0 (socket says 172.21.15.37:36484) 2024-04-25T10:29:56.273 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:56.270+0000 7f516163c640 1 -- 172.21.15.37:0/67012354 learned_addr learned my addr 172.21.15.37:0/67012354 (peer_addr_for_me v2:172.21.15.37:0/0) 2024-04-25T10:29:56.273 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:56.270+0000 7f516163c640 1 -- 172.21.15.37:0/67012354 >> v2:172.21.15.203:3300/0 conn(0x7f515c0b2dc0 msgr2=0x7f515c0d2860 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:29:56.273 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:56.270+0000 7f5161e3d640 1 --2- 172.21.15.37:0/67012354 >> v2:172.21.15.203:3300/0 conn(0x7f515c0b2dc0 0x7f515c0d2860 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:29:56.273 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:56.270+0000 7f516163c640 1 --2- 172.21.15.37:0/67012354 >> v2:172.21.15.203:3300/0 conn(0x7f515c0b2dc0 0x7f515c0d2860 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:29:56.273 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:56.270+0000 7f516163c640 1 -- 172.21.15.37:0/67012354 >> v1:172.21.15.203:6789/0 conn(0x7f515c06e440 legacy=0x7f515c08cc90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:29:56.274 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:56.270+0000 7f516163c640 1 -- 172.21.15.37:0/67012354 --> v2:172.21.15.37:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f515c0d3120 con 0x7f515c06daf0 2024-04-25T10:29:56.274 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:56.270+0000 7f5161e3d640 1 --2- 172.21.15.37:0/67012354 >> v2:172.21.15.203:3300/0 conn(0x7f515c0b2dc0 0x7f515c0d2860 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T10:29:56.275 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:56.271+0000 7f516163c640 1 --2- 172.21.15.37:0/67012354 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f515c06daf0 0x7f515c06df00 secure :-1 s=READY pgs=336 cs=0 l=1 rev1=1 crypto rx=0x7f515c070b40 tx=0x7f515002a230 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ddfb4a74c1c0d902 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:29:56.275 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:56.271+0000 7f5160e3b640 1 -- 172.21.15.37:0/67012354 <== mon.0 v2:172.21.15.37:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5150037070 con 0x7f515c06daf0 2024-04-25T10:29:56.275 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:56.271+0000 7f5160e3b640 1 -- 172.21.15.37:0/67012354 <== mon.0 v2:172.21.15.37:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f5150032440 con 0x7f515c06daf0 2024-04-25T10:29:56.275 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:56.271+0000 7f5160e3b640 1 -- 172.21.15.37:0/67012354 <== mon.0 v2:172.21.15.37:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5150032810 con 0x7f515c06daf0 2024-04-25T10:29:56.275 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:56.272+0000 7f51640a1640 1 -- 172.21.15.37:0/67012354 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f515c06daf0 msgr2=0x7f515c06df00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:29:56.275 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:56.272+0000 7f51640a1640 1 --2- 172.21.15.37:0/67012354 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f515c06daf0 0x7f515c06df00 secure :-1 s=READY pgs=336 cs=0 l=1 rev1=1 crypto rx=0x7f515c070b40 tx=0x7f515002a230 comp rx=0 tx=0).stop 2024-04-25T10:29:56.275 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:56.272+0000 7f51640a1640 1 -- 172.21.15.37:0/67012354 shutdown_connections 2024-04-25T10:29:56.275 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:56.272+0000 7f51640a1640 1 --2- 172.21.15.37:0/67012354 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f515c06daf0 0x7f515c06df00 unknown :-1 s=CLOSED pgs=336 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:29:56.276 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:56.272+0000 7f51640a1640 1 --2- 172.21.15.37:0/67012354 >> v2:172.21.15.203:3300/0 conn(0x7f515c0b2dc0 0x7f515c0d2860 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:29:56.276 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:56.272+0000 7f51640a1640 1 -- 172.21.15.37:0/67012354 >> 172.21.15.37:0/67012354 conn(0x7f515c0b5440 msgr2=0x7f515c0b5850 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:29:56.276 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:56.272+0000 7f51640a1640 1 -- 172.21.15.37:0/67012354 shutdown_connections 2024-04-25T10:29:56.276 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:56.272+0000 7f51640a1640 1 -- 172.21.15.37:0/67012354 wait complete. 2024-04-25T10:29:56.276 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:56.274+0000 7f51640a1640 1 Processor -- start 2024-04-25T10:29:56.276 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:56.274+0000 7f51640a1640 1 -- start start 2024-04-25T10:29:56.279 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:56.275+0000 7f51640a1640 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f515c06daf0 0x7f515c0c7480 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:29:56.279 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:56.275+0000 7f51640a1640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f515c06e440 0x7f515c1f99a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:29:56.279 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:56.275+0000 7f51640a1640 1 --2- >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f515c0b2dc0 0x7f515c162c40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:29:56.279 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:56.275+0000 7f51640a1640 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 -- 0x7f515c06e850 con 0x7f515c06e440 2024-04-25T10:29:56.279 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:56.275+0000 7f51640a1640 1 -- --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_getmap magic: 0 -- 0x7f515c0d36e0 con 0x7f515c06daf0 2024-04-25T10:29:56.279 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:56.275+0000 7f51640a1640 1 -- --> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] -- mon_getmap magic: 0 -- 0x7f515c0cf570 con 0x7f515c0b2dc0 2024-04-25T10:29:56.279 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:56.275+0000 7f516163c640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f515c06e440 0x7f515c1f99a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:29:56.279 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:56.275+0000 7f516163c640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f515c06e440 0x7f515c1f99a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.37:3300/0 says I am v2:172.21.15.37:36500/0 (socket says 172.21.15.37:36500) 2024-04-25T10:29:56.279 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:56.275+0000 7f516163c640 1 -- 172.21.15.37:0/1196558598 learned_addr learned my addr 172.21.15.37:0/1196558598 (peer_addr_for_me v2:172.21.15.37:0/0) 2024-04-25T10:29:56.279 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:56.276+0000 7f516163c640 1 -- 172.21.15.37:0/1196558598 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f515c0b2dc0 msgr2=0x7f515c162c40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:29:56.280 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:56.276+0000 7f5161e3d640 1 --2- 172.21.15.37:0/1196558598 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f515c06daf0 0x7f515c0c7480 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:29:56.280 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:56.276+0000 7f516263e640 1 --2- 172.21.15.37:0/1196558598 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f515c0b2dc0 0x7f515c162c40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:29:56.280 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:56.276+0000 7f516163c640 1 --2- 172.21.15.37:0/1196558598 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f515c0b2dc0 0x7f515c162c40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:29:56.280 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:56.276+0000 7f516163c640 1 -- 172.21.15.37:0/1196558598 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f515c06daf0 msgr2=0x7f515c0c7480 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:29:56.280 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:56.276+0000 7f516163c640 1 --2- 172.21.15.37:0/1196558598 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f515c06daf0 0x7f515c0c7480 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:29:56.280 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:56.276+0000 7f516163c640 1 -- 172.21.15.37:0/1196558598 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f515c0ca020 con 0x7f515c06e440 2024-04-25T10:29:56.280 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:56.276+0000 7f516163c640 1 --2- 172.21.15.37:0/1196558598 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f515c06e440 0x7f515c1f99a0 secure :-1 s=READY pgs=337 cs=0 l=1 rev1=1 crypto rx=0x7f515002a270 tx=0x7f51500046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:29:56.280 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:56.276+0000 7f515a7fc640 1 -- 172.21.15.37:0/1196558598 <== mon.0 v2:172.21.15.37:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5150037040 con 0x7f515c06e440 2024-04-25T10:29:56.280 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:56.276+0000 7f515a7fc640 1 -- 172.21.15.37:0/1196558598 <== mon.0 v2:172.21.15.37:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f5150042070 con 0x7f515c06e440 2024-04-25T10:29:56.280 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:56.276+0000 7f51640a1640 1 -- 172.21.15.37:0/1196558598 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f515c0ca300 con 0x7f515c06e440 2024-04-25T10:29:56.280 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:56.276+0000 7f515a7fc640 1 -- 172.21.15.37:0/1196558598 <== mon.0 v2:172.21.15.37:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5150030b90 con 0x7f515c06e440 2024-04-25T10:29:56.280 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:56.276+0000 7f51640a1640 1 -- 172.21.15.37:0/1196558598 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f515c0b0d10 con 0x7f515c06e440 2024-04-25T10:29:56.280 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:56.277+0000 7f51640a1640 1 -- 172.21.15.37:0/1196558598 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f515c06a140 con 0x7f515c06e440 2024-04-25T10:29:56.281 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:56.278+0000 7f515a7fc640 1 -- 172.21.15.37:0/1196558598 <== mon.0 v2:172.21.15.37:3300/0 4 ==== mgrmap(e 5) ==== 96782+0+0 (secure 0 0 0) 0x7f5150047460 con 0x7f515c06e440 2024-04-25T10:29:56.281 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:56.278+0000 7f515a7fc640 1 --2- 172.21.15.37:0/1196558598 >> [v2:172.21.15.203:6832/2418205237,v1:172.21.15.203:6833/2418205237] conn(0x7f51480744b0 0x7f51480769c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:29:56.281 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:56.279+0000 7f515a7fc640 1 -- 172.21.15.37:0/1196558598 <== mon.0 v2:172.21.15.37:3300/0 5 ==== osd_map(25..25 src has 1..25) ==== 5738+0+0 (secure 0 0 0) 0x7f51500b5fc0 con 0x7f515c06e440 2024-04-25T10:29:56.281 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:56.279+0000 7f5161e3d640 1 --2- 172.21.15.37:0/1196558598 >> [v2:172.21.15.203:6832/2418205237,v1:172.21.15.203:6833/2418205237] conn(0x7f51480744b0 0x7f51480769c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:29:56.282 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:56.279+0000 7f5161e3d640 1 --2- 172.21.15.37:0/1196558598 >> [v2:172.21.15.203:6832/2418205237,v1:172.21.15.203:6833/2418205237] conn(0x7f51480744b0 0x7f51480769c0 secure :-1 s=READY pgs=174 cs=0 l=1 rev1=1 crypto rx=0x7f515c074cc0 tx=0x7f514c0084a0 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:29:56.285 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:56.283+0000 7f515a7fc640 1 -- 172.21.15.37:0/1196558598 <== mon.0 v2:172.21.15.37:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187246 (secure 0 0 0) 0x7f51500800d0 con 0x7f515c06e440 2024-04-25T10:29:56.478 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:56.476+0000 7f51640a1640 1 -- 172.21.15.37:0/1196558598 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7f515c0d2b80 con 0x7f515c06e440 2024-04-25T10:29:56.479 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:56.477+0000 7f515a7fc640 1 -- 172.21.15.37:0/1196558598 <== mon.0 v2:172.21.15.37:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v25) ==== 74+0+16670 (secure 0 0 0) 0x7f51500843d0 con 0x7f515c06e440 2024-04-25T10:29:56.479 INFO:teuthology.orchestra.run.smithi037.stdout: 2024-04-25T10:29:56.480 INFO:teuthology.orchestra.run.smithi037.stdout:{"epoch":25,"fsid":"f4acda28-be34-492a-b286-994afd36f61d","created":"2024-04-25T10:28:43.985718+0000","modified":"2024-04-25T10:29:49.325719+0000","last_up_change":"2024-04-25T10:28:54.900654+0000","last_in_change":"2024-04-25T10:28:46.216973+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":3,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"luminous","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-04-25T10:28:56.202153+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"19","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2024-04-25T10:29:06.507244+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"22","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":2.25,"score_stable":2.25,"optimal_score":1,"raw_score_acting":2.25,"raw_score_stable":2.25,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2024-04-25T10:29:07.347844+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"23","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"a2f79028-b332-4166-9ebb-a7024a37ea22","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":21,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.37:6816","nonce":2746074720},{"type":"v1","addr":"172.21.15.37:6817","nonce":2746074720}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.37:6818","nonce":2746074720},{"type":"v1","addr":"172.21.15.37:6819","nonce":2746074720}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.37:6822","nonce":2746074720},{"type":"v1","addr":"172.21.15.37:6823","nonce":2746074720}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.37:6820","nonce":2746074720},{"type":"v1","addr":"172.21.15.37:6821","nonce":2746074720}]},"public_addr":"172.21.15.37:6817/2746074720","cluster_addr":"172.21.15.37:6819/2746074720","heartbeat_back_addr":"172.21.15.37:6823/2746074720","heartbeat_front_addr":"172.21.15.37:6821/2746074720","state":["exists","up"]},{"osd":1,"uuid":"973159de-c2ea-4083-aecc-69373d1c1132","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":21,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.37:6800","nonce":1690341662},{"type":"v1","addr":"172.21.15.37:6801","nonce":1690341662}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.37:6802","nonce":1690341662},{"type":"v1","addr":"172.21.15.37:6803","nonce":1690341662}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.37:6806","nonce":1690341662},{"type":"v1","addr":"172.21.15.37:6807","nonce":1690341662}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.37:6804","nonce":1690341662},{"type":"v1","addr":"172.21.15.37:6805","nonce":1690341662}]},"public_addr":"172.21.15.37:6801/1690341662","cluster_addr":"172.21.15.37:6803/1690341662","heartbeat_back_addr":"172.21.15.37:6807/1690341662","heartbeat_front_addr":"172.21.15.37:6805/1690341662","state":["exists","up"]},{"osd":2,"uuid":"4ae7a0e9-cb3f-484c-a75f-f3766b721c47","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":21,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.37:6808","nonce":2754669649},{"type":"v1","addr":"172.21.15.37:6809","nonce":2754669649}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.37:6810","nonce":2754669649},{"type":"v1","addr":"172.21.15.37:6811","nonce":2754669649}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.37:6814","nonce":2754669649},{"type":"v1","addr":"172.21.15.37:6815","nonce":2754669649}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.37:6812","nonce":2754669649},{"type":"v1","addr":"172.21.15.37:6813","nonce":2754669649}]},"public_addr":"172.21.15.37:6809/2754669649","cluster_addr":"172.21.15.37:6811/2754669649","heartbeat_back_addr":"172.21.15.37:6815/2754669649","heartbeat_front_addr":"172.21.15.37:6813/2754669649","state":["exists","up"]},{"osd":3,"uuid":"1ba4a746-1608-47a7-ace2-960a5cfae934","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":21,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.37:6824","nonce":1815290618},{"type":"v1","addr":"172.21.15.37:6825","nonce":1815290618}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.37:6826","nonce":1815290618},{"type":"v1","addr":"172.21.15.37:6827","nonce":1815290618}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.37:6830","nonce":1815290618},{"type":"v1","addr":"172.21.15.37:6831","nonce":1815290618}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.37:6828","nonce":1815290618},{"type":"v1","addr":"172.21.15.37:6829","nonce":1815290618}]},"public_addr":"172.21.15.37:6825/1815290618","cluster_addr":"172.21.15.37:6827/1815290618","heartbeat_back_addr":"172.21.15.37:6831/1815290618","heartbeat_front_addr":"172.21.15.37:6829/1815290618","state":["exists","up"]},{"osd":4,"uuid":"d4f59f5d-0413-4b79-87ee-bb212c1fcf62","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":21,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6800","nonce":4051532063},{"type":"v1","addr":"172.21.15.203:6801","nonce":4051532063}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6802","nonce":4051532063},{"type":"v1","addr":"172.21.15.203:6803","nonce":4051532063}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6806","nonce":4051532063},{"type":"v1","addr":"172.21.15.203:6807","nonce":4051532063}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6804","nonce":4051532063},{"type":"v1","addr":"172.21.15.203:6805","nonce":4051532063}]},"public_addr":"172.21.15.203:6801/4051532063","cluster_addr":"172.21.15.203:6803/4051532063","heartbeat_back_addr":"172.21.15.203:6807/4051532063","heartbeat_front_addr":"172.21.15.203:6805/4051532063","state":["exists","up"]},{"osd":5,"uuid":"69710a0e-41d8-4e7f-9a10-17ce540b67be","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":24,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6824","nonce":1203822890},{"type":"v1","addr":"172.21.15.203:6825","nonce":1203822890}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6826","nonce":1203822890},{"type":"v1","addr":"172.21.15.203:6827","nonce":1203822890}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6830","nonce":1203822890},{"type":"v1","addr":"172.21.15.203:6831","nonce":1203822890}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6828","nonce":1203822890},{"type":"v1","addr":"172.21.15.203:6829","nonce":1203822890}]},"public_addr":"172.21.15.203:6825/1203822890","cluster_addr":"172.21.15.203:6827/1203822890","heartbeat_back_addr":"172.21.15.203:6831/1203822890","heartbeat_front_addr":"172.21.15.203:6829/1203822890","state":["exists","up"]},{"osd":6,"uuid":"f29dd047-3b48-473d-a235-6642a6293f20","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":21,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6810","nonce":2309615014},{"type":"v1","addr":"172.21.15.203:6812","nonce":2309615014}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6814","nonce":2309615014},{"type":"v1","addr":"172.21.15.203:6816","nonce":2309615014}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6822","nonce":2309615014},{"type":"v1","addr":"172.21.15.203:6823","nonce":2309615014}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6818","nonce":2309615014},{"type":"v1","addr":"172.21.15.203:6820","nonce":2309615014}]},"public_addr":"172.21.15.203:6812/2309615014","cluster_addr":"172.21.15.203:6816/2309615014","heartbeat_back_addr":"172.21.15.203:6823/2309615014","heartbeat_front_addr":"172.21.15.203:6820/2309615014","state":["exists","up"]},{"osd":7,"uuid":"6e691644-378b-4fdc-8e86-fb5f56e31f29","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":21,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6808","nonce":3688099014},{"type":"v1","addr":"172.21.15.203:6809","nonce":3688099014}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6811","nonce":3688099014},{"type":"v1","addr":"172.21.15.203:6813","nonce":3688099014}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6819","nonce":3688099014},{"type":"v1","addr":"172.21.15.203:6821","nonce":3688099014}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6815","nonce":3688099014},{"type":"v1","addr":"172.21.15.203:6817","nonce":3688099014}]},"public_addr":"172.21.15.203:6809/3688099014","cluster_addr":"172.21.15.203:6813/3688099014","heartbeat_back_addr":"172.21.15.203:6821/3688099014","heartbeat_front_addr":"172.21.15.203:6817/3688099014","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T10:28:51.419141+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T10:28:51.366448+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T10:28:51.348276+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T10:28:51.380519+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T10:28:51.105812+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T10:28:51.384496+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T10:28:51.388755+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T10:28:51.366958+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[{"pgid":"3.31","mappings":[{"from":0,"to":7}]}],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-04-25T10:29:56.481 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:56.479+0000 7f51640a1640 1 -- 172.21.15.37:0/1196558598 >> [v2:172.21.15.203:6832/2418205237,v1:172.21.15.203:6833/2418205237] conn(0x7f51480744b0 msgr2=0x7f51480769c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:29:56.481 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:56.479+0000 7f51640a1640 1 --2- 172.21.15.37:0/1196558598 >> [v2:172.21.15.203:6832/2418205237,v1:172.21.15.203:6833/2418205237] conn(0x7f51480744b0 0x7f51480769c0 secure :-1 s=READY pgs=174 cs=0 l=1 rev1=1 crypto rx=0x7f515c074cc0 tx=0x7f514c0084a0 comp rx=0 tx=0).stop 2024-04-25T10:29:56.482 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:56.479+0000 7f51640a1640 1 -- 172.21.15.37:0/1196558598 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f515c06e440 msgr2=0x7f515c1f99a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:29:56.482 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:56.479+0000 7f51640a1640 1 --2- 172.21.15.37:0/1196558598 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f515c06e440 0x7f515c1f99a0 secure :-1 s=READY pgs=337 cs=0 l=1 rev1=1 crypto rx=0x7f515002a270 tx=0x7f51500046c0 comp rx=0 tx=0).stop 2024-04-25T10:29:56.482 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:56.479+0000 7f51640a1640 1 -- 172.21.15.37:0/1196558598 shutdown_connections 2024-04-25T10:29:56.482 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:56.479+0000 7f51640a1640 1 --2- 172.21.15.37:0/1196558598 >> [v2:172.21.15.203:6832/2418205237,v1:172.21.15.203:6833/2418205237] conn(0x7f51480744b0 0x7f51480769c0 unknown :-1 s=CLOSED pgs=174 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:29:56.482 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:56.479+0000 7f51640a1640 1 --2- 172.21.15.37:0/1196558598 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f515c0b2dc0 0x7f515c162c40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:29:56.482 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:56.479+0000 7f51640a1640 1 --2- 172.21.15.37:0/1196558598 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f515c06e440 0x7f515c1f99a0 unknown :-1 s=CLOSED pgs=337 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:29:56.482 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:56.479+0000 7f51640a1640 1 --2- 172.21.15.37:0/1196558598 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f515c06daf0 0x7f515c0c7480 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:29:56.482 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:56.479+0000 7f51640a1640 1 -- 172.21.15.37:0/1196558598 >> 172.21.15.37:0/1196558598 conn(0x7f515c0b5440 msgr2=0x7f515c0cffa0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:29:56.482 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:56.479+0000 7f51640a1640 1 -- 172.21.15.37:0/1196558598 shutdown_connections 2024-04-25T10:29:56.482 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:56.479+0000 7f51640a1640 1 -- 172.21.15.37:0/1196558598 wait complete. 2024-04-25T10:29:56.493 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2024-04-25T10:28:56.202153+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '19', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 8, 'score_stable': 8, 'optimal_score': 0.25, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 2, 'pool_name': 'cephfs_metadata', 'create_time': '2024-04-25T10:29:06.507244+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '22', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_autoscale_bias': 4, 'pg_num_min': 16, 'recovery_priority': 5}, 'application_metadata': {'cephfs': {'metadata': 'cephfs'}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 2.25, 'score_stable': 2.25, 'optimal_score': 1, 'raw_score_acting': 2.25, 'raw_score_stable': 2.25, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 3, 'pool_name': 'cephfs_data', 'create_time': '2024-04-25T10:29:07.347844+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 64, 'pg_placement_num': 64, 'pg_placement_num_target': 64, 'pg_num_target': 64, 'pg_num_pending': 64, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '23', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.8999999761581421}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 1.5, 'score_stable': 1.5, 'optimal_score': 1, 'raw_score_acting': 1.5, 'raw_score_stable': 1.5, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}] 2024-04-25T10:29:56.493 DEBUG:teuthology.orchestra.run.smithi037:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2024-04-25T10:29:56.607 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:56.605+0000 7f1964c42640 1 Processor -- start 2024-04-25T10:29:56.607 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:56.605+0000 7f1964c42640 1 -- start start 2024-04-25T10:29:56.609 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:56.607+0000 7f1964c42640 1 --2- >> v2:172.21.15.203:3300/0 conn(0x7f19600d5160 0x7f19600b4e80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:29:56.610 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:56.607+0000 7f1964c42640 1 --2- >> v2:172.21.15.37:3300/0 conn(0x7f19600d0990 0x7f19600d2b20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:29:56.610 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:56.607+0000 7f1964c42640 1 -- --> v1:172.21.15.203:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f19600d1370 con 0x7f19600d0da0 2024-04-25T10:29:56.610 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:56.607+0000 7f1964c42640 1 -- --> v2:172.21.15.37:3300/0 -- mon_getmap magic: 0 -- 0x7f19600d1650 con 0x7f19600d0990 2024-04-25T10:29:56.610 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:56.607+0000 7f1964c42640 1 -- --> v2:172.21.15.203:3300/0 -- mon_getmap magic: 0 -- 0x7f19600d4680 con 0x7f19600d5160 2024-04-25T10:29:56.610 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:56.607+0000 7f195dd9b640 1 --2- >> v2:172.21.15.37:3300/0 conn(0x7f19600d0990 0x7f19600d2b20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:29:56.610 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:56.607+0000 7f195dd9b640 1 --2- >> v2:172.21.15.37:3300/0 conn(0x7f19600d0990 0x7f19600d2b20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.37:3300/0 says I am v2:172.21.15.37:36502/0 (socket says 172.21.15.37:36502) 2024-04-25T10:29:56.610 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:56.607+0000 7f195dd9b640 1 -- 172.21.15.37:0/572990389 learned_addr learned my addr 172.21.15.37:0/572990389 (peer_addr_for_me v2:172.21.15.37:0/0) 2024-04-25T10:29:56.610 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:56.607+0000 7f195e59c640 1 --2- 172.21.15.37:0/572990389 >> v2:172.21.15.203:3300/0 conn(0x7f19600d5160 0x7f19600b4e80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:29:56.610 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:56.607+0000 7f195dd9b640 1 -- 172.21.15.37:0/572990389 >> v2:172.21.15.203:3300/0 conn(0x7f19600d5160 msgr2=0x7f19600b4e80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:29:56.610 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:56.607+0000 7f195dd9b640 1 --2- 172.21.15.37:0/572990389 >> v2:172.21.15.203:3300/0 conn(0x7f19600d5160 0x7f19600b4e80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:29:56.610 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:56.607+0000 7f195dd9b640 1 -- 172.21.15.37:0/572990389 >> v1:172.21.15.203:6789/0 conn(0x7f19600d0da0 legacy=0x7f196006db10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:29:56.610 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:56.607+0000 7f195dd9b640 1 -- 172.21.15.37:0/572990389 --> v2:172.21.15.37:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f19600d4960 con 0x7f19600d0990 2024-04-25T10:29:56.611 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:56.608+0000 7f195dd9b640 1 --2- 172.21.15.37:0/572990389 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f19600d0990 0x7f19600d2b20 secure :-1 s=READY pgs=338 cs=0 l=1 rev1=1 crypto rx=0x7f19600d6b80 tx=0x7f195402a0d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=1010aa11abd86fb0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:29:56.612 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:56.608+0000 7f195d59a640 1 -- 172.21.15.37:0/572990389 <== mon.0 v2:172.21.15.37:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1954037070 con 0x7f19600d0990 2024-04-25T10:29:56.612 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:56.608+0000 7f195d59a640 1 -- 172.21.15.37:0/572990389 <== mon.0 v2:172.21.15.37:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f195402ade0 con 0x7f19600d0990 2024-04-25T10:29:56.612 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:56.608+0000 7f195d59a640 1 -- 172.21.15.37:0/572990389 <== mon.0 v2:172.21.15.37:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1954032660 con 0x7f19600d0990 2024-04-25T10:29:56.612 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:56.609+0000 7f1964c42640 1 -- 172.21.15.37:0/572990389 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f19600d0990 msgr2=0x7f19600d2b20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:29:56.612 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:56.609+0000 7f1964c42640 1 --2- 172.21.15.37:0/572990389 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f19600d0990 0x7f19600d2b20 secure :-1 s=READY pgs=338 cs=0 l=1 rev1=1 crypto rx=0x7f19600d6b80 tx=0x7f195402a0d0 comp rx=0 tx=0).stop 2024-04-25T10:29:56.612 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:56.609+0000 7f1964c42640 1 -- 172.21.15.37:0/572990389 shutdown_connections 2024-04-25T10:29:56.612 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:56.609+0000 7f1964c42640 1 --2- 172.21.15.37:0/572990389 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f19600d0990 0x7f19600d2b20 unknown :-1 s=CLOSED pgs=338 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:29:56.612 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:56.609+0000 7f1964c42640 1 --2- 172.21.15.37:0/572990389 >> v2:172.21.15.203:3300/0 conn(0x7f19600d5160 0x7f19600b4e80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:29:56.612 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:56.609+0000 7f1964c42640 1 -- 172.21.15.37:0/572990389 >> 172.21.15.37:0/572990389 conn(0x7f19600b7a80 msgr2=0x7f19600b7e90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:29:56.612 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:56.609+0000 7f1964c42640 1 -- 172.21.15.37:0/572990389 shutdown_connections 2024-04-25T10:29:56.612 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:56.609+0000 7f1964c42640 1 -- 172.21.15.37:0/572990389 wait complete. 2024-04-25T10:29:56.613 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:56.611+0000 7f1964c42640 1 Processor -- start 2024-04-25T10:29:56.613 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:56.611+0000 7f1964c42640 1 -- start start 2024-04-25T10:29:56.615 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:56.612+0000 7f1964c42640 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f19600d0990 0x7f1960161d40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:29:56.615 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:56.612+0000 7f1964c42640 1 --2- >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f19600d0da0 0x7f1960147c70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:29:56.615 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:56.612+0000 7f1964c42640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f19600d5160 0x7f19601f90f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:29:56.615 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:56.612+0000 7f1964c42640 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 -- 0x7f1960164c40 con 0x7f19600d5160 2024-04-25T10:29:56.616 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:56.612+0000 7f1964c42640 1 -- --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_getmap magic: 0 -- 0x7f1960164ac0 con 0x7f19600d0990 2024-04-25T10:29:56.616 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:56.612+0000 7f1964c42640 1 -- --> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] -- mon_getmap magic: 0 -- 0x7f19600d3060 con 0x7f19600d0da0 2024-04-25T10:29:56.616 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:56.612+0000 7f195ed9d640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f19600d5160 0x7f19601f90f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:29:56.616 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:56.612+0000 7f195ed9d640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f19600d5160 0x7f19601f90f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.37:3300/0 says I am v2:172.21.15.37:36506/0 (socket says 172.21.15.37:36506) 2024-04-25T10:29:56.616 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:56.612+0000 7f195ed9d640 1 -- 172.21.15.37:0/4212011056 learned_addr learned my addr 172.21.15.37:0/4212011056 (peer_addr_for_me v2:172.21.15.37:0/0) 2024-04-25T10:29:56.616 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:56.612+0000 7f195e59c640 1 --2- 172.21.15.37:0/4212011056 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f19600d0990 0x7f1960161d40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:29:56.616 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:56.612+0000 7f195dd9b640 1 --2- 172.21.15.37:0/4212011056 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f19600d0da0 0x7f1960147c70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:29:56.616 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:56.613+0000 7f195ed9d640 1 -- 172.21.15.37:0/4212011056 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f19600d0da0 msgr2=0x7f1960147c70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:29:56.616 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:56.613+0000 7f195ed9d640 1 --2- 172.21.15.37:0/4212011056 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f19600d0da0 0x7f1960147c70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:29:56.616 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:56.613+0000 7f195ed9d640 1 -- 172.21.15.37:0/4212011056 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f19600d0990 msgr2=0x7f1960161d40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:29:56.616 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:56.613+0000 7f195ed9d640 1 --2- 172.21.15.37:0/4212011056 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f19600d0990 0x7f1960161d40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:29:56.616 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:56.613+0000 7f195ed9d640 1 -- 172.21.15.37:0/4212011056 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f1960162500 con 0x7f19600d5160 2024-04-25T10:29:56.616 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:56.613+0000 7f195ed9d640 1 --2- 172.21.15.37:0/4212011056 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f19600d5160 0x7f19601f90f0 secure :-1 s=READY pgs=339 cs=0 l=1 rev1=1 crypto rx=0x7f195800b970 tx=0x7f195800be30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:29:56.616 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:56.613+0000 7f194effd640 1 -- 172.21.15.37:0/4212011056 <== mon.0 v2:172.21.15.37:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1958019070 con 0x7f19600d5160 2024-04-25T10:29:56.617 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:56.613+0000 7f194effd640 1 -- 172.21.15.37:0/4212011056 <== mon.0 v2:172.21.15.37:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f195800cb40 con 0x7f19600d5160 2024-04-25T10:29:56.617 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:56.613+0000 7f1964c42640 1 -- 172.21.15.37:0/4212011056 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f19601627e0 con 0x7f19600d5160 2024-04-25T10:29:56.617 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:56.613+0000 7f194effd640 1 -- 172.21.15.37:0/4212011056 <== mon.0 v2:172.21.15.37:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1958014440 con 0x7f19600d5160 2024-04-25T10:29:56.617 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:56.613+0000 7f1964c42640 1 -- 172.21.15.37:0/4212011056 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f1960162da0 con 0x7f19600d5160 2024-04-25T10:29:56.617 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:56.614+0000 7f1964c42640 1 -- 172.21.15.37:0/4212011056 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f196006a140 con 0x7f19600d5160 2024-04-25T10:29:56.618 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:56.615+0000 7f194effd640 1 -- 172.21.15.37:0/4212011056 <== mon.0 v2:172.21.15.37:3300/0 4 ==== mgrmap(e 5) ==== 96782+0+0 (secure 0 0 0) 0x7f195801c070 con 0x7f19600d5160 2024-04-25T10:29:56.618 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:56.615+0000 7f194effd640 1 --2- 172.21.15.37:0/4212011056 >> [v2:172.21.15.203:6832/2418205237,v1:172.21.15.203:6833/2418205237] conn(0x7f19400744e0 0x7f19400769f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:29:56.618 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:56.615+0000 7f194effd640 1 -- 172.21.15.37:0/4212011056 <== mon.0 v2:172.21.15.37:3300/0 5 ==== osd_map(25..25 src has 1..25) ==== 5738+0+0 (secure 0 0 0) 0x7f1958094d70 con 0x7f19600d5160 2024-04-25T10:29:56.618 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:56.615+0000 7f195e59c640 1 --2- 172.21.15.37:0/4212011056 >> [v2:172.21.15.203:6832/2418205237,v1:172.21.15.203:6833/2418205237] conn(0x7f19400744e0 0x7f19400769f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:29:56.618 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:56.616+0000 7f195e59c640 1 --2- 172.21.15.37:0/4212011056 >> [v2:172.21.15.203:6832/2418205237,v1:172.21.15.203:6833/2418205237] conn(0x7f19400744e0 0x7f19400769f0 secure :-1 s=READY pgs=175 cs=0 l=1 rev1=1 crypto rx=0x7f19500079a0 tx=0x7f1950008040 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:29:56.622 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:56.620+0000 7f194effd640 1 -- 172.21.15.37:0/4212011056 <== mon.0 v2:172.21.15.37:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187246 (secure 0 0 0) 0x7f195805ef20 con 0x7f19600d5160 2024-04-25T10:29:56.805 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:56.803+0000 7f1964c42640 1 -- 172.21.15.37:0/4212011056 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) -- 0x7f1960162290 con 0x7f19600d5160 2024-04-25T10:29:56.806 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:56.803+0000 7f194effd640 1 -- 172.21.15.37:0/4212011056 <== mon.0 v2:172.21.15.37:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v25) ==== 93+0+10 (secure 0 0 0) 0x7f1958063220 con 0x7f19600d5160 2024-04-25T10:29:56.806 INFO:teuthology.orchestra.run.smithi037.stdout:pg_num: 1 2024-04-25T10:29:56.808 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:56.805+0000 7f1964c42640 1 -- 172.21.15.37:0/4212011056 >> [v2:172.21.15.203:6832/2418205237,v1:172.21.15.203:6833/2418205237] conn(0x7f19400744e0 msgr2=0x7f19400769f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:29:56.808 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:56.805+0000 7f1964c42640 1 --2- 172.21.15.37:0/4212011056 >> [v2:172.21.15.203:6832/2418205237,v1:172.21.15.203:6833/2418205237] conn(0x7f19400744e0 0x7f19400769f0 secure :-1 s=READY pgs=175 cs=0 l=1 rev1=1 crypto rx=0x7f19500079a0 tx=0x7f1950008040 comp rx=0 tx=0).stop 2024-04-25T10:29:56.808 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:56.805+0000 7f1964c42640 1 -- 172.21.15.37:0/4212011056 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f19600d5160 msgr2=0x7f19601f90f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:29:56.808 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:56.805+0000 7f1964c42640 1 --2- 172.21.15.37:0/4212011056 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f19600d5160 0x7f19601f90f0 secure :-1 s=READY pgs=339 cs=0 l=1 rev1=1 crypto rx=0x7f195800b970 tx=0x7f195800be30 comp rx=0 tx=0).stop 2024-04-25T10:29:56.808 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:56.806+0000 7f1964c42640 1 -- 172.21.15.37:0/4212011056 shutdown_connections 2024-04-25T10:29:56.808 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:56.806+0000 7f1964c42640 1 --2- 172.21.15.37:0/4212011056 >> [v2:172.21.15.203:6832/2418205237,v1:172.21.15.203:6833/2418205237] conn(0x7f19400744e0 0x7f19400769f0 unknown :-1 s=CLOSED pgs=175 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:29:56.808 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:56.806+0000 7f1964c42640 1 --2- 172.21.15.37:0/4212011056 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f19600d5160 0x7f19601f90f0 unknown :-1 s=CLOSED pgs=339 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:29:56.808 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:56.806+0000 7f1964c42640 1 --2- 172.21.15.37:0/4212011056 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f19600d0da0 0x7f1960147c70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:29:56.808 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:56.806+0000 7f1964c42640 1 --2- 172.21.15.37:0/4212011056 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f19600d0990 0x7f1960161d40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:29:56.809 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:56.806+0000 7f1964c42640 1 -- 172.21.15.37:0/4212011056 >> 172.21.15.37:0/4212011056 conn(0x7f19600b7a80 msgr2=0x7f19600b54d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:29:56.809 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:56.806+0000 7f1964c42640 1 -- 172.21.15.37:0/4212011056 shutdown_connections 2024-04-25T10:29:56.809 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:56.806+0000 7f1964c42640 1 -- 172.21.15.37:0/4212011056 wait complete. 2024-04-25T10:29:56.819 DEBUG:teuthology.orchestra.run.smithi037:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_metadata pg_num 2024-04-25T10:29:56.934 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:56.932+0000 7f2a29492640 1 Processor -- start 2024-04-25T10:29:56.935 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:56.932+0000 7f2a29492640 1 -- start start 2024-04-25T10:29:56.936 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:56.934+0000 7f2a29492640 1 --2- >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f2a2415a9f0 0x7f2a2406c540 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:29:56.937 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:56.934+0000 7f2a29492640 1 --2- >> v2:172.21.15.37:3300/0 conn(0x7f2a24161020 0x7f2a240b5860 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:29:56.937 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:56.934+0000 7f2a29492640 1 -- --> v1:172.21.15.203:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f2a24161430 con 0x7f2a2406ca80 2024-04-25T10:29:56.937 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:56.934+0000 7f2a29492640 1 -- --> v2:172.21.15.37:3300/0 -- mon_getmap magic: 0 -- 0x7f2a2406ce90 con 0x7f2a24161020 2024-04-25T10:29:56.937 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:56.934+0000 7f2a29492640 1 -- --> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] -- mon_getmap magic: 0 -- 0x7f2a241616d0 con 0x7f2a2415a9f0 2024-04-25T10:29:56.937 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:56.934+0000 7f2a237fe640 1 --2- >> v2:172.21.15.37:3300/0 conn(0x7f2a24161020 0x7f2a240b5860 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:29:56.937 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:56.934+0000 7f2a237fe640 1 --2- >> v2:172.21.15.37:3300/0 conn(0x7f2a24161020 0x7f2a240b5860 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.37:3300/0 says I am v2:172.21.15.37:36522/0 (socket says 172.21.15.37:36522) 2024-04-25T10:29:56.937 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:56.934+0000 7f2a237fe640 1 -- 172.21.15.37:0/2826456745 learned_addr learned my addr 172.21.15.37:0/2826456745 (peer_addr_for_me v2:172.21.15.37:0/0) 2024-04-25T10:29:56.937 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:56.934+0000 7f2a22ffd640 1 --2- 172.21.15.37:0/2826456745 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f2a2415a9f0 0x7f2a2406c540 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:29:56.937 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:56.934+0000 7f2a237fe640 1 -- 172.21.15.37:0/2826456745 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f2a2415a9f0 msgr2=0x7f2a2406c540 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:29:56.937 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:56.934+0000 7f2a237fe640 1 --2- 172.21.15.37:0/2826456745 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f2a2415a9f0 0x7f2a2406c540 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:29:56.937 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:56.934+0000 7f2a237fe640 1 -- 172.21.15.37:0/2826456745 >> v1:172.21.15.203:6789/0 conn(0x7f2a2406ca80 legacy=0x7f2a24160910 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:29:56.937 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:56.934+0000 7f2a237fe640 1 -- 172.21.15.37:0/2826456745 --> v2:172.21.15.37:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f2a24163e00 con 0x7f2a24161020 2024-04-25T10:29:56.939 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:56.935+0000 7f2a237fe640 1 --2- 172.21.15.37:0/2826456745 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f2a24161020 0x7f2a240b5860 secure :-1 s=READY pgs=340 cs=0 l=1 rev1=1 crypto rx=0x7f2a14009e50 tx=0x7f2a1402a710 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d2d3739f257c62 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:29:56.939 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:56.935+0000 7f2a21ffb640 1 -- 172.21.15.37:0/2826456745 <== mon.0 v2:172.21.15.37:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2a14036070 con 0x7f2a24161020 2024-04-25T10:29:56.939 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:56.935+0000 7f2a21ffb640 1 -- 172.21.15.37:0/2826456745 <== mon.0 v2:172.21.15.37:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f2a14031600 con 0x7f2a24161020 2024-04-25T10:29:56.939 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:56.935+0000 7f2a21ffb640 1 -- 172.21.15.37:0/2826456745 <== mon.0 v2:172.21.15.37:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2a140319d0 con 0x7f2a24161020 2024-04-25T10:29:56.939 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:56.936+0000 7f2a29492640 1 -- 172.21.15.37:0/2826456745 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f2a24161020 msgr2=0x7f2a240b5860 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:29:56.939 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:56.936+0000 7f2a29492640 1 --2- 172.21.15.37:0/2826456745 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f2a24161020 0x7f2a240b5860 secure :-1 s=READY pgs=340 cs=0 l=1 rev1=1 crypto rx=0x7f2a14009e50 tx=0x7f2a1402a710 comp rx=0 tx=0).stop 2024-04-25T10:29:56.939 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:56.936+0000 7f2a29492640 1 -- 172.21.15.37:0/2826456745 shutdown_connections 2024-04-25T10:29:56.939 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:56.936+0000 7f2a29492640 1 --2- 172.21.15.37:0/2826456745 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f2a24161020 0x7f2a240b5860 unknown :-1 s=CLOSED pgs=340 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:29:56.939 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:56.936+0000 7f2a29492640 1 --2- 172.21.15.37:0/2826456745 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f2a2415a9f0 0x7f2a2406c540 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:29:56.939 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:56.936+0000 7f2a29492640 1 -- 172.21.15.37:0/2826456745 >> 172.21.15.37:0/2826456745 conn(0x7f2a240b4560 msgr2=0x7f2a240b4970 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:29:56.939 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:56.936+0000 7f2a29492640 1 -- 172.21.15.37:0/2826456745 shutdown_connections 2024-04-25T10:29:56.939 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:56.936+0000 7f2a29492640 1 -- 172.21.15.37:0/2826456745 wait complete. 2024-04-25T10:29:56.940 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:56.938+0000 7f2a29492640 1 Processor -- start 2024-04-25T10:29:56.940 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:56.938+0000 7f2a29492640 1 -- start start 2024-04-25T10:29:56.942 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:56.939+0000 7f2a29492640 1 --2- >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f2a2406ca80 0x7f2a240cddd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:29:56.943 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:56.939+0000 7f2a29492640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f2a2415a9f0 0x7f2a2415ca30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:29:56.943 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:56.939+0000 7f2a29492640 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f2a24161020 0x7f2a240b2490 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:29:56.943 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:56.939+0000 7f2a29492640 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 -- 0x7f2a241646a0 con 0x7f2a2415a9f0 2024-04-25T10:29:56.943 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:56.939+0000 7f2a29492640 1 -- --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_getmap magic: 0 -- 0x7f2a241643c0 con 0x7f2a24161020 2024-04-25T10:29:56.943 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:56.939+0000 7f2a29492640 1 -- --> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] -- mon_getmap magic: 0 -- 0x7f2a24164bc0 con 0x7f2a2406ca80 2024-04-25T10:29:56.943 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:56.939+0000 7f2a227fc640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f2a2415a9f0 0x7f2a2415ca30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:29:56.943 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:56.939+0000 7f2a227fc640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f2a2415a9f0 0x7f2a2415ca30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.37:3300/0 says I am v2:172.21.15.37:36534/0 (socket says 172.21.15.37:36534) 2024-04-25T10:29:56.943 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:56.939+0000 7f2a227fc640 1 -- 172.21.15.37:0/304878343 learned_addr learned my addr 172.21.15.37:0/304878343 (peer_addr_for_me v2:172.21.15.37:0/0) 2024-04-25T10:29:56.943 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:56.939+0000 7f2a22ffd640 1 --2- 172.21.15.37:0/304878343 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f2a2406ca80 0x7f2a240cddd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:29:56.943 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:56.939+0000 7f2a237fe640 1 --2- 172.21.15.37:0/304878343 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f2a24161020 0x7f2a240b2490 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:29:56.943 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:56.940+0000 7f2a227fc640 1 -- 172.21.15.37:0/304878343 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f2a2406ca80 msgr2=0x7f2a240cddd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:29:56.943 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:56.940+0000 7f2a227fc640 1 --2- 172.21.15.37:0/304878343 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f2a2406ca80 0x7f2a240cddd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:29:56.943 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:56.940+0000 7f2a227fc640 1 -- 172.21.15.37:0/304878343 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f2a24161020 msgr2=0x7f2a240b2490 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:29:56.943 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:56.940+0000 7f2a227fc640 1 --2- 172.21.15.37:0/304878343 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f2a24161020 0x7f2a240b2490 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:29:56.943 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:56.940+0000 7f2a227fc640 1 -- 172.21.15.37:0/304878343 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f2a2415f250 con 0x7f2a2415a9f0 2024-04-25T10:29:56.944 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:56.940+0000 7f2a227fc640 1 --2- 172.21.15.37:0/304878343 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f2a2415a9f0 0x7f2a2415ca30 secure :-1 s=READY pgs=341 cs=0 l=1 rev1=1 crypto rx=0x7f2a0c00c3c0 tx=0x7f2a0c00c880 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:29:56.944 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:56.940+0000 7f2a1b7fe640 1 -- 172.21.15.37:0/304878343 <== mon.0 v2:172.21.15.37:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2a0c0052d0 con 0x7f2a2415a9f0 2024-04-25T10:29:56.944 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:56.940+0000 7f2a1b7fe640 1 -- 172.21.15.37:0/304878343 <== mon.0 v2:172.21.15.37:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f2a0c005490 con 0x7f2a2415a9f0 2024-04-25T10:29:56.944 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:56.940+0000 7f2a1b7fe640 1 -- 172.21.15.37:0/304878343 <== mon.0 v2:172.21.15.37:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2a0c007260 con 0x7f2a2415a9f0 2024-04-25T10:29:56.944 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:56.940+0000 7f2a29492640 1 -- 172.21.15.37:0/304878343 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f2a2415f530 con 0x7f2a2415a9f0 2024-04-25T10:29:56.944 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:56.940+0000 7f2a29492640 1 -- 172.21.15.37:0/304878343 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f2a2415beb0 con 0x7f2a2415a9f0 2024-04-25T10:29:56.945 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:56.941+0000 7f2a29492640 1 -- 172.21.15.37:0/304878343 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f2a240ce390 con 0x7f2a2415a9f0 2024-04-25T10:29:56.945 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:56.942+0000 7f2a1b7fe640 1 -- 172.21.15.37:0/304878343 <== mon.0 v2:172.21.15.37:3300/0 4 ==== mgrmap(e 5) ==== 96782+0+0 (secure 0 0 0) 0x7f2a0c007400 con 0x7f2a2415a9f0 2024-04-25T10:29:56.945 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:56.942+0000 7f2a1b7fe640 1 --2- 172.21.15.37:0/304878343 >> [v2:172.21.15.203:6832/2418205237,v1:172.21.15.203:6833/2418205237] conn(0x7f2a040744b0 0x7f2a040769c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:29:56.945 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:56.942+0000 7f2a1b7fe640 1 -- 172.21.15.37:0/304878343 <== mon.0 v2:172.21.15.37:3300/0 5 ==== osd_map(25..25 src has 1..25) ==== 5738+0+0 (secure 0 0 0) 0x7f2a0c094660 con 0x7f2a2415a9f0 2024-04-25T10:29:56.945 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:56.943+0000 7f2a22ffd640 1 --2- 172.21.15.37:0/304878343 >> [v2:172.21.15.203:6832/2418205237,v1:172.21.15.203:6833/2418205237] conn(0x7f2a040744b0 0x7f2a040769c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:29:56.945 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:56.943+0000 7f2a22ffd640 1 --2- 172.21.15.37:0/304878343 >> [v2:172.21.15.203:6832/2418205237,v1:172.21.15.203:6833/2418205237] conn(0x7f2a040744b0 0x7f2a040769c0 secure :-1 s=READY pgs=176 cs=0 l=1 rev1=1 crypto rx=0x7f2a24073140 tx=0x7f2a1c009190 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:29:56.949 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:56.947+0000 7f2a1b7fe640 1 -- 172.21.15.37:0/304878343 <== mon.0 v2:172.21.15.37:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187246 (secure 0 0 0) 0x7f2a0c05e780 con 0x7f2a2415a9f0 2024-04-25T10:29:57.142 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:57.139+0000 7f2a29492640 1 -- 172.21.15.37:0/304878343 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"} v 0) -- 0x7f2a2406a140 con 0x7f2a2415a9f0 2024-04-25T10:29:57.142 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:57.140+0000 7f2a1b7fe640 1 -- 172.21.15.37:0/304878343 <== mon.0 v2:172.21.15.37:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]=0 v25) ==== 104+0+11 (secure 0 0 0) 0x7f2a0c062a80 con 0x7f2a2415a9f0 2024-04-25T10:29:57.142 INFO:teuthology.orchestra.run.smithi037.stdout:pg_num: 32 2024-04-25T10:29:57.145 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:57.142+0000 7f2a29492640 1 -- 172.21.15.37:0/304878343 >> [v2:172.21.15.203:6832/2418205237,v1:172.21.15.203:6833/2418205237] conn(0x7f2a040744b0 msgr2=0x7f2a040769c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:29:57.145 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:57.142+0000 7f2a29492640 1 --2- 172.21.15.37:0/304878343 >> [v2:172.21.15.203:6832/2418205237,v1:172.21.15.203:6833/2418205237] conn(0x7f2a040744b0 0x7f2a040769c0 secure :-1 s=READY pgs=176 cs=0 l=1 rev1=1 crypto rx=0x7f2a24073140 tx=0x7f2a1c009190 comp rx=0 tx=0).stop 2024-04-25T10:29:57.145 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:57.142+0000 7f2a29492640 1 -- 172.21.15.37:0/304878343 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f2a2415a9f0 msgr2=0x7f2a2415ca30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:29:57.145 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:57.142+0000 7f2a29492640 1 --2- 172.21.15.37:0/304878343 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f2a2415a9f0 0x7f2a2415ca30 secure :-1 s=READY pgs=341 cs=0 l=1 rev1=1 crypto rx=0x7f2a0c00c3c0 tx=0x7f2a0c00c880 comp rx=0 tx=0).stop 2024-04-25T10:29:57.145 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:57.142+0000 7f2a29492640 1 -- 172.21.15.37:0/304878343 shutdown_connections 2024-04-25T10:29:57.145 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:57.142+0000 7f2a29492640 1 --2- 172.21.15.37:0/304878343 >> [v2:172.21.15.203:6832/2418205237,v1:172.21.15.203:6833/2418205237] conn(0x7f2a040744b0 0x7f2a040769c0 unknown :-1 s=CLOSED pgs=176 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:29:57.145 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:57.142+0000 7f2a29492640 1 --2- 172.21.15.37:0/304878343 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f2a24161020 0x7f2a240b2490 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:29:57.145 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:57.142+0000 7f2a29492640 1 --2- 172.21.15.37:0/304878343 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f2a2415a9f0 0x7f2a2415ca30 unknown :-1 s=CLOSED pgs=341 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:29:57.145 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:57.142+0000 7f2a29492640 1 --2- 172.21.15.37:0/304878343 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f2a2406ca80 0x7f2a240cddd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:29:57.145 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:57.142+0000 7f2a29492640 1 -- 172.21.15.37:0/304878343 >> 172.21.15.37:0/304878343 conn(0x7f2a240b4560 msgr2=0x7f2a24161f70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:29:57.145 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:57.142+0000 7f2a29492640 1 -- 172.21.15.37:0/304878343 shutdown_connections 2024-04-25T10:29:57.145 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:57.142+0000 7f2a29492640 1 -- 172.21.15.37:0/304878343 wait complete. 2024-04-25T10:29:57.156 DEBUG:teuthology.orchestra.run.smithi037:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data pg_num 2024-04-25T10:29:57.269 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:57.267+0000 7f70b7e30640 1 Processor -- start 2024-04-25T10:29:57.269 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:57.267+0000 7f70b7e30640 1 -- start start 2024-04-25T10:29:57.273 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:57.269+0000 7f70b7e30640 1 --2- >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f70b00cf570 0x7f70b016ad90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:29:57.273 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:57.269+0000 7f70b7e30640 1 -- --> v1:172.21.15.37:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f70b006e910 con 0x7f70b00b2dc0 2024-04-25T10:29:57.273 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:57.269+0000 7f70b7e30640 1 -- --> v1:172.21.15.203:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f70b00d2e40 con 0x7f70b00d2860 2024-04-25T10:29:57.273 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:57.269+0000 7f70b7e30640 1 -- --> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] -- mon_getmap magic: 0 -- 0x7f70b00b92c0 con 0x7f70b00cf570 2024-04-25T10:29:57.273 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:57.269+0000 7f70b5bcc640 1 --1- >> v1:172.21.15.37:6789/0 conn(0x7f70b00b2dc0 0x7f70b006daf0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.37:6789/0 says I am v1:172.21.15.37:56850/0 (socket says 172.21.15.37:56850) 2024-04-25T10:29:57.273 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:57.269+0000 7f70b5bcc640 1 -- 172.21.15.37:0/2921536300 learned_addr learned my addr 172.21.15.37:0/2921536300 (peer_addr_for_me v1:172.21.15.37:0/0) 2024-04-25T10:29:57.273 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:57.269+0000 7f70b4bca640 1 -- 172.21.15.37:0/2921536300 <== mon.0 v1:172.21.15.37:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 1809546915 0 0) 0x7f70b006e910 con 0x7f70b00b2dc0 2024-04-25T10:29:57.273 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:57.269+0000 7f70b4bca640 1 -- 172.21.15.37:0/2921536300 --> v1:172.21.15.37:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f7098003520 con 0x7f70b00b2dc0 2024-04-25T10:29:57.273 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:57.269+0000 7f70b63cd640 1 --2- 172.21.15.37:0/2921536300 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f70b00cf570 0x7f70b016ad90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:29:57.274 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:57.269+0000 7f70b4bca640 1 -- 172.21.15.37:0/2921536300 <== mon.0 v1:172.21.15.37:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 338134062 0 0) 0x7f7098003520 con 0x7f70b00b2dc0 2024-04-25T10:29:57.274 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:57.269+0000 7f70b4bca640 1 -- 172.21.15.37:0/2921536300 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f70b00cf570 msgr2=0x7f70b016ad90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:29:57.274 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:57.269+0000 7f70b4bca640 1 --2- 172.21.15.37:0/2921536300 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f70b00cf570 0x7f70b016ad90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:29:57.274 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:57.269+0000 7f70b4bca640 1 -- 172.21.15.37:0/2921536300 >> v1:172.21.15.203:6789/0 conn(0x7f70b00d2860 legacy=0x7f70b006e200 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:29:57.274 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:57.269+0000 7f70b4bca640 1 -- 172.21.15.37:0/2921536300 --> v1:172.21.15.37:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f70b00d3120 con 0x7f70b00b2dc0 2024-04-25T10:29:57.274 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:57.269+0000 7f70b4bca640 1 -- 172.21.15.37:0/2921536300 <== mon.0 v1:172.21.15.37:6789/0 3 ==== mon_map magic: 0 ==== 413+0+0 (unknown 1577794600 0 0) 0x7f70a0001b70 con 0x7f70b00b2dc0 2024-04-25T10:29:57.274 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:57.269+0000 7f70b4bca640 1 -- 172.21.15.37:0/2921536300 >> v1:172.21.15.37:6789/0 conn(0x7f70b00b2dc0 legacy=0x7f70b006daf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:29:57.274 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:57.270+0000 7f70b4bca640 1 --2- 172.21.15.37:0/2921536300 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f70b00cf570 0x7f70980039e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:29:57.274 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:57.270+0000 7f70b4bca640 1 --2- 172.21.15.37:0/2921536300 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f7098003f60 0x7f7098004800 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:29:57.274 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:57.270+0000 7f70b4bca640 1 --2- 172.21.15.37:0/2921536300 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f7098004d40 0x7f7098005150 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:29:57.274 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:57.270+0000 7f70b4bca640 1 -- 172.21.15.37:0/2921536300 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 -- 0x7f709800bb50 con 0x7f7098003f60 2024-04-25T10:29:57.274 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:57.270+0000 7f70b4bca640 1 -- 172.21.15.37:0/2921536300 --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_getmap magic: 0 -- 0x7f709800bcd0 con 0x7f7098004d40 2024-04-25T10:29:57.274 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:57.270+0000 7f70b4bca640 1 -- 172.21.15.37:0/2921536300 --> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] -- mon_getmap magic: 0 -- 0x7f709800bed0 con 0x7f70b00cf570 2024-04-25T10:29:57.274 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:57.270+0000 7f70b53cb640 1 --2- 172.21.15.37:0/2921536300 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f7098003f60 0x7f7098004800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:29:57.275 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:57.270+0000 7f70b5bcc640 1 --2- 172.21.15.37:0/2921536300 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f70b00cf570 0x7f70980039e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:29:57.275 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:57.270+0000 7f70b63cd640 1 --2- 172.21.15.37:0/2921536300 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f7098004d40 0x7f7098005150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:29:57.275 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:57.270+0000 7f70b53cb640 1 -- 172.21.15.37:0/2921536300 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f70b00cf570 msgr2=0x7f70980039e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:29:57.275 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:57.270+0000 7f70b53cb640 1 --2- 172.21.15.37:0/2921536300 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f70b00cf570 0x7f70980039e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:29:57.275 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:57.270+0000 7f70b53cb640 1 -- 172.21.15.37:0/2921536300 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f7098004d40 msgr2=0x7f7098005150 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:29:57.275 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:57.270+0000 7f70b53cb640 1 --2- 172.21.15.37:0/2921536300 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f7098004d40 0x7f7098005150 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:29:57.275 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:57.270+0000 7f70b53cb640 1 -- 172.21.15.37:0/2921536300 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f7098003520 con 0x7f7098003f60 2024-04-25T10:29:57.275 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:57.271+0000 7f70b53cb640 1 --2- 172.21.15.37:0/2921536300 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f7098003f60 0x7f7098004800 secure :-1 s=READY pgs=343 cs=0 l=1 rev1=1 crypto rx=0x7f70a4001720 tx=0x7f7098005830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=14770006cf69edab server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:29:57.275 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:57.271+0000 7f70b4bca640 1 -- 172.21.15.37:0/2921536300 <== mon.0 v2:172.21.15.37:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f70a4001370 con 0x7f7098003f60 2024-04-25T10:29:57.275 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:57.271+0000 7f70b4bca640 1 -- 172.21.15.37:0/2921536300 <== mon.0 v2:172.21.15.37:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f70a40052d0 con 0x7f7098003f60 2024-04-25T10:29:57.275 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:57.271+0000 7f70b4bca640 1 -- 172.21.15.37:0/2921536300 <== mon.0 v2:172.21.15.37:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f70a402abc0 con 0x7f7098003f60 2024-04-25T10:29:57.278 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:57.271+0000 7f70b7e30640 1 -- 172.21.15.37:0/2921536300 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f7098003f60 msgr2=0x7f7098004800 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:29:57.278 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:57.271+0000 7f70b7e30640 1 --2- 172.21.15.37:0/2921536300 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f7098003f60 0x7f7098004800 secure :-1 s=READY pgs=343 cs=0 l=1 rev1=1 crypto rx=0x7f70a4001720 tx=0x7f7098005830 comp rx=0 tx=0).stop 2024-04-25T10:29:57.278 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:57.272+0000 7f70b63cd640 1 -- 172.21.15.37:0/2921536300 reap_dead start 2024-04-25T10:29:57.278 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:57.272+0000 7f70b7e30640 1 -- 172.21.15.37:0/2921536300 shutdown_connections 2024-04-25T10:29:57.278 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:57.272+0000 7f70b7e30640 1 -- 172.21.15.37:0/2921536300 >> 172.21.15.37:0/2921536300 conn(0x7f70b00b5440 msgr2=0x7f70b00b5850 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:29:57.278 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:57.272+0000 7f70b7e30640 1 -- 172.21.15.37:0/2921536300 shutdown_connections 2024-04-25T10:29:57.278 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:57.272+0000 7f70b7e30640 1 -- 172.21.15.37:0/2921536300 wait complete. 2024-04-25T10:29:57.278 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:57.273+0000 7f70b7e30640 1 Processor -- start 2024-04-25T10:29:57.278 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:57.273+0000 7f70b7e30640 1 -- start start 2024-04-25T10:29:57.278 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:57.275+0000 7f70b7e30640 1 --2- >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f7098003f60 0x7f70b014be70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:29:57.279 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:57.275+0000 7f70b7e30640 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f70b0160850 0x7f70b00cbf70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:29:57.279 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:57.275+0000 7f70b7e30640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f70b0159d30 0x7f70b00cc4b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:29:57.279 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:57.275+0000 7f70b7e30640 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 -- 0x7f70b00d36e0 con 0x7f70b0159d30 2024-04-25T10:29:57.279 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:57.275+0000 7f70b7e30640 1 -- --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_getmap magic: 0 -- 0x7f70b00ce0d0 con 0x7f70b0160850 2024-04-25T10:29:57.279 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:57.275+0000 7f70b7e30640 1 -- --> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] -- mon_getmap magic: 0 -- 0x7f70b0168d80 con 0x7f7098003f60 2024-04-25T10:29:57.279 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:57.275+0000 7f70b63cd640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f70b0159d30 0x7f70b00cc4b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:29:57.279 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:57.275+0000 7f70b63cd640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f70b0159d30 0x7f70b00cc4b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.37:3300/0 says I am v2:172.21.15.37:36566/0 (socket says 172.21.15.37:36566) 2024-04-25T10:29:57.279 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:57.275+0000 7f70b63cd640 1 -- 172.21.15.37:0/4245034002 learned_addr learned my addr 172.21.15.37:0/4245034002 (peer_addr_for_me v2:172.21.15.37:0/0) 2024-04-25T10:29:57.279 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:57.275+0000 7f70b53cb640 1 --2- 172.21.15.37:0/4245034002 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f70b0160850 0x7f70b00cbf70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:29:57.279 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:57.275+0000 7f70b5bcc640 1 --2- 172.21.15.37:0/4245034002 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f7098003f60 0x7f70b014be70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:29:57.279 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:57.275+0000 7f70b63cd640 1 -- 172.21.15.37:0/4245034002 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f7098003f60 msgr2=0x7f70b014be70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:29:57.279 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:57.275+0000 7f70b63cd640 1 --2- 172.21.15.37:0/4245034002 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f7098003f60 0x7f70b014be70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:29:57.279 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:57.275+0000 7f70b63cd640 1 -- 172.21.15.37:0/4245034002 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f70b0160850 msgr2=0x7f70b00cbf70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:29:57.279 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:57.275+0000 7f70b63cd640 1 --2- 172.21.15.37:0/4245034002 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f70b0160850 0x7f70b00cbf70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:29:57.280 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:57.275+0000 7f70b63cd640 1 -- 172.21.15.37:0/4245034002 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f70b0161030 con 0x7f70b0159d30 2024-04-25T10:29:57.280 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:57.275+0000 7f70b63cd640 1 --2- 172.21.15.37:0/4245034002 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f70b0159d30 0x7f70b00cc4b0 secure :-1 s=READY pgs=344 cs=0 l=1 rev1=1 crypto rx=0x7f70b0074120 tx=0x7f70a8011920 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:29:57.280 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:57.275+0000 7f70ae7fc640 1 -- 172.21.15.37:0/4245034002 <== mon.0 v2:172.21.15.37:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f70a8003610 con 0x7f70b0159d30 2024-04-25T10:29:57.280 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:57.276+0000 7f70ae7fc640 1 -- 172.21.15.37:0/4245034002 <== mon.0 v2:172.21.15.37:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f70a801b070 con 0x7f70b0159d30 2024-04-25T10:29:57.280 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:57.276+0000 7f70b7e30640 1 -- 172.21.15.37:0/4245034002 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f70b0161310 con 0x7f70b0159d30 2024-04-25T10:29:57.280 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:57.276+0000 7f70ae7fc640 1 -- 172.21.15.37:0/4245034002 <== mon.0 v2:172.21.15.37:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f70a8003a30 con 0x7f70b0159d30 2024-04-25T10:29:57.280 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:57.276+0000 7f70b7e30640 1 -- 172.21.15.37:0/4245034002 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f70b01618d0 con 0x7f70b0159d30 2024-04-25T10:29:57.281 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:57.277+0000 7f70b7e30640 1 -- 172.21.15.37:0/4245034002 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f70b006a140 con 0x7f70b0159d30 2024-04-25T10:29:57.281 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:57.277+0000 7f70ae7fc640 1 -- 172.21.15.37:0/4245034002 <== mon.0 v2:172.21.15.37:3300/0 4 ==== mgrmap(e 5) ==== 96782+0+0 (secure 0 0 0) 0x7f70a8003bd0 con 0x7f70b0159d30 2024-04-25T10:29:57.281 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:57.278+0000 7f70ae7fc640 1 --2- 172.21.15.37:0/4245034002 >> [v2:172.21.15.203:6832/2418205237,v1:172.21.15.203:6833/2418205237] conn(0x7f709c0744b0 0x7f709c0769c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:29:57.281 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:57.278+0000 7f70ae7fc640 1 -- 172.21.15.37:0/4245034002 <== mon.0 v2:172.21.15.37:3300/0 5 ==== osd_map(25..25 src has 1..25) ==== 5738+0+0 (secure 0 0 0) 0x7f70a80953d0 con 0x7f70b0159d30 2024-04-25T10:29:57.281 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:57.278+0000 7f70b5bcc640 1 --2- 172.21.15.37:0/4245034002 >> [v2:172.21.15.203:6832/2418205237,v1:172.21.15.203:6833/2418205237] conn(0x7f709c0744b0 0x7f709c0769c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:29:57.281 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:57.279+0000 7f70b5bcc640 1 --2- 172.21.15.37:0/4245034002 >> [v2:172.21.15.203:6832/2418205237,v1:172.21.15.203:6833/2418205237] conn(0x7f709c0744b0 0x7f709c0769c0 secure :-1 s=READY pgs=177 cs=0 l=1 rev1=1 crypto rx=0x7f70a0001550 tx=0x7f70a0009040 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:29:57.284 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:57.283+0000 7f70ae7fc640 1 -- 172.21.15.37:0/4245034002 <== mon.0 v2:172.21.15.37:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187246 (secure 0 0 0) 0x7f70a8025020 con 0x7f70b0159d30 2024-04-25T10:29:57.470 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:57.468+0000 7f70b7e30640 1 -- 172.21.15.37:0/4245034002 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"} v 0) -- 0x7f70b00d0f80 con 0x7f70b0159d30 2024-04-25T10:29:57.470 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:57.468+0000 7f70ae7fc640 1 -- 172.21.15.37:0/4245034002 <== mon.0 v2:172.21.15.37:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]=0 v25) ==== 100+0+11 (secure 0 0 0) 0x7f70a805fb40 con 0x7f70b0159d30 2024-04-25T10:29:57.471 INFO:teuthology.orchestra.run.smithi037.stdout:pg_num: 64 2024-04-25T10:29:57.472 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:57.470+0000 7f70b7e30640 1 -- 172.21.15.37:0/4245034002 >> [v2:172.21.15.203:6832/2418205237,v1:172.21.15.203:6833/2418205237] conn(0x7f709c0744b0 msgr2=0x7f709c0769c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:29:57.473 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:57.470+0000 7f70b7e30640 1 --2- 172.21.15.37:0/4245034002 >> [v2:172.21.15.203:6832/2418205237,v1:172.21.15.203:6833/2418205237] conn(0x7f709c0744b0 0x7f709c0769c0 secure :-1 s=READY pgs=177 cs=0 l=1 rev1=1 crypto rx=0x7f70a0001550 tx=0x7f70a0009040 comp rx=0 tx=0).stop 2024-04-25T10:29:57.473 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:57.470+0000 7f70b7e30640 1 -- 172.21.15.37:0/4245034002 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f70b0159d30 msgr2=0x7f70b00cc4b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:29:57.473 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:57.470+0000 7f70b7e30640 1 --2- 172.21.15.37:0/4245034002 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f70b0159d30 0x7f70b00cc4b0 secure :-1 s=READY pgs=344 cs=0 l=1 rev1=1 crypto rx=0x7f70b0074120 tx=0x7f70a8011920 comp rx=0 tx=0).stop 2024-04-25T10:29:57.473 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:57.470+0000 7f70b7e30640 1 -- 172.21.15.37:0/4245034002 shutdown_connections 2024-04-25T10:29:57.473 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:57.470+0000 7f70b7e30640 1 --2- 172.21.15.37:0/4245034002 >> [v2:172.21.15.203:6832/2418205237,v1:172.21.15.203:6833/2418205237] conn(0x7f709c0744b0 0x7f709c0769c0 unknown :-1 s=CLOSED pgs=177 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:29:57.473 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:57.470+0000 7f70b7e30640 1 --2- 172.21.15.37:0/4245034002 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f70b0159d30 0x7f70b00cc4b0 unknown :-1 s=CLOSED pgs=344 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:29:57.473 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:57.470+0000 7f70b7e30640 1 --2- 172.21.15.37:0/4245034002 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f70b0160850 0x7f70b00cbf70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:29:57.473 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:57.470+0000 7f70b7e30640 1 --2- 172.21.15.37:0/4245034002 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f7098003f60 0x7f70b014be70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:29:57.473 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:57.470+0000 7f70b7e30640 1 -- 172.21.15.37:0/4245034002 >> 172.21.15.37:0/4245034002 conn(0x7f70b00b5440 msgr2=0x7f70b006e500 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:29:57.473 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:57.470+0000 7f70b7e30640 1 -- 172.21.15.37:0/4245034002 shutdown_connections 2024-04-25T10:29:57.473 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:57.470+0000 7f70b7e30640 1 -- 172.21.15.37:0/4245034002 wait complete. 2024-04-25T10:29:57.484 DEBUG:teuthology.orchestra.run.smithi037:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd blocklist ls 2024-04-25T10:29:57.595 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:57.593+0000 7fae56757640 1 Processor -- start 2024-04-25T10:29:57.595 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:57.593+0000 7fae56757640 1 -- start start 2024-04-25T10:29:57.599 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:57.594+0000 7fae56757640 1 --2- >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7fae5008cc90 0x7fae5006e0a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:29:57.599 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:57.594+0000 7fae56757640 1 --2- >> v2:172.21.15.203:3300/0 conn(0x7fae5006e9f0 0x7fae50164f00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:29:57.599 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:57.594+0000 7fae56757640 1 -- --> v1:172.21.15.37:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fae500d2ed0 con 0x7fae5006e5e0 2024-04-25T10:29:57.599 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:57.594+0000 7fae56757640 1 -- --> v2:172.21.15.203:3300/0 -- mon_getmap magic: 0 -- 0x7fae5006ee00 con 0x7fae5006e9f0 2024-04-25T10:29:57.599 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:57.594+0000 7fae56757640 1 -- --> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] -- mon_getmap magic: 0 -- 0x7fae500d3200 con 0x7fae5008cc90 2024-04-25T10:29:57.599 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:57.594+0000 7fae4f7fe640 1 --1- >> v1:172.21.15.37:6789/0 conn(0x7fae5006e5e0 0x7fae501647f0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.37:6789/0 says I am v1:172.21.15.37:56856/0 (socket says 172.21.15.37:56856) 2024-04-25T10:29:57.599 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:57.594+0000 7fae4f7fe640 1 -- 172.21.15.37:0/3342531430 learned_addr learned my addr 172.21.15.37:0/3342531430 (peer_addr_for_me v1:172.21.15.37:0/0) 2024-04-25T10:29:57.599 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:57.595+0000 7fae4ffff640 1 --2- 172.21.15.37:0/3342531430 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7fae5008cc90 0x7fae5006e0a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:29:57.599 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:57.595+0000 7fae54cf4640 1 --2- 172.21.15.37:0/3342531430 >> v2:172.21.15.203:3300/0 conn(0x7fae5006e9f0 0x7fae50164f00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:29:57.599 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:57.595+0000 7fae4effd640 1 -- 172.21.15.37:0/3342531430 <== mon.0 v1:172.21.15.37:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 3867481089 0 0) 0x7fae500d2ed0 con 0x7fae5006e5e0 2024-04-25T10:29:57.599 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:57.595+0000 7fae4effd640 1 -- 172.21.15.37:0/3342531430 --> v1:172.21.15.37:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7fae38003520 con 0x7fae5006e5e0 2024-04-25T10:29:57.599 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:57.595+0000 7fae4effd640 1 -- 172.21.15.37:0/3342531430 <== mon.0 v1:172.21.15.37:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 3026591899 0 0) 0x7fae38003520 con 0x7fae5006e5e0 2024-04-25T10:29:57.600 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:57.595+0000 7fae4effd640 1 -- 172.21.15.37:0/3342531430 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7fae5008cc90 msgr2=0x7fae5006e0a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:29:57.600 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:57.595+0000 7fae4effd640 1 --2- 172.21.15.37:0/3342531430 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7fae5008cc90 0x7fae5006e0a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:29:57.600 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:57.595+0000 7fae4effd640 1 -- 172.21.15.37:0/3342531430 >> v2:172.21.15.203:3300/0 conn(0x7fae5006e9f0 msgr2=0x7fae50164f00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:29:57.600 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:57.595+0000 7fae4effd640 1 --2- 172.21.15.37:0/3342531430 >> v2:172.21.15.203:3300/0 conn(0x7fae5006e9f0 0x7fae50164f00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:29:57.600 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:57.595+0000 7fae4effd640 1 -- 172.21.15.37:0/3342531430 --> v1:172.21.15.37:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fae5009cd40 con 0x7fae5006e5e0 2024-04-25T10:29:57.600 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:57.595+0000 7fae4effd640 1 -- 172.21.15.37:0/3342531430 <== mon.0 v1:172.21.15.37:6789/0 3 ==== mon_map magic: 0 ==== 413+0+0 (unknown 1577794600 0 0) 0x7fae44001670 con 0x7fae5006e5e0 2024-04-25T10:29:57.600 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:57.595+0000 7fae4effd640 1 -- 172.21.15.37:0/3342531430 >> v1:172.21.15.37:6789/0 conn(0x7fae5006e5e0 legacy=0x7fae501647f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:29:57.600 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:57.595+0000 7fae4effd640 1 --2- 172.21.15.37:0/3342531430 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7fae380039b0 0x7fae38003dc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:29:57.600 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:57.595+0000 7fae4effd640 1 --2- 172.21.15.37:0/3342531430 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fae38004340 0x7fae38004860 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:29:57.600 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:57.595+0000 7fae4effd640 1 --2- 172.21.15.37:0/3342531430 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fae38004ed0 0x7fae3800b9c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:29:57.600 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:57.595+0000 7fae4effd640 1 -- 172.21.15.37:0/3342531430 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 -- 0x7fae5006ee00 con 0x7fae38004ed0 2024-04-25T10:29:57.600 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:57.595+0000 7fae4effd640 1 -- 172.21.15.37:0/3342531430 --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_getmap magic: 0 -- 0x7fae500d3200 con 0x7fae38004340 2024-04-25T10:29:57.600 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:57.595+0000 7fae4effd640 1 -- 172.21.15.37:0/3342531430 --> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] -- mon_getmap magic: 0 -- 0x7fae3800bf00 con 0x7fae380039b0 2024-04-25T10:29:57.600 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:57.595+0000 7fae54cf4640 1 --2- 172.21.15.37:0/3342531430 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fae38004ed0 0x7fae3800b9c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:29:57.601 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:57.595+0000 7fae4ffff640 1 --2- 172.21.15.37:0/3342531430 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7fae380039b0 0x7fae38003dc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:29:57.601 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:57.595+0000 7fae4f7fe640 1 --2- 172.21.15.37:0/3342531430 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fae38004340 0x7fae38004860 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:29:57.601 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:57.596+0000 7fae54cf4640 1 -- 172.21.15.37:0/3342531430 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7fae380039b0 msgr2=0x7fae38003dc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:29:57.601 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:57.596+0000 7fae54cf4640 1 --2- 172.21.15.37:0/3342531430 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7fae380039b0 0x7fae38003dc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:29:57.601 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:57.596+0000 7fae54cf4640 1 -- 172.21.15.37:0/3342531430 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fae38004340 msgr2=0x7fae38004860 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:29:57.601 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:57.596+0000 7fae54cf4640 1 --2- 172.21.15.37:0/3342531430 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fae38004340 0x7fae38004860 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:29:57.601 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:57.596+0000 7fae54cf4640 1 -- 172.21.15.37:0/3342531430 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fae38003520 con 0x7fae38004ed0 2024-04-25T10:29:57.601 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:57.596+0000 7fae4f7fe640 1 --2- 172.21.15.37:0/3342531430 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fae38004340 0x7fae38004860 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T10:29:57.601 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:57.596+0000 7fae54cf4640 1 --2- 172.21.15.37:0/3342531430 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fae38004ed0 0x7fae3800b9c0 secure :-1 s=READY pgs=346 cs=0 l=1 rev1=1 crypto rx=0x7fae48009f90 tx=0x7fae38005830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f0759fab32b87137 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:29:57.601 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:57.596+0000 7fae4effd640 1 -- 172.21.15.37:0/3342531430 <== mon.0 v2:172.21.15.37:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fae48009c10 con 0x7fae38004ed0 2024-04-25T10:29:57.601 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:57.596+0000 7fae4effd640 1 -- 172.21.15.37:0/3342531430 <== mon.0 v2:172.21.15.37:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fae48036070 con 0x7fae38004ed0 2024-04-25T10:29:57.601 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:57.596+0000 7fae4effd640 1 -- 172.21.15.37:0/3342531430 <== mon.0 v2:172.21.15.37:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fae48031880 con 0x7fae38004ed0 2024-04-25T10:29:57.604 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:57.597+0000 7fae56757640 1 -- 172.21.15.37:0/3342531430 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fae38004ed0 msgr2=0x7fae3800b9c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:29:57.604 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:57.597+0000 7fae56757640 1 --2- 172.21.15.37:0/3342531430 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fae38004ed0 0x7fae3800b9c0 secure :-1 s=READY pgs=346 cs=0 l=1 rev1=1 crypto rx=0x7fae48009f90 tx=0x7fae38005830 comp rx=0 tx=0).stop 2024-04-25T10:29:57.604 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:57.597+0000 7fae54cf4640 1 -- 172.21.15.37:0/3342531430 reap_dead start 2024-04-25T10:29:57.604 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:57.597+0000 7fae56757640 1 -- 172.21.15.37:0/3342531430 shutdown_connections 2024-04-25T10:29:57.604 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:57.597+0000 7fae56757640 1 -- 172.21.15.37:0/3342531430 >> 172.21.15.37:0/3342531430 conn(0x7fae500d03d0 msgr2=0x7fae500d07e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:29:57.604 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:57.597+0000 7fae56757640 1 -- 172.21.15.37:0/3342531430 shutdown_connections 2024-04-25T10:29:57.604 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:57.597+0000 7fae56757640 1 -- 172.21.15.37:0/3342531430 wait complete. 2024-04-25T10:29:57.604 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:57.599+0000 7fae56757640 1 Processor -- start 2024-04-25T10:29:57.604 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:57.599+0000 7fae56757640 1 -- start start 2024-04-25T10:29:57.604 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:57.600+0000 7fae56757640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fae500c8230 0x7fae50159cb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:29:57.604 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:57.600+0000 7fae56757640 1 --2- >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7fae50159040 0x7fae500c8f00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:29:57.604 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:57.600+0000 7fae56757640 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fae50159450 0x7fae500ccc50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:29:57.605 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:57.600+0000 7fae56757640 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 -- 0x7fae5006da50 con 0x7fae500c8230 2024-04-25T10:29:57.605 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:57.600+0000 7fae56757640 1 -- --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_getmap magic: 0 -- 0x7fae5009d000 con 0x7fae50159450 2024-04-25T10:29:57.605 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:57.600+0000 7fae56757640 1 -- --> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] -- mon_getmap magic: 0 -- 0x7fae500d3690 con 0x7fae50159040 2024-04-25T10:29:57.605 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:57.600+0000 7fae4ffff640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fae500c8230 0x7fae50159cb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:29:57.605 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:57.600+0000 7fae4ffff640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fae500c8230 0x7fae50159cb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.37:3300/0 says I am v2:172.21.15.37:36592/0 (socket says 172.21.15.37:36592) 2024-04-25T10:29:57.605 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:57.600+0000 7fae4ffff640 1 -- 172.21.15.37:0/2340228611 learned_addr learned my addr 172.21.15.37:0/2340228611 (peer_addr_for_me v2:172.21.15.37:0/0) 2024-04-25T10:29:57.605 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:57.601+0000 7fae54cf4640 1 --2- 172.21.15.37:0/2340228611 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fae50159450 0x7fae500ccc50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:29:57.605 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:57.601+0000 7fae4f7fe640 1 --2- 172.21.15.37:0/2340228611 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7fae50159040 0x7fae500c8f00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:29:57.605 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:57.601+0000 7fae4ffff640 1 -- 172.21.15.37:0/2340228611 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7fae50159040 msgr2=0x7fae500c8f00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:29:57.605 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:57.601+0000 7fae4ffff640 1 --2- 172.21.15.37:0/2340228611 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7fae50159040 0x7fae500c8f00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:29:57.605 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:57.601+0000 7fae4ffff640 1 -- 172.21.15.37:0/2340228611 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fae50159450 msgr2=0x7fae500ccc50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:29:57.605 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:57.601+0000 7fae4ffff640 1 --2- 172.21.15.37:0/2340228611 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fae50159450 0x7fae500ccc50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:29:57.605 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:57.601+0000 7fae4ffff640 1 -- 172.21.15.37:0/2340228611 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fae500cab80 con 0x7fae500c8230 2024-04-25T10:29:57.605 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:57.601+0000 7fae4f7fe640 1 --2- 172.21.15.37:0/2340228611 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7fae50159040 0x7fae500c8f00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T10:29:57.605 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:57.601+0000 7fae54cf4640 1 --2- 172.21.15.37:0/2340228611 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fae50159450 0x7fae500ccc50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T10:29:57.606 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:57.601+0000 7fae4ffff640 1 --2- 172.21.15.37:0/2340228611 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fae500c8230 0x7fae50159cb0 secure :-1 s=READY pgs=347 cs=0 l=1 rev1=1 crypto rx=0x7fae400118e0 tx=0x7fae40011da0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:29:57.606 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:57.601+0000 7fae4cff9640 1 -- 172.21.15.37:0/2340228611 <== mon.0 v2:172.21.15.37:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fae4000faf0 con 0x7fae500c8230 2024-04-25T10:29:57.606 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:57.601+0000 7fae4cff9640 1 -- 172.21.15.37:0/2340228611 <== mon.0 v2:172.21.15.37:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fae4000fc90 con 0x7fae500c8230 2024-04-25T10:29:57.606 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:57.601+0000 7fae56757640 1 -- 172.21.15.37:0/2340228611 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fae500cae60 con 0x7fae500c8230 2024-04-25T10:29:57.606 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:57.601+0000 7fae4cff9640 1 -- 172.21.15.37:0/2340228611 <== mon.0 v2:172.21.15.37:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fae40014440 con 0x7fae500c8230 2024-04-25T10:29:57.606 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:57.601+0000 7fae56757640 1 -- 172.21.15.37:0/2340228611 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fae5015a1f0 con 0x7fae500c8230 2024-04-25T10:29:57.607 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:57.602+0000 7fae56757640 1 -- 172.21.15.37:0/2340228611 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fae5006a140 con 0x7fae500c8230 2024-04-25T10:29:57.607 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:57.603+0000 7fae4cff9640 1 -- 172.21.15.37:0/2340228611 <== mon.0 v2:172.21.15.37:3300/0 4 ==== mgrmap(e 5) ==== 96782+0+0 (secure 0 0 0) 0x7fae40012040 con 0x7fae500c8230 2024-04-25T10:29:57.607 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:57.603+0000 7fae4cff9640 1 --2- 172.21.15.37:0/2340228611 >> [v2:172.21.15.203:6832/2418205237,v1:172.21.15.203:6833/2418205237] conn(0x7fae3c0744b0 0x7fae3c0769c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:29:57.607 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:57.604+0000 7fae4cff9640 1 -- 172.21.15.37:0/2340228611 <== mon.0 v2:172.21.15.37:3300/0 5 ==== osd_map(25..25 src has 1..25) ==== 5738+0+0 (secure 0 0 0) 0x7fae40095950 con 0x7fae500c8230 2024-04-25T10:29:57.607 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:57.604+0000 7fae4f7fe640 1 --2- 172.21.15.37:0/2340228611 >> [v2:172.21.15.203:6832/2418205237,v1:172.21.15.203:6833/2418205237] conn(0x7fae3c0744b0 0x7fae3c0769c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:29:57.607 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:57.604+0000 7fae4f7fe640 1 --2- 172.21.15.37:0/2340228611 >> [v2:172.21.15.203:6832/2418205237,v1:172.21.15.203:6833/2418205237] conn(0x7fae3c0744b0 0x7fae3c0769c0 secure :-1 s=READY pgs=178 cs=0 l=1 rev1=1 crypto rx=0x7fae44001c50 tx=0x7fae44001bc0 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:29:57.610 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:57.608+0000 7fae4cff9640 1 -- 172.21.15.37:0/2340228611 <== mon.0 v2:172.21.15.37:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187246 (secure 0 0 0) 0x7fae40019030 con 0x7fae500c8230 2024-04-25T10:29:57.808 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:57.806+0000 7fae56757640 1 -- 172.21.15.37:0/2340228611 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_command({"prefix": "osd blocklist ls"} v 0) -- 0x7fae500b9480 con 0x7fae500c8230 2024-04-25T10:29:57.809 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:57.806+0000 7fae4cff9640 1 -- 172.21.15.37:0/2340228611 <== mon.0 v2:172.21.15.37:3300/0 7 ==== mon_command_ack([{"prefix": "osd blocklist ls"}]=0 listed 0 entries v25) ==== 80+0+0 (secure 0 0 0) 0x7fae40060150 con 0x7fae500c8230 2024-04-25T10:29:57.809 INFO:teuthology.orchestra.run.smithi037.stderr:listed 0 entries 2024-04-25T10:29:57.810 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:57.808+0000 7fae56757640 1 -- 172.21.15.37:0/2340228611 >> [v2:172.21.15.203:6832/2418205237,v1:172.21.15.203:6833/2418205237] conn(0x7fae3c0744b0 msgr2=0x7fae3c0769c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:29:57.811 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:57.808+0000 7fae56757640 1 --2- 172.21.15.37:0/2340228611 >> [v2:172.21.15.203:6832/2418205237,v1:172.21.15.203:6833/2418205237] conn(0x7fae3c0744b0 0x7fae3c0769c0 secure :-1 s=READY pgs=178 cs=0 l=1 rev1=1 crypto rx=0x7fae44001c50 tx=0x7fae44001bc0 comp rx=0 tx=0).stop 2024-04-25T10:29:57.811 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:57.808+0000 7fae56757640 1 -- 172.21.15.37:0/2340228611 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fae500c8230 msgr2=0x7fae50159cb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:29:57.811 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:57.808+0000 7fae56757640 1 --2- 172.21.15.37:0/2340228611 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fae500c8230 0x7fae50159cb0 secure :-1 s=READY pgs=347 cs=0 l=1 rev1=1 crypto rx=0x7fae400118e0 tx=0x7fae40011da0 comp rx=0 tx=0).stop 2024-04-25T10:29:57.811 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:57.808+0000 7fae56757640 1 -- 172.21.15.37:0/2340228611 shutdown_connections 2024-04-25T10:29:57.811 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:57.808+0000 7fae56757640 1 --2- 172.21.15.37:0/2340228611 >> [v2:172.21.15.203:6832/2418205237,v1:172.21.15.203:6833/2418205237] conn(0x7fae3c0744b0 0x7fae3c0769c0 unknown :-1 s=CLOSED pgs=178 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:29:57.811 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:57.808+0000 7fae56757640 1 --2- 172.21.15.37:0/2340228611 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fae50159450 0x7fae500ccc50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:29:57.811 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:57.808+0000 7fae56757640 1 --2- 172.21.15.37:0/2340228611 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7fae50159040 0x7fae500c8f00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:29:57.811 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:57.808+0000 7fae56757640 1 --2- 172.21.15.37:0/2340228611 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fae500c8230 0x7fae50159cb0 unknown :-1 s=CLOSED pgs=347 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:29:57.811 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:57.808+0000 7fae56757640 1 -- 172.21.15.37:0/2340228611 >> 172.21.15.37:0/2340228611 conn(0x7fae500d03d0 msgr2=0x7fae50162c00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:29:57.811 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:57.808+0000 7fae56757640 1 -- 172.21.15.37:0/2340228611 shutdown_connections 2024-04-25T10:29:57.811 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:57.808+0000 7fae56757640 1 -- 172.21.15.37:0/2340228611 wait complete. 2024-04-25T10:29:57.821 INFO:tasks.cephfs.fuse_mount:Running fusermount -u on ubuntu@smithi203.front.sepia.ceph.com... 2024-04-25T10:29:57.821 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T10:29:57.822 DEBUG:teuthology.orchestra.run.smithi203:> sudo fusermount -u /home/ubuntu/cephtest/mnt.3 2024-04-25T10:29:57.851 INFO:teuthology.orchestra.run:waiting for 300 2024-04-25T10:29:57.851 INFO:tasks.cephfs.mount:Cleaning up mount ubuntu@smithi203.front.sepia.ceph.com 2024-04-25T10:29:57.851 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T10:29:57.851 DEBUG:teuthology.orchestra.run.smithi203:> (cd /home/ubuntu/cephtest && exec rmdir -- /home/ubuntu/cephtest/mnt.3) 2024-04-25T10:29:57.909 INFO:tasks.cephfs.cephfs_test_case:[, , , ] 2024-04-25T10:29:57.909 DEBUG:teuthology.orchestra.run.smithi037:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-04-25T10:29:58.021 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:58.019+0000 7fc09ddc0640 1 Processor -- start 2024-04-25T10:29:58.021 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:58.019+0000 7fc09ddc0640 1 -- start start 2024-04-25T10:29:58.023 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:58.020+0000 7fc09ddc0640 1 --2- >> v2:172.21.15.37:3300/0 conn(0x7fc0980b7600 0x7fc0980b7a10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:29:58.023 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:58.020+0000 7fc09ddc0640 1 --2- >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7fc0980d4560 0x7fc0980d4970 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:29:58.023 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:58.020+0000 7fc09ddc0640 1 -- --> v1:172.21.15.203:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fc0980b7f50 con 0x7fc0980b4bb0 2024-04-25T10:29:58.023 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:58.020+0000 7fc09ddc0640 1 -- --> v2:172.21.15.37:3300/0 -- mon_getmap magic: 0 -- 0x7fc098161190 con 0x7fc0980b7600 2024-04-25T10:29:58.023 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:58.020+0000 7fc09ddc0640 1 -- --> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] -- mon_getmap magic: 0 -- 0x7fc098161470 con 0x7fc0980d4560 2024-04-25T10:29:58.023 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:58.020+0000 7fc096ffd640 1 --2- >> v2:172.21.15.37:3300/0 conn(0x7fc0980b7600 0x7fc0980b7a10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:29:58.023 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:58.020+0000 7fc096ffd640 1 --2- >> v2:172.21.15.37:3300/0 conn(0x7fc0980b7600 0x7fc0980b7a10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.37:3300/0 says I am v2:172.21.15.37:36602/0 (socket says 172.21.15.37:36602) 2024-04-25T10:29:58.023 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:58.020+0000 7fc096ffd640 1 -- 172.21.15.37:0/2465644157 learned_addr learned my addr 172.21.15.37:0/2465644157 (peer_addr_for_me v2:172.21.15.37:0/0) 2024-04-25T10:29:58.023 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:58.020+0000 7fc097fff640 1 --2- 172.21.15.37:0/2465644157 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7fc0980d4560 0x7fc0980d4970 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:29:58.023 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:58.021+0000 7fc096ffd640 1 -- 172.21.15.37:0/2465644157 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7fc0980d4560 msgr2=0x7fc0980d4970 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:29:58.023 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:58.021+0000 7fc096ffd640 1 --2- 172.21.15.37:0/2465644157 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7fc0980d4560 0x7fc0980d4970 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:29:58.023 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:58.021+0000 7fc096ffd640 1 -- 172.21.15.37:0/2465644157 >> v1:172.21.15.203:6789/0 conn(0x7fc0980b4bb0 legacy=0x7fc0981646b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:29:58.024 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:58.021+0000 7fc096ffd640 1 -- 172.21.15.37:0/2465644157 --> v2:172.21.15.37:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fc098163eb0 con 0x7fc0980b7600 2024-04-25T10:29:58.024 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:58.021+0000 7fc097fff640 1 --2- 172.21.15.37:0/2465644157 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7fc0980d4560 0x7fc0980d4970 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T10:29:58.025 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:58.021+0000 7fc096ffd640 1 --2- 172.21.15.37:0/2465644157 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fc0980b7600 0x7fc0980b7a10 secure :-1 s=READY pgs=348 cs=0 l=1 rev1=1 crypto rx=0x7fc08c009870 tx=0x7fc08c02a210 comp rx=0 tx=0).ready entity=mon.0 client_cookie=bea6ade5e72dc73c server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:29:58.025 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:58.021+0000 7fc0967fc640 1 -- 172.21.15.37:0/2465644157 <== mon.0 v2:172.21.15.37:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc08c037070 con 0x7fc0980b7600 2024-04-25T10:29:58.025 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:58.022+0000 7fc0967fc640 1 -- 172.21.15.37:0/2465644157 <== mon.0 v2:172.21.15.37:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fc08c032440 con 0x7fc0980b7600 2024-04-25T10:29:58.025 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:58.022+0000 7fc0967fc640 1 -- 172.21.15.37:0/2465644157 <== mon.0 v2:172.21.15.37:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc08c032810 con 0x7fc0980b7600 2024-04-25T10:29:58.025 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:58.022+0000 7fc09ddc0640 1 -- 172.21.15.37:0/2465644157 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fc0980b7600 msgr2=0x7fc0980b7a10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:29:58.025 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:58.022+0000 7fc09ddc0640 1 --2- 172.21.15.37:0/2465644157 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fc0980b7600 0x7fc0980b7a10 secure :-1 s=READY pgs=348 cs=0 l=1 rev1=1 crypto rx=0x7fc08c009870 tx=0x7fc08c02a210 comp rx=0 tx=0).stop 2024-04-25T10:29:58.025 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:58.022+0000 7fc09ddc0640 1 -- 172.21.15.37:0/2465644157 shutdown_connections 2024-04-25T10:29:58.025 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:58.022+0000 7fc09ddc0640 1 --2- 172.21.15.37:0/2465644157 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7fc0980d4560 0x7fc0980d4970 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:29:58.026 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:58.022+0000 7fc09ddc0640 1 --2- 172.21.15.37:0/2465644157 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fc0980b7600 0x7fc0980b7a10 unknown :-1 s=CLOSED pgs=348 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:29:58.026 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:58.022+0000 7fc09ddc0640 1 -- 172.21.15.37:0/2465644157 >> 172.21.15.37:0/2465644157 conn(0x7fc0980d27e0 msgr2=0x7fc0980d2bf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:29:58.026 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:58.022+0000 7fc09ddc0640 1 -- 172.21.15.37:0/2465644157 shutdown_connections 2024-04-25T10:29:58.026 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:58.022+0000 7fc09ddc0640 1 -- 172.21.15.37:0/2465644157 wait complete. 2024-04-25T10:29:58.026 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:58.024+0000 7fc09ddc0640 1 Processor -- start 2024-04-25T10:29:58.026 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:58.024+0000 7fc09ddc0640 1 -- start start 2024-04-25T10:29:58.029 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:58.025+0000 7fc09ddc0640 1 --2- >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7fc0980b4bb0 0x7fc0981f6400 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:29:58.029 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:58.025+0000 7fc09ddc0640 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fc0980b7600 0x7fc09815fad0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:29:58.030 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:58.025+0000 7fc09ddc0640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fc0980d4560 0x7fc0980cb170 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:29:58.030 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:58.025+0000 7fc09ddc0640 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 -- 0x7fc0980d09e0 con 0x7fc0980d4560 2024-04-25T10:29:58.030 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:58.025+0000 7fc09ddc0640 1 -- --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_getmap magic: 0 -- 0x7fc0980b4fc0 con 0x7fc0980b7600 2024-04-25T10:29:58.030 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:58.025+0000 7fc09ddc0640 1 -- --> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] -- mon_getmap magic: 0 -- 0x7fc0980d0b60 con 0x7fc0980b4bb0 2024-04-25T10:29:58.030 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:58.026+0000 7fc097fff640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fc0980d4560 0x7fc0980cb170 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:29:58.030 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:58.026+0000 7fc097fff640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fc0980d4560 0x7fc0980cb170 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.37:3300/0 says I am v2:172.21.15.37:36604/0 (socket says 172.21.15.37:36604) 2024-04-25T10:29:58.030 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:58.026+0000 7fc097fff640 1 -- 172.21.15.37:0/192868913 learned_addr learned my addr 172.21.15.37:0/192868913 (peer_addr_for_me v2:172.21.15.37:0/0) 2024-04-25T10:29:58.030 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:58.026+0000 7fc096ffd640 1 --2- 172.21.15.37:0/192868913 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fc0980b7600 0x7fc09815fad0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:29:58.030 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:58.026+0000 7fc0977fe640 1 --2- 172.21.15.37:0/192868913 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7fc0980b4bb0 0x7fc0981f6400 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:29:58.030 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:58.026+0000 7fc097fff640 1 -- 172.21.15.37:0/192868913 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7fc0980b4bb0 msgr2=0x7fc0981f6400 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:29:58.030 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:58.026+0000 7fc097fff640 1 --2- 172.21.15.37:0/192868913 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7fc0980b4bb0 0x7fc0981f6400 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:29:58.030 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:58.026+0000 7fc097fff640 1 -- 172.21.15.37:0/192868913 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fc0980b7600 msgr2=0x7fc09815fad0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:29:58.030 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:58.026+0000 7fc097fff640 1 --2- 172.21.15.37:0/192868913 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fc0980b7600 0x7fc09815fad0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:29:58.030 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:58.026+0000 7fc097fff640 1 -- 172.21.15.37:0/192868913 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fc09815dcf0 con 0x7fc0980d4560 2024-04-25T10:29:58.031 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:58.026+0000 7fc0977fe640 1 --2- 172.21.15.37:0/192868913 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7fc0980b4bb0 0x7fc0981f6400 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T10:29:58.031 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:58.026+0000 7fc097fff640 1 --2- 172.21.15.37:0/192868913 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fc0980d4560 0x7fc0980cb170 secure :-1 s=READY pgs=349 cs=0 l=1 rev1=1 crypto rx=0x7fc09000cd50 tx=0x7fc09000a650 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:29:58.031 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:58.026+0000 7fc07ffff640 1 -- 172.21.15.37:0/192868913 <== mon.0 v2:172.21.15.37:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc09000f680 con 0x7fc0980d4560 2024-04-25T10:29:58.031 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:58.026+0000 7fc07ffff640 1 -- 172.21.15.37:0/192868913 <== mon.0 v2:172.21.15.37:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fc09000f840 con 0x7fc0980d4560 2024-04-25T10:29:58.031 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:58.026+0000 7fc09ddc0640 1 -- 172.21.15.37:0/192868913 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fc09815dfd0 con 0x7fc0980d4560 2024-04-25T10:29:58.031 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:58.026+0000 7fc07ffff640 1 -- 172.21.15.37:0/192868913 <== mon.0 v2:172.21.15.37:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc09000fc30 con 0x7fc0980d4560 2024-04-25T10:29:58.031 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:58.026+0000 7fc09ddc0640 1 -- 172.21.15.37:0/192868913 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fc09815e590 con 0x7fc0980d4560 2024-04-25T10:29:58.031 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:58.027+0000 7fc09ddc0640 1 -- 172.21.15.37:0/192868913 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fc09806a140 con 0x7fc0980d4560 2024-04-25T10:29:58.032 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:58.028+0000 7fc07ffff640 1 -- 172.21.15.37:0/192868913 <== mon.0 v2:172.21.15.37:3300/0 4 ==== mgrmap(e 5) ==== 96782+0+0 (secure 0 0 0) 0x7fc0900213f0 con 0x7fc0980d4560 2024-04-25T10:29:58.032 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:58.028+0000 7fc07ffff640 1 --2- 172.21.15.37:0/192868913 >> [v2:172.21.15.203:6832/2418205237,v1:172.21.15.203:6833/2418205237] conn(0x7fc0840744b0 0x7fc0840769c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:29:58.032 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:58.029+0000 7fc07ffff640 1 -- 172.21.15.37:0/192868913 <== mon.0 v2:172.21.15.37:3300/0 5 ==== osd_map(25..25 src has 1..25) ==== 5738+0+0 (secure 0 0 0) 0x7fc090096020 con 0x7fc0980d4560 2024-04-25T10:29:58.032 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:58.029+0000 7fc0977fe640 1 --2- 172.21.15.37:0/192868913 >> [v2:172.21.15.203:6832/2418205237,v1:172.21.15.203:6833/2418205237] conn(0x7fc0840744b0 0x7fc0840769c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:29:58.032 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:58.029+0000 7fc0977fe640 1 --2- 172.21.15.37:0/192868913 >> [v2:172.21.15.203:6832/2418205237,v1:172.21.15.203:6833/2418205237] conn(0x7fc0840744b0 0x7fc0840769c0 secure :-1 s=READY pgs=179 cs=0 l=1 rev1=1 crypto rx=0x7fc0980d6c00 tx=0x7fc088008040 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:29:58.035 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:58.033+0000 7fc07ffff640 1 -- 172.21.15.37:0/192868913 <== mon.0 v2:172.21.15.37:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187246 (secure 0 0 0) 0x7fc090014030 con 0x7fc0980d4560 2024-04-25T10:29:58.261 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:58.259+0000 7fc09ddc0640 1 -- 172.21.15.37:0/192868913 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7fc0980c9420 con 0x7fc0980d4560 2024-04-25T10:29:58.261 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:58.259+0000 7fc07ffff640 1 -- 172.21.15.37:0/192868913 <== mon.0 v2:172.21.15.37:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 8 v8) ==== 93+0+5178 (secure 0 0 0) 0x7fc090060790 con 0x7fc0980d4560 2024-04-25T10:29:58.262 INFO:teuthology.orchestra.run.smithi037.stdout: 2024-04-25T10:29:58.262 INFO:teuthology.orchestra.run.smithi037.stdout:{"epoch":8,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":4232,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.37:6835/1911169480","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.37:6833","nonce":1911169480},{"type":"v1","addr":"172.21.15.37:6835","nonce":1911169480}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3},{"gid":4241,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.203:6835/3873243328","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6834","nonce":3873243328},{"type":"v1","addr":"172.21.15.203:6835","nonce":3873243328}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3},{"gid":4278,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.37:6834/179723013","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.37:6832","nonce":179723013},{"type":"v1","addr":"172.21.15.37:6834","nonce":179723013}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":2}],"filesystems":[{"mdsmap":{"epoch":8,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-04-25T10:29:08.397829+0000","modified":"2024-04-25T10:29:11.407694+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":1,"in":[0],"up":{"mds_0":4279},"failed":[],"damaged":[],"stopped":[],"info":{"gid_4279":{"gid":4279,"name":"d","rank":0,"incarnation":4,"state":"up:active","state_seq":3,"addr":"172.21.15.203:6837/3215191422","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6836","nonce":3215191422},{"type":"v1","addr":"172.21.15.203:6837","nonce":3215191422}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":4279,"qdb_cluster":[4279]},"id":1}]} 2024-04-25T10:29:58.262 INFO:teuthology.orchestra.run.smithi037.stderr:dumped fsmap epoch 8 2024-04-25T10:29:58.264 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:58.261+0000 7fc09ddc0640 1 -- 172.21.15.37:0/192868913 >> [v2:172.21.15.203:6832/2418205237,v1:172.21.15.203:6833/2418205237] conn(0x7fc0840744b0 msgr2=0x7fc0840769c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:29:58.264 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:58.261+0000 7fc09ddc0640 1 --2- 172.21.15.37:0/192868913 >> [v2:172.21.15.203:6832/2418205237,v1:172.21.15.203:6833/2418205237] conn(0x7fc0840744b0 0x7fc0840769c0 secure :-1 s=READY pgs=179 cs=0 l=1 rev1=1 crypto rx=0x7fc0980d6c00 tx=0x7fc088008040 comp rx=0 tx=0).stop 2024-04-25T10:29:58.264 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:58.261+0000 7fc09ddc0640 1 -- 172.21.15.37:0/192868913 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fc0980d4560 msgr2=0x7fc0980cb170 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:29:58.264 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:58.261+0000 7fc09ddc0640 1 --2- 172.21.15.37:0/192868913 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fc0980d4560 0x7fc0980cb170 secure :-1 s=READY pgs=349 cs=0 l=1 rev1=1 crypto rx=0x7fc09000cd50 tx=0x7fc09000a650 comp rx=0 tx=0).stop 2024-04-25T10:29:58.264 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:58.261+0000 7fc09ddc0640 1 -- 172.21.15.37:0/192868913 shutdown_connections 2024-04-25T10:29:58.264 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:58.261+0000 7fc09ddc0640 1 --2- 172.21.15.37:0/192868913 >> [v2:172.21.15.203:6832/2418205237,v1:172.21.15.203:6833/2418205237] conn(0x7fc0840744b0 0x7fc0840769c0 unknown :-1 s=CLOSED pgs=179 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:29:58.264 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:58.261+0000 7fc09ddc0640 1 --2- 172.21.15.37:0/192868913 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fc0980d4560 0x7fc0980cb170 unknown :-1 s=CLOSED pgs=349 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:29:58.264 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:58.261+0000 7fc09ddc0640 1 --2- 172.21.15.37:0/192868913 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fc0980b7600 0x7fc09815fad0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:29:58.264 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:58.261+0000 7fc09ddc0640 1 --2- 172.21.15.37:0/192868913 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7fc0980b4bb0 0x7fc0981f6400 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:29:58.264 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:58.261+0000 7fc09ddc0640 1 -- 172.21.15.37:0/192868913 >> 172.21.15.37:0/192868913 conn(0x7fc0980d27e0 msgr2=0x7fc098162fe0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:29:58.264 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:58.261+0000 7fc09ddc0640 1 -- 172.21.15.37:0/192868913 shutdown_connections 2024-04-25T10:29:58.264 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:58.261+0000 7fc09ddc0640 1 -- 172.21.15.37:0/192868913 wait complete. 2024-04-25T10:29:58.276 DEBUG:teuthology.orchestra.run.smithi037:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-04-25T10:29:58.388 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:58.386+0000 7faf66c27640 1 Processor -- start 2024-04-25T10:29:58.389 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:58.386+0000 7faf66c27640 1 -- start start 2024-04-25T10:29:58.390 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:58.388+0000 7faf66c27640 1 --2- >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7faf600d5160 0x7faf600b4e80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:29:58.390 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:58.388+0000 7faf66c27640 1 --2- >> v2:172.21.15.37:3300/0 conn(0x7faf600d3230 0x7faf600b9060 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:29:58.390 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:58.388+0000 7faf66c27640 1 -- --> v1:172.21.15.203:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7faf600d1370 con 0x7faf600d0990 2024-04-25T10:29:58.391 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:58.388+0000 7faf66c27640 1 -- --> v2:172.21.15.37:3300/0 -- mon_getmap magic: 0 -- 0x7faf600d1650 con 0x7faf600d3230 2024-04-25T10:29:58.391 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:58.388+0000 7faf66c27640 1 -- --> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] -- mon_getmap magic: 0 -- 0x7faf600d4680 con 0x7faf600d5160 2024-04-25T10:29:58.391 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:58.388+0000 7faf651c4640 1 --2- >> v2:172.21.15.37:3300/0 conn(0x7faf600d3230 0x7faf600b9060 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:29:58.391 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:58.388+0000 7faf651c4640 1 --2- >> v2:172.21.15.37:3300/0 conn(0x7faf600d3230 0x7faf600b9060 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.37:3300/0 says I am v2:172.21.15.37:36616/0 (socket says 172.21.15.37:36616) 2024-04-25T10:29:58.391 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:58.388+0000 7faf651c4640 1 -- 172.21.15.37:0/3265337033 learned_addr learned my addr 172.21.15.37:0/3265337033 (peer_addr_for_me v2:172.21.15.37:0/0) 2024-04-25T10:29:58.391 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:58.388+0000 7faf649c3640 1 --2- 172.21.15.37:0/3265337033 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7faf600d5160 0x7faf600b4e80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:29:58.391 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:58.388+0000 7faf651c4640 1 -- 172.21.15.37:0/3265337033 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7faf600d5160 msgr2=0x7faf600b4e80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:29:58.391 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:58.388+0000 7faf651c4640 1 --2- 172.21.15.37:0/3265337033 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7faf600d5160 0x7faf600b4e80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:29:58.391 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:58.388+0000 7faf651c4640 1 -- 172.21.15.37:0/3265337033 >> v1:172.21.15.203:6789/0 conn(0x7faf600d0990 legacy=0x7faf600d2b20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:29:58.391 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:58.388+0000 7faf651c4640 1 -- 172.21.15.37:0/3265337033 --> v2:172.21.15.37:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7faf600d4960 con 0x7faf600d3230 2024-04-25T10:29:58.392 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:58.389+0000 7faf651c4640 1 --2- 172.21.15.37:0/3265337033 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7faf600d3230 0x7faf600b9060 secure :-1 s=READY pgs=350 cs=0 l=1 rev1=1 crypto rx=0x7faf58009e50 tx=0x7faf5802a610 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a3dab6704896c1a3 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:29:58.393 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:58.389+0000 7faf5f7fe640 1 -- 172.21.15.37:0/3265337033 <== mon.0 v2:172.21.15.37:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7faf58036070 con 0x7faf600d3230 2024-04-25T10:29:58.393 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:58.389+0000 7faf5f7fe640 1 -- 172.21.15.37:0/3265337033 <== mon.0 v2:172.21.15.37:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7faf58031600 con 0x7faf600d3230 2024-04-25T10:29:58.393 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:58.389+0000 7faf5f7fe640 1 -- 172.21.15.37:0/3265337033 <== mon.0 v2:172.21.15.37:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7faf580319d0 con 0x7faf600d3230 2024-04-25T10:29:58.393 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:58.390+0000 7faf66c27640 1 -- 172.21.15.37:0/3265337033 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7faf600d3230 msgr2=0x7faf600b9060 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:29:58.393 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:58.390+0000 7faf66c27640 1 --2- 172.21.15.37:0/3265337033 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7faf600d3230 0x7faf600b9060 secure :-1 s=READY pgs=350 cs=0 l=1 rev1=1 crypto rx=0x7faf58009e50 tx=0x7faf5802a610 comp rx=0 tx=0).stop 2024-04-25T10:29:58.393 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:58.390+0000 7faf66c27640 1 -- 172.21.15.37:0/3265337033 shutdown_connections 2024-04-25T10:29:58.393 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:58.390+0000 7faf66c27640 1 --2- 172.21.15.37:0/3265337033 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7faf600d3230 0x7faf600b9060 unknown :-1 s=CLOSED pgs=350 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:29:58.393 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:58.390+0000 7faf66c27640 1 --2- 172.21.15.37:0/3265337033 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7faf600d5160 0x7faf600b4e80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:29:58.393 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:58.390+0000 7faf66c27640 1 -- 172.21.15.37:0/3265337033 >> 172.21.15.37:0/3265337033 conn(0x7faf600b7a80 msgr2=0x7faf600b7e90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:29:58.393 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:58.390+0000 7faf66c27640 1 -- 172.21.15.37:0/3265337033 shutdown_connections 2024-04-25T10:29:58.393 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:58.390+0000 7faf66c27640 1 -- 172.21.15.37:0/3265337033 wait complete. 2024-04-25T10:29:58.394 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:58.391+0000 7faf66c27640 1 Processor -- start 2024-04-25T10:29:58.394 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:58.392+0000 7faf66c27640 1 -- start start 2024-04-25T10:29:58.396 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:58.393+0000 7faf66c27640 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7faf600d0990 0x7faf600ac5a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:29:58.397 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:58.393+0000 7faf66c27640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7faf600d5160 0x7faf6014a5f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:29:58.397 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:58.393+0000 7faf66c27640 1 --2- >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7faf6014ab30 0x7faf60157da0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:29:58.397 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:58.393+0000 7faf66c27640 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 -- 0x7faf60164c40 con 0x7faf600d5160 2024-04-25T10:29:58.397 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:58.393+0000 7faf66c27640 1 -- --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_getmap magic: 0 -- 0x7faf60164ac0 con 0x7faf600d0990 2024-04-25T10:29:58.397 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:58.393+0000 7faf66c27640 1 -- --> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] -- mon_getmap magic: 0 -- 0x7faf600d0da0 con 0x7faf6014ab30 2024-04-25T10:29:58.397 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:58.393+0000 7faf5ffff640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7faf600d5160 0x7faf6014a5f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:29:58.397 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:58.393+0000 7faf5ffff640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7faf600d5160 0x7faf6014a5f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.37:3300/0 says I am v2:172.21.15.37:36618/0 (socket says 172.21.15.37:36618) 2024-04-25T10:29:58.397 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:58.393+0000 7faf5ffff640 1 -- 172.21.15.37:0/2924243333 learned_addr learned my addr 172.21.15.37:0/2924243333 (peer_addr_for_me v2:172.21.15.37:0/0) 2024-04-25T10:29:58.397 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:58.393+0000 7faf649c3640 1 --2- 172.21.15.37:0/2924243333 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7faf600d0990 0x7faf600ac5a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:29:58.397 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:58.393+0000 7faf651c4640 1 --2- 172.21.15.37:0/2924243333 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7faf6014ab30 0x7faf60157da0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:29:58.397 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:58.394+0000 7faf5ffff640 1 -- 172.21.15.37:0/2924243333 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7faf6014ab30 msgr2=0x7faf60157da0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:29:58.397 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:58.394+0000 7faf5ffff640 1 --2- 172.21.15.37:0/2924243333 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7faf6014ab30 0x7faf60157da0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:29:58.397 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:58.394+0000 7faf5ffff640 1 -- 172.21.15.37:0/2924243333 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7faf600d0990 msgr2=0x7faf600ac5a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:29:58.398 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:58.394+0000 7faf5ffff640 1 --2- 172.21.15.37:0/2924243333 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7faf600d0990 0x7faf600ac5a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:29:58.398 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:58.394+0000 7faf5ffff640 1 -- 172.21.15.37:0/2924243333 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7faf6014b220 con 0x7faf600d5160 2024-04-25T10:29:58.398 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:58.394+0000 7faf651c4640 1 --2- 172.21.15.37:0/2924243333 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7faf6014ab30 0x7faf60157da0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-25T10:29:58.398 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:58.394+0000 7faf5ffff640 1 --2- 172.21.15.37:0/2924243333 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7faf600d5160 0x7faf6014a5f0 secure :-1 s=READY pgs=351 cs=0 l=1 rev1=1 crypto rx=0x7faf5400c390 tx=0x7faf5400c850 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:29:58.398 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:58.394+0000 7faf5d7fa640 1 -- 172.21.15.37:0/2924243333 <== mon.0 v2:172.21.15.37:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7faf540052d0 con 0x7faf600d5160 2024-04-25T10:29:58.398 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:58.394+0000 7faf5d7fa640 1 -- 172.21.15.37:0/2924243333 <== mon.0 v2:172.21.15.37:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7faf54005490 con 0x7faf600d5160 2024-04-25T10:29:58.398 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:58.394+0000 7faf66c27640 1 -- 172.21.15.37:0/2924243333 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7faf60161820 con 0x7faf600d5160 2024-04-25T10:29:58.398 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:58.394+0000 7faf5d7fa640 1 -- 172.21.15.37:0/2924243333 <== mon.0 v2:172.21.15.37:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7faf54007260 con 0x7faf600d5160 2024-04-25T10:29:58.398 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:58.394+0000 7faf66c27640 1 -- 172.21.15.37:0/2924243333 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7faf60161de0 con 0x7faf600d5160 2024-04-25T10:29:58.399 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:58.395+0000 7faf66c27640 1 -- 172.21.15.37:0/2924243333 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7faf6006a140 con 0x7faf600d5160 2024-04-25T10:29:58.399 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:58.396+0000 7faf5d7fa640 1 -- 172.21.15.37:0/2924243333 <== mon.0 v2:172.21.15.37:3300/0 4 ==== mgrmap(e 5) ==== 96782+0+0 (secure 0 0 0) 0x7faf54007400 con 0x7faf600d5160 2024-04-25T10:29:58.399 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:58.396+0000 7faf5d7fa640 1 --2- 172.21.15.37:0/2924243333 >> [v2:172.21.15.203:6832/2418205237,v1:172.21.15.203:6833/2418205237] conn(0x7faf4c0744b0 0x7faf4c0769c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:29:58.399 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:58.396+0000 7faf5d7fa640 1 -- 172.21.15.37:0/2924243333 <== mon.0 v2:172.21.15.37:3300/0 5 ==== osd_map(25..25 src has 1..25) ==== 5738+0+0 (secure 0 0 0) 0x7faf54094570 con 0x7faf600d5160 2024-04-25T10:29:58.400 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:58.397+0000 7faf649c3640 1 --2- 172.21.15.37:0/2924243333 >> [v2:172.21.15.203:6832/2418205237,v1:172.21.15.203:6833/2418205237] conn(0x7faf4c0744b0 0x7faf4c0769c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:29:58.400 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:58.397+0000 7faf649c3640 1 --2- 172.21.15.37:0/2924243333 >> [v2:172.21.15.203:6832/2418205237,v1:172.21.15.203:6833/2418205237] conn(0x7faf4c0744b0 0x7faf4c0769c0 secure :-1 s=READY pgs=180 cs=0 l=1 rev1=1 crypto rx=0x7faf50009aa0 tx=0x7faf50009190 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:29:58.403 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:58.401+0000 7faf5d7fa640 1 -- 172.21.15.37:0/2924243333 <== mon.0 v2:172.21.15.37:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187246 (secure 0 0 0) 0x7faf5405e690 con 0x7faf600d5160 2024-04-25T10:29:58.589 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:58.587+0000 7faf66c27640 1 -- 172.21.15.37:0/2924243333 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7faf600d3f50 con 0x7faf600d5160 2024-04-25T10:29:58.590 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:58.588+0000 7faf5d7fa640 1 -- 172.21.15.37:0/2924243333 <== mon.0 v2:172.21.15.37:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v25) ==== 74+0+16670 (secure 0 0 0) 0x7faf54062990 con 0x7faf600d5160 2024-04-25T10:29:58.590 INFO:teuthology.orchestra.run.smithi037.stdout: 2024-04-25T10:29:58.590 INFO:teuthology.orchestra.run.smithi037.stdout:{"epoch":25,"fsid":"f4acda28-be34-492a-b286-994afd36f61d","created":"2024-04-25T10:28:43.985718+0000","modified":"2024-04-25T10:29:49.325719+0000","last_up_change":"2024-04-25T10:28:54.900654+0000","last_in_change":"2024-04-25T10:28:46.216973+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":3,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"luminous","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-04-25T10:28:56.202153+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"19","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2024-04-25T10:29:06.507244+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"22","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":2.25,"score_stable":2.25,"optimal_score":1,"raw_score_acting":2.25,"raw_score_stable":2.25,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2024-04-25T10:29:07.347844+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"23","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"a2f79028-b332-4166-9ebb-a7024a37ea22","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":21,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.37:6816","nonce":2746074720},{"type":"v1","addr":"172.21.15.37:6817","nonce":2746074720}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.37:6818","nonce":2746074720},{"type":"v1","addr":"172.21.15.37:6819","nonce":2746074720}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.37:6822","nonce":2746074720},{"type":"v1","addr":"172.21.15.37:6823","nonce":2746074720}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.37:6820","nonce":2746074720},{"type":"v1","addr":"172.21.15.37:6821","nonce":2746074720}]},"public_addr":"172.21.15.37:6817/2746074720","cluster_addr":"172.21.15.37:6819/2746074720","heartbeat_back_addr":"172.21.15.37:6823/2746074720","heartbeat_front_addr":"172.21.15.37:6821/2746074720","state":["exists","up"]},{"osd":1,"uuid":"973159de-c2ea-4083-aecc-69373d1c1132","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":21,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.37:6800","nonce":1690341662},{"type":"v1","addr":"172.21.15.37:6801","nonce":1690341662}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.37:6802","nonce":1690341662},{"type":"v1","addr":"172.21.15.37:6803","nonce":1690341662}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.37:6806","nonce":1690341662},{"type":"v1","addr":"172.21.15.37:6807","nonce":1690341662}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.37:6804","nonce":1690341662},{"type":"v1","addr":"172.21.15.37:6805","nonce":1690341662}]},"public_addr":"172.21.15.37:6801/1690341662","cluster_addr":"172.21.15.37:6803/1690341662","heartbeat_back_addr":"172.21.15.37:6807/1690341662","heartbeat_front_addr":"172.21.15.37:6805/1690341662","state":["exists","up"]},{"osd":2,"uuid":"4ae7a0e9-cb3f-484c-a75f-f3766b721c47","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":21,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.37:6808","nonce":2754669649},{"type":"v1","addr":"172.21.15.37:6809","nonce":2754669649}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.37:6810","nonce":2754669649},{"type":"v1","addr":"172.21.15.37:6811","nonce":2754669649}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.37:6814","nonce":2754669649},{"type":"v1","addr":"172.21.15.37:6815","nonce":2754669649}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.37:6812","nonce":2754669649},{"type":"v1","addr":"172.21.15.37:6813","nonce":2754669649}]},"public_addr":"172.21.15.37:6809/2754669649","cluster_addr":"172.21.15.37:6811/2754669649","heartbeat_back_addr":"172.21.15.37:6815/2754669649","heartbeat_front_addr":"172.21.15.37:6813/2754669649","state":["exists","up"]},{"osd":3,"uuid":"1ba4a746-1608-47a7-ace2-960a5cfae934","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":21,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.37:6824","nonce":1815290618},{"type":"v1","addr":"172.21.15.37:6825","nonce":1815290618}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.37:6826","nonce":1815290618},{"type":"v1","addr":"172.21.15.37:6827","nonce":1815290618}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.37:6830","nonce":1815290618},{"type":"v1","addr":"172.21.15.37:6831","nonce":1815290618}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.37:6828","nonce":1815290618},{"type":"v1","addr":"172.21.15.37:6829","nonce":1815290618}]},"public_addr":"172.21.15.37:6825/1815290618","cluster_addr":"172.21.15.37:6827/1815290618","heartbeat_back_addr":"172.21.15.37:6831/1815290618","heartbeat_front_addr":"172.21.15.37:6829/1815290618","state":["exists","up"]},{"osd":4,"uuid":"d4f59f5d-0413-4b79-87ee-bb212c1fcf62","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":21,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6800","nonce":4051532063},{"type":"v1","addr":"172.21.15.203:6801","nonce":4051532063}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6802","nonce":4051532063},{"type":"v1","addr":"172.21.15.203:6803","nonce":4051532063}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6806","nonce":4051532063},{"type":"v1","addr":"172.21.15.203:6807","nonce":4051532063}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6804","nonce":4051532063},{"type":"v1","addr":"172.21.15.203:6805","nonce":4051532063}]},"public_addr":"172.21.15.203:6801/4051532063","cluster_addr":"172.21.15.203:6803/4051532063","heartbeat_back_addr":"172.21.15.203:6807/4051532063","heartbeat_front_addr":"172.21.15.203:6805/4051532063","state":["exists","up"]},{"osd":5,"uuid":"69710a0e-41d8-4e7f-9a10-17ce540b67be","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":24,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6824","nonce":1203822890},{"type":"v1","addr":"172.21.15.203:6825","nonce":1203822890}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6826","nonce":1203822890},{"type":"v1","addr":"172.21.15.203:6827","nonce":1203822890}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6830","nonce":1203822890},{"type":"v1","addr":"172.21.15.203:6831","nonce":1203822890}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6828","nonce":1203822890},{"type":"v1","addr":"172.21.15.203:6829","nonce":1203822890}]},"public_addr":"172.21.15.203:6825/1203822890","cluster_addr":"172.21.15.203:6827/1203822890","heartbeat_back_addr":"172.21.15.203:6831/1203822890","heartbeat_front_addr":"172.21.15.203:6829/1203822890","state":["exists","up"]},{"osd":6,"uuid":"f29dd047-3b48-473d-a235-6642a6293f20","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":21,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6810","nonce":2309615014},{"type":"v1","addr":"172.21.15.203:6812","nonce":2309615014}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6814","nonce":2309615014},{"type":"v1","addr":"172.21.15.203:6816","nonce":2309615014}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6822","nonce":2309615014},{"type":"v1","addr":"172.21.15.203:6823","nonce":2309615014}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6818","nonce":2309615014},{"type":"v1","addr":"172.21.15.203:6820","nonce":2309615014}]},"public_addr":"172.21.15.203:6812/2309615014","cluster_addr":"172.21.15.203:6816/2309615014","heartbeat_back_addr":"172.21.15.203:6823/2309615014","heartbeat_front_addr":"172.21.15.203:6820/2309615014","state":["exists","up"]},{"osd":7,"uuid":"6e691644-378b-4fdc-8e86-fb5f56e31f29","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":21,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6808","nonce":3688099014},{"type":"v1","addr":"172.21.15.203:6809","nonce":3688099014}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6811","nonce":3688099014},{"type":"v1","addr":"172.21.15.203:6813","nonce":3688099014}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6819","nonce":3688099014},{"type":"v1","addr":"172.21.15.203:6821","nonce":3688099014}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6815","nonce":3688099014},{"type":"v1","addr":"172.21.15.203:6817","nonce":3688099014}]},"public_addr":"172.21.15.203:6809/3688099014","cluster_addr":"172.21.15.203:6813/3688099014","heartbeat_back_addr":"172.21.15.203:6821/3688099014","heartbeat_front_addr":"172.21.15.203:6817/3688099014","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T10:28:51.419141+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T10:28:51.366448+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T10:28:51.348276+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T10:28:51.380519+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T10:28:51.105812+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T10:28:51.384496+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T10:28:51.388755+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T10:28:51.366958+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[{"pgid":"3.31","mappings":[{"from":0,"to":7}]}],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-04-25T10:29:58.592 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:58.590+0000 7faf66c27640 1 -- 172.21.15.37:0/2924243333 >> [v2:172.21.15.203:6832/2418205237,v1:172.21.15.203:6833/2418205237] conn(0x7faf4c0744b0 msgr2=0x7faf4c0769c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:29:58.592 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:58.590+0000 7faf66c27640 1 --2- 172.21.15.37:0/2924243333 >> [v2:172.21.15.203:6832/2418205237,v1:172.21.15.203:6833/2418205237] conn(0x7faf4c0744b0 0x7faf4c0769c0 secure :-1 s=READY pgs=180 cs=0 l=1 rev1=1 crypto rx=0x7faf50009aa0 tx=0x7faf50009190 comp rx=0 tx=0).stop 2024-04-25T10:29:58.592 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:58.590+0000 7faf66c27640 1 -- 172.21.15.37:0/2924243333 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7faf600d5160 msgr2=0x7faf6014a5f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:29:58.592 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:58.590+0000 7faf66c27640 1 --2- 172.21.15.37:0/2924243333 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7faf600d5160 0x7faf6014a5f0 secure :-1 s=READY pgs=351 cs=0 l=1 rev1=1 crypto rx=0x7faf5400c390 tx=0x7faf5400c850 comp rx=0 tx=0).stop 2024-04-25T10:29:58.593 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:58.590+0000 7faf66c27640 1 -- 172.21.15.37:0/2924243333 shutdown_connections 2024-04-25T10:29:58.593 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:58.590+0000 7faf66c27640 1 --2- 172.21.15.37:0/2924243333 >> [v2:172.21.15.203:6832/2418205237,v1:172.21.15.203:6833/2418205237] conn(0x7faf4c0744b0 0x7faf4c0769c0 unknown :-1 s=CLOSED pgs=180 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:29:58.593 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:58.590+0000 7faf66c27640 1 --2- 172.21.15.37:0/2924243333 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7faf6014ab30 0x7faf60157da0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:29:58.593 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:58.590+0000 7faf66c27640 1 --2- 172.21.15.37:0/2924243333 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7faf600d5160 0x7faf6014a5f0 unknown :-1 s=CLOSED pgs=351 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:29:58.593 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:58.590+0000 7faf66c27640 1 --2- 172.21.15.37:0/2924243333 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7faf600d0990 0x7faf600ac5a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:29:58.593 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:58.590+0000 7faf66c27640 1 -- 172.21.15.37:0/2924243333 >> 172.21.15.37:0/2924243333 conn(0x7faf600b7a80 msgr2=0x7faf600ba830 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:29:58.593 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:58.590+0000 7faf66c27640 1 -- 172.21.15.37:0/2924243333 shutdown_connections 2024-04-25T10:29:58.593 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:58.590+0000 7faf66c27640 1 -- 172.21.15.37:0/2924243333 wait complete. 2024-04-25T10:29:58.606 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2024-04-25T10:28:56.202153+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '19', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 8, 'score_stable': 8, 'optimal_score': 0.25, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 2, 'pool_name': 'cephfs_metadata', 'create_time': '2024-04-25T10:29:06.507244+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '22', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_autoscale_bias': 4, 'pg_num_min': 16, 'recovery_priority': 5}, 'application_metadata': {'cephfs': {'metadata': 'cephfs'}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 2.25, 'score_stable': 2.25, 'optimal_score': 1, 'raw_score_acting': 2.25, 'raw_score_stable': 2.25, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 3, 'pool_name': 'cephfs_data', 'create_time': '2024-04-25T10:29:07.347844+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 64, 'pg_placement_num': 64, 'pg_placement_num_target': 64, 'pg_num_target': 64, 'pg_num_pending': 64, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '23', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.8999999761581421}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 1.5, 'score_stable': 1.5, 'optimal_score': 1, 'raw_score_acting': 1.5, 'raw_score_stable': 1.5, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}] 2024-04-25T10:29:58.606 DEBUG:teuthology.orchestra.run.smithi037:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2024-04-25T10:29:58.721 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:58.719+0000 7fed3bab1640 1 Processor -- start 2024-04-25T10:29:58.721 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:58.719+0000 7fed3bab1640 1 -- start start 2024-04-25T10:29:58.723 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:58.720+0000 7fed3bab1640 1 --2- >> v2:172.21.15.203:3300/0 conn(0x7fed340b3de0 0x7fed3416ade0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:29:58.723 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:58.720+0000 7fed3bab1640 1 -- --> v1:172.21.15.37:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fed341612e0 con 0x7fed3409b700 2024-04-25T10:29:58.723 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:58.720+0000 7fed3bab1640 1 -- --> v1:172.21.15.203:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fed341615c0 con 0x7fed3415a940 2024-04-25T10:29:58.723 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:58.720+0000 7fed3bab1640 1 -- --> v2:172.21.15.203:3300/0 -- mon_getmap magic: 0 -- 0x7fed340b2940 con 0x7fed340b3de0 2024-04-25T10:29:58.723 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:58.720+0000 7fed3904c640 1 --1- >> v1:172.21.15.37:6789/0 conn(0x7fed3409b700 0x7fed3409bb10 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.37:6789/0 says I am v1:172.21.15.37:56860/0 (socket says 172.21.15.37:56860) 2024-04-25T10:29:58.724 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:58.720+0000 7fed3904c640 1 -- 172.21.15.37:0/1891119634 learned_addr learned my addr 172.21.15.37:0/1891119634 (peer_addr_for_me v1:172.21.15.37:0/0) 2024-04-25T10:29:58.724 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:58.720+0000 7fed3a04e640 1 --2- 172.21.15.37:0/1891119634 >> v2:172.21.15.203:3300/0 conn(0x7fed340b3de0 0x7fed3416ade0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:29:58.724 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:58.721+0000 7fed3884b640 1 -- 172.21.15.37:0/1891119634 <== mon.0 v1:172.21.15.37:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 3599970798 0 0) 0x7fed341612e0 con 0x7fed3409b700 2024-04-25T10:29:58.724 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:58.721+0000 7fed3884b640 1 -- 172.21.15.37:0/1891119634 --> v1:172.21.15.37:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7fed20003520 con 0x7fed3409b700 2024-04-25T10:29:58.724 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:58.721+0000 7fed3884b640 1 -- 172.21.15.37:0/1891119634 <== mon.1 v1:172.21.15.203:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 3591038515 0 0) 0x7fed341615c0 con 0x7fed3415a940 2024-04-25T10:29:58.724 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:58.721+0000 7fed3884b640 1 -- 172.21.15.37:0/1891119634 --> v1:172.21.15.203:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7fed341612e0 con 0x7fed3415a940 2024-04-25T10:29:58.724 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:58.721+0000 7fed3a04e640 1 -- 172.21.15.37:0/1891119634 >> v1:172.21.15.203:6789/0 conn(0x7fed3415a940 legacy=0x7fed341607b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:29:58.724 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:58.721+0000 7fed3a04e640 1 -- 172.21.15.37:0/1891119634 >> v1:172.21.15.37:6789/0 conn(0x7fed3409b700 legacy=0x7fed3409bb10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:29:58.724 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:58.721+0000 7fed3884b640 1 -- 172.21.15.37:0/1891119634 <== mon.0 v1:172.21.15.37:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 2319021244 0 0) 0x7fed20003520 con 0x7fed3409b700 2024-04-25T10:29:58.724 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:58.721+0000 7fed3a04e640 1 -- 172.21.15.37:0/1891119634 --> v2:172.21.15.203:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fed340b25d0 con 0x7fed340b3de0 2024-04-25T10:29:58.726 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:58.722+0000 7fed3a04e640 1 --2- 172.21.15.37:0/1891119634 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fed340b3de0 0x7fed3416ade0 secure :-1 s=READY pgs=155 cs=0 l=1 rev1=1 crypto rx=0x7fed300097a0 tx=0x7fed3002a130 comp rx=0 tx=0).ready entity=mon.1 client_cookie=7d35caa2383f8327 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:29:58.726 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:58.722+0000 7fed3884b640 1 -- 172.21.15.37:0/1891119634 <== mon.1 v2:172.21.15.203:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fed30037070 con 0x7fed340b3de0 2024-04-25T10:29:58.726 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:58.722+0000 7fed3884b640 1 -- 172.21.15.37:0/1891119634 <== mon.1 v2:172.21.15.203:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fed3002abc0 con 0x7fed340b3de0 2024-04-25T10:29:58.726 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:58.722+0000 7fed3884b640 1 -- 172.21.15.37:0/1891119634 <== mon.1 v2:172.21.15.203:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fed24001670 con 0x7fed340b3de0 2024-04-25T10:29:58.726 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:58.723+0000 7fed3bab1640 1 -- 172.21.15.37:0/1891119634 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fed340b3de0 msgr2=0x7fed3416ade0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:29:58.726 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:58.723+0000 7fed3bab1640 1 --2- 172.21.15.37:0/1891119634 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fed340b3de0 0x7fed3416ade0 secure :-1 s=READY pgs=155 cs=0 l=1 rev1=1 crypto rx=0x7fed300097a0 tx=0x7fed3002a130 comp rx=0 tx=0).stop 2024-04-25T10:29:58.726 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:58.723+0000 7fed3bab1640 1 -- 172.21.15.37:0/1891119634 shutdown_connections 2024-04-25T10:29:58.726 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:58.723+0000 7fed3bab1640 1 --2- 172.21.15.37:0/1891119634 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fed340b3de0 0x7fed3416ade0 unknown :-1 s=CLOSED pgs=155 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:29:58.726 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:58.723+0000 7fed3bab1640 1 -- 172.21.15.37:0/1891119634 >> 172.21.15.37:0/1891119634 conn(0x7fed340b7600 msgr2=0x7fed340b7c30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:29:58.726 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:58.723+0000 7fed3bab1640 1 -- 172.21.15.37:0/1891119634 shutdown_connections 2024-04-25T10:29:58.726 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:58.723+0000 7fed3bab1640 1 -- 172.21.15.37:0/1891119634 wait complete. 2024-04-25T10:29:58.727 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:58.725+0000 7fed3bab1640 1 Processor -- start 2024-04-25T10:29:58.727 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:58.725+0000 7fed3bab1640 1 -- start start 2024-04-25T10:29:58.730 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:58.726+0000 7fed3bab1640 1 --2- >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7fed3409b700 0x7fed34149470 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:29:58.730 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:58.726+0000 7fed3bab1640 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fed340b3de0 0x7fed3414c610 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:29:58.730 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:58.726+0000 7fed3bab1640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fed3415a940 0x7fed34070880 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:29:58.730 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:58.726+0000 7fed3bab1640 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 -- 0x7fed34161040 con 0x7fed3415a940 2024-04-25T10:29:58.730 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:58.726+0000 7fed3bab1640 1 -- --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_getmap magic: 0 -- 0x7fed34160ec0 con 0x7fed340b3de0 2024-04-25T10:29:58.730 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:58.726+0000 7fed3bab1640 1 -- --> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] -- mon_getmap magic: 0 -- 0x7fed3416b320 con 0x7fed3409b700 2024-04-25T10:29:58.730 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:58.726+0000 7fed3a04e640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fed3415a940 0x7fed34070880 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:29:58.730 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:58.726+0000 7fed3a04e640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fed3415a940 0x7fed34070880 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.37:3300/0 says I am v2:172.21.15.37:36622/0 (socket says 172.21.15.37:36622) 2024-04-25T10:29:58.730 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:58.726+0000 7fed3a04e640 1 -- 172.21.15.37:0/3821337216 learned_addr learned my addr 172.21.15.37:0/3821337216 (peer_addr_for_me v2:172.21.15.37:0/0) 2024-04-25T10:29:58.730 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:58.726+0000 7fed3904c640 1 --2- 172.21.15.37:0/3821337216 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fed340b3de0 0x7fed3414c610 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:29:58.730 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:58.726+0000 7fed3984d640 1 --2- 172.21.15.37:0/3821337216 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7fed3409b700 0x7fed34149470 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:29:58.730 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:58.726+0000 7fed3a04e640 1 -- 172.21.15.37:0/3821337216 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7fed3409b700 msgr2=0x7fed34149470 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:29:58.731 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:58.726+0000 7fed3a04e640 1 --2- 172.21.15.37:0/3821337216 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7fed3409b700 0x7fed34149470 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:29:58.731 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:58.726+0000 7fed3a04e640 1 -- 172.21.15.37:0/3821337216 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fed340b3de0 msgr2=0x7fed3414c610 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:29:58.731 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:58.726+0000 7fed3a04e640 1 --2- 172.21.15.37:0/3821337216 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fed340b3de0 0x7fed3414c610 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:29:58.731 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:58.726+0000 7fed3a04e640 1 -- 172.21.15.37:0/3821337216 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fed3406e4b0 con 0x7fed3415a940 2024-04-25T10:29:58.731 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:58.727+0000 7fed3a04e640 1 --2- 172.21.15.37:0/3821337216 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fed3415a940 0x7fed34070880 secure :-1 s=READY pgs=353 cs=0 l=1 rev1=1 crypto rx=0x7fed300098c0 tx=0x7fed300046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:29:58.731 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:58.727+0000 7fed2a7fc640 1 -- 172.21.15.37:0/3821337216 <== mon.0 v2:172.21.15.37:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fed30042020 con 0x7fed3415a940 2024-04-25T10:29:58.731 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:58.727+0000 7fed2a7fc640 1 -- 172.21.15.37:0/3821337216 <== mon.0 v2:172.21.15.37:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fed30003a40 con 0x7fed3415a940 2024-04-25T10:29:58.731 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:58.727+0000 7fed3bab1640 1 -- 172.21.15.37:0/3821337216 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fed3406c670 con 0x7fed3415a940 2024-04-25T10:29:58.731 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:58.727+0000 7fed2a7fc640 1 -- 172.21.15.37:0/3821337216 <== mon.0 v2:172.21.15.37:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fed30037040 con 0x7fed3415a940 2024-04-25T10:29:58.731 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:58.727+0000 7fed3bab1640 1 -- 172.21.15.37:0/3821337216 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fed3406cc30 con 0x7fed3415a940 2024-04-25T10:29:58.731 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:58.728+0000 7fed3bab1640 1 -- 172.21.15.37:0/3821337216 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fed3406a140 con 0x7fed3415a940 2024-04-25T10:29:58.732 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:58.728+0000 7fed2a7fc640 1 -- 172.21.15.37:0/3821337216 <== mon.0 v2:172.21.15.37:3300/0 4 ==== mgrmap(e 5) ==== 96782+0+0 (secure 0 0 0) 0x7fed30045020 con 0x7fed3415a940 2024-04-25T10:29:58.732 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:58.729+0000 7fed2a7fc640 1 --2- 172.21.15.37:0/3821337216 >> [v2:172.21.15.203:6832/2418205237,v1:172.21.15.203:6833/2418205237] conn(0x7fed180744e0 0x7fed180769f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:29:58.732 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:58.729+0000 7fed2a7fc640 1 -- 172.21.15.37:0/3821337216 <== mon.0 v2:172.21.15.37:3300/0 5 ==== osd_map(25..25 src has 1..25) ==== 5738+0+0 (secure 0 0 0) 0x7fed300b56e0 con 0x7fed3415a940 2024-04-25T10:29:58.732 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:58.729+0000 7fed3984d640 1 --2- 172.21.15.37:0/3821337216 >> [v2:172.21.15.203:6832/2418205237,v1:172.21.15.203:6833/2418205237] conn(0x7fed180744e0 0x7fed180769f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:29:58.732 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:58.730+0000 7fed3984d640 1 --2- 172.21.15.37:0/3821337216 >> [v2:172.21.15.203:6832/2418205237,v1:172.21.15.203:6833/2418205237] conn(0x7fed180744e0 0x7fed180769f0 secure :-1 s=READY pgs=181 cs=0 l=1 rev1=1 crypto rx=0x7fed2c001560 tx=0x7fed2c009040 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:29:58.735 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:58.733+0000 7fed2a7fc640 1 -- 172.21.15.37:0/3821337216 <== mon.0 v2:172.21.15.37:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187246 (secure 0 0 0) 0x7fed3004c030 con 0x7fed3415a940 2024-04-25T10:29:58.922 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:58.920+0000 7fed3bab1640 1 -- 172.21.15.37:0/3821337216 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) -- 0x7fed340d3000 con 0x7fed3415a940 2024-04-25T10:29:58.922 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:58.920+0000 7fed2a7fc640 1 -- 172.21.15.37:0/3821337216 <== mon.0 v2:172.21.15.37:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v25) ==== 93+0+10 (secure 0 0 0) 0x7fed3007fee0 con 0x7fed3415a940 2024-04-25T10:29:58.922 INFO:teuthology.orchestra.run.smithi037.stdout:pg_num: 1 2024-04-25T10:29:58.925 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:58.922+0000 7fed3bab1640 1 -- 172.21.15.37:0/3821337216 >> [v2:172.21.15.203:6832/2418205237,v1:172.21.15.203:6833/2418205237] conn(0x7fed180744e0 msgr2=0x7fed180769f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:29:58.925 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:58.922+0000 7fed3bab1640 1 --2- 172.21.15.37:0/3821337216 >> [v2:172.21.15.203:6832/2418205237,v1:172.21.15.203:6833/2418205237] conn(0x7fed180744e0 0x7fed180769f0 secure :-1 s=READY pgs=181 cs=0 l=1 rev1=1 crypto rx=0x7fed2c001560 tx=0x7fed2c009040 comp rx=0 tx=0).stop 2024-04-25T10:29:58.925 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:58.922+0000 7fed3bab1640 1 -- 172.21.15.37:0/3821337216 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fed3415a940 msgr2=0x7fed34070880 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:29:58.925 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:58.922+0000 7fed3bab1640 1 --2- 172.21.15.37:0/3821337216 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fed3415a940 0x7fed34070880 secure :-1 s=READY pgs=353 cs=0 l=1 rev1=1 crypto rx=0x7fed300098c0 tx=0x7fed300046c0 comp rx=0 tx=0).stop 2024-04-25T10:29:58.925 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:58.922+0000 7fed3bab1640 1 -- 172.21.15.37:0/3821337216 shutdown_connections 2024-04-25T10:29:58.925 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:58.922+0000 7fed3bab1640 1 --2- 172.21.15.37:0/3821337216 >> [v2:172.21.15.203:6832/2418205237,v1:172.21.15.203:6833/2418205237] conn(0x7fed180744e0 0x7fed180769f0 unknown :-1 s=CLOSED pgs=181 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:29:58.925 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:58.922+0000 7fed3bab1640 1 --2- 172.21.15.37:0/3821337216 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fed3415a940 0x7fed34070880 unknown :-1 s=CLOSED pgs=353 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:29:58.925 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:58.922+0000 7fed3bab1640 1 --2- 172.21.15.37:0/3821337216 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fed340b3de0 0x7fed3414c610 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:29:58.925 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:58.922+0000 7fed3bab1640 1 --2- 172.21.15.37:0/3821337216 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7fed3409b700 0x7fed34149470 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:29:58.925 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:58.922+0000 7fed3bab1640 1 -- 172.21.15.37:0/3821337216 >> 172.21.15.37:0/3821337216 conn(0x7fed340b7600 msgr2=0x7fed340afc50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:29:58.925 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:58.922+0000 7fed3bab1640 1 -- 172.21.15.37:0/3821337216 shutdown_connections 2024-04-25T10:29:58.925 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:58.922+0000 7fed3bab1640 1 -- 172.21.15.37:0/3821337216 wait complete. 2024-04-25T10:29:58.936 DEBUG:teuthology.orchestra.run.smithi037:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_metadata pg_num 2024-04-25T10:29:59.050 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:59.048+0000 7f3205b9d640 1 Processor -- start 2024-04-25T10:29:59.050 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:59.048+0000 7f3205b9d640 1 -- start start 2024-04-25T10:29:59.052 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:59.049+0000 7f3205b9d640 1 --2- >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f32000718e0 0x7f3200071cf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:29:59.052 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:59.049+0000 7f3205b9d640 1 --2- >> v2:172.21.15.203:3300/0 conn(0x7f32000d0a30 0x7f32000d0e40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:29:59.052 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:59.049+0000 7f3205b9d640 1 -- --> v1:172.21.15.203:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f3200072230 con 0x7f3200144420 2024-04-25T10:29:59.052 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:59.049+0000 7f3205b9d640 1 -- --> v2:172.21.15.203:3300/0 -- mon_getmap magic: 0 -- 0x7f3200144830 con 0x7f32000d0a30 2024-04-25T10:29:59.052 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:59.049+0000 7f3205b9d640 1 -- --> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] -- mon_getmap magic: 0 -- 0x7f320015ea00 con 0x7f32000718e0 2024-04-25T10:29:59.052 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:59.049+0000 7f31feffd640 1 --2- >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f32000718e0 0x7f3200071cf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:29:59.052 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:59.049+0000 7f31ff7fe640 1 --1- >> v1:172.21.15.203:6789/0 conn(0x7f3200144420 0x7f320006c540 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.203:6789/0 says I am v1:172.21.15.37:52772/0 (socket says 172.21.15.37:52772) 2024-04-25T10:29:59.052 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:59.049+0000 7f31ff7fe640 1 -- 172.21.15.37:0/3203368836 learned_addr learned my addr 172.21.15.37:0/3203368836 (peer_addr_for_me v1:172.21.15.37:0/0) 2024-04-25T10:29:59.052 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:59.049+0000 7f31fffff640 1 --2- 172.21.15.37:0/3203368836 >> v2:172.21.15.203:3300/0 conn(0x7f32000d0a30 0x7f32000d0e40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:29:59.052 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:59.050+0000 7f31fe7fc640 1 -- 172.21.15.37:0/3203368836 <== mon.1 v1:172.21.15.203:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 3800513612 0 0) 0x7f31f0001110 con 0x7f3200144420 2024-04-25T10:29:59.053 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:59.050+0000 7f31fe7fc640 1 -- 172.21.15.37:0/3203368836 --> v1:172.21.15.203:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f31e8003520 con 0x7f3200144420 2024-04-25T10:29:59.053 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:59.050+0000 7f31feffd640 1 -- 172.21.15.37:0/3203368836 >> v2:172.21.15.203:3300/0 conn(0x7f32000d0a30 msgr2=0x7f32000d0e40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:29:59.053 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:59.050+0000 7f31feffd640 1 --2- 172.21.15.37:0/3203368836 >> v2:172.21.15.203:3300/0 conn(0x7f32000d0a30 0x7f32000d0e40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:29:59.053 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:59.050+0000 7f31feffd640 1 -- 172.21.15.37:0/3203368836 >> v1:172.21.15.203:6789/0 conn(0x7f3200144420 legacy=0x7f320006c540 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:29:59.053 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:59.050+0000 7f31feffd640 1 -- 172.21.15.37:0/3203368836 --> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f320015ece0 con 0x7f32000718e0 2024-04-25T10:29:59.053 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:59.050+0000 7f31fffff640 1 --2- 172.21.15.37:0/3203368836 >> v2:172.21.15.203:3300/0 conn(0x7f32000d0a30 0x7f32000d0e40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-25T10:29:59.054 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:59.050+0000 7f31feffd640 1 --2- 172.21.15.37:0/3203368836 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f32000718e0 0x7f3200071cf0 secure :-1 s=READY pgs=48 cs=0 l=1 rev1=1 crypto rx=0x7f31f40097a0 tx=0x7f31f402a060 comp rx=0 tx=0).ready entity=mon.2 client_cookie=5bb1b711f20e85db server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:29:59.054 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:59.051+0000 7f31fe7fc640 1 -- 172.21.15.37:0/3203368836 <== mon.2 v2:172.21.15.203:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f31f4037070 con 0x7f32000718e0 2024-04-25T10:29:59.054 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:59.051+0000 7f31fe7fc640 1 -- 172.21.15.37:0/3203368836 <== mon.2 v2:172.21.15.203:3301/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f31f402adb0 con 0x7f32000718e0 2024-04-25T10:29:59.054 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:59.051+0000 7f31fe7fc640 1 -- 172.21.15.37:0/3203368836 <== mon.2 v2:172.21.15.203:3301/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f31f4032640 con 0x7f32000718e0 2024-04-25T10:29:59.054 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:59.051+0000 7f3205b9d640 1 -- 172.21.15.37:0/3203368836 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f32000718e0 msgr2=0x7f3200071cf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:29:59.054 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:59.051+0000 7f3205b9d640 1 --2- 172.21.15.37:0/3203368836 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f32000718e0 0x7f3200071cf0 secure :-1 s=READY pgs=48 cs=0 l=1 rev1=1 crypto rx=0x7f31f40097a0 tx=0x7f31f402a060 comp rx=0 tx=0).stop 2024-04-25T10:29:59.054 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:59.051+0000 7f3205b9d640 1 -- 172.21.15.37:0/3203368836 shutdown_connections 2024-04-25T10:29:59.055 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:59.051+0000 7f3205b9d640 1 --2- 172.21.15.37:0/3203368836 >> v2:172.21.15.203:3300/0 conn(0x7f32000d0a30 0x7f32000d0e40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:29:59.055 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:59.051+0000 7f3205b9d640 1 --2- 172.21.15.37:0/3203368836 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f32000718e0 0x7f3200071cf0 unknown :-1 s=CLOSED pgs=48 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:29:59.055 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:59.051+0000 7f3205b9d640 1 -- 172.21.15.37:0/3203368836 >> 172.21.15.37:0/3203368836 conn(0x7f32000d2850 msgr2=0x7f32000d2c60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:29:59.055 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:59.051+0000 7f3205b9d640 1 -- 172.21.15.37:0/3203368836 shutdown_connections 2024-04-25T10:29:59.055 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:59.051+0000 7f3205b9d640 1 -- 172.21.15.37:0/3203368836 wait complete. 2024-04-25T10:29:59.055 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:59.053+0000 7f3205b9d640 1 Processor -- start 2024-04-25T10:29:59.055 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:59.053+0000 7f3205b9d640 1 -- start start 2024-04-25T10:29:59.058 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:59.054+0000 7f3205b9d640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f32000718e0 0x7f3200132900 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:29:59.058 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:59.055+0000 7f3205b9d640 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f32000d0a30 0x7f3200159b80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:29:59.058 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:59.055+0000 7f3205b9d640 1 --2- >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f3200144420 0x7f3200154370 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:29:59.058 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:59.055+0000 7f3205b9d640 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 -- 0x7f320015f580 con 0x7f32000718e0 2024-04-25T10:29:59.058 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:59.055+0000 7f3205b9d640 1 -- --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_getmap magic: 0 -- 0x7f320015f2a0 con 0x7f32000d0a30 2024-04-25T10:29:59.058 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:59.055+0000 7f3205b9d640 1 -- --> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] -- mon_getmap magic: 0 -- 0x7f320006cc50 con 0x7f3200144420 2024-04-25T10:29:59.058 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:59.055+0000 7f31ff7fe640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f32000718e0 0x7f3200132900 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:29:59.058 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:59.055+0000 7f31ff7fe640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f32000718e0 0x7f3200132900 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.37:3300/0 says I am v2:172.21.15.37:36634/0 (socket says 172.21.15.37:36634) 2024-04-25T10:29:59.058 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:59.055+0000 7f31ff7fe640 1 -- 172.21.15.37:0/2889704254 learned_addr learned my addr 172.21.15.37:0/2889704254 (peer_addr_for_me v2:172.21.15.37:0/0) 2024-04-25T10:29:59.058 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:59.055+0000 7f31feffd640 1 --2- 172.21.15.37:0/2889704254 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f32000d0a30 0x7f3200159b80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:29:59.058 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:59.055+0000 7f31fffff640 1 --2- 172.21.15.37:0/2889704254 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f3200144420 0x7f3200154370 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:29:59.059 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:59.055+0000 7f31ff7fe640 1 -- 172.21.15.37:0/2889704254 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f3200144420 msgr2=0x7f3200154370 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:29:59.059 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:59.055+0000 7f31ff7fe640 1 --2- 172.21.15.37:0/2889704254 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f3200144420 0x7f3200154370 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:29:59.059 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:59.055+0000 7f31ff7fe640 1 -- 172.21.15.37:0/2889704254 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f32000d0a30 msgr2=0x7f3200159b80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:29:59.059 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:59.055+0000 7f31ff7fe640 1 --2- 172.21.15.37:0/2889704254 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f32000d0a30 0x7f3200159b80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:29:59.059 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:59.055+0000 7f31ff7fe640 1 -- 172.21.15.37:0/2889704254 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f32001581c0 con 0x7f32000718e0 2024-04-25T10:29:59.059 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:59.055+0000 7f31ff7fe640 1 --2- 172.21.15.37:0/2889704254 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f32000718e0 0x7f3200132900 secure :-1 s=READY pgs=354 cs=0 l=1 rev1=1 crypto rx=0x7f31f000c340 tx=0x7f31f000c740 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:29:59.059 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:59.055+0000 7f31e7fff640 1 -- 172.21.15.37:0/2889704254 <== mon.0 v2:172.21.15.37:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f31f0017070 con 0x7f32000718e0 2024-04-25T10:29:59.059 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:59.055+0000 7f31e7fff640 1 -- 172.21.15.37:0/2889704254 <== mon.0 v2:172.21.15.37:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f31f0005ce0 con 0x7f32000718e0 2024-04-25T10:29:59.059 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:59.055+0000 7f3205b9d640 1 -- 172.21.15.37:0/2889704254 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f3200133150 con 0x7f32000718e0 2024-04-25T10:29:59.059 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:59.056+0000 7f31e7fff640 1 -- 172.21.15.37:0/2889704254 <== mon.0 v2:172.21.15.37:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f31f00072f0 con 0x7f32000718e0 2024-04-25T10:29:59.059 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:59.056+0000 7f3205b9d640 1 -- 172.21.15.37:0/2889704254 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f3200133710 con 0x7f32000718e0 2024-04-25T10:29:59.060 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:59.056+0000 7f3205b9d640 1 -- 172.21.15.37:0/2889704254 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f320006a140 con 0x7f32000718e0 2024-04-25T10:29:59.060 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:59.057+0000 7f31e7fff640 1 -- 172.21.15.37:0/2889704254 <== mon.0 v2:172.21.15.37:3300/0 4 ==== mgrmap(e 5) ==== 96782+0+0 (secure 0 0 0) 0x7f31f001a070 con 0x7f32000718e0 2024-04-25T10:29:59.060 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:59.058+0000 7f31e7fff640 1 --2- 172.21.15.37:0/2889704254 >> [v2:172.21.15.203:6832/2418205237,v1:172.21.15.203:6833/2418205237] conn(0x7f31ec0744b0 0x7f31ec0769c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:29:59.060 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:59.058+0000 7f31e7fff640 1 -- 172.21.15.37:0/2889704254 <== mon.0 v2:172.21.15.37:3300/0 5 ==== osd_map(25..25 src has 1..25) ==== 5738+0+0 (secure 0 0 0) 0x7f31f0094490 con 0x7f32000718e0 2024-04-25T10:29:59.060 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:59.058+0000 7f31feffd640 1 --2- 172.21.15.37:0/2889704254 >> [v2:172.21.15.203:6832/2418205237,v1:172.21.15.203:6833/2418205237] conn(0x7f31ec0744b0 0x7f31ec0769c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:29:59.061 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:59.058+0000 7f31feffd640 1 --2- 172.21.15.37:0/2889704254 >> [v2:172.21.15.203:6832/2418205237,v1:172.21.15.203:6833/2418205237] conn(0x7f31ec0744b0 0x7f31ec0769c0 secure :-1 s=READY pgs=182 cs=0 l=1 rev1=1 crypto rx=0x7f31f4034000 tx=0x7f31f402e000 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:29:59.064 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:59.062+0000 7f31e7fff640 1 -- 172.21.15.37:0/2889704254 <== mon.0 v2:172.21.15.37:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187246 (secure 0 0 0) 0x7f3200072230 con 0x7f32000718e0 2024-04-25T10:29:59.252 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:59.250+0000 7f3205b9d640 1 -- 172.21.15.37:0/2889704254 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"} v 0) -- 0x7f32000d3640 con 0x7f32000718e0 2024-04-25T10:29:59.253 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:59.250+0000 7f31e7fff640 1 -- 172.21.15.37:0/2889704254 <== mon.0 v2:172.21.15.37:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]=0 v25) ==== 104+0+11 (secure 0 0 0) 0x7f31f005e5b0 con 0x7f32000718e0 2024-04-25T10:29:59.253 INFO:teuthology.orchestra.run.smithi037.stdout:pg_num: 32 2024-04-25T10:29:59.255 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:59.252+0000 7f3205b9d640 1 -- 172.21.15.37:0/2889704254 >> [v2:172.21.15.203:6832/2418205237,v1:172.21.15.203:6833/2418205237] conn(0x7f31ec0744b0 msgr2=0x7f31ec0769c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:29:59.255 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:59.252+0000 7f3205b9d640 1 --2- 172.21.15.37:0/2889704254 >> [v2:172.21.15.203:6832/2418205237,v1:172.21.15.203:6833/2418205237] conn(0x7f31ec0744b0 0x7f31ec0769c0 secure :-1 s=READY pgs=182 cs=0 l=1 rev1=1 crypto rx=0x7f31f4034000 tx=0x7f31f402e000 comp rx=0 tx=0).stop 2024-04-25T10:29:59.255 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:59.252+0000 7f3205b9d640 1 -- 172.21.15.37:0/2889704254 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f32000718e0 msgr2=0x7f3200132900 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:29:59.255 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:59.252+0000 7f3205b9d640 1 --2- 172.21.15.37:0/2889704254 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f32000718e0 0x7f3200132900 secure :-1 s=READY pgs=354 cs=0 l=1 rev1=1 crypto rx=0x7f31f000c340 tx=0x7f31f000c740 comp rx=0 tx=0).stop 2024-04-25T10:29:59.255 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:59.252+0000 7f3205b9d640 1 -- 172.21.15.37:0/2889704254 shutdown_connections 2024-04-25T10:29:59.255 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:59.252+0000 7f3205b9d640 1 --2- 172.21.15.37:0/2889704254 >> [v2:172.21.15.203:6832/2418205237,v1:172.21.15.203:6833/2418205237] conn(0x7f31ec0744b0 0x7f31ec0769c0 unknown :-1 s=CLOSED pgs=182 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:29:59.255 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:59.252+0000 7f3205b9d640 1 --2- 172.21.15.37:0/2889704254 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f3200144420 0x7f3200154370 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:29:59.255 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:59.252+0000 7f3205b9d640 1 --2- 172.21.15.37:0/2889704254 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f32000d0a30 0x7f3200159b80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:29:59.255 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:59.252+0000 7f3205b9d640 1 --2- 172.21.15.37:0/2889704254 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f32000718e0 0x7f3200132900 unknown :-1 s=CLOSED pgs=354 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:29:59.255 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:59.252+0000 7f3205b9d640 1 -- 172.21.15.37:0/2889704254 >> 172.21.15.37:0/2889704254 conn(0x7f32000d2850 msgr2=0x7f32000b2410 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:29:59.255 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:59.252+0000 7f3205b9d640 1 -- 172.21.15.37:0/2889704254 shutdown_connections 2024-04-25T10:29:59.256 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:59.252+0000 7f3205b9d640 1 -- 172.21.15.37:0/2889704254 wait complete. 2024-04-25T10:29:59.266 DEBUG:teuthology.orchestra.run.smithi037:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data pg_num 2024-04-25T10:29:59.380 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:59.378+0000 7fe52151f640 1 Processor -- start 2024-04-25T10:29:59.380 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:59.378+0000 7fe52151f640 1 -- start start 2024-04-25T10:29:59.382 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:59.379+0000 7fe52151f640 1 --2- >> v2:172.21.15.37:3300/0 conn(0x7fe51c0b7cd0 0x7fe51c160a00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:29:59.382 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:59.379+0000 7fe52151f640 1 --2- >> v2:172.21.15.203:3300/0 conn(0x7fe51c160f40 0x7fe51c161880 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:29:59.382 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:59.379+0000 7fe52151f640 1 -- --> v1:172.21.15.203:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fe51c0d5840 con 0x7fe51c0d5430 2024-04-25T10:29:59.382 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:59.379+0000 7fe52151f640 1 -- --> v2:172.21.15.37:3300/0 -- mon_getmap magic: 0 -- 0x7fe51c0d4420 con 0x7fe51c0b7cd0 2024-04-25T10:29:59.382 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:59.379+0000 7fe52151f640 1 -- --> v2:172.21.15.203:3300/0 -- mon_getmap magic: 0 -- 0x7fe51c0d4670 con 0x7fe51c160f40 2024-04-25T10:29:59.383 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:59.380+0000 7fe51a7fc640 1 --2- >> v2:172.21.15.37:3300/0 conn(0x7fe51c0b7cd0 0x7fe51c160a00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:29:59.383 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:59.380+0000 7fe51a7fc640 1 --2- >> v2:172.21.15.37:3300/0 conn(0x7fe51c0b7cd0 0x7fe51c160a00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.37:3300/0 says I am v2:172.21.15.37:36648/0 (socket says 172.21.15.37:36648) 2024-04-25T10:29:59.383 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:59.380+0000 7fe51a7fc640 1 -- 172.21.15.37:0/3783664111 learned_addr learned my addr 172.21.15.37:0/3783664111 (peer_addr_for_me v2:172.21.15.37:0/0) 2024-04-25T10:29:59.383 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:59.380+0000 7fe51a7fc640 1 -- 172.21.15.37:0/3783664111 >> v2:172.21.15.203:3300/0 conn(0x7fe51c160f40 msgr2=0x7fe51c161880 unknown :-1 s=STATE_CONNECTING_RE l=0).mark_down 2024-04-25T10:29:59.383 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:59.380+0000 7fe51a7fc640 1 --2- 172.21.15.37:0/3783664111 >> v2:172.21.15.203:3300/0 conn(0x7fe51c160f40 0x7fe51c161880 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:29:59.383 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:59.380+0000 7fe51a7fc640 1 -- 172.21.15.37:0/3783664111 >> v1:172.21.15.203:6789/0 conn(0x7fe51c0d5430 legacy=0x7fe51c0b75c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:29:59.383 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:59.380+0000 7fe51a7fc640 1 -- 172.21.15.37:0/3783664111 --> v2:172.21.15.37:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fe51c0d4950 con 0x7fe51c0b7cd0 2024-04-25T10:29:59.384 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:59.381+0000 7fe51a7fc640 1 --2- 172.21.15.37:0/3783664111 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fe51c0b7cd0 0x7fe51c160a00 secure :-1 s=READY pgs=355 cs=0 l=1 rev1=1 crypto rx=0x7fe51c0d3cb0 tx=0x7fe51002a210 comp rx=0 tx=0).ready entity=mon.0 client_cookie=6d8af227c415ee86 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:29:59.384 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:59.381+0000 7fe519ffb640 1 -- 172.21.15.37:0/3783664111 <== mon.0 v2:172.21.15.37:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe510037070 con 0x7fe51c0b7cd0 2024-04-25T10:29:59.384 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:59.381+0000 7fe519ffb640 1 -- 172.21.15.37:0/3783664111 <== mon.0 v2:172.21.15.37:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fe510032440 con 0x7fe51c0b7cd0 2024-04-25T10:29:59.384 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:59.381+0000 7fe519ffb640 1 -- 172.21.15.37:0/3783664111 <== mon.0 v2:172.21.15.37:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe510032810 con 0x7fe51c0b7cd0 2024-04-25T10:29:59.385 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:59.381+0000 7fe52151f640 1 -- 172.21.15.37:0/3783664111 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fe51c0b7cd0 msgr2=0x7fe51c160a00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:29:59.385 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:59.381+0000 7fe52151f640 1 --2- 172.21.15.37:0/3783664111 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fe51c0b7cd0 0x7fe51c160a00 secure :-1 s=READY pgs=355 cs=0 l=1 rev1=1 crypto rx=0x7fe51c0d3cb0 tx=0x7fe51002a210 comp rx=0 tx=0).stop 2024-04-25T10:29:59.385 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:59.381+0000 7fe52151f640 1 -- 172.21.15.37:0/3783664111 shutdown_connections 2024-04-25T10:29:59.385 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:59.381+0000 7fe52151f640 1 --2- 172.21.15.37:0/3783664111 >> v2:172.21.15.203:3300/0 conn(0x7fe51c160f40 0x7fe51c161880 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:29:59.385 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:59.381+0000 7fe52151f640 1 --2- 172.21.15.37:0/3783664111 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fe51c0b7cd0 0x7fe51c160a00 unknown :-1 s=CLOSED pgs=355 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:29:59.385 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:59.381+0000 7fe52151f640 1 -- 172.21.15.37:0/3783664111 >> 172.21.15.37:0/3783664111 conn(0x7fe51c0d27a0 msgr2=0x7fe51c0d2bb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:29:59.385 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:59.382+0000 7fe52151f640 1 -- 172.21.15.37:0/3783664111 shutdown_connections 2024-04-25T10:29:59.385 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:59.382+0000 7fe52151f640 1 -- 172.21.15.37:0/3783664111 wait complete. 2024-04-25T10:29:59.386 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:59.383+0000 7fe52151f640 1 Processor -- start 2024-04-25T10:29:59.386 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:59.383+0000 7fe52151f640 1 -- start start 2024-04-25T10:29:59.388 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:59.385+0000 7fe52151f640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fe51c0d5430 0x7fe51c0b3c10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:29:59.388 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:59.385+0000 7fe52151f640 1 --2- >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7fe51c160f40 0x7fe51c0b65d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:29:59.388 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:59.385+0000 7fe52151f640 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fe51c15b3d0 0x7fe51c0b6b10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:29:59.388 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:59.385+0000 7fe52151f640 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 -- 0x7fe51c161350 con 0x7fe51c0d5430 2024-04-25T10:29:59.388 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:59.385+0000 7fe52151f640 1 -- --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_getmap magic: 0 -- 0x7fe51c0d4f10 con 0x7fe51c15b3d0 2024-04-25T10:29:59.388 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:59.385+0000 7fe52151f640 1 -- --> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] -- mon_getmap magic: 0 -- 0x7fe51c1614d0 con 0x7fe51c160f40 2024-04-25T10:29:59.389 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:59.385+0000 7fe51affd640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fe51c0d5430 0x7fe51c0b3c10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:29:59.389 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:59.385+0000 7fe51affd640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fe51c0d5430 0x7fe51c0b3c10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.37:3300/0 says I am v2:172.21.15.37:36650/0 (socket says 172.21.15.37:36650) 2024-04-25T10:29:59.389 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:59.385+0000 7fe51affd640 1 -- 172.21.15.37:0/2693588471 learned_addr learned my addr 172.21.15.37:0/2693588471 (peer_addr_for_me v2:172.21.15.37:0/0) 2024-04-25T10:29:59.389 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:59.385+0000 7fe51b7fe640 1 --2- 172.21.15.37:0/2693588471 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fe51c15b3d0 0x7fe51c0b6b10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:29:59.389 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:59.385+0000 7fe51a7fc640 1 --2- 172.21.15.37:0/2693588471 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7fe51c160f40 0x7fe51c0b65d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:29:59.389 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:59.385+0000 7fe51affd640 1 -- 172.21.15.37:0/2693588471 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7fe51c160f40 msgr2=0x7fe51c0b65d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:29:59.389 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:59.385+0000 7fe51affd640 1 --2- 172.21.15.37:0/2693588471 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7fe51c160f40 0x7fe51c0b65d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:29:59.389 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:59.385+0000 7fe51affd640 1 -- 172.21.15.37:0/2693588471 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fe51c15b3d0 msgr2=0x7fe51c0b6b10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:29:59.389 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:59.385+0000 7fe51affd640 1 --2- 172.21.15.37:0/2693588471 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fe51c15b3d0 0x7fe51c0b6b10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:29:59.389 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:59.385+0000 7fe51affd640 1 -- 172.21.15.37:0/2693588471 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fe51c0b4150 con 0x7fe51c0d5430 2024-04-25T10:29:59.389 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:59.385+0000 7fe51affd640 1 --2- 172.21.15.37:0/2693588471 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fe51c0d5430 0x7fe51c0b3c10 secure :-1 s=READY pgs=356 cs=0 l=1 rev1=1 crypto rx=0x7fe50c00c6e0 tx=0x7fe50c00cba0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:29:59.389 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:59.385+0000 7fe5037fe640 1 -- 172.21.15.37:0/2693588471 <== mon.0 v2:172.21.15.37:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe50c016020 con 0x7fe51c0d5430 2024-04-25T10:29:59.390 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:59.385+0000 7fe5037fe640 1 -- 172.21.15.37:0/2693588471 <== mon.0 v2:172.21.15.37:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fe50c006d50 con 0x7fe51c0d5430 2024-04-25T10:29:59.390 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:59.385+0000 7fe52151f640 1 -- 172.21.15.37:0/2693588471 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fe51c0ccf30 con 0x7fe51c0d5430 2024-04-25T10:29:59.390 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:59.385+0000 7fe5037fe640 1 -- 172.21.15.37:0/2693588471 <== mon.0 v2:172.21.15.37:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe50c003bb0 con 0x7fe51c0d5430 2024-04-25T10:29:59.390 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:59.386+0000 7fe52151f640 1 -- 172.21.15.37:0/2693588471 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fe51c0cd4f0 con 0x7fe51c0d5430 2024-04-25T10:29:59.391 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:59.386+0000 7fe52151f640 1 -- 172.21.15.37:0/2693588471 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fe51c06a140 con 0x7fe51c0d5430 2024-04-25T10:29:59.391 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:59.387+0000 7fe5037fe640 1 -- 172.21.15.37:0/2693588471 <== mon.0 v2:172.21.15.37:3300/0 4 ==== mgrmap(e 5) ==== 96782+0+0 (secure 0 0 0) 0x7fe50c0050d0 con 0x7fe51c0d5430 2024-04-25T10:29:59.391 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:59.388+0000 7fe5037fe640 1 --2- 172.21.15.37:0/2693588471 >> [v2:172.21.15.203:6832/2418205237,v1:172.21.15.203:6833/2418205237] conn(0x7fe5080744b0 0x7fe5080769c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:29:59.391 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:59.388+0000 7fe5037fe640 1 -- 172.21.15.37:0/2693588471 <== mon.0 v2:172.21.15.37:3300/0 5 ==== osd_map(25..25 src has 1..25) ==== 5738+0+0 (secure 0 0 0) 0x7fe50c094cd0 con 0x7fe51c0d5430 2024-04-25T10:29:59.391 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:59.388+0000 7fe51a7fc640 1 --2- 172.21.15.37:0/2693588471 >> [v2:172.21.15.203:6832/2418205237,v1:172.21.15.203:6833/2418205237] conn(0x7fe5080744b0 0x7fe5080769c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:29:59.391 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:59.388+0000 7fe51a7fc640 1 --2- 172.21.15.37:0/2693588471 >> [v2:172.21.15.203:6832/2418205237,v1:172.21.15.203:6833/2418205237] conn(0x7fe5080744b0 0x7fe5080769c0 secure :-1 s=READY pgs=183 cs=0 l=1 rev1=1 crypto rx=0x7fe510034000 tx=0x7fe51002e000 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:29:59.394 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:59.392+0000 7fe5037fe640 1 -- 172.21.15.37:0/2693588471 <== mon.0 v2:172.21.15.37:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187246 (secure 0 0 0) 0x7fe50c020020 con 0x7fe51c0d5430 2024-04-25T10:29:59.579 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:59.577+0000 7fe52151f640 1 -- 172.21.15.37:0/2693588471 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"} v 0) -- 0x7fe51c0b45c0 con 0x7fe51c0d5430 2024-04-25T10:29:59.579 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:59.577+0000 7fe5037fe640 1 -- 172.21.15.37:0/2693588471 <== mon.0 v2:172.21.15.37:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]=0 v25) ==== 100+0+11 (secure 0 0 0) 0x7fe51c0b4150 con 0x7fe51c0d5430 2024-04-25T10:29:59.579 INFO:teuthology.orchestra.run.smithi037.stdout:pg_num: 64 2024-04-25T10:29:59.582 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:59.579+0000 7fe52151f640 1 -- 172.21.15.37:0/2693588471 >> [v2:172.21.15.203:6832/2418205237,v1:172.21.15.203:6833/2418205237] conn(0x7fe5080744b0 msgr2=0x7fe5080769c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:29:59.582 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:59.579+0000 7fe52151f640 1 --2- 172.21.15.37:0/2693588471 >> [v2:172.21.15.203:6832/2418205237,v1:172.21.15.203:6833/2418205237] conn(0x7fe5080744b0 0x7fe5080769c0 secure :-1 s=READY pgs=183 cs=0 l=1 rev1=1 crypto rx=0x7fe510034000 tx=0x7fe51002e000 comp rx=0 tx=0).stop 2024-04-25T10:29:59.582 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:59.579+0000 7fe52151f640 1 -- 172.21.15.37:0/2693588471 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fe51c0d5430 msgr2=0x7fe51c0b3c10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:29:59.582 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:59.579+0000 7fe52151f640 1 --2- 172.21.15.37:0/2693588471 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fe51c0d5430 0x7fe51c0b3c10 secure :-1 s=READY pgs=356 cs=0 l=1 rev1=1 crypto rx=0x7fe50c00c6e0 tx=0x7fe50c00cba0 comp rx=0 tx=0).stop 2024-04-25T10:29:59.582 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:59.579+0000 7fe52151f640 1 -- 172.21.15.37:0/2693588471 shutdown_connections 2024-04-25T10:29:59.582 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:59.579+0000 7fe52151f640 1 --2- 172.21.15.37:0/2693588471 >> [v2:172.21.15.203:6832/2418205237,v1:172.21.15.203:6833/2418205237] conn(0x7fe5080744b0 0x7fe5080769c0 unknown :-1 s=CLOSED pgs=183 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:29:59.582 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:59.579+0000 7fe52151f640 1 --2- 172.21.15.37:0/2693588471 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fe51c15b3d0 0x7fe51c0b6b10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:29:59.582 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:59.579+0000 7fe52151f640 1 --2- 172.21.15.37:0/2693588471 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7fe51c160f40 0x7fe51c0b65d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:29:59.583 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:59.579+0000 7fe52151f640 1 --2- 172.21.15.37:0/2693588471 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fe51c0d5430 0x7fe51c0b3c10 unknown :-1 s=CLOSED pgs=356 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:29:59.583 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:59.579+0000 7fe52151f640 1 -- 172.21.15.37:0/2693588471 >> 172.21.15.37:0/2693588471 conn(0x7fe51c0d27a0 msgr2=0x7fe51c0d5b20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:29:59.583 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:59.579+0000 7fe52151f640 1 -- 172.21.15.37:0/2693588471 shutdown_connections 2024-04-25T10:29:59.583 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:59.579+0000 7fe52151f640 1 -- 172.21.15.37:0/2693588471 wait complete. 2024-04-25T10:29:59.593 DEBUG:teuthology.orchestra.run.smithi037:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-04-25T10:29:59.704 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:59.702+0000 7f58239ee640 1 Processor -- start 2024-04-25T10:29:59.704 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:59.702+0000 7f58239ee640 1 -- start start 2024-04-25T10:29:59.706 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:59.703+0000 7f58239ee640 1 --2- >> v2:172.21.15.37:3300/0 conn(0x7f581c0b4bb0 0x7f581c1646b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:29:59.706 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:59.703+0000 7f58239ee640 1 --2- >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f581c0b7600 0x7f581c0b7a10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:29:59.706 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:59.703+0000 7f58239ee640 1 -- --> v1:172.21.15.203:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f581c0b7f50 con 0x7f581c0d4560 2024-04-25T10:29:59.706 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:59.703+0000 7f58239ee640 1 -- --> v2:172.21.15.37:3300/0 -- mon_getmap magic: 0 -- 0x7f581c0d5080 con 0x7f581c0b4bb0 2024-04-25T10:29:59.706 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:59.703+0000 7f58239ee640 1 -- --> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] -- mon_getmap magic: 0 -- 0x7f581c161190 con 0x7f581c0b7600 2024-04-25T10:29:59.706 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:59.703+0000 7f582178a640 1 --2- >> v2:172.21.15.37:3300/0 conn(0x7f581c0b4bb0 0x7f581c1646b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:29:59.707 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:59.703+0000 7f582178a640 1 --2- >> v2:172.21.15.37:3300/0 conn(0x7f581c0b4bb0 0x7f581c1646b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.37:3300/0 says I am v2:172.21.15.37:54458/0 (socket says 172.21.15.37:54458) 2024-04-25T10:29:59.707 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:59.703+0000 7f582178a640 1 -- 172.21.15.37:0/580202360 learned_addr learned my addr 172.21.15.37:0/580202360 (peer_addr_for_me v2:172.21.15.37:0/0) 2024-04-25T10:29:59.707 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:59.704+0000 7f582178a640 1 -- 172.21.15.37:0/580202360 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f581c0b7600 msgr2=0x7f581c0b7a10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:29:59.707 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:59.704+0000 7f582178a640 1 --2- 172.21.15.37:0/580202360 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f581c0b7600 0x7f581c0b7a10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:29:59.707 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:59.704+0000 7f582178a640 1 -- 172.21.15.37:0/580202360 >> v1:172.21.15.203:6789/0 conn(0x7f581c0d4560 legacy=0x7f581c0d4970 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:29:59.707 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:59.704+0000 7f582178a640 1 -- 172.21.15.37:0/580202360 --> v2:172.21.15.37:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f581c161470 con 0x7f581c0b4bb0 2024-04-25T10:29:59.708 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:59.704+0000 7f582178a640 1 --2- 172.21.15.37:0/580202360 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f581c0b4bb0 0x7f581c1646b0 secure :-1 s=READY pgs=357 cs=0 l=1 rev1=1 crypto rx=0x7f5818009870 tx=0x7f581802a210 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f73ef1812dcae3af server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:29:59.708 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:59.705+0000 7f5813fff640 1 -- 172.21.15.37:0/580202360 <== mon.0 v2:172.21.15.37:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5818037070 con 0x7f581c0b4bb0 2024-04-25T10:29:59.708 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:59.705+0000 7f5813fff640 1 -- 172.21.15.37:0/580202360 <== mon.0 v2:172.21.15.37:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f5818032440 con 0x7f581c0b4bb0 2024-04-25T10:29:59.708 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:59.705+0000 7f5813fff640 1 -- 172.21.15.37:0/580202360 <== mon.0 v2:172.21.15.37:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5818032810 con 0x7f581c0b4bb0 2024-04-25T10:29:59.708 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:59.705+0000 7f58239ee640 1 -- 172.21.15.37:0/580202360 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f581c0b4bb0 msgr2=0x7f581c1646b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:29:59.708 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:59.705+0000 7f58239ee640 1 --2- 172.21.15.37:0/580202360 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f581c0b4bb0 0x7f581c1646b0 secure :-1 s=READY pgs=357 cs=0 l=1 rev1=1 crypto rx=0x7f5818009870 tx=0x7f581802a210 comp rx=0 tx=0).stop 2024-04-25T10:29:59.709 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:59.705+0000 7f58239ee640 1 -- 172.21.15.37:0/580202360 shutdown_connections 2024-04-25T10:29:59.709 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:59.705+0000 7f58239ee640 1 --2- 172.21.15.37:0/580202360 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f581c0b7600 0x7f581c0b7a10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:29:59.709 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:59.705+0000 7f58239ee640 1 --2- 172.21.15.37:0/580202360 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f581c0b4bb0 0x7f581c1646b0 unknown :-1 s=CLOSED pgs=357 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:29:59.709 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:59.705+0000 7f58239ee640 1 -- 172.21.15.37:0/580202360 >> 172.21.15.37:0/580202360 conn(0x7f581c0d27e0 msgr2=0x7f581c0d2bf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:29:59.709 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:59.705+0000 7f58239ee640 1 -- 172.21.15.37:0/580202360 shutdown_connections 2024-04-25T10:29:59.709 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:59.705+0000 7f58239ee640 1 -- 172.21.15.37:0/580202360 wait complete. 2024-04-25T10:29:59.709 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:59.707+0000 7f58239ee640 1 Processor -- start 2024-04-25T10:29:59.709 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:59.707+0000 7f58239ee640 1 -- start start 2024-04-25T10:29:59.712 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:59.708+0000 7f58239ee640 1 --2- >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f581c0b4bb0 0x7f581c0cef20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:29:59.712 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:59.708+0000 7f58239ee640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f581c0b7600 0x7f581c156d70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:29:59.712 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:59.709+0000 7f58239ee640 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f581c0d4560 0x7f581c0c9d30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:29:59.712 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:59.709+0000 7f58239ee640 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 -- 0x7f581c0b4fc0 con 0x7f581c0b7600 2024-04-25T10:29:59.712 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:59.709+0000 7f58239ee640 1 -- --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_getmap magic: 0 -- 0x7f581c164190 con 0x7f581c0d4560 2024-04-25T10:29:59.712 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:59.709+0000 7f58239ee640 1 -- --> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] -- mon_getmap magic: 0 -- 0x7f581c164bf0 con 0x7f581c0b4bb0 2024-04-25T10:29:59.712 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:59.709+0000 7f5820f89640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f581c0b7600 0x7f581c156d70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:29:59.712 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:59.709+0000 7f5820f89640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f581c0b7600 0x7f581c156d70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.37:3300/0 says I am v2:172.21.15.37:54460/0 (socket says 172.21.15.37:54460) 2024-04-25T10:29:59.713 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:59.709+0000 7f5820f89640 1 -- 172.21.15.37:0/2703359555 learned_addr learned my addr 172.21.15.37:0/2703359555 (peer_addr_for_me v2:172.21.15.37:0/0) 2024-04-25T10:29:59.713 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:59.709+0000 7f5820f89640 1 -- 172.21.15.37:0/2703359555 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f581c0b4bb0 msgr2=0x7f581c0cef20 unknown :-1 s=STATE_CONNECTING_RE l=1).mark_down 2024-04-25T10:29:59.713 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:59.709+0000 7f5820f89640 1 --2- 172.21.15.37:0/2703359555 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f581c0b4bb0 0x7f581c0cef20 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:29:59.713 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:59.709+0000 7f5820f89640 1 -- 172.21.15.37:0/2703359555 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f581c0d4560 msgr2=0x7f581c0c9d30 unknown :-1 s=STATE_CONNECTING_RE l=1).mark_down 2024-04-25T10:29:59.713 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:59.709+0000 7f5820f89640 1 --2- 172.21.15.37:0/2703359555 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f581c0d4560 0x7f581c0c9d30 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:29:59.713 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:59.709+0000 7f5820f89640 1 -- 172.21.15.37:0/2703359555 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f581c1ead70 con 0x7f581c0b7600 2024-04-25T10:29:59.713 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:59.709+0000 7f5820f89640 1 --2- 172.21.15.37:0/2703359555 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f581c0b7600 0x7f581c156d70 secure :-1 s=READY pgs=358 cs=0 l=1 rev1=1 crypto rx=0x7f580c00b410 tx=0x7f580c00b8d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:29:59.713 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:59.709+0000 7f5811ffb640 1 -- 172.21.15.37:0/2703359555 <== mon.0 v2:172.21.15.37:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f580c015020 con 0x7f581c0b7600 2024-04-25T10:29:59.713 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:59.709+0000 7f5811ffb640 1 -- 172.21.15.37:0/2703359555 <== mon.0 v2:172.21.15.37:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f580c011850 con 0x7f581c0b7600 2024-04-25T10:29:59.713 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:59.709+0000 7f58239ee640 1 -- 172.21.15.37:0/2703359555 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f581c1eb050 con 0x7f581c0b7600 2024-04-25T10:29:59.713 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:59.709+0000 7f5811ffb640 1 -- 172.21.15.37:0/2703359555 <== mon.0 v2:172.21.15.37:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f580c011c40 con 0x7f581c0b7600 2024-04-25T10:29:59.713 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:59.710+0000 7f58239ee640 1 -- 172.21.15.37:0/2703359555 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f581c0ca870 con 0x7f581c0b7600 2024-04-25T10:29:59.714 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:59.710+0000 7f58239ee640 1 -- 172.21.15.37:0/2703359555 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f581c06a140 con 0x7f581c0b7600 2024-04-25T10:29:59.714 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:59.711+0000 7f5811ffb640 1 -- 172.21.15.37:0/2703359555 <== mon.0 v2:172.21.15.37:3300/0 4 ==== mgrmap(e 5) ==== 96782+0+0 (secure 0 0 0) 0x7f580c018020 con 0x7f581c0b7600 2024-04-25T10:29:59.714 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:59.712+0000 7f5811ffb640 1 --2- 172.21.15.37:0/2703359555 >> [v2:172.21.15.203:6832/2418205237,v1:172.21.15.203:6833/2418205237] conn(0x7f58000744b0 0x7f58000769c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:29:59.714 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:59.712+0000 7f5811ffb640 1 -- 172.21.15.37:0/2703359555 <== mon.0 v2:172.21.15.37:3300/0 5 ==== osd_map(25..25 src has 1..25) ==== 5738+0+0 (secure 0 0 0) 0x7f580c095230 con 0x7f581c0b7600 2024-04-25T10:29:59.715 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:59.712+0000 7f582178a640 1 --2- 172.21.15.37:0/2703359555 >> [v2:172.21.15.203:6832/2418205237,v1:172.21.15.203:6833/2418205237] conn(0x7f58000744b0 0x7f58000769c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:29:59.715 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:59.712+0000 7f582178a640 1 --2- 172.21.15.37:0/2703359555 >> [v2:172.21.15.203:6832/2418205237,v1:172.21.15.203:6833/2418205237] conn(0x7f58000744b0 0x7f58000769c0 secure :-1 s=READY pgs=184 cs=0 l=1 rev1=1 crypto rx=0x7f5818034000 tx=0x7f581802e000 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:29:59.718 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:59.716+0000 7f5811ffb640 1 -- 172.21.15.37:0/2703359555 <== mon.0 v2:172.21.15.37:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187246 (secure 0 0 0) 0x7f580c01f020 con 0x7f581c0b7600 2024-04-25T10:29:59.942 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:59.940+0000 7f58239ee640 1 -- 172.21.15.37:0/2703359555 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f581c0d3ff0 con 0x7f581c0b7600 2024-04-25T10:29:59.942 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:59.940+0000 7f5811ffb640 1 -- 172.21.15.37:0/2703359555 <== mon.0 v2:172.21.15.37:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 8 v8) ==== 93+0+5178 (secure 0 0 0) 0x7f580c05f900 con 0x7f581c0b7600 2024-04-25T10:29:59.943 INFO:teuthology.orchestra.run.smithi037.stdout: 2024-04-25T10:29:59.943 INFO:teuthology.orchestra.run.smithi037.stdout:{"epoch":8,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":4232,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.37:6835/1911169480","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.37:6833","nonce":1911169480},{"type":"v1","addr":"172.21.15.37:6835","nonce":1911169480}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3},{"gid":4241,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.203:6835/3873243328","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6834","nonce":3873243328},{"type":"v1","addr":"172.21.15.203:6835","nonce":3873243328}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3},{"gid":4278,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.37:6834/179723013","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.37:6832","nonce":179723013},{"type":"v1","addr":"172.21.15.37:6834","nonce":179723013}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":2}],"filesystems":[{"mdsmap":{"epoch":8,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-04-25T10:29:08.397829+0000","modified":"2024-04-25T10:29:11.407694+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":1,"in":[0],"up":{"mds_0":4279},"failed":[],"damaged":[],"stopped":[],"info":{"gid_4279":{"gid":4279,"name":"d","rank":0,"incarnation":4,"state":"up:active","state_seq":3,"addr":"172.21.15.203:6837/3215191422","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6836","nonce":3215191422},{"type":"v1","addr":"172.21.15.203:6837","nonce":3215191422}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":4279,"qdb_cluster":[4279]},"id":1}]} 2024-04-25T10:29:59.943 INFO:teuthology.orchestra.run.smithi037.stderr:dumped fsmap epoch 8 2024-04-25T10:29:59.945 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:59.942+0000 7f58239ee640 1 -- 172.21.15.37:0/2703359555 >> [v2:172.21.15.203:6832/2418205237,v1:172.21.15.203:6833/2418205237] conn(0x7f58000744b0 msgr2=0x7f58000769c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:29:59.945 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:59.942+0000 7f58239ee640 1 --2- 172.21.15.37:0/2703359555 >> [v2:172.21.15.203:6832/2418205237,v1:172.21.15.203:6833/2418205237] conn(0x7f58000744b0 0x7f58000769c0 secure :-1 s=READY pgs=184 cs=0 l=1 rev1=1 crypto rx=0x7f5818034000 tx=0x7f581802e000 comp rx=0 tx=0).stop 2024-04-25T10:29:59.945 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:59.942+0000 7f58239ee640 1 -- 172.21.15.37:0/2703359555 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f581c0b7600 msgr2=0x7f581c156d70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:29:59.945 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:59.942+0000 7f58239ee640 1 --2- 172.21.15.37:0/2703359555 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f581c0b7600 0x7f581c156d70 secure :-1 s=READY pgs=358 cs=0 l=1 rev1=1 crypto rx=0x7f580c00b410 tx=0x7f580c00b8d0 comp rx=0 tx=0).stop 2024-04-25T10:29:59.945 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:59.942+0000 7f58239ee640 1 -- 172.21.15.37:0/2703359555 shutdown_connections 2024-04-25T10:29:59.945 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:59.942+0000 7f58239ee640 1 --2- 172.21.15.37:0/2703359555 >> [v2:172.21.15.203:6832/2418205237,v1:172.21.15.203:6833/2418205237] conn(0x7f58000744b0 0x7f58000769c0 unknown :-1 s=CLOSED pgs=184 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:29:59.945 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:59.942+0000 7f58239ee640 1 --2- 172.21.15.37:0/2703359555 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f581c0d4560 0x7f581c0c9d30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:29:59.945 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:59.942+0000 7f58239ee640 1 --2- 172.21.15.37:0/2703359555 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f581c0b7600 0x7f581c156d70 unknown :-1 s=CLOSED pgs=358 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:29:59.945 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:59.942+0000 7f58239ee640 1 --2- 172.21.15.37:0/2703359555 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f581c0b4bb0 0x7f581c0cef20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:29:59.945 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:59.942+0000 7f58239ee640 1 -- 172.21.15.37:0/2703359555 >> 172.21.15.37:0/2703359555 conn(0x7f581c0d27e0 msgr2=0x7f581c0d2bf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:29:59.945 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:59.942+0000 7f58239ee640 1 -- 172.21.15.37:0/2703359555 shutdown_connections 2024-04-25T10:29:59.946 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:29:59.942+0000 7f58239ee640 1 -- 172.21.15.37:0/2703359555 wait complete. 2024-04-25T10:29:59.956 DEBUG:teuthology.orchestra.run.smithi037:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-04-25T10:30:00.068 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:00.065+0000 7fc603a9f640 1 Processor -- start 2024-04-25T10:30:00.068 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:00.066+0000 7fc603a9f640 1 -- start start 2024-04-25T10:30:00.070 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:00.067+0000 7fc603a9f640 1 --2- >> v2:172.21.15.203:3300/0 conn(0x7fc5fc143500 0x7fc5fc06f100 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:30:00.070 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:00.067+0000 7fc603a9f640 1 --2- >> v2:172.21.15.37:3300/0 conn(0x7fc5fc1460e0 0x7fc5fc0b6640 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:30:00.070 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:00.067+0000 7fc603a9f640 1 --2- >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7fc5fc1609a0 0x7fc5fc160db0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:30:00.070 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:00.067+0000 7fc603a9f640 1 -- --> v2:172.21.15.37:3300/0 -- mon_getmap magic: 0 -- 0x7fc5fc161730 con 0x7fc5fc1460e0 2024-04-25T10:30:00.070 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:00.067+0000 7fc603a9f640 1 -- --> v2:172.21.15.203:3300/0 -- mon_getmap magic: 0 -- 0x7fc5fc0b5b60 con 0x7fc5fc143500 2024-04-25T10:30:00.070 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:00.067+0000 7fc603a9f640 1 -- --> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] -- mon_getmap magic: 0 -- 0x7fc5fc0b5e40 con 0x7fc5fc1609a0 2024-04-25T10:30:00.070 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:00.067+0000 7fc60103a640 1 --2- >> v2:172.21.15.37:3300/0 conn(0x7fc5fc1460e0 0x7fc5fc0b6640 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:30:00.070 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:00.067+0000 7fc60103a640 1 --2- >> v2:172.21.15.37:3300/0 conn(0x7fc5fc1460e0 0x7fc5fc0b6640 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.37:3300/0 says I am v2:172.21.15.37:54462/0 (socket says 172.21.15.37:54462) 2024-04-25T10:30:00.070 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:00.067+0000 7fc60103a640 1 -- 172.21.15.37:0/1302053793 learned_addr learned my addr 172.21.15.37:0/1302053793 (peer_addr_for_me v2:172.21.15.37:0/0) 2024-04-25T10:30:00.071 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:00.067+0000 7fc60183b640 1 --2- 172.21.15.37:0/1302053793 >> v2:172.21.15.203:3300/0 conn(0x7fc5fc143500 0x7fc5fc06f100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:30:00.071 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:00.067+0000 7fc60103a640 1 -- 172.21.15.37:0/1302053793 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7fc5fc1609a0 msgr2=0x7fc5fc160db0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:30:00.071 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:00.068+0000 7fc60103a640 1 --2- 172.21.15.37:0/1302053793 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7fc5fc1609a0 0x7fc5fc160db0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:30:00.071 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:00.068+0000 7fc60103a640 1 -- 172.21.15.37:0/1302053793 >> v2:172.21.15.203:3300/0 conn(0x7fc5fc143500 msgr2=0x7fc5fc06f100 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:30:00.071 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:00.068+0000 7fc60103a640 1 --2- 172.21.15.37:0/1302053793 >> v2:172.21.15.203:3300/0 conn(0x7fc5fc143500 0x7fc5fc06f100 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:30:00.071 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:00.068+0000 7fc60103a640 1 -- 172.21.15.37:0/1302053793 --> v2:172.21.15.37:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fc5fc0b6120 con 0x7fc5fc1460e0 2024-04-25T10:30:00.071 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:00.068+0000 7fc60183b640 1 --2- 172.21.15.37:0/1302053793 >> v2:172.21.15.203:3300/0 conn(0x7fc5fc143500 0x7fc5fc06f100 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T10:30:00.072 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:00.068+0000 7fc60103a640 1 --2- 172.21.15.37:0/1302053793 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fc5fc1460e0 0x7fc5fc0b6640 secure :-1 s=READY pgs=359 cs=0 l=1 rev1=1 crypto rx=0x7fc5fc072030 tx=0x7fc5f002a1e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=cb1c88e8d051ed89 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:30:00.072 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:00.068+0000 7fc600839640 1 -- 172.21.15.37:0/1302053793 <== mon.0 v2:172.21.15.37:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc5f0037070 con 0x7fc5fc1460e0 2024-04-25T10:30:00.072 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:00.069+0000 7fc600839640 1 -- 172.21.15.37:0/1302053793 <== mon.0 v2:172.21.15.37:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fc5f0032440 con 0x7fc5fc1460e0 2024-04-25T10:30:00.072 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:00.069+0000 7fc600839640 1 -- 172.21.15.37:0/1302053793 <== mon.0 v2:172.21.15.37:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc5f0032810 con 0x7fc5fc1460e0 2024-04-25T10:30:00.073 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:00.069+0000 7fc603a9f640 1 -- 172.21.15.37:0/1302053793 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fc5fc1460e0 msgr2=0x7fc5fc0b6640 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:30:00.073 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:00.069+0000 7fc603a9f640 1 --2- 172.21.15.37:0/1302053793 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fc5fc1460e0 0x7fc5fc0b6640 secure :-1 s=READY pgs=359 cs=0 l=1 rev1=1 crypto rx=0x7fc5fc072030 tx=0x7fc5f002a1e0 comp rx=0 tx=0).stop 2024-04-25T10:30:00.073 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:00.069+0000 7fc603a9f640 1 -- 172.21.15.37:0/1302053793 shutdown_connections 2024-04-25T10:30:00.073 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:00.069+0000 7fc603a9f640 1 --2- 172.21.15.37:0/1302053793 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fc5fc1460e0 0x7fc5fc0b6640 unknown :-1 s=CLOSED pgs=359 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:30:00.073 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:00.069+0000 7fc603a9f640 1 --2- 172.21.15.37:0/1302053793 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7fc5fc1609a0 0x7fc5fc160db0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:30:00.073 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:00.069+0000 7fc603a9f640 1 --2- 172.21.15.37:0/1302053793 >> v2:172.21.15.203:3300/0 conn(0x7fc5fc143500 0x7fc5fc06f100 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:30:00.073 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:00.069+0000 7fc603a9f640 1 -- 172.21.15.37:0/1302053793 >> 172.21.15.37:0/1302053793 conn(0x7fc5fc0b36b0 msgr2=0x7fc5fc0b3ac0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:30:00.073 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:00.069+0000 7fc603a9f640 1 -- 172.21.15.37:0/1302053793 shutdown_connections 2024-04-25T10:30:00.073 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:00.069+0000 7fc603a9f640 1 -- 172.21.15.37:0/1302053793 wait complete. 2024-04-25T10:30:00.074 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:00.071+0000 7fc603a9f640 1 Processor -- start 2024-04-25T10:30:00.074 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:00.071+0000 7fc603a9f640 1 -- start start 2024-04-25T10:30:00.076 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:00.072+0000 7fc603a9f640 1 --2- >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7fc5fc1460e0 0x7fc5fc0c6920 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:30:00.076 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:00.072+0000 7fc603a9f640 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fc5fc1609a0 0x7fc5fc0c6e60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:30:00.076 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:00.072+0000 7fc603a9f640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fc5fc15f9b0 0x7fc5fc0c84a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:30:00.076 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:00.072+0000 7fc603a9f640 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 -- 0x7fc5fc143cb0 con 0x7fc5fc15f9b0 2024-04-25T10:30:00.076 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:00.072+0000 7fc603a9f640 1 -- --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_getmap magic: 0 -- 0x7fc5fc143b30 con 0x7fc5fc1609a0 2024-04-25T10:30:00.076 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:00.072+0000 7fc603a9f640 1 -- --> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] -- mon_getmap magic: 0 -- 0x7fc5fc143e30 con 0x7fc5fc1460e0 2024-04-25T10:30:00.076 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:00.072+0000 7fc60203c640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fc5fc15f9b0 0x7fc5fc0c84a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:30:00.076 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:00.073+0000 7fc60203c640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fc5fc15f9b0 0x7fc5fc0c84a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.37:3300/0 says I am v2:172.21.15.37:54474/0 (socket says 172.21.15.37:54474) 2024-04-25T10:30:00.076 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:00.073+0000 7fc60203c640 1 -- 172.21.15.37:0/714408441 learned_addr learned my addr 172.21.15.37:0/714408441 (peer_addr_for_me v2:172.21.15.37:0/0) 2024-04-25T10:30:00.077 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:00.073+0000 7fc60103a640 1 --2- 172.21.15.37:0/714408441 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fc5fc1609a0 0x7fc5fc0c6e60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:30:00.077 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:00.073+0000 7fc60183b640 1 --2- 172.21.15.37:0/714408441 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7fc5fc1460e0 0x7fc5fc0c6920 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:30:00.077 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:00.073+0000 7fc60203c640 1 -- 172.21.15.37:0/714408441 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7fc5fc1460e0 msgr2=0x7fc5fc0c6920 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:30:00.077 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:00.073+0000 7fc60203c640 1 --2- 172.21.15.37:0/714408441 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7fc5fc1460e0 0x7fc5fc0c6920 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:30:00.077 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:00.073+0000 7fc60203c640 1 -- 172.21.15.37:0/714408441 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fc5fc1609a0 msgr2=0x7fc5fc0c6e60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:30:00.077 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:00.073+0000 7fc60203c640 1 --2- 172.21.15.37:0/714408441 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fc5fc1609a0 0x7fc5fc0c6e60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:30:00.077 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:00.073+0000 7fc60203c640 1 -- 172.21.15.37:0/714408441 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fc5fc160120 con 0x7fc5fc15f9b0 2024-04-25T10:30:00.077 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:00.073+0000 7fc60183b640 1 --2- 172.21.15.37:0/714408441 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7fc5fc1460e0 0x7fc5fc0c6920 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T10:30:00.077 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:00.073+0000 7fc60203c640 1 --2- 172.21.15.37:0/714408441 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fc5fc15f9b0 0x7fc5fc0c84a0 secure :-1 s=READY pgs=360 cs=0 l=1 rev1=1 crypto rx=0x7fc5ec00b9e0 tx=0x7fc5ec00bea0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:30:00.077 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:00.073+0000 7fc5fa7fc640 1 -- 172.21.15.37:0/714408441 <== mon.0 v2:172.21.15.37:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc5ec019070 con 0x7fc5fc15f9b0 2024-04-25T10:30:00.077 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:00.073+0000 7fc5fa7fc640 1 -- 172.21.15.37:0/714408441 <== mon.0 v2:172.21.15.37:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fc5ec00cb40 con 0x7fc5fc15f9b0 2024-04-25T10:30:00.077 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:00.073+0000 7fc603a9f640 1 -- 172.21.15.37:0/714408441 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fc5fc15d620 con 0x7fc5fc15f9b0 2024-04-25T10:30:00.077 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:00.073+0000 7fc5fa7fc640 1 -- 172.21.15.37:0/714408441 <== mon.0 v2:172.21.15.37:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc5ec014440 con 0x7fc5fc15f9b0 2024-04-25T10:30:00.077 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:00.073+0000 7fc603a9f640 1 -- 172.21.15.37:0/714408441 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fc5fc1f1e40 con 0x7fc5fc15f9b0 2024-04-25T10:30:00.078 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:00.074+0000 7fc603a9f640 1 -- 172.21.15.37:0/714408441 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fc5fc15fe90 con 0x7fc5fc15f9b0 2024-04-25T10:30:00.079 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:00.075+0000 7fc5fa7fc640 1 -- 172.21.15.37:0/714408441 <== mon.0 v2:172.21.15.37:3300/0 4 ==== mgrmap(e 5) ==== 96782+0+0 (secure 0 0 0) 0x7fc5ec01c070 con 0x7fc5fc15f9b0 2024-04-25T10:30:00.079 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:00.076+0000 7fc5fa7fc640 1 --2- 172.21.15.37:0/714408441 >> [v2:172.21.15.203:6832/2418205237,v1:172.21.15.203:6833/2418205237] conn(0x7fc5e80744e0 0x7fc5e80769f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:30:00.079 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:00.076+0000 7fc5fa7fc640 1 -- 172.21.15.37:0/714408441 <== mon.0 v2:172.21.15.37:3300/0 5 ==== osd_map(25..25 src has 1..25) ==== 5738+0+0 (secure 0 0 0) 0x7fc5ec0951d0 con 0x7fc5fc15f9b0 2024-04-25T10:30:00.079 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:00.076+0000 7fc60183b640 1 --2- 172.21.15.37:0/714408441 >> [v2:172.21.15.203:6832/2418205237,v1:172.21.15.203:6833/2418205237] conn(0x7fc5e80744e0 0x7fc5e80769f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:30:00.079 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:00.076+0000 7fc60183b640 1 --2- 172.21.15.37:0/714408441 >> [v2:172.21.15.203:6832/2418205237,v1:172.21.15.203:6833/2418205237] conn(0x7fc5e80744e0 0x7fc5e80769f0 secure :-1 s=READY pgs=185 cs=0 l=1 rev1=1 crypto rx=0x7fc5f4006fd0 tx=0x7fc5f4008040 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:30:00.081 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:00.079+0000 7fc5fa7fc640 1 -- 172.21.15.37:0/714408441 <== mon.0 v2:172.21.15.37:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187246 (secure 0 0 0) 0x7fc5ec023020 con 0x7fc5fc15f9b0 2024-04-25T10:30:00.268 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:00.266+0000 7fc603a9f640 1 -- 172.21.15.37:0/714408441 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7fc5fc06a140 con 0x7fc5fc15f9b0 2024-04-25T10:30:00.270 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:00.267+0000 7fc5fa7fc640 1 -- 172.21.15.37:0/714408441 <== mon.0 v2:172.21.15.37:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v25) ==== 74+0+16670 (secure 0 0 0) 0x7fc5ec05f460 con 0x7fc5fc15f9b0 2024-04-25T10:30:00.270 INFO:teuthology.orchestra.run.smithi037.stdout: 2024-04-25T10:30:00.270 INFO:teuthology.orchestra.run.smithi037.stdout:{"epoch":25,"fsid":"f4acda28-be34-492a-b286-994afd36f61d","created":"2024-04-25T10:28:43.985718+0000","modified":"2024-04-25T10:29:49.325719+0000","last_up_change":"2024-04-25T10:28:54.900654+0000","last_in_change":"2024-04-25T10:28:46.216973+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":3,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"luminous","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-04-25T10:28:56.202153+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"19","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2024-04-25T10:29:06.507244+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"22","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":2.25,"score_stable":2.25,"optimal_score":1,"raw_score_acting":2.25,"raw_score_stable":2.25,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2024-04-25T10:29:07.347844+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"23","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"a2f79028-b332-4166-9ebb-a7024a37ea22","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":21,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.37:6816","nonce":2746074720},{"type":"v1","addr":"172.21.15.37:6817","nonce":2746074720}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.37:6818","nonce":2746074720},{"type":"v1","addr":"172.21.15.37:6819","nonce":2746074720}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.37:6822","nonce":2746074720},{"type":"v1","addr":"172.21.15.37:6823","nonce":2746074720}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.37:6820","nonce":2746074720},{"type":"v1","addr":"172.21.15.37:6821","nonce":2746074720}]},"public_addr":"172.21.15.37:6817/2746074720","cluster_addr":"172.21.15.37:6819/2746074720","heartbeat_back_addr":"172.21.15.37:6823/2746074720","heartbeat_front_addr":"172.21.15.37:6821/2746074720","state":["exists","up"]},{"osd":1,"uuid":"973159de-c2ea-4083-aecc-69373d1c1132","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":21,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.37:6800","nonce":1690341662},{"type":"v1","addr":"172.21.15.37:6801","nonce":1690341662}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.37:6802","nonce":1690341662},{"type":"v1","addr":"172.21.15.37:6803","nonce":1690341662}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.37:6806","nonce":1690341662},{"type":"v1","addr":"172.21.15.37:6807","nonce":1690341662}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.37:6804","nonce":1690341662},{"type":"v1","addr":"172.21.15.37:6805","nonce":1690341662}]},"public_addr":"172.21.15.37:6801/1690341662","cluster_addr":"172.21.15.37:6803/1690341662","heartbeat_back_addr":"172.21.15.37:6807/1690341662","heartbeat_front_addr":"172.21.15.37:6805/1690341662","state":["exists","up"]},{"osd":2,"uuid":"4ae7a0e9-cb3f-484c-a75f-f3766b721c47","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":21,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.37:6808","nonce":2754669649},{"type":"v1","addr":"172.21.15.37:6809","nonce":2754669649}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.37:6810","nonce":2754669649},{"type":"v1","addr":"172.21.15.37:6811","nonce":2754669649}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.37:6814","nonce":2754669649},{"type":"v1","addr":"172.21.15.37:6815","nonce":2754669649}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.37:6812","nonce":2754669649},{"type":"v1","addr":"172.21.15.37:6813","nonce":2754669649}]},"public_addr":"172.21.15.37:6809/2754669649","cluster_addr":"172.21.15.37:6811/2754669649","heartbeat_back_addr":"172.21.15.37:6815/2754669649","heartbeat_front_addr":"172.21.15.37:6813/2754669649","state":["exists","up"]},{"osd":3,"uuid":"1ba4a746-1608-47a7-ace2-960a5cfae934","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":21,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.37:6824","nonce":1815290618},{"type":"v1","addr":"172.21.15.37:6825","nonce":1815290618}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.37:6826","nonce":1815290618},{"type":"v1","addr":"172.21.15.37:6827","nonce":1815290618}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.37:6830","nonce":1815290618},{"type":"v1","addr":"172.21.15.37:6831","nonce":1815290618}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.37:6828","nonce":1815290618},{"type":"v1","addr":"172.21.15.37:6829","nonce":1815290618}]},"public_addr":"172.21.15.37:6825/1815290618","cluster_addr":"172.21.15.37:6827/1815290618","heartbeat_back_addr":"172.21.15.37:6831/1815290618","heartbeat_front_addr":"172.21.15.37:6829/1815290618","state":["exists","up"]},{"osd":4,"uuid":"d4f59f5d-0413-4b79-87ee-bb212c1fcf62","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":21,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6800","nonce":4051532063},{"type":"v1","addr":"172.21.15.203:6801","nonce":4051532063}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6802","nonce":4051532063},{"type":"v1","addr":"172.21.15.203:6803","nonce":4051532063}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6806","nonce":4051532063},{"type":"v1","addr":"172.21.15.203:6807","nonce":4051532063}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6804","nonce":4051532063},{"type":"v1","addr":"172.21.15.203:6805","nonce":4051532063}]},"public_addr":"172.21.15.203:6801/4051532063","cluster_addr":"172.21.15.203:6803/4051532063","heartbeat_back_addr":"172.21.15.203:6807/4051532063","heartbeat_front_addr":"172.21.15.203:6805/4051532063","state":["exists","up"]},{"osd":5,"uuid":"69710a0e-41d8-4e7f-9a10-17ce540b67be","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":24,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6824","nonce":1203822890},{"type":"v1","addr":"172.21.15.203:6825","nonce":1203822890}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6826","nonce":1203822890},{"type":"v1","addr":"172.21.15.203:6827","nonce":1203822890}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6830","nonce":1203822890},{"type":"v1","addr":"172.21.15.203:6831","nonce":1203822890}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6828","nonce":1203822890},{"type":"v1","addr":"172.21.15.203:6829","nonce":1203822890}]},"public_addr":"172.21.15.203:6825/1203822890","cluster_addr":"172.21.15.203:6827/1203822890","heartbeat_back_addr":"172.21.15.203:6831/1203822890","heartbeat_front_addr":"172.21.15.203:6829/1203822890","state":["exists","up"]},{"osd":6,"uuid":"f29dd047-3b48-473d-a235-6642a6293f20","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":21,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6810","nonce":2309615014},{"type":"v1","addr":"172.21.15.203:6812","nonce":2309615014}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6814","nonce":2309615014},{"type":"v1","addr":"172.21.15.203:6816","nonce":2309615014}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6822","nonce":2309615014},{"type":"v1","addr":"172.21.15.203:6823","nonce":2309615014}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6818","nonce":2309615014},{"type":"v1","addr":"172.21.15.203:6820","nonce":2309615014}]},"public_addr":"172.21.15.203:6812/2309615014","cluster_addr":"172.21.15.203:6816/2309615014","heartbeat_back_addr":"172.21.15.203:6823/2309615014","heartbeat_front_addr":"172.21.15.203:6820/2309615014","state":["exists","up"]},{"osd":7,"uuid":"6e691644-378b-4fdc-8e86-fb5f56e31f29","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":21,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6808","nonce":3688099014},{"type":"v1","addr":"172.21.15.203:6809","nonce":3688099014}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6811","nonce":3688099014},{"type":"v1","addr":"172.21.15.203:6813","nonce":3688099014}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6819","nonce":3688099014},{"type":"v1","addr":"172.21.15.203:6821","nonce":3688099014}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6815","nonce":3688099014},{"type":"v1","addr":"172.21.15.203:6817","nonce":3688099014}]},"public_addr":"172.21.15.203:6809/3688099014","cluster_addr":"172.21.15.203:6813/3688099014","heartbeat_back_addr":"172.21.15.203:6821/3688099014","heartbeat_front_addr":"172.21.15.203:6817/3688099014","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T10:28:51.419141+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T10:28:51.366448+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T10:28:51.348276+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T10:28:51.380519+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T10:28:51.105812+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T10:28:51.384496+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T10:28:51.388755+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T10:28:51.366958+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[{"pgid":"3.31","mappings":[{"from":0,"to":7}]}],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-04-25T10:30:00.272 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:00.269+0000 7fc603a9f640 1 -- 172.21.15.37:0/714408441 >> [v2:172.21.15.203:6832/2418205237,v1:172.21.15.203:6833/2418205237] conn(0x7fc5e80744e0 msgr2=0x7fc5e80769f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:30:00.272 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:00.269+0000 7fc603a9f640 1 --2- 172.21.15.37:0/714408441 >> [v2:172.21.15.203:6832/2418205237,v1:172.21.15.203:6833/2418205237] conn(0x7fc5e80744e0 0x7fc5e80769f0 secure :-1 s=READY pgs=185 cs=0 l=1 rev1=1 crypto rx=0x7fc5f4006fd0 tx=0x7fc5f4008040 comp rx=0 tx=0).stop 2024-04-25T10:30:00.272 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:00.269+0000 7fc603a9f640 1 -- 172.21.15.37:0/714408441 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fc5fc15f9b0 msgr2=0x7fc5fc0c84a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:30:00.272 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:00.269+0000 7fc603a9f640 1 --2- 172.21.15.37:0/714408441 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fc5fc15f9b0 0x7fc5fc0c84a0 secure :-1 s=READY pgs=360 cs=0 l=1 rev1=1 crypto rx=0x7fc5ec00b9e0 tx=0x7fc5ec00bea0 comp rx=0 tx=0).stop 2024-04-25T10:30:00.272 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:00.269+0000 7fc603a9f640 1 -- 172.21.15.37:0/714408441 shutdown_connections 2024-04-25T10:30:00.272 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:00.269+0000 7fc603a9f640 1 --2- 172.21.15.37:0/714408441 >> [v2:172.21.15.203:6832/2418205237,v1:172.21.15.203:6833/2418205237] conn(0x7fc5e80744e0 0x7fc5e80769f0 unknown :-1 s=CLOSED pgs=185 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:30:00.272 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:00.269+0000 7fc603a9f640 1 --2- 172.21.15.37:0/714408441 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fc5fc15f9b0 0x7fc5fc0c84a0 unknown :-1 s=CLOSED pgs=360 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:30:00.272 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:00.269+0000 7fc603a9f640 1 --2- 172.21.15.37:0/714408441 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fc5fc1609a0 0x7fc5fc0c6e60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:30:00.272 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:00.269+0000 7fc603a9f640 1 --2- 172.21.15.37:0/714408441 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7fc5fc1460e0 0x7fc5fc0c6920 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:30:00.272 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:00.269+0000 7fc603a9f640 1 -- 172.21.15.37:0/714408441 >> 172.21.15.37:0/714408441 conn(0x7fc5fc0b36b0 msgr2=0x7fc5fc070500 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:30:00.273 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:00.269+0000 7fc603a9f640 1 -- 172.21.15.37:0/714408441 shutdown_connections 2024-04-25T10:30:00.273 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:00.269+0000 7fc603a9f640 1 -- 172.21.15.37:0/714408441 wait complete. 2024-04-25T10:30:00.283 INFO:tasks.cephfs.filesystem:Destroying file system cephfs and related pools 2024-04-25T10:30:00.283 DEBUG:teuthology.orchestra.run.smithi037:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-04-25T10:30:00.397 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:00.394+0000 7fa4bd09f640 1 Processor -- start 2024-04-25T10:30:00.397 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:00.394+0000 7fa4bd09f640 1 -- start start 2024-04-25T10:30:00.400 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:00.396+0000 7fa4bd09f640 1 --2- >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7fa4b80d3230 0x7fa4b816b0c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:30:00.400 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:00.396+0000 7fa4bd09f640 1 -- --> v1:172.21.15.37:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fa4b80d10a0 con 0x7fa4b80d5160 2024-04-25T10:30:00.400 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:00.396+0000 7fa4bd09f640 1 -- --> v1:172.21.15.203:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fa4b80d1370 con 0x7fa4b80b4e80 2024-04-25T10:30:00.400 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:00.396+0000 7fa4bd09f640 1 -- --> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] -- mon_getmap magic: 0 -- 0x7fa4b80d1650 con 0x7fa4b80d3230 2024-04-25T10:30:00.400 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:00.396+0000 7fa4b6d9d640 1 --1- >> v1:172.21.15.37:6789/0 conn(0x7fa4b80d5160 0x7fa4b80d0990 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.37:6789/0 says I am v1:172.21.15.37:60640/0 (socket says 172.21.15.37:60640) 2024-04-25T10:30:00.400 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:00.396+0000 7fa4b6d9d640 1 -- 172.21.15.37:0/3666118867 learned_addr learned my addr 172.21.15.37:0/3666118867 (peer_addr_for_me v1:172.21.15.37:0/0) 2024-04-25T10:30:00.400 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:00.396+0000 7fa4b5d9b640 1 -- 172.21.15.37:0/3666118867 <== mon.0 v1:172.21.15.37:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 1853933555 0 0) 0x7fa4b80d10a0 con 0x7fa4b80d5160 2024-04-25T10:30:00.400 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:00.396+0000 7fa4b759e640 1 --2- 172.21.15.37:0/3666118867 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7fa4b80d3230 0x7fa4b816b0c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:30:00.400 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:00.396+0000 7fa4b5d9b640 1 -- 172.21.15.37:0/3666118867 --> v1:172.21.15.37:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7fa4a0003520 con 0x7fa4b80d5160 2024-04-25T10:30:00.400 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:00.396+0000 7fa4b5d9b640 1 -- 172.21.15.37:0/3666118867 <== mon.0 v1:172.21.15.37:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 665357030 0 0) 0x7fa4a0003520 con 0x7fa4b80d5160 2024-04-25T10:30:00.400 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:00.396+0000 7fa4b5d9b640 1 -- 172.21.15.37:0/3666118867 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7fa4b80d3230 msgr2=0x7fa4b816b0c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:30:00.401 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:00.396+0000 7fa4b5d9b640 1 --2- 172.21.15.37:0/3666118867 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7fa4b80d3230 0x7fa4b816b0c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:30:00.401 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:00.396+0000 7fa4b5d9b640 1 -- 172.21.15.37:0/3666118867 >> v1:172.21.15.203:6789/0 conn(0x7fa4b80b4e80 legacy=0x7fa4b80d2b20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:30:00.401 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:00.396+0000 7fa4b5d9b640 1 -- 172.21.15.37:0/3666118867 --> v1:172.21.15.37:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fa4b80d4680 con 0x7fa4b80d5160 2024-04-25T10:30:00.401 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:00.396+0000 7fa4b5d9b640 1 -- 172.21.15.37:0/3666118867 <== mon.0 v1:172.21.15.37:6789/0 3 ==== mon_map magic: 0 ==== 413+0+0 (unknown 1577794600 0 0) 0x7fa4b0001690 con 0x7fa4b80d5160 2024-04-25T10:30:00.401 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:00.396+0000 7fa4b5d9b640 1 -- 172.21.15.37:0/3666118867 >> v1:172.21.15.37:6789/0 conn(0x7fa4b80d5160 legacy=0x7fa4b80d0990 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:30:00.401 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:00.397+0000 7fa4b5d9b640 1 --2- 172.21.15.37:0/3666118867 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fa4a00039e0 0x7fa4a0003e50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:30:00.401 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:00.397+0000 7fa4b5d9b640 1 --2- 172.21.15.37:0/3666118867 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7fa4a0004800 0x7fa4a0004c10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:30:00.401 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:00.397+0000 7fa4b5d9b640 1 --2- 172.21.15.37:0/3666118867 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fa4a0005150 0x7fa4a000b9c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:30:00.401 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:00.397+0000 7fa4b5d9b640 1 -- 172.21.15.37:0/3666118867 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 -- 0x7fa4b80d1650 con 0x7fa4a0005150 2024-04-25T10:30:00.401 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:00.397+0000 7fa4b5d9b640 1 -- 172.21.15.37:0/3666118867 --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_getmap magic: 0 -- 0x7fa4a000bfe0 con 0x7fa4a00039e0 2024-04-25T10:30:00.401 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:00.397+0000 7fa4b5d9b640 1 -- 172.21.15.37:0/3666118867 --> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] -- mon_getmap magic: 0 -- 0x7fa4a000c1e0 con 0x7fa4a0004800 2024-04-25T10:30:00.401 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:00.397+0000 7fa4b759e640 1 --2- 172.21.15.37:0/3666118867 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fa4a0005150 0x7fa4a000b9c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:30:00.401 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:00.397+0000 7fa4b6d9d640 1 --2- 172.21.15.37:0/3666118867 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fa4a00039e0 0x7fa4a0003e50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:30:00.402 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:00.397+0000 7fa4b659c640 1 --2- 172.21.15.37:0/3666118867 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7fa4a0004800 0x7fa4a0004c10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:30:00.402 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:00.397+0000 7fa4b759e640 1 -- 172.21.15.37:0/3666118867 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7fa4a0004800 msgr2=0x7fa4a0004c10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:30:00.402 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:00.397+0000 7fa4b759e640 1 --2- 172.21.15.37:0/3666118867 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7fa4a0004800 0x7fa4a0004c10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:30:00.402 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:00.397+0000 7fa4b759e640 1 -- 172.21.15.37:0/3666118867 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fa4a00039e0 msgr2=0x7fa4a0003e50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:30:00.402 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:00.397+0000 7fa4b759e640 1 --2- 172.21.15.37:0/3666118867 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fa4a00039e0 0x7fa4a0003e50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:30:00.402 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:00.397+0000 7fa4b759e640 1 -- 172.21.15.37:0/3666118867 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fa4a0003520 con 0x7fa4a0005150 2024-04-25T10:30:00.402 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:00.398+0000 7fa4b759e640 1 --2- 172.21.15.37:0/3666118867 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fa4a0005150 0x7fa4a000b9c0 secure :-1 s=READY pgs=362 cs=0 l=1 rev1=1 crypto rx=0x7fa4b808f070 tx=0x7fa4a0005830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=8ea63deccdc08b60 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:30:00.402 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:00.398+0000 7fa4b5d9b640 1 -- 172.21.15.37:0/3666118867 <== mon.0 v2:172.21.15.37:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa4a8009c10 con 0x7fa4a0005150 2024-04-25T10:30:00.402 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:00.398+0000 7fa4b5d9b640 1 -- 172.21.15.37:0/3666118867 <== mon.0 v2:172.21.15.37:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fa4a8036070 con 0x7fa4a0005150 2024-04-25T10:30:00.402 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:00.398+0000 7fa4b5d9b640 1 -- 172.21.15.37:0/3666118867 <== mon.0 v2:172.21.15.37:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa4a8031830 con 0x7fa4a0005150 2024-04-25T10:30:00.403 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:00.398+0000 7fa4bd09f640 1 -- 172.21.15.37:0/3666118867 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fa4a0005150 msgr2=0x7fa4a000b9c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:30:00.403 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:00.398+0000 7fa4bd09f640 1 --2- 172.21.15.37:0/3666118867 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fa4a0005150 0x7fa4a000b9c0 secure :-1 s=READY pgs=362 cs=0 l=1 rev1=1 crypto rx=0x7fa4b808f070 tx=0x7fa4a0005830 comp rx=0 tx=0).stop 2024-04-25T10:30:00.403 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:00.398+0000 7fa4b759e640 1 -- 172.21.15.37:0/3666118867 reap_dead start 2024-04-25T10:30:00.403 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:00.398+0000 7fa4bd09f640 1 -- 172.21.15.37:0/3666118867 shutdown_connections 2024-04-25T10:30:00.404 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:00.398+0000 7fa4bd09f640 1 -- 172.21.15.37:0/3666118867 >> 172.21.15.37:0/3666118867 conn(0x7fa4b80b7a80 msgr2=0x7fa4b80b7e90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:30:00.404 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:00.399+0000 7fa4bd09f640 1 -- 172.21.15.37:0/3666118867 shutdown_connections 2024-04-25T10:30:00.404 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:00.399+0000 7fa4bd09f640 1 -- 172.21.15.37:0/3666118867 wait complete. 2024-04-25T10:30:00.404 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:00.401+0000 7fa4bd09f640 1 Processor -- start 2024-04-25T10:30:00.404 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:00.401+0000 7fa4bd09f640 1 -- start start 2024-04-25T10:30:00.406 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:00.402+0000 7fa4bd09f640 1 --2- >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7fa4a0005150 0x7fa4b80cc5d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:30:00.406 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:00.402+0000 7fa4bd09f640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fa4b8151960 0x7fa4b814e870 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:30:00.406 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:00.402+0000 7fa4bd09f640 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fa4b80cedd0 0x7fa4b80cf1e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:30:00.406 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:00.402+0000 7fa4bd09f640 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 -- 0x7fa4b80d4c40 con 0x7fa4b8151960 2024-04-25T10:30:00.406 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:00.402+0000 7fa4bd09f640 1 -- --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_getmap magic: 0 -- 0x7fa4b80b5290 con 0x7fa4b80cedd0 2024-04-25T10:30:00.406 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:00.402+0000 7fa4bd09f640 1 -- --> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] -- mon_getmap magic: 0 -- 0x7fa4b8164ac0 con 0x7fa4a0005150 2024-04-25T10:30:00.406 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:00.402+0000 7fa4b659c640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fa4b8151960 0x7fa4b814e870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:30:00.406 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:00.402+0000 7fa4b659c640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fa4b8151960 0x7fa4b814e870 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.37:3300/0 says I am v2:172.21.15.37:54488/0 (socket says 172.21.15.37:54488) 2024-04-25T10:30:00.407 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:00.402+0000 7fa4b659c640 1 -- 172.21.15.37:0/796169545 learned_addr learned my addr 172.21.15.37:0/796169545 (peer_addr_for_me v2:172.21.15.37:0/0) 2024-04-25T10:30:00.407 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:00.402+0000 7fa4b6d9d640 1 --2- 172.21.15.37:0/796169545 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7fa4a0005150 0x7fa4b80cc5d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:30:00.407 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:00.402+0000 7fa4b759e640 1 --2- 172.21.15.37:0/796169545 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fa4b80cedd0 0x7fa4b80cf1e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:30:00.407 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:00.402+0000 7fa4b659c640 1 -- 172.21.15.37:0/796169545 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7fa4a0005150 msgr2=0x7fa4b80cc5d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:30:00.407 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:00.402+0000 7fa4b659c640 1 --2- 172.21.15.37:0/796169545 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7fa4a0005150 0x7fa4b80cc5d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:30:00.407 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:00.402+0000 7fa4b659c640 1 -- 172.21.15.37:0/796169545 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fa4b80cedd0 msgr2=0x7fa4b80cf1e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:30:00.407 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:00.402+0000 7fa4b659c640 1 --2- 172.21.15.37:0/796169545 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fa4b80cedd0 0x7fa4b80cf1e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:30:00.407 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:00.402+0000 7fa4b659c640 1 -- 172.21.15.37:0/796169545 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fa4b80cf720 con 0x7fa4b8151960 2024-04-25T10:30:00.407 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:00.403+0000 7fa4b659c640 1 --2- 172.21.15.37:0/796169545 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fa4b8151960 0x7fa4b814e870 secure :-1 s=READY pgs=363 cs=0 l=1 rev1=1 crypto rx=0x7fa4ac00f820 tx=0x7fa4ac00fce0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:30:00.407 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:00.403+0000 7fa49f7fe640 1 -- 172.21.15.37:0/796169545 <== mon.0 v2:172.21.15.37:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa4ac0050d0 con 0x7fa4b8151960 2024-04-25T10:30:00.407 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:00.403+0000 7fa49f7fe640 1 -- 172.21.15.37:0/796169545 <== mon.0 v2:172.21.15.37:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fa4ac018070 con 0x7fa4b8151960 2024-04-25T10:30:00.407 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:00.403+0000 7fa49f7fe640 1 -- 172.21.15.37:0/796169545 <== mon.0 v2:172.21.15.37:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa4ac005490 con 0x7fa4b8151960 2024-04-25T10:30:00.407 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:00.403+0000 7fa4bd09f640 1 -- 172.21.15.37:0/796169545 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fa4b814f0d0 con 0x7fa4b8151960 2024-04-25T10:30:00.408 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:00.403+0000 7fa4bd09f640 1 -- 172.21.15.37:0/796169545 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fa4b814f690 con 0x7fa4b8151960 2024-04-25T10:30:00.408 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:00.404+0000 7fa4bd09f640 1 -- 172.21.15.37:0/796169545 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fa4b806a140 con 0x7fa4b8151960 2024-04-25T10:30:00.409 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:00.405+0000 7fa49f7fe640 1 -- 172.21.15.37:0/796169545 <== mon.0 v2:172.21.15.37:3300/0 4 ==== mgrmap(e 5) ==== 96782+0+0 (secure 0 0 0) 0x7fa4ac00d730 con 0x7fa4b8151960 2024-04-25T10:30:00.409 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:00.405+0000 7fa49f7fe640 1 --2- 172.21.15.37:0/796169545 >> [v2:172.21.15.203:6832/2418205237,v1:172.21.15.203:6833/2418205237] conn(0x7fa4a40744b0 0x7fa4a40769c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:30:00.409 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:00.405+0000 7fa49f7fe640 1 -- 172.21.15.37:0/796169545 <== mon.0 v2:172.21.15.37:3300/0 5 ==== osd_map(25..25 src has 1..25) ==== 5738+0+0 (secure 0 0 0) 0x7fa4ac094f90 con 0x7fa4b8151960 2024-04-25T10:30:00.409 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:00.405+0000 7fa4b6d9d640 1 --2- 172.21.15.37:0/796169545 >> [v2:172.21.15.203:6832/2418205237,v1:172.21.15.203:6833/2418205237] conn(0x7fa4a40744b0 0x7fa4a40769c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:30:00.409 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:00.406+0000 7fa4b6d9d640 1 --2- 172.21.15.37:0/796169545 >> [v2:172.21.15.203:6832/2418205237,v1:172.21.15.203:6833/2418205237] conn(0x7fa4a40744b0 0x7fa4a40769c0 secure :-1 s=READY pgs=186 cs=0 l=1 rev1=1 crypto rx=0x7fa4b0001800 tx=0x7fa4b0001450 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:30:00.411 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:00.410+0000 7fa49f7fe640 1 -- 172.21.15.37:0/796169545 <== mon.0 v2:172.21.15.37:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187246 (secure 0 0 0) 0x7fa4b80cf720 con 0x7fa4b8151960 2024-04-25T10:30:00.637 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:00.634+0000 7fa4bd09f640 1 -- 172.21.15.37:0/796169545 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7fa4b80d3f50 con 0x7fa4b8151960 2024-04-25T10:30:00.637 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:00.635+0000 7fa49f7fe640 1 -- 172.21.15.37:0/796169545 <== mon.0 v2:172.21.15.37:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 8 v8) ==== 93+0+5178 (secure 0 0 0) 0x7fa4ac05f0b0 con 0x7fa4b8151960 2024-04-25T10:30:00.637 INFO:teuthology.orchestra.run.smithi037.stdout: 2024-04-25T10:30:00.637 INFO:teuthology.orchestra.run.smithi037.stdout:{"epoch":8,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":4232,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.37:6835/1911169480","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.37:6833","nonce":1911169480},{"type":"v1","addr":"172.21.15.37:6835","nonce":1911169480}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3},{"gid":4241,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.203:6835/3873243328","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6834","nonce":3873243328},{"type":"v1","addr":"172.21.15.203:6835","nonce":3873243328}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3},{"gid":4278,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.37:6834/179723013","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.37:6832","nonce":179723013},{"type":"v1","addr":"172.21.15.37:6834","nonce":179723013}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":2}],"filesystems":[{"mdsmap":{"epoch":8,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-04-25T10:29:08.397829+0000","modified":"2024-04-25T10:29:11.407694+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":1,"in":[0],"up":{"mds_0":4279},"failed":[],"damaged":[],"stopped":[],"info":{"gid_4279":{"gid":4279,"name":"d","rank":0,"incarnation":4,"state":"up:active","state_seq":3,"addr":"172.21.15.203:6837/3215191422","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6836","nonce":3215191422},{"type":"v1","addr":"172.21.15.203:6837","nonce":3215191422}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":4279,"qdb_cluster":[4279]},"id":1}]} 2024-04-25T10:30:00.638 INFO:teuthology.orchestra.run.smithi037.stderr:dumped fsmap epoch 8 2024-04-25T10:30:00.639 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:00.636+0000 7fa4bd09f640 1 -- 172.21.15.37:0/796169545 >> [v2:172.21.15.203:6832/2418205237,v1:172.21.15.203:6833/2418205237] conn(0x7fa4a40744b0 msgr2=0x7fa4a40769c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:30:00.639 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:00.636+0000 7fa4bd09f640 1 --2- 172.21.15.37:0/796169545 >> [v2:172.21.15.203:6832/2418205237,v1:172.21.15.203:6833/2418205237] conn(0x7fa4a40744b0 0x7fa4a40769c0 secure :-1 s=READY pgs=186 cs=0 l=1 rev1=1 crypto rx=0x7fa4b0001800 tx=0x7fa4b0001450 comp rx=0 tx=0).stop 2024-04-25T10:30:00.639 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:00.636+0000 7fa4bd09f640 1 -- 172.21.15.37:0/796169545 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fa4b8151960 msgr2=0x7fa4b814e870 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:30:00.639 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:00.636+0000 7fa4bd09f640 1 --2- 172.21.15.37:0/796169545 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fa4b8151960 0x7fa4b814e870 secure :-1 s=READY pgs=363 cs=0 l=1 rev1=1 crypto rx=0x7fa4ac00f820 tx=0x7fa4ac00fce0 comp rx=0 tx=0).stop 2024-04-25T10:30:00.639 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:00.636+0000 7fa4bd09f640 1 -- 172.21.15.37:0/796169545 shutdown_connections 2024-04-25T10:30:00.639 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:00.636+0000 7fa4bd09f640 1 --2- 172.21.15.37:0/796169545 >> [v2:172.21.15.203:6832/2418205237,v1:172.21.15.203:6833/2418205237] conn(0x7fa4a40744b0 0x7fa4a40769c0 unknown :-1 s=CLOSED pgs=186 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:30:00.639 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:00.636+0000 7fa4bd09f640 1 --2- 172.21.15.37:0/796169545 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fa4b80cedd0 0x7fa4b80cf1e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:30:00.639 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:00.636+0000 7fa4bd09f640 1 --2- 172.21.15.37:0/796169545 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fa4b8151960 0x7fa4b814e870 unknown :-1 s=CLOSED pgs=363 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:30:00.639 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:00.636+0000 7fa4bd09f640 1 --2- 172.21.15.37:0/796169545 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7fa4a0005150 0x7fa4b80cc5d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:30:00.639 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:00.636+0000 7fa4bd09f640 1 -- 172.21.15.37:0/796169545 >> 172.21.15.37:0/796169545 conn(0x7fa4b80b7a80 msgr2=0x7fa4b80b5f00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:30:00.640 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:00.637+0000 7fa4bd09f640 1 -- 172.21.15.37:0/796169545 shutdown_connections 2024-04-25T10:30:00.640 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:00.637+0000 7fa4bd09f640 1 -- 172.21.15.37:0/796169545 wait complete. 2024-04-25T10:30:00.650 DEBUG:teuthology.orchestra.run.smithi037:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-04-25T10:30:00.765 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:00.763+0000 7fe0a9eaf640 1 Processor -- start 2024-04-25T10:30:00.765 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:00.763+0000 7fe0a9eaf640 1 -- start start 2024-04-25T10:30:00.767 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:00.764+0000 7fe0a9eaf640 1 --2- >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7fe0a40d5430 0x7fe0a40b75c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:30:00.767 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:00.764+0000 7fe0a9eaf640 1 --2- >> v2:172.21.15.37:3300/0 conn(0x7fe0a40b7b00 0x7fe0a4160a00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:30:00.767 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:00.764+0000 7fe0a9eaf640 1 -- --> v1:172.21.15.37:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fe0a40b7f10 con 0x7fe0a4160f40 2024-04-25T10:30:00.767 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:00.764+0000 7fe0a9eaf640 1 -- --> v2:172.21.15.37:3300/0 -- mon_getmap magic: 0 -- 0x7fe0a40d4420 con 0x7fe0a40b7b00 2024-04-25T10:30:00.767 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:00.764+0000 7fe0a9eaf640 1 -- --> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] -- mon_getmap magic: 0 -- 0x7fe0a40d5840 con 0x7fe0a40d5430 2024-04-25T10:30:00.767 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:00.764+0000 7fe0a2ffd640 1 --2- >> v2:172.21.15.37:3300/0 conn(0x7fe0a40b7b00 0x7fe0a4160a00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:30:00.767 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:00.764+0000 7fe0a3fff640 1 --1- >> v1:172.21.15.37:6789/0 conn(0x7fe0a4160f40 0x7fe0a4161880 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.37:6789/0 says I am v1:172.21.15.37:60648/0 (socket says 172.21.15.37:60648) 2024-04-25T10:30:00.767 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:00.764+0000 7fe0a3fff640 1 -- 172.21.15.37:0/2233431529 learned_addr learned my addr 172.21.15.37:0/2233431529 (peer_addr_for_me v1:172.21.15.37:0/0) 2024-04-25T10:30:00.767 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:00.764+0000 7fe0a2ffd640 1 --2- >> v2:172.21.15.37:3300/0 conn(0x7fe0a40b7b00 0x7fe0a4160a00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.37:3300/0 says I am v2:172.21.15.37:54502/0 (socket says 172.21.15.37:54502) 2024-04-25T10:30:00.767 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:00.764+0000 7fe0a37fe640 1 --2- 172.21.15.37:0/2233431529 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7fe0a40d5430 0x7fe0a40b75c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:30:00.767 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:00.764+0000 7fe0a2ffd640 1 -- 172.21.15.37:0/2233431529 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7fe0a40d5430 msgr2=0x7fe0a40b75c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:30:00.768 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:00.764+0000 7fe0a2ffd640 1 --2- 172.21.15.37:0/2233431529 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7fe0a40d5430 0x7fe0a40b75c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:30:00.768 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:00.764+0000 7fe0a2ffd640 1 -- 172.21.15.37:0/2233431529 >> v1:172.21.15.37:6789/0 conn(0x7fe0a4160f40 legacy=0x7fe0a4161880 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:30:00.768 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:00.764+0000 7fe0a27fc640 1 -- 172.21.15.37:0/2233431529 <== mon.0 v1:172.21.15.37:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 340799429 0 0) 0x7fe09c001490 con 0x7fe0a4160f40 2024-04-25T10:30:00.768 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:00.764+0000 7fe0a2ffd640 1 -- 172.21.15.37:0/2233431529 --> v2:172.21.15.37:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fe0a40d4670 con 0x7fe0a40b7b00 2024-04-25T10:30:00.770 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:00.765+0000 7fe0a2ffd640 1 --2- 172.21.15.37:0/2233431529 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fe0a40b7b00 0x7fe0a4160a00 secure :-1 s=READY pgs=365 cs=0 l=1 rev1=1 crypto rx=0x7fe0980097a0 tx=0x7fe09802a100 comp rx=0 tx=0).ready entity=mon.0 client_cookie=10a5dda0b05b849f server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:30:00.770 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:00.765+0000 7fe0a27fc640 1 -- 172.21.15.37:0/2233431529 <== mon.0 v2:172.21.15.37:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe098037070 con 0x7fe0a40b7b00 2024-04-25T10:30:00.770 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:00.765+0000 7fe0a27fc640 1 -- 172.21.15.37:0/2233431529 <== mon.0 v2:172.21.15.37:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fe09802ae50 con 0x7fe0a40b7b00 2024-04-25T10:30:00.770 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:00.765+0000 7fe0a27fc640 1 -- 172.21.15.37:0/2233431529 <== mon.0 v2:172.21.15.37:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe098032660 con 0x7fe0a40b7b00 2024-04-25T10:30:00.770 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:00.766+0000 7fe0a9eaf640 1 -- 172.21.15.37:0/2233431529 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fe0a40b7b00 msgr2=0x7fe0a4160a00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:30:00.770 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:00.766+0000 7fe0a9eaf640 1 --2- 172.21.15.37:0/2233431529 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fe0a40b7b00 0x7fe0a4160a00 secure :-1 s=READY pgs=365 cs=0 l=1 rev1=1 crypto rx=0x7fe0980097a0 tx=0x7fe09802a100 comp rx=0 tx=0).stop 2024-04-25T10:30:00.770 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:00.766+0000 7fe0a9eaf640 1 -- 172.21.15.37:0/2233431529 shutdown_connections 2024-04-25T10:30:00.770 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:00.766+0000 7fe0a9eaf640 1 --2- 172.21.15.37:0/2233431529 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fe0a40b7b00 0x7fe0a4160a00 unknown :-1 s=CLOSED pgs=365 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:30:00.770 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:00.766+0000 7fe0a9eaf640 1 --2- 172.21.15.37:0/2233431529 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7fe0a40d5430 0x7fe0a40b75c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:30:00.770 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:00.766+0000 7fe0a9eaf640 1 -- 172.21.15.37:0/2233431529 >> 172.21.15.37:0/2233431529 conn(0x7fe0a40d27a0 msgr2=0x7fe0a40d2bb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:30:00.771 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:00.766+0000 7fe0a9eaf640 1 -- 172.21.15.37:0/2233431529 shutdown_connections 2024-04-25T10:30:00.771 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:00.766+0000 7fe0a9eaf640 1 -- 172.21.15.37:0/2233431529 wait complete. 2024-04-25T10:30:00.771 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:00.768+0000 7fe0a9eaf640 1 Processor -- start 2024-04-25T10:30:00.771 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:00.768+0000 7fe0a9eaf640 1 -- start start 2024-04-25T10:30:00.773 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:00.769+0000 7fe0a9eaf640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fe0a40b7b00 0x7fe0a40ccbf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:30:00.773 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:00.769+0000 7fe0a9eaf640 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fe0a40d5430 0x7fe0a4162d70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:30:00.773 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:00.769+0000 7fe0a9eaf640 1 --2- >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7fe0a4160f40 0x7fe0a41508d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:30:00.773 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:00.769+0000 7fe0a9eaf640 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 -- 0x7fe0a40d4f10 con 0x7fe0a40b7b00 2024-04-25T10:30:00.774 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:00.769+0000 7fe0a9eaf640 1 -- --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_getmap magic: 0 -- 0x7fe0a40d4c30 con 0x7fe0a40d5430 2024-04-25T10:30:00.774 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:00.769+0000 7fe0a9eaf640 1 -- --> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] -- mon_getmap magic: 0 -- 0x7fe0a4161350 con 0x7fe0a4160f40 2024-04-25T10:30:00.774 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:00.769+0000 7fe0a37fe640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fe0a40b7b00 0x7fe0a40ccbf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:30:00.774 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:00.769+0000 7fe0a37fe640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fe0a40b7b00 0x7fe0a40ccbf0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.37:3300/0 says I am v2:172.21.15.37:54514/0 (socket says 172.21.15.37:54514) 2024-04-25T10:30:00.774 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:00.769+0000 7fe0a37fe640 1 -- 172.21.15.37:0/1677943743 learned_addr learned my addr 172.21.15.37:0/1677943743 (peer_addr_for_me v2:172.21.15.37:0/0) 2024-04-25T10:30:00.774 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:00.770+0000 7fe0a3fff640 1 --2- 172.21.15.37:0/1677943743 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7fe0a4160f40 0x7fe0a41508d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:30:00.774 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:00.770+0000 7fe0a2ffd640 1 --2- 172.21.15.37:0/1677943743 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fe0a40d5430 0x7fe0a4162d70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:30:00.774 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:00.770+0000 7fe0a37fe640 1 -- 172.21.15.37:0/1677943743 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7fe0a4160f40 msgr2=0x7fe0a41508d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:30:00.774 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:00.770+0000 7fe0a37fe640 1 --2- 172.21.15.37:0/1677943743 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7fe0a4160f40 0x7fe0a41508d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:30:00.774 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:00.770+0000 7fe0a37fe640 1 -- 172.21.15.37:0/1677943743 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fe0a40d5430 msgr2=0x7fe0a4162d70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:30:00.774 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:00.770+0000 7fe0a37fe640 1 --2- 172.21.15.37:0/1677943743 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fe0a40d5430 0x7fe0a4162d70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:30:00.774 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:00.770+0000 7fe0a37fe640 1 -- 172.21.15.37:0/1677943743 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fe0a414e2f0 con 0x7fe0a40b7b00 2024-04-25T10:30:00.775 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:00.770+0000 7fe0a3fff640 1 --2- 172.21.15.37:0/1677943743 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7fe0a4160f40 0x7fe0a41508d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T10:30:00.775 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:00.770+0000 7fe0a2ffd640 1 --2- 172.21.15.37:0/1677943743 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fe0a40d5430 0x7fe0a4162d70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T10:30:00.775 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:00.770+0000 7fe0a37fe640 1 --2- 172.21.15.37:0/1677943743 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fe0a40b7b00 0x7fe0a40ccbf0 secure :-1 s=READY pgs=366 cs=0 l=1 rev1=1 crypto rx=0x7fe09400e870 tx=0x7fe09400ed30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:30:00.775 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:00.770+0000 7fe08bfff640 1 -- 172.21.15.37:0/1677943743 <== mon.0 v2:172.21.15.37:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe094017070 con 0x7fe0a40b7b00 2024-04-25T10:30:00.775 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:00.770+0000 7fe08bfff640 1 -- 172.21.15.37:0/1677943743 <== mon.0 v2:172.21.15.37:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fe09400c940 con 0x7fe0a40b7b00 2024-04-25T10:30:00.775 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:00.770+0000 7fe0a9eaf640 1 -- 172.21.15.37:0/1677943743 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fe0a41f9820 con 0x7fe0a40b7b00 2024-04-25T10:30:00.775 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:00.770+0000 7fe08bfff640 1 -- 172.21.15.37:0/1677943743 <== mon.0 v2:172.21.15.37:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe09400cd10 con 0x7fe0a40b7b00 2024-04-25T10:30:00.775 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:00.770+0000 7fe0a9eaf640 1 -- 172.21.15.37:0/1677943743 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fe0a41f9de0 con 0x7fe0a40b7b00 2024-04-25T10:30:00.775 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:00.771+0000 7fe0a9eaf640 1 -- 172.21.15.37:0/1677943743 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fe0a406a140 con 0x7fe0a40b7b00 2024-04-25T10:30:00.777 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:00.772+0000 7fe08bfff640 1 -- 172.21.15.37:0/1677943743 <== mon.0 v2:172.21.15.37:3300/0 4 ==== mgrmap(e 5) ==== 96782+0+0 (secure 0 0 0) 0x7fe09401a070 con 0x7fe0a40b7b00 2024-04-25T10:30:00.777 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:00.772+0000 7fe08bfff640 1 --2- 172.21.15.37:0/1677943743 >> [v2:172.21.15.203:6832/2418205237,v1:172.21.15.203:6833/2418205237] conn(0x7fe0900744e0 0x7fe0900769f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:30:00.777 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:00.772+0000 7fe08bfff640 1 -- 172.21.15.37:0/1677943743 <== mon.0 v2:172.21.15.37:3300/0 5 ==== osd_map(25..25 src has 1..25) ==== 5738+0+0 (secure 0 0 0) 0x7fe0940953f0 con 0x7fe0a40b7b00 2024-04-25T10:30:00.777 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:00.772+0000 7fe0a2ffd640 1 --2- 172.21.15.37:0/1677943743 >> [v2:172.21.15.203:6832/2418205237,v1:172.21.15.203:6833/2418205237] conn(0x7fe0900744e0 0x7fe0900769f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:30:00.778 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:00.773+0000 7fe0a2ffd640 1 --2- 172.21.15.37:0/1677943743 >> [v2:172.21.15.203:6832/2418205237,v1:172.21.15.203:6833/2418205237] conn(0x7fe0900744e0 0x7fe0900769f0 secure :-1 s=READY pgs=187 cs=0 l=1 rev1=1 crypto rx=0x7fe098034000 tx=0x7fe09802e000 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:30:00.779 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:00.777+0000 7fe08bfff640 1 -- 172.21.15.37:0/1677943743 <== mon.0 v2:172.21.15.37:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187246 (secure 0 0 0) 0x7fe09405fbf0 con 0x7fe0a40b7b00 2024-04-25T10:30:01.001 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:00.999+0000 7fe0a9eaf640 1 -- 172.21.15.37:0/1677943743 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7fe0a40d0f00 con 0x7fe0a40b7b00 2024-04-25T10:30:01.001 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:00.999+0000 7fe08bfff640 1 -- 172.21.15.37:0/1677943743 <== mon.0 v2:172.21.15.37:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 8 v8) ==== 93+0+5178 (secure 0 0 0) 0x7fe09405f4f0 con 0x7fe0a40b7b00 2024-04-25T10:30:01.002 INFO:teuthology.orchestra.run.smithi037.stdout: 2024-04-25T10:30:01.002 INFO:teuthology.orchestra.run.smithi037.stdout:{"epoch":8,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":4232,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.37:6835/1911169480","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.37:6833","nonce":1911169480},{"type":"v1","addr":"172.21.15.37:6835","nonce":1911169480}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3},{"gid":4241,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.203:6835/3873243328","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6834","nonce":3873243328},{"type":"v1","addr":"172.21.15.203:6835","nonce":3873243328}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3},{"gid":4278,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.37:6834/179723013","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.37:6832","nonce":179723013},{"type":"v1","addr":"172.21.15.37:6834","nonce":179723013}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":2}],"filesystems":[{"mdsmap":{"epoch":8,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-04-25T10:29:08.397829+0000","modified":"2024-04-25T10:29:11.407694+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":1,"in":[0],"up":{"mds_0":4279},"failed":[],"damaged":[],"stopped":[],"info":{"gid_4279":{"gid":4279,"name":"d","rank":0,"incarnation":4,"state":"up:active","state_seq":3,"addr":"172.21.15.203:6837/3215191422","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6836","nonce":3215191422},{"type":"v1","addr":"172.21.15.203:6837","nonce":3215191422}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":4279,"qdb_cluster":[4279]},"id":1}]} 2024-04-25T10:30:01.002 INFO:teuthology.orchestra.run.smithi037.stderr:dumped fsmap epoch 8 2024-04-25T10:30:01.004 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:01.001+0000 7fe0a9eaf640 1 -- 172.21.15.37:0/1677943743 >> [v2:172.21.15.203:6832/2418205237,v1:172.21.15.203:6833/2418205237] conn(0x7fe0900744e0 msgr2=0x7fe0900769f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:30:01.004 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:01.001+0000 7fe0a9eaf640 1 --2- 172.21.15.37:0/1677943743 >> [v2:172.21.15.203:6832/2418205237,v1:172.21.15.203:6833/2418205237] conn(0x7fe0900744e0 0x7fe0900769f0 secure :-1 s=READY pgs=187 cs=0 l=1 rev1=1 crypto rx=0x7fe098034000 tx=0x7fe09802e000 comp rx=0 tx=0).stop 2024-04-25T10:30:01.004 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:01.002+0000 7fe0a9eaf640 1 -- 172.21.15.37:0/1677943743 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fe0a40b7b00 msgr2=0x7fe0a40ccbf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:30:01.004 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:01.002+0000 7fe0a9eaf640 1 --2- 172.21.15.37:0/1677943743 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fe0a40b7b00 0x7fe0a40ccbf0 secure :-1 s=READY pgs=366 cs=0 l=1 rev1=1 crypto rx=0x7fe09400e870 tx=0x7fe09400ed30 comp rx=0 tx=0).stop 2024-04-25T10:30:01.004 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:01.002+0000 7fe0a9eaf640 1 -- 172.21.15.37:0/1677943743 shutdown_connections 2024-04-25T10:30:01.004 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:01.002+0000 7fe0a9eaf640 1 --2- 172.21.15.37:0/1677943743 >> [v2:172.21.15.203:6832/2418205237,v1:172.21.15.203:6833/2418205237] conn(0x7fe0900744e0 0x7fe0900769f0 unknown :-1 s=CLOSED pgs=187 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:30:01.004 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:01.002+0000 7fe0a9eaf640 1 --2- 172.21.15.37:0/1677943743 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7fe0a4160f40 0x7fe0a41508d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:30:01.004 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:01.002+0000 7fe0a9eaf640 1 --2- 172.21.15.37:0/1677943743 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fe0a40d5430 0x7fe0a4162d70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:30:01.004 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:01.002+0000 7fe0a9eaf640 1 --2- 172.21.15.37:0/1677943743 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fe0a40b7b00 0x7fe0a40ccbf0 unknown :-1 s=CLOSED pgs=366 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:30:01.005 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:01.002+0000 7fe0a9eaf640 1 -- 172.21.15.37:0/1677943743 >> 172.21.15.37:0/1677943743 conn(0x7fe0a40d27a0 msgr2=0x7fe0a415f440 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:30:01.005 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:01.002+0000 7fe0a9eaf640 1 -- 172.21.15.37:0/1677943743 shutdown_connections 2024-04-25T10:30:01.005 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:01.002+0000 7fe0a9eaf640 1 -- 172.21.15.37:0/1677943743 wait complete. 2024-04-25T10:30:01.015 DEBUG:teuthology.orchestra.run.smithi037:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-04-25T10:30:01.127 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:01.125+0000 7f6f02a6e640 1 Processor -- start 2024-04-25T10:30:01.127 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:01.125+0000 7f6f02a6e640 1 -- start start 2024-04-25T10:30:01.130 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:01.126+0000 7f6f02a6e640 1 --2- >> v2:172.21.15.37:3300/0 conn(0x7f6efc0d5160 0x7f6efc0b4e80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:30:01.130 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:01.126+0000 7f6f02a6e640 1 --2- >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f6efc0d0990 0x7f6efc0d2b20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:30:01.130 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:01.126+0000 7f6f02a6e640 1 -- --> v1:172.21.15.37:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f6efc0d1370 con 0x7f6efc0d0da0 2024-04-25T10:30:01.130 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:01.126+0000 7f6f02a6e640 1 -- --> v2:172.21.15.37:3300/0 -- mon_getmap magic: 0 -- 0x7f6efc0d1650 con 0x7f6efc0d5160 2024-04-25T10:30:01.130 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:01.126+0000 7f6f02a6e640 1 -- --> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] -- mon_getmap magic: 0 -- 0x7f6efc0d4680 con 0x7f6efc0d0990 2024-04-25T10:30:01.130 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:01.127+0000 7f6f0080a640 1 --2- >> v2:172.21.15.37:3300/0 conn(0x7f6efc0d5160 0x7f6efc0b4e80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:30:01.130 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:01.127+0000 7f6f0100b640 1 --1- >> v1:172.21.15.37:6789/0 conn(0x7f6efc0d0da0 0x7f6efc06db10 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.37:6789/0 says I am v1:172.21.15.37:60660/0 (socket says 172.21.15.37:60660) 2024-04-25T10:30:01.130 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:01.127+0000 7f6f0080a640 1 --2- >> v2:172.21.15.37:3300/0 conn(0x7f6efc0d5160 0x7f6efc0b4e80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.37:3300/0 says I am v2:172.21.15.37:54522/0 (socket says 172.21.15.37:54522) 2024-04-25T10:30:01.130 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:01.127+0000 7f6f0100b640 1 -- 172.21.15.37:0/3592026212 learned_addr learned my addr 172.21.15.37:0/3592026212 (peer_addr_for_me v1:172.21.15.37:0/0) 2024-04-25T10:30:01.131 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:01.127+0000 7f6efbfff640 1 --2- 172.21.15.37:0/3592026212 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f6efc0d0990 0x7f6efc0d2b20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:30:01.131 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:01.127+0000 7f6f0080a640 1 -- 172.21.15.37:0/3592026212 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f6efc0d0990 msgr2=0x7f6efc0d2b20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:30:01.131 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:01.127+0000 7f6f0080a640 1 --2- 172.21.15.37:0/3592026212 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f6efc0d0990 0x7f6efc0d2b20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:30:01.131 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:01.127+0000 7f6f0080a640 1 -- 172.21.15.37:0/3592026212 >> v1:172.21.15.37:6789/0 conn(0x7f6efc0d0da0 legacy=0x7f6efc06db10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:30:01.131 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:01.127+0000 7f6efb7fe640 1 -- 172.21.15.37:0/3592026212 <== mon.0 v1:172.21.15.37:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 2058260666 0 0) 0x7f6efc0d1370 con 0x7f6efc0d0da0 2024-04-25T10:30:01.131 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:01.127+0000 7f6f0080a640 1 -- 172.21.15.37:0/3592026212 --> v2:172.21.15.37:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f6efc0d4960 con 0x7f6efc0d5160 2024-04-25T10:30:01.131 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:01.127+0000 7f6efbfff640 1 --2- 172.21.15.37:0/3592026212 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f6efc0d0990 0x7f6efc0d2b20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T10:30:01.131 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:01.128+0000 7f6f0080a640 1 --2- 172.21.15.37:0/3592026212 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f6efc0d5160 0x7f6efc0b4e80 secure :-1 s=READY pgs=368 cs=0 l=1 rev1=1 crypto rx=0x7f6ef00097a0 tx=0x7f6ef002a080 comp rx=0 tx=0).ready entity=mon.0 client_cookie=992127b93aebd0bd server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:30:01.131 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:01.128+0000 7f6efb7fe640 1 -- 172.21.15.37:0/3592026212 <== mon.0 v2:172.21.15.37:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6ef0037070 con 0x7f6efc0d5160 2024-04-25T10:30:01.131 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:01.128+0000 7f6efb7fe640 1 -- 172.21.15.37:0/3592026212 <== mon.0 v2:172.21.15.37:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f6ef002add0 con 0x7f6efc0d5160 2024-04-25T10:30:01.131 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:01.128+0000 7f6efb7fe640 1 -- 172.21.15.37:0/3592026212 <== mon.0 v2:172.21.15.37:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6ef0032660 con 0x7f6efc0d5160 2024-04-25T10:30:01.132 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:01.128+0000 7f6f02a6e640 1 -- 172.21.15.37:0/3592026212 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f6efc0d5160 msgr2=0x7f6efc0b4e80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:30:01.132 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:01.128+0000 7f6f02a6e640 1 --2- 172.21.15.37:0/3592026212 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f6efc0d5160 0x7f6efc0b4e80 secure :-1 s=READY pgs=368 cs=0 l=1 rev1=1 crypto rx=0x7f6ef00097a0 tx=0x7f6ef002a080 comp rx=0 tx=0).stop 2024-04-25T10:30:01.132 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:01.129+0000 7f6f02a6e640 1 -- 172.21.15.37:0/3592026212 shutdown_connections 2024-04-25T10:30:01.133 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:01.129+0000 7f6f02a6e640 1 --2- 172.21.15.37:0/3592026212 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f6efc0d0990 0x7f6efc0d2b20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:30:01.133 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:01.129+0000 7f6f02a6e640 1 --2- 172.21.15.37:0/3592026212 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f6efc0d5160 0x7f6efc0b4e80 unknown :-1 s=CLOSED pgs=368 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:30:01.133 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:01.129+0000 7f6f02a6e640 1 -- 172.21.15.37:0/3592026212 >> 172.21.15.37:0/3592026212 conn(0x7f6efc0b7a80 msgr2=0x7f6efc0b7e90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:30:01.133 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:01.129+0000 7f6f02a6e640 1 -- 172.21.15.37:0/3592026212 shutdown_connections 2024-04-25T10:30:01.133 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:01.129+0000 7f6f02a6e640 1 -- 172.21.15.37:0/3592026212 wait complete. 2024-04-25T10:30:01.133 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:01.130+0000 7f6f02a6e640 1 Processor -- start 2024-04-25T10:30:01.133 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:01.130+0000 7f6f02a6e640 1 -- start start 2024-04-25T10:30:01.136 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:01.132+0000 7f6f02a6e640 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f6efc0d0990 0x7f6efc150de0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:30:01.136 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:01.132+0000 7f6f02a6e640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f6efc0d0da0 0x7f6efc0cee70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:30:01.136 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:01.132+0000 7f6f02a6e640 1 --2- >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f6efc0d5160 0x7f6efc161f20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:30:01.136 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:01.132+0000 7f6f02a6e640 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 -- 0x7f6efc164c40 con 0x7f6efc0d0da0 2024-04-25T10:30:01.136 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:01.132+0000 7f6f02a6e640 1 -- --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_getmap magic: 0 -- 0x7f6efc164ac0 con 0x7f6efc0d0990 2024-04-25T10:30:01.136 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:01.132+0000 7f6f02a6e640 1 -- --> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] -- mon_getmap magic: 0 -- 0x7f6efc0d3060 con 0x7f6efc0d5160 2024-04-25T10:30:01.136 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:01.132+0000 7f6efbfff640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f6efc0d0da0 0x7f6efc0cee70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:30:01.136 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:01.132+0000 7f6efbfff640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f6efc0d0da0 0x7f6efc0cee70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.37:3300/0 says I am v2:172.21.15.37:54538/0 (socket says 172.21.15.37:54538) 2024-04-25T10:30:01.136 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:01.132+0000 7f6efbfff640 1 -- 172.21.15.37:0/2017788917 learned_addr learned my addr 172.21.15.37:0/2017788917 (peer_addr_for_me v2:172.21.15.37:0/0) 2024-04-25T10:30:01.136 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:01.132+0000 7f6efbfff640 1 -- 172.21.15.37:0/2017788917 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f6efc0d5160 msgr2=0x7f6efc161f20 unknown :-1 s=STATE_CONNECTING_RE l=1).mark_down 2024-04-25T10:30:01.136 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:01.132+0000 7f6efbfff640 1 --2- 172.21.15.37:0/2017788917 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f6efc0d5160 0x7f6efc161f20 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:30:01.136 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:01.132+0000 7f6efbfff640 1 -- 172.21.15.37:0/2017788917 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f6efc0d0990 msgr2=0x7f6efc150de0 unknown :-1 s=STATE_CONNECTING_RE l=1).mark_down 2024-04-25T10:30:01.137 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:01.132+0000 7f6efbfff640 1 --2- 172.21.15.37:0/2017788917 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f6efc0d0990 0x7f6efc150de0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:30:01.137 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:01.132+0000 7f6efbfff640 1 -- 172.21.15.37:0/2017788917 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f6efc15e1a0 con 0x7f6efc0d0da0 2024-04-25T10:30:01.137 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:01.132+0000 7f6efbfff640 1 --2- 172.21.15.37:0/2017788917 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f6efc0d0da0 0x7f6efc0cee70 secure :-1 s=READY pgs=369 cs=0 l=1 rev1=1 crypto rx=0x7f6ee800c820 tx=0x7f6ee800cce0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:30:01.137 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:01.133+0000 7f6ef97fa640 1 -- 172.21.15.37:0/2017788917 <== mon.0 v2:172.21.15.37:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6ee8015070 con 0x7f6efc0d0da0 2024-04-25T10:30:01.137 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:01.133+0000 7f6ef97fa640 1 -- 172.21.15.37:0/2017788917 <== mon.0 v2:172.21.15.37:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f6ee8007a30 con 0x7f6efc0d0da0 2024-04-25T10:30:01.137 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:01.133+0000 7f6f02a6e640 1 -- 172.21.15.37:0/2017788917 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f6efc1e9c00 con 0x7f6efc0d0da0 2024-04-25T10:30:01.137 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:01.133+0000 7f6ef97fa640 1 -- 172.21.15.37:0/2017788917 <== mon.0 v2:172.21.15.37:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6ee8007de0 con 0x7f6efc0d0da0 2024-04-25T10:30:01.137 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:01.133+0000 7f6f02a6e640 1 -- 172.21.15.37:0/2017788917 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f6efc1ea1c0 con 0x7f6efc0d0da0 2024-04-25T10:30:01.137 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:01.134+0000 7f6f02a6e640 1 -- 172.21.15.37:0/2017788917 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f6efc0cf7a0 con 0x7f6efc0d0da0 2024-04-25T10:30:01.138 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:01.134+0000 7f6ef97fa640 1 -- 172.21.15.37:0/2017788917 <== mon.0 v2:172.21.15.37:3300/0 4 ==== mgrmap(e 5) ==== 96782+0+0 (secure 0 0 0) 0x7f6ee8018070 con 0x7f6efc0d0da0 2024-04-25T10:30:01.138 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:01.135+0000 7f6ef97fa640 1 --2- 172.21.15.37:0/2017788917 >> [v2:172.21.15.203:6832/2418205237,v1:172.21.15.203:6833/2418205237] conn(0x7f6ee00744b0 0x7f6ee00769c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:30:01.138 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:01.135+0000 7f6ef97fa640 1 -- 172.21.15.37:0/2017788917 <== mon.0 v2:172.21.15.37:3300/0 5 ==== osd_map(25..25 src has 1..25) ==== 5738+0+0 (secure 0 0 0) 0x7f6ee80950e0 con 0x7f6efc0d0da0 2024-04-25T10:30:01.139 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:01.137+0000 7f6f0080a640 1 --2- 172.21.15.37:0/2017788917 >> [v2:172.21.15.203:6832/2418205237,v1:172.21.15.203:6833/2418205237] conn(0x7f6ee00744b0 0x7f6ee00769c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:30:01.141 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:01.139+0000 7f6ef97fa640 1 -- 172.21.15.37:0/2017788917 <== mon.0 v2:172.21.15.37:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187246 (secure 0 0 0) 0x7f6efc15e1a0 con 0x7f6efc0d0da0 2024-04-25T10:30:01.142 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:01.140+0000 7f6f0080a640 1 --2- 172.21.15.37:0/2017788917 >> [v2:172.21.15.203:6832/2418205237,v1:172.21.15.203:6833/2418205237] conn(0x7f6ee00744b0 0x7f6ee00769c0 secure :-1 s=READY pgs=188 cs=0 l=1 rev1=1 crypto rx=0x7f6ef0034000 tx=0x7f6ef002e000 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:30:01.330 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:01.328+0000 7f6f02a6e640 1 -- 172.21.15.37:0/2017788917 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7f6efc06a140 con 0x7f6efc0d0da0 2024-04-25T10:30:01.332 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:01.330+0000 7f6ef97fa640 1 -- 172.21.15.37:0/2017788917 <== mon.0 v2:172.21.15.37:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v25) ==== 74+0+16670 (secure 0 0 0) 0x7f6ee805f8e0 con 0x7f6efc0d0da0 2024-04-25T10:30:01.332 INFO:teuthology.orchestra.run.smithi037.stdout: 2024-04-25T10:30:01.332 INFO:teuthology.orchestra.run.smithi037.stdout:{"epoch":25,"fsid":"f4acda28-be34-492a-b286-994afd36f61d","created":"2024-04-25T10:28:43.985718+0000","modified":"2024-04-25T10:29:49.325719+0000","last_up_change":"2024-04-25T10:28:54.900654+0000","last_in_change":"2024-04-25T10:28:46.216973+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":3,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"luminous","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-04-25T10:28:56.202153+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"19","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2024-04-25T10:29:06.507244+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"22","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":2.25,"score_stable":2.25,"optimal_score":1,"raw_score_acting":2.25,"raw_score_stable":2.25,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2024-04-25T10:29:07.347844+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"23","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"a2f79028-b332-4166-9ebb-a7024a37ea22","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":21,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.37:6816","nonce":2746074720},{"type":"v1","addr":"172.21.15.37:6817","nonce":2746074720}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.37:6818","nonce":2746074720},{"type":"v1","addr":"172.21.15.37:6819","nonce":2746074720}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.37:6822","nonce":2746074720},{"type":"v1","addr":"172.21.15.37:6823","nonce":2746074720}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.37:6820","nonce":2746074720},{"type":"v1","addr":"172.21.15.37:6821","nonce":2746074720}]},"public_addr":"172.21.15.37:6817/2746074720","cluster_addr":"172.21.15.37:6819/2746074720","heartbeat_back_addr":"172.21.15.37:6823/2746074720","heartbeat_front_addr":"172.21.15.37:6821/2746074720","state":["exists","up"]},{"osd":1,"uuid":"973159de-c2ea-4083-aecc-69373d1c1132","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":21,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.37:6800","nonce":1690341662},{"type":"v1","addr":"172.21.15.37:6801","nonce":1690341662}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.37:6802","nonce":1690341662},{"type":"v1","addr":"172.21.15.37:6803","nonce":1690341662}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.37:6806","nonce":1690341662},{"type":"v1","addr":"172.21.15.37:6807","nonce":1690341662}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.37:6804","nonce":1690341662},{"type":"v1","addr":"172.21.15.37:6805","nonce":1690341662}]},"public_addr":"172.21.15.37:6801/1690341662","cluster_addr":"172.21.15.37:6803/1690341662","heartbeat_back_addr":"172.21.15.37:6807/1690341662","heartbeat_front_addr":"172.21.15.37:6805/1690341662","state":["exists","up"]},{"osd":2,"uuid":"4ae7a0e9-cb3f-484c-a75f-f3766b721c47","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":21,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.37:6808","nonce":2754669649},{"type":"v1","addr":"172.21.15.37:6809","nonce":2754669649}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.37:6810","nonce":2754669649},{"type":"v1","addr":"172.21.15.37:6811","nonce":2754669649}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.37:6814","nonce":2754669649},{"type":"v1","addr":"172.21.15.37:6815","nonce":2754669649}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.37:6812","nonce":2754669649},{"type":"v1","addr":"172.21.15.37:6813","nonce":2754669649}]},"public_addr":"172.21.15.37:6809/2754669649","cluster_addr":"172.21.15.37:6811/2754669649","heartbeat_back_addr":"172.21.15.37:6815/2754669649","heartbeat_front_addr":"172.21.15.37:6813/2754669649","state":["exists","up"]},{"osd":3,"uuid":"1ba4a746-1608-47a7-ace2-960a5cfae934","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":21,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.37:6824","nonce":1815290618},{"type":"v1","addr":"172.21.15.37:6825","nonce":1815290618}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.37:6826","nonce":1815290618},{"type":"v1","addr":"172.21.15.37:6827","nonce":1815290618}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.37:6830","nonce":1815290618},{"type":"v1","addr":"172.21.15.37:6831","nonce":1815290618}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.37:6828","nonce":1815290618},{"type":"v1","addr":"172.21.15.37:6829","nonce":1815290618}]},"public_addr":"172.21.15.37:6825/1815290618","cluster_addr":"172.21.15.37:6827/1815290618","heartbeat_back_addr":"172.21.15.37:6831/1815290618","heartbeat_front_addr":"172.21.15.37:6829/1815290618","state":["exists","up"]},{"osd":4,"uuid":"d4f59f5d-0413-4b79-87ee-bb212c1fcf62","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":21,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6800","nonce":4051532063},{"type":"v1","addr":"172.21.15.203:6801","nonce":4051532063}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6802","nonce":4051532063},{"type":"v1","addr":"172.21.15.203:6803","nonce":4051532063}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6806","nonce":4051532063},{"type":"v1","addr":"172.21.15.203:6807","nonce":4051532063}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6804","nonce":4051532063},{"type":"v1","addr":"172.21.15.203:6805","nonce":4051532063}]},"public_addr":"172.21.15.203:6801/4051532063","cluster_addr":"172.21.15.203:6803/4051532063","heartbeat_back_addr":"172.21.15.203:6807/4051532063","heartbeat_front_addr":"172.21.15.203:6805/4051532063","state":["exists","up"]},{"osd":5,"uuid":"69710a0e-41d8-4e7f-9a10-17ce540b67be","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":24,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6824","nonce":1203822890},{"type":"v1","addr":"172.21.15.203:6825","nonce":1203822890}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6826","nonce":1203822890},{"type":"v1","addr":"172.21.15.203:6827","nonce":1203822890}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6830","nonce":1203822890},{"type":"v1","addr":"172.21.15.203:6831","nonce":1203822890}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6828","nonce":1203822890},{"type":"v1","addr":"172.21.15.203:6829","nonce":1203822890}]},"public_addr":"172.21.15.203:6825/1203822890","cluster_addr":"172.21.15.203:6827/1203822890","heartbeat_back_addr":"172.21.15.203:6831/1203822890","heartbeat_front_addr":"172.21.15.203:6829/1203822890","state":["exists","up"]},{"osd":6,"uuid":"f29dd047-3b48-473d-a235-6642a6293f20","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":21,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6810","nonce":2309615014},{"type":"v1","addr":"172.21.15.203:6812","nonce":2309615014}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6814","nonce":2309615014},{"type":"v1","addr":"172.21.15.203:6816","nonce":2309615014}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6822","nonce":2309615014},{"type":"v1","addr":"172.21.15.203:6823","nonce":2309615014}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6818","nonce":2309615014},{"type":"v1","addr":"172.21.15.203:6820","nonce":2309615014}]},"public_addr":"172.21.15.203:6812/2309615014","cluster_addr":"172.21.15.203:6816/2309615014","heartbeat_back_addr":"172.21.15.203:6823/2309615014","heartbeat_front_addr":"172.21.15.203:6820/2309615014","state":["exists","up"]},{"osd":7,"uuid":"6e691644-378b-4fdc-8e86-fb5f56e31f29","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":21,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6808","nonce":3688099014},{"type":"v1","addr":"172.21.15.203:6809","nonce":3688099014}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6811","nonce":3688099014},{"type":"v1","addr":"172.21.15.203:6813","nonce":3688099014}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6819","nonce":3688099014},{"type":"v1","addr":"172.21.15.203:6821","nonce":3688099014}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6815","nonce":3688099014},{"type":"v1","addr":"172.21.15.203:6817","nonce":3688099014}]},"public_addr":"172.21.15.203:6809/3688099014","cluster_addr":"172.21.15.203:6813/3688099014","heartbeat_back_addr":"172.21.15.203:6821/3688099014","heartbeat_front_addr":"172.21.15.203:6817/3688099014","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T10:28:51.419141+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T10:28:51.366448+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T10:28:51.348276+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T10:28:51.380519+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T10:28:51.105812+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T10:28:51.384496+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T10:28:51.388755+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T10:28:51.366958+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[{"pgid":"3.31","mappings":[{"from":0,"to":7}]}],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-04-25T10:30:01.334 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:01.332+0000 7f6f02a6e640 1 -- 172.21.15.37:0/2017788917 >> [v2:172.21.15.203:6832/2418205237,v1:172.21.15.203:6833/2418205237] conn(0x7f6ee00744b0 msgr2=0x7f6ee00769c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:30:01.334 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:01.332+0000 7f6f02a6e640 1 --2- 172.21.15.37:0/2017788917 >> [v2:172.21.15.203:6832/2418205237,v1:172.21.15.203:6833/2418205237] conn(0x7f6ee00744b0 0x7f6ee00769c0 secure :-1 s=READY pgs=188 cs=0 l=1 rev1=1 crypto rx=0x7f6ef0034000 tx=0x7f6ef002e000 comp rx=0 tx=0).stop 2024-04-25T10:30:01.334 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:01.332+0000 7f6f02a6e640 1 -- 172.21.15.37:0/2017788917 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f6efc0d0da0 msgr2=0x7f6efc0cee70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:30:01.334 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:01.332+0000 7f6f02a6e640 1 --2- 172.21.15.37:0/2017788917 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f6efc0d0da0 0x7f6efc0cee70 secure :-1 s=READY pgs=369 cs=0 l=1 rev1=1 crypto rx=0x7f6ee800c820 tx=0x7f6ee800cce0 comp rx=0 tx=0).stop 2024-04-25T10:30:01.334 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:01.332+0000 7f6f02a6e640 1 -- 172.21.15.37:0/2017788917 shutdown_connections 2024-04-25T10:30:01.334 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:01.332+0000 7f6f02a6e640 1 --2- 172.21.15.37:0/2017788917 >> [v2:172.21.15.203:6832/2418205237,v1:172.21.15.203:6833/2418205237] conn(0x7f6ee00744b0 0x7f6ee00769c0 unknown :-1 s=CLOSED pgs=188 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:30:01.335 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:01.332+0000 7f6f02a6e640 1 --2- 172.21.15.37:0/2017788917 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f6efc0d5160 0x7f6efc161f20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:30:01.335 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:01.332+0000 7f6f02a6e640 1 --2- 172.21.15.37:0/2017788917 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f6efc0d0da0 0x7f6efc0cee70 unknown :-1 s=CLOSED pgs=369 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:30:01.335 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:01.332+0000 7f6f02a6e640 1 --2- 172.21.15.37:0/2017788917 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f6efc0d0990 0x7f6efc150de0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:30:01.335 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:01.332+0000 7f6f02a6e640 1 -- 172.21.15.37:0/2017788917 >> 172.21.15.37:0/2017788917 conn(0x7f6efc0b7a80 msgr2=0x7f6efc0b7e90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:30:01.335 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:01.332+0000 7f6f02a6e640 1 -- 172.21.15.37:0/2017788917 shutdown_connections 2024-04-25T10:30:01.335 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:01.332+0000 7f6f02a6e640 1 -- 172.21.15.37:0/2017788917 wait complete. 2024-04-25T10:30:01.346 DEBUG:teuthology.orchestra.run.smithi037:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs fail cephfs 2024-04-25T10:30:01.458 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:01.455+0000 7fab6780c640 1 Processor -- start 2024-04-25T10:30:01.458 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:01.456+0000 7fab6780c640 1 -- start start 2024-04-25T10:30:01.459 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:01.457+0000 7fab6780c640 1 --2- >> v2:172.21.15.37:3300/0 conn(0x7fab600d5430 0x7fab600b75c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:30:01.460 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:01.457+0000 7fab6780c640 1 -- --> v1:172.21.15.37:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fab600b7f10 con 0x7fab60161110 2024-04-25T10:30:01.460 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:01.457+0000 7fab6780c640 1 -- --> v1:172.21.15.203:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fab60161520 con 0x7fab600b7b00 2024-04-25T10:30:01.460 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:01.457+0000 7fab6780c640 1 -- --> v2:172.21.15.37:3300/0 -- mon_getmap magic: 0 -- 0x7fab600d4420 con 0x7fab600d5430 2024-04-25T10:30:01.460 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:01.457+0000 7fab655a8640 1 --2- >> v2:172.21.15.37:3300/0 conn(0x7fab600d5430 0x7fab600b75c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:30:01.460 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:01.457+0000 7fab65da9640 1 --1- >> v1:172.21.15.37:6789/0 conn(0x7fab60161110 0x7fab60161880 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.37:6789/0 says I am v1:172.21.15.37:60670/0 (socket says 172.21.15.37:60670) 2024-04-25T10:30:01.460 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:01.457+0000 7fab65da9640 1 -- 172.21.15.37:0/1485339340 learned_addr learned my addr 172.21.15.37:0/1485339340 (peer_addr_for_me v1:172.21.15.37:0/0) 2024-04-25T10:30:01.460 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:01.457+0000 7fab655a8640 1 -- 172.21.15.37:0/1485339340 >> v1:172.21.15.203:6789/0 conn(0x7fab600b7b00 legacy=0x7fab60160a00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:30:01.460 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:01.457+0000 7fab655a8640 1 -- 172.21.15.37:0/1485339340 >> v1:172.21.15.37:6789/0 conn(0x7fab60161110 legacy=0x7fab60161880 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:30:01.460 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:01.457+0000 7fab57fff640 1 -- 172.21.15.37:0/1485339340 <== mon.0 v1:172.21.15.37:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 2000357401 0 0) 0x7fab5c001490 con 0x7fab60161110 2024-04-25T10:30:01.460 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:01.457+0000 7fab655a8640 1 -- 172.21.15.37:0/1485339340 --> v2:172.21.15.37:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fab600d5840 con 0x7fab600d5430 2024-04-25T10:30:01.461 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:01.458+0000 7fab655a8640 1 --2- 172.21.15.37:0/1485339340 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fab600d5430 0x7fab600b75c0 secure :-1 s=READY pgs=371 cs=0 l=1 rev1=1 crypto rx=0x7fab601616f0 tx=0x7fab5802a130 comp rx=0 tx=0).ready entity=mon.0 client_cookie=7e4697e6a024e78c server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:30:01.461 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:01.458+0000 7fab57fff640 1 -- 172.21.15.37:0/1485339340 <== mon.0 v2:172.21.15.37:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fab58037070 con 0x7fab600d5430 2024-04-25T10:30:01.462 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:01.458+0000 7fab57fff640 1 -- 172.21.15.37:0/1485339340 <== mon.0 v2:172.21.15.37:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fab5802ae40 con 0x7fab600d5430 2024-04-25T10:30:01.462 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:01.458+0000 7fab57fff640 1 -- 172.21.15.37:0/1485339340 <== mon.0 v2:172.21.15.37:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fab58032660 con 0x7fab600d5430 2024-04-25T10:30:01.462 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:01.459+0000 7fab6780c640 1 -- 172.21.15.37:0/1485339340 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fab600d5430 msgr2=0x7fab600b75c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:30:01.462 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:01.459+0000 7fab6780c640 1 --2- 172.21.15.37:0/1485339340 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fab600d5430 0x7fab600b75c0 secure :-1 s=READY pgs=371 cs=0 l=1 rev1=1 crypto rx=0x7fab601616f0 tx=0x7fab5802a130 comp rx=0 tx=0).stop 2024-04-25T10:30:01.462 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:01.459+0000 7fab6780c640 1 -- 172.21.15.37:0/1485339340 shutdown_connections 2024-04-25T10:30:01.462 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:01.459+0000 7fab6780c640 1 --2- 172.21.15.37:0/1485339340 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fab600d5430 0x7fab600b75c0 unknown :-1 s=CLOSED pgs=371 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:30:01.462 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:01.459+0000 7fab6780c640 1 -- 172.21.15.37:0/1485339340 >> 172.21.15.37:0/1485339340 conn(0x7fab600d27a0 msgr2=0x7fab600d2bb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:30:01.462 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:01.459+0000 7fab6780c640 1 -- 172.21.15.37:0/1485339340 shutdown_connections 2024-04-25T10:30:01.462 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:01.459+0000 7fab6780c640 1 -- 172.21.15.37:0/1485339340 wait complete. 2024-04-25T10:30:01.463 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:01.461+0000 7fab6780c640 1 Processor -- start 2024-04-25T10:30:01.463 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:01.461+0000 7fab6780c640 1 -- start start 2024-04-25T10:30:01.466 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:01.462+0000 7fab6780c640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fab600b7b00 0x7fab60156aa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:30:01.466 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:01.462+0000 7fab6780c640 1 --2- >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7fab600d5430 0x7fab600cd6a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:30:01.466 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:01.462+0000 7fab6780c640 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fab60161110 0x7fab601508d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:30:01.466 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:01.462+0000 7fab6780c640 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 -- 0x7fab600d4c30 con 0x7fab600b7b00 2024-04-25T10:30:01.466 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:01.462+0000 7fab6780c640 1 -- --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_getmap magic: 0 -- 0x7fab600d4950 con 0x7fab60161110 2024-04-25T10:30:01.466 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:01.462+0000 7fab6780c640 1 -- --> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] -- mon_getmap magic: 0 -- 0x7fab600d4f10 con 0x7fab600d5430 2024-04-25T10:30:01.466 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:01.462+0000 7fab655a8640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fab600b7b00 0x7fab60156aa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:30:01.466 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:01.462+0000 7fab655a8640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fab600b7b00 0x7fab60156aa0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.37:3300/0 says I am v2:172.21.15.37:54550/0 (socket says 172.21.15.37:54550) 2024-04-25T10:30:01.466 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:01.462+0000 7fab655a8640 1 -- 172.21.15.37:0/602198984 learned_addr learned my addr 172.21.15.37:0/602198984 (peer_addr_for_me v2:172.21.15.37:0/0) 2024-04-25T10:30:01.466 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:01.463+0000 7fab64da7640 1 --2- 172.21.15.37:0/602198984 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7fab600d5430 0x7fab600cd6a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:30:01.466 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:01.463+0000 7fab65da9640 1 --2- 172.21.15.37:0/602198984 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fab60161110 0x7fab601508d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:30:01.466 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:01.463+0000 7fab655a8640 1 -- 172.21.15.37:0/602198984 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7fab600d5430 msgr2=0x7fab600cd6a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:30:01.466 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:01.463+0000 7fab655a8640 1 --2- 172.21.15.37:0/602198984 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7fab600d5430 0x7fab600cd6a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:30:01.467 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:01.463+0000 7fab655a8640 1 -- 172.21.15.37:0/602198984 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fab60161110 msgr2=0x7fab601508d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:30:01.467 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:01.463+0000 7fab655a8640 1 --2- 172.21.15.37:0/602198984 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fab60161110 0x7fab601508d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:30:01.467 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:01.463+0000 7fab655a8640 1 -- 172.21.15.37:0/602198984 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fab6015b2d0 con 0x7fab600b7b00 2024-04-25T10:30:01.467 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:01.463+0000 7fab655a8640 1 --2- 172.21.15.37:0/602198984 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fab600b7b00 0x7fab60156aa0 secure :-1 s=READY pgs=372 cs=0 l=1 rev1=1 crypto rx=0x7fab58009890 tx=0x7fab580046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:30:01.467 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:01.463+0000 7fab55ffb640 1 -- 172.21.15.37:0/602198984 <== mon.0 v2:172.21.15.37:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fab58037040 con 0x7fab600b7b00 2024-04-25T10:30:01.467 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:01.463+0000 7fab55ffb640 1 -- 172.21.15.37:0/602198984 <== mon.0 v2:172.21.15.37:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fab58004030 con 0x7fab600b7b00 2024-04-25T10:30:01.469 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:01.463+0000 7fab55ffb640 1 -- 172.21.15.37:0/602198984 <== mon.0 v2:172.21.15.37:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fab58006e10 con 0x7fab600b7b00 2024-04-25T10:30:01.470 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:01.463+0000 7fab6780c640 1 -- 172.21.15.37:0/602198984 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fab6015b5b0 con 0x7fab600b7b00 2024-04-25T10:30:01.470 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:01.463+0000 7fab6780c640 1 -- 172.21.15.37:0/602198984 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fab6015bb70 con 0x7fab600b7b00 2024-04-25T10:30:01.470 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:01.464+0000 7fab6780c640 1 -- 172.21.15.37:0/602198984 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fab6006a140 con 0x7fab600b7b00 2024-04-25T10:30:01.471 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:01.465+0000 7fab55ffb640 1 -- 172.21.15.37:0/602198984 <== mon.0 v2:172.21.15.37:3300/0 4 ==== mgrmap(e 5) ==== 96782+0+0 (secure 0 0 0) 0x7fab5803d650 con 0x7fab600b7b00 2024-04-25T10:30:01.471 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:01.465+0000 7fab55ffb640 1 --2- 172.21.15.37:0/602198984 >> [v2:172.21.15.203:6832/2418205237,v1:172.21.15.203:6833/2418205237] conn(0x7fab440744e0 0x7fab440769f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:30:01.471 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:01.466+0000 7fab55ffb640 1 -- 172.21.15.37:0/602198984 <== mon.0 v2:172.21.15.37:3300/0 5 ==== osd_map(25..25 src has 1..25) ==== 5738+0+0 (secure 0 0 0) 0x7fab580b5780 con 0x7fab600b7b00 2024-04-25T10:30:01.471 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:01.466+0000 7fab64da7640 1 --2- 172.21.15.37:0/602198984 >> [v2:172.21.15.203:6832/2418205237,v1:172.21.15.203:6833/2418205237] conn(0x7fab440744e0 0x7fab440769f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:30:01.471 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:01.466+0000 7fab64da7640 1 --2- 172.21.15.37:0/602198984 >> [v2:172.21.15.203:6832/2418205237,v1:172.21.15.203:6833/2418205237] conn(0x7fab440744e0 0x7fab440769f0 secure :-1 s=READY pgs=189 cs=0 l=1 rev1=1 crypto rx=0x7fab50006fd0 tx=0x7fab50008040 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:30:01.472 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:01.470+0000 7fab55ffb640 1 -- 172.21.15.37:0/602198984 <== mon.0 v2:172.21.15.37:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187246 (secure 0 0 0) 0x7fab58042020 con 0x7fab600b7b00 2024-04-25T10:30:01.696 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:01.694+0000 7fab6780c640 1 -- 172.21.15.37:0/602198984 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_command({"prefix": "fs fail", "fs_name": "cephfs"} v 0) -- 0x7fab6015dc20 con 0x7fab600b7b00 2024-04-25T10:30:01.696 INFO:tasks.ceph.mon.a.smithi037.stderr:2024-04-25T10:30:01.694+0000 7fb25155b640 -1 log_channel(cluster) log [ERR] : Health check failed: 1 filesystem is offline (MDS_ALL_DOWN) 2024-04-25T10:30:01.698 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:01.696+0000 7fab55ffb640 1 -- 172.21.15.37:0/602198984 <== mon.0 v2:172.21.15.37:3300/0 7 ==== mon_command_ack([{"prefix": "fs fail", "fs_name": "cephfs"}]=0 cephfs marked not joinable; MDS cannot join the cluster. All MDS ranks marked failed. v9) ==== 161+0+0 (secure 0 0 0) 0x7fab5807ff70 con 0x7fab600b7b00 2024-04-25T10:30:01.698 INFO:teuthology.orchestra.run.smithi037.stderr:cephfs marked not joinable; MDS cannot join the cluster. All MDS ranks marked failed. 2024-04-25T10:30:01.701 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:01.698+0000 7fab6780c640 1 -- 172.21.15.37:0/602198984 >> [v2:172.21.15.203:6832/2418205237,v1:172.21.15.203:6833/2418205237] conn(0x7fab440744e0 msgr2=0x7fab440769f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:30:01.701 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:01.698+0000 7fab6780c640 1 --2- 172.21.15.37:0/602198984 >> [v2:172.21.15.203:6832/2418205237,v1:172.21.15.203:6833/2418205237] conn(0x7fab440744e0 0x7fab440769f0 secure :-1 s=READY pgs=189 cs=0 l=1 rev1=1 crypto rx=0x7fab50006fd0 tx=0x7fab50008040 comp rx=0 tx=0).stop 2024-04-25T10:30:01.701 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:01.698+0000 7fab6780c640 1 -- 172.21.15.37:0/602198984 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fab600b7b00 msgr2=0x7fab60156aa0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:30:01.701 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:01.698+0000 7fab6780c640 1 --2- 172.21.15.37:0/602198984 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fab600b7b00 0x7fab60156aa0 secure :-1 s=READY pgs=372 cs=0 l=1 rev1=1 crypto rx=0x7fab58009890 tx=0x7fab580046c0 comp rx=0 tx=0).stop 2024-04-25T10:30:01.701 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:01.698+0000 7fab6780c640 1 -- 172.21.15.37:0/602198984 shutdown_connections 2024-04-25T10:30:01.701 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:01.698+0000 7fab6780c640 1 --2- 172.21.15.37:0/602198984 >> [v2:172.21.15.203:6832/2418205237,v1:172.21.15.203:6833/2418205237] conn(0x7fab440744e0 0x7fab440769f0 unknown :-1 s=CLOSED pgs=189 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:30:01.702 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:01.699+0000 7fab6780c640 1 --2- 172.21.15.37:0/602198984 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fab60161110 0x7fab601508d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:30:01.702 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:01.699+0000 7fab6780c640 1 --2- 172.21.15.37:0/602198984 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7fab600d5430 0x7fab600cd6a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:30:01.702 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:01.699+0000 7fab6780c640 1 --2- 172.21.15.37:0/602198984 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fab600b7b00 0x7fab60156aa0 unknown :-1 s=CLOSED pgs=372 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:30:01.702 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:01.699+0000 7fab6780c640 1 -- 172.21.15.37:0/602198984 >> 172.21.15.37:0/602198984 conn(0x7fab600d27a0 msgr2=0x7fab6015e140 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:30:01.702 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:01.699+0000 7fab6780c640 1 -- 172.21.15.37:0/602198984 shutdown_connections 2024-04-25T10:30:01.702 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:01.699+0000 7fab6780c640 1 -- 172.21.15.37:0/602198984 wait complete. 2024-04-25T10:30:01.704 INFO:tasks.ceph.mds.d.smithi203.stderr: -3212> 2024-04-25T10:29:12.904+0000 7fa45031c640 -1 mds.pinger is_rank_lagging: rank=0 was never sent ping request. 2024-04-25T10:30:01.712 DEBUG:teuthology.orchestra.run.smithi037:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs rm cephfs --yes-i-really-mean-it 2024-04-25T10:30:01.760 INFO:tasks.ceph.mds.d.smithi203.stdout:starting mds.d at 2024-04-25T10:30:01.826 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:01.824+0000 7f80831c5640 1 Processor -- start 2024-04-25T10:30:01.826 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:01.824+0000 7f80831c5640 1 -- start start 2024-04-25T10:30:01.829 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:01.825+0000 7f80831c5640 1 --2- >> v2:172.21.15.203:3300/0 conn(0x7f807c0b4e80 0x7f807c0d2b20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:30:01.829 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:01.825+0000 7f80831c5640 1 --2- >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f807c0d3060 0x7f807c0b9060 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:30:01.829 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:01.825+0000 7f80831c5640 1 -- --> v1:172.21.15.37:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f807c0d3470 con 0x7f807c0d5160 2024-04-25T10:30:01.829 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:01.825+0000 7f80831c5640 1 -- --> v2:172.21.15.203:3300/0 -- mon_getmap magic: 0 -- 0x7f807c0d1650 con 0x7f807c0b4e80 2024-04-25T10:30:01.829 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:01.825+0000 7f80831c5640 1 -- --> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] -- mon_getmap magic: 0 -- 0x7f807c0b5290 con 0x7f807c0d3060 2024-04-25T10:30:01.829 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:01.825+0000 7f8080f61640 1 --1- >> v1:172.21.15.37:6789/0 conn(0x7f807c0d5160 0x7f807c0d0990 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.37:6789/0 says I am v1:172.21.15.37:60672/0 (socket says 172.21.15.37:60672) 2024-04-25T10:30:01.829 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:01.825+0000 7f8080f61640 1 -- 172.21.15.37:0/3502947968 learned_addr learned my addr 172.21.15.37:0/3502947968 (peer_addr_for_me v1:172.21.15.37:0/0) 2024-04-25T10:30:01.830 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:01.825+0000 7f807bfff640 1 --2- 172.21.15.37:0/3502947968 >> v2:172.21.15.203:3300/0 conn(0x7f807c0b4e80 0x7f807c0d2b20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:30:01.830 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:01.825+0000 7f8081762640 1 --2- 172.21.15.37:0/3502947968 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f807c0d3060 0x7f807c0b9060 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:30:01.830 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:01.825+0000 7f807b7fe640 1 -- 172.21.15.37:0/3502947968 <== mon.0 v1:172.21.15.37:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 2249382645 0 0) 0x7f8074001490 con 0x7f807c0d5160 2024-04-25T10:30:01.830 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:01.825+0000 7f807b7fe640 1 -- 172.21.15.37:0/3502947968 --> v1:172.21.15.37:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f8068003520 con 0x7f807c0d5160 2024-04-25T10:30:01.830 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:01.826+0000 7f807b7fe640 1 -- 172.21.15.37:0/3502947968 <== mon.0 v1:172.21.15.37:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 3266010157 0 0) 0x7f8068003520 con 0x7f807c0d5160 2024-04-25T10:30:01.830 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:01.826+0000 7f807b7fe640 1 -- 172.21.15.37:0/3502947968 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f807c0d3060 msgr2=0x7f807c0b9060 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:30:01.830 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:01.826+0000 7f807b7fe640 1 --2- 172.21.15.37:0/3502947968 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f807c0d3060 0x7f807c0b9060 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:30:01.830 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:01.826+0000 7f807b7fe640 1 -- 172.21.15.37:0/3502947968 >> v2:172.21.15.203:3300/0 conn(0x7f807c0b4e80 msgr2=0x7f807c0d2b20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:30:01.830 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:01.826+0000 7f807b7fe640 1 --2- 172.21.15.37:0/3502947968 >> v2:172.21.15.203:3300/0 conn(0x7f807c0b4e80 0x7f807c0d2b20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:30:01.830 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:01.826+0000 7f807b7fe640 1 -- 172.21.15.37:0/3502947968 --> v1:172.21.15.37:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f807c0d10a0 con 0x7f807c0d5160 2024-04-25T10:30:01.830 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:01.826+0000 7f807b7fe640 1 -- 172.21.15.37:0/3502947968 <== mon.0 v1:172.21.15.37:6789/0 3 ==== mon_map magic: 0 ==== 413+0+0 (unknown 1577794600 0 0) 0x7f8074001be0 con 0x7f807c0d5160 2024-04-25T10:30:01.830 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:01.826+0000 7f807b7fe640 1 -- 172.21.15.37:0/3502947968 >> v1:172.21.15.37:6789/0 conn(0x7f807c0d5160 legacy=0x7f807c0d0990 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:30:01.830 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:01.826+0000 7f807b7fe640 1 --2- 172.21.15.37:0/3502947968 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f807c0d3060 0x7f80680039b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:30:01.831 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:01.826+0000 7f807b7fe640 1 --2- 172.21.15.37:0/3502947968 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f8068003f30 0x7f8068004800 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:30:01.831 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:01.826+0000 7f807b7fe640 1 --2- 172.21.15.37:0/3502947968 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f8068004d40 0x7f8068005150 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:30:01.831 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:01.826+0000 7f807b7fe640 1 -- 172.21.15.37:0/3502947968 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 -- 0x7f807c0d1650 con 0x7f8068004d40 2024-04-25T10:30:01.831 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:01.826+0000 7f807b7fe640 1 -- 172.21.15.37:0/3502947968 --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_getmap magic: 0 -- 0x7f807c0b5290 con 0x7f8068003f30 2024-04-25T10:30:01.831 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:01.826+0000 7f807b7fe640 1 -- 172.21.15.37:0/3502947968 --> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] -- mon_getmap magic: 0 -- 0x7f806800bac0 con 0x7f807c0d3060 2024-04-25T10:30:01.831 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:01.826+0000 7f8081762640 1 --2- 172.21.15.37:0/3502947968 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f8068004d40 0x7f8068005150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:30:01.831 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:01.826+0000 7f8080f61640 1 --2- 172.21.15.37:0/3502947968 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f807c0d3060 0x7f80680039b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:30:01.831 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:01.826+0000 7f807bfff640 1 --2- 172.21.15.37:0/3502947968 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f8068003f30 0x7f8068004800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:30:01.831 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:01.826+0000 7f8081762640 1 -- 172.21.15.37:0/3502947968 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f807c0d3060 msgr2=0x7f80680039b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:30:01.831 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:01.826+0000 7f8081762640 1 --2- 172.21.15.37:0/3502947968 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f807c0d3060 0x7f80680039b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:30:01.831 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:01.826+0000 7f8081762640 1 -- 172.21.15.37:0/3502947968 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f8068003f30 msgr2=0x7f8068004800 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:30:01.831 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:01.826+0000 7f8081762640 1 --2- 172.21.15.37:0/3502947968 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f8068003f30 0x7f8068004800 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:30:01.831 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:01.826+0000 7f8081762640 1 -- 172.21.15.37:0/3502947968 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f8068003520 con 0x7f8068004d40 2024-04-25T10:30:01.831 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:01.827+0000 7f8081762640 1 --2- 172.21.15.37:0/3502947968 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f8068004d40 0x7f8068005150 secure :-1 s=READY pgs=376 cs=0 l=1 rev1=1 crypto rx=0x7f807c08f070 tx=0x7f8068005830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=290f244b1df38c18 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:30:01.831 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:01.827+0000 7f807b7fe640 1 -- 172.21.15.37:0/3502947968 <== mon.0 v2:172.21.15.37:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8064036070 con 0x7f8068004d40 2024-04-25T10:30:01.832 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:01.827+0000 7f807b7fe640 1 -- 172.21.15.37:0/3502947968 <== mon.0 v2:172.21.15.37:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f8064031400 con 0x7f8068004d40 2024-04-25T10:30:01.832 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:01.827+0000 7f807b7fe640 1 -- 172.21.15.37:0/3502947968 <== mon.0 v2:172.21.15.37:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8064009690 con 0x7f8068004d40 2024-04-25T10:30:01.833 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:01.828+0000 7f80831c5640 1 -- 172.21.15.37:0/3502947968 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f8068004d40 msgr2=0x7f8068005150 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:30:01.833 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:01.828+0000 7f80831c5640 1 --2- 172.21.15.37:0/3502947968 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f8068004d40 0x7f8068005150 secure :-1 s=READY pgs=376 cs=0 l=1 rev1=1 crypto rx=0x7f807c08f070 tx=0x7f8068005830 comp rx=0 tx=0).stop 2024-04-25T10:30:01.833 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:01.828+0000 7f8081762640 1 -- 172.21.15.37:0/3502947968 reap_dead start 2024-04-25T10:30:01.833 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:01.828+0000 7f80831c5640 1 -- 172.21.15.37:0/3502947968 shutdown_connections 2024-04-25T10:30:01.833 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:01.828+0000 7f80831c5640 1 -- 172.21.15.37:0/3502947968 >> 172.21.15.37:0/3502947968 conn(0x7f807c0b7a80 msgr2=0x7f807c0b7e90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:30:01.833 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:01.828+0000 7f80831c5640 1 -- 172.21.15.37:0/3502947968 shutdown_connections 2024-04-25T10:30:01.833 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:01.828+0000 7f80831c5640 1 -- 172.21.15.37:0/3502947968 wait complete. 2024-04-25T10:30:01.833 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:01.830+0000 7f80831c5640 1 Processor -- start 2024-04-25T10:30:01.833 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:01.830+0000 7f80831c5640 1 -- start start 2024-04-25T10:30:01.835 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:01.831+0000 7f80831c5640 1 --2- >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f807c0cf7b0 0x7f807c151bd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:30:01.835 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:01.831+0000 7f80831c5640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f807c0ccc00 0x7f807c0cd010 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:30:01.835 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:01.831+0000 7f80831c5640 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f807c15d850 0x7f807c15dc60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:30:01.835 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:01.831+0000 7f80831c5640 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 -- 0x7f807c0d4960 con 0x7f807c0ccc00 2024-04-25T10:30:01.835 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:01.831+0000 7f80831c5640 1 -- --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_getmap magic: 0 -- 0x7f807c0d4680 con 0x7f807c15d850 2024-04-25T10:30:01.835 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:01.831+0000 7f80831c5640 1 -- --> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] -- mon_getmap magic: 0 -- 0x7f807c0d4c40 con 0x7f807c0cf7b0 2024-04-25T10:30:01.835 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:01.831+0000 7f807bfff640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f807c0ccc00 0x7f807c0cd010 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:30:01.836 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:01.831+0000 7f807bfff640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f807c0ccc00 0x7f807c0cd010 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.37:3300/0 says I am v2:172.21.15.37:54564/0 (socket says 172.21.15.37:54564) 2024-04-25T10:30:01.836 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:01.831+0000 7f807bfff640 1 -- 172.21.15.37:0/48970133 learned_addr learned my addr 172.21.15.37:0/48970133 (peer_addr_for_me v2:172.21.15.37:0/0) 2024-04-25T10:30:01.836 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:01.832+0000 7f807bfff640 1 -- 172.21.15.37:0/48970133 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f807c0cf7b0 msgr2=0x7f807c151bd0 unknown :-1 s=STATE_CONNECTING_RE l=1).mark_down 2024-04-25T10:30:01.836 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:01.832+0000 7f807bfff640 1 --2- 172.21.15.37:0/48970133 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f807c0cf7b0 0x7f807c151bd0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:30:01.836 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:01.832+0000 7f807bfff640 1 -- 172.21.15.37:0/48970133 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f807c15d850 msgr2=0x7f807c15dc60 unknown :-1 s=STATE_CONNECTING_RE l=1).mark_down 2024-04-25T10:30:01.836 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:01.832+0000 7f807bfff640 1 --2- 172.21.15.37:0/48970133 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f807c15d850 0x7f807c15dc60 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:30:01.836 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:01.832+0000 7f807bfff640 1 -- 172.21.15.37:0/48970133 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f807c15e1a0 con 0x7f807c0ccc00 2024-04-25T10:30:01.836 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:01.832+0000 7f807bfff640 1 --2- 172.21.15.37:0/48970133 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f807c0ccc00 0x7f807c0cd010 secure :-1 s=READY pgs=377 cs=0 l=1 rev1=1 crypto rx=0x7f806c011960 tx=0x7f806c011e20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:30:01.836 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:01.832+0000 7f80797fa640 1 -- 172.21.15.37:0/48970133 <== mon.0 v2:172.21.15.37:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f806c00fa70 con 0x7f807c0ccc00 2024-04-25T10:30:01.836 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:01.832+0000 7f80797fa640 1 -- 172.21.15.37:0/48970133 <== mon.0 v2:172.21.15.37:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f806c00fc10 con 0x7f807c0ccc00 2024-04-25T10:30:01.836 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:01.832+0000 7f80797fa640 1 -- 172.21.15.37:0/48970133 <== mon.0 v2:172.21.15.37:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f806c014440 con 0x7f807c0ccc00 2024-04-25T10:30:01.836 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:01.832+0000 7f80831c5640 1 -- 172.21.15.37:0/48970133 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f807c152110 con 0x7f807c0ccc00 2024-04-25T10:30:01.836 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:01.832+0000 7f80831c5640 1 -- 172.21.15.37:0/48970133 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f807c0cff70 con 0x7f807c0ccc00 2024-04-25T10:30:01.836 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:01.833+0000 7f80831c5640 1 -- 172.21.15.37:0/48970133 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f807c06a140 con 0x7f807c0ccc00 2024-04-25T10:30:01.837 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:01.834+0000 7f80797fa640 1 -- 172.21.15.37:0/48970133 <== mon.0 v2:172.21.15.37:3300/0 4 ==== mgrmap(e 5) ==== 96782+0+0 (secure 0 0 0) 0x7f806c012040 con 0x7f807c0ccc00 2024-04-25T10:30:01.837 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:01.834+0000 7f80797fa640 1 --2- 172.21.15.37:0/48970133 >> [v2:172.21.15.203:6832/2418205237,v1:172.21.15.203:6833/2418205237] conn(0x7f805c0744b0 0x7f805c0769c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:30:01.837 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:01.834+0000 7f80797fa640 1 -- 172.21.15.37:0/48970133 <== mon.0 v2:172.21.15.37:3300/0 5 ==== osd_map(26..26 src has 1..26) ==== 5824+0+0 (secure 0 0 0) 0x7f806c095900 con 0x7f807c0ccc00 2024-04-25T10:30:01.837 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:01.835+0000 7f8080f61640 1 --2- 172.21.15.37:0/48970133 >> [v2:172.21.15.203:6832/2418205237,v1:172.21.15.203:6833/2418205237] conn(0x7f805c0744b0 0x7f805c0769c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:30:01.837 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:01.835+0000 7f8080f61640 1 --2- 172.21.15.37:0/48970133 >> [v2:172.21.15.203:6832/2418205237,v1:172.21.15.203:6833/2418205237] conn(0x7f805c0744b0 0x7f805c0769c0 secure :-1 s=READY pgs=190 cs=0 l=1 rev1=1 crypto rx=0x7f8074001630 tx=0x7f8074004b60 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:30:01.841 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:01.839+0000 7f80797fa640 1 -- 172.21.15.37:0/48970133 <== mon.0 v2:172.21.15.37:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187246 (secure 0 0 0) 0x7f807c15e1a0 con 0x7f807c0ccc00 2024-04-25T10:30:02.068 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:02.066+0000 7f80831c5640 1 -- 172.21.15.37:0/48970133 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_command({"prefix": "fs rm", "fs_name": "cephfs", "yes_i_really_mean_it": true} v 0) -- 0x7f807c15f820 con 0x7f807c0ccc00 2024-04-25T10:30:02.704 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:02.702+0000 7f80797fa640 1 -- 172.21.15.37:0/48970133 <== mon.0 v2:172.21.15.37:3300/0 7 ==== mon_command_ack([{"prefix": "fs rm", "fs_name": "cephfs", "yes_i_really_mean_it": true}]=0 v10) ==== 104+0+0 (secure 0 0 0) 0x7f806c019030 con 0x7f807c0ccc00 2024-04-25T10:30:02.707 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:02.705+0000 7f80831c5640 1 -- 172.21.15.37:0/48970133 >> [v2:172.21.15.203:6832/2418205237,v1:172.21.15.203:6833/2418205237] conn(0x7f805c0744b0 msgr2=0x7f805c0769c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:30:02.707 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:02.705+0000 7f80831c5640 1 --2- 172.21.15.37:0/48970133 >> [v2:172.21.15.203:6832/2418205237,v1:172.21.15.203:6833/2418205237] conn(0x7f805c0744b0 0x7f805c0769c0 secure :-1 s=READY pgs=190 cs=0 l=1 rev1=1 crypto rx=0x7f8074001630 tx=0x7f8074004b60 comp rx=0 tx=0).stop 2024-04-25T10:30:02.707 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:02.705+0000 7f80831c5640 1 -- 172.21.15.37:0/48970133 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f807c0ccc00 msgr2=0x7f807c0cd010 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:30:02.707 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:02.705+0000 7f80831c5640 1 --2- 172.21.15.37:0/48970133 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f807c0ccc00 0x7f807c0cd010 secure :-1 s=READY pgs=377 cs=0 l=1 rev1=1 crypto rx=0x7f806c011960 tx=0x7f806c011e20 comp rx=0 tx=0).stop 2024-04-25T10:30:02.707 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:02.705+0000 7f80831c5640 1 -- 172.21.15.37:0/48970133 shutdown_connections 2024-04-25T10:30:02.707 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:02.705+0000 7f80831c5640 1 --2- 172.21.15.37:0/48970133 >> [v2:172.21.15.203:6832/2418205237,v1:172.21.15.203:6833/2418205237] conn(0x7f805c0744b0 0x7f805c0769c0 unknown :-1 s=CLOSED pgs=190 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:30:02.707 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:02.705+0000 7f80831c5640 1 --2- 172.21.15.37:0/48970133 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f807c15d850 0x7f807c15dc60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:30:02.708 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:02.705+0000 7f80831c5640 1 --2- 172.21.15.37:0/48970133 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f807c0ccc00 0x7f807c0cd010 unknown :-1 s=CLOSED pgs=377 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:30:02.708 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:02.705+0000 7f80831c5640 1 --2- 172.21.15.37:0/48970133 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f807c0cf7b0 0x7f807c151bd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:30:02.708 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:02.705+0000 7f80831c5640 1 -- 172.21.15.37:0/48970133 >> 172.21.15.37:0/48970133 conn(0x7f807c0b7a80 msgr2=0x7f807c075fa0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:30:02.708 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:02.705+0000 7f80831c5640 1 -- 172.21.15.37:0/48970133 shutdown_connections 2024-04-25T10:30:02.708 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:02.705+0000 7f80831c5640 1 -- 172.21.15.37:0/48970133 wait complete. 2024-04-25T10:30:02.721 DEBUG:teuthology.orchestra.run.smithi037:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool rm cephfs_metadata cephfs_metadata --yes-i-really-really-mean-it 2024-04-25T10:30:02.843 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:02.841+0000 7f98679f3640 1 Processor -- start 2024-04-25T10:30:02.843 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:02.841+0000 7f98679f3640 1 -- start start 2024-04-25T10:30:02.848 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:02.842+0000 7f98679f3640 1 --2- >> v2:172.21.15.203:3300/0 conn(0x7f98601603c0 0x7f98600d13f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:30:02.848 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:02.842+0000 7f98679f3640 1 --2- >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f9860074380 0x7f98600d4610 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:30:02.848 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:02.842+0000 7f98679f3640 1 -- --> v1:172.21.15.37:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f9860163ec0 con 0x7f98600b4840 2024-04-25T10:30:02.848 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:02.842+0000 7f98679f3640 1 -- --> v2:172.21.15.203:3300/0 -- mon_getmap magic: 0 -- 0x7f9860164c80 con 0x7f98601603c0 2024-04-25T10:30:02.848 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:02.842+0000 7f98679f3640 1 -- --> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] -- mon_getmap magic: 0 -- 0x7f98601641a0 con 0x7f9860074380 2024-04-25T10:30:02.848 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:02.842+0000 7f9864f8e640 1 --1- >> v1:172.21.15.37:6789/0 conn(0x7f98600b4840 0x7f9860073c70 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.37:6789/0 says I am v1:172.21.15.37:60678/0 (socket says 172.21.15.37:60678) 2024-04-25T10:30:02.848 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:02.842+0000 7f9864f8e640 1 -- 172.21.15.37:0/4208999971 learned_addr learned my addr 172.21.15.37:0/4208999971 (peer_addr_for_me v1:172.21.15.37:0/0) 2024-04-25T10:30:02.848 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:02.843+0000 7f9865f90640 1 --2- 172.21.15.37:0/4208999971 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f9860074380 0x7f98600d4610 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:30:02.848 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:02.843+0000 7f986578f640 1 --2- 172.21.15.37:0/4208999971 >> v2:172.21.15.203:3300/0 conn(0x7f98601603c0 0x7f98600d13f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:30:02.848 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:02.843+0000 7f9857fff640 1 -- 172.21.15.37:0/4208999971 <== mon.0 v1:172.21.15.37:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 1637036202 0 0) 0x7f9860163ec0 con 0x7f98600b4840 2024-04-25T10:30:02.848 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:02.843+0000 7f9857fff640 1 -- 172.21.15.37:0/4208999971 --> v1:172.21.15.37:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f984c003520 con 0x7f98600b4840 2024-04-25T10:30:02.848 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:02.843+0000 7f9857fff640 1 -- 172.21.15.37:0/4208999971 <== mon.0 v1:172.21.15.37:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 2984552016 0 0) 0x7f984c003520 con 0x7f98600b4840 2024-04-25T10:30:02.848 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:02.843+0000 7f9857fff640 1 -- 172.21.15.37:0/4208999971 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f9860074380 msgr2=0x7f98600d4610 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:30:02.849 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:02.843+0000 7f9857fff640 1 --2- 172.21.15.37:0/4208999971 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f9860074380 0x7f98600d4610 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:30:02.849 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:02.843+0000 7f9857fff640 1 -- 172.21.15.37:0/4208999971 >> v2:172.21.15.203:3300/0 conn(0x7f98601603c0 msgr2=0x7f98600d13f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:30:02.849 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:02.843+0000 7f9857fff640 1 --2- 172.21.15.37:0/4208999971 >> v2:172.21.15.203:3300/0 conn(0x7f98601603c0 0x7f98600d13f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:30:02.849 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:02.843+0000 7f9857fff640 1 -- 172.21.15.37:0/4208999971 --> v1:172.21.15.37:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f9860164480 con 0x7f98600b4840 2024-04-25T10:30:02.849 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:02.843+0000 7f9857fff640 1 -- 172.21.15.37:0/4208999971 <== mon.0 v1:172.21.15.37:6789/0 3 ==== mon_map magic: 0 ==== 413+0+0 (unknown 1577794600 0 0) 0x7f9850001670 con 0x7f98600b4840 2024-04-25T10:30:02.849 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:02.843+0000 7f986578f640 1 --2- 172.21.15.37:0/4208999971 >> v2:172.21.15.203:3300/0 conn(0x7f98601603c0 0x7f98600d13f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-25T10:30:02.849 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:02.843+0000 7f9857fff640 1 -- 172.21.15.37:0/4208999971 >> v1:172.21.15.37:6789/0 conn(0x7f98600b4840 legacy=0x7f9860073c70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:30:02.849 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:02.843+0000 7f9857fff640 1 --2- 172.21.15.37:0/4208999971 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f984c0039b0 0x7f984c003dc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:30:02.849 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:02.843+0000 7f9857fff640 1 --2- 172.21.15.37:0/4208999971 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f984c004340 0x7f984c004860 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:30:02.849 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:02.843+0000 7f9857fff640 1 --2- 172.21.15.37:0/4208999971 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f984c004ed0 0x7f984c00b9c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:30:02.849 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:02.843+0000 7f9857fff640 1 -- 172.21.15.37:0/4208999971 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 -- 0x7f98601641a0 con 0x7f984c004340 2024-04-25T10:30:02.849 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:02.843+0000 7f9857fff640 1 -- 172.21.15.37:0/4208999971 --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_getmap magic: 0 -- 0x7f984c00bf00 con 0x7f984c004ed0 2024-04-25T10:30:02.849 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:02.843+0000 7f9857fff640 1 -- 172.21.15.37:0/4208999971 --> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] -- mon_getmap magic: 0 -- 0x7f984c00c080 con 0x7f984c0039b0 2024-04-25T10:30:02.849 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:02.843+0000 7f9864f8e640 1 --2- 172.21.15.37:0/4208999971 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f984c004340 0x7f984c004860 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:30:02.849 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:02.843+0000 7f986578f640 1 --2- 172.21.15.37:0/4208999971 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f984c0039b0 0x7f984c003dc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:30:02.849 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:02.843+0000 7f9865f90640 1 --2- 172.21.15.37:0/4208999971 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f984c004ed0 0x7f984c00b9c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:30:02.850 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:02.844+0000 7f9864f8e640 1 -- 172.21.15.37:0/4208999971 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f984c0039b0 msgr2=0x7f984c003dc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:30:02.850 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:02.844+0000 7f9864f8e640 1 --2- 172.21.15.37:0/4208999971 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f984c0039b0 0x7f984c003dc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:30:02.850 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:02.844+0000 7f9864f8e640 1 -- 172.21.15.37:0/4208999971 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f984c004ed0 msgr2=0x7f984c00b9c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:30:02.850 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:02.844+0000 7f9864f8e640 1 --2- 172.21.15.37:0/4208999971 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f984c004ed0 0x7f984c00b9c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:30:02.850 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:02.844+0000 7f9864f8e640 1 -- 172.21.15.37:0/4208999971 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f984c003520 con 0x7f984c004340 2024-04-25T10:30:02.850 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:02.844+0000 7f9864f8e640 1 --2- 172.21.15.37:0/4208999971 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f984c004340 0x7f984c004860 secure :-1 s=READY pgs=379 cs=0 l=1 rev1=1 crypto rx=0x7f9850007fd0 tx=0x7f984c005830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=5ca9cc3bac9922fe server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:30:02.850 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:02.845+0000 7f9857fff640 1 -- 172.21.15.37:0/4208999971 <== mon.0 v2:172.21.15.37:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9850007d50 con 0x7f984c004340 2024-04-25T10:30:02.850 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:02.845+0000 7f9857fff640 1 -- 172.21.15.37:0/4208999971 <== mon.0 v2:172.21.15.37:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f9850005ad0 con 0x7f984c004340 2024-04-25T10:30:02.850 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:02.845+0000 7f9857fff640 1 -- 172.21.15.37:0/4208999971 <== mon.0 v2:172.21.15.37:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9850007070 con 0x7f984c004340 2024-04-25T10:30:02.850 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:02.845+0000 7f98679f3640 1 -- 172.21.15.37:0/4208999971 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f984c004340 msgr2=0x7f984c004860 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:30:02.850 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:02.845+0000 7f98679f3640 1 --2- 172.21.15.37:0/4208999971 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f984c004340 0x7f984c004860 secure :-1 s=READY pgs=379 cs=0 l=1 rev1=1 crypto rx=0x7f9850007fd0 tx=0x7f984c005830 comp rx=0 tx=0).stop 2024-04-25T10:30:02.850 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:02.845+0000 7f9865f90640 1 -- 172.21.15.37:0/4208999971 reap_dead start 2024-04-25T10:30:02.850 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:02.845+0000 7f98679f3640 1 -- 172.21.15.37:0/4208999971 shutdown_connections 2024-04-25T10:30:02.850 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:02.845+0000 7f98679f3640 1 -- 172.21.15.37:0/4208999971 >> 172.21.15.37:0/4208999971 conn(0x7f98600d5d60 msgr2=0x7f98600d6170 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:30:02.850 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:02.845+0000 7f98679f3640 1 -- 172.21.15.37:0/4208999971 shutdown_connections 2024-04-25T10:30:02.851 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:02.845+0000 7f98679f3640 1 -- 172.21.15.37:0/4208999971 wait complete. 2024-04-25T10:30:02.853 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:02.847+0000 7f98679f3640 1 Processor -- start 2024-04-25T10:30:02.853 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:02.847+0000 7f98679f3640 1 -- start start 2024-04-25T10:30:02.853 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:02.848+0000 7f98679f3640 1 --2- >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f984c004340 0x7f9860155c30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:30:02.853 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:02.848+0000 7f98679f3640 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f98600c8d90 0x7f98601592e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:30:02.853 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:02.848+0000 7f98679f3640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f98600c91a0 0x7f9860159820 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:30:02.853 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:02.848+0000 7f98679f3640 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 -- 0x7f98600b92c0 con 0x7f98600c91a0 2024-04-25T10:30:02.853 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:02.848+0000 7f98679f3640 1 -- --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_getmap magic: 0 -- 0x7f98600b9140 con 0x7f98600c8d90 2024-04-25T10:30:02.853 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:02.848+0000 7f98679f3640 1 -- --> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] -- mon_getmap magic: 0 -- 0x7f98600b4d90 con 0x7f984c004340 2024-04-25T10:30:02.853 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:02.849+0000 7f9865f90640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f98600c91a0 0x7f9860159820 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:30:02.853 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:02.849+0000 7f9865f90640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f98600c91a0 0x7f9860159820 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.37:3300/0 says I am v2:172.21.15.37:54580/0 (socket says 172.21.15.37:54580) 2024-04-25T10:30:02.853 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:02.849+0000 7f9865f90640 1 -- 172.21.15.37:0/4165097247 learned_addr learned my addr 172.21.15.37:0/4165097247 (peer_addr_for_me v2:172.21.15.37:0/0) 2024-04-25T10:30:02.853 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:02.849+0000 7f986578f640 1 --2- 172.21.15.37:0/4165097247 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f984c004340 0x7f9860155c30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:30:02.853 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:02.849+0000 7f9864f8e640 1 --2- 172.21.15.37:0/4165097247 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f98600c8d90 0x7f98601592e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:30:02.854 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:02.849+0000 7f9865f90640 1 -- 172.21.15.37:0/4165097247 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f984c004340 msgr2=0x7f9860155c30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:30:02.854 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:02.849+0000 7f9865f90640 1 --2- 172.21.15.37:0/4165097247 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f984c004340 0x7f9860155c30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:30:02.854 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:02.849+0000 7f9865f90640 1 -- 172.21.15.37:0/4165097247 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f98600c8d90 msgr2=0x7f98601592e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:30:02.854 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:02.849+0000 7f9865f90640 1 --2- 172.21.15.37:0/4165097247 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f98600c8d90 0x7f98601592e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:30:02.854 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:02.849+0000 7f9865f90640 1 -- 172.21.15.37:0/4165097247 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f9860156610 con 0x7f98600c91a0 2024-04-25T10:30:02.854 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:02.849+0000 7f986578f640 1 --2- 172.21.15.37:0/4165097247 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f984c004340 0x7f9860155c30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T10:30:02.854 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:02.849+0000 7f9864f8e640 1 --2- 172.21.15.37:0/4165097247 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f98600c8d90 0x7f98601592e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T10:30:02.854 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:02.849+0000 7f9865f90640 1 --2- 172.21.15.37:0/4165097247 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f98600c91a0 0x7f9860159820 secure :-1 s=READY pgs=380 cs=0 l=1 rev1=1 crypto rx=0x7f98600b7680 tx=0x7f985800f580 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:30:02.854 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:02.849+0000 7f9855ffb640 1 -- 172.21.15.37:0/4165097247 <== mon.0 v2:172.21.15.37:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9858011e90 con 0x7f98600c91a0 2024-04-25T10:30:02.854 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:02.849+0000 7f9855ffb640 1 -- 172.21.15.37:0/4165097247 <== mon.0 v2:172.21.15.37:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f985801c070 con 0x7f98600c91a0 2024-04-25T10:30:02.854 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:02.849+0000 7f98679f3640 1 -- 172.21.15.37:0/4165097247 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f98601568f0 con 0x7f98600c91a0 2024-04-25T10:30:02.854 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:02.849+0000 7f9855ffb640 1 -- 172.21.15.37:0/4165097247 <== mon.0 v2:172.21.15.37:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9858003bd0 con 0x7f98600c91a0 2024-04-25T10:30:02.854 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:02.850+0000 7f98679f3640 1 -- 172.21.15.37:0/4165097247 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f986015c0c0 con 0x7f98600c91a0 2024-04-25T10:30:02.854 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:02.850+0000 7f98679f3640 1 -- 172.21.15.37:0/4165097247 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f986006a140 con 0x7f98600c91a0 2024-04-25T10:30:02.855 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:02.851+0000 7f9855ffb640 1 -- 172.21.15.37:0/4165097247 <== mon.0 v2:172.21.15.37:3300/0 4 ==== mgrmap(e 5) ==== 96782+0+0 (secure 0 0 0) 0x7f9858004030 con 0x7f98600c91a0 2024-04-25T10:30:02.855 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:02.852+0000 7f9855ffb640 1 --2- 172.21.15.37:0/4165097247 >> [v2:172.21.15.203:6832/2418205237,v1:172.21.15.203:6833/2418205237] conn(0x7f9844074650 0x7f9844076b60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:30:02.855 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:02.852+0000 7f9855ffb640 1 -- 172.21.15.37:0/4165097247 <== mon.0 v2:172.21.15.37:3300/0 5 ==== osd_map(26..26 src has 1..26) ==== 5824+0+0 (secure 0 0 0) 0x7f9858095b40 con 0x7f98600c91a0 2024-04-25T10:30:02.855 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:02.852+0000 7f986578f640 1 --2- 172.21.15.37:0/4165097247 >> [v2:172.21.15.203:6832/2418205237,v1:172.21.15.203:6833/2418205237] conn(0x7f9844074650 0x7f9844076b60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:30:02.855 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:02.853+0000 7f986578f640 1 --2- 172.21.15.37:0/4165097247 >> [v2:172.21.15.203:6832/2418205237,v1:172.21.15.203:6833/2418205237] conn(0x7f9844074650 0x7f9844076b60 secure :-1 s=READY pgs=192 cs=0 l=1 rev1=1 crypto rx=0x7f985c009c40 tx=0x7f985c008040 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:30:02.858 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:02.857+0000 7f9855ffb640 1 -- 172.21.15.37:0/4165097247 <== mon.0 v2:172.21.15.37:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187246 (secure 0 0 0) 0x7f9858010040 con 0x7f98600c91a0 2024-04-25T10:30:03.042 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:03.039+0000 7f98679f3640 1 -- 172.21.15.37:0/4165097247 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_command({"prefix": "osd pool rm", "pool": "cephfs_metadata", "pool2": "cephfs_metadata", "yes_i_really_really_mean_it": true} v 0) -- 0x7f9860156380 con 0x7f98600c91a0 2024-04-25T10:30:03.707 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:03.704+0000 7f9855ffb640 1 -- 172.21.15.37:0/4165097247 <== mon.0 v2:172.21.15.37:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool rm", "pool": "cephfs_metadata", "pool2": "cephfs_metadata", "yes_i_really_really_mean_it": true}]=0 pool 'cephfs_metadata' removed v27) ==== 181+0+0 (secure 0 0 0) 0x7f98580602f0 con 0x7f98600c91a0 2024-04-25T10:30:03.707 INFO:teuthology.orchestra.run.smithi037.stderr:pool 'cephfs_metadata' removed 2024-04-25T10:30:03.709 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:03.707+0000 7f98679f3640 1 -- 172.21.15.37:0/4165097247 >> [v2:172.21.15.203:6832/2418205237,v1:172.21.15.203:6833/2418205237] conn(0x7f9844074650 msgr2=0x7f9844076b60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:30:03.709 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:03.707+0000 7f98679f3640 1 --2- 172.21.15.37:0/4165097247 >> [v2:172.21.15.203:6832/2418205237,v1:172.21.15.203:6833/2418205237] conn(0x7f9844074650 0x7f9844076b60 secure :-1 s=READY pgs=192 cs=0 l=1 rev1=1 crypto rx=0x7f985c009c40 tx=0x7f985c008040 comp rx=0 tx=0).stop 2024-04-25T10:30:03.710 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:03.707+0000 7f98679f3640 1 -- 172.21.15.37:0/4165097247 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f98600c91a0 msgr2=0x7f9860159820 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:30:03.710 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:03.707+0000 7f98679f3640 1 --2- 172.21.15.37:0/4165097247 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f98600c91a0 0x7f9860159820 secure :-1 s=READY pgs=380 cs=0 l=1 rev1=1 crypto rx=0x7f98600b7680 tx=0x7f985800f580 comp rx=0 tx=0).stop 2024-04-25T10:30:03.711 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:03.708+0000 7f98679f3640 1 -- 172.21.15.37:0/4165097247 shutdown_connections 2024-04-25T10:30:03.711 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:03.709+0000 7f98679f3640 1 --2- 172.21.15.37:0/4165097247 >> [v2:172.21.15.203:6832/2418205237,v1:172.21.15.203:6833/2418205237] conn(0x7f9844074650 0x7f9844076b60 unknown :-1 s=CLOSED pgs=192 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:30:03.711 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:03.709+0000 7f98679f3640 1 --2- 172.21.15.37:0/4165097247 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f98600c91a0 0x7f9860159820 unknown :-1 s=CLOSED pgs=380 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:30:03.711 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:03.709+0000 7f98679f3640 1 --2- 172.21.15.37:0/4165097247 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f98600c8d90 0x7f98601592e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:30:03.711 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:03.709+0000 7f98679f3640 1 --2- 172.21.15.37:0/4165097247 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f984c004340 0x7f9860155c30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:30:03.711 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:03.709+0000 7f98679f3640 1 -- 172.21.15.37:0/4165097247 >> 172.21.15.37:0/4165097247 conn(0x7f98600d5d60 msgr2=0x7f98600b66a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:30:03.712 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:03.709+0000 7f98679f3640 1 -- 172.21.15.37:0/4165097247 shutdown_connections 2024-04-25T10:30:03.712 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:03.709+0000 7f98679f3640 1 -- 172.21.15.37:0/4165097247 wait complete. 2024-04-25T10:30:03.727 DEBUG:teuthology.orchestra.run.smithi037:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool rm cephfs_data cephfs_data --yes-i-really-really-mean-it 2024-04-25T10:30:03.849 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:03.847+0000 7f26bcc92640 1 Processor -- start 2024-04-25T10:30:03.849 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:03.847+0000 7f26bcc92640 1 -- start start 2024-04-25T10:30:03.851 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:03.848+0000 7f26bcc92640 1 --2- >> v2:172.21.15.37:3300/0 conn(0x7f26b8160260 0x7f26b80d4e30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:30:03.851 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:03.848+0000 7f26bcc92640 1 -- --> v1:172.21.15.37:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f26b80d0f70 con 0x7f26b80d0940 2024-04-25T10:30:03.851 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:03.848+0000 7f26bcc92640 1 -- --> v1:172.21.15.203:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f26b80d1250 con 0x7f26b80b79f0 2024-04-25T10:30:03.851 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:03.848+0000 7f26bcc92640 1 -- --> v2:172.21.15.37:3300/0 -- mon_getmap magic: 0 -- 0x7f26b80d0d50 con 0x7f26b8160260 2024-04-25T10:30:03.851 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:03.848+0000 7f26b5d9b640 1 --2- >> v2:172.21.15.37:3300/0 conn(0x7f26b8160260 0x7f26b80d4e30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:30:03.851 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:03.848+0000 7f26b5d9b640 1 --2- >> v2:172.21.15.37:3300/0 conn(0x7f26b8160260 0x7f26b80d4e30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.37:3300/0 says I am v2:172.21.15.37:54592/0 (socket says 172.21.15.37:54592) 2024-04-25T10:30:03.851 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:03.848+0000 7f26b5d9b640 1 -- 172.21.15.37:0/1911301449 learned_addr learned my addr 172.21.15.37:0/1911301449 (peer_addr_for_me v2:172.21.15.37:0/0) 2024-04-25T10:30:03.851 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:03.848+0000 7f26b659c640 1 --1- >> v1:172.21.15.37:6789/0 conn(0x7f26b80d0940 0x7f26b81647b0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.37:6789/0 says I am v1:172.21.15.37:60694/0 (socket says 172.21.15.37:60694) 2024-04-25T10:30:03.852 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:03.849+0000 7f26b5d9b640 1 -- 172.21.15.37:0/1911301449 >> v1:172.21.15.203:6789/0 conn(0x7f26b80b79f0 legacy=0x7f26b80b7e00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:30:03.852 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:03.849+0000 7f26b559a640 1 -- 172.21.15.37:0/1911301449 <== mon.0 v1:172.21.15.37:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 21523516 0 0) 0x7f26b80d0f70 con 0x7f26b80d0940 2024-04-25T10:30:03.852 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:03.849+0000 7f26b5d9b640 1 -- 172.21.15.37:0/1911301449 >> v1:172.21.15.37:6789/0 conn(0x7f26b80d0940 legacy=0x7f26b81647b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:30:03.852 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:03.849+0000 7f26b5d9b640 1 -- 172.21.15.37:0/1911301449 --> v2:172.21.15.37:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f26b80d4630 con 0x7f26b8160260 2024-04-25T10:30:03.853 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:03.850+0000 7f26b5d9b640 1 --2- 172.21.15.37:0/1911301449 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f26b8160260 0x7f26b80d4e30 secure :-1 s=READY pgs=382 cs=0 l=1 rev1=1 crypto rx=0x7f26ac009870 tx=0x7f26ac02a300 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e958d58a68f75b07 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:30:03.853 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:03.850+0000 7f26b559a640 1 -- 172.21.15.37:0/1911301449 <== mon.0 v2:172.21.15.37:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f26ac037070 con 0x7f26b8160260 2024-04-25T10:30:03.853 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:03.850+0000 7f26b559a640 1 -- 172.21.15.37:0/1911301449 <== mon.0 v2:172.21.15.37:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f26ac02ae50 con 0x7f26b8160260 2024-04-25T10:30:03.853 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:03.850+0000 7f26b559a640 1 -- 172.21.15.37:0/1911301449 <== mon.0 v2:172.21.15.37:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f26ac032860 con 0x7f26b8160260 2024-04-25T10:30:03.853 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:03.850+0000 7f26bcc92640 1 -- 172.21.15.37:0/1911301449 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f26b8160260 msgr2=0x7f26b80d4e30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:30:03.853 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:03.850+0000 7f26bcc92640 1 --2- 172.21.15.37:0/1911301449 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f26b8160260 0x7f26b80d4e30 secure :-1 s=READY pgs=382 cs=0 l=1 rev1=1 crypto rx=0x7f26ac009870 tx=0x7f26ac02a300 comp rx=0 tx=0).stop 2024-04-25T10:30:03.853 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:03.851+0000 7f26bcc92640 1 -- 172.21.15.37:0/1911301449 shutdown_connections 2024-04-25T10:30:03.854 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:03.851+0000 7f26bcc92640 1 --2- 172.21.15.37:0/1911301449 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f26b8160260 0x7f26b80d4e30 unknown :-1 s=CLOSED pgs=382 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:30:03.854 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:03.851+0000 7f26bcc92640 1 -- 172.21.15.37:0/1911301449 >> 172.21.15.37:0/1911301449 conn(0x7f26b80d5d70 msgr2=0x7f26b80d6180 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:30:03.854 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:03.851+0000 7f26bcc92640 1 -- 172.21.15.37:0/1911301449 shutdown_connections 2024-04-25T10:30:03.854 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:03.851+0000 7f26bcc92640 1 -- 172.21.15.37:0/1911301449 wait complete. 2024-04-25T10:30:03.855 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:03.853+0000 7f26bcc92640 1 Processor -- start 2024-04-25T10:30:03.855 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:03.853+0000 7f26bcc92640 1 -- start start 2024-04-25T10:30:03.857 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:03.854+0000 7f26bcc92640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f26b80b79f0 0x7f26b8161f60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:30:03.857 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:03.854+0000 7f26bcc92640 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f26b80d0940 0x7f26b8163260 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:30:03.858 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:03.854+0000 7f26bcc92640 1 --2- >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f26b8160260 0x7f26b80cb700 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:30:03.858 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:03.854+0000 7f26bcc92640 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 -- 0x7f26b80d1530 con 0x7f26b80b79f0 2024-04-25T10:30:03.858 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:03.854+0000 7f26bcc92640 1 -- --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_getmap magic: 0 -- 0x7f26b81606b0 con 0x7f26b80d0940 2024-04-25T10:30:03.858 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:03.854+0000 7f26bcc92640 1 -- --> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] -- mon_getmap magic: 0 -- 0x7f26b80d5370 con 0x7f26b8160260 2024-04-25T10:30:03.858 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:03.854+0000 7f26b659c640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f26b80b79f0 0x7f26b8161f60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:30:03.858 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:03.854+0000 7f26b659c640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f26b80b79f0 0x7f26b8161f60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.37:3300/0 says I am v2:172.21.15.37:54594/0 (socket says 172.21.15.37:54594) 2024-04-25T10:30:03.858 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:03.854+0000 7f26b659c640 1 -- 172.21.15.37:0/1872219901 learned_addr learned my addr 172.21.15.37:0/1872219901 (peer_addr_for_me v2:172.21.15.37:0/0) 2024-04-25T10:30:03.858 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:03.854+0000 7f26b5d9b640 1 --2- 172.21.15.37:0/1872219901 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f26b80d0940 0x7f26b8163260 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:30:03.858 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:03.854+0000 7f26b6d9d640 1 --2- 172.21.15.37:0/1872219901 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f26b8160260 0x7f26b80cb700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:30:03.858 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:03.854+0000 7f26b659c640 1 -- 172.21.15.37:0/1872219901 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f26b8160260 msgr2=0x7f26b80cb700 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:30:03.858 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:03.854+0000 7f26b659c640 1 --2- 172.21.15.37:0/1872219901 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f26b8160260 0x7f26b80cb700 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:30:03.858 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:03.854+0000 7f26b659c640 1 -- 172.21.15.37:0/1872219901 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f26b80d0940 msgr2=0x7f26b8163260 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:30:03.858 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:03.854+0000 7f26b659c640 1 --2- 172.21.15.37:0/1872219901 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f26b80d0940 0x7f26b8163260 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:30:03.859 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:03.854+0000 7f26b659c640 1 -- 172.21.15.37:0/1872219901 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f26b8160ec0 con 0x7f26b80b79f0 2024-04-25T10:30:03.859 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:03.854+0000 7f26b5d9b640 1 --2- 172.21.15.37:0/1872219901 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f26b80d0940 0x7f26b8163260 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T10:30:03.859 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:03.855+0000 7f26b659c640 1 --2- 172.21.15.37:0/1872219901 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f26b80b79f0 0x7f26b8161f60 secure :-1 s=READY pgs=383 cs=0 l=1 rev1=1 crypto rx=0x7f26a800c3f0 tx=0x7f26a800c8b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:30:03.859 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:03.855+0000 7f26a6ffd640 1 -- 172.21.15.37:0/1872219901 <== mon.0 v2:172.21.15.37:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f26a80074e0 con 0x7f26b80b79f0 2024-04-25T10:30:03.859 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:03.855+0000 7f26a6ffd640 1 -- 172.21.15.37:0/1872219901 <== mon.0 v2:172.21.15.37:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f26a80047e0 con 0x7f26b80b79f0 2024-04-25T10:30:03.859 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:03.855+0000 7f26bcc92640 1 -- 172.21.15.37:0/1872219901 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f26b81611a0 con 0x7f26b80b79f0 2024-04-25T10:30:03.859 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:03.855+0000 7f26a6ffd640 1 -- 172.21.15.37:0/1872219901 <== mon.0 v2:172.21.15.37:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f26a8004bd0 con 0x7f26b80b79f0 2024-04-25T10:30:03.859 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:03.855+0000 7f26bcc92640 1 -- 172.21.15.37:0/1872219901 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f26b8161760 con 0x7f26b80b79f0 2024-04-25T10:30:03.860 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:03.856+0000 7f26bcc92640 1 -- 172.21.15.37:0/1872219901 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f26b806a140 con 0x7f26b80b79f0 2024-04-25T10:30:03.860 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:03.857+0000 7f26a6ffd640 1 -- 172.21.15.37:0/1872219901 <== mon.0 v2:172.21.15.37:3300/0 4 ==== mgrmap(e 5) ==== 96782+0+0 (secure 0 0 0) 0x7f26a8011070 con 0x7f26b80b79f0 2024-04-25T10:30:03.860 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:03.857+0000 7f26a6ffd640 1 --2- 172.21.15.37:0/1872219901 >> [v2:172.21.15.203:6832/2418205237,v1:172.21.15.203:6833/2418205237] conn(0x7f26980744b0 0x7f26980769c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:30:03.860 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:03.857+0000 7f26a6ffd640 1 -- 172.21.15.37:0/1872219901 <== mon.0 v2:172.21.15.37:3300/0 5 ==== osd_map(27..27 src has 1..27) ==== 5386+0+0 (secure 0 0 0) 0x7f26a8094d00 con 0x7f26b80b79f0 2024-04-25T10:30:03.860 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:03.857+0000 7f26b5d9b640 1 --2- 172.21.15.37:0/1872219901 >> [v2:172.21.15.203:6832/2418205237,v1:172.21.15.203:6833/2418205237] conn(0x7f26980744b0 0x7f26980769c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:30:03.860 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:03.858+0000 7f26b5d9b640 1 --2- 172.21.15.37:0/1872219901 >> [v2:172.21.15.203:6832/2418205237,v1:172.21.15.203:6833/2418205237] conn(0x7f26980744b0 0x7f26980769c0 secure :-1 s=READY pgs=193 cs=0 l=1 rev1=1 crypto rx=0x7f26ac034000 tx=0x7f26ac02e000 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:30:03.864 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:03.862+0000 7f26a6ffd640 1 -- 172.21.15.37:0/1872219901 <== mon.0 v2:172.21.15.37:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187246 (secure 0 0 0) 0x7f26a805f5d0 con 0x7f26b80b79f0 2024-04-25T10:30:04.057 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:04.055+0000 7f26bcc92640 1 -- 172.21.15.37:0/1872219901 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_command({"prefix": "osd pool rm", "pool": "cephfs_data", "pool2": "cephfs_data", "yes_i_really_really_mean_it": true} v 0) -- 0x7f26b80d3f90 con 0x7f26b80b79f0 2024-04-25T10:30:04.714 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:04.711+0000 7f26a6ffd640 1 -- 172.21.15.37:0/1872219901 <== mon.0 v2:172.21.15.37:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool rm", "pool": "cephfs_data", "pool2": "cephfs_data", "yes_i_really_really_mean_it": true}]=0 pool 'cephfs_data' removed v28) ==== 169+0+0 (secure 0 0 0) 0x7f26a805eed0 con 0x7f26b80b79f0 2024-04-25T10:30:04.714 INFO:teuthology.orchestra.run.smithi037.stderr:pool 'cephfs_data' removed 2024-04-25T10:30:04.717 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:04.714+0000 7f26bcc92640 1 -- 172.21.15.37:0/1872219901 >> [v2:172.21.15.203:6832/2418205237,v1:172.21.15.203:6833/2418205237] conn(0x7f26980744b0 msgr2=0x7f26980769c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:30:04.717 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:04.714+0000 7f26bcc92640 1 --2- 172.21.15.37:0/1872219901 >> [v2:172.21.15.203:6832/2418205237,v1:172.21.15.203:6833/2418205237] conn(0x7f26980744b0 0x7f26980769c0 secure :-1 s=READY pgs=193 cs=0 l=1 rev1=1 crypto rx=0x7f26ac034000 tx=0x7f26ac02e000 comp rx=0 tx=0).stop 2024-04-25T10:30:04.717 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:04.714+0000 7f26bcc92640 1 -- 172.21.15.37:0/1872219901 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f26b80b79f0 msgr2=0x7f26b8161f60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:30:04.717 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:04.714+0000 7f26bcc92640 1 --2- 172.21.15.37:0/1872219901 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f26b80b79f0 0x7f26b8161f60 secure :-1 s=READY pgs=383 cs=0 l=1 rev1=1 crypto rx=0x7f26a800c3f0 tx=0x7f26a800c8b0 comp rx=0 tx=0).stop 2024-04-25T10:30:04.717 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:04.715+0000 7f26bcc92640 1 -- 172.21.15.37:0/1872219901 shutdown_connections 2024-04-25T10:30:04.718 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:04.715+0000 7f26bcc92640 1 --2- 172.21.15.37:0/1872219901 >> [v2:172.21.15.203:6832/2418205237,v1:172.21.15.203:6833/2418205237] conn(0x7f26980744b0 0x7f26980769c0 unknown :-1 s=CLOSED pgs=193 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:30:04.718 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:04.715+0000 7f26bcc92640 1 --2- 172.21.15.37:0/1872219901 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f26b8160260 0x7f26b80cb700 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:30:04.718 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:04.715+0000 7f26bcc92640 1 --2- 172.21.15.37:0/1872219901 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f26b80d0940 0x7f26b8163260 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:30:04.718 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:04.715+0000 7f26bcc92640 1 --2- 172.21.15.37:0/1872219901 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f26b80b79f0 0x7f26b8161f60 unknown :-1 s=CLOSED pgs=383 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:30:04.718 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:04.715+0000 7f26bcc92640 1 -- 172.21.15.37:0/1872219901 >> 172.21.15.37:0/1872219901 conn(0x7f26b80d5d70 msgr2=0x7f26b8075d00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:30:04.718 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:04.716+0000 7f26bcc92640 1 -- 172.21.15.37:0/1872219901 shutdown_connections 2024-04-25T10:30:04.718 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:04.716+0000 7f26bcc92640 1 -- 172.21.15.37:0/1872219901 wait complete. 2024-04-25T10:30:04.740 INFO:tasks.ceph.mds.a:Restarting daemon 2024-04-25T10:30:04.740 INFO:tasks.ceph.mds.a:Stopping old one... 2024-04-25T10:30:04.740 DEBUG:tasks.ceph.mds.a:waiting for process to exit 2024-04-25T10:30:04.740 INFO:teuthology.orchestra.run:waiting for 300 2024-04-25T10:30:04.741 INFO:tasks.ceph.mds.c:Restarting daemon 2024-04-25T10:30:04.741 INFO:tasks.ceph.mds.c:Stopping old one... 2024-04-25T10:30:04.741 DEBUG:tasks.ceph.mds.c:waiting for process to exit 2024-04-25T10:30:04.741 INFO:teuthology.orchestra.run:waiting for 300 2024-04-25T10:30:04.741 INFO:tasks.ceph.mds.b:Restarting daemon 2024-04-25T10:30:04.741 INFO:tasks.ceph.mds.b:Stopping old one... 2024-04-25T10:30:04.741 DEBUG:tasks.ceph.mds.b:waiting for process to exit 2024-04-25T10:30:04.742 INFO:teuthology.orchestra.run:waiting for 300 2024-04-25T10:30:04.742 INFO:tasks.ceph.mds.d:Restarting daemon 2024-04-25T10:30:04.742 INFO:tasks.ceph.mds.d:Stopping old one... 2024-04-25T10:30:04.742 DEBUG:tasks.ceph.mds.d:waiting for process to exit 2024-04-25T10:30:04.745 INFO:teuthology.orchestra.run:waiting for 300 2024-04-25T10:30:04.752 INFO:tasks.ceph.mds.a:Stopped 2024-04-25T10:30:04.752 DEBUG:teuthology.orchestra.run.smithi037:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mds -f --cluster ceph -i a 2024-04-25T10:30:04.753 INFO:tasks.ceph.mds.c:Stopped 2024-04-25T10:30:04.753 DEBUG:teuthology.orchestra.run.smithi037:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mds -f --cluster ceph -i c 2024-04-25T10:30:04.754 INFO:tasks.ceph.mds.b:Stopped 2024-04-25T10:30:04.754 DEBUG:teuthology.orchestra.run.smithi203:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mds -f --cluster ceph -i b 2024-04-25T10:30:04.756 INFO:tasks.ceph.mds.d:Stopped 2024-04-25T10:30:04.756 DEBUG:teuthology.orchestra.run.smithi203:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mds -f --cluster ceph -i d 2024-04-25T10:30:04.795 INFO:tasks.ceph.mds.a:Started 2024-04-25T10:30:04.795 DEBUG:teuthology.parallel:result is None 2024-04-25T10:30:04.796 INFO:tasks.ceph.mds.c:Started 2024-04-25T10:30:04.796 DEBUG:teuthology.parallel:result is None 2024-04-25T10:30:04.798 INFO:tasks.ceph.mds.b:Started 2024-04-25T10:30:04.799 DEBUG:teuthology.parallel:result is None 2024-04-25T10:30:04.800 INFO:tasks.ceph.mds.d:Started 2024-04-25T10:30:04.800 DEBUG:teuthology.parallel:result is None 2024-04-25T10:30:04.800 DEBUG:teuthology.orchestra.run.smithi037:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd blocklist clear 2024-04-25T10:30:04.890 INFO:tasks.ceph.mds.a.smithi037.stdout:starting mds.a at 2024-04-25T10:30:04.891 INFO:tasks.ceph.mds.c.smithi037.stdout:starting mds.c at 2024-04-25T10:30:04.891 INFO:tasks.ceph.mds.d.smithi203.stdout:starting mds.d at 2024-04-25T10:30:04.891 INFO:tasks.ceph.mds.b.smithi203.stdout:starting mds.b at 2024-04-25T10:30:04.925 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:04.922+0000 7feca8b2d640 1 Processor -- start 2024-04-25T10:30:04.925 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:04.922+0000 7feca8b2d640 1 -- start start 2024-04-25T10:30:04.926 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:04.924+0000 7feca8b2d640 1 --2- >> v2:172.21.15.203:3300/0 conn(0x7feca40b3c80 0x7feca40b4090 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:30:04.926 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:04.924+0000 7feca8b2d640 1 --2- >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7feca40d1ff0 0x7feca40d2400 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:30:04.926 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:04.924+0000 7feca8b2d640 1 -- --> v1:172.21.15.203:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7feca40b8d10 con 0x7feca41625b0 2024-04-25T10:30:04.926 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:04.924+0000 7feca8b2d640 1 -- --> v2:172.21.15.203:3300/0 -- mon_getmap magic: 0 -- 0x7feca408ceb0 con 0x7feca40b3c80 2024-04-25T10:30:04.926 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:04.924+0000 7feca8b2d640 1 -- --> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] -- mon_getmap magic: 0 -- 0x7feca40b9300 con 0x7feca40d1ff0 2024-04-25T10:30:04.927 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:04.925+0000 7feca259c640 1 --2- >> v2:172.21.15.203:3300/0 conn(0x7feca40b3c80 0x7feca40b4090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:30:04.927 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:04.925+0000 7feca1d9b640 1 --2- >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7feca40d1ff0 0x7feca40d2400 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:30:04.927 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:04.925+0000 7feca1d9b640 1 --2- >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7feca40d1ff0 0x7feca40d2400 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.203:3301/0 says I am v2:172.21.15.37:44606/0 (socket says 172.21.15.37:44606) 2024-04-25T10:30:04.927 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:04.925+0000 7feca1d9b640 1 -- 172.21.15.37:0/2150389483 learned_addr learned my addr 172.21.15.37:0/2150389483 (peer_addr_for_me v2:172.21.15.37:0/0) 2024-04-25T10:30:04.927 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:04.925+0000 7feca259c640 1 --2- >> v2:172.21.15.203:3300/0 conn(0x7feca40b3c80 0x7feca40b4090 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.203:3300/0 says I am v2:172.21.15.37:41206/0 (socket says 172.21.15.37:41206) 2024-04-25T10:30:04.928 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:04.926+0000 7feca259c640 1 -- 172.21.15.37:0/2150389483 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7feca40d1ff0 msgr2=0x7feca40d2400 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:30:04.928 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:04.926+0000 7feca259c640 1 --2- 172.21.15.37:0/2150389483 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7feca40d1ff0 0x7feca40d2400 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:30:04.928 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:04.926+0000 7feca259c640 1 -- 172.21.15.37:0/2150389483 >> v1:172.21.15.203:6789/0 conn(0x7feca41625b0 legacy=0x7feca41629c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:30:04.928 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:04.926+0000 7feca259c640 1 -- 172.21.15.37:0/2150389483 --> v2:172.21.15.203:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7feca40b8a30 con 0x7feca40b3c80 2024-04-25T10:30:04.929 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:04.927+0000 7feca259c640 1 --2- 172.21.15.37:0/2150389483 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7feca40b3c80 0x7feca40b4090 secure :-1 s=READY pgs=170 cs=0 l=1 rev1=1 crypto rx=0x7fec940097a0 tx=0x7fec9402a000 comp rx=0 tx=0).ready entity=mon.1 client_cookie=f4e6577ca95dccd4 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:30:04.929 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:04.927+0000 7feca159a640 1 -- 172.21.15.37:0/2150389483 <== mon.1 v2:172.21.15.203:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fec94037070 con 0x7feca40b3c80 2024-04-25T10:30:04.929 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:04.927+0000 7feca159a640 1 -- 172.21.15.37:0/2150389483 <== mon.1 v2:172.21.15.203:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fec9402ac90 con 0x7feca40b3c80 2024-04-25T10:30:04.929 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:04.927+0000 7feca159a640 1 -- 172.21.15.37:0/2150389483 <== mon.1 v2:172.21.15.203:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fec94032660 con 0x7feca40b3c80 2024-04-25T10:30:04.930 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:04.928+0000 7feca8b2d640 1 -- 172.21.15.37:0/2150389483 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7feca40b3c80 msgr2=0x7feca40b4090 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:30:04.930 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:04.928+0000 7feca8b2d640 1 --2- 172.21.15.37:0/2150389483 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7feca40b3c80 0x7feca40b4090 secure :-1 s=READY pgs=170 cs=0 l=1 rev1=1 crypto rx=0x7fec940097a0 tx=0x7fec9402a000 comp rx=0 tx=0).stop 2024-04-25T10:30:04.930 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:04.928+0000 7feca8b2d640 1 -- 172.21.15.37:0/2150389483 shutdown_connections 2024-04-25T10:30:04.930 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:04.928+0000 7feca8b2d640 1 --2- 172.21.15.37:0/2150389483 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7feca40d1ff0 0x7feca40d2400 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:30:04.930 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:04.928+0000 7feca8b2d640 1 --2- 172.21.15.37:0/2150389483 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7feca40b3c80 0x7feca40b4090 unknown :-1 s=CLOSED pgs=170 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:30:04.931 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:04.928+0000 7feca8b2d640 1 -- 172.21.15.37:0/2150389483 >> 172.21.15.37:0/2150389483 conn(0x7feca40d0390 msgr2=0x7feca40d07a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:30:04.931 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:04.928+0000 7feca8b2d640 1 -- 172.21.15.37:0/2150389483 shutdown_connections 2024-04-25T10:30:04.931 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:04.928+0000 7feca8b2d640 1 -- 172.21.15.37:0/2150389483 wait complete. 2024-04-25T10:30:04.932 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:04.930+0000 7feca8b2d640 1 Processor -- start 2024-04-25T10:30:04.932 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:04.930+0000 7feca8b2d640 1 -- start start 2024-04-25T10:30:04.935 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:04.931+0000 7feca8b2d640 1 --2- >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7feca40b3c80 0x7feca4151e00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:30:04.935 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:04.931+0000 7feca8b2d640 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7feca40d1ff0 0x7feca415af90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:30:04.935 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:04.931+0000 7feca8b2d640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7feca41625b0 0x7feca415b4d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:30:04.935 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:04.931+0000 7feca8b2d640 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 -- 0x7feca40b45d0 con 0x7feca41625b0 2024-04-25T10:30:04.935 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:04.931+0000 7feca8b2d640 1 -- --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_getmap magic: 0 -- 0x7feca409ce90 con 0x7feca40d1ff0 2024-04-25T10:30:04.935 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:04.931+0000 7feca8b2d640 1 -- --> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] -- mon_getmap magic: 0 -- 0x7feca40d2940 con 0x7feca40b3c80 2024-04-25T10:30:04.935 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:04.931+0000 7feca2d9d640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7feca41625b0 0x7feca415b4d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:30:04.935 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:04.931+0000 7feca2d9d640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7feca41625b0 0x7feca415b4d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.37:3300/0 says I am v2:172.21.15.37:54626/0 (socket says 172.21.15.37:54626) 2024-04-25T10:30:04.935 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:04.931+0000 7feca2d9d640 1 -- 172.21.15.37:0/4037720154 learned_addr learned my addr 172.21.15.37:0/4037720154 (peer_addr_for_me v2:172.21.15.37:0/0) 2024-04-25T10:30:04.935 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:04.931+0000 7feca259c640 1 --2- 172.21.15.37:0/4037720154 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7feca40b3c80 0x7feca4151e00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:30:04.935 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:04.932+0000 7feca2d9d640 1 -- 172.21.15.37:0/4037720154 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7feca40b3c80 msgr2=0x7feca4151e00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:30:04.936 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:04.932+0000 7feca2d9d640 1 --2- 172.21.15.37:0/4037720154 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7feca40b3c80 0x7feca4151e00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:30:04.936 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:04.932+0000 7feca2d9d640 1 -- 172.21.15.37:0/4037720154 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7feca40d1ff0 msgr2=0x7feca415af90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:30:04.936 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:04.932+0000 7feca2d9d640 1 --2- 172.21.15.37:0/4037720154 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7feca40d1ff0 0x7feca415af90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:30:04.936 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:04.932+0000 7feca2d9d640 1 -- 172.21.15.37:0/4037720154 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7feca40b0f10 con 0x7feca41625b0 2024-04-25T10:30:04.936 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:04.932+0000 7feca2d9d640 1 --2- 172.21.15.37:0/4037720154 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7feca41625b0 0x7feca415b4d0 secure :-1 s=READY pgs=389 cs=0 l=1 rev1=1 crypto rx=0x7fec9c00c850 tx=0x7fec9c00cd10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:30:04.936 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:04.932+0000 7fec92ffd640 1 -- 172.21.15.37:0/4037720154 <== mon.0 v2:172.21.15.37:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fec9c017070 con 0x7feca41625b0 2024-04-25T10:30:04.936 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:04.932+0000 7fec92ffd640 1 -- 172.21.15.37:0/4037720154 <== mon.0 v2:172.21.15.37:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fec9c003da0 con 0x7feca41625b0 2024-04-25T10:30:04.936 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:04.932+0000 7fec92ffd640 1 -- 172.21.15.37:0/4037720154 <== mon.0 v2:172.21.15.37:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fec9c005290 con 0x7feca41625b0 2024-04-25T10:30:04.936 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:04.932+0000 7feca8b2d640 1 -- 172.21.15.37:0/4037720154 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7feca40b11f0 con 0x7feca41625b0 2024-04-25T10:30:04.936 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:04.932+0000 7feca8b2d640 1 -- 172.21.15.37:0/4037720154 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7feca40b17b0 con 0x7feca41625b0 2024-04-25T10:30:04.937 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:04.933+0000 7feca8b2d640 1 -- 172.21.15.37:0/4037720154 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7feca406a140 con 0x7feca41625b0 2024-04-25T10:30:04.937 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:04.934+0000 7fec92ffd640 1 -- 172.21.15.37:0/4037720154 <== mon.0 v2:172.21.15.37:3300/0 4 ==== mgrmap(e 5) ==== 96782+0+0 (secure 0 0 0) 0x7fec9c01a070 con 0x7feca41625b0 2024-04-25T10:30:04.937 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:04.934+0000 7fec92ffd640 1 --2- 172.21.15.37:0/4037720154 >> [v2:172.21.15.203:6832/2418205237,v1:172.21.15.203:6833/2418205237] conn(0x7fec840744b0 0x7fec840769c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:30:04.937 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:04.934+0000 7fec92ffd640 1 -- 172.21.15.37:0/4037720154 <== mon.0 v2:172.21.15.37:3300/0 5 ==== osd_map(28..28 src has 1..28) ==== 4943+0+0 (secure 0 0 0) 0x7fec9c094630 con 0x7feca41625b0 2024-04-25T10:30:04.937 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:04.935+0000 7feca259c640 1 --2- 172.21.15.37:0/4037720154 >> [v2:172.21.15.203:6832/2418205237,v1:172.21.15.203:6833/2418205237] conn(0x7fec840744b0 0x7fec840769c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:30:04.938 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:04.936+0000 7feca259c640 1 --2- 172.21.15.37:0/4037720154 >> [v2:172.21.15.203:6832/2418205237,v1:172.21.15.203:6833/2418205237] conn(0x7fec840744b0 0x7fec840769c0 secure :-1 s=READY pgs=194 cs=0 l=1 rev1=1 crypto rx=0x7fec94034000 tx=0x7fec9402e000 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:30:04.941 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:04.939+0000 7fec92ffd640 1 -- 172.21.15.37:0/4037720154 <== mon.0 v2:172.21.15.37:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187246 (secure 0 0 0) 0x7fec9c05ea60 con 0x7feca41625b0 2024-04-25T10:30:05.144 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:05.142+0000 7feca8b2d640 1 -- 172.21.15.37:0/4037720154 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_command({"prefix": "osd blocklist clear"} v 0) -- 0x7feca415a320 con 0x7feca41625b0 2024-04-25T10:30:05.727 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:05.725+0000 7fec92ffd640 1 -- 172.21.15.37:0/4037720154 <== mon.0 v2:172.21.15.37:3300/0 7 ==== mon_command_ack([{"prefix": "osd blocklist clear"}]=0 removed all blocklist entries v30) ==== 97+0+0 (secure 0 0 0) 0x7fec9c0670a0 con 0x7feca41625b0 2024-04-25T10:30:05.727 INFO:teuthology.orchestra.run.smithi037.stderr: removed all blocklist entries 2024-04-25T10:30:05.730 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:05.727+0000 7feca8b2d640 1 -- 172.21.15.37:0/4037720154 >> [v2:172.21.15.203:6832/2418205237,v1:172.21.15.203:6833/2418205237] conn(0x7fec840744b0 msgr2=0x7fec840769c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:30:05.730 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:05.727+0000 7feca8b2d640 1 --2- 172.21.15.37:0/4037720154 >> [v2:172.21.15.203:6832/2418205237,v1:172.21.15.203:6833/2418205237] conn(0x7fec840744b0 0x7fec840769c0 secure :-1 s=READY pgs=194 cs=0 l=1 rev1=1 crypto rx=0x7fec94034000 tx=0x7fec9402e000 comp rx=0 tx=0).stop 2024-04-25T10:30:05.730 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:05.727+0000 7feca8b2d640 1 -- 172.21.15.37:0/4037720154 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7feca41625b0 msgr2=0x7feca415b4d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:30:05.730 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:05.727+0000 7feca8b2d640 1 --2- 172.21.15.37:0/4037720154 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7feca41625b0 0x7feca415b4d0 secure :-1 s=READY pgs=389 cs=0 l=1 rev1=1 crypto rx=0x7fec9c00c850 tx=0x7fec9c00cd10 comp rx=0 tx=0).stop 2024-04-25T10:30:05.730 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:05.727+0000 7feca8b2d640 1 -- 172.21.15.37:0/4037720154 shutdown_connections 2024-04-25T10:30:05.730 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:05.727+0000 7feca8b2d640 1 --2- 172.21.15.37:0/4037720154 >> [v2:172.21.15.203:6832/2418205237,v1:172.21.15.203:6833/2418205237] conn(0x7fec840744b0 0x7fec840769c0 unknown :-1 s=CLOSED pgs=194 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:30:05.730 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:05.727+0000 7feca8b2d640 1 --2- 172.21.15.37:0/4037720154 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7feca41625b0 0x7feca415b4d0 unknown :-1 s=CLOSED pgs=389 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:30:05.730 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:05.727+0000 7feca8b2d640 1 --2- 172.21.15.37:0/4037720154 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7feca40d1ff0 0x7feca415af90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:30:05.730 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:05.727+0000 7feca8b2d640 1 --2- 172.21.15.37:0/4037720154 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7feca40b3c80 0x7feca4151e00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:30:05.730 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:05.727+0000 7feca8b2d640 1 -- 172.21.15.37:0/4037720154 >> 172.21.15.37:0/4037720154 conn(0x7feca40d0390 msgr2=0x7feca40712f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:30:05.731 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:05.727+0000 7feca8b2d640 1 -- 172.21.15.37:0/4037720154 shutdown_connections 2024-04-25T10:30:05.731 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:05.727+0000 7feca8b2d640 1 -- 172.21.15.37:0/4037720154 wait complete. 2024-04-25T10:30:05.742 DEBUG:teuthology.orchestra.run.smithi037:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph auth ls --format=json-pretty 2024-04-25T10:30:05.854 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:05.852+0000 7f6d272f9640 1 Processor -- start 2024-04-25T10:30:05.854 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:05.852+0000 7f6d272f9640 1 -- start start 2024-04-25T10:30:05.856 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:05.853+0000 7f6d272f9640 1 --2- >> v2:172.21.15.37:3300/0 conn(0x7f6d200b2dc0 0x7f6d200d2860 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:30:05.856 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:05.853+0000 7f6d272f9640 1 --2- >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f6d2006e610 0x7f6d200ce560 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:30:05.856 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:05.853+0000 7f6d272f9640 1 -- --> v1:172.21.15.203:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f6d200d2e40 con 0x7f6d2006daf0 2024-04-25T10:30:05.856 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:05.853+0000 7f6d272f9640 1 -- --> v2:172.21.15.37:3300/0 -- mon_getmap magic: 0 -- 0x7f6d200b92c0 con 0x7f6d200b2dc0 2024-04-25T10:30:05.856 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:05.853+0000 7f6d272f9640 1 -- --> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] -- mon_getmap magic: 0 -- 0x7f6d200ce0d0 con 0x7f6d2006e610 2024-04-25T10:30:05.856 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:05.853+0000 7f6d25095640 1 --2- >> v2:172.21.15.37:3300/0 conn(0x7f6d200b2dc0 0x7f6d200d2860 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:30:05.856 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:05.854+0000 7f6d25095640 1 --2- >> v2:172.21.15.37:3300/0 conn(0x7f6d200b2dc0 0x7f6d200d2860 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.37:3300/0 says I am v2:172.21.15.37:54640/0 (socket says 172.21.15.37:54640) 2024-04-25T10:30:05.856 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:05.854+0000 7f6d25095640 1 -- 172.21.15.37:0/3216660523 learned_addr learned my addr 172.21.15.37:0/3216660523 (peer_addr_for_me v2:172.21.15.37:0/0) 2024-04-25T10:30:05.857 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:05.854+0000 7f6d25095640 1 -- 172.21.15.37:0/3216660523 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f6d2006e610 msgr2=0x7f6d200ce560 unknown :-1 s=STATE_CONNECTING_RE l=0).mark_down 2024-04-25T10:30:05.857 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:05.854+0000 7f6d25095640 1 --2- 172.21.15.37:0/3216660523 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f6d2006e610 0x7f6d200ce560 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:30:05.857 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:05.854+0000 7f6d25095640 1 -- 172.21.15.37:0/3216660523 >> v1:172.21.15.203:6789/0 conn(0x7f6d2006daf0 legacy=0x7f6d2006df00 unknown :-1 s=STATE_CONNECTING_RE l=0).mark_down 2024-04-25T10:30:05.857 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:05.854+0000 7f6d25095640 1 -- 172.21.15.37:0/3216660523 --> v2:172.21.15.37:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f6d200d3120 con 0x7f6d200b2dc0 2024-04-25T10:30:05.858 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:05.854+0000 7f6d25095640 1 --2- 172.21.15.37:0/3216660523 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f6d200b2dc0 0x7f6d200d2860 secure :-1 s=READY pgs=390 cs=0 l=1 rev1=1 crypto rx=0x7f6d1c009850 tx=0x7f6d1c02a210 comp rx=0 tx=0).ready entity=mon.0 client_cookie=965a864ac8e59873 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:30:05.858 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:05.855+0000 7f6d17fff640 1 -- 172.21.15.37:0/3216660523 <== mon.0 v2:172.21.15.37:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6d1c004030 con 0x7f6d200b2dc0 2024-04-25T10:30:05.858 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:05.855+0000 7f6d17fff640 1 -- 172.21.15.37:0/3216660523 <== mon.0 v2:172.21.15.37:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f6d1c0041d0 con 0x7f6d200b2dc0 2024-04-25T10:30:05.858 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:05.855+0000 7f6d17fff640 1 -- 172.21.15.37:0/3216660523 <== mon.0 v2:172.21.15.37:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6d1c003610 con 0x7f6d200b2dc0 2024-04-25T10:30:05.858 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:05.855+0000 7f6d272f9640 1 -- 172.21.15.37:0/3216660523 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f6d200b2dc0 msgr2=0x7f6d200d2860 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:30:05.858 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:05.855+0000 7f6d272f9640 1 --2- 172.21.15.37:0/3216660523 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f6d200b2dc0 0x7f6d200d2860 secure :-1 s=READY pgs=390 cs=0 l=1 rev1=1 crypto rx=0x7f6d1c009850 tx=0x7f6d1c02a210 comp rx=0 tx=0).stop 2024-04-25T10:30:05.858 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:05.855+0000 7f6d272f9640 1 -- 172.21.15.37:0/3216660523 shutdown_connections 2024-04-25T10:30:05.859 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:05.855+0000 7f6d272f9640 1 --2- 172.21.15.37:0/3216660523 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f6d2006e610 0x7f6d200ce560 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:30:05.859 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:05.855+0000 7f6d272f9640 1 --2- 172.21.15.37:0/3216660523 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f6d200b2dc0 0x7f6d200d2860 unknown :-1 s=CLOSED pgs=390 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:30:05.859 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:05.855+0000 7f6d272f9640 1 -- 172.21.15.37:0/3216660523 >> 172.21.15.37:0/3216660523 conn(0x7f6d200b5440 msgr2=0x7f6d200b5850 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:30:05.859 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:05.855+0000 7f6d272f9640 1 -- 172.21.15.37:0/3216660523 shutdown_connections 2024-04-25T10:30:05.859 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:05.855+0000 7f6d272f9640 1 -- 172.21.15.37:0/3216660523 wait complete. 2024-04-25T10:30:05.859 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:05.857+0000 7f6d272f9640 1 Processor -- start 2024-04-25T10:30:05.859 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:05.857+0000 7f6d272f9640 1 -- start start 2024-04-25T10:30:05.862 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:05.858+0000 7f6d272f9640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f6d2006daf0 0x7f6d201607e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:30:05.862 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:05.858+0000 7f6d272f9640 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f6d2006e610 0x7f6d20153d40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:30:05.863 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:05.858+0000 7f6d272f9640 1 --2- >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f6d200b2dc0 0x7f6d200c96d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:30:05.863 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:05.858+0000 7f6d272f9640 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 -- 0x7f6d200ceaa0 con 0x7f6d2006daf0 2024-04-25T10:30:05.863 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:05.858+0000 7f6d272f9640 1 -- --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_getmap magic: 0 -- 0x7f6d200d36e0 con 0x7f6d2006e610 2024-04-25T10:30:05.863 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:05.858+0000 7f6d272f9640 1 -- --> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] -- mon_getmap magic: 0 -- 0x7f6d200cec20 con 0x7f6d200b2dc0 2024-04-25T10:30:05.863 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:05.859+0000 7f6d25095640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f6d2006daf0 0x7f6d201607e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:30:05.863 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:05.859+0000 7f6d25095640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f6d2006daf0 0x7f6d201607e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.37:3300/0 says I am v2:172.21.15.37:54648/0 (socket says 172.21.15.37:54648) 2024-04-25T10:30:05.863 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:05.859+0000 7f6d25095640 1 -- 172.21.15.37:0/1158308136 learned_addr learned my addr 172.21.15.37:0/1158308136 (peer_addr_for_me v2:172.21.15.37:0/0) 2024-04-25T10:30:05.863 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:05.859+0000 7f6d25896640 1 --2- 172.21.15.37:0/1158308136 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f6d200b2dc0 0x7f6d200c96d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:30:05.863 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:05.859+0000 7f6d24894640 1 --2- 172.21.15.37:0/1158308136 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f6d2006e610 0x7f6d20153d40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:30:05.863 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:05.859+0000 7f6d25095640 1 -- 172.21.15.37:0/1158308136 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f6d200b2dc0 msgr2=0x7f6d200c96d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:30:05.863 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:05.859+0000 7f6d25095640 1 --2- 172.21.15.37:0/1158308136 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f6d200b2dc0 0x7f6d200c96d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:30:05.863 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:05.859+0000 7f6d25095640 1 -- 172.21.15.37:0/1158308136 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f6d2006e610 msgr2=0x7f6d20153d40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:30:05.863 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:05.859+0000 7f6d25095640 1 --2- 172.21.15.37:0/1158308136 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f6d2006e610 0x7f6d20153d40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:30:05.864 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:05.859+0000 7f6d25095640 1 -- 172.21.15.37:0/1158308136 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f6d201610e0 con 0x7f6d2006daf0 2024-04-25T10:30:05.864 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:05.859+0000 7f6d24894640 1 --2- 172.21.15.37:0/1158308136 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f6d2006e610 0x7f6d20153d40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T10:30:05.864 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:05.859+0000 7f6d25095640 1 --2- 172.21.15.37:0/1158308136 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f6d2006daf0 0x7f6d201607e0 secure :-1 s=READY pgs=391 cs=0 l=1 rev1=1 crypto rx=0x7f6d1c000c00 tx=0x7f6d1c0046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:30:05.864 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:05.859+0000 7f6d15ffb640 1 -- 172.21.15.37:0/1158308136 <== mon.0 v2:172.21.15.37:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6d1c041020 con 0x7f6d2006daf0 2024-04-25T10:30:05.864 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:05.859+0000 7f6d15ffb640 1 -- 172.21.15.37:0/1158308136 <== mon.0 v2:172.21.15.37:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f6d1c003c50 con 0x7f6d2006daf0 2024-04-25T10:30:05.864 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:05.859+0000 7f6d272f9640 1 -- 172.21.15.37:0/1158308136 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f6d201613c0 con 0x7f6d2006daf0 2024-04-25T10:30:05.864 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:05.859+0000 7f6d15ffb640 1 -- 172.21.15.37:0/1158308136 <== mon.0 v2:172.21.15.37:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6d1c004030 con 0x7f6d2006daf0 2024-04-25T10:30:05.864 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:05.860+0000 7f6d272f9640 1 -- 172.21.15.37:0/1158308136 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f6d20161980 con 0x7f6d2006daf0 2024-04-25T10:30:05.864 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:05.860+0000 7f6d272f9640 1 -- 172.21.15.37:0/1158308136 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f6d2006a140 con 0x7f6d2006daf0 2024-04-25T10:30:05.865 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:05.861+0000 7f6d15ffb640 1 -- 172.21.15.37:0/1158308136 <== mon.0 v2:172.21.15.37:3300/0 4 ==== mgrmap(e 5) ==== 96782+0+0 (secure 0 0 0) 0x7f6d1c044020 con 0x7f6d2006daf0 2024-04-25T10:30:05.865 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:05.861+0000 7f6d15ffb640 1 --2- 172.21.15.37:0/1158308136 >> [v2:172.21.15.203:6832/2418205237,v1:172.21.15.203:6833/2418205237] conn(0x7f6d040744b0 0x7f6d040769c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:30:05.865 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:05.862+0000 7f6d15ffb640 1 -- 172.21.15.37:0/1158308136 <== mon.0 v2:172.21.15.37:3300/0 5 ==== osd_map(32..32 src has 1..32) ==== 4857+0+0 (secure 0 0 0) 0x7f6d1c0b5c60 con 0x7f6d2006daf0 2024-04-25T10:30:05.865 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:05.862+0000 7f6d24894640 1 --2- 172.21.15.37:0/1158308136 >> [v2:172.21.15.203:6832/2418205237,v1:172.21.15.203:6833/2418205237] conn(0x7f6d040744b0 0x7f6d040769c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:30:05.865 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:05.863+0000 7f6d24894640 1 --2- 172.21.15.37:0/1158308136 >> [v2:172.21.15.203:6832/2418205237,v1:172.21.15.203:6833/2418205237] conn(0x7f6d040744b0 0x7f6d040769c0 secure :-1 s=READY pgs=196 cs=0 l=1 rev1=1 crypto rx=0x7f6d200b8850 tx=0x7f6d10008040 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:30:05.868 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:05.866+0000 7f6d15ffb640 1 -- 172.21.15.37:0/1158308136 <== mon.0 v2:172.21.15.37:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187246 (secure 0 0 0) 0x7f6d1c04b030 con 0x7f6d2006daf0 2024-04-25T10:30:06.126 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:06.124+0000 7f6d272f9640 1 -- 172.21.15.37:0/1158308136 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_command({"prefix": "auth ls", "format": "json-pretty"} v 0) -- 0x7f6d20162190 con 0x7f6d2006daf0 2024-04-25T10:30:06.127 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:06.125+0000 7f6d15ffb640 1 -- 172.21.15.37:0/1158308136 <== mon.0 v2:172.21.15.37:3300/0 7 ==== mon_command_ack([{"prefix": "auth ls", "format": "json-pretty"}]=0 v2) ==== 80+0+6645 (secure 0 0 0) 0x7f6d1c080740 con 0x7f6d2006daf0 2024-04-25T10:30:06.127 INFO:teuthology.orchestra.run.smithi037.stdout: 2024-04-25T10:30:06.127 INFO:teuthology.orchestra.run.smithi037.stdout:{ 2024-04-25T10:30:06.127 INFO:teuthology.orchestra.run.smithi037.stdout: "auth_dump": [ 2024-04-25T10:30:06.128 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-04-25T10:30:06.128 INFO:teuthology.orchestra.run.smithi037.stdout: "entity": "mds.a", 2024-04-25T10:30:06.128 INFO:teuthology.orchestra.run.smithi037.stdout: "key": "AQA6MCpmOqPOAxAAA6+9BBhumKh0ZpFfmoGY9g==", 2024-04-25T10:30:06.128 INFO:teuthology.orchestra.run.smithi037.stdout: "caps": { 2024-04-25T10:30:06.128 INFO:teuthology.orchestra.run.smithi037.stdout: "mds": "allow", 2024-04-25T10:30:06.128 INFO:teuthology.orchestra.run.smithi037.stdout: "mgr": "allow *", 2024-04-25T10:30:06.128 INFO:teuthology.orchestra.run.smithi037.stdout: "mon": "allow *", 2024-04-25T10:30:06.128 INFO:teuthology.orchestra.run.smithi037.stdout: "osd": "allow *" 2024-04-25T10:30:06.128 INFO:teuthology.orchestra.run.smithi037.stdout: } 2024-04-25T10:30:06.128 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-04-25T10:30:06.128 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-04-25T10:30:06.128 INFO:teuthology.orchestra.run.smithi037.stdout: "entity": "mds.b", 2024-04-25T10:30:06.128 INFO:teuthology.orchestra.run.smithi037.stdout: "key": "AQA6MCpm8nFTEBAAHcOblByoU8A86+xc++RP4g==", 2024-04-25T10:30:06.129 INFO:teuthology.orchestra.run.smithi037.stdout: "caps": { 2024-04-25T10:30:06.129 INFO:teuthology.orchestra.run.smithi037.stdout: "mds": "allow", 2024-04-25T10:30:06.129 INFO:teuthology.orchestra.run.smithi037.stdout: "mgr": "allow *", 2024-04-25T10:30:06.129 INFO:teuthology.orchestra.run.smithi037.stdout: "mon": "allow *", 2024-04-25T10:30:06.129 INFO:teuthology.orchestra.run.smithi037.stdout: "osd": "allow *" 2024-04-25T10:30:06.129 INFO:teuthology.orchestra.run.smithi037.stdout: } 2024-04-25T10:30:06.129 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-04-25T10:30:06.129 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-04-25T10:30:06.129 INFO:teuthology.orchestra.run.smithi037.stdout: "entity": "mds.c", 2024-04-25T10:30:06.129 INFO:teuthology.orchestra.run.smithi037.stdout: "key": "AQA6MCpmxhNNCxAA96FDgnsZN+3HKcD7mcXOKQ==", 2024-04-25T10:30:06.129 INFO:teuthology.orchestra.run.smithi037.stdout: "caps": { 2024-04-25T10:30:06.129 INFO:teuthology.orchestra.run.smithi037.stdout: "mds": "allow", 2024-04-25T10:30:06.129 INFO:teuthology.orchestra.run.smithi037.stdout: "mgr": "allow *", 2024-04-25T10:30:06.129 INFO:teuthology.orchestra.run.smithi037.stdout: "mon": "allow *", 2024-04-25T10:30:06.129 INFO:teuthology.orchestra.run.smithi037.stdout: "osd": "allow *" 2024-04-25T10:30:06.130 INFO:teuthology.orchestra.run.smithi037.stdout: } 2024-04-25T10:30:06.130 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-04-25T10:30:06.130 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-04-25T10:30:06.130 INFO:teuthology.orchestra.run.smithi037.stdout: "entity": "mds.d", 2024-04-25T10:30:06.130 INFO:teuthology.orchestra.run.smithi037.stdout: "key": "AQA6MCpmkcehFxAAwy8W6aXfuVc30DJQg3ubTQ==", 2024-04-25T10:30:06.130 INFO:teuthology.orchestra.run.smithi037.stdout: "caps": { 2024-04-25T10:30:06.130 INFO:teuthology.orchestra.run.smithi037.stdout: "mds": "allow", 2024-04-25T10:30:06.130 INFO:teuthology.orchestra.run.smithi037.stdout: "mgr": "allow *", 2024-04-25T10:30:06.130 INFO:teuthology.orchestra.run.smithi037.stdout: "mon": "allow *", 2024-04-25T10:30:06.130 INFO:teuthology.orchestra.run.smithi037.stdout: "osd": "allow *" 2024-04-25T10:30:06.130 INFO:teuthology.orchestra.run.smithi037.stdout: } 2024-04-25T10:30:06.130 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-04-25T10:30:06.130 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-04-25T10:30:06.131 INFO:teuthology.orchestra.run.smithi037.stdout: "entity": "osd.0", 2024-04-25T10:30:06.131 INFO:teuthology.orchestra.run.smithi037.stdout: "key": "AQA9MCpmY2W2NhAA9bGxmGeGHUKB9mkN+aVQeQ==", 2024-04-25T10:30:06.131 INFO:teuthology.orchestra.run.smithi037.stdout: "caps": { 2024-04-25T10:30:06.131 INFO:teuthology.orchestra.run.smithi037.stdout: "mgr": "allow profile osd", 2024-04-25T10:30:06.131 INFO:teuthology.orchestra.run.smithi037.stdout: "mon": "allow profile osd", 2024-04-25T10:30:06.131 INFO:teuthology.orchestra.run.smithi037.stdout: "osd": "allow *" 2024-04-25T10:30:06.131 INFO:teuthology.orchestra.run.smithi037.stdout: } 2024-04-25T10:30:06.131 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-04-25T10:30:06.131 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-04-25T10:30:06.131 INFO:teuthology.orchestra.run.smithi037.stdout: "entity": "osd.1", 2024-04-25T10:30:06.131 INFO:teuthology.orchestra.run.smithi037.stdout: "key": "AQBAMCpmVaZMNhAA0f+Rbv1taIO6L4ghaHc0mw==", 2024-04-25T10:30:06.131 INFO:teuthology.orchestra.run.smithi037.stdout: "caps": { 2024-04-25T10:30:06.131 INFO:teuthology.orchestra.run.smithi037.stdout: "mgr": "allow profile osd", 2024-04-25T10:30:06.131 INFO:teuthology.orchestra.run.smithi037.stdout: "mon": "allow profile osd", 2024-04-25T10:30:06.131 INFO:teuthology.orchestra.run.smithi037.stdout: "osd": "allow *" 2024-04-25T10:30:06.132 INFO:teuthology.orchestra.run.smithi037.stdout: } 2024-04-25T10:30:06.132 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-04-25T10:30:06.132 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-04-25T10:30:06.132 INFO:teuthology.orchestra.run.smithi037.stdout: "entity": "osd.2", 2024-04-25T10:30:06.132 INFO:teuthology.orchestra.run.smithi037.stdout: "key": "AQBDMCpmjj4FNhAAwOpaIswpIllQpGT1shypQw==", 2024-04-25T10:30:06.132 INFO:teuthology.orchestra.run.smithi037.stdout: "caps": { 2024-04-25T10:30:06.132 INFO:teuthology.orchestra.run.smithi037.stdout: "mgr": "allow profile osd", 2024-04-25T10:30:06.132 INFO:teuthology.orchestra.run.smithi037.stdout: "mon": "allow profile osd", 2024-04-25T10:30:06.132 INFO:teuthology.orchestra.run.smithi037.stdout: "osd": "allow *" 2024-04-25T10:30:06.132 INFO:teuthology.orchestra.run.smithi037.stdout: } 2024-04-25T10:30:06.132 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-04-25T10:30:06.132 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-04-25T10:30:06.132 INFO:teuthology.orchestra.run.smithi037.stdout: "entity": "osd.3", 2024-04-25T10:30:06.132 INFO:teuthology.orchestra.run.smithi037.stdout: "key": "AQBGMCpmIFb7NRAA3/p5NiVot6OJ/4zCIgfgBQ==", 2024-04-25T10:30:06.133 INFO:teuthology.orchestra.run.smithi037.stdout: "caps": { 2024-04-25T10:30:06.133 INFO:teuthology.orchestra.run.smithi037.stdout: "mgr": "allow profile osd", 2024-04-25T10:30:06.133 INFO:teuthology.orchestra.run.smithi037.stdout: "mon": "allow profile osd", 2024-04-25T10:30:06.133 INFO:teuthology.orchestra.run.smithi037.stdout: "osd": "allow *" 2024-04-25T10:30:06.133 INFO:teuthology.orchestra.run.smithi037.stdout: } 2024-04-25T10:30:06.133 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-04-25T10:30:06.133 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-04-25T10:30:06.133 INFO:teuthology.orchestra.run.smithi037.stdout: "entity": "osd.4", 2024-04-25T10:30:06.133 INFO:teuthology.orchestra.run.smithi037.stdout: "key": "AQBNMCpm93QfAxAAEWVT2WCL9FZGei9lOGVklQ==", 2024-04-25T10:30:06.133 INFO:teuthology.orchestra.run.smithi037.stdout: "caps": { 2024-04-25T10:30:06.133 INFO:teuthology.orchestra.run.smithi037.stdout: "mgr": "allow profile osd", 2024-04-25T10:30:06.133 INFO:teuthology.orchestra.run.smithi037.stdout: "mon": "allow profile osd", 2024-04-25T10:30:06.133 INFO:teuthology.orchestra.run.smithi037.stdout: "osd": "allow *" 2024-04-25T10:30:06.133 INFO:teuthology.orchestra.run.smithi037.stdout: } 2024-04-25T10:30:06.134 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-04-25T10:30:06.134 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-04-25T10:30:06.134 INFO:teuthology.orchestra.run.smithi037.stdout: "entity": "osd.5", 2024-04-25T10:30:06.134 INFO:teuthology.orchestra.run.smithi037.stdout: "key": "AQBQMCpm1ivWARAAQFZjKw9Fma0rQJB7W/9mdQ==", 2024-04-25T10:30:06.134 INFO:teuthology.orchestra.run.smithi037.stdout: "caps": { 2024-04-25T10:30:06.134 INFO:teuthology.orchestra.run.smithi037.stdout: "mgr": "allow profile osd", 2024-04-25T10:30:06.134 INFO:teuthology.orchestra.run.smithi037.stdout: "mon": "allow profile osd", 2024-04-25T10:30:06.134 INFO:teuthology.orchestra.run.smithi037.stdout: "osd": "allow *" 2024-04-25T10:30:06.134 INFO:teuthology.orchestra.run.smithi037.stdout: } 2024-04-25T10:30:06.134 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-04-25T10:30:06.134 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-04-25T10:30:06.134 INFO:teuthology.orchestra.run.smithi037.stdout: "entity": "osd.6", 2024-04-25T10:30:06.134 INFO:teuthology.orchestra.run.smithi037.stdout: "key": "AQBTMCpm5h3YABAAvQ+ACgMvLhelhESFoNkdbQ==", 2024-04-25T10:30:06.134 INFO:teuthology.orchestra.run.smithi037.stdout: "caps": { 2024-04-25T10:30:06.135 INFO:teuthology.orchestra.run.smithi037.stdout: "mgr": "allow profile osd", 2024-04-25T10:30:06.135 INFO:teuthology.orchestra.run.smithi037.stdout: "mon": "allow profile osd", 2024-04-25T10:30:06.135 INFO:teuthology.orchestra.run.smithi037.stdout: "osd": "allow *" 2024-04-25T10:30:06.135 INFO:teuthology.orchestra.run.smithi037.stdout: } 2024-04-25T10:30:06.135 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-04-25T10:30:06.135 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-04-25T10:30:06.135 INFO:teuthology.orchestra.run.smithi037.stdout: "entity": "osd.7", 2024-04-25T10:30:06.135 INFO:teuthology.orchestra.run.smithi037.stdout: "key": "AQBVMCpmTjVfOxAAj6Zg4/eeQqNxiVlZS0by4w==", 2024-04-25T10:30:06.135 INFO:teuthology.orchestra.run.smithi037.stdout: "caps": { 2024-04-25T10:30:06.135 INFO:teuthology.orchestra.run.smithi037.stdout: "mgr": "allow profile osd", 2024-04-25T10:30:06.135 INFO:teuthology.orchestra.run.smithi037.stdout: "mon": "allow profile osd", 2024-04-25T10:30:06.135 INFO:teuthology.orchestra.run.smithi037.stdout: "osd": "allow *" 2024-04-25T10:30:06.135 INFO:teuthology.orchestra.run.smithi037.stdout: } 2024-04-25T10:30:06.135 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-04-25T10:30:06.135 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-04-25T10:30:06.136 INFO:teuthology.orchestra.run.smithi037.stdout: "entity": "client.0", 2024-04-25T10:30:06.136 INFO:teuthology.orchestra.run.smithi037.stdout: "key": "AQA6MCpmQ308HBAAvMknoENSGUD2vblm1SMnBw==", 2024-04-25T10:30:06.136 INFO:teuthology.orchestra.run.smithi037.stdout: "caps": { 2024-04-25T10:30:06.136 INFO:teuthology.orchestra.run.smithi037.stdout: "mds": "allow", 2024-04-25T10:30:06.136 INFO:teuthology.orchestra.run.smithi037.stdout: "mgr": "allow r", 2024-04-25T10:30:06.136 INFO:teuthology.orchestra.run.smithi037.stdout: "mon": "allow rw", 2024-04-25T10:30:06.136 INFO:teuthology.orchestra.run.smithi037.stdout: "osd": "allow rwx" 2024-04-25T10:30:06.136 INFO:teuthology.orchestra.run.smithi037.stdout: } 2024-04-25T10:30:06.136 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-04-25T10:30:06.136 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-04-25T10:30:06.136 INFO:teuthology.orchestra.run.smithi037.stdout: "entity": "client.1", 2024-04-25T10:30:06.136 INFO:teuthology.orchestra.run.smithi037.stdout: "key": "AQA6MCpmpPyRIxAAic9ZcD+WF6YRCzQu9u0jCg==", 2024-04-25T10:30:06.136 INFO:teuthology.orchestra.run.smithi037.stdout: "caps": { 2024-04-25T10:30:06.136 INFO:teuthology.orchestra.run.smithi037.stdout: "mds": "allow", 2024-04-25T10:30:06.137 INFO:teuthology.orchestra.run.smithi037.stdout: "mgr": "allow r", 2024-04-25T10:30:06.137 INFO:teuthology.orchestra.run.smithi037.stdout: "mon": "allow rw", 2024-04-25T10:30:06.137 INFO:teuthology.orchestra.run.smithi037.stdout: "osd": "allow rwx" 2024-04-25T10:30:06.137 INFO:teuthology.orchestra.run.smithi037.stdout: } 2024-04-25T10:30:06.137 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-04-25T10:30:06.137 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-04-25T10:30:06.137 INFO:teuthology.orchestra.run.smithi037.stdout: "entity": "client.2", 2024-04-25T10:30:06.137 INFO:teuthology.orchestra.run.smithi037.stdout: "key": "AQA6MCpmJuXpHxAAvIdzKkZOqcGEhbcmTQ15FQ==", 2024-04-25T10:30:06.137 INFO:teuthology.orchestra.run.smithi037.stdout: "caps": { 2024-04-25T10:30:06.137 INFO:teuthology.orchestra.run.smithi037.stdout: "mds": "allow", 2024-04-25T10:30:06.137 INFO:teuthology.orchestra.run.smithi037.stdout: "mgr": "allow r", 2024-04-25T10:30:06.137 INFO:teuthology.orchestra.run.smithi037.stdout: "mon": "allow rw", 2024-04-25T10:30:06.137 INFO:teuthology.orchestra.run.smithi037.stdout: "osd": "allow rwx" 2024-04-25T10:30:06.137 INFO:teuthology.orchestra.run.smithi037.stdout: } 2024-04-25T10:30:06.137 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-04-25T10:30:06.138 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-04-25T10:30:06.138 INFO:teuthology.orchestra.run.smithi037.stdout: "entity": "client.3", 2024-04-25T10:30:06.138 INFO:teuthology.orchestra.run.smithi037.stdout: "key": "AQA6MCpmlk0vJxAAi5on23wOapMW7UdNXB71Lw==", 2024-04-25T10:30:06.138 INFO:teuthology.orchestra.run.smithi037.stdout: "caps": { 2024-04-25T10:30:06.138 INFO:teuthology.orchestra.run.smithi037.stdout: "mds": "allow", 2024-04-25T10:30:06.138 INFO:teuthology.orchestra.run.smithi037.stdout: "mgr": "allow r", 2024-04-25T10:30:06.138 INFO:teuthology.orchestra.run.smithi037.stdout: "mon": "allow rw", 2024-04-25T10:30:06.138 INFO:teuthology.orchestra.run.smithi037.stdout: "osd": "allow rwx" 2024-04-25T10:30:06.138 INFO:teuthology.orchestra.run.smithi037.stdout: } 2024-04-25T10:30:06.138 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-04-25T10:30:06.138 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-04-25T10:30:06.138 INFO:teuthology.orchestra.run.smithi037.stdout: "entity": "client.admin", 2024-04-25T10:30:06.138 INFO:teuthology.orchestra.run.smithi037.stdout: "key": "AQA5MCpmocDaIRAAOjmbZ7WkBxdiJIKyLCPwHA==", 2024-04-25T10:30:06.138 INFO:teuthology.orchestra.run.smithi037.stdout: "caps": { 2024-04-25T10:30:06.139 INFO:teuthology.orchestra.run.smithi037.stdout: "mds": "allow *", 2024-04-25T10:30:06.139 INFO:teuthology.orchestra.run.smithi037.stdout: "mgr": "allow *", 2024-04-25T10:30:06.139 INFO:teuthology.orchestra.run.smithi037.stdout: "mon": "allow *", 2024-04-25T10:30:06.139 INFO:teuthology.orchestra.run.smithi037.stdout: "osd": "allow *" 2024-04-25T10:30:06.139 INFO:teuthology.orchestra.run.smithi037.stdout: } 2024-04-25T10:30:06.139 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-04-25T10:30:06.139 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-04-25T10:30:06.139 INFO:teuthology.orchestra.run.smithi037.stdout: "entity": "client.bootstrap-mds", 2024-04-25T10:30:06.139 INFO:teuthology.orchestra.run.smithi037.stdout: "key": "AQBbMCpmvdXLOhAA2f+CGZgmPgPdwvBVUW/Ptw==", 2024-04-25T10:30:06.139 INFO:teuthology.orchestra.run.smithi037.stdout: "caps": { 2024-04-25T10:30:06.139 INFO:teuthology.orchestra.run.smithi037.stdout: "mon": "allow profile bootstrap-mds" 2024-04-25T10:30:06.139 INFO:teuthology.orchestra.run.smithi037.stdout: } 2024-04-25T10:30:06.139 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-04-25T10:30:06.139 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-04-25T10:30:06.140 INFO:teuthology.orchestra.run.smithi037.stdout: "entity": "client.bootstrap-mgr", 2024-04-25T10:30:06.140 INFO:teuthology.orchestra.run.smithi037.stdout: "key": "AQBbMCpmm97LOhAAIX/3gv2Ucjwwvkq4bjPHtw==", 2024-04-25T10:30:06.140 INFO:teuthology.orchestra.run.smithi037.stdout: "caps": { 2024-04-25T10:30:06.140 INFO:teuthology.orchestra.run.smithi037.stdout: "mon": "allow profile bootstrap-mgr" 2024-04-25T10:30:06.140 INFO:teuthology.orchestra.run.smithi037.stdout: } 2024-04-25T10:30:06.140 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-04-25T10:30:06.140 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-04-25T10:30:06.140 INFO:teuthology.orchestra.run.smithi037.stdout: "entity": "client.bootstrap-osd", 2024-04-25T10:30:06.140 INFO:teuthology.orchestra.run.smithi037.stdout: "key": "AQBbMCpmE+bLOhAAT7ZuSZJIMI/LtMDzBF6B1A==", 2024-04-25T10:30:06.140 INFO:teuthology.orchestra.run.smithi037.stdout: "caps": { 2024-04-25T10:30:06.140 INFO:teuthology.orchestra.run.smithi037.stdout: "mon": "allow profile bootstrap-osd" 2024-04-25T10:30:06.140 INFO:teuthology.orchestra.run.smithi037.stdout: } 2024-04-25T10:30:06.140 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-04-25T10:30:06.140 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-04-25T10:30:06.141 INFO:teuthology.orchestra.run.smithi037.stdout: "entity": "client.bootstrap-rbd", 2024-04-25T10:30:06.141 INFO:teuthology.orchestra.run.smithi037.stdout: "key": "AQBbMCpm6u7LOhAASC5LFNwk1wJElwNW1qkTVw==", 2024-04-25T10:30:06.141 INFO:teuthology.orchestra.run.smithi037.stdout: "caps": { 2024-04-25T10:30:06.141 INFO:teuthology.orchestra.run.smithi037.stdout: "mon": "allow profile bootstrap-rbd" 2024-04-25T10:30:06.141 INFO:teuthology.orchestra.run.smithi037.stdout: } 2024-04-25T10:30:06.141 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-04-25T10:30:06.141 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-04-25T10:30:06.141 INFO:teuthology.orchestra.run.smithi037.stdout: "entity": "client.bootstrap-rbd-mirror", 2024-04-25T10:30:06.141 INFO:teuthology.orchestra.run.smithi037.stdout: "key": "AQBbMCpmd/bLOhAATQcBHFGiZTV/S7IaAO7jpw==", 2024-04-25T10:30:06.141 INFO:teuthology.orchestra.run.smithi037.stdout: "caps": { 2024-04-25T10:30:06.141 INFO:teuthology.orchestra.run.smithi037.stdout: "mon": "allow profile bootstrap-rbd-mirror" 2024-04-25T10:30:06.141 INFO:teuthology.orchestra.run.smithi037.stdout: } 2024-04-25T10:30:06.141 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-04-25T10:30:06.141 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-04-25T10:30:06.142 INFO:teuthology.orchestra.run.smithi037.stdout: "entity": "client.bootstrap-rgw", 2024-04-25T10:30:06.142 INFO:teuthology.orchestra.run.smithi037.stdout: "key": "AQBbMCpmZv7LOhAAJlzWA4D3hXEt6w71D4nELA==", 2024-04-25T10:30:06.142 INFO:teuthology.orchestra.run.smithi037.stdout: "caps": { 2024-04-25T10:30:06.142 INFO:teuthology.orchestra.run.smithi037.stdout: "mon": "allow profile bootstrap-rgw" 2024-04-25T10:30:06.142 INFO:teuthology.orchestra.run.smithi037.stdout: } 2024-04-25T10:30:06.142 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-04-25T10:30:06.142 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-04-25T10:30:06.142 INFO:teuthology.orchestra.run.smithi037.stdout: "entity": "mgr.x", 2024-04-25T10:30:06.142 INFO:teuthology.orchestra.run.smithi037.stdout: "key": "AQA6MCpmyf8OABAAiqPbP7X6j/lHlk9pzU6ABQ==", 2024-04-25T10:30:06.142 INFO:teuthology.orchestra.run.smithi037.stdout: "caps": { 2024-04-25T10:30:06.142 INFO:teuthology.orchestra.run.smithi037.stdout: "mds": "allow *", 2024-04-25T10:30:06.142 INFO:teuthology.orchestra.run.smithi037.stdout: "mon": "allow profile mgr", 2024-04-25T10:30:06.142 INFO:teuthology.orchestra.run.smithi037.stdout: "osd": "allow *" 2024-04-25T10:30:06.143 INFO:teuthology.orchestra.run.smithi037.stdout: } 2024-04-25T10:30:06.143 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-04-25T10:30:06.143 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-04-25T10:30:06.143 INFO:teuthology.orchestra.run.smithi037.stdout: "entity": "mgr.y", 2024-04-25T10:30:06.143 INFO:teuthology.orchestra.run.smithi037.stdout: "key": "AQA5MCpmaXHDNhAA4ipqVxc6vQM915k93tOJIQ==", 2024-04-25T10:30:06.143 INFO:teuthology.orchestra.run.smithi037.stdout: "caps": { 2024-04-25T10:30:06.143 INFO:teuthology.orchestra.run.smithi037.stdout: "mds": "allow *", 2024-04-25T10:30:06.143 INFO:teuthology.orchestra.run.smithi037.stdout: "mon": "allow profile mgr", 2024-04-25T10:30:06.143 INFO:teuthology.orchestra.run.smithi037.stdout: "osd": "allow *" 2024-04-25T10:30:06.143 INFO:teuthology.orchestra.run.smithi037.stdout: } 2024-04-25T10:30:06.143 INFO:teuthology.orchestra.run.smithi037.stdout: } 2024-04-25T10:30:06.143 INFO:teuthology.orchestra.run.smithi037.stdout: ] 2024-04-25T10:30:06.143 INFO:teuthology.orchestra.run.smithi037.stdout:} 2024-04-25T10:30:06.145 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:06.127+0000 7f6d272f9640 1 -- 172.21.15.37:0/1158308136 >> [v2:172.21.15.203:6832/2418205237,v1:172.21.15.203:6833/2418205237] conn(0x7f6d040744b0 msgr2=0x7f6d040769c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:30:06.145 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:06.127+0000 7f6d272f9640 1 --2- 172.21.15.37:0/1158308136 >> [v2:172.21.15.203:6832/2418205237,v1:172.21.15.203:6833/2418205237] conn(0x7f6d040744b0 0x7f6d040769c0 secure :-1 s=READY pgs=196 cs=0 l=1 rev1=1 crypto rx=0x7f6d200b8850 tx=0x7f6d10008040 comp rx=0 tx=0).stop 2024-04-25T10:30:06.145 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:06.127+0000 7f6d272f9640 1 -- 172.21.15.37:0/1158308136 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f6d2006daf0 msgr2=0x7f6d201607e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:30:06.145 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:06.127+0000 7f6d272f9640 1 --2- 172.21.15.37:0/1158308136 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f6d2006daf0 0x7f6d201607e0 secure :-1 s=READY pgs=391 cs=0 l=1 rev1=1 crypto rx=0x7f6d1c000c00 tx=0x7f6d1c0046c0 comp rx=0 tx=0).stop 2024-04-25T10:30:06.145 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:06.127+0000 7f6d272f9640 1 -- 172.21.15.37:0/1158308136 shutdown_connections 2024-04-25T10:30:06.145 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:06.127+0000 7f6d272f9640 1 --2- 172.21.15.37:0/1158308136 >> [v2:172.21.15.203:6832/2418205237,v1:172.21.15.203:6833/2418205237] conn(0x7f6d040744b0 0x7f6d040769c0 unknown :-1 s=CLOSED pgs=196 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:30:06.145 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:06.127+0000 7f6d272f9640 1 --2- 172.21.15.37:0/1158308136 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f6d200b2dc0 0x7f6d200c96d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:30:06.146 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:06.127+0000 7f6d272f9640 1 --2- 172.21.15.37:0/1158308136 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f6d2006e610 0x7f6d20153d40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:30:06.146 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:06.127+0000 7f6d272f9640 1 --2- 172.21.15.37:0/1158308136 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f6d2006daf0 0x7f6d201607e0 unknown :-1 s=CLOSED pgs=391 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:30:06.146 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:06.127+0000 7f6d272f9640 1 -- 172.21.15.37:0/1158308136 >> 172.21.15.37:0/1158308136 conn(0x7f6d200b5440 msgr2=0x7f6d2006df00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:30:06.146 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:06.127+0000 7f6d272f9640 1 -- 172.21.15.37:0/1158308136 shutdown_connections 2024-04-25T10:30:06.146 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:06.127+0000 7f6d272f9640 1 -- 172.21.15.37:0/1158308136 wait complete. 2024-04-25T10:30:06.146 DEBUG:teuthology.orchestra.run.smithi037:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph auth del client.bootstrap-mds 2024-04-25T10:30:06.258 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:06.256+0000 7fc902c9f640 1 Processor -- start 2024-04-25T10:30:06.258 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:06.256+0000 7fc902c9f640 1 -- start start 2024-04-25T10:30:06.260 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:06.257+0000 7fc902c9f640 1 --2- >> v2:172.21.15.37:3300/0 conn(0x7fc8fc0cf570 0x7fc8fc16ad90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:30:06.260 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:06.257+0000 7fc902c9f640 1 -- --> v1:172.21.15.37:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fc8fc06e910 con 0x7fc8fc0d2860 2024-04-25T10:30:06.260 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:06.257+0000 7fc902c9f640 1 -- --> v1:172.21.15.203:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fc8fc0d2e40 con 0x7fc8fc0b2dc0 2024-04-25T10:30:06.260 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:06.257+0000 7fc902c9f640 1 -- --> v2:172.21.15.37:3300/0 -- mon_getmap magic: 0 -- 0x7fc8fc0b92c0 con 0x7fc8fc0cf570 2024-04-25T10:30:06.260 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:06.257+0000 7fc90123c640 1 --2- >> v2:172.21.15.37:3300/0 conn(0x7fc8fc0cf570 0x7fc8fc16ad90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:30:06.260 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:06.257+0000 7fc90123c640 1 --2- >> v2:172.21.15.37:3300/0 conn(0x7fc8fc0cf570 0x7fc8fc16ad90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.37:3300/0 says I am v2:172.21.15.37:54660/0 (socket says 172.21.15.37:54660) 2024-04-25T10:30:06.260 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:06.258+0000 7fc90123c640 1 -- 172.21.15.37:0/87999591 learned_addr learned my addr 172.21.15.37:0/87999591 (peer_addr_for_me v2:172.21.15.37:0/0) 2024-04-25T10:30:06.261 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:06.257+0000 7fc8fbfff640 1 --1- 172.21.15.37:0/87999591 >> v1:172.21.15.37:6789/0 conn(0x7fc8fc0d2860 0x7fc8fc06e200 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.37:6789/0 says I am v1:172.21.15.37:60720/0 (socket says 172.21.15.37:60720) 2024-04-25T10:30:06.261 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:06.258+0000 7fc90123c640 1 -- 172.21.15.37:0/87999591 >> v1:172.21.15.203:6789/0 conn(0x7fc8fc0b2dc0 legacy=0x7fc8fc06daf0 unknown :-1 s=STATE_CONNECTING_RE l=0).mark_down 2024-04-25T10:30:06.261 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:06.258+0000 7fc90123c640 1 -- 172.21.15.37:0/87999591 >> v1:172.21.15.37:6789/0 conn(0x7fc8fc0d2860 legacy=0x7fc8fc06e200 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:30:06.261 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:06.258+0000 7fc8fb7fe640 1 -- 172.21.15.37:0/87999591 <== mon.0 v1:172.21.15.37:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 841283960 0 0) 0x7fc8fc06e910 con 0x7fc8fc0d2860 2024-04-25T10:30:06.261 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:06.258+0000 7fc90123c640 1 -- 172.21.15.37:0/87999591 --> v2:172.21.15.37:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fc8fc0d3120 con 0x7fc8fc0cf570 2024-04-25T10:30:06.262 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:06.259+0000 7fc90123c640 1 --2- 172.21.15.37:0/87999591 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fc8fc0cf570 0x7fc8fc16ad90 secure :-1 s=READY pgs=393 cs=0 l=1 rev1=1 crypto rx=0x7fc8f4009e50 tx=0x7fc8f402a760 comp rx=0 tx=0).ready entity=mon.0 client_cookie=9f4178d309c3968a server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:30:06.262 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:06.259+0000 7fc8fb7fe640 1 -- 172.21.15.37:0/87999591 <== mon.0 v2:172.21.15.37:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc8f4036070 con 0x7fc8fc0cf570 2024-04-25T10:30:06.262 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:06.259+0000 7fc8fb7fe640 1 -- 172.21.15.37:0/87999591 <== mon.0 v2:172.21.15.37:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fc8f4031600 con 0x7fc8fc0cf570 2024-04-25T10:30:06.262 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:06.259+0000 7fc8fb7fe640 1 -- 172.21.15.37:0/87999591 <== mon.0 v2:172.21.15.37:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc8f40319d0 con 0x7fc8fc0cf570 2024-04-25T10:30:06.262 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:06.259+0000 7fc902c9f640 1 -- 172.21.15.37:0/87999591 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fc8fc0cf570 msgr2=0x7fc8fc16ad90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:30:06.263 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:06.259+0000 7fc902c9f640 1 --2- 172.21.15.37:0/87999591 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fc8fc0cf570 0x7fc8fc16ad90 secure :-1 s=READY pgs=393 cs=0 l=1 rev1=1 crypto rx=0x7fc8f4009e50 tx=0x7fc8f402a760 comp rx=0 tx=0).stop 2024-04-25T10:30:06.263 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:06.259+0000 7fc902c9f640 1 -- 172.21.15.37:0/87999591 shutdown_connections 2024-04-25T10:30:06.263 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:06.259+0000 7fc902c9f640 1 --2- 172.21.15.37:0/87999591 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fc8fc0cf570 0x7fc8fc16ad90 unknown :-1 s=CLOSED pgs=393 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:30:06.263 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:06.259+0000 7fc902c9f640 1 -- 172.21.15.37:0/87999591 >> 172.21.15.37:0/87999591 conn(0x7fc8fc0b5440 msgr2=0x7fc8fc0b5850 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:30:06.263 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:06.259+0000 7fc902c9f640 1 -- 172.21.15.37:0/87999591 shutdown_connections 2024-04-25T10:30:06.263 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:06.259+0000 7fc902c9f640 1 -- 172.21.15.37:0/87999591 wait complete. 2024-04-25T10:30:06.263 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:06.261+0000 7fc902c9f640 1 Processor -- start 2024-04-25T10:30:06.263 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:06.261+0000 7fc902c9f640 1 -- start start 2024-04-25T10:30:06.266 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:06.263+0000 7fc902c9f640 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fc8fc0b2dc0 0x7fc8fc0ccf30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:30:06.266 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:06.263+0000 7fc902c9f640 1 --2- >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7fc8fc0cf570 0x7fc8fc15b980 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:30:06.266 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:06.263+0000 7fc902c9f640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fc8fc0d2860 0x7fc8fc14f4c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:30:06.266 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:06.263+0000 7fc902c9f640 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 -- 0x7fc8fc0d36e0 con 0x7fc8fc0d2860 2024-04-25T10:30:06.267 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:06.263+0000 7fc902c9f640 1 -- --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_getmap magic: 0 -- 0x7fc8fc0ce0d0 con 0x7fc8fc0b2dc0 2024-04-25T10:30:06.267 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:06.263+0000 7fc902c9f640 1 -- --> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] -- mon_getmap magic: 0 -- 0x7fc8fc16b2d0 con 0x7fc8fc0cf570 2024-04-25T10:30:06.267 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:06.263+0000 7fc90123c640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fc8fc0d2860 0x7fc8fc14f4c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:30:06.267 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:06.263+0000 7fc90123c640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fc8fc0d2860 0x7fc8fc14f4c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.37:3300/0 says I am v2:172.21.15.37:54676/0 (socket says 172.21.15.37:54676) 2024-04-25T10:30:06.267 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:06.263+0000 7fc90123c640 1 -- 172.21.15.37:0/3194847850 learned_addr learned my addr 172.21.15.37:0/3194847850 (peer_addr_for_me v2:172.21.15.37:0/0) 2024-04-25T10:30:06.267 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:06.263+0000 7fc8fbfff640 1 --2- 172.21.15.37:0/3194847850 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7fc8fc0cf570 0x7fc8fc15b980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:30:06.267 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:06.263+0000 7fc90123c640 1 -- 172.21.15.37:0/3194847850 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7fc8fc0cf570 msgr2=0x7fc8fc15b980 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:30:06.267 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:06.263+0000 7fc90123c640 1 --2- 172.21.15.37:0/3194847850 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7fc8fc0cf570 0x7fc8fc15b980 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:30:06.267 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:06.263+0000 7fc90123c640 1 -- 172.21.15.37:0/3194847850 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fc8fc0b2dc0 msgr2=0x7fc8fc0ccf30 unknown :-1 s=STATE_CONNECTING_RE l=1).mark_down 2024-04-25T10:30:06.267 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:06.263+0000 7fc90123c640 1 --2- 172.21.15.37:0/3194847850 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fc8fc0b2dc0 0x7fc8fc0ccf30 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:30:06.267 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:06.263+0000 7fc90123c640 1 -- 172.21.15.37:0/3194847850 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fc8fc15a8e0 con 0x7fc8fc0d2860 2024-04-25T10:30:06.267 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:06.263+0000 7fc8fbfff640 1 --2- 172.21.15.37:0/3194847850 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7fc8fc0cf570 0x7fc8fc15b980 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T10:30:06.267 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:06.263+0000 7fc90123c640 1 --2- 172.21.15.37:0/3194847850 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fc8fc0d2860 0x7fc8fc14f4c0 secure :-1 s=READY pgs=394 cs=0 l=1 rev1=1 crypto rx=0x7fc8f4009e50 tx=0x7fc8f4004a40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:30:06.268 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:06.263+0000 7fc8f97fa640 1 -- 172.21.15.37:0/3194847850 <== mon.0 v2:172.21.15.37:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc8f4041020 con 0x7fc8fc0d2860 2024-04-25T10:30:06.268 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:06.263+0000 7fc8f97fa640 1 -- 172.21.15.37:0/3194847850 <== mon.0 v2:172.21.15.37:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fc8f4007780 con 0x7fc8fc0d2860 2024-04-25T10:30:06.268 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:06.264+0000 7fc902c9f640 1 -- 172.21.15.37:0/3194847850 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fc8fc15abc0 con 0x7fc8fc0d2860 2024-04-25T10:30:06.268 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:06.264+0000 7fc8f97fa640 1 -- 172.21.15.37:0/3194847850 <== mon.0 v2:172.21.15.37:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc8f4036040 con 0x7fc8fc0d2860 2024-04-25T10:30:06.268 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:06.264+0000 7fc902c9f640 1 -- 172.21.15.37:0/3194847850 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fc8fc15b180 con 0x7fc8fc0d2860 2024-04-25T10:30:06.268 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:06.264+0000 7fc902c9f640 1 -- 172.21.15.37:0/3194847850 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fc8fc0cd680 con 0x7fc8fc0d2860 2024-04-25T10:30:06.269 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:06.265+0000 7fc8f97fa640 1 -- 172.21.15.37:0/3194847850 <== mon.0 v2:172.21.15.37:3300/0 4 ==== mgrmap(e 5) ==== 96782+0+0 (secure 0 0 0) 0x7fc8f4044020 con 0x7fc8fc0d2860 2024-04-25T10:30:06.269 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:06.266+0000 7fc8f97fa640 1 --2- 172.21.15.37:0/3194847850 >> [v2:172.21.15.203:6832/2418205237,v1:172.21.15.203:6833/2418205237] conn(0x7fc8e00744e0 0x7fc8e00769f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:30:06.269 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:06.266+0000 7fc8f97fa640 1 -- 172.21.15.37:0/3194847850 <== mon.0 v2:172.21.15.37:3300/0 5 ==== osd_map(32..32 src has 1..32) ==== 4857+0+0 (secure 0 0 0) 0x7fc8f40b5b00 con 0x7fc8fc0d2860 2024-04-25T10:30:06.269 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:06.266+0000 7fc900a3b640 1 --2- 172.21.15.37:0/3194847850 >> [v2:172.21.15.203:6832/2418205237,v1:172.21.15.203:6833/2418205237] conn(0x7fc8e00744e0 0x7fc8e00769f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:30:06.269 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:06.267+0000 7fc900a3b640 1 --2- 172.21.15.37:0/3194847850 >> [v2:172.21.15.203:6832/2418205237,v1:172.21.15.203:6833/2418205237] conn(0x7fc8e00744e0 0x7fc8e00769f0 secure :-1 s=READY pgs=197 cs=0 l=1 rev1=1 crypto rx=0x7fc8fc074cc0 tx=0x7fc8f0006b20 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:30:06.272 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:06.270+0000 7fc8f97fa640 1 -- 172.21.15.37:0/3194847850 <== mon.0 v2:172.21.15.37:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187246 (secure 0 0 0) 0x7fc8f404b030 con 0x7fc8fc0d2860 2024-04-25T10:30:06.529 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:06.527+0000 7fc902c9f640 1 -- 172.21.15.37:0/3194847850 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_command({"prefix": "auth del", "entity": "client.bootstrap-mds"} v 0) -- 0x7fc8fc06a140 con 0x7fc8fc0d2860 2024-04-25T10:30:06.531 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:06.529+0000 7fc8f97fa640 1 -- 172.21.15.37:0/3194847850 <== mon.0 v2:172.21.15.37:3300/0 7 ==== mon_command_ack([{"prefix": "auth del", "entity": "client.bootstrap-mds"}]=0 v3) ==== 90+0+0 (secure 0 0 0) 0x7fc8f40807c0 con 0x7fc8fc0d2860 2024-04-25T10:30:06.534 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:06.531+0000 7fc902c9f640 1 -- 172.21.15.37:0/3194847850 >> [v2:172.21.15.203:6832/2418205237,v1:172.21.15.203:6833/2418205237] conn(0x7fc8e00744e0 msgr2=0x7fc8e00769f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:30:06.534 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:06.531+0000 7fc902c9f640 1 --2- 172.21.15.37:0/3194847850 >> [v2:172.21.15.203:6832/2418205237,v1:172.21.15.203:6833/2418205237] conn(0x7fc8e00744e0 0x7fc8e00769f0 secure :-1 s=READY pgs=197 cs=0 l=1 rev1=1 crypto rx=0x7fc8fc074cc0 tx=0x7fc8f0006b20 comp rx=0 tx=0).stop 2024-04-25T10:30:06.534 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:06.531+0000 7fc902c9f640 1 -- 172.21.15.37:0/3194847850 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fc8fc0d2860 msgr2=0x7fc8fc14f4c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:30:06.534 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:06.531+0000 7fc902c9f640 1 --2- 172.21.15.37:0/3194847850 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fc8fc0d2860 0x7fc8fc14f4c0 secure :-1 s=READY pgs=394 cs=0 l=1 rev1=1 crypto rx=0x7fc8f4009e50 tx=0x7fc8f4004a40 comp rx=0 tx=0).stop 2024-04-25T10:30:06.534 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:06.531+0000 7fc902c9f640 1 -- 172.21.15.37:0/3194847850 shutdown_connections 2024-04-25T10:30:06.534 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:06.531+0000 7fc902c9f640 1 --2- 172.21.15.37:0/3194847850 >> [v2:172.21.15.203:6832/2418205237,v1:172.21.15.203:6833/2418205237] conn(0x7fc8e00744e0 0x7fc8e00769f0 unknown :-1 s=CLOSED pgs=197 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:30:06.534 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:06.531+0000 7fc902c9f640 1 --2- 172.21.15.37:0/3194847850 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fc8fc0d2860 0x7fc8fc14f4c0 unknown :-1 s=CLOSED pgs=394 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:30:06.534 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:06.531+0000 7fc902c9f640 1 --2- 172.21.15.37:0/3194847850 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7fc8fc0cf570 0x7fc8fc15b980 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:30:06.535 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:06.531+0000 7fc902c9f640 1 --2- 172.21.15.37:0/3194847850 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fc8fc0b2dc0 0x7fc8fc0ccf30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:30:06.535 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:06.531+0000 7fc902c9f640 1 -- 172.21.15.37:0/3194847850 >> 172.21.15.37:0/3194847850 conn(0x7fc8fc0b5440 msgr2=0x7fc8fc0d0b60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:30:06.535 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:06.532+0000 7fc902c9f640 1 -- 172.21.15.37:0/3194847850 shutdown_connections 2024-04-25T10:30:06.535 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:06.532+0000 7fc902c9f640 1 -- 172.21.15.37:0/3194847850 wait complete. 2024-04-25T10:30:06.545 DEBUG:teuthology.orchestra.run.smithi037:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph auth del client.bootstrap-mgr 2024-04-25T10:30:06.657 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:06.655+0000 7f33b61e3640 1 Processor -- start 2024-04-25T10:30:06.657 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:06.655+0000 7f33b61e3640 1 -- start start 2024-04-25T10:30:06.659 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:06.656+0000 7f33b61e3640 1 --2- >> v2:172.21.15.37:3300/0 conn(0x7f33b00b9070 0x7f33b00d2370 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:30:06.659 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:06.656+0000 7f33b61e3640 1 --2- >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f33b00d0330 0x7f33b00d0740 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:30:06.659 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:06.656+0000 7f33b61e3640 1 -- --> v1:172.21.15.203:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f33b00d0c80 con 0x7f33b00b4fd0 2024-04-25T10:30:06.659 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:06.656+0000 7f33b61e3640 1 -- --> v2:172.21.15.37:3300/0 -- mon_getmap magic: 0 -- 0x7f33b00b9480 con 0x7f33b00b9070 2024-04-25T10:30:06.659 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:06.656+0000 7f33b61e3640 1 -- --> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] -- mon_getmap magic: 0 -- 0x7f33b00d47b0 con 0x7f33b00d0330 2024-04-25T10:30:06.659 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:06.656+0000 7f33aeffd640 1 --2- >> v2:172.21.15.37:3300/0 conn(0x7f33b00b9070 0x7f33b00d2370 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:30:06.659 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:06.656+0000 7f33aeffd640 1 --2- >> v2:172.21.15.37:3300/0 conn(0x7f33b00b9070 0x7f33b00d2370 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.37:3300/0 says I am v2:172.21.15.37:54692/0 (socket says 172.21.15.37:54692) 2024-04-25T10:30:06.659 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:06.656+0000 7f33aeffd640 1 -- 172.21.15.37:0/2620977124 learned_addr learned my addr 172.21.15.37:0/2620977124 (peer_addr_for_me v2:172.21.15.37:0/0) 2024-04-25T10:30:06.659 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:06.656+0000 7f33affff640 1 --2- 172.21.15.37:0/2620977124 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f33b00d0330 0x7f33b00d0740 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:30:06.660 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:06.657+0000 7f33aeffd640 1 -- 172.21.15.37:0/2620977124 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f33b00d0330 msgr2=0x7f33b00d0740 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:30:06.660 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:06.657+0000 7f33aeffd640 1 --2- 172.21.15.37:0/2620977124 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f33b00d0330 0x7f33b00d0740 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:30:06.660 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:06.657+0000 7f33aeffd640 1 -- 172.21.15.37:0/2620977124 >> v1:172.21.15.203:6789/0 conn(0x7f33b00b4fd0 legacy=0x7f33b00cf5d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:30:06.660 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:06.657+0000 7f33aeffd640 1 -- 172.21.15.37:0/2620977124 --> v2:172.21.15.37:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f33b00ce810 con 0x7f33b00b9070 2024-04-25T10:30:06.661 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:06.657+0000 7f33aeffd640 1 --2- 172.21.15.37:0/2620977124 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f33b00b9070 0x7f33b00d2370 secure :-1 s=READY pgs=395 cs=0 l=1 rev1=1 crypto rx=0x7f33a4009870 tx=0x7f33a402a1c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=6c1237ff0fed63cf server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:30:06.661 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:06.657+0000 7f33ae7fc640 1 -- 172.21.15.37:0/2620977124 <== mon.0 v2:172.21.15.37:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f33a4037070 con 0x7f33b00b9070 2024-04-25T10:30:06.661 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:06.658+0000 7f33ae7fc640 1 -- 172.21.15.37:0/2620977124 <== mon.0 v2:172.21.15.37:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f33a4032440 con 0x7f33b00b9070 2024-04-25T10:30:06.661 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:06.658+0000 7f33ae7fc640 1 -- 172.21.15.37:0/2620977124 <== mon.0 v2:172.21.15.37:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f33a4032810 con 0x7f33b00b9070 2024-04-25T10:30:06.661 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:06.658+0000 7f33b61e3640 1 -- 172.21.15.37:0/2620977124 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f33b00b9070 msgr2=0x7f33b00d2370 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:30:06.661 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:06.658+0000 7f33b61e3640 1 --2- 172.21.15.37:0/2620977124 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f33b00b9070 0x7f33b00d2370 secure :-1 s=READY pgs=395 cs=0 l=1 rev1=1 crypto rx=0x7f33a4009870 tx=0x7f33a402a1c0 comp rx=0 tx=0).stop 2024-04-25T10:30:06.661 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:06.658+0000 7f33b61e3640 1 -- 172.21.15.37:0/2620977124 shutdown_connections 2024-04-25T10:30:06.661 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:06.658+0000 7f33b61e3640 1 --2- 172.21.15.37:0/2620977124 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f33b00d0330 0x7f33b00d0740 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:30:06.662 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:06.658+0000 7f33b61e3640 1 --2- 172.21.15.37:0/2620977124 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f33b00b9070 0x7f33b00d2370 unknown :-1 s=CLOSED pgs=395 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:30:06.662 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:06.658+0000 7f33b61e3640 1 -- 172.21.15.37:0/2620977124 >> 172.21.15.37:0/2620977124 conn(0x7f33b00d39f0 msgr2=0x7f33b00d3e00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:30:06.662 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:06.658+0000 7f33b61e3640 1 -- 172.21.15.37:0/2620977124 shutdown_connections 2024-04-25T10:30:06.662 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:06.658+0000 7f33b61e3640 1 -- 172.21.15.37:0/2620977124 wait complete. 2024-04-25T10:30:06.662 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:06.660+0000 7f33b61e3640 1 Processor -- start 2024-04-25T10:30:06.662 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:06.660+0000 7f33b61e3640 1 -- start start 2024-04-25T10:30:06.665 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:06.661+0000 7f33b61e3640 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f33b00b4fd0 0x7f33b01542d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:30:06.665 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:06.661+0000 7f33b61e3640 1 --2- >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f33b00b9070 0x7f33b01f4810 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:30:06.665 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:06.661+0000 7f33b61e3640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f33b00d0330 0x7f33b01f4d50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:30:06.665 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:06.661+0000 7f33b61e3640 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 -- 0x7f33b00cf0b0 con 0x7f33b00d0330 2024-04-25T10:30:06.665 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:06.661+0000 7f33b61e3640 1 -- --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_getmap magic: 0 -- 0x7f33b00cedd0 con 0x7f33b00b4fd0 2024-04-25T10:30:06.665 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:06.661+0000 7f33b61e3640 1 -- --> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] -- mon_getmap magic: 0 -- 0x7f33b00d2a70 con 0x7f33b00b9070 2024-04-25T10:30:06.665 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:06.662+0000 7f33affff640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f33b00d0330 0x7f33b01f4d50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:30:06.665 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:06.662+0000 7f33affff640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f33b00d0330 0x7f33b01f4d50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.37:3300/0 says I am v2:172.21.15.37:54704/0 (socket says 172.21.15.37:54704) 2024-04-25T10:30:06.665 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:06.662+0000 7f33affff640 1 -- 172.21.15.37:0/1957971924 learned_addr learned my addr 172.21.15.37:0/1957971924 (peer_addr_for_me v2:172.21.15.37:0/0) 2024-04-25T10:30:06.665 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:06.662+0000 7f33affff640 1 -- 172.21.15.37:0/1957971924 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f33b00b9070 msgr2=0x7f33b01f4810 unknown :-1 s=STATE_CONNECTING_RE l=1).mark_down 2024-04-25T10:30:06.665 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:06.662+0000 7f33affff640 1 --2- 172.21.15.37:0/1957971924 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f33b00b9070 0x7f33b01f4810 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:30:06.666 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:06.662+0000 7f33affff640 1 -- 172.21.15.37:0/1957971924 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f33b00b4fd0 msgr2=0x7f33b01542d0 unknown :-1 s=STATE_CONNECTING_RE l=1).mark_down 2024-04-25T10:30:06.666 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:06.662+0000 7f33affff640 1 --2- 172.21.15.37:0/1957971924 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f33b00b4fd0 0x7f33b01542d0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:30:06.666 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:06.662+0000 7f33affff640 1 -- 172.21.15.37:0/1957971924 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f33b0156b80 con 0x7f33b00d0330 2024-04-25T10:30:06.666 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:06.662+0000 7f33affff640 1 --2- 172.21.15.37:0/1957971924 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f33b00d0330 0x7f33b01f4d50 secure :-1 s=READY pgs=396 cs=0 l=1 rev1=1 crypto rx=0x7f33a8009fd0 tx=0x7f33a800e4b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:30:06.666 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:06.662+0000 7f3397fff640 1 -- 172.21.15.37:0/1957971924 <== mon.0 v2:172.21.15.37:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f33a8014420 con 0x7f33b00d0330 2024-04-25T10:30:06.666 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:06.662+0000 7f3397fff640 1 -- 172.21.15.37:0/1957971924 <== mon.0 v2:172.21.15.37:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f33a8019070 con 0x7f33b00d0330 2024-04-25T10:30:06.666 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:06.662+0000 7f33b61e3640 1 -- 172.21.15.37:0/1957971924 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f33b01e8ef0 con 0x7f33b00d0330 2024-04-25T10:30:06.666 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:06.662+0000 7f3397fff640 1 -- 172.21.15.37:0/1957971924 <== mon.0 v2:172.21.15.37:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f33a8014800 con 0x7f33b00d0330 2024-04-25T10:30:06.666 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:06.662+0000 7f33b61e3640 1 -- 172.21.15.37:0/1957971924 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f33b01e94b0 con 0x7f33b00d0330 2024-04-25T10:30:06.667 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:06.664+0000 7f33b61e3640 1 -- 172.21.15.37:0/1957971924 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f33b006a140 con 0x7f33b00d0330 2024-04-25T10:30:06.667 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:06.664+0000 7f3397fff640 1 -- 172.21.15.37:0/1957971924 <== mon.0 v2:172.21.15.37:3300/0 4 ==== mgrmap(e 5) ==== 96782+0+0 (secure 0 0 0) 0x7f33a80149a0 con 0x7f33b00d0330 2024-04-25T10:30:06.667 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:06.665+0000 7f3397fff640 1 --2- 172.21.15.37:0/1957971924 >> [v2:172.21.15.203:6832/2418205237,v1:172.21.15.203:6833/2418205237] conn(0x7f339c0744b0 0x7f339c0769c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:30:06.667 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:06.665+0000 7f3397fff640 1 -- 172.21.15.37:0/1957971924 <== mon.0 v2:172.21.15.37:3300/0 5 ==== osd_map(32..32 src has 1..32) ==== 4857+0+0 (secure 0 0 0) 0x7f33a80956e0 con 0x7f33b00d0330 2024-04-25T10:30:06.667 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:06.665+0000 7f33af7fe640 1 --2- 172.21.15.37:0/1957971924 >> [v2:172.21.15.203:6832/2418205237,v1:172.21.15.203:6833/2418205237] conn(0x7f339c0744b0 0x7f339c0769c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:30:06.668 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:06.666+0000 7f33af7fe640 1 --2- 172.21.15.37:0/1957971924 >> [v2:172.21.15.203:6832/2418205237,v1:172.21.15.203:6833/2418205237] conn(0x7f339c0744b0 0x7f339c0769c0 secure :-1 s=READY pgs=198 cs=0 l=1 rev1=1 crypto rx=0x7f33b0161da0 tx=0x7f33a00079d0 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:30:06.671 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:06.669+0000 7f3397fff640 1 -- 172.21.15.37:0/1957971924 <== mon.0 v2:172.21.15.37:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187246 (secure 0 0 0) 0x7f33a80601c0 con 0x7f33b00d0330 2024-04-25T10:30:06.932 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:06.930+0000 7f33b61e3640 1 -- 172.21.15.37:0/1957971924 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_command({"prefix": "auth del", "entity": "client.bootstrap-mgr"} v 0) -- 0x7f33b00a1780 con 0x7f33b00d0330 2024-04-25T10:30:06.935 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:06.933+0000 7f3397fff640 1 -- 172.21.15.37:0/1957971924 <== mon.0 v2:172.21.15.37:3300/0 7 ==== mon_command_ack([{"prefix": "auth del", "entity": "client.bootstrap-mgr"}]=0 v4) ==== 90+0+0 (secure 0 0 0) 0x7f33a805fac0 con 0x7f33b00d0330 2024-04-25T10:30:06.938 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:06.935+0000 7f33b61e3640 1 -- 172.21.15.37:0/1957971924 >> [v2:172.21.15.203:6832/2418205237,v1:172.21.15.203:6833/2418205237] conn(0x7f339c0744b0 msgr2=0x7f339c0769c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:30:06.938 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:06.935+0000 7f33b61e3640 1 --2- 172.21.15.37:0/1957971924 >> [v2:172.21.15.203:6832/2418205237,v1:172.21.15.203:6833/2418205237] conn(0x7f339c0744b0 0x7f339c0769c0 secure :-1 s=READY pgs=198 cs=0 l=1 rev1=1 crypto rx=0x7f33b0161da0 tx=0x7f33a00079d0 comp rx=0 tx=0).stop 2024-04-25T10:30:06.938 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:06.935+0000 7f33b61e3640 1 -- 172.21.15.37:0/1957971924 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f33b00d0330 msgr2=0x7f33b01f4d50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:30:06.938 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:06.935+0000 7f33b61e3640 1 --2- 172.21.15.37:0/1957971924 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f33b00d0330 0x7f33b01f4d50 secure :-1 s=READY pgs=396 cs=0 l=1 rev1=1 crypto rx=0x7f33a8009fd0 tx=0x7f33a800e4b0 comp rx=0 tx=0).stop 2024-04-25T10:30:06.938 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:06.935+0000 7f33b61e3640 1 -- 172.21.15.37:0/1957971924 shutdown_connections 2024-04-25T10:30:06.938 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:06.935+0000 7f33b61e3640 1 --2- 172.21.15.37:0/1957971924 >> [v2:172.21.15.203:6832/2418205237,v1:172.21.15.203:6833/2418205237] conn(0x7f339c0744b0 0x7f339c0769c0 unknown :-1 s=CLOSED pgs=198 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:30:06.938 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:06.935+0000 7f33b61e3640 1 --2- 172.21.15.37:0/1957971924 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f33b00d0330 0x7f33b01f4d50 unknown :-1 s=CLOSED pgs=396 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:30:06.938 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:06.935+0000 7f33b61e3640 1 --2- 172.21.15.37:0/1957971924 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f33b00b9070 0x7f33b01f4810 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:30:06.938 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:06.935+0000 7f33b61e3640 1 --2- 172.21.15.37:0/1957971924 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f33b00b4fd0 0x7f33b01542d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:30:06.938 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:06.935+0000 7f33b61e3640 1 -- 172.21.15.37:0/1957971924 >> 172.21.15.37:0/1957971924 conn(0x7f33b00d39f0 msgr2=0x7f33b015df30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:30:06.938 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:06.935+0000 7f33b61e3640 1 -- 172.21.15.37:0/1957971924 shutdown_connections 2024-04-25T10:30:06.938 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:06.935+0000 7f33b61e3640 1 -- 172.21.15.37:0/1957971924 wait complete. 2024-04-25T10:30:06.948 DEBUG:teuthology.orchestra.run.smithi037:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph auth del client.bootstrap-osd 2024-04-25T10:30:07.061 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:07.059+0000 7f97c239d640 1 Processor -- start 2024-04-25T10:30:07.061 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:07.059+0000 7f97c239d640 1 -- start start 2024-04-25T10:30:07.063 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:07.060+0000 7f97c239d640 1 --2- >> v2:172.21.15.203:3300/0 conn(0x7f97bc0d02c0 0x7f97bc0b2d60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:30:07.063 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:07.060+0000 7f97c239d640 1 --2- >> v2:172.21.15.37:3300/0 conn(0x7f97bc0b3990 0x7f97bc0cf7c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:30:07.063 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:07.060+0000 7f97c239d640 1 --2- >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f97bc0cfd00 0x7f97bc0d1a00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:30:07.063 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:07.060+0000 7f97c239d640 1 -- --> v2:172.21.15.37:3300/0 -- mon_getmap magic: 0 -- 0x7f97bc06d3e0 con 0x7f97bc0b3990 2024-04-25T10:30:07.063 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:07.060+0000 7f97c239d640 1 -- --> v2:172.21.15.203:3300/0 -- mon_getmap magic: 0 -- 0x7f97bc0cea00 con 0x7f97bc0d02c0 2024-04-25T10:30:07.063 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:07.060+0000 7f97c239d640 1 -- --> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] -- mon_getmap magic: 0 -- 0x7f97bc0cece0 con 0x7f97bc0cfd00 2024-04-25T10:30:07.063 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:07.060+0000 7f97c0b9a640 1 --2- >> v2:172.21.15.37:3300/0 conn(0x7f97bc0b3990 0x7f97bc0cf7c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:30:07.063 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:07.060+0000 7f97c0b9a640 1 --2- >> v2:172.21.15.37:3300/0 conn(0x7f97bc0b3990 0x7f97bc0cf7c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.37:3300/0 says I am v2:172.21.15.37:54718/0 (socket says 172.21.15.37:54718) 2024-04-25T10:30:07.063 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:07.060+0000 7f97c0b9a640 1 -- 172.21.15.37:0/4213948547 learned_addr learned my addr 172.21.15.37:0/4213948547 (peer_addr_for_me v2:172.21.15.37:0/0) 2024-04-25T10:30:07.064 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:07.060+0000 7f97c1b9c640 1 --2- 172.21.15.37:0/4213948547 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f97bc0cfd00 0x7f97bc0d1a00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:30:07.064 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:07.061+0000 7f97c0b9a640 1 -- 172.21.15.37:0/4213948547 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f97bc0cfd00 msgr2=0x7f97bc0d1a00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:30:07.064 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:07.061+0000 7f97c0b9a640 1 --2- 172.21.15.37:0/4213948547 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f97bc0cfd00 0x7f97bc0d1a00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:30:07.064 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:07.061+0000 7f97c0b9a640 1 -- 172.21.15.37:0/4213948547 >> v2:172.21.15.203:3300/0 conn(0x7f97bc0d02c0 msgr2=0x7f97bc0b2d60 unknown :-1 s=STATE_CONNECTING_RE l=0).mark_down 2024-04-25T10:30:07.064 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:07.061+0000 7f97c0b9a640 1 --2- 172.21.15.37:0/4213948547 >> v2:172.21.15.203:3300/0 conn(0x7f97bc0d02c0 0x7f97bc0b2d60 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:30:07.064 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:07.061+0000 7f97c0b9a640 1 -- 172.21.15.37:0/4213948547 --> v2:172.21.15.37:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f97bc0cefc0 con 0x7f97bc0b3990 2024-04-25T10:30:07.066 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:07.061+0000 7f97c0b9a640 1 --2- 172.21.15.37:0/4213948547 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f97bc0b3990 0x7f97bc0cf7c0 secure :-1 s=READY pgs=397 cs=0 l=1 rev1=1 crypto rx=0x7f97bc06e5c0 tx=0x7f97ac02a200 comp rx=0 tx=0).ready entity=mon.0 client_cookie=9d1c6e247696f0c0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:30:07.066 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:07.062+0000 7f97b3fff640 1 -- 172.21.15.37:0/4213948547 <== mon.0 v2:172.21.15.37:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f97ac037070 con 0x7f97bc0b3990 2024-04-25T10:30:07.066 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:07.062+0000 7f97b3fff640 1 -- 172.21.15.37:0/4213948547 <== mon.0 v2:172.21.15.37:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f97ac032440 con 0x7f97bc0b3990 2024-04-25T10:30:07.066 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:07.062+0000 7f97b3fff640 1 -- 172.21.15.37:0/4213948547 <== mon.0 v2:172.21.15.37:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f97ac032810 con 0x7f97bc0b3990 2024-04-25T10:30:07.066 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:07.062+0000 7f97c239d640 1 -- 172.21.15.37:0/4213948547 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f97bc0b3990 msgr2=0x7f97bc0cf7c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:30:07.066 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:07.062+0000 7f97c239d640 1 --2- 172.21.15.37:0/4213948547 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f97bc0b3990 0x7f97bc0cf7c0 secure :-1 s=READY pgs=397 cs=0 l=1 rev1=1 crypto rx=0x7f97bc06e5c0 tx=0x7f97ac02a200 comp rx=0 tx=0).stop 2024-04-25T10:30:07.066 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:07.062+0000 7f97c239d640 1 -- 172.21.15.37:0/4213948547 shutdown_connections 2024-04-25T10:30:07.067 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:07.062+0000 7f97c239d640 1 --2- 172.21.15.37:0/4213948547 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f97bc0b3990 0x7f97bc0cf7c0 unknown :-1 s=CLOSED pgs=397 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:30:07.067 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:07.062+0000 7f97c239d640 1 --2- 172.21.15.37:0/4213948547 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f97bc0cfd00 0x7f97bc0d1a00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:30:07.067 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:07.062+0000 7f97c239d640 1 --2- 172.21.15.37:0/4213948547 >> v2:172.21.15.203:3300/0 conn(0x7f97bc0d02c0 0x7f97bc0b2d60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:30:07.067 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:07.062+0000 7f97c239d640 1 -- 172.21.15.37:0/4213948547 >> 172.21.15.37:0/4213948547 conn(0x7f97bc0b4220 msgr2=0x7f97bc0b4630 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:30:07.067 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:07.062+0000 7f97c239d640 1 -- 172.21.15.37:0/4213948547 shutdown_connections 2024-04-25T10:30:07.067 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:07.062+0000 7f97c239d640 1 -- 172.21.15.37:0/4213948547 wait complete. 2024-04-25T10:30:07.067 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:07.064+0000 7f97c239d640 1 Processor -- start 2024-04-25T10:30:07.067 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:07.064+0000 7f97c239d640 1 -- start start 2024-04-25T10:30:07.069 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:07.065+0000 7f97c239d640 1 --2- >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f97bc0b3990 0x7f97bc0b0550 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:30:07.069 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:07.065+0000 7f97c239d640 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f97bc0cfd00 0x7f97bc0cb910 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:30:07.069 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:07.065+0000 7f97c239d640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f97bc0d02c0 0x7f97bc0cbe50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:30:07.069 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:07.065+0000 7f97c239d640 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 -- 0x7f97bc06d0d0 con 0x7f97bc0d02c0 2024-04-25T10:30:07.069 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:07.065+0000 7f97c239d640 1 -- --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_getmap magic: 0 -- 0x7f97bc06cf50 con 0x7f97bc0cfd00 2024-04-25T10:30:07.069 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:07.065+0000 7f97c239d640 1 -- --> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] -- mon_getmap magic: 0 -- 0x7f97bc0d1f40 con 0x7f97bc0b3990 2024-04-25T10:30:07.069 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:07.065+0000 7f97c1b9c640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f97bc0d02c0 0x7f97bc0cbe50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:30:07.069 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:07.065+0000 7f97c1b9c640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f97bc0d02c0 0x7f97bc0cbe50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.37:3300/0 says I am v2:172.21.15.37:54732/0 (socket says 172.21.15.37:54732) 2024-04-25T10:30:07.069 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:07.066+0000 7f97c1b9c640 1 -- 172.21.15.37:0/504974951 learned_addr learned my addr 172.21.15.37:0/504974951 (peer_addr_for_me v2:172.21.15.37:0/0) 2024-04-25T10:30:07.069 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:07.066+0000 7f97c1b9c640 1 -- 172.21.15.37:0/504974951 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f97bc0b3990 msgr2=0x7f97bc0b0550 unknown :-1 s=STATE_CONNECTING_RE l=1).mark_down 2024-04-25T10:30:07.069 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:07.066+0000 7f97c1b9c640 1 --2- 172.21.15.37:0/504974951 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f97bc0b3990 0x7f97bc0b0550 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:30:07.069 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:07.066+0000 7f97c1b9c640 1 -- 172.21.15.37:0/504974951 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f97bc0cfd00 msgr2=0x7f97bc0cb910 unknown :-1 s=STATE_CONNECTING_RE l=1).mark_down 2024-04-25T10:30:07.069 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:07.066+0000 7f97c1b9c640 1 --2- 172.21.15.37:0/504974951 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f97bc0cfd00 0x7f97bc0cb910 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:30:07.070 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:07.066+0000 7f97c1b9c640 1 -- 172.21.15.37:0/504974951 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f97bc0b1590 con 0x7f97bc0d02c0 2024-04-25T10:30:07.070 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:07.066+0000 7f97c1b9c640 1 --2- 172.21.15.37:0/504974951 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f97bc0d02c0 0x7f97bc0cbe50 secure :-1 s=READY pgs=398 cs=0 l=1 rev1=1 crypto rx=0x7f97b400efc0 tx=0x7f97b400c490 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:30:07.070 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:07.066+0000 7f97b1ffb640 1 -- 172.21.15.37:0/504974951 <== mon.0 v2:172.21.15.37:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f97b4018020 con 0x7f97bc0d02c0 2024-04-25T10:30:07.070 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:07.066+0000 7f97b1ffb640 1 -- 172.21.15.37:0/504974951 <== mon.0 v2:172.21.15.37:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f97b400ccd0 con 0x7f97bc0d02c0 2024-04-25T10:30:07.070 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:07.066+0000 7f97c239d640 1 -- 172.21.15.37:0/504974951 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f97bc0b0fd0 con 0x7f97bc0d02c0 2024-04-25T10:30:07.070 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:07.066+0000 7f97b1ffb640 1 -- 172.21.15.37:0/504974951 <== mon.0 v2:172.21.15.37:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f97b40146a0 con 0x7f97bc0d02c0 2024-04-25T10:30:07.070 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:07.066+0000 7f97c239d640 1 -- 172.21.15.37:0/504974951 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f97bc0c62f0 con 0x7f97bc0d02c0 2024-04-25T10:30:07.071 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:07.067+0000 7f97c239d640 1 -- 172.21.15.37:0/504974951 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f97bc076bc0 con 0x7f97bc0d02c0 2024-04-25T10:30:07.071 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:07.068+0000 7f97b1ffb640 1 -- 172.21.15.37:0/504974951 <== mon.0 v2:172.21.15.37:3300/0 4 ==== mgrmap(e 5) ==== 96782+0+0 (secure 0 0 0) 0x7f97b4005d10 con 0x7f97bc0d02c0 2024-04-25T10:30:07.071 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:07.069+0000 7f97b1ffb640 1 --2- 172.21.15.37:0/504974951 >> [v2:172.21.15.203:6832/2418205237,v1:172.21.15.203:6833/2418205237] conn(0x7f97a00744e0 0x7f97a00769f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:30:07.071 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:07.069+0000 7f97b1ffb640 1 -- 172.21.15.37:0/504974951 <== mon.0 v2:172.21.15.37:3300/0 5 ==== osd_map(32..32 src has 1..32) ==== 4857+0+0 (secure 0 0 0) 0x7f97b4095ba0 con 0x7f97bc0d02c0 2024-04-25T10:30:07.072 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:07.070+0000 7f97c139b640 1 --2- 172.21.15.37:0/504974951 >> [v2:172.21.15.203:6832/2418205237,v1:172.21.15.203:6833/2418205237] conn(0x7f97a00744e0 0x7f97a00769f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:30:07.075 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:07.073+0000 7f97c139b640 1 --2- 172.21.15.37:0/504974951 >> [v2:172.21.15.203:6832/2418205237,v1:172.21.15.203:6833/2418205237] conn(0x7f97a00744e0 0x7f97a00769f0 secure :-1 s=READY pgs=202 cs=0 l=1 rev1=1 crypto rx=0x7f97b80095d0 tx=0x7f97b8006b20 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:30:07.075 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:07.073+0000 7f97b1ffb640 1 -- 172.21.15.37:0/504974951 <== mon.0 v2:172.21.15.37:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187246 (secure 0 0 0) 0x7f97b4022030 con 0x7f97bc0d02c0 2024-04-25T10:30:07.331 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:07.329+0000 7f97c239d640 1 -- 172.21.15.37:0/504974951 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_command({"prefix": "auth del", "entity": "client.bootstrap-osd"} v 0) -- 0x7f97bc076380 con 0x7f97bc0d02c0 2024-04-25T10:30:07.334 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:07.332+0000 7f97b1ffb640 1 -- 172.21.15.37:0/504974951 <== mon.0 v2:172.21.15.37:3300/0 7 ==== mon_command_ack([{"prefix": "auth del", "entity": "client.bootstrap-osd"}]=0 v5) ==== 90+0+0 (secure 0 0 0) 0x7f97b4010040 con 0x7f97bc0d02c0 2024-04-25T10:30:07.336 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:07.334+0000 7f97c239d640 1 -- 172.21.15.37:0/504974951 >> [v2:172.21.15.203:6832/2418205237,v1:172.21.15.203:6833/2418205237] conn(0x7f97a00744e0 msgr2=0x7f97a00769f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:30:07.336 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:07.334+0000 7f97c239d640 1 --2- 172.21.15.37:0/504974951 >> [v2:172.21.15.203:6832/2418205237,v1:172.21.15.203:6833/2418205237] conn(0x7f97a00744e0 0x7f97a00769f0 secure :-1 s=READY pgs=202 cs=0 l=1 rev1=1 crypto rx=0x7f97b80095d0 tx=0x7f97b8006b20 comp rx=0 tx=0).stop 2024-04-25T10:30:07.336 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:07.334+0000 7f97c239d640 1 -- 172.21.15.37:0/504974951 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f97bc0d02c0 msgr2=0x7f97bc0cbe50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:30:07.336 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:07.334+0000 7f97c239d640 1 --2- 172.21.15.37:0/504974951 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f97bc0d02c0 0x7f97bc0cbe50 secure :-1 s=READY pgs=398 cs=0 l=1 rev1=1 crypto rx=0x7f97b400efc0 tx=0x7f97b400c490 comp rx=0 tx=0).stop 2024-04-25T10:30:07.336 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:07.334+0000 7f97c239d640 1 -- 172.21.15.37:0/504974951 shutdown_connections 2024-04-25T10:30:07.336 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:07.334+0000 7f97c239d640 1 --2- 172.21.15.37:0/504974951 >> [v2:172.21.15.203:6832/2418205237,v1:172.21.15.203:6833/2418205237] conn(0x7f97a00744e0 0x7f97a00769f0 unknown :-1 s=CLOSED pgs=202 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:30:07.337 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:07.334+0000 7f97c239d640 1 --2- 172.21.15.37:0/504974951 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f97bc0d02c0 0x7f97bc0cbe50 unknown :-1 s=CLOSED pgs=398 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:30:07.337 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:07.334+0000 7f97c239d640 1 --2- 172.21.15.37:0/504974951 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f97bc0cfd00 0x7f97bc0cb910 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:30:07.337 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:07.334+0000 7f97c239d640 1 --2- 172.21.15.37:0/504974951 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f97bc0b3990 0x7f97bc0b0550 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:30:07.337 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:07.334+0000 7f97c239d640 1 -- 172.21.15.37:0/504974951 >> 172.21.15.37:0/504974951 conn(0x7f97bc0b4220 msgr2=0x7f97bc0b7600 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:30:07.337 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:07.334+0000 7f97c239d640 1 -- 172.21.15.37:0/504974951 shutdown_connections 2024-04-25T10:30:07.337 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:07.334+0000 7f97c239d640 1 -- 172.21.15.37:0/504974951 wait complete. 2024-04-25T10:30:07.347 DEBUG:teuthology.orchestra.run.smithi037:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph auth del client.bootstrap-rbd 2024-04-25T10:30:07.459 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:07.457+0000 7f0574e70640 1 Processor -- start 2024-04-25T10:30:07.459 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:07.457+0000 7f0574e70640 1 -- start start 2024-04-25T10:30:07.461 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:07.458+0000 7f0574e70640 1 --2- >> v2:172.21.15.203:3300/0 conn(0x7f05700d3230 0x7f057016b0c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:30:07.461 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:07.458+0000 7f0574e70640 1 -- --> v1:172.21.15.37:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f05700d10a0 con 0x7f05700d5160 2024-04-25T10:30:07.461 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:07.458+0000 7f0574e70640 1 -- --> v1:172.21.15.203:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f05700d1370 con 0x7f05700b4e80 2024-04-25T10:30:07.461 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:07.458+0000 7f0574e70640 1 -- --> v2:172.21.15.203:3300/0 -- mon_getmap magic: 0 -- 0x7f05700d1650 con 0x7f05700d3230 2024-04-25T10:30:07.461 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:07.459+0000 7f056e59c640 1 --1- >> v1:172.21.15.37:6789/0 conn(0x7f05700d5160 0x7f05700d0990 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.37:6789/0 says I am v1:172.21.15.37:60730/0 (socket says 172.21.15.37:60730) 2024-04-25T10:30:07.461 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:07.459+0000 7f056e59c640 1 -- 172.21.15.37:0/239076502 learned_addr learned my addr 172.21.15.37:0/239076502 (peer_addr_for_me v1:172.21.15.37:0/0) 2024-04-25T10:30:07.462 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:07.459+0000 7f056ed9d640 1 --2- 172.21.15.37:0/239076502 >> v2:172.21.15.203:3300/0 conn(0x7f05700d3230 0x7f057016b0c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:30:07.462 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:07.459+0000 7f056d59a640 1 -- 172.21.15.37:0/239076502 <== mon.0 v1:172.21.15.37:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 547482757 0 0) 0x7f05700d10a0 con 0x7f05700d5160 2024-04-25T10:30:07.462 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:07.459+0000 7f056d59a640 1 -- 172.21.15.37:0/239076502 --> v1:172.21.15.37:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f0558003520 con 0x7f05700d5160 2024-04-25T10:30:07.462 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:07.459+0000 7f056d59a640 1 -- 172.21.15.37:0/239076502 <== mon.0 v1:172.21.15.37:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 3258195767 0 0) 0x7f0558003520 con 0x7f05700d5160 2024-04-25T10:30:07.462 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:07.459+0000 7f056ed9d640 1 -- 172.21.15.37:0/239076502 >> v1:172.21.15.203:6789/0 conn(0x7f05700b4e80 legacy=0x7f05700d2b20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:30:07.462 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:07.459+0000 7f056ed9d640 1 -- 172.21.15.37:0/239076502 >> v1:172.21.15.37:6789/0 conn(0x7f05700d5160 legacy=0x7f05700d0990 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:30:07.462 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:07.459+0000 7f056ed9d640 1 -- 172.21.15.37:0/239076502 --> v2:172.21.15.203:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f05700d4680 con 0x7f05700d3230 2024-04-25T10:30:07.463 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:07.460+0000 7f056ed9d640 1 --2- 172.21.15.37:0/239076502 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f05700d3230 0x7f057016b0c0 secure :-1 s=READY pgs=173 cs=0 l=1 rev1=1 crypto rx=0x7f05680097a0 tx=0x7f056802a130 comp rx=0 tx=0).ready entity=mon.1 client_cookie=4f8022d90ce075df server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:30:07.463 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:07.460+0000 7f056d59a640 1 -- 172.21.15.37:0/239076502 <== mon.1 v2:172.21.15.203:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0568037070 con 0x7f05700d3230 2024-04-25T10:30:07.463 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:07.460+0000 7f056d59a640 1 -- 172.21.15.37:0/239076502 <== mon.1 v2:172.21.15.203:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f056802abc0 con 0x7f05700d3230 2024-04-25T10:30:07.463 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:07.460+0000 7f056d59a640 1 -- 172.21.15.37:0/239076502 <== mon.1 v2:172.21.15.203:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0560001a10 con 0x7f05700d3230 2024-04-25T10:30:07.464 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:07.461+0000 7f0574e70640 1 -- 172.21.15.37:0/239076502 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f05700d3230 msgr2=0x7f057016b0c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:30:07.464 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:07.461+0000 7f0574e70640 1 --2- 172.21.15.37:0/239076502 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f05700d3230 0x7f057016b0c0 secure :-1 s=READY pgs=173 cs=0 l=1 rev1=1 crypto rx=0x7f05680097a0 tx=0x7f056802a130 comp rx=0 tx=0).stop 2024-04-25T10:30:07.464 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:07.461+0000 7f0574e70640 1 -- 172.21.15.37:0/239076502 shutdown_connections 2024-04-25T10:30:07.464 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:07.461+0000 7f0574e70640 1 --2- 172.21.15.37:0/239076502 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f05700d3230 0x7f057016b0c0 unknown :-1 s=CLOSED pgs=173 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:30:07.464 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:07.461+0000 7f0574e70640 1 -- 172.21.15.37:0/239076502 >> 172.21.15.37:0/239076502 conn(0x7f05700b7a80 msgr2=0x7f05700b7e90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:30:07.464 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:07.461+0000 7f0574e70640 1 -- 172.21.15.37:0/239076502 shutdown_connections 2024-04-25T10:30:07.464 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:07.461+0000 7f0574e70640 1 -- 172.21.15.37:0/239076502 wait complete. 2024-04-25T10:30:07.465 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:07.463+0000 7f0574e70640 1 Processor -- start 2024-04-25T10:30:07.465 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:07.463+0000 7f0574e70640 1 -- start start 2024-04-25T10:30:07.467 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:07.464+0000 7f0574e70640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f05700b4e80 0x7f0570162fb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:30:07.467 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:07.464+0000 7f0574e70640 1 --2- >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f05700d5160 0x7f05701612e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:30:07.468 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:07.464+0000 7f0574e70640 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f05701f6430 0x7f05700d0740 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:30:07.468 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:07.464+0000 7f0574e70640 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 -- 0x7f05700d4c40 con 0x7f05700b4e80 2024-04-25T10:30:07.468 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:07.464+0000 7f0574e70640 1 -- --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_getmap magic: 0 -- 0x7f05700b5290 con 0x7f05701f6430 2024-04-25T10:30:07.468 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:07.464+0000 7f0574e70640 1 -- --> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] -- mon_getmap magic: 0 -- 0x7f0570164ac0 con 0x7f05700d5160 2024-04-25T10:30:07.468 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:07.464+0000 7f056e59c640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f05700b4e80 0x7f0570162fb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:30:07.468 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:07.464+0000 7f056e59c640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f05700b4e80 0x7f0570162fb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.37:3300/0 says I am v2:172.21.15.37:54734/0 (socket says 172.21.15.37:54734) 2024-04-25T10:30:07.468 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:07.464+0000 7f056e59c640 1 -- 172.21.15.37:0/3746278777 learned_addr learned my addr 172.21.15.37:0/3746278777 (peer_addr_for_me v2:172.21.15.37:0/0) 2024-04-25T10:30:07.468 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:07.465+0000 7f056dd9b640 1 --2- 172.21.15.37:0/3746278777 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f05700d5160 0x7f05701612e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:30:07.468 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:07.465+0000 7f056e59c640 1 -- 172.21.15.37:0/3746278777 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f05700d5160 msgr2=0x7f05701612e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:30:07.468 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:07.465+0000 7f056e59c640 1 --2- 172.21.15.37:0/3746278777 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f05700d5160 0x7f05701612e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:30:07.468 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:07.465+0000 7f056e59c640 1 -- 172.21.15.37:0/3746278777 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f05701f6430 msgr2=0x7f05700d0740 unknown :-1 s=STATE_CONNECTING_RE l=1).mark_down 2024-04-25T10:30:07.468 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:07.465+0000 7f056e59c640 1 --2- 172.21.15.37:0/3746278777 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f05701f6430 0x7f05700d0740 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:30:07.468 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:07.465+0000 7f056e59c640 1 -- 172.21.15.37:0/3746278777 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f0570160240 con 0x7f05700b4e80 2024-04-25T10:30:07.469 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:07.465+0000 7f056dd9b640 1 --2- 172.21.15.37:0/3746278777 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f05700d5160 0x7f05701612e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T10:30:07.469 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:07.465+0000 7f056e59c640 1 --2- 172.21.15.37:0/3746278777 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f05700b4e80 0x7f0570162fb0 secure :-1 s=READY pgs=400 cs=0 l=1 rev1=1 crypto rx=0x7f056000c8c0 tx=0x7f056000cd80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:30:07.469 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:07.465+0000 7f055effd640 1 -- 172.21.15.37:0/3746278777 <== mon.0 v2:172.21.15.37:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0560018070 con 0x7f05700b4e80 2024-04-25T10:30:07.469 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:07.465+0000 7f055effd640 1 -- 172.21.15.37:0/3746278777 <== mon.0 v2:172.21.15.37:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f056000a5e0 con 0x7f05700b4e80 2024-04-25T10:30:07.469 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:07.465+0000 7f0574e70640 1 -- 172.21.15.37:0/3746278777 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f0570160520 con 0x7f05700b4e80 2024-04-25T10:30:07.469 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:07.465+0000 7f055effd640 1 -- 172.21.15.37:0/3746278777 <== mon.0 v2:172.21.15.37:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f056000a9d0 con 0x7f05700b4e80 2024-04-25T10:30:07.469 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:07.465+0000 7f0574e70640 1 -- 172.21.15.37:0/3746278777 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f0570160ae0 con 0x7f05700b4e80 2024-04-25T10:30:07.470 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:07.466+0000 7f0574e70640 1 -- 172.21.15.37:0/3746278777 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f057006a140 con 0x7f05700b4e80 2024-04-25T10:30:07.470 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:07.467+0000 7f055effd640 1 -- 172.21.15.37:0/3746278777 <== mon.0 v2:172.21.15.37:3300/0 4 ==== mgrmap(e 5) ==== 96782+0+0 (secure 0 0 0) 0x7f056000ab70 con 0x7f05700b4e80 2024-04-25T10:30:07.470 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:07.467+0000 7f055effd640 1 --2- 172.21.15.37:0/3746278777 >> [v2:172.21.15.203:6832/2418205237,v1:172.21.15.203:6833/2418205237] conn(0x7f05500744e0 0x7f05500769f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:30:07.470 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:07.468+0000 7f055effd640 1 -- 172.21.15.37:0/3746278777 <== mon.0 v2:172.21.15.37:3300/0 5 ==== osd_map(32..32 src has 1..32) ==== 4857+0+0 (secure 0 0 0) 0x7f0560094730 con 0x7f05700b4e80 2024-04-25T10:30:07.470 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:07.468+0000 7f056dd9b640 1 --2- 172.21.15.37:0/3746278777 >> [v2:172.21.15.203:6832/2418205237,v1:172.21.15.203:6833/2418205237] conn(0x7f05500744e0 0x7f05500769f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:30:07.470 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:07.468+0000 7f056dd9b640 1 --2- 172.21.15.37:0/3746278777 >> [v2:172.21.15.203:6832/2418205237,v1:172.21.15.203:6833/2418205237] conn(0x7f05500744e0 0x7f05500769f0 secure :-1 s=READY pgs=203 cs=0 l=1 rev1=1 crypto rx=0x7f05700b81f0 tx=0x7f0564009040 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:30:07.474 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:07.472+0000 7f055effd640 1 -- 172.21.15.37:0/3746278777 <== mon.0 v2:172.21.15.37:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187246 (secure 0 0 0) 0x7f056005f180 con 0x7f05700b4e80 2024-04-25T10:30:07.732 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:07.730+0000 7f0574e70640 1 -- 172.21.15.37:0/3746278777 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_command({"prefix": "auth del", "entity": "client.bootstrap-rbd"} v 0) -- 0x7f05700d3f50 con 0x7f05700b4e80 2024-04-25T10:30:07.734 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:07.732+0000 7f055effd640 1 -- 172.21.15.37:0/3746278777 <== mon.0 v2:172.21.15.37:3300/0 7 ==== mon_command_ack([{"prefix": "auth del", "entity": "client.bootstrap-rbd"}]=0 v6) ==== 90+0+0 (secure 0 0 0) 0x7f056005ea80 con 0x7f05700b4e80 2024-04-25T10:30:07.736 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:07.734+0000 7f0574e70640 1 -- 172.21.15.37:0/3746278777 >> [v2:172.21.15.203:6832/2418205237,v1:172.21.15.203:6833/2418205237] conn(0x7f05500744e0 msgr2=0x7f05500769f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:30:07.736 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:07.734+0000 7f0574e70640 1 --2- 172.21.15.37:0/3746278777 >> [v2:172.21.15.203:6832/2418205237,v1:172.21.15.203:6833/2418205237] conn(0x7f05500744e0 0x7f05500769f0 secure :-1 s=READY pgs=203 cs=0 l=1 rev1=1 crypto rx=0x7f05700b81f0 tx=0x7f0564009040 comp rx=0 tx=0).stop 2024-04-25T10:30:07.737 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:07.734+0000 7f0574e70640 1 -- 172.21.15.37:0/3746278777 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f05700b4e80 msgr2=0x7f0570162fb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:30:07.737 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:07.734+0000 7f0574e70640 1 --2- 172.21.15.37:0/3746278777 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f05700b4e80 0x7f0570162fb0 secure :-1 s=READY pgs=400 cs=0 l=1 rev1=1 crypto rx=0x7f056000c8c0 tx=0x7f056000cd80 comp rx=0 tx=0).stop 2024-04-25T10:30:07.737 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:07.734+0000 7f0574e70640 1 -- 172.21.15.37:0/3746278777 shutdown_connections 2024-04-25T10:30:07.737 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:07.734+0000 7f0574e70640 1 --2- 172.21.15.37:0/3746278777 >> [v2:172.21.15.203:6832/2418205237,v1:172.21.15.203:6833/2418205237] conn(0x7f05500744e0 0x7f05500769f0 unknown :-1 s=CLOSED pgs=203 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:30:07.737 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:07.734+0000 7f0574e70640 1 --2- 172.21.15.37:0/3746278777 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f05701f6430 0x7f05700d0740 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:30:07.737 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:07.734+0000 7f0574e70640 1 --2- 172.21.15.37:0/3746278777 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f05700d5160 0x7f05701612e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:30:07.737 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:07.734+0000 7f0574e70640 1 --2- 172.21.15.37:0/3746278777 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f05700b4e80 0x7f0570162fb0 unknown :-1 s=CLOSED pgs=400 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:30:07.737 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:07.734+0000 7f0574e70640 1 -- 172.21.15.37:0/3746278777 >> 172.21.15.37:0/3746278777 conn(0x7f05700b7a80 msgr2=0x7f05700d5ae0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:30:07.737 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:07.734+0000 7f0574e70640 1 -- 172.21.15.37:0/3746278777 shutdown_connections 2024-04-25T10:30:07.737 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:07.734+0000 7f0574e70640 1 -- 172.21.15.37:0/3746278777 wait complete. 2024-04-25T10:30:07.747 DEBUG:teuthology.orchestra.run.smithi037:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-25T10:30:07.748 DEBUG:teuthology.orchestra.run.smithi037:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph auth del client.bootstrap-rbd-mirror 2024-04-25T10:30:07.790 DEBUG:teuthology.orchestra.run.smithi203:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-25T10:30:07.902 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:07.899+0000 7f0104dee640 1 Processor -- start 2024-04-25T10:30:07.902 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:07.900+0000 7f0104dee640 1 -- start start 2024-04-25T10:30:07.904 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:07.901+0000 7f0104dee640 1 --2- >> v2:172.21.15.203:3300/0 conn(0x7f010006e390 0x7f010015eec0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:30:07.904 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:07.901+0000 7f0104dee640 1 --2- >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f010006c8f0 0x7f010009b700 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:30:07.905 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:07.901+0000 7f0104dee640 1 --2- >> v2:172.21.15.37:3300/0 conn(0x7f010006cd00 0x7f010009bc40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:30:07.905 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:07.901+0000 7f0104dee640 1 -- --> v2:172.21.15.37:3300/0 -- mon_getmap magic: 0 -- 0x7f01000b4240 con 0x7f010006cd00 2024-04-25T10:30:07.905 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:07.901+0000 7f0104dee640 1 -- --> v2:172.21.15.203:3300/0 -- mon_getmap magic: 0 -- 0x7f010006eb00 con 0x7f010006e390 2024-04-25T10:30:07.905 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:07.901+0000 7f0104dee640 1 -- --> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] -- mon_getmap magic: 0 -- 0x7f010006ede0 con 0x7f010006c8f0 2024-04-25T10:30:07.905 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:07.901+0000 7f00fed9d640 1 --2- >> v2:172.21.15.37:3300/0 conn(0x7f010006cd00 0x7f010009bc40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:30:07.905 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:07.901+0000 7f00fed9d640 1 --2- >> v2:172.21.15.37:3300/0 conn(0x7f010006cd00 0x7f010009bc40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.37:3300/0 says I am v2:172.21.15.37:54742/0 (socket says 172.21.15.37:54742) 2024-04-25T10:30:07.905 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:07.901+0000 7f00fed9d640 1 -- 172.21.15.37:0/1237234809 learned_addr learned my addr 172.21.15.37:0/1237234809 (peer_addr_for_me v2:172.21.15.37:0/0) 2024-04-25T10:30:07.905 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:07.901+0000 7f00fdd9b640 1 --2- 172.21.15.37:0/1237234809 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f010006c8f0 0x7f010009b700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:30:07.905 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:07.902+0000 7f00fe59c640 1 --2- 172.21.15.37:0/1237234809 >> v2:172.21.15.203:3300/0 conn(0x7f010006e390 0x7f010015eec0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:30:07.905 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:07.902+0000 7f00fed9d640 1 -- 172.21.15.37:0/1237234809 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f010006c8f0 msgr2=0x7f010009b700 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:30:07.905 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:07.902+0000 7f00fed9d640 1 --2- 172.21.15.37:0/1237234809 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f010006c8f0 0x7f010009b700 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:30:07.905 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:07.902+0000 7f00fed9d640 1 -- 172.21.15.37:0/1237234809 >> v2:172.21.15.203:3300/0 conn(0x7f010006e390 msgr2=0x7f010015eec0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:30:07.906 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:07.902+0000 7f00fed9d640 1 --2- 172.21.15.37:0/1237234809 >> v2:172.21.15.203:3300/0 conn(0x7f010006e390 0x7f010015eec0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:30:07.906 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:07.902+0000 7f00fed9d640 1 -- 172.21.15.37:0/1237234809 --> v2:172.21.15.37:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f010015e6c0 con 0x7f010006cd00 2024-04-25T10:30:07.906 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:07.902+0000 7f00fdd9b640 1 --2- 172.21.15.37:0/1237234809 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f010006c8f0 0x7f010009b700 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T10:30:07.906 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:07.902+0000 7f00fe59c640 1 --2- 172.21.15.37:0/1237234809 >> v2:172.21.15.203:3300/0 conn(0x7f010006e390 0x7f010015eec0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T10:30:07.906 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:07.902+0000 7f00fed9d640 1 --2- 172.21.15.37:0/1237234809 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f010006cd00 0x7f010009bc40 secure :-1 s=READY pgs=401 cs=0 l=1 rev1=1 crypto rx=0x7f01000d38c0 tx=0x7f00f002a6a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b933cafe89dbb43a server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:30:07.906 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:07.903+0000 7f00fd59a640 1 -- 172.21.15.37:0/1237234809 <== mon.0 v2:172.21.15.37:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f00f0036070 con 0x7f010006cd00 2024-04-25T10:30:07.906 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:07.903+0000 7f00fd59a640 1 -- 172.21.15.37:0/1237234809 <== mon.0 v2:172.21.15.37:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f00f0031600 con 0x7f010006cd00 2024-04-25T10:30:07.906 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:07.903+0000 7f00fd59a640 1 -- 172.21.15.37:0/1237234809 <== mon.0 v2:172.21.15.37:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f00f00319d0 con 0x7f010006cd00 2024-04-25T10:30:07.907 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:07.903+0000 7f0104dee640 1 -- 172.21.15.37:0/1237234809 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f010006cd00 msgr2=0x7f010009bc40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:30:07.907 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:07.903+0000 7f0104dee640 1 --2- 172.21.15.37:0/1237234809 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f010006cd00 0x7f010009bc40 secure :-1 s=READY pgs=401 cs=0 l=1 rev1=1 crypto rx=0x7f01000d38c0 tx=0x7f00f002a6a0 comp rx=0 tx=0).stop 2024-04-25T10:30:07.907 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:07.903+0000 7f0104dee640 1 -- 172.21.15.37:0/1237234809 shutdown_connections 2024-04-25T10:30:07.907 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:07.903+0000 7f0104dee640 1 --2- 172.21.15.37:0/1237234809 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f010006cd00 0x7f010009bc40 unknown :-1 s=CLOSED pgs=401 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:30:07.908 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:07.903+0000 7f0104dee640 1 --2- 172.21.15.37:0/1237234809 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f010006c8f0 0x7f010009b700 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:30:07.908 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:07.903+0000 7f0104dee640 1 --2- 172.21.15.37:0/1237234809 >> v2:172.21.15.203:3300/0 conn(0x7f010006e390 0x7f010015eec0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:30:07.908 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:07.903+0000 7f0104dee640 1 -- 172.21.15.37:0/1237234809 >> 172.21.15.37:0/1237234809 conn(0x7f01000b7600 msgr2=0x7f01000b7c30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:30:07.908 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:07.903+0000 7f0104dee640 1 -- 172.21.15.37:0/1237234809 shutdown_connections 2024-04-25T10:30:07.908 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:07.903+0000 7f0104dee640 1 -- 172.21.15.37:0/1237234809 wait complete. 2024-04-25T10:30:07.908 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:07.905+0000 7f0104dee640 1 Processor -- start 2024-04-25T10:30:07.908 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:07.905+0000 7f0104dee640 1 -- start start 2024-04-25T10:30:07.910 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:07.907+0000 7f0104dee640 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f010006c8f0 0x7f01000b0f10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:30:07.911 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:07.907+0000 7f0104dee640 1 --2- >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f010006cd00 0x7f01000cfe90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:30:07.911 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:07.907+0000 7f0104dee640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f010006e390 0x7f01000d03d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:30:07.911 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:07.907+0000 7f0104dee640 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 -- 0x7f010006e7a0 con 0x7f010006e390 2024-04-25T10:30:07.911 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:07.907+0000 7f0104dee640 1 -- --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_getmap magic: 0 -- 0x7f010015f400 con 0x7f010006c8f0 2024-04-25T10:30:07.911 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:07.907+0000 7f0104dee640 1 -- --> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] -- mon_getmap magic: 0 -- 0x7f010015bd00 con 0x7f010006cd00 2024-04-25T10:30:07.911 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:07.907+0000 7f00fed9d640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f010006e390 0x7f01000d03d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:30:07.911 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:07.907+0000 7f00fed9d640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f010006e390 0x7f01000d03d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.37:3300/0 says I am v2:172.21.15.37:54746/0 (socket says 172.21.15.37:54746) 2024-04-25T10:30:07.911 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:07.907+0000 7f00fed9d640 1 -- 172.21.15.37:0/4143144880 learned_addr learned my addr 172.21.15.37:0/4143144880 (peer_addr_for_me v2:172.21.15.37:0/0) 2024-04-25T10:30:07.911 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:07.907+0000 7f00fdd9b640 1 --2- 172.21.15.37:0/4143144880 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f010006cd00 0x7f01000cfe90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:30:07.911 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:07.907+0000 7f00fed9d640 1 -- 172.21.15.37:0/4143144880 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f010006cd00 msgr2=0x7f01000cfe90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:30:07.911 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:07.907+0000 7f00fed9d640 1 --2- 172.21.15.37:0/4143144880 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f010006cd00 0x7f01000cfe90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:30:07.911 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:07.907+0000 7f00fed9d640 1 -- 172.21.15.37:0/4143144880 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f010006c8f0 msgr2=0x7f01000b0f10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:30:07.912 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:07.907+0000 7f00fed9d640 1 --2- 172.21.15.37:0/4143144880 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f010006c8f0 0x7f01000b0f10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:30:07.912 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:07.907+0000 7f00fed9d640 1 -- 172.21.15.37:0/4143144880 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f01000cba90 con 0x7f010006e390 2024-04-25T10:30:07.912 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:07.908+0000 7f00fed9d640 1 --2- 172.21.15.37:0/4143144880 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f010006e390 0x7f01000d03d0 secure :-1 s=READY pgs=402 cs=0 l=1 rev1=1 crypto rx=0x7f00f0009f40 tx=0x7f00f0004a40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:30:07.912 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:07.908+0000 7f00f6ffd640 1 -- 172.21.15.37:0/4143144880 <== mon.0 v2:172.21.15.37:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f00f0036050 con 0x7f010006e390 2024-04-25T10:30:07.912 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:07.908+0000 7f00f6ffd640 1 -- 172.21.15.37:0/4143144880 <== mon.0 v2:172.21.15.37:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f00f0042070 con 0x7f010006e390 2024-04-25T10:30:07.912 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:07.908+0000 7f0104dee640 1 -- 172.21.15.37:0/4143144880 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f01001f65f0 con 0x7f010006e390 2024-04-25T10:30:07.912 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:07.908+0000 7f00f6ffd640 1 -- 172.21.15.37:0/4143144880 <== mon.0 v2:172.21.15.37:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f00f0009510 con 0x7f010006e390 2024-04-25T10:30:07.912 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:07.908+0000 7f0104dee640 1 -- 172.21.15.37:0/4143144880 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f01001f6bb0 con 0x7f010006e390 2024-04-25T10:30:07.912 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:07.909+0000 7f0104dee640 1 -- 172.21.15.37:0/4143144880 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f01000cbc60 con 0x7f010006e390 2024-04-25T10:30:07.913 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:07.909+0000 7f00f6ffd640 1 -- 172.21.15.37:0/4143144880 <== mon.0 v2:172.21.15.37:3300/0 4 ==== mgrmap(e 5) ==== 96782+0+0 (secure 0 0 0) 0x7f00f0007bd0 con 0x7f010006e390 2024-04-25T10:30:07.913 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:07.910+0000 7f00f6ffd640 1 --2- 172.21.15.37:0/4143144880 >> [v2:172.21.15.203:6832/2418205237,v1:172.21.15.203:6833/2418205237] conn(0x7f00e00744b0 0x7f00e00769c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:30:07.913 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:07.910+0000 7f00f6ffd640 1 -- 172.21.15.37:0/4143144880 <== mon.0 v2:172.21.15.37:3300/0 5 ==== osd_map(32..32 src has 1..32) ==== 4857+0+0 (secure 0 0 0) 0x7f00f00b5dd0 con 0x7f010006e390 2024-04-25T10:30:07.913 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:07.912+0000 7f00fe59c640 1 --2- 172.21.15.37:0/4143144880 >> [v2:172.21.15.203:6832/2418205237,v1:172.21.15.203:6833/2418205237] conn(0x7f00e00744b0 0x7f00e00769c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:30:07.916 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:07.914+0000 7f00fe59c640 1 --2- 172.21.15.37:0/4143144880 >> [v2:172.21.15.203:6832/2418205237,v1:172.21.15.203:6833/2418205237] conn(0x7f00e00744b0 0x7f00e00769c0 secure :-1 s=READY pgs=204 cs=0 l=1 rev1=1 crypto rx=0x7f00f8006b10 tx=0x7f00f8008040 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:30:07.916 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:07.915+0000 7f00f6ffd640 1 -- 172.21.15.37:0/4143144880 <== mon.0 v2:172.21.15.37:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187246 (secure 0 0 0) 0x7f00f0080940 con 0x7f010006e390 2024-04-25T10:30:08.180 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:08.178+0000 7f0104dee640 1 -- 172.21.15.37:0/4143144880 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_command({"prefix": "auth del", "entity": "client.bootstrap-rbd-mirror"} v 0) -- 0x7f01001f6360 con 0x7f010006e390 2024-04-25T10:30:08.183 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:08.181+0000 7f00f6ffd640 1 -- 172.21.15.37:0/4143144880 <== mon.0 v2:172.21.15.37:3300/0 7 ==== mon_command_ack([{"prefix": "auth del", "entity": "client.bootstrap-rbd-mirror"}]=0 v7) ==== 97+0+0 (secure 0 0 0) 0x7f00f0080240 con 0x7f010006e390 2024-04-25T10:30:08.186 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:08.183+0000 7f0104dee640 1 -- 172.21.15.37:0/4143144880 >> [v2:172.21.15.203:6832/2418205237,v1:172.21.15.203:6833/2418205237] conn(0x7f00e00744b0 msgr2=0x7f00e00769c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:30:08.186 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:08.183+0000 7f0104dee640 1 --2- 172.21.15.37:0/4143144880 >> [v2:172.21.15.203:6832/2418205237,v1:172.21.15.203:6833/2418205237] conn(0x7f00e00744b0 0x7f00e00769c0 secure :-1 s=READY pgs=204 cs=0 l=1 rev1=1 crypto rx=0x7f00f8006b10 tx=0x7f00f8008040 comp rx=0 tx=0).stop 2024-04-25T10:30:08.186 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:08.183+0000 7f0104dee640 1 -- 172.21.15.37:0/4143144880 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f010006e390 msgr2=0x7f01000d03d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:30:08.186 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:08.183+0000 7f0104dee640 1 --2- 172.21.15.37:0/4143144880 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f010006e390 0x7f01000d03d0 secure :-1 s=READY pgs=402 cs=0 l=1 rev1=1 crypto rx=0x7f00f0009f40 tx=0x7f00f0004a40 comp rx=0 tx=0).stop 2024-04-25T10:30:08.186 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:08.183+0000 7f0104dee640 1 -- 172.21.15.37:0/4143144880 shutdown_connections 2024-04-25T10:30:08.186 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:08.183+0000 7f0104dee640 1 --2- 172.21.15.37:0/4143144880 >> [v2:172.21.15.203:6832/2418205237,v1:172.21.15.203:6833/2418205237] conn(0x7f00e00744b0 0x7f00e00769c0 unknown :-1 s=CLOSED pgs=204 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:30:08.186 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:08.183+0000 7f0104dee640 1 --2- 172.21.15.37:0/4143144880 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f010006e390 0x7f01000d03d0 unknown :-1 s=CLOSED pgs=402 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:30:08.186 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:08.183+0000 7f0104dee640 1 --2- 172.21.15.37:0/4143144880 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f010006cd00 0x7f01000cfe90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:30:08.186 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:08.183+0000 7f0104dee640 1 --2- 172.21.15.37:0/4143144880 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f010006c8f0 0x7f01000b0f10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:30:08.187 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:08.183+0000 7f0104dee640 1 -- 172.21.15.37:0/4143144880 >> 172.21.15.37:0/4143144880 conn(0x7f01000b7600 msgr2=0x7f0100159b70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:30:08.187 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:08.184+0000 7f0104dee640 1 -- 172.21.15.37:0/4143144880 shutdown_connections 2024-04-25T10:30:08.187 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:08.184+0000 7f0104dee640 1 -- 172.21.15.37:0/4143144880 wait complete. 2024-04-25T10:30:08.197 DEBUG:teuthology.orchestra.run.smithi037:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph auth del client.bootstrap-rgw 2024-04-25T10:30:08.349 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:08.347+0000 7fbb9889f640 1 Processor -- start 2024-04-25T10:30:08.350 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:08.347+0000 7fbb9889f640 1 -- start start 2024-04-25T10:30:08.351 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:08.349+0000 7fbb9889f640 1 --2- >> v2:172.21.15.37:3300/0 conn(0x7fbb940b2dc0 0x7fbb940d2860 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:30:08.351 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:08.349+0000 7fbb9889f640 1 --2- >> v2:172.21.15.203:3300/0 conn(0x7fbb9406daf0 0x7fbb9406df00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:30:08.351 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:08.349+0000 7fbb9889f640 1 -- --> v1:172.21.15.203:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fbb940d2e40 con 0x7fbb9406e440 2024-04-25T10:30:08.352 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:08.349+0000 7fbb9889f640 1 -- --> v2:172.21.15.37:3300/0 -- mon_getmap magic: 0 -- 0x7fbb940b92c0 con 0x7fbb940b2dc0 2024-04-25T10:30:08.352 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:08.349+0000 7fbb9889f640 1 -- --> v2:172.21.15.203:3300/0 -- mon_getmap magic: 0 -- 0x7fbb940ce0d0 con 0x7fbb9406daf0 2024-04-25T10:30:08.352 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:08.349+0000 7fbb9259c640 1 --2- >> v2:172.21.15.37:3300/0 conn(0x7fbb940b2dc0 0x7fbb940d2860 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:30:08.352 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:08.349+0000 7fbb9259c640 1 --2- >> v2:172.21.15.37:3300/0 conn(0x7fbb940b2dc0 0x7fbb940d2860 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.37:3300/0 says I am v2:172.21.15.37:54762/0 (socket says 172.21.15.37:54762) 2024-04-25T10:30:08.352 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:08.349+0000 7fbb9259c640 1 -- 172.21.15.37:0/309788964 learned_addr learned my addr 172.21.15.37:0/309788964 (peer_addr_for_me v2:172.21.15.37:0/0) 2024-04-25T10:30:08.352 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:08.349+0000 7fbb91d9b640 1 --2- 172.21.15.37:0/309788964 >> v2:172.21.15.203:3300/0 conn(0x7fbb9406daf0 0x7fbb9406df00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:30:08.352 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:08.349+0000 7fbb9259c640 1 -- 172.21.15.37:0/309788964 >> v2:172.21.15.203:3300/0 conn(0x7fbb9406daf0 msgr2=0x7fbb9406df00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:30:08.352 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:08.349+0000 7fbb9259c640 1 --2- 172.21.15.37:0/309788964 >> v2:172.21.15.203:3300/0 conn(0x7fbb9406daf0 0x7fbb9406df00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:30:08.352 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:08.349+0000 7fbb9259c640 1 -- 172.21.15.37:0/309788964 >> v1:172.21.15.203:6789/0 conn(0x7fbb9406e440 legacy=0x7fbb9408cc90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:30:08.352 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:08.349+0000 7fbb9259c640 1 -- 172.21.15.37:0/309788964 --> v2:172.21.15.37:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fbb940d3120 con 0x7fbb940b2dc0 2024-04-25T10:30:08.353 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:08.350+0000 7fbb9259c640 1 --2- 172.21.15.37:0/309788964 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fbb940b2dc0 0x7fbb940d2860 secure :-1 s=READY pgs=403 cs=0 l=1 rev1=1 crypto rx=0x7fbb94070ab0 tx=0x7fbb8c02a210 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b028ef105e2d4e28 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:30:08.353 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:08.350+0000 7fbb9159a640 1 -- 172.21.15.37:0/309788964 <== mon.0 v2:172.21.15.37:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbb8c004030 con 0x7fbb940b2dc0 2024-04-25T10:30:08.354 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:08.350+0000 7fbb9159a640 1 -- 172.21.15.37:0/309788964 <== mon.0 v2:172.21.15.37:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fbb8c0041d0 con 0x7fbb940b2dc0 2024-04-25T10:30:08.354 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:08.350+0000 7fbb9159a640 1 -- 172.21.15.37:0/309788964 <== mon.0 v2:172.21.15.37:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbb8c003610 con 0x7fbb940b2dc0 2024-04-25T10:30:08.354 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:08.351+0000 7fbb9889f640 1 -- 172.21.15.37:0/309788964 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fbb940b2dc0 msgr2=0x7fbb940d2860 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:30:08.354 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:08.351+0000 7fbb9889f640 1 --2- 172.21.15.37:0/309788964 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fbb940b2dc0 0x7fbb940d2860 secure :-1 s=READY pgs=403 cs=0 l=1 rev1=1 crypto rx=0x7fbb94070ab0 tx=0x7fbb8c02a210 comp rx=0 tx=0).stop 2024-04-25T10:30:08.354 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:08.351+0000 7fbb9889f640 1 -- 172.21.15.37:0/309788964 shutdown_connections 2024-04-25T10:30:08.354 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:08.351+0000 7fbb9889f640 1 --2- 172.21.15.37:0/309788964 >> v2:172.21.15.203:3300/0 conn(0x7fbb9406daf0 0x7fbb9406df00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:30:08.354 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:08.351+0000 7fbb9889f640 1 --2- 172.21.15.37:0/309788964 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fbb940b2dc0 0x7fbb940d2860 unknown :-1 s=CLOSED pgs=403 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:30:08.354 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:08.351+0000 7fbb9889f640 1 -- 172.21.15.37:0/309788964 >> 172.21.15.37:0/309788964 conn(0x7fbb940b5440 msgr2=0x7fbb940b5850 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:30:08.354 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:08.351+0000 7fbb9889f640 1 -- 172.21.15.37:0/309788964 shutdown_connections 2024-04-25T10:30:08.354 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:08.351+0000 7fbb9889f640 1 -- 172.21.15.37:0/309788964 wait complete. 2024-04-25T10:30:08.355 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:08.353+0000 7fbb9889f640 1 Processor -- start 2024-04-25T10:30:08.355 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:08.353+0000 7fbb9889f640 1 -- start start 2024-04-25T10:30:08.357 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:08.354+0000 7fbb9889f640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fbb9406daf0 0x7fbb940b5e10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:30:08.358 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:08.354+0000 7fbb9889f640 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fbb9406e440 0x7fbb9415ece0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:30:08.358 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:08.354+0000 7fbb9889f640 1 --2- >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7fbb940b2dc0 0x7fbb940c8990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:30:08.358 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:08.354+0000 7fbb9889f640 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 -- 0x7fbb9406e850 con 0x7fbb9406daf0 2024-04-25T10:30:08.358 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:08.354+0000 7fbb9889f640 1 -- --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_getmap magic: 0 -- 0x7fbb940d36e0 con 0x7fbb9406e440 2024-04-25T10:30:08.358 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:08.354+0000 7fbb9889f640 1 -- --> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] -- mon_getmap magic: 0 -- 0x7fbb940cf570 con 0x7fbb940b2dc0 2024-04-25T10:30:08.358 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:08.354+0000 7fbb9259c640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fbb9406daf0 0x7fbb940b5e10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:30:08.358 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:08.354+0000 7fbb9259c640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fbb9406daf0 0x7fbb940b5e10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.37:3300/0 says I am v2:172.21.15.37:54778/0 (socket says 172.21.15.37:54778) 2024-04-25T10:30:08.358 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:08.354+0000 7fbb9259c640 1 -- 172.21.15.37:0/381519679 learned_addr learned my addr 172.21.15.37:0/381519679 (peer_addr_for_me v2:172.21.15.37:0/0) 2024-04-25T10:30:08.358 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:08.354+0000 7fbb91d9b640 1 --2- 172.21.15.37:0/381519679 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fbb9406e440 0x7fbb9415ece0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:30:08.358 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:08.354+0000 7fbb92d9d640 1 --2- 172.21.15.37:0/381519679 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7fbb940b2dc0 0x7fbb940c8990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:30:08.358 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:08.354+0000 7fbb9259c640 1 -- 172.21.15.37:0/381519679 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7fbb940b2dc0 msgr2=0x7fbb940c8990 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:30:08.358 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:08.354+0000 7fbb9259c640 1 --2- 172.21.15.37:0/381519679 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7fbb940b2dc0 0x7fbb940c8990 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:30:08.359 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:08.354+0000 7fbb9259c640 1 -- 172.21.15.37:0/381519679 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fbb9406e440 msgr2=0x7fbb9415ece0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:30:08.359 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:08.354+0000 7fbb9259c640 1 --2- 172.21.15.37:0/381519679 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fbb9406e440 0x7fbb9415ece0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:30:08.359 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:08.355+0000 7fbb9259c640 1 -- 172.21.15.37:0/381519679 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fbb940c7480 con 0x7fbb9406daf0 2024-04-25T10:30:08.359 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:08.355+0000 7fbb9259c640 1 --2- 172.21.15.37:0/381519679 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fbb9406daf0 0x7fbb940b5e10 secure :-1 s=READY pgs=404 cs=0 l=1 rev1=1 crypto rx=0x7fbb8c009940 tx=0x7fbb8c0046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:30:08.359 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:08.355+0000 7fbb8affd640 1 -- 172.21.15.37:0/381519679 <== mon.0 v2:172.21.15.37:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbb8c041030 con 0x7fbb9406daf0 2024-04-25T10:30:08.359 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:08.355+0000 7fbb8affd640 1 -- 172.21.15.37:0/381519679 <== mon.0 v2:172.21.15.37:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fbb8c003c50 con 0x7fbb9406daf0 2024-04-25T10:30:08.359 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:08.355+0000 7fbb9889f640 1 -- 172.21.15.37:0/381519679 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fbb940b6350 con 0x7fbb9406daf0 2024-04-25T10:30:08.359 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:08.355+0000 7fbb8affd640 1 -- 172.21.15.37:0/381519679 <== mon.0 v2:172.21.15.37:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbb8c004030 con 0x7fbb9406daf0 2024-04-25T10:30:08.359 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:08.355+0000 7fbb9889f640 1 -- 172.21.15.37:0/381519679 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fbb940c78f0 con 0x7fbb9406daf0 2024-04-25T10:30:08.360 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:08.356+0000 7fbb9889f640 1 -- 172.21.15.37:0/381519679 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fbb9406a140 con 0x7fbb9406daf0 2024-04-25T10:30:08.360 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:08.357+0000 7fbb8affd640 1 -- 172.21.15.37:0/381519679 <== mon.0 v2:172.21.15.37:3300/0 4 ==== mgrmap(e 5) ==== 96782+0+0 (secure 0 0 0) 0x7fbb8c044070 con 0x7fbb9406daf0 2024-04-25T10:30:08.360 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:08.357+0000 7fbb8affd640 1 --2- 172.21.15.37:0/381519679 >> [v2:172.21.15.203:6832/2418205237,v1:172.21.15.203:6833/2418205237] conn(0x7fbb740744b0 0x7fbb740769c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:30:08.360 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:08.357+0000 7fbb8affd640 1 -- 172.21.15.37:0/381519679 <== mon.0 v2:172.21.15.37:3300/0 5 ==== osd_map(32..32 src has 1..32) ==== 4857+0+0 (secure 0 0 0) 0x7fbb8c0b5ad0 con 0x7fbb9406daf0 2024-04-25T10:30:08.360 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:08.357+0000 7fbb91d9b640 1 --2- 172.21.15.37:0/381519679 >> [v2:172.21.15.203:6832/2418205237,v1:172.21.15.203:6833/2418205237] conn(0x7fbb740744b0 0x7fbb740769c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:30:08.361 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:08.358+0000 7fbb91d9b640 1 --2- 172.21.15.37:0/381519679 >> [v2:172.21.15.203:6832/2418205237,v1:172.21.15.203:6833/2418205237] conn(0x7fbb740744b0 0x7fbb740769c0 secure :-1 s=READY pgs=205 cs=0 l=1 rev1=1 crypto rx=0x7fbb7c006fd0 tx=0x7fbb7c008040 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:30:08.364 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:08.362+0000 7fbb8affd640 1 -- 172.21.15.37:0/381519679 <== mon.0 v2:172.21.15.37:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187246 (secure 0 0 0) 0x7fbb8c0805b0 con 0x7fbb9406daf0 2024-04-25T10:30:08.622 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:08.620+0000 7fbb9889f640 1 -- 172.21.15.37:0/381519679 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_command({"prefix": "auth del", "entity": "client.bootstrap-rgw"} v 0) -- 0x7fbb9406a140 con 0x7fbb9406daf0 2024-04-25T10:30:08.625 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:08.623+0000 7fbb8affd640 1 -- 172.21.15.37:0/381519679 <== mon.0 v2:172.21.15.37:3300/0 7 ==== mon_command_ack([{"prefix": "auth del", "entity": "client.bootstrap-rgw"}]=0 v8) ==== 90+0+0 (secure 0 0 0) 0x7fbb8c07feb0 con 0x7fbb9406daf0 2024-04-25T10:30:08.627 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:08.625+0000 7fbb9889f640 1 -- 172.21.15.37:0/381519679 >> [v2:172.21.15.203:6832/2418205237,v1:172.21.15.203:6833/2418205237] conn(0x7fbb740744b0 msgr2=0x7fbb740769c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:30:08.628 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:08.625+0000 7fbb9889f640 1 --2- 172.21.15.37:0/381519679 >> [v2:172.21.15.203:6832/2418205237,v1:172.21.15.203:6833/2418205237] conn(0x7fbb740744b0 0x7fbb740769c0 secure :-1 s=READY pgs=205 cs=0 l=1 rev1=1 crypto rx=0x7fbb7c006fd0 tx=0x7fbb7c008040 comp rx=0 tx=0).stop 2024-04-25T10:30:08.628 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:08.625+0000 7fbb9889f640 1 -- 172.21.15.37:0/381519679 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fbb9406daf0 msgr2=0x7fbb940b5e10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:30:08.628 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:08.625+0000 7fbb9889f640 1 --2- 172.21.15.37:0/381519679 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fbb9406daf0 0x7fbb940b5e10 secure :-1 s=READY pgs=404 cs=0 l=1 rev1=1 crypto rx=0x7fbb8c009940 tx=0x7fbb8c0046c0 comp rx=0 tx=0).stop 2024-04-25T10:30:08.628 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:08.625+0000 7fbb9889f640 1 -- 172.21.15.37:0/381519679 shutdown_connections 2024-04-25T10:30:08.628 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:08.625+0000 7fbb9889f640 1 --2- 172.21.15.37:0/381519679 >> [v2:172.21.15.203:6832/2418205237,v1:172.21.15.203:6833/2418205237] conn(0x7fbb740744b0 0x7fbb740769c0 unknown :-1 s=CLOSED pgs=205 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:30:08.628 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:08.625+0000 7fbb9889f640 1 --2- 172.21.15.37:0/381519679 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7fbb940b2dc0 0x7fbb940c8990 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:30:08.628 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:08.625+0000 7fbb9889f640 1 --2- 172.21.15.37:0/381519679 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fbb9406e440 0x7fbb9415ece0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:30:08.628 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:08.625+0000 7fbb9889f640 1 --2- 172.21.15.37:0/381519679 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fbb9406daf0 0x7fbb940b5e10 unknown :-1 s=CLOSED pgs=404 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:30:08.628 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:08.625+0000 7fbb9889f640 1 -- 172.21.15.37:0/381519679 >> 172.21.15.37:0/381519679 conn(0x7fbb940b5440 msgr2=0x7fbb940cec20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:30:08.628 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:08.625+0000 7fbb9889f640 1 -- 172.21.15.37:0/381519679 shutdown_connections 2024-04-25T10:30:08.628 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:08.625+0000 7fbb9889f640 1 -- 172.21.15.37:0/381519679 wait complete. 2024-04-25T10:30:08.639 DEBUG:teuthology.orchestra.run.smithi037:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-04-25T10:30:08.750 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:08.748+0000 7f06d98aa640 1 Processor -- start 2024-04-25T10:30:08.751 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:08.748+0000 7f06d98aa640 1 -- start start 2024-04-25T10:30:08.752 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:08.749+0000 7f06d98aa640 1 --2- >> v2:172.21.15.37:3300/0 conn(0x7f06d415a9f0 0x7f06d406c540 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:30:08.752 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:08.749+0000 7f06d98aa640 1 --2- >> v2:172.21.15.203:3300/0 conn(0x7f06d406ca80 0x7f06d4160910 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:30:08.752 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:08.749+0000 7f06d98aa640 1 -- --> v1:172.21.15.203:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f06d406ce90 con 0x7f06d4160e50 2024-04-25T10:30:08.752 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:08.749+0000 7f06d98aa640 1 -- --> v2:172.21.15.37:3300/0 -- mon_getmap magic: 0 -- 0x7f06d40b6870 con 0x7f06d415a9f0 2024-04-25T10:30:08.752 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:08.749+0000 7f06d98aa640 1 -- --> v2:172.21.15.203:3300/0 -- mon_getmap magic: 0 -- 0x7f06d41616d0 con 0x7f06d406ca80 2024-04-25T10:30:08.753 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:08.750+0000 7f06d2ffd640 1 --2- >> v2:172.21.15.37:3300/0 conn(0x7f06d415a9f0 0x7f06d406c540 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:30:08.753 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:08.750+0000 7f06d2ffd640 1 --2- >> v2:172.21.15.37:3300/0 conn(0x7f06d415a9f0 0x7f06d406c540 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.37:3300/0 says I am v2:172.21.15.37:54782/0 (socket says 172.21.15.37:54782) 2024-04-25T10:30:08.753 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:08.750+0000 7f06d2ffd640 1 -- 172.21.15.37:0/2841107881 learned_addr learned my addr 172.21.15.37:0/2841107881 (peer_addr_for_me v2:172.21.15.37:0/0) 2024-04-25T10:30:08.753 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:08.750+0000 7f06d27fc640 1 --2- 172.21.15.37:0/2841107881 >> v2:172.21.15.203:3300/0 conn(0x7f06d406ca80 0x7f06d4160910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:30:08.753 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:08.750+0000 7f06d2ffd640 1 -- 172.21.15.37:0/2841107881 >> v2:172.21.15.203:3300/0 conn(0x7f06d406ca80 msgr2=0x7f06d4160910 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:30:08.753 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:08.750+0000 7f06d2ffd640 1 --2- 172.21.15.37:0/2841107881 >> v2:172.21.15.203:3300/0 conn(0x7f06d406ca80 0x7f06d4160910 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:30:08.753 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:08.750+0000 7f06d2ffd640 1 -- 172.21.15.37:0/2841107881 >> v1:172.21.15.203:6789/0 conn(0x7f06d4160e50 legacy=0x7f06d4145a30 unknown :-1 s=STATE_CONNECTING_RE l=0).mark_down 2024-04-25T10:30:08.753 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:08.750+0000 7f06d2ffd640 1 -- 172.21.15.37:0/2841107881 --> v2:172.21.15.37:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f06d4163e00 con 0x7f06d415a9f0 2024-04-25T10:30:08.754 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:08.751+0000 7f06d2ffd640 1 --2- 172.21.15.37:0/2841107881 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f06d415a9f0 0x7f06d406c540 secure :-1 s=READY pgs=405 cs=0 l=1 rev1=1 crypto rx=0x7f06d40b2300 tx=0x7f06cc02a1a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=9adb72ba0f26c227 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:30:08.754 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:08.751+0000 7f06d1ffb640 1 -- 172.21.15.37:0/2841107881 <== mon.0 v2:172.21.15.37:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f06cc037070 con 0x7f06d415a9f0 2024-04-25T10:30:08.755 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:08.751+0000 7f06d1ffb640 1 -- 172.21.15.37:0/2841107881 <== mon.0 v2:172.21.15.37:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f06cc032440 con 0x7f06d415a9f0 2024-04-25T10:30:08.755 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:08.751+0000 7f06d1ffb640 1 -- 172.21.15.37:0/2841107881 <== mon.0 v2:172.21.15.37:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f06cc032810 con 0x7f06d415a9f0 2024-04-25T10:30:08.755 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:08.751+0000 7f06d98aa640 1 -- 172.21.15.37:0/2841107881 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f06d415a9f0 msgr2=0x7f06d406c540 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:30:08.755 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:08.751+0000 7f06d98aa640 1 --2- 172.21.15.37:0/2841107881 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f06d415a9f0 0x7f06d406c540 secure :-1 s=READY pgs=405 cs=0 l=1 rev1=1 crypto rx=0x7f06d40b2300 tx=0x7f06cc02a1a0 comp rx=0 tx=0).stop 2024-04-25T10:30:08.755 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:08.751+0000 7f06d98aa640 1 -- 172.21.15.37:0/2841107881 shutdown_connections 2024-04-25T10:30:08.755 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:08.751+0000 7f06d98aa640 1 --2- 172.21.15.37:0/2841107881 >> v2:172.21.15.203:3300/0 conn(0x7f06d406ca80 0x7f06d4160910 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:30:08.755 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:08.751+0000 7f06d98aa640 1 --2- 172.21.15.37:0/2841107881 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f06d415a9f0 0x7f06d406c540 unknown :-1 s=CLOSED pgs=405 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:30:08.755 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:08.751+0000 7f06d98aa640 1 -- 172.21.15.37:0/2841107881 >> 172.21.15.37:0/2841107881 conn(0x7f06d40b4560 msgr2=0x7f06d40b4970 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:30:08.755 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:08.752+0000 7f06d98aa640 1 -- 172.21.15.37:0/2841107881 shutdown_connections 2024-04-25T10:30:08.755 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:08.752+0000 7f06d98aa640 1 -- 172.21.15.37:0/2841107881 wait complete. 2024-04-25T10:30:08.756 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:08.753+0000 7f06d98aa640 1 Processor -- start 2024-04-25T10:30:08.756 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:08.753+0000 7f06d98aa640 1 -- start start 2024-04-25T10:30:08.758 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:08.755+0000 7f06d98aa640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f06d406ca80 0x7f06d4140730 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:30:08.758 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:08.755+0000 7f06d98aa640 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f06d415a9f0 0x7f06d41f5eb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:30:08.758 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:08.755+0000 7f06d98aa640 1 --2- >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f06d4160e50 0x7f06d41eb490 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:30:08.758 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:08.755+0000 7f06d98aa640 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 -- 0x7f06d41646a0 con 0x7f06d406ca80 2024-04-25T10:30:08.758 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:08.755+0000 7f06d98aa640 1 -- --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_getmap magic: 0 -- 0x7f06d41643c0 con 0x7f06d415a9f0 2024-04-25T10:30:08.759 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:08.755+0000 7f06d98aa640 1 -- --> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] -- mon_getmap magic: 0 -- 0x7f06d4164bc0 con 0x7f06d4160e50 2024-04-25T10:30:08.759 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:08.755+0000 7f06d2ffd640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f06d406ca80 0x7f06d4140730 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:30:08.759 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:08.755+0000 7f06d2ffd640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f06d406ca80 0x7f06d4140730 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.37:3300/0 says I am v2:172.21.15.37:54786/0 (socket says 172.21.15.37:54786) 2024-04-25T10:30:08.759 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:08.755+0000 7f06d2ffd640 1 -- 172.21.15.37:0/1078454652 learned_addr learned my addr 172.21.15.37:0/1078454652 (peer_addr_for_me v2:172.21.15.37:0/0) 2024-04-25T10:30:08.759 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:08.755+0000 7f06d27fc640 1 --2- 172.21.15.37:0/1078454652 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f06d415a9f0 0x7f06d41f5eb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:30:08.759 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:08.755+0000 7f06d2ffd640 1 -- 172.21.15.37:0/1078454652 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f06d4160e50 msgr2=0x7f06d41eb490 unknown :-1 s=STATE_CONNECTING_RE l=1).mark_down 2024-04-25T10:30:08.759 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:08.755+0000 7f06d2ffd640 1 --2- 172.21.15.37:0/1078454652 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f06d4160e50 0x7f06d41eb490 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:30:08.759 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:08.755+0000 7f06d2ffd640 1 -- 172.21.15.37:0/1078454652 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f06d415a9f0 msgr2=0x7f06d41f5eb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:30:08.759 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:08.755+0000 7f06d2ffd640 1 --2- 172.21.15.37:0/1078454652 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f06d415a9f0 0x7f06d41f5eb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:30:08.759 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:08.755+0000 7f06d2ffd640 1 -- 172.21.15.37:0/1078454652 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f06d41f9c00 con 0x7f06d406ca80 2024-04-25T10:30:08.759 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:08.755+0000 7f06d27fc640 1 --2- 172.21.15.37:0/1078454652 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f06d415a9f0 0x7f06d41f5eb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T10:30:08.759 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:08.755+0000 7f06d2ffd640 1 --2- 172.21.15.37:0/1078454652 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f06d406ca80 0x7f06d4140730 secure :-1 s=READY pgs=406 cs=0 l=1 rev1=1 crypto rx=0x7f06cc000c00 tx=0x7f06cc0046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:30:08.759 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:08.755+0000 7f06bbfff640 1 -- 172.21.15.37:0/1078454652 <== mon.0 v2:172.21.15.37:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f06cc041070 con 0x7f06d406ca80 2024-04-25T10:30:08.760 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:08.756+0000 7f06bbfff640 1 -- 172.21.15.37:0/1078454652 <== mon.0 v2:172.21.15.37:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f06cc03c870 con 0x7f06d406ca80 2024-04-25T10:30:08.760 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:08.756+0000 7f06d98aa640 1 -- 172.21.15.37:0/1078454652 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f06d412e9c0 con 0x7f06d406ca80 2024-04-25T10:30:08.760 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:08.756+0000 7f06bbfff640 1 -- 172.21.15.37:0/1078454652 <== mon.0 v2:172.21.15.37:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f06cc037040 con 0x7f06d406ca80 2024-04-25T10:30:08.760 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:08.756+0000 7f06d98aa640 1 -- 172.21.15.37:0/1078454652 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f06d41fa100 con 0x7f06d406ca80 2024-04-25T10:30:08.761 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:08.757+0000 7f06d98aa640 1 -- 172.21.15.37:0/1078454652 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f06d406a140 con 0x7f06d406ca80 2024-04-25T10:30:08.761 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:08.757+0000 7f06bbfff640 1 -- 172.21.15.37:0/1078454652 <== mon.0 v2:172.21.15.37:3300/0 4 ==== mgrmap(e 5) ==== 96782+0+0 (secure 0 0 0) 0x7f06cc044070 con 0x7f06d406ca80 2024-04-25T10:30:08.761 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:08.758+0000 7f06bbfff640 1 --2- 172.21.15.37:0/1078454652 >> [v2:172.21.15.203:6832/2418205237,v1:172.21.15.203:6833/2418205237] conn(0x7f06c00744e0 0x7f06c00769f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:30:08.761 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:08.758+0000 7f06bbfff640 1 -- 172.21.15.37:0/1078454652 <== mon.0 v2:172.21.15.37:3300/0 5 ==== osd_map(32..32 src has 1..32) ==== 4857+0+0 (secure 0 0 0) 0x7f06cc0b5990 con 0x7f06d406ca80 2024-04-25T10:30:08.761 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:08.759+0000 7f06d27fc640 1 --2- 172.21.15.37:0/1078454652 >> [v2:172.21.15.203:6832/2418205237,v1:172.21.15.203:6833/2418205237] conn(0x7f06c00744e0 0x7f06c00769f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:30:08.762 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:08.761+0000 7f06d27fc640 1 --2- 172.21.15.37:0/1078454652 >> [v2:172.21.15.203:6832/2418205237,v1:172.21.15.203:6833/2418205237] conn(0x7f06c00744e0 0x7f06c00769f0 secure :-1 s=READY pgs=206 cs=0 l=1 rev1=1 crypto rx=0x7f06c8006fd0 tx=0x7f06c8008040 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:30:08.764 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:08.762+0000 7f06bbfff640 1 -- 172.21.15.37:0/1078454652 <== mon.0 v2:172.21.15.37:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187246 (secure 0 0 0) 0x7f06cc080480 con 0x7f06d406ca80 2024-04-25T10:30:08.949 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:08.947+0000 7f06d98aa640 1 -- 172.21.15.37:0/1078454652 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7f06d4145810 con 0x7f06d406ca80 2024-04-25T10:30:09.035 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:08.948+0000 7f06bbfff640 1 -- 172.21.15.37:0/1078454652 <== mon.0 v2:172.21.15.37:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v32) ==== 74+0+12903 (secure 0 0 0) 0x7f06cc07fd80 con 0x7f06d406ca80 2024-04-25T10:30:09.035 INFO:teuthology.orchestra.run.smithi037.stdout: 2024-04-25T10:30:09.036 INFO:teuthology.orchestra.run.smithi037.stdout:{"epoch":32,"fsid":"f4acda28-be34-492a-b286-994afd36f61d","created":"2024-04-25T10:28:43.985718+0000","modified":"2024-04-25T10:30:05.739507+0000","last_up_change":"2024-04-25T10:28:54.900654+0000","last_in_change":"2024-04-25T10:28:46.216973+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":3,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-04-25T10:28:56.202153+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"19","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"a2f79028-b332-4166-9ebb-a7024a37ea22","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":21,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.37:6816","nonce":2746074720},{"type":"v1","addr":"172.21.15.37:6817","nonce":2746074720}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.37:6818","nonce":2746074720},{"type":"v1","addr":"172.21.15.37:6819","nonce":2746074720}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.37:6822","nonce":2746074720},{"type":"v1","addr":"172.21.15.37:6823","nonce":2746074720}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.37:6820","nonce":2746074720},{"type":"v1","addr":"172.21.15.37:6821","nonce":2746074720}]},"public_addr":"172.21.15.37:6817/2746074720","cluster_addr":"172.21.15.37:6819/2746074720","heartbeat_back_addr":"172.21.15.37:6823/2746074720","heartbeat_front_addr":"172.21.15.37:6821/2746074720","state":["exists","up"]},{"osd":1,"uuid":"973159de-c2ea-4083-aecc-69373d1c1132","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":21,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.37:6800","nonce":1690341662},{"type":"v1","addr":"172.21.15.37:6801","nonce":1690341662}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.37:6802","nonce":1690341662},{"type":"v1","addr":"172.21.15.37:6803","nonce":1690341662}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.37:6806","nonce":1690341662},{"type":"v1","addr":"172.21.15.37:6807","nonce":1690341662}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.37:6804","nonce":1690341662},{"type":"v1","addr":"172.21.15.37:6805","nonce":1690341662}]},"public_addr":"172.21.15.37:6801/1690341662","cluster_addr":"172.21.15.37:6803/1690341662","heartbeat_back_addr":"172.21.15.37:6807/1690341662","heartbeat_front_addr":"172.21.15.37:6805/1690341662","state":["exists","up"]},{"osd":2,"uuid":"4ae7a0e9-cb3f-484c-a75f-f3766b721c47","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":21,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.37:6808","nonce":2754669649},{"type":"v1","addr":"172.21.15.37:6809","nonce":2754669649}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.37:6810","nonce":2754669649},{"type":"v1","addr":"172.21.15.37:6811","nonce":2754669649}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.37:6814","nonce":2754669649},{"type":"v1","addr":"172.21.15.37:6815","nonce":2754669649}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.37:6812","nonce":2754669649},{"type":"v1","addr":"172.21.15.37:6813","nonce":2754669649}]},"public_addr":"172.21.15.37:6809/2754669649","cluster_addr":"172.21.15.37:6811/2754669649","heartbeat_back_addr":"172.21.15.37:6815/2754669649","heartbeat_front_addr":"172.21.15.37:6813/2754669649","state":["exists","up"]},{"osd":3,"uuid":"1ba4a746-1608-47a7-ace2-960a5cfae934","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":21,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.37:6824","nonce":1815290618},{"type":"v1","addr":"172.21.15.37:6825","nonce":1815290618}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.37:6826","nonce":1815290618},{"type":"v1","addr":"172.21.15.37:6827","nonce":1815290618}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.37:6830","nonce":1815290618},{"type":"v1","addr":"172.21.15.37:6831","nonce":1815290618}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.37:6828","nonce":1815290618},{"type":"v1","addr":"172.21.15.37:6829","nonce":1815290618}]},"public_addr":"172.21.15.37:6825/1815290618","cluster_addr":"172.21.15.37:6827/1815290618","heartbeat_back_addr":"172.21.15.37:6831/1815290618","heartbeat_front_addr":"172.21.15.37:6829/1815290618","state":["exists","up"]},{"osd":4,"uuid":"d4f59f5d-0413-4b79-87ee-bb212c1fcf62","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":21,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6800","nonce":4051532063},{"type":"v1","addr":"172.21.15.203:6801","nonce":4051532063}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6802","nonce":4051532063},{"type":"v1","addr":"172.21.15.203:6803","nonce":4051532063}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6806","nonce":4051532063},{"type":"v1","addr":"172.21.15.203:6807","nonce":4051532063}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6804","nonce":4051532063},{"type":"v1","addr":"172.21.15.203:6805","nonce":4051532063}]},"public_addr":"172.21.15.203:6801/4051532063","cluster_addr":"172.21.15.203:6803/4051532063","heartbeat_back_addr":"172.21.15.203:6807/4051532063","heartbeat_front_addr":"172.21.15.203:6805/4051532063","state":["exists","up"]},{"osd":5,"uuid":"69710a0e-41d8-4e7f-9a10-17ce540b67be","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":24,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6824","nonce":1203822890},{"type":"v1","addr":"172.21.15.203:6825","nonce":1203822890}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6826","nonce":1203822890},{"type":"v1","addr":"172.21.15.203:6827","nonce":1203822890}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6830","nonce":1203822890},{"type":"v1","addr":"172.21.15.203:6831","nonce":1203822890}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6828","nonce":1203822890},{"type":"v1","addr":"172.21.15.203:6829","nonce":1203822890}]},"public_addr":"172.21.15.203:6825/1203822890","cluster_addr":"172.21.15.203:6827/1203822890","heartbeat_back_addr":"172.21.15.203:6831/1203822890","heartbeat_front_addr":"172.21.15.203:6829/1203822890","state":["exists","up"]},{"osd":6,"uuid":"f29dd047-3b48-473d-a235-6642a6293f20","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":21,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6810","nonce":2309615014},{"type":"v1","addr":"172.21.15.203:6812","nonce":2309615014}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6814","nonce":2309615014},{"type":"v1","addr":"172.21.15.203:6816","nonce":2309615014}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6822","nonce":2309615014},{"type":"v1","addr":"172.21.15.203:6823","nonce":2309615014}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6818","nonce":2309615014},{"type":"v1","addr":"172.21.15.203:6820","nonce":2309615014}]},"public_addr":"172.21.15.203:6812/2309615014","cluster_addr":"172.21.15.203:6816/2309615014","heartbeat_back_addr":"172.21.15.203:6823/2309615014","heartbeat_front_addr":"172.21.15.203:6820/2309615014","state":["exists","up"]},{"osd":7,"uuid":"6e691644-378b-4fdc-8e86-fb5f56e31f29","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":21,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6808","nonce":3688099014},{"type":"v1","addr":"172.21.15.203:6809","nonce":3688099014}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6811","nonce":3688099014},{"type":"v1","addr":"172.21.15.203:6813","nonce":3688099014}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6819","nonce":3688099014},{"type":"v1","addr":"172.21.15.203:6821","nonce":3688099014}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6815","nonce":3688099014},{"type":"v1","addr":"172.21.15.203:6817","nonce":3688099014}]},"public_addr":"172.21.15.203:6809/3688099014","cluster_addr":"172.21.15.203:6813/3688099014","heartbeat_back_addr":"172.21.15.203:6821/3688099014","heartbeat_front_addr":"172.21.15.203:6817/3688099014","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T10:28:51.419141+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T10:28:51.366448+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T10:28:51.348276+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T10:28:51.380519+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T10:28:51.105812+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T10:28:51.384496+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T10:28:51.388755+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T10:28:51.366958+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-04-25T10:30:09.037 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:08.950+0000 7f06d98aa640 1 -- 172.21.15.37:0/1078454652 >> [v2:172.21.15.203:6832/2418205237,v1:172.21.15.203:6833/2418205237] conn(0x7f06c00744e0 msgr2=0x7f06c00769f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:30:09.037 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:08.950+0000 7f06d98aa640 1 --2- 172.21.15.37:0/1078454652 >> [v2:172.21.15.203:6832/2418205237,v1:172.21.15.203:6833/2418205237] conn(0x7f06c00744e0 0x7f06c00769f0 secure :-1 s=READY pgs=206 cs=0 l=1 rev1=1 crypto rx=0x7f06c8006fd0 tx=0x7f06c8008040 comp rx=0 tx=0).stop 2024-04-25T10:30:09.037 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:08.950+0000 7f06d98aa640 1 -- 172.21.15.37:0/1078454652 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f06d406ca80 msgr2=0x7f06d4140730 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:30:09.038 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:08.950+0000 7f06d98aa640 1 --2- 172.21.15.37:0/1078454652 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f06d406ca80 0x7f06d4140730 secure :-1 s=READY pgs=406 cs=0 l=1 rev1=1 crypto rx=0x7f06cc000c00 tx=0x7f06cc0046c0 comp rx=0 tx=0).stop 2024-04-25T10:30:09.038 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:08.950+0000 7f06d98aa640 1 -- 172.21.15.37:0/1078454652 shutdown_connections 2024-04-25T10:30:09.038 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:08.950+0000 7f06d98aa640 1 --2- 172.21.15.37:0/1078454652 >> [v2:172.21.15.203:6832/2418205237,v1:172.21.15.203:6833/2418205237] conn(0x7f06c00744e0 0x7f06c00769f0 unknown :-1 s=CLOSED pgs=206 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:30:09.038 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:08.950+0000 7f06d98aa640 1 --2- 172.21.15.37:0/1078454652 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f06d4160e50 0x7f06d41eb490 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:30:09.038 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:08.950+0000 7f06d98aa640 1 --2- 172.21.15.37:0/1078454652 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f06d415a9f0 0x7f06d41f5eb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:30:09.038 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:08.950+0000 7f06d98aa640 1 --2- 172.21.15.37:0/1078454652 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f06d406ca80 0x7f06d4140730 unknown :-1 s=CLOSED pgs=406 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:30:09.038 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:08.950+0000 7f06d98aa640 1 -- 172.21.15.37:0/1078454652 >> 172.21.15.37:0/1078454652 conn(0x7f06d40b4560 msgr2=0x7f06d40b4970 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:30:09.038 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:08.950+0000 7f06d98aa640 1 -- 172.21.15.37:0/1078454652 shutdown_connections 2024-04-25T10:30:09.038 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:08.950+0000 7f06d98aa640 1 -- 172.21.15.37:0/1078454652 wait complete. 2024-04-25T10:30:09.039 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2024-04-25T10:28:56.202153+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '19', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 8, 'score_stable': 8, 'optimal_score': 0.25, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}] 2024-04-25T10:30:09.039 DEBUG:teuthology.orchestra.run.smithi037:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2024-04-25T10:30:09.153 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:09.151+0000 7fee5d334640 1 Processor -- start 2024-04-25T10:30:09.153 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:09.151+0000 7fee5d334640 1 -- start start 2024-04-25T10:30:09.157 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:09.152+0000 7fee5d334640 1 --2- >> v2:172.21.15.203:3300/0 conn(0x7fee580d3230 0x7fee5816b0c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:30:09.157 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:09.152+0000 7fee5d334640 1 -- --> v1:172.21.15.37:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fee580d10a0 con 0x7fee580d5160 2024-04-25T10:30:09.157 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:09.152+0000 7fee5d334640 1 -- --> v1:172.21.15.203:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fee580d1370 con 0x7fee580b4e80 2024-04-25T10:30:09.157 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:09.152+0000 7fee5d334640 1 -- --> v2:172.21.15.203:3300/0 -- mon_getmap magic: 0 -- 0x7fee580d1650 con 0x7fee580d3230 2024-04-25T10:30:09.157 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:09.153+0000 7fee56ffd640 1 --1- >> v1:172.21.15.37:6789/0 conn(0x7fee580d5160 0x7fee580d0990 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.37:6789/0 says I am v1:172.21.15.37:60742/0 (socket says 172.21.15.37:60742) 2024-04-25T10:30:09.157 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:09.153+0000 7fee56ffd640 1 -- 172.21.15.37:0/3127134920 learned_addr learned my addr 172.21.15.37:0/3127134920 (peer_addr_for_me v1:172.21.15.37:0/0) 2024-04-25T10:30:09.157 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:09.153+0000 7fee55ffb640 1 -- 172.21.15.37:0/3127134920 <== mon.0 v1:172.21.15.37:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 248936755 0 0) 0x7fee580d10a0 con 0x7fee580d5160 2024-04-25T10:30:09.157 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:09.153+0000 7fee55ffb640 1 -- 172.21.15.37:0/3127134920 --> v1:172.21.15.37:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7fee40003520 con 0x7fee580d5160 2024-04-25T10:30:09.157 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:09.153+0000 7fee577fe640 1 --2- 172.21.15.37:0/3127134920 >> v2:172.21.15.203:3300/0 conn(0x7fee580d3230 0x7fee5816b0c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:30:09.157 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:09.153+0000 7fee55ffb640 1 -- 172.21.15.37:0/3127134920 <== mon.0 v1:172.21.15.37:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 3683706524 0 0) 0x7fee40003520 con 0x7fee580d5160 2024-04-25T10:30:09.158 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:09.153+0000 7fee55ffb640 1 -- 172.21.15.37:0/3127134920 >> v2:172.21.15.203:3300/0 conn(0x7fee580d3230 msgr2=0x7fee5816b0c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:30:09.158 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:09.153+0000 7fee55ffb640 1 --2- 172.21.15.37:0/3127134920 >> v2:172.21.15.203:3300/0 conn(0x7fee580d3230 0x7fee5816b0c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:30:09.158 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:09.153+0000 7fee55ffb640 1 -- 172.21.15.37:0/3127134920 >> v1:172.21.15.203:6789/0 conn(0x7fee580b4e80 legacy=0x7fee580d2b20 unknown :-1 s=STATE_CONNECTING_RE l=0).mark_down 2024-04-25T10:30:09.158 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:09.153+0000 7fee55ffb640 1 -- 172.21.15.37:0/3127134920 --> v1:172.21.15.37:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fee580d4680 con 0x7fee580d5160 2024-04-25T10:30:09.158 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:09.153+0000 7fee55ffb640 1 -- 172.21.15.37:0/3127134920 <== mon.0 v1:172.21.15.37:6789/0 3 ==== mon_map magic: 0 ==== 413+0+0 (unknown 1577794600 0 0) 0x7fee50001690 con 0x7fee580d5160 2024-04-25T10:30:09.158 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:09.153+0000 7fee55ffb640 1 -- 172.21.15.37:0/3127134920 >> v1:172.21.15.37:6789/0 conn(0x7fee580d5160 legacy=0x7fee580d0990 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:30:09.158 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:09.153+0000 7fee55ffb640 1 --2- 172.21.15.37:0/3127134920 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fee400039b0 0x7fee40003e00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:30:09.158 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:09.153+0000 7fee55ffb640 1 --2- 172.21.15.37:0/3127134920 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fee40004800 0x7fee40004c10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:30:09.158 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:09.153+0000 7fee55ffb640 1 --2- 172.21.15.37:0/3127134920 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7fee40005270 0x7fee4000b9c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:30:09.158 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:09.153+0000 7fee55ffb640 1 -- 172.21.15.37:0/3127134920 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 -- 0x7fee580d1650 con 0x7fee40004800 2024-04-25T10:30:09.158 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:09.153+0000 7fee55ffb640 1 -- 172.21.15.37:0/3127134920 --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_getmap magic: 0 -- 0x7fee4000c1c0 con 0x7fee400039b0 2024-04-25T10:30:09.158 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:09.153+0000 7fee55ffb640 1 -- 172.21.15.37:0/3127134920 --> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] -- mon_getmap magic: 0 -- 0x7fee4000c3c0 con 0x7fee40005270 2024-04-25T10:30:09.158 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:09.153+0000 7fee567fc640 1 --2- 172.21.15.37:0/3127134920 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fee40004800 0x7fee40004c10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:30:09.159 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:09.154+0000 7fee577fe640 1 --2- 172.21.15.37:0/3127134920 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7fee40005270 0x7fee4000b9c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:30:09.159 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:09.154+0000 7fee567fc640 1 -- 172.21.15.37:0/3127134920 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7fee40005270 msgr2=0x7fee4000b9c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:30:09.159 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:09.154+0000 7fee567fc640 1 --2- 172.21.15.37:0/3127134920 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7fee40005270 0x7fee4000b9c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:30:09.159 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:09.154+0000 7fee567fc640 1 -- 172.21.15.37:0/3127134920 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fee400039b0 msgr2=0x7fee40003e00 unknown :-1 s=STATE_CONNECTING_RE l=0).mark_down 2024-04-25T10:30:09.159 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:09.154+0000 7fee567fc640 1 --2- 172.21.15.37:0/3127134920 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fee400039b0 0x7fee40003e00 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:30:09.159 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:09.154+0000 7fee567fc640 1 -- 172.21.15.37:0/3127134920 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fee40003520 con 0x7fee40004800 2024-04-25T10:30:09.159 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:09.154+0000 7fee577fe640 1 -- 172.21.15.37:0/3127134920 reap_dead start 2024-04-25T10:30:09.159 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:09.154+0000 7fee567fc640 1 --2- 172.21.15.37:0/3127134920 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fee40004800 0x7fee40004c10 secure :-1 s=READY pgs=408 cs=0 l=1 rev1=1 crypto rx=0x7fee4c009860 tx=0x7fee40005830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=3ec027ea77f94cf3 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:30:09.159 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:09.155+0000 7fee55ffb640 1 -- 172.21.15.37:0/3127134920 <== mon.0 v2:172.21.15.37:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fee4c037070 con 0x7fee40004800 2024-04-25T10:30:09.159 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:09.155+0000 7fee55ffb640 1 -- 172.21.15.37:0/3127134920 <== mon.0 v2:172.21.15.37:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fee4c02a9e0 con 0x7fee40004800 2024-04-25T10:30:09.159 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:09.155+0000 7fee55ffb640 1 -- 172.21.15.37:0/3127134920 <== mon.0 v2:172.21.15.37:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fee4c0093b0 con 0x7fee40004800 2024-04-25T10:30:09.162 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:09.155+0000 7fee5d334640 1 -- 172.21.15.37:0/3127134920 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fee40004800 msgr2=0x7fee40004c10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:30:09.162 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:09.155+0000 7fee5d334640 1 --2- 172.21.15.37:0/3127134920 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fee40004800 0x7fee40004c10 secure :-1 s=READY pgs=408 cs=0 l=1 rev1=1 crypto rx=0x7fee4c009860 tx=0x7fee40005830 comp rx=0 tx=0).stop 2024-04-25T10:30:09.162 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:09.155+0000 7fee5d334640 1 -- 172.21.15.37:0/3127134920 shutdown_connections 2024-04-25T10:30:09.162 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:09.155+0000 7fee5d334640 1 --2- 172.21.15.37:0/3127134920 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fee40004800 0x7fee40004c10 unknown :-1 s=CLOSED pgs=408 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:30:09.162 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:09.155+0000 7fee5d334640 1 -- 172.21.15.37:0/3127134920 >> 172.21.15.37:0/3127134920 conn(0x7fee580b7a80 msgr2=0x7fee580b7e90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:30:09.162 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:09.155+0000 7fee5d334640 1 -- 172.21.15.37:0/3127134920 shutdown_connections 2024-04-25T10:30:09.162 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:09.155+0000 7fee5d334640 1 -- 172.21.15.37:0/3127134920 wait complete. 2024-04-25T10:30:09.162 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:09.157+0000 7fee5d334640 1 Processor -- start 2024-04-25T10:30:09.162 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:09.157+0000 7fee5d334640 1 -- start start 2024-04-25T10:30:09.162 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:09.159+0000 7fee5d334640 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fee40004800 0x7fee58162e60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:30:09.162 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:09.159+0000 7fee5d334640 1 --2- >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7fee580c9680 0x7fee581633a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:30:09.163 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:09.159+0000 7fee5d334640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fee58147fc0 0x7fee581483d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:30:09.163 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:09.159+0000 7fee5d334640 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 -- 0x7fee580d4c40 con 0x7fee58147fc0 2024-04-25T10:30:09.163 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:09.159+0000 7fee5d334640 1 -- --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_getmap magic: 0 -- 0x7fee580b5290 con 0x7fee40004800 2024-04-25T10:30:09.163 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:09.159+0000 7fee5d334640 1 -- --> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] -- mon_getmap magic: 0 -- 0x7fee58164ac0 con 0x7fee580c9680 2024-04-25T10:30:09.163 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:09.159+0000 7fee577fe640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fee58147fc0 0x7fee581483d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:30:09.163 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:09.159+0000 7fee577fe640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fee58147fc0 0x7fee581483d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.37:3300/0 says I am v2:172.21.15.37:54802/0 (socket says 172.21.15.37:54802) 2024-04-25T10:30:09.163 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:09.159+0000 7fee577fe640 1 -- 172.21.15.37:0/992740706 learned_addr learned my addr 172.21.15.37:0/992740706 (peer_addr_for_me v2:172.21.15.37:0/0) 2024-04-25T10:30:09.163 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:09.159+0000 7fee577fe640 1 -- 172.21.15.37:0/992740706 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7fee580c9680 msgr2=0x7fee581633a0 unknown :-1 s=STATE_CONNECTING_RE l=1).mark_down 2024-04-25T10:30:09.163 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:09.159+0000 7fee577fe640 1 --2- 172.21.15.37:0/992740706 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7fee580c9680 0x7fee581633a0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:30:09.163 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:09.159+0000 7fee577fe640 1 -- 172.21.15.37:0/992740706 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fee40004800 msgr2=0x7fee58162e60 unknown :-1 s=STATE_CONNECTING_RE l=1).mark_down 2024-04-25T10:30:09.163 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:09.159+0000 7fee577fe640 1 --2- 172.21.15.37:0/992740706 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fee40004800 0x7fee58162e60 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:30:09.163 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:09.159+0000 7fee577fe640 1 -- 172.21.15.37:0/992740706 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fee58159880 con 0x7fee58147fc0 2024-04-25T10:30:09.163 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:09.159+0000 7fee577fe640 1 --2- 172.21.15.37:0/992740706 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fee58147fc0 0x7fee581483d0 secure :-1 s=READY pgs=409 cs=0 l=1 rev1=1 crypto rx=0x7fee580ba070 tx=0x7fee4800f580 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:30:09.163 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:09.159+0000 7fee3f7fe640 1 -- 172.21.15.37:0/992740706 <== mon.0 v2:172.21.15.37:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fee4800fde0 con 0x7fee58147fc0 2024-04-25T10:30:09.164 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:09.159+0000 7fee3f7fe640 1 -- 172.21.15.37:0/992740706 <== mon.0 v2:172.21.15.37:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fee4801d070 con 0x7fee58147fc0 2024-04-25T10:30:09.164 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:09.159+0000 7fee5d334640 1 -- 172.21.15.37:0/992740706 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fee58159b60 con 0x7fee58147fc0 2024-04-25T10:30:09.164 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:09.159+0000 7fee3f7fe640 1 -- 172.21.15.37:0/992740706 <== mon.0 v2:172.21.15.37:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fee48015670 con 0x7fee58147fc0 2024-04-25T10:30:09.164 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:09.160+0000 7fee5d334640 1 -- 172.21.15.37:0/992740706 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fee5815a120 con 0x7fee58147fc0 2024-04-25T10:30:09.165 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:09.161+0000 7fee5d334640 1 -- 172.21.15.37:0/992740706 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fee5806a140 con 0x7fee58147fc0 2024-04-25T10:30:09.165 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:09.161+0000 7fee3f7fe640 1 -- 172.21.15.37:0/992740706 <== mon.0 v2:172.21.15.37:3300/0 4 ==== mgrmap(e 5) ==== 96782+0+0 (secure 0 0 0) 0x7fee48015810 con 0x7fee58147fc0 2024-04-25T10:30:09.165 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:09.162+0000 7fee3f7fe640 1 --2- 172.21.15.37:0/992740706 >> [v2:172.21.15.203:6832/2418205237,v1:172.21.15.203:6833/2418205237] conn(0x7fee440744b0 0x7fee440769c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:30:09.165 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:09.162+0000 7fee3f7fe640 1 -- 172.21.15.37:0/992740706 <== mon.0 v2:172.21.15.37:3300/0 5 ==== osd_map(32..32 src has 1..32) ==== 4857+0+0 (secure 0 0 0) 0x7fee48095910 con 0x7fee58147fc0 2024-04-25T10:30:09.165 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:09.162+0000 7fee56ffd640 1 --2- 172.21.15.37:0/992740706 >> [v2:172.21.15.203:6832/2418205237,v1:172.21.15.203:6833/2418205237] conn(0x7fee440744b0 0x7fee440769c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:30:09.165 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:09.163+0000 7fee56ffd640 1 --2- 172.21.15.37:0/992740706 >> [v2:172.21.15.203:6832/2418205237,v1:172.21.15.203:6833/2418205237] conn(0x7fee440744b0 0x7fee440769c0 secure :-1 s=READY pgs=207 cs=0 l=1 rev1=1 crypto rx=0x7fee580ba0a0 tx=0x7fee500014e0 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:30:09.168 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:09.166+0000 7fee3f7fe640 1 -- 172.21.15.37:0/992740706 <== mon.0 v2:172.21.15.37:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187246 (secure 0 0 0) 0x7fee48010040 con 0x7fee58147fc0 2024-04-25T10:30:09.351 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:09.349+0000 7fee5d334640 1 -- 172.21.15.37:0/992740706 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) -- 0x7fee580c9ab0 con 0x7fee58147fc0 2024-04-25T10:30:09.352 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:09.349+0000 7fee3f7fe640 1 -- 172.21.15.37:0/992740706 <== mon.0 v2:172.21.15.37:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v32) ==== 93+0+10 (secure 0 0 0) 0x7fee480603e0 con 0x7fee58147fc0 2024-04-25T10:30:09.352 INFO:teuthology.orchestra.run.smithi037.stdout:pg_num: 1 2024-04-25T10:30:09.354 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:09.351+0000 7fee5d334640 1 -- 172.21.15.37:0/992740706 >> [v2:172.21.15.203:6832/2418205237,v1:172.21.15.203:6833/2418205237] conn(0x7fee440744b0 msgr2=0x7fee440769c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:30:09.354 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:09.351+0000 7fee5d334640 1 --2- 172.21.15.37:0/992740706 >> [v2:172.21.15.203:6832/2418205237,v1:172.21.15.203:6833/2418205237] conn(0x7fee440744b0 0x7fee440769c0 secure :-1 s=READY pgs=207 cs=0 l=1 rev1=1 crypto rx=0x7fee580ba0a0 tx=0x7fee500014e0 comp rx=0 tx=0).stop 2024-04-25T10:30:09.354 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:09.351+0000 7fee5d334640 1 -- 172.21.15.37:0/992740706 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fee58147fc0 msgr2=0x7fee581483d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:30:09.354 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:09.351+0000 7fee5d334640 1 --2- 172.21.15.37:0/992740706 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fee58147fc0 0x7fee581483d0 secure :-1 s=READY pgs=409 cs=0 l=1 rev1=1 crypto rx=0x7fee580ba070 tx=0x7fee4800f580 comp rx=0 tx=0).stop 2024-04-25T10:30:09.354 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:09.351+0000 7fee5d334640 1 -- 172.21.15.37:0/992740706 shutdown_connections 2024-04-25T10:30:09.354 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:09.351+0000 7fee5d334640 1 --2- 172.21.15.37:0/992740706 >> [v2:172.21.15.203:6832/2418205237,v1:172.21.15.203:6833/2418205237] conn(0x7fee440744b0 0x7fee440769c0 unknown :-1 s=CLOSED pgs=207 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:30:09.354 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:09.351+0000 7fee5d334640 1 --2- 172.21.15.37:0/992740706 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fee58147fc0 0x7fee581483d0 unknown :-1 s=CLOSED pgs=409 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:30:09.354 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:09.351+0000 7fee5d334640 1 --2- 172.21.15.37:0/992740706 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7fee580c9680 0x7fee581633a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:30:09.354 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:09.351+0000 7fee5d334640 1 --2- 172.21.15.37:0/992740706 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fee40004800 0x7fee58162e60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:30:09.354 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:09.351+0000 7fee5d334640 1 -- 172.21.15.37:0/992740706 >> 172.21.15.37:0/992740706 conn(0x7fee580b7a80 msgr2=0x7fee580d6250 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:30:09.354 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:09.351+0000 7fee5d334640 1 -- 172.21.15.37:0/992740706 shutdown_connections 2024-04-25T10:30:09.355 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:09.351+0000 7fee5d334640 1 -- 172.21.15.37:0/992740706 wait complete. 2024-04-25T10:30:09.365 DEBUG:teuthology.orchestra.run.smithi037:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph --format=json-pretty osd lspools 2024-04-25T10:30:09.477 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:09.475+0000 7f3ffe69f640 1 Processor -- start 2024-04-25T10:30:09.477 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:09.475+0000 7f3ffe69f640 1 -- start start 2024-04-25T10:30:09.479 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:09.476+0000 7f3ffe69f640 1 --2- >> v2:172.21.15.203:3300/0 conn(0x7f3ff808cc90 0x7f3ff806e0a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:30:09.479 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:09.476+0000 7f3ffe69f640 1 --2- >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f3ff806e5e0 0x7f3ff806e9f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:30:09.479 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:09.476+0000 7f3ffe69f640 1 -- --> v1:172.21.15.37:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f3ff80d2ed0 con 0x7f3ff81647f0 2024-04-25T10:30:09.479 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:09.476+0000 7f3ffe69f640 1 -- --> v2:172.21.15.203:3300/0 -- mon_getmap magic: 0 -- 0x7f3ff80d3200 con 0x7f3ff808cc90 2024-04-25T10:30:09.479 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:09.476+0000 7f3ffe69f640 1 -- --> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] -- mon_getmap magic: 0 -- 0x7f3ff809cd40 con 0x7f3ff806e5e0 2024-04-25T10:30:09.479 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:09.476+0000 7f3ffcc3c640 1 --1- >> v1:172.21.15.37:6789/0 conn(0x7f3ff81647f0 0x7f3ff8164c00 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.37:6789/0 says I am v1:172.21.15.37:60744/0 (socket says 172.21.15.37:60744) 2024-04-25T10:30:09.480 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:09.476+0000 7f3ffcc3c640 1 -- 172.21.15.37:0/1600544013 learned_addr learned my addr 172.21.15.37:0/1600544013 (peer_addr_for_me v1:172.21.15.37:0/0) 2024-04-25T10:30:09.480 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:09.476+0000 7f3ff77fe640 1 --2- 172.21.15.37:0/1600544013 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f3ff806e5e0 0x7f3ff806e9f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:30:09.480 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:09.476+0000 7f3ff7fff640 1 --2- 172.21.15.37:0/1600544013 >> v2:172.21.15.203:3300/0 conn(0x7f3ff808cc90 0x7f3ff806e0a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:30:09.480 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:09.477+0000 7f3ff6ffd640 1 -- 172.21.15.37:0/1600544013 <== mon.0 v1:172.21.15.37:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 2410779140 0 0) 0x7f3ff80d2ed0 con 0x7f3ff81647f0 2024-04-25T10:30:09.480 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:09.477+0000 7f3ff6ffd640 1 -- 172.21.15.37:0/1600544013 --> v1:172.21.15.37:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f3fe0003520 con 0x7f3ff81647f0 2024-04-25T10:30:09.480 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:09.477+0000 7f3ff6ffd640 1 -- 172.21.15.37:0/1600544013 <== mon.0 v1:172.21.15.37:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 1549131904 0 0) 0x7f3fe0003520 con 0x7f3ff81647f0 2024-04-25T10:30:09.480 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:09.477+0000 7f3ff77fe640 1 -- 172.21.15.37:0/1600544013 >> v2:172.21.15.203:3300/0 conn(0x7f3ff808cc90 msgr2=0x7f3ff806e0a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:30:09.480 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:09.477+0000 7f3ff77fe640 1 --2- 172.21.15.37:0/1600544013 >> v2:172.21.15.203:3300/0 conn(0x7f3ff808cc90 0x7f3ff806e0a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:30:09.480 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:09.477+0000 7f3ff77fe640 1 -- 172.21.15.37:0/1600544013 >> v1:172.21.15.37:6789/0 conn(0x7f3ff81647f0 legacy=0x7f3ff8164c00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:30:09.480 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:09.477+0000 7f3ff77fe640 1 -- 172.21.15.37:0/1600544013 --> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f3ff80d1100 con 0x7f3ff806e5e0 2024-04-25T10:30:09.480 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:09.477+0000 7f3ff7fff640 1 --2- 172.21.15.37:0/1600544013 >> v2:172.21.15.203:3300/0 conn(0x7f3ff808cc90 0x7f3ff806e0a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-25T10:30:09.481 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:09.478+0000 7f3ff77fe640 1 --2- 172.21.15.37:0/1600544013 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f3ff806e5e0 0x7f3ff806e9f0 secure :-1 s=READY pgs=50 cs=0 l=1 rev1=1 crypto rx=0x7f3fec0097a0 tx=0x7f3fec02a110 comp rx=0 tx=0).ready entity=mon.2 client_cookie=deaf6ceaa37531d7 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:30:09.482 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:09.478+0000 7f3ff6ffd640 1 -- 172.21.15.37:0/1600544013 <== mon.2 v2:172.21.15.203:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3fec037070 con 0x7f3ff806e5e0 2024-04-25T10:30:09.482 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:09.478+0000 7f3ff6ffd640 1 -- 172.21.15.37:0/1600544013 <== mon.2 v2:172.21.15.203:3301/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f3fec02aba0 con 0x7f3ff806e5e0 2024-04-25T10:30:09.482 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:09.478+0000 7f3ff6ffd640 1 -- 172.21.15.37:0/1600544013 <== mon.2 v2:172.21.15.203:3301/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3ff00019f0 con 0x7f3ff806e5e0 2024-04-25T10:30:09.482 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:09.479+0000 7f3ffe69f640 1 -- 172.21.15.37:0/1600544013 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f3ff806e5e0 msgr2=0x7f3ff806e9f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:30:09.482 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:09.479+0000 7f3ffe69f640 1 --2- 172.21.15.37:0/1600544013 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f3ff806e5e0 0x7f3ff806e9f0 secure :-1 s=READY pgs=50 cs=0 l=1 rev1=1 crypto rx=0x7f3fec0097a0 tx=0x7f3fec02a110 comp rx=0 tx=0).stop 2024-04-25T10:30:09.482 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:09.479+0000 7f3ffe69f640 1 -- 172.21.15.37:0/1600544013 shutdown_connections 2024-04-25T10:30:09.482 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:09.479+0000 7f3ffe69f640 1 --2- 172.21.15.37:0/1600544013 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f3ff806e5e0 0x7f3ff806e9f0 unknown :-1 s=CLOSED pgs=50 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:30:09.482 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:09.479+0000 7f3ffe69f640 1 --2- 172.21.15.37:0/1600544013 >> v2:172.21.15.203:3300/0 conn(0x7f3ff808cc90 0x7f3ff806e0a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:30:09.482 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:09.479+0000 7f3ffe69f640 1 -- 172.21.15.37:0/1600544013 >> 172.21.15.37:0/1600544013 conn(0x7f3ff80d03d0 msgr2=0x7f3ff80d07e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:30:09.482 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:09.479+0000 7f3ffe69f640 1 -- 172.21.15.37:0/1600544013 shutdown_connections 2024-04-25T10:30:09.482 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:09.479+0000 7f3ffe69f640 1 -- 172.21.15.37:0/1600544013 wait complete. 2024-04-25T10:30:09.483 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:09.480+0000 7f3ffe69f640 1 Processor -- start 2024-04-25T10:30:09.483 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:09.481+0000 7f3ffe69f640 1 -- start start 2024-04-25T10:30:09.485 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:09.482+0000 7f3ffe69f640 1 --2- >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f3ff806e5e0 0x7f3ff81f6e10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:30:09.485 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:09.482+0000 7f3ffe69f640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f3ff81647f0 0x7f3ff80b77f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:30:09.485 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:09.482+0000 7f3ffe69f640 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f3ff81f5f60 0x7f3ff80b7d30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:30:09.485 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:09.482+0000 7f3ffe69f640 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 -- 0x7f3ff80d3690 con 0x7f3ff81647f0 2024-04-25T10:30:09.485 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:09.482+0000 7f3ffe69f640 1 -- --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_getmap magic: 0 -- 0x7f3ff806da50 con 0x7f3ff81f5f60 2024-04-25T10:30:09.486 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:09.482+0000 7f3ffe69f640 1 -- --> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] -- mon_getmap magic: 0 -- 0x7f3ff8165310 con 0x7f3ff806e5e0 2024-04-25T10:30:09.486 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:09.482+0000 7f3ff77fe640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f3ff81647f0 0x7f3ff80b77f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:30:09.486 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:09.482+0000 7f3ff77fe640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f3ff81647f0 0x7f3ff80b77f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.37:3300/0 says I am v2:172.21.15.37:54814/0 (socket says 172.21.15.37:54814) 2024-04-25T10:30:09.486 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:09.482+0000 7f3ff77fe640 1 -- 172.21.15.37:0/2306705031 learned_addr learned my addr 172.21.15.37:0/2306705031 (peer_addr_for_me v2:172.21.15.37:0/0) 2024-04-25T10:30:09.486 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:09.482+0000 7f3ff7fff640 1 --2- 172.21.15.37:0/2306705031 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f3ff806e5e0 0x7f3ff81f6e10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:30:09.486 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:09.482+0000 7f3ffcc3c640 1 --2- 172.21.15.37:0/2306705031 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f3ff81f5f60 0x7f3ff80b7d30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:30:09.486 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:09.482+0000 7f3ff77fe640 1 -- 172.21.15.37:0/2306705031 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f3ff806e5e0 msgr2=0x7f3ff81f6e10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:30:09.486 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:09.482+0000 7f3ff77fe640 1 --2- 172.21.15.37:0/2306705031 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f3ff806e5e0 0x7f3ff81f6e10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:30:09.486 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:09.482+0000 7f3ff77fe640 1 -- 172.21.15.37:0/2306705031 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f3ff81f5f60 msgr2=0x7f3ff80b7d30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:30:09.486 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:09.482+0000 7f3ff77fe640 1 --2- 172.21.15.37:0/2306705031 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f3ff81f5f60 0x7f3ff80b7d30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:30:09.486 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:09.482+0000 7f3ff77fe640 1 -- 172.21.15.37:0/2306705031 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f3ff81f7350 con 0x7f3ff81647f0 2024-04-25T10:30:09.486 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:09.482+0000 7f3ff7fff640 1 --2- 172.21.15.37:0/2306705031 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f3ff806e5e0 0x7f3ff81f6e10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T10:30:09.487 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:09.482+0000 7f3ffcc3c640 1 --2- 172.21.15.37:0/2306705031 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f3ff81f5f60 0x7f3ff80b7d30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T10:30:09.487 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:09.483+0000 7f3ff77fe640 1 --2- 172.21.15.37:0/2306705031 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f3ff81647f0 0x7f3ff80b77f0 secure :-1 s=READY pgs=411 cs=0 l=1 rev1=1 crypto rx=0x7f3fec035040 tx=0x7f3fec0046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:30:09.487 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:09.483+0000 7f3ff4ff9640 1 -- 172.21.15.37:0/2306705031 <== mon.0 v2:172.21.15.37:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3fec041020 con 0x7f3ff81647f0 2024-04-25T10:30:09.487 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:09.483+0000 7f3ffe69f640 1 -- 172.21.15.37:0/2306705031 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f3ff80c9050 con 0x7f3ff81647f0 2024-04-25T10:30:09.487 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:09.483+0000 7f3ff4ff9640 1 -- 172.21.15.37:0/2306705031 <== mon.0 v2:172.21.15.37:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f3fec03d620 con 0x7f3ff81647f0 2024-04-25T10:30:09.487 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:09.483+0000 7f3ff4ff9640 1 -- 172.21.15.37:0/2306705031 <== mon.0 v2:172.21.15.37:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3fec037040 con 0x7f3ff81647f0 2024-04-25T10:30:09.487 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:09.483+0000 7f3ffe69f640 1 -- 172.21.15.37:0/2306705031 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f3ff80cfbc0 con 0x7f3ff81647f0 2024-04-25T10:30:09.488 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:09.484+0000 7f3ffe69f640 1 -- 172.21.15.37:0/2306705031 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f3ff80cf930 con 0x7f3ff81647f0 2024-04-25T10:30:09.488 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:09.485+0000 7f3ff4ff9640 1 -- 172.21.15.37:0/2306705031 <== mon.0 v2:172.21.15.37:3300/0 4 ==== mgrmap(e 5) ==== 96782+0+0 (secure 0 0 0) 0x7f3fec044020 con 0x7f3ff81647f0 2024-04-25T10:30:09.488 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:09.485+0000 7f3ff4ff9640 1 --2- 172.21.15.37:0/2306705031 >> [v2:172.21.15.203:6832/2418205237,v1:172.21.15.203:6833/2418205237] conn(0x7f3fe40744e0 0x7f3fe40769f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:30:09.488 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:09.485+0000 7f3ff4ff9640 1 -- 172.21.15.37:0/2306705031 <== mon.0 v2:172.21.15.37:3300/0 5 ==== osd_map(32..32 src has 1..32) ==== 4857+0+0 (secure 0 0 0) 0x7f3fec0b5460 con 0x7f3ff81647f0 2024-04-25T10:30:09.488 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:09.485+0000 7f3ff7fff640 1 --2- 172.21.15.37:0/2306705031 >> [v2:172.21.15.203:6832/2418205237,v1:172.21.15.203:6833/2418205237] conn(0x7f3fe40744e0 0x7f3fe40769f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:30:09.488 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:09.486+0000 7f3ff7fff640 1 --2- 172.21.15.37:0/2306705031 >> [v2:172.21.15.203:6832/2418205237,v1:172.21.15.203:6833/2418205237] conn(0x7f3fe40744e0 0x7f3fe40769f0 secure :-1 s=READY pgs=208 cs=0 l=1 rev1=1 crypto rx=0x7f3fe8009a20 tx=0x7f3fe8008040 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:30:09.492 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:09.490+0000 7f3ff4ff9640 1 -- 172.21.15.37:0/2306705031 <== mon.0 v2:172.21.15.37:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187246 (secure 0 0 0) 0x7f3fec04b030 con 0x7f3ff81647f0 2024-04-25T10:30:09.678 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:09.676+0000 7f3ffe69f640 1 -- 172.21.15.37:0/2306705031 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_command({"prefix": "osd lspools", "format": "json-pretty"} v 0) -- 0x7f3ff814d6a0 con 0x7f3ff81647f0 2024-04-25T10:30:09.678 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:09.676+0000 7f3ff4ff9640 1 -- 172.21.15.37:0/2306705031 <== mon.0 v2:172.21.15.37:3300/0 7 ==== mon_command_ack([{"prefix": "osd lspools", "format": "json-pretty"}]=0 v32) ==== 84+0+65 (secure 0 0 0) 0x7f3ff814d6a0 con 0x7f3ff81647f0 2024-04-25T10:30:09.678 INFO:teuthology.orchestra.run.smithi037.stdout: 2024-04-25T10:30:09.679 INFO:teuthology.orchestra.run.smithi037.stdout:[ 2024-04-25T10:30:09.679 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-04-25T10:30:09.679 INFO:teuthology.orchestra.run.smithi037.stdout: "poolnum": 1, 2024-04-25T10:30:09.679 INFO:teuthology.orchestra.run.smithi037.stdout: "poolname": ".mgr" 2024-04-25T10:30:09.679 INFO:teuthology.orchestra.run.smithi037.stdout: } 2024-04-25T10:30:09.679 INFO:teuthology.orchestra.run.smithi037.stdout:] 2024-04-25T10:30:09.680 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:09.678+0000 7f3ffe69f640 1 -- 172.21.15.37:0/2306705031 >> [v2:172.21.15.203:6832/2418205237,v1:172.21.15.203:6833/2418205237] conn(0x7f3fe40744e0 msgr2=0x7f3fe40769f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:30:09.680 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:09.678+0000 7f3ffe69f640 1 --2- 172.21.15.37:0/2306705031 >> [v2:172.21.15.203:6832/2418205237,v1:172.21.15.203:6833/2418205237] conn(0x7f3fe40744e0 0x7f3fe40769f0 secure :-1 s=READY pgs=208 cs=0 l=1 rev1=1 crypto rx=0x7f3fe8009a20 tx=0x7f3fe8008040 comp rx=0 tx=0).stop 2024-04-25T10:30:09.680 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:09.678+0000 7f3ffe69f640 1 -- 172.21.15.37:0/2306705031 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f3ff81647f0 msgr2=0x7f3ff80b77f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:30:09.680 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:09.678+0000 7f3ffe69f640 1 --2- 172.21.15.37:0/2306705031 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f3ff81647f0 0x7f3ff80b77f0 secure :-1 s=READY pgs=411 cs=0 l=1 rev1=1 crypto rx=0x7f3fec035040 tx=0x7f3fec0046c0 comp rx=0 tx=0).stop 2024-04-25T10:30:09.681 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:09.678+0000 7f3ffe69f640 1 -- 172.21.15.37:0/2306705031 shutdown_connections 2024-04-25T10:30:09.681 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:09.678+0000 7f3ffe69f640 1 --2- 172.21.15.37:0/2306705031 >> [v2:172.21.15.203:6832/2418205237,v1:172.21.15.203:6833/2418205237] conn(0x7f3fe40744e0 0x7f3fe40769f0 unknown :-1 s=CLOSED pgs=208 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:30:09.681 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:09.678+0000 7f3ffe69f640 1 --2- 172.21.15.37:0/2306705031 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f3ff81f5f60 0x7f3ff80b7d30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:30:09.681 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:09.678+0000 7f3ffe69f640 1 --2- 172.21.15.37:0/2306705031 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f3ff81647f0 0x7f3ff80b77f0 unknown :-1 s=CLOSED pgs=411 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:30:09.681 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:09.678+0000 7f3ffe69f640 1 --2- 172.21.15.37:0/2306705031 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f3ff806e5e0 0x7f3ff81f6e10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:30:09.681 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:09.678+0000 7f3ffe69f640 1 -- 172.21.15.37:0/2306705031 >> 172.21.15.37:0/2306705031 conn(0x7f3ff80d03d0 msgr2=0x7f3ff815efc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:30:09.681 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:09.678+0000 7f3ffe69f640 1 -- 172.21.15.37:0/2306705031 shutdown_connections 2024-04-25T10:30:09.681 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:09.678+0000 7f3ffe69f640 1 -- 172.21.15.37:0/2306705031 wait complete. 2024-04-25T10:30:09.691 DEBUG:tasks.cephfs.filesystem:Creating filesystem 'cephfs' 2024-04-25T10:30:09.692 DEBUG:teuthology.orchestra.run.smithi037:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool create cephfs_metadata --pg_num_min 64 2024-04-25T10:30:09.808 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:09.805+0000 7fd8fed84640 1 Processor -- start 2024-04-25T10:30:09.808 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:09.805+0000 7fd8fed84640 1 -- start start 2024-04-25T10:30:09.810 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:09.807+0000 7fd8fed84640 1 --2- >> v2:172.21.15.203:3300/0 conn(0x7fd8f80d0940 0x7fd8f8160260 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:30:09.810 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:09.807+0000 7fd8fed84640 1 --2- >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7fd8f81647b0 0x7fd8f80d4e30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:30:09.810 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:09.807+0000 7fd8fed84640 1 -- --> v1:172.21.15.37:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fd8f80d0f70 con 0x7fd8f80b79f0 2024-04-25T10:30:09.810 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:09.807+0000 7fd8fed84640 1 -- --> v2:172.21.15.203:3300/0 -- mon_getmap magic: 0 -- 0x7fd8f80d0d50 con 0x7fd8f80d0940 2024-04-25T10:30:09.810 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:09.807+0000 7fd8fed84640 1 -- --> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] -- mon_getmap magic: 0 -- 0x7fd8f80d1250 con 0x7fd8f81647b0 2024-04-25T10:30:09.810 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:09.807+0000 7fd8fd321640 1 --1- >> v1:172.21.15.37:6789/0 conn(0x7fd8f80b79f0 0x7fd8f80b7e00 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.37:6789/0 says I am v1:172.21.15.37:41038/0 (socket says 172.21.15.37:41038) 2024-04-25T10:30:09.810 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:09.807+0000 7fd8fd321640 1 -- 172.21.15.37:0/3020724501 learned_addr learned my addr 172.21.15.37:0/3020724501 (peer_addr_for_me v1:172.21.15.37:0/0) 2024-04-25T10:30:09.810 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:09.807+0000 7fd8fcb20640 1 --2- 172.21.15.37:0/3020724501 >> v2:172.21.15.203:3300/0 conn(0x7fd8f80d0940 0x7fd8f8160260 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:30:09.810 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:09.807+0000 7fd8f7fff640 1 --2- 172.21.15.37:0/3020724501 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7fd8f81647b0 0x7fd8f80d4e30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:30:09.810 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:09.807+0000 7fd8f77fe640 1 -- 172.21.15.37:0/3020724501 <== mon.0 v1:172.21.15.37:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 1062974202 0 0) 0x7fd8f80d0f70 con 0x7fd8f80b79f0 2024-04-25T10:30:09.810 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:09.807+0000 7fd8f77fe640 1 -- 172.21.15.37:0/3020724501 --> v1:172.21.15.37:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7fd8e0003520 con 0x7fd8f80b79f0 2024-04-25T10:30:09.811 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:09.807+0000 7fd8f7fff640 1 -- 172.21.15.37:0/3020724501 >> v2:172.21.15.203:3300/0 conn(0x7fd8f80d0940 msgr2=0x7fd8f8160260 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:30:09.811 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:09.807+0000 7fd8f7fff640 1 --2- 172.21.15.37:0/3020724501 >> v2:172.21.15.203:3300/0 conn(0x7fd8f80d0940 0x7fd8f8160260 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:30:09.811 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:09.807+0000 7fd8f7fff640 1 -- 172.21.15.37:0/3020724501 >> v1:172.21.15.37:6789/0 conn(0x7fd8f80b79f0 legacy=0x7fd8f80b7e00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:30:09.811 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:09.807+0000 7fd8f7fff640 1 -- 172.21.15.37:0/3020724501 --> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fd8f80d4630 con 0x7fd8f81647b0 2024-04-25T10:30:09.811 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:09.807+0000 7fd8fcb20640 1 --2- 172.21.15.37:0/3020724501 >> v2:172.21.15.203:3300/0 conn(0x7fd8f80d0940 0x7fd8f8160260 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-25T10:30:09.812 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:09.808+0000 7fd8f7fff640 1 --2- 172.21.15.37:0/3020724501 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7fd8f81647b0 0x7fd8f80d4e30 secure :-1 s=READY pgs=51 cs=0 l=1 rev1=1 crypto rx=0x7fd8ec0097a0 tx=0x7fd8ec02a180 comp rx=0 tx=0).ready entity=mon.2 client_cookie=cb6d533bbf34a14 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:30:09.812 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:09.808+0000 7fd8f77fe640 1 -- 172.21.15.37:0/3020724501 <== mon.2 v2:172.21.15.203:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd8ec037070 con 0x7fd8f81647b0 2024-04-25T10:30:09.812 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:09.809+0000 7fd8f77fe640 1 -- 172.21.15.37:0/3020724501 <== mon.2 v2:172.21.15.203:3301/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fd8ec032440 con 0x7fd8f81647b0 2024-04-25T10:30:09.812 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:09.809+0000 7fd8f77fe640 1 -- 172.21.15.37:0/3020724501 <== mon.2 v2:172.21.15.203:3301/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd8ec032810 con 0x7fd8f81647b0 2024-04-25T10:30:09.812 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:09.809+0000 7fd8fed84640 1 -- 172.21.15.37:0/3020724501 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7fd8f81647b0 msgr2=0x7fd8f80d4e30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:30:09.812 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:09.809+0000 7fd8fed84640 1 --2- 172.21.15.37:0/3020724501 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7fd8f81647b0 0x7fd8f80d4e30 secure :-1 s=READY pgs=51 cs=0 l=1 rev1=1 crypto rx=0x7fd8ec0097a0 tx=0x7fd8ec02a180 comp rx=0 tx=0).stop 2024-04-25T10:30:09.813 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:09.809+0000 7fd8fed84640 1 -- 172.21.15.37:0/3020724501 shutdown_connections 2024-04-25T10:30:09.813 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:09.809+0000 7fd8fed84640 1 --2- 172.21.15.37:0/3020724501 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7fd8f81647b0 0x7fd8f80d4e30 unknown :-1 s=CLOSED pgs=51 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:30:09.813 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:09.809+0000 7fd8fed84640 1 --2- 172.21.15.37:0/3020724501 >> v2:172.21.15.203:3300/0 conn(0x7fd8f80d0940 0x7fd8f8160260 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:30:09.813 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:09.809+0000 7fd8fed84640 1 -- 172.21.15.37:0/3020724501 >> 172.21.15.37:0/3020724501 conn(0x7fd8f80d5d70 msgr2=0x7fd8f80d6180 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:30:09.813 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:09.809+0000 7fd8fed84640 1 -- 172.21.15.37:0/3020724501 shutdown_connections 2024-04-25T10:30:09.813 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:09.809+0000 7fd8fed84640 1 -- 172.21.15.37:0/3020724501 wait complete. 2024-04-25T10:30:09.813 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:09.811+0000 7fd8fed84640 1 Processor -- start 2024-04-25T10:30:09.813 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:09.811+0000 7fd8fed84640 1 -- start start 2024-04-25T10:30:09.816 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:09.812+0000 7fd8fed84640 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fd8f80b79f0 0x7fd8f80cb350 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:30:09.816 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:09.812+0000 7fd8fed84640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fd8f80d0940 0x7fd8f80cb890 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:30:09.816 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:09.812+0000 7fd8fed84640 1 --2- >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7fd8f81647b0 0x7fd8f80ca040 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:30:09.816 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:09.812+0000 7fd8fed84640 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 -- 0x7fd8f8164bc0 con 0x7fd8f80d0940 2024-04-25T10:30:09.816 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:09.812+0000 7fd8fed84640 1 -- --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_getmap magic: 0 -- 0x7fd8f80d1530 con 0x7fd8f80b79f0 2024-04-25T10:30:09.816 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:09.812+0000 7fd8fed84640 1 -- --> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] -- mon_getmap magic: 0 -- 0x7fd8f8164d40 con 0x7fd8f81647b0 2024-04-25T10:30:09.816 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:09.813+0000 7fd8f7fff640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fd8f80d0940 0x7fd8f80cb890 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:30:09.816 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:09.813+0000 7fd8f7fff640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fd8f80d0940 0x7fd8f80cb890 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.37:3300/0 says I am v2:172.21.15.37:52554/0 (socket says 172.21.15.37:52554) 2024-04-25T10:30:09.816 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:09.813+0000 7fd8f7fff640 1 -- 172.21.15.37:0/2806555049 learned_addr learned my addr 172.21.15.37:0/2806555049 (peer_addr_for_me v2:172.21.15.37:0/0) 2024-04-25T10:30:09.817 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:09.813+0000 7fd8fd321640 1 --2- 172.21.15.37:0/2806555049 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7fd8f81647b0 0x7fd8f80ca040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:30:09.817 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:09.813+0000 7fd8fcb20640 1 --2- 172.21.15.37:0/2806555049 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fd8f80b79f0 0x7fd8f80cb350 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:30:09.817 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:09.813+0000 7fd8f7fff640 1 -- 172.21.15.37:0/2806555049 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7fd8f81647b0 msgr2=0x7fd8f80ca040 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:30:09.817 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:09.813+0000 7fd8f7fff640 1 --2- 172.21.15.37:0/2806555049 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7fd8f81647b0 0x7fd8f80ca040 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:30:09.817 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:09.813+0000 7fd8f7fff640 1 -- 172.21.15.37:0/2806555049 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fd8f80b79f0 msgr2=0x7fd8f80cb350 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:30:09.817 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:09.813+0000 7fd8f7fff640 1 --2- 172.21.15.37:0/2806555049 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fd8f80b79f0 0x7fd8f80cb350 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:30:09.817 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:09.813+0000 7fd8f7fff640 1 -- 172.21.15.37:0/2806555049 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fd8f8160930 con 0x7fd8f80d0940 2024-04-25T10:30:09.817 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:09.813+0000 7fd8fd321640 1 --2- 172.21.15.37:0/2806555049 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7fd8f81647b0 0x7fd8f80ca040 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T10:30:09.817 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:09.813+0000 7fd8f7fff640 1 --2- 172.21.15.37:0/2806555049 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fd8f80d0940 0x7fd8f80cb890 secure :-1 s=READY pgs=413 cs=0 l=1 rev1=1 crypto rx=0x7fd8ec02a1c0 tx=0x7fd8ec0046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:30:09.817 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:09.813+0000 7fd8f57fa640 1 -- 172.21.15.37:0/2806555049 <== mon.0 v2:172.21.15.37:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd8ec037040 con 0x7fd8f80d0940 2024-04-25T10:30:09.817 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:09.813+0000 7fd8f57fa640 1 -- 172.21.15.37:0/2806555049 <== mon.0 v2:172.21.15.37:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fd8ec042070 con 0x7fd8f80d0940 2024-04-25T10:30:09.817 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:09.813+0000 7fd8fed84640 1 -- 172.21.15.37:0/2806555049 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fd8f81f76f0 con 0x7fd8f80d0940 2024-04-25T10:30:09.817 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:09.813+0000 7fd8f57fa640 1 -- 172.21.15.37:0/2806555049 <== mon.0 v2:172.21.15.37:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd8ec03b930 con 0x7fd8f80d0940 2024-04-25T10:30:09.818 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:09.813+0000 7fd8fed84640 1 -- 172.21.15.37:0/2806555049 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fd8f8160ba0 con 0x7fd8f80d0940 2024-04-25T10:30:09.819 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:09.815+0000 7fd8fed84640 1 -- 172.21.15.37:0/2806555049 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fd8f806a140 con 0x7fd8f80d0940 2024-04-25T10:30:09.819 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:09.815+0000 7fd8f57fa640 1 -- 172.21.15.37:0/2806555049 <== mon.0 v2:172.21.15.37:3300/0 4 ==== mgrmap(e 5) ==== 96782+0+0 (secure 0 0 0) 0x7fd8ec03bad0 con 0x7fd8f80d0940 2024-04-25T10:30:09.819 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:09.816+0000 7fd8f57fa640 1 --2- 172.21.15.37:0/2806555049 >> [v2:172.21.15.203:6832/2418205237,v1:172.21.15.203:6833/2418205237] conn(0x7fd8e40744b0 0x7fd8e40769c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:30:09.819 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:09.816+0000 7fd8f57fa640 1 -- 172.21.15.37:0/2806555049 <== mon.0 v2:172.21.15.37:3300/0 5 ==== osd_map(32..32 src has 1..32) ==== 4857+0+0 (secure 0 0 0) 0x7fd8ec0b56c0 con 0x7fd8f80d0940 2024-04-25T10:30:09.819 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:09.816+0000 7fd8fcb20640 1 --2- 172.21.15.37:0/2806555049 >> [v2:172.21.15.203:6832/2418205237,v1:172.21.15.203:6833/2418205237] conn(0x7fd8e40744b0 0x7fd8e40769c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:30:09.819 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:09.816+0000 7fd8fcb20640 1 --2- 172.21.15.37:0/2806555049 >> [v2:172.21.15.203:6832/2418205237,v1:172.21.15.203:6833/2418205237] conn(0x7fd8e40744b0 0x7fd8e40769c0 secure :-1 s=READY pgs=209 cs=0 l=1 rev1=1 crypto rx=0x7fd8f0007440 tx=0x7fd8f000b040 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:30:09.823 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:09.820+0000 7fd8f57fa640 1 -- 172.21.15.37:0/2806555049 <== mon.0 v2:172.21.15.37:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187246 (secure 0 0 0) 0x7fd8ec04c020 con 0x7fd8f80d0940 2024-04-25T10:30:10.005 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:10.003+0000 7fd8fed84640 1 -- 172.21.15.37:0/2806555049 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_command({"prefix": "osd pool create", "pool": "cephfs_metadata", "pg_num_min": 64} v 0) -- 0x7fd8f81636e0 con 0x7fd8f80d0940 2024-04-25T10:30:10.735 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:10.733+0000 7fd8f57fa640 1 -- 172.21.15.37:0/2806555049 <== mon.0 v2:172.21.15.37:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool create", "pool": "cephfs_metadata", "pg_num_min": 64}]=0 pool 'cephfs_metadata' created v33) ==== 138+0+0 (secure 0 0 0) 0x7fd8ec0801a0 con 0x7fd8f80d0940 2024-04-25T10:30:10.735 INFO:teuthology.orchestra.run.smithi037.stderr:pool 'cephfs_metadata' created 2024-04-25T10:30:10.738 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:10.735+0000 7fd8fed84640 1 -- 172.21.15.37:0/2806555049 >> [v2:172.21.15.203:6832/2418205237,v1:172.21.15.203:6833/2418205237] conn(0x7fd8e40744b0 msgr2=0x7fd8e40769c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:30:10.738 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:10.735+0000 7fd8fed84640 1 --2- 172.21.15.37:0/2806555049 >> [v2:172.21.15.203:6832/2418205237,v1:172.21.15.203:6833/2418205237] conn(0x7fd8e40744b0 0x7fd8e40769c0 secure :-1 s=READY pgs=209 cs=0 l=1 rev1=1 crypto rx=0x7fd8f0007440 tx=0x7fd8f000b040 comp rx=0 tx=0).stop 2024-04-25T10:30:10.738 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:10.735+0000 7fd8fed84640 1 -- 172.21.15.37:0/2806555049 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fd8f80d0940 msgr2=0x7fd8f80cb890 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:30:10.738 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:10.735+0000 7fd8fed84640 1 --2- 172.21.15.37:0/2806555049 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fd8f80d0940 0x7fd8f80cb890 secure :-1 s=READY pgs=413 cs=0 l=1 rev1=1 crypto rx=0x7fd8ec02a1c0 tx=0x7fd8ec0046c0 comp rx=0 tx=0).stop 2024-04-25T10:30:10.738 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:10.735+0000 7fd8fed84640 1 -- 172.21.15.37:0/2806555049 shutdown_connections 2024-04-25T10:30:10.738 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:10.735+0000 7fd8fed84640 1 --2- 172.21.15.37:0/2806555049 >> [v2:172.21.15.203:6832/2418205237,v1:172.21.15.203:6833/2418205237] conn(0x7fd8e40744b0 0x7fd8e40769c0 unknown :-1 s=CLOSED pgs=209 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:30:10.738 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:10.735+0000 7fd8fed84640 1 --2- 172.21.15.37:0/2806555049 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7fd8f81647b0 0x7fd8f80ca040 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:30:10.738 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:10.735+0000 7fd8fed84640 1 --2- 172.21.15.37:0/2806555049 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fd8f80d0940 0x7fd8f80cb890 unknown :-1 s=CLOSED pgs=413 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:30:10.738 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:10.735+0000 7fd8fed84640 1 --2- 172.21.15.37:0/2806555049 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fd8f80b79f0 0x7fd8f80cb350 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:30:10.738 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:10.736+0000 7fd8fed84640 1 -- 172.21.15.37:0/2806555049 >> 172.21.15.37:0/2806555049 conn(0x7fd8f80d5d70 msgr2=0x7fd8f815ceb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:30:10.739 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:10.736+0000 7fd8fed84640 1 -- 172.21.15.37:0/2806555049 shutdown_connections 2024-04-25T10:30:10.739 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:10.736+0000 7fd8fed84640 1 -- 172.21.15.37:0/2806555049 wait complete. 2024-04-25T10:30:10.755 DEBUG:teuthology.orchestra.run.smithi037:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool create cephfs_data 64 --pg_num_min 64 --target_size_ratio 0.9 2024-04-25T10:30:10.880 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:10.877+0000 7fb156500640 1 Processor -- start 2024-04-25T10:30:10.880 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:10.877+0000 7fb156500640 1 -- start start 2024-04-25T10:30:10.883 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:10.879+0000 7fb156500640 1 --2- >> v2:172.21.15.203:3300/0 conn(0x7fb1500d1ec0 0x7fb1500d22d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:30:10.883 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:10.879+0000 7fb156500640 1 --2- >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7fb1500d2810 0x7fb1500b3c80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:30:10.883 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:10.879+0000 7fb156500640 1 -- --> v1:172.21.15.37:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fb15009ce70 con 0x7fb15006e260 2024-04-25T10:30:10.883 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:10.879+0000 7fb156500640 1 -- --> v2:172.21.15.203:3300/0 -- mon_getmap magic: 0 -- 0x7fb1500d3370 con 0x7fb1500d1ec0 2024-04-25T10:30:10.883 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:10.879+0000 7fb156500640 1 -- --> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] -- mon_getmap magic: 0 -- 0x7fb1500d3090 con 0x7fb1500d2810 2024-04-25T10:30:10.883 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:10.879+0000 7fb14ffff640 1 --1- >> v1:172.21.15.37:6789/0 conn(0x7fb15006e260 0x7fb15006e670 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.37:6789/0 says I am v1:172.21.15.37:41054/0 (socket says 172.21.15.37:41054) 2024-04-25T10:30:10.883 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:10.879+0000 7fb14ffff640 1 -- 172.21.15.37:0/1250741017 learned_addr learned my addr 172.21.15.37:0/1250741017 (peer_addr_for_me v1:172.21.15.37:0/0) 2024-04-25T10:30:10.884 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:10.879+0000 7fb154a9d640 1 --2- 172.21.15.37:0/1250741017 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7fb1500d2810 0x7fb1500b3c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:30:10.884 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:10.879+0000 7fb14f7fe640 1 --2- 172.21.15.37:0/1250741017 >> v2:172.21.15.203:3300/0 conn(0x7fb1500d1ec0 0x7fb1500d22d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:30:10.884 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:10.879+0000 7fb14effd640 1 -- 172.21.15.37:0/1250741017 <== mon.0 v1:172.21.15.37:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 2553074810 0 0) 0x7fb15009ce70 con 0x7fb15006e260 2024-04-25T10:30:10.884 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:10.879+0000 7fb14effd640 1 -- 172.21.15.37:0/1250741017 --> v1:172.21.15.37:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7fb13c003520 con 0x7fb15006e260 2024-04-25T10:30:10.884 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:10.879+0000 7fb14effd640 1 -- 172.21.15.37:0/1250741017 <== mon.0 v1:172.21.15.37:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 2210808790 0 0) 0x7fb13c003520 con 0x7fb15006e260 2024-04-25T10:30:10.884 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:10.880+0000 7fb14effd640 1 -- 172.21.15.37:0/1250741017 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7fb1500d2810 msgr2=0x7fb1500b3c80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:30:10.884 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:10.880+0000 7fb14effd640 1 --2- 172.21.15.37:0/1250741017 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7fb1500d2810 0x7fb1500b3c80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:30:10.884 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:10.880+0000 7fb14effd640 1 -- 172.21.15.37:0/1250741017 >> v2:172.21.15.203:3300/0 conn(0x7fb1500d1ec0 msgr2=0x7fb1500d22d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:30:10.884 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:10.880+0000 7fb14effd640 1 --2- 172.21.15.37:0/1250741017 >> v2:172.21.15.203:3300/0 conn(0x7fb1500d1ec0 0x7fb1500d22d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:30:10.884 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:10.880+0000 7fb14effd640 1 -- 172.21.15.37:0/1250741017 --> v1:172.21.15.37:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fb15009d150 con 0x7fb15006e260 2024-04-25T10:30:10.884 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:10.880+0000 7fb14effd640 1 -- 172.21.15.37:0/1250741017 <== mon.0 v1:172.21.15.37:6789/0 3 ==== mon_map magic: 0 ==== 413+0+0 (unknown 1577794600 0 0) 0x7fb138001670 con 0x7fb15006e260 2024-04-25T10:30:10.884 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:10.880+0000 7fb14effd640 1 -- 172.21.15.37:0/1250741017 >> v1:172.21.15.37:6789/0 conn(0x7fb15006e260 legacy=0x7fb15006e670 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:30:10.884 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:10.880+0000 7fb14effd640 1 --2- 172.21.15.37:0/1250741017 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fb13c0039b0 0x7fb13c003e20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:30:10.884 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:10.880+0000 7fb14effd640 1 --2- 172.21.15.37:0/1250741017 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7fb1500d2810 0x7fb13c004800 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:30:10.885 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:10.880+0000 7fb14effd640 1 --2- 172.21.15.37:0/1250741017 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fb13c004d40 0x7fb13c005150 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:30:10.885 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:10.880+0000 7fb14effd640 1 -- 172.21.15.37:0/1250741017 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 -- 0x7fb1500d3090 con 0x7fb13c004d40 2024-04-25T10:30:10.885 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:10.880+0000 7fb14effd640 1 -- 172.21.15.37:0/1250741017 --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_getmap magic: 0 -- 0x7fb13c00b9c0 con 0x7fb13c0039b0 2024-04-25T10:30:10.885 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:10.880+0000 7fb14effd640 1 -- 172.21.15.37:0/1250741017 --> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] -- mon_getmap magic: 0 -- 0x7fb13c00bbc0 con 0x7fb1500d2810 2024-04-25T10:30:10.885 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:10.880+0000 7fb154a9d640 1 --2- 172.21.15.37:0/1250741017 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fb13c004d40 0x7fb13c005150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:30:10.885 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:10.880+0000 7fb14f7fe640 1 --2- 172.21.15.37:0/1250741017 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7fb1500d2810 0x7fb13c004800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:30:10.885 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:10.880+0000 7fb14ffff640 1 --2- 172.21.15.37:0/1250741017 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fb13c0039b0 0x7fb13c003e20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:30:10.885 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:10.880+0000 7fb154a9d640 1 -- 172.21.15.37:0/1250741017 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7fb1500d2810 msgr2=0x7fb13c004800 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:30:10.885 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:10.880+0000 7fb154a9d640 1 --2- 172.21.15.37:0/1250741017 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7fb1500d2810 0x7fb13c004800 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:30:10.885 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:10.880+0000 7fb154a9d640 1 -- 172.21.15.37:0/1250741017 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fb13c0039b0 msgr2=0x7fb13c003e20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:30:10.885 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:10.880+0000 7fb154a9d640 1 --2- 172.21.15.37:0/1250741017 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fb13c0039b0 0x7fb13c003e20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:30:10.885 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:10.880+0000 7fb154a9d640 1 -- 172.21.15.37:0/1250741017 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fb13c003520 con 0x7fb13c004d40 2024-04-25T10:30:10.885 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:10.881+0000 7fb154a9d640 1 --2- 172.21.15.37:0/1250741017 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fb13c004d40 0x7fb13c005150 secure :-1 s=READY pgs=415 cs=0 l=1 rev1=1 crypto rx=0x7fb148009f90 tx=0x7fb13c005830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d8354ba0c51eadce server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:30:10.886 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:10.881+0000 7fb14effd640 1 -- 172.21.15.37:0/1250741017 <== mon.0 v2:172.21.15.37:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb148009c10 con 0x7fb13c004d40 2024-04-25T10:30:10.886 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:10.881+0000 7fb14effd640 1 -- 172.21.15.37:0/1250741017 <== mon.0 v2:172.21.15.37:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fb148036070 con 0x7fb13c004d40 2024-04-25T10:30:10.886 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:10.881+0000 7fb14effd640 1 -- 172.21.15.37:0/1250741017 <== mon.0 v2:172.21.15.37:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb148031810 con 0x7fb13c004d40 2024-04-25T10:30:10.889 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:10.882+0000 7fb156500640 1 -- 172.21.15.37:0/1250741017 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fb13c004d40 msgr2=0x7fb13c005150 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:30:10.889 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:10.882+0000 7fb156500640 1 --2- 172.21.15.37:0/1250741017 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fb13c004d40 0x7fb13c005150 secure :-1 s=READY pgs=415 cs=0 l=1 rev1=1 crypto rx=0x7fb148009f90 tx=0x7fb13c005830 comp rx=0 tx=0).stop 2024-04-25T10:30:10.889 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:10.882+0000 7fb154a9d640 1 -- 172.21.15.37:0/1250741017 reap_dead start 2024-04-25T10:30:10.889 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:10.882+0000 7fb156500640 1 -- 172.21.15.37:0/1250741017 shutdown_connections 2024-04-25T10:30:10.889 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:10.882+0000 7fb156500640 1 -- 172.21.15.37:0/1250741017 >> 172.21.15.37:0/1250741017 conn(0x7fb1500d0390 msgr2=0x7fb1500d07a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:30:10.889 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:10.882+0000 7fb156500640 1 -- 172.21.15.37:0/1250741017 shutdown_connections 2024-04-25T10:30:10.889 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:10.882+0000 7fb156500640 1 -- 172.21.15.37:0/1250741017 wait complete. 2024-04-25T10:30:10.889 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:10.884+0000 7fb156500640 1 Processor -- start 2024-04-25T10:30:10.889 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:10.884+0000 7fb156500640 1 -- start start 2024-04-25T10:30:10.889 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:10.885+0000 7fb156500640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fb1500b84d0 0x7fb15009e340 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:30:10.889 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:10.885+0000 7fb156500640 1 --2- >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7fb15009e880 0x7fb1501d76a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:30:10.890 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:10.885+0000 7fb156500640 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fb1501d7be0 0x7fb1500b11d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:30:10.890 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:10.885+0000 7fb156500640 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 -- 0x7fb1500b9000 con 0x7fb1500b84d0 2024-04-25T10:30:10.890 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:10.885+0000 7fb156500640 1 -- --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_getmap magic: 0 -- 0x7fb15006d830 con 0x7fb1501d7be0 2024-04-25T10:30:10.890 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:10.885+0000 7fb156500640 1 -- --> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] -- mon_getmap magic: 0 -- 0x7fb1500b9180 con 0x7fb15009e880 2024-04-25T10:30:10.890 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:10.885+0000 7fb14ffff640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fb1500b84d0 0x7fb15009e340 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:30:10.890 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:10.885+0000 7fb14ffff640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fb1500b84d0 0x7fb15009e340 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.37:3300/0 says I am v2:172.21.15.37:52566/0 (socket says 172.21.15.37:52566) 2024-04-25T10:30:10.890 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:10.885+0000 7fb14ffff640 1 -- 172.21.15.37:0/1715299781 learned_addr learned my addr 172.21.15.37:0/1715299781 (peer_addr_for_me v2:172.21.15.37:0/0) 2024-04-25T10:30:10.890 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:10.885+0000 7fb14f7fe640 1 --2- 172.21.15.37:0/1715299781 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7fb15009e880 0x7fb1501d76a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:30:10.890 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:10.886+0000 7fb14ffff640 1 -- 172.21.15.37:0/1715299781 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7fb15009e880 msgr2=0x7fb1501d76a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:30:10.890 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:10.886+0000 7fb154a9d640 1 --2- 172.21.15.37:0/1715299781 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fb1501d7be0 0x7fb1500b11d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:30:10.890 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:10.886+0000 7fb14ffff640 1 --2- 172.21.15.37:0/1715299781 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7fb15009e880 0x7fb1501d76a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:30:10.890 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:10.886+0000 7fb14ffff640 1 -- 172.21.15.37:0/1715299781 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fb1501d7be0 msgr2=0x7fb1500b11d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:30:10.890 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:10.886+0000 7fb14ffff640 1 --2- 172.21.15.37:0/1715299781 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fb1501d7be0 0x7fb1500b11d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:30:10.890 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:10.886+0000 7fb14ffff640 1 -- 172.21.15.37:0/1715299781 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fb1500b7a60 con 0x7fb1500b84d0 2024-04-25T10:30:10.891 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:10.886+0000 7fb14f7fe640 1 --2- 172.21.15.37:0/1715299781 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7fb15009e880 0x7fb1501d76a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T10:30:10.891 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:10.886+0000 7fb14ffff640 1 --2- 172.21.15.37:0/1715299781 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fb1500b84d0 0x7fb15009e340 secure :-1 s=READY pgs=416 cs=0 l=1 rev1=1 crypto rx=0x7fb138007290 tx=0x7fb138007750 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:30:10.891 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:10.886+0000 7fb14cff9640 1 -- 172.21.15.37:0/1715299781 <== mon.0 v2:172.21.15.37:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb13800f9c0 con 0x7fb1500b84d0 2024-04-25T10:30:10.891 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:10.886+0000 7fb14cff9640 1 -- 172.21.15.37:0/1715299781 <== mon.0 v2:172.21.15.37:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fb138018070 con 0x7fb1500b84d0 2024-04-25T10:30:10.891 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:10.886+0000 7fb14cff9640 1 -- 172.21.15.37:0/1715299781 <== mon.0 v2:172.21.15.37:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb13800fda0 con 0x7fb1500b84d0 2024-04-25T10:30:10.891 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:10.886+0000 7fb156500640 1 -- 172.21.15.37:0/1715299781 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fb1500b88e0 con 0x7fb1500b84d0 2024-04-25T10:30:10.891 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:10.886+0000 7fb156500640 1 -- 172.21.15.37:0/1715299781 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fb1500b7cd0 con 0x7fb1500b84d0 2024-04-25T10:30:10.891 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:10.887+0000 7fb156500640 1 -- 172.21.15.37:0/1715299781 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fb1500b8110 con 0x7fb1500b84d0 2024-04-25T10:30:10.892 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:10.888+0000 7fb14cff9640 1 -- 172.21.15.37:0/1715299781 <== mon.0 v2:172.21.15.37:3300/0 4 ==== mgrmap(e 5) ==== 96782+0+0 (secure 0 0 0) 0x7fb13801d460 con 0x7fb1500b84d0 2024-04-25T10:30:10.892 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:10.888+0000 7fb14cff9640 1 --2- 172.21.15.37:0/1715299781 >> [v2:172.21.15.203:6832/2418205237,v1:172.21.15.203:6833/2418205237] conn(0x7fb1300744b0 0x7fb1300769c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:30:10.892 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:10.888+0000 7fb14cff9640 1 -- 172.21.15.37:0/1715299781 <== mon.0 v2:172.21.15.37:3300/0 5 ==== osd_map(33..33 src has 1..33) ==== 5227+0+0 (secure 0 0 0) 0x7fb1380950d0 con 0x7fb1500b84d0 2024-04-25T10:30:10.892 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:10.889+0000 7fb14f7fe640 1 --2- 172.21.15.37:0/1715299781 >> [v2:172.21.15.203:6832/2418205237,v1:172.21.15.203:6833/2418205237] conn(0x7fb1300744b0 0x7fb1300769c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:30:10.892 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:10.889+0000 7fb14f7fe640 1 --2- 172.21.15.37:0/1715299781 >> [v2:172.21.15.203:6832/2418205237,v1:172.21.15.203:6833/2418205237] conn(0x7fb1300744b0 0x7fb1300769c0 secure :-1 s=READY pgs=210 cs=0 l=1 rev1=1 crypto rx=0x7fb140009b30 tx=0x7fb140008040 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:30:10.895 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:10.893+0000 7fb14cff9640 1 -- 172.21.15.37:0/1715299781 <== mon.0 v2:172.21.15.37:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187246 (secure 0 0 0) 0x7fb13805fb60 con 0x7fb1500b84d0 2024-04-25T10:30:11.082 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:11.080+0000 7fb156500640 1 -- 172.21.15.37:0/1715299781 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_command({"prefix": "osd pool create", "pool": "cephfs_data", "pg_num": 64, "pg_num_min": 64, "target_size_ratio": 0.9} v 0) -- 0x7fb15006a140 con 0x7fb1500b84d0 2024-04-25T10:30:11.738 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:11.735+0000 7fb14cff9640 1 -- 172.21.15.37:0/1715299781 <== mon.0 v2:172.21.15.37:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool create", "pool": "cephfs_data", "pg_num": 64, "pg_num_min": 64, "target_size_ratio": 0.9}]=0 pool 'cephfs_data' created v34) ==== 170+0+0 (secure 0 0 0) 0x7fb13805f460 con 0x7fb1500b84d0 2024-04-25T10:30:11.738 INFO:teuthology.orchestra.run.smithi037.stderr:pool 'cephfs_data' created 2024-04-25T10:30:11.742 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:11.739+0000 7fb156500640 1 -- 172.21.15.37:0/1715299781 >> [v2:172.21.15.203:6832/2418205237,v1:172.21.15.203:6833/2418205237] conn(0x7fb1300744b0 msgr2=0x7fb1300769c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:30:11.742 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:11.739+0000 7fb156500640 1 --2- 172.21.15.37:0/1715299781 >> [v2:172.21.15.203:6832/2418205237,v1:172.21.15.203:6833/2418205237] conn(0x7fb1300744b0 0x7fb1300769c0 secure :-1 s=READY pgs=210 cs=0 l=1 rev1=1 crypto rx=0x7fb140009b30 tx=0x7fb140008040 comp rx=0 tx=0).stop 2024-04-25T10:30:11.742 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:11.739+0000 7fb156500640 1 -- 172.21.15.37:0/1715299781 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fb1500b84d0 msgr2=0x7fb15009e340 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:30:11.742 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:11.739+0000 7fb156500640 1 --2- 172.21.15.37:0/1715299781 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fb1500b84d0 0x7fb15009e340 secure :-1 s=READY pgs=416 cs=0 l=1 rev1=1 crypto rx=0x7fb138007290 tx=0x7fb138007750 comp rx=0 tx=0).stop 2024-04-25T10:30:11.742 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:11.739+0000 7fb156500640 1 -- 172.21.15.37:0/1715299781 shutdown_connections 2024-04-25T10:30:11.742 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:11.739+0000 7fb156500640 1 --2- 172.21.15.37:0/1715299781 >> [v2:172.21.15.203:6832/2418205237,v1:172.21.15.203:6833/2418205237] conn(0x7fb1300744b0 0x7fb1300769c0 unknown :-1 s=CLOSED pgs=210 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:30:11.742 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:11.739+0000 7fb156500640 1 --2- 172.21.15.37:0/1715299781 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fb1501d7be0 0x7fb1500b11d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:30:11.742 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:11.739+0000 7fb156500640 1 --2- 172.21.15.37:0/1715299781 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7fb15009e880 0x7fb1501d76a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:30:11.742 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:11.739+0000 7fb156500640 1 --2- 172.21.15.37:0/1715299781 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fb1500b84d0 0x7fb15009e340 unknown :-1 s=CLOSED pgs=416 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:30:11.742 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:11.739+0000 7fb156500640 1 -- 172.21.15.37:0/1715299781 >> 172.21.15.37:0/1715299781 conn(0x7fb1500d0390 msgr2=0x7fb1500cec50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:30:11.743 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:11.740+0000 7fb156500640 1 -- 172.21.15.37:0/1715299781 shutdown_connections 2024-04-25T10:30:11.743 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:11.740+0000 7fb156500640 1 -- 172.21.15.37:0/1715299781 wait complete. 2024-04-25T10:30:11.767 DEBUG:teuthology.orchestra.run.smithi037:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs new cephfs cephfs_metadata cephfs_data 2024-04-25T10:30:11.887 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:11.885+0000 7f7c5f242640 1 Processor -- start 2024-04-25T10:30:11.887 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:11.885+0000 7f7c5f242640 1 -- start start 2024-04-25T10:30:11.889 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:11.886+0000 7f7c5f242640 1 --2- >> v2:172.21.15.203:3300/0 conn(0x7f7c580b7cd0 0x7f7c58160a00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:30:11.889 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:11.886+0000 7f7c5f242640 1 -- --> v1:172.21.15.37:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f7c580d5840 con 0x7f7c580d5430 2024-04-25T10:30:11.890 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:11.886+0000 7f7c5f242640 1 -- --> v1:172.21.15.203:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f7c580d4670 con 0x7f7c58160f40 2024-04-25T10:30:11.890 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:11.886+0000 7f7c5f242640 1 -- --> v2:172.21.15.203:3300/0 -- mon_getmap magic: 0 -- 0x7f7c580d4420 con 0x7f7c580b7cd0 2024-04-25T10:30:11.890 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:11.886+0000 7f7c5cfde640 1 --1- >> v1:172.21.15.37:6789/0 conn(0x7f7c580d5430 0x7f7c580b75c0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.37:6789/0 says I am v1:172.21.15.37:41056/0 (socket says 172.21.15.37:41056) 2024-04-25T10:30:11.890 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:11.886+0000 7f7c5cfde640 1 -- 172.21.15.37:0/909686071 learned_addr learned my addr 172.21.15.37:0/909686071 (peer_addr_for_me v1:172.21.15.37:0/0) 2024-04-25T10:30:11.890 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:11.886+0000 7f7c577fe640 1 -- 172.21.15.37:0/909686071 <== mon.0 v1:172.21.15.37:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 3261057183 0 0) 0x7f7c580d5840 con 0x7f7c580d5430 2024-04-25T10:30:11.890 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:11.886+0000 7f7c57fff640 1 --2- 172.21.15.37:0/909686071 >> v2:172.21.15.203:3300/0 conn(0x7f7c580b7cd0 0x7f7c58160a00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:30:11.890 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:11.886+0000 7f7c577fe640 1 -- 172.21.15.37:0/909686071 --> v1:172.21.15.37:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f7c44003520 con 0x7f7c580d5430 2024-04-25T10:30:11.890 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:11.887+0000 7f7c577fe640 1 -- 172.21.15.37:0/909686071 <== mon.0 v1:172.21.15.37:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 3392931535 0 0) 0x7f7c44003520 con 0x7f7c580d5430 2024-04-25T10:30:11.890 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:11.887+0000 7f7c577fe640 1 -- 172.21.15.37:0/909686071 >> v2:172.21.15.203:3300/0 conn(0x7f7c580b7cd0 msgr2=0x7f7c58160a00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:30:11.890 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:11.887+0000 7f7c577fe640 1 --2- 172.21.15.37:0/909686071 >> v2:172.21.15.203:3300/0 conn(0x7f7c580b7cd0 0x7f7c58160a00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:30:11.890 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:11.887+0000 7f7c577fe640 1 -- 172.21.15.37:0/909686071 >> v1:172.21.15.203:6789/0 conn(0x7f7c58160f40 legacy=0x7f7c58161880 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:30:11.890 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:11.887+0000 7f7c577fe640 1 -- 172.21.15.37:0/909686071 --> v1:172.21.15.37:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f7c580d4950 con 0x7f7c580d5430 2024-04-25T10:30:11.890 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:11.887+0000 7f7c577fe640 1 -- 172.21.15.37:0/909686071 <== mon.0 v1:172.21.15.37:6789/0 3 ==== mon_map magic: 0 ==== 413+0+0 (unknown 1577794600 0 0) 0x7f7c500019f0 con 0x7f7c580d5430 2024-04-25T10:30:11.891 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:11.887+0000 7f7c577fe640 1 -- 172.21.15.37:0/909686071 >> v1:172.21.15.37:6789/0 conn(0x7f7c580d5430 legacy=0x7f7c580b75c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:30:11.891 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:11.887+0000 7f7c577fe640 1 --2- 172.21.15.37:0/909686071 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f7c440039b0 0x7f7c44003e20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:30:11.891 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:11.887+0000 7f7c577fe640 1 --2- 172.21.15.37:0/909686071 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f7c44004800 0x7f7c44004c10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:30:11.891 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:11.887+0000 7f7c577fe640 1 --2- 172.21.15.37:0/909686071 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f7c44005250 0x7f7c4400b9c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:30:11.891 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:11.887+0000 7f7c577fe640 1 -- 172.21.15.37:0/909686071 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 -- 0x7f7c580d4420 con 0x7f7c44004800 2024-04-25T10:30:11.891 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:11.887+0000 7f7c577fe640 1 -- 172.21.15.37:0/909686071 --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_getmap magic: 0 -- 0x7f7c4400c1e0 con 0x7f7c44005250 2024-04-25T10:30:11.891 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:11.887+0000 7f7c577fe640 1 -- 172.21.15.37:0/909686071 --> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] -- mon_getmap magic: 0 -- 0x7f7c4400c3e0 con 0x7f7c440039b0 2024-04-25T10:30:11.891 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:11.887+0000 7f7c57fff640 1 --2- 172.21.15.37:0/909686071 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f7c44004800 0x7f7c44004c10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:30:11.891 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:11.887+0000 7f7c5cfde640 1 --2- 172.21.15.37:0/909686071 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f7c440039b0 0x7f7c44003e20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:30:11.891 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:11.887+0000 7f7c5d7df640 1 --2- 172.21.15.37:0/909686071 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f7c44005250 0x7f7c4400b9c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:30:11.891 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:11.887+0000 7f7c57fff640 1 -- 172.21.15.37:0/909686071 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f7c440039b0 msgr2=0x7f7c44003e20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:30:11.891 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:11.887+0000 7f7c57fff640 1 --2- 172.21.15.37:0/909686071 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f7c440039b0 0x7f7c44003e20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:30:11.891 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:11.887+0000 7f7c57fff640 1 -- 172.21.15.37:0/909686071 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f7c44005250 msgr2=0x7f7c4400b9c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:30:11.891 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:11.887+0000 7f7c57fff640 1 --2- 172.21.15.37:0/909686071 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f7c44005250 0x7f7c4400b9c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:30:11.892 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:11.887+0000 7f7c5d7df640 1 -- 172.21.15.37:0/909686071 reap_dead start 2024-04-25T10:30:11.892 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:11.887+0000 7f7c57fff640 1 -- 172.21.15.37:0/909686071 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f7c44003520 con 0x7f7c44004800 2024-04-25T10:30:11.893 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:11.888+0000 7f7c57fff640 1 --2- 172.21.15.37:0/909686071 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f7c44004800 0x7f7c44004c10 secure :-1 s=READY pgs=418 cs=0 l=1 rev1=1 crypto rx=0x7f7c40009ac0 tx=0x7f7c44005830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=4747509718dbe05c server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:30:11.893 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:11.888+0000 7f7c577fe640 1 -- 172.21.15.37:0/909686071 <== mon.0 v2:172.21.15.37:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7c40036070 con 0x7f7c44004800 2024-04-25T10:30:11.893 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:11.888+0000 7f7c577fe640 1 -- 172.21.15.37:0/909686071 <== mon.0 v2:172.21.15.37:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f7c40031400 con 0x7f7c44004800 2024-04-25T10:30:11.893 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:11.888+0000 7f7c577fe640 1 -- 172.21.15.37:0/909686071 <== mon.0 v2:172.21.15.37:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7c40009690 con 0x7f7c44004800 2024-04-25T10:30:11.893 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:11.889+0000 7f7c5f242640 1 -- 172.21.15.37:0/909686071 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f7c44004800 msgr2=0x7f7c44004c10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:30:11.893 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:11.889+0000 7f7c5f242640 1 --2- 172.21.15.37:0/909686071 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f7c44004800 0x7f7c44004c10 secure :-1 s=READY pgs=418 cs=0 l=1 rev1=1 crypto rx=0x7f7c40009ac0 tx=0x7f7c44005830 comp rx=0 tx=0).stop 2024-04-25T10:30:11.893 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:11.889+0000 7f7c5f242640 1 -- 172.21.15.37:0/909686071 shutdown_connections 2024-04-25T10:30:11.893 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:11.889+0000 7f7c5f242640 1 --2- 172.21.15.37:0/909686071 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f7c44004800 0x7f7c44004c10 unknown :-1 s=CLOSED pgs=418 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:30:11.893 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:11.889+0000 7f7c5f242640 1 -- 172.21.15.37:0/909686071 >> 172.21.15.37:0/909686071 conn(0x7f7c580d27a0 msgr2=0x7f7c580d2bb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:30:11.893 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:11.889+0000 7f7c5f242640 1 -- 172.21.15.37:0/909686071 shutdown_connections 2024-04-25T10:30:11.894 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:11.889+0000 7f7c5f242640 1 -- 172.21.15.37:0/909686071 wait complete. 2024-04-25T10:30:11.894 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:11.891+0000 7f7c5f242640 1 Processor -- start 2024-04-25T10:30:11.894 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:11.891+0000 7f7c5f242640 1 -- start start 2024-04-25T10:30:11.895 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:11.892+0000 7f7c5f242640 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f7c44004800 0x7f7c58151ff0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:30:11.896 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:11.892+0000 7f7c5f242640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f7c58153580 0x7f7c580cecb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:30:11.896 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:11.892+0000 7f7c5f242640 1 --2- >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f7c580cf1f0 0x7f7c580b90a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:30:11.896 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:11.892+0000 7f7c5f242640 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 -- 0x7f7c58161350 con 0x7f7c58153580 2024-04-25T10:30:11.896 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:11.892+0000 7f7c5f242640 1 -- --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_getmap magic: 0 -- 0x7f7c580d4f10 con 0x7f7c44004800 2024-04-25T10:30:11.896 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:11.892+0000 7f7c5f242640 1 -- --> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] -- mon_getmap magic: 0 -- 0x7f7c581614d0 con 0x7f7c580cf1f0 2024-04-25T10:30:11.896 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:11.892+0000 7f7c57fff640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f7c58153580 0x7f7c580cecb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:30:11.896 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:11.892+0000 7f7c57fff640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f7c58153580 0x7f7c580cecb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.37:3300/0 says I am v2:172.21.15.37:52586/0 (socket says 172.21.15.37:52586) 2024-04-25T10:30:11.896 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:11.892+0000 7f7c57fff640 1 -- 172.21.15.37:0/211657747 learned_addr learned my addr 172.21.15.37:0/211657747 (peer_addr_for_me v2:172.21.15.37:0/0) 2024-04-25T10:30:11.896 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:11.892+0000 7f7c5d7df640 1 --2- 172.21.15.37:0/211657747 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f7c580cf1f0 0x7f7c580b90a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:30:11.896 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:11.893+0000 7f7c5cfde640 1 --2- 172.21.15.37:0/211657747 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f7c44004800 0x7f7c58151ff0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:30:11.896 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:11.893+0000 7f7c57fff640 1 -- 172.21.15.37:0/211657747 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f7c580cf1f0 msgr2=0x7f7c580b90a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:30:11.896 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:11.893+0000 7f7c57fff640 1 --2- 172.21.15.37:0/211657747 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f7c580cf1f0 0x7f7c580b90a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:30:11.896 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:11.893+0000 7f7c57fff640 1 -- 172.21.15.37:0/211657747 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f7c44004800 msgr2=0x7f7c58151ff0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:30:11.896 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:11.893+0000 7f7c57fff640 1 --2- 172.21.15.37:0/211657747 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f7c44004800 0x7f7c58151ff0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:30:11.896 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:11.893+0000 7f7c57fff640 1 -- 172.21.15.37:0/211657747 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f7c580cf600 con 0x7f7c58153580 2024-04-25T10:30:11.896 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:11.893+0000 7f7c5d7df640 1 --2- 172.21.15.37:0/211657747 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f7c580cf1f0 0x7f7c580b90a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T10:30:11.897 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:11.893+0000 7f7c57fff640 1 --2- 172.21.15.37:0/211657747 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f7c58153580 0x7f7c580cecb0 secure :-1 s=READY pgs=419 cs=0 l=1 rev1=1 crypto rx=0x7f7c40009be0 tx=0x7f7c40004730 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:30:11.897 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:11.893+0000 7f7c557fa640 1 -- 172.21.15.37:0/211657747 <== mon.0 v2:172.21.15.37:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7c4003f020 con 0x7f7c58153580 2024-04-25T10:30:11.897 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:11.893+0000 7f7c557fa640 1 -- 172.21.15.37:0/211657747 <== mon.0 v2:172.21.15.37:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f7c40037040 con 0x7f7c58153580 2024-04-25T10:30:11.897 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:11.893+0000 7f7c5f242640 1 -- 172.21.15.37:0/211657747 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f7c580cd260 con 0x7f7c58153580 2024-04-25T10:30:11.897 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:11.893+0000 7f7c557fa640 1 -- 172.21.15.37:0/211657747 <== mon.0 v2:172.21.15.37:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7c40036070 con 0x7f7c58153580 2024-04-25T10:30:11.897 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:11.893+0000 7f7c5f242640 1 -- 172.21.15.37:0/211657747 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f7c580cd820 con 0x7f7c58153580 2024-04-25T10:30:11.897 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:11.894+0000 7f7c5f242640 1 -- 172.21.15.37:0/211657747 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f7c5806a140 con 0x7f7c58153580 2024-04-25T10:30:11.897 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:11.895+0000 7f7c557fa640 1 -- 172.21.15.37:0/211657747 <== mon.0 v2:172.21.15.37:3300/0 4 ==== mgrmap(e 5) ==== 96782+0+0 (secure 0 0 0) 0x7f7c40042020 con 0x7f7c58153580 2024-04-25T10:30:11.898 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:11.896+0000 7f7c557fa640 1 --2- 172.21.15.37:0/211657747 >> [v2:172.21.15.203:6832/2418205237,v1:172.21.15.203:6833/2418205237] conn(0x7f7c380744b0 0x7f7c380769c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:30:11.898 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:11.896+0000 7f7c557fa640 1 -- 172.21.15.37:0/211657747 <== mon.0 v2:172.21.15.37:3300/0 5 ==== osd_map(34..34 src has 1..34) ==== 5609+0+0 (secure 0 0 0) 0x7f7c400b5640 con 0x7f7c58153580 2024-04-25T10:30:11.898 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:11.896+0000 7f7c5cfde640 1 --2- 172.21.15.37:0/211657747 >> [v2:172.21.15.203:6832/2418205237,v1:172.21.15.203:6833/2418205237] conn(0x7f7c380744b0 0x7f7c380769c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:30:11.899 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:11.897+0000 7f7c5cfde640 1 --2- 172.21.15.37:0/211657747 >> [v2:172.21.15.203:6832/2418205237,v1:172.21.15.203:6833/2418205237] conn(0x7f7c380744b0 0x7f7c380769c0 secure :-1 s=READY pgs=211 cs=0 l=1 rev1=1 crypto rx=0x7f7c500013d0 tx=0x7f7c50006040 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:30:11.902 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:11.900+0000 7f7c557fa640 1 -- 172.21.15.37:0/211657747 <== mon.0 v2:172.21.15.37:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187246 (secure 0 0 0) 0x7f7c40049030 con 0x7f7c58153580 2024-04-25T10:30:12.131 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:12.129+0000 7f7c5f242640 1 -- 172.21.15.37:0/211657747 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_command({"prefix": "fs new", "fs_name": "cephfs", "metadata": "cephfs_metadata", "data": "cephfs_data"} v 0) -- 0x7f7c580b43c0 con 0x7f7c58153580 2024-04-25T10:30:12.759 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:12.757+0000 7f7c557fa640 1 -- 172.21.15.37:0/211657747 <== mon.0 v2:172.21.15.37:3300/0 7 ==== mon_command_ack([{"prefix": "fs new", "fs_name": "cephfs", "metadata": "cephfs_metadata", "data": "cephfs_data"}]=0 new fs with metadata pool 4 and data pool 5 v13) ==== 172+0+0 (secure 0 0 0) 0x7f7c580cf600 con 0x7f7c58153580 2024-04-25T10:30:12.760 INFO:teuthology.orchestra.run.smithi037.stderr:new fs with metadata pool 4 and data pool 5 2024-04-25T10:30:12.778 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:12.774+0000 7f7c5f242640 1 -- 172.21.15.37:0/211657747 >> [v2:172.21.15.203:6832/2418205237,v1:172.21.15.203:6833/2418205237] conn(0x7f7c380744b0 msgr2=0x7f7c380769c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:30:12.779 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:12.774+0000 7f7c5f242640 1 --2- 172.21.15.37:0/211657747 >> [v2:172.21.15.203:6832/2418205237,v1:172.21.15.203:6833/2418205237] conn(0x7f7c380744b0 0x7f7c380769c0 secure :-1 s=READY pgs=211 cs=0 l=1 rev1=1 crypto rx=0x7f7c500013d0 tx=0x7f7c50006040 comp rx=0 tx=0).stop 2024-04-25T10:30:12.779 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:12.774+0000 7f7c5f242640 1 -- 172.21.15.37:0/211657747 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f7c58153580 msgr2=0x7f7c580cecb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:30:12.779 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:12.774+0000 7f7c5f242640 1 --2- 172.21.15.37:0/211657747 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f7c58153580 0x7f7c580cecb0 secure :-1 s=READY pgs=419 cs=0 l=1 rev1=1 crypto rx=0x7f7c40009be0 tx=0x7f7c40004730 comp rx=0 tx=0).stop 2024-04-25T10:30:12.779 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:12.777+0000 7f7c5f242640 1 -- 172.21.15.37:0/211657747 shutdown_connections 2024-04-25T10:30:12.779 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:12.777+0000 7f7c5f242640 1 --2- 172.21.15.37:0/211657747 >> [v2:172.21.15.203:6832/2418205237,v1:172.21.15.203:6833/2418205237] conn(0x7f7c380744b0 0x7f7c380769c0 unknown :-1 s=CLOSED pgs=211 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:30:12.779 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:12.777+0000 7f7c5f242640 1 --2- 172.21.15.37:0/211657747 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f7c580cf1f0 0x7f7c580b90a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:30:12.780 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:12.777+0000 7f7c5f242640 1 --2- 172.21.15.37:0/211657747 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f7c58153580 0x7f7c580cecb0 unknown :-1 s=CLOSED pgs=419 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:30:12.780 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:12.777+0000 7f7c5f242640 1 --2- 172.21.15.37:0/211657747 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f7c44004800 0x7f7c58151ff0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:30:12.780 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:12.777+0000 7f7c5f242640 1 -- 172.21.15.37:0/211657747 >> 172.21.15.37:0/211657747 conn(0x7f7c580d27a0 msgr2=0x7f7c580d6270 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:30:12.782 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:12.778+0000 7f7c5f242640 1 -- 172.21.15.37:0/211657747 shutdown_connections 2024-04-25T10:30:12.782 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:12.779+0000 7f7c5f242640 1 -- 172.21.15.37:0/211657747 wait complete. 2024-04-25T10:30:12.796 DEBUG:teuthology.orchestra.run.smithi037:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-04-25T10:30:12.909 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:12.907+0000 7f7db2896640 1 Processor -- start 2024-04-25T10:30:12.909 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:12.907+0000 7f7db2896640 1 -- start start 2024-04-25T10:30:12.911 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:12.908+0000 7f7db2896640 1 --2- >> v2:172.21.15.203:3300/0 conn(0x7f7dac1460e0 0x7f7dac06f100 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:30:12.911 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:12.908+0000 7f7db2896640 1 --2- >> v2:172.21.15.37:3300/0 conn(0x7f7dac1610b0 0x7f7dac0b6640 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:30:12.911 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:12.908+0000 7f7db2896640 1 -- --> v1:172.21.15.37:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f7dac1614c0 con 0x7f7dac143500 2024-04-25T10:30:12.911 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:12.908+0000 7f7db2896640 1 -- --> v2:172.21.15.37:3300/0 -- mon_getmap magic: 0 -- 0x7f7dac161730 con 0x7f7dac1610b0 2024-04-25T10:30:12.911 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:12.908+0000 7f7db2896640 1 -- --> v2:172.21.15.203:3300/0 -- mon_getmap magic: 0 -- 0x7f7dac0b5b60 con 0x7f7dac1460e0 2024-04-25T10:30:12.911 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:12.908+0000 7f7db0e33640 1 --2- >> v2:172.21.15.37:3300/0 conn(0x7f7dac1610b0 0x7f7dac0b6640 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:30:12.911 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:12.908+0000 7f7db0e33640 1 --2- >> v2:172.21.15.37:3300/0 conn(0x7f7dac1610b0 0x7f7dac0b6640 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.37:3300/0 says I am v2:172.21.15.37:52596/0 (socket says 172.21.15.37:52596) 2024-04-25T10:30:12.911 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:12.908+0000 7f7db0e33640 1 -- 172.21.15.37:0/2135990417 learned_addr learned my addr 172.21.15.37:0/2135990417 (peer_addr_for_me v2:172.21.15.37:0/0) 2024-04-25T10:30:12.911 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:12.908+0000 7f7dabfff640 1 --1- 172.21.15.37:0/2135990417 >> v1:172.21.15.37:6789/0 conn(0x7f7dac143500 0x7f7dac1609a0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.37:6789/0 says I am v1:172.21.15.37:41060/0 (socket says 172.21.15.37:41060) 2024-04-25T10:30:12.912 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:12.909+0000 7f7dab7fe640 1 --2- 172.21.15.37:0/2135990417 >> v2:172.21.15.203:3300/0 conn(0x7f7dac1460e0 0x7f7dac06f100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:30:12.912 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:12.909+0000 7f7db0e33640 1 -- 172.21.15.37:0/2135990417 >> v2:172.21.15.203:3300/0 conn(0x7f7dac1460e0 msgr2=0x7f7dac06f100 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:30:12.912 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:12.909+0000 7f7db0e33640 1 --2- 172.21.15.37:0/2135990417 >> v2:172.21.15.203:3300/0 conn(0x7f7dac1460e0 0x7f7dac06f100 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:30:12.912 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:12.909+0000 7f7db0e33640 1 -- 172.21.15.37:0/2135990417 >> v1:172.21.15.37:6789/0 conn(0x7f7dac143500 legacy=0x7f7dac1609a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:30:12.912 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:12.909+0000 7f7db0e33640 1 -- 172.21.15.37:0/2135990417 --> v2:172.21.15.37:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f7dac0b5e40 con 0x7f7dac1610b0 2024-04-25T10:30:12.913 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:12.909+0000 7f7db0e33640 1 --2- 172.21.15.37:0/2135990417 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f7dac1610b0 0x7f7dac0b6640 secure :-1 s=READY pgs=421 cs=0 l=1 rev1=1 crypto rx=0x7f7dac0725f0 tx=0x7f7da402a730 comp rx=0 tx=0).ready entity=mon.0 client_cookie=448e6d56331fe9c7 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:30:12.913 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:12.910+0000 7f7daaffd640 1 -- 172.21.15.37:0/2135990417 <== mon.0 v2:172.21.15.37:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7da4036070 con 0x7f7dac1610b0 2024-04-25T10:30:12.913 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:12.910+0000 7f7daaffd640 1 -- 172.21.15.37:0/2135990417 <== mon.0 v2:172.21.15.37:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f7da4031600 con 0x7f7dac1610b0 2024-04-25T10:30:12.913 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:12.910+0000 7f7daaffd640 1 -- 172.21.15.37:0/2135990417 <== mon.0 v2:172.21.15.37:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7da40319d0 con 0x7f7dac1610b0 2024-04-25T10:30:12.913 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:12.910+0000 7f7db2896640 1 -- 172.21.15.37:0/2135990417 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f7dac1610b0 msgr2=0x7f7dac0b6640 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:30:12.914 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:12.910+0000 7f7db2896640 1 --2- 172.21.15.37:0/2135990417 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f7dac1610b0 0x7f7dac0b6640 secure :-1 s=READY pgs=421 cs=0 l=1 rev1=1 crypto rx=0x7f7dac0725f0 tx=0x7f7da402a730 comp rx=0 tx=0).stop 2024-04-25T10:30:12.914 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:12.910+0000 7f7db2896640 1 -- 172.21.15.37:0/2135990417 shutdown_connections 2024-04-25T10:30:12.914 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:12.910+0000 7f7db2896640 1 --2- 172.21.15.37:0/2135990417 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f7dac1610b0 0x7f7dac0b6640 unknown :-1 s=CLOSED pgs=421 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:30:12.914 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:12.910+0000 7f7db2896640 1 --2- 172.21.15.37:0/2135990417 >> v2:172.21.15.203:3300/0 conn(0x7f7dac1460e0 0x7f7dac06f100 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:30:12.914 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:12.910+0000 7f7db2896640 1 -- 172.21.15.37:0/2135990417 >> 172.21.15.37:0/2135990417 conn(0x7f7dac0b36b0 msgr2=0x7f7dac0b3ac0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:30:12.914 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:12.910+0000 7f7db2896640 1 -- 172.21.15.37:0/2135990417 shutdown_connections 2024-04-25T10:30:12.914 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:12.910+0000 7f7db2896640 1 -- 172.21.15.37:0/2135990417 wait complete. 2024-04-25T10:30:12.915 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:12.912+0000 7f7db2896640 1 Processor -- start 2024-04-25T10:30:12.915 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:12.912+0000 7f7db2896640 1 -- start start 2024-04-25T10:30:12.917 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:12.913+0000 7f7db2896640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f7dac1460e0 0x7f7dac15fc20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:30:12.917 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:12.914+0000 7f7db2896640 1 --2- >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f7dac1610b0 0x7f7dac06fb70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:30:12.917 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:12.914+0000 7f7db2896640 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f7dac0700b0 0x7f7dac15c500 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:30:12.917 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:12.914+0000 7f7db2896640 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 -- 0x7f7dac143cb0 con 0x7f7dac1460e0 2024-04-25T10:30:12.917 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:12.914+0000 7f7db2896640 1 -- --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_getmap magic: 0 -- 0x7f7dac143b30 con 0x7f7dac0700b0 2024-04-25T10:30:12.917 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:12.914+0000 7f7db2896640 1 -- --> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] -- mon_getmap magic: 0 -- 0x7f7dac143e30 con 0x7f7dac1610b0 2024-04-25T10:30:12.917 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:12.914+0000 7f7dabfff640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f7dac1460e0 0x7f7dac15fc20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:30:12.918 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:12.914+0000 7f7dabfff640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f7dac1460e0 0x7f7dac15fc20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.37:3300/0 says I am v2:172.21.15.37:52598/0 (socket says 172.21.15.37:52598) 2024-04-25T10:30:12.918 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:12.914+0000 7f7dabfff640 1 -- 172.21.15.37:0/4195219098 learned_addr learned my addr 172.21.15.37:0/4195219098 (peer_addr_for_me v2:172.21.15.37:0/0) 2024-04-25T10:30:12.918 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:12.914+0000 7f7dab7fe640 1 --2- 172.21.15.37:0/4195219098 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f7dac1610b0 0x7f7dac06fb70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:30:12.918 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:12.914+0000 7f7db0e33640 1 --2- 172.21.15.37:0/4195219098 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f7dac0700b0 0x7f7dac15c500 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:30:12.918 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:12.914+0000 7f7dabfff640 1 -- 172.21.15.37:0/4195219098 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f7dac1610b0 msgr2=0x7f7dac06fb70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:30:12.918 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:12.914+0000 7f7dabfff640 1 --2- 172.21.15.37:0/4195219098 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f7dac1610b0 0x7f7dac06fb70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:30:12.918 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:12.914+0000 7f7dabfff640 1 -- 172.21.15.37:0/4195219098 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f7dac0700b0 msgr2=0x7f7dac15c500 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:30:12.918 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:12.914+0000 7f7dabfff640 1 --2- 172.21.15.37:0/4195219098 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f7dac0700b0 0x7f7dac15c500 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:30:12.918 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:12.914+0000 7f7dabfff640 1 -- 172.21.15.37:0/4195219098 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f7dac12f850 con 0x7f7dac1460e0 2024-04-25T10:30:12.918 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:12.914+0000 7f7db0e33640 1 --2- 172.21.15.37:0/4195219098 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f7dac0700b0 0x7f7dac15c500 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T10:30:12.918 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:12.914+0000 7f7dabfff640 1 --2- 172.21.15.37:0/4195219098 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f7dac1460e0 0x7f7dac15fc20 secure :-1 s=READY pgs=422 cs=0 l=1 rev1=1 crypto rx=0x7f7d9c00c3f0 tx=0x7f7d9c00c8b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:30:12.918 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:12.914+0000 7f7da8ff9640 1 -- 172.21.15.37:0/4195219098 <== mon.0 v2:172.21.15.37:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7d9c0074e0 con 0x7f7dac1460e0 2024-04-25T10:30:12.918 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:12.914+0000 7f7da8ff9640 1 -- 172.21.15.37:0/4195219098 <== mon.0 v2:172.21.15.37:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f7d9c007680 con 0x7f7dac1460e0 2024-04-25T10:30:12.919 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:12.914+0000 7f7db2896640 1 -- 172.21.15.37:0/4195219098 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f7dac1f99d0 con 0x7f7dac1460e0 2024-04-25T10:30:12.919 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:12.914+0000 7f7da8ff9640 1 -- 172.21.15.37:0/4195219098 <== mon.0 v2:172.21.15.37:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7d9c004a00 con 0x7f7dac1460e0 2024-04-25T10:30:12.919 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:12.914+0000 7f7db2896640 1 -- 172.21.15.37:0/4195219098 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f7dac1f9f90 con 0x7f7dac1460e0 2024-04-25T10:30:12.919 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:12.915+0000 7f7db2896640 1 -- 172.21.15.37:0/4195219098 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f7dac06a140 con 0x7f7dac1460e0 2024-04-25T10:30:12.920 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:12.916+0000 7f7da8ff9640 1 -- 172.21.15.37:0/4195219098 <== mon.0 v2:172.21.15.37:3300/0 4 ==== mgrmap(e 5) ==== 96782+0+0 (secure 0 0 0) 0x7f7d9c01a020 con 0x7f7dac1460e0 2024-04-25T10:30:12.920 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:12.917+0000 7f7da8ff9640 1 --2- 172.21.15.37:0/4195219098 >> [v2:172.21.15.203:6832/2418205237,v1:172.21.15.203:6833/2418205237] conn(0x7f7d8c0744b0 0x7f7d8c0769c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:30:12.920 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:12.917+0000 7f7da8ff9640 1 -- 172.21.15.37:0/4195219098 <== mon.0 v2:172.21.15.37:3300/0 5 ==== osd_map(35..35 src has 1..35) ==== 5709+0+0 (secure 0 0 0) 0x7f7d9c094a30 con 0x7f7dac1460e0 2024-04-25T10:30:12.920 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:12.917+0000 7f7dab7fe640 1 --2- 172.21.15.37:0/4195219098 >> [v2:172.21.15.203:6832/2418205237,v1:172.21.15.203:6833/2418205237] conn(0x7f7d8c0744b0 0x7f7d8c0769c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:30:12.920 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:12.917+0000 7f7dab7fe640 1 --2- 172.21.15.37:0/4195219098 >> [v2:172.21.15.203:6832/2418205237,v1:172.21.15.203:6833/2418205237] conn(0x7f7d8c0744b0 0x7f7d8c0769c0 secure :-1 s=READY pgs=212 cs=0 l=1 rev1=1 crypto rx=0x7f7d94006fd0 tx=0x7f7d94008040 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:30:12.923 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:12.921+0000 7f7da8ff9640 1 -- 172.21.15.37:0/4195219098 <== mon.0 v2:172.21.15.37:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187246 (secure 0 0 0) 0x7f7dac12f850 con 0x7f7dac1460e0 2024-04-25T10:30:13.110 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:13.108+0000 7f7db2896640 1 -- 172.21.15.37:0/4195219098 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7f7dac144200 con 0x7f7dac1460e0 2024-04-25T10:30:13.111 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:13.109+0000 7f7da8ff9640 1 -- 172.21.15.37:0/4195219098 <== mon.0 v2:172.21.15.37:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v35) ==== 74+0+16634 (secure 0 0 0) 0x7f7d9c05f140 con 0x7f7dac1460e0 2024-04-25T10:30:13.112 INFO:teuthology.orchestra.run.smithi037.stdout: 2024-04-25T10:30:13.112 INFO:teuthology.orchestra.run.smithi037.stdout:{"epoch":35,"fsid":"f4acda28-be34-492a-b286-994afd36f61d","created":"2024-04-25T10:28:43.985718+0000","modified":"2024-04-25T10:30:12.130516+0000","last_up_change":"2024-04-25T10:28:54.900654+0000","last_in_change":"2024-04-25T10:28:46.216973+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":5,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-04-25T10:28:56.202153+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"19","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":4,"pool_name":"cephfs_metadata","create_time":"2024-04-25T10:30:10.008132+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"35","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.75,"score_stable":1.75,"optimal_score":1,"raw_score_acting":1.75,"raw_score_stable":1.75,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":5,"pool_name":"cephfs_data","create_time":"2024-04-25T10:30:11.084827+0000","flags":32769,"flags_names":"hashpspool,creating","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"35","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"a2f79028-b332-4166-9ebb-a7024a37ea22","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":34,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.37:6816","nonce":2746074720},{"type":"v1","addr":"172.21.15.37:6817","nonce":2746074720}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.37:6818","nonce":2746074720},{"type":"v1","addr":"172.21.15.37:6819","nonce":2746074720}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.37:6822","nonce":2746074720},{"type":"v1","addr":"172.21.15.37:6823","nonce":2746074720}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.37:6820","nonce":2746074720},{"type":"v1","addr":"172.21.15.37:6821","nonce":2746074720}]},"public_addr":"172.21.15.37:6817/2746074720","cluster_addr":"172.21.15.37:6819/2746074720","heartbeat_back_addr":"172.21.15.37:6823/2746074720","heartbeat_front_addr":"172.21.15.37:6821/2746074720","state":["exists","up"]},{"osd":1,"uuid":"973159de-c2ea-4083-aecc-69373d1c1132","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":34,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.37:6800","nonce":1690341662},{"type":"v1","addr":"172.21.15.37:6801","nonce":1690341662}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.37:6802","nonce":1690341662},{"type":"v1","addr":"172.21.15.37:6803","nonce":1690341662}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.37:6806","nonce":1690341662},{"type":"v1","addr":"172.21.15.37:6807","nonce":1690341662}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.37:6804","nonce":1690341662},{"type":"v1","addr":"172.21.15.37:6805","nonce":1690341662}]},"public_addr":"172.21.15.37:6801/1690341662","cluster_addr":"172.21.15.37:6803/1690341662","heartbeat_back_addr":"172.21.15.37:6807/1690341662","heartbeat_front_addr":"172.21.15.37:6805/1690341662","state":["exists","up"]},{"osd":2,"uuid":"4ae7a0e9-cb3f-484c-a75f-f3766b721c47","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":34,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.37:6808","nonce":2754669649},{"type":"v1","addr":"172.21.15.37:6809","nonce":2754669649}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.37:6810","nonce":2754669649},{"type":"v1","addr":"172.21.15.37:6811","nonce":2754669649}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.37:6814","nonce":2754669649},{"type":"v1","addr":"172.21.15.37:6815","nonce":2754669649}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.37:6812","nonce":2754669649},{"type":"v1","addr":"172.21.15.37:6813","nonce":2754669649}]},"public_addr":"172.21.15.37:6809/2754669649","cluster_addr":"172.21.15.37:6811/2754669649","heartbeat_back_addr":"172.21.15.37:6815/2754669649","heartbeat_front_addr":"172.21.15.37:6813/2754669649","state":["exists","up"]},{"osd":3,"uuid":"1ba4a746-1608-47a7-ace2-960a5cfae934","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":34,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.37:6824","nonce":1815290618},{"type":"v1","addr":"172.21.15.37:6825","nonce":1815290618}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.37:6826","nonce":1815290618},{"type":"v1","addr":"172.21.15.37:6827","nonce":1815290618}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.37:6830","nonce":1815290618},{"type":"v1","addr":"172.21.15.37:6831","nonce":1815290618}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.37:6828","nonce":1815290618},{"type":"v1","addr":"172.21.15.37:6829","nonce":1815290618}]},"public_addr":"172.21.15.37:6825/1815290618","cluster_addr":"172.21.15.37:6827/1815290618","heartbeat_back_addr":"172.21.15.37:6831/1815290618","heartbeat_front_addr":"172.21.15.37:6829/1815290618","state":["exists","up"]},{"osd":4,"uuid":"d4f59f5d-0413-4b79-87ee-bb212c1fcf62","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":34,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6800","nonce":4051532063},{"type":"v1","addr":"172.21.15.203:6801","nonce":4051532063}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6802","nonce":4051532063},{"type":"v1","addr":"172.21.15.203:6803","nonce":4051532063}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6806","nonce":4051532063},{"type":"v1","addr":"172.21.15.203:6807","nonce":4051532063}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6804","nonce":4051532063},{"type":"v1","addr":"172.21.15.203:6805","nonce":4051532063}]},"public_addr":"172.21.15.203:6801/4051532063","cluster_addr":"172.21.15.203:6803/4051532063","heartbeat_back_addr":"172.21.15.203:6807/4051532063","heartbeat_front_addr":"172.21.15.203:6805/4051532063","state":["exists","up"]},{"osd":5,"uuid":"69710a0e-41d8-4e7f-9a10-17ce540b67be","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":34,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6824","nonce":1203822890},{"type":"v1","addr":"172.21.15.203:6825","nonce":1203822890}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6826","nonce":1203822890},{"type":"v1","addr":"172.21.15.203:6827","nonce":1203822890}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6830","nonce":1203822890},{"type":"v1","addr":"172.21.15.203:6831","nonce":1203822890}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6828","nonce":1203822890},{"type":"v1","addr":"172.21.15.203:6829","nonce":1203822890}]},"public_addr":"172.21.15.203:6825/1203822890","cluster_addr":"172.21.15.203:6827/1203822890","heartbeat_back_addr":"172.21.15.203:6831/1203822890","heartbeat_front_addr":"172.21.15.203:6829/1203822890","state":["exists","up"]},{"osd":6,"uuid":"f29dd047-3b48-473d-a235-6642a6293f20","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":34,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6810","nonce":2309615014},{"type":"v1","addr":"172.21.15.203:6812","nonce":2309615014}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6814","nonce":2309615014},{"type":"v1","addr":"172.21.15.203:6816","nonce":2309615014}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6822","nonce":2309615014},{"type":"v1","addr":"172.21.15.203:6823","nonce":2309615014}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6818","nonce":2309615014},{"type":"v1","addr":"172.21.15.203:6820","nonce":2309615014}]},"public_addr":"172.21.15.203:6812/2309615014","cluster_addr":"172.21.15.203:6816/2309615014","heartbeat_back_addr":"172.21.15.203:6823/2309615014","heartbeat_front_addr":"172.21.15.203:6820/2309615014","state":["exists","up"]},{"osd":7,"uuid":"6e691644-378b-4fdc-8e86-fb5f56e31f29","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":34,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6808","nonce":3688099014},{"type":"v1","addr":"172.21.15.203:6809","nonce":3688099014}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6811","nonce":3688099014},{"type":"v1","addr":"172.21.15.203:6813","nonce":3688099014}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6819","nonce":3688099014},{"type":"v1","addr":"172.21.15.203:6821","nonce":3688099014}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6815","nonce":3688099014},{"type":"v1","addr":"172.21.15.203:6817","nonce":3688099014}]},"public_addr":"172.21.15.203:6809/3688099014","cluster_addr":"172.21.15.203:6813/3688099014","heartbeat_back_addr":"172.21.15.203:6821/3688099014","heartbeat_front_addr":"172.21.15.203:6817/3688099014","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T10:28:51.419141+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T10:28:51.366448+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T10:28:51.348276+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T10:28:51.380519+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T10:28:51.105812+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T10:28:51.384496+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T10:28:51.388755+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T10:28:51.366958+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-04-25T10:30:13.114 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:13.111+0000 7f7db2896640 1 -- 172.21.15.37:0/4195219098 >> [v2:172.21.15.203:6832/2418205237,v1:172.21.15.203:6833/2418205237] conn(0x7f7d8c0744b0 msgr2=0x7f7d8c0769c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:30:13.114 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:13.111+0000 7f7db2896640 1 --2- 172.21.15.37:0/4195219098 >> [v2:172.21.15.203:6832/2418205237,v1:172.21.15.203:6833/2418205237] conn(0x7f7d8c0744b0 0x7f7d8c0769c0 secure :-1 s=READY pgs=212 cs=0 l=1 rev1=1 crypto rx=0x7f7d94006fd0 tx=0x7f7d94008040 comp rx=0 tx=0).stop 2024-04-25T10:30:13.114 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:13.111+0000 7f7db2896640 1 -- 172.21.15.37:0/4195219098 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f7dac1460e0 msgr2=0x7f7dac15fc20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:30:13.114 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:13.111+0000 7f7db2896640 1 --2- 172.21.15.37:0/4195219098 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f7dac1460e0 0x7f7dac15fc20 secure :-1 s=READY pgs=422 cs=0 l=1 rev1=1 crypto rx=0x7f7d9c00c3f0 tx=0x7f7d9c00c8b0 comp rx=0 tx=0).stop 2024-04-25T10:30:13.114 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:13.111+0000 7f7db2896640 1 -- 172.21.15.37:0/4195219098 shutdown_connections 2024-04-25T10:30:13.114 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:13.111+0000 7f7db2896640 1 --2- 172.21.15.37:0/4195219098 >> [v2:172.21.15.203:6832/2418205237,v1:172.21.15.203:6833/2418205237] conn(0x7f7d8c0744b0 0x7f7d8c0769c0 unknown :-1 s=CLOSED pgs=212 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:30:13.114 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:13.111+0000 7f7db2896640 1 --2- 172.21.15.37:0/4195219098 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f7dac0700b0 0x7f7dac15c500 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:30:13.114 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:13.111+0000 7f7db2896640 1 --2- 172.21.15.37:0/4195219098 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f7dac1610b0 0x7f7dac06fb70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:30:13.114 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:13.111+0000 7f7db2896640 1 --2- 172.21.15.37:0/4195219098 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f7dac1460e0 0x7f7dac15fc20 unknown :-1 s=CLOSED pgs=422 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:30:13.115 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:13.111+0000 7f7db2896640 1 -- 172.21.15.37:0/4195219098 >> 172.21.15.37:0/4195219098 conn(0x7f7dac0b36b0 msgr2=0x7f7dac1608d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:30:13.115 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:13.111+0000 7f7db2896640 1 -- 172.21.15.37:0/4195219098 shutdown_connections 2024-04-25T10:30:13.115 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:13.111+0000 7f7db2896640 1 -- 172.21.15.37:0/4195219098 wait complete. 2024-04-25T10:30:13.125 DEBUG:teuthology.orchestra.run.smithi037:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-04-25T10:30:13.237 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:13.234+0000 7f893196c640 1 Processor -- start 2024-04-25T10:30:13.237 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:13.234+0000 7f893196c640 1 -- start start 2024-04-25T10:30:13.238 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:13.236+0000 7f893196c640 1 --2- >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f892c0d5160 0x7f892c0b4e80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:30:13.238 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:13.236+0000 7f893196c640 1 --2- >> v2:172.21.15.37:3300/0 conn(0x7f892c0d0990 0x7f892c0d2b20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:30:13.238 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:13.236+0000 7f893196c640 1 -- --> v1:172.21.15.203:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f892c0d1370 con 0x7f892c0d0da0 2024-04-25T10:30:13.239 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:13.236+0000 7f893196c640 1 -- --> v2:172.21.15.37:3300/0 -- mon_getmap magic: 0 -- 0x7f892c0d1650 con 0x7f892c0d0990 2024-04-25T10:30:13.239 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:13.236+0000 7f893196c640 1 -- --> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] -- mon_getmap magic: 0 -- 0x7f892c0d4680 con 0x7f892c0d5160 2024-04-25T10:30:13.239 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:13.236+0000 7f892a7fc640 1 --2- >> v2:172.21.15.37:3300/0 conn(0x7f892c0d0990 0x7f892c0d2b20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:30:13.239 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:13.236+0000 7f892a7fc640 1 --2- >> v2:172.21.15.37:3300/0 conn(0x7f892c0d0990 0x7f892c0d2b20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.37:3300/0 says I am v2:172.21.15.37:52612/0 (socket says 172.21.15.37:52612) 2024-04-25T10:30:13.239 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:13.236+0000 7f892a7fc640 1 -- 172.21.15.37:0/194251150 learned_addr learned my addr 172.21.15.37:0/194251150 (peer_addr_for_me v2:172.21.15.37:0/0) 2024-04-25T10:30:13.239 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:13.236+0000 7f892affd640 1 --2- 172.21.15.37:0/194251150 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f892c0d5160 0x7f892c0b4e80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:30:13.239 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:13.236+0000 7f892a7fc640 1 -- 172.21.15.37:0/194251150 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f892c0d5160 msgr2=0x7f892c0b4e80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:30:13.239 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:13.236+0000 7f892a7fc640 1 --2- 172.21.15.37:0/194251150 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f892c0d5160 0x7f892c0b4e80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:30:13.239 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:13.236+0000 7f892a7fc640 1 -- 172.21.15.37:0/194251150 >> v1:172.21.15.203:6789/0 conn(0x7f892c0d0da0 legacy=0x7f892c06db10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:30:13.239 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:13.236+0000 7f892a7fc640 1 -- 172.21.15.37:0/194251150 --> v2:172.21.15.37:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f892c0d4960 con 0x7f892c0d0990 2024-04-25T10:30:13.240 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:13.237+0000 7f892a7fc640 1 --2- 172.21.15.37:0/194251150 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f892c0d0990 0x7f892c0d2b20 secure :-1 s=READY pgs=423 cs=0 l=1 rev1=1 crypto rx=0x7f8914009870 tx=0x7f891402a130 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e7610edd8cad6400 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:30:13.240 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:13.237+0000 7f8929ffb640 1 -- 172.21.15.37:0/194251150 <== mon.0 v2:172.21.15.37:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8914037070 con 0x7f892c0d0990 2024-04-25T10:30:13.240 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:13.237+0000 7f8929ffb640 1 -- 172.21.15.37:0/194251150 <== mon.0 v2:172.21.15.37:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f8914032440 con 0x7f892c0d0990 2024-04-25T10:30:13.241 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:13.237+0000 7f8929ffb640 1 -- 172.21.15.37:0/194251150 <== mon.0 v2:172.21.15.37:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8914032810 con 0x7f892c0d0990 2024-04-25T10:30:13.241 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:13.238+0000 7f893196c640 1 -- 172.21.15.37:0/194251150 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f892c0d0990 msgr2=0x7f892c0d2b20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:30:13.241 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:13.238+0000 7f893196c640 1 --2- 172.21.15.37:0/194251150 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f892c0d0990 0x7f892c0d2b20 secure :-1 s=READY pgs=423 cs=0 l=1 rev1=1 crypto rx=0x7f8914009870 tx=0x7f891402a130 comp rx=0 tx=0).stop 2024-04-25T10:30:13.241 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:13.238+0000 7f893196c640 1 -- 172.21.15.37:0/194251150 shutdown_connections 2024-04-25T10:30:13.241 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:13.238+0000 7f893196c640 1 --2- 172.21.15.37:0/194251150 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f892c0d0990 0x7f892c0d2b20 unknown :-1 s=CLOSED pgs=423 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:30:13.241 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:13.238+0000 7f893196c640 1 --2- 172.21.15.37:0/194251150 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f892c0d5160 0x7f892c0b4e80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:30:13.241 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:13.238+0000 7f893196c640 1 -- 172.21.15.37:0/194251150 >> 172.21.15.37:0/194251150 conn(0x7f892c0b7a80 msgr2=0x7f892c0b7e90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:30:13.241 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:13.238+0000 7f893196c640 1 -- 172.21.15.37:0/194251150 shutdown_connections 2024-04-25T10:30:13.241 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:13.238+0000 7f893196c640 1 -- 172.21.15.37:0/194251150 wait complete. 2024-04-25T10:30:13.242 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:13.240+0000 7f893196c640 1 Processor -- start 2024-04-25T10:30:13.242 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:13.240+0000 7f893196c640 1 -- start start 2024-04-25T10:30:13.245 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:13.241+0000 7f893196c640 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f892c0d0990 0x7f892c1565a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:30:13.245 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:13.241+0000 7f893196c640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f892c0d0da0 0x7f892c15bd80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:30:13.245 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:13.241+0000 7f893196c640 1 --2- >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f892c0d5160 0x7f892c157430 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:30:13.245 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:13.241+0000 7f893196c640 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 -- 0x7f892c164c40 con 0x7f892c0d0da0 2024-04-25T10:30:13.245 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:13.241+0000 7f893196c640 1 -- --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_getmap magic: 0 -- 0x7f892c164ac0 con 0x7f892c0d0990 2024-04-25T10:30:13.245 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:13.241+0000 7f893196c640 1 -- --> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] -- mon_getmap magic: 0 -- 0x7f892c0d3060 con 0x7f892c0d5160 2024-04-25T10:30:13.245 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:13.241+0000 7f892a7fc640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f892c0d0da0 0x7f892c15bd80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:30:13.245 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:13.241+0000 7f892a7fc640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f892c0d0da0 0x7f892c15bd80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.37:3300/0 says I am v2:172.21.15.37:52614/0 (socket says 172.21.15.37:52614) 2024-04-25T10:30:13.245 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:13.241+0000 7f892a7fc640 1 -- 172.21.15.37:0/582601766 learned_addr learned my addr 172.21.15.37:0/582601766 (peer_addr_for_me v2:172.21.15.37:0/0) 2024-04-25T10:30:13.245 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:13.242+0000 7f892affd640 1 --2- 172.21.15.37:0/582601766 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f892c0d0990 0x7f892c1565a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:30:13.245 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:13.242+0000 7f892b7fe640 1 --2- 172.21.15.37:0/582601766 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f892c0d5160 0x7f892c157430 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:30:13.245 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:13.242+0000 7f892a7fc640 1 -- 172.21.15.37:0/582601766 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f892c0d5160 msgr2=0x7f892c157430 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:30:13.246 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:13.242+0000 7f892a7fc640 1 --2- 172.21.15.37:0/582601766 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f892c0d5160 0x7f892c157430 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:30:13.246 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:13.242+0000 7f892a7fc640 1 -- 172.21.15.37:0/582601766 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f892c0d0990 msgr2=0x7f892c1565a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:30:13.246 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:13.242+0000 7f892a7fc640 1 --2- 172.21.15.37:0/582601766 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f892c0d0990 0x7f892c1565a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:30:13.246 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:13.242+0000 7f892a7fc640 1 -- 172.21.15.37:0/582601766 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f892c156ae0 con 0x7f892c0d0da0 2024-04-25T10:30:13.246 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:13.242+0000 7f892b7fe640 1 --2- 172.21.15.37:0/582601766 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f892c0d5160 0x7f892c157430 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T10:30:13.246 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:13.242+0000 7f892affd640 1 --2- 172.21.15.37:0/582601766 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f892c0d0990 0x7f892c1565a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T10:30:13.246 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:13.242+0000 7f892a7fc640 1 --2- 172.21.15.37:0/582601766 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f892c0d0da0 0x7f892c15bd80 secure :-1 s=READY pgs=424 cs=0 l=1 rev1=1 crypto rx=0x7f8914000c00 tx=0x7f89140046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:30:13.246 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:13.242+0000 7f8923fff640 1 -- 172.21.15.37:0/582601766 <== mon.0 v2:172.21.15.37:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8914037040 con 0x7f892c0d0da0 2024-04-25T10:30:13.246 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:13.242+0000 7f8923fff640 1 -- 172.21.15.37:0/582601766 <== mon.0 v2:172.21.15.37:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f8914042070 con 0x7f892c0d0da0 2024-04-25T10:30:13.246 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:13.242+0000 7f893196c640 1 -- 172.21.15.37:0/582601766 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f892c159560 con 0x7f892c0d0da0 2024-04-25T10:30:13.246 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:13.242+0000 7f8923fff640 1 -- 172.21.15.37:0/582601766 <== mon.0 v2:172.21.15.37:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8914030880 con 0x7f892c0d0da0 2024-04-25T10:30:13.246 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:13.242+0000 7f893196c640 1 -- 172.21.15.37:0/582601766 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f892c155da0 con 0x7f892c0d0da0 2024-04-25T10:30:13.247 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:13.243+0000 7f893196c640 1 -- 172.21.15.37:0/582601766 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f892c14b290 con 0x7f892c0d0da0 2024-04-25T10:30:13.247 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:13.244+0000 7f8923fff640 1 -- 172.21.15.37:0/582601766 <== mon.0 v2:172.21.15.37:3300/0 4 ==== mgrmap(e 5) ==== 96782+0+0 (secure 0 0 0) 0x7f8914030a20 con 0x7f892c0d0da0 2024-04-25T10:30:13.247 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:13.244+0000 7f8923fff640 1 --2- 172.21.15.37:0/582601766 >> [v2:172.21.15.203:6832/2418205237,v1:172.21.15.203:6833/2418205237] conn(0x7f890c0744b0 0x7f890c0769c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:30:13.247 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:13.244+0000 7f8923fff640 1 -- 172.21.15.37:0/582601766 <== mon.0 v2:172.21.15.37:3300/0 5 ==== osd_map(35..35 src has 1..35) ==== 5709+0+0 (secure 0 0 0) 0x7f89140b5a10 con 0x7f892c0d0da0 2024-04-25T10:30:13.248 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:13.245+0000 7f892affd640 1 --2- 172.21.15.37:0/582601766 >> [v2:172.21.15.203:6832/2418205237,v1:172.21.15.203:6833/2418205237] conn(0x7f890c0744b0 0x7f890c0769c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:30:13.248 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:13.245+0000 7f892affd640 1 --2- 172.21.15.37:0/582601766 >> [v2:172.21.15.203:6832/2418205237,v1:172.21.15.203:6833/2418205237] conn(0x7f890c0744b0 0x7f890c0769c0 secure :-1 s=READY pgs=213 cs=0 l=1 rev1=1 crypto rx=0x7f8924009a20 tx=0x7f8924008040 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:30:13.251 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:13.249+0000 7f8923fff640 1 -- 172.21.15.37:0/582601766 <== mon.0 v2:172.21.15.37:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187246 (secure 0 0 0) 0x7f89140802c0 con 0x7f892c0d0da0 2024-04-25T10:30:13.436 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:13.434+0000 7f893196c640 1 -- 172.21.15.37:0/582601766 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7f892c1592f0 con 0x7f892c0d0da0 2024-04-25T10:30:13.437 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:13.435+0000 7f8923fff640 1 -- 172.21.15.37:0/582601766 <== mon.0 v2:172.21.15.37:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v35) ==== 74+0+16634 (secure 0 0 0) 0x7f89140882b0 con 0x7f892c0d0da0 2024-04-25T10:30:13.437 INFO:teuthology.orchestra.run.smithi037.stdout: 2024-04-25T10:30:13.437 INFO:teuthology.orchestra.run.smithi037.stdout:{"epoch":35,"fsid":"f4acda28-be34-492a-b286-994afd36f61d","created":"2024-04-25T10:28:43.985718+0000","modified":"2024-04-25T10:30:12.130516+0000","last_up_change":"2024-04-25T10:28:54.900654+0000","last_in_change":"2024-04-25T10:28:46.216973+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":5,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-04-25T10:28:56.202153+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"19","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":4,"pool_name":"cephfs_metadata","create_time":"2024-04-25T10:30:10.008132+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"35","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.75,"score_stable":1.75,"optimal_score":1,"raw_score_acting":1.75,"raw_score_stable":1.75,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":5,"pool_name":"cephfs_data","create_time":"2024-04-25T10:30:11.084827+0000","flags":32769,"flags_names":"hashpspool,creating","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"35","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"a2f79028-b332-4166-9ebb-a7024a37ea22","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":34,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.37:6816","nonce":2746074720},{"type":"v1","addr":"172.21.15.37:6817","nonce":2746074720}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.37:6818","nonce":2746074720},{"type":"v1","addr":"172.21.15.37:6819","nonce":2746074720}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.37:6822","nonce":2746074720},{"type":"v1","addr":"172.21.15.37:6823","nonce":2746074720}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.37:6820","nonce":2746074720},{"type":"v1","addr":"172.21.15.37:6821","nonce":2746074720}]},"public_addr":"172.21.15.37:6817/2746074720","cluster_addr":"172.21.15.37:6819/2746074720","heartbeat_back_addr":"172.21.15.37:6823/2746074720","heartbeat_front_addr":"172.21.15.37:6821/2746074720","state":["exists","up"]},{"osd":1,"uuid":"973159de-c2ea-4083-aecc-69373d1c1132","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":34,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.37:6800","nonce":1690341662},{"type":"v1","addr":"172.21.15.37:6801","nonce":1690341662}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.37:6802","nonce":1690341662},{"type":"v1","addr":"172.21.15.37:6803","nonce":1690341662}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.37:6806","nonce":1690341662},{"type":"v1","addr":"172.21.15.37:6807","nonce":1690341662}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.37:6804","nonce":1690341662},{"type":"v1","addr":"172.21.15.37:6805","nonce":1690341662}]},"public_addr":"172.21.15.37:6801/1690341662","cluster_addr":"172.21.15.37:6803/1690341662","heartbeat_back_addr":"172.21.15.37:6807/1690341662","heartbeat_front_addr":"172.21.15.37:6805/1690341662","state":["exists","up"]},{"osd":2,"uuid":"4ae7a0e9-cb3f-484c-a75f-f3766b721c47","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":34,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.37:6808","nonce":2754669649},{"type":"v1","addr":"172.21.15.37:6809","nonce":2754669649}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.37:6810","nonce":2754669649},{"type":"v1","addr":"172.21.15.37:6811","nonce":2754669649}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.37:6814","nonce":2754669649},{"type":"v1","addr":"172.21.15.37:6815","nonce":2754669649}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.37:6812","nonce":2754669649},{"type":"v1","addr":"172.21.15.37:6813","nonce":2754669649}]},"public_addr":"172.21.15.37:6809/2754669649","cluster_addr":"172.21.15.37:6811/2754669649","heartbeat_back_addr":"172.21.15.37:6815/2754669649","heartbeat_front_addr":"172.21.15.37:6813/2754669649","state":["exists","up"]},{"osd":3,"uuid":"1ba4a746-1608-47a7-ace2-960a5cfae934","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":34,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.37:6824","nonce":1815290618},{"type":"v1","addr":"172.21.15.37:6825","nonce":1815290618}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.37:6826","nonce":1815290618},{"type":"v1","addr":"172.21.15.37:6827","nonce":1815290618}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.37:6830","nonce":1815290618},{"type":"v1","addr":"172.21.15.37:6831","nonce":1815290618}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.37:6828","nonce":1815290618},{"type":"v1","addr":"172.21.15.37:6829","nonce":1815290618}]},"public_addr":"172.21.15.37:6825/1815290618","cluster_addr":"172.21.15.37:6827/1815290618","heartbeat_back_addr":"172.21.15.37:6831/1815290618","heartbeat_front_addr":"172.21.15.37:6829/1815290618","state":["exists","up"]},{"osd":4,"uuid":"d4f59f5d-0413-4b79-87ee-bb212c1fcf62","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":34,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6800","nonce":4051532063},{"type":"v1","addr":"172.21.15.203:6801","nonce":4051532063}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6802","nonce":4051532063},{"type":"v1","addr":"172.21.15.203:6803","nonce":4051532063}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6806","nonce":4051532063},{"type":"v1","addr":"172.21.15.203:6807","nonce":4051532063}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6804","nonce":4051532063},{"type":"v1","addr":"172.21.15.203:6805","nonce":4051532063}]},"public_addr":"172.21.15.203:6801/4051532063","cluster_addr":"172.21.15.203:6803/4051532063","heartbeat_back_addr":"172.21.15.203:6807/4051532063","heartbeat_front_addr":"172.21.15.203:6805/4051532063","state":["exists","up"]},{"osd":5,"uuid":"69710a0e-41d8-4e7f-9a10-17ce540b67be","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":34,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6824","nonce":1203822890},{"type":"v1","addr":"172.21.15.203:6825","nonce":1203822890}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6826","nonce":1203822890},{"type":"v1","addr":"172.21.15.203:6827","nonce":1203822890}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6830","nonce":1203822890},{"type":"v1","addr":"172.21.15.203:6831","nonce":1203822890}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6828","nonce":1203822890},{"type":"v1","addr":"172.21.15.203:6829","nonce":1203822890}]},"public_addr":"172.21.15.203:6825/1203822890","cluster_addr":"172.21.15.203:6827/1203822890","heartbeat_back_addr":"172.21.15.203:6831/1203822890","heartbeat_front_addr":"172.21.15.203:6829/1203822890","state":["exists","up"]},{"osd":6,"uuid":"f29dd047-3b48-473d-a235-6642a6293f20","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":34,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6810","nonce":2309615014},{"type":"v1","addr":"172.21.15.203:6812","nonce":2309615014}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6814","nonce":2309615014},{"type":"v1","addr":"172.21.15.203:6816","nonce":2309615014}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6822","nonce":2309615014},{"type":"v1","addr":"172.21.15.203:6823","nonce":2309615014}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6818","nonce":2309615014},{"type":"v1","addr":"172.21.15.203:6820","nonce":2309615014}]},"public_addr":"172.21.15.203:6812/2309615014","cluster_addr":"172.21.15.203:6816/2309615014","heartbeat_back_addr":"172.21.15.203:6823/2309615014","heartbeat_front_addr":"172.21.15.203:6820/2309615014","state":["exists","up"]},{"osd":7,"uuid":"6e691644-378b-4fdc-8e86-fb5f56e31f29","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":34,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6808","nonce":3688099014},{"type":"v1","addr":"172.21.15.203:6809","nonce":3688099014}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6811","nonce":3688099014},{"type":"v1","addr":"172.21.15.203:6813","nonce":3688099014}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6819","nonce":3688099014},{"type":"v1","addr":"172.21.15.203:6821","nonce":3688099014}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6815","nonce":3688099014},{"type":"v1","addr":"172.21.15.203:6817","nonce":3688099014}]},"public_addr":"172.21.15.203:6809/3688099014","cluster_addr":"172.21.15.203:6813/3688099014","heartbeat_back_addr":"172.21.15.203:6821/3688099014","heartbeat_front_addr":"172.21.15.203:6817/3688099014","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T10:28:51.419141+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T10:28:51.366448+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T10:28:51.348276+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T10:28:51.380519+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T10:28:51.105812+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T10:28:51.384496+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T10:28:51.388755+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T10:28:51.366958+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-04-25T10:30:13.439 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:13.437+0000 7f893196c640 1 -- 172.21.15.37:0/582601766 >> [v2:172.21.15.203:6832/2418205237,v1:172.21.15.203:6833/2418205237] conn(0x7f890c0744b0 msgr2=0x7f890c0769c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:30:13.439 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:13.437+0000 7f893196c640 1 --2- 172.21.15.37:0/582601766 >> [v2:172.21.15.203:6832/2418205237,v1:172.21.15.203:6833/2418205237] conn(0x7f890c0744b0 0x7f890c0769c0 secure :-1 s=READY pgs=213 cs=0 l=1 rev1=1 crypto rx=0x7f8924009a20 tx=0x7f8924008040 comp rx=0 tx=0).stop 2024-04-25T10:30:13.440 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:13.437+0000 7f893196c640 1 -- 172.21.15.37:0/582601766 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f892c0d0da0 msgr2=0x7f892c15bd80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:30:13.440 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:13.437+0000 7f893196c640 1 --2- 172.21.15.37:0/582601766 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f892c0d0da0 0x7f892c15bd80 secure :-1 s=READY pgs=424 cs=0 l=1 rev1=1 crypto rx=0x7f8914000c00 tx=0x7f89140046c0 comp rx=0 tx=0).stop 2024-04-25T10:30:13.440 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:13.437+0000 7f893196c640 1 -- 172.21.15.37:0/582601766 shutdown_connections 2024-04-25T10:30:13.440 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:13.437+0000 7f893196c640 1 --2- 172.21.15.37:0/582601766 >> [v2:172.21.15.203:6832/2418205237,v1:172.21.15.203:6833/2418205237] conn(0x7f890c0744b0 0x7f890c0769c0 unknown :-1 s=CLOSED pgs=213 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:30:13.440 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:13.437+0000 7f893196c640 1 --2- 172.21.15.37:0/582601766 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f892c0d5160 0x7f892c157430 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:30:13.440 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:13.437+0000 7f893196c640 1 --2- 172.21.15.37:0/582601766 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f892c0d0da0 0x7f892c15bd80 unknown :-1 s=CLOSED pgs=424 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:30:13.440 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:13.437+0000 7f893196c640 1 --2- 172.21.15.37:0/582601766 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f892c0d0990 0x7f892c1565a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:30:13.440 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:13.437+0000 7f893196c640 1 -- 172.21.15.37:0/582601766 >> 172.21.15.37:0/582601766 conn(0x7f892c0b7a80 msgr2=0x7f892c162d80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:30:13.440 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:13.437+0000 7f893196c640 1 -- 172.21.15.37:0/582601766 shutdown_connections 2024-04-25T10:30:13.440 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:13.437+0000 7f893196c640 1 -- 172.21.15.37:0/582601766 wait complete. 2024-04-25T10:30:13.451 DEBUG:teuthology.orchestra.run.smithi037:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs set cephfs standby_count_wanted 0 2024-04-25T10:30:13.566 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:13.563+0000 7fbb9f6e7640 1 Processor -- start 2024-04-25T10:30:13.566 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:13.564+0000 7fbb9f6e7640 1 -- start start 2024-04-25T10:30:13.568 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:13.565+0000 7fbb9f6e7640 1 --2- >> v2:172.21.15.203:3300/0 conn(0x7fbb98143500 0x7fbb9806f100 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:30:13.568 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:13.565+0000 7fbb9f6e7640 1 -- --> v1:172.21.15.37:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fbb981616a0 con 0x7fbb981460e0 2024-04-25T10:30:13.568 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:13.565+0000 7fbb9f6e7640 1 -- --> v1:172.21.15.203:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fbb980b5b60 con 0x7fbb981610b0 2024-04-25T10:30:13.568 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:13.565+0000 7fbb9f6e7640 1 -- --> v2:172.21.15.203:3300/0 -- mon_getmap magic: 0 -- 0x7fbb980b5e40 con 0x7fbb98143500 2024-04-25T10:30:13.568 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:13.565+0000 7fbb9cc82640 1 --1- >> v1:172.21.15.37:6789/0 conn(0x7fbb981460e0 0x7fbb981609a0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.37:6789/0 says I am v1:172.21.15.37:41068/0 (socket says 172.21.15.37:41068) 2024-04-25T10:30:13.568 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:13.565+0000 7fbb9cc82640 1 -- 172.21.15.37:0/1808642341 learned_addr learned my addr 172.21.15.37:0/1808642341 (peer_addr_for_me v1:172.21.15.37:0/0) 2024-04-25T10:30:13.568 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:13.565+0000 7fbb9d483640 1 --2- 172.21.15.37:0/1808642341 >> v2:172.21.15.203:3300/0 conn(0x7fbb98143500 0x7fbb9806f100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:30:13.568 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:13.565+0000 7fbb97fff640 1 -- 172.21.15.37:0/1808642341 <== mon.0 v1:172.21.15.37:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 1661108206 0 0) 0x7fbb981616a0 con 0x7fbb981460e0 2024-04-25T10:30:13.568 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:13.565+0000 7fbb97fff640 1 -- 172.21.15.37:0/1808642341 --> v1:172.21.15.37:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7fbb80003520 con 0x7fbb981460e0 2024-04-25T10:30:13.569 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:13.566+0000 7fbb97fff640 1 -- 172.21.15.37:0/1808642341 <== mon.1 v1:172.21.15.203:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 2914002631 0 0) 0x7fbb980b5b60 con 0x7fbb981610b0 2024-04-25T10:30:13.569 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:13.566+0000 7fbb97fff640 1 -- 172.21.15.37:0/1808642341 --> v1:172.21.15.203:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7fbb981616a0 con 0x7fbb981610b0 2024-04-25T10:30:13.569 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:13.566+0000 7fbb9d483640 1 -- 172.21.15.37:0/1808642341 >> v1:172.21.15.203:6789/0 conn(0x7fbb981610b0 legacy=0x7fbb98143b30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:30:13.569 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:13.566+0000 7fbb9d483640 1 -- 172.21.15.37:0/1808642341 >> v1:172.21.15.37:6789/0 conn(0x7fbb981460e0 legacy=0x7fbb981609a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:30:13.569 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:13.566+0000 7fbb97fff640 1 -- 172.21.15.37:0/1808642341 <== mon.0 v1:172.21.15.37:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 3742875752 0 0) 0x7fbb80003520 con 0x7fbb981460e0 2024-04-25T10:30:13.569 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:13.566+0000 7fbb9d483640 1 -- 172.21.15.37:0/1808642341 --> v2:172.21.15.203:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fbb980b6120 con 0x7fbb98143500 2024-04-25T10:30:13.570 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:13.567+0000 7fbb9d483640 1 --2- 172.21.15.37:0/1808642341 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fbb98143500 0x7fbb9806f100 secure :-1 s=READY pgs=178 cs=0 l=1 rev1=1 crypto rx=0x7fbb900097a0 tx=0x7fbb9002a130 comp rx=0 tx=0).ready entity=mon.1 client_cookie=9a9030f8f6bffd10 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:30:13.570 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:13.567+0000 7fbb97fff640 1 -- 172.21.15.37:0/1808642341 <== mon.1 v2:172.21.15.203:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbb90037070 con 0x7fbb98143500 2024-04-25T10:30:13.570 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:13.567+0000 7fbb97fff640 1 -- 172.21.15.37:0/1808642341 <== mon.1 v2:172.21.15.203:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fbb9002abc0 con 0x7fbb98143500 2024-04-25T10:30:13.570 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:13.567+0000 7fbb97fff640 1 -- 172.21.15.37:0/1808642341 <== mon.1 v2:172.21.15.203:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbb8c001670 con 0x7fbb98143500 2024-04-25T10:30:13.570 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:13.567+0000 7fbb9f6e7640 1 -- 172.21.15.37:0/1808642341 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fbb98143500 msgr2=0x7fbb9806f100 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:30:13.571 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:13.567+0000 7fbb9f6e7640 1 --2- 172.21.15.37:0/1808642341 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fbb98143500 0x7fbb9806f100 secure :-1 s=READY pgs=178 cs=0 l=1 rev1=1 crypto rx=0x7fbb900097a0 tx=0x7fbb9002a130 comp rx=0 tx=0).stop 2024-04-25T10:30:13.571 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:13.567+0000 7fbb9f6e7640 1 -- 172.21.15.37:0/1808642341 shutdown_connections 2024-04-25T10:30:13.571 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:13.567+0000 7fbb9f6e7640 1 --2- 172.21.15.37:0/1808642341 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fbb98143500 0x7fbb9806f100 unknown :-1 s=CLOSED pgs=178 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:30:13.571 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:13.567+0000 7fbb9f6e7640 1 -- 172.21.15.37:0/1808642341 >> 172.21.15.37:0/1808642341 conn(0x7fbb980b36b0 msgr2=0x7fbb980b3ac0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:30:13.571 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:13.567+0000 7fbb9f6e7640 1 -- 172.21.15.37:0/1808642341 shutdown_connections 2024-04-25T10:30:13.571 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:13.567+0000 7fbb9f6e7640 1 -- 172.21.15.37:0/1808642341 wait complete. 2024-04-25T10:30:13.571 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:13.569+0000 7fbb9f6e7640 1 Processor -- start 2024-04-25T10:30:13.572 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:13.569+0000 7fbb9f6e7640 1 -- start start 2024-04-25T10:30:13.574 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:13.570+0000 7fbb9f6e7640 1 --2- >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7fbb981460e0 0x7fbb9815a6e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:30:13.574 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:13.570+0000 7fbb9f6e7640 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fbb981610b0 0x7fbb98136f80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:30:13.574 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:13.571+0000 7fbb9f6e7640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fbb9815f450 0x7fbb9815f860 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:30:13.574 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:13.571+0000 7fbb9f6e7640 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 -- 0x7fbb980b6a90 con 0x7fbb9815f450 2024-04-25T10:30:13.574 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:13.571+0000 7fbb9f6e7640 1 -- --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_getmap magic: 0 -- 0x7fbb980b6910 con 0x7fbb981610b0 2024-04-25T10:30:13.574 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:13.571+0000 7fbb9f6e7640 1 -- --> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] -- mon_getmap magic: 0 -- 0x7fbb98144240 con 0x7fbb981460e0 2024-04-25T10:30:13.574 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:13.571+0000 7fbb9dc84640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fbb9815f450 0x7fbb9815f860 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:30:13.575 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:13.571+0000 7fbb9dc84640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fbb9815f450 0x7fbb9815f860 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.37:3300/0 says I am v2:172.21.15.37:52628/0 (socket says 172.21.15.37:52628) 2024-04-25T10:30:13.575 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:13.571+0000 7fbb9dc84640 1 -- 172.21.15.37:0/490505984 learned_addr learned my addr 172.21.15.37:0/490505984 (peer_addr_for_me v2:172.21.15.37:0/0) 2024-04-25T10:30:13.575 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:13.571+0000 7fbb9d483640 1 --2- 172.21.15.37:0/490505984 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7fbb981460e0 0x7fbb9815a6e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:30:13.575 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:13.571+0000 7fbb9cc82640 1 --2- 172.21.15.37:0/490505984 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fbb981610b0 0x7fbb98136f80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:30:13.575 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:13.571+0000 7fbb9dc84640 1 -- 172.21.15.37:0/490505984 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7fbb981460e0 msgr2=0x7fbb9815a6e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:30:13.575 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:13.571+0000 7fbb9dc84640 1 --2- 172.21.15.37:0/490505984 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7fbb981460e0 0x7fbb9815a6e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:30:13.575 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:13.571+0000 7fbb9dc84640 1 -- 172.21.15.37:0/490505984 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fbb981610b0 msgr2=0x7fbb98136f80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:30:13.575 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:13.571+0000 7fbb9dc84640 1 --2- 172.21.15.37:0/490505984 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fbb981610b0 0x7fbb98136f80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:30:13.575 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:13.571+0000 7fbb9dc84640 1 -- 172.21.15.37:0/490505984 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fbb9815fda0 con 0x7fbb9815f450 2024-04-25T10:30:13.575 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:13.571+0000 7fbb9cc82640 1 --2- 172.21.15.37:0/490505984 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fbb981610b0 0x7fbb98136f80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T10:30:13.575 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:13.571+0000 7fbb9dc84640 1 --2- 172.21.15.37:0/490505984 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fbb9815f450 0x7fbb9815f860 secure :-1 s=READY pgs=426 cs=0 l=1 rev1=1 crypto rx=0x7fbb8800c6e0 tx=0x7fbb8800cba0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:30:13.575 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:13.571+0000 7fbb95ffb640 1 -- 172.21.15.37:0/490505984 <== mon.0 v2:172.21.15.37:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbb88003b70 con 0x7fbb9815f450 2024-04-25T10:30:13.575 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:13.571+0000 7fbb95ffb640 1 -- 172.21.15.37:0/490505984 <== mon.0 v2:172.21.15.37:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fbb88003d10 con 0x7fbb9815f450 2024-04-25T10:30:13.575 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:13.571+0000 7fbb9f6e7640 1 -- 172.21.15.37:0/490505984 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fbb98158270 con 0x7fbb9815f450 2024-04-25T10:30:13.575 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:13.571+0000 7fbb95ffb640 1 -- 172.21.15.37:0/490505984 <== mon.0 v2:172.21.15.37:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbb880052d0 con 0x7fbb9815f450 2024-04-25T10:30:13.576 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:13.572+0000 7fbb9f6e7640 1 -- 172.21.15.37:0/490505984 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fbb98158830 con 0x7fbb9815f450 2024-04-25T10:30:13.576 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:13.573+0000 7fbb9f6e7640 1 -- 172.21.15.37:0/490505984 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fbb9815ea60 con 0x7fbb9815f450 2024-04-25T10:30:13.577 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:13.573+0000 7fbb95ffb640 1 -- 172.21.15.37:0/490505984 <== mon.0 v2:172.21.15.37:3300/0 4 ==== mgrmap(e 5) ==== 96782+0+0 (secure 0 0 0) 0x7fbb88005470 con 0x7fbb9815f450 2024-04-25T10:30:13.577 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:13.574+0000 7fbb95ffb640 1 --2- 172.21.15.37:0/490505984 >> [v2:172.21.15.203:6832/2418205237,v1:172.21.15.203:6833/2418205237] conn(0x7fbb840744e0 0x7fbb840769f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:30:13.577 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:13.574+0000 7fbb95ffb640 1 -- 172.21.15.37:0/490505984 <== mon.0 v2:172.21.15.37:3300/0 5 ==== osd_map(35..35 src has 1..35) ==== 5709+0+0 (secure 0 0 0) 0x7fbb88095100 con 0x7fbb9815f450 2024-04-25T10:30:13.577 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:13.574+0000 7fbb9d483640 1 --2- 172.21.15.37:0/490505984 >> [v2:172.21.15.203:6832/2418205237,v1:172.21.15.203:6833/2418205237] conn(0x7fbb840744e0 0x7fbb840769f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:30:13.577 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:13.575+0000 7fbb9d483640 1 --2- 172.21.15.37:0/490505984 >> [v2:172.21.15.203:6832/2418205237,v1:172.21.15.203:6833/2418205237] conn(0x7fbb840744e0 0x7fbb840769f0 secure :-1 s=READY pgs=214 cs=0 l=1 rev1=1 crypto rx=0x7fbb90034000 tx=0x7fbb9002e000 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:30:13.580 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:13.578+0000 7fbb95ffb640 1 -- 172.21.15.37:0/490505984 <== mon.0 v2:172.21.15.37:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187246 (secure 0 0 0) 0x7fbb9815fda0 con 0x7fbb9815f450 2024-04-25T10:30:13.805 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:13.803+0000 7fbb9f6e7640 1 -- 172.21.15.37:0/490505984 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_command({"prefix": "fs set", "fs_name": "cephfs", "var": "standby_count_wanted", "val": "0"} v 0) -- 0x7fbb9806a140 con 0x7fbb9815f450 2024-04-25T10:30:14.779 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:14.777+0000 7fbb95ffb640 1 -- 172.21.15.37:0/490505984 <== mon.0 v2:172.21.15.37:3300/0 7 ==== mon_command_ack([{"prefix": "fs set", "fs_name": "cephfs", "var": "standby_count_wanted", "val": "0"}]=0 v16) ==== 118+0+0 (secure 0 0 0) 0x7fbb8805f7f0 con 0x7fbb9815f450 2024-04-25T10:30:14.782 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:14.780+0000 7fbb9f6e7640 1 -- 172.21.15.37:0/490505984 >> [v2:172.21.15.203:6832/2418205237,v1:172.21.15.203:6833/2418205237] conn(0x7fbb840744e0 msgr2=0x7fbb840769f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:30:14.782 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:14.780+0000 7fbb9f6e7640 1 --2- 172.21.15.37:0/490505984 >> [v2:172.21.15.203:6832/2418205237,v1:172.21.15.203:6833/2418205237] conn(0x7fbb840744e0 0x7fbb840769f0 secure :-1 s=READY pgs=214 cs=0 l=1 rev1=1 crypto rx=0x7fbb90034000 tx=0x7fbb9002e000 comp rx=0 tx=0).stop 2024-04-25T10:30:14.782 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:14.780+0000 7fbb9f6e7640 1 -- 172.21.15.37:0/490505984 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fbb9815f450 msgr2=0x7fbb9815f860 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:30:14.782 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:14.780+0000 7fbb9f6e7640 1 --2- 172.21.15.37:0/490505984 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fbb9815f450 0x7fbb9815f860 secure :-1 s=READY pgs=426 cs=0 l=1 rev1=1 crypto rx=0x7fbb8800c6e0 tx=0x7fbb8800cba0 comp rx=0 tx=0).stop 2024-04-25T10:30:14.783 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:14.780+0000 7fbb9f6e7640 1 -- 172.21.15.37:0/490505984 shutdown_connections 2024-04-25T10:30:14.783 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:14.780+0000 7fbb9f6e7640 1 --2- 172.21.15.37:0/490505984 >> [v2:172.21.15.203:6832/2418205237,v1:172.21.15.203:6833/2418205237] conn(0x7fbb840744e0 0x7fbb840769f0 unknown :-1 s=CLOSED pgs=214 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:30:14.783 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:14.780+0000 7fbb9f6e7640 1 --2- 172.21.15.37:0/490505984 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fbb9815f450 0x7fbb9815f860 unknown :-1 s=CLOSED pgs=426 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:30:14.783 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:14.780+0000 7fbb9f6e7640 1 --2- 172.21.15.37:0/490505984 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fbb981610b0 0x7fbb98136f80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:30:14.783 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:14.780+0000 7fbb9f6e7640 1 --2- 172.21.15.37:0/490505984 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7fbb981460e0 0x7fbb9815a6e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:30:14.783 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:14.780+0000 7fbb9f6e7640 1 -- 172.21.15.37:0/490505984 >> 172.21.15.37:0/490505984 conn(0x7fbb980b36b0 msgr2=0x7fbb98144e90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:30:14.783 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:14.780+0000 7fbb9f6e7640 1 -- 172.21.15.37:0/490505984 shutdown_connections 2024-04-25T10:30:14.783 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:14.780+0000 7fbb9f6e7640 1 -- 172.21.15.37:0/490505984 wait complete. 2024-04-25T10:30:14.795 DEBUG:tasks.cephfs.filesystem:fs_config: {} 2024-04-25T10:30:14.795 DEBUG:teuthology.orchestra.run.smithi037:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs set cephfs allow_standby_replay false 2024-04-25T10:30:14.917 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:14.914+0000 7fb42b484640 1 Processor -- start 2024-04-25T10:30:14.917 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:14.915+0000 7fb42b484640 1 -- start start 2024-04-25T10:30:14.919 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:14.916+0000 7fb42b484640 1 --2- >> v2:172.21.15.203:3300/0 conn(0x7fb4240b2dc0 0x7fb4240d2860 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:30:14.919 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:14.916+0000 7fb42b484640 1 --2- >> v2:172.21.15.37:3300/0 conn(0x7fb42406daf0 0x7fb42406df00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:30:14.919 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:14.916+0000 7fb42b484640 1 -- --> v1:172.21.15.203:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fb4240d2e40 con 0x7fb42406e440 2024-04-25T10:30:14.919 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:14.916+0000 7fb42b484640 1 -- --> v2:172.21.15.37:3300/0 -- mon_getmap magic: 0 -- 0x7fb4240b92c0 con 0x7fb42406daf0 2024-04-25T10:30:14.919 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:14.916+0000 7fb42b484640 1 -- --> v2:172.21.15.203:3300/0 -- mon_getmap magic: 0 -- 0x7fb4240ce0d0 con 0x7fb4240b2dc0 2024-04-25T10:30:14.919 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:14.916+0000 7fb428a1f640 1 --2- >> v2:172.21.15.37:3300/0 conn(0x7fb42406daf0 0x7fb42406df00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:30:14.919 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:14.916+0000 7fb428a1f640 1 --2- >> v2:172.21.15.37:3300/0 conn(0x7fb42406daf0 0x7fb42406df00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.37:3300/0 says I am v2:172.21.15.37:52636/0 (socket says 172.21.15.37:52636) 2024-04-25T10:30:14.919 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:14.916+0000 7fb428a1f640 1 -- 172.21.15.37:0/357542236 learned_addr learned my addr 172.21.15.37:0/357542236 (peer_addr_for_me v2:172.21.15.37:0/0) 2024-04-25T10:30:14.919 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:14.916+0000 7fb428a1f640 1 -- 172.21.15.37:0/357542236 >> v2:172.21.15.203:3300/0 conn(0x7fb4240b2dc0 msgr2=0x7fb4240d2860 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:30:14.919 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:14.916+0000 7fb428a1f640 1 --2- 172.21.15.37:0/357542236 >> v2:172.21.15.203:3300/0 conn(0x7fb4240b2dc0 0x7fb4240d2860 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:30:14.919 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:14.916+0000 7fb428a1f640 1 -- 172.21.15.37:0/357542236 >> v1:172.21.15.203:6789/0 conn(0x7fb42406e440 legacy=0x7fb42408cc90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:30:14.919 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:14.917+0000 7fb428a1f640 1 -- 172.21.15.37:0/357542236 --> v2:172.21.15.37:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fb4240d3120 con 0x7fb42406daf0 2024-04-25T10:30:14.921 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:14.917+0000 7fb428a1f640 1 --2- 172.21.15.37:0/357542236 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fb42406daf0 0x7fb42406df00 secure :-1 s=READY pgs=427 cs=0 l=1 rev1=1 crypto rx=0x7fb424070b40 tx=0x7fb41802a230 comp rx=0 tx=0).ready entity=mon.0 client_cookie=6f35299c59aad70e server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:30:14.921 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:14.917+0000 7fb423fff640 1 -- 172.21.15.37:0/357542236 <== mon.0 v2:172.21.15.37:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb418037070 con 0x7fb42406daf0 2024-04-25T10:30:14.921 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:14.917+0000 7fb423fff640 1 -- 172.21.15.37:0/357542236 <== mon.0 v2:172.21.15.37:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fb418032440 con 0x7fb42406daf0 2024-04-25T10:30:14.921 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:14.917+0000 7fb423fff640 1 -- 172.21.15.37:0/357542236 <== mon.0 v2:172.21.15.37:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb418032810 con 0x7fb42406daf0 2024-04-25T10:30:14.921 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:14.918+0000 7fb42b484640 1 -- 172.21.15.37:0/357542236 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fb42406daf0 msgr2=0x7fb42406df00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:30:14.921 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:14.918+0000 7fb42b484640 1 --2- 172.21.15.37:0/357542236 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fb42406daf0 0x7fb42406df00 secure :-1 s=READY pgs=427 cs=0 l=1 rev1=1 crypto rx=0x7fb424070b40 tx=0x7fb41802a230 comp rx=0 tx=0).stop 2024-04-25T10:30:14.921 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:14.918+0000 7fb42b484640 1 -- 172.21.15.37:0/357542236 shutdown_connections 2024-04-25T10:30:14.921 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:14.918+0000 7fb42b484640 1 --2- 172.21.15.37:0/357542236 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fb42406daf0 0x7fb42406df00 unknown :-1 s=CLOSED pgs=427 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:30:14.921 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:14.918+0000 7fb42b484640 1 --2- 172.21.15.37:0/357542236 >> v2:172.21.15.203:3300/0 conn(0x7fb4240b2dc0 0x7fb4240d2860 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:30:14.921 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:14.918+0000 7fb42b484640 1 -- 172.21.15.37:0/357542236 >> 172.21.15.37:0/357542236 conn(0x7fb4240b5440 msgr2=0x7fb4240b5850 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:30:14.921 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:14.918+0000 7fb42b484640 1 -- 172.21.15.37:0/357542236 shutdown_connections 2024-04-25T10:30:14.921 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:14.918+0000 7fb42b484640 1 -- 172.21.15.37:0/357542236 wait complete. 2024-04-25T10:30:14.922 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:14.920+0000 7fb42b484640 1 Processor -- start 2024-04-25T10:30:14.922 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:14.920+0000 7fb42b484640 1 -- start start 2024-04-25T10:30:14.925 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:14.921+0000 7fb42b484640 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fb42406daf0 0x7fb4240c8550 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:30:14.925 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:14.921+0000 7fb42b484640 1 --2- >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7fb42406e440 0x7fb4240c9d90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:30:14.925 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:14.921+0000 7fb42b484640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fb4240b2dc0 0x7fb4240b8120 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:30:14.925 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:14.922+0000 7fb42b484640 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 -- 0x7fb42406e850 con 0x7fb4240b2dc0 2024-04-25T10:30:14.925 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:14.922+0000 7fb42b484640 1 -- --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_getmap magic: 0 -- 0x7fb4240d36e0 con 0x7fb42406daf0 2024-04-25T10:30:14.926 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:14.922+0000 7fb42b484640 1 -- --> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] -- mon_getmap magic: 0 -- 0x7fb4240cf570 con 0x7fb42406e440 2024-04-25T10:30:14.926 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:14.922+0000 7fb429a21640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fb4240b2dc0 0x7fb4240b8120 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:30:14.926 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:14.922+0000 7fb429a21640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fb4240b2dc0 0x7fb4240b8120 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.37:3300/0 says I am v2:172.21.15.37:52648/0 (socket says 172.21.15.37:52648) 2024-04-25T10:30:14.926 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:14.922+0000 7fb429a21640 1 -- 172.21.15.37:0/3871146509 learned_addr learned my addr 172.21.15.37:0/3871146509 (peer_addr_for_me v2:172.21.15.37:0/0) 2024-04-25T10:30:14.926 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:14.922+0000 7fb429220640 1 --2- 172.21.15.37:0/3871146509 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fb42406daf0 0x7fb4240c8550 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:30:14.926 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:14.922+0000 7fb428a1f640 1 --2- 172.21.15.37:0/3871146509 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7fb42406e440 0x7fb4240c9d90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:30:14.926 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:14.922+0000 7fb429a21640 1 -- 172.21.15.37:0/3871146509 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7fb42406e440 msgr2=0x7fb4240c9d90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:30:14.926 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:14.922+0000 7fb429a21640 1 --2- 172.21.15.37:0/3871146509 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7fb42406e440 0x7fb4240c9d90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:30:14.926 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:14.922+0000 7fb429a21640 1 -- 172.21.15.37:0/3871146509 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fb42406daf0 msgr2=0x7fb4240c8550 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:30:14.926 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:14.922+0000 7fb429a21640 1 --2- 172.21.15.37:0/3871146509 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fb42406daf0 0x7fb4240c8550 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:30:14.926 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:14.922+0000 7fb429a21640 1 -- 172.21.15.37:0/3871146509 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fb4240c7520 con 0x7fb4240b2dc0 2024-04-25T10:30:14.926 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:14.922+0000 7fb429220640 1 --2- 172.21.15.37:0/3871146509 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fb42406daf0 0x7fb4240c8550 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T10:30:14.926 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:14.922+0000 7fb429a21640 1 --2- 172.21.15.37:0/3871146509 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fb4240b2dc0 0x7fb4240b8120 secure :-1 s=READY pgs=428 cs=0 l=1 rev1=1 crypto rx=0x7fb41400b990 tx=0x7fb41400be50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:30:14.927 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:14.922+0000 7fb421ffb640 1 -- 172.21.15.37:0/3871146509 <== mon.0 v2:172.21.15.37:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb414019070 con 0x7fb4240b2dc0 2024-04-25T10:30:14.927 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:14.922+0000 7fb421ffb640 1 -- 172.21.15.37:0/3871146509 <== mon.0 v2:172.21.15.37:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fb41400cba0 con 0x7fb4240b2dc0 2024-04-25T10:30:14.927 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:14.922+0000 7fb421ffb640 1 -- 172.21.15.37:0/3871146509 <== mon.0 v2:172.21.15.37:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb414014440 con 0x7fb4240b2dc0 2024-04-25T10:30:14.927 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:14.922+0000 7fb42b484640 1 -- 172.21.15.37:0/3871146509 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fb4240b8660 con 0x7fb4240b2dc0 2024-04-25T10:30:14.927 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:14.923+0000 7fb42b484640 1 -- 172.21.15.37:0/3871146509 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fb4240c7a70 con 0x7fb4240b2dc0 2024-04-25T10:30:14.928 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:14.924+0000 7fb42b484640 1 -- 172.21.15.37:0/3871146509 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fb42406a140 con 0x7fb4240b2dc0 2024-04-25T10:30:14.928 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:14.924+0000 7fb421ffb640 1 -- 172.21.15.37:0/3871146509 <== mon.0 v2:172.21.15.37:3300/0 4 ==== mgrmap(e 5) ==== 96782+0+0 (secure 0 0 0) 0x7fb41401c070 con 0x7fb4240b2dc0 2024-04-25T10:30:14.928 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:14.925+0000 7fb421ffb640 1 --2- 172.21.15.37:0/3871146509 >> [v2:172.21.15.203:6832/2418205237,v1:172.21.15.203:6833/2418205237] conn(0x7fb4100744e0 0x7fb4100769f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:30:14.928 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:14.925+0000 7fb421ffb640 1 -- 172.21.15.37:0/3871146509 <== mon.0 v2:172.21.15.37:3300/0 5 ==== osd_map(36..36 src has 1..36) ==== 5709+0+0 (secure 0 0 0) 0x7fb414094f30 con 0x7fb4240b2dc0 2024-04-25T10:30:14.928 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:14.925+0000 7fb429220640 1 --2- 172.21.15.37:0/3871146509 >> [v2:172.21.15.203:6832/2418205237,v1:172.21.15.203:6833/2418205237] conn(0x7fb4100744e0 0x7fb4100769f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:30:14.928 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:14.925+0000 7fb429220640 1 --2- 172.21.15.37:0/3871146509 >> [v2:172.21.15.203:6832/2418205237,v1:172.21.15.203:6833/2418205237] conn(0x7fb4100744e0 0x7fb4100769f0 secure :-1 s=READY pgs=215 cs=0 l=1 rev1=1 crypto rx=0x7fb424074cc0 tx=0x7fb41c008040 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:30:14.931 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:14.930+0000 7fb421ffb640 1 -- 172.21.15.37:0/3871146509 <== mon.0 v2:172.21.15.37:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187246 (secure 0 0 0) 0x7fb41405f750 con 0x7fb4240b2dc0 2024-04-25T10:30:15.158 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:15.155+0000 7fb42b484640 1 -- 172.21.15.37:0/3871146509 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_command({"prefix": "fs set", "fs_name": "cephfs", "var": "allow_standby_replay", "val": "false"} v 0) -- 0x7fb4240ca2e0 con 0x7fb4240b2dc0 2024-04-25T10:30:15.780 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:15.778+0000 7fb421ffb640 1 -- 172.21.15.37:0/3871146509 <== mon.0 v2:172.21.15.37:3300/0 7 ==== mon_command_ack([{"prefix": "fs set", "fs_name": "cephfs", "var": "allow_standby_replay", "val": "false"}]=0 v17) ==== 122+0+0 (secure 0 0 0) 0x7fb41405f050 con 0x7fb4240b2dc0 2024-04-25T10:30:15.783 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:15.781+0000 7fb42b484640 1 -- 172.21.15.37:0/3871146509 >> [v2:172.21.15.203:6832/2418205237,v1:172.21.15.203:6833/2418205237] conn(0x7fb4100744e0 msgr2=0x7fb4100769f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:30:15.783 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:15.781+0000 7fb42b484640 1 --2- 172.21.15.37:0/3871146509 >> [v2:172.21.15.203:6832/2418205237,v1:172.21.15.203:6833/2418205237] conn(0x7fb4100744e0 0x7fb4100769f0 secure :-1 s=READY pgs=215 cs=0 l=1 rev1=1 crypto rx=0x7fb424074cc0 tx=0x7fb41c008040 comp rx=0 tx=0).stop 2024-04-25T10:30:15.783 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:15.781+0000 7fb42b484640 1 -- 172.21.15.37:0/3871146509 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fb4240b2dc0 msgr2=0x7fb4240b8120 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:30:15.783 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:15.781+0000 7fb42b484640 1 --2- 172.21.15.37:0/3871146509 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fb4240b2dc0 0x7fb4240b8120 secure :-1 s=READY pgs=428 cs=0 l=1 rev1=1 crypto rx=0x7fb41400b990 tx=0x7fb41400be50 comp rx=0 tx=0).stop 2024-04-25T10:30:15.784 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:15.781+0000 7fb42b484640 1 -- 172.21.15.37:0/3871146509 shutdown_connections 2024-04-25T10:30:15.784 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:15.781+0000 7fb42b484640 1 --2- 172.21.15.37:0/3871146509 >> [v2:172.21.15.203:6832/2418205237,v1:172.21.15.203:6833/2418205237] conn(0x7fb4100744e0 0x7fb4100769f0 unknown :-1 s=CLOSED pgs=215 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:30:15.784 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:15.781+0000 7fb42b484640 1 --2- 172.21.15.37:0/3871146509 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fb4240b2dc0 0x7fb4240b8120 unknown :-1 s=CLOSED pgs=428 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:30:15.784 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:15.781+0000 7fb42b484640 1 --2- 172.21.15.37:0/3871146509 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7fb42406e440 0x7fb4240c9d90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:30:15.784 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:15.781+0000 7fb42b484640 1 --2- 172.21.15.37:0/3871146509 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fb42406daf0 0x7fb4240c8550 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:30:15.784 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:15.781+0000 7fb42b484640 1 -- 172.21.15.37:0/3871146509 >> 172.21.15.37:0/3871146509 conn(0x7fb4240b5440 msgr2=0x7fb42408cc90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:30:15.784 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:15.781+0000 7fb42b484640 1 -- 172.21.15.37:0/3871146509 shutdown_connections 2024-04-25T10:30:15.784 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:15.781+0000 7fb42b484640 1 -- 172.21.15.37:0/3871146509 wait complete. 2024-04-25T10:30:15.796 DEBUG:tasks.cephfs.filesystem:Not Creating any subvols for filesystem 'cephfs' 2024-04-25T10:30:15.796 DEBUG:teuthology.orchestra.run.smithi037:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-04-25T10:30:15.912 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:15.910+0000 7f23ffc67640 1 Processor -- start 2024-04-25T10:30:15.913 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:15.910+0000 7f23ffc67640 1 -- start start 2024-04-25T10:30:15.914 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:15.911+0000 7f23ffc67640 1 --2- >> v2:172.21.15.37:3300/0 conn(0x7f23f815a9f0 0x7f23f806c540 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:30:15.914 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:15.911+0000 7f23ffc67640 1 --2- >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f23f8161020 0x7f23f80b5860 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:30:15.914 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:15.911+0000 7f23ffc67640 1 -- --> v1:172.21.15.203:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f23f8161430 con 0x7f23f806ca80 2024-04-25T10:30:15.914 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:15.911+0000 7f23ffc67640 1 -- --> v2:172.21.15.37:3300/0 -- mon_getmap magic: 0 -- 0x7f23f806ce90 con 0x7f23f815a9f0 2024-04-25T10:30:15.914 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:15.911+0000 7f23ffc67640 1 -- --> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] -- mon_getmap magic: 0 -- 0x7f23f81616d0 con 0x7f23f8161020 2024-04-25T10:30:15.915 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:15.912+0000 7f23fda03640 1 --2- >> v2:172.21.15.37:3300/0 conn(0x7f23f815a9f0 0x7f23f806c540 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:30:15.915 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:15.912+0000 7f23fda03640 1 --2- >> v2:172.21.15.37:3300/0 conn(0x7f23f815a9f0 0x7f23f806c540 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.37:3300/0 says I am v2:172.21.15.37:52662/0 (socket says 172.21.15.37:52662) 2024-04-25T10:30:15.915 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:15.912+0000 7f23fda03640 1 -- 172.21.15.37:0/3600226881 learned_addr learned my addr 172.21.15.37:0/3600226881 (peer_addr_for_me v2:172.21.15.37:0/0) 2024-04-25T10:30:15.915 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:15.912+0000 7f23fe204640 1 --2- 172.21.15.37:0/3600226881 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f23f8161020 0x7f23f80b5860 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:30:15.915 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:15.912+0000 7f23fda03640 1 -- 172.21.15.37:0/3600226881 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f23f8161020 msgr2=0x7f23f80b5860 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:30:15.915 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:15.912+0000 7f23fda03640 1 --2- 172.21.15.37:0/3600226881 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f23f8161020 0x7f23f80b5860 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:30:15.915 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:15.912+0000 7f23fda03640 1 -- 172.21.15.37:0/3600226881 >> v1:172.21.15.203:6789/0 conn(0x7f23f806ca80 legacy=0x7f23f8160910 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:30:15.915 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:15.912+0000 7f23fda03640 1 -- 172.21.15.37:0/3600226881 --> v2:172.21.15.37:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f23f8163e00 con 0x7f23f815a9f0 2024-04-25T10:30:15.916 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:15.913+0000 7f23fda03640 1 --2- 172.21.15.37:0/3600226881 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f23f815a9f0 0x7f23f806c540 secure :-1 s=READY pgs=429 cs=0 l=1 rev1=1 crypto rx=0x7f23f0009870 tx=0x7f23f002a270 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d45d1026822dc8f3 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:30:15.917 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:15.913+0000 7f23fca01640 1 -- 172.21.15.37:0/3600226881 <== mon.0 v2:172.21.15.37:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f23f0037070 con 0x7f23f815a9f0 2024-04-25T10:30:15.917 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:15.913+0000 7f23fca01640 1 -- 172.21.15.37:0/3600226881 <== mon.0 v2:172.21.15.37:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f23f0032440 con 0x7f23f815a9f0 2024-04-25T10:30:15.917 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:15.913+0000 7f23fca01640 1 -- 172.21.15.37:0/3600226881 <== mon.0 v2:172.21.15.37:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f23f0032810 con 0x7f23f815a9f0 2024-04-25T10:30:15.917 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:15.914+0000 7f23ffc67640 1 -- 172.21.15.37:0/3600226881 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f23f815a9f0 msgr2=0x7f23f806c540 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:30:15.917 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:15.914+0000 7f23ffc67640 1 --2- 172.21.15.37:0/3600226881 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f23f815a9f0 0x7f23f806c540 secure :-1 s=READY pgs=429 cs=0 l=1 rev1=1 crypto rx=0x7f23f0009870 tx=0x7f23f002a270 comp rx=0 tx=0).stop 2024-04-25T10:30:15.917 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:15.914+0000 7f23ffc67640 1 -- 172.21.15.37:0/3600226881 shutdown_connections 2024-04-25T10:30:15.917 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:15.914+0000 7f23ffc67640 1 --2- 172.21.15.37:0/3600226881 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f23f8161020 0x7f23f80b5860 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:30:15.917 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:15.914+0000 7f23ffc67640 1 --2- 172.21.15.37:0/3600226881 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f23f815a9f0 0x7f23f806c540 unknown :-1 s=CLOSED pgs=429 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:30:15.917 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:15.914+0000 7f23ffc67640 1 -- 172.21.15.37:0/3600226881 >> 172.21.15.37:0/3600226881 conn(0x7f23f80b4560 msgr2=0x7f23f80b4970 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:30:15.917 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:15.914+0000 7f23ffc67640 1 -- 172.21.15.37:0/3600226881 shutdown_connections 2024-04-25T10:30:15.917 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:15.914+0000 7f23ffc67640 1 -- 172.21.15.37:0/3600226881 wait complete. 2024-04-25T10:30:15.918 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:15.916+0000 7f23ffc67640 1 Processor -- start 2024-04-25T10:30:15.918 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:15.916+0000 7f23ffc67640 1 -- start start 2024-04-25T10:30:15.921 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:15.917+0000 7f23ffc67640 1 --2- >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f23f806ca80 0x7f23f81ea620 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:30:15.921 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:15.917+0000 7f23ffc67640 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f23f815a9f0 0x7f23f815ae00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:30:15.921 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:15.917+0000 7f23ffc67640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f23f8161020 0x7f23f815b340 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:30:15.921 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:15.917+0000 7f23ffc67640 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 -- 0x7f23f81646a0 con 0x7f23f8161020 2024-04-25T10:30:15.921 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:15.917+0000 7f23ffc67640 1 -- --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_getmap magic: 0 -- 0x7f23f81643c0 con 0x7f23f815a9f0 2024-04-25T10:30:15.921 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:15.917+0000 7f23ffc67640 1 -- --> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] -- mon_getmap magic: 0 -- 0x7f23f8164bc0 con 0x7f23f806ca80 2024-04-25T10:30:15.921 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:15.917+0000 7f23fe204640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f23f8161020 0x7f23f815b340 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:30:15.922 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:15.917+0000 7f23fe204640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f23f8161020 0x7f23f815b340 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.37:3300/0 says I am v2:172.21.15.37:52672/0 (socket says 172.21.15.37:52672) 2024-04-25T10:30:15.922 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:15.917+0000 7f23fe204640 1 -- 172.21.15.37:0/2799329538 learned_addr learned my addr 172.21.15.37:0/2799329538 (peer_addr_for_me v2:172.21.15.37:0/0) 2024-04-25T10:30:15.922 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:15.918+0000 7f23fda03640 1 --2- 172.21.15.37:0/2799329538 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f23f806ca80 0x7f23f81ea620 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:30:15.922 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:15.918+0000 7f23fd202640 1 --2- 172.21.15.37:0/2799329538 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f23f815a9f0 0x7f23f815ae00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:30:15.922 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:15.918+0000 7f23fe204640 1 -- 172.21.15.37:0/2799329538 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f23f806ca80 msgr2=0x7f23f81ea620 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:30:15.922 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:15.918+0000 7f23fe204640 1 --2- 172.21.15.37:0/2799329538 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f23f806ca80 0x7f23f81ea620 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:30:15.922 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:15.918+0000 7f23fe204640 1 -- 172.21.15.37:0/2799329538 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f23f815a9f0 msgr2=0x7f23f815ae00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:30:15.922 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:15.918+0000 7f23fe204640 1 --2- 172.21.15.37:0/2799329538 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f23f815a9f0 0x7f23f815ae00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:30:15.922 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:15.918+0000 7f23fe204640 1 -- 172.21.15.37:0/2799329538 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f23f81eab60 con 0x7f23f8161020 2024-04-25T10:30:15.922 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:15.918+0000 7f23fda03640 1 --2- 172.21.15.37:0/2799329538 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f23f806ca80 0x7f23f81ea620 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T10:30:15.922 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:15.918+0000 7f23fe204640 1 --2- 172.21.15.37:0/2799329538 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f23f8161020 0x7f23f815b340 secure :-1 s=READY pgs=430 cs=0 l=1 rev1=1 crypto rx=0x7f23e800ccb0 tx=0x7f23e800a5c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:30:15.922 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:15.918+0000 7f23f67fc640 1 -- 172.21.15.37:0/2799329538 <== mon.0 v2:172.21.15.37:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f23e8003990 con 0x7f23f8161020 2024-04-25T10:30:15.922 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:15.918+0000 7f23f67fc640 1 -- 172.21.15.37:0/2799329538 <== mon.0 v2:172.21.15.37:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f23e8003b50 con 0x7f23f8161020 2024-04-25T10:30:15.922 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:15.918+0000 7f23ffc67640 1 -- 172.21.15.37:0/2799329538 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f23f815d190 con 0x7f23f8161020 2024-04-25T10:30:15.923 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:15.918+0000 7f23f67fc640 1 -- 172.21.15.37:0/2799329538 <== mon.0 v2:172.21.15.37:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f23e8004020 con 0x7f23f8161020 2024-04-25T10:30:15.923 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:15.918+0000 7f23ffc67640 1 -- 172.21.15.37:0/2799329538 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f23f815d750 con 0x7f23f8161020 2024-04-25T10:30:15.924 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:15.919+0000 7f23ffc67640 1 -- 172.21.15.37:0/2799329538 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f23f815cef0 con 0x7f23f8161020 2024-04-25T10:30:15.924 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:15.920+0000 7f23f67fc640 1 -- 172.21.15.37:0/2799329538 <== mon.0 v2:172.21.15.37:3300/0 4 ==== mgrmap(e 5) ==== 96782+0+0 (secure 0 0 0) 0x7f23e80041c0 con 0x7f23f8161020 2024-04-25T10:30:15.924 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:15.920+0000 7f23f67fc640 1 --2- 172.21.15.37:0/2799329538 >> [v2:172.21.15.203:6832/2418205237,v1:172.21.15.203:6833/2418205237] conn(0x7f23e40744e0 0x7f23e40769f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:30:15.924 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:15.921+0000 7f23f67fc640 1 -- 172.21.15.37:0/2799329538 <== mon.0 v2:172.21.15.37:3300/0 5 ==== osd_map(36..36 src has 1..36) ==== 5709+0+0 (secure 0 0 0) 0x7f23e8095620 con 0x7f23f8161020 2024-04-25T10:30:15.924 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:15.921+0000 7f23fda03640 1 --2- 172.21.15.37:0/2799329538 >> [v2:172.21.15.203:6832/2418205237,v1:172.21.15.203:6833/2418205237] conn(0x7f23e40744e0 0x7f23e40769f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:30:15.924 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:15.921+0000 7f23fda03640 1 --2- 172.21.15.37:0/2799329538 >> [v2:172.21.15.203:6832/2418205237,v1:172.21.15.203:6833/2418205237] conn(0x7f23e40744e0 0x7f23e40769f0 secure :-1 s=READY pgs=216 cs=0 l=1 rev1=1 crypto rx=0x7f23f0034000 tx=0x7f23f002e000 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:30:15.927 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:15.925+0000 7f23f67fc640 1 -- 172.21.15.37:0/2799329538 <== mon.0 v2:172.21.15.37:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187246 (secure 0 0 0) 0x7f23f815cef0 con 0x7f23f8161020 2024-04-25T10:30:16.158 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:16.156+0000 7f23ffc67640 1 -- 172.21.15.37:0/2799329538 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f23f806a140 con 0x7f23f8161020 2024-04-25T10:30:16.158 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:16.156+0000 7f23f67fc640 1 -- 172.21.15.37:0/2799329538 <== mon.0 v2:172.21.15.37:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 17 v17) ==== 94+0+5181 (secure 0 0 0) 0x7f23e8014030 con 0x7f23f8161020 2024-04-25T10:30:16.159 INFO:teuthology.orchestra.run.smithi037.stdout: 2024-04-25T10:30:16.159 INFO:teuthology.orchestra.run.smithi037.stdout:{"epoch":17,"default_fscid":2,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":4790,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.203:6837/2245361396","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6836","nonce":2245361396},{"type":"v1","addr":"172.21.15.203:6837","nonce":2245361396}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":11},{"gid":5005,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.203:6835/3353571013","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6834","nonce":3353571013},{"type":"v1","addr":"172.21.15.203:6835","nonce":3353571013}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":12},{"gid":5313,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.37:6834/824801265","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.37:6833","nonce":824801265},{"type":"v1","addr":"172.21.15.37:6834","nonce":824801265}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":12}],"filesystems":[{"mdsmap":{"epoch":17,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-04-25T10:30:12.130473+0000","modified":"2024-04-25T10:30:15.157110+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":1,"in":[0],"up":{"mds_0":5316},"failed":[],"damaged":[],"stopped":[],"info":{"gid_5316":{"gid":5316,"name":"a","rank":0,"incarnation":13,"state":"up:active","state_seq":3,"addr":"172.21.15.37:6835/311552410","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.37:6832","nonce":311552410},{"type":"v1","addr":"172.21.15.37:6835","nonce":311552410}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[5],"metadata_pool":4,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":5316,"qdb_cluster":[5316]},"id":2}]} 2024-04-25T10:30:16.159 INFO:teuthology.orchestra.run.smithi037.stderr:dumped fsmap epoch 17 2024-04-25T10:30:16.162 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:16.158+0000 7f23ffc67640 1 -- 172.21.15.37:0/2799329538 >> [v2:172.21.15.203:6832/2418205237,v1:172.21.15.203:6833/2418205237] conn(0x7f23e40744e0 msgr2=0x7f23e40769f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:30:16.162 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:16.158+0000 7f23ffc67640 1 --2- 172.21.15.37:0/2799329538 >> [v2:172.21.15.203:6832/2418205237,v1:172.21.15.203:6833/2418205237] conn(0x7f23e40744e0 0x7f23e40769f0 secure :-1 s=READY pgs=216 cs=0 l=1 rev1=1 crypto rx=0x7f23f0034000 tx=0x7f23f002e000 comp rx=0 tx=0).stop 2024-04-25T10:30:16.162 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:16.158+0000 7f23ffc67640 1 -- 172.21.15.37:0/2799329538 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f23f8161020 msgr2=0x7f23f815b340 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:30:16.162 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:16.158+0000 7f23ffc67640 1 --2- 172.21.15.37:0/2799329538 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f23f8161020 0x7f23f815b340 secure :-1 s=READY pgs=430 cs=0 l=1 rev1=1 crypto rx=0x7f23e800ccb0 tx=0x7f23e800a5c0 comp rx=0 tx=0).stop 2024-04-25T10:30:16.162 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:16.159+0000 7f23ffc67640 1 -- 172.21.15.37:0/2799329538 shutdown_connections 2024-04-25T10:30:16.162 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:16.159+0000 7f23ffc67640 1 --2- 172.21.15.37:0/2799329538 >> [v2:172.21.15.203:6832/2418205237,v1:172.21.15.203:6833/2418205237] conn(0x7f23e40744e0 0x7f23e40769f0 unknown :-1 s=CLOSED pgs=216 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:30:16.162 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:16.159+0000 7f23ffc67640 1 --2- 172.21.15.37:0/2799329538 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f23f8161020 0x7f23f815b340 unknown :-1 s=CLOSED pgs=430 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:30:16.162 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:16.159+0000 7f23ffc67640 1 --2- 172.21.15.37:0/2799329538 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f23f815a9f0 0x7f23f815ae00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:30:16.162 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:16.159+0000 7f23ffc67640 1 --2- 172.21.15.37:0/2799329538 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f23f806ca80 0x7f23f81ea620 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:30:16.163 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:16.159+0000 7f23ffc67640 1 -- 172.21.15.37:0/2799329538 >> 172.21.15.37:0/2799329538 conn(0x7f23f80b4560 msgr2=0x7f23f80d2090 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:30:16.163 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:16.159+0000 7f23ffc67640 1 -- 172.21.15.37:0/2799329538 shutdown_connections 2024-04-25T10:30:16.163 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:16.159+0000 7f23ffc67640 1 -- 172.21.15.37:0/2799329538 wait complete. 2024-04-25T10:30:16.172 DEBUG:teuthology.orchestra.run.smithi037:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-04-25T10:30:16.284 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:16.282+0000 7fe370ae2640 1 Processor -- start 2024-04-25T10:30:16.284 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:16.282+0000 7fe370ae2640 1 -- start start 2024-04-25T10:30:16.286 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:16.283+0000 7fe370ae2640 1 --2- >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7fe36c0b2dc0 0x7fe36c0d2860 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:30:16.286 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:16.283+0000 7fe370ae2640 1 --2- >> v2:172.21.15.37:3300/0 conn(0x7fe36c06daf0 0x7fe36c06df00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:30:16.286 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:16.283+0000 7fe370ae2640 1 -- --> v1:172.21.15.37:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fe36c0d2e40 con 0x7fe36c06e440 2024-04-25T10:30:16.286 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:16.283+0000 7fe370ae2640 1 -- --> v2:172.21.15.37:3300/0 -- mon_getmap magic: 0 -- 0x7fe36c0b92c0 con 0x7fe36c06daf0 2024-04-25T10:30:16.286 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:16.283+0000 7fe370ae2640 1 -- --> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] -- mon_getmap magic: 0 -- 0x7fe36c0ce0d0 con 0x7fe36c0b2dc0 2024-04-25T10:30:16.286 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:16.283+0000 7fe361d9b640 1 --2- >> v2:172.21.15.37:3300/0 conn(0x7fe36c06daf0 0x7fe36c06df00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:30:16.286 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:16.283+0000 7fe36ad9d640 1 --1- >> v1:172.21.15.37:6789/0 conn(0x7fe36c06e440 0x7fe36c08cc90 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.37:6789/0 says I am v1:172.21.15.37:41074/0 (socket says 172.21.15.37:41074) 2024-04-25T10:30:16.286 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:16.283+0000 7fe361d9b640 1 --2- >> v2:172.21.15.37:3300/0 conn(0x7fe36c06daf0 0x7fe36c06df00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.37:3300/0 says I am v2:172.21.15.37:52674/0 (socket says 172.21.15.37:52674) 2024-04-25T10:30:16.287 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:16.283+0000 7fe36ad9d640 1 -- 172.21.15.37:0/2268430767 learned_addr learned my addr 172.21.15.37:0/2268430767 (peer_addr_for_me v1:172.21.15.37:0/0) 2024-04-25T10:30:16.287 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:16.284+0000 7fe36a59c640 1 --2- 172.21.15.37:0/2268430767 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7fe36c0b2dc0 0x7fe36c0d2860 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:30:16.287 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:16.284+0000 7fe361d9b640 1 -- 172.21.15.37:0/2268430767 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7fe36c0b2dc0 msgr2=0x7fe36c0d2860 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:30:16.287 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:16.284+0000 7fe361d9b640 1 --2- 172.21.15.37:0/2268430767 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7fe36c0b2dc0 0x7fe36c0d2860 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:30:16.287 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:16.284+0000 7fe361d9b640 1 -- 172.21.15.37:0/2268430767 >> v1:172.21.15.37:6789/0 conn(0x7fe36c06e440 legacy=0x7fe36c08cc90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:30:16.287 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:16.284+0000 7fe361d9b640 1 -- 172.21.15.37:0/2268430767 --> v2:172.21.15.37:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fe36c0d3120 con 0x7fe36c06daf0 2024-04-25T10:30:16.287 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:16.284+0000 7fe369d9b640 1 -- 172.21.15.37:0/2268430767 <== mon.0 v1:172.21.15.37:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 2503340646 0 0) 0x7fe36c0d2e40 con 0x7fe36c06e440 2024-04-25T10:30:16.287 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:16.284+0000 7fe36a59c640 1 --2- 172.21.15.37:0/2268430767 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7fe36c0b2dc0 0x7fe36c0d2860 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T10:30:16.288 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:16.284+0000 7fe361d9b640 1 --2- 172.21.15.37:0/2268430767 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fe36c06daf0 0x7fe36c06df00 secure :-1 s=READY pgs=432 cs=0 l=1 rev1=1 crypto rx=0x7fe3540097a0 tx=0x7fe35402a180 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d84266e87e6b0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:30:16.288 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:16.285+0000 7fe369d9b640 1 -- 172.21.15.37:0/2268430767 <== mon.0 v2:172.21.15.37:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe354037070 con 0x7fe36c06daf0 2024-04-25T10:30:16.288 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:16.285+0000 7fe369d9b640 1 -- 172.21.15.37:0/2268430767 <== mon.0 v2:172.21.15.37:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fe354032440 con 0x7fe36c06daf0 2024-04-25T10:30:16.288 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:16.285+0000 7fe369d9b640 1 -- 172.21.15.37:0/2268430767 <== mon.0 v2:172.21.15.37:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe354032810 con 0x7fe36c06daf0 2024-04-25T10:30:16.289 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:16.285+0000 7fe370ae2640 1 -- 172.21.15.37:0/2268430767 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fe36c06daf0 msgr2=0x7fe36c06df00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:30:16.289 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:16.285+0000 7fe370ae2640 1 --2- 172.21.15.37:0/2268430767 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fe36c06daf0 0x7fe36c06df00 secure :-1 s=READY pgs=432 cs=0 l=1 rev1=1 crypto rx=0x7fe3540097a0 tx=0x7fe35402a180 comp rx=0 tx=0).stop 2024-04-25T10:30:16.289 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:16.285+0000 7fe370ae2640 1 -- 172.21.15.37:0/2268430767 shutdown_connections 2024-04-25T10:30:16.289 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:16.285+0000 7fe370ae2640 1 --2- 172.21.15.37:0/2268430767 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fe36c06daf0 0x7fe36c06df00 unknown :-1 s=CLOSED pgs=432 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:30:16.289 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:16.285+0000 7fe370ae2640 1 --2- 172.21.15.37:0/2268430767 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7fe36c0b2dc0 0x7fe36c0d2860 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:30:16.289 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:16.285+0000 7fe370ae2640 1 -- 172.21.15.37:0/2268430767 >> 172.21.15.37:0/2268430767 conn(0x7fe36c0b5440 msgr2=0x7fe36c0b5850 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:30:16.289 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:16.285+0000 7fe370ae2640 1 -- 172.21.15.37:0/2268430767 shutdown_connections 2024-04-25T10:30:16.289 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:16.285+0000 7fe370ae2640 1 -- 172.21.15.37:0/2268430767 wait complete. 2024-04-25T10:30:16.289 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:16.287+0000 7fe370ae2640 1 Processor -- start 2024-04-25T10:30:16.290 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:16.287+0000 7fe370ae2640 1 -- start start 2024-04-25T10:30:16.292 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:16.289+0000 7fe370ae2640 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fe36c06daf0 0x7fe36c1fa060 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:30:16.292 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:16.289+0000 7fe370ae2640 1 --2- >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7fe36c06e440 0x7fe36c155f10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:30:16.292 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:16.289+0000 7fe370ae2640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fe36c0b2dc0 0x7fe36c0cc2c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:30:16.292 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:16.289+0000 7fe370ae2640 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 -- 0x7fe36c06e850 con 0x7fe36c0b2dc0 2024-04-25T10:30:16.292 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:16.289+0000 7fe370ae2640 1 -- --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_getmap magic: 0 -- 0x7fe36c0d36e0 con 0x7fe36c06daf0 2024-04-25T10:30:16.292 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:16.289+0000 7fe370ae2640 1 -- --> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] -- mon_getmap magic: 0 -- 0x7fe36c0cf570 con 0x7fe36c06e440 2024-04-25T10:30:16.292 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:16.289+0000 7fe36ad9d640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fe36c0b2dc0 0x7fe36c0cc2c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:30:16.293 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:16.289+0000 7fe36ad9d640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fe36c0b2dc0 0x7fe36c0cc2c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.37:3300/0 says I am v2:172.21.15.37:52688/0 (socket says 172.21.15.37:52688) 2024-04-25T10:30:16.293 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:16.289+0000 7fe36ad9d640 1 -- 172.21.15.37:0/2190665527 learned_addr learned my addr 172.21.15.37:0/2190665527 (peer_addr_for_me v2:172.21.15.37:0/0) 2024-04-25T10:30:16.293 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:16.289+0000 7fe361d9b640 1 --2- 172.21.15.37:0/2190665527 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7fe36c06e440 0x7fe36c155f10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:30:16.293 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:16.289+0000 7fe36a59c640 1 --2- 172.21.15.37:0/2190665527 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fe36c06daf0 0x7fe36c1fa060 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:30:16.293 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:16.289+0000 7fe36ad9d640 1 -- 172.21.15.37:0/2190665527 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7fe36c06e440 msgr2=0x7fe36c155f10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:30:16.293 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:16.289+0000 7fe36ad9d640 1 --2- 172.21.15.37:0/2190665527 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7fe36c06e440 0x7fe36c155f10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:30:16.293 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:16.289+0000 7fe36ad9d640 1 -- 172.21.15.37:0/2190665527 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fe36c06daf0 msgr2=0x7fe36c1fa060 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:30:16.293 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:16.289+0000 7fe36ad9d640 1 --2- 172.21.15.37:0/2190665527 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fe36c06daf0 0x7fe36c1fa060 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:30:16.293 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:16.289+0000 7fe36ad9d640 1 -- 172.21.15.37:0/2190665527 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fe36c0c93d0 con 0x7fe36c0b2dc0 2024-04-25T10:30:16.293 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:16.289+0000 7fe36a59c640 1 --2- 172.21.15.37:0/2190665527 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fe36c06daf0 0x7fe36c1fa060 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T10:30:16.293 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:16.289+0000 7fe36ad9d640 1 --2- 172.21.15.37:0/2190665527 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fe36c0b2dc0 0x7fe36c0cc2c0 secure :-1 s=READY pgs=433 cs=0 l=1 rev1=1 crypto rx=0x7fe36400c700 tx=0x7fe36400cbc0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:30:16.293 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:16.290+0000 7fe3637fe640 1 -- 172.21.15.37:0/2190665527 <== mon.0 v2:172.21.15.37:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe364018070 con 0x7fe36c0b2dc0 2024-04-25T10:30:16.293 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:16.290+0000 7fe3637fe640 1 -- 172.21.15.37:0/2190665527 <== mon.0 v2:172.21.15.37:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fe36400ce10 con 0x7fe36c0b2dc0 2024-04-25T10:30:16.293 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:16.290+0000 7fe370ae2640 1 -- 172.21.15.37:0/2190665527 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fe36c0c96b0 con 0x7fe36c0b2dc0 2024-04-25T10:30:16.294 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:16.290+0000 7fe3637fe640 1 -- 172.21.15.37:0/2190665527 <== mon.0 v2:172.21.15.37:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe36400a820 con 0x7fe36c0b2dc0 2024-04-25T10:30:16.294 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:16.290+0000 7fe370ae2640 1 -- 172.21.15.37:0/2190665527 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fe36c0c9c70 con 0x7fe36c0b2dc0 2024-04-25T10:30:16.294 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:16.291+0000 7fe370ae2640 1 -- 172.21.15.37:0/2190665527 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fe36c0c9120 con 0x7fe36c0b2dc0 2024-04-25T10:30:16.294 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:16.291+0000 7fe3637fe640 1 -- 172.21.15.37:0/2190665527 <== mon.0 v2:172.21.15.37:3300/0 4 ==== mgrmap(e 5) ==== 96782+0+0 (secure 0 0 0) 0x7fe36401b070 con 0x7fe36c0b2dc0 2024-04-25T10:30:16.295 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:16.292+0000 7fe3637fe640 1 --2- 172.21.15.37:0/2190665527 >> [v2:172.21.15.203:6832/2418205237,v1:172.21.15.203:6833/2418205237] conn(0x7fe34c0744b0 0x7fe34c0769c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:30:16.295 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:16.292+0000 7fe3637fe640 1 -- 172.21.15.37:0/2190665527 <== mon.0 v2:172.21.15.37:3300/0 5 ==== osd_map(36..36 src has 1..36) ==== 5709+0+0 (secure 0 0 0) 0x7fe364094680 con 0x7fe36c0b2dc0 2024-04-25T10:30:16.295 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:16.292+0000 7fe36a59c640 1 --2- 172.21.15.37:0/2190665527 >> [v2:172.21.15.203:6832/2418205237,v1:172.21.15.203:6833/2418205237] conn(0x7fe34c0744b0 0x7fe34c0769c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:30:16.295 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:16.293+0000 7fe36a59c640 1 --2- 172.21.15.37:0/2190665527 >> [v2:172.21.15.203:6832/2418205237,v1:172.21.15.203:6833/2418205237] conn(0x7fe34c0744b0 0x7fe34c0769c0 secure :-1 s=READY pgs=217 cs=0 l=1 rev1=1 crypto rx=0x7fe35c006fd0 tx=0x7fe35c008040 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:30:16.299 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:16.297+0000 7fe3637fe640 1 -- 172.21.15.37:0/2190665527 <== mon.0 v2:172.21.15.37:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187246 (secure 0 0 0) 0x7fe36405e7c0 con 0x7fe36c0b2dc0 2024-04-25T10:30:16.483 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:16.482+0000 7fe370ae2640 1 -- 172.21.15.37:0/2190665527 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7fe36c06a140 con 0x7fe36c0b2dc0 2024-04-25T10:30:16.485 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:16.483+0000 7fe3637fe640 1 -- 172.21.15.37:0/2190665527 <== mon.0 v2:172.21.15.37:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v36) ==== 74+0+16621 (secure 0 0 0) 0x7fe364062ac0 con 0x7fe36c0b2dc0 2024-04-25T10:30:16.485 INFO:teuthology.orchestra.run.smithi037.stdout: 2024-04-25T10:30:16.485 INFO:teuthology.orchestra.run.smithi037.stdout:{"epoch":36,"fsid":"f4acda28-be34-492a-b286-994afd36f61d","created":"2024-04-25T10:28:43.985718+0000","modified":"2024-04-25T10:30:13.756895+0000","last_up_change":"2024-04-25T10:28:54.900654+0000","last_in_change":"2024-04-25T10:28:46.216973+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":5,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-04-25T10:28:56.202153+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"19","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":4,"pool_name":"cephfs_metadata","create_time":"2024-04-25T10:30:10.008132+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"35","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.75,"score_stable":1.75,"optimal_score":1,"raw_score_acting":1.75,"raw_score_stable":1.75,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":5,"pool_name":"cephfs_data","create_time":"2024-04-25T10:30:11.084827+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"36","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"a2f79028-b332-4166-9ebb-a7024a37ea22","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":34,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.37:6816","nonce":2746074720},{"type":"v1","addr":"172.21.15.37:6817","nonce":2746074720}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.37:6818","nonce":2746074720},{"type":"v1","addr":"172.21.15.37:6819","nonce":2746074720}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.37:6822","nonce":2746074720},{"type":"v1","addr":"172.21.15.37:6823","nonce":2746074720}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.37:6820","nonce":2746074720},{"type":"v1","addr":"172.21.15.37:6821","nonce":2746074720}]},"public_addr":"172.21.15.37:6817/2746074720","cluster_addr":"172.21.15.37:6819/2746074720","heartbeat_back_addr":"172.21.15.37:6823/2746074720","heartbeat_front_addr":"172.21.15.37:6821/2746074720","state":["exists","up"]},{"osd":1,"uuid":"973159de-c2ea-4083-aecc-69373d1c1132","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":34,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.37:6800","nonce":1690341662},{"type":"v1","addr":"172.21.15.37:6801","nonce":1690341662}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.37:6802","nonce":1690341662},{"type":"v1","addr":"172.21.15.37:6803","nonce":1690341662}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.37:6806","nonce":1690341662},{"type":"v1","addr":"172.21.15.37:6807","nonce":1690341662}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.37:6804","nonce":1690341662},{"type":"v1","addr":"172.21.15.37:6805","nonce":1690341662}]},"public_addr":"172.21.15.37:6801/1690341662","cluster_addr":"172.21.15.37:6803/1690341662","heartbeat_back_addr":"172.21.15.37:6807/1690341662","heartbeat_front_addr":"172.21.15.37:6805/1690341662","state":["exists","up"]},{"osd":2,"uuid":"4ae7a0e9-cb3f-484c-a75f-f3766b721c47","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":34,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.37:6808","nonce":2754669649},{"type":"v1","addr":"172.21.15.37:6809","nonce":2754669649}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.37:6810","nonce":2754669649},{"type":"v1","addr":"172.21.15.37:6811","nonce":2754669649}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.37:6814","nonce":2754669649},{"type":"v1","addr":"172.21.15.37:6815","nonce":2754669649}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.37:6812","nonce":2754669649},{"type":"v1","addr":"172.21.15.37:6813","nonce":2754669649}]},"public_addr":"172.21.15.37:6809/2754669649","cluster_addr":"172.21.15.37:6811/2754669649","heartbeat_back_addr":"172.21.15.37:6815/2754669649","heartbeat_front_addr":"172.21.15.37:6813/2754669649","state":["exists","up"]},{"osd":3,"uuid":"1ba4a746-1608-47a7-ace2-960a5cfae934","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":34,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.37:6824","nonce":1815290618},{"type":"v1","addr":"172.21.15.37:6825","nonce":1815290618}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.37:6826","nonce":1815290618},{"type":"v1","addr":"172.21.15.37:6827","nonce":1815290618}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.37:6830","nonce":1815290618},{"type":"v1","addr":"172.21.15.37:6831","nonce":1815290618}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.37:6828","nonce":1815290618},{"type":"v1","addr":"172.21.15.37:6829","nonce":1815290618}]},"public_addr":"172.21.15.37:6825/1815290618","cluster_addr":"172.21.15.37:6827/1815290618","heartbeat_back_addr":"172.21.15.37:6831/1815290618","heartbeat_front_addr":"172.21.15.37:6829/1815290618","state":["exists","up"]},{"osd":4,"uuid":"d4f59f5d-0413-4b79-87ee-bb212c1fcf62","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":34,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6800","nonce":4051532063},{"type":"v1","addr":"172.21.15.203:6801","nonce":4051532063}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6802","nonce":4051532063},{"type":"v1","addr":"172.21.15.203:6803","nonce":4051532063}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6806","nonce":4051532063},{"type":"v1","addr":"172.21.15.203:6807","nonce":4051532063}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6804","nonce":4051532063},{"type":"v1","addr":"172.21.15.203:6805","nonce":4051532063}]},"public_addr":"172.21.15.203:6801/4051532063","cluster_addr":"172.21.15.203:6803/4051532063","heartbeat_back_addr":"172.21.15.203:6807/4051532063","heartbeat_front_addr":"172.21.15.203:6805/4051532063","state":["exists","up"]},{"osd":5,"uuid":"69710a0e-41d8-4e7f-9a10-17ce540b67be","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":34,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6824","nonce":1203822890},{"type":"v1","addr":"172.21.15.203:6825","nonce":1203822890}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6826","nonce":1203822890},{"type":"v1","addr":"172.21.15.203:6827","nonce":1203822890}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6830","nonce":1203822890},{"type":"v1","addr":"172.21.15.203:6831","nonce":1203822890}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6828","nonce":1203822890},{"type":"v1","addr":"172.21.15.203:6829","nonce":1203822890}]},"public_addr":"172.21.15.203:6825/1203822890","cluster_addr":"172.21.15.203:6827/1203822890","heartbeat_back_addr":"172.21.15.203:6831/1203822890","heartbeat_front_addr":"172.21.15.203:6829/1203822890","state":["exists","up"]},{"osd":6,"uuid":"f29dd047-3b48-473d-a235-6642a6293f20","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":34,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6810","nonce":2309615014},{"type":"v1","addr":"172.21.15.203:6812","nonce":2309615014}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6814","nonce":2309615014},{"type":"v1","addr":"172.21.15.203:6816","nonce":2309615014}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6822","nonce":2309615014},{"type":"v1","addr":"172.21.15.203:6823","nonce":2309615014}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6818","nonce":2309615014},{"type":"v1","addr":"172.21.15.203:6820","nonce":2309615014}]},"public_addr":"172.21.15.203:6812/2309615014","cluster_addr":"172.21.15.203:6816/2309615014","heartbeat_back_addr":"172.21.15.203:6823/2309615014","heartbeat_front_addr":"172.21.15.203:6820/2309615014","state":["exists","up"]},{"osd":7,"uuid":"6e691644-378b-4fdc-8e86-fb5f56e31f29","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":34,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6808","nonce":3688099014},{"type":"v1","addr":"172.21.15.203:6809","nonce":3688099014}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6811","nonce":3688099014},{"type":"v1","addr":"172.21.15.203:6813","nonce":3688099014}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6819","nonce":3688099014},{"type":"v1","addr":"172.21.15.203:6821","nonce":3688099014}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6815","nonce":3688099014},{"type":"v1","addr":"172.21.15.203:6817","nonce":3688099014}]},"public_addr":"172.21.15.203:6809/3688099014","cluster_addr":"172.21.15.203:6813/3688099014","heartbeat_back_addr":"172.21.15.203:6821/3688099014","heartbeat_front_addr":"172.21.15.203:6817/3688099014","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T10:28:51.419141+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T10:28:51.366448+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T10:28:51.348276+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T10:28:51.380519+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T10:28:51.105812+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T10:28:51.384496+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T10:28:51.388755+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T10:28:51.366958+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-04-25T10:30:16.487 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:16.485+0000 7fe370ae2640 1 -- 172.21.15.37:0/2190665527 >> [v2:172.21.15.203:6832/2418205237,v1:172.21.15.203:6833/2418205237] conn(0x7fe34c0744b0 msgr2=0x7fe34c0769c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:30:16.487 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:16.485+0000 7fe370ae2640 1 --2- 172.21.15.37:0/2190665527 >> [v2:172.21.15.203:6832/2418205237,v1:172.21.15.203:6833/2418205237] conn(0x7fe34c0744b0 0x7fe34c0769c0 secure :-1 s=READY pgs=217 cs=0 l=1 rev1=1 crypto rx=0x7fe35c006fd0 tx=0x7fe35c008040 comp rx=0 tx=0).stop 2024-04-25T10:30:16.487 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:16.485+0000 7fe370ae2640 1 -- 172.21.15.37:0/2190665527 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fe36c0b2dc0 msgr2=0x7fe36c0cc2c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:30:16.487 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:16.485+0000 7fe370ae2640 1 --2- 172.21.15.37:0/2190665527 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fe36c0b2dc0 0x7fe36c0cc2c0 secure :-1 s=READY pgs=433 cs=0 l=1 rev1=1 crypto rx=0x7fe36400c700 tx=0x7fe36400cbc0 comp rx=0 tx=0).stop 2024-04-25T10:30:16.487 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:16.485+0000 7fe370ae2640 1 -- 172.21.15.37:0/2190665527 shutdown_connections 2024-04-25T10:30:16.488 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:16.485+0000 7fe370ae2640 1 --2- 172.21.15.37:0/2190665527 >> [v2:172.21.15.203:6832/2418205237,v1:172.21.15.203:6833/2418205237] conn(0x7fe34c0744b0 0x7fe34c0769c0 unknown :-1 s=CLOSED pgs=217 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:30:16.488 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:16.485+0000 7fe370ae2640 1 --2- 172.21.15.37:0/2190665527 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fe36c0b2dc0 0x7fe36c0cc2c0 unknown :-1 s=CLOSED pgs=433 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:30:16.488 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:16.485+0000 7fe370ae2640 1 --2- 172.21.15.37:0/2190665527 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7fe36c06e440 0x7fe36c155f10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:30:16.488 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:16.485+0000 7fe370ae2640 1 --2- 172.21.15.37:0/2190665527 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fe36c06daf0 0x7fe36c1fa060 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:30:16.488 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:16.485+0000 7fe370ae2640 1 -- 172.21.15.37:0/2190665527 >> 172.21.15.37:0/2190665527 conn(0x7fe36c0b5440 msgr2=0x7fe36c072420 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:30:16.488 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:16.485+0000 7fe370ae2640 1 -- 172.21.15.37:0/2190665527 shutdown_connections 2024-04-25T10:30:16.488 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:16.485+0000 7fe370ae2640 1 -- 172.21.15.37:0/2190665527 wait complete. 2024-04-25T10:30:16.499 INFO:tasks.cephfs.filesystem.ceph_manager:waiting for clean 2024-04-25T10:30:16.499 DEBUG:teuthology.orchestra.run.smithi037:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-04-25T10:30:16.610 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:16.607+0000 7f341c43c640 1 Processor -- start 2024-04-25T10:30:16.610 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:16.607+0000 7f341c43c640 1 -- start start 2024-04-25T10:30:16.612 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:16.609+0000 7f341c43c640 1 --2- >> v2:172.21.15.37:3300/0 conn(0x7f341406e260 0x7f341406e670 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:30:16.612 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:16.609+0000 7f341c43c640 1 --2- >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f341406ebb0 0x7f34141645d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:30:16.612 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:16.609+0000 7f341c43c640 1 -- --> v1:172.21.15.203:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f341409ce70 con 0x7f3414164b10 2024-04-25T10:30:16.612 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:16.609+0000 7f341c43c640 1 -- --> v2:172.21.15.37:3300/0 -- mon_getmap magic: 0 -- 0x7f34140d3370 con 0x7f341406e260 2024-04-25T10:30:16.612 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:16.609+0000 7f341c43c640 1 -- --> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] -- mon_getmap magic: 0 -- 0x7f34140d3090 con 0x7f341406ebb0 2024-04-25T10:30:16.612 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:16.609+0000 7f341a1d8640 1 --2- >> v2:172.21.15.37:3300/0 conn(0x7f341406e260 0x7f341406e670 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:30:16.612 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:16.609+0000 7f341a1d8640 1 --2- >> v2:172.21.15.37:3300/0 conn(0x7f341406e260 0x7f341406e670 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.37:3300/0 says I am v2:172.21.15.37:52704/0 (socket says 172.21.15.37:52704) 2024-04-25T10:30:16.612 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:16.609+0000 7f341a1d8640 1 -- 172.21.15.37:0/3088831691 learned_addr learned my addr 172.21.15.37:0/3088831691 (peer_addr_for_me v2:172.21.15.37:0/0) 2024-04-25T10:30:16.612 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:16.609+0000 7f34199d7640 1 --2- 172.21.15.37:0/3088831691 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f341406ebb0 0x7f34141645d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:30:16.612 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:16.609+0000 7f341a1d8640 1 -- 172.21.15.37:0/3088831691 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f341406ebb0 msgr2=0x7f34141645d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:30:16.612 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:16.609+0000 7f341a1d8640 1 --2- 172.21.15.37:0/3088831691 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f341406ebb0 0x7f34141645d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:30:16.612 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:16.609+0000 7f341a1d8640 1 -- 172.21.15.37:0/3088831691 >> v1:172.21.15.203:6789/0 conn(0x7f3414164b10 legacy=0x7f34140d1ec0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:30:16.612 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:16.609+0000 7f341a1d8640 1 -- 172.21.15.37:0/3088831691 --> v2:172.21.15.37:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f341409d150 con 0x7f341406e260 2024-04-25T10:30:16.613 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:16.610+0000 7f341a1d8640 1 --2- 172.21.15.37:0/3088831691 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f341406e260 0x7f341406e670 secure :-1 s=READY pgs=434 cs=0 l=1 rev1=1 crypto rx=0x7f340c009870 tx=0x7f340c02a180 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ac0d3cae81e3939d server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:30:16.613 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:16.610+0000 7f34191d6640 1 -- 172.21.15.37:0/3088831691 <== mon.0 v2:172.21.15.37:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f340c037070 con 0x7f341406e260 2024-04-25T10:30:16.614 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:16.610+0000 7f34191d6640 1 -- 172.21.15.37:0/3088831691 <== mon.0 v2:172.21.15.37:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f340c032440 con 0x7f341406e260 2024-04-25T10:30:16.614 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:16.610+0000 7f34191d6640 1 -- 172.21.15.37:0/3088831691 <== mon.0 v2:172.21.15.37:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f340c032810 con 0x7f341406e260 2024-04-25T10:30:16.614 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:16.611+0000 7f341c43c640 1 -- 172.21.15.37:0/3088831691 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f341406e260 msgr2=0x7f341406e670 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:30:16.614 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:16.611+0000 7f341c43c640 1 --2- 172.21.15.37:0/3088831691 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f341406e260 0x7f341406e670 secure :-1 s=READY pgs=434 cs=0 l=1 rev1=1 crypto rx=0x7f340c009870 tx=0x7f340c02a180 comp rx=0 tx=0).stop 2024-04-25T10:30:16.614 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:16.611+0000 7f341c43c640 1 -- 172.21.15.37:0/3088831691 shutdown_connections 2024-04-25T10:30:16.614 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:16.611+0000 7f341c43c640 1 --2- 172.21.15.37:0/3088831691 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f341406ebb0 0x7f34141645d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:30:16.614 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:16.611+0000 7f341c43c640 1 --2- 172.21.15.37:0/3088831691 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f341406e260 0x7f341406e670 unknown :-1 s=CLOSED pgs=434 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:30:16.614 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:16.611+0000 7f341c43c640 1 -- 172.21.15.37:0/3088831691 >> 172.21.15.37:0/3088831691 conn(0x7f34140d0390 msgr2=0x7f34140d07a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:30:16.614 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:16.611+0000 7f341c43c640 1 -- 172.21.15.37:0/3088831691 shutdown_connections 2024-04-25T10:30:16.614 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:16.611+0000 7f341c43c640 1 -- 172.21.15.37:0/3088831691 wait complete. 2024-04-25T10:30:16.615 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:16.613+0000 7f341c43c640 1 Processor -- start 2024-04-25T10:30:16.615 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:16.613+0000 7f341c43c640 1 -- start start 2024-04-25T10:30:16.618 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:16.614+0000 7f341c43c640 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f341406e260 0x7f3414160640 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:30:16.618 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:16.614+0000 7f341c43c640 1 --2- >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f341406ebb0 0x7f341415dea0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:30:16.618 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:16.614+0000 7f341c43c640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f3414164b10 0x7f34140b77b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:30:16.618 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:16.614+0000 7f341c43c640 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 -- 0x7f341406db30 con 0x7f3414164b10 2024-04-25T10:30:16.618 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:16.614+0000 7f341c43c640 1 -- --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_getmap magic: 0 -- 0x7f341406d830 con 0x7f341406e260 2024-04-25T10:30:16.618 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:16.614+0000 7f341c43c640 1 -- --> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] -- mon_getmap magic: 0 -- 0x7f341406dcb0 con 0x7f341406ebb0 2024-04-25T10:30:16.618 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:16.614+0000 7f341a9d9640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f3414164b10 0x7f34140b77b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:30:16.618 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:16.614+0000 7f341a9d9640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f3414164b10 0x7f34140b77b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.37:3300/0 says I am v2:172.21.15.37:52712/0 (socket says 172.21.15.37:52712) 2024-04-25T10:30:16.618 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:16.614+0000 7f341a9d9640 1 -- 172.21.15.37:0/3902983572 learned_addr learned my addr 172.21.15.37:0/3902983572 (peer_addr_for_me v2:172.21.15.37:0/0) 2024-04-25T10:30:16.618 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:16.615+0000 7f341a1d8640 1 --2- 172.21.15.37:0/3902983572 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f341406e260 0x7f3414160640 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:30:16.618 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:16.615+0000 7f34199d7640 1 --2- 172.21.15.37:0/3902983572 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f341406ebb0 0x7f341415dea0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:30:16.618 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:16.615+0000 7f341a9d9640 1 -- 172.21.15.37:0/3902983572 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f341406ebb0 msgr2=0x7f341415dea0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:30:16.618 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:16.615+0000 7f341a9d9640 1 --2- 172.21.15.37:0/3902983572 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f341406ebb0 0x7f341415dea0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:30:16.619 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:16.615+0000 7f341a9d9640 1 -- 172.21.15.37:0/3902983572 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f341406e260 msgr2=0x7f3414160640 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:30:16.619 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:16.615+0000 7f341a9d9640 1 --2- 172.21.15.37:0/3902983572 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f341406e260 0x7f3414160640 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:30:16.619 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:16.615+0000 7f341a9d9640 1 -- 172.21.15.37:0/3902983572 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f34141f6140 con 0x7f3414164b10 2024-04-25T10:30:16.619 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:16.615+0000 7f341a9d9640 1 --2- 172.21.15.37:0/3902983572 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f3414164b10 0x7f34140b77b0 secure :-1 s=READY pgs=435 cs=0 l=1 rev1=1 crypto rx=0x7f340400ba10 tx=0x7f340400bed0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:30:16.619 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:16.615+0000 7f3412ffd640 1 -- 172.21.15.37:0/3902983572 <== mon.0 v2:172.21.15.37:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3404019070 con 0x7f3414164b10 2024-04-25T10:30:16.619 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:16.615+0000 7f3412ffd640 1 -- 172.21.15.37:0/3902983572 <== mon.0 v2:172.21.15.37:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f340400cb40 con 0x7f3414164b10 2024-04-25T10:30:16.619 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:16.615+0000 7f3412ffd640 1 -- 172.21.15.37:0/3902983572 <== mon.0 v2:172.21.15.37:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3404014440 con 0x7f3414164b10 2024-04-25T10:30:16.619 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:16.615+0000 7f341c43c640 1 -- 172.21.15.37:0/3902983572 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f341415ed70 con 0x7f3414164b10 2024-04-25T10:30:16.619 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:16.615+0000 7f341c43c640 1 -- 172.21.15.37:0/3902983572 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f341415f330 con 0x7f3414164b10 2024-04-25T10:30:16.620 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:16.616+0000 7f341c43c640 1 -- 172.21.15.37:0/3902983572 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f341406a140 con 0x7f3414164b10 2024-04-25T10:30:16.620 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:16.617+0000 7f3412ffd640 1 -- 172.21.15.37:0/3902983572 <== mon.0 v2:172.21.15.37:3300/0 4 ==== mgrmap(e 5) ==== 96782+0+0 (secure 0 0 0) 0x7f340401c070 con 0x7f3414164b10 2024-04-25T10:30:16.620 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:16.617+0000 7f3412ffd640 1 --2- 172.21.15.37:0/3902983572 >> [v2:172.21.15.203:6832/2418205237,v1:172.21.15.203:6833/2418205237] conn(0x7f34000744b0 0x7f34000769c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:30:16.620 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:16.617+0000 7f3412ffd640 1 -- 172.21.15.37:0/3902983572 <== mon.0 v2:172.21.15.37:3300/0 5 ==== osd_map(36..36 src has 1..36) ==== 5709+0+0 (secure 0 0 0) 0x7f3404094d40 con 0x7f3414164b10 2024-04-25T10:30:16.620 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:16.617+0000 7f341a1d8640 1 --2- 172.21.15.37:0/3902983572 >> [v2:172.21.15.203:6832/2418205237,v1:172.21.15.203:6833/2418205237] conn(0x7f34000744b0 0x7f34000769c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:30:16.620 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:16.618+0000 7f341a1d8640 1 --2- 172.21.15.37:0/3902983572 >> [v2:172.21.15.203:6832/2418205237,v1:172.21.15.203:6833/2418205237] conn(0x7f34000744b0 0x7f34000769c0 secure :-1 s=READY pgs=218 cs=0 l=1 rev1=1 crypto rx=0x7f340c034000 tx=0x7f340c02e000 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:30:16.624 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:16.622+0000 7f3412ffd640 1 -- 172.21.15.37:0/3902983572 <== mon.0 v2:172.21.15.37:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187246 (secure 0 0 0) 0x7f340405ee80 con 0x7f3414164b10 2024-04-25T10:30:16.731 INFO:tasks.ceph.mds.a.smithi037.stderr:2024-04-25T10:30:16.729+0000 7f5d17c43640 -1 mds.pinger is_rank_lagging: rank=0 was never sent ping request. 2024-04-25T10:30:16.804 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:16.802+0000 7f341c43c640 1 -- 172.21.15.37:0/3902983572 --> [v2:172.21.15.203:6832/2418205237,v1:172.21.15.203:6833/2418205237] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) -- 0x7f34141f6310 con 0x7f34000744b0 2024-04-25T10:30:16.811 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:16.809+0000 7f3412ffd640 1 -- 172.21.15.37:0/3902983572 <== mgr.4100 v2:172.21.15.203:6832/2418205237 1 ==== mgr_command_reply(tid 0: 0 dumped all) ==== 18+0+251930 (secure 0 0 0) 0x7f340c0326f0 con 0x7f34000744b0 2024-04-25T10:30:16.815 INFO:teuthology.orchestra.run.smithi037.stdout: 2024-04-25T10:30:16.816 INFO:teuthology.orchestra.run.smithi037.stderr:dumped all 2024-04-25T10:30:16.818 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:16.812+0000 7f341c43c640 1 -- 172.21.15.37:0/3902983572 >> [v2:172.21.15.203:6832/2418205237,v1:172.21.15.203:6833/2418205237] conn(0x7f34000744b0 msgr2=0x7f34000769c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:30:16.818 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:16.812+0000 7f341c43c640 1 --2- 172.21.15.37:0/3902983572 >> [v2:172.21.15.203:6832/2418205237,v1:172.21.15.203:6833/2418205237] conn(0x7f34000744b0 0x7f34000769c0 secure :-1 s=READY pgs=218 cs=0 l=1 rev1=1 crypto rx=0x7f340c034000 tx=0x7f340c02e000 comp rx=0 tx=0).stop 2024-04-25T10:30:16.818 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:16.812+0000 7f341c43c640 1 -- 172.21.15.37:0/3902983572 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f3414164b10 msgr2=0x7f34140b77b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:30:16.818 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:16.812+0000 7f341c43c640 1 --2- 172.21.15.37:0/3902983572 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f3414164b10 0x7f34140b77b0 secure :-1 s=READY pgs=435 cs=0 l=1 rev1=1 crypto rx=0x7f340400ba10 tx=0x7f340400bed0 comp rx=0 tx=0).stop 2024-04-25T10:30:16.818 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:16.812+0000 7f341c43c640 1 -- 172.21.15.37:0/3902983572 shutdown_connections 2024-04-25T10:30:16.818 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:16.812+0000 7f341c43c640 1 --2- 172.21.15.37:0/3902983572 >> [v2:172.21.15.203:6832/2418205237,v1:172.21.15.203:6833/2418205237] conn(0x7f34000744b0 0x7f34000769c0 unknown :-1 s=CLOSED pgs=218 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:30:16.818 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:16.812+0000 7f341c43c640 1 --2- 172.21.15.37:0/3902983572 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f3414164b10 0x7f34140b77b0 unknown :-1 s=CLOSED pgs=435 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:30:16.818 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:16.812+0000 7f341c43c640 1 --2- 172.21.15.37:0/3902983572 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f341406ebb0 0x7f341415dea0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:30:16.819 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:16.812+0000 7f341c43c640 1 --2- 172.21.15.37:0/3902983572 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f341406e260 0x7f3414160640 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:30:16.819 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:16.812+0000 7f341c43c640 1 -- 172.21.15.37:0/3902983572 >> 172.21.15.37:0/3902983572 conn(0x7f34140d0390 msgr2=0x7f34140b3270 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:30:16.819 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:16.812+0000 7f341c43c640 1 -- 172.21.15.37:0/3902983572 shutdown_connections 2024-04-25T10:30:16.819 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:16.812+0000 7f341c43c640 1 -- 172.21.15.37:0/3902983572 wait complete. 2024-04-25T10:30:16.826 INFO:teuthology.orchestra.run.smithi037.stdout:{"pg_ready":true,"pg_map":{"version":71,"stamp":"2024-04-25T10:30:16.203949+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":592778,"num_objects":24,"num_object_clones":0,"num_object_copies":48,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":24,"num_whiteouts":0,"num_read":66,"num_read_kb":55,"num_write":146,"num_write_kb":1642,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":14,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":111,"ondisk_log_size":111,"up":194,"acting":194,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":126,"num_osds":8,"num_per_pool_osds":8,"num_per_pool_omap_osds":8,"kb":754974720,"kb_used":220572,"kb_used_data":4348,"kb_used_omap":12,"kb_used_meta":216051,"kb_avail":754754148,"statfs":{"total":773094113280,"available":772868247552,"internally_reserved":0,"allocated":4452352,"data_stored":2430640,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":12752,"internal_metadata":221236784},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":30,"apply_latency_ms":30,"commit_latency_ns":30000000,"apply_latency_ns":30000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":2410,"num_objects":22,"num_object_clones":0,"num_object_copies":44,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":22,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":45,"num_write_kb":14,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":14,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"5.464383"},"pg_stats":[{"pgid":"5.3f","version":"0'0","reported_seq":12,"reported_epoch":36,"state":"active+clean","last_fresh":"2024-04-25T10:30:13.762926+0000","last_change":"2024-04-25T10:30:12.772509+0000","last_active":"2024-04-25T10:30:13.762926+0000","last_peered":"2024-04-25T10:30:13.762926+0000","last_clean":"2024-04-25T10:30:13.762926+0000","last_became_active":"2024-04-25T10:30:12.772018+0000","last_became_peered":"2024-04-25T10:30:12.772018+0000","last_unstale":"2024-04-25T10:30:13.762926+0000","last_undegraded":"2024-04-25T10:30:13.762926+0000","last_fullsized":"2024-04-25T10:30:13.762926+0000","mapping_epoch":34,"log_start":"0'0","ondisk_log_start":"0'0","created":34,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:30:11.733978+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:30:11.733978+0000","last_clean_scrub_stamp":"2024-04-25T10:30:11.733978+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T19:25:06.397830+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"5.3e","version":"0'0","reported_seq":12,"reported_epoch":36,"state":"active+clean","last_fresh":"2024-04-25T10:30:14.411090+0000","last_change":"2024-04-25T10:30:12.780210+0000","last_active":"2024-04-25T10:30:14.411090+0000","last_peered":"2024-04-25T10:30:14.411090+0000","last_clean":"2024-04-25T10:30:14.411090+0000","last_became_active":"2024-04-25T10:30:12.779867+0000","last_became_peered":"2024-04-25T10:30:12.779867+0000","last_unstale":"2024-04-25T10:30:14.411090+0000","last_undegraded":"2024-04-25T10:30:14.411090+0000","last_fullsized":"2024-04-25T10:30:14.411090+0000","mapping_epoch":34,"log_start":"0'0","ondisk_log_start":"0'0","created":34,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:30:11.733978+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:30:11.733978+0000","last_clean_scrub_stamp":"2024-04-25T10:30:11.733978+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T12:27:54.564692+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"5.3d","version":"0'0","reported_seq":12,"reported_epoch":36,"state":"active+clean","last_fresh":"2024-04-25T10:30:14.411226+0000","last_change":"2024-04-25T10:30:12.768292+0000","last_active":"2024-04-25T10:30:14.411226+0000","last_peered":"2024-04-25T10:30:14.411226+0000","last_clean":"2024-04-25T10:30:14.411226+0000","last_became_active":"2024-04-25T10:30:12.767909+0000","last_became_peered":"2024-04-25T10:30:12.767909+0000","last_unstale":"2024-04-25T10:30:14.411226+0000","last_undegraded":"2024-04-25T10:30:14.411226+0000","last_fullsized":"2024-04-25T10:30:14.411226+0000","mapping_epoch":34,"log_start":"0'0","ondisk_log_start":"0'0","created":34,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:30:11.733978+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:30:11.733978+0000","last_clean_scrub_stamp":"2024-04-25T10:30:11.733978+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T16:09:05.086563+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"5.3c","version":"0'0","reported_seq":9,"reported_epoch":35,"state":"active+clean","last_fresh":"2024-04-25T10:30:12.785547+0000","last_change":"2024-04-25T10:30:12.781723+0000","last_active":"2024-04-25T10:30:12.785547+0000","last_peered":"2024-04-25T10:30:12.785547+0000","last_clean":"2024-04-25T10:30:12.785547+0000","last_became_active":"2024-04-25T10:30:12.781357+0000","last_became_peered":"2024-04-25T10:30:12.781357+0000","last_unstale":"2024-04-25T10:30:12.785547+0000","last_undegraded":"2024-04-25T10:30:12.785547+0000","last_fullsized":"2024-04-25T10:30:12.785547+0000","mapping_epoch":34,"log_start":"0'0","ondisk_log_start":"0'0","created":34,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:30:11.733978+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:30:11.733978+0000","last_clean_scrub_stamp":"2024-04-25T10:30:11.733978+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T16:17:39.250622+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.3b","version":"0'0","reported_seq":12,"reported_epoch":36,"state":"active+clean","last_fresh":"2024-04-25T10:30:14.411497+0000","last_change":"2024-04-25T10:30:12.762141+0000","last_active":"2024-04-25T10:30:14.411497+0000","last_peered":"2024-04-25T10:30:14.411497+0000","last_clean":"2024-04-25T10:30:14.411497+0000","last_became_active":"2024-04-25T10:30:12.761805+0000","last_became_peered":"2024-04-25T10:30:12.761805+0000","last_unstale":"2024-04-25T10:30:14.411497+0000","last_undegraded":"2024-04-25T10:30:14.411497+0000","last_fullsized":"2024-04-25T10:30:14.411497+0000","mapping_epoch":34,"log_start":"0'0","ondisk_log_start":"0'0","created":34,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:30:11.733978+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:30:11.733978+0000","last_clean_scrub_stamp":"2024-04-25T10:30:11.733978+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T18:52:45.539758+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6],"acting":[5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"5.3a","version":"0'0","reported_seq":12,"reported_epoch":36,"state":"active+clean","last_fresh":"2024-04-25T10:30:14.411756+0000","last_change":"2024-04-25T10:30:12.775534+0000","last_active":"2024-04-25T10:30:14.411756+0000","last_peered":"2024-04-25T10:30:14.411756+0000","last_clean":"2024-04-25T10:30:14.411756+0000","last_became_active":"2024-04-25T10:30:12.775180+0000","last_became_peered":"2024-04-25T10:30:12.775180+0000","last_unstale":"2024-04-25T10:30:14.411756+0000","last_undegraded":"2024-04-25T10:30:14.411756+0000","last_fullsized":"2024-04-25T10:30:14.411756+0000","mapping_epoch":34,"log_start":"0'0","ondisk_log_start":"0'0","created":34,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:30:11.733978+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:30:11.733978+0000","last_clean_scrub_stamp":"2024-04-25T10:30:11.733978+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T17:36:19.165820+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"5.39","version":"0'0","reported_seq":12,"reported_epoch":36,"state":"active+clean","last_fresh":"2024-04-25T10:30:13.762763+0000","last_change":"2024-04-25T10:30:12.781210+0000","last_active":"2024-04-25T10:30:13.762763+0000","last_peered":"2024-04-25T10:30:13.762763+0000","last_clean":"2024-04-25T10:30:13.762763+0000","last_became_active":"2024-04-25T10:30:12.780874+0000","last_became_peered":"2024-04-25T10:30:12.780874+0000","last_unstale":"2024-04-25T10:30:13.762763+0000","last_undegraded":"2024-04-25T10:30:13.762763+0000","last_fullsized":"2024-04-25T10:30:13.762763+0000","mapping_epoch":34,"log_start":"0'0","ondisk_log_start":"0'0","created":34,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:30:11.733978+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:30:11.733978+0000","last_clean_scrub_stamp":"2024-04-25T10:30:11.733978+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T14:16:42.719469+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"5.38","version":"0'0","reported_seq":12,"reported_epoch":36,"state":"active+clean","last_fresh":"2024-04-25T10:30:13.762365+0000","last_change":"2024-04-25T10:30:12.774614+0000","last_active":"2024-04-25T10:30:13.762365+0000","last_peered":"2024-04-25T10:30:13.762365+0000","last_clean":"2024-04-25T10:30:13.762365+0000","last_became_active":"2024-04-25T10:30:12.774213+0000","last_became_peered":"2024-04-25T10:30:12.774213+0000","last_unstale":"2024-04-25T10:30:13.762365+0000","last_undegraded":"2024-04-25T10:30:13.762365+0000","last_fullsized":"2024-04-25T10:30:13.762365+0000","mapping_epoch":34,"log_start":"0'0","ondisk_log_start":"0'0","created":34,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:30:11.733978+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:30:11.733978+0000","last_clean_scrub_stamp":"2024-04-25T10:30:11.733978+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T15:17:30.002665+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,6],"acting":[2,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"5.37","version":"0'0","reported_seq":12,"reported_epoch":36,"state":"active+clean","last_fresh":"2024-04-25T10:30:14.411269+0000","last_change":"2024-04-25T10:30:12.773184+0000","last_active":"2024-04-25T10:30:14.411269+0000","last_peered":"2024-04-25T10:30:14.411269+0000","last_clean":"2024-04-25T10:30:14.411269+0000","last_became_active":"2024-04-25T10:30:12.772866+0000","last_became_peered":"2024-04-25T10:30:12.772866+0000","last_unstale":"2024-04-25T10:30:14.411269+0000","last_undegraded":"2024-04-25T10:30:14.411269+0000","last_fullsized":"2024-04-25T10:30:14.411269+0000","mapping_epoch":34,"log_start":"0'0","ondisk_log_start":"0'0","created":34,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:30:11.733978+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:30:11.733978+0000","last_clean_scrub_stamp":"2024-04-25T10:30:11.733978+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T17:24:14.853132+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"5.36","version":"0'0","reported_seq":9,"reported_epoch":35,"state":"active+clean","last_fresh":"2024-04-25T10:30:12.778693+0000","last_change":"2024-04-25T10:30:12.773138+0000","last_active":"2024-04-25T10:30:12.778693+0000","last_peered":"2024-04-25T10:30:12.778693+0000","last_clean":"2024-04-25T10:30:12.778693+0000","last_became_active":"2024-04-25T10:30:12.772854+0000","last_became_peered":"2024-04-25T10:30:12.772854+0000","last_unstale":"2024-04-25T10:30:12.778693+0000","last_undegraded":"2024-04-25T10:30:12.778693+0000","last_fullsized":"2024-04-25T10:30:12.778693+0000","mapping_epoch":34,"log_start":"0'0","ondisk_log_start":"0'0","created":34,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:30:11.733978+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:30:11.733978+0000","last_clean_scrub_stamp":"2024-04-25T10:30:11.733978+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T12:08:30.050992+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5],"acting":[4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.35","version":"0'0","reported_seq":12,"reported_epoch":36,"state":"active+clean","last_fresh":"2024-04-25T10:30:13.760760+0000","last_change":"2024-04-25T10:30:12.776164+0000","last_active":"2024-04-25T10:30:13.760760+0000","last_peered":"2024-04-25T10:30:13.760760+0000","last_clean":"2024-04-25T10:30:13.760760+0000","last_became_active":"2024-04-25T10:30:12.775455+0000","last_became_peered":"2024-04-25T10:30:12.775455+0000","last_unstale":"2024-04-25T10:30:13.760760+0000","last_undegraded":"2024-04-25T10:30:13.760760+0000","last_fullsized":"2024-04-25T10:30:13.760760+0000","mapping_epoch":34,"log_start":"0'0","ondisk_log_start":"0'0","created":34,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:30:11.733978+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:30:11.733978+0000","last_clean_scrub_stamp":"2024-04-25T10:30:11.733978+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T14:08:35.756973+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.34","version":"0'0","reported_seq":12,"reported_epoch":36,"state":"active+clean","last_fresh":"2024-04-25T10:30:14.411114+0000","last_change":"2024-04-25T10:30:12.761162+0000","last_active":"2024-04-25T10:30:14.411114+0000","last_peered":"2024-04-25T10:30:14.411114+0000","last_clean":"2024-04-25T10:30:14.411114+0000","last_became_active":"2024-04-25T10:30:12.760835+0000","last_became_peered":"2024-04-25T10:30:12.760835+0000","last_unstale":"2024-04-25T10:30:14.411114+0000","last_undegraded":"2024-04-25T10:30:14.411114+0000","last_fullsized":"2024-04-25T10:30:14.411114+0000","mapping_epoch":34,"log_start":"0'0","ondisk_log_start":"0'0","created":34,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:30:11.733978+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:30:11.733978+0000","last_clean_scrub_stamp":"2024-04-25T10:30:11.733978+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T16:30:12.714228+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"5.33","version":"0'0","reported_seq":12,"reported_epoch":36,"state":"active+clean","last_fresh":"2024-04-25T10:30:13.762122+0000","last_change":"2024-04-25T10:30:12.777499+0000","last_active":"2024-04-25T10:30:13.762122+0000","last_peered":"2024-04-25T10:30:13.762122+0000","last_clean":"2024-04-25T10:30:13.762122+0000","last_became_active":"2024-04-25T10:30:12.776773+0000","last_became_peered":"2024-04-25T10:30:12.776773+0000","last_unstale":"2024-04-25T10:30:13.762122+0000","last_undegraded":"2024-04-25T10:30:13.762122+0000","last_fullsized":"2024-04-25T10:30:13.762122+0000","mapping_epoch":34,"log_start":"0'0","ondisk_log_start":"0'0","created":34,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:30:11.733978+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:30:11.733978+0000","last_clean_scrub_stamp":"2024-04-25T10:30:11.733978+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T16:49:00.268423+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"5.32","version":"0'0","reported_seq":9,"reported_epoch":35,"state":"active+clean","last_fresh":"2024-04-25T10:30:12.778573+0000","last_change":"2024-04-25T10:30:12.772243+0000","last_active":"2024-04-25T10:30:12.778573+0000","last_peered":"2024-04-25T10:30:12.778573+0000","last_clean":"2024-04-25T10:30:12.778573+0000","last_became_active":"2024-04-25T10:30:12.771936+0000","last_became_peered":"2024-04-25T10:30:12.771936+0000","last_unstale":"2024-04-25T10:30:12.778573+0000","last_undegraded":"2024-04-25T10:30:12.778573+0000","last_fullsized":"2024-04-25T10:30:12.778573+0000","mapping_epoch":34,"log_start":"0'0","ondisk_log_start":"0'0","created":34,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:30:11.733978+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:30:11.733978+0000","last_clean_scrub_stamp":"2024-04-25T10:30:11.733978+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T14:32:32.178874+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5],"acting":[4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.31","version":"0'0","reported_seq":12,"reported_epoch":36,"state":"active+clean","last_fresh":"2024-04-25T10:30:14.195534+0000","last_change":"2024-04-25T10:30:12.777324+0000","last_active":"2024-04-25T10:30:14.195534+0000","last_peered":"2024-04-25T10:30:14.195534+0000","last_clean":"2024-04-25T10:30:14.195534+0000","last_became_active":"2024-04-25T10:30:12.776983+0000","last_became_peered":"2024-04-25T10:30:12.776983+0000","last_unstale":"2024-04-25T10:30:14.195534+0000","last_undegraded":"2024-04-25T10:30:14.195534+0000","last_fullsized":"2024-04-25T10:30:14.195534+0000","mapping_epoch":34,"log_start":"0'0","ondisk_log_start":"0'0","created":34,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:30:11.733978+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:30:11.733978+0000","last_clean_scrub_stamp":"2024-04-25T10:30:11.733978+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T12:27:54.564692+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"5.30","version":"0'0","reported_seq":12,"reported_epoch":36,"state":"active+clean","last_fresh":"2024-04-25T10:30:14.195428+0000","last_change":"2024-04-25T10:30:12.765989+0000","last_active":"2024-04-25T10:30:14.195428+0000","last_peered":"2024-04-25T10:30:14.195428+0000","last_clean":"2024-04-25T10:30:14.195428+0000","last_became_active":"2024-04-25T10:30:12.765648+0000","last_became_peered":"2024-04-25T10:30:12.765648+0000","last_unstale":"2024-04-25T10:30:14.195428+0000","last_undegraded":"2024-04-25T10:30:14.195428+0000","last_fullsized":"2024-04-25T10:30:14.195428+0000","mapping_epoch":34,"log_start":"0'0","ondisk_log_start":"0'0","created":34,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:30:11.733978+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:30:11.733978+0000","last_clean_scrub_stamp":"2024-04-25T10:30:11.733978+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T16:32:01.312412+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"5.2f","version":"0'0","reported_seq":12,"reported_epoch":36,"state":"active+clean","last_fresh":"2024-04-25T10:30:13.760553+0000","last_change":"2024-04-25T10:30:12.766616+0000","last_active":"2024-04-25T10:30:13.760553+0000","last_peered":"2024-04-25T10:30:13.760553+0000","last_clean":"2024-04-25T10:30:13.760553+0000","last_became_active":"2024-04-25T10:30:12.766290+0000","last_became_peered":"2024-04-25T10:30:12.766290+0000","last_unstale":"2024-04-25T10:30:13.760553+0000","last_undegraded":"2024-04-25T10:30:13.760553+0000","last_fullsized":"2024-04-25T10:30:13.760553+0000","mapping_epoch":34,"log_start":"0'0","ondisk_log_start":"0'0","created":34,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:30:11.733978+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:30:11.733978+0000","last_clean_scrub_stamp":"2024-04-25T10:30:11.733978+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T17:35:33.561138+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.2e","version":"0'0","reported_seq":12,"reported_epoch":36,"state":"active+clean","last_fresh":"2024-04-25T10:30:14.411550+0000","last_change":"2024-04-25T10:30:12.766393+0000","last_active":"2024-04-25T10:30:14.411550+0000","last_peered":"2024-04-25T10:30:14.411550+0000","last_clean":"2024-04-25T10:30:14.411550+0000","last_became_active":"2024-04-25T10:30:12.766096+0000","last_became_peered":"2024-04-25T10:30:12.766096+0000","last_unstale":"2024-04-25T10:30:14.411550+0000","last_undegraded":"2024-04-25T10:30:14.411550+0000","last_fullsized":"2024-04-25T10:30:14.411550+0000","mapping_epoch":34,"log_start":"0'0","ondisk_log_start":"0'0","created":34,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:30:11.733978+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:30:11.733978+0000","last_clean_scrub_stamp":"2024-04-25T10:30:11.733978+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T17:58:01.091931+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"5.2d","version":"0'0","reported_seq":12,"reported_epoch":36,"state":"active+clean","last_fresh":"2024-04-25T10:30:14.411042+0000","last_change":"2024-04-25T10:30:12.768688+0000","last_active":"2024-04-25T10:30:14.411042+0000","last_peered":"2024-04-25T10:30:14.411042+0000","last_clean":"2024-04-25T10:30:14.411042+0000","last_became_active":"2024-04-25T10:30:12.768382+0000","last_became_peered":"2024-04-25T10:30:12.768382+0000","last_unstale":"2024-04-25T10:30:14.411042+0000","last_undegraded":"2024-04-25T10:30:14.411042+0000","last_fullsized":"2024-04-25T10:30:14.411042+0000","mapping_epoch":34,"log_start":"0'0","ondisk_log_start":"0'0","created":34,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:30:11.733978+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:30:11.733978+0000","last_clean_scrub_stamp":"2024-04-25T10:30:11.733978+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T17:24:14.853132+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"5.2c","version":"0'0","reported_seq":12,"reported_epoch":36,"state":"active+clean","last_fresh":"2024-04-25T10:30:13.762150+0000","last_change":"2024-04-25T10:30:12.772663+0000","last_active":"2024-04-25T10:30:13.762150+0000","last_peered":"2024-04-25T10:30:13.762150+0000","last_clean":"2024-04-25T10:30:13.762150+0000","last_became_active":"2024-04-25T10:30:12.772356+0000","last_became_peered":"2024-04-25T10:30:12.772356+0000","last_unstale":"2024-04-25T10:30:13.762150+0000","last_undegraded":"2024-04-25T10:30:13.762150+0000","last_fullsized":"2024-04-25T10:30:13.762150+0000","mapping_epoch":34,"log_start":"0'0","ondisk_log_start":"0'0","created":34,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:30:11.733978+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:30:11.733978+0000","last_clean_scrub_stamp":"2024-04-25T10:30:11.733978+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T16:08:29.987819+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"5.2b","version":"0'0","reported_seq":12,"reported_epoch":36,"state":"active+clean","last_fresh":"2024-04-25T10:30:14.195727+0000","last_change":"2024-04-25T10:30:12.771344+0000","last_active":"2024-04-25T10:30:14.195727+0000","last_peered":"2024-04-25T10:30:14.195727+0000","last_clean":"2024-04-25T10:30:14.195727+0000","last_became_active":"2024-04-25T10:30:12.770949+0000","last_became_peered":"2024-04-25T10:30:12.770949+0000","last_unstale":"2024-04-25T10:30:14.195727+0000","last_undegraded":"2024-04-25T10:30:14.195727+0000","last_fullsized":"2024-04-25T10:30:14.195727+0000","mapping_epoch":34,"log_start":"0'0","ondisk_log_start":"0'0","created":34,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:30:11.733978+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:30:11.733978+0000","last_clean_scrub_stamp":"2024-04-25T10:30:11.733978+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T15:47:40.336983+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"5.2a","version":"0'0","reported_seq":12,"reported_epoch":36,"state":"active+clean","last_fresh":"2024-04-25T10:30:13.762475+0000","last_change":"2024-04-25T10:30:12.782730+0000","last_active":"2024-04-25T10:30:13.762475+0000","last_peered":"2024-04-25T10:30:13.762475+0000","last_clean":"2024-04-25T10:30:13.762475+0000","last_became_active":"2024-04-25T10:30:12.782261+0000","last_became_peered":"2024-04-25T10:30:12.782261+0000","last_unstale":"2024-04-25T10:30:13.762475+0000","last_undegraded":"2024-04-25T10:30:13.762475+0000","last_fullsized":"2024-04-25T10:30:13.762475+0000","mapping_epoch":34,"log_start":"0'0","ondisk_log_start":"0'0","created":34,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:30:11.733978+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:30:11.733978+0000","last_clean_scrub_stamp":"2024-04-25T10:30:11.733978+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T16:08:29.987819+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4],"acting":[2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"5.29","version":"0'0","reported_seq":12,"reported_epoch":36,"state":"active+clean","last_fresh":"2024-04-25T10:30:13.761684+0000","last_change":"2024-04-25T10:30:12.763911+0000","last_active":"2024-04-25T10:30:13.761684+0000","last_peered":"2024-04-25T10:30:13.761684+0000","last_clean":"2024-04-25T10:30:13.761684+0000","last_became_active":"2024-04-25T10:30:12.763578+0000","last_became_peered":"2024-04-25T10:30:12.763578+0000","last_unstale":"2024-04-25T10:30:13.761684+0000","last_undegraded":"2024-04-25T10:30:13.761684+0000","last_fullsized":"2024-04-25T10:30:13.761684+0000","mapping_epoch":34,"log_start":"0'0","ondisk_log_start":"0'0","created":34,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:30:11.733978+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:30:11.733978+0000","last_clean_scrub_stamp":"2024-04-25T10:30:11.733978+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T18:26:06.164667+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,1],"acting":[6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"5.28","version":"0'0","reported_seq":12,"reported_epoch":36,"state":"active+clean","last_fresh":"2024-04-25T10:30:13.762434+0000","last_change":"2024-04-25T10:30:12.783394+0000","last_active":"2024-04-25T10:30:13.762434+0000","last_peered":"2024-04-25T10:30:13.762434+0000","last_clean":"2024-04-25T10:30:13.762434+0000","last_became_active":"2024-04-25T10:30:12.783066+0000","last_became_peered":"2024-04-25T10:30:12.783066+0000","last_unstale":"2024-04-25T10:30:13.762434+0000","last_undegraded":"2024-04-25T10:30:13.762434+0000","last_fullsized":"2024-04-25T10:30:13.762434+0000","mapping_epoch":34,"log_start":"0'0","ondisk_log_start":"0'0","created":34,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:30:11.733978+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:30:11.733978+0000","last_clean_scrub_stamp":"2024-04-25T10:30:11.733978+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T16:49:00.268423+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4],"acting":[2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"5.27","version":"0'0","reported_seq":9,"reported_epoch":35,"state":"active+clean","last_fresh":"2024-04-25T10:30:12.785555+0000","last_change":"2024-04-25T10:30:12.779972+0000","last_active":"2024-04-25T10:30:12.785555+0000","last_peered":"2024-04-25T10:30:12.785555+0000","last_clean":"2024-04-25T10:30:12.785555+0000","last_became_active":"2024-04-25T10:30:12.779526+0000","last_became_peered":"2024-04-25T10:30:12.779526+0000","last_unstale":"2024-04-25T10:30:12.785555+0000","last_undegraded":"2024-04-25T10:30:12.785555+0000","last_fullsized":"2024-04-25T10:30:12.785555+0000","mapping_epoch":34,"log_start":"0'0","ondisk_log_start":"0'0","created":34,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:30:11.733978+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:30:11.733978+0000","last_clean_scrub_stamp":"2024-04-25T10:30:11.733978+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T13:39:19.978606+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.26","version":"0'0","reported_seq":9,"reported_epoch":35,"state":"active+clean","last_fresh":"2024-04-25T10:30:12.778668+0000","last_change":"2024-04-25T10:30:12.771420+0000","last_active":"2024-04-25T10:30:12.778668+0000","last_peered":"2024-04-25T10:30:12.778668+0000","last_clean":"2024-04-25T10:30:12.778668+0000","last_became_active":"2024-04-25T10:30:12.771117+0000","last_became_peered":"2024-04-25T10:30:12.771117+0000","last_unstale":"2024-04-25T10:30:12.778668+0000","last_undegraded":"2024-04-25T10:30:12.778668+0000","last_fullsized":"2024-04-25T10:30:12.778668+0000","mapping_epoch":34,"log_start":"0'0","ondisk_log_start":"0'0","created":34,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:30:11.733978+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:30:11.733978+0000","last_clean_scrub_stamp":"2024-04-25T10:30:11.733978+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T15:06:28.094728+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.25","version":"0'0","reported_seq":12,"reported_epoch":36,"state":"active+clean","last_fresh":"2024-04-25T10:30:14.411287+0000","last_change":"2024-04-25T10:30:12.767505+0000","last_active":"2024-04-25T10:30:14.411287+0000","last_peered":"2024-04-25T10:30:14.411287+0000","last_clean":"2024-04-25T10:30:14.411287+0000","last_became_active":"2024-04-25T10:30:12.767149+0000","last_became_peered":"2024-04-25T10:30:12.767149+0000","last_unstale":"2024-04-25T10:30:14.411287+0000","last_undegraded":"2024-04-25T10:30:14.411287+0000","last_fullsized":"2024-04-25T10:30:14.411287+0000","mapping_epoch":34,"log_start":"0'0","ondisk_log_start":"0'0","created":34,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:30:11.733978+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:30:11.733978+0000","last_clean_scrub_stamp":"2024-04-25T10:30:11.733978+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T19:25:06.397830+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"5.24","version":"0'0","reported_seq":12,"reported_epoch":36,"state":"active+clean","last_fresh":"2024-04-25T10:30:13.762844+0000","last_change":"2024-04-25T10:30:12.775515+0000","last_active":"2024-04-25T10:30:13.762844+0000","last_peered":"2024-04-25T10:30:13.762844+0000","last_clean":"2024-04-25T10:30:13.762844+0000","last_became_active":"2024-04-25T10:30:12.775167+0000","last_became_peered":"2024-04-25T10:30:12.775167+0000","last_unstale":"2024-04-25T10:30:13.762844+0000","last_undegraded":"2024-04-25T10:30:13.762844+0000","last_fullsized":"2024-04-25T10:30:13.762844+0000","mapping_epoch":34,"log_start":"0'0","ondisk_log_start":"0'0","created":34,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:30:11.733978+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:30:11.733978+0000","last_clean_scrub_stamp":"2024-04-25T10:30:11.733978+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T17:41:34.910195+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,6],"acting":[2,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"5.23","version":"0'0","reported_seq":12,"reported_epoch":36,"state":"active+clean","last_fresh":"2024-04-25T10:30:14.195712+0000","last_change":"2024-04-25T10:30:12.781995+0000","last_active":"2024-04-25T10:30:14.195712+0000","last_peered":"2024-04-25T10:30:14.195712+0000","last_clean":"2024-04-25T10:30:14.195712+0000","last_became_active":"2024-04-25T10:30:12.781358+0000","last_became_peered":"2024-04-25T10:30:12.781358+0000","last_unstale":"2024-04-25T10:30:14.195712+0000","last_undegraded":"2024-04-25T10:30:14.195712+0000","last_fullsized":"2024-04-25T10:30:14.195712+0000","mapping_epoch":34,"log_start":"0'0","ondisk_log_start":"0'0","created":34,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:30:11.733978+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:30:11.733978+0000","last_clean_scrub_stamp":"2024-04-25T10:30:11.733978+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T18:21:32.178524+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"5.22","version":"0'0","reported_seq":12,"reported_epoch":36,"state":"active+clean","last_fresh":"2024-04-25T10:30:14.195443+0000","last_change":"2024-04-25T10:30:12.769958+0000","last_active":"2024-04-25T10:30:14.195443+0000","last_peered":"2024-04-25T10:30:14.195443+0000","last_clean":"2024-04-25T10:30:14.195443+0000","last_became_active":"2024-04-25T10:30:12.769590+0000","last_became_peered":"2024-04-25T10:30:12.769590+0000","last_unstale":"2024-04-25T10:30:14.195443+0000","last_undegraded":"2024-04-25T10:30:14.195443+0000","last_fullsized":"2024-04-25T10:30:14.195443+0000","mapping_epoch":34,"log_start":"0'0","ondisk_log_start":"0'0","created":34,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:30:11.733978+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:30:11.733978+0000","last_clean_scrub_stamp":"2024-04-25T10:30:11.733978+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T17:41:34.910195+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"5.21","version":"0'0","reported_seq":12,"reported_epoch":36,"state":"active+clean","last_fresh":"2024-04-25T10:30:13.762540+0000","last_change":"2024-04-25T10:30:12.774245+0000","last_active":"2024-04-25T10:30:13.762540+0000","last_peered":"2024-04-25T10:30:13.762540+0000","last_clean":"2024-04-25T10:30:13.762540+0000","last_became_active":"2024-04-25T10:30:12.773923+0000","last_became_peered":"2024-04-25T10:30:12.773923+0000","last_unstale":"2024-04-25T10:30:13.762540+0000","last_undegraded":"2024-04-25T10:30:13.762540+0000","last_fullsized":"2024-04-25T10:30:13.762540+0000","mapping_epoch":34,"log_start":"0'0","ondisk_log_start":"0'0","created":34,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:30:11.733978+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:30:11.733978+0000","last_clean_scrub_stamp":"2024-04-25T10:30:11.733978+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T11:32:45.617923+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,7],"acting":[2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"5.20","version":"0'0","reported_seq":12,"reported_epoch":36,"state":"active+clean","last_fresh":"2024-04-25T10:30:13.760265+0000","last_change":"2024-04-25T10:30:12.766692+0000","last_active":"2024-04-25T10:30:13.760265+0000","last_peered":"2024-04-25T10:30:13.760265+0000","last_clean":"2024-04-25T10:30:13.760265+0000","last_became_active":"2024-04-25T10:30:12.766336+0000","last_became_peered":"2024-04-25T10:30:12.766336+0000","last_unstale":"2024-04-25T10:30:13.760265+0000","last_undegraded":"2024-04-25T10:30:13.760265+0000","last_fullsized":"2024-04-25T10:30:13.760265+0000","mapping_epoch":34,"log_start":"0'0","ondisk_log_start":"0'0","created":34,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:30:11.733978+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:30:11.733978+0000","last_clean_scrub_stamp":"2024-04-25T10:30:11.733978+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T22:09:01.889527+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.1e","version":"0'0","reported_seq":12,"reported_epoch":36,"state":"active+clean","last_fresh":"2024-04-25T10:30:13.762215+0000","last_change":"2024-04-25T10:30:12.769800+0000","last_active":"2024-04-25T10:30:13.762215+0000","last_peered":"2024-04-25T10:30:13.762215+0000","last_clean":"2024-04-25T10:30:13.762215+0000","last_became_active":"2024-04-25T10:30:12.769470+0000","last_became_peered":"2024-04-25T10:30:12.769470+0000","last_unstale":"2024-04-25T10:30:13.762215+0000","last_undegraded":"2024-04-25T10:30:13.762215+0000","last_fullsized":"2024-04-25T10:30:13.762215+0000","mapping_epoch":34,"log_start":"0'0","ondisk_log_start":"0'0","created":34,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:30:11.733978+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:30:11.733978+0000","last_clean_scrub_stamp":"2024-04-25T10:30:11.733978+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T11:32:45.617923+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.1f","version":"35'4","reported_seq":18,"reported_epoch":36,"state":"active+clean","last_fresh":"2024-04-25T10:30:13.760656+0000","last_change":"2024-04-25T10:30:11.756978+0000","last_active":"2024-04-25T10:30:13.760656+0000","last_peered":"2024-04-25T10:30:13.760656+0000","last_clean":"2024-04-25T10:30:13.760656+0000","last_became_active":"2024-04-25T10:30:11.756617+0000","last_became_peered":"2024-04-25T10:30:11.756617+0000","last_unstale":"2024-04-25T10:30:13.760656+0000","last_undegraded":"2024-04-25T10:30:13.760656+0000","last_fullsized":"2024-04-25T10:30:13.760656+0000","mapping_epoch":33,"log_start":"0'0","ondisk_log_start":"0'0","created":33,"last_epoch_clean":34,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:30:10.729165+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:30:10.729165+0000","last_clean_scrub_stamp":"2024-04-25T10:30:10.729165+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T12:27:53.559880+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":582,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.1f","version":"0'0","reported_seq":12,"reported_epoch":36,"state":"active+clean","last_fresh":"2024-04-25T10:30:13.762989+0000","last_change":"2024-04-25T10:30:12.772257+0000","last_active":"2024-04-25T10:30:13.762989+0000","last_peered":"2024-04-25T10:30:13.762989+0000","last_clean":"2024-04-25T10:30:13.762989+0000","last_became_active":"2024-04-25T10:30:12.771923+0000","last_became_peered":"2024-04-25T10:30:12.771923+0000","last_unstale":"2024-04-25T10:30:13.762989+0000","last_undegraded":"2024-04-25T10:30:13.762989+0000","last_fullsized":"2024-04-25T10:30:13.762989+0000","mapping_epoch":34,"log_start":"0'0","ondisk_log_start":"0'0","created":34,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:30:11.733978+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:30:11.733978+0000","last_clean_scrub_stamp":"2024-04-25T10:30:11.733978+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T18:52:45.539758+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.1e","version":"35'1","reported_seq":13,"reported_epoch":35,"state":"active+clean","last_fresh":"2024-04-25T10:30:12.786487+0000","last_change":"2024-04-25T10:30:11.761160+0000","last_active":"2024-04-25T10:30:12.786487+0000","last_peered":"2024-04-25T10:30:12.786487+0000","last_clean":"2024-04-25T10:30:12.786487+0000","last_became_active":"2024-04-25T10:30:11.760819+0000","last_became_peered":"2024-04-25T10:30:11.760819+0000","last_unstale":"2024-04-25T10:30:12.786487+0000","last_undegraded":"2024-04-25T10:30:12.786487+0000","last_fullsized":"2024-04-25T10:30:12.786487+0000","mapping_epoch":33,"log_start":"0'0","ondisk_log_start":"0'0","created":33,"last_epoch_clean":34,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:30:10.729165+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:30:10.729165+0000","last_clean_scrub_stamp":"2024-04-25T10:30:10.729165+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T13:00:22.139475+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.1c","version":"0'0","reported_seq":12,"reported_epoch":36,"state":"active+clean","last_fresh":"2024-04-25T10:30:13.762885+0000","last_change":"2024-04-25T10:30:12.781913+0000","last_active":"2024-04-25T10:30:13.762885+0000","last_peered":"2024-04-25T10:30:13.762885+0000","last_clean":"2024-04-25T10:30:13.762885+0000","last_became_active":"2024-04-25T10:30:12.781111+0000","last_became_peered":"2024-04-25T10:30:12.781111+0000","last_unstale":"2024-04-25T10:30:13.762885+0000","last_undegraded":"2024-04-25T10:30:13.762885+0000","last_fullsized":"2024-04-25T10:30:13.762885+0000","mapping_epoch":34,"log_start":"0'0","ondisk_log_start":"0'0","created":34,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:30:11.733978+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:30:11.733978+0000","last_clean_scrub_stamp":"2024-04-25T10:30:11.733978+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T12:27:54.564692+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4],"acting":[2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.1d","version":"35'2","reported_seq":16,"reported_epoch":36,"state":"active+clean","last_fresh":"2024-04-25T10:30:14.411566+0000","last_change":"2024-04-25T10:30:11.752439+0000","last_active":"2024-04-25T10:30:14.411566+0000","last_peered":"2024-04-25T10:30:14.411566+0000","last_clean":"2024-04-25T10:30:14.411566+0000","last_became_active":"2024-04-25T10:30:11.752050+0000","last_became_peered":"2024-04-25T10:30:11.752050+0000","last_unstale":"2024-04-25T10:30:14.411566+0000","last_undegraded":"2024-04-25T10:30:14.411566+0000","last_fullsized":"2024-04-25T10:30:14.411566+0000","mapping_epoch":33,"log_start":"0'0","ondisk_log_start":"0'0","created":33,"last_epoch_clean":34,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:30:10.729165+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:30:10.729165+0000","last_clean_scrub_stamp":"2024-04-25T10:30:10.729165+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T11:41:21.358729+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"5.1d","version":"0'0","reported_seq":9,"reported_epoch":35,"state":"active+clean","last_fresh":"2024-04-25T10:30:12.779046+0000","last_change":"2024-04-25T10:30:12.773474+0000","last_active":"2024-04-25T10:30:12.779046+0000","last_peered":"2024-04-25T10:30:12.779046+0000","last_clean":"2024-04-25T10:30:12.779046+0000","last_became_active":"2024-04-25T10:30:12.773133+0000","last_became_peered":"2024-04-25T10:30:12.773133+0000","last_unstale":"2024-04-25T10:30:12.779046+0000","last_undegraded":"2024-04-25T10:30:12.779046+0000","last_fullsized":"2024-04-25T10:30:12.779046+0000","mapping_epoch":34,"log_start":"0'0","ondisk_log_start":"0'0","created":34,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:30:11.733978+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:30:11.733978+0000","last_clean_scrub_stamp":"2024-04-25T10:30:11.733978+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T15:53:06.665247+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.1c","version":"35'2","reported_seq":14,"reported_epoch":35,"state":"active+clean","last_fresh":"2024-04-25T10:30:12.785935+0000","last_change":"2024-04-25T10:30:11.759829+0000","last_active":"2024-04-25T10:30:12.785935+0000","last_peered":"2024-04-25T10:30:12.785935+0000","last_clean":"2024-04-25T10:30:12.785935+0000","last_became_active":"2024-04-25T10:30:11.759465+0000","last_became_peered":"2024-04-25T10:30:11.759465+0000","last_unstale":"2024-04-25T10:30:12.785935+0000","last_undegraded":"2024-04-25T10:30:12.785935+0000","last_fullsized":"2024-04-25T10:30:12.785935+0000","mapping_epoch":33,"log_start":"0'0","ondisk_log_start":"0'0","created":33,"last_epoch_clean":34,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:30:10.729165+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:30:10.729165+0000","last_clean_scrub_stamp":"2024-04-25T10:30:10.729165+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T13:31:53.608355+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.1a","version":"0'0","reported_seq":12,"reported_epoch":36,"state":"active+clean","last_fresh":"2024-04-25T10:30:14.411115+0000","last_change":"2024-04-25T10:30:12.773835+0000","last_active":"2024-04-25T10:30:14.411115+0000","last_peered":"2024-04-25T10:30:14.411115+0000","last_clean":"2024-04-25T10:30:14.411115+0000","last_became_active":"2024-04-25T10:30:12.773538+0000","last_became_peered":"2024-04-25T10:30:12.773538+0000","last_unstale":"2024-04-25T10:30:14.411115+0000","last_undegraded":"2024-04-25T10:30:14.411115+0000","last_fullsized":"2024-04-25T10:30:14.411115+0000","mapping_epoch":34,"log_start":"0'0","ondisk_log_start":"0'0","created":34,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:30:11.733978+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:30:11.733978+0000","last_clean_scrub_stamp":"2024-04-25T10:30:11.733978+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T16:17:55.381353+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.1b","version":"35'1","reported_seq":15,"reported_epoch":36,"state":"active+clean","last_fresh":"2024-04-25T10:30:13.762774+0000","last_change":"2024-04-25T10:30:11.753667+0000","last_active":"2024-04-25T10:30:13.762774+0000","last_peered":"2024-04-25T10:30:13.762774+0000","last_clean":"2024-04-25T10:30:13.762774+0000","last_became_active":"2024-04-25T10:30:11.753340+0000","last_became_peered":"2024-04-25T10:30:11.753340+0000","last_unstale":"2024-04-25T10:30:13.762774+0000","last_undegraded":"2024-04-25T10:30:13.762774+0000","last_fullsized":"2024-04-25T10:30:13.762774+0000","mapping_epoch":33,"log_start":"0'0","ondisk_log_start":"0'0","created":33,"last_epoch_clean":34,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:30:10.729165+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:30:10.729165+0000","last_clean_scrub_stamp":"2024-04-25T10:30:10.729165+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T16:09:04.081750+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"5.1b","version":"0'0","reported_seq":12,"reported_epoch":36,"state":"active+clean","last_fresh":"2024-04-25T10:30:13.762652+0000","last_change":"2024-04-25T10:30:12.784767+0000","last_active":"2024-04-25T10:30:13.762652+0000","last_peered":"2024-04-25T10:30:13.762652+0000","last_clean":"2024-04-25T10:30:13.762652+0000","last_became_active":"2024-04-25T10:30:12.784311+0000","last_became_peered":"2024-04-25T10:30:12.784311+0000","last_unstale":"2024-04-25T10:30:13.762652+0000","last_undegraded":"2024-04-25T10:30:13.762652+0000","last_fullsized":"2024-04-25T10:30:13.762652+0000","mapping_epoch":34,"log_start":"0'0","ondisk_log_start":"0'0","created":34,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:30:11.733978+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:30:11.733978+0000","last_clean_scrub_stamp":"2024-04-25T10:30:11.733978+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T16:32:01.312412+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,7],"acting":[2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.1a","version":"35'1","reported_seq":15,"reported_epoch":36,"state":"active+clean","last_fresh":"2024-04-25T10:30:13.762501+0000","last_change":"2024-04-25T10:30:11.758678+0000","last_active":"2024-04-25T10:30:13.762501+0000","last_peered":"2024-04-25T10:30:13.762501+0000","last_clean":"2024-04-25T10:30:13.762501+0000","last_became_active":"2024-04-25T10:30:11.758384+0000","last_became_peered":"2024-04-25T10:30:11.758384+0000","last_unstale":"2024-04-25T10:30:13.762501+0000","last_undegraded":"2024-04-25T10:30:13.762501+0000","last_fullsized":"2024-04-25T10:30:13.762501+0000","mapping_epoch":33,"log_start":"0'0","ondisk_log_start":"0'0","created":33,"last_epoch_clean":34,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:30:10.729165+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:30:10.729165+0000","last_clean_scrub_stamp":"2024-04-25T10:30:10.729165+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T16:17:54.376541+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,6],"acting":[2,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"5.18","version":"0'0","reported_seq":12,"reported_epoch":36,"state":"active+clean","last_fresh":"2024-04-25T10:30:14.410937+0000","last_change":"2024-04-25T10:30:12.769294+0000","last_active":"2024-04-25T10:30:14.410937+0000","last_peered":"2024-04-25T10:30:14.410937+0000","last_clean":"2024-04-25T10:30:14.410937+0000","last_became_active":"2024-04-25T10:30:12.768543+0000","last_became_peered":"2024-04-25T10:30:12.768543+0000","last_unstale":"2024-04-25T10:30:14.410937+0000","last_undegraded":"2024-04-25T10:30:14.410937+0000","last_fullsized":"2024-04-25T10:30:14.410937+0000","mapping_epoch":34,"log_start":"0'0","ondisk_log_start":"0'0","created":34,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:30:11.733978+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:30:11.733978+0000","last_clean_scrub_stamp":"2024-04-25T10:30:11.733978+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T16:49:21.091826+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.19","version":"0'0","reported_seq":14,"reported_epoch":36,"state":"active+clean","last_fresh":"2024-04-25T10:30:14.410830+0000","last_change":"2024-04-25T10:30:11.757481+0000","last_active":"2024-04-25T10:30:14.410830+0000","last_peered":"2024-04-25T10:30:14.410830+0000","last_clean":"2024-04-25T10:30:14.410830+0000","last_became_active":"2024-04-25T10:30:11.757149+0000","last_became_peered":"2024-04-25T10:30:11.757149+0000","last_unstale":"2024-04-25T10:30:14.410830+0000","last_undegraded":"2024-04-25T10:30:14.410830+0000","last_fullsized":"2024-04-25T10:30:14.410830+0000","mapping_epoch":33,"log_start":"0'0","ondisk_log_start":"0'0","created":33,"last_epoch_clean":34,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:30:10.729165+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:30:10.729165+0000","last_clean_scrub_stamp":"2024-04-25T10:30:10.729165+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T13:23:15.814583+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"5.19","version":"0'0","reported_seq":9,"reported_epoch":35,"state":"active+clean","last_fresh":"2024-04-25T10:30:12.785526+0000","last_change":"2024-04-25T10:30:12.780437+0000","last_active":"2024-04-25T10:30:12.785526+0000","last_peered":"2024-04-25T10:30:12.785526+0000","last_clean":"2024-04-25T10:30:12.785526+0000","last_became_active":"2024-04-25T10:30:12.779762+0000","last_became_peered":"2024-04-25T10:30:12.779762+0000","last_unstale":"2024-04-25T10:30:12.785526+0000","last_undegraded":"2024-04-25T10:30:12.785526+0000","last_fullsized":"2024-04-25T10:30:12.785526+0000","mapping_epoch":34,"log_start":"0'0","ondisk_log_start":"0'0","created":34,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:30:11.733978+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:30:11.733978+0000","last_clean_scrub_stamp":"2024-04-25T10:30:11.733978+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T13:43:38.026914+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.18","version":"0'0","reported_seq":14,"reported_epoch":36,"state":"active+clean","last_fresh":"2024-04-25T10:30:13.762556+0000","last_change":"2024-04-25T10:30:11.756627+0000","last_active":"2024-04-25T10:30:13.762556+0000","last_peered":"2024-04-25T10:30:13.762556+0000","last_clean":"2024-04-25T10:30:13.762556+0000","last_became_active":"2024-04-25T10:30:11.756323+0000","last_became_peered":"2024-04-25T10:30:11.756323+0000","last_unstale":"2024-04-25T10:30:13.762556+0000","last_undegraded":"2024-04-25T10:30:13.762556+0000","last_fullsized":"2024-04-25T10:30:13.762556+0000","mapping_epoch":33,"log_start":"0'0","ondisk_log_start":"0'0","created":33,"last_epoch_clean":34,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:30:10.729165+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:30:10.729165+0000","last_clean_scrub_stamp":"2024-04-25T10:30:10.729165+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T16:49:20.087013+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4],"acting":[2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"5.16","version":"0'0","reported_seq":12,"reported_epoch":36,"state":"active+clean","last_fresh":"2024-04-25T10:30:13.760859+0000","last_change":"2024-04-25T10:30:12.770216+0000","last_active":"2024-04-25T10:30:13.760859+0000","last_peered":"2024-04-25T10:30:13.760859+0000","last_clean":"2024-04-25T10:30:13.760859+0000","last_became_active":"2024-04-25T10:30:12.769785+0000","last_became_peered":"2024-04-25T10:30:12.769785+0000","last_unstale":"2024-04-25T10:30:13.760859+0000","last_undegraded":"2024-04-25T10:30:13.760859+0000","last_fullsized":"2024-04-25T10:30:13.760859+0000","mapping_epoch":34,"log_start":"0'0","ondisk_log_start":"0'0","created":34,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:30:11.733978+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:30:11.733978+0000","last_clean_scrub_stamp":"2024-04-25T10:30:11.733978+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T18:12:41.326304+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.17","version":"0'0","reported_seq":14,"reported_epoch":36,"state":"active+clean","last_fresh":"2024-04-25T10:30:14.411352+0000","last_change":"2024-04-25T10:30:11.758416+0000","last_active":"2024-04-25T10:30:14.411352+0000","last_peered":"2024-04-25T10:30:14.411352+0000","last_clean":"2024-04-25T10:30:14.411352+0000","last_became_active":"2024-04-25T10:30:11.758049+0000","last_became_peered":"2024-04-25T10:30:11.758049+0000","last_unstale":"2024-04-25T10:30:14.411352+0000","last_undegraded":"2024-04-25T10:30:14.411352+0000","last_fullsized":"2024-04-25T10:30:14.411352+0000","mapping_epoch":33,"log_start":"0'0","ondisk_log_start":"0'0","created":33,"last_epoch_clean":34,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:30:10.729165+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:30:10.729165+0000","last_clean_scrub_stamp":"2024-04-25T10:30:10.729165+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T18:26:05.159854+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"5.17","version":"0'0","reported_seq":9,"reported_epoch":35,"state":"active+clean","last_fresh":"2024-04-25T10:30:12.778640+0000","last_change":"2024-04-25T10:30:12.773113+0000","last_active":"2024-04-25T10:30:12.778640+0000","last_peered":"2024-04-25T10:30:12.778640+0000","last_clean":"2024-04-25T10:30:12.778640+0000","last_became_active":"2024-04-25T10:30:12.772786+0000","last_became_peered":"2024-04-25T10:30:12.772786+0000","last_unstale":"2024-04-25T10:30:12.778640+0000","last_undegraded":"2024-04-25T10:30:12.778640+0000","last_fullsized":"2024-04-25T10:30:12.778640+0000","mapping_epoch":34,"log_start":"0'0","ondisk_log_start":"0'0","created":34,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:30:11.733978+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:30:11.733978+0000","last_clean_scrub_stamp":"2024-04-25T10:30:11.733978+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T18:06:42.983247+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5],"acting":[4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.16","version":"0'0","reported_seq":14,"reported_epoch":36,"state":"active+clean","last_fresh":"2024-04-25T10:30:14.411211+0000","last_change":"2024-04-25T10:30:11.749473+0000","last_active":"2024-04-25T10:30:14.411211+0000","last_peered":"2024-04-25T10:30:14.411211+0000","last_clean":"2024-04-25T10:30:14.411211+0000","last_became_active":"2024-04-25T10:30:11.749135+0000","last_became_peered":"2024-04-25T10:30:11.749135+0000","last_unstale":"2024-04-25T10:30:14.411211+0000","last_undegraded":"2024-04-25T10:30:14.411211+0000","last_fullsized":"2024-04-25T10:30:14.411211+0000","mapping_epoch":33,"log_start":"0'0","ondisk_log_start":"0'0","created":33,"last_epoch_clean":34,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:30:10.729165+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:30:10.729165+0000","last_clean_scrub_stamp":"2024-04-25T10:30:10.729165+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T15:17:28.997853+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"5.14","version":"0'0","reported_seq":9,"reported_epoch":35,"state":"active+clean","last_fresh":"2024-04-25T10:30:12.778630+0000","last_change":"2024-04-25T10:30:12.770588+0000","last_active":"2024-04-25T10:30:12.778630+0000","last_peered":"2024-04-25T10:30:12.778630+0000","last_clean":"2024-04-25T10:30:12.778630+0000","last_became_active":"2024-04-25T10:30:12.770264+0000","last_became_peered":"2024-04-25T10:30:12.770264+0000","last_unstale":"2024-04-25T10:30:12.778630+0000","last_undegraded":"2024-04-25T10:30:12.778630+0000","last_fullsized":"2024-04-25T10:30:12.778630+0000","mapping_epoch":34,"log_start":"0'0","ondisk_log_start":"0'0","created":34,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:30:11.733978+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:30:11.733978+0000","last_clean_scrub_stamp":"2024-04-25T10:30:11.733978+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T18:06:36.650571+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.15","version":"35'3","reported_seq":17,"reported_epoch":36,"state":"active+clean","last_fresh":"2024-04-25T10:30:13.760573+0000","last_change":"2024-04-25T10:30:11.755959+0000","last_active":"2024-04-25T10:30:13.760573+0000","last_peered":"2024-04-25T10:30:13.760573+0000","last_clean":"2024-04-25T10:30:13.760573+0000","last_became_active":"2024-04-25T10:30:11.755603+0000","last_became_peered":"2024-04-25T10:30:11.755603+0000","last_unstale":"2024-04-25T10:30:13.760573+0000","last_undegraded":"2024-04-25T10:30:13.760573+0000","last_fullsized":"2024-04-25T10:30:13.760573+0000","mapping_epoch":33,"log_start":"0'0","ondisk_log_start":"0'0","created":33,"last_epoch_clean":34,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:30:10.729165+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:30:10.729165+0000","last_clean_scrub_stamp":"2024-04-25T10:30:10.729165+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T19:25:05.393017+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.15","version":"0'0","reported_seq":12,"reported_epoch":36,"state":"active+clean","last_fresh":"2024-04-25T10:30:13.760709+0000","last_change":"2024-04-25T10:30:12.776432+0000","last_active":"2024-04-25T10:30:13.760709+0000","last_peered":"2024-04-25T10:30:13.760709+0000","last_clean":"2024-04-25T10:30:13.760709+0000","last_became_active":"2024-04-25T10:30:12.775857+0000","last_became_peered":"2024-04-25T10:30:12.775857+0000","last_unstale":"2024-04-25T10:30:13.760709+0000","last_undegraded":"2024-04-25T10:30:13.760709+0000","last_fullsized":"2024-04-25T10:30:13.760709+0000","mapping_epoch":34,"log_start":"0'0","ondisk_log_start":"0'0","created":34,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:30:11.733978+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:30:11.733978+0000","last_clean_scrub_stamp":"2024-04-25T10:30:11.733978+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T14:37:28.612068+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.14","version":"35'6","reported_seq":20,"reported_epoch":36,"state":"active+clean","last_fresh":"2024-04-25T10:30:13.762316+0000","last_change":"2024-04-25T10:30:11.757323+0000","last_active":"2024-04-25T10:30:13.762316+0000","last_peered":"2024-04-25T10:30:13.762316+0000","last_clean":"2024-04-25T10:30:13.762316+0000","last_became_active":"2024-04-25T10:30:11.757011+0000","last_became_peered":"2024-04-25T10:30:11.757011+0000","last_unstale":"2024-04-25T10:30:13.762316+0000","last_undegraded":"2024-04-25T10:30:13.762316+0000","last_fullsized":"2024-04-25T10:30:13.762316+0000","mapping_epoch":33,"log_start":"0'0","ondisk_log_start":"0'0","created":33,"last_epoch_clean":34,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:30:10.729165+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:30:10.729165+0000","last_clean_scrub_stamp":"2024-04-25T10:30:10.729165+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T19:02:53.584992+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1612,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"5.12","version":"0'0","reported_seq":12,"reported_epoch":36,"state":"active+clean","last_fresh":"2024-04-25T10:30:13.761928+0000","last_change":"2024-04-25T10:30:12.770496+0000","last_active":"2024-04-25T10:30:13.761928+0000","last_peered":"2024-04-25T10:30:13.761928+0000","last_clean":"2024-04-25T10:30:13.761928+0000","last_became_active":"2024-04-25T10:30:12.770193+0000","last_became_peered":"2024-04-25T10:30:12.770193+0000","last_unstale":"2024-04-25T10:30:13.761928+0000","last_undegraded":"2024-04-25T10:30:13.761928+0000","last_fullsized":"2024-04-25T10:30:13.761928+0000","mapping_epoch":34,"log_start":"0'0","ondisk_log_start":"0'0","created":34,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:30:11.733978+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:30:11.733978+0000","last_clean_scrub_stamp":"2024-04-25T10:30:11.733978+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T15:17:30.002665+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,1],"acting":[6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.13","version":"35'2","reported_seq":16,"reported_epoch":36,"state":"active+clean","last_fresh":"2024-04-25T10:30:13.762908+0000","last_change":"2024-04-25T10:30:11.764993+0000","last_active":"2024-04-25T10:30:13.762908+0000","last_peered":"2024-04-25T10:30:13.762908+0000","last_clean":"2024-04-25T10:30:13.762908+0000","last_became_active":"2024-04-25T10:30:11.764637+0000","last_became_peered":"2024-04-25T10:30:11.764637+0000","last_unstale":"2024-04-25T10:30:13.762908+0000","last_undegraded":"2024-04-25T10:30:13.762908+0000","last_fullsized":"2024-04-25T10:30:13.762908+0000","mapping_epoch":33,"log_start":"0'0","ondisk_log_start":"0'0","created":33,"last_epoch_clean":34,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:30:10.729165+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:30:10.729165+0000","last_clean_scrub_stamp":"2024-04-25T10:30:10.729165+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T17:24:13.848319+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":34,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":7,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"5.13","version":"0'0","reported_seq":9,"reported_epoch":35,"state":"active+clean","last_fresh":"2024-04-25T10:30:12.779318+0000","last_change":"2024-04-25T10:30:12.775269+0000","last_active":"2024-04-25T10:30:12.779318+0000","last_peered":"2024-04-25T10:30:12.779318+0000","last_clean":"2024-04-25T10:30:12.779318+0000","last_became_active":"2024-04-25T10:30:12.774865+0000","last_became_peered":"2024-04-25T10:30:12.774865+0000","last_unstale":"2024-04-25T10:30:12.779318+0000","last_undegraded":"2024-04-25T10:30:12.779318+0000","last_fullsized":"2024-04-25T10:30:12.779318+0000","mapping_epoch":34,"log_start":"0'0","ondisk_log_start":"0'0","created":34,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:30:11.733978+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:30:11.733978+0000","last_clean_scrub_stamp":"2024-04-25T10:30:11.733978+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T16:21:52.033922+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.12","version":"0'0","reported_seq":12,"reported_epoch":35,"state":"active+clean","last_fresh":"2024-04-25T10:30:12.770611+0000","last_change":"2024-04-25T10:30:11.762094+0000","last_active":"2024-04-25T10:30:12.770611+0000","last_peered":"2024-04-25T10:30:12.770611+0000","last_clean":"2024-04-25T10:30:12.770611+0000","last_became_active":"2024-04-25T10:30:11.761456+0000","last_became_peered":"2024-04-25T10:30:11.761456+0000","last_unstale":"2024-04-25T10:30:12.770611+0000","last_undegraded":"2024-04-25T10:30:12.770611+0000","last_fullsized":"2024-04-25T10:30:12.770611+0000","mapping_epoch":33,"log_start":"0'0","ondisk_log_start":"0'0","created":33,"last_epoch_clean":34,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:30:10.729165+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:30:10.729165+0000","last_clean_scrub_stamp":"2024-04-25T10:30:10.729165+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T21:58:31.211023+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.10","version":"0'0","reported_seq":12,"reported_epoch":36,"state":"active+clean","last_fresh":"2024-04-25T10:30:13.762551+0000","last_change":"2024-04-25T10:30:12.783285+0000","last_active":"2024-04-25T10:30:13.762551+0000","last_peered":"2024-04-25T10:30:13.762551+0000","last_clean":"2024-04-25T10:30:13.762551+0000","last_became_active":"2024-04-25T10:30:12.782617+0000","last_became_peered":"2024-04-25T10:30:12.782617+0000","last_unstale":"2024-04-25T10:30:13.762551+0000","last_undegraded":"2024-04-25T10:30:13.762551+0000","last_fullsized":"2024-04-25T10:30:13.762551+0000","mapping_epoch":34,"log_start":"0'0","ondisk_log_start":"0'0","created":34,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:30:11.733978+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:30:11.733978+0000","last_clean_scrub_stamp":"2024-04-25T10:30:11.733978+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T20:14:15.241563+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.11","version":"0'0","reported_seq":12,"reported_epoch":35,"state":"active+clean","last_fresh":"2024-04-25T10:30:12.768436+0000","last_change":"2024-04-25T10:30:11.755706+0000","last_active":"2024-04-25T10:30:12.768436+0000","last_peered":"2024-04-25T10:30:12.768436+0000","last_clean":"2024-04-25T10:30:12.768436+0000","last_became_active":"2024-04-25T10:30:11.755348+0000","last_became_peered":"2024-04-25T10:30:11.755348+0000","last_unstale":"2024-04-25T10:30:12.768436+0000","last_undegraded":"2024-04-25T10:30:12.768436+0000","last_fullsized":"2024-04-25T10:30:12.768436+0000","mapping_epoch":33,"log_start":"0'0","ondisk_log_start":"0'0","created":33,"last_epoch_clean":34,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:30:10.729165+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:30:10.729165+0000","last_clean_scrub_stamp":"2024-04-25T10:30:10.729165+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T10:42:20.657474+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.11","version":"0'0","reported_seq":12,"reported_epoch":36,"state":"active+clean","last_fresh":"2024-04-25T10:30:13.760319+0000","last_change":"2024-04-25T10:30:12.777184+0000","last_active":"2024-04-25T10:30:13.760319+0000","last_peered":"2024-04-25T10:30:13.760319+0000","last_clean":"2024-04-25T10:30:13.760319+0000","last_became_active":"2024-04-25T10:30:12.776874+0000","last_became_peered":"2024-04-25T10:30:12.776874+0000","last_unstale":"2024-04-25T10:30:13.760319+0000","last_undegraded":"2024-04-25T10:30:13.760319+0000","last_fullsized":"2024-04-25T10:30:13.760319+0000","mapping_epoch":34,"log_start":"0'0","ondisk_log_start":"0'0","created":34,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:30:11.733978+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:30:11.733978+0000","last_clean_scrub_stamp":"2024-04-25T10:30:11.733978+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T11:57:29.979255+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.10","version":"35'8","reported_seq":22,"reported_epoch":36,"state":"active+clean","last_fresh":"2024-04-25T10:30:14.411032+0000","last_change":"2024-04-25T10:30:11.755358+0000","last_active":"2024-04-25T10:30:14.411032+0000","last_peered":"2024-04-25T10:30:14.411032+0000","last_clean":"2024-04-25T10:30:14.411032+0000","last_became_active":"2024-04-25T10:30:11.754992+0000","last_became_peered":"2024-04-25T10:30:11.754992+0000","last_unstale":"2024-04-25T10:30:14.411032+0000","last_undegraded":"2024-04-25T10:30:14.411032+0000","last_fullsized":"2024-04-25T10:30:14.411032+0000","mapping_epoch":33,"log_start":"0'0","ondisk_log_start":"0'0","created":33,"last_epoch_clean":34,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:30:10.729165+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:30:10.729165+0000","last_clean_scrub_stamp":"2024-04-25T10:30:10.729165+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T19:57:30.397817+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":12,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3],"acting":[7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"5.e","version":"0'0","reported_seq":12,"reported_epoch":36,"state":"active+clean","last_fresh":"2024-04-25T10:30:13.760700+0000","last_change":"2024-04-25T10:30:12.780850+0000","last_active":"2024-04-25T10:30:13.760700+0000","last_peered":"2024-04-25T10:30:13.760700+0000","last_clean":"2024-04-25T10:30:13.760700+0000","last_became_active":"2024-04-25T10:30:12.780187+0000","last_became_peered":"2024-04-25T10:30:12.780187+0000","last_unstale":"2024-04-25T10:30:13.760700+0000","last_undegraded":"2024-04-25T10:30:13.760700+0000","last_fullsized":"2024-04-25T10:30:13.760700+0000","mapping_epoch":34,"log_start":"0'0","ondisk_log_start":"0'0","created":34,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:30:11.733978+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:30:11.733978+0000","last_clean_scrub_stamp":"2024-04-25T10:30:11.733978+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T14:04:42.520921+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.f","version":"35'1","reported_seq":15,"reported_epoch":36,"state":"active+clean","last_fresh":"2024-04-25T10:30:14.411495+0000","last_change":"2024-04-25T10:30:11.750348+0000","last_active":"2024-04-25T10:30:14.411495+0000","last_peered":"2024-04-25T10:30:14.411495+0000","last_clean":"2024-04-25T10:30:14.411495+0000","last_became_active":"2024-04-25T10:30:11.749970+0000","last_became_peered":"2024-04-25T10:30:11.749970+0000","last_unstale":"2024-04-25T10:30:14.411495+0000","last_undegraded":"2024-04-25T10:30:14.411495+0000","last_fullsized":"2024-04-25T10:30:14.411495+0000","mapping_epoch":33,"log_start":"0'0","ondisk_log_start":"0'0","created":33,"last_epoch_clean":34,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:30:10.729165+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:30:10.729165+0000","last_clean_scrub_stamp":"2024-04-25T10:30:10.729165+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T20:41:22.574408+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"5.f","version":"0'0","reported_seq":12,"reported_epoch":36,"state":"active+clean","last_fresh":"2024-04-25T10:30:14.411347+0000","last_change":"2024-04-25T10:30:12.769270+0000","last_active":"2024-04-25T10:30:14.411347+0000","last_peered":"2024-04-25T10:30:14.411347+0000","last_clean":"2024-04-25T10:30:14.411347+0000","last_became_active":"2024-04-25T10:30:12.768931+0000","last_became_peered":"2024-04-25T10:30:12.768931+0000","last_unstale":"2024-04-25T10:30:14.411347+0000","last_undegraded":"2024-04-25T10:30:14.411347+0000","last_fullsized":"2024-04-25T10:30:14.411347+0000","mapping_epoch":34,"log_start":"0'0","ondisk_log_start":"0'0","created":34,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:30:11.733978+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:30:11.733978+0000","last_clean_scrub_stamp":"2024-04-25T10:30:11.733978+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T19:02:54.589804+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.e","version":"35'2","reported_seq":16,"reported_epoch":36,"state":"active+clean","last_fresh":"2024-04-25T10:30:13.762774+0000","last_change":"2024-04-25T10:30:11.765106+0000","last_active":"2024-04-25T10:30:13.762774+0000","last_peered":"2024-04-25T10:30:13.762774+0000","last_clean":"2024-04-25T10:30:13.762774+0000","last_became_active":"2024-04-25T10:30:11.764710+0000","last_became_peered":"2024-04-25T10:30:11.764710+0000","last_unstale":"2024-04-25T10:30:13.762774+0000","last_undegraded":"2024-04-25T10:30:13.762774+0000","last_fullsized":"2024-04-25T10:30:13.762774+0000","mapping_epoch":33,"log_start":"0'0","ondisk_log_start":"0'0","created":33,"last_epoch_clean":34,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:30:10.729165+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:30:10.729165+0000","last_clean_scrub_stamp":"2024-04-25T10:30:10.729165+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T16:30:11.709416+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"5.c","version":"0'0","reported_seq":12,"reported_epoch":36,"state":"active+clean","last_fresh":"2024-04-25T10:30:14.410924+0000","last_change":"2024-04-25T10:30:12.765790+0000","last_active":"2024-04-25T10:30:14.410924+0000","last_peered":"2024-04-25T10:30:14.410924+0000","last_clean":"2024-04-25T10:30:14.410924+0000","last_became_active":"2024-04-25T10:30:12.765468+0000","last_became_peered":"2024-04-25T10:30:12.765468+0000","last_unstale":"2024-04-25T10:30:14.410924+0000","last_undegraded":"2024-04-25T10:30:14.410924+0000","last_fullsized":"2024-04-25T10:30:14.410924+0000","mapping_epoch":34,"log_start":"0'0","ondisk_log_start":"0'0","created":34,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:30:11.733978+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:30:11.733978+0000","last_clean_scrub_stamp":"2024-04-25T10:30:11.733978+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T17:41:34.910195+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.d","version":"35'1","reported_seq":15,"reported_epoch":36,"state":"active+clean","last_fresh":"2024-04-25T10:30:13.760826+0000","last_change":"2024-04-25T10:30:11.758390+0000","last_active":"2024-04-25T10:30:13.760826+0000","last_peered":"2024-04-25T10:30:13.760826+0000","last_clean":"2024-04-25T10:30:13.760826+0000","last_became_active":"2024-04-25T10:30:11.757989+0000","last_became_peered":"2024-04-25T10:30:11.757989+0000","last_unstale":"2024-04-25T10:30:13.760826+0000","last_undegraded":"2024-04-25T10:30:13.760826+0000","last_fullsized":"2024-04-25T10:30:13.760826+0000","mapping_epoch":33,"log_start":"0'0","ondisk_log_start":"0'0","created":33,"last_epoch_clean":34,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:30:10.729165+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:30:10.729165+0000","last_clean_scrub_stamp":"2024-04-25T10:30:10.729165+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T18:52:44.534946+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":70,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.d","version":"0'0","reported_seq":12,"reported_epoch":36,"state":"active+clean","last_fresh":"2024-04-25T10:30:14.411240+0000","last_change":"2024-04-25T10:30:12.771743+0000","last_active":"2024-04-25T10:30:14.411240+0000","last_peered":"2024-04-25T10:30:14.411240+0000","last_clean":"2024-04-25T10:30:14.411240+0000","last_became_active":"2024-04-25T10:30:12.771387+0000","last_became_peered":"2024-04-25T10:30:12.771387+0000","last_unstale":"2024-04-25T10:30:14.411240+0000","last_undegraded":"2024-04-25T10:30:14.411240+0000","last_fullsized":"2024-04-25T10:30:14.411240+0000","mapping_epoch":34,"log_start":"0'0","ondisk_log_start":"0'0","created":34,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:30:11.733978+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:30:11.733978+0000","last_clean_scrub_stamp":"2024-04-25T10:30:11.733978+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T19:02:54.589804+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.c","version":"0'0","reported_seq":14,"reported_epoch":36,"state":"active+clean","last_fresh":"2024-04-25T10:30:14.195574+0000","last_change":"2024-04-25T10:30:11.753067+0000","last_active":"2024-04-25T10:30:14.195574+0000","last_peered":"2024-04-25T10:30:14.195574+0000","last_clean":"2024-04-25T10:30:14.195574+0000","last_became_active":"2024-04-25T10:30:11.752595+0000","last_became_peered":"2024-04-25T10:30:11.752595+0000","last_unstale":"2024-04-25T10:30:14.195574+0000","last_undegraded":"2024-04-25T10:30:14.195574+0000","last_fullsized":"2024-04-25T10:30:14.195574+0000","mapping_epoch":33,"log_start":"0'0","ondisk_log_start":"0'0","created":33,"last_epoch_clean":34,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:30:10.729165+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:30:10.729165+0000","last_clean_scrub_stamp":"2024-04-25T10:30:10.729165+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T18:59:19.688948+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"5.a","version":"0'0","reported_seq":12,"reported_epoch":36,"state":"active+clean","last_fresh":"2024-04-25T10:30:13.761924+0000","last_change":"2024-04-25T10:30:12.762940+0000","last_active":"2024-04-25T10:30:13.761924+0000","last_peered":"2024-04-25T10:30:13.761924+0000","last_clean":"2024-04-25T10:30:13.761924+0000","last_became_active":"2024-04-25T10:30:12.762628+0000","last_became_peered":"2024-04-25T10:30:12.762628+0000","last_unstale":"2024-04-25T10:30:13.761924+0000","last_undegraded":"2024-04-25T10:30:13.761924+0000","last_fullsized":"2024-04-25T10:30:13.761924+0000","mapping_epoch":34,"log_start":"0'0","ondisk_log_start":"0'0","created":34,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:30:11.733978+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:30:11.733978+0000","last_clean_scrub_stamp":"2024-04-25T10:30:11.733978+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T14:01:06.791796+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.b","version":"35'2","reported_seq":16,"reported_epoch":36,"state":"active+clean","last_fresh":"2024-04-25T10:30:14.411713+0000","last_change":"2024-04-25T10:30:11.752902+0000","last_active":"2024-04-25T10:30:14.411713+0000","last_peered":"2024-04-25T10:30:14.411713+0000","last_clean":"2024-04-25T10:30:14.411713+0000","last_became_active":"2024-04-25T10:30:11.752599+0000","last_became_peered":"2024-04-25T10:30:11.752599+0000","last_unstale":"2024-04-25T10:30:14.411713+0000","last_undegraded":"2024-04-25T10:30:14.411713+0000","last_fullsized":"2024-04-25T10:30:14.411713+0000","mapping_epoch":33,"log_start":"0'0","ondisk_log_start":"0'0","created":33,"last_epoch_clean":34,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:30:10.729165+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:30:10.729165+0000","last_clean_scrub_stamp":"2024-04-25T10:30:10.729165+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T14:01:05.786983+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"5.b","version":"0'0","reported_seq":12,"reported_epoch":36,"state":"active+clean","last_fresh":"2024-04-25T10:30:14.411684+0000","last_change":"2024-04-25T10:30:12.766544+0000","last_active":"2024-04-25T10:30:14.411684+0000","last_peered":"2024-04-25T10:30:14.411684+0000","last_clean":"2024-04-25T10:30:14.411684+0000","last_became_active":"2024-04-25T10:30:12.766228+0000","last_became_peered":"2024-04-25T10:30:12.766228+0000","last_unstale":"2024-04-25T10:30:14.411684+0000","last_undegraded":"2024-04-25T10:30:14.411684+0000","last_fullsized":"2024-04-25T10:30:14.411684+0000","mapping_epoch":34,"log_start":"0'0","ondisk_log_start":"0'0","created":34,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:30:11.733978+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:30:11.733978+0000","last_clean_scrub_stamp":"2024-04-25T10:30:11.733978+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T18:21:32.178524+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.a","version":"0'0","reported_seq":14,"reported_epoch":36,"state":"active+clean","last_fresh":"2024-04-25T10:30:13.762599+0000","last_change":"2024-04-25T10:30:11.762745+0000","last_active":"2024-04-25T10:30:13.762599+0000","last_peered":"2024-04-25T10:30:13.762599+0000","last_clean":"2024-04-25T10:30:13.762599+0000","last_became_active":"2024-04-25T10:30:11.762377+0000","last_became_peered":"2024-04-25T10:30:11.762377+0000","last_unstale":"2024-04-25T10:30:13.762599+0000","last_undegraded":"2024-04-25T10:30:13.762599+0000","last_fullsized":"2024-04-25T10:30:13.762599+0000","mapping_epoch":33,"log_start":"0'0","ondisk_log_start":"0'0","created":33,"last_epoch_clean":34,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:30:10.729165+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:30:10.729165+0000","last_clean_scrub_stamp":"2024-04-25T10:30:10.729165+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T17:15:13.597075+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"5.8","version":"0'0","reported_seq":12,"reported_epoch":36,"state":"active+clean","last_fresh":"2024-04-25T10:30:13.761619+0000","last_change":"2024-04-25T10:30:12.777060+0000","last_active":"2024-04-25T10:30:13.761619+0000","last_peered":"2024-04-25T10:30:13.761619+0000","last_clean":"2024-04-25T10:30:13.761619+0000","last_became_active":"2024-04-25T10:30:12.776749+0000","last_became_peered":"2024-04-25T10:30:12.776749+0000","last_unstale":"2024-04-25T10:30:13.761619+0000","last_undegraded":"2024-04-25T10:30:13.761619+0000","last_fullsized":"2024-04-25T10:30:13.761619+0000","mapping_epoch":34,"log_start":"0'0","ondisk_log_start":"0'0","created":34,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:30:11.733978+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:30:11.733978+0000","last_clean_scrub_stamp":"2024-04-25T10:30:11.733978+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T18:31:49.260513+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.9","version":"0'0","reported_seq":14,"reported_epoch":36,"state":"active+clean","last_fresh":"2024-04-25T10:30:14.411012+0000","last_change":"2024-04-25T10:30:11.759807+0000","last_active":"2024-04-25T10:30:14.411012+0000","last_peered":"2024-04-25T10:30:14.411012+0000","last_clean":"2024-04-25T10:30:14.411012+0000","last_became_active":"2024-04-25T10:30:11.758558+0000","last_became_peered":"2024-04-25T10:30:11.758558+0000","last_unstale":"2024-04-25T10:30:14.411012+0000","last_undegraded":"2024-04-25T10:30:14.411012+0000","last_fullsized":"2024-04-25T10:30:14.411012+0000","mapping_epoch":33,"log_start":"0'0","ondisk_log_start":"0'0","created":33,"last_epoch_clean":34,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:30:10.729165+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:30:10.729165+0000","last_clean_scrub_stamp":"2024-04-25T10:30:10.729165+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T20:38:23.460836+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"5.9","version":"0'0","reported_seq":12,"reported_epoch":36,"state":"active+clean","last_fresh":"2024-04-25T10:30:14.411216+0000","last_change":"2024-04-25T10:30:12.768782+0000","last_active":"2024-04-25T10:30:14.411216+0000","last_peered":"2024-04-25T10:30:14.411216+0000","last_clean":"2024-04-25T10:30:14.411216+0000","last_became_active":"2024-04-25T10:30:12.768464+0000","last_became_peered":"2024-04-25T10:30:12.768464+0000","last_unstale":"2024-04-25T10:30:14.411216+0000","last_undegraded":"2024-04-25T10:30:14.411216+0000","last_fullsized":"2024-04-25T10:30:14.411216+0000","mapping_epoch":34,"log_start":"0'0","ondisk_log_start":"0'0","created":34,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:30:11.733978+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:30:11.733978+0000","last_clean_scrub_stamp":"2024-04-25T10:30:11.733978+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T17:15:14.601888+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.8","version":"0'0","reported_seq":14,"reported_epoch":36,"state":"active+clean","last_fresh":"2024-04-25T10:30:13.760291+0000","last_change":"2024-04-25T10:30:11.758581+0000","last_active":"2024-04-25T10:30:13.760291+0000","last_peered":"2024-04-25T10:30:13.760291+0000","last_clean":"2024-04-25T10:30:13.760291+0000","last_became_active":"2024-04-25T10:30:11.758239+0000","last_became_peered":"2024-04-25T10:30:11.758239+0000","last_unstale":"2024-04-25T10:30:13.760291+0000","last_undegraded":"2024-04-25T10:30:13.760291+0000","last_fullsized":"2024-04-25T10:30:13.760291+0000","mapping_epoch":33,"log_start":"0'0","ondisk_log_start":"0'0","created":33,"last_epoch_clean":34,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:30:10.729165+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:30:10.729165+0000","last_clean_scrub_stamp":"2024-04-25T10:30:10.729165+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T17:41:33.905382+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.6","version":"0'0","reported_seq":12,"reported_epoch":36,"state":"active+clean","last_fresh":"2024-04-25T10:30:14.411468+0000","last_change":"2024-04-25T10:30:12.772456+0000","last_active":"2024-04-25T10:30:14.411468+0000","last_peered":"2024-04-25T10:30:14.411468+0000","last_clean":"2024-04-25T10:30:14.411468+0000","last_became_active":"2024-04-25T10:30:12.772070+0000","last_became_peered":"2024-04-25T10:30:12.772070+0000","last_unstale":"2024-04-25T10:30:14.411468+0000","last_undegraded":"2024-04-25T10:30:14.411468+0000","last_fullsized":"2024-04-25T10:30:14.411468+0000","mapping_epoch":34,"log_start":"0'0","ondisk_log_start":"0'0","created":34,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:30:11.733978+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:30:11.733978+0000","last_clean_scrub_stamp":"2024-04-25T10:30:11.733978+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T16:09:46.519973+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.7","version":"35'2","reported_seq":16,"reported_epoch":36,"state":"active+clean","last_fresh":"2024-04-25T10:30:13.761834+0000","last_change":"2024-04-25T10:30:11.756948+0000","last_active":"2024-04-25T10:30:13.761834+0000","last_peered":"2024-04-25T10:30:13.761834+0000","last_clean":"2024-04-25T10:30:13.761834+0000","last_became_active":"2024-04-25T10:30:11.756607+0000","last_became_peered":"2024-04-25T10:30:11.756607+0000","last_unstale":"2024-04-25T10:30:13.761834+0000","last_undegraded":"2024-04-25T10:30:13.761834+0000","last_fullsized":"2024-04-25T10:30:13.761834+0000","mapping_epoch":33,"log_start":"0'0","ondisk_log_start":"0'0","created":33,"last_epoch_clean":34,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:30:10.729165+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:30:10.729165+0000","last_clean_scrub_stamp":"2024-04-25T10:30:10.729165+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T17:15:13.597075+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"5.7","version":"0'0","reported_seq":12,"reported_epoch":36,"state":"active+clean","last_fresh":"2024-04-25T10:30:13.760659+0000","last_change":"2024-04-25T10:30:12.768932+0000","last_active":"2024-04-25T10:30:13.760659+0000","last_peered":"2024-04-25T10:30:13.760659+0000","last_clean":"2024-04-25T10:30:13.760659+0000","last_became_active":"2024-04-25T10:30:12.768609+0000","last_became_peered":"2024-04-25T10:30:12.768609+0000","last_unstale":"2024-04-25T10:30:13.760659+0000","last_undegraded":"2024-04-25T10:30:13.760659+0000","last_fullsized":"2024-04-25T10:30:13.760659+0000","mapping_epoch":34,"log_start":"0'0","ondisk_log_start":"0'0","created":34,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:30:11.733978+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:30:11.733978+0000","last_clean_scrub_stamp":"2024-04-25T10:30:11.733978+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T18:35:51.085554+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.6","version":"36'1","reported_seq":15,"reported_epoch":36,"state":"active+clean","last_fresh":"2024-04-25T10:30:14.765515+0000","last_change":"2024-04-25T10:30:11.750312+0000","last_active":"2024-04-25T10:30:14.765515+0000","last_peered":"2024-04-25T10:30:14.765515+0000","last_clean":"2024-04-25T10:30:14.765515+0000","last_became_active":"2024-04-25T10:30:11.750024+0000","last_became_peered":"2024-04-25T10:30:11.750024+0000","last_unstale":"2024-04-25T10:30:14.765515+0000","last_undegraded":"2024-04-25T10:30:14.765515+0000","last_fullsized":"2024-04-25T10:30:14.765515+0000","mapping_epoch":33,"log_start":"0'0","ondisk_log_start":"0'0","created":33,"last_epoch_clean":34,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:30:10.729165+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:30:10.729165+0000","last_clean_scrub_stamp":"2024-04-25T10:30:10.729165+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T16:48:59.263610+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"5.5","version":"0'0","reported_seq":12,"reported_epoch":36,"state":"active+clean","last_fresh":"2024-04-25T10:30:14.411435+0000","last_change":"2024-04-25T10:30:12.782521+0000","last_active":"2024-04-25T10:30:14.411435+0000","last_peered":"2024-04-25T10:30:14.411435+0000","last_clean":"2024-04-25T10:30:14.411435+0000","last_became_active":"2024-04-25T10:30:12.781802+0000","last_became_peered":"2024-04-25T10:30:12.781802+0000","last_unstale":"2024-04-25T10:30:14.411435+0000","last_undegraded":"2024-04-25T10:30:14.411435+0000","last_fullsized":"2024-04-25T10:30:14.411435+0000","mapping_epoch":34,"log_start":"0'0","ondisk_log_start":"0'0","created":34,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:30:11.733978+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:30:11.733978+0000","last_clean_scrub_stamp":"2024-04-25T10:30:11.733978+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T13:03:09.680861+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.4","version":"0'0","reported_seq":14,"reported_epoch":36,"state":"active+clean","last_fresh":"2024-04-25T10:30:13.762316+0000","last_change":"2024-04-25T10:30:11.757601+0000","last_active":"2024-04-25T10:30:13.762316+0000","last_peered":"2024-04-25T10:30:13.762316+0000","last_clean":"2024-04-25T10:30:13.762316+0000","last_became_active":"2024-04-25T10:30:11.756626+0000","last_became_peered":"2024-04-25T10:30:11.756626+0000","last_unstale":"2024-04-25T10:30:13.762316+0000","last_undegraded":"2024-04-25T10:30:13.762316+0000","last_fullsized":"2024-04-25T10:30:13.762316+0000","mapping_epoch":33,"log_start":"0'0","ondisk_log_start":"0'0","created":33,"last_epoch_clean":34,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:30:10.729165+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:30:10.729165+0000","last_clean_scrub_stamp":"2024-04-25T10:30:10.729165+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T13:03:08.676049+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"5.2","version":"0'0","reported_seq":12,"reported_epoch":36,"state":"active+clean","last_fresh":"2024-04-25T10:30:14.195648+0000","last_change":"2024-04-25T10:30:12.765903+0000","last_active":"2024-04-25T10:30:14.195648+0000","last_peered":"2024-04-25T10:30:14.195648+0000","last_clean":"2024-04-25T10:30:14.195648+0000","last_became_active":"2024-04-25T10:30:12.765611+0000","last_became_peered":"2024-04-25T10:30:12.765611+0000","last_unstale":"2024-04-25T10:30:14.195648+0000","last_undegraded":"2024-04-25T10:30:14.195648+0000","last_fullsized":"2024-04-25T10:30:14.195648+0000","mapping_epoch":34,"log_start":"0'0","ondisk_log_start":"0'0","created":34,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:30:11.733978+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:30:11.733978+0000","last_clean_scrub_stamp":"2024-04-25T10:30:11.733978+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T18:52:45.539758+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.3","version":"0'0","reported_seq":14,"reported_epoch":36,"state":"active+clean","last_fresh":"2024-04-25T10:30:13.762155+0000","last_change":"2024-04-25T10:30:11.762286+0000","last_active":"2024-04-25T10:30:13.762155+0000","last_peered":"2024-04-25T10:30:13.762155+0000","last_clean":"2024-04-25T10:30:13.762155+0000","last_became_active":"2024-04-25T10:30:11.761975+0000","last_became_peered":"2024-04-25T10:30:11.761975+0000","last_unstale":"2024-04-25T10:30:13.762155+0000","last_undegraded":"2024-04-25T10:30:13.762155+0000","last_fullsized":"2024-04-25T10:30:13.762155+0000","mapping_epoch":33,"log_start":"0'0","ondisk_log_start":"0'0","created":33,"last_epoch_clean":34,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:30:10.729165+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:30:10.729165+0000","last_clean_scrub_stamp":"2024-04-25T10:30:10.729165+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T16:09:04.081750+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"5.3","version":"0'0","reported_seq":9,"reported_epoch":35,"state":"active+clean","last_fresh":"2024-04-25T10:30:12.779343+0000","last_change":"2024-04-25T10:30:12.775959+0000","last_active":"2024-04-25T10:30:12.779343+0000","last_peered":"2024-04-25T10:30:12.779343+0000","last_clean":"2024-04-25T10:30:12.779343+0000","last_became_active":"2024-04-25T10:30:12.774532+0000","last_became_peered":"2024-04-25T10:30:12.774532+0000","last_unstale":"2024-04-25T10:30:12.779343+0000","last_undegraded":"2024-04-25T10:30:12.779343+0000","last_fullsized":"2024-04-25T10:30:12.779343+0000","mapping_epoch":34,"log_start":"0'0","ondisk_log_start":"0'0","created":34,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:30:11.733978+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:30:11.733978+0000","last_clean_scrub_stamp":"2024-04-25T10:30:11.733978+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T16:18:26.764587+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.2","version":"0'0","reported_seq":14,"reported_epoch":36,"state":"active+clean","last_fresh":"2024-04-25T10:30:13.762069+0000","last_change":"2024-04-25T10:30:11.760769+0000","last_active":"2024-04-25T10:30:13.762069+0000","last_peered":"2024-04-25T10:30:13.762069+0000","last_clean":"2024-04-25T10:30:13.762069+0000","last_became_active":"2024-04-25T10:30:11.760425+0000","last_became_peered":"2024-04-25T10:30:11.760425+0000","last_unstale":"2024-04-25T10:30:13.762069+0000","last_undegraded":"2024-04-25T10:30:13.762069+0000","last_fullsized":"2024-04-25T10:30:13.762069+0000","mapping_epoch":33,"log_start":"0'0","ondisk_log_start":"0'0","created":33,"last_epoch_clean":34,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:30:10.729165+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:30:10.729165+0000","last_clean_scrub_stamp":"2024-04-25T10:30:10.729165+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T17:24:13.848319+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"5.0","version":"0'0","reported_seq":9,"reported_epoch":35,"state":"active+clean","last_fresh":"2024-04-25T10:30:12.787795+0000","last_change":"2024-04-25T10:30:12.781642+0000","last_active":"2024-04-25T10:30:12.787795+0000","last_peered":"2024-04-25T10:30:12.787795+0000","last_clean":"2024-04-25T10:30:12.787795+0000","last_became_active":"2024-04-25T10:30:12.781303+0000","last_became_peered":"2024-04-25T10:30:12.781303+0000","last_unstale":"2024-04-25T10:30:12.787795+0000","last_undegraded":"2024-04-25T10:30:12.787795+0000","last_fullsized":"2024-04-25T10:30:12.787795+0000","mapping_epoch":34,"log_start":"0'0","ondisk_log_start":"0'0","created":34,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:30:11.733978+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:30:11.733978+0000","last_clean_scrub_stamp":"2024-04-25T10:30:11.733978+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T22:06:18.432116+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.1","version":"0'0","reported_seq":14,"reported_epoch":36,"state":"active+clean","last_fresh":"2024-04-25T10:30:13.762656+0000","last_change":"2024-04-25T10:30:11.765156+0000","last_active":"2024-04-25T10:30:13.762656+0000","last_peered":"2024-04-25T10:30:13.762656+0000","last_clean":"2024-04-25T10:30:13.762656+0000","last_became_active":"2024-04-25T10:30:11.764704+0000","last_became_peered":"2024-04-25T10:30:11.764704+0000","last_unstale":"2024-04-25T10:30:13.762656+0000","last_undegraded":"2024-04-25T10:30:13.762656+0000","last_fullsized":"2024-04-25T10:30:13.762656+0000","mapping_epoch":33,"log_start":"0'0","ondisk_log_start":"0'0","created":33,"last_epoch_clean":34,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:30:10.729165+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:30:10.729165+0000","last_clean_scrub_stamp":"2024-04-25T10:30:10.729165+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T19:02:53.584992+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"5.1","version":"0'0","reported_seq":12,"reported_epoch":36,"state":"active+clean","last_fresh":"2024-04-25T10:30:13.760478+0000","last_change":"2024-04-25T10:30:12.769006+0000","last_active":"2024-04-25T10:30:13.760478+0000","last_peered":"2024-04-25T10:30:13.760478+0000","last_clean":"2024-04-25T10:30:13.760478+0000","last_became_active":"2024-04-25T10:30:12.768671+0000","last_became_peered":"2024-04-25T10:30:12.768671+0000","last_unstale":"2024-04-25T10:30:13.760478+0000","last_undegraded":"2024-04-25T10:30:13.760478+0000","last_fullsized":"2024-04-25T10:30:13.760478+0000","mapping_epoch":34,"log_start":"0'0","ondisk_log_start":"0'0","created":34,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:30:11.733978+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:30:11.733978+0000","last_clean_scrub_stamp":"2024-04-25T10:30:11.733978+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T15:39:31.890064+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.0","version":"0'0","reported_seq":12,"reported_epoch":35,"state":"active+clean","last_fresh":"2024-04-25T10:30:12.768988+0000","last_change":"2024-04-25T10:30:11.761764+0000","last_active":"2024-04-25T10:30:12.768988+0000","last_peered":"2024-04-25T10:30:12.768988+0000","last_clean":"2024-04-25T10:30:12.768988+0000","last_became_active":"2024-04-25T10:30:11.761281+0000","last_became_peered":"2024-04-25T10:30:11.761281+0000","last_unstale":"2024-04-25T10:30:12.768988+0000","last_undegraded":"2024-04-25T10:30:12.768988+0000","last_fullsized":"2024-04-25T10:30:12.768988+0000","mapping_epoch":33,"log_start":"0'0","ondisk_log_start":"0'0","created":33,"last_epoch_clean":34,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:30:10.729165+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:30:10.729165+0000","last_clean_scrub_stamp":"2024-04-25T10:30:10.729165+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T13:05:33.012421+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.4","version":"0'0","reported_seq":12,"reported_epoch":36,"state":"active+clean","last_fresh":"2024-04-25T10:30:14.195540+0000","last_change":"2024-04-25T10:30:12.773161+0000","last_active":"2024-04-25T10:30:14.195540+0000","last_peered":"2024-04-25T10:30:14.195540+0000","last_clean":"2024-04-25T10:30:14.195540+0000","last_became_active":"2024-04-25T10:30:12.772831+0000","last_became_peered":"2024-04-25T10:30:12.772831+0000","last_unstale":"2024-04-25T10:30:14.195540+0000","last_undegraded":"2024-04-25T10:30:14.195540+0000","last_fullsized":"2024-04-25T10:30:14.195540+0000","mapping_epoch":34,"log_start":"0'0","ondisk_log_start":"0'0","created":34,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:30:11.733978+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:30:11.733978+0000","last_clean_scrub_stamp":"2024-04-25T10:30:11.733978+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T19:25:06.397830+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.5","version":"0'0","reported_seq":14,"reported_epoch":36,"state":"active+clean","last_fresh":"2024-04-25T10:30:13.760838+0000","last_change":"2024-04-25T10:30:11.759459+0000","last_active":"2024-04-25T10:30:13.760838+0000","last_peered":"2024-04-25T10:30:13.760838+0000","last_clean":"2024-04-25T10:30:13.760838+0000","last_became_active":"2024-04-25T10:30:11.757880+0000","last_became_peered":"2024-04-25T10:30:11.757880+0000","last_unstale":"2024-04-25T10:30:13.760838+0000","last_undegraded":"2024-04-25T10:30:13.760838+0000","last_fullsized":"2024-04-25T10:30:13.760838+0000","mapping_epoch":33,"log_start":"0'0","ondisk_log_start":"0'0","created":33,"last_epoch_clean":34,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:30:10.729165+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:30:10.729165+0000","last_clean_scrub_stamp":"2024-04-25T10:30:10.729165+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T18:21:31.173712+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.0","version":"18'72","reported_seq":133,"reported_epoch":36,"state":"active+clean","last_fresh":"2024-04-25T10:30:14.410873+0000","last_change":"2024-04-25T10:28:57.972696+0000","last_active":"2024-04-25T10:30:14.410873+0000","last_peered":"2024-04-25T10:30:14.410873+0000","last_clean":"2024-04-25T10:30:14.410873+0000","last_became_active":"2024-04-25T10:28:57.972373+0000","last_became_peered":"2024-04-25T10:28:57.972373+0000","last_unstale":"2024-04-25T10:30:14.410873+0000","last_undegraded":"2024-04-25T10:30:14.410873+0000","last_fullsized":"2024-04-25T10:30:14.410873+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:28:56.959439+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:28:56.959439+0000","last_clean_scrub_stamp":"2024-04-25T10:28:56.959439+0000","objects_scrubbed":0,"log_size":72,"log_dups_size":0,"ondisk_log_size":72,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T17:34:34.057408+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":66,"num_read_kb":55,"num_write":101,"num_write_kb":1628,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]}],"pool_stats":[{"poolid":5,"num_pg":64,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":128,"acting":128,"num_store_stats":8},{"poolid":4,"num_pg":32,"stat_sum":{"num_bytes":2410,"num_objects":22,"num_object_clones":0,"num_object_copies":44,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":22,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":45,"num_write_kb":14,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":14,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":65536,"data_stored":4820,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":39,"ondisk_log_size":39,"up":64,"acting":64,"num_store_stats":8},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":66,"num_read_kb":55,"num_write":101,"num_write_kb":1628,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1187840,"data_stored":1180736,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":72,"ondisk_log_size":72,"up":2,"acting":2,"num_store_stats":2}],"osd_stats":[{"osd":7,"up_from":15,"seq":64424509459,"num_pgs":21,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27424,"kb_used_data":396,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344416,"statfs":{"total":96636764160,"available":96608681984,"internally_reserved":0,"allocated":405504,"data_stored":156445,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1590,"internal_metadata":27654602},"hb_peers":[0,1,2,3,4,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[]},{"osd":6,"up_from":15,"seq":64424509460,"num_pgs":25,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27432,"kb_used_data":404,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344408,"statfs":{"total":96636764160,"available":96608673792,"internally_reserved":0,"allocated":413696,"data_stored":158035,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":27654598},"hb_peers":[0,1,2,3,4,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[]},{"osd":5,"up_from":15,"seq":64424509461,"num_pgs":14,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":28004,"kb_used_data":976,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94343836,"statfs":{"total":96636764160,"available":96608088064,"internally_reserved":0,"allocated":999424,"data_stored":747373,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]},{"osd":3,"up_from":15,"seq":64424509461,"num_pgs":25,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27436,"kb_used_data":408,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344404,"statfs":{"total":96636764160,"available":96608669696,"internally_reserved":0,"allocated":417792,"data_stored":157199,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1593,"internal_metadata":27654599},"hb_peers":[0,1,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[]},{"osd":2,"up_from":15,"seq":64424509461,"num_pgs":18,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27424,"kb_used_data":396,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344416,"statfs":{"total":96636764160,"available":96608681984,"internally_reserved":0,"allocated":405504,"data_stored":156457,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":5,"apply_latency_ms":5,"commit_latency_ns":5000000,"apply_latency_ns":5000000},"alerts":[]},{"osd":1,"up_from":15,"seq":64424509459,"num_pgs":2,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":28004,"kb_used_data":976,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94343836,"statfs":{"total":96636764160,"available":96608088064,"internally_reserved":0,"allocated":999424,"data_stored":746861,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":5,"apply_latency_ms":5,"commit_latency_ns":5000000,"apply_latency_ns":5000000},"alerts":[]},{"osd":0,"up_from":15,"seq":64424509460,"num_pgs":21,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27424,"kb_used_data":396,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344416,"statfs":{"total":96636764160,"available":96608681984,"internally_reserved":0,"allocated":405504,"data_stored":156513,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[1,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[]},{"osd":4,"up_from":13,"seq":55834574867,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27424,"kb_used_data":396,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344416,"statfs":{"total":96636764160,"available":96608681984,"internally_reserved":0,"allocated":405504,"data_stored":151757,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":90,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":70,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":34,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":16384,"data_stored":776,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":16384,"data_stored":1634,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":582,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":1612,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":22,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-04-25T10:30:16.831 DEBUG:teuthology.orchestra.run.smithi037:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-04-25T10:30:16.977 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:16.975+0000 7f875ce70640 1 Processor -- start 2024-04-25T10:30:16.977 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:16.975+0000 7f875ce70640 1 -- start start 2024-04-25T10:30:16.979 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:16.976+0000 7f875ce70640 1 --2- >> v2:172.21.15.37:3300/0 conn(0x7f87580b7600 0x7f87580b7a10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:30:16.979 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:16.976+0000 7f875ce70640 1 --2- >> v2:172.21.15.203:3300/0 conn(0x7f87580d4560 0x7f87580d4970 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:30:16.980 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:16.976+0000 7f875ce70640 1 -- --> v1:172.21.15.37:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f87580b7f50 con 0x7f87580b4bb0 2024-04-25T10:30:16.980 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:16.976+0000 7f875ce70640 1 -- --> v2:172.21.15.37:3300/0 -- mon_getmap magic: 0 -- 0x7f8758161190 con 0x7f87580b7600 2024-04-25T10:30:16.980 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:16.976+0000 7f875ce70640 1 -- --> v2:172.21.15.203:3300/0 -- mon_getmap magic: 0 -- 0x7f8758161470 con 0x7f87580d4560 2024-04-25T10:30:16.980 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:16.976+0000 7f874ffff640 1 --2- >> v2:172.21.15.37:3300/0 conn(0x7f87580b7600 0x7f87580b7a10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:30:16.980 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:16.976+0000 7f875659c640 1 --1- >> v1:172.21.15.37:6789/0 conn(0x7f87580b4bb0 0x7f87581646b0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.37:6789/0 says I am v1:172.21.15.37:41088/0 (socket says 172.21.15.37:41088) 2024-04-25T10:30:16.980 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:16.976+0000 7f875659c640 1 -- 172.21.15.37:0/2345516632 learned_addr learned my addr 172.21.15.37:0/2345516632 (peer_addr_for_me v1:172.21.15.37:0/0) 2024-04-25T10:30:16.980 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:16.977+0000 7f8756d9d640 1 --2- 172.21.15.37:0/2345516632 >> v2:172.21.15.203:3300/0 conn(0x7f87580d4560 0x7f87580d4970 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:30:16.980 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:16.977+0000 7f874ffff640 1 -- 172.21.15.37:0/2345516632 >> v2:172.21.15.203:3300/0 conn(0x7f87580d4560 msgr2=0x7f87580d4970 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:30:16.980 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:16.977+0000 7f874ffff640 1 --2- 172.21.15.37:0/2345516632 >> v2:172.21.15.203:3300/0 conn(0x7f87580d4560 0x7f87580d4970 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:30:16.980 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:16.977+0000 7f874ffff640 1 -- 172.21.15.37:0/2345516632 >> v1:172.21.15.37:6789/0 conn(0x7f87580b4bb0 legacy=0x7f87581646b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:30:16.980 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:16.977+0000 7f8755d9b640 1 -- 172.21.15.37:0/2345516632 <== mon.0 v1:172.21.15.37:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 3412322088 0 0) 0x7f8748001110 con 0x7f87580b4bb0 2024-04-25T10:30:16.980 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:16.977+0000 7f874ffff640 1 -- 172.21.15.37:0/2345516632 --> v2:172.21.15.37:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f8758163eb0 con 0x7f87580b7600 2024-04-25T10:30:16.980 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:16.977+0000 7f8756d9d640 1 --2- 172.21.15.37:0/2345516632 >> v2:172.21.15.203:3300/0 conn(0x7f87580d4560 0x7f87580d4970 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T10:30:16.980 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:16.977+0000 7f874ffff640 1 --2- 172.21.15.37:0/2345516632 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f87580b7600 0x7f87580b7a10 secure :-1 s=READY pgs=437 cs=0 l=1 rev1=1 crypto rx=0x7f87580d3b00 tx=0x7f874002a1a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=cb47ea460faeeb3d server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:30:16.981 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:16.977+0000 7f8755d9b640 1 -- 172.21.15.37:0/2345516632 <== mon.0 v2:172.21.15.37:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8740037070 con 0x7f87580b7600 2024-04-25T10:30:16.981 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:16.978+0000 7f8755d9b640 1 -- 172.21.15.37:0/2345516632 <== mon.0 v2:172.21.15.37:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f8740032440 con 0x7f87580b7600 2024-04-25T10:30:16.981 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:16.978+0000 7f8755d9b640 1 -- 172.21.15.37:0/2345516632 <== mon.0 v2:172.21.15.37:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8740032810 con 0x7f87580b7600 2024-04-25T10:30:16.981 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:16.978+0000 7f875ce70640 1 -- 172.21.15.37:0/2345516632 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f87580b7600 msgr2=0x7f87580b7a10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:30:16.981 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:16.978+0000 7f875ce70640 1 --2- 172.21.15.37:0/2345516632 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f87580b7600 0x7f87580b7a10 secure :-1 s=READY pgs=437 cs=0 l=1 rev1=1 crypto rx=0x7f87580d3b00 tx=0x7f874002a1a0 comp rx=0 tx=0).stop 2024-04-25T10:30:16.982 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:16.978+0000 7f875ce70640 1 -- 172.21.15.37:0/2345516632 shutdown_connections 2024-04-25T10:30:16.982 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:16.978+0000 7f875ce70640 1 --2- 172.21.15.37:0/2345516632 >> v2:172.21.15.203:3300/0 conn(0x7f87580d4560 0x7f87580d4970 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:30:16.982 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:16.978+0000 7f875ce70640 1 --2- 172.21.15.37:0/2345516632 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f87580b7600 0x7f87580b7a10 unknown :-1 s=CLOSED pgs=437 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:30:16.982 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:16.978+0000 7f875ce70640 1 -- 172.21.15.37:0/2345516632 >> 172.21.15.37:0/2345516632 conn(0x7f87580d27e0 msgr2=0x7f87580d2bf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:30:16.982 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:16.978+0000 7f875ce70640 1 -- 172.21.15.37:0/2345516632 shutdown_connections 2024-04-25T10:30:16.982 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:16.978+0000 7f875ce70640 1 -- 172.21.15.37:0/2345516632 wait complete. 2024-04-25T10:30:16.982 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:16.980+0000 7f875ce70640 1 Processor -- start 2024-04-25T10:30:16.983 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:16.980+0000 7f875ce70640 1 -- start start 2024-04-25T10:30:16.985 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:16.981+0000 7f875ce70640 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f87580b4bb0 0x7f87581e9b00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:30:16.985 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:16.981+0000 7f875ce70640 1 --2- >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f87580b7600 0x7f8758156340 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:30:16.985 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:16.981+0000 7f875ce70640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f87580d4560 0x7f875815f8d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:30:16.985 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:16.982+0000 7f875ce70640 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 -- 0x7f87580d09e0 con 0x7f87580d4560 2024-04-25T10:30:16.985 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:16.982+0000 7f875ce70640 1 -- --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_getmap magic: 0 -- 0x7f87580b4fc0 con 0x7f87580b4bb0 2024-04-25T10:30:16.985 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:16.982+0000 7f875ce70640 1 -- --> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] -- mon_getmap magic: 0 -- 0x7f87580d0b60 con 0x7f87580b7600 2024-04-25T10:30:16.985 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:16.982+0000 7f8756d9d640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f87580d4560 0x7f875815f8d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:30:16.985 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:16.982+0000 7f8756d9d640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f87580d4560 0x7f875815f8d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.37:3300/0 says I am v2:172.21.15.37:52730/0 (socket says 172.21.15.37:52730) 2024-04-25T10:30:16.985 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:16.982+0000 7f8756d9d640 1 -- 172.21.15.37:0/4133816987 learned_addr learned my addr 172.21.15.37:0/4133816987 (peer_addr_for_me v2:172.21.15.37:0/0) 2024-04-25T10:30:16.985 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:16.982+0000 7f875659c640 1 --2- 172.21.15.37:0/4133816987 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f87580b4bb0 0x7f87581e9b00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:30:16.985 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:16.982+0000 7f874ffff640 1 --2- 172.21.15.37:0/4133816987 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f87580b7600 0x7f8758156340 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:30:16.986 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:16.982+0000 7f8756d9d640 1 -- 172.21.15.37:0/4133816987 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f87580b7600 msgr2=0x7f8758156340 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:30:16.986 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:16.982+0000 7f8756d9d640 1 --2- 172.21.15.37:0/4133816987 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f87580b7600 0x7f8758156340 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:30:16.986 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:16.982+0000 7f8756d9d640 1 -- 172.21.15.37:0/4133816987 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f87580b4bb0 msgr2=0x7f87581e9b00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:30:16.986 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:16.982+0000 7f8756d9d640 1 --2- 172.21.15.37:0/4133816987 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f87580b4bb0 0x7f87581e9b00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:30:16.986 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:16.982+0000 7f8756d9d640 1 -- 172.21.15.37:0/4133816987 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f87581ea390 con 0x7f87580d4560 2024-04-25T10:30:16.986 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:16.982+0000 7f8756d9d640 1 --2- 172.21.15.37:0/4133816987 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f87580d4560 0x7f875815f8d0 secure :-1 s=READY pgs=438 cs=0 l=1 rev1=1 crypto rx=0x7f875000cdb0 tx=0x7f875000a650 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:30:16.986 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:16.982+0000 7f874effd640 1 -- 172.21.15.37:0/4133816987 <== mon.0 v2:172.21.15.37:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8750003b90 con 0x7f87580d4560 2024-04-25T10:30:16.986 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:16.982+0000 7f874effd640 1 -- 172.21.15.37:0/4133816987 <== mon.0 v2:172.21.15.37:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f8750003d50 con 0x7f87580d4560 2024-04-25T10:30:16.986 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:16.982+0000 7f875ce70640 1 -- 172.21.15.37:0/4133816987 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f87581ea670 con 0x7f87580d4560 2024-04-25T10:30:16.986 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:16.982+0000 7f874effd640 1 -- 172.21.15.37:0/4133816987 <== mon.0 v2:172.21.15.37:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8750004260 con 0x7f87580d4560 2024-04-25T10:30:16.986 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:16.982+0000 7f875ce70640 1 -- 172.21.15.37:0/4133816987 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f87580cecb0 con 0x7f87580d4560 2024-04-25T10:30:16.987 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:16.983+0000 7f875ce70640 1 -- 172.21.15.37:0/4133816987 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f87580cead0 con 0x7f87580d4560 2024-04-25T10:30:16.987 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:16.984+0000 7f874effd640 1 -- 172.21.15.37:0/4133816987 <== mon.0 v2:172.21.15.37:3300/0 4 ==== mgrmap(e 5) ==== 96782+0+0 (secure 0 0 0) 0x7f8750004400 con 0x7f87580d4560 2024-04-25T10:30:16.988 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:16.985+0000 7f874effd640 1 --2- 172.21.15.37:0/4133816987 >> [v2:172.21.15.203:6832/2418205237,v1:172.21.15.203:6833/2418205237] conn(0x7f87380744b0 0x7f87380769c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:30:16.988 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:16.985+0000 7f874effd640 1 -- 172.21.15.37:0/4133816987 <== mon.0 v2:172.21.15.37:3300/0 5 ==== osd_map(36..36 src has 1..36) ==== 5709+0+0 (secure 0 0 0) 0x7f8750095c90 con 0x7f87580d4560 2024-04-25T10:30:16.988 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:16.985+0000 7f875659c640 1 --2- 172.21.15.37:0/4133816987 >> [v2:172.21.15.203:6832/2418205237,v1:172.21.15.203:6833/2418205237] conn(0x7f87380744b0 0x7f87380769c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:30:16.988 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:16.985+0000 7f875659c640 1 --2- 172.21.15.37:0/4133816987 >> [v2:172.21.15.203:6832/2418205237,v1:172.21.15.203:6833/2418205237] conn(0x7f87380744b0 0x7f87380769c0 secure :-1 s=READY pgs=219 cs=0 l=1 rev1=1 crypto rx=0x7f8748000f60 tx=0x7f8748007c00 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:30:16.991 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:16.989+0000 7f874effd640 1 -- 172.21.15.37:0/4133816987 <== mon.0 v2:172.21.15.37:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187246 (secure 0 0 0) 0x7f87580cead0 con 0x7f87580d4560 2024-04-25T10:30:17.172 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:17.170+0000 7f875ce70640 1 -- 172.21.15.37:0/4133816987 --> [v2:172.21.15.203:6832/2418205237,v1:172.21.15.203:6833/2418205237] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) -- 0x7f87581ea120 con 0x7f87380744b0 2024-04-25T10:30:17.179 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:17.177+0000 7f874effd640 1 -- 172.21.15.37:0/4133816987 <== mgr.4100 v2:172.21.15.203:6832/2418205237 1 ==== mgr_command_reply(tid 0: 0 dumped all) ==== 18+0+251930 (secure 0 0 0) 0x7f87581ea120 con 0x7f87380744b0 2024-04-25T10:30:17.182 INFO:teuthology.orchestra.run.smithi037.stdout: 2024-04-25T10:30:17.183 INFO:teuthology.orchestra.run.smithi037.stderr:dumped all 2024-04-25T10:30:17.184 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:17.180+0000 7f875ce70640 1 -- 172.21.15.37:0/4133816987 >> [v2:172.21.15.203:6832/2418205237,v1:172.21.15.203:6833/2418205237] conn(0x7f87380744b0 msgr2=0x7f87380769c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:30:17.185 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:17.180+0000 7f875ce70640 1 --2- 172.21.15.37:0/4133816987 >> [v2:172.21.15.203:6832/2418205237,v1:172.21.15.203:6833/2418205237] conn(0x7f87380744b0 0x7f87380769c0 secure :-1 s=READY pgs=219 cs=0 l=1 rev1=1 crypto rx=0x7f8748000f60 tx=0x7f8748007c00 comp rx=0 tx=0).stop 2024-04-25T10:30:17.185 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:17.180+0000 7f875ce70640 1 -- 172.21.15.37:0/4133816987 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f87580d4560 msgr2=0x7f875815f8d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:30:17.185 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:17.180+0000 7f875ce70640 1 --2- 172.21.15.37:0/4133816987 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f87580d4560 0x7f875815f8d0 secure :-1 s=READY pgs=438 cs=0 l=1 rev1=1 crypto rx=0x7f875000cdb0 tx=0x7f875000a650 comp rx=0 tx=0).stop 2024-04-25T10:30:17.185 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:17.180+0000 7f875ce70640 1 -- 172.21.15.37:0/4133816987 shutdown_connections 2024-04-25T10:30:17.185 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:17.180+0000 7f875ce70640 1 --2- 172.21.15.37:0/4133816987 >> [v2:172.21.15.203:6832/2418205237,v1:172.21.15.203:6833/2418205237] conn(0x7f87380744b0 0x7f87380769c0 unknown :-1 s=CLOSED pgs=219 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:30:17.185 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:17.180+0000 7f875ce70640 1 --2- 172.21.15.37:0/4133816987 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f87580d4560 0x7f875815f8d0 unknown :-1 s=CLOSED pgs=438 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:30:17.185 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:17.180+0000 7f875ce70640 1 --2- 172.21.15.37:0/4133816987 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f87580b7600 0x7f8758156340 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:30:17.185 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:17.180+0000 7f875ce70640 1 --2- 172.21.15.37:0/4133816987 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f87580b4bb0 0x7f87581e9b00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:30:17.185 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:17.180+0000 7f875ce70640 1 -- 172.21.15.37:0/4133816987 >> 172.21.15.37:0/4133816987 conn(0x7f87580d27e0 msgr2=0x7f87580d4970 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:30:17.185 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:17.180+0000 7f875ce70640 1 -- 172.21.15.37:0/4133816987 shutdown_connections 2024-04-25T10:30:17.186 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:17.180+0000 7f875ce70640 1 -- 172.21.15.37:0/4133816987 wait complete. 2024-04-25T10:30:17.193 INFO:teuthology.orchestra.run.smithi037.stdout:{"pg_ready":true,"pg_map":{"version":71,"stamp":"2024-04-25T10:30:16.203949+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":592778,"num_objects":24,"num_object_clones":0,"num_object_copies":48,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":24,"num_whiteouts":0,"num_read":66,"num_read_kb":55,"num_write":146,"num_write_kb":1642,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":14,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":111,"ondisk_log_size":111,"up":194,"acting":194,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":126,"num_osds":8,"num_per_pool_osds":8,"num_per_pool_omap_osds":8,"kb":754974720,"kb_used":220572,"kb_used_data":4348,"kb_used_omap":12,"kb_used_meta":216051,"kb_avail":754754148,"statfs":{"total":773094113280,"available":772868247552,"internally_reserved":0,"allocated":4452352,"data_stored":2430640,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":12752,"internal_metadata":221236784},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":30,"apply_latency_ms":30,"commit_latency_ns":30000000,"apply_latency_ns":30000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":2410,"num_objects":22,"num_object_clones":0,"num_object_copies":44,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":22,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":45,"num_write_kb":14,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":14,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"5.464383"},"pg_stats":[{"pgid":"5.3f","version":"0'0","reported_seq":12,"reported_epoch":36,"state":"active+clean","last_fresh":"2024-04-25T10:30:13.762926+0000","last_change":"2024-04-25T10:30:12.772509+0000","last_active":"2024-04-25T10:30:13.762926+0000","last_peered":"2024-04-25T10:30:13.762926+0000","last_clean":"2024-04-25T10:30:13.762926+0000","last_became_active":"2024-04-25T10:30:12.772018+0000","last_became_peered":"2024-04-25T10:30:12.772018+0000","last_unstale":"2024-04-25T10:30:13.762926+0000","last_undegraded":"2024-04-25T10:30:13.762926+0000","last_fullsized":"2024-04-25T10:30:13.762926+0000","mapping_epoch":34,"log_start":"0'0","ondisk_log_start":"0'0","created":34,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:30:11.733978+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:30:11.733978+0000","last_clean_scrub_stamp":"2024-04-25T10:30:11.733978+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T19:25:06.397830+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"5.3e","version":"0'0","reported_seq":12,"reported_epoch":36,"state":"active+clean","last_fresh":"2024-04-25T10:30:14.411090+0000","last_change":"2024-04-25T10:30:12.780210+0000","last_active":"2024-04-25T10:30:14.411090+0000","last_peered":"2024-04-25T10:30:14.411090+0000","last_clean":"2024-04-25T10:30:14.411090+0000","last_became_active":"2024-04-25T10:30:12.779867+0000","last_became_peered":"2024-04-25T10:30:12.779867+0000","last_unstale":"2024-04-25T10:30:14.411090+0000","last_undegraded":"2024-04-25T10:30:14.411090+0000","last_fullsized":"2024-04-25T10:30:14.411090+0000","mapping_epoch":34,"log_start":"0'0","ondisk_log_start":"0'0","created":34,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:30:11.733978+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:30:11.733978+0000","last_clean_scrub_stamp":"2024-04-25T10:30:11.733978+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T12:27:54.564692+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"5.3d","version":"0'0","reported_seq":12,"reported_epoch":36,"state":"active+clean","last_fresh":"2024-04-25T10:30:14.411226+0000","last_change":"2024-04-25T10:30:12.768292+0000","last_active":"2024-04-25T10:30:14.411226+0000","last_peered":"2024-04-25T10:30:14.411226+0000","last_clean":"2024-04-25T10:30:14.411226+0000","last_became_active":"2024-04-25T10:30:12.767909+0000","last_became_peered":"2024-04-25T10:30:12.767909+0000","last_unstale":"2024-04-25T10:30:14.411226+0000","last_undegraded":"2024-04-25T10:30:14.411226+0000","last_fullsized":"2024-04-25T10:30:14.411226+0000","mapping_epoch":34,"log_start":"0'0","ondisk_log_start":"0'0","created":34,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:30:11.733978+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:30:11.733978+0000","last_clean_scrub_stamp":"2024-04-25T10:30:11.733978+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T16:09:05.086563+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"5.3c","version":"0'0","reported_seq":9,"reported_epoch":35,"state":"active+clean","last_fresh":"2024-04-25T10:30:12.785547+0000","last_change":"2024-04-25T10:30:12.781723+0000","last_active":"2024-04-25T10:30:12.785547+0000","last_peered":"2024-04-25T10:30:12.785547+0000","last_clean":"2024-04-25T10:30:12.785547+0000","last_became_active":"2024-04-25T10:30:12.781357+0000","last_became_peered":"2024-04-25T10:30:12.781357+0000","last_unstale":"2024-04-25T10:30:12.785547+0000","last_undegraded":"2024-04-25T10:30:12.785547+0000","last_fullsized":"2024-04-25T10:30:12.785547+0000","mapping_epoch":34,"log_start":"0'0","ondisk_log_start":"0'0","created":34,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:30:11.733978+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:30:11.733978+0000","last_clean_scrub_stamp":"2024-04-25T10:30:11.733978+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T16:17:39.250622+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.3b","version":"0'0","reported_seq":12,"reported_epoch":36,"state":"active+clean","last_fresh":"2024-04-25T10:30:14.411497+0000","last_change":"2024-04-25T10:30:12.762141+0000","last_active":"2024-04-25T10:30:14.411497+0000","last_peered":"2024-04-25T10:30:14.411497+0000","last_clean":"2024-04-25T10:30:14.411497+0000","last_became_active":"2024-04-25T10:30:12.761805+0000","last_became_peered":"2024-04-25T10:30:12.761805+0000","last_unstale":"2024-04-25T10:30:14.411497+0000","last_undegraded":"2024-04-25T10:30:14.411497+0000","last_fullsized":"2024-04-25T10:30:14.411497+0000","mapping_epoch":34,"log_start":"0'0","ondisk_log_start":"0'0","created":34,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:30:11.733978+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:30:11.733978+0000","last_clean_scrub_stamp":"2024-04-25T10:30:11.733978+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T18:52:45.539758+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6],"acting":[5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"5.3a","version":"0'0","reported_seq":12,"reported_epoch":36,"state":"active+clean","last_fresh":"2024-04-25T10:30:14.411756+0000","last_change":"2024-04-25T10:30:12.775534+0000","last_active":"2024-04-25T10:30:14.411756+0000","last_peered":"2024-04-25T10:30:14.411756+0000","last_clean":"2024-04-25T10:30:14.411756+0000","last_became_active":"2024-04-25T10:30:12.775180+0000","last_became_peered":"2024-04-25T10:30:12.775180+0000","last_unstale":"2024-04-25T10:30:14.411756+0000","last_undegraded":"2024-04-25T10:30:14.411756+0000","last_fullsized":"2024-04-25T10:30:14.411756+0000","mapping_epoch":34,"log_start":"0'0","ondisk_log_start":"0'0","created":34,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:30:11.733978+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:30:11.733978+0000","last_clean_scrub_stamp":"2024-04-25T10:30:11.733978+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T17:36:19.165820+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"5.39","version":"0'0","reported_seq":12,"reported_epoch":36,"state":"active+clean","last_fresh":"2024-04-25T10:30:13.762763+0000","last_change":"2024-04-25T10:30:12.781210+0000","last_active":"2024-04-25T10:30:13.762763+0000","last_peered":"2024-04-25T10:30:13.762763+0000","last_clean":"2024-04-25T10:30:13.762763+0000","last_became_active":"2024-04-25T10:30:12.780874+0000","last_became_peered":"2024-04-25T10:30:12.780874+0000","last_unstale":"2024-04-25T10:30:13.762763+0000","last_undegraded":"2024-04-25T10:30:13.762763+0000","last_fullsized":"2024-04-25T10:30:13.762763+0000","mapping_epoch":34,"log_start":"0'0","ondisk_log_start":"0'0","created":34,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:30:11.733978+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:30:11.733978+0000","last_clean_scrub_stamp":"2024-04-25T10:30:11.733978+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T14:16:42.719469+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"5.38","version":"0'0","reported_seq":12,"reported_epoch":36,"state":"active+clean","last_fresh":"2024-04-25T10:30:13.762365+0000","last_change":"2024-04-25T10:30:12.774614+0000","last_active":"2024-04-25T10:30:13.762365+0000","last_peered":"2024-04-25T10:30:13.762365+0000","last_clean":"2024-04-25T10:30:13.762365+0000","last_became_active":"2024-04-25T10:30:12.774213+0000","last_became_peered":"2024-04-25T10:30:12.774213+0000","last_unstale":"2024-04-25T10:30:13.762365+0000","last_undegraded":"2024-04-25T10:30:13.762365+0000","last_fullsized":"2024-04-25T10:30:13.762365+0000","mapping_epoch":34,"log_start":"0'0","ondisk_log_start":"0'0","created":34,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:30:11.733978+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:30:11.733978+0000","last_clean_scrub_stamp":"2024-04-25T10:30:11.733978+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T15:17:30.002665+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,6],"acting":[2,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"5.37","version":"0'0","reported_seq":12,"reported_epoch":36,"state":"active+clean","last_fresh":"2024-04-25T10:30:14.411269+0000","last_change":"2024-04-25T10:30:12.773184+0000","last_active":"2024-04-25T10:30:14.411269+0000","last_peered":"2024-04-25T10:30:14.411269+0000","last_clean":"2024-04-25T10:30:14.411269+0000","last_became_active":"2024-04-25T10:30:12.772866+0000","last_became_peered":"2024-04-25T10:30:12.772866+0000","last_unstale":"2024-04-25T10:30:14.411269+0000","last_undegraded":"2024-04-25T10:30:14.411269+0000","last_fullsized":"2024-04-25T10:30:14.411269+0000","mapping_epoch":34,"log_start":"0'0","ondisk_log_start":"0'0","created":34,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:30:11.733978+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:30:11.733978+0000","last_clean_scrub_stamp":"2024-04-25T10:30:11.733978+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T17:24:14.853132+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"5.36","version":"0'0","reported_seq":9,"reported_epoch":35,"state":"active+clean","last_fresh":"2024-04-25T10:30:12.778693+0000","last_change":"2024-04-25T10:30:12.773138+0000","last_active":"2024-04-25T10:30:12.778693+0000","last_peered":"2024-04-25T10:30:12.778693+0000","last_clean":"2024-04-25T10:30:12.778693+0000","last_became_active":"2024-04-25T10:30:12.772854+0000","last_became_peered":"2024-04-25T10:30:12.772854+0000","last_unstale":"2024-04-25T10:30:12.778693+0000","last_undegraded":"2024-04-25T10:30:12.778693+0000","last_fullsized":"2024-04-25T10:30:12.778693+0000","mapping_epoch":34,"log_start":"0'0","ondisk_log_start":"0'0","created":34,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:30:11.733978+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:30:11.733978+0000","last_clean_scrub_stamp":"2024-04-25T10:30:11.733978+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T12:08:30.050992+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5],"acting":[4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.35","version":"0'0","reported_seq":12,"reported_epoch":36,"state":"active+clean","last_fresh":"2024-04-25T10:30:13.760760+0000","last_change":"2024-04-25T10:30:12.776164+0000","last_active":"2024-04-25T10:30:13.760760+0000","last_peered":"2024-04-25T10:30:13.760760+0000","last_clean":"2024-04-25T10:30:13.760760+0000","last_became_active":"2024-04-25T10:30:12.775455+0000","last_became_peered":"2024-04-25T10:30:12.775455+0000","last_unstale":"2024-04-25T10:30:13.760760+0000","last_undegraded":"2024-04-25T10:30:13.760760+0000","last_fullsized":"2024-04-25T10:30:13.760760+0000","mapping_epoch":34,"log_start":"0'0","ondisk_log_start":"0'0","created":34,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:30:11.733978+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:30:11.733978+0000","last_clean_scrub_stamp":"2024-04-25T10:30:11.733978+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T14:08:35.756973+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.34","version":"0'0","reported_seq":12,"reported_epoch":36,"state":"active+clean","last_fresh":"2024-04-25T10:30:14.411114+0000","last_change":"2024-04-25T10:30:12.761162+0000","last_active":"2024-04-25T10:30:14.411114+0000","last_peered":"2024-04-25T10:30:14.411114+0000","last_clean":"2024-04-25T10:30:14.411114+0000","last_became_active":"2024-04-25T10:30:12.760835+0000","last_became_peered":"2024-04-25T10:30:12.760835+0000","last_unstale":"2024-04-25T10:30:14.411114+0000","last_undegraded":"2024-04-25T10:30:14.411114+0000","last_fullsized":"2024-04-25T10:30:14.411114+0000","mapping_epoch":34,"log_start":"0'0","ondisk_log_start":"0'0","created":34,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:30:11.733978+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:30:11.733978+0000","last_clean_scrub_stamp":"2024-04-25T10:30:11.733978+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T16:30:12.714228+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"5.33","version":"0'0","reported_seq":12,"reported_epoch":36,"state":"active+clean","last_fresh":"2024-04-25T10:30:13.762122+0000","last_change":"2024-04-25T10:30:12.777499+0000","last_active":"2024-04-25T10:30:13.762122+0000","last_peered":"2024-04-25T10:30:13.762122+0000","last_clean":"2024-04-25T10:30:13.762122+0000","last_became_active":"2024-04-25T10:30:12.776773+0000","last_became_peered":"2024-04-25T10:30:12.776773+0000","last_unstale":"2024-04-25T10:30:13.762122+0000","last_undegraded":"2024-04-25T10:30:13.762122+0000","last_fullsized":"2024-04-25T10:30:13.762122+0000","mapping_epoch":34,"log_start":"0'0","ondisk_log_start":"0'0","created":34,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:30:11.733978+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:30:11.733978+0000","last_clean_scrub_stamp":"2024-04-25T10:30:11.733978+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T16:49:00.268423+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"5.32","version":"0'0","reported_seq":9,"reported_epoch":35,"state":"active+clean","last_fresh":"2024-04-25T10:30:12.778573+0000","last_change":"2024-04-25T10:30:12.772243+0000","last_active":"2024-04-25T10:30:12.778573+0000","last_peered":"2024-04-25T10:30:12.778573+0000","last_clean":"2024-04-25T10:30:12.778573+0000","last_became_active":"2024-04-25T10:30:12.771936+0000","last_became_peered":"2024-04-25T10:30:12.771936+0000","last_unstale":"2024-04-25T10:30:12.778573+0000","last_undegraded":"2024-04-25T10:30:12.778573+0000","last_fullsized":"2024-04-25T10:30:12.778573+0000","mapping_epoch":34,"log_start":"0'0","ondisk_log_start":"0'0","created":34,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:30:11.733978+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:30:11.733978+0000","last_clean_scrub_stamp":"2024-04-25T10:30:11.733978+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T14:32:32.178874+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5],"acting":[4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.31","version":"0'0","reported_seq":12,"reported_epoch":36,"state":"active+clean","last_fresh":"2024-04-25T10:30:14.195534+0000","last_change":"2024-04-25T10:30:12.777324+0000","last_active":"2024-04-25T10:30:14.195534+0000","last_peered":"2024-04-25T10:30:14.195534+0000","last_clean":"2024-04-25T10:30:14.195534+0000","last_became_active":"2024-04-25T10:30:12.776983+0000","last_became_peered":"2024-04-25T10:30:12.776983+0000","last_unstale":"2024-04-25T10:30:14.195534+0000","last_undegraded":"2024-04-25T10:30:14.195534+0000","last_fullsized":"2024-04-25T10:30:14.195534+0000","mapping_epoch":34,"log_start":"0'0","ondisk_log_start":"0'0","created":34,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:30:11.733978+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:30:11.733978+0000","last_clean_scrub_stamp":"2024-04-25T10:30:11.733978+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T12:27:54.564692+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"5.30","version":"0'0","reported_seq":12,"reported_epoch":36,"state":"active+clean","last_fresh":"2024-04-25T10:30:14.195428+0000","last_change":"2024-04-25T10:30:12.765989+0000","last_active":"2024-04-25T10:30:14.195428+0000","last_peered":"2024-04-25T10:30:14.195428+0000","last_clean":"2024-04-25T10:30:14.195428+0000","last_became_active":"2024-04-25T10:30:12.765648+0000","last_became_peered":"2024-04-25T10:30:12.765648+0000","last_unstale":"2024-04-25T10:30:14.195428+0000","last_undegraded":"2024-04-25T10:30:14.195428+0000","last_fullsized":"2024-04-25T10:30:14.195428+0000","mapping_epoch":34,"log_start":"0'0","ondisk_log_start":"0'0","created":34,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:30:11.733978+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:30:11.733978+0000","last_clean_scrub_stamp":"2024-04-25T10:30:11.733978+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T16:32:01.312412+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"5.2f","version":"0'0","reported_seq":12,"reported_epoch":36,"state":"active+clean","last_fresh":"2024-04-25T10:30:13.760553+0000","last_change":"2024-04-25T10:30:12.766616+0000","last_active":"2024-04-25T10:30:13.760553+0000","last_peered":"2024-04-25T10:30:13.760553+0000","last_clean":"2024-04-25T10:30:13.760553+0000","last_became_active":"2024-04-25T10:30:12.766290+0000","last_became_peered":"2024-04-25T10:30:12.766290+0000","last_unstale":"2024-04-25T10:30:13.760553+0000","last_undegraded":"2024-04-25T10:30:13.760553+0000","last_fullsized":"2024-04-25T10:30:13.760553+0000","mapping_epoch":34,"log_start":"0'0","ondisk_log_start":"0'0","created":34,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:30:11.733978+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:30:11.733978+0000","last_clean_scrub_stamp":"2024-04-25T10:30:11.733978+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T17:35:33.561138+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.2e","version":"0'0","reported_seq":12,"reported_epoch":36,"state":"active+clean","last_fresh":"2024-04-25T10:30:14.411550+0000","last_change":"2024-04-25T10:30:12.766393+0000","last_active":"2024-04-25T10:30:14.411550+0000","last_peered":"2024-04-25T10:30:14.411550+0000","last_clean":"2024-04-25T10:30:14.411550+0000","last_became_active":"2024-04-25T10:30:12.766096+0000","last_became_peered":"2024-04-25T10:30:12.766096+0000","last_unstale":"2024-04-25T10:30:14.411550+0000","last_undegraded":"2024-04-25T10:30:14.411550+0000","last_fullsized":"2024-04-25T10:30:14.411550+0000","mapping_epoch":34,"log_start":"0'0","ondisk_log_start":"0'0","created":34,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:30:11.733978+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:30:11.733978+0000","last_clean_scrub_stamp":"2024-04-25T10:30:11.733978+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T17:58:01.091931+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"5.2d","version":"0'0","reported_seq":12,"reported_epoch":36,"state":"active+clean","last_fresh":"2024-04-25T10:30:14.411042+0000","last_change":"2024-04-25T10:30:12.768688+0000","last_active":"2024-04-25T10:30:14.411042+0000","last_peered":"2024-04-25T10:30:14.411042+0000","last_clean":"2024-04-25T10:30:14.411042+0000","last_became_active":"2024-04-25T10:30:12.768382+0000","last_became_peered":"2024-04-25T10:30:12.768382+0000","last_unstale":"2024-04-25T10:30:14.411042+0000","last_undegraded":"2024-04-25T10:30:14.411042+0000","last_fullsized":"2024-04-25T10:30:14.411042+0000","mapping_epoch":34,"log_start":"0'0","ondisk_log_start":"0'0","created":34,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:30:11.733978+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:30:11.733978+0000","last_clean_scrub_stamp":"2024-04-25T10:30:11.733978+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T17:24:14.853132+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"5.2c","version":"0'0","reported_seq":12,"reported_epoch":36,"state":"active+clean","last_fresh":"2024-04-25T10:30:13.762150+0000","last_change":"2024-04-25T10:30:12.772663+0000","last_active":"2024-04-25T10:30:13.762150+0000","last_peered":"2024-04-25T10:30:13.762150+0000","last_clean":"2024-04-25T10:30:13.762150+0000","last_became_active":"2024-04-25T10:30:12.772356+0000","last_became_peered":"2024-04-25T10:30:12.772356+0000","last_unstale":"2024-04-25T10:30:13.762150+0000","last_undegraded":"2024-04-25T10:30:13.762150+0000","last_fullsized":"2024-04-25T10:30:13.762150+0000","mapping_epoch":34,"log_start":"0'0","ondisk_log_start":"0'0","created":34,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:30:11.733978+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:30:11.733978+0000","last_clean_scrub_stamp":"2024-04-25T10:30:11.733978+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T16:08:29.987819+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"5.2b","version":"0'0","reported_seq":12,"reported_epoch":36,"state":"active+clean","last_fresh":"2024-04-25T10:30:14.195727+0000","last_change":"2024-04-25T10:30:12.771344+0000","last_active":"2024-04-25T10:30:14.195727+0000","last_peered":"2024-04-25T10:30:14.195727+0000","last_clean":"2024-04-25T10:30:14.195727+0000","last_became_active":"2024-04-25T10:30:12.770949+0000","last_became_peered":"2024-04-25T10:30:12.770949+0000","last_unstale":"2024-04-25T10:30:14.195727+0000","last_undegraded":"2024-04-25T10:30:14.195727+0000","last_fullsized":"2024-04-25T10:30:14.195727+0000","mapping_epoch":34,"log_start":"0'0","ondisk_log_start":"0'0","created":34,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:30:11.733978+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:30:11.733978+0000","last_clean_scrub_stamp":"2024-04-25T10:30:11.733978+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T15:47:40.336983+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"5.2a","version":"0'0","reported_seq":12,"reported_epoch":36,"state":"active+clean","last_fresh":"2024-04-25T10:30:13.762475+0000","last_change":"2024-04-25T10:30:12.782730+0000","last_active":"2024-04-25T10:30:13.762475+0000","last_peered":"2024-04-25T10:30:13.762475+0000","last_clean":"2024-04-25T10:30:13.762475+0000","last_became_active":"2024-04-25T10:30:12.782261+0000","last_became_peered":"2024-04-25T10:30:12.782261+0000","last_unstale":"2024-04-25T10:30:13.762475+0000","last_undegraded":"2024-04-25T10:30:13.762475+0000","last_fullsized":"2024-04-25T10:30:13.762475+0000","mapping_epoch":34,"log_start":"0'0","ondisk_log_start":"0'0","created":34,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:30:11.733978+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:30:11.733978+0000","last_clean_scrub_stamp":"2024-04-25T10:30:11.733978+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T16:08:29.987819+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4],"acting":[2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"5.29","version":"0'0","reported_seq":12,"reported_epoch":36,"state":"active+clean","last_fresh":"2024-04-25T10:30:13.761684+0000","last_change":"2024-04-25T10:30:12.763911+0000","last_active":"2024-04-25T10:30:13.761684+0000","last_peered":"2024-04-25T10:30:13.761684+0000","last_clean":"2024-04-25T10:30:13.761684+0000","last_became_active":"2024-04-25T10:30:12.763578+0000","last_became_peered":"2024-04-25T10:30:12.763578+0000","last_unstale":"2024-04-25T10:30:13.761684+0000","last_undegraded":"2024-04-25T10:30:13.761684+0000","last_fullsized":"2024-04-25T10:30:13.761684+0000","mapping_epoch":34,"log_start":"0'0","ondisk_log_start":"0'0","created":34,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:30:11.733978+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:30:11.733978+0000","last_clean_scrub_stamp":"2024-04-25T10:30:11.733978+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T18:26:06.164667+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,1],"acting":[6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"5.28","version":"0'0","reported_seq":12,"reported_epoch":36,"state":"active+clean","last_fresh":"2024-04-25T10:30:13.762434+0000","last_change":"2024-04-25T10:30:12.783394+0000","last_active":"2024-04-25T10:30:13.762434+0000","last_peered":"2024-04-25T10:30:13.762434+0000","last_clean":"2024-04-25T10:30:13.762434+0000","last_became_active":"2024-04-25T10:30:12.783066+0000","last_became_peered":"2024-04-25T10:30:12.783066+0000","last_unstale":"2024-04-25T10:30:13.762434+0000","last_undegraded":"2024-04-25T10:30:13.762434+0000","last_fullsized":"2024-04-25T10:30:13.762434+0000","mapping_epoch":34,"log_start":"0'0","ondisk_log_start":"0'0","created":34,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:30:11.733978+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:30:11.733978+0000","last_clean_scrub_stamp":"2024-04-25T10:30:11.733978+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T16:49:00.268423+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4],"acting":[2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"5.27","version":"0'0","reported_seq":9,"reported_epoch":35,"state":"active+clean","last_fresh":"2024-04-25T10:30:12.785555+0000","last_change":"2024-04-25T10:30:12.779972+0000","last_active":"2024-04-25T10:30:12.785555+0000","last_peered":"2024-04-25T10:30:12.785555+0000","last_clean":"2024-04-25T10:30:12.785555+0000","last_became_active":"2024-04-25T10:30:12.779526+0000","last_became_peered":"2024-04-25T10:30:12.779526+0000","last_unstale":"2024-04-25T10:30:12.785555+0000","last_undegraded":"2024-04-25T10:30:12.785555+0000","last_fullsized":"2024-04-25T10:30:12.785555+0000","mapping_epoch":34,"log_start":"0'0","ondisk_log_start":"0'0","created":34,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:30:11.733978+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:30:11.733978+0000","last_clean_scrub_stamp":"2024-04-25T10:30:11.733978+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T13:39:19.978606+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.26","version":"0'0","reported_seq":9,"reported_epoch":35,"state":"active+clean","last_fresh":"2024-04-25T10:30:12.778668+0000","last_change":"2024-04-25T10:30:12.771420+0000","last_active":"2024-04-25T10:30:12.778668+0000","last_peered":"2024-04-25T10:30:12.778668+0000","last_clean":"2024-04-25T10:30:12.778668+0000","last_became_active":"2024-04-25T10:30:12.771117+0000","last_became_peered":"2024-04-25T10:30:12.771117+0000","last_unstale":"2024-04-25T10:30:12.778668+0000","last_undegraded":"2024-04-25T10:30:12.778668+0000","last_fullsized":"2024-04-25T10:30:12.778668+0000","mapping_epoch":34,"log_start":"0'0","ondisk_log_start":"0'0","created":34,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:30:11.733978+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:30:11.733978+0000","last_clean_scrub_stamp":"2024-04-25T10:30:11.733978+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T15:06:28.094728+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.25","version":"0'0","reported_seq":12,"reported_epoch":36,"state":"active+clean","last_fresh":"2024-04-25T10:30:14.411287+0000","last_change":"2024-04-25T10:30:12.767505+0000","last_active":"2024-04-25T10:30:14.411287+0000","last_peered":"2024-04-25T10:30:14.411287+0000","last_clean":"2024-04-25T10:30:14.411287+0000","last_became_active":"2024-04-25T10:30:12.767149+0000","last_became_peered":"2024-04-25T10:30:12.767149+0000","last_unstale":"2024-04-25T10:30:14.411287+0000","last_undegraded":"2024-04-25T10:30:14.411287+0000","last_fullsized":"2024-04-25T10:30:14.411287+0000","mapping_epoch":34,"log_start":"0'0","ondisk_log_start":"0'0","created":34,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:30:11.733978+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:30:11.733978+0000","last_clean_scrub_stamp":"2024-04-25T10:30:11.733978+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T19:25:06.397830+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"5.24","version":"0'0","reported_seq":12,"reported_epoch":36,"state":"active+clean","last_fresh":"2024-04-25T10:30:13.762844+0000","last_change":"2024-04-25T10:30:12.775515+0000","last_active":"2024-04-25T10:30:13.762844+0000","last_peered":"2024-04-25T10:30:13.762844+0000","last_clean":"2024-04-25T10:30:13.762844+0000","last_became_active":"2024-04-25T10:30:12.775167+0000","last_became_peered":"2024-04-25T10:30:12.775167+0000","last_unstale":"2024-04-25T10:30:13.762844+0000","last_undegraded":"2024-04-25T10:30:13.762844+0000","last_fullsized":"2024-04-25T10:30:13.762844+0000","mapping_epoch":34,"log_start":"0'0","ondisk_log_start":"0'0","created":34,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:30:11.733978+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:30:11.733978+0000","last_clean_scrub_stamp":"2024-04-25T10:30:11.733978+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T17:41:34.910195+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,6],"acting":[2,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"5.23","version":"0'0","reported_seq":12,"reported_epoch":36,"state":"active+clean","last_fresh":"2024-04-25T10:30:14.195712+0000","last_change":"2024-04-25T10:30:12.781995+0000","last_active":"2024-04-25T10:30:14.195712+0000","last_peered":"2024-04-25T10:30:14.195712+0000","last_clean":"2024-04-25T10:30:14.195712+0000","last_became_active":"2024-04-25T10:30:12.781358+0000","last_became_peered":"2024-04-25T10:30:12.781358+0000","last_unstale":"2024-04-25T10:30:14.195712+0000","last_undegraded":"2024-04-25T10:30:14.195712+0000","last_fullsized":"2024-04-25T10:30:14.195712+0000","mapping_epoch":34,"log_start":"0'0","ondisk_log_start":"0'0","created":34,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:30:11.733978+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:30:11.733978+0000","last_clean_scrub_stamp":"2024-04-25T10:30:11.733978+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T18:21:32.178524+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"5.22","version":"0'0","reported_seq":12,"reported_epoch":36,"state":"active+clean","last_fresh":"2024-04-25T10:30:14.195443+0000","last_change":"2024-04-25T10:30:12.769958+0000","last_active":"2024-04-25T10:30:14.195443+0000","last_peered":"2024-04-25T10:30:14.195443+0000","last_clean":"2024-04-25T10:30:14.195443+0000","last_became_active":"2024-04-25T10:30:12.769590+0000","last_became_peered":"2024-04-25T10:30:12.769590+0000","last_unstale":"2024-04-25T10:30:14.195443+0000","last_undegraded":"2024-04-25T10:30:14.195443+0000","last_fullsized":"2024-04-25T10:30:14.195443+0000","mapping_epoch":34,"log_start":"0'0","ondisk_log_start":"0'0","created":34,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:30:11.733978+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:30:11.733978+0000","last_clean_scrub_stamp":"2024-04-25T10:30:11.733978+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T17:41:34.910195+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"5.21","version":"0'0","reported_seq":12,"reported_epoch":36,"state":"active+clean","last_fresh":"2024-04-25T10:30:13.762540+0000","last_change":"2024-04-25T10:30:12.774245+0000","last_active":"2024-04-25T10:30:13.762540+0000","last_peered":"2024-04-25T10:30:13.762540+0000","last_clean":"2024-04-25T10:30:13.762540+0000","last_became_active":"2024-04-25T10:30:12.773923+0000","last_became_peered":"2024-04-25T10:30:12.773923+0000","last_unstale":"2024-04-25T10:30:13.762540+0000","last_undegraded":"2024-04-25T10:30:13.762540+0000","last_fullsized":"2024-04-25T10:30:13.762540+0000","mapping_epoch":34,"log_start":"0'0","ondisk_log_start":"0'0","created":34,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:30:11.733978+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:30:11.733978+0000","last_clean_scrub_stamp":"2024-04-25T10:30:11.733978+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T11:32:45.617923+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,7],"acting":[2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"5.20","version":"0'0","reported_seq":12,"reported_epoch":36,"state":"active+clean","last_fresh":"2024-04-25T10:30:13.760265+0000","last_change":"2024-04-25T10:30:12.766692+0000","last_active":"2024-04-25T10:30:13.760265+0000","last_peered":"2024-04-25T10:30:13.760265+0000","last_clean":"2024-04-25T10:30:13.760265+0000","last_became_active":"2024-04-25T10:30:12.766336+0000","last_became_peered":"2024-04-25T10:30:12.766336+0000","last_unstale":"2024-04-25T10:30:13.760265+0000","last_undegraded":"2024-04-25T10:30:13.760265+0000","last_fullsized":"2024-04-25T10:30:13.760265+0000","mapping_epoch":34,"log_start":"0'0","ondisk_log_start":"0'0","created":34,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:30:11.733978+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:30:11.733978+0000","last_clean_scrub_stamp":"2024-04-25T10:30:11.733978+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T22:09:01.889527+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.1e","version":"0'0","reported_seq":12,"reported_epoch":36,"state":"active+clean","last_fresh":"2024-04-25T10:30:13.762215+0000","last_change":"2024-04-25T10:30:12.769800+0000","last_active":"2024-04-25T10:30:13.762215+0000","last_peered":"2024-04-25T10:30:13.762215+0000","last_clean":"2024-04-25T10:30:13.762215+0000","last_became_active":"2024-04-25T10:30:12.769470+0000","last_became_peered":"2024-04-25T10:30:12.769470+0000","last_unstale":"2024-04-25T10:30:13.762215+0000","last_undegraded":"2024-04-25T10:30:13.762215+0000","last_fullsized":"2024-04-25T10:30:13.762215+0000","mapping_epoch":34,"log_start":"0'0","ondisk_log_start":"0'0","created":34,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:30:11.733978+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:30:11.733978+0000","last_clean_scrub_stamp":"2024-04-25T10:30:11.733978+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T11:32:45.617923+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.1f","version":"35'4","reported_seq":18,"reported_epoch":36,"state":"active+clean","last_fresh":"2024-04-25T10:30:13.760656+0000","last_change":"2024-04-25T10:30:11.756978+0000","last_active":"2024-04-25T10:30:13.760656+0000","last_peered":"2024-04-25T10:30:13.760656+0000","last_clean":"2024-04-25T10:30:13.760656+0000","last_became_active":"2024-04-25T10:30:11.756617+0000","last_became_peered":"2024-04-25T10:30:11.756617+0000","last_unstale":"2024-04-25T10:30:13.760656+0000","last_undegraded":"2024-04-25T10:30:13.760656+0000","last_fullsized":"2024-04-25T10:30:13.760656+0000","mapping_epoch":33,"log_start":"0'0","ondisk_log_start":"0'0","created":33,"last_epoch_clean":34,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:30:10.729165+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:30:10.729165+0000","last_clean_scrub_stamp":"2024-04-25T10:30:10.729165+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T12:27:53.559880+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":582,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.1f","version":"0'0","reported_seq":12,"reported_epoch":36,"state":"active+clean","last_fresh":"2024-04-25T10:30:13.762989+0000","last_change":"2024-04-25T10:30:12.772257+0000","last_active":"2024-04-25T10:30:13.762989+0000","last_peered":"2024-04-25T10:30:13.762989+0000","last_clean":"2024-04-25T10:30:13.762989+0000","last_became_active":"2024-04-25T10:30:12.771923+0000","last_became_peered":"2024-04-25T10:30:12.771923+0000","last_unstale":"2024-04-25T10:30:13.762989+0000","last_undegraded":"2024-04-25T10:30:13.762989+0000","last_fullsized":"2024-04-25T10:30:13.762989+0000","mapping_epoch":34,"log_start":"0'0","ondisk_log_start":"0'0","created":34,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:30:11.733978+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:30:11.733978+0000","last_clean_scrub_stamp":"2024-04-25T10:30:11.733978+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T18:52:45.539758+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.1e","version":"35'1","reported_seq":13,"reported_epoch":35,"state":"active+clean","last_fresh":"2024-04-25T10:30:12.786487+0000","last_change":"2024-04-25T10:30:11.761160+0000","last_active":"2024-04-25T10:30:12.786487+0000","last_peered":"2024-04-25T10:30:12.786487+0000","last_clean":"2024-04-25T10:30:12.786487+0000","last_became_active":"2024-04-25T10:30:11.760819+0000","last_became_peered":"2024-04-25T10:30:11.760819+0000","last_unstale":"2024-04-25T10:30:12.786487+0000","last_undegraded":"2024-04-25T10:30:12.786487+0000","last_fullsized":"2024-04-25T10:30:12.786487+0000","mapping_epoch":33,"log_start":"0'0","ondisk_log_start":"0'0","created":33,"last_epoch_clean":34,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:30:10.729165+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:30:10.729165+0000","last_clean_scrub_stamp":"2024-04-25T10:30:10.729165+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T13:00:22.139475+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.1c","version":"0'0","reported_seq":12,"reported_epoch":36,"state":"active+clean","last_fresh":"2024-04-25T10:30:13.762885+0000","last_change":"2024-04-25T10:30:12.781913+0000","last_active":"2024-04-25T10:30:13.762885+0000","last_peered":"2024-04-25T10:30:13.762885+0000","last_clean":"2024-04-25T10:30:13.762885+0000","last_became_active":"2024-04-25T10:30:12.781111+0000","last_became_peered":"2024-04-25T10:30:12.781111+0000","last_unstale":"2024-04-25T10:30:13.762885+0000","last_undegraded":"2024-04-25T10:30:13.762885+0000","last_fullsized":"2024-04-25T10:30:13.762885+0000","mapping_epoch":34,"log_start":"0'0","ondisk_log_start":"0'0","created":34,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:30:11.733978+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:30:11.733978+0000","last_clean_scrub_stamp":"2024-04-25T10:30:11.733978+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T12:27:54.564692+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4],"acting":[2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.1d","version":"35'2","reported_seq":16,"reported_epoch":36,"state":"active+clean","last_fresh":"2024-04-25T10:30:14.411566+0000","last_change":"2024-04-25T10:30:11.752439+0000","last_active":"2024-04-25T10:30:14.411566+0000","last_peered":"2024-04-25T10:30:14.411566+0000","last_clean":"2024-04-25T10:30:14.411566+0000","last_became_active":"2024-04-25T10:30:11.752050+0000","last_became_peered":"2024-04-25T10:30:11.752050+0000","last_unstale":"2024-04-25T10:30:14.411566+0000","last_undegraded":"2024-04-25T10:30:14.411566+0000","last_fullsized":"2024-04-25T10:30:14.411566+0000","mapping_epoch":33,"log_start":"0'0","ondisk_log_start":"0'0","created":33,"last_epoch_clean":34,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:30:10.729165+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:30:10.729165+0000","last_clean_scrub_stamp":"2024-04-25T10:30:10.729165+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T11:41:21.358729+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"5.1d","version":"0'0","reported_seq":9,"reported_epoch":35,"state":"active+clean","last_fresh":"2024-04-25T10:30:12.779046+0000","last_change":"2024-04-25T10:30:12.773474+0000","last_active":"2024-04-25T10:30:12.779046+0000","last_peered":"2024-04-25T10:30:12.779046+0000","last_clean":"2024-04-25T10:30:12.779046+0000","last_became_active":"2024-04-25T10:30:12.773133+0000","last_became_peered":"2024-04-25T10:30:12.773133+0000","last_unstale":"2024-04-25T10:30:12.779046+0000","last_undegraded":"2024-04-25T10:30:12.779046+0000","last_fullsized":"2024-04-25T10:30:12.779046+0000","mapping_epoch":34,"log_start":"0'0","ondisk_log_start":"0'0","created":34,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:30:11.733978+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:30:11.733978+0000","last_clean_scrub_stamp":"2024-04-25T10:30:11.733978+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T15:53:06.665247+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.1c","version":"35'2","reported_seq":14,"reported_epoch":35,"state":"active+clean","last_fresh":"2024-04-25T10:30:12.785935+0000","last_change":"2024-04-25T10:30:11.759829+0000","last_active":"2024-04-25T10:30:12.785935+0000","last_peered":"2024-04-25T10:30:12.785935+0000","last_clean":"2024-04-25T10:30:12.785935+0000","last_became_active":"2024-04-25T10:30:11.759465+0000","last_became_peered":"2024-04-25T10:30:11.759465+0000","last_unstale":"2024-04-25T10:30:12.785935+0000","last_undegraded":"2024-04-25T10:30:12.785935+0000","last_fullsized":"2024-04-25T10:30:12.785935+0000","mapping_epoch":33,"log_start":"0'0","ondisk_log_start":"0'0","created":33,"last_epoch_clean":34,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:30:10.729165+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:30:10.729165+0000","last_clean_scrub_stamp":"2024-04-25T10:30:10.729165+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T13:31:53.608355+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.1a","version":"0'0","reported_seq":12,"reported_epoch":36,"state":"active+clean","last_fresh":"2024-04-25T10:30:14.411115+0000","last_change":"2024-04-25T10:30:12.773835+0000","last_active":"2024-04-25T10:30:14.411115+0000","last_peered":"2024-04-25T10:30:14.411115+0000","last_clean":"2024-04-25T10:30:14.411115+0000","last_became_active":"2024-04-25T10:30:12.773538+0000","last_became_peered":"2024-04-25T10:30:12.773538+0000","last_unstale":"2024-04-25T10:30:14.411115+0000","last_undegraded":"2024-04-25T10:30:14.411115+0000","last_fullsized":"2024-04-25T10:30:14.411115+0000","mapping_epoch":34,"log_start":"0'0","ondisk_log_start":"0'0","created":34,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:30:11.733978+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:30:11.733978+0000","last_clean_scrub_stamp":"2024-04-25T10:30:11.733978+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T16:17:55.381353+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.1b","version":"35'1","reported_seq":15,"reported_epoch":36,"state":"active+clean","last_fresh":"2024-04-25T10:30:13.762774+0000","last_change":"2024-04-25T10:30:11.753667+0000","last_active":"2024-04-25T10:30:13.762774+0000","last_peered":"2024-04-25T10:30:13.762774+0000","last_clean":"2024-04-25T10:30:13.762774+0000","last_became_active":"2024-04-25T10:30:11.753340+0000","last_became_peered":"2024-04-25T10:30:11.753340+0000","last_unstale":"2024-04-25T10:30:13.762774+0000","last_undegraded":"2024-04-25T10:30:13.762774+0000","last_fullsized":"2024-04-25T10:30:13.762774+0000","mapping_epoch":33,"log_start":"0'0","ondisk_log_start":"0'0","created":33,"last_epoch_clean":34,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:30:10.729165+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:30:10.729165+0000","last_clean_scrub_stamp":"2024-04-25T10:30:10.729165+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T16:09:04.081750+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"5.1b","version":"0'0","reported_seq":12,"reported_epoch":36,"state":"active+clean","last_fresh":"2024-04-25T10:30:13.762652+0000","last_change":"2024-04-25T10:30:12.784767+0000","last_active":"2024-04-25T10:30:13.762652+0000","last_peered":"2024-04-25T10:30:13.762652+0000","last_clean":"2024-04-25T10:30:13.762652+0000","last_became_active":"2024-04-25T10:30:12.784311+0000","last_became_peered":"2024-04-25T10:30:12.784311+0000","last_unstale":"2024-04-25T10:30:13.762652+0000","last_undegraded":"2024-04-25T10:30:13.762652+0000","last_fullsized":"2024-04-25T10:30:13.762652+0000","mapping_epoch":34,"log_start":"0'0","ondisk_log_start":"0'0","created":34,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:30:11.733978+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:30:11.733978+0000","last_clean_scrub_stamp":"2024-04-25T10:30:11.733978+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T16:32:01.312412+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,7],"acting":[2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.1a","version":"35'1","reported_seq":15,"reported_epoch":36,"state":"active+clean","last_fresh":"2024-04-25T10:30:13.762501+0000","last_change":"2024-04-25T10:30:11.758678+0000","last_active":"2024-04-25T10:30:13.762501+0000","last_peered":"2024-04-25T10:30:13.762501+0000","last_clean":"2024-04-25T10:30:13.762501+0000","last_became_active":"2024-04-25T10:30:11.758384+0000","last_became_peered":"2024-04-25T10:30:11.758384+0000","last_unstale":"2024-04-25T10:30:13.762501+0000","last_undegraded":"2024-04-25T10:30:13.762501+0000","last_fullsized":"2024-04-25T10:30:13.762501+0000","mapping_epoch":33,"log_start":"0'0","ondisk_log_start":"0'0","created":33,"last_epoch_clean":34,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:30:10.729165+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:30:10.729165+0000","last_clean_scrub_stamp":"2024-04-25T10:30:10.729165+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T16:17:54.376541+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,6],"acting":[2,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"5.18","version":"0'0","reported_seq":12,"reported_epoch":36,"state":"active+clean","last_fresh":"2024-04-25T10:30:14.410937+0000","last_change":"2024-04-25T10:30:12.769294+0000","last_active":"2024-04-25T10:30:14.410937+0000","last_peered":"2024-04-25T10:30:14.410937+0000","last_clean":"2024-04-25T10:30:14.410937+0000","last_became_active":"2024-04-25T10:30:12.768543+0000","last_became_peered":"2024-04-25T10:30:12.768543+0000","last_unstale":"2024-04-25T10:30:14.410937+0000","last_undegraded":"2024-04-25T10:30:14.410937+0000","last_fullsized":"2024-04-25T10:30:14.410937+0000","mapping_epoch":34,"log_start":"0'0","ondisk_log_start":"0'0","created":34,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:30:11.733978+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:30:11.733978+0000","last_clean_scrub_stamp":"2024-04-25T10:30:11.733978+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T16:49:21.091826+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.19","version":"0'0","reported_seq":14,"reported_epoch":36,"state":"active+clean","last_fresh":"2024-04-25T10:30:14.410830+0000","last_change":"2024-04-25T10:30:11.757481+0000","last_active":"2024-04-25T10:30:14.410830+0000","last_peered":"2024-04-25T10:30:14.410830+0000","last_clean":"2024-04-25T10:30:14.410830+0000","last_became_active":"2024-04-25T10:30:11.757149+0000","last_became_peered":"2024-04-25T10:30:11.757149+0000","last_unstale":"2024-04-25T10:30:14.410830+0000","last_undegraded":"2024-04-25T10:30:14.410830+0000","last_fullsized":"2024-04-25T10:30:14.410830+0000","mapping_epoch":33,"log_start":"0'0","ondisk_log_start":"0'0","created":33,"last_epoch_clean":34,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:30:10.729165+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:30:10.729165+0000","last_clean_scrub_stamp":"2024-04-25T10:30:10.729165+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T13:23:15.814583+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"5.19","version":"0'0","reported_seq":9,"reported_epoch":35,"state":"active+clean","last_fresh":"2024-04-25T10:30:12.785526+0000","last_change":"2024-04-25T10:30:12.780437+0000","last_active":"2024-04-25T10:30:12.785526+0000","last_peered":"2024-04-25T10:30:12.785526+0000","last_clean":"2024-04-25T10:30:12.785526+0000","last_became_active":"2024-04-25T10:30:12.779762+0000","last_became_peered":"2024-04-25T10:30:12.779762+0000","last_unstale":"2024-04-25T10:30:12.785526+0000","last_undegraded":"2024-04-25T10:30:12.785526+0000","last_fullsized":"2024-04-25T10:30:12.785526+0000","mapping_epoch":34,"log_start":"0'0","ondisk_log_start":"0'0","created":34,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:30:11.733978+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:30:11.733978+0000","last_clean_scrub_stamp":"2024-04-25T10:30:11.733978+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T13:43:38.026914+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.18","version":"0'0","reported_seq":14,"reported_epoch":36,"state":"active+clean","last_fresh":"2024-04-25T10:30:13.762556+0000","last_change":"2024-04-25T10:30:11.756627+0000","last_active":"2024-04-25T10:30:13.762556+0000","last_peered":"2024-04-25T10:30:13.762556+0000","last_clean":"2024-04-25T10:30:13.762556+0000","last_became_active":"2024-04-25T10:30:11.756323+0000","last_became_peered":"2024-04-25T10:30:11.756323+0000","last_unstale":"2024-04-25T10:30:13.762556+0000","last_undegraded":"2024-04-25T10:30:13.762556+0000","last_fullsized":"2024-04-25T10:30:13.762556+0000","mapping_epoch":33,"log_start":"0'0","ondisk_log_start":"0'0","created":33,"last_epoch_clean":34,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:30:10.729165+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:30:10.729165+0000","last_clean_scrub_stamp":"2024-04-25T10:30:10.729165+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T16:49:20.087013+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4],"acting":[2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"5.16","version":"0'0","reported_seq":12,"reported_epoch":36,"state":"active+clean","last_fresh":"2024-04-25T10:30:13.760859+0000","last_change":"2024-04-25T10:30:12.770216+0000","last_active":"2024-04-25T10:30:13.760859+0000","last_peered":"2024-04-25T10:30:13.760859+0000","last_clean":"2024-04-25T10:30:13.760859+0000","last_became_active":"2024-04-25T10:30:12.769785+0000","last_became_peered":"2024-04-25T10:30:12.769785+0000","last_unstale":"2024-04-25T10:30:13.760859+0000","last_undegraded":"2024-04-25T10:30:13.760859+0000","last_fullsized":"2024-04-25T10:30:13.760859+0000","mapping_epoch":34,"log_start":"0'0","ondisk_log_start":"0'0","created":34,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:30:11.733978+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:30:11.733978+0000","last_clean_scrub_stamp":"2024-04-25T10:30:11.733978+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T18:12:41.326304+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.17","version":"0'0","reported_seq":14,"reported_epoch":36,"state":"active+clean","last_fresh":"2024-04-25T10:30:14.411352+0000","last_change":"2024-04-25T10:30:11.758416+0000","last_active":"2024-04-25T10:30:14.411352+0000","last_peered":"2024-04-25T10:30:14.411352+0000","last_clean":"2024-04-25T10:30:14.411352+0000","last_became_active":"2024-04-25T10:30:11.758049+0000","last_became_peered":"2024-04-25T10:30:11.758049+0000","last_unstale":"2024-04-25T10:30:14.411352+0000","last_undegraded":"2024-04-25T10:30:14.411352+0000","last_fullsized":"2024-04-25T10:30:14.411352+0000","mapping_epoch":33,"log_start":"0'0","ondisk_log_start":"0'0","created":33,"last_epoch_clean":34,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:30:10.729165+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:30:10.729165+0000","last_clean_scrub_stamp":"2024-04-25T10:30:10.729165+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T18:26:05.159854+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"5.17","version":"0'0","reported_seq":9,"reported_epoch":35,"state":"active+clean","last_fresh":"2024-04-25T10:30:12.778640+0000","last_change":"2024-04-25T10:30:12.773113+0000","last_active":"2024-04-25T10:30:12.778640+0000","last_peered":"2024-04-25T10:30:12.778640+0000","last_clean":"2024-04-25T10:30:12.778640+0000","last_became_active":"2024-04-25T10:30:12.772786+0000","last_became_peered":"2024-04-25T10:30:12.772786+0000","last_unstale":"2024-04-25T10:30:12.778640+0000","last_undegraded":"2024-04-25T10:30:12.778640+0000","last_fullsized":"2024-04-25T10:30:12.778640+0000","mapping_epoch":34,"log_start":"0'0","ondisk_log_start":"0'0","created":34,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:30:11.733978+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:30:11.733978+0000","last_clean_scrub_stamp":"2024-04-25T10:30:11.733978+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T18:06:42.983247+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5],"acting":[4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.16","version":"0'0","reported_seq":14,"reported_epoch":36,"state":"active+clean","last_fresh":"2024-04-25T10:30:14.411211+0000","last_change":"2024-04-25T10:30:11.749473+0000","last_active":"2024-04-25T10:30:14.411211+0000","last_peered":"2024-04-25T10:30:14.411211+0000","last_clean":"2024-04-25T10:30:14.411211+0000","last_became_active":"2024-04-25T10:30:11.749135+0000","last_became_peered":"2024-04-25T10:30:11.749135+0000","last_unstale":"2024-04-25T10:30:14.411211+0000","last_undegraded":"2024-04-25T10:30:14.411211+0000","last_fullsized":"2024-04-25T10:30:14.411211+0000","mapping_epoch":33,"log_start":"0'0","ondisk_log_start":"0'0","created":33,"last_epoch_clean":34,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:30:10.729165+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:30:10.729165+0000","last_clean_scrub_stamp":"2024-04-25T10:30:10.729165+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T15:17:28.997853+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"5.14","version":"0'0","reported_seq":9,"reported_epoch":35,"state":"active+clean","last_fresh":"2024-04-25T10:30:12.778630+0000","last_change":"2024-04-25T10:30:12.770588+0000","last_active":"2024-04-25T10:30:12.778630+0000","last_peered":"2024-04-25T10:30:12.778630+0000","last_clean":"2024-04-25T10:30:12.778630+0000","last_became_active":"2024-04-25T10:30:12.770264+0000","last_became_peered":"2024-04-25T10:30:12.770264+0000","last_unstale":"2024-04-25T10:30:12.778630+0000","last_undegraded":"2024-04-25T10:30:12.778630+0000","last_fullsized":"2024-04-25T10:30:12.778630+0000","mapping_epoch":34,"log_start":"0'0","ondisk_log_start":"0'0","created":34,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:30:11.733978+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:30:11.733978+0000","last_clean_scrub_stamp":"2024-04-25T10:30:11.733978+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T18:06:36.650571+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.15","version":"35'3","reported_seq":17,"reported_epoch":36,"state":"active+clean","last_fresh":"2024-04-25T10:30:13.760573+0000","last_change":"2024-04-25T10:30:11.755959+0000","last_active":"2024-04-25T10:30:13.760573+0000","last_peered":"2024-04-25T10:30:13.760573+0000","last_clean":"2024-04-25T10:30:13.760573+0000","last_became_active":"2024-04-25T10:30:11.755603+0000","last_became_peered":"2024-04-25T10:30:11.755603+0000","last_unstale":"2024-04-25T10:30:13.760573+0000","last_undegraded":"2024-04-25T10:30:13.760573+0000","last_fullsized":"2024-04-25T10:30:13.760573+0000","mapping_epoch":33,"log_start":"0'0","ondisk_log_start":"0'0","created":33,"last_epoch_clean":34,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:30:10.729165+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:30:10.729165+0000","last_clean_scrub_stamp":"2024-04-25T10:30:10.729165+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T19:25:05.393017+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.15","version":"0'0","reported_seq":12,"reported_epoch":36,"state":"active+clean","last_fresh":"2024-04-25T10:30:13.760709+0000","last_change":"2024-04-25T10:30:12.776432+0000","last_active":"2024-04-25T10:30:13.760709+0000","last_peered":"2024-04-25T10:30:13.760709+0000","last_clean":"2024-04-25T10:30:13.760709+0000","last_became_active":"2024-04-25T10:30:12.775857+0000","last_became_peered":"2024-04-25T10:30:12.775857+0000","last_unstale":"2024-04-25T10:30:13.760709+0000","last_undegraded":"2024-04-25T10:30:13.760709+0000","last_fullsized":"2024-04-25T10:30:13.760709+0000","mapping_epoch":34,"log_start":"0'0","ondisk_log_start":"0'0","created":34,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:30:11.733978+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:30:11.733978+0000","last_clean_scrub_stamp":"2024-04-25T10:30:11.733978+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T14:37:28.612068+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.14","version":"35'6","reported_seq":20,"reported_epoch":36,"state":"active+clean","last_fresh":"2024-04-25T10:30:13.762316+0000","last_change":"2024-04-25T10:30:11.757323+0000","last_active":"2024-04-25T10:30:13.762316+0000","last_peered":"2024-04-25T10:30:13.762316+0000","last_clean":"2024-04-25T10:30:13.762316+0000","last_became_active":"2024-04-25T10:30:11.757011+0000","last_became_peered":"2024-04-25T10:30:11.757011+0000","last_unstale":"2024-04-25T10:30:13.762316+0000","last_undegraded":"2024-04-25T10:30:13.762316+0000","last_fullsized":"2024-04-25T10:30:13.762316+0000","mapping_epoch":33,"log_start":"0'0","ondisk_log_start":"0'0","created":33,"last_epoch_clean":34,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:30:10.729165+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:30:10.729165+0000","last_clean_scrub_stamp":"2024-04-25T10:30:10.729165+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T19:02:53.584992+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1612,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"5.12","version":"0'0","reported_seq":12,"reported_epoch":36,"state":"active+clean","last_fresh":"2024-04-25T10:30:13.761928+0000","last_change":"2024-04-25T10:30:12.770496+0000","last_active":"2024-04-25T10:30:13.761928+0000","last_peered":"2024-04-25T10:30:13.761928+0000","last_clean":"2024-04-25T10:30:13.761928+0000","last_became_active":"2024-04-25T10:30:12.770193+0000","last_became_peered":"2024-04-25T10:30:12.770193+0000","last_unstale":"2024-04-25T10:30:13.761928+0000","last_undegraded":"2024-04-25T10:30:13.761928+0000","last_fullsized":"2024-04-25T10:30:13.761928+0000","mapping_epoch":34,"log_start":"0'0","ondisk_log_start":"0'0","created":34,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:30:11.733978+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:30:11.733978+0000","last_clean_scrub_stamp":"2024-04-25T10:30:11.733978+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T15:17:30.002665+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,1],"acting":[6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.13","version":"35'2","reported_seq":16,"reported_epoch":36,"state":"active+clean","last_fresh":"2024-04-25T10:30:13.762908+0000","last_change":"2024-04-25T10:30:11.764993+0000","last_active":"2024-04-25T10:30:13.762908+0000","last_peered":"2024-04-25T10:30:13.762908+0000","last_clean":"2024-04-25T10:30:13.762908+0000","last_became_active":"2024-04-25T10:30:11.764637+0000","last_became_peered":"2024-04-25T10:30:11.764637+0000","last_unstale":"2024-04-25T10:30:13.762908+0000","last_undegraded":"2024-04-25T10:30:13.762908+0000","last_fullsized":"2024-04-25T10:30:13.762908+0000","mapping_epoch":33,"log_start":"0'0","ondisk_log_start":"0'0","created":33,"last_epoch_clean":34,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:30:10.729165+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:30:10.729165+0000","last_clean_scrub_stamp":"2024-04-25T10:30:10.729165+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T17:24:13.848319+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":34,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":7,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"5.13","version":"0'0","reported_seq":9,"reported_epoch":35,"state":"active+clean","last_fresh":"2024-04-25T10:30:12.779318+0000","last_change":"2024-04-25T10:30:12.775269+0000","last_active":"2024-04-25T10:30:12.779318+0000","last_peered":"2024-04-25T10:30:12.779318+0000","last_clean":"2024-04-25T10:30:12.779318+0000","last_became_active":"2024-04-25T10:30:12.774865+0000","last_became_peered":"2024-04-25T10:30:12.774865+0000","last_unstale":"2024-04-25T10:30:12.779318+0000","last_undegraded":"2024-04-25T10:30:12.779318+0000","last_fullsized":"2024-04-25T10:30:12.779318+0000","mapping_epoch":34,"log_start":"0'0","ondisk_log_start":"0'0","created":34,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:30:11.733978+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:30:11.733978+0000","last_clean_scrub_stamp":"2024-04-25T10:30:11.733978+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T16:21:52.033922+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.12","version":"0'0","reported_seq":12,"reported_epoch":35,"state":"active+clean","last_fresh":"2024-04-25T10:30:12.770611+0000","last_change":"2024-04-25T10:30:11.762094+0000","last_active":"2024-04-25T10:30:12.770611+0000","last_peered":"2024-04-25T10:30:12.770611+0000","last_clean":"2024-04-25T10:30:12.770611+0000","last_became_active":"2024-04-25T10:30:11.761456+0000","last_became_peered":"2024-04-25T10:30:11.761456+0000","last_unstale":"2024-04-25T10:30:12.770611+0000","last_undegraded":"2024-04-25T10:30:12.770611+0000","last_fullsized":"2024-04-25T10:30:12.770611+0000","mapping_epoch":33,"log_start":"0'0","ondisk_log_start":"0'0","created":33,"last_epoch_clean":34,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:30:10.729165+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:30:10.729165+0000","last_clean_scrub_stamp":"2024-04-25T10:30:10.729165+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T21:58:31.211023+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.10","version":"0'0","reported_seq":12,"reported_epoch":36,"state":"active+clean","last_fresh":"2024-04-25T10:30:13.762551+0000","last_change":"2024-04-25T10:30:12.783285+0000","last_active":"2024-04-25T10:30:13.762551+0000","last_peered":"2024-04-25T10:30:13.762551+0000","last_clean":"2024-04-25T10:30:13.762551+0000","last_became_active":"2024-04-25T10:30:12.782617+0000","last_became_peered":"2024-04-25T10:30:12.782617+0000","last_unstale":"2024-04-25T10:30:13.762551+0000","last_undegraded":"2024-04-25T10:30:13.762551+0000","last_fullsized":"2024-04-25T10:30:13.762551+0000","mapping_epoch":34,"log_start":"0'0","ondisk_log_start":"0'0","created":34,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:30:11.733978+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:30:11.733978+0000","last_clean_scrub_stamp":"2024-04-25T10:30:11.733978+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T20:14:15.241563+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.11","version":"0'0","reported_seq":12,"reported_epoch":35,"state":"active+clean","last_fresh":"2024-04-25T10:30:12.768436+0000","last_change":"2024-04-25T10:30:11.755706+0000","last_active":"2024-04-25T10:30:12.768436+0000","last_peered":"2024-04-25T10:30:12.768436+0000","last_clean":"2024-04-25T10:30:12.768436+0000","last_became_active":"2024-04-25T10:30:11.755348+0000","last_became_peered":"2024-04-25T10:30:11.755348+0000","last_unstale":"2024-04-25T10:30:12.768436+0000","last_undegraded":"2024-04-25T10:30:12.768436+0000","last_fullsized":"2024-04-25T10:30:12.768436+0000","mapping_epoch":33,"log_start":"0'0","ondisk_log_start":"0'0","created":33,"last_epoch_clean":34,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:30:10.729165+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:30:10.729165+0000","last_clean_scrub_stamp":"2024-04-25T10:30:10.729165+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T10:42:20.657474+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.11","version":"0'0","reported_seq":12,"reported_epoch":36,"state":"active+clean","last_fresh":"2024-04-25T10:30:13.760319+0000","last_change":"2024-04-25T10:30:12.777184+0000","last_active":"2024-04-25T10:30:13.760319+0000","last_peered":"2024-04-25T10:30:13.760319+0000","last_clean":"2024-04-25T10:30:13.760319+0000","last_became_active":"2024-04-25T10:30:12.776874+0000","last_became_peered":"2024-04-25T10:30:12.776874+0000","last_unstale":"2024-04-25T10:30:13.760319+0000","last_undegraded":"2024-04-25T10:30:13.760319+0000","last_fullsized":"2024-04-25T10:30:13.760319+0000","mapping_epoch":34,"log_start":"0'0","ondisk_log_start":"0'0","created":34,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:30:11.733978+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:30:11.733978+0000","last_clean_scrub_stamp":"2024-04-25T10:30:11.733978+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T11:57:29.979255+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.10","version":"35'8","reported_seq":22,"reported_epoch":36,"state":"active+clean","last_fresh":"2024-04-25T10:30:14.411032+0000","last_change":"2024-04-25T10:30:11.755358+0000","last_active":"2024-04-25T10:30:14.411032+0000","last_peered":"2024-04-25T10:30:14.411032+0000","last_clean":"2024-04-25T10:30:14.411032+0000","last_became_active":"2024-04-25T10:30:11.754992+0000","last_became_peered":"2024-04-25T10:30:11.754992+0000","last_unstale":"2024-04-25T10:30:14.411032+0000","last_undegraded":"2024-04-25T10:30:14.411032+0000","last_fullsized":"2024-04-25T10:30:14.411032+0000","mapping_epoch":33,"log_start":"0'0","ondisk_log_start":"0'0","created":33,"last_epoch_clean":34,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:30:10.729165+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:30:10.729165+0000","last_clean_scrub_stamp":"2024-04-25T10:30:10.729165+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T19:57:30.397817+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":12,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3],"acting":[7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"5.e","version":"0'0","reported_seq":12,"reported_epoch":36,"state":"active+clean","last_fresh":"2024-04-25T10:30:13.760700+0000","last_change":"2024-04-25T10:30:12.780850+0000","last_active":"2024-04-25T10:30:13.760700+0000","last_peered":"2024-04-25T10:30:13.760700+0000","last_clean":"2024-04-25T10:30:13.760700+0000","last_became_active":"2024-04-25T10:30:12.780187+0000","last_became_peered":"2024-04-25T10:30:12.780187+0000","last_unstale":"2024-04-25T10:30:13.760700+0000","last_undegraded":"2024-04-25T10:30:13.760700+0000","last_fullsized":"2024-04-25T10:30:13.760700+0000","mapping_epoch":34,"log_start":"0'0","ondisk_log_start":"0'0","created":34,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:30:11.733978+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:30:11.733978+0000","last_clean_scrub_stamp":"2024-04-25T10:30:11.733978+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T14:04:42.520921+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.f","version":"35'1","reported_seq":15,"reported_epoch":36,"state":"active+clean","last_fresh":"2024-04-25T10:30:14.411495+0000","last_change":"2024-04-25T10:30:11.750348+0000","last_active":"2024-04-25T10:30:14.411495+0000","last_peered":"2024-04-25T10:30:14.411495+0000","last_clean":"2024-04-25T10:30:14.411495+0000","last_became_active":"2024-04-25T10:30:11.749970+0000","last_became_peered":"2024-04-25T10:30:11.749970+0000","last_unstale":"2024-04-25T10:30:14.411495+0000","last_undegraded":"2024-04-25T10:30:14.411495+0000","last_fullsized":"2024-04-25T10:30:14.411495+0000","mapping_epoch":33,"log_start":"0'0","ondisk_log_start":"0'0","created":33,"last_epoch_clean":34,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:30:10.729165+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:30:10.729165+0000","last_clean_scrub_stamp":"2024-04-25T10:30:10.729165+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T20:41:22.574408+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"5.f","version":"0'0","reported_seq":12,"reported_epoch":36,"state":"active+clean","last_fresh":"2024-04-25T10:30:14.411347+0000","last_change":"2024-04-25T10:30:12.769270+0000","last_active":"2024-04-25T10:30:14.411347+0000","last_peered":"2024-04-25T10:30:14.411347+0000","last_clean":"2024-04-25T10:30:14.411347+0000","last_became_active":"2024-04-25T10:30:12.768931+0000","last_became_peered":"2024-04-25T10:30:12.768931+0000","last_unstale":"2024-04-25T10:30:14.411347+0000","last_undegraded":"2024-04-25T10:30:14.411347+0000","last_fullsized":"2024-04-25T10:30:14.411347+0000","mapping_epoch":34,"log_start":"0'0","ondisk_log_start":"0'0","created":34,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:30:11.733978+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:30:11.733978+0000","last_clean_scrub_stamp":"2024-04-25T10:30:11.733978+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T19:02:54.589804+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.e","version":"35'2","reported_seq":16,"reported_epoch":36,"state":"active+clean","last_fresh":"2024-04-25T10:30:13.762774+0000","last_change":"2024-04-25T10:30:11.765106+0000","last_active":"2024-04-25T10:30:13.762774+0000","last_peered":"2024-04-25T10:30:13.762774+0000","last_clean":"2024-04-25T10:30:13.762774+0000","last_became_active":"2024-04-25T10:30:11.764710+0000","last_became_peered":"2024-04-25T10:30:11.764710+0000","last_unstale":"2024-04-25T10:30:13.762774+0000","last_undegraded":"2024-04-25T10:30:13.762774+0000","last_fullsized":"2024-04-25T10:30:13.762774+0000","mapping_epoch":33,"log_start":"0'0","ondisk_log_start":"0'0","created":33,"last_epoch_clean":34,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:30:10.729165+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:30:10.729165+0000","last_clean_scrub_stamp":"2024-04-25T10:30:10.729165+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T16:30:11.709416+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"5.c","version":"0'0","reported_seq":12,"reported_epoch":36,"state":"active+clean","last_fresh":"2024-04-25T10:30:14.410924+0000","last_change":"2024-04-25T10:30:12.765790+0000","last_active":"2024-04-25T10:30:14.410924+0000","last_peered":"2024-04-25T10:30:14.410924+0000","last_clean":"2024-04-25T10:30:14.410924+0000","last_became_active":"2024-04-25T10:30:12.765468+0000","last_became_peered":"2024-04-25T10:30:12.765468+0000","last_unstale":"2024-04-25T10:30:14.410924+0000","last_undegraded":"2024-04-25T10:30:14.410924+0000","last_fullsized":"2024-04-25T10:30:14.410924+0000","mapping_epoch":34,"log_start":"0'0","ondisk_log_start":"0'0","created":34,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:30:11.733978+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:30:11.733978+0000","last_clean_scrub_stamp":"2024-04-25T10:30:11.733978+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T17:41:34.910195+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.d","version":"35'1","reported_seq":15,"reported_epoch":36,"state":"active+clean","last_fresh":"2024-04-25T10:30:13.760826+0000","last_change":"2024-04-25T10:30:11.758390+0000","last_active":"2024-04-25T10:30:13.760826+0000","last_peered":"2024-04-25T10:30:13.760826+0000","last_clean":"2024-04-25T10:30:13.760826+0000","last_became_active":"2024-04-25T10:30:11.757989+0000","last_became_peered":"2024-04-25T10:30:11.757989+0000","last_unstale":"2024-04-25T10:30:13.760826+0000","last_undegraded":"2024-04-25T10:30:13.760826+0000","last_fullsized":"2024-04-25T10:30:13.760826+0000","mapping_epoch":33,"log_start":"0'0","ondisk_log_start":"0'0","created":33,"last_epoch_clean":34,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:30:10.729165+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:30:10.729165+0000","last_clean_scrub_stamp":"2024-04-25T10:30:10.729165+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T18:52:44.534946+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":70,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.d","version":"0'0","reported_seq":12,"reported_epoch":36,"state":"active+clean","last_fresh":"2024-04-25T10:30:14.411240+0000","last_change":"2024-04-25T10:30:12.771743+0000","last_active":"2024-04-25T10:30:14.411240+0000","last_peered":"2024-04-25T10:30:14.411240+0000","last_clean":"2024-04-25T10:30:14.411240+0000","last_became_active":"2024-04-25T10:30:12.771387+0000","last_became_peered":"2024-04-25T10:30:12.771387+0000","last_unstale":"2024-04-25T10:30:14.411240+0000","last_undegraded":"2024-04-25T10:30:14.411240+0000","last_fullsized":"2024-04-25T10:30:14.411240+0000","mapping_epoch":34,"log_start":"0'0","ondisk_log_start":"0'0","created":34,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:30:11.733978+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:30:11.733978+0000","last_clean_scrub_stamp":"2024-04-25T10:30:11.733978+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T19:02:54.589804+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.c","version":"0'0","reported_seq":14,"reported_epoch":36,"state":"active+clean","last_fresh":"2024-04-25T10:30:14.195574+0000","last_change":"2024-04-25T10:30:11.753067+0000","last_active":"2024-04-25T10:30:14.195574+0000","last_peered":"2024-04-25T10:30:14.195574+0000","last_clean":"2024-04-25T10:30:14.195574+0000","last_became_active":"2024-04-25T10:30:11.752595+0000","last_became_peered":"2024-04-25T10:30:11.752595+0000","last_unstale":"2024-04-25T10:30:14.195574+0000","last_undegraded":"2024-04-25T10:30:14.195574+0000","last_fullsized":"2024-04-25T10:30:14.195574+0000","mapping_epoch":33,"log_start":"0'0","ondisk_log_start":"0'0","created":33,"last_epoch_clean":34,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:30:10.729165+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:30:10.729165+0000","last_clean_scrub_stamp":"2024-04-25T10:30:10.729165+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T18:59:19.688948+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"5.a","version":"0'0","reported_seq":12,"reported_epoch":36,"state":"active+clean","last_fresh":"2024-04-25T10:30:13.761924+0000","last_change":"2024-04-25T10:30:12.762940+0000","last_active":"2024-04-25T10:30:13.761924+0000","last_peered":"2024-04-25T10:30:13.761924+0000","last_clean":"2024-04-25T10:30:13.761924+0000","last_became_active":"2024-04-25T10:30:12.762628+0000","last_became_peered":"2024-04-25T10:30:12.762628+0000","last_unstale":"2024-04-25T10:30:13.761924+0000","last_undegraded":"2024-04-25T10:30:13.761924+0000","last_fullsized":"2024-04-25T10:30:13.761924+0000","mapping_epoch":34,"log_start":"0'0","ondisk_log_start":"0'0","created":34,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:30:11.733978+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:30:11.733978+0000","last_clean_scrub_stamp":"2024-04-25T10:30:11.733978+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T14:01:06.791796+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.b","version":"35'2","reported_seq":16,"reported_epoch":36,"state":"active+clean","last_fresh":"2024-04-25T10:30:14.411713+0000","last_change":"2024-04-25T10:30:11.752902+0000","last_active":"2024-04-25T10:30:14.411713+0000","last_peered":"2024-04-25T10:30:14.411713+0000","last_clean":"2024-04-25T10:30:14.411713+0000","last_became_active":"2024-04-25T10:30:11.752599+0000","last_became_peered":"2024-04-25T10:30:11.752599+0000","last_unstale":"2024-04-25T10:30:14.411713+0000","last_undegraded":"2024-04-25T10:30:14.411713+0000","last_fullsized":"2024-04-25T10:30:14.411713+0000","mapping_epoch":33,"log_start":"0'0","ondisk_log_start":"0'0","created":33,"last_epoch_clean":34,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:30:10.729165+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:30:10.729165+0000","last_clean_scrub_stamp":"2024-04-25T10:30:10.729165+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T14:01:05.786983+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"5.b","version":"0'0","reported_seq":12,"reported_epoch":36,"state":"active+clean","last_fresh":"2024-04-25T10:30:14.411684+0000","last_change":"2024-04-25T10:30:12.766544+0000","last_active":"2024-04-25T10:30:14.411684+0000","last_peered":"2024-04-25T10:30:14.411684+0000","last_clean":"2024-04-25T10:30:14.411684+0000","last_became_active":"2024-04-25T10:30:12.766228+0000","last_became_peered":"2024-04-25T10:30:12.766228+0000","last_unstale":"2024-04-25T10:30:14.411684+0000","last_undegraded":"2024-04-25T10:30:14.411684+0000","last_fullsized":"2024-04-25T10:30:14.411684+0000","mapping_epoch":34,"log_start":"0'0","ondisk_log_start":"0'0","created":34,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:30:11.733978+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:30:11.733978+0000","last_clean_scrub_stamp":"2024-04-25T10:30:11.733978+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T18:21:32.178524+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.a","version":"0'0","reported_seq":14,"reported_epoch":36,"state":"active+clean","last_fresh":"2024-04-25T10:30:13.762599+0000","last_change":"2024-04-25T10:30:11.762745+0000","last_active":"2024-04-25T10:30:13.762599+0000","last_peered":"2024-04-25T10:30:13.762599+0000","last_clean":"2024-04-25T10:30:13.762599+0000","last_became_active":"2024-04-25T10:30:11.762377+0000","last_became_peered":"2024-04-25T10:30:11.762377+0000","last_unstale":"2024-04-25T10:30:13.762599+0000","last_undegraded":"2024-04-25T10:30:13.762599+0000","last_fullsized":"2024-04-25T10:30:13.762599+0000","mapping_epoch":33,"log_start":"0'0","ondisk_log_start":"0'0","created":33,"last_epoch_clean":34,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:30:10.729165+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:30:10.729165+0000","last_clean_scrub_stamp":"2024-04-25T10:30:10.729165+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T17:15:13.597075+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"5.8","version":"0'0","reported_seq":12,"reported_epoch":36,"state":"active+clean","last_fresh":"2024-04-25T10:30:13.761619+0000","last_change":"2024-04-25T10:30:12.777060+0000","last_active":"2024-04-25T10:30:13.761619+0000","last_peered":"2024-04-25T10:30:13.761619+0000","last_clean":"2024-04-25T10:30:13.761619+0000","last_became_active":"2024-04-25T10:30:12.776749+0000","last_became_peered":"2024-04-25T10:30:12.776749+0000","last_unstale":"2024-04-25T10:30:13.761619+0000","last_undegraded":"2024-04-25T10:30:13.761619+0000","last_fullsized":"2024-04-25T10:30:13.761619+0000","mapping_epoch":34,"log_start":"0'0","ondisk_log_start":"0'0","created":34,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:30:11.733978+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:30:11.733978+0000","last_clean_scrub_stamp":"2024-04-25T10:30:11.733978+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T18:31:49.260513+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.9","version":"0'0","reported_seq":14,"reported_epoch":36,"state":"active+clean","last_fresh":"2024-04-25T10:30:14.411012+0000","last_change":"2024-04-25T10:30:11.759807+0000","last_active":"2024-04-25T10:30:14.411012+0000","last_peered":"2024-04-25T10:30:14.411012+0000","last_clean":"2024-04-25T10:30:14.411012+0000","last_became_active":"2024-04-25T10:30:11.758558+0000","last_became_peered":"2024-04-25T10:30:11.758558+0000","last_unstale":"2024-04-25T10:30:14.411012+0000","last_undegraded":"2024-04-25T10:30:14.411012+0000","last_fullsized":"2024-04-25T10:30:14.411012+0000","mapping_epoch":33,"log_start":"0'0","ondisk_log_start":"0'0","created":33,"last_epoch_clean":34,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:30:10.729165+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:30:10.729165+0000","last_clean_scrub_stamp":"2024-04-25T10:30:10.729165+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T20:38:23.460836+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"5.9","version":"0'0","reported_seq":12,"reported_epoch":36,"state":"active+clean","last_fresh":"2024-04-25T10:30:14.411216+0000","last_change":"2024-04-25T10:30:12.768782+0000","last_active":"2024-04-25T10:30:14.411216+0000","last_peered":"2024-04-25T10:30:14.411216+0000","last_clean":"2024-04-25T10:30:14.411216+0000","last_became_active":"2024-04-25T10:30:12.768464+0000","last_became_peered":"2024-04-25T10:30:12.768464+0000","last_unstale":"2024-04-25T10:30:14.411216+0000","last_undegraded":"2024-04-25T10:30:14.411216+0000","last_fullsized":"2024-04-25T10:30:14.411216+0000","mapping_epoch":34,"log_start":"0'0","ondisk_log_start":"0'0","created":34,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:30:11.733978+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:30:11.733978+0000","last_clean_scrub_stamp":"2024-04-25T10:30:11.733978+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T17:15:14.601888+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.8","version":"0'0","reported_seq":14,"reported_epoch":36,"state":"active+clean","last_fresh":"2024-04-25T10:30:13.760291+0000","last_change":"2024-04-25T10:30:11.758581+0000","last_active":"2024-04-25T10:30:13.760291+0000","last_peered":"2024-04-25T10:30:13.760291+0000","last_clean":"2024-04-25T10:30:13.760291+0000","last_became_active":"2024-04-25T10:30:11.758239+0000","last_became_peered":"2024-04-25T10:30:11.758239+0000","last_unstale":"2024-04-25T10:30:13.760291+0000","last_undegraded":"2024-04-25T10:30:13.760291+0000","last_fullsized":"2024-04-25T10:30:13.760291+0000","mapping_epoch":33,"log_start":"0'0","ondisk_log_start":"0'0","created":33,"last_epoch_clean":34,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:30:10.729165+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:30:10.729165+0000","last_clean_scrub_stamp":"2024-04-25T10:30:10.729165+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T17:41:33.905382+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.6","version":"0'0","reported_seq":12,"reported_epoch":36,"state":"active+clean","last_fresh":"2024-04-25T10:30:14.411468+0000","last_change":"2024-04-25T10:30:12.772456+0000","last_active":"2024-04-25T10:30:14.411468+0000","last_peered":"2024-04-25T10:30:14.411468+0000","last_clean":"2024-04-25T10:30:14.411468+0000","last_became_active":"2024-04-25T10:30:12.772070+0000","last_became_peered":"2024-04-25T10:30:12.772070+0000","last_unstale":"2024-04-25T10:30:14.411468+0000","last_undegraded":"2024-04-25T10:30:14.411468+0000","last_fullsized":"2024-04-25T10:30:14.411468+0000","mapping_epoch":34,"log_start":"0'0","ondisk_log_start":"0'0","created":34,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:30:11.733978+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:30:11.733978+0000","last_clean_scrub_stamp":"2024-04-25T10:30:11.733978+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T16:09:46.519973+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.7","version":"35'2","reported_seq":16,"reported_epoch":36,"state":"active+clean","last_fresh":"2024-04-25T10:30:13.761834+0000","last_change":"2024-04-25T10:30:11.756948+0000","last_active":"2024-04-25T10:30:13.761834+0000","last_peered":"2024-04-25T10:30:13.761834+0000","last_clean":"2024-04-25T10:30:13.761834+0000","last_became_active":"2024-04-25T10:30:11.756607+0000","last_became_peered":"2024-04-25T10:30:11.756607+0000","last_unstale":"2024-04-25T10:30:13.761834+0000","last_undegraded":"2024-04-25T10:30:13.761834+0000","last_fullsized":"2024-04-25T10:30:13.761834+0000","mapping_epoch":33,"log_start":"0'0","ondisk_log_start":"0'0","created":33,"last_epoch_clean":34,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:30:10.729165+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:30:10.729165+0000","last_clean_scrub_stamp":"2024-04-25T10:30:10.729165+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T17:15:13.597075+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"5.7","version":"0'0","reported_seq":12,"reported_epoch":36,"state":"active+clean","last_fresh":"2024-04-25T10:30:13.760659+0000","last_change":"2024-04-25T10:30:12.768932+0000","last_active":"2024-04-25T10:30:13.760659+0000","last_peered":"2024-04-25T10:30:13.760659+0000","last_clean":"2024-04-25T10:30:13.760659+0000","last_became_active":"2024-04-25T10:30:12.768609+0000","last_became_peered":"2024-04-25T10:30:12.768609+0000","last_unstale":"2024-04-25T10:30:13.760659+0000","last_undegraded":"2024-04-25T10:30:13.760659+0000","last_fullsized":"2024-04-25T10:30:13.760659+0000","mapping_epoch":34,"log_start":"0'0","ondisk_log_start":"0'0","created":34,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:30:11.733978+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:30:11.733978+0000","last_clean_scrub_stamp":"2024-04-25T10:30:11.733978+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T18:35:51.085554+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.6","version":"36'1","reported_seq":15,"reported_epoch":36,"state":"active+clean","last_fresh":"2024-04-25T10:30:14.765515+0000","last_change":"2024-04-25T10:30:11.750312+0000","last_active":"2024-04-25T10:30:14.765515+0000","last_peered":"2024-04-25T10:30:14.765515+0000","last_clean":"2024-04-25T10:30:14.765515+0000","last_became_active":"2024-04-25T10:30:11.750024+0000","last_became_peered":"2024-04-25T10:30:11.750024+0000","last_unstale":"2024-04-25T10:30:14.765515+0000","last_undegraded":"2024-04-25T10:30:14.765515+0000","last_fullsized":"2024-04-25T10:30:14.765515+0000","mapping_epoch":33,"log_start":"0'0","ondisk_log_start":"0'0","created":33,"last_epoch_clean":34,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:30:10.729165+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:30:10.729165+0000","last_clean_scrub_stamp":"2024-04-25T10:30:10.729165+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T16:48:59.263610+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"5.5","version":"0'0","reported_seq":12,"reported_epoch":36,"state":"active+clean","last_fresh":"2024-04-25T10:30:14.411435+0000","last_change":"2024-04-25T10:30:12.782521+0000","last_active":"2024-04-25T10:30:14.411435+0000","last_peered":"2024-04-25T10:30:14.411435+0000","last_clean":"2024-04-25T10:30:14.411435+0000","last_became_active":"2024-04-25T10:30:12.781802+0000","last_became_peered":"2024-04-25T10:30:12.781802+0000","last_unstale":"2024-04-25T10:30:14.411435+0000","last_undegraded":"2024-04-25T10:30:14.411435+0000","last_fullsized":"2024-04-25T10:30:14.411435+0000","mapping_epoch":34,"log_start":"0'0","ondisk_log_start":"0'0","created":34,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:30:11.733978+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:30:11.733978+0000","last_clean_scrub_stamp":"2024-04-25T10:30:11.733978+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T13:03:09.680861+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.4","version":"0'0","reported_seq":14,"reported_epoch":36,"state":"active+clean","last_fresh":"2024-04-25T10:30:13.762316+0000","last_change":"2024-04-25T10:30:11.757601+0000","last_active":"2024-04-25T10:30:13.762316+0000","last_peered":"2024-04-25T10:30:13.762316+0000","last_clean":"2024-04-25T10:30:13.762316+0000","last_became_active":"2024-04-25T10:30:11.756626+0000","last_became_peered":"2024-04-25T10:30:11.756626+0000","last_unstale":"2024-04-25T10:30:13.762316+0000","last_undegraded":"2024-04-25T10:30:13.762316+0000","last_fullsized":"2024-04-25T10:30:13.762316+0000","mapping_epoch":33,"log_start":"0'0","ondisk_log_start":"0'0","created":33,"last_epoch_clean":34,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:30:10.729165+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:30:10.729165+0000","last_clean_scrub_stamp":"2024-04-25T10:30:10.729165+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T13:03:08.676049+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"5.2","version":"0'0","reported_seq":12,"reported_epoch":36,"state":"active+clean","last_fresh":"2024-04-25T10:30:14.195648+0000","last_change":"2024-04-25T10:30:12.765903+0000","last_active":"2024-04-25T10:30:14.195648+0000","last_peered":"2024-04-25T10:30:14.195648+0000","last_clean":"2024-04-25T10:30:14.195648+0000","last_became_active":"2024-04-25T10:30:12.765611+0000","last_became_peered":"2024-04-25T10:30:12.765611+0000","last_unstale":"2024-04-25T10:30:14.195648+0000","last_undegraded":"2024-04-25T10:30:14.195648+0000","last_fullsized":"2024-04-25T10:30:14.195648+0000","mapping_epoch":34,"log_start":"0'0","ondisk_log_start":"0'0","created":34,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:30:11.733978+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:30:11.733978+0000","last_clean_scrub_stamp":"2024-04-25T10:30:11.733978+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T18:52:45.539758+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.3","version":"0'0","reported_seq":14,"reported_epoch":36,"state":"active+clean","last_fresh":"2024-04-25T10:30:13.762155+0000","last_change":"2024-04-25T10:30:11.762286+0000","last_active":"2024-04-25T10:30:13.762155+0000","last_peered":"2024-04-25T10:30:13.762155+0000","last_clean":"2024-04-25T10:30:13.762155+0000","last_became_active":"2024-04-25T10:30:11.761975+0000","last_became_peered":"2024-04-25T10:30:11.761975+0000","last_unstale":"2024-04-25T10:30:13.762155+0000","last_undegraded":"2024-04-25T10:30:13.762155+0000","last_fullsized":"2024-04-25T10:30:13.762155+0000","mapping_epoch":33,"log_start":"0'0","ondisk_log_start":"0'0","created":33,"last_epoch_clean":34,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:30:10.729165+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:30:10.729165+0000","last_clean_scrub_stamp":"2024-04-25T10:30:10.729165+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T16:09:04.081750+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"5.3","version":"0'0","reported_seq":9,"reported_epoch":35,"state":"active+clean","last_fresh":"2024-04-25T10:30:12.779343+0000","last_change":"2024-04-25T10:30:12.775959+0000","last_active":"2024-04-25T10:30:12.779343+0000","last_peered":"2024-04-25T10:30:12.779343+0000","last_clean":"2024-04-25T10:30:12.779343+0000","last_became_active":"2024-04-25T10:30:12.774532+0000","last_became_peered":"2024-04-25T10:30:12.774532+0000","last_unstale":"2024-04-25T10:30:12.779343+0000","last_undegraded":"2024-04-25T10:30:12.779343+0000","last_fullsized":"2024-04-25T10:30:12.779343+0000","mapping_epoch":34,"log_start":"0'0","ondisk_log_start":"0'0","created":34,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:30:11.733978+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:30:11.733978+0000","last_clean_scrub_stamp":"2024-04-25T10:30:11.733978+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T16:18:26.764587+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.2","version":"0'0","reported_seq":14,"reported_epoch":36,"state":"active+clean","last_fresh":"2024-04-25T10:30:13.762069+0000","last_change":"2024-04-25T10:30:11.760769+0000","last_active":"2024-04-25T10:30:13.762069+0000","last_peered":"2024-04-25T10:30:13.762069+0000","last_clean":"2024-04-25T10:30:13.762069+0000","last_became_active":"2024-04-25T10:30:11.760425+0000","last_became_peered":"2024-04-25T10:30:11.760425+0000","last_unstale":"2024-04-25T10:30:13.762069+0000","last_undegraded":"2024-04-25T10:30:13.762069+0000","last_fullsized":"2024-04-25T10:30:13.762069+0000","mapping_epoch":33,"log_start":"0'0","ondisk_log_start":"0'0","created":33,"last_epoch_clean":34,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:30:10.729165+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:30:10.729165+0000","last_clean_scrub_stamp":"2024-04-25T10:30:10.729165+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T17:24:13.848319+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"5.0","version":"0'0","reported_seq":9,"reported_epoch":35,"state":"active+clean","last_fresh":"2024-04-25T10:30:12.787795+0000","last_change":"2024-04-25T10:30:12.781642+0000","last_active":"2024-04-25T10:30:12.787795+0000","last_peered":"2024-04-25T10:30:12.787795+0000","last_clean":"2024-04-25T10:30:12.787795+0000","last_became_active":"2024-04-25T10:30:12.781303+0000","last_became_peered":"2024-04-25T10:30:12.781303+0000","last_unstale":"2024-04-25T10:30:12.787795+0000","last_undegraded":"2024-04-25T10:30:12.787795+0000","last_fullsized":"2024-04-25T10:30:12.787795+0000","mapping_epoch":34,"log_start":"0'0","ondisk_log_start":"0'0","created":34,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:30:11.733978+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:30:11.733978+0000","last_clean_scrub_stamp":"2024-04-25T10:30:11.733978+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T22:06:18.432116+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.1","version":"0'0","reported_seq":14,"reported_epoch":36,"state":"active+clean","last_fresh":"2024-04-25T10:30:13.762656+0000","last_change":"2024-04-25T10:30:11.765156+0000","last_active":"2024-04-25T10:30:13.762656+0000","last_peered":"2024-04-25T10:30:13.762656+0000","last_clean":"2024-04-25T10:30:13.762656+0000","last_became_active":"2024-04-25T10:30:11.764704+0000","last_became_peered":"2024-04-25T10:30:11.764704+0000","last_unstale":"2024-04-25T10:30:13.762656+0000","last_undegraded":"2024-04-25T10:30:13.762656+0000","last_fullsized":"2024-04-25T10:30:13.762656+0000","mapping_epoch":33,"log_start":"0'0","ondisk_log_start":"0'0","created":33,"last_epoch_clean":34,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:30:10.729165+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:30:10.729165+0000","last_clean_scrub_stamp":"2024-04-25T10:30:10.729165+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T19:02:53.584992+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"5.1","version":"0'0","reported_seq":12,"reported_epoch":36,"state":"active+clean","last_fresh":"2024-04-25T10:30:13.760478+0000","last_change":"2024-04-25T10:30:12.769006+0000","last_active":"2024-04-25T10:30:13.760478+0000","last_peered":"2024-04-25T10:30:13.760478+0000","last_clean":"2024-04-25T10:30:13.760478+0000","last_became_active":"2024-04-25T10:30:12.768671+0000","last_became_peered":"2024-04-25T10:30:12.768671+0000","last_unstale":"2024-04-25T10:30:13.760478+0000","last_undegraded":"2024-04-25T10:30:13.760478+0000","last_fullsized":"2024-04-25T10:30:13.760478+0000","mapping_epoch":34,"log_start":"0'0","ondisk_log_start":"0'0","created":34,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:30:11.733978+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:30:11.733978+0000","last_clean_scrub_stamp":"2024-04-25T10:30:11.733978+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T15:39:31.890064+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.0","version":"0'0","reported_seq":12,"reported_epoch":35,"state":"active+clean","last_fresh":"2024-04-25T10:30:12.768988+0000","last_change":"2024-04-25T10:30:11.761764+0000","last_active":"2024-04-25T10:30:12.768988+0000","last_peered":"2024-04-25T10:30:12.768988+0000","last_clean":"2024-04-25T10:30:12.768988+0000","last_became_active":"2024-04-25T10:30:11.761281+0000","last_became_peered":"2024-04-25T10:30:11.761281+0000","last_unstale":"2024-04-25T10:30:12.768988+0000","last_undegraded":"2024-04-25T10:30:12.768988+0000","last_fullsized":"2024-04-25T10:30:12.768988+0000","mapping_epoch":33,"log_start":"0'0","ondisk_log_start":"0'0","created":33,"last_epoch_clean":34,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:30:10.729165+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:30:10.729165+0000","last_clean_scrub_stamp":"2024-04-25T10:30:10.729165+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T13:05:33.012421+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.4","version":"0'0","reported_seq":12,"reported_epoch":36,"state":"active+clean","last_fresh":"2024-04-25T10:30:14.195540+0000","last_change":"2024-04-25T10:30:12.773161+0000","last_active":"2024-04-25T10:30:14.195540+0000","last_peered":"2024-04-25T10:30:14.195540+0000","last_clean":"2024-04-25T10:30:14.195540+0000","last_became_active":"2024-04-25T10:30:12.772831+0000","last_became_peered":"2024-04-25T10:30:12.772831+0000","last_unstale":"2024-04-25T10:30:14.195540+0000","last_undegraded":"2024-04-25T10:30:14.195540+0000","last_fullsized":"2024-04-25T10:30:14.195540+0000","mapping_epoch":34,"log_start":"0'0","ondisk_log_start":"0'0","created":34,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:30:11.733978+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:30:11.733978+0000","last_clean_scrub_stamp":"2024-04-25T10:30:11.733978+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T19:25:06.397830+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.5","version":"0'0","reported_seq":14,"reported_epoch":36,"state":"active+clean","last_fresh":"2024-04-25T10:30:13.760838+0000","last_change":"2024-04-25T10:30:11.759459+0000","last_active":"2024-04-25T10:30:13.760838+0000","last_peered":"2024-04-25T10:30:13.760838+0000","last_clean":"2024-04-25T10:30:13.760838+0000","last_became_active":"2024-04-25T10:30:11.757880+0000","last_became_peered":"2024-04-25T10:30:11.757880+0000","last_unstale":"2024-04-25T10:30:13.760838+0000","last_undegraded":"2024-04-25T10:30:13.760838+0000","last_fullsized":"2024-04-25T10:30:13.760838+0000","mapping_epoch":33,"log_start":"0'0","ondisk_log_start":"0'0","created":33,"last_epoch_clean":34,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:30:10.729165+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:30:10.729165+0000","last_clean_scrub_stamp":"2024-04-25T10:30:10.729165+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T18:21:31.173712+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.0","version":"18'72","reported_seq":133,"reported_epoch":36,"state":"active+clean","last_fresh":"2024-04-25T10:30:14.410873+0000","last_change":"2024-04-25T10:28:57.972696+0000","last_active":"2024-04-25T10:30:14.410873+0000","last_peered":"2024-04-25T10:30:14.410873+0000","last_clean":"2024-04-25T10:30:14.410873+0000","last_became_active":"2024-04-25T10:28:57.972373+0000","last_became_peered":"2024-04-25T10:28:57.972373+0000","last_unstale":"2024-04-25T10:30:14.410873+0000","last_undegraded":"2024-04-25T10:30:14.410873+0000","last_fullsized":"2024-04-25T10:30:14.410873+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:28:56.959439+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:28:56.959439+0000","last_clean_scrub_stamp":"2024-04-25T10:28:56.959439+0000","objects_scrubbed":0,"log_size":72,"log_dups_size":0,"ondisk_log_size":72,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T17:34:34.057408+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":66,"num_read_kb":55,"num_write":101,"num_write_kb":1628,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]}],"pool_stats":[{"poolid":5,"num_pg":64,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":128,"acting":128,"num_store_stats":8},{"poolid":4,"num_pg":32,"stat_sum":{"num_bytes":2410,"num_objects":22,"num_object_clones":0,"num_object_copies":44,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":22,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":45,"num_write_kb":14,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":14,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":65536,"data_stored":4820,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":39,"ondisk_log_size":39,"up":64,"acting":64,"num_store_stats":8},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":66,"num_read_kb":55,"num_write":101,"num_write_kb":1628,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1187840,"data_stored":1180736,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":72,"ondisk_log_size":72,"up":2,"acting":2,"num_store_stats":2}],"osd_stats":[{"osd":7,"up_from":15,"seq":64424509459,"num_pgs":21,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27424,"kb_used_data":396,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344416,"statfs":{"total":96636764160,"available":96608681984,"internally_reserved":0,"allocated":405504,"data_stored":156445,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1590,"internal_metadata":27654602},"hb_peers":[0,1,2,3,4,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[]},{"osd":6,"up_from":15,"seq":64424509460,"num_pgs":25,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27432,"kb_used_data":404,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344408,"statfs":{"total":96636764160,"available":96608673792,"internally_reserved":0,"allocated":413696,"data_stored":158035,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":27654598},"hb_peers":[0,1,2,3,4,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[]},{"osd":5,"up_from":15,"seq":64424509461,"num_pgs":14,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":28004,"kb_used_data":976,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94343836,"statfs":{"total":96636764160,"available":96608088064,"internally_reserved":0,"allocated":999424,"data_stored":747373,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]},{"osd":3,"up_from":15,"seq":64424509461,"num_pgs":25,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27436,"kb_used_data":408,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344404,"statfs":{"total":96636764160,"available":96608669696,"internally_reserved":0,"allocated":417792,"data_stored":157199,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1593,"internal_metadata":27654599},"hb_peers":[0,1,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[]},{"osd":2,"up_from":15,"seq":64424509461,"num_pgs":18,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27424,"kb_used_data":396,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344416,"statfs":{"total":96636764160,"available":96608681984,"internally_reserved":0,"allocated":405504,"data_stored":156457,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":5,"apply_latency_ms":5,"commit_latency_ns":5000000,"apply_latency_ns":5000000},"alerts":[]},{"osd":1,"up_from":15,"seq":64424509459,"num_pgs":2,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":28004,"kb_used_data":976,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94343836,"statfs":{"total":96636764160,"available":96608088064,"internally_reserved":0,"allocated":999424,"data_stored":746861,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":5,"apply_latency_ms":5,"commit_latency_ns":5000000,"apply_latency_ns":5000000},"alerts":[]},{"osd":0,"up_from":15,"seq":64424509460,"num_pgs":21,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27424,"kb_used_data":396,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344416,"statfs":{"total":96636764160,"available":96608681984,"internally_reserved":0,"allocated":405504,"data_stored":156513,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[1,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[]},{"osd":4,"up_from":13,"seq":55834574867,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27424,"kb_used_data":396,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344416,"statfs":{"total":96636764160,"available":96608681984,"internally_reserved":0,"allocated":405504,"data_stored":151757,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":90,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":70,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":34,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":16384,"data_stored":776,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":16384,"data_stored":1634,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":582,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":1612,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":22,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-04-25T10:30:17.197 INFO:tasks.cephfs.filesystem.ceph_manager:clean! 2024-04-25T10:30:17.198 DEBUG:teuthology.orchestra.run.smithi037:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph auth caps client.0 mon 'allow r' osd 'allow rw tag cephfs data=cephfs' mds allow 2024-04-25T10:30:17.352 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:17.350+0000 7f4634420640 1 Processor -- start 2024-04-25T10:30:17.353 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:17.350+0000 7f4634420640 1 -- start start 2024-04-25T10:30:17.355 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:17.352+0000 7f4634420640 1 --2- >> v2:172.21.15.37:3300/0 conn(0x7f462c0d1ec0 0x7f462c0d22d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:30:17.355 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:17.352+0000 7f4634420640 1 --2- >> v2:172.21.15.203:3300/0 conn(0x7f462c0d2810 0x7f462c0b3c80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:30:17.355 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:17.352+0000 7f4634420640 1 -- --> v1:172.21.15.203:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f462c09ce70 con 0x7f462c06e260 2024-04-25T10:30:17.355 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:17.352+0000 7f4634420640 1 -- --> v2:172.21.15.37:3300/0 -- mon_getmap magic: 0 -- 0x7f462c0d3370 con 0x7f462c0d1ec0 2024-04-25T10:30:17.355 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:17.352+0000 7f4634420640 1 -- --> v2:172.21.15.203:3300/0 -- mon_getmap magic: 0 -- 0x7f462c0d3090 con 0x7f462c0d2810 2024-04-25T10:30:17.355 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:17.352+0000 7f46319bb640 1 --2- >> v2:172.21.15.37:3300/0 conn(0x7f462c0d1ec0 0x7f462c0d22d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:30:17.355 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:17.352+0000 7f46319bb640 1 --2- >> v2:172.21.15.37:3300/0 conn(0x7f462c0d1ec0 0x7f462c0d22d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.37:3300/0 says I am v2:172.21.15.37:52742/0 (socket says 172.21.15.37:52742) 2024-04-25T10:30:17.355 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:17.352+0000 7f46319bb640 1 -- 172.21.15.37:0/1660587795 learned_addr learned my addr 172.21.15.37:0/1660587795 (peer_addr_for_me v2:172.21.15.37:0/0) 2024-04-25T10:30:17.355 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:17.352+0000 7f46329bd640 1 --2- 172.21.15.37:0/1660587795 >> v2:172.21.15.203:3300/0 conn(0x7f462c0d2810 0x7f462c0b3c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:30:17.355 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:17.352+0000 7f46319bb640 1 -- 172.21.15.37:0/1660587795 >> v2:172.21.15.203:3300/0 conn(0x7f462c0d2810 msgr2=0x7f462c0b3c80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:30:17.356 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:17.352+0000 7f46319bb640 1 --2- 172.21.15.37:0/1660587795 >> v2:172.21.15.203:3300/0 conn(0x7f462c0d2810 0x7f462c0b3c80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:30:17.356 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:17.352+0000 7f46319bb640 1 -- 172.21.15.37:0/1660587795 >> v1:172.21.15.203:6789/0 conn(0x7f462c06e260 legacy=0x7f462c06e670 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:30:17.356 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:17.352+0000 7f46319bb640 1 -- 172.21.15.37:0/1660587795 --> v2:172.21.15.37:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f462c09d150 con 0x7f462c0d1ec0 2024-04-25T10:30:17.356 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:17.353+0000 7f46319bb640 1 --2- 172.21.15.37:0/1660587795 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f462c0d1ec0 0x7f462c0d22d0 secure :-1 s=READY pgs=439 cs=0 l=1 rev1=1 crypto rx=0x7f462c08cfc0 tx=0x7f462002a1e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f92d276322391efe server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:30:17.361 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:17.353+0000 7f46311ba640 1 -- 172.21.15.37:0/1660587795 <== mon.0 v2:172.21.15.37:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4620037070 con 0x7f462c0d1ec0 2024-04-25T10:30:17.361 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:17.353+0000 7f46311ba640 1 -- 172.21.15.37:0/1660587795 <== mon.0 v2:172.21.15.37:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f4620032440 con 0x7f462c0d1ec0 2024-04-25T10:30:17.361 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:17.353+0000 7f46311ba640 1 -- 172.21.15.37:0/1660587795 <== mon.0 v2:172.21.15.37:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4620032810 con 0x7f462c0d1ec0 2024-04-25T10:30:17.361 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:17.354+0000 7f4634420640 1 -- 172.21.15.37:0/1660587795 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f462c0d1ec0 msgr2=0x7f462c0d22d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:30:17.361 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:17.354+0000 7f4634420640 1 --2- 172.21.15.37:0/1660587795 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f462c0d1ec0 0x7f462c0d22d0 secure :-1 s=READY pgs=439 cs=0 l=1 rev1=1 crypto rx=0x7f462c08cfc0 tx=0x7f462002a1e0 comp rx=0 tx=0).stop 2024-04-25T10:30:17.361 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:17.354+0000 7f4634420640 1 -- 172.21.15.37:0/1660587795 shutdown_connections 2024-04-25T10:30:17.361 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:17.354+0000 7f4634420640 1 --2- 172.21.15.37:0/1660587795 >> v2:172.21.15.203:3300/0 conn(0x7f462c0d2810 0x7f462c0b3c80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:30:17.361 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:17.354+0000 7f4634420640 1 --2- 172.21.15.37:0/1660587795 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f462c0d1ec0 0x7f462c0d22d0 unknown :-1 s=CLOSED pgs=439 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:30:17.361 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:17.354+0000 7f4634420640 1 -- 172.21.15.37:0/1660587795 >> 172.21.15.37:0/1660587795 conn(0x7f462c0d0390 msgr2=0x7f462c0d07a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:30:17.361 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:17.354+0000 7f4634420640 1 -- 172.21.15.37:0/1660587795 shutdown_connections 2024-04-25T10:30:17.361 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:17.354+0000 7f4634420640 1 -- 172.21.15.37:0/1660587795 wait complete. 2024-04-25T10:30:17.361 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:17.356+0000 7f4634420640 1 Processor -- start 2024-04-25T10:30:17.362 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:17.356+0000 7f4634420640 1 -- start start 2024-04-25T10:30:17.362 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:17.357+0000 7f4634420640 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f462c06e260 0x7f462c1d5f20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:30:17.362 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:17.357+0000 7f4634420640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f462c0d1ec0 0x7f462c09e740 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:30:17.362 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:17.357+0000 7f4634420640 1 --2- >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f462c0d2810 0x7f462c1ebe10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:30:17.362 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:17.357+0000 7f4634420640 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 -- 0x7f462c0b9000 con 0x7f462c0d1ec0 2024-04-25T10:30:17.362 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:17.357+0000 7f4634420640 1 -- --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_getmap magic: 0 -- 0x7f462c06d830 con 0x7f462c06e260 2024-04-25T10:30:17.362 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:17.357+0000 7f4634420640 1 -- --> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] -- mon_getmap magic: 0 -- 0x7f462c0b9180 con 0x7f462c0d2810 2024-04-25T10:30:17.362 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:17.357+0000 7f46319bb640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f462c0d1ec0 0x7f462c09e740 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:30:17.362 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:17.357+0000 7f46319bb640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f462c0d1ec0 0x7f462c09e740 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.37:3300/0 says I am v2:172.21.15.37:52748/0 (socket says 172.21.15.37:52748) 2024-04-25T10:30:17.362 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:17.357+0000 7f46319bb640 1 -- 172.21.15.37:0/206347787 learned_addr learned my addr 172.21.15.37:0/206347787 (peer_addr_for_me v2:172.21.15.37:0/0) 2024-04-25T10:30:17.362 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:17.357+0000 7f46329bd640 1 --2- 172.21.15.37:0/206347787 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f462c0d2810 0x7f462c1ebe10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:30:17.362 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:17.357+0000 7f46321bc640 1 --2- 172.21.15.37:0/206347787 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f462c06e260 0x7f462c1d5f20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:30:17.362 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:17.358+0000 7f46319bb640 1 -- 172.21.15.37:0/206347787 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f462c0d2810 msgr2=0x7f462c1ebe10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:30:17.362 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:17.358+0000 7f46319bb640 1 --2- 172.21.15.37:0/206347787 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f462c0d2810 0x7f462c1ebe10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:30:17.363 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:17.358+0000 7f46319bb640 1 -- 172.21.15.37:0/206347787 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f462c06e260 msgr2=0x7f462c1d5f20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:30:17.363 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:17.358+0000 7f46319bb640 1 --2- 172.21.15.37:0/206347787 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f462c06e260 0x7f462c1d5f20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:30:17.363 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:17.358+0000 7f46319bb640 1 -- 172.21.15.37:0/206347787 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f462c1d6770 con 0x7f462c0d1ec0 2024-04-25T10:30:17.363 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:17.358+0000 7f46319bb640 1 --2- 172.21.15.37:0/206347787 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f462c0d1ec0 0x7f462c09e740 secure :-1 s=READY pgs=440 cs=0 l=1 rev1=1 crypto rx=0x7f462002a220 tx=0x7f46200046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:30:17.363 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:17.358+0000 7f462affd640 1 -- 172.21.15.37:0/206347787 <== mon.0 v2:172.21.15.37:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4620037040 con 0x7f462c0d1ec0 2024-04-25T10:30:17.363 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:17.358+0000 7f462affd640 1 -- 172.21.15.37:0/206347787 <== mon.0 v2:172.21.15.37:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f4620042070 con 0x7f462c0d1ec0 2024-04-25T10:30:17.363 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:17.358+0000 7f4634420640 1 -- 172.21.15.37:0/206347787 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f462c1d6a50 con 0x7f462c0d1ec0 2024-04-25T10:30:17.363 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:17.358+0000 7f462affd640 1 -- 172.21.15.37:0/206347787 <== mon.0 v2:172.21.15.37:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4620030880 con 0x7f462c0d1ec0 2024-04-25T10:30:17.363 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:17.358+0000 7f4634420640 1 -- 172.21.15.37:0/206347787 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f462c1d7010 con 0x7f462c0d1ec0 2024-04-25T10:30:17.364 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:17.359+0000 7f4634420640 1 -- 172.21.15.37:0/206347787 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f462c06a140 con 0x7f462c0d1ec0 2024-04-25T10:30:17.364 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:17.360+0000 7f462affd640 1 -- 172.21.15.37:0/206347787 <== mon.0 v2:172.21.15.37:3300/0 4 ==== mgrmap(e 5) ==== 96782+0+0 (secure 0 0 0) 0x7f4620030a20 con 0x7f462c0d1ec0 2024-04-25T10:30:17.364 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:17.360+0000 7f462affd640 1 --2- 172.21.15.37:0/206347787 >> [v2:172.21.15.203:6832/2418205237,v1:172.21.15.203:6833/2418205237] conn(0x7f46180744e0 0x7f46180769f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:30:17.364 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:17.360+0000 7f462affd640 1 -- 172.21.15.37:0/206347787 <== mon.0 v2:172.21.15.37:3300/0 5 ==== osd_map(36..36 src has 1..36) ==== 5709+0+0 (secure 0 0 0) 0x7f46200b59e0 con 0x7f462c0d1ec0 2024-04-25T10:30:17.364 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:17.361+0000 7f46321bc640 1 --2- 172.21.15.37:0/206347787 >> [v2:172.21.15.203:6832/2418205237,v1:172.21.15.203:6833/2418205237] conn(0x7f46180744e0 0x7f46180769f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:30:17.364 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:17.361+0000 7f46321bc640 1 --2- 172.21.15.37:0/206347787 >> [v2:172.21.15.203:6832/2418205237,v1:172.21.15.203:6833/2418205237] conn(0x7f46180744e0 0x7f46180769f0 secure :-1 s=READY pgs=220 cs=0 l=1 rev1=1 crypto rx=0x7f4624007fd0 tx=0x7f4624009040 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:30:17.367 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:17.365+0000 7f462affd640 1 -- 172.21.15.37:0/206347787 <== mon.0 v2:172.21.15.37:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187246 (secure 0 0 0) 0x7f4620080290 con 0x7f462c0d1ec0 2024-04-25T10:30:17.620 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:17.618+0000 7f4634420640 1 -- 172.21.15.37:0/206347787 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_command({"prefix": "auth caps", "entity": "client.0", "caps": ["mon", "allow r", "osd", "allow rw tag cephfs data=cephfs", "mds", "allow"]} v 0) -- 0x7f462c1d6500 con 0x7f462c0d1ec0 2024-04-25T10:30:17.622 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:17.621+0000 7f462affd640 1 -- 172.21.15.37:0/206347787 <== mon.0 v2:172.21.15.37:3300/0 7 ==== mon_command_ack([{"prefix": "auth caps", "entity": "client.0", "caps": ["mon", "allow r", "osd", "allow rw tag cephfs data=cephfs", "mds", "allow"]}]=0 updated caps for client.0 v9) ==== 190+0+147 (secure 0 0 0) 0x7f462007fb90 con 0x7f462c0d1ec0 2024-04-25T10:30:17.623 INFO:teuthology.orchestra.run.smithi037.stdout:[client.0] 2024-04-25T10:30:17.623 INFO:teuthology.orchestra.run.smithi037.stdout: key = AQA6MCpmQ308HBAAvMknoENSGUD2vblm1SMnBw== 2024-04-25T10:30:17.623 INFO:teuthology.orchestra.run.smithi037.stdout: caps mds = "allow" 2024-04-25T10:30:17.623 INFO:teuthology.orchestra.run.smithi037.stdout: caps mon = "allow r" 2024-04-25T10:30:17.623 INFO:teuthology.orchestra.run.smithi037.stdout: caps osd = "allow rw tag cephfs data=cephfs" 2024-04-25T10:30:17.623 INFO:teuthology.orchestra.run.smithi037.stderr:updated caps for client.0 2024-04-25T10:30:17.625 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:17.623+0000 7f4634420640 1 -- 172.21.15.37:0/206347787 >> [v2:172.21.15.203:6832/2418205237,v1:172.21.15.203:6833/2418205237] conn(0x7f46180744e0 msgr2=0x7f46180769f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:30:17.625 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:17.623+0000 7f4634420640 1 --2- 172.21.15.37:0/206347787 >> [v2:172.21.15.203:6832/2418205237,v1:172.21.15.203:6833/2418205237] conn(0x7f46180744e0 0x7f46180769f0 secure :-1 s=READY pgs=220 cs=0 l=1 rev1=1 crypto rx=0x7f4624007fd0 tx=0x7f4624009040 comp rx=0 tx=0).stop 2024-04-25T10:30:17.625 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:17.623+0000 7f4634420640 1 -- 172.21.15.37:0/206347787 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f462c0d1ec0 msgr2=0x7f462c09e740 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:30:17.625 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:17.623+0000 7f4634420640 1 --2- 172.21.15.37:0/206347787 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f462c0d1ec0 0x7f462c09e740 secure :-1 s=READY pgs=440 cs=0 l=1 rev1=1 crypto rx=0x7f462002a220 tx=0x7f46200046c0 comp rx=0 tx=0).stop 2024-04-25T10:30:17.625 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:17.623+0000 7f4634420640 1 -- 172.21.15.37:0/206347787 shutdown_connections 2024-04-25T10:30:17.625 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:17.623+0000 7f4634420640 1 --2- 172.21.15.37:0/206347787 >> [v2:172.21.15.203:6832/2418205237,v1:172.21.15.203:6833/2418205237] conn(0x7f46180744e0 0x7f46180769f0 unknown :-1 s=CLOSED pgs=220 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:30:17.626 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:17.623+0000 7f4634420640 1 --2- 172.21.15.37:0/206347787 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f462c0d2810 0x7f462c1ebe10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:30:17.626 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:17.623+0000 7f4634420640 1 --2- 172.21.15.37:0/206347787 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f462c0d1ec0 0x7f462c09e740 unknown :-1 s=CLOSED pgs=440 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:30:17.626 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:17.623+0000 7f4634420640 1 --2- 172.21.15.37:0/206347787 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f462c06e260 0x7f462c1d5f20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:30:17.626 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:17.623+0000 7f4634420640 1 -- 172.21.15.37:0/206347787 >> 172.21.15.37:0/206347787 conn(0x7f462c0d0390 msgr2=0x7f462c0b0b30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:30:17.626 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:17.623+0000 7f4634420640 1 -- 172.21.15.37:0/206347787 shutdown_connections 2024-04-25T10:30:17.626 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:17.623+0000 7f4634420640 1 -- 172.21.15.37:0/206347787 wait complete. 2024-04-25T10:30:17.636 DEBUG:teuthology.orchestra.run.smithi037:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-04-25T10:30:17.747 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:17.745+0000 7fed0b834640 1 Processor -- start 2024-04-25T10:30:17.747 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:17.745+0000 7fed0b834640 1 -- start start 2024-04-25T10:30:17.749 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:17.746+0000 7fed0b834640 1 --2- >> v2:172.21.15.37:3300/0 conn(0x7fed040b4bb0 0x7fed041646b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:30:17.749 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:17.746+0000 7fed0b834640 1 --2- >> v2:172.21.15.203:3300/0 conn(0x7fed040b7600 0x7fed040b7a10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:30:17.749 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:17.746+0000 7fed0b834640 1 -- --> v1:172.21.15.203:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fed040b7f50 con 0x7fed040d4560 2024-04-25T10:30:17.749 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:17.746+0000 7fed0b834640 1 -- --> v2:172.21.15.37:3300/0 -- mon_getmap magic: 0 -- 0x7fed040d5080 con 0x7fed040b4bb0 2024-04-25T10:30:17.749 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:17.746+0000 7fed0b834640 1 -- --> v2:172.21.15.203:3300/0 -- mon_getmap magic: 0 -- 0x7fed04161190 con 0x7fed040b7600 2024-04-25T10:30:17.749 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:17.746+0000 7fed095d0640 1 --2- >> v2:172.21.15.37:3300/0 conn(0x7fed040b4bb0 0x7fed041646b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:30:17.749 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:17.746+0000 7fed095d0640 1 --2- >> v2:172.21.15.37:3300/0 conn(0x7fed040b4bb0 0x7fed041646b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.37:3300/0 says I am v2:172.21.15.37:52756/0 (socket says 172.21.15.37:52756) 2024-04-25T10:30:17.749 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:17.746+0000 7fed095d0640 1 -- 172.21.15.37:0/2137242668 learned_addr learned my addr 172.21.15.37:0/2137242668 (peer_addr_for_me v2:172.21.15.37:0/0) 2024-04-25T10:30:17.749 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:17.747+0000 7fed08dcf640 1 --2- 172.21.15.37:0/2137242668 >> v2:172.21.15.203:3300/0 conn(0x7fed040b7600 0x7fed040b7a10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:30:17.749 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:17.747+0000 7fed095d0640 1 -- 172.21.15.37:0/2137242668 >> v2:172.21.15.203:3300/0 conn(0x7fed040b7600 msgr2=0x7fed040b7a10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:30:17.750 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:17.747+0000 7fed095d0640 1 --2- 172.21.15.37:0/2137242668 >> v2:172.21.15.203:3300/0 conn(0x7fed040b7600 0x7fed040b7a10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:30:17.750 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:17.747+0000 7fed095d0640 1 -- 172.21.15.37:0/2137242668 >> v1:172.21.15.203:6789/0 conn(0x7fed040d4560 legacy=0x7fed040d4970 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:30:17.750 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:17.747+0000 7fed095d0640 1 -- 172.21.15.37:0/2137242668 --> v2:172.21.15.37:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fed04161470 con 0x7fed040b4bb0 2024-04-25T10:30:17.751 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:17.747+0000 7fed095d0640 1 --2- 172.21.15.37:0/2137242668 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fed040b4bb0 0x7fed041646b0 secure :-1 s=READY pgs=441 cs=0 l=1 rev1=1 crypto rx=0x7fed040ccbe0 tx=0x7fecfc02a270 comp rx=0 tx=0).ready entity=mon.0 client_cookie=8b7fdc0ac091552b server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:30:17.751 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:17.748+0000 7fed03fff640 1 -- 172.21.15.37:0/2137242668 <== mon.0 v2:172.21.15.37:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fecfc037070 con 0x7fed040b4bb0 2024-04-25T10:30:17.751 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:17.748+0000 7fed03fff640 1 -- 172.21.15.37:0/2137242668 <== mon.0 v2:172.21.15.37:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fecfc032440 con 0x7fed040b4bb0 2024-04-25T10:30:17.751 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:17.748+0000 7fed03fff640 1 -- 172.21.15.37:0/2137242668 <== mon.0 v2:172.21.15.37:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fecfc032810 con 0x7fed040b4bb0 2024-04-25T10:30:17.751 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:17.748+0000 7fed0b834640 1 -- 172.21.15.37:0/2137242668 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fed040b4bb0 msgr2=0x7fed041646b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:30:17.751 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:17.748+0000 7fed0b834640 1 --2- 172.21.15.37:0/2137242668 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fed040b4bb0 0x7fed041646b0 secure :-1 s=READY pgs=441 cs=0 l=1 rev1=1 crypto rx=0x7fed040ccbe0 tx=0x7fecfc02a270 comp rx=0 tx=0).stop 2024-04-25T10:30:17.751 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:17.748+0000 7fed0b834640 1 -- 172.21.15.37:0/2137242668 shutdown_connections 2024-04-25T10:30:17.751 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:17.748+0000 7fed0b834640 1 --2- 172.21.15.37:0/2137242668 >> v2:172.21.15.203:3300/0 conn(0x7fed040b7600 0x7fed040b7a10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:30:17.751 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:17.748+0000 7fed0b834640 1 --2- 172.21.15.37:0/2137242668 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fed040b4bb0 0x7fed041646b0 unknown :-1 s=CLOSED pgs=441 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:30:17.751 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:17.748+0000 7fed0b834640 1 -- 172.21.15.37:0/2137242668 >> 172.21.15.37:0/2137242668 conn(0x7fed040d27e0 msgr2=0x7fed040d2bf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:30:17.752 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:17.748+0000 7fed0b834640 1 -- 172.21.15.37:0/2137242668 shutdown_connections 2024-04-25T10:30:17.752 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:17.748+0000 7fed0b834640 1 -- 172.21.15.37:0/2137242668 wait complete. 2024-04-25T10:30:17.752 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:17.750+0000 7fed0b834640 1 Processor -- start 2024-04-25T10:30:17.752 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:17.750+0000 7fed0b834640 1 -- start start 2024-04-25T10:30:17.755 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:17.751+0000 7fed0b834640 1 --2- >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7fed040b4bb0 0x7fed040c84f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:30:17.755 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:17.751+0000 7fed0b834640 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fed040b7600 0x7fed0415acf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:30:17.755 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:17.751+0000 7fed0b834640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fed040d4560 0x7fed040ba120 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:30:17.755 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:17.752+0000 7fed0b834640 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 -- 0x7fed040b4fc0 con 0x7fed040d4560 2024-04-25T10:30:17.755 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:17.752+0000 7fed0b834640 1 -- --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_getmap magic: 0 -- 0x7fed04164190 con 0x7fed040b7600 2024-04-25T10:30:17.755 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:17.752+0000 7fed0b834640 1 -- --> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] -- mon_getmap magic: 0 -- 0x7fed04164bf0 con 0x7fed040b4bb0 2024-04-25T10:30:17.755 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:17.752+0000 7fed09dd1640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fed040d4560 0x7fed040ba120 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:30:17.755 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:17.752+0000 7fed09dd1640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fed040d4560 0x7fed040ba120 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.37:3300/0 says I am v2:172.21.15.37:52762/0 (socket says 172.21.15.37:52762) 2024-04-25T10:30:17.756 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:17.752+0000 7fed09dd1640 1 -- 172.21.15.37:0/2688438621 learned_addr learned my addr 172.21.15.37:0/2688438621 (peer_addr_for_me v2:172.21.15.37:0/0) 2024-04-25T10:30:17.756 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:17.752+0000 7fed095d0640 1 --2- 172.21.15.37:0/2688438621 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7fed040b4bb0 0x7fed040c84f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:30:17.756 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:17.752+0000 7fed08dcf640 1 --2- 172.21.15.37:0/2688438621 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fed040b7600 0x7fed0415acf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:30:17.756 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:17.752+0000 7fed09dd1640 1 -- 172.21.15.37:0/2688438621 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7fed040b4bb0 msgr2=0x7fed040c84f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:30:17.756 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:17.752+0000 7fed09dd1640 1 --2- 172.21.15.37:0/2688438621 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7fed040b4bb0 0x7fed040c84f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:30:17.756 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:17.752+0000 7fed09dd1640 1 -- 172.21.15.37:0/2688438621 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fed040b7600 msgr2=0x7fed0415acf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:30:17.756 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:17.752+0000 7fed09dd1640 1 --2- 172.21.15.37:0/2688438621 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fed040b7600 0x7fed0415acf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:30:17.756 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:17.752+0000 7fed09dd1640 1 -- 172.21.15.37:0/2688438621 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fed041e9d10 con 0x7fed040d4560 2024-04-25T10:30:17.756 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:17.752+0000 7fed095d0640 1 --2- 172.21.15.37:0/2688438621 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7fed040b4bb0 0x7fed040c84f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T10:30:17.756 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:17.752+0000 7fed09dd1640 1 --2- 172.21.15.37:0/2688438621 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fed040d4560 0x7fed040ba120 secure :-1 s=READY pgs=442 cs=0 l=1 rev1=1 crypto rx=0x7fecf400c680 tx=0x7fecf400cb40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:30:17.756 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:17.752+0000 7fed01ffb640 1 -- 172.21.15.37:0/2688438621 <== mon.0 v2:172.21.15.37:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fecf4003b70 con 0x7fed040d4560 2024-04-25T10:30:17.756 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:17.752+0000 7fed01ffb640 1 -- 172.21.15.37:0/2688438621 <== mon.0 v2:172.21.15.37:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fecf4003d10 con 0x7fed040d4560 2024-04-25T10:30:17.756 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:17.752+0000 7fed0b834640 1 -- 172.21.15.37:0/2688438621 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fed041e9ff0 con 0x7fed040d4560 2024-04-25T10:30:17.756 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:17.752+0000 7fed01ffb640 1 -- 172.21.15.37:0/2688438621 <== mon.0 v2:172.21.15.37:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fecf4005310 con 0x7fed040d4560 2024-04-25T10:30:17.757 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:17.753+0000 7fed0b834640 1 -- 172.21.15.37:0/2688438621 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fed041ea5b0 con 0x7fed040d4560 2024-04-25T10:30:17.757 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:17.754+0000 7fed0b834640 1 -- 172.21.15.37:0/2688438621 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fed040d4970 con 0x7fed040d4560 2024-04-25T10:30:17.757 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:17.754+0000 7fed01ffb640 1 -- 172.21.15.37:0/2688438621 <== mon.0 v2:172.21.15.37:3300/0 4 ==== mgrmap(e 5) ==== 96782+0+0 (secure 0 0 0) 0x7fecf40054b0 con 0x7fed040d4560 2024-04-25T10:30:17.758 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:17.754+0000 7fed01ffb640 1 --2- 172.21.15.37:0/2688438621 >> [v2:172.21.15.203:6832/2418205237,v1:172.21.15.203:6833/2418205237] conn(0x7fecf00744b0 0x7fecf00769c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:30:17.758 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:17.755+0000 7fed01ffb640 1 -- 172.21.15.37:0/2688438621 <== mon.0 v2:172.21.15.37:3300/0 5 ==== osd_map(36..36 src has 1..36) ==== 5709+0+0 (secure 0 0 0) 0x7fecf40950d0 con 0x7fed040d4560 2024-04-25T10:30:17.758 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:17.755+0000 7fed095d0640 1 --2- 172.21.15.37:0/2688438621 >> [v2:172.21.15.203:6832/2418205237,v1:172.21.15.203:6833/2418205237] conn(0x7fecf00744b0 0x7fecf00769c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:30:17.758 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:17.755+0000 7fed095d0640 1 --2- 172.21.15.37:0/2688438621 >> [v2:172.21.15.203:6832/2418205237,v1:172.21.15.203:6833/2418205237] conn(0x7fecf00744b0 0x7fecf00769c0 secure :-1 s=READY pgs=221 cs=0 l=1 rev1=1 crypto rx=0x7fecfc034000 tx=0x7fecfc02e000 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:30:17.761 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:17.759+0000 7fed01ffb640 1 -- 172.21.15.37:0/2688438621 <== mon.0 v2:172.21.15.37:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187246 (secure 0 0 0) 0x7fed040d4970 con 0x7fed040d4560 2024-04-25T10:30:17.987 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:17.985+0000 7fed0b834640 1 -- 172.21.15.37:0/2688438621 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7fed0406a140 con 0x7fed040d4560 2024-04-25T10:30:17.988 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:17.986+0000 7fed01ffb640 1 -- 172.21.15.37:0/2688438621 <== mon.0 v2:172.21.15.37:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 17 v17) ==== 94+0+5181 (secure 0 0 0) 0x7fed040b7f50 con 0x7fed040d4560 2024-04-25T10:30:17.988 INFO:teuthology.orchestra.run.smithi037.stdout: 2024-04-25T10:30:17.988 INFO:teuthology.orchestra.run.smithi037.stdout:{"epoch":17,"default_fscid":2,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":4790,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.203:6837/2245361396","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6836","nonce":2245361396},{"type":"v1","addr":"172.21.15.203:6837","nonce":2245361396}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":11},{"gid":5005,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.203:6835/3353571013","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6834","nonce":3353571013},{"type":"v1","addr":"172.21.15.203:6835","nonce":3353571013}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":12},{"gid":5313,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.37:6834/824801265","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.37:6833","nonce":824801265},{"type":"v1","addr":"172.21.15.37:6834","nonce":824801265}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":12}],"filesystems":[{"mdsmap":{"epoch":17,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-04-25T10:30:12.130473+0000","modified":"2024-04-25T10:30:15.157110+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":1,"in":[0],"up":{"mds_0":5316},"failed":[],"damaged":[],"stopped":[],"info":{"gid_5316":{"gid":5316,"name":"a","rank":0,"incarnation":13,"state":"up:active","state_seq":3,"addr":"172.21.15.37:6835/311552410","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.37:6832","nonce":311552410},{"type":"v1","addr":"172.21.15.37:6835","nonce":311552410}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[5],"metadata_pool":4,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":5316,"qdb_cluster":[5316]},"id":2}]} 2024-04-25T10:30:17.988 INFO:teuthology.orchestra.run.smithi037.stderr:dumped fsmap epoch 17 2024-04-25T10:30:17.990 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:17.988+0000 7fed0b834640 1 -- 172.21.15.37:0/2688438621 >> [v2:172.21.15.203:6832/2418205237,v1:172.21.15.203:6833/2418205237] conn(0x7fecf00744b0 msgr2=0x7fecf00769c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:30:17.990 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:17.988+0000 7fed0b834640 1 --2- 172.21.15.37:0/2688438621 >> [v2:172.21.15.203:6832/2418205237,v1:172.21.15.203:6833/2418205237] conn(0x7fecf00744b0 0x7fecf00769c0 secure :-1 s=READY pgs=221 cs=0 l=1 rev1=1 crypto rx=0x7fecfc034000 tx=0x7fecfc02e000 comp rx=0 tx=0).stop 2024-04-25T10:30:17.990 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:17.988+0000 7fed0b834640 1 -- 172.21.15.37:0/2688438621 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fed040d4560 msgr2=0x7fed040ba120 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:30:17.990 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:17.988+0000 7fed0b834640 1 --2- 172.21.15.37:0/2688438621 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fed040d4560 0x7fed040ba120 secure :-1 s=READY pgs=442 cs=0 l=1 rev1=1 crypto rx=0x7fecf400c680 tx=0x7fecf400cb40 comp rx=0 tx=0).stop 2024-04-25T10:30:17.991 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:17.988+0000 7fed0b834640 1 -- 172.21.15.37:0/2688438621 shutdown_connections 2024-04-25T10:30:17.991 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:17.988+0000 7fed0b834640 1 --2- 172.21.15.37:0/2688438621 >> [v2:172.21.15.203:6832/2418205237,v1:172.21.15.203:6833/2418205237] conn(0x7fecf00744b0 0x7fecf00769c0 unknown :-1 s=CLOSED pgs=221 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:30:17.991 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:17.988+0000 7fed0b834640 1 --2- 172.21.15.37:0/2688438621 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fed040d4560 0x7fed040ba120 unknown :-1 s=CLOSED pgs=442 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:30:17.991 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:17.988+0000 7fed0b834640 1 --2- 172.21.15.37:0/2688438621 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fed040b7600 0x7fed0415acf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:30:17.991 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:17.988+0000 7fed0b834640 1 --2- 172.21.15.37:0/2688438621 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7fed040b4bb0 0x7fed040c84f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:30:17.991 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:17.988+0000 7fed0b834640 1 -- 172.21.15.37:0/2688438621 >> 172.21.15.37:0/2688438621 conn(0x7fed040d27e0 msgr2=0x7fed040b8270 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:30:17.991 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:17.988+0000 7fed0b834640 1 -- 172.21.15.37:0/2688438621 shutdown_connections 2024-04-25T10:30:17.991 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:17.988+0000 7fed0b834640 1 -- 172.21.15.37:0/2688438621 wait complete. 2024-04-25T10:30:18.403 DEBUG:tasks.cephfs.filesystem:are_daemons_healthy: mds map: {'epoch': 17, 'flags': 18, 'flags_state': {'joinable': True, 'allow_snaps': True, 'allow_multimds_snaps': True, 'allow_standby_replay': False, 'refuse_client_session': False, 'refuse_standby_for_another_fs': False, 'balance_automate': False}, 'ever_allowed_features': 0, 'explicitly_allowed_features': 0, 'created': '2024-04-25T10:30:12.130473+0000', 'modified': '2024-04-25T10:30:15.157110+0000', 'tableserver': 0, 'root': 0, 'session_timeout': 60, 'session_autoclose': 300, 'required_client_features': {}, 'max_file_size': 1099511627776, 'max_xattr_size': 65536, 'last_failure': 0, 'last_failure_osd_epoch': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments', 'feature_12': 'quiesce subvolumes'}}, 'max_mds': 1, 'in': [0], 'up': {'mds_0': 5316}, 'failed': [], 'damaged': [], 'stopped': [], 'info': {'gid_5316': {'gid': 5316, 'name': 'a', 'rank': 0, 'incarnation': 13, 'state': 'up:active', 'state_seq': 3, 'addr': '172.21.15.37:6835/311552410', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.37:6832', 'nonce': 311552410}, {'type': 'v1', 'addr': '172.21.15.37:6835', 'nonce': 311552410}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540701547738038271, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments', 'feature_12': 'quiesce subvolumes'}}}}, 'data_pools': [5], 'metadata_pool': 4, 'enabled': True, 'fs_name': 'cephfs', 'balancer': '', 'bal_rank_mask': '-1', 'standby_count_wanted': 0, 'qdb_leader': 5316, 'qdb_cluster': [5316]} 2024-04-25T10:30:18.403 DEBUG:tasks.cephfs.filesystem:are_daemons_healthy: 1/1 2024-04-25T10:30:18.404 DEBUG:teuthology.orchestra.run.smithi037:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.a status 2024-04-25T10:30:18.520 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:18.518+0000 7f150104c640 1 Processor -- start 2024-04-25T10:30:18.521 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:18.518+0000 7f150104c640 1 -- start start 2024-04-25T10:30:18.522 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:18.520+0000 7f150104c640 1 --2- >> v2:172.21.15.37:3300/0 conn(0x7f14fc0d0c60 0x7f14fc0ce8d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:30:18.522 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:18.520+0000 7f150104c640 1 --2- >> v2:172.21.15.203:3300/0 conn(0x7f14fc0cee10 0x7f14fc06f3a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:30:18.523 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:18.520+0000 7f150104c640 1 -- --> v1:172.21.15.203:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f14fc0cf600 con 0x7f14fc1441d0 2024-04-25T10:30:18.523 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:18.520+0000 7f150104c640 1 -- --> v2:172.21.15.37:3300/0 -- mon_getmap magic: 0 -- 0x7f14fc0cf8e0 con 0x7f14fc0d0c60 2024-04-25T10:30:18.523 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:18.520+0000 7f150104c640 1 -- --> v2:172.21.15.203:3300/0 -- mon_getmap magic: 0 -- 0x7f14fc0cfbc0 con 0x7f14fc0cee10 2024-04-25T10:30:18.523 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:18.520+0000 7f14fa59c640 1 --2- >> v2:172.21.15.37:3300/0 conn(0x7f14fc0d0c60 0x7f14fc0ce8d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:30:18.523 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:18.520+0000 7f14fa59c640 1 --2- >> v2:172.21.15.37:3300/0 conn(0x7f14fc0d0c60 0x7f14fc0ce8d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.37:3300/0 says I am v2:172.21.15.37:52770/0 (socket says 172.21.15.37:52770) 2024-04-25T10:30:18.523 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:18.520+0000 7f14fa59c640 1 -- 172.21.15.37:0/2110204235 learned_addr learned my addr 172.21.15.37:0/2110204235 (peer_addr_for_me v2:172.21.15.37:0/0) 2024-04-25T10:30:18.523 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:18.520+0000 7f14fb59e640 1 --2- 172.21.15.37:0/2110204235 >> v2:172.21.15.203:3300/0 conn(0x7f14fc0cee10 0x7f14fc06f3a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:30:18.523 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:18.520+0000 7f14fa59c640 1 -- 172.21.15.37:0/2110204235 >> v2:172.21.15.203:3300/0 conn(0x7f14fc0cee10 msgr2=0x7f14fc06f3a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:30:18.523 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:18.520+0000 7f14fa59c640 1 --2- 172.21.15.37:0/2110204235 >> v2:172.21.15.203:3300/0 conn(0x7f14fc0cee10 0x7f14fc06f3a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:30:18.523 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:18.520+0000 7f14fa59c640 1 -- 172.21.15.37:0/2110204235 >> v1:172.21.15.203:6789/0 conn(0x7f14fc1441d0 legacy=0x7f14fc0d0550 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:30:18.523 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:18.520+0000 7f14fa59c640 1 -- 172.21.15.37:0/2110204235 --> v2:172.21.15.37:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f14fc0cfea0 con 0x7f14fc0d0c60 2024-04-25T10:30:18.525 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:18.521+0000 7f14fa59c640 1 --2- 172.21.15.37:0/2110204235 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f14fc0d0c60 0x7f14fc0ce8d0 secure :-1 s=READY pgs=443 cs=0 l=1 rev1=1 crypto rx=0x7f14fc0ce330 tx=0x7f14e402a230 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ec0c93d92b4e8bfd server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:30:18.525 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:18.521+0000 7f14f9d9b640 1 -- 172.21.15.37:0/2110204235 <== mon.0 v2:172.21.15.37:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f14e4037070 con 0x7f14fc0d0c60 2024-04-25T10:30:18.525 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:18.521+0000 7f14f9d9b640 1 -- 172.21.15.37:0/2110204235 <== mon.0 v2:172.21.15.37:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f14e4032440 con 0x7f14fc0d0c60 2024-04-25T10:30:18.525 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:18.521+0000 7f14f9d9b640 1 -- 172.21.15.37:0/2110204235 <== mon.0 v2:172.21.15.37:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f14e4032810 con 0x7f14fc0d0c60 2024-04-25T10:30:18.525 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:18.522+0000 7f150104c640 1 -- 172.21.15.37:0/2110204235 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f14fc0d0c60 msgr2=0x7f14fc0ce8d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:30:18.525 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:18.522+0000 7f150104c640 1 --2- 172.21.15.37:0/2110204235 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f14fc0d0c60 0x7f14fc0ce8d0 secure :-1 s=READY pgs=443 cs=0 l=1 rev1=1 crypto rx=0x7f14fc0ce330 tx=0x7f14e402a230 comp rx=0 tx=0).stop 2024-04-25T10:30:18.525 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:18.522+0000 7f150104c640 1 -- 172.21.15.37:0/2110204235 shutdown_connections 2024-04-25T10:30:18.525 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:18.522+0000 7f150104c640 1 --2- 172.21.15.37:0/2110204235 >> v2:172.21.15.203:3300/0 conn(0x7f14fc0cee10 0x7f14fc06f3a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:30:18.525 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:18.522+0000 7f150104c640 1 --2- 172.21.15.37:0/2110204235 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f14fc0d0c60 0x7f14fc0ce8d0 unknown :-1 s=CLOSED pgs=443 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:30:18.525 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:18.522+0000 7f150104c640 1 -- 172.21.15.37:0/2110204235 >> 172.21.15.37:0/2110204235 conn(0x7f14fc0b36b0 msgr2=0x7f14fc0b3ac0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:30:18.526 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:18.522+0000 7f150104c640 1 -- 172.21.15.37:0/2110204235 shutdown_connections 2024-04-25T10:30:18.526 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:18.522+0000 7f150104c640 1 -- 172.21.15.37:0/2110204235 wait complete. 2024-04-25T10:30:18.526 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:18.524+0000 7f150104c640 1 Processor -- start 2024-04-25T10:30:18.526 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:18.524+0000 7f150104c640 1 -- start start 2024-04-25T10:30:18.529 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:18.525+0000 7f150104c640 1 --2- >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f14fc0cee10 0x7f14fc0b0550 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:30:18.529 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:18.525+0000 7f150104c640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f14fc1441d0 0x7f14fc147450 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:30:18.529 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:18.525+0000 7f150104c640 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f14fc1f6620 0x7f14fc1f6a30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:30:18.529 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:18.525+0000 7f150104c640 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 -- 0x7f14fc0cf3a0 con 0x7f14fc1441d0 2024-04-25T10:30:18.529 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:18.525+0000 7f150104c640 1 -- --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_getmap magic: 0 -- 0x7f14fc0cf220 con 0x7f14fc1f6620 2024-04-25T10:30:18.529 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:18.525+0000 7f150104c640 1 -- --> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] -- mon_getmap magic: 0 -- 0x7f14fc1445e0 con 0x7f14fc0cee10 2024-04-25T10:30:18.529 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:18.525+0000 7f14fa59c640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f14fc1441d0 0x7f14fc147450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:30:18.529 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:18.525+0000 7f14fa59c640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f14fc1441d0 0x7f14fc147450 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.37:3300/0 says I am v2:172.21.15.37:52786/0 (socket says 172.21.15.37:52786) 2024-04-25T10:30:18.530 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:18.525+0000 7f14fa59c640 1 -- 172.21.15.37:0/1182015337 learned_addr learned my addr 172.21.15.37:0/1182015337 (peer_addr_for_me v2:172.21.15.37:0/0) 2024-04-25T10:30:18.530 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:18.525+0000 7f14fb59e640 1 --2- 172.21.15.37:0/1182015337 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f14fc1f6620 0x7f14fc1f6a30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:30:18.530 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:18.525+0000 7f14fad9d640 1 --2- 172.21.15.37:0/1182015337 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f14fc0cee10 0x7f14fc0b0550 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:30:18.530 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:18.526+0000 7f14fa59c640 1 -- 172.21.15.37:0/1182015337 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f14fc0cee10 msgr2=0x7f14fc0b0550 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:30:18.530 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:18.526+0000 7f14fa59c640 1 --2- 172.21.15.37:0/1182015337 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f14fc0cee10 0x7f14fc0b0550 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:30:18.530 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:18.526+0000 7f14fa59c640 1 -- 172.21.15.37:0/1182015337 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f14fc1f6620 msgr2=0x7f14fc1f6a30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:30:18.530 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:18.526+0000 7f14fa59c640 1 --2- 172.21.15.37:0/1182015337 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f14fc1f6620 0x7f14fc1f6a30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:30:18.530 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:18.526+0000 7f14fa59c640 1 -- 172.21.15.37:0/1182015337 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f14fc0b0a90 con 0x7f14fc1441d0 2024-04-25T10:30:18.530 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:18.526+0000 7f14fb59e640 1 --2- 172.21.15.37:0/1182015337 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f14fc1f6620 0x7f14fc1f6a30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T10:30:18.530 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:18.526+0000 7f14fa59c640 1 --2- 172.21.15.37:0/1182015337 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f14fc1441d0 0x7f14fc147450 secure :-1 s=READY pgs=444 cs=0 l=1 rev1=1 crypto rx=0x7f14e402a270 tx=0x7f14e40046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:30:18.530 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:18.526+0000 7f14f37fe640 1 -- 172.21.15.37:0/1182015337 <== mon.0 v2:172.21.15.37:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f14e4041070 con 0x7f14fc1441d0 2024-04-25T10:30:18.530 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:18.526+0000 7f14f37fe640 1 -- 172.21.15.37:0/1182015337 <== mon.0 v2:172.21.15.37:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f14e403c870 con 0x7f14fc1441d0 2024-04-25T10:30:18.531 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:18.526+0000 7f150104c640 1 -- 172.21.15.37:0/1182015337 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f14fc0af4b0 con 0x7f14fc1441d0 2024-04-25T10:30:18.531 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:18.526+0000 7f14f37fe640 1 -- 172.21.15.37:0/1182015337 <== mon.0 v2:172.21.15.37:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f14e4037040 con 0x7f14fc1441d0 2024-04-25T10:30:18.531 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:18.526+0000 7f150104c640 1 -- 172.21.15.37:0/1182015337 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f14fc0afa70 con 0x7f14fc1441d0 2024-04-25T10:30:18.532 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:18.528+0000 7f14f37fe640 1 -- 172.21.15.37:0/1182015337 <== mon.0 v2:172.21.15.37:3300/0 4 ==== mgrmap(e 5) ==== 96782+0+0 (secure 0 0 0) 0x7f14e4044070 con 0x7f14fc1441d0 2024-04-25T10:30:18.532 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:18.528+0000 7f14f37fe640 1 --2- 172.21.15.37:0/1182015337 >> [v2:172.21.15.203:6832/2418205237,v1:172.21.15.203:6833/2418205237] conn(0x7f14dc0744b0 0x7f14dc0769c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:30:18.532 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:18.529+0000 7f14f37fe640 1 -- 172.21.15.37:0/1182015337 <== mon.0 v2:172.21.15.37:3300/0 5 ==== osd_map(36..36 src has 1..36) ==== 5709+0+0 (secure 0 0 0) 0x7f14e40b5f90 con 0x7f14fc1441d0 2024-04-25T10:30:18.532 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:18.529+0000 7f14fad9d640 1 --2- 172.21.15.37:0/1182015337 >> [v2:172.21.15.203:6832/2418205237,v1:172.21.15.203:6833/2418205237] conn(0x7f14dc0744b0 0x7f14dc0769c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:30:18.532 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:18.529+0000 7f14fad9d640 1 --2- 172.21.15.37:0/1182015337 >> [v2:172.21.15.203:6832/2418205237,v1:172.21.15.203:6833/2418205237] conn(0x7f14dc0744b0 0x7f14dc0769c0 secure :-1 s=READY pgs=222 cs=0 l=1 rev1=1 crypto rx=0x7f14f4006fd0 tx=0x7f14f4008040 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:30:18.532 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:18.530+0000 7f1507fdb740 1 Processor -- start 2024-04-25T10:30:18.532 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:18.530+0000 7f1507fdb740 1 -- start start 2024-04-25T10:30:18.535 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:18.531+0000 7f1507fdb740 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x55b877363270 0x55b87772f600 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:30:18.535 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:18.531+0000 7f1507fdb740 1 --2- >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x55b87733a3b0 0x55b87772bfa0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:30:18.535 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:18.531+0000 7f1507fdb740 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55b877342590 0x55b877705260 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:30:18.535 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:18.531+0000 7f1507fdb740 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 -- 0x55b877669740 con 0x55b877342590 2024-04-25T10:30:18.535 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:18.531+0000 7f1507fdb740 1 -- --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_getmap magic: 0 -- 0x55b8776053a0 con 0x55b877363270 2024-04-25T10:30:18.535 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:18.531+0000 7f1507fdb740 1 -- --> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] -- mon_getmap magic: 0 -- 0x55b877665700 con 0x55b87733a3b0 2024-04-25T10:30:18.535 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:18.531+0000 7f14fb59e640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55b877342590 0x55b877705260 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:30:18.535 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:18.531+0000 7f14fb59e640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55b877342590 0x55b877705260 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.37:3300/0 says I am v2:172.21.15.37:52788/0 (socket says 172.21.15.37:52788) 2024-04-25T10:30:18.535 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:18.531+0000 7f14fb59e640 1 -- 172.21.15.37:0/537237642 learned_addr learned my addr 172.21.15.37:0/537237642 (peer_addr_for_me v2:172.21.15.37:0/0) 2024-04-25T10:30:18.535 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:18.531+0000 7f14fa59c640 1 --2- 172.21.15.37:0/537237642 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x55b87733a3b0 0x55b87772bfa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:30:18.535 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:18.531+0000 7f14fad9d640 1 --2- 172.21.15.37:0/537237642 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x55b877363270 0x55b87772f600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:30:18.536 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:18.531+0000 7f14fb59e640 1 -- 172.21.15.37:0/537237642 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x55b87733a3b0 msgr2=0x55b87772bfa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:30:18.536 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:18.531+0000 7f14fb59e640 1 --2- 172.21.15.37:0/537237642 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x55b87733a3b0 0x55b87772bfa0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:30:18.536 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:18.531+0000 7f14fb59e640 1 -- 172.21.15.37:0/537237642 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x55b877363270 msgr2=0x55b87772f600 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:30:18.536 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:18.531+0000 7f14fb59e640 1 --2- 172.21.15.37:0/537237642 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x55b877363270 0x55b87772f600 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:30:18.536 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:18.531+0000 7f14fb59e640 1 -- 172.21.15.37:0/537237642 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55b877621750 con 0x55b877342590 2024-04-25T10:30:18.536 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:18.532+0000 7f14fb59e640 1 --2- 172.21.15.37:0/537237642 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55b877342590 0x55b877705260 secure :-1 s=READY pgs=445 cs=0 l=1 rev1=1 crypto rx=0x7f14ec007ec0 tx=0x7f14ec00e6c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=616a905eb0afc2fc server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:30:18.536 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:18.532+0000 7f14f0ff9640 1 -- 172.21.15.37:0/537237642 <== mon.0 v2:172.21.15.37:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f14ec015030 con 0x55b877342590 2024-04-25T10:30:18.536 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:18.532+0000 7f14f0ff9640 1 -- 172.21.15.37:0/537237642 <== mon.0 v2:172.21.15.37:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f14ec011470 con 0x55b877342590 2024-04-25T10:30:18.536 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:18.532+0000 7f14f0ff9640 1 -- 172.21.15.37:0/537237642 <== mon.0 v2:172.21.15.37:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f14ec011620 con 0x55b877342590 2024-04-25T10:30:18.536 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:18.533+0000 7f1507fdb740 1 -- 172.21.15.37:0/537237642 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55b877342590 msgr2=0x55b877705260 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:30:18.536 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:18.533+0000 7f1507fdb740 1 --2- 172.21.15.37:0/537237642 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55b877342590 0x55b877705260 secure :-1 s=READY pgs=445 cs=0 l=1 rev1=1 crypto rx=0x7f14ec007ec0 tx=0x7f14ec00e6c0 comp rx=0 tx=0).stop 2024-04-25T10:30:18.536 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:18.533+0000 7f1507fdb740 1 -- 172.21.15.37:0/537237642 shutdown_connections 2024-04-25T10:30:18.536 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:18.533+0000 7f1507fdb740 1 --2- 172.21.15.37:0/537237642 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55b877342590 0x55b877705260 unknown :-1 s=CLOSED pgs=445 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:30:18.536 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:18.533+0000 7f1507fdb740 1 --2- 172.21.15.37:0/537237642 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x55b87733a3b0 0x55b87772bfa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:30:18.537 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:18.533+0000 7f1507fdb740 1 --2- 172.21.15.37:0/537237642 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x55b877363270 0x55b87772f600 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:30:18.537 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:18.533+0000 7f1507fdb740 1 -- 172.21.15.37:0/537237642 >> 172.21.15.37:0/537237642 conn(0x55b8773573c0 msgr2=0x55b8777049f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:30:18.537 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:18.533+0000 7f1507fdb740 1 -- 172.21.15.37:0/537237642 shutdown_connections 2024-04-25T10:30:18.537 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:18.533+0000 7f1507fdb740 1 -- 172.21.15.37:0/537237642 wait complete. 2024-04-25T10:30:18.542 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:18.534+0000 7f1507fdb740 1 -- start start 2024-04-25T10:30:18.542 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:18.534+0000 7f1507fdb740 1 Processor -- start 2024-04-25T10:30:18.542 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:18.535+0000 7f1507fdb740 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x55b87733a3b0 0x55b87775d2a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:30:18.542 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:18.535+0000 7f1507fdb740 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55b877342590 0x55b87772ab30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:30:18.542 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:18.535+0000 7f1507fdb740 1 --2- >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x55b877363270 0x55b87772b070 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:30:18.542 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:18.535+0000 7f1507fdb740 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 -- 0x55b8775f0fe0 con 0x55b877342590 2024-04-25T10:30:18.542 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:18.535+0000 7f1507fdb740 1 -- --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_getmap magic: 0 -- 0x55b8777332c0 con 0x55b87733a3b0 2024-04-25T10:30:18.542 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:18.535+0000 7f1507fdb740 1 -- --> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] -- mon_getmap magic: 0 -- 0x55b87772c9b0 con 0x55b877363270 2024-04-25T10:30:18.542 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:18.535+0000 7f1507fdb740 10 client.0 _ll_register_callbacks cb 0x55b877607150 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-25T10:30:18.543 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:18.535+0000 7f14fa59c640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55b877342590 0x55b87772ab30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:30:18.543 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:18.535+0000 7f14fa59c640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55b877342590 0x55b87772ab30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.37:3300/0 says I am v2:172.21.15.37:52804/0 (socket says 172.21.15.37:52804) 2024-04-25T10:30:18.543 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:18.535+0000 7f14fa59c640 1 -- 172.21.15.37:0/1719454315 learned_addr learned my addr 172.21.15.37:0/1719454315 (peer_addr_for_me v2:172.21.15.37:0/0) 2024-04-25T10:30:18.543 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:18.535+0000 7f14fad9d640 1 --2- 172.21.15.37:0/1719454315 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x55b87733a3b0 0x55b87775d2a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:30:18.543 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:18.535+0000 7f14fb59e640 1 --2- 172.21.15.37:0/1719454315 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x55b877363270 0x55b87772b070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:30:18.543 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:18.535+0000 7f14fa59c640 1 -- 172.21.15.37:0/1719454315 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x55b877363270 msgr2=0x55b87772b070 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:30:18.543 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:18.535+0000 7f14fa59c640 1 --2- 172.21.15.37:0/1719454315 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x55b877363270 0x55b87772b070 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:30:18.543 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:18.536+0000 7f14fa59c640 1 -- 172.21.15.37:0/1719454315 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x55b87733a3b0 msgr2=0x55b87775d2a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:30:18.543 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:18.536+0000 7f14fa59c640 1 --2- 172.21.15.37:0/1719454315 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x55b87733a3b0 0x55b87775d2a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:30:18.543 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:18.536+0000 7f14fa59c640 1 -- 172.21.15.37:0/1719454315 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55b87737b2d0 con 0x55b877342590 2024-04-25T10:30:18.543 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:18.536+0000 7f14fa59c640 1 --2- 172.21.15.37:0/1719454315 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55b877342590 0x55b87772ab30 secure :-1 s=READY pgs=446 cs=0 l=1 rev1=1 crypto rx=0x7f14e40b4420 tx=0x7f14e403cee0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ccf757ab515f7ccf server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:30:18.543 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:18.536+0000 7f14d9ffb640 10 client.0 ms_handle_connect on v2:172.21.15.37:3300/0 2024-04-25T10:30:18.543 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:18.536+0000 7f14d9ffb640 1 -- 172.21.15.37:0/1719454315 <== mon.0 v2:172.21.15.37:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f14e404b040 con 0x55b877342590 2024-04-25T10:30:18.543 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:18.536+0000 7f14d9ffb640 1 -- 172.21.15.37:0/1719454315 <== mon.0 v2:172.21.15.37:3300/0 2 ==== osd_map(36..36 src has 1..36) ==== 5709+0+0 (secure 0 0 0) 0x7f14e407f1e0 con 0x55b877342590 2024-04-25T10:30:18.544 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:18.536+0000 7f1507fdb740 1 -- 172.21.15.37:0/1719454315 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f14ec011470 con 0x55b877342590 2024-04-25T10:30:18.544 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:18.536+0000 7f14d9ffb640 1 -- 172.21.15.37:0/1719454315 <== mon.0 v2:172.21.15.37:3300/0 3 ==== mon_get_version_reply(handle=1 version=17) ==== 24+0+0 (secure 0 0 0) 0x7f14e4080680 con 0x55b877342590 2024-04-25T10:30:18.544 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:18.536+0000 7f1507fdb740 10 client.5481 fetch_fsmap learned FSMap version 17 2024-04-25T10:30:18.544 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:18.536+0000 7f1507fdb740 1 -- 172.21.15.37:0/1719454315 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({config=0+,fsmap=17,monmap=2+}) -- 0x55b8774456b0 con 0x55b877342590 2024-04-25T10:30:18.544 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:18.536+0000 7f14d9ffb640 1 -- 172.21.15.37:0/1719454315 <== mon.0 v2:172.21.15.37:3300/0 4 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f14ec011470 con 0x55b877342590 2024-04-25T10:30:18.544 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:18.536+0000 7f14d9ffb640 1 -- 172.21.15.37:0/1719454315 <== mon.0 v2:172.21.15.37:3300/0 5 ==== fsmap(e 17) ==== 3494+0+0 (secure 0 0 0) 0x7f14e407b8f0 con 0x55b877342590 2024-04-25T10:30:18.544 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:18.536+0000 7f1507fdb740 10 client.5481 fetch_fsmap finished waiting for FSMap version 17 2024-04-25T10:30:18.544 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:18.536+0000 7f1507fdb740 10 client.5481 resolve_mds: resolved name 'a' to daemon mds.a 2024-04-25T10:30:18.544 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:18.536+0000 7f1507fdb740 20 client.5481 populate_metadata read hostname 'smithi037' 2024-04-25T10:30:18.544 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:18.536+0000 7f1507fdb740 1 --2- 172.21.15.37:0/1719454315 >> [v2:172.21.15.37:6832/311552410,v1:172.21.15.37:6835/311552410] conn(0x55b877321e40 0x55b8777321e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:30:18.544 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:18.536+0000 7f1507fdb740 4 client.5481 mds_command: new command op to 5316 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-25T10:30:18.544 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:18.536+0000 7f1507fdb740 1 -- 172.21.15.37:0/1719454315 --> [v2:172.21.15.37:6832/311552410,v1:172.21.15.37:6835/311552410] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55b877677350 con 0x55b877321e40 2024-04-25T10:30:18.544 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:18.536+0000 7f14fad9d640 1 --2- 172.21.15.37:0/1719454315 >> [v2:172.21.15.37:6832/311552410,v1:172.21.15.37:6835/311552410] conn(0x55b877321e40 0x55b8777321e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:30:18.544 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:18.537+0000 7f14fad9d640 1 --2- 172.21.15.37:0/1719454315 >> [v2:172.21.15.37:6832/311552410,v1:172.21.15.37:6835/311552410] conn(0x55b877321e40 0x55b8777321e0 crc :-1 s=READY pgs=9 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=3186edae5ca1f7c server_cookie=b711ab6475d2c186 in_seq=0 out_seq=0 2024-04-25T10:30:18.544 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:18.537+0000 7f14d9ffb640 10 client.5481 ms_handle_connect on v2:172.21.15.37:6832/311552410 2024-04-25T10:30:18.545 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:18.537+0000 7f14d9ffb640 1 -- 172.21.15.37:0/1719454315 <== mds.0 v2:172.21.15.37:6832/311552410 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23648 (crc 0 0 0) 0x55b877677350 con 0x55b877321e40 2024-04-25T10:30:18.545 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:18.537+0000 7f14d9ffb640 10 client.5481 handle_command_reply: tid=0 2024-04-25T10:30:18.545 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:18.537+0000 7f1507fdb740 1 client.5481 shutdown 2024-04-25T10:30:18.545 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:18.538+0000 7f1507fdb740 1 -- 172.21.15.37:0/1719454315 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55b877342590 msgr2=0x55b87772ab30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:30:18.545 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:18.538+0000 7f1507fdb740 1 --2- 172.21.15.37:0/1719454315 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55b877342590 0x55b87772ab30 secure :-1 s=READY pgs=446 cs=0 l=1 rev1=1 crypto rx=0x7f14e40b4420 tx=0x7f14e403cee0 comp rx=0 tx=0).stop 2024-04-25T10:30:18.545 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:18.538+0000 7f1507fdb740 1 -- 172.21.15.37:0/1719454315 shutdown_connections 2024-04-25T10:30:18.545 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:18.538+0000 7f1507fdb740 1 --2- 172.21.15.37:0/1719454315 >> [v2:172.21.15.37:6832/311552410,v1:172.21.15.37:6835/311552410] conn(0x55b877321e40 0x55b8777321e0 crc :-1 s=READY pgs=9 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:30:18.545 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:18.538+0000 7f1507fdb740 1 --2- 172.21.15.37:0/1719454315 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x55b877363270 0x55b87772b070 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:30:18.545 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:18.538+0000 7f1507fdb740 1 --2- 172.21.15.37:0/1719454315 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55b877342590 0x55b87772ab30 unknown :-1 s=CLOSED pgs=446 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:30:18.545 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:18.538+0000 7f14d9ffb640 0 client.5481 ms_handle_reset on v2:172.21.15.37:6832/311552410 2024-04-25T10:30:18.545 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:18.538+0000 7f1507fdb740 1 --2- 172.21.15.37:0/1719454315 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x55b87733a3b0 0x55b87775d2a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:30:18.545 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:18.538+0000 7f1507fdb740 1 -- 172.21.15.37:0/1719454315 >> 172.21.15.37:0/1719454315 conn(0x55b8773573c0 msgr2=0x55b8776fed30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:30:18.545 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:18.538+0000 7f1507fdb740 1 -- 172.21.15.37:0/1719454315 shutdown_connections 2024-04-25T10:30:18.545 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:18.538+0000 7f1507fdb740 1 -- 172.21.15.37:0/1719454315 wait complete. 2024-04-25T10:30:18.546 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:18.538+0000 7f1507fdb740 20 client.5481 trim_cache size 0 max 16384 2024-04-25T10:30:18.559 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:18.557+0000 7f1507fdb740 1 Processor -- start 2024-04-25T10:30:18.559 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:18.557+0000 7f1507fdb740 1 -- start start 2024-04-25T10:30:18.562 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:18.558+0000 7f1507fdb740 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55b87733a3b0 0x55b877732ac0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:30:18.563 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:18.558+0000 7f1507fdb740 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x55b877342590 0x55b87772aa70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:30:18.563 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:18.558+0000 7f1507fdb740 1 --2- >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x55b877363270 0x55b87772afb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:30:18.563 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:18.558+0000 7f1507fdb740 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 -- 0x7f14fc201e30 con 0x55b87733a3b0 2024-04-25T10:30:18.563 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:18.558+0000 7f1507fdb740 1 -- --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_getmap magic: 0 -- 0x7f14fc1f9e90 con 0x55b877342590 2024-04-25T10:30:18.563 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:18.558+0000 7f1507fdb740 1 -- --> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] -- mon_getmap magic: 0 -- 0x7f14fc2018f0 con 0x55b877363270 2024-04-25T10:30:18.563 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:18.558+0000 7f14fad9d640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55b87733a3b0 0x55b877732ac0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:30:18.563 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:18.558+0000 7f14fad9d640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55b87733a3b0 0x55b877732ac0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.37:3300/0 says I am v2:172.21.15.37:52806/0 (socket says 172.21.15.37:52806) 2024-04-25T10:30:18.563 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:18.558+0000 7f14fad9d640 1 -- 172.21.15.37:0/3999357521 learned_addr learned my addr 172.21.15.37:0/3999357521 (peer_addr_for_me v2:172.21.15.37:0/0) 2024-04-25T10:30:18.563 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:18.559+0000 7f14fad9d640 1 -- 172.21.15.37:0/3999357521 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x55b877363270 msgr2=0x55b87772afb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:30:18.563 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:18.559+0000 7f14fad9d640 1 --2- 172.21.15.37:0/3999357521 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x55b877363270 0x55b87772afb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:30:18.563 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:18.559+0000 7f14fad9d640 1 -- 172.21.15.37:0/3999357521 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x55b877342590 msgr2=0x55b87772aa70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:30:18.563 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:18.559+0000 7f14fad9d640 1 --2- 172.21.15.37:0/3999357521 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x55b877342590 0x55b87772aa70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:30:18.564 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:18.559+0000 7f14fad9d640 1 -- 172.21.15.37:0/3999357521 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55b87776d540 con 0x55b87733a3b0 2024-04-25T10:30:18.564 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:18.559+0000 7f14fad9d640 1 --2- 172.21.15.37:0/3999357521 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55b87733a3b0 0x55b877732ac0 secure :-1 s=READY pgs=447 cs=0 l=1 rev1=1 crypto rx=0x7f14fc147d90 tx=0x7f14f4013000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c74c9bc14fdc25b5 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:30:18.564 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:18.559+0000 7f14dbfff640 1 -- 172.21.15.37:0/3999357521 <== mon.0 v2:172.21.15.37:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f14f402f070 con 0x55b87733a3b0 2024-04-25T10:30:18.564 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:18.559+0000 7f14dbfff640 1 -- 172.21.15.37:0/3999357521 <== mon.0 v2:172.21.15.37:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f14f400de70 con 0x55b87733a3b0 2024-04-25T10:30:18.564 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:18.559+0000 7f14dbfff640 1 -- 172.21.15.37:0/3999357521 <== mon.0 v2:172.21.15.37:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f14f4022860 con 0x55b87733a3b0 2024-04-25T10:30:18.564 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:18.560+0000 7f1507fdb740 1 -- 172.21.15.37:0/3999357521 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55b87733a3b0 msgr2=0x55b877732ac0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:30:18.564 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:18.560+0000 7f1507fdb740 1 --2- 172.21.15.37:0/3999357521 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55b87733a3b0 0x55b877732ac0 secure :-1 s=READY pgs=447 cs=0 l=1 rev1=1 crypto rx=0x7f14fc147d90 tx=0x7f14f4013000 comp rx=0 tx=0).stop 2024-04-25T10:30:18.564 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:18.560+0000 7f1507fdb740 1 -- 172.21.15.37:0/3999357521 shutdown_connections 2024-04-25T10:30:18.564 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:18.560+0000 7f1507fdb740 1 --2- 172.21.15.37:0/3999357521 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x55b877363270 0x55b87772afb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:30:18.564 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:18.560+0000 7f1507fdb740 1 --2- 172.21.15.37:0/3999357521 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x55b877342590 0x55b87772aa70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:30:18.564 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:18.560+0000 7f1507fdb740 1 --2- 172.21.15.37:0/3999357521 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55b87733a3b0 0x55b877732ac0 unknown :-1 s=CLOSED pgs=447 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:30:18.564 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:18.560+0000 7f1507fdb740 1 -- 172.21.15.37:0/3999357521 >> 172.21.15.37:0/3999357521 conn(0x55b8773573c0 msgr2=0x55b877702ea0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:30:18.564 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:18.560+0000 7f1507fdb740 1 -- 172.21.15.37:0/3999357521 shutdown_connections 2024-04-25T10:30:18.564 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:18.560+0000 7f1507fdb740 1 -- 172.21.15.37:0/3999357521 wait complete. 2024-04-25T10:30:18.564 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:18.560+0000 7f1507fdb740 1 -- start start 2024-04-25T10:30:18.565 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:18.561+0000 7f1507fdb740 1 Processor -- start 2024-04-25T10:30:18.570 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:18.562+0000 7f1507fdb740 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55b87733a3b0 0x55b8777c88e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:30:18.570 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:18.562+0000 7f1507fdb740 1 --2- >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x55b877342590 0x55b877704dd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:30:18.570 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:18.562+0000 7f1507fdb740 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x55b877363270 0x55b87776bd50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:30:18.570 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:18.562+0000 7f1507fdb740 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 -- 0x55b87772c750 con 0x55b87733a3b0 2024-04-25T10:30:18.570 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:18.562+0000 7f1507fdb740 1 -- --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_getmap magic: 0 -- 0x55b87772d780 con 0x55b877363270 2024-04-25T10:30:18.570 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:18.562+0000 7f1507fdb740 1 -- --> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] -- mon_getmap magic: 0 -- 0x55b877730840 con 0x55b877342590 2024-04-25T10:30:18.570 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:18.562+0000 7f1507fdb740 10 client.0 _ll_register_callbacks cb 0x7f14e407be80 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-25T10:30:18.570 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:18.562+0000 7f14fad9d640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55b87733a3b0 0x55b8777c88e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:30:18.570 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:18.562+0000 7f14fad9d640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55b87733a3b0 0x55b8777c88e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.37:3300/0 says I am v2:172.21.15.37:52816/0 (socket says 172.21.15.37:52816) 2024-04-25T10:30:18.571 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:18.562+0000 7f14fad9d640 1 -- 172.21.15.37:0/4067880637 learned_addr learned my addr 172.21.15.37:0/4067880637 (peer_addr_for_me v2:172.21.15.37:0/0) 2024-04-25T10:30:18.571 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:18.562+0000 7f14fa59c640 1 --2- 172.21.15.37:0/4067880637 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x55b877342590 0x55b877704dd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:30:18.571 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:18.562+0000 7f14fb59e640 1 --2- 172.21.15.37:0/4067880637 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x55b877363270 0x55b87776bd50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:30:18.571 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:18.562+0000 7f14fad9d640 1 -- 172.21.15.37:0/4067880637 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x55b877342590 msgr2=0x55b877704dd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:30:18.571 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:18.562+0000 7f14fad9d640 1 --2- 172.21.15.37:0/4067880637 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x55b877342590 0x55b877704dd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:30:18.571 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:18.562+0000 7f14fad9d640 1 -- 172.21.15.37:0/4067880637 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x55b877363270 msgr2=0x55b87776bd50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:30:18.571 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:18.562+0000 7f14fad9d640 1 --2- 172.21.15.37:0/4067880637 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x55b877363270 0x55b87776bd50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:30:18.571 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:18.562+0000 7f14fad9d640 1 -- 172.21.15.37:0/4067880637 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55b877732180 con 0x55b87733a3b0 2024-04-25T10:30:18.571 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:18.562+0000 7f14fad9d640 1 --2- 172.21.15.37:0/4067880637 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55b87733a3b0 0x55b8777c88e0 secure :-1 s=READY pgs=448 cs=0 l=1 rev1=1 crypto rx=0x7f14f402d040 tx=0x7f14f4004640 comp rx=0 tx=0).ready entity=mon.0 client_cookie=2ac7bcff93740f89 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:30:18.571 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:18.562+0000 7f14d9ffb640 10 client.0 ms_handle_connect on v2:172.21.15.37:3300/0 2024-04-25T10:30:18.571 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:18.563+0000 7f14d9ffb640 1 -- 172.21.15.37:0/4067880637 <== mon.0 v2:172.21.15.37:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f14f402f070 con 0x55b87733a3b0 2024-04-25T10:30:18.571 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:18.563+0000 7f14d9ffb640 1 -- 172.21.15.37:0/4067880637 <== mon.0 v2:172.21.15.37:3300/0 2 ==== osd_map(36..36 src has 1..36) ==== 5709+0+0 (secure 0 0 0) 0x7f14f4022a70 con 0x55b87733a3b0 2024-04-25T10:30:18.571 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:18.563+0000 7f1507fdb740 1 -- 172.21.15.37:0/4067880637 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f14f400de70 con 0x55b87733a3b0 2024-04-25T10:30:18.571 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:18.563+0000 7f14d9ffb640 1 -- 172.21.15.37:0/4067880637 <== mon.0 v2:172.21.15.37:3300/0 3 ==== mon_get_version_reply(handle=1 version=17) ==== 24+0+0 (secure 0 0 0) 0x7f14f4022c70 con 0x55b87733a3b0 2024-04-25T10:30:18.572 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:18.563+0000 7f1507fdb740 10 client.5487 fetch_fsmap learned FSMap version 17 2024-04-25T10:30:18.572 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:18.563+0000 7f1507fdb740 1 -- 172.21.15.37:0/4067880637 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({config=0+,fsmap=17,monmap=2+}) -- 0x55b877705830 con 0x55b87733a3b0 2024-04-25T10:30:18.572 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:18.563+0000 7f14d9ffb640 1 -- 172.21.15.37:0/4067880637 <== mon.0 v2:172.21.15.37:3300/0 4 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f14f400de70 con 0x55b87733a3b0 2024-04-25T10:30:18.572 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:18.563+0000 7f14d9ffb640 1 -- 172.21.15.37:0/4067880637 <== mon.0 v2:172.21.15.37:3300/0 5 ==== fsmap(e 17) ==== 3494+0+0 (secure 0 0 0) 0x7f14f4029ce0 con 0x55b87733a3b0 2024-04-25T10:30:18.572 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:18.563+0000 7f1507fdb740 10 client.5487 fetch_fsmap finished waiting for FSMap version 17 2024-04-25T10:30:18.572 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:18.563+0000 7f1507fdb740 10 client.5487 resolve_mds: resolved name 'a' to daemon mds.a 2024-04-25T10:30:18.572 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:18.563+0000 7f1507fdb740 20 client.5487 populate_metadata read hostname 'smithi037' 2024-04-25T10:30:18.572 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:18.563+0000 7f1507fdb740 1 --2- 172.21.15.37:0/4067880637 >> [v2:172.21.15.37:6832/311552410,v1:172.21.15.37:6835/311552410] conn(0x55b877724890 0x55b8776fe300 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:30:18.572 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:18.563+0000 7f1507fdb740 4 client.5487 mds_command: new command op to 5316 tid=0 multi_id=0 [{"prefix": "status"}] 2024-04-25T10:30:18.572 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:18.563+0000 7f1507fdb740 1 -- 172.21.15.37:0/4067880637 --> [v2:172.21.15.37:6832/311552410,v1:172.21.15.37:6835/311552410] -- command(tid 0: {"prefix": "status"}) -- 0x55b8776711c0 con 0x55b877724890 2024-04-25T10:30:18.572 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:18.563+0000 7f14fa59c640 1 --2- 172.21.15.37:0/4067880637 >> [v2:172.21.15.37:6832/311552410,v1:172.21.15.37:6835/311552410] conn(0x55b877724890 0x55b8776fe300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:30:18.573 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:18.564+0000 7f14d9ffb640 10 client.5487 ms_handle_connect on v2:172.21.15.37:6832/311552410 2024-04-25T10:30:18.573 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:18.564+0000 7f14fa59c640 1 --2- 172.21.15.37:0/4067880637 >> [v2:172.21.15.37:6832/311552410,v1:172.21.15.37:6835/311552410] conn(0x55b877724890 0x55b8776fe300 crc :-1 s=READY pgs=10 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=e69f0ac20c7e20c6 server_cookie=f2bf49c60e11bf4 in_seq=0 out_seq=0 2024-04-25T10:30:18.573 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:18.564+0000 7f14d9ffb640 1 -- 172.21.15.37:0/4067880637 <== mds.0 v2:172.21.15.37:6832/311552410 1 ==== command_reply(tid 0: 0 ) ==== 8+0+331 (crc 0 0 0) 0x55b8776711c0 con 0x55b877724890 2024-04-25T10:30:18.573 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:18.564+0000 7f14d9ffb640 10 client.5487 handle_command_reply: tid=0 2024-04-25T10:30:18.573 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:18.564+0000 7f1507fdb740 1 client.5487 shutdown 2024-04-25T10:30:18.573 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:18.564+0000 7f1507fdb740 1 -- 172.21.15.37:0/4067880637 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55b87733a3b0 msgr2=0x55b8777c88e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:30:18.573 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:18.564+0000 7f1507fdb740 1 --2- 172.21.15.37:0/4067880637 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55b87733a3b0 0x55b8777c88e0 secure :-1 s=READY pgs=448 cs=0 l=1 rev1=1 crypto rx=0x7f14f402d040 tx=0x7f14f4004640 comp rx=0 tx=0).stop 2024-04-25T10:30:18.573 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:18.564+0000 7f1507fdb740 1 -- 172.21.15.37:0/4067880637 shutdown_connections 2024-04-25T10:30:18.573 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:18.564+0000 7f1507fdb740 1 --2- 172.21.15.37:0/4067880637 >> [v2:172.21.15.37:6832/311552410,v1:172.21.15.37:6835/311552410] conn(0x55b877724890 0x55b8776fe300 crc :-1 s=READY pgs=10 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:30:18.573 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:18.564+0000 7f1507fdb740 1 --2- 172.21.15.37:0/4067880637 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x55b877363270 0x55b87776bd50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:30:18.573 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:18.564+0000 7f1507fdb740 1 --2- 172.21.15.37:0/4067880637 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x55b877342590 0x55b877704dd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:30:18.573 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:18.564+0000 7f14d9ffb640 0 client.5487 ms_handle_reset on v2:172.21.15.37:6832/311552410 2024-04-25T10:30:18.573 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:18.564+0000 7f1507fdb740 1 --2- 172.21.15.37:0/4067880637 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55b87733a3b0 0x55b8777c88e0 unknown :-1 s=CLOSED pgs=448 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:30:18.574 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:18.564+0000 7f1507fdb740 1 -- 172.21.15.37:0/4067880637 >> 172.21.15.37:0/4067880637 conn(0x55b8773573c0 msgr2=0x55b87772bbb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:30:18.574 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:18.564+0000 7f1507fdb740 1 -- 172.21.15.37:0/4067880637 shutdown_connections 2024-04-25T10:30:18.574 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:18.564+0000 7f1507fdb740 1 -- 172.21.15.37:0/4067880637 wait complete. 2024-04-25T10:30:18.574 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:18.564+0000 7f1507fdb740 20 client.5487 trim_cache size 0 max 16384 2024-04-25T10:30:18.574 INFO:teuthology.orchestra.run.smithi037.stdout:{ 2024-04-25T10:30:18.574 INFO:teuthology.orchestra.run.smithi037.stdout: "cluster_fsid": "f4acda28-be34-492a-b286-994afd36f61d", 2024-04-25T10:30:18.574 INFO:teuthology.orchestra.run.smithi037.stdout: "whoami": 0, 2024-04-25T10:30:18.574 INFO:teuthology.orchestra.run.smithi037.stdout: "id": 5316, 2024-04-25T10:30:18.574 INFO:teuthology.orchestra.run.smithi037.stdout: "want_state": "up:active", 2024-04-25T10:30:18.575 INFO:teuthology.orchestra.run.smithi037.stdout: "state": "up:active", 2024-04-25T10:30:18.575 INFO:teuthology.orchestra.run.smithi037.stdout: "fs_name": "cephfs", 2024-04-25T10:30:18.575 INFO:teuthology.orchestra.run.smithi037.stdout: "rank_uptime": 5.8064783770000004, 2024-04-25T10:30:18.575 INFO:teuthology.orchestra.run.smithi037.stdout: "mdsmap_epoch": 17, 2024-04-25T10:30:18.575 INFO:teuthology.orchestra.run.smithi037.stdout: "osdmap_epoch": 36, 2024-04-25T10:30:18.575 INFO:teuthology.orchestra.run.smithi037.stdout: "osdmap_epoch_barrier": 36, 2024-04-25T10:30:18.575 INFO:teuthology.orchestra.run.smithi037.stdout: "uptime": 13.672530099999999 2024-04-25T10:30:18.575 INFO:teuthology.orchestra.run.smithi037.stdout:} 2024-04-25T10:30:18.577 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:18.565+0000 7f150104c640 1 -- 172.21.15.37:0/1182015337 >> [v2:172.21.15.203:6832/2418205237,v1:172.21.15.203:6833/2418205237] conn(0x7f14dc0744b0 msgr2=0x7f14dc0769c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:30:18.577 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:18.565+0000 7f150104c640 1 --2- 172.21.15.37:0/1182015337 >> [v2:172.21.15.203:6832/2418205237,v1:172.21.15.203:6833/2418205237] conn(0x7f14dc0744b0 0x7f14dc0769c0 secure :-1 s=READY pgs=222 cs=0 l=1 rev1=1 crypto rx=0x7f14f4006fd0 tx=0x7f14f4008040 comp rx=0 tx=0).stop 2024-04-25T10:30:18.577 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:18.565+0000 7f150104c640 1 -- 172.21.15.37:0/1182015337 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f14fc1441d0 msgr2=0x7f14fc147450 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:30:18.577 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:18.565+0000 7f150104c640 1 --2- 172.21.15.37:0/1182015337 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f14fc1441d0 0x7f14fc147450 secure :-1 s=READY pgs=444 cs=0 l=1 rev1=1 crypto rx=0x7f14e402a270 tx=0x7f14e40046c0 comp rx=0 tx=0).stop 2024-04-25T10:30:18.577 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:18.565+0000 7f150104c640 1 -- 172.21.15.37:0/1182015337 shutdown_connections 2024-04-25T10:30:18.577 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:18.565+0000 7f150104c640 1 --2- 172.21.15.37:0/1182015337 >> [v2:172.21.15.203:6832/2418205237,v1:172.21.15.203:6833/2418205237] conn(0x7f14dc0744b0 0x7f14dc0769c0 unknown :-1 s=CLOSED pgs=222 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:30:18.577 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:18.565+0000 7f150104c640 1 --2- 172.21.15.37:0/1182015337 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f14fc1f6620 0x7f14fc1f6a30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:30:18.577 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:18.565+0000 7f150104c640 1 --2- 172.21.15.37:0/1182015337 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f14fc1441d0 0x7f14fc147450 unknown :-1 s=CLOSED pgs=444 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:30:18.577 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:18.565+0000 7f150104c640 1 --2- 172.21.15.37:0/1182015337 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f14fc0cee10 0x7f14fc0b0550 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:30:18.577 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:18.565+0000 7f150104c640 1 -- 172.21.15.37:0/1182015337 >> 172.21.15.37:0/1182015337 conn(0x7f14fc0b36b0 msgr2=0x7f14fc1406b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:30:18.577 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:18.565+0000 7f150104c640 1 -- 172.21.15.37:0/1182015337 shutdown_connections 2024-04-25T10:30:18.577 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:18.565+0000 7f150104c640 1 -- 172.21.15.37:0/1182015337 wait complete. 2024-04-25T10:30:18.579 DEBUG:teuthology.orchestra.run.smithi037:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs ls 2024-04-25T10:30:18.689 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:18.687+0000 7ff96c586640 1 Processor -- start 2024-04-25T10:30:18.690 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:18.687+0000 7ff96c586640 1 -- start start 2024-04-25T10:30:18.691 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:18.689+0000 7ff96c586640 1 --2- >> v2:172.21.15.37:3300/0 conn(0x7ff964144490 0x7ff96415f760 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:30:18.691 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:18.689+0000 7ff96c586640 1 -- --> v1:172.21.15.37:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7ff9640d0e90 con 0x7ff96406c540 2024-04-25T10:30:18.691 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:18.689+0000 7ff96c586640 1 -- --> v1:172.21.15.203:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7ff96415eb80 con 0x7ff96415e570 2024-04-25T10:30:18.692 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:18.689+0000 7ff96c586640 1 -- --> v2:172.21.15.37:3300/0 -- mon_getmap magic: 0 -- 0x7ff96415ee60 con 0x7ff964144490 2024-04-25T10:30:18.692 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:18.689+0000 7ff96a322640 1 --2- >> v2:172.21.15.37:3300/0 conn(0x7ff964144490 0x7ff96415f760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:30:18.692 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:18.689+0000 7ff96a322640 1 --2- >> v2:172.21.15.37:3300/0 conn(0x7ff964144490 0x7ff96415f760 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.37:3300/0 says I am v2:172.21.15.37:52818/0 (socket says 172.21.15.37:52818) 2024-04-25T10:30:18.692 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:18.689+0000 7ff96ab23640 1 --1- >> v1:172.21.15.37:6789/0 conn(0x7ff96406c540 0x7ff96406c950 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.37:6789/0 says I am v1:172.21.15.37:41096/0 (socket says 172.21.15.37:41096) 2024-04-25T10:30:18.692 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:18.689+0000 7ff96a322640 1 -- 172.21.15.37:0/2190616576 learned_addr learned my addr 172.21.15.37:0/2190616576 (peer_addr_for_me v2:172.21.15.37:0/0) 2024-04-25T10:30:18.692 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:18.689+0000 7ff96a322640 1 -- 172.21.15.37:0/2190616576 >> v1:172.21.15.203:6789/0 conn(0x7ff96415e570 legacy=0x7ff9640d0780 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:30:18.692 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:18.689+0000 7ff96a322640 1 -- 172.21.15.37:0/2190616576 >> v1:172.21.15.37:6789/0 conn(0x7ff96406c540 legacy=0x7ff96406c950 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:30:18.692 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:18.689+0000 7ff969320640 1 -- 172.21.15.37:0/2190616576 <== mon.0 v1:172.21.15.37:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 2004593027 0 0) 0x7ff954001490 con 0x7ff96406c540 2024-04-25T10:30:18.692 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:18.689+0000 7ff96a322640 1 -- 172.21.15.37:0/2190616576 --> v2:172.21.15.37:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ff96415f140 con 0x7ff964144490 2024-04-25T10:30:18.693 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:18.690+0000 7ff96a322640 1 --2- 172.21.15.37:0/2190616576 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7ff964144490 0x7ff96415f760 secure :-1 s=READY pgs=450 cs=0 l=1 rev1=1 crypto rx=0x7ff95c009870 tx=0x7ff95c02a300 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b38d4a75b3f4ad7a server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:30:18.693 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:18.690+0000 7ff969320640 1 -- 172.21.15.37:0/2190616576 <== mon.0 v2:172.21.15.37:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff95c037070 con 0x7ff964144490 2024-04-25T10:30:18.694 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:18.690+0000 7ff969320640 1 -- 172.21.15.37:0/2190616576 <== mon.0 v2:172.21.15.37:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7ff95c02ae50 con 0x7ff964144490 2024-04-25T10:30:18.694 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:18.690+0000 7ff969320640 1 -- 172.21.15.37:0/2190616576 <== mon.0 v2:172.21.15.37:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff95c032860 con 0x7ff964144490 2024-04-25T10:30:18.694 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:18.691+0000 7ff96c586640 1 -- 172.21.15.37:0/2190616576 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7ff964144490 msgr2=0x7ff96415f760 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:30:18.694 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:18.691+0000 7ff96c586640 1 --2- 172.21.15.37:0/2190616576 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7ff964144490 0x7ff96415f760 secure :-1 s=READY pgs=450 cs=0 l=1 rev1=1 crypto rx=0x7ff95c009870 tx=0x7ff95c02a300 comp rx=0 tx=0).stop 2024-04-25T10:30:18.694 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:18.691+0000 7ff96c586640 1 -- 172.21.15.37:0/2190616576 shutdown_connections 2024-04-25T10:30:18.694 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:18.691+0000 7ff96c586640 1 --2- 172.21.15.37:0/2190616576 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7ff964144490 0x7ff96415f760 unknown :-1 s=CLOSED pgs=450 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:30:18.694 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:18.691+0000 7ff96c586640 1 -- 172.21.15.37:0/2190616576 >> 172.21.15.37:0/2190616576 conn(0x7ff9640b4560 msgr2=0x7ff9640b4970 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:30:18.694 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:18.691+0000 7ff96c586640 1 -- 172.21.15.37:0/2190616576 shutdown_connections 2024-04-25T10:30:18.694 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:18.691+0000 7ff96c586640 1 -- 172.21.15.37:0/2190616576 wait complete. 2024-04-25T10:30:18.695 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:18.693+0000 7ff96c586640 1 Processor -- start 2024-04-25T10:30:18.695 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:18.693+0000 7ff96c586640 1 -- start start 2024-04-25T10:30:18.698 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:18.694+0000 7ff96c586640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7ff96406c540 0x7ff96415aac0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:30:18.698 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:18.694+0000 7ff96c586640 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7ff964144490 0x7ff9641404e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:30:18.698 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:18.694+0000 7ff96c586640 1 --2- >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7ff96415e570 0x7ff964159280 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:30:18.698 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:18.694+0000 7ff96c586640 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 -- 0x7ff9640d2470 con 0x7ff96406c540 2024-04-25T10:30:18.698 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:18.694+0000 7ff96c586640 1 -- --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_getmap magic: 0 -- 0x7ff9640d22f0 con 0x7ff964144490 2024-04-25T10:30:18.698 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:18.694+0000 7ff96c586640 1 -- --> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] -- mon_getmap magic: 0 -- 0x7ff9640d28d0 con 0x7ff96415e570 2024-04-25T10:30:18.698 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:18.694+0000 7ff96a322640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7ff96406c540 0x7ff96415aac0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:30:18.698 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:18.694+0000 7ff96a322640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7ff96406c540 0x7ff96415aac0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.37:3300/0 says I am v2:172.21.15.37:52820/0 (socket says 172.21.15.37:52820) 2024-04-25T10:30:18.698 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:18.694+0000 7ff96a322640 1 -- 172.21.15.37:0/816431755 learned_addr learned my addr 172.21.15.37:0/816431755 (peer_addr_for_me v2:172.21.15.37:0/0) 2024-04-25T10:30:18.699 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:18.694+0000 7ff96ab23640 1 --2- 172.21.15.37:0/816431755 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7ff96415e570 0x7ff964159280 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:30:18.699 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:18.694+0000 7ff969b21640 1 --2- 172.21.15.37:0/816431755 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7ff964144490 0x7ff9641404e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:30:18.699 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:18.695+0000 7ff96a322640 1 -- 172.21.15.37:0/816431755 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7ff96415e570 msgr2=0x7ff964159280 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:30:18.699 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:18.695+0000 7ff96a322640 1 --2- 172.21.15.37:0/816431755 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7ff96415e570 0x7ff964159280 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:30:18.699 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:18.695+0000 7ff96a322640 1 -- 172.21.15.37:0/816431755 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7ff964144490 msgr2=0x7ff9641404e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:30:18.699 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:18.695+0000 7ff96a322640 1 --2- 172.21.15.37:0/816431755 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7ff964144490 0x7ff9641404e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:30:18.699 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:18.695+0000 7ff96a322640 1 -- 172.21.15.37:0/816431755 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ff9641597c0 con 0x7ff96406c540 2024-04-25T10:30:18.699 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:18.695+0000 7ff96a322640 1 --2- 172.21.15.37:0/816431755 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7ff96406c540 0x7ff96415aac0 secure :-1 s=READY pgs=451 cs=0 l=1 rev1=1 crypto rx=0x7ff95c009870 tx=0x7ff95c0046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:30:18.699 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:18.695+0000 7ff962ffd640 1 -- 172.21.15.37:0/816431755 <== mon.0 v2:172.21.15.37:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff95c037040 con 0x7ff96406c540 2024-04-25T10:30:18.699 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:18.695+0000 7ff962ffd640 1 -- 172.21.15.37:0/816431755 <== mon.0 v2:172.21.15.37:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7ff95c004230 con 0x7ff96406c540 2024-04-25T10:30:18.699 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:18.695+0000 7ff96c586640 1 -- 172.21.15.37:0/816431755 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7ff9641581e0 con 0x7ff96406c540 2024-04-25T10:30:18.699 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:18.695+0000 7ff962ffd640 1 -- 172.21.15.37:0/816431755 <== mon.0 v2:172.21.15.37:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff95c032e40 con 0x7ff96406c540 2024-04-25T10:30:18.699 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:18.695+0000 7ff96c586640 1 -- 172.21.15.37:0/816431755 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7ff9641587a0 con 0x7ff96406c540 2024-04-25T10:30:18.699 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:18.696+0000 7ff96c586640 1 -- 172.21.15.37:0/816431755 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7ff96406a140 con 0x7ff96406c540 2024-04-25T10:30:18.700 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:18.697+0000 7ff962ffd640 1 -- 172.21.15.37:0/816431755 <== mon.0 v2:172.21.15.37:3300/0 4 ==== mgrmap(e 5) ==== 96782+0+0 (secure 0 0 0) 0x7ff95c03a070 con 0x7ff96406c540 2024-04-25T10:30:18.700 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:18.697+0000 7ff962ffd640 1 --2- 172.21.15.37:0/816431755 >> [v2:172.21.15.203:6832/2418205237,v1:172.21.15.203:6833/2418205237] conn(0x7ff9500744e0 0x7ff9500769f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:30:18.701 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:18.697+0000 7ff962ffd640 1 -- 172.21.15.37:0/816431755 <== mon.0 v2:172.21.15.37:3300/0 5 ==== osd_map(36..36 src has 1..36) ==== 5709+0+0 (secure 0 0 0) 0x7ff95c0b5900 con 0x7ff96406c540 2024-04-25T10:30:18.701 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:18.697+0000 7ff969b21640 1 --2- 172.21.15.37:0/816431755 >> [v2:172.21.15.203:6832/2418205237,v1:172.21.15.203:6833/2418205237] conn(0x7ff9500744e0 0x7ff9500769f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:30:18.701 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:18.698+0000 7ff969b21640 1 --2- 172.21.15.37:0/816431755 >> [v2:172.21.15.203:6832/2418205237,v1:172.21.15.203:6833/2418205237] conn(0x7ff9500744e0 0x7ff9500769f0 secure :-1 s=READY pgs=223 cs=0 l=1 rev1=1 crypto rx=0x7ff9640d3830 tx=0x7ff958009040 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:30:18.704 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:18.702+0000 7ff962ffd640 1 -- 172.21.15.37:0/816431755 <== mon.0 v2:172.21.15.37:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187246 (secure 0 0 0) 0x7ff9641597c0 con 0x7ff96406c540 2024-04-25T10:30:18.929 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:18.926+0000 7ff96c586640 1 -- 172.21.15.37:0/816431755 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_command({"prefix": "fs ls"} v 0) -- 0x7ff9641602d0 con 0x7ff96406c540 2024-04-25T10:30:18.929 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:18.927+0000 7ff962ffd640 1 -- 172.21.15.37:0/816431755 <== mon.0 v2:172.21.15.37:3300/0 7 ==== mon_command_ack([{"prefix": "fs ls"}]=0 v17) ==== 53+0+73 (secure 0 0 0) 0x7ff95c041020 con 0x7ff96406c540 2024-04-25T10:30:18.929 INFO:teuthology.orchestra.run.smithi037.stdout:name: cephfs, metadata pool: cephfs_metadata, data pools: [cephfs_data ] 2024-04-25T10:30:18.931 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:18.929+0000 7ff96c586640 1 -- 172.21.15.37:0/816431755 >> [v2:172.21.15.203:6832/2418205237,v1:172.21.15.203:6833/2418205237] conn(0x7ff9500744e0 msgr2=0x7ff9500769f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:30:18.931 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:18.929+0000 7ff96c586640 1 --2- 172.21.15.37:0/816431755 >> [v2:172.21.15.203:6832/2418205237,v1:172.21.15.203:6833/2418205237] conn(0x7ff9500744e0 0x7ff9500769f0 secure :-1 s=READY pgs=223 cs=0 l=1 rev1=1 crypto rx=0x7ff9640d3830 tx=0x7ff958009040 comp rx=0 tx=0).stop 2024-04-25T10:30:18.931 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:18.929+0000 7ff96c586640 1 -- 172.21.15.37:0/816431755 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7ff96406c540 msgr2=0x7ff96415aac0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:30:18.931 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:18.929+0000 7ff96c586640 1 --2- 172.21.15.37:0/816431755 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7ff96406c540 0x7ff96415aac0 secure :-1 s=READY pgs=451 cs=0 l=1 rev1=1 crypto rx=0x7ff95c009870 tx=0x7ff95c0046c0 comp rx=0 tx=0).stop 2024-04-25T10:30:18.931 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:18.929+0000 7ff96c586640 1 -- 172.21.15.37:0/816431755 shutdown_connections 2024-04-25T10:30:18.931 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:18.929+0000 7ff96c586640 1 --2- 172.21.15.37:0/816431755 >> [v2:172.21.15.203:6832/2418205237,v1:172.21.15.203:6833/2418205237] conn(0x7ff9500744e0 0x7ff9500769f0 unknown :-1 s=CLOSED pgs=223 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:30:18.932 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:18.929+0000 7ff96c586640 1 --2- 172.21.15.37:0/816431755 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7ff96415e570 0x7ff964159280 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:30:18.932 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:18.929+0000 7ff96c586640 1 --2- 172.21.15.37:0/816431755 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7ff964144490 0x7ff9641404e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:30:18.932 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:18.929+0000 7ff96c586640 1 --2- 172.21.15.37:0/816431755 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7ff96406c540 0x7ff96415aac0 unknown :-1 s=CLOSED pgs=451 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:30:18.932 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:18.929+0000 7ff96c586640 1 -- 172.21.15.37:0/816431755 >> 172.21.15.37:0/816431755 conn(0x7ff9640b4560 msgr2=0x7ff964160df0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:30:18.932 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:18.929+0000 7ff96c586640 1 -- 172.21.15.37:0/816431755 shutdown_connections 2024-04-25T10:30:18.932 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:18.929+0000 7ff96c586640 1 -- 172.21.15.37:0/816431755 wait complete. 2024-04-25T10:30:18.942 INFO:tasks.cephfs.mount:Mounting default Ceph FS; just confirmed its presence on cluster 2024-04-25T10:30:18.942 INFO:tasks.cephfs.mount:Mounting Ceph FS. Following are details of mount; remember "None" represents Python type None - 2024-04-25T10:30:18.942 INFO:tasks.cephfs.mount:self.client_remote.hostname = smithi037.front.sepia.ceph.com 2024-04-25T10:30:18.942 INFO:tasks.cephfs.mount:self.client.name = client.0 2024-04-25T10:30:18.942 INFO:tasks.cephfs.mount:self.hostfs_mntpt = /home/ubuntu/cephtest/mnt.0 2024-04-25T10:30:18.942 INFO:tasks.cephfs.mount:self.cephfs_name = None 2024-04-25T10:30:18.942 INFO:tasks.cephfs.mount:self.cephfs_mntpt = None 2024-04-25T10:30:18.942 INFO:tasks.cephfs.mount:self.client_keyring_path = None 2024-04-25T10:30:18.942 INFO:tasks.cephfs.mount:Setting the 'ceph-ns--home-ubuntu-cephtest-mnt.0' netns for '/home/ubuntu/cephtest/mnt.0' 2024-04-25T10:30:18.943 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T10:30:18.943 DEBUG:teuthology.orchestra.run.smithi037:> ip addr 2024-04-25T10:30:18.959 INFO:teuthology.orchestra.run.smithi037.stdout:1: lo: mtu 65536 qdisc noqueue state UNKNOWN group default qlen 1000 2024-04-25T10:30:18.959 INFO:teuthology.orchestra.run.smithi037.stdout: link/loopback 00:00:00:00:00:00 brd 00:00:00:00:00:00 2024-04-25T10:30:18.959 INFO:teuthology.orchestra.run.smithi037.stdout: inet 127.0.0.1/8 scope host lo 2024-04-25T10:30:18.959 INFO:teuthology.orchestra.run.smithi037.stdout: valid_lft forever preferred_lft forever 2024-04-25T10:30:18.959 INFO:teuthology.orchestra.run.smithi037.stdout: inet6 ::1/128 scope host 2024-04-25T10:30:18.959 INFO:teuthology.orchestra.run.smithi037.stdout: valid_lft forever preferred_lft forever 2024-04-25T10:30:18.959 INFO:teuthology.orchestra.run.smithi037.stdout:2: eno1: mtu 1500 qdisc mq state DOWN group default qlen 1000 2024-04-25T10:30:18.959 INFO:teuthology.orchestra.run.smithi037.stdout: link/ether 0c:c4:7a:6b:fd:18 brd ff:ff:ff:ff:ff:ff 2024-04-25T10:30:18.960 INFO:teuthology.orchestra.run.smithi037.stdout: altname enp7s0f0 2024-04-25T10:30:18.960 INFO:teuthology.orchestra.run.smithi037.stdout:3: eno2: mtu 1500 qdisc mq state DOWN group default qlen 1000 2024-04-25T10:30:18.960 INFO:teuthology.orchestra.run.smithi037.stdout: link/ether 0c:c4:7a:6b:fd:19 brd ff:ff:ff:ff:ff:ff 2024-04-25T10:30:18.960 INFO:teuthology.orchestra.run.smithi037.stdout: altname enp7s0f1 2024-04-25T10:30:18.960 INFO:teuthology.orchestra.run.smithi037.stdout:4: ens1f0: mtu 1500 qdisc mq state UP group default qlen 1000 2024-04-25T10:30:18.960 INFO:teuthology.orchestra.run.smithi037.stdout: link/ether 0c:c4:7a:88:77:94 brd ff:ff:ff:ff:ff:ff 2024-04-25T10:30:18.960 INFO:teuthology.orchestra.run.smithi037.stdout: altname enp1s0f0 2024-04-25T10:30:18.960 INFO:teuthology.orchestra.run.smithi037.stdout: inet 172.21.15.37/20 brd 172.21.15.255 scope global dynamic noprefixroute ens1f0 2024-04-25T10:30:18.960 INFO:teuthology.orchestra.run.smithi037.stdout: valid_lft 42649sec preferred_lft 42649sec 2024-04-25T10:30:18.960 INFO:teuthology.orchestra.run.smithi037.stdout: inet6 fe80::ec4:7aff:fe88:7794/64 scope link noprefixroute 2024-04-25T10:30:18.960 INFO:teuthology.orchestra.run.smithi037.stdout: valid_lft forever preferred_lft forever 2024-04-25T10:30:18.960 INFO:teuthology.orchestra.run.smithi037.stdout:5: ens1f1: mtu 1500 qdisc mq state DOWN group default qlen 1000 2024-04-25T10:30:18.960 INFO:teuthology.orchestra.run.smithi037.stdout: link/ether 0c:c4:7a:88:77:95 brd ff:ff:ff:ff:ff:ff 2024-04-25T10:30:18.961 INFO:teuthology.orchestra.run.smithi037.stdout: altname enp1s0f1 2024-04-25T10:30:18.961 INFO:teuthology.orchestra.run.smithi037.stdout:6: ceph-brx: mtu 1500 qdisc noqueue state UP group default qlen 1000 2024-04-25T10:30:18.961 INFO:teuthology.orchestra.run.smithi037.stdout: link/ether ce:87:f0:58:c7:15 brd ff:ff:ff:ff:ff:ff 2024-04-25T10:30:18.961 INFO:teuthology.orchestra.run.smithi037.stdout: inet 192.168.255.254/16 brd 192.168.255.255 scope global ceph-brx 2024-04-25T10:30:18.961 INFO:teuthology.orchestra.run.smithi037.stdout: valid_lft forever preferred_lft forever 2024-04-25T10:30:18.961 INFO:teuthology.orchestra.run.smithi037.stdout: inet6 fe80::9827:f7ff:febf:df95/64 scope link 2024-04-25T10:30:18.961 INFO:teuthology.orchestra.run.smithi037.stdout: valid_lft forever preferred_lft forever 2024-04-25T10:30:18.961 INFO:teuthology.orchestra.run.smithi037.stdout:7: brx.0@if2: mtu 1500 qdisc noqueue master ceph-brx state UP group default qlen 1000 2024-04-25T10:30:18.961 INFO:teuthology.orchestra.run.smithi037.stdout: link/ether ce:87:f0:58:c7:15 brd ff:ff:ff:ff:ff:ff link-netns ceph-ns--home-ubuntu-cephtest-mnt.0 2024-04-25T10:30:18.961 INFO:teuthology.orchestra.run.smithi037.stdout: inet6 fe80::cc87:f0ff:fe58:c715/64 scope link 2024-04-25T10:30:18.961 INFO:teuthology.orchestra.run.smithi037.stdout: valid_lft forever preferred_lft forever 2024-04-25T10:30:18.961 INFO:teuthology.orchestra.run.smithi037.stdout:8: brx.1@if2: mtu 1500 qdisc noqueue master ceph-brx state UP group default qlen 1000 2024-04-25T10:30:18.961 INFO:teuthology.orchestra.run.smithi037.stdout: link/ether fa:eb:96:ac:44:0e brd ff:ff:ff:ff:ff:ff link-netns ceph-ns--home-ubuntu-cephtest-mnt.2 2024-04-25T10:30:18.961 INFO:teuthology.orchestra.run.smithi037.stdout: inet6 fe80::f8eb:96ff:feac:440e/64 scope link 2024-04-25T10:30:18.961 INFO:teuthology.orchestra.run.smithi037.stdout: valid_lft forever preferred_lft forever 2024-04-25T10:30:18.962 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T10:30:18.962 DEBUG:teuthology.orchestra.run.smithi037:> echo 1 | sudo tee /proc/sys/net/ipv4/ip_forward 2024-04-25T10:30:19.022 INFO:teuthology.orchestra.run.smithi037.stdout:1 2024-04-25T10:30:19.023 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T10:30:19.023 DEBUG:teuthology.orchestra.run.smithi037:> route 2024-04-25T10:30:19.081 INFO:teuthology.orchestra.run.smithi037.stdout:Kernel IP routing table 2024-04-25T10:30:19.081 INFO:teuthology.orchestra.run.smithi037.stdout:Destination Gateway Genmask Flags Metric Ref Use Iface 2024-04-25T10:30:19.081 INFO:teuthology.orchestra.run.smithi037.stdout:default _gateway 0.0.0.0 UG 100 0 0 ens1f0 2024-04-25T10:30:19.081 INFO:teuthology.orchestra.run.smithi037.stdout:172.21.0.0 0.0.0.0 255.255.240.0 U 100 0 0 ens1f0 2024-04-25T10:30:19.081 INFO:teuthology.orchestra.run.smithi037.stdout:192.168.0.0 0.0.0.0 255.255.0.0 U 0 0 0 ceph-brx 2024-04-25T10:30:19.081 DEBUG:teuthology.orchestra.run.smithi037:> (cd / && exec stdin-killer --timeout=300 -- bash -c ' 2024-04-25T10:30:19.082 DEBUG:teuthology.orchestra.run.smithi037:> set -e 2024-04-25T10:30:19.082 DEBUG:teuthology.orchestra.run.smithi037:> sudo iptables -A FORWARD -o ens1f0 -i ceph-brx -j ACCEPT 2024-04-25T10:30:19.082 DEBUG:teuthology.orchestra.run.smithi037:> sudo iptables -A FORWARD -i ens1f0 -o ceph-brx -j ACCEPT 2024-04-25T10:30:19.082 DEBUG:teuthology.orchestra.run.smithi037:> sudo iptables -t nat -A POSTROUTING -s 192.168.255.254/16 -o ens1f0 -j MASQUERADE 2024-04-25T10:30:19.082 DEBUG:teuthology.orchestra.run.smithi037:> ') 2024-04-25T10:30:19.170 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:19 stdin-killer INFO: expiration expected; waiting 300 seconds for command to complete 2024-04-25T10:30:19.201 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:19 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-04-25T10:30:19.207 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T10:30:19.207 DEBUG:teuthology.orchestra.run.smithi037:> ip netns list 2024-04-25T10:30:19.223 INFO:teuthology.orchestra.run.smithi037.stdout:ceph-ns--home-ubuntu-cephtest-mnt.2 (id: 1) 2024-04-25T10:30:19.223 INFO:teuthology.orchestra.run.smithi037.stdout:ceph-ns--home-ubuntu-cephtest-mnt.0 (id: 0) 2024-04-25T10:30:19.223 INFO:tasks.cephfs.mount:Resuming the 'ceph-ns--home-ubuntu-cephtest-mnt.0' netns for '/home/ubuntu/cephtest/mnt.0' 2024-04-25T10:30:19.223 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T10:30:19.223 DEBUG:teuthology.orchestra.run.smithi037:> sudo ip link set brx.0 up 2024-04-25T10:30:19.287 INFO:tasks.cephfs.fuse_mount:Client client.0 config is {} 2024-04-25T10:30:19.287 INFO:teuthology.orchestra.run:Running command with timeout 60 2024-04-25T10:30:19.288 DEBUG:teuthology.orchestra.run.smithi037:> mkdir -p -v /home/ubuntu/cephtest/mnt.0 2024-04-25T10:30:19.342 INFO:teuthology.orchestra.run.smithi037.stdout:mkdir: created directory '/home/ubuntu/cephtest/mnt.0' 2024-04-25T10:30:19.343 INFO:teuthology.orchestra.run:Running command with timeout 60 2024-04-25T10:30:19.343 DEBUG:teuthology.orchestra.run.smithi037:> chmod 0000 /home/ubuntu/cephtest/mnt.0 2024-04-25T10:30:19.398 DEBUG:teuthology.orchestra.run.smithi037:> sudo modprobe fuse 2024-04-25T10:30:19.463 DEBUG:teuthology.orchestra.run.smithi037:> cat /proc/self/mounts | awk '{print $2}' 2024-04-25T10:30:19.519 INFO:teuthology.orchestra.run.smithi037.stdout:/proc 2024-04-25T10:30:19.519 INFO:teuthology.orchestra.run.smithi037.stdout:/sys 2024-04-25T10:30:19.519 INFO:teuthology.orchestra.run.smithi037.stdout:/dev 2024-04-25T10:30:19.520 INFO:teuthology.orchestra.run.smithi037.stdout:/sys/kernel/security 2024-04-25T10:30:19.520 INFO:teuthology.orchestra.run.smithi037.stdout:/dev/shm 2024-04-25T10:30:19.520 INFO:teuthology.orchestra.run.smithi037.stdout:/dev/pts 2024-04-25T10:30:19.520 INFO:teuthology.orchestra.run.smithi037.stdout:/run 2024-04-25T10:30:19.520 INFO:teuthology.orchestra.run.smithi037.stdout:/sys/fs/cgroup 2024-04-25T10:30:19.520 INFO:teuthology.orchestra.run.smithi037.stdout:/sys/fs/pstore 2024-04-25T10:30:19.520 INFO:teuthology.orchestra.run.smithi037.stdout:/sys/fs/bpf 2024-04-25T10:30:19.520 INFO:teuthology.orchestra.run.smithi037.stdout:/ 2024-04-25T10:30:19.520 INFO:teuthology.orchestra.run.smithi037.stdout:/sys/fs/selinux 2024-04-25T10:30:19.520 INFO:teuthology.orchestra.run.smithi037.stdout:/proc/sys/fs/binfmt_misc 2024-04-25T10:30:19.520 INFO:teuthology.orchestra.run.smithi037.stdout:/dev/hugepages 2024-04-25T10:30:19.520 INFO:teuthology.orchestra.run.smithi037.stdout:/dev/mqueue 2024-04-25T10:30:19.521 INFO:teuthology.orchestra.run.smithi037.stdout:/sys/kernel/debug 2024-04-25T10:30:19.521 INFO:teuthology.orchestra.run.smithi037.stdout:/sys/kernel/tracing 2024-04-25T10:30:19.521 INFO:teuthology.orchestra.run.smithi037.stdout:/sys/fs/fuse/connections 2024-04-25T10:30:19.521 INFO:teuthology.orchestra.run.smithi037.stdout:/sys/kernel/config 2024-04-25T10:30:19.521 INFO:teuthology.orchestra.run.smithi037.stdout:/run/credentials/systemd-sysusers.service 2024-04-25T10:30:19.521 INFO:teuthology.orchestra.run.smithi037.stdout:/run/credentials/systemd-sysctl.service 2024-04-25T10:30:19.521 INFO:teuthology.orchestra.run.smithi037.stdout:/run/credentials/systemd-tmpfiles-setup-dev.service 2024-04-25T10:30:19.521 INFO:teuthology.orchestra.run.smithi037.stdout:/run/credentials/systemd-tmpfiles-setup.service 2024-04-25T10:30:19.521 INFO:teuthology.orchestra.run.smithi037.stdout:/var/lib/nfs/rpc_pipefs 2024-04-25T10:30:19.521 INFO:teuthology.orchestra.run.smithi037.stdout:/run/user/1000 2024-04-25T10:30:19.521 INFO:teuthology.orchestra.run.smithi037.stdout:/var/lib/ceph 2024-04-25T10:30:19.521 INFO:teuthology.orchestra.run.smithi037.stdout:/var/lib/ceph/osd/ceph-0 2024-04-25T10:30:19.522 INFO:teuthology.orchestra.run.smithi037.stdout:/var/lib/ceph/osd/ceph-1 2024-04-25T10:30:19.522 INFO:teuthology.orchestra.run.smithi037.stdout:/var/lib/ceph/osd/ceph-2 2024-04-25T10:30:19.522 INFO:teuthology.orchestra.run.smithi037.stdout:/var/lib/ceph/osd/ceph-3 2024-04-25T10:30:19.522 INFO:teuthology.orchestra.run.smithi037.stdout:/run/netns 2024-04-25T10:30:19.522 INFO:teuthology.orchestra.run.smithi037.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 2024-04-25T10:30:19.522 INFO:teuthology.orchestra.run.smithi037.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 2024-04-25T10:30:19.522 INFO:teuthology.orchestra.run.smithi037.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 2024-04-25T10:30:19.522 INFO:teuthology.orchestra.run.smithi037.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 2024-04-25T10:30:19.522 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T10:30:19.523 DEBUG:teuthology.orchestra.run.smithi037:> ls /sys/fs/fuse/connections 2024-04-25T10:30:19.576 INFO:tasks.cephfs.fuse_mount:Pre-mount connections: [] 2024-04-25T10:30:19.576 DEBUG:teuthology.orchestra.run.smithi037:> (cd /home/ubuntu/cephtest && exec sudo nsenter --net=/var/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-fuse -f --admin-socket '/var/run/ceph/$cluster-$name.$pid.asok' /home/ubuntu/cephtest/mnt.0 --id 0) 2024-04-25T10:30:19.618 DEBUG:teuthology.orchestra.run.smithi037:> sudo modprobe fuse 2024-04-25T10:30:19.643 DEBUG:teuthology.orchestra.run.smithi037:> cat /proc/self/mounts | awk '{print $2}' 2024-04-25T10:30:19.700 INFO:teuthology.orchestra.run.smithi037.stdout:/proc 2024-04-25T10:30:19.700 INFO:teuthology.orchestra.run.smithi037.stdout:/sys 2024-04-25T10:30:19.700 INFO:teuthology.orchestra.run.smithi037.stdout:/dev 2024-04-25T10:30:19.700 INFO:teuthology.orchestra.run.smithi037.stdout:/sys/kernel/security 2024-04-25T10:30:19.700 INFO:teuthology.orchestra.run.smithi037.stdout:/dev/shm 2024-04-25T10:30:19.700 INFO:teuthology.orchestra.run.smithi037.stdout:/dev/pts 2024-04-25T10:30:19.700 INFO:teuthology.orchestra.run.smithi037.stdout:/run 2024-04-25T10:30:19.700 INFO:teuthology.orchestra.run.smithi037.stdout:/sys/fs/cgroup 2024-04-25T10:30:19.701 INFO:teuthology.orchestra.run.smithi037.stdout:/sys/fs/pstore 2024-04-25T10:30:19.701 INFO:teuthology.orchestra.run.smithi037.stdout:/sys/fs/bpf 2024-04-25T10:30:19.701 INFO:teuthology.orchestra.run.smithi037.stdout:/ 2024-04-25T10:30:19.701 INFO:teuthology.orchestra.run.smithi037.stdout:/sys/fs/selinux 2024-04-25T10:30:19.701 INFO:teuthology.orchestra.run.smithi037.stdout:/proc/sys/fs/binfmt_misc 2024-04-25T10:30:19.701 INFO:teuthology.orchestra.run.smithi037.stdout:/dev/hugepages 2024-04-25T10:30:19.701 INFO:teuthology.orchestra.run.smithi037.stdout:/dev/mqueue 2024-04-25T10:30:19.701 INFO:teuthology.orchestra.run.smithi037.stdout:/sys/kernel/debug 2024-04-25T10:30:19.701 INFO:teuthology.orchestra.run.smithi037.stdout:/sys/kernel/tracing 2024-04-25T10:30:19.701 INFO:teuthology.orchestra.run.smithi037.stdout:/sys/fs/fuse/connections 2024-04-25T10:30:19.701 INFO:teuthology.orchestra.run.smithi037.stdout:/sys/kernel/config 2024-04-25T10:30:19.701 INFO:teuthology.orchestra.run.smithi037.stdout:/run/credentials/systemd-sysusers.service 2024-04-25T10:30:19.701 INFO:teuthology.orchestra.run.smithi037.stdout:/run/credentials/systemd-sysctl.service 2024-04-25T10:30:19.701 INFO:teuthology.orchestra.run.smithi037.stdout:/run/credentials/systemd-tmpfiles-setup-dev.service 2024-04-25T10:30:19.701 INFO:teuthology.orchestra.run.smithi037.stdout:/run/credentials/systemd-tmpfiles-setup.service 2024-04-25T10:30:19.701 INFO:teuthology.orchestra.run.smithi037.stdout:/var/lib/nfs/rpc_pipefs 2024-04-25T10:30:19.701 INFO:teuthology.orchestra.run.smithi037.stdout:/run/user/1000 2024-04-25T10:30:19.702 INFO:teuthology.orchestra.run.smithi037.stdout:/var/lib/ceph 2024-04-25T10:30:19.702 INFO:teuthology.orchestra.run.smithi037.stdout:/var/lib/ceph/osd/ceph-0 2024-04-25T10:30:19.702 INFO:teuthology.orchestra.run.smithi037.stdout:/var/lib/ceph/osd/ceph-1 2024-04-25T10:30:19.702 INFO:teuthology.orchestra.run.smithi037.stdout:/var/lib/ceph/osd/ceph-2 2024-04-25T10:30:19.702 INFO:teuthology.orchestra.run.smithi037.stdout:/var/lib/ceph/osd/ceph-3 2024-04-25T10:30:19.702 INFO:teuthology.orchestra.run.smithi037.stdout:/run/netns 2024-04-25T10:30:19.702 INFO:teuthology.orchestra.run.smithi037.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 2024-04-25T10:30:19.702 INFO:teuthology.orchestra.run.smithi037.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 2024-04-25T10:30:19.702 INFO:teuthology.orchestra.run.smithi037.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 2024-04-25T10:30:19.702 INFO:teuthology.orchestra.run.smithi037.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 2024-04-25T10:30:19.702 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T10:30:19.702 DEBUG:teuthology.orchestra.run.smithi037:> ls /sys/fs/fuse/connections 2024-04-25T10:30:19.710 INFO:tasks.cephfs.fuse_mount.ceph-fuse.0.smithi037.stderr:2024-04-25T10:30:19.708+0000 7f8cfee12480 -1 init, newargv = 0x7f8cec001210 newargc=15 2024-04-25T10:30:19.710 INFO:tasks.cephfs.fuse_mount.ceph-fuse.0.smithi037.stderr:ceph-fuse[48855]: starting ceph client 2024-04-25T10:30:19.715 INFO:tasks.cephfs.fuse_mount.ceph-fuse.0.smithi037.stderr:ceph-fuse[48855]: starting fuse 2024-04-25T10:30:19.725 INFO:teuthology.orchestra.run.smithi037.stdout:43 2024-04-25T10:30:19.725 INFO:tasks.cephfs.fuse_mount:Post-mount connections: [43] 2024-04-25T10:30:19.725 DEBUG:teuthology.orchestra.run.smithi037:> sudo stdin-killer -- python3 -c ' 2024-04-25T10:30:19.726 DEBUG:teuthology.orchestra.run.smithi037:> import glob 2024-04-25T10:30:19.726 DEBUG:teuthology.orchestra.run.smithi037:> import re 2024-04-25T10:30:19.726 DEBUG:teuthology.orchestra.run.smithi037:> import os 2024-04-25T10:30:19.726 DEBUG:teuthology.orchestra.run.smithi037:> import subprocess 2024-04-25T10:30:19.726 DEBUG:teuthology.orchestra.run.smithi037:> 2024-04-25T10:30:19.726 DEBUG:teuthology.orchestra.run.smithi037:> def _find_admin_socket(client_name): 2024-04-25T10:30:19.726 DEBUG:teuthology.orchestra.run.smithi037:> asok_path = "/var/run/ceph/ceph-client.0.*.asok" 2024-04-25T10:30:19.726 DEBUG:teuthology.orchestra.run.smithi037:> files = glob.glob(asok_path) 2024-04-25T10:30:19.726 DEBUG:teuthology.orchestra.run.smithi037:> mountpoint = "/home/ubuntu/cephtest/mnt.0" 2024-04-25T10:30:19.726 DEBUG:teuthology.orchestra.run.smithi037:> 2024-04-25T10:30:19.726 DEBUG:teuthology.orchestra.run.smithi037:> # Given a non-glob path, it better be there 2024-04-25T10:30:19.726 DEBUG:teuthology.orchestra.run.smithi037:> if "*" not in asok_path: 2024-04-25T10:30:19.726 DEBUG:teuthology.orchestra.run.smithi037:> assert(len(files) == 1) 2024-04-25T10:30:19.726 DEBUG:teuthology.orchestra.run.smithi037:> return files[0] 2024-04-25T10:30:19.726 DEBUG:teuthology.orchestra.run.smithi037:> 2024-04-25T10:30:19.727 DEBUG:teuthology.orchestra.run.smithi037:> for f in files: 2024-04-25T10:30:19.727 DEBUG:teuthology.orchestra.run.smithi037:> pid = re.match(".*\.(\d+)\.asok$", f).group(1) 2024-04-25T10:30:19.727 DEBUG:teuthology.orchestra.run.smithi037:> if os.path.exists("/proc/{0}".format(pid)): 2024-04-25T10:30:19.727 DEBUG:teuthology.orchestra.run.smithi037:> with open("/proc/{0}/cmdline".format(pid), '"'"'r'"'"') as proc_f: 2024-04-25T10:30:19.727 DEBUG:teuthology.orchestra.run.smithi037:> contents = proc_f.read() 2024-04-25T10:30:19.727 DEBUG:teuthology.orchestra.run.smithi037:> if mountpoint in contents: 2024-04-25T10:30:19.727 DEBUG:teuthology.orchestra.run.smithi037:> return f 2024-04-25T10:30:19.727 DEBUG:teuthology.orchestra.run.smithi037:> raise RuntimeError("Client socket {0} not found".format(client_name)) 2024-04-25T10:30:19.727 DEBUG:teuthology.orchestra.run.smithi037:> 2024-04-25T10:30:19.727 DEBUG:teuthology.orchestra.run.smithi037:> print(_find_admin_socket("client.0")) 2024-04-25T10:30:19.727 DEBUG:teuthology.orchestra.run.smithi037:> ' 2024-04-25T10:30:19.844 INFO:teuthology.orchestra.run.smithi037.stdout:/var/run/ceph/ceph-client.0.48855.asok 2024-04-25T10:30:19.847 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:19 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-04-25T10:30:19.854 INFO:tasks.cephfs.fuse_mount:Found client admin socket at /var/run/ceph/ceph-client.0.48855.asok 2024-04-25T10:30:19.854 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T10:30:19.855 DEBUG:teuthology.orchestra.run.smithi037:> sudo ceph --admin-daemon /var/run/ceph/ceph-client.0.48855.asok status 2024-04-25T10:30:19.999 INFO:teuthology.orchestra.run.smithi037.stdout:{ 2024-04-25T10:30:19.999 INFO:teuthology.orchestra.run.smithi037.stdout: "metadata": { 2024-04-25T10:30:19.999 INFO:teuthology.orchestra.run.smithi037.stdout: "ceph_sha1": "994e2f20102a20fa9a27966c14b625cc01884df8", 2024-04-25T10:30:19.999 INFO:teuthology.orchestra.run.smithi037.stdout: "ceph_version": "ceph version 19.0.0-3291-g994e2f20 (994e2f20102a20fa9a27966c14b625cc01884df8) squid (dev)", 2024-04-25T10:30:19.999 INFO:teuthology.orchestra.run.smithi037.stdout: "entity_id": "0", 2024-04-25T10:30:20.000 INFO:teuthology.orchestra.run.smithi037.stdout: "hostname": "smithi037", 2024-04-25T10:30:20.000 INFO:teuthology.orchestra.run.smithi037.stdout: "mount_point": "/home/ubuntu/cephtest/mnt.0", 2024-04-25T10:30:20.000 INFO:teuthology.orchestra.run.smithi037.stdout: "pid": "48855", 2024-04-25T10:30:20.000 INFO:teuthology.orchestra.run.smithi037.stdout: "root": "/" 2024-04-25T10:30:20.000 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-04-25T10:30:20.000 INFO:teuthology.orchestra.run.smithi037.stdout: "dentry_count": 0, 2024-04-25T10:30:20.000 INFO:teuthology.orchestra.run.smithi037.stdout: "dentry_pinned_count": 0, 2024-04-25T10:30:20.000 INFO:teuthology.orchestra.run.smithi037.stdout: "id": 5505, 2024-04-25T10:30:20.000 INFO:teuthology.orchestra.run.smithi037.stdout: "inst": { 2024-04-25T10:30:20.000 INFO:teuthology.orchestra.run.smithi037.stdout: "name": { 2024-04-25T10:30:20.000 INFO:teuthology.orchestra.run.smithi037.stdout: "type": "client", 2024-04-25T10:30:20.000 INFO:teuthology.orchestra.run.smithi037.stdout: "num": 5505 2024-04-25T10:30:20.000 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-04-25T10:30:20.000 INFO:teuthology.orchestra.run.smithi037.stdout: "addr": { 2024-04-25T10:30:20.000 INFO:teuthology.orchestra.run.smithi037.stdout: "type": "v1", 2024-04-25T10:30:20.001 INFO:teuthology.orchestra.run.smithi037.stdout: "addr": "192.168.0.1:0", 2024-04-25T10:30:20.001 INFO:teuthology.orchestra.run.smithi037.stdout: "nonce": 1694662164 2024-04-25T10:30:20.001 INFO:teuthology.orchestra.run.smithi037.stdout: } 2024-04-25T10:30:20.001 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-04-25T10:30:20.001 INFO:teuthology.orchestra.run.smithi037.stdout: "addr": { 2024-04-25T10:30:20.001 INFO:teuthology.orchestra.run.smithi037.stdout: "type": "v1", 2024-04-25T10:30:20.001 INFO:teuthology.orchestra.run.smithi037.stdout: "addr": "192.168.0.1:0", 2024-04-25T10:30:20.001 INFO:teuthology.orchestra.run.smithi037.stdout: "nonce": 1694662164 2024-04-25T10:30:20.001 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-04-25T10:30:20.001 INFO:teuthology.orchestra.run.smithi037.stdout: "inst_str": "client.5505 192.168.0.1:0/1694662164", 2024-04-25T10:30:20.001 INFO:teuthology.orchestra.run.smithi037.stdout: "addr_str": "192.168.0.1:0/1694662164", 2024-04-25T10:30:20.001 INFO:teuthology.orchestra.run.smithi037.stdout: "inode_count": 1, 2024-04-25T10:30:20.001 INFO:teuthology.orchestra.run.smithi037.stdout: "mds_epoch": 17, 2024-04-25T10:30:20.001 INFO:teuthology.orchestra.run.smithi037.stdout: "osd_epoch": 36, 2024-04-25T10:30:20.001 INFO:teuthology.orchestra.run.smithi037.stdout: "osd_epoch_barrier": 0, 2024-04-25T10:30:20.002 INFO:teuthology.orchestra.run.smithi037.stdout: "blocklisted": false, 2024-04-25T10:30:20.002 INFO:teuthology.orchestra.run.smithi037.stdout: "fs_name": "cephfs" 2024-04-25T10:30:20.002 INFO:teuthology.orchestra.run.smithi037.stdout:} 2024-04-25T10:30:20.008 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T10:30:20.008 DEBUG:teuthology.orchestra.run.smithi037:> stat --file-system '--printf=%T 2024-04-25T10:30:20.008 DEBUG:teuthology.orchestra.run.smithi037:> ' -- /home/ubuntu/cephtest/mnt.0 2024-04-25T10:30:20.024 INFO:teuthology.orchestra.run.smithi037.stdout:fuseblk 2024-04-25T10:30:20.024 INFO:tasks.cephfs.fuse_mount:ceph-fuse is mounted on /home/ubuntu/cephtest/mnt.0 2024-04-25T10:30:20.025 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T10:30:20.025 DEBUG:teuthology.orchestra.run.smithi037:> sudo chmod 1777 /home/ubuntu/cephtest/mnt.0 2024-04-25T10:30:20.090 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T10:30:20.091 DEBUG:teuthology.orchestra.run.smithi037:> (cd /home/ubuntu/cephtest/mnt.0 && exec mkdir subdir) 2024-04-25T10:30:20.152 DEBUG:teuthology.orchestra.run.smithi037:> set -ex 2024-04-25T10:30:20.153 DEBUG:teuthology.orchestra.run.smithi037:> dd if=/proc/self/mounts of=/dev/stdout 2024-04-25T10:30:20.209 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T10:30:20.209 DEBUG:teuthology.orchestra.run.smithi037:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=subdir/sixmegs bs=1M conv=fdatasync count=6 seek=0) 2024-04-25T10:30:20.311 INFO:teuthology.orchestra.run.smithi037.stderr:6+0 records in 2024-04-25T10:30:20.311 INFO:teuthology.orchestra.run.smithi037.stderr:6+0 records out 2024-04-25T10:30:20.312 INFO:teuthology.orchestra.run.smithi037.stderr:6291456 bytes (6.3 MB, 6.0 MiB) copied, 0.0438399 s, 144 MB/s 2024-04-25T10:30:20.312 DEBUG:teuthology.orchestra.run.smithi037:> stdin-killer -- python3 -c ' 2024-04-25T10:30:20.312 DEBUG:teuthology.orchestra.run.smithi037:> import os 2024-04-25T10:30:20.312 DEBUG:teuthology.orchestra.run.smithi037:> import stat 2024-04-25T10:30:20.312 DEBUG:teuthology.orchestra.run.smithi037:> import json 2024-04-25T10:30:20.312 DEBUG:teuthology.orchestra.run.smithi037:> import sys 2024-04-25T10:30:20.312 DEBUG:teuthology.orchestra.run.smithi037:> 2024-04-25T10:30:20.312 DEBUG:teuthology.orchestra.run.smithi037:> try: 2024-04-25T10:30:20.313 DEBUG:teuthology.orchestra.run.smithi037:> s = os.stat('"'"'/home/ubuntu/cephtest/mnt.0/subdir/sixmegs'"'"') 2024-04-25T10:30:20.313 DEBUG:teuthology.orchestra.run.smithi037:> except OSError as e: 2024-04-25T10:30:20.313 DEBUG:teuthology.orchestra.run.smithi037:> sys.exit(e.errno) 2024-04-25T10:30:20.313 DEBUG:teuthology.orchestra.run.smithi037:> 2024-04-25T10:30:20.313 DEBUG:teuthology.orchestra.run.smithi037:> attrs = ["st_mode", "st_ino", "st_dev", "st_nlink", "st_uid", "st_gid", "st_size", "st_atime", "st_mtime", "st_ctime"] 2024-04-25T10:30:20.313 DEBUG:teuthology.orchestra.run.smithi037:> print(json.dumps( 2024-04-25T10:30:20.313 DEBUG:teuthology.orchestra.run.smithi037:> dict([(a, getattr(s, a)) for a in attrs]), 2024-04-25T10:30:20.313 DEBUG:teuthology.orchestra.run.smithi037:> indent=2)) 2024-04-25T10:30:20.313 DEBUG:teuthology.orchestra.run.smithi037:> ' 2024-04-25T10:30:20.382 INFO:teuthology.orchestra.run.smithi037.stdout:{ 2024-04-25T10:30:20.382 INFO:teuthology.orchestra.run.smithi037.stdout: "st_mode": 33188, 2024-04-25T10:30:20.382 INFO:teuthology.orchestra.run.smithi037.stdout: "st_ino": 1099511627777, 2024-04-25T10:30:20.382 INFO:teuthology.orchestra.run.smithi037.stdout: "st_dev": 43, 2024-04-25T10:30:20.382 INFO:teuthology.orchestra.run.smithi037.stdout: "st_nlink": 1, 2024-04-25T10:30:20.382 INFO:teuthology.orchestra.run.smithi037.stdout: "st_uid": 1000, 2024-04-25T10:30:20.382 INFO:teuthology.orchestra.run.smithi037.stdout: "st_gid": 1316, 2024-04-25T10:30:20.382 INFO:teuthology.orchestra.run.smithi037.stdout: "st_size": 6291456, 2024-04-25T10:30:20.382 INFO:teuthology.orchestra.run.smithi037.stdout: "st_atime": 1714041020.2651925, 2024-04-25T10:30:20.382 INFO:teuthology.orchestra.run.smithi037.stdout: "st_mtime": 1714041020.2932866, 2024-04-25T10:30:20.382 INFO:teuthology.orchestra.run.smithi037.stdout: "st_ctime": 1714041020.2932866 2024-04-25T10:30:20.383 INFO:teuthology.orchestra.run.smithi037.stdout:} 2024-04-25T10:30:20.384 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:20 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-04-25T10:30:20.390 DEBUG:teuthology.orchestra.run.smithi037:> set -ex 2024-04-25T10:30:20.390 DEBUG:teuthology.orchestra.run.smithi037:> dd if=/proc/self/mounts of=/dev/stdout 2024-04-25T10:30:20.447 DEBUG:teuthology.orchestra.run.smithi037:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-04-25T10:30:20.598 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:20.596+0000 7f2c464bd640 1 Processor -- start 2024-04-25T10:30:20.599 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:20.596+0000 7f2c464bd640 1 -- start start 2024-04-25T10:30:20.601 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:20.598+0000 7f2c464bd640 1 --2- >> v2:172.21.15.203:3300/0 conn(0x7f2c400b4bb0 0x7f2c401646b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:30:20.601 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:20.598+0000 7f2c464bd640 1 --2- >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f2c400d4560 0x7f2c400d4970 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:30:20.601 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:20.598+0000 7f2c464bd640 1 -- --> v1:172.21.15.203:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f2c40161190 con 0x7f2c400b7600 2024-04-25T10:30:20.601 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:20.598+0000 7f2c464bd640 1 -- --> v2:172.21.15.203:3300/0 -- mon_getmap magic: 0 -- 0x7f2c40161470 con 0x7f2c400b4bb0 2024-04-25T10:30:20.601 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:20.598+0000 7f2c464bd640 1 -- --> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] -- mon_getmap magic: 0 -- 0x7f2c40163eb0 con 0x7f2c400d4560 2024-04-25T10:30:20.601 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:20.598+0000 7f2c3ffff640 1 --2- >> v2:172.21.15.203:3300/0 conn(0x7f2c400b4bb0 0x7f2c401646b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:30:20.601 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:20.598+0000 7f2c44a5a640 1 --2- >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f2c400d4560 0x7f2c400d4970 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:30:20.601 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:20.598+0000 7f2c3f7fe640 1 --1- >> v1:172.21.15.203:6789/0 conn(0x7f2c400b7600 0x7f2c400b7a10 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.203:6789/0 says I am v1:172.21.15.37:54452/0 (socket says 172.21.15.37:54452) 2024-04-25T10:30:20.601 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:20.598+0000 7f2c3f7fe640 1 -- 172.21.15.37:0/4049640797 learned_addr learned my addr 172.21.15.37:0/4049640797 (peer_addr_for_me v1:172.21.15.37:0/0) 2024-04-25T10:30:20.601 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:20.598+0000 7f2c3ffff640 1 --2- >> v2:172.21.15.203:3300/0 conn(0x7f2c400b4bb0 0x7f2c401646b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.203:3300/0 says I am v2:172.21.15.37:38824/0 (socket says 172.21.15.37:38824) 2024-04-25T10:30:20.601 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:20.598+0000 7f2c44a5a640 1 -- 172.21.15.37:0/4049640797 >> v2:172.21.15.203:3300/0 conn(0x7f2c400b4bb0 msgr2=0x7f2c401646b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:30:20.601 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:20.598+0000 7f2c44a5a640 1 --2- 172.21.15.37:0/4049640797 >> v2:172.21.15.203:3300/0 conn(0x7f2c400b4bb0 0x7f2c401646b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:30:20.601 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:20.598+0000 7f2c44a5a640 1 -- 172.21.15.37:0/4049640797 >> v1:172.21.15.203:6789/0 conn(0x7f2c400b7600 legacy=0x7f2c400b7a10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:30:20.601 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:20.598+0000 7f2c44a5a640 1 -- 172.21.15.37:0/4049640797 --> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f2c40164190 con 0x7f2c400d4560 2024-04-25T10:30:20.602 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:20.598+0000 7f2c3ffff640 1 --2- 172.21.15.37:0/4049640797 >> v2:172.21.15.203:3300/0 conn(0x7f2c400b4bb0 0x7f2c401646b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-25T10:30:20.603 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:20.599+0000 7f2c44a5a640 1 --2- 172.21.15.37:0/4049640797 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f2c400d4560 0x7f2c400d4970 secure :-1 s=READY pgs=52 cs=0 l=1 rev1=1 crypto rx=0x7f2c38009d90 tx=0x7f2c3802a660 comp rx=0 tx=0).ready entity=mon.2 client_cookie=30c9c9365c4bc299 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:30:20.603 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:20.599+0000 7f2c3effd640 1 -- 172.21.15.37:0/4049640797 <== mon.2 v2:172.21.15.203:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2c38007740 con 0x7f2c400d4560 2024-04-25T10:30:20.603 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:20.600+0000 7f2c3effd640 1 -- 172.21.15.37:0/4049640797 <== mon.2 v2:172.21.15.203:3301/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f2c380078e0 con 0x7f2c400d4560 2024-04-25T10:30:20.603 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:20.600+0000 7f2c3effd640 1 -- 172.21.15.37:0/4049640797 <== mon.2 v2:172.21.15.203:3301/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2c38007cb0 con 0x7f2c400d4560 2024-04-25T10:30:20.603 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:20.600+0000 7f2c464bd640 1 -- 172.21.15.37:0/4049640797 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f2c400d4560 msgr2=0x7f2c400d4970 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:30:20.603 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:20.600+0000 7f2c464bd640 1 --2- 172.21.15.37:0/4049640797 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f2c400d4560 0x7f2c400d4970 secure :-1 s=READY pgs=52 cs=0 l=1 rev1=1 crypto rx=0x7f2c38009d90 tx=0x7f2c3802a660 comp rx=0 tx=0).stop 2024-04-25T10:30:20.603 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:20.600+0000 7f2c464bd640 1 -- 172.21.15.37:0/4049640797 shutdown_connections 2024-04-25T10:30:20.603 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:20.600+0000 7f2c464bd640 1 --2- 172.21.15.37:0/4049640797 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f2c400d4560 0x7f2c400d4970 unknown :-1 s=CLOSED pgs=52 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:30:20.603 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:20.600+0000 7f2c464bd640 1 --2- 172.21.15.37:0/4049640797 >> v2:172.21.15.203:3300/0 conn(0x7f2c400b4bb0 0x7f2c401646b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:30:20.603 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:20.600+0000 7f2c464bd640 1 -- 172.21.15.37:0/4049640797 >> 172.21.15.37:0/4049640797 conn(0x7f2c400d27e0 msgr2=0x7f2c400d2bf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:30:20.603 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:20.600+0000 7f2c464bd640 1 -- 172.21.15.37:0/4049640797 shutdown_connections 2024-04-25T10:30:20.603 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:20.600+0000 7f2c464bd640 1 -- 172.21.15.37:0/4049640797 wait complete. 2024-04-25T10:30:20.604 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:20.602+0000 7f2c464bd640 1 Processor -- start 2024-04-25T10:30:20.604 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:20.602+0000 7f2c464bd640 1 -- start start 2024-04-25T10:30:20.607 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:20.603+0000 7f2c464bd640 1 --2- >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f2c400b4bb0 0x7f2c401fa4b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:30:20.607 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:20.603+0000 7f2c464bd640 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f2c400b7600 0x7f2c400ba1b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:30:20.607 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:20.603+0000 7f2c464bd640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f2c400d4560 0x7f2c400ba6f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:30:20.607 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:20.603+0000 7f2c464bd640 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 -- 0x7f2c400d09e0 con 0x7f2c400d4560 2024-04-25T10:30:20.607 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:20.603+0000 7f2c464bd640 1 -- --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_getmap magic: 0 -- 0x7f2c40164bf0 con 0x7f2c400b7600 2024-04-25T10:30:20.608 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:20.603+0000 7f2c464bd640 1 -- --> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] -- mon_getmap magic: 0 -- 0x7f2c400d0b60 con 0x7f2c400b4bb0 2024-04-25T10:30:20.608 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:20.604+0000 7f2c44a5a640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f2c400d4560 0x7f2c400ba6f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:30:20.608 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:20.604+0000 7f2c44a5a640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f2c400d4560 0x7f2c400ba6f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.37:3300/0 says I am v2:172.21.15.37:59544/0 (socket says 172.21.15.37:59544) 2024-04-25T10:30:20.608 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:20.604+0000 7f2c44a5a640 1 -- 172.21.15.37:0/362741910 learned_addr learned my addr 172.21.15.37:0/362741910 (peer_addr_for_me v2:172.21.15.37:0/0) 2024-04-25T10:30:20.608 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:20.604+0000 7f2c3ffff640 1 --2- 172.21.15.37:0/362741910 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f2c400b4bb0 0x7f2c401fa4b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:30:20.608 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:20.604+0000 7f2c3f7fe640 1 --2- 172.21.15.37:0/362741910 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f2c400b7600 0x7f2c400ba1b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:30:20.608 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:20.604+0000 7f2c44a5a640 1 -- 172.21.15.37:0/362741910 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f2c400b4bb0 msgr2=0x7f2c401fa4b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:30:20.608 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:20.604+0000 7f2c44a5a640 1 --2- 172.21.15.37:0/362741910 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f2c400b4bb0 0x7f2c401fa4b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:30:20.608 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:20.604+0000 7f2c44a5a640 1 -- 172.21.15.37:0/362741910 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f2c400b7600 msgr2=0x7f2c400ba1b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:30:20.608 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:20.604+0000 7f2c44a5a640 1 --2- 172.21.15.37:0/362741910 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f2c400b7600 0x7f2c400ba1b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:30:20.608 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:20.604+0000 7f2c44a5a640 1 -- 172.21.15.37:0/362741910 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f2c4015a340 con 0x7f2c400d4560 2024-04-25T10:30:20.608 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:20.604+0000 7f2c3f7fe640 1 --2- 172.21.15.37:0/362741910 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f2c400b7600 0x7f2c400ba1b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T10:30:20.608 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:20.604+0000 7f2c44a5a640 1 --2- 172.21.15.37:0/362741910 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f2c400d4560 0x7f2c400ba6f0 secure :-1 s=READY pgs=455 cs=0 l=1 rev1=1 crypto rx=0x7f2c38009eb0 tx=0x7f2c38004a40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:30:20.608 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:20.604+0000 7f2c3cff9640 1 -- 172.21.15.37:0/362741910 <== mon.0 v2:172.21.15.37:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2c3803f070 con 0x7f2c400d4560 2024-04-25T10:30:20.608 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:20.604+0000 7f2c3cff9640 1 -- 172.21.15.37:0/362741910 <== mon.0 v2:172.21.15.37:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f2c38031c60 con 0x7f2c400d4560 2024-04-25T10:30:20.608 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:20.604+0000 7f2c3cff9640 1 -- 172.21.15.37:0/362741910 <== mon.0 v2:172.21.15.37:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2c38009450 con 0x7f2c400d4560 2024-04-25T10:30:20.609 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:20.604+0000 7f2c464bd640 1 -- 172.21.15.37:0/362741910 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f2c4015a620 con 0x7f2c400d4560 2024-04-25T10:30:20.609 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:20.604+0000 7f2c464bd640 1 -- 172.21.15.37:0/362741910 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f2c4015abe0 con 0x7f2c400d4560 2024-04-25T10:30:20.609 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:20.605+0000 7f2c464bd640 1 -- 172.21.15.37:0/362741910 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f2c4006a140 con 0x7f2c400d4560 2024-04-25T10:30:20.609 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:20.606+0000 7f2c3cff9640 1 -- 172.21.15.37:0/362741910 <== mon.0 v2:172.21.15.37:3300/0 4 ==== mgrmap(e 5) ==== 96782+0+0 (secure 0 0 0) 0x7f2c3803b410 con 0x7f2c400d4560 2024-04-25T10:30:20.609 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:20.607+0000 7f2c3cff9640 1 --2- 172.21.15.37:0/362741910 >> [v2:172.21.15.203:6832/2418205237,v1:172.21.15.203:6833/2418205237] conn(0x7f2c2c0744b0 0x7f2c2c0769c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:30:20.610 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:20.607+0000 7f2c3cff9640 1 -- 172.21.15.37:0/362741910 <== mon.0 v2:172.21.15.37:3300/0 5 ==== osd_map(36..36 src has 1..36) ==== 5709+0+0 (secure 0 0 0) 0x7f2c380b64f0 con 0x7f2c400d4560 2024-04-25T10:30:20.610 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:20.607+0000 7f2c3ffff640 1 --2- 172.21.15.37:0/362741910 >> [v2:172.21.15.203:6832/2418205237,v1:172.21.15.203:6833/2418205237] conn(0x7f2c2c0744b0 0x7f2c2c0769c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:30:20.610 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:20.608+0000 7f2c3ffff640 1 --2- 172.21.15.37:0/362741910 >> [v2:172.21.15.203:6832/2418205237,v1:172.21.15.203:6833/2418205237] conn(0x7f2c2c0744b0 0x7f2c2c0769c0 secure :-1 s=READY pgs=224 cs=0 l=1 rev1=1 crypto rx=0x7f2c30009560 tx=0x7f2c30009190 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:30:20.613 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:20.611+0000 7f2c3cff9640 1 -- 172.21.15.37:0/362741910 <== mon.0 v2:172.21.15.37:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187246 (secure 0 0 0) 0x7f2c38080d10 con 0x7f2c400d4560 2024-04-25T10:30:20.801 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:20.799+0000 7f2c464bd640 1 -- 172.21.15.37:0/362741910 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7f2c40162850 con 0x7f2c400d4560 2024-04-25T10:30:20.803 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:20.801+0000 7f2c3cff9640 1 -- 172.21.15.37:0/362741910 <== mon.0 v2:172.21.15.37:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v36) ==== 74+0+16621 (secure 0 0 0) 0x7f2c38080610 con 0x7f2c400d4560 2024-04-25T10:30:20.803 INFO:teuthology.orchestra.run.smithi037.stdout: 2024-04-25T10:30:20.803 INFO:teuthology.orchestra.run.smithi037.stdout:{"epoch":36,"fsid":"f4acda28-be34-492a-b286-994afd36f61d","created":"2024-04-25T10:28:43.985718+0000","modified":"2024-04-25T10:30:13.756895+0000","last_up_change":"2024-04-25T10:28:54.900654+0000","last_in_change":"2024-04-25T10:28:46.216973+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":5,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-04-25T10:28:56.202153+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"19","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":4,"pool_name":"cephfs_metadata","create_time":"2024-04-25T10:30:10.008132+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"35","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.75,"score_stable":1.75,"optimal_score":1,"raw_score_acting":1.75,"raw_score_stable":1.75,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":5,"pool_name":"cephfs_data","create_time":"2024-04-25T10:30:11.084827+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"36","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"a2f79028-b332-4166-9ebb-a7024a37ea22","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":34,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.37:6816","nonce":2746074720},{"type":"v1","addr":"172.21.15.37:6817","nonce":2746074720}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.37:6818","nonce":2746074720},{"type":"v1","addr":"172.21.15.37:6819","nonce":2746074720}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.37:6822","nonce":2746074720},{"type":"v1","addr":"172.21.15.37:6823","nonce":2746074720}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.37:6820","nonce":2746074720},{"type":"v1","addr":"172.21.15.37:6821","nonce":2746074720}]},"public_addr":"172.21.15.37:6817/2746074720","cluster_addr":"172.21.15.37:6819/2746074720","heartbeat_back_addr":"172.21.15.37:6823/2746074720","heartbeat_front_addr":"172.21.15.37:6821/2746074720","state":["exists","up"]},{"osd":1,"uuid":"973159de-c2ea-4083-aecc-69373d1c1132","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":34,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.37:6800","nonce":1690341662},{"type":"v1","addr":"172.21.15.37:6801","nonce":1690341662}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.37:6802","nonce":1690341662},{"type":"v1","addr":"172.21.15.37:6803","nonce":1690341662}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.37:6806","nonce":1690341662},{"type":"v1","addr":"172.21.15.37:6807","nonce":1690341662}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.37:6804","nonce":1690341662},{"type":"v1","addr":"172.21.15.37:6805","nonce":1690341662}]},"public_addr":"172.21.15.37:6801/1690341662","cluster_addr":"172.21.15.37:6803/1690341662","heartbeat_back_addr":"172.21.15.37:6807/1690341662","heartbeat_front_addr":"172.21.15.37:6805/1690341662","state":["exists","up"]},{"osd":2,"uuid":"4ae7a0e9-cb3f-484c-a75f-f3766b721c47","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":34,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.37:6808","nonce":2754669649},{"type":"v1","addr":"172.21.15.37:6809","nonce":2754669649}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.37:6810","nonce":2754669649},{"type":"v1","addr":"172.21.15.37:6811","nonce":2754669649}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.37:6814","nonce":2754669649},{"type":"v1","addr":"172.21.15.37:6815","nonce":2754669649}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.37:6812","nonce":2754669649},{"type":"v1","addr":"172.21.15.37:6813","nonce":2754669649}]},"public_addr":"172.21.15.37:6809/2754669649","cluster_addr":"172.21.15.37:6811/2754669649","heartbeat_back_addr":"172.21.15.37:6815/2754669649","heartbeat_front_addr":"172.21.15.37:6813/2754669649","state":["exists","up"]},{"osd":3,"uuid":"1ba4a746-1608-47a7-ace2-960a5cfae934","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":34,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.37:6824","nonce":1815290618},{"type":"v1","addr":"172.21.15.37:6825","nonce":1815290618}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.37:6826","nonce":1815290618},{"type":"v1","addr":"172.21.15.37:6827","nonce":1815290618}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.37:6830","nonce":1815290618},{"type":"v1","addr":"172.21.15.37:6831","nonce":1815290618}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.37:6828","nonce":1815290618},{"type":"v1","addr":"172.21.15.37:6829","nonce":1815290618}]},"public_addr":"172.21.15.37:6825/1815290618","cluster_addr":"172.21.15.37:6827/1815290618","heartbeat_back_addr":"172.21.15.37:6831/1815290618","heartbeat_front_addr":"172.21.15.37:6829/1815290618","state":["exists","up"]},{"osd":4,"uuid":"d4f59f5d-0413-4b79-87ee-bb212c1fcf62","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":34,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6800","nonce":4051532063},{"type":"v1","addr":"172.21.15.203:6801","nonce":4051532063}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6802","nonce":4051532063},{"type":"v1","addr":"172.21.15.203:6803","nonce":4051532063}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6806","nonce":4051532063},{"type":"v1","addr":"172.21.15.203:6807","nonce":4051532063}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6804","nonce":4051532063},{"type":"v1","addr":"172.21.15.203:6805","nonce":4051532063}]},"public_addr":"172.21.15.203:6801/4051532063","cluster_addr":"172.21.15.203:6803/4051532063","heartbeat_back_addr":"172.21.15.203:6807/4051532063","heartbeat_front_addr":"172.21.15.203:6805/4051532063","state":["exists","up"]},{"osd":5,"uuid":"69710a0e-41d8-4e7f-9a10-17ce540b67be","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":34,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6824","nonce":1203822890},{"type":"v1","addr":"172.21.15.203:6825","nonce":1203822890}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6826","nonce":1203822890},{"type":"v1","addr":"172.21.15.203:6827","nonce":1203822890}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6830","nonce":1203822890},{"type":"v1","addr":"172.21.15.203:6831","nonce":1203822890}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6828","nonce":1203822890},{"type":"v1","addr":"172.21.15.203:6829","nonce":1203822890}]},"public_addr":"172.21.15.203:6825/1203822890","cluster_addr":"172.21.15.203:6827/1203822890","heartbeat_back_addr":"172.21.15.203:6831/1203822890","heartbeat_front_addr":"172.21.15.203:6829/1203822890","state":["exists","up"]},{"osd":6,"uuid":"f29dd047-3b48-473d-a235-6642a6293f20","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":34,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6810","nonce":2309615014},{"type":"v1","addr":"172.21.15.203:6812","nonce":2309615014}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6814","nonce":2309615014},{"type":"v1","addr":"172.21.15.203:6816","nonce":2309615014}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6822","nonce":2309615014},{"type":"v1","addr":"172.21.15.203:6823","nonce":2309615014}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6818","nonce":2309615014},{"type":"v1","addr":"172.21.15.203:6820","nonce":2309615014}]},"public_addr":"172.21.15.203:6812/2309615014","cluster_addr":"172.21.15.203:6816/2309615014","heartbeat_back_addr":"172.21.15.203:6823/2309615014","heartbeat_front_addr":"172.21.15.203:6820/2309615014","state":["exists","up"]},{"osd":7,"uuid":"6e691644-378b-4fdc-8e86-fb5f56e31f29","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":34,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6808","nonce":3688099014},{"type":"v1","addr":"172.21.15.203:6809","nonce":3688099014}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6811","nonce":3688099014},{"type":"v1","addr":"172.21.15.203:6813","nonce":3688099014}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6819","nonce":3688099014},{"type":"v1","addr":"172.21.15.203:6821","nonce":3688099014}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6815","nonce":3688099014},{"type":"v1","addr":"172.21.15.203:6817","nonce":3688099014}]},"public_addr":"172.21.15.203:6809/3688099014","cluster_addr":"172.21.15.203:6813/3688099014","heartbeat_back_addr":"172.21.15.203:6821/3688099014","heartbeat_front_addr":"172.21.15.203:6817/3688099014","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T10:28:51.419141+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T10:28:51.366448+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T10:28:51.348276+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T10:28:51.380519+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T10:28:51.105812+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T10:28:51.384496+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T10:28:51.388755+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T10:28:51.366958+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-04-25T10:30:20.805 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:20.803+0000 7f2c464bd640 1 -- 172.21.15.37:0/362741910 >> [v2:172.21.15.203:6832/2418205237,v1:172.21.15.203:6833/2418205237] conn(0x7f2c2c0744b0 msgr2=0x7f2c2c0769c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:30:20.805 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:20.803+0000 7f2c464bd640 1 --2- 172.21.15.37:0/362741910 >> [v2:172.21.15.203:6832/2418205237,v1:172.21.15.203:6833/2418205237] conn(0x7f2c2c0744b0 0x7f2c2c0769c0 secure :-1 s=READY pgs=224 cs=0 l=1 rev1=1 crypto rx=0x7f2c30009560 tx=0x7f2c30009190 comp rx=0 tx=0).stop 2024-04-25T10:30:20.805 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:20.803+0000 7f2c464bd640 1 -- 172.21.15.37:0/362741910 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f2c400d4560 msgr2=0x7f2c400ba6f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:30:20.806 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:20.803+0000 7f2c464bd640 1 --2- 172.21.15.37:0/362741910 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f2c400d4560 0x7f2c400ba6f0 secure :-1 s=READY pgs=455 cs=0 l=1 rev1=1 crypto rx=0x7f2c38009eb0 tx=0x7f2c38004a40 comp rx=0 tx=0).stop 2024-04-25T10:30:20.806 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:20.803+0000 7f2c464bd640 1 -- 172.21.15.37:0/362741910 shutdown_connections 2024-04-25T10:30:20.806 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:20.803+0000 7f2c464bd640 1 --2- 172.21.15.37:0/362741910 >> [v2:172.21.15.203:6832/2418205237,v1:172.21.15.203:6833/2418205237] conn(0x7f2c2c0744b0 0x7f2c2c0769c0 unknown :-1 s=CLOSED pgs=224 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:30:20.806 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:20.803+0000 7f2c464bd640 1 --2- 172.21.15.37:0/362741910 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f2c400d4560 0x7f2c400ba6f0 unknown :-1 s=CLOSED pgs=455 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:30:20.806 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:20.803+0000 7f2c464bd640 1 --2- 172.21.15.37:0/362741910 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f2c400b7600 0x7f2c400ba1b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:30:20.806 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:20.803+0000 7f2c464bd640 1 --2- 172.21.15.37:0/362741910 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f2c400b4bb0 0x7f2c401fa4b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:30:20.806 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:20.803+0000 7f2c464bd640 1 -- 172.21.15.37:0/362741910 >> 172.21.15.37:0/362741910 conn(0x7f2c400d27e0 msgr2=0x7f2c4015f3d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:30:20.806 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:20.803+0000 7f2c464bd640 1 -- 172.21.15.37:0/362741910 shutdown_connections 2024-04-25T10:30:20.806 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:20.803+0000 7f2c464bd640 1 -- 172.21.15.37:0/362741910 wait complete. 2024-04-25T10:30:20.816 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2024-04-25T10:28:56.202153+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '19', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 8, 'score_stable': 8, 'optimal_score': 0.25, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 4, 'pool_name': 'cephfs_metadata', 'create_time': '2024-04-25T10:30:10.008132+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '35', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_autoscale_bias': 4, 'pg_num_min': 16, 'recovery_priority': 5}, 'application_metadata': {'cephfs': {'metadata': 'cephfs'}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 1.75, 'score_stable': 1.75, 'optimal_score': 1, 'raw_score_acting': 1.75, 'raw_score_stable': 1.75, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 5, 'pool_name': 'cephfs_data', 'create_time': '2024-04-25T10:30:11.084827+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 64, 'pg_placement_num': 64, 'pg_placement_num_target': 64, 'pg_num_target': 64, 'pg_num_pending': 64, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '36', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.8999999761581421}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 1.5, 'score_stable': 1.5, 'optimal_score': 1, 'raw_score_acting': 1.5, 'raw_score_stable': 1.5, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}] 2024-04-25T10:30:20.817 DEBUG:teuthology.orchestra.run.smithi037:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2024-04-25T10:30:20.932 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:20.929+0000 7fd19ebd7640 1 Processor -- start 2024-04-25T10:30:20.932 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:20.929+0000 7fd19ebd7640 1 -- start start 2024-04-25T10:30:20.936 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:20.931+0000 7fd19ebd7640 1 --2- >> v2:172.21.15.203:3300/0 conn(0x7fd19806e260 0x7fd19806e670 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:30:20.936 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:20.931+0000 7fd19ebd7640 1 -- --> v1:172.21.15.37:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fd19809ce70 con 0x7fd19806ebb0 2024-04-25T10:30:20.936 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:20.931+0000 7fd19ebd7640 1 -- --> v1:172.21.15.203:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fd19809d150 con 0x7fd198164ce0 2024-04-25T10:30:20.936 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:20.931+0000 7fd19ebd7640 1 -- --> v2:172.21.15.203:3300/0 -- mon_getmap magic: 0 -- 0x7fd1980d3370 con 0x7fd19806e260 2024-04-25T10:30:20.936 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:20.931+0000 7fd197fff640 1 --1- >> v1:172.21.15.37:6789/0 conn(0x7fd19806ebb0 0x7fd1981645d0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.37:6789/0 says I am v1:172.21.15.37:55074/0 (socket says 172.21.15.37:55074) 2024-04-25T10:30:20.936 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:20.931+0000 7fd197fff640 1 -- 172.21.15.37:0/2678934795 learned_addr learned my addr 172.21.15.37:0/2678934795 (peer_addr_for_me v1:172.21.15.37:0/0) 2024-04-25T10:30:20.936 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:20.931+0000 7fd19c973640 1 --2- 172.21.15.37:0/2678934795 >> v2:172.21.15.203:3300/0 conn(0x7fd19806e260 0x7fd19806e670 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:30:20.936 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:20.931+0000 7fd1977fe640 1 -- 172.21.15.37:0/2678934795 <== mon.0 v1:172.21.15.37:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 1426190493 0 0) 0x7fd19809ce70 con 0x7fd19806ebb0 2024-04-25T10:30:20.936 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:20.931+0000 7fd1977fe640 1 -- 172.21.15.37:0/2678934795 --> v1:172.21.15.37:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7fd180003520 con 0x7fd19806ebb0 2024-04-25T10:30:20.936 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:20.931+0000 7fd1977fe640 1 -- 172.21.15.37:0/2678934795 <== mon.0 v1:172.21.15.37:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 1925464698 0 0) 0x7fd180003520 con 0x7fd19806ebb0 2024-04-25T10:30:20.936 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:20.931+0000 7fd1977fe640 1 -- 172.21.15.37:0/2678934795 >> v2:172.21.15.203:3300/0 conn(0x7fd19806e260 msgr2=0x7fd19806e670 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:30:20.936 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:20.931+0000 7fd1977fe640 1 --2- 172.21.15.37:0/2678934795 >> v2:172.21.15.203:3300/0 conn(0x7fd19806e260 0x7fd19806e670 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:30:20.937 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:20.931+0000 7fd1977fe640 1 -- 172.21.15.37:0/2678934795 >> v1:172.21.15.203:6789/0 conn(0x7fd198164ce0 legacy=0x7fd1980b3c80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:30:20.937 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:20.931+0000 7fd1977fe640 1 -- 172.21.15.37:0/2678934795 --> v1:172.21.15.37:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fd1980b9000 con 0x7fd19806ebb0 2024-04-25T10:30:20.937 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:20.931+0000 7fd1977fe640 1 -- 172.21.15.37:0/2678934795 <== mon.0 v1:172.21.15.37:6789/0 3 ==== mon_map magic: 0 ==== 413+0+0 (unknown 1577794600 0 0) 0x7fd18c001670 con 0x7fd19806ebb0 2024-04-25T10:30:20.937 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:20.931+0000 7fd1977fe640 1 -- 172.21.15.37:0/2678934795 >> v1:172.21.15.37:6789/0 conn(0x7fd19806ebb0 legacy=0x7fd1981645d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:30:20.937 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:20.931+0000 7fd1977fe640 1 --2- 172.21.15.37:0/2678934795 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7fd1800039b0 0x7fd180003e20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:30:20.937 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:20.931+0000 7fd1977fe640 1 --2- 172.21.15.37:0/2678934795 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fd180004800 0x7fd180004c10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:30:20.937 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:20.931+0000 7fd1977fe640 1 --2- 172.21.15.37:0/2678934795 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fd180005250 0x7fd18000b9c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:30:20.937 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:20.932+0000 7fd1977fe640 1 -- 172.21.15.37:0/2678934795 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 -- 0x7fd18000c1b0 con 0x7fd180005250 2024-04-25T10:30:20.937 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:20.932+0000 7fd1977fe640 1 -- 172.21.15.37:0/2678934795 --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_getmap magic: 0 -- 0x7fd18000c360 con 0x7fd180004800 2024-04-25T10:30:20.937 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:20.932+0000 7fd1977fe640 1 -- 172.21.15.37:0/2678934795 --> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] -- mon_getmap magic: 0 -- 0x7fd18000c560 con 0x7fd1800039b0 2024-04-25T10:30:20.938 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:20.932+0000 7fd19d174640 1 --2- 172.21.15.37:0/2678934795 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fd180005250 0x7fd18000b9c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:30:20.938 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:20.932+0000 7fd197fff640 1 --2- 172.21.15.37:0/2678934795 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fd180004800 0x7fd180004c10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:30:20.938 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:20.932+0000 7fd19c973640 1 --2- 172.21.15.37:0/2678934795 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7fd1800039b0 0x7fd180003e20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:30:20.938 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:20.932+0000 7fd19d174640 1 -- 172.21.15.37:0/2678934795 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7fd1800039b0 msgr2=0x7fd180003e20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:30:20.938 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:20.932+0000 7fd19d174640 1 --2- 172.21.15.37:0/2678934795 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7fd1800039b0 0x7fd180003e20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:30:20.938 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:20.932+0000 7fd19d174640 1 -- 172.21.15.37:0/2678934795 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fd180004800 msgr2=0x7fd180004c10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:30:20.938 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:20.932+0000 7fd19d174640 1 --2- 172.21.15.37:0/2678934795 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fd180004800 0x7fd180004c10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:30:20.938 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:20.932+0000 7fd19d174640 1 -- 172.21.15.37:0/2678934795 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fd180003520 con 0x7fd180005250 2024-04-25T10:30:20.938 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:20.932+0000 7fd197fff640 1 --2- 172.21.15.37:0/2678934795 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fd180004800 0x7fd180004c10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T10:30:20.938 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:20.933+0000 7fd19d174640 1 -- 172.21.15.37:0/2678934795 reap_dead start 2024-04-25T10:30:20.938 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:20.933+0000 7fd19d174640 1 --2- 172.21.15.37:0/2678934795 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fd180005250 0x7fd18000b9c0 secure :-1 s=READY pgs=457 cs=0 l=1 rev1=1 crypto rx=0x7fd188001720 tx=0x7fd180005830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=923562ae14eff1d0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:30:20.938 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:20.933+0000 7fd1977fe640 1 -- 172.21.15.37:0/2678934795 <== mon.0 v2:172.21.15.37:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd188001370 con 0x7fd180005250 2024-04-25T10:30:20.938 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:20.933+0000 7fd1977fe640 1 -- 172.21.15.37:0/2678934795 <== mon.0 v2:172.21.15.37:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fd1880052d0 con 0x7fd180005250 2024-04-25T10:30:20.939 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:20.933+0000 7fd1977fe640 1 -- 172.21.15.37:0/2678934795 <== mon.0 v2:172.21.15.37:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd18802aba0 con 0x7fd180005250 2024-04-25T10:30:20.939 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:20.933+0000 7fd19ebd7640 1 -- 172.21.15.37:0/2678934795 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fd180005250 msgr2=0x7fd18000b9c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:30:20.939 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:20.933+0000 7fd19ebd7640 1 --2- 172.21.15.37:0/2678934795 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fd180005250 0x7fd18000b9c0 secure :-1 s=READY pgs=457 cs=0 l=1 rev1=1 crypto rx=0x7fd188001720 tx=0x7fd180005830 comp rx=0 tx=0).stop 2024-04-25T10:30:20.939 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:20.933+0000 7fd19ebd7640 1 -- 172.21.15.37:0/2678934795 shutdown_connections 2024-04-25T10:30:20.939 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:20.933+0000 7fd19ebd7640 1 --2- 172.21.15.37:0/2678934795 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fd180005250 0x7fd18000b9c0 unknown :-1 s=CLOSED pgs=457 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:30:20.939 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:20.934+0000 7fd19ebd7640 1 -- 172.21.15.37:0/2678934795 >> 172.21.15.37:0/2678934795 conn(0x7fd1980d0390 msgr2=0x7fd1980d07a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:30:20.939 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:20.934+0000 7fd19ebd7640 1 -- 172.21.15.37:0/2678934795 shutdown_connections 2024-04-25T10:30:20.939 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:20.934+0000 7fd19ebd7640 1 -- 172.21.15.37:0/2678934795 wait complete. 2024-04-25T10:30:20.942 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:20.935+0000 7fd19ebd7640 1 Processor -- start 2024-04-25T10:30:20.942 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:20.935+0000 7fd19ebd7640 1 -- start start 2024-04-25T10:30:20.942 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:20.937+0000 7fd19ebd7640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fd180005250 0x7fd198156290 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:30:20.942 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:20.937+0000 7fd19ebd7640 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fd1980b1550 0x7fd1980b1960 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:30:20.942 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:20.937+0000 7fd19ebd7640 1 --2- >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7fd1981539a0 0x7fd198153db0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:30:20.942 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:20.937+0000 7fd19ebd7640 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 -- 0x7fd1980d3090 con 0x7fd180005250 2024-04-25T10:30:20.942 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:20.937+0000 7fd19ebd7640 1 -- --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_getmap magic: 0 -- 0x7fd19806dd40 con 0x7fd1980b1550 2024-04-25T10:30:20.942 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:20.937+0000 7fd19ebd7640 1 -- --> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] -- mon_getmap magic: 0 -- 0x7fd1980b91f0 con 0x7fd1981539a0 2024-04-25T10:30:20.942 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:20.937+0000 7fd19c973640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fd180005250 0x7fd198156290 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:30:20.942 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:20.937+0000 7fd19c973640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fd180005250 0x7fd198156290 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.37:3300/0 says I am v2:172.21.15.37:59554/0 (socket says 172.21.15.37:59554) 2024-04-25T10:30:20.943 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:20.937+0000 7fd19c973640 1 -- 172.21.15.37:0/1876796369 learned_addr learned my addr 172.21.15.37:0/1876796369 (peer_addr_for_me v2:172.21.15.37:0/0) 2024-04-25T10:30:20.943 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:20.937+0000 7fd197fff640 1 --2- 172.21.15.37:0/1876796369 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fd1980b1550 0x7fd1980b1960 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:30:20.943 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:20.937+0000 7fd19c973640 1 -- 172.21.15.37:0/1876796369 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7fd1981539a0 msgr2=0x7fd198153db0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:30:20.943 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:20.937+0000 7fd19d174640 1 --2- 172.21.15.37:0/1876796369 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7fd1981539a0 0x7fd198153db0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:30:20.943 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:20.937+0000 7fd19c973640 1 --2- 172.21.15.37:0/1876796369 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7fd1981539a0 0x7fd198153db0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:30:20.943 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:20.937+0000 7fd19c973640 1 -- 172.21.15.37:0/1876796369 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fd1980b1550 msgr2=0x7fd1980b1960 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:30:20.943 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:20.937+0000 7fd19c973640 1 --2- 172.21.15.37:0/1876796369 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fd1980b1550 0x7fd1980b1960 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:30:20.943 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:20.937+0000 7fd19c973640 1 -- 172.21.15.37:0/1876796369 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fd1980b1ea0 con 0x7fd180005250 2024-04-25T10:30:20.943 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:20.937+0000 7fd197fff640 1 --2- 172.21.15.37:0/1876796369 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fd1980b1550 0x7fd1980b1960 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T10:30:20.943 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:20.937+0000 7fd19d174640 1 --2- 172.21.15.37:0/1876796369 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7fd1981539a0 0x7fd198153db0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T10:30:20.943 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:20.937+0000 7fd19c973640 1 --2- 172.21.15.37:0/1876796369 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fd180005250 0x7fd198156290 secure :-1 s=READY pgs=458 cs=0 l=1 rev1=1 crypto rx=0x7fd190009f50 tx=0x7fd19000f580 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:30:20.943 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:20.937+0000 7fd1957fa640 1 -- 172.21.15.37:0/1876796369 <== mon.0 v2:172.21.15.37:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd190015430 con 0x7fd180005250 2024-04-25T10:30:20.943 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:20.938+0000 7fd1957fa640 1 -- 172.21.15.37:0/1876796369 <== mon.0 v2:172.21.15.37:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fd1900155d0 con 0x7fd180005250 2024-04-25T10:30:20.943 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:20.938+0000 7fd19ebd7640 1 -- 172.21.15.37:0/1876796369 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fd1980b2320 con 0x7fd180005250 2024-04-25T10:30:20.944 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:20.938+0000 7fd1957fa640 1 -- 172.21.15.37:0/1876796369 <== mon.0 v2:172.21.15.37:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd190015980 con 0x7fd180005250 2024-04-25T10:30:20.944 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:20.938+0000 7fd19ebd7640 1 -- 172.21.15.37:0/1876796369 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fd19809f450 con 0x7fd180005250 2024-04-25T10:30:20.944 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:20.939+0000 7fd19ebd7640 1 -- 172.21.15.37:0/1876796369 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fd19806a140 con 0x7fd180005250 2024-04-25T10:30:20.944 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:20.939+0000 7fd1957fa640 1 -- 172.21.15.37:0/1876796369 <== mon.0 v2:172.21.15.37:3300/0 4 ==== mgrmap(e 5) ==== 96782+0+0 (secure 0 0 0) 0x7fd190015b20 con 0x7fd180005250 2024-04-25T10:30:20.944 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:20.940+0000 7fd1957fa640 1 --2- 172.21.15.37:0/1876796369 >> [v2:172.21.15.203:6832/2418205237,v1:172.21.15.203:6833/2418205237] conn(0x7fd1840744b0 0x7fd1840769c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:30:20.944 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:20.940+0000 7fd1957fa640 1 -- 172.21.15.37:0/1876796369 <== mon.0 v2:172.21.15.37:3300/0 5 ==== osd_map(36..36 src has 1..36) ==== 5709+0+0 (secure 0 0 0) 0x7fd1900963d0 con 0x7fd180005250 2024-04-25T10:30:20.944 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:20.940+0000 7fd197fff640 1 --2- 172.21.15.37:0/1876796369 >> [v2:172.21.15.203:6832/2418205237,v1:172.21.15.203:6833/2418205237] conn(0x7fd1840744b0 0x7fd1840769c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:30:20.944 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:20.941+0000 7fd197fff640 1 --2- 172.21.15.37:0/1876796369 >> [v2:172.21.15.203:6832/2418205237,v1:172.21.15.203:6833/2418205237] conn(0x7fd1840744b0 0x7fd1840769c0 secure :-1 s=READY pgs=225 cs=0 l=1 rev1=1 crypto rx=0x7fd18c001200 tx=0x7fd18c001190 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:30:20.946 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:20.945+0000 7fd1957fa640 1 -- 172.21.15.37:0/1876796369 <== mon.0 v2:172.21.15.37:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187246 (secure 0 0 0) 0x7fd1980b1ea0 con 0x7fd180005250 2024-04-25T10:30:21.131 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:21.129+0000 7fd19ebd7640 1 -- 172.21.15.37:0/1876796369 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) -- 0x7fd198070210 con 0x7fd180005250 2024-04-25T10:30:21.131 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:21.129+0000 7fd1957fa640 1 -- 172.21.15.37:0/1876796369 <== mon.0 v2:172.21.15.37:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v36) ==== 93+0+10 (secure 0 0 0) 0x7fd190019030 con 0x7fd180005250 2024-04-25T10:30:21.132 INFO:teuthology.orchestra.run.smithi037.stdout:pg_num: 1 2024-04-25T10:30:21.134 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:21.131+0000 7fd19ebd7640 1 -- 172.21.15.37:0/1876796369 >> [v2:172.21.15.203:6832/2418205237,v1:172.21.15.203:6833/2418205237] conn(0x7fd1840744b0 msgr2=0x7fd1840769c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:30:21.134 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:21.131+0000 7fd19ebd7640 1 --2- 172.21.15.37:0/1876796369 >> [v2:172.21.15.203:6832/2418205237,v1:172.21.15.203:6833/2418205237] conn(0x7fd1840744b0 0x7fd1840769c0 secure :-1 s=READY pgs=225 cs=0 l=1 rev1=1 crypto rx=0x7fd18c001200 tx=0x7fd18c001190 comp rx=0 tx=0).stop 2024-04-25T10:30:21.134 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:21.131+0000 7fd19ebd7640 1 -- 172.21.15.37:0/1876796369 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fd180005250 msgr2=0x7fd198156290 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:30:21.134 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:21.131+0000 7fd19ebd7640 1 --2- 172.21.15.37:0/1876796369 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fd180005250 0x7fd198156290 secure :-1 s=READY pgs=458 cs=0 l=1 rev1=1 crypto rx=0x7fd190009f50 tx=0x7fd19000f580 comp rx=0 tx=0).stop 2024-04-25T10:30:21.134 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:21.131+0000 7fd19ebd7640 1 -- 172.21.15.37:0/1876796369 shutdown_connections 2024-04-25T10:30:21.134 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:21.131+0000 7fd19ebd7640 1 --2- 172.21.15.37:0/1876796369 >> [v2:172.21.15.203:6832/2418205237,v1:172.21.15.203:6833/2418205237] conn(0x7fd1840744b0 0x7fd1840769c0 unknown :-1 s=CLOSED pgs=225 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:30:21.134 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:21.131+0000 7fd19ebd7640 1 --2- 172.21.15.37:0/1876796369 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7fd1981539a0 0x7fd198153db0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:30:21.134 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:21.131+0000 7fd19ebd7640 1 --2- 172.21.15.37:0/1876796369 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fd1980b1550 0x7fd1980b1960 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:30:21.134 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:21.131+0000 7fd19ebd7640 1 --2- 172.21.15.37:0/1876796369 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fd180005250 0x7fd198156290 unknown :-1 s=CLOSED pgs=458 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:30:21.134 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:21.131+0000 7fd19ebd7640 1 -- 172.21.15.37:0/1876796369 >> 172.21.15.37:0/1876796369 conn(0x7fd1980d0390 msgr2=0x7fd198161da0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:30:21.134 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:21.131+0000 7fd19ebd7640 1 -- 172.21.15.37:0/1876796369 shutdown_connections 2024-04-25T10:30:21.134 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:21.131+0000 7fd19ebd7640 1 -- 172.21.15.37:0/1876796369 wait complete. 2024-04-25T10:30:21.145 DEBUG:teuthology.orchestra.run.smithi037:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_metadata pg_num 2024-04-25T10:30:21.259 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:21.256+0000 7fbca9306640 1 Processor -- start 2024-04-25T10:30:21.259 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:21.257+0000 7fbca9306640 1 -- start start 2024-04-25T10:30:21.263 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:21.258+0000 7fbca9306640 1 --2- >> v2:172.21.15.37:3300/0 conn(0x7fbca40b2dc0 0x7fbca40d2860 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:30:21.263 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:21.258+0000 7fbca9306640 1 --2- >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7fbca406daf0 0x7fbca406df00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:30:21.263 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:21.258+0000 7fbca9306640 1 --2- >> v2:172.21.15.203:3300/0 conn(0x7fbca406e440 0x7fbca408cc90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:30:21.263 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:21.258+0000 7fbca9306640 1 -- --> v2:172.21.15.37:3300/0 -- mon_getmap magic: 0 -- 0x7fbca40b92c0 con 0x7fbca40b2dc0 2024-04-25T10:30:21.263 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:21.258+0000 7fbca9306640 1 -- --> v2:172.21.15.203:3300/0 -- mon_getmap magic: 0 -- 0x7fbca40ce0d0 con 0x7fbca406e440 2024-04-25T10:30:21.263 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:21.258+0000 7fbca9306640 1 -- --> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] -- mon_getmap magic: 0 -- 0x7fbca40d2e40 con 0x7fbca406daf0 2024-04-25T10:30:21.263 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:21.258+0000 7fbca2ffd640 1 --2- >> v2:172.21.15.37:3300/0 conn(0x7fbca40b2dc0 0x7fbca40d2860 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:30:21.263 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:21.258+0000 7fbca2ffd640 1 --2- >> v2:172.21.15.37:3300/0 conn(0x7fbca40b2dc0 0x7fbca40d2860 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.37:3300/0 says I am v2:172.21.15.37:59566/0 (socket says 172.21.15.37:59566) 2024-04-25T10:30:21.263 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:21.258+0000 7fbca2ffd640 1 -- 172.21.15.37:0/888899942 learned_addr learned my addr 172.21.15.37:0/888899942 (peer_addr_for_me v2:172.21.15.37:0/0) 2024-04-25T10:30:21.263 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:21.258+0000 7fbca27fc640 1 --2- 172.21.15.37:0/888899942 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7fbca406daf0 0x7fbca406df00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:30:21.264 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:21.258+0000 7fbca37fe640 1 --2- 172.21.15.37:0/888899942 >> v2:172.21.15.203:3300/0 conn(0x7fbca406e440 0x7fbca408cc90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:30:21.264 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:21.258+0000 7fbca2ffd640 1 -- 172.21.15.37:0/888899942 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7fbca406daf0 msgr2=0x7fbca406df00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:30:21.264 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:21.258+0000 7fbca2ffd640 1 --2- 172.21.15.37:0/888899942 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7fbca406daf0 0x7fbca406df00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:30:21.264 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:21.258+0000 7fbca2ffd640 1 -- 172.21.15.37:0/888899942 >> v2:172.21.15.203:3300/0 conn(0x7fbca406e440 msgr2=0x7fbca408cc90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:30:21.264 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:21.258+0000 7fbca2ffd640 1 --2- 172.21.15.37:0/888899942 >> v2:172.21.15.203:3300/0 conn(0x7fbca406e440 0x7fbca408cc90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:30:21.264 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:21.258+0000 7fbca2ffd640 1 -- 172.21.15.37:0/888899942 --> v2:172.21.15.37:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fbca40d3120 con 0x7fbca40b2dc0 2024-04-25T10:30:21.264 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:21.258+0000 7fbca37fe640 1 --2- 172.21.15.37:0/888899942 >> v2:172.21.15.203:3300/0 conn(0x7fbca406e440 0x7fbca408cc90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T10:30:21.265 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:21.259+0000 7fbca2ffd640 1 --2- 172.21.15.37:0/888899942 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fbca40b2dc0 0x7fbca40d2860 secure :-1 s=READY pgs=459 cs=0 l=1 rev1=1 crypto rx=0x7fbca4074120 tx=0x7fbc9402a6f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b1f962310f5838e9 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:30:21.265 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:21.259+0000 7fbca1ffb640 1 -- 172.21.15.37:0/888899942 <== mon.0 v2:172.21.15.37:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbc94036070 con 0x7fbca40b2dc0 2024-04-25T10:30:21.265 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:21.259+0000 7fbca1ffb640 1 -- 172.21.15.37:0/888899942 <== mon.0 v2:172.21.15.37:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fbc94031600 con 0x7fbca40b2dc0 2024-04-25T10:30:21.265 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:21.259+0000 7fbca1ffb640 1 -- 172.21.15.37:0/888899942 <== mon.0 v2:172.21.15.37:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbc940319d0 con 0x7fbca40b2dc0 2024-04-25T10:30:21.265 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:21.260+0000 7fbca9306640 1 -- 172.21.15.37:0/888899942 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fbca40b2dc0 msgr2=0x7fbca40d2860 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:30:21.265 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:21.260+0000 7fbca9306640 1 --2- 172.21.15.37:0/888899942 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fbca40b2dc0 0x7fbca40d2860 secure :-1 s=READY pgs=459 cs=0 l=1 rev1=1 crypto rx=0x7fbca4074120 tx=0x7fbc9402a6f0 comp rx=0 tx=0).stop 2024-04-25T10:30:21.265 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:21.260+0000 7fbca9306640 1 -- 172.21.15.37:0/888899942 shutdown_connections 2024-04-25T10:30:21.266 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:21.260+0000 7fbca9306640 1 --2- 172.21.15.37:0/888899942 >> v2:172.21.15.203:3300/0 conn(0x7fbca406e440 0x7fbca408cc90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:30:21.266 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:21.260+0000 7fbca9306640 1 --2- 172.21.15.37:0/888899942 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7fbca406daf0 0x7fbca406df00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:30:21.266 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:21.260+0000 7fbca9306640 1 --2- 172.21.15.37:0/888899942 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fbca40b2dc0 0x7fbca40d2860 unknown :-1 s=CLOSED pgs=459 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:30:21.266 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:21.260+0000 7fbca9306640 1 -- 172.21.15.37:0/888899942 >> 172.21.15.37:0/888899942 conn(0x7fbca40b5440 msgr2=0x7fbca40b5850 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:30:21.266 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:21.260+0000 7fbca9306640 1 -- 172.21.15.37:0/888899942 shutdown_connections 2024-04-25T10:30:21.266 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:21.260+0000 7fbca9306640 1 -- 172.21.15.37:0/888899942 wait complete. 2024-04-25T10:30:21.271 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:21.262+0000 7fbca9306640 1 Processor -- start 2024-04-25T10:30:21.272 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:21.262+0000 7fbca9306640 1 -- start start 2024-04-25T10:30:21.272 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:21.263+0000 7fbca9306640 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fbca406daf0 0x7fbca41f5d10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:30:21.272 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:21.263+0000 7fbca9306640 1 --2- >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7fbca406e440 0x7fbca40b76f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:30:21.272 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:21.263+0000 7fbca9306640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fbca40b2dc0 0x7fbca40c7b20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:30:21.272 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:21.263+0000 7fbca9306640 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 -- 0x7fbca406e850 con 0x7fbca40b2dc0 2024-04-25T10:30:21.272 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:21.263+0000 7fbca9306640 1 -- --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_getmap magic: 0 -- 0x7fbca40d36e0 con 0x7fbca406daf0 2024-04-25T10:30:21.272 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:21.263+0000 7fbca9306640 1 -- --> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] -- mon_getmap magic: 0 -- 0x7fbca408d1d0 con 0x7fbca406e440 2024-04-25T10:30:21.273 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:21.263+0000 7fbca37fe640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fbca40b2dc0 0x7fbca40c7b20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:30:21.273 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:21.263+0000 7fbca37fe640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fbca40b2dc0 0x7fbca40c7b20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.37:3300/0 says I am v2:172.21.15.37:59572/0 (socket says 172.21.15.37:59572) 2024-04-25T10:30:21.273 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:21.263+0000 7fbca37fe640 1 -- 172.21.15.37:0/1637236820 learned_addr learned my addr 172.21.15.37:0/1637236820 (peer_addr_for_me v2:172.21.15.37:0/0) 2024-04-25T10:30:21.273 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:21.264+0000 7fbca27fc640 1 --2- 172.21.15.37:0/1637236820 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7fbca406e440 0x7fbca40b76f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:30:21.273 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:21.264+0000 7fbca37fe640 1 -- 172.21.15.37:0/1637236820 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7fbca406e440 msgr2=0x7fbca40b76f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:30:21.273 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:21.264+0000 7fbca37fe640 1 --2- 172.21.15.37:0/1637236820 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7fbca406e440 0x7fbca40b76f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:30:21.273 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:21.264+0000 7fbca2ffd640 1 --2- 172.21.15.37:0/1637236820 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fbca406daf0 0x7fbca41f5d10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:30:21.273 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:21.264+0000 7fbca37fe640 1 -- 172.21.15.37:0/1637236820 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fbca406daf0 msgr2=0x7fbca41f5d10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:30:21.274 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:21.264+0000 7fbca37fe640 1 --2- 172.21.15.37:0/1637236820 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fbca406daf0 0x7fbca41f5d10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:30:21.274 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:21.264+0000 7fbca37fe640 1 -- 172.21.15.37:0/1637236820 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fbca40b8260 con 0x7fbca40b2dc0 2024-04-25T10:30:21.274 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:21.264+0000 7fbca27fc640 1 --2- 172.21.15.37:0/1637236820 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7fbca406e440 0x7fbca40b76f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T10:30:21.274 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:21.264+0000 7fbca37fe640 1 --2- 172.21.15.37:0/1637236820 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fbca40b2dc0 0x7fbca40c7b20 secure :-1 s=READY pgs=460 cs=0 l=1 rev1=1 crypto rx=0x7fbc9c00c7f0 tx=0x7fbc9c00ccb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:30:21.274 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:21.264+0000 7fbc9b7fe640 1 -- 172.21.15.37:0/1637236820 <== mon.0 v2:172.21.15.37:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbc9c015020 con 0x7fbca40b2dc0 2024-04-25T10:30:21.274 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:21.264+0000 7fbc9b7fe640 1 -- 172.21.15.37:0/1637236820 <== mon.0 v2:172.21.15.37:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fbc9c004230 con 0x7fbca40b2dc0 2024-04-25T10:30:21.274 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:21.264+0000 7fbca9306640 1 -- 172.21.15.37:0/1637236820 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fbca40b8540 con 0x7fbca40b2dc0 2024-04-25T10:30:21.274 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:21.264+0000 7fbc9b7fe640 1 -- 172.21.15.37:0/1637236820 <== mon.0 v2:172.21.15.37:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbc9c007620 con 0x7fbca40b2dc0 2024-04-25T10:30:21.274 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:21.264+0000 7fbca9306640 1 -- 172.21.15.37:0/1637236820 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fbca40c9d20 con 0x7fbca40b2dc0 2024-04-25T10:30:21.275 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:21.265+0000 7fbca9306640 1 -- 172.21.15.37:0/1637236820 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fbca406a140 con 0x7fbca40b2dc0 2024-04-25T10:30:21.275 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:21.266+0000 7fbc9b7fe640 1 -- 172.21.15.37:0/1637236820 <== mon.0 v2:172.21.15.37:3300/0 4 ==== mgrmap(e 5) ==== 96782+0+0 (secure 0 0 0) 0x7fbc9c018020 con 0x7fbca40b2dc0 2024-04-25T10:30:21.275 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:21.266+0000 7fbc9b7fe640 1 --2- 172.21.15.37:0/1637236820 >> [v2:172.21.15.203:6832/2418205237,v1:172.21.15.203:6833/2418205237] conn(0x7fbc840744b0 0x7fbc840769c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:30:21.275 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:21.266+0000 7fbc9b7fe640 1 -- 172.21.15.37:0/1637236820 <== mon.0 v2:172.21.15.37:3300/0 5 ==== osd_map(36..36 src has 1..36) ==== 5709+0+0 (secure 0 0 0) 0x7fbc9c095490 con 0x7fbca40b2dc0 2024-04-25T10:30:21.275 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:21.267+0000 7fbca2ffd640 1 --2- 172.21.15.37:0/1637236820 >> [v2:172.21.15.203:6832/2418205237,v1:172.21.15.203:6833/2418205237] conn(0x7fbc840744b0 0x7fbc840769c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:30:21.275 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:21.267+0000 7fbca2ffd640 1 --2- 172.21.15.37:0/1637236820 >> [v2:172.21.15.203:6832/2418205237,v1:172.21.15.203:6833/2418205237] conn(0x7fbc840744b0 0x7fbc840769c0 secure :-1 s=READY pgs=226 cs=0 l=1 rev1=1 crypto rx=0x7fbc94033000 tx=0x7fbc94009000 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:30:21.276 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:21.271+0000 7fbc9b7fe640 1 -- 172.21.15.37:0/1637236820 <== mon.0 v2:172.21.15.37:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187246 (secure 0 0 0) 0x7fbc9c01f020 con 0x7fbca40b2dc0 2024-04-25T10:30:21.457 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:21.455+0000 7fbca9306640 1 -- 172.21.15.37:0/1637236820 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"} v 0) -- 0x7fbca4076450 con 0x7fbca40b2dc0 2024-04-25T10:30:21.458 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:21.455+0000 7fbc9b7fe640 1 -- 172.21.15.37:0/1637236820 <== mon.0 v2:172.21.15.37:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]=0 v36) ==== 104+0+11 (secure 0 0 0) 0x7fbc9c05fcd0 con 0x7fbca40b2dc0 2024-04-25T10:30:21.458 INFO:teuthology.orchestra.run.smithi037.stdout:pg_num: 32 2024-04-25T10:30:21.460 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:21.457+0000 7fbca9306640 1 -- 172.21.15.37:0/1637236820 >> [v2:172.21.15.203:6832/2418205237,v1:172.21.15.203:6833/2418205237] conn(0x7fbc840744b0 msgr2=0x7fbc840769c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:30:21.460 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:21.457+0000 7fbca9306640 1 --2- 172.21.15.37:0/1637236820 >> [v2:172.21.15.203:6832/2418205237,v1:172.21.15.203:6833/2418205237] conn(0x7fbc840744b0 0x7fbc840769c0 secure :-1 s=READY pgs=226 cs=0 l=1 rev1=1 crypto rx=0x7fbc94033000 tx=0x7fbc94009000 comp rx=0 tx=0).stop 2024-04-25T10:30:21.460 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:21.457+0000 7fbca9306640 1 -- 172.21.15.37:0/1637236820 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fbca40b2dc0 msgr2=0x7fbca40c7b20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:30:21.461 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:21.457+0000 7fbca9306640 1 --2- 172.21.15.37:0/1637236820 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fbca40b2dc0 0x7fbca40c7b20 secure :-1 s=READY pgs=460 cs=0 l=1 rev1=1 crypto rx=0x7fbc9c00c7f0 tx=0x7fbc9c00ccb0 comp rx=0 tx=0).stop 2024-04-25T10:30:21.461 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:21.457+0000 7fbca9306640 1 -- 172.21.15.37:0/1637236820 shutdown_connections 2024-04-25T10:30:21.461 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:21.457+0000 7fbca9306640 1 --2- 172.21.15.37:0/1637236820 >> [v2:172.21.15.203:6832/2418205237,v1:172.21.15.203:6833/2418205237] conn(0x7fbc840744b0 0x7fbc840769c0 unknown :-1 s=CLOSED pgs=226 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:30:21.461 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:21.457+0000 7fbca9306640 1 --2- 172.21.15.37:0/1637236820 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fbca40b2dc0 0x7fbca40c7b20 unknown :-1 s=CLOSED pgs=460 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:30:21.461 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:21.457+0000 7fbca9306640 1 --2- 172.21.15.37:0/1637236820 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7fbca406e440 0x7fbca40b76f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:30:21.462 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:21.457+0000 7fbca9306640 1 --2- 172.21.15.37:0/1637236820 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fbca406daf0 0x7fbca41f5d10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:30:21.462 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:21.457+0000 7fbca9306640 1 -- 172.21.15.37:0/1637236820 >> 172.21.15.37:0/1637236820 conn(0x7fbca40b5440 msgr2=0x7fbca40cf830 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:30:21.462 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:21.457+0000 7fbca9306640 1 -- 172.21.15.37:0/1637236820 shutdown_connections 2024-04-25T10:30:21.462 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:21.457+0000 7fbca9306640 1 -- 172.21.15.37:0/1637236820 wait complete. 2024-04-25T10:30:21.470 DEBUG:teuthology.orchestra.run.smithi037:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data pg_num 2024-04-25T10:30:21.584 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:21.582+0000 7fa533372640 1 Processor -- start 2024-04-25T10:30:21.585 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:21.582+0000 7fa533372640 1 -- start start 2024-04-25T10:30:21.587 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:21.584+0000 7fa533372640 1 --2- >> v2:172.21.15.203:3300/0 conn(0x7fa52c0b2dc0 0x7fa52c0d2860 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:30:21.587 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:21.584+0000 7fa533372640 1 --2- >> v2:172.21.15.37:3300/0 conn(0x7fa52c06e610 0x7fa52c0ce560 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:30:21.587 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:21.584+0000 7fa533372640 1 -- --> v1:172.21.15.37:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fa52c0d2e40 con 0x7fa52c06daf0 2024-04-25T10:30:21.587 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:21.584+0000 7fa533372640 1 -- --> v2:172.21.15.37:3300/0 -- mon_getmap magic: 0 -- 0x7fa52c0b92c0 con 0x7fa52c06e610 2024-04-25T10:30:21.587 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:21.584+0000 7fa533372640 1 -- --> v2:172.21.15.203:3300/0 -- mon_getmap magic: 0 -- 0x7fa52c0ce0d0 con 0x7fa52c0b2dc0 2024-04-25T10:30:21.587 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:21.584+0000 7fa53190f640 1 --2- >> v2:172.21.15.37:3300/0 conn(0x7fa52c06e610 0x7fa52c0ce560 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:30:21.587 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:21.584+0000 7fa53090d640 1 --1- >> v1:172.21.15.37:6789/0 conn(0x7fa52c06daf0 0x7fa52c06df00 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.37:6789/0 says I am v1:172.21.15.37:55076/0 (socket says 172.21.15.37:55076) 2024-04-25T10:30:21.587 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:21.584+0000 7fa53090d640 1 -- 172.21.15.37:0/2211378885 learned_addr learned my addr 172.21.15.37:0/2211378885 (peer_addr_for_me v1:172.21.15.37:0/0) 2024-04-25T10:30:21.587 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:21.584+0000 7fa53110e640 1 --2- 172.21.15.37:0/2211378885 >> v2:172.21.15.203:3300/0 conn(0x7fa52c0b2dc0 0x7fa52c0d2860 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:30:21.587 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:21.584+0000 7fa53190f640 1 -- 172.21.15.37:0/2211378885 >> v2:172.21.15.203:3300/0 conn(0x7fa52c0b2dc0 msgr2=0x7fa52c0d2860 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:30:21.587 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:21.584+0000 7fa53190f640 1 --2- 172.21.15.37:0/2211378885 >> v2:172.21.15.203:3300/0 conn(0x7fa52c0b2dc0 0x7fa52c0d2860 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:30:21.587 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:21.584+0000 7fa53190f640 1 -- 172.21.15.37:0/2211378885 >> v1:172.21.15.37:6789/0 conn(0x7fa52c06daf0 legacy=0x7fa52c06df00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:30:21.587 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:21.584+0000 7fa52bfff640 1 -- 172.21.15.37:0/2211378885 <== mon.0 v1:172.21.15.37:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 212774494 0 0) 0x7fa52c0d2e40 con 0x7fa52c06daf0 2024-04-25T10:30:21.588 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:21.584+0000 7fa53190f640 1 -- 172.21.15.37:0/2211378885 --> v2:172.21.15.37:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fa52c0d3120 con 0x7fa52c06e610 2024-04-25T10:30:21.589 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:21.585+0000 7fa53190f640 1 --2- 172.21.15.37:0/2211378885 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fa52c06e610 0x7fa52c0ce560 secure :-1 s=READY pgs=462 cs=0 l=1 rev1=1 crypto rx=0x7fa52c070b40 tx=0x7fa51c02a680 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f94deec2f1b89f3a server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:30:21.589 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:21.585+0000 7fa52bfff640 1 -- 172.21.15.37:0/2211378885 <== mon.0 v2:172.21.15.37:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa51c007740 con 0x7fa52c06e610 2024-04-25T10:30:21.589 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:21.585+0000 7fa52bfff640 1 -- 172.21.15.37:0/2211378885 <== mon.0 v2:172.21.15.37:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fa51c0078e0 con 0x7fa52c06e610 2024-04-25T10:30:21.589 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:21.585+0000 7fa52bfff640 1 -- 172.21.15.37:0/2211378885 <== mon.0 v2:172.21.15.37:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa51c007cb0 con 0x7fa52c06e610 2024-04-25T10:30:21.589 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:21.586+0000 7fa533372640 1 -- 172.21.15.37:0/2211378885 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fa52c06e610 msgr2=0x7fa52c0ce560 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:30:21.589 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:21.586+0000 7fa533372640 1 --2- 172.21.15.37:0/2211378885 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fa52c06e610 0x7fa52c0ce560 secure :-1 s=READY pgs=462 cs=0 l=1 rev1=1 crypto rx=0x7fa52c070b40 tx=0x7fa51c02a680 comp rx=0 tx=0).stop 2024-04-25T10:30:21.589 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:21.586+0000 7fa533372640 1 -- 172.21.15.37:0/2211378885 shutdown_connections 2024-04-25T10:30:21.589 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:21.586+0000 7fa533372640 1 --2- 172.21.15.37:0/2211378885 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fa52c06e610 0x7fa52c0ce560 unknown :-1 s=CLOSED pgs=462 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:30:21.589 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:21.586+0000 7fa533372640 1 --2- 172.21.15.37:0/2211378885 >> v2:172.21.15.203:3300/0 conn(0x7fa52c0b2dc0 0x7fa52c0d2860 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:30:21.589 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:21.586+0000 7fa533372640 1 -- 172.21.15.37:0/2211378885 >> 172.21.15.37:0/2211378885 conn(0x7fa52c0b5440 msgr2=0x7fa52c0b5850 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:30:21.589 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:21.586+0000 7fa533372640 1 -- 172.21.15.37:0/2211378885 shutdown_connections 2024-04-25T10:30:21.590 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:21.586+0000 7fa533372640 1 -- 172.21.15.37:0/2211378885 wait complete. 2024-04-25T10:30:21.590 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:21.588+0000 7fa533372640 1 Processor -- start 2024-04-25T10:30:21.590 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:21.588+0000 7fa533372640 1 -- start start 2024-04-25T10:30:21.592 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:21.589+0000 7fa533372640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fa52c06daf0 0x7fa52c163040 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:30:21.592 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:21.589+0000 7fa533372640 1 --2- >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7fa52c06e610 0x7fa52c0c9620 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:30:21.592 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:21.589+0000 7fa533372640 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fa52c0b2dc0 0x7fa52c1f98c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:30:21.592 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:21.589+0000 7fa533372640 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 -- 0x7fa52c0ceaa0 con 0x7fa52c06daf0 2024-04-25T10:30:21.592 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:21.589+0000 7fa533372640 1 -- --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_getmap magic: 0 -- 0x7fa52c0d36e0 con 0x7fa52c0b2dc0 2024-04-25T10:30:21.592 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:21.589+0000 7fa533372640 1 -- --> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] -- mon_getmap magic: 0 -- 0x7fa52c0cec20 con 0x7fa52c06e610 2024-04-25T10:30:21.592 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:21.589+0000 7fa53110e640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fa52c06daf0 0x7fa52c163040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:30:21.593 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:21.589+0000 7fa53110e640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fa52c06daf0 0x7fa52c163040 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.37:3300/0 says I am v2:172.21.15.37:59600/0 (socket says 172.21.15.37:59600) 2024-04-25T10:30:21.593 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:21.589+0000 7fa53110e640 1 -- 172.21.15.37:0/1206109496 learned_addr learned my addr 172.21.15.37:0/1206109496 (peer_addr_for_me v2:172.21.15.37:0/0) 2024-04-25T10:30:21.593 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:21.590+0000 7fa53110e640 1 -- 172.21.15.37:0/1206109496 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7fa52c06e610 msgr2=0x7fa52c0c9620 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:30:21.593 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:21.590+0000 7fa53110e640 1 --2- 172.21.15.37:0/1206109496 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7fa52c06e610 0x7fa52c0c9620 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:30:21.593 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:21.590+0000 7fa53110e640 1 -- 172.21.15.37:0/1206109496 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fa52c0b2dc0 msgr2=0x7fa52c1f98c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:30:21.593 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:21.590+0000 7fa53110e640 1 --2- 172.21.15.37:0/1206109496 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fa52c0b2dc0 0x7fa52c1f98c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:30:21.593 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:21.590+0000 7fa53110e640 1 -- 172.21.15.37:0/1206109496 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fa52c0cb960 con 0x7fa52c06daf0 2024-04-25T10:30:21.593 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:21.590+0000 7fa53110e640 1 --2- 172.21.15.37:0/1206109496 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fa52c06daf0 0x7fa52c163040 secure :-1 s=READY pgs=463 cs=0 l=1 rev1=1 crypto rx=0x7fa52400e870 tx=0x7fa52400ed30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:30:21.593 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:21.590+0000 7fa529ffb640 1 -- 172.21.15.37:0/1206109496 <== mon.0 v2:172.21.15.37:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa524017070 con 0x7fa52c06daf0 2024-04-25T10:30:21.593 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:21.590+0000 7fa529ffb640 1 -- 172.21.15.37:0/1206109496 <== mon.0 v2:172.21.15.37:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fa52400c940 con 0x7fa52c06daf0 2024-04-25T10:30:21.593 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:21.590+0000 7fa533372640 1 -- 172.21.15.37:0/1206109496 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fa52c0cbbd0 con 0x7fa52c06daf0 2024-04-25T10:30:21.593 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:21.590+0000 7fa529ffb640 1 -- 172.21.15.37:0/1206109496 <== mon.0 v2:172.21.15.37:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa52400cd10 con 0x7fa52c06daf0 2024-04-25T10:30:21.593 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:21.590+0000 7fa533372640 1 -- 172.21.15.37:0/1206109496 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fa52c14be30 con 0x7fa52c06daf0 2024-04-25T10:30:21.594 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:21.591+0000 7fa533372640 1 -- 172.21.15.37:0/1206109496 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fa52c06a140 con 0x7fa52c06daf0 2024-04-25T10:30:21.594 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:21.592+0000 7fa529ffb640 1 -- 172.21.15.37:0/1206109496 <== mon.0 v2:172.21.15.37:3300/0 4 ==== mgrmap(e 5) ==== 96782+0+0 (secure 0 0 0) 0x7fa52401a070 con 0x7fa52c06daf0 2024-04-25T10:30:21.594 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:21.592+0000 7fa529ffb640 1 --2- 172.21.15.37:0/1206109496 >> [v2:172.21.15.203:6832/2418205237,v1:172.21.15.203:6833/2418205237] conn(0x7fa5180744b0 0x7fa5180769c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:30:21.594 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:21.592+0000 7fa529ffb640 1 -- 172.21.15.37:0/1206109496 <== mon.0 v2:172.21.15.37:3300/0 5 ==== osd_map(36..36 src has 1..36) ==== 5709+0+0 (secure 0 0 0) 0x7fa524095520 con 0x7fa52c06daf0 2024-04-25T10:30:21.595 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:21.593+0000 7fa53090d640 1 --2- 172.21.15.37:0/1206109496 >> [v2:172.21.15.203:6832/2418205237,v1:172.21.15.203:6833/2418205237] conn(0x7fa5180744b0 0x7fa5180769c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:30:21.595 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:21.593+0000 7fa53090d640 1 --2- 172.21.15.37:0/1206109496 >> [v2:172.21.15.203:6832/2418205237,v1:172.21.15.203:6833/2418205237] conn(0x7fa5180744b0 0x7fa5180769c0 secure :-1 s=READY pgs=227 cs=0 l=1 rev1=1 crypto rx=0x7fa520007fd0 tx=0x7fa520007960 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:30:21.599 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:21.597+0000 7fa529ffb640 1 -- 172.21.15.37:0/1206109496 <== mon.0 v2:172.21.15.37:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187246 (secure 0 0 0) 0x7fa52405fe00 con 0x7fa52c06daf0 2024-04-25T10:30:21.784 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:21.781+0000 7fa533372640 1 -- 172.21.15.37:0/1206109496 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"} v 0) -- 0x7fa52c06df00 con 0x7fa52c06daf0 2024-04-25T10:30:21.784 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:21.781+0000 7fa529ffb640 1 -- 172.21.15.37:0/1206109496 <== mon.0 v2:172.21.15.37:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]=0 v36) ==== 100+0+11 (secure 0 0 0) 0x7fa52405f700 con 0x7fa52c06daf0 2024-04-25T10:30:21.784 INFO:teuthology.orchestra.run.smithi037.stdout:pg_num: 64 2024-04-25T10:30:21.786 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:21.783+0000 7fa533372640 1 -- 172.21.15.37:0/1206109496 >> [v2:172.21.15.203:6832/2418205237,v1:172.21.15.203:6833/2418205237] conn(0x7fa5180744b0 msgr2=0x7fa5180769c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:30:21.786 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:21.783+0000 7fa533372640 1 --2- 172.21.15.37:0/1206109496 >> [v2:172.21.15.203:6832/2418205237,v1:172.21.15.203:6833/2418205237] conn(0x7fa5180744b0 0x7fa5180769c0 secure :-1 s=READY pgs=227 cs=0 l=1 rev1=1 crypto rx=0x7fa520007fd0 tx=0x7fa520007960 comp rx=0 tx=0).stop 2024-04-25T10:30:21.786 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:21.783+0000 7fa533372640 1 -- 172.21.15.37:0/1206109496 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fa52c06daf0 msgr2=0x7fa52c163040 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:30:21.786 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:21.783+0000 7fa533372640 1 --2- 172.21.15.37:0/1206109496 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fa52c06daf0 0x7fa52c163040 secure :-1 s=READY pgs=463 cs=0 l=1 rev1=1 crypto rx=0x7fa52400e870 tx=0x7fa52400ed30 comp rx=0 tx=0).stop 2024-04-25T10:30:21.786 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:21.783+0000 7fa533372640 1 -- 172.21.15.37:0/1206109496 shutdown_connections 2024-04-25T10:30:21.786 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:21.783+0000 7fa533372640 1 --2- 172.21.15.37:0/1206109496 >> [v2:172.21.15.203:6832/2418205237,v1:172.21.15.203:6833/2418205237] conn(0x7fa5180744b0 0x7fa5180769c0 unknown :-1 s=CLOSED pgs=227 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:30:21.786 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:21.783+0000 7fa533372640 1 --2- 172.21.15.37:0/1206109496 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fa52c0b2dc0 0x7fa52c1f98c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:30:21.787 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:21.783+0000 7fa533372640 1 --2- 172.21.15.37:0/1206109496 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7fa52c06e610 0x7fa52c0c9620 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:30:21.787 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:21.783+0000 7fa533372640 1 --2- 172.21.15.37:0/1206109496 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fa52c06daf0 0x7fa52c163040 unknown :-1 s=CLOSED pgs=463 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:30:21.787 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:21.783+0000 7fa533372640 1 -- 172.21.15.37:0/1206109496 >> 172.21.15.37:0/1206109496 conn(0x7fa52c0b5440 msgr2=0x7fa52c0b0740 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:30:21.787 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:21.784+0000 7fa533372640 1 -- 172.21.15.37:0/1206109496 shutdown_connections 2024-04-25T10:30:21.787 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:21.784+0000 7fa533372640 1 -- 172.21.15.37:0/1206109496 wait complete. 2024-04-25T10:30:21.797 DEBUG:teuthology.orchestra.run.smithi037:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd blocklist ls 2024-04-25T10:30:21.909 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:21.907+0000 7feff7ce7640 1 Processor -- start 2024-04-25T10:30:21.909 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:21.907+0000 7feff7ce7640 1 -- start start 2024-04-25T10:30:21.912 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:21.909+0000 7feff7ce7640 1 --2- >> v2:172.21.15.37:3300/0 conn(0x7feff006e260 0x7feff006e670 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:30:21.912 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:21.909+0000 7feff7ce7640 1 --2- >> v2:172.21.15.203:3300/0 conn(0x7feff006ebb0 0x7feff01645d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:30:21.912 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:21.909+0000 7feff7ce7640 1 -- --> v1:172.21.15.203:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7feff009ce70 con 0x7feff0164b10 2024-04-25T10:30:21.912 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:21.909+0000 7feff7ce7640 1 -- --> v2:172.21.15.37:3300/0 -- mon_getmap magic: 0 -- 0x7feff00d3370 con 0x7feff006e260 2024-04-25T10:30:21.912 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:21.909+0000 7feff7ce7640 1 -- --> v2:172.21.15.203:3300/0 -- mon_getmap magic: 0 -- 0x7feff00d3090 con 0x7feff006ebb0 2024-04-25T10:30:21.912 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:21.909+0000 7feff5a83640 1 --2- >> v2:172.21.15.37:3300/0 conn(0x7feff006e260 0x7feff006e670 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:30:21.912 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:21.909+0000 7feff5a83640 1 --2- >> v2:172.21.15.37:3300/0 conn(0x7feff006e260 0x7feff006e670 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.37:3300/0 says I am v2:172.21.15.37:59616/0 (socket says 172.21.15.37:59616) 2024-04-25T10:30:21.912 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:21.909+0000 7feff5a83640 1 -- 172.21.15.37:0/180132080 learned_addr learned my addr 172.21.15.37:0/180132080 (peer_addr_for_me v2:172.21.15.37:0/0) 2024-04-25T10:30:21.913 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:21.909+0000 7feff5282640 1 --2- 172.21.15.37:0/180132080 >> v2:172.21.15.203:3300/0 conn(0x7feff006ebb0 0x7feff01645d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:30:21.913 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:21.910+0000 7feff5a83640 1 -- 172.21.15.37:0/180132080 >> v2:172.21.15.203:3300/0 conn(0x7feff006ebb0 msgr2=0x7feff01645d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:30:21.913 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:21.910+0000 7feff5a83640 1 --2- 172.21.15.37:0/180132080 >> v2:172.21.15.203:3300/0 conn(0x7feff006ebb0 0x7feff01645d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:30:21.913 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:21.910+0000 7feff5a83640 1 -- 172.21.15.37:0/180132080 >> v1:172.21.15.203:6789/0 conn(0x7feff0164b10 legacy=0x7feff00d1ec0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:30:21.913 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:21.910+0000 7feff5a83640 1 -- 172.21.15.37:0/180132080 --> v2:172.21.15.37:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7feff009d150 con 0x7feff006e260 2024-04-25T10:30:21.914 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:21.910+0000 7feff5a83640 1 --2- 172.21.15.37:0/180132080 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7feff006e260 0x7feff006e670 secure :-1 s=READY pgs=464 cs=0 l=1 rev1=1 crypto rx=0x7feff0074b20 tx=0x7fefe802a1e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=de86fdc6e6e5a545 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:30:21.914 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:21.910+0000 7feff4a81640 1 -- 172.21.15.37:0/180132080 <== mon.0 v2:172.21.15.37:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fefe8037070 con 0x7feff006e260 2024-04-25T10:30:21.914 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:21.911+0000 7feff4a81640 1 -- 172.21.15.37:0/180132080 <== mon.0 v2:172.21.15.37:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fefe8032440 con 0x7feff006e260 2024-04-25T10:30:21.914 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:21.911+0000 7feff4a81640 1 -- 172.21.15.37:0/180132080 <== mon.0 v2:172.21.15.37:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fefe8032810 con 0x7feff006e260 2024-04-25T10:30:21.914 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:21.911+0000 7feff7ce7640 1 -- 172.21.15.37:0/180132080 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7feff006e260 msgr2=0x7feff006e670 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:30:21.914 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:21.911+0000 7feff7ce7640 1 --2- 172.21.15.37:0/180132080 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7feff006e260 0x7feff006e670 secure :-1 s=READY pgs=464 cs=0 l=1 rev1=1 crypto rx=0x7feff0074b20 tx=0x7fefe802a1e0 comp rx=0 tx=0).stop 2024-04-25T10:30:21.914 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:21.911+0000 7feff7ce7640 1 -- 172.21.15.37:0/180132080 shutdown_connections 2024-04-25T10:30:21.914 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:21.911+0000 7feff7ce7640 1 --2- 172.21.15.37:0/180132080 >> v2:172.21.15.203:3300/0 conn(0x7feff006ebb0 0x7feff01645d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:30:21.915 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:21.911+0000 7feff7ce7640 1 --2- 172.21.15.37:0/180132080 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7feff006e260 0x7feff006e670 unknown :-1 s=CLOSED pgs=464 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:30:21.915 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:21.911+0000 7feff7ce7640 1 -- 172.21.15.37:0/180132080 >> 172.21.15.37:0/180132080 conn(0x7feff00d0390 msgr2=0x7feff00d07a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:30:21.915 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:21.911+0000 7feff7ce7640 1 -- 172.21.15.37:0/180132080 shutdown_connections 2024-04-25T10:30:21.915 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:21.911+0000 7feff7ce7640 1 -- 172.21.15.37:0/180132080 wait complete. 2024-04-25T10:30:21.915 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:21.913+0000 7feff7ce7640 1 Processor -- start 2024-04-25T10:30:21.915 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:21.913+0000 7feff7ce7640 1 -- start start 2024-04-25T10:30:21.918 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:21.915+0000 7feff7ce7640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7feff006e260 0x7feff00be0d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:30:21.918 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:21.915+0000 7feff7ce7640 1 --2- >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7feff006ebb0 0x7feff0159910 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:30:21.919 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:21.915+0000 7feff7ce7640 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7feff0164b10 0x7feff00b7bf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:30:21.919 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:21.915+0000 7feff7ce7640 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 -- 0x7feff006db30 con 0x7feff006e260 2024-04-25T10:30:21.919 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:21.915+0000 7feff7ce7640 1 -- --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_getmap magic: 0 -- 0x7feff006d830 con 0x7feff0164b10 2024-04-25T10:30:21.919 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:21.915+0000 7feff7ce7640 1 -- --> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] -- mon_getmap magic: 0 -- 0x7feff006dcb0 con 0x7feff006ebb0 2024-04-25T10:30:21.919 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:21.915+0000 7feff5a83640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7feff006e260 0x7feff00be0d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:30:21.919 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:21.915+0000 7feff5a83640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7feff006e260 0x7feff00be0d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.37:3300/0 says I am v2:172.21.15.37:59626/0 (socket says 172.21.15.37:59626) 2024-04-25T10:30:21.919 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:21.915+0000 7feff5a83640 1 -- 172.21.15.37:0/3892162862 learned_addr learned my addr 172.21.15.37:0/3892162862 (peer_addr_for_me v2:172.21.15.37:0/0) 2024-04-25T10:30:21.919 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:21.915+0000 7feff5282640 1 --2- 172.21.15.37:0/3892162862 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7feff006ebb0 0x7feff0159910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:30:21.919 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:21.915+0000 7feff6284640 1 --2- 172.21.15.37:0/3892162862 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7feff0164b10 0x7feff00b7bf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:30:21.919 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:21.915+0000 7feff5a83640 1 -- 172.21.15.37:0/3892162862 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7feff006ebb0 msgr2=0x7feff0159910 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:30:21.919 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:21.915+0000 7feff5a83640 1 --2- 172.21.15.37:0/3892162862 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7feff006ebb0 0x7feff0159910 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:30:21.919 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:21.915+0000 7feff5a83640 1 -- 172.21.15.37:0/3892162862 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7feff0164b10 msgr2=0x7feff00b7bf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:30:21.919 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:21.915+0000 7feff5a83640 1 --2- 172.21.15.37:0/3892162862 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7feff0164b10 0x7feff00b7bf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:30:21.919 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:21.915+0000 7feff5a83640 1 -- 172.21.15.37:0/3892162862 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7feff00bec80 con 0x7feff006e260 2024-04-25T10:30:21.920 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:21.915+0000 7feff5282640 1 --2- 172.21.15.37:0/3892162862 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7feff006ebb0 0x7feff0159910 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T10:30:21.920 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:21.915+0000 7feff6284640 1 --2- 172.21.15.37:0/3892162862 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7feff0164b10 0x7feff00b7bf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T10:30:21.920 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:21.915+0000 7feff5a83640 1 --2- 172.21.15.37:0/3892162862 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7feff006e260 0x7feff00be0d0 secure :-1 s=READY pgs=465 cs=0 l=1 rev1=1 crypto rx=0x7fefe802a220 tx=0x7fefe80046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:30:21.920 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:21.915+0000 7fefee7fc640 1 -- 172.21.15.37:0/3892162862 <== mon.0 v2:172.21.15.37:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fefe8037040 con 0x7feff006e260 2024-04-25T10:30:21.920 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:21.915+0000 7fefee7fc640 1 -- 172.21.15.37:0/3892162862 <== mon.0 v2:172.21.15.37:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fefe8042070 con 0x7feff006e260 2024-04-25T10:30:21.920 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:21.915+0000 7feff7ce7640 1 -- 172.21.15.37:0/3892162862 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7feff00bef60 con 0x7feff006e260 2024-04-25T10:30:21.920 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:21.915+0000 7fefee7fc640 1 -- 172.21.15.37:0/3892162862 <== mon.0 v2:172.21.15.37:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fefe8030880 con 0x7feff006e260 2024-04-25T10:30:21.920 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:21.915+0000 7feff7ce7640 1 -- 172.21.15.37:0/3892162862 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7feff00a0d90 con 0x7feff006e260 2024-04-25T10:30:21.920 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:21.916+0000 7feff7ce7640 1 -- 172.21.15.37:0/3892162862 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7feff00d12e0 con 0x7feff006e260 2024-04-25T10:30:21.921 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:21.917+0000 7fefee7fc640 1 -- 172.21.15.37:0/3892162862 <== mon.0 v2:172.21.15.37:3300/0 4 ==== mgrmap(e 5) ==== 96782+0+0 (secure 0 0 0) 0x7fefe8030a20 con 0x7feff006e260 2024-04-25T10:30:21.921 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:21.917+0000 7fefee7fc640 1 --2- 172.21.15.37:0/3892162862 >> [v2:172.21.15.203:6832/2418205237,v1:172.21.15.203:6833/2418205237] conn(0x7fefdc0744b0 0x7fefdc0769c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:30:21.921 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:21.918+0000 7fefee7fc640 1 -- 172.21.15.37:0/3892162862 <== mon.0 v2:172.21.15.37:3300/0 5 ==== osd_map(36..36 src has 1..36) ==== 5709+0+0 (secure 0 0 0) 0x7fefe80b5b80 con 0x7feff006e260 2024-04-25T10:30:21.921 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:21.918+0000 7feff5282640 1 --2- 172.21.15.37:0/3892162862 >> [v2:172.21.15.203:6832/2418205237,v1:172.21.15.203:6833/2418205237] conn(0x7fefdc0744b0 0x7fefdc0769c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:30:21.921 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:21.919+0000 7feff5282640 1 --2- 172.21.15.37:0/3892162862 >> [v2:172.21.15.203:6832/2418205237,v1:172.21.15.203:6833/2418205237] conn(0x7fefdc0744b0 0x7fefdc0769c0 secure :-1 s=READY pgs=228 cs=0 l=1 rev1=1 crypto rx=0x7feff00cda60 tx=0x7fefe4008040 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:30:21.924 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:21.922+0000 7fefee7fc640 1 -- 172.21.15.37:0/3892162862 <== mon.0 v2:172.21.15.37:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187246 (secure 0 0 0) 0x7fefe8080310 con 0x7feff006e260 2024-04-25T10:30:22.122 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:22.119+0000 7feff7ce7640 1 -- 172.21.15.37:0/3892162862 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_command({"prefix": "osd blocklist ls"} v 0) -- 0x7feff006a140 con 0x7feff006e260 2024-04-25T10:30:22.122 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:22.119+0000 7fefee7fc640 1 -- 172.21.15.37:0/3892162862 <== mon.0 v2:172.21.15.37:3300/0 7 ==== mon_command_ack([{"prefix": "osd blocklist ls"}]=0 listed 0 entries v36) ==== 80+0+0 (secure 0 0 0) 0x7fefe807fc10 con 0x7feff006e260 2024-04-25T10:30:22.122 INFO:teuthology.orchestra.run.smithi037.stderr:listed 0 entries 2024-04-25T10:30:22.124 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:22.121+0000 7feff7ce7640 1 -- 172.21.15.37:0/3892162862 >> [v2:172.21.15.203:6832/2418205237,v1:172.21.15.203:6833/2418205237] conn(0x7fefdc0744b0 msgr2=0x7fefdc0769c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:30:22.124 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:22.121+0000 7feff7ce7640 1 --2- 172.21.15.37:0/3892162862 >> [v2:172.21.15.203:6832/2418205237,v1:172.21.15.203:6833/2418205237] conn(0x7fefdc0744b0 0x7fefdc0769c0 secure :-1 s=READY pgs=228 cs=0 l=1 rev1=1 crypto rx=0x7feff00cda60 tx=0x7fefe4008040 comp rx=0 tx=0).stop 2024-04-25T10:30:22.124 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:22.121+0000 7feff7ce7640 1 -- 172.21.15.37:0/3892162862 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7feff006e260 msgr2=0x7feff00be0d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:30:22.124 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:22.121+0000 7feff7ce7640 1 --2- 172.21.15.37:0/3892162862 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7feff006e260 0x7feff00be0d0 secure :-1 s=READY pgs=465 cs=0 l=1 rev1=1 crypto rx=0x7fefe802a220 tx=0x7fefe80046c0 comp rx=0 tx=0).stop 2024-04-25T10:30:22.124 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:22.121+0000 7feff7ce7640 1 -- 172.21.15.37:0/3892162862 shutdown_connections 2024-04-25T10:30:22.124 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:22.121+0000 7feff7ce7640 1 --2- 172.21.15.37:0/3892162862 >> [v2:172.21.15.203:6832/2418205237,v1:172.21.15.203:6833/2418205237] conn(0x7fefdc0744b0 0x7fefdc0769c0 unknown :-1 s=CLOSED pgs=228 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:30:22.124 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:22.121+0000 7feff7ce7640 1 --2- 172.21.15.37:0/3892162862 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7feff0164b10 0x7feff00b7bf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:30:22.124 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:22.121+0000 7feff7ce7640 1 --2- 172.21.15.37:0/3892162862 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7feff006ebb0 0x7feff0159910 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:30:22.124 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:22.121+0000 7feff7ce7640 1 --2- 172.21.15.37:0/3892162862 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7feff006e260 0x7feff00be0d0 unknown :-1 s=CLOSED pgs=465 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:30:22.124 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:22.121+0000 7feff7ce7640 1 -- 172.21.15.37:0/3892162862 >> 172.21.15.37:0/3892162862 conn(0x7feff00d0390 msgr2=0x7feff00712f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:30:22.124 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:22.121+0000 7feff7ce7640 1 -- 172.21.15.37:0/3892162862 shutdown_connections 2024-04-25T10:30:22.125 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:22.121+0000 7feff7ce7640 1 -- 172.21.15.37:0/3892162862 wait complete. 2024-04-25T10:30:22.134 DEBUG:teuthology.orchestra.run.smithi037:> set -ex 2024-04-25T10:30:22.135 DEBUG:teuthology.orchestra.run.smithi037:> dd if=/proc/self/mounts of=/dev/stdout 2024-04-25T10:30:22.151 DEBUG:teuthology.orchestra.run.smithi037:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-04-25T10:30:22.302 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:22.300+0000 7f036051f640 1 Processor -- start 2024-04-25T10:30:22.302 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:22.300+0000 7f036051f640 1 -- start start 2024-04-25T10:30:22.306 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:22.301+0000 7f036051f640 1 --2- >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f035809ced0 0x7f03580b71d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:30:22.306 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:22.301+0000 7f036051f640 1 --2- >> v2:172.21.15.203:3300/0 conn(0x7f03580b7710 0x7f03580b7b20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:30:22.306 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:22.301+0000 7f036051f640 1 -- --> v1:172.21.15.37:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f03580b8dc0 con 0x7f035808cc90 2024-04-25T10:30:22.306 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:22.301+0000 7f036051f640 1 -- --> v2:172.21.15.203:3300/0 -- mon_getmap magic: 0 -- 0x7f03580d1e10 con 0x7f03580b7710 2024-04-25T10:30:22.306 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:22.301+0000 7f036051f640 1 -- --> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] -- mon_getmap magic: 0 -- 0x7f035809d2e0 con 0x7f035809ced0 2024-04-25T10:30:22.306 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:22.301+0000 7f035e2bb640 1 --1- >> v1:172.21.15.37:6789/0 conn(0x7f035808cc90 0x7f03580d21c0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.37:6789/0 says I am v1:172.21.15.37:55084/0 (socket says 172.21.15.37:55084) 2024-04-25T10:30:22.306 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:22.301+0000 7f035e2bb640 1 -- 172.21.15.37:0/1260263945 learned_addr learned my addr 172.21.15.37:0/1260263945 (peer_addr_for_me v1:172.21.15.37:0/0) 2024-04-25T10:30:22.307 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:22.302+0000 7f035daba640 1 --2- 172.21.15.37:0/1260263945 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f035809ced0 0x7f03580b71d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:30:22.307 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:22.302+0000 7f035eabc640 1 --2- 172.21.15.37:0/1260263945 >> v2:172.21.15.203:3300/0 conn(0x7f03580b7710 0x7f03580b7b20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:30:22.307 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:22.302+0000 7f035d2b9640 1 -- 172.21.15.37:0/1260263945 <== mon.0 v1:172.21.15.37:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 1719565509 0 0) 0x7f03580b8dc0 con 0x7f035808cc90 2024-04-25T10:30:22.307 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:22.302+0000 7f035d2b9640 1 -- 172.21.15.37:0/1260263945 --> v1:172.21.15.37:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f0344003520 con 0x7f035808cc90 2024-04-25T10:30:22.307 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:22.302+0000 7f035d2b9640 1 -- 172.21.15.37:0/1260263945 <== mon.0 v1:172.21.15.37:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 2758447626 0 0) 0x7f0344003520 con 0x7f035808cc90 2024-04-25T10:30:22.307 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:22.302+0000 7f035d2b9640 1 -- 172.21.15.37:0/1260263945 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f035809ced0 msgr2=0x7f03580b71d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:30:22.307 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:22.302+0000 7f035d2b9640 1 --2- 172.21.15.37:0/1260263945 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f035809ced0 0x7f03580b71d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:30:22.307 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:22.302+0000 7f035d2b9640 1 -- 172.21.15.37:0/1260263945 >> v2:172.21.15.203:3300/0 conn(0x7f03580b7710 msgr2=0x7f03580b7b20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:30:22.307 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:22.302+0000 7f035d2b9640 1 --2- 172.21.15.37:0/1260263945 >> v2:172.21.15.203:3300/0 conn(0x7f03580b7710 0x7f03580b7b20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:30:22.307 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:22.302+0000 7f035d2b9640 1 -- 172.21.15.37:0/1260263945 --> v1:172.21.15.37:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f03580d1820 con 0x7f035808cc90 2024-04-25T10:30:22.307 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:22.302+0000 7f035d2b9640 1 -- 172.21.15.37:0/1260263945 <== mon.0 v1:172.21.15.37:6789/0 3 ==== mon_map magic: 0 ==== 413+0+0 (unknown 1577794600 0 0) 0x7f03540017f0 con 0x7f035808cc90 2024-04-25T10:30:22.307 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:22.302+0000 7f035d2b9640 1 -- 172.21.15.37:0/1260263945 >> v1:172.21.15.37:6789/0 conn(0x7f035808cc90 legacy=0x7f03580d21c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:30:22.307 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:22.302+0000 7f035d2b9640 1 --2- 172.21.15.37:0/1260263945 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f03440039b0 0x7f0344003e20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:30:22.307 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:22.302+0000 7f035d2b9640 1 --2- 172.21.15.37:0/1260263945 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f0344004800 0x7f0344004c10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:30:22.308 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:22.302+0000 7f035d2b9640 1 --2- 172.21.15.37:0/1260263945 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f035809ced0 0x7f0344005170 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:30:22.308 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:22.302+0000 7f035d2b9640 1 -- 172.21.15.37:0/1260263945 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 -- 0x7f03580d1e10 con 0x7f03440039b0 2024-04-25T10:30:22.308 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:22.302+0000 7f035d2b9640 1 -- 172.21.15.37:0/1260263945 --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_getmap magic: 0 -- 0x7f035809d2e0 con 0x7f0344004800 2024-04-25T10:30:22.308 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:22.302+0000 7f035d2b9640 1 -- 172.21.15.37:0/1260263945 --> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] -- mon_getmap magic: 0 -- 0x7f034400baf0 con 0x7f035809ced0 2024-04-25T10:30:22.308 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:22.302+0000 7f035e2bb640 1 --2- 172.21.15.37:0/1260263945 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f03440039b0 0x7f0344003e20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:30:22.308 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:22.302+0000 7f035daba640 1 --2- 172.21.15.37:0/1260263945 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f0344004800 0x7f0344004c10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:30:22.308 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:22.302+0000 7f035eabc640 1 --2- 172.21.15.37:0/1260263945 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f035809ced0 0x7f0344005170 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:30:22.308 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:22.302+0000 7f035e2bb640 1 -- 172.21.15.37:0/1260263945 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f035809ced0 msgr2=0x7f0344005170 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:30:22.308 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:22.302+0000 7f035e2bb640 1 --2- 172.21.15.37:0/1260263945 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f035809ced0 0x7f0344005170 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:30:22.308 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:22.302+0000 7f035e2bb640 1 -- 172.21.15.37:0/1260263945 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f0344004800 msgr2=0x7f0344004c10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:30:22.308 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:22.302+0000 7f035e2bb640 1 --2- 172.21.15.37:0/1260263945 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f0344004800 0x7f0344004c10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:30:22.308 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:22.302+0000 7f035e2bb640 1 -- 172.21.15.37:0/1260263945 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f0344003520 con 0x7f03440039b0 2024-04-25T10:30:22.308 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:22.302+0000 7f035eabc640 1 --2- 172.21.15.37:0/1260263945 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f035809ced0 0x7f0344005170 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T10:30:22.308 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:22.303+0000 7f035e2bb640 1 --2- 172.21.15.37:0/1260263945 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f03440039b0 0x7f0344003e20 secure :-1 s=READY pgs=467 cs=0 l=1 rev1=1 crypto rx=0x7f035400c210 tx=0x7f0344005830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=3f5a23957045530 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:30:22.309 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:22.303+0000 7f035d2b9640 1 -- 172.21.15.37:0/1260263945 <== mon.0 v2:172.21.15.37:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0354007d20 con 0x7f03440039b0 2024-04-25T10:30:22.309 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:22.303+0000 7f035d2b9640 1 -- 172.21.15.37:0/1260263945 <== mon.0 v2:172.21.15.37:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f0354007290 con 0x7f03440039b0 2024-04-25T10:30:22.309 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:22.303+0000 7f035d2b9640 1 -- 172.21.15.37:0/1260263945 <== mon.0 v2:172.21.15.37:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f03540057a0 con 0x7f03440039b0 2024-04-25T10:30:22.309 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:22.304+0000 7f036051f640 1 -- 172.21.15.37:0/1260263945 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f03440039b0 msgr2=0x7f0344003e20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:30:22.309 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:22.304+0000 7f036051f640 1 --2- 172.21.15.37:0/1260263945 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f03440039b0 0x7f0344003e20 secure :-1 s=READY pgs=467 cs=0 l=1 rev1=1 crypto rx=0x7f035400c210 tx=0x7f0344005830 comp rx=0 tx=0).stop 2024-04-25T10:30:22.309 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:22.304+0000 7f035eabc640 1 -- 172.21.15.37:0/1260263945 reap_dead start 2024-04-25T10:30:22.309 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:22.304+0000 7f036051f640 1 -- 172.21.15.37:0/1260263945 shutdown_connections 2024-04-25T10:30:22.309 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:22.304+0000 7f036051f640 1 -- 172.21.15.37:0/1260263945 >> 172.21.15.37:0/1260263945 conn(0x7f03580d03f0 msgr2=0x7f03580d0800 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:30:22.309 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:22.304+0000 7f036051f640 1 -- 172.21.15.37:0/1260263945 shutdown_connections 2024-04-25T10:30:22.309 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:22.304+0000 7f036051f640 1 -- 172.21.15.37:0/1260263945 wait complete. 2024-04-25T10:30:22.311 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:22.306+0000 7f036051f640 1 Processor -- start 2024-04-25T10:30:22.311 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:22.306+0000 7f036051f640 1 -- start start 2024-04-25T10:30:22.312 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:22.307+0000 7f036051f640 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f03440039b0 0x7f03581dcdf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:30:22.312 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:22.307+0000 7f036051f640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f03580cb5c0 0x7f03581e8650 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:30:22.312 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:22.307+0000 7f036051f640 1 --2- >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f03580cce50 0x7f03580cd260 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:30:22.312 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:22.307+0000 7f036051f640 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 -- 0x7f03580b9210 con 0x7f03580cb5c0 2024-04-25T10:30:22.312 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:22.307+0000 7f036051f640 1 -- --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_getmap magic: 0 -- 0x7f03580d1b30 con 0x7f03440039b0 2024-04-25T10:30:22.312 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:22.307+0000 7f036051f640 1 -- --> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] -- mon_getmap magic: 0 -- 0x7f03580d28d0 con 0x7f03580cce50 2024-04-25T10:30:22.312 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:22.307+0000 7f035daba640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f03580cb5c0 0x7f03581e8650 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:30:22.312 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:22.307+0000 7f035daba640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f03580cb5c0 0x7f03581e8650 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.37:3300/0 says I am v2:172.21.15.37:59654/0 (socket says 172.21.15.37:59654) 2024-04-25T10:30:22.312 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:22.307+0000 7f035daba640 1 -- 172.21.15.37:0/4082044491 learned_addr learned my addr 172.21.15.37:0/4082044491 (peer_addr_for_me v2:172.21.15.37:0/0) 2024-04-25T10:30:22.312 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:22.308+0000 7f035eabc640 1 --2- 172.21.15.37:0/4082044491 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f03580cce50 0x7f03580cd260 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:30:22.312 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:22.308+0000 7f035e2bb640 1 --2- 172.21.15.37:0/4082044491 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f03440039b0 0x7f03581dcdf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:30:22.312 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:22.308+0000 7f035daba640 1 -- 172.21.15.37:0/4082044491 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f03580cce50 msgr2=0x7f03580cd260 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:30:22.312 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:22.308+0000 7f035daba640 1 --2- 172.21.15.37:0/4082044491 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f03580cce50 0x7f03580cd260 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:30:22.312 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:22.308+0000 7f035daba640 1 -- 172.21.15.37:0/4082044491 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f03440039b0 msgr2=0x7f03581dcdf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:30:22.313 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:22.308+0000 7f035daba640 1 --2- 172.21.15.37:0/4082044491 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f03440039b0 0x7f03581dcdf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:30:22.313 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:22.308+0000 7f035daba640 1 -- 172.21.15.37:0/4082044491 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f03581e8b90 con 0x7f03580cb5c0 2024-04-25T10:30:22.313 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:22.308+0000 7f035daba640 1 --2- 172.21.15.37:0/4082044491 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f03580cb5c0 0x7f03581e8650 secure :-1 s=READY pgs=468 cs=0 l=1 rev1=1 crypto rx=0x7f0348011940 tx=0x7f0348011e00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:30:22.313 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:22.308+0000 7f034effd640 1 -- 172.21.15.37:0/4082044491 <== mon.0 v2:172.21.15.37:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f034800faf0 con 0x7f03580cb5c0 2024-04-25T10:30:22.313 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:22.308+0000 7f034effd640 1 -- 172.21.15.37:0/4082044491 <== mon.0 v2:172.21.15.37:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f034800fc90 con 0x7f03580cb5c0 2024-04-25T10:30:22.313 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:22.308+0000 7f036051f640 1 -- 172.21.15.37:0/4082044491 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f03580cd7a0 con 0x7f03580cb5c0 2024-04-25T10:30:22.313 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:22.308+0000 7f034effd640 1 -- 172.21.15.37:0/4082044491 <== mon.0 v2:172.21.15.37:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0348014440 con 0x7f03580cb5c0 2024-04-25T10:30:22.313 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:22.308+0000 7f036051f640 1 -- 172.21.15.37:0/4082044491 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f03581e0530 con 0x7f03580cb5c0 2024-04-25T10:30:22.313 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:22.309+0000 7f036051f640 1 -- 172.21.15.37:0/4082044491 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f035806a140 con 0x7f03580cb5c0 2024-04-25T10:30:22.314 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:22.310+0000 7f034effd640 1 -- 172.21.15.37:0/4082044491 <== mon.0 v2:172.21.15.37:3300/0 4 ==== mgrmap(e 5) ==== 96782+0+0 (secure 0 0 0) 0x7f0348012040 con 0x7f03580cb5c0 2024-04-25T10:30:22.314 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:22.310+0000 7f034effd640 1 --2- 172.21.15.37:0/4082044491 >> [v2:172.21.15.203:6832/2418205237,v1:172.21.15.203:6833/2418205237] conn(0x7f033c0744e0 0x7f033c0769f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:30:22.314 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:22.311+0000 7f034effd640 1 -- 172.21.15.37:0/4082044491 <== mon.0 v2:172.21.15.37:3300/0 5 ==== osd_map(36..36 src has 1..36) ==== 5709+0+0 (secure 0 0 0) 0x7f03480957c0 con 0x7f03580cb5c0 2024-04-25T10:30:22.314 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:22.311+0000 7f035e2bb640 1 --2- 172.21.15.37:0/4082044491 >> [v2:172.21.15.203:6832/2418205237,v1:172.21.15.203:6833/2418205237] conn(0x7f033c0744e0 0x7f033c0769f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:30:22.314 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:22.311+0000 7f035e2bb640 1 --2- 172.21.15.37:0/4082044491 >> [v2:172.21.15.203:6832/2418205237,v1:172.21.15.203:6833/2418205237] conn(0x7f033c0744e0 0x7f033c0769f0 secure :-1 s=READY pgs=229 cs=0 l=1 rev1=1 crypto rx=0x7f035400c210 tx=0x7f035402e000 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:30:22.317 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:22.315+0000 7f034effd640 1 -- 172.21.15.37:0/4082044491 <== mon.0 v2:172.21.15.37:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187246 (secure 0 0 0) 0x7f03580cd7a0 con 0x7f03580cb5c0 2024-04-25T10:30:22.502 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:22.500+0000 7f036051f640 1 -- 172.21.15.37:0/4082044491 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7f03580cf200 con 0x7f03580cb5c0 2024-04-25T10:30:22.503 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:22.501+0000 7f034effd640 1 -- 172.21.15.37:0/4082044491 <== mon.0 v2:172.21.15.37:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v36) ==== 74+0+16621 (secure 0 0 0) 0x7f0348064e20 con 0x7f03580cb5c0 2024-04-25T10:30:22.504 INFO:teuthology.orchestra.run.smithi037.stdout: 2024-04-25T10:30:22.504 INFO:teuthology.orchestra.run.smithi037.stdout:{"epoch":36,"fsid":"f4acda28-be34-492a-b286-994afd36f61d","created":"2024-04-25T10:28:43.985718+0000","modified":"2024-04-25T10:30:13.756895+0000","last_up_change":"2024-04-25T10:28:54.900654+0000","last_in_change":"2024-04-25T10:28:46.216973+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":5,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-04-25T10:28:56.202153+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"19","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":4,"pool_name":"cephfs_metadata","create_time":"2024-04-25T10:30:10.008132+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"35","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.75,"score_stable":1.75,"optimal_score":1,"raw_score_acting":1.75,"raw_score_stable":1.75,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":5,"pool_name":"cephfs_data","create_time":"2024-04-25T10:30:11.084827+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"36","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"a2f79028-b332-4166-9ebb-a7024a37ea22","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":34,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.37:6816","nonce":2746074720},{"type":"v1","addr":"172.21.15.37:6817","nonce":2746074720}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.37:6818","nonce":2746074720},{"type":"v1","addr":"172.21.15.37:6819","nonce":2746074720}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.37:6822","nonce":2746074720},{"type":"v1","addr":"172.21.15.37:6823","nonce":2746074720}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.37:6820","nonce":2746074720},{"type":"v1","addr":"172.21.15.37:6821","nonce":2746074720}]},"public_addr":"172.21.15.37:6817/2746074720","cluster_addr":"172.21.15.37:6819/2746074720","heartbeat_back_addr":"172.21.15.37:6823/2746074720","heartbeat_front_addr":"172.21.15.37:6821/2746074720","state":["exists","up"]},{"osd":1,"uuid":"973159de-c2ea-4083-aecc-69373d1c1132","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":34,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.37:6800","nonce":1690341662},{"type":"v1","addr":"172.21.15.37:6801","nonce":1690341662}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.37:6802","nonce":1690341662},{"type":"v1","addr":"172.21.15.37:6803","nonce":1690341662}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.37:6806","nonce":1690341662},{"type":"v1","addr":"172.21.15.37:6807","nonce":1690341662}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.37:6804","nonce":1690341662},{"type":"v1","addr":"172.21.15.37:6805","nonce":1690341662}]},"public_addr":"172.21.15.37:6801/1690341662","cluster_addr":"172.21.15.37:6803/1690341662","heartbeat_back_addr":"172.21.15.37:6807/1690341662","heartbeat_front_addr":"172.21.15.37:6805/1690341662","state":["exists","up"]},{"osd":2,"uuid":"4ae7a0e9-cb3f-484c-a75f-f3766b721c47","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":34,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.37:6808","nonce":2754669649},{"type":"v1","addr":"172.21.15.37:6809","nonce":2754669649}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.37:6810","nonce":2754669649},{"type":"v1","addr":"172.21.15.37:6811","nonce":2754669649}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.37:6814","nonce":2754669649},{"type":"v1","addr":"172.21.15.37:6815","nonce":2754669649}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.37:6812","nonce":2754669649},{"type":"v1","addr":"172.21.15.37:6813","nonce":2754669649}]},"public_addr":"172.21.15.37:6809/2754669649","cluster_addr":"172.21.15.37:6811/2754669649","heartbeat_back_addr":"172.21.15.37:6815/2754669649","heartbeat_front_addr":"172.21.15.37:6813/2754669649","state":["exists","up"]},{"osd":3,"uuid":"1ba4a746-1608-47a7-ace2-960a5cfae934","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":34,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.37:6824","nonce":1815290618},{"type":"v1","addr":"172.21.15.37:6825","nonce":1815290618}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.37:6826","nonce":1815290618},{"type":"v1","addr":"172.21.15.37:6827","nonce":1815290618}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.37:6830","nonce":1815290618},{"type":"v1","addr":"172.21.15.37:6831","nonce":1815290618}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.37:6828","nonce":1815290618},{"type":"v1","addr":"172.21.15.37:6829","nonce":1815290618}]},"public_addr":"172.21.15.37:6825/1815290618","cluster_addr":"172.21.15.37:6827/1815290618","heartbeat_back_addr":"172.21.15.37:6831/1815290618","heartbeat_front_addr":"172.21.15.37:6829/1815290618","state":["exists","up"]},{"osd":4,"uuid":"d4f59f5d-0413-4b79-87ee-bb212c1fcf62","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":34,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6800","nonce":4051532063},{"type":"v1","addr":"172.21.15.203:6801","nonce":4051532063}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6802","nonce":4051532063},{"type":"v1","addr":"172.21.15.203:6803","nonce":4051532063}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6806","nonce":4051532063},{"type":"v1","addr":"172.21.15.203:6807","nonce":4051532063}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6804","nonce":4051532063},{"type":"v1","addr":"172.21.15.203:6805","nonce":4051532063}]},"public_addr":"172.21.15.203:6801/4051532063","cluster_addr":"172.21.15.203:6803/4051532063","heartbeat_back_addr":"172.21.15.203:6807/4051532063","heartbeat_front_addr":"172.21.15.203:6805/4051532063","state":["exists","up"]},{"osd":5,"uuid":"69710a0e-41d8-4e7f-9a10-17ce540b67be","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":34,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6824","nonce":1203822890},{"type":"v1","addr":"172.21.15.203:6825","nonce":1203822890}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6826","nonce":1203822890},{"type":"v1","addr":"172.21.15.203:6827","nonce":1203822890}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6830","nonce":1203822890},{"type":"v1","addr":"172.21.15.203:6831","nonce":1203822890}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6828","nonce":1203822890},{"type":"v1","addr":"172.21.15.203:6829","nonce":1203822890}]},"public_addr":"172.21.15.203:6825/1203822890","cluster_addr":"172.21.15.203:6827/1203822890","heartbeat_back_addr":"172.21.15.203:6831/1203822890","heartbeat_front_addr":"172.21.15.203:6829/1203822890","state":["exists","up"]},{"osd":6,"uuid":"f29dd047-3b48-473d-a235-6642a6293f20","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":34,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6810","nonce":2309615014},{"type":"v1","addr":"172.21.15.203:6812","nonce":2309615014}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6814","nonce":2309615014},{"type":"v1","addr":"172.21.15.203:6816","nonce":2309615014}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6822","nonce":2309615014},{"type":"v1","addr":"172.21.15.203:6823","nonce":2309615014}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6818","nonce":2309615014},{"type":"v1","addr":"172.21.15.203:6820","nonce":2309615014}]},"public_addr":"172.21.15.203:6812/2309615014","cluster_addr":"172.21.15.203:6816/2309615014","heartbeat_back_addr":"172.21.15.203:6823/2309615014","heartbeat_front_addr":"172.21.15.203:6820/2309615014","state":["exists","up"]},{"osd":7,"uuid":"6e691644-378b-4fdc-8e86-fb5f56e31f29","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":34,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6808","nonce":3688099014},{"type":"v1","addr":"172.21.15.203:6809","nonce":3688099014}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6811","nonce":3688099014},{"type":"v1","addr":"172.21.15.203:6813","nonce":3688099014}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6819","nonce":3688099014},{"type":"v1","addr":"172.21.15.203:6821","nonce":3688099014}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6815","nonce":3688099014},{"type":"v1","addr":"172.21.15.203:6817","nonce":3688099014}]},"public_addr":"172.21.15.203:6809/3688099014","cluster_addr":"172.21.15.203:6813/3688099014","heartbeat_back_addr":"172.21.15.203:6821/3688099014","heartbeat_front_addr":"172.21.15.203:6817/3688099014","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T10:28:51.419141+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T10:28:51.366448+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T10:28:51.348276+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T10:28:51.380519+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T10:28:51.105812+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T10:28:51.384496+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T10:28:51.388755+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T10:28:51.366958+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-04-25T10:30:22.506 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:22.503+0000 7f036051f640 1 -- 172.21.15.37:0/4082044491 >> [v2:172.21.15.203:6832/2418205237,v1:172.21.15.203:6833/2418205237] conn(0x7f033c0744e0 msgr2=0x7f033c0769f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:30:22.506 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:22.503+0000 7f036051f640 1 --2- 172.21.15.37:0/4082044491 >> [v2:172.21.15.203:6832/2418205237,v1:172.21.15.203:6833/2418205237] conn(0x7f033c0744e0 0x7f033c0769f0 secure :-1 s=READY pgs=229 cs=0 l=1 rev1=1 crypto rx=0x7f035400c210 tx=0x7f035402e000 comp rx=0 tx=0).stop 2024-04-25T10:30:22.506 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:22.503+0000 7f036051f640 1 -- 172.21.15.37:0/4082044491 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f03580cb5c0 msgr2=0x7f03581e8650 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:30:22.506 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:22.503+0000 7f036051f640 1 --2- 172.21.15.37:0/4082044491 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f03580cb5c0 0x7f03581e8650 secure :-1 s=READY pgs=468 cs=0 l=1 rev1=1 crypto rx=0x7f0348011940 tx=0x7f0348011e00 comp rx=0 tx=0).stop 2024-04-25T10:30:22.506 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:22.503+0000 7f036051f640 1 -- 172.21.15.37:0/4082044491 shutdown_connections 2024-04-25T10:30:22.506 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:22.503+0000 7f036051f640 1 --2- 172.21.15.37:0/4082044491 >> [v2:172.21.15.203:6832/2418205237,v1:172.21.15.203:6833/2418205237] conn(0x7f033c0744e0 0x7f033c0769f0 unknown :-1 s=CLOSED pgs=229 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:30:22.506 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:22.503+0000 7f036051f640 1 --2- 172.21.15.37:0/4082044491 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f03580cce50 0x7f03580cd260 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:30:22.506 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:22.503+0000 7f036051f640 1 --2- 172.21.15.37:0/4082044491 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f03580cb5c0 0x7f03581e8650 unknown :-1 s=CLOSED pgs=468 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:30:22.506 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:22.503+0000 7f036051f640 1 --2- 172.21.15.37:0/4082044491 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f03440039b0 0x7f03581dcdf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:30:22.506 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:22.503+0000 7f036051f640 1 -- 172.21.15.37:0/4082044491 >> 172.21.15.37:0/4082044491 conn(0x7f03580d03f0 msgr2=0x7f03581e9810 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:30:22.507 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:22.503+0000 7f036051f640 1 -- 172.21.15.37:0/4082044491 shutdown_connections 2024-04-25T10:30:22.507 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:22.503+0000 7f036051f640 1 -- 172.21.15.37:0/4082044491 wait complete. 2024-04-25T10:30:22.517 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2024-04-25T10:28:56.202153+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '19', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 8, 'score_stable': 8, 'optimal_score': 0.25, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 4, 'pool_name': 'cephfs_metadata', 'create_time': '2024-04-25T10:30:10.008132+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '35', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_autoscale_bias': 4, 'pg_num_min': 16, 'recovery_priority': 5}, 'application_metadata': {'cephfs': {'metadata': 'cephfs'}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 1.75, 'score_stable': 1.75, 'optimal_score': 1, 'raw_score_acting': 1.75, 'raw_score_stable': 1.75, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 5, 'pool_name': 'cephfs_data', 'create_time': '2024-04-25T10:30:11.084827+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 64, 'pg_placement_num': 64, 'pg_placement_num_target': 64, 'pg_num_target': 64, 'pg_num_pending': 64, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '36', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.8999999761581421}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 1.5, 'score_stable': 1.5, 'optimal_score': 1, 'raw_score_acting': 1.5, 'raw_score_stable': 1.5, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}] 2024-04-25T10:30:22.518 DEBUG:teuthology.orchestra.run.smithi037:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2024-04-25T10:30:22.632 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:22.630+0000 7f3e36c84640 1 Processor -- start 2024-04-25T10:30:22.632 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:22.630+0000 7f3e36c84640 1 -- start start 2024-04-25T10:30:22.634 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:22.631+0000 7f3e36c84640 1 --2- >> v2:172.21.15.37:3300/0 conn(0x7f3e300d1ff0 0x7f3e300d2400 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:30:22.634 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:22.631+0000 7f3e36c84640 1 -- --> v1:172.21.15.37:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f3e300b8d10 con 0x7f3e300ce750 2024-04-25T10:30:22.634 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:22.631+0000 7f3e36c84640 1 -- --> v1:172.21.15.203:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f3e300b8a30 con 0x7f3e300b3c80 2024-04-25T10:30:22.635 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:22.631+0000 7f3e36c84640 1 -- --> v2:172.21.15.37:3300/0 -- mon_getmap magic: 0 -- 0x7f3e3008ceb0 con 0x7f3e300d1ff0 2024-04-25T10:30:22.635 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:22.631+0000 7f3e2ffff640 1 --2- >> v2:172.21.15.37:3300/0 conn(0x7f3e300d1ff0 0x7f3e300d2400 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:30:22.635 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:22.631+0000 7f3e35221640 1 --1- >> v1:172.21.15.37:6789/0 conn(0x7f3e300ce750 0x7f3e300ceb60 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.37:6789/0 says I am v1:172.21.15.37:55096/0 (socket says 172.21.15.37:55096) 2024-04-25T10:30:22.635 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:22.631+0000 7f3e35221640 1 -- 172.21.15.37:0/1925993968 learned_addr learned my addr 172.21.15.37:0/1925993968 (peer_addr_for_me v1:172.21.15.37:0/0) 2024-04-25T10:30:22.635 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:22.632+0000 7f3e2ffff640 1 -- 172.21.15.37:0/1925993968 >> v1:172.21.15.203:6789/0 conn(0x7f3e300b3c80 legacy=0x7f3e300b4090 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:30:22.635 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:22.632+0000 7f3e2ffff640 1 -- 172.21.15.37:0/1925993968 >> v1:172.21.15.37:6789/0 conn(0x7f3e300ce750 legacy=0x7f3e300ceb60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:30:22.635 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:22.632+0000 7f3e2ffff640 1 -- 172.21.15.37:0/1925993968 --> v2:172.21.15.37:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f3e300b9020 con 0x7f3e300d1ff0 2024-04-25T10:30:22.635 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:22.632+0000 7f3e2f7fe640 1 -- 172.21.15.37:0/1925993968 <== mon.0 v1:172.21.15.37:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 2902829684 0 0) 0x7f3e300b8d10 con 0x7f3e300ce750 2024-04-25T10:30:22.635 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:22.632+0000 7f3e2ffff640 1 --2- 172.21.15.37:0/1925993968 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f3e300d1ff0 0x7f3e300d2400 secure :-1 s=READY pgs=470 cs=0 l=1 rev1=1 crypto rx=0x7f3e1c0097a0 tx=0x7f3e1c02a140 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a943313dcfa16f16 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:30:22.636 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:22.633+0000 7f3e2f7fe640 1 -- 172.21.15.37:0/1925993968 <== mon.0 v2:172.21.15.37:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3e1c037070 con 0x7f3e300d1ff0 2024-04-25T10:30:22.636 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:22.633+0000 7f3e2f7fe640 1 -- 172.21.15.37:0/1925993968 <== mon.0 v2:172.21.15.37:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f3e1c02ae50 con 0x7f3e300d1ff0 2024-04-25T10:30:22.636 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:22.633+0000 7f3e2f7fe640 1 -- 172.21.15.37:0/1925993968 <== mon.0 v2:172.21.15.37:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3e1c032660 con 0x7f3e300d1ff0 2024-04-25T10:30:22.637 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:22.633+0000 7f3e36c84640 1 -- 172.21.15.37:0/1925993968 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f3e300d1ff0 msgr2=0x7f3e300d2400 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:30:22.637 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:22.633+0000 7f3e36c84640 1 --2- 172.21.15.37:0/1925993968 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f3e300d1ff0 0x7f3e300d2400 secure :-1 s=READY pgs=470 cs=0 l=1 rev1=1 crypto rx=0x7f3e1c0097a0 tx=0x7f3e1c02a140 comp rx=0 tx=0).stop 2024-04-25T10:30:22.637 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:22.633+0000 7f3e36c84640 1 -- 172.21.15.37:0/1925993968 shutdown_connections 2024-04-25T10:30:22.637 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:22.633+0000 7f3e36c84640 1 --2- 172.21.15.37:0/1925993968 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f3e300d1ff0 0x7f3e300d2400 unknown :-1 s=CLOSED pgs=470 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:30:22.637 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:22.633+0000 7f3e36c84640 1 -- 172.21.15.37:0/1925993968 >> 172.21.15.37:0/1925993968 conn(0x7f3e300d0390 msgr2=0x7f3e300d07a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:30:22.637 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:22.633+0000 7f3e36c84640 1 -- 172.21.15.37:0/1925993968 shutdown_connections 2024-04-25T10:30:22.637 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:22.633+0000 7f3e36c84640 1 -- 172.21.15.37:0/1925993968 wait complete. 2024-04-25T10:30:22.637 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:22.635+0000 7f3e36c84640 1 Processor -- start 2024-04-25T10:30:22.638 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:22.635+0000 7f3e36c84640 1 -- start start 2024-04-25T10:30:22.640 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:22.637+0000 7f3e36c84640 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f3e300b3c80 0x7f3e301e4550 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:30:22.641 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:22.637+0000 7f3e36c84640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f3e300ce750 0x7f3e301e8d60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:30:22.641 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:22.637+0000 7f3e36c84640 1 --2- >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f3e300d1ff0 0x7f3e300b1550 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:30:22.641 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:22.637+0000 7f3e36c84640 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 -- 0x7f3e3009ce90 con 0x7f3e300ce750 2024-04-25T10:30:22.641 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:22.637+0000 7f3e36c84640 1 -- --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_getmap magic: 0 -- 0x7f3e300b9300 con 0x7f3e300b3c80 2024-04-25T10:30:22.641 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:22.637+0000 7f3e36c84640 1 -- --> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] -- mon_getmap magic: 0 -- 0x7f3e300d18c0 con 0x7f3e300d1ff0 2024-04-25T10:30:22.641 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:22.637+0000 7f3e2ffff640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f3e300ce750 0x7f3e301e8d60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:30:22.641 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:22.637+0000 7f3e2ffff640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f3e300ce750 0x7f3e301e8d60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.37:3300/0 says I am v2:172.21.15.37:59662/0 (socket says 172.21.15.37:59662) 2024-04-25T10:30:22.641 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:22.637+0000 7f3e2ffff640 1 -- 172.21.15.37:0/863826772 learned_addr learned my addr 172.21.15.37:0/863826772 (peer_addr_for_me v2:172.21.15.37:0/0) 2024-04-25T10:30:22.641 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:22.637+0000 7f3e34a20640 1 --2- 172.21.15.37:0/863826772 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f3e300b3c80 0x7f3e301e4550 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:30:22.641 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:22.637+0000 7f3e35221640 1 --2- 172.21.15.37:0/863826772 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f3e300d1ff0 0x7f3e300b1550 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:30:22.641 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:22.637+0000 7f3e2ffff640 1 -- 172.21.15.37:0/863826772 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f3e300d1ff0 msgr2=0x7f3e300b1550 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:30:22.641 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:22.637+0000 7f3e2ffff640 1 --2- 172.21.15.37:0/863826772 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f3e300d1ff0 0x7f3e300b1550 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:30:22.641 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:22.637+0000 7f3e2ffff640 1 -- 172.21.15.37:0/863826772 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f3e300b3c80 msgr2=0x7f3e301e4550 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:30:22.641 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:22.637+0000 7f3e2ffff640 1 --2- 172.21.15.37:0/863826772 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f3e300b3c80 0x7f3e301e4550 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:30:22.642 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:22.637+0000 7f3e2ffff640 1 -- 172.21.15.37:0/863826772 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f3e301e4e10 con 0x7f3e300ce750 2024-04-25T10:30:22.642 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:22.637+0000 7f3e34a20640 1 --2- 172.21.15.37:0/863826772 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f3e300b3c80 0x7f3e301e4550 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T10:30:22.642 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:22.637+0000 7f3e2ffff640 1 --2- 172.21.15.37:0/863826772 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f3e300ce750 0x7f3e301e8d60 secure :-1 s=READY pgs=471 cs=0 l=1 rev1=1 crypto rx=0x7f3e1c0098c0 tx=0x7f3e1c0046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:30:22.642 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:22.638+0000 7f3e2d7fa640 1 -- 172.21.15.37:0/863826772 <== mon.0 v2:172.21.15.37:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3e1c042070 con 0x7f3e300ce750 2024-04-25T10:30:22.642 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:22.638+0000 7f3e2d7fa640 1 -- 172.21.15.37:0/863826772 <== mon.0 v2:172.21.15.37:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f3e1c03e670 con 0x7f3e300ce750 2024-04-25T10:30:22.642 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:22.638+0000 7f3e2d7fa640 1 -- 172.21.15.37:0/863826772 <== mon.0 v2:172.21.15.37:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3e1c037040 con 0x7f3e300ce750 2024-04-25T10:30:22.642 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:22.638+0000 7f3e36c84640 1 -- 172.21.15.37:0/863826772 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f3e301e50f0 con 0x7f3e300ce750 2024-04-25T10:30:22.642 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:22.638+0000 7f3e36c84640 1 -- 172.21.15.37:0/863826772 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f3e301e8370 con 0x7f3e300ce750 2024-04-25T10:30:22.642 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:22.639+0000 7f3e36c84640 1 -- 172.21.15.37:0/863826772 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f3e301de450 con 0x7f3e300ce750 2024-04-25T10:30:22.643 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:22.639+0000 7f3e2d7fa640 1 -- 172.21.15.37:0/863826772 <== mon.0 v2:172.21.15.37:3300/0 4 ==== mgrmap(e 5) ==== 96782+0+0 (secure 0 0 0) 0x7f3e1c045070 con 0x7f3e300ce750 2024-04-25T10:30:22.643 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:22.640+0000 7f3e2d7fa640 1 --2- 172.21.15.37:0/863826772 >> [v2:172.21.15.203:6832/2418205237,v1:172.21.15.203:6833/2418205237] conn(0x7f3e140744b0 0x7f3e140769c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:30:22.643 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:22.640+0000 7f3e2d7fa640 1 -- 172.21.15.37:0/863826772 <== mon.0 v2:172.21.15.37:3300/0 5 ==== osd_map(36..36 src has 1..36) ==== 5709+0+0 (secure 0 0 0) 0x7f3e1c0b5af0 con 0x7f3e300ce750 2024-04-25T10:30:22.643 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:22.640+0000 7f3e34a20640 1 --2- 172.21.15.37:0/863826772 >> [v2:172.21.15.203:6832/2418205237,v1:172.21.15.203:6833/2418205237] conn(0x7f3e140744b0 0x7f3e140769c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:30:22.643 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:22.641+0000 7f3e34a20640 1 --2- 172.21.15.37:0/863826772 >> [v2:172.21.15.203:6832/2418205237,v1:172.21.15.203:6833/2418205237] conn(0x7f3e140744b0 0x7f3e140769c0 secure :-1 s=READY pgs=230 cs=0 l=1 rev1=1 crypto rx=0x7f3e24006fd0 tx=0x7f3e24008040 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:30:22.646 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:22.644+0000 7f3e2d7fa640 1 -- 172.21.15.37:0/863826772 <== mon.0 v2:172.21.15.37:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187246 (secure 0 0 0) 0x7f3e1c080290 con 0x7f3e300ce750 2024-04-25T10:30:22.833 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:22.831+0000 7f3e36c84640 1 -- 172.21.15.37:0/863826772 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) -- 0x7f3e3006a140 con 0x7f3e300ce750 2024-04-25T10:30:22.834 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:22.831+0000 7f3e2d7fa640 1 -- 172.21.15.37:0/863826772 <== mon.0 v2:172.21.15.37:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v36) ==== 93+0+10 (secure 0 0 0) 0x7f3e1c07fb90 con 0x7f3e300ce750 2024-04-25T10:30:22.834 INFO:teuthology.orchestra.run.smithi037.stdout:pg_num: 1 2024-04-25T10:30:22.836 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:22.833+0000 7f3e36c84640 1 -- 172.21.15.37:0/863826772 >> [v2:172.21.15.203:6832/2418205237,v1:172.21.15.203:6833/2418205237] conn(0x7f3e140744b0 msgr2=0x7f3e140769c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:30:22.836 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:22.833+0000 7f3e36c84640 1 --2- 172.21.15.37:0/863826772 >> [v2:172.21.15.203:6832/2418205237,v1:172.21.15.203:6833/2418205237] conn(0x7f3e140744b0 0x7f3e140769c0 secure :-1 s=READY pgs=230 cs=0 l=1 rev1=1 crypto rx=0x7f3e24006fd0 tx=0x7f3e24008040 comp rx=0 tx=0).stop 2024-04-25T10:30:22.836 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:22.833+0000 7f3e36c84640 1 -- 172.21.15.37:0/863826772 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f3e300ce750 msgr2=0x7f3e301e8d60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:30:22.836 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:22.833+0000 7f3e36c84640 1 --2- 172.21.15.37:0/863826772 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f3e300ce750 0x7f3e301e8d60 secure :-1 s=READY pgs=471 cs=0 l=1 rev1=1 crypto rx=0x7f3e1c0098c0 tx=0x7f3e1c0046c0 comp rx=0 tx=0).stop 2024-04-25T10:30:22.836 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:22.833+0000 7f3e36c84640 1 -- 172.21.15.37:0/863826772 shutdown_connections 2024-04-25T10:30:22.836 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:22.833+0000 7f3e36c84640 1 --2- 172.21.15.37:0/863826772 >> [v2:172.21.15.203:6832/2418205237,v1:172.21.15.203:6833/2418205237] conn(0x7f3e140744b0 0x7f3e140769c0 unknown :-1 s=CLOSED pgs=230 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:30:22.836 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:22.833+0000 7f3e36c84640 1 --2- 172.21.15.37:0/863826772 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f3e300d1ff0 0x7f3e300b1550 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:30:22.836 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:22.833+0000 7f3e36c84640 1 --2- 172.21.15.37:0/863826772 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f3e300ce750 0x7f3e301e8d60 unknown :-1 s=CLOSED pgs=471 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:30:22.836 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:22.833+0000 7f3e36c84640 1 --2- 172.21.15.37:0/863826772 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f3e300b3c80 0x7f3e301e4550 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:30:22.836 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:22.833+0000 7f3e36c84640 1 -- 172.21.15.37:0/863826772 >> 172.21.15.37:0/863826772 conn(0x7f3e300d0390 msgr2=0x7f3e300712f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:30:22.837 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:22.834+0000 7f3e36c84640 1 -- 172.21.15.37:0/863826772 shutdown_connections 2024-04-25T10:30:22.837 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:22.834+0000 7f3e36c84640 1 -- 172.21.15.37:0/863826772 wait complete. 2024-04-25T10:30:22.847 DEBUG:teuthology.orchestra.run.smithi037:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_metadata pg_num 2024-04-25T10:30:22.961 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:22.959+0000 7f80064fa640 1 Processor -- start 2024-04-25T10:30:22.961 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:22.959+0000 7f80064fa640 1 -- start start 2024-04-25T10:30:22.965 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:22.960+0000 7f80064fa640 1 --2- >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f80000d1ec0 0x7f80000d22d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:30:22.965 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:22.960+0000 7f80064fa640 1 -- --> v1:172.21.15.37:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f800009ce70 con 0x7f80000d2810 2024-04-25T10:30:22.965 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:22.960+0000 7f80064fa640 1 -- --> v1:172.21.15.203:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f800009d150 con 0x7f800006e260 2024-04-25T10:30:22.965 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:22.960+0000 7f80064fa640 1 -- --> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] -- mon_getmap magic: 0 -- 0x7f80000d3370 con 0x7f80000d1ec0 2024-04-25T10:30:22.965 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:22.960+0000 7f8004a97640 1 --1- >> v1:172.21.15.37:6789/0 conn(0x7f80000d2810 0x7f80000b3c80 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.37:6789/0 says I am v1:172.21.15.37:55112/0 (socket says 172.21.15.37:55112) 2024-04-25T10:30:22.965 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:22.960+0000 7f8004a97640 1 -- 172.21.15.37:0/644213976 learned_addr learned my addr 172.21.15.37:0/644213976 (peer_addr_for_me v1:172.21.15.37:0/0) 2024-04-25T10:30:22.966 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:22.960+0000 7f7ffeffd640 1 -- 172.21.15.37:0/644213976 <== mon.0 v1:172.21.15.37:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 1079655758 0 0) 0x7f800009ce70 con 0x7f80000d2810 2024-04-25T10:30:22.966 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:22.960+0000 7f7fff7fe640 1 --2- 172.21.15.37:0/644213976 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f80000d1ec0 0x7f80000d22d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:30:22.966 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:22.960+0000 7f7ffeffd640 1 -- 172.21.15.37:0/644213976 --> v1:172.21.15.37:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f7fe8003520 con 0x7f80000d2810 2024-04-25T10:30:22.966 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:22.961+0000 7f7ffeffd640 1 -- 172.21.15.37:0/644213976 <== mon.0 v1:172.21.15.37:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 2088458815 0 0) 0x7f7fe8003520 con 0x7f80000d2810 2024-04-25T10:30:22.966 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:22.961+0000 7f7ffeffd640 1 -- 172.21.15.37:0/644213976 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f80000d1ec0 msgr2=0x7f80000d22d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:30:22.966 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:22.961+0000 7f7ffeffd640 1 --2- 172.21.15.37:0/644213976 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f80000d1ec0 0x7f80000d22d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:30:22.966 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:22.961+0000 7f7ffeffd640 1 -- 172.21.15.37:0/644213976 >> v1:172.21.15.203:6789/0 conn(0x7f800006e260 legacy=0x7f800006e670 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:30:22.966 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:22.961+0000 7f7ffeffd640 1 -- 172.21.15.37:0/644213976 --> v1:172.21.15.37:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f800006ed80 con 0x7f80000d2810 2024-04-25T10:30:22.966 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:22.961+0000 7f7ffeffd640 1 -- 172.21.15.37:0/644213976 <== mon.0 v1:172.21.15.37:6789/0 3 ==== mon_map magic: 0 ==== 413+0+0 (unknown 1577794600 0 0) 0x7f7ff80019f0 con 0x7f80000d2810 2024-04-25T10:30:22.966 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:22.961+0000 7f7ffeffd640 1 -- 172.21.15.37:0/644213976 >> v1:172.21.15.37:6789/0 conn(0x7f80000d2810 legacy=0x7f80000b3c80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:30:22.966 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:22.961+0000 7f7ffeffd640 1 --2- 172.21.15.37:0/644213976 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f7fe80039e0 0x7f7fe8003e50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:30:22.967 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:22.961+0000 7f7ffeffd640 1 --2- 172.21.15.37:0/644213976 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f80000d1ec0 0x7f7fe8004800 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:30:22.967 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:22.961+0000 7f7ffeffd640 1 --2- 172.21.15.37:0/644213976 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f7fe8004d40 0x7f7fe80051b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:30:22.967 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:22.961+0000 7f7ffeffd640 1 -- 172.21.15.37:0/644213976 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 -- 0x7f7fe800bbc0 con 0x7f7fe8004d40 2024-04-25T10:30:22.967 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:22.961+0000 7f7ffeffd640 1 -- 172.21.15.37:0/644213976 --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_getmap magic: 0 -- 0x7f7fe800bd70 con 0x7f7fe80039e0 2024-04-25T10:30:22.967 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:22.961+0000 7f7ffeffd640 1 -- 172.21.15.37:0/644213976 --> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] -- mon_getmap magic: 0 -- 0x7f7fe800bf70 con 0x7f80000d1ec0 2024-04-25T10:30:22.967 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:22.961+0000 7f8004a97640 1 --2- 172.21.15.37:0/644213976 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f7fe8004d40 0x7f7fe80051b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:30:22.967 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:22.961+0000 7f7ffffff640 1 --2- 172.21.15.37:0/644213976 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f7fe80039e0 0x7f7fe8003e50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:30:22.967 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:22.961+0000 7f7fff7fe640 1 --2- 172.21.15.37:0/644213976 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f80000d1ec0 0x7f7fe8004800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:30:22.967 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:22.961+0000 7f8004a97640 1 -- 172.21.15.37:0/644213976 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f80000d1ec0 msgr2=0x7f7fe8004800 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:30:22.967 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:22.961+0000 7f8004a97640 1 --2- 172.21.15.37:0/644213976 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f80000d1ec0 0x7f7fe8004800 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:30:22.967 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:22.961+0000 7f8004a97640 1 -- 172.21.15.37:0/644213976 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f7fe80039e0 msgr2=0x7f7fe8003e50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:30:22.967 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:22.961+0000 7f8004a97640 1 --2- 172.21.15.37:0/644213976 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f7fe80039e0 0x7f7fe8003e50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:30:22.967 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:22.961+0000 7f8004a97640 1 -- 172.21.15.37:0/644213976 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f7fe8003520 con 0x7f7fe8004d40 2024-04-25T10:30:22.967 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:22.961+0000 7f7fff7fe640 1 --2- 172.21.15.37:0/644213976 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f80000d1ec0 0x7f7fe8004800 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T10:30:22.968 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:22.962+0000 7f8004a97640 1 --2- 172.21.15.37:0/644213976 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f7fe8004d40 0x7f7fe80051b0 secure :-1 s=READY pgs=473 cs=0 l=1 rev1=1 crypto rx=0x7f7ff800a780 tx=0x7f7fe8005830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=aa097b9500fdb3ad server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:30:22.968 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:22.962+0000 7f7ffeffd640 1 -- 172.21.15.37:0/644213976 <== mon.0 v2:172.21.15.37:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7ff800a3d0 con 0x7f7fe8004d40 2024-04-25T10:30:22.968 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:22.962+0000 7f7ffeffd640 1 -- 172.21.15.37:0/644213976 <== mon.0 v2:172.21.15.37:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f7ff802f070 con 0x7f7fe8004d40 2024-04-25T10:30:22.968 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:22.962+0000 7f7ffeffd640 1 -- 172.21.15.37:0/644213976 <== mon.0 v2:172.21.15.37:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7ff802bc40 con 0x7f7fe8004d40 2024-04-25T10:30:22.968 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:22.963+0000 7f80064fa640 1 -- 172.21.15.37:0/644213976 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f7fe8004d40 msgr2=0x7f7fe80051b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:30:22.968 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:22.963+0000 7f80064fa640 1 --2- 172.21.15.37:0/644213976 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f7fe8004d40 0x7f7fe80051b0 secure :-1 s=READY pgs=473 cs=0 l=1 rev1=1 crypto rx=0x7f7ff800a780 tx=0x7f7fe8005830 comp rx=0 tx=0).stop 2024-04-25T10:30:22.968 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:22.963+0000 7f8004a97640 1 -- 172.21.15.37:0/644213976 reap_dead start 2024-04-25T10:30:22.968 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:22.963+0000 7f80064fa640 1 -- 172.21.15.37:0/644213976 shutdown_connections 2024-04-25T10:30:22.968 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:22.963+0000 7f80064fa640 1 -- 172.21.15.37:0/644213976 >> 172.21.15.37:0/644213976 conn(0x7f80000d0390 msgr2=0x7f80000d07a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:30:22.968 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:22.963+0000 7f80064fa640 1 -- 172.21.15.37:0/644213976 shutdown_connections 2024-04-25T10:30:22.968 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:22.963+0000 7f80064fa640 1 -- 172.21.15.37:0/644213976 wait complete. 2024-04-25T10:30:22.971 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:22.965+0000 7f80064fa640 1 Processor -- start 2024-04-25T10:30:22.971 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:22.965+0000 7f80064fa640 1 -- start start 2024-04-25T10:30:22.971 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:22.966+0000 7f80064fa640 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f800009ebf0 0x7f80001d55b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:30:22.972 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:22.966+0000 7f80064fa640 1 --2- >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f800009e020 0x7f800009e430 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:30:22.972 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:22.966+0000 7f80064fa640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f80001eb380 0x7f80001eb790 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:30:22.972 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:22.966+0000 7f80064fa640 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 -- 0x7f80000d3090 con 0x7f80001eb380 2024-04-25T10:30:22.972 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:22.966+0000 7f80064fa640 1 -- --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_getmap magic: 0 -- 0x7f800006ef70 con 0x7f800009ebf0 2024-04-25T10:30:22.972 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:22.966+0000 7f80064fa640 1 -- --> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] -- mon_getmap magic: 0 -- 0x7f800006d830 con 0x7f800009e020 2024-04-25T10:30:22.972 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:22.966+0000 7f8004a97640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f80001eb380 0x7f80001eb790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:30:22.972 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:22.966+0000 7f8004a97640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f80001eb380 0x7f80001eb790 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.37:3300/0 says I am v2:172.21.15.37:59682/0 (socket says 172.21.15.37:59682) 2024-04-25T10:30:22.972 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:22.966+0000 7f8004a97640 1 -- 172.21.15.37:0/257691158 learned_addr learned my addr 172.21.15.37:0/257691158 (peer_addr_for_me v2:172.21.15.37:0/0) 2024-04-25T10:30:22.972 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:22.966+0000 7f7fff7fe640 1 --2- 172.21.15.37:0/257691158 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f800009e020 0x7f800009e430 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:30:22.972 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:22.966+0000 7f7ffffff640 1 --2- 172.21.15.37:0/257691158 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f800009ebf0 0x7f80001d55b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:30:22.972 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:22.967+0000 7f8004a97640 1 -- 172.21.15.37:0/257691158 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f800009e020 msgr2=0x7f800009e430 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:30:22.972 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:22.967+0000 7f8004a97640 1 --2- 172.21.15.37:0/257691158 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f800009e020 0x7f800009e430 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:30:22.972 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:22.967+0000 7f8004a97640 1 -- 172.21.15.37:0/257691158 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f800009ebf0 msgr2=0x7f80001d55b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:30:22.972 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:22.967+0000 7f8004a97640 1 --2- 172.21.15.37:0/257691158 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f800009ebf0 0x7f80001d55b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:30:22.972 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:22.967+0000 7f8004a97640 1 -- 172.21.15.37:0/257691158 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f80001ebcd0 con 0x7f80001eb380 2024-04-25T10:30:22.973 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:22.967+0000 7f8004a97640 1 --2- 172.21.15.37:0/257691158 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f80001eb380 0x7f80001eb790 secure :-1 s=READY pgs=474 cs=0 l=1 rev1=1 crypto rx=0x7f7ff800a780 tx=0x7f7ff802d070 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:30:22.973 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:22.967+0000 7f7ffcff9640 1 -- 172.21.15.37:0/257691158 <== mon.0 v2:172.21.15.37:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7ff8045070 con 0x7f80001eb380 2024-04-25T10:30:22.973 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:22.967+0000 7f7ffcff9640 1 -- 172.21.15.37:0/257691158 <== mon.0 v2:172.21.15.37:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f7ff802b9b0 con 0x7f80001eb380 2024-04-25T10:30:22.973 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:22.967+0000 7f80064fa640 1 -- 172.21.15.37:0/257691158 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f80001d6000 con 0x7f80001eb380 2024-04-25T10:30:22.973 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:22.967+0000 7f7ffcff9640 1 -- 172.21.15.37:0/257691158 <== mon.0 v2:172.21.15.37:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7ff802bda0 con 0x7f80001eb380 2024-04-25T10:30:22.973 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:22.967+0000 7f80064fa640 1 -- 172.21.15.37:0/257691158 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f80001d65c0 con 0x7f80001eb380 2024-04-25T10:30:22.973 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:22.968+0000 7f80064fa640 1 -- 172.21.15.37:0/257691158 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f800006a140 con 0x7f80001eb380 2024-04-25T10:30:22.973 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:22.969+0000 7f7ffcff9640 1 -- 172.21.15.37:0/257691158 <== mon.0 v2:172.21.15.37:3300/0 4 ==== mgrmap(e 5) ==== 96782+0+0 (secure 0 0 0) 0x7f7ff8048070 con 0x7f80001eb380 2024-04-25T10:30:22.973 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:22.969+0000 7f7ffcff9640 1 --2- 172.21.15.37:0/257691158 >> [v2:172.21.15.203:6832/2418205237,v1:172.21.15.203:6833/2418205237] conn(0x7f7fec0744b0 0x7f7fec0769c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:30:22.973 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:22.969+0000 7f7ffcff9640 1 -- 172.21.15.37:0/257691158 <== mon.0 v2:172.21.15.37:3300/0 5 ==== osd_map(36..36 src has 1..36) ==== 5709+0+0 (secure 0 0 0) 0x7f7ff80b5110 con 0x7f80001eb380 2024-04-25T10:30:22.973 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:22.969+0000 7f7ffffff640 1 --2- 172.21.15.37:0/257691158 >> [v2:172.21.15.203:6832/2418205237,v1:172.21.15.203:6833/2418205237] conn(0x7f7fec0744b0 0x7f7fec0769c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:30:22.973 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:22.970+0000 7f7ffffff640 1 --2- 172.21.15.37:0/257691158 >> [v2:172.21.15.203:6832/2418205237,v1:172.21.15.203:6833/2418205237] conn(0x7f7fec0744b0 0x7f7fec0769c0 secure :-1 s=READY pgs=231 cs=0 l=1 rev1=1 crypto rx=0x7f80001ec340 tx=0x7f7ff4009040 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:30:22.976 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:22.974+0000 7f7ffcff9640 1 -- 172.21.15.37:0/257691158 <== mon.0 v2:172.21.15.37:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187246 (secure 0 0 0) 0x7f80001ebcd0 con 0x7f80001eb380 2024-04-25T10:30:23.163 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:23.161+0000 7f80064fa640 1 -- 172.21.15.37:0/257691158 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"} v 0) -- 0x7f80000cefc0 con 0x7f80001eb380 2024-04-25T10:30:23.163 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:23.161+0000 7f7ffcff9640 1 -- 172.21.15.37:0/257691158 <== mon.0 v2:172.21.15.37:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]=0 v36) ==== 104+0+11 (secure 0 0 0) 0x7f7ff807f2e0 con 0x7f80001eb380 2024-04-25T10:30:23.164 INFO:teuthology.orchestra.run.smithi037.stdout:pg_num: 32 2024-04-25T10:30:23.166 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:23.163+0000 7f80064fa640 1 -- 172.21.15.37:0/257691158 >> [v2:172.21.15.203:6832/2418205237,v1:172.21.15.203:6833/2418205237] conn(0x7f7fec0744b0 msgr2=0x7f7fec0769c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:30:23.166 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:23.163+0000 7f80064fa640 1 --2- 172.21.15.37:0/257691158 >> [v2:172.21.15.203:6832/2418205237,v1:172.21.15.203:6833/2418205237] conn(0x7f7fec0744b0 0x7f7fec0769c0 secure :-1 s=READY pgs=231 cs=0 l=1 rev1=1 crypto rx=0x7f80001ec340 tx=0x7f7ff4009040 comp rx=0 tx=0).stop 2024-04-25T10:30:23.166 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:23.163+0000 7f80064fa640 1 -- 172.21.15.37:0/257691158 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f80001eb380 msgr2=0x7f80001eb790 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:30:23.166 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:23.163+0000 7f80064fa640 1 --2- 172.21.15.37:0/257691158 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f80001eb380 0x7f80001eb790 secure :-1 s=READY pgs=474 cs=0 l=1 rev1=1 crypto rx=0x7f7ff800a780 tx=0x7f7ff802d070 comp rx=0 tx=0).stop 2024-04-25T10:30:23.166 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:23.163+0000 7f80064fa640 1 -- 172.21.15.37:0/257691158 shutdown_connections 2024-04-25T10:30:23.166 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:23.163+0000 7f80064fa640 1 --2- 172.21.15.37:0/257691158 >> [v2:172.21.15.203:6832/2418205237,v1:172.21.15.203:6833/2418205237] conn(0x7f7fec0744b0 0x7f7fec0769c0 unknown :-1 s=CLOSED pgs=231 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:30:23.166 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:23.163+0000 7f80064fa640 1 --2- 172.21.15.37:0/257691158 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f80001eb380 0x7f80001eb790 unknown :-1 s=CLOSED pgs=474 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:30:23.166 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:23.163+0000 7f80064fa640 1 --2- 172.21.15.37:0/257691158 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f800009e020 0x7f800009e430 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:30:23.166 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:23.163+0000 7f80064fa640 1 --2- 172.21.15.37:0/257691158 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f800009ebf0 0x7f80001d55b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:30:23.167 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:23.163+0000 7f80064fa640 1 -- 172.21.15.37:0/257691158 >> 172.21.15.37:0/257691158 conn(0x7f80000d0390 msgr2=0x7f80000b3ca0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:30:23.167 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:23.163+0000 7f80064fa640 1 -- 172.21.15.37:0/257691158 shutdown_connections 2024-04-25T10:30:23.167 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:23.164+0000 7f80064fa640 1 -- 172.21.15.37:0/257691158 wait complete. 2024-04-25T10:30:23.177 DEBUG:teuthology.orchestra.run.smithi037:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data pg_num 2024-04-25T10:30:23.290 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:23.288+0000 7f84545ce640 1 Processor -- start 2024-04-25T10:30:23.290 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:23.288+0000 7f84545ce640 1 -- start start 2024-04-25T10:30:23.293 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:23.289+0000 7f84545ce640 1 --2- >> v2:172.21.15.203:3300/0 conn(0x7f844c06e260 0x7f844c06e670 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:30:23.293 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:23.290+0000 7f84545ce640 1 --2- >> v2:172.21.15.37:3300/0 conn(0x7f844c06ebb0 0x7f844c1645d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:30:23.293 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:23.290+0000 7f84545ce640 1 --2- >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f844c164b10 0x7f844c164f20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:30:23.293 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:23.290+0000 7f84545ce640 1 -- --> v2:172.21.15.37:3300/0 -- mon_getmap magic: 0 -- 0x7f844c165460 con 0x7f844c06ebb0 2024-04-25T10:30:23.293 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:23.290+0000 7f84545ce640 1 -- --> v2:172.21.15.203:3300/0 -- mon_getmap magic: 0 -- 0x7f844c0d3370 con 0x7f844c06e260 2024-04-25T10:30:23.293 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:23.290+0000 7f84545ce640 1 -- --> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] -- mon_getmap magic: 0 -- 0x7f844c0d3090 con 0x7f844c164b10 2024-04-25T10:30:23.293 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:23.290+0000 7f8451b69640 1 --2- >> v2:172.21.15.37:3300/0 conn(0x7f844c06ebb0 0x7f844c1645d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:30:23.293 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:23.290+0000 7f8451b69640 1 --2- >> v2:172.21.15.37:3300/0 conn(0x7f844c06ebb0 0x7f844c1645d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.37:3300/0 says I am v2:172.21.15.37:59684/0 (socket says 172.21.15.37:59684) 2024-04-25T10:30:23.294 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:23.290+0000 7f8451b69640 1 -- 172.21.15.37:0/3504824698 learned_addr learned my addr 172.21.15.37:0/3504824698 (peer_addr_for_me v2:172.21.15.37:0/0) 2024-04-25T10:30:23.294 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:23.290+0000 7f845236a640 1 --2- 172.21.15.37:0/3504824698 >> v2:172.21.15.203:3300/0 conn(0x7f844c06e260 0x7f844c06e670 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:30:23.294 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:23.290+0000 7f8452b6b640 1 --2- 172.21.15.37:0/3504824698 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f844c164b10 0x7f844c164f20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:30:23.294 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:23.290+0000 7f8451b69640 1 -- 172.21.15.37:0/3504824698 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f844c164b10 msgr2=0x7f844c164f20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:30:23.294 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:23.290+0000 7f8451b69640 1 --2- 172.21.15.37:0/3504824698 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f844c164b10 0x7f844c164f20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:30:23.294 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:23.290+0000 7f8451b69640 1 -- 172.21.15.37:0/3504824698 >> v2:172.21.15.203:3300/0 conn(0x7f844c06e260 msgr2=0x7f844c06e670 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:30:23.294 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:23.290+0000 7f8451b69640 1 --2- 172.21.15.37:0/3504824698 >> v2:172.21.15.203:3300/0 conn(0x7f844c06e260 0x7f844c06e670 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:30:23.294 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:23.290+0000 7f8451b69640 1 -- 172.21.15.37:0/3504824698 --> v2:172.21.15.37:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f844c09ce70 con 0x7f844c06ebb0 2024-04-25T10:30:23.294 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:23.291+0000 7f8451b69640 1 --2- 172.21.15.37:0/3504824698 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f844c06ebb0 0x7f844c1645d0 secure :-1 s=READY pgs=475 cs=0 l=1 rev1=1 crypto rx=0x7f844c074460 tx=0x7f844002a140 comp rx=0 tx=0).ready entity=mon.0 client_cookie=bdcd3fdce5055cc3 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:30:23.294 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:23.291+0000 7f8451368640 1 -- 172.21.15.37:0/3504824698 <== mon.0 v2:172.21.15.37:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8440037070 con 0x7f844c06ebb0 2024-04-25T10:30:23.294 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:23.291+0000 7f8451368640 1 -- 172.21.15.37:0/3504824698 <== mon.0 v2:172.21.15.37:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f8440032440 con 0x7f844c06ebb0 2024-04-25T10:30:23.294 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:23.291+0000 7f8451368640 1 -- 172.21.15.37:0/3504824698 <== mon.0 v2:172.21.15.37:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8440032810 con 0x7f844c06ebb0 2024-04-25T10:30:23.295 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:23.291+0000 7f84545ce640 1 -- 172.21.15.37:0/3504824698 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f844c06ebb0 msgr2=0x7f844c1645d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:30:23.296 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:23.291+0000 7f84545ce640 1 --2- 172.21.15.37:0/3504824698 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f844c06ebb0 0x7f844c1645d0 secure :-1 s=READY pgs=475 cs=0 l=1 rev1=1 crypto rx=0x7f844c074460 tx=0x7f844002a140 comp rx=0 tx=0).stop 2024-04-25T10:30:23.296 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:23.292+0000 7f84545ce640 1 -- 172.21.15.37:0/3504824698 shutdown_connections 2024-04-25T10:30:23.296 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:23.292+0000 7f84545ce640 1 --2- 172.21.15.37:0/3504824698 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f844c06ebb0 0x7f844c1645d0 unknown :-1 s=CLOSED pgs=475 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:30:23.296 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:23.292+0000 7f84545ce640 1 --2- 172.21.15.37:0/3504824698 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f844c164b10 0x7f844c164f20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:30:23.296 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:23.292+0000 7f84545ce640 1 --2- 172.21.15.37:0/3504824698 >> v2:172.21.15.203:3300/0 conn(0x7f844c06e260 0x7f844c06e670 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:30:23.296 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:23.292+0000 7f84545ce640 1 -- 172.21.15.37:0/3504824698 >> 172.21.15.37:0/3504824698 conn(0x7f844c0d0390 msgr2=0x7f844c0d07a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:30:23.296 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:23.292+0000 7f84545ce640 1 -- 172.21.15.37:0/3504824698 shutdown_connections 2024-04-25T10:30:23.296 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:23.292+0000 7f84545ce640 1 -- 172.21.15.37:0/3504824698 wait complete. 2024-04-25T10:30:23.296 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:23.293+0000 7f84545ce640 1 Processor -- start 2024-04-25T10:30:23.296 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:23.293+0000 7f84545ce640 1 -- start start 2024-04-25T10:30:23.298 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:23.295+0000 7f84545ce640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f844c06e260 0x7f844c1610a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:30:23.298 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:23.295+0000 7f84545ce640 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f844c06ebb0 0x7f844c0a0560 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:30:23.298 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:23.295+0000 7f84545ce640 1 --2- >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f844c164b10 0x7f844c1f61e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:30:23.298 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:23.295+0000 7f84545ce640 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 -- 0x7f844c0b9000 con 0x7f844c06e260 2024-04-25T10:30:23.299 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:23.295+0000 7f84545ce640 1 -- --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_getmap magic: 0 -- 0x7f844c06d830 con 0x7f844c06ebb0 2024-04-25T10:30:23.299 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:23.295+0000 7f84545ce640 1 -- --> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] -- mon_getmap magic: 0 -- 0x7f844c0b9180 con 0x7f844c164b10 2024-04-25T10:30:23.299 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:23.295+0000 7f845236a640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f844c06e260 0x7f844c1610a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:30:23.299 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:23.295+0000 7f845236a640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f844c06e260 0x7f844c1610a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.37:3300/0 says I am v2:172.21.15.37:59696/0 (socket says 172.21.15.37:59696) 2024-04-25T10:30:23.299 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:23.295+0000 7f845236a640 1 -- 172.21.15.37:0/2366819390 learned_addr learned my addr 172.21.15.37:0/2366819390 (peer_addr_for_me v2:172.21.15.37:0/0) 2024-04-25T10:30:23.299 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:23.295+0000 7f8451b69640 1 --2- 172.21.15.37:0/2366819390 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f844c06ebb0 0x7f844c0a0560 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:30:23.299 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:23.295+0000 7f8452b6b640 1 --2- 172.21.15.37:0/2366819390 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f844c164b10 0x7f844c1f61e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:30:23.299 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:23.295+0000 7f845236a640 1 -- 172.21.15.37:0/2366819390 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f844c164b10 msgr2=0x7f844c1f61e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:30:23.299 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:23.295+0000 7f845236a640 1 --2- 172.21.15.37:0/2366819390 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f844c164b10 0x7f844c1f61e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:30:23.299 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:23.295+0000 7f845236a640 1 -- 172.21.15.37:0/2366819390 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f844c06ebb0 msgr2=0x7f844c0a0560 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:30:23.299 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:23.295+0000 7f845236a640 1 --2- 172.21.15.37:0/2366819390 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f844c06ebb0 0x7f844c0a0560 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:30:23.299 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:23.295+0000 7f845236a640 1 -- 172.21.15.37:0/2366819390 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f844c1615e0 con 0x7f844c06e260 2024-04-25T10:30:23.300 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:23.295+0000 7f845236a640 1 --2- 172.21.15.37:0/2366819390 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f844c06e260 0x7f844c1610a0 secure :-1 s=READY pgs=476 cs=0 l=1 rev1=1 crypto rx=0x7f844400d600 tx=0x7f844400dac0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:30:23.300 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:23.295+0000 7f844affd640 1 -- 172.21.15.37:0/2366819390 <== mon.0 v2:172.21.15.37:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8444004240 con 0x7f844c06e260 2024-04-25T10:30:23.300 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:23.296+0000 7f844affd640 1 -- 172.21.15.37:0/2366819390 <== mon.0 v2:172.21.15.37:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f84440043e0 con 0x7f844c06e260 2024-04-25T10:30:23.300 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:23.296+0000 7f84545ce640 1 -- 172.21.15.37:0/2366819390 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f844c0a0fe0 con 0x7f844c06e260 2024-04-25T10:30:23.300 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:23.296+0000 7f844affd640 1 -- 172.21.15.37:0/2366819390 <== mon.0 v2:172.21.15.37:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8444003830 con 0x7f844c06e260 2024-04-25T10:30:23.300 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:23.296+0000 7f84545ce640 1 -- 172.21.15.37:0/2366819390 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f844c0b69b0 con 0x7f844c06e260 2024-04-25T10:30:23.301 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:23.297+0000 7f84545ce640 1 -- 172.21.15.37:0/2366819390 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f844c06a140 con 0x7f844c06e260 2024-04-25T10:30:23.301 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:23.297+0000 7f844affd640 1 -- 172.21.15.37:0/2366819390 <== mon.0 v2:172.21.15.37:3300/0 4 ==== mgrmap(e 5) ==== 96782+0+0 (secure 0 0 0) 0x7f84440039d0 con 0x7f844c06e260 2024-04-25T10:30:23.301 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:23.298+0000 7f844affd640 1 --2- 172.21.15.37:0/2366819390 >> [v2:172.21.15.203:6832/2418205237,v1:172.21.15.203:6833/2418205237] conn(0x7f84380744e0 0x7f84380769f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:30:23.301 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:23.298+0000 7f844affd640 1 -- 172.21.15.37:0/2366819390 <== mon.0 v2:172.21.15.37:3300/0 5 ==== osd_map(36..36 src has 1..36) ==== 5709+0+0 (secure 0 0 0) 0x7f84440952e0 con 0x7f844c06e260 2024-04-25T10:30:23.301 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:23.298+0000 7f8451b69640 1 --2- 172.21.15.37:0/2366819390 >> [v2:172.21.15.203:6832/2418205237,v1:172.21.15.203:6833/2418205237] conn(0x7f84380744e0 0x7f84380769f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:30:23.301 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:23.298+0000 7f8451b69640 1 --2- 172.21.15.37:0/2366819390 >> [v2:172.21.15.203:6832/2418205237,v1:172.21.15.203:6833/2418205237] conn(0x7f84380744e0 0x7f84380769f0 secure :-1 s=READY pgs=232 cs=0 l=1 rev1=1 crypto rx=0x7f8440034000 tx=0x7f844002e000 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:30:23.304 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:23.302+0000 7f844affd640 1 -- 172.21.15.37:0/2366819390 <== mon.0 v2:172.21.15.37:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187246 (secure 0 0 0) 0x7f844405f4b0 con 0x7f844c06e260 2024-04-25T10:30:23.487 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:23.485+0000 7f84545ce640 1 -- 172.21.15.37:0/2366819390 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"} v 0) -- 0x7f844c0a0aa0 con 0x7f844c06e260 2024-04-25T10:30:23.487 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:23.485+0000 7f844affd640 1 -- 172.21.15.37:0/2366819390 <== mon.0 v2:172.21.15.37:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]=0 v36) ==== 100+0+11 (secure 0 0 0) 0x7f84440637b0 con 0x7f844c06e260 2024-04-25T10:30:23.487 INFO:teuthology.orchestra.run.smithi037.stdout:pg_num: 64 2024-04-25T10:30:23.489 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:23.487+0000 7f84545ce640 1 -- 172.21.15.37:0/2366819390 >> [v2:172.21.15.203:6832/2418205237,v1:172.21.15.203:6833/2418205237] conn(0x7f84380744e0 msgr2=0x7f84380769f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:30:23.489 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:23.487+0000 7f84545ce640 1 --2- 172.21.15.37:0/2366819390 >> [v2:172.21.15.203:6832/2418205237,v1:172.21.15.203:6833/2418205237] conn(0x7f84380744e0 0x7f84380769f0 secure :-1 s=READY pgs=232 cs=0 l=1 rev1=1 crypto rx=0x7f8440034000 tx=0x7f844002e000 comp rx=0 tx=0).stop 2024-04-25T10:30:23.490 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:23.487+0000 7f84545ce640 1 -- 172.21.15.37:0/2366819390 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f844c06e260 msgr2=0x7f844c1610a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:30:23.490 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:23.487+0000 7f84545ce640 1 --2- 172.21.15.37:0/2366819390 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f844c06e260 0x7f844c1610a0 secure :-1 s=READY pgs=476 cs=0 l=1 rev1=1 crypto rx=0x7f844400d600 tx=0x7f844400dac0 comp rx=0 tx=0).stop 2024-04-25T10:30:23.490 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:23.487+0000 7f84545ce640 1 -- 172.21.15.37:0/2366819390 shutdown_connections 2024-04-25T10:30:23.490 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:23.487+0000 7f84545ce640 1 --2- 172.21.15.37:0/2366819390 >> [v2:172.21.15.203:6832/2418205237,v1:172.21.15.203:6833/2418205237] conn(0x7f84380744e0 0x7f84380769f0 unknown :-1 s=CLOSED pgs=232 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:30:23.490 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:23.487+0000 7f84545ce640 1 --2- 172.21.15.37:0/2366819390 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f844c164b10 0x7f844c1f61e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:30:23.490 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:23.487+0000 7f84545ce640 1 --2- 172.21.15.37:0/2366819390 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f844c06ebb0 0x7f844c0a0560 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:30:23.490 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:23.487+0000 7f84545ce640 1 --2- 172.21.15.37:0/2366819390 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f844c06e260 0x7f844c1610a0 unknown :-1 s=CLOSED pgs=476 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:30:23.490 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:23.487+0000 7f84545ce640 1 -- 172.21.15.37:0/2366819390 >> 172.21.15.37:0/2366819390 conn(0x7f844c0d0390 msgr2=0x7f844c161d80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:30:23.490 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:23.487+0000 7f84545ce640 1 -- 172.21.15.37:0/2366819390 shutdown_connections 2024-04-25T10:30:23.490 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:23.487+0000 7f84545ce640 1 -- 172.21.15.37:0/2366819390 wait complete. 2024-04-25T10:30:23.500 DEBUG:teuthology.orchestra.run.smithi037:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd blocklist ls 2024-04-25T10:30:23.612 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:23.610+0000 7f0cb93e3640 1 Processor -- start 2024-04-25T10:30:23.612 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:23.610+0000 7f0cb93e3640 1 -- start start 2024-04-25T10:30:23.614 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:23.611+0000 7f0cb93e3640 1 --2- >> v2:172.21.15.203:3300/0 conn(0x7f0cb40cf570 0x7f0cb416ad90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:30:23.615 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:23.611+0000 7f0cb93e3640 1 -- --> v1:172.21.15.37:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f0cb406e910 con 0x7f0cb40b2dc0 2024-04-25T10:30:23.615 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:23.611+0000 7f0cb93e3640 1 -- --> v1:172.21.15.203:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f0cb40d2e40 con 0x7f0cb40d2860 2024-04-25T10:30:23.615 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:23.611+0000 7f0cb93e3640 1 -- --> v2:172.21.15.203:3300/0 -- mon_getmap magic: 0 -- 0x7f0cb40b92c0 con 0x7f0cb40cf570 2024-04-25T10:30:23.615 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:23.612+0000 7f0cb2ffd640 1 --1- >> v1:172.21.15.37:6789/0 conn(0x7f0cb40b2dc0 0x7f0cb406daf0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.37:6789/0 says I am v1:172.21.15.37:55120/0 (socket says 172.21.15.37:55120) 2024-04-25T10:30:23.615 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:23.612+0000 7f0cb2ffd640 1 -- 172.21.15.37:0/3726119640 learned_addr learned my addr 172.21.15.37:0/3726119640 (peer_addr_for_me v1:172.21.15.37:0/0) 2024-04-25T10:30:23.615 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:23.612+0000 7f0cb37fe640 1 --2- 172.21.15.37:0/3726119640 >> v2:172.21.15.203:3300/0 conn(0x7f0cb40cf570 0x7f0cb416ad90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:30:23.615 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:23.612+0000 7f0cb1ffb640 1 -- 172.21.15.37:0/3726119640 <== mon.0 v1:172.21.15.37:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 2756142743 0 0) 0x7f0cb406e910 con 0x7f0cb40b2dc0 2024-04-25T10:30:23.615 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:23.612+0000 7f0cb1ffb640 1 -- 172.21.15.37:0/3726119640 --> v1:172.21.15.37:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f0ca0003520 con 0x7f0cb40b2dc0 2024-04-25T10:30:23.615 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:23.612+0000 7f0cb1ffb640 1 -- 172.21.15.37:0/3726119640 <== mon.0 v1:172.21.15.37:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 2508564168 0 0) 0x7f0ca0003520 con 0x7f0cb40b2dc0 2024-04-25T10:30:23.615 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:23.612+0000 7f0cb37fe640 1 -- 172.21.15.37:0/3726119640 >> v1:172.21.15.203:6789/0 conn(0x7f0cb40d2860 legacy=0x7f0cb406e200 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:30:23.615 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:23.612+0000 7f0cb37fe640 1 -- 172.21.15.37:0/3726119640 >> v1:172.21.15.37:6789/0 conn(0x7f0cb40b2dc0 legacy=0x7f0cb406daf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:30:23.615 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:23.612+0000 7f0cb37fe640 1 -- 172.21.15.37:0/3726119640 --> v2:172.21.15.203:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f0cb40d3120 con 0x7f0cb40cf570 2024-04-25T10:30:23.617 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:23.613+0000 7f0cb37fe640 1 --2- 172.21.15.37:0/3726119640 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f0cb40cf570 0x7f0cb416ad90 secure :-1 s=READY pgs=191 cs=0 l=1 rev1=1 crypto rx=0x7f0ca4009d90 tx=0x7f0ca402a5b0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=1f04a2d656ab36f9 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:30:23.617 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:23.613+0000 7f0cb1ffb640 1 -- 172.21.15.37:0/3726119640 <== mon.1 v2:172.21.15.203:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0ca4035020 con 0x7f0cb40cf570 2024-04-25T10:30:23.617 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:23.613+0000 7f0cb1ffb640 1 -- 172.21.15.37:0/3726119640 <== mon.1 v2:172.21.15.203:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f0ca4007540 con 0x7f0cb40cf570 2024-04-25T10:30:23.617 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:23.613+0000 7f0cb1ffb640 1 -- 172.21.15.37:0/3726119640 <== mon.1 v2:172.21.15.203:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0cac0017f0 con 0x7f0cb40cf570 2024-04-25T10:30:23.617 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:23.614+0000 7f0cb93e3640 1 -- 172.21.15.37:0/3726119640 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f0cb40cf570 msgr2=0x7f0cb416ad90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:30:23.617 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:23.614+0000 7f0cb93e3640 1 --2- 172.21.15.37:0/3726119640 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f0cb40cf570 0x7f0cb416ad90 secure :-1 s=READY pgs=191 cs=0 l=1 rev1=1 crypto rx=0x7f0ca4009d90 tx=0x7f0ca402a5b0 comp rx=0 tx=0).stop 2024-04-25T10:30:23.617 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:23.614+0000 7f0cb93e3640 1 -- 172.21.15.37:0/3726119640 shutdown_connections 2024-04-25T10:30:23.617 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:23.614+0000 7f0cb93e3640 1 --2- 172.21.15.37:0/3726119640 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f0cb40cf570 0x7f0cb416ad90 unknown :-1 s=CLOSED pgs=191 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:30:23.617 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:23.614+0000 7f0cb93e3640 1 -- 172.21.15.37:0/3726119640 >> 172.21.15.37:0/3726119640 conn(0x7f0cb40b5440 msgr2=0x7f0cb40b5850 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:30:23.617 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:23.614+0000 7f0cb93e3640 1 -- 172.21.15.37:0/3726119640 shutdown_connections 2024-04-25T10:30:23.617 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:23.614+0000 7f0cb93e3640 1 -- 172.21.15.37:0/3726119640 wait complete. 2024-04-25T10:30:23.618 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:23.616+0000 7f0cb93e3640 1 Processor -- start 2024-04-25T10:30:23.618 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:23.616+0000 7f0cb93e3640 1 -- start start 2024-04-25T10:30:23.620 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:23.617+0000 7f0cb93e3640 1 --2- >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f0cb40b2dc0 0x7f0cb415c000 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:30:23.621 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:23.617+0000 7f0cb93e3640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f0cb40cf570 0x7f0cb40cbe50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:30:23.621 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:23.617+0000 7f0cb93e3640 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f0cb40d2860 0x7f0cb40b1870 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:30:23.621 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:23.617+0000 7f0cb93e3640 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 -- 0x7f0cb40d36e0 con 0x7f0cb40cf570 2024-04-25T10:30:23.621 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:23.617+0000 7f0cb93e3640 1 -- --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_getmap magic: 0 -- 0x7f0cb40ce0d0 con 0x7f0cb40d2860 2024-04-25T10:30:23.621 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:23.617+0000 7f0cb93e3640 1 -- --> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] -- mon_getmap magic: 0 -- 0x7f0cb416b2d0 con 0x7f0cb40b2dc0 2024-04-25T10:30:23.621 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:23.617+0000 7f0cb27fc640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f0cb40cf570 0x7f0cb40cbe50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:30:23.621 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:23.617+0000 7f0cb27fc640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f0cb40cf570 0x7f0cb40cbe50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.37:3300/0 says I am v2:172.21.15.37:59706/0 (socket says 172.21.15.37:59706) 2024-04-25T10:30:23.621 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:23.617+0000 7f0cb27fc640 1 -- 172.21.15.37:0/2307683022 learned_addr learned my addr 172.21.15.37:0/2307683022 (peer_addr_for_me v2:172.21.15.37:0/0) 2024-04-25T10:30:23.621 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:23.618+0000 7f0cb2ffd640 1 --2- 172.21.15.37:0/2307683022 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f0cb40b2dc0 0x7f0cb415c000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:30:23.621 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:23.618+0000 7f0cb37fe640 1 --2- 172.21.15.37:0/2307683022 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f0cb40d2860 0x7f0cb40b1870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:30:23.621 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:23.618+0000 7f0cb27fc640 1 -- 172.21.15.37:0/2307683022 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f0cb40b2dc0 msgr2=0x7f0cb415c000 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:30:23.622 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:23.618+0000 7f0cb27fc640 1 --2- 172.21.15.37:0/2307683022 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f0cb40b2dc0 0x7f0cb415c000 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:30:23.622 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:23.618+0000 7f0cb27fc640 1 -- 172.21.15.37:0/2307683022 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f0cb40d2860 msgr2=0x7f0cb40b1870 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:30:23.622 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:23.618+0000 7f0cb27fc640 1 --2- 172.21.15.37:0/2307683022 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f0cb40d2860 0x7f0cb40b1870 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:30:23.622 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:23.618+0000 7f0cb27fc640 1 -- 172.21.15.37:0/2307683022 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f0cb415cba0 con 0x7f0cb40cf570 2024-04-25T10:30:23.622 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:23.618+0000 7f0cb27fc640 1 --2- 172.21.15.37:0/2307683022 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f0cb40cf570 0x7f0cb40cbe50 secure :-1 s=READY pgs=478 cs=0 l=1 rev1=1 crypto rx=0x7f0c9c00c4a0 tx=0x7f0c9c00c960 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:30:23.622 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:23.618+0000 7f0cab7fe640 1 -- 172.21.15.37:0/2307683022 <== mon.0 v2:172.21.15.37:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0c9c017070 con 0x7f0cb40cf570 2024-04-25T10:30:23.622 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:23.618+0000 7f0cab7fe640 1 -- 172.21.15.37:0/2307683022 <== mon.0 v2:172.21.15.37:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f0c9c005480 con 0x7f0cb40cf570 2024-04-25T10:30:23.622 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:23.618+0000 7f0cab7fe640 1 -- 172.21.15.37:0/2307683022 <== mon.0 v2:172.21.15.37:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0c9c003810 con 0x7f0cb40cf570 2024-04-25T10:30:23.622 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:23.618+0000 7f0cb93e3640 1 -- 172.21.15.37:0/2307683022 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f0cb415ce80 con 0x7f0cb40cf570 2024-04-25T10:30:23.622 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:23.618+0000 7f0cb93e3640 1 -- 172.21.15.37:0/2307683022 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f0cb415d440 con 0x7f0cb40cf570 2024-04-25T10:30:23.623 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:23.619+0000 7f0cb93e3640 1 -- 172.21.15.37:0/2307683022 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f0cb406a140 con 0x7f0cb40cf570 2024-04-25T10:30:23.623 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:23.620+0000 7f0cab7fe640 1 -- 172.21.15.37:0/2307683022 <== mon.0 v2:172.21.15.37:3300/0 4 ==== mgrmap(e 5) ==== 96782+0+0 (secure 0 0 0) 0x7f0c9c01a070 con 0x7f0cb40cf570 2024-04-25T10:30:23.623 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:23.620+0000 7f0cab7fe640 1 --2- 172.21.15.37:0/2307683022 >> [v2:172.21.15.203:6832/2418205237,v1:172.21.15.203:6833/2418205237] conn(0x7f0c940744e0 0x7f0c940769f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:30:23.623 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:23.620+0000 7f0cab7fe640 1 -- 172.21.15.37:0/2307683022 <== mon.0 v2:172.21.15.37:3300/0 5 ==== osd_map(36..36 src has 1..36) ==== 5709+0+0 (secure 0 0 0) 0x7f0c9c094590 con 0x7f0cb40cf570 2024-04-25T10:30:23.623 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:23.621+0000 7f0cb2ffd640 1 --2- 172.21.15.37:0/2307683022 >> [v2:172.21.15.203:6832/2418205237,v1:172.21.15.203:6833/2418205237] conn(0x7f0c940744e0 0x7f0c940769f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:30:23.623 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:23.621+0000 7f0cb2ffd640 1 --2- 172.21.15.37:0/2307683022 >> [v2:172.21.15.203:6832/2418205237,v1:172.21.15.203:6833/2418205237] conn(0x7f0c940744e0 0x7f0c940769f0 secure :-1 s=READY pgs=233 cs=0 l=1 rev1=1 crypto rx=0x7f0cac001360 tx=0x7f0cac001e00 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:30:23.627 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:23.625+0000 7f0cab7fe640 1 -- 172.21.15.37:0/2307683022 <== mon.0 v2:172.21.15.37:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187246 (secure 0 0 0) 0x7f0c9c05e6d0 con 0x7f0cb40cf570 2024-04-25T10:30:23.823 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:23.821+0000 7f0cb93e3640 1 -- 172.21.15.37:0/2307683022 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_command({"prefix": "osd blocklist ls"} v 0) -- 0x7f0cb4076cb0 con 0x7f0cb40cf570 2024-04-25T10:30:23.823 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:23.821+0000 7f0cab7fe640 1 -- 172.21.15.37:0/2307683022 <== mon.0 v2:172.21.15.37:3300/0 7 ==== mon_command_ack([{"prefix": "osd blocklist ls"}]=0 listed 0 entries v36) ==== 80+0+0 (secure 0 0 0) 0x7f0c9c0629d0 con 0x7f0cb40cf570 2024-04-25T10:30:23.823 INFO:teuthology.orchestra.run.smithi037.stderr:listed 0 entries 2024-04-25T10:30:23.825 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:23.823+0000 7f0cb93e3640 1 -- 172.21.15.37:0/2307683022 >> [v2:172.21.15.203:6832/2418205237,v1:172.21.15.203:6833/2418205237] conn(0x7f0c940744e0 msgr2=0x7f0c940769f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:30:23.825 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:23.823+0000 7f0cb93e3640 1 --2- 172.21.15.37:0/2307683022 >> [v2:172.21.15.203:6832/2418205237,v1:172.21.15.203:6833/2418205237] conn(0x7f0c940744e0 0x7f0c940769f0 secure :-1 s=READY pgs=233 cs=0 l=1 rev1=1 crypto rx=0x7f0cac001360 tx=0x7f0cac001e00 comp rx=0 tx=0).stop 2024-04-25T10:30:23.825 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:23.823+0000 7f0cb93e3640 1 -- 172.21.15.37:0/2307683022 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f0cb40cf570 msgr2=0x7f0cb40cbe50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:30:23.825 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:23.823+0000 7f0cb93e3640 1 --2- 172.21.15.37:0/2307683022 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f0cb40cf570 0x7f0cb40cbe50 secure :-1 s=READY pgs=478 cs=0 l=1 rev1=1 crypto rx=0x7f0c9c00c4a0 tx=0x7f0c9c00c960 comp rx=0 tx=0).stop 2024-04-25T10:30:23.825 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:23.823+0000 7f0cb93e3640 1 -- 172.21.15.37:0/2307683022 shutdown_connections 2024-04-25T10:30:23.826 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:23.823+0000 7f0cb93e3640 1 --2- 172.21.15.37:0/2307683022 >> [v2:172.21.15.203:6832/2418205237,v1:172.21.15.203:6833/2418205237] conn(0x7f0c940744e0 0x7f0c940769f0 unknown :-1 s=CLOSED pgs=233 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:30:23.826 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:23.823+0000 7f0cb93e3640 1 --2- 172.21.15.37:0/2307683022 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f0cb40d2860 0x7f0cb40b1870 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:30:23.826 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:23.823+0000 7f0cb93e3640 1 --2- 172.21.15.37:0/2307683022 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f0cb40cf570 0x7f0cb40cbe50 unknown :-1 s=CLOSED pgs=478 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:30:23.826 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:23.823+0000 7f0cb93e3640 1 --2- 172.21.15.37:0/2307683022 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f0cb40b2dc0 0x7f0cb415c000 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:30:23.826 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:23.823+0000 7f0cb93e3640 1 -- 172.21.15.37:0/2307683022 >> 172.21.15.37:0/2307683022 conn(0x7f0cb40b5440 msgr2=0x7f0cb4071c50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:30:23.826 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:23.823+0000 7f0cb93e3640 1 -- 172.21.15.37:0/2307683022 shutdown_connections 2024-04-25T10:30:23.826 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:23.823+0000 7f0cb93e3640 1 -- 172.21.15.37:0/2307683022 wait complete. 2024-04-25T10:30:23.836 INFO:tasks.cephfs.fuse_mount:Running fusermount -u on ubuntu@smithi037.front.sepia.ceph.com... 2024-04-25T10:30:23.836 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T10:30:23.836 DEBUG:teuthology.orchestra.run.smithi037:> sudo fusermount -u /home/ubuntu/cephtest/mnt.0 2024-04-25T10:30:23.859 INFO:tasks.cephfs.fuse_mount.ceph-fuse.0.smithi037.stderr:ceph-fuse[48855]: fuse finished with error 0 and tester_r 0 2024-04-25T10:30:23.861 INFO:teuthology.orchestra.run:waiting for 300 2024-04-25T10:30:23.902 INFO:tasks.cephfs.mount:Cleaning up mount ubuntu@smithi037.front.sepia.ceph.com 2024-04-25T10:30:23.902 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T10:30:23.902 DEBUG:teuthology.orchestra.run.smithi037:> (cd /home/ubuntu/cephtest && exec rmdir -- /home/ubuntu/cephtest/mnt.0) 2024-04-25T10:30:23.958 DEBUG:teuthology.orchestra.run.smithi037:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-04-25T10:30:24.110 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:24.107+0000 7fbab006e640 1 Processor -- start 2024-04-25T10:30:24.110 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:24.108+0000 7fbab006e640 1 -- start start 2024-04-25T10:30:24.113 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:24.109+0000 7fbab006e640 1 --2- >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7fbaa80b2dc0 0x7fbaa80d2860 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:30:24.113 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:24.109+0000 7fbab006e640 1 --2- >> v2:172.21.15.203:3300/0 conn(0x7fbaa806e610 0x7fbaa80ce560 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:30:24.113 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:24.109+0000 7fbab006e640 1 -- --> v1:172.21.15.37:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fbaa80d2e40 con 0x7fbaa806daf0 2024-04-25T10:30:24.113 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:24.109+0000 7fbab006e640 1 -- --> v2:172.21.15.203:3300/0 -- mon_getmap magic: 0 -- 0x7fbaa80b92c0 con 0x7fbaa806e610 2024-04-25T10:30:24.114 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:24.109+0000 7fbab006e640 1 -- --> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] -- mon_getmap magic: 0 -- 0x7fbaa80ce0d0 con 0x7fbaa80b2dc0 2024-04-25T10:30:24.114 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:24.109+0000 7fbaad609640 1 --1- >> v1:172.21.15.37:6789/0 conn(0x7fbaa806daf0 0x7fbaa806df00 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.37:6789/0 says I am v1:172.21.15.37:55134/0 (socket says 172.21.15.37:55134) 2024-04-25T10:30:24.114 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:24.109+0000 7fbaad609640 1 -- 172.21.15.37:0/1225901070 learned_addr learned my addr 172.21.15.37:0/1225901070 (peer_addr_for_me v1:172.21.15.37:0/0) 2024-04-25T10:30:24.114 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:24.109+0000 7fbaade0a640 1 --2- 172.21.15.37:0/1225901070 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7fbaa80b2dc0 0x7fbaa80d2860 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:30:24.114 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:24.109+0000 7fbaae60b640 1 --2- 172.21.15.37:0/1225901070 >> v2:172.21.15.203:3300/0 conn(0x7fbaa806e610 0x7fbaa80ce560 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:30:24.114 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:24.109+0000 7fbaace08640 1 -- 172.21.15.37:0/1225901070 <== mon.0 v1:172.21.15.37:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 3726547420 0 0) 0x7fbaa80d2e40 con 0x7fbaa806daf0 2024-04-25T10:30:24.114 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:24.109+0000 7fbaace08640 1 -- 172.21.15.37:0/1225901070 --> v1:172.21.15.37:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7fba90003520 con 0x7fbaa806daf0 2024-04-25T10:30:24.114 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:24.110+0000 7fbaace08640 1 -- 172.21.15.37:0/1225901070 <== mon.0 v1:172.21.15.37:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 1745732190 0 0) 0x7fba90003520 con 0x7fbaa806daf0 2024-04-25T10:30:24.114 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:24.110+0000 7fbaace08640 1 -- 172.21.15.37:0/1225901070 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7fbaa80b2dc0 msgr2=0x7fbaa80d2860 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:30:24.114 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:24.110+0000 7fbaace08640 1 --2- 172.21.15.37:0/1225901070 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7fbaa80b2dc0 0x7fbaa80d2860 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:30:24.114 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:24.110+0000 7fbaace08640 1 -- 172.21.15.37:0/1225901070 >> v2:172.21.15.203:3300/0 conn(0x7fbaa806e610 msgr2=0x7fbaa80ce560 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:30:24.114 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:24.110+0000 7fbaace08640 1 --2- 172.21.15.37:0/1225901070 >> v2:172.21.15.203:3300/0 conn(0x7fbaa806e610 0x7fbaa80ce560 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:30:24.114 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:24.110+0000 7fbaace08640 1 -- 172.21.15.37:0/1225901070 --> v1:172.21.15.37:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fbaa80d3120 con 0x7fbaa806daf0 2024-04-25T10:30:24.115 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:24.110+0000 7fbaace08640 1 -- 172.21.15.37:0/1225901070 <== mon.0 v1:172.21.15.37:6789/0 3 ==== mon_map magic: 0 ==== 413+0+0 (unknown 1577794600 0 0) 0x7fba9c001670 con 0x7fbaa806daf0 2024-04-25T10:30:24.115 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:24.110+0000 7fbaace08640 1 -- 172.21.15.37:0/1225901070 >> v1:172.21.15.37:6789/0 conn(0x7fbaa806daf0 legacy=0x7fbaa806df00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:30:24.115 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:24.110+0000 7fbaace08640 1 --2- 172.21.15.37:0/1225901070 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fba900039b0 0x7fba90003e20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:30:24.115 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:24.110+0000 7fbaace08640 1 --2- 172.21.15.37:0/1225901070 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fba90004800 0x7fba90004c10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:30:24.115 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:24.110+0000 7fbaace08640 1 --2- 172.21.15.37:0/1225901070 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7fbaa80b2dc0 0x7fba90005150 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:30:24.115 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:24.110+0000 7fbaace08640 1 -- 172.21.15.37:0/1225901070 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 -- 0x7fbaa80ce0d0 con 0x7fba90004800 2024-04-25T10:30:24.115 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:24.110+0000 7fbaace08640 1 -- 172.21.15.37:0/1225901070 --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_getmap magic: 0 -- 0x7fba9000b9c0 con 0x7fba900039b0 2024-04-25T10:30:24.115 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:24.110+0000 7fbaace08640 1 -- 172.21.15.37:0/1225901070 --> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] -- mon_getmap magic: 0 -- 0x7fba9000bb40 con 0x7fbaa80b2dc0 2024-04-25T10:30:24.115 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:24.110+0000 7fbaad609640 1 --2- 172.21.15.37:0/1225901070 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fba90004800 0x7fba90004c10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:30:24.115 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:24.110+0000 7fbaade0a640 1 --2- 172.21.15.37:0/1225901070 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fba900039b0 0x7fba90003e20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:30:24.115 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:24.110+0000 7fbaae60b640 1 --2- 172.21.15.37:0/1225901070 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7fbaa80b2dc0 0x7fba90005150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:30:24.116 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:24.110+0000 7fbaad609640 1 -- 172.21.15.37:0/1225901070 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7fbaa80b2dc0 msgr2=0x7fba90005150 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:30:24.116 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:24.110+0000 7fbaad609640 1 --2- 172.21.15.37:0/1225901070 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7fbaa80b2dc0 0x7fba90005150 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:30:24.116 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:24.110+0000 7fbaad609640 1 -- 172.21.15.37:0/1225901070 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fba900039b0 msgr2=0x7fba90003e20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:30:24.116 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:24.110+0000 7fbaad609640 1 --2- 172.21.15.37:0/1225901070 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fba900039b0 0x7fba90003e20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:30:24.116 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:24.110+0000 7fbaad609640 1 -- 172.21.15.37:0/1225901070 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fba90003520 con 0x7fba90004800 2024-04-25T10:30:24.116 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:24.111+0000 7fbaad609640 1 --2- 172.21.15.37:0/1225901070 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fba90004800 0x7fba90004c10 secure :-1 s=READY pgs=480 cs=0 l=1 rev1=1 crypto rx=0x7fba9c00c210 tx=0x7fba90005830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=4ff656d1aba3a5ad server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:30:24.116 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:24.111+0000 7fbaace08640 1 -- 172.21.15.37:0/1225901070 <== mon.0 v2:172.21.15.37:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fba9c007d50 con 0x7fba90004800 2024-04-25T10:30:24.116 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:24.111+0000 7fbaace08640 1 -- 172.21.15.37:0/1225901070 <== mon.0 v2:172.21.15.37:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fba9c005af0 con 0x7fba90004800 2024-04-25T10:30:24.116 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:24.111+0000 7fbaace08640 1 -- 172.21.15.37:0/1225901070 <== mon.0 v2:172.21.15.37:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fba9c007070 con 0x7fba90004800 2024-04-25T10:30:24.119 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:24.112+0000 7fbab006e640 1 -- 172.21.15.37:0/1225901070 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fba90004800 msgr2=0x7fba90004c10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:30:24.119 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:24.112+0000 7fbab006e640 1 --2- 172.21.15.37:0/1225901070 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fba90004800 0x7fba90004c10 secure :-1 s=READY pgs=480 cs=0 l=1 rev1=1 crypto rx=0x7fba9c00c210 tx=0x7fba90005830 comp rx=0 tx=0).stop 2024-04-25T10:30:24.119 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:24.112+0000 7fbaae60b640 1 -- 172.21.15.37:0/1225901070 reap_dead start 2024-04-25T10:30:24.119 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:24.112+0000 7fbab006e640 1 -- 172.21.15.37:0/1225901070 shutdown_connections 2024-04-25T10:30:24.119 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:24.112+0000 7fbab006e640 1 -- 172.21.15.37:0/1225901070 >> 172.21.15.37:0/1225901070 conn(0x7fbaa80b5440 msgr2=0x7fbaa80b5850 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:30:24.119 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:24.112+0000 7fbab006e640 1 -- 172.21.15.37:0/1225901070 shutdown_connections 2024-04-25T10:30:24.119 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:24.112+0000 7fbab006e640 1 -- 172.21.15.37:0/1225901070 wait complete. 2024-04-25T10:30:24.119 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:24.114+0000 7fbab006e640 1 Processor -- start 2024-04-25T10:30:24.119 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:24.114+0000 7fbab006e640 1 -- start start 2024-04-25T10:30:24.120 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:24.115+0000 7fbab006e640 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fba90004800 0x7fbaa8155e40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:30:24.120 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:24.115+0000 7fbab006e640 1 --2- >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7fbaa8162980 0x7fbaa80caf30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:30:24.120 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:24.115+0000 7fbab006e640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fbaa80cb470 0x7fbaa80cb880 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:30:24.120 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:24.115+0000 7fbab006e640 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 -- 0x7fbaa80d2860 con 0x7fbaa80cb470 2024-04-25T10:30:24.120 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:24.115+0000 7fbab006e640 1 -- --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_getmap magic: 0 -- 0x7fbaa80d36e0 con 0x7fba90004800 2024-04-25T10:30:24.120 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:24.115+0000 7fbab006e640 1 -- --> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] -- mon_getmap magic: 0 -- 0x7fbaa80d29e0 con 0x7fbaa8162980 2024-04-25T10:30:24.120 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:24.115+0000 7fbaae60b640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fbaa80cb470 0x7fbaa80cb880 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:30:24.120 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:24.115+0000 7fbaae60b640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fbaa80cb470 0x7fbaa80cb880 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.37:3300/0 says I am v2:172.21.15.37:59726/0 (socket says 172.21.15.37:59726) 2024-04-25T10:30:24.120 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:24.115+0000 7fbaae60b640 1 -- 172.21.15.37:0/3527255103 learned_addr learned my addr 172.21.15.37:0/3527255103 (peer_addr_for_me v2:172.21.15.37:0/0) 2024-04-25T10:30:24.120 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:24.115+0000 7fbaad609640 1 --2- 172.21.15.37:0/3527255103 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7fbaa8162980 0x7fbaa80caf30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:30:24.120 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:24.115+0000 7fbaade0a640 1 --2- 172.21.15.37:0/3527255103 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fba90004800 0x7fbaa8155e40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:30:24.120 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:24.115+0000 7fbaae60b640 1 -- 172.21.15.37:0/3527255103 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7fbaa8162980 msgr2=0x7fbaa80caf30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:30:24.120 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:24.115+0000 7fbaae60b640 1 --2- 172.21.15.37:0/3527255103 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7fbaa8162980 0x7fbaa80caf30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:30:24.120 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:24.115+0000 7fbaae60b640 1 -- 172.21.15.37:0/3527255103 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fba90004800 msgr2=0x7fbaa8155e40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:30:24.120 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:24.115+0000 7fbaae60b640 1 --2- 172.21.15.37:0/3527255103 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fba90004800 0x7fbaa8155e40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:30:24.121 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:24.115+0000 7fbaae60b640 1 -- 172.21.15.37:0/3527255103 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fbaa80c95c0 con 0x7fbaa80cb470 2024-04-25T10:30:24.121 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:24.116+0000 7fbaae60b640 1 --2- 172.21.15.37:0/3527255103 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fbaa80cb470 0x7fbaa80cb880 secure :-1 s=READY pgs=481 cs=0 l=1 rev1=1 crypto rx=0x7fbaa8074400 tx=0x7fba9800f580 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:30:24.121 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:24.116+0000 7fbaa67fc640 1 -- 172.21.15.37:0/3527255103 <== mon.0 v2:172.21.15.37:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fba98011e60 con 0x7fbaa80cb470 2024-04-25T10:30:24.121 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:24.116+0000 7fbaa67fc640 1 -- 172.21.15.37:0/3527255103 <== mon.0 v2:172.21.15.37:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fba9801d070 con 0x7fbaa80cb470 2024-04-25T10:30:24.121 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:24.116+0000 7fbaa67fc640 1 -- 172.21.15.37:0/3527255103 <== mon.0 v2:172.21.15.37:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fba98015670 con 0x7fbaa80cb470 2024-04-25T10:30:24.121 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:24.116+0000 7fbab006e640 1 -- 172.21.15.37:0/3527255103 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fbaa8156380 con 0x7fbaa80cb470 2024-04-25T10:30:24.121 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:24.116+0000 7fbab006e640 1 -- 172.21.15.37:0/3527255103 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fbaa80c8800 con 0x7fbaa80cb470 2024-04-25T10:30:24.121 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:24.117+0000 7fbab006e640 1 -- 172.21.15.37:0/3527255103 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fbaa806a140 con 0x7fbaa80cb470 2024-04-25T10:30:24.122 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:24.117+0000 7fbaa67fc640 1 -- 172.21.15.37:0/3527255103 <== mon.0 v2:172.21.15.37:3300/0 4 ==== mgrmap(e 5) ==== 96782+0+0 (secure 0 0 0) 0x7fba98015810 con 0x7fbaa80cb470 2024-04-25T10:30:24.123 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:24.118+0000 7fbaa67fc640 1 --2- 172.21.15.37:0/3527255103 >> [v2:172.21.15.203:6832/2418205237,v1:172.21.15.203:6833/2418205237] conn(0x7fba940744e0 0x7fba940769f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:30:24.123 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:24.118+0000 7fbaa67fc640 1 -- 172.21.15.37:0/3527255103 <== mon.0 v2:172.21.15.37:3300/0 5 ==== osd_map(36..36 src has 1..36) ==== 5709+0+0 (secure 0 0 0) 0x7fba98095a10 con 0x7fbaa80cb470 2024-04-25T10:30:24.123 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:24.118+0000 7fbaade0a640 1 --2- 172.21.15.37:0/3527255103 >> [v2:172.21.15.203:6832/2418205237,v1:172.21.15.203:6833/2418205237] conn(0x7fba940744e0 0x7fba940769f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:30:24.123 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:24.119+0000 7fbaade0a640 1 --2- 172.21.15.37:0/3527255103 >> [v2:172.21.15.203:6832/2418205237,v1:172.21.15.203:6833/2418205237] conn(0x7fba940744e0 0x7fba940769f0 secure :-1 s=READY pgs=234 cs=0 l=1 rev1=1 crypto rx=0x7fbaa8074cc0 tx=0x7fbaa0009110 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:30:24.125 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:24.123+0000 7fbaa67fc640 1 -- 172.21.15.37:0/3527255103 <== mon.0 v2:172.21.15.37:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187246 (secure 0 0 0) 0x7fba98010040 con 0x7fbaa80cb470 2024-04-25T10:30:24.347 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:24.345+0000 7fbab006e640 1 -- 172.21.15.37:0/3527255103 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7fbaa806ed60 con 0x7fbaa80cb470 2024-04-25T10:30:24.348 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:24.346+0000 7fbaa67fc640 1 -- 172.21.15.37:0/3527255103 <== mon.0 v2:172.21.15.37:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 17 v17) ==== 94+0+5181 (secure 0 0 0) 0x7fba98060230 con 0x7fbaa80cb470 2024-04-25T10:30:24.348 INFO:teuthology.orchestra.run.smithi037.stdout: 2024-04-25T10:30:24.348 INFO:teuthology.orchestra.run.smithi037.stdout:{"epoch":17,"default_fscid":2,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":4790,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.203:6837/2245361396","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6836","nonce":2245361396},{"type":"v1","addr":"172.21.15.203:6837","nonce":2245361396}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":11},{"gid":5005,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.203:6835/3353571013","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6834","nonce":3353571013},{"type":"v1","addr":"172.21.15.203:6835","nonce":3353571013}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":12},{"gid":5313,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.37:6834/824801265","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.37:6833","nonce":824801265},{"type":"v1","addr":"172.21.15.37:6834","nonce":824801265}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":12}],"filesystems":[{"mdsmap":{"epoch":17,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-04-25T10:30:12.130473+0000","modified":"2024-04-25T10:30:15.157110+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":1,"in":[0],"up":{"mds_0":5316},"failed":[],"damaged":[],"stopped":[],"info":{"gid_5316":{"gid":5316,"name":"a","rank":0,"incarnation":13,"state":"up:active","state_seq":3,"addr":"172.21.15.37:6835/311552410","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.37:6832","nonce":311552410},{"type":"v1","addr":"172.21.15.37:6835","nonce":311552410}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[5],"metadata_pool":4,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":5316,"qdb_cluster":[5316]},"id":2}]} 2024-04-25T10:30:24.348 INFO:teuthology.orchestra.run.smithi037.stderr:dumped fsmap epoch 17 2024-04-25T10:30:24.350 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:24.348+0000 7fbab006e640 1 -- 172.21.15.37:0/3527255103 >> [v2:172.21.15.203:6832/2418205237,v1:172.21.15.203:6833/2418205237] conn(0x7fba940744e0 msgr2=0x7fba940769f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:30:24.350 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:24.348+0000 7fbab006e640 1 --2- 172.21.15.37:0/3527255103 >> [v2:172.21.15.203:6832/2418205237,v1:172.21.15.203:6833/2418205237] conn(0x7fba940744e0 0x7fba940769f0 secure :-1 s=READY pgs=234 cs=0 l=1 rev1=1 crypto rx=0x7fbaa8074cc0 tx=0x7fbaa0009110 comp rx=0 tx=0).stop 2024-04-25T10:30:24.350 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:24.348+0000 7fbab006e640 1 -- 172.21.15.37:0/3527255103 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fbaa80cb470 msgr2=0x7fbaa80cb880 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:30:24.350 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:24.348+0000 7fbab006e640 1 --2- 172.21.15.37:0/3527255103 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fbaa80cb470 0x7fbaa80cb880 secure :-1 s=READY pgs=481 cs=0 l=1 rev1=1 crypto rx=0x7fbaa8074400 tx=0x7fba9800f580 comp rx=0 tx=0).stop 2024-04-25T10:30:24.350 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:24.348+0000 7fbab006e640 1 -- 172.21.15.37:0/3527255103 shutdown_connections 2024-04-25T10:30:24.350 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:24.348+0000 7fbab006e640 1 --2- 172.21.15.37:0/3527255103 >> [v2:172.21.15.203:6832/2418205237,v1:172.21.15.203:6833/2418205237] conn(0x7fba940744e0 0x7fba940769f0 unknown :-1 s=CLOSED pgs=234 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:30:24.351 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:24.348+0000 7fbab006e640 1 --2- 172.21.15.37:0/3527255103 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fbaa80cb470 0x7fbaa80cb880 unknown :-1 s=CLOSED pgs=481 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:30:24.351 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:24.348+0000 7fbab006e640 1 --2- 172.21.15.37:0/3527255103 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7fbaa8162980 0x7fbaa80caf30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:30:24.351 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:24.348+0000 7fbab006e640 1 --2- 172.21.15.37:0/3527255103 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fba90004800 0x7fbaa8155e40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:30:24.351 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:24.348+0000 7fbab006e640 1 -- 172.21.15.37:0/3527255103 >> 172.21.15.37:0/3527255103 conn(0x7fbaa80b5440 msgr2=0x7fbaa80ce460 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:30:24.351 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:24.348+0000 7fbab006e640 1 -- 172.21.15.37:0/3527255103 shutdown_connections 2024-04-25T10:30:24.351 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:24.348+0000 7fbab006e640 1 -- 172.21.15.37:0/3527255103 wait complete. 2024-04-25T10:30:24.361 DEBUG:teuthology.orchestra.run.smithi037:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 300 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-mds.a.asok --format=json flush journal 2024-04-25T10:30:24.500 INFO:teuthology.orchestra.run.smithi037.stdout:{"message":"","return_code":0} 2024-04-25T10:30:24.501 DEBUG:tasks.cephfs.filesystem:_json_asok output { "message": "", "return_code": 0 } 2024-04-25T10:30:24.501 DEBUG:teuthology.orchestra.run.smithi037:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs fail cephfs 2024-04-25T10:30:24.613 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:24.611+0000 7fd12a6d9640 1 Processor -- start 2024-04-25T10:30:24.614 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:24.611+0000 7fd12a6d9640 1 -- start start 2024-04-25T10:30:24.616 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:24.613+0000 7fd12a6d9640 1 --2- >> v2:172.21.15.203:3300/0 conn(0x7fd124143300 0x7fd1240719b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:30:24.616 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:24.613+0000 7fd12a6d9640 1 --2- >> v2:172.21.15.37:3300/0 conn(0x7fd124071ef0 0x7fd1240cf3d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:30:24.616 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:24.613+0000 7fd12a6d9640 1 --2- >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7fd1240cf910 0x7fd124160aa0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:30:24.616 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:24.613+0000 7fd12a6d9640 1 -- --> v2:172.21.15.37:3300/0 -- mon_getmap magic: 0 -- 0x7fd1240b7990 con 0x7fd124071ef0 2024-04-25T10:30:24.616 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:24.613+0000 7fd12a6d9640 1 -- --> v2:172.21.15.203:3300/0 -- mon_getmap magic: 0 -- 0x7fd1240b6b30 con 0x7fd124143300 2024-04-25T10:30:24.617 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:24.613+0000 7fd12a6d9640 1 -- --> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] -- mon_getmap magic: 0 -- 0x7fd124072300 con 0x7fd1240cf910 2024-04-25T10:30:24.617 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:24.613+0000 7fd1237fe640 1 --2- >> v2:172.21.15.37:3300/0 conn(0x7fd124071ef0 0x7fd1240cf3d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:30:24.617 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:24.613+0000 7fd1237fe640 1 --2- >> v2:172.21.15.37:3300/0 conn(0x7fd124071ef0 0x7fd1240cf3d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.37:3300/0 says I am v2:172.21.15.37:59728/0 (socket says 172.21.15.37:59728) 2024-04-25T10:30:24.617 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:24.613+0000 7fd1237fe640 1 -- 172.21.15.37:0/558482578 learned_addr learned my addr 172.21.15.37:0/558482578 (peer_addr_for_me v2:172.21.15.37:0/0) 2024-04-25T10:30:24.617 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:24.613+0000 7fd128c76640 1 --2- 172.21.15.37:0/558482578 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7fd1240cf910 0x7fd124160aa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:30:24.617 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:24.613+0000 7fd123fff640 1 --2- 172.21.15.37:0/558482578 >> v2:172.21.15.203:3300/0 conn(0x7fd124143300 0x7fd1240719b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:30:24.617 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:24.613+0000 7fd1237fe640 1 -- 172.21.15.37:0/558482578 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7fd1240cf910 msgr2=0x7fd124160aa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:30:24.617 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:24.613+0000 7fd1237fe640 1 --2- 172.21.15.37:0/558482578 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7fd1240cf910 0x7fd124160aa0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:30:24.617 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:24.613+0000 7fd1237fe640 1 -- 172.21.15.37:0/558482578 >> v2:172.21.15.203:3300/0 conn(0x7fd124143300 msgr2=0x7fd1240719b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:30:24.617 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:24.613+0000 7fd1237fe640 1 --2- 172.21.15.37:0/558482578 >> v2:172.21.15.203:3300/0 conn(0x7fd124143300 0x7fd1240719b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:30:24.617 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:24.613+0000 7fd1237fe640 1 -- 172.21.15.37:0/558482578 --> v2:172.21.15.37:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fd1240cfd20 con 0x7fd124071ef0 2024-04-25T10:30:24.617 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:24.614+0000 7fd1237fe640 1 --2- 172.21.15.37:0/558482578 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fd124071ef0 0x7fd1240cf3d0 secure :-1 s=READY pgs=482 cs=0 l=1 rev1=1 crypto rx=0x7fd1240724b0 tx=0x7fd11402c890 comp rx=0 tx=0).ready entity=mon.0 client_cookie=479701e6f7a111ce server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:30:24.617 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:24.614+0000 7fd122ffd640 1 -- 172.21.15.37:0/558482578 <== mon.0 v2:172.21.15.37:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd114032790 con 0x7fd124071ef0 2024-04-25T10:30:24.617 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:24.614+0000 7fd122ffd640 1 -- 172.21.15.37:0/558482578 <== mon.0 v2:172.21.15.37:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fd114038070 con 0x7fd124071ef0 2024-04-25T10:30:24.618 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:24.614+0000 7fd122ffd640 1 -- 172.21.15.37:0/558482578 <== mon.0 v2:172.21.15.37:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd114032b50 con 0x7fd124071ef0 2024-04-25T10:30:24.619 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:24.615+0000 7fd12a6d9640 1 -- 172.21.15.37:0/558482578 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fd124071ef0 msgr2=0x7fd1240cf3d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:30:24.619 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:24.615+0000 7fd12a6d9640 1 --2- 172.21.15.37:0/558482578 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fd124071ef0 0x7fd1240cf3d0 secure :-1 s=READY pgs=482 cs=0 l=1 rev1=1 crypto rx=0x7fd1240724b0 tx=0x7fd11402c890 comp rx=0 tx=0).stop 2024-04-25T10:30:24.619 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:24.615+0000 7fd12a6d9640 1 -- 172.21.15.37:0/558482578 shutdown_connections 2024-04-25T10:30:24.619 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:24.615+0000 7fd12a6d9640 1 --2- 172.21.15.37:0/558482578 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fd124071ef0 0x7fd1240cf3d0 unknown :-1 s=CLOSED pgs=482 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:30:24.619 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:24.615+0000 7fd12a6d9640 1 --2- 172.21.15.37:0/558482578 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7fd1240cf910 0x7fd124160aa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:30:24.619 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:24.615+0000 7fd12a6d9640 1 --2- 172.21.15.37:0/558482578 >> v2:172.21.15.203:3300/0 conn(0x7fd124143300 0x7fd1240719b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:30:24.619 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:24.615+0000 7fd12a6d9640 1 -- 172.21.15.37:0/558482578 >> 172.21.15.37:0/558482578 conn(0x7fd1240b45d0 msgr2=0x7fd1240b49e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:30:24.619 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:24.615+0000 7fd12a6d9640 1 -- 172.21.15.37:0/558482578 shutdown_connections 2024-04-25T10:30:24.619 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:24.615+0000 7fd12a6d9640 1 -- 172.21.15.37:0/558482578 wait complete. 2024-04-25T10:30:24.619 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:24.616+0000 7fd12a6d9640 1 Processor -- start 2024-04-25T10:30:24.619 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:24.617+0000 7fd12a6d9640 1 -- start start 2024-04-25T10:30:24.621 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:24.618+0000 7fd12a6d9640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fd124071ef0 0x7fd124141490 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:30:24.621 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:24.618+0000 7fd12a6d9640 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fd1240cf910 0x7fd1241f5c50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:30:24.621 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:24.618+0000 7fd12a6d9640 1 --2- >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7fd124143300 0x7fd124070210 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:30:24.621 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:24.618+0000 7fd12a6d9640 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 -- 0x7fd1240b7340 con 0x7fd124071ef0 2024-04-25T10:30:24.621 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:24.618+0000 7fd12a6d9640 1 -- --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_getmap magic: 0 -- 0x7fd1240b7060 con 0x7fd1240cf910 2024-04-25T10:30:24.621 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:24.618+0000 7fd12a6d9640 1 -- --> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] -- mon_getmap magic: 0 -- 0x7fd1240b7620 con 0x7fd124143300 2024-04-25T10:30:24.621 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:24.618+0000 7fd123fff640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fd124071ef0 0x7fd124141490 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:30:24.622 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:24.618+0000 7fd123fff640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fd124071ef0 0x7fd124141490 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.37:3300/0 says I am v2:172.21.15.37:59742/0 (socket says 172.21.15.37:59742) 2024-04-25T10:30:24.622 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:24.618+0000 7fd123fff640 1 -- 172.21.15.37:0/4072088441 learned_addr learned my addr 172.21.15.37:0/4072088441 (peer_addr_for_me v2:172.21.15.37:0/0) 2024-04-25T10:30:24.622 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:24.618+0000 7fd1237fe640 1 --2- 172.21.15.37:0/4072088441 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fd1240cf910 0x7fd1241f5c50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:30:24.622 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:24.618+0000 7fd128c76640 1 --2- 172.21.15.37:0/4072088441 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7fd124143300 0x7fd124070210 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:30:24.622 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:24.618+0000 7fd123fff640 1 -- 172.21.15.37:0/4072088441 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7fd124143300 msgr2=0x7fd124070210 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:30:24.622 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:24.618+0000 7fd123fff640 1 --2- 172.21.15.37:0/4072088441 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7fd124143300 0x7fd124070210 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:30:24.622 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:24.618+0000 7fd123fff640 1 -- 172.21.15.37:0/4072088441 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fd1240cf910 msgr2=0x7fd1241f5c50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:30:24.622 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:24.618+0000 7fd123fff640 1 --2- 172.21.15.37:0/4072088441 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fd1240cf910 0x7fd1241f5c50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:30:24.622 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:24.618+0000 7fd123fff640 1 -- 172.21.15.37:0/4072088441 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fd1240c7ed0 con 0x7fd124071ef0 2024-04-25T10:30:24.622 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:24.619+0000 7fd123fff640 1 --2- 172.21.15.37:0/4072088441 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fd124071ef0 0x7fd124141490 secure :-1 s=READY pgs=483 cs=0 l=1 rev1=1 crypto rx=0x7fd10c00d5a0 tx=0x7fd10c00da60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:30:24.622 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:24.619+0000 7fd120ff9640 1 -- 172.21.15.37:0/4072088441 <== mon.0 v2:172.21.15.37:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd10c019070 con 0x7fd124071ef0 2024-04-25T10:30:24.622 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:24.619+0000 7fd120ff9640 1 -- 172.21.15.37:0/4072088441 <== mon.0 v2:172.21.15.37:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fd10c004240 con 0x7fd124071ef0 2024-04-25T10:30:24.622 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:24.619+0000 7fd12a6d9640 1 -- 172.21.15.37:0/4072088441 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fd1240c81b0 con 0x7fd124071ef0 2024-04-25T10:30:24.622 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:24.619+0000 7fd120ff9640 1 -- 172.21.15.37:0/4072088441 <== mon.0 v2:172.21.15.37:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd10c003610 con 0x7fd124071ef0 2024-04-25T10:30:24.623 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:24.619+0000 7fd12a6d9640 1 -- 172.21.15.37:0/4072088441 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fd1240cd070 con 0x7fd124071ef0 2024-04-25T10:30:24.623 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:24.620+0000 7fd12a6d9640 1 -- 172.21.15.37:0/4072088441 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fd12406a140 con 0x7fd124071ef0 2024-04-25T10:30:24.623 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:24.621+0000 7fd120ff9640 1 -- 172.21.15.37:0/4072088441 <== mon.0 v2:172.21.15.37:3300/0 4 ==== mgrmap(e 5) ==== 96782+0+0 (secure 0 0 0) 0x7fd10c01c070 con 0x7fd124071ef0 2024-04-25T10:30:24.623 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:24.621+0000 7fd120ff9640 1 --2- 172.21.15.37:0/4072088441 >> [v2:172.21.15.203:6832/2418205237,v1:172.21.15.203:6833/2418205237] conn(0x7fd1040744b0 0x7fd1040769c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:30:24.624 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:24.621+0000 7fd120ff9640 1 -- 172.21.15.37:0/4072088441 <== mon.0 v2:172.21.15.37:3300/0 5 ==== osd_map(36..36 src has 1..36) ==== 5709+0+0 (secure 0 0 0) 0x7fd10c0949a0 con 0x7fd124071ef0 2024-04-25T10:30:24.624 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:24.621+0000 7fd1237fe640 1 --2- 172.21.15.37:0/4072088441 >> [v2:172.21.15.203:6832/2418205237,v1:172.21.15.203:6833/2418205237] conn(0x7fd1040744b0 0x7fd1040769c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:30:24.624 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:24.622+0000 7fd1237fe640 1 --2- 172.21.15.37:0/4072088441 >> [v2:172.21.15.203:6832/2418205237,v1:172.21.15.203:6833/2418205237] conn(0x7fd1040744b0 0x7fd1040769c0 secure :-1 s=READY pgs=235 cs=0 l=1 rev1=1 crypto rx=0x7fd114037000 tx=0x7fd114032000 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:30:24.628 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:24.626+0000 7fd120ff9640 1 -- 172.21.15.37:0/4072088441 <== mon.0 v2:172.21.15.37:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187246 (secure 0 0 0) 0x7fd10c05eb80 con 0x7fd124071ef0 2024-04-25T10:30:24.854 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:24.852+0000 7fd12a6d9640 1 -- 172.21.15.37:0/4072088441 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_command({"prefix": "fs fail", "fs_name": "cephfs"} v 0) -- 0x7fd124158bf0 con 0x7fd124071ef0 2024-04-25T10:30:24.854 INFO:tasks.ceph.mon.a.smithi037.stderr:2024-04-25T10:30:24.852+0000 7fb25155b640 -1 log_channel(cluster) log [ERR] : Health check failed: 1 filesystem is offline (MDS_ALL_DOWN) 2024-04-25T10:30:24.856 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:24.854+0000 7fd120ff9640 1 -- 172.21.15.37:0/4072088441 <== mon.0 v2:172.21.15.37:3300/0 7 ==== mon_command_ack([{"prefix": "fs fail", "fs_name": "cephfs"}]=0 cephfs marked not joinable; MDS cannot join the cluster. All MDS ranks marked failed. v18) ==== 161+0+0 (secure 0 0 0) 0x7fd10c062e80 con 0x7fd124071ef0 2024-04-25T10:30:24.856 INFO:teuthology.orchestra.run.smithi037.stderr:cephfs marked not joinable; MDS cannot join the cluster. All MDS ranks marked failed. 2024-04-25T10:30:24.859 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:24.856+0000 7fd12a6d9640 1 -- 172.21.15.37:0/4072088441 >> [v2:172.21.15.203:6832/2418205237,v1:172.21.15.203:6833/2418205237] conn(0x7fd1040744b0 msgr2=0x7fd1040769c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:30:24.859 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:24.856+0000 7fd12a6d9640 1 --2- 172.21.15.37:0/4072088441 >> [v2:172.21.15.203:6832/2418205237,v1:172.21.15.203:6833/2418205237] conn(0x7fd1040744b0 0x7fd1040769c0 secure :-1 s=READY pgs=235 cs=0 l=1 rev1=1 crypto rx=0x7fd114037000 tx=0x7fd114032000 comp rx=0 tx=0).stop 2024-04-25T10:30:24.859 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:24.857+0000 7fd12a6d9640 1 -- 172.21.15.37:0/4072088441 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fd124071ef0 msgr2=0x7fd124141490 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:30:24.859 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:24.857+0000 7fd12a6d9640 1 --2- 172.21.15.37:0/4072088441 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fd124071ef0 0x7fd124141490 secure :-1 s=READY pgs=483 cs=0 l=1 rev1=1 crypto rx=0x7fd10c00d5a0 tx=0x7fd10c00da60 comp rx=0 tx=0).stop 2024-04-25T10:30:24.859 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:24.857+0000 7fd12a6d9640 1 -- 172.21.15.37:0/4072088441 shutdown_connections 2024-04-25T10:30:24.859 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:24.857+0000 7fd12a6d9640 1 --2- 172.21.15.37:0/4072088441 >> [v2:172.21.15.203:6832/2418205237,v1:172.21.15.203:6833/2418205237] conn(0x7fd1040744b0 0x7fd1040769c0 unknown :-1 s=CLOSED pgs=235 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:30:24.860 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:24.857+0000 7fd12a6d9640 1 --2- 172.21.15.37:0/4072088441 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7fd124143300 0x7fd124070210 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:30:24.860 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:24.857+0000 7fd12a6d9640 1 --2- 172.21.15.37:0/4072088441 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fd1240cf910 0x7fd1241f5c50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:30:24.860 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:24.857+0000 7fd12a6d9640 1 --2- 172.21.15.37:0/4072088441 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fd124071ef0 0x7fd124141490 unknown :-1 s=CLOSED pgs=483 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:30:24.860 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:24.857+0000 7fd12a6d9640 1 -- 172.21.15.37:0/4072088441 >> 172.21.15.37:0/4072088441 conn(0x7fd1240b45d0 msgr2=0x7fd1240b2e80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:30:24.860 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:24.857+0000 7fd12a6d9640 1 -- 172.21.15.37:0/4072088441 shutdown_connections 2024-04-25T10:30:24.860 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:24.857+0000 7fd12a6d9640 1 -- 172.21.15.37:0/4072088441 wait complete. 2024-04-25T10:30:24.860 INFO:tasks.ceph.mds.a.smithi037.stderr: -3201> 2024-04-25T10:30:16.729+0000 7f5d17c43640 -1 mds.pinger is_rank_lagging: rank=0 was never sent ping request. 2024-04-25T10:30:24.872 DEBUG:teuthology.orchestra.run.smithi037:> sudo mkdir -p /etc/ceph && sudo chmod 0755 /etc/ceph && sudo tee /etc/ceph/ceph.conf && sudo chmod 0644 /etc/ceph/ceph.conf > /dev/null 2024-04-25T10:30:24.875 DEBUG:teuthology.orchestra.run.smithi203:> sudo mkdir -p /etc/ceph && sudo chmod 0755 /etc/ceph && sudo tee /etc/ceph/ceph.conf && sudo chmod 0644 /etc/ceph/ceph.conf > /dev/null 2024-04-25T10:30:24.918 INFO:tasks.ceph.mds.a.smithi037.stdout:starting mds.a at 2024-04-25T10:30:24.918 INFO:teuthology.orchestra.run.smithi037.stdout:[global] 2024-04-25T10:30:24.918 INFO:teuthology.orchestra.run.smithi037.stdout: chdir = "" 2024-04-25T10:30:24.918 INFO:teuthology.orchestra.run.smithi037.stdout: pid file = /var/run/ceph/$cluster-$name.pid 2024-04-25T10:30:24.918 INFO:teuthology.orchestra.run.smithi037.stdout: auth supported = cephx 2024-04-25T10:30:24.918 INFO:teuthology.orchestra.run.smithi037.stdout: 2024-04-25T10:30:24.918 INFO:teuthology.orchestra.run.smithi037.stdout: filestore xattr use omap = true 2024-04-25T10:30:24.919 INFO:teuthology.orchestra.run.smithi037.stdout: 2024-04-25T10:30:24.919 INFO:teuthology.orchestra.run.smithi037.stdout: mon clock drift allowed = 1.000 2024-04-25T10:30:24.919 INFO:teuthology.orchestra.run.smithi037.stdout: 2024-04-25T10:30:24.919 INFO:teuthology.orchestra.run.smithi037.stdout: osd crush chooseleaf type = 0 2024-04-25T10:30:24.919 INFO:teuthology.orchestra.run.smithi037.stdout: auth debug = true 2024-04-25T10:30:24.919 INFO:teuthology.orchestra.run.smithi037.stdout: 2024-04-25T10:30:24.919 INFO:teuthology.orchestra.run.smithi037.stdout: ms die on old message = true 2024-04-25T10:30:24.919 INFO:teuthology.orchestra.run.smithi037.stdout: ms die on bug = true 2024-04-25T10:30:24.919 INFO:teuthology.orchestra.run.smithi037.stdout: 2024-04-25T10:30:24.919 INFO:teuthology.orchestra.run.smithi037.stdout: mon max pg per osd = 10000 # >= luminous 2024-04-25T10:30:24.919 INFO:teuthology.orchestra.run.smithi037.stdout: mon pg warn max object skew = 0 2024-04-25T10:30:24.919 INFO:teuthology.orchestra.run.smithi037.stdout: 2024-04-25T10:30:24.919 INFO:teuthology.orchestra.run.smithi037.stdout: # disable pg_autoscaler by default for new pools 2024-04-25T10:30:24.919 INFO:teuthology.orchestra.run.smithi037.stdout: osd_pool_default_pg_autoscale_mode = off 2024-04-25T10:30:24.919 INFO:teuthology.orchestra.run.smithi037.stdout: 2024-04-25T10:30:24.920 INFO:teuthology.orchestra.run.smithi037.stdout: osd pool default size = 2 2024-04-25T10:30:24.920 INFO:teuthology.orchestra.run.smithi037.stdout: 2024-04-25T10:30:24.920 INFO:teuthology.orchestra.run.smithi037.stdout: mon osd allow primary affinity = true 2024-04-25T10:30:24.920 INFO:teuthology.orchestra.run.smithi037.stdout: mon osd allow pg remap = true 2024-04-25T10:30:24.920 INFO:teuthology.orchestra.run.smithi037.stdout: mon warn on legacy crush tunables = false 2024-04-25T10:30:24.920 INFO:teuthology.orchestra.run.smithi037.stdout: mon warn on crush straw calc version zero = false 2024-04-25T10:30:24.920 INFO:teuthology.orchestra.run.smithi037.stdout: mon warn on no sortbitwise = false 2024-04-25T10:30:24.920 INFO:teuthology.orchestra.run.smithi037.stdout: mon warn on osd down out interval zero = false 2024-04-25T10:30:24.920 INFO:teuthology.orchestra.run.smithi037.stdout: mon warn on too few osds = false 2024-04-25T10:30:24.920 INFO:teuthology.orchestra.run.smithi037.stdout: mon_warn_on_pool_pg_num_not_power_of_two = false 2024-04-25T10:30:24.920 INFO:teuthology.orchestra.run.smithi037.stdout: mon_warn_on_pool_no_redundancy = false 2024-04-25T10:30:24.920 INFO:teuthology.orchestra.run.smithi037.stdout: mon_allow_pool_size_one = true 2024-04-25T10:30:24.920 INFO:teuthology.orchestra.run.smithi037.stdout: 2024-04-25T10:30:24.920 INFO:teuthology.orchestra.run.smithi037.stdout: osd pool default erasure code profile = plugin=jerasure technique=reed_sol_van k=2 m=1 crush-failure-domain=osd 2024-04-25T10:30:24.920 INFO:teuthology.orchestra.run.smithi037.stdout: 2024-04-25T10:30:24.921 INFO:teuthology.orchestra.run.smithi037.stdout: osd default data pool replay window = 5 2024-04-25T10:30:24.921 INFO:teuthology.orchestra.run.smithi037.stdout: 2024-04-25T10:30:24.921 INFO:teuthology.orchestra.run.smithi037.stdout: mon allow pool delete = true 2024-04-25T10:30:24.921 INFO:teuthology.orchestra.run.smithi037.stdout: 2024-04-25T10:30:24.921 INFO:teuthology.orchestra.run.smithi037.stdout: mon cluster log file level = debug 2024-04-25T10:30:24.921 INFO:teuthology.orchestra.run.smithi037.stdout: debug asserts on shutdown = true 2024-04-25T10:30:24.921 INFO:teuthology.orchestra.run.smithi037.stdout: mon health detail to clog = false 2024-04-25T10:30:24.921 INFO:teuthology.orchestra.run.smithi037.stdout: mon host = "172.21.15.37,172.21.15.203,[v2:172.21.15.203:3301,v1:172.21.15.203:6790]" 2024-04-25T10:30:24.921 INFO:teuthology.orchestra.run.smithi037.stdout: fsid = f4acda28-be34-492a-b286-994afd36f61d 2024-04-25T10:30:24.921 INFO:teuthology.orchestra.run.smithi037.stdout: 2024-04-25T10:30:24.921 INFO:teuthology.orchestra.run.smithi037.stdout:[osd] 2024-04-25T10:30:24.921 INFO:teuthology.orchestra.run.smithi037.stdout: osd journal size = 100 2024-04-25T10:30:24.921 INFO:teuthology.orchestra.run.smithi037.stdout: 2024-04-25T10:30:24.921 INFO:teuthology.orchestra.run.smithi037.stdout: osd scrub load threshold = 5.0 2024-04-25T10:30:24.921 INFO:teuthology.orchestra.run.smithi037.stdout: osd scrub max interval = 600 2024-04-25T10:30:24.922 INFO:teuthology.orchestra.run.smithi037.stdout: osd mclock profile = high_recovery_ops 2024-04-25T10:30:24.922 INFO:teuthology.orchestra.run.smithi037.stdout: 2024-04-25T10:30:24.922 INFO:teuthology.orchestra.run.smithi037.stdout: osd recover clone overlap = true 2024-04-25T10:30:24.922 INFO:teuthology.orchestra.run.smithi037.stdout: osd recovery max chunk = 1048576 2024-04-25T10:30:24.922 INFO:teuthology.orchestra.run.smithi037.stdout: 2024-04-25T10:30:24.922 INFO:teuthology.orchestra.run.smithi037.stdout: osd debug shutdown = true 2024-04-25T10:30:24.922 INFO:teuthology.orchestra.run.smithi037.stdout: osd debug op order = true 2024-04-25T10:30:24.922 INFO:teuthology.orchestra.run.smithi037.stdout: osd debug verify stray on activate = true 2024-04-25T10:30:24.922 INFO:teuthology.orchestra.run.smithi037.stdout: osd debug trim objects = true 2024-04-25T10:30:24.922 INFO:teuthology.orchestra.run.smithi037.stdout: 2024-04-25T10:30:24.922 INFO:teuthology.orchestra.run.smithi037.stdout: osd open classes on start = true 2024-04-25T10:30:24.922 INFO:teuthology.orchestra.run.smithi037.stdout: osd debug pg log writeout = true 2024-04-25T10:30:24.922 INFO:teuthology.orchestra.run.smithi037.stdout: 2024-04-25T10:30:24.922 INFO:teuthology.orchestra.run.smithi037.stdout: osd deep scrub update digest min age = 30 2024-04-25T10:30:24.923 INFO:teuthology.orchestra.run.smithi037.stdout: 2024-04-25T10:30:24.923 INFO:teuthology.orchestra.run.smithi037.stdout: osd map max advance = 10 2024-04-25T10:30:24.923 INFO:teuthology.orchestra.run.smithi037.stdout: 2024-04-25T10:30:24.923 INFO:teuthology.orchestra.run.smithi037.stdout: journal zero on create = true 2024-04-25T10:30:24.923 INFO:teuthology.orchestra.run.smithi037.stdout: 2024-04-25T10:30:24.923 INFO:teuthology.orchestra.run.smithi037.stdout: filestore ondisk finisher threads = 3 2024-04-25T10:30:24.923 INFO:teuthology.orchestra.run.smithi037.stdout: filestore apply finisher threads = 3 2024-04-25T10:30:24.923 INFO:teuthology.orchestra.run.smithi037.stdout: 2024-04-25T10:30:24.923 INFO:teuthology.orchestra.run.smithi037.stdout: bdev debug aio = true 2024-04-25T10:30:24.923 INFO:teuthology.orchestra.run.smithi037.stdout: osd debug misdirected ops = true 2024-04-25T10:30:24.923 INFO:teuthology.orchestra.run.smithi037.stdout: bdev async discard = True 2024-04-25T10:30:24.923 INFO:teuthology.orchestra.run.smithi037.stdout: bdev enable discard = True 2024-04-25T10:30:24.923 INFO:teuthology.orchestra.run.smithi037.stdout: bluestore allocator = bitmap 2024-04-25T10:30:24.923 INFO:teuthology.orchestra.run.smithi037.stdout: bluestore block size = 96636764160 2024-04-25T10:30:24.923 INFO:teuthology.orchestra.run.smithi037.stdout: bluestore fsck on mount = True 2024-04-25T10:30:24.923 INFO:teuthology.orchestra.run.smithi037.stdout: debug bluefs = 1/20 2024-04-25T10:30:24.924 INFO:teuthology.orchestra.run.smithi037.stdout: debug bluestore = 1/20 2024-04-25T10:30:24.924 INFO:teuthology.orchestra.run.smithi037.stdout: debug ms = 1 2024-04-25T10:30:24.924 INFO:teuthology.orchestra.run.smithi037.stdout: debug osd = 20 2024-04-25T10:30:24.924 INFO:teuthology.orchestra.run.smithi037.stdout: debug rocksdb = 4/10 2024-04-25T10:30:24.924 INFO:teuthology.orchestra.run.smithi037.stdout: mon osd backfillfull_ratio = 0.85 2024-04-25T10:30:24.924 INFO:teuthology.orchestra.run.smithi037.stdout: mon osd full ratio = 0.9 2024-04-25T10:30:24.924 INFO:teuthology.orchestra.run.smithi037.stdout: mon osd nearfull ratio = 0.8 2024-04-25T10:30:24.924 INFO:teuthology.orchestra.run.smithi037.stdout: osd failsafe full ratio = 0.95 2024-04-25T10:30:24.924 INFO:teuthology.orchestra.run.smithi037.stdout: osd objectstore = bluestore 2024-04-25T10:30:24.924 INFO:teuthology.orchestra.run.smithi037.stdout: osd op complaint time = 180 2024-04-25T10:30:24.924 INFO:teuthology.orchestra.run.smithi037.stdout: 2024-04-25T10:30:24.924 INFO:teuthology.orchestra.run.smithi037.stdout:[mgr] 2024-04-25T10:30:24.924 INFO:teuthology.orchestra.run.smithi037.stdout: debug ms = 1 2024-04-25T10:30:24.924 INFO:teuthology.orchestra.run.smithi037.stdout: debug mgr = 20 2024-04-25T10:30:24.924 INFO:teuthology.orchestra.run.smithi037.stdout: debug mon = 20 2024-04-25T10:30:24.924 INFO:teuthology.orchestra.run.smithi037.stdout: debug auth = 20 2024-04-25T10:30:24.924 INFO:teuthology.orchestra.run.smithi037.stdout: mon reweight min pgs per osd = 4 2024-04-25T10:30:24.925 INFO:teuthology.orchestra.run.smithi037.stdout: mon reweight min bytes per osd = 10 2024-04-25T10:30:24.925 INFO:teuthology.orchestra.run.smithi037.stdout: mgr/telemetry/nag = false 2024-04-25T10:30:24.925 INFO:teuthology.orchestra.run.smithi037.stdout: debug client = 20 2024-04-25T10:30:24.925 INFO:teuthology.orchestra.run.smithi037.stdout: 2024-04-25T10:30:24.925 INFO:teuthology.orchestra.run.smithi037.stdout:[mon] 2024-04-25T10:30:24.925 INFO:teuthology.orchestra.run.smithi037.stdout: debug ms = 1 2024-04-25T10:30:24.925 INFO:teuthology.orchestra.run.smithi037.stdout: debug mon = 20 2024-04-25T10:30:24.925 INFO:teuthology.orchestra.run.smithi037.stdout: debug paxos = 20 2024-04-25T10:30:24.925 INFO:teuthology.orchestra.run.smithi037.stdout: debug auth = 20 2024-04-25T10:30:24.925 INFO:teuthology.orchestra.run.smithi037.stdout: mon data avail warn = 5 2024-04-25T10:30:24.925 INFO:teuthology.orchestra.run.smithi037.stdout: mon mgr mkfs grace = 240 2024-04-25T10:30:24.925 INFO:teuthology.orchestra.run.smithi037.stdout: mon reweight min pgs per osd = 4 2024-04-25T10:30:24.925 INFO:teuthology.orchestra.run.smithi037.stdout: mon osd reporter subtree level = osd 2024-04-25T10:30:24.925 INFO:teuthology.orchestra.run.smithi037.stdout: mon osd prime pg temp = true 2024-04-25T10:30:24.925 INFO:teuthology.orchestra.run.smithi037.stdout: mon reweight min bytes per osd = 10 2024-04-25T10:30:24.925 INFO:teuthology.orchestra.run.smithi037.stdout: 2024-04-25T10:30:24.926 INFO:teuthology.orchestra.run.smithi037.stdout: # rotate auth tickets quickly to exercise renewal paths 2024-04-25T10:30:24.926 INFO:teuthology.orchestra.run.smithi037.stdout: auth mon ticket ttl = 660 # 11m 2024-04-25T10:30:24.926 INFO:teuthology.orchestra.run.smithi037.stdout: auth service ticket ttl = 240 # 4m 2024-04-25T10:30:24.926 INFO:teuthology.orchestra.run.smithi037.stdout: 2024-04-25T10:30:24.926 INFO:teuthology.orchestra.run.smithi037.stdout: # don't complain about insecure global_id in the test suite 2024-04-25T10:30:24.926 INFO:teuthology.orchestra.run.smithi037.stdout: mon_warn_on_insecure_global_id_reclaim = false 2024-04-25T10:30:24.926 INFO:teuthology.orchestra.run.smithi037.stdout: mon_warn_on_insecure_global_id_reclaim_allowed = false 2024-04-25T10:30:24.926 INFO:teuthology.orchestra.run.smithi037.stdout: 2024-04-25T10:30:24.926 INFO:teuthology.orchestra.run.smithi037.stdout: # 1m isn't quite enough 2024-04-25T10:30:24.926 INFO:teuthology.orchestra.run.smithi037.stdout: mon_down_mkfs_grace = 2m 2024-04-25T10:30:24.926 INFO:teuthology.orchestra.run.smithi037.stdout: 2024-04-25T10:30:24.926 INFO:teuthology.orchestra.run.smithi037.stdout: mon_warn_on_filestore_osds = false 2024-04-25T10:30:24.926 INFO:teuthology.orchestra.run.smithi037.stdout: mon down mkfs grace = 300 2024-04-25T10:30:24.926 INFO:teuthology.orchestra.run.smithi037.stdout: mon op complaint time = 120 2024-04-25T10:30:24.926 INFO:teuthology.orchestra.run.smithi037.stdout: 2024-04-25T10:30:24.927 INFO:teuthology.orchestra.run.smithi037.stdout:[client] 2024-04-25T10:30:24.927 INFO:teuthology.orchestra.run.smithi037.stdout: rgw cache enabled = true 2024-04-25T10:30:24.927 INFO:teuthology.orchestra.run.smithi037.stdout: rgw enable ops log = true 2024-04-25T10:30:24.927 INFO:teuthology.orchestra.run.smithi037.stdout: rgw enable usage log = true 2024-04-25T10:30:24.927 INFO:teuthology.orchestra.run.smithi037.stdout: log file = /var/log/ceph/$cluster-$name.$pid.log 2024-04-25T10:30:24.927 INFO:teuthology.orchestra.run.smithi037.stdout: admin socket = /var/run/ceph/$cluster-$name.$pid.asok 2024-04-25T10:30:24.927 INFO:teuthology.orchestra.run.smithi037.stdout: client mount timeout = 600 2024-04-25T10:30:24.927 INFO:teuthology.orchestra.run.smithi037.stdout: debug client = 20 2024-04-25T10:30:24.927 INFO:teuthology.orchestra.run.smithi037.stdout: debug ms = 1 2024-04-25T10:30:24.927 INFO:teuthology.orchestra.run.smithi037.stdout: pid file = "" 2024-04-25T10:30:24.927 INFO:teuthology.orchestra.run.smithi037.stdout: rados mon op timeout = 900 2024-04-25T10:30:24.927 INFO:teuthology.orchestra.run.smithi037.stdout: rados osd op timeout = 900 2024-04-25T10:30:24.927 INFO:teuthology.orchestra.run.smithi037.stdout:[mon.a] 2024-04-25T10:30:24.928 INFO:teuthology.orchestra.run.smithi037.stdout:[mon.b] 2024-04-25T10:30:24.928 INFO:teuthology.orchestra.run.smithi037.stdout:[mon.c] 2024-04-25T10:30:24.928 INFO:teuthology.orchestra.run.smithi037.stdout:[mds.a] 2024-04-25T10:30:24.928 INFO:teuthology.orchestra.run.smithi037.stdout:[mds.c] 2024-04-25T10:30:24.928 INFO:teuthology.orchestra.run.smithi037.stdout:[mds.b] 2024-04-25T10:30:24.928 INFO:teuthology.orchestra.run.smithi037.stdout:[mds.d] 2024-04-25T10:30:24.928 INFO:teuthology.orchestra.run.smithi037.stdout:[mds] 2024-04-25T10:30:24.928 INFO:teuthology.orchestra.run.smithi037.stdout: debug mds = 20 2024-04-25T10:30:24.928 INFO:teuthology.orchestra.run.smithi037.stdout: debug mds balancer = 20 2024-04-25T10:30:24.928 INFO:teuthology.orchestra.run.smithi037.stdout: debug ms = 1 2024-04-25T10:30:24.928 INFO:teuthology.orchestra.run.smithi037.stdout: mds debug frag = True 2024-04-25T10:30:24.928 INFO:teuthology.orchestra.run.smithi037.stdout: mds debug scatterstat = True 2024-04-25T10:30:24.929 INFO:teuthology.orchestra.run.smithi037.stdout: mds op complaint time = 180 2024-04-25T10:30:24.929 INFO:teuthology.orchestra.run.smithi037.stdout: mds verify scatter = False 2024-04-25T10:30:24.929 INFO:teuthology.orchestra.run.smithi037.stdout: osd op complaint time = 180 2024-04-25T10:30:24.929 INFO:teuthology.orchestra.run.smithi037.stdout: rados mon op timeout = 900 2024-04-25T10:30:24.929 INFO:teuthology.orchestra.run.smithi037.stdout: rados osd op timeout = 900 2024-04-25T10:30:24.929 INFO:teuthology.orchestra.run.smithi203.stdout:[global] 2024-04-25T10:30:24.929 INFO:teuthology.orchestra.run.smithi203.stdout: chdir = "" 2024-04-25T10:30:24.929 INFO:teuthology.orchestra.run.smithi203.stdout: pid file = /var/run/ceph/$cluster-$name.pid 2024-04-25T10:30:24.930 INFO:teuthology.orchestra.run.smithi203.stdout: auth supported = cephx 2024-04-25T10:30:24.930 INFO:teuthology.orchestra.run.smithi203.stdout: 2024-04-25T10:30:24.930 INFO:teuthology.orchestra.run.smithi203.stdout: filestore xattr use omap = true 2024-04-25T10:30:24.930 INFO:teuthology.orchestra.run.smithi203.stdout: 2024-04-25T10:30:24.930 INFO:teuthology.orchestra.run.smithi203.stdout: mon clock drift allowed = 1.000 2024-04-25T10:30:24.930 INFO:teuthology.orchestra.run.smithi203.stdout: 2024-04-25T10:30:24.930 INFO:teuthology.orchestra.run.smithi203.stdout: osd crush chooseleaf type = 0 2024-04-25T10:30:24.930 INFO:teuthology.orchestra.run.smithi203.stdout: auth debug = true 2024-04-25T10:30:24.930 INFO:teuthology.orchestra.run.smithi203.stdout: 2024-04-25T10:30:24.930 INFO:teuthology.orchestra.run.smithi203.stdout: ms die on old message = true 2024-04-25T10:30:24.930 INFO:teuthology.orchestra.run.smithi203.stdout: ms die on bug = true 2024-04-25T10:30:24.930 INFO:teuthology.orchestra.run.smithi203.stdout: 2024-04-25T10:30:24.930 INFO:teuthology.orchestra.run.smithi203.stdout: mon max pg per osd = 10000 # >= luminous 2024-04-25T10:30:24.930 INFO:teuthology.orchestra.run.smithi203.stdout: mon pg warn max object skew = 0 2024-04-25T10:30:24.931 INFO:teuthology.orchestra.run.smithi203.stdout: 2024-04-25T10:30:24.931 INFO:teuthology.orchestra.run.smithi203.stdout: # disable pg_autoscaler by default for new pools 2024-04-25T10:30:24.931 INFO:teuthology.orchestra.run.smithi203.stdout: osd_pool_default_pg_autoscale_mode = off 2024-04-25T10:30:24.931 INFO:teuthology.orchestra.run.smithi203.stdout: 2024-04-25T10:30:24.931 INFO:teuthology.orchestra.run.smithi203.stdout: osd pool default size = 2 2024-04-25T10:30:24.931 INFO:teuthology.orchestra.run.smithi203.stdout: 2024-04-25T10:30:24.931 INFO:teuthology.orchestra.run.smithi203.stdout: mon osd allow primary affinity = true 2024-04-25T10:30:24.931 INFO:teuthology.orchestra.run.smithi203.stdout: mon osd allow pg remap = true 2024-04-25T10:30:24.931 INFO:teuthology.orchestra.run.smithi203.stdout: mon warn on legacy crush tunables = false 2024-04-25T10:30:24.931 INFO:teuthology.orchestra.run.smithi203.stdout: mon warn on crush straw calc version zero = false 2024-04-25T10:30:24.931 INFO:teuthology.orchestra.run.smithi203.stdout: mon warn on no sortbitwise = false 2024-04-25T10:30:24.931 INFO:teuthology.orchestra.run.smithi203.stdout: mon warn on osd down out interval zero = false 2024-04-25T10:30:24.931 INFO:teuthology.orchestra.run.smithi203.stdout: mon warn on too few osds = false 2024-04-25T10:30:24.931 INFO:teuthology.orchestra.run.smithi203.stdout: mon_warn_on_pool_pg_num_not_power_of_two = false 2024-04-25T10:30:24.932 INFO:teuthology.orchestra.run.smithi203.stdout: mon_warn_on_pool_no_redundancy = false 2024-04-25T10:30:24.932 INFO:teuthology.orchestra.run.smithi203.stdout: mon_allow_pool_size_one = true 2024-04-25T10:30:24.932 INFO:teuthology.orchestra.run.smithi203.stdout: 2024-04-25T10:30:24.932 INFO:teuthology.orchestra.run.smithi203.stdout: osd pool default erasure code profile = plugin=jerasure technique=reed_sol_van k=2 m=1 crush-failure-domain=osd 2024-04-25T10:30:24.932 INFO:teuthology.orchestra.run.smithi203.stdout: 2024-04-25T10:30:24.932 INFO:teuthology.orchestra.run.smithi203.stdout: osd default data pool replay window = 5 2024-04-25T10:30:24.932 INFO:teuthology.orchestra.run.smithi203.stdout: 2024-04-25T10:30:24.932 INFO:teuthology.orchestra.run.smithi203.stdout: mon allow pool delete = true 2024-04-25T10:30:24.932 INFO:teuthology.orchestra.run.smithi203.stdout: 2024-04-25T10:30:24.932 INFO:teuthology.orchestra.run.smithi203.stdout: mon cluster log file level = debug 2024-04-25T10:30:24.932 INFO:teuthology.orchestra.run.smithi203.stdout: debug asserts on shutdown = true 2024-04-25T10:30:24.932 INFO:teuthology.orchestra.run.smithi203.stdout: mon health detail to clog = false 2024-04-25T10:30:24.932 INFO:teuthology.orchestra.run.smithi203.stdout: mon host = "172.21.15.37,172.21.15.203,[v2:172.21.15.203:3301,v1:172.21.15.203:6790]" 2024-04-25T10:30:24.932 INFO:teuthology.orchestra.run.smithi203.stdout: fsid = f4acda28-be34-492a-b286-994afd36f61d 2024-04-25T10:30:24.933 INFO:teuthology.orchestra.run.smithi203.stdout: 2024-04-25T10:30:24.933 INFO:teuthology.orchestra.run.smithi203.stdout:[osd] 2024-04-25T10:30:24.933 INFO:teuthology.orchestra.run.smithi203.stdout: osd journal size = 100 2024-04-25T10:30:24.933 INFO:teuthology.orchestra.run.smithi203.stdout: 2024-04-25T10:30:24.933 INFO:teuthology.orchestra.run.smithi203.stdout: osd scrub load threshold = 5.0 2024-04-25T10:30:24.933 INFO:teuthology.orchestra.run.smithi203.stdout: osd scrub max interval = 600 2024-04-25T10:30:24.933 INFO:teuthology.orchestra.run.smithi203.stdout: osd mclock profile = high_recovery_ops 2024-04-25T10:30:24.933 INFO:teuthology.orchestra.run.smithi203.stdout: 2024-04-25T10:30:24.933 INFO:teuthology.orchestra.run.smithi203.stdout: osd recover clone overlap = true 2024-04-25T10:30:24.933 INFO:teuthology.orchestra.run.smithi203.stdout: osd recovery max chunk = 1048576 2024-04-25T10:30:24.933 INFO:teuthology.orchestra.run.smithi203.stdout: 2024-04-25T10:30:24.933 INFO:teuthology.orchestra.run.smithi203.stdout: osd debug shutdown = true 2024-04-25T10:30:24.933 INFO:teuthology.orchestra.run.smithi203.stdout: osd debug op order = true 2024-04-25T10:30:24.933 INFO:teuthology.orchestra.run.smithi203.stdout: osd debug verify stray on activate = true 2024-04-25T10:30:24.933 INFO:teuthology.orchestra.run.smithi203.stdout: osd debug trim objects = true 2024-04-25T10:30:24.934 INFO:teuthology.orchestra.run.smithi203.stdout: 2024-04-25T10:30:24.934 INFO:teuthology.orchestra.run.smithi203.stdout: osd open classes on start = true 2024-04-25T10:30:24.934 INFO:teuthology.orchestra.run.smithi203.stdout: osd debug pg log writeout = true 2024-04-25T10:30:24.934 INFO:teuthology.orchestra.run.smithi203.stdout: 2024-04-25T10:30:24.934 INFO:teuthology.orchestra.run.smithi203.stdout: osd deep scrub update digest min age = 30 2024-04-25T10:30:24.934 INFO:teuthology.orchestra.run.smithi203.stdout: 2024-04-25T10:30:24.934 INFO:teuthology.orchestra.run.smithi203.stdout: osd map max advance = 10 2024-04-25T10:30:24.934 INFO:teuthology.orchestra.run.smithi203.stdout: 2024-04-25T10:30:24.934 INFO:teuthology.orchestra.run.smithi203.stdout: journal zero on create = true 2024-04-25T10:30:24.934 INFO:teuthology.orchestra.run.smithi203.stdout: 2024-04-25T10:30:24.934 INFO:teuthology.orchestra.run.smithi203.stdout: filestore ondisk finisher threads = 3 2024-04-25T10:30:24.934 INFO:teuthology.orchestra.run.smithi203.stdout: filestore apply finisher threads = 3 2024-04-25T10:30:24.934 INFO:teuthology.orchestra.run.smithi203.stdout: 2024-04-25T10:30:24.934 INFO:teuthology.orchestra.run.smithi203.stdout: bdev debug aio = true 2024-04-25T10:30:24.935 INFO:teuthology.orchestra.run.smithi203.stdout: osd debug misdirected ops = true 2024-04-25T10:30:24.935 INFO:teuthology.orchestra.run.smithi203.stdout: bdev async discard = True 2024-04-25T10:30:24.935 INFO:teuthology.orchestra.run.smithi203.stdout: bdev enable discard = True 2024-04-25T10:30:24.935 INFO:teuthology.orchestra.run.smithi203.stdout: bluestore allocator = bitmap 2024-04-25T10:30:24.935 INFO:teuthology.orchestra.run.smithi203.stdout: bluestore block size = 96636764160 2024-04-25T10:30:24.935 INFO:teuthology.orchestra.run.smithi203.stdout: bluestore fsck on mount = True 2024-04-25T10:30:24.935 INFO:teuthology.orchestra.run.smithi203.stdout: debug bluefs = 1/20 2024-04-25T10:30:24.935 INFO:teuthology.orchestra.run.smithi203.stdout: debug bluestore = 1/20 2024-04-25T10:30:24.935 INFO:teuthology.orchestra.run.smithi203.stdout: debug ms = 1 2024-04-25T10:30:24.935 INFO:teuthology.orchestra.run.smithi203.stdout: debug osd = 20 2024-04-25T10:30:24.936 INFO:teuthology.orchestra.run.smithi203.stdout: debug rocksdb = 4/10 2024-04-25T10:30:24.936 INFO:teuthology.orchestra.run.smithi203.stdout: mon osd backfillfull_ratio = 0.85 2024-04-25T10:30:24.936 INFO:teuthology.orchestra.run.smithi203.stdout: mon osd full ratio = 0.9 2024-04-25T10:30:24.936 INFO:teuthology.orchestra.run.smithi203.stdout: mon osd nearfull ratio = 0.8 2024-04-25T10:30:24.936 INFO:teuthology.orchestra.run.smithi203.stdout: osd failsafe full ratio = 0.95 2024-04-25T10:30:24.936 INFO:teuthology.orchestra.run.smithi203.stdout: osd objectstore = bluestore 2024-04-25T10:30:24.936 INFO:teuthology.orchestra.run.smithi203.stdout: osd op complaint time = 180 2024-04-25T10:30:24.936 INFO:teuthology.orchestra.run.smithi203.stdout: 2024-04-25T10:30:24.936 INFO:teuthology.orchestra.run.smithi203.stdout:[mgr] 2024-04-25T10:30:24.936 INFO:teuthology.orchestra.run.smithi203.stdout: debug ms = 1 2024-04-25T10:30:24.936 INFO:teuthology.orchestra.run.smithi203.stdout: debug mgr = 20 2024-04-25T10:30:24.936 INFO:teuthology.orchestra.run.smithi203.stdout: debug mon = 20 2024-04-25T10:30:24.937 INFO:teuthology.orchestra.run.smithi203.stdout: debug auth = 20 2024-04-25T10:30:24.937 INFO:teuthology.orchestra.run.smithi203.stdout: mon reweight min pgs per osd = 4 2024-04-25T10:30:24.937 INFO:teuthology.orchestra.run.smithi203.stdout: mon reweight min bytes per osd = 10 2024-04-25T10:30:24.937 INFO:teuthology.orchestra.run.smithi203.stdout: mgr/telemetry/nag = false 2024-04-25T10:30:24.937 INFO:teuthology.orchestra.run.smithi203.stdout: debug client = 20 2024-04-25T10:30:24.937 INFO:teuthology.orchestra.run.smithi203.stdout: 2024-04-25T10:30:24.937 INFO:teuthology.orchestra.run.smithi203.stdout:[mon] 2024-04-25T10:30:24.937 INFO:teuthology.orchestra.run.smithi203.stdout: debug ms = 1 2024-04-25T10:30:24.937 INFO:teuthology.orchestra.run.smithi203.stdout: debug mon = 20 2024-04-25T10:30:24.937 INFO:teuthology.orchestra.run.smithi203.stdout: debug paxos = 20 2024-04-25T10:30:24.937 INFO:teuthology.orchestra.run.smithi203.stdout: debug auth = 20 2024-04-25T10:30:24.937 INFO:teuthology.orchestra.run.smithi203.stdout: mon data avail warn = 5 2024-04-25T10:30:24.938 INFO:teuthology.orchestra.run.smithi203.stdout: mon mgr mkfs grace = 240 2024-04-25T10:30:24.938 INFO:teuthology.orchestra.run.smithi203.stdout: mon reweight min pgs per osd = 4 2024-04-25T10:30:24.938 INFO:teuthology.orchestra.run.smithi203.stdout: mon osd reporter subtree level = osd 2024-04-25T10:30:24.938 INFO:teuthology.orchestra.run.smithi203.stdout: mon osd prime pg temp = true 2024-04-25T10:30:24.938 INFO:teuthology.orchestra.run.smithi203.stdout: mon reweight min bytes per osd = 10 2024-04-25T10:30:24.938 INFO:teuthology.orchestra.run.smithi203.stdout: 2024-04-25T10:30:24.938 INFO:teuthology.orchestra.run.smithi203.stdout: # rotate auth tickets quickly to exercise renewal paths 2024-04-25T10:30:24.938 INFO:teuthology.orchestra.run.smithi203.stdout: auth mon ticket ttl = 660 # 11m 2024-04-25T10:30:24.938 INFO:teuthology.orchestra.run.smithi203.stdout: auth service ticket ttl = 240 # 4m 2024-04-25T10:30:24.938 INFO:teuthology.orchestra.run.smithi203.stdout: 2024-04-25T10:30:24.938 INFO:teuthology.orchestra.run.smithi203.stdout: # don't complain about insecure global_id in the test suite 2024-04-25T10:30:24.938 INFO:teuthology.orchestra.run.smithi203.stdout: mon_warn_on_insecure_global_id_reclaim = false 2024-04-25T10:30:24.938 INFO:teuthology.orchestra.run.smithi203.stdout: mon_warn_on_insecure_global_id_reclaim_allowed = false 2024-04-25T10:30:24.939 INFO:teuthology.orchestra.run.smithi203.stdout: 2024-04-25T10:30:24.939 INFO:teuthology.orchestra.run.smithi203.stdout: # 1m isn't quite enough 2024-04-25T10:30:24.939 INFO:teuthology.orchestra.run.smithi203.stdout: mon_down_mkfs_grace = 2m 2024-04-25T10:30:24.939 INFO:teuthology.orchestra.run.smithi203.stdout: 2024-04-25T10:30:24.939 INFO:teuthology.orchestra.run.smithi203.stdout: mon_warn_on_filestore_osds = false 2024-04-25T10:30:24.939 INFO:teuthology.orchestra.run.smithi203.stdout: mon down mkfs grace = 300 2024-04-25T10:30:24.939 INFO:teuthology.orchestra.run.smithi203.stdout: mon op complaint time = 120 2024-04-25T10:30:24.939 INFO:teuthology.orchestra.run.smithi203.stdout: 2024-04-25T10:30:24.939 INFO:teuthology.orchestra.run.smithi203.stdout:[client] 2024-04-25T10:30:24.939 INFO:teuthology.orchestra.run.smithi203.stdout: rgw cache enabled = true 2024-04-25T10:30:24.939 INFO:teuthology.orchestra.run.smithi203.stdout: rgw enable ops log = true 2024-04-25T10:30:24.939 INFO:teuthology.orchestra.run.smithi203.stdout: rgw enable usage log = true 2024-04-25T10:30:24.939 INFO:teuthology.orchestra.run.smithi203.stdout: log file = /var/log/ceph/$cluster-$name.$pid.log 2024-04-25T10:30:24.939 INFO:teuthology.orchestra.run.smithi203.stdout: admin socket = /var/run/ceph/$cluster-$name.$pid.asok 2024-04-25T10:30:24.939 INFO:teuthology.orchestra.run.smithi203.stdout: client mount timeout = 600 2024-04-25T10:30:24.940 INFO:teuthology.orchestra.run.smithi203.stdout: debug client = 20 2024-04-25T10:30:24.940 INFO:teuthology.orchestra.run.smithi203.stdout: debug ms = 1 2024-04-25T10:30:24.940 INFO:teuthology.orchestra.run.smithi203.stdout: pid file = "" 2024-04-25T10:30:24.940 INFO:teuthology.orchestra.run.smithi203.stdout: rados mon op timeout = 900 2024-04-25T10:30:24.940 INFO:teuthology.orchestra.run.smithi203.stdout: rados osd op timeout = 900 2024-04-25T10:30:24.940 INFO:teuthology.orchestra.run.smithi203.stdout:[mon.a] 2024-04-25T10:30:24.940 INFO:teuthology.orchestra.run.smithi203.stdout:[mon.b] 2024-04-25T10:30:24.940 INFO:teuthology.orchestra.run.smithi203.stdout:[mon.c] 2024-04-25T10:30:24.940 INFO:teuthology.orchestra.run.smithi203.stdout:[mds.a] 2024-04-25T10:30:24.940 INFO:teuthology.orchestra.run.smithi203.stdout:[mds.c] 2024-04-25T10:30:24.940 INFO:teuthology.orchestra.run.smithi203.stdout:[mds.b] 2024-04-25T10:30:24.940 INFO:teuthology.orchestra.run.smithi203.stdout:[mds.d] 2024-04-25T10:30:24.940 INFO:teuthology.orchestra.run.smithi203.stdout:[mds] 2024-04-25T10:30:24.941 INFO:teuthology.orchestra.run.smithi203.stdout: debug mds = 20 2024-04-25T10:30:24.941 INFO:teuthology.orchestra.run.smithi203.stdout: debug mds balancer = 20 2024-04-25T10:30:24.941 INFO:teuthology.orchestra.run.smithi203.stdout: debug ms = 1 2024-04-25T10:30:24.941 INFO:teuthology.orchestra.run.smithi203.stdout: mds debug frag = True 2024-04-25T10:30:24.941 INFO:teuthology.orchestra.run.smithi203.stdout: mds debug scatterstat = True 2024-04-25T10:30:24.941 INFO:teuthology.orchestra.run.smithi203.stdout: mds op complaint time = 180 2024-04-25T10:30:24.941 INFO:teuthology.orchestra.run.smithi203.stdout: mds verify scatter = False 2024-04-25T10:30:24.941 INFO:teuthology.orchestra.run.smithi203.stdout: osd op complaint time = 180 2024-04-25T10:30:24.941 INFO:teuthology.orchestra.run.smithi203.stdout: rados mon op timeout = 900 2024-04-25T10:30:24.941 INFO:teuthology.orchestra.run.smithi203.stdout: rados osd op timeout = 900 2024-04-25T10:30:24.943 DEBUG:teuthology.orchestra.run.smithi037:> sudo mkdir -p /etc/ceph && sudo chmod 0755 /etc/ceph && sudo tee /etc/ceph/ceph.conf && sudo chmod 0644 /etc/ceph/ceph.conf > /dev/null 2024-04-25T10:30:24.985 DEBUG:teuthology.orchestra.run.smithi203:> sudo mkdir -p /etc/ceph && sudo chmod 0755 /etc/ceph && sudo tee /etc/ceph/ceph.conf && sudo chmod 0644 /etc/ceph/ceph.conf > /dev/null 2024-04-25T10:30:25.024 INFO:teuthology.orchestra.run.smithi037.stdout:[global] 2024-04-25T10:30:25.024 INFO:teuthology.orchestra.run.smithi037.stdout: chdir = "" 2024-04-25T10:30:25.024 INFO:teuthology.orchestra.run.smithi037.stdout: pid file = /var/run/ceph/$cluster-$name.pid 2024-04-25T10:30:25.024 INFO:teuthology.orchestra.run.smithi037.stdout: auth supported = cephx 2024-04-25T10:30:25.025 INFO:teuthology.orchestra.run.smithi037.stdout: 2024-04-25T10:30:25.025 INFO:teuthology.orchestra.run.smithi037.stdout: filestore xattr use omap = true 2024-04-25T10:30:25.025 INFO:teuthology.orchestra.run.smithi037.stdout: 2024-04-25T10:30:25.025 INFO:teuthology.orchestra.run.smithi037.stdout: mon clock drift allowed = 1.000 2024-04-25T10:30:25.025 INFO:teuthology.orchestra.run.smithi037.stdout: 2024-04-25T10:30:25.025 INFO:teuthology.orchestra.run.smithi037.stdout: osd crush chooseleaf type = 0 2024-04-25T10:30:25.025 INFO:teuthology.orchestra.run.smithi037.stdout: auth debug = true 2024-04-25T10:30:25.025 INFO:teuthology.orchestra.run.smithi037.stdout: 2024-04-25T10:30:25.025 INFO:teuthology.orchestra.run.smithi037.stdout: ms die on old message = true 2024-04-25T10:30:25.025 INFO:teuthology.orchestra.run.smithi037.stdout: ms die on bug = true 2024-04-25T10:30:25.025 INFO:teuthology.orchestra.run.smithi037.stdout: 2024-04-25T10:30:25.025 INFO:teuthology.orchestra.run.smithi037.stdout: mon max pg per osd = 10000 # >= luminous 2024-04-25T10:30:25.025 INFO:teuthology.orchestra.run.smithi037.stdout: mon pg warn max object skew = 0 2024-04-25T10:30:25.025 INFO:teuthology.orchestra.run.smithi037.stdout: 2024-04-25T10:30:25.025 INFO:teuthology.orchestra.run.smithi037.stdout: # disable pg_autoscaler by default for new pools 2024-04-25T10:30:25.025 INFO:teuthology.orchestra.run.smithi037.stdout: osd_pool_default_pg_autoscale_mode = off 2024-04-25T10:30:25.026 INFO:teuthology.orchestra.run.smithi037.stdout: 2024-04-25T10:30:25.026 INFO:teuthology.orchestra.run.smithi037.stdout: osd pool default size = 2 2024-04-25T10:30:25.026 INFO:teuthology.orchestra.run.smithi037.stdout: 2024-04-25T10:30:25.026 INFO:teuthology.orchestra.run.smithi037.stdout: mon osd allow primary affinity = true 2024-04-25T10:30:25.026 INFO:teuthology.orchestra.run.smithi037.stdout: mon osd allow pg remap = true 2024-04-25T10:30:25.026 INFO:teuthology.orchestra.run.smithi037.stdout: mon warn on legacy crush tunables = false 2024-04-25T10:30:25.026 INFO:teuthology.orchestra.run.smithi037.stdout: mon warn on crush straw calc version zero = false 2024-04-25T10:30:25.026 INFO:teuthology.orchestra.run.smithi037.stdout: mon warn on no sortbitwise = false 2024-04-25T10:30:25.026 INFO:teuthology.orchestra.run.smithi037.stdout: mon warn on osd down out interval zero = false 2024-04-25T10:30:25.026 INFO:teuthology.orchestra.run.smithi037.stdout: mon warn on too few osds = false 2024-04-25T10:30:25.026 INFO:teuthology.orchestra.run.smithi037.stdout: mon_warn_on_pool_pg_num_not_power_of_two = false 2024-04-25T10:30:25.026 INFO:teuthology.orchestra.run.smithi037.stdout: mon_warn_on_pool_no_redundancy = false 2024-04-25T10:30:25.026 INFO:teuthology.orchestra.run.smithi037.stdout: mon_allow_pool_size_one = true 2024-04-25T10:30:25.026 INFO:teuthology.orchestra.run.smithi037.stdout: 2024-04-25T10:30:25.027 INFO:teuthology.orchestra.run.smithi037.stdout: osd pool default erasure code profile = plugin=jerasure technique=reed_sol_van k=2 m=1 crush-failure-domain=osd 2024-04-25T10:30:25.027 INFO:teuthology.orchestra.run.smithi037.stdout: 2024-04-25T10:30:25.027 INFO:teuthology.orchestra.run.smithi037.stdout: osd default data pool replay window = 5 2024-04-25T10:30:25.027 INFO:teuthology.orchestra.run.smithi037.stdout: 2024-04-25T10:30:25.027 INFO:teuthology.orchestra.run.smithi037.stdout: mon allow pool delete = true 2024-04-25T10:30:25.027 INFO:teuthology.orchestra.run.smithi037.stdout: 2024-04-25T10:30:25.027 INFO:teuthology.orchestra.run.smithi037.stdout: mon cluster log file level = debug 2024-04-25T10:30:25.027 INFO:teuthology.orchestra.run.smithi037.stdout: debug asserts on shutdown = true 2024-04-25T10:30:25.027 INFO:teuthology.orchestra.run.smithi037.stdout: mon health detail to clog = false 2024-04-25T10:30:25.027 INFO:teuthology.orchestra.run.smithi037.stdout: mon host = "172.21.15.37,172.21.15.203,[v2:172.21.15.203:3301,v1:172.21.15.203:6790]" 2024-04-25T10:30:25.027 INFO:teuthology.orchestra.run.smithi037.stdout: fsid = f4acda28-be34-492a-b286-994afd36f61d 2024-04-25T10:30:25.027 INFO:teuthology.orchestra.run.smithi037.stdout: 2024-04-25T10:30:25.027 INFO:teuthology.orchestra.run.smithi037.stdout:[osd] 2024-04-25T10:30:25.028 INFO:teuthology.orchestra.run.smithi037.stdout: osd journal size = 100 2024-04-25T10:30:25.028 INFO:teuthology.orchestra.run.smithi037.stdout: 2024-04-25T10:30:25.028 INFO:teuthology.orchestra.run.smithi037.stdout: osd scrub load threshold = 5.0 2024-04-25T10:30:25.028 INFO:teuthology.orchestra.run.smithi037.stdout: osd scrub max interval = 600 2024-04-25T10:30:25.028 INFO:teuthology.orchestra.run.smithi037.stdout: osd mclock profile = high_recovery_ops 2024-04-25T10:30:25.028 INFO:teuthology.orchestra.run.smithi037.stdout: 2024-04-25T10:30:25.028 INFO:teuthology.orchestra.run.smithi037.stdout: osd recover clone overlap = true 2024-04-25T10:30:25.028 INFO:teuthology.orchestra.run.smithi037.stdout: osd recovery max chunk = 1048576 2024-04-25T10:30:25.028 INFO:teuthology.orchestra.run.smithi037.stdout: 2024-04-25T10:30:25.028 INFO:teuthology.orchestra.run.smithi037.stdout: osd debug shutdown = true 2024-04-25T10:30:25.029 INFO:teuthology.orchestra.run.smithi037.stdout: osd debug op order = true 2024-04-25T10:30:25.029 INFO:teuthology.orchestra.run.smithi037.stdout: osd debug verify stray on activate = true 2024-04-25T10:30:25.029 INFO:teuthology.orchestra.run.smithi037.stdout: osd debug trim objects = true 2024-04-25T10:30:25.029 INFO:teuthology.orchestra.run.smithi037.stdout: 2024-04-25T10:30:25.029 INFO:teuthology.orchestra.run.smithi037.stdout: osd open classes on start = true 2024-04-25T10:30:25.029 INFO:teuthology.orchestra.run.smithi037.stdout: osd debug pg log writeout = true 2024-04-25T10:30:25.029 INFO:teuthology.orchestra.run.smithi037.stdout: 2024-04-25T10:30:25.029 INFO:teuthology.orchestra.run.smithi037.stdout: osd deep scrub update digest min age = 30 2024-04-25T10:30:25.029 INFO:teuthology.orchestra.run.smithi037.stdout: 2024-04-25T10:30:25.029 INFO:teuthology.orchestra.run.smithi037.stdout: osd map max advance = 10 2024-04-25T10:30:25.029 INFO:teuthology.orchestra.run.smithi037.stdout: 2024-04-25T10:30:25.029 INFO:teuthology.orchestra.run.smithi037.stdout: journal zero on create = true 2024-04-25T10:30:25.029 INFO:teuthology.orchestra.run.smithi037.stdout: 2024-04-25T10:30:25.029 INFO:teuthology.orchestra.run.smithi037.stdout: filestore ondisk finisher threads = 3 2024-04-25T10:30:25.030 INFO:teuthology.orchestra.run.smithi037.stdout: filestore apply finisher threads = 3 2024-04-25T10:30:25.030 INFO:teuthology.orchestra.run.smithi037.stdout: 2024-04-25T10:30:25.030 INFO:teuthology.orchestra.run.smithi037.stdout: bdev debug aio = true 2024-04-25T10:30:25.030 INFO:teuthology.orchestra.run.smithi037.stdout: osd debug misdirected ops = true 2024-04-25T10:30:25.030 INFO:teuthology.orchestra.run.smithi037.stdout: bdev async discard = True 2024-04-25T10:30:25.030 INFO:teuthology.orchestra.run.smithi037.stdout: bdev enable discard = True 2024-04-25T10:30:25.030 INFO:teuthology.orchestra.run.smithi037.stdout: bluestore allocator = bitmap 2024-04-25T10:30:25.030 INFO:teuthology.orchestra.run.smithi037.stdout: bluestore block size = 96636764160 2024-04-25T10:30:25.030 INFO:teuthology.orchestra.run.smithi037.stdout: bluestore fsck on mount = True 2024-04-25T10:30:25.030 INFO:teuthology.orchestra.run.smithi037.stdout: debug bluefs = 1/20 2024-04-25T10:30:25.030 INFO:teuthology.orchestra.run.smithi037.stdout: debug bluestore = 1/20 2024-04-25T10:30:25.030 INFO:teuthology.orchestra.run.smithi037.stdout: debug ms = 1 2024-04-25T10:30:25.030 INFO:teuthology.orchestra.run.smithi037.stdout: debug osd = 20 2024-04-25T10:30:25.030 INFO:teuthology.orchestra.run.smithi037.stdout: debug rocksdb = 4/10 2024-04-25T10:30:25.031 INFO:teuthology.orchestra.run.smithi037.stdout: mon osd backfillfull_ratio = 0.85 2024-04-25T10:30:25.031 INFO:teuthology.orchestra.run.smithi037.stdout: mon osd full ratio = 0.9 2024-04-25T10:30:25.031 INFO:teuthology.orchestra.run.smithi037.stdout: mon osd nearfull ratio = 0.8 2024-04-25T10:30:25.031 INFO:teuthology.orchestra.run.smithi037.stdout: osd failsafe full ratio = 0.95 2024-04-25T10:30:25.031 INFO:teuthology.orchestra.run.smithi037.stdout: osd objectstore = bluestore 2024-04-25T10:30:25.031 INFO:teuthology.orchestra.run.smithi037.stdout: osd op complaint time = 180 2024-04-25T10:30:25.031 INFO:teuthology.orchestra.run.smithi037.stdout: 2024-04-25T10:30:25.031 INFO:teuthology.orchestra.run.smithi037.stdout:[mgr] 2024-04-25T10:30:25.031 INFO:teuthology.orchestra.run.smithi037.stdout: debug ms = 1 2024-04-25T10:30:25.031 INFO:teuthology.orchestra.run.smithi037.stdout: debug mgr = 20 2024-04-25T10:30:25.031 INFO:teuthology.orchestra.run.smithi037.stdout: debug mon = 20 2024-04-25T10:30:25.031 INFO:teuthology.orchestra.run.smithi037.stdout: debug auth = 20 2024-04-25T10:30:25.031 INFO:teuthology.orchestra.run.smithi037.stdout: mon reweight min pgs per osd = 4 2024-04-25T10:30:25.031 INFO:teuthology.orchestra.run.smithi037.stdout: mon reweight min bytes per osd = 10 2024-04-25T10:30:25.031 INFO:teuthology.orchestra.run.smithi037.stdout: mgr/telemetry/nag = false 2024-04-25T10:30:25.031 INFO:teuthology.orchestra.run.smithi037.stdout: debug client = 20 2024-04-25T10:30:25.032 INFO:teuthology.orchestra.run.smithi037.stdout: 2024-04-25T10:30:25.032 INFO:teuthology.orchestra.run.smithi037.stdout:[mon] 2024-04-25T10:30:25.032 INFO:teuthology.orchestra.run.smithi037.stdout: debug ms = 1 2024-04-25T10:30:25.032 INFO:teuthology.orchestra.run.smithi037.stdout: debug mon = 20 2024-04-25T10:30:25.032 INFO:teuthology.orchestra.run.smithi037.stdout: debug paxos = 20 2024-04-25T10:30:25.032 INFO:teuthology.orchestra.run.smithi037.stdout: debug auth = 20 2024-04-25T10:30:25.032 INFO:teuthology.orchestra.run.smithi037.stdout: mon data avail warn = 5 2024-04-25T10:30:25.032 INFO:teuthology.orchestra.run.smithi037.stdout: mon mgr mkfs grace = 240 2024-04-25T10:30:25.032 INFO:teuthology.orchestra.run.smithi037.stdout: mon reweight min pgs per osd = 4 2024-04-25T10:30:25.032 INFO:teuthology.orchestra.run.smithi037.stdout: mon osd reporter subtree level = osd 2024-04-25T10:30:25.032 INFO:teuthology.orchestra.run.smithi037.stdout: mon osd prime pg temp = true 2024-04-25T10:30:25.032 INFO:teuthology.orchestra.run.smithi037.stdout: mon reweight min bytes per osd = 10 2024-04-25T10:30:25.032 INFO:teuthology.orchestra.run.smithi037.stdout: 2024-04-25T10:30:25.032 INFO:teuthology.orchestra.run.smithi037.stdout: # rotate auth tickets quickly to exercise renewal paths 2024-04-25T10:30:25.032 INFO:teuthology.orchestra.run.smithi037.stdout: auth mon ticket ttl = 660 # 11m 2024-04-25T10:30:25.033 INFO:teuthology.orchestra.run.smithi037.stdout: auth service ticket ttl = 240 # 4m 2024-04-25T10:30:25.033 INFO:teuthology.orchestra.run.smithi037.stdout: 2024-04-25T10:30:25.033 INFO:teuthology.orchestra.run.smithi037.stdout: # don't complain about insecure global_id in the test suite 2024-04-25T10:30:25.033 INFO:teuthology.orchestra.run.smithi037.stdout: mon_warn_on_insecure_global_id_reclaim = false 2024-04-25T10:30:25.033 INFO:teuthology.orchestra.run.smithi037.stdout: mon_warn_on_insecure_global_id_reclaim_allowed = false 2024-04-25T10:30:25.033 INFO:teuthology.orchestra.run.smithi037.stdout: 2024-04-25T10:30:25.033 INFO:teuthology.orchestra.run.smithi037.stdout: # 1m isn't quite enough 2024-04-25T10:30:25.033 INFO:teuthology.orchestra.run.smithi037.stdout: mon_down_mkfs_grace = 2m 2024-04-25T10:30:25.033 INFO:teuthology.orchestra.run.smithi037.stdout: 2024-04-25T10:30:25.033 INFO:teuthology.orchestra.run.smithi037.stdout: mon_warn_on_filestore_osds = false 2024-04-25T10:30:25.033 INFO:teuthology.orchestra.run.smithi037.stdout: mon down mkfs grace = 300 2024-04-25T10:30:25.033 INFO:teuthology.orchestra.run.smithi037.stdout: mon op complaint time = 120 2024-04-25T10:30:25.033 INFO:teuthology.orchestra.run.smithi037.stdout: 2024-04-25T10:30:25.033 INFO:teuthology.orchestra.run.smithi037.stdout:[client] 2024-04-25T10:30:25.034 INFO:teuthology.orchestra.run.smithi037.stdout: rgw cache enabled = true 2024-04-25T10:30:25.034 INFO:teuthology.orchestra.run.smithi037.stdout: rgw enable ops log = true 2024-04-25T10:30:25.034 INFO:teuthology.orchestra.run.smithi037.stdout: rgw enable usage log = true 2024-04-25T10:30:25.034 INFO:teuthology.orchestra.run.smithi037.stdout: log file = /var/log/ceph/$cluster-$name.$pid.log 2024-04-25T10:30:25.034 INFO:teuthology.orchestra.run.smithi037.stdout: admin socket = /var/run/ceph/$cluster-$name.$pid.asok 2024-04-25T10:30:25.034 INFO:teuthology.orchestra.run.smithi037.stdout: client mount timeout = 600 2024-04-25T10:30:25.034 INFO:teuthology.orchestra.run.smithi037.stdout: debug client = 20 2024-04-25T10:30:25.034 INFO:teuthology.orchestra.run.smithi037.stdout: debug ms = 1 2024-04-25T10:30:25.034 INFO:teuthology.orchestra.run.smithi037.stdout: pid file = "" 2024-04-25T10:30:25.034 INFO:teuthology.orchestra.run.smithi037.stdout: rados mon op timeout = 900 2024-04-25T10:30:25.034 INFO:teuthology.orchestra.run.smithi037.stdout: rados osd op timeout = 900 2024-04-25T10:30:25.034 INFO:teuthology.orchestra.run.smithi037.stdout:[mon.a] 2024-04-25T10:30:25.035 INFO:teuthology.orchestra.run.smithi037.stdout:[mon.b] 2024-04-25T10:30:25.035 INFO:teuthology.orchestra.run.smithi037.stdout:[mon.c] 2024-04-25T10:30:25.036 INFO:teuthology.orchestra.run.smithi037.stdout:[mds.a] 2024-04-25T10:30:25.036 INFO:teuthology.orchestra.run.smithi037.stdout:[mds.c] 2024-04-25T10:30:25.036 INFO:teuthology.orchestra.run.smithi037.stdout:[mds.b] 2024-04-25T10:30:25.036 INFO:teuthology.orchestra.run.smithi037.stdout:[mds.d] 2024-04-25T10:30:25.036 INFO:teuthology.orchestra.run.smithi037.stdout:[mds] 2024-04-25T10:30:25.036 INFO:teuthology.orchestra.run.smithi037.stdout: debug mds = 20 2024-04-25T10:30:25.036 INFO:teuthology.orchestra.run.smithi037.stdout: debug mds balancer = 20 2024-04-25T10:30:25.036 INFO:teuthology.orchestra.run.smithi037.stdout: debug ms = 1 2024-04-25T10:30:25.036 INFO:teuthology.orchestra.run.smithi037.stdout: mds debug frag = True 2024-04-25T10:30:25.036 INFO:teuthology.orchestra.run.smithi037.stdout: mds debug scatterstat = False 2024-04-25T10:30:25.036 INFO:teuthology.orchestra.run.smithi037.stdout: mds op complaint time = 180 2024-04-25T10:30:25.036 INFO:teuthology.orchestra.run.smithi037.stdout: mds verify scatter = False 2024-04-25T10:30:25.037 INFO:teuthology.orchestra.run.smithi037.stdout: osd op complaint time = 180 2024-04-25T10:30:25.037 INFO:teuthology.orchestra.run.smithi037.stdout: rados mon op timeout = 900 2024-04-25T10:30:25.037 INFO:teuthology.orchestra.run.smithi037.stdout: rados osd op timeout = 900 2024-04-25T10:30:25.038 INFO:teuthology.orchestra.run.smithi203.stdout:[global] 2024-04-25T10:30:25.038 INFO:teuthology.orchestra.run.smithi203.stdout: chdir = "" 2024-04-25T10:30:25.038 INFO:teuthology.orchestra.run.smithi203.stdout: pid file = /var/run/ceph/$cluster-$name.pid 2024-04-25T10:30:25.038 INFO:teuthology.orchestra.run.smithi203.stdout: auth supported = cephx 2024-04-25T10:30:25.038 INFO:teuthology.orchestra.run.smithi203.stdout: 2024-04-25T10:30:25.038 INFO:teuthology.orchestra.run.smithi203.stdout: filestore xattr use omap = true 2024-04-25T10:30:25.038 INFO:teuthology.orchestra.run.smithi203.stdout: 2024-04-25T10:30:25.039 INFO:teuthology.orchestra.run.smithi203.stdout: mon clock drift allowed = 1.000 2024-04-25T10:30:25.039 INFO:teuthology.orchestra.run.smithi203.stdout: 2024-04-25T10:30:25.039 INFO:teuthology.orchestra.run.smithi203.stdout: osd crush chooseleaf type = 0 2024-04-25T10:30:25.039 INFO:teuthology.orchestra.run.smithi203.stdout: auth debug = true 2024-04-25T10:30:25.039 INFO:teuthology.orchestra.run.smithi203.stdout: 2024-04-25T10:30:25.039 INFO:teuthology.orchestra.run.smithi203.stdout: ms die on old message = true 2024-04-25T10:30:25.039 INFO:teuthology.orchestra.run.smithi203.stdout: ms die on bug = true 2024-04-25T10:30:25.039 INFO:teuthology.orchestra.run.smithi203.stdout: 2024-04-25T10:30:25.039 INFO:teuthology.orchestra.run.smithi203.stdout: mon max pg per osd = 10000 # >= luminous 2024-04-25T10:30:25.039 INFO:teuthology.orchestra.run.smithi203.stdout: mon pg warn max object skew = 0 2024-04-25T10:30:25.039 INFO:teuthology.orchestra.run.smithi203.stdout: 2024-04-25T10:30:25.039 INFO:teuthology.orchestra.run.smithi203.stdout: # disable pg_autoscaler by default for new pools 2024-04-25T10:30:25.039 INFO:teuthology.orchestra.run.smithi203.stdout: osd_pool_default_pg_autoscale_mode = off 2024-04-25T10:30:25.040 INFO:teuthology.orchestra.run.smithi203.stdout: 2024-04-25T10:30:25.040 INFO:teuthology.orchestra.run.smithi203.stdout: osd pool default size = 2 2024-04-25T10:30:25.040 INFO:teuthology.orchestra.run.smithi203.stdout: 2024-04-25T10:30:25.040 INFO:teuthology.orchestra.run.smithi203.stdout: mon osd allow primary affinity = true 2024-04-25T10:30:25.040 INFO:teuthology.orchestra.run.smithi203.stdout: mon osd allow pg remap = true 2024-04-25T10:30:25.040 INFO:teuthology.orchestra.run.smithi203.stdout: mon warn on legacy crush tunables = false 2024-04-25T10:30:25.040 INFO:teuthology.orchestra.run.smithi203.stdout: mon warn on crush straw calc version zero = false 2024-04-25T10:30:25.040 INFO:teuthology.orchestra.run.smithi203.stdout: mon warn on no sortbitwise = false 2024-04-25T10:30:25.040 INFO:teuthology.orchestra.run.smithi203.stdout: mon warn on osd down out interval zero = false 2024-04-25T10:30:25.040 INFO:teuthology.orchestra.run.smithi203.stdout: mon warn on too few osds = false 2024-04-25T10:30:25.040 INFO:teuthology.orchestra.run.smithi203.stdout: mon_warn_on_pool_pg_num_not_power_of_two = false 2024-04-25T10:30:25.040 INFO:teuthology.orchestra.run.smithi203.stdout: mon_warn_on_pool_no_redundancy = false 2024-04-25T10:30:25.040 INFO:teuthology.orchestra.run.smithi203.stdout: mon_allow_pool_size_one = true 2024-04-25T10:30:25.040 INFO:teuthology.orchestra.run.smithi203.stdout: 2024-04-25T10:30:25.040 INFO:teuthology.orchestra.run.smithi203.stdout: osd pool default erasure code profile = plugin=jerasure technique=reed_sol_van k=2 m=1 crush-failure-domain=osd 2024-04-25T10:30:25.041 INFO:teuthology.orchestra.run.smithi203.stdout: 2024-04-25T10:30:25.041 INFO:teuthology.orchestra.run.smithi203.stdout: osd default data pool replay window = 5 2024-04-25T10:30:25.041 INFO:teuthology.orchestra.run.smithi203.stdout: 2024-04-25T10:30:25.041 INFO:teuthology.orchestra.run.smithi203.stdout: mon allow pool delete = true 2024-04-25T10:30:25.041 INFO:teuthology.orchestra.run.smithi203.stdout: 2024-04-25T10:30:25.041 INFO:teuthology.orchestra.run.smithi203.stdout: mon cluster log file level = debug 2024-04-25T10:30:25.041 INFO:teuthology.orchestra.run.smithi203.stdout: debug asserts on shutdown = true 2024-04-25T10:30:25.041 INFO:teuthology.orchestra.run.smithi203.stdout: mon health detail to clog = false 2024-04-25T10:30:25.041 INFO:teuthology.orchestra.run.smithi203.stdout: mon host = "172.21.15.37,172.21.15.203,[v2:172.21.15.203:3301,v1:172.21.15.203:6790]" 2024-04-25T10:30:25.041 INFO:teuthology.orchestra.run.smithi203.stdout: fsid = f4acda28-be34-492a-b286-994afd36f61d 2024-04-25T10:30:25.041 INFO:teuthology.orchestra.run.smithi203.stdout: 2024-04-25T10:30:25.041 INFO:teuthology.orchestra.run.smithi203.stdout:[osd] 2024-04-25T10:30:25.041 INFO:teuthology.orchestra.run.smithi203.stdout: osd journal size = 100 2024-04-25T10:30:25.041 INFO:teuthology.orchestra.run.smithi203.stdout: 2024-04-25T10:30:25.042 INFO:teuthology.orchestra.run.smithi203.stdout: osd scrub load threshold = 5.0 2024-04-25T10:30:25.042 INFO:teuthology.orchestra.run.smithi203.stdout: osd scrub max interval = 600 2024-04-25T10:30:25.042 INFO:teuthology.orchestra.run.smithi203.stdout: osd mclock profile = high_recovery_ops 2024-04-25T10:30:25.042 INFO:teuthology.orchestra.run.smithi203.stdout: 2024-04-25T10:30:25.042 INFO:teuthology.orchestra.run.smithi203.stdout: osd recover clone overlap = true 2024-04-25T10:30:25.042 INFO:teuthology.orchestra.run.smithi203.stdout: osd recovery max chunk = 1048576 2024-04-25T10:30:25.042 INFO:teuthology.orchestra.run.smithi203.stdout: 2024-04-25T10:30:25.042 INFO:teuthology.orchestra.run.smithi203.stdout: osd debug shutdown = true 2024-04-25T10:30:25.042 INFO:teuthology.orchestra.run.smithi203.stdout: osd debug op order = true 2024-04-25T10:30:25.042 INFO:teuthology.orchestra.run.smithi203.stdout: osd debug verify stray on activate = true 2024-04-25T10:30:25.042 INFO:teuthology.orchestra.run.smithi203.stdout: osd debug trim objects = true 2024-04-25T10:30:25.042 INFO:teuthology.orchestra.run.smithi203.stdout: 2024-04-25T10:30:25.042 INFO:teuthology.orchestra.run.smithi203.stdout: osd open classes on start = true 2024-04-25T10:30:25.043 INFO:teuthology.orchestra.run.smithi203.stdout: osd debug pg log writeout = true 2024-04-25T10:30:25.043 INFO:teuthology.orchestra.run.smithi203.stdout: 2024-04-25T10:30:25.043 INFO:teuthology.orchestra.run.smithi203.stdout: osd deep scrub update digest min age = 30 2024-04-25T10:30:25.043 INFO:teuthology.orchestra.run.smithi203.stdout: 2024-04-25T10:30:25.043 INFO:teuthology.orchestra.run.smithi203.stdout: osd map max advance = 10 2024-04-25T10:30:25.043 INFO:teuthology.orchestra.run.smithi203.stdout: 2024-04-25T10:30:25.043 INFO:teuthology.orchestra.run.smithi203.stdout: journal zero on create = true 2024-04-25T10:30:25.043 INFO:teuthology.orchestra.run.smithi203.stdout: 2024-04-25T10:30:25.043 INFO:teuthology.orchestra.run.smithi203.stdout: filestore ondisk finisher threads = 3 2024-04-25T10:30:25.043 INFO:teuthology.orchestra.run.smithi203.stdout: filestore apply finisher threads = 3 2024-04-25T10:30:25.043 INFO:teuthology.orchestra.run.smithi203.stdout: 2024-04-25T10:30:25.043 INFO:teuthology.orchestra.run.smithi203.stdout: bdev debug aio = true 2024-04-25T10:30:25.043 INFO:teuthology.orchestra.run.smithi203.stdout: osd debug misdirected ops = true 2024-04-25T10:30:25.043 INFO:teuthology.orchestra.run.smithi203.stdout: bdev async discard = True 2024-04-25T10:30:25.044 INFO:teuthology.orchestra.run.smithi203.stdout: bdev enable discard = True 2024-04-25T10:30:25.044 INFO:teuthology.orchestra.run.smithi203.stdout: bluestore allocator = bitmap 2024-04-25T10:30:25.044 INFO:teuthology.orchestra.run.smithi203.stdout: bluestore block size = 96636764160 2024-04-25T10:30:25.044 INFO:teuthology.orchestra.run.smithi203.stdout: bluestore fsck on mount = True 2024-04-25T10:30:25.044 INFO:teuthology.orchestra.run.smithi203.stdout: debug bluefs = 1/20 2024-04-25T10:30:25.044 INFO:teuthology.orchestra.run.smithi203.stdout: debug bluestore = 1/20 2024-04-25T10:30:25.044 INFO:teuthology.orchestra.run.smithi203.stdout: debug ms = 1 2024-04-25T10:30:25.044 INFO:teuthology.orchestra.run.smithi203.stdout: debug osd = 20 2024-04-25T10:30:25.044 INFO:teuthology.orchestra.run.smithi203.stdout: debug rocksdb = 4/10 2024-04-25T10:30:25.044 INFO:teuthology.orchestra.run.smithi203.stdout: mon osd backfillfull_ratio = 0.85 2024-04-25T10:30:25.044 INFO:teuthology.orchestra.run.smithi203.stdout: mon osd full ratio = 0.9 2024-04-25T10:30:25.044 INFO:teuthology.orchestra.run.smithi203.stdout: mon osd nearfull ratio = 0.8 2024-04-25T10:30:25.044 INFO:teuthology.orchestra.run.smithi203.stdout: osd failsafe full ratio = 0.95 2024-04-25T10:30:25.044 INFO:teuthology.orchestra.run.smithi203.stdout: osd objectstore = bluestore 2024-04-25T10:30:25.044 INFO:teuthology.orchestra.run.smithi203.stdout: osd op complaint time = 180 2024-04-25T10:30:25.045 INFO:teuthology.orchestra.run.smithi203.stdout: 2024-04-25T10:30:25.045 INFO:teuthology.orchestra.run.smithi203.stdout:[mgr] 2024-04-25T10:30:25.045 INFO:teuthology.orchestra.run.smithi203.stdout: debug ms = 1 2024-04-25T10:30:25.045 INFO:teuthology.orchestra.run.smithi203.stdout: debug mgr = 20 2024-04-25T10:30:25.045 INFO:teuthology.orchestra.run.smithi203.stdout: debug mon = 20 2024-04-25T10:30:25.045 INFO:teuthology.orchestra.run.smithi203.stdout: debug auth = 20 2024-04-25T10:30:25.045 INFO:teuthology.orchestra.run.smithi203.stdout: mon reweight min pgs per osd = 4 2024-04-25T10:30:25.045 INFO:teuthology.orchestra.run.smithi203.stdout: mon reweight min bytes per osd = 10 2024-04-25T10:30:25.045 INFO:teuthology.orchestra.run.smithi203.stdout: mgr/telemetry/nag = false 2024-04-25T10:30:25.045 INFO:teuthology.orchestra.run.smithi203.stdout: debug client = 20 2024-04-25T10:30:25.045 INFO:teuthology.orchestra.run.smithi203.stdout: 2024-04-25T10:30:25.045 INFO:teuthology.orchestra.run.smithi203.stdout:[mon] 2024-04-25T10:30:25.045 INFO:teuthology.orchestra.run.smithi203.stdout: debug ms = 1 2024-04-25T10:30:25.045 INFO:teuthology.orchestra.run.smithi203.stdout: debug mon = 20 2024-04-25T10:30:25.046 INFO:teuthology.orchestra.run.smithi203.stdout: debug paxos = 20 2024-04-25T10:30:25.046 INFO:teuthology.orchestra.run.smithi203.stdout: debug auth = 20 2024-04-25T10:30:25.046 INFO:teuthology.orchestra.run.smithi203.stdout: mon data avail warn = 5 2024-04-25T10:30:25.046 INFO:teuthology.orchestra.run.smithi203.stdout: mon mgr mkfs grace = 240 2024-04-25T10:30:25.046 INFO:teuthology.orchestra.run.smithi203.stdout: mon reweight min pgs per osd = 4 2024-04-25T10:30:25.046 INFO:teuthology.orchestra.run.smithi203.stdout: mon osd reporter subtree level = osd 2024-04-25T10:30:25.046 INFO:teuthology.orchestra.run.smithi203.stdout: mon osd prime pg temp = true 2024-04-25T10:30:25.046 INFO:teuthology.orchestra.run.smithi203.stdout: mon reweight min bytes per osd = 10 2024-04-25T10:30:25.046 INFO:teuthology.orchestra.run.smithi203.stdout: 2024-04-25T10:30:25.046 INFO:teuthology.orchestra.run.smithi203.stdout: # rotate auth tickets quickly to exercise renewal paths 2024-04-25T10:30:25.046 INFO:teuthology.orchestra.run.smithi203.stdout: auth mon ticket ttl = 660 # 11m 2024-04-25T10:30:25.046 INFO:teuthology.orchestra.run.smithi203.stdout: auth service ticket ttl = 240 # 4m 2024-04-25T10:30:25.047 INFO:teuthology.orchestra.run.smithi203.stdout: 2024-04-25T10:30:25.047 INFO:teuthology.orchestra.run.smithi203.stdout: # don't complain about insecure global_id in the test suite 2024-04-25T10:30:25.047 INFO:teuthology.orchestra.run.smithi203.stdout: mon_warn_on_insecure_global_id_reclaim = false 2024-04-25T10:30:25.047 INFO:teuthology.orchestra.run.smithi203.stdout: mon_warn_on_insecure_global_id_reclaim_allowed = false 2024-04-25T10:30:25.047 INFO:teuthology.orchestra.run.smithi203.stdout: 2024-04-25T10:30:25.047 INFO:teuthology.orchestra.run.smithi203.stdout: # 1m isn't quite enough 2024-04-25T10:30:25.047 INFO:teuthology.orchestra.run.smithi203.stdout: mon_down_mkfs_grace = 2m 2024-04-25T10:30:25.047 INFO:teuthology.orchestra.run.smithi203.stdout: 2024-04-25T10:30:25.047 INFO:teuthology.orchestra.run.smithi203.stdout: mon_warn_on_filestore_osds = false 2024-04-25T10:30:25.047 INFO:teuthology.orchestra.run.smithi203.stdout: mon down mkfs grace = 300 2024-04-25T10:30:25.047 INFO:teuthology.orchestra.run.smithi203.stdout: mon op complaint time = 120 2024-04-25T10:30:25.047 INFO:teuthology.orchestra.run.smithi203.stdout: 2024-04-25T10:30:25.047 INFO:teuthology.orchestra.run.smithi203.stdout:[client] 2024-04-25T10:30:25.047 INFO:teuthology.orchestra.run.smithi203.stdout: rgw cache enabled = true 2024-04-25T10:30:25.048 INFO:teuthology.orchestra.run.smithi203.stdout: rgw enable ops log = true 2024-04-25T10:30:25.048 INFO:teuthology.orchestra.run.smithi203.stdout: rgw enable usage log = true 2024-04-25T10:30:25.048 INFO:teuthology.orchestra.run.smithi203.stdout: log file = /var/log/ceph/$cluster-$name.$pid.log 2024-04-25T10:30:25.048 INFO:teuthology.orchestra.run.smithi203.stdout: admin socket = /var/run/ceph/$cluster-$name.$pid.asok 2024-04-25T10:30:25.048 INFO:teuthology.orchestra.run.smithi203.stdout: client mount timeout = 600 2024-04-25T10:30:25.048 INFO:teuthology.orchestra.run.smithi203.stdout: debug client = 20 2024-04-25T10:30:25.048 INFO:teuthology.orchestra.run.smithi203.stdout: debug ms = 1 2024-04-25T10:30:25.048 INFO:teuthology.orchestra.run.smithi203.stdout: pid file = "" 2024-04-25T10:30:25.048 INFO:teuthology.orchestra.run.smithi203.stdout: rados mon op timeout = 900 2024-04-25T10:30:25.048 INFO:teuthology.orchestra.run.smithi203.stdout: rados osd op timeout = 900 2024-04-25T10:30:25.048 INFO:teuthology.orchestra.run.smithi203.stdout:[mon.a] 2024-04-25T10:30:25.048 INFO:teuthology.orchestra.run.smithi203.stdout:[mon.b] 2024-04-25T10:30:25.048 INFO:teuthology.orchestra.run.smithi203.stdout:[mon.c] 2024-04-25T10:30:25.049 INFO:teuthology.orchestra.run.smithi203.stdout:[mds.a] 2024-04-25T10:30:25.049 INFO:teuthology.orchestra.run.smithi203.stdout:[mds.c] 2024-04-25T10:30:25.049 INFO:teuthology.orchestra.run.smithi203.stdout:[mds.b] 2024-04-25T10:30:25.049 INFO:teuthology.orchestra.run.smithi203.stdout:[mds.d] 2024-04-25T10:30:25.049 INFO:teuthology.orchestra.run.smithi203.stdout:[mds] 2024-04-25T10:30:25.049 INFO:teuthology.orchestra.run.smithi203.stdout: debug mds = 20 2024-04-25T10:30:25.049 INFO:teuthology.orchestra.run.smithi203.stdout: debug mds balancer = 20 2024-04-25T10:30:25.049 INFO:teuthology.orchestra.run.smithi203.stdout: debug ms = 1 2024-04-25T10:30:25.049 INFO:teuthology.orchestra.run.smithi203.stdout: mds debug frag = True 2024-04-25T10:30:25.049 INFO:teuthology.orchestra.run.smithi203.stdout: mds debug scatterstat = False 2024-04-25T10:30:25.049 INFO:teuthology.orchestra.run.smithi203.stdout: mds op complaint time = 180 2024-04-25T10:30:25.049 INFO:teuthology.orchestra.run.smithi203.stdout: mds verify scatter = False 2024-04-25T10:30:25.049 INFO:teuthology.orchestra.run.smithi203.stdout: osd op complaint time = 180 2024-04-25T10:30:25.049 INFO:teuthology.orchestra.run.smithi203.stdout: rados mon op timeout = 900 2024-04-25T10:30:25.049 INFO:teuthology.orchestra.run.smithi203.stdout: rados osd op timeout = 900 2024-04-25T10:30:25.051 INFO:tasks.ceph.mds.a:Restarting daemon 2024-04-25T10:30:25.051 INFO:tasks.ceph.mds.a:Stopping old one... 2024-04-25T10:30:25.052 DEBUG:tasks.ceph.mds.a:waiting for process to exit 2024-04-25T10:30:25.052 INFO:teuthology.orchestra.run:waiting for 300 2024-04-25T10:30:25.052 INFO:tasks.ceph.mds.c:Restarting daemon 2024-04-25T10:30:25.052 INFO:tasks.ceph.mds.c:Stopping old one... 2024-04-25T10:30:25.052 DEBUG:tasks.ceph.mds.c:waiting for process to exit 2024-04-25T10:30:25.052 INFO:teuthology.orchestra.run:waiting for 300 2024-04-25T10:30:25.052 INFO:tasks.ceph.mds.b:Restarting daemon 2024-04-25T10:30:25.052 INFO:tasks.ceph.mds.b:Stopping old one... 2024-04-25T10:30:25.053 DEBUG:tasks.ceph.mds.b:waiting for process to exit 2024-04-25T10:30:25.053 INFO:teuthology.orchestra.run:waiting for 300 2024-04-25T10:30:25.053 INFO:tasks.ceph.mds.d:Restarting daemon 2024-04-25T10:30:25.053 INFO:tasks.ceph.mds.d:Stopping old one... 2024-04-25T10:30:25.053 DEBUG:tasks.ceph.mds.d:waiting for process to exit 2024-04-25T10:30:25.053 INFO:teuthology.orchestra.run:waiting for 300 2024-04-25T10:30:25.101 INFO:tasks.ceph.mds.c:Stopped 2024-04-25T10:30:25.102 DEBUG:teuthology.orchestra.run.smithi037:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mds -f --cluster ceph -i c 2024-04-25T10:30:25.103 INFO:tasks.ceph.mds.b:Stopped 2024-04-25T10:30:25.103 DEBUG:teuthology.orchestra.run.smithi203:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mds -f --cluster ceph -i b 2024-04-25T10:30:25.104 INFO:tasks.ceph.mds.d:Stopped 2024-04-25T10:30:25.104 DEBUG:teuthology.orchestra.run.smithi203:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mds -f --cluster ceph -i d 2024-04-25T10:30:25.105 INFO:tasks.ceph.mds.a:Stopped 2024-04-25T10:30:25.105 DEBUG:teuthology.orchestra.run.smithi037:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mds -f --cluster ceph -i a 2024-04-25T10:30:25.145 INFO:tasks.ceph.mds.b:Started 2024-04-25T10:30:25.146 DEBUG:teuthology.parallel:result is None 2024-04-25T10:30:25.146 INFO:tasks.ceph.mds.d:Started 2024-04-25T10:30:25.146 DEBUG:teuthology.parallel:result is None 2024-04-25T10:30:25.147 INFO:tasks.ceph.mds.c:Started 2024-04-25T10:30:25.147 DEBUG:teuthology.parallel:result is None 2024-04-25T10:30:25.148 INFO:tasks.ceph.mds.a:Started 2024-04-25T10:30:25.149 DEBUG:teuthology.parallel:result is None 2024-04-25T10:30:25.149 DEBUG:teuthology.orchestra.run.smithi037:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage rados --cluster ceph purge cephfs_metadata --yes-i-really-really-mean-it 2024-04-25T10:30:25.203 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:25.201+0000 7fd406716f40 1 Processor -- start 2024-04-25T10:30:25.203 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:25.201+0000 7fd406716f40 1 -- start start 2024-04-25T10:30:25.205 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:25.202+0000 7fd406716f40 1 --2- >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x5566fa91c200 0x5566fa9f4fd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:30:25.205 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:25.202+0000 7fd406716f40 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x5566fa9c5ea0 0x5566fa91d520 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:30:25.206 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:25.202+0000 7fd406716f40 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x5566fa9d8ec0 0x5566fa9ee1e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:30:25.206 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:25.202+0000 7fd406716f40 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 -- 0x5566fa8c4e40 con 0x5566fa9d8ec0 2024-04-25T10:30:25.206 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:25.202+0000 7fd406716f40 1 -- --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_getmap magic: 0 -- 0x5566fa8c8e80 con 0x5566fa9c5ea0 2024-04-25T10:30:25.206 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:25.202+0000 7fd406716f40 1 -- --> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] -- mon_getmap magic: 0 -- 0x5566fa9da5a0 con 0x5566fa91c200 2024-04-25T10:30:25.206 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:25.202+0000 7fd4009e1640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x5566fa9d8ec0 0x5566fa9ee1e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:30:25.206 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:25.202+0000 7fd4009e1640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x5566fa9d8ec0 0x5566fa9ee1e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.37:3300/0 says I am v2:172.21.15.37:59790/0 (socket says 172.21.15.37:59790) 2024-04-25T10:30:25.206 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:25.202+0000 7fd4009e1640 1 -- 172.21.15.37:0/2903687145 learned_addr learned my addr 172.21.15.37:0/2903687145 (peer_addr_for_me v2:172.21.15.37:0/0) 2024-04-25T10:30:25.206 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:25.203+0000 7fd4001e0640 1 --2- 172.21.15.37:0/2903687145 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x5566fa91c200 0x5566fa9f4fd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:30:25.206 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:25.203+0000 7fd3ff9df640 1 --2- 172.21.15.37:0/2903687145 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x5566fa9c5ea0 0x5566fa91d520 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:30:25.206 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:25.203+0000 7fd4009e1640 1 -- 172.21.15.37:0/2903687145 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x5566fa91c200 msgr2=0x5566fa9f4fd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:30:25.206 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:25.203+0000 7fd4009e1640 1 --2- 172.21.15.37:0/2903687145 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x5566fa91c200 0x5566fa9f4fd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:30:25.206 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:25.203+0000 7fd4009e1640 1 -- 172.21.15.37:0/2903687145 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x5566fa9c5ea0 msgr2=0x5566fa91d520 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:30:25.207 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:25.203+0000 7fd4009e1640 1 --2- 172.21.15.37:0/2903687145 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x5566fa9c5ea0 0x5566fa91d520 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:30:25.207 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:25.203+0000 7fd4009e1640 1 -- 172.21.15.37:0/2903687145 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5566fa967190 con 0x5566fa9d8ec0 2024-04-25T10:30:25.207 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:25.203+0000 7fd4001e0640 1 --2- 172.21.15.37:0/2903687145 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x5566fa91c200 0x5566fa9f4fd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T10:30:25.207 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:25.203+0000 7fd3ff9df640 1 --2- 172.21.15.37:0/2903687145 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x5566fa9c5ea0 0x5566fa91d520 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T10:30:25.207 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:25.203+0000 7fd4009e1640 1 --2- 172.21.15.37:0/2903687145 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x5566fa9d8ec0 0x5566fa9ee1e0 secure :-1 s=READY pgs=489 cs=0 l=1 rev1=1 crypto rx=0x7fd3f8025b90 tx=0x7fd3f8051000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a1720ef408a1e16 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:30:25.207 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:25.203+0000 7fd3fe9dd640 1 -- 172.21.15.37:0/2903687145 <== mon.0 v2:172.21.15.37:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd3f804fb70 con 0x5566fa9d8ec0 2024-04-25T10:30:25.207 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:25.203+0000 7fd3fe9dd640 1 -- 172.21.15.37:0/2903687145 <== mon.0 v2:172.21.15.37:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fd3f8067070 con 0x5566fa9d8ec0 2024-04-25T10:30:25.207 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:25.203+0000 7fd3fe9dd640 1 -- 172.21.15.37:0/2903687145 <== mon.0 v2:172.21.15.37:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd3f804f610 con 0x5566fa9d8ec0 2024-04-25T10:30:25.208 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:25.204+0000 7fd406716f40 1 -- 172.21.15.37:0/2903687145 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x5566fa9d8ec0 msgr2=0x5566fa9ee1e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:30:25.208 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:25.204+0000 7fd406716f40 1 --2- 172.21.15.37:0/2903687145 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x5566fa9d8ec0 0x5566fa9ee1e0 secure :-1 s=READY pgs=489 cs=0 l=1 rev1=1 crypto rx=0x7fd3f8025b90 tx=0x7fd3f8051000 comp rx=0 tx=0).stop 2024-04-25T10:30:25.208 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:25.204+0000 7fd406716f40 1 -- 172.21.15.37:0/2903687145 shutdown_connections 2024-04-25T10:30:25.208 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:25.204+0000 7fd406716f40 1 --2- 172.21.15.37:0/2903687145 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x5566fa9d8ec0 0x5566fa9ee1e0 unknown :-1 s=CLOSED pgs=489 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:30:25.209 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:25.204+0000 7fd406716f40 1 --2- 172.21.15.37:0/2903687145 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x5566fa9c5ea0 0x5566fa91d520 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:30:25.209 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:25.204+0000 7fd406716f40 1 --2- 172.21.15.37:0/2903687145 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x5566fa91c200 0x5566fa9f4fd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:30:25.209 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:25.204+0000 7fd406716f40 1 -- 172.21.15.37:0/2903687145 >> 172.21.15.37:0/2903687145 conn(0x5566fa840ac0 msgr2=0x5566fa9f70b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:30:25.209 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:25.204+0000 7fd406716f40 1 -- 172.21.15.37:0/2903687145 shutdown_connections 2024-04-25T10:30:25.209 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:25.204+0000 7fd406716f40 1 -- 172.21.15.37:0/2903687145 wait complete. 2024-04-25T10:30:25.209 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:25.206+0000 7fd406716f40 1 Processor -- start 2024-04-25T10:30:25.209 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:25.206+0000 7fd406716f40 1 -- start start 2024-04-25T10:30:25.211 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:25.208+0000 7fd406716f40 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x5566fa91c200 0x5566fa9f4410 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:30:25.211 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:25.208+0000 7fd406716f40 1 --2- >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x5566fa9c5ea0 0x5566fa9f2af0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:30:25.211 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:25.208+0000 7fd406716f40 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x5566fa9d8ec0 0x5566fa9c43b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:30:25.211 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:25.208+0000 7fd406716f40 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 -- 0x5566fa91cfe0 con 0x5566fa9d8ec0 2024-04-25T10:30:25.212 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:25.208+0000 7fd406716f40 1 -- --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_getmap magic: 0 -- 0x5566fa9681c0 con 0x5566fa91c200 2024-04-25T10:30:25.212 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:25.208+0000 7fd406716f40 1 -- --> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] -- mon_getmap magic: 0 -- 0x5566fa9edc90 con 0x5566fa9c5ea0 2024-04-25T10:30:25.212 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:25.208+0000 7fd4009e1640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x5566fa9d8ec0 0x5566fa9c43b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:30:25.212 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:25.208+0000 7fd4009e1640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x5566fa9d8ec0 0x5566fa9c43b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.37:3300/0 says I am v2:172.21.15.37:59804/0 (socket says 172.21.15.37:59804) 2024-04-25T10:30:25.212 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:25.208+0000 7fd4009e1640 1 -- 172.21.15.37:0/3348722167 learned_addr learned my addr 172.21.15.37:0/3348722167 (peer_addr_for_me v2:172.21.15.37:0/0) 2024-04-25T10:30:25.212 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:25.208+0000 7fd3ff9df640 1 --2- 172.21.15.37:0/3348722167 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x5566fa9c5ea0 0x5566fa9f2af0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:30:25.212 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:25.208+0000 7fd4001e0640 1 --2- 172.21.15.37:0/3348722167 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x5566fa91c200 0x5566fa9f4410 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:30:25.212 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:25.208+0000 7fd4009e1640 1 -- 172.21.15.37:0/3348722167 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x5566fa9c5ea0 msgr2=0x5566fa9f2af0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:30:25.212 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:25.208+0000 7fd4009e1640 1 --2- 172.21.15.37:0/3348722167 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x5566fa9c5ea0 0x5566fa9f2af0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:30:25.212 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:25.208+0000 7fd4009e1640 1 -- 172.21.15.37:0/3348722167 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x5566fa91c200 msgr2=0x5566fa9f4410 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:30:25.212 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:25.208+0000 7fd4009e1640 1 --2- 172.21.15.37:0/3348722167 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x5566fa91c200 0x5566fa9f4410 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:30:25.212 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:25.208+0000 7fd4009e1640 1 -- 172.21.15.37:0/3348722167 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5566fa9ee280 con 0x5566fa9d8ec0 2024-04-25T10:30:25.212 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:25.209+0000 7fd4009e1640 1 --2- 172.21.15.37:0/3348722167 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x5566fa9d8ec0 0x5566fa9c43b0 secure :-1 s=READY pgs=490 cs=0 l=1 rev1=1 crypto rx=0x7fd3f8064040 tx=0x7fd3f8003330 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:30:25.212 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:25.209+0000 7fd3fd9db640 1 -- 172.21.15.37:0/3348722167 <== mon.0 v2:172.21.15.37:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd3f804fd10 con 0x5566fa9d8ec0 2024-04-25T10:30:25.213 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:25.209+0000 7fd3fd9db640 1 -- 172.21.15.37:0/3348722167 <== mon.0 v2:172.21.15.37:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fd3f8005020 con 0x5566fa9d8ec0 2024-04-25T10:30:25.213 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:25.209+0000 7fd406716f40 1 -- 172.21.15.37:0/3348722167 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x5566fa9f63b0 con 0x5566fa9d8ec0 2024-04-25T10:30:25.213 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:25.209+0000 7fd3fd9db640 1 -- 172.21.15.37:0/3348722167 <== mon.0 v2:172.21.15.37:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd3f8005410 con 0x5566fa9d8ec0 2024-04-25T10:30:25.213 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:25.209+0000 7fd406716f40 1 -- 172.21.15.37:0/3348722167 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5566fa9ee560 con 0x5566fa9d8ec0 2024-04-25T10:30:25.213 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:25.211+0000 7fd3fd9db640 1 -- 172.21.15.37:0/3348722167 <== mon.0 v2:172.21.15.37:3300/0 4 ==== mgrmap(e 5) ==== 96782+0+0 (secure 0 0 0) 0x7fd3f8075020 con 0x5566fa9d8ec0 2024-04-25T10:30:25.213 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:25.211+0000 7fd3fd9db640 1 --2- 172.21.15.37:0/3348722167 >> [v2:172.21.15.203:6832/2418205237,v1:172.21.15.203:6833/2418205237] conn(0x7fd3ec072e90 0x7fd3ec0753a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:30:25.214 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:25.211+0000 7fd3fd9db640 1 -- 172.21.15.37:0/3348722167 <== mon.0 v2:172.21.15.37:3300/0 5 ==== osd_map(37..37 src has 1..37) ==== 5795+0+0 (secure 0 0 0) 0x7fd3f80d8960 con 0x5566fa9d8ec0 2024-04-25T10:30:25.214 INFO:teuthology.orchestra.run.smithi037.stdout:Warning: using slow linear search 2024-04-25T10:30:25.215 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:25.212+0000 7fd406716f40 1 --2- 172.21.15.37:0/3348722167 >> [v2:172.21.15.203:6800/4051532063,v1:172.21.15.203:6801/4051532063] conn(0x5566fa9f1fd0 0x5566faa98750 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:30:25.215 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:25.212+0000 7fd406716f40 1 -- 172.21.15.37:0/3348722167 --> [v2:172.21.15.203:6800/4051532063,v1:172.21.15.203:6801/4051532063] -- osd_op(unknown.0.0:1 4.0 4:00000000:%01:::head [pgnls start_epoch 37 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e37) -- 0x5566fa9f23e0 con 0x5566fa9f1fd0 2024-04-25T10:30:25.215 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:25.212+0000 7fd4001e0640 1 --2- 172.21.15.37:0/3348722167 >> [v2:172.21.15.203:6832/2418205237,v1:172.21.15.203:6833/2418205237] conn(0x7fd3ec072e90 0x7fd3ec0753a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:30:25.215 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:25.212+0000 7fd3ff9df640 1 --2- 172.21.15.37:0/3348722167 >> [v2:172.21.15.203:6800/4051532063,v1:172.21.15.203:6801/4051532063] conn(0x5566fa9f1fd0 0x5566faa98750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:30:25.215 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:25.213+0000 7fd3ff9df640 1 --2- 172.21.15.37:0/3348722167 >> [v2:172.21.15.203:6800/4051532063,v1:172.21.15.203:6801/4051532063] conn(0x5566fa9f1fd0 0x5566faa98750 crc :-1 s=READY pgs=11 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.4 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:30:25.215 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:25.213+0000 7fd4001e0640 1 --2- 172.21.15.37:0/3348722167 >> [v2:172.21.15.203:6832/2418205237,v1:172.21.15.203:6833/2418205237] conn(0x7fd3ec072e90 0x7fd3ec0753a0 secure :-1 s=READY pgs=236 cs=0 l=1 rev1=1 crypto rx=0x7fd3f0000c00 tx=0x7fd3f0003810 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:30:25.216 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:25.213+0000 7fd3ff9df640 1 -- 172.21.15.37:0/3348722167 <== osd.4 v2:172.21.15.203:6800/4051532063 1 ==== osd_op_reply(1 [pgnls start_epoch 37 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7fd3f4010720 con 0x5566fa9f1fd0 2024-04-25T10:30:25.216 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:25.213+0000 7fd3ff9df640 1 --2- 172.21.15.37:0/3348722167 >> [v2:172.21.15.203:6808/3688099014,v1:172.21.15.203:6809/3688099014] conn(0x7fd3f4011900 0x7fd3f400e300 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:30:25.216 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:25.213+0000 7fd3ff9df640 1 -- 172.21.15.37:0/3348722167 --> [v2:172.21.15.203:6808/3688099014,v1:172.21.15.203:6809/3688099014] -- osd_op(unknown.0.0:2 4.10 4:08000000:%01:::head [pgnls start_epoch 37 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e37) -- 0x5566fa9f23e0 con 0x7fd3f4011900 2024-04-25T10:30:25.216 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:25.214+0000 7fd4001e0640 1 --2- 172.21.15.37:0/3348722167 >> [v2:172.21.15.203:6808/3688099014,v1:172.21.15.203:6809/3688099014] conn(0x7fd3f4011900 0x7fd3f400e300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:30:25.216 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:25.214+0000 7fd4001e0640 1 --2- 172.21.15.37:0/3348722167 >> [v2:172.21.15.203:6808/3688099014,v1:172.21.15.203:6809/3688099014] conn(0x7fd3f4011900 0x7fd3f400e300 crc :-1 s=READY pgs=12 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.7 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:30:25.220 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:25.217+0000 7fd4001e0640 1 -- 172.21.15.37:0/3348722167 <== osd.7 v2:172.21.15.203:6808/3688099014 1 ==== osd_op_reply(2 [pgnls start_epoch 37 out=177b] v36'10 uv10 ondisk = 1) ==== 144+0+177 (crc 0 0 0) 0x7fd3f001b040 con 0x7fd3f4011900 2024-04-25T10:30:25.220 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:25.217+0000 7fd4001e0640 1 --2- 172.21.15.37:0/3348722167 >> [v2:172.21.15.37:6824/1815290618,v1:172.21.15.37:6825/1815290618] conn(0x7fd3f001e8d0 0x7fd3f00113c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:30:25.220 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:25.217+0000 7fd4001e0640 1 -- 172.21.15.37:0/3348722167 --> [v2:172.21.15.37:6824/1815290618,v1:172.21.15.37:6825/1815290618] -- osd_op(unknown.0.0:3 4.8 4:10000000:%01:::head [pgnls start_epoch 37 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e37) -- 0x5566fa9f23e0 con 0x7fd3f001e8d0 2024-04-25T10:30:25.220 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:25.218+0000 7fd3ff9df640 1 --2- 172.21.15.37:0/3348722167 >> [v2:172.21.15.37:6824/1815290618,v1:172.21.15.37:6825/1815290618] conn(0x7fd3f001e8d0 0x7fd3f00113c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:30:25.221 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:25.218+0000 7fd3ff9df640 1 --2- 172.21.15.37:0/3348722167 >> [v2:172.21.15.37:6824/1815290618,v1:172.21.15.37:6825/1815290618] conn(0x7fd3f001e8d0 0x7fd3f00113c0 crc :-1 s=READY pgs=13 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.3 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:30:25.221 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:25.218+0000 7fd3ff9df640 1 -- 172.21.15.37:0/3348722167 <== osd.3 v2:172.21.15.37:6824/1815290618 1 ==== osd_op_reply(3 [pgnls start_epoch 37 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7fd3f401e040 con 0x7fd3f001e8d0 2024-04-25T10:30:25.221 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:25.219+0000 7fd3ff9df640 1 --2- 172.21.15.37:0/3348722167 >> [v2:172.21.15.37:6808/2754669649,v1:172.21.15.37:6809/2754669649] conn(0x7fd3f401e7e0 0x7fd3f400d2f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:30:25.221 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:25.219+0000 7fd3ff9df640 1 -- 172.21.15.37:0/3348722167 --> [v2:172.21.15.37:6808/2754669649,v1:172.21.15.37:6809/2754669649] -- osd_op(unknown.0.0:4 4.18 4:18000000:%01:::head [pgnls start_epoch 37 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e37) -- 0x5566fa9f23e0 con 0x7fd3f401e7e0 2024-04-25T10:30:25.221 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:25.219+0000 7fd4009e1640 1 --2- 172.21.15.37:0/3348722167 >> [v2:172.21.15.37:6808/2754669649,v1:172.21.15.37:6809/2754669649] conn(0x7fd3f401e7e0 0x7fd3f400d2f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:30:25.221 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:25.219+0000 7fd4009e1640 1 --2- 172.21.15.37:0/3348722167 >> [v2:172.21.15.37:6808/2754669649,v1:172.21.15.37:6809/2754669649] conn(0x7fd3f401e7e0 0x7fd3f400d2f0 crc :-1 s=READY pgs=12 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:30:25.222 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:25.220+0000 7fd4009e1640 1 -- 172.21.15.37:0/3348722167 <== osd.2 v2:172.21.15.37:6808/2754669649 1 ==== osd_op_reply(4 [pgnls start_epoch 37 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7fd3f809edf0 con 0x7fd3f401e7e0 2024-04-25T10:30:25.222 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:25.220+0000 7fd4009e1640 1 --2- 172.21.15.37:0/3348722167 >> [v2:172.21.15.203:6810/2309615014,v1:172.21.15.203:6812/2309615014] conn(0x7fd3f8022b50 0x7fd3f80d82c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:30:25.222 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:25.220+0000 7fd4009e1640 1 -- 172.21.15.37:0/3348722167 --> [v2:172.21.15.203:6810/2309615014,v1:172.21.15.203:6812/2309615014] -- osd_op(unknown.0.0:5 4.4 4:20000000:%01:::head [pgnls start_epoch 37 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e37) -- 0x5566fa9f23e0 con 0x7fd3f8022b50 2024-04-25T10:30:25.222 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:25.220+0000 7fd4001e0640 1 --2- 172.21.15.37:0/3348722167 >> [v2:172.21.15.203:6810/2309615014,v1:172.21.15.203:6812/2309615014] conn(0x7fd3f8022b50 0x7fd3f80d82c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:30:25.223 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:25.221+0000 7fd4001e0640 1 --2- 172.21.15.37:0/3348722167 >> [v2:172.21.15.203:6810/2309615014,v1:172.21.15.203:6812/2309615014] conn(0x7fd3f8022b50 0x7fd3f80d82c0 crc :-1 s=READY pgs=11 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.6 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:30:25.225 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:25.223+0000 7fd4001e0640 1 -- 172.21.15.37:0/3348722167 <== osd.6 v2:172.21.15.203:6810/2309615014 1 ==== osd_op_reply(5 [pgnls start_epoch 37 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7fd3f001b040 con 0x7fd3f8022b50 2024-04-25T10:30:25.225 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:25.223+0000 7fd4001e0640 1 -- 172.21.15.37:0/3348722167 --> [v2:172.21.15.203:6810/2309615014,v1:172.21.15.203:6812/2309615014] -- osd_op(unknown.0.0:6 4.14 4:28000000:%01:::head [pgnls start_epoch 37 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e37) -- 0x5566fa9f23e0 con 0x7fd3f8022b50 2024-04-25T10:30:25.226 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:25.224+0000 7fd4001e0640 1 -- 172.21.15.37:0/3348722167 <== osd.6 v2:172.21.15.203:6810/2309615014 2 ==== osd_op_reply(6 [pgnls start_epoch 37 out=149b] v36'19 uv19 ondisk = 1) ==== 144+0+149 (crc 0 0 0) 0x7fd3f001b040 con 0x7fd3f8022b50 2024-04-25T10:30:25.226 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:25.224+0000 7fd4001e0640 1 --2- 172.21.15.37:0/3348722167 >> [v2:172.21.15.37:6816/2746074720,v1:172.21.15.37:6817/2746074720] conn(0x7fd3f0021950 0x7fd3f001d350 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:30:25.226 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:25.224+0000 7fd4001e0640 1 -- 172.21.15.37:0/3348722167 --> [v2:172.21.15.37:6816/2746074720,v1:172.21.15.37:6817/2746074720] -- osd_op(unknown.0.0:7 4.c 4:30000000:%01:::head [pgnls start_epoch 37 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e37) -- 0x5566fa9f23e0 con 0x7fd3f0021950 2024-04-25T10:30:25.226 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:25.224+0000 7fd3ff9df640 1 --2- 172.21.15.37:0/3348722167 >> [v2:172.21.15.37:6816/2746074720,v1:172.21.15.37:6817/2746074720] conn(0x7fd3f0021950 0x7fd3f001d350 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:30:25.226 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:25.224+0000 7fd3ff9df640 1 --2- 172.21.15.37:0/3348722167 >> [v2:172.21.15.37:6816/2746074720,v1:172.21.15.37:6817/2746074720] conn(0x7fd3f0021950 0x7fd3f001d350 crc :-1 s=READY pgs=12 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:30:25.229 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:25.227+0000 7fd3ff9df640 1 -- 172.21.15.37:0/3348722167 <== osd.0 v2:172.21.15.37:6816/2746074720 1 ==== osd_op_reply(7 [pgnls start_epoch 37 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7fd3f401e040 con 0x7fd3f0021950 2024-04-25T10:30:25.229 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:25.227+0000 7fd3ff9df640 1 -- 172.21.15.37:0/3348722167 --> [v2:172.21.15.203:6800/4051532063,v1:172.21.15.203:6801/4051532063] -- osd_op(unknown.0.0:8 4.1c 4:38000000:%01:::head [pgnls start_epoch 37 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e37) -- 0x5566fa9f23e0 con 0x5566fa9f1fd0 2024-04-25T10:30:25.230 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:25.227+0000 7fd3ff9df640 1 -- 172.21.15.37:0/3348722167 <== osd.4 v2:172.21.15.203:6800/4051532063 2 ==== osd_op_reply(8 [pgnls start_epoch 37 out=73b] v35'2 uv2 ondisk = 1) ==== 144+0+73 (crc 0 0 0) 0x7fd3f401e040 con 0x5566fa9f1fd0 2024-04-25T10:30:25.230 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:25.227+0000 7fd3ff9df640 1 -- 172.21.15.37:0/3348722167 --> [v2:172.21.15.203:6810/2309615014,v1:172.21.15.203:6812/2309615014] -- osd_op(unknown.0.0:9 4.2 4:40000000:%01:::head [pgnls start_epoch 37 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e37) -- 0x5566fa9f23e0 con 0x7fd3f8022b50 2024-04-25T10:30:25.230 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:25.228+0000 7fd4001e0640 1 -- 172.21.15.37:0/3348722167 <== osd.6 v2:172.21.15.203:6810/2309615014 3 ==== osd_op_reply(9 [pgnls start_epoch 37 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7fd3f001b040 con 0x7fd3f8022b50 2024-04-25T10:30:25.230 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:25.228+0000 7fd4001e0640 1 -- 172.21.15.37:0/3348722167 --> [v2:172.21.15.203:6800/4051532063,v1:172.21.15.203:6801/4051532063] -- osd_op(unknown.0.0:10 4.12 4:48000000:%01:::head [pgnls start_epoch 37 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e37) -- 0x5566fa9f23e0 con 0x5566fa9f1fd0 2024-04-25T10:30:25.231 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:25.228+0000 7fd3ff9df640 1 -- 172.21.15.37:0/3348722167 <== osd.4 v2:172.21.15.203:6800/4051532063 3 ==== osd_op_reply(10 [pgnls start_epoch 37 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7fd3f401e040 con 0x5566fa9f1fd0 2024-04-25T10:30:25.231 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:25.228+0000 7fd3ff9df640 1 -- 172.21.15.37:0/3348722167 --> [v2:172.21.15.37:6808/2754669649,v1:172.21.15.37:6809/2754669649] -- osd_op(unknown.0.0:11 4.a 4:50000000:%01:::head [pgnls start_epoch 37 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e37) -- 0x5566fa9f23e0 con 0x7fd3f401e7e0 2024-04-25T10:30:25.231 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:25.229+0000 7fd4009e1640 1 -- 172.21.15.37:0/3348722167 <== osd.2 v2:172.21.15.37:6808/2754669649 2 ==== osd_op_reply(11 [pgnls start_epoch 37 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7fd3f809edf0 con 0x7fd3f401e7e0 2024-04-25T10:30:25.231 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:25.229+0000 7fd4009e1640 1 -- 172.21.15.37:0/3348722167 --> [v2:172.21.15.37:6808/2754669649,v1:172.21.15.37:6809/2754669649] -- osd_op(unknown.0.0:12 4.1a 4:58000000:%01:::head [pgnls start_epoch 37 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e37) -- 0x5566fa9f23e0 con 0x7fd3f401e7e0 2024-04-25T10:30:25.231 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:25.229+0000 7fd4009e1640 1 -- 172.21.15.37:0/3348722167 <== osd.2 v2:172.21.15.37:6808/2754669649 3 ==== osd_op_reply(12 [pgnls start_epoch 37 out=49b] v35'1 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7fd3f809edf0 con 0x7fd3f401e7e0 2024-04-25T10:30:25.231 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:25.229+0000 7fd4009e1640 1 --2- 172.21.15.37:0/3348722167 >> [v2:172.21.15.203:6824/1203822890,v1:172.21.15.203:6825/1203822890] conn(0x7fd3f801f4b0 0x7fd3f8004600 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:30:25.231 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:25.229+0000 7fd4009e1640 1 -- 172.21.15.37:0/3348722167 --> [v2:172.21.15.203:6824/1203822890,v1:172.21.15.203:6825/1203822890] -- osd_op(unknown.0.0:13 4.6 4:60000000:%01:::head [pgnls start_epoch 37 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e37) -- 0x5566fa9f23e0 con 0x7fd3f801f4b0 2024-04-25T10:30:25.232 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:25.229+0000 7fd4009e1640 1 --2- 172.21.15.37:0/3348722167 >> [v2:172.21.15.203:6824/1203822890,v1:172.21.15.203:6825/1203822890] conn(0x7fd3f801f4b0 0x7fd3f8004600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:30:25.232 INFO:tasks.ceph.mds.b.smithi203.stdout:starting mds.b at 2024-04-25T10:30:25.232 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:25.230+0000 7fd4009e1640 1 --2- 172.21.15.37:0/3348722167 >> [v2:172.21.15.203:6824/1203822890,v1:172.21.15.203:6825/1203822890] conn(0x7fd3f801f4b0 0x7fd3f8004600 crc :-1 s=READY pgs=12 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.5 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:30:25.232 INFO:tasks.ceph.mds.d.smithi203.stdout:starting mds.d at 2024-04-25T10:30:25.234 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:25.232+0000 7fd4009e1640 1 -- 172.21.15.37:0/3348722167 <== osd.5 v2:172.21.15.203:6824/1203822890 1 ==== osd_op_reply(13 [pgnls start_epoch 37 out=77b] v36'3 uv3 ondisk = 1) ==== 144+0+77 (crc 0 0 0) 0x7fd3f809edf0 con 0x7fd3f801f4b0 2024-04-25T10:30:25.234 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:25.232+0000 7fd4009e1640 1 -- 172.21.15.37:0/3348722167 --> [v2:172.21.15.203:6824/1203822890,v1:172.21.15.203:6825/1203822890] -- osd_op(unknown.0.0:14 4.16 4:68000000:%01:::head [pgnls start_epoch 37 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e37) -- 0x5566fa9f23e0 con 0x7fd3f801f4b0 2024-04-25T10:30:25.235 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:25.232+0000 7fd4009e1640 1 -- 172.21.15.37:0/3348722167 <== osd.5 v2:172.21.15.203:6824/1203822890 2 ==== osd_op_reply(14 [pgnls start_epoch 37 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7fd3f809edf0 con 0x7fd3f801f4b0 2024-04-25T10:30:25.235 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:25.232+0000 7fd4009e1640 1 -- 172.21.15.37:0/3348722167 --> [v2:172.21.15.37:6808/2754669649,v1:172.21.15.37:6809/2754669649] -- osd_op(unknown.0.0:15 4.e 4:70000000:%01:::head [pgnls start_epoch 37 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e37) -- 0x5566fa9f23e0 con 0x7fd3f401e7e0 2024-04-25T10:30:25.235 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:25.233+0000 7fd4009e1640 1 -- 172.21.15.37:0/3348722167 <== osd.2 v2:172.21.15.37:6808/2754669649 4 ==== osd_op_reply(15 [pgnls start_epoch 37 out=73b] v35'2 uv2 ondisk = 1) ==== 144+0+73 (crc 0 0 0) 0x7fd3f809edf0 con 0x7fd3f401e7e0 2024-04-25T10:30:25.235 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:25.233+0000 7fd4009e1640 1 -- 172.21.15.37:0/3348722167 --> [v2:172.21.15.203:6800/4051532063,v1:172.21.15.203:6801/4051532063] -- osd_op(unknown.0.0:16 4.1e 4:78000000:%01:::head [pgnls start_epoch 37 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e37) -- 0x5566fa9f23e0 con 0x5566fa9f1fd0 2024-04-25T10:30:25.236 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:25.233+0000 7fd3ff9df640 1 -- 172.21.15.37:0/3348722167 <== osd.4 v2:172.21.15.203:6800/4051532063 4 ==== osd_op_reply(16 [pgnls start_epoch 37 out=49b] v35'1 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7fd3f401e040 con 0x5566fa9f1fd0 2024-04-25T10:30:25.236 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:25.233+0000 7fd3ff9df640 1 -- 172.21.15.37:0/3348722167 --> [v2:172.21.15.37:6808/2754669649,v1:172.21.15.37:6809/2754669649] -- osd_op(unknown.0.0:17 4.1 4:80000000:%01:::head [pgnls start_epoch 37 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e37) -- 0x5566fa9f23e0 con 0x7fd3f401e7e0 2024-04-25T10:30:25.236 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:25.234+0000 7fd4009e1640 1 -- 172.21.15.37:0/3348722167 <== osd.2 v2:172.21.15.37:6808/2754669649 5 ==== osd_op_reply(17 [pgnls start_epoch 37 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7fd3f809edf0 con 0x7fd3f401e7e0 2024-04-25T10:30:25.236 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:25.234+0000 7fd4009e1640 1 -- 172.21.15.37:0/3348722167 --> [v2:172.21.15.203:6800/4051532063,v1:172.21.15.203:6801/4051532063] -- osd_op(unknown.0.0:18 4.11 4:88000000:%01:::head [pgnls start_epoch 37 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e37) -- 0x5566fa9f23e0 con 0x5566fa9f1fd0 2024-04-25T10:30:25.237 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:25.234+0000 7fd3ff9df640 1 -- 172.21.15.37:0/3348722167 <== osd.4 v2:172.21.15.203:6800/4051532063 5 ==== osd_op_reply(18 [pgnls start_epoch 37 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7fd3f401e040 con 0x5566fa9f1fd0 2024-04-25T10:30:25.237 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:25.234+0000 7fd3ff9df640 1 --2- 172.21.15.37:0/3348722167 >> [v2:172.21.15.37:6800/1690341662,v1:172.21.15.37:6801/1690341662] conn(0x7fd3f4023940 0x7fd3f40253f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:30:25.237 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:25.234+0000 7fd3ff9df640 1 -- 172.21.15.37:0/3348722167 --> [v2:172.21.15.37:6800/1690341662,v1:172.21.15.37:6801/1690341662] -- osd_op(unknown.0.0:19 4.9 4:90000000:%01:::head [pgnls start_epoch 37 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e37) -- 0x5566fa9f23e0 con 0x7fd3f4023940 2024-04-25T10:30:25.237 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:25.234+0000 7fd4001e0640 1 --2- 172.21.15.37:0/3348722167 >> [v2:172.21.15.37:6800/1690341662,v1:172.21.15.37:6801/1690341662] conn(0x7fd3f4023940 0x7fd3f40253f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:30:25.237 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:25.235+0000 7fd4001e0640 1 --2- 172.21.15.37:0/3348722167 >> [v2:172.21.15.37:6800/1690341662,v1:172.21.15.37:6801/1690341662] conn(0x7fd3f4023940 0x7fd3f40253f0 crc :-1 s=READY pgs=13 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:30:25.237 INFO:tasks.ceph.mds.c.smithi037.stdout:starting mds.c at 2024-04-25T10:30:25.238 INFO:tasks.ceph.mds.a.smithi037.stdout:starting mds.a at 2024-04-25T10:30:25.239 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:25.238+0000 7fd4001e0640 1 -- 172.21.15.37:0/3348722167 <== osd.1 v2:172.21.15.37:6800/1690341662 1 ==== osd_op_reply(19 [pgnls start_epoch 37 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7fd3f001b040 con 0x7fd3f4023940 2024-04-25T10:30:25.240 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:25.238+0000 7fd4001e0640 1 -- 172.21.15.37:0/3348722167 --> [v2:172.21.15.203:6808/3688099014,v1:172.21.15.203:6809/3688099014] -- osd_op(unknown.0.0:20 4.19 4:98000000:%01:::head [pgnls start_epoch 37 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e37) -- 0x5566fa9f23e0 con 0x7fd3f4011900 2024-04-25T10:30:25.240 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:25.238+0000 7fd4001e0640 1 -- 172.21.15.37:0/3348722167 <== osd.7 v2:172.21.15.203:6808/3688099014 2 ==== osd_op_reply(20 [pgnls start_epoch 37 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7fd3f001b040 con 0x7fd3f4011900 2024-04-25T10:30:25.240 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:25.238+0000 7fd4001e0640 1 -- 172.21.15.37:0/3348722167 --> [v2:172.21.15.37:6824/1815290618,v1:172.21.15.37:6825/1815290618] -- osd_op(unknown.0.0:21 4.5 4:a0000000:%01:::head [pgnls start_epoch 37 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e37) -- 0x5566fa9f23e0 con 0x7fd3f001e8d0 2024-04-25T10:30:25.241 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:25.239+0000 7fd3ff9df640 1 -- 172.21.15.37:0/3348722167 <== osd.3 v2:172.21.15.37:6824/1815290618 2 ==== osd_op_reply(21 [pgnls start_epoch 37 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7fd3f401e040 con 0x7fd3f001e8d0 2024-04-25T10:30:25.241 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:25.239+0000 7fd3ff9df640 1 -- 172.21.15.37:0/3348722167 --> [v2:172.21.15.37:6824/1815290618,v1:172.21.15.37:6825/1815290618] -- osd_op(unknown.0.0:22 4.15 4:a8000000:%01:::head [pgnls start_epoch 37 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e37) -- 0x5566fa9f23e0 con 0x7fd3f001e8d0 2024-04-25T10:30:25.241 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:25.239+0000 7fd3ff9df640 1 -- 172.21.15.37:0/3348722167 <== osd.3 v2:172.21.15.37:6824/1815290618 3 ==== osd_op_reply(22 [pgnls start_epoch 37 out=97b] v35'3 uv3 ondisk = 1) ==== 144+0+97 (crc 0 0 0) 0x7fd3f401e040 con 0x7fd3f001e8d0 2024-04-25T10:30:25.241 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:25.239+0000 7fd3ff9df640 1 -- 172.21.15.37:0/3348722167 --> [v2:172.21.15.37:6824/1815290618,v1:172.21.15.37:6825/1815290618] -- osd_op(unknown.0.0:23 4.d 4:b0000000:%01:::head [pgnls start_epoch 37 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e37) -- 0x5566fa9f23e0 con 0x7fd3f001e8d0 2024-04-25T10:30:25.242 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:25.240+0000 7fd3ff9df640 1 -- 172.21.15.37:0/3348722167 <== osd.3 v2:172.21.15.37:6824/1815290618 4 ==== osd_op_reply(23 [pgnls start_epoch 37 out=74b] v35'1 uv1 ondisk = 1) ==== 144+0+74 (crc 0 0 0) 0x7fd3f401e040 con 0x7fd3f001e8d0 2024-04-25T10:30:25.242 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:25.240+0000 7fd3ff9df640 1 -- 172.21.15.37:0/3348722167 --> [v2:172.21.15.203:6808/3688099014,v1:172.21.15.203:6809/3688099014] -- osd_op(unknown.0.0:24 4.1d 4:b8000000:%01:::head [pgnls start_epoch 37 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e37) -- 0x5566fa9f23e0 con 0x7fd3f4011900 2024-04-25T10:30:25.242 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:25.240+0000 7fd4001e0640 1 -- 172.21.15.37:0/3348722167 <== osd.7 v2:172.21.15.203:6808/3688099014 3 ==== osd_op_reply(24 [pgnls start_epoch 37 out=73b] v35'2 uv2 ondisk = 1) ==== 144+0+73 (crc 0 0 0) 0x7fd3f001b040 con 0x7fd3f4011900 2024-04-25T10:30:25.242 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:25.240+0000 7fd4001e0640 1 -- 172.21.15.37:0/3348722167 --> [v2:172.21.15.203:6810/2309615014,v1:172.21.15.203:6812/2309615014] -- osd_op(unknown.0.0:25 4.3 4:c0000000:%01:::head [pgnls start_epoch 37 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e37) -- 0x5566fa9f23e0 con 0x7fd3f8022b50 2024-04-25T10:30:25.242 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:25.240+0000 7fd4001e0640 1 -- 172.21.15.37:0/3348722167 <== osd.6 v2:172.21.15.203:6810/2309615014 4 ==== osd_op_reply(25 [pgnls start_epoch 37 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7fd3f001b040 con 0x7fd3f8022b50 2024-04-25T10:30:25.243 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:25.240+0000 7fd4001e0640 1 -- 172.21.15.37:0/3348722167 --> [v2:172.21.15.37:6808/2754669649,v1:172.21.15.37:6809/2754669649] -- osd_op(unknown.0.0:26 4.13 4:c8000000:%01:::head [pgnls start_epoch 37 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e37) -- 0x5566fa9f23e0 con 0x7fd3f401e7e0 2024-04-25T10:30:25.243 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:25.241+0000 7fd4009e1640 1 -- 172.21.15.37:0/3348722167 <== osd.2 v2:172.21.15.37:6808/2754669649 6 ==== osd_op_reply(26 [pgnls start_epoch 37 out=98b] v36'3 uv3 ondisk = 1) ==== 144+0+98 (crc 0 0 0) 0x7fd3f809edf0 con 0x7fd3f401e7e0 2024-04-25T10:30:25.243 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:25.241+0000 7fd4009e1640 1 -- 172.21.15.37:0/3348722167 --> [v2:172.21.15.203:6824/1203822890,v1:172.21.15.203:6825/1203822890] -- osd_op(unknown.0.0:27 4.b 4:d0000000:%01:::head [pgnls start_epoch 37 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e37) -- 0x5566fa9f23e0 con 0x7fd3f801f4b0 2024-04-25T10:30:25.243 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:25.241+0000 7fd4009e1640 1 -- 172.21.15.37:0/3348722167 <== osd.5 v2:172.21.15.203:6824/1203822890 3 ==== osd_op_reply(27 [pgnls start_epoch 37 out=76b] v36'3 uv3 ondisk = 1) ==== 144+0+76 (crc 0 0 0) 0x7fd3f809edf0 con 0x7fd3f801f4b0 2024-04-25T10:30:25.243 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:25.241+0000 7fd4009e1640 1 -- 172.21.15.37:0/3348722167 --> [v2:172.21.15.37:6808/2754669649,v1:172.21.15.37:6809/2754669649] -- osd_op(unknown.0.0:28 4.1b 4:d8000000:%01:::head [pgnls start_epoch 37 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e37) -- 0x5566fa9f23e0 con 0x7fd3f401e7e0 2024-04-25T10:30:25.244 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:25.242+0000 7fd4009e1640 1 -- 172.21.15.37:0/3348722167 <== osd.2 v2:172.21.15.37:6808/2754669649 7 ==== osd_op_reply(28 [pgnls start_epoch 37 out=49b] v35'1 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7fd3f809edf0 con 0x7fd3f401e7e0 2024-04-25T10:30:25.244 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:25.242+0000 7fd4009e1640 1 -- 172.21.15.37:0/3348722167 --> [v2:172.21.15.203:6810/2309615014,v1:172.21.15.203:6812/2309615014] -- osd_op(unknown.0.0:29 4.7 4:e0000000:%01:::head [pgnls start_epoch 37 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e37) -- 0x5566fa9f23e0 con 0x7fd3f8022b50 2024-04-25T10:30:25.245 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:25.242+0000 7fd4001e0640 1 -- 172.21.15.37:0/3348722167 <== osd.6 v2:172.21.15.203:6810/2309615014 5 ==== osd_op_reply(29 [pgnls start_epoch 37 out=73b] v35'2 uv2 ondisk = 1) ==== 144+0+73 (crc 0 0 0) 0x7fd3f001b040 con 0x7fd3f8022b50 2024-04-25T10:30:25.245 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:25.242+0000 7fd4001e0640 1 -- 172.21.15.37:0/3348722167 --> [v2:172.21.15.203:6824/1203822890,v1:172.21.15.203:6825/1203822890] -- osd_op(unknown.0.0:30 4.17 4:e8000000:%01:::head [pgnls start_epoch 37 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e37) -- 0x5566fa9f23e0 con 0x7fd3f801f4b0 2024-04-25T10:30:25.245 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:25.243+0000 7fd4009e1640 1 -- 172.21.15.37:0/3348722167 <== osd.5 v2:172.21.15.203:6824/1203822890 4 ==== osd_op_reply(30 [pgnls start_epoch 37 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7fd3f809edf0 con 0x7fd3f801f4b0 2024-04-25T10:30:25.245 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:25.243+0000 7fd4009e1640 1 -- 172.21.15.37:0/3348722167 --> [v2:172.21.15.203:6808/3688099014,v1:172.21.15.203:6809/3688099014] -- osd_op(unknown.0.0:31 4.f 4:f0000000:%01:::head [pgnls start_epoch 37 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e37) -- 0x5566fa9f23e0 con 0x7fd3f4011900 2024-04-25T10:30:25.247 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:25.243+0000 7fd4001e0640 1 -- 172.21.15.37:0/3348722167 <== osd.7 v2:172.21.15.203:6808/3688099014 4 ==== osd_op_reply(31 [pgnls start_epoch 37 out=73b] v35'1 uv1 ondisk = 1) ==== 144+0+73 (crc 0 0 0) 0x7fd3f001b040 con 0x7fd3f4011900 2024-04-25T10:30:25.247 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:25.243+0000 7fd4001e0640 1 -- 172.21.15.37:0/3348722167 --> [v2:172.21.15.37:6824/1815290618,v1:172.21.15.37:6825/1815290618] -- osd_op(unknown.0.0:32 4.1f 4:f8000000:%01:::head [pgnls start_epoch 37 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e37) -- 0x5566fa9f23e0 con 0x7fd3f001e8d0 2024-04-25T10:30:25.247 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:25.243+0000 7fd3ff9df640 1 -- 172.21.15.37:0/3348722167 <== osd.3 v2:172.21.15.37:6824/1815290618 5 ==== osd_op_reply(32 [pgnls start_epoch 37 out=101b] v36'6 uv6 ondisk = 1) ==== 144+0+101 (crc 0 0 0) 0x7fd3f401e040 con 0x7fd3f001e8d0 2024-04-25T10:30:25.247 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:25.243+0000 7fd406716f40 1 -- 172.21.15.37:0/3348722167 --> [v2:172.21.15.203:6808/3688099014,v1:172.21.15.203:6809/3688099014] -- osd_op(unknown.0.0:33 4.10 4:09401a81:::601.00000000:head [delete] snapc 0=[] ondisk+write+known_if_redirected+full_try+supports_pool_eio e37) -- 0x5566faa958e0 con 0x7fd3f4011900 2024-04-25T10:30:25.247 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:25.243+0000 7fd406716f40 1 -- 172.21.15.37:0/3348722167 --> [v2:172.21.15.203:6808/3688099014,v1:172.21.15.203:6809/3688099014] -- osd_op(unknown.0.0:34 4.10 4:0bd6d154:::602.00000000:head [delete] snapc 0=[] ondisk+write+known_if_redirected+full_try+supports_pool_eio e37) -- 0x5566faa961b0 con 0x7fd3f4011900 2024-04-25T10:30:25.247 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:25.243+0000 7fd406716f40 1 -- 172.21.15.37:0/3348722167 --> [v2:172.21.15.203:6808/3688099014,v1:172.21.15.203:6809/3688099014] -- osd_op(unknown.0.0:35 4.10 4:0cf6ad0f:::10000000000.00000000:head [delete] snapc 0=[] ondisk+write+known_if_redirected+full_try+supports_pool_eio e37) -- 0x5566faa99140 con 0x7fd3f4011900 2024-04-25T10:30:25.247 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:25.243+0000 7fd406716f40 1 -- 172.21.15.37:0/3348722167 --> [v2:172.21.15.203:6808/3688099014,v1:172.21.15.203:6809/3688099014] -- osd_op(unknown.0.0:36 4.10 4:0d82a743:::600.00000000:head [delete] snapc 0=[] ondisk+write+known_if_redirected+full_try+supports_pool_eio e37) -- 0x5566faa99990 con 0x7fd3f4011900 2024-04-25T10:30:25.247 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:25.243+0000 7fd406716f40 1 -- 172.21.15.37:0/3348722167 --> [v2:172.21.15.203:6808/3688099014,v1:172.21.15.203:6809/3688099014] -- osd_op(unknown.0.0:37 4.10 4:0d89b25e:::603.00000000:head [delete] snapc 0=[] ondisk+write+known_if_redirected+full_try+supports_pool_eio e37) -- 0x5566faa9a1e0 con 0x7fd3f4011900 2024-04-25T10:30:25.247 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:25.243+0000 7fd406716f40 1 -- 172.21.15.37:0/3348722167 --> [v2:172.21.15.203:6810/2309615014,v1:172.21.15.203:6812/2309615014] -- osd_op(unknown.0.0:38 4.14 4:287034c4:::1.00000000.inode:head [delete] snapc 0=[] ondisk+write+known_if_redirected+full_try+supports_pool_eio e37) -- 0x5566faa9aa30 con 0x7fd3f8022b50 2024-04-25T10:30:25.248 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:25.243+0000 7fd406716f40 1 -- 172.21.15.37:0/3348722167 --> [v2:172.21.15.203:6810/2309615014,v1:172.21.15.203:6812/2309615014] -- osd_op(unknown.0.0:39 4.14 4:292cf221:::200.00000000:head [delete] snapc 0=[] ondisk+write+known_if_redirected+full_try+supports_pool_eio e37) -- 0x5566faa9b280 con 0x7fd3f8022b50 2024-04-25T10:30:25.248 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:25.243+0000 7fd406716f40 1 -- 172.21.15.37:0/3348722167 --> [v2:172.21.15.203:6810/2309615014,v1:172.21.15.203:6812/2309615014] -- osd_op(unknown.0.0:40 4.14 4:2e2fa760:::200.00000001:head [delete] snapc 0=[] ondisk+write+known_if_redirected+full_try+supports_pool_eio e37) -- 0x5566faa9bad0 con 0x7fd3f8022b50 2024-04-25T10:30:25.248 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:25.243+0000 7fd406716f40 1 -- 172.21.15.37:0/3348722167 --> [v2:172.21.15.203:6810/2309615014,v1:172.21.15.203:6812/2309615014] -- osd_op(unknown.0.0:41 4.14 4:2f55791f:::606.00000000:head [delete] snapc 0=[] ondisk+write+known_if_redirected+full_try+supports_pool_eio e37) -- 0x5566faa9c470 con 0x7fd3f8022b50 2024-04-25T10:30:25.248 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:25.243+0000 7fd406716f40 1 -- 172.21.15.37:0/3348722167 --> [v2:172.21.15.203:6800/4051532063,v1:172.21.15.203:6801/4051532063] -- osd_op(unknown.0.0:42 4.1c 4:3ed09add:::607.00000000:head [delete] snapc 0=[] ondisk+write+known_if_redirected+full_try+supports_pool_eio e37) -- 0x5566faa9cf30 con 0x5566fa9f1fd0 2024-04-25T10:30:25.248 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:25.243+0000 7fd406716f40 1 -- 172.21.15.37:0/3348722167 --> [v2:172.21.15.203:6824/1203822890,v1:172.21.15.203:6825/1203822890] -- osd_op(unknown.0.0:43 4.6 4:654134d2:::mds0_openfiles.0:head [delete] snapc 0=[] ondisk+write+known_if_redirected+full_try+supports_pool_eio e37) -- 0x5566faa9dae0 con 0x7fd3f801f4b0 2024-04-25T10:30:25.248 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:25.243+0000 7fd406716f40 1 -- 172.21.15.37:0/3348722167 --> [v2:172.21.15.37:6808/2754669649,v1:172.21.15.37:6809/2754669649] -- osd_op(unknown.0.0:44 4.e 4:717a0223:::608.00000000:head [delete] snapc 0=[] ondisk+write+known_if_redirected+full_try+supports_pool_eio e37) -- 0x5566faa9e630 con 0x7fd3f401e7e0 2024-04-25T10:30:25.248 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:25.243+0000 7fd406716f40 1 -- 172.21.15.37:0/3348722167 --> [v2:172.21.15.37:6824/1815290618,v1:172.21.15.37:6825/1815290618] -- osd_op(unknown.0.0:45 4.15 4:a93a17c2:::604.00000000:head [delete] snapc 0=[] ondisk+write+known_if_redirected+full_try+supports_pool_eio e37) -- 0x5566faa9f1a0 con 0x7fd3f001e8d0 2024-04-25T10:30:25.248 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:25.243+0000 7fd406716f40 1 -- 172.21.15.37:0/3348722167 --> [v2:172.21.15.37:6824/1815290618,v1:172.21.15.37:6825/1815290618] -- osd_op(unknown.0.0:46 4.15 4:aa448500:::500.00000000:head [delete] snapc 0=[] ondisk+write+known_if_redirected+full_try+supports_pool_eio e37) -- 0x5566faa9fd20 con 0x7fd3f001e8d0 2024-04-25T10:30:25.248 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:25.243+0000 7fd406716f40 1 -- 172.21.15.37:0/3348722167 --> [v2:172.21.15.37:6824/1815290618,v1:172.21.15.37:6825/1815290618] -- osd_op(unknown.0.0:47 4.d 4:b50e409b:::mds_snaptable:head [delete] snapc 0=[] ondisk+write+known_if_redirected+full_try+supports_pool_eio e37) -- 0x5566faa9bad0 con 0x7fd3f001e8d0 2024-04-25T10:30:25.248 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:25.243+0000 7fd406716f40 1 -- 172.21.15.37:0/3348722167 --> [v2:172.21.15.203:6808/3688099014,v1:172.21.15.203:6809/3688099014] -- osd_op(unknown.0.0:48 4.1d 4:b871830b:::605.00000000:head [delete] snapc 0=[] ondisk+write+known_if_redirected+full_try+supports_pool_eio e37) -- 0x5566faaa0cf0 con 0x7fd3f4011900 2024-04-25T10:30:25.249 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:25.246+0000 7fd4009e1640 1 -- 172.21.15.37:0/3348722167 <== osd.2 v2:172.21.15.37:6808/2754669649 8 ==== osd_op_reply(44 608.00000000 [delete] v37'3 uv3 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7fd3f809edf0 con 0x7fd3f401e7e0 2024-04-25T10:30:25.249 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:25.246+0000 7fd3ff9df640 1 -- 172.21.15.37:0/3348722167 <== osd.3 v2:172.21.15.37:6824/1815290618 6 ==== osd_op_reply(45 604.00000000 [delete] v37'4 uv4 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7fd3f401e040 con 0x7fd3f001e8d0 2024-04-25T10:30:25.249 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:25.246+0000 7fd406716f40 1 -- 172.21.15.37:0/3348722167 --> [v2:172.21.15.37:6808/2754669649,v1:172.21.15.37:6809/2754669649] -- osd_op(unknown.0.0:49 4.13 4:c91d4a1d:::mds0_inotable:head [delete] snapc 0=[] ondisk+write+known_if_redirected+full_try+supports_pool_eio e37) -- 0x5566faa9ebd0 con 0x7fd3f401e7e0 2024-04-25T10:30:25.249 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:25.246+0000 7fd406716f40 1 -- 172.21.15.37:0/3348722167 --> [v2:172.21.15.37:6808/2754669649,v1:172.21.15.37:6809/2754669649] -- osd_op(unknown.0.0:50 4.13 4:cd5a64a3:::100.00000000:head [delete] snapc 0=[] ondisk+write+known_if_redirected+full_try+supports_pool_eio e37) -- 0x5566faaa11a0 con 0x7fd3f401e7e0 2024-04-25T10:30:25.250 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:25.248+0000 7fd4001e0640 1 -- 172.21.15.37:0/3348722167 <== osd.7 v2:172.21.15.203:6808/3688099014 5 ==== osd_op_reply(33 601.00000000 [delete] v37'11 uv11 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7fd3f001b040 con 0x7fd3f4011900 2024-04-25T10:30:25.250 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:25.248+0000 7fd406716f40 1 -- 172.21.15.37:0/3348722167 --> [v2:172.21.15.203:6824/1203822890,v1:172.21.15.203:6825/1203822890] -- osd_op(unknown.0.0:51 4.b 4:d0630e4c:::mds0_sessionmap:head [delete] snapc 0=[] ondisk+write+known_if_redirected+full_try+supports_pool_eio e37) -- 0x5566faaa1540 con 0x7fd3f801f4b0 2024-04-25T10:30:25.250 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:25.248+0000 7fd4001e0640 1 -- 172.21.15.37:0/3348722167 <== osd.6 v2:172.21.15.203:6810/2309615014 6 ==== osd_op_reply(38 1.00000000.inode [delete] v37'20 uv20 ondisk = 0) ==== 160+0+0 (crc 0 0 0) 0x7fd3f001b040 con 0x7fd3f8022b50 2024-04-25T10:30:25.250 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:25.248+0000 7fd4001e0640 1 -- 172.21.15.37:0/3348722167 <== osd.7 v2:172.21.15.203:6808/3688099014 6 ==== osd_op_reply(48 605.00000000 [delete] v37'3 uv3 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7fd3f001b040 con 0x7fd3f4011900 2024-04-25T10:30:25.251 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:25.248+0000 7fd406716f40 1 -- 172.21.15.37:0/3348722167 --> [v2:172.21.15.203:6810/2309615014,v1:172.21.15.203:6812/2309615014] -- osd_op(unknown.0.0:52 4.7 4:e0b41d06:::609.00000000:head [delete] snapc 0=[] ondisk+write+known_if_redirected+full_try+supports_pool_eio e37) -- 0x5566faaa18e0 con 0x7fd3f8022b50 2024-04-25T10:30:25.251 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:25.248+0000 7fd3ff9df640 1 -- 172.21.15.37:0/3348722167 <== osd.3 v2:172.21.15.37:6824/1815290618 7 ==== osd_op_reply(47 mds_snaptable [delete] v37'2 uv2 ondisk = 0) ==== 157+0+0 (crc 0 0 0) 0x7fd3f401e040 con 0x7fd3f001e8d0 2024-04-25T10:30:25.251 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:25.248+0000 7fd406716f40 1 -- 172.21.15.37:0/3348722167 --> [v2:172.21.15.203:6808/3688099014,v1:172.21.15.203:6809/3688099014] -- osd_op(unknown.0.0:53 4.f 4:f1f69726:::400.00000000:head [delete] snapc 0=[] ondisk+write+known_if_redirected+full_try+supports_pool_eio e37) -- 0x5566faaa18e0 con 0x7fd3f4011900 2024-04-25T10:30:25.251 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:25.248+0000 7fd406716f40 1 -- 172.21.15.37:0/3348722167 --> [v2:172.21.15.37:6824/1815290618,v1:172.21.15.37:6825/1815290618] -- osd_op(unknown.0.0:54 4.1f 4:fe07bba1:::100.00000000.inode:head [delete] snapc 0=[] ondisk+write+known_if_redirected+full_try+supports_pool_eio e37) -- 0x5566faaa1d30 con 0x7fd3f001e8d0 2024-04-25T10:30:25.251 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:25.248+0000 7fd3ff9df640 1 -- 172.21.15.37:0/3348722167 <== osd.4 v2:172.21.15.203:6800/4051532063 6 ==== osd_op_reply(42 607.00000000 [delete] v37'3 uv3 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7fd3f401e040 con 0x5566fa9f1fd0 2024-04-25T10:30:25.251 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:25.248+0000 7fd406716f40 1 -- 172.21.15.37:0/3348722167 --> [v2:172.21.15.37:6824/1815290618,v1:172.21.15.37:6825/1815290618] -- osd_op(unknown.0.0:55 4.1f 4:ff5b34d6:::1.00000000:head [delete] snapc 0=[] ondisk+write+known_if_redirected+full_try+supports_pool_eio e37) -- 0x5566faaa18e0 con 0x7fd3f001e8d0 2024-04-25T10:30:25.253 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:25.251+0000 7fd4001e0640 1 -- 172.21.15.37:0/3348722167 <== osd.7 v2:172.21.15.203:6808/3688099014 7 ==== osd_op_reply(34 602.00000000 [delete] v37'12 uv12 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7fd3f001b040 con 0x7fd3f4011900 2024-04-25T10:30:25.254 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:25.251+0000 7fd4009e1640 1 -- 172.21.15.37:0/3348722167 <== osd.5 v2:172.21.15.203:6824/1203822890 5 ==== osd_op_reply(43 mds0_openfiles.0 [delete] v37'4 uv4 ondisk = 0) ==== 160+0+0 (crc 0 0 0) 0x7fd3f809edf0 con 0x7fd3f801f4b0 2024-04-25T10:30:25.254 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:25.251+0000 7fd3ff9df640 1 -- 172.21.15.37:0/3348722167 <== osd.3 v2:172.21.15.37:6824/1815290618 8 ==== osd_op_reply(46 500.00000000 [delete] v37'5 uv5 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7fd3f401e040 con 0x7fd3f001e8d0 2024-04-25T10:30:25.254 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:25.251+0000 7fd4001e0640 1 -- 172.21.15.37:0/3348722167 <== osd.6 v2:172.21.15.203:6810/2309615014 7 ==== osd_op_reply(39 200.00000000 [delete] v37'21 uv21 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7fd3f001b040 con 0x7fd3f8022b50 2024-04-25T10:30:25.254 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:25.251+0000 7fd4009e1640 1 -- 172.21.15.37:0/3348722167 <== osd.2 v2:172.21.15.37:6808/2754669649 9 ==== osd_op_reply(49 mds0_inotable [delete] v37'4 uv4 ondisk = 0) ==== 157+0+0 (crc 0 0 0) 0x7fd3f809edf0 con 0x7fd3f401e7e0 2024-04-25T10:30:25.254 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:25.251+0000 7fd4001e0640 1 -- 172.21.15.37:0/3348722167 <== osd.7 v2:172.21.15.203:6808/3688099014 8 ==== osd_op_reply(35 10000000000.00000000 [delete] v37'13 uv13 ondisk = 0) ==== 164+0+0 (crc 0 0 0) 0x7fd3f001b040 con 0x7fd3f4011900 2024-04-25T10:30:25.254 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:25.251+0000 7fd4001e0640 1 -- 172.21.15.37:0/3348722167 <== osd.6 v2:172.21.15.203:6810/2309615014 8 ==== osd_op_reply(40 200.00000001 [delete] v37'22 uv22 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7fd3f001b040 con 0x7fd3f8022b50 2024-04-25T10:30:25.254 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:25.251+0000 7fd4009e1640 1 -- 172.21.15.37:0/3348722167 <== osd.2 v2:172.21.15.37:6808/2754669649 10 ==== osd_op_reply(50 100.00000000 [delete] v37'5 uv5 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7fd3f809edf0 con 0x7fd3f401e7e0 2024-04-25T10:30:25.254 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:25.251+0000 7fd4001e0640 1 -- 172.21.15.37:0/3348722167 <== osd.6 v2:172.21.15.203:6810/2309615014 9 ==== osd_op_reply(41 606.00000000 [delete] v37'23 uv23 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7fd3f001b040 con 0x7fd3f8022b50 2024-04-25T10:30:25.254 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:25.251+0000 7fd4001e0640 1 -- 172.21.15.37:0/3348722167 <== osd.7 v2:172.21.15.203:6808/3688099014 9 ==== osd_op_reply(36 600.00000000 [delete] v37'14 uv14 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7fd3f001b040 con 0x7fd3f4011900 2024-04-25T10:30:25.254 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:25.251+0000 7fd4001e0640 1 -- 172.21.15.37:0/3348722167 <== osd.7 v2:172.21.15.203:6808/3688099014 10 ==== osd_op_reply(37 603.00000000 [delete] v37'15 uv15 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7fd3f001b040 con 0x7fd3f4011900 2024-04-25T10:30:25.254 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:25.252+0000 7fd4001e0640 1 -- 172.21.15.37:0/3348722167 <== osd.7 v2:172.21.15.203:6808/3688099014 11 ==== osd_op_reply(53 400.00000000 [delete] v37'2 uv2 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7fd3f001b040 con 0x7fd3f4011900 2024-04-25T10:30:25.254 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:25.252+0000 7fd4009e1640 1 -- 172.21.15.37:0/3348722167 <== osd.5 v2:172.21.15.203:6824/1203822890 6 ==== osd_op_reply(51 mds0_sessionmap [delete] v37'4 uv4 ondisk = 0) ==== 159+0+0 (crc 0 0 0) 0x7fd3f809edf0 con 0x7fd3f801f4b0 2024-04-25T10:30:25.255 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:25.252+0000 7fd4001e0640 1 -- 172.21.15.37:0/3348722167 <== osd.6 v2:172.21.15.203:6810/2309615014 10 ==== osd_op_reply(52 609.00000000 [delete] v37'3 uv3 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7fd3f001b040 con 0x7fd3f8022b50 2024-04-25T10:30:25.256 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:25.254+0000 7fd3ff9df640 1 -- 172.21.15.37:0/3348722167 <== osd.3 v2:172.21.15.37:6824/1815290618 9 ==== osd_op_reply(54 100.00000000.inode [delete] v37'7 uv7 ondisk = 0) ==== 162+0+0 (crc 0 0 0) 0x7fd3f401e040 con 0x7fd3f001e8d0 2024-04-25T10:30:25.256 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:25.254+0000 7fd3ff9df640 1 -- 172.21.15.37:0/3348722167 <== osd.3 v2:172.21.15.37:6824/1815290618 10 ==== osd_op_reply(55 1.00000000 [delete] v37'8 uv8 ondisk = 0) ==== 154+0+0 (crc 0 0 0) 0x7fd3f401e040 con 0x7fd3f001e8d0 2024-04-25T10:30:25.256 INFO:teuthology.orchestra.run.smithi037.stdout:Removed 23 objects 2024-04-25T10:30:25.256 INFO:teuthology.orchestra.run.smithi037.stdout:successfully purged pool cephfs_metadata 2024-04-25T10:30:25.258 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:25.254+0000 7fd406716f40 1 -- 172.21.15.37:0/3348722167 >> [v2:172.21.15.37:6816/2746074720,v1:172.21.15.37:6817/2746074720] conn(0x7fd3f0021950 msgr2=0x7fd3f001d350 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:30:25.258 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:25.254+0000 7fd406716f40 1 --2- 172.21.15.37:0/3348722167 >> [v2:172.21.15.37:6816/2746074720,v1:172.21.15.37:6817/2746074720] conn(0x7fd3f0021950 0x7fd3f001d350 crc :-1 s=READY pgs=12 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:30:25.258 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:25.254+0000 7fd406716f40 1 -- 172.21.15.37:0/3348722167 >> [v2:172.21.15.37:6800/1690341662,v1:172.21.15.37:6801/1690341662] conn(0x7fd3f4023940 msgr2=0x7fd3f40253f0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:30:25.258 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:25.254+0000 7fd406716f40 1 --2- 172.21.15.37:0/3348722167 >> [v2:172.21.15.37:6800/1690341662,v1:172.21.15.37:6801/1690341662] conn(0x7fd3f4023940 0x7fd3f40253f0 crc :-1 s=READY pgs=13 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:30:25.258 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:25.254+0000 7fd406716f40 1 -- 172.21.15.37:0/3348722167 >> [v2:172.21.15.37:6808/2754669649,v1:172.21.15.37:6809/2754669649] conn(0x7fd3f401e7e0 msgr2=0x7fd3f400d2f0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:30:25.258 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:25.254+0000 7fd406716f40 1 --2- 172.21.15.37:0/3348722167 >> [v2:172.21.15.37:6808/2754669649,v1:172.21.15.37:6809/2754669649] conn(0x7fd3f401e7e0 0x7fd3f400d2f0 crc :-1 s=READY pgs=12 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:30:25.259 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:25.254+0000 7fd406716f40 1 -- 172.21.15.37:0/3348722167 >> [v2:172.21.15.37:6824/1815290618,v1:172.21.15.37:6825/1815290618] conn(0x7fd3f001e8d0 msgr2=0x7fd3f00113c0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:30:25.259 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:25.254+0000 7fd406716f40 1 --2- 172.21.15.37:0/3348722167 >> [v2:172.21.15.37:6824/1815290618,v1:172.21.15.37:6825/1815290618] conn(0x7fd3f001e8d0 0x7fd3f00113c0 crc :-1 s=READY pgs=13 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:30:25.259 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:25.254+0000 7fd4009e1640 1 -- 172.21.15.37:0/3348722167 reap_dead start 2024-04-25T10:30:25.259 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:25.254+0000 7fd406716f40 1 -- 172.21.15.37:0/3348722167 >> [v2:172.21.15.203:6800/4051532063,v1:172.21.15.203:6801/4051532063] conn(0x5566fa9f1fd0 msgr2=0x5566faa98750 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:30:25.259 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:25.254+0000 7fd406716f40 1 --2- 172.21.15.37:0/3348722167 >> [v2:172.21.15.203:6800/4051532063,v1:172.21.15.203:6801/4051532063] conn(0x5566fa9f1fd0 0x5566faa98750 crc :-1 s=READY pgs=11 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:30:25.259 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:25.254+0000 7fd406716f40 1 -- 172.21.15.37:0/3348722167 >> [v2:172.21.15.203:6824/1203822890,v1:172.21.15.203:6825/1203822890] conn(0x7fd3f801f4b0 msgr2=0x7fd3f8004600 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:30:25.259 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:25.254+0000 7fd406716f40 1 --2- 172.21.15.37:0/3348722167 >> [v2:172.21.15.203:6824/1203822890,v1:172.21.15.203:6825/1203822890] conn(0x7fd3f801f4b0 0x7fd3f8004600 crc :-1 s=READY pgs=12 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:30:25.259 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:25.254+0000 7fd406716f40 1 -- 172.21.15.37:0/3348722167 >> [v2:172.21.15.203:6810/2309615014,v1:172.21.15.203:6812/2309615014] conn(0x7fd3f8022b50 msgr2=0x7fd3f80d82c0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:30:25.259 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:25.254+0000 7fd406716f40 1 --2- 172.21.15.37:0/3348722167 >> [v2:172.21.15.203:6810/2309615014,v1:172.21.15.203:6812/2309615014] conn(0x7fd3f8022b50 0x7fd3f80d82c0 crc :-1 s=READY pgs=11 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:30:25.259 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:25.254+0000 7fd406716f40 1 -- 172.21.15.37:0/3348722167 >> [v2:172.21.15.203:6808/3688099014,v1:172.21.15.203:6809/3688099014] conn(0x7fd3f4011900 msgr2=0x7fd3f400e300 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:30:25.259 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:25.254+0000 7fd406716f40 1 --2- 172.21.15.37:0/3348722167 >> [v2:172.21.15.203:6808/3688099014,v1:172.21.15.203:6809/3688099014] conn(0x7fd3f4011900 0x7fd3f400e300 crc :-1 s=READY pgs=12 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:30:25.259 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:25.254+0000 7fd4009e1640 1 -- 172.21.15.37:0/3348722167 reap_dead start 2024-04-25T10:30:25.259 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:25.254+0000 7fd406716f40 1 -- 172.21.15.37:0/3348722167 >> [v2:172.21.15.203:6832/2418205237,v1:172.21.15.203:6833/2418205237] conn(0x7fd3ec072e90 msgr2=0x7fd3ec0753a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:30:25.259 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:25.254+0000 7fd406716f40 1 --2- 172.21.15.37:0/3348722167 >> [v2:172.21.15.203:6832/2418205237,v1:172.21.15.203:6833/2418205237] conn(0x7fd3ec072e90 0x7fd3ec0753a0 secure :-1 s=READY pgs=236 cs=0 l=1 rev1=1 crypto rx=0x7fd3f0000c00 tx=0x7fd3f0003810 comp rx=0 tx=0).stop 2024-04-25T10:30:25.259 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:25.254+0000 7fd406716f40 1 -- 172.21.15.37:0/3348722167 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x5566fa9d8ec0 msgr2=0x5566fa9c43b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:30:25.260 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:25.254+0000 7fd406716f40 1 --2- 172.21.15.37:0/3348722167 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x5566fa9d8ec0 0x5566fa9c43b0 secure :-1 s=READY pgs=490 cs=0 l=1 rev1=1 crypto rx=0x7fd3f8064040 tx=0x7fd3f8003330 comp rx=0 tx=0).stop 2024-04-25T10:30:25.260 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:25.254+0000 7fd406716f40 1 -- 172.21.15.37:0/3348722167 shutdown_connections 2024-04-25T10:30:25.260 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:25.254+0000 7fd406716f40 1 --2- 172.21.15.37:0/3348722167 >> [v2:172.21.15.203:6832/2418205237,v1:172.21.15.203:6833/2418205237] conn(0x7fd3ec072e90 0x7fd3ec0753a0 unknown :-1 s=CLOSED pgs=236 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:30:25.260 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:25.254+0000 7fd406716f40 1 --2- 172.21.15.37:0/3348722167 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x5566fa9d8ec0 0x5566fa9c43b0 unknown :-1 s=CLOSED pgs=490 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:30:25.260 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:25.254+0000 7fd406716f40 1 -- 172.21.15.37:0/3348722167 >> 172.21.15.37:0/3348722167 conn(0x5566fa840ac0 msgr2=0x5566fa9f4ec0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:30:25.260 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:25.254+0000 7fd406716f40 1 -- 172.21.15.37:0/3348722167 shutdown_connections 2024-04-25T10:30:25.260 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:25.254+0000 7fd406716f40 1 -- 172.21.15.37:0/3348722167 wait complete. 2024-04-25T10:30:25.261 DEBUG:teuthology.orchestra.run.smithi037:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-04-25T10:30:25.372 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:25.370+0000 7f76b37a8640 1 Processor -- start 2024-04-25T10:30:25.372 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:25.370+0000 7f76b37a8640 1 -- start start 2024-04-25T10:30:25.374 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:25.371+0000 7f76b37a8640 1 --2- >> v2:172.21.15.203:3300/0 conn(0x7f76ac0b4c90 0x7f76ac0d29e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:30:25.374 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:25.371+0000 7f76b37a8640 1 -- --> v1:172.21.15.37:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f76ac09ce70 con 0x7f76ac06e260 2024-04-25T10:30:25.374 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:25.372+0000 7f76b37a8640 1 -- --> v1:172.21.15.203:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f76ac09d150 con 0x7f76ac0d1ec0 2024-04-25T10:30:25.375 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:25.372+0000 7f76b37a8640 1 -- --> v2:172.21.15.203:3300/0 -- mon_getmap magic: 0 -- 0x7f76ac0d3370 con 0x7f76ac0b4c90 2024-04-25T10:30:25.375 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:25.372+0000 7f76b1544640 1 --1- >> v1:172.21.15.37:6789/0 conn(0x7f76ac06e260 0x7f76ac06e670 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.37:6789/0 says I am v1:172.21.15.37:55182/0 (socket says 172.21.15.37:55182) 2024-04-25T10:30:25.375 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:25.372+0000 7f76b1544640 1 -- 172.21.15.37:0/1897549479 learned_addr learned my addr 172.21.15.37:0/1897549479 (peer_addr_for_me v1:172.21.15.37:0/0) 2024-04-25T10:30:25.375 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:25.372+0000 7f76b1d45640 1 --2- 172.21.15.37:0/1897549479 >> v2:172.21.15.203:3300/0 conn(0x7f76ac0b4c90 0x7f76ac0d29e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:30:25.375 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:25.372+0000 7f76a3fff640 1 -- 172.21.15.37:0/1897549479 <== mon.0 v1:172.21.15.37:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 3525262988 0 0) 0x7f76ac09ce70 con 0x7f76ac06e260 2024-04-25T10:30:25.375 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:25.372+0000 7f76a3fff640 1 -- 172.21.15.37:0/1897549479 --> v1:172.21.15.37:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f7698003520 con 0x7f76ac06e260 2024-04-25T10:30:25.375 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:25.372+0000 7f76a3fff640 1 -- 172.21.15.37:0/1897549479 <== mon.1 v1:172.21.15.203:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 2271248240 0 0) 0x7f76ac09d150 con 0x7f76ac0d1ec0 2024-04-25T10:30:25.375 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:25.372+0000 7f76a3fff640 1 -- 172.21.15.37:0/1897549479 --> v1:172.21.15.203:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f76ac09ce70 con 0x7f76ac0d1ec0 2024-04-25T10:30:25.375 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:25.372+0000 7f76b1d45640 1 -- 172.21.15.37:0/1897549479 >> v1:172.21.15.203:6789/0 conn(0x7f76ac0d1ec0 legacy=0x7f76ac0d22d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:30:25.375 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:25.372+0000 7f76b1d45640 1 -- 172.21.15.37:0/1897549479 >> v1:172.21.15.37:6789/0 conn(0x7f76ac06e260 legacy=0x7f76ac06e670 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:30:25.375 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:25.372+0000 7f76b1d45640 1 -- 172.21.15.37:0/1897549479 --> v2:172.21.15.203:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f76ac06ed80 con 0x7f76ac0b4c90 2024-04-25T10:30:25.377 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:25.373+0000 7f76b1d45640 1 --2- 172.21.15.37:0/1897549479 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f76ac0b4c90 0x7f76ac0d29e0 secure :-1 s=READY pgs=200 cs=0 l=1 rev1=1 crypto rx=0x7f76a4009d90 tx=0x7f76a402a660 comp rx=0 tx=0).ready entity=mon.1 client_cookie=b3dfe756921ce833 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:30:25.377 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:25.373+0000 7f76a3fff640 1 -- 172.21.15.37:0/1897549479 <== mon.1 v2:172.21.15.203:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f76a4007740 con 0x7f76ac0b4c90 2024-04-25T10:30:25.377 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:25.373+0000 7f76a3fff640 1 -- 172.21.15.37:0/1897549479 <== mon.1 v2:172.21.15.203:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f76a40078e0 con 0x7f76ac0b4c90 2024-04-25T10:30:25.377 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:25.373+0000 7f76a3fff640 1 -- 172.21.15.37:0/1897549479 <== mon.1 v2:172.21.15.203:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f76a4007c90 con 0x7f76ac0b4c90 2024-04-25T10:30:25.377 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:25.374+0000 7f76b37a8640 1 -- 172.21.15.37:0/1897549479 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f76ac0b4c90 msgr2=0x7f76ac0d29e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:30:25.377 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:25.374+0000 7f76b37a8640 1 --2- 172.21.15.37:0/1897549479 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f76ac0b4c90 0x7f76ac0d29e0 secure :-1 s=READY pgs=200 cs=0 l=1 rev1=1 crypto rx=0x7f76a4009d90 tx=0x7f76a402a660 comp rx=0 tx=0).stop 2024-04-25T10:30:25.377 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:25.374+0000 7f76b37a8640 1 -- 172.21.15.37:0/1897549479 shutdown_connections 2024-04-25T10:30:25.377 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:25.374+0000 7f76b37a8640 1 --2- 172.21.15.37:0/1897549479 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f76ac0b4c90 0x7f76ac0d29e0 unknown :-1 s=CLOSED pgs=200 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:30:25.377 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:25.374+0000 7f76b37a8640 1 -- 172.21.15.37:0/1897549479 >> 172.21.15.37:0/1897549479 conn(0x7f76ac0d0390 msgr2=0x7f76ac0d07a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:30:25.378 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:25.374+0000 7f76b37a8640 1 -- 172.21.15.37:0/1897549479 shutdown_connections 2024-04-25T10:30:25.378 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:25.374+0000 7f76b37a8640 1 -- 172.21.15.37:0/1897549479 wait complete. 2024-04-25T10:30:25.378 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:25.376+0000 7f76b37a8640 1 Processor -- start 2024-04-25T10:30:25.378 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:25.376+0000 7f76b37a8640 1 -- start start 2024-04-25T10:30:25.380 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:25.377+0000 7f76b37a8640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f76ac06e260 0x7f76ac0ccc60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:30:25.380 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:25.377+0000 7f76b37a8640 1 --2- >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f76ac0b4c90 0x7f76ac09d6a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:30:25.380 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:25.377+0000 7f76b37a8640 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f76ac0d1ec0 0x7f76ac09dbe0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:30:25.381 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:25.377+0000 7f76b37a8640 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 -- 0x7f76ac06d830 con 0x7f76ac06e260 2024-04-25T10:30:25.381 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:25.377+0000 7f76b37a8640 1 -- --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_getmap magic: 0 -- 0x7f76ac0d3090 con 0x7f76ac0d1ec0 2024-04-25T10:30:25.381 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:25.377+0000 7f76b37a8640 1 -- --> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] -- mon_getmap magic: 0 -- 0x7f76ac06db30 con 0x7f76ac0b4c90 2024-04-25T10:30:25.381 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:25.377+0000 7f76b1544640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f76ac06e260 0x7f76ac0ccc60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:30:25.381 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:25.377+0000 7f76b1544640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f76ac06e260 0x7f76ac0ccc60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.37:3300/0 says I am v2:172.21.15.37:59846/0 (socket says 172.21.15.37:59846) 2024-04-25T10:30:25.381 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:25.377+0000 7f76b1544640 1 -- 172.21.15.37:0/3485888200 learned_addr learned my addr 172.21.15.37:0/3485888200 (peer_addr_for_me v2:172.21.15.37:0/0) 2024-04-25T10:30:25.381 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:25.377+0000 7f76b1544640 1 -- 172.21.15.37:0/3485888200 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f76ac0b4c90 msgr2=0x7f76ac09d6a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:30:25.381 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:25.377+0000 7f76b0d43640 1 --2- 172.21.15.37:0/3485888200 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f76ac0b4c90 0x7f76ac09d6a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:30:25.381 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:25.377+0000 7f76b1d45640 1 --2- 172.21.15.37:0/3485888200 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f76ac0d1ec0 0x7f76ac09dbe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:30:25.381 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:25.378+0000 7f76b1544640 1 --2- 172.21.15.37:0/3485888200 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f76ac0b4c90 0x7f76ac09d6a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:30:25.381 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:25.378+0000 7f76b1544640 1 -- 172.21.15.37:0/3485888200 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f76ac0d1ec0 msgr2=0x7f76ac09dbe0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:30:25.381 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:25.378+0000 7f76b1544640 1 --2- 172.21.15.37:0/3485888200 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f76ac0d1ec0 0x7f76ac09dbe0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:30:25.381 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:25.378+0000 7f76b1544640 1 -- 172.21.15.37:0/3485888200 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f76ac09e440 con 0x7f76ac06e260 2024-04-25T10:30:25.381 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:25.378+0000 7f76b1544640 1 --2- 172.21.15.37:0/3485888200 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f76ac06e260 0x7f76ac0ccc60 secure :-1 s=READY pgs=497 cs=0 l=1 rev1=1 crypto rx=0x7f76a800c430 tx=0x7f76a800c8f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:30:25.382 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:25.378+0000 7f76a1ffb640 1 -- 172.21.15.37:0/3485888200 <== mon.0 v2:172.21.15.37:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f76a8017070 con 0x7f76ac06e260 2024-04-25T10:30:25.382 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:25.378+0000 7f76a1ffb640 1 -- 172.21.15.37:0/3485888200 <== mon.0 v2:172.21.15.37:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f76a8005ce0 con 0x7f76ac06e260 2024-04-25T10:30:25.382 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:25.378+0000 7f76b37a8640 1 -- 172.21.15.37:0/3485888200 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f76ac09e720 con 0x7f76ac06e260 2024-04-25T10:30:25.382 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:25.378+0000 7f76a1ffb640 1 -- 172.21.15.37:0/3485888200 <== mon.0 v2:172.21.15.37:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f76a80072f0 con 0x7f76ac06e260 2024-04-25T10:30:25.382 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:25.378+0000 7f76b37a8640 1 -- 172.21.15.37:0/3485888200 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f76ac09ece0 con 0x7f76ac06e260 2024-04-25T10:30:25.383 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:25.379+0000 7f76b37a8640 1 -- 172.21.15.37:0/3485888200 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f76ac06a140 con 0x7f76ac06e260 2024-04-25T10:30:25.383 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:25.379+0000 7f76a1ffb640 1 -- 172.21.15.37:0/3485888200 <== mon.0 v2:172.21.15.37:3300/0 4 ==== mgrmap(e 5) ==== 96782+0+0 (secure 0 0 0) 0x7f76a801a070 con 0x7f76ac06e260 2024-04-25T10:30:25.383 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:25.380+0000 7f76a1ffb640 1 --2- 172.21.15.37:0/3485888200 >> [v2:172.21.15.203:6832/2418205237,v1:172.21.15.203:6833/2418205237] conn(0x7f76900744e0 0x7f76900769f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:30:25.383 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:25.380+0000 7f76a1ffb640 1 -- 172.21.15.37:0/3485888200 <== mon.0 v2:172.21.15.37:3300/0 5 ==== osd_map(37..37 src has 1..37) ==== 5795+0+0 (secure 0 0 0) 0x7f76a80946f0 con 0x7f76ac06e260 2024-04-25T10:30:25.383 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:25.380+0000 7f76b0d43640 1 --2- 172.21.15.37:0/3485888200 >> [v2:172.21.15.203:6832/2418205237,v1:172.21.15.203:6833/2418205237] conn(0x7f76900744e0 0x7f76900769f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:30:25.383 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:25.381+0000 7f76b0d43640 1 --2- 172.21.15.37:0/3485888200 >> [v2:172.21.15.203:6832/2418205237,v1:172.21.15.203:6833/2418205237] conn(0x7f76900744e0 0x7f76900769f0 secure :-1 s=READY pgs=237 cs=0 l=1 rev1=1 crypto rx=0x7f769c007fd0 tx=0x7f769c009040 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:30:25.386 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:25.384+0000 7f76a1ffb640 1 -- 172.21.15.37:0/3485888200 <== mon.0 v2:172.21.15.37:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187246 (secure 0 0 0) 0x7f76a805ee20 con 0x7f76ac06e260 2024-04-25T10:30:25.577 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:25.575+0000 7f76b37a8640 1 -- 172.21.15.37:0/3485888200 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7f76ac06e7a0 con 0x7f76ac06e260 2024-04-25T10:30:25.578 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:25.576+0000 7f76a1ffb640 1 -- 172.21.15.37:0/3485888200 <== mon.0 v2:172.21.15.37:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v37) ==== 74+0+16748 (secure 0 0 0) 0x7f76a805e720 con 0x7f76ac06e260 2024-04-25T10:30:25.578 INFO:teuthology.orchestra.run.smithi037.stdout: 2024-04-25T10:30:25.579 INFO:teuthology.orchestra.run.smithi037.stdout:{"epoch":37,"fsid":"f4acda28-be34-492a-b286-994afd36f61d","created":"2024-04-25T10:28:43.985718+0000","modified":"2024-04-25T10:30:24.853759+0000","last_up_change":"2024-04-25T10:28:54.900654+0000","last_in_change":"2024-04-25T10:28:46.216973+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":5,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-04-25T10:28:56.202153+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"19","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":4,"pool_name":"cephfs_metadata","create_time":"2024-04-25T10:30:10.008132+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"35","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.75,"score_stable":1.75,"optimal_score":1,"raw_score_acting":1.75,"raw_score_stable":1.75,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":5,"pool_name":"cephfs_data","create_time":"2024-04-25T10:30:11.084827+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"36","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"a2f79028-b332-4166-9ebb-a7024a37ea22","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":34,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.37:6816","nonce":2746074720},{"type":"v1","addr":"172.21.15.37:6817","nonce":2746074720}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.37:6818","nonce":2746074720},{"type":"v1","addr":"172.21.15.37:6819","nonce":2746074720}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.37:6822","nonce":2746074720},{"type":"v1","addr":"172.21.15.37:6823","nonce":2746074720}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.37:6820","nonce":2746074720},{"type":"v1","addr":"172.21.15.37:6821","nonce":2746074720}]},"public_addr":"172.21.15.37:6817/2746074720","cluster_addr":"172.21.15.37:6819/2746074720","heartbeat_back_addr":"172.21.15.37:6823/2746074720","heartbeat_front_addr":"172.21.15.37:6821/2746074720","state":["exists","up"]},{"osd":1,"uuid":"973159de-c2ea-4083-aecc-69373d1c1132","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":34,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.37:6800","nonce":1690341662},{"type":"v1","addr":"172.21.15.37:6801","nonce":1690341662}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.37:6802","nonce":1690341662},{"type":"v1","addr":"172.21.15.37:6803","nonce":1690341662}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.37:6806","nonce":1690341662},{"type":"v1","addr":"172.21.15.37:6807","nonce":1690341662}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.37:6804","nonce":1690341662},{"type":"v1","addr":"172.21.15.37:6805","nonce":1690341662}]},"public_addr":"172.21.15.37:6801/1690341662","cluster_addr":"172.21.15.37:6803/1690341662","heartbeat_back_addr":"172.21.15.37:6807/1690341662","heartbeat_front_addr":"172.21.15.37:6805/1690341662","state":["exists","up"]},{"osd":2,"uuid":"4ae7a0e9-cb3f-484c-a75f-f3766b721c47","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":34,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.37:6808","nonce":2754669649},{"type":"v1","addr":"172.21.15.37:6809","nonce":2754669649}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.37:6810","nonce":2754669649},{"type":"v1","addr":"172.21.15.37:6811","nonce":2754669649}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.37:6814","nonce":2754669649},{"type":"v1","addr":"172.21.15.37:6815","nonce":2754669649}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.37:6812","nonce":2754669649},{"type":"v1","addr":"172.21.15.37:6813","nonce":2754669649}]},"public_addr":"172.21.15.37:6809/2754669649","cluster_addr":"172.21.15.37:6811/2754669649","heartbeat_back_addr":"172.21.15.37:6815/2754669649","heartbeat_front_addr":"172.21.15.37:6813/2754669649","state":["exists","up"]},{"osd":3,"uuid":"1ba4a746-1608-47a7-ace2-960a5cfae934","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":34,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.37:6824","nonce":1815290618},{"type":"v1","addr":"172.21.15.37:6825","nonce":1815290618}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.37:6826","nonce":1815290618},{"type":"v1","addr":"172.21.15.37:6827","nonce":1815290618}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.37:6830","nonce":1815290618},{"type":"v1","addr":"172.21.15.37:6831","nonce":1815290618}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.37:6828","nonce":1815290618},{"type":"v1","addr":"172.21.15.37:6829","nonce":1815290618}]},"public_addr":"172.21.15.37:6825/1815290618","cluster_addr":"172.21.15.37:6827/1815290618","heartbeat_back_addr":"172.21.15.37:6831/1815290618","heartbeat_front_addr":"172.21.15.37:6829/1815290618","state":["exists","up"]},{"osd":4,"uuid":"d4f59f5d-0413-4b79-87ee-bb212c1fcf62","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":34,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6800","nonce":4051532063},{"type":"v1","addr":"172.21.15.203:6801","nonce":4051532063}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6802","nonce":4051532063},{"type":"v1","addr":"172.21.15.203:6803","nonce":4051532063}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6806","nonce":4051532063},{"type":"v1","addr":"172.21.15.203:6807","nonce":4051532063}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6804","nonce":4051532063},{"type":"v1","addr":"172.21.15.203:6805","nonce":4051532063}]},"public_addr":"172.21.15.203:6801/4051532063","cluster_addr":"172.21.15.203:6803/4051532063","heartbeat_back_addr":"172.21.15.203:6807/4051532063","heartbeat_front_addr":"172.21.15.203:6805/4051532063","state":["exists","up"]},{"osd":5,"uuid":"69710a0e-41d8-4e7f-9a10-17ce540b67be","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":34,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6824","nonce":1203822890},{"type":"v1","addr":"172.21.15.203:6825","nonce":1203822890}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6826","nonce":1203822890},{"type":"v1","addr":"172.21.15.203:6827","nonce":1203822890}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6830","nonce":1203822890},{"type":"v1","addr":"172.21.15.203:6831","nonce":1203822890}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6828","nonce":1203822890},{"type":"v1","addr":"172.21.15.203:6829","nonce":1203822890}]},"public_addr":"172.21.15.203:6825/1203822890","cluster_addr":"172.21.15.203:6827/1203822890","heartbeat_back_addr":"172.21.15.203:6831/1203822890","heartbeat_front_addr":"172.21.15.203:6829/1203822890","state":["exists","up"]},{"osd":6,"uuid":"f29dd047-3b48-473d-a235-6642a6293f20","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":34,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6810","nonce":2309615014},{"type":"v1","addr":"172.21.15.203:6812","nonce":2309615014}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6814","nonce":2309615014},{"type":"v1","addr":"172.21.15.203:6816","nonce":2309615014}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6822","nonce":2309615014},{"type":"v1","addr":"172.21.15.203:6823","nonce":2309615014}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6818","nonce":2309615014},{"type":"v1","addr":"172.21.15.203:6820","nonce":2309615014}]},"public_addr":"172.21.15.203:6812/2309615014","cluster_addr":"172.21.15.203:6816/2309615014","heartbeat_back_addr":"172.21.15.203:6823/2309615014","heartbeat_front_addr":"172.21.15.203:6820/2309615014","state":["exists","up"]},{"osd":7,"uuid":"6e691644-378b-4fdc-8e86-fb5f56e31f29","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":34,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6808","nonce":3688099014},{"type":"v1","addr":"172.21.15.203:6809","nonce":3688099014}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6811","nonce":3688099014},{"type":"v1","addr":"172.21.15.203:6813","nonce":3688099014}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6819","nonce":3688099014},{"type":"v1","addr":"172.21.15.203:6821","nonce":3688099014}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6815","nonce":3688099014},{"type":"v1","addr":"172.21.15.203:6817","nonce":3688099014}]},"public_addr":"172.21.15.203:6809/3688099014","cluster_addr":"172.21.15.203:6813/3688099014","heartbeat_back_addr":"172.21.15.203:6821/3688099014","heartbeat_front_addr":"172.21.15.203:6817/3688099014","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T10:28:51.419141+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T10:28:51.366448+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T10:28:51.348276+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T10:28:51.380519+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T10:28:51.105812+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T10:28:51.384496+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T10:28:51.388755+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T10:28:51.366958+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.37:6835/311552410":"2024-04-26T10:30:24.853732+0000","172.21.15.37:6832/311552410":"2024-04-26T10:30:24.853732+0000"},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-04-25T10:30:25.581 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:25.578+0000 7f76b37a8640 1 -- 172.21.15.37:0/3485888200 >> [v2:172.21.15.203:6832/2418205237,v1:172.21.15.203:6833/2418205237] conn(0x7f76900744e0 msgr2=0x7f76900769f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:30:25.581 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:25.578+0000 7f76b37a8640 1 --2- 172.21.15.37:0/3485888200 >> [v2:172.21.15.203:6832/2418205237,v1:172.21.15.203:6833/2418205237] conn(0x7f76900744e0 0x7f76900769f0 secure :-1 s=READY pgs=237 cs=0 l=1 rev1=1 crypto rx=0x7f769c007fd0 tx=0x7f769c009040 comp rx=0 tx=0).stop 2024-04-25T10:30:25.581 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:25.578+0000 7f76b37a8640 1 -- 172.21.15.37:0/3485888200 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f76ac06e260 msgr2=0x7f76ac0ccc60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:30:25.581 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:25.578+0000 7f76b37a8640 1 --2- 172.21.15.37:0/3485888200 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f76ac06e260 0x7f76ac0ccc60 secure :-1 s=READY pgs=497 cs=0 l=1 rev1=1 crypto rx=0x7f76a800c430 tx=0x7f76a800c8f0 comp rx=0 tx=0).stop 2024-04-25T10:30:25.581 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:25.578+0000 7f76b37a8640 1 -- 172.21.15.37:0/3485888200 shutdown_connections 2024-04-25T10:30:25.581 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:25.578+0000 7f76b37a8640 1 --2- 172.21.15.37:0/3485888200 >> [v2:172.21.15.203:6832/2418205237,v1:172.21.15.203:6833/2418205237] conn(0x7f76900744e0 0x7f76900769f0 unknown :-1 s=CLOSED pgs=237 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:30:25.581 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:25.578+0000 7f76b37a8640 1 --2- 172.21.15.37:0/3485888200 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f76ac0d1ec0 0x7f76ac09dbe0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:30:25.581 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:25.578+0000 7f76b37a8640 1 --2- 172.21.15.37:0/3485888200 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f76ac0b4c90 0x7f76ac09d6a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:30:25.581 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:25.578+0000 7f76b37a8640 1 --2- 172.21.15.37:0/3485888200 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f76ac06e260 0x7f76ac0ccc60 unknown :-1 s=CLOSED pgs=497 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:30:25.581 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:25.578+0000 7f76b37a8640 1 -- 172.21.15.37:0/3485888200 >> 172.21.15.37:0/3485888200 conn(0x7f76ac0d0390 msgr2=0x7f76ac072a00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:30:25.581 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:25.578+0000 7f76b37a8640 1 -- 172.21.15.37:0/3485888200 shutdown_connections 2024-04-25T10:30:25.581 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:25.578+0000 7f76b37a8640 1 -- 172.21.15.37:0/3485888200 wait complete. 2024-04-25T10:30:25.594 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2024-04-25T10:28:56.202153+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '19', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 8, 'score_stable': 8, 'optimal_score': 0.25, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 4, 'pool_name': 'cephfs_metadata', 'create_time': '2024-04-25T10:30:10.008132+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '35', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_autoscale_bias': 4, 'pg_num_min': 16, 'recovery_priority': 5}, 'application_metadata': {'cephfs': {'metadata': 'cephfs'}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 1.75, 'score_stable': 1.75, 'optimal_score': 1, 'raw_score_acting': 1.75, 'raw_score_stable': 1.75, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 5, 'pool_name': 'cephfs_data', 'create_time': '2024-04-25T10:30:11.084827+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 64, 'pg_placement_num': 64, 'pg_placement_num_target': 64, 'pg_num_target': 64, 'pg_num_pending': 64, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '36', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.8999999761581421}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 1.5, 'score_stable': 1.5, 'optimal_score': 1, 'raw_score_acting': 1.5, 'raw_score_stable': 1.5, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}] 2024-04-25T10:30:25.594 DEBUG:teuthology.orchestra.run.smithi037:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2024-04-25T10:30:25.708 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:25.706+0000 7fb6a6946640 1 Processor -- start 2024-04-25T10:30:25.709 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:25.706+0000 7fb6a6946640 1 -- start start 2024-04-25T10:30:25.710 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:25.708+0000 7fb6a6946640 1 --2- >> v2:172.21.15.203:3300/0 conn(0x7fb6a00b5fd0 0x7fb6a00b2ca0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:30:25.710 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:25.708+0000 7fb6a6946640 1 --2- >> v2:172.21.15.37:3300/0 conn(0x7fb6a008cc90 0x7fb6a008d0a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:30:25.710 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:25.708+0000 7fb6a6946640 1 --2- >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7fb6a008d5e0 0x7fb6a00d18c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:30:25.711 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:25.708+0000 7fb6a6946640 1 -- --> v2:172.21.15.37:3300/0 -- mon_getmap magic: 0 -- 0x7fb6a00cf120 con 0x7fb6a008cc90 2024-04-25T10:30:25.711 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:25.708+0000 7fb6a6946640 1 -- --> v2:172.21.15.203:3300/0 -- mon_getmap magic: 0 -- 0x7fb6a00cf400 con 0x7fb6a00b5fd0 2024-04-25T10:30:25.711 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:25.708+0000 7fb6a6946640 1 -- --> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] -- mon_getmap magic: 0 -- 0x7fb6a00cf6e0 con 0x7fb6a008d5e0 2024-04-25T10:30:25.711 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:25.708+0000 7fb69f7fe640 1 --2- >> v2:172.21.15.37:3300/0 conn(0x7fb6a008cc90 0x7fb6a008d0a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:30:25.711 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:25.708+0000 7fb69f7fe640 1 --2- >> v2:172.21.15.37:3300/0 conn(0x7fb6a008cc90 0x7fb6a008d0a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.37:3300/0 says I am v2:172.21.15.37:59852/0 (socket says 172.21.15.37:59852) 2024-04-25T10:30:25.711 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:25.708+0000 7fb69f7fe640 1 -- 172.21.15.37:0/4022096822 learned_addr learned my addr 172.21.15.37:0/4022096822 (peer_addr_for_me v2:172.21.15.37:0/0) 2024-04-25T10:30:25.711 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:25.708+0000 7fb6a4ee3640 1 --2- 172.21.15.37:0/4022096822 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7fb6a008d5e0 0x7fb6a00d18c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:30:25.711 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:25.708+0000 7fb69ffff640 1 --2- 172.21.15.37:0/4022096822 >> v2:172.21.15.203:3300/0 conn(0x7fb6a00b5fd0 0x7fb6a00b2ca0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:30:25.711 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:25.708+0000 7fb69f7fe640 1 -- 172.21.15.37:0/4022096822 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7fb6a008d5e0 msgr2=0x7fb6a00d18c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:30:25.711 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:25.708+0000 7fb69f7fe640 1 --2- 172.21.15.37:0/4022096822 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7fb6a008d5e0 0x7fb6a00d18c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:30:25.711 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:25.708+0000 7fb69f7fe640 1 -- 172.21.15.37:0/4022096822 >> v2:172.21.15.203:3300/0 conn(0x7fb6a00b5fd0 msgr2=0x7fb6a00b2ca0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:30:25.711 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:25.708+0000 7fb69f7fe640 1 --2- 172.21.15.37:0/4022096822 >> v2:172.21.15.203:3300/0 conn(0x7fb6a00b5fd0 0x7fb6a00b2ca0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:30:25.711 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:25.708+0000 7fb69f7fe640 1 -- 172.21.15.37:0/4022096822 --> v2:172.21.15.37:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fb6a00cf9c0 con 0x7fb6a008cc90 2024-04-25T10:30:25.713 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:25.709+0000 7fb69f7fe640 1 --2- 172.21.15.37:0/4022096822 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fb6a008cc90 0x7fb6a008d0a0 secure :-1 s=READY pgs=498 cs=0 l=1 rev1=1 crypto rx=0x7fb6a0070b40 tx=0x7fb69002a140 comp rx=0 tx=0).ready entity=mon.0 client_cookie=30c0e077ca9effff server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:30:25.713 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:25.709+0000 7fb69effd640 1 -- 172.21.15.37:0/4022096822 <== mon.0 v2:172.21.15.37:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb690037070 con 0x7fb6a008cc90 2024-04-25T10:30:25.713 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:25.709+0000 7fb69effd640 1 -- 172.21.15.37:0/4022096822 <== mon.0 v2:172.21.15.37:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fb690032440 con 0x7fb6a008cc90 2024-04-25T10:30:25.713 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:25.709+0000 7fb69effd640 1 -- 172.21.15.37:0/4022096822 <== mon.0 v2:172.21.15.37:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb690032810 con 0x7fb6a008cc90 2024-04-25T10:30:25.713 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:25.710+0000 7fb6a6946640 1 -- 172.21.15.37:0/4022096822 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fb6a008cc90 msgr2=0x7fb6a008d0a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:30:25.713 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:25.710+0000 7fb6a6946640 1 --2- 172.21.15.37:0/4022096822 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fb6a008cc90 0x7fb6a008d0a0 secure :-1 s=READY pgs=498 cs=0 l=1 rev1=1 crypto rx=0x7fb6a0070b40 tx=0x7fb69002a140 comp rx=0 tx=0).stop 2024-04-25T10:30:25.713 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:25.710+0000 7fb6a6946640 1 -- 172.21.15.37:0/4022096822 shutdown_connections 2024-04-25T10:30:25.713 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:25.710+0000 7fb6a6946640 1 --2- 172.21.15.37:0/4022096822 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fb6a008cc90 0x7fb6a008d0a0 unknown :-1 s=CLOSED pgs=498 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:30:25.713 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:25.710+0000 7fb6a6946640 1 --2- 172.21.15.37:0/4022096822 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7fb6a008d5e0 0x7fb6a00d18c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:30:25.713 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:25.710+0000 7fb6a6946640 1 --2- 172.21.15.37:0/4022096822 >> v2:172.21.15.203:3300/0 conn(0x7fb6a00b5fd0 0x7fb6a00b2ca0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:30:25.714 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:25.710+0000 7fb6a6946640 1 -- 172.21.15.37:0/4022096822 >> 172.21.15.37:0/4022096822 conn(0x7fb6a00b5440 msgr2=0x7fb6a00b5850 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:30:25.714 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:25.710+0000 7fb6a6946640 1 -- 172.21.15.37:0/4022096822 shutdown_connections 2024-04-25T10:30:25.714 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:25.710+0000 7fb6a6946640 1 -- 172.21.15.37:0/4022096822 wait complete. 2024-04-25T10:30:25.714 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:25.711+0000 7fb6a6946640 1 Processor -- start 2024-04-25T10:30:25.714 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:25.712+0000 7fb6a6946640 1 -- start start 2024-04-25T10:30:25.716 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:25.713+0000 7fb6a6946640 1 --2- >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7fb6a008cc90 0x7fb6a01587d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:30:25.716 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:25.713+0000 7fb6a6946640 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fb6a008d5e0 0x7fb6a00cc810 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:30:25.716 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:25.713+0000 7fb6a6946640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fb6a00b5fd0 0x7fb6a00ccd50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:30:25.717 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:25.713+0000 7fb6a6946640 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 -- 0x7fb6a00d1f80 con 0x7fb6a00b5fd0 2024-04-25T10:30:25.717 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:25.713+0000 7fb6a6946640 1 -- --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_getmap magic: 0 -- 0x7fb6a00d1e00 con 0x7fb6a008d5e0 2024-04-25T10:30:25.717 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:25.713+0000 7fb6a6946640 1 -- --> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] -- mon_getmap magic: 0 -- 0x7fb6a00d2100 con 0x7fb6a008cc90 2024-04-25T10:30:25.717 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:25.713+0000 7fb6a4ee3640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fb6a00b5fd0 0x7fb6a00ccd50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:30:25.717 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:25.713+0000 7fb6a4ee3640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fb6a00b5fd0 0x7fb6a00ccd50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.37:3300/0 says I am v2:172.21.15.37:59862/0 (socket says 172.21.15.37:59862) 2024-04-25T10:30:25.717 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:25.713+0000 7fb6a4ee3640 1 -- 172.21.15.37:0/2286013361 learned_addr learned my addr 172.21.15.37:0/2286013361 (peer_addr_for_me v2:172.21.15.37:0/0) 2024-04-25T10:30:25.717 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:25.713+0000 7fb69f7fe640 1 --2- 172.21.15.37:0/2286013361 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fb6a008d5e0 0x7fb6a00cc810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:30:25.717 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:25.713+0000 7fb69ffff640 1 --2- 172.21.15.37:0/2286013361 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7fb6a008cc90 0x7fb6a01587d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:30:25.717 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:25.714+0000 7fb6a4ee3640 1 -- 172.21.15.37:0/2286013361 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7fb6a008cc90 msgr2=0x7fb6a01587d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:30:25.717 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:25.714+0000 7fb6a4ee3640 1 --2- 172.21.15.37:0/2286013361 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7fb6a008cc90 0x7fb6a01587d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:30:25.717 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:25.714+0000 7fb6a4ee3640 1 -- 172.21.15.37:0/2286013361 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fb6a008d5e0 msgr2=0x7fb6a00cc810 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:30:25.717 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:25.714+0000 7fb6a4ee3640 1 --2- 172.21.15.37:0/2286013361 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fb6a008d5e0 0x7fb6a00cc810 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:30:25.717 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:25.714+0000 7fb6a4ee3640 1 -- 172.21.15.37:0/2286013361 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fb6a00cd7c0 con 0x7fb6a00b5fd0 2024-04-25T10:30:25.717 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:25.714+0000 7fb6a4ee3640 1 --2- 172.21.15.37:0/2286013361 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fb6a00b5fd0 0x7fb6a00ccd50 secure :-1 s=READY pgs=499 cs=0 l=1 rev1=1 crypto rx=0x7fb69800ef00 tx=0x7fb69800b480 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:30:25.718 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:25.714+0000 7fb69cff9640 1 -- 172.21.15.37:0/2286013361 <== mon.0 v2:172.21.15.37:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb698018020 con 0x7fb6a00b5fd0 2024-04-25T10:30:25.718 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:25.714+0000 7fb69cff9640 1 -- 172.21.15.37:0/2286013361 <== mon.0 v2:172.21.15.37:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fb69800bd20 con 0x7fb6a00b5fd0 2024-04-25T10:30:25.718 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:25.714+0000 7fb6a6946640 1 -- 172.21.15.37:0/2286013361 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fb6a00cd4e0 con 0x7fb6a00b5fd0 2024-04-25T10:30:25.718 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:25.714+0000 7fb69cff9640 1 -- 172.21.15.37:0/2286013361 <== mon.0 v2:172.21.15.37:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb698014640 con 0x7fb6a00b5fd0 2024-04-25T10:30:25.718 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:25.714+0000 7fb6a6946640 1 -- 172.21.15.37:0/2286013361 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fb6a015f5d0 con 0x7fb6a00b5fd0 2024-04-25T10:30:25.719 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:25.715+0000 7fb6a6946640 1 -- 172.21.15.37:0/2286013361 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fb6a006a140 con 0x7fb6a00b5fd0 2024-04-25T10:30:25.719 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:25.716+0000 7fb69cff9640 1 -- 172.21.15.37:0/2286013361 <== mon.0 v2:172.21.15.37:3300/0 4 ==== mgrmap(e 5) ==== 96782+0+0 (secure 0 0 0) 0x7fb698005d10 con 0x7fb6a00b5fd0 2024-04-25T10:30:25.719 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:25.716+0000 7fb69cff9640 1 --2- 172.21.15.37:0/2286013361 >> [v2:172.21.15.203:6832/2418205237,v1:172.21.15.203:6833/2418205237] conn(0x7fb68c0744b0 0x7fb68c0769c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:30:25.719 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:25.716+0000 7fb69cff9640 1 -- 172.21.15.37:0/2286013361 <== mon.0 v2:172.21.15.37:3300/0 5 ==== osd_map(37..37 src has 1..37) ==== 5795+0+0 (secure 0 0 0) 0x7fb698095730 con 0x7fb6a00b5fd0 2024-04-25T10:30:25.719 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:25.716+0000 7fb69ffff640 1 --2- 172.21.15.37:0/2286013361 >> [v2:172.21.15.203:6832/2418205237,v1:172.21.15.203:6833/2418205237] conn(0x7fb68c0744b0 0x7fb68c0769c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:30:25.719 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:25.717+0000 7fb69ffff640 1 --2- 172.21.15.37:0/2286013361 >> [v2:172.21.15.203:6832/2418205237,v1:172.21.15.203:6833/2418205237] conn(0x7fb68c0744b0 0x7fb68c0769c0 secure :-1 s=READY pgs=238 cs=0 l=1 rev1=1 crypto rx=0x7fb6a0073c80 tx=0x7fb694009110 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:30:25.723 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:25.721+0000 7fb69cff9640 1 -- 172.21.15.37:0/2286013361 <== mon.0 v2:172.21.15.37:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187246 (secure 0 0 0) 0x7fb698022070 con 0x7fb6a00b5fd0 2024-04-25T10:30:25.913 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:25.911+0000 7fb6a6946640 1 -- 172.21.15.37:0/2286013361 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) -- 0x7fb6a00ce700 con 0x7fb6a00b5fd0 2024-04-25T10:30:25.913 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:25.911+0000 7fb69cff9640 1 -- 172.21.15.37:0/2286013361 <== mon.0 v2:172.21.15.37:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v41) ==== 93+0+10 (secure 0 0 0) 0x7fb6a00cd7c0 con 0x7fb6a00b5fd0 2024-04-25T10:30:25.913 INFO:teuthology.orchestra.run.smithi037.stdout:pg_num: 1 2024-04-25T10:30:25.916 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:25.913+0000 7fb6a6946640 1 -- 172.21.15.37:0/2286013361 >> [v2:172.21.15.203:6832/2418205237,v1:172.21.15.203:6833/2418205237] conn(0x7fb68c0744b0 msgr2=0x7fb68c0769c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:30:25.916 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:25.913+0000 7fb6a6946640 1 --2- 172.21.15.37:0/2286013361 >> [v2:172.21.15.203:6832/2418205237,v1:172.21.15.203:6833/2418205237] conn(0x7fb68c0744b0 0x7fb68c0769c0 secure :-1 s=READY pgs=238 cs=0 l=1 rev1=1 crypto rx=0x7fb6a0073c80 tx=0x7fb694009110 comp rx=0 tx=0).stop 2024-04-25T10:30:25.916 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:25.913+0000 7fb6a6946640 1 -- 172.21.15.37:0/2286013361 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fb6a00b5fd0 msgr2=0x7fb6a00ccd50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:30:25.916 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:25.913+0000 7fb6a6946640 1 --2- 172.21.15.37:0/2286013361 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fb6a00b5fd0 0x7fb6a00ccd50 secure :-1 s=READY pgs=499 cs=0 l=1 rev1=1 crypto rx=0x7fb69800ef00 tx=0x7fb69800b480 comp rx=0 tx=0).stop 2024-04-25T10:30:25.916 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:25.913+0000 7fb6a6946640 1 -- 172.21.15.37:0/2286013361 shutdown_connections 2024-04-25T10:30:25.916 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:25.913+0000 7fb6a6946640 1 --2- 172.21.15.37:0/2286013361 >> [v2:172.21.15.203:6832/2418205237,v1:172.21.15.203:6833/2418205237] conn(0x7fb68c0744b0 0x7fb68c0769c0 unknown :-1 s=CLOSED pgs=238 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:30:25.916 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:25.913+0000 7fb6a6946640 1 --2- 172.21.15.37:0/2286013361 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fb6a00b5fd0 0x7fb6a00ccd50 unknown :-1 s=CLOSED pgs=499 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:30:25.916 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:25.913+0000 7fb6a6946640 1 --2- 172.21.15.37:0/2286013361 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fb6a008d5e0 0x7fb6a00cc810 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:30:25.916 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:25.913+0000 7fb6a6946640 1 --2- 172.21.15.37:0/2286013361 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7fb6a008cc90 0x7fb6a01587d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:30:25.916 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:25.913+0000 7fb6a6946640 1 -- 172.21.15.37:0/2286013361 >> 172.21.15.37:0/2286013361 conn(0x7fb6a00b5440 msgr2=0x7fb6a00b8580 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:30:25.916 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:25.913+0000 7fb6a6946640 1 -- 172.21.15.37:0/2286013361 shutdown_connections 2024-04-25T10:30:25.916 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:25.913+0000 7fb6a6946640 1 -- 172.21.15.37:0/2286013361 wait complete. 2024-04-25T10:30:25.927 DEBUG:teuthology.orchestra.run.smithi037:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_metadata pg_num 2024-04-25T10:30:26.040 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:26.038+0000 7f12a46aa640 1 Processor -- start 2024-04-25T10:30:26.041 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:26.038+0000 7f12a46aa640 1 -- start start 2024-04-25T10:30:26.043 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:26.040+0000 7f12a46aa640 1 --2- >> v2:172.21.15.37:3300/0 conn(0x7f129c0b3c80 0x7f129c0b4090 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:30:26.043 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:26.040+0000 7f12a46aa640 1 --2- >> v2:172.21.15.203:3300/0 conn(0x7f129c0d1ff0 0x7f129c0d2400 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:30:26.043 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:26.040+0000 7f12a46aa640 1 -- --> v1:172.21.15.37:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f129c0b8d10 con 0x7f129c1625b0 2024-04-25T10:30:26.043 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:26.040+0000 7f12a46aa640 1 -- --> v2:172.21.15.37:3300/0 -- mon_getmap magic: 0 -- 0x7f129c08ceb0 con 0x7f129c0b3c80 2024-04-25T10:30:26.043 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:26.040+0000 7f12a46aa640 1 -- --> v2:172.21.15.203:3300/0 -- mon_getmap magic: 0 -- 0x7f129c0b9300 con 0x7f129c0d1ff0 2024-04-25T10:30:26.043 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:26.040+0000 7f12a2446640 1 --2- >> v2:172.21.15.37:3300/0 conn(0x7f129c0b3c80 0x7f129c0b4090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:30:26.043 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:26.040+0000 7f12a2c47640 1 --1- >> v1:172.21.15.37:6789/0 conn(0x7f129c1625b0 0x7f129c1629c0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.37:6789/0 says I am v1:172.21.15.37:55194/0 (socket says 172.21.15.37:55194) 2024-04-25T10:30:26.043 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:26.040+0000 7f12a2446640 1 --2- >> v2:172.21.15.37:3300/0 conn(0x7f129c0b3c80 0x7f129c0b4090 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.37:3300/0 says I am v2:172.21.15.37:59872/0 (socket says 172.21.15.37:59872) 2024-04-25T10:30:26.043 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:26.040+0000 7f12a2c47640 1 -- 172.21.15.37:0/2935447342 learned_addr learned my addr 172.21.15.37:0/2935447342 (peer_addr_for_me v1:172.21.15.37:0/0) 2024-04-25T10:30:26.043 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:26.040+0000 7f12a1c45640 1 --2- 172.21.15.37:0/2935447342 >> v2:172.21.15.203:3300/0 conn(0x7f129c0d1ff0 0x7f129c0d2400 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:30:26.043 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:26.040+0000 7f12a2446640 1 -- 172.21.15.37:0/2935447342 >> v2:172.21.15.203:3300/0 conn(0x7f129c0d1ff0 msgr2=0x7f129c0d2400 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:30:26.043 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:26.040+0000 7f12a2446640 1 --2- 172.21.15.37:0/2935447342 >> v2:172.21.15.203:3300/0 conn(0x7f129c0d1ff0 0x7f129c0d2400 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:30:26.044 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:26.040+0000 7f12a2446640 1 -- 172.21.15.37:0/2935447342 >> v1:172.21.15.37:6789/0 conn(0x7f129c1625b0 legacy=0x7f129c1629c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:30:26.044 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:26.040+0000 7f12a2446640 1 -- 172.21.15.37:0/2935447342 --> v2:172.21.15.37:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f129c0b8a30 con 0x7f129c0b3c80 2024-04-25T10:30:26.044 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:26.040+0000 7f12a1444640 1 -- 172.21.15.37:0/2935447342 <== mon.0 v1:172.21.15.37:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 3434605319 0 0) 0x7f129c0b8d10 con 0x7f129c1625b0 2024-04-25T10:30:26.045 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:26.041+0000 7f12a2446640 1 --2- 172.21.15.37:0/2935447342 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f129c0b3c80 0x7f129c0b4090 secure :-1 s=READY pgs=501 cs=0 l=1 rev1=1 crypto rx=0x7f129c0b9610 tx=0x7f129402a110 comp rx=0 tx=0).ready entity=mon.0 client_cookie=1eadf537e2c4eb4f server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:30:26.045 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:26.041+0000 7f12a1444640 1 -- 172.21.15.37:0/2935447342 <== mon.0 v2:172.21.15.37:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1294037070 con 0x7f129c0b3c80 2024-04-25T10:30:26.045 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:26.041+0000 7f12a1444640 1 -- 172.21.15.37:0/2935447342 <== mon.0 v2:172.21.15.37:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f1294032440 con 0x7f129c0b3c80 2024-04-25T10:30:26.045 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:26.041+0000 7f12a1444640 1 -- 172.21.15.37:0/2935447342 <== mon.0 v2:172.21.15.37:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f129402ae60 con 0x7f129c0b3c80 2024-04-25T10:30:26.045 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:26.042+0000 7f12a46aa640 1 -- 172.21.15.37:0/2935447342 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f129c0b3c80 msgr2=0x7f129c0b4090 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:30:26.045 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:26.042+0000 7f12a46aa640 1 --2- 172.21.15.37:0/2935447342 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f129c0b3c80 0x7f129c0b4090 secure :-1 s=READY pgs=501 cs=0 l=1 rev1=1 crypto rx=0x7f129c0b9610 tx=0x7f129402a110 comp rx=0 tx=0).stop 2024-04-25T10:30:26.045 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:26.042+0000 7f12a46aa640 1 -- 172.21.15.37:0/2935447342 shutdown_connections 2024-04-25T10:30:26.045 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:26.042+0000 7f12a46aa640 1 --2- 172.21.15.37:0/2935447342 >> v2:172.21.15.203:3300/0 conn(0x7f129c0d1ff0 0x7f129c0d2400 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:30:26.045 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:26.042+0000 7f12a46aa640 1 --2- 172.21.15.37:0/2935447342 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f129c0b3c80 0x7f129c0b4090 unknown :-1 s=CLOSED pgs=501 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:30:26.045 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:26.042+0000 7f12a46aa640 1 -- 172.21.15.37:0/2935447342 >> 172.21.15.37:0/2935447342 conn(0x7f129c0d0390 msgr2=0x7f129c0d07a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:30:26.046 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:26.042+0000 7f12a46aa640 1 -- 172.21.15.37:0/2935447342 shutdown_connections 2024-04-25T10:30:26.046 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:26.042+0000 7f12a46aa640 1 -- 172.21.15.37:0/2935447342 wait complete. 2024-04-25T10:30:26.046 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:26.044+0000 7f12a46aa640 1 Processor -- start 2024-04-25T10:30:26.046 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:26.044+0000 7f12a46aa640 1 -- start start 2024-04-25T10:30:26.048 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:26.045+0000 7f12a46aa640 1 --2- >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f129c0b3c80 0x7f129c14cdb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:30:26.048 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:26.045+0000 7f12a46aa640 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f129c0d1ff0 0x7f129c1e8a10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:30:26.049 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:26.045+0000 7f12a46aa640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f129c1625b0 0x7f129c1f7510 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:30:26.049 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:26.045+0000 7f12a46aa640 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 -- 0x7f129c0b45d0 con 0x7f129c1625b0 2024-04-25T10:30:26.049 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:26.045+0000 7f12a46aa640 1 -- --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_getmap magic: 0 -- 0x7f129c09ce90 con 0x7f129c0d1ff0 2024-04-25T10:30:26.049 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:26.045+0000 7f12a46aa640 1 -- --> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] -- mon_getmap magic: 0 -- 0x7f129c0d2940 con 0x7f129c0b3c80 2024-04-25T10:30:26.049 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:26.045+0000 7f12a2c47640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f129c1625b0 0x7f129c1f7510 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:30:26.049 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:26.045+0000 7f12a2c47640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f129c1625b0 0x7f129c1f7510 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.37:3300/0 says I am v2:172.21.15.37:59886/0 (socket says 172.21.15.37:59886) 2024-04-25T10:30:26.049 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:26.045+0000 7f12a2c47640 1 -- 172.21.15.37:0/273274271 learned_addr learned my addr 172.21.15.37:0/273274271 (peer_addr_for_me v2:172.21.15.37:0/0) 2024-04-25T10:30:26.049 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:26.045+0000 7f12a2446640 1 --2- 172.21.15.37:0/273274271 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f129c0b3c80 0x7f129c14cdb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:30:26.049 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:26.045+0000 7f12a1c45640 1 --2- 172.21.15.37:0/273274271 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f129c0d1ff0 0x7f129c1e8a10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:30:26.049 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:26.046+0000 7f12a2c47640 1 -- 172.21.15.37:0/273274271 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f129c0b3c80 msgr2=0x7f129c14cdb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:30:26.049 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:26.046+0000 7f12a2c47640 1 --2- 172.21.15.37:0/273274271 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f129c0b3c80 0x7f129c14cdb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:30:26.049 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:26.046+0000 7f12a2c47640 1 -- 172.21.15.37:0/273274271 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f129c0d1ff0 msgr2=0x7f129c1e8a10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:30:26.049 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:26.046+0000 7f12a2c47640 1 --2- 172.21.15.37:0/273274271 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f129c0d1ff0 0x7f129c1e8a10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:30:26.050 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:26.046+0000 7f12a2c47640 1 -- 172.21.15.37:0/273274271 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f129c09daa0 con 0x7f129c1625b0 2024-04-25T10:30:26.050 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:26.046+0000 7f12a1c45640 1 --2- 172.21.15.37:0/273274271 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f129c0d1ff0 0x7f129c1e8a10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T10:30:26.050 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:26.046+0000 7f12a2c47640 1 --2- 172.21.15.37:0/273274271 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f129c1625b0 0x7f129c1f7510 secure :-1 s=READY pgs=502 cs=0 l=1 rev1=1 crypto rx=0x7f128c00c6d0 tx=0x7f128c00cb90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:30:26.050 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:26.046+0000 7f129affd640 1 -- 172.21.15.37:0/273274271 <== mon.0 v2:172.21.15.37:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f128c018070 con 0x7f129c1625b0 2024-04-25T10:30:26.050 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:26.046+0000 7f129affd640 1 -- 172.21.15.37:0/273274271 <== mon.0 v2:172.21.15.37:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f128c004d60 con 0x7f129c1625b0 2024-04-25T10:30:26.050 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:26.046+0000 7f12a46aa640 1 -- 172.21.15.37:0/273274271 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f129c09dd80 con 0x7f129c1625b0 2024-04-25T10:30:26.050 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:26.046+0000 7f129affd640 1 -- 172.21.15.37:0/273274271 <== mon.0 v2:172.21.15.37:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f128c00a880 con 0x7f129c1625b0 2024-04-25T10:30:26.050 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:26.046+0000 7f12a46aa640 1 -- 172.21.15.37:0/273274271 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f129c09e340 con 0x7f129c1625b0 2024-04-25T10:30:26.051 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:26.047+0000 7f12a46aa640 1 -- 172.21.15.37:0/273274271 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f129c09d7f0 con 0x7f129c1625b0 2024-04-25T10:30:26.051 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:26.048+0000 7f129affd640 1 -- 172.21.15.37:0/273274271 <== mon.0 v2:172.21.15.37:3300/0 4 ==== mgrmap(e 5) ==== 96782+0+0 (secure 0 0 0) 0x7f128c01b070 con 0x7f129c1625b0 2024-04-25T10:30:26.051 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:26.048+0000 7f129affd640 1 --2- 172.21.15.37:0/273274271 >> [v2:172.21.15.203:6832/2418205237,v1:172.21.15.203:6833/2418205237] conn(0x7f12880744b0 0x7f12880769c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:30:26.051 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:26.048+0000 7f129affd640 1 -- 172.21.15.37:0/273274271 <== mon.0 v2:172.21.15.37:3300/0 5 ==== osd_map(41..41 src has 1..41) ==== 5795+0+0 (secure 0 0 0) 0x7f128c0947b0 con 0x7f129c1625b0 2024-04-25T10:30:26.051 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:26.048+0000 7f12a2446640 1 --2- 172.21.15.37:0/273274271 >> [v2:172.21.15.203:6832/2418205237,v1:172.21.15.203:6833/2418205237] conn(0x7f12880744b0 0x7f12880769c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:30:26.051 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:26.049+0000 7f12a2446640 1 --2- 172.21.15.37:0/273274271 >> [v2:172.21.15.203:6832/2418205237,v1:172.21.15.203:6833/2418205237] conn(0x7f12880744b0 0x7f12880769c0 secure :-1 s=READY pgs=243 cs=0 l=1 rev1=1 crypto rx=0x7f1294034000 tx=0x7f129402e000 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:30:26.054 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:26.053+0000 7f129affd640 1 -- 172.21.15.37:0/273274271 <== mon.0 v2:172.21.15.37:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187246 (secure 0 0 0) 0x7f128c096e20 con 0x7f129c1625b0 2024-04-25T10:30:26.240 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:26.238+0000 7f12a46aa640 1 -- 172.21.15.37:0/273274271 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"} v 0) -- 0x7f129c06a140 con 0x7f129c1625b0 2024-04-25T10:30:26.240 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:26.238+0000 7f129affd640 1 -- 172.21.15.37:0/273274271 <== mon.0 v2:172.21.15.37:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]=0 v41) ==== 104+0+11 (secure 0 0 0) 0x7f128c05ef70 con 0x7f129c1625b0 2024-04-25T10:30:26.240 INFO:teuthology.orchestra.run.smithi037.stdout:pg_num: 32 2024-04-25T10:30:26.242 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:26.240+0000 7f12a46aa640 1 -- 172.21.15.37:0/273274271 >> [v2:172.21.15.203:6832/2418205237,v1:172.21.15.203:6833/2418205237] conn(0x7f12880744b0 msgr2=0x7f12880769c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:30:26.242 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:26.240+0000 7f12a46aa640 1 --2- 172.21.15.37:0/273274271 >> [v2:172.21.15.203:6832/2418205237,v1:172.21.15.203:6833/2418205237] conn(0x7f12880744b0 0x7f12880769c0 secure :-1 s=READY pgs=243 cs=0 l=1 rev1=1 crypto rx=0x7f1294034000 tx=0x7f129402e000 comp rx=0 tx=0).stop 2024-04-25T10:30:26.242 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:26.240+0000 7f12a46aa640 1 -- 172.21.15.37:0/273274271 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f129c1625b0 msgr2=0x7f129c1f7510 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:30:26.243 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:26.240+0000 7f12a46aa640 1 --2- 172.21.15.37:0/273274271 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f129c1625b0 0x7f129c1f7510 secure :-1 s=READY pgs=502 cs=0 l=1 rev1=1 crypto rx=0x7f128c00c6d0 tx=0x7f128c00cb90 comp rx=0 tx=0).stop 2024-04-25T10:30:26.243 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:26.240+0000 7f12a46aa640 1 -- 172.21.15.37:0/273274271 shutdown_connections 2024-04-25T10:30:26.243 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:26.240+0000 7f12a46aa640 1 --2- 172.21.15.37:0/273274271 >> [v2:172.21.15.203:6832/2418205237,v1:172.21.15.203:6833/2418205237] conn(0x7f12880744b0 0x7f12880769c0 unknown :-1 s=CLOSED pgs=243 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:30:26.243 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:26.240+0000 7f12a46aa640 1 --2- 172.21.15.37:0/273274271 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f129c1625b0 0x7f129c1f7510 unknown :-1 s=CLOSED pgs=502 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:30:26.243 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:26.240+0000 7f12a46aa640 1 --2- 172.21.15.37:0/273274271 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f129c0d1ff0 0x7f129c1e8a10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:30:26.243 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:26.240+0000 7f12a46aa640 1 --2- 172.21.15.37:0/273274271 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f129c0b3c80 0x7f129c14cdb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:30:26.243 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:26.240+0000 7f12a46aa640 1 -- 172.21.15.37:0/273274271 >> 172.21.15.37:0/273274271 conn(0x7f129c0d0390 msgr2=0x7f129c1609c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:30:26.243 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:26.240+0000 7f12a46aa640 1 -- 172.21.15.37:0/273274271 shutdown_connections 2024-04-25T10:30:26.243 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:26.240+0000 7f12a46aa640 1 -- 172.21.15.37:0/273274271 wait complete. 2024-04-25T10:30:26.254 DEBUG:teuthology.orchestra.run.smithi037:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data pg_num 2024-04-25T10:30:26.368 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:26.366+0000 7f2a4f80c640 1 Processor -- start 2024-04-25T10:30:26.368 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:26.366+0000 7f2a4f80c640 1 -- start start 2024-04-25T10:30:26.370 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:26.367+0000 7f2a4f80c640 1 --2- >> v2:172.21.15.37:3300/0 conn(0x7f2a4806e260 0x7f2a4806e670 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:30:26.370 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:26.367+0000 7f2a4f80c640 1 -- --> v1:172.21.15.37:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f2a4809ce70 con 0x7f2a4806ebb0 2024-04-25T10:30:26.370 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:26.367+0000 7f2a4f80c640 1 -- --> v1:172.21.15.203:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f2a4809d150 con 0x7f2a48164ce0 2024-04-25T10:30:26.370 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:26.367+0000 7f2a4f80c640 1 -- --> v2:172.21.15.37:3300/0 -- mon_getmap magic: 0 -- 0x7f2a480d3370 con 0x7f2a4806e260 2024-04-25T10:30:26.370 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:26.367+0000 7f2a4d5a8640 1 --2- >> v2:172.21.15.37:3300/0 conn(0x7f2a4806e260 0x7f2a4806e670 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:30:26.370 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:26.367+0000 7f2a4cda7640 1 --1- >> v1:172.21.15.37:6789/0 conn(0x7f2a4806ebb0 0x7f2a481645d0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.37:6789/0 says I am v1:172.21.15.37:55198/0 (socket says 172.21.15.37:55198) 2024-04-25T10:30:26.370 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:26.367+0000 7f2a4cda7640 1 -- 172.21.15.37:0/4138399927 learned_addr learned my addr 172.21.15.37:0/4138399927 (peer_addr_for_me v1:172.21.15.37:0/0) 2024-04-25T10:30:26.370 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:26.368+0000 7f2a4d5a8640 1 -- 172.21.15.37:0/4138399927 >> v1:172.21.15.203:6789/0 conn(0x7f2a48164ce0 legacy=0x7f2a480b3c80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:30:26.370 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:26.368+0000 7f2a4d5a8640 1 -- 172.21.15.37:0/4138399927 >> v1:172.21.15.37:6789/0 conn(0x7f2a4806ebb0 legacy=0x7f2a481645d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:30:26.370 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:26.368+0000 7f2a4d5a8640 1 -- 172.21.15.37:0/4138399927 --> v2:172.21.15.37:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f2a480b9000 con 0x7f2a4806e260 2024-04-25T10:30:26.370 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:26.368+0000 7f2a3ffff640 1 -- 172.21.15.37:0/4138399927 <== mon.0 v1:172.21.15.37:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 773822158 0 0) 0x7f2a4809ce70 con 0x7f2a4806ebb0 2024-04-25T10:30:26.371 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:26.368+0000 7f2a4d5a8640 1 --2- 172.21.15.37:0/4138399927 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f2a4806e260 0x7f2a4806e670 secure :-1 s=READY pgs=504 cs=0 l=1 rev1=1 crypto rx=0x7f2a440097a0 tx=0x7f2a4402a140 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d7b304e27542beb7 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:30:26.372 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:26.369+0000 7f2a3ffff640 1 -- 172.21.15.37:0/4138399927 <== mon.0 v2:172.21.15.37:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2a44037070 con 0x7f2a4806e260 2024-04-25T10:30:26.372 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:26.369+0000 7f2a3ffff640 1 -- 172.21.15.37:0/4138399927 <== mon.0 v2:172.21.15.37:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f2a4402ae50 con 0x7f2a4806e260 2024-04-25T10:30:26.372 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:26.369+0000 7f2a3ffff640 1 -- 172.21.15.37:0/4138399927 <== mon.0 v2:172.21.15.37:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2a44032660 con 0x7f2a4806e260 2024-04-25T10:30:26.372 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:26.369+0000 7f2a4f80c640 1 -- 172.21.15.37:0/4138399927 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f2a4806e260 msgr2=0x7f2a4806e670 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:30:26.372 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:26.369+0000 7f2a4f80c640 1 --2- 172.21.15.37:0/4138399927 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f2a4806e260 0x7f2a4806e670 secure :-1 s=READY pgs=504 cs=0 l=1 rev1=1 crypto rx=0x7f2a440097a0 tx=0x7f2a4402a140 comp rx=0 tx=0).stop 2024-04-25T10:30:26.372 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:26.369+0000 7f2a4f80c640 1 -- 172.21.15.37:0/4138399927 shutdown_connections 2024-04-25T10:30:26.372 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:26.369+0000 7f2a4f80c640 1 --2- 172.21.15.37:0/4138399927 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f2a4806e260 0x7f2a4806e670 unknown :-1 s=CLOSED pgs=504 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:30:26.372 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:26.369+0000 7f2a4f80c640 1 -- 172.21.15.37:0/4138399927 >> 172.21.15.37:0/4138399927 conn(0x7f2a480d0390 msgr2=0x7f2a480d07a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:30:26.372 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:26.369+0000 7f2a4f80c640 1 -- 172.21.15.37:0/4138399927 shutdown_connections 2024-04-25T10:30:26.372 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:26.369+0000 7f2a4f80c640 1 -- 172.21.15.37:0/4138399927 wait complete. 2024-04-25T10:30:26.373 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:26.371+0000 7f2a4f80c640 1 Processor -- start 2024-04-25T10:30:26.373 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:26.371+0000 7f2a4f80c640 1 -- start start 2024-04-25T10:30:26.376 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:26.372+0000 7f2a4f80c640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f2a4806e260 0x7f2a480cdf40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:30:26.376 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:26.372+0000 7f2a4f80c640 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f2a4806ebb0 0x7f2a481f63b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:30:26.376 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:26.372+0000 7f2a4f80c640 1 --2- >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f2a48164ce0 0x7f2a481f68f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:30:26.376 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:26.372+0000 7f2a4f80c640 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 -- 0x7f2a4806d830 con 0x7f2a4806e260 2024-04-25T10:30:26.376 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:26.372+0000 7f2a4f80c640 1 -- --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_getmap magic: 0 -- 0x7f2a480d3090 con 0x7f2a4806ebb0 2024-04-25T10:30:26.376 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:26.372+0000 7f2a4f80c640 1 -- --> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] -- mon_getmap magic: 0 -- 0x7f2a481650f0 con 0x7f2a48164ce0 2024-04-25T10:30:26.376 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:26.373+0000 7f2a4d5a8640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f2a4806e260 0x7f2a480cdf40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:30:26.376 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:26.373+0000 7f2a4d5a8640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f2a4806e260 0x7f2a480cdf40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.37:3300/0 says I am v2:172.21.15.37:59898/0 (socket says 172.21.15.37:59898) 2024-04-25T10:30:26.376 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:26.373+0000 7f2a4d5a8640 1 -- 172.21.15.37:0/1886901884 learned_addr learned my addr 172.21.15.37:0/1886901884 (peer_addr_for_me v2:172.21.15.37:0/0) 2024-04-25T10:30:26.376 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:26.373+0000 7f2a4cda7640 1 --2- 172.21.15.37:0/1886901884 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f2a4806ebb0 0x7f2a481f63b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:30:26.376 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:26.373+0000 7f2a4dda9640 1 --2- 172.21.15.37:0/1886901884 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f2a48164ce0 0x7f2a481f68f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:30:26.376 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:26.373+0000 7f2a4d5a8640 1 -- 172.21.15.37:0/1886901884 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f2a48164ce0 msgr2=0x7f2a481f68f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:30:26.376 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:26.373+0000 7f2a4d5a8640 1 --2- 172.21.15.37:0/1886901884 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f2a48164ce0 0x7f2a481f68f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:30:26.376 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:26.373+0000 7f2a4d5a8640 1 -- 172.21.15.37:0/1886901884 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f2a4806ebb0 msgr2=0x7f2a481f63b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:30:26.377 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:26.373+0000 7f2a4d5a8640 1 --2- 172.21.15.37:0/1886901884 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f2a4806ebb0 0x7f2a481f63b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:30:26.377 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:26.373+0000 7f2a4d5a8640 1 -- 172.21.15.37:0/1886901884 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f2a480cf830 con 0x7f2a4806e260 2024-04-25T10:30:26.377 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:26.373+0000 7f2a4dda9640 1 --2- 172.21.15.37:0/1886901884 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f2a48164ce0 0x7f2a481f68f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T10:30:26.377 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:26.373+0000 7f2a4cda7640 1 --2- 172.21.15.37:0/1886901884 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f2a4806ebb0 0x7f2a481f63b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T10:30:26.377 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:26.373+0000 7f2a4d5a8640 1 --2- 172.21.15.37:0/1886901884 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f2a4806e260 0x7f2a480cdf40 secure :-1 s=READY pgs=505 cs=0 l=1 rev1=1 crypto rx=0x7f2a44035040 tx=0x7f2a440046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:30:26.377 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:26.373+0000 7f2a3dffb640 1 -- 172.21.15.37:0/1886901884 <== mon.0 v2:172.21.15.37:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2a44037040 con 0x7f2a4806e260 2024-04-25T10:30:26.377 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:26.373+0000 7f2a3dffb640 1 -- 172.21.15.37:0/1886901884 <== mon.0 v2:172.21.15.37:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f2a44004030 con 0x7f2a4806e260 2024-04-25T10:30:26.377 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:26.373+0000 7f2a4f80c640 1 -- 172.21.15.37:0/1886901884 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f2a480ce790 con 0x7f2a4806e260 2024-04-25T10:30:26.377 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:26.373+0000 7f2a3dffb640 1 -- 172.21.15.37:0/1886901884 <== mon.0 v2:172.21.15.37:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2a44006e10 con 0x7f2a4806e260 2024-04-25T10:30:26.377 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:26.374+0000 7f2a4f80c640 1 -- 172.21.15.37:0/1886901884 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f2a480ced50 con 0x7f2a4806e260 2024-04-25T10:30:26.378 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:26.375+0000 7f2a4f80c640 1 -- 172.21.15.37:0/1886901884 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f2a4806a140 con 0x7f2a4806e260 2024-04-25T10:30:26.378 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:26.375+0000 7f2a3dffb640 1 -- 172.21.15.37:0/1886901884 <== mon.0 v2:172.21.15.37:3300/0 4 ==== mgrmap(e 5) ==== 96782+0+0 (secure 0 0 0) 0x7f2a4403d650 con 0x7f2a4806e260 2024-04-25T10:30:26.378 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:26.376+0000 7f2a3dffb640 1 --2- 172.21.15.37:0/1886901884 >> [v2:172.21.15.203:6832/2418205237,v1:172.21.15.203:6833/2418205237] conn(0x7f2a2c0744e0 0x7f2a2c0769f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:30:26.378 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:26.376+0000 7f2a3dffb640 1 -- 172.21.15.37:0/1886901884 <== mon.0 v2:172.21.15.37:3300/0 5 ==== osd_map(41..41 src has 1..41) ==== 5795+0+0 (secure 0 0 0) 0x7f2a440b56d0 con 0x7f2a4806e260 2024-04-25T10:30:26.378 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:26.376+0000 7f2a4cda7640 1 --2- 172.21.15.37:0/1886901884 >> [v2:172.21.15.203:6832/2418205237,v1:172.21.15.203:6833/2418205237] conn(0x7f2a2c0744e0 0x7f2a2c0769f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:30:26.379 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:26.377+0000 7f2a4cda7640 1 --2- 172.21.15.37:0/1886901884 >> [v2:172.21.15.203:6832/2418205237,v1:172.21.15.203:6833/2418205237] conn(0x7f2a2c0744e0 0x7f2a2c0769f0 secure :-1 s=READY pgs=244 cs=0 l=1 rev1=1 crypto rx=0x7f2a48164150 tx=0x7f2a38007cc0 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:30:26.382 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:26.381+0000 7f2a3dffb640 1 -- 172.21.15.37:0/1886901884 <== mon.0 v2:172.21.15.37:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187246 (secure 0 0 0) 0x7f2a44042020 con 0x7f2a4806e260 2024-04-25T10:30:26.569 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:26.567+0000 7f2a4f80c640 1 -- 172.21.15.37:0/1886901884 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"} v 0) -- 0x7f2a4809f770 con 0x7f2a4806e260 2024-04-25T10:30:26.569 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:26.567+0000 7f2a3dffb640 1 -- 172.21.15.37:0/1886901884 <== mon.0 v2:172.21.15.37:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]=0 v41) ==== 100+0+11 (secure 0 0 0) 0x7f2a4407fe00 con 0x7f2a4806e260 2024-04-25T10:30:26.569 INFO:teuthology.orchestra.run.smithi037.stdout:pg_num: 64 2024-04-25T10:30:26.572 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:26.569+0000 7f2a4f80c640 1 -- 172.21.15.37:0/1886901884 >> [v2:172.21.15.203:6832/2418205237,v1:172.21.15.203:6833/2418205237] conn(0x7f2a2c0744e0 msgr2=0x7f2a2c0769f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:30:26.572 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:26.569+0000 7f2a4f80c640 1 --2- 172.21.15.37:0/1886901884 >> [v2:172.21.15.203:6832/2418205237,v1:172.21.15.203:6833/2418205237] conn(0x7f2a2c0744e0 0x7f2a2c0769f0 secure :-1 s=READY pgs=244 cs=0 l=1 rev1=1 crypto rx=0x7f2a48164150 tx=0x7f2a38007cc0 comp rx=0 tx=0).stop 2024-04-25T10:30:26.572 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:26.569+0000 7f2a4f80c640 1 -- 172.21.15.37:0/1886901884 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f2a4806e260 msgr2=0x7f2a480cdf40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:30:26.572 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:26.569+0000 7f2a4f80c640 1 --2- 172.21.15.37:0/1886901884 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f2a4806e260 0x7f2a480cdf40 secure :-1 s=READY pgs=505 cs=0 l=1 rev1=1 crypto rx=0x7f2a44035040 tx=0x7f2a440046c0 comp rx=0 tx=0).stop 2024-04-25T10:30:26.572 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:26.569+0000 7f2a4f80c640 1 -- 172.21.15.37:0/1886901884 shutdown_connections 2024-04-25T10:30:26.572 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:26.569+0000 7f2a4f80c640 1 --2- 172.21.15.37:0/1886901884 >> [v2:172.21.15.203:6832/2418205237,v1:172.21.15.203:6833/2418205237] conn(0x7f2a2c0744e0 0x7f2a2c0769f0 unknown :-1 s=CLOSED pgs=244 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:30:26.572 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:26.569+0000 7f2a4f80c640 1 --2- 172.21.15.37:0/1886901884 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f2a48164ce0 0x7f2a481f68f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:30:26.572 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:26.569+0000 7f2a4f80c640 1 --2- 172.21.15.37:0/1886901884 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f2a4806ebb0 0x7f2a481f63b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:30:26.573 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:26.569+0000 7f2a4f80c640 1 --2- 172.21.15.37:0/1886901884 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f2a4806e260 0x7f2a480cdf40 unknown :-1 s=CLOSED pgs=505 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:30:26.573 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:26.570+0000 7f2a4f80c640 1 -- 172.21.15.37:0/1886901884 >> 172.21.15.37:0/1886901884 conn(0x7f2a480d0390 msgr2=0x7f2a48162c90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:30:26.573 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:26.570+0000 7f2a4f80c640 1 -- 172.21.15.37:0/1886901884 shutdown_connections 2024-04-25T10:30:26.573 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:26.570+0000 7f2a4f80c640 1 -- 172.21.15.37:0/1886901884 wait complete. 2024-04-25T10:30:26.583 DEBUG:tasks.cephfs.filesystem:Creating filesystem 'recovery_fs' 2024-04-25T10:30:26.583 DEBUG:teuthology.orchestra.run.smithi037:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool create recovery_fs_metadata --pg_num_min 64 2024-04-25T10:30:26.698 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:26.696+0000 7f1410868640 1 Processor -- start 2024-04-25T10:30:26.699 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:26.696+0000 7f1410868640 1 -- start start 2024-04-25T10:30:26.701 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:26.698+0000 7f1410868640 1 --2- >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f140c160260 0x7f140c0d4e30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:30:26.701 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:26.698+0000 7f1410868640 1 -- --> v1:172.21.15.37:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f140c0d0f70 con 0x7f140c0b79f0 2024-04-25T10:30:26.701 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:26.698+0000 7f1410868640 1 -- --> v1:172.21.15.203:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f140c0d1250 con 0x7f140c0d0940 2024-04-25T10:30:26.701 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:26.698+0000 7f1410868640 1 -- --> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] -- mon_getmap magic: 0 -- 0x7f140c0d0d50 con 0x7f140c160260 2024-04-25T10:30:26.701 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:26.698+0000 7f140ad9d640 1 --1- >> v1:172.21.15.37:6789/0 conn(0x7f140c0b79f0 0x7f140c0b7e00 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.37:6789/0 says I am v1:172.21.15.37:55204/0 (socket says 172.21.15.37:55204) 2024-04-25T10:30:26.701 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:26.698+0000 7f140ad9d640 1 -- 172.21.15.37:0/2054181477 learned_addr learned my addr 172.21.15.37:0/2054181477 (peer_addr_for_me v1:172.21.15.37:0/0) 2024-04-25T10:30:26.701 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:26.698+0000 7f1409d9b640 1 --2- 172.21.15.37:0/2054181477 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f140c160260 0x7f140c0d4e30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:30:26.701 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:26.698+0000 7f140959a640 1 -- 172.21.15.37:0/2054181477 <== mon.0 v1:172.21.15.37:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 3011624621 0 0) 0x7f140c0d0f70 con 0x7f140c0b79f0 2024-04-25T10:30:26.701 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:26.698+0000 7f140959a640 1 -- 172.21.15.37:0/2054181477 --> v1:172.21.15.37:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f13f8003520 con 0x7f140c0b79f0 2024-04-25T10:30:26.702 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:26.698+0000 7f140959a640 1 -- 172.21.15.37:0/2054181477 <== mon.1 v1:172.21.15.203:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 2402144843 0 0) 0x7f140c0d1250 con 0x7f140c0d0940 2024-04-25T10:30:26.702 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:26.698+0000 7f140959a640 1 -- 172.21.15.37:0/2054181477 --> v1:172.21.15.203:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f140c0d0f70 con 0x7f140c0d0940 2024-04-25T10:30:26.702 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:26.698+0000 7f1409d9b640 1 -- 172.21.15.37:0/2054181477 >> v1:172.21.15.203:6789/0 conn(0x7f140c0d0940 legacy=0x7f140c1647b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:30:26.702 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:26.698+0000 7f1409d9b640 1 -- 172.21.15.37:0/2054181477 >> v1:172.21.15.37:6789/0 conn(0x7f140c0b79f0 legacy=0x7f140c0b7e00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:30:26.702 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:26.698+0000 7f1409d9b640 1 -- 172.21.15.37:0/2054181477 --> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f140c0d4630 con 0x7f140c160260 2024-04-25T10:30:26.703 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:26.699+0000 7f1409d9b640 1 --2- 172.21.15.37:0/2054181477 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f140c160260 0x7f140c0d4e30 secure :-1 s=READY pgs=55 cs=0 l=1 rev1=1 crypto rx=0x7f13f40097d0 tx=0x7f13f402a210 comp rx=0 tx=0).ready entity=mon.2 client_cookie=7b36835e02fcd9cd server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:30:26.703 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:26.699+0000 7f140959a640 1 -- 172.21.15.37:0/2054181477 <== mon.2 v2:172.21.15.203:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f13f4037070 con 0x7f140c160260 2024-04-25T10:30:26.703 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:26.699+0000 7f140959a640 1 -- 172.21.15.37:0/2054181477 <== mon.2 v2:172.21.15.203:3301/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f13f4032440 con 0x7f140c160260 2024-04-25T10:30:26.703 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:26.699+0000 7f140959a640 1 -- 172.21.15.37:0/2054181477 <== mon.2 v2:172.21.15.203:3301/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f13f40327f0 con 0x7f140c160260 2024-04-25T10:30:26.704 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:26.700+0000 7f1410868640 1 -- 172.21.15.37:0/2054181477 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f140c160260 msgr2=0x7f140c0d4e30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:30:26.704 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:26.700+0000 7f1410868640 1 --2- 172.21.15.37:0/2054181477 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f140c160260 0x7f140c0d4e30 secure :-1 s=READY pgs=55 cs=0 l=1 rev1=1 crypto rx=0x7f13f40097d0 tx=0x7f13f402a210 comp rx=0 tx=0).stop 2024-04-25T10:30:26.704 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:26.700+0000 7f1410868640 1 -- 172.21.15.37:0/2054181477 shutdown_connections 2024-04-25T10:30:26.704 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:26.700+0000 7f1410868640 1 --2- 172.21.15.37:0/2054181477 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f140c160260 0x7f140c0d4e30 unknown :-1 s=CLOSED pgs=55 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:30:26.704 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:26.700+0000 7f1410868640 1 -- 172.21.15.37:0/2054181477 >> 172.21.15.37:0/2054181477 conn(0x7f140c0d5d70 msgr2=0x7f140c0d6180 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:30:26.704 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:26.700+0000 7f1410868640 1 -- 172.21.15.37:0/2054181477 shutdown_connections 2024-04-25T10:30:26.704 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:26.700+0000 7f1410868640 1 -- 172.21.15.37:0/2054181477 wait complete. 2024-04-25T10:30:26.704 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:26.702+0000 7f1410868640 1 Processor -- start 2024-04-25T10:30:26.705 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:26.702+0000 7f1410868640 1 -- start start 2024-04-25T10:30:26.707 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:26.703+0000 7f1410868640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f140c0b79f0 0x7f140c145c20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:30:26.707 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:26.703+0000 7f1410868640 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f140c0d0940 0x7f140c1f9b90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:30:26.707 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:26.703+0000 7f1410868640 1 --2- >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f140c160260 0x7f140c1fa0d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:30:26.707 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:26.703+0000 7f1410868640 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 -- 0x7f140c0d1530 con 0x7f140c0b79f0 2024-04-25T10:30:26.707 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:26.703+0000 7f1410868640 1 -- --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_getmap magic: 0 -- 0x7f140c1606b0 con 0x7f140c0d0940 2024-04-25T10:30:26.707 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:26.703+0000 7f1410868640 1 -- --> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] -- mon_getmap magic: 0 -- 0x7f140c0d5370 con 0x7f140c160260 2024-04-25T10:30:26.707 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:26.704+0000 7f140a59c640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f140c0b79f0 0x7f140c145c20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:30:26.708 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:26.704+0000 7f140a59c640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f140c0b79f0 0x7f140c145c20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.37:3300/0 says I am v2:172.21.15.37:59902/0 (socket says 172.21.15.37:59902) 2024-04-25T10:30:26.708 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:26.704+0000 7f140a59c640 1 -- 172.21.15.37:0/2667222601 learned_addr learned my addr 172.21.15.37:0/2667222601 (peer_addr_for_me v2:172.21.15.37:0/0) 2024-04-25T10:30:26.708 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:26.704+0000 7f1409d9b640 1 --2- 172.21.15.37:0/2667222601 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f140c0d0940 0x7f140c1f9b90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:30:26.708 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:26.704+0000 7f140ad9d640 1 --2- 172.21.15.37:0/2667222601 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f140c160260 0x7f140c1fa0d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:30:26.708 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:26.704+0000 7f140a59c640 1 -- 172.21.15.37:0/2667222601 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f140c160260 msgr2=0x7f140c1fa0d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:30:26.708 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:26.704+0000 7f140a59c640 1 --2- 172.21.15.37:0/2667222601 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f140c160260 0x7f140c1fa0d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:30:26.708 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:26.704+0000 7f140a59c640 1 -- 172.21.15.37:0/2667222601 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f140c0d0940 msgr2=0x7f140c1f9b90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:30:26.708 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:26.704+0000 7f140a59c640 1 --2- 172.21.15.37:0/2667222601 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f140c0d0940 0x7f140c1f9b90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:30:26.708 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:26.704+0000 7f140a59c640 1 -- 172.21.15.37:0/2667222601 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f140c0c9ee0 con 0x7f140c0b79f0 2024-04-25T10:30:26.708 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:26.704+0000 7f140a59c640 1 --2- 172.21.15.37:0/2667222601 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f140c0b79f0 0x7f140c145c20 secure :-1 s=READY pgs=507 cs=0 l=1 rev1=1 crypto rx=0x7f140400c460 tx=0x7f140400c920 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:30:26.708 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:26.704+0000 7f1402ffd640 1 -- 172.21.15.37:0/2667222601 <== mon.0 v2:172.21.15.37:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1404017070 con 0x7f140c0b79f0 2024-04-25T10:30:26.708 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:26.704+0000 7f1402ffd640 1 -- 172.21.15.37:0/2667222601 <== mon.0 v2:172.21.15.37:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f14040052f0 con 0x7f140c0b79f0 2024-04-25T10:30:26.709 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:26.704+0000 7f1410868640 1 -- 172.21.15.37:0/2667222601 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f140c146160 con 0x7f140c0b79f0 2024-04-25T10:30:26.709 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:26.704+0000 7f1402ffd640 1 -- 172.21.15.37:0/2667222601 <== mon.0 v2:172.21.15.37:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1404003830 con 0x7f140c0b79f0 2024-04-25T10:30:26.709 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:26.704+0000 7f1410868640 1 -- 172.21.15.37:0/2667222601 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f140c163370 con 0x7f140c0b79f0 2024-04-25T10:30:26.710 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:26.706+0000 7f1410868640 1 -- 172.21.15.37:0/2667222601 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f140c06a140 con 0x7f140c0b79f0 2024-04-25T10:30:26.710 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:26.706+0000 7f1402ffd640 1 -- 172.21.15.37:0/2667222601 <== mon.0 v2:172.21.15.37:3300/0 4 ==== mgrmap(e 5) ==== 96782+0+0 (secure 0 0 0) 0x7f140401a070 con 0x7f140c0b79f0 2024-04-25T10:30:26.710 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:26.707+0000 7f1402ffd640 1 --2- 172.21.15.37:0/2667222601 >> [v2:172.21.15.203:6832/2418205237,v1:172.21.15.203:6833/2418205237] conn(0x7f13ec0744b0 0x7f13ec0769c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:30:26.710 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:26.707+0000 7f1402ffd640 1 -- 172.21.15.37:0/2667222601 <== mon.0 v2:172.21.15.37:3300/0 5 ==== osd_map(41..41 src has 1..41) ==== 5795+0+0 (secure 0 0 0) 0x7f14040945c0 con 0x7f140c0b79f0 2024-04-25T10:30:26.710 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:26.707+0000 7f1409d9b640 1 --2- 172.21.15.37:0/2667222601 >> [v2:172.21.15.203:6832/2418205237,v1:172.21.15.203:6833/2418205237] conn(0x7f13ec0744b0 0x7f13ec0769c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:30:26.710 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:26.708+0000 7f1409d9b640 1 --2- 172.21.15.37:0/2667222601 >> [v2:172.21.15.203:6832/2418205237,v1:172.21.15.203:6833/2418205237] conn(0x7f13ec0744b0 0x7f13ec0769c0 secure :-1 s=READY pgs=245 cs=0 l=1 rev1=1 crypto rx=0x7f13f4034000 tx=0x7f13f402e000 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:30:26.713 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:26.711+0000 7f1402ffd640 1 -- 172.21.15.37:0/2667222601 <== mon.0 v2:172.21.15.37:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187246 (secure 0 0 0) 0x7f140405e6b0 con 0x7f140c0b79f0 2024-04-25T10:30:26.901 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:26.899+0000 7f1410868640 1 -- 172.21.15.37:0/2667222601 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_command({"prefix": "osd pool create", "pool": "recovery_fs_metadata", "pg_num_min": 64} v 0) -- 0x7f140c0d3f90 con 0x7f140c0b79f0 2024-04-25T10:30:27.867 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:27.864+0000 7f1402ffd640 1 -- 172.21.15.37:0/2667222601 <== mon.0 v2:172.21.15.37:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool create", "pool": "recovery_fs_metadata", "pg_num_min": 64}]=0 pool 'recovery_fs_metadata' created v42) ==== 148+0+0 (secure 0 0 0) 0x7f14040629b0 con 0x7f140c0b79f0 2024-04-25T10:30:27.867 INFO:teuthology.orchestra.run.smithi037.stderr:pool 'recovery_fs_metadata' created 2024-04-25T10:30:27.870 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:27.868+0000 7f1410868640 1 -- 172.21.15.37:0/2667222601 >> [v2:172.21.15.203:6832/2418205237,v1:172.21.15.203:6833/2418205237] conn(0x7f13ec0744b0 msgr2=0x7f13ec0769c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:30:27.870 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:27.868+0000 7f1410868640 1 --2- 172.21.15.37:0/2667222601 >> [v2:172.21.15.203:6832/2418205237,v1:172.21.15.203:6833/2418205237] conn(0x7f13ec0744b0 0x7f13ec0769c0 secure :-1 s=READY pgs=245 cs=0 l=1 rev1=1 crypto rx=0x7f13f4034000 tx=0x7f13f402e000 comp rx=0 tx=0).stop 2024-04-25T10:30:27.870 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:27.868+0000 7f1410868640 1 -- 172.21.15.37:0/2667222601 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f140c0b79f0 msgr2=0x7f140c145c20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:30:27.870 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:27.868+0000 7f1410868640 1 --2- 172.21.15.37:0/2667222601 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f140c0b79f0 0x7f140c145c20 secure :-1 s=READY pgs=507 cs=0 l=1 rev1=1 crypto rx=0x7f140400c460 tx=0x7f140400c920 comp rx=0 tx=0).stop 2024-04-25T10:30:27.871 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:27.868+0000 7f1410868640 1 -- 172.21.15.37:0/2667222601 shutdown_connections 2024-04-25T10:30:27.871 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:27.868+0000 7f1410868640 1 --2- 172.21.15.37:0/2667222601 >> [v2:172.21.15.203:6832/2418205237,v1:172.21.15.203:6833/2418205237] conn(0x7f13ec0744b0 0x7f13ec0769c0 unknown :-1 s=CLOSED pgs=245 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:30:27.871 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:27.868+0000 7f1410868640 1 --2- 172.21.15.37:0/2667222601 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f140c160260 0x7f140c1fa0d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:30:27.871 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:27.868+0000 7f1410868640 1 --2- 172.21.15.37:0/2667222601 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f140c0d0940 0x7f140c1f9b90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:30:27.871 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:27.868+0000 7f1410868640 1 --2- 172.21.15.37:0/2667222601 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f140c0b79f0 0x7f140c145c20 unknown :-1 s=CLOSED pgs=507 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:30:27.871 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:27.868+0000 7f1410868640 1 -- 172.21.15.37:0/2667222601 >> 172.21.15.37:0/2667222601 conn(0x7f140c0d5d70 msgr2=0x7f140c0d6180 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:30:27.871 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:27.868+0000 7f1410868640 1 -- 172.21.15.37:0/2667222601 shutdown_connections 2024-04-25T10:30:27.871 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:27.868+0000 7f1410868640 1 -- 172.21.15.37:0/2667222601 wait complete. 2024-04-25T10:30:27.889 DEBUG:teuthology.orchestra.run.smithi037:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool create cephfs_data 64 --pg_num_min 64 --target_size_ratio 0.9 2024-04-25T10:30:28.014 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:28.012+0000 7f642e20c640 1 Processor -- start 2024-04-25T10:30:28.014 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:28.012+0000 7f642e20c640 1 -- start start 2024-04-25T10:30:28.019 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:28.013+0000 7f642e20c640 1 --2- >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f64280b2ca0 0x7f642808d3a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:30:28.019 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:28.013+0000 7f642e20c640 1 --2- >> v2:172.21.15.203:3300/0 conn(0x7f64280b3670 0x7f64280b3a80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:30:28.019 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:28.013+0000 7f642e20c640 1 -- --> v1:172.21.15.37:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f64280cf120 con 0x7f64280b5fd0 2024-04-25T10:30:28.019 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:28.013+0000 7f642e20c640 1 -- --> v2:172.21.15.203:3300/0 -- mon_getmap magic: 0 -- 0x7f64280b30b0 con 0x7f64280b3670 2024-04-25T10:30:28.019 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:28.013+0000 7f642e20c640 1 -- --> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] -- mon_getmap magic: 0 -- 0x7f64280cf400 con 0x7f64280b2ca0 2024-04-25T10:30:28.019 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:28.013+0000 7f64277fe640 1 --1- >> v1:172.21.15.37:6789/0 conn(0x7f64280b5fd0 0x7f642808cc90 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.37:6789/0 says I am v1:172.21.15.37:55210/0 (socket says 172.21.15.37:55210) 2024-04-25T10:30:28.019 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:28.013+0000 7f64277fe640 1 -- 172.21.15.37:0/3264198344 learned_addr learned my addr 172.21.15.37:0/3264198344 (peer_addr_for_me v1:172.21.15.37:0/0) 2024-04-25T10:30:28.019 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:28.014+0000 7f6426ffd640 1 --2- 172.21.15.37:0/3264198344 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f64280b2ca0 0x7f642808d3a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:30:28.019 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:28.014+0000 7f6427fff640 1 --2- 172.21.15.37:0/3264198344 >> v2:172.21.15.203:3300/0 conn(0x7f64280b3670 0x7f64280b3a80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:30:28.019 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:28.014+0000 7f64267fc640 1 -- 172.21.15.37:0/3264198344 <== mon.0 v1:172.21.15.37:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 2929741828 0 0) 0x7f64280cf120 con 0x7f64280b5fd0 2024-04-25T10:30:28.020 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:28.014+0000 7f64267fc640 1 -- 172.21.15.37:0/3264198344 --> v1:172.21.15.37:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f6410003520 con 0x7f64280b5fd0 2024-04-25T10:30:28.020 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:28.014+0000 7f64267fc640 1 -- 172.21.15.37:0/3264198344 <== mon.0 v1:172.21.15.37:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 3165705976 0 0) 0x7f6410003520 con 0x7f64280b5fd0 2024-04-25T10:30:28.020 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:28.014+0000 7f64267fc640 1 -- 172.21.15.37:0/3264198344 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f64280b2ca0 msgr2=0x7f642808d3a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:30:28.020 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:28.014+0000 7f64267fc640 1 --2- 172.21.15.37:0/3264198344 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f64280b2ca0 0x7f642808d3a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:30:28.020 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:28.014+0000 7f64267fc640 1 -- 172.21.15.37:0/3264198344 >> v2:172.21.15.203:3300/0 conn(0x7f64280b3670 msgr2=0x7f64280b3a80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:30:28.020 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:28.014+0000 7f64267fc640 1 --2- 172.21.15.37:0/3264198344 >> v2:172.21.15.203:3300/0 conn(0x7f64280b3670 0x7f64280b3a80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:30:28.020 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:28.014+0000 7f64267fc640 1 -- 172.21.15.37:0/3264198344 --> v1:172.21.15.37:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f64280cf6e0 con 0x7f64280b5fd0 2024-04-25T10:30:28.020 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:28.014+0000 7f64267fc640 1 -- 172.21.15.37:0/3264198344 <== mon.0 v1:172.21.15.37:6789/0 3 ==== mon_map magic: 0 ==== 413+0+0 (unknown 1577794600 0 0) 0x7f64180017f0 con 0x7f64280b5fd0 2024-04-25T10:30:28.020 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:28.014+0000 7f64267fc640 1 -- 172.21.15.37:0/3264198344 >> v1:172.21.15.37:6789/0 conn(0x7f64280b5fd0 legacy=0x7f642808cc90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:30:28.020 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:28.014+0000 7f64267fc640 1 --2- 172.21.15.37:0/3264198344 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f64100039b0 0x7f6410003e20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:30:28.021 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:28.014+0000 7f64267fc640 1 --2- 172.21.15.37:0/3264198344 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f6410004800 0x7f6410004c10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:30:28.021 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:28.014+0000 7f64267fc640 1 --2- 172.21.15.37:0/3264198344 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f64280b2ca0 0x7f6410005150 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:30:28.021 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:28.014+0000 7f64267fc640 1 -- 172.21.15.37:0/3264198344 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 -- 0x7f64280b30b0 con 0x7f6410004800 2024-04-25T10:30:28.021 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:28.014+0000 7f64267fc640 1 -- 172.21.15.37:0/3264198344 --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_getmap magic: 0 -- 0x7f64280cf400 con 0x7f64100039b0 2024-04-25T10:30:28.021 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:28.014+0000 7f64267fc640 1 -- 172.21.15.37:0/3264198344 --> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] -- mon_getmap magic: 0 -- 0x7f641000b9f0 con 0x7f64280b2ca0 2024-04-25T10:30:28.021 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:28.014+0000 7f6426ffd640 1 --2- 172.21.15.37:0/3264198344 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f6410004800 0x7f6410004c10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:30:28.021 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:28.014+0000 7f64277fe640 1 --2- 172.21.15.37:0/3264198344 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f64100039b0 0x7f6410003e20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:30:28.021 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:28.014+0000 7f6427fff640 1 --2- 172.21.15.37:0/3264198344 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f64280b2ca0 0x7f6410005150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:30:28.021 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:28.015+0000 7f6426ffd640 1 -- 172.21.15.37:0/3264198344 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f64280b2ca0 msgr2=0x7f6410005150 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:30:28.022 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:28.015+0000 7f6426ffd640 1 --2- 172.21.15.37:0/3264198344 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f64280b2ca0 0x7f6410005150 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:30:28.022 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:28.015+0000 7f6426ffd640 1 -- 172.21.15.37:0/3264198344 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f64100039b0 msgr2=0x7f6410003e20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:30:28.022 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:28.015+0000 7f6426ffd640 1 --2- 172.21.15.37:0/3264198344 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f64100039b0 0x7f6410003e20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:30:28.022 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:28.015+0000 7f6426ffd640 1 -- 172.21.15.37:0/3264198344 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f6410003520 con 0x7f6410004800 2024-04-25T10:30:28.022 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:28.015+0000 7f6426ffd640 1 --2- 172.21.15.37:0/3264198344 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f6410004800 0x7f6410004c10 secure :-1 s=READY pgs=509 cs=0 l=1 rev1=1 crypto rx=0x7f641c009a90 tx=0x7f6410005830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=30891fa632d5cd11 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:30:28.022 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:28.016+0000 7f64267fc640 1 -- 172.21.15.37:0/3264198344 <== mon.0 v2:172.21.15.37:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f641c036070 con 0x7f6410004800 2024-04-25T10:30:28.022 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:28.016+0000 7f64267fc640 1 -- 172.21.15.37:0/3264198344 <== mon.0 v2:172.21.15.37:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f641c031400 con 0x7f6410004800 2024-04-25T10:30:28.023 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:28.016+0000 7f64267fc640 1 -- 172.21.15.37:0/3264198344 <== mon.0 v2:172.21.15.37:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f641c009690 con 0x7f6410004800 2024-04-25T10:30:28.023 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:28.016+0000 7f642e20c640 1 -- 172.21.15.37:0/3264198344 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f6410004800 msgr2=0x7f6410004c10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:30:28.023 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:28.016+0000 7f642e20c640 1 --2- 172.21.15.37:0/3264198344 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f6410004800 0x7f6410004c10 secure :-1 s=READY pgs=509 cs=0 l=1 rev1=1 crypto rx=0x7f641c009a90 tx=0x7f6410005830 comp rx=0 tx=0).stop 2024-04-25T10:30:28.023 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:28.016+0000 7f6427fff640 1 -- 172.21.15.37:0/3264198344 reap_dead start 2024-04-25T10:30:28.023 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:28.016+0000 7f642e20c640 1 -- 172.21.15.37:0/3264198344 shutdown_connections 2024-04-25T10:30:28.023 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:28.016+0000 7f642e20c640 1 -- 172.21.15.37:0/3264198344 >> 172.21.15.37:0/3264198344 conn(0x7f64280b5440 msgr2=0x7f64280b5850 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:30:28.023 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:28.016+0000 7f642e20c640 1 -- 172.21.15.37:0/3264198344 shutdown_connections 2024-04-25T10:30:28.023 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:28.016+0000 7f642e20c640 1 -- 172.21.15.37:0/3264198344 wait complete. 2024-04-25T10:30:28.027 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:28.018+0000 7f642e20c640 1 Processor -- start 2024-04-25T10:30:28.028 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:28.018+0000 7f642e20c640 1 -- start start 2024-04-25T10:30:28.028 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:28.020+0000 7f642e20c640 1 --2- >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f6410004800 0x7f64281efdb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:30:28.028 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:28.020+0000 7f642e20c640 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f64280b88a0 0x7f6428160990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:30:28.028 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:28.020+0000 7f642e20c640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f6428160ed0 0x7f642806d650 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:30:28.028 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:28.020+0000 7f642e20c640 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 -- 0x7f642808d3a0 con 0x7f6428160ed0 2024-04-25T10:30:28.028 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:28.020+0000 7f642e20c640 1 -- --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_getmap magic: 0 -- 0x7f64280d28d0 con 0x7f64280b88a0 2024-04-25T10:30:28.029 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:28.020+0000 7f642e20c640 1 -- --> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] -- mon_getmap magic: 0 -- 0x7f642808d520 con 0x7f6410004800 2024-04-25T10:30:28.029 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:28.020+0000 7f6427fff640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f6428160ed0 0x7f642806d650 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:30:28.029 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:28.020+0000 7f6427fff640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f6428160ed0 0x7f642806d650 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.37:3300/0 says I am v2:172.21.15.37:59920/0 (socket says 172.21.15.37:59920) 2024-04-25T10:30:28.029 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:28.020+0000 7f6427fff640 1 -- 172.21.15.37:0/3968849135 learned_addr learned my addr 172.21.15.37:0/3968849135 (peer_addr_for_me v2:172.21.15.37:0/0) 2024-04-25T10:30:28.029 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:28.020+0000 7f64277fe640 1 --2- 172.21.15.37:0/3968849135 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f6410004800 0x7f64281efdb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:30:28.029 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:28.020+0000 7f6426ffd640 1 --2- 172.21.15.37:0/3968849135 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f64280b88a0 0x7f6428160990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:30:28.029 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:28.020+0000 7f6427fff640 1 -- 172.21.15.37:0/3968849135 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f6410004800 msgr2=0x7f64281efdb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:30:28.029 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:28.020+0000 7f6427fff640 1 --2- 172.21.15.37:0/3968849135 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f6410004800 0x7f64281efdb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:30:28.030 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:28.020+0000 7f6427fff640 1 -- 172.21.15.37:0/3968849135 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f64280b88a0 msgr2=0x7f6428160990 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:30:28.030 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:28.020+0000 7f6427fff640 1 --2- 172.21.15.37:0/3968849135 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f64280b88a0 0x7f6428160990 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:30:28.030 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:28.020+0000 7f6427fff640 1 -- 172.21.15.37:0/3968849135 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f64281612e0 con 0x7f6428160ed0 2024-04-25T10:30:28.030 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:28.020+0000 7f6427fff640 1 --2- 172.21.15.37:0/3968849135 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f6428160ed0 0x7f642806d650 secure :-1 s=READY pgs=510 cs=0 l=1 rev1=1 crypto rx=0x7f642815e720 tx=0x7f642000f580 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:30:28.030 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:28.020+0000 7f640ffff640 1 -- 172.21.15.37:0/3968849135 <== mon.0 v2:172.21.15.37:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f642000fe10 con 0x7f6428160ed0 2024-04-25T10:30:28.030 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:28.020+0000 7f640ffff640 1 -- 172.21.15.37:0/3968849135 <== mon.0 v2:172.21.15.37:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f642001d070 con 0x7f6428160ed0 2024-04-25T10:30:28.030 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:28.020+0000 7f642e20c640 1 -- 172.21.15.37:0/3968849135 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f64280cb890 con 0x7f6428160ed0 2024-04-25T10:30:28.030 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:28.020+0000 7f640ffff640 1 -- 172.21.15.37:0/3968849135 <== mon.0 v2:172.21.15.37:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6420015670 con 0x7f6428160ed0 2024-04-25T10:30:28.031 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:28.021+0000 7f642e20c640 1 -- 172.21.15.37:0/3968849135 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f64280cbb30 con 0x7f6428160ed0 2024-04-25T10:30:28.031 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:28.022+0000 7f642e20c640 1 -- 172.21.15.37:0/3968849135 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f642806a140 con 0x7f6428160ed0 2024-04-25T10:30:28.031 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:28.023+0000 7f640ffff640 1 -- 172.21.15.37:0/3968849135 <== mon.0 v2:172.21.15.37:3300/0 4 ==== mgrmap(e 5) ==== 96782+0+0 (secure 0 0 0) 0x7f6420015810 con 0x7f6428160ed0 2024-04-25T10:30:28.031 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:28.023+0000 7f640ffff640 1 --2- 172.21.15.37:0/3968849135 >> [v2:172.21.15.203:6832/2418205237,v1:172.21.15.203:6833/2418205237] conn(0x7f64140744b0 0x7f64140769c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:30:28.031 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:28.023+0000 7f640ffff640 1 -- 172.21.15.37:0/3968849135 <== mon.0 v2:172.21.15.37:3300/0 5 ==== osd_map(42..42 src has 1..42) ==== 6170+0+0 (secure 0 0 0) 0x7f6420095e70 con 0x7f6428160ed0 2024-04-25T10:30:28.031 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:28.023+0000 7f64277fe640 1 --2- 172.21.15.37:0/3968849135 >> [v2:172.21.15.203:6832/2418205237,v1:172.21.15.203:6833/2418205237] conn(0x7f64140744b0 0x7f64140769c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:30:28.031 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:28.024+0000 7f64277fe640 1 --2- 172.21.15.37:0/3968849135 >> [v2:172.21.15.203:6832/2418205237,v1:172.21.15.203:6833/2418205237] conn(0x7f64140744b0 0x7f64140769c0 secure :-1 s=READY pgs=246 cs=0 l=1 rev1=1 crypto rx=0x7f64180011d0 tx=0x7f6418009040 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:30:28.032 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:28.028+0000 7f640ffff640 1 -- 172.21.15.37:0/3968849135 <== mon.0 v2:172.21.15.37:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187246 (secure 0 0 0) 0x7f64281612e0 con 0x7f6428160ed0 2024-04-25T10:30:28.215 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:28.213+0000 7f642e20c640 1 -- 172.21.15.37:0/3968849135 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_command({"prefix": "osd pool create", "pool": "cephfs_data", "pg_num": 64, "pg_num_min": 64, "target_size_ratio": 0.9} v 0) -- 0x7f642815e130 con 0x7f6428160ed0 2024-04-25T10:30:28.215 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:28.213+0000 7f640ffff640 1 -- 172.21.15.37:0/3968849135 <== mon.0 v2:172.21.15.37:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool create", "pool": "cephfs_data", "pg_num": 64, "pg_num_min": 64, "target_size_ratio": 0.9}]=0 pool 'cephfs_data' already exists v42) ==== 177+0+0 (secure 0 0 0) 0x7f6420010040 con 0x7f6428160ed0 2024-04-25T10:30:28.215 INFO:teuthology.orchestra.run.smithi037.stderr:pool 'cephfs_data' already exists 2024-04-25T10:30:28.218 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:28.215+0000 7f642e20c640 1 -- 172.21.15.37:0/3968849135 >> [v2:172.21.15.203:6832/2418205237,v1:172.21.15.203:6833/2418205237] conn(0x7f64140744b0 msgr2=0x7f64140769c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:30:28.218 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:28.215+0000 7f642e20c640 1 --2- 172.21.15.37:0/3968849135 >> [v2:172.21.15.203:6832/2418205237,v1:172.21.15.203:6833/2418205237] conn(0x7f64140744b0 0x7f64140769c0 secure :-1 s=READY pgs=246 cs=0 l=1 rev1=1 crypto rx=0x7f64180011d0 tx=0x7f6418009040 comp rx=0 tx=0).stop 2024-04-25T10:30:28.218 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:28.215+0000 7f642e20c640 1 -- 172.21.15.37:0/3968849135 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f6428160ed0 msgr2=0x7f642806d650 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:30:28.218 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:28.215+0000 7f642e20c640 1 --2- 172.21.15.37:0/3968849135 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f6428160ed0 0x7f642806d650 secure :-1 s=READY pgs=510 cs=0 l=1 rev1=1 crypto rx=0x7f642815e720 tx=0x7f642000f580 comp rx=0 tx=0).stop 2024-04-25T10:30:28.218 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:28.215+0000 7f642e20c640 1 -- 172.21.15.37:0/3968849135 shutdown_connections 2024-04-25T10:30:28.218 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:28.215+0000 7f642e20c640 1 --2- 172.21.15.37:0/3968849135 >> [v2:172.21.15.203:6832/2418205237,v1:172.21.15.203:6833/2418205237] conn(0x7f64140744b0 0x7f64140769c0 unknown :-1 s=CLOSED pgs=246 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:30:28.218 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:28.215+0000 7f642e20c640 1 --2- 172.21.15.37:0/3968849135 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f6428160ed0 0x7f642806d650 unknown :-1 s=CLOSED pgs=510 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:30:28.218 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:28.215+0000 7f642e20c640 1 --2- 172.21.15.37:0/3968849135 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f64280b88a0 0x7f6428160990 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:30:28.218 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:28.215+0000 7f642e20c640 1 --2- 172.21.15.37:0/3968849135 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f6410004800 0x7f64281efdb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:30:28.218 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:28.215+0000 7f642e20c640 1 -- 172.21.15.37:0/3968849135 >> 172.21.15.37:0/3968849135 conn(0x7f64280b5440 msgr2=0x7f64280726f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:30:28.218 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:28.216+0000 7f642e20c640 1 -- 172.21.15.37:0/3968849135 shutdown_connections 2024-04-25T10:30:28.219 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:28.216+0000 7f642e20c640 1 -- 172.21.15.37:0/3968849135 wait complete. 2024-04-25T10:30:28.229 DEBUG:teuthology.orchestra.run.smithi037:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs new recovery_fs recovery_fs_metadata cephfs_data --recover --allow-dangerous-metadata-overlay 2024-04-25T10:30:28.346 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:28.343+0000 7f4f35cd1640 1 Processor -- start 2024-04-25T10:30:28.346 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:28.343+0000 7f4f35cd1640 1 -- start start 2024-04-25T10:30:28.347 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:28.345+0000 7f4f35cd1640 1 --2- >> v2:172.21.15.37:3300/0 conn(0x7f4f300d13c0 0x7f4f30145a20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:30:28.347 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:28.345+0000 7f4f35cd1640 1 --2- >> v2:172.21.15.203:3300/0 conn(0x7f4f30073b80 0x7f4f300702e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:30:28.347 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:28.345+0000 7f4f35cd1640 1 -- --> v1:172.21.15.37:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f4f30070c30 con 0x7f4f30070820 2024-04-25T10:30:28.348 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:28.345+0000 7f4f35cd1640 1 -- --> v2:172.21.15.37:3300/0 -- mon_getmap magic: 0 -- 0x7f4f30145f60 con 0x7f4f300d13c0 2024-04-25T10:30:28.348 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:28.345+0000 7f4f35cd1640 1 -- --> v2:172.21.15.203:3300/0 -- mon_getmap magic: 0 -- 0x7f4f300d0f30 con 0x7f4f30073b80 2024-04-25T10:30:28.348 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:28.345+0000 7f4f2f7fe640 1 --2- >> v2:172.21.15.37:3300/0 conn(0x7f4f300d13c0 0x7f4f30145a20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:30:28.348 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:28.345+0000 7f4f2ffff640 1 --1- >> v1:172.21.15.37:6789/0 conn(0x7f4f30070820 0x7f4f301609c0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.37:6789/0 says I am v1:172.21.15.37:55216/0 (socket says 172.21.15.37:55216) 2024-04-25T10:30:28.348 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:28.345+0000 7f4f2f7fe640 1 --2- >> v2:172.21.15.37:3300/0 conn(0x7f4f300d13c0 0x7f4f30145a20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.37:3300/0 says I am v2:172.21.15.37:59922/0 (socket says 172.21.15.37:59922) 2024-04-25T10:30:28.348 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:28.345+0000 7f4f2ffff640 1 -- 172.21.15.37:0/961613681 learned_addr learned my addr 172.21.15.37:0/961613681 (peer_addr_for_me v1:172.21.15.37:0/0) 2024-04-25T10:30:28.348 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:28.345+0000 7f4f2effd640 1 --2- 172.21.15.37:0/961613681 >> v2:172.21.15.203:3300/0 conn(0x7f4f30073b80 0x7f4f300702e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:30:28.348 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:28.345+0000 7f4f2f7fe640 1 -- 172.21.15.37:0/961613681 >> v2:172.21.15.203:3300/0 conn(0x7f4f30073b80 msgr2=0x7f4f300702e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:30:28.348 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:28.345+0000 7f4f2f7fe640 1 --2- 172.21.15.37:0/961613681 >> v2:172.21.15.203:3300/0 conn(0x7f4f30073b80 0x7f4f300702e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:30:28.348 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:28.345+0000 7f4f2f7fe640 1 -- 172.21.15.37:0/961613681 >> v1:172.21.15.37:6789/0 conn(0x7f4f30070820 legacy=0x7f4f301609c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:30:28.348 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:28.345+0000 7f4f2f7fe640 1 -- 172.21.15.37:0/961613681 --> v2:172.21.15.37:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f4f300d0320 con 0x7f4f300d13c0 2024-04-25T10:30:28.348 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:28.345+0000 7f4f2e7fc640 1 -- 172.21.15.37:0/961613681 <== mon.0 v1:172.21.15.37:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 3145326693 0 0) 0x7f4f28001490 con 0x7f4f30070820 2024-04-25T10:30:28.348 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:28.345+0000 7f4f2effd640 1 --2- 172.21.15.37:0/961613681 >> v2:172.21.15.203:3300/0 conn(0x7f4f30073b80 0x7f4f300702e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T10:30:28.350 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:28.346+0000 7f4f2f7fe640 1 --2- 172.21.15.37:0/961613681 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f4f300d13c0 0x7f4f30145a20 secure :-1 s=READY pgs=512 cs=0 l=1 rev1=1 crypto rx=0x7f4f3006eb90 tx=0x7f4f2002a110 comp rx=0 tx=0).ready entity=mon.0 client_cookie=454317125a342ed8 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:30:28.350 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:28.346+0000 7f4f2e7fc640 1 -- 172.21.15.37:0/961613681 <== mon.0 v2:172.21.15.37:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4f20037070 con 0x7f4f300d13c0 2024-04-25T10:30:28.350 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:28.346+0000 7f4f2e7fc640 1 -- 172.21.15.37:0/961613681 <== mon.0 v2:172.21.15.37:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f4f20032440 con 0x7f4f300d13c0 2024-04-25T10:30:28.350 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:28.346+0000 7f4f2e7fc640 1 -- 172.21.15.37:0/961613681 <== mon.0 v2:172.21.15.37:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4f2002ae60 con 0x7f4f300d13c0 2024-04-25T10:30:28.350 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:28.347+0000 7f4f35cd1640 1 -- 172.21.15.37:0/961613681 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f4f300d13c0 msgr2=0x7f4f30145a20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:30:28.350 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:28.347+0000 7f4f35cd1640 1 --2- 172.21.15.37:0/961613681 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f4f300d13c0 0x7f4f30145a20 secure :-1 s=READY pgs=512 cs=0 l=1 rev1=1 crypto rx=0x7f4f3006eb90 tx=0x7f4f2002a110 comp rx=0 tx=0).stop 2024-04-25T10:30:28.350 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:28.347+0000 7f4f35cd1640 1 -- 172.21.15.37:0/961613681 shutdown_connections 2024-04-25T10:30:28.350 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:28.347+0000 7f4f35cd1640 1 --2- 172.21.15.37:0/961613681 >> v2:172.21.15.203:3300/0 conn(0x7f4f30073b80 0x7f4f300702e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:30:28.350 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:28.347+0000 7f4f35cd1640 1 --2- 172.21.15.37:0/961613681 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f4f300d13c0 0x7f4f30145a20 unknown :-1 s=CLOSED pgs=512 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:30:28.350 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:28.347+0000 7f4f35cd1640 1 -- 172.21.15.37:0/961613681 >> 172.21.15.37:0/961613681 conn(0x7f4f300b35c0 msgr2=0x7f4f300b39d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:30:28.350 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:28.347+0000 7f4f35cd1640 1 -- 172.21.15.37:0/961613681 shutdown_connections 2024-04-25T10:30:28.350 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:28.347+0000 7f4f35cd1640 1 -- 172.21.15.37:0/961613681 wait complete. 2024-04-25T10:30:28.351 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:28.349+0000 7f4f35cd1640 1 Processor -- start 2024-04-25T10:30:28.351 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:28.349+0000 7f4f35cd1640 1 -- start start 2024-04-25T10:30:28.354 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:28.350+0000 7f4f35cd1640 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f4f30070820 0x7f4f3013f580 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:30:28.354 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:28.350+0000 7f4f35cd1640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f4f30073b80 0x7f4f3015c530 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:30:28.354 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:28.350+0000 7f4f35cd1640 1 --2- >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f4f301e7830 0x7f4f301e7c40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:30:28.354 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:28.350+0000 7f4f35cd1640 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 -- 0x7f4f300d0bc0 con 0x7f4f30073b80 2024-04-25T10:30:28.354 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:28.350+0000 7f4f35cd1640 1 -- --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_getmap magic: 0 -- 0x7f4f300d08e0 con 0x7f4f30070820 2024-04-25T10:30:28.354 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:28.350+0000 7f4f35cd1640 1 -- --> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] -- mon_getmap magic: 0 -- 0x7f4f300cffd0 con 0x7f4f301e7830 2024-04-25T10:30:28.354 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:28.350+0000 7f4f2effd640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f4f30073b80 0x7f4f3015c530 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:30:28.354 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:28.351+0000 7f4f2effd640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f4f30073b80 0x7f4f3015c530 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.37:3300/0 says I am v2:172.21.15.37:59934/0 (socket says 172.21.15.37:59934) 2024-04-25T10:30:28.354 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:28.351+0000 7f4f2effd640 1 -- 172.21.15.37:0/3705536662 learned_addr learned my addr 172.21.15.37:0/3705536662 (peer_addr_for_me v2:172.21.15.37:0/0) 2024-04-25T10:30:28.354 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:28.351+0000 7f4f2ffff640 1 --2- 172.21.15.37:0/3705536662 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f4f301e7830 0x7f4f301e7c40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:30:28.355 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:28.351+0000 7f4f2f7fe640 1 --2- 172.21.15.37:0/3705536662 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f4f30070820 0x7f4f3013f580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:30:28.355 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:28.351+0000 7f4f2effd640 1 -- 172.21.15.37:0/3705536662 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f4f301e7830 msgr2=0x7f4f301e7c40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:30:28.355 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:28.351+0000 7f4f2effd640 1 --2- 172.21.15.37:0/3705536662 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f4f301e7830 0x7f4f301e7c40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:30:28.355 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:28.351+0000 7f4f2effd640 1 -- 172.21.15.37:0/3705536662 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f4f30070820 msgr2=0x7f4f3013f580 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:30:28.355 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:28.351+0000 7f4f2effd640 1 --2- 172.21.15.37:0/3705536662 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f4f30070820 0x7f4f3013f580 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:30:28.355 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:28.351+0000 7f4f2effd640 1 -- 172.21.15.37:0/3705536662 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f4f301e8180 con 0x7f4f30073b80 2024-04-25T10:30:28.355 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:28.351+0000 7f4f2effd640 1 --2- 172.21.15.37:0/3705536662 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f4f30073b80 0x7f4f3015c530 secure :-1 s=READY pgs=513 cs=0 l=1 rev1=1 crypto rx=0x7f4f2400d830 tx=0x7f4f2400dcf0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:30:28.355 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:28.351+0000 7f4f17fff640 1 -- 172.21.15.37:0/3705536662 <== mon.0 v2:172.21.15.37:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4f2400b6c0 con 0x7f4f30073b80 2024-04-25T10:30:28.355 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:28.351+0000 7f4f17fff640 1 -- 172.21.15.37:0/3705536662 <== mon.0 v2:172.21.15.37:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f4f24019070 con 0x7f4f30073b80 2024-04-25T10:30:28.355 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:28.351+0000 7f4f17fff640 1 -- 172.21.15.37:0/3705536662 <== mon.0 v2:172.21.15.37:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4f2400ba80 con 0x7f4f30073b80 2024-04-25T10:30:28.355 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:28.351+0000 7f4f35cd1640 1 -- 172.21.15.37:0/3705536662 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f4f3013fac0 con 0x7f4f30073b80 2024-04-25T10:30:28.355 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:28.351+0000 7f4f35cd1640 1 -- 172.21.15.37:0/3705536662 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f4f301402b0 con 0x7f4f30073b80 2024-04-25T10:30:28.356 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:28.352+0000 7f4f35cd1640 1 -- 172.21.15.37:0/3705536662 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f4f30145630 con 0x7f4f30073b80 2024-04-25T10:30:28.356 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:28.353+0000 7f4f17fff640 1 -- 172.21.15.37:0/3705536662 <== mon.0 v2:172.21.15.37:3300/0 4 ==== mgrmap(e 5) ==== 96782+0+0 (secure 0 0 0) 0x7f4f2401e460 con 0x7f4f30073b80 2024-04-25T10:30:28.357 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:28.353+0000 7f4f17fff640 1 --2- 172.21.15.37:0/3705536662 >> [v2:172.21.15.203:6832/2418205237,v1:172.21.15.203:6833/2418205237] conn(0x7f4f1c0744e0 0x7f4f1c0769f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:30:28.357 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:28.353+0000 7f4f17fff640 1 -- 172.21.15.37:0/3705536662 <== mon.0 v2:172.21.15.37:3300/0 5 ==== osd_map(42..42 src has 1..42) ==== 6170+0+0 (secure 0 0 0) 0x7f4f240959f0 con 0x7f4f30073b80 2024-04-25T10:30:28.357 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:28.354+0000 7f4f2f7fe640 1 --2- 172.21.15.37:0/3705536662 >> [v2:172.21.15.203:6832/2418205237,v1:172.21.15.203:6833/2418205237] conn(0x7f4f1c0744e0 0x7f4f1c0769f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:30:28.357 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:28.354+0000 7f4f2f7fe640 1 --2- 172.21.15.37:0/3705536662 >> [v2:172.21.15.203:6832/2418205237,v1:172.21.15.203:6833/2418205237] conn(0x7f4f1c0744e0 0x7f4f1c0769f0 secure :-1 s=READY pgs=247 cs=0 l=1 rev1=1 crypto rx=0x7f4f20034000 tx=0x7f4f2002e000 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:30:28.360 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:28.358+0000 7f4f17fff640 1 -- 172.21.15.37:0/3705536662 <== mon.0 v2:172.21.15.37:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187246 (secure 0 0 0) 0x7f4f30145630 con 0x7f4f30073b80 2024-04-25T10:30:28.589 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:28.587+0000 7f4f35cd1640 1 -- 172.21.15.37:0/3705536662 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_command({"prefix": "fs new", "fs_name": "recovery_fs", "metadata": "recovery_fs_metadata", "data": "cephfs_data", "recover": true, "allow_dangerous_metadata_overlay": true} v 0) -- 0x7f4f3013fd60 con 0x7f4f30073b80 2024-04-25T10:30:28.876 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:28.874+0000 7f4f17fff640 1 -- 172.21.15.37:0/3705536662 <== mon.0 v2:172.21.15.37:3300/0 7 ==== mon_command_ack([{"prefix": "fs new", "fs_name": "recovery_fs", "metadata": "recovery_fs_metadata", "data": "cephfs_data", "recover": true, "allow_dangerous_metadata_overlay": true}]=0 new fs with metadata pool 6 and data pool 5 v20) ==== 241+0+0 (secure 0 0 0) 0x7f4f301e8180 con 0x7f4f30073b80 2024-04-25T10:30:28.876 INFO:teuthology.orchestra.run.smithi037.stderr:new fs with metadata pool 6 and data pool 5 2024-04-25T10:30:28.886 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:28.883+0000 7f4f35cd1640 1 -- 172.21.15.37:0/3705536662 >> [v2:172.21.15.203:6832/2418205237,v1:172.21.15.203:6833/2418205237] conn(0x7f4f1c0744e0 msgr2=0x7f4f1c0769f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:30:28.886 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:28.883+0000 7f4f35cd1640 1 --2- 172.21.15.37:0/3705536662 >> [v2:172.21.15.203:6832/2418205237,v1:172.21.15.203:6833/2418205237] conn(0x7f4f1c0744e0 0x7f4f1c0769f0 secure :-1 s=READY pgs=247 cs=0 l=1 rev1=1 crypto rx=0x7f4f20034000 tx=0x7f4f2002e000 comp rx=0 tx=0).stop 2024-04-25T10:30:28.886 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:28.883+0000 7f4f35cd1640 1 -- 172.21.15.37:0/3705536662 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f4f30073b80 msgr2=0x7f4f3015c530 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:30:28.886 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:28.883+0000 7f4f35cd1640 1 --2- 172.21.15.37:0/3705536662 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f4f30073b80 0x7f4f3015c530 secure :-1 s=READY pgs=513 cs=0 l=1 rev1=1 crypto rx=0x7f4f2400d830 tx=0x7f4f2400dcf0 comp rx=0 tx=0).stop 2024-04-25T10:30:28.886 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:28.883+0000 7f4f35cd1640 1 -- 172.21.15.37:0/3705536662 shutdown_connections 2024-04-25T10:30:28.887 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:28.883+0000 7f4f35cd1640 1 --2- 172.21.15.37:0/3705536662 >> [v2:172.21.15.203:6832/2418205237,v1:172.21.15.203:6833/2418205237] conn(0x7f4f1c0744e0 0x7f4f1c0769f0 secure :-1 s=CLOSED pgs=247 cs=0 l=1 rev1=1 crypto rx=0x7f4f20034000 tx=0x7f4f2002e000 comp rx=0 tx=0).stop 2024-04-25T10:30:28.887 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:28.883+0000 7f4f35cd1640 1 --2- 172.21.15.37:0/3705536662 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f4f301e7830 0x7f4f301e7c40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:30:28.887 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:28.883+0000 7f4f35cd1640 1 --2- 172.21.15.37:0/3705536662 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f4f30073b80 0x7f4f3015c530 secure :-1 s=CLOSED pgs=513 cs=0 l=1 rev1=1 crypto rx=0x7f4f2400d830 tx=0x7f4f2400dcf0 comp rx=0 tx=0).stop 2024-04-25T10:30:28.887 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:28.883+0000 7f4f35cd1640 1 --2- 172.21.15.37:0/3705536662 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f4f30070820 0x7f4f3013f580 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:30:28.887 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:28.883+0000 7f4f35cd1640 1 -- 172.21.15.37:0/3705536662 >> 172.21.15.37:0/3705536662 conn(0x7f4f300b35c0 msgr2=0x7f4f3015fdd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:30:28.887 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:28.884+0000 7f4f35cd1640 1 -- 172.21.15.37:0/3705536662 shutdown_connections 2024-04-25T10:30:28.887 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:28.884+0000 7f4f35cd1640 1 -- 172.21.15.37:0/3705536662 wait complete. 2024-04-25T10:30:28.898 DEBUG:teuthology.orchestra.run.smithi037:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-04-25T10:30:29.010 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:29.008+0000 7fc03f146640 1 Processor -- start 2024-04-25T10:30:29.010 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:29.008+0000 7fc03f146640 1 -- start start 2024-04-25T10:30:29.015 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:29.009+0000 7fc03f146640 1 --2- >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7fc03815f760 0x7fc03815e570 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:30:29.015 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:29.009+0000 7fc03f146640 1 --2- >> v2:172.21.15.37:3300/0 conn(0x7fc03806c540 0x7fc03806c950 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:30:29.015 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:29.009+0000 7fc03f146640 1 -- --> v1:172.21.15.203:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fc0380d0e90 con 0x7fc038144490 2024-04-25T10:30:29.015 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:29.009+0000 7fc03f146640 1 -- --> v2:172.21.15.37:3300/0 -- mon_getmap magic: 0 -- 0x7fc03806ce90 con 0x7fc03806c540 2024-04-25T10:30:29.015 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:29.009+0000 7fc03f146640 1 -- --> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] -- mon_getmap magic: 0 -- 0x7fc03815eb80 con 0x7fc03815f760 2024-04-25T10:30:29.015 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:29.009+0000 7fc03d6e3640 1 --2- >> v2:172.21.15.37:3300/0 conn(0x7fc03806c540 0x7fc03806c950 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:30:29.016 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:29.009+0000 7fc03d6e3640 1 --2- >> v2:172.21.15.37:3300/0 conn(0x7fc03806c540 0x7fc03806c950 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.37:3300/0 says I am v2:172.21.15.37:59944/0 (socket says 172.21.15.37:59944) 2024-04-25T10:30:29.016 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:29.009+0000 7fc03d6e3640 1 -- 172.21.15.37:0/364025286 learned_addr learned my addr 172.21.15.37:0/364025286 (peer_addr_for_me v2:172.21.15.37:0/0) 2024-04-25T10:30:29.016 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:29.009+0000 7fc037fff640 1 --2- 172.21.15.37:0/364025286 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7fc03815f760 0x7fc03815e570 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:30:29.016 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:29.009+0000 7fc03d6e3640 1 -- 172.21.15.37:0/364025286 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7fc03815f760 msgr2=0x7fc03815e570 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:30:29.016 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:29.009+0000 7fc03d6e3640 1 --2- 172.21.15.37:0/364025286 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7fc03815f760 0x7fc03815e570 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:30:29.016 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:29.010+0000 7fc03d6e3640 1 -- 172.21.15.37:0/364025286 >> v1:172.21.15.203:6789/0 conn(0x7fc038144490 legacy=0x7fc0380d0780 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:30:29.016 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:29.010+0000 7fc03d6e3640 1 -- 172.21.15.37:0/364025286 --> v2:172.21.15.37:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fc03815ee60 con 0x7fc03806c540 2024-04-25T10:30:29.017 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:29.010+0000 7fc03d6e3640 1 --2- 172.21.15.37:0/364025286 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fc03806c540 0x7fc03806c950 secure :-1 s=READY pgs=514 cs=0 l=1 rev1=1 crypto rx=0x7fc030009e50 tx=0x7fc03002a710 comp rx=0 tx=0).ready entity=mon.0 client_cookie=11f117fd0ef7ba30 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:30:29.017 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:29.010+0000 7fc0377fe640 1 -- 172.21.15.37:0/364025286 <== mon.0 v2:172.21.15.37:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc030036070 con 0x7fc03806c540 2024-04-25T10:30:29.017 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:29.010+0000 7fc0377fe640 1 -- 172.21.15.37:0/364025286 <== mon.0 v2:172.21.15.37:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fc030031600 con 0x7fc03806c540 2024-04-25T10:30:29.017 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:29.010+0000 7fc0377fe640 1 -- 172.21.15.37:0/364025286 <== mon.0 v2:172.21.15.37:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc0300319d0 con 0x7fc03806c540 2024-04-25T10:30:29.017 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:29.011+0000 7fc03f146640 1 -- 172.21.15.37:0/364025286 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fc03806c540 msgr2=0x7fc03806c950 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:30:29.017 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:29.011+0000 7fc03f146640 1 --2- 172.21.15.37:0/364025286 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fc03806c540 0x7fc03806c950 secure :-1 s=READY pgs=514 cs=0 l=1 rev1=1 crypto rx=0x7fc030009e50 tx=0x7fc03002a710 comp rx=0 tx=0).stop 2024-04-25T10:30:29.018 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:29.011+0000 7fc03f146640 1 -- 172.21.15.37:0/364025286 shutdown_connections 2024-04-25T10:30:29.018 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:29.011+0000 7fc03f146640 1 --2- 172.21.15.37:0/364025286 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fc03806c540 0x7fc03806c950 unknown :-1 s=CLOSED pgs=514 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:30:29.018 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:29.011+0000 7fc03f146640 1 --2- 172.21.15.37:0/364025286 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7fc03815f760 0x7fc03815e570 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:30:29.018 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:29.011+0000 7fc03f146640 1 -- 172.21.15.37:0/364025286 >> 172.21.15.37:0/364025286 conn(0x7fc0380b4560 msgr2=0x7fc0380b4970 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:30:29.018 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:29.011+0000 7fc03f146640 1 -- 172.21.15.37:0/364025286 shutdown_connections 2024-04-25T10:30:29.018 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:29.011+0000 7fc03f146640 1 -- 172.21.15.37:0/364025286 wait complete. 2024-04-25T10:30:29.022 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:29.013+0000 7fc03f146640 1 Processor -- start 2024-04-25T10:30:29.023 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:29.013+0000 7fc03f146640 1 -- start start 2024-04-25T10:30:29.023 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:29.014+0000 7fc03f146640 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fc03806c540 0x7fc0381e7240 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:30:29.023 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:29.014+0000 7fc03f146640 1 --2- >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7fc038144490 0x7fc0380c7a00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:30:29.023 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:29.014+0000 7fc03f146640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fc03815f760 0x7fc038134f70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:30:29.023 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:29.014+0000 7fc03f146640 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 -- 0x7fc0380d22f0 con 0x7fc03815f760 2024-04-25T10:30:29.023 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:29.014+0000 7fc03f146640 1 -- --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_getmap magic: 0 -- 0x7fc03815f420 con 0x7fc03806c540 2024-04-25T10:30:29.023 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:29.014+0000 7fc03f146640 1 -- --> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] -- mon_getmap magic: 0 -- 0x7fc0380d2470 con 0x7fc038144490 2024-04-25T10:30:29.023 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:29.015+0000 7fc03d6e3640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fc03815f760 0x7fc038134f70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:30:29.023 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:29.015+0000 7fc03d6e3640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fc03815f760 0x7fc038134f70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.37:3300/0 says I am v2:172.21.15.37:59956/0 (socket says 172.21.15.37:59956) 2024-04-25T10:30:29.023 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:29.015+0000 7fc03d6e3640 1 -- 172.21.15.37:0/2059685709 learned_addr learned my addr 172.21.15.37:0/2059685709 (peer_addr_for_me v2:172.21.15.37:0/0) 2024-04-25T10:30:29.023 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:29.015+0000 7fc037fff640 1 --2- 172.21.15.37:0/2059685709 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7fc038144490 0x7fc0380c7a00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:30:29.023 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:29.015+0000 7fc03cee2640 1 --2- 172.21.15.37:0/2059685709 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fc03806c540 0x7fc0381e7240 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:30:29.023 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:29.015+0000 7fc03d6e3640 1 -- 172.21.15.37:0/2059685709 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7fc038144490 msgr2=0x7fc0380c7a00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:30:29.023 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:29.015+0000 7fc03d6e3640 1 --2- 172.21.15.37:0/2059685709 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7fc038144490 0x7fc0380c7a00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:30:29.024 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:29.015+0000 7fc03d6e3640 1 -- 172.21.15.37:0/2059685709 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fc03806c540 msgr2=0x7fc0381e7240 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:30:29.024 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:29.015+0000 7fc03d6e3640 1 --2- 172.21.15.37:0/2059685709 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fc03806c540 0x7fc0381e7240 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:30:29.024 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:29.015+0000 7fc03d6e3640 1 -- 172.21.15.37:0/2059685709 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fc0381f4220 con 0x7fc03815f760 2024-04-25T10:30:29.024 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:29.015+0000 7fc03d6e3640 1 --2- 172.21.15.37:0/2059685709 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fc03815f760 0x7fc038134f70 secure :-1 s=READY pgs=515 cs=0 l=1 rev1=1 crypto rx=0x7fc030009e50 tx=0x7fc030004a40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:30:29.024 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:29.015+0000 7fc0357fa640 1 -- 172.21.15.37:0/2059685709 <== mon.0 v2:172.21.15.37:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc030036050 con 0x7fc03815f760 2024-04-25T10:30:29.024 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:29.015+0000 7fc0357fa640 1 -- 172.21.15.37:0/2059685709 <== mon.0 v2:172.21.15.37:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fc030042070 con 0x7fc03815f760 2024-04-25T10:30:29.024 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:29.015+0000 7fc03f146640 1 -- 172.21.15.37:0/2059685709 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fc0381e7a00 con 0x7fc03815f760 2024-04-25T10:30:29.024 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:29.015+0000 7fc0357fa640 1 -- 172.21.15.37:0/2059685709 <== mon.0 v2:172.21.15.37:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc030009510 con 0x7fc03815f760 2024-04-25T10:30:29.024 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:29.015+0000 7fc03f146640 1 -- 172.21.15.37:0/2059685709 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fc0381e7fc0 con 0x7fc03815f760 2024-04-25T10:30:29.024 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:29.017+0000 7fc03f146640 1 -- 172.21.15.37:0/2059685709 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fc0380d31a0 con 0x7fc03815f760 2024-04-25T10:30:29.024 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:29.017+0000 7fc0357fa640 1 -- 172.21.15.37:0/2059685709 <== mon.0 v2:172.21.15.37:3300/0 4 ==== mgrmap(e 5) ==== 96782+0+0 (secure 0 0 0) 0x7fc030007bd0 con 0x7fc03815f760 2024-04-25T10:30:29.024 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:29.017+0000 7fc0357fa640 1 --2- 172.21.15.37:0/2059685709 >> [v2:172.21.15.203:6832/2418205237,v1:172.21.15.203:6833/2418205237] conn(0x7fc0240744e0 0x7fc0240769f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:30:29.024 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:29.018+0000 7fc0357fa640 1 -- 172.21.15.37:0/2059685709 <== mon.0 v2:172.21.15.37:3300/0 5 ==== osd_map(43..43 src has 1..43) ==== 6248+0+0 (secure 0 0 0) 0x7fc0300b61f0 con 0x7fc03815f760 2024-04-25T10:30:29.025 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:29.018+0000 7fc03cee2640 1 --2- 172.21.15.37:0/2059685709 >> [v2:172.21.15.203:6832/2418205237,v1:172.21.15.203:6833/2418205237] conn(0x7fc0240744e0 0x7fc0240769f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:30:29.025 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:29.018+0000 7fc03cee2640 1 --2- 172.21.15.37:0/2059685709 >> [v2:172.21.15.203:6832/2418205237,v1:172.21.15.203:6833/2418205237] conn(0x7fc0240744e0 0x7fc0240769f0 secure :-1 s=READY pgs=248 cs=0 l=1 rev1=1 crypto rx=0x7fc0280016e0 tx=0x7fc028009040 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:30:29.025 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:29.022+0000 7fc0357fa640 1 -- 172.21.15.37:0/2059685709 <== mon.0 v2:172.21.15.37:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187246 (secure 0 0 0) 0x7fc0380d31a0 con 0x7fc03815f760 2024-04-25T10:30:29.213 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:29.210+0000 7fc03f146640 1 -- 172.21.15.37:0/2059685709 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7fc0381e7790 con 0x7fc03815f760 2024-04-25T10:30:29.214 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:29.212+0000 7fc0357fa640 1 -- 172.21.15.37:0/2059685709 <== mon.0 v2:172.21.15.37:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v43) ==== 74+0+18639 (secure 0 0 0) 0x7fc0381f4220 con 0x7fc03815f760 2024-04-25T10:30:29.214 INFO:teuthology.orchestra.run.smithi037.stdout: 2024-04-25T10:30:29.214 INFO:teuthology.orchestra.run.smithi037.stdout:{"epoch":43,"fsid":"f4acda28-be34-492a-b286-994afd36f61d","created":"2024-04-25T10:28:43.985718+0000","modified":"2024-04-25T10:30:28.588660+0000","last_up_change":"2024-04-25T10:28:54.900654+0000","last_in_change":"2024-04-25T10:28:46.216973+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":6,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-04-25T10:28:56.202153+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"19","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":4,"pool_name":"cephfs_metadata","create_time":"2024-04-25T10:30:10.008132+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"35","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.75,"score_stable":1.75,"optimal_score":1,"raw_score_acting":1.75,"raw_score_stable":1.75,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":5,"pool_name":"cephfs_data","create_time":"2024-04-25T10:30:11.084827+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"43","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"recovery_fs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":6,"pool_name":"recovery_fs_metadata","create_time":"2024-04-25T10:30:26.903737+0000","flags":32769,"flags_names":"hashpspool,creating","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"43","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"recovery_fs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"a2f79028-b332-4166-9ebb-a7024a37ea22","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":42,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.37:6816","nonce":2746074720},{"type":"v1","addr":"172.21.15.37:6817","nonce":2746074720}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.37:6818","nonce":2746074720},{"type":"v1","addr":"172.21.15.37:6819","nonce":2746074720}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.37:6822","nonce":2746074720},{"type":"v1","addr":"172.21.15.37:6823","nonce":2746074720}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.37:6820","nonce":2746074720},{"type":"v1","addr":"172.21.15.37:6821","nonce":2746074720}]},"public_addr":"172.21.15.37:6817/2746074720","cluster_addr":"172.21.15.37:6819/2746074720","heartbeat_back_addr":"172.21.15.37:6823/2746074720","heartbeat_front_addr":"172.21.15.37:6821/2746074720","state":["exists","up"]},{"osd":1,"uuid":"973159de-c2ea-4083-aecc-69373d1c1132","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":42,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.37:6800","nonce":1690341662},{"type":"v1","addr":"172.21.15.37:6801","nonce":1690341662}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.37:6802","nonce":1690341662},{"type":"v1","addr":"172.21.15.37:6803","nonce":1690341662}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.37:6806","nonce":1690341662},{"type":"v1","addr":"172.21.15.37:6807","nonce":1690341662}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.37:6804","nonce":1690341662},{"type":"v1","addr":"172.21.15.37:6805","nonce":1690341662}]},"public_addr":"172.21.15.37:6801/1690341662","cluster_addr":"172.21.15.37:6803/1690341662","heartbeat_back_addr":"172.21.15.37:6807/1690341662","heartbeat_front_addr":"172.21.15.37:6805/1690341662","state":["exists","up"]},{"osd":2,"uuid":"4ae7a0e9-cb3f-484c-a75f-f3766b721c47","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":42,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.37:6808","nonce":2754669649},{"type":"v1","addr":"172.21.15.37:6809","nonce":2754669649}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.37:6810","nonce":2754669649},{"type":"v1","addr":"172.21.15.37:6811","nonce":2754669649}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.37:6814","nonce":2754669649},{"type":"v1","addr":"172.21.15.37:6815","nonce":2754669649}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.37:6812","nonce":2754669649},{"type":"v1","addr":"172.21.15.37:6813","nonce":2754669649}]},"public_addr":"172.21.15.37:6809/2754669649","cluster_addr":"172.21.15.37:6811/2754669649","heartbeat_back_addr":"172.21.15.37:6815/2754669649","heartbeat_front_addr":"172.21.15.37:6813/2754669649","state":["exists","up"]},{"osd":3,"uuid":"1ba4a746-1608-47a7-ace2-960a5cfae934","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":42,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.37:6824","nonce":1815290618},{"type":"v1","addr":"172.21.15.37:6825","nonce":1815290618}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.37:6826","nonce":1815290618},{"type":"v1","addr":"172.21.15.37:6827","nonce":1815290618}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.37:6830","nonce":1815290618},{"type":"v1","addr":"172.21.15.37:6831","nonce":1815290618}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.37:6828","nonce":1815290618},{"type":"v1","addr":"172.21.15.37:6829","nonce":1815290618}]},"public_addr":"172.21.15.37:6825/1815290618","cluster_addr":"172.21.15.37:6827/1815290618","heartbeat_back_addr":"172.21.15.37:6831/1815290618","heartbeat_front_addr":"172.21.15.37:6829/1815290618","state":["exists","up"]},{"osd":4,"uuid":"d4f59f5d-0413-4b79-87ee-bb212c1fcf62","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":42,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6800","nonce":4051532063},{"type":"v1","addr":"172.21.15.203:6801","nonce":4051532063}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6802","nonce":4051532063},{"type":"v1","addr":"172.21.15.203:6803","nonce":4051532063}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6806","nonce":4051532063},{"type":"v1","addr":"172.21.15.203:6807","nonce":4051532063}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6804","nonce":4051532063},{"type":"v1","addr":"172.21.15.203:6805","nonce":4051532063}]},"public_addr":"172.21.15.203:6801/4051532063","cluster_addr":"172.21.15.203:6803/4051532063","heartbeat_back_addr":"172.21.15.203:6807/4051532063","heartbeat_front_addr":"172.21.15.203:6805/4051532063","state":["exists","up"]},{"osd":5,"uuid":"69710a0e-41d8-4e7f-9a10-17ce540b67be","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":42,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6824","nonce":1203822890},{"type":"v1","addr":"172.21.15.203:6825","nonce":1203822890}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6826","nonce":1203822890},{"type":"v1","addr":"172.21.15.203:6827","nonce":1203822890}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6830","nonce":1203822890},{"type":"v1","addr":"172.21.15.203:6831","nonce":1203822890}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6828","nonce":1203822890},{"type":"v1","addr":"172.21.15.203:6829","nonce":1203822890}]},"public_addr":"172.21.15.203:6825/1203822890","cluster_addr":"172.21.15.203:6827/1203822890","heartbeat_back_addr":"172.21.15.203:6831/1203822890","heartbeat_front_addr":"172.21.15.203:6829/1203822890","state":["exists","up"]},{"osd":6,"uuid":"f29dd047-3b48-473d-a235-6642a6293f20","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":42,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6810","nonce":2309615014},{"type":"v1","addr":"172.21.15.203:6812","nonce":2309615014}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6814","nonce":2309615014},{"type":"v1","addr":"172.21.15.203:6816","nonce":2309615014}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6822","nonce":2309615014},{"type":"v1","addr":"172.21.15.203:6823","nonce":2309615014}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6818","nonce":2309615014},{"type":"v1","addr":"172.21.15.203:6820","nonce":2309615014}]},"public_addr":"172.21.15.203:6812/2309615014","cluster_addr":"172.21.15.203:6816/2309615014","heartbeat_back_addr":"172.21.15.203:6823/2309615014","heartbeat_front_addr":"172.21.15.203:6820/2309615014","state":["exists","up"]},{"osd":7,"uuid":"6e691644-378b-4fdc-8e86-fb5f56e31f29","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":42,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6808","nonce":3688099014},{"type":"v1","addr":"172.21.15.203:6809","nonce":3688099014}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6811","nonce":3688099014},{"type":"v1","addr":"172.21.15.203:6813","nonce":3688099014}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6819","nonce":3688099014},{"type":"v1","addr":"172.21.15.203:6821","nonce":3688099014}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6815","nonce":3688099014},{"type":"v1","addr":"172.21.15.203:6817","nonce":3688099014}]},"public_addr":"172.21.15.203:6809/3688099014","cluster_addr":"172.21.15.203:6813/3688099014","heartbeat_back_addr":"172.21.15.203:6821/3688099014","heartbeat_front_addr":"172.21.15.203:6817/3688099014","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T10:28:51.419141+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T10:28:51.366448+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T10:28:51.348276+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T10:28:51.380519+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T10:28:51.105812+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T10:28:51.384496+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T10:28:51.388755+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T10:28:51.366958+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.37:6835/311552410":"2024-04-26T10:30:24.853732+0000","172.21.15.37:6832/311552410":"2024-04-26T10:30:24.853732+0000"},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-04-25T10:30:29.216 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:29.214+0000 7fc03f146640 1 -- 172.21.15.37:0/2059685709 >> [v2:172.21.15.203:6832/2418205237,v1:172.21.15.203:6833/2418205237] conn(0x7fc0240744e0 msgr2=0x7fc0240769f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:30:29.216 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:29.214+0000 7fc03f146640 1 --2- 172.21.15.37:0/2059685709 >> [v2:172.21.15.203:6832/2418205237,v1:172.21.15.203:6833/2418205237] conn(0x7fc0240744e0 0x7fc0240769f0 secure :-1 s=READY pgs=248 cs=0 l=1 rev1=1 crypto rx=0x7fc0280016e0 tx=0x7fc028009040 comp rx=0 tx=0).stop 2024-04-25T10:30:29.216 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:29.214+0000 7fc03f146640 1 -- 172.21.15.37:0/2059685709 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fc03815f760 msgr2=0x7fc038134f70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:30:29.216 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:29.214+0000 7fc03f146640 1 --2- 172.21.15.37:0/2059685709 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fc03815f760 0x7fc038134f70 secure :-1 s=READY pgs=515 cs=0 l=1 rev1=1 crypto rx=0x7fc030009e50 tx=0x7fc030004a40 comp rx=0 tx=0).stop 2024-04-25T10:30:29.216 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:29.214+0000 7fc03f146640 1 -- 172.21.15.37:0/2059685709 shutdown_connections 2024-04-25T10:30:29.216 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:29.214+0000 7fc03f146640 1 --2- 172.21.15.37:0/2059685709 >> [v2:172.21.15.203:6832/2418205237,v1:172.21.15.203:6833/2418205237] conn(0x7fc0240744e0 0x7fc0240769f0 unknown :-1 s=CLOSED pgs=248 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:30:29.217 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:29.214+0000 7fc03f146640 1 --2- 172.21.15.37:0/2059685709 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fc03815f760 0x7fc038134f70 unknown :-1 s=CLOSED pgs=515 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:30:29.217 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:29.214+0000 7fc03f146640 1 --2- 172.21.15.37:0/2059685709 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7fc038144490 0x7fc0380c7a00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:30:29.217 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:29.214+0000 7fc03f146640 1 --2- 172.21.15.37:0/2059685709 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fc03806c540 0x7fc0381e7240 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:30:29.217 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:29.214+0000 7fc03f146640 1 -- 172.21.15.37:0/2059685709 >> 172.21.15.37:0/2059685709 conn(0x7fc0380b4560 msgr2=0x7fc0380d2a90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:30:29.217 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:29.214+0000 7fc03f146640 1 -- 172.21.15.37:0/2059685709 shutdown_connections 2024-04-25T10:30:29.217 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:29.214+0000 7fc03f146640 1 -- 172.21.15.37:0/2059685709 wait complete. 2024-04-25T10:30:29.228 DEBUG:teuthology.orchestra.run.smithi037:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-04-25T10:30:29.339 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:29.337+0000 7f1782434640 1 Processor -- start 2024-04-25T10:30:29.340 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:29.337+0000 7f1782434640 1 -- start start 2024-04-25T10:30:29.341 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:29.339+0000 7f1782434640 1 --2- >> v2:172.21.15.37:3300/0 conn(0x7f177c0b4e80 0x7f177c0d2b20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:30:29.342 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:29.339+0000 7f1782434640 1 --2- >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f177c0d3060 0x7f177c0b9060 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:30:29.342 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:29.339+0000 7f1782434640 1 -- --> v1:172.21.15.37:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f177c0d3470 con 0x7f177c0d5160 2024-04-25T10:30:29.342 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:29.339+0000 7f1782434640 1 -- --> v2:172.21.15.37:3300/0 -- mon_getmap magic: 0 -- 0x7f177c0d1650 con 0x7f177c0b4e80 2024-04-25T10:30:29.342 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:29.339+0000 7f1782434640 1 -- --> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] -- mon_getmap magic: 0 -- 0x7f177c0b5290 con 0x7f177c0d3060 2024-04-25T10:30:29.342 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:29.339+0000 7f177b7fe640 1 --2- >> v2:172.21.15.37:3300/0 conn(0x7f177c0b4e80 0x7f177c0d2b20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:30:29.342 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:29.339+0000 7f177bfff640 1 --1- >> v1:172.21.15.37:6789/0 conn(0x7f177c0d5160 0x7f177c0d0990 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.37:6789/0 says I am v1:172.21.15.37:55222/0 (socket says 172.21.15.37:55222) 2024-04-25T10:30:29.342 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:29.339+0000 7f177bfff640 1 -- 172.21.15.37:0/2073001936 learned_addr learned my addr 172.21.15.37:0/2073001936 (peer_addr_for_me v1:172.21.15.37:0/0) 2024-04-25T10:30:29.342 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:29.339+0000 7f17809d1640 1 --2- 172.21.15.37:0/2073001936 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f177c0d3060 0x7f177c0b9060 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:30:29.342 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:29.339+0000 7f177affd640 1 -- 172.21.15.37:0/2073001936 <== mon.0 v1:172.21.15.37:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 763053902 0 0) 0x7f176c001110 con 0x7f177c0d5160 2024-04-25T10:30:29.342 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:29.339+0000 7f177affd640 1 -- 172.21.15.37:0/2073001936 --> v1:172.21.15.37:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f1764003520 con 0x7f177c0d5160 2024-04-25T10:30:29.342 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:29.339+0000 7f177b7fe640 1 -- 172.21.15.37:0/2073001936 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f177c0d3060 msgr2=0x7f177c0b9060 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:30:29.342 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:29.339+0000 7f177b7fe640 1 --2- 172.21.15.37:0/2073001936 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f177c0d3060 0x7f177c0b9060 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:30:29.342 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:29.339+0000 7f177b7fe640 1 -- 172.21.15.37:0/2073001936 >> v1:172.21.15.37:6789/0 conn(0x7f177c0d5160 legacy=0x7f177c0d0990 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:30:29.343 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:29.339+0000 7f177b7fe640 1 -- 172.21.15.37:0/2073001936 --> v2:172.21.15.37:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f177c0d10a0 con 0x7f177c0b4e80 2024-04-25T10:30:29.344 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:29.340+0000 7f177b7fe640 1 --2- 172.21.15.37:0/2073001936 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f177c0b4e80 0x7f177c0d2b20 secure :-1 s=READY pgs=517 cs=0 l=1 rev1=1 crypto rx=0x7f17700097a0 tx=0x7f177002a100 comp rx=0 tx=0).ready entity=mon.0 client_cookie=259daccf0f028ab8 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:30:29.344 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:29.340+0000 7f177affd640 1 -- 172.21.15.37:0/2073001936 <== mon.0 v2:172.21.15.37:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1770037070 con 0x7f177c0b4e80 2024-04-25T10:30:29.344 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:29.340+0000 7f177affd640 1 -- 172.21.15.37:0/2073001936 <== mon.0 v2:172.21.15.37:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f177002ae50 con 0x7f177c0b4e80 2024-04-25T10:30:29.344 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:29.340+0000 7f177affd640 1 -- 172.21.15.37:0/2073001936 <== mon.0 v2:172.21.15.37:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1770032640 con 0x7f177c0b4e80 2024-04-25T10:30:29.344 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:29.341+0000 7f1782434640 1 -- 172.21.15.37:0/2073001936 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f177c0b4e80 msgr2=0x7f177c0d2b20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:30:29.344 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:29.341+0000 7f1782434640 1 --2- 172.21.15.37:0/2073001936 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f177c0b4e80 0x7f177c0d2b20 secure :-1 s=READY pgs=517 cs=0 l=1 rev1=1 crypto rx=0x7f17700097a0 tx=0x7f177002a100 comp rx=0 tx=0).stop 2024-04-25T10:30:29.344 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:29.341+0000 7f1782434640 1 -- 172.21.15.37:0/2073001936 shutdown_connections 2024-04-25T10:30:29.344 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:29.341+0000 7f1782434640 1 --2- 172.21.15.37:0/2073001936 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f177c0d3060 0x7f177c0b9060 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:30:29.344 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:29.341+0000 7f1782434640 1 --2- 172.21.15.37:0/2073001936 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f177c0b4e80 0x7f177c0d2b20 unknown :-1 s=CLOSED pgs=517 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:30:29.344 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:29.341+0000 7f1782434640 1 -- 172.21.15.37:0/2073001936 >> 172.21.15.37:0/2073001936 conn(0x7f177c0b7a80 msgr2=0x7f177c0b7e90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:30:29.344 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:29.341+0000 7f1782434640 1 -- 172.21.15.37:0/2073001936 shutdown_connections 2024-04-25T10:30:29.345 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:29.341+0000 7f1782434640 1 -- 172.21.15.37:0/2073001936 wait complete. 2024-04-25T10:30:29.345 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:29.343+0000 7f1782434640 1 Processor -- start 2024-04-25T10:30:29.345 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:29.343+0000 7f1782434640 1 -- start start 2024-04-25T10:30:29.348 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:29.344+0000 7f1782434640 1 --2- >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f177c0b4e80 0x7f177c1f6340 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:30:29.348 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:29.344+0000 7f1782434640 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f177c0d3060 0x7f177c163420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:30:29.348 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:29.344+0000 7f1782434640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f177c0d5160 0x7f177c14cf40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:30:29.348 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:29.344+0000 7f1782434640 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 -- 0x7f177c0d4960 con 0x7f177c0d5160 2024-04-25T10:30:29.348 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:29.344+0000 7f1782434640 1 -- --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_getmap magic: 0 -- 0x7f177c0d4680 con 0x7f177c0d3060 2024-04-25T10:30:29.348 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:29.344+0000 7f1782434640 1 -- --> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] -- mon_getmap magic: 0 -- 0x7f177c0d4c40 con 0x7f177c0b4e80 2024-04-25T10:30:29.348 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:29.344+0000 7f17809d1640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f177c0d5160 0x7f177c14cf40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:30:29.348 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:29.344+0000 7f17809d1640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f177c0d5160 0x7f177c14cf40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.37:3300/0 says I am v2:172.21.15.37:59976/0 (socket says 172.21.15.37:59976) 2024-04-25T10:30:29.348 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:29.344+0000 7f17809d1640 1 -- 172.21.15.37:0/2638346274 learned_addr learned my addr 172.21.15.37:0/2638346274 (peer_addr_for_me v2:172.21.15.37:0/0) 2024-04-25T10:30:29.348 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:29.344+0000 7f177bfff640 1 --2- 172.21.15.37:0/2638346274 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f177c0b4e80 0x7f177c1f6340 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:30:29.348 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:29.344+0000 7f177b7fe640 1 --2- 172.21.15.37:0/2638346274 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f177c0d3060 0x7f177c163420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:30:29.348 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:29.344+0000 7f17809d1640 1 -- 172.21.15.37:0/2638346274 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f177c0b4e80 msgr2=0x7f177c1f6340 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:30:29.348 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:29.344+0000 7f17809d1640 1 --2- 172.21.15.37:0/2638346274 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f177c0b4e80 0x7f177c1f6340 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:30:29.349 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:29.344+0000 7f17809d1640 1 -- 172.21.15.37:0/2638346274 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f177c0d3060 msgr2=0x7f177c163420 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:30:29.349 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:29.344+0000 7f17809d1640 1 --2- 172.21.15.37:0/2638346274 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f177c0d3060 0x7f177c163420 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:30:29.349 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:29.344+0000 7f17809d1640 1 -- 172.21.15.37:0/2638346274 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f177c14d480 con 0x7f177c0d5160 2024-04-25T10:30:29.349 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:29.345+0000 7f177b7fe640 1 --2- 172.21.15.37:0/2638346274 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f177c0d3060 0x7f177c163420 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T10:30:29.349 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:29.345+0000 7f17809d1640 1 --2- 172.21.15.37:0/2638346274 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f177c0d5160 0x7f177c14cf40 secure :-1 s=READY pgs=518 cs=0 l=1 rev1=1 crypto rx=0x7f177400efc0 tx=0x7f177400c490 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:30:29.349 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:29.345+0000 7f1778ff9640 1 -- 172.21.15.37:0/2638346274 <== mon.0 v2:172.21.15.37:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1774018020 con 0x7f177c0d5160 2024-04-25T10:30:29.349 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:29.345+0000 7f1778ff9640 1 -- 172.21.15.37:0/2638346274 <== mon.0 v2:172.21.15.37:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f1774014480 con 0x7f177c0d5160 2024-04-25T10:30:29.349 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:29.345+0000 7f1782434640 1 -- 172.21.15.37:0/2638346274 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f177c0c9bd0 con 0x7f177c0d5160 2024-04-25T10:30:29.349 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:29.345+0000 7f1778ff9640 1 -- 172.21.15.37:0/2638346274 <== mon.0 v2:172.21.15.37:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1774009e60 con 0x7f177c0d5160 2024-04-25T10:30:29.349 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:29.345+0000 7f1782434640 1 -- 172.21.15.37:0/2638346274 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f177c0cc440 con 0x7f177c0d5160 2024-04-25T10:30:29.349 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:29.346+0000 7f1782434640 1 -- 172.21.15.37:0/2638346274 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f177c150730 con 0x7f177c0d5160 2024-04-25T10:30:29.350 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:29.347+0000 7f1778ff9640 1 -- 172.21.15.37:0/2638346274 <== mon.0 v2:172.21.15.37:3300/0 4 ==== mgrmap(e 5) ==== 96782+0+0 (secure 0 0 0) 0x7f1774005d10 con 0x7f177c0d5160 2024-04-25T10:30:29.350 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:29.347+0000 7f1778ff9640 1 --2- 172.21.15.37:0/2638346274 >> [v2:172.21.15.203:6832/2418205237,v1:172.21.15.203:6833/2418205237] conn(0x7f17680744b0 0x7f17680769c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:30:29.350 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:29.347+0000 7f1778ff9640 1 -- 172.21.15.37:0/2638346274 <== mon.0 v2:172.21.15.37:3300/0 5 ==== osd_map(43..43 src has 1..43) ==== 6248+0+0 (secure 0 0 0) 0x7f17740960a0 con 0x7f177c0d5160 2024-04-25T10:30:29.350 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:29.347+0000 7f177bfff640 1 --2- 172.21.15.37:0/2638346274 >> [v2:172.21.15.203:6832/2418205237,v1:172.21.15.203:6833/2418205237] conn(0x7f17680744b0 0x7f17680769c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:30:29.350 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:29.348+0000 7f177bfff640 1 --2- 172.21.15.37:0/2638346274 >> [v2:172.21.15.203:6832/2418205237,v1:172.21.15.203:6833/2418205237] conn(0x7f17680744b0 0x7f17680769c0 secure :-1 s=READY pgs=249 cs=0 l=1 rev1=1 crypto rx=0x7f176c0013d0 tx=0x7f176c007b50 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:30:29.353 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:29.352+0000 7f1778ff9640 1 -- 172.21.15.37:0/2638346274 <== mon.0 v2:172.21.15.37:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187246 (secure 0 0 0) 0x7f1774022030 con 0x7f177c0d5160 2024-04-25T10:30:29.541 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:29.539+0000 7f1782434640 1 -- 172.21.15.37:0/2638346274 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7f177c06a140 con 0x7f177c0d5160 2024-04-25T10:30:29.542 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:29.540+0000 7f1778ff9640 1 -- 172.21.15.37:0/2638346274 <== mon.0 v2:172.21.15.37:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v43) ==== 74+0+18639 (secure 0 0 0) 0x7f1774010040 con 0x7f177c0d5160 2024-04-25T10:30:29.543 INFO:teuthology.orchestra.run.smithi037.stdout: 2024-04-25T10:30:29.543 INFO:teuthology.orchestra.run.smithi037.stdout:{"epoch":43,"fsid":"f4acda28-be34-492a-b286-994afd36f61d","created":"2024-04-25T10:28:43.985718+0000","modified":"2024-04-25T10:30:28.588660+0000","last_up_change":"2024-04-25T10:28:54.900654+0000","last_in_change":"2024-04-25T10:28:46.216973+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":6,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-04-25T10:28:56.202153+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"19","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":4,"pool_name":"cephfs_metadata","create_time":"2024-04-25T10:30:10.008132+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"35","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.75,"score_stable":1.75,"optimal_score":1,"raw_score_acting":1.75,"raw_score_stable":1.75,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":5,"pool_name":"cephfs_data","create_time":"2024-04-25T10:30:11.084827+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"43","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"recovery_fs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":6,"pool_name":"recovery_fs_metadata","create_time":"2024-04-25T10:30:26.903737+0000","flags":32769,"flags_names":"hashpspool,creating","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"43","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"recovery_fs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"a2f79028-b332-4166-9ebb-a7024a37ea22","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":42,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.37:6816","nonce":2746074720},{"type":"v1","addr":"172.21.15.37:6817","nonce":2746074720}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.37:6818","nonce":2746074720},{"type":"v1","addr":"172.21.15.37:6819","nonce":2746074720}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.37:6822","nonce":2746074720},{"type":"v1","addr":"172.21.15.37:6823","nonce":2746074720}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.37:6820","nonce":2746074720},{"type":"v1","addr":"172.21.15.37:6821","nonce":2746074720}]},"public_addr":"172.21.15.37:6817/2746074720","cluster_addr":"172.21.15.37:6819/2746074720","heartbeat_back_addr":"172.21.15.37:6823/2746074720","heartbeat_front_addr":"172.21.15.37:6821/2746074720","state":["exists","up"]},{"osd":1,"uuid":"973159de-c2ea-4083-aecc-69373d1c1132","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":42,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.37:6800","nonce":1690341662},{"type":"v1","addr":"172.21.15.37:6801","nonce":1690341662}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.37:6802","nonce":1690341662},{"type":"v1","addr":"172.21.15.37:6803","nonce":1690341662}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.37:6806","nonce":1690341662},{"type":"v1","addr":"172.21.15.37:6807","nonce":1690341662}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.37:6804","nonce":1690341662},{"type":"v1","addr":"172.21.15.37:6805","nonce":1690341662}]},"public_addr":"172.21.15.37:6801/1690341662","cluster_addr":"172.21.15.37:6803/1690341662","heartbeat_back_addr":"172.21.15.37:6807/1690341662","heartbeat_front_addr":"172.21.15.37:6805/1690341662","state":["exists","up"]},{"osd":2,"uuid":"4ae7a0e9-cb3f-484c-a75f-f3766b721c47","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":42,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.37:6808","nonce":2754669649},{"type":"v1","addr":"172.21.15.37:6809","nonce":2754669649}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.37:6810","nonce":2754669649},{"type":"v1","addr":"172.21.15.37:6811","nonce":2754669649}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.37:6814","nonce":2754669649},{"type":"v1","addr":"172.21.15.37:6815","nonce":2754669649}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.37:6812","nonce":2754669649},{"type":"v1","addr":"172.21.15.37:6813","nonce":2754669649}]},"public_addr":"172.21.15.37:6809/2754669649","cluster_addr":"172.21.15.37:6811/2754669649","heartbeat_back_addr":"172.21.15.37:6815/2754669649","heartbeat_front_addr":"172.21.15.37:6813/2754669649","state":["exists","up"]},{"osd":3,"uuid":"1ba4a746-1608-47a7-ace2-960a5cfae934","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":42,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.37:6824","nonce":1815290618},{"type":"v1","addr":"172.21.15.37:6825","nonce":1815290618}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.37:6826","nonce":1815290618},{"type":"v1","addr":"172.21.15.37:6827","nonce":1815290618}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.37:6830","nonce":1815290618},{"type":"v1","addr":"172.21.15.37:6831","nonce":1815290618}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.37:6828","nonce":1815290618},{"type":"v1","addr":"172.21.15.37:6829","nonce":1815290618}]},"public_addr":"172.21.15.37:6825/1815290618","cluster_addr":"172.21.15.37:6827/1815290618","heartbeat_back_addr":"172.21.15.37:6831/1815290618","heartbeat_front_addr":"172.21.15.37:6829/1815290618","state":["exists","up"]},{"osd":4,"uuid":"d4f59f5d-0413-4b79-87ee-bb212c1fcf62","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":42,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6800","nonce":4051532063},{"type":"v1","addr":"172.21.15.203:6801","nonce":4051532063}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6802","nonce":4051532063},{"type":"v1","addr":"172.21.15.203:6803","nonce":4051532063}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6806","nonce":4051532063},{"type":"v1","addr":"172.21.15.203:6807","nonce":4051532063}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6804","nonce":4051532063},{"type":"v1","addr":"172.21.15.203:6805","nonce":4051532063}]},"public_addr":"172.21.15.203:6801/4051532063","cluster_addr":"172.21.15.203:6803/4051532063","heartbeat_back_addr":"172.21.15.203:6807/4051532063","heartbeat_front_addr":"172.21.15.203:6805/4051532063","state":["exists","up"]},{"osd":5,"uuid":"69710a0e-41d8-4e7f-9a10-17ce540b67be","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":42,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6824","nonce":1203822890},{"type":"v1","addr":"172.21.15.203:6825","nonce":1203822890}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6826","nonce":1203822890},{"type":"v1","addr":"172.21.15.203:6827","nonce":1203822890}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6830","nonce":1203822890},{"type":"v1","addr":"172.21.15.203:6831","nonce":1203822890}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6828","nonce":1203822890},{"type":"v1","addr":"172.21.15.203:6829","nonce":1203822890}]},"public_addr":"172.21.15.203:6825/1203822890","cluster_addr":"172.21.15.203:6827/1203822890","heartbeat_back_addr":"172.21.15.203:6831/1203822890","heartbeat_front_addr":"172.21.15.203:6829/1203822890","state":["exists","up"]},{"osd":6,"uuid":"f29dd047-3b48-473d-a235-6642a6293f20","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":42,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6810","nonce":2309615014},{"type":"v1","addr":"172.21.15.203:6812","nonce":2309615014}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6814","nonce":2309615014},{"type":"v1","addr":"172.21.15.203:6816","nonce":2309615014}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6822","nonce":2309615014},{"type":"v1","addr":"172.21.15.203:6823","nonce":2309615014}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6818","nonce":2309615014},{"type":"v1","addr":"172.21.15.203:6820","nonce":2309615014}]},"public_addr":"172.21.15.203:6812/2309615014","cluster_addr":"172.21.15.203:6816/2309615014","heartbeat_back_addr":"172.21.15.203:6823/2309615014","heartbeat_front_addr":"172.21.15.203:6820/2309615014","state":["exists","up"]},{"osd":7,"uuid":"6e691644-378b-4fdc-8e86-fb5f56e31f29","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":42,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6808","nonce":3688099014},{"type":"v1","addr":"172.21.15.203:6809","nonce":3688099014}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6811","nonce":3688099014},{"type":"v1","addr":"172.21.15.203:6813","nonce":3688099014}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6819","nonce":3688099014},{"type":"v1","addr":"172.21.15.203:6821","nonce":3688099014}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6815","nonce":3688099014},{"type":"v1","addr":"172.21.15.203:6817","nonce":3688099014}]},"public_addr":"172.21.15.203:6809/3688099014","cluster_addr":"172.21.15.203:6813/3688099014","heartbeat_back_addr":"172.21.15.203:6821/3688099014","heartbeat_front_addr":"172.21.15.203:6817/3688099014","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T10:28:51.419141+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T10:28:51.366448+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T10:28:51.348276+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T10:28:51.380519+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T10:28:51.105812+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T10:28:51.384496+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T10:28:51.388755+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T10:28:51.366958+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.37:6835/311552410":"2024-04-26T10:30:24.853732+0000","172.21.15.37:6832/311552410":"2024-04-26T10:30:24.853732+0000"},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-04-25T10:30:29.545 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:29.542+0000 7f1782434640 1 -- 172.21.15.37:0/2638346274 >> [v2:172.21.15.203:6832/2418205237,v1:172.21.15.203:6833/2418205237] conn(0x7f17680744b0 msgr2=0x7f17680769c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:30:29.545 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:29.542+0000 7f1782434640 1 --2- 172.21.15.37:0/2638346274 >> [v2:172.21.15.203:6832/2418205237,v1:172.21.15.203:6833/2418205237] conn(0x7f17680744b0 0x7f17680769c0 secure :-1 s=READY pgs=249 cs=0 l=1 rev1=1 crypto rx=0x7f176c0013d0 tx=0x7f176c007b50 comp rx=0 tx=0).stop 2024-04-25T10:30:29.545 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:29.542+0000 7f1782434640 1 -- 172.21.15.37:0/2638346274 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f177c0d5160 msgr2=0x7f177c14cf40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:30:29.545 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:29.542+0000 7f1782434640 1 --2- 172.21.15.37:0/2638346274 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f177c0d5160 0x7f177c14cf40 secure :-1 s=READY pgs=518 cs=0 l=1 rev1=1 crypto rx=0x7f177400efc0 tx=0x7f177400c490 comp rx=0 tx=0).stop 2024-04-25T10:30:29.545 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:29.542+0000 7f1782434640 1 -- 172.21.15.37:0/2638346274 shutdown_connections 2024-04-25T10:30:29.545 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:29.542+0000 7f1782434640 1 --2- 172.21.15.37:0/2638346274 >> [v2:172.21.15.203:6832/2418205237,v1:172.21.15.203:6833/2418205237] conn(0x7f17680744b0 0x7f17680769c0 unknown :-1 s=CLOSED pgs=249 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:30:29.545 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:29.542+0000 7f1782434640 1 --2- 172.21.15.37:0/2638346274 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f177c0d5160 0x7f177c14cf40 unknown :-1 s=CLOSED pgs=518 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:30:29.545 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:29.542+0000 7f1782434640 1 --2- 172.21.15.37:0/2638346274 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f177c0d3060 0x7f177c163420 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:30:29.545 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:29.542+0000 7f1782434640 1 --2- 172.21.15.37:0/2638346274 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f177c0b4e80 0x7f177c1f6340 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:30:29.545 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:29.543+0000 7f1782434640 1 -- 172.21.15.37:0/2638346274 >> 172.21.15.37:0/2638346274 conn(0x7f177c0b7a80 msgr2=0x7f177c15ed50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:30:29.545 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:29.543+0000 7f1782434640 1 -- 172.21.15.37:0/2638346274 shutdown_connections 2024-04-25T10:30:29.546 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:29.543+0000 7f1782434640 1 -- 172.21.15.37:0/2638346274 wait complete. 2024-04-25T10:30:29.556 DEBUG:teuthology.orchestra.run.smithi037:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs set recovery_fs standby_count_wanted 0 2024-04-25T10:30:29.671 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:29.668+0000 7fea8ef7a640 1 Processor -- start 2024-04-25T10:30:29.671 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:29.669+0000 7fea8ef7a640 1 -- start start 2024-04-25T10:30:29.673 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:29.670+0000 7fea8ef7a640 1 --2- >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7fea880b4bb0 0x7fea881646b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:30:29.673 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:29.670+0000 7fea8ef7a640 1 -- --> v1:172.21.15.37:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fea880d5080 con 0x7fea880d4560 2024-04-25T10:30:29.673 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:29.670+0000 7fea8ef7a640 1 -- --> v1:172.21.15.203:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fea88161190 con 0x7fea880b7600 2024-04-25T10:30:29.673 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:29.670+0000 7fea8ef7a640 1 -- --> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] -- mon_getmap magic: 0 -- 0x7fea88161470 con 0x7fea880b4bb0 2024-04-25T10:30:29.673 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:29.670+0000 7fea8d517640 1 --1- >> v1:172.21.15.37:6789/0 conn(0x7fea880d4560 0x7fea880d4970 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.37:6789/0 says I am v1:172.21.15.37:40880/0 (socket says 172.21.15.37:40880) 2024-04-25T10:30:29.673 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:29.670+0000 7fea8d517640 1 -- 172.21.15.37:0/3041113878 learned_addr learned my addr 172.21.15.37:0/3041113878 (peer_addr_for_me v1:172.21.15.37:0/0) 2024-04-25T10:30:29.673 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:29.670+0000 7fea8cd16640 1 --2- 172.21.15.37:0/3041113878 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7fea880b4bb0 0x7fea881646b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:30:29.674 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:29.670+0000 7fea877fe640 1 -- 172.21.15.37:0/3041113878 <== mon.0 v1:172.21.15.37:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 2365732180 0 0) 0x7fea880d5080 con 0x7fea880d4560 2024-04-25T10:30:29.674 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:29.670+0000 7fea877fe640 1 -- 172.21.15.37:0/3041113878 --> v1:172.21.15.37:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7fea70003520 con 0x7fea880d4560 2024-04-25T10:30:29.674 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:29.671+0000 7fea8cd16640 1 -- 172.21.15.37:0/3041113878 >> v1:172.21.15.203:6789/0 conn(0x7fea880b7600 legacy=0x7fea880b7a10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:30:29.674 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:29.671+0000 7fea877fe640 1 -- 172.21.15.37:0/3041113878 <== mon.1 v1:172.21.15.203:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 3122571196 0 0) 0x7fea88161190 con 0x7fea880b7600 2024-04-25T10:30:29.674 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:29.671+0000 7fea8cd16640 1 -- 172.21.15.37:0/3041113878 >> v1:172.21.15.37:6789/0 conn(0x7fea880d4560 legacy=0x7fea880d4970 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:30:29.674 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:29.671+0000 7fea8cd16640 1 -- 172.21.15.37:0/3041113878 --> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fea88163eb0 con 0x7fea880b4bb0 2024-04-25T10:30:29.675 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:29.671+0000 7fea8cd16640 1 --2- 172.21.15.37:0/3041113878 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7fea880b4bb0 0x7fea881646b0 secure :-1 s=READY pgs=56 cs=0 l=1 rev1=1 crypto rx=0x7fea80009dc0 tx=0x7fea8002a640 comp rx=0 tx=0).ready entity=mon.2 client_cookie=599e1fad51ec532d server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:30:29.676 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:29.672+0000 7fea877fe640 1 -- 172.21.15.37:0/3041113878 <== mon.2 v2:172.21.15.203:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fea80035020 con 0x7fea880b4bb0 2024-04-25T10:30:29.676 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:29.672+0000 7fea877fe640 1 -- 172.21.15.37:0/3041113878 <== mon.2 v2:172.21.15.203:3301/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fea80007540 con 0x7fea880b4bb0 2024-04-25T10:30:29.676 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:29.672+0000 7fea877fe640 1 -- 172.21.15.37:0/3041113878 <== mon.2 v2:172.21.15.203:3301/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fea78001670 con 0x7fea880b4bb0 2024-04-25T10:30:29.676 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:29.672+0000 7fea8ef7a640 1 -- 172.21.15.37:0/3041113878 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7fea880b4bb0 msgr2=0x7fea881646b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:30:29.676 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:29.672+0000 7fea8ef7a640 1 --2- 172.21.15.37:0/3041113878 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7fea880b4bb0 0x7fea881646b0 secure :-1 s=READY pgs=56 cs=0 l=1 rev1=1 crypto rx=0x7fea80009dc0 tx=0x7fea8002a640 comp rx=0 tx=0).stop 2024-04-25T10:30:29.676 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:29.672+0000 7fea8ef7a640 1 -- 172.21.15.37:0/3041113878 shutdown_connections 2024-04-25T10:30:29.676 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:29.672+0000 7fea8ef7a640 1 --2- 172.21.15.37:0/3041113878 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7fea880b4bb0 0x7fea881646b0 unknown :-1 s=CLOSED pgs=56 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:30:29.676 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:29.672+0000 7fea8ef7a640 1 -- 172.21.15.37:0/3041113878 >> 172.21.15.37:0/3041113878 conn(0x7fea880d27e0 msgr2=0x7fea880d2bf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:30:29.676 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:29.672+0000 7fea8ef7a640 1 -- 172.21.15.37:0/3041113878 shutdown_connections 2024-04-25T10:30:29.676 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:29.672+0000 7fea8ef7a640 1 -- 172.21.15.37:0/3041113878 wait complete. 2024-04-25T10:30:29.677 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:29.674+0000 7fea8ef7a640 1 Processor -- start 2024-04-25T10:30:29.677 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:29.674+0000 7fea8ef7a640 1 -- start start 2024-04-25T10:30:29.679 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:29.676+0000 7fea8ef7a640 1 --2- >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7fea880b4bb0 0x7fea881525a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:30:29.679 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:29.676+0000 7fea8ef7a640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fea880b7600 0x7fea880ce920 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:30:29.679 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:29.676+0000 7fea8ef7a640 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fea880d4560 0x7fea880cb530 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:30:29.679 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:29.676+0000 7fea8ef7a640 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 -- 0x7fea88164bf0 con 0x7fea880b7600 2024-04-25T10:30:29.679 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:29.676+0000 7fea8ef7a640 1 -- --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_getmap magic: 0 -- 0x7fea880b4fc0 con 0x7fea880d4560 2024-04-25T10:30:29.679 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:29.676+0000 7fea8ef7a640 1 -- --> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] -- mon_getmap magic: 0 -- 0x7fea880d09e0 con 0x7fea880b4bb0 2024-04-25T10:30:29.679 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:29.676+0000 7fea87fff640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fea880b7600 0x7fea880ce920 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:30:29.679 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:29.676+0000 7fea87fff640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fea880b7600 0x7fea880ce920 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.37:3300/0 says I am v2:172.21.15.37:55340/0 (socket says 172.21.15.37:55340) 2024-04-25T10:30:29.680 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:29.676+0000 7fea87fff640 1 -- 172.21.15.37:0/657813815 learned_addr learned my addr 172.21.15.37:0/657813815 (peer_addr_for_me v2:172.21.15.37:0/0) 2024-04-25T10:30:29.680 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:29.676+0000 7fea8cd16640 1 --2- 172.21.15.37:0/657813815 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7fea880b4bb0 0x7fea881525a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:30:29.680 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:29.676+0000 7fea8d517640 1 --2- 172.21.15.37:0/657813815 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fea880d4560 0x7fea880cb530 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:30:29.680 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:29.676+0000 7fea87fff640 1 -- 172.21.15.37:0/657813815 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7fea880b4bb0 msgr2=0x7fea881525a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:30:29.680 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:29.676+0000 7fea87fff640 1 --2- 172.21.15.37:0/657813815 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7fea880b4bb0 0x7fea881525a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:30:29.680 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:29.676+0000 7fea87fff640 1 -- 172.21.15.37:0/657813815 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fea880d4560 msgr2=0x7fea880cb530 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:30:29.680 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:29.676+0000 7fea87fff640 1 --2- 172.21.15.37:0/657813815 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fea880d4560 0x7fea880cb530 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:30:29.680 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:29.676+0000 7fea87fff640 1 -- 172.21.15.37:0/657813815 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fea880ac610 con 0x7fea880b7600 2024-04-25T10:30:29.680 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:29.676+0000 7fea8cd16640 1 --2- 172.21.15.37:0/657813815 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7fea880b4bb0 0x7fea881525a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T10:30:29.680 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:29.676+0000 7fea8d517640 1 --2- 172.21.15.37:0/657813815 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fea880d4560 0x7fea880cb530 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T10:30:29.680 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:29.676+0000 7fea87fff640 1 --2- 172.21.15.37:0/657813815 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fea880b7600 0x7fea880ce920 secure :-1 s=READY pgs=520 cs=0 l=1 rev1=1 crypto rx=0x7fea7c00c410 tx=0x7fea7c00c8d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:30:29.680 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:29.676+0000 7fea857fa640 1 -- 172.21.15.37:0/657813815 <== mon.0 v2:172.21.15.37:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fea7c017070 con 0x7fea880b7600 2024-04-25T10:30:29.681 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:29.676+0000 7fea857fa640 1 -- 172.21.15.37:0/657813815 <== mon.0 v2:172.21.15.37:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fea7c0035f0 con 0x7fea880b7600 2024-04-25T10:30:29.681 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:29.676+0000 7fea8ef7a640 1 -- 172.21.15.37:0/657813815 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fea880cf650 con 0x7fea880b7600 2024-04-25T10:30:29.681 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:29.677+0000 7fea857fa640 1 -- 172.21.15.37:0/657813815 <== mon.0 v2:172.21.15.37:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fea7c0039e0 con 0x7fea880b7600 2024-04-25T10:30:29.681 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:29.677+0000 7fea8ef7a640 1 -- 172.21.15.37:0/657813815 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fea880cfc10 con 0x7fea880b7600 2024-04-25T10:30:29.682 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:29.677+0000 7fea8ef7a640 1 -- 172.21.15.37:0/657813815 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fea8806a140 con 0x7fea880b7600 2024-04-25T10:30:29.682 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:29.678+0000 7fea857fa640 1 -- 172.21.15.37:0/657813815 <== mon.0 v2:172.21.15.37:3300/0 4 ==== mgrmap(e 5) ==== 96782+0+0 (secure 0 0 0) 0x7fea7c01a070 con 0x7fea880b7600 2024-04-25T10:30:29.682 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:29.679+0000 7fea857fa640 1 --2- 172.21.15.37:0/657813815 >> [v2:172.21.15.203:6832/2418205237,v1:172.21.15.203:6833/2418205237] conn(0x7fea740744b0 0x7fea740769c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:30:29.682 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:29.679+0000 7fea857fa640 1 -- 172.21.15.37:0/657813815 <== mon.0 v2:172.21.15.37:3300/0 5 ==== osd_map(43..43 src has 1..43) ==== 6248+0+0 (secure 0 0 0) 0x7fea7c094870 con 0x7fea880b7600 2024-04-25T10:30:29.682 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:29.679+0000 7fea8cd16640 1 --2- 172.21.15.37:0/657813815 >> [v2:172.21.15.203:6832/2418205237,v1:172.21.15.203:6833/2418205237] conn(0x7fea740744b0 0x7fea740769c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:30:29.682 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:29.679+0000 7fea8cd16640 1 --2- 172.21.15.37:0/657813815 >> [v2:172.21.15.203:6832/2418205237,v1:172.21.15.203:6833/2418205237] conn(0x7fea740744b0 0x7fea740769c0 secure :-1 s=READY pgs=250 cs=0 l=1 rev1=1 crypto rx=0x7fea80033000 tx=0x7fea8002f000 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:30:29.686 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:29.683+0000 7fea857fa640 1 -- 172.21.15.37:0/657813815 <== mon.0 v2:172.21.15.37:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187246 (secure 0 0 0) 0x7fea7c05ee70 con 0x7fea880b7600 2024-04-25T10:30:29.914 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:29.911+0000 7fea8ef7a640 1 -- 172.21.15.37:0/657813815 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_command({"prefix": "fs set", "fs_name": "recovery_fs", "var": "standby_count_wanted", "val": "0"} v 0) -- 0x7fea880d3ff0 con 0x7fea880b7600 2024-04-25T10:30:30.886 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:30.884+0000 7fea857fa640 1 -- 172.21.15.37:0/657813815 <== mon.0 v2:172.21.15.37:3300/0 7 ==== mon_command_ack([{"prefix": "fs set", "fs_name": "recovery_fs", "var": "standby_count_wanted", "val": "0"}]=0 v21) ==== 123+0+0 (secure 0 0 0) 0x7fea7c05e770 con 0x7fea880b7600 2024-04-25T10:30:30.889 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:30.887+0000 7fea8ef7a640 1 -- 172.21.15.37:0/657813815 >> [v2:172.21.15.203:6832/2418205237,v1:172.21.15.203:6833/2418205237] conn(0x7fea740744b0 msgr2=0x7fea740769c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:30:30.889 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:30.887+0000 7fea8ef7a640 1 --2- 172.21.15.37:0/657813815 >> [v2:172.21.15.203:6832/2418205237,v1:172.21.15.203:6833/2418205237] conn(0x7fea740744b0 0x7fea740769c0 secure :-1 s=READY pgs=250 cs=0 l=1 rev1=1 crypto rx=0x7fea80033000 tx=0x7fea8002f000 comp rx=0 tx=0).stop 2024-04-25T10:30:30.889 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:30.887+0000 7fea8ef7a640 1 -- 172.21.15.37:0/657813815 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fea880b7600 msgr2=0x7fea880ce920 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:30:30.889 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:30.887+0000 7fea8ef7a640 1 --2- 172.21.15.37:0/657813815 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fea880b7600 0x7fea880ce920 secure :-1 s=READY pgs=520 cs=0 l=1 rev1=1 crypto rx=0x7fea7c00c410 tx=0x7fea7c00c8d0 comp rx=0 tx=0).stop 2024-04-25T10:30:30.889 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:30.887+0000 7fea8ef7a640 1 -- 172.21.15.37:0/657813815 shutdown_connections 2024-04-25T10:30:30.889 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:30.887+0000 7fea8ef7a640 1 --2- 172.21.15.37:0/657813815 >> [v2:172.21.15.203:6832/2418205237,v1:172.21.15.203:6833/2418205237] conn(0x7fea740744b0 0x7fea740769c0 unknown :-1 s=CLOSED pgs=250 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:30:30.890 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:30.887+0000 7fea8ef7a640 1 --2- 172.21.15.37:0/657813815 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fea880d4560 0x7fea880cb530 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:30:30.890 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:30.887+0000 7fea8ef7a640 1 --2- 172.21.15.37:0/657813815 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fea880b7600 0x7fea880ce920 unknown :-1 s=CLOSED pgs=520 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:30:30.890 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:30.887+0000 7fea8ef7a640 1 --2- 172.21.15.37:0/657813815 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7fea880b4bb0 0x7fea881525a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:30:30.890 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:30.887+0000 7fea8ef7a640 1 -- 172.21.15.37:0/657813815 >> 172.21.15.37:0/657813815 conn(0x7fea880d27e0 msgr2=0x7fea880d0d20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:30:30.890 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:30.887+0000 7fea8ef7a640 1 -- 172.21.15.37:0/657813815 shutdown_connections 2024-04-25T10:30:30.890 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:30.887+0000 7fea8ef7a640 1 -- 172.21.15.37:0/657813815 wait complete. 2024-04-25T10:30:30.903 DEBUG:tasks.cephfs.filesystem:fs_config: {} 2024-04-25T10:30:30.903 DEBUG:teuthology.orchestra.run.smithi037:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs set recovery_fs allow_standby_replay false 2024-04-25T10:30:31.025 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:31.023+0000 7f5f979b2640 1 Processor -- start 2024-04-25T10:30:31.025 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:31.023+0000 7f5f979b2640 1 -- start start 2024-04-25T10:30:31.030 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:31.024+0000 7f5f979b2640 1 --2- >> v2:172.21.15.203:3300/0 conn(0x7f5f900d5160 0x7f5f900b4e80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:30:31.030 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:31.024+0000 7f5f979b2640 1 --2- >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f5f900d3230 0x7f5f900b9060 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:30:31.030 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:31.024+0000 7f5f979b2640 1 -- --> v1:172.21.15.37:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f5f900d1370 con 0x7f5f900d0990 2024-04-25T10:30:31.030 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:31.024+0000 7f5f979b2640 1 -- --> v2:172.21.15.203:3300/0 -- mon_getmap magic: 0 -- 0x7f5f900d1650 con 0x7f5f900d5160 2024-04-25T10:30:31.030 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:31.024+0000 7f5f979b2640 1 -- --> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] -- mon_getmap magic: 0 -- 0x7f5f900d4680 con 0x7f5f900d3230 2024-04-25T10:30:31.030 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:31.025+0000 7f5f94f4d640 1 --1- >> v1:172.21.15.37:6789/0 conn(0x7f5f900d0990 0x7f5f900d2b20 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.37:6789/0 says I am v1:172.21.15.37:40884/0 (socket says 172.21.15.37:40884) 2024-04-25T10:30:31.030 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:31.025+0000 7f5f94f4d640 1 -- 172.21.15.37:0/3744278642 learned_addr learned my addr 172.21.15.37:0/3744278642 (peer_addr_for_me v1:172.21.15.37:0/0) 2024-04-25T10:30:31.031 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:31.025+0000 7f5f87fff640 1 -- 172.21.15.37:0/3744278642 <== mon.0 v1:172.21.15.37:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 416868498 0 0) 0x7f5f900d1370 con 0x7f5f900d0990 2024-04-25T10:30:31.031 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:31.025+0000 7f5f87fff640 1 -- 172.21.15.37:0/3744278642 --> v1:172.21.15.37:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f5f7c003520 con 0x7f5f900d0990 2024-04-25T10:30:31.031 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:31.025+0000 7f5f95f4f640 1 --2- 172.21.15.37:0/3744278642 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f5f900d3230 0x7f5f900b9060 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:30:31.031 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:31.025+0000 7f5f9574e640 1 --2- 172.21.15.37:0/3744278642 >> v2:172.21.15.203:3300/0 conn(0x7f5f900d5160 0x7f5f900b4e80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:30:31.031 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:31.025+0000 7f5f87fff640 1 -- 172.21.15.37:0/3744278642 <== mon.0 v1:172.21.15.37:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 3234325800 0 0) 0x7f5f7c003520 con 0x7f5f900d0990 2024-04-25T10:30:31.031 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:31.025+0000 7f5f87fff640 1 -- 172.21.15.37:0/3744278642 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f5f900d3230 msgr2=0x7f5f900b9060 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:30:31.031 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:31.025+0000 7f5f87fff640 1 --2- 172.21.15.37:0/3744278642 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f5f900d3230 0x7f5f900b9060 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:30:31.031 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:31.025+0000 7f5f87fff640 1 -- 172.21.15.37:0/3744278642 >> v2:172.21.15.203:3300/0 conn(0x7f5f900d5160 msgr2=0x7f5f900b4e80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:30:31.032 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:31.025+0000 7f5f87fff640 1 --2- 172.21.15.37:0/3744278642 >> v2:172.21.15.203:3300/0 conn(0x7f5f900d5160 0x7f5f900b4e80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:30:31.032 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:31.025+0000 7f5f87fff640 1 -- 172.21.15.37:0/3744278642 --> v1:172.21.15.37:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f5f900d4960 con 0x7f5f900d0990 2024-04-25T10:30:31.032 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:31.025+0000 7f5f87fff640 1 -- 172.21.15.37:0/3744278642 <== mon.0 v1:172.21.15.37:6789/0 3 ==== mon_map magic: 0 ==== 413+0+0 (unknown 1577794600 0 0) 0x7f5f80001690 con 0x7f5f900d0990 2024-04-25T10:30:31.032 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:31.025+0000 7f5f9574e640 1 --2- 172.21.15.37:0/3744278642 >> v2:172.21.15.203:3300/0 conn(0x7f5f900d5160 0x7f5f900b4e80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T10:30:31.032 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:31.025+0000 7f5f87fff640 1 -- 172.21.15.37:0/3744278642 >> v1:172.21.15.37:6789/0 conn(0x7f5f900d0990 legacy=0x7f5f900d2b20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:30:31.032 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:31.025+0000 7f5f87fff640 1 --2- 172.21.15.37:0/3744278642 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f5f7c0039b0 0x7f5f7c003dc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:30:31.032 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:31.025+0000 7f5f87fff640 1 --2- 172.21.15.37:0/3744278642 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f5f7c004340 0x7f5f7c004860 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:30:31.032 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:31.025+0000 7f5f87fff640 1 --2- 172.21.15.37:0/3744278642 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f5f7c005020 0x7f5f7c00b9c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:30:31.032 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:31.025+0000 7f5f87fff640 1 -- 172.21.15.37:0/3744278642 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 -- 0x7f5f900d1650 con 0x7f5f7c004340 2024-04-25T10:30:31.032 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:31.025+0000 7f5f87fff640 1 -- 172.21.15.37:0/3744278642 --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_getmap magic: 0 -- 0x7f5f900d4680 con 0x7f5f7c005020 2024-04-25T10:30:31.032 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:31.025+0000 7f5f87fff640 1 -- 172.21.15.37:0/3744278642 --> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] -- mon_getmap magic: 0 -- 0x7f5f7c00c000 con 0x7f5f7c0039b0 2024-04-25T10:30:31.032 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:31.025+0000 7f5f94f4d640 1 --2- 172.21.15.37:0/3744278642 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f5f7c004340 0x7f5f7c004860 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:30:31.032 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:31.025+0000 7f5f9574e640 1 --2- 172.21.15.37:0/3744278642 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f5f7c0039b0 0x7f5f7c003dc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:30:31.032 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:31.025+0000 7f5f95f4f640 1 --2- 172.21.15.37:0/3744278642 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f5f7c005020 0x7f5f7c00b9c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:30:31.033 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:31.026+0000 7f5f94f4d640 1 -- 172.21.15.37:0/3744278642 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f5f7c0039b0 msgr2=0x7f5f7c003dc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:30:31.033 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:31.026+0000 7f5f94f4d640 1 --2- 172.21.15.37:0/3744278642 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f5f7c0039b0 0x7f5f7c003dc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:30:31.033 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:31.026+0000 7f5f94f4d640 1 -- 172.21.15.37:0/3744278642 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f5f7c005020 msgr2=0x7f5f7c00b9c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:30:31.033 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:31.026+0000 7f5f94f4d640 1 --2- 172.21.15.37:0/3744278642 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f5f7c005020 0x7f5f7c00b9c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:30:31.033 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:31.026+0000 7f5f94f4d640 1 -- 172.21.15.37:0/3744278642 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f5f7c003520 con 0x7f5f7c004340 2024-04-25T10:30:31.033 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:31.026+0000 7f5f94f4d640 1 --2- 172.21.15.37:0/3744278642 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f5f7c004340 0x7f5f7c004860 secure :-1 s=READY pgs=522 cs=0 l=1 rev1=1 crypto rx=0x7f5f80007fd0 tx=0x7f5f7c005830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c7d8e32dc0f55122 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:30:31.033 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:31.027+0000 7f5f87fff640 1 -- 172.21.15.37:0/3744278642 <== mon.0 v2:172.21.15.37:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5f80007d50 con 0x7f5f7c004340 2024-04-25T10:30:31.033 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:31.027+0000 7f5f87fff640 1 -- 172.21.15.37:0/3744278642 <== mon.0 v2:172.21.15.37:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f5f80005ad0 con 0x7f5f7c004340 2024-04-25T10:30:31.033 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:31.027+0000 7f5f87fff640 1 -- 172.21.15.37:0/3744278642 <== mon.0 v2:172.21.15.37:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5f80007070 con 0x7f5f7c004340 2024-04-25T10:30:31.033 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:31.027+0000 7f5f979b2640 1 -- 172.21.15.37:0/3744278642 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f5f7c004340 msgr2=0x7f5f7c004860 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:30:31.033 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:31.027+0000 7f5f979b2640 1 --2- 172.21.15.37:0/3744278642 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f5f7c004340 0x7f5f7c004860 secure :-1 s=READY pgs=522 cs=0 l=1 rev1=1 crypto rx=0x7f5f80007fd0 tx=0x7f5f7c005830 comp rx=0 tx=0).stop 2024-04-25T10:30:31.033 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:31.027+0000 7f5f95f4f640 1 -- 172.21.15.37:0/3744278642 reap_dead start 2024-04-25T10:30:31.033 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:31.027+0000 7f5f979b2640 1 -- 172.21.15.37:0/3744278642 shutdown_connections 2024-04-25T10:30:31.033 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:31.027+0000 7f5f979b2640 1 -- 172.21.15.37:0/3744278642 >> 172.21.15.37:0/3744278642 conn(0x7f5f900b7a80 msgr2=0x7f5f900b7e90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:30:31.034 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:31.027+0000 7f5f979b2640 1 -- 172.21.15.37:0/3744278642 shutdown_connections 2024-04-25T10:30:31.034 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:31.027+0000 7f5f979b2640 1 -- 172.21.15.37:0/3744278642 wait complete. 2024-04-25T10:30:31.037 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:31.029+0000 7f5f979b2640 1 Processor -- start 2024-04-25T10:30:31.037 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:31.029+0000 7f5f979b2640 1 -- start start 2024-04-25T10:30:31.037 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:31.030+0000 7f5f979b2640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f5f7c004340 0x7f5f900c9680 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:30:31.037 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:31.030+0000 7f5f979b2640 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f5f900cd760 0x7f5f90151c80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:30:31.037 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:31.030+0000 7f5f979b2640 1 --2- >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f5f900cec60 0x7f5f900cf070 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:30:31.037 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:31.030+0000 7f5f979b2640 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 -- 0x7f5f90164c40 con 0x7f5f7c004340 2024-04-25T10:30:31.037 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:31.030+0000 7f5f979b2640 1 -- --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_getmap magic: 0 -- 0x7f5f90164ac0 con 0x7f5f900cd760 2024-04-25T10:30:31.037 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:31.030+0000 7f5f979b2640 1 -- --> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] -- mon_getmap magic: 0 -- 0x7f5f900d3230 con 0x7f5f900cec60 2024-04-25T10:30:31.038 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:31.031+0000 7f5f9574e640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f5f7c004340 0x7f5f900c9680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:30:31.038 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:31.031+0000 7f5f9574e640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f5f7c004340 0x7f5f900c9680 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.37:3300/0 says I am v2:172.21.15.37:55368/0 (socket says 172.21.15.37:55368) 2024-04-25T10:30:31.038 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:31.031+0000 7f5f9574e640 1 -- 172.21.15.37:0/3407425873 learned_addr learned my addr 172.21.15.37:0/3407425873 (peer_addr_for_me v2:172.21.15.37:0/0) 2024-04-25T10:30:31.038 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:31.031+0000 7f5f95f4f640 1 --2- 172.21.15.37:0/3407425873 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f5f900cec60 0x7f5f900cf070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:30:31.038 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:31.031+0000 7f5f94f4d640 1 --2- 172.21.15.37:0/3407425873 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f5f900cd760 0x7f5f90151c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:30:31.038 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:31.031+0000 7f5f9574e640 1 -- 172.21.15.37:0/3407425873 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f5f900cec60 msgr2=0x7f5f900cf070 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:30:31.038 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:31.031+0000 7f5f9574e640 1 --2- 172.21.15.37:0/3407425873 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f5f900cec60 0x7f5f900cf070 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:30:31.038 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:31.031+0000 7f5f9574e640 1 -- 172.21.15.37:0/3407425873 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f5f900cd760 msgr2=0x7f5f90151c80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:30:31.038 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:31.031+0000 7f5f9574e640 1 --2- 172.21.15.37:0/3407425873 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f5f900cd760 0x7f5f90151c80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:30:31.038 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:31.031+0000 7f5f9574e640 1 -- 172.21.15.37:0/3407425873 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f5f900cf5b0 con 0x7f5f7c004340 2024-04-25T10:30:31.038 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:31.031+0000 7f5f9574e640 1 --2- 172.21.15.37:0/3407425873 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f5f7c004340 0x7f5f900c9680 secure :-1 s=READY pgs=523 cs=0 l=1 rev1=1 crypto rx=0x7f5f8c0115d0 tx=0x7f5f8c011a90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:30:31.038 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:31.031+0000 7f5f85ffb640 1 -- 172.21.15.37:0/3407425873 <== mon.0 v2:172.21.15.37:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5f8c019070 con 0x7f5f7c004340 2024-04-25T10:30:31.038 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:31.031+0000 7f5f85ffb640 1 -- 172.21.15.37:0/3407425873 <== mon.0 v2:172.21.15.37:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f5f8c00f9b0 con 0x7f5f7c004340 2024-04-25T10:30:31.038 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:31.031+0000 7f5f979b2640 1 -- 172.21.15.37:0/3407425873 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f5f900cdeb0 con 0x7f5f7c004340 2024-04-25T10:30:31.039 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:31.031+0000 7f5f85ffb640 1 -- 172.21.15.37:0/3407425873 <== mon.0 v2:172.21.15.37:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5f8c00f040 con 0x7f5f7c004340 2024-04-25T10:30:31.039 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:31.031+0000 7f5f979b2640 1 -- 172.21.15.37:0/3407425873 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f5f900ce190 con 0x7f5f7c004340 2024-04-25T10:30:31.039 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:31.032+0000 7f5f979b2640 1 -- 172.21.15.37:0/3407425873 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f5f9006a140 con 0x7f5f7c004340 2024-04-25T10:30:31.039 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:31.033+0000 7f5f85ffb640 1 -- 172.21.15.37:0/3407425873 <== mon.0 v2:172.21.15.37:3300/0 4 ==== mgrmap(e 5) ==== 96782+0+0 (secure 0 0 0) 0x7f5f8c01c070 con 0x7f5f7c004340 2024-04-25T10:30:31.039 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:31.033+0000 7f5f85ffb640 1 --2- 172.21.15.37:0/3407425873 >> [v2:172.21.15.203:6832/2418205237,v1:172.21.15.203:6833/2418205237] conn(0x7f5f740744e0 0x7f5f740769f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:30:31.039 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:31.034+0000 7f5f85ffb640 1 -- 172.21.15.37:0/3407425873 <== mon.0 v2:172.21.15.37:3300/0 5 ==== osd_map(44..44 src has 1..44) ==== 6248+0+0 (secure 0 0 0) 0x7f5f8c0957d0 con 0x7f5f7c004340 2024-04-25T10:30:31.039 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:31.034+0000 7f5f94f4d640 1 --2- 172.21.15.37:0/3407425873 >> [v2:172.21.15.203:6832/2418205237,v1:172.21.15.203:6833/2418205237] conn(0x7f5f740744e0 0x7f5f740769f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:30:31.039 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:31.034+0000 7f5f94f4d640 1 --2- 172.21.15.37:0/3407425873 >> [v2:172.21.15.203:6832/2418205237,v1:172.21.15.203:6833/2418205237] conn(0x7f5f740744e0 0x7f5f740769f0 secure :-1 s=READY pgs=251 cs=0 l=1 rev1=1 crypto rx=0x7f5f80033000 tx=0x7f5f80009030 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:30:31.040 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:31.038+0000 7f5f85ffb640 1 -- 172.21.15.37:0/3407425873 <== mon.0 v2:172.21.15.37:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187246 (secure 0 0 0) 0x7f5f900cf5b0 con 0x7f5f7c004340 2024-04-25T10:30:31.263 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:31.261+0000 7f5f979b2640 1 -- 172.21.15.37:0/3407425873 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_command({"prefix": "fs set", "fs_name": "recovery_fs", "var": "allow_standby_replay", "val": "false"} v 0) -- 0x7f5f900d2e40 con 0x7f5f7c004340 2024-04-25T10:30:31.892 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:31.890+0000 7f5f85ffb640 1 -- 172.21.15.37:0/3407425873 <== mon.0 v2:172.21.15.37:3300/0 7 ==== mon_command_ack([{"prefix": "fs set", "fs_name": "recovery_fs", "var": "allow_standby_replay", "val": "false"}]=0 v22) ==== 127+0+0 (secure 0 0 0) 0x7f5f8c05fd30 con 0x7f5f7c004340 2024-04-25T10:30:31.895 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:31.892+0000 7f5f979b2640 1 -- 172.21.15.37:0/3407425873 >> [v2:172.21.15.203:6832/2418205237,v1:172.21.15.203:6833/2418205237] conn(0x7f5f740744e0 msgr2=0x7f5f740769f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:30:31.895 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:31.892+0000 7f5f979b2640 1 --2- 172.21.15.37:0/3407425873 >> [v2:172.21.15.203:6832/2418205237,v1:172.21.15.203:6833/2418205237] conn(0x7f5f740744e0 0x7f5f740769f0 secure :-1 s=READY pgs=251 cs=0 l=1 rev1=1 crypto rx=0x7f5f80033000 tx=0x7f5f80009030 comp rx=0 tx=0).stop 2024-04-25T10:30:31.895 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:31.892+0000 7f5f979b2640 1 -- 172.21.15.37:0/3407425873 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f5f7c004340 msgr2=0x7f5f900c9680 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:30:31.895 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:31.892+0000 7f5f979b2640 1 --2- 172.21.15.37:0/3407425873 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f5f7c004340 0x7f5f900c9680 secure :-1 s=READY pgs=523 cs=0 l=1 rev1=1 crypto rx=0x7f5f8c0115d0 tx=0x7f5f8c011a90 comp rx=0 tx=0).stop 2024-04-25T10:30:31.895 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:31.892+0000 7f5f979b2640 1 -- 172.21.15.37:0/3407425873 shutdown_connections 2024-04-25T10:30:31.895 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:31.892+0000 7f5f979b2640 1 --2- 172.21.15.37:0/3407425873 >> [v2:172.21.15.203:6832/2418205237,v1:172.21.15.203:6833/2418205237] conn(0x7f5f740744e0 0x7f5f740769f0 unknown :-1 s=CLOSED pgs=251 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:30:31.895 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:31.892+0000 7f5f979b2640 1 --2- 172.21.15.37:0/3407425873 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f5f900cec60 0x7f5f900cf070 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:30:31.895 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:31.892+0000 7f5f979b2640 1 --2- 172.21.15.37:0/3407425873 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f5f900cd760 0x7f5f90151c80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:30:31.895 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:31.892+0000 7f5f979b2640 1 --2- 172.21.15.37:0/3407425873 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f5f7c004340 0x7f5f900c9680 unknown :-1 s=CLOSED pgs=523 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:30:31.896 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:31.892+0000 7f5f979b2640 1 -- 172.21.15.37:0/3407425873 >> 172.21.15.37:0/3407425873 conn(0x7f5f900b7a80 msgr2=0x7f5f9015e8c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:30:31.896 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:31.892+0000 7f5f979b2640 1 -- 172.21.15.37:0/3407425873 shutdown_connections 2024-04-25T10:30:31.896 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:31.892+0000 7f5f979b2640 1 -- 172.21.15.37:0/3407425873 wait complete. 2024-04-25T10:30:31.909 DEBUG:tasks.cephfs.filesystem:Not Creating any subvols for filesystem 'recovery_fs' 2024-04-25T10:30:31.909 DEBUG:teuthology.orchestra.run.smithi037:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-04-25T10:30:32.027 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:32.025+0000 7f0386d46640 1 Processor -- start 2024-04-25T10:30:32.027 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:32.025+0000 7f0386d46640 1 -- start start 2024-04-25T10:30:32.032 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:32.026+0000 7f0386d46640 1 --2- >> v2:172.21.15.203:3300/0 conn(0x7f03800b02b0 0x7f03800b06c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:30:32.032 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:32.026+0000 7f0386d46640 1 -- --> v1:172.21.15.37:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f03800b36d0 con 0x7f038009b700 2024-04-25T10:30:32.032 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:32.026+0000 7f0386d46640 1 -- --> v1:172.21.15.203:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f03800b39b0 con 0x7f03800d3430 2024-04-25T10:30:32.032 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:32.026+0000 7f0386d46640 1 -- --> v2:172.21.15.203:3300/0 -- mon_getmap magic: 0 -- 0x7f038006e1c0 con 0x7f03800b02b0 2024-04-25T10:30:32.032 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:32.026+0000 7f03852e3640 1 --1- >> v1:172.21.15.37:6789/0 conn(0x7f038009b700 0x7f038009bb10 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.37:6789/0 says I am v1:172.21.15.37:40900/0 (socket says 172.21.15.37:40900) 2024-04-25T10:30:32.033 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:32.027+0000 7f03852e3640 1 -- 172.21.15.37:0/1652142437 learned_addr learned my addr 172.21.15.37:0/1652142437 (peer_addr_for_me v1:172.21.15.37:0/0) 2024-04-25T10:30:32.033 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:32.027+0000 7f037ffff640 1 --2- 172.21.15.37:0/1652142437 >> v2:172.21.15.203:3300/0 conn(0x7f03800b02b0 0x7f03800b06c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:30:32.033 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:32.027+0000 7f037f7fe640 1 -- 172.21.15.37:0/1652142437 <== mon.0 v1:172.21.15.37:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 1588058555 0 0) 0x7f03800b36d0 con 0x7f038009b700 2024-04-25T10:30:32.033 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:32.027+0000 7f037f7fe640 1 -- 172.21.15.37:0/1652142437 --> v1:172.21.15.37:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f0370003520 con 0x7f038009b700 2024-04-25T10:30:32.033 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:32.027+0000 7f037f7fe640 1 -- 172.21.15.37:0/1652142437 <== mon.1 v1:172.21.15.203:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 2183078228 0 0) 0x7f03800b39b0 con 0x7f03800d3430 2024-04-25T10:30:32.033 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:32.027+0000 7f037f7fe640 1 -- 172.21.15.37:0/1652142437 --> v1:172.21.15.203:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f03800b36d0 con 0x7f03800d3430 2024-04-25T10:30:32.033 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:32.027+0000 7f037f7fe640 1 -- 172.21.15.37:0/1652142437 <== mon.0 v1:172.21.15.37:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 1922189209 0 0) 0x7f0370003520 con 0x7f038009b700 2024-04-25T10:30:32.033 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:32.027+0000 7f037f7fe640 1 -- 172.21.15.37:0/1652142437 >> v2:172.21.15.203:3300/0 conn(0x7f03800b02b0 msgr2=0x7f03800b06c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:30:32.033 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:32.027+0000 7f037f7fe640 1 --2- 172.21.15.37:0/1652142437 >> v2:172.21.15.203:3300/0 conn(0x7f03800b02b0 0x7f03800b06c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:30:32.033 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:32.027+0000 7f037f7fe640 1 -- 172.21.15.37:0/1652142437 >> v1:172.21.15.203:6789/0 conn(0x7f03800d3430 legacy=0x7f03800b2e00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:30:32.034 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:32.027+0000 7f037f7fe640 1 -- 172.21.15.37:0/1652142437 --> v1:172.21.15.37:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f038006db70 con 0x7f038009b700 2024-04-25T10:30:32.034 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:32.027+0000 7f037f7fe640 1 -- 172.21.15.37:0/1652142437 <== mon.0 v1:172.21.15.37:6789/0 3 ==== mon_map magic: 0 ==== 413+0+0 (unknown 1577794600 0 0) 0x7f03780019f0 con 0x7f038009b700 2024-04-25T10:30:32.034 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:32.027+0000 7f037f7fe640 1 -- 172.21.15.37:0/1652142437 >> v1:172.21.15.37:6789/0 conn(0x7f038009b700 legacy=0x7f038009bb10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:30:32.034 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:32.027+0000 7f037f7fe640 1 --2- 172.21.15.37:0/1652142437 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f0370003b40 0x7f0370003f90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:30:32.034 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:32.027+0000 7f037f7fe640 1 --2- 172.21.15.37:0/1652142437 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f0370004800 0x7f0370004c10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:30:32.034 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:32.027+0000 7f037f7fe640 1 --2- 172.21.15.37:0/1652142437 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f03700051a0 0x7f037000b9c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:30:32.034 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:32.027+0000 7f037f7fe640 1 -- 172.21.15.37:0/1652142437 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 -- 0x7f038006e1c0 con 0x7f0370003b40 2024-04-25T10:30:32.034 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:32.027+0000 7f037f7fe640 1 -- 172.21.15.37:0/1652142437 --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_getmap magic: 0 -- 0x7f037000c140 con 0x7f0370004800 2024-04-25T10:30:32.034 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:32.027+0000 7f037f7fe640 1 -- 172.21.15.37:0/1652142437 --> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] -- mon_getmap magic: 0 -- 0x7f037000c340 con 0x7f03700051a0 2024-04-25T10:30:32.034 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:32.027+0000 7f0384ae2640 1 --2- 172.21.15.37:0/1652142437 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f0370003b40 0x7f0370003f90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:30:32.035 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:32.027+0000 7f037ffff640 1 --2- 172.21.15.37:0/1652142437 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f0370004800 0x7f0370004c10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:30:32.035 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:32.027+0000 7f03852e3640 1 --2- 172.21.15.37:0/1652142437 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f03700051a0 0x7f037000b9c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:30:32.035 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:32.028+0000 7f0384ae2640 1 -- 172.21.15.37:0/1652142437 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f03700051a0 msgr2=0x7f037000b9c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:30:32.035 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:32.028+0000 7f0384ae2640 1 --2- 172.21.15.37:0/1652142437 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f03700051a0 0x7f037000b9c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:30:32.035 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:32.028+0000 7f0384ae2640 1 -- 172.21.15.37:0/1652142437 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f0370004800 msgr2=0x7f0370004c10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:30:32.035 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:32.028+0000 7f0384ae2640 1 --2- 172.21.15.37:0/1652142437 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f0370004800 0x7f0370004c10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:30:32.035 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:32.028+0000 7f03852e3640 1 -- 172.21.15.37:0/1652142437 reap_dead start 2024-04-25T10:30:32.035 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:32.028+0000 7f0384ae2640 1 -- 172.21.15.37:0/1652142437 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f0370003520 con 0x7f0370003b40 2024-04-25T10:30:32.036 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:32.028+0000 7f0384ae2640 1 --2- 172.21.15.37:0/1652142437 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f0370003b40 0x7f0370003f90 secure :-1 s=READY pgs=525 cs=0 l=1 rev1=1 crypto rx=0x7f03700057e0 tx=0x7f0370005830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=904c986568225d66 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:30:32.036 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:32.029+0000 7f037f7fe640 1 -- 172.21.15.37:0/1652142437 <== mon.0 v2:172.21.15.37:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f03740014f0 con 0x7f0370003b40 2024-04-25T10:30:32.036 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:32.029+0000 7f037f7fe640 1 -- 172.21.15.37:0/1652142437 <== mon.0 v2:172.21.15.37:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f03740052d0 con 0x7f0370003b40 2024-04-25T10:30:32.036 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:32.029+0000 7f037f7fe640 1 -- 172.21.15.37:0/1652142437 <== mon.0 v2:172.21.15.37:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f037402abb0 con 0x7f0370003b40 2024-04-25T10:30:32.036 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:32.029+0000 7f0386d46640 1 -- 172.21.15.37:0/1652142437 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f0370003b40 msgr2=0x7f0370003f90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:30:32.036 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:32.029+0000 7f0386d46640 1 --2- 172.21.15.37:0/1652142437 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f0370003b40 0x7f0370003f90 secure :-1 s=READY pgs=525 cs=0 l=1 rev1=1 crypto rx=0x7f03700057e0 tx=0x7f0370005830 comp rx=0 tx=0).stop 2024-04-25T10:30:32.036 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:32.029+0000 7f0386d46640 1 -- 172.21.15.37:0/1652142437 shutdown_connections 2024-04-25T10:30:32.036 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:32.029+0000 7f0386d46640 1 --2- 172.21.15.37:0/1652142437 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f0370003b40 0x7f0370003f90 unknown :-1 s=CLOSED pgs=525 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:30:32.036 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:32.029+0000 7f0386d46640 1 -- 172.21.15.37:0/1652142437 >> 172.21.15.37:0/1652142437 conn(0x7f03800b7600 msgr2=0x7f03800b7c30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:30:32.036 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:32.029+0000 7f0386d46640 1 -- 172.21.15.37:0/1652142437 shutdown_connections 2024-04-25T10:30:32.036 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:32.029+0000 7f0386d46640 1 -- 172.21.15.37:0/1652142437 wait complete. 2024-04-25T10:30:32.041 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:32.031+0000 7f0386d46640 1 Processor -- start 2024-04-25T10:30:32.041 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:32.031+0000 7f0386d46640 1 -- start start 2024-04-25T10:30:32.041 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:32.033+0000 7f0386d46640 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f0370003b40 0x7f038015a050 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:30:32.041 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:32.033+0000 7f0386d46640 1 --2- >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f03800cfa50 0x7f038009ff90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:30:32.041 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:32.033+0000 7f0386d46640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f03800d0c80 0x7f03800d1090 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:30:32.041 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:32.033+0000 7f0386d46640 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 -- 0x7f0380160720 con 0x7f03800d0c80 2024-04-25T10:30:32.041 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:32.033+0000 7f0386d46640 1 -- --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_getmap magic: 0 -- 0x7f03801605a0 con 0x7f0370003b40 2024-04-25T10:30:32.041 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:32.033+0000 7f0386d46640 1 -- --> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] -- mon_getmap magic: 0 -- 0x7f03800b06c0 con 0x7f03800cfa50 2024-04-25T10:30:32.042 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:32.033+0000 7f03852e3640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f03800d0c80 0x7f03800d1090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:30:32.042 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:32.033+0000 7f03852e3640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f03800d0c80 0x7f03800d1090 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.37:3300/0 says I am v2:172.21.15.37:55386/0 (socket says 172.21.15.37:55386) 2024-04-25T10:30:32.042 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:32.033+0000 7f03852e3640 1 -- 172.21.15.37:0/4005957164 learned_addr learned my addr 172.21.15.37:0/4005957164 (peer_addr_for_me v2:172.21.15.37:0/0) 2024-04-25T10:30:32.042 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:32.033+0000 7f037ffff640 1 --2- 172.21.15.37:0/4005957164 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f03800cfa50 0x7f038009ff90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:30:32.042 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:32.033+0000 7f0384ae2640 1 --2- 172.21.15.37:0/4005957164 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f0370003b40 0x7f038015a050 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:30:32.042 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:32.033+0000 7f03852e3640 1 -- 172.21.15.37:0/4005957164 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f03800cfa50 msgr2=0x7f038009ff90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:30:32.042 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:32.033+0000 7f03852e3640 1 --2- 172.21.15.37:0/4005957164 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f03800cfa50 0x7f038009ff90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:30:32.042 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:32.033+0000 7f03852e3640 1 -- 172.21.15.37:0/4005957164 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f0370003b40 msgr2=0x7f038015a050 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:30:32.042 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:32.033+0000 7f03852e3640 1 --2- 172.21.15.37:0/4005957164 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f0370003b40 0x7f038015a050 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:30:32.042 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:32.033+0000 7f03852e3640 1 -- 172.21.15.37:0/4005957164 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f03800d15d0 con 0x7f03800d0c80 2024-04-25T10:30:32.042 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:32.033+0000 7f0384ae2640 1 --2- 172.21.15.37:0/4005957164 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f0370003b40 0x7f038015a050 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T10:30:32.042 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:32.033+0000 7f03852e3640 1 --2- 172.21.15.37:0/4005957164 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f03800d0c80 0x7f03800d1090 secure :-1 s=READY pgs=526 cs=0 l=1 rev1=1 crypto rx=0x7f03800b43e0 tx=0x7f0378007bd0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:30:32.042 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:32.033+0000 7f037d7fa640 1 -- 172.21.15.37:0/4005957164 <== mon.0 v2:172.21.15.37:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f037800ab90 con 0x7f03800d0c80 2024-04-25T10:30:32.043 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:32.034+0000 7f037d7fa640 1 -- 172.21.15.37:0/4005957164 <== mon.0 v2:172.21.15.37:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f0378018070 con 0x7f03800d0c80 2024-04-25T10:30:32.043 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:32.034+0000 7f037d7fa640 1 -- 172.21.15.37:0/4005957164 <== mon.0 v2:172.21.15.37:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f03780133f0 con 0x7f03800d0c80 2024-04-25T10:30:32.043 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:32.034+0000 7f0386d46640 1 -- 172.21.15.37:0/4005957164 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f038015a9a0 con 0x7f03800d0c80 2024-04-25T10:30:32.043 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:32.034+0000 7f0386d46640 1 -- 172.21.15.37:0/4005957164 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f03800cf250 con 0x7f03800d0c80 2024-04-25T10:30:32.043 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:32.035+0000 7f0386d46640 1 -- 172.21.15.37:0/4005957164 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f038006a140 con 0x7f03800d0c80 2024-04-25T10:30:32.043 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:32.035+0000 7f037d7fa640 1 -- 172.21.15.37:0/4005957164 <== mon.0 v2:172.21.15.37:3300/0 4 ==== mgrmap(e 5) ==== 96782+0+0 (secure 0 0 0) 0x7f0378013590 con 0x7f03800d0c80 2024-04-25T10:30:32.043 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:32.036+0000 7f037d7fa640 1 --2- 172.21.15.37:0/4005957164 >> [v2:172.21.15.203:6832/2418205237,v1:172.21.15.203:6833/2418205237] conn(0x7f03640744b0 0x7f03640769c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:30:32.043 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:32.036+0000 7f037d7fa640 1 -- 172.21.15.37:0/4005957164 <== mon.0 v2:172.21.15.37:3300/0 5 ==== osd_map(44..44 src has 1..44) ==== 6248+0+0 (secure 0 0 0) 0x7f0378095a10 con 0x7f03800d0c80 2024-04-25T10:30:32.043 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:32.036+0000 7f0384ae2640 1 --2- 172.21.15.37:0/4005957164 >> [v2:172.21.15.203:6832/2418205237,v1:172.21.15.203:6833/2418205237] conn(0x7f03640744b0 0x7f03640769c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:30:32.043 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:32.036+0000 7f0384ae2640 1 --2- 172.21.15.37:0/4005957164 >> [v2:172.21.15.203:6832/2418205237,v1:172.21.15.203:6833/2418205237] conn(0x7f03640744b0 0x7f03640769c0 secure :-1 s=READY pgs=252 cs=0 l=1 rev1=1 crypto rx=0x7f0374032000 tx=0x7f0374009030 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:30:32.045 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:32.040+0000 7f037d7fa640 1 -- 172.21.15.37:0/4005957164 <== mon.0 v2:172.21.15.37:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187246 (secure 0 0 0) 0x7f0378060000 con 0x7f03800d0c80 2024-04-25T10:30:32.264 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:32.262+0000 7f0386d46640 1 -- 172.21.15.37:0/4005957164 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f03800cf630 con 0x7f03800d0c80 2024-04-25T10:30:32.265 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:32.263+0000 7f037d7fa640 1 -- 172.21.15.37:0/4005957164 <== mon.0 v2:172.21.15.37:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 22 v22) ==== 94+0+6474 (secure 0 0 0) 0x7f037805f900 con 0x7f03800d0c80 2024-04-25T10:30:32.265 INFO:teuthology.orchestra.run.smithi037.stdout: 2024-04-25T10:30:32.266 INFO:teuthology.orchestra.run.smithi037.stdout:{"epoch":22,"default_fscid":2,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":4946,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.37:6835/683655741","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.37:6834","nonce":683655741},{"type":"v1","addr":"172.21.15.37:6835","nonce":683655741}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":19},{"gid":5215,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.203:6836/3927045001","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6835","nonce":3927045001},{"type":"v1","addr":"172.21.15.203:6836","nonce":3927045001}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":19},{"gid":5218,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.203:6837/3108183946","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6834","nonce":3108183946},{"type":"v1","addr":"172.21.15.203:6837","nonce":3108183946}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":19},{"gid":5625,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.37:6833/2353203722","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.37:6832","nonce":2353203722},{"type":"v1","addr":"172.21.15.37:6833","nonce":2353203722}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":19}],"filesystems":[{"mdsmap":{"epoch":18,"flags":19,"flags_state":{"joinable":false,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-04-25T10:30:12.130473+0000","modified":"2024-04-25T10:30:24.853915+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":37,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":1,"in":[0],"up":{},"failed":[0],"damaged":[],"stopped":[],"info":{},"data_pools":[5],"metadata_pool":4,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":0,"qdb_cluster":[]},"id":2},{"mdsmap":{"epoch":22,"flags":19,"flags_state":{"joinable":false,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-04-25T10:30:28.588617+0000","modified":"2024-04-25T10:30:31.262530+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":1,"in":[0],"up":{},"failed":[0],"damaged":[],"stopped":[],"info":{},"data_pools":[5],"metadata_pool":6,"enabled":true,"fs_name":"recovery_fs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":0,"qdb_cluster":[]},"id":3}]} 2024-04-25T10:30:32.266 INFO:teuthology.orchestra.run.smithi037.stderr:dumped fsmap epoch 22 2024-04-25T10:30:32.267 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:32.265+0000 7f0386d46640 1 -- 172.21.15.37:0/4005957164 >> [v2:172.21.15.203:6832/2418205237,v1:172.21.15.203:6833/2418205237] conn(0x7f03640744b0 msgr2=0x7f03640769c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:30:32.267 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:32.265+0000 7f0386d46640 1 --2- 172.21.15.37:0/4005957164 >> [v2:172.21.15.203:6832/2418205237,v1:172.21.15.203:6833/2418205237] conn(0x7f03640744b0 0x7f03640769c0 secure :-1 s=READY pgs=252 cs=0 l=1 rev1=1 crypto rx=0x7f0374032000 tx=0x7f0374009030 comp rx=0 tx=0).stop 2024-04-25T10:30:32.267 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:32.265+0000 7f0386d46640 1 -- 172.21.15.37:0/4005957164 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f03800d0c80 msgr2=0x7f03800d1090 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:30:32.268 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:32.265+0000 7f0386d46640 1 --2- 172.21.15.37:0/4005957164 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f03800d0c80 0x7f03800d1090 secure :-1 s=READY pgs=526 cs=0 l=1 rev1=1 crypto rx=0x7f03800b43e0 tx=0x7f0378007bd0 comp rx=0 tx=0).stop 2024-04-25T10:30:32.268 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:32.265+0000 7f0386d46640 1 -- 172.21.15.37:0/4005957164 shutdown_connections 2024-04-25T10:30:32.268 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:32.265+0000 7f0386d46640 1 --2- 172.21.15.37:0/4005957164 >> [v2:172.21.15.203:6832/2418205237,v1:172.21.15.203:6833/2418205237] conn(0x7f03640744b0 0x7f03640769c0 unknown :-1 s=CLOSED pgs=252 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:30:32.268 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:32.265+0000 7f0386d46640 1 --2- 172.21.15.37:0/4005957164 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f03800d0c80 0x7f03800d1090 unknown :-1 s=CLOSED pgs=526 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:30:32.268 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:32.265+0000 7f0386d46640 1 --2- 172.21.15.37:0/4005957164 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f03800cfa50 0x7f038009ff90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:30:32.268 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:32.265+0000 7f0386d46640 1 --2- 172.21.15.37:0/4005957164 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f0370003b40 0x7f038015a050 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:30:32.268 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:32.265+0000 7f0386d46640 1 -- 172.21.15.37:0/4005957164 >> 172.21.15.37:0/4005957164 conn(0x7f03800b7600 msgr2=0x7f0380160df0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:30:32.268 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:32.265+0000 7f0386d46640 1 -- 172.21.15.37:0/4005957164 shutdown_connections 2024-04-25T10:30:32.268 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:32.265+0000 7f0386d46640 1 -- 172.21.15.37:0/4005957164 wait complete. 2024-04-25T10:30:32.279 DEBUG:teuthology.orchestra.run.smithi037:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-04-25T10:30:32.390 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:32.388+0000 7f8ff0ab1640 1 Processor -- start 2024-04-25T10:30:32.390 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:32.388+0000 7f8ff0ab1640 1 -- start start 2024-04-25T10:30:32.392 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:32.389+0000 7f8ff0ab1640 1 --2- >> v2:172.21.15.203:3300/0 conn(0x7f8fec0b2dc0 0x7f8fec0d2860 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:30:32.392 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:32.389+0000 7f8ff0ab1640 1 --2- >> v2:172.21.15.37:3300/0 conn(0x7f8fec06daf0 0x7f8fec06df00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:30:32.392 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:32.389+0000 7f8ff0ab1640 1 --2- >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f8fec06e440 0x7f8fec08cc90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:30:32.392 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:32.389+0000 7f8ff0ab1640 1 -- --> v2:172.21.15.37:3300/0 -- mon_getmap magic: 0 -- 0x7f8fec0b92c0 con 0x7f8fec06daf0 2024-04-25T10:30:32.392 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:32.389+0000 7f8ff0ab1640 1 -- --> v2:172.21.15.203:3300/0 -- mon_getmap magic: 0 -- 0x7f8fec0ce0d0 con 0x7f8fec0b2dc0 2024-04-25T10:30:32.393 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:32.389+0000 7f8ff0ab1640 1 -- --> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] -- mon_getmap magic: 0 -- 0x7f8fec0d2e40 con 0x7f8fec06e440 2024-04-25T10:30:32.393 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:32.390+0000 7f8fe9d9b640 1 --2- >> v2:172.21.15.37:3300/0 conn(0x7f8fec06daf0 0x7f8fec06df00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:30:32.393 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:32.390+0000 7f8fe9d9b640 1 --2- >> v2:172.21.15.37:3300/0 conn(0x7f8fec06daf0 0x7f8fec06df00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.37:3300/0 says I am v2:172.21.15.37:55400/0 (socket says 172.21.15.37:55400) 2024-04-25T10:30:32.393 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:32.390+0000 7f8fe9d9b640 1 -- 172.21.15.37:0/2939070805 learned_addr learned my addr 172.21.15.37:0/2939070805 (peer_addr_for_me v2:172.21.15.37:0/0) 2024-04-25T10:30:32.393 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:32.390+0000 7f8fead9d640 1 --2- 172.21.15.37:0/2939070805 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f8fec06e440 0x7f8fec08cc90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:30:32.393 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:32.390+0000 7f8fea59c640 1 --2- 172.21.15.37:0/2939070805 >> v2:172.21.15.203:3300/0 conn(0x7f8fec0b2dc0 0x7f8fec0d2860 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:30:32.393 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:32.390+0000 7f8fe9d9b640 1 -- 172.21.15.37:0/2939070805 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f8fec06e440 msgr2=0x7f8fec08cc90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:30:32.393 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:32.390+0000 7f8fe9d9b640 1 --2- 172.21.15.37:0/2939070805 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f8fec06e440 0x7f8fec08cc90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:30:32.393 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:32.390+0000 7f8fe9d9b640 1 -- 172.21.15.37:0/2939070805 >> v2:172.21.15.203:3300/0 conn(0x7f8fec0b2dc0 msgr2=0x7f8fec0d2860 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:30:32.393 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:32.390+0000 7f8fe9d9b640 1 --2- 172.21.15.37:0/2939070805 >> v2:172.21.15.203:3300/0 conn(0x7f8fec0b2dc0 0x7f8fec0d2860 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:30:32.393 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:32.390+0000 7f8fe9d9b640 1 -- 172.21.15.37:0/2939070805 --> v2:172.21.15.37:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f8fec0d3120 con 0x7f8fec06daf0 2024-04-25T10:30:32.394 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:32.391+0000 7f8fe9d9b640 1 --2- 172.21.15.37:0/2939070805 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f8fec06daf0 0x7f8fec06df00 secure :-1 s=READY pgs=527 cs=0 l=1 rev1=1 crypto rx=0x7f8fec070b40 tx=0x7f8fe002a1e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=7391cf5d054446ac server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:30:32.395 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:32.391+0000 7f8fe959a640 1 -- 172.21.15.37:0/2939070805 <== mon.0 v2:172.21.15.37:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8fe0037070 con 0x7f8fec06daf0 2024-04-25T10:30:32.395 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:32.391+0000 7f8fe959a640 1 -- 172.21.15.37:0/2939070805 <== mon.0 v2:172.21.15.37:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f8fe0032440 con 0x7f8fec06daf0 2024-04-25T10:30:32.395 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:32.391+0000 7f8fe959a640 1 -- 172.21.15.37:0/2939070805 <== mon.0 v2:172.21.15.37:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8fe0032810 con 0x7f8fec06daf0 2024-04-25T10:30:32.395 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:32.392+0000 7f8ff0ab1640 1 -- 172.21.15.37:0/2939070805 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f8fec06daf0 msgr2=0x7f8fec06df00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:30:32.395 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:32.392+0000 7f8ff0ab1640 1 --2- 172.21.15.37:0/2939070805 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f8fec06daf0 0x7f8fec06df00 secure :-1 s=READY pgs=527 cs=0 l=1 rev1=1 crypto rx=0x7f8fec070b40 tx=0x7f8fe002a1e0 comp rx=0 tx=0).stop 2024-04-25T10:30:32.395 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:32.392+0000 7f8ff0ab1640 1 -- 172.21.15.37:0/2939070805 shutdown_connections 2024-04-25T10:30:32.395 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:32.392+0000 7f8ff0ab1640 1 --2- 172.21.15.37:0/2939070805 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f8fec06daf0 0x7f8fec06df00 unknown :-1 s=CLOSED pgs=527 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:30:32.395 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:32.392+0000 7f8ff0ab1640 1 --2- 172.21.15.37:0/2939070805 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f8fec06e440 0x7f8fec08cc90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:30:32.395 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:32.392+0000 7f8ff0ab1640 1 --2- 172.21.15.37:0/2939070805 >> v2:172.21.15.203:3300/0 conn(0x7f8fec0b2dc0 0x7f8fec0d2860 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:30:32.395 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:32.392+0000 7f8ff0ab1640 1 -- 172.21.15.37:0/2939070805 >> 172.21.15.37:0/2939070805 conn(0x7f8fec0b5440 msgr2=0x7f8fec0b5850 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:30:32.395 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:32.392+0000 7f8ff0ab1640 1 -- 172.21.15.37:0/2939070805 shutdown_connections 2024-04-25T10:30:32.395 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:32.392+0000 7f8ff0ab1640 1 -- 172.21.15.37:0/2939070805 wait complete. 2024-04-25T10:30:32.396 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:32.394+0000 7f8ff0ab1640 1 Processor -- start 2024-04-25T10:30:32.396 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:32.394+0000 7f8ff0ab1640 1 -- start start 2024-04-25T10:30:32.398 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:32.395+0000 7f8ff0ab1640 1 --2- >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f8fec06daf0 0x7f8fec0cf330 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:30:32.398 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:32.395+0000 7f8ff0ab1640 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f8fec06e440 0x7f8fec15e190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:30:32.398 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:32.395+0000 7f8ff0ab1640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f8fec0b2dc0 0x7f8fec0c7440 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:30:32.398 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:32.395+0000 7f8ff0ab1640 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 -- 0x7f8fec06e850 con 0x7f8fec0b2dc0 2024-04-25T10:30:32.398 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:32.395+0000 7f8ff0ab1640 1 -- --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_getmap magic: 0 -- 0x7f8fec0d36e0 con 0x7f8fec06e440 2024-04-25T10:30:32.399 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:32.395+0000 7f8ff0ab1640 1 -- --> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] -- mon_getmap magic: 0 -- 0x7f8fec08d1d0 con 0x7f8fec06daf0 2024-04-25T10:30:32.399 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:32.395+0000 7f8fead9d640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f8fec0b2dc0 0x7f8fec0c7440 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:30:32.399 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:32.395+0000 7f8fead9d640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f8fec0b2dc0 0x7f8fec0c7440 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.37:3300/0 says I am v2:172.21.15.37:55406/0 (socket says 172.21.15.37:55406) 2024-04-25T10:30:32.399 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:32.395+0000 7f8fead9d640 1 -- 172.21.15.37:0/2741914474 learned_addr learned my addr 172.21.15.37:0/2741914474 (peer_addr_for_me v2:172.21.15.37:0/0) 2024-04-25T10:30:32.399 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:32.395+0000 7f8fea59c640 1 --2- 172.21.15.37:0/2741914474 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f8fec06daf0 0x7f8fec0cf330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:30:32.399 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:32.395+0000 7f8fe9d9b640 1 --2- 172.21.15.37:0/2741914474 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f8fec06e440 0x7f8fec15e190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:30:32.399 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:32.395+0000 7f8fead9d640 1 -- 172.21.15.37:0/2741914474 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f8fec06daf0 msgr2=0x7f8fec0cf330 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:30:32.399 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:32.395+0000 7f8fead9d640 1 --2- 172.21.15.37:0/2741914474 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f8fec06daf0 0x7f8fec0cf330 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:30:32.399 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:32.395+0000 7f8fead9d640 1 -- 172.21.15.37:0/2741914474 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f8fec06e440 msgr2=0x7f8fec15e190 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:30:32.399 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:32.395+0000 7f8fead9d640 1 --2- 172.21.15.37:0/2741914474 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f8fec06e440 0x7f8fec15e190 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:30:32.399 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:32.395+0000 7f8fead9d640 1 -- 172.21.15.37:0/2741914474 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f8fec0cf870 con 0x7f8fec0b2dc0 2024-04-25T10:30:32.399 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:32.396+0000 7f8fead9d640 1 --2- 172.21.15.37:0/2741914474 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f8fec0b2dc0 0x7f8fec0c7440 secure :-1 s=READY pgs=528 cs=0 l=1 rev1=1 crypto rx=0x7f8fe400e7f0 tx=0x7f8fe400ebf0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:30:32.399 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:32.396+0000 7f8fdaffd640 1 -- 172.21.15.37:0/2741914474 <== mon.0 v2:172.21.15.37:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8fe4017070 con 0x7f8fec0b2dc0 2024-04-25T10:30:32.399 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:32.396+0000 7f8fdaffd640 1 -- 172.21.15.37:0/2741914474 <== mon.0 v2:172.21.15.37:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f8fe400ee40 con 0x7f8fec0b2dc0 2024-04-25T10:30:32.399 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:32.396+0000 7f8ff0ab1640 1 -- 172.21.15.37:0/2741914474 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f8fec15e6d0 con 0x7f8fec0b2dc0 2024-04-25T10:30:32.400 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:32.396+0000 7f8fdaffd640 1 -- 172.21.15.37:0/2741914474 <== mon.0 v2:172.21.15.37:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8fe400cb80 con 0x7f8fec0b2dc0 2024-04-25T10:30:32.400 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:32.396+0000 7f8ff0ab1640 1 -- 172.21.15.37:0/2741914474 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f8fec0b58f0 con 0x7f8fec0b2dc0 2024-04-25T10:30:32.400 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:32.397+0000 7f8ff0ab1640 1 -- 172.21.15.37:0/2741914474 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f8fec0cfab0 con 0x7f8fec0b2dc0 2024-04-25T10:30:32.400 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:32.397+0000 7f8fdaffd640 1 -- 172.21.15.37:0/2741914474 <== mon.0 v2:172.21.15.37:3300/0 4 ==== mgrmap(e 5) ==== 96782+0+0 (secure 0 0 0) 0x7f8fe401a070 con 0x7f8fec0b2dc0 2024-04-25T10:30:32.400 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:32.398+0000 7f8fdaffd640 1 --2- 172.21.15.37:0/2741914474 >> [v2:172.21.15.203:6832/2418205237,v1:172.21.15.203:6833/2418205237] conn(0x7f8fcc0744b0 0x7f8fcc0769c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:30:32.401 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:32.398+0000 7f8fdaffd640 1 -- 172.21.15.37:0/2741914474 <== mon.0 v2:172.21.15.37:3300/0 5 ==== osd_map(44..44 src has 1..44) ==== 6248+0+0 (secure 0 0 0) 0x7f8fe4095260 con 0x7f8fec0b2dc0 2024-04-25T10:30:32.401 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:32.398+0000 7f8fea59c640 1 --2- 172.21.15.37:0/2741914474 >> [v2:172.21.15.203:6832/2418205237,v1:172.21.15.203:6833/2418205237] conn(0x7f8fcc0744b0 0x7f8fcc0769c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:30:32.401 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:32.399+0000 7f8fea59c640 1 --2- 172.21.15.37:0/2741914474 >> [v2:172.21.15.203:6832/2418205237,v1:172.21.15.203:6833/2418205237] conn(0x7f8fcc0744b0 0x7f8fcc0769c0 secure :-1 s=READY pgs=253 cs=0 l=1 rev1=1 crypto rx=0x7f8fec074cc0 tx=0x7f8fdc00b040 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:30:32.404 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:32.402+0000 7f8fdaffd640 1 -- 172.21.15.37:0/2741914474 <== mon.0 v2:172.21.15.37:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187246 (secure 0 0 0) 0x7f8fe405f180 con 0x7f8fec0b2dc0 2024-04-25T10:30:32.593 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:32.591+0000 7f8ff0ab1640 1 -- 172.21.15.37:0/2741914474 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7f8fec0b0ec0 con 0x7f8fec0b2dc0 2024-04-25T10:30:32.594 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:32.592+0000 7f8fdaffd640 1 -- 172.21.15.37:0/2741914474 <== mon.0 v2:172.21.15.37:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v44) ==== 74+0+18626 (secure 0 0 0) 0x7f8fe4063480 con 0x7f8fec0b2dc0 2024-04-25T10:30:32.595 INFO:teuthology.orchestra.run.smithi037.stdout: 2024-04-25T10:30:32.595 INFO:teuthology.orchestra.run.smithi037.stdout:{"epoch":44,"fsid":"f4acda28-be34-492a-b286-994afd36f61d","created":"2024-04-25T10:28:43.985718+0000","modified":"2024-04-25T10:30:29.873697+0000","last_up_change":"2024-04-25T10:28:54.900654+0000","last_in_change":"2024-04-25T10:28:46.216973+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":6,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-04-25T10:28:56.202153+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"19","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":4,"pool_name":"cephfs_metadata","create_time":"2024-04-25T10:30:10.008132+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"35","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.75,"score_stable":1.75,"optimal_score":1,"raw_score_acting":1.75,"raw_score_stable":1.75,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":5,"pool_name":"cephfs_data","create_time":"2024-04-25T10:30:11.084827+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"43","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"recovery_fs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":6,"pool_name":"recovery_fs_metadata","create_time":"2024-04-25T10:30:26.903737+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"44","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"recovery_fs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"a2f79028-b332-4166-9ebb-a7024a37ea22","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":42,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.37:6816","nonce":2746074720},{"type":"v1","addr":"172.21.15.37:6817","nonce":2746074720}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.37:6818","nonce":2746074720},{"type":"v1","addr":"172.21.15.37:6819","nonce":2746074720}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.37:6822","nonce":2746074720},{"type":"v1","addr":"172.21.15.37:6823","nonce":2746074720}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.37:6820","nonce":2746074720},{"type":"v1","addr":"172.21.15.37:6821","nonce":2746074720}]},"public_addr":"172.21.15.37:6817/2746074720","cluster_addr":"172.21.15.37:6819/2746074720","heartbeat_back_addr":"172.21.15.37:6823/2746074720","heartbeat_front_addr":"172.21.15.37:6821/2746074720","state":["exists","up"]},{"osd":1,"uuid":"973159de-c2ea-4083-aecc-69373d1c1132","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":42,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.37:6800","nonce":1690341662},{"type":"v1","addr":"172.21.15.37:6801","nonce":1690341662}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.37:6802","nonce":1690341662},{"type":"v1","addr":"172.21.15.37:6803","nonce":1690341662}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.37:6806","nonce":1690341662},{"type":"v1","addr":"172.21.15.37:6807","nonce":1690341662}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.37:6804","nonce":1690341662},{"type":"v1","addr":"172.21.15.37:6805","nonce":1690341662}]},"public_addr":"172.21.15.37:6801/1690341662","cluster_addr":"172.21.15.37:6803/1690341662","heartbeat_back_addr":"172.21.15.37:6807/1690341662","heartbeat_front_addr":"172.21.15.37:6805/1690341662","state":["exists","up"]},{"osd":2,"uuid":"4ae7a0e9-cb3f-484c-a75f-f3766b721c47","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":42,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.37:6808","nonce":2754669649},{"type":"v1","addr":"172.21.15.37:6809","nonce":2754669649}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.37:6810","nonce":2754669649},{"type":"v1","addr":"172.21.15.37:6811","nonce":2754669649}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.37:6814","nonce":2754669649},{"type":"v1","addr":"172.21.15.37:6815","nonce":2754669649}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.37:6812","nonce":2754669649},{"type":"v1","addr":"172.21.15.37:6813","nonce":2754669649}]},"public_addr":"172.21.15.37:6809/2754669649","cluster_addr":"172.21.15.37:6811/2754669649","heartbeat_back_addr":"172.21.15.37:6815/2754669649","heartbeat_front_addr":"172.21.15.37:6813/2754669649","state":["exists","up"]},{"osd":3,"uuid":"1ba4a746-1608-47a7-ace2-960a5cfae934","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":42,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.37:6824","nonce":1815290618},{"type":"v1","addr":"172.21.15.37:6825","nonce":1815290618}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.37:6826","nonce":1815290618},{"type":"v1","addr":"172.21.15.37:6827","nonce":1815290618}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.37:6830","nonce":1815290618},{"type":"v1","addr":"172.21.15.37:6831","nonce":1815290618}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.37:6828","nonce":1815290618},{"type":"v1","addr":"172.21.15.37:6829","nonce":1815290618}]},"public_addr":"172.21.15.37:6825/1815290618","cluster_addr":"172.21.15.37:6827/1815290618","heartbeat_back_addr":"172.21.15.37:6831/1815290618","heartbeat_front_addr":"172.21.15.37:6829/1815290618","state":["exists","up"]},{"osd":4,"uuid":"d4f59f5d-0413-4b79-87ee-bb212c1fcf62","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":42,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6800","nonce":4051532063},{"type":"v1","addr":"172.21.15.203:6801","nonce":4051532063}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6802","nonce":4051532063},{"type":"v1","addr":"172.21.15.203:6803","nonce":4051532063}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6806","nonce":4051532063},{"type":"v1","addr":"172.21.15.203:6807","nonce":4051532063}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6804","nonce":4051532063},{"type":"v1","addr":"172.21.15.203:6805","nonce":4051532063}]},"public_addr":"172.21.15.203:6801/4051532063","cluster_addr":"172.21.15.203:6803/4051532063","heartbeat_back_addr":"172.21.15.203:6807/4051532063","heartbeat_front_addr":"172.21.15.203:6805/4051532063","state":["exists","up"]},{"osd":5,"uuid":"69710a0e-41d8-4e7f-9a10-17ce540b67be","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":42,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6824","nonce":1203822890},{"type":"v1","addr":"172.21.15.203:6825","nonce":1203822890}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6826","nonce":1203822890},{"type":"v1","addr":"172.21.15.203:6827","nonce":1203822890}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6830","nonce":1203822890},{"type":"v1","addr":"172.21.15.203:6831","nonce":1203822890}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6828","nonce":1203822890},{"type":"v1","addr":"172.21.15.203:6829","nonce":1203822890}]},"public_addr":"172.21.15.203:6825/1203822890","cluster_addr":"172.21.15.203:6827/1203822890","heartbeat_back_addr":"172.21.15.203:6831/1203822890","heartbeat_front_addr":"172.21.15.203:6829/1203822890","state":["exists","up"]},{"osd":6,"uuid":"f29dd047-3b48-473d-a235-6642a6293f20","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":42,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6810","nonce":2309615014},{"type":"v1","addr":"172.21.15.203:6812","nonce":2309615014}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6814","nonce":2309615014},{"type":"v1","addr":"172.21.15.203:6816","nonce":2309615014}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6822","nonce":2309615014},{"type":"v1","addr":"172.21.15.203:6823","nonce":2309615014}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6818","nonce":2309615014},{"type":"v1","addr":"172.21.15.203:6820","nonce":2309615014}]},"public_addr":"172.21.15.203:6812/2309615014","cluster_addr":"172.21.15.203:6816/2309615014","heartbeat_back_addr":"172.21.15.203:6823/2309615014","heartbeat_front_addr":"172.21.15.203:6820/2309615014","state":["exists","up"]},{"osd":7,"uuid":"6e691644-378b-4fdc-8e86-fb5f56e31f29","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":42,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6808","nonce":3688099014},{"type":"v1","addr":"172.21.15.203:6809","nonce":3688099014}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6811","nonce":3688099014},{"type":"v1","addr":"172.21.15.203:6813","nonce":3688099014}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6819","nonce":3688099014},{"type":"v1","addr":"172.21.15.203:6821","nonce":3688099014}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6815","nonce":3688099014},{"type":"v1","addr":"172.21.15.203:6817","nonce":3688099014}]},"public_addr":"172.21.15.203:6809/3688099014","cluster_addr":"172.21.15.203:6813/3688099014","heartbeat_back_addr":"172.21.15.203:6821/3688099014","heartbeat_front_addr":"172.21.15.203:6817/3688099014","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T10:28:51.419141+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T10:28:51.366448+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T10:28:51.348276+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T10:28:51.380519+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T10:28:51.105812+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T10:28:51.384496+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T10:28:51.388755+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T10:28:51.366958+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.37:6835/311552410":"2024-04-26T10:30:24.853732+0000","172.21.15.37:6832/311552410":"2024-04-26T10:30:24.853732+0000"},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-04-25T10:30:32.596 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:32.594+0000 7f8ff0ab1640 1 -- 172.21.15.37:0/2741914474 >> [v2:172.21.15.203:6832/2418205237,v1:172.21.15.203:6833/2418205237] conn(0x7f8fcc0744b0 msgr2=0x7f8fcc0769c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:30:32.596 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:32.594+0000 7f8ff0ab1640 1 --2- 172.21.15.37:0/2741914474 >> [v2:172.21.15.203:6832/2418205237,v1:172.21.15.203:6833/2418205237] conn(0x7f8fcc0744b0 0x7f8fcc0769c0 secure :-1 s=READY pgs=253 cs=0 l=1 rev1=1 crypto rx=0x7f8fec074cc0 tx=0x7f8fdc00b040 comp rx=0 tx=0).stop 2024-04-25T10:30:32.597 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:32.594+0000 7f8ff0ab1640 1 -- 172.21.15.37:0/2741914474 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f8fec0b2dc0 msgr2=0x7f8fec0c7440 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:30:32.597 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:32.594+0000 7f8ff0ab1640 1 --2- 172.21.15.37:0/2741914474 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f8fec0b2dc0 0x7f8fec0c7440 secure :-1 s=READY pgs=528 cs=0 l=1 rev1=1 crypto rx=0x7f8fe400e7f0 tx=0x7f8fe400ebf0 comp rx=0 tx=0).stop 2024-04-25T10:30:32.597 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:32.594+0000 7f8ff0ab1640 1 -- 172.21.15.37:0/2741914474 shutdown_connections 2024-04-25T10:30:32.597 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:32.594+0000 7f8ff0ab1640 1 --2- 172.21.15.37:0/2741914474 >> [v2:172.21.15.203:6832/2418205237,v1:172.21.15.203:6833/2418205237] conn(0x7f8fcc0744b0 0x7f8fcc0769c0 unknown :-1 s=CLOSED pgs=253 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:30:32.597 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:32.594+0000 7f8ff0ab1640 1 --2- 172.21.15.37:0/2741914474 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f8fec0b2dc0 0x7f8fec0c7440 unknown :-1 s=CLOSED pgs=528 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:30:32.597 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:32.594+0000 7f8ff0ab1640 1 --2- 172.21.15.37:0/2741914474 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f8fec06e440 0x7f8fec15e190 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:30:32.597 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:32.594+0000 7f8ff0ab1640 1 --2- 172.21.15.37:0/2741914474 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f8fec06daf0 0x7f8fec0cf330 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:30:32.597 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:32.594+0000 7f8ff0ab1640 1 -- 172.21.15.37:0/2741914474 >> 172.21.15.37:0/2741914474 conn(0x7f8fec0b5440 msgr2=0x7f8fec0712b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:30:32.597 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:32.594+0000 7f8ff0ab1640 1 -- 172.21.15.37:0/2741914474 shutdown_connections 2024-04-25T10:30:32.597 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:32.594+0000 7f8ff0ab1640 1 -- 172.21.15.37:0/2741914474 wait complete. 2024-04-25T10:30:32.608 INFO:tasks.cephfs.filesystem.ceph_manager:waiting for clean 2024-04-25T10:30:32.608 DEBUG:teuthology.orchestra.run.smithi037:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-04-25T10:30:32.719 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:32.717+0000 7f2d3b6af640 1 Processor -- start 2024-04-25T10:30:32.720 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:32.717+0000 7f2d3b6af640 1 -- start start 2024-04-25T10:30:32.723 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:32.719+0000 7f2d3b6af640 1 --2- >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f2d340d5160 0x7f2d340b4e80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:30:32.723 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:32.719+0000 7f2d3b6af640 1 -- --> v1:172.21.15.37:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f2d340d1370 con 0x7f2d340d0990 2024-04-25T10:30:32.723 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:32.719+0000 7f2d3b6af640 1 -- --> v1:172.21.15.203:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f2d340d4680 con 0x7f2d340d3230 2024-04-25T10:30:32.723 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:32.719+0000 7f2d3b6af640 1 -- --> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] -- mon_getmap magic: 0 -- 0x7f2d340d1650 con 0x7f2d340d5160 2024-04-25T10:30:32.723 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:32.719+0000 7f2d38c4a640 1 --1- >> v1:172.21.15.37:6789/0 conn(0x7f2d340d0990 0x7f2d340d2b20 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.37:6789/0 says I am v1:172.21.15.37:40902/0 (socket says 172.21.15.37:40902) 2024-04-25T10:30:32.723 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:32.719+0000 7f2d38c4a640 1 -- 172.21.15.37:0/3064882241 learned_addr learned my addr 172.21.15.37:0/3064882241 (peer_addr_for_me v1:172.21.15.37:0/0) 2024-04-25T10:30:32.723 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:32.719+0000 7f2d3944b640 1 --2- 172.21.15.37:0/3064882241 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f2d340d5160 0x7f2d340b4e80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:30:32.724 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:32.719+0000 7f2d33fff640 1 -- 172.21.15.37:0/3064882241 <== mon.0 v1:172.21.15.37:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 1288268362 0 0) 0x7f2d340d1370 con 0x7f2d340d0990 2024-04-25T10:30:32.724 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:32.719+0000 7f2d33fff640 1 -- 172.21.15.37:0/3064882241 --> v1:172.21.15.37:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f2d1c003520 con 0x7f2d340d0990 2024-04-25T10:30:32.724 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:32.719+0000 7f2d33fff640 1 -- 172.21.15.37:0/3064882241 <== mon.0 v1:172.21.15.37:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 1689335779 0 0) 0x7f2d1c003520 con 0x7f2d340d0990 2024-04-25T10:30:32.724 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:32.719+0000 7f2d33fff640 1 -- 172.21.15.37:0/3064882241 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f2d340d5160 msgr2=0x7f2d340b4e80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:30:32.724 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:32.719+0000 7f2d33fff640 1 --2- 172.21.15.37:0/3064882241 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f2d340d5160 0x7f2d340b4e80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:30:32.724 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:32.719+0000 7f2d33fff640 1 -- 172.21.15.37:0/3064882241 >> v1:172.21.15.203:6789/0 conn(0x7f2d340d3230 legacy=0x7f2d340b9060 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:30:32.724 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:32.720+0000 7f2d33fff640 1 -- 172.21.15.37:0/3064882241 --> v1:172.21.15.37:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f2d340d4960 con 0x7f2d340d0990 2024-04-25T10:30:32.724 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:32.720+0000 7f2d33fff640 1 -- 172.21.15.37:0/3064882241 <== mon.0 v1:172.21.15.37:6789/0 3 ==== mon_map magic: 0 ==== 413+0+0 (unknown 1577794600 0 0) 0x7f2d28001670 con 0x7f2d340d0990 2024-04-25T10:30:32.724 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:32.720+0000 7f2d33fff640 1 -- 172.21.15.37:0/3064882241 >> v1:172.21.15.37:6789/0 conn(0x7f2d340d0990 legacy=0x7f2d340d2b20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:30:32.724 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:32.720+0000 7f2d33fff640 1 --2- 172.21.15.37:0/3064882241 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f2d1c0039e0 0x7f2d1c003e50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:30:32.724 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:32.720+0000 7f2d33fff640 1 --2- 172.21.15.37:0/3064882241 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f2d1c004800 0x7f2d1c004c10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:30:32.724 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:32.720+0000 7f2d33fff640 1 --2- 172.21.15.37:0/3064882241 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f2d340d5160 0x7f2d1c00b9c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:30:32.724 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:32.720+0000 7f2d33fff640 1 -- 172.21.15.37:0/3064882241 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 -- 0x7f2d340d1650 con 0x7f2d1c0039e0 2024-04-25T10:30:32.724 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:32.720+0000 7f2d33fff640 1 -- 172.21.15.37:0/3064882241 --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_getmap magic: 0 -- 0x7f2d1c005590 con 0x7f2d1c004800 2024-04-25T10:30:32.724 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:32.720+0000 7f2d33fff640 1 -- 172.21.15.37:0/3064882241 --> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] -- mon_getmap magic: 0 -- 0x7f2d1c00bf00 con 0x7f2d340d5160 2024-04-25T10:30:32.725 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:32.720+0000 7f2d3944b640 1 --2- 172.21.15.37:0/3064882241 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f2d1c0039e0 0x7f2d1c003e50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:30:32.725 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:32.720+0000 7f2d38c4a640 1 --2- 172.21.15.37:0/3064882241 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f2d1c004800 0x7f2d1c004c10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:30:32.725 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:32.720+0000 7f2d39c4c640 1 --2- 172.21.15.37:0/3064882241 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f2d340d5160 0x7f2d1c00b9c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:30:32.725 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:32.720+0000 7f2d3944b640 1 -- 172.21.15.37:0/3064882241 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f2d340d5160 msgr2=0x7f2d1c00b9c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:30:32.725 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:32.720+0000 7f2d3944b640 1 --2- 172.21.15.37:0/3064882241 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f2d340d5160 0x7f2d1c00b9c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:30:32.725 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:32.720+0000 7f2d3944b640 1 -- 172.21.15.37:0/3064882241 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f2d1c004800 msgr2=0x7f2d1c004c10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:30:32.725 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:32.720+0000 7f2d3944b640 1 --2- 172.21.15.37:0/3064882241 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f2d1c004800 0x7f2d1c004c10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:30:32.725 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:32.720+0000 7f2d3944b640 1 -- 172.21.15.37:0/3064882241 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f2d1c003520 con 0x7f2d1c0039e0 2024-04-25T10:30:32.725 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:32.721+0000 7f2d3944b640 1 --2- 172.21.15.37:0/3064882241 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f2d1c0039e0 0x7f2d1c003e50 secure :-1 s=READY pgs=530 cs=0 l=1 rev1=1 crypto rx=0x7f2d2c009ac0 tx=0x7f2d1c005830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a1672abe72b75b8 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:30:32.725 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:32.721+0000 7f2d33fff640 1 -- 172.21.15.37:0/3064882241 <== mon.0 v2:172.21.15.37:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2d2c036070 con 0x7f2d1c0039e0 2024-04-25T10:30:32.725 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:32.721+0000 7f2d33fff640 1 -- 172.21.15.37:0/3064882241 <== mon.0 v2:172.21.15.37:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f2d2c031400 con 0x7f2d1c0039e0 2024-04-25T10:30:32.725 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:32.721+0000 7f2d33fff640 1 -- 172.21.15.37:0/3064882241 <== mon.0 v2:172.21.15.37:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2d2c009690 con 0x7f2d1c0039e0 2024-04-25T10:30:32.728 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:32.722+0000 7f2d3b6af640 1 -- 172.21.15.37:0/3064882241 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f2d1c0039e0 msgr2=0x7f2d1c003e50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:30:32.728 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:32.722+0000 7f2d3b6af640 1 --2- 172.21.15.37:0/3064882241 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f2d1c0039e0 0x7f2d1c003e50 secure :-1 s=READY pgs=530 cs=0 l=1 rev1=1 crypto rx=0x7f2d2c009ac0 tx=0x7f2d1c005830 comp rx=0 tx=0).stop 2024-04-25T10:30:32.728 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:32.722+0000 7f2d39c4c640 1 -- 172.21.15.37:0/3064882241 reap_dead start 2024-04-25T10:30:32.728 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:32.722+0000 7f2d3b6af640 1 -- 172.21.15.37:0/3064882241 shutdown_connections 2024-04-25T10:30:32.728 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:32.722+0000 7f2d3b6af640 1 -- 172.21.15.37:0/3064882241 >> 172.21.15.37:0/3064882241 conn(0x7f2d340b7a80 msgr2=0x7f2d340b7e90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:30:32.728 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:32.722+0000 7f2d3b6af640 1 -- 172.21.15.37:0/3064882241 shutdown_connections 2024-04-25T10:30:32.728 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:32.722+0000 7f2d3b6af640 1 -- 172.21.15.37:0/3064882241 wait complete. 2024-04-25T10:30:32.728 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:32.723+0000 7f2d3b6af640 1 Processor -- start 2024-04-25T10:30:32.729 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:32.724+0000 7f2d3b6af640 1 -- start start 2024-04-25T10:30:32.729 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:32.725+0000 7f2d3b6af640 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f2d1c0039e0 0x7f2d340b3990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:30:32.729 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:32.725+0000 7f2d3b6af640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f2d341527d0 0x7f2d340cdc30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:30:32.729 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:32.725+0000 7f2d3b6af640 1 --2- >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f2d340ce170 0x7f2d340b6a80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:30:32.729 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:32.725+0000 7f2d3b6af640 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 -- 0x7f2d34164c40 con 0x7f2d341527d0 2024-04-25T10:30:32.729 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:32.725+0000 7f2d3b6af640 1 -- --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_getmap magic: 0 -- 0x7f2d34164ac0 con 0x7f2d1c0039e0 2024-04-25T10:30:32.729 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:32.725+0000 7f2d3b6af640 1 -- --> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] -- mon_getmap magic: 0 -- 0x7f2d340d0da0 con 0x7f2d340ce170 2024-04-25T10:30:32.729 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:32.725+0000 7f2d38c4a640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f2d341527d0 0x7f2d340cdc30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:30:32.729 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:32.725+0000 7f2d38c4a640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f2d341527d0 0x7f2d340cdc30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.37:3300/0 says I am v2:172.21.15.37:55414/0 (socket says 172.21.15.37:55414) 2024-04-25T10:30:32.729 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:32.725+0000 7f2d38c4a640 1 -- 172.21.15.37:0/2385758388 learned_addr learned my addr 172.21.15.37:0/2385758388 (peer_addr_for_me v2:172.21.15.37:0/0) 2024-04-25T10:30:32.729 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:32.725+0000 7f2d39c4c640 1 --2- 172.21.15.37:0/2385758388 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f2d340ce170 0x7f2d340b6a80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:30:32.729 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:32.725+0000 7f2d3944b640 1 --2- 172.21.15.37:0/2385758388 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f2d1c0039e0 0x7f2d340b3990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:30:32.729 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:32.725+0000 7f2d38c4a640 1 -- 172.21.15.37:0/2385758388 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f2d340ce170 msgr2=0x7f2d340b6a80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:30:32.730 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:32.725+0000 7f2d38c4a640 1 --2- 172.21.15.37:0/2385758388 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f2d340ce170 0x7f2d340b6a80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:30:32.730 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:32.725+0000 7f2d38c4a640 1 -- 172.21.15.37:0/2385758388 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f2d1c0039e0 msgr2=0x7f2d340b3990 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:30:32.730 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:32.725+0000 7f2d38c4a640 1 --2- 172.21.15.37:0/2385758388 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f2d1c0039e0 0x7f2d340b3990 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:30:32.730 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:32.725+0000 7f2d38c4a640 1 -- 172.21.15.37:0/2385758388 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f2d340ce8d0 con 0x7f2d341527d0 2024-04-25T10:30:32.730 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:32.725+0000 7f2d38c4a640 1 --2- 172.21.15.37:0/2385758388 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f2d341527d0 0x7f2d340cdc30 secure :-1 s=READY pgs=531 cs=0 l=1 rev1=1 crypto rx=0x7f2d280017e0 tx=0x7f2d28007430 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:30:32.730 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:32.726+0000 7f2d31ffb640 1 -- 172.21.15.37:0/2385758388 <== mon.0 v2:172.21.15.37:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2d2800f720 con 0x7f2d341527d0 2024-04-25T10:30:32.730 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:32.726+0000 7f2d31ffb640 1 -- 172.21.15.37:0/2385758388 <== mon.0 v2:172.21.15.37:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f2d28018070 con 0x7f2d341527d0 2024-04-25T10:30:32.730 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:32.726+0000 7f2d31ffb640 1 -- 172.21.15.37:0/2385758388 <== mon.0 v2:172.21.15.37:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2d2800fb00 con 0x7f2d341527d0 2024-04-25T10:30:32.730 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:32.726+0000 7f2d3b6af640 1 -- 172.21.15.37:0/2385758388 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f2d340cebb0 con 0x7f2d341527d0 2024-04-25T10:30:32.730 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:32.726+0000 7f2d3b6af640 1 -- 172.21.15.37:0/2385758388 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f2d340cf170 con 0x7f2d341527d0 2024-04-25T10:30:32.731 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:32.727+0000 7f2d3b6af640 1 -- 172.21.15.37:0/2385758388 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f2d3406a140 con 0x7f2d341527d0 2024-04-25T10:30:32.731 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:32.728+0000 7f2d31ffb640 1 -- 172.21.15.37:0/2385758388 <== mon.0 v2:172.21.15.37:3300/0 4 ==== mgrmap(e 5) ==== 96782+0+0 (secure 0 0 0) 0x7f2d2800fca0 con 0x7f2d341527d0 2024-04-25T10:30:32.731 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:32.728+0000 7f2d31ffb640 1 --2- 172.21.15.37:0/2385758388 >> [v2:172.21.15.203:6832/2418205237,v1:172.21.15.203:6833/2418205237] conn(0x7f2d200744e0 0x7f2d200769f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:30:32.731 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:32.728+0000 7f2d31ffb640 1 -- 172.21.15.37:0/2385758388 <== mon.0 v2:172.21.15.37:3300/0 5 ==== osd_map(44..44 src has 1..44) ==== 6248+0+0 (secure 0 0 0) 0x7f2d28095220 con 0x7f2d341527d0 2024-04-25T10:30:32.731 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:32.728+0000 7f2d3944b640 1 --2- 172.21.15.37:0/2385758388 >> [v2:172.21.15.203:6832/2418205237,v1:172.21.15.203:6833/2418205237] conn(0x7f2d200744e0 0x7f2d200769f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:30:32.731 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:32.729+0000 7f2d3944b640 1 --2- 172.21.15.37:0/2385758388 >> [v2:172.21.15.203:6832/2418205237,v1:172.21.15.203:6833/2418205237] conn(0x7f2d200744e0 0x7f2d200769f0 secure :-1 s=READY pgs=254 cs=0 l=1 rev1=1 crypto rx=0x7f2d2c033000 tx=0x7f2d2c008030 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:30:32.735 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:32.733+0000 7f2d31ffb640 1 -- 172.21.15.37:0/2385758388 <== mon.0 v2:172.21.15.37:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187246 (secure 0 0 0) 0x7f2d2805f140 con 0x7f2d341527d0 2024-04-25T10:30:32.920 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:32.918+0000 7f2d3b6af640 1 -- 172.21.15.37:0/2385758388 --> [v2:172.21.15.203:6832/2418205237,v1:172.21.15.203:6833/2418205237] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) -- 0x7f2d341634e0 con 0x7f2d200744e0 2024-04-25T10:30:32.928 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:32.926+0000 7f2d31ffb640 1 -- 172.21.15.37:0/2385758388 <== mgr.4100 v2:172.21.15.203:6832/2418205237 1 ==== mgr_command_reply(tid 0: 0 dumped all) ==== 18+0+332031 (secure 0 0 0) 0x7f2d341634e0 con 0x7f2d200744e0 2024-04-25T10:30:32.932 INFO:teuthology.orchestra.run.smithi037.stdout: 2024-04-25T10:30:32.934 INFO:teuthology.orchestra.run.smithi037.stderr:dumped all 2024-04-25T10:30:32.936 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:32.930+0000 7f2d3b6af640 1 -- 172.21.15.37:0/2385758388 >> [v2:172.21.15.203:6832/2418205237,v1:172.21.15.203:6833/2418205237] conn(0x7f2d200744e0 msgr2=0x7f2d200769f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:30:32.936 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:32.930+0000 7f2d3b6af640 1 --2- 172.21.15.37:0/2385758388 >> [v2:172.21.15.203:6832/2418205237,v1:172.21.15.203:6833/2418205237] conn(0x7f2d200744e0 0x7f2d200769f0 secure :-1 s=READY pgs=254 cs=0 l=1 rev1=1 crypto rx=0x7f2d2c033000 tx=0x7f2d2c008030 comp rx=0 tx=0).stop 2024-04-25T10:30:32.936 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:32.930+0000 7f2d3b6af640 1 -- 172.21.15.37:0/2385758388 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f2d341527d0 msgr2=0x7f2d340cdc30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:30:32.936 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:32.930+0000 7f2d3b6af640 1 --2- 172.21.15.37:0/2385758388 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f2d341527d0 0x7f2d340cdc30 secure :-1 s=READY pgs=531 cs=0 l=1 rev1=1 crypto rx=0x7f2d280017e0 tx=0x7f2d28007430 comp rx=0 tx=0).stop 2024-04-25T10:30:32.936 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:32.930+0000 7f2d3b6af640 1 -- 172.21.15.37:0/2385758388 shutdown_connections 2024-04-25T10:30:32.936 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:32.930+0000 7f2d3b6af640 1 --2- 172.21.15.37:0/2385758388 >> [v2:172.21.15.203:6832/2418205237,v1:172.21.15.203:6833/2418205237] conn(0x7f2d200744e0 0x7f2d200769f0 unknown :-1 s=CLOSED pgs=254 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:30:32.937 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:32.930+0000 7f2d3b6af640 1 --2- 172.21.15.37:0/2385758388 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f2d340ce170 0x7f2d340b6a80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:30:32.937 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:32.930+0000 7f2d3b6af640 1 --2- 172.21.15.37:0/2385758388 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f2d341527d0 0x7f2d340cdc30 unknown :-1 s=CLOSED pgs=531 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:30:32.937 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:32.930+0000 7f2d3b6af640 1 --2- 172.21.15.37:0/2385758388 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f2d1c0039e0 0x7f2d340b3990 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:30:32.937 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:32.930+0000 7f2d3b6af640 1 -- 172.21.15.37:0/2385758388 >> 172.21.15.37:0/2385758388 conn(0x7f2d340b7a80 msgr2=0x7f2d340b9fd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:30:32.937 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:32.930+0000 7f2d3b6af640 1 -- 172.21.15.37:0/2385758388 shutdown_connections 2024-04-25T10:30:32.937 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:32.930+0000 7f2d3b6af640 1 -- 172.21.15.37:0/2385758388 wait complete. 2024-04-25T10:30:32.943 INFO:teuthology.orchestra.run.smithi037.stdout:{"pg_ready":true,"pg_map":{"version":87,"stamp":"2024-04-25T10:30:32.207732+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":6881824,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":67,"num_read_kb":55,"num_write":201,"num_write_kb":7812,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":659,"ondisk_log_size":659,"up":258,"acting":258,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":197,"num_osds":8,"num_per_pool_osds":8,"num_per_pool_omap_osds":8,"kb":754974720,"kb_used":233448,"kb_used_data":17232,"kb_used_omap":12,"kb_used_meta":216051,"kb_avail":754741272,"statfs":{"total":773094113280,"available":772855062528,"internally_reserved":0,"allocated":17645568,"data_stored":15354143,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":12752,"internal_metadata":221236784},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":14,"apply_latency_ms":14,"commit_latency_ns":14000000,"apply_latency_ns":14000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":-16365,"num_objects":-23,"num_object_clones":0,"num_object_copies":-46,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":-23,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":23,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":-15,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"6.001255"},"pg_stats":[{"pgid":"1.0","version":"18'72","reported_seq":145,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-04-25T10:30:28.875202+0000","last_change":"2024-04-25T10:28:57.972696+0000","last_active":"2024-04-25T10:30:28.875202+0000","last_peered":"2024-04-25T10:30:28.875202+0000","last_clean":"2024-04-25T10:30:28.875202+0000","last_became_active":"2024-04-25T10:28:57.972373+0000","last_became_peered":"2024-04-25T10:28:57.972373+0000","last_unstale":"2024-04-25T10:30:28.875202+0000","last_undegraded":"2024-04-25T10:30:28.875202+0000","last_fullsized":"2024-04-25T10:30:28.875202+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:28:56.959439+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:28:56.959439+0000","last_clean_scrub_stamp":"2024-04-25T10:28:56.959439+0000","objects_scrubbed":0,"log_size":72,"log_dups_size":0,"ondisk_log_size":72,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T17:34:34.057408+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":66,"num_read_kb":55,"num_write":101,"num_write_kb":1628,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.5","version":"0'0","reported_seq":30,"reported_epoch":44,"state":"active+clean","last_fresh":"2024-04-25T10:30:29.882532+0000","last_change":"2024-04-25T10:30:11.759459+0000","last_active":"2024-04-25T10:30:29.882532+0000","last_peered":"2024-04-25T10:30:29.882532+0000","last_clean":"2024-04-25T10:30:29.882532+0000","last_became_active":"2024-04-25T10:30:11.757880+0000","last_became_peered":"2024-04-25T10:30:11.757880+0000","last_unstale":"2024-04-25T10:30:29.882532+0000","last_undegraded":"2024-04-25T10:30:29.882532+0000","last_fullsized":"2024-04-25T10:30:29.882532+0000","mapping_epoch":33,"log_start":"0'0","ondisk_log_start":"0'0","created":33,"last_epoch_clean":34,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:30:10.729165+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:30:10.729165+0000","last_clean_scrub_stamp":"2024-04-25T10:30:10.729165+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T18:21:31.173712+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.4","version":"36'9","reported_seq":33,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-04-25T10:30:28.874920+0000","last_change":"2024-04-25T10:30:12.773161+0000","last_active":"2024-04-25T10:30:28.874920+0000","last_peered":"2024-04-25T10:30:28.874920+0000","last_clean":"2024-04-25T10:30:28.874920+0000","last_became_active":"2024-04-25T10:30:12.772831+0000","last_became_peered":"2024-04-25T10:30:12.772831+0000","last_unstale":"2024-04-25T10:30:28.874920+0000","last_undegraded":"2024-04-25T10:30:28.874920+0000","last_fullsized":"2024-04-25T10:30:28.874920+0000","mapping_epoch":34,"log_start":"0'0","ondisk_log_start":"0'0","created":34,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:30:11.733978+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:30:11.733978+0000","last_clean_scrub_stamp":"2024-04-25T10:30:11.733978+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T19:56:57.765496+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"6.7","version":"0'0","reported_seq":12,"reported_epoch":44,"state":"active+clean","last_fresh":"2024-04-25T10:30:29.882905+0000","last_change":"2024-04-25T10:30:28.881711+0000","last_active":"2024-04-25T10:30:29.882905+0000","last_peered":"2024-04-25T10:30:29.882905+0000","last_clean":"2024-04-25T10:30:29.882905+0000","last_became_active":"2024-04-25T10:30:28.881407+0000","last_became_peered":"2024-04-25T10:30:28.881407+0000","last_unstale":"2024-04-25T10:30:29.882905+0000","last_undegraded":"2024-04-25T10:30:29.882905+0000","last_fullsized":"2024-04-25T10:30:29.882905+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":42,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:30:27.860024+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:30:27.860024+0000","last_clean_scrub_stamp":"2024-04-25T10:30:27.860024+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T16:30:29.338622+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.0","version":"0'0","reported_seq":25,"reported_epoch":42,"state":"active+clean","last_fresh":"2024-04-25T10:30:27.875763+0000","last_change":"2024-04-25T10:30:11.761764+0000","last_active":"2024-04-25T10:30:27.875763+0000","last_peered":"2024-04-25T10:30:27.875763+0000","last_clean":"2024-04-25T10:30:27.875763+0000","last_became_active":"2024-04-25T10:30:11.761281+0000","last_became_peered":"2024-04-25T10:30:11.761281+0000","last_unstale":"2024-04-25T10:30:27.875763+0000","last_undegraded":"2024-04-25T10:30:27.875763+0000","last_fullsized":"2024-04-25T10:30:27.875763+0000","mapping_epoch":33,"log_start":"0'0","ondisk_log_start":"0'0","created":33,"last_epoch_clean":34,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:30:10.729165+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:30:10.729165+0000","last_clean_scrub_stamp":"2024-04-25T10:30:10.729165+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T13:05:33.012421+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.1","version":"36'5","reported_seq":34,"reported_epoch":44,"state":"active+clean","last_fresh":"2024-04-25T10:30:29.881926+0000","last_change":"2024-04-25T10:30:12.769006+0000","last_active":"2024-04-25T10:30:29.881926+0000","last_peered":"2024-04-25T10:30:29.881926+0000","last_clean":"2024-04-25T10:30:29.881926+0000","last_became_active":"2024-04-25T10:30:12.768671+0000","last_became_peered":"2024-04-25T10:30:12.768671+0000","last_unstale":"2024-04-25T10:30:29.881926+0000","last_undegraded":"2024-04-25T10:30:29.881926+0000","last_fullsized":"2024-04-25T10:30:29.881926+0000","mapping_epoch":34,"log_start":"0'0","ondisk_log_start":"0'0","created":34,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:30:11.733978+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:30:11.733978+0000","last_clean_scrub_stamp":"2024-04-25T10:30:11.733978+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T15:00:26.818072+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"6.2","version":"0'0","reported_seq":12,"reported_epoch":44,"state":"active+clean","last_fresh":"2024-04-25T10:30:29.882577+0000","last_change":"2024-04-25T10:30:28.875554+0000","last_active":"2024-04-25T10:30:29.882577+0000","last_peered":"2024-04-25T10:30:29.882577+0000","last_clean":"2024-04-25T10:30:29.882577+0000","last_became_active":"2024-04-25T10:30:28.875249+0000","last_became_peered":"2024-04-25T10:30:28.875249+0000","last_unstale":"2024-04-25T10:30:29.882577+0000","last_undegraded":"2024-04-25T10:30:29.882577+0000","last_fullsized":"2024-04-25T10:30:29.882577+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":42,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:30:27.860024+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:30:27.860024+0000","last_clean_scrub_stamp":"2024-04-25T10:30:27.860024+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T17:20:57.234773+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.1","version":"0'0","reported_seq":30,"reported_epoch":44,"state":"active+clean","last_fresh":"2024-04-25T10:30:29.882995+0000","last_change":"2024-04-25T10:30:11.765156+0000","last_active":"2024-04-25T10:30:29.882995+0000","last_peered":"2024-04-25T10:30:29.882995+0000","last_clean":"2024-04-25T10:30:29.882995+0000","last_became_active":"2024-04-25T10:30:11.764704+0000","last_became_peered":"2024-04-25T10:30:11.764704+0000","last_unstale":"2024-04-25T10:30:29.882995+0000","last_undegraded":"2024-04-25T10:30:29.882995+0000","last_fullsized":"2024-04-25T10:30:29.882995+0000","mapping_epoch":33,"log_start":"0'0","ondisk_log_start":"0'0","created":33,"last_epoch_clean":34,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:30:10.729165+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:30:10.729165+0000","last_clean_scrub_stamp":"2024-04-25T10:30:10.729165+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T19:02:53.584992+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"5.0","version":"36'8","reported_seq":31,"reported_epoch":42,"state":"active+clean","last_fresh":"2024-04-25T10:30:27.875633+0000","last_change":"2024-04-25T10:30:12.781642+0000","last_active":"2024-04-25T10:30:27.875633+0000","last_peered":"2024-04-25T10:30:27.875633+0000","last_clean":"2024-04-25T10:30:27.875633+0000","last_became_active":"2024-04-25T10:30:12.781303+0000","last_became_peered":"2024-04-25T10:30:12.781303+0000","last_unstale":"2024-04-25T10:30:27.875633+0000","last_undegraded":"2024-04-25T10:30:27.875633+0000","last_fullsized":"2024-04-25T10:30:27.875633+0000","mapping_epoch":34,"log_start":"0'0","ondisk_log_start":"0'0","created":34,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:30:11.733978+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:30:11.733978+0000","last_clean_scrub_stamp":"2024-04-25T10:30:11.733978+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T12:21:06.638842+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"6.3","version":"0'0","reported_seq":9,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-04-25T10:30:28.885536+0000","last_change":"2024-04-25T10:30:28.883569+0000","last_active":"2024-04-25T10:30:28.885536+0000","last_peered":"2024-04-25T10:30:28.885536+0000","last_clean":"2024-04-25T10:30:28.885536+0000","last_became_active":"2024-04-25T10:30:28.883246+0000","last_became_peered":"2024-04-25T10:30:28.883246+0000","last_unstale":"2024-04-25T10:30:28.885536+0000","last_undegraded":"2024-04-25T10:30:28.885536+0000","last_fullsized":"2024-04-25T10:30:28.885536+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":42,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:30:27.860024+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:30:27.860024+0000","last_clean_scrub_stamp":"2024-04-25T10:30:27.860024+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T14:04:58.646968+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.2","version":"0'0","reported_seq":28,"reported_epoch":44,"state":"active+clean","last_fresh":"2024-04-25T10:30:29.882904+0000","last_change":"2024-04-25T10:30:11.760769+0000","last_active":"2024-04-25T10:30:29.882904+0000","last_peered":"2024-04-25T10:30:29.882904+0000","last_clean":"2024-04-25T10:30:29.882904+0000","last_became_active":"2024-04-25T10:30:11.760425+0000","last_became_peered":"2024-04-25T10:30:11.760425+0000","last_unstale":"2024-04-25T10:30:29.882904+0000","last_undegraded":"2024-04-25T10:30:29.882904+0000","last_fullsized":"2024-04-25T10:30:29.882904+0000","mapping_epoch":33,"log_start":"0'0","ondisk_log_start":"0'0","created":33,"last_epoch_clean":34,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:30:10.729165+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:30:10.729165+0000","last_clean_scrub_stamp":"2024-04-25T10:30:10.729165+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T17:24:13.848319+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"5.3","version":"36'7","reported_seq":30,"reported_epoch":42,"state":"active+clean","last_fresh":"2024-04-25T10:30:27.876884+0000","last_change":"2024-04-25T10:30:12.775959+0000","last_active":"2024-04-25T10:30:27.876884+0000","last_peered":"2024-04-25T10:30:27.876884+0000","last_clean":"2024-04-25T10:30:27.876884+0000","last_became_active":"2024-04-25T10:30:12.774532+0000","last_became_peered":"2024-04-25T10:30:12.774532+0000","last_unstale":"2024-04-25T10:30:27.876884+0000","last_undegraded":"2024-04-25T10:30:27.876884+0000","last_fullsized":"2024-04-25T10:30:27.876884+0000","mapping_epoch":34,"log_start":"0'0","ondisk_log_start":"0'0","created":34,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:30:11.733978+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:30:11.733978+0000","last_clean_scrub_stamp":"2024-04-25T10:30:11.733978+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T11:38:09.717996+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"6.0","version":"0'0","reported_seq":9,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-04-25T10:30:28.884563+0000","last_change":"2024-04-25T10:30:28.883801+0000","last_active":"2024-04-25T10:30:28.884563+0000","last_peered":"2024-04-25T10:30:28.884563+0000","last_clean":"2024-04-25T10:30:28.884563+0000","last_became_active":"2024-04-25T10:30:28.883485+0000","last_became_peered":"2024-04-25T10:30:28.883485+0000","last_unstale":"2024-04-25T10:30:28.884563+0000","last_undegraded":"2024-04-25T10:30:28.884563+0000","last_fullsized":"2024-04-25T10:30:28.884563+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":42,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:30:27.860024+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:30:27.860024+0000","last_clean_scrub_stamp":"2024-04-25T10:30:27.860024+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T14:16:58.845516+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.3","version":"0'0","reported_seq":28,"reported_epoch":44,"state":"active+clean","last_fresh":"2024-04-25T10:30:29.883455+0000","last_change":"2024-04-25T10:30:11.762286+0000","last_active":"2024-04-25T10:30:29.883455+0000","last_peered":"2024-04-25T10:30:29.883455+0000","last_clean":"2024-04-25T10:30:29.883455+0000","last_became_active":"2024-04-25T10:30:11.761975+0000","last_became_peered":"2024-04-25T10:30:11.761975+0000","last_unstale":"2024-04-25T10:30:29.883455+0000","last_undegraded":"2024-04-25T10:30:29.883455+0000","last_fullsized":"2024-04-25T10:30:29.883455+0000","mapping_epoch":33,"log_start":"0'0","ondisk_log_start":"0'0","created":33,"last_epoch_clean":34,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:30:10.729165+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:30:10.729165+0000","last_clean_scrub_stamp":"2024-04-25T10:30:10.729165+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T16:09:04.081750+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"5.2","version":"36'8","reported_seq":32,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-04-25T10:30:28.877071+0000","last_change":"2024-04-25T10:30:12.765903+0000","last_active":"2024-04-25T10:30:28.877071+0000","last_peered":"2024-04-25T10:30:28.877071+0000","last_clean":"2024-04-25T10:30:28.877071+0000","last_became_active":"2024-04-25T10:30:12.765611+0000","last_became_peered":"2024-04-25T10:30:12.765611+0000","last_unstale":"2024-04-25T10:30:28.877071+0000","last_undegraded":"2024-04-25T10:30:28.877071+0000","last_fullsized":"2024-04-25T10:30:28.877071+0000","mapping_epoch":34,"log_start":"0'0","ondisk_log_start":"0'0","created":34,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:30:11.733978+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:30:11.733978+0000","last_clean_scrub_stamp":"2024-04-25T10:30:11.733978+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T22:09:01.889527+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"6.1","version":"0'0","reported_seq":12,"reported_epoch":44,"state":"active+clean","last_fresh":"2024-04-25T10:30:29.882707+0000","last_change":"2024-04-25T10:30:28.884434+0000","last_active":"2024-04-25T10:30:29.882707+0000","last_peered":"2024-04-25T10:30:29.882707+0000","last_clean":"2024-04-25T10:30:29.882707+0000","last_became_active":"2024-04-25T10:30:28.883801+0000","last_became_peered":"2024-04-25T10:30:28.883801+0000","last_unstale":"2024-04-25T10:30:29.882707+0000","last_undegraded":"2024-04-25T10:30:29.882707+0000","last_fullsized":"2024-04-25T10:30:29.882707+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":42,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:30:27.860024+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:30:27.860024+0000","last_clean_scrub_stamp":"2024-04-25T10:30:27.860024+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T11:57:46.105302+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.4","version":"0'0","reported_seq":28,"reported_epoch":44,"state":"active+clean","last_fresh":"2024-04-25T10:30:29.883736+0000","last_change":"2024-04-25T10:30:11.757601+0000","last_active":"2024-04-25T10:30:29.883736+0000","last_peered":"2024-04-25T10:30:29.883736+0000","last_clean":"2024-04-25T10:30:29.883736+0000","last_became_active":"2024-04-25T10:30:11.756626+0000","last_became_peered":"2024-04-25T10:30:11.756626+0000","last_unstale":"2024-04-25T10:30:29.883736+0000","last_undegraded":"2024-04-25T10:30:29.883736+0000","last_fullsized":"2024-04-25T10:30:29.883736+0000","mapping_epoch":33,"log_start":"0'0","ondisk_log_start":"0'0","created":33,"last_epoch_clean":34,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:30:10.729165+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:30:10.729165+0000","last_clean_scrub_stamp":"2024-04-25T10:30:10.729165+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T13:03:08.676049+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"5.5","version":"36'5","reported_seq":29,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-04-25T10:30:28.875752+0000","last_change":"2024-04-25T10:30:12.782521+0000","last_active":"2024-04-25T10:30:28.875752+0000","last_peered":"2024-04-25T10:30:28.875752+0000","last_clean":"2024-04-25T10:30:28.875752+0000","last_became_active":"2024-04-25T10:30:12.781802+0000","last_became_peered":"2024-04-25T10:30:12.781802+0000","last_unstale":"2024-04-25T10:30:28.875752+0000","last_undegraded":"2024-04-25T10:30:28.875752+0000","last_fullsized":"2024-04-25T10:30:28.875752+0000","mapping_epoch":34,"log_start":"0'0","ondisk_log_start":"0'0","created":34,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:30:11.733978+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:30:11.733978+0000","last_clean_scrub_stamp":"2024-04-25T10:30:11.733978+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T15:17:30.002665+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"6.6","version":"0'0","reported_seq":9,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-04-25T10:30:28.886860+0000","last_change":"2024-04-25T10:30:28.885896+0000","last_active":"2024-04-25T10:30:28.886860+0000","last_peered":"2024-04-25T10:30:28.886860+0000","last_clean":"2024-04-25T10:30:28.886860+0000","last_became_active":"2024-04-25T10:30:28.885575+0000","last_became_peered":"2024-04-25T10:30:28.885575+0000","last_unstale":"2024-04-25T10:30:28.886860+0000","last_undegraded":"2024-04-25T10:30:28.886860+0000","last_fullsized":"2024-04-25T10:30:28.886860+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":42,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:30:27.860024+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:30:27.860024+0000","last_clean_scrub_stamp":"2024-04-25T10:30:27.860024+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T22:09:18.015574+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.6","version":"37'4","reported_seq":29,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-04-25T10:30:28.879222+0000","last_change":"2024-04-25T10:30:11.750312+0000","last_active":"2024-04-25T10:30:28.879222+0000","last_peered":"2024-04-25T10:30:28.879222+0000","last_clean":"2024-04-25T10:30:28.879222+0000","last_became_active":"2024-04-25T10:30:11.750024+0000","last_became_peered":"2024-04-25T10:30:11.750024+0000","last_unstale":"2024-04-25T10:30:28.879222+0000","last_undegraded":"2024-04-25T10:30:28.879222+0000","last_fullsized":"2024-04-25T10:30:28.879222+0000","mapping_epoch":33,"log_start":"0'0","ondisk_log_start":"0'0","created":33,"last_epoch_clean":34,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:30:10.729165+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:30:10.729165+0000","last_clean_scrub_stamp":"2024-04-25T10:30:10.729165+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T16:48:59.263610+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"5.7","version":"36'8","reported_seq":37,"reported_epoch":44,"state":"active+clean","last_fresh":"2024-04-25T10:30:29.882715+0000","last_change":"2024-04-25T10:30:12.768932+0000","last_active":"2024-04-25T10:30:29.882715+0000","last_peered":"2024-04-25T10:30:29.882715+0000","last_clean":"2024-04-25T10:30:29.882715+0000","last_became_active":"2024-04-25T10:30:12.768609+0000","last_became_peered":"2024-04-25T10:30:12.768609+0000","last_unstale":"2024-04-25T10:30:29.882715+0000","last_undegraded":"2024-04-25T10:30:29.882715+0000","last_fullsized":"2024-04-25T10:30:29.882715+0000","mapping_epoch":34,"log_start":"0'0","ondisk_log_start":"0'0","created":34,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:30:11.733978+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:30:11.733978+0000","last_clean_scrub_stamp":"2024-04-25T10:30:11.733978+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T11:24:05.006967+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":4194304,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":4096,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"6.4","version":"0'0","reported_seq":9,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-04-25T10:30:28.881060+0000","last_change":"2024-04-25T10:30:28.880128+0000","last_active":"2024-04-25T10:30:28.881060+0000","last_peered":"2024-04-25T10:30:28.881060+0000","last_clean":"2024-04-25T10:30:28.881060+0000","last_became_active":"2024-04-25T10:30:28.879771+0000","last_became_peered":"2024-04-25T10:30:28.879771+0000","last_unstale":"2024-04-25T10:30:28.881060+0000","last_undegraded":"2024-04-25T10:30:28.881060+0000","last_fullsized":"2024-04-25T10:30:28.881060+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":42,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:30:27.860024+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:30:27.860024+0000","last_clean_scrub_stamp":"2024-04-25T10:30:27.860024+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T11:33:01.743970+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.7","version":"37'3","reported_seq":31,"reported_epoch":44,"state":"active+clean","last_fresh":"2024-04-25T10:30:29.883302+0000","last_change":"2024-04-25T10:30:11.756948+0000","last_active":"2024-04-25T10:30:29.883302+0000","last_peered":"2024-04-25T10:30:29.883302+0000","last_clean":"2024-04-25T10:30:29.883302+0000","last_became_active":"2024-04-25T10:30:11.756607+0000","last_became_peered":"2024-04-25T10:30:11.756607+0000","last_unstale":"2024-04-25T10:30:29.883302+0000","last_undegraded":"2024-04-25T10:30:29.883302+0000","last_fullsized":"2024-04-25T10:30:29.883302+0000","mapping_epoch":33,"log_start":"0'0","ondisk_log_start":"0'0","created":33,"last_epoch_clean":34,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:30:10.729165+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:30:10.729165+0000","last_clean_scrub_stamp":"2024-04-25T10:30:10.729165+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T17:15:13.597075+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"5.6","version":"36'7","reported_seq":31,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-04-25T10:30:28.876259+0000","last_change":"2024-04-25T10:30:12.772456+0000","last_active":"2024-04-25T10:30:28.876259+0000","last_peered":"2024-04-25T10:30:28.876259+0000","last_clean":"2024-04-25T10:30:28.876259+0000","last_became_active":"2024-04-25T10:30:12.772070+0000","last_became_peered":"2024-04-25T10:30:12.772070+0000","last_unstale":"2024-04-25T10:30:28.876259+0000","last_undegraded":"2024-04-25T10:30:28.876259+0000","last_fullsized":"2024-04-25T10:30:28.876259+0000","mapping_epoch":34,"log_start":"0'0","ondisk_log_start":"0'0","created":34,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:30:11.733978+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:30:11.733978+0000","last_clean_scrub_stamp":"2024-04-25T10:30:11.733978+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T16:08:29.987819+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"6.5","version":"0'0","reported_seq":9,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-04-25T10:30:28.882547+0000","last_change":"2024-04-25T10:30:28.881260+0000","last_active":"2024-04-25T10:30:28.882547+0000","last_peered":"2024-04-25T10:30:28.882547+0000","last_clean":"2024-04-25T10:30:28.882547+0000","last_became_active":"2024-04-25T10:30:28.880995+0000","last_became_peered":"2024-04-25T10:30:28.880995+0000","last_unstale":"2024-04-25T10:30:28.882547+0000","last_undegraded":"2024-04-25T10:30:28.882547+0000","last_fullsized":"2024-04-25T10:30:28.882547+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":42,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:30:27.860024+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:30:27.860024+0000","last_clean_scrub_stamp":"2024-04-25T10:30:27.860024+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T15:39:48.016111+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.8","version":"0'0","reported_seq":30,"reported_epoch":44,"state":"active+clean","last_fresh":"2024-04-25T10:30:29.881839+0000","last_change":"2024-04-25T10:30:11.758581+0000","last_active":"2024-04-25T10:30:29.881839+0000","last_peered":"2024-04-25T10:30:29.881839+0000","last_clean":"2024-04-25T10:30:29.881839+0000","last_became_active":"2024-04-25T10:30:11.758239+0000","last_became_peered":"2024-04-25T10:30:11.758239+0000","last_unstale":"2024-04-25T10:30:29.881839+0000","last_undegraded":"2024-04-25T10:30:29.881839+0000","last_fullsized":"2024-04-25T10:30:29.881839+0000","mapping_epoch":33,"log_start":"0'0","ondisk_log_start":"0'0","created":33,"last_epoch_clean":34,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:30:10.729165+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:30:10.729165+0000","last_clean_scrub_stamp":"2024-04-25T10:30:10.729165+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T17:41:33.905382+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.9","version":"36'2","reported_seq":27,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-04-25T10:30:28.874577+0000","last_change":"2024-04-25T10:30:12.768782+0000","last_active":"2024-04-25T10:30:28.874577+0000","last_peered":"2024-04-25T10:30:28.874577+0000","last_clean":"2024-04-25T10:30:28.874577+0000","last_became_active":"2024-04-25T10:30:12.768464+0000","last_became_peered":"2024-04-25T10:30:12.768464+0000","last_unstale":"2024-04-25T10:30:28.874577+0000","last_undegraded":"2024-04-25T10:30:28.874577+0000","last_fullsized":"2024-04-25T10:30:28.874577+0000","mapping_epoch":34,"log_start":"0'0","ondisk_log_start":"0'0","created":34,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:30:11.733978+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:30:11.733978+0000","last_clean_scrub_stamp":"2024-04-25T10:30:11.733978+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T13:03:21.170197+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"6.a","version":"0'0","reported_seq":9,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-04-25T10:30:28.884130+0000","last_change":"2024-04-25T10:30:28.882147+0000","last_active":"2024-04-25T10:30:28.884130+0000","last_peered":"2024-04-25T10:30:28.884130+0000","last_clean":"2024-04-25T10:30:28.884130+0000","last_became_active":"2024-04-25T10:30:28.881783+0000","last_became_peered":"2024-04-25T10:30:28.881783+0000","last_unstale":"2024-04-25T10:30:28.884130+0000","last_undegraded":"2024-04-25T10:30:28.884130+0000","last_fullsized":"2024-04-25T10:30:28.884130+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":42,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:30:27.860024+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:30:27.860024+0000","last_clean_scrub_stamp":"2024-04-25T10:30:27.860024+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T18:36:07.211600+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.9","version":"0'0","reported_seq":26,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-04-25T10:30:28.874568+0000","last_change":"2024-04-25T10:30:11.759807+0000","last_active":"2024-04-25T10:30:28.874568+0000","last_peered":"2024-04-25T10:30:28.874568+0000","last_clean":"2024-04-25T10:30:28.874568+0000","last_became_active":"2024-04-25T10:30:11.758558+0000","last_became_peered":"2024-04-25T10:30:11.758558+0000","last_unstale":"2024-04-25T10:30:28.874568+0000","last_undegraded":"2024-04-25T10:30:28.874568+0000","last_fullsized":"2024-04-25T10:30:28.874568+0000","mapping_epoch":33,"log_start":"0'0","ondisk_log_start":"0'0","created":33,"last_epoch_clean":34,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:30:10.729165+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:30:10.729165+0000","last_clean_scrub_stamp":"2024-04-25T10:30:10.729165+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T20:38:23.460836+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"5.8","version":"36'9","reported_seq":36,"reported_epoch":44,"state":"active+clean","last_fresh":"2024-04-25T10:30:29.882627+0000","last_change":"2024-04-25T10:30:12.777060+0000","last_active":"2024-04-25T10:30:29.882627+0000","last_peered":"2024-04-25T10:30:29.882627+0000","last_clean":"2024-04-25T10:30:29.882627+0000","last_became_active":"2024-04-25T10:30:12.776749+0000","last_became_peered":"2024-04-25T10:30:12.776749+0000","last_unstale":"2024-04-25T10:30:29.882627+0000","last_undegraded":"2024-04-25T10:30:29.882627+0000","last_fullsized":"2024-04-25T10:30:29.882627+0000","mapping_epoch":34,"log_start":"0'0","ondisk_log_start":"0'0","created":34,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:30:11.733978+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:30:11.733978+0000","last_clean_scrub_stamp":"2024-04-25T10:30:11.733978+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T15:47:40.336983+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"6.b","version":"0'0","reported_seq":1,"reported_epoch":42,"state":"creating+peering","last_fresh":"2024-04-25T10:30:27.876378+0000","last_change":"2024-04-25T10:30:27.876378+0000","last_active":"2024-04-25T10:30:27.860024+0000","last_peered":"2024-04-25T10:30:27.860024+0000","last_clean":"2024-04-25T10:30:27.860024+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T10:30:27.876378+0000","last_undegraded":"2024-04-25T10:30:27.876378+0000","last_fullsized":"2024-04-25T10:30:27.876378+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":42,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:30:27.860024+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:30:27.860024+0000","last_clean_scrub_stamp":"2024-04-25T10:30:27.860024+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.a","version":"0'0","reported_seq":30,"reported_epoch":44,"state":"active+clean","last_fresh":"2024-04-25T10:30:29.883112+0000","last_change":"2024-04-25T10:30:11.762745+0000","last_active":"2024-04-25T10:30:29.883112+0000","last_peered":"2024-04-25T10:30:29.883112+0000","last_clean":"2024-04-25T10:30:29.883112+0000","last_became_active":"2024-04-25T10:30:11.762377+0000","last_became_peered":"2024-04-25T10:30:11.762377+0000","last_unstale":"2024-04-25T10:30:29.883112+0000","last_undegraded":"2024-04-25T10:30:29.883112+0000","last_fullsized":"2024-04-25T10:30:29.883112+0000","mapping_epoch":33,"log_start":"0'0","ondisk_log_start":"0'0","created":33,"last_epoch_clean":34,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:30:10.729165+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:30:10.729165+0000","last_clean_scrub_stamp":"2024-04-25T10:30:10.729165+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T17:15:13.597075+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"5.b","version":"36'8","reported_seq":32,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-04-25T10:30:28.875847+0000","last_change":"2024-04-25T10:30:12.766544+0000","last_active":"2024-04-25T10:30:28.875847+0000","last_peered":"2024-04-25T10:30:28.875847+0000","last_clean":"2024-04-25T10:30:28.875847+0000","last_became_active":"2024-04-25T10:30:12.766228+0000","last_became_peered":"2024-04-25T10:30:12.766228+0000","last_unstale":"2024-04-25T10:30:28.875847+0000","last_undegraded":"2024-04-25T10:30:28.875847+0000","last_fullsized":"2024-04-25T10:30:28.875847+0000","mapping_epoch":34,"log_start":"0'0","ondisk_log_start":"0'0","created":34,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:30:11.733978+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:30:11.733978+0000","last_clean_scrub_stamp":"2024-04-25T10:30:11.733978+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T18:00:26.511360+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"6.8","version":"0'0","reported_seq":12,"reported_epoch":44,"state":"active+clean","last_fresh":"2024-04-25T10:30:29.881772+0000","last_change":"2024-04-25T10:30:28.881118+0000","last_active":"2024-04-25T10:30:29.881772+0000","last_peered":"2024-04-25T10:30:29.881772+0000","last_clean":"2024-04-25T10:30:29.881772+0000","last_became_active":"2024-04-25T10:30:28.880810+0000","last_became_peered":"2024-04-25T10:30:28.880810+0000","last_unstale":"2024-04-25T10:30:29.881772+0000","last_undegraded":"2024-04-25T10:30:29.881772+0000","last_fullsized":"2024-04-25T10:30:29.881772+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":42,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:30:27.860024+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:30:27.860024+0000","last_clean_scrub_stamp":"2024-04-25T10:30:27.860024+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T15:53:38.989476+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.b","version":"37'4","reported_seq":29,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-04-25T10:30:28.875996+0000","last_change":"2024-04-25T10:30:11.752902+0000","last_active":"2024-04-25T10:30:28.875996+0000","last_peered":"2024-04-25T10:30:28.875996+0000","last_clean":"2024-04-25T10:30:28.875996+0000","last_became_active":"2024-04-25T10:30:11.752599+0000","last_became_peered":"2024-04-25T10:30:11.752599+0000","last_unstale":"2024-04-25T10:30:28.875996+0000","last_undegraded":"2024-04-25T10:30:28.875996+0000","last_fullsized":"2024-04-25T10:30:28.875996+0000","mapping_epoch":33,"log_start":"0'0","ondisk_log_start":"0'0","created":33,"last_epoch_clean":34,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:30:10.729165+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:30:10.729165+0000","last_clean_scrub_stamp":"2024-04-25T10:30:10.729165+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T14:01:05.786983+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"5.a","version":"36'5","reported_seq":32,"reported_epoch":44,"state":"active+clean","last_fresh":"2024-04-25T10:30:29.882793+0000","last_change":"2024-04-25T10:30:12.762940+0000","last_active":"2024-04-25T10:30:29.882793+0000","last_peered":"2024-04-25T10:30:29.882793+0000","last_clean":"2024-04-25T10:30:29.882793+0000","last_became_active":"2024-04-25T10:30:12.762628+0000","last_became_peered":"2024-04-25T10:30:12.762628+0000","last_unstale":"2024-04-25T10:30:29.882793+0000","last_undegraded":"2024-04-25T10:30:29.882793+0000","last_fullsized":"2024-04-25T10:30:29.882793+0000","mapping_epoch":34,"log_start":"0'0","ondisk_log_start":"0'0","created":34,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:30:11.733978+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:30:11.733978+0000","last_clean_scrub_stamp":"2024-04-25T10:30:11.733978+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T19:25:06.397830+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"6.9","version":"0'0","reported_seq":1,"reported_epoch":42,"state":"creating+peering","last_fresh":"2024-04-25T10:30:27.871775+0000","last_change":"2024-04-25T10:30:27.871775+0000","last_active":"2024-04-25T10:30:27.860024+0000","last_peered":"2024-04-25T10:30:27.860024+0000","last_clean":"2024-04-25T10:30:27.860024+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T10:30:27.871775+0000","last_undegraded":"2024-04-25T10:30:27.871775+0000","last_fullsized":"2024-04-25T10:30:27.871775+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":42,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:30:27.860024+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:30:27.860024+0000","last_clean_scrub_stamp":"2024-04-25T10:30:27.860024+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.c","version":"0'0","reported_seq":25,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-04-25T10:30:28.875034+0000","last_change":"2024-04-25T10:30:11.753067+0000","last_active":"2024-04-25T10:30:28.875034+0000","last_peered":"2024-04-25T10:30:28.875034+0000","last_clean":"2024-04-25T10:30:28.875034+0000","last_became_active":"2024-04-25T10:30:11.752595+0000","last_became_peered":"2024-04-25T10:30:11.752595+0000","last_unstale":"2024-04-25T10:30:28.875034+0000","last_undegraded":"2024-04-25T10:30:28.875034+0000","last_fullsized":"2024-04-25T10:30:28.875034+0000","mapping_epoch":33,"log_start":"0'0","ondisk_log_start":"0'0","created":33,"last_epoch_clean":34,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:30:10.729165+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:30:10.729165+0000","last_clean_scrub_stamp":"2024-04-25T10:30:10.729165+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T18:59:19.688948+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"5.d","version":"36'7","reported_seq":31,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-04-25T10:30:28.875563+0000","last_change":"2024-04-25T10:30:12.771743+0000","last_active":"2024-04-25T10:30:28.875563+0000","last_peered":"2024-04-25T10:30:28.875563+0000","last_clean":"2024-04-25T10:30:28.875563+0000","last_became_active":"2024-04-25T10:30:12.771387+0000","last_became_peered":"2024-04-25T10:30:12.771387+0000","last_unstale":"2024-04-25T10:30:28.875563+0000","last_undegraded":"2024-04-25T10:30:28.875563+0000","last_fullsized":"2024-04-25T10:30:28.875563+0000","mapping_epoch":34,"log_start":"0'0","ondisk_log_start":"0'0","created":34,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:30:11.733978+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:30:11.733978+0000","last_clean_scrub_stamp":"2024-04-25T10:30:11.733978+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T14:01:06.791796+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"6.e","version":"0'0","reported_seq":9,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-04-25T10:30:28.885894+0000","last_change":"2024-04-25T10:30:28.884467+0000","last_active":"2024-04-25T10:30:28.885894+0000","last_peered":"2024-04-25T10:30:28.885894+0000","last_clean":"2024-04-25T10:30:28.885894+0000","last_became_active":"2024-04-25T10:30:28.884173+0000","last_became_peered":"2024-04-25T10:30:28.884173+0000","last_unstale":"2024-04-25T10:30:28.885894+0000","last_undegraded":"2024-04-25T10:30:28.885894+0000","last_fullsized":"2024-04-25T10:30:28.885894+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":42,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:30:27.860024+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:30:27.860024+0000","last_clean_scrub_stamp":"2024-04-25T10:30:27.860024+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T20:14:31.367609+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.d","version":"37'2","reported_seq":32,"reported_epoch":44,"state":"active+clean","last_fresh":"2024-04-25T10:30:29.882419+0000","last_change":"2024-04-25T10:30:11.758390+0000","last_active":"2024-04-25T10:30:29.882419+0000","last_peered":"2024-04-25T10:30:29.882419+0000","last_clean":"2024-04-25T10:30:29.882419+0000","last_became_active":"2024-04-25T10:30:11.757989+0000","last_became_peered":"2024-04-25T10:30:11.757989+0000","last_unstale":"2024-04-25T10:30:29.882419+0000","last_undegraded":"2024-04-25T10:30:29.882419+0000","last_fullsized":"2024-04-25T10:30:29.882419+0000","mapping_epoch":33,"log_start":"0'0","ondisk_log_start":"0'0","created":33,"last_epoch_clean":34,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:30:10.729165+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:30:10.729165+0000","last_clean_scrub_stamp":"2024-04-25T10:30:10.729165+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T18:52:44.534946+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.c","version":"36'6","reported_seq":30,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-04-25T10:30:28.876757+0000","last_change":"2024-04-25T10:30:12.765790+0000","last_active":"2024-04-25T10:30:28.876757+0000","last_peered":"2024-04-25T10:30:28.876757+0000","last_clean":"2024-04-25T10:30:28.876757+0000","last_became_active":"2024-04-25T10:30:12.765468+0000","last_became_peered":"2024-04-25T10:30:12.765468+0000","last_unstale":"2024-04-25T10:30:28.876757+0000","last_undegraded":"2024-04-25T10:30:28.876757+0000","last_fullsized":"2024-04-25T10:30:28.876757+0000","mapping_epoch":34,"log_start":"0'0","ondisk_log_start":"0'0","created":34,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:30:11.733978+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:30:11.733978+0000","last_clean_scrub_stamp":"2024-04-25T10:30:11.733978+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T16:29:40.621322+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"6.f","version":"0'0","reported_seq":9,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-04-25T10:30:28.887668+0000","last_change":"2024-04-25T10:30:28.886506+0000","last_active":"2024-04-25T10:30:28.887668+0000","last_peered":"2024-04-25T10:30:28.887668+0000","last_clean":"2024-04-25T10:30:28.887668+0000","last_became_active":"2024-04-25T10:30:28.886157+0000","last_became_peered":"2024-04-25T10:30:28.886157+0000","last_unstale":"2024-04-25T10:30:28.887668+0000","last_undegraded":"2024-04-25T10:30:28.887668+0000","last_fullsized":"2024-04-25T10:30:28.887668+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":42,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:30:27.860024+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:30:27.860024+0000","last_clean_scrub_stamp":"2024-04-25T10:30:27.860024+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T11:57:46.105302+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.e","version":"37'3","reported_seq":33,"reported_epoch":44,"state":"active+clean","last_fresh":"2024-04-25T10:30:29.883403+0000","last_change":"2024-04-25T10:30:11.765106+0000","last_active":"2024-04-25T10:30:29.883403+0000","last_peered":"2024-04-25T10:30:29.883403+0000","last_clean":"2024-04-25T10:30:29.883403+0000","last_became_active":"2024-04-25T10:30:11.764710+0000","last_became_peered":"2024-04-25T10:30:11.764710+0000","last_unstale":"2024-04-25T10:30:29.883403+0000","last_undegraded":"2024-04-25T10:30:29.883403+0000","last_fullsized":"2024-04-25T10:30:29.883403+0000","mapping_epoch":33,"log_start":"0'0","ondisk_log_start":"0'0","created":33,"last_epoch_clean":34,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:30:10.729165+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:30:10.729165+0000","last_clean_scrub_stamp":"2024-04-25T10:30:10.729165+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T16:30:11.709416+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"5.f","version":"36'9","reported_seq":34,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-04-25T10:30:28.875269+0000","last_change":"2024-04-25T10:30:12.769270+0000","last_active":"2024-04-25T10:30:28.875269+0000","last_peered":"2024-04-25T10:30:28.875269+0000","last_clean":"2024-04-25T10:30:28.875269+0000","last_became_active":"2024-04-25T10:30:12.768931+0000","last_became_peered":"2024-04-25T10:30:12.768931+0000","last_unstale":"2024-04-25T10:30:28.875269+0000","last_undegraded":"2024-04-25T10:30:28.875269+0000","last_fullsized":"2024-04-25T10:30:28.875269+0000","mapping_epoch":34,"log_start":"0'0","ondisk_log_start":"0'0","created":34,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:30:11.733978+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:30:11.733978+0000","last_clean_scrub_stamp":"2024-04-25T10:30:11.733978+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T16:49:00.268423+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"6.c","version":"0'0","reported_seq":12,"reported_epoch":44,"state":"active+clean","last_fresh":"2024-04-25T10:30:29.883642+0000","last_change":"2024-04-25T10:30:28.887055+0000","last_active":"2024-04-25T10:30:29.883642+0000","last_peered":"2024-04-25T10:30:29.883642+0000","last_clean":"2024-04-25T10:30:29.883642+0000","last_became_active":"2024-04-25T10:30:28.886675+0000","last_became_peered":"2024-04-25T10:30:28.886675+0000","last_unstale":"2024-04-25T10:30:29.883642+0000","last_undegraded":"2024-04-25T10:30:29.883642+0000","last_fullsized":"2024-04-25T10:30:29.883642+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":42,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:30:27.860024+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:30:27.860024+0000","last_clean_scrub_stamp":"2024-04-25T10:30:27.860024+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T14:04:58.646968+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,1],"acting":[6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.f","version":"37'2","reported_seq":27,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-04-25T10:30:28.874709+0000","last_change":"2024-04-25T10:30:11.750348+0000","last_active":"2024-04-25T10:30:28.874709+0000","last_peered":"2024-04-25T10:30:28.874709+0000","last_clean":"2024-04-25T10:30:28.874709+0000","last_became_active":"2024-04-25T10:30:11.749970+0000","last_became_peered":"2024-04-25T10:30:11.749970+0000","last_unstale":"2024-04-25T10:30:28.874709+0000","last_undegraded":"2024-04-25T10:30:28.874709+0000","last_fullsized":"2024-04-25T10:30:28.874709+0000","mapping_epoch":33,"log_start":"0'0","ondisk_log_start":"0'0","created":33,"last_epoch_clean":34,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:30:10.729165+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:30:10.729165+0000","last_clean_scrub_stamp":"2024-04-25T10:30:10.729165+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T20:41:22.574408+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"5.e","version":"36'5","reported_seq":34,"reported_epoch":44,"state":"active+clean","last_fresh":"2024-04-25T10:30:29.882392+0000","last_change":"2024-04-25T10:30:12.780850+0000","last_active":"2024-04-25T10:30:29.882392+0000","last_peered":"2024-04-25T10:30:29.882392+0000","last_clean":"2024-04-25T10:30:29.882392+0000","last_became_active":"2024-04-25T10:30:12.780187+0000","last_became_peered":"2024-04-25T10:30:12.780187+0000","last_unstale":"2024-04-25T10:30:29.882392+0000","last_undegraded":"2024-04-25T10:30:29.882392+0000","last_fullsized":"2024-04-25T10:30:29.882392+0000","mapping_epoch":34,"log_start":"0'0","ondisk_log_start":"0'0","created":34,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:30:11.733978+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:30:11.733978+0000","last_clean_scrub_stamp":"2024-04-25T10:30:11.733978+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T12:49:30.093333+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"6.d","version":"0'0","reported_seq":9,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-04-25T10:30:28.880134+0000","last_change":"2024-04-25T10:30:28.878873+0000","last_active":"2024-04-25T10:30:28.880134+0000","last_peered":"2024-04-25T10:30:28.880134+0000","last_clean":"2024-04-25T10:30:28.880134+0000","last_became_active":"2024-04-25T10:30:28.878596+0000","last_became_peered":"2024-04-25T10:30:28.878596+0000","last_unstale":"2024-04-25T10:30:28.880134+0000","last_undegraded":"2024-04-25T10:30:28.880134+0000","last_fullsized":"2024-04-25T10:30:28.880134+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":42,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:30:27.860024+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:30:27.860024+0000","last_clean_scrub_stamp":"2024-04-25T10:30:27.860024+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T11:33:01.743970+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.10","version":"37'15","reported_seq":40,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-04-25T10:30:28.873989+0000","last_change":"2024-04-25T10:30:11.755358+0000","last_active":"2024-04-25T10:30:28.873989+0000","last_peered":"2024-04-25T10:30:28.873989+0000","last_clean":"2024-04-25T10:30:28.873989+0000","last_became_active":"2024-04-25T10:30:11.754992+0000","last_became_peered":"2024-04-25T10:30:11.754992+0000","last_unstale":"2024-04-25T10:30:28.873989+0000","last_undegraded":"2024-04-25T10:30:28.873989+0000","last_fullsized":"2024-04-25T10:30:28.873989+0000","mapping_epoch":33,"log_start":"0'0","ondisk_log_start":"0'0","created":33,"last_epoch_clean":34,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:30:10.729165+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:30:10.729165+0000","last_clean_scrub_stamp":"2024-04-25T10:30:10.729165+0000","objects_scrubbed":0,"log_size":15,"log_dups_size":0,"ondisk_log_size":15,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T19:57:30.397817+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":21,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3],"acting":[7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"5.11","version":"36'6","reported_seq":35,"reported_epoch":44,"state":"active+clean","last_fresh":"2024-04-25T10:30:29.881885+0000","last_change":"2024-04-25T10:30:12.777184+0000","last_active":"2024-04-25T10:30:29.881885+0000","last_peered":"2024-04-25T10:30:29.881885+0000","last_clean":"2024-04-25T10:30:29.881885+0000","last_became_active":"2024-04-25T10:30:12.776874+0000","last_became_peered":"2024-04-25T10:30:12.776874+0000","last_unstale":"2024-04-25T10:30:29.881885+0000","last_undegraded":"2024-04-25T10:30:29.881885+0000","last_fullsized":"2024-04-25T10:30:29.881885+0000","mapping_epoch":34,"log_start":"0'0","ondisk_log_start":"0'0","created":34,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:30:11.733978+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:30:11.733978+0000","last_clean_scrub_stamp":"2024-04-25T10:30:11.733978+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T16:41:33.609972+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"6.12","version":"0'0","reported_seq":9,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-04-25T10:30:28.882650+0000","last_change":"2024-04-25T10:30:28.881494+0000","last_active":"2024-04-25T10:30:28.882650+0000","last_peered":"2024-04-25T10:30:28.882650+0000","last_clean":"2024-04-25T10:30:28.882650+0000","last_became_active":"2024-04-25T10:30:28.880682+0000","last_became_peered":"2024-04-25T10:30:28.880682+0000","last_unstale":"2024-04-25T10:30:28.882650+0000","last_undegraded":"2024-04-25T10:30:28.882650+0000","last_fullsized":"2024-04-25T10:30:28.882650+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":42,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:30:27.860024+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:30:27.860024+0000","last_clean_scrub_stamp":"2024-04-25T10:30:27.860024+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T17:35:49.687185+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.11","version":"0'0","reported_seq":25,"reported_epoch":42,"state":"active+clean","last_fresh":"2024-04-25T10:30:27.876406+0000","last_change":"2024-04-25T10:30:11.755706+0000","last_active":"2024-04-25T10:30:27.876406+0000","last_peered":"2024-04-25T10:30:27.876406+0000","last_clean":"2024-04-25T10:30:27.876406+0000","last_became_active":"2024-04-25T10:30:11.755348+0000","last_became_peered":"2024-04-25T10:30:11.755348+0000","last_unstale":"2024-04-25T10:30:27.876406+0000","last_undegraded":"2024-04-25T10:30:27.876406+0000","last_fullsized":"2024-04-25T10:30:27.876406+0000","mapping_epoch":33,"log_start":"0'0","ondisk_log_start":"0'0","created":33,"last_epoch_clean":34,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:30:10.729165+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:30:10.729165+0000","last_clean_scrub_stamp":"2024-04-25T10:30:10.729165+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T10:42:20.657474+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.10","version":"36'6","reported_seq":35,"reported_epoch":44,"state":"active+clean","last_fresh":"2024-04-25T10:30:29.882755+0000","last_change":"2024-04-25T10:30:12.783285+0000","last_active":"2024-04-25T10:30:29.882755+0000","last_peered":"2024-04-25T10:30:29.882755+0000","last_clean":"2024-04-25T10:30:29.882755+0000","last_became_active":"2024-04-25T10:30:12.782617+0000","last_became_peered":"2024-04-25T10:30:12.782617+0000","last_unstale":"2024-04-25T10:30:29.882755+0000","last_undegraded":"2024-04-25T10:30:29.882755+0000","last_fullsized":"2024-04-25T10:30:29.882755+0000","mapping_epoch":34,"log_start":"0'0","ondisk_log_start":"0'0","created":34,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:30:11.733978+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:30:11.733978+0000","last_clean_scrub_stamp":"2024-04-25T10:30:11.733978+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T18:35:51.085554+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"6.13","version":"0'0","reported_seq":9,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-04-25T10:30:28.882643+0000","last_change":"2024-04-25T10:30:28.880997+0000","last_active":"2024-04-25T10:30:28.882643+0000","last_peered":"2024-04-25T10:30:28.882643+0000","last_clean":"2024-04-25T10:30:28.882643+0000","last_became_active":"2024-04-25T10:30:28.880686+0000","last_became_peered":"2024-04-25T10:30:28.880686+0000","last_unstale":"2024-04-25T10:30:28.882643+0000","last_undegraded":"2024-04-25T10:30:28.882643+0000","last_fullsized":"2024-04-25T10:30:28.882643+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":42,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:30:27.860024+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:30:27.860024+0000","last_clean_scrub_stamp":"2024-04-25T10:30:27.860024+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T20:14:31.367609+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.12","version":"0'0","reported_seq":25,"reported_epoch":42,"state":"active+clean","last_fresh":"2024-04-25T10:30:27.876694+0000","last_change":"2024-04-25T10:30:11.762094+0000","last_active":"2024-04-25T10:30:27.876694+0000","last_peered":"2024-04-25T10:30:27.876694+0000","last_clean":"2024-04-25T10:30:27.876694+0000","last_became_active":"2024-04-25T10:30:11.761456+0000","last_became_peered":"2024-04-25T10:30:11.761456+0000","last_unstale":"2024-04-25T10:30:27.876694+0000","last_undegraded":"2024-04-25T10:30:27.876694+0000","last_fullsized":"2024-04-25T10:30:27.876694+0000","mapping_epoch":33,"log_start":"0'0","ondisk_log_start":"0'0","created":33,"last_epoch_clean":34,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:30:10.729165+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:30:10.729165+0000","last_clean_scrub_stamp":"2024-04-25T10:30:10.729165+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T21:58:31.211023+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.13","version":"36'15","reported_seq":38,"reported_epoch":42,"state":"active+clean","last_fresh":"2024-04-25T10:30:27.876744+0000","last_change":"2024-04-25T10:30:12.775269+0000","last_active":"2024-04-25T10:30:27.876744+0000","last_peered":"2024-04-25T10:30:27.876744+0000","last_clean":"2024-04-25T10:30:27.876744+0000","last_became_active":"2024-04-25T10:30:12.774865+0000","last_became_peered":"2024-04-25T10:30:12.774865+0000","last_unstale":"2024-04-25T10:30:27.876744+0000","last_undegraded":"2024-04-25T10:30:27.876744+0000","last_fullsized":"2024-04-25T10:30:27.876744+0000","mapping_epoch":34,"log_start":"0'0","ondisk_log_start":"0'0","created":34,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:30:11.733978+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:30:11.733978+0000","last_clean_scrub_stamp":"2024-04-25T10:30:11.733978+0000","objects_scrubbed":0,"log_size":15,"log_dups_size":0,"ondisk_log_size":15,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T20:14:36.991455+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"6.10","version":"0'0","reported_seq":1,"reported_epoch":42,"state":"creating+peering","last_fresh":"2024-04-25T10:30:27.875123+0000","last_change":"2024-04-25T10:30:27.875123+0000","last_active":"2024-04-25T10:30:27.860024+0000","last_peered":"2024-04-25T10:30:27.860024+0000","last_clean":"2024-04-25T10:30:27.860024+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T10:30:27.875123+0000","last_undegraded":"2024-04-25T10:30:27.875123+0000","last_fullsized":"2024-04-25T10:30:27.875123+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":42,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:30:27.860024+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:30:27.860024+0000","last_clean_scrub_stamp":"2024-04-25T10:30:27.860024+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.13","version":"37'5","reported_seq":35,"reported_epoch":44,"state":"active+clean","last_fresh":"2024-04-25T10:30:29.883254+0000","last_change":"2024-04-25T10:30:11.764993+0000","last_active":"2024-04-25T10:30:29.883254+0000","last_peered":"2024-04-25T10:30:29.883254+0000","last_clean":"2024-04-25T10:30:29.883254+0000","last_became_active":"2024-04-25T10:30:11.764637+0000","last_became_peered":"2024-04-25T10:30:11.764637+0000","last_unstale":"2024-04-25T10:30:29.883254+0000","last_undegraded":"2024-04-25T10:30:29.883254+0000","last_fullsized":"2024-04-25T10:30:29.883254+0000","mapping_epoch":33,"log_start":"0'0","ondisk_log_start":"0'0","created":33,"last_epoch_clean":34,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:30:10.729165+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:30:10.729165+0000","last_clean_scrub_stamp":"2024-04-25T10:30:10.729165+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T17:24:13.848319+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6,"num_write_kb":8,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"5.12","version":"36'10","reported_seq":37,"reported_epoch":44,"state":"active+clean","last_fresh":"2024-04-25T10:30:29.882763+0000","last_change":"2024-04-25T10:30:12.770496+0000","last_active":"2024-04-25T10:30:29.882763+0000","last_peered":"2024-04-25T10:30:29.882763+0000","last_clean":"2024-04-25T10:30:29.882763+0000","last_became_active":"2024-04-25T10:30:12.770193+0000","last_became_peered":"2024-04-25T10:30:12.770193+0000","last_unstale":"2024-04-25T10:30:29.882763+0000","last_undegraded":"2024-04-25T10:30:29.882763+0000","last_fullsized":"2024-04-25T10:30:29.882763+0000","mapping_epoch":34,"log_start":"0'0","ondisk_log_start":"0'0","created":34,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:30:11.733978+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:30:11.733978+0000","last_clean_scrub_stamp":"2024-04-25T10:30:11.733978+0000","objects_scrubbed":0,"log_size":10,"log_dups_size":0,"ondisk_log_size":10,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T21:35:23.984378+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,1],"acting":[6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"6.11","version":"0'0","reported_seq":12,"reported_epoch":44,"state":"active+clean","last_fresh":"2024-04-25T10:30:29.882707+0000","last_change":"2024-04-25T10:30:28.884488+0000","last_active":"2024-04-25T10:30:29.882707+0000","last_peered":"2024-04-25T10:30:29.882707+0000","last_clean":"2024-04-25T10:30:29.882707+0000","last_became_active":"2024-04-25T10:30:28.884188+0000","last_became_peered":"2024-04-25T10:30:28.884188+0000","last_unstale":"2024-04-25T10:30:29.882707+0000","last_undegraded":"2024-04-25T10:30:29.882707+0000","last_fullsized":"2024-04-25T10:30:29.882707+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":42,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:30:27.860024+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:30:27.860024+0000","last_clean_scrub_stamp":"2024-04-25T10:30:27.860024+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T15:39:48.016111+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,1],"acting":[6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.14","version":"37'23","reported_seq":51,"reported_epoch":44,"state":"active+clean","last_fresh":"2024-04-25T10:30:29.883688+0000","last_change":"2024-04-25T10:30:11.757323+0000","last_active":"2024-04-25T10:30:29.883688+0000","last_peered":"2024-04-25T10:30:29.883688+0000","last_clean":"2024-04-25T10:30:29.883688+0000","last_became_active":"2024-04-25T10:30:11.757011+0000","last_became_peered":"2024-04-25T10:30:11.757011+0000","last_unstale":"2024-04-25T10:30:29.883688+0000","last_undegraded":"2024-04-25T10:30:29.883688+0000","last_fullsized":"2024-04-25T10:30:29.883688+0000","mapping_epoch":33,"log_start":"0'0","ondisk_log_start":"0'0","created":33,"last_epoch_clean":34,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:30:10.729165+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:30:10.729165+0000","last_clean_scrub_stamp":"2024-04-25T10:30:10.729165+0000","objects_scrubbed":0,"log_size":23,"log_dups_size":0,"ondisk_log_size":23,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T19:02:53.584992+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":23,"num_write_kb":25,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"5.15","version":"36'6","reported_seq":35,"reported_epoch":44,"state":"active+clean","last_fresh":"2024-04-25T10:30:29.881889+0000","last_change":"2024-04-25T10:30:12.776432+0000","last_active":"2024-04-25T10:30:29.881889+0000","last_peered":"2024-04-25T10:30:29.881889+0000","last_clean":"2024-04-25T10:30:29.881889+0000","last_became_active":"2024-04-25T10:30:12.775857+0000","last_became_peered":"2024-04-25T10:30:12.775857+0000","last_unstale":"2024-04-25T10:30:29.881889+0000","last_undegraded":"2024-04-25T10:30:29.881889+0000","last_fullsized":"2024-04-25T10:30:29.881889+0000","mapping_epoch":34,"log_start":"0'0","ondisk_log_start":"0'0","created":34,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:30:11.733978+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:30:11.733978+0000","last_clean_scrub_stamp":"2024-04-25T10:30:11.733978+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T14:29:10.343115+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"6.16","version":"0'0","reported_seq":1,"reported_epoch":42,"state":"creating+peering","last_fresh":"2024-04-25T10:30:27.876799+0000","last_change":"2024-04-25T10:30:27.876799+0000","last_active":"2024-04-25T10:30:27.860024+0000","last_peered":"2024-04-25T10:30:27.860024+0000","last_clean":"2024-04-25T10:30:27.860024+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T10:30:27.876799+0000","last_undegraded":"2024-04-25T10:30:27.876799+0000","last_fullsized":"2024-04-25T10:30:27.876799+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":42,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:30:27.860024+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:30:27.860024+0000","last_clean_scrub_stamp":"2024-04-25T10:30:27.860024+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.15","version":"37'5","reported_seq":35,"reported_epoch":44,"state":"active+clean","last_fresh":"2024-04-25T10:30:29.882033+0000","last_change":"2024-04-25T10:30:11.755959+0000","last_active":"2024-04-25T10:30:29.882033+0000","last_peered":"2024-04-25T10:30:29.882033+0000","last_clean":"2024-04-25T10:30:29.882033+0000","last_became_active":"2024-04-25T10:30:11.755603+0000","last_became_peered":"2024-04-25T10:30:11.755603+0000","last_unstale":"2024-04-25T10:30:29.882033+0000","last_undegraded":"2024-04-25T10:30:29.882033+0000","last_fullsized":"2024-04-25T10:30:29.882033+0000","mapping_epoch":33,"log_start":"0'0","ondisk_log_start":"0'0","created":33,"last_epoch_clean":34,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:30:10.729165+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:30:10.729165+0000","last_clean_scrub_stamp":"2024-04-25T10:30:10.729165+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T19:25:05.393017+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.14","version":"36'11","reported_seq":34,"reported_epoch":42,"state":"active+clean","last_fresh":"2024-04-25T10:30:27.877515+0000","last_change":"2024-04-25T10:30:12.770588+0000","last_active":"2024-04-25T10:30:27.877515+0000","last_peered":"2024-04-25T10:30:27.877515+0000","last_clean":"2024-04-25T10:30:27.877515+0000","last_became_active":"2024-04-25T10:30:12.770264+0000","last_became_peered":"2024-04-25T10:30:12.770264+0000","last_unstale":"2024-04-25T10:30:27.877515+0000","last_undegraded":"2024-04-25T10:30:27.877515+0000","last_fullsized":"2024-04-25T10:30:27.877515+0000","mapping_epoch":34,"log_start":"0'0","ondisk_log_start":"0'0","created":34,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:30:11.733978+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:30:11.733978+0000","last_clean_scrub_stamp":"2024-04-25T10:30:11.733978+0000","objects_scrubbed":0,"log_size":11,"log_dups_size":0,"ondisk_log_size":11,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T20:41:50.045543+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"6.17","version":"0'0","reported_seq":9,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-04-25T10:30:28.880577+0000","last_change":"2024-04-25T10:30:28.878754+0000","last_active":"2024-04-25T10:30:28.880577+0000","last_peered":"2024-04-25T10:30:28.880577+0000","last_clean":"2024-04-25T10:30:28.880577+0000","last_became_active":"2024-04-25T10:30:28.878242+0000","last_became_peered":"2024-04-25T10:30:28.878242+0000","last_unstale":"2024-04-25T10:30:28.880577+0000","last_undegraded":"2024-04-25T10:30:28.880577+0000","last_fullsized":"2024-04-25T10:30:28.880577+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":42,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:30:27.860024+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:30:27.860024+0000","last_clean_scrub_stamp":"2024-04-25T10:30:27.860024+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T16:08:46.113866+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7],"acting":[1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.16","version":"0'0","reported_seq":25,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-04-25T10:30:28.879092+0000","last_change":"2024-04-25T10:30:11.749473+0000","last_active":"2024-04-25T10:30:28.879092+0000","last_peered":"2024-04-25T10:30:28.879092+0000","last_clean":"2024-04-25T10:30:28.879092+0000","last_became_active":"2024-04-25T10:30:11.749135+0000","last_became_peered":"2024-04-25T10:30:11.749135+0000","last_unstale":"2024-04-25T10:30:28.879092+0000","last_undegraded":"2024-04-25T10:30:28.879092+0000","last_fullsized":"2024-04-25T10:30:28.879092+0000","mapping_epoch":33,"log_start":"0'0","ondisk_log_start":"0'0","created":33,"last_epoch_clean":34,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:30:10.729165+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:30:10.729165+0000","last_clean_scrub_stamp":"2024-04-25T10:30:10.729165+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T15:17:28.997853+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"5.17","version":"36'9","reported_seq":32,"reported_epoch":42,"state":"active+clean","last_fresh":"2024-04-25T10:30:27.872862+0000","last_change":"2024-04-25T10:30:12.773113+0000","last_active":"2024-04-25T10:30:27.872862+0000","last_peered":"2024-04-25T10:30:27.872862+0000","last_clean":"2024-04-25T10:30:27.872862+0000","last_became_active":"2024-04-25T10:30:12.772786+0000","last_became_peered":"2024-04-25T10:30:12.772786+0000","last_unstale":"2024-04-25T10:30:27.872862+0000","last_undegraded":"2024-04-25T10:30:27.872862+0000","last_fullsized":"2024-04-25T10:30:27.872862+0000","mapping_epoch":34,"log_start":"0'0","ondisk_log_start":"0'0","created":34,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:30:11.733978+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:30:11.733978+0000","last_clean_scrub_stamp":"2024-04-25T10:30:11.733978+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T18:45:03.004387+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5],"acting":[4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"6.14","version":"0'0","reported_seq":1,"reported_epoch":42,"state":"creating+peering","last_fresh":"2024-04-25T10:30:27.876893+0000","last_change":"2024-04-25T10:30:27.876893+0000","last_active":"2024-04-25T10:30:27.860024+0000","last_peered":"2024-04-25T10:30:27.860024+0000","last_clean":"2024-04-25T10:30:27.860024+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T10:30:27.876893+0000","last_undegraded":"2024-04-25T10:30:27.876893+0000","last_fullsized":"2024-04-25T10:30:27.876893+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":42,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:30:27.860024+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:30:27.860024+0000","last_clean_scrub_stamp":"2024-04-25T10:30:27.860024+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.17","version":"0'0","reported_seq":25,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-04-25T10:30:28.878199+0000","last_change":"2024-04-25T10:30:11.758416+0000","last_active":"2024-04-25T10:30:28.878199+0000","last_peered":"2024-04-25T10:30:28.878199+0000","last_clean":"2024-04-25T10:30:28.878199+0000","last_became_active":"2024-04-25T10:30:11.758049+0000","last_became_peered":"2024-04-25T10:30:11.758049+0000","last_unstale":"2024-04-25T10:30:28.878199+0000","last_undegraded":"2024-04-25T10:30:28.878199+0000","last_fullsized":"2024-04-25T10:30:28.878199+0000","mapping_epoch":33,"log_start":"0'0","ondisk_log_start":"0'0","created":33,"last_epoch_clean":34,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:30:10.729165+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:30:10.729165+0000","last_clean_scrub_stamp":"2024-04-25T10:30:10.729165+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T18:26:05.159854+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"5.16","version":"36'10","reported_seq":39,"reported_epoch":44,"state":"active+clean","last_fresh":"2024-04-25T10:30:29.882265+0000","last_change":"2024-04-25T10:30:12.770216+0000","last_active":"2024-04-25T10:30:29.882265+0000","last_peered":"2024-04-25T10:30:29.882265+0000","last_clean":"2024-04-25T10:30:29.882265+0000","last_became_active":"2024-04-25T10:30:12.769785+0000","last_became_peered":"2024-04-25T10:30:12.769785+0000","last_unstale":"2024-04-25T10:30:29.882265+0000","last_undegraded":"2024-04-25T10:30:29.882265+0000","last_fullsized":"2024-04-25T10:30:29.882265+0000","mapping_epoch":34,"log_start":"0'0","ondisk_log_start":"0'0","created":34,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:30:11.733978+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:30:11.733978+0000","last_clean_scrub_stamp":"2024-04-25T10:30:11.733978+0000","objects_scrubbed":0,"log_size":10,"log_dups_size":0,"ondisk_log_size":10,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T16:58:09.648786+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"6.15","version":"0'0","reported_seq":12,"reported_epoch":44,"state":"active+clean","last_fresh":"2024-04-25T10:30:29.883532+0000","last_change":"2024-04-25T10:30:28.879590+0000","last_active":"2024-04-25T10:30:29.883532+0000","last_peered":"2024-04-25T10:30:29.883532+0000","last_clean":"2024-04-25T10:30:29.883532+0000","last_became_active":"2024-04-25T10:30:28.879277+0000","last_became_peered":"2024-04-25T10:30:28.879277+0000","last_unstale":"2024-04-25T10:30:29.883532+0000","last_undegraded":"2024-04-25T10:30:29.883532+0000","last_fullsized":"2024-04-25T10:30:29.883532+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":42,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:30:27.860024+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:30:27.860024+0000","last_clean_scrub_stamp":"2024-04-25T10:30:27.860024+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T14:04:58.646968+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.18","version":"0'0","reported_seq":30,"reported_epoch":44,"state":"active+clean","last_fresh":"2024-04-25T10:30:29.882866+0000","last_change":"2024-04-25T10:30:11.756627+0000","last_active":"2024-04-25T10:30:29.882866+0000","last_peered":"2024-04-25T10:30:29.882866+0000","last_clean":"2024-04-25T10:30:29.882866+0000","last_became_active":"2024-04-25T10:30:11.756323+0000","last_became_peered":"2024-04-25T10:30:11.756323+0000","last_unstale":"2024-04-25T10:30:29.882866+0000","last_undegraded":"2024-04-25T10:30:29.882866+0000","last_fullsized":"2024-04-25T10:30:29.882866+0000","mapping_epoch":33,"log_start":"0'0","ondisk_log_start":"0'0","created":33,"last_epoch_clean":34,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:30:10.729165+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:30:10.729165+0000","last_clean_scrub_stamp":"2024-04-25T10:30:10.729165+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T16:49:20.087013+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4],"acting":[2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"5.19","version":"36'10","reported_seq":33,"reported_epoch":42,"state":"active+clean","last_fresh":"2024-04-25T10:30:27.876263+0000","last_change":"2024-04-25T10:30:12.780437+0000","last_active":"2024-04-25T10:30:27.876263+0000","last_peered":"2024-04-25T10:30:27.876263+0000","last_clean":"2024-04-25T10:30:27.876263+0000","last_became_active":"2024-04-25T10:30:12.779762+0000","last_became_peered":"2024-04-25T10:30:12.779762+0000","last_unstale":"2024-04-25T10:30:27.876263+0000","last_undegraded":"2024-04-25T10:30:27.876263+0000","last_fullsized":"2024-04-25T10:30:27.876263+0000","mapping_epoch":34,"log_start":"0'0","ondisk_log_start":"0'0","created":34,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:30:11.733978+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:30:11.733978+0000","last_clean_scrub_stamp":"2024-04-25T10:30:11.733978+0000","objects_scrubbed":0,"log_size":10,"log_dups_size":0,"ondisk_log_size":10,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T14:22:37.474013+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"6.1a","version":"0'0","reported_seq":12,"reported_epoch":44,"state":"active+clean","last_fresh":"2024-04-25T10:30:29.882257+0000","last_change":"2024-04-25T10:30:28.879591+0000","last_active":"2024-04-25T10:30:29.882257+0000","last_peered":"2024-04-25T10:30:29.882257+0000","last_clean":"2024-04-25T10:30:29.882257+0000","last_became_active":"2024-04-25T10:30:28.879271+0000","last_became_peered":"2024-04-25T10:30:28.879271+0000","last_unstale":"2024-04-25T10:30:29.882257+0000","last_undegraded":"2024-04-25T10:30:29.882257+0000","last_fullsized":"2024-04-25T10:30:29.882257+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":42,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:30:27.860024+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:30:27.860024+0000","last_clean_scrub_stamp":"2024-04-25T10:30:27.860024+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T20:14:35.145802+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.19","version":"0'0","reported_seq":25,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-04-25T10:30:28.874176+0000","last_change":"2024-04-25T10:30:11.757481+0000","last_active":"2024-04-25T10:30:28.874176+0000","last_peered":"2024-04-25T10:30:28.874176+0000","last_clean":"2024-04-25T10:30:28.874176+0000","last_became_active":"2024-04-25T10:30:11.757149+0000","last_became_peered":"2024-04-25T10:30:11.757149+0000","last_unstale":"2024-04-25T10:30:28.874176+0000","last_undegraded":"2024-04-25T10:30:28.874176+0000","last_fullsized":"2024-04-25T10:30:28.874176+0000","mapping_epoch":33,"log_start":"0'0","ondisk_log_start":"0'0","created":33,"last_epoch_clean":34,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:30:10.729165+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:30:10.729165+0000","last_clean_scrub_stamp":"2024-04-25T10:30:10.729165+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T13:23:15.814583+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"5.18","version":"36'8","reported_seq":33,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-04-25T10:30:28.874986+0000","last_change":"2024-04-25T10:30:12.769294+0000","last_active":"2024-04-25T10:30:28.874986+0000","last_peered":"2024-04-25T10:30:28.874986+0000","last_clean":"2024-04-25T10:30:28.874986+0000","last_became_active":"2024-04-25T10:30:12.768543+0000","last_became_peered":"2024-04-25T10:30:12.768543+0000","last_unstale":"2024-04-25T10:30:28.874986+0000","last_undegraded":"2024-04-25T10:30:28.874986+0000","last_fullsized":"2024-04-25T10:30:28.874986+0000","mapping_epoch":34,"log_start":"0'0","ondisk_log_start":"0'0","created":34,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:30:11.733978+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:30:11.733978+0000","last_clean_scrub_stamp":"2024-04-25T10:30:11.733978+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T18:26:06.164667+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"6.1b","version":"0'0","reported_seq":12,"reported_epoch":44,"state":"active+clean","last_fresh":"2024-04-25T10:30:29.883121+0000","last_change":"2024-04-25T10:30:28.885011+0000","last_active":"2024-04-25T10:30:29.883121+0000","last_peered":"2024-04-25T10:30:29.883121+0000","last_clean":"2024-04-25T10:30:29.883121+0000","last_became_active":"2024-04-25T10:30:28.883516+0000","last_became_peered":"2024-04-25T10:30:28.883516+0000","last_unstale":"2024-04-25T10:30:29.883121+0000","last_undegraded":"2024-04-25T10:30:29.883121+0000","last_fullsized":"2024-04-25T10:30:29.883121+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":42,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:30:27.860024+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:30:27.860024+0000","last_clean_scrub_stamp":"2024-04-25T10:30:27.860024+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T17:35:49.687185+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,2],"acting":[6,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.1a","version":"35'1","reported_seq":31,"reported_epoch":44,"state":"active+clean","last_fresh":"2024-04-25T10:30:29.883044+0000","last_change":"2024-04-25T10:30:11.758678+0000","last_active":"2024-04-25T10:30:29.883044+0000","last_peered":"2024-04-25T10:30:29.883044+0000","last_clean":"2024-04-25T10:30:29.883044+0000","last_became_active":"2024-04-25T10:30:11.758384+0000","last_became_peered":"2024-04-25T10:30:11.758384+0000","last_unstale":"2024-04-25T10:30:29.883044+0000","last_undegraded":"2024-04-25T10:30:29.883044+0000","last_fullsized":"2024-04-25T10:30:29.883044+0000","mapping_epoch":33,"log_start":"0'0","ondisk_log_start":"0'0","created":33,"last_epoch_clean":34,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:30:10.729165+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:30:10.729165+0000","last_clean_scrub_stamp":"2024-04-25T10:30:10.729165+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T16:17:54.376541+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,6],"acting":[2,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"5.1b","version":"36'8","reported_seq":37,"reported_epoch":44,"state":"active+clean","last_fresh":"2024-04-25T10:30:29.883109+0000","last_change":"2024-04-25T10:30:12.784767+0000","last_active":"2024-04-25T10:30:29.883109+0000","last_peered":"2024-04-25T10:30:29.883109+0000","last_clean":"2024-04-25T10:30:29.883109+0000","last_became_active":"2024-04-25T10:30:12.784311+0000","last_became_peered":"2024-04-25T10:30:12.784311+0000","last_unstale":"2024-04-25T10:30:29.883109+0000","last_undegraded":"2024-04-25T10:30:29.883109+0000","last_fullsized":"2024-04-25T10:30:29.883109+0000","mapping_epoch":34,"log_start":"0'0","ondisk_log_start":"0'0","created":34,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:30:11.733978+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:30:11.733978+0000","last_clean_scrub_stamp":"2024-04-25T10:30:11.733978+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T19:56:57.765496+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,7],"acting":[2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"6.18","version":"0'0","reported_seq":1,"reported_epoch":42,"state":"creating+peering","last_fresh":"2024-04-25T10:30:27.874480+0000","last_change":"2024-04-25T10:30:27.874480+0000","last_active":"2024-04-25T10:30:27.860024+0000","last_peered":"2024-04-25T10:30:27.860024+0000","last_clean":"2024-04-25T10:30:27.860024+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T10:30:27.874480+0000","last_undegraded":"2024-04-25T10:30:27.874480+0000","last_fullsized":"2024-04-25T10:30:27.874480+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":42,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:30:27.860024+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:30:27.860024+0000","last_clean_scrub_stamp":"2024-04-25T10:30:27.860024+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5],"acting":[4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.1b","version":"35'1","reported_seq":31,"reported_epoch":44,"state":"active+clean","last_fresh":"2024-04-25T10:30:29.883205+0000","last_change":"2024-04-25T10:30:11.753667+0000","last_active":"2024-04-25T10:30:29.883205+0000","last_peered":"2024-04-25T10:30:29.883205+0000","last_clean":"2024-04-25T10:30:29.883205+0000","last_became_active":"2024-04-25T10:30:11.753340+0000","last_became_peered":"2024-04-25T10:30:11.753340+0000","last_unstale":"2024-04-25T10:30:29.883205+0000","last_undegraded":"2024-04-25T10:30:29.883205+0000","last_fullsized":"2024-04-25T10:30:29.883205+0000","mapping_epoch":33,"log_start":"0'0","ondisk_log_start":"0'0","created":33,"last_epoch_clean":34,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:30:10.729165+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:30:10.729165+0000","last_clean_scrub_stamp":"2024-04-25T10:30:10.729165+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T16:09:04.081750+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"5.1a","version":"36'7","reported_seq":32,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-04-25T10:30:28.874760+0000","last_change":"2024-04-25T10:30:12.773835+0000","last_active":"2024-04-25T10:30:28.874760+0000","last_peered":"2024-04-25T10:30:28.874760+0000","last_clean":"2024-04-25T10:30:28.874760+0000","last_became_active":"2024-04-25T10:30:12.773538+0000","last_became_peered":"2024-04-25T10:30:12.773538+0000","last_unstale":"2024-04-25T10:30:28.874760+0000","last_undegraded":"2024-04-25T10:30:28.874760+0000","last_fullsized":"2024-04-25T10:30:28.874760+0000","mapping_epoch":34,"log_start":"0'0","ondisk_log_start":"0'0","created":34,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:30:11.733978+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:30:11.733978+0000","last_clean_scrub_stamp":"2024-04-25T10:30:11.733978+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T18:31:49.260513+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"6.19","version":"0'0","reported_seq":9,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-04-25T10:30:28.885557+0000","last_change":"2024-04-25T10:30:28.883543+0000","last_active":"2024-04-25T10:30:28.885557+0000","last_peered":"2024-04-25T10:30:28.885557+0000","last_clean":"2024-04-25T10:30:28.885557+0000","last_became_active":"2024-04-25T10:30:28.883229+0000","last_became_peered":"2024-04-25T10:30:28.883229+0000","last_unstale":"2024-04-25T10:30:28.885557+0000","last_undegraded":"2024-04-25T10:30:28.885557+0000","last_fullsized":"2024-04-25T10:30:28.885557+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":42,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:30:27.860024+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:30:27.860024+0000","last_clean_scrub_stamp":"2024-04-25T10:30:27.860024+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T18:12:57.452350+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"6.1e","version":"0'0","reported_seq":12,"reported_epoch":44,"state":"active+clean","last_fresh":"2024-04-25T10:30:29.882435+0000","last_change":"2024-04-25T10:30:28.874417+0000","last_active":"2024-04-25T10:30:29.882435+0000","last_peered":"2024-04-25T10:30:29.882435+0000","last_clean":"2024-04-25T10:30:29.882435+0000","last_became_active":"2024-04-25T10:30:28.874102+0000","last_became_peered":"2024-04-25T10:30:28.874102+0000","last_unstale":"2024-04-25T10:30:29.882435+0000","last_undegraded":"2024-04-25T10:30:29.882435+0000","last_fullsized":"2024-04-25T10:30:29.882435+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":42,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:30:27.860024+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:30:27.860024+0000","last_clean_scrub_stamp":"2024-04-25T10:30:27.860024+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T21:25:07.511482+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.1c","version":"37'3","reported_seq":28,"reported_epoch":42,"state":"active+clean","last_fresh":"2024-04-25T10:30:27.877716+0000","last_change":"2024-04-25T10:30:11.759829+0000","last_active":"2024-04-25T10:30:27.877716+0000","last_peered":"2024-04-25T10:30:27.877716+0000","last_clean":"2024-04-25T10:30:27.877716+0000","last_became_active":"2024-04-25T10:30:11.759465+0000","last_became_peered":"2024-04-25T10:30:11.759465+0000","last_unstale":"2024-04-25T10:30:27.877716+0000","last_undegraded":"2024-04-25T10:30:27.877716+0000","last_fullsized":"2024-04-25T10:30:27.877716+0000","mapping_epoch":33,"log_start":"0'0","ondisk_log_start":"0'0","created":33,"last_epoch_clean":34,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:30:10.729165+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:30:10.729165+0000","last_clean_scrub_stamp":"2024-04-25T10:30:10.729165+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T13:31:53.608355+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.1d","version":"36'15","reported_seq":38,"reported_epoch":42,"state":"active+clean","last_fresh":"2024-04-25T10:30:27.876677+0000","last_change":"2024-04-25T10:30:12.773474+0000","last_active":"2024-04-25T10:30:27.876677+0000","last_peered":"2024-04-25T10:30:27.876677+0000","last_clean":"2024-04-25T10:30:27.876677+0000","last_became_active":"2024-04-25T10:30:12.773133+0000","last_became_peered":"2024-04-25T10:30:12.773133+0000","last_unstale":"2024-04-25T10:30:27.876677+0000","last_undegraded":"2024-04-25T10:30:27.876677+0000","last_fullsized":"2024-04-25T10:30:27.876677+0000","mapping_epoch":34,"log_start":"0'0","ondisk_log_start":"0'0","created":34,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:30:11.733978+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:30:11.733978+0000","last_clean_scrub_stamp":"2024-04-25T10:30:11.733978+0000","objects_scrubbed":0,"log_size":15,"log_dups_size":0,"ondisk_log_size":15,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T17:24:23.630916+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"6.1f","version":"0'0","reported_seq":12,"reported_epoch":44,"state":"active+clean","last_fresh":"2024-04-25T10:30:29.883478+0000","last_change":"2024-04-25T10:30:28.884033+0000","last_active":"2024-04-25T10:30:29.883478+0000","last_peered":"2024-04-25T10:30:29.883478+0000","last_clean":"2024-04-25T10:30:29.883478+0000","last_became_active":"2024-04-25T10:30:28.883610+0000","last_became_peered":"2024-04-25T10:30:28.883610+0000","last_unstale":"2024-04-25T10:30:29.883478+0000","last_undegraded":"2024-04-25T10:30:29.883478+0000","last_fullsized":"2024-04-25T10:30:29.883478+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":42,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:30:27.860024+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:30:27.860024+0000","last_clean_scrub_stamp":"2024-04-25T10:30:27.860024+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T18:12:57.452350+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.1d","version":"37'3","reported_seq":28,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-04-25T10:30:28.875875+0000","last_change":"2024-04-25T10:30:11.752439+0000","last_active":"2024-04-25T10:30:28.875875+0000","last_peered":"2024-04-25T10:30:28.875875+0000","last_clean":"2024-04-25T10:30:28.875875+0000","last_became_active":"2024-04-25T10:30:11.752050+0000","last_became_peered":"2024-04-25T10:30:11.752050+0000","last_unstale":"2024-04-25T10:30:28.875875+0000","last_undegraded":"2024-04-25T10:30:28.875875+0000","last_fullsized":"2024-04-25T10:30:28.875875+0000","mapping_epoch":33,"log_start":"0'0","ondisk_log_start":"0'0","created":33,"last_epoch_clean":34,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:30:10.729165+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:30:10.729165+0000","last_clean_scrub_stamp":"2024-04-25T10:30:10.729165+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T11:41:21.358729+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"5.1c","version":"36'9","reported_seq":38,"reported_epoch":44,"state":"active+clean","last_fresh":"2024-04-25T10:30:29.883403+0000","last_change":"2024-04-25T10:30:12.781913+0000","last_active":"2024-04-25T10:30:29.883403+0000","last_peered":"2024-04-25T10:30:29.883403+0000","last_clean":"2024-04-25T10:30:29.883403+0000","last_became_active":"2024-04-25T10:30:12.781111+0000","last_became_peered":"2024-04-25T10:30:12.781111+0000","last_unstale":"2024-04-25T10:30:29.883403+0000","last_undegraded":"2024-04-25T10:30:29.883403+0000","last_fullsized":"2024-04-25T10:30:29.883403+0000","mapping_epoch":34,"log_start":"0'0","ondisk_log_start":"0'0","created":34,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:30:11.733978+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:30:11.733978+0000","last_clean_scrub_stamp":"2024-04-25T10:30:11.733978+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T16:29:27.103512+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4],"acting":[2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.1e","version":"35'1","reported_seq":26,"reported_epoch":42,"state":"active+clean","last_fresh":"2024-04-25T10:30:27.877804+0000","last_change":"2024-04-25T10:30:11.761160+0000","last_active":"2024-04-25T10:30:27.877804+0000","last_peered":"2024-04-25T10:30:27.877804+0000","last_clean":"2024-04-25T10:30:27.877804+0000","last_became_active":"2024-04-25T10:30:11.760819+0000","last_became_peered":"2024-04-25T10:30:11.760819+0000","last_unstale":"2024-04-25T10:30:27.877804+0000","last_undegraded":"2024-04-25T10:30:27.877804+0000","last_fullsized":"2024-04-25T10:30:27.877804+0000","mapping_epoch":33,"log_start":"0'0","ondisk_log_start":"0'0","created":33,"last_epoch_clean":34,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:30:10.729165+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:30:10.729165+0000","last_clean_scrub_stamp":"2024-04-25T10:30:10.729165+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T13:00:22.139475+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.1f","version":"36'6","reported_seq":35,"reported_epoch":44,"state":"active+clean","last_fresh":"2024-04-25T10:30:29.883526+0000","last_change":"2024-04-25T10:30:12.772257+0000","last_active":"2024-04-25T10:30:29.883526+0000","last_peered":"2024-04-25T10:30:29.883526+0000","last_clean":"2024-04-25T10:30:29.883526+0000","last_became_active":"2024-04-25T10:30:12.771923+0000","last_became_peered":"2024-04-25T10:30:12.771923+0000","last_unstale":"2024-04-25T10:30:29.883526+0000","last_undegraded":"2024-04-25T10:30:29.883526+0000","last_fullsized":"2024-04-25T10:30:29.883526+0000","mapping_epoch":34,"log_start":"0'0","ondisk_log_start":"0'0","created":34,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:30:11.733978+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:30:11.733978+0000","last_clean_scrub_stamp":"2024-04-25T10:30:11.733978+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T17:35:33.561138+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"6.1c","version":"0'0","reported_seq":12,"reported_epoch":44,"state":"active+clean","last_fresh":"2024-04-25T10:30:29.882853+0000","last_change":"2024-04-25T10:30:28.875422+0000","last_active":"2024-04-25T10:30:29.882853+0000","last_peered":"2024-04-25T10:30:29.882853+0000","last_clean":"2024-04-25T10:30:29.882853+0000","last_became_active":"2024-04-25T10:30:28.875079+0000","last_became_peered":"2024-04-25T10:30:28.875079+0000","last_unstale":"2024-04-25T10:30:29.882853+0000","last_undegraded":"2024-04-25T10:30:29.882853+0000","last_fullsized":"2024-04-25T10:30:29.882853+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":42,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:30:27.860024+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:30:27.860024+0000","last_clean_scrub_stamp":"2024-04-25T10:30:27.860024+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T13:23:55.301869+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.1f","version":"37'8","reported_seq":38,"reported_epoch":44,"state":"active+clean","last_fresh":"2024-04-25T10:30:29.882826+0000","last_change":"2024-04-25T10:30:11.756978+0000","last_active":"2024-04-25T10:30:29.882826+0000","last_peered":"2024-04-25T10:30:29.882826+0000","last_clean":"2024-04-25T10:30:29.882826+0000","last_became_active":"2024-04-25T10:30:11.756617+0000","last_became_peered":"2024-04-25T10:30:11.756617+0000","last_unstale":"2024-04-25T10:30:29.882826+0000","last_undegraded":"2024-04-25T10:30:29.882826+0000","last_fullsized":"2024-04-25T10:30:29.882826+0000","mapping_epoch":33,"log_start":"0'0","ondisk_log_start":"0'0","created":33,"last_epoch_clean":34,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:30:10.729165+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:30:10.729165+0000","last_clean_scrub_stamp":"2024-04-25T10:30:10.729165+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T12:27:53.559880+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":1,"num_read_kb":0,"num_write":10,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.1e","version":"36'5","reported_seq":32,"reported_epoch":44,"state":"active+clean","last_fresh":"2024-04-25T10:30:29.883469+0000","last_change":"2024-04-25T10:30:12.769800+0000","last_active":"2024-04-25T10:30:29.883469+0000","last_peered":"2024-04-25T10:30:29.883469+0000","last_clean":"2024-04-25T10:30:29.883469+0000","last_became_active":"2024-04-25T10:30:12.769470+0000","last_became_peered":"2024-04-25T10:30:12.769470+0000","last_unstale":"2024-04-25T10:30:29.883469+0000","last_undegraded":"2024-04-25T10:30:29.883469+0000","last_fullsized":"2024-04-25T10:30:29.883469+0000","mapping_epoch":34,"log_start":"0'0","ondisk_log_start":"0'0","created":34,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:30:11.733978+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:30:11.733978+0000","last_clean_scrub_stamp":"2024-04-25T10:30:11.733978+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T13:11:54.897586+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"6.1d","version":"0'0","reported_seq":12,"reported_epoch":44,"state":"active+clean","last_fresh":"2024-04-25T10:30:29.883295+0000","last_change":"2024-04-25T10:30:28.884277+0000","last_active":"2024-04-25T10:30:29.883295+0000","last_peered":"2024-04-25T10:30:29.883295+0000","last_clean":"2024-04-25T10:30:29.883295+0000","last_became_active":"2024-04-25T10:30:28.883764+0000","last_became_peered":"2024-04-25T10:30:28.883764+0000","last_unstale":"2024-04-25T10:30:29.883295+0000","last_undegraded":"2024-04-25T10:30:29.883295+0000","last_fullsized":"2024-04-25T10:30:29.883295+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":42,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:30:27.860024+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:30:27.860024+0000","last_clean_scrub_stamp":"2024-04-25T10:30:27.860024+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T18:36:07.211600+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"5.20","version":"36'9","reported_seq":38,"reported_epoch":44,"state":"active+clean","last_fresh":"2024-04-25T10:30:29.881619+0000","last_change":"2024-04-25T10:30:12.766692+0000","last_active":"2024-04-25T10:30:29.881619+0000","last_peered":"2024-04-25T10:30:29.881619+0000","last_clean":"2024-04-25T10:30:29.881619+0000","last_became_active":"2024-04-25T10:30:12.766336+0000","last_became_peered":"2024-04-25T10:30:12.766336+0000","last_unstale":"2024-04-25T10:30:29.881619+0000","last_undegraded":"2024-04-25T10:30:29.881619+0000","last_fullsized":"2024-04-25T10:30:29.881619+0000","mapping_epoch":34,"log_start":"0'0","ondisk_log_start":"0'0","created":34,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:30:11.733978+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:30:11.733978+0000","last_clean_scrub_stamp":"2024-04-25T10:30:11.733978+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T18:11:27.798982+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.21","version":"36'12","reported_seq":41,"reported_epoch":44,"state":"active+clean","last_fresh":"2024-04-25T10:30:29.882888+0000","last_change":"2024-04-25T10:30:12.774245+0000","last_active":"2024-04-25T10:30:29.882888+0000","last_peered":"2024-04-25T10:30:29.882888+0000","last_clean":"2024-04-25T10:30:29.882888+0000","last_became_active":"2024-04-25T10:30:12.773923+0000","last_became_peered":"2024-04-25T10:30:12.773923+0000","last_unstale":"2024-04-25T10:30:29.882888+0000","last_undegraded":"2024-04-25T10:30:29.882888+0000","last_fullsized":"2024-04-25T10:30:29.882888+0000","mapping_epoch":34,"log_start":"0'0","ondisk_log_start":"0'0","created":34,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:30:11.733978+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:30:11.733978+0000","last_clean_scrub_stamp":"2024-04-25T10:30:11.733978+0000","objects_scrubbed":0,"log_size":12,"log_dups_size":0,"ondisk_log_size":12,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T22:09:01.889527+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,7],"acting":[2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"5.22","version":"36'8","reported_seq":32,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-04-25T10:30:28.877192+0000","last_change":"2024-04-25T10:30:12.769958+0000","last_active":"2024-04-25T10:30:28.877192+0000","last_peered":"2024-04-25T10:30:28.877192+0000","last_clean":"2024-04-25T10:30:28.877192+0000","last_became_active":"2024-04-25T10:30:12.769590+0000","last_became_peered":"2024-04-25T10:30:12.769590+0000","last_unstale":"2024-04-25T10:30:28.877192+0000","last_undegraded":"2024-04-25T10:30:28.877192+0000","last_fullsized":"2024-04-25T10:30:28.877192+0000","mapping_epoch":34,"log_start":"0'0","ondisk_log_start":"0'0","created":34,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:30:11.733978+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:30:11.733978+0000","last_clean_scrub_stamp":"2024-04-25T10:30:11.733978+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T11:57:29.979255+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"5.23","version":"36'7","reported_seq":31,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-04-25T10:30:28.876441+0000","last_change":"2024-04-25T10:30:12.781995+0000","last_active":"2024-04-25T10:30:28.876441+0000","last_peered":"2024-04-25T10:30:28.876441+0000","last_clean":"2024-04-25T10:30:28.876441+0000","last_became_active":"2024-04-25T10:30:12.781358+0000","last_became_peered":"2024-04-25T10:30:12.781358+0000","last_unstale":"2024-04-25T10:30:28.876441+0000","last_undegraded":"2024-04-25T10:30:28.876441+0000","last_fullsized":"2024-04-25T10:30:28.876441+0000","mapping_epoch":34,"log_start":"0'0","ondisk_log_start":"0'0","created":34,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:30:11.733978+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:30:11.733978+0000","last_clean_scrub_stamp":"2024-04-25T10:30:11.733978+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T18:00:26.511360+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"5.24","version":"36'4","reported_seq":33,"reported_epoch":44,"state":"active+clean","last_fresh":"2024-04-25T10:30:29.883349+0000","last_change":"2024-04-25T10:30:12.775515+0000","last_active":"2024-04-25T10:30:29.883349+0000","last_peered":"2024-04-25T10:30:29.883349+0000","last_clean":"2024-04-25T10:30:29.883349+0000","last_became_active":"2024-04-25T10:30:12.775167+0000","last_became_peered":"2024-04-25T10:30:12.775167+0000","last_unstale":"2024-04-25T10:30:29.883349+0000","last_undegraded":"2024-04-25T10:30:29.883349+0000","last_fullsized":"2024-04-25T10:30:29.883349+0000","mapping_epoch":34,"log_start":"0'0","ondisk_log_start":"0'0","created":34,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:30:11.733978+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:30:11.733978+0000","last_clean_scrub_stamp":"2024-04-25T10:30:11.733978+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T21:35:23.984378+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,6],"acting":[2,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"5.25","version":"36'6","reported_seq":30,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-04-25T10:30:28.876545+0000","last_change":"2024-04-25T10:30:12.767505+0000","last_active":"2024-04-25T10:30:28.876545+0000","last_peered":"2024-04-25T10:30:28.876545+0000","last_clean":"2024-04-25T10:30:28.876545+0000","last_became_active":"2024-04-25T10:30:12.767149+0000","last_became_peered":"2024-04-25T10:30:12.767149+0000","last_unstale":"2024-04-25T10:30:28.876545+0000","last_undegraded":"2024-04-25T10:30:28.876545+0000","last_fullsized":"2024-04-25T10:30:28.876545+0000","mapping_epoch":34,"log_start":"0'0","ondisk_log_start":"0'0","created":34,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:30:11.733978+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:30:11.733978+0000","last_clean_scrub_stamp":"2024-04-25T10:30:11.733978+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T16:29:27.103512+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"5.26","version":"36'15","reported_seq":38,"reported_epoch":42,"state":"active+clean","last_fresh":"2024-04-25T10:30:27.877674+0000","last_change":"2024-04-25T10:30:12.771420+0000","last_active":"2024-04-25T10:30:27.877674+0000","last_peered":"2024-04-25T10:30:27.877674+0000","last_clean":"2024-04-25T10:30:27.877674+0000","last_became_active":"2024-04-25T10:30:12.771117+0000","last_became_peered":"2024-04-25T10:30:12.771117+0000","last_unstale":"2024-04-25T10:30:27.877674+0000","last_undegraded":"2024-04-25T10:30:27.877674+0000","last_fullsized":"2024-04-25T10:30:27.877674+0000","mapping_epoch":34,"log_start":"0'0","ondisk_log_start":"0'0","created":34,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:30:11.733978+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:30:11.733978+0000","last_clean_scrub_stamp":"2024-04-25T10:30:11.733978+0000","objects_scrubbed":0,"log_size":15,"log_dups_size":0,"ondisk_log_size":15,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T16:08:42.820761+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.27","version":"36'2","reported_seq":25,"reported_epoch":42,"state":"active+clean","last_fresh":"2024-04-25T10:30:27.873896+0000","last_change":"2024-04-25T10:30:12.779972+0000","last_active":"2024-04-25T10:30:27.873896+0000","last_peered":"2024-04-25T10:30:27.873896+0000","last_clean":"2024-04-25T10:30:27.873896+0000","last_became_active":"2024-04-25T10:30:12.779526+0000","last_became_peered":"2024-04-25T10:30:12.779526+0000","last_unstale":"2024-04-25T10:30:27.873896+0000","last_undegraded":"2024-04-25T10:30:27.873896+0000","last_fullsized":"2024-04-25T10:30:27.873896+0000","mapping_epoch":34,"log_start":"0'0","ondisk_log_start":"0'0","created":34,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:30:11.733978+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:30:11.733978+0000","last_clean_scrub_stamp":"2024-04-25T10:30:11.733978+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T22:04:21.879881+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.28","version":"36'6","reported_seq":35,"reported_epoch":44,"state":"active+clean","last_fresh":"2024-04-25T10:30:29.882628+0000","last_change":"2024-04-25T10:30:12.783394+0000","last_active":"2024-04-25T10:30:29.882628+0000","last_peered":"2024-04-25T10:30:29.882628+0000","last_clean":"2024-04-25T10:30:29.882628+0000","last_became_active":"2024-04-25T10:30:12.783066+0000","last_became_peered":"2024-04-25T10:30:12.783066+0000","last_unstale":"2024-04-25T10:30:29.882628+0000","last_undegraded":"2024-04-25T10:30:29.882628+0000","last_fullsized":"2024-04-25T10:30:29.882628+0000","mapping_epoch":34,"log_start":"0'0","ondisk_log_start":"0'0","created":34,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:30:11.733978+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:30:11.733978+0000","last_clean_scrub_stamp":"2024-04-25T10:30:11.733978+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T15:39:31.890064+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4],"acting":[2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"5.29","version":"36'8","reported_seq":35,"reported_epoch":44,"state":"active+clean","last_fresh":"2024-04-25T10:30:29.882807+0000","last_change":"2024-04-25T10:30:12.763911+0000","last_active":"2024-04-25T10:30:29.882807+0000","last_peered":"2024-04-25T10:30:29.882807+0000","last_clean":"2024-04-25T10:30:29.882807+0000","last_became_active":"2024-04-25T10:30:12.763578+0000","last_became_peered":"2024-04-25T10:30:12.763578+0000","last_unstale":"2024-04-25T10:30:29.882807+0000","last_undegraded":"2024-04-25T10:30:29.882807+0000","last_fullsized":"2024-04-25T10:30:29.882807+0000","mapping_epoch":34,"log_start":"0'0","ondisk_log_start":"0'0","created":34,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:30:11.733978+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:30:11.733978+0000","last_clean_scrub_stamp":"2024-04-25T10:30:11.733978+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T18:52:45.539758+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,1],"acting":[6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"5.2a","version":"36'9","reported_seq":38,"reported_epoch":44,"state":"active+clean","last_fresh":"2024-04-25T10:30:29.882996+0000","last_change":"2024-04-25T10:30:12.782730+0000","last_active":"2024-04-25T10:30:29.882996+0000","last_peered":"2024-04-25T10:30:29.882996+0000","last_clean":"2024-04-25T10:30:29.882996+0000","last_became_active":"2024-04-25T10:30:12.782261+0000","last_became_peered":"2024-04-25T10:30:12.782261+0000","last_unstale":"2024-04-25T10:30:29.882996+0000","last_undegraded":"2024-04-25T10:30:29.882996+0000","last_fullsized":"2024-04-25T10:30:29.882996+0000","mapping_epoch":34,"log_start":"0'0","ondisk_log_start":"0'0","created":34,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:30:11.733978+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:30:11.733978+0000","last_clean_scrub_stamp":"2024-04-25T10:30:11.733978+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T18:00:26.511360+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4],"acting":[2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"5.2b","version":"36'11","reported_seq":35,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-04-25T10:30:28.876504+0000","last_change":"2024-04-25T10:30:12.771344+0000","last_active":"2024-04-25T10:30:28.876504+0000","last_peered":"2024-04-25T10:30:28.876504+0000","last_clean":"2024-04-25T10:30:28.876504+0000","last_became_active":"2024-04-25T10:30:12.770949+0000","last_became_peered":"2024-04-25T10:30:12.770949+0000","last_unstale":"2024-04-25T10:30:28.876504+0000","last_undegraded":"2024-04-25T10:30:28.876504+0000","last_fullsized":"2024-04-25T10:30:28.876504+0000","mapping_epoch":34,"log_start":"0'0","ondisk_log_start":"0'0","created":34,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:30:11.733978+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:30:11.733978+0000","last_clean_scrub_stamp":"2024-04-25T10:30:11.733978+0000","objects_scrubbed":0,"log_size":11,"log_dups_size":0,"ondisk_log_size":11,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T16:29:27.103512+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"5.2c","version":"36'8","reported_seq":35,"reported_epoch":44,"state":"active+clean","last_fresh":"2024-04-25T10:30:29.883520+0000","last_change":"2024-04-25T10:30:12.772663+0000","last_active":"2024-04-25T10:30:29.883520+0000","last_peered":"2024-04-25T10:30:29.883520+0000","last_clean":"2024-04-25T10:30:29.883520+0000","last_became_active":"2024-04-25T10:30:12.772356+0000","last_became_peered":"2024-04-25T10:30:12.772356+0000","last_unstale":"2024-04-25T10:30:29.883520+0000","last_undegraded":"2024-04-25T10:30:29.883520+0000","last_fullsized":"2024-04-25T10:30:29.883520+0000","mapping_epoch":34,"log_start":"0'0","ondisk_log_start":"0'0","created":34,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:30:11.733978+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:30:11.733978+0000","last_clean_scrub_stamp":"2024-04-25T10:30:11.733978+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T12:21:33.158775+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"5.2d","version":"36'7","reported_seq":31,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-04-25T10:30:28.875358+0000","last_change":"2024-04-25T10:30:12.768688+0000","last_active":"2024-04-25T10:30:28.875358+0000","last_peered":"2024-04-25T10:30:28.875358+0000","last_clean":"2024-04-25T10:30:28.875358+0000","last_became_active":"2024-04-25T10:30:12.768382+0000","last_became_peered":"2024-04-25T10:30:12.768382+0000","last_unstale":"2024-04-25T10:30:28.875358+0000","last_undegraded":"2024-04-25T10:30:28.875358+0000","last_fullsized":"2024-04-25T10:30:28.875358+0000","mapping_epoch":34,"log_start":"0'0","ondisk_log_start":"0'0","created":34,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:30:11.733978+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:30:11.733978+0000","last_clean_scrub_stamp":"2024-04-25T10:30:11.733978+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T18:26:06.164667+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"5.2e","version":"36'10","reported_seq":34,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-04-25T10:30:28.876239+0000","last_change":"2024-04-25T10:30:12.766393+0000","last_active":"2024-04-25T10:30:28.876239+0000","last_peered":"2024-04-25T10:30:28.876239+0000","last_clean":"2024-04-25T10:30:28.876239+0000","last_became_active":"2024-04-25T10:30:12.766096+0000","last_became_peered":"2024-04-25T10:30:12.766096+0000","last_unstale":"2024-04-25T10:30:28.876239+0000","last_undegraded":"2024-04-25T10:30:28.876239+0000","last_fullsized":"2024-04-25T10:30:28.876239+0000","mapping_epoch":34,"log_start":"0'0","ondisk_log_start":"0'0","created":34,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:30:11.733978+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:30:11.733978+0000","last_clean_scrub_stamp":"2024-04-25T10:30:11.733978+0000","objects_scrubbed":0,"log_size":10,"log_dups_size":0,"ondisk_log_size":10,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T16:49:00.268423+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"5.2f","version":"36'9","reported_seq":38,"reported_epoch":44,"state":"active+clean","last_fresh":"2024-04-25T10:30:29.882690+0000","last_change":"2024-04-25T10:30:12.766616+0000","last_active":"2024-04-25T10:30:29.882690+0000","last_peered":"2024-04-25T10:30:29.882690+0000","last_clean":"2024-04-25T10:30:29.882690+0000","last_became_active":"2024-04-25T10:30:12.766290+0000","last_became_peered":"2024-04-25T10:30:12.766290+0000","last_unstale":"2024-04-25T10:30:29.882690+0000","last_undegraded":"2024-04-25T10:30:29.882690+0000","last_fullsized":"2024-04-25T10:30:29.882690+0000","mapping_epoch":34,"log_start":"0'0","ondisk_log_start":"0'0","created":34,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:30:11.733978+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:30:11.733978+0000","last_clean_scrub_stamp":"2024-04-25T10:30:11.733978+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T13:22:50.975199+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":2097152,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":2048,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.30","version":"36'9","reported_seq":33,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-04-25T10:30:28.876615+0000","last_change":"2024-04-25T10:30:12.765989+0000","last_active":"2024-04-25T10:30:28.876615+0000","last_peered":"2024-04-25T10:30:28.876615+0000","last_clean":"2024-04-25T10:30:28.876615+0000","last_became_active":"2024-04-25T10:30:12.765648+0000","last_became_peered":"2024-04-25T10:30:12.765648+0000","last_unstale":"2024-04-25T10:30:28.876615+0000","last_undegraded":"2024-04-25T10:30:28.876615+0000","last_fullsized":"2024-04-25T10:30:28.876615+0000","mapping_epoch":34,"log_start":"0'0","ondisk_log_start":"0'0","created":34,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:30:11.733978+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:30:11.733978+0000","last_clean_scrub_stamp":"2024-04-25T10:30:11.733978+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T16:29:40.621322+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"5.31","version":"36'7","reported_seq":31,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-04-25T10:30:28.877047+0000","last_change":"2024-04-25T10:30:12.777324+0000","last_active":"2024-04-25T10:30:28.877047+0000","last_peered":"2024-04-25T10:30:28.877047+0000","last_clean":"2024-04-25T10:30:28.877047+0000","last_became_active":"2024-04-25T10:30:12.776983+0000","last_became_peered":"2024-04-25T10:30:12.776983+0000","last_unstale":"2024-04-25T10:30:28.877047+0000","last_undegraded":"2024-04-25T10:30:28.877047+0000","last_fullsized":"2024-04-25T10:30:28.877047+0000","mapping_epoch":34,"log_start":"0'0","ondisk_log_start":"0'0","created":34,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:30:11.733978+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:30:11.733978+0000","last_clean_scrub_stamp":"2024-04-25T10:30:11.733978+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T20:33:24.458243+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"5.32","version":"36'8","reported_seq":31,"reported_epoch":42,"state":"active+clean","last_fresh":"2024-04-25T10:30:27.876591+0000","last_change":"2024-04-25T10:30:12.772243+0000","last_active":"2024-04-25T10:30:27.876591+0000","last_peered":"2024-04-25T10:30:27.876591+0000","last_clean":"2024-04-25T10:30:27.876591+0000","last_became_active":"2024-04-25T10:30:12.771936+0000","last_became_peered":"2024-04-25T10:30:12.771936+0000","last_unstale":"2024-04-25T10:30:27.876591+0000","last_undegraded":"2024-04-25T10:30:27.876591+0000","last_fullsized":"2024-04-25T10:30:27.876591+0000","mapping_epoch":34,"log_start":"0'0","ondisk_log_start":"0'0","created":34,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:30:11.733978+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:30:11.733978+0000","last_clean_scrub_stamp":"2024-04-25T10:30:11.733978+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T13:11:22.432926+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5],"acting":[4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.33","version":"36'10","reported_seq":37,"reported_epoch":44,"state":"active+clean","last_fresh":"2024-04-25T10:30:29.883324+0000","last_change":"2024-04-25T10:30:12.777499+0000","last_active":"2024-04-25T10:30:29.883324+0000","last_peered":"2024-04-25T10:30:29.883324+0000","last_clean":"2024-04-25T10:30:29.883324+0000","last_became_active":"2024-04-25T10:30:12.776773+0000","last_became_peered":"2024-04-25T10:30:12.776773+0000","last_unstale":"2024-04-25T10:30:29.883324+0000","last_undegraded":"2024-04-25T10:30:29.883324+0000","last_fullsized":"2024-04-25T10:30:29.883324+0000","mapping_epoch":34,"log_start":"0'0","ondisk_log_start":"0'0","created":34,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:30:11.733978+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:30:11.733978+0000","last_clean_scrub_stamp":"2024-04-25T10:30:11.733978+0000","objects_scrubbed":0,"log_size":10,"log_dups_size":0,"ondisk_log_size":10,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T18:21:32.178524+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"5.34","version":"36'7","reported_seq":32,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-04-25T10:30:28.874618+0000","last_change":"2024-04-25T10:30:12.761162+0000","last_active":"2024-04-25T10:30:28.874618+0000","last_peered":"2024-04-25T10:30:28.874618+0000","last_clean":"2024-04-25T10:30:28.874618+0000","last_became_active":"2024-04-25T10:30:12.760835+0000","last_became_peered":"2024-04-25T10:30:12.760835+0000","last_unstale":"2024-04-25T10:30:28.874618+0000","last_undegraded":"2024-04-25T10:30:28.874618+0000","last_fullsized":"2024-04-25T10:30:28.874618+0000","mapping_epoch":34,"log_start":"0'0","ondisk_log_start":"0'0","created":34,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:30:11.733978+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:30:11.733978+0000","last_clean_scrub_stamp":"2024-04-25T10:30:11.733978+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T18:59:20.693761+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"5.35","version":"36'6","reported_seq":35,"reported_epoch":44,"state":"active+clean","last_fresh":"2024-04-25T10:30:29.882303+0000","last_change":"2024-04-25T10:30:12.776164+0000","last_active":"2024-04-25T10:30:29.882303+0000","last_peered":"2024-04-25T10:30:29.882303+0000","last_clean":"2024-04-25T10:30:29.882303+0000","last_became_active":"2024-04-25T10:30:12.775455+0000","last_became_peered":"2024-04-25T10:30:12.775455+0000","last_unstale":"2024-04-25T10:30:29.882303+0000","last_undegraded":"2024-04-25T10:30:29.882303+0000","last_fullsized":"2024-04-25T10:30:29.882303+0000","mapping_epoch":34,"log_start":"0'0","ondisk_log_start":"0'0","created":34,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:30:11.733978+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:30:11.733978+0000","last_clean_scrub_stamp":"2024-04-25T10:30:11.733978+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T21:45:24.780979+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.36","version":"36'15","reported_seq":38,"reported_epoch":42,"state":"active+clean","last_fresh":"2024-04-25T10:30:27.877229+0000","last_change":"2024-04-25T10:30:12.773138+0000","last_active":"2024-04-25T10:30:27.877229+0000","last_peered":"2024-04-25T10:30:27.877229+0000","last_clean":"2024-04-25T10:30:27.877229+0000","last_became_active":"2024-04-25T10:30:12.772854+0000","last_became_peered":"2024-04-25T10:30:12.772854+0000","last_unstale":"2024-04-25T10:30:27.877229+0000","last_undegraded":"2024-04-25T10:30:27.877229+0000","last_fullsized":"2024-04-25T10:30:27.877229+0000","mapping_epoch":34,"log_start":"0'0","ondisk_log_start":"0'0","created":34,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:30:11.733978+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:30:11.733978+0000","last_clean_scrub_stamp":"2024-04-25T10:30:11.733978+0000","objects_scrubbed":0,"log_size":15,"log_dups_size":0,"ondisk_log_size":15,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T19:24:03.893622+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5],"acting":[4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.37","version":"36'11","reported_seq":36,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-04-25T10:30:28.875064+0000","last_change":"2024-04-25T10:30:12.773184+0000","last_active":"2024-04-25T10:30:28.875064+0000","last_peered":"2024-04-25T10:30:28.875064+0000","last_clean":"2024-04-25T10:30:28.875064+0000","last_became_active":"2024-04-25T10:30:12.772866+0000","last_became_peered":"2024-04-25T10:30:12.772866+0000","last_unstale":"2024-04-25T10:30:28.875064+0000","last_undegraded":"2024-04-25T10:30:28.875064+0000","last_fullsized":"2024-04-25T10:30:28.875064+0000","mapping_epoch":34,"log_start":"0'0","ondisk_log_start":"0'0","created":34,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:30:11.733978+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:30:11.733978+0000","last_clean_scrub_stamp":"2024-04-25T10:30:11.733978+0000","objects_scrubbed":0,"log_size":11,"log_dups_size":0,"ondisk_log_size":11,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T14:01:06.791796+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"5.38","version":"36'9","reported_seq":38,"reported_epoch":44,"state":"active+clean","last_fresh":"2024-04-25T10:30:29.882611+0000","last_change":"2024-04-25T10:30:12.774614+0000","last_active":"2024-04-25T10:30:29.882611+0000","last_peered":"2024-04-25T10:30:29.882611+0000","last_clean":"2024-04-25T10:30:29.882611+0000","last_became_active":"2024-04-25T10:30:12.774213+0000","last_became_peered":"2024-04-25T10:30:12.774213+0000","last_unstale":"2024-04-25T10:30:29.882611+0000","last_undegraded":"2024-04-25T10:30:29.882611+0000","last_fullsized":"2024-04-25T10:30:29.882611+0000","mapping_epoch":34,"log_start":"0'0","ondisk_log_start":"0'0","created":34,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:30:11.733978+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:30:11.733978+0000","last_clean_scrub_stamp":"2024-04-25T10:30:11.733978+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T20:33:24.458243+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,6],"acting":[2,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"5.39","version":"36'5","reported_seq":34,"reported_epoch":44,"state":"active+clean","last_fresh":"2024-04-25T10:30:29.882763+0000","last_change":"2024-04-25T10:30:12.781210+0000","last_active":"2024-04-25T10:30:29.882763+0000","last_peered":"2024-04-25T10:30:29.882763+0000","last_clean":"2024-04-25T10:30:29.882763+0000","last_became_active":"2024-04-25T10:30:12.780874+0000","last_became_peered":"2024-04-25T10:30:12.780874+0000","last_unstale":"2024-04-25T10:30:29.882763+0000","last_undegraded":"2024-04-25T10:30:29.882763+0000","last_fullsized":"2024-04-25T10:30:29.882763+0000","mapping_epoch":34,"log_start":"0'0","ondisk_log_start":"0'0","created":34,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:30:11.733978+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:30:11.733978+0000","last_clean_scrub_stamp":"2024-04-25T10:30:11.733978+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T16:29:40.621322+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"5.3a","version":"36'6","reported_seq":30,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-04-25T10:30:28.874520+0000","last_change":"2024-04-25T10:30:12.775534+0000","last_active":"2024-04-25T10:30:28.874520+0000","last_peered":"2024-04-25T10:30:28.874520+0000","last_clean":"2024-04-25T10:30:28.874520+0000","last_became_active":"2024-04-25T10:30:12.775180+0000","last_became_peered":"2024-04-25T10:30:12.775180+0000","last_unstale":"2024-04-25T10:30:28.874520+0000","last_undegraded":"2024-04-25T10:30:28.874520+0000","last_fullsized":"2024-04-25T10:30:28.874520+0000","mapping_epoch":34,"log_start":"0'0","ondisk_log_start":"0'0","created":34,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:30:11.733978+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:30:11.733978+0000","last_clean_scrub_stamp":"2024-04-25T10:30:11.733978+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T18:31:49.260513+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"5.3b","version":"36'4","reported_seq":28,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-04-25T10:30:28.875655+0000","last_change":"2024-04-25T10:30:12.762141+0000","last_active":"2024-04-25T10:30:28.875655+0000","last_peered":"2024-04-25T10:30:28.875655+0000","last_clean":"2024-04-25T10:30:28.875655+0000","last_became_active":"2024-04-25T10:30:12.761805+0000","last_became_peered":"2024-04-25T10:30:12.761805+0000","last_unstale":"2024-04-25T10:30:28.875655+0000","last_undegraded":"2024-04-25T10:30:28.875655+0000","last_fullsized":"2024-04-25T10:30:28.875655+0000","mapping_epoch":34,"log_start":"0'0","ondisk_log_start":"0'0","created":34,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:30:11.733978+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:30:11.733978+0000","last_clean_scrub_stamp":"2024-04-25T10:30:11.733978+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T19:56:57.765496+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6],"acting":[5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"5.3c","version":"36'8","reported_seq":31,"reported_epoch":42,"state":"active+clean","last_fresh":"2024-04-25T10:30:27.877379+0000","last_change":"2024-04-25T10:30:12.781723+0000","last_active":"2024-04-25T10:30:27.877379+0000","last_peered":"2024-04-25T10:30:27.877379+0000","last_clean":"2024-04-25T10:30:27.877379+0000","last_became_active":"2024-04-25T10:30:12.781357+0000","last_became_peered":"2024-04-25T10:30:12.781357+0000","last_unstale":"2024-04-25T10:30:27.877379+0000","last_undegraded":"2024-04-25T10:30:27.877379+0000","last_fullsized":"2024-04-25T10:30:27.877379+0000","mapping_epoch":34,"log_start":"0'0","ondisk_log_start":"0'0","created":34,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:30:11.733978+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:30:11.733978+0000","last_clean_scrub_stamp":"2024-04-25T10:30:11.733978+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T12:06:29.897651+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.3d","version":"36'7","reported_seq":32,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-04-25T10:30:28.875135+0000","last_change":"2024-04-25T10:30:12.768292+0000","last_active":"2024-04-25T10:30:28.875135+0000","last_peered":"2024-04-25T10:30:28.875135+0000","last_clean":"2024-04-25T10:30:28.875135+0000","last_became_active":"2024-04-25T10:30:12.767909+0000","last_became_peered":"2024-04-25T10:30:12.767909+0000","last_unstale":"2024-04-25T10:30:28.875135+0000","last_undegraded":"2024-04-25T10:30:28.875135+0000","last_fullsized":"2024-04-25T10:30:28.875135+0000","mapping_epoch":34,"log_start":"0'0","ondisk_log_start":"0'0","created":34,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:30:11.733978+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:30:11.733978+0000","last_clean_scrub_stamp":"2024-04-25T10:30:11.733978+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T15:17:30.002665+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"5.3e","version":"36'4","reported_seq":28,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-04-25T10:30:28.878982+0000","last_change":"2024-04-25T10:30:12.780210+0000","last_active":"2024-04-25T10:30:28.878982+0000","last_peered":"2024-04-25T10:30:28.878982+0000","last_clean":"2024-04-25T10:30:28.878982+0000","last_became_active":"2024-04-25T10:30:12.779867+0000","last_became_peered":"2024-04-25T10:30:12.779867+0000","last_unstale":"2024-04-25T10:30:28.878982+0000","last_undegraded":"2024-04-25T10:30:28.878982+0000","last_fullsized":"2024-04-25T10:30:28.878982+0000","mapping_epoch":34,"log_start":"0'0","ondisk_log_start":"0'0","created":34,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:30:11.733978+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:30:11.733978+0000","last_clean_scrub_stamp":"2024-04-25T10:30:11.733978+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T20:33:24.458243+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"5.3f","version":"36'8","reported_seq":37,"reported_epoch":44,"state":"active+clean","last_fresh":"2024-04-25T10:30:29.883448+0000","last_change":"2024-04-25T10:30:12.772509+0000","last_active":"2024-04-25T10:30:29.883448+0000","last_peered":"2024-04-25T10:30:29.883448+0000","last_clean":"2024-04-25T10:30:29.883448+0000","last_became_active":"2024-04-25T10:30:12.772018+0000","last_became_peered":"2024-04-25T10:30:12.772018+0000","last_unstale":"2024-04-25T10:30:29.883448+0000","last_undegraded":"2024-04-25T10:30:29.883448+0000","last_fullsized":"2024-04-25T10:30:29.883448+0000","mapping_epoch":34,"log_start":"0'0","ondisk_log_start":"0'0","created":34,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:30:11.733978+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:30:11.733978+0000","last_clean_scrub_stamp":"2024-04-25T10:30:11.733978+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T11:57:29.979255+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]}],"pool_stats":[{"poolid":6,"num_pg":32,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":64,"acting":64,"num_store_stats":8},{"poolid":5,"num_pg":64,"stat_sum":{"num_bytes":6291456,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":6144,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":12582912,"data_stored":12582912,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":504,"ondisk_log_size":504,"up":128,"acting":128,"num_store_stats":8},{"poolid":4,"num_pg":32,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":1,"num_read_kb":0,"num_write":96,"num_write_kb":40,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":83,"ondisk_log_size":83,"up":64,"acting":64,"num_store_stats":8},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":66,"num_read_kb":55,"num_write":101,"num_write_kb":1628,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1187840,"data_stored":1180736,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":72,"ondisk_log_size":72,"up":2,"acting":2,"num_store_stats":2}],"osd_stats":[{"osd":7,"up_from":15,"seq":64424509462,"num_pgs":21,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":31596,"kb_used_data":4572,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94340244,"statfs":{"total":96636764160,"available":96604409856,"internally_reserved":0,"allocated":4681728,"data_stored":4394877,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1590,"internal_metadata":27654602},"hb_peers":[0,1,2,3,4,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]},{"osd":6,"up_from":15,"seq":64424509463,"num_pgs":25,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27504,"kb_used_data":476,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344336,"statfs":{"total":96636764160,"available":96608600064,"internally_reserved":0,"allocated":487424,"data_stored":200573,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":27654598},"hb_peers":[0,1,2,3,4,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]},{"osd":5,"up_from":15,"seq":64424509464,"num_pgs":18,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":28084,"kb_used_data":1056,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94343756,"statfs":{"total":96636764160,"available":96608006144,"internally_reserved":0,"allocated":1081344,"data_stored":790941,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]},{"osd":3,"up_from":15,"seq":64424509464,"num_pgs":25,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":33644,"kb_used_data":6620,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94338196,"statfs":{"total":96636764160,"available":96602312704,"internally_reserved":0,"allocated":6778880,"data_stored":6492029,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1593,"internal_metadata":27654599},"hb_peers":[0,1,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]},{"osd":2,"up_from":15,"seq":64424509464,"num_pgs":27,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":29548,"kb_used_data":2524,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94342292,"statfs":{"total":96636764160,"available":96606507008,"internally_reserved":0,"allocated":2584576,"data_stored":2297725,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]},{"osd":1,"up_from":15,"seq":64424509462,"num_pgs":25,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":28088,"kb_used_data":1056,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94343752,"statfs":{"total":96636764160,"available":96608002048,"internally_reserved":0,"allocated":1081344,"data_stored":790941,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]},{"osd":0,"up_from":15,"seq":64424509463,"num_pgs":21,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27504,"kb_used_data":476,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344336,"statfs":{"total":96636764160,"available":96608600064,"internally_reserved":0,"allocated":487424,"data_stored":200573,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[1,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]},{"osd":4,"up_from":13,"seq":55834574870,"num_pgs":35,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27480,"kb_used_data":452,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344360,"statfs":{"total":96636764160,"available":96608624640,"internally_reserved":0,"allocated":462848,"data_stored":186484,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":2097152,"data_stored":2097152,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":6291456,"data_stored":6291456,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":4194304,"data_stored":4194304,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-04-25T10:30:32.949 DEBUG:teuthology.orchestra.run.smithi037:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-04-25T10:30:33.096 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:33.094+0000 7fe686f3f640 1 Processor -- start 2024-04-25T10:30:33.096 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:33.094+0000 7fe686f3f640 1 -- start start 2024-04-25T10:30:33.098 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:33.095+0000 7fe686f3f640 1 --2- >> v2:172.21.15.37:3300/0 conn(0x7fe6800b4bb0 0x7fe6801646b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:30:33.098 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:33.095+0000 7fe686f3f640 1 --2- >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7fe6800d4560 0x7fe6800d4970 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:30:33.098 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:33.095+0000 7fe686f3f640 1 -- --> v1:172.21.15.203:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fe680161190 con 0x7fe6800b7600 2024-04-25T10:30:33.098 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:33.095+0000 7fe686f3f640 1 -- --> v2:172.21.15.37:3300/0 -- mon_getmap magic: 0 -- 0x7fe680161470 con 0x7fe6800b4bb0 2024-04-25T10:30:33.098 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:33.095+0000 7fe686f3f640 1 -- --> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] -- mon_getmap magic: 0 -- 0x7fe680163eb0 con 0x7fe6800d4560 2024-04-25T10:30:33.098 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:33.095+0000 7fe684cdb640 1 --2- >> v2:172.21.15.37:3300/0 conn(0x7fe6800b4bb0 0x7fe6801646b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:30:33.098 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:33.095+0000 7fe684cdb640 1 --2- >> v2:172.21.15.37:3300/0 conn(0x7fe6800b4bb0 0x7fe6801646b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.37:3300/0 says I am v2:172.21.15.37:55424/0 (socket says 172.21.15.37:55424) 2024-04-25T10:30:33.098 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:33.095+0000 7fe684cdb640 1 -- 172.21.15.37:0/2259501645 learned_addr learned my addr 172.21.15.37:0/2259501645 (peer_addr_for_me v2:172.21.15.37:0/0) 2024-04-25T10:30:33.098 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:33.096+0000 7fe6854dc640 1 --2- 172.21.15.37:0/2259501645 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7fe6800d4560 0x7fe6800d4970 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:30:33.099 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:33.096+0000 7fe684cdb640 1 -- 172.21.15.37:0/2259501645 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7fe6800d4560 msgr2=0x7fe6800d4970 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:30:33.099 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:33.096+0000 7fe684cdb640 1 --2- 172.21.15.37:0/2259501645 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7fe6800d4560 0x7fe6800d4970 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:30:33.099 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:33.096+0000 7fe684cdb640 1 -- 172.21.15.37:0/2259501645 >> v1:172.21.15.203:6789/0 conn(0x7fe6800b7600 legacy=0x7fe6800b7a10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:30:33.099 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:33.096+0000 7fe684cdb640 1 -- 172.21.15.37:0/2259501645 --> v2:172.21.15.37:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fe680164190 con 0x7fe6800b4bb0 2024-04-25T10:30:33.100 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:33.096+0000 7fe684cdb640 1 --2- 172.21.15.37:0/2259501645 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fe6800b4bb0 0x7fe6801646b0 secure :-1 s=READY pgs=532 cs=0 l=1 rev1=1 crypto rx=0x7fe674009870 tx=0x7fe67402a210 comp rx=0 tx=0).ready entity=mon.0 client_cookie=41b1a860e688299 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:30:33.100 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:33.097+0000 7fe67f7fe640 1 -- 172.21.15.37:0/2259501645 <== mon.0 v2:172.21.15.37:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe674037070 con 0x7fe6800b4bb0 2024-04-25T10:30:33.100 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:33.097+0000 7fe67f7fe640 1 -- 172.21.15.37:0/2259501645 <== mon.0 v2:172.21.15.37:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fe674032440 con 0x7fe6800b4bb0 2024-04-25T10:30:33.100 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:33.097+0000 7fe67f7fe640 1 -- 172.21.15.37:0/2259501645 <== mon.0 v2:172.21.15.37:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe674032810 con 0x7fe6800b4bb0 2024-04-25T10:30:33.100 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:33.097+0000 7fe686f3f640 1 -- 172.21.15.37:0/2259501645 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fe6800b4bb0 msgr2=0x7fe6801646b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:30:33.100 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:33.097+0000 7fe686f3f640 1 --2- 172.21.15.37:0/2259501645 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fe6800b4bb0 0x7fe6801646b0 secure :-1 s=READY pgs=532 cs=0 l=1 rev1=1 crypto rx=0x7fe674009870 tx=0x7fe67402a210 comp rx=0 tx=0).stop 2024-04-25T10:30:33.100 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:33.097+0000 7fe686f3f640 1 -- 172.21.15.37:0/2259501645 shutdown_connections 2024-04-25T10:30:33.100 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:33.097+0000 7fe686f3f640 1 --2- 172.21.15.37:0/2259501645 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7fe6800d4560 0x7fe6800d4970 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:30:33.100 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:33.097+0000 7fe686f3f640 1 --2- 172.21.15.37:0/2259501645 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fe6800b4bb0 0x7fe6801646b0 unknown :-1 s=CLOSED pgs=532 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:30:33.101 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:33.097+0000 7fe686f3f640 1 -- 172.21.15.37:0/2259501645 >> 172.21.15.37:0/2259501645 conn(0x7fe6800d27e0 msgr2=0x7fe6800d2bf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:30:33.101 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:33.097+0000 7fe686f3f640 1 -- 172.21.15.37:0/2259501645 shutdown_connections 2024-04-25T10:30:33.101 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:33.097+0000 7fe686f3f640 1 -- 172.21.15.37:0/2259501645 wait complete. 2024-04-25T10:30:33.101 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:33.099+0000 7fe686f3f640 1 Processor -- start 2024-04-25T10:30:33.101 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:33.099+0000 7fe686f3f640 1 -- start start 2024-04-25T10:30:33.104 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:33.100+0000 7fe686f3f640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fe6800b4bb0 0x7fe6800d0050 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:30:33.104 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:33.100+0000 7fe686f3f640 1 --2- >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7fe6800b7600 0x7fe68015a3c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:30:33.104 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:33.100+0000 7fe686f3f640 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fe6800d4560 0x7fe68015a900 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:30:33.104 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:33.100+0000 7fe686f3f640 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 -- 0x7fe6800d09e0 con 0x7fe6800b4bb0 2024-04-25T10:30:33.104 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:33.100+0000 7fe686f3f640 1 -- --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_getmap magic: 0 -- 0x7fe680164bf0 con 0x7fe6800d4560 2024-04-25T10:30:33.104 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:33.100+0000 7fe686f3f640 1 -- --> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] -- mon_getmap magic: 0 -- 0x7fe6800d0b60 con 0x7fe6800b7600 2024-04-25T10:30:33.104 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:33.101+0000 7fe684cdb640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fe6800b4bb0 0x7fe6800d0050 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:30:33.104 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:33.101+0000 7fe684cdb640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fe6800b4bb0 0x7fe6800d0050 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.37:3300/0 says I am v2:172.21.15.37:55438/0 (socket says 172.21.15.37:55438) 2024-04-25T10:30:33.104 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:33.101+0000 7fe684cdb640 1 -- 172.21.15.37:0/2488448319 learned_addr learned my addr 172.21.15.37:0/2488448319 (peer_addr_for_me v2:172.21.15.37:0/0) 2024-04-25T10:30:33.105 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:33.101+0000 7fe67ffff640 1 --2- 172.21.15.37:0/2488448319 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7fe6800b7600 0x7fe68015a3c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:30:33.105 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:33.101+0000 7fe6854dc640 1 --2- 172.21.15.37:0/2488448319 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fe6800d4560 0x7fe68015a900 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:30:33.105 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:33.101+0000 7fe684cdb640 1 -- 172.21.15.37:0/2488448319 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7fe6800b7600 msgr2=0x7fe68015a3c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:30:33.105 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:33.101+0000 7fe684cdb640 1 --2- 172.21.15.37:0/2488448319 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7fe6800b7600 0x7fe68015a3c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:30:33.105 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:33.101+0000 7fe684cdb640 1 -- 172.21.15.37:0/2488448319 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fe6800d4560 msgr2=0x7fe68015a900 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:30:33.105 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:33.101+0000 7fe684cdb640 1 --2- 172.21.15.37:0/2488448319 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fe6800d4560 0x7fe68015a900 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:30:33.105 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:33.101+0000 7fe684cdb640 1 -- 172.21.15.37:0/2488448319 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fe68014abd0 con 0x7fe6800b4bb0 2024-04-25T10:30:33.105 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:33.101+0000 7fe684cdb640 1 --2- 172.21.15.37:0/2488448319 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fe6800b4bb0 0x7fe6800d0050 secure :-1 s=READY pgs=533 cs=0 l=1 rev1=1 crypto rx=0x7fe674009870 tx=0x7fe6740046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:30:33.105 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:33.101+0000 7fe67d7fa640 1 -- 172.21.15.37:0/2488448319 <== mon.0 v2:172.21.15.37:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe674041020 con 0x7fe6800b4bb0 2024-04-25T10:30:33.105 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:33.101+0000 7fe67d7fa640 1 -- 172.21.15.37:0/2488448319 <== mon.0 v2:172.21.15.37:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fe6740306c0 con 0x7fe6800b4bb0 2024-04-25T10:30:33.105 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:33.101+0000 7fe686f3f640 1 -- 172.21.15.37:0/2488448319 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fe68015b140 con 0x7fe6800b4bb0 2024-04-25T10:30:33.105 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:33.101+0000 7fe67d7fa640 1 -- 172.21.15.37:0/2488448319 <== mon.0 v2:172.21.15.37:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe674037040 con 0x7fe6800b4bb0 2024-04-25T10:30:33.105 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:33.101+0000 7fe686f3f640 1 -- 172.21.15.37:0/2488448319 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fe68015b700 con 0x7fe6800b4bb0 2024-04-25T10:30:33.106 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:33.102+0000 7fe686f3f640 1 -- 172.21.15.37:0/2488448319 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fe68006a140 con 0x7fe6800b4bb0 2024-04-25T10:30:33.106 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:33.103+0000 7fe67d7fa640 1 -- 172.21.15.37:0/2488448319 <== mon.0 v2:172.21.15.37:3300/0 4 ==== mgrmap(e 5) ==== 96782+0+0 (secure 0 0 0) 0x7fe674044020 con 0x7fe6800b4bb0 2024-04-25T10:30:33.106 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:33.103+0000 7fe67d7fa640 1 --2- 172.21.15.37:0/2488448319 >> [v2:172.21.15.203:6832/2418205237,v1:172.21.15.203:6833/2418205237] conn(0x7fe6640744e0 0x7fe6640769f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:30:33.107 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:33.103+0000 7fe67d7fa640 1 -- 172.21.15.37:0/2488448319 <== mon.0 v2:172.21.15.37:3300/0 5 ==== osd_map(44..44 src has 1..44) ==== 6248+0+0 (secure 0 0 0) 0x7fe6740b5ca0 con 0x7fe6800b4bb0 2024-04-25T10:30:33.107 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:33.104+0000 7fe67ffff640 1 --2- 172.21.15.37:0/2488448319 >> [v2:172.21.15.203:6832/2418205237,v1:172.21.15.203:6833/2418205237] conn(0x7fe6640744e0 0x7fe6640769f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:30:33.107 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:33.104+0000 7fe67ffff640 1 --2- 172.21.15.37:0/2488448319 >> [v2:172.21.15.203:6832/2418205237,v1:172.21.15.203:6833/2418205237] conn(0x7fe6640744e0 0x7fe6640769f0 secure :-1 s=READY pgs=255 cs=0 l=1 rev1=1 crypto rx=0x7fe66c006fd0 tx=0x7fe66c008040 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:30:33.110 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:33.108+0000 7fe67d7fa640 1 -- 172.21.15.37:0/2488448319 <== mon.0 v2:172.21.15.37:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187246 (secure 0 0 0) 0x7fe67404b030 con 0x7fe6800b4bb0 2024-04-25T10:30:33.291 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:33.289+0000 7fe686f3f640 1 -- 172.21.15.37:0/2488448319 --> [v2:172.21.15.203:6832/2418205237,v1:172.21.15.203:6833/2418205237] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) -- 0x7fe6800d0590 con 0x7fe6640744e0 2024-04-25T10:30:33.300 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:33.298+0000 7fe67d7fa640 1 -- 172.21.15.37:0/2488448319 <== mgr.4100 v2:172.21.15.203:6832/2418205237 1 ==== mgr_command_reply(tid 0: 0 dumped all) ==== 18+0+332031 (secure 0 0 0) 0x7fe66c00bda0 con 0x7fe6640744e0 2024-04-25T10:30:33.304 INFO:teuthology.orchestra.run.smithi037.stdout: 2024-04-25T10:30:33.305 INFO:teuthology.orchestra.run.smithi037.stderr:dumped all 2024-04-25T10:30:33.307 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:33.301+0000 7fe686f3f640 1 -- 172.21.15.37:0/2488448319 >> [v2:172.21.15.203:6832/2418205237,v1:172.21.15.203:6833/2418205237] conn(0x7fe6640744e0 msgr2=0x7fe6640769f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:30:33.307 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:33.301+0000 7fe686f3f640 1 --2- 172.21.15.37:0/2488448319 >> [v2:172.21.15.203:6832/2418205237,v1:172.21.15.203:6833/2418205237] conn(0x7fe6640744e0 0x7fe6640769f0 secure :-1 s=READY pgs=255 cs=0 l=1 rev1=1 crypto rx=0x7fe66c006fd0 tx=0x7fe66c008040 comp rx=0 tx=0).stop 2024-04-25T10:30:33.307 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:33.301+0000 7fe686f3f640 1 -- 172.21.15.37:0/2488448319 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fe6800b4bb0 msgr2=0x7fe6800d0050 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:30:33.307 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:33.301+0000 7fe686f3f640 1 --2- 172.21.15.37:0/2488448319 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fe6800b4bb0 0x7fe6800d0050 secure :-1 s=READY pgs=533 cs=0 l=1 rev1=1 crypto rx=0x7fe674009870 tx=0x7fe6740046c0 comp rx=0 tx=0).stop 2024-04-25T10:30:33.307 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:33.301+0000 7fe686f3f640 1 -- 172.21.15.37:0/2488448319 shutdown_connections 2024-04-25T10:30:33.307 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:33.301+0000 7fe686f3f640 1 --2- 172.21.15.37:0/2488448319 >> [v2:172.21.15.203:6832/2418205237,v1:172.21.15.203:6833/2418205237] conn(0x7fe6640744e0 0x7fe6640769f0 unknown :-1 s=CLOSED pgs=255 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:30:33.307 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:33.301+0000 7fe686f3f640 1 --2- 172.21.15.37:0/2488448319 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fe6800d4560 0x7fe68015a900 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:30:33.307 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:33.301+0000 7fe686f3f640 1 --2- 172.21.15.37:0/2488448319 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7fe6800b7600 0x7fe68015a3c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:30:33.307 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:33.301+0000 7fe686f3f640 1 --2- 172.21.15.37:0/2488448319 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fe6800b4bb0 0x7fe6800d0050 unknown :-1 s=CLOSED pgs=533 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:30:33.307 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:33.301+0000 7fe686f3f640 1 -- 172.21.15.37:0/2488448319 >> 172.21.15.37:0/2488448319 conn(0x7fe6800d27e0 msgr2=0x7fe680163080 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:30:33.308 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:33.301+0000 7fe686f3f640 1 -- 172.21.15.37:0/2488448319 shutdown_connections 2024-04-25T10:30:33.308 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:33.301+0000 7fe686f3f640 1 -- 172.21.15.37:0/2488448319 wait complete. 2024-04-25T10:30:33.315 INFO:teuthology.orchestra.run.smithi037.stdout:{"pg_ready":true,"pg_map":{"version":87,"stamp":"2024-04-25T10:30:32.207732+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":6881824,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":67,"num_read_kb":55,"num_write":201,"num_write_kb":7812,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":659,"ondisk_log_size":659,"up":258,"acting":258,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":197,"num_osds":8,"num_per_pool_osds":8,"num_per_pool_omap_osds":8,"kb":754974720,"kb_used":233448,"kb_used_data":17232,"kb_used_omap":12,"kb_used_meta":216051,"kb_avail":754741272,"statfs":{"total":773094113280,"available":772855062528,"internally_reserved":0,"allocated":17645568,"data_stored":15354143,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":12752,"internal_metadata":221236784},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":14,"apply_latency_ms":14,"commit_latency_ns":14000000,"apply_latency_ns":14000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":-16365,"num_objects":-23,"num_object_clones":0,"num_object_copies":-46,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":-23,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":23,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":-15,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"6.001255"},"pg_stats":[{"pgid":"1.0","version":"18'72","reported_seq":145,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-04-25T10:30:28.875202+0000","last_change":"2024-04-25T10:28:57.972696+0000","last_active":"2024-04-25T10:30:28.875202+0000","last_peered":"2024-04-25T10:30:28.875202+0000","last_clean":"2024-04-25T10:30:28.875202+0000","last_became_active":"2024-04-25T10:28:57.972373+0000","last_became_peered":"2024-04-25T10:28:57.972373+0000","last_unstale":"2024-04-25T10:30:28.875202+0000","last_undegraded":"2024-04-25T10:30:28.875202+0000","last_fullsized":"2024-04-25T10:30:28.875202+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:28:56.959439+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:28:56.959439+0000","last_clean_scrub_stamp":"2024-04-25T10:28:56.959439+0000","objects_scrubbed":0,"log_size":72,"log_dups_size":0,"ondisk_log_size":72,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T17:34:34.057408+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":66,"num_read_kb":55,"num_write":101,"num_write_kb":1628,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.5","version":"0'0","reported_seq":30,"reported_epoch":44,"state":"active+clean","last_fresh":"2024-04-25T10:30:29.882532+0000","last_change":"2024-04-25T10:30:11.759459+0000","last_active":"2024-04-25T10:30:29.882532+0000","last_peered":"2024-04-25T10:30:29.882532+0000","last_clean":"2024-04-25T10:30:29.882532+0000","last_became_active":"2024-04-25T10:30:11.757880+0000","last_became_peered":"2024-04-25T10:30:11.757880+0000","last_unstale":"2024-04-25T10:30:29.882532+0000","last_undegraded":"2024-04-25T10:30:29.882532+0000","last_fullsized":"2024-04-25T10:30:29.882532+0000","mapping_epoch":33,"log_start":"0'0","ondisk_log_start":"0'0","created":33,"last_epoch_clean":34,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:30:10.729165+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:30:10.729165+0000","last_clean_scrub_stamp":"2024-04-25T10:30:10.729165+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T18:21:31.173712+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.4","version":"36'9","reported_seq":33,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-04-25T10:30:28.874920+0000","last_change":"2024-04-25T10:30:12.773161+0000","last_active":"2024-04-25T10:30:28.874920+0000","last_peered":"2024-04-25T10:30:28.874920+0000","last_clean":"2024-04-25T10:30:28.874920+0000","last_became_active":"2024-04-25T10:30:12.772831+0000","last_became_peered":"2024-04-25T10:30:12.772831+0000","last_unstale":"2024-04-25T10:30:28.874920+0000","last_undegraded":"2024-04-25T10:30:28.874920+0000","last_fullsized":"2024-04-25T10:30:28.874920+0000","mapping_epoch":34,"log_start":"0'0","ondisk_log_start":"0'0","created":34,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:30:11.733978+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:30:11.733978+0000","last_clean_scrub_stamp":"2024-04-25T10:30:11.733978+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T19:56:57.765496+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"6.7","version":"0'0","reported_seq":12,"reported_epoch":44,"state":"active+clean","last_fresh":"2024-04-25T10:30:29.882905+0000","last_change":"2024-04-25T10:30:28.881711+0000","last_active":"2024-04-25T10:30:29.882905+0000","last_peered":"2024-04-25T10:30:29.882905+0000","last_clean":"2024-04-25T10:30:29.882905+0000","last_became_active":"2024-04-25T10:30:28.881407+0000","last_became_peered":"2024-04-25T10:30:28.881407+0000","last_unstale":"2024-04-25T10:30:29.882905+0000","last_undegraded":"2024-04-25T10:30:29.882905+0000","last_fullsized":"2024-04-25T10:30:29.882905+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":42,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:30:27.860024+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:30:27.860024+0000","last_clean_scrub_stamp":"2024-04-25T10:30:27.860024+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T16:30:29.338622+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.0","version":"0'0","reported_seq":25,"reported_epoch":42,"state":"active+clean","last_fresh":"2024-04-25T10:30:27.875763+0000","last_change":"2024-04-25T10:30:11.761764+0000","last_active":"2024-04-25T10:30:27.875763+0000","last_peered":"2024-04-25T10:30:27.875763+0000","last_clean":"2024-04-25T10:30:27.875763+0000","last_became_active":"2024-04-25T10:30:11.761281+0000","last_became_peered":"2024-04-25T10:30:11.761281+0000","last_unstale":"2024-04-25T10:30:27.875763+0000","last_undegraded":"2024-04-25T10:30:27.875763+0000","last_fullsized":"2024-04-25T10:30:27.875763+0000","mapping_epoch":33,"log_start":"0'0","ondisk_log_start":"0'0","created":33,"last_epoch_clean":34,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:30:10.729165+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:30:10.729165+0000","last_clean_scrub_stamp":"2024-04-25T10:30:10.729165+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T13:05:33.012421+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.1","version":"36'5","reported_seq":34,"reported_epoch":44,"state":"active+clean","last_fresh":"2024-04-25T10:30:29.881926+0000","last_change":"2024-04-25T10:30:12.769006+0000","last_active":"2024-04-25T10:30:29.881926+0000","last_peered":"2024-04-25T10:30:29.881926+0000","last_clean":"2024-04-25T10:30:29.881926+0000","last_became_active":"2024-04-25T10:30:12.768671+0000","last_became_peered":"2024-04-25T10:30:12.768671+0000","last_unstale":"2024-04-25T10:30:29.881926+0000","last_undegraded":"2024-04-25T10:30:29.881926+0000","last_fullsized":"2024-04-25T10:30:29.881926+0000","mapping_epoch":34,"log_start":"0'0","ondisk_log_start":"0'0","created":34,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:30:11.733978+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:30:11.733978+0000","last_clean_scrub_stamp":"2024-04-25T10:30:11.733978+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T15:00:26.818072+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"6.2","version":"0'0","reported_seq":12,"reported_epoch":44,"state":"active+clean","last_fresh":"2024-04-25T10:30:29.882577+0000","last_change":"2024-04-25T10:30:28.875554+0000","last_active":"2024-04-25T10:30:29.882577+0000","last_peered":"2024-04-25T10:30:29.882577+0000","last_clean":"2024-04-25T10:30:29.882577+0000","last_became_active":"2024-04-25T10:30:28.875249+0000","last_became_peered":"2024-04-25T10:30:28.875249+0000","last_unstale":"2024-04-25T10:30:29.882577+0000","last_undegraded":"2024-04-25T10:30:29.882577+0000","last_fullsized":"2024-04-25T10:30:29.882577+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":42,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:30:27.860024+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:30:27.860024+0000","last_clean_scrub_stamp":"2024-04-25T10:30:27.860024+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T17:20:57.234773+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.1","version":"0'0","reported_seq":30,"reported_epoch":44,"state":"active+clean","last_fresh":"2024-04-25T10:30:29.882995+0000","last_change":"2024-04-25T10:30:11.765156+0000","last_active":"2024-04-25T10:30:29.882995+0000","last_peered":"2024-04-25T10:30:29.882995+0000","last_clean":"2024-04-25T10:30:29.882995+0000","last_became_active":"2024-04-25T10:30:11.764704+0000","last_became_peered":"2024-04-25T10:30:11.764704+0000","last_unstale":"2024-04-25T10:30:29.882995+0000","last_undegraded":"2024-04-25T10:30:29.882995+0000","last_fullsized":"2024-04-25T10:30:29.882995+0000","mapping_epoch":33,"log_start":"0'0","ondisk_log_start":"0'0","created":33,"last_epoch_clean":34,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:30:10.729165+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:30:10.729165+0000","last_clean_scrub_stamp":"2024-04-25T10:30:10.729165+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T19:02:53.584992+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"5.0","version":"36'8","reported_seq":31,"reported_epoch":42,"state":"active+clean","last_fresh":"2024-04-25T10:30:27.875633+0000","last_change":"2024-04-25T10:30:12.781642+0000","last_active":"2024-04-25T10:30:27.875633+0000","last_peered":"2024-04-25T10:30:27.875633+0000","last_clean":"2024-04-25T10:30:27.875633+0000","last_became_active":"2024-04-25T10:30:12.781303+0000","last_became_peered":"2024-04-25T10:30:12.781303+0000","last_unstale":"2024-04-25T10:30:27.875633+0000","last_undegraded":"2024-04-25T10:30:27.875633+0000","last_fullsized":"2024-04-25T10:30:27.875633+0000","mapping_epoch":34,"log_start":"0'0","ondisk_log_start":"0'0","created":34,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:30:11.733978+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:30:11.733978+0000","last_clean_scrub_stamp":"2024-04-25T10:30:11.733978+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T12:21:06.638842+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"6.3","version":"0'0","reported_seq":9,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-04-25T10:30:28.885536+0000","last_change":"2024-04-25T10:30:28.883569+0000","last_active":"2024-04-25T10:30:28.885536+0000","last_peered":"2024-04-25T10:30:28.885536+0000","last_clean":"2024-04-25T10:30:28.885536+0000","last_became_active":"2024-04-25T10:30:28.883246+0000","last_became_peered":"2024-04-25T10:30:28.883246+0000","last_unstale":"2024-04-25T10:30:28.885536+0000","last_undegraded":"2024-04-25T10:30:28.885536+0000","last_fullsized":"2024-04-25T10:30:28.885536+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":42,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:30:27.860024+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:30:27.860024+0000","last_clean_scrub_stamp":"2024-04-25T10:30:27.860024+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T14:04:58.646968+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.2","version":"0'0","reported_seq":28,"reported_epoch":44,"state":"active+clean","last_fresh":"2024-04-25T10:30:29.882904+0000","last_change":"2024-04-25T10:30:11.760769+0000","last_active":"2024-04-25T10:30:29.882904+0000","last_peered":"2024-04-25T10:30:29.882904+0000","last_clean":"2024-04-25T10:30:29.882904+0000","last_became_active":"2024-04-25T10:30:11.760425+0000","last_became_peered":"2024-04-25T10:30:11.760425+0000","last_unstale":"2024-04-25T10:30:29.882904+0000","last_undegraded":"2024-04-25T10:30:29.882904+0000","last_fullsized":"2024-04-25T10:30:29.882904+0000","mapping_epoch":33,"log_start":"0'0","ondisk_log_start":"0'0","created":33,"last_epoch_clean":34,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:30:10.729165+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:30:10.729165+0000","last_clean_scrub_stamp":"2024-04-25T10:30:10.729165+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T17:24:13.848319+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"5.3","version":"36'7","reported_seq":30,"reported_epoch":42,"state":"active+clean","last_fresh":"2024-04-25T10:30:27.876884+0000","last_change":"2024-04-25T10:30:12.775959+0000","last_active":"2024-04-25T10:30:27.876884+0000","last_peered":"2024-04-25T10:30:27.876884+0000","last_clean":"2024-04-25T10:30:27.876884+0000","last_became_active":"2024-04-25T10:30:12.774532+0000","last_became_peered":"2024-04-25T10:30:12.774532+0000","last_unstale":"2024-04-25T10:30:27.876884+0000","last_undegraded":"2024-04-25T10:30:27.876884+0000","last_fullsized":"2024-04-25T10:30:27.876884+0000","mapping_epoch":34,"log_start":"0'0","ondisk_log_start":"0'0","created":34,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:30:11.733978+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:30:11.733978+0000","last_clean_scrub_stamp":"2024-04-25T10:30:11.733978+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T11:38:09.717996+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"6.0","version":"0'0","reported_seq":9,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-04-25T10:30:28.884563+0000","last_change":"2024-04-25T10:30:28.883801+0000","last_active":"2024-04-25T10:30:28.884563+0000","last_peered":"2024-04-25T10:30:28.884563+0000","last_clean":"2024-04-25T10:30:28.884563+0000","last_became_active":"2024-04-25T10:30:28.883485+0000","last_became_peered":"2024-04-25T10:30:28.883485+0000","last_unstale":"2024-04-25T10:30:28.884563+0000","last_undegraded":"2024-04-25T10:30:28.884563+0000","last_fullsized":"2024-04-25T10:30:28.884563+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":42,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:30:27.860024+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:30:27.860024+0000","last_clean_scrub_stamp":"2024-04-25T10:30:27.860024+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T14:16:58.845516+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.3","version":"0'0","reported_seq":28,"reported_epoch":44,"state":"active+clean","last_fresh":"2024-04-25T10:30:29.883455+0000","last_change":"2024-04-25T10:30:11.762286+0000","last_active":"2024-04-25T10:30:29.883455+0000","last_peered":"2024-04-25T10:30:29.883455+0000","last_clean":"2024-04-25T10:30:29.883455+0000","last_became_active":"2024-04-25T10:30:11.761975+0000","last_became_peered":"2024-04-25T10:30:11.761975+0000","last_unstale":"2024-04-25T10:30:29.883455+0000","last_undegraded":"2024-04-25T10:30:29.883455+0000","last_fullsized":"2024-04-25T10:30:29.883455+0000","mapping_epoch":33,"log_start":"0'0","ondisk_log_start":"0'0","created":33,"last_epoch_clean":34,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:30:10.729165+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:30:10.729165+0000","last_clean_scrub_stamp":"2024-04-25T10:30:10.729165+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T16:09:04.081750+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"5.2","version":"36'8","reported_seq":32,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-04-25T10:30:28.877071+0000","last_change":"2024-04-25T10:30:12.765903+0000","last_active":"2024-04-25T10:30:28.877071+0000","last_peered":"2024-04-25T10:30:28.877071+0000","last_clean":"2024-04-25T10:30:28.877071+0000","last_became_active":"2024-04-25T10:30:12.765611+0000","last_became_peered":"2024-04-25T10:30:12.765611+0000","last_unstale":"2024-04-25T10:30:28.877071+0000","last_undegraded":"2024-04-25T10:30:28.877071+0000","last_fullsized":"2024-04-25T10:30:28.877071+0000","mapping_epoch":34,"log_start":"0'0","ondisk_log_start":"0'0","created":34,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:30:11.733978+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:30:11.733978+0000","last_clean_scrub_stamp":"2024-04-25T10:30:11.733978+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T22:09:01.889527+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"6.1","version":"0'0","reported_seq":12,"reported_epoch":44,"state":"active+clean","last_fresh":"2024-04-25T10:30:29.882707+0000","last_change":"2024-04-25T10:30:28.884434+0000","last_active":"2024-04-25T10:30:29.882707+0000","last_peered":"2024-04-25T10:30:29.882707+0000","last_clean":"2024-04-25T10:30:29.882707+0000","last_became_active":"2024-04-25T10:30:28.883801+0000","last_became_peered":"2024-04-25T10:30:28.883801+0000","last_unstale":"2024-04-25T10:30:29.882707+0000","last_undegraded":"2024-04-25T10:30:29.882707+0000","last_fullsized":"2024-04-25T10:30:29.882707+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":42,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:30:27.860024+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:30:27.860024+0000","last_clean_scrub_stamp":"2024-04-25T10:30:27.860024+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T11:57:46.105302+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.4","version":"0'0","reported_seq":28,"reported_epoch":44,"state":"active+clean","last_fresh":"2024-04-25T10:30:29.883736+0000","last_change":"2024-04-25T10:30:11.757601+0000","last_active":"2024-04-25T10:30:29.883736+0000","last_peered":"2024-04-25T10:30:29.883736+0000","last_clean":"2024-04-25T10:30:29.883736+0000","last_became_active":"2024-04-25T10:30:11.756626+0000","last_became_peered":"2024-04-25T10:30:11.756626+0000","last_unstale":"2024-04-25T10:30:29.883736+0000","last_undegraded":"2024-04-25T10:30:29.883736+0000","last_fullsized":"2024-04-25T10:30:29.883736+0000","mapping_epoch":33,"log_start":"0'0","ondisk_log_start":"0'0","created":33,"last_epoch_clean":34,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:30:10.729165+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:30:10.729165+0000","last_clean_scrub_stamp":"2024-04-25T10:30:10.729165+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T13:03:08.676049+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"5.5","version":"36'5","reported_seq":29,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-04-25T10:30:28.875752+0000","last_change":"2024-04-25T10:30:12.782521+0000","last_active":"2024-04-25T10:30:28.875752+0000","last_peered":"2024-04-25T10:30:28.875752+0000","last_clean":"2024-04-25T10:30:28.875752+0000","last_became_active":"2024-04-25T10:30:12.781802+0000","last_became_peered":"2024-04-25T10:30:12.781802+0000","last_unstale":"2024-04-25T10:30:28.875752+0000","last_undegraded":"2024-04-25T10:30:28.875752+0000","last_fullsized":"2024-04-25T10:30:28.875752+0000","mapping_epoch":34,"log_start":"0'0","ondisk_log_start":"0'0","created":34,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:30:11.733978+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:30:11.733978+0000","last_clean_scrub_stamp":"2024-04-25T10:30:11.733978+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T15:17:30.002665+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"6.6","version":"0'0","reported_seq":9,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-04-25T10:30:28.886860+0000","last_change":"2024-04-25T10:30:28.885896+0000","last_active":"2024-04-25T10:30:28.886860+0000","last_peered":"2024-04-25T10:30:28.886860+0000","last_clean":"2024-04-25T10:30:28.886860+0000","last_became_active":"2024-04-25T10:30:28.885575+0000","last_became_peered":"2024-04-25T10:30:28.885575+0000","last_unstale":"2024-04-25T10:30:28.886860+0000","last_undegraded":"2024-04-25T10:30:28.886860+0000","last_fullsized":"2024-04-25T10:30:28.886860+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":42,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:30:27.860024+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:30:27.860024+0000","last_clean_scrub_stamp":"2024-04-25T10:30:27.860024+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T22:09:18.015574+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.6","version":"37'4","reported_seq":29,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-04-25T10:30:28.879222+0000","last_change":"2024-04-25T10:30:11.750312+0000","last_active":"2024-04-25T10:30:28.879222+0000","last_peered":"2024-04-25T10:30:28.879222+0000","last_clean":"2024-04-25T10:30:28.879222+0000","last_became_active":"2024-04-25T10:30:11.750024+0000","last_became_peered":"2024-04-25T10:30:11.750024+0000","last_unstale":"2024-04-25T10:30:28.879222+0000","last_undegraded":"2024-04-25T10:30:28.879222+0000","last_fullsized":"2024-04-25T10:30:28.879222+0000","mapping_epoch":33,"log_start":"0'0","ondisk_log_start":"0'0","created":33,"last_epoch_clean":34,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:30:10.729165+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:30:10.729165+0000","last_clean_scrub_stamp":"2024-04-25T10:30:10.729165+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T16:48:59.263610+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"5.7","version":"36'8","reported_seq":37,"reported_epoch":44,"state":"active+clean","last_fresh":"2024-04-25T10:30:29.882715+0000","last_change":"2024-04-25T10:30:12.768932+0000","last_active":"2024-04-25T10:30:29.882715+0000","last_peered":"2024-04-25T10:30:29.882715+0000","last_clean":"2024-04-25T10:30:29.882715+0000","last_became_active":"2024-04-25T10:30:12.768609+0000","last_became_peered":"2024-04-25T10:30:12.768609+0000","last_unstale":"2024-04-25T10:30:29.882715+0000","last_undegraded":"2024-04-25T10:30:29.882715+0000","last_fullsized":"2024-04-25T10:30:29.882715+0000","mapping_epoch":34,"log_start":"0'0","ondisk_log_start":"0'0","created":34,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:30:11.733978+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:30:11.733978+0000","last_clean_scrub_stamp":"2024-04-25T10:30:11.733978+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T11:24:05.006967+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":4194304,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":4096,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"6.4","version":"0'0","reported_seq":9,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-04-25T10:30:28.881060+0000","last_change":"2024-04-25T10:30:28.880128+0000","last_active":"2024-04-25T10:30:28.881060+0000","last_peered":"2024-04-25T10:30:28.881060+0000","last_clean":"2024-04-25T10:30:28.881060+0000","last_became_active":"2024-04-25T10:30:28.879771+0000","last_became_peered":"2024-04-25T10:30:28.879771+0000","last_unstale":"2024-04-25T10:30:28.881060+0000","last_undegraded":"2024-04-25T10:30:28.881060+0000","last_fullsized":"2024-04-25T10:30:28.881060+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":42,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:30:27.860024+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:30:27.860024+0000","last_clean_scrub_stamp":"2024-04-25T10:30:27.860024+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T11:33:01.743970+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.7","version":"37'3","reported_seq":31,"reported_epoch":44,"state":"active+clean","last_fresh":"2024-04-25T10:30:29.883302+0000","last_change":"2024-04-25T10:30:11.756948+0000","last_active":"2024-04-25T10:30:29.883302+0000","last_peered":"2024-04-25T10:30:29.883302+0000","last_clean":"2024-04-25T10:30:29.883302+0000","last_became_active":"2024-04-25T10:30:11.756607+0000","last_became_peered":"2024-04-25T10:30:11.756607+0000","last_unstale":"2024-04-25T10:30:29.883302+0000","last_undegraded":"2024-04-25T10:30:29.883302+0000","last_fullsized":"2024-04-25T10:30:29.883302+0000","mapping_epoch":33,"log_start":"0'0","ondisk_log_start":"0'0","created":33,"last_epoch_clean":34,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:30:10.729165+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:30:10.729165+0000","last_clean_scrub_stamp":"2024-04-25T10:30:10.729165+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T17:15:13.597075+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"5.6","version":"36'7","reported_seq":31,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-04-25T10:30:28.876259+0000","last_change":"2024-04-25T10:30:12.772456+0000","last_active":"2024-04-25T10:30:28.876259+0000","last_peered":"2024-04-25T10:30:28.876259+0000","last_clean":"2024-04-25T10:30:28.876259+0000","last_became_active":"2024-04-25T10:30:12.772070+0000","last_became_peered":"2024-04-25T10:30:12.772070+0000","last_unstale":"2024-04-25T10:30:28.876259+0000","last_undegraded":"2024-04-25T10:30:28.876259+0000","last_fullsized":"2024-04-25T10:30:28.876259+0000","mapping_epoch":34,"log_start":"0'0","ondisk_log_start":"0'0","created":34,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:30:11.733978+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:30:11.733978+0000","last_clean_scrub_stamp":"2024-04-25T10:30:11.733978+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T16:08:29.987819+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"6.5","version":"0'0","reported_seq":9,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-04-25T10:30:28.882547+0000","last_change":"2024-04-25T10:30:28.881260+0000","last_active":"2024-04-25T10:30:28.882547+0000","last_peered":"2024-04-25T10:30:28.882547+0000","last_clean":"2024-04-25T10:30:28.882547+0000","last_became_active":"2024-04-25T10:30:28.880995+0000","last_became_peered":"2024-04-25T10:30:28.880995+0000","last_unstale":"2024-04-25T10:30:28.882547+0000","last_undegraded":"2024-04-25T10:30:28.882547+0000","last_fullsized":"2024-04-25T10:30:28.882547+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":42,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:30:27.860024+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:30:27.860024+0000","last_clean_scrub_stamp":"2024-04-25T10:30:27.860024+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T15:39:48.016111+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.8","version":"0'0","reported_seq":30,"reported_epoch":44,"state":"active+clean","last_fresh":"2024-04-25T10:30:29.881839+0000","last_change":"2024-04-25T10:30:11.758581+0000","last_active":"2024-04-25T10:30:29.881839+0000","last_peered":"2024-04-25T10:30:29.881839+0000","last_clean":"2024-04-25T10:30:29.881839+0000","last_became_active":"2024-04-25T10:30:11.758239+0000","last_became_peered":"2024-04-25T10:30:11.758239+0000","last_unstale":"2024-04-25T10:30:29.881839+0000","last_undegraded":"2024-04-25T10:30:29.881839+0000","last_fullsized":"2024-04-25T10:30:29.881839+0000","mapping_epoch":33,"log_start":"0'0","ondisk_log_start":"0'0","created":33,"last_epoch_clean":34,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:30:10.729165+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:30:10.729165+0000","last_clean_scrub_stamp":"2024-04-25T10:30:10.729165+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T17:41:33.905382+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.9","version":"36'2","reported_seq":27,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-04-25T10:30:28.874577+0000","last_change":"2024-04-25T10:30:12.768782+0000","last_active":"2024-04-25T10:30:28.874577+0000","last_peered":"2024-04-25T10:30:28.874577+0000","last_clean":"2024-04-25T10:30:28.874577+0000","last_became_active":"2024-04-25T10:30:12.768464+0000","last_became_peered":"2024-04-25T10:30:12.768464+0000","last_unstale":"2024-04-25T10:30:28.874577+0000","last_undegraded":"2024-04-25T10:30:28.874577+0000","last_fullsized":"2024-04-25T10:30:28.874577+0000","mapping_epoch":34,"log_start":"0'0","ondisk_log_start":"0'0","created":34,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:30:11.733978+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:30:11.733978+0000","last_clean_scrub_stamp":"2024-04-25T10:30:11.733978+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T13:03:21.170197+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"6.a","version":"0'0","reported_seq":9,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-04-25T10:30:28.884130+0000","last_change":"2024-04-25T10:30:28.882147+0000","last_active":"2024-04-25T10:30:28.884130+0000","last_peered":"2024-04-25T10:30:28.884130+0000","last_clean":"2024-04-25T10:30:28.884130+0000","last_became_active":"2024-04-25T10:30:28.881783+0000","last_became_peered":"2024-04-25T10:30:28.881783+0000","last_unstale":"2024-04-25T10:30:28.884130+0000","last_undegraded":"2024-04-25T10:30:28.884130+0000","last_fullsized":"2024-04-25T10:30:28.884130+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":42,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:30:27.860024+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:30:27.860024+0000","last_clean_scrub_stamp":"2024-04-25T10:30:27.860024+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T18:36:07.211600+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.9","version":"0'0","reported_seq":26,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-04-25T10:30:28.874568+0000","last_change":"2024-04-25T10:30:11.759807+0000","last_active":"2024-04-25T10:30:28.874568+0000","last_peered":"2024-04-25T10:30:28.874568+0000","last_clean":"2024-04-25T10:30:28.874568+0000","last_became_active":"2024-04-25T10:30:11.758558+0000","last_became_peered":"2024-04-25T10:30:11.758558+0000","last_unstale":"2024-04-25T10:30:28.874568+0000","last_undegraded":"2024-04-25T10:30:28.874568+0000","last_fullsized":"2024-04-25T10:30:28.874568+0000","mapping_epoch":33,"log_start":"0'0","ondisk_log_start":"0'0","created":33,"last_epoch_clean":34,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:30:10.729165+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:30:10.729165+0000","last_clean_scrub_stamp":"2024-04-25T10:30:10.729165+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T20:38:23.460836+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"5.8","version":"36'9","reported_seq":36,"reported_epoch":44,"state":"active+clean","last_fresh":"2024-04-25T10:30:29.882627+0000","last_change":"2024-04-25T10:30:12.777060+0000","last_active":"2024-04-25T10:30:29.882627+0000","last_peered":"2024-04-25T10:30:29.882627+0000","last_clean":"2024-04-25T10:30:29.882627+0000","last_became_active":"2024-04-25T10:30:12.776749+0000","last_became_peered":"2024-04-25T10:30:12.776749+0000","last_unstale":"2024-04-25T10:30:29.882627+0000","last_undegraded":"2024-04-25T10:30:29.882627+0000","last_fullsized":"2024-04-25T10:30:29.882627+0000","mapping_epoch":34,"log_start":"0'0","ondisk_log_start":"0'0","created":34,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:30:11.733978+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:30:11.733978+0000","last_clean_scrub_stamp":"2024-04-25T10:30:11.733978+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T15:47:40.336983+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"6.b","version":"0'0","reported_seq":1,"reported_epoch":42,"state":"creating+peering","last_fresh":"2024-04-25T10:30:27.876378+0000","last_change":"2024-04-25T10:30:27.876378+0000","last_active":"2024-04-25T10:30:27.860024+0000","last_peered":"2024-04-25T10:30:27.860024+0000","last_clean":"2024-04-25T10:30:27.860024+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T10:30:27.876378+0000","last_undegraded":"2024-04-25T10:30:27.876378+0000","last_fullsized":"2024-04-25T10:30:27.876378+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":42,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:30:27.860024+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:30:27.860024+0000","last_clean_scrub_stamp":"2024-04-25T10:30:27.860024+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.a","version":"0'0","reported_seq":30,"reported_epoch":44,"state":"active+clean","last_fresh":"2024-04-25T10:30:29.883112+0000","last_change":"2024-04-25T10:30:11.762745+0000","last_active":"2024-04-25T10:30:29.883112+0000","last_peered":"2024-04-25T10:30:29.883112+0000","last_clean":"2024-04-25T10:30:29.883112+0000","last_became_active":"2024-04-25T10:30:11.762377+0000","last_became_peered":"2024-04-25T10:30:11.762377+0000","last_unstale":"2024-04-25T10:30:29.883112+0000","last_undegraded":"2024-04-25T10:30:29.883112+0000","last_fullsized":"2024-04-25T10:30:29.883112+0000","mapping_epoch":33,"log_start":"0'0","ondisk_log_start":"0'0","created":33,"last_epoch_clean":34,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:30:10.729165+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:30:10.729165+0000","last_clean_scrub_stamp":"2024-04-25T10:30:10.729165+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T17:15:13.597075+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"5.b","version":"36'8","reported_seq":32,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-04-25T10:30:28.875847+0000","last_change":"2024-04-25T10:30:12.766544+0000","last_active":"2024-04-25T10:30:28.875847+0000","last_peered":"2024-04-25T10:30:28.875847+0000","last_clean":"2024-04-25T10:30:28.875847+0000","last_became_active":"2024-04-25T10:30:12.766228+0000","last_became_peered":"2024-04-25T10:30:12.766228+0000","last_unstale":"2024-04-25T10:30:28.875847+0000","last_undegraded":"2024-04-25T10:30:28.875847+0000","last_fullsized":"2024-04-25T10:30:28.875847+0000","mapping_epoch":34,"log_start":"0'0","ondisk_log_start":"0'0","created":34,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:30:11.733978+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:30:11.733978+0000","last_clean_scrub_stamp":"2024-04-25T10:30:11.733978+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T18:00:26.511360+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"6.8","version":"0'0","reported_seq":12,"reported_epoch":44,"state":"active+clean","last_fresh":"2024-04-25T10:30:29.881772+0000","last_change":"2024-04-25T10:30:28.881118+0000","last_active":"2024-04-25T10:30:29.881772+0000","last_peered":"2024-04-25T10:30:29.881772+0000","last_clean":"2024-04-25T10:30:29.881772+0000","last_became_active":"2024-04-25T10:30:28.880810+0000","last_became_peered":"2024-04-25T10:30:28.880810+0000","last_unstale":"2024-04-25T10:30:29.881772+0000","last_undegraded":"2024-04-25T10:30:29.881772+0000","last_fullsized":"2024-04-25T10:30:29.881772+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":42,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:30:27.860024+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:30:27.860024+0000","last_clean_scrub_stamp":"2024-04-25T10:30:27.860024+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T15:53:38.989476+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.b","version":"37'4","reported_seq":29,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-04-25T10:30:28.875996+0000","last_change":"2024-04-25T10:30:11.752902+0000","last_active":"2024-04-25T10:30:28.875996+0000","last_peered":"2024-04-25T10:30:28.875996+0000","last_clean":"2024-04-25T10:30:28.875996+0000","last_became_active":"2024-04-25T10:30:11.752599+0000","last_became_peered":"2024-04-25T10:30:11.752599+0000","last_unstale":"2024-04-25T10:30:28.875996+0000","last_undegraded":"2024-04-25T10:30:28.875996+0000","last_fullsized":"2024-04-25T10:30:28.875996+0000","mapping_epoch":33,"log_start":"0'0","ondisk_log_start":"0'0","created":33,"last_epoch_clean":34,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:30:10.729165+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:30:10.729165+0000","last_clean_scrub_stamp":"2024-04-25T10:30:10.729165+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T14:01:05.786983+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"5.a","version":"36'5","reported_seq":32,"reported_epoch":44,"state":"active+clean","last_fresh":"2024-04-25T10:30:29.882793+0000","last_change":"2024-04-25T10:30:12.762940+0000","last_active":"2024-04-25T10:30:29.882793+0000","last_peered":"2024-04-25T10:30:29.882793+0000","last_clean":"2024-04-25T10:30:29.882793+0000","last_became_active":"2024-04-25T10:30:12.762628+0000","last_became_peered":"2024-04-25T10:30:12.762628+0000","last_unstale":"2024-04-25T10:30:29.882793+0000","last_undegraded":"2024-04-25T10:30:29.882793+0000","last_fullsized":"2024-04-25T10:30:29.882793+0000","mapping_epoch":34,"log_start":"0'0","ondisk_log_start":"0'0","created":34,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:30:11.733978+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:30:11.733978+0000","last_clean_scrub_stamp":"2024-04-25T10:30:11.733978+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T19:25:06.397830+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"6.9","version":"0'0","reported_seq":1,"reported_epoch":42,"state":"creating+peering","last_fresh":"2024-04-25T10:30:27.871775+0000","last_change":"2024-04-25T10:30:27.871775+0000","last_active":"2024-04-25T10:30:27.860024+0000","last_peered":"2024-04-25T10:30:27.860024+0000","last_clean":"2024-04-25T10:30:27.860024+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T10:30:27.871775+0000","last_undegraded":"2024-04-25T10:30:27.871775+0000","last_fullsized":"2024-04-25T10:30:27.871775+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":42,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:30:27.860024+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:30:27.860024+0000","last_clean_scrub_stamp":"2024-04-25T10:30:27.860024+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.c","version":"0'0","reported_seq":25,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-04-25T10:30:28.875034+0000","last_change":"2024-04-25T10:30:11.753067+0000","last_active":"2024-04-25T10:30:28.875034+0000","last_peered":"2024-04-25T10:30:28.875034+0000","last_clean":"2024-04-25T10:30:28.875034+0000","last_became_active":"2024-04-25T10:30:11.752595+0000","last_became_peered":"2024-04-25T10:30:11.752595+0000","last_unstale":"2024-04-25T10:30:28.875034+0000","last_undegraded":"2024-04-25T10:30:28.875034+0000","last_fullsized":"2024-04-25T10:30:28.875034+0000","mapping_epoch":33,"log_start":"0'0","ondisk_log_start":"0'0","created":33,"last_epoch_clean":34,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:30:10.729165+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:30:10.729165+0000","last_clean_scrub_stamp":"2024-04-25T10:30:10.729165+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T18:59:19.688948+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"5.d","version":"36'7","reported_seq":31,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-04-25T10:30:28.875563+0000","last_change":"2024-04-25T10:30:12.771743+0000","last_active":"2024-04-25T10:30:28.875563+0000","last_peered":"2024-04-25T10:30:28.875563+0000","last_clean":"2024-04-25T10:30:28.875563+0000","last_became_active":"2024-04-25T10:30:12.771387+0000","last_became_peered":"2024-04-25T10:30:12.771387+0000","last_unstale":"2024-04-25T10:30:28.875563+0000","last_undegraded":"2024-04-25T10:30:28.875563+0000","last_fullsized":"2024-04-25T10:30:28.875563+0000","mapping_epoch":34,"log_start":"0'0","ondisk_log_start":"0'0","created":34,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:30:11.733978+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:30:11.733978+0000","last_clean_scrub_stamp":"2024-04-25T10:30:11.733978+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T14:01:06.791796+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"6.e","version":"0'0","reported_seq":9,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-04-25T10:30:28.885894+0000","last_change":"2024-04-25T10:30:28.884467+0000","last_active":"2024-04-25T10:30:28.885894+0000","last_peered":"2024-04-25T10:30:28.885894+0000","last_clean":"2024-04-25T10:30:28.885894+0000","last_became_active":"2024-04-25T10:30:28.884173+0000","last_became_peered":"2024-04-25T10:30:28.884173+0000","last_unstale":"2024-04-25T10:30:28.885894+0000","last_undegraded":"2024-04-25T10:30:28.885894+0000","last_fullsized":"2024-04-25T10:30:28.885894+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":42,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:30:27.860024+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:30:27.860024+0000","last_clean_scrub_stamp":"2024-04-25T10:30:27.860024+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T20:14:31.367609+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.d","version":"37'2","reported_seq":32,"reported_epoch":44,"state":"active+clean","last_fresh":"2024-04-25T10:30:29.882419+0000","last_change":"2024-04-25T10:30:11.758390+0000","last_active":"2024-04-25T10:30:29.882419+0000","last_peered":"2024-04-25T10:30:29.882419+0000","last_clean":"2024-04-25T10:30:29.882419+0000","last_became_active":"2024-04-25T10:30:11.757989+0000","last_became_peered":"2024-04-25T10:30:11.757989+0000","last_unstale":"2024-04-25T10:30:29.882419+0000","last_undegraded":"2024-04-25T10:30:29.882419+0000","last_fullsized":"2024-04-25T10:30:29.882419+0000","mapping_epoch":33,"log_start":"0'0","ondisk_log_start":"0'0","created":33,"last_epoch_clean":34,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:30:10.729165+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:30:10.729165+0000","last_clean_scrub_stamp":"2024-04-25T10:30:10.729165+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T18:52:44.534946+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.c","version":"36'6","reported_seq":30,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-04-25T10:30:28.876757+0000","last_change":"2024-04-25T10:30:12.765790+0000","last_active":"2024-04-25T10:30:28.876757+0000","last_peered":"2024-04-25T10:30:28.876757+0000","last_clean":"2024-04-25T10:30:28.876757+0000","last_became_active":"2024-04-25T10:30:12.765468+0000","last_became_peered":"2024-04-25T10:30:12.765468+0000","last_unstale":"2024-04-25T10:30:28.876757+0000","last_undegraded":"2024-04-25T10:30:28.876757+0000","last_fullsized":"2024-04-25T10:30:28.876757+0000","mapping_epoch":34,"log_start":"0'0","ondisk_log_start":"0'0","created":34,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:30:11.733978+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:30:11.733978+0000","last_clean_scrub_stamp":"2024-04-25T10:30:11.733978+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T16:29:40.621322+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"6.f","version":"0'0","reported_seq":9,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-04-25T10:30:28.887668+0000","last_change":"2024-04-25T10:30:28.886506+0000","last_active":"2024-04-25T10:30:28.887668+0000","last_peered":"2024-04-25T10:30:28.887668+0000","last_clean":"2024-04-25T10:30:28.887668+0000","last_became_active":"2024-04-25T10:30:28.886157+0000","last_became_peered":"2024-04-25T10:30:28.886157+0000","last_unstale":"2024-04-25T10:30:28.887668+0000","last_undegraded":"2024-04-25T10:30:28.887668+0000","last_fullsized":"2024-04-25T10:30:28.887668+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":42,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:30:27.860024+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:30:27.860024+0000","last_clean_scrub_stamp":"2024-04-25T10:30:27.860024+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T11:57:46.105302+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.e","version":"37'3","reported_seq":33,"reported_epoch":44,"state":"active+clean","last_fresh":"2024-04-25T10:30:29.883403+0000","last_change":"2024-04-25T10:30:11.765106+0000","last_active":"2024-04-25T10:30:29.883403+0000","last_peered":"2024-04-25T10:30:29.883403+0000","last_clean":"2024-04-25T10:30:29.883403+0000","last_became_active":"2024-04-25T10:30:11.764710+0000","last_became_peered":"2024-04-25T10:30:11.764710+0000","last_unstale":"2024-04-25T10:30:29.883403+0000","last_undegraded":"2024-04-25T10:30:29.883403+0000","last_fullsized":"2024-04-25T10:30:29.883403+0000","mapping_epoch":33,"log_start":"0'0","ondisk_log_start":"0'0","created":33,"last_epoch_clean":34,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:30:10.729165+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:30:10.729165+0000","last_clean_scrub_stamp":"2024-04-25T10:30:10.729165+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T16:30:11.709416+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"5.f","version":"36'9","reported_seq":34,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-04-25T10:30:28.875269+0000","last_change":"2024-04-25T10:30:12.769270+0000","last_active":"2024-04-25T10:30:28.875269+0000","last_peered":"2024-04-25T10:30:28.875269+0000","last_clean":"2024-04-25T10:30:28.875269+0000","last_became_active":"2024-04-25T10:30:12.768931+0000","last_became_peered":"2024-04-25T10:30:12.768931+0000","last_unstale":"2024-04-25T10:30:28.875269+0000","last_undegraded":"2024-04-25T10:30:28.875269+0000","last_fullsized":"2024-04-25T10:30:28.875269+0000","mapping_epoch":34,"log_start":"0'0","ondisk_log_start":"0'0","created":34,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:30:11.733978+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:30:11.733978+0000","last_clean_scrub_stamp":"2024-04-25T10:30:11.733978+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T16:49:00.268423+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"6.c","version":"0'0","reported_seq":12,"reported_epoch":44,"state":"active+clean","last_fresh":"2024-04-25T10:30:29.883642+0000","last_change":"2024-04-25T10:30:28.887055+0000","last_active":"2024-04-25T10:30:29.883642+0000","last_peered":"2024-04-25T10:30:29.883642+0000","last_clean":"2024-04-25T10:30:29.883642+0000","last_became_active":"2024-04-25T10:30:28.886675+0000","last_became_peered":"2024-04-25T10:30:28.886675+0000","last_unstale":"2024-04-25T10:30:29.883642+0000","last_undegraded":"2024-04-25T10:30:29.883642+0000","last_fullsized":"2024-04-25T10:30:29.883642+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":42,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:30:27.860024+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:30:27.860024+0000","last_clean_scrub_stamp":"2024-04-25T10:30:27.860024+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T14:04:58.646968+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,1],"acting":[6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.f","version":"37'2","reported_seq":27,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-04-25T10:30:28.874709+0000","last_change":"2024-04-25T10:30:11.750348+0000","last_active":"2024-04-25T10:30:28.874709+0000","last_peered":"2024-04-25T10:30:28.874709+0000","last_clean":"2024-04-25T10:30:28.874709+0000","last_became_active":"2024-04-25T10:30:11.749970+0000","last_became_peered":"2024-04-25T10:30:11.749970+0000","last_unstale":"2024-04-25T10:30:28.874709+0000","last_undegraded":"2024-04-25T10:30:28.874709+0000","last_fullsized":"2024-04-25T10:30:28.874709+0000","mapping_epoch":33,"log_start":"0'0","ondisk_log_start":"0'0","created":33,"last_epoch_clean":34,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:30:10.729165+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:30:10.729165+0000","last_clean_scrub_stamp":"2024-04-25T10:30:10.729165+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T20:41:22.574408+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"5.e","version":"36'5","reported_seq":34,"reported_epoch":44,"state":"active+clean","last_fresh":"2024-04-25T10:30:29.882392+0000","last_change":"2024-04-25T10:30:12.780850+0000","last_active":"2024-04-25T10:30:29.882392+0000","last_peered":"2024-04-25T10:30:29.882392+0000","last_clean":"2024-04-25T10:30:29.882392+0000","last_became_active":"2024-04-25T10:30:12.780187+0000","last_became_peered":"2024-04-25T10:30:12.780187+0000","last_unstale":"2024-04-25T10:30:29.882392+0000","last_undegraded":"2024-04-25T10:30:29.882392+0000","last_fullsized":"2024-04-25T10:30:29.882392+0000","mapping_epoch":34,"log_start":"0'0","ondisk_log_start":"0'0","created":34,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:30:11.733978+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:30:11.733978+0000","last_clean_scrub_stamp":"2024-04-25T10:30:11.733978+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T12:49:30.093333+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"6.d","version":"0'0","reported_seq":9,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-04-25T10:30:28.880134+0000","last_change":"2024-04-25T10:30:28.878873+0000","last_active":"2024-04-25T10:30:28.880134+0000","last_peered":"2024-04-25T10:30:28.880134+0000","last_clean":"2024-04-25T10:30:28.880134+0000","last_became_active":"2024-04-25T10:30:28.878596+0000","last_became_peered":"2024-04-25T10:30:28.878596+0000","last_unstale":"2024-04-25T10:30:28.880134+0000","last_undegraded":"2024-04-25T10:30:28.880134+0000","last_fullsized":"2024-04-25T10:30:28.880134+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":42,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:30:27.860024+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:30:27.860024+0000","last_clean_scrub_stamp":"2024-04-25T10:30:27.860024+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T11:33:01.743970+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.10","version":"37'15","reported_seq":40,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-04-25T10:30:28.873989+0000","last_change":"2024-04-25T10:30:11.755358+0000","last_active":"2024-04-25T10:30:28.873989+0000","last_peered":"2024-04-25T10:30:28.873989+0000","last_clean":"2024-04-25T10:30:28.873989+0000","last_became_active":"2024-04-25T10:30:11.754992+0000","last_became_peered":"2024-04-25T10:30:11.754992+0000","last_unstale":"2024-04-25T10:30:28.873989+0000","last_undegraded":"2024-04-25T10:30:28.873989+0000","last_fullsized":"2024-04-25T10:30:28.873989+0000","mapping_epoch":33,"log_start":"0'0","ondisk_log_start":"0'0","created":33,"last_epoch_clean":34,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:30:10.729165+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:30:10.729165+0000","last_clean_scrub_stamp":"2024-04-25T10:30:10.729165+0000","objects_scrubbed":0,"log_size":15,"log_dups_size":0,"ondisk_log_size":15,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T19:57:30.397817+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":21,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3],"acting":[7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"5.11","version":"36'6","reported_seq":35,"reported_epoch":44,"state":"active+clean","last_fresh":"2024-04-25T10:30:29.881885+0000","last_change":"2024-04-25T10:30:12.777184+0000","last_active":"2024-04-25T10:30:29.881885+0000","last_peered":"2024-04-25T10:30:29.881885+0000","last_clean":"2024-04-25T10:30:29.881885+0000","last_became_active":"2024-04-25T10:30:12.776874+0000","last_became_peered":"2024-04-25T10:30:12.776874+0000","last_unstale":"2024-04-25T10:30:29.881885+0000","last_undegraded":"2024-04-25T10:30:29.881885+0000","last_fullsized":"2024-04-25T10:30:29.881885+0000","mapping_epoch":34,"log_start":"0'0","ondisk_log_start":"0'0","created":34,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:30:11.733978+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:30:11.733978+0000","last_clean_scrub_stamp":"2024-04-25T10:30:11.733978+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T16:41:33.609972+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"6.12","version":"0'0","reported_seq":9,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-04-25T10:30:28.882650+0000","last_change":"2024-04-25T10:30:28.881494+0000","last_active":"2024-04-25T10:30:28.882650+0000","last_peered":"2024-04-25T10:30:28.882650+0000","last_clean":"2024-04-25T10:30:28.882650+0000","last_became_active":"2024-04-25T10:30:28.880682+0000","last_became_peered":"2024-04-25T10:30:28.880682+0000","last_unstale":"2024-04-25T10:30:28.882650+0000","last_undegraded":"2024-04-25T10:30:28.882650+0000","last_fullsized":"2024-04-25T10:30:28.882650+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":42,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:30:27.860024+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:30:27.860024+0000","last_clean_scrub_stamp":"2024-04-25T10:30:27.860024+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T17:35:49.687185+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.11","version":"0'0","reported_seq":25,"reported_epoch":42,"state":"active+clean","last_fresh":"2024-04-25T10:30:27.876406+0000","last_change":"2024-04-25T10:30:11.755706+0000","last_active":"2024-04-25T10:30:27.876406+0000","last_peered":"2024-04-25T10:30:27.876406+0000","last_clean":"2024-04-25T10:30:27.876406+0000","last_became_active":"2024-04-25T10:30:11.755348+0000","last_became_peered":"2024-04-25T10:30:11.755348+0000","last_unstale":"2024-04-25T10:30:27.876406+0000","last_undegraded":"2024-04-25T10:30:27.876406+0000","last_fullsized":"2024-04-25T10:30:27.876406+0000","mapping_epoch":33,"log_start":"0'0","ondisk_log_start":"0'0","created":33,"last_epoch_clean":34,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:30:10.729165+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:30:10.729165+0000","last_clean_scrub_stamp":"2024-04-25T10:30:10.729165+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T10:42:20.657474+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.10","version":"36'6","reported_seq":35,"reported_epoch":44,"state":"active+clean","last_fresh":"2024-04-25T10:30:29.882755+0000","last_change":"2024-04-25T10:30:12.783285+0000","last_active":"2024-04-25T10:30:29.882755+0000","last_peered":"2024-04-25T10:30:29.882755+0000","last_clean":"2024-04-25T10:30:29.882755+0000","last_became_active":"2024-04-25T10:30:12.782617+0000","last_became_peered":"2024-04-25T10:30:12.782617+0000","last_unstale":"2024-04-25T10:30:29.882755+0000","last_undegraded":"2024-04-25T10:30:29.882755+0000","last_fullsized":"2024-04-25T10:30:29.882755+0000","mapping_epoch":34,"log_start":"0'0","ondisk_log_start":"0'0","created":34,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:30:11.733978+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:30:11.733978+0000","last_clean_scrub_stamp":"2024-04-25T10:30:11.733978+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T18:35:51.085554+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"6.13","version":"0'0","reported_seq":9,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-04-25T10:30:28.882643+0000","last_change":"2024-04-25T10:30:28.880997+0000","last_active":"2024-04-25T10:30:28.882643+0000","last_peered":"2024-04-25T10:30:28.882643+0000","last_clean":"2024-04-25T10:30:28.882643+0000","last_became_active":"2024-04-25T10:30:28.880686+0000","last_became_peered":"2024-04-25T10:30:28.880686+0000","last_unstale":"2024-04-25T10:30:28.882643+0000","last_undegraded":"2024-04-25T10:30:28.882643+0000","last_fullsized":"2024-04-25T10:30:28.882643+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":42,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:30:27.860024+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:30:27.860024+0000","last_clean_scrub_stamp":"2024-04-25T10:30:27.860024+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T20:14:31.367609+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.12","version":"0'0","reported_seq":25,"reported_epoch":42,"state":"active+clean","last_fresh":"2024-04-25T10:30:27.876694+0000","last_change":"2024-04-25T10:30:11.762094+0000","last_active":"2024-04-25T10:30:27.876694+0000","last_peered":"2024-04-25T10:30:27.876694+0000","last_clean":"2024-04-25T10:30:27.876694+0000","last_became_active":"2024-04-25T10:30:11.761456+0000","last_became_peered":"2024-04-25T10:30:11.761456+0000","last_unstale":"2024-04-25T10:30:27.876694+0000","last_undegraded":"2024-04-25T10:30:27.876694+0000","last_fullsized":"2024-04-25T10:30:27.876694+0000","mapping_epoch":33,"log_start":"0'0","ondisk_log_start":"0'0","created":33,"last_epoch_clean":34,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:30:10.729165+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:30:10.729165+0000","last_clean_scrub_stamp":"2024-04-25T10:30:10.729165+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T21:58:31.211023+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.13","version":"36'15","reported_seq":38,"reported_epoch":42,"state":"active+clean","last_fresh":"2024-04-25T10:30:27.876744+0000","last_change":"2024-04-25T10:30:12.775269+0000","last_active":"2024-04-25T10:30:27.876744+0000","last_peered":"2024-04-25T10:30:27.876744+0000","last_clean":"2024-04-25T10:30:27.876744+0000","last_became_active":"2024-04-25T10:30:12.774865+0000","last_became_peered":"2024-04-25T10:30:12.774865+0000","last_unstale":"2024-04-25T10:30:27.876744+0000","last_undegraded":"2024-04-25T10:30:27.876744+0000","last_fullsized":"2024-04-25T10:30:27.876744+0000","mapping_epoch":34,"log_start":"0'0","ondisk_log_start":"0'0","created":34,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:30:11.733978+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:30:11.733978+0000","last_clean_scrub_stamp":"2024-04-25T10:30:11.733978+0000","objects_scrubbed":0,"log_size":15,"log_dups_size":0,"ondisk_log_size":15,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T20:14:36.991455+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"6.10","version":"0'0","reported_seq":1,"reported_epoch":42,"state":"creating+peering","last_fresh":"2024-04-25T10:30:27.875123+0000","last_change":"2024-04-25T10:30:27.875123+0000","last_active":"2024-04-25T10:30:27.860024+0000","last_peered":"2024-04-25T10:30:27.860024+0000","last_clean":"2024-04-25T10:30:27.860024+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T10:30:27.875123+0000","last_undegraded":"2024-04-25T10:30:27.875123+0000","last_fullsized":"2024-04-25T10:30:27.875123+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":42,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:30:27.860024+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:30:27.860024+0000","last_clean_scrub_stamp":"2024-04-25T10:30:27.860024+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.13","version":"37'5","reported_seq":35,"reported_epoch":44,"state":"active+clean","last_fresh":"2024-04-25T10:30:29.883254+0000","last_change":"2024-04-25T10:30:11.764993+0000","last_active":"2024-04-25T10:30:29.883254+0000","last_peered":"2024-04-25T10:30:29.883254+0000","last_clean":"2024-04-25T10:30:29.883254+0000","last_became_active":"2024-04-25T10:30:11.764637+0000","last_became_peered":"2024-04-25T10:30:11.764637+0000","last_unstale":"2024-04-25T10:30:29.883254+0000","last_undegraded":"2024-04-25T10:30:29.883254+0000","last_fullsized":"2024-04-25T10:30:29.883254+0000","mapping_epoch":33,"log_start":"0'0","ondisk_log_start":"0'0","created":33,"last_epoch_clean":34,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:30:10.729165+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:30:10.729165+0000","last_clean_scrub_stamp":"2024-04-25T10:30:10.729165+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T17:24:13.848319+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6,"num_write_kb":8,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"5.12","version":"36'10","reported_seq":37,"reported_epoch":44,"state":"active+clean","last_fresh":"2024-04-25T10:30:29.882763+0000","last_change":"2024-04-25T10:30:12.770496+0000","last_active":"2024-04-25T10:30:29.882763+0000","last_peered":"2024-04-25T10:30:29.882763+0000","last_clean":"2024-04-25T10:30:29.882763+0000","last_became_active":"2024-04-25T10:30:12.770193+0000","last_became_peered":"2024-04-25T10:30:12.770193+0000","last_unstale":"2024-04-25T10:30:29.882763+0000","last_undegraded":"2024-04-25T10:30:29.882763+0000","last_fullsized":"2024-04-25T10:30:29.882763+0000","mapping_epoch":34,"log_start":"0'0","ondisk_log_start":"0'0","created":34,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:30:11.733978+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:30:11.733978+0000","last_clean_scrub_stamp":"2024-04-25T10:30:11.733978+0000","objects_scrubbed":0,"log_size":10,"log_dups_size":0,"ondisk_log_size":10,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T21:35:23.984378+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,1],"acting":[6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"6.11","version":"0'0","reported_seq":12,"reported_epoch":44,"state":"active+clean","last_fresh":"2024-04-25T10:30:29.882707+0000","last_change":"2024-04-25T10:30:28.884488+0000","last_active":"2024-04-25T10:30:29.882707+0000","last_peered":"2024-04-25T10:30:29.882707+0000","last_clean":"2024-04-25T10:30:29.882707+0000","last_became_active":"2024-04-25T10:30:28.884188+0000","last_became_peered":"2024-04-25T10:30:28.884188+0000","last_unstale":"2024-04-25T10:30:29.882707+0000","last_undegraded":"2024-04-25T10:30:29.882707+0000","last_fullsized":"2024-04-25T10:30:29.882707+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":42,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:30:27.860024+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:30:27.860024+0000","last_clean_scrub_stamp":"2024-04-25T10:30:27.860024+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T15:39:48.016111+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,1],"acting":[6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.14","version":"37'23","reported_seq":51,"reported_epoch":44,"state":"active+clean","last_fresh":"2024-04-25T10:30:29.883688+0000","last_change":"2024-04-25T10:30:11.757323+0000","last_active":"2024-04-25T10:30:29.883688+0000","last_peered":"2024-04-25T10:30:29.883688+0000","last_clean":"2024-04-25T10:30:29.883688+0000","last_became_active":"2024-04-25T10:30:11.757011+0000","last_became_peered":"2024-04-25T10:30:11.757011+0000","last_unstale":"2024-04-25T10:30:29.883688+0000","last_undegraded":"2024-04-25T10:30:29.883688+0000","last_fullsized":"2024-04-25T10:30:29.883688+0000","mapping_epoch":33,"log_start":"0'0","ondisk_log_start":"0'0","created":33,"last_epoch_clean":34,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:30:10.729165+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:30:10.729165+0000","last_clean_scrub_stamp":"2024-04-25T10:30:10.729165+0000","objects_scrubbed":0,"log_size":23,"log_dups_size":0,"ondisk_log_size":23,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T19:02:53.584992+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":23,"num_write_kb":25,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"5.15","version":"36'6","reported_seq":35,"reported_epoch":44,"state":"active+clean","last_fresh":"2024-04-25T10:30:29.881889+0000","last_change":"2024-04-25T10:30:12.776432+0000","last_active":"2024-04-25T10:30:29.881889+0000","last_peered":"2024-04-25T10:30:29.881889+0000","last_clean":"2024-04-25T10:30:29.881889+0000","last_became_active":"2024-04-25T10:30:12.775857+0000","last_became_peered":"2024-04-25T10:30:12.775857+0000","last_unstale":"2024-04-25T10:30:29.881889+0000","last_undegraded":"2024-04-25T10:30:29.881889+0000","last_fullsized":"2024-04-25T10:30:29.881889+0000","mapping_epoch":34,"log_start":"0'0","ondisk_log_start":"0'0","created":34,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:30:11.733978+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:30:11.733978+0000","last_clean_scrub_stamp":"2024-04-25T10:30:11.733978+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T14:29:10.343115+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"6.16","version":"0'0","reported_seq":1,"reported_epoch":42,"state":"creating+peering","last_fresh":"2024-04-25T10:30:27.876799+0000","last_change":"2024-04-25T10:30:27.876799+0000","last_active":"2024-04-25T10:30:27.860024+0000","last_peered":"2024-04-25T10:30:27.860024+0000","last_clean":"2024-04-25T10:30:27.860024+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T10:30:27.876799+0000","last_undegraded":"2024-04-25T10:30:27.876799+0000","last_fullsized":"2024-04-25T10:30:27.876799+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":42,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:30:27.860024+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:30:27.860024+0000","last_clean_scrub_stamp":"2024-04-25T10:30:27.860024+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.15","version":"37'5","reported_seq":35,"reported_epoch":44,"state":"active+clean","last_fresh":"2024-04-25T10:30:29.882033+0000","last_change":"2024-04-25T10:30:11.755959+0000","last_active":"2024-04-25T10:30:29.882033+0000","last_peered":"2024-04-25T10:30:29.882033+0000","last_clean":"2024-04-25T10:30:29.882033+0000","last_became_active":"2024-04-25T10:30:11.755603+0000","last_became_peered":"2024-04-25T10:30:11.755603+0000","last_unstale":"2024-04-25T10:30:29.882033+0000","last_undegraded":"2024-04-25T10:30:29.882033+0000","last_fullsized":"2024-04-25T10:30:29.882033+0000","mapping_epoch":33,"log_start":"0'0","ondisk_log_start":"0'0","created":33,"last_epoch_clean":34,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:30:10.729165+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:30:10.729165+0000","last_clean_scrub_stamp":"2024-04-25T10:30:10.729165+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T19:25:05.393017+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.14","version":"36'11","reported_seq":34,"reported_epoch":42,"state":"active+clean","last_fresh":"2024-04-25T10:30:27.877515+0000","last_change":"2024-04-25T10:30:12.770588+0000","last_active":"2024-04-25T10:30:27.877515+0000","last_peered":"2024-04-25T10:30:27.877515+0000","last_clean":"2024-04-25T10:30:27.877515+0000","last_became_active":"2024-04-25T10:30:12.770264+0000","last_became_peered":"2024-04-25T10:30:12.770264+0000","last_unstale":"2024-04-25T10:30:27.877515+0000","last_undegraded":"2024-04-25T10:30:27.877515+0000","last_fullsized":"2024-04-25T10:30:27.877515+0000","mapping_epoch":34,"log_start":"0'0","ondisk_log_start":"0'0","created":34,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:30:11.733978+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:30:11.733978+0000","last_clean_scrub_stamp":"2024-04-25T10:30:11.733978+0000","objects_scrubbed":0,"log_size":11,"log_dups_size":0,"ondisk_log_size":11,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T20:41:50.045543+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"6.17","version":"0'0","reported_seq":9,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-04-25T10:30:28.880577+0000","last_change":"2024-04-25T10:30:28.878754+0000","last_active":"2024-04-25T10:30:28.880577+0000","last_peered":"2024-04-25T10:30:28.880577+0000","last_clean":"2024-04-25T10:30:28.880577+0000","last_became_active":"2024-04-25T10:30:28.878242+0000","last_became_peered":"2024-04-25T10:30:28.878242+0000","last_unstale":"2024-04-25T10:30:28.880577+0000","last_undegraded":"2024-04-25T10:30:28.880577+0000","last_fullsized":"2024-04-25T10:30:28.880577+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":42,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:30:27.860024+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:30:27.860024+0000","last_clean_scrub_stamp":"2024-04-25T10:30:27.860024+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T16:08:46.113866+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7],"acting":[1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.16","version":"0'0","reported_seq":25,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-04-25T10:30:28.879092+0000","last_change":"2024-04-25T10:30:11.749473+0000","last_active":"2024-04-25T10:30:28.879092+0000","last_peered":"2024-04-25T10:30:28.879092+0000","last_clean":"2024-04-25T10:30:28.879092+0000","last_became_active":"2024-04-25T10:30:11.749135+0000","last_became_peered":"2024-04-25T10:30:11.749135+0000","last_unstale":"2024-04-25T10:30:28.879092+0000","last_undegraded":"2024-04-25T10:30:28.879092+0000","last_fullsized":"2024-04-25T10:30:28.879092+0000","mapping_epoch":33,"log_start":"0'0","ondisk_log_start":"0'0","created":33,"last_epoch_clean":34,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:30:10.729165+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:30:10.729165+0000","last_clean_scrub_stamp":"2024-04-25T10:30:10.729165+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T15:17:28.997853+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"5.17","version":"36'9","reported_seq":32,"reported_epoch":42,"state":"active+clean","last_fresh":"2024-04-25T10:30:27.872862+0000","last_change":"2024-04-25T10:30:12.773113+0000","last_active":"2024-04-25T10:30:27.872862+0000","last_peered":"2024-04-25T10:30:27.872862+0000","last_clean":"2024-04-25T10:30:27.872862+0000","last_became_active":"2024-04-25T10:30:12.772786+0000","last_became_peered":"2024-04-25T10:30:12.772786+0000","last_unstale":"2024-04-25T10:30:27.872862+0000","last_undegraded":"2024-04-25T10:30:27.872862+0000","last_fullsized":"2024-04-25T10:30:27.872862+0000","mapping_epoch":34,"log_start":"0'0","ondisk_log_start":"0'0","created":34,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:30:11.733978+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:30:11.733978+0000","last_clean_scrub_stamp":"2024-04-25T10:30:11.733978+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T18:45:03.004387+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5],"acting":[4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"6.14","version":"0'0","reported_seq":1,"reported_epoch":42,"state":"creating+peering","last_fresh":"2024-04-25T10:30:27.876893+0000","last_change":"2024-04-25T10:30:27.876893+0000","last_active":"2024-04-25T10:30:27.860024+0000","last_peered":"2024-04-25T10:30:27.860024+0000","last_clean":"2024-04-25T10:30:27.860024+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T10:30:27.876893+0000","last_undegraded":"2024-04-25T10:30:27.876893+0000","last_fullsized":"2024-04-25T10:30:27.876893+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":42,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:30:27.860024+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:30:27.860024+0000","last_clean_scrub_stamp":"2024-04-25T10:30:27.860024+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.17","version":"0'0","reported_seq":25,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-04-25T10:30:28.878199+0000","last_change":"2024-04-25T10:30:11.758416+0000","last_active":"2024-04-25T10:30:28.878199+0000","last_peered":"2024-04-25T10:30:28.878199+0000","last_clean":"2024-04-25T10:30:28.878199+0000","last_became_active":"2024-04-25T10:30:11.758049+0000","last_became_peered":"2024-04-25T10:30:11.758049+0000","last_unstale":"2024-04-25T10:30:28.878199+0000","last_undegraded":"2024-04-25T10:30:28.878199+0000","last_fullsized":"2024-04-25T10:30:28.878199+0000","mapping_epoch":33,"log_start":"0'0","ondisk_log_start":"0'0","created":33,"last_epoch_clean":34,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:30:10.729165+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:30:10.729165+0000","last_clean_scrub_stamp":"2024-04-25T10:30:10.729165+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T18:26:05.159854+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"5.16","version":"36'10","reported_seq":39,"reported_epoch":44,"state":"active+clean","last_fresh":"2024-04-25T10:30:29.882265+0000","last_change":"2024-04-25T10:30:12.770216+0000","last_active":"2024-04-25T10:30:29.882265+0000","last_peered":"2024-04-25T10:30:29.882265+0000","last_clean":"2024-04-25T10:30:29.882265+0000","last_became_active":"2024-04-25T10:30:12.769785+0000","last_became_peered":"2024-04-25T10:30:12.769785+0000","last_unstale":"2024-04-25T10:30:29.882265+0000","last_undegraded":"2024-04-25T10:30:29.882265+0000","last_fullsized":"2024-04-25T10:30:29.882265+0000","mapping_epoch":34,"log_start":"0'0","ondisk_log_start":"0'0","created":34,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:30:11.733978+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:30:11.733978+0000","last_clean_scrub_stamp":"2024-04-25T10:30:11.733978+0000","objects_scrubbed":0,"log_size":10,"log_dups_size":0,"ondisk_log_size":10,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T16:58:09.648786+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"6.15","version":"0'0","reported_seq":12,"reported_epoch":44,"state":"active+clean","last_fresh":"2024-04-25T10:30:29.883532+0000","last_change":"2024-04-25T10:30:28.879590+0000","last_active":"2024-04-25T10:30:29.883532+0000","last_peered":"2024-04-25T10:30:29.883532+0000","last_clean":"2024-04-25T10:30:29.883532+0000","last_became_active":"2024-04-25T10:30:28.879277+0000","last_became_peered":"2024-04-25T10:30:28.879277+0000","last_unstale":"2024-04-25T10:30:29.883532+0000","last_undegraded":"2024-04-25T10:30:29.883532+0000","last_fullsized":"2024-04-25T10:30:29.883532+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":42,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:30:27.860024+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:30:27.860024+0000","last_clean_scrub_stamp":"2024-04-25T10:30:27.860024+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T14:04:58.646968+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.18","version":"0'0","reported_seq":30,"reported_epoch":44,"state":"active+clean","last_fresh":"2024-04-25T10:30:29.882866+0000","last_change":"2024-04-25T10:30:11.756627+0000","last_active":"2024-04-25T10:30:29.882866+0000","last_peered":"2024-04-25T10:30:29.882866+0000","last_clean":"2024-04-25T10:30:29.882866+0000","last_became_active":"2024-04-25T10:30:11.756323+0000","last_became_peered":"2024-04-25T10:30:11.756323+0000","last_unstale":"2024-04-25T10:30:29.882866+0000","last_undegraded":"2024-04-25T10:30:29.882866+0000","last_fullsized":"2024-04-25T10:30:29.882866+0000","mapping_epoch":33,"log_start":"0'0","ondisk_log_start":"0'0","created":33,"last_epoch_clean":34,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:30:10.729165+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:30:10.729165+0000","last_clean_scrub_stamp":"2024-04-25T10:30:10.729165+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T16:49:20.087013+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4],"acting":[2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"5.19","version":"36'10","reported_seq":33,"reported_epoch":42,"state":"active+clean","last_fresh":"2024-04-25T10:30:27.876263+0000","last_change":"2024-04-25T10:30:12.780437+0000","last_active":"2024-04-25T10:30:27.876263+0000","last_peered":"2024-04-25T10:30:27.876263+0000","last_clean":"2024-04-25T10:30:27.876263+0000","last_became_active":"2024-04-25T10:30:12.779762+0000","last_became_peered":"2024-04-25T10:30:12.779762+0000","last_unstale":"2024-04-25T10:30:27.876263+0000","last_undegraded":"2024-04-25T10:30:27.876263+0000","last_fullsized":"2024-04-25T10:30:27.876263+0000","mapping_epoch":34,"log_start":"0'0","ondisk_log_start":"0'0","created":34,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:30:11.733978+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:30:11.733978+0000","last_clean_scrub_stamp":"2024-04-25T10:30:11.733978+0000","objects_scrubbed":0,"log_size":10,"log_dups_size":0,"ondisk_log_size":10,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T14:22:37.474013+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"6.1a","version":"0'0","reported_seq":12,"reported_epoch":44,"state":"active+clean","last_fresh":"2024-04-25T10:30:29.882257+0000","last_change":"2024-04-25T10:30:28.879591+0000","last_active":"2024-04-25T10:30:29.882257+0000","last_peered":"2024-04-25T10:30:29.882257+0000","last_clean":"2024-04-25T10:30:29.882257+0000","last_became_active":"2024-04-25T10:30:28.879271+0000","last_became_peered":"2024-04-25T10:30:28.879271+0000","last_unstale":"2024-04-25T10:30:29.882257+0000","last_undegraded":"2024-04-25T10:30:29.882257+0000","last_fullsized":"2024-04-25T10:30:29.882257+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":42,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:30:27.860024+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:30:27.860024+0000","last_clean_scrub_stamp":"2024-04-25T10:30:27.860024+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T20:14:35.145802+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.19","version":"0'0","reported_seq":25,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-04-25T10:30:28.874176+0000","last_change":"2024-04-25T10:30:11.757481+0000","last_active":"2024-04-25T10:30:28.874176+0000","last_peered":"2024-04-25T10:30:28.874176+0000","last_clean":"2024-04-25T10:30:28.874176+0000","last_became_active":"2024-04-25T10:30:11.757149+0000","last_became_peered":"2024-04-25T10:30:11.757149+0000","last_unstale":"2024-04-25T10:30:28.874176+0000","last_undegraded":"2024-04-25T10:30:28.874176+0000","last_fullsized":"2024-04-25T10:30:28.874176+0000","mapping_epoch":33,"log_start":"0'0","ondisk_log_start":"0'0","created":33,"last_epoch_clean":34,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:30:10.729165+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:30:10.729165+0000","last_clean_scrub_stamp":"2024-04-25T10:30:10.729165+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T13:23:15.814583+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"5.18","version":"36'8","reported_seq":33,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-04-25T10:30:28.874986+0000","last_change":"2024-04-25T10:30:12.769294+0000","last_active":"2024-04-25T10:30:28.874986+0000","last_peered":"2024-04-25T10:30:28.874986+0000","last_clean":"2024-04-25T10:30:28.874986+0000","last_became_active":"2024-04-25T10:30:12.768543+0000","last_became_peered":"2024-04-25T10:30:12.768543+0000","last_unstale":"2024-04-25T10:30:28.874986+0000","last_undegraded":"2024-04-25T10:30:28.874986+0000","last_fullsized":"2024-04-25T10:30:28.874986+0000","mapping_epoch":34,"log_start":"0'0","ondisk_log_start":"0'0","created":34,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:30:11.733978+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:30:11.733978+0000","last_clean_scrub_stamp":"2024-04-25T10:30:11.733978+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T18:26:06.164667+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"6.1b","version":"0'0","reported_seq":12,"reported_epoch":44,"state":"active+clean","last_fresh":"2024-04-25T10:30:29.883121+0000","last_change":"2024-04-25T10:30:28.885011+0000","last_active":"2024-04-25T10:30:29.883121+0000","last_peered":"2024-04-25T10:30:29.883121+0000","last_clean":"2024-04-25T10:30:29.883121+0000","last_became_active":"2024-04-25T10:30:28.883516+0000","last_became_peered":"2024-04-25T10:30:28.883516+0000","last_unstale":"2024-04-25T10:30:29.883121+0000","last_undegraded":"2024-04-25T10:30:29.883121+0000","last_fullsized":"2024-04-25T10:30:29.883121+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":42,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:30:27.860024+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:30:27.860024+0000","last_clean_scrub_stamp":"2024-04-25T10:30:27.860024+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T17:35:49.687185+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,2],"acting":[6,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.1a","version":"35'1","reported_seq":31,"reported_epoch":44,"state":"active+clean","last_fresh":"2024-04-25T10:30:29.883044+0000","last_change":"2024-04-25T10:30:11.758678+0000","last_active":"2024-04-25T10:30:29.883044+0000","last_peered":"2024-04-25T10:30:29.883044+0000","last_clean":"2024-04-25T10:30:29.883044+0000","last_became_active":"2024-04-25T10:30:11.758384+0000","last_became_peered":"2024-04-25T10:30:11.758384+0000","last_unstale":"2024-04-25T10:30:29.883044+0000","last_undegraded":"2024-04-25T10:30:29.883044+0000","last_fullsized":"2024-04-25T10:30:29.883044+0000","mapping_epoch":33,"log_start":"0'0","ondisk_log_start":"0'0","created":33,"last_epoch_clean":34,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:30:10.729165+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:30:10.729165+0000","last_clean_scrub_stamp":"2024-04-25T10:30:10.729165+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T16:17:54.376541+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,6],"acting":[2,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"5.1b","version":"36'8","reported_seq":37,"reported_epoch":44,"state":"active+clean","last_fresh":"2024-04-25T10:30:29.883109+0000","last_change":"2024-04-25T10:30:12.784767+0000","last_active":"2024-04-25T10:30:29.883109+0000","last_peered":"2024-04-25T10:30:29.883109+0000","last_clean":"2024-04-25T10:30:29.883109+0000","last_became_active":"2024-04-25T10:30:12.784311+0000","last_became_peered":"2024-04-25T10:30:12.784311+0000","last_unstale":"2024-04-25T10:30:29.883109+0000","last_undegraded":"2024-04-25T10:30:29.883109+0000","last_fullsized":"2024-04-25T10:30:29.883109+0000","mapping_epoch":34,"log_start":"0'0","ondisk_log_start":"0'0","created":34,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:30:11.733978+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:30:11.733978+0000","last_clean_scrub_stamp":"2024-04-25T10:30:11.733978+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T19:56:57.765496+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,7],"acting":[2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"6.18","version":"0'0","reported_seq":1,"reported_epoch":42,"state":"creating+peering","last_fresh":"2024-04-25T10:30:27.874480+0000","last_change":"2024-04-25T10:30:27.874480+0000","last_active":"2024-04-25T10:30:27.860024+0000","last_peered":"2024-04-25T10:30:27.860024+0000","last_clean":"2024-04-25T10:30:27.860024+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T10:30:27.874480+0000","last_undegraded":"2024-04-25T10:30:27.874480+0000","last_fullsized":"2024-04-25T10:30:27.874480+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":42,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:30:27.860024+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:30:27.860024+0000","last_clean_scrub_stamp":"2024-04-25T10:30:27.860024+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5],"acting":[4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.1b","version":"35'1","reported_seq":31,"reported_epoch":44,"state":"active+clean","last_fresh":"2024-04-25T10:30:29.883205+0000","last_change":"2024-04-25T10:30:11.753667+0000","last_active":"2024-04-25T10:30:29.883205+0000","last_peered":"2024-04-25T10:30:29.883205+0000","last_clean":"2024-04-25T10:30:29.883205+0000","last_became_active":"2024-04-25T10:30:11.753340+0000","last_became_peered":"2024-04-25T10:30:11.753340+0000","last_unstale":"2024-04-25T10:30:29.883205+0000","last_undegraded":"2024-04-25T10:30:29.883205+0000","last_fullsized":"2024-04-25T10:30:29.883205+0000","mapping_epoch":33,"log_start":"0'0","ondisk_log_start":"0'0","created":33,"last_epoch_clean":34,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:30:10.729165+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:30:10.729165+0000","last_clean_scrub_stamp":"2024-04-25T10:30:10.729165+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T16:09:04.081750+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"5.1a","version":"36'7","reported_seq":32,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-04-25T10:30:28.874760+0000","last_change":"2024-04-25T10:30:12.773835+0000","last_active":"2024-04-25T10:30:28.874760+0000","last_peered":"2024-04-25T10:30:28.874760+0000","last_clean":"2024-04-25T10:30:28.874760+0000","last_became_active":"2024-04-25T10:30:12.773538+0000","last_became_peered":"2024-04-25T10:30:12.773538+0000","last_unstale":"2024-04-25T10:30:28.874760+0000","last_undegraded":"2024-04-25T10:30:28.874760+0000","last_fullsized":"2024-04-25T10:30:28.874760+0000","mapping_epoch":34,"log_start":"0'0","ondisk_log_start":"0'0","created":34,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:30:11.733978+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:30:11.733978+0000","last_clean_scrub_stamp":"2024-04-25T10:30:11.733978+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T18:31:49.260513+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"6.19","version":"0'0","reported_seq":9,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-04-25T10:30:28.885557+0000","last_change":"2024-04-25T10:30:28.883543+0000","last_active":"2024-04-25T10:30:28.885557+0000","last_peered":"2024-04-25T10:30:28.885557+0000","last_clean":"2024-04-25T10:30:28.885557+0000","last_became_active":"2024-04-25T10:30:28.883229+0000","last_became_peered":"2024-04-25T10:30:28.883229+0000","last_unstale":"2024-04-25T10:30:28.885557+0000","last_undegraded":"2024-04-25T10:30:28.885557+0000","last_fullsized":"2024-04-25T10:30:28.885557+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":42,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:30:27.860024+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:30:27.860024+0000","last_clean_scrub_stamp":"2024-04-25T10:30:27.860024+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T18:12:57.452350+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"6.1e","version":"0'0","reported_seq":12,"reported_epoch":44,"state":"active+clean","last_fresh":"2024-04-25T10:30:29.882435+0000","last_change":"2024-04-25T10:30:28.874417+0000","last_active":"2024-04-25T10:30:29.882435+0000","last_peered":"2024-04-25T10:30:29.882435+0000","last_clean":"2024-04-25T10:30:29.882435+0000","last_became_active":"2024-04-25T10:30:28.874102+0000","last_became_peered":"2024-04-25T10:30:28.874102+0000","last_unstale":"2024-04-25T10:30:29.882435+0000","last_undegraded":"2024-04-25T10:30:29.882435+0000","last_fullsized":"2024-04-25T10:30:29.882435+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":42,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:30:27.860024+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:30:27.860024+0000","last_clean_scrub_stamp":"2024-04-25T10:30:27.860024+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T21:25:07.511482+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.1c","version":"37'3","reported_seq":28,"reported_epoch":42,"state":"active+clean","last_fresh":"2024-04-25T10:30:27.877716+0000","last_change":"2024-04-25T10:30:11.759829+0000","last_active":"2024-04-25T10:30:27.877716+0000","last_peered":"2024-04-25T10:30:27.877716+0000","last_clean":"2024-04-25T10:30:27.877716+0000","last_became_active":"2024-04-25T10:30:11.759465+0000","last_became_peered":"2024-04-25T10:30:11.759465+0000","last_unstale":"2024-04-25T10:30:27.877716+0000","last_undegraded":"2024-04-25T10:30:27.877716+0000","last_fullsized":"2024-04-25T10:30:27.877716+0000","mapping_epoch":33,"log_start":"0'0","ondisk_log_start":"0'0","created":33,"last_epoch_clean":34,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:30:10.729165+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:30:10.729165+0000","last_clean_scrub_stamp":"2024-04-25T10:30:10.729165+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T13:31:53.608355+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.1d","version":"36'15","reported_seq":38,"reported_epoch":42,"state":"active+clean","last_fresh":"2024-04-25T10:30:27.876677+0000","last_change":"2024-04-25T10:30:12.773474+0000","last_active":"2024-04-25T10:30:27.876677+0000","last_peered":"2024-04-25T10:30:27.876677+0000","last_clean":"2024-04-25T10:30:27.876677+0000","last_became_active":"2024-04-25T10:30:12.773133+0000","last_became_peered":"2024-04-25T10:30:12.773133+0000","last_unstale":"2024-04-25T10:30:27.876677+0000","last_undegraded":"2024-04-25T10:30:27.876677+0000","last_fullsized":"2024-04-25T10:30:27.876677+0000","mapping_epoch":34,"log_start":"0'0","ondisk_log_start":"0'0","created":34,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:30:11.733978+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:30:11.733978+0000","last_clean_scrub_stamp":"2024-04-25T10:30:11.733978+0000","objects_scrubbed":0,"log_size":15,"log_dups_size":0,"ondisk_log_size":15,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T17:24:23.630916+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"6.1f","version":"0'0","reported_seq":12,"reported_epoch":44,"state":"active+clean","last_fresh":"2024-04-25T10:30:29.883478+0000","last_change":"2024-04-25T10:30:28.884033+0000","last_active":"2024-04-25T10:30:29.883478+0000","last_peered":"2024-04-25T10:30:29.883478+0000","last_clean":"2024-04-25T10:30:29.883478+0000","last_became_active":"2024-04-25T10:30:28.883610+0000","last_became_peered":"2024-04-25T10:30:28.883610+0000","last_unstale":"2024-04-25T10:30:29.883478+0000","last_undegraded":"2024-04-25T10:30:29.883478+0000","last_fullsized":"2024-04-25T10:30:29.883478+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":42,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:30:27.860024+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:30:27.860024+0000","last_clean_scrub_stamp":"2024-04-25T10:30:27.860024+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T18:12:57.452350+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.1d","version":"37'3","reported_seq":28,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-04-25T10:30:28.875875+0000","last_change":"2024-04-25T10:30:11.752439+0000","last_active":"2024-04-25T10:30:28.875875+0000","last_peered":"2024-04-25T10:30:28.875875+0000","last_clean":"2024-04-25T10:30:28.875875+0000","last_became_active":"2024-04-25T10:30:11.752050+0000","last_became_peered":"2024-04-25T10:30:11.752050+0000","last_unstale":"2024-04-25T10:30:28.875875+0000","last_undegraded":"2024-04-25T10:30:28.875875+0000","last_fullsized":"2024-04-25T10:30:28.875875+0000","mapping_epoch":33,"log_start":"0'0","ondisk_log_start":"0'0","created":33,"last_epoch_clean":34,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:30:10.729165+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:30:10.729165+0000","last_clean_scrub_stamp":"2024-04-25T10:30:10.729165+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T11:41:21.358729+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"5.1c","version":"36'9","reported_seq":38,"reported_epoch":44,"state":"active+clean","last_fresh":"2024-04-25T10:30:29.883403+0000","last_change":"2024-04-25T10:30:12.781913+0000","last_active":"2024-04-25T10:30:29.883403+0000","last_peered":"2024-04-25T10:30:29.883403+0000","last_clean":"2024-04-25T10:30:29.883403+0000","last_became_active":"2024-04-25T10:30:12.781111+0000","last_became_peered":"2024-04-25T10:30:12.781111+0000","last_unstale":"2024-04-25T10:30:29.883403+0000","last_undegraded":"2024-04-25T10:30:29.883403+0000","last_fullsized":"2024-04-25T10:30:29.883403+0000","mapping_epoch":34,"log_start":"0'0","ondisk_log_start":"0'0","created":34,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:30:11.733978+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:30:11.733978+0000","last_clean_scrub_stamp":"2024-04-25T10:30:11.733978+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T16:29:27.103512+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4],"acting":[2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.1e","version":"35'1","reported_seq":26,"reported_epoch":42,"state":"active+clean","last_fresh":"2024-04-25T10:30:27.877804+0000","last_change":"2024-04-25T10:30:11.761160+0000","last_active":"2024-04-25T10:30:27.877804+0000","last_peered":"2024-04-25T10:30:27.877804+0000","last_clean":"2024-04-25T10:30:27.877804+0000","last_became_active":"2024-04-25T10:30:11.760819+0000","last_became_peered":"2024-04-25T10:30:11.760819+0000","last_unstale":"2024-04-25T10:30:27.877804+0000","last_undegraded":"2024-04-25T10:30:27.877804+0000","last_fullsized":"2024-04-25T10:30:27.877804+0000","mapping_epoch":33,"log_start":"0'0","ondisk_log_start":"0'0","created":33,"last_epoch_clean":34,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:30:10.729165+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:30:10.729165+0000","last_clean_scrub_stamp":"2024-04-25T10:30:10.729165+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T13:00:22.139475+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.1f","version":"36'6","reported_seq":35,"reported_epoch":44,"state":"active+clean","last_fresh":"2024-04-25T10:30:29.883526+0000","last_change":"2024-04-25T10:30:12.772257+0000","last_active":"2024-04-25T10:30:29.883526+0000","last_peered":"2024-04-25T10:30:29.883526+0000","last_clean":"2024-04-25T10:30:29.883526+0000","last_became_active":"2024-04-25T10:30:12.771923+0000","last_became_peered":"2024-04-25T10:30:12.771923+0000","last_unstale":"2024-04-25T10:30:29.883526+0000","last_undegraded":"2024-04-25T10:30:29.883526+0000","last_fullsized":"2024-04-25T10:30:29.883526+0000","mapping_epoch":34,"log_start":"0'0","ondisk_log_start":"0'0","created":34,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:30:11.733978+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:30:11.733978+0000","last_clean_scrub_stamp":"2024-04-25T10:30:11.733978+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T17:35:33.561138+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"6.1c","version":"0'0","reported_seq":12,"reported_epoch":44,"state":"active+clean","last_fresh":"2024-04-25T10:30:29.882853+0000","last_change":"2024-04-25T10:30:28.875422+0000","last_active":"2024-04-25T10:30:29.882853+0000","last_peered":"2024-04-25T10:30:29.882853+0000","last_clean":"2024-04-25T10:30:29.882853+0000","last_became_active":"2024-04-25T10:30:28.875079+0000","last_became_peered":"2024-04-25T10:30:28.875079+0000","last_unstale":"2024-04-25T10:30:29.882853+0000","last_undegraded":"2024-04-25T10:30:29.882853+0000","last_fullsized":"2024-04-25T10:30:29.882853+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":42,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:30:27.860024+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:30:27.860024+0000","last_clean_scrub_stamp":"2024-04-25T10:30:27.860024+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T13:23:55.301869+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.1f","version":"37'8","reported_seq":38,"reported_epoch":44,"state":"active+clean","last_fresh":"2024-04-25T10:30:29.882826+0000","last_change":"2024-04-25T10:30:11.756978+0000","last_active":"2024-04-25T10:30:29.882826+0000","last_peered":"2024-04-25T10:30:29.882826+0000","last_clean":"2024-04-25T10:30:29.882826+0000","last_became_active":"2024-04-25T10:30:11.756617+0000","last_became_peered":"2024-04-25T10:30:11.756617+0000","last_unstale":"2024-04-25T10:30:29.882826+0000","last_undegraded":"2024-04-25T10:30:29.882826+0000","last_fullsized":"2024-04-25T10:30:29.882826+0000","mapping_epoch":33,"log_start":"0'0","ondisk_log_start":"0'0","created":33,"last_epoch_clean":34,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:30:10.729165+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:30:10.729165+0000","last_clean_scrub_stamp":"2024-04-25T10:30:10.729165+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T12:27:53.559880+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":1,"num_read_kb":0,"num_write":10,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.1e","version":"36'5","reported_seq":32,"reported_epoch":44,"state":"active+clean","last_fresh":"2024-04-25T10:30:29.883469+0000","last_change":"2024-04-25T10:30:12.769800+0000","last_active":"2024-04-25T10:30:29.883469+0000","last_peered":"2024-04-25T10:30:29.883469+0000","last_clean":"2024-04-25T10:30:29.883469+0000","last_became_active":"2024-04-25T10:30:12.769470+0000","last_became_peered":"2024-04-25T10:30:12.769470+0000","last_unstale":"2024-04-25T10:30:29.883469+0000","last_undegraded":"2024-04-25T10:30:29.883469+0000","last_fullsized":"2024-04-25T10:30:29.883469+0000","mapping_epoch":34,"log_start":"0'0","ondisk_log_start":"0'0","created":34,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:30:11.733978+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:30:11.733978+0000","last_clean_scrub_stamp":"2024-04-25T10:30:11.733978+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T13:11:54.897586+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"6.1d","version":"0'0","reported_seq":12,"reported_epoch":44,"state":"active+clean","last_fresh":"2024-04-25T10:30:29.883295+0000","last_change":"2024-04-25T10:30:28.884277+0000","last_active":"2024-04-25T10:30:29.883295+0000","last_peered":"2024-04-25T10:30:29.883295+0000","last_clean":"2024-04-25T10:30:29.883295+0000","last_became_active":"2024-04-25T10:30:28.883764+0000","last_became_peered":"2024-04-25T10:30:28.883764+0000","last_unstale":"2024-04-25T10:30:29.883295+0000","last_undegraded":"2024-04-25T10:30:29.883295+0000","last_fullsized":"2024-04-25T10:30:29.883295+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":42,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:30:27.860024+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:30:27.860024+0000","last_clean_scrub_stamp":"2024-04-25T10:30:27.860024+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T18:36:07.211600+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"5.20","version":"36'9","reported_seq":38,"reported_epoch":44,"state":"active+clean","last_fresh":"2024-04-25T10:30:29.881619+0000","last_change":"2024-04-25T10:30:12.766692+0000","last_active":"2024-04-25T10:30:29.881619+0000","last_peered":"2024-04-25T10:30:29.881619+0000","last_clean":"2024-04-25T10:30:29.881619+0000","last_became_active":"2024-04-25T10:30:12.766336+0000","last_became_peered":"2024-04-25T10:30:12.766336+0000","last_unstale":"2024-04-25T10:30:29.881619+0000","last_undegraded":"2024-04-25T10:30:29.881619+0000","last_fullsized":"2024-04-25T10:30:29.881619+0000","mapping_epoch":34,"log_start":"0'0","ondisk_log_start":"0'0","created":34,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:30:11.733978+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:30:11.733978+0000","last_clean_scrub_stamp":"2024-04-25T10:30:11.733978+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T18:11:27.798982+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.21","version":"36'12","reported_seq":41,"reported_epoch":44,"state":"active+clean","last_fresh":"2024-04-25T10:30:29.882888+0000","last_change":"2024-04-25T10:30:12.774245+0000","last_active":"2024-04-25T10:30:29.882888+0000","last_peered":"2024-04-25T10:30:29.882888+0000","last_clean":"2024-04-25T10:30:29.882888+0000","last_became_active":"2024-04-25T10:30:12.773923+0000","last_became_peered":"2024-04-25T10:30:12.773923+0000","last_unstale":"2024-04-25T10:30:29.882888+0000","last_undegraded":"2024-04-25T10:30:29.882888+0000","last_fullsized":"2024-04-25T10:30:29.882888+0000","mapping_epoch":34,"log_start":"0'0","ondisk_log_start":"0'0","created":34,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:30:11.733978+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:30:11.733978+0000","last_clean_scrub_stamp":"2024-04-25T10:30:11.733978+0000","objects_scrubbed":0,"log_size":12,"log_dups_size":0,"ondisk_log_size":12,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T22:09:01.889527+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,7],"acting":[2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"5.22","version":"36'8","reported_seq":32,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-04-25T10:30:28.877192+0000","last_change":"2024-04-25T10:30:12.769958+0000","last_active":"2024-04-25T10:30:28.877192+0000","last_peered":"2024-04-25T10:30:28.877192+0000","last_clean":"2024-04-25T10:30:28.877192+0000","last_became_active":"2024-04-25T10:30:12.769590+0000","last_became_peered":"2024-04-25T10:30:12.769590+0000","last_unstale":"2024-04-25T10:30:28.877192+0000","last_undegraded":"2024-04-25T10:30:28.877192+0000","last_fullsized":"2024-04-25T10:30:28.877192+0000","mapping_epoch":34,"log_start":"0'0","ondisk_log_start":"0'0","created":34,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:30:11.733978+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:30:11.733978+0000","last_clean_scrub_stamp":"2024-04-25T10:30:11.733978+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T11:57:29.979255+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"5.23","version":"36'7","reported_seq":31,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-04-25T10:30:28.876441+0000","last_change":"2024-04-25T10:30:12.781995+0000","last_active":"2024-04-25T10:30:28.876441+0000","last_peered":"2024-04-25T10:30:28.876441+0000","last_clean":"2024-04-25T10:30:28.876441+0000","last_became_active":"2024-04-25T10:30:12.781358+0000","last_became_peered":"2024-04-25T10:30:12.781358+0000","last_unstale":"2024-04-25T10:30:28.876441+0000","last_undegraded":"2024-04-25T10:30:28.876441+0000","last_fullsized":"2024-04-25T10:30:28.876441+0000","mapping_epoch":34,"log_start":"0'0","ondisk_log_start":"0'0","created":34,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:30:11.733978+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:30:11.733978+0000","last_clean_scrub_stamp":"2024-04-25T10:30:11.733978+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T18:00:26.511360+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"5.24","version":"36'4","reported_seq":33,"reported_epoch":44,"state":"active+clean","last_fresh":"2024-04-25T10:30:29.883349+0000","last_change":"2024-04-25T10:30:12.775515+0000","last_active":"2024-04-25T10:30:29.883349+0000","last_peered":"2024-04-25T10:30:29.883349+0000","last_clean":"2024-04-25T10:30:29.883349+0000","last_became_active":"2024-04-25T10:30:12.775167+0000","last_became_peered":"2024-04-25T10:30:12.775167+0000","last_unstale":"2024-04-25T10:30:29.883349+0000","last_undegraded":"2024-04-25T10:30:29.883349+0000","last_fullsized":"2024-04-25T10:30:29.883349+0000","mapping_epoch":34,"log_start":"0'0","ondisk_log_start":"0'0","created":34,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:30:11.733978+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:30:11.733978+0000","last_clean_scrub_stamp":"2024-04-25T10:30:11.733978+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T21:35:23.984378+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,6],"acting":[2,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"5.25","version":"36'6","reported_seq":30,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-04-25T10:30:28.876545+0000","last_change":"2024-04-25T10:30:12.767505+0000","last_active":"2024-04-25T10:30:28.876545+0000","last_peered":"2024-04-25T10:30:28.876545+0000","last_clean":"2024-04-25T10:30:28.876545+0000","last_became_active":"2024-04-25T10:30:12.767149+0000","last_became_peered":"2024-04-25T10:30:12.767149+0000","last_unstale":"2024-04-25T10:30:28.876545+0000","last_undegraded":"2024-04-25T10:30:28.876545+0000","last_fullsized":"2024-04-25T10:30:28.876545+0000","mapping_epoch":34,"log_start":"0'0","ondisk_log_start":"0'0","created":34,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:30:11.733978+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:30:11.733978+0000","last_clean_scrub_stamp":"2024-04-25T10:30:11.733978+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T16:29:27.103512+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"5.26","version":"36'15","reported_seq":38,"reported_epoch":42,"state":"active+clean","last_fresh":"2024-04-25T10:30:27.877674+0000","last_change":"2024-04-25T10:30:12.771420+0000","last_active":"2024-04-25T10:30:27.877674+0000","last_peered":"2024-04-25T10:30:27.877674+0000","last_clean":"2024-04-25T10:30:27.877674+0000","last_became_active":"2024-04-25T10:30:12.771117+0000","last_became_peered":"2024-04-25T10:30:12.771117+0000","last_unstale":"2024-04-25T10:30:27.877674+0000","last_undegraded":"2024-04-25T10:30:27.877674+0000","last_fullsized":"2024-04-25T10:30:27.877674+0000","mapping_epoch":34,"log_start":"0'0","ondisk_log_start":"0'0","created":34,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:30:11.733978+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:30:11.733978+0000","last_clean_scrub_stamp":"2024-04-25T10:30:11.733978+0000","objects_scrubbed":0,"log_size":15,"log_dups_size":0,"ondisk_log_size":15,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T16:08:42.820761+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.27","version":"36'2","reported_seq":25,"reported_epoch":42,"state":"active+clean","last_fresh":"2024-04-25T10:30:27.873896+0000","last_change":"2024-04-25T10:30:12.779972+0000","last_active":"2024-04-25T10:30:27.873896+0000","last_peered":"2024-04-25T10:30:27.873896+0000","last_clean":"2024-04-25T10:30:27.873896+0000","last_became_active":"2024-04-25T10:30:12.779526+0000","last_became_peered":"2024-04-25T10:30:12.779526+0000","last_unstale":"2024-04-25T10:30:27.873896+0000","last_undegraded":"2024-04-25T10:30:27.873896+0000","last_fullsized":"2024-04-25T10:30:27.873896+0000","mapping_epoch":34,"log_start":"0'0","ondisk_log_start":"0'0","created":34,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:30:11.733978+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:30:11.733978+0000","last_clean_scrub_stamp":"2024-04-25T10:30:11.733978+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T22:04:21.879881+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.28","version":"36'6","reported_seq":35,"reported_epoch":44,"state":"active+clean","last_fresh":"2024-04-25T10:30:29.882628+0000","last_change":"2024-04-25T10:30:12.783394+0000","last_active":"2024-04-25T10:30:29.882628+0000","last_peered":"2024-04-25T10:30:29.882628+0000","last_clean":"2024-04-25T10:30:29.882628+0000","last_became_active":"2024-04-25T10:30:12.783066+0000","last_became_peered":"2024-04-25T10:30:12.783066+0000","last_unstale":"2024-04-25T10:30:29.882628+0000","last_undegraded":"2024-04-25T10:30:29.882628+0000","last_fullsized":"2024-04-25T10:30:29.882628+0000","mapping_epoch":34,"log_start":"0'0","ondisk_log_start":"0'0","created":34,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:30:11.733978+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:30:11.733978+0000","last_clean_scrub_stamp":"2024-04-25T10:30:11.733978+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T15:39:31.890064+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4],"acting":[2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"5.29","version":"36'8","reported_seq":35,"reported_epoch":44,"state":"active+clean","last_fresh":"2024-04-25T10:30:29.882807+0000","last_change":"2024-04-25T10:30:12.763911+0000","last_active":"2024-04-25T10:30:29.882807+0000","last_peered":"2024-04-25T10:30:29.882807+0000","last_clean":"2024-04-25T10:30:29.882807+0000","last_became_active":"2024-04-25T10:30:12.763578+0000","last_became_peered":"2024-04-25T10:30:12.763578+0000","last_unstale":"2024-04-25T10:30:29.882807+0000","last_undegraded":"2024-04-25T10:30:29.882807+0000","last_fullsized":"2024-04-25T10:30:29.882807+0000","mapping_epoch":34,"log_start":"0'0","ondisk_log_start":"0'0","created":34,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:30:11.733978+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:30:11.733978+0000","last_clean_scrub_stamp":"2024-04-25T10:30:11.733978+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T18:52:45.539758+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,1],"acting":[6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"5.2a","version":"36'9","reported_seq":38,"reported_epoch":44,"state":"active+clean","last_fresh":"2024-04-25T10:30:29.882996+0000","last_change":"2024-04-25T10:30:12.782730+0000","last_active":"2024-04-25T10:30:29.882996+0000","last_peered":"2024-04-25T10:30:29.882996+0000","last_clean":"2024-04-25T10:30:29.882996+0000","last_became_active":"2024-04-25T10:30:12.782261+0000","last_became_peered":"2024-04-25T10:30:12.782261+0000","last_unstale":"2024-04-25T10:30:29.882996+0000","last_undegraded":"2024-04-25T10:30:29.882996+0000","last_fullsized":"2024-04-25T10:30:29.882996+0000","mapping_epoch":34,"log_start":"0'0","ondisk_log_start":"0'0","created":34,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:30:11.733978+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:30:11.733978+0000","last_clean_scrub_stamp":"2024-04-25T10:30:11.733978+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T18:00:26.511360+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4],"acting":[2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"5.2b","version":"36'11","reported_seq":35,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-04-25T10:30:28.876504+0000","last_change":"2024-04-25T10:30:12.771344+0000","last_active":"2024-04-25T10:30:28.876504+0000","last_peered":"2024-04-25T10:30:28.876504+0000","last_clean":"2024-04-25T10:30:28.876504+0000","last_became_active":"2024-04-25T10:30:12.770949+0000","last_became_peered":"2024-04-25T10:30:12.770949+0000","last_unstale":"2024-04-25T10:30:28.876504+0000","last_undegraded":"2024-04-25T10:30:28.876504+0000","last_fullsized":"2024-04-25T10:30:28.876504+0000","mapping_epoch":34,"log_start":"0'0","ondisk_log_start":"0'0","created":34,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:30:11.733978+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:30:11.733978+0000","last_clean_scrub_stamp":"2024-04-25T10:30:11.733978+0000","objects_scrubbed":0,"log_size":11,"log_dups_size":0,"ondisk_log_size":11,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T16:29:27.103512+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"5.2c","version":"36'8","reported_seq":35,"reported_epoch":44,"state":"active+clean","last_fresh":"2024-04-25T10:30:29.883520+0000","last_change":"2024-04-25T10:30:12.772663+0000","last_active":"2024-04-25T10:30:29.883520+0000","last_peered":"2024-04-25T10:30:29.883520+0000","last_clean":"2024-04-25T10:30:29.883520+0000","last_became_active":"2024-04-25T10:30:12.772356+0000","last_became_peered":"2024-04-25T10:30:12.772356+0000","last_unstale":"2024-04-25T10:30:29.883520+0000","last_undegraded":"2024-04-25T10:30:29.883520+0000","last_fullsized":"2024-04-25T10:30:29.883520+0000","mapping_epoch":34,"log_start":"0'0","ondisk_log_start":"0'0","created":34,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:30:11.733978+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:30:11.733978+0000","last_clean_scrub_stamp":"2024-04-25T10:30:11.733978+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T12:21:33.158775+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"5.2d","version":"36'7","reported_seq":31,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-04-25T10:30:28.875358+0000","last_change":"2024-04-25T10:30:12.768688+0000","last_active":"2024-04-25T10:30:28.875358+0000","last_peered":"2024-04-25T10:30:28.875358+0000","last_clean":"2024-04-25T10:30:28.875358+0000","last_became_active":"2024-04-25T10:30:12.768382+0000","last_became_peered":"2024-04-25T10:30:12.768382+0000","last_unstale":"2024-04-25T10:30:28.875358+0000","last_undegraded":"2024-04-25T10:30:28.875358+0000","last_fullsized":"2024-04-25T10:30:28.875358+0000","mapping_epoch":34,"log_start":"0'0","ondisk_log_start":"0'0","created":34,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:30:11.733978+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:30:11.733978+0000","last_clean_scrub_stamp":"2024-04-25T10:30:11.733978+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T18:26:06.164667+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"5.2e","version":"36'10","reported_seq":34,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-04-25T10:30:28.876239+0000","last_change":"2024-04-25T10:30:12.766393+0000","last_active":"2024-04-25T10:30:28.876239+0000","last_peered":"2024-04-25T10:30:28.876239+0000","last_clean":"2024-04-25T10:30:28.876239+0000","last_became_active":"2024-04-25T10:30:12.766096+0000","last_became_peered":"2024-04-25T10:30:12.766096+0000","last_unstale":"2024-04-25T10:30:28.876239+0000","last_undegraded":"2024-04-25T10:30:28.876239+0000","last_fullsized":"2024-04-25T10:30:28.876239+0000","mapping_epoch":34,"log_start":"0'0","ondisk_log_start":"0'0","created":34,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:30:11.733978+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:30:11.733978+0000","last_clean_scrub_stamp":"2024-04-25T10:30:11.733978+0000","objects_scrubbed":0,"log_size":10,"log_dups_size":0,"ondisk_log_size":10,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T16:49:00.268423+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"5.2f","version":"36'9","reported_seq":38,"reported_epoch":44,"state":"active+clean","last_fresh":"2024-04-25T10:30:29.882690+0000","last_change":"2024-04-25T10:30:12.766616+0000","last_active":"2024-04-25T10:30:29.882690+0000","last_peered":"2024-04-25T10:30:29.882690+0000","last_clean":"2024-04-25T10:30:29.882690+0000","last_became_active":"2024-04-25T10:30:12.766290+0000","last_became_peered":"2024-04-25T10:30:12.766290+0000","last_unstale":"2024-04-25T10:30:29.882690+0000","last_undegraded":"2024-04-25T10:30:29.882690+0000","last_fullsized":"2024-04-25T10:30:29.882690+0000","mapping_epoch":34,"log_start":"0'0","ondisk_log_start":"0'0","created":34,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:30:11.733978+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:30:11.733978+0000","last_clean_scrub_stamp":"2024-04-25T10:30:11.733978+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T13:22:50.975199+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":2097152,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":2048,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.30","version":"36'9","reported_seq":33,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-04-25T10:30:28.876615+0000","last_change":"2024-04-25T10:30:12.765989+0000","last_active":"2024-04-25T10:30:28.876615+0000","last_peered":"2024-04-25T10:30:28.876615+0000","last_clean":"2024-04-25T10:30:28.876615+0000","last_became_active":"2024-04-25T10:30:12.765648+0000","last_became_peered":"2024-04-25T10:30:12.765648+0000","last_unstale":"2024-04-25T10:30:28.876615+0000","last_undegraded":"2024-04-25T10:30:28.876615+0000","last_fullsized":"2024-04-25T10:30:28.876615+0000","mapping_epoch":34,"log_start":"0'0","ondisk_log_start":"0'0","created":34,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:30:11.733978+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:30:11.733978+0000","last_clean_scrub_stamp":"2024-04-25T10:30:11.733978+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T16:29:40.621322+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"5.31","version":"36'7","reported_seq":31,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-04-25T10:30:28.877047+0000","last_change":"2024-04-25T10:30:12.777324+0000","last_active":"2024-04-25T10:30:28.877047+0000","last_peered":"2024-04-25T10:30:28.877047+0000","last_clean":"2024-04-25T10:30:28.877047+0000","last_became_active":"2024-04-25T10:30:12.776983+0000","last_became_peered":"2024-04-25T10:30:12.776983+0000","last_unstale":"2024-04-25T10:30:28.877047+0000","last_undegraded":"2024-04-25T10:30:28.877047+0000","last_fullsized":"2024-04-25T10:30:28.877047+0000","mapping_epoch":34,"log_start":"0'0","ondisk_log_start":"0'0","created":34,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:30:11.733978+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:30:11.733978+0000","last_clean_scrub_stamp":"2024-04-25T10:30:11.733978+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T20:33:24.458243+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"5.32","version":"36'8","reported_seq":31,"reported_epoch":42,"state":"active+clean","last_fresh":"2024-04-25T10:30:27.876591+0000","last_change":"2024-04-25T10:30:12.772243+0000","last_active":"2024-04-25T10:30:27.876591+0000","last_peered":"2024-04-25T10:30:27.876591+0000","last_clean":"2024-04-25T10:30:27.876591+0000","last_became_active":"2024-04-25T10:30:12.771936+0000","last_became_peered":"2024-04-25T10:30:12.771936+0000","last_unstale":"2024-04-25T10:30:27.876591+0000","last_undegraded":"2024-04-25T10:30:27.876591+0000","last_fullsized":"2024-04-25T10:30:27.876591+0000","mapping_epoch":34,"log_start":"0'0","ondisk_log_start":"0'0","created":34,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:30:11.733978+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:30:11.733978+0000","last_clean_scrub_stamp":"2024-04-25T10:30:11.733978+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T13:11:22.432926+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5],"acting":[4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.33","version":"36'10","reported_seq":37,"reported_epoch":44,"state":"active+clean","last_fresh":"2024-04-25T10:30:29.883324+0000","last_change":"2024-04-25T10:30:12.777499+0000","last_active":"2024-04-25T10:30:29.883324+0000","last_peered":"2024-04-25T10:30:29.883324+0000","last_clean":"2024-04-25T10:30:29.883324+0000","last_became_active":"2024-04-25T10:30:12.776773+0000","last_became_peered":"2024-04-25T10:30:12.776773+0000","last_unstale":"2024-04-25T10:30:29.883324+0000","last_undegraded":"2024-04-25T10:30:29.883324+0000","last_fullsized":"2024-04-25T10:30:29.883324+0000","mapping_epoch":34,"log_start":"0'0","ondisk_log_start":"0'0","created":34,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:30:11.733978+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:30:11.733978+0000","last_clean_scrub_stamp":"2024-04-25T10:30:11.733978+0000","objects_scrubbed":0,"log_size":10,"log_dups_size":0,"ondisk_log_size":10,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T18:21:32.178524+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"5.34","version":"36'7","reported_seq":32,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-04-25T10:30:28.874618+0000","last_change":"2024-04-25T10:30:12.761162+0000","last_active":"2024-04-25T10:30:28.874618+0000","last_peered":"2024-04-25T10:30:28.874618+0000","last_clean":"2024-04-25T10:30:28.874618+0000","last_became_active":"2024-04-25T10:30:12.760835+0000","last_became_peered":"2024-04-25T10:30:12.760835+0000","last_unstale":"2024-04-25T10:30:28.874618+0000","last_undegraded":"2024-04-25T10:30:28.874618+0000","last_fullsized":"2024-04-25T10:30:28.874618+0000","mapping_epoch":34,"log_start":"0'0","ondisk_log_start":"0'0","created":34,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:30:11.733978+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:30:11.733978+0000","last_clean_scrub_stamp":"2024-04-25T10:30:11.733978+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T18:59:20.693761+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"5.35","version":"36'6","reported_seq":35,"reported_epoch":44,"state":"active+clean","last_fresh":"2024-04-25T10:30:29.882303+0000","last_change":"2024-04-25T10:30:12.776164+0000","last_active":"2024-04-25T10:30:29.882303+0000","last_peered":"2024-04-25T10:30:29.882303+0000","last_clean":"2024-04-25T10:30:29.882303+0000","last_became_active":"2024-04-25T10:30:12.775455+0000","last_became_peered":"2024-04-25T10:30:12.775455+0000","last_unstale":"2024-04-25T10:30:29.882303+0000","last_undegraded":"2024-04-25T10:30:29.882303+0000","last_fullsized":"2024-04-25T10:30:29.882303+0000","mapping_epoch":34,"log_start":"0'0","ondisk_log_start":"0'0","created":34,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:30:11.733978+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:30:11.733978+0000","last_clean_scrub_stamp":"2024-04-25T10:30:11.733978+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T21:45:24.780979+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.36","version":"36'15","reported_seq":38,"reported_epoch":42,"state":"active+clean","last_fresh":"2024-04-25T10:30:27.877229+0000","last_change":"2024-04-25T10:30:12.773138+0000","last_active":"2024-04-25T10:30:27.877229+0000","last_peered":"2024-04-25T10:30:27.877229+0000","last_clean":"2024-04-25T10:30:27.877229+0000","last_became_active":"2024-04-25T10:30:12.772854+0000","last_became_peered":"2024-04-25T10:30:12.772854+0000","last_unstale":"2024-04-25T10:30:27.877229+0000","last_undegraded":"2024-04-25T10:30:27.877229+0000","last_fullsized":"2024-04-25T10:30:27.877229+0000","mapping_epoch":34,"log_start":"0'0","ondisk_log_start":"0'0","created":34,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:30:11.733978+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:30:11.733978+0000","last_clean_scrub_stamp":"2024-04-25T10:30:11.733978+0000","objects_scrubbed":0,"log_size":15,"log_dups_size":0,"ondisk_log_size":15,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T19:24:03.893622+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5],"acting":[4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.37","version":"36'11","reported_seq":36,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-04-25T10:30:28.875064+0000","last_change":"2024-04-25T10:30:12.773184+0000","last_active":"2024-04-25T10:30:28.875064+0000","last_peered":"2024-04-25T10:30:28.875064+0000","last_clean":"2024-04-25T10:30:28.875064+0000","last_became_active":"2024-04-25T10:30:12.772866+0000","last_became_peered":"2024-04-25T10:30:12.772866+0000","last_unstale":"2024-04-25T10:30:28.875064+0000","last_undegraded":"2024-04-25T10:30:28.875064+0000","last_fullsized":"2024-04-25T10:30:28.875064+0000","mapping_epoch":34,"log_start":"0'0","ondisk_log_start":"0'0","created":34,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:30:11.733978+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:30:11.733978+0000","last_clean_scrub_stamp":"2024-04-25T10:30:11.733978+0000","objects_scrubbed":0,"log_size":11,"log_dups_size":0,"ondisk_log_size":11,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T14:01:06.791796+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"5.38","version":"36'9","reported_seq":38,"reported_epoch":44,"state":"active+clean","last_fresh":"2024-04-25T10:30:29.882611+0000","last_change":"2024-04-25T10:30:12.774614+0000","last_active":"2024-04-25T10:30:29.882611+0000","last_peered":"2024-04-25T10:30:29.882611+0000","last_clean":"2024-04-25T10:30:29.882611+0000","last_became_active":"2024-04-25T10:30:12.774213+0000","last_became_peered":"2024-04-25T10:30:12.774213+0000","last_unstale":"2024-04-25T10:30:29.882611+0000","last_undegraded":"2024-04-25T10:30:29.882611+0000","last_fullsized":"2024-04-25T10:30:29.882611+0000","mapping_epoch":34,"log_start":"0'0","ondisk_log_start":"0'0","created":34,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:30:11.733978+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:30:11.733978+0000","last_clean_scrub_stamp":"2024-04-25T10:30:11.733978+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T20:33:24.458243+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,6],"acting":[2,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"5.39","version":"36'5","reported_seq":34,"reported_epoch":44,"state":"active+clean","last_fresh":"2024-04-25T10:30:29.882763+0000","last_change":"2024-04-25T10:30:12.781210+0000","last_active":"2024-04-25T10:30:29.882763+0000","last_peered":"2024-04-25T10:30:29.882763+0000","last_clean":"2024-04-25T10:30:29.882763+0000","last_became_active":"2024-04-25T10:30:12.780874+0000","last_became_peered":"2024-04-25T10:30:12.780874+0000","last_unstale":"2024-04-25T10:30:29.882763+0000","last_undegraded":"2024-04-25T10:30:29.882763+0000","last_fullsized":"2024-04-25T10:30:29.882763+0000","mapping_epoch":34,"log_start":"0'0","ondisk_log_start":"0'0","created":34,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:30:11.733978+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:30:11.733978+0000","last_clean_scrub_stamp":"2024-04-25T10:30:11.733978+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T16:29:40.621322+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"5.3a","version":"36'6","reported_seq":30,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-04-25T10:30:28.874520+0000","last_change":"2024-04-25T10:30:12.775534+0000","last_active":"2024-04-25T10:30:28.874520+0000","last_peered":"2024-04-25T10:30:28.874520+0000","last_clean":"2024-04-25T10:30:28.874520+0000","last_became_active":"2024-04-25T10:30:12.775180+0000","last_became_peered":"2024-04-25T10:30:12.775180+0000","last_unstale":"2024-04-25T10:30:28.874520+0000","last_undegraded":"2024-04-25T10:30:28.874520+0000","last_fullsized":"2024-04-25T10:30:28.874520+0000","mapping_epoch":34,"log_start":"0'0","ondisk_log_start":"0'0","created":34,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:30:11.733978+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:30:11.733978+0000","last_clean_scrub_stamp":"2024-04-25T10:30:11.733978+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T18:31:49.260513+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"5.3b","version":"36'4","reported_seq":28,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-04-25T10:30:28.875655+0000","last_change":"2024-04-25T10:30:12.762141+0000","last_active":"2024-04-25T10:30:28.875655+0000","last_peered":"2024-04-25T10:30:28.875655+0000","last_clean":"2024-04-25T10:30:28.875655+0000","last_became_active":"2024-04-25T10:30:12.761805+0000","last_became_peered":"2024-04-25T10:30:12.761805+0000","last_unstale":"2024-04-25T10:30:28.875655+0000","last_undegraded":"2024-04-25T10:30:28.875655+0000","last_fullsized":"2024-04-25T10:30:28.875655+0000","mapping_epoch":34,"log_start":"0'0","ondisk_log_start":"0'0","created":34,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:30:11.733978+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:30:11.733978+0000","last_clean_scrub_stamp":"2024-04-25T10:30:11.733978+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T19:56:57.765496+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6],"acting":[5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"5.3c","version":"36'8","reported_seq":31,"reported_epoch":42,"state":"active+clean","last_fresh":"2024-04-25T10:30:27.877379+0000","last_change":"2024-04-25T10:30:12.781723+0000","last_active":"2024-04-25T10:30:27.877379+0000","last_peered":"2024-04-25T10:30:27.877379+0000","last_clean":"2024-04-25T10:30:27.877379+0000","last_became_active":"2024-04-25T10:30:12.781357+0000","last_became_peered":"2024-04-25T10:30:12.781357+0000","last_unstale":"2024-04-25T10:30:27.877379+0000","last_undegraded":"2024-04-25T10:30:27.877379+0000","last_fullsized":"2024-04-25T10:30:27.877379+0000","mapping_epoch":34,"log_start":"0'0","ondisk_log_start":"0'0","created":34,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:30:11.733978+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:30:11.733978+0000","last_clean_scrub_stamp":"2024-04-25T10:30:11.733978+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T12:06:29.897651+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.3d","version":"36'7","reported_seq":32,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-04-25T10:30:28.875135+0000","last_change":"2024-04-25T10:30:12.768292+0000","last_active":"2024-04-25T10:30:28.875135+0000","last_peered":"2024-04-25T10:30:28.875135+0000","last_clean":"2024-04-25T10:30:28.875135+0000","last_became_active":"2024-04-25T10:30:12.767909+0000","last_became_peered":"2024-04-25T10:30:12.767909+0000","last_unstale":"2024-04-25T10:30:28.875135+0000","last_undegraded":"2024-04-25T10:30:28.875135+0000","last_fullsized":"2024-04-25T10:30:28.875135+0000","mapping_epoch":34,"log_start":"0'0","ondisk_log_start":"0'0","created":34,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:30:11.733978+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:30:11.733978+0000","last_clean_scrub_stamp":"2024-04-25T10:30:11.733978+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T15:17:30.002665+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"5.3e","version":"36'4","reported_seq":28,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-04-25T10:30:28.878982+0000","last_change":"2024-04-25T10:30:12.780210+0000","last_active":"2024-04-25T10:30:28.878982+0000","last_peered":"2024-04-25T10:30:28.878982+0000","last_clean":"2024-04-25T10:30:28.878982+0000","last_became_active":"2024-04-25T10:30:12.779867+0000","last_became_peered":"2024-04-25T10:30:12.779867+0000","last_unstale":"2024-04-25T10:30:28.878982+0000","last_undegraded":"2024-04-25T10:30:28.878982+0000","last_fullsized":"2024-04-25T10:30:28.878982+0000","mapping_epoch":34,"log_start":"0'0","ondisk_log_start":"0'0","created":34,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:30:11.733978+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:30:11.733978+0000","last_clean_scrub_stamp":"2024-04-25T10:30:11.733978+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T20:33:24.458243+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"5.3f","version":"36'8","reported_seq":37,"reported_epoch":44,"state":"active+clean","last_fresh":"2024-04-25T10:30:29.883448+0000","last_change":"2024-04-25T10:30:12.772509+0000","last_active":"2024-04-25T10:30:29.883448+0000","last_peered":"2024-04-25T10:30:29.883448+0000","last_clean":"2024-04-25T10:30:29.883448+0000","last_became_active":"2024-04-25T10:30:12.772018+0000","last_became_peered":"2024-04-25T10:30:12.772018+0000","last_unstale":"2024-04-25T10:30:29.883448+0000","last_undegraded":"2024-04-25T10:30:29.883448+0000","last_fullsized":"2024-04-25T10:30:29.883448+0000","mapping_epoch":34,"log_start":"0'0","ondisk_log_start":"0'0","created":34,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:30:11.733978+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:30:11.733978+0000","last_clean_scrub_stamp":"2024-04-25T10:30:11.733978+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T11:57:29.979255+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]}],"pool_stats":[{"poolid":6,"num_pg":32,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":64,"acting":64,"num_store_stats":8},{"poolid":5,"num_pg":64,"stat_sum":{"num_bytes":6291456,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":6144,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":12582912,"data_stored":12582912,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":504,"ondisk_log_size":504,"up":128,"acting":128,"num_store_stats":8},{"poolid":4,"num_pg":32,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":1,"num_read_kb":0,"num_write":96,"num_write_kb":40,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":83,"ondisk_log_size":83,"up":64,"acting":64,"num_store_stats":8},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":66,"num_read_kb":55,"num_write":101,"num_write_kb":1628,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1187840,"data_stored":1180736,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":72,"ondisk_log_size":72,"up":2,"acting":2,"num_store_stats":2}],"osd_stats":[{"osd":7,"up_from":15,"seq":64424509462,"num_pgs":21,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":31596,"kb_used_data":4572,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94340244,"statfs":{"total":96636764160,"available":96604409856,"internally_reserved":0,"allocated":4681728,"data_stored":4394877,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1590,"internal_metadata":27654602},"hb_peers":[0,1,2,3,4,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]},{"osd":6,"up_from":15,"seq":64424509463,"num_pgs":25,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27504,"kb_used_data":476,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344336,"statfs":{"total":96636764160,"available":96608600064,"internally_reserved":0,"allocated":487424,"data_stored":200573,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":27654598},"hb_peers":[0,1,2,3,4,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]},{"osd":5,"up_from":15,"seq":64424509464,"num_pgs":18,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":28084,"kb_used_data":1056,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94343756,"statfs":{"total":96636764160,"available":96608006144,"internally_reserved":0,"allocated":1081344,"data_stored":790941,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]},{"osd":3,"up_from":15,"seq":64424509464,"num_pgs":25,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":33644,"kb_used_data":6620,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94338196,"statfs":{"total":96636764160,"available":96602312704,"internally_reserved":0,"allocated":6778880,"data_stored":6492029,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1593,"internal_metadata":27654599},"hb_peers":[0,1,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]},{"osd":2,"up_from":15,"seq":64424509464,"num_pgs":27,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":29548,"kb_used_data":2524,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94342292,"statfs":{"total":96636764160,"available":96606507008,"internally_reserved":0,"allocated":2584576,"data_stored":2297725,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]},{"osd":1,"up_from":15,"seq":64424509462,"num_pgs":25,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":28088,"kb_used_data":1056,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94343752,"statfs":{"total":96636764160,"available":96608002048,"internally_reserved":0,"allocated":1081344,"data_stored":790941,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]},{"osd":0,"up_from":15,"seq":64424509463,"num_pgs":21,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27504,"kb_used_data":476,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344336,"statfs":{"total":96636764160,"available":96608600064,"internally_reserved":0,"allocated":487424,"data_stored":200573,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[1,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]},{"osd":4,"up_from":13,"seq":55834574870,"num_pgs":35,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27480,"kb_used_data":452,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344360,"statfs":{"total":96636764160,"available":96608624640,"internally_reserved":0,"allocated":462848,"data_stored":186484,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":2097152,"data_stored":2097152,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":6291456,"data_stored":6291456,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":4194304,"data_stored":4194304,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-04-25T10:30:33.320 DEBUG:teuthology.orchestra.run.smithi037:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-04-25T10:30:33.467 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:33.465+0000 7f3225227640 1 Processor -- start 2024-04-25T10:30:33.467 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:33.465+0000 7f3225227640 1 -- start start 2024-04-25T10:30:33.469 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:33.466+0000 7f3225227640 1 --2- >> v2:172.21.15.37:3300/0 conn(0x7f32200b7600 0x7f32200b7a10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:30:33.469 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:33.466+0000 7f3225227640 1 --2- >> v2:172.21.15.203:3300/0 conn(0x7f32200d4560 0x7f32200d4970 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:30:33.469 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:33.466+0000 7f3225227640 1 -- --> v1:172.21.15.203:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f32200b7f50 con 0x7f32200b4bb0 2024-04-25T10:30:33.469 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:33.466+0000 7f3225227640 1 -- --> v2:172.21.15.37:3300/0 -- mon_getmap magic: 0 -- 0x7f3220161190 con 0x7f32200b7600 2024-04-25T10:30:33.469 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:33.466+0000 7f3225227640 1 -- --> v2:172.21.15.203:3300/0 -- mon_getmap magic: 0 -- 0x7f3220161470 con 0x7f32200d4560 2024-04-25T10:30:33.469 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:33.466+0000 7f321e59c640 1 --2- >> v2:172.21.15.37:3300/0 conn(0x7f32200b7600 0x7f32200b7a10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:30:33.469 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:33.466+0000 7f321e59c640 1 --2- >> v2:172.21.15.37:3300/0 conn(0x7f32200b7600 0x7f32200b7a10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.37:3300/0 says I am v2:172.21.15.37:55450/0 (socket says 172.21.15.37:55450) 2024-04-25T10:30:33.469 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:33.466+0000 7f321e59c640 1 -- 172.21.15.37:0/1623561173 learned_addr learned my addr 172.21.15.37:0/1623561173 (peer_addr_for_me v2:172.21.15.37:0/0) 2024-04-25T10:30:33.469 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:33.466+0000 7f321f59e640 1 --2- 172.21.15.37:0/1623561173 >> v2:172.21.15.203:3300/0 conn(0x7f32200d4560 0x7f32200d4970 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:30:33.469 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:33.466+0000 7f321e59c640 1 -- 172.21.15.37:0/1623561173 >> v2:172.21.15.203:3300/0 conn(0x7f32200d4560 msgr2=0x7f32200d4970 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:30:33.469 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:33.466+0000 7f321e59c640 1 --2- 172.21.15.37:0/1623561173 >> v2:172.21.15.203:3300/0 conn(0x7f32200d4560 0x7f32200d4970 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:30:33.469 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:33.466+0000 7f321e59c640 1 -- 172.21.15.37:0/1623561173 >> v1:172.21.15.203:6789/0 conn(0x7f32200b4bb0 legacy=0x7f32201646b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:30:33.470 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:33.466+0000 7f321e59c640 1 -- 172.21.15.37:0/1623561173 --> v2:172.21.15.37:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f3220163eb0 con 0x7f32200b7600 2024-04-25T10:30:33.471 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:33.467+0000 7f321e59c640 1 --2- 172.21.15.37:0/1623561173 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f32200b7600 0x7f32200b7a10 secure :-1 s=READY pgs=534 cs=0 l=1 rev1=1 crypto rx=0x7f32200d3b00 tx=0x7f321402a270 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ed26d63becde66bd server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:30:33.471 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:33.467+0000 7f321dd9b640 1 -- 172.21.15.37:0/1623561173 <== mon.0 v2:172.21.15.37:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3214037070 con 0x7f32200b7600 2024-04-25T10:30:33.471 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:33.467+0000 7f321dd9b640 1 -- 172.21.15.37:0/1623561173 <== mon.0 v2:172.21.15.37:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f3214032440 con 0x7f32200b7600 2024-04-25T10:30:33.471 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:33.467+0000 7f321dd9b640 1 -- 172.21.15.37:0/1623561173 <== mon.0 v2:172.21.15.37:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3214032810 con 0x7f32200b7600 2024-04-25T10:30:33.471 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:33.468+0000 7f3225227640 1 -- 172.21.15.37:0/1623561173 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f32200b7600 msgr2=0x7f32200b7a10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:30:33.471 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:33.468+0000 7f3225227640 1 --2- 172.21.15.37:0/1623561173 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f32200b7600 0x7f32200b7a10 secure :-1 s=READY pgs=534 cs=0 l=1 rev1=1 crypto rx=0x7f32200d3b00 tx=0x7f321402a270 comp rx=0 tx=0).stop 2024-04-25T10:30:33.471 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:33.468+0000 7f3225227640 1 -- 172.21.15.37:0/1623561173 shutdown_connections 2024-04-25T10:30:33.471 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:33.468+0000 7f3225227640 1 --2- 172.21.15.37:0/1623561173 >> v2:172.21.15.203:3300/0 conn(0x7f32200d4560 0x7f32200d4970 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:30:33.471 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:33.468+0000 7f3225227640 1 --2- 172.21.15.37:0/1623561173 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f32200b7600 0x7f32200b7a10 unknown :-1 s=CLOSED pgs=534 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:30:33.471 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:33.468+0000 7f3225227640 1 -- 172.21.15.37:0/1623561173 >> 172.21.15.37:0/1623561173 conn(0x7f32200d27e0 msgr2=0x7f32200d2bf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:30:33.471 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:33.468+0000 7f3225227640 1 -- 172.21.15.37:0/1623561173 shutdown_connections 2024-04-25T10:30:33.471 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:33.468+0000 7f3225227640 1 -- 172.21.15.37:0/1623561173 wait complete. 2024-04-25T10:30:33.472 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:33.470+0000 7f3225227640 1 Processor -- start 2024-04-25T10:30:33.472 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:33.470+0000 7f3225227640 1 -- start start 2024-04-25T10:30:33.475 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:33.471+0000 7f3225227640 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f32200b4bb0 0x7f32200c83f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:30:33.475 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:33.471+0000 7f3225227640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f32200b7600 0x7f3220156b10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:30:33.475 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:33.471+0000 7f3225227640 1 --2- >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f32200d4560 0x7f3220157050 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:30:33.475 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:33.471+0000 7f3225227640 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 -- 0x7f32200d09e0 con 0x7f32200b7600 2024-04-25T10:30:33.475 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:33.471+0000 7f3225227640 1 -- --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_getmap magic: 0 -- 0x7f32200b4fc0 con 0x7f32200b4bb0 2024-04-25T10:30:33.475 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:33.471+0000 7f3225227640 1 -- --> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] -- mon_getmap magic: 0 -- 0x7f32200d0b60 con 0x7f32200d4560 2024-04-25T10:30:33.475 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:33.472+0000 7f321e59c640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f32200b7600 0x7f3220156b10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:30:33.475 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:33.472+0000 7f321e59c640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f32200b7600 0x7f3220156b10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.37:3300/0 says I am v2:172.21.15.37:55456/0 (socket says 172.21.15.37:55456) 2024-04-25T10:30:33.475 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:33.472+0000 7f321e59c640 1 -- 172.21.15.37:0/4163485090 learned_addr learned my addr 172.21.15.37:0/4163485090 (peer_addr_for_me v2:172.21.15.37:0/0) 2024-04-25T10:30:33.476 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:33.472+0000 7f321ed9d640 1 --2- 172.21.15.37:0/4163485090 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f32200b4bb0 0x7f32200c83f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:30:33.476 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:33.472+0000 7f321f59e640 1 --2- 172.21.15.37:0/4163485090 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f32200d4560 0x7f3220157050 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:30:33.476 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:33.472+0000 7f321e59c640 1 -- 172.21.15.37:0/4163485090 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f32200d4560 msgr2=0x7f3220157050 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:30:33.476 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:33.472+0000 7f321e59c640 1 --2- 172.21.15.37:0/4163485090 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f32200d4560 0x7f3220157050 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:30:33.476 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:33.472+0000 7f321e59c640 1 -- 172.21.15.37:0/4163485090 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f32200b4bb0 msgr2=0x7f32200c83f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:30:33.476 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:33.472+0000 7f321e59c640 1 --2- 172.21.15.37:0/4163485090 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f32200b4bb0 0x7f32200c83f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:30:33.476 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:33.472+0000 7f321e59c640 1 -- 172.21.15.37:0/4163485090 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f32201578e0 con 0x7f32200b7600 2024-04-25T10:30:33.476 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:33.472+0000 7f321e59c640 1 --2- 172.21.15.37:0/4163485090 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f32200b7600 0x7f3220156b10 secure :-1 s=READY pgs=535 cs=0 l=1 rev1=1 crypto rx=0x7f321402a2b0 tx=0x7f32140046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:30:33.476 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:33.472+0000 7f32077fe640 1 -- 172.21.15.37:0/4163485090 <== mon.0 v2:172.21.15.37:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3214037040 con 0x7f32200b7600 2024-04-25T10:30:33.476 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:33.472+0000 7f32077fe640 1 -- 172.21.15.37:0/4163485090 <== mon.0 v2:172.21.15.37:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f3214042070 con 0x7f32200b7600 2024-04-25T10:30:33.476 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:33.473+0000 7f3225227640 1 -- 172.21.15.37:0/4163485090 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f3220157bc0 con 0x7f32200b7600 2024-04-25T10:30:33.476 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:33.473+0000 7f32077fe640 1 -- 172.21.15.37:0/4163485090 <== mon.0 v2:172.21.15.37:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3214030880 con 0x7f32200b7600 2024-04-25T10:30:33.476 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:33.473+0000 7f3225227640 1 -- 172.21.15.37:0/4163485090 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f3220158180 con 0x7f32200b7600 2024-04-25T10:30:33.477 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:33.473+0000 7f3225227640 1 -- 172.21.15.37:0/4163485090 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f322006a140 con 0x7f32200b7600 2024-04-25T10:30:33.477 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:33.474+0000 7f32077fe640 1 -- 172.21.15.37:0/4163485090 <== mon.0 v2:172.21.15.37:3300/0 4 ==== mgrmap(e 5) ==== 96782+0+0 (secure 0 0 0) 0x7f3214030a20 con 0x7f32200b7600 2024-04-25T10:30:33.477 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:33.475+0000 7f32077fe640 1 --2- 172.21.15.37:0/4163485090 >> [v2:172.21.15.203:6832/2418205237,v1:172.21.15.203:6833/2418205237] conn(0x7f320c0744e0 0x7f320c0769f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:30:33.478 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:33.475+0000 7f32077fe640 1 -- 172.21.15.37:0/4163485090 <== mon.0 v2:172.21.15.37:3300/0 5 ==== osd_map(44..44 src has 1..44) ==== 6248+0+0 (secure 0 0 0) 0x7f32140b5cb0 con 0x7f32200b7600 2024-04-25T10:30:33.478 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:33.475+0000 7f321ed9d640 1 --2- 172.21.15.37:0/4163485090 >> [v2:172.21.15.203:6832/2418205237,v1:172.21.15.203:6833/2418205237] conn(0x7f320c0744e0 0x7f320c0769f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:30:33.478 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:33.476+0000 7f321ed9d640 1 --2- 172.21.15.37:0/4163485090 >> [v2:172.21.15.203:6832/2418205237,v1:172.21.15.203:6833/2418205237] conn(0x7f320c0744e0 0x7f320c0769f0 secure :-1 s=READY pgs=256 cs=0 l=1 rev1=1 crypto rx=0x7f32200d6c00 tx=0x7f3210009040 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:30:33.481 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:33.479+0000 7f32077fe640 1 -- 172.21.15.37:0/4163485090 <== mon.0 v2:172.21.15.37:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187246 (secure 0 0 0) 0x7f321407fab0 con 0x7f32200b7600 2024-04-25T10:30:33.666 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:33.664+0000 7f3225227640 1 -- 172.21.15.37:0/4163485090 --> [v2:172.21.15.203:6832/2418205237,v1:172.21.15.203:6833/2418205237] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) -- 0x7f32200b6e90 con 0x7f320c0744e0 2024-04-25T10:30:33.674 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:33.672+0000 7f32077fe640 1 -- 172.21.15.37:0/4163485090 <== mgr.4100 v2:172.21.15.203:6832/2418205237 1 ==== mgr_command_reply(tid 0: 0 dumped all) ==== 18+0+332031 (secure 0 0 0) 0x7f32200b6e90 con 0x7f320c0744e0 2024-04-25T10:30:33.678 INFO:teuthology.orchestra.run.smithi037.stdout: 2024-04-25T10:30:33.679 INFO:teuthology.orchestra.run.smithi037.stderr:dumped all 2024-04-25T10:30:33.681 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:33.676+0000 7f3225227640 1 -- 172.21.15.37:0/4163485090 >> [v2:172.21.15.203:6832/2418205237,v1:172.21.15.203:6833/2418205237] conn(0x7f320c0744e0 msgr2=0x7f320c0769f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:30:33.681 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:33.676+0000 7f3225227640 1 --2- 172.21.15.37:0/4163485090 >> [v2:172.21.15.203:6832/2418205237,v1:172.21.15.203:6833/2418205237] conn(0x7f320c0744e0 0x7f320c0769f0 secure :-1 s=READY pgs=256 cs=0 l=1 rev1=1 crypto rx=0x7f32200d6c00 tx=0x7f3210009040 comp rx=0 tx=0).stop 2024-04-25T10:30:33.681 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:33.676+0000 7f3225227640 1 -- 172.21.15.37:0/4163485090 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f32200b7600 msgr2=0x7f3220156b10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:30:33.681 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:33.676+0000 7f3225227640 1 --2- 172.21.15.37:0/4163485090 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f32200b7600 0x7f3220156b10 secure :-1 s=READY pgs=535 cs=0 l=1 rev1=1 crypto rx=0x7f321402a2b0 tx=0x7f32140046c0 comp rx=0 tx=0).stop 2024-04-25T10:30:33.682 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:33.676+0000 7f3225227640 1 -- 172.21.15.37:0/4163485090 shutdown_connections 2024-04-25T10:30:33.682 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:33.676+0000 7f3225227640 1 --2- 172.21.15.37:0/4163485090 >> [v2:172.21.15.203:6832/2418205237,v1:172.21.15.203:6833/2418205237] conn(0x7f320c0744e0 0x7f320c0769f0 unknown :-1 s=CLOSED pgs=256 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:30:33.682 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:33.676+0000 7f3225227640 1 --2- 172.21.15.37:0/4163485090 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f32200d4560 0x7f3220157050 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:30:33.682 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:33.676+0000 7f3225227640 1 --2- 172.21.15.37:0/4163485090 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f32200b7600 0x7f3220156b10 unknown :-1 s=CLOSED pgs=535 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:30:33.682 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:33.676+0000 7f3225227640 1 --2- 172.21.15.37:0/4163485090 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f32200b4bb0 0x7f32200c83f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:30:33.682 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:33.676+0000 7f3225227640 1 -- 172.21.15.37:0/4163485090 >> 172.21.15.37:0/4163485090 conn(0x7f32200d27e0 msgr2=0x7f32201631e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:30:33.682 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:33.676+0000 7f3225227640 1 -- 172.21.15.37:0/4163485090 shutdown_connections 2024-04-25T10:30:33.682 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:33.676+0000 7f3225227640 1 -- 172.21.15.37:0/4163485090 wait complete. 2024-04-25T10:30:33.689 INFO:teuthology.orchestra.run.smithi037.stdout:{"pg_ready":true,"pg_map":{"version":87,"stamp":"2024-04-25T10:30:32.207732+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":6881824,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":67,"num_read_kb":55,"num_write":201,"num_write_kb":7812,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":659,"ondisk_log_size":659,"up":258,"acting":258,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":197,"num_osds":8,"num_per_pool_osds":8,"num_per_pool_omap_osds":8,"kb":754974720,"kb_used":233448,"kb_used_data":17232,"kb_used_omap":12,"kb_used_meta":216051,"kb_avail":754741272,"statfs":{"total":773094113280,"available":772855062528,"internally_reserved":0,"allocated":17645568,"data_stored":15354143,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":12752,"internal_metadata":221236784},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":14,"apply_latency_ms":14,"commit_latency_ns":14000000,"apply_latency_ns":14000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":-16365,"num_objects":-23,"num_object_clones":0,"num_object_copies":-46,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":-23,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":23,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":-15,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"6.001255"},"pg_stats":[{"pgid":"1.0","version":"18'72","reported_seq":145,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-04-25T10:30:28.875202+0000","last_change":"2024-04-25T10:28:57.972696+0000","last_active":"2024-04-25T10:30:28.875202+0000","last_peered":"2024-04-25T10:30:28.875202+0000","last_clean":"2024-04-25T10:30:28.875202+0000","last_became_active":"2024-04-25T10:28:57.972373+0000","last_became_peered":"2024-04-25T10:28:57.972373+0000","last_unstale":"2024-04-25T10:30:28.875202+0000","last_undegraded":"2024-04-25T10:30:28.875202+0000","last_fullsized":"2024-04-25T10:30:28.875202+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:28:56.959439+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:28:56.959439+0000","last_clean_scrub_stamp":"2024-04-25T10:28:56.959439+0000","objects_scrubbed":0,"log_size":72,"log_dups_size":0,"ondisk_log_size":72,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T17:34:34.057408+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":66,"num_read_kb":55,"num_write":101,"num_write_kb":1628,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.5","version":"0'0","reported_seq":30,"reported_epoch":44,"state":"active+clean","last_fresh":"2024-04-25T10:30:29.882532+0000","last_change":"2024-04-25T10:30:11.759459+0000","last_active":"2024-04-25T10:30:29.882532+0000","last_peered":"2024-04-25T10:30:29.882532+0000","last_clean":"2024-04-25T10:30:29.882532+0000","last_became_active":"2024-04-25T10:30:11.757880+0000","last_became_peered":"2024-04-25T10:30:11.757880+0000","last_unstale":"2024-04-25T10:30:29.882532+0000","last_undegraded":"2024-04-25T10:30:29.882532+0000","last_fullsized":"2024-04-25T10:30:29.882532+0000","mapping_epoch":33,"log_start":"0'0","ondisk_log_start":"0'0","created":33,"last_epoch_clean":34,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:30:10.729165+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:30:10.729165+0000","last_clean_scrub_stamp":"2024-04-25T10:30:10.729165+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T18:21:31.173712+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.4","version":"36'9","reported_seq":33,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-04-25T10:30:28.874920+0000","last_change":"2024-04-25T10:30:12.773161+0000","last_active":"2024-04-25T10:30:28.874920+0000","last_peered":"2024-04-25T10:30:28.874920+0000","last_clean":"2024-04-25T10:30:28.874920+0000","last_became_active":"2024-04-25T10:30:12.772831+0000","last_became_peered":"2024-04-25T10:30:12.772831+0000","last_unstale":"2024-04-25T10:30:28.874920+0000","last_undegraded":"2024-04-25T10:30:28.874920+0000","last_fullsized":"2024-04-25T10:30:28.874920+0000","mapping_epoch":34,"log_start":"0'0","ondisk_log_start":"0'0","created":34,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:30:11.733978+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:30:11.733978+0000","last_clean_scrub_stamp":"2024-04-25T10:30:11.733978+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T19:56:57.765496+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"6.7","version":"0'0","reported_seq":12,"reported_epoch":44,"state":"active+clean","last_fresh":"2024-04-25T10:30:29.882905+0000","last_change":"2024-04-25T10:30:28.881711+0000","last_active":"2024-04-25T10:30:29.882905+0000","last_peered":"2024-04-25T10:30:29.882905+0000","last_clean":"2024-04-25T10:30:29.882905+0000","last_became_active":"2024-04-25T10:30:28.881407+0000","last_became_peered":"2024-04-25T10:30:28.881407+0000","last_unstale":"2024-04-25T10:30:29.882905+0000","last_undegraded":"2024-04-25T10:30:29.882905+0000","last_fullsized":"2024-04-25T10:30:29.882905+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":42,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:30:27.860024+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:30:27.860024+0000","last_clean_scrub_stamp":"2024-04-25T10:30:27.860024+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T16:30:29.338622+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.0","version":"0'0","reported_seq":25,"reported_epoch":42,"state":"active+clean","last_fresh":"2024-04-25T10:30:27.875763+0000","last_change":"2024-04-25T10:30:11.761764+0000","last_active":"2024-04-25T10:30:27.875763+0000","last_peered":"2024-04-25T10:30:27.875763+0000","last_clean":"2024-04-25T10:30:27.875763+0000","last_became_active":"2024-04-25T10:30:11.761281+0000","last_became_peered":"2024-04-25T10:30:11.761281+0000","last_unstale":"2024-04-25T10:30:27.875763+0000","last_undegraded":"2024-04-25T10:30:27.875763+0000","last_fullsized":"2024-04-25T10:30:27.875763+0000","mapping_epoch":33,"log_start":"0'0","ondisk_log_start":"0'0","created":33,"last_epoch_clean":34,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:30:10.729165+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:30:10.729165+0000","last_clean_scrub_stamp":"2024-04-25T10:30:10.729165+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T13:05:33.012421+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.1","version":"36'5","reported_seq":34,"reported_epoch":44,"state":"active+clean","last_fresh":"2024-04-25T10:30:29.881926+0000","last_change":"2024-04-25T10:30:12.769006+0000","last_active":"2024-04-25T10:30:29.881926+0000","last_peered":"2024-04-25T10:30:29.881926+0000","last_clean":"2024-04-25T10:30:29.881926+0000","last_became_active":"2024-04-25T10:30:12.768671+0000","last_became_peered":"2024-04-25T10:30:12.768671+0000","last_unstale":"2024-04-25T10:30:29.881926+0000","last_undegraded":"2024-04-25T10:30:29.881926+0000","last_fullsized":"2024-04-25T10:30:29.881926+0000","mapping_epoch":34,"log_start":"0'0","ondisk_log_start":"0'0","created":34,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:30:11.733978+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:30:11.733978+0000","last_clean_scrub_stamp":"2024-04-25T10:30:11.733978+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T15:00:26.818072+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"6.2","version":"0'0","reported_seq":12,"reported_epoch":44,"state":"active+clean","last_fresh":"2024-04-25T10:30:29.882577+0000","last_change":"2024-04-25T10:30:28.875554+0000","last_active":"2024-04-25T10:30:29.882577+0000","last_peered":"2024-04-25T10:30:29.882577+0000","last_clean":"2024-04-25T10:30:29.882577+0000","last_became_active":"2024-04-25T10:30:28.875249+0000","last_became_peered":"2024-04-25T10:30:28.875249+0000","last_unstale":"2024-04-25T10:30:29.882577+0000","last_undegraded":"2024-04-25T10:30:29.882577+0000","last_fullsized":"2024-04-25T10:30:29.882577+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":42,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:30:27.860024+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:30:27.860024+0000","last_clean_scrub_stamp":"2024-04-25T10:30:27.860024+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T17:20:57.234773+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.1","version":"0'0","reported_seq":30,"reported_epoch":44,"state":"active+clean","last_fresh":"2024-04-25T10:30:29.882995+0000","last_change":"2024-04-25T10:30:11.765156+0000","last_active":"2024-04-25T10:30:29.882995+0000","last_peered":"2024-04-25T10:30:29.882995+0000","last_clean":"2024-04-25T10:30:29.882995+0000","last_became_active":"2024-04-25T10:30:11.764704+0000","last_became_peered":"2024-04-25T10:30:11.764704+0000","last_unstale":"2024-04-25T10:30:29.882995+0000","last_undegraded":"2024-04-25T10:30:29.882995+0000","last_fullsized":"2024-04-25T10:30:29.882995+0000","mapping_epoch":33,"log_start":"0'0","ondisk_log_start":"0'0","created":33,"last_epoch_clean":34,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:30:10.729165+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:30:10.729165+0000","last_clean_scrub_stamp":"2024-04-25T10:30:10.729165+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T19:02:53.584992+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"5.0","version":"36'8","reported_seq":31,"reported_epoch":42,"state":"active+clean","last_fresh":"2024-04-25T10:30:27.875633+0000","last_change":"2024-04-25T10:30:12.781642+0000","last_active":"2024-04-25T10:30:27.875633+0000","last_peered":"2024-04-25T10:30:27.875633+0000","last_clean":"2024-04-25T10:30:27.875633+0000","last_became_active":"2024-04-25T10:30:12.781303+0000","last_became_peered":"2024-04-25T10:30:12.781303+0000","last_unstale":"2024-04-25T10:30:27.875633+0000","last_undegraded":"2024-04-25T10:30:27.875633+0000","last_fullsized":"2024-04-25T10:30:27.875633+0000","mapping_epoch":34,"log_start":"0'0","ondisk_log_start":"0'0","created":34,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:30:11.733978+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:30:11.733978+0000","last_clean_scrub_stamp":"2024-04-25T10:30:11.733978+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T12:21:06.638842+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"6.3","version":"0'0","reported_seq":9,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-04-25T10:30:28.885536+0000","last_change":"2024-04-25T10:30:28.883569+0000","last_active":"2024-04-25T10:30:28.885536+0000","last_peered":"2024-04-25T10:30:28.885536+0000","last_clean":"2024-04-25T10:30:28.885536+0000","last_became_active":"2024-04-25T10:30:28.883246+0000","last_became_peered":"2024-04-25T10:30:28.883246+0000","last_unstale":"2024-04-25T10:30:28.885536+0000","last_undegraded":"2024-04-25T10:30:28.885536+0000","last_fullsized":"2024-04-25T10:30:28.885536+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":42,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:30:27.860024+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:30:27.860024+0000","last_clean_scrub_stamp":"2024-04-25T10:30:27.860024+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T14:04:58.646968+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.2","version":"0'0","reported_seq":28,"reported_epoch":44,"state":"active+clean","last_fresh":"2024-04-25T10:30:29.882904+0000","last_change":"2024-04-25T10:30:11.760769+0000","last_active":"2024-04-25T10:30:29.882904+0000","last_peered":"2024-04-25T10:30:29.882904+0000","last_clean":"2024-04-25T10:30:29.882904+0000","last_became_active":"2024-04-25T10:30:11.760425+0000","last_became_peered":"2024-04-25T10:30:11.760425+0000","last_unstale":"2024-04-25T10:30:29.882904+0000","last_undegraded":"2024-04-25T10:30:29.882904+0000","last_fullsized":"2024-04-25T10:30:29.882904+0000","mapping_epoch":33,"log_start":"0'0","ondisk_log_start":"0'0","created":33,"last_epoch_clean":34,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:30:10.729165+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:30:10.729165+0000","last_clean_scrub_stamp":"2024-04-25T10:30:10.729165+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T17:24:13.848319+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"5.3","version":"36'7","reported_seq":30,"reported_epoch":42,"state":"active+clean","last_fresh":"2024-04-25T10:30:27.876884+0000","last_change":"2024-04-25T10:30:12.775959+0000","last_active":"2024-04-25T10:30:27.876884+0000","last_peered":"2024-04-25T10:30:27.876884+0000","last_clean":"2024-04-25T10:30:27.876884+0000","last_became_active":"2024-04-25T10:30:12.774532+0000","last_became_peered":"2024-04-25T10:30:12.774532+0000","last_unstale":"2024-04-25T10:30:27.876884+0000","last_undegraded":"2024-04-25T10:30:27.876884+0000","last_fullsized":"2024-04-25T10:30:27.876884+0000","mapping_epoch":34,"log_start":"0'0","ondisk_log_start":"0'0","created":34,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:30:11.733978+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:30:11.733978+0000","last_clean_scrub_stamp":"2024-04-25T10:30:11.733978+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T11:38:09.717996+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"6.0","version":"0'0","reported_seq":9,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-04-25T10:30:28.884563+0000","last_change":"2024-04-25T10:30:28.883801+0000","last_active":"2024-04-25T10:30:28.884563+0000","last_peered":"2024-04-25T10:30:28.884563+0000","last_clean":"2024-04-25T10:30:28.884563+0000","last_became_active":"2024-04-25T10:30:28.883485+0000","last_became_peered":"2024-04-25T10:30:28.883485+0000","last_unstale":"2024-04-25T10:30:28.884563+0000","last_undegraded":"2024-04-25T10:30:28.884563+0000","last_fullsized":"2024-04-25T10:30:28.884563+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":42,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:30:27.860024+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:30:27.860024+0000","last_clean_scrub_stamp":"2024-04-25T10:30:27.860024+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T14:16:58.845516+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.3","version":"0'0","reported_seq":28,"reported_epoch":44,"state":"active+clean","last_fresh":"2024-04-25T10:30:29.883455+0000","last_change":"2024-04-25T10:30:11.762286+0000","last_active":"2024-04-25T10:30:29.883455+0000","last_peered":"2024-04-25T10:30:29.883455+0000","last_clean":"2024-04-25T10:30:29.883455+0000","last_became_active":"2024-04-25T10:30:11.761975+0000","last_became_peered":"2024-04-25T10:30:11.761975+0000","last_unstale":"2024-04-25T10:30:29.883455+0000","last_undegraded":"2024-04-25T10:30:29.883455+0000","last_fullsized":"2024-04-25T10:30:29.883455+0000","mapping_epoch":33,"log_start":"0'0","ondisk_log_start":"0'0","created":33,"last_epoch_clean":34,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:30:10.729165+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:30:10.729165+0000","last_clean_scrub_stamp":"2024-04-25T10:30:10.729165+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T16:09:04.081750+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"5.2","version":"36'8","reported_seq":32,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-04-25T10:30:28.877071+0000","last_change":"2024-04-25T10:30:12.765903+0000","last_active":"2024-04-25T10:30:28.877071+0000","last_peered":"2024-04-25T10:30:28.877071+0000","last_clean":"2024-04-25T10:30:28.877071+0000","last_became_active":"2024-04-25T10:30:12.765611+0000","last_became_peered":"2024-04-25T10:30:12.765611+0000","last_unstale":"2024-04-25T10:30:28.877071+0000","last_undegraded":"2024-04-25T10:30:28.877071+0000","last_fullsized":"2024-04-25T10:30:28.877071+0000","mapping_epoch":34,"log_start":"0'0","ondisk_log_start":"0'0","created":34,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:30:11.733978+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:30:11.733978+0000","last_clean_scrub_stamp":"2024-04-25T10:30:11.733978+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T22:09:01.889527+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"6.1","version":"0'0","reported_seq":12,"reported_epoch":44,"state":"active+clean","last_fresh":"2024-04-25T10:30:29.882707+0000","last_change":"2024-04-25T10:30:28.884434+0000","last_active":"2024-04-25T10:30:29.882707+0000","last_peered":"2024-04-25T10:30:29.882707+0000","last_clean":"2024-04-25T10:30:29.882707+0000","last_became_active":"2024-04-25T10:30:28.883801+0000","last_became_peered":"2024-04-25T10:30:28.883801+0000","last_unstale":"2024-04-25T10:30:29.882707+0000","last_undegraded":"2024-04-25T10:30:29.882707+0000","last_fullsized":"2024-04-25T10:30:29.882707+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":42,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:30:27.860024+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:30:27.860024+0000","last_clean_scrub_stamp":"2024-04-25T10:30:27.860024+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T11:57:46.105302+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.4","version":"0'0","reported_seq":28,"reported_epoch":44,"state":"active+clean","last_fresh":"2024-04-25T10:30:29.883736+0000","last_change":"2024-04-25T10:30:11.757601+0000","last_active":"2024-04-25T10:30:29.883736+0000","last_peered":"2024-04-25T10:30:29.883736+0000","last_clean":"2024-04-25T10:30:29.883736+0000","last_became_active":"2024-04-25T10:30:11.756626+0000","last_became_peered":"2024-04-25T10:30:11.756626+0000","last_unstale":"2024-04-25T10:30:29.883736+0000","last_undegraded":"2024-04-25T10:30:29.883736+0000","last_fullsized":"2024-04-25T10:30:29.883736+0000","mapping_epoch":33,"log_start":"0'0","ondisk_log_start":"0'0","created":33,"last_epoch_clean":34,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:30:10.729165+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:30:10.729165+0000","last_clean_scrub_stamp":"2024-04-25T10:30:10.729165+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T13:03:08.676049+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"5.5","version":"36'5","reported_seq":29,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-04-25T10:30:28.875752+0000","last_change":"2024-04-25T10:30:12.782521+0000","last_active":"2024-04-25T10:30:28.875752+0000","last_peered":"2024-04-25T10:30:28.875752+0000","last_clean":"2024-04-25T10:30:28.875752+0000","last_became_active":"2024-04-25T10:30:12.781802+0000","last_became_peered":"2024-04-25T10:30:12.781802+0000","last_unstale":"2024-04-25T10:30:28.875752+0000","last_undegraded":"2024-04-25T10:30:28.875752+0000","last_fullsized":"2024-04-25T10:30:28.875752+0000","mapping_epoch":34,"log_start":"0'0","ondisk_log_start":"0'0","created":34,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:30:11.733978+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:30:11.733978+0000","last_clean_scrub_stamp":"2024-04-25T10:30:11.733978+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T15:17:30.002665+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"6.6","version":"0'0","reported_seq":9,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-04-25T10:30:28.886860+0000","last_change":"2024-04-25T10:30:28.885896+0000","last_active":"2024-04-25T10:30:28.886860+0000","last_peered":"2024-04-25T10:30:28.886860+0000","last_clean":"2024-04-25T10:30:28.886860+0000","last_became_active":"2024-04-25T10:30:28.885575+0000","last_became_peered":"2024-04-25T10:30:28.885575+0000","last_unstale":"2024-04-25T10:30:28.886860+0000","last_undegraded":"2024-04-25T10:30:28.886860+0000","last_fullsized":"2024-04-25T10:30:28.886860+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":42,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:30:27.860024+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:30:27.860024+0000","last_clean_scrub_stamp":"2024-04-25T10:30:27.860024+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T22:09:18.015574+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.6","version":"37'4","reported_seq":29,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-04-25T10:30:28.879222+0000","last_change":"2024-04-25T10:30:11.750312+0000","last_active":"2024-04-25T10:30:28.879222+0000","last_peered":"2024-04-25T10:30:28.879222+0000","last_clean":"2024-04-25T10:30:28.879222+0000","last_became_active":"2024-04-25T10:30:11.750024+0000","last_became_peered":"2024-04-25T10:30:11.750024+0000","last_unstale":"2024-04-25T10:30:28.879222+0000","last_undegraded":"2024-04-25T10:30:28.879222+0000","last_fullsized":"2024-04-25T10:30:28.879222+0000","mapping_epoch":33,"log_start":"0'0","ondisk_log_start":"0'0","created":33,"last_epoch_clean":34,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:30:10.729165+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:30:10.729165+0000","last_clean_scrub_stamp":"2024-04-25T10:30:10.729165+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T16:48:59.263610+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"5.7","version":"36'8","reported_seq":37,"reported_epoch":44,"state":"active+clean","last_fresh":"2024-04-25T10:30:29.882715+0000","last_change":"2024-04-25T10:30:12.768932+0000","last_active":"2024-04-25T10:30:29.882715+0000","last_peered":"2024-04-25T10:30:29.882715+0000","last_clean":"2024-04-25T10:30:29.882715+0000","last_became_active":"2024-04-25T10:30:12.768609+0000","last_became_peered":"2024-04-25T10:30:12.768609+0000","last_unstale":"2024-04-25T10:30:29.882715+0000","last_undegraded":"2024-04-25T10:30:29.882715+0000","last_fullsized":"2024-04-25T10:30:29.882715+0000","mapping_epoch":34,"log_start":"0'0","ondisk_log_start":"0'0","created":34,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:30:11.733978+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:30:11.733978+0000","last_clean_scrub_stamp":"2024-04-25T10:30:11.733978+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T11:24:05.006967+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":4194304,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":4096,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"6.4","version":"0'0","reported_seq":9,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-04-25T10:30:28.881060+0000","last_change":"2024-04-25T10:30:28.880128+0000","last_active":"2024-04-25T10:30:28.881060+0000","last_peered":"2024-04-25T10:30:28.881060+0000","last_clean":"2024-04-25T10:30:28.881060+0000","last_became_active":"2024-04-25T10:30:28.879771+0000","last_became_peered":"2024-04-25T10:30:28.879771+0000","last_unstale":"2024-04-25T10:30:28.881060+0000","last_undegraded":"2024-04-25T10:30:28.881060+0000","last_fullsized":"2024-04-25T10:30:28.881060+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":42,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:30:27.860024+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:30:27.860024+0000","last_clean_scrub_stamp":"2024-04-25T10:30:27.860024+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T11:33:01.743970+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.7","version":"37'3","reported_seq":31,"reported_epoch":44,"state":"active+clean","last_fresh":"2024-04-25T10:30:29.883302+0000","last_change":"2024-04-25T10:30:11.756948+0000","last_active":"2024-04-25T10:30:29.883302+0000","last_peered":"2024-04-25T10:30:29.883302+0000","last_clean":"2024-04-25T10:30:29.883302+0000","last_became_active":"2024-04-25T10:30:11.756607+0000","last_became_peered":"2024-04-25T10:30:11.756607+0000","last_unstale":"2024-04-25T10:30:29.883302+0000","last_undegraded":"2024-04-25T10:30:29.883302+0000","last_fullsized":"2024-04-25T10:30:29.883302+0000","mapping_epoch":33,"log_start":"0'0","ondisk_log_start":"0'0","created":33,"last_epoch_clean":34,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:30:10.729165+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:30:10.729165+0000","last_clean_scrub_stamp":"2024-04-25T10:30:10.729165+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T17:15:13.597075+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"5.6","version":"36'7","reported_seq":31,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-04-25T10:30:28.876259+0000","last_change":"2024-04-25T10:30:12.772456+0000","last_active":"2024-04-25T10:30:28.876259+0000","last_peered":"2024-04-25T10:30:28.876259+0000","last_clean":"2024-04-25T10:30:28.876259+0000","last_became_active":"2024-04-25T10:30:12.772070+0000","last_became_peered":"2024-04-25T10:30:12.772070+0000","last_unstale":"2024-04-25T10:30:28.876259+0000","last_undegraded":"2024-04-25T10:30:28.876259+0000","last_fullsized":"2024-04-25T10:30:28.876259+0000","mapping_epoch":34,"log_start":"0'0","ondisk_log_start":"0'0","created":34,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:30:11.733978+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:30:11.733978+0000","last_clean_scrub_stamp":"2024-04-25T10:30:11.733978+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T16:08:29.987819+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"6.5","version":"0'0","reported_seq":9,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-04-25T10:30:28.882547+0000","last_change":"2024-04-25T10:30:28.881260+0000","last_active":"2024-04-25T10:30:28.882547+0000","last_peered":"2024-04-25T10:30:28.882547+0000","last_clean":"2024-04-25T10:30:28.882547+0000","last_became_active":"2024-04-25T10:30:28.880995+0000","last_became_peered":"2024-04-25T10:30:28.880995+0000","last_unstale":"2024-04-25T10:30:28.882547+0000","last_undegraded":"2024-04-25T10:30:28.882547+0000","last_fullsized":"2024-04-25T10:30:28.882547+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":42,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:30:27.860024+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:30:27.860024+0000","last_clean_scrub_stamp":"2024-04-25T10:30:27.860024+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T15:39:48.016111+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.8","version":"0'0","reported_seq":30,"reported_epoch":44,"state":"active+clean","last_fresh":"2024-04-25T10:30:29.881839+0000","last_change":"2024-04-25T10:30:11.758581+0000","last_active":"2024-04-25T10:30:29.881839+0000","last_peered":"2024-04-25T10:30:29.881839+0000","last_clean":"2024-04-25T10:30:29.881839+0000","last_became_active":"2024-04-25T10:30:11.758239+0000","last_became_peered":"2024-04-25T10:30:11.758239+0000","last_unstale":"2024-04-25T10:30:29.881839+0000","last_undegraded":"2024-04-25T10:30:29.881839+0000","last_fullsized":"2024-04-25T10:30:29.881839+0000","mapping_epoch":33,"log_start":"0'0","ondisk_log_start":"0'0","created":33,"last_epoch_clean":34,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:30:10.729165+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:30:10.729165+0000","last_clean_scrub_stamp":"2024-04-25T10:30:10.729165+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T17:41:33.905382+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.9","version":"36'2","reported_seq":27,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-04-25T10:30:28.874577+0000","last_change":"2024-04-25T10:30:12.768782+0000","last_active":"2024-04-25T10:30:28.874577+0000","last_peered":"2024-04-25T10:30:28.874577+0000","last_clean":"2024-04-25T10:30:28.874577+0000","last_became_active":"2024-04-25T10:30:12.768464+0000","last_became_peered":"2024-04-25T10:30:12.768464+0000","last_unstale":"2024-04-25T10:30:28.874577+0000","last_undegraded":"2024-04-25T10:30:28.874577+0000","last_fullsized":"2024-04-25T10:30:28.874577+0000","mapping_epoch":34,"log_start":"0'0","ondisk_log_start":"0'0","created":34,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:30:11.733978+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:30:11.733978+0000","last_clean_scrub_stamp":"2024-04-25T10:30:11.733978+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T13:03:21.170197+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"6.a","version":"0'0","reported_seq":9,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-04-25T10:30:28.884130+0000","last_change":"2024-04-25T10:30:28.882147+0000","last_active":"2024-04-25T10:30:28.884130+0000","last_peered":"2024-04-25T10:30:28.884130+0000","last_clean":"2024-04-25T10:30:28.884130+0000","last_became_active":"2024-04-25T10:30:28.881783+0000","last_became_peered":"2024-04-25T10:30:28.881783+0000","last_unstale":"2024-04-25T10:30:28.884130+0000","last_undegraded":"2024-04-25T10:30:28.884130+0000","last_fullsized":"2024-04-25T10:30:28.884130+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":42,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:30:27.860024+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:30:27.860024+0000","last_clean_scrub_stamp":"2024-04-25T10:30:27.860024+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T18:36:07.211600+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.9","version":"0'0","reported_seq":26,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-04-25T10:30:28.874568+0000","last_change":"2024-04-25T10:30:11.759807+0000","last_active":"2024-04-25T10:30:28.874568+0000","last_peered":"2024-04-25T10:30:28.874568+0000","last_clean":"2024-04-25T10:30:28.874568+0000","last_became_active":"2024-04-25T10:30:11.758558+0000","last_became_peered":"2024-04-25T10:30:11.758558+0000","last_unstale":"2024-04-25T10:30:28.874568+0000","last_undegraded":"2024-04-25T10:30:28.874568+0000","last_fullsized":"2024-04-25T10:30:28.874568+0000","mapping_epoch":33,"log_start":"0'0","ondisk_log_start":"0'0","created":33,"last_epoch_clean":34,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:30:10.729165+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:30:10.729165+0000","last_clean_scrub_stamp":"2024-04-25T10:30:10.729165+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T20:38:23.460836+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"5.8","version":"36'9","reported_seq":36,"reported_epoch":44,"state":"active+clean","last_fresh":"2024-04-25T10:30:29.882627+0000","last_change":"2024-04-25T10:30:12.777060+0000","last_active":"2024-04-25T10:30:29.882627+0000","last_peered":"2024-04-25T10:30:29.882627+0000","last_clean":"2024-04-25T10:30:29.882627+0000","last_became_active":"2024-04-25T10:30:12.776749+0000","last_became_peered":"2024-04-25T10:30:12.776749+0000","last_unstale":"2024-04-25T10:30:29.882627+0000","last_undegraded":"2024-04-25T10:30:29.882627+0000","last_fullsized":"2024-04-25T10:30:29.882627+0000","mapping_epoch":34,"log_start":"0'0","ondisk_log_start":"0'0","created":34,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:30:11.733978+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:30:11.733978+0000","last_clean_scrub_stamp":"2024-04-25T10:30:11.733978+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T15:47:40.336983+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"6.b","version":"0'0","reported_seq":1,"reported_epoch":42,"state":"creating+peering","last_fresh":"2024-04-25T10:30:27.876378+0000","last_change":"2024-04-25T10:30:27.876378+0000","last_active":"2024-04-25T10:30:27.860024+0000","last_peered":"2024-04-25T10:30:27.860024+0000","last_clean":"2024-04-25T10:30:27.860024+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T10:30:27.876378+0000","last_undegraded":"2024-04-25T10:30:27.876378+0000","last_fullsized":"2024-04-25T10:30:27.876378+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":42,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:30:27.860024+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:30:27.860024+0000","last_clean_scrub_stamp":"2024-04-25T10:30:27.860024+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.a","version":"0'0","reported_seq":30,"reported_epoch":44,"state":"active+clean","last_fresh":"2024-04-25T10:30:29.883112+0000","last_change":"2024-04-25T10:30:11.762745+0000","last_active":"2024-04-25T10:30:29.883112+0000","last_peered":"2024-04-25T10:30:29.883112+0000","last_clean":"2024-04-25T10:30:29.883112+0000","last_became_active":"2024-04-25T10:30:11.762377+0000","last_became_peered":"2024-04-25T10:30:11.762377+0000","last_unstale":"2024-04-25T10:30:29.883112+0000","last_undegraded":"2024-04-25T10:30:29.883112+0000","last_fullsized":"2024-04-25T10:30:29.883112+0000","mapping_epoch":33,"log_start":"0'0","ondisk_log_start":"0'0","created":33,"last_epoch_clean":34,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:30:10.729165+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:30:10.729165+0000","last_clean_scrub_stamp":"2024-04-25T10:30:10.729165+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T17:15:13.597075+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"5.b","version":"36'8","reported_seq":32,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-04-25T10:30:28.875847+0000","last_change":"2024-04-25T10:30:12.766544+0000","last_active":"2024-04-25T10:30:28.875847+0000","last_peered":"2024-04-25T10:30:28.875847+0000","last_clean":"2024-04-25T10:30:28.875847+0000","last_became_active":"2024-04-25T10:30:12.766228+0000","last_became_peered":"2024-04-25T10:30:12.766228+0000","last_unstale":"2024-04-25T10:30:28.875847+0000","last_undegraded":"2024-04-25T10:30:28.875847+0000","last_fullsized":"2024-04-25T10:30:28.875847+0000","mapping_epoch":34,"log_start":"0'0","ondisk_log_start":"0'0","created":34,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:30:11.733978+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:30:11.733978+0000","last_clean_scrub_stamp":"2024-04-25T10:30:11.733978+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T18:00:26.511360+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"6.8","version":"0'0","reported_seq":12,"reported_epoch":44,"state":"active+clean","last_fresh":"2024-04-25T10:30:29.881772+0000","last_change":"2024-04-25T10:30:28.881118+0000","last_active":"2024-04-25T10:30:29.881772+0000","last_peered":"2024-04-25T10:30:29.881772+0000","last_clean":"2024-04-25T10:30:29.881772+0000","last_became_active":"2024-04-25T10:30:28.880810+0000","last_became_peered":"2024-04-25T10:30:28.880810+0000","last_unstale":"2024-04-25T10:30:29.881772+0000","last_undegraded":"2024-04-25T10:30:29.881772+0000","last_fullsized":"2024-04-25T10:30:29.881772+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":42,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:30:27.860024+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:30:27.860024+0000","last_clean_scrub_stamp":"2024-04-25T10:30:27.860024+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T15:53:38.989476+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.b","version":"37'4","reported_seq":29,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-04-25T10:30:28.875996+0000","last_change":"2024-04-25T10:30:11.752902+0000","last_active":"2024-04-25T10:30:28.875996+0000","last_peered":"2024-04-25T10:30:28.875996+0000","last_clean":"2024-04-25T10:30:28.875996+0000","last_became_active":"2024-04-25T10:30:11.752599+0000","last_became_peered":"2024-04-25T10:30:11.752599+0000","last_unstale":"2024-04-25T10:30:28.875996+0000","last_undegraded":"2024-04-25T10:30:28.875996+0000","last_fullsized":"2024-04-25T10:30:28.875996+0000","mapping_epoch":33,"log_start":"0'0","ondisk_log_start":"0'0","created":33,"last_epoch_clean":34,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:30:10.729165+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:30:10.729165+0000","last_clean_scrub_stamp":"2024-04-25T10:30:10.729165+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T14:01:05.786983+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"5.a","version":"36'5","reported_seq":32,"reported_epoch":44,"state":"active+clean","last_fresh":"2024-04-25T10:30:29.882793+0000","last_change":"2024-04-25T10:30:12.762940+0000","last_active":"2024-04-25T10:30:29.882793+0000","last_peered":"2024-04-25T10:30:29.882793+0000","last_clean":"2024-04-25T10:30:29.882793+0000","last_became_active":"2024-04-25T10:30:12.762628+0000","last_became_peered":"2024-04-25T10:30:12.762628+0000","last_unstale":"2024-04-25T10:30:29.882793+0000","last_undegraded":"2024-04-25T10:30:29.882793+0000","last_fullsized":"2024-04-25T10:30:29.882793+0000","mapping_epoch":34,"log_start":"0'0","ondisk_log_start":"0'0","created":34,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:30:11.733978+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:30:11.733978+0000","last_clean_scrub_stamp":"2024-04-25T10:30:11.733978+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T19:25:06.397830+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"6.9","version":"0'0","reported_seq":1,"reported_epoch":42,"state":"creating+peering","last_fresh":"2024-04-25T10:30:27.871775+0000","last_change":"2024-04-25T10:30:27.871775+0000","last_active":"2024-04-25T10:30:27.860024+0000","last_peered":"2024-04-25T10:30:27.860024+0000","last_clean":"2024-04-25T10:30:27.860024+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T10:30:27.871775+0000","last_undegraded":"2024-04-25T10:30:27.871775+0000","last_fullsized":"2024-04-25T10:30:27.871775+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":42,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:30:27.860024+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:30:27.860024+0000","last_clean_scrub_stamp":"2024-04-25T10:30:27.860024+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.c","version":"0'0","reported_seq":25,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-04-25T10:30:28.875034+0000","last_change":"2024-04-25T10:30:11.753067+0000","last_active":"2024-04-25T10:30:28.875034+0000","last_peered":"2024-04-25T10:30:28.875034+0000","last_clean":"2024-04-25T10:30:28.875034+0000","last_became_active":"2024-04-25T10:30:11.752595+0000","last_became_peered":"2024-04-25T10:30:11.752595+0000","last_unstale":"2024-04-25T10:30:28.875034+0000","last_undegraded":"2024-04-25T10:30:28.875034+0000","last_fullsized":"2024-04-25T10:30:28.875034+0000","mapping_epoch":33,"log_start":"0'0","ondisk_log_start":"0'0","created":33,"last_epoch_clean":34,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:30:10.729165+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:30:10.729165+0000","last_clean_scrub_stamp":"2024-04-25T10:30:10.729165+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T18:59:19.688948+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"5.d","version":"36'7","reported_seq":31,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-04-25T10:30:28.875563+0000","last_change":"2024-04-25T10:30:12.771743+0000","last_active":"2024-04-25T10:30:28.875563+0000","last_peered":"2024-04-25T10:30:28.875563+0000","last_clean":"2024-04-25T10:30:28.875563+0000","last_became_active":"2024-04-25T10:30:12.771387+0000","last_became_peered":"2024-04-25T10:30:12.771387+0000","last_unstale":"2024-04-25T10:30:28.875563+0000","last_undegraded":"2024-04-25T10:30:28.875563+0000","last_fullsized":"2024-04-25T10:30:28.875563+0000","mapping_epoch":34,"log_start":"0'0","ondisk_log_start":"0'0","created":34,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:30:11.733978+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:30:11.733978+0000","last_clean_scrub_stamp":"2024-04-25T10:30:11.733978+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T14:01:06.791796+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"6.e","version":"0'0","reported_seq":9,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-04-25T10:30:28.885894+0000","last_change":"2024-04-25T10:30:28.884467+0000","last_active":"2024-04-25T10:30:28.885894+0000","last_peered":"2024-04-25T10:30:28.885894+0000","last_clean":"2024-04-25T10:30:28.885894+0000","last_became_active":"2024-04-25T10:30:28.884173+0000","last_became_peered":"2024-04-25T10:30:28.884173+0000","last_unstale":"2024-04-25T10:30:28.885894+0000","last_undegraded":"2024-04-25T10:30:28.885894+0000","last_fullsized":"2024-04-25T10:30:28.885894+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":42,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:30:27.860024+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:30:27.860024+0000","last_clean_scrub_stamp":"2024-04-25T10:30:27.860024+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T20:14:31.367609+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.d","version":"37'2","reported_seq":32,"reported_epoch":44,"state":"active+clean","last_fresh":"2024-04-25T10:30:29.882419+0000","last_change":"2024-04-25T10:30:11.758390+0000","last_active":"2024-04-25T10:30:29.882419+0000","last_peered":"2024-04-25T10:30:29.882419+0000","last_clean":"2024-04-25T10:30:29.882419+0000","last_became_active":"2024-04-25T10:30:11.757989+0000","last_became_peered":"2024-04-25T10:30:11.757989+0000","last_unstale":"2024-04-25T10:30:29.882419+0000","last_undegraded":"2024-04-25T10:30:29.882419+0000","last_fullsized":"2024-04-25T10:30:29.882419+0000","mapping_epoch":33,"log_start":"0'0","ondisk_log_start":"0'0","created":33,"last_epoch_clean":34,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:30:10.729165+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:30:10.729165+0000","last_clean_scrub_stamp":"2024-04-25T10:30:10.729165+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T18:52:44.534946+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.c","version":"36'6","reported_seq":30,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-04-25T10:30:28.876757+0000","last_change":"2024-04-25T10:30:12.765790+0000","last_active":"2024-04-25T10:30:28.876757+0000","last_peered":"2024-04-25T10:30:28.876757+0000","last_clean":"2024-04-25T10:30:28.876757+0000","last_became_active":"2024-04-25T10:30:12.765468+0000","last_became_peered":"2024-04-25T10:30:12.765468+0000","last_unstale":"2024-04-25T10:30:28.876757+0000","last_undegraded":"2024-04-25T10:30:28.876757+0000","last_fullsized":"2024-04-25T10:30:28.876757+0000","mapping_epoch":34,"log_start":"0'0","ondisk_log_start":"0'0","created":34,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:30:11.733978+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:30:11.733978+0000","last_clean_scrub_stamp":"2024-04-25T10:30:11.733978+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T16:29:40.621322+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"6.f","version":"0'0","reported_seq":9,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-04-25T10:30:28.887668+0000","last_change":"2024-04-25T10:30:28.886506+0000","last_active":"2024-04-25T10:30:28.887668+0000","last_peered":"2024-04-25T10:30:28.887668+0000","last_clean":"2024-04-25T10:30:28.887668+0000","last_became_active":"2024-04-25T10:30:28.886157+0000","last_became_peered":"2024-04-25T10:30:28.886157+0000","last_unstale":"2024-04-25T10:30:28.887668+0000","last_undegraded":"2024-04-25T10:30:28.887668+0000","last_fullsized":"2024-04-25T10:30:28.887668+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":42,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:30:27.860024+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:30:27.860024+0000","last_clean_scrub_stamp":"2024-04-25T10:30:27.860024+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T11:57:46.105302+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.e","version":"37'3","reported_seq":33,"reported_epoch":44,"state":"active+clean","last_fresh":"2024-04-25T10:30:29.883403+0000","last_change":"2024-04-25T10:30:11.765106+0000","last_active":"2024-04-25T10:30:29.883403+0000","last_peered":"2024-04-25T10:30:29.883403+0000","last_clean":"2024-04-25T10:30:29.883403+0000","last_became_active":"2024-04-25T10:30:11.764710+0000","last_became_peered":"2024-04-25T10:30:11.764710+0000","last_unstale":"2024-04-25T10:30:29.883403+0000","last_undegraded":"2024-04-25T10:30:29.883403+0000","last_fullsized":"2024-04-25T10:30:29.883403+0000","mapping_epoch":33,"log_start":"0'0","ondisk_log_start":"0'0","created":33,"last_epoch_clean":34,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:30:10.729165+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:30:10.729165+0000","last_clean_scrub_stamp":"2024-04-25T10:30:10.729165+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T16:30:11.709416+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"5.f","version":"36'9","reported_seq":34,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-04-25T10:30:28.875269+0000","last_change":"2024-04-25T10:30:12.769270+0000","last_active":"2024-04-25T10:30:28.875269+0000","last_peered":"2024-04-25T10:30:28.875269+0000","last_clean":"2024-04-25T10:30:28.875269+0000","last_became_active":"2024-04-25T10:30:12.768931+0000","last_became_peered":"2024-04-25T10:30:12.768931+0000","last_unstale":"2024-04-25T10:30:28.875269+0000","last_undegraded":"2024-04-25T10:30:28.875269+0000","last_fullsized":"2024-04-25T10:30:28.875269+0000","mapping_epoch":34,"log_start":"0'0","ondisk_log_start":"0'0","created":34,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:30:11.733978+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:30:11.733978+0000","last_clean_scrub_stamp":"2024-04-25T10:30:11.733978+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T16:49:00.268423+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"6.c","version":"0'0","reported_seq":12,"reported_epoch":44,"state":"active+clean","last_fresh":"2024-04-25T10:30:29.883642+0000","last_change":"2024-04-25T10:30:28.887055+0000","last_active":"2024-04-25T10:30:29.883642+0000","last_peered":"2024-04-25T10:30:29.883642+0000","last_clean":"2024-04-25T10:30:29.883642+0000","last_became_active":"2024-04-25T10:30:28.886675+0000","last_became_peered":"2024-04-25T10:30:28.886675+0000","last_unstale":"2024-04-25T10:30:29.883642+0000","last_undegraded":"2024-04-25T10:30:29.883642+0000","last_fullsized":"2024-04-25T10:30:29.883642+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":42,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:30:27.860024+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:30:27.860024+0000","last_clean_scrub_stamp":"2024-04-25T10:30:27.860024+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T14:04:58.646968+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,1],"acting":[6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.f","version":"37'2","reported_seq":27,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-04-25T10:30:28.874709+0000","last_change":"2024-04-25T10:30:11.750348+0000","last_active":"2024-04-25T10:30:28.874709+0000","last_peered":"2024-04-25T10:30:28.874709+0000","last_clean":"2024-04-25T10:30:28.874709+0000","last_became_active":"2024-04-25T10:30:11.749970+0000","last_became_peered":"2024-04-25T10:30:11.749970+0000","last_unstale":"2024-04-25T10:30:28.874709+0000","last_undegraded":"2024-04-25T10:30:28.874709+0000","last_fullsized":"2024-04-25T10:30:28.874709+0000","mapping_epoch":33,"log_start":"0'0","ondisk_log_start":"0'0","created":33,"last_epoch_clean":34,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:30:10.729165+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:30:10.729165+0000","last_clean_scrub_stamp":"2024-04-25T10:30:10.729165+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T20:41:22.574408+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"5.e","version":"36'5","reported_seq":34,"reported_epoch":44,"state":"active+clean","last_fresh":"2024-04-25T10:30:29.882392+0000","last_change":"2024-04-25T10:30:12.780850+0000","last_active":"2024-04-25T10:30:29.882392+0000","last_peered":"2024-04-25T10:30:29.882392+0000","last_clean":"2024-04-25T10:30:29.882392+0000","last_became_active":"2024-04-25T10:30:12.780187+0000","last_became_peered":"2024-04-25T10:30:12.780187+0000","last_unstale":"2024-04-25T10:30:29.882392+0000","last_undegraded":"2024-04-25T10:30:29.882392+0000","last_fullsized":"2024-04-25T10:30:29.882392+0000","mapping_epoch":34,"log_start":"0'0","ondisk_log_start":"0'0","created":34,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:30:11.733978+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:30:11.733978+0000","last_clean_scrub_stamp":"2024-04-25T10:30:11.733978+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T12:49:30.093333+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"6.d","version":"0'0","reported_seq":9,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-04-25T10:30:28.880134+0000","last_change":"2024-04-25T10:30:28.878873+0000","last_active":"2024-04-25T10:30:28.880134+0000","last_peered":"2024-04-25T10:30:28.880134+0000","last_clean":"2024-04-25T10:30:28.880134+0000","last_became_active":"2024-04-25T10:30:28.878596+0000","last_became_peered":"2024-04-25T10:30:28.878596+0000","last_unstale":"2024-04-25T10:30:28.880134+0000","last_undegraded":"2024-04-25T10:30:28.880134+0000","last_fullsized":"2024-04-25T10:30:28.880134+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":42,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:30:27.860024+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:30:27.860024+0000","last_clean_scrub_stamp":"2024-04-25T10:30:27.860024+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T11:33:01.743970+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.10","version":"37'15","reported_seq":40,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-04-25T10:30:28.873989+0000","last_change":"2024-04-25T10:30:11.755358+0000","last_active":"2024-04-25T10:30:28.873989+0000","last_peered":"2024-04-25T10:30:28.873989+0000","last_clean":"2024-04-25T10:30:28.873989+0000","last_became_active":"2024-04-25T10:30:11.754992+0000","last_became_peered":"2024-04-25T10:30:11.754992+0000","last_unstale":"2024-04-25T10:30:28.873989+0000","last_undegraded":"2024-04-25T10:30:28.873989+0000","last_fullsized":"2024-04-25T10:30:28.873989+0000","mapping_epoch":33,"log_start":"0'0","ondisk_log_start":"0'0","created":33,"last_epoch_clean":34,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:30:10.729165+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:30:10.729165+0000","last_clean_scrub_stamp":"2024-04-25T10:30:10.729165+0000","objects_scrubbed":0,"log_size":15,"log_dups_size":0,"ondisk_log_size":15,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T19:57:30.397817+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":21,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3],"acting":[7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"5.11","version":"36'6","reported_seq":35,"reported_epoch":44,"state":"active+clean","last_fresh":"2024-04-25T10:30:29.881885+0000","last_change":"2024-04-25T10:30:12.777184+0000","last_active":"2024-04-25T10:30:29.881885+0000","last_peered":"2024-04-25T10:30:29.881885+0000","last_clean":"2024-04-25T10:30:29.881885+0000","last_became_active":"2024-04-25T10:30:12.776874+0000","last_became_peered":"2024-04-25T10:30:12.776874+0000","last_unstale":"2024-04-25T10:30:29.881885+0000","last_undegraded":"2024-04-25T10:30:29.881885+0000","last_fullsized":"2024-04-25T10:30:29.881885+0000","mapping_epoch":34,"log_start":"0'0","ondisk_log_start":"0'0","created":34,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:30:11.733978+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:30:11.733978+0000","last_clean_scrub_stamp":"2024-04-25T10:30:11.733978+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T16:41:33.609972+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"6.12","version":"0'0","reported_seq":9,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-04-25T10:30:28.882650+0000","last_change":"2024-04-25T10:30:28.881494+0000","last_active":"2024-04-25T10:30:28.882650+0000","last_peered":"2024-04-25T10:30:28.882650+0000","last_clean":"2024-04-25T10:30:28.882650+0000","last_became_active":"2024-04-25T10:30:28.880682+0000","last_became_peered":"2024-04-25T10:30:28.880682+0000","last_unstale":"2024-04-25T10:30:28.882650+0000","last_undegraded":"2024-04-25T10:30:28.882650+0000","last_fullsized":"2024-04-25T10:30:28.882650+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":42,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:30:27.860024+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:30:27.860024+0000","last_clean_scrub_stamp":"2024-04-25T10:30:27.860024+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T17:35:49.687185+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.11","version":"0'0","reported_seq":25,"reported_epoch":42,"state":"active+clean","last_fresh":"2024-04-25T10:30:27.876406+0000","last_change":"2024-04-25T10:30:11.755706+0000","last_active":"2024-04-25T10:30:27.876406+0000","last_peered":"2024-04-25T10:30:27.876406+0000","last_clean":"2024-04-25T10:30:27.876406+0000","last_became_active":"2024-04-25T10:30:11.755348+0000","last_became_peered":"2024-04-25T10:30:11.755348+0000","last_unstale":"2024-04-25T10:30:27.876406+0000","last_undegraded":"2024-04-25T10:30:27.876406+0000","last_fullsized":"2024-04-25T10:30:27.876406+0000","mapping_epoch":33,"log_start":"0'0","ondisk_log_start":"0'0","created":33,"last_epoch_clean":34,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:30:10.729165+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:30:10.729165+0000","last_clean_scrub_stamp":"2024-04-25T10:30:10.729165+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T10:42:20.657474+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.10","version":"36'6","reported_seq":35,"reported_epoch":44,"state":"active+clean","last_fresh":"2024-04-25T10:30:29.882755+0000","last_change":"2024-04-25T10:30:12.783285+0000","last_active":"2024-04-25T10:30:29.882755+0000","last_peered":"2024-04-25T10:30:29.882755+0000","last_clean":"2024-04-25T10:30:29.882755+0000","last_became_active":"2024-04-25T10:30:12.782617+0000","last_became_peered":"2024-04-25T10:30:12.782617+0000","last_unstale":"2024-04-25T10:30:29.882755+0000","last_undegraded":"2024-04-25T10:30:29.882755+0000","last_fullsized":"2024-04-25T10:30:29.882755+0000","mapping_epoch":34,"log_start":"0'0","ondisk_log_start":"0'0","created":34,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:30:11.733978+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:30:11.733978+0000","last_clean_scrub_stamp":"2024-04-25T10:30:11.733978+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T18:35:51.085554+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"6.13","version":"0'0","reported_seq":9,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-04-25T10:30:28.882643+0000","last_change":"2024-04-25T10:30:28.880997+0000","last_active":"2024-04-25T10:30:28.882643+0000","last_peered":"2024-04-25T10:30:28.882643+0000","last_clean":"2024-04-25T10:30:28.882643+0000","last_became_active":"2024-04-25T10:30:28.880686+0000","last_became_peered":"2024-04-25T10:30:28.880686+0000","last_unstale":"2024-04-25T10:30:28.882643+0000","last_undegraded":"2024-04-25T10:30:28.882643+0000","last_fullsized":"2024-04-25T10:30:28.882643+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":42,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:30:27.860024+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:30:27.860024+0000","last_clean_scrub_stamp":"2024-04-25T10:30:27.860024+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T20:14:31.367609+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.12","version":"0'0","reported_seq":25,"reported_epoch":42,"state":"active+clean","last_fresh":"2024-04-25T10:30:27.876694+0000","last_change":"2024-04-25T10:30:11.762094+0000","last_active":"2024-04-25T10:30:27.876694+0000","last_peered":"2024-04-25T10:30:27.876694+0000","last_clean":"2024-04-25T10:30:27.876694+0000","last_became_active":"2024-04-25T10:30:11.761456+0000","last_became_peered":"2024-04-25T10:30:11.761456+0000","last_unstale":"2024-04-25T10:30:27.876694+0000","last_undegraded":"2024-04-25T10:30:27.876694+0000","last_fullsized":"2024-04-25T10:30:27.876694+0000","mapping_epoch":33,"log_start":"0'0","ondisk_log_start":"0'0","created":33,"last_epoch_clean":34,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:30:10.729165+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:30:10.729165+0000","last_clean_scrub_stamp":"2024-04-25T10:30:10.729165+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T21:58:31.211023+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.13","version":"36'15","reported_seq":38,"reported_epoch":42,"state":"active+clean","last_fresh":"2024-04-25T10:30:27.876744+0000","last_change":"2024-04-25T10:30:12.775269+0000","last_active":"2024-04-25T10:30:27.876744+0000","last_peered":"2024-04-25T10:30:27.876744+0000","last_clean":"2024-04-25T10:30:27.876744+0000","last_became_active":"2024-04-25T10:30:12.774865+0000","last_became_peered":"2024-04-25T10:30:12.774865+0000","last_unstale":"2024-04-25T10:30:27.876744+0000","last_undegraded":"2024-04-25T10:30:27.876744+0000","last_fullsized":"2024-04-25T10:30:27.876744+0000","mapping_epoch":34,"log_start":"0'0","ondisk_log_start":"0'0","created":34,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:30:11.733978+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:30:11.733978+0000","last_clean_scrub_stamp":"2024-04-25T10:30:11.733978+0000","objects_scrubbed":0,"log_size":15,"log_dups_size":0,"ondisk_log_size":15,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T20:14:36.991455+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"6.10","version":"0'0","reported_seq":1,"reported_epoch":42,"state":"creating+peering","last_fresh":"2024-04-25T10:30:27.875123+0000","last_change":"2024-04-25T10:30:27.875123+0000","last_active":"2024-04-25T10:30:27.860024+0000","last_peered":"2024-04-25T10:30:27.860024+0000","last_clean":"2024-04-25T10:30:27.860024+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T10:30:27.875123+0000","last_undegraded":"2024-04-25T10:30:27.875123+0000","last_fullsized":"2024-04-25T10:30:27.875123+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":42,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:30:27.860024+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:30:27.860024+0000","last_clean_scrub_stamp":"2024-04-25T10:30:27.860024+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.13","version":"37'5","reported_seq":35,"reported_epoch":44,"state":"active+clean","last_fresh":"2024-04-25T10:30:29.883254+0000","last_change":"2024-04-25T10:30:11.764993+0000","last_active":"2024-04-25T10:30:29.883254+0000","last_peered":"2024-04-25T10:30:29.883254+0000","last_clean":"2024-04-25T10:30:29.883254+0000","last_became_active":"2024-04-25T10:30:11.764637+0000","last_became_peered":"2024-04-25T10:30:11.764637+0000","last_unstale":"2024-04-25T10:30:29.883254+0000","last_undegraded":"2024-04-25T10:30:29.883254+0000","last_fullsized":"2024-04-25T10:30:29.883254+0000","mapping_epoch":33,"log_start":"0'0","ondisk_log_start":"0'0","created":33,"last_epoch_clean":34,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:30:10.729165+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:30:10.729165+0000","last_clean_scrub_stamp":"2024-04-25T10:30:10.729165+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T17:24:13.848319+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6,"num_write_kb":8,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"5.12","version":"36'10","reported_seq":37,"reported_epoch":44,"state":"active+clean","last_fresh":"2024-04-25T10:30:29.882763+0000","last_change":"2024-04-25T10:30:12.770496+0000","last_active":"2024-04-25T10:30:29.882763+0000","last_peered":"2024-04-25T10:30:29.882763+0000","last_clean":"2024-04-25T10:30:29.882763+0000","last_became_active":"2024-04-25T10:30:12.770193+0000","last_became_peered":"2024-04-25T10:30:12.770193+0000","last_unstale":"2024-04-25T10:30:29.882763+0000","last_undegraded":"2024-04-25T10:30:29.882763+0000","last_fullsized":"2024-04-25T10:30:29.882763+0000","mapping_epoch":34,"log_start":"0'0","ondisk_log_start":"0'0","created":34,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:30:11.733978+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:30:11.733978+0000","last_clean_scrub_stamp":"2024-04-25T10:30:11.733978+0000","objects_scrubbed":0,"log_size":10,"log_dups_size":0,"ondisk_log_size":10,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T21:35:23.984378+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,1],"acting":[6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"6.11","version":"0'0","reported_seq":12,"reported_epoch":44,"state":"active+clean","last_fresh":"2024-04-25T10:30:29.882707+0000","last_change":"2024-04-25T10:30:28.884488+0000","last_active":"2024-04-25T10:30:29.882707+0000","last_peered":"2024-04-25T10:30:29.882707+0000","last_clean":"2024-04-25T10:30:29.882707+0000","last_became_active":"2024-04-25T10:30:28.884188+0000","last_became_peered":"2024-04-25T10:30:28.884188+0000","last_unstale":"2024-04-25T10:30:29.882707+0000","last_undegraded":"2024-04-25T10:30:29.882707+0000","last_fullsized":"2024-04-25T10:30:29.882707+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":42,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:30:27.860024+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:30:27.860024+0000","last_clean_scrub_stamp":"2024-04-25T10:30:27.860024+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T15:39:48.016111+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,1],"acting":[6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.14","version":"37'23","reported_seq":51,"reported_epoch":44,"state":"active+clean","last_fresh":"2024-04-25T10:30:29.883688+0000","last_change":"2024-04-25T10:30:11.757323+0000","last_active":"2024-04-25T10:30:29.883688+0000","last_peered":"2024-04-25T10:30:29.883688+0000","last_clean":"2024-04-25T10:30:29.883688+0000","last_became_active":"2024-04-25T10:30:11.757011+0000","last_became_peered":"2024-04-25T10:30:11.757011+0000","last_unstale":"2024-04-25T10:30:29.883688+0000","last_undegraded":"2024-04-25T10:30:29.883688+0000","last_fullsized":"2024-04-25T10:30:29.883688+0000","mapping_epoch":33,"log_start":"0'0","ondisk_log_start":"0'0","created":33,"last_epoch_clean":34,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:30:10.729165+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:30:10.729165+0000","last_clean_scrub_stamp":"2024-04-25T10:30:10.729165+0000","objects_scrubbed":0,"log_size":23,"log_dups_size":0,"ondisk_log_size":23,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T19:02:53.584992+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":23,"num_write_kb":25,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"5.15","version":"36'6","reported_seq":35,"reported_epoch":44,"state":"active+clean","last_fresh":"2024-04-25T10:30:29.881889+0000","last_change":"2024-04-25T10:30:12.776432+0000","last_active":"2024-04-25T10:30:29.881889+0000","last_peered":"2024-04-25T10:30:29.881889+0000","last_clean":"2024-04-25T10:30:29.881889+0000","last_became_active":"2024-04-25T10:30:12.775857+0000","last_became_peered":"2024-04-25T10:30:12.775857+0000","last_unstale":"2024-04-25T10:30:29.881889+0000","last_undegraded":"2024-04-25T10:30:29.881889+0000","last_fullsized":"2024-04-25T10:30:29.881889+0000","mapping_epoch":34,"log_start":"0'0","ondisk_log_start":"0'0","created":34,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:30:11.733978+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:30:11.733978+0000","last_clean_scrub_stamp":"2024-04-25T10:30:11.733978+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T14:29:10.343115+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"6.16","version":"0'0","reported_seq":1,"reported_epoch":42,"state":"creating+peering","last_fresh":"2024-04-25T10:30:27.876799+0000","last_change":"2024-04-25T10:30:27.876799+0000","last_active":"2024-04-25T10:30:27.860024+0000","last_peered":"2024-04-25T10:30:27.860024+0000","last_clean":"2024-04-25T10:30:27.860024+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T10:30:27.876799+0000","last_undegraded":"2024-04-25T10:30:27.876799+0000","last_fullsized":"2024-04-25T10:30:27.876799+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":42,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:30:27.860024+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:30:27.860024+0000","last_clean_scrub_stamp":"2024-04-25T10:30:27.860024+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.15","version":"37'5","reported_seq":35,"reported_epoch":44,"state":"active+clean","last_fresh":"2024-04-25T10:30:29.882033+0000","last_change":"2024-04-25T10:30:11.755959+0000","last_active":"2024-04-25T10:30:29.882033+0000","last_peered":"2024-04-25T10:30:29.882033+0000","last_clean":"2024-04-25T10:30:29.882033+0000","last_became_active":"2024-04-25T10:30:11.755603+0000","last_became_peered":"2024-04-25T10:30:11.755603+0000","last_unstale":"2024-04-25T10:30:29.882033+0000","last_undegraded":"2024-04-25T10:30:29.882033+0000","last_fullsized":"2024-04-25T10:30:29.882033+0000","mapping_epoch":33,"log_start":"0'0","ondisk_log_start":"0'0","created":33,"last_epoch_clean":34,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:30:10.729165+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:30:10.729165+0000","last_clean_scrub_stamp":"2024-04-25T10:30:10.729165+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T19:25:05.393017+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.14","version":"36'11","reported_seq":34,"reported_epoch":42,"state":"active+clean","last_fresh":"2024-04-25T10:30:27.877515+0000","last_change":"2024-04-25T10:30:12.770588+0000","last_active":"2024-04-25T10:30:27.877515+0000","last_peered":"2024-04-25T10:30:27.877515+0000","last_clean":"2024-04-25T10:30:27.877515+0000","last_became_active":"2024-04-25T10:30:12.770264+0000","last_became_peered":"2024-04-25T10:30:12.770264+0000","last_unstale":"2024-04-25T10:30:27.877515+0000","last_undegraded":"2024-04-25T10:30:27.877515+0000","last_fullsized":"2024-04-25T10:30:27.877515+0000","mapping_epoch":34,"log_start":"0'0","ondisk_log_start":"0'0","created":34,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:30:11.733978+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:30:11.733978+0000","last_clean_scrub_stamp":"2024-04-25T10:30:11.733978+0000","objects_scrubbed":0,"log_size":11,"log_dups_size":0,"ondisk_log_size":11,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T20:41:50.045543+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"6.17","version":"0'0","reported_seq":9,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-04-25T10:30:28.880577+0000","last_change":"2024-04-25T10:30:28.878754+0000","last_active":"2024-04-25T10:30:28.880577+0000","last_peered":"2024-04-25T10:30:28.880577+0000","last_clean":"2024-04-25T10:30:28.880577+0000","last_became_active":"2024-04-25T10:30:28.878242+0000","last_became_peered":"2024-04-25T10:30:28.878242+0000","last_unstale":"2024-04-25T10:30:28.880577+0000","last_undegraded":"2024-04-25T10:30:28.880577+0000","last_fullsized":"2024-04-25T10:30:28.880577+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":42,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:30:27.860024+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:30:27.860024+0000","last_clean_scrub_stamp":"2024-04-25T10:30:27.860024+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T16:08:46.113866+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7],"acting":[1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.16","version":"0'0","reported_seq":25,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-04-25T10:30:28.879092+0000","last_change":"2024-04-25T10:30:11.749473+0000","last_active":"2024-04-25T10:30:28.879092+0000","last_peered":"2024-04-25T10:30:28.879092+0000","last_clean":"2024-04-25T10:30:28.879092+0000","last_became_active":"2024-04-25T10:30:11.749135+0000","last_became_peered":"2024-04-25T10:30:11.749135+0000","last_unstale":"2024-04-25T10:30:28.879092+0000","last_undegraded":"2024-04-25T10:30:28.879092+0000","last_fullsized":"2024-04-25T10:30:28.879092+0000","mapping_epoch":33,"log_start":"0'0","ondisk_log_start":"0'0","created":33,"last_epoch_clean":34,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:30:10.729165+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:30:10.729165+0000","last_clean_scrub_stamp":"2024-04-25T10:30:10.729165+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T15:17:28.997853+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"5.17","version":"36'9","reported_seq":32,"reported_epoch":42,"state":"active+clean","last_fresh":"2024-04-25T10:30:27.872862+0000","last_change":"2024-04-25T10:30:12.773113+0000","last_active":"2024-04-25T10:30:27.872862+0000","last_peered":"2024-04-25T10:30:27.872862+0000","last_clean":"2024-04-25T10:30:27.872862+0000","last_became_active":"2024-04-25T10:30:12.772786+0000","last_became_peered":"2024-04-25T10:30:12.772786+0000","last_unstale":"2024-04-25T10:30:27.872862+0000","last_undegraded":"2024-04-25T10:30:27.872862+0000","last_fullsized":"2024-04-25T10:30:27.872862+0000","mapping_epoch":34,"log_start":"0'0","ondisk_log_start":"0'0","created":34,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:30:11.733978+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:30:11.733978+0000","last_clean_scrub_stamp":"2024-04-25T10:30:11.733978+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T18:45:03.004387+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5],"acting":[4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"6.14","version":"0'0","reported_seq":1,"reported_epoch":42,"state":"creating+peering","last_fresh":"2024-04-25T10:30:27.876893+0000","last_change":"2024-04-25T10:30:27.876893+0000","last_active":"2024-04-25T10:30:27.860024+0000","last_peered":"2024-04-25T10:30:27.860024+0000","last_clean":"2024-04-25T10:30:27.860024+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T10:30:27.876893+0000","last_undegraded":"2024-04-25T10:30:27.876893+0000","last_fullsized":"2024-04-25T10:30:27.876893+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":42,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:30:27.860024+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:30:27.860024+0000","last_clean_scrub_stamp":"2024-04-25T10:30:27.860024+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.17","version":"0'0","reported_seq":25,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-04-25T10:30:28.878199+0000","last_change":"2024-04-25T10:30:11.758416+0000","last_active":"2024-04-25T10:30:28.878199+0000","last_peered":"2024-04-25T10:30:28.878199+0000","last_clean":"2024-04-25T10:30:28.878199+0000","last_became_active":"2024-04-25T10:30:11.758049+0000","last_became_peered":"2024-04-25T10:30:11.758049+0000","last_unstale":"2024-04-25T10:30:28.878199+0000","last_undegraded":"2024-04-25T10:30:28.878199+0000","last_fullsized":"2024-04-25T10:30:28.878199+0000","mapping_epoch":33,"log_start":"0'0","ondisk_log_start":"0'0","created":33,"last_epoch_clean":34,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:30:10.729165+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:30:10.729165+0000","last_clean_scrub_stamp":"2024-04-25T10:30:10.729165+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T18:26:05.159854+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"5.16","version":"36'10","reported_seq":39,"reported_epoch":44,"state":"active+clean","last_fresh":"2024-04-25T10:30:29.882265+0000","last_change":"2024-04-25T10:30:12.770216+0000","last_active":"2024-04-25T10:30:29.882265+0000","last_peered":"2024-04-25T10:30:29.882265+0000","last_clean":"2024-04-25T10:30:29.882265+0000","last_became_active":"2024-04-25T10:30:12.769785+0000","last_became_peered":"2024-04-25T10:30:12.769785+0000","last_unstale":"2024-04-25T10:30:29.882265+0000","last_undegraded":"2024-04-25T10:30:29.882265+0000","last_fullsized":"2024-04-25T10:30:29.882265+0000","mapping_epoch":34,"log_start":"0'0","ondisk_log_start":"0'0","created":34,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:30:11.733978+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:30:11.733978+0000","last_clean_scrub_stamp":"2024-04-25T10:30:11.733978+0000","objects_scrubbed":0,"log_size":10,"log_dups_size":0,"ondisk_log_size":10,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T16:58:09.648786+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"6.15","version":"0'0","reported_seq":12,"reported_epoch":44,"state":"active+clean","last_fresh":"2024-04-25T10:30:29.883532+0000","last_change":"2024-04-25T10:30:28.879590+0000","last_active":"2024-04-25T10:30:29.883532+0000","last_peered":"2024-04-25T10:30:29.883532+0000","last_clean":"2024-04-25T10:30:29.883532+0000","last_became_active":"2024-04-25T10:30:28.879277+0000","last_became_peered":"2024-04-25T10:30:28.879277+0000","last_unstale":"2024-04-25T10:30:29.883532+0000","last_undegraded":"2024-04-25T10:30:29.883532+0000","last_fullsized":"2024-04-25T10:30:29.883532+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":42,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:30:27.860024+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:30:27.860024+0000","last_clean_scrub_stamp":"2024-04-25T10:30:27.860024+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T14:04:58.646968+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.18","version":"0'0","reported_seq":30,"reported_epoch":44,"state":"active+clean","last_fresh":"2024-04-25T10:30:29.882866+0000","last_change":"2024-04-25T10:30:11.756627+0000","last_active":"2024-04-25T10:30:29.882866+0000","last_peered":"2024-04-25T10:30:29.882866+0000","last_clean":"2024-04-25T10:30:29.882866+0000","last_became_active":"2024-04-25T10:30:11.756323+0000","last_became_peered":"2024-04-25T10:30:11.756323+0000","last_unstale":"2024-04-25T10:30:29.882866+0000","last_undegraded":"2024-04-25T10:30:29.882866+0000","last_fullsized":"2024-04-25T10:30:29.882866+0000","mapping_epoch":33,"log_start":"0'0","ondisk_log_start":"0'0","created":33,"last_epoch_clean":34,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:30:10.729165+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:30:10.729165+0000","last_clean_scrub_stamp":"2024-04-25T10:30:10.729165+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T16:49:20.087013+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4],"acting":[2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"5.19","version":"36'10","reported_seq":33,"reported_epoch":42,"state":"active+clean","last_fresh":"2024-04-25T10:30:27.876263+0000","last_change":"2024-04-25T10:30:12.780437+0000","last_active":"2024-04-25T10:30:27.876263+0000","last_peered":"2024-04-25T10:30:27.876263+0000","last_clean":"2024-04-25T10:30:27.876263+0000","last_became_active":"2024-04-25T10:30:12.779762+0000","last_became_peered":"2024-04-25T10:30:12.779762+0000","last_unstale":"2024-04-25T10:30:27.876263+0000","last_undegraded":"2024-04-25T10:30:27.876263+0000","last_fullsized":"2024-04-25T10:30:27.876263+0000","mapping_epoch":34,"log_start":"0'0","ondisk_log_start":"0'0","created":34,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:30:11.733978+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:30:11.733978+0000","last_clean_scrub_stamp":"2024-04-25T10:30:11.733978+0000","objects_scrubbed":0,"log_size":10,"log_dups_size":0,"ondisk_log_size":10,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T14:22:37.474013+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"6.1a","version":"0'0","reported_seq":12,"reported_epoch":44,"state":"active+clean","last_fresh":"2024-04-25T10:30:29.882257+0000","last_change":"2024-04-25T10:30:28.879591+0000","last_active":"2024-04-25T10:30:29.882257+0000","last_peered":"2024-04-25T10:30:29.882257+0000","last_clean":"2024-04-25T10:30:29.882257+0000","last_became_active":"2024-04-25T10:30:28.879271+0000","last_became_peered":"2024-04-25T10:30:28.879271+0000","last_unstale":"2024-04-25T10:30:29.882257+0000","last_undegraded":"2024-04-25T10:30:29.882257+0000","last_fullsized":"2024-04-25T10:30:29.882257+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":42,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:30:27.860024+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:30:27.860024+0000","last_clean_scrub_stamp":"2024-04-25T10:30:27.860024+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T20:14:35.145802+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.19","version":"0'0","reported_seq":25,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-04-25T10:30:28.874176+0000","last_change":"2024-04-25T10:30:11.757481+0000","last_active":"2024-04-25T10:30:28.874176+0000","last_peered":"2024-04-25T10:30:28.874176+0000","last_clean":"2024-04-25T10:30:28.874176+0000","last_became_active":"2024-04-25T10:30:11.757149+0000","last_became_peered":"2024-04-25T10:30:11.757149+0000","last_unstale":"2024-04-25T10:30:28.874176+0000","last_undegraded":"2024-04-25T10:30:28.874176+0000","last_fullsized":"2024-04-25T10:30:28.874176+0000","mapping_epoch":33,"log_start":"0'0","ondisk_log_start":"0'0","created":33,"last_epoch_clean":34,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:30:10.729165+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:30:10.729165+0000","last_clean_scrub_stamp":"2024-04-25T10:30:10.729165+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T13:23:15.814583+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"5.18","version":"36'8","reported_seq":33,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-04-25T10:30:28.874986+0000","last_change":"2024-04-25T10:30:12.769294+0000","last_active":"2024-04-25T10:30:28.874986+0000","last_peered":"2024-04-25T10:30:28.874986+0000","last_clean":"2024-04-25T10:30:28.874986+0000","last_became_active":"2024-04-25T10:30:12.768543+0000","last_became_peered":"2024-04-25T10:30:12.768543+0000","last_unstale":"2024-04-25T10:30:28.874986+0000","last_undegraded":"2024-04-25T10:30:28.874986+0000","last_fullsized":"2024-04-25T10:30:28.874986+0000","mapping_epoch":34,"log_start":"0'0","ondisk_log_start":"0'0","created":34,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:30:11.733978+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:30:11.733978+0000","last_clean_scrub_stamp":"2024-04-25T10:30:11.733978+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T18:26:06.164667+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"6.1b","version":"0'0","reported_seq":12,"reported_epoch":44,"state":"active+clean","last_fresh":"2024-04-25T10:30:29.883121+0000","last_change":"2024-04-25T10:30:28.885011+0000","last_active":"2024-04-25T10:30:29.883121+0000","last_peered":"2024-04-25T10:30:29.883121+0000","last_clean":"2024-04-25T10:30:29.883121+0000","last_became_active":"2024-04-25T10:30:28.883516+0000","last_became_peered":"2024-04-25T10:30:28.883516+0000","last_unstale":"2024-04-25T10:30:29.883121+0000","last_undegraded":"2024-04-25T10:30:29.883121+0000","last_fullsized":"2024-04-25T10:30:29.883121+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":42,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:30:27.860024+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:30:27.860024+0000","last_clean_scrub_stamp":"2024-04-25T10:30:27.860024+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T17:35:49.687185+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,2],"acting":[6,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.1a","version":"35'1","reported_seq":31,"reported_epoch":44,"state":"active+clean","last_fresh":"2024-04-25T10:30:29.883044+0000","last_change":"2024-04-25T10:30:11.758678+0000","last_active":"2024-04-25T10:30:29.883044+0000","last_peered":"2024-04-25T10:30:29.883044+0000","last_clean":"2024-04-25T10:30:29.883044+0000","last_became_active":"2024-04-25T10:30:11.758384+0000","last_became_peered":"2024-04-25T10:30:11.758384+0000","last_unstale":"2024-04-25T10:30:29.883044+0000","last_undegraded":"2024-04-25T10:30:29.883044+0000","last_fullsized":"2024-04-25T10:30:29.883044+0000","mapping_epoch":33,"log_start":"0'0","ondisk_log_start":"0'0","created":33,"last_epoch_clean":34,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:30:10.729165+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:30:10.729165+0000","last_clean_scrub_stamp":"2024-04-25T10:30:10.729165+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T16:17:54.376541+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,6],"acting":[2,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"5.1b","version":"36'8","reported_seq":37,"reported_epoch":44,"state":"active+clean","last_fresh":"2024-04-25T10:30:29.883109+0000","last_change":"2024-04-25T10:30:12.784767+0000","last_active":"2024-04-25T10:30:29.883109+0000","last_peered":"2024-04-25T10:30:29.883109+0000","last_clean":"2024-04-25T10:30:29.883109+0000","last_became_active":"2024-04-25T10:30:12.784311+0000","last_became_peered":"2024-04-25T10:30:12.784311+0000","last_unstale":"2024-04-25T10:30:29.883109+0000","last_undegraded":"2024-04-25T10:30:29.883109+0000","last_fullsized":"2024-04-25T10:30:29.883109+0000","mapping_epoch":34,"log_start":"0'0","ondisk_log_start":"0'0","created":34,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:30:11.733978+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:30:11.733978+0000","last_clean_scrub_stamp":"2024-04-25T10:30:11.733978+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T19:56:57.765496+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,7],"acting":[2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"6.18","version":"0'0","reported_seq":1,"reported_epoch":42,"state":"creating+peering","last_fresh":"2024-04-25T10:30:27.874480+0000","last_change":"2024-04-25T10:30:27.874480+0000","last_active":"2024-04-25T10:30:27.860024+0000","last_peered":"2024-04-25T10:30:27.860024+0000","last_clean":"2024-04-25T10:30:27.860024+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T10:30:27.874480+0000","last_undegraded":"2024-04-25T10:30:27.874480+0000","last_fullsized":"2024-04-25T10:30:27.874480+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":42,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:30:27.860024+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:30:27.860024+0000","last_clean_scrub_stamp":"2024-04-25T10:30:27.860024+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5],"acting":[4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.1b","version":"35'1","reported_seq":31,"reported_epoch":44,"state":"active+clean","last_fresh":"2024-04-25T10:30:29.883205+0000","last_change":"2024-04-25T10:30:11.753667+0000","last_active":"2024-04-25T10:30:29.883205+0000","last_peered":"2024-04-25T10:30:29.883205+0000","last_clean":"2024-04-25T10:30:29.883205+0000","last_became_active":"2024-04-25T10:30:11.753340+0000","last_became_peered":"2024-04-25T10:30:11.753340+0000","last_unstale":"2024-04-25T10:30:29.883205+0000","last_undegraded":"2024-04-25T10:30:29.883205+0000","last_fullsized":"2024-04-25T10:30:29.883205+0000","mapping_epoch":33,"log_start":"0'0","ondisk_log_start":"0'0","created":33,"last_epoch_clean":34,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:30:10.729165+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:30:10.729165+0000","last_clean_scrub_stamp":"2024-04-25T10:30:10.729165+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T16:09:04.081750+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"5.1a","version":"36'7","reported_seq":32,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-04-25T10:30:28.874760+0000","last_change":"2024-04-25T10:30:12.773835+0000","last_active":"2024-04-25T10:30:28.874760+0000","last_peered":"2024-04-25T10:30:28.874760+0000","last_clean":"2024-04-25T10:30:28.874760+0000","last_became_active":"2024-04-25T10:30:12.773538+0000","last_became_peered":"2024-04-25T10:30:12.773538+0000","last_unstale":"2024-04-25T10:30:28.874760+0000","last_undegraded":"2024-04-25T10:30:28.874760+0000","last_fullsized":"2024-04-25T10:30:28.874760+0000","mapping_epoch":34,"log_start":"0'0","ondisk_log_start":"0'0","created":34,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:30:11.733978+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:30:11.733978+0000","last_clean_scrub_stamp":"2024-04-25T10:30:11.733978+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T18:31:49.260513+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"6.19","version":"0'0","reported_seq":9,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-04-25T10:30:28.885557+0000","last_change":"2024-04-25T10:30:28.883543+0000","last_active":"2024-04-25T10:30:28.885557+0000","last_peered":"2024-04-25T10:30:28.885557+0000","last_clean":"2024-04-25T10:30:28.885557+0000","last_became_active":"2024-04-25T10:30:28.883229+0000","last_became_peered":"2024-04-25T10:30:28.883229+0000","last_unstale":"2024-04-25T10:30:28.885557+0000","last_undegraded":"2024-04-25T10:30:28.885557+0000","last_fullsized":"2024-04-25T10:30:28.885557+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":42,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:30:27.860024+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:30:27.860024+0000","last_clean_scrub_stamp":"2024-04-25T10:30:27.860024+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T18:12:57.452350+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"6.1e","version":"0'0","reported_seq":12,"reported_epoch":44,"state":"active+clean","last_fresh":"2024-04-25T10:30:29.882435+0000","last_change":"2024-04-25T10:30:28.874417+0000","last_active":"2024-04-25T10:30:29.882435+0000","last_peered":"2024-04-25T10:30:29.882435+0000","last_clean":"2024-04-25T10:30:29.882435+0000","last_became_active":"2024-04-25T10:30:28.874102+0000","last_became_peered":"2024-04-25T10:30:28.874102+0000","last_unstale":"2024-04-25T10:30:29.882435+0000","last_undegraded":"2024-04-25T10:30:29.882435+0000","last_fullsized":"2024-04-25T10:30:29.882435+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":42,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:30:27.860024+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:30:27.860024+0000","last_clean_scrub_stamp":"2024-04-25T10:30:27.860024+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T21:25:07.511482+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.1c","version":"37'3","reported_seq":28,"reported_epoch":42,"state":"active+clean","last_fresh":"2024-04-25T10:30:27.877716+0000","last_change":"2024-04-25T10:30:11.759829+0000","last_active":"2024-04-25T10:30:27.877716+0000","last_peered":"2024-04-25T10:30:27.877716+0000","last_clean":"2024-04-25T10:30:27.877716+0000","last_became_active":"2024-04-25T10:30:11.759465+0000","last_became_peered":"2024-04-25T10:30:11.759465+0000","last_unstale":"2024-04-25T10:30:27.877716+0000","last_undegraded":"2024-04-25T10:30:27.877716+0000","last_fullsized":"2024-04-25T10:30:27.877716+0000","mapping_epoch":33,"log_start":"0'0","ondisk_log_start":"0'0","created":33,"last_epoch_clean":34,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:30:10.729165+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:30:10.729165+0000","last_clean_scrub_stamp":"2024-04-25T10:30:10.729165+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T13:31:53.608355+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.1d","version":"36'15","reported_seq":38,"reported_epoch":42,"state":"active+clean","last_fresh":"2024-04-25T10:30:27.876677+0000","last_change":"2024-04-25T10:30:12.773474+0000","last_active":"2024-04-25T10:30:27.876677+0000","last_peered":"2024-04-25T10:30:27.876677+0000","last_clean":"2024-04-25T10:30:27.876677+0000","last_became_active":"2024-04-25T10:30:12.773133+0000","last_became_peered":"2024-04-25T10:30:12.773133+0000","last_unstale":"2024-04-25T10:30:27.876677+0000","last_undegraded":"2024-04-25T10:30:27.876677+0000","last_fullsized":"2024-04-25T10:30:27.876677+0000","mapping_epoch":34,"log_start":"0'0","ondisk_log_start":"0'0","created":34,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:30:11.733978+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:30:11.733978+0000","last_clean_scrub_stamp":"2024-04-25T10:30:11.733978+0000","objects_scrubbed":0,"log_size":15,"log_dups_size":0,"ondisk_log_size":15,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T17:24:23.630916+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"6.1f","version":"0'0","reported_seq":12,"reported_epoch":44,"state":"active+clean","last_fresh":"2024-04-25T10:30:29.883478+0000","last_change":"2024-04-25T10:30:28.884033+0000","last_active":"2024-04-25T10:30:29.883478+0000","last_peered":"2024-04-25T10:30:29.883478+0000","last_clean":"2024-04-25T10:30:29.883478+0000","last_became_active":"2024-04-25T10:30:28.883610+0000","last_became_peered":"2024-04-25T10:30:28.883610+0000","last_unstale":"2024-04-25T10:30:29.883478+0000","last_undegraded":"2024-04-25T10:30:29.883478+0000","last_fullsized":"2024-04-25T10:30:29.883478+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":42,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:30:27.860024+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:30:27.860024+0000","last_clean_scrub_stamp":"2024-04-25T10:30:27.860024+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T18:12:57.452350+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.1d","version":"37'3","reported_seq":28,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-04-25T10:30:28.875875+0000","last_change":"2024-04-25T10:30:11.752439+0000","last_active":"2024-04-25T10:30:28.875875+0000","last_peered":"2024-04-25T10:30:28.875875+0000","last_clean":"2024-04-25T10:30:28.875875+0000","last_became_active":"2024-04-25T10:30:11.752050+0000","last_became_peered":"2024-04-25T10:30:11.752050+0000","last_unstale":"2024-04-25T10:30:28.875875+0000","last_undegraded":"2024-04-25T10:30:28.875875+0000","last_fullsized":"2024-04-25T10:30:28.875875+0000","mapping_epoch":33,"log_start":"0'0","ondisk_log_start":"0'0","created":33,"last_epoch_clean":34,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:30:10.729165+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:30:10.729165+0000","last_clean_scrub_stamp":"2024-04-25T10:30:10.729165+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T11:41:21.358729+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"5.1c","version":"36'9","reported_seq":38,"reported_epoch":44,"state":"active+clean","last_fresh":"2024-04-25T10:30:29.883403+0000","last_change":"2024-04-25T10:30:12.781913+0000","last_active":"2024-04-25T10:30:29.883403+0000","last_peered":"2024-04-25T10:30:29.883403+0000","last_clean":"2024-04-25T10:30:29.883403+0000","last_became_active":"2024-04-25T10:30:12.781111+0000","last_became_peered":"2024-04-25T10:30:12.781111+0000","last_unstale":"2024-04-25T10:30:29.883403+0000","last_undegraded":"2024-04-25T10:30:29.883403+0000","last_fullsized":"2024-04-25T10:30:29.883403+0000","mapping_epoch":34,"log_start":"0'0","ondisk_log_start":"0'0","created":34,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:30:11.733978+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:30:11.733978+0000","last_clean_scrub_stamp":"2024-04-25T10:30:11.733978+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T16:29:27.103512+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4],"acting":[2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.1e","version":"35'1","reported_seq":26,"reported_epoch":42,"state":"active+clean","last_fresh":"2024-04-25T10:30:27.877804+0000","last_change":"2024-04-25T10:30:11.761160+0000","last_active":"2024-04-25T10:30:27.877804+0000","last_peered":"2024-04-25T10:30:27.877804+0000","last_clean":"2024-04-25T10:30:27.877804+0000","last_became_active":"2024-04-25T10:30:11.760819+0000","last_became_peered":"2024-04-25T10:30:11.760819+0000","last_unstale":"2024-04-25T10:30:27.877804+0000","last_undegraded":"2024-04-25T10:30:27.877804+0000","last_fullsized":"2024-04-25T10:30:27.877804+0000","mapping_epoch":33,"log_start":"0'0","ondisk_log_start":"0'0","created":33,"last_epoch_clean":34,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:30:10.729165+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:30:10.729165+0000","last_clean_scrub_stamp":"2024-04-25T10:30:10.729165+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T13:00:22.139475+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.1f","version":"36'6","reported_seq":35,"reported_epoch":44,"state":"active+clean","last_fresh":"2024-04-25T10:30:29.883526+0000","last_change":"2024-04-25T10:30:12.772257+0000","last_active":"2024-04-25T10:30:29.883526+0000","last_peered":"2024-04-25T10:30:29.883526+0000","last_clean":"2024-04-25T10:30:29.883526+0000","last_became_active":"2024-04-25T10:30:12.771923+0000","last_became_peered":"2024-04-25T10:30:12.771923+0000","last_unstale":"2024-04-25T10:30:29.883526+0000","last_undegraded":"2024-04-25T10:30:29.883526+0000","last_fullsized":"2024-04-25T10:30:29.883526+0000","mapping_epoch":34,"log_start":"0'0","ondisk_log_start":"0'0","created":34,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:30:11.733978+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:30:11.733978+0000","last_clean_scrub_stamp":"2024-04-25T10:30:11.733978+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T17:35:33.561138+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"6.1c","version":"0'0","reported_seq":12,"reported_epoch":44,"state":"active+clean","last_fresh":"2024-04-25T10:30:29.882853+0000","last_change":"2024-04-25T10:30:28.875422+0000","last_active":"2024-04-25T10:30:29.882853+0000","last_peered":"2024-04-25T10:30:29.882853+0000","last_clean":"2024-04-25T10:30:29.882853+0000","last_became_active":"2024-04-25T10:30:28.875079+0000","last_became_peered":"2024-04-25T10:30:28.875079+0000","last_unstale":"2024-04-25T10:30:29.882853+0000","last_undegraded":"2024-04-25T10:30:29.882853+0000","last_fullsized":"2024-04-25T10:30:29.882853+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":42,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:30:27.860024+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:30:27.860024+0000","last_clean_scrub_stamp":"2024-04-25T10:30:27.860024+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T13:23:55.301869+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.1f","version":"37'8","reported_seq":38,"reported_epoch":44,"state":"active+clean","last_fresh":"2024-04-25T10:30:29.882826+0000","last_change":"2024-04-25T10:30:11.756978+0000","last_active":"2024-04-25T10:30:29.882826+0000","last_peered":"2024-04-25T10:30:29.882826+0000","last_clean":"2024-04-25T10:30:29.882826+0000","last_became_active":"2024-04-25T10:30:11.756617+0000","last_became_peered":"2024-04-25T10:30:11.756617+0000","last_unstale":"2024-04-25T10:30:29.882826+0000","last_undegraded":"2024-04-25T10:30:29.882826+0000","last_fullsized":"2024-04-25T10:30:29.882826+0000","mapping_epoch":33,"log_start":"0'0","ondisk_log_start":"0'0","created":33,"last_epoch_clean":34,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:30:10.729165+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:30:10.729165+0000","last_clean_scrub_stamp":"2024-04-25T10:30:10.729165+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T12:27:53.559880+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":1,"num_read_kb":0,"num_write":10,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.1e","version":"36'5","reported_seq":32,"reported_epoch":44,"state":"active+clean","last_fresh":"2024-04-25T10:30:29.883469+0000","last_change":"2024-04-25T10:30:12.769800+0000","last_active":"2024-04-25T10:30:29.883469+0000","last_peered":"2024-04-25T10:30:29.883469+0000","last_clean":"2024-04-25T10:30:29.883469+0000","last_became_active":"2024-04-25T10:30:12.769470+0000","last_became_peered":"2024-04-25T10:30:12.769470+0000","last_unstale":"2024-04-25T10:30:29.883469+0000","last_undegraded":"2024-04-25T10:30:29.883469+0000","last_fullsized":"2024-04-25T10:30:29.883469+0000","mapping_epoch":34,"log_start":"0'0","ondisk_log_start":"0'0","created":34,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:30:11.733978+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:30:11.733978+0000","last_clean_scrub_stamp":"2024-04-25T10:30:11.733978+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T13:11:54.897586+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"6.1d","version":"0'0","reported_seq":12,"reported_epoch":44,"state":"active+clean","last_fresh":"2024-04-25T10:30:29.883295+0000","last_change":"2024-04-25T10:30:28.884277+0000","last_active":"2024-04-25T10:30:29.883295+0000","last_peered":"2024-04-25T10:30:29.883295+0000","last_clean":"2024-04-25T10:30:29.883295+0000","last_became_active":"2024-04-25T10:30:28.883764+0000","last_became_peered":"2024-04-25T10:30:28.883764+0000","last_unstale":"2024-04-25T10:30:29.883295+0000","last_undegraded":"2024-04-25T10:30:29.883295+0000","last_fullsized":"2024-04-25T10:30:29.883295+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":42,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:30:27.860024+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:30:27.860024+0000","last_clean_scrub_stamp":"2024-04-25T10:30:27.860024+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T18:36:07.211600+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"5.20","version":"36'9","reported_seq":38,"reported_epoch":44,"state":"active+clean","last_fresh":"2024-04-25T10:30:29.881619+0000","last_change":"2024-04-25T10:30:12.766692+0000","last_active":"2024-04-25T10:30:29.881619+0000","last_peered":"2024-04-25T10:30:29.881619+0000","last_clean":"2024-04-25T10:30:29.881619+0000","last_became_active":"2024-04-25T10:30:12.766336+0000","last_became_peered":"2024-04-25T10:30:12.766336+0000","last_unstale":"2024-04-25T10:30:29.881619+0000","last_undegraded":"2024-04-25T10:30:29.881619+0000","last_fullsized":"2024-04-25T10:30:29.881619+0000","mapping_epoch":34,"log_start":"0'0","ondisk_log_start":"0'0","created":34,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:30:11.733978+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:30:11.733978+0000","last_clean_scrub_stamp":"2024-04-25T10:30:11.733978+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T18:11:27.798982+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.21","version":"36'12","reported_seq":41,"reported_epoch":44,"state":"active+clean","last_fresh":"2024-04-25T10:30:29.882888+0000","last_change":"2024-04-25T10:30:12.774245+0000","last_active":"2024-04-25T10:30:29.882888+0000","last_peered":"2024-04-25T10:30:29.882888+0000","last_clean":"2024-04-25T10:30:29.882888+0000","last_became_active":"2024-04-25T10:30:12.773923+0000","last_became_peered":"2024-04-25T10:30:12.773923+0000","last_unstale":"2024-04-25T10:30:29.882888+0000","last_undegraded":"2024-04-25T10:30:29.882888+0000","last_fullsized":"2024-04-25T10:30:29.882888+0000","mapping_epoch":34,"log_start":"0'0","ondisk_log_start":"0'0","created":34,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:30:11.733978+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:30:11.733978+0000","last_clean_scrub_stamp":"2024-04-25T10:30:11.733978+0000","objects_scrubbed":0,"log_size":12,"log_dups_size":0,"ondisk_log_size":12,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T22:09:01.889527+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,7],"acting":[2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"5.22","version":"36'8","reported_seq":32,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-04-25T10:30:28.877192+0000","last_change":"2024-04-25T10:30:12.769958+0000","last_active":"2024-04-25T10:30:28.877192+0000","last_peered":"2024-04-25T10:30:28.877192+0000","last_clean":"2024-04-25T10:30:28.877192+0000","last_became_active":"2024-04-25T10:30:12.769590+0000","last_became_peered":"2024-04-25T10:30:12.769590+0000","last_unstale":"2024-04-25T10:30:28.877192+0000","last_undegraded":"2024-04-25T10:30:28.877192+0000","last_fullsized":"2024-04-25T10:30:28.877192+0000","mapping_epoch":34,"log_start":"0'0","ondisk_log_start":"0'0","created":34,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:30:11.733978+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:30:11.733978+0000","last_clean_scrub_stamp":"2024-04-25T10:30:11.733978+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T11:57:29.979255+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"5.23","version":"36'7","reported_seq":31,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-04-25T10:30:28.876441+0000","last_change":"2024-04-25T10:30:12.781995+0000","last_active":"2024-04-25T10:30:28.876441+0000","last_peered":"2024-04-25T10:30:28.876441+0000","last_clean":"2024-04-25T10:30:28.876441+0000","last_became_active":"2024-04-25T10:30:12.781358+0000","last_became_peered":"2024-04-25T10:30:12.781358+0000","last_unstale":"2024-04-25T10:30:28.876441+0000","last_undegraded":"2024-04-25T10:30:28.876441+0000","last_fullsized":"2024-04-25T10:30:28.876441+0000","mapping_epoch":34,"log_start":"0'0","ondisk_log_start":"0'0","created":34,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:30:11.733978+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:30:11.733978+0000","last_clean_scrub_stamp":"2024-04-25T10:30:11.733978+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T18:00:26.511360+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"5.24","version":"36'4","reported_seq":33,"reported_epoch":44,"state":"active+clean","last_fresh":"2024-04-25T10:30:29.883349+0000","last_change":"2024-04-25T10:30:12.775515+0000","last_active":"2024-04-25T10:30:29.883349+0000","last_peered":"2024-04-25T10:30:29.883349+0000","last_clean":"2024-04-25T10:30:29.883349+0000","last_became_active":"2024-04-25T10:30:12.775167+0000","last_became_peered":"2024-04-25T10:30:12.775167+0000","last_unstale":"2024-04-25T10:30:29.883349+0000","last_undegraded":"2024-04-25T10:30:29.883349+0000","last_fullsized":"2024-04-25T10:30:29.883349+0000","mapping_epoch":34,"log_start":"0'0","ondisk_log_start":"0'0","created":34,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:30:11.733978+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:30:11.733978+0000","last_clean_scrub_stamp":"2024-04-25T10:30:11.733978+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T21:35:23.984378+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,6],"acting":[2,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"5.25","version":"36'6","reported_seq":30,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-04-25T10:30:28.876545+0000","last_change":"2024-04-25T10:30:12.767505+0000","last_active":"2024-04-25T10:30:28.876545+0000","last_peered":"2024-04-25T10:30:28.876545+0000","last_clean":"2024-04-25T10:30:28.876545+0000","last_became_active":"2024-04-25T10:30:12.767149+0000","last_became_peered":"2024-04-25T10:30:12.767149+0000","last_unstale":"2024-04-25T10:30:28.876545+0000","last_undegraded":"2024-04-25T10:30:28.876545+0000","last_fullsized":"2024-04-25T10:30:28.876545+0000","mapping_epoch":34,"log_start":"0'0","ondisk_log_start":"0'0","created":34,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:30:11.733978+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:30:11.733978+0000","last_clean_scrub_stamp":"2024-04-25T10:30:11.733978+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T16:29:27.103512+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"5.26","version":"36'15","reported_seq":38,"reported_epoch":42,"state":"active+clean","last_fresh":"2024-04-25T10:30:27.877674+0000","last_change":"2024-04-25T10:30:12.771420+0000","last_active":"2024-04-25T10:30:27.877674+0000","last_peered":"2024-04-25T10:30:27.877674+0000","last_clean":"2024-04-25T10:30:27.877674+0000","last_became_active":"2024-04-25T10:30:12.771117+0000","last_became_peered":"2024-04-25T10:30:12.771117+0000","last_unstale":"2024-04-25T10:30:27.877674+0000","last_undegraded":"2024-04-25T10:30:27.877674+0000","last_fullsized":"2024-04-25T10:30:27.877674+0000","mapping_epoch":34,"log_start":"0'0","ondisk_log_start":"0'0","created":34,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:30:11.733978+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:30:11.733978+0000","last_clean_scrub_stamp":"2024-04-25T10:30:11.733978+0000","objects_scrubbed":0,"log_size":15,"log_dups_size":0,"ondisk_log_size":15,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T16:08:42.820761+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.27","version":"36'2","reported_seq":25,"reported_epoch":42,"state":"active+clean","last_fresh":"2024-04-25T10:30:27.873896+0000","last_change":"2024-04-25T10:30:12.779972+0000","last_active":"2024-04-25T10:30:27.873896+0000","last_peered":"2024-04-25T10:30:27.873896+0000","last_clean":"2024-04-25T10:30:27.873896+0000","last_became_active":"2024-04-25T10:30:12.779526+0000","last_became_peered":"2024-04-25T10:30:12.779526+0000","last_unstale":"2024-04-25T10:30:27.873896+0000","last_undegraded":"2024-04-25T10:30:27.873896+0000","last_fullsized":"2024-04-25T10:30:27.873896+0000","mapping_epoch":34,"log_start":"0'0","ondisk_log_start":"0'0","created":34,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:30:11.733978+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:30:11.733978+0000","last_clean_scrub_stamp":"2024-04-25T10:30:11.733978+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T22:04:21.879881+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.28","version":"36'6","reported_seq":35,"reported_epoch":44,"state":"active+clean","last_fresh":"2024-04-25T10:30:29.882628+0000","last_change":"2024-04-25T10:30:12.783394+0000","last_active":"2024-04-25T10:30:29.882628+0000","last_peered":"2024-04-25T10:30:29.882628+0000","last_clean":"2024-04-25T10:30:29.882628+0000","last_became_active":"2024-04-25T10:30:12.783066+0000","last_became_peered":"2024-04-25T10:30:12.783066+0000","last_unstale":"2024-04-25T10:30:29.882628+0000","last_undegraded":"2024-04-25T10:30:29.882628+0000","last_fullsized":"2024-04-25T10:30:29.882628+0000","mapping_epoch":34,"log_start":"0'0","ondisk_log_start":"0'0","created":34,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:30:11.733978+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:30:11.733978+0000","last_clean_scrub_stamp":"2024-04-25T10:30:11.733978+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T15:39:31.890064+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4],"acting":[2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"5.29","version":"36'8","reported_seq":35,"reported_epoch":44,"state":"active+clean","last_fresh":"2024-04-25T10:30:29.882807+0000","last_change":"2024-04-25T10:30:12.763911+0000","last_active":"2024-04-25T10:30:29.882807+0000","last_peered":"2024-04-25T10:30:29.882807+0000","last_clean":"2024-04-25T10:30:29.882807+0000","last_became_active":"2024-04-25T10:30:12.763578+0000","last_became_peered":"2024-04-25T10:30:12.763578+0000","last_unstale":"2024-04-25T10:30:29.882807+0000","last_undegraded":"2024-04-25T10:30:29.882807+0000","last_fullsized":"2024-04-25T10:30:29.882807+0000","mapping_epoch":34,"log_start":"0'0","ondisk_log_start":"0'0","created":34,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:30:11.733978+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:30:11.733978+0000","last_clean_scrub_stamp":"2024-04-25T10:30:11.733978+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T18:52:45.539758+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,1],"acting":[6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"5.2a","version":"36'9","reported_seq":38,"reported_epoch":44,"state":"active+clean","last_fresh":"2024-04-25T10:30:29.882996+0000","last_change":"2024-04-25T10:30:12.782730+0000","last_active":"2024-04-25T10:30:29.882996+0000","last_peered":"2024-04-25T10:30:29.882996+0000","last_clean":"2024-04-25T10:30:29.882996+0000","last_became_active":"2024-04-25T10:30:12.782261+0000","last_became_peered":"2024-04-25T10:30:12.782261+0000","last_unstale":"2024-04-25T10:30:29.882996+0000","last_undegraded":"2024-04-25T10:30:29.882996+0000","last_fullsized":"2024-04-25T10:30:29.882996+0000","mapping_epoch":34,"log_start":"0'0","ondisk_log_start":"0'0","created":34,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:30:11.733978+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:30:11.733978+0000","last_clean_scrub_stamp":"2024-04-25T10:30:11.733978+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T18:00:26.511360+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4],"acting":[2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"5.2b","version":"36'11","reported_seq":35,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-04-25T10:30:28.876504+0000","last_change":"2024-04-25T10:30:12.771344+0000","last_active":"2024-04-25T10:30:28.876504+0000","last_peered":"2024-04-25T10:30:28.876504+0000","last_clean":"2024-04-25T10:30:28.876504+0000","last_became_active":"2024-04-25T10:30:12.770949+0000","last_became_peered":"2024-04-25T10:30:12.770949+0000","last_unstale":"2024-04-25T10:30:28.876504+0000","last_undegraded":"2024-04-25T10:30:28.876504+0000","last_fullsized":"2024-04-25T10:30:28.876504+0000","mapping_epoch":34,"log_start":"0'0","ondisk_log_start":"0'0","created":34,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:30:11.733978+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:30:11.733978+0000","last_clean_scrub_stamp":"2024-04-25T10:30:11.733978+0000","objects_scrubbed":0,"log_size":11,"log_dups_size":0,"ondisk_log_size":11,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T16:29:27.103512+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"5.2c","version":"36'8","reported_seq":35,"reported_epoch":44,"state":"active+clean","last_fresh":"2024-04-25T10:30:29.883520+0000","last_change":"2024-04-25T10:30:12.772663+0000","last_active":"2024-04-25T10:30:29.883520+0000","last_peered":"2024-04-25T10:30:29.883520+0000","last_clean":"2024-04-25T10:30:29.883520+0000","last_became_active":"2024-04-25T10:30:12.772356+0000","last_became_peered":"2024-04-25T10:30:12.772356+0000","last_unstale":"2024-04-25T10:30:29.883520+0000","last_undegraded":"2024-04-25T10:30:29.883520+0000","last_fullsized":"2024-04-25T10:30:29.883520+0000","mapping_epoch":34,"log_start":"0'0","ondisk_log_start":"0'0","created":34,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:30:11.733978+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:30:11.733978+0000","last_clean_scrub_stamp":"2024-04-25T10:30:11.733978+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T12:21:33.158775+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"5.2d","version":"36'7","reported_seq":31,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-04-25T10:30:28.875358+0000","last_change":"2024-04-25T10:30:12.768688+0000","last_active":"2024-04-25T10:30:28.875358+0000","last_peered":"2024-04-25T10:30:28.875358+0000","last_clean":"2024-04-25T10:30:28.875358+0000","last_became_active":"2024-04-25T10:30:12.768382+0000","last_became_peered":"2024-04-25T10:30:12.768382+0000","last_unstale":"2024-04-25T10:30:28.875358+0000","last_undegraded":"2024-04-25T10:30:28.875358+0000","last_fullsized":"2024-04-25T10:30:28.875358+0000","mapping_epoch":34,"log_start":"0'0","ondisk_log_start":"0'0","created":34,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:30:11.733978+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:30:11.733978+0000","last_clean_scrub_stamp":"2024-04-25T10:30:11.733978+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T18:26:06.164667+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"5.2e","version":"36'10","reported_seq":34,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-04-25T10:30:28.876239+0000","last_change":"2024-04-25T10:30:12.766393+0000","last_active":"2024-04-25T10:30:28.876239+0000","last_peered":"2024-04-25T10:30:28.876239+0000","last_clean":"2024-04-25T10:30:28.876239+0000","last_became_active":"2024-04-25T10:30:12.766096+0000","last_became_peered":"2024-04-25T10:30:12.766096+0000","last_unstale":"2024-04-25T10:30:28.876239+0000","last_undegraded":"2024-04-25T10:30:28.876239+0000","last_fullsized":"2024-04-25T10:30:28.876239+0000","mapping_epoch":34,"log_start":"0'0","ondisk_log_start":"0'0","created":34,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:30:11.733978+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:30:11.733978+0000","last_clean_scrub_stamp":"2024-04-25T10:30:11.733978+0000","objects_scrubbed":0,"log_size":10,"log_dups_size":0,"ondisk_log_size":10,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T16:49:00.268423+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"5.2f","version":"36'9","reported_seq":38,"reported_epoch":44,"state":"active+clean","last_fresh":"2024-04-25T10:30:29.882690+0000","last_change":"2024-04-25T10:30:12.766616+0000","last_active":"2024-04-25T10:30:29.882690+0000","last_peered":"2024-04-25T10:30:29.882690+0000","last_clean":"2024-04-25T10:30:29.882690+0000","last_became_active":"2024-04-25T10:30:12.766290+0000","last_became_peered":"2024-04-25T10:30:12.766290+0000","last_unstale":"2024-04-25T10:30:29.882690+0000","last_undegraded":"2024-04-25T10:30:29.882690+0000","last_fullsized":"2024-04-25T10:30:29.882690+0000","mapping_epoch":34,"log_start":"0'0","ondisk_log_start":"0'0","created":34,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:30:11.733978+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:30:11.733978+0000","last_clean_scrub_stamp":"2024-04-25T10:30:11.733978+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T13:22:50.975199+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":2097152,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":2048,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.30","version":"36'9","reported_seq":33,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-04-25T10:30:28.876615+0000","last_change":"2024-04-25T10:30:12.765989+0000","last_active":"2024-04-25T10:30:28.876615+0000","last_peered":"2024-04-25T10:30:28.876615+0000","last_clean":"2024-04-25T10:30:28.876615+0000","last_became_active":"2024-04-25T10:30:12.765648+0000","last_became_peered":"2024-04-25T10:30:12.765648+0000","last_unstale":"2024-04-25T10:30:28.876615+0000","last_undegraded":"2024-04-25T10:30:28.876615+0000","last_fullsized":"2024-04-25T10:30:28.876615+0000","mapping_epoch":34,"log_start":"0'0","ondisk_log_start":"0'0","created":34,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:30:11.733978+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:30:11.733978+0000","last_clean_scrub_stamp":"2024-04-25T10:30:11.733978+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T16:29:40.621322+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"5.31","version":"36'7","reported_seq":31,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-04-25T10:30:28.877047+0000","last_change":"2024-04-25T10:30:12.777324+0000","last_active":"2024-04-25T10:30:28.877047+0000","last_peered":"2024-04-25T10:30:28.877047+0000","last_clean":"2024-04-25T10:30:28.877047+0000","last_became_active":"2024-04-25T10:30:12.776983+0000","last_became_peered":"2024-04-25T10:30:12.776983+0000","last_unstale":"2024-04-25T10:30:28.877047+0000","last_undegraded":"2024-04-25T10:30:28.877047+0000","last_fullsized":"2024-04-25T10:30:28.877047+0000","mapping_epoch":34,"log_start":"0'0","ondisk_log_start":"0'0","created":34,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:30:11.733978+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:30:11.733978+0000","last_clean_scrub_stamp":"2024-04-25T10:30:11.733978+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T20:33:24.458243+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"5.32","version":"36'8","reported_seq":31,"reported_epoch":42,"state":"active+clean","last_fresh":"2024-04-25T10:30:27.876591+0000","last_change":"2024-04-25T10:30:12.772243+0000","last_active":"2024-04-25T10:30:27.876591+0000","last_peered":"2024-04-25T10:30:27.876591+0000","last_clean":"2024-04-25T10:30:27.876591+0000","last_became_active":"2024-04-25T10:30:12.771936+0000","last_became_peered":"2024-04-25T10:30:12.771936+0000","last_unstale":"2024-04-25T10:30:27.876591+0000","last_undegraded":"2024-04-25T10:30:27.876591+0000","last_fullsized":"2024-04-25T10:30:27.876591+0000","mapping_epoch":34,"log_start":"0'0","ondisk_log_start":"0'0","created":34,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:30:11.733978+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:30:11.733978+0000","last_clean_scrub_stamp":"2024-04-25T10:30:11.733978+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T13:11:22.432926+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5],"acting":[4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.33","version":"36'10","reported_seq":37,"reported_epoch":44,"state":"active+clean","last_fresh":"2024-04-25T10:30:29.883324+0000","last_change":"2024-04-25T10:30:12.777499+0000","last_active":"2024-04-25T10:30:29.883324+0000","last_peered":"2024-04-25T10:30:29.883324+0000","last_clean":"2024-04-25T10:30:29.883324+0000","last_became_active":"2024-04-25T10:30:12.776773+0000","last_became_peered":"2024-04-25T10:30:12.776773+0000","last_unstale":"2024-04-25T10:30:29.883324+0000","last_undegraded":"2024-04-25T10:30:29.883324+0000","last_fullsized":"2024-04-25T10:30:29.883324+0000","mapping_epoch":34,"log_start":"0'0","ondisk_log_start":"0'0","created":34,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:30:11.733978+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:30:11.733978+0000","last_clean_scrub_stamp":"2024-04-25T10:30:11.733978+0000","objects_scrubbed":0,"log_size":10,"log_dups_size":0,"ondisk_log_size":10,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T18:21:32.178524+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"5.34","version":"36'7","reported_seq":32,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-04-25T10:30:28.874618+0000","last_change":"2024-04-25T10:30:12.761162+0000","last_active":"2024-04-25T10:30:28.874618+0000","last_peered":"2024-04-25T10:30:28.874618+0000","last_clean":"2024-04-25T10:30:28.874618+0000","last_became_active":"2024-04-25T10:30:12.760835+0000","last_became_peered":"2024-04-25T10:30:12.760835+0000","last_unstale":"2024-04-25T10:30:28.874618+0000","last_undegraded":"2024-04-25T10:30:28.874618+0000","last_fullsized":"2024-04-25T10:30:28.874618+0000","mapping_epoch":34,"log_start":"0'0","ondisk_log_start":"0'0","created":34,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:30:11.733978+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:30:11.733978+0000","last_clean_scrub_stamp":"2024-04-25T10:30:11.733978+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T18:59:20.693761+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"5.35","version":"36'6","reported_seq":35,"reported_epoch":44,"state":"active+clean","last_fresh":"2024-04-25T10:30:29.882303+0000","last_change":"2024-04-25T10:30:12.776164+0000","last_active":"2024-04-25T10:30:29.882303+0000","last_peered":"2024-04-25T10:30:29.882303+0000","last_clean":"2024-04-25T10:30:29.882303+0000","last_became_active":"2024-04-25T10:30:12.775455+0000","last_became_peered":"2024-04-25T10:30:12.775455+0000","last_unstale":"2024-04-25T10:30:29.882303+0000","last_undegraded":"2024-04-25T10:30:29.882303+0000","last_fullsized":"2024-04-25T10:30:29.882303+0000","mapping_epoch":34,"log_start":"0'0","ondisk_log_start":"0'0","created":34,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:30:11.733978+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:30:11.733978+0000","last_clean_scrub_stamp":"2024-04-25T10:30:11.733978+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T21:45:24.780979+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.36","version":"36'15","reported_seq":38,"reported_epoch":42,"state":"active+clean","last_fresh":"2024-04-25T10:30:27.877229+0000","last_change":"2024-04-25T10:30:12.773138+0000","last_active":"2024-04-25T10:30:27.877229+0000","last_peered":"2024-04-25T10:30:27.877229+0000","last_clean":"2024-04-25T10:30:27.877229+0000","last_became_active":"2024-04-25T10:30:12.772854+0000","last_became_peered":"2024-04-25T10:30:12.772854+0000","last_unstale":"2024-04-25T10:30:27.877229+0000","last_undegraded":"2024-04-25T10:30:27.877229+0000","last_fullsized":"2024-04-25T10:30:27.877229+0000","mapping_epoch":34,"log_start":"0'0","ondisk_log_start":"0'0","created":34,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:30:11.733978+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:30:11.733978+0000","last_clean_scrub_stamp":"2024-04-25T10:30:11.733978+0000","objects_scrubbed":0,"log_size":15,"log_dups_size":0,"ondisk_log_size":15,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T19:24:03.893622+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5],"acting":[4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.37","version":"36'11","reported_seq":36,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-04-25T10:30:28.875064+0000","last_change":"2024-04-25T10:30:12.773184+0000","last_active":"2024-04-25T10:30:28.875064+0000","last_peered":"2024-04-25T10:30:28.875064+0000","last_clean":"2024-04-25T10:30:28.875064+0000","last_became_active":"2024-04-25T10:30:12.772866+0000","last_became_peered":"2024-04-25T10:30:12.772866+0000","last_unstale":"2024-04-25T10:30:28.875064+0000","last_undegraded":"2024-04-25T10:30:28.875064+0000","last_fullsized":"2024-04-25T10:30:28.875064+0000","mapping_epoch":34,"log_start":"0'0","ondisk_log_start":"0'0","created":34,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:30:11.733978+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:30:11.733978+0000","last_clean_scrub_stamp":"2024-04-25T10:30:11.733978+0000","objects_scrubbed":0,"log_size":11,"log_dups_size":0,"ondisk_log_size":11,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T14:01:06.791796+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"5.38","version":"36'9","reported_seq":38,"reported_epoch":44,"state":"active+clean","last_fresh":"2024-04-25T10:30:29.882611+0000","last_change":"2024-04-25T10:30:12.774614+0000","last_active":"2024-04-25T10:30:29.882611+0000","last_peered":"2024-04-25T10:30:29.882611+0000","last_clean":"2024-04-25T10:30:29.882611+0000","last_became_active":"2024-04-25T10:30:12.774213+0000","last_became_peered":"2024-04-25T10:30:12.774213+0000","last_unstale":"2024-04-25T10:30:29.882611+0000","last_undegraded":"2024-04-25T10:30:29.882611+0000","last_fullsized":"2024-04-25T10:30:29.882611+0000","mapping_epoch":34,"log_start":"0'0","ondisk_log_start":"0'0","created":34,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:30:11.733978+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:30:11.733978+0000","last_clean_scrub_stamp":"2024-04-25T10:30:11.733978+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T20:33:24.458243+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,6],"acting":[2,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"5.39","version":"36'5","reported_seq":34,"reported_epoch":44,"state":"active+clean","last_fresh":"2024-04-25T10:30:29.882763+0000","last_change":"2024-04-25T10:30:12.781210+0000","last_active":"2024-04-25T10:30:29.882763+0000","last_peered":"2024-04-25T10:30:29.882763+0000","last_clean":"2024-04-25T10:30:29.882763+0000","last_became_active":"2024-04-25T10:30:12.780874+0000","last_became_peered":"2024-04-25T10:30:12.780874+0000","last_unstale":"2024-04-25T10:30:29.882763+0000","last_undegraded":"2024-04-25T10:30:29.882763+0000","last_fullsized":"2024-04-25T10:30:29.882763+0000","mapping_epoch":34,"log_start":"0'0","ondisk_log_start":"0'0","created":34,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:30:11.733978+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:30:11.733978+0000","last_clean_scrub_stamp":"2024-04-25T10:30:11.733978+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T16:29:40.621322+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"5.3a","version":"36'6","reported_seq":30,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-04-25T10:30:28.874520+0000","last_change":"2024-04-25T10:30:12.775534+0000","last_active":"2024-04-25T10:30:28.874520+0000","last_peered":"2024-04-25T10:30:28.874520+0000","last_clean":"2024-04-25T10:30:28.874520+0000","last_became_active":"2024-04-25T10:30:12.775180+0000","last_became_peered":"2024-04-25T10:30:12.775180+0000","last_unstale":"2024-04-25T10:30:28.874520+0000","last_undegraded":"2024-04-25T10:30:28.874520+0000","last_fullsized":"2024-04-25T10:30:28.874520+0000","mapping_epoch":34,"log_start":"0'0","ondisk_log_start":"0'0","created":34,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:30:11.733978+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:30:11.733978+0000","last_clean_scrub_stamp":"2024-04-25T10:30:11.733978+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T18:31:49.260513+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"5.3b","version":"36'4","reported_seq":28,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-04-25T10:30:28.875655+0000","last_change":"2024-04-25T10:30:12.762141+0000","last_active":"2024-04-25T10:30:28.875655+0000","last_peered":"2024-04-25T10:30:28.875655+0000","last_clean":"2024-04-25T10:30:28.875655+0000","last_became_active":"2024-04-25T10:30:12.761805+0000","last_became_peered":"2024-04-25T10:30:12.761805+0000","last_unstale":"2024-04-25T10:30:28.875655+0000","last_undegraded":"2024-04-25T10:30:28.875655+0000","last_fullsized":"2024-04-25T10:30:28.875655+0000","mapping_epoch":34,"log_start":"0'0","ondisk_log_start":"0'0","created":34,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:30:11.733978+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:30:11.733978+0000","last_clean_scrub_stamp":"2024-04-25T10:30:11.733978+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T19:56:57.765496+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6],"acting":[5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"5.3c","version":"36'8","reported_seq":31,"reported_epoch":42,"state":"active+clean","last_fresh":"2024-04-25T10:30:27.877379+0000","last_change":"2024-04-25T10:30:12.781723+0000","last_active":"2024-04-25T10:30:27.877379+0000","last_peered":"2024-04-25T10:30:27.877379+0000","last_clean":"2024-04-25T10:30:27.877379+0000","last_became_active":"2024-04-25T10:30:12.781357+0000","last_became_peered":"2024-04-25T10:30:12.781357+0000","last_unstale":"2024-04-25T10:30:27.877379+0000","last_undegraded":"2024-04-25T10:30:27.877379+0000","last_fullsized":"2024-04-25T10:30:27.877379+0000","mapping_epoch":34,"log_start":"0'0","ondisk_log_start":"0'0","created":34,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:30:11.733978+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:30:11.733978+0000","last_clean_scrub_stamp":"2024-04-25T10:30:11.733978+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T12:06:29.897651+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.3d","version":"36'7","reported_seq":32,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-04-25T10:30:28.875135+0000","last_change":"2024-04-25T10:30:12.768292+0000","last_active":"2024-04-25T10:30:28.875135+0000","last_peered":"2024-04-25T10:30:28.875135+0000","last_clean":"2024-04-25T10:30:28.875135+0000","last_became_active":"2024-04-25T10:30:12.767909+0000","last_became_peered":"2024-04-25T10:30:12.767909+0000","last_unstale":"2024-04-25T10:30:28.875135+0000","last_undegraded":"2024-04-25T10:30:28.875135+0000","last_fullsized":"2024-04-25T10:30:28.875135+0000","mapping_epoch":34,"log_start":"0'0","ondisk_log_start":"0'0","created":34,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:30:11.733978+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:30:11.733978+0000","last_clean_scrub_stamp":"2024-04-25T10:30:11.733978+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T15:17:30.002665+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"5.3e","version":"36'4","reported_seq":28,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-04-25T10:30:28.878982+0000","last_change":"2024-04-25T10:30:12.780210+0000","last_active":"2024-04-25T10:30:28.878982+0000","last_peered":"2024-04-25T10:30:28.878982+0000","last_clean":"2024-04-25T10:30:28.878982+0000","last_became_active":"2024-04-25T10:30:12.779867+0000","last_became_peered":"2024-04-25T10:30:12.779867+0000","last_unstale":"2024-04-25T10:30:28.878982+0000","last_undegraded":"2024-04-25T10:30:28.878982+0000","last_fullsized":"2024-04-25T10:30:28.878982+0000","mapping_epoch":34,"log_start":"0'0","ondisk_log_start":"0'0","created":34,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:30:11.733978+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:30:11.733978+0000","last_clean_scrub_stamp":"2024-04-25T10:30:11.733978+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T20:33:24.458243+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"5.3f","version":"36'8","reported_seq":37,"reported_epoch":44,"state":"active+clean","last_fresh":"2024-04-25T10:30:29.883448+0000","last_change":"2024-04-25T10:30:12.772509+0000","last_active":"2024-04-25T10:30:29.883448+0000","last_peered":"2024-04-25T10:30:29.883448+0000","last_clean":"2024-04-25T10:30:29.883448+0000","last_became_active":"2024-04-25T10:30:12.772018+0000","last_became_peered":"2024-04-25T10:30:12.772018+0000","last_unstale":"2024-04-25T10:30:29.883448+0000","last_undegraded":"2024-04-25T10:30:29.883448+0000","last_fullsized":"2024-04-25T10:30:29.883448+0000","mapping_epoch":34,"log_start":"0'0","ondisk_log_start":"0'0","created":34,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:30:11.733978+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:30:11.733978+0000","last_clean_scrub_stamp":"2024-04-25T10:30:11.733978+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T11:57:29.979255+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]}],"pool_stats":[{"poolid":6,"num_pg":32,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":64,"acting":64,"num_store_stats":8},{"poolid":5,"num_pg":64,"stat_sum":{"num_bytes":6291456,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":6144,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":12582912,"data_stored":12582912,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":504,"ondisk_log_size":504,"up":128,"acting":128,"num_store_stats":8},{"poolid":4,"num_pg":32,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":1,"num_read_kb":0,"num_write":96,"num_write_kb":40,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":83,"ondisk_log_size":83,"up":64,"acting":64,"num_store_stats":8},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":66,"num_read_kb":55,"num_write":101,"num_write_kb":1628,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1187840,"data_stored":1180736,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":72,"ondisk_log_size":72,"up":2,"acting":2,"num_store_stats":2}],"osd_stats":[{"osd":7,"up_from":15,"seq":64424509462,"num_pgs":21,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":31596,"kb_used_data":4572,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94340244,"statfs":{"total":96636764160,"available":96604409856,"internally_reserved":0,"allocated":4681728,"data_stored":4394877,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1590,"internal_metadata":27654602},"hb_peers":[0,1,2,3,4,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]},{"osd":6,"up_from":15,"seq":64424509463,"num_pgs":25,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27504,"kb_used_data":476,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344336,"statfs":{"total":96636764160,"available":96608600064,"internally_reserved":0,"allocated":487424,"data_stored":200573,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":27654598},"hb_peers":[0,1,2,3,4,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]},{"osd":5,"up_from":15,"seq":64424509464,"num_pgs":18,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":28084,"kb_used_data":1056,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94343756,"statfs":{"total":96636764160,"available":96608006144,"internally_reserved":0,"allocated":1081344,"data_stored":790941,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]},{"osd":3,"up_from":15,"seq":64424509464,"num_pgs":25,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":33644,"kb_used_data":6620,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94338196,"statfs":{"total":96636764160,"available":96602312704,"internally_reserved":0,"allocated":6778880,"data_stored":6492029,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1593,"internal_metadata":27654599},"hb_peers":[0,1,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]},{"osd":2,"up_from":15,"seq":64424509464,"num_pgs":27,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":29548,"kb_used_data":2524,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94342292,"statfs":{"total":96636764160,"available":96606507008,"internally_reserved":0,"allocated":2584576,"data_stored":2297725,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]},{"osd":1,"up_from":15,"seq":64424509462,"num_pgs":25,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":28088,"kb_used_data":1056,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94343752,"statfs":{"total":96636764160,"available":96608002048,"internally_reserved":0,"allocated":1081344,"data_stored":790941,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]},{"osd":0,"up_from":15,"seq":64424509463,"num_pgs":21,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27504,"kb_used_data":476,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344336,"statfs":{"total":96636764160,"available":96608600064,"internally_reserved":0,"allocated":487424,"data_stored":200573,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[1,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]},{"osd":4,"up_from":13,"seq":55834574870,"num_pgs":35,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27480,"kb_used_data":452,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344360,"statfs":{"total":96636764160,"available":96608624640,"internally_reserved":0,"allocated":462848,"data_stored":186484,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":2097152,"data_stored":2097152,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":6291456,"data_stored":6291456,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":4194304,"data_stored":4194304,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-04-25T10:30:33.694 INFO:tasks.cephfs.filesystem.ceph_manager:PG 6.b is not active+clean 2024-04-25T10:30:33.694 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '6.b', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 42, 'state': 'creating+peering', 'last_fresh': '2024-04-25T10:30:27.876378+0000', 'last_change': '2024-04-25T10:30:27.876378+0000', 'last_active': '2024-04-25T10:30:27.860024+0000', 'last_peered': '2024-04-25T10:30:27.860024+0000', 'last_clean': '2024-04-25T10:30:27.860024+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-04-25T10:30:27.876378+0000', 'last_undegraded': '2024-04-25T10:30:27.876378+0000', 'last_fullsized': '2024-04-25T10:30:27.876378+0000', 'mapping_epoch': 42, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 42, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-04-25T10:30:27.860024+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-04-25T10:30:27.860024+0000', 'last_clean_scrub_stamp': '2024-04-25T10:30:27.860024+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [4, 1], 'acting': [4, 1], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 4, 'acting_primary': 4, 'purged_snaps': []} 2024-04-25T10:30:33.694 INFO:tasks.cephfs.filesystem.ceph_manager:PG 6.9 is not active+clean 2024-04-25T10:30:33.694 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '6.9', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 42, 'state': 'creating+peering', 'last_fresh': '2024-04-25T10:30:27.871775+0000', 'last_change': '2024-04-25T10:30:27.871775+0000', 'last_active': '2024-04-25T10:30:27.860024+0000', 'last_peered': '2024-04-25T10:30:27.860024+0000', 'last_clean': '2024-04-25T10:30:27.860024+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-04-25T10:30:27.871775+0000', 'last_undegraded': '2024-04-25T10:30:27.871775+0000', 'last_fullsized': '2024-04-25T10:30:27.871775+0000', 'mapping_epoch': 42, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 42, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-04-25T10:30:27.860024+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-04-25T10:30:27.860024+0000', 'last_clean_scrub_stamp': '2024-04-25T10:30:27.860024+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [4, 1], 'acting': [4, 1], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 4, 'acting_primary': 4, 'purged_snaps': []} 2024-04-25T10:30:33.694 INFO:tasks.cephfs.filesystem.ceph_manager:PG 6.10 is not active+clean 2024-04-25T10:30:33.694 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '6.10', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 42, 'state': 'creating+peering', 'last_fresh': '2024-04-25T10:30:27.875123+0000', 'last_change': '2024-04-25T10:30:27.875123+0000', 'last_active': '2024-04-25T10:30:27.860024+0000', 'last_peered': '2024-04-25T10:30:27.860024+0000', 'last_clean': '2024-04-25T10:30:27.860024+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-04-25T10:30:27.875123+0000', 'last_undegraded': '2024-04-25T10:30:27.875123+0000', 'last_fullsized': '2024-04-25T10:30:27.875123+0000', 'mapping_epoch': 42, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 42, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-04-25T10:30:27.860024+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-04-25T10:30:27.860024+0000', 'last_clean_scrub_stamp': '2024-04-25T10:30:27.860024+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [4, 2], 'acting': [4, 2], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 4, 'acting_primary': 4, 'purged_snaps': []} 2024-04-25T10:30:33.695 INFO:tasks.cephfs.filesystem.ceph_manager:PG 6.16 is not active+clean 2024-04-25T10:30:33.695 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '6.16', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 42, 'state': 'creating+peering', 'last_fresh': '2024-04-25T10:30:27.876799+0000', 'last_change': '2024-04-25T10:30:27.876799+0000', 'last_active': '2024-04-25T10:30:27.860024+0000', 'last_peered': '2024-04-25T10:30:27.860024+0000', 'last_clean': '2024-04-25T10:30:27.860024+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-04-25T10:30:27.876799+0000', 'last_undegraded': '2024-04-25T10:30:27.876799+0000', 'last_fullsized': '2024-04-25T10:30:27.876799+0000', 'mapping_epoch': 42, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 42, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-04-25T10:30:27.860024+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-04-25T10:30:27.860024+0000', 'last_clean_scrub_stamp': '2024-04-25T10:30:27.860024+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [4, 1], 'acting': [4, 1], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 4, 'acting_primary': 4, 'purged_snaps': []} 2024-04-25T10:30:33.695 INFO:tasks.cephfs.filesystem.ceph_manager:PG 6.14 is not active+clean 2024-04-25T10:30:33.695 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '6.14', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 42, 'state': 'creating+peering', 'last_fresh': '2024-04-25T10:30:27.876893+0000', 'last_change': '2024-04-25T10:30:27.876893+0000', 'last_active': '2024-04-25T10:30:27.860024+0000', 'last_peered': '2024-04-25T10:30:27.860024+0000', 'last_clean': '2024-04-25T10:30:27.860024+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-04-25T10:30:27.876893+0000', 'last_undegraded': '2024-04-25T10:30:27.876893+0000', 'last_fullsized': '2024-04-25T10:30:27.876893+0000', 'mapping_epoch': 42, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 42, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-04-25T10:30:27.860024+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-04-25T10:30:27.860024+0000', 'last_clean_scrub_stamp': '2024-04-25T10:30:27.860024+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [4, 3], 'acting': [4, 3], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 4, 'acting_primary': 4, 'purged_snaps': []} 2024-04-25T10:30:33.695 INFO:tasks.cephfs.filesystem.ceph_manager:PG 6.18 is not active+clean 2024-04-25T10:30:33.695 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '6.18', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 42, 'state': 'creating+peering', 'last_fresh': '2024-04-25T10:30:27.874480+0000', 'last_change': '2024-04-25T10:30:27.874480+0000', 'last_active': '2024-04-25T10:30:27.860024+0000', 'last_peered': '2024-04-25T10:30:27.860024+0000', 'last_clean': '2024-04-25T10:30:27.860024+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-04-25T10:30:27.874480+0000', 'last_undegraded': '2024-04-25T10:30:27.874480+0000', 'last_fullsized': '2024-04-25T10:30:27.874480+0000', 'mapping_epoch': 42, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 42, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-04-25T10:30:27.860024+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-04-25T10:30:27.860024+0000', 'last_clean_scrub_stamp': '2024-04-25T10:30:27.860024+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [4, 5], 'acting': [4, 5], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 4, 'acting_primary': 4, 'purged_snaps': []} 2024-04-25T10:30:33.695 DEBUG:teuthology.orchestra.run.smithi037:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph status --format=json 2024-04-25T10:30:33.785 INFO:tasks.ceph.mon.a.smithi037.stderr:2024-04-25T10:30:33.783+0000 7fb253d60640 -1 log_channel(cluster) log [ERR] : Health check update: 2 filesystems are offline (MDS_ALL_DOWN) 2024-04-25T10:30:33.841 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:33.838+0000 7fde1a892640 1 Processor -- start 2024-04-25T10:30:33.841 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:33.838+0000 7fde1a892640 1 -- start start 2024-04-25T10:30:33.843 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:33.840+0000 7fde1a892640 1 --2- >> v2:172.21.15.203:3300/0 conn(0x7fde140d1ec0 0x7fde140d22d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:30:33.843 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:33.840+0000 7fde1a892640 1 --2- >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7fde140d2810 0x7fde140b3c80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:30:33.843 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:33.840+0000 7fde1a892640 1 -- --> v1:172.21.15.37:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fde1409ce70 con 0x7fde1406e260 2024-04-25T10:30:33.843 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:33.840+0000 7fde1a892640 1 -- --> v2:172.21.15.203:3300/0 -- mon_getmap magic: 0 -- 0x7fde140d3370 con 0x7fde140d1ec0 2024-04-25T10:30:33.843 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:33.840+0000 7fde1a892640 1 -- --> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] -- mon_getmap magic: 0 -- 0x7fde140d3090 con 0x7fde140d2810 2024-04-25T10:30:33.843 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:33.840+0000 7fde13fff640 1 --1- >> v1:172.21.15.37:6789/0 conn(0x7fde1406e260 0x7fde1406e670 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.37:6789/0 says I am v1:172.21.15.37:40912/0 (socket says 172.21.15.37:40912) 2024-04-25T10:30:33.843 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:33.840+0000 7fde13fff640 1 -- 172.21.15.37:0/2981168494 learned_addr learned my addr 172.21.15.37:0/2981168494 (peer_addr_for_me v1:172.21.15.37:0/0) 2024-04-25T10:30:33.843 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:33.840+0000 7fde18e2f640 1 --2- 172.21.15.37:0/2981168494 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7fde140d2810 0x7fde140b3c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:30:33.843 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:33.840+0000 7fde137fe640 1 --2- 172.21.15.37:0/2981168494 >> v2:172.21.15.203:3300/0 conn(0x7fde140d1ec0 0x7fde140d22d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:30:33.843 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:33.840+0000 7fde12ffd640 1 -- 172.21.15.37:0/2981168494 <== mon.0 v1:172.21.15.37:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 2559298429 0 0) 0x7fde1409ce70 con 0x7fde1406e260 2024-04-25T10:30:33.844 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:33.840+0000 7fde12ffd640 1 -- 172.21.15.37:0/2981168494 --> v1:172.21.15.37:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7fddfc003520 con 0x7fde1406e260 2024-04-25T10:30:33.844 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:33.840+0000 7fde137fe640 1 -- 172.21.15.37:0/2981168494 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7fde140d2810 msgr2=0x7fde140b3c80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:30:33.844 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:33.840+0000 7fde137fe640 1 --2- 172.21.15.37:0/2981168494 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7fde140d2810 0x7fde140b3c80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:30:33.844 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:33.840+0000 7fde137fe640 1 -- 172.21.15.37:0/2981168494 >> v1:172.21.15.37:6789/0 conn(0x7fde1406e260 legacy=0x7fde1406e670 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:30:33.844 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:33.840+0000 7fde137fe640 1 -- 172.21.15.37:0/2981168494 --> v2:172.21.15.203:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fde1409d150 con 0x7fde140d1ec0 2024-04-25T10:30:33.844 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:33.840+0000 7fde18e2f640 1 --2- 172.21.15.37:0/2981168494 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7fde140d2810 0x7fde140b3c80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-25T10:30:33.845 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:33.841+0000 7fde137fe640 1 --2- 172.21.15.37:0/2981168494 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fde140d1ec0 0x7fde140d22d0 secure :-1 s=READY pgs=209 cs=0 l=1 rev1=1 crypto rx=0x7fde040097a0 tx=0x7fde0402a100 comp rx=0 tx=0).ready entity=mon.1 client_cookie=f3fe9d23d206a4d2 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:30:33.845 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:33.842+0000 7fde12ffd640 1 -- 172.21.15.37:0/2981168494 <== mon.1 v2:172.21.15.203:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fde04037070 con 0x7fde140d1ec0 2024-04-25T10:30:33.845 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:33.842+0000 7fde12ffd640 1 -- 172.21.15.37:0/2981168494 <== mon.1 v2:172.21.15.203:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fde0402ae50 con 0x7fde140d1ec0 2024-04-25T10:30:33.845 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:33.842+0000 7fde12ffd640 1 -- 172.21.15.37:0/2981168494 <== mon.1 v2:172.21.15.203:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fde04032660 con 0x7fde140d1ec0 2024-04-25T10:30:33.846 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:33.842+0000 7fde1a892640 1 -- 172.21.15.37:0/2981168494 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fde140d1ec0 msgr2=0x7fde140d22d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:30:33.846 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:33.842+0000 7fde1a892640 1 --2- 172.21.15.37:0/2981168494 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fde140d1ec0 0x7fde140d22d0 secure :-1 s=READY pgs=209 cs=0 l=1 rev1=1 crypto rx=0x7fde040097a0 tx=0x7fde0402a100 comp rx=0 tx=0).stop 2024-04-25T10:30:33.846 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:33.842+0000 7fde1a892640 1 -- 172.21.15.37:0/2981168494 shutdown_connections 2024-04-25T10:30:33.846 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:33.842+0000 7fde1a892640 1 --2- 172.21.15.37:0/2981168494 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7fde140d2810 0x7fde140b3c80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:30:33.846 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:33.842+0000 7fde1a892640 1 --2- 172.21.15.37:0/2981168494 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fde140d1ec0 0x7fde140d22d0 unknown :-1 s=CLOSED pgs=209 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:30:33.846 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:33.842+0000 7fde1a892640 1 -- 172.21.15.37:0/2981168494 >> 172.21.15.37:0/2981168494 conn(0x7fde140d0390 msgr2=0x7fde140d07a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:30:33.846 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:33.842+0000 7fde1a892640 1 -- 172.21.15.37:0/2981168494 shutdown_connections 2024-04-25T10:30:33.846 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:33.842+0000 7fde1a892640 1 -- 172.21.15.37:0/2981168494 wait complete. 2024-04-25T10:30:33.847 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:33.844+0000 7fde1a892640 1 Processor -- start 2024-04-25T10:30:33.847 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:33.844+0000 7fde1a892640 1 -- start start 2024-04-25T10:30:33.849 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:33.845+0000 7fde1a892640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fde1406e260 0x7fde140be0d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:30:33.849 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:33.845+0000 7fde1a892640 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fde140d1ec0 0x7fde141d5b00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:30:33.849 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:33.845+0000 7fde1a892640 1 --2- >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7fde140d2810 0x7fde141eac00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:30:33.849 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:33.845+0000 7fde1a892640 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 -- 0x7fde140b9000 con 0x7fde1406e260 2024-04-25T10:30:33.849 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:33.846+0000 7fde1a892640 1 -- --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_getmap magic: 0 -- 0x7fde1406d830 con 0x7fde140d1ec0 2024-04-25T10:30:33.849 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:33.846+0000 7fde1a892640 1 -- --> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] -- mon_getmap magic: 0 -- 0x7fde140b9180 con 0x7fde140d2810 2024-04-25T10:30:33.849 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:33.846+0000 7fde13fff640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fde1406e260 0x7fde140be0d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:30:33.849 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:33.846+0000 7fde13fff640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fde1406e260 0x7fde140be0d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.37:3300/0 says I am v2:172.21.15.37:55470/0 (socket says 172.21.15.37:55470) 2024-04-25T10:30:33.849 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:33.846+0000 7fde13fff640 1 -- 172.21.15.37:0/4065014849 learned_addr learned my addr 172.21.15.37:0/4065014849 (peer_addr_for_me v2:172.21.15.37:0/0) 2024-04-25T10:30:33.850 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:33.846+0000 7fde137fe640 1 --2- 172.21.15.37:0/4065014849 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fde140d1ec0 0x7fde141d5b00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:30:33.850 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:33.846+0000 7fde18e2f640 1 --2- 172.21.15.37:0/4065014849 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7fde140d2810 0x7fde141eac00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:30:33.850 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:33.846+0000 7fde13fff640 1 -- 172.21.15.37:0/4065014849 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7fde140d2810 msgr2=0x7fde141eac00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:30:33.850 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:33.846+0000 7fde13fff640 1 --2- 172.21.15.37:0/4065014849 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7fde140d2810 0x7fde141eac00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:30:33.850 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:33.846+0000 7fde13fff640 1 -- 172.21.15.37:0/4065014849 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fde140d1ec0 msgr2=0x7fde141d5b00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:30:33.850 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:33.846+0000 7fde13fff640 1 --2- 172.21.15.37:0/4065014849 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fde140d1ec0 0x7fde141d5b00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:30:33.850 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:33.846+0000 7fde13fff640 1 -- 172.21.15.37:0/4065014849 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fde141d6450 con 0x7fde1406e260 2024-04-25T10:30:33.850 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:33.846+0000 7fde13fff640 1 --2- 172.21.15.37:0/4065014849 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fde1406e260 0x7fde140be0d0 secure :-1 s=READY pgs=537 cs=0 l=1 rev1=1 crypto rx=0x7fde0800c390 tx=0x7fde0800c850 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:30:33.850 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:33.846+0000 7fde10ff9640 1 -- 172.21.15.37:0/4065014849 <== mon.0 v2:172.21.15.37:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fde08017070 con 0x7fde1406e260 2024-04-25T10:30:33.850 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:33.846+0000 7fde10ff9640 1 -- 172.21.15.37:0/4065014849 <== mon.0 v2:172.21.15.37:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fde08005d00 con 0x7fde1406e260 2024-04-25T10:30:33.850 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:33.846+0000 7fde10ff9640 1 -- 172.21.15.37:0/4065014849 <== mon.0 v2:172.21.15.37:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fde08007310 con 0x7fde1406e260 2024-04-25T10:30:33.850 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:33.846+0000 7fde1a892640 1 -- 172.21.15.37:0/4065014849 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fde141d6730 con 0x7fde1406e260 2024-04-25T10:30:33.850 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:33.847+0000 7fde1a892640 1 -- 172.21.15.37:0/4065014849 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fde1409ea80 con 0x7fde1406e260 2024-04-25T10:30:33.851 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:33.848+0000 7fde1a892640 1 -- 172.21.15.37:0/4065014849 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fde140b9810 con 0x7fde1406e260 2024-04-25T10:30:33.851 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:33.848+0000 7fde10ff9640 1 -- 172.21.15.37:0/4065014849 <== mon.0 v2:172.21.15.37:3300/0 4 ==== mgrmap(e 5) ==== 96782+0+0 (secure 0 0 0) 0x7fde0801a070 con 0x7fde1406e260 2024-04-25T10:30:33.852 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:33.849+0000 7fde10ff9640 1 --2- 172.21.15.37:0/4065014849 >> [v2:172.21.15.203:6832/2418205237,v1:172.21.15.203:6833/2418205237] conn(0x7fde000744b0 0x7fde000769c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:30:33.852 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:33.849+0000 7fde10ff9640 1 -- 172.21.15.37:0/4065014849 <== mon.0 v2:172.21.15.37:3300/0 5 ==== osd_map(44..44 src has 1..44) ==== 6248+0+0 (secure 0 0 0) 0x7fde08094850 con 0x7fde1406e260 2024-04-25T10:30:33.852 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:33.849+0000 7fde137fe640 1 --2- 172.21.15.37:0/4065014849 >> [v2:172.21.15.203:6832/2418205237,v1:172.21.15.203:6833/2418205237] conn(0x7fde000744b0 0x7fde000769c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:30:33.852 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:33.849+0000 7fde137fe640 1 --2- 172.21.15.37:0/4065014849 >> [v2:172.21.15.203:6832/2418205237,v1:172.21.15.203:6833/2418205237] conn(0x7fde000744b0 0x7fde000769c0 secure :-1 s=READY pgs=257 cs=0 l=1 rev1=1 crypto rx=0x7fde04034000 tx=0x7fde0402e000 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:30:33.855 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:33.853+0000 7fde10ff9640 1 -- 172.21.15.37:0/4065014849 <== mon.0 v2:172.21.15.37:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187246 (secure 0 0 0) 0x7fde140b9810 con 0x7fde1406e260 2024-04-25T10:30:34.111 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:34.109+0000 7fde1a892640 1 -- 172.21.15.37:0/4065014849 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_command({"prefix": "status", "format": "json"} v 0) -- 0x7fde1406a140 con 0x7fde1406e260 2024-04-25T10:30:34.112 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:34.110+0000 7fde10ff9640 1 -- 172.21.15.37:0/4065014849 <== mon.0 v2:172.21.15.37:3300/0 7 ==== mon_command_ack([{"prefix": "status", "format": "json"}]=0 v0) ==== 72+0+1459 (secure 0 0 0) 0x7fde0805edb0 con 0x7fde1406e260 2024-04-25T10:30:34.112 INFO:teuthology.orchestra.run.smithi037.stdout: 2024-04-25T10:30:34.112 INFO:teuthology.orchestra.run.smithi037.stdout:{"fsid":"f4acda28-be34-492a-b286-994afd36f61d","health":{"status":"HEALTH_ERR","checks":{"FS_DEGRADED":{"severity":"HEALTH_WARN","summary":{"message":"2 filesystems are degraded","count":2},"muted":false},"MDS_ALL_DOWN":{"severity":"HEALTH_ERR","summary":{"message":"2 filesystems are offline","count":2},"muted":false}},"mutes":[]},"election_epoch":4,"quorum":[0,1,2],"quorum_names":["a","b","c"],"quorum_age":110,"monmap":{"epoch":1,"min_mon_release_name":"squid","num_mons":3},"osdmap":{"epoch":44,"num_osds":8,"num_up_osds":8,"osd_up_since":1714040934,"num_in_osds":8,"osd_in_since":1714040926,"num_remapped_pgs":0},"pgmap":{"pgs_by_state":[{"state_name":"active+clean","count":123},{"state_name":"creating+peering","count":6}],"num_pgs":129,"num_pools":4,"num_objects":4,"data_bytes":6881824,"bytes_used":239050752,"bytes_avail":772855062528,"bytes_total":773094113280,"inactive_pgs_ratio":0.046511627733707428,"write_bytes_sec":0,"read_op_per_sec":0,"write_op_per_sec":3},"fsmap":{"epoch":22,"id":2,"up":0,"in":1,"max":1,"id":3,"up":0,"in":1,"max":1,"by_rank":[],"up:standby":4,"failed":2},"mgrmap":{"available":true,"num_standbys":1,"modules":["iostat","nfs","restful"],"services":{}},"servicemap":{"epoch":6,"modified":"2024-04-25T10:30:04.201662+0000","services":{"mds":{"daemons":{"summary":"","d":{"start_epoch":0,"start_stamp":"0.000000","gid":0,"addr":"(unrecognized address family 0)/0","metadata":{},"task_status":{}}}}}},"progress_events":{}} 2024-04-25T10:30:34.114 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:34.111+0000 7fde1a892640 1 -- 172.21.15.37:0/4065014849 >> [v2:172.21.15.203:6832/2418205237,v1:172.21.15.203:6833/2418205237] conn(0x7fde000744b0 msgr2=0x7fde000769c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:30:34.114 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:34.111+0000 7fde1a892640 1 --2- 172.21.15.37:0/4065014849 >> [v2:172.21.15.203:6832/2418205237,v1:172.21.15.203:6833/2418205237] conn(0x7fde000744b0 0x7fde000769c0 secure :-1 s=READY pgs=257 cs=0 l=1 rev1=1 crypto rx=0x7fde04034000 tx=0x7fde0402e000 comp rx=0 tx=0).stop 2024-04-25T10:30:34.114 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:34.111+0000 7fde1a892640 1 -- 172.21.15.37:0/4065014849 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fde1406e260 msgr2=0x7fde140be0d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:30:34.114 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:34.111+0000 7fde1a892640 1 --2- 172.21.15.37:0/4065014849 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fde1406e260 0x7fde140be0d0 secure :-1 s=READY pgs=537 cs=0 l=1 rev1=1 crypto rx=0x7fde0800c390 tx=0x7fde0800c850 comp rx=0 tx=0).stop 2024-04-25T10:30:34.114 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:34.111+0000 7fde1a892640 1 -- 172.21.15.37:0/4065014849 shutdown_connections 2024-04-25T10:30:34.114 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:34.111+0000 7fde1a892640 1 --2- 172.21.15.37:0/4065014849 >> [v2:172.21.15.203:6832/2418205237,v1:172.21.15.203:6833/2418205237] conn(0x7fde000744b0 0x7fde000769c0 unknown :-1 s=CLOSED pgs=257 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:30:34.114 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:34.111+0000 7fde1a892640 1 --2- 172.21.15.37:0/4065014849 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7fde140d2810 0x7fde141eac00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:30:34.114 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:34.111+0000 7fde1a892640 1 --2- 172.21.15.37:0/4065014849 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fde140d1ec0 0x7fde141d5b00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:30:34.114 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:34.111+0000 7fde1a892640 1 --2- 172.21.15.37:0/4065014849 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fde1406e260 0x7fde140be0d0 unknown :-1 s=CLOSED pgs=537 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:30:34.114 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:34.111+0000 7fde1a892640 1 -- 172.21.15.37:0/4065014849 >> 172.21.15.37:0/4065014849 conn(0x7fde140d0390 msgr2=0x7fde140728e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:30:34.115 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:34.112+0000 7fde1a892640 1 -- 172.21.15.37:0/4065014849 shutdown_connections 2024-04-25T10:30:34.115 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:34.112+0000 7fde1a892640 1 -- 172.21.15.37:0/4065014849 wait complete. 2024-04-25T10:30:34.125 INFO:tasks.cephfs.filesystem.ceph_manager:no progress seen, keeping timeout for now 2024-04-25T10:30:34.125 DEBUG:teuthology.orchestra.run.smithi037:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-04-25T10:30:34.237 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:34.234+0000 7f15ce4ca640 1 Processor -- start 2024-04-25T10:30:34.237 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:34.235+0000 7f15ce4ca640 1 -- start start 2024-04-25T10:30:34.240 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:34.236+0000 7f15ce4ca640 1 --2- >> v2:172.21.15.203:3300/0 conn(0x7f15c806e260 0x7f15c806e670 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:30:34.240 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:34.236+0000 7f15ce4ca640 1 --2- >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f15c806ebb0 0x7f15c81645d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:30:34.240 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:34.236+0000 7f15ce4ca640 1 --2- >> v2:172.21.15.37:3300/0 conn(0x7f15c8164b10 0x7f15c8164f20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:30:34.240 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:34.236+0000 7f15ce4ca640 1 -- --> v2:172.21.15.37:3300/0 -- mon_getmap magic: 0 -- 0x7f15c8165460 con 0x7f15c8164b10 2024-04-25T10:30:34.240 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:34.236+0000 7f15ce4ca640 1 -- --> v2:172.21.15.203:3300/0 -- mon_getmap magic: 0 -- 0x7f15c80d3370 con 0x7f15c806e260 2024-04-25T10:30:34.240 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:34.236+0000 7f15ce4ca640 1 -- --> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] -- mon_getmap magic: 0 -- 0x7f15c80d3090 con 0x7f15c806ebb0 2024-04-25T10:30:34.240 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:34.236+0000 7f15cca67640 1 --2- >> v2:172.21.15.37:3300/0 conn(0x7f15c8164b10 0x7f15c8164f20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:30:34.240 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:34.236+0000 7f15cca67640 1 --2- >> v2:172.21.15.37:3300/0 conn(0x7f15c8164b10 0x7f15c8164f20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.37:3300/0 says I am v2:172.21.15.37:55484/0 (socket says 172.21.15.37:55484) 2024-04-25T10:30:34.240 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:34.236+0000 7f15cca67640 1 -- 172.21.15.37:0/1297387796 learned_addr learned my addr 172.21.15.37:0/1297387796 (peer_addr_for_me v2:172.21.15.37:0/0) 2024-04-25T10:30:34.240 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:34.236+0000 7f15c77fe640 1 --2- 172.21.15.37:0/1297387796 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f15c806ebb0 0x7f15c81645d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:30:34.240 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:34.236+0000 7f15c7fff640 1 --2- 172.21.15.37:0/1297387796 >> v2:172.21.15.203:3300/0 conn(0x7f15c806e260 0x7f15c806e670 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:30:34.240 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:34.236+0000 7f15cca67640 1 -- 172.21.15.37:0/1297387796 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f15c806ebb0 msgr2=0x7f15c81645d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:30:34.241 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:34.236+0000 7f15cca67640 1 --2- 172.21.15.37:0/1297387796 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f15c806ebb0 0x7f15c81645d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:30:34.241 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:34.236+0000 7f15cca67640 1 -- 172.21.15.37:0/1297387796 >> v2:172.21.15.203:3300/0 conn(0x7f15c806e260 msgr2=0x7f15c806e670 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:30:34.241 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:34.236+0000 7f15cca67640 1 --2- 172.21.15.37:0/1297387796 >> v2:172.21.15.203:3300/0 conn(0x7f15c806e260 0x7f15c806e670 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:30:34.241 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:34.236+0000 7f15cca67640 1 -- 172.21.15.37:0/1297387796 --> v2:172.21.15.37:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f15c809ce70 con 0x7f15c8164b10 2024-04-25T10:30:34.241 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:34.236+0000 7f15c7fff640 1 --2- 172.21.15.37:0/1297387796 >> v2:172.21.15.203:3300/0 conn(0x7f15c806e260 0x7f15c806e670 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T10:30:34.241 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:34.236+0000 7f15c77fe640 1 --2- 172.21.15.37:0/1297387796 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f15c806ebb0 0x7f15c81645d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T10:30:34.241 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:34.237+0000 7f15cca67640 1 --2- 172.21.15.37:0/1297387796 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f15c8164b10 0x7f15c8164f20 secure :-1 s=READY pgs=538 cs=0 l=1 rev1=1 crypto rx=0x7f15c8074460 tx=0x7f15c002a610 comp rx=0 tx=0).ready entity=mon.0 client_cookie=6faebc39b2d58fec server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:30:34.241 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:34.237+0000 7f15c6ffd640 1 -- 172.21.15.37:0/1297387796 <== mon.0 v2:172.21.15.37:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f15c0036070 con 0x7f15c8164b10 2024-04-25T10:30:34.241 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:34.237+0000 7f15c6ffd640 1 -- 172.21.15.37:0/1297387796 <== mon.0 v2:172.21.15.37:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f15c0031600 con 0x7f15c8164b10 2024-04-25T10:30:34.241 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:34.237+0000 7f15c6ffd640 1 -- 172.21.15.37:0/1297387796 <== mon.0 v2:172.21.15.37:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f15c00319d0 con 0x7f15c8164b10 2024-04-25T10:30:34.242 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:34.238+0000 7f15ce4ca640 1 -- 172.21.15.37:0/1297387796 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f15c8164b10 msgr2=0x7f15c8164f20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:30:34.242 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:34.238+0000 7f15ce4ca640 1 --2- 172.21.15.37:0/1297387796 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f15c8164b10 0x7f15c8164f20 secure :-1 s=READY pgs=538 cs=0 l=1 rev1=1 crypto rx=0x7f15c8074460 tx=0x7f15c002a610 comp rx=0 tx=0).stop 2024-04-25T10:30:34.242 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:34.238+0000 7f15ce4ca640 1 -- 172.21.15.37:0/1297387796 shutdown_connections 2024-04-25T10:30:34.242 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:34.238+0000 7f15ce4ca640 1 --2- 172.21.15.37:0/1297387796 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f15c8164b10 0x7f15c8164f20 unknown :-1 s=CLOSED pgs=538 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:30:34.243 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:34.238+0000 7f15ce4ca640 1 --2- 172.21.15.37:0/1297387796 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f15c806ebb0 0x7f15c81645d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:30:34.243 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:34.238+0000 7f15ce4ca640 1 --2- 172.21.15.37:0/1297387796 >> v2:172.21.15.203:3300/0 conn(0x7f15c806e260 0x7f15c806e670 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:30:34.243 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:34.238+0000 7f15ce4ca640 1 -- 172.21.15.37:0/1297387796 >> 172.21.15.37:0/1297387796 conn(0x7f15c80d0390 msgr2=0x7f15c80d07a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:30:34.243 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:34.238+0000 7f15ce4ca640 1 -- 172.21.15.37:0/1297387796 shutdown_connections 2024-04-25T10:30:34.243 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:34.238+0000 7f15ce4ca640 1 -- 172.21.15.37:0/1297387796 wait complete. 2024-04-25T10:30:34.243 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:34.240+0000 7f15ce4ca640 1 Processor -- start 2024-04-25T10:30:34.243 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:34.240+0000 7f15ce4ca640 1 -- start start 2024-04-25T10:30:34.245 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:34.241+0000 7f15ce4ca640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f15c806e260 0x7f15c80cec10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:30:34.245 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:34.241+0000 7f15ce4ca640 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f15c806ebb0 0x7f15c81f53a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:30:34.245 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:34.241+0000 7f15ce4ca640 1 --2- >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f15c8164b10 0x7f15c81f58e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:30:34.245 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:34.241+0000 7f15ce4ca640 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 -- 0x7f15c80b9000 con 0x7f15c806e260 2024-04-25T10:30:34.245 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:34.241+0000 7f15ce4ca640 1 -- --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_getmap magic: 0 -- 0x7f15c806d830 con 0x7f15c806ebb0 2024-04-25T10:30:34.245 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:34.241+0000 7f15ce4ca640 1 -- --> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] -- mon_getmap magic: 0 -- 0x7f15c80b9180 con 0x7f15c8164b10 2024-04-25T10:30:34.245 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:34.242+0000 7f15c7fff640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f15c806e260 0x7f15c80cec10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:30:34.245 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:34.242+0000 7f15c7fff640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f15c806e260 0x7f15c80cec10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.37:3300/0 says I am v2:172.21.15.37:55494/0 (socket says 172.21.15.37:55494) 2024-04-25T10:30:34.245 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:34.242+0000 7f15c7fff640 1 -- 172.21.15.37:0/2784911558 learned_addr learned my addr 172.21.15.37:0/2784911558 (peer_addr_for_me v2:172.21.15.37:0/0) 2024-04-25T10:30:34.245 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:34.242+0000 7f15c7fff640 1 -- 172.21.15.37:0/2784911558 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f15c8164b10 msgr2=0x7f15c81f58e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:30:34.245 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:34.242+0000 7f15c7fff640 1 --2- 172.21.15.37:0/2784911558 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f15c8164b10 0x7f15c81f58e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:30:34.246 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:34.242+0000 7f15c7fff640 1 -- 172.21.15.37:0/2784911558 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f15c806ebb0 msgr2=0x7f15c81f53a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:30:34.246 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:34.242+0000 7f15c77fe640 1 --2- 172.21.15.37:0/2784911558 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f15c806ebb0 0x7f15c81f53a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:30:34.246 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:34.242+0000 7f15c7fff640 1 --2- 172.21.15.37:0/2784911558 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f15c806ebb0 0x7f15c81f53a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:30:34.246 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:34.242+0000 7f15c7fff640 1 -- 172.21.15.37:0/2784911558 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f15c8158f10 con 0x7f15c806e260 2024-04-25T10:30:34.246 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:34.242+0000 7f15c7fff640 1 --2- 172.21.15.37:0/2784911558 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f15c806e260 0x7f15c80cec10 secure :-1 s=READY pgs=539 cs=0 l=1 rev1=1 crypto rx=0x7f15b000d800 tx=0x7f15b000dcc0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:30:34.246 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:34.242+0000 7f15c4ff9640 1 -- 172.21.15.37:0/2784911558 <== mon.0 v2:172.21.15.37:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f15b0018030 con 0x7f15c806e260 2024-04-25T10:30:34.246 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:34.242+0000 7f15c4ff9640 1 -- 172.21.15.37:0/2784911558 <== mon.0 v2:172.21.15.37:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f15b00096a0 con 0x7f15c806e260 2024-04-25T10:30:34.246 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:34.242+0000 7f15c4ff9640 1 -- 172.21.15.37:0/2784911558 <== mon.0 v2:172.21.15.37:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f15b0009870 con 0x7f15c806e260 2024-04-25T10:30:34.246 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:34.242+0000 7f15ce4ca640 1 -- 172.21.15.37:0/2784911558 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f15c80bf030 con 0x7f15c806e260 2024-04-25T10:30:34.246 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:34.242+0000 7f15ce4ca640 1 -- 172.21.15.37:0/2784911558 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f15c80bf5f0 con 0x7f15c806e260 2024-04-25T10:30:34.247 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:34.243+0000 7f15ce4ca640 1 -- 172.21.15.37:0/2784911558 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f15c806a140 con 0x7f15c806e260 2024-04-25T10:30:34.247 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:34.244+0000 7f15c4ff9640 1 -- 172.21.15.37:0/2784911558 <== mon.0 v2:172.21.15.37:3300/0 4 ==== mgrmap(e 5) ==== 96782+0+0 (secure 0 0 0) 0x7f15b001b070 con 0x7f15c806e260 2024-04-25T10:30:34.247 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:34.244+0000 7f15c4ff9640 1 --2- 172.21.15.37:0/2784911558 >> [v2:172.21.15.203:6832/2418205237,v1:172.21.15.203:6833/2418205237] conn(0x7f15a80744b0 0x7f15a80769c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:30:34.247 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:34.245+0000 7f15c4ff9640 1 -- 172.21.15.37:0/2784911558 <== mon.0 v2:172.21.15.37:3300/0 5 ==== osd_map(44..44 src has 1..44) ==== 6248+0+0 (secure 0 0 0) 0x7f15b00956a0 con 0x7f15c806e260 2024-04-25T10:30:34.247 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:34.245+0000 7f15c77fe640 1 --2- 172.21.15.37:0/2784911558 >> [v2:172.21.15.203:6832/2418205237,v1:172.21.15.203:6833/2418205237] conn(0x7f15a80744b0 0x7f15a80769c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:30:34.247 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:34.245+0000 7f15c77fe640 1 --2- 172.21.15.37:0/2784911558 >> [v2:172.21.15.203:6832/2418205237,v1:172.21.15.203:6833/2418205237] conn(0x7f15a80744b0 0x7f15a80769c0 secure :-1 s=READY pgs=258 cs=0 l=1 rev1=1 crypto rx=0x7f15b8006fd0 tx=0x7f15b8008040 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:30:34.251 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:34.249+0000 7f15c4ff9640 1 -- 172.21.15.37:0/2784911558 <== mon.0 v2:172.21.15.37:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187246 (secure 0 0 0) 0x7f15b005fc10 con 0x7f15c806e260 2024-04-25T10:30:34.431 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:34.429+0000 7f15ce4ca640 1 -- 172.21.15.37:0/2784911558 --> [v2:172.21.15.203:6832/2418205237,v1:172.21.15.203:6833/2418205237] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) -- 0x7f15c80cf150 con 0x7f15a80744b0 2024-04-25T10:30:34.439 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:34.437+0000 7f15c4ff9640 1 -- 172.21.15.37:0/2784911558 <== mgr.4100 v2:172.21.15.203:6832/2418205237 1 ==== mgr_command_reply(tid 0: 0 dumped all) ==== 18+0+332517 (secure 0 0 0) 0x7f15c80cf150 con 0x7f15a80744b0 2024-04-25T10:30:34.443 INFO:teuthology.orchestra.run.smithi037.stdout: 2024-04-25T10:30:34.445 INFO:teuthology.orchestra.run.smithi037.stderr:dumped all 2024-04-25T10:30:34.446 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:34.441+0000 7f15ce4ca640 1 -- 172.21.15.37:0/2784911558 >> [v2:172.21.15.203:6832/2418205237,v1:172.21.15.203:6833/2418205237] conn(0x7f15a80744b0 msgr2=0x7f15a80769c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:30:34.446 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:34.441+0000 7f15ce4ca640 1 --2- 172.21.15.37:0/2784911558 >> [v2:172.21.15.203:6832/2418205237,v1:172.21.15.203:6833/2418205237] conn(0x7f15a80744b0 0x7f15a80769c0 secure :-1 s=READY pgs=258 cs=0 l=1 rev1=1 crypto rx=0x7f15b8006fd0 tx=0x7f15b8008040 comp rx=0 tx=0).stop 2024-04-25T10:30:34.446 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:34.441+0000 7f15ce4ca640 1 -- 172.21.15.37:0/2784911558 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f15c806e260 msgr2=0x7f15c80cec10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:30:34.446 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:34.441+0000 7f15ce4ca640 1 --2- 172.21.15.37:0/2784911558 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f15c806e260 0x7f15c80cec10 secure :-1 s=READY pgs=539 cs=0 l=1 rev1=1 crypto rx=0x7f15b000d800 tx=0x7f15b000dcc0 comp rx=0 tx=0).stop 2024-04-25T10:30:34.447 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:34.441+0000 7f15ce4ca640 1 -- 172.21.15.37:0/2784911558 shutdown_connections 2024-04-25T10:30:34.447 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:34.441+0000 7f15ce4ca640 1 --2- 172.21.15.37:0/2784911558 >> [v2:172.21.15.203:6832/2418205237,v1:172.21.15.203:6833/2418205237] conn(0x7f15a80744b0 0x7f15a80769c0 unknown :-1 s=CLOSED pgs=258 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:30:34.447 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:34.441+0000 7f15ce4ca640 1 --2- 172.21.15.37:0/2784911558 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f15c8164b10 0x7f15c81f58e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:30:34.447 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:34.441+0000 7f15ce4ca640 1 --2- 172.21.15.37:0/2784911558 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f15c806ebb0 0x7f15c81f53a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:30:34.447 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:34.441+0000 7f15ce4ca640 1 --2- 172.21.15.37:0/2784911558 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f15c806e260 0x7f15c80cec10 unknown :-1 s=CLOSED pgs=539 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:30:34.447 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:34.441+0000 7f15ce4ca640 1 -- 172.21.15.37:0/2784911558 >> 172.21.15.37:0/2784911558 conn(0x7f15c80d0390 msgr2=0x7f15c8163fd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:30:34.447 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:34.441+0000 7f15ce4ca640 1 -- 172.21.15.37:0/2784911558 shutdown_connections 2024-04-25T10:30:34.447 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:34.441+0000 7f15ce4ca640 1 -- 172.21.15.37:0/2784911558 wait complete. 2024-04-25T10:30:34.454 INFO:teuthology.orchestra.run.smithi037.stdout:{"pg_ready":true,"pg_map":{"version":88,"stamp":"2024-04-25T10:30:34.208614+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":6881824,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":67,"num_read_kb":55,"num_write":201,"num_write_kb":7812,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":659,"ondisk_log_size":659,"up":258,"acting":258,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":207,"num_osds":8,"num_per_pool_osds":8,"num_per_pool_omap_osds":8,"kb":754974720,"kb_used":237640,"kb_used_data":17268,"kb_used_omap":12,"kb_used_meta":220211,"kb_avail":754737080,"statfs":{"total":773094113280,"available":772850769920,"internally_reserved":0,"allocated":17682432,"data_stored":15375096,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":12752,"internal_metadata":225496624},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":13,"apply_latency_ms":13,"commit_latency_ns":13000000,"apply_latency_ns":13000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":-16365,"num_objects":-23,"num_object_clones":0,"num_object_copies":-46,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":-23,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":23,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":-15,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"6.339486"},"pg_stats":[{"pgid":"1.0","version":"18'72","reported_seq":145,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-04-25T10:30:28.875202+0000","last_change":"2024-04-25T10:28:57.972696+0000","last_active":"2024-04-25T10:30:28.875202+0000","last_peered":"2024-04-25T10:30:28.875202+0000","last_clean":"2024-04-25T10:30:28.875202+0000","last_became_active":"2024-04-25T10:28:57.972373+0000","last_became_peered":"2024-04-25T10:28:57.972373+0000","last_unstale":"2024-04-25T10:30:28.875202+0000","last_undegraded":"2024-04-25T10:30:28.875202+0000","last_fullsized":"2024-04-25T10:30:28.875202+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:28:56.959439+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:28:56.959439+0000","last_clean_scrub_stamp":"2024-04-25T10:28:56.959439+0000","objects_scrubbed":0,"log_size":72,"log_dups_size":0,"ondisk_log_size":72,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T17:34:34.057408+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":66,"num_read_kb":55,"num_write":101,"num_write_kb":1628,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.5","version":"0'0","reported_seq":30,"reported_epoch":44,"state":"active+clean","last_fresh":"2024-04-25T10:30:29.882532+0000","last_change":"2024-04-25T10:30:11.759459+0000","last_active":"2024-04-25T10:30:29.882532+0000","last_peered":"2024-04-25T10:30:29.882532+0000","last_clean":"2024-04-25T10:30:29.882532+0000","last_became_active":"2024-04-25T10:30:11.757880+0000","last_became_peered":"2024-04-25T10:30:11.757880+0000","last_unstale":"2024-04-25T10:30:29.882532+0000","last_undegraded":"2024-04-25T10:30:29.882532+0000","last_fullsized":"2024-04-25T10:30:29.882532+0000","mapping_epoch":33,"log_start":"0'0","ondisk_log_start":"0'0","created":33,"last_epoch_clean":34,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:30:10.729165+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:30:10.729165+0000","last_clean_scrub_stamp":"2024-04-25T10:30:10.729165+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T18:21:31.173712+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.4","version":"36'9","reported_seq":33,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-04-25T10:30:28.874920+0000","last_change":"2024-04-25T10:30:12.773161+0000","last_active":"2024-04-25T10:30:28.874920+0000","last_peered":"2024-04-25T10:30:28.874920+0000","last_clean":"2024-04-25T10:30:28.874920+0000","last_became_active":"2024-04-25T10:30:12.772831+0000","last_became_peered":"2024-04-25T10:30:12.772831+0000","last_unstale":"2024-04-25T10:30:28.874920+0000","last_undegraded":"2024-04-25T10:30:28.874920+0000","last_fullsized":"2024-04-25T10:30:28.874920+0000","mapping_epoch":34,"log_start":"0'0","ondisk_log_start":"0'0","created":34,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:30:11.733978+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:30:11.733978+0000","last_clean_scrub_stamp":"2024-04-25T10:30:11.733978+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T19:56:57.765496+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"6.7","version":"0'0","reported_seq":12,"reported_epoch":44,"state":"active+clean","last_fresh":"2024-04-25T10:30:29.882905+0000","last_change":"2024-04-25T10:30:28.881711+0000","last_active":"2024-04-25T10:30:29.882905+0000","last_peered":"2024-04-25T10:30:29.882905+0000","last_clean":"2024-04-25T10:30:29.882905+0000","last_became_active":"2024-04-25T10:30:28.881407+0000","last_became_peered":"2024-04-25T10:30:28.881407+0000","last_unstale":"2024-04-25T10:30:29.882905+0000","last_undegraded":"2024-04-25T10:30:29.882905+0000","last_fullsized":"2024-04-25T10:30:29.882905+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":42,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:30:27.860024+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:30:27.860024+0000","last_clean_scrub_stamp":"2024-04-25T10:30:27.860024+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T16:30:29.338622+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.0","version":"0'0","reported_seq":29,"reported_epoch":44,"state":"active+clean","last_fresh":"2024-04-25T10:30:30.800360+0000","last_change":"2024-04-25T10:30:11.761764+0000","last_active":"2024-04-25T10:30:30.800360+0000","last_peered":"2024-04-25T10:30:30.800360+0000","last_clean":"2024-04-25T10:30:30.800360+0000","last_became_active":"2024-04-25T10:30:11.761281+0000","last_became_peered":"2024-04-25T10:30:11.761281+0000","last_unstale":"2024-04-25T10:30:30.800360+0000","last_undegraded":"2024-04-25T10:30:30.800360+0000","last_fullsized":"2024-04-25T10:30:30.800360+0000","mapping_epoch":33,"log_start":"0'0","ondisk_log_start":"0'0","created":33,"last_epoch_clean":34,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:30:10.729165+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:30:10.729165+0000","last_clean_scrub_stamp":"2024-04-25T10:30:10.729165+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T13:05:33.012421+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.1","version":"36'5","reported_seq":34,"reported_epoch":44,"state":"active+clean","last_fresh":"2024-04-25T10:30:29.881926+0000","last_change":"2024-04-25T10:30:12.769006+0000","last_active":"2024-04-25T10:30:29.881926+0000","last_peered":"2024-04-25T10:30:29.881926+0000","last_clean":"2024-04-25T10:30:29.881926+0000","last_became_active":"2024-04-25T10:30:12.768671+0000","last_became_peered":"2024-04-25T10:30:12.768671+0000","last_unstale":"2024-04-25T10:30:29.881926+0000","last_undegraded":"2024-04-25T10:30:29.881926+0000","last_fullsized":"2024-04-25T10:30:29.881926+0000","mapping_epoch":34,"log_start":"0'0","ondisk_log_start":"0'0","created":34,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:30:11.733978+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:30:11.733978+0000","last_clean_scrub_stamp":"2024-04-25T10:30:11.733978+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T15:00:26.818072+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"6.2","version":"0'0","reported_seq":12,"reported_epoch":44,"state":"active+clean","last_fresh":"2024-04-25T10:30:29.882577+0000","last_change":"2024-04-25T10:30:28.875554+0000","last_active":"2024-04-25T10:30:29.882577+0000","last_peered":"2024-04-25T10:30:29.882577+0000","last_clean":"2024-04-25T10:30:29.882577+0000","last_became_active":"2024-04-25T10:30:28.875249+0000","last_became_peered":"2024-04-25T10:30:28.875249+0000","last_unstale":"2024-04-25T10:30:29.882577+0000","last_undegraded":"2024-04-25T10:30:29.882577+0000","last_fullsized":"2024-04-25T10:30:29.882577+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":42,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:30:27.860024+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:30:27.860024+0000","last_clean_scrub_stamp":"2024-04-25T10:30:27.860024+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T17:20:57.234773+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.1","version":"0'0","reported_seq":30,"reported_epoch":44,"state":"active+clean","last_fresh":"2024-04-25T10:30:29.882995+0000","last_change":"2024-04-25T10:30:11.765156+0000","last_active":"2024-04-25T10:30:29.882995+0000","last_peered":"2024-04-25T10:30:29.882995+0000","last_clean":"2024-04-25T10:30:29.882995+0000","last_became_active":"2024-04-25T10:30:11.764704+0000","last_became_peered":"2024-04-25T10:30:11.764704+0000","last_unstale":"2024-04-25T10:30:29.882995+0000","last_undegraded":"2024-04-25T10:30:29.882995+0000","last_fullsized":"2024-04-25T10:30:29.882995+0000","mapping_epoch":33,"log_start":"0'0","ondisk_log_start":"0'0","created":33,"last_epoch_clean":34,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:30:10.729165+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:30:10.729165+0000","last_clean_scrub_stamp":"2024-04-25T10:30:10.729165+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T19:02:53.584992+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"5.0","version":"36'8","reported_seq":36,"reported_epoch":44,"state":"active+clean","last_fresh":"2024-04-25T10:30:30.800119+0000","last_change":"2024-04-25T10:30:12.781642+0000","last_active":"2024-04-25T10:30:30.800119+0000","last_peered":"2024-04-25T10:30:30.800119+0000","last_clean":"2024-04-25T10:30:30.800119+0000","last_became_active":"2024-04-25T10:30:12.781303+0000","last_became_peered":"2024-04-25T10:30:12.781303+0000","last_unstale":"2024-04-25T10:30:30.800119+0000","last_undegraded":"2024-04-25T10:30:30.800119+0000","last_fullsized":"2024-04-25T10:30:30.800119+0000","mapping_epoch":34,"log_start":"0'0","ondisk_log_start":"0'0","created":34,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:30:11.733978+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:30:11.733978+0000","last_clean_scrub_stamp":"2024-04-25T10:30:11.733978+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T16:14:36.620049+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"6.3","version":"0'0","reported_seq":9,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-04-25T10:30:28.885536+0000","last_change":"2024-04-25T10:30:28.883569+0000","last_active":"2024-04-25T10:30:28.885536+0000","last_peered":"2024-04-25T10:30:28.885536+0000","last_clean":"2024-04-25T10:30:28.885536+0000","last_became_active":"2024-04-25T10:30:28.883246+0000","last_became_peered":"2024-04-25T10:30:28.883246+0000","last_unstale":"2024-04-25T10:30:28.885536+0000","last_undegraded":"2024-04-25T10:30:28.885536+0000","last_fullsized":"2024-04-25T10:30:28.885536+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":42,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:30:27.860024+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:30:27.860024+0000","last_clean_scrub_stamp":"2024-04-25T10:30:27.860024+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T14:04:58.646968+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.2","version":"0'0","reported_seq":28,"reported_epoch":44,"state":"active+clean","last_fresh":"2024-04-25T10:30:29.882904+0000","last_change":"2024-04-25T10:30:11.760769+0000","last_active":"2024-04-25T10:30:29.882904+0000","last_peered":"2024-04-25T10:30:29.882904+0000","last_clean":"2024-04-25T10:30:29.882904+0000","last_became_active":"2024-04-25T10:30:11.760425+0000","last_became_peered":"2024-04-25T10:30:11.760425+0000","last_unstale":"2024-04-25T10:30:29.882904+0000","last_undegraded":"2024-04-25T10:30:29.882904+0000","last_fullsized":"2024-04-25T10:30:29.882904+0000","mapping_epoch":33,"log_start":"0'0","ondisk_log_start":"0'0","created":33,"last_epoch_clean":34,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:30:10.729165+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:30:10.729165+0000","last_clean_scrub_stamp":"2024-04-25T10:30:10.729165+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T17:24:13.848319+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"5.3","version":"36'7","reported_seq":35,"reported_epoch":44,"state":"active+clean","last_fresh":"2024-04-25T10:30:30.800298+0000","last_change":"2024-04-25T10:30:12.775959+0000","last_active":"2024-04-25T10:30:30.800298+0000","last_peered":"2024-04-25T10:30:30.800298+0000","last_clean":"2024-04-25T10:30:30.800298+0000","last_became_active":"2024-04-25T10:30:12.774532+0000","last_became_peered":"2024-04-25T10:30:12.774532+0000","last_unstale":"2024-04-25T10:30:30.800298+0000","last_undegraded":"2024-04-25T10:30:30.800298+0000","last_fullsized":"2024-04-25T10:30:30.800298+0000","mapping_epoch":34,"log_start":"0'0","ondisk_log_start":"0'0","created":34,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:30:11.733978+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:30:11.733978+0000","last_clean_scrub_stamp":"2024-04-25T10:30:11.733978+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T12:46:39.795946+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"6.0","version":"0'0","reported_seq":9,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-04-25T10:30:28.884563+0000","last_change":"2024-04-25T10:30:28.883801+0000","last_active":"2024-04-25T10:30:28.884563+0000","last_peered":"2024-04-25T10:30:28.884563+0000","last_clean":"2024-04-25T10:30:28.884563+0000","last_became_active":"2024-04-25T10:30:28.883485+0000","last_became_peered":"2024-04-25T10:30:28.883485+0000","last_unstale":"2024-04-25T10:30:28.884563+0000","last_undegraded":"2024-04-25T10:30:28.884563+0000","last_fullsized":"2024-04-25T10:30:28.884563+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":42,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:30:27.860024+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:30:27.860024+0000","last_clean_scrub_stamp":"2024-04-25T10:30:27.860024+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T14:16:58.845516+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.3","version":"0'0","reported_seq":28,"reported_epoch":44,"state":"active+clean","last_fresh":"2024-04-25T10:30:29.883455+0000","last_change":"2024-04-25T10:30:11.762286+0000","last_active":"2024-04-25T10:30:29.883455+0000","last_peered":"2024-04-25T10:30:29.883455+0000","last_clean":"2024-04-25T10:30:29.883455+0000","last_became_active":"2024-04-25T10:30:11.761975+0000","last_became_peered":"2024-04-25T10:30:11.761975+0000","last_unstale":"2024-04-25T10:30:29.883455+0000","last_undegraded":"2024-04-25T10:30:29.883455+0000","last_fullsized":"2024-04-25T10:30:29.883455+0000","mapping_epoch":33,"log_start":"0'0","ondisk_log_start":"0'0","created":33,"last_epoch_clean":34,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:30:10.729165+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:30:10.729165+0000","last_clean_scrub_stamp":"2024-04-25T10:30:10.729165+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T16:09:04.081750+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"5.2","version":"36'8","reported_seq":32,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-04-25T10:30:28.877071+0000","last_change":"2024-04-25T10:30:12.765903+0000","last_active":"2024-04-25T10:30:28.877071+0000","last_peered":"2024-04-25T10:30:28.877071+0000","last_clean":"2024-04-25T10:30:28.877071+0000","last_became_active":"2024-04-25T10:30:12.765611+0000","last_became_peered":"2024-04-25T10:30:12.765611+0000","last_unstale":"2024-04-25T10:30:28.877071+0000","last_undegraded":"2024-04-25T10:30:28.877071+0000","last_fullsized":"2024-04-25T10:30:28.877071+0000","mapping_epoch":34,"log_start":"0'0","ondisk_log_start":"0'0","created":34,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:30:11.733978+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:30:11.733978+0000","last_clean_scrub_stamp":"2024-04-25T10:30:11.733978+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T22:09:01.889527+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"6.1","version":"0'0","reported_seq":12,"reported_epoch":44,"state":"active+clean","last_fresh":"2024-04-25T10:30:29.882707+0000","last_change":"2024-04-25T10:30:28.884434+0000","last_active":"2024-04-25T10:30:29.882707+0000","last_peered":"2024-04-25T10:30:29.882707+0000","last_clean":"2024-04-25T10:30:29.882707+0000","last_became_active":"2024-04-25T10:30:28.883801+0000","last_became_peered":"2024-04-25T10:30:28.883801+0000","last_unstale":"2024-04-25T10:30:29.882707+0000","last_undegraded":"2024-04-25T10:30:29.882707+0000","last_fullsized":"2024-04-25T10:30:29.882707+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":42,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:30:27.860024+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:30:27.860024+0000","last_clean_scrub_stamp":"2024-04-25T10:30:27.860024+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T11:57:46.105302+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.4","version":"0'0","reported_seq":28,"reported_epoch":44,"state":"active+clean","last_fresh":"2024-04-25T10:30:29.883736+0000","last_change":"2024-04-25T10:30:11.757601+0000","last_active":"2024-04-25T10:30:29.883736+0000","last_peered":"2024-04-25T10:30:29.883736+0000","last_clean":"2024-04-25T10:30:29.883736+0000","last_became_active":"2024-04-25T10:30:11.756626+0000","last_became_peered":"2024-04-25T10:30:11.756626+0000","last_unstale":"2024-04-25T10:30:29.883736+0000","last_undegraded":"2024-04-25T10:30:29.883736+0000","last_fullsized":"2024-04-25T10:30:29.883736+0000","mapping_epoch":33,"log_start":"0'0","ondisk_log_start":"0'0","created":33,"last_epoch_clean":34,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:30:10.729165+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:30:10.729165+0000","last_clean_scrub_stamp":"2024-04-25T10:30:10.729165+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T13:03:08.676049+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"5.5","version":"36'5","reported_seq":29,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-04-25T10:30:28.875752+0000","last_change":"2024-04-25T10:30:12.782521+0000","last_active":"2024-04-25T10:30:28.875752+0000","last_peered":"2024-04-25T10:30:28.875752+0000","last_clean":"2024-04-25T10:30:28.875752+0000","last_became_active":"2024-04-25T10:30:12.781802+0000","last_became_peered":"2024-04-25T10:30:12.781802+0000","last_unstale":"2024-04-25T10:30:28.875752+0000","last_undegraded":"2024-04-25T10:30:28.875752+0000","last_fullsized":"2024-04-25T10:30:28.875752+0000","mapping_epoch":34,"log_start":"0'0","ondisk_log_start":"0'0","created":34,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:30:11.733978+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:30:11.733978+0000","last_clean_scrub_stamp":"2024-04-25T10:30:11.733978+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T15:17:30.002665+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"6.6","version":"0'0","reported_seq":9,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-04-25T10:30:28.886860+0000","last_change":"2024-04-25T10:30:28.885896+0000","last_active":"2024-04-25T10:30:28.886860+0000","last_peered":"2024-04-25T10:30:28.886860+0000","last_clean":"2024-04-25T10:30:28.886860+0000","last_became_active":"2024-04-25T10:30:28.885575+0000","last_became_peered":"2024-04-25T10:30:28.885575+0000","last_unstale":"2024-04-25T10:30:28.886860+0000","last_undegraded":"2024-04-25T10:30:28.886860+0000","last_fullsized":"2024-04-25T10:30:28.886860+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":42,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:30:27.860024+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:30:27.860024+0000","last_clean_scrub_stamp":"2024-04-25T10:30:27.860024+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T22:09:18.015574+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.6","version":"37'4","reported_seq":29,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-04-25T10:30:28.879222+0000","last_change":"2024-04-25T10:30:11.750312+0000","last_active":"2024-04-25T10:30:28.879222+0000","last_peered":"2024-04-25T10:30:28.879222+0000","last_clean":"2024-04-25T10:30:28.879222+0000","last_became_active":"2024-04-25T10:30:11.750024+0000","last_became_peered":"2024-04-25T10:30:11.750024+0000","last_unstale":"2024-04-25T10:30:28.879222+0000","last_undegraded":"2024-04-25T10:30:28.879222+0000","last_fullsized":"2024-04-25T10:30:28.879222+0000","mapping_epoch":33,"log_start":"0'0","ondisk_log_start":"0'0","created":33,"last_epoch_clean":34,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:30:10.729165+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:30:10.729165+0000","last_clean_scrub_stamp":"2024-04-25T10:30:10.729165+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T16:48:59.263610+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"5.7","version":"36'8","reported_seq":37,"reported_epoch":44,"state":"active+clean","last_fresh":"2024-04-25T10:30:29.882715+0000","last_change":"2024-04-25T10:30:12.768932+0000","last_active":"2024-04-25T10:30:29.882715+0000","last_peered":"2024-04-25T10:30:29.882715+0000","last_clean":"2024-04-25T10:30:29.882715+0000","last_became_active":"2024-04-25T10:30:12.768609+0000","last_became_peered":"2024-04-25T10:30:12.768609+0000","last_unstale":"2024-04-25T10:30:29.882715+0000","last_undegraded":"2024-04-25T10:30:29.882715+0000","last_fullsized":"2024-04-25T10:30:29.882715+0000","mapping_epoch":34,"log_start":"0'0","ondisk_log_start":"0'0","created":34,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:30:11.733978+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:30:11.733978+0000","last_clean_scrub_stamp":"2024-04-25T10:30:11.733978+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T11:24:05.006967+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":4194304,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":4096,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"6.4","version":"0'0","reported_seq":9,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-04-25T10:30:28.881060+0000","last_change":"2024-04-25T10:30:28.880128+0000","last_active":"2024-04-25T10:30:28.881060+0000","last_peered":"2024-04-25T10:30:28.881060+0000","last_clean":"2024-04-25T10:30:28.881060+0000","last_became_active":"2024-04-25T10:30:28.879771+0000","last_became_peered":"2024-04-25T10:30:28.879771+0000","last_unstale":"2024-04-25T10:30:28.881060+0000","last_undegraded":"2024-04-25T10:30:28.881060+0000","last_fullsized":"2024-04-25T10:30:28.881060+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":42,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:30:27.860024+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:30:27.860024+0000","last_clean_scrub_stamp":"2024-04-25T10:30:27.860024+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T11:33:01.743970+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.7","version":"37'3","reported_seq":31,"reported_epoch":44,"state":"active+clean","last_fresh":"2024-04-25T10:30:29.883302+0000","last_change":"2024-04-25T10:30:11.756948+0000","last_active":"2024-04-25T10:30:29.883302+0000","last_peered":"2024-04-25T10:30:29.883302+0000","last_clean":"2024-04-25T10:30:29.883302+0000","last_became_active":"2024-04-25T10:30:11.756607+0000","last_became_peered":"2024-04-25T10:30:11.756607+0000","last_unstale":"2024-04-25T10:30:29.883302+0000","last_undegraded":"2024-04-25T10:30:29.883302+0000","last_fullsized":"2024-04-25T10:30:29.883302+0000","mapping_epoch":33,"log_start":"0'0","ondisk_log_start":"0'0","created":33,"last_epoch_clean":34,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:30:10.729165+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:30:10.729165+0000","last_clean_scrub_stamp":"2024-04-25T10:30:10.729165+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T17:15:13.597075+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"5.6","version":"36'7","reported_seq":31,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-04-25T10:30:28.876259+0000","last_change":"2024-04-25T10:30:12.772456+0000","last_active":"2024-04-25T10:30:28.876259+0000","last_peered":"2024-04-25T10:30:28.876259+0000","last_clean":"2024-04-25T10:30:28.876259+0000","last_became_active":"2024-04-25T10:30:12.772070+0000","last_became_peered":"2024-04-25T10:30:12.772070+0000","last_unstale":"2024-04-25T10:30:28.876259+0000","last_undegraded":"2024-04-25T10:30:28.876259+0000","last_fullsized":"2024-04-25T10:30:28.876259+0000","mapping_epoch":34,"log_start":"0'0","ondisk_log_start":"0'0","created":34,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:30:11.733978+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:30:11.733978+0000","last_clean_scrub_stamp":"2024-04-25T10:30:11.733978+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T16:08:29.987819+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"6.5","version":"0'0","reported_seq":9,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-04-25T10:30:28.882547+0000","last_change":"2024-04-25T10:30:28.881260+0000","last_active":"2024-04-25T10:30:28.882547+0000","last_peered":"2024-04-25T10:30:28.882547+0000","last_clean":"2024-04-25T10:30:28.882547+0000","last_became_active":"2024-04-25T10:30:28.880995+0000","last_became_peered":"2024-04-25T10:30:28.880995+0000","last_unstale":"2024-04-25T10:30:28.882547+0000","last_undegraded":"2024-04-25T10:30:28.882547+0000","last_fullsized":"2024-04-25T10:30:28.882547+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":42,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:30:27.860024+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:30:27.860024+0000","last_clean_scrub_stamp":"2024-04-25T10:30:27.860024+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T15:39:48.016111+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.8","version":"0'0","reported_seq":30,"reported_epoch":44,"state":"active+clean","last_fresh":"2024-04-25T10:30:29.881839+0000","last_change":"2024-04-25T10:30:11.758581+0000","last_active":"2024-04-25T10:30:29.881839+0000","last_peered":"2024-04-25T10:30:29.881839+0000","last_clean":"2024-04-25T10:30:29.881839+0000","last_became_active":"2024-04-25T10:30:11.758239+0000","last_became_peered":"2024-04-25T10:30:11.758239+0000","last_unstale":"2024-04-25T10:30:29.881839+0000","last_undegraded":"2024-04-25T10:30:29.881839+0000","last_fullsized":"2024-04-25T10:30:29.881839+0000","mapping_epoch":33,"log_start":"0'0","ondisk_log_start":"0'0","created":33,"last_epoch_clean":34,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:30:10.729165+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:30:10.729165+0000","last_clean_scrub_stamp":"2024-04-25T10:30:10.729165+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T17:41:33.905382+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.9","version":"36'2","reported_seq":27,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-04-25T10:30:28.874577+0000","last_change":"2024-04-25T10:30:12.768782+0000","last_active":"2024-04-25T10:30:28.874577+0000","last_peered":"2024-04-25T10:30:28.874577+0000","last_clean":"2024-04-25T10:30:28.874577+0000","last_became_active":"2024-04-25T10:30:12.768464+0000","last_became_peered":"2024-04-25T10:30:12.768464+0000","last_unstale":"2024-04-25T10:30:28.874577+0000","last_undegraded":"2024-04-25T10:30:28.874577+0000","last_fullsized":"2024-04-25T10:30:28.874577+0000","mapping_epoch":34,"log_start":"0'0","ondisk_log_start":"0'0","created":34,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:30:11.733978+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:30:11.733978+0000","last_clean_scrub_stamp":"2024-04-25T10:30:11.733978+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T13:03:21.170197+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"6.a","version":"0'0","reported_seq":9,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-04-25T10:30:28.884130+0000","last_change":"2024-04-25T10:30:28.882147+0000","last_active":"2024-04-25T10:30:28.884130+0000","last_peered":"2024-04-25T10:30:28.884130+0000","last_clean":"2024-04-25T10:30:28.884130+0000","last_became_active":"2024-04-25T10:30:28.881783+0000","last_became_peered":"2024-04-25T10:30:28.881783+0000","last_unstale":"2024-04-25T10:30:28.884130+0000","last_undegraded":"2024-04-25T10:30:28.884130+0000","last_fullsized":"2024-04-25T10:30:28.884130+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":42,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:30:27.860024+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:30:27.860024+0000","last_clean_scrub_stamp":"2024-04-25T10:30:27.860024+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T18:36:07.211600+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.9","version":"0'0","reported_seq":26,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-04-25T10:30:28.874568+0000","last_change":"2024-04-25T10:30:11.759807+0000","last_active":"2024-04-25T10:30:28.874568+0000","last_peered":"2024-04-25T10:30:28.874568+0000","last_clean":"2024-04-25T10:30:28.874568+0000","last_became_active":"2024-04-25T10:30:11.758558+0000","last_became_peered":"2024-04-25T10:30:11.758558+0000","last_unstale":"2024-04-25T10:30:28.874568+0000","last_undegraded":"2024-04-25T10:30:28.874568+0000","last_fullsized":"2024-04-25T10:30:28.874568+0000","mapping_epoch":33,"log_start":"0'0","ondisk_log_start":"0'0","created":33,"last_epoch_clean":34,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:30:10.729165+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:30:10.729165+0000","last_clean_scrub_stamp":"2024-04-25T10:30:10.729165+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T20:38:23.460836+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"5.8","version":"36'9","reported_seq":36,"reported_epoch":44,"state":"active+clean","last_fresh":"2024-04-25T10:30:29.882627+0000","last_change":"2024-04-25T10:30:12.777060+0000","last_active":"2024-04-25T10:30:29.882627+0000","last_peered":"2024-04-25T10:30:29.882627+0000","last_clean":"2024-04-25T10:30:29.882627+0000","last_became_active":"2024-04-25T10:30:12.776749+0000","last_became_peered":"2024-04-25T10:30:12.776749+0000","last_unstale":"2024-04-25T10:30:29.882627+0000","last_undegraded":"2024-04-25T10:30:29.882627+0000","last_fullsized":"2024-04-25T10:30:29.882627+0000","mapping_epoch":34,"log_start":"0'0","ondisk_log_start":"0'0","created":34,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:30:11.733978+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:30:11.733978+0000","last_clean_scrub_stamp":"2024-04-25T10:30:11.733978+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T15:47:40.336983+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"6.b","version":"0'0","reported_seq":12,"reported_epoch":44,"state":"active+clean","last_fresh":"2024-04-25T10:30:30.799735+0000","last_change":"2024-04-25T10:30:28.881916+0000","last_active":"2024-04-25T10:30:30.799735+0000","last_peered":"2024-04-25T10:30:30.799735+0000","last_clean":"2024-04-25T10:30:30.799735+0000","last_became_active":"2024-04-25T10:30:28.881605+0000","last_became_peered":"2024-04-25T10:30:28.881605+0000","last_unstale":"2024-04-25T10:30:30.799735+0000","last_undegraded":"2024-04-25T10:30:30.799735+0000","last_fullsized":"2024-04-25T10:30:30.799735+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":42,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:30:27.860024+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:30:27.860024+0000","last_clean_scrub_stamp":"2024-04-25T10:30:27.860024+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T15:18:43.432122+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.a","version":"0'0","reported_seq":30,"reported_epoch":44,"state":"active+clean","last_fresh":"2024-04-25T10:30:29.883112+0000","last_change":"2024-04-25T10:30:11.762745+0000","last_active":"2024-04-25T10:30:29.883112+0000","last_peered":"2024-04-25T10:30:29.883112+0000","last_clean":"2024-04-25T10:30:29.883112+0000","last_became_active":"2024-04-25T10:30:11.762377+0000","last_became_peered":"2024-04-25T10:30:11.762377+0000","last_unstale":"2024-04-25T10:30:29.883112+0000","last_undegraded":"2024-04-25T10:30:29.883112+0000","last_fullsized":"2024-04-25T10:30:29.883112+0000","mapping_epoch":33,"log_start":"0'0","ondisk_log_start":"0'0","created":33,"last_epoch_clean":34,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:30:10.729165+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:30:10.729165+0000","last_clean_scrub_stamp":"2024-04-25T10:30:10.729165+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T17:15:13.597075+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"5.b","version":"36'8","reported_seq":32,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-04-25T10:30:28.875847+0000","last_change":"2024-04-25T10:30:12.766544+0000","last_active":"2024-04-25T10:30:28.875847+0000","last_peered":"2024-04-25T10:30:28.875847+0000","last_clean":"2024-04-25T10:30:28.875847+0000","last_became_active":"2024-04-25T10:30:12.766228+0000","last_became_peered":"2024-04-25T10:30:12.766228+0000","last_unstale":"2024-04-25T10:30:28.875847+0000","last_undegraded":"2024-04-25T10:30:28.875847+0000","last_fullsized":"2024-04-25T10:30:28.875847+0000","mapping_epoch":34,"log_start":"0'0","ondisk_log_start":"0'0","created":34,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:30:11.733978+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:30:11.733978+0000","last_clean_scrub_stamp":"2024-04-25T10:30:11.733978+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T18:00:26.511360+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"6.8","version":"0'0","reported_seq":12,"reported_epoch":44,"state":"active+clean","last_fresh":"2024-04-25T10:30:29.881772+0000","last_change":"2024-04-25T10:30:28.881118+0000","last_active":"2024-04-25T10:30:29.881772+0000","last_peered":"2024-04-25T10:30:29.881772+0000","last_clean":"2024-04-25T10:30:29.881772+0000","last_became_active":"2024-04-25T10:30:28.880810+0000","last_became_peered":"2024-04-25T10:30:28.880810+0000","last_unstale":"2024-04-25T10:30:29.881772+0000","last_undegraded":"2024-04-25T10:30:29.881772+0000","last_fullsized":"2024-04-25T10:30:29.881772+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":42,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:30:27.860024+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:30:27.860024+0000","last_clean_scrub_stamp":"2024-04-25T10:30:27.860024+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T15:53:38.989476+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.b","version":"37'4","reported_seq":29,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-04-25T10:30:28.875996+0000","last_change":"2024-04-25T10:30:11.752902+0000","last_active":"2024-04-25T10:30:28.875996+0000","last_peered":"2024-04-25T10:30:28.875996+0000","last_clean":"2024-04-25T10:30:28.875996+0000","last_became_active":"2024-04-25T10:30:11.752599+0000","last_became_peered":"2024-04-25T10:30:11.752599+0000","last_unstale":"2024-04-25T10:30:28.875996+0000","last_undegraded":"2024-04-25T10:30:28.875996+0000","last_fullsized":"2024-04-25T10:30:28.875996+0000","mapping_epoch":33,"log_start":"0'0","ondisk_log_start":"0'0","created":33,"last_epoch_clean":34,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:30:10.729165+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:30:10.729165+0000","last_clean_scrub_stamp":"2024-04-25T10:30:10.729165+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T14:01:05.786983+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"5.a","version":"36'5","reported_seq":32,"reported_epoch":44,"state":"active+clean","last_fresh":"2024-04-25T10:30:29.882793+0000","last_change":"2024-04-25T10:30:12.762940+0000","last_active":"2024-04-25T10:30:29.882793+0000","last_peered":"2024-04-25T10:30:29.882793+0000","last_clean":"2024-04-25T10:30:29.882793+0000","last_became_active":"2024-04-25T10:30:12.762628+0000","last_became_peered":"2024-04-25T10:30:12.762628+0000","last_unstale":"2024-04-25T10:30:29.882793+0000","last_undegraded":"2024-04-25T10:30:29.882793+0000","last_fullsized":"2024-04-25T10:30:29.882793+0000","mapping_epoch":34,"log_start":"0'0","ondisk_log_start":"0'0","created":34,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:30:11.733978+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:30:11.733978+0000","last_clean_scrub_stamp":"2024-04-25T10:30:11.733978+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T19:25:06.397830+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"6.9","version":"0'0","reported_seq":12,"reported_epoch":44,"state":"active+clean","last_fresh":"2024-04-25T10:30:30.799857+0000","last_change":"2024-04-25T10:30:28.885090+0000","last_active":"2024-04-25T10:30:30.799857+0000","last_peered":"2024-04-25T10:30:30.799857+0000","last_clean":"2024-04-25T10:30:30.799857+0000","last_became_active":"2024-04-25T10:30:28.882960+0000","last_became_peered":"2024-04-25T10:30:28.882960+0000","last_unstale":"2024-04-25T10:30:30.799857+0000","last_undegraded":"2024-04-25T10:30:30.799857+0000","last_fullsized":"2024-04-25T10:30:30.799857+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":42,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:30:27.860024+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:30:27.860024+0000","last_clean_scrub_stamp":"2024-04-25T10:30:27.860024+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T17:25:53.823227+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.c","version":"0'0","reported_seq":25,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-04-25T10:30:28.875034+0000","last_change":"2024-04-25T10:30:11.753067+0000","last_active":"2024-04-25T10:30:28.875034+0000","last_peered":"2024-04-25T10:30:28.875034+0000","last_clean":"2024-04-25T10:30:28.875034+0000","last_became_active":"2024-04-25T10:30:11.752595+0000","last_became_peered":"2024-04-25T10:30:11.752595+0000","last_unstale":"2024-04-25T10:30:28.875034+0000","last_undegraded":"2024-04-25T10:30:28.875034+0000","last_fullsized":"2024-04-25T10:30:28.875034+0000","mapping_epoch":33,"log_start":"0'0","ondisk_log_start":"0'0","created":33,"last_epoch_clean":34,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:30:10.729165+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:30:10.729165+0000","last_clean_scrub_stamp":"2024-04-25T10:30:10.729165+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T18:59:19.688948+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"5.d","version":"36'7","reported_seq":31,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-04-25T10:30:28.875563+0000","last_change":"2024-04-25T10:30:12.771743+0000","last_active":"2024-04-25T10:30:28.875563+0000","last_peered":"2024-04-25T10:30:28.875563+0000","last_clean":"2024-04-25T10:30:28.875563+0000","last_became_active":"2024-04-25T10:30:12.771387+0000","last_became_peered":"2024-04-25T10:30:12.771387+0000","last_unstale":"2024-04-25T10:30:28.875563+0000","last_undegraded":"2024-04-25T10:30:28.875563+0000","last_fullsized":"2024-04-25T10:30:28.875563+0000","mapping_epoch":34,"log_start":"0'0","ondisk_log_start":"0'0","created":34,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:30:11.733978+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:30:11.733978+0000","last_clean_scrub_stamp":"2024-04-25T10:30:11.733978+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T14:01:06.791796+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"6.e","version":"0'0","reported_seq":9,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-04-25T10:30:28.885894+0000","last_change":"2024-04-25T10:30:28.884467+0000","last_active":"2024-04-25T10:30:28.885894+0000","last_peered":"2024-04-25T10:30:28.885894+0000","last_clean":"2024-04-25T10:30:28.885894+0000","last_became_active":"2024-04-25T10:30:28.884173+0000","last_became_peered":"2024-04-25T10:30:28.884173+0000","last_unstale":"2024-04-25T10:30:28.885894+0000","last_undegraded":"2024-04-25T10:30:28.885894+0000","last_fullsized":"2024-04-25T10:30:28.885894+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":42,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:30:27.860024+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:30:27.860024+0000","last_clean_scrub_stamp":"2024-04-25T10:30:27.860024+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T20:14:31.367609+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.d","version":"37'2","reported_seq":32,"reported_epoch":44,"state":"active+clean","last_fresh":"2024-04-25T10:30:29.882419+0000","last_change":"2024-04-25T10:30:11.758390+0000","last_active":"2024-04-25T10:30:29.882419+0000","last_peered":"2024-04-25T10:30:29.882419+0000","last_clean":"2024-04-25T10:30:29.882419+0000","last_became_active":"2024-04-25T10:30:11.757989+0000","last_became_peered":"2024-04-25T10:30:11.757989+0000","last_unstale":"2024-04-25T10:30:29.882419+0000","last_undegraded":"2024-04-25T10:30:29.882419+0000","last_fullsized":"2024-04-25T10:30:29.882419+0000","mapping_epoch":33,"log_start":"0'0","ondisk_log_start":"0'0","created":33,"last_epoch_clean":34,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:30:10.729165+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:30:10.729165+0000","last_clean_scrub_stamp":"2024-04-25T10:30:10.729165+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T18:52:44.534946+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.c","version":"36'6","reported_seq":30,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-04-25T10:30:28.876757+0000","last_change":"2024-04-25T10:30:12.765790+0000","last_active":"2024-04-25T10:30:28.876757+0000","last_peered":"2024-04-25T10:30:28.876757+0000","last_clean":"2024-04-25T10:30:28.876757+0000","last_became_active":"2024-04-25T10:30:12.765468+0000","last_became_peered":"2024-04-25T10:30:12.765468+0000","last_unstale":"2024-04-25T10:30:28.876757+0000","last_undegraded":"2024-04-25T10:30:28.876757+0000","last_fullsized":"2024-04-25T10:30:28.876757+0000","mapping_epoch":34,"log_start":"0'0","ondisk_log_start":"0'0","created":34,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:30:11.733978+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:30:11.733978+0000","last_clean_scrub_stamp":"2024-04-25T10:30:11.733978+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T16:29:40.621322+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"6.f","version":"0'0","reported_seq":9,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-04-25T10:30:28.887668+0000","last_change":"2024-04-25T10:30:28.886506+0000","last_active":"2024-04-25T10:30:28.887668+0000","last_peered":"2024-04-25T10:30:28.887668+0000","last_clean":"2024-04-25T10:30:28.887668+0000","last_became_active":"2024-04-25T10:30:28.886157+0000","last_became_peered":"2024-04-25T10:30:28.886157+0000","last_unstale":"2024-04-25T10:30:28.887668+0000","last_undegraded":"2024-04-25T10:30:28.887668+0000","last_fullsized":"2024-04-25T10:30:28.887668+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":42,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:30:27.860024+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:30:27.860024+0000","last_clean_scrub_stamp":"2024-04-25T10:30:27.860024+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T11:57:46.105302+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.e","version":"37'3","reported_seq":33,"reported_epoch":44,"state":"active+clean","last_fresh":"2024-04-25T10:30:29.883403+0000","last_change":"2024-04-25T10:30:11.765106+0000","last_active":"2024-04-25T10:30:29.883403+0000","last_peered":"2024-04-25T10:30:29.883403+0000","last_clean":"2024-04-25T10:30:29.883403+0000","last_became_active":"2024-04-25T10:30:11.764710+0000","last_became_peered":"2024-04-25T10:30:11.764710+0000","last_unstale":"2024-04-25T10:30:29.883403+0000","last_undegraded":"2024-04-25T10:30:29.883403+0000","last_fullsized":"2024-04-25T10:30:29.883403+0000","mapping_epoch":33,"log_start":"0'0","ondisk_log_start":"0'0","created":33,"last_epoch_clean":34,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:30:10.729165+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:30:10.729165+0000","last_clean_scrub_stamp":"2024-04-25T10:30:10.729165+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T16:30:11.709416+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"5.f","version":"36'9","reported_seq":34,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-04-25T10:30:28.875269+0000","last_change":"2024-04-25T10:30:12.769270+0000","last_active":"2024-04-25T10:30:28.875269+0000","last_peered":"2024-04-25T10:30:28.875269+0000","last_clean":"2024-04-25T10:30:28.875269+0000","last_became_active":"2024-04-25T10:30:12.768931+0000","last_became_peered":"2024-04-25T10:30:12.768931+0000","last_unstale":"2024-04-25T10:30:28.875269+0000","last_undegraded":"2024-04-25T10:30:28.875269+0000","last_fullsized":"2024-04-25T10:30:28.875269+0000","mapping_epoch":34,"log_start":"0'0","ondisk_log_start":"0'0","created":34,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:30:11.733978+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:30:11.733978+0000","last_clean_scrub_stamp":"2024-04-25T10:30:11.733978+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T16:49:00.268423+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"6.c","version":"0'0","reported_seq":12,"reported_epoch":44,"state":"active+clean","last_fresh":"2024-04-25T10:30:29.883642+0000","last_change":"2024-04-25T10:30:28.887055+0000","last_active":"2024-04-25T10:30:29.883642+0000","last_peered":"2024-04-25T10:30:29.883642+0000","last_clean":"2024-04-25T10:30:29.883642+0000","last_became_active":"2024-04-25T10:30:28.886675+0000","last_became_peered":"2024-04-25T10:30:28.886675+0000","last_unstale":"2024-04-25T10:30:29.883642+0000","last_undegraded":"2024-04-25T10:30:29.883642+0000","last_fullsized":"2024-04-25T10:30:29.883642+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":42,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:30:27.860024+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:30:27.860024+0000","last_clean_scrub_stamp":"2024-04-25T10:30:27.860024+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T14:04:58.646968+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,1],"acting":[6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.f","version":"37'2","reported_seq":27,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-04-25T10:30:28.874709+0000","last_change":"2024-04-25T10:30:11.750348+0000","last_active":"2024-04-25T10:30:28.874709+0000","last_peered":"2024-04-25T10:30:28.874709+0000","last_clean":"2024-04-25T10:30:28.874709+0000","last_became_active":"2024-04-25T10:30:11.749970+0000","last_became_peered":"2024-04-25T10:30:11.749970+0000","last_unstale":"2024-04-25T10:30:28.874709+0000","last_undegraded":"2024-04-25T10:30:28.874709+0000","last_fullsized":"2024-04-25T10:30:28.874709+0000","mapping_epoch":33,"log_start":"0'0","ondisk_log_start":"0'0","created":33,"last_epoch_clean":34,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:30:10.729165+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:30:10.729165+0000","last_clean_scrub_stamp":"2024-04-25T10:30:10.729165+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T20:41:22.574408+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"5.e","version":"36'5","reported_seq":34,"reported_epoch":44,"state":"active+clean","last_fresh":"2024-04-25T10:30:29.882392+0000","last_change":"2024-04-25T10:30:12.780850+0000","last_active":"2024-04-25T10:30:29.882392+0000","last_peered":"2024-04-25T10:30:29.882392+0000","last_clean":"2024-04-25T10:30:29.882392+0000","last_became_active":"2024-04-25T10:30:12.780187+0000","last_became_peered":"2024-04-25T10:30:12.780187+0000","last_unstale":"2024-04-25T10:30:29.882392+0000","last_undegraded":"2024-04-25T10:30:29.882392+0000","last_fullsized":"2024-04-25T10:30:29.882392+0000","mapping_epoch":34,"log_start":"0'0","ondisk_log_start":"0'0","created":34,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:30:11.733978+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:30:11.733978+0000","last_clean_scrub_stamp":"2024-04-25T10:30:11.733978+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T12:49:30.093333+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"6.d","version":"0'0","reported_seq":9,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-04-25T10:30:28.880134+0000","last_change":"2024-04-25T10:30:28.878873+0000","last_active":"2024-04-25T10:30:28.880134+0000","last_peered":"2024-04-25T10:30:28.880134+0000","last_clean":"2024-04-25T10:30:28.880134+0000","last_became_active":"2024-04-25T10:30:28.878596+0000","last_became_peered":"2024-04-25T10:30:28.878596+0000","last_unstale":"2024-04-25T10:30:28.880134+0000","last_undegraded":"2024-04-25T10:30:28.880134+0000","last_fullsized":"2024-04-25T10:30:28.880134+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":42,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:30:27.860024+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:30:27.860024+0000","last_clean_scrub_stamp":"2024-04-25T10:30:27.860024+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T11:33:01.743970+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.10","version":"37'15","reported_seq":40,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-04-25T10:30:28.873989+0000","last_change":"2024-04-25T10:30:11.755358+0000","last_active":"2024-04-25T10:30:28.873989+0000","last_peered":"2024-04-25T10:30:28.873989+0000","last_clean":"2024-04-25T10:30:28.873989+0000","last_became_active":"2024-04-25T10:30:11.754992+0000","last_became_peered":"2024-04-25T10:30:11.754992+0000","last_unstale":"2024-04-25T10:30:28.873989+0000","last_undegraded":"2024-04-25T10:30:28.873989+0000","last_fullsized":"2024-04-25T10:30:28.873989+0000","mapping_epoch":33,"log_start":"0'0","ondisk_log_start":"0'0","created":33,"last_epoch_clean":34,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:30:10.729165+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:30:10.729165+0000","last_clean_scrub_stamp":"2024-04-25T10:30:10.729165+0000","objects_scrubbed":0,"log_size":15,"log_dups_size":0,"ondisk_log_size":15,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T19:57:30.397817+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":21,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3],"acting":[7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"5.11","version":"36'6","reported_seq":35,"reported_epoch":44,"state":"active+clean","last_fresh":"2024-04-25T10:30:29.881885+0000","last_change":"2024-04-25T10:30:12.777184+0000","last_active":"2024-04-25T10:30:29.881885+0000","last_peered":"2024-04-25T10:30:29.881885+0000","last_clean":"2024-04-25T10:30:29.881885+0000","last_became_active":"2024-04-25T10:30:12.776874+0000","last_became_peered":"2024-04-25T10:30:12.776874+0000","last_unstale":"2024-04-25T10:30:29.881885+0000","last_undegraded":"2024-04-25T10:30:29.881885+0000","last_fullsized":"2024-04-25T10:30:29.881885+0000","mapping_epoch":34,"log_start":"0'0","ondisk_log_start":"0'0","created":34,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:30:11.733978+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:30:11.733978+0000","last_clean_scrub_stamp":"2024-04-25T10:30:11.733978+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T16:41:33.609972+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"6.12","version":"0'0","reported_seq":9,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-04-25T10:30:28.882650+0000","last_change":"2024-04-25T10:30:28.881494+0000","last_active":"2024-04-25T10:30:28.882650+0000","last_peered":"2024-04-25T10:30:28.882650+0000","last_clean":"2024-04-25T10:30:28.882650+0000","last_became_active":"2024-04-25T10:30:28.880682+0000","last_became_peered":"2024-04-25T10:30:28.880682+0000","last_unstale":"2024-04-25T10:30:28.882650+0000","last_undegraded":"2024-04-25T10:30:28.882650+0000","last_fullsized":"2024-04-25T10:30:28.882650+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":42,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:30:27.860024+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:30:27.860024+0000","last_clean_scrub_stamp":"2024-04-25T10:30:27.860024+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T17:35:49.687185+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.11","version":"0'0","reported_seq":29,"reported_epoch":44,"state":"active+clean","last_fresh":"2024-04-25T10:30:30.800150+0000","last_change":"2024-04-25T10:30:11.755706+0000","last_active":"2024-04-25T10:30:30.800150+0000","last_peered":"2024-04-25T10:30:30.800150+0000","last_clean":"2024-04-25T10:30:30.800150+0000","last_became_active":"2024-04-25T10:30:11.755348+0000","last_became_peered":"2024-04-25T10:30:11.755348+0000","last_unstale":"2024-04-25T10:30:30.800150+0000","last_undegraded":"2024-04-25T10:30:30.800150+0000","last_fullsized":"2024-04-25T10:30:30.800150+0000","mapping_epoch":33,"log_start":"0'0","ondisk_log_start":"0'0","created":33,"last_epoch_clean":34,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:30:10.729165+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:30:10.729165+0000","last_clean_scrub_stamp":"2024-04-25T10:30:10.729165+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T10:42:20.657474+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.10","version":"36'6","reported_seq":35,"reported_epoch":44,"state":"active+clean","last_fresh":"2024-04-25T10:30:29.882755+0000","last_change":"2024-04-25T10:30:12.783285+0000","last_active":"2024-04-25T10:30:29.882755+0000","last_peered":"2024-04-25T10:30:29.882755+0000","last_clean":"2024-04-25T10:30:29.882755+0000","last_became_active":"2024-04-25T10:30:12.782617+0000","last_became_peered":"2024-04-25T10:30:12.782617+0000","last_unstale":"2024-04-25T10:30:29.882755+0000","last_undegraded":"2024-04-25T10:30:29.882755+0000","last_fullsized":"2024-04-25T10:30:29.882755+0000","mapping_epoch":34,"log_start":"0'0","ondisk_log_start":"0'0","created":34,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:30:11.733978+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:30:11.733978+0000","last_clean_scrub_stamp":"2024-04-25T10:30:11.733978+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T18:35:51.085554+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"6.13","version":"0'0","reported_seq":9,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-04-25T10:30:28.882643+0000","last_change":"2024-04-25T10:30:28.880997+0000","last_active":"2024-04-25T10:30:28.882643+0000","last_peered":"2024-04-25T10:30:28.882643+0000","last_clean":"2024-04-25T10:30:28.882643+0000","last_became_active":"2024-04-25T10:30:28.880686+0000","last_became_peered":"2024-04-25T10:30:28.880686+0000","last_unstale":"2024-04-25T10:30:28.882643+0000","last_undegraded":"2024-04-25T10:30:28.882643+0000","last_fullsized":"2024-04-25T10:30:28.882643+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":42,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:30:27.860024+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:30:27.860024+0000","last_clean_scrub_stamp":"2024-04-25T10:30:27.860024+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T20:14:31.367609+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.12","version":"0'0","reported_seq":29,"reported_epoch":44,"state":"active+clean","last_fresh":"2024-04-25T10:30:30.800203+0000","last_change":"2024-04-25T10:30:11.762094+0000","last_active":"2024-04-25T10:30:30.800203+0000","last_peered":"2024-04-25T10:30:30.800203+0000","last_clean":"2024-04-25T10:30:30.800203+0000","last_became_active":"2024-04-25T10:30:11.761456+0000","last_became_peered":"2024-04-25T10:30:11.761456+0000","last_unstale":"2024-04-25T10:30:30.800203+0000","last_undegraded":"2024-04-25T10:30:30.800203+0000","last_fullsized":"2024-04-25T10:30:30.800203+0000","mapping_epoch":33,"log_start":"0'0","ondisk_log_start":"0'0","created":33,"last_epoch_clean":34,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:30:10.729165+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:30:10.729165+0000","last_clean_scrub_stamp":"2024-04-25T10:30:10.729165+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T21:58:31.211023+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.13","version":"36'15","reported_seq":43,"reported_epoch":44,"state":"active+clean","last_fresh":"2024-04-25T10:30:30.799948+0000","last_change":"2024-04-25T10:30:12.775269+0000","last_active":"2024-04-25T10:30:30.799948+0000","last_peered":"2024-04-25T10:30:30.799948+0000","last_clean":"2024-04-25T10:30:30.799948+0000","last_became_active":"2024-04-25T10:30:12.774865+0000","last_became_peered":"2024-04-25T10:30:12.774865+0000","last_unstale":"2024-04-25T10:30:30.799948+0000","last_undegraded":"2024-04-25T10:30:30.799948+0000","last_fullsized":"2024-04-25T10:30:30.799948+0000","mapping_epoch":34,"log_start":"0'0","ondisk_log_start":"0'0","created":34,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:30:11.733978+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:30:11.733978+0000","last_clean_scrub_stamp":"2024-04-25T10:30:11.733978+0000","objects_scrubbed":0,"log_size":15,"log_dups_size":0,"ondisk_log_size":15,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T19:52:00.482437+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"6.10","version":"0'0","reported_seq":12,"reported_epoch":44,"state":"active+clean","last_fresh":"2024-04-25T10:30:30.799595+0000","last_change":"2024-04-25T10:30:28.874537+0000","last_active":"2024-04-25T10:30:30.799595+0000","last_peered":"2024-04-25T10:30:30.799595+0000","last_clean":"2024-04-25T10:30:30.799595+0000","last_became_active":"2024-04-25T10:30:28.874226+0000","last_became_peered":"2024-04-25T10:30:28.874226+0000","last_unstale":"2024-04-25T10:30:30.799595+0000","last_undegraded":"2024-04-25T10:30:30.799595+0000","last_fullsized":"2024-04-25T10:30:30.799595+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":42,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:30:27.860024+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:30:27.860024+0000","last_clean_scrub_stamp":"2024-04-25T10:30:27.860024+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T19:27:12.449150+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.13","version":"37'5","reported_seq":35,"reported_epoch":44,"state":"active+clean","last_fresh":"2024-04-25T10:30:29.883254+0000","last_change":"2024-04-25T10:30:11.764993+0000","last_active":"2024-04-25T10:30:29.883254+0000","last_peered":"2024-04-25T10:30:29.883254+0000","last_clean":"2024-04-25T10:30:29.883254+0000","last_became_active":"2024-04-25T10:30:11.764637+0000","last_became_peered":"2024-04-25T10:30:11.764637+0000","last_unstale":"2024-04-25T10:30:29.883254+0000","last_undegraded":"2024-04-25T10:30:29.883254+0000","last_fullsized":"2024-04-25T10:30:29.883254+0000","mapping_epoch":33,"log_start":"0'0","ondisk_log_start":"0'0","created":33,"last_epoch_clean":34,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:30:10.729165+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:30:10.729165+0000","last_clean_scrub_stamp":"2024-04-25T10:30:10.729165+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T17:24:13.848319+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6,"num_write_kb":8,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"5.12","version":"36'10","reported_seq":37,"reported_epoch":44,"state":"active+clean","last_fresh":"2024-04-25T10:30:29.882763+0000","last_change":"2024-04-25T10:30:12.770496+0000","last_active":"2024-04-25T10:30:29.882763+0000","last_peered":"2024-04-25T10:30:29.882763+0000","last_clean":"2024-04-25T10:30:29.882763+0000","last_became_active":"2024-04-25T10:30:12.770193+0000","last_became_peered":"2024-04-25T10:30:12.770193+0000","last_unstale":"2024-04-25T10:30:29.882763+0000","last_undegraded":"2024-04-25T10:30:29.882763+0000","last_fullsized":"2024-04-25T10:30:29.882763+0000","mapping_epoch":34,"log_start":"0'0","ondisk_log_start":"0'0","created":34,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:30:11.733978+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:30:11.733978+0000","last_clean_scrub_stamp":"2024-04-25T10:30:11.733978+0000","objects_scrubbed":0,"log_size":10,"log_dups_size":0,"ondisk_log_size":10,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T21:35:23.984378+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,1],"acting":[6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"6.11","version":"0'0","reported_seq":12,"reported_epoch":44,"state":"active+clean","last_fresh":"2024-04-25T10:30:29.882707+0000","last_change":"2024-04-25T10:30:28.884488+0000","last_active":"2024-04-25T10:30:29.882707+0000","last_peered":"2024-04-25T10:30:29.882707+0000","last_clean":"2024-04-25T10:30:29.882707+0000","last_became_active":"2024-04-25T10:30:28.884188+0000","last_became_peered":"2024-04-25T10:30:28.884188+0000","last_unstale":"2024-04-25T10:30:29.882707+0000","last_undegraded":"2024-04-25T10:30:29.882707+0000","last_fullsized":"2024-04-25T10:30:29.882707+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":42,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:30:27.860024+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:30:27.860024+0000","last_clean_scrub_stamp":"2024-04-25T10:30:27.860024+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T15:39:48.016111+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,1],"acting":[6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.14","version":"37'23","reported_seq":51,"reported_epoch":44,"state":"active+clean","last_fresh":"2024-04-25T10:30:29.883688+0000","last_change":"2024-04-25T10:30:11.757323+0000","last_active":"2024-04-25T10:30:29.883688+0000","last_peered":"2024-04-25T10:30:29.883688+0000","last_clean":"2024-04-25T10:30:29.883688+0000","last_became_active":"2024-04-25T10:30:11.757011+0000","last_became_peered":"2024-04-25T10:30:11.757011+0000","last_unstale":"2024-04-25T10:30:29.883688+0000","last_undegraded":"2024-04-25T10:30:29.883688+0000","last_fullsized":"2024-04-25T10:30:29.883688+0000","mapping_epoch":33,"log_start":"0'0","ondisk_log_start":"0'0","created":33,"last_epoch_clean":34,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:30:10.729165+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:30:10.729165+0000","last_clean_scrub_stamp":"2024-04-25T10:30:10.729165+0000","objects_scrubbed":0,"log_size":23,"log_dups_size":0,"ondisk_log_size":23,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T19:02:53.584992+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":23,"num_write_kb":25,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"5.15","version":"36'6","reported_seq":35,"reported_epoch":44,"state":"active+clean","last_fresh":"2024-04-25T10:30:29.881889+0000","last_change":"2024-04-25T10:30:12.776432+0000","last_active":"2024-04-25T10:30:29.881889+0000","last_peered":"2024-04-25T10:30:29.881889+0000","last_clean":"2024-04-25T10:30:29.881889+0000","last_became_active":"2024-04-25T10:30:12.775857+0000","last_became_peered":"2024-04-25T10:30:12.775857+0000","last_unstale":"2024-04-25T10:30:29.881889+0000","last_undegraded":"2024-04-25T10:30:29.881889+0000","last_fullsized":"2024-04-25T10:30:29.881889+0000","mapping_epoch":34,"log_start":"0'0","ondisk_log_start":"0'0","created":34,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:30:11.733978+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:30:11.733978+0000","last_clean_scrub_stamp":"2024-04-25T10:30:11.733978+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T14:29:10.343115+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"6.16","version":"0'0","reported_seq":12,"reported_epoch":44,"state":"active+clean","last_fresh":"2024-04-25T10:30:30.799863+0000","last_change":"2024-04-25T10:30:28.881222+0000","last_active":"2024-04-25T10:30:30.799863+0000","last_peered":"2024-04-25T10:30:30.799863+0000","last_clean":"2024-04-25T10:30:30.799863+0000","last_became_active":"2024-04-25T10:30:28.880786+0000","last_became_peered":"2024-04-25T10:30:28.880786+0000","last_unstale":"2024-04-25T10:30:30.799863+0000","last_undegraded":"2024-04-25T10:30:30.799863+0000","last_fullsized":"2024-04-25T10:30:30.799863+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":42,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:30:27.860024+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:30:27.860024+0000","last_clean_scrub_stamp":"2024-04-25T10:30:27.860024+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T21:03:30.612823+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.15","version":"37'5","reported_seq":35,"reported_epoch":44,"state":"active+clean","last_fresh":"2024-04-25T10:30:29.882033+0000","last_change":"2024-04-25T10:30:11.755959+0000","last_active":"2024-04-25T10:30:29.882033+0000","last_peered":"2024-04-25T10:30:29.882033+0000","last_clean":"2024-04-25T10:30:29.882033+0000","last_became_active":"2024-04-25T10:30:11.755603+0000","last_became_peered":"2024-04-25T10:30:11.755603+0000","last_unstale":"2024-04-25T10:30:29.882033+0000","last_undegraded":"2024-04-25T10:30:29.882033+0000","last_fullsized":"2024-04-25T10:30:29.882033+0000","mapping_epoch":33,"log_start":"0'0","ondisk_log_start":"0'0","created":33,"last_epoch_clean":34,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:30:10.729165+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:30:10.729165+0000","last_clean_scrub_stamp":"2024-04-25T10:30:10.729165+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T19:25:05.393017+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.14","version":"36'11","reported_seq":39,"reported_epoch":44,"state":"active+clean","last_fresh":"2024-04-25T10:30:30.800775+0000","last_change":"2024-04-25T10:30:12.770588+0000","last_active":"2024-04-25T10:30:30.800775+0000","last_peered":"2024-04-25T10:30:30.800775+0000","last_clean":"2024-04-25T10:30:30.800775+0000","last_became_active":"2024-04-25T10:30:12.770264+0000","last_became_peered":"2024-04-25T10:30:12.770264+0000","last_unstale":"2024-04-25T10:30:30.800775+0000","last_undegraded":"2024-04-25T10:30:30.800775+0000","last_fullsized":"2024-04-25T10:30:30.800775+0000","mapping_epoch":34,"log_start":"0'0","ondisk_log_start":"0'0","created":34,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:30:11.733978+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:30:11.733978+0000","last_clean_scrub_stamp":"2024-04-25T10:30:11.733978+0000","objects_scrubbed":0,"log_size":11,"log_dups_size":0,"ondisk_log_size":11,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T21:20:12.031650+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"6.17","version":"0'0","reported_seq":9,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-04-25T10:30:28.880577+0000","last_change":"2024-04-25T10:30:28.878754+0000","last_active":"2024-04-25T10:30:28.880577+0000","last_peered":"2024-04-25T10:30:28.880577+0000","last_clean":"2024-04-25T10:30:28.880577+0000","last_became_active":"2024-04-25T10:30:28.878242+0000","last_became_peered":"2024-04-25T10:30:28.878242+0000","last_unstale":"2024-04-25T10:30:28.880577+0000","last_undegraded":"2024-04-25T10:30:28.880577+0000","last_fullsized":"2024-04-25T10:30:28.880577+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":42,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:30:27.860024+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:30:27.860024+0000","last_clean_scrub_stamp":"2024-04-25T10:30:27.860024+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T16:08:46.113866+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7],"acting":[1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.16","version":"0'0","reported_seq":25,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-04-25T10:30:28.879092+0000","last_change":"2024-04-25T10:30:11.749473+0000","last_active":"2024-04-25T10:30:28.879092+0000","last_peered":"2024-04-25T10:30:28.879092+0000","last_clean":"2024-04-25T10:30:28.879092+0000","last_became_active":"2024-04-25T10:30:11.749135+0000","last_became_peered":"2024-04-25T10:30:11.749135+0000","last_unstale":"2024-04-25T10:30:28.879092+0000","last_undegraded":"2024-04-25T10:30:28.879092+0000","last_fullsized":"2024-04-25T10:30:28.879092+0000","mapping_epoch":33,"log_start":"0'0","ondisk_log_start":"0'0","created":33,"last_epoch_clean":34,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:30:10.729165+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:30:10.729165+0000","last_clean_scrub_stamp":"2024-04-25T10:30:10.729165+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T15:17:28.997853+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"5.17","version":"36'9","reported_seq":37,"reported_epoch":44,"state":"active+clean","last_fresh":"2024-04-25T10:30:30.800520+0000","last_change":"2024-04-25T10:30:12.773113+0000","last_active":"2024-04-25T10:30:30.800520+0000","last_peered":"2024-04-25T10:30:30.800520+0000","last_clean":"2024-04-25T10:30:30.800520+0000","last_became_active":"2024-04-25T10:30:12.772786+0000","last_became_peered":"2024-04-25T10:30:12.772786+0000","last_unstale":"2024-04-25T10:30:30.800520+0000","last_undegraded":"2024-04-25T10:30:30.800520+0000","last_fullsized":"2024-04-25T10:30:30.800520+0000","mapping_epoch":34,"log_start":"0'0","ondisk_log_start":"0'0","created":34,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:30:11.733978+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:30:11.733978+0000","last_clean_scrub_stamp":"2024-04-25T10:30:11.733978+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T21:30:14.051378+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5],"acting":[4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"6.14","version":"0'0","reported_seq":12,"reported_epoch":44,"state":"active+clean","last_fresh":"2024-04-25T10:30:30.800723+0000","last_change":"2024-04-25T10:30:28.878325+0000","last_active":"2024-04-25T10:30:30.800723+0000","last_peered":"2024-04-25T10:30:30.800723+0000","last_clean":"2024-04-25T10:30:30.800723+0000","last_became_active":"2024-04-25T10:30:28.878014+0000","last_became_peered":"2024-04-25T10:30:28.878014+0000","last_unstale":"2024-04-25T10:30:30.800723+0000","last_undegraded":"2024-04-25T10:30:30.800723+0000","last_fullsized":"2024-04-25T10:30:30.800723+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":42,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:30:27.860024+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:30:27.860024+0000","last_clean_scrub_stamp":"2024-04-25T10:30:27.860024+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T11:04:24.909990+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.17","version":"0'0","reported_seq":25,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-04-25T10:30:28.878199+0000","last_change":"2024-04-25T10:30:11.758416+0000","last_active":"2024-04-25T10:30:28.878199+0000","last_peered":"2024-04-25T10:30:28.878199+0000","last_clean":"2024-04-25T10:30:28.878199+0000","last_became_active":"2024-04-25T10:30:11.758049+0000","last_became_peered":"2024-04-25T10:30:11.758049+0000","last_unstale":"2024-04-25T10:30:28.878199+0000","last_undegraded":"2024-04-25T10:30:28.878199+0000","last_fullsized":"2024-04-25T10:30:28.878199+0000","mapping_epoch":33,"log_start":"0'0","ondisk_log_start":"0'0","created":33,"last_epoch_clean":34,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:30:10.729165+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:30:10.729165+0000","last_clean_scrub_stamp":"2024-04-25T10:30:10.729165+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T18:26:05.159854+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"5.16","version":"36'10","reported_seq":39,"reported_epoch":44,"state":"active+clean","last_fresh":"2024-04-25T10:30:29.882265+0000","last_change":"2024-04-25T10:30:12.770216+0000","last_active":"2024-04-25T10:30:29.882265+0000","last_peered":"2024-04-25T10:30:29.882265+0000","last_clean":"2024-04-25T10:30:29.882265+0000","last_became_active":"2024-04-25T10:30:12.769785+0000","last_became_peered":"2024-04-25T10:30:12.769785+0000","last_unstale":"2024-04-25T10:30:29.882265+0000","last_undegraded":"2024-04-25T10:30:29.882265+0000","last_fullsized":"2024-04-25T10:30:29.882265+0000","mapping_epoch":34,"log_start":"0'0","ondisk_log_start":"0'0","created":34,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:30:11.733978+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:30:11.733978+0000","last_clean_scrub_stamp":"2024-04-25T10:30:11.733978+0000","objects_scrubbed":0,"log_size":10,"log_dups_size":0,"ondisk_log_size":10,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T16:58:09.648786+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"6.15","version":"0'0","reported_seq":12,"reported_epoch":44,"state":"active+clean","last_fresh":"2024-04-25T10:30:29.883532+0000","last_change":"2024-04-25T10:30:28.879590+0000","last_active":"2024-04-25T10:30:29.883532+0000","last_peered":"2024-04-25T10:30:29.883532+0000","last_clean":"2024-04-25T10:30:29.883532+0000","last_became_active":"2024-04-25T10:30:28.879277+0000","last_became_peered":"2024-04-25T10:30:28.879277+0000","last_unstale":"2024-04-25T10:30:29.883532+0000","last_undegraded":"2024-04-25T10:30:29.883532+0000","last_fullsized":"2024-04-25T10:30:29.883532+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":42,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:30:27.860024+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:30:27.860024+0000","last_clean_scrub_stamp":"2024-04-25T10:30:27.860024+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T14:04:58.646968+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.18","version":"0'0","reported_seq":30,"reported_epoch":44,"state":"active+clean","last_fresh":"2024-04-25T10:30:29.882866+0000","last_change":"2024-04-25T10:30:11.756627+0000","last_active":"2024-04-25T10:30:29.882866+0000","last_peered":"2024-04-25T10:30:29.882866+0000","last_clean":"2024-04-25T10:30:29.882866+0000","last_became_active":"2024-04-25T10:30:11.756323+0000","last_became_peered":"2024-04-25T10:30:11.756323+0000","last_unstale":"2024-04-25T10:30:29.882866+0000","last_undegraded":"2024-04-25T10:30:29.882866+0000","last_fullsized":"2024-04-25T10:30:29.882866+0000","mapping_epoch":33,"log_start":"0'0","ondisk_log_start":"0'0","created":33,"last_epoch_clean":34,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:30:10.729165+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:30:10.729165+0000","last_clean_scrub_stamp":"2024-04-25T10:30:10.729165+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T16:49:20.087013+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4],"acting":[2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"5.19","version":"36'10","reported_seq":38,"reported_epoch":44,"state":"active+clean","last_fresh":"2024-04-25T10:30:30.799990+0000","last_change":"2024-04-25T10:30:12.780437+0000","last_active":"2024-04-25T10:30:30.799990+0000","last_peered":"2024-04-25T10:30:30.799990+0000","last_clean":"2024-04-25T10:30:30.799990+0000","last_became_active":"2024-04-25T10:30:12.779762+0000","last_became_peered":"2024-04-25T10:30:12.779762+0000","last_unstale":"2024-04-25T10:30:30.799990+0000","last_undegraded":"2024-04-25T10:30:30.799990+0000","last_fullsized":"2024-04-25T10:30:30.799990+0000","mapping_epoch":34,"log_start":"0'0","ondisk_log_start":"0'0","created":34,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:30:11.733978+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:30:11.733978+0000","last_clean_scrub_stamp":"2024-04-25T10:30:11.733978+0000","objects_scrubbed":0,"log_size":10,"log_dups_size":0,"ondisk_log_size":10,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T12:12:16.175153+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"6.1a","version":"0'0","reported_seq":12,"reported_epoch":44,"state":"active+clean","last_fresh":"2024-04-25T10:30:29.882257+0000","last_change":"2024-04-25T10:30:28.879591+0000","last_active":"2024-04-25T10:30:29.882257+0000","last_peered":"2024-04-25T10:30:29.882257+0000","last_clean":"2024-04-25T10:30:29.882257+0000","last_became_active":"2024-04-25T10:30:28.879271+0000","last_became_peered":"2024-04-25T10:30:28.879271+0000","last_unstale":"2024-04-25T10:30:29.882257+0000","last_undegraded":"2024-04-25T10:30:29.882257+0000","last_fullsized":"2024-04-25T10:30:29.882257+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":42,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:30:27.860024+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:30:27.860024+0000","last_clean_scrub_stamp":"2024-04-25T10:30:27.860024+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T20:14:35.145802+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.19","version":"0'0","reported_seq":25,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-04-25T10:30:28.874176+0000","last_change":"2024-04-25T10:30:11.757481+0000","last_active":"2024-04-25T10:30:28.874176+0000","last_peered":"2024-04-25T10:30:28.874176+0000","last_clean":"2024-04-25T10:30:28.874176+0000","last_became_active":"2024-04-25T10:30:11.757149+0000","last_became_peered":"2024-04-25T10:30:11.757149+0000","last_unstale":"2024-04-25T10:30:28.874176+0000","last_undegraded":"2024-04-25T10:30:28.874176+0000","last_fullsized":"2024-04-25T10:30:28.874176+0000","mapping_epoch":33,"log_start":"0'0","ondisk_log_start":"0'0","created":33,"last_epoch_clean":34,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:30:10.729165+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:30:10.729165+0000","last_clean_scrub_stamp":"2024-04-25T10:30:10.729165+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T13:23:15.814583+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"5.18","version":"36'8","reported_seq":33,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-04-25T10:30:28.874986+0000","last_change":"2024-04-25T10:30:12.769294+0000","last_active":"2024-04-25T10:30:28.874986+0000","last_peered":"2024-04-25T10:30:28.874986+0000","last_clean":"2024-04-25T10:30:28.874986+0000","last_became_active":"2024-04-25T10:30:12.768543+0000","last_became_peered":"2024-04-25T10:30:12.768543+0000","last_unstale":"2024-04-25T10:30:28.874986+0000","last_undegraded":"2024-04-25T10:30:28.874986+0000","last_fullsized":"2024-04-25T10:30:28.874986+0000","mapping_epoch":34,"log_start":"0'0","ondisk_log_start":"0'0","created":34,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:30:11.733978+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:30:11.733978+0000","last_clean_scrub_stamp":"2024-04-25T10:30:11.733978+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T18:26:06.164667+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"6.1b","version":"0'0","reported_seq":12,"reported_epoch":44,"state":"active+clean","last_fresh":"2024-04-25T10:30:29.883121+0000","last_change":"2024-04-25T10:30:28.885011+0000","last_active":"2024-04-25T10:30:29.883121+0000","last_peered":"2024-04-25T10:30:29.883121+0000","last_clean":"2024-04-25T10:30:29.883121+0000","last_became_active":"2024-04-25T10:30:28.883516+0000","last_became_peered":"2024-04-25T10:30:28.883516+0000","last_unstale":"2024-04-25T10:30:29.883121+0000","last_undegraded":"2024-04-25T10:30:29.883121+0000","last_fullsized":"2024-04-25T10:30:29.883121+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":42,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:30:27.860024+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:30:27.860024+0000","last_clean_scrub_stamp":"2024-04-25T10:30:27.860024+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T17:35:49.687185+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,2],"acting":[6,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.1a","version":"35'1","reported_seq":31,"reported_epoch":44,"state":"active+clean","last_fresh":"2024-04-25T10:30:29.883044+0000","last_change":"2024-04-25T10:30:11.758678+0000","last_active":"2024-04-25T10:30:29.883044+0000","last_peered":"2024-04-25T10:30:29.883044+0000","last_clean":"2024-04-25T10:30:29.883044+0000","last_became_active":"2024-04-25T10:30:11.758384+0000","last_became_peered":"2024-04-25T10:30:11.758384+0000","last_unstale":"2024-04-25T10:30:29.883044+0000","last_undegraded":"2024-04-25T10:30:29.883044+0000","last_fullsized":"2024-04-25T10:30:29.883044+0000","mapping_epoch":33,"log_start":"0'0","ondisk_log_start":"0'0","created":33,"last_epoch_clean":34,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:30:10.729165+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:30:10.729165+0000","last_clean_scrub_stamp":"2024-04-25T10:30:10.729165+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T16:17:54.376541+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,6],"acting":[2,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"5.1b","version":"36'8","reported_seq":37,"reported_epoch":44,"state":"active+clean","last_fresh":"2024-04-25T10:30:29.883109+0000","last_change":"2024-04-25T10:30:12.784767+0000","last_active":"2024-04-25T10:30:29.883109+0000","last_peered":"2024-04-25T10:30:29.883109+0000","last_clean":"2024-04-25T10:30:29.883109+0000","last_became_active":"2024-04-25T10:30:12.784311+0000","last_became_peered":"2024-04-25T10:30:12.784311+0000","last_unstale":"2024-04-25T10:30:29.883109+0000","last_undegraded":"2024-04-25T10:30:29.883109+0000","last_fullsized":"2024-04-25T10:30:29.883109+0000","mapping_epoch":34,"log_start":"0'0","ondisk_log_start":"0'0","created":34,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:30:11.733978+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:30:11.733978+0000","last_clean_scrub_stamp":"2024-04-25T10:30:11.733978+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T19:56:57.765496+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,7],"acting":[2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"6.18","version":"0'0","reported_seq":12,"reported_epoch":44,"state":"active+clean","last_fresh":"2024-04-25T10:30:30.800349+0000","last_change":"2024-04-25T10:30:28.882157+0000","last_active":"2024-04-25T10:30:30.800349+0000","last_peered":"2024-04-25T10:30:30.800349+0000","last_clean":"2024-04-25T10:30:30.800349+0000","last_became_active":"2024-04-25T10:30:28.881823+0000","last_became_peered":"2024-04-25T10:30:28.881823+0000","last_unstale":"2024-04-25T10:30:30.800349+0000","last_undegraded":"2024-04-25T10:30:30.800349+0000","last_fullsized":"2024-04-25T10:30:30.800349+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":42,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:30:27.860024+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:30:27.860024+0000","last_clean_scrub_stamp":"2024-04-25T10:30:27.860024+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T12:12:35.591767+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5],"acting":[4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.1b","version":"35'1","reported_seq":31,"reported_epoch":44,"state":"active+clean","last_fresh":"2024-04-25T10:30:29.883205+0000","last_change":"2024-04-25T10:30:11.753667+0000","last_active":"2024-04-25T10:30:29.883205+0000","last_peered":"2024-04-25T10:30:29.883205+0000","last_clean":"2024-04-25T10:30:29.883205+0000","last_became_active":"2024-04-25T10:30:11.753340+0000","last_became_peered":"2024-04-25T10:30:11.753340+0000","last_unstale":"2024-04-25T10:30:29.883205+0000","last_undegraded":"2024-04-25T10:30:29.883205+0000","last_fullsized":"2024-04-25T10:30:29.883205+0000","mapping_epoch":33,"log_start":"0'0","ondisk_log_start":"0'0","created":33,"last_epoch_clean":34,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:30:10.729165+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:30:10.729165+0000","last_clean_scrub_stamp":"2024-04-25T10:30:10.729165+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T16:09:04.081750+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"5.1a","version":"36'7","reported_seq":32,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-04-25T10:30:28.874760+0000","last_change":"2024-04-25T10:30:12.773835+0000","last_active":"2024-04-25T10:30:28.874760+0000","last_peered":"2024-04-25T10:30:28.874760+0000","last_clean":"2024-04-25T10:30:28.874760+0000","last_became_active":"2024-04-25T10:30:12.773538+0000","last_became_peered":"2024-04-25T10:30:12.773538+0000","last_unstale":"2024-04-25T10:30:28.874760+0000","last_undegraded":"2024-04-25T10:30:28.874760+0000","last_fullsized":"2024-04-25T10:30:28.874760+0000","mapping_epoch":34,"log_start":"0'0","ondisk_log_start":"0'0","created":34,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:30:11.733978+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:30:11.733978+0000","last_clean_scrub_stamp":"2024-04-25T10:30:11.733978+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T18:31:49.260513+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"6.19","version":"0'0","reported_seq":9,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-04-25T10:30:28.885557+0000","last_change":"2024-04-25T10:30:28.883543+0000","last_active":"2024-04-25T10:30:28.885557+0000","last_peered":"2024-04-25T10:30:28.885557+0000","last_clean":"2024-04-25T10:30:28.885557+0000","last_became_active":"2024-04-25T10:30:28.883229+0000","last_became_peered":"2024-04-25T10:30:28.883229+0000","last_unstale":"2024-04-25T10:30:28.885557+0000","last_undegraded":"2024-04-25T10:30:28.885557+0000","last_fullsized":"2024-04-25T10:30:28.885557+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":42,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:30:27.860024+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:30:27.860024+0000","last_clean_scrub_stamp":"2024-04-25T10:30:27.860024+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T18:12:57.452350+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"6.1e","version":"0'0","reported_seq":12,"reported_epoch":44,"state":"active+clean","last_fresh":"2024-04-25T10:30:29.882435+0000","last_change":"2024-04-25T10:30:28.874417+0000","last_active":"2024-04-25T10:30:29.882435+0000","last_peered":"2024-04-25T10:30:29.882435+0000","last_clean":"2024-04-25T10:30:29.882435+0000","last_became_active":"2024-04-25T10:30:28.874102+0000","last_became_peered":"2024-04-25T10:30:28.874102+0000","last_unstale":"2024-04-25T10:30:29.882435+0000","last_undegraded":"2024-04-25T10:30:29.882435+0000","last_fullsized":"2024-04-25T10:30:29.882435+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":42,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:30:27.860024+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:30:27.860024+0000","last_clean_scrub_stamp":"2024-04-25T10:30:27.860024+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T21:25:07.511482+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.1c","version":"37'3","reported_seq":32,"reported_epoch":44,"state":"active+clean","last_fresh":"2024-04-25T10:30:30.800852+0000","last_change":"2024-04-25T10:30:11.759829+0000","last_active":"2024-04-25T10:30:30.800852+0000","last_peered":"2024-04-25T10:30:30.800852+0000","last_clean":"2024-04-25T10:30:30.800852+0000","last_became_active":"2024-04-25T10:30:11.759465+0000","last_became_peered":"2024-04-25T10:30:11.759465+0000","last_unstale":"2024-04-25T10:30:30.800852+0000","last_undegraded":"2024-04-25T10:30:30.800852+0000","last_fullsized":"2024-04-25T10:30:30.800852+0000","mapping_epoch":33,"log_start":"0'0","ondisk_log_start":"0'0","created":33,"last_epoch_clean":34,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:30:10.729165+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:30:10.729165+0000","last_clean_scrub_stamp":"2024-04-25T10:30:10.729165+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T13:31:53.608355+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.1d","version":"36'15","reported_seq":43,"reported_epoch":44,"state":"active+clean","last_fresh":"2024-04-25T10:30:30.799497+0000","last_change":"2024-04-25T10:30:12.773474+0000","last_active":"2024-04-25T10:30:30.799497+0000","last_peered":"2024-04-25T10:30:30.799497+0000","last_clean":"2024-04-25T10:30:30.799497+0000","last_became_active":"2024-04-25T10:30:12.773133+0000","last_became_peered":"2024-04-25T10:30:12.773133+0000","last_unstale":"2024-04-25T10:30:30.799497+0000","last_undegraded":"2024-04-25T10:30:30.799497+0000","last_fullsized":"2024-04-25T10:30:30.799497+0000","mapping_epoch":34,"log_start":"0'0","ondisk_log_start":"0'0","created":34,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:30:11.733978+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:30:11.733978+0000","last_clean_scrub_stamp":"2024-04-25T10:30:11.733978+0000","objects_scrubbed":0,"log_size":15,"log_dups_size":0,"ondisk_log_size":15,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T21:37:31.551319+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"6.1f","version":"0'0","reported_seq":12,"reported_epoch":44,"state":"active+clean","last_fresh":"2024-04-25T10:30:29.883478+0000","last_change":"2024-04-25T10:30:28.884033+0000","last_active":"2024-04-25T10:30:29.883478+0000","last_peered":"2024-04-25T10:30:29.883478+0000","last_clean":"2024-04-25T10:30:29.883478+0000","last_became_active":"2024-04-25T10:30:28.883610+0000","last_became_peered":"2024-04-25T10:30:28.883610+0000","last_unstale":"2024-04-25T10:30:29.883478+0000","last_undegraded":"2024-04-25T10:30:29.883478+0000","last_fullsized":"2024-04-25T10:30:29.883478+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":42,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:30:27.860024+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:30:27.860024+0000","last_clean_scrub_stamp":"2024-04-25T10:30:27.860024+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T18:12:57.452350+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.1d","version":"37'3","reported_seq":28,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-04-25T10:30:28.875875+0000","last_change":"2024-04-25T10:30:11.752439+0000","last_active":"2024-04-25T10:30:28.875875+0000","last_peered":"2024-04-25T10:30:28.875875+0000","last_clean":"2024-04-25T10:30:28.875875+0000","last_became_active":"2024-04-25T10:30:11.752050+0000","last_became_peered":"2024-04-25T10:30:11.752050+0000","last_unstale":"2024-04-25T10:30:28.875875+0000","last_undegraded":"2024-04-25T10:30:28.875875+0000","last_fullsized":"2024-04-25T10:30:28.875875+0000","mapping_epoch":33,"log_start":"0'0","ondisk_log_start":"0'0","created":33,"last_epoch_clean":34,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:30:10.729165+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:30:10.729165+0000","last_clean_scrub_stamp":"2024-04-25T10:30:10.729165+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T11:41:21.358729+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"5.1c","version":"36'9","reported_seq":38,"reported_epoch":44,"state":"active+clean","last_fresh":"2024-04-25T10:30:29.883403+0000","last_change":"2024-04-25T10:30:12.781913+0000","last_active":"2024-04-25T10:30:29.883403+0000","last_peered":"2024-04-25T10:30:29.883403+0000","last_clean":"2024-04-25T10:30:29.883403+0000","last_became_active":"2024-04-25T10:30:12.781111+0000","last_became_peered":"2024-04-25T10:30:12.781111+0000","last_unstale":"2024-04-25T10:30:29.883403+0000","last_undegraded":"2024-04-25T10:30:29.883403+0000","last_fullsized":"2024-04-25T10:30:29.883403+0000","mapping_epoch":34,"log_start":"0'0","ondisk_log_start":"0'0","created":34,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:30:11.733978+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:30:11.733978+0000","last_clean_scrub_stamp":"2024-04-25T10:30:11.733978+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T16:29:27.103512+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4],"acting":[2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.1e","version":"35'1","reported_seq":30,"reported_epoch":44,"state":"active+clean","last_fresh":"2024-04-25T10:30:30.800499+0000","last_change":"2024-04-25T10:30:11.761160+0000","last_active":"2024-04-25T10:30:30.800499+0000","last_peered":"2024-04-25T10:30:30.800499+0000","last_clean":"2024-04-25T10:30:30.800499+0000","last_became_active":"2024-04-25T10:30:11.760819+0000","last_became_peered":"2024-04-25T10:30:11.760819+0000","last_unstale":"2024-04-25T10:30:30.800499+0000","last_undegraded":"2024-04-25T10:30:30.800499+0000","last_fullsized":"2024-04-25T10:30:30.800499+0000","mapping_epoch":33,"log_start":"0'0","ondisk_log_start":"0'0","created":33,"last_epoch_clean":34,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:30:10.729165+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:30:10.729165+0000","last_clean_scrub_stamp":"2024-04-25T10:30:10.729165+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T13:00:22.139475+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.1f","version":"36'6","reported_seq":35,"reported_epoch":44,"state":"active+clean","last_fresh":"2024-04-25T10:30:29.883526+0000","last_change":"2024-04-25T10:30:12.772257+0000","last_active":"2024-04-25T10:30:29.883526+0000","last_peered":"2024-04-25T10:30:29.883526+0000","last_clean":"2024-04-25T10:30:29.883526+0000","last_became_active":"2024-04-25T10:30:12.771923+0000","last_became_peered":"2024-04-25T10:30:12.771923+0000","last_unstale":"2024-04-25T10:30:29.883526+0000","last_undegraded":"2024-04-25T10:30:29.883526+0000","last_fullsized":"2024-04-25T10:30:29.883526+0000","mapping_epoch":34,"log_start":"0'0","ondisk_log_start":"0'0","created":34,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:30:11.733978+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:30:11.733978+0000","last_clean_scrub_stamp":"2024-04-25T10:30:11.733978+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T17:35:33.561138+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"6.1c","version":"0'0","reported_seq":12,"reported_epoch":44,"state":"active+clean","last_fresh":"2024-04-25T10:30:29.882853+0000","last_change":"2024-04-25T10:30:28.875422+0000","last_active":"2024-04-25T10:30:29.882853+0000","last_peered":"2024-04-25T10:30:29.882853+0000","last_clean":"2024-04-25T10:30:29.882853+0000","last_became_active":"2024-04-25T10:30:28.875079+0000","last_became_peered":"2024-04-25T10:30:28.875079+0000","last_unstale":"2024-04-25T10:30:29.882853+0000","last_undegraded":"2024-04-25T10:30:29.882853+0000","last_fullsized":"2024-04-25T10:30:29.882853+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":42,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:30:27.860024+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:30:27.860024+0000","last_clean_scrub_stamp":"2024-04-25T10:30:27.860024+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T13:23:55.301869+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.1f","version":"37'8","reported_seq":38,"reported_epoch":44,"state":"active+clean","last_fresh":"2024-04-25T10:30:29.882826+0000","last_change":"2024-04-25T10:30:11.756978+0000","last_active":"2024-04-25T10:30:29.882826+0000","last_peered":"2024-04-25T10:30:29.882826+0000","last_clean":"2024-04-25T10:30:29.882826+0000","last_became_active":"2024-04-25T10:30:11.756617+0000","last_became_peered":"2024-04-25T10:30:11.756617+0000","last_unstale":"2024-04-25T10:30:29.882826+0000","last_undegraded":"2024-04-25T10:30:29.882826+0000","last_fullsized":"2024-04-25T10:30:29.882826+0000","mapping_epoch":33,"log_start":"0'0","ondisk_log_start":"0'0","created":33,"last_epoch_clean":34,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:30:10.729165+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:30:10.729165+0000","last_clean_scrub_stamp":"2024-04-25T10:30:10.729165+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T12:27:53.559880+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":1,"num_read_kb":0,"num_write":10,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.1e","version":"36'5","reported_seq":32,"reported_epoch":44,"state":"active+clean","last_fresh":"2024-04-25T10:30:29.883469+0000","last_change":"2024-04-25T10:30:12.769800+0000","last_active":"2024-04-25T10:30:29.883469+0000","last_peered":"2024-04-25T10:30:29.883469+0000","last_clean":"2024-04-25T10:30:29.883469+0000","last_became_active":"2024-04-25T10:30:12.769470+0000","last_became_peered":"2024-04-25T10:30:12.769470+0000","last_unstale":"2024-04-25T10:30:29.883469+0000","last_undegraded":"2024-04-25T10:30:29.883469+0000","last_fullsized":"2024-04-25T10:30:29.883469+0000","mapping_epoch":34,"log_start":"0'0","ondisk_log_start":"0'0","created":34,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:30:11.733978+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:30:11.733978+0000","last_clean_scrub_stamp":"2024-04-25T10:30:11.733978+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T13:11:54.897586+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"6.1d","version":"0'0","reported_seq":12,"reported_epoch":44,"state":"active+clean","last_fresh":"2024-04-25T10:30:29.883295+0000","last_change":"2024-04-25T10:30:28.884277+0000","last_active":"2024-04-25T10:30:29.883295+0000","last_peered":"2024-04-25T10:30:29.883295+0000","last_clean":"2024-04-25T10:30:29.883295+0000","last_became_active":"2024-04-25T10:30:28.883764+0000","last_became_peered":"2024-04-25T10:30:28.883764+0000","last_unstale":"2024-04-25T10:30:29.883295+0000","last_undegraded":"2024-04-25T10:30:29.883295+0000","last_fullsized":"2024-04-25T10:30:29.883295+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":42,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:30:27.860024+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:30:27.860024+0000","last_clean_scrub_stamp":"2024-04-25T10:30:27.860024+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T18:36:07.211600+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"5.20","version":"36'9","reported_seq":38,"reported_epoch":44,"state":"active+clean","last_fresh":"2024-04-25T10:30:29.881619+0000","last_change":"2024-04-25T10:30:12.766692+0000","last_active":"2024-04-25T10:30:29.881619+0000","last_peered":"2024-04-25T10:30:29.881619+0000","last_clean":"2024-04-25T10:30:29.881619+0000","last_became_active":"2024-04-25T10:30:12.766336+0000","last_became_peered":"2024-04-25T10:30:12.766336+0000","last_unstale":"2024-04-25T10:30:29.881619+0000","last_undegraded":"2024-04-25T10:30:29.881619+0000","last_fullsized":"2024-04-25T10:30:29.881619+0000","mapping_epoch":34,"log_start":"0'0","ondisk_log_start":"0'0","created":34,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:30:11.733978+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:30:11.733978+0000","last_clean_scrub_stamp":"2024-04-25T10:30:11.733978+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T18:11:27.798982+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.21","version":"36'12","reported_seq":41,"reported_epoch":44,"state":"active+clean","last_fresh":"2024-04-25T10:30:29.882888+0000","last_change":"2024-04-25T10:30:12.774245+0000","last_active":"2024-04-25T10:30:29.882888+0000","last_peered":"2024-04-25T10:30:29.882888+0000","last_clean":"2024-04-25T10:30:29.882888+0000","last_became_active":"2024-04-25T10:30:12.773923+0000","last_became_peered":"2024-04-25T10:30:12.773923+0000","last_unstale":"2024-04-25T10:30:29.882888+0000","last_undegraded":"2024-04-25T10:30:29.882888+0000","last_fullsized":"2024-04-25T10:30:29.882888+0000","mapping_epoch":34,"log_start":"0'0","ondisk_log_start":"0'0","created":34,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:30:11.733978+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:30:11.733978+0000","last_clean_scrub_stamp":"2024-04-25T10:30:11.733978+0000","objects_scrubbed":0,"log_size":12,"log_dups_size":0,"ondisk_log_size":12,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T22:09:01.889527+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,7],"acting":[2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"5.22","version":"36'8","reported_seq":32,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-04-25T10:30:28.877192+0000","last_change":"2024-04-25T10:30:12.769958+0000","last_active":"2024-04-25T10:30:28.877192+0000","last_peered":"2024-04-25T10:30:28.877192+0000","last_clean":"2024-04-25T10:30:28.877192+0000","last_became_active":"2024-04-25T10:30:12.769590+0000","last_became_peered":"2024-04-25T10:30:12.769590+0000","last_unstale":"2024-04-25T10:30:28.877192+0000","last_undegraded":"2024-04-25T10:30:28.877192+0000","last_fullsized":"2024-04-25T10:30:28.877192+0000","mapping_epoch":34,"log_start":"0'0","ondisk_log_start":"0'0","created":34,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:30:11.733978+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:30:11.733978+0000","last_clean_scrub_stamp":"2024-04-25T10:30:11.733978+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T11:57:29.979255+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"5.23","version":"36'7","reported_seq":31,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-04-25T10:30:28.876441+0000","last_change":"2024-04-25T10:30:12.781995+0000","last_active":"2024-04-25T10:30:28.876441+0000","last_peered":"2024-04-25T10:30:28.876441+0000","last_clean":"2024-04-25T10:30:28.876441+0000","last_became_active":"2024-04-25T10:30:12.781358+0000","last_became_peered":"2024-04-25T10:30:12.781358+0000","last_unstale":"2024-04-25T10:30:28.876441+0000","last_undegraded":"2024-04-25T10:30:28.876441+0000","last_fullsized":"2024-04-25T10:30:28.876441+0000","mapping_epoch":34,"log_start":"0'0","ondisk_log_start":"0'0","created":34,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:30:11.733978+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:30:11.733978+0000","last_clean_scrub_stamp":"2024-04-25T10:30:11.733978+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T18:00:26.511360+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"5.24","version":"36'4","reported_seq":33,"reported_epoch":44,"state":"active+clean","last_fresh":"2024-04-25T10:30:29.883349+0000","last_change":"2024-04-25T10:30:12.775515+0000","last_active":"2024-04-25T10:30:29.883349+0000","last_peered":"2024-04-25T10:30:29.883349+0000","last_clean":"2024-04-25T10:30:29.883349+0000","last_became_active":"2024-04-25T10:30:12.775167+0000","last_became_peered":"2024-04-25T10:30:12.775167+0000","last_unstale":"2024-04-25T10:30:29.883349+0000","last_undegraded":"2024-04-25T10:30:29.883349+0000","last_fullsized":"2024-04-25T10:30:29.883349+0000","mapping_epoch":34,"log_start":"0'0","ondisk_log_start":"0'0","created":34,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:30:11.733978+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:30:11.733978+0000","last_clean_scrub_stamp":"2024-04-25T10:30:11.733978+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T21:35:23.984378+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,6],"acting":[2,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"5.25","version":"36'6","reported_seq":30,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-04-25T10:30:28.876545+0000","last_change":"2024-04-25T10:30:12.767505+0000","last_active":"2024-04-25T10:30:28.876545+0000","last_peered":"2024-04-25T10:30:28.876545+0000","last_clean":"2024-04-25T10:30:28.876545+0000","last_became_active":"2024-04-25T10:30:12.767149+0000","last_became_peered":"2024-04-25T10:30:12.767149+0000","last_unstale":"2024-04-25T10:30:28.876545+0000","last_undegraded":"2024-04-25T10:30:28.876545+0000","last_fullsized":"2024-04-25T10:30:28.876545+0000","mapping_epoch":34,"log_start":"0'0","ondisk_log_start":"0'0","created":34,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:30:11.733978+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:30:11.733978+0000","last_clean_scrub_stamp":"2024-04-25T10:30:11.733978+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T16:29:27.103512+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"5.26","version":"36'15","reported_seq":43,"reported_epoch":44,"state":"active+clean","last_fresh":"2024-04-25T10:30:30.800340+0000","last_change":"2024-04-25T10:30:12.771420+0000","last_active":"2024-04-25T10:30:30.800340+0000","last_peered":"2024-04-25T10:30:30.800340+0000","last_clean":"2024-04-25T10:30:30.800340+0000","last_became_active":"2024-04-25T10:30:12.771117+0000","last_became_peered":"2024-04-25T10:30:12.771117+0000","last_unstale":"2024-04-25T10:30:30.800340+0000","last_undegraded":"2024-04-25T10:30:30.800340+0000","last_fullsized":"2024-04-25T10:30:30.800340+0000","mapping_epoch":34,"log_start":"0'0","ondisk_log_start":"0'0","created":34,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:30:11.733978+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:30:11.733978+0000","last_clean_scrub_stamp":"2024-04-25T10:30:11.733978+0000","objects_scrubbed":0,"log_size":15,"log_dups_size":0,"ondisk_log_size":15,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T21:06:20.749496+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.27","version":"36'2","reported_seq":30,"reported_epoch":44,"state":"active+clean","last_fresh":"2024-04-25T10:30:30.800377+0000","last_change":"2024-04-25T10:30:12.779972+0000","last_active":"2024-04-25T10:30:30.800377+0000","last_peered":"2024-04-25T10:30:30.800377+0000","last_clean":"2024-04-25T10:30:30.800377+0000","last_became_active":"2024-04-25T10:30:12.779526+0000","last_became_peered":"2024-04-25T10:30:12.779526+0000","last_unstale":"2024-04-25T10:30:30.800377+0000","last_undegraded":"2024-04-25T10:30:30.800377+0000","last_fullsized":"2024-04-25T10:30:30.800377+0000","mapping_epoch":34,"log_start":"0'0","ondisk_log_start":"0'0","created":34,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:30:11.733978+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:30:11.733978+0000","last_clean_scrub_stamp":"2024-04-25T10:30:11.733978+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T15:41:59.020749+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.28","version":"36'6","reported_seq":35,"reported_epoch":44,"state":"active+clean","last_fresh":"2024-04-25T10:30:29.882628+0000","last_change":"2024-04-25T10:30:12.783394+0000","last_active":"2024-04-25T10:30:29.882628+0000","last_peered":"2024-04-25T10:30:29.882628+0000","last_clean":"2024-04-25T10:30:29.882628+0000","last_became_active":"2024-04-25T10:30:12.783066+0000","last_became_peered":"2024-04-25T10:30:12.783066+0000","last_unstale":"2024-04-25T10:30:29.882628+0000","last_undegraded":"2024-04-25T10:30:29.882628+0000","last_fullsized":"2024-04-25T10:30:29.882628+0000","mapping_epoch":34,"log_start":"0'0","ondisk_log_start":"0'0","created":34,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:30:11.733978+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:30:11.733978+0000","last_clean_scrub_stamp":"2024-04-25T10:30:11.733978+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T15:39:31.890064+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4],"acting":[2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"5.29","version":"36'8","reported_seq":35,"reported_epoch":44,"state":"active+clean","last_fresh":"2024-04-25T10:30:29.882807+0000","last_change":"2024-04-25T10:30:12.763911+0000","last_active":"2024-04-25T10:30:29.882807+0000","last_peered":"2024-04-25T10:30:29.882807+0000","last_clean":"2024-04-25T10:30:29.882807+0000","last_became_active":"2024-04-25T10:30:12.763578+0000","last_became_peered":"2024-04-25T10:30:12.763578+0000","last_unstale":"2024-04-25T10:30:29.882807+0000","last_undegraded":"2024-04-25T10:30:29.882807+0000","last_fullsized":"2024-04-25T10:30:29.882807+0000","mapping_epoch":34,"log_start":"0'0","ondisk_log_start":"0'0","created":34,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:30:11.733978+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:30:11.733978+0000","last_clean_scrub_stamp":"2024-04-25T10:30:11.733978+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T18:52:45.539758+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,1],"acting":[6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"5.2a","version":"36'9","reported_seq":38,"reported_epoch":44,"state":"active+clean","last_fresh":"2024-04-25T10:30:29.882996+0000","last_change":"2024-04-25T10:30:12.782730+0000","last_active":"2024-04-25T10:30:29.882996+0000","last_peered":"2024-04-25T10:30:29.882996+0000","last_clean":"2024-04-25T10:30:29.882996+0000","last_became_active":"2024-04-25T10:30:12.782261+0000","last_became_peered":"2024-04-25T10:30:12.782261+0000","last_unstale":"2024-04-25T10:30:29.882996+0000","last_undegraded":"2024-04-25T10:30:29.882996+0000","last_fullsized":"2024-04-25T10:30:29.882996+0000","mapping_epoch":34,"log_start":"0'0","ondisk_log_start":"0'0","created":34,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:30:11.733978+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:30:11.733978+0000","last_clean_scrub_stamp":"2024-04-25T10:30:11.733978+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T18:00:26.511360+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4],"acting":[2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"5.2b","version":"36'11","reported_seq":35,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-04-25T10:30:28.876504+0000","last_change":"2024-04-25T10:30:12.771344+0000","last_active":"2024-04-25T10:30:28.876504+0000","last_peered":"2024-04-25T10:30:28.876504+0000","last_clean":"2024-04-25T10:30:28.876504+0000","last_became_active":"2024-04-25T10:30:12.770949+0000","last_became_peered":"2024-04-25T10:30:12.770949+0000","last_unstale":"2024-04-25T10:30:28.876504+0000","last_undegraded":"2024-04-25T10:30:28.876504+0000","last_fullsized":"2024-04-25T10:30:28.876504+0000","mapping_epoch":34,"log_start":"0'0","ondisk_log_start":"0'0","created":34,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:30:11.733978+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:30:11.733978+0000","last_clean_scrub_stamp":"2024-04-25T10:30:11.733978+0000","objects_scrubbed":0,"log_size":11,"log_dups_size":0,"ondisk_log_size":11,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T16:29:27.103512+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"5.2c","version":"36'8","reported_seq":35,"reported_epoch":44,"state":"active+clean","last_fresh":"2024-04-25T10:30:29.883520+0000","last_change":"2024-04-25T10:30:12.772663+0000","last_active":"2024-04-25T10:30:29.883520+0000","last_peered":"2024-04-25T10:30:29.883520+0000","last_clean":"2024-04-25T10:30:29.883520+0000","last_became_active":"2024-04-25T10:30:12.772356+0000","last_became_peered":"2024-04-25T10:30:12.772356+0000","last_unstale":"2024-04-25T10:30:29.883520+0000","last_undegraded":"2024-04-25T10:30:29.883520+0000","last_fullsized":"2024-04-25T10:30:29.883520+0000","mapping_epoch":34,"log_start":"0'0","ondisk_log_start":"0'0","created":34,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:30:11.733978+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:30:11.733978+0000","last_clean_scrub_stamp":"2024-04-25T10:30:11.733978+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T12:21:33.158775+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"5.2d","version":"36'7","reported_seq":31,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-04-25T10:30:28.875358+0000","last_change":"2024-04-25T10:30:12.768688+0000","last_active":"2024-04-25T10:30:28.875358+0000","last_peered":"2024-04-25T10:30:28.875358+0000","last_clean":"2024-04-25T10:30:28.875358+0000","last_became_active":"2024-04-25T10:30:12.768382+0000","last_became_peered":"2024-04-25T10:30:12.768382+0000","last_unstale":"2024-04-25T10:30:28.875358+0000","last_undegraded":"2024-04-25T10:30:28.875358+0000","last_fullsized":"2024-04-25T10:30:28.875358+0000","mapping_epoch":34,"log_start":"0'0","ondisk_log_start":"0'0","created":34,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:30:11.733978+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:30:11.733978+0000","last_clean_scrub_stamp":"2024-04-25T10:30:11.733978+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T18:26:06.164667+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"5.2e","version":"36'10","reported_seq":34,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-04-25T10:30:28.876239+0000","last_change":"2024-04-25T10:30:12.766393+0000","last_active":"2024-04-25T10:30:28.876239+0000","last_peered":"2024-04-25T10:30:28.876239+0000","last_clean":"2024-04-25T10:30:28.876239+0000","last_became_active":"2024-04-25T10:30:12.766096+0000","last_became_peered":"2024-04-25T10:30:12.766096+0000","last_unstale":"2024-04-25T10:30:28.876239+0000","last_undegraded":"2024-04-25T10:30:28.876239+0000","last_fullsized":"2024-04-25T10:30:28.876239+0000","mapping_epoch":34,"log_start":"0'0","ondisk_log_start":"0'0","created":34,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:30:11.733978+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:30:11.733978+0000","last_clean_scrub_stamp":"2024-04-25T10:30:11.733978+0000","objects_scrubbed":0,"log_size":10,"log_dups_size":0,"ondisk_log_size":10,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T16:49:00.268423+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"5.2f","version":"36'9","reported_seq":38,"reported_epoch":44,"state":"active+clean","last_fresh":"2024-04-25T10:30:29.882690+0000","last_change":"2024-04-25T10:30:12.766616+0000","last_active":"2024-04-25T10:30:29.882690+0000","last_peered":"2024-04-25T10:30:29.882690+0000","last_clean":"2024-04-25T10:30:29.882690+0000","last_became_active":"2024-04-25T10:30:12.766290+0000","last_became_peered":"2024-04-25T10:30:12.766290+0000","last_unstale":"2024-04-25T10:30:29.882690+0000","last_undegraded":"2024-04-25T10:30:29.882690+0000","last_fullsized":"2024-04-25T10:30:29.882690+0000","mapping_epoch":34,"log_start":"0'0","ondisk_log_start":"0'0","created":34,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:30:11.733978+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:30:11.733978+0000","last_clean_scrub_stamp":"2024-04-25T10:30:11.733978+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T13:22:50.975199+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":2097152,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":2048,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.30","version":"36'9","reported_seq":33,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-04-25T10:30:28.876615+0000","last_change":"2024-04-25T10:30:12.765989+0000","last_active":"2024-04-25T10:30:28.876615+0000","last_peered":"2024-04-25T10:30:28.876615+0000","last_clean":"2024-04-25T10:30:28.876615+0000","last_became_active":"2024-04-25T10:30:12.765648+0000","last_became_peered":"2024-04-25T10:30:12.765648+0000","last_unstale":"2024-04-25T10:30:28.876615+0000","last_undegraded":"2024-04-25T10:30:28.876615+0000","last_fullsized":"2024-04-25T10:30:28.876615+0000","mapping_epoch":34,"log_start":"0'0","ondisk_log_start":"0'0","created":34,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:30:11.733978+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:30:11.733978+0000","last_clean_scrub_stamp":"2024-04-25T10:30:11.733978+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T16:29:40.621322+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"5.31","version":"36'7","reported_seq":31,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-04-25T10:30:28.877047+0000","last_change":"2024-04-25T10:30:12.777324+0000","last_active":"2024-04-25T10:30:28.877047+0000","last_peered":"2024-04-25T10:30:28.877047+0000","last_clean":"2024-04-25T10:30:28.877047+0000","last_became_active":"2024-04-25T10:30:12.776983+0000","last_became_peered":"2024-04-25T10:30:12.776983+0000","last_unstale":"2024-04-25T10:30:28.877047+0000","last_undegraded":"2024-04-25T10:30:28.877047+0000","last_fullsized":"2024-04-25T10:30:28.877047+0000","mapping_epoch":34,"log_start":"0'0","ondisk_log_start":"0'0","created":34,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:30:11.733978+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:30:11.733978+0000","last_clean_scrub_stamp":"2024-04-25T10:30:11.733978+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T20:33:24.458243+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"5.32","version":"36'8","reported_seq":36,"reported_epoch":44,"state":"active+clean","last_fresh":"2024-04-25T10:30:30.800183+0000","last_change":"2024-04-25T10:30:12.772243+0000","last_active":"2024-04-25T10:30:30.800183+0000","last_peered":"2024-04-25T10:30:30.800183+0000","last_clean":"2024-04-25T10:30:30.800183+0000","last_became_active":"2024-04-25T10:30:12.771936+0000","last_became_peered":"2024-04-25T10:30:12.771936+0000","last_unstale":"2024-04-25T10:30:30.800183+0000","last_undegraded":"2024-04-25T10:30:30.800183+0000","last_fullsized":"2024-04-25T10:30:30.800183+0000","mapping_epoch":34,"log_start":"0'0","ondisk_log_start":"0'0","created":34,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:30:11.733978+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:30:11.733978+0000","last_clean_scrub_stamp":"2024-04-25T10:30:11.733978+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T14:03:40.703735+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5],"acting":[4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.33","version":"36'10","reported_seq":37,"reported_epoch":44,"state":"active+clean","last_fresh":"2024-04-25T10:30:29.883324+0000","last_change":"2024-04-25T10:30:12.777499+0000","last_active":"2024-04-25T10:30:29.883324+0000","last_peered":"2024-04-25T10:30:29.883324+0000","last_clean":"2024-04-25T10:30:29.883324+0000","last_became_active":"2024-04-25T10:30:12.776773+0000","last_became_peered":"2024-04-25T10:30:12.776773+0000","last_unstale":"2024-04-25T10:30:29.883324+0000","last_undegraded":"2024-04-25T10:30:29.883324+0000","last_fullsized":"2024-04-25T10:30:29.883324+0000","mapping_epoch":34,"log_start":"0'0","ondisk_log_start":"0'0","created":34,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:30:11.733978+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:30:11.733978+0000","last_clean_scrub_stamp":"2024-04-25T10:30:11.733978+0000","objects_scrubbed":0,"log_size":10,"log_dups_size":0,"ondisk_log_size":10,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T18:21:32.178524+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"5.34","version":"36'7","reported_seq":32,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-04-25T10:30:28.874618+0000","last_change":"2024-04-25T10:30:12.761162+0000","last_active":"2024-04-25T10:30:28.874618+0000","last_peered":"2024-04-25T10:30:28.874618+0000","last_clean":"2024-04-25T10:30:28.874618+0000","last_became_active":"2024-04-25T10:30:12.760835+0000","last_became_peered":"2024-04-25T10:30:12.760835+0000","last_unstale":"2024-04-25T10:30:28.874618+0000","last_undegraded":"2024-04-25T10:30:28.874618+0000","last_fullsized":"2024-04-25T10:30:28.874618+0000","mapping_epoch":34,"log_start":"0'0","ondisk_log_start":"0'0","created":34,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:30:11.733978+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:30:11.733978+0000","last_clean_scrub_stamp":"2024-04-25T10:30:11.733978+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T18:59:20.693761+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"5.35","version":"36'6","reported_seq":35,"reported_epoch":44,"state":"active+clean","last_fresh":"2024-04-25T10:30:29.882303+0000","last_change":"2024-04-25T10:30:12.776164+0000","last_active":"2024-04-25T10:30:29.882303+0000","last_peered":"2024-04-25T10:30:29.882303+0000","last_clean":"2024-04-25T10:30:29.882303+0000","last_became_active":"2024-04-25T10:30:12.775455+0000","last_became_peered":"2024-04-25T10:30:12.775455+0000","last_unstale":"2024-04-25T10:30:29.882303+0000","last_undegraded":"2024-04-25T10:30:29.882303+0000","last_fullsized":"2024-04-25T10:30:29.882303+0000","mapping_epoch":34,"log_start":"0'0","ondisk_log_start":"0'0","created":34,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:30:11.733978+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:30:11.733978+0000","last_clean_scrub_stamp":"2024-04-25T10:30:11.733978+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T21:45:24.780979+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.36","version":"36'15","reported_seq":43,"reported_epoch":44,"state":"active+clean","last_fresh":"2024-04-25T10:30:30.800175+0000","last_change":"2024-04-25T10:30:12.773138+0000","last_active":"2024-04-25T10:30:30.800175+0000","last_peered":"2024-04-25T10:30:30.800175+0000","last_clean":"2024-04-25T10:30:30.800175+0000","last_became_active":"2024-04-25T10:30:12.772854+0000","last_became_peered":"2024-04-25T10:30:12.772854+0000","last_unstale":"2024-04-25T10:30:30.800175+0000","last_undegraded":"2024-04-25T10:30:30.800175+0000","last_fullsized":"2024-04-25T10:30:30.800175+0000","mapping_epoch":34,"log_start":"0'0","ondisk_log_start":"0'0","created":34,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:30:11.733978+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:30:11.733978+0000","last_clean_scrub_stamp":"2024-04-25T10:30:11.733978+0000","objects_scrubbed":0,"log_size":15,"log_dups_size":0,"ondisk_log_size":15,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T12:33:38.324566+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5],"acting":[4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.37","version":"36'11","reported_seq":36,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-04-25T10:30:28.875064+0000","last_change":"2024-04-25T10:30:12.773184+0000","last_active":"2024-04-25T10:30:28.875064+0000","last_peered":"2024-04-25T10:30:28.875064+0000","last_clean":"2024-04-25T10:30:28.875064+0000","last_became_active":"2024-04-25T10:30:12.772866+0000","last_became_peered":"2024-04-25T10:30:12.772866+0000","last_unstale":"2024-04-25T10:30:28.875064+0000","last_undegraded":"2024-04-25T10:30:28.875064+0000","last_fullsized":"2024-04-25T10:30:28.875064+0000","mapping_epoch":34,"log_start":"0'0","ondisk_log_start":"0'0","created":34,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:30:11.733978+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:30:11.733978+0000","last_clean_scrub_stamp":"2024-04-25T10:30:11.733978+0000","objects_scrubbed":0,"log_size":11,"log_dups_size":0,"ondisk_log_size":11,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T14:01:06.791796+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"5.38","version":"36'9","reported_seq":38,"reported_epoch":44,"state":"active+clean","last_fresh":"2024-04-25T10:30:29.882611+0000","last_change":"2024-04-25T10:30:12.774614+0000","last_active":"2024-04-25T10:30:29.882611+0000","last_peered":"2024-04-25T10:30:29.882611+0000","last_clean":"2024-04-25T10:30:29.882611+0000","last_became_active":"2024-04-25T10:30:12.774213+0000","last_became_peered":"2024-04-25T10:30:12.774213+0000","last_unstale":"2024-04-25T10:30:29.882611+0000","last_undegraded":"2024-04-25T10:30:29.882611+0000","last_fullsized":"2024-04-25T10:30:29.882611+0000","mapping_epoch":34,"log_start":"0'0","ondisk_log_start":"0'0","created":34,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:30:11.733978+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:30:11.733978+0000","last_clean_scrub_stamp":"2024-04-25T10:30:11.733978+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T20:33:24.458243+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,6],"acting":[2,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"5.39","version":"36'5","reported_seq":34,"reported_epoch":44,"state":"active+clean","last_fresh":"2024-04-25T10:30:29.882763+0000","last_change":"2024-04-25T10:30:12.781210+0000","last_active":"2024-04-25T10:30:29.882763+0000","last_peered":"2024-04-25T10:30:29.882763+0000","last_clean":"2024-04-25T10:30:29.882763+0000","last_became_active":"2024-04-25T10:30:12.780874+0000","last_became_peered":"2024-04-25T10:30:12.780874+0000","last_unstale":"2024-04-25T10:30:29.882763+0000","last_undegraded":"2024-04-25T10:30:29.882763+0000","last_fullsized":"2024-04-25T10:30:29.882763+0000","mapping_epoch":34,"log_start":"0'0","ondisk_log_start":"0'0","created":34,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:30:11.733978+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:30:11.733978+0000","last_clean_scrub_stamp":"2024-04-25T10:30:11.733978+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T16:29:40.621322+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"5.3a","version":"36'6","reported_seq":30,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-04-25T10:30:28.874520+0000","last_change":"2024-04-25T10:30:12.775534+0000","last_active":"2024-04-25T10:30:28.874520+0000","last_peered":"2024-04-25T10:30:28.874520+0000","last_clean":"2024-04-25T10:30:28.874520+0000","last_became_active":"2024-04-25T10:30:12.775180+0000","last_became_peered":"2024-04-25T10:30:12.775180+0000","last_unstale":"2024-04-25T10:30:28.874520+0000","last_undegraded":"2024-04-25T10:30:28.874520+0000","last_fullsized":"2024-04-25T10:30:28.874520+0000","mapping_epoch":34,"log_start":"0'0","ondisk_log_start":"0'0","created":34,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:30:11.733978+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:30:11.733978+0000","last_clean_scrub_stamp":"2024-04-25T10:30:11.733978+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T18:31:49.260513+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"5.3b","version":"36'4","reported_seq":28,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-04-25T10:30:28.875655+0000","last_change":"2024-04-25T10:30:12.762141+0000","last_active":"2024-04-25T10:30:28.875655+0000","last_peered":"2024-04-25T10:30:28.875655+0000","last_clean":"2024-04-25T10:30:28.875655+0000","last_became_active":"2024-04-25T10:30:12.761805+0000","last_became_peered":"2024-04-25T10:30:12.761805+0000","last_unstale":"2024-04-25T10:30:28.875655+0000","last_undegraded":"2024-04-25T10:30:28.875655+0000","last_fullsized":"2024-04-25T10:30:28.875655+0000","mapping_epoch":34,"log_start":"0'0","ondisk_log_start":"0'0","created":34,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:30:11.733978+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:30:11.733978+0000","last_clean_scrub_stamp":"2024-04-25T10:30:11.733978+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T19:56:57.765496+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6],"acting":[5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"5.3c","version":"36'8","reported_seq":36,"reported_epoch":44,"state":"active+clean","last_fresh":"2024-04-25T10:30:30.800691+0000","last_change":"2024-04-25T10:30:12.781723+0000","last_active":"2024-04-25T10:30:30.800691+0000","last_peered":"2024-04-25T10:30:30.800691+0000","last_clean":"2024-04-25T10:30:30.800691+0000","last_became_active":"2024-04-25T10:30:12.781357+0000","last_became_peered":"2024-04-25T10:30:12.781357+0000","last_unstale":"2024-04-25T10:30:30.800691+0000","last_undegraded":"2024-04-25T10:30:30.800691+0000","last_fullsized":"2024-04-25T10:30:30.800691+0000","mapping_epoch":34,"log_start":"0'0","ondisk_log_start":"0'0","created":34,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:30:11.733978+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:30:11.733978+0000","last_clean_scrub_stamp":"2024-04-25T10:30:11.733978+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T15:28:31.731686+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.3d","version":"36'7","reported_seq":32,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-04-25T10:30:28.875135+0000","last_change":"2024-04-25T10:30:12.768292+0000","last_active":"2024-04-25T10:30:28.875135+0000","last_peered":"2024-04-25T10:30:28.875135+0000","last_clean":"2024-04-25T10:30:28.875135+0000","last_became_active":"2024-04-25T10:30:12.767909+0000","last_became_peered":"2024-04-25T10:30:12.767909+0000","last_unstale":"2024-04-25T10:30:28.875135+0000","last_undegraded":"2024-04-25T10:30:28.875135+0000","last_fullsized":"2024-04-25T10:30:28.875135+0000","mapping_epoch":34,"log_start":"0'0","ondisk_log_start":"0'0","created":34,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:30:11.733978+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:30:11.733978+0000","last_clean_scrub_stamp":"2024-04-25T10:30:11.733978+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T15:17:30.002665+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"5.3e","version":"36'4","reported_seq":28,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-04-25T10:30:28.878982+0000","last_change":"2024-04-25T10:30:12.780210+0000","last_active":"2024-04-25T10:30:28.878982+0000","last_peered":"2024-04-25T10:30:28.878982+0000","last_clean":"2024-04-25T10:30:28.878982+0000","last_became_active":"2024-04-25T10:30:12.779867+0000","last_became_peered":"2024-04-25T10:30:12.779867+0000","last_unstale":"2024-04-25T10:30:28.878982+0000","last_undegraded":"2024-04-25T10:30:28.878982+0000","last_fullsized":"2024-04-25T10:30:28.878982+0000","mapping_epoch":34,"log_start":"0'0","ondisk_log_start":"0'0","created":34,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:30:11.733978+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:30:11.733978+0000","last_clean_scrub_stamp":"2024-04-25T10:30:11.733978+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T20:33:24.458243+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"5.3f","version":"36'8","reported_seq":37,"reported_epoch":44,"state":"active+clean","last_fresh":"2024-04-25T10:30:29.883448+0000","last_change":"2024-04-25T10:30:12.772509+0000","last_active":"2024-04-25T10:30:29.883448+0000","last_peered":"2024-04-25T10:30:29.883448+0000","last_clean":"2024-04-25T10:30:29.883448+0000","last_became_active":"2024-04-25T10:30:12.772018+0000","last_became_peered":"2024-04-25T10:30:12.772018+0000","last_unstale":"2024-04-25T10:30:29.883448+0000","last_undegraded":"2024-04-25T10:30:29.883448+0000","last_fullsized":"2024-04-25T10:30:29.883448+0000","mapping_epoch":34,"log_start":"0'0","ondisk_log_start":"0'0","created":34,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:30:11.733978+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:30:11.733978+0000","last_clean_scrub_stamp":"2024-04-25T10:30:11.733978+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T11:57:29.979255+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]}],"pool_stats":[{"poolid":6,"num_pg":32,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":64,"acting":64,"num_store_stats":8},{"poolid":5,"num_pg":64,"stat_sum":{"num_bytes":6291456,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":6144,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":12582912,"data_stored":12582912,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":504,"ondisk_log_size":504,"up":128,"acting":128,"num_store_stats":8},{"poolid":4,"num_pg":32,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":1,"num_read_kb":0,"num_write":96,"num_write_kb":40,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":83,"ondisk_log_size":83,"up":64,"acting":64,"num_store_stats":8},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":66,"num_read_kb":55,"num_write":101,"num_write_kb":1628,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1187840,"data_stored":1180736,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":72,"ondisk_log_size":72,"up":2,"acting":2,"num_store_stats":2}],"osd_stats":[{"osd":7,"up_from":15,"seq":64424509462,"num_pgs":21,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":31596,"kb_used_data":4572,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94340244,"statfs":{"total":96636764160,"available":96604409856,"internally_reserved":0,"allocated":4681728,"data_stored":4394877,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1590,"internal_metadata":27654602},"hb_peers":[0,1,2,3,4,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]},{"osd":6,"up_from":15,"seq":64424509463,"num_pgs":25,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27504,"kb_used_data":476,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344336,"statfs":{"total":96636764160,"available":96608600064,"internally_reserved":0,"allocated":487424,"data_stored":200573,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":27654598},"hb_peers":[0,1,2,3,4,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]},{"osd":5,"up_from":15,"seq":64424509464,"num_pgs":18,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":28084,"kb_used_data":1056,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94343756,"statfs":{"total":96636764160,"available":96608006144,"internally_reserved":0,"allocated":1081344,"data_stored":790941,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]},{"osd":3,"up_from":15,"seq":64424509464,"num_pgs":25,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":33644,"kb_used_data":6620,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94338196,"statfs":{"total":96636764160,"available":96602312704,"internally_reserved":0,"allocated":6778880,"data_stored":6492029,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1593,"internal_metadata":27654599},"hb_peers":[0,1,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]},{"osd":2,"up_from":15,"seq":64424509464,"num_pgs":27,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":29548,"kb_used_data":2524,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94342292,"statfs":{"total":96636764160,"available":96606507008,"internally_reserved":0,"allocated":2584576,"data_stored":2297725,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]},{"osd":1,"up_from":15,"seq":64424509462,"num_pgs":25,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":28088,"kb_used_data":1056,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94343752,"statfs":{"total":96636764160,"available":96608002048,"internally_reserved":0,"allocated":1081344,"data_stored":790941,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]},{"osd":0,"up_from":15,"seq":64424509463,"num_pgs":21,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27504,"kb_used_data":476,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344336,"statfs":{"total":96636764160,"available":96608600064,"internally_reserved":0,"allocated":487424,"data_stored":200573,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[1,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]},{"osd":4,"up_from":13,"seq":55834574871,"num_pgs":45,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":31672,"kb_used_data":488,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94340168,"statfs":{"total":96636764160,"available":96604332032,"internally_reserved":0,"allocated":499712,"data_stored":207437,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[0,1,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":2097152,"data_stored":2097152,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":6291456,"data_stored":6291456,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":4194304,"data_stored":4194304,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-04-25T10:30:37.461 DEBUG:teuthology.orchestra.run.smithi037:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-04-25T10:30:37.588 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:37.585+0000 7f5ad7ee7640 1 Processor -- start 2024-04-25T10:30:37.588 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:37.585+0000 7f5ad7ee7640 1 -- start start 2024-04-25T10:30:37.589 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:37.587+0000 7f5ad7ee7640 1 --2- >> v2:172.21.15.203:3300/0 conn(0x7f5ad00b7cd0 0x7f5ad00d0d90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:30:37.589 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:37.587+0000 7f5ad7ee7640 1 -- --> v1:172.21.15.37:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f5ad00d5520 con 0x7f5ad00d0980 2024-04-25T10:30:37.589 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:37.587+0000 7f5ad7ee7640 1 -- --> v1:172.21.15.203:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f5ad00d1370 con 0x7f5ad00d5110 2024-04-25T10:30:37.590 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:37.587+0000 7f5ad7ee7640 1 -- --> v2:172.21.15.203:3300/0 -- mon_getmap magic: 0 -- 0x7f5ad00d4630 con 0x7f5ad00b7cd0 2024-04-25T10:30:37.590 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:37.587+0000 7f5ad5482640 1 --1- >> v1:172.21.15.37:6789/0 conn(0x7f5ad00d0980 0x7f5ad00b75c0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.37:6789/0 says I am v1:172.21.15.37:40928/0 (socket says 172.21.15.37:40928) 2024-04-25T10:30:37.590 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:37.587+0000 7f5ad5482640 1 -- 172.21.15.37:0/2899794603 learned_addr learned my addr 172.21.15.37:0/2899794603 (peer_addr_for_me v1:172.21.15.37:0/0) 2024-04-25T10:30:37.590 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:37.587+0000 7f5ad6484640 1 --2- 172.21.15.37:0/2899794603 >> v2:172.21.15.203:3300/0 conn(0x7f5ad00b7cd0 0x7f5ad00d0d90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:30:37.590 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:37.587+0000 7f5ad4c81640 1 -- 172.21.15.37:0/2899794603 <== mon.0 v1:172.21.15.37:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 174919601 0 0) 0x7f5ad00d5520 con 0x7f5ad00d0980 2024-04-25T10:30:37.590 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:37.587+0000 7f5ad4c81640 1 -- 172.21.15.37:0/2899794603 --> v1:172.21.15.37:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f5abc003520 con 0x7f5ad00d0980 2024-04-25T10:30:37.590 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:37.587+0000 7f5ad6484640 1 -- 172.21.15.37:0/2899794603 >> v1:172.21.15.203:6789/0 conn(0x7f5ad00d5110 legacy=0x7f5ad00b4d20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:30:37.590 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:37.587+0000 7f5ad4c81640 1 -- 172.21.15.37:0/2899794603 <== mon.0 v1:172.21.15.37:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 2176868664 0 0) 0x7f5abc003520 con 0x7f5ad00d0980 2024-04-25T10:30:37.590 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:37.587+0000 7f5ad6484640 1 -- 172.21.15.37:0/2899794603 >> v1:172.21.15.37:6789/0 conn(0x7f5ad00d0980 legacy=0x7f5ad00b75c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:30:37.590 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:37.587+0000 7f5ad6484640 1 -- 172.21.15.37:0/2899794603 --> v2:172.21.15.203:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f5ad00d4910 con 0x7f5ad00b7cd0 2024-04-25T10:30:37.591 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:37.588+0000 7f5ad6484640 1 --2- 172.21.15.37:0/2899794603 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f5ad00b7cd0 0x7f5ad00d0d90 secure :-1 s=READY pgs=211 cs=0 l=1 rev1=1 crypto rx=0x7f5ac8009d90 tx=0x7f5ac802a5b0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=e2d01d490f288ce4 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:30:37.591 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:37.588+0000 7f5ad4c81640 1 -- 172.21.15.37:0/2899794603 <== mon.1 v2:172.21.15.203:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5ac8035020 con 0x7f5ad00b7cd0 2024-04-25T10:30:37.592 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:37.589+0000 7f5ad4c81640 1 -- 172.21.15.37:0/2899794603 <== mon.1 v2:172.21.15.203:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f5ac8007540 con 0x7f5ad00b7cd0 2024-04-25T10:30:37.592 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:37.589+0000 7f5ad4c81640 1 -- 172.21.15.37:0/2899794603 <== mon.1 v2:172.21.15.203:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5ac0001670 con 0x7f5ad00b7cd0 2024-04-25T10:30:37.592 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:37.589+0000 7f5ad7ee7640 1 -- 172.21.15.37:0/2899794603 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f5ad00b7cd0 msgr2=0x7f5ad00d0d90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:30:37.592 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:37.589+0000 7f5ad7ee7640 1 --2- 172.21.15.37:0/2899794603 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f5ad00b7cd0 0x7f5ad00d0d90 secure :-1 s=READY pgs=211 cs=0 l=1 rev1=1 crypto rx=0x7f5ac8009d90 tx=0x7f5ac802a5b0 comp rx=0 tx=0).stop 2024-04-25T10:30:37.592 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:37.589+0000 7f5ad7ee7640 1 -- 172.21.15.37:0/2899794603 shutdown_connections 2024-04-25T10:30:37.592 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:37.589+0000 7f5ad7ee7640 1 --2- 172.21.15.37:0/2899794603 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f5ad00b7cd0 0x7f5ad00d0d90 unknown :-1 s=CLOSED pgs=211 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:30:37.592 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:37.589+0000 7f5ad7ee7640 1 -- 172.21.15.37:0/2899794603 >> 172.21.15.37:0/2899794603 conn(0x7f5ad00d27a0 msgr2=0x7f5ad00d2bb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:30:37.592 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:37.589+0000 7f5ad7ee7640 1 -- 172.21.15.37:0/2899794603 shutdown_connections 2024-04-25T10:30:37.592 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:37.589+0000 7f5ad7ee7640 1 -- 172.21.15.37:0/2899794603 wait complete. 2024-04-25T10:30:37.593 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:37.591+0000 7f5ad7ee7640 1 Processor -- start 2024-04-25T10:30:37.593 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:37.591+0000 7f5ad7ee7640 1 -- start start 2024-04-25T10:30:37.596 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:37.592+0000 7f5ad7ee7640 1 --2- >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f5ad00d0980 0x7f5ad00cbc30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:30:37.596 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:37.592+0000 7f5ad7ee7640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f5ad00d5110 0x7f5ad00cc170 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:30:37.596 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:37.592+0000 7f5ad7ee7640 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f5ad00cda50 0x7f5ad01f6100 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:30:37.596 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:37.592+0000 7f5ad7ee7640 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 -- 0x7f5ad0164ae0 con 0x7f5ad00d5110 2024-04-25T10:30:37.596 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:37.592+0000 7f5ad7ee7640 1 -- --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_getmap magic: 0 -- 0x7f5ad00d1690 con 0x7f5ad00cda50 2024-04-25T10:30:37.596 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:37.592+0000 7f5ad7ee7640 1 -- --> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] -- mon_getmap magic: 0 -- 0x7f5ad0164c60 con 0x7f5ad00d0980 2024-04-25T10:30:37.596 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:37.592+0000 7f5ad5482640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f5ad00d5110 0x7f5ad00cc170 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:30:37.596 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:37.593+0000 7f5ad5482640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f5ad00d5110 0x7f5ad00cc170 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.37:3300/0 says I am v2:172.21.15.37:55506/0 (socket says 172.21.15.37:55506) 2024-04-25T10:30:37.596 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:37.593+0000 7f5ad5482640 1 -- 172.21.15.37:0/3675460001 learned_addr learned my addr 172.21.15.37:0/3675460001 (peer_addr_for_me v2:172.21.15.37:0/0) 2024-04-25T10:30:37.596 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:37.593+0000 7f5ad6484640 1 --2- 172.21.15.37:0/3675460001 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f5ad00cda50 0x7f5ad01f6100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:30:37.597 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:37.593+0000 7f5ad5c83640 1 --2- 172.21.15.37:0/3675460001 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f5ad00d0980 0x7f5ad00cbc30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:30:37.597 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:37.593+0000 7f5ad5482640 1 -- 172.21.15.37:0/3675460001 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f5ad00d0980 msgr2=0x7f5ad00cbc30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:30:37.597 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:37.593+0000 7f5ad5482640 1 --2- 172.21.15.37:0/3675460001 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f5ad00d0980 0x7f5ad00cbc30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:30:37.597 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:37.593+0000 7f5ad5482640 1 -- 172.21.15.37:0/3675460001 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f5ad00cda50 msgr2=0x7f5ad01f6100 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:30:37.597 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:37.593+0000 7f5ad5482640 1 --2- 172.21.15.37:0/3675460001 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f5ad00cda50 0x7f5ad01f6100 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:30:37.597 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:37.593+0000 7f5ad5482640 1 -- 172.21.15.37:0/3675460001 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f5ad00cc9b0 con 0x7f5ad00d5110 2024-04-25T10:30:37.597 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:37.593+0000 7f5ad6484640 1 --2- 172.21.15.37:0/3675460001 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f5ad00cda50 0x7f5ad01f6100 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T10:30:37.597 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:37.593+0000 7f5ad5c83640 1 --2- 172.21.15.37:0/3675460001 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f5ad00d0980 0x7f5ad00cbc30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T10:30:37.597 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:37.593+0000 7f5ad5482640 1 --2- 172.21.15.37:0/3675460001 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f5ad00d5110 0x7f5ad00cc170 secure :-1 s=READY pgs=541 cs=0 l=1 rev1=1 crypto rx=0x7f5ac0007fd0 tx=0x7f5ac000c9b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:30:37.597 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:37.593+0000 7f5ac67fc640 1 -- 172.21.15.37:0/3675460001 <== mon.0 v2:172.21.15.37:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5ac0017070 con 0x7f5ad00d5110 2024-04-25T10:30:37.597 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:37.593+0000 7f5ac67fc640 1 -- 172.21.15.37:0/3675460001 <== mon.0 v2:172.21.15.37:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f5ac0005cc0 con 0x7f5ad00d5110 2024-04-25T10:30:37.597 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:37.593+0000 7f5ad7ee7640 1 -- 172.21.15.37:0/3675460001 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f5ad00ccc90 con 0x7f5ad00d5110 2024-04-25T10:30:37.597 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:37.593+0000 7f5ac67fc640 1 -- 172.21.15.37:0/3675460001 <== mon.0 v2:172.21.15.37:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5ac0007270 con 0x7f5ad00d5110 2024-04-25T10:30:37.598 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:37.593+0000 7f5ad7ee7640 1 -- 172.21.15.37:0/3675460001 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f5ad00cd250 con 0x7f5ad00d5110 2024-04-25T10:30:37.598 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:37.594+0000 7f5ad7ee7640 1 -- 172.21.15.37:0/3675460001 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f5ad00cc720 con 0x7f5ad00d5110 2024-04-25T10:30:37.599 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:37.595+0000 7f5ac67fc640 1 -- 172.21.15.37:0/3675460001 <== mon.0 v2:172.21.15.37:3300/0 4 ==== mgrmap(e 5) ==== 96782+0+0 (secure 0 0 0) 0x7f5ac001a070 con 0x7f5ad00d5110 2024-04-25T10:30:37.599 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:37.596+0000 7f5ac67fc640 1 --2- 172.21.15.37:0/3675460001 >> [v2:172.21.15.203:6832/2418205237,v1:172.21.15.203:6833/2418205237] conn(0x7f5ab40744e0 0x7f5ab40769f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:30:37.599 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:37.596+0000 7f5ac67fc640 1 -- 172.21.15.37:0/3675460001 <== mon.0 v2:172.21.15.37:3300/0 5 ==== osd_map(44..44 src has 1..44) ==== 6248+0+0 (secure 0 0 0) 0x7f5ac0094760 con 0x7f5ad00d5110 2024-04-25T10:30:37.599 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:37.596+0000 7f5ad5c83640 1 --2- 172.21.15.37:0/3675460001 >> [v2:172.21.15.203:6832/2418205237,v1:172.21.15.203:6833/2418205237] conn(0x7f5ab40744e0 0x7f5ab40769f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:30:37.599 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:37.597+0000 7f5ad5c83640 1 --2- 172.21.15.37:0/3675460001 >> [v2:172.21.15.203:6832/2418205237,v1:172.21.15.203:6833/2418205237] conn(0x7f5ab40744e0 0x7f5ab40769f0 secure :-1 s=READY pgs=259 cs=0 l=1 rev1=1 crypto rx=0x7f5acc007fd0 tx=0x7f5acc009040 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:30:37.602 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:37.600+0000 7f5ac67fc640 1 -- 172.21.15.37:0/3675460001 <== mon.0 v2:172.21.15.37:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187246 (secure 0 0 0) 0x7f5ac005e680 con 0x7f5ad00d5110 2024-04-25T10:30:37.784 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:37.782+0000 7f5ad7ee7640 1 -- 172.21.15.37:0/3675460001 --> [v2:172.21.15.203:6832/2418205237,v1:172.21.15.203:6833/2418205237] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) -- 0x7f5ad006a140 con 0x7f5ab40744e0 2024-04-25T10:30:37.792 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:37.791+0000 7f5ac67fc640 1 -- 172.21.15.37:0/3675460001 <== mgr.4100 v2:172.21.15.203:6832/2418205237 1 ==== mgr_command_reply(tid 0: 0 dumped all) ==== 18+0+332490 (secure 0 0 0) 0x7f5ad006a140 con 0x7f5ab40744e0 2024-04-25T10:30:37.796 INFO:teuthology.orchestra.run.smithi037.stdout: 2024-04-25T10:30:37.798 INFO:teuthology.orchestra.run.smithi037.stderr:dumped all 2024-04-25T10:30:37.800 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:37.794+0000 7f5ad7ee7640 1 -- 172.21.15.37:0/3675460001 >> [v2:172.21.15.203:6832/2418205237,v1:172.21.15.203:6833/2418205237] conn(0x7f5ab40744e0 msgr2=0x7f5ab40769f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:30:37.800 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:37.794+0000 7f5ad7ee7640 1 --2- 172.21.15.37:0/3675460001 >> [v2:172.21.15.203:6832/2418205237,v1:172.21.15.203:6833/2418205237] conn(0x7f5ab40744e0 0x7f5ab40769f0 secure :-1 s=READY pgs=259 cs=0 l=1 rev1=1 crypto rx=0x7f5acc007fd0 tx=0x7f5acc009040 comp rx=0 tx=0).stop 2024-04-25T10:30:37.800 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:37.794+0000 7f5ad7ee7640 1 -- 172.21.15.37:0/3675460001 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f5ad00d5110 msgr2=0x7f5ad00cc170 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:30:37.800 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:37.794+0000 7f5ad7ee7640 1 --2- 172.21.15.37:0/3675460001 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f5ad00d5110 0x7f5ad00cc170 secure :-1 s=READY pgs=541 cs=0 l=1 rev1=1 crypto rx=0x7f5ac0007fd0 tx=0x7f5ac000c9b0 comp rx=0 tx=0).stop 2024-04-25T10:30:37.800 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:37.794+0000 7f5ad7ee7640 1 -- 172.21.15.37:0/3675460001 shutdown_connections 2024-04-25T10:30:37.801 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:37.794+0000 7f5ad7ee7640 1 --2- 172.21.15.37:0/3675460001 >> [v2:172.21.15.203:6832/2418205237,v1:172.21.15.203:6833/2418205237] conn(0x7f5ab40744e0 0x7f5ab40769f0 unknown :-1 s=CLOSED pgs=259 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:30:37.801 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:37.794+0000 7f5ad7ee7640 1 --2- 172.21.15.37:0/3675460001 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f5ad00cda50 0x7f5ad01f6100 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:30:37.801 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:37.794+0000 7f5ad7ee7640 1 --2- 172.21.15.37:0/3675460001 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f5ad00d5110 0x7f5ad00cc170 unknown :-1 s=CLOSED pgs=541 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:30:37.801 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:37.794+0000 7f5ad7ee7640 1 --2- 172.21.15.37:0/3675460001 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f5ad00d0980 0x7f5ad00cbc30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:30:37.801 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:37.794+0000 7f5ad7ee7640 1 -- 172.21.15.37:0/3675460001 >> 172.21.15.37:0/3675460001 conn(0x7f5ad00d27a0 msgr2=0x7f5ad0162070 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:30:37.801 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:37.794+0000 7f5ad7ee7640 1 -- 172.21.15.37:0/3675460001 shutdown_connections 2024-04-25T10:30:37.801 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:37.794+0000 7f5ad7ee7640 1 -- 172.21.15.37:0/3675460001 wait complete. 2024-04-25T10:30:37.807 INFO:teuthology.orchestra.run.smithi037.stdout:{"pg_ready":true,"pg_map":{"version":89,"stamp":"2024-04-25T10:30:36.209062+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":6881824,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":67,"num_read_kb":55,"num_write":201,"num_write_kb":7812,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":659,"ondisk_log_size":659,"up":258,"acting":258,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":258,"num_osds":8,"num_per_pool_osds":8,"num_per_pool_omap_osds":8,"kb":754974720,"kb_used":237676,"kb_used_data":17352,"kb_used_omap":12,"kb_used_meta":220211,"kb_avail":754737044,"statfs":{"total":773094113280,"available":772850733056,"internally_reserved":0,"allocated":17768448,"data_stored":15423144,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":12752,"internal_metadata":225496624},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":6,"apply_latency_ms":6,"commit_latency_ns":6000000,"apply_latency_ns":6000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":-16365,"num_objects":-23,"num_object_clones":0,"num_object_copies":-46,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":-23,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":23,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":-15,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"8.002117"},"pg_stats":[{"pgid":"1.0","version":"18'72","reported_seq":147,"reported_epoch":44,"state":"active+clean","last_fresh":"2024-04-25T10:30:30.799141+0000","last_change":"2024-04-25T10:28:57.972696+0000","last_active":"2024-04-25T10:30:30.799141+0000","last_peered":"2024-04-25T10:30:30.799141+0000","last_clean":"2024-04-25T10:30:30.799141+0000","last_became_active":"2024-04-25T10:28:57.972373+0000","last_became_peered":"2024-04-25T10:28:57.972373+0000","last_unstale":"2024-04-25T10:30:30.799141+0000","last_undegraded":"2024-04-25T10:30:30.799141+0000","last_fullsized":"2024-04-25T10:30:30.799141+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:28:56.959439+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:28:56.959439+0000","last_clean_scrub_stamp":"2024-04-25T10:28:56.959439+0000","objects_scrubbed":0,"log_size":72,"log_dups_size":0,"ondisk_log_size":72,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T17:34:34.057408+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":66,"num_read_kb":55,"num_write":101,"num_write_kb":1628,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.5","version":"0'0","reported_seq":30,"reported_epoch":44,"state":"active+clean","last_fresh":"2024-04-25T10:30:29.882532+0000","last_change":"2024-04-25T10:30:11.759459+0000","last_active":"2024-04-25T10:30:29.882532+0000","last_peered":"2024-04-25T10:30:29.882532+0000","last_clean":"2024-04-25T10:30:29.882532+0000","last_became_active":"2024-04-25T10:30:11.757880+0000","last_became_peered":"2024-04-25T10:30:11.757880+0000","last_unstale":"2024-04-25T10:30:29.882532+0000","last_undegraded":"2024-04-25T10:30:29.882532+0000","last_fullsized":"2024-04-25T10:30:29.882532+0000","mapping_epoch":33,"log_start":"0'0","ondisk_log_start":"0'0","created":33,"last_epoch_clean":34,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:30:10.729165+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:30:10.729165+0000","last_clean_scrub_stamp":"2024-04-25T10:30:10.729165+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T18:21:31.173712+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.4","version":"36'9","reported_seq":35,"reported_epoch":44,"state":"active+clean","last_fresh":"2024-04-25T10:30:30.798765+0000","last_change":"2024-04-25T10:30:12.773161+0000","last_active":"2024-04-25T10:30:30.798765+0000","last_peered":"2024-04-25T10:30:30.798765+0000","last_clean":"2024-04-25T10:30:30.798765+0000","last_became_active":"2024-04-25T10:30:12.772831+0000","last_became_peered":"2024-04-25T10:30:12.772831+0000","last_unstale":"2024-04-25T10:30:30.798765+0000","last_undegraded":"2024-04-25T10:30:30.798765+0000","last_fullsized":"2024-04-25T10:30:30.798765+0000","mapping_epoch":34,"log_start":"0'0","ondisk_log_start":"0'0","created":34,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:30:11.733978+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:30:11.733978+0000","last_clean_scrub_stamp":"2024-04-25T10:30:11.733978+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T19:56:57.765496+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"6.7","version":"0'0","reported_seq":12,"reported_epoch":44,"state":"active+clean","last_fresh":"2024-04-25T10:30:29.882905+0000","last_change":"2024-04-25T10:30:28.881711+0000","last_active":"2024-04-25T10:30:29.882905+0000","last_peered":"2024-04-25T10:30:29.882905+0000","last_clean":"2024-04-25T10:30:29.882905+0000","last_became_active":"2024-04-25T10:30:28.881407+0000","last_became_peered":"2024-04-25T10:30:28.881407+0000","last_unstale":"2024-04-25T10:30:29.882905+0000","last_undegraded":"2024-04-25T10:30:29.882905+0000","last_fullsized":"2024-04-25T10:30:29.882905+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":42,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:30:27.860024+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:30:27.860024+0000","last_clean_scrub_stamp":"2024-04-25T10:30:27.860024+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T16:30:29.338622+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.0","version":"0'0","reported_seq":29,"reported_epoch":44,"state":"active+clean","last_fresh":"2024-04-25T10:30:30.800360+0000","last_change":"2024-04-25T10:30:11.761764+0000","last_active":"2024-04-25T10:30:30.800360+0000","last_peered":"2024-04-25T10:30:30.800360+0000","last_clean":"2024-04-25T10:30:30.800360+0000","last_became_active":"2024-04-25T10:30:11.761281+0000","last_became_peered":"2024-04-25T10:30:11.761281+0000","last_unstale":"2024-04-25T10:30:30.800360+0000","last_undegraded":"2024-04-25T10:30:30.800360+0000","last_fullsized":"2024-04-25T10:30:30.800360+0000","mapping_epoch":33,"log_start":"0'0","ondisk_log_start":"0'0","created":33,"last_epoch_clean":34,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:30:10.729165+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:30:10.729165+0000","last_clean_scrub_stamp":"2024-04-25T10:30:10.729165+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T13:05:33.012421+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.1","version":"36'5","reported_seq":34,"reported_epoch":44,"state":"active+clean","last_fresh":"2024-04-25T10:30:29.881926+0000","last_change":"2024-04-25T10:30:12.769006+0000","last_active":"2024-04-25T10:30:29.881926+0000","last_peered":"2024-04-25T10:30:29.881926+0000","last_clean":"2024-04-25T10:30:29.881926+0000","last_became_active":"2024-04-25T10:30:12.768671+0000","last_became_peered":"2024-04-25T10:30:12.768671+0000","last_unstale":"2024-04-25T10:30:29.881926+0000","last_undegraded":"2024-04-25T10:30:29.881926+0000","last_fullsized":"2024-04-25T10:30:29.881926+0000","mapping_epoch":34,"log_start":"0'0","ondisk_log_start":"0'0","created":34,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:30:11.733978+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:30:11.733978+0000","last_clean_scrub_stamp":"2024-04-25T10:30:11.733978+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T15:00:26.818072+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"6.2","version":"0'0","reported_seq":12,"reported_epoch":44,"state":"active+clean","last_fresh":"2024-04-25T10:30:29.882577+0000","last_change":"2024-04-25T10:30:28.875554+0000","last_active":"2024-04-25T10:30:29.882577+0000","last_peered":"2024-04-25T10:30:29.882577+0000","last_clean":"2024-04-25T10:30:29.882577+0000","last_became_active":"2024-04-25T10:30:28.875249+0000","last_became_peered":"2024-04-25T10:30:28.875249+0000","last_unstale":"2024-04-25T10:30:29.882577+0000","last_undegraded":"2024-04-25T10:30:29.882577+0000","last_fullsized":"2024-04-25T10:30:29.882577+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":42,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:30:27.860024+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:30:27.860024+0000","last_clean_scrub_stamp":"2024-04-25T10:30:27.860024+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T17:20:57.234773+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.1","version":"0'0","reported_seq":30,"reported_epoch":44,"state":"active+clean","last_fresh":"2024-04-25T10:30:29.882995+0000","last_change":"2024-04-25T10:30:11.765156+0000","last_active":"2024-04-25T10:30:29.882995+0000","last_peered":"2024-04-25T10:30:29.882995+0000","last_clean":"2024-04-25T10:30:29.882995+0000","last_became_active":"2024-04-25T10:30:11.764704+0000","last_became_peered":"2024-04-25T10:30:11.764704+0000","last_unstale":"2024-04-25T10:30:29.882995+0000","last_undegraded":"2024-04-25T10:30:29.882995+0000","last_fullsized":"2024-04-25T10:30:29.882995+0000","mapping_epoch":33,"log_start":"0'0","ondisk_log_start":"0'0","created":33,"last_epoch_clean":34,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:30:10.729165+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:30:10.729165+0000","last_clean_scrub_stamp":"2024-04-25T10:30:10.729165+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T19:02:53.584992+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"5.0","version":"36'8","reported_seq":36,"reported_epoch":44,"state":"active+clean","last_fresh":"2024-04-25T10:30:30.800119+0000","last_change":"2024-04-25T10:30:12.781642+0000","last_active":"2024-04-25T10:30:30.800119+0000","last_peered":"2024-04-25T10:30:30.800119+0000","last_clean":"2024-04-25T10:30:30.800119+0000","last_became_active":"2024-04-25T10:30:12.781303+0000","last_became_peered":"2024-04-25T10:30:12.781303+0000","last_unstale":"2024-04-25T10:30:30.800119+0000","last_undegraded":"2024-04-25T10:30:30.800119+0000","last_fullsized":"2024-04-25T10:30:30.800119+0000","mapping_epoch":34,"log_start":"0'0","ondisk_log_start":"0'0","created":34,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:30:11.733978+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:30:11.733978+0000","last_clean_scrub_stamp":"2024-04-25T10:30:11.733978+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T16:14:36.620049+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"6.3","version":"0'0","reported_seq":12,"reported_epoch":44,"state":"active+clean","last_fresh":"2024-04-25T10:30:30.799030+0000","last_change":"2024-04-25T10:30:28.883569+0000","last_active":"2024-04-25T10:30:30.799030+0000","last_peered":"2024-04-25T10:30:30.799030+0000","last_clean":"2024-04-25T10:30:30.799030+0000","last_became_active":"2024-04-25T10:30:28.883246+0000","last_became_peered":"2024-04-25T10:30:28.883246+0000","last_unstale":"2024-04-25T10:30:30.799030+0000","last_undegraded":"2024-04-25T10:30:30.799030+0000","last_fullsized":"2024-04-25T10:30:30.799030+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":42,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:30:27.860024+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:30:27.860024+0000","last_clean_scrub_stamp":"2024-04-25T10:30:27.860024+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T17:35:53.704785+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.2","version":"0'0","reported_seq":28,"reported_epoch":44,"state":"active+clean","last_fresh":"2024-04-25T10:30:29.882904+0000","last_change":"2024-04-25T10:30:11.760769+0000","last_active":"2024-04-25T10:30:29.882904+0000","last_peered":"2024-04-25T10:30:29.882904+0000","last_clean":"2024-04-25T10:30:29.882904+0000","last_became_active":"2024-04-25T10:30:11.760425+0000","last_became_peered":"2024-04-25T10:30:11.760425+0000","last_unstale":"2024-04-25T10:30:29.882904+0000","last_undegraded":"2024-04-25T10:30:29.882904+0000","last_fullsized":"2024-04-25T10:30:29.882904+0000","mapping_epoch":33,"log_start":"0'0","ondisk_log_start":"0'0","created":33,"last_epoch_clean":34,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:30:10.729165+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:30:10.729165+0000","last_clean_scrub_stamp":"2024-04-25T10:30:10.729165+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T17:24:13.848319+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"5.3","version":"36'7","reported_seq":35,"reported_epoch":44,"state":"active+clean","last_fresh":"2024-04-25T10:30:30.800298+0000","last_change":"2024-04-25T10:30:12.775959+0000","last_active":"2024-04-25T10:30:30.800298+0000","last_peered":"2024-04-25T10:30:30.800298+0000","last_clean":"2024-04-25T10:30:30.800298+0000","last_became_active":"2024-04-25T10:30:12.774532+0000","last_became_peered":"2024-04-25T10:30:12.774532+0000","last_unstale":"2024-04-25T10:30:30.800298+0000","last_undegraded":"2024-04-25T10:30:30.800298+0000","last_fullsized":"2024-04-25T10:30:30.800298+0000","mapping_epoch":34,"log_start":"0'0","ondisk_log_start":"0'0","created":34,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:30:11.733978+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:30:11.733978+0000","last_clean_scrub_stamp":"2024-04-25T10:30:11.733978+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T12:46:39.795946+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"6.0","version":"0'0","reported_seq":12,"reported_epoch":44,"state":"active+clean","last_fresh":"2024-04-25T10:30:30.798877+0000","last_change":"2024-04-25T10:30:28.883801+0000","last_active":"2024-04-25T10:30:30.798877+0000","last_peered":"2024-04-25T10:30:30.798877+0000","last_clean":"2024-04-25T10:30:30.798877+0000","last_became_active":"2024-04-25T10:30:28.883485+0000","last_became_peered":"2024-04-25T10:30:28.883485+0000","last_unstale":"2024-04-25T10:30:30.798877+0000","last_undegraded":"2024-04-25T10:30:30.798877+0000","last_fullsized":"2024-04-25T10:30:30.798877+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":42,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:30:27.860024+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:30:27.860024+0000","last_clean_scrub_stamp":"2024-04-25T10:30:27.860024+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T16:32:17.438458+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.3","version":"0'0","reported_seq":28,"reported_epoch":44,"state":"active+clean","last_fresh":"2024-04-25T10:30:29.883455+0000","last_change":"2024-04-25T10:30:11.762286+0000","last_active":"2024-04-25T10:30:29.883455+0000","last_peered":"2024-04-25T10:30:29.883455+0000","last_clean":"2024-04-25T10:30:29.883455+0000","last_became_active":"2024-04-25T10:30:11.761975+0000","last_became_peered":"2024-04-25T10:30:11.761975+0000","last_unstale":"2024-04-25T10:30:29.883455+0000","last_undegraded":"2024-04-25T10:30:29.883455+0000","last_fullsized":"2024-04-25T10:30:29.883455+0000","mapping_epoch":33,"log_start":"0'0","ondisk_log_start":"0'0","created":33,"last_epoch_clean":34,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:30:10.729165+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:30:10.729165+0000","last_clean_scrub_stamp":"2024-04-25T10:30:10.729165+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T16:09:04.081750+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"5.2","version":"36'8","reported_seq":34,"reported_epoch":44,"state":"active+clean","last_fresh":"2024-04-25T10:30:30.799391+0000","last_change":"2024-04-25T10:30:12.765903+0000","last_active":"2024-04-25T10:30:30.799391+0000","last_peered":"2024-04-25T10:30:30.799391+0000","last_clean":"2024-04-25T10:30:30.799391+0000","last_became_active":"2024-04-25T10:30:12.765611+0000","last_became_peered":"2024-04-25T10:30:12.765611+0000","last_unstale":"2024-04-25T10:30:30.799391+0000","last_undegraded":"2024-04-25T10:30:30.799391+0000","last_fullsized":"2024-04-25T10:30:30.799391+0000","mapping_epoch":34,"log_start":"0'0","ondisk_log_start":"0'0","created":34,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:30:11.733978+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:30:11.733978+0000","last_clean_scrub_stamp":"2024-04-25T10:30:11.733978+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T22:09:01.889527+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"6.1","version":"0'0","reported_seq":12,"reported_epoch":44,"state":"active+clean","last_fresh":"2024-04-25T10:30:29.882707+0000","last_change":"2024-04-25T10:30:28.884434+0000","last_active":"2024-04-25T10:30:29.882707+0000","last_peered":"2024-04-25T10:30:29.882707+0000","last_clean":"2024-04-25T10:30:29.882707+0000","last_became_active":"2024-04-25T10:30:28.883801+0000","last_became_peered":"2024-04-25T10:30:28.883801+0000","last_unstale":"2024-04-25T10:30:29.882707+0000","last_undegraded":"2024-04-25T10:30:29.882707+0000","last_fullsized":"2024-04-25T10:30:29.882707+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":42,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:30:27.860024+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:30:27.860024+0000","last_clean_scrub_stamp":"2024-04-25T10:30:27.860024+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T11:57:46.105302+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.4","version":"0'0","reported_seq":28,"reported_epoch":44,"state":"active+clean","last_fresh":"2024-04-25T10:30:29.883736+0000","last_change":"2024-04-25T10:30:11.757601+0000","last_active":"2024-04-25T10:30:29.883736+0000","last_peered":"2024-04-25T10:30:29.883736+0000","last_clean":"2024-04-25T10:30:29.883736+0000","last_became_active":"2024-04-25T10:30:11.756626+0000","last_became_peered":"2024-04-25T10:30:11.756626+0000","last_unstale":"2024-04-25T10:30:29.883736+0000","last_undegraded":"2024-04-25T10:30:29.883736+0000","last_fullsized":"2024-04-25T10:30:29.883736+0000","mapping_epoch":33,"log_start":"0'0","ondisk_log_start":"0'0","created":33,"last_epoch_clean":34,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:30:10.729165+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:30:10.729165+0000","last_clean_scrub_stamp":"2024-04-25T10:30:10.729165+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T13:03:08.676049+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"5.5","version":"36'5","reported_seq":31,"reported_epoch":44,"state":"active+clean","last_fresh":"2024-04-25T10:30:30.799684+0000","last_change":"2024-04-25T10:30:12.782521+0000","last_active":"2024-04-25T10:30:30.799684+0000","last_peered":"2024-04-25T10:30:30.799684+0000","last_clean":"2024-04-25T10:30:30.799684+0000","last_became_active":"2024-04-25T10:30:12.781802+0000","last_became_peered":"2024-04-25T10:30:12.781802+0000","last_unstale":"2024-04-25T10:30:30.799684+0000","last_undegraded":"2024-04-25T10:30:30.799684+0000","last_fullsized":"2024-04-25T10:30:30.799684+0000","mapping_epoch":34,"log_start":"0'0","ondisk_log_start":"0'0","created":34,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:30:11.733978+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:30:11.733978+0000","last_clean_scrub_stamp":"2024-04-25T10:30:11.733978+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T15:17:30.002665+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"6.6","version":"0'0","reported_seq":12,"reported_epoch":44,"state":"active+clean","last_fresh":"2024-04-25T10:30:30.799093+0000","last_change":"2024-04-25T10:30:28.885896+0000","last_active":"2024-04-25T10:30:30.799093+0000","last_peered":"2024-04-25T10:30:30.799093+0000","last_clean":"2024-04-25T10:30:30.799093+0000","last_became_active":"2024-04-25T10:30:28.885575+0000","last_became_peered":"2024-04-25T10:30:28.885575+0000","last_unstale":"2024-04-25T10:30:30.799093+0000","last_undegraded":"2024-04-25T10:30:30.799093+0000","last_fullsized":"2024-04-25T10:30:30.799093+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":42,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:30:27.860024+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:30:27.860024+0000","last_clean_scrub_stamp":"2024-04-25T10:30:27.860024+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T15:39:48.016111+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.6","version":"37'4","reported_seq":31,"reported_epoch":44,"state":"active+clean","last_fresh":"2024-04-25T10:30:30.800176+0000","last_change":"2024-04-25T10:30:11.750312+0000","last_active":"2024-04-25T10:30:30.800176+0000","last_peered":"2024-04-25T10:30:30.800176+0000","last_clean":"2024-04-25T10:30:30.800176+0000","last_became_active":"2024-04-25T10:30:11.750024+0000","last_became_peered":"2024-04-25T10:30:11.750024+0000","last_unstale":"2024-04-25T10:30:30.800176+0000","last_undegraded":"2024-04-25T10:30:30.800176+0000","last_fullsized":"2024-04-25T10:30:30.800176+0000","mapping_epoch":33,"log_start":"0'0","ondisk_log_start":"0'0","created":33,"last_epoch_clean":34,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:30:10.729165+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:30:10.729165+0000","last_clean_scrub_stamp":"2024-04-25T10:30:10.729165+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T16:48:59.263610+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"5.7","version":"36'8","reported_seq":37,"reported_epoch":44,"state":"active+clean","last_fresh":"2024-04-25T10:30:29.882715+0000","last_change":"2024-04-25T10:30:12.768932+0000","last_active":"2024-04-25T10:30:29.882715+0000","last_peered":"2024-04-25T10:30:29.882715+0000","last_clean":"2024-04-25T10:30:29.882715+0000","last_became_active":"2024-04-25T10:30:12.768609+0000","last_became_peered":"2024-04-25T10:30:12.768609+0000","last_unstale":"2024-04-25T10:30:29.882715+0000","last_undegraded":"2024-04-25T10:30:29.882715+0000","last_fullsized":"2024-04-25T10:30:29.882715+0000","mapping_epoch":34,"log_start":"0'0","ondisk_log_start":"0'0","created":34,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:30:11.733978+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:30:11.733978+0000","last_clean_scrub_stamp":"2024-04-25T10:30:11.733978+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T11:24:05.006967+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":4194304,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":4096,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"6.4","version":"0'0","reported_seq":12,"reported_epoch":44,"state":"active+clean","last_fresh":"2024-04-25T10:30:30.799544+0000","last_change":"2024-04-25T10:30:28.880128+0000","last_active":"2024-04-25T10:30:30.799544+0000","last_peered":"2024-04-25T10:30:30.799544+0000","last_clean":"2024-04-25T10:30:30.799544+0000","last_became_active":"2024-04-25T10:30:28.879771+0000","last_became_peered":"2024-04-25T10:30:28.879771+0000","last_unstale":"2024-04-25T10:30:30.799544+0000","last_undegraded":"2024-04-25T10:30:30.799544+0000","last_fullsized":"2024-04-25T10:30:30.799544+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":42,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:30:27.860024+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:30:27.860024+0000","last_clean_scrub_stamp":"2024-04-25T10:30:27.860024+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T12:28:10.690739+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.7","version":"37'3","reported_seq":31,"reported_epoch":44,"state":"active+clean","last_fresh":"2024-04-25T10:30:29.883302+0000","last_change":"2024-04-25T10:30:11.756948+0000","last_active":"2024-04-25T10:30:29.883302+0000","last_peered":"2024-04-25T10:30:29.883302+0000","last_clean":"2024-04-25T10:30:29.883302+0000","last_became_active":"2024-04-25T10:30:11.756607+0000","last_became_peered":"2024-04-25T10:30:11.756607+0000","last_unstale":"2024-04-25T10:30:29.883302+0000","last_undegraded":"2024-04-25T10:30:29.883302+0000","last_fullsized":"2024-04-25T10:30:29.883302+0000","mapping_epoch":33,"log_start":"0'0","ondisk_log_start":"0'0","created":33,"last_epoch_clean":34,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:30:10.729165+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:30:10.729165+0000","last_clean_scrub_stamp":"2024-04-25T10:30:10.729165+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T17:15:13.597075+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"5.6","version":"36'7","reported_seq":33,"reported_epoch":44,"state":"active+clean","last_fresh":"2024-04-25T10:30:30.799445+0000","last_change":"2024-04-25T10:30:12.772456+0000","last_active":"2024-04-25T10:30:30.799445+0000","last_peered":"2024-04-25T10:30:30.799445+0000","last_clean":"2024-04-25T10:30:30.799445+0000","last_became_active":"2024-04-25T10:30:12.772070+0000","last_became_peered":"2024-04-25T10:30:12.772070+0000","last_unstale":"2024-04-25T10:30:30.799445+0000","last_undegraded":"2024-04-25T10:30:30.799445+0000","last_fullsized":"2024-04-25T10:30:30.799445+0000","mapping_epoch":34,"log_start":"0'0","ondisk_log_start":"0'0","created":34,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:30:11.733978+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:30:11.733978+0000","last_clean_scrub_stamp":"2024-04-25T10:30:11.733978+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T16:08:29.987819+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"6.5","version":"0'0","reported_seq":12,"reported_epoch":44,"state":"active+clean","last_fresh":"2024-04-25T10:30:30.799259+0000","last_change":"2024-04-25T10:30:28.881260+0000","last_active":"2024-04-25T10:30:30.799259+0000","last_peered":"2024-04-25T10:30:30.799259+0000","last_clean":"2024-04-25T10:30:30.799259+0000","last_became_active":"2024-04-25T10:30:28.880995+0000","last_became_peered":"2024-04-25T10:30:28.880995+0000","last_unstale":"2024-04-25T10:30:30.799259+0000","last_undegraded":"2024-04-25T10:30:30.799259+0000","last_fullsized":"2024-04-25T10:30:30.799259+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":42,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:30:27.860024+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:30:27.860024+0000","last_clean_scrub_stamp":"2024-04-25T10:30:27.860024+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T20:27:40.417464+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.8","version":"0'0","reported_seq":30,"reported_epoch":44,"state":"active+clean","last_fresh":"2024-04-25T10:30:29.881839+0000","last_change":"2024-04-25T10:30:11.758581+0000","last_active":"2024-04-25T10:30:29.881839+0000","last_peered":"2024-04-25T10:30:29.881839+0000","last_clean":"2024-04-25T10:30:29.881839+0000","last_became_active":"2024-04-25T10:30:11.758239+0000","last_became_peered":"2024-04-25T10:30:11.758239+0000","last_unstale":"2024-04-25T10:30:29.881839+0000","last_undegraded":"2024-04-25T10:30:29.881839+0000","last_fullsized":"2024-04-25T10:30:29.881839+0000","mapping_epoch":33,"log_start":"0'0","ondisk_log_start":"0'0","created":33,"last_epoch_clean":34,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:30:10.729165+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:30:10.729165+0000","last_clean_scrub_stamp":"2024-04-25T10:30:10.729165+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T17:41:33.905382+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.9","version":"36'2","reported_seq":29,"reported_epoch":44,"state":"active+clean","last_fresh":"2024-04-25T10:30:30.799369+0000","last_change":"2024-04-25T10:30:12.768782+0000","last_active":"2024-04-25T10:30:30.799369+0000","last_peered":"2024-04-25T10:30:30.799369+0000","last_clean":"2024-04-25T10:30:30.799369+0000","last_became_active":"2024-04-25T10:30:12.768464+0000","last_became_peered":"2024-04-25T10:30:12.768464+0000","last_unstale":"2024-04-25T10:30:30.799369+0000","last_undegraded":"2024-04-25T10:30:30.799369+0000","last_fullsized":"2024-04-25T10:30:30.799369+0000","mapping_epoch":34,"log_start":"0'0","ondisk_log_start":"0'0","created":34,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:30:11.733978+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:30:11.733978+0000","last_clean_scrub_stamp":"2024-04-25T10:30:11.733978+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T13:03:21.170197+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"6.a","version":"0'0","reported_seq":12,"reported_epoch":44,"state":"active+clean","last_fresh":"2024-04-25T10:30:30.798627+0000","last_change":"2024-04-25T10:30:28.882147+0000","last_active":"2024-04-25T10:30:30.798627+0000","last_peered":"2024-04-25T10:30:30.798627+0000","last_clean":"2024-04-25T10:30:30.798627+0000","last_became_active":"2024-04-25T10:30:28.881783+0000","last_became_peered":"2024-04-25T10:30:28.881783+0000","last_unstale":"2024-04-25T10:30:30.798627+0000","last_undegraded":"2024-04-25T10:30:30.798627+0000","last_fullsized":"2024-04-25T10:30:30.798627+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":42,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:30:27.860024+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:30:27.860024+0000","last_clean_scrub_stamp":"2024-04-25T10:30:27.860024+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T14:08:51.883019+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.9","version":"0'0","reported_seq":28,"reported_epoch":44,"state":"active+clean","last_fresh":"2024-04-25T10:30:30.799521+0000","last_change":"2024-04-25T10:30:11.759807+0000","last_active":"2024-04-25T10:30:30.799521+0000","last_peered":"2024-04-25T10:30:30.799521+0000","last_clean":"2024-04-25T10:30:30.799521+0000","last_became_active":"2024-04-25T10:30:11.758558+0000","last_became_peered":"2024-04-25T10:30:11.758558+0000","last_unstale":"2024-04-25T10:30:30.799521+0000","last_undegraded":"2024-04-25T10:30:30.799521+0000","last_fullsized":"2024-04-25T10:30:30.799521+0000","mapping_epoch":33,"log_start":"0'0","ondisk_log_start":"0'0","created":33,"last_epoch_clean":34,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:30:10.729165+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:30:10.729165+0000","last_clean_scrub_stamp":"2024-04-25T10:30:10.729165+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T20:38:23.460836+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"5.8","version":"36'9","reported_seq":36,"reported_epoch":44,"state":"active+clean","last_fresh":"2024-04-25T10:30:29.882627+0000","last_change":"2024-04-25T10:30:12.777060+0000","last_active":"2024-04-25T10:30:29.882627+0000","last_peered":"2024-04-25T10:30:29.882627+0000","last_clean":"2024-04-25T10:30:29.882627+0000","last_became_active":"2024-04-25T10:30:12.776749+0000","last_became_peered":"2024-04-25T10:30:12.776749+0000","last_unstale":"2024-04-25T10:30:29.882627+0000","last_undegraded":"2024-04-25T10:30:29.882627+0000","last_fullsized":"2024-04-25T10:30:29.882627+0000","mapping_epoch":34,"log_start":"0'0","ondisk_log_start":"0'0","created":34,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:30:11.733978+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:30:11.733978+0000","last_clean_scrub_stamp":"2024-04-25T10:30:11.733978+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T15:47:40.336983+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"6.b","version":"0'0","reported_seq":12,"reported_epoch":44,"state":"active+clean","last_fresh":"2024-04-25T10:30:30.799735+0000","last_change":"2024-04-25T10:30:28.881916+0000","last_active":"2024-04-25T10:30:30.799735+0000","last_peered":"2024-04-25T10:30:30.799735+0000","last_clean":"2024-04-25T10:30:30.799735+0000","last_became_active":"2024-04-25T10:30:28.881605+0000","last_became_peered":"2024-04-25T10:30:28.881605+0000","last_unstale":"2024-04-25T10:30:30.799735+0000","last_undegraded":"2024-04-25T10:30:30.799735+0000","last_fullsized":"2024-04-25T10:30:30.799735+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":42,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:30:27.860024+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:30:27.860024+0000","last_clean_scrub_stamp":"2024-04-25T10:30:27.860024+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T15:18:43.432122+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.a","version":"0'0","reported_seq":30,"reported_epoch":44,"state":"active+clean","last_fresh":"2024-04-25T10:30:29.883112+0000","last_change":"2024-04-25T10:30:11.762745+0000","last_active":"2024-04-25T10:30:29.883112+0000","last_peered":"2024-04-25T10:30:29.883112+0000","last_clean":"2024-04-25T10:30:29.883112+0000","last_became_active":"2024-04-25T10:30:11.762377+0000","last_became_peered":"2024-04-25T10:30:11.762377+0000","last_unstale":"2024-04-25T10:30:29.883112+0000","last_undegraded":"2024-04-25T10:30:29.883112+0000","last_fullsized":"2024-04-25T10:30:29.883112+0000","mapping_epoch":33,"log_start":"0'0","ondisk_log_start":"0'0","created":33,"last_epoch_clean":34,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:30:10.729165+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:30:10.729165+0000","last_clean_scrub_stamp":"2024-04-25T10:30:10.729165+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T17:15:13.597075+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"5.b","version":"36'8","reported_seq":34,"reported_epoch":44,"state":"active+clean","last_fresh":"2024-04-25T10:30:30.799019+0000","last_change":"2024-04-25T10:30:12.766544+0000","last_active":"2024-04-25T10:30:30.799019+0000","last_peered":"2024-04-25T10:30:30.799019+0000","last_clean":"2024-04-25T10:30:30.799019+0000","last_became_active":"2024-04-25T10:30:12.766228+0000","last_became_peered":"2024-04-25T10:30:12.766228+0000","last_unstale":"2024-04-25T10:30:30.799019+0000","last_undegraded":"2024-04-25T10:30:30.799019+0000","last_fullsized":"2024-04-25T10:30:30.799019+0000","mapping_epoch":34,"log_start":"0'0","ondisk_log_start":"0'0","created":34,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:30:11.733978+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:30:11.733978+0000","last_clean_scrub_stamp":"2024-04-25T10:30:11.733978+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T18:00:26.511360+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"6.8","version":"0'0","reported_seq":12,"reported_epoch":44,"state":"active+clean","last_fresh":"2024-04-25T10:30:29.881772+0000","last_change":"2024-04-25T10:30:28.881118+0000","last_active":"2024-04-25T10:30:29.881772+0000","last_peered":"2024-04-25T10:30:29.881772+0000","last_clean":"2024-04-25T10:30:29.881772+0000","last_became_active":"2024-04-25T10:30:28.880810+0000","last_became_peered":"2024-04-25T10:30:28.880810+0000","last_unstale":"2024-04-25T10:30:29.881772+0000","last_undegraded":"2024-04-25T10:30:29.881772+0000","last_fullsized":"2024-04-25T10:30:29.881772+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":42,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:30:27.860024+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:30:27.860024+0000","last_clean_scrub_stamp":"2024-04-25T10:30:27.860024+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T15:53:38.989476+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.b","version":"37'4","reported_seq":31,"reported_epoch":44,"state":"active+clean","last_fresh":"2024-04-25T10:30:30.799159+0000","last_change":"2024-04-25T10:30:11.752902+0000","last_active":"2024-04-25T10:30:30.799159+0000","last_peered":"2024-04-25T10:30:30.799159+0000","last_clean":"2024-04-25T10:30:30.799159+0000","last_became_active":"2024-04-25T10:30:11.752599+0000","last_became_peered":"2024-04-25T10:30:11.752599+0000","last_unstale":"2024-04-25T10:30:30.799159+0000","last_undegraded":"2024-04-25T10:30:30.799159+0000","last_fullsized":"2024-04-25T10:30:30.799159+0000","mapping_epoch":33,"log_start":"0'0","ondisk_log_start":"0'0","created":33,"last_epoch_clean":34,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:30:10.729165+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:30:10.729165+0000","last_clean_scrub_stamp":"2024-04-25T10:30:10.729165+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T14:01:05.786983+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"5.a","version":"36'5","reported_seq":32,"reported_epoch":44,"state":"active+clean","last_fresh":"2024-04-25T10:30:29.882793+0000","last_change":"2024-04-25T10:30:12.762940+0000","last_active":"2024-04-25T10:30:29.882793+0000","last_peered":"2024-04-25T10:30:29.882793+0000","last_clean":"2024-04-25T10:30:29.882793+0000","last_became_active":"2024-04-25T10:30:12.762628+0000","last_became_peered":"2024-04-25T10:30:12.762628+0000","last_unstale":"2024-04-25T10:30:29.882793+0000","last_undegraded":"2024-04-25T10:30:29.882793+0000","last_fullsized":"2024-04-25T10:30:29.882793+0000","mapping_epoch":34,"log_start":"0'0","ondisk_log_start":"0'0","created":34,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:30:11.733978+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:30:11.733978+0000","last_clean_scrub_stamp":"2024-04-25T10:30:11.733978+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T19:25:06.397830+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"6.9","version":"0'0","reported_seq":12,"reported_epoch":44,"state":"active+clean","last_fresh":"2024-04-25T10:30:30.799857+0000","last_change":"2024-04-25T10:30:28.885090+0000","last_active":"2024-04-25T10:30:30.799857+0000","last_peered":"2024-04-25T10:30:30.799857+0000","last_clean":"2024-04-25T10:30:30.799857+0000","last_became_active":"2024-04-25T10:30:28.882960+0000","last_became_peered":"2024-04-25T10:30:28.882960+0000","last_unstale":"2024-04-25T10:30:30.799857+0000","last_undegraded":"2024-04-25T10:30:30.799857+0000","last_fullsized":"2024-04-25T10:30:30.799857+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":42,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:30:27.860024+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:30:27.860024+0000","last_clean_scrub_stamp":"2024-04-25T10:30:27.860024+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T17:25:53.823227+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.c","version":"0'0","reported_seq":27,"reported_epoch":44,"state":"active+clean","last_fresh":"2024-04-25T10:30:30.798875+0000","last_change":"2024-04-25T10:30:11.753067+0000","last_active":"2024-04-25T10:30:30.798875+0000","last_peered":"2024-04-25T10:30:30.798875+0000","last_clean":"2024-04-25T10:30:30.798875+0000","last_became_active":"2024-04-25T10:30:11.752595+0000","last_became_peered":"2024-04-25T10:30:11.752595+0000","last_unstale":"2024-04-25T10:30:30.798875+0000","last_undegraded":"2024-04-25T10:30:30.798875+0000","last_fullsized":"2024-04-25T10:30:30.798875+0000","mapping_epoch":33,"log_start":"0'0","ondisk_log_start":"0'0","created":33,"last_epoch_clean":34,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:30:10.729165+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:30:10.729165+0000","last_clean_scrub_stamp":"2024-04-25T10:30:10.729165+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T18:59:19.688948+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"5.d","version":"36'7","reported_seq":33,"reported_epoch":44,"state":"active+clean","last_fresh":"2024-04-25T10:30:30.799693+0000","last_change":"2024-04-25T10:30:12.771743+0000","last_active":"2024-04-25T10:30:30.799693+0000","last_peered":"2024-04-25T10:30:30.799693+0000","last_clean":"2024-04-25T10:30:30.799693+0000","last_became_active":"2024-04-25T10:30:12.771387+0000","last_became_peered":"2024-04-25T10:30:12.771387+0000","last_unstale":"2024-04-25T10:30:30.799693+0000","last_undegraded":"2024-04-25T10:30:30.799693+0000","last_fullsized":"2024-04-25T10:30:30.799693+0000","mapping_epoch":34,"log_start":"0'0","ondisk_log_start":"0'0","created":34,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:30:11.733978+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:30:11.733978+0000","last_clean_scrub_stamp":"2024-04-25T10:30:11.733978+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T14:01:06.791796+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"6.e","version":"0'0","reported_seq":12,"reported_epoch":44,"state":"active+clean","last_fresh":"2024-04-25T10:30:30.799515+0000","last_change":"2024-04-25T10:30:28.884467+0000","last_active":"2024-04-25T10:30:30.799515+0000","last_peered":"2024-04-25T10:30:30.799515+0000","last_clean":"2024-04-25T10:30:30.799515+0000","last_became_active":"2024-04-25T10:30:28.884173+0000","last_became_peered":"2024-04-25T10:30:28.884173+0000","last_unstale":"2024-04-25T10:30:30.799515+0000","last_undegraded":"2024-04-25T10:30:30.799515+0000","last_fullsized":"2024-04-25T10:30:30.799515+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":42,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:30:27.860024+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:30:27.860024+0000","last_clean_scrub_stamp":"2024-04-25T10:30:27.860024+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T14:16:58.845516+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.d","version":"37'2","reported_seq":32,"reported_epoch":44,"state":"active+clean","last_fresh":"2024-04-25T10:30:29.882419+0000","last_change":"2024-04-25T10:30:11.758390+0000","last_active":"2024-04-25T10:30:29.882419+0000","last_peered":"2024-04-25T10:30:29.882419+0000","last_clean":"2024-04-25T10:30:29.882419+0000","last_became_active":"2024-04-25T10:30:11.757989+0000","last_became_peered":"2024-04-25T10:30:11.757989+0000","last_unstale":"2024-04-25T10:30:29.882419+0000","last_undegraded":"2024-04-25T10:30:29.882419+0000","last_fullsized":"2024-04-25T10:30:29.882419+0000","mapping_epoch":33,"log_start":"0'0","ondisk_log_start":"0'0","created":33,"last_epoch_clean":34,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:30:10.729165+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:30:10.729165+0000","last_clean_scrub_stamp":"2024-04-25T10:30:10.729165+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T18:52:44.534946+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.c","version":"36'6","reported_seq":32,"reported_epoch":44,"state":"active+clean","last_fresh":"2024-04-25T10:30:30.798991+0000","last_change":"2024-04-25T10:30:12.765790+0000","last_active":"2024-04-25T10:30:30.798991+0000","last_peered":"2024-04-25T10:30:30.798991+0000","last_clean":"2024-04-25T10:30:30.798991+0000","last_became_active":"2024-04-25T10:30:12.765468+0000","last_became_peered":"2024-04-25T10:30:12.765468+0000","last_unstale":"2024-04-25T10:30:30.798991+0000","last_undegraded":"2024-04-25T10:30:30.798991+0000","last_fullsized":"2024-04-25T10:30:30.798991+0000","mapping_epoch":34,"log_start":"0'0","ondisk_log_start":"0'0","created":34,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:30:11.733978+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:30:11.733978+0000","last_clean_scrub_stamp":"2024-04-25T10:30:11.733978+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T16:29:40.621322+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"6.f","version":"0'0","reported_seq":12,"reported_epoch":44,"state":"active+clean","last_fresh":"2024-04-25T10:30:30.799276+0000","last_change":"2024-04-25T10:30:28.886506+0000","last_active":"2024-04-25T10:30:30.799276+0000","last_peered":"2024-04-25T10:30:30.799276+0000","last_clean":"2024-04-25T10:30:30.799276+0000","last_became_active":"2024-04-25T10:30:28.886157+0000","last_became_peered":"2024-04-25T10:30:28.886157+0000","last_unstale":"2024-04-25T10:30:30.799276+0000","last_undegraded":"2024-04-25T10:30:30.799276+0000","last_fullsized":"2024-04-25T10:30:30.799276+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":42,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:30:27.860024+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:30:27.860024+0000","last_clean_scrub_stamp":"2024-04-25T10:30:27.860024+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T17:35:49.687185+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.e","version":"37'3","reported_seq":33,"reported_epoch":44,"state":"active+clean","last_fresh":"2024-04-25T10:30:29.883403+0000","last_change":"2024-04-25T10:30:11.765106+0000","last_active":"2024-04-25T10:30:29.883403+0000","last_peered":"2024-04-25T10:30:29.883403+0000","last_clean":"2024-04-25T10:30:29.883403+0000","last_became_active":"2024-04-25T10:30:11.764710+0000","last_became_peered":"2024-04-25T10:30:11.764710+0000","last_unstale":"2024-04-25T10:30:29.883403+0000","last_undegraded":"2024-04-25T10:30:29.883403+0000","last_fullsized":"2024-04-25T10:30:29.883403+0000","mapping_epoch":33,"log_start":"0'0","ondisk_log_start":"0'0","created":33,"last_epoch_clean":34,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:30:10.729165+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:30:10.729165+0000","last_clean_scrub_stamp":"2024-04-25T10:30:10.729165+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T16:30:11.709416+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"5.f","version":"36'9","reported_seq":36,"reported_epoch":44,"state":"active+clean","last_fresh":"2024-04-25T10:30:30.799889+0000","last_change":"2024-04-25T10:30:12.769270+0000","last_active":"2024-04-25T10:30:30.799889+0000","last_peered":"2024-04-25T10:30:30.799889+0000","last_clean":"2024-04-25T10:30:30.799889+0000","last_became_active":"2024-04-25T10:30:12.768931+0000","last_became_peered":"2024-04-25T10:30:12.768931+0000","last_unstale":"2024-04-25T10:30:30.799889+0000","last_undegraded":"2024-04-25T10:30:30.799889+0000","last_fullsized":"2024-04-25T10:30:30.799889+0000","mapping_epoch":34,"log_start":"0'0","ondisk_log_start":"0'0","created":34,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:30:11.733978+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:30:11.733978+0000","last_clean_scrub_stamp":"2024-04-25T10:30:11.733978+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T16:49:00.268423+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"6.c","version":"0'0","reported_seq":12,"reported_epoch":44,"state":"active+clean","last_fresh":"2024-04-25T10:30:29.883642+0000","last_change":"2024-04-25T10:30:28.887055+0000","last_active":"2024-04-25T10:30:29.883642+0000","last_peered":"2024-04-25T10:30:29.883642+0000","last_clean":"2024-04-25T10:30:29.883642+0000","last_became_active":"2024-04-25T10:30:28.886675+0000","last_became_peered":"2024-04-25T10:30:28.886675+0000","last_unstale":"2024-04-25T10:30:29.883642+0000","last_undegraded":"2024-04-25T10:30:29.883642+0000","last_fullsized":"2024-04-25T10:30:29.883642+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":42,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:30:27.860024+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:30:27.860024+0000","last_clean_scrub_stamp":"2024-04-25T10:30:27.860024+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T14:04:58.646968+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,1],"acting":[6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.f","version":"37'2","reported_seq":29,"reported_epoch":44,"state":"active+clean","last_fresh":"2024-04-25T10:30:30.799567+0000","last_change":"2024-04-25T10:30:11.750348+0000","last_active":"2024-04-25T10:30:30.799567+0000","last_peered":"2024-04-25T10:30:30.799567+0000","last_clean":"2024-04-25T10:30:30.799567+0000","last_became_active":"2024-04-25T10:30:11.749970+0000","last_became_peered":"2024-04-25T10:30:11.749970+0000","last_unstale":"2024-04-25T10:30:30.799567+0000","last_undegraded":"2024-04-25T10:30:30.799567+0000","last_fullsized":"2024-04-25T10:30:30.799567+0000","mapping_epoch":33,"log_start":"0'0","ondisk_log_start":"0'0","created":33,"last_epoch_clean":34,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:30:10.729165+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:30:10.729165+0000","last_clean_scrub_stamp":"2024-04-25T10:30:10.729165+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T20:41:22.574408+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"5.e","version":"36'5","reported_seq":34,"reported_epoch":44,"state":"active+clean","last_fresh":"2024-04-25T10:30:29.882392+0000","last_change":"2024-04-25T10:30:12.780850+0000","last_active":"2024-04-25T10:30:29.882392+0000","last_peered":"2024-04-25T10:30:29.882392+0000","last_clean":"2024-04-25T10:30:29.882392+0000","last_became_active":"2024-04-25T10:30:12.780187+0000","last_became_peered":"2024-04-25T10:30:12.780187+0000","last_unstale":"2024-04-25T10:30:29.882392+0000","last_undegraded":"2024-04-25T10:30:29.882392+0000","last_fullsized":"2024-04-25T10:30:29.882392+0000","mapping_epoch":34,"log_start":"0'0","ondisk_log_start":"0'0","created":34,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:30:11.733978+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:30:11.733978+0000","last_clean_scrub_stamp":"2024-04-25T10:30:11.733978+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T12:49:30.093333+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"6.d","version":"0'0","reported_seq":12,"reported_epoch":44,"state":"active+clean","last_fresh":"2024-04-25T10:30:30.799752+0000","last_change":"2024-04-25T10:30:28.878873+0000","last_active":"2024-04-25T10:30:30.799752+0000","last_peered":"2024-04-25T10:30:30.799752+0000","last_clean":"2024-04-25T10:30:30.799752+0000","last_became_active":"2024-04-25T10:30:28.878596+0000","last_became_peered":"2024-04-25T10:30:28.878596+0000","last_unstale":"2024-04-25T10:30:30.799752+0000","last_undegraded":"2024-04-25T10:30:30.799752+0000","last_fullsized":"2024-04-25T10:30:30.799752+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":42,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:30:27.860024+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:30:27.860024+0000","last_clean_scrub_stamp":"2024-04-25T10:30:27.860024+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T16:32:17.438458+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.10","version":"37'15","reported_seq":42,"reported_epoch":44,"state":"active+clean","last_fresh":"2024-04-25T10:30:30.798919+0000","last_change":"2024-04-25T10:30:11.755358+0000","last_active":"2024-04-25T10:30:30.798919+0000","last_peered":"2024-04-25T10:30:30.798919+0000","last_clean":"2024-04-25T10:30:30.798919+0000","last_became_active":"2024-04-25T10:30:11.754992+0000","last_became_peered":"2024-04-25T10:30:11.754992+0000","last_unstale":"2024-04-25T10:30:30.798919+0000","last_undegraded":"2024-04-25T10:30:30.798919+0000","last_fullsized":"2024-04-25T10:30:30.798919+0000","mapping_epoch":33,"log_start":"0'0","ondisk_log_start":"0'0","created":33,"last_epoch_clean":34,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:30:10.729165+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:30:10.729165+0000","last_clean_scrub_stamp":"2024-04-25T10:30:10.729165+0000","objects_scrubbed":0,"log_size":15,"log_dups_size":0,"ondisk_log_size":15,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T19:57:30.397817+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":21,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3],"acting":[7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"5.11","version":"36'6","reported_seq":35,"reported_epoch":44,"state":"active+clean","last_fresh":"2024-04-25T10:30:29.881885+0000","last_change":"2024-04-25T10:30:12.777184+0000","last_active":"2024-04-25T10:30:29.881885+0000","last_peered":"2024-04-25T10:30:29.881885+0000","last_clean":"2024-04-25T10:30:29.881885+0000","last_became_active":"2024-04-25T10:30:12.776874+0000","last_became_peered":"2024-04-25T10:30:12.776874+0000","last_unstale":"2024-04-25T10:30:29.881885+0000","last_undegraded":"2024-04-25T10:30:29.881885+0000","last_fullsized":"2024-04-25T10:30:29.881885+0000","mapping_epoch":34,"log_start":"0'0","ondisk_log_start":"0'0","created":34,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:30:11.733978+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:30:11.733978+0000","last_clean_scrub_stamp":"2024-04-25T10:30:11.733978+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T16:41:33.609972+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"6.12","version":"0'0","reported_seq":12,"reported_epoch":44,"state":"active+clean","last_fresh":"2024-04-25T10:30:30.799248+0000","last_change":"2024-04-25T10:30:28.881494+0000","last_active":"2024-04-25T10:30:30.799248+0000","last_peered":"2024-04-25T10:30:30.799248+0000","last_clean":"2024-04-25T10:30:30.799248+0000","last_became_active":"2024-04-25T10:30:28.880682+0000","last_became_peered":"2024-04-25T10:30:28.880682+0000","last_unstale":"2024-04-25T10:30:30.799248+0000","last_undegraded":"2024-04-25T10:30:30.799248+0000","last_fullsized":"2024-04-25T10:30:30.799248+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":42,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:30:27.860024+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:30:27.860024+0000","last_clean_scrub_stamp":"2024-04-25T10:30:27.860024+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T19:25:03.006802+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.11","version":"0'0","reported_seq":29,"reported_epoch":44,"state":"active+clean","last_fresh":"2024-04-25T10:30:30.800150+0000","last_change":"2024-04-25T10:30:11.755706+0000","last_active":"2024-04-25T10:30:30.800150+0000","last_peered":"2024-04-25T10:30:30.800150+0000","last_clean":"2024-04-25T10:30:30.800150+0000","last_became_active":"2024-04-25T10:30:11.755348+0000","last_became_peered":"2024-04-25T10:30:11.755348+0000","last_unstale":"2024-04-25T10:30:30.800150+0000","last_undegraded":"2024-04-25T10:30:30.800150+0000","last_fullsized":"2024-04-25T10:30:30.800150+0000","mapping_epoch":33,"log_start":"0'0","ondisk_log_start":"0'0","created":33,"last_epoch_clean":34,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:30:10.729165+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:30:10.729165+0000","last_clean_scrub_stamp":"2024-04-25T10:30:10.729165+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T10:42:20.657474+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.10","version":"36'6","reported_seq":35,"reported_epoch":44,"state":"active+clean","last_fresh":"2024-04-25T10:30:29.882755+0000","last_change":"2024-04-25T10:30:12.783285+0000","last_active":"2024-04-25T10:30:29.882755+0000","last_peered":"2024-04-25T10:30:29.882755+0000","last_clean":"2024-04-25T10:30:29.882755+0000","last_became_active":"2024-04-25T10:30:12.782617+0000","last_became_peered":"2024-04-25T10:30:12.782617+0000","last_unstale":"2024-04-25T10:30:29.882755+0000","last_undegraded":"2024-04-25T10:30:29.882755+0000","last_fullsized":"2024-04-25T10:30:29.882755+0000","mapping_epoch":34,"log_start":"0'0","ondisk_log_start":"0'0","created":34,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:30:11.733978+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:30:11.733978+0000","last_clean_scrub_stamp":"2024-04-25T10:30:11.733978+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T18:35:51.085554+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"6.13","version":"0'0","reported_seq":12,"reported_epoch":44,"state":"active+clean","last_fresh":"2024-04-25T10:30:30.799157+0000","last_change":"2024-04-25T10:30:28.880997+0000","last_active":"2024-04-25T10:30:30.799157+0000","last_peered":"2024-04-25T10:30:30.799157+0000","last_clean":"2024-04-25T10:30:30.799157+0000","last_became_active":"2024-04-25T10:30:28.880686+0000","last_became_peered":"2024-04-25T10:30:28.880686+0000","last_unstale":"2024-04-25T10:30:30.799157+0000","last_undegraded":"2024-04-25T10:30:30.799157+0000","last_fullsized":"2024-04-25T10:30:30.799157+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":42,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:30:27.860024+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:30:27.860024+0000","last_clean_scrub_stamp":"2024-04-25T10:30:27.860024+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T17:41:51.036241+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.12","version":"0'0","reported_seq":29,"reported_epoch":44,"state":"active+clean","last_fresh":"2024-04-25T10:30:30.800203+0000","last_change":"2024-04-25T10:30:11.762094+0000","last_active":"2024-04-25T10:30:30.800203+0000","last_peered":"2024-04-25T10:30:30.800203+0000","last_clean":"2024-04-25T10:30:30.800203+0000","last_became_active":"2024-04-25T10:30:11.761456+0000","last_became_peered":"2024-04-25T10:30:11.761456+0000","last_unstale":"2024-04-25T10:30:30.800203+0000","last_undegraded":"2024-04-25T10:30:30.800203+0000","last_fullsized":"2024-04-25T10:30:30.800203+0000","mapping_epoch":33,"log_start":"0'0","ondisk_log_start":"0'0","created":33,"last_epoch_clean":34,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:30:10.729165+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:30:10.729165+0000","last_clean_scrub_stamp":"2024-04-25T10:30:10.729165+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T21:58:31.211023+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.13","version":"36'15","reported_seq":43,"reported_epoch":44,"state":"active+clean","last_fresh":"2024-04-25T10:30:30.799948+0000","last_change":"2024-04-25T10:30:12.775269+0000","last_active":"2024-04-25T10:30:30.799948+0000","last_peered":"2024-04-25T10:30:30.799948+0000","last_clean":"2024-04-25T10:30:30.799948+0000","last_became_active":"2024-04-25T10:30:12.774865+0000","last_became_peered":"2024-04-25T10:30:12.774865+0000","last_unstale":"2024-04-25T10:30:30.799948+0000","last_undegraded":"2024-04-25T10:30:30.799948+0000","last_fullsized":"2024-04-25T10:30:30.799948+0000","mapping_epoch":34,"log_start":"0'0","ondisk_log_start":"0'0","created":34,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:30:11.733978+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:30:11.733978+0000","last_clean_scrub_stamp":"2024-04-25T10:30:11.733978+0000","objects_scrubbed":0,"log_size":15,"log_dups_size":0,"ondisk_log_size":15,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T19:52:00.482437+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"6.10","version":"0'0","reported_seq":12,"reported_epoch":44,"state":"active+clean","last_fresh":"2024-04-25T10:30:30.799595+0000","last_change":"2024-04-25T10:30:28.874537+0000","last_active":"2024-04-25T10:30:30.799595+0000","last_peered":"2024-04-25T10:30:30.799595+0000","last_clean":"2024-04-25T10:30:30.799595+0000","last_became_active":"2024-04-25T10:30:28.874226+0000","last_became_peered":"2024-04-25T10:30:28.874226+0000","last_unstale":"2024-04-25T10:30:30.799595+0000","last_undegraded":"2024-04-25T10:30:30.799595+0000","last_fullsized":"2024-04-25T10:30:30.799595+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":42,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:30:27.860024+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:30:27.860024+0000","last_clean_scrub_stamp":"2024-04-25T10:30:27.860024+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T19:27:12.449150+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.13","version":"37'5","reported_seq":35,"reported_epoch":44,"state":"active+clean","last_fresh":"2024-04-25T10:30:29.883254+0000","last_change":"2024-04-25T10:30:11.764993+0000","last_active":"2024-04-25T10:30:29.883254+0000","last_peered":"2024-04-25T10:30:29.883254+0000","last_clean":"2024-04-25T10:30:29.883254+0000","last_became_active":"2024-04-25T10:30:11.764637+0000","last_became_peered":"2024-04-25T10:30:11.764637+0000","last_unstale":"2024-04-25T10:30:29.883254+0000","last_undegraded":"2024-04-25T10:30:29.883254+0000","last_fullsized":"2024-04-25T10:30:29.883254+0000","mapping_epoch":33,"log_start":"0'0","ondisk_log_start":"0'0","created":33,"last_epoch_clean":34,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:30:10.729165+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:30:10.729165+0000","last_clean_scrub_stamp":"2024-04-25T10:30:10.729165+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T17:24:13.848319+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6,"num_write_kb":8,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"5.12","version":"36'10","reported_seq":37,"reported_epoch":44,"state":"active+clean","last_fresh":"2024-04-25T10:30:29.882763+0000","last_change":"2024-04-25T10:30:12.770496+0000","last_active":"2024-04-25T10:30:29.882763+0000","last_peered":"2024-04-25T10:30:29.882763+0000","last_clean":"2024-04-25T10:30:29.882763+0000","last_became_active":"2024-04-25T10:30:12.770193+0000","last_became_peered":"2024-04-25T10:30:12.770193+0000","last_unstale":"2024-04-25T10:30:29.882763+0000","last_undegraded":"2024-04-25T10:30:29.882763+0000","last_fullsized":"2024-04-25T10:30:29.882763+0000","mapping_epoch":34,"log_start":"0'0","ondisk_log_start":"0'0","created":34,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:30:11.733978+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:30:11.733978+0000","last_clean_scrub_stamp":"2024-04-25T10:30:11.733978+0000","objects_scrubbed":0,"log_size":10,"log_dups_size":0,"ondisk_log_size":10,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T21:35:23.984378+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,1],"acting":[6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"6.11","version":"0'0","reported_seq":12,"reported_epoch":44,"state":"active+clean","last_fresh":"2024-04-25T10:30:29.882707+0000","last_change":"2024-04-25T10:30:28.884488+0000","last_active":"2024-04-25T10:30:29.882707+0000","last_peered":"2024-04-25T10:30:29.882707+0000","last_clean":"2024-04-25T10:30:29.882707+0000","last_became_active":"2024-04-25T10:30:28.884188+0000","last_became_peered":"2024-04-25T10:30:28.884188+0000","last_unstale":"2024-04-25T10:30:29.882707+0000","last_undegraded":"2024-04-25T10:30:29.882707+0000","last_fullsized":"2024-04-25T10:30:29.882707+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":42,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:30:27.860024+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:30:27.860024+0000","last_clean_scrub_stamp":"2024-04-25T10:30:27.860024+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T15:39:48.016111+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,1],"acting":[6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.14","version":"37'23","reported_seq":51,"reported_epoch":44,"state":"active+clean","last_fresh":"2024-04-25T10:30:29.883688+0000","last_change":"2024-04-25T10:30:11.757323+0000","last_active":"2024-04-25T10:30:29.883688+0000","last_peered":"2024-04-25T10:30:29.883688+0000","last_clean":"2024-04-25T10:30:29.883688+0000","last_became_active":"2024-04-25T10:30:11.757011+0000","last_became_peered":"2024-04-25T10:30:11.757011+0000","last_unstale":"2024-04-25T10:30:29.883688+0000","last_undegraded":"2024-04-25T10:30:29.883688+0000","last_fullsized":"2024-04-25T10:30:29.883688+0000","mapping_epoch":33,"log_start":"0'0","ondisk_log_start":"0'0","created":33,"last_epoch_clean":34,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:30:10.729165+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:30:10.729165+0000","last_clean_scrub_stamp":"2024-04-25T10:30:10.729165+0000","objects_scrubbed":0,"log_size":23,"log_dups_size":0,"ondisk_log_size":23,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T19:02:53.584992+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":23,"num_write_kb":25,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"5.15","version":"36'6","reported_seq":35,"reported_epoch":44,"state":"active+clean","last_fresh":"2024-04-25T10:30:29.881889+0000","last_change":"2024-04-25T10:30:12.776432+0000","last_active":"2024-04-25T10:30:29.881889+0000","last_peered":"2024-04-25T10:30:29.881889+0000","last_clean":"2024-04-25T10:30:29.881889+0000","last_became_active":"2024-04-25T10:30:12.775857+0000","last_became_peered":"2024-04-25T10:30:12.775857+0000","last_unstale":"2024-04-25T10:30:29.881889+0000","last_undegraded":"2024-04-25T10:30:29.881889+0000","last_fullsized":"2024-04-25T10:30:29.881889+0000","mapping_epoch":34,"log_start":"0'0","ondisk_log_start":"0'0","created":34,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:30:11.733978+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:30:11.733978+0000","last_clean_scrub_stamp":"2024-04-25T10:30:11.733978+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T14:29:10.343115+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"6.16","version":"0'0","reported_seq":12,"reported_epoch":44,"state":"active+clean","last_fresh":"2024-04-25T10:30:30.799863+0000","last_change":"2024-04-25T10:30:28.881222+0000","last_active":"2024-04-25T10:30:30.799863+0000","last_peered":"2024-04-25T10:30:30.799863+0000","last_clean":"2024-04-25T10:30:30.799863+0000","last_became_active":"2024-04-25T10:30:28.880786+0000","last_became_peered":"2024-04-25T10:30:28.880786+0000","last_unstale":"2024-04-25T10:30:30.799863+0000","last_undegraded":"2024-04-25T10:30:30.799863+0000","last_fullsized":"2024-04-25T10:30:30.799863+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":42,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:30:27.860024+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:30:27.860024+0000","last_clean_scrub_stamp":"2024-04-25T10:30:27.860024+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T21:03:30.612823+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.15","version":"37'5","reported_seq":35,"reported_epoch":44,"state":"active+clean","last_fresh":"2024-04-25T10:30:29.882033+0000","last_change":"2024-04-25T10:30:11.755959+0000","last_active":"2024-04-25T10:30:29.882033+0000","last_peered":"2024-04-25T10:30:29.882033+0000","last_clean":"2024-04-25T10:30:29.882033+0000","last_became_active":"2024-04-25T10:30:11.755603+0000","last_became_peered":"2024-04-25T10:30:11.755603+0000","last_unstale":"2024-04-25T10:30:29.882033+0000","last_undegraded":"2024-04-25T10:30:29.882033+0000","last_fullsized":"2024-04-25T10:30:29.882033+0000","mapping_epoch":33,"log_start":"0'0","ondisk_log_start":"0'0","created":33,"last_epoch_clean":34,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:30:10.729165+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:30:10.729165+0000","last_clean_scrub_stamp":"2024-04-25T10:30:10.729165+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T19:25:05.393017+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.14","version":"36'11","reported_seq":39,"reported_epoch":44,"state":"active+clean","last_fresh":"2024-04-25T10:30:30.800775+0000","last_change":"2024-04-25T10:30:12.770588+0000","last_active":"2024-04-25T10:30:30.800775+0000","last_peered":"2024-04-25T10:30:30.800775+0000","last_clean":"2024-04-25T10:30:30.800775+0000","last_became_active":"2024-04-25T10:30:12.770264+0000","last_became_peered":"2024-04-25T10:30:12.770264+0000","last_unstale":"2024-04-25T10:30:30.800775+0000","last_undegraded":"2024-04-25T10:30:30.800775+0000","last_fullsized":"2024-04-25T10:30:30.800775+0000","mapping_epoch":34,"log_start":"0'0","ondisk_log_start":"0'0","created":34,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:30:11.733978+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:30:11.733978+0000","last_clean_scrub_stamp":"2024-04-25T10:30:11.733978+0000","objects_scrubbed":0,"log_size":11,"log_dups_size":0,"ondisk_log_size":11,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T21:20:12.031650+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"6.17","version":"0'0","reported_seq":12,"reported_epoch":44,"state":"active+clean","last_fresh":"2024-04-25T10:30:30.799771+0000","last_change":"2024-04-25T10:30:28.878754+0000","last_active":"2024-04-25T10:30:30.799771+0000","last_peered":"2024-04-25T10:30:30.799771+0000","last_clean":"2024-04-25T10:30:30.799771+0000","last_became_active":"2024-04-25T10:30:28.878242+0000","last_became_peered":"2024-04-25T10:30:28.878242+0000","last_unstale":"2024-04-25T10:30:30.799771+0000","last_undegraded":"2024-04-25T10:30:30.799771+0000","last_fullsized":"2024-04-25T10:30:30.799771+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":42,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:30:27.860024+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:30:27.860024+0000","last_clean_scrub_stamp":"2024-04-25T10:30:27.860024+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T17:41:51.036241+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7],"acting":[1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.16","version":"0'0","reported_seq":27,"reported_epoch":44,"state":"active+clean","last_fresh":"2024-04-25T10:30:30.800054+0000","last_change":"2024-04-25T10:30:11.749473+0000","last_active":"2024-04-25T10:30:30.800054+0000","last_peered":"2024-04-25T10:30:30.800054+0000","last_clean":"2024-04-25T10:30:30.800054+0000","last_became_active":"2024-04-25T10:30:11.749135+0000","last_became_peered":"2024-04-25T10:30:11.749135+0000","last_unstale":"2024-04-25T10:30:30.800054+0000","last_undegraded":"2024-04-25T10:30:30.800054+0000","last_fullsized":"2024-04-25T10:30:30.800054+0000","mapping_epoch":33,"log_start":"0'0","ondisk_log_start":"0'0","created":33,"last_epoch_clean":34,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:30:10.729165+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:30:10.729165+0000","last_clean_scrub_stamp":"2024-04-25T10:30:10.729165+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T15:17:28.997853+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"5.17","version":"36'9","reported_seq":37,"reported_epoch":44,"state":"active+clean","last_fresh":"2024-04-25T10:30:30.800520+0000","last_change":"2024-04-25T10:30:12.773113+0000","last_active":"2024-04-25T10:30:30.800520+0000","last_peered":"2024-04-25T10:30:30.800520+0000","last_clean":"2024-04-25T10:30:30.800520+0000","last_became_active":"2024-04-25T10:30:12.772786+0000","last_became_peered":"2024-04-25T10:30:12.772786+0000","last_unstale":"2024-04-25T10:30:30.800520+0000","last_undegraded":"2024-04-25T10:30:30.800520+0000","last_fullsized":"2024-04-25T10:30:30.800520+0000","mapping_epoch":34,"log_start":"0'0","ondisk_log_start":"0'0","created":34,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:30:11.733978+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:30:11.733978+0000","last_clean_scrub_stamp":"2024-04-25T10:30:11.733978+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T21:30:14.051378+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5],"acting":[4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"6.14","version":"0'0","reported_seq":12,"reported_epoch":44,"state":"active+clean","last_fresh":"2024-04-25T10:30:30.800723+0000","last_change":"2024-04-25T10:30:28.878325+0000","last_active":"2024-04-25T10:30:30.800723+0000","last_peered":"2024-04-25T10:30:30.800723+0000","last_clean":"2024-04-25T10:30:30.800723+0000","last_became_active":"2024-04-25T10:30:28.878014+0000","last_became_peered":"2024-04-25T10:30:28.878014+0000","last_unstale":"2024-04-25T10:30:30.800723+0000","last_undegraded":"2024-04-25T10:30:30.800723+0000","last_fullsized":"2024-04-25T10:30:30.800723+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":42,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:30:27.860024+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:30:27.860024+0000","last_clean_scrub_stamp":"2024-04-25T10:30:27.860024+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T11:04:24.909990+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.17","version":"0'0","reported_seq":27,"reported_epoch":44,"state":"active+clean","last_fresh":"2024-04-25T10:30:30.799599+0000","last_change":"2024-04-25T10:30:11.758416+0000","last_active":"2024-04-25T10:30:30.799599+0000","last_peered":"2024-04-25T10:30:30.799599+0000","last_clean":"2024-04-25T10:30:30.799599+0000","last_became_active":"2024-04-25T10:30:11.758049+0000","last_became_peered":"2024-04-25T10:30:11.758049+0000","last_unstale":"2024-04-25T10:30:30.799599+0000","last_undegraded":"2024-04-25T10:30:30.799599+0000","last_fullsized":"2024-04-25T10:30:30.799599+0000","mapping_epoch":33,"log_start":"0'0","ondisk_log_start":"0'0","created":33,"last_epoch_clean":34,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:30:10.729165+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:30:10.729165+0000","last_clean_scrub_stamp":"2024-04-25T10:30:10.729165+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T18:26:05.159854+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"5.16","version":"36'10","reported_seq":39,"reported_epoch":44,"state":"active+clean","last_fresh":"2024-04-25T10:30:29.882265+0000","last_change":"2024-04-25T10:30:12.770216+0000","last_active":"2024-04-25T10:30:29.882265+0000","last_peered":"2024-04-25T10:30:29.882265+0000","last_clean":"2024-04-25T10:30:29.882265+0000","last_became_active":"2024-04-25T10:30:12.769785+0000","last_became_peered":"2024-04-25T10:30:12.769785+0000","last_unstale":"2024-04-25T10:30:29.882265+0000","last_undegraded":"2024-04-25T10:30:29.882265+0000","last_fullsized":"2024-04-25T10:30:29.882265+0000","mapping_epoch":34,"log_start":"0'0","ondisk_log_start":"0'0","created":34,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:30:11.733978+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:30:11.733978+0000","last_clean_scrub_stamp":"2024-04-25T10:30:11.733978+0000","objects_scrubbed":0,"log_size":10,"log_dups_size":0,"ondisk_log_size":10,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T16:58:09.648786+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"6.15","version":"0'0","reported_seq":12,"reported_epoch":44,"state":"active+clean","last_fresh":"2024-04-25T10:30:29.883532+0000","last_change":"2024-04-25T10:30:28.879590+0000","last_active":"2024-04-25T10:30:29.883532+0000","last_peered":"2024-04-25T10:30:29.883532+0000","last_clean":"2024-04-25T10:30:29.883532+0000","last_became_active":"2024-04-25T10:30:28.879277+0000","last_became_peered":"2024-04-25T10:30:28.879277+0000","last_unstale":"2024-04-25T10:30:29.883532+0000","last_undegraded":"2024-04-25T10:30:29.883532+0000","last_fullsized":"2024-04-25T10:30:29.883532+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":42,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:30:27.860024+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:30:27.860024+0000","last_clean_scrub_stamp":"2024-04-25T10:30:27.860024+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T14:04:58.646968+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.18","version":"0'0","reported_seq":30,"reported_epoch":44,"state":"active+clean","last_fresh":"2024-04-25T10:30:29.882866+0000","last_change":"2024-04-25T10:30:11.756627+0000","last_active":"2024-04-25T10:30:29.882866+0000","last_peered":"2024-04-25T10:30:29.882866+0000","last_clean":"2024-04-25T10:30:29.882866+0000","last_became_active":"2024-04-25T10:30:11.756323+0000","last_became_peered":"2024-04-25T10:30:11.756323+0000","last_unstale":"2024-04-25T10:30:29.882866+0000","last_undegraded":"2024-04-25T10:30:29.882866+0000","last_fullsized":"2024-04-25T10:30:29.882866+0000","mapping_epoch":33,"log_start":"0'0","ondisk_log_start":"0'0","created":33,"last_epoch_clean":34,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:30:10.729165+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:30:10.729165+0000","last_clean_scrub_stamp":"2024-04-25T10:30:10.729165+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T16:49:20.087013+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4],"acting":[2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"5.19","version":"36'10","reported_seq":38,"reported_epoch":44,"state":"active+clean","last_fresh":"2024-04-25T10:30:30.799990+0000","last_change":"2024-04-25T10:30:12.780437+0000","last_active":"2024-04-25T10:30:30.799990+0000","last_peered":"2024-04-25T10:30:30.799990+0000","last_clean":"2024-04-25T10:30:30.799990+0000","last_became_active":"2024-04-25T10:30:12.779762+0000","last_became_peered":"2024-04-25T10:30:12.779762+0000","last_unstale":"2024-04-25T10:30:30.799990+0000","last_undegraded":"2024-04-25T10:30:30.799990+0000","last_fullsized":"2024-04-25T10:30:30.799990+0000","mapping_epoch":34,"log_start":"0'0","ondisk_log_start":"0'0","created":34,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:30:11.733978+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:30:11.733978+0000","last_clean_scrub_stamp":"2024-04-25T10:30:11.733978+0000","objects_scrubbed":0,"log_size":10,"log_dups_size":0,"ondisk_log_size":10,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T12:12:16.175153+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"6.1a","version":"0'0","reported_seq":12,"reported_epoch":44,"state":"active+clean","last_fresh":"2024-04-25T10:30:29.882257+0000","last_change":"2024-04-25T10:30:28.879591+0000","last_active":"2024-04-25T10:30:29.882257+0000","last_peered":"2024-04-25T10:30:29.882257+0000","last_clean":"2024-04-25T10:30:29.882257+0000","last_became_active":"2024-04-25T10:30:28.879271+0000","last_became_peered":"2024-04-25T10:30:28.879271+0000","last_unstale":"2024-04-25T10:30:29.882257+0000","last_undegraded":"2024-04-25T10:30:29.882257+0000","last_fullsized":"2024-04-25T10:30:29.882257+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":42,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:30:27.860024+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:30:27.860024+0000","last_clean_scrub_stamp":"2024-04-25T10:30:27.860024+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T20:14:35.145802+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.19","version":"0'0","reported_seq":27,"reported_epoch":44,"state":"active+clean","last_fresh":"2024-04-25T10:30:30.799075+0000","last_change":"2024-04-25T10:30:11.757481+0000","last_active":"2024-04-25T10:30:30.799075+0000","last_peered":"2024-04-25T10:30:30.799075+0000","last_clean":"2024-04-25T10:30:30.799075+0000","last_became_active":"2024-04-25T10:30:11.757149+0000","last_became_peered":"2024-04-25T10:30:11.757149+0000","last_unstale":"2024-04-25T10:30:30.799075+0000","last_undegraded":"2024-04-25T10:30:30.799075+0000","last_fullsized":"2024-04-25T10:30:30.799075+0000","mapping_epoch":33,"log_start":"0'0","ondisk_log_start":"0'0","created":33,"last_epoch_clean":34,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:30:10.729165+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:30:10.729165+0000","last_clean_scrub_stamp":"2024-04-25T10:30:10.729165+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T13:23:15.814583+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"5.18","version":"36'8","reported_seq":35,"reported_epoch":44,"state":"active+clean","last_fresh":"2024-04-25T10:30:30.798887+0000","last_change":"2024-04-25T10:30:12.769294+0000","last_active":"2024-04-25T10:30:30.798887+0000","last_peered":"2024-04-25T10:30:30.798887+0000","last_clean":"2024-04-25T10:30:30.798887+0000","last_became_active":"2024-04-25T10:30:12.768543+0000","last_became_peered":"2024-04-25T10:30:12.768543+0000","last_unstale":"2024-04-25T10:30:30.798887+0000","last_undegraded":"2024-04-25T10:30:30.798887+0000","last_fullsized":"2024-04-25T10:30:30.798887+0000","mapping_epoch":34,"log_start":"0'0","ondisk_log_start":"0'0","created":34,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:30:11.733978+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:30:11.733978+0000","last_clean_scrub_stamp":"2024-04-25T10:30:11.733978+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T18:26:06.164667+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"6.1b","version":"0'0","reported_seq":12,"reported_epoch":44,"state":"active+clean","last_fresh":"2024-04-25T10:30:29.883121+0000","last_change":"2024-04-25T10:30:28.885011+0000","last_active":"2024-04-25T10:30:29.883121+0000","last_peered":"2024-04-25T10:30:29.883121+0000","last_clean":"2024-04-25T10:30:29.883121+0000","last_became_active":"2024-04-25T10:30:28.883516+0000","last_became_peered":"2024-04-25T10:30:28.883516+0000","last_unstale":"2024-04-25T10:30:29.883121+0000","last_undegraded":"2024-04-25T10:30:29.883121+0000","last_fullsized":"2024-04-25T10:30:29.883121+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":42,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:30:27.860024+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:30:27.860024+0000","last_clean_scrub_stamp":"2024-04-25T10:30:27.860024+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T17:35:49.687185+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,2],"acting":[6,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.1a","version":"35'1","reported_seq":31,"reported_epoch":44,"state":"active+clean","last_fresh":"2024-04-25T10:30:29.883044+0000","last_change":"2024-04-25T10:30:11.758678+0000","last_active":"2024-04-25T10:30:29.883044+0000","last_peered":"2024-04-25T10:30:29.883044+0000","last_clean":"2024-04-25T10:30:29.883044+0000","last_became_active":"2024-04-25T10:30:11.758384+0000","last_became_peered":"2024-04-25T10:30:11.758384+0000","last_unstale":"2024-04-25T10:30:29.883044+0000","last_undegraded":"2024-04-25T10:30:29.883044+0000","last_fullsized":"2024-04-25T10:30:29.883044+0000","mapping_epoch":33,"log_start":"0'0","ondisk_log_start":"0'0","created":33,"last_epoch_clean":34,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:30:10.729165+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:30:10.729165+0000","last_clean_scrub_stamp":"2024-04-25T10:30:10.729165+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T16:17:54.376541+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,6],"acting":[2,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"5.1b","version":"36'8","reported_seq":37,"reported_epoch":44,"state":"active+clean","last_fresh":"2024-04-25T10:30:29.883109+0000","last_change":"2024-04-25T10:30:12.784767+0000","last_active":"2024-04-25T10:30:29.883109+0000","last_peered":"2024-04-25T10:30:29.883109+0000","last_clean":"2024-04-25T10:30:29.883109+0000","last_became_active":"2024-04-25T10:30:12.784311+0000","last_became_peered":"2024-04-25T10:30:12.784311+0000","last_unstale":"2024-04-25T10:30:29.883109+0000","last_undegraded":"2024-04-25T10:30:29.883109+0000","last_fullsized":"2024-04-25T10:30:29.883109+0000","mapping_epoch":34,"log_start":"0'0","ondisk_log_start":"0'0","created":34,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:30:11.733978+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:30:11.733978+0000","last_clean_scrub_stamp":"2024-04-25T10:30:11.733978+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T19:56:57.765496+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,7],"acting":[2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"6.18","version":"0'0","reported_seq":12,"reported_epoch":44,"state":"active+clean","last_fresh":"2024-04-25T10:30:30.800349+0000","last_change":"2024-04-25T10:30:28.882157+0000","last_active":"2024-04-25T10:30:30.800349+0000","last_peered":"2024-04-25T10:30:30.800349+0000","last_clean":"2024-04-25T10:30:30.800349+0000","last_became_active":"2024-04-25T10:30:28.881823+0000","last_became_peered":"2024-04-25T10:30:28.881823+0000","last_unstale":"2024-04-25T10:30:30.800349+0000","last_undegraded":"2024-04-25T10:30:30.800349+0000","last_fullsized":"2024-04-25T10:30:30.800349+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":42,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:30:27.860024+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:30:27.860024+0000","last_clean_scrub_stamp":"2024-04-25T10:30:27.860024+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T12:12:35.591767+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5],"acting":[4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.1b","version":"35'1","reported_seq":31,"reported_epoch":44,"state":"active+clean","last_fresh":"2024-04-25T10:30:29.883205+0000","last_change":"2024-04-25T10:30:11.753667+0000","last_active":"2024-04-25T10:30:29.883205+0000","last_peered":"2024-04-25T10:30:29.883205+0000","last_clean":"2024-04-25T10:30:29.883205+0000","last_became_active":"2024-04-25T10:30:11.753340+0000","last_became_peered":"2024-04-25T10:30:11.753340+0000","last_unstale":"2024-04-25T10:30:29.883205+0000","last_undegraded":"2024-04-25T10:30:29.883205+0000","last_fullsized":"2024-04-25T10:30:29.883205+0000","mapping_epoch":33,"log_start":"0'0","ondisk_log_start":"0'0","created":33,"last_epoch_clean":34,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:30:10.729165+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:30:10.729165+0000","last_clean_scrub_stamp":"2024-04-25T10:30:10.729165+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T16:09:04.081750+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"5.1a","version":"36'7","reported_seq":34,"reported_epoch":44,"state":"active+clean","last_fresh":"2024-04-25T10:30:30.799658+0000","last_change":"2024-04-25T10:30:12.773835+0000","last_active":"2024-04-25T10:30:30.799658+0000","last_peered":"2024-04-25T10:30:30.799658+0000","last_clean":"2024-04-25T10:30:30.799658+0000","last_became_active":"2024-04-25T10:30:12.773538+0000","last_became_peered":"2024-04-25T10:30:12.773538+0000","last_unstale":"2024-04-25T10:30:30.799658+0000","last_undegraded":"2024-04-25T10:30:30.799658+0000","last_fullsized":"2024-04-25T10:30:30.799658+0000","mapping_epoch":34,"log_start":"0'0","ondisk_log_start":"0'0","created":34,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:30:11.733978+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:30:11.733978+0000","last_clean_scrub_stamp":"2024-04-25T10:30:11.733978+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T18:31:49.260513+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"6.19","version":"0'0","reported_seq":12,"reported_epoch":44,"state":"active+clean","last_fresh":"2024-04-25T10:30:30.798549+0000","last_change":"2024-04-25T10:30:28.883543+0000","last_active":"2024-04-25T10:30:30.798549+0000","last_peered":"2024-04-25T10:30:30.798549+0000","last_clean":"2024-04-25T10:30:30.798549+0000","last_became_active":"2024-04-25T10:30:28.883229+0000","last_became_peered":"2024-04-25T10:30:28.883229+0000","last_unstale":"2024-04-25T10:30:30.798549+0000","last_undegraded":"2024-04-25T10:30:30.798549+0000","last_fullsized":"2024-04-25T10:30:30.798549+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":42,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:30:27.860024+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:30:27.860024+0000","last_clean_scrub_stamp":"2024-04-25T10:30:27.860024+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T14:37:44.738115+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"6.1e","version":"0'0","reported_seq":12,"reported_epoch":44,"state":"active+clean","last_fresh":"2024-04-25T10:30:29.882435+0000","last_change":"2024-04-25T10:30:28.874417+0000","last_active":"2024-04-25T10:30:29.882435+0000","last_peered":"2024-04-25T10:30:29.882435+0000","last_clean":"2024-04-25T10:30:29.882435+0000","last_became_active":"2024-04-25T10:30:28.874102+0000","last_became_peered":"2024-04-25T10:30:28.874102+0000","last_unstale":"2024-04-25T10:30:29.882435+0000","last_undegraded":"2024-04-25T10:30:29.882435+0000","last_fullsized":"2024-04-25T10:30:29.882435+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":42,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:30:27.860024+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:30:27.860024+0000","last_clean_scrub_stamp":"2024-04-25T10:30:27.860024+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T21:25:07.511482+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.1c","version":"37'3","reported_seq":32,"reported_epoch":44,"state":"active+clean","last_fresh":"2024-04-25T10:30:30.800852+0000","last_change":"2024-04-25T10:30:11.759829+0000","last_active":"2024-04-25T10:30:30.800852+0000","last_peered":"2024-04-25T10:30:30.800852+0000","last_clean":"2024-04-25T10:30:30.800852+0000","last_became_active":"2024-04-25T10:30:11.759465+0000","last_became_peered":"2024-04-25T10:30:11.759465+0000","last_unstale":"2024-04-25T10:30:30.800852+0000","last_undegraded":"2024-04-25T10:30:30.800852+0000","last_fullsized":"2024-04-25T10:30:30.800852+0000","mapping_epoch":33,"log_start":"0'0","ondisk_log_start":"0'0","created":33,"last_epoch_clean":34,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:30:10.729165+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:30:10.729165+0000","last_clean_scrub_stamp":"2024-04-25T10:30:10.729165+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T13:31:53.608355+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.1d","version":"36'15","reported_seq":43,"reported_epoch":44,"state":"active+clean","last_fresh":"2024-04-25T10:30:30.799497+0000","last_change":"2024-04-25T10:30:12.773474+0000","last_active":"2024-04-25T10:30:30.799497+0000","last_peered":"2024-04-25T10:30:30.799497+0000","last_clean":"2024-04-25T10:30:30.799497+0000","last_became_active":"2024-04-25T10:30:12.773133+0000","last_became_peered":"2024-04-25T10:30:12.773133+0000","last_unstale":"2024-04-25T10:30:30.799497+0000","last_undegraded":"2024-04-25T10:30:30.799497+0000","last_fullsized":"2024-04-25T10:30:30.799497+0000","mapping_epoch":34,"log_start":"0'0","ondisk_log_start":"0'0","created":34,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:30:11.733978+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:30:11.733978+0000","last_clean_scrub_stamp":"2024-04-25T10:30:11.733978+0000","objects_scrubbed":0,"log_size":15,"log_dups_size":0,"ondisk_log_size":15,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T21:37:31.551319+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"6.1f","version":"0'0","reported_seq":12,"reported_epoch":44,"state":"active+clean","last_fresh":"2024-04-25T10:30:29.883478+0000","last_change":"2024-04-25T10:30:28.884033+0000","last_active":"2024-04-25T10:30:29.883478+0000","last_peered":"2024-04-25T10:30:29.883478+0000","last_clean":"2024-04-25T10:30:29.883478+0000","last_became_active":"2024-04-25T10:30:28.883610+0000","last_became_peered":"2024-04-25T10:30:28.883610+0000","last_unstale":"2024-04-25T10:30:29.883478+0000","last_undegraded":"2024-04-25T10:30:29.883478+0000","last_fullsized":"2024-04-25T10:30:29.883478+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":42,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:30:27.860024+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:30:27.860024+0000","last_clean_scrub_stamp":"2024-04-25T10:30:27.860024+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T18:12:57.452350+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.1d","version":"37'3","reported_seq":30,"reported_epoch":44,"state":"active+clean","last_fresh":"2024-04-25T10:30:30.799815+0000","last_change":"2024-04-25T10:30:11.752439+0000","last_active":"2024-04-25T10:30:30.799815+0000","last_peered":"2024-04-25T10:30:30.799815+0000","last_clean":"2024-04-25T10:30:30.799815+0000","last_became_active":"2024-04-25T10:30:11.752050+0000","last_became_peered":"2024-04-25T10:30:11.752050+0000","last_unstale":"2024-04-25T10:30:30.799815+0000","last_undegraded":"2024-04-25T10:30:30.799815+0000","last_fullsized":"2024-04-25T10:30:30.799815+0000","mapping_epoch":33,"log_start":"0'0","ondisk_log_start":"0'0","created":33,"last_epoch_clean":34,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:30:10.729165+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:30:10.729165+0000","last_clean_scrub_stamp":"2024-04-25T10:30:10.729165+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T11:41:21.358729+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"5.1c","version":"36'9","reported_seq":38,"reported_epoch":44,"state":"active+clean","last_fresh":"2024-04-25T10:30:29.883403+0000","last_change":"2024-04-25T10:30:12.781913+0000","last_active":"2024-04-25T10:30:29.883403+0000","last_peered":"2024-04-25T10:30:29.883403+0000","last_clean":"2024-04-25T10:30:29.883403+0000","last_became_active":"2024-04-25T10:30:12.781111+0000","last_became_peered":"2024-04-25T10:30:12.781111+0000","last_unstale":"2024-04-25T10:30:29.883403+0000","last_undegraded":"2024-04-25T10:30:29.883403+0000","last_fullsized":"2024-04-25T10:30:29.883403+0000","mapping_epoch":34,"log_start":"0'0","ondisk_log_start":"0'0","created":34,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:30:11.733978+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:30:11.733978+0000","last_clean_scrub_stamp":"2024-04-25T10:30:11.733978+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T16:29:27.103512+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4],"acting":[2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.1e","version":"35'1","reported_seq":30,"reported_epoch":44,"state":"active+clean","last_fresh":"2024-04-25T10:30:30.800499+0000","last_change":"2024-04-25T10:30:11.761160+0000","last_active":"2024-04-25T10:30:30.800499+0000","last_peered":"2024-04-25T10:30:30.800499+0000","last_clean":"2024-04-25T10:30:30.800499+0000","last_became_active":"2024-04-25T10:30:11.760819+0000","last_became_peered":"2024-04-25T10:30:11.760819+0000","last_unstale":"2024-04-25T10:30:30.800499+0000","last_undegraded":"2024-04-25T10:30:30.800499+0000","last_fullsized":"2024-04-25T10:30:30.800499+0000","mapping_epoch":33,"log_start":"0'0","ondisk_log_start":"0'0","created":33,"last_epoch_clean":34,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:30:10.729165+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:30:10.729165+0000","last_clean_scrub_stamp":"2024-04-25T10:30:10.729165+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T13:00:22.139475+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.1f","version":"36'6","reported_seq":35,"reported_epoch":44,"state":"active+clean","last_fresh":"2024-04-25T10:30:29.883526+0000","last_change":"2024-04-25T10:30:12.772257+0000","last_active":"2024-04-25T10:30:29.883526+0000","last_peered":"2024-04-25T10:30:29.883526+0000","last_clean":"2024-04-25T10:30:29.883526+0000","last_became_active":"2024-04-25T10:30:12.771923+0000","last_became_peered":"2024-04-25T10:30:12.771923+0000","last_unstale":"2024-04-25T10:30:29.883526+0000","last_undegraded":"2024-04-25T10:30:29.883526+0000","last_fullsized":"2024-04-25T10:30:29.883526+0000","mapping_epoch":34,"log_start":"0'0","ondisk_log_start":"0'0","created":34,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:30:11.733978+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:30:11.733978+0000","last_clean_scrub_stamp":"2024-04-25T10:30:11.733978+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T17:35:33.561138+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"6.1c","version":"0'0","reported_seq":12,"reported_epoch":44,"state":"active+clean","last_fresh":"2024-04-25T10:30:29.882853+0000","last_change":"2024-04-25T10:30:28.875422+0000","last_active":"2024-04-25T10:30:29.882853+0000","last_peered":"2024-04-25T10:30:29.882853+0000","last_clean":"2024-04-25T10:30:29.882853+0000","last_became_active":"2024-04-25T10:30:28.875079+0000","last_became_peered":"2024-04-25T10:30:28.875079+0000","last_unstale":"2024-04-25T10:30:29.882853+0000","last_undegraded":"2024-04-25T10:30:29.882853+0000","last_fullsized":"2024-04-25T10:30:29.882853+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":42,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:30:27.860024+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:30:27.860024+0000","last_clean_scrub_stamp":"2024-04-25T10:30:27.860024+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T13:23:55.301869+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.1f","version":"37'8","reported_seq":38,"reported_epoch":44,"state":"active+clean","last_fresh":"2024-04-25T10:30:29.882826+0000","last_change":"2024-04-25T10:30:11.756978+0000","last_active":"2024-04-25T10:30:29.882826+0000","last_peered":"2024-04-25T10:30:29.882826+0000","last_clean":"2024-04-25T10:30:29.882826+0000","last_became_active":"2024-04-25T10:30:11.756617+0000","last_became_peered":"2024-04-25T10:30:11.756617+0000","last_unstale":"2024-04-25T10:30:29.882826+0000","last_undegraded":"2024-04-25T10:30:29.882826+0000","last_fullsized":"2024-04-25T10:30:29.882826+0000","mapping_epoch":33,"log_start":"0'0","ondisk_log_start":"0'0","created":33,"last_epoch_clean":34,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:30:10.729165+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:30:10.729165+0000","last_clean_scrub_stamp":"2024-04-25T10:30:10.729165+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T12:27:53.559880+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":1,"num_read_kb":0,"num_write":10,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.1e","version":"36'5","reported_seq":32,"reported_epoch":44,"state":"active+clean","last_fresh":"2024-04-25T10:30:29.883469+0000","last_change":"2024-04-25T10:30:12.769800+0000","last_active":"2024-04-25T10:30:29.883469+0000","last_peered":"2024-04-25T10:30:29.883469+0000","last_clean":"2024-04-25T10:30:29.883469+0000","last_became_active":"2024-04-25T10:30:12.769470+0000","last_became_peered":"2024-04-25T10:30:12.769470+0000","last_unstale":"2024-04-25T10:30:29.883469+0000","last_undegraded":"2024-04-25T10:30:29.883469+0000","last_fullsized":"2024-04-25T10:30:29.883469+0000","mapping_epoch":34,"log_start":"0'0","ondisk_log_start":"0'0","created":34,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:30:11.733978+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:30:11.733978+0000","last_clean_scrub_stamp":"2024-04-25T10:30:11.733978+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T13:11:54.897586+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"6.1d","version":"0'0","reported_seq":12,"reported_epoch":44,"state":"active+clean","last_fresh":"2024-04-25T10:30:29.883295+0000","last_change":"2024-04-25T10:30:28.884277+0000","last_active":"2024-04-25T10:30:29.883295+0000","last_peered":"2024-04-25T10:30:29.883295+0000","last_clean":"2024-04-25T10:30:29.883295+0000","last_became_active":"2024-04-25T10:30:28.883764+0000","last_became_peered":"2024-04-25T10:30:28.883764+0000","last_unstale":"2024-04-25T10:30:29.883295+0000","last_undegraded":"2024-04-25T10:30:29.883295+0000","last_fullsized":"2024-04-25T10:30:29.883295+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":42,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:30:27.860024+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:30:27.860024+0000","last_clean_scrub_stamp":"2024-04-25T10:30:27.860024+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T18:36:07.211600+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"5.20","version":"36'9","reported_seq":38,"reported_epoch":44,"state":"active+clean","last_fresh":"2024-04-25T10:30:29.881619+0000","last_change":"2024-04-25T10:30:12.766692+0000","last_active":"2024-04-25T10:30:29.881619+0000","last_peered":"2024-04-25T10:30:29.881619+0000","last_clean":"2024-04-25T10:30:29.881619+0000","last_became_active":"2024-04-25T10:30:12.766336+0000","last_became_peered":"2024-04-25T10:30:12.766336+0000","last_unstale":"2024-04-25T10:30:29.881619+0000","last_undegraded":"2024-04-25T10:30:29.881619+0000","last_fullsized":"2024-04-25T10:30:29.881619+0000","mapping_epoch":34,"log_start":"0'0","ondisk_log_start":"0'0","created":34,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:30:11.733978+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:30:11.733978+0000","last_clean_scrub_stamp":"2024-04-25T10:30:11.733978+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T18:11:27.798982+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.21","version":"36'12","reported_seq":41,"reported_epoch":44,"state":"active+clean","last_fresh":"2024-04-25T10:30:29.882888+0000","last_change":"2024-04-25T10:30:12.774245+0000","last_active":"2024-04-25T10:30:29.882888+0000","last_peered":"2024-04-25T10:30:29.882888+0000","last_clean":"2024-04-25T10:30:29.882888+0000","last_became_active":"2024-04-25T10:30:12.773923+0000","last_became_peered":"2024-04-25T10:30:12.773923+0000","last_unstale":"2024-04-25T10:30:29.882888+0000","last_undegraded":"2024-04-25T10:30:29.882888+0000","last_fullsized":"2024-04-25T10:30:29.882888+0000","mapping_epoch":34,"log_start":"0'0","ondisk_log_start":"0'0","created":34,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:30:11.733978+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:30:11.733978+0000","last_clean_scrub_stamp":"2024-04-25T10:30:11.733978+0000","objects_scrubbed":0,"log_size":12,"log_dups_size":0,"ondisk_log_size":12,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T22:09:01.889527+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,7],"acting":[2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"5.22","version":"36'8","reported_seq":34,"reported_epoch":44,"state":"active+clean","last_fresh":"2024-04-25T10:30:30.799017+0000","last_change":"2024-04-25T10:30:12.769958+0000","last_active":"2024-04-25T10:30:30.799017+0000","last_peered":"2024-04-25T10:30:30.799017+0000","last_clean":"2024-04-25T10:30:30.799017+0000","last_became_active":"2024-04-25T10:30:12.769590+0000","last_became_peered":"2024-04-25T10:30:12.769590+0000","last_unstale":"2024-04-25T10:30:30.799017+0000","last_undegraded":"2024-04-25T10:30:30.799017+0000","last_fullsized":"2024-04-25T10:30:30.799017+0000","mapping_epoch":34,"log_start":"0'0","ondisk_log_start":"0'0","created":34,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:30:11.733978+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:30:11.733978+0000","last_clean_scrub_stamp":"2024-04-25T10:30:11.733978+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T11:57:29.979255+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"5.23","version":"36'7","reported_seq":33,"reported_epoch":44,"state":"active+clean","last_fresh":"2024-04-25T10:30:30.798639+0000","last_change":"2024-04-25T10:30:12.781995+0000","last_active":"2024-04-25T10:30:30.798639+0000","last_peered":"2024-04-25T10:30:30.798639+0000","last_clean":"2024-04-25T10:30:30.798639+0000","last_became_active":"2024-04-25T10:30:12.781358+0000","last_became_peered":"2024-04-25T10:30:12.781358+0000","last_unstale":"2024-04-25T10:30:30.798639+0000","last_undegraded":"2024-04-25T10:30:30.798639+0000","last_fullsized":"2024-04-25T10:30:30.798639+0000","mapping_epoch":34,"log_start":"0'0","ondisk_log_start":"0'0","created":34,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:30:11.733978+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:30:11.733978+0000","last_clean_scrub_stamp":"2024-04-25T10:30:11.733978+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T18:00:26.511360+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"5.24","version":"36'4","reported_seq":33,"reported_epoch":44,"state":"active+clean","last_fresh":"2024-04-25T10:30:29.883349+0000","last_change":"2024-04-25T10:30:12.775515+0000","last_active":"2024-04-25T10:30:29.883349+0000","last_peered":"2024-04-25T10:30:29.883349+0000","last_clean":"2024-04-25T10:30:29.883349+0000","last_became_active":"2024-04-25T10:30:12.775167+0000","last_became_peered":"2024-04-25T10:30:12.775167+0000","last_unstale":"2024-04-25T10:30:29.883349+0000","last_undegraded":"2024-04-25T10:30:29.883349+0000","last_fullsized":"2024-04-25T10:30:29.883349+0000","mapping_epoch":34,"log_start":"0'0","ondisk_log_start":"0'0","created":34,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:30:11.733978+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:30:11.733978+0000","last_clean_scrub_stamp":"2024-04-25T10:30:11.733978+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T21:35:23.984378+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,6],"acting":[2,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"5.25","version":"36'6","reported_seq":32,"reported_epoch":44,"state":"active+clean","last_fresh":"2024-04-25T10:30:30.799258+0000","last_change":"2024-04-25T10:30:12.767505+0000","last_active":"2024-04-25T10:30:30.799258+0000","last_peered":"2024-04-25T10:30:30.799258+0000","last_clean":"2024-04-25T10:30:30.799258+0000","last_became_active":"2024-04-25T10:30:12.767149+0000","last_became_peered":"2024-04-25T10:30:12.767149+0000","last_unstale":"2024-04-25T10:30:30.799258+0000","last_undegraded":"2024-04-25T10:30:30.799258+0000","last_fullsized":"2024-04-25T10:30:30.799258+0000","mapping_epoch":34,"log_start":"0'0","ondisk_log_start":"0'0","created":34,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:30:11.733978+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:30:11.733978+0000","last_clean_scrub_stamp":"2024-04-25T10:30:11.733978+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T16:29:27.103512+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"5.26","version":"36'15","reported_seq":43,"reported_epoch":44,"state":"active+clean","last_fresh":"2024-04-25T10:30:30.800340+0000","last_change":"2024-04-25T10:30:12.771420+0000","last_active":"2024-04-25T10:30:30.800340+0000","last_peered":"2024-04-25T10:30:30.800340+0000","last_clean":"2024-04-25T10:30:30.800340+0000","last_became_active":"2024-04-25T10:30:12.771117+0000","last_became_peered":"2024-04-25T10:30:12.771117+0000","last_unstale":"2024-04-25T10:30:30.800340+0000","last_undegraded":"2024-04-25T10:30:30.800340+0000","last_fullsized":"2024-04-25T10:30:30.800340+0000","mapping_epoch":34,"log_start":"0'0","ondisk_log_start":"0'0","created":34,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:30:11.733978+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:30:11.733978+0000","last_clean_scrub_stamp":"2024-04-25T10:30:11.733978+0000","objects_scrubbed":0,"log_size":15,"log_dups_size":0,"ondisk_log_size":15,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T21:06:20.749496+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.27","version":"36'2","reported_seq":30,"reported_epoch":44,"state":"active+clean","last_fresh":"2024-04-25T10:30:30.800377+0000","last_change":"2024-04-25T10:30:12.779972+0000","last_active":"2024-04-25T10:30:30.800377+0000","last_peered":"2024-04-25T10:30:30.800377+0000","last_clean":"2024-04-25T10:30:30.800377+0000","last_became_active":"2024-04-25T10:30:12.779526+0000","last_became_peered":"2024-04-25T10:30:12.779526+0000","last_unstale":"2024-04-25T10:30:30.800377+0000","last_undegraded":"2024-04-25T10:30:30.800377+0000","last_fullsized":"2024-04-25T10:30:30.800377+0000","mapping_epoch":34,"log_start":"0'0","ondisk_log_start":"0'0","created":34,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:30:11.733978+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:30:11.733978+0000","last_clean_scrub_stamp":"2024-04-25T10:30:11.733978+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T15:41:59.020749+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.28","version":"36'6","reported_seq":35,"reported_epoch":44,"state":"active+clean","last_fresh":"2024-04-25T10:30:29.882628+0000","last_change":"2024-04-25T10:30:12.783394+0000","last_active":"2024-04-25T10:30:29.882628+0000","last_peered":"2024-04-25T10:30:29.882628+0000","last_clean":"2024-04-25T10:30:29.882628+0000","last_became_active":"2024-04-25T10:30:12.783066+0000","last_became_peered":"2024-04-25T10:30:12.783066+0000","last_unstale":"2024-04-25T10:30:29.882628+0000","last_undegraded":"2024-04-25T10:30:29.882628+0000","last_fullsized":"2024-04-25T10:30:29.882628+0000","mapping_epoch":34,"log_start":"0'0","ondisk_log_start":"0'0","created":34,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:30:11.733978+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:30:11.733978+0000","last_clean_scrub_stamp":"2024-04-25T10:30:11.733978+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T15:39:31.890064+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4],"acting":[2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"5.29","version":"36'8","reported_seq":35,"reported_epoch":44,"state":"active+clean","last_fresh":"2024-04-25T10:30:29.882807+0000","last_change":"2024-04-25T10:30:12.763911+0000","last_active":"2024-04-25T10:30:29.882807+0000","last_peered":"2024-04-25T10:30:29.882807+0000","last_clean":"2024-04-25T10:30:29.882807+0000","last_became_active":"2024-04-25T10:30:12.763578+0000","last_became_peered":"2024-04-25T10:30:12.763578+0000","last_unstale":"2024-04-25T10:30:29.882807+0000","last_undegraded":"2024-04-25T10:30:29.882807+0000","last_fullsized":"2024-04-25T10:30:29.882807+0000","mapping_epoch":34,"log_start":"0'0","ondisk_log_start":"0'0","created":34,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:30:11.733978+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:30:11.733978+0000","last_clean_scrub_stamp":"2024-04-25T10:30:11.733978+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T18:52:45.539758+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,1],"acting":[6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"5.2a","version":"36'9","reported_seq":38,"reported_epoch":44,"state":"active+clean","last_fresh":"2024-04-25T10:30:29.882996+0000","last_change":"2024-04-25T10:30:12.782730+0000","last_active":"2024-04-25T10:30:29.882996+0000","last_peered":"2024-04-25T10:30:29.882996+0000","last_clean":"2024-04-25T10:30:29.882996+0000","last_became_active":"2024-04-25T10:30:12.782261+0000","last_became_peered":"2024-04-25T10:30:12.782261+0000","last_unstale":"2024-04-25T10:30:29.882996+0000","last_undegraded":"2024-04-25T10:30:29.882996+0000","last_fullsized":"2024-04-25T10:30:29.882996+0000","mapping_epoch":34,"log_start":"0'0","ondisk_log_start":"0'0","created":34,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:30:11.733978+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:30:11.733978+0000","last_clean_scrub_stamp":"2024-04-25T10:30:11.733978+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T18:00:26.511360+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4],"acting":[2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"5.2b","version":"36'11","reported_seq":37,"reported_epoch":44,"state":"active+clean","last_fresh":"2024-04-25T10:30:30.798903+0000","last_change":"2024-04-25T10:30:12.771344+0000","last_active":"2024-04-25T10:30:30.798903+0000","last_peered":"2024-04-25T10:30:30.798903+0000","last_clean":"2024-04-25T10:30:30.798903+0000","last_became_active":"2024-04-25T10:30:12.770949+0000","last_became_peered":"2024-04-25T10:30:12.770949+0000","last_unstale":"2024-04-25T10:30:30.798903+0000","last_undegraded":"2024-04-25T10:30:30.798903+0000","last_fullsized":"2024-04-25T10:30:30.798903+0000","mapping_epoch":34,"log_start":"0'0","ondisk_log_start":"0'0","created":34,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:30:11.733978+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:30:11.733978+0000","last_clean_scrub_stamp":"2024-04-25T10:30:11.733978+0000","objects_scrubbed":0,"log_size":11,"log_dups_size":0,"ondisk_log_size":11,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T16:29:27.103512+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"5.2c","version":"36'8","reported_seq":35,"reported_epoch":44,"state":"active+clean","last_fresh":"2024-04-25T10:30:29.883520+0000","last_change":"2024-04-25T10:30:12.772663+0000","last_active":"2024-04-25T10:30:29.883520+0000","last_peered":"2024-04-25T10:30:29.883520+0000","last_clean":"2024-04-25T10:30:29.883520+0000","last_became_active":"2024-04-25T10:30:12.772356+0000","last_became_peered":"2024-04-25T10:30:12.772356+0000","last_unstale":"2024-04-25T10:30:29.883520+0000","last_undegraded":"2024-04-25T10:30:29.883520+0000","last_fullsized":"2024-04-25T10:30:29.883520+0000","mapping_epoch":34,"log_start":"0'0","ondisk_log_start":"0'0","created":34,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:30:11.733978+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:30:11.733978+0000","last_clean_scrub_stamp":"2024-04-25T10:30:11.733978+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T12:21:33.158775+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"5.2d","version":"36'7","reported_seq":33,"reported_epoch":44,"state":"active+clean","last_fresh":"2024-04-25T10:30:30.799380+0000","last_change":"2024-04-25T10:30:12.768688+0000","last_active":"2024-04-25T10:30:30.799380+0000","last_peered":"2024-04-25T10:30:30.799380+0000","last_clean":"2024-04-25T10:30:30.799380+0000","last_became_active":"2024-04-25T10:30:12.768382+0000","last_became_peered":"2024-04-25T10:30:12.768382+0000","last_unstale":"2024-04-25T10:30:30.799380+0000","last_undegraded":"2024-04-25T10:30:30.799380+0000","last_fullsized":"2024-04-25T10:30:30.799380+0000","mapping_epoch":34,"log_start":"0'0","ondisk_log_start":"0'0","created":34,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:30:11.733978+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:30:11.733978+0000","last_clean_scrub_stamp":"2024-04-25T10:30:11.733978+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T18:26:06.164667+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"5.2e","version":"36'10","reported_seq":36,"reported_epoch":44,"state":"active+clean","last_fresh":"2024-04-25T10:30:30.799258+0000","last_change":"2024-04-25T10:30:12.766393+0000","last_active":"2024-04-25T10:30:30.799258+0000","last_peered":"2024-04-25T10:30:30.799258+0000","last_clean":"2024-04-25T10:30:30.799258+0000","last_became_active":"2024-04-25T10:30:12.766096+0000","last_became_peered":"2024-04-25T10:30:12.766096+0000","last_unstale":"2024-04-25T10:30:30.799258+0000","last_undegraded":"2024-04-25T10:30:30.799258+0000","last_fullsized":"2024-04-25T10:30:30.799258+0000","mapping_epoch":34,"log_start":"0'0","ondisk_log_start":"0'0","created":34,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:30:11.733978+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:30:11.733978+0000","last_clean_scrub_stamp":"2024-04-25T10:30:11.733978+0000","objects_scrubbed":0,"log_size":10,"log_dups_size":0,"ondisk_log_size":10,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T16:49:00.268423+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"5.2f","version":"36'9","reported_seq":38,"reported_epoch":44,"state":"active+clean","last_fresh":"2024-04-25T10:30:29.882690+0000","last_change":"2024-04-25T10:30:12.766616+0000","last_active":"2024-04-25T10:30:29.882690+0000","last_peered":"2024-04-25T10:30:29.882690+0000","last_clean":"2024-04-25T10:30:29.882690+0000","last_became_active":"2024-04-25T10:30:12.766290+0000","last_became_peered":"2024-04-25T10:30:12.766290+0000","last_unstale":"2024-04-25T10:30:29.882690+0000","last_undegraded":"2024-04-25T10:30:29.882690+0000","last_fullsized":"2024-04-25T10:30:29.882690+0000","mapping_epoch":34,"log_start":"0'0","ondisk_log_start":"0'0","created":34,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:30:11.733978+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:30:11.733978+0000","last_clean_scrub_stamp":"2024-04-25T10:30:11.733978+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T13:22:50.975199+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":2097152,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":2048,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.30","version":"36'9","reported_seq":35,"reported_epoch":44,"state":"active+clean","last_fresh":"2024-04-25T10:30:30.798435+0000","last_change":"2024-04-25T10:30:12.765989+0000","last_active":"2024-04-25T10:30:30.798435+0000","last_peered":"2024-04-25T10:30:30.798435+0000","last_clean":"2024-04-25T10:30:30.798435+0000","last_became_active":"2024-04-25T10:30:12.765648+0000","last_became_peered":"2024-04-25T10:30:12.765648+0000","last_unstale":"2024-04-25T10:30:30.798435+0000","last_undegraded":"2024-04-25T10:30:30.798435+0000","last_fullsized":"2024-04-25T10:30:30.798435+0000","mapping_epoch":34,"log_start":"0'0","ondisk_log_start":"0'0","created":34,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:30:11.733978+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:30:11.733978+0000","last_clean_scrub_stamp":"2024-04-25T10:30:11.733978+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T16:29:40.621322+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"5.31","version":"36'7","reported_seq":33,"reported_epoch":44,"state":"active+clean","last_fresh":"2024-04-25T10:30:30.798713+0000","last_change":"2024-04-25T10:30:12.777324+0000","last_active":"2024-04-25T10:30:30.798713+0000","last_peered":"2024-04-25T10:30:30.798713+0000","last_clean":"2024-04-25T10:30:30.798713+0000","last_became_active":"2024-04-25T10:30:12.776983+0000","last_became_peered":"2024-04-25T10:30:12.776983+0000","last_unstale":"2024-04-25T10:30:30.798713+0000","last_undegraded":"2024-04-25T10:30:30.798713+0000","last_fullsized":"2024-04-25T10:30:30.798713+0000","mapping_epoch":34,"log_start":"0'0","ondisk_log_start":"0'0","created":34,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:30:11.733978+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:30:11.733978+0000","last_clean_scrub_stamp":"2024-04-25T10:30:11.733978+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T20:33:24.458243+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"5.32","version":"36'8","reported_seq":36,"reported_epoch":44,"state":"active+clean","last_fresh":"2024-04-25T10:30:30.800183+0000","last_change":"2024-04-25T10:30:12.772243+0000","last_active":"2024-04-25T10:30:30.800183+0000","last_peered":"2024-04-25T10:30:30.800183+0000","last_clean":"2024-04-25T10:30:30.800183+0000","last_became_active":"2024-04-25T10:30:12.771936+0000","last_became_peered":"2024-04-25T10:30:12.771936+0000","last_unstale":"2024-04-25T10:30:30.800183+0000","last_undegraded":"2024-04-25T10:30:30.800183+0000","last_fullsized":"2024-04-25T10:30:30.800183+0000","mapping_epoch":34,"log_start":"0'0","ondisk_log_start":"0'0","created":34,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:30:11.733978+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:30:11.733978+0000","last_clean_scrub_stamp":"2024-04-25T10:30:11.733978+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T14:03:40.703735+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5],"acting":[4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.33","version":"36'10","reported_seq":37,"reported_epoch":44,"state":"active+clean","last_fresh":"2024-04-25T10:30:29.883324+0000","last_change":"2024-04-25T10:30:12.777499+0000","last_active":"2024-04-25T10:30:29.883324+0000","last_peered":"2024-04-25T10:30:29.883324+0000","last_clean":"2024-04-25T10:30:29.883324+0000","last_became_active":"2024-04-25T10:30:12.776773+0000","last_became_peered":"2024-04-25T10:30:12.776773+0000","last_unstale":"2024-04-25T10:30:29.883324+0000","last_undegraded":"2024-04-25T10:30:29.883324+0000","last_fullsized":"2024-04-25T10:30:29.883324+0000","mapping_epoch":34,"log_start":"0'0","ondisk_log_start":"0'0","created":34,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:30:11.733978+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:30:11.733978+0000","last_clean_scrub_stamp":"2024-04-25T10:30:11.733978+0000","objects_scrubbed":0,"log_size":10,"log_dups_size":0,"ondisk_log_size":10,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T18:21:32.178524+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"5.34","version":"36'7","reported_seq":34,"reported_epoch":44,"state":"active+clean","last_fresh":"2024-04-25T10:30:30.799962+0000","last_change":"2024-04-25T10:30:12.761162+0000","last_active":"2024-04-25T10:30:30.799962+0000","last_peered":"2024-04-25T10:30:30.799962+0000","last_clean":"2024-04-25T10:30:30.799962+0000","last_became_active":"2024-04-25T10:30:12.760835+0000","last_became_peered":"2024-04-25T10:30:12.760835+0000","last_unstale":"2024-04-25T10:30:30.799962+0000","last_undegraded":"2024-04-25T10:30:30.799962+0000","last_fullsized":"2024-04-25T10:30:30.799962+0000","mapping_epoch":34,"log_start":"0'0","ondisk_log_start":"0'0","created":34,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:30:11.733978+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:30:11.733978+0000","last_clean_scrub_stamp":"2024-04-25T10:30:11.733978+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T18:59:20.693761+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"5.35","version":"36'6","reported_seq":35,"reported_epoch":44,"state":"active+clean","last_fresh":"2024-04-25T10:30:29.882303+0000","last_change":"2024-04-25T10:30:12.776164+0000","last_active":"2024-04-25T10:30:29.882303+0000","last_peered":"2024-04-25T10:30:29.882303+0000","last_clean":"2024-04-25T10:30:29.882303+0000","last_became_active":"2024-04-25T10:30:12.775455+0000","last_became_peered":"2024-04-25T10:30:12.775455+0000","last_unstale":"2024-04-25T10:30:29.882303+0000","last_undegraded":"2024-04-25T10:30:29.882303+0000","last_fullsized":"2024-04-25T10:30:29.882303+0000","mapping_epoch":34,"log_start":"0'0","ondisk_log_start":"0'0","created":34,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:30:11.733978+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:30:11.733978+0000","last_clean_scrub_stamp":"2024-04-25T10:30:11.733978+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T21:45:24.780979+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.36","version":"36'15","reported_seq":43,"reported_epoch":44,"state":"active+clean","last_fresh":"2024-04-25T10:30:30.800175+0000","last_change":"2024-04-25T10:30:12.773138+0000","last_active":"2024-04-25T10:30:30.800175+0000","last_peered":"2024-04-25T10:30:30.800175+0000","last_clean":"2024-04-25T10:30:30.800175+0000","last_became_active":"2024-04-25T10:30:12.772854+0000","last_became_peered":"2024-04-25T10:30:12.772854+0000","last_unstale":"2024-04-25T10:30:30.800175+0000","last_undegraded":"2024-04-25T10:30:30.800175+0000","last_fullsized":"2024-04-25T10:30:30.800175+0000","mapping_epoch":34,"log_start":"0'0","ondisk_log_start":"0'0","created":34,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:30:11.733978+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:30:11.733978+0000","last_clean_scrub_stamp":"2024-04-25T10:30:11.733978+0000","objects_scrubbed":0,"log_size":15,"log_dups_size":0,"ondisk_log_size":15,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T12:33:38.324566+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5],"acting":[4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.37","version":"36'11","reported_seq":38,"reported_epoch":44,"state":"active+clean","last_fresh":"2024-04-25T10:30:30.799736+0000","last_change":"2024-04-25T10:30:12.773184+0000","last_active":"2024-04-25T10:30:30.799736+0000","last_peered":"2024-04-25T10:30:30.799736+0000","last_clean":"2024-04-25T10:30:30.799736+0000","last_became_active":"2024-04-25T10:30:12.772866+0000","last_became_peered":"2024-04-25T10:30:12.772866+0000","last_unstale":"2024-04-25T10:30:30.799736+0000","last_undegraded":"2024-04-25T10:30:30.799736+0000","last_fullsized":"2024-04-25T10:30:30.799736+0000","mapping_epoch":34,"log_start":"0'0","ondisk_log_start":"0'0","created":34,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:30:11.733978+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:30:11.733978+0000","last_clean_scrub_stamp":"2024-04-25T10:30:11.733978+0000","objects_scrubbed":0,"log_size":11,"log_dups_size":0,"ondisk_log_size":11,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T14:01:06.791796+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"5.38","version":"36'9","reported_seq":38,"reported_epoch":44,"state":"active+clean","last_fresh":"2024-04-25T10:30:29.882611+0000","last_change":"2024-04-25T10:30:12.774614+0000","last_active":"2024-04-25T10:30:29.882611+0000","last_peered":"2024-04-25T10:30:29.882611+0000","last_clean":"2024-04-25T10:30:29.882611+0000","last_became_active":"2024-04-25T10:30:12.774213+0000","last_became_peered":"2024-04-25T10:30:12.774213+0000","last_unstale":"2024-04-25T10:30:29.882611+0000","last_undegraded":"2024-04-25T10:30:29.882611+0000","last_fullsized":"2024-04-25T10:30:29.882611+0000","mapping_epoch":34,"log_start":"0'0","ondisk_log_start":"0'0","created":34,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:30:11.733978+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:30:11.733978+0000","last_clean_scrub_stamp":"2024-04-25T10:30:11.733978+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T20:33:24.458243+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,6],"acting":[2,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"5.39","version":"36'5","reported_seq":34,"reported_epoch":44,"state":"active+clean","last_fresh":"2024-04-25T10:30:29.882763+0000","last_change":"2024-04-25T10:30:12.781210+0000","last_active":"2024-04-25T10:30:29.882763+0000","last_peered":"2024-04-25T10:30:29.882763+0000","last_clean":"2024-04-25T10:30:29.882763+0000","last_became_active":"2024-04-25T10:30:12.780874+0000","last_became_peered":"2024-04-25T10:30:12.780874+0000","last_unstale":"2024-04-25T10:30:29.882763+0000","last_undegraded":"2024-04-25T10:30:29.882763+0000","last_fullsized":"2024-04-25T10:30:29.882763+0000","mapping_epoch":34,"log_start":"0'0","ondisk_log_start":"0'0","created":34,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:30:11.733978+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:30:11.733978+0000","last_clean_scrub_stamp":"2024-04-25T10:30:11.733978+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T16:29:40.621322+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"5.3a","version":"36'6","reported_seq":32,"reported_epoch":44,"state":"active+clean","last_fresh":"2024-04-25T10:30:30.799382+0000","last_change":"2024-04-25T10:30:12.775534+0000","last_active":"2024-04-25T10:30:30.799382+0000","last_peered":"2024-04-25T10:30:30.799382+0000","last_clean":"2024-04-25T10:30:30.799382+0000","last_became_active":"2024-04-25T10:30:12.775180+0000","last_became_peered":"2024-04-25T10:30:12.775180+0000","last_unstale":"2024-04-25T10:30:30.799382+0000","last_undegraded":"2024-04-25T10:30:30.799382+0000","last_fullsized":"2024-04-25T10:30:30.799382+0000","mapping_epoch":34,"log_start":"0'0","ondisk_log_start":"0'0","created":34,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:30:11.733978+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:30:11.733978+0000","last_clean_scrub_stamp":"2024-04-25T10:30:11.733978+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T18:31:49.260513+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"5.3b","version":"36'4","reported_seq":30,"reported_epoch":44,"state":"active+clean","last_fresh":"2024-04-25T10:30:30.798879+0000","last_change":"2024-04-25T10:30:12.762141+0000","last_active":"2024-04-25T10:30:30.798879+0000","last_peered":"2024-04-25T10:30:30.798879+0000","last_clean":"2024-04-25T10:30:30.798879+0000","last_became_active":"2024-04-25T10:30:12.761805+0000","last_became_peered":"2024-04-25T10:30:12.761805+0000","last_unstale":"2024-04-25T10:30:30.798879+0000","last_undegraded":"2024-04-25T10:30:30.798879+0000","last_fullsized":"2024-04-25T10:30:30.798879+0000","mapping_epoch":34,"log_start":"0'0","ondisk_log_start":"0'0","created":34,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:30:11.733978+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:30:11.733978+0000","last_clean_scrub_stamp":"2024-04-25T10:30:11.733978+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T19:56:57.765496+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6],"acting":[5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"5.3c","version":"36'8","reported_seq":36,"reported_epoch":44,"state":"active+clean","last_fresh":"2024-04-25T10:30:30.800691+0000","last_change":"2024-04-25T10:30:12.781723+0000","last_active":"2024-04-25T10:30:30.800691+0000","last_peered":"2024-04-25T10:30:30.800691+0000","last_clean":"2024-04-25T10:30:30.800691+0000","last_became_active":"2024-04-25T10:30:12.781357+0000","last_became_peered":"2024-04-25T10:30:12.781357+0000","last_unstale":"2024-04-25T10:30:30.800691+0000","last_undegraded":"2024-04-25T10:30:30.800691+0000","last_fullsized":"2024-04-25T10:30:30.800691+0000","mapping_epoch":34,"log_start":"0'0","ondisk_log_start":"0'0","created":34,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:30:11.733978+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:30:11.733978+0000","last_clean_scrub_stamp":"2024-04-25T10:30:11.733978+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T15:28:31.731686+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.3d","version":"36'7","reported_seq":34,"reported_epoch":44,"state":"active+clean","last_fresh":"2024-04-25T10:30:30.799661+0000","last_change":"2024-04-25T10:30:12.768292+0000","last_active":"2024-04-25T10:30:30.799661+0000","last_peered":"2024-04-25T10:30:30.799661+0000","last_clean":"2024-04-25T10:30:30.799661+0000","last_became_active":"2024-04-25T10:30:12.767909+0000","last_became_peered":"2024-04-25T10:30:12.767909+0000","last_unstale":"2024-04-25T10:30:30.799661+0000","last_undegraded":"2024-04-25T10:30:30.799661+0000","last_fullsized":"2024-04-25T10:30:30.799661+0000","mapping_epoch":34,"log_start":"0'0","ondisk_log_start":"0'0","created":34,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:30:11.733978+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:30:11.733978+0000","last_clean_scrub_stamp":"2024-04-25T10:30:11.733978+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T15:17:30.002665+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"5.3e","version":"36'4","reported_seq":30,"reported_epoch":44,"state":"active+clean","last_fresh":"2024-04-25T10:30:30.799933+0000","last_change":"2024-04-25T10:30:12.780210+0000","last_active":"2024-04-25T10:30:30.799933+0000","last_peered":"2024-04-25T10:30:30.799933+0000","last_clean":"2024-04-25T10:30:30.799933+0000","last_became_active":"2024-04-25T10:30:12.779867+0000","last_became_peered":"2024-04-25T10:30:12.779867+0000","last_unstale":"2024-04-25T10:30:30.799933+0000","last_undegraded":"2024-04-25T10:30:30.799933+0000","last_fullsized":"2024-04-25T10:30:30.799933+0000","mapping_epoch":34,"log_start":"0'0","ondisk_log_start":"0'0","created":34,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:30:11.733978+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:30:11.733978+0000","last_clean_scrub_stamp":"2024-04-25T10:30:11.733978+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T20:33:24.458243+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"5.3f","version":"36'8","reported_seq":37,"reported_epoch":44,"state":"active+clean","last_fresh":"2024-04-25T10:30:29.883448+0000","last_change":"2024-04-25T10:30:12.772509+0000","last_active":"2024-04-25T10:30:29.883448+0000","last_peered":"2024-04-25T10:30:29.883448+0000","last_clean":"2024-04-25T10:30:29.883448+0000","last_became_active":"2024-04-25T10:30:12.772018+0000","last_became_peered":"2024-04-25T10:30:12.772018+0000","last_unstale":"2024-04-25T10:30:29.883448+0000","last_undegraded":"2024-04-25T10:30:29.883448+0000","last_fullsized":"2024-04-25T10:30:29.883448+0000","mapping_epoch":34,"log_start":"0'0","ondisk_log_start":"0'0","created":34,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:30:11.733978+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:30:11.733978+0000","last_clean_scrub_stamp":"2024-04-25T10:30:11.733978+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T11:57:29.979255+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]}],"pool_stats":[{"poolid":6,"num_pg":32,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":64,"acting":64,"num_store_stats":8},{"poolid":5,"num_pg":64,"stat_sum":{"num_bytes":6291456,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":6144,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":12582912,"data_stored":12582912,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":504,"ondisk_log_size":504,"up":128,"acting":128,"num_store_stats":8},{"poolid":4,"num_pg":32,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":1,"num_read_kb":0,"num_write":96,"num_write_kb":40,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":83,"ondisk_log_size":83,"up":64,"acting":64,"num_store_stats":8},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":66,"num_read_kb":55,"num_write":101,"num_write_kb":1628,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1187840,"data_stored":1180736,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":72,"ondisk_log_size":72,"up":2,"acting":2,"num_store_stats":2}],"osd_stats":[{"osd":7,"up_from":15,"seq":64424509463,"num_pgs":26,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":31604,"kb_used_data":4584,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94340236,"statfs":{"total":96636764160,"available":96604401664,"internally_reserved":0,"allocated":4694016,"data_stored":4401741,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1590,"internal_metadata":27654602},"hb_peers":[0,1,2,3,4,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]},{"osd":6,"up_from":15,"seq":64424509464,"num_pgs":34,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27508,"kb_used_data":488,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344332,"statfs":{"total":96636764160,"available":96608595968,"internally_reserved":0,"allocated":499712,"data_stored":207437,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":27654598},"hb_peers":[0,1,2,3,4,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":5,"up_from":15,"seq":64424509465,"num_pgs":24,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":28088,"kb_used_data":1068,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94343752,"statfs":{"total":96636764160,"available":96608002048,"internally_reserved":0,"allocated":1093632,"data_stored":797805,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]},{"osd":3,"up_from":15,"seq":64424509465,"num_pgs":34,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":33652,"kb_used_data":6632,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94338188,"statfs":{"total":96636764160,"available":96602304512,"internally_reserved":0,"allocated":6791168,"data_stored":6498893,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1593,"internal_metadata":27654599},"hb_peers":[0,1,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":2,"up_from":15,"seq":64424509465,"num_pgs":32,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":29556,"kb_used_data":2536,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94342284,"statfs":{"total":96636764160,"available":96606498816,"internally_reserved":0,"allocated":2596864,"data_stored":2304589,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":1,"up_from":15,"seq":64424509463,"num_pgs":33,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":28088,"kb_used_data":1068,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94343752,"statfs":{"total":96636764160,"available":96608002048,"internally_reserved":0,"allocated":1093632,"data_stored":797805,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]},{"osd":0,"up_from":15,"seq":64424509464,"num_pgs":30,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27508,"kb_used_data":488,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344332,"statfs":{"total":96636764160,"available":96608595968,"internally_reserved":0,"allocated":499712,"data_stored":207437,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[1,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]},{"osd":4,"up_from":13,"seq":55834574871,"num_pgs":45,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":31672,"kb_used_data":488,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94340168,"statfs":{"total":96636764160,"available":96604332032,"internally_reserved":0,"allocated":499712,"data_stored":207437,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[0,1,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":2097152,"data_stored":2097152,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":6291456,"data_stored":6291456,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":4194304,"data_stored":4194304,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-04-25T10:30:37.812 INFO:tasks.cephfs.filesystem.ceph_manager:clean! 2024-04-25T10:30:37.812 DEBUG:teuthology.orchestra.run.smithi037:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph -s 2024-04-25T10:30:37.879 DEBUG:teuthology.orchestra.run.smithi037:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-25T10:30:37.894 DEBUG:teuthology.orchestra.run.smithi203:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-25T10:30:37.956 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:37.954+0000 7f596b242640 1 Processor -- start 2024-04-25T10:30:37.957 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:37.954+0000 7f596b242640 1 -- start start 2024-04-25T10:30:37.958 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:37.956+0000 7f596b242640 1 --2- >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f59640c8270 0x7f59640b7720 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:30:37.959 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:37.956+0000 7f596b242640 1 --2- >> v2:172.21.15.37:3300/0 conn(0x7f59640c8940 0x7f59640c6f10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:30:37.959 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:37.956+0000 7f596b242640 1 -- --> v1:172.21.15.37:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f596406cba0 con 0x7f596406d300 2024-04-25T10:30:37.959 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:37.956+0000 7f596b242640 1 -- --> v2:172.21.15.37:3300/0 -- mon_getmap magic: 0 -- 0x7f59640c8ee0 con 0x7f59640c8940 2024-04-25T10:30:37.959 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:37.956+0000 7f596b242640 1 -- --> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] -- mon_getmap magic: 0 -- 0x7f596406c5e0 con 0x7f59640c8270 2024-04-25T10:30:37.959 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:37.956+0000 7f5963fff640 1 --2- >> v2:172.21.15.37:3300/0 conn(0x7f59640c8940 0x7f59640c6f10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:30:37.959 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:37.956+0000 7f59697df640 1 --1- >> v1:172.21.15.37:6789/0 conn(0x7f596406d300 0x7f596406d710 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.37:6789/0 says I am v1:172.21.15.37:40938/0 (socket says 172.21.15.37:40938) 2024-04-25T10:30:37.959 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:37.956+0000 7f59697df640 1 -- 172.21.15.37:0/2565544000 learned_addr learned my addr 172.21.15.37:0/2565544000 (peer_addr_for_me v1:172.21.15.37:0/0) 2024-04-25T10:30:37.959 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:37.956+0000 7f5968fde640 1 --2- 172.21.15.37:0/2565544000 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f59640c8270 0x7f59640b7720 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:30:37.959 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:37.956+0000 7f5963fff640 1 -- 172.21.15.37:0/2565544000 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f59640c8270 msgr2=0x7f59640b7720 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:30:37.959 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:37.956+0000 7f5963fff640 1 --2- 172.21.15.37:0/2565544000 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f59640c8270 0x7f59640b7720 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:30:37.959 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:37.956+0000 7f5963fff640 1 -- 172.21.15.37:0/2565544000 >> v1:172.21.15.37:6789/0 conn(0x7f596406d300 legacy=0x7f596406d710 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:30:37.959 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:37.956+0000 7f59637fe640 1 -- 172.21.15.37:0/2565544000 <== mon.0 v1:172.21.15.37:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 1446327365 0 0) 0x7f596406cba0 con 0x7f596406d300 2024-04-25T10:30:37.959 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:37.956+0000 7f5963fff640 1 -- 172.21.15.37:0/2565544000 --> v2:172.21.15.37:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f596406c8c0 con 0x7f59640c8940 2024-04-25T10:30:37.960 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:37.956+0000 7f5968fde640 1 --2- 172.21.15.37:0/2565544000 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f59640c8270 0x7f59640b7720 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T10:30:37.961 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:37.957+0000 7f5963fff640 1 --2- 172.21.15.37:0/2565544000 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f59640c8940 0x7f59640c6f10 secure :-1 s=READY pgs=543 cs=0 l=1 rev1=1 crypto rx=0x7f59500097a0 tx=0x7f595002a060 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ace274e3db49346f server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:30:37.961 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:37.957+0000 7f59637fe640 1 -- 172.21.15.37:0/2565544000 <== mon.0 v2:172.21.15.37:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5950037070 con 0x7f59640c8940 2024-04-25T10:30:37.961 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:37.957+0000 7f59637fe640 1 -- 172.21.15.37:0/2565544000 <== mon.0 v2:172.21.15.37:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f595002adb0 con 0x7f59640c8940 2024-04-25T10:30:37.961 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:37.957+0000 7f59637fe640 1 -- 172.21.15.37:0/2565544000 <== mon.0 v2:172.21.15.37:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5950032660 con 0x7f59640c8940 2024-04-25T10:30:37.961 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:37.958+0000 7f596b242640 1 -- 172.21.15.37:0/2565544000 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f59640c8940 msgr2=0x7f59640c6f10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:30:37.961 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:37.958+0000 7f596b242640 1 --2- 172.21.15.37:0/2565544000 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f59640c8940 0x7f59640c6f10 secure :-1 s=READY pgs=543 cs=0 l=1 rev1=1 crypto rx=0x7f59500097a0 tx=0x7f595002a060 comp rx=0 tx=0).stop 2024-04-25T10:30:37.961 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:37.958+0000 7f596b242640 1 -- 172.21.15.37:0/2565544000 shutdown_connections 2024-04-25T10:30:37.961 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:37.958+0000 7f596b242640 1 --2- 172.21.15.37:0/2565544000 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f59640c8940 0x7f59640c6f10 unknown :-1 s=CLOSED pgs=543 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:30:37.961 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:37.958+0000 7f596b242640 1 --2- 172.21.15.37:0/2565544000 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f59640c8270 0x7f59640b7720 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:30:37.961 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:37.958+0000 7f596b242640 1 -- 172.21.15.37:0/2565544000 >> 172.21.15.37:0/2565544000 conn(0x7f596409b700 msgr2=0x7f59640b6f80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:30:37.961 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:37.958+0000 7f596b242640 1 -- 172.21.15.37:0/2565544000 shutdown_connections 2024-04-25T10:30:37.961 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:37.958+0000 7f596b242640 1 -- 172.21.15.37:0/2565544000 wait complete. 2024-04-25T10:30:37.962 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:37.960+0000 7f596b242640 1 Processor -- start 2024-04-25T10:30:37.962 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:37.960+0000 7f596b242640 1 -- start start 2024-04-25T10:30:37.965 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:37.961+0000 7f596b242640 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f596406d300 0x7f59640aff60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:30:37.965 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:37.961+0000 7f596b242640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f59640c8270 0x7f596415c9b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:30:37.965 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:37.961+0000 7f596b242640 1 --2- >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f59640c8940 0x7f59640ca270 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:30:37.965 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:37.961+0000 7f596b242640 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 -- 0x7f59640c75d0 con 0x7f59640c8270 2024-04-25T10:30:37.965 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:37.961+0000 7f596b242640 1 -- --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_getmap magic: 0 -- 0x7f59640c7450 con 0x7f596406d300 2024-04-25T10:30:37.965 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:37.961+0000 7f596b242640 1 -- --> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] -- mon_getmap magic: 0 -- 0x7f59640cef00 con 0x7f59640c8940 2024-04-25T10:30:37.965 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:37.961+0000 7f5963fff640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f59640c8270 0x7f596415c9b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:30:37.965 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:37.961+0000 7f5963fff640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f59640c8270 0x7f596415c9b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.37:3300/0 says I am v2:172.21.15.37:55522/0 (socket says 172.21.15.37:55522) 2024-04-25T10:30:37.965 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:37.961+0000 7f5963fff640 1 -- 172.21.15.37:0/3132573825 learned_addr learned my addr 172.21.15.37:0/3132573825 (peer_addr_for_me v2:172.21.15.37:0/0) 2024-04-25T10:30:37.965 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:37.962+0000 7f59697df640 1 --2- 172.21.15.37:0/3132573825 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f59640c8940 0x7f59640ca270 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:30:37.965 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:37.962+0000 7f5968fde640 1 --2- 172.21.15.37:0/3132573825 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f596406d300 0x7f59640aff60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:30:37.965 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:37.962+0000 7f5963fff640 1 -- 172.21.15.37:0/3132573825 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f59640c8940 msgr2=0x7f59640ca270 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:30:37.965 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:37.962+0000 7f5963fff640 1 --2- 172.21.15.37:0/3132573825 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f59640c8940 0x7f59640ca270 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:30:37.966 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:37.962+0000 7f5963fff640 1 -- 172.21.15.37:0/3132573825 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f596406d300 msgr2=0x7f59640aff60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:30:37.966 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:37.962+0000 7f5963fff640 1 --2- 172.21.15.37:0/3132573825 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f596406d300 0x7f59640aff60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:30:37.966 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:37.962+0000 7f5963fff640 1 -- 172.21.15.37:0/3132573825 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f59640c91f0 con 0x7f59640c8270 2024-04-25T10:30:37.966 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:37.962+0000 7f5968fde640 1 --2- 172.21.15.37:0/3132573825 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f596406d300 0x7f59640aff60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T10:30:37.966 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:37.962+0000 7f5963fff640 1 --2- 172.21.15.37:0/3132573825 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f59640c8270 0x7f596415c9b0 secure :-1 s=READY pgs=544 cs=0 l=1 rev1=1 crypto rx=0x7f5950035040 tx=0x7f59500046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:30:37.966 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:37.962+0000 7f59617fa640 1 -- 172.21.15.37:0/3132573825 <== mon.0 v2:172.21.15.37:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5950042020 con 0x7f59640c8270 2024-04-25T10:30:37.966 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:37.962+0000 7f59617fa640 1 -- 172.21.15.37:0/3132573825 <== mon.0 v2:172.21.15.37:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f595003d670 con 0x7f59640c8270 2024-04-25T10:30:37.966 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:37.962+0000 7f596b242640 1 -- 172.21.15.37:0/3132573825 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f59640c94b0 con 0x7f59640c8270 2024-04-25T10:30:37.966 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:37.962+0000 7f59617fa640 1 -- 172.21.15.37:0/3132573825 <== mon.0 v2:172.21.15.37:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5950037040 con 0x7f59640c8270 2024-04-25T10:30:37.966 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:37.962+0000 7f596b242640 1 -- 172.21.15.37:0/3132573825 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f59640c9a70 con 0x7f59640c8270 2024-04-25T10:30:37.967 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:37.963+0000 7f596b242640 1 -- 172.21.15.37:0/3132573825 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f596406a140 con 0x7f59640c8270 2024-04-25T10:30:37.967 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:37.964+0000 7f59617fa640 1 -- 172.21.15.37:0/3132573825 <== mon.0 v2:172.21.15.37:3300/0 4 ==== mgrmap(e 5) ==== 96782+0+0 (secure 0 0 0) 0x7f5950045020 con 0x7f59640c8270 2024-04-25T10:30:37.967 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:37.964+0000 7f59617fa640 1 --2- 172.21.15.37:0/3132573825 >> [v2:172.21.15.203:6832/2418205237,v1:172.21.15.203:6833/2418205237] conn(0x7f59480744e0 0x7f59480769f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:30:37.967 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:37.964+0000 7f59617fa640 1 -- 172.21.15.37:0/3132573825 <== mon.0 v2:172.21.15.37:3300/0 5 ==== osd_map(44..44 src has 1..44) ==== 6248+0+0 (secure 0 0 0) 0x7f59500b5d30 con 0x7f59640c8270 2024-04-25T10:30:37.967 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:37.965+0000 7f5968fde640 1 --2- 172.21.15.37:0/3132573825 >> [v2:172.21.15.203:6832/2418205237,v1:172.21.15.203:6833/2418205237] conn(0x7f59480744e0 0x7f59480769f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:30:37.967 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:37.965+0000 7f5968fde640 1 --2- 172.21.15.37:0/3132573825 >> [v2:172.21.15.203:6832/2418205237,v1:172.21.15.203:6833/2418205237] conn(0x7f59480744e0 0x7f59480769f0 secure :-1 s=READY pgs=260 cs=0 l=1 rev1=1 crypto rx=0x7f5964072a70 tx=0x7f5958008040 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:30:37.971 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:37.969+0000 7f59617fa640 1 -- 172.21.15.37:0/3132573825 <== mon.0 v2:172.21.15.37:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187246 (secure 0 0 0) 0x7f595004c030 con 0x7f59640c8270 2024-04-25T10:30:38.228 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:38.226+0000 7f596b242640 1 -- 172.21.15.37:0/3132573825 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_command({"prefix": "status"} v 0) -- 0x7f59640c7b30 con 0x7f59640c8270 2024-04-25T10:30:38.228 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:38.226+0000 7f59617fa640 1 -- 172.21.15.37:0/3132573825 <== mon.0 v2:172.21.15.37:3300/0 7 ==== mon_command_ack([{"prefix": "status"}]=0 v0) ==== 54+0+605 (secure 0 0 0) 0x7f5950080200 con 0x7f59640c8270 2024-04-25T10:30:38.229 INFO:teuthology.orchestra.run.smithi037.stdout: cluster: 2024-04-25T10:30:38.229 INFO:teuthology.orchestra.run.smithi037.stdout: id: f4acda28-be34-492a-b286-994afd36f61d 2024-04-25T10:30:38.229 INFO:teuthology.orchestra.run.smithi037.stdout: health: HEALTH_ERR 2024-04-25T10:30:38.229 INFO:teuthology.orchestra.run.smithi037.stdout: 2 filesystems are degraded 2024-04-25T10:30:38.229 INFO:teuthology.orchestra.run.smithi037.stdout: 2 filesystems are offline 2024-04-25T10:30:38.229 INFO:teuthology.orchestra.run.smithi037.stdout: 2024-04-25T10:30:38.229 INFO:teuthology.orchestra.run.smithi037.stdout: services: 2024-04-25T10:30:38.229 INFO:teuthology.orchestra.run.smithi037.stdout: mon: 3 daemons, quorum a,b,c (age 114s) 2024-04-25T10:30:38.229 INFO:teuthology.orchestra.run.smithi037.stdout: mgr: x(active, since 110s), standbys: y 2024-04-25T10:30:38.229 INFO:teuthology.orchestra.run.smithi037.stdout: mds: 0/2 daemons up (2 failed), 4 standby 2024-04-25T10:30:38.229 INFO:teuthology.orchestra.run.smithi037.stdout: osd: 8 osds: 8 up (since 103s), 8 in (since 112s) 2024-04-25T10:30:38.230 INFO:teuthology.orchestra.run.smithi037.stdout: 2024-04-25T10:30:38.230 INFO:teuthology.orchestra.run.smithi037.stdout: data: 2024-04-25T10:30:38.230 INFO:teuthology.orchestra.run.smithi037.stdout: volumes: 0/2 healthy, 2 failed 2024-04-25T10:30:38.230 INFO:teuthology.orchestra.run.smithi037.stdout: pools: 4 pools, 129 pgs 2024-04-25T10:30:38.230 INFO:teuthology.orchestra.run.smithi037.stdout: objects: 4 objects, 6.6 MiB 2024-04-25T10:30:38.230 INFO:teuthology.orchestra.run.smithi037.stdout: usage: 232 MiB used, 720 GiB / 720 GiB avail 2024-04-25T10:30:38.230 INFO:teuthology.orchestra.run.smithi037.stdout: pgs: 129 active+clean 2024-04-25T10:30:38.230 INFO:teuthology.orchestra.run.smithi037.stdout: 2024-04-25T10:30:38.230 INFO:teuthology.orchestra.run.smithi037.stdout: io: 2024-04-25T10:30:38.230 INFO:teuthology.orchestra.run.smithi037.stdout: client: 0 B/s wr, 0 op/s rd, 2 op/s wr 2024-04-25T10:30:38.230 INFO:teuthology.orchestra.run.smithi037.stdout: 2024-04-25T10:30:38.231 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:38.228+0000 7f596b242640 1 -- 172.21.15.37:0/3132573825 >> [v2:172.21.15.203:6832/2418205237,v1:172.21.15.203:6833/2418205237] conn(0x7f59480744e0 msgr2=0x7f59480769f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:30:38.231 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:38.228+0000 7f596b242640 1 --2- 172.21.15.37:0/3132573825 >> [v2:172.21.15.203:6832/2418205237,v1:172.21.15.203:6833/2418205237] conn(0x7f59480744e0 0x7f59480769f0 secure :-1 s=READY pgs=260 cs=0 l=1 rev1=1 crypto rx=0x7f5964072a70 tx=0x7f5958008040 comp rx=0 tx=0).stop 2024-04-25T10:30:38.231 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:38.228+0000 7f596b242640 1 -- 172.21.15.37:0/3132573825 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f59640c8270 msgr2=0x7f596415c9b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:30:38.231 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:38.228+0000 7f596b242640 1 --2- 172.21.15.37:0/3132573825 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f59640c8270 0x7f596415c9b0 secure :-1 s=READY pgs=544 cs=0 l=1 rev1=1 crypto rx=0x7f5950035040 tx=0x7f59500046c0 comp rx=0 tx=0).stop 2024-04-25T10:30:38.231 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:38.229+0000 7f596b242640 1 -- 172.21.15.37:0/3132573825 shutdown_connections 2024-04-25T10:30:38.232 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:38.229+0000 7f596b242640 1 --2- 172.21.15.37:0/3132573825 >> [v2:172.21.15.203:6832/2418205237,v1:172.21.15.203:6833/2418205237] conn(0x7f59480744e0 0x7f59480769f0 unknown :-1 s=CLOSED pgs=260 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:30:38.232 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:38.229+0000 7f596b242640 1 --2- 172.21.15.37:0/3132573825 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f59640c8940 0x7f59640ca270 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:30:38.232 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:38.229+0000 7f596b242640 1 --2- 172.21.15.37:0/3132573825 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f59640c8270 0x7f596415c9b0 unknown :-1 s=CLOSED pgs=544 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:30:38.232 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:38.229+0000 7f596b242640 1 --2- 172.21.15.37:0/3132573825 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f596406d300 0x7f59640aff60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:30:38.232 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:38.229+0000 7f596b242640 1 -- 172.21.15.37:0/3132573825 >> 172.21.15.37:0/3132573825 conn(0x7f596409b700 msgr2=0x7f59640cd2a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:30:38.232 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:38.229+0000 7f596b242640 1 -- 172.21.15.37:0/3132573825 shutdown_connections 2024-04-25T10:30:38.232 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:38.229+0000 7f596b242640 1 -- 172.21.15.37:0/3132573825 wait complete. 2024-04-25T10:30:38.242 DEBUG:teuthology.orchestra.run.smithi037:> cephfs-data-scan --debug-mds=20 --debug-ms=1 --debug-objecter=1 init --force-init --filesystem recovery_fs --alternate-pool recovery_fs_metadata 2024-04-25T10:30:38.337 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:38.335+0000 7f4cf8a1d740 1 -- start start 2024-04-25T10:30:38.337 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:38.335+0000 7f4cf8a1d740 1 Processor -- start 2024-04-25T10:30:38.339 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:38.336+0000 7f4cf8a1d740 1 --2- >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x5575c3d81680 0x5575c3d5a580 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:30:38.340 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:38.336+0000 7f4cf8a1d740 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x5575c3d81b00 0x5575c3d5a000 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:30:38.340 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:38.336+0000 7f4cf8a1d740 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x5575c3d81f80 0x5575c3d5ab00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:30:38.340 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:38.336+0000 7f4cf8a1d740 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 -- 0x5575c3ca6780 con 0x5575c3d81b00 2024-04-25T10:30:38.340 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:38.336+0000 7f4cf8a1d740 1 -- --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_getmap magic: 0 -- 0x5575c3ca6600 con 0x5575c3d81f80 2024-04-25T10:30:38.340 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:38.336+0000 7f4cf8a1d740 1 -- --> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] -- mon_getmap magic: 0 -- 0x5575c3ca6480 con 0x5575c3d81680 2024-04-25T10:30:38.340 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:38.336+0000 7f4cf5f0d640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x5575c3d81b00 0x5575c3d5a000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:30:38.340 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:38.336+0000 7f4cf5f0d640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x5575c3d81b00 0x5575c3d5a000 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.37:3300/0 says I am v2:172.21.15.37:55530/0 (socket says 172.21.15.37:55530) 2024-04-25T10:30:38.340 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:38.336+0000 7f4cf5f0d640 1 -- 172.21.15.37:0/1454804361 learned_addr learned my addr 172.21.15.37:0/1454804361 (peer_addr_for_me v2:172.21.15.37:0/0) 2024-04-25T10:30:38.340 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:38.336+0000 7f4cf6f0f640 1 --2- 172.21.15.37:0/1454804361 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x5575c3d81f80 0x5575c3d5ab00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:30:38.340 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:38.336+0000 7f4cf670e640 1 --2- 172.21.15.37:0/1454804361 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x5575c3d81680 0x5575c3d5a580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:30:38.340 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:38.336+0000 7f4cf5f0d640 1 -- 172.21.15.37:0/1454804361 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x5575c3d81680 msgr2=0x5575c3d5a580 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:30:38.340 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:38.336+0000 7f4cf5f0d640 1 --2- 172.21.15.37:0/1454804361 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x5575c3d81680 0x5575c3d5a580 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:30:38.340 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:38.336+0000 7f4cf5f0d640 1 -- 172.21.15.37:0/1454804361 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x5575c3d81f80 msgr2=0x5575c3d5ab00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:30:38.341 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:38.336+0000 7f4cf5f0d640 1 --2- 172.21.15.37:0/1454804361 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x5575c3d81f80 0x5575c3d5ab00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:30:38.341 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:38.336+0000 7f4cf5f0d640 1 -- 172.21.15.37:0/1454804361 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5575bfbe5680 con 0x5575c3d81b00 2024-04-25T10:30:38.341 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:38.336+0000 7f4cf6f0f640 1 --2- 172.21.15.37:0/1454804361 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x5575c3d81f80 0x5575c3d5ab00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-25T10:30:38.341 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:38.336+0000 7f4cf670e640 1 --2- 172.21.15.37:0/1454804361 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x5575c3d81680 0x5575c3d5a580 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-25T10:30:38.341 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:38.337+0000 7f4cf5f0d640 1 --2- 172.21.15.37:0/1454804361 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x5575c3d81b00 0x5575c3d5a000 secure :-1 s=READY pgs=545 cs=0 l=1 rev1=1 crypto rx=0x5575c4a11200 tx=0x5575c3ce8a20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ac66c0736a3179eb server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:30:38.341 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:38.337+0000 7f4cf3f09640 1 -- 172.21.15.37:0/1454804361 <== mon.0 v2:172.21.15.37:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x5575bfbde9c0 con 0x5575c3d81b00 2024-04-25T10:30:38.341 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:38.337+0000 7f4cf3f09640 1 -- 172.21.15.37:0/1454804361 <== mon.0 v2:172.21.15.37:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x5575c3c30e00 con 0x5575c3d81b00 2024-04-25T10:30:38.341 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:38.337+0000 7f4cf3f09640 1 -- 172.21.15.37:0/1454804361 <== mon.0 v2:172.21.15.37:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x5575bfbde680 con 0x5575c3d81b00 2024-04-25T10:30:38.341 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:38.337+0000 7f4cf8a1d740 1 -- 172.21.15.37:0/1454804361 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5575bfbe5860 con 0x5575c3d81b00 2024-04-25T10:30:38.341 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:38.337+0000 7f4cf8a1d740 1 -- 172.21.15.37:0/1454804361 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({fsmap=0}) -- 0x5575bfbe52c0 con 0x5575c3d81b00 2024-04-25T10:30:38.341 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:38.337+0000 7f4cf8a1d740 4 waiting for MDS map... 2024-04-25T10:30:38.341 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:38.337+0000 7f4cf3f09640 1 -- 172.21.15.37:0/1454804361 <== mon.0 v2:172.21.15.37:3300/0 4 ==== osd_map(44..44 src has 1..44) ==== 6248+0+0 (secure 0 0 0) 0x5575c4acc600 con 0x5575c3d81b00 2024-04-25T10:30:38.341 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:38.337+0000 7f4cf3f09640 1 -- 172.21.15.37:0/1454804361 <== mon.0 v2:172.21.15.37:3300/0 5 ==== fsmap(e 22) ==== 4135+0+0 (secure 0 0 0) 0x5575c3c41180 con 0x5575c3d81b00 2024-04-25T10:30:38.341 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:38.337+0000 7f4cf8a1d740 4 Got MDS map 22 2024-04-25T10:30:38.341 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:38.337+0000 7f4cf8a1d740 4 datascan.main: Using metadata pool output 2024-04-25T10:30:38.342 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:38.337+0000 7f4cf8a1d740 4 datascan.main: connecting to RADOS... 2024-04-25T10:30:38.346 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:38.338+0000 7f4cf8a1d740 1 Processor -- start 2024-04-25T10:30:38.346 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:38.338+0000 7f4cf8a1d740 1 -- start start 2024-04-25T10:30:38.346 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:38.339+0000 7f4cf8a1d740 1 --2- >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x5575c3d80d80 0x5575c3d5b080 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:30:38.346 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:38.339+0000 7f4cf8a1d740 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x5575c3d80900 0x5575c3d5b600 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:30:38.346 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:38.339+0000 7f4cf8a1d740 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x5575c3d82880 0x5575c3d5bb80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:30:38.346 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:38.339+0000 7f4cf8a1d740 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 -- 0x5575c3ca6300 con 0x5575c3d82880 2024-04-25T10:30:38.346 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:38.339+0000 7f4cf8a1d740 1 -- --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_getmap magic: 0 -- 0x5575c3ca6000 con 0x5575c3d80900 2024-04-25T10:30:38.346 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:38.339+0000 7f4cf8a1d740 1 -- --> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] -- mon_getmap magic: 0 -- 0x5575c3c48900 con 0x5575c3d80d80 2024-04-25T10:30:38.346 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:38.339+0000 7f4cf5f0d640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x5575c3d82880 0x5575c3d5bb80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:30:38.347 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:38.339+0000 7f4cf5f0d640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x5575c3d82880 0x5575c3d5bb80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.37:3300/0 says I am v2:172.21.15.37:55532/0 (socket says 172.21.15.37:55532) 2024-04-25T10:30:38.347 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:38.339+0000 7f4cf5f0d640 1 -- 172.21.15.37:0/1126798696 learned_addr learned my addr 172.21.15.37:0/1126798696 (peer_addr_for_me v2:172.21.15.37:0/0) 2024-04-25T10:30:38.347 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:38.339+0000 7f4cf6f0f640 1 --2- 172.21.15.37:0/1126798696 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x5575c3d80d80 0x5575c3d5b080 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:30:38.347 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:38.339+0000 7f4cf670e640 1 --2- 172.21.15.37:0/1126798696 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x5575c3d80900 0x5575c3d5b600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:30:38.347 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:38.340+0000 7f4cf5f0d640 1 -- 172.21.15.37:0/1126798696 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x5575c3d80d80 msgr2=0x5575c3d5b080 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:30:38.347 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:38.340+0000 7f4cf5f0d640 1 --2- 172.21.15.37:0/1126798696 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x5575c3d80d80 0x5575c3d5b080 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:30:38.347 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:38.340+0000 7f4cf5f0d640 1 -- 172.21.15.37:0/1126798696 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x5575c3d80900 msgr2=0x5575c3d5b600 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:30:38.347 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:38.340+0000 7f4cf5f0d640 1 --2- 172.21.15.37:0/1126798696 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x5575c3d80900 0x5575c3d5b600 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:30:38.347 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:38.340+0000 7f4cf5f0d640 1 -- 172.21.15.37:0/1126798696 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5575bfbe54a0 con 0x5575c3d82880 2024-04-25T10:30:38.347 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:38.340+0000 7f4cf5f0d640 1 --2- 172.21.15.37:0/1126798696 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x5575c3d82880 0x5575c3d5bb80 secure :-1 s=READY pgs=546 cs=0 l=1 rev1=1 crypto rx=0x5575c4abf0e0 tx=0x5575c3caa000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=eab8d5ec6ee99ec4 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:30:38.347 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:38.340+0000 7f4cf1f05640 1 -- 172.21.15.37:0/1126798696 <== mon.0 v2:172.21.15.37:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x5575bfbdeb60 con 0x5575c3d82880 2024-04-25T10:30:38.347 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:38.340+0000 7f4cf1f05640 1 -- 172.21.15.37:0/1126798696 <== mon.0 v2:172.21.15.37:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x5575c3c30c40 con 0x5575c3d82880 2024-04-25T10:30:38.347 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:38.340+0000 7f4cf1f05640 1 -- 172.21.15.37:0/1126798696 <== mon.0 v2:172.21.15.37:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x5575bfbde820 con 0x5575c3d82880 2024-04-25T10:30:38.348 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:38.341+0000 7f4cf8a1d740 1 -- 172.21.15.37:0/1126798696 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x5575c3d82880 msgr2=0x5575c3d5bb80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:30:38.348 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:38.341+0000 7f4cf8a1d740 1 --2- 172.21.15.37:0/1126798696 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x5575c3d82880 0x5575c3d5bb80 secure :-1 s=READY pgs=546 cs=0 l=1 rev1=1 crypto rx=0x5575c4abf0e0 tx=0x5575c3caa000 comp rx=0 tx=0).stop 2024-04-25T10:30:38.348 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:38.341+0000 7f4cf8a1d740 1 -- 172.21.15.37:0/1126798696 shutdown_connections 2024-04-25T10:30:38.348 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:38.341+0000 7f4cf8a1d740 1 --2- 172.21.15.37:0/1126798696 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x5575c3d82880 0x5575c3d5bb80 unknown :-1 s=CLOSED pgs=546 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:30:38.348 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:38.341+0000 7f4cf8a1d740 1 --2- 172.21.15.37:0/1126798696 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x5575c3d80900 0x5575c3d5b600 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:30:38.348 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:38.341+0000 7f4cf8a1d740 1 --2- 172.21.15.37:0/1126798696 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x5575c3d80d80 0x5575c3d5b080 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:30:38.348 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:38.341+0000 7f4cf8a1d740 1 -- 172.21.15.37:0/1126798696 >> 172.21.15.37:0/1126798696 conn(0x5575c3d81200 msgr2=0x5575c3ccf800 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:30:38.348 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:38.341+0000 7f4cf8a1d740 1 -- 172.21.15.37:0/1126798696 shutdown_connections 2024-04-25T10:30:38.348 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:38.341+0000 7f4cf8a1d740 1 -- 172.21.15.37:0/1126798696 wait complete. 2024-04-25T10:30:38.348 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:38.341+0000 7f4cf8a1d740 1 Processor -- start 2024-04-25T10:30:38.348 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:38.341+0000 7f4cf8a1d740 1 -- start start 2024-04-25T10:30:38.348 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:38.343+0000 7f4cf8a1d740 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x5575c3d80900 0x5575c3d5b600 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:30:38.348 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:38.343+0000 7f4cf8a1d740 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x5575c3d80d80 0x5575c3d5b080 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:30:38.348 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:38.343+0000 7f4cf8a1d740 1 --2- >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x5575c3d82880 0x5575c3d5bb80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:30:38.349 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:38.343+0000 7f4cf8a1d740 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 -- 0x5575c3ca7500 con 0x5575c3d80900 2024-04-25T10:30:38.349 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:38.343+0000 7f4cf8a1d740 1 -- --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_getmap magic: 0 -- 0x5575c3ca7680 con 0x5575c3d80d80 2024-04-25T10:30:38.349 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:38.343+0000 7f4cf8a1d740 1 -- --> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] -- mon_getmap magic: 0 -- 0x5575c3ca6c00 con 0x5575c3d82880 2024-04-25T10:30:38.349 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:38.343+0000 7f4cf6f0f640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x5575c3d80900 0x5575c3d5b600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:30:38.349 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:38.343+0000 7f4cf670e640 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x5575c3d80d80 0x5575c3d5b080 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:30:38.349 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:38.343+0000 7f4cf6f0f640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x5575c3d80900 0x5575c3d5b600 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.37:3300/0 says I am v2:172.21.15.37:55548/0 (socket says 172.21.15.37:55548) 2024-04-25T10:30:38.349 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:38.343+0000 7f4cf5f0d640 1 --2- >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x5575c3d82880 0x5575c3d5bb80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:30:38.349 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:38.343+0000 7f4cf6f0f640 1 -- 172.21.15.37:0/1994611522 learned_addr learned my addr 172.21.15.37:0/1994611522 (peer_addr_for_me v2:172.21.15.37:0/0) 2024-04-25T10:30:38.349 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:38.343+0000 7f4cf6f0f640 1 -- 172.21.15.37:0/1994611522 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x5575c3d82880 msgr2=0x5575c3d5bb80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:30:38.349 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:38.343+0000 7f4cf6f0f640 1 --2- 172.21.15.37:0/1994611522 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x5575c3d82880 0x5575c3d5bb80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:30:38.349 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:38.343+0000 7f4cf6f0f640 1 -- 172.21.15.37:0/1994611522 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x5575c3d80d80 msgr2=0x5575c3d5b080 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:30:38.349 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:38.343+0000 7f4cf6f0f640 1 --2- 172.21.15.37:0/1994611522 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x5575c3d80d80 0x5575c3d5b080 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:30:38.349 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:38.343+0000 7f4cf6f0f640 1 -- 172.21.15.37:0/1994611522 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5575bfbe50e0 con 0x5575c3d80900 2024-04-25T10:30:38.349 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:38.343+0000 7f4cf5f0d640 1 --2- 172.21.15.37:0/1994611522 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x5575c3d82880 0x5575c3d5bb80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-25T10:30:38.349 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:38.343+0000 7f4cf6f0f640 1 --2- 172.21.15.37:0/1994611522 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x5575c3d80900 0x5575c3d5b600 secure :-1 s=READY pgs=547 cs=0 l=1 rev1=1 crypto rx=0x5575c4abff80 tx=0x5575c4ad8c00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:30:38.350 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:38.343+0000 7f4cf0702640 1 -- 172.21.15.37:0/1994611522 <== mon.0 v2:172.21.15.37:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x5575bfbdeb60 con 0x5575c3d80900 2024-04-25T10:30:38.350 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:38.343+0000 7f4cf0702640 1 -- 172.21.15.37:0/1994611522 <== mon.0 v2:172.21.15.37:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x5575c3c30a80 con 0x5575c3d80900 2024-04-25T10:30:38.350 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:38.343+0000 7f4cf8a1d740 1 -- 172.21.15.37:0/1994611522 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x5575bfbe5a40 con 0x5575c3d80900 2024-04-25T10:30:38.350 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:38.343+0000 7f4cf0702640 1 -- 172.21.15.37:0/1994611522 <== mon.0 v2:172.21.15.37:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x5575bfbde820 con 0x5575c3d80900 2024-04-25T10:30:38.350 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:38.343+0000 7f4cf8a1d740 1 -- 172.21.15.37:0/1994611522 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5575bfbe43c0 con 0x5575c3d80900 2024-04-25T10:30:38.350 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:38.344+0000 7f4cf8a1d740 4 datascan.init: forcing metadata pool 'recovery_fs_metadata' 2024-04-25T10:30:38.352 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:38.345+0000 7f4cf0702640 1 -- 172.21.15.37:0/1994611522 <== mon.0 v2:172.21.15.37:3300/0 4 ==== mgrmap(e 5) ==== 96782+0+0 (secure 0 0 0) 0x5575c4b12900 con 0x5575c3d80900 2024-04-25T10:30:38.352 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:38.345+0000 7f4cf0702640 1 --2- 172.21.15.37:0/1994611522 >> [v2:172.21.15.203:6832/2418205237,v1:172.21.15.203:6833/2418205237] conn(0x5575c3d83a80 0x5575c3d5c100 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:30:38.352 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:38.345+0000 7f4cf0702640 1 -- 172.21.15.37:0/1994611522 <== mon.0 v2:172.21.15.37:3300/0 5 ==== osd_map(44..44 src has 1..44) ==== 6248+0+0 (secure 0 0 0) 0x5575c4b2b000 con 0x5575c3d80900 2024-04-25T10:30:38.352 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:38.345+0000 7f4cf8a1d740 1 --2- 172.21.15.37:0/1994611522 >> [v2:172.21.15.203:6800/4051532063,v1:172.21.15.203:6801/4051532063] conn(0x5575c3d82d00 0x5575c3d5c680 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:30:38.352 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:38.345+0000 7f4cf8a1d740 1 -- 172.21.15.37:0/1994611522 --> [v2:172.21.15.203:6800/4051532063,v1:172.21.15.203:6801/4051532063] -- osd_op(unknown.0.0:1 6.14 6:287034c4:::1.00000000.inode:head [stat] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e44) -- 0x5575c4b3ec00 con 0x5575c3d82d00 2024-04-25T10:30:38.352 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:38.346+0000 7f4cf670e640 1 --2- 172.21.15.37:0/1994611522 >> [v2:172.21.15.203:6832/2418205237,v1:172.21.15.203:6833/2418205237] conn(0x5575c3d83a80 0x5575c3d5c100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:30:38.352 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:38.346+0000 7f4cf5f0d640 1 --2- 172.21.15.37:0/1994611522 >> [v2:172.21.15.203:6800/4051532063,v1:172.21.15.203:6801/4051532063] conn(0x5575c3d82d00 0x5575c3d5c680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:30:38.352 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:38.346+0000 7f4cf670e640 1 --2- 172.21.15.37:0/1994611522 >> [v2:172.21.15.203:6832/2418205237,v1:172.21.15.203:6833/2418205237] conn(0x5575c3d83a80 0x5575c3d5c100 secure :-1 s=READY pgs=261 cs=0 l=1 rev1=1 crypto rx=0x5575c3ceb1d0 tx=0x5575c4ad8f00 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:30:38.353 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:38.346+0000 7f4cf5f0d640 1 --2- 172.21.15.37:0/1994611522 >> [v2:172.21.15.203:6800/4051532063,v1:172.21.15.203:6801/4051532063] conn(0x5575c3d82d00 0x5575c3d5c680 crc :-1 s=READY pgs=12 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.4 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:30:38.353 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:38.347+0000 7f4cf5f0d640 1 -- 172.21.15.37:0/1994611522 <== osd.4 v2:172.21.15.203:6800/4051532063 1 ==== osd_op_reply(1 1.00000000.inode [stat] v0'0 uv0 ondisk = -2 ((2) No such file or directory)) ==== 160+0+0 (crc 0 0 0) 0x5575c3cd0000 con 0x5575c3d82d00 2024-04-25T10:30:38.353 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:38.347+0000 7f4cf8a1d740 1 -- 172.21.15.37:0/1994611522 --> [v2:172.21.15.203:6800/4051532063,v1:172.21.15.203:6801/4051532063] -- osd_op(unknown.0.0:2 6.14 6:287034c4:::1.00000000.inode:head [writefull 0~582 in=582b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e44) -- 0x5575c4b3f000 con 0x5575c3d82d00 2024-04-25T10:30:38.353 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:38.348+0000 7f4cf5f0d640 1 -- 172.21.15.37:0/1994611522 <== osd.4 v2:172.21.15.203:6800/4051532063 2 ==== osd_op_reply(2 1.00000000.inode [writefull 0~582] v44'1 uv1 ondisk = 0) ==== 160+0+0 (crc 0 0 0) 0x5575c3cd0000 con 0x5575c3d82d00 2024-04-25T10:30:38.353 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:38.348+0000 7f4cf8a1d740 1 --2- 172.21.15.37:0/1994611522 >> [v2:172.21.15.203:6810/2309615014,v1:172.21.15.203:6812/2309615014] conn(0x5575c3d83180 0x5575c3d5cc00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:30:38.353 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:38.348+0000 7f4cf8a1d740 1 -- 172.21.15.37:0/1994611522 --> [v2:172.21.15.203:6810/2309615014,v1:172.21.15.203:6812/2309615014] -- osd_op(unknown.0.0:3 6.1f 6:fe07bba1:::100.00000000.inode:head [stat] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e44) -- 0x5575c4ac9400 con 0x5575c3d83180 2024-04-25T10:30:38.353 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:38.349+0000 7f4cf6f0f640 1 --2- 172.21.15.37:0/1994611522 >> [v2:172.21.15.203:6810/2309615014,v1:172.21.15.203:6812/2309615014] conn(0x5575c3d83180 0x5575c3d5cc00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:30:38.353 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:38.349+0000 7f4cf6f0f640 1 --2- 172.21.15.37:0/1994611522 >> [v2:172.21.15.203:6810/2309615014,v1:172.21.15.203:6812/2309615014] conn(0x5575c3d83180 0x5575c3d5cc00 crc :-1 s=READY pgs=12 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.6 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:30:38.353 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:38.349+0000 7f4cf6f0f640 1 -- 172.21.15.37:0/1994611522 <== osd.6 v2:172.21.15.203:6810/2309615014 1 ==== osd_op_reply(3 100.00000000.inode [stat] v0'0 uv0 ondisk = -2 ((2) No such file or directory)) ==== 162+0+0 (crc 0 0 0) 0x5575c3cd0500 con 0x5575c3d83180 2024-04-25T10:30:38.353 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:38.349+0000 7f4cf8a1d740 1 -- 172.21.15.37:0/1994611522 --> [v2:172.21.15.203:6810/2309615014,v1:172.21.15.203:6812/2309615014] -- osd_op(unknown.0.0:4 6.1f 6:fe07bba1:::100.00000000.inode:head [writefull 0~582 in=582b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e44) -- 0x5575c4ac9000 con 0x5575c3d83180 2024-04-25T10:30:38.354 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:38.350+0000 7f4cf6f0f640 1 -- 172.21.15.37:0/1994611522 <== osd.6 v2:172.21.15.203:6810/2309615014 2 ==== osd_op_reply(4 100.00000000.inode [writefull 0~582] v44'1 uv1 ondisk = 0) ==== 162+0+0 (crc 0 0 0) 0x5575c3cd0500 con 0x5575c3d83180 2024-04-25T10:30:38.354 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:38.351+0000 7f4cf8a1d740 1 --2- 172.21.15.37:0/1994611522 >> [v2:172.21.15.203:6808/3688099014,v1:172.21.15.203:6809/3688099014] conn(0x5575c3d83600 0x5575c3d5d180 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:30:38.354 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:38.351+0000 7f4cf8a1d740 1 -- 172.21.15.37:0/1994611522 --> [v2:172.21.15.203:6808/3688099014,v1:172.21.15.203:6809/3688099014] -- osd_op(unknown.0.0:5 6.13 6:cd5a64a3:::100.00000000:head [omap-get-header] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e44) -- 0x5575c4ac8c00 con 0x5575c3d83600 2024-04-25T10:30:38.354 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:38.351+0000 7f4cf670e640 1 --2- 172.21.15.37:0/1994611522 >> [v2:172.21.15.203:6808/3688099014,v1:172.21.15.203:6809/3688099014] conn(0x5575c3d83600 0x5575c3d5d180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:30:38.355 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:38.351+0000 7f4cf670e640 1 --2- 172.21.15.37:0/1994611522 >> [v2:172.21.15.203:6808/3688099014,v1:172.21.15.203:6809/3688099014] conn(0x5575c3d83600 0x5575c3d5d180 crc :-1 s=READY pgs=13 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.7 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:30:38.355 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:38.351+0000 7f4cf670e640 1 -- 172.21.15.37:0/1994611522 <== osd.7 v2:172.21.15.203:6808/3688099014 1 ==== osd_op_reply(5 100.00000000 [omap-get-header] v0'0 uv0 ondisk = -2 ((2) No such file or directory)) ==== 156+0+0 (crc 0 0 0) 0x5575c3cd0780 con 0x5575c3d83600 2024-04-25T10:30:38.355 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:38.351+0000 7f4cf8a1d740 10 datascan.find_or_create_dirfrag: read_version = 0 2024-04-25T10:30:38.355 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:38.351+0000 7f4cf8a1d740 1 -- 172.21.15.37:0/1994611522 --> [v2:172.21.15.203:6808/3688099014,v1:172.21.15.203:6809/3688099014] -- osd_op(unknown.0.0:6 6.13 6:cd5a64a3:::100.00000000:head [create,omap-set-header in=274b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e44) -- 0x5575c4b3ec00 con 0x5575c3d83600 2024-04-25T10:30:38.359 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:38.355+0000 7f4cf670e640 1 -- 172.21.15.37:0/1994611522 <== osd.7 v2:172.21.15.203:6808/3688099014 2 ==== osd_op_reply(6 100.00000000 [create,omap-set-header] v44'1 uv1 ondisk = 0) ==== 198+0+0 (crc 0 0 0) 0x5575c3cd0780 con 0x5575c3d83600 2024-04-25T10:30:38.359 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:38.355+0000 7f4cf8a1d740 10 datascan.find_or_create_dirfrag: Created dirfrag: 0x0x100 2024-04-25T10:30:38.359 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:38.355+0000 7f4cf8a1d740 1 -- 172.21.15.37:0/1994611522 >> [v2:172.21.15.203:6800/4051532063,v1:172.21.15.203:6801/4051532063] conn(0x5575c3d82d00 msgr2=0x5575c3d5c680 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:30:38.359 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:38.355+0000 7f4cf8a1d740 1 --2- 172.21.15.37:0/1994611522 >> [v2:172.21.15.203:6800/4051532063,v1:172.21.15.203:6801/4051532063] conn(0x5575c3d82d00 0x5575c3d5c680 crc :-1 s=READY pgs=12 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:30:38.359 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:38.356+0000 7f4cf8a1d740 1 -- 172.21.15.37:0/1994611522 >> [v2:172.21.15.203:6810/2309615014,v1:172.21.15.203:6812/2309615014] conn(0x5575c3d83180 msgr2=0x5575c3d5cc00 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:30:38.359 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:38.356+0000 7f4cf8a1d740 1 --2- 172.21.15.37:0/1994611522 >> [v2:172.21.15.203:6810/2309615014,v1:172.21.15.203:6812/2309615014] conn(0x5575c3d83180 0x5575c3d5cc00 crc :-1 s=READY pgs=12 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:30:38.359 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:38.356+0000 7f4cf8a1d740 1 -- 172.21.15.37:0/1994611522 >> [v2:172.21.15.203:6808/3688099014,v1:172.21.15.203:6809/3688099014] conn(0x5575c3d83600 msgr2=0x5575c3d5d180 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:30:38.359 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:38.356+0000 7f4cf8a1d740 1 --2- 172.21.15.37:0/1994611522 >> [v2:172.21.15.203:6808/3688099014,v1:172.21.15.203:6809/3688099014] conn(0x5575c3d83600 0x5575c3d5d180 crc :-1 s=READY pgs=13 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:30:38.360 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:38.356+0000 7f4cf670e640 1 -- 172.21.15.37:0/1994611522 reap_dead start 2024-04-25T10:30:38.360 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:38.356+0000 7f4cf8a1d740 1 -- 172.21.15.37:0/1994611522 >> [v2:172.21.15.203:6832/2418205237,v1:172.21.15.203:6833/2418205237] conn(0x5575c3d83a80 msgr2=0x5575c3d5c100 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:30:38.360 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:38.356+0000 7f4cf8a1d740 1 --2- 172.21.15.37:0/1994611522 >> [v2:172.21.15.203:6832/2418205237,v1:172.21.15.203:6833/2418205237] conn(0x5575c3d83a80 0x5575c3d5c100 secure :-1 s=READY pgs=261 cs=0 l=1 rev1=1 crypto rx=0x5575c3ceb1d0 tx=0x5575c4ad8f00 comp rx=0 tx=0).stop 2024-04-25T10:30:38.360 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:38.356+0000 7f4cf8a1d740 1 -- 172.21.15.37:0/1994611522 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x5575c3d80900 msgr2=0x5575c3d5b600 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:30:38.360 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:38.356+0000 7f4cf8a1d740 1 --2- 172.21.15.37:0/1994611522 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x5575c3d80900 0x5575c3d5b600 secure :-1 s=READY pgs=547 cs=0 l=1 rev1=1 crypto rx=0x5575c4abff80 tx=0x5575c4ad8c00 comp rx=0 tx=0).stop 2024-04-25T10:30:38.360 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:38.356+0000 7f4cf8a1d740 1 -- 172.21.15.37:0/1994611522 shutdown_connections 2024-04-25T10:30:38.360 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:38.356+0000 7f4cf8a1d740 1 --2- 172.21.15.37:0/1994611522 >> [v2:172.21.15.203:6832/2418205237,v1:172.21.15.203:6833/2418205237] conn(0x5575c3d83a80 0x5575c3d5c100 unknown :-1 s=CLOSED pgs=261 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:30:38.360 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:38.356+0000 7f4cf8a1d740 1 --2- 172.21.15.37:0/1994611522 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x5575c3d80900 0x5575c3d5b600 unknown :-1 s=CLOSED pgs=547 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:30:38.360 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:38.356+0000 7f4cf8a1d740 1 -- 172.21.15.37:0/1994611522 >> 172.21.15.37:0/1994611522 conn(0x5575c3d81200 msgr2=0x5575c3ccf800 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:30:38.360 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:38.356+0000 7f4cf8a1d740 1 -- 172.21.15.37:0/1994611522 shutdown_connections 2024-04-25T10:30:38.360 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:38.356+0000 7f4cf8a1d740 1 -- 172.21.15.37:0/1994611522 wait complete. 2024-04-25T10:30:38.360 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:38.356+0000 7f4cf8a1d740 1 -- 172.21.15.37:0/1454804361 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x5575c3d81b00 msgr2=0x5575c3d5a000 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:30:38.360 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:38.356+0000 7f4cf8a1d740 1 --2- 172.21.15.37:0/1454804361 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x5575c3d81b00 0x5575c3d5a000 secure :-1 s=READY pgs=545 cs=0 l=1 rev1=1 crypto rx=0x5575c4a11200 tx=0x5575c3ce8a20 comp rx=0 tx=0).stop 2024-04-25T10:30:38.360 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:38.356+0000 7f4cf8a1d740 1 -- 172.21.15.37:0/1454804361 shutdown_connections 2024-04-25T10:30:38.360 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:38.356+0000 7f4cf8a1d740 1 --2- 172.21.15.37:0/1454804361 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x5575c3d81f80 0x5575c3d5ab00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:30:38.361 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:38.356+0000 7f4cf8a1d740 1 --2- 172.21.15.37:0/1454804361 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x5575c3d81b00 0x5575c3d5a000 unknown :-1 s=CLOSED pgs=545 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:30:38.361 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:38.356+0000 7f4cf8a1d740 1 --2- 172.21.15.37:0/1454804361 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x5575c3d81680 0x5575c3d5a580 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:30:38.361 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:38.356+0000 7f4cf8a1d740 1 -- 172.21.15.37:0/1454804361 >> 172.21.15.37:0/1454804361 conn(0x5575c3d82400 msgr2=0x5575c3ccf000 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:30:38.361 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:38.356+0000 7f4cf8a1d740 1 -- 172.21.15.37:0/1454804361 shutdown_connections 2024-04-25T10:30:38.361 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:38.356+0000 7f4cf8a1d740 1 -- 172.21.15.37:0/1454804361 wait complete. 2024-04-25T10:30:38.362 DEBUG:tasks.cephfs.filesystem:Ran ['cephfs-data-scan', '--debug-mds=20', '--debug-ms=1', '--debug-objecter=1', 'init', '--force-init', '--filesystem', 'recovery_fs', '--alternate-pool', 'recovery_fs_metadata'] in time 0:00:00.119624, result: 2024-04-25T10:30:38.362 DEBUG:teuthology.orchestra.run.smithi037:> cephfs-table-tool --debug-mds=20 --debug-ms=1 --debug-objecter=1 recovery_fs:0 reset session 2024-04-25T10:30:38.413 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:38.411+0000 7f4265f1f280 1 -- start start 2024-04-25T10:30:38.413 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:38.411+0000 7f4265f1f280 1 Processor -- start 2024-04-25T10:30:38.416 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:38.412+0000 7f4265f1f280 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x5603ad9bfa80 0x5603ad8e0000 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:30:38.416 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:38.412+0000 7f4265f1f280 1 --2- >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x5603ad9bf180 0x5603ad8e0580 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:30:38.416 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:38.412+0000 7f4265f1f280 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x5603ae602000 0x5603ad8e0b00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:30:38.416 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:38.412+0000 7f4265f1f280 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 -- 0x5603ad8e4780 con 0x5603ad9bfa80 2024-04-25T10:30:38.417 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:38.412+0000 7f4265f1f280 1 -- --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_getmap magic: 0 -- 0x5603ad8e4600 con 0x5603ae602000 2024-04-25T10:30:38.417 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:38.412+0000 7f4265f1f280 1 -- --> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] -- mon_getmap magic: 0 -- 0x5603ad8e4480 con 0x5603ad9bf180 2024-04-25T10:30:38.417 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:38.412+0000 7f4263db2640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x5603ad9bfa80 0x5603ad8e0000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:30:38.417 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:38.412+0000 7f4263db2640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x5603ad9bfa80 0x5603ad8e0000 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.37:3300/0 says I am v2:172.21.15.37:55556/0 (socket says 172.21.15.37:55556) 2024-04-25T10:30:38.417 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:38.412+0000 7f4263db2640 1 -- 172.21.15.37:0/3123934490 learned_addr learned my addr 172.21.15.37:0/3123934490 (peer_addr_for_me v2:172.21.15.37:0/0) 2024-04-25T10:30:38.417 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:38.412+0000 7f42635b1640 1 --2- 172.21.15.37:0/3123934490 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x5603ad9bf180 0x5603ad8e0580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:30:38.417 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:38.412+0000 7f42645b3640 1 --2- 172.21.15.37:0/3123934490 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x5603ae602000 0x5603ad8e0b00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:30:38.417 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:38.413+0000 7f4263db2640 1 -- 172.21.15.37:0/3123934490 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x5603ad9bf180 msgr2=0x5603ad8e0580 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:30:38.417 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:38.413+0000 7f4263db2640 1 --2- 172.21.15.37:0/3123934490 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x5603ad9bf180 0x5603ad8e0580 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:30:38.417 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:38.413+0000 7f4263db2640 1 -- 172.21.15.37:0/3123934490 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x5603ae602000 msgr2=0x5603ad8e0b00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:30:38.417 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:38.413+0000 7f4263db2640 1 --2- 172.21.15.37:0/3123934490 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x5603ae602000 0x5603ad8e0b00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:30:38.417 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:38.413+0000 7f4263db2640 1 -- 172.21.15.37:0/3123934490 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5603a982e780 con 0x5603ad9bfa80 2024-04-25T10:30:38.417 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:38.413+0000 7f4263db2640 1 --2- 172.21.15.37:0/3123934490 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x5603ad9bfa80 0x5603ad8e0000 secure :-1 s=READY pgs=549 cs=0 l=1 rev1=1 crypto rx=0x5603ae65f350 tx=0x5603ad924a20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b3cc2f6cf54d4220 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:30:38.417 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:38.413+0000 7f42615ad640 1 -- 172.21.15.37:0/3123934490 <== mon.0 v2:172.21.15.37:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x5603a981e9c0 con 0x5603ad9bfa80 2024-04-25T10:30:38.418 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:38.413+0000 7f42615ad640 1 -- 172.21.15.37:0/3123934490 <== mon.0 v2:172.21.15.37:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x5603a9824700 con 0x5603ad9bfa80 2024-04-25T10:30:38.418 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:38.413+0000 7f42615ad640 1 -- 172.21.15.37:0/3123934490 <== mon.0 v2:172.21.15.37:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x5603a981e680 con 0x5603ad9bfa80 2024-04-25T10:30:38.418 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:38.413+0000 7f4265f1f280 1 -- 172.21.15.37:0/3123934490 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5603a982e3c0 con 0x5603ad9bfa80 2024-04-25T10:30:38.418 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:38.413+0000 7f4265f1f280 1 -- 172.21.15.37:0/3123934490 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({fsmap=0}) -- 0x5603a982f680 con 0x5603ad9bfa80 2024-04-25T10:30:38.418 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:38.413+0000 7f4265f1f280 4 waiting for MDS map... 2024-04-25T10:30:38.418 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:38.413+0000 7f42615ad640 1 -- 172.21.15.37:0/3123934490 <== mon.0 v2:172.21.15.37:3300/0 4 ==== osd_map(44..44 src has 1..44) ==== 6248+0+0 (secure 0 0 0) 0x5603ae704200 con 0x5603ad9bfa80 2024-04-25T10:30:38.418 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:38.413+0000 7f42615ad640 1 -- 172.21.15.37:0/3123934490 <== mon.0 v2:172.21.15.37:3300/0 5 ==== fsmap(e 22) ==== 4135+0+0 (secure 0 0 0) 0x5603ad883180 con 0x5603ad9bfa80 2024-04-25T10:30:38.418 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:38.413+0000 7f4265f1f280 4 Got MDS map 22 2024-04-25T10:30:38.418 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:38.413+0000 7f4265f1f280 10 main: main 2024-04-25T10:30:38.418 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:38.414+0000 7f4265f1f280 4 main: connecting to RADOS... 2024-04-25T10:30:38.418 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:38.414+0000 7f4265f1f280 1 Processor -- start 2024-04-25T10:30:38.418 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:38.414+0000 7f4265f1f280 1 -- start start 2024-04-25T10:30:38.423 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:38.415+0000 7f4265f1f280 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x5603ae602900 0x5603ad8e1080 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:30:38.423 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:38.415+0000 7f4265f1f280 1 --2- >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x5603ae602d80 0x5603ad8e1600 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:30:38.423 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:38.416+0000 7f4265f1f280 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x5603ae603200 0x5603ad8e1b80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:30:38.423 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:38.416+0000 7f4265f1f280 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 -- 0x5603ad8e4300 con 0x5603ae602900 2024-04-25T10:30:38.423 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:38.416+0000 7f4265f1f280 1 -- --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_getmap magic: 0 -- 0x5603ad8e4000 con 0x5603ae603200 2024-04-25T10:30:38.423 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:38.416+0000 7f4265f1f280 1 -- --> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] -- mon_getmap magic: 0 -- 0x5603ad88a900 con 0x5603ae602d80 2024-04-25T10:30:38.423 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:38.416+0000 7f42645b3640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x5603ae602900 0x5603ad8e1080 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:30:38.423 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:38.416+0000 7f42645b3640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x5603ae602900 0x5603ad8e1080 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.37:3300/0 says I am v2:172.21.15.37:55558/0 (socket says 172.21.15.37:55558) 2024-04-25T10:30:38.423 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:38.416+0000 7f42645b3640 1 -- 172.21.15.37:0/1166920179 learned_addr learned my addr 172.21.15.37:0/1166920179 (peer_addr_for_me v2:172.21.15.37:0/0) 2024-04-25T10:30:38.424 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:38.416+0000 7f4263db2640 1 --2- 172.21.15.37:0/1166920179 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x5603ae602d80 0x5603ad8e1600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:30:38.424 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:38.416+0000 7f42635b1640 1 --2- 172.21.15.37:0/1166920179 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x5603ae603200 0x5603ad8e1b80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:30:38.424 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:38.416+0000 7f42645b3640 1 -- 172.21.15.37:0/1166920179 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x5603ae602d80 msgr2=0x5603ad8e1600 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:30:38.424 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:38.416+0000 7f42645b3640 1 --2- 172.21.15.37:0/1166920179 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x5603ae602d80 0x5603ad8e1600 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:30:38.424 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:38.416+0000 7f42645b3640 1 -- 172.21.15.37:0/1166920179 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x5603ae603200 msgr2=0x5603ad8e1b80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:30:38.424 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:38.416+0000 7f42645b3640 1 --2- 172.21.15.37:0/1166920179 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x5603ae603200 0x5603ad8e1b80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:30:38.424 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:38.416+0000 7f42645b3640 1 -- 172.21.15.37:0/1166920179 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5603a982f4a0 con 0x5603ae602900 2024-04-25T10:30:38.424 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:38.416+0000 7f42645b3640 1 --2- 172.21.15.37:0/1166920179 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x5603ae602900 0x5603ad8e1080 secure :-1 s=READY pgs=550 cs=0 l=1 rev1=1 crypto rx=0x5603ae63a870 tx=0x5603ae620c00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f34f5f2ff2aadf96 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:30:38.424 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:38.416+0000 7f425f5a9640 1 -- 172.21.15.37:0/1166920179 <== mon.0 v2:172.21.15.37:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x5603a981e9c0 con 0x5603ae602900 2024-04-25T10:30:38.424 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:38.416+0000 7f425f5a9640 1 -- 172.21.15.37:0/1166920179 <== mon.0 v2:172.21.15.37:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x5603a9824e00 con 0x5603ae602900 2024-04-25T10:30:38.424 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:38.416+0000 7f425f5a9640 1 -- 172.21.15.37:0/1166920179 <== mon.0 v2:172.21.15.37:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x5603a981e680 con 0x5603ae602900 2024-04-25T10:30:38.424 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:38.417+0000 7f4265f1f280 1 -- 172.21.15.37:0/1166920179 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x5603ae602900 msgr2=0x5603ad8e1080 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:30:38.424 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:38.417+0000 7f4265f1f280 1 --2- 172.21.15.37:0/1166920179 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x5603ae602900 0x5603ad8e1080 secure :-1 s=READY pgs=550 cs=0 l=1 rev1=1 crypto rx=0x5603ae63a870 tx=0x5603ae620c00 comp rx=0 tx=0).stop 2024-04-25T10:30:38.425 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:38.417+0000 7f4265f1f280 1 -- 172.21.15.37:0/1166920179 shutdown_connections 2024-04-25T10:30:38.425 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:38.417+0000 7f4265f1f280 1 --2- 172.21.15.37:0/1166920179 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x5603ae603200 0x5603ad8e1b80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:30:38.425 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:38.417+0000 7f4265f1f280 1 --2- 172.21.15.37:0/1166920179 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x5603ae602d80 0x5603ad8e1600 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:30:38.425 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:38.417+0000 7f4265f1f280 1 --2- 172.21.15.37:0/1166920179 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x5603ae602900 0x5603ad8e1080 unknown :-1 s=CLOSED pgs=550 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:30:38.425 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:38.417+0000 7f4265f1f280 1 -- 172.21.15.37:0/1166920179 >> 172.21.15.37:0/1166920179 conn(0x5603ae602480 msgr2=0x5603ad955800 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:30:38.425 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:38.417+0000 7f4265f1f280 1 -- 172.21.15.37:0/1166920179 shutdown_connections 2024-04-25T10:30:38.425 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:38.417+0000 7f4265f1f280 1 -- 172.21.15.37:0/1166920179 wait complete. 2024-04-25T10:30:38.425 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:38.418+0000 7f4265f1f280 1 Processor -- start 2024-04-25T10:30:38.425 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:38.418+0000 7f4265f1f280 1 -- start start 2024-04-25T10:30:38.425 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:38.419+0000 7f4265f1f280 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x5603ae602900 0x5603ad8e1080 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:30:38.425 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:38.419+0000 7f4265f1f280 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x5603ae602d80 0x5603ad8e1600 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:30:38.425 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:38.419+0000 7f4265f1f280 1 --2- >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x5603ae603200 0x5603ad8e1b80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:30:38.425 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:38.419+0000 7f4265f1f280 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 -- 0x5603ad8e4000 con 0x5603ae602d80 2024-04-25T10:30:38.425 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:38.419+0000 7f4265f1f280 1 -- --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_getmap magic: 0 -- 0x5603ad8e5500 con 0x5603ae602900 2024-04-25T10:30:38.425 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:38.419+0000 7f4265f1f280 1 -- --> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] -- mon_getmap magic: 0 -- 0x5603ad8e4300 con 0x5603ae603200 2024-04-25T10:30:38.426 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:38.419+0000 7f4263db2640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x5603ae602d80 0x5603ad8e1600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:30:38.426 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:38.419+0000 7f4263db2640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x5603ae602d80 0x5603ad8e1600 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.37:3300/0 says I am v2:172.21.15.37:55570/0 (socket says 172.21.15.37:55570) 2024-04-25T10:30:38.426 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:38.419+0000 7f4263db2640 1 -- 172.21.15.37:0/1159520047 learned_addr learned my addr 172.21.15.37:0/1159520047 (peer_addr_for_me v2:172.21.15.37:0/0) 2024-04-25T10:30:38.426 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:38.419+0000 7f42635b1640 1 --2- 172.21.15.37:0/1159520047 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x5603ae603200 0x5603ad8e1b80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:30:38.426 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:38.419+0000 7f42645b3640 1 --2- 172.21.15.37:0/1159520047 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x5603ae602900 0x5603ad8e1080 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:30:38.426 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:38.419+0000 7f4263db2640 1 -- 172.21.15.37:0/1159520047 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x5603ae603200 msgr2=0x5603ad8e1b80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:30:38.426 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:38.419+0000 7f4263db2640 1 --2- 172.21.15.37:0/1159520047 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x5603ae603200 0x5603ad8e1b80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:30:38.426 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:38.419+0000 7f4263db2640 1 -- 172.21.15.37:0/1159520047 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x5603ae602900 msgr2=0x5603ad8e1080 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:30:38.426 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:38.419+0000 7f4263db2640 1 --2- 172.21.15.37:0/1159520047 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x5603ae602900 0x5603ad8e1080 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:30:38.426 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:38.419+0000 7f4263db2640 1 -- 172.21.15.37:0/1159520047 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5603a982e000 con 0x5603ae602d80 2024-04-25T10:30:38.427 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:38.420+0000 7f4263db2640 1 --2- 172.21.15.37:0/1159520047 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x5603ae602d80 0x5603ad8e1600 secure :-1 s=READY pgs=551 cs=0 l=1 rev1=1 crypto rx=0x5603ae65fa10 tx=0x5603ad925f80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:30:38.427 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:38.420+0000 7f425dda6640 1 -- 172.21.15.37:0/1159520047 <== mon.0 v2:172.21.15.37:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x5603a981eb60 con 0x5603ae602d80 2024-04-25T10:30:38.427 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:38.420+0000 7f425dda6640 1 -- 172.21.15.37:0/1159520047 <== mon.0 v2:172.21.15.37:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x5603a9824fc0 con 0x5603ae602d80 2024-04-25T10:30:38.427 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:38.420+0000 7f4265f1f280 1 -- 172.21.15.37:0/1159520047 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x5603a982e960 con 0x5603ae602d80 2024-04-25T10:30:38.427 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:38.420+0000 7f425dda6640 1 -- 172.21.15.37:0/1159520047 <== mon.0 v2:172.21.15.37:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x5603a981e820 con 0x5603ae602d80 2024-04-25T10:30:38.427 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:38.420+0000 7f4265f1f280 1 -- 172.21.15.37:0/1159520047 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5603a982f860 con 0x5603ae602d80 2024-04-25T10:30:38.427 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:38.420+0000 7f4265f1f280 4 main: resolving pool 6 2024-04-25T10:30:38.429 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:38.421+0000 7f425dda6640 1 -- 172.21.15.37:0/1159520047 <== mon.0 v2:172.21.15.37:3300/0 4 ==== mgrmap(e 5) ==== 96782+0+0 (secure 0 0 0) 0x5603ae74c900 con 0x5603ae602d80 2024-04-25T10:30:38.429 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:38.422+0000 7f425dda6640 1 --2- 172.21.15.37:0/1159520047 >> [v2:172.21.15.203:6832/2418205237,v1:172.21.15.203:6833/2418205237] conn(0x5603ae605600 0x5603ad8e2100 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:30:38.429 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:38.422+0000 7f425dda6640 1 -- 172.21.15.37:0/1159520047 <== mon.0 v2:172.21.15.37:3300/0 5 ==== osd_map(44..44 src has 1..44) ==== 6248+0+0 (secure 0 0 0) 0x5603ae75b200 con 0x5603ae602d80 2024-04-25T10:30:38.429 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:38.422+0000 7f4265f1f280 4 main: creating IoCtx.. 2024-04-25T10:30:38.429 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:38.422+0000 7f42645b3640 1 --2- 172.21.15.37:0/1159520047 >> [v2:172.21.15.203:6832/2418205237,v1:172.21.15.203:6833/2418205237] conn(0x5603ae605600 0x5603ad8e2100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:30:38.429 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:38.422+0000 7f4265f1f280 1 --2- 172.21.15.37:0/1159520047 >> [v2:172.21.15.203:6800/4051532063,v1:172.21.15.203:6801/4051532063] conn(0x5603ae603680 0x5603ad8e2680 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:30:38.429 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:38.422+0000 7f4265f1f280 1 -- 172.21.15.37:0/1159520047 --> [v2:172.21.15.203:6800/4051532063,v1:172.21.15.203:6801/4051532063] -- osd_op(unknown.0.0:1 6.b 6:d0630e4c:::mds0_sessionmap:head [omap-clear,omap-set-header in=14b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e44) -- 0x5603ae76bc00 con 0x5603ae603680 2024-04-25T10:30:38.429 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:38.422+0000 7f42635b1640 1 --2- 172.21.15.37:0/1159520047 >> [v2:172.21.15.203:6800/4051532063,v1:172.21.15.203:6801/4051532063] conn(0x5603ae603680 0x5603ad8e2680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:30:38.429 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:38.422+0000 7f42645b3640 1 --2- 172.21.15.37:0/1159520047 >> [v2:172.21.15.203:6832/2418205237,v1:172.21.15.203:6833/2418205237] conn(0x5603ae605600 0x5603ad8e2100 secure :-1 s=READY pgs=262 cs=0 l=1 rev1=1 crypto rx=0x5603ae63a870 tx=0x5603ad924a80 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:30:38.429 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:38.423+0000 7f42635b1640 1 --2- 172.21.15.37:0/1159520047 >> [v2:172.21.15.203:6800/4051532063,v1:172.21.15.203:6801/4051532063] conn(0x5603ae603680 0x5603ad8e2680 crc :-1 s=READY pgs=13 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.4 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:30:38.429 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:38.424+0000 7f42635b1640 1 -- 172.21.15.37:0/1159520047 <== osd.4 v2:172.21.15.203:6800/4051532063 1 ==== osd_op_reply(1 mds0_sessionmap [omap-clear,omap-set-header] v44'1 uv1 ondisk = 0) ==== 201+0+0 (crc 0 0 0) 0x5603ad90e000 con 0x5603ae603680 2024-04-25T10:30:38.430 INFO:teuthology.orchestra.run.smithi037.stdout:{ 2024-04-25T10:30:38.430 INFO:teuthology.orchestra.run.smithi037.stdout: "0": { 2024-04-25T10:30:38.430 INFO:teuthology.orchestra.run.smithi037.stdout: "data": {}, 2024-04-25T10:30:38.430 INFO:teuthology.orchestra.run.smithi037.stdout: "result": 0 2024-04-25T10:30:38.430 INFO:teuthology.orchestra.run.smithi037.stdout: } 2024-04-25T10:30:38.430 INFO:teuthology.orchestra.run.smithi037.stdout:} 2024-04-25T10:30:38.430 INFO:teuthology.orchestra.run.smithi037.stdout: 2024-04-25T10:30:38.432 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:38.424+0000 7f4265f1f280 1 -- 172.21.15.37:0/1159520047 >> [v2:172.21.15.203:6800/4051532063,v1:172.21.15.203:6801/4051532063] conn(0x5603ae603680 msgr2=0x5603ad8e2680 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:30:38.432 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:38.424+0000 7f4265f1f280 1 --2- 172.21.15.37:0/1159520047 >> [v2:172.21.15.203:6800/4051532063,v1:172.21.15.203:6801/4051532063] conn(0x5603ae603680 0x5603ad8e2680 crc :-1 s=READY pgs=13 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:30:38.432 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:38.424+0000 7f4265f1f280 1 -- 172.21.15.37:0/1159520047 >> [v2:172.21.15.203:6832/2418205237,v1:172.21.15.203:6833/2418205237] conn(0x5603ae605600 msgr2=0x5603ad8e2100 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:30:38.432 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:38.424+0000 7f4265f1f280 1 --2- 172.21.15.37:0/1159520047 >> [v2:172.21.15.203:6832/2418205237,v1:172.21.15.203:6833/2418205237] conn(0x5603ae605600 0x5603ad8e2100 secure :-1 s=READY pgs=262 cs=0 l=1 rev1=1 crypto rx=0x5603ae63a870 tx=0x5603ad924a80 comp rx=0 tx=0).stop 2024-04-25T10:30:38.433 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:38.424+0000 7f4265f1f280 1 -- 172.21.15.37:0/1159520047 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x5603ae602d80 msgr2=0x5603ad8e1600 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:30:38.433 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:38.424+0000 7f4265f1f280 1 --2- 172.21.15.37:0/1159520047 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x5603ae602d80 0x5603ad8e1600 secure :-1 s=READY pgs=551 cs=0 l=1 rev1=1 crypto rx=0x5603ae65fa10 tx=0x5603ad925f80 comp rx=0 tx=0).stop 2024-04-25T10:30:38.433 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:38.424+0000 7f42635b1640 1 -- 172.21.15.37:0/1159520047 reap_dead start 2024-04-25T10:30:38.433 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:38.424+0000 7f4265f1f280 1 -- 172.21.15.37:0/1159520047 shutdown_connections 2024-04-25T10:30:38.433 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:38.424+0000 7f4265f1f280 1 -- 172.21.15.37:0/1159520047 >> 172.21.15.37:0/1159520047 conn(0x5603ae602480 msgr2=0x5603ad955800 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:30:38.433 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:38.424+0000 7f4265f1f280 1 -- 172.21.15.37:0/1159520047 shutdown_connections 2024-04-25T10:30:38.433 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:38.424+0000 7f4265f1f280 1 -- 172.21.15.37:0/1159520047 wait complete. 2024-04-25T10:30:38.433 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:38.425+0000 7f4265f1f280 1 -- 172.21.15.37:0/3123934490 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x5603ad9bfa80 msgr2=0x5603ad8e0000 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:30:38.433 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:38.425+0000 7f4265f1f280 1 --2- 172.21.15.37:0/3123934490 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x5603ad9bfa80 0x5603ad8e0000 secure :-1 s=READY pgs=549 cs=0 l=1 rev1=1 crypto rx=0x5603ae65f350 tx=0x5603ad924a20 comp rx=0 tx=0).stop 2024-04-25T10:30:38.433 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:38.425+0000 7f4265f1f280 1 -- 172.21.15.37:0/3123934490 shutdown_connections 2024-04-25T10:30:38.433 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:38.425+0000 7f4265f1f280 1 --2- 172.21.15.37:0/3123934490 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x5603ae602000 0x5603ad8e0b00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:30:38.433 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:38.425+0000 7f4265f1f280 1 --2- 172.21.15.37:0/3123934490 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x5603ad9bf180 0x5603ad8e0580 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:30:38.433 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:38.425+0000 7f4265f1f280 1 --2- 172.21.15.37:0/3123934490 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x5603ad9bfa80 0x5603ad8e0000 unknown :-1 s=CLOSED pgs=549 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:30:38.433 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:38.425+0000 7f4265f1f280 1 -- 172.21.15.37:0/3123934490 >> 172.21.15.37:0/3123934490 conn(0x5603ad9bf600 msgr2=0x5603ad955000 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:30:38.434 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:38.425+0000 7f4265f1f280 1 -- 172.21.15.37:0/3123934490 shutdown_connections 2024-04-25T10:30:38.434 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:38.425+0000 7f4265f1f280 1 -- 172.21.15.37:0/3123934490 wait complete. 2024-04-25T10:30:38.434 DEBUG:tasks.cephfs.filesystem:Ran ['cephfs-table-tool', '--debug-mds=20', '--debug-ms=1', '--debug-objecter=1', 'recovery_fs:0', 'reset', 'session'] in time 0:00:00.071974, result: { "0": { "data": {}, "result": 0 } } 2024-04-25T10:30:38.434 DEBUG:teuthology.orchestra.run.smithi037:> cephfs-table-tool --debug-mds=20 --debug-ms=1 --debug-objecter=1 recovery_fs:0 reset snap 2024-04-25T10:30:38.486 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:38.484+0000 7f0d90850280 1 -- start start 2024-04-25T10:30:38.486 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:38.484+0000 7f0d90850280 1 Processor -- start 2024-04-25T10:30:38.489 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:38.485+0000 7f0d90850280 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x5592b69fba80 0x5592b6914000 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:30:38.489 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:38.485+0000 7f0d90850280 1 --2- >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x5592b69fb180 0x5592b6914580 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:30:38.489 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:38.485+0000 7f0d90850280 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x5592b7732000 0x5592b6914b00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:30:38.490 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:38.485+0000 7f0d90850280 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 -- 0x5592b6920780 con 0x5592b7732000 2024-04-25T10:30:38.490 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:38.485+0000 7f0d90850280 1 -- --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_getmap magic: 0 -- 0x5592b6920600 con 0x5592b69fba80 2024-04-25T10:30:38.490 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:38.485+0000 7f0d90850280 1 -- --> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] -- mon_getmap magic: 0 -- 0x5592b6920480 con 0x5592b69fb180 2024-04-25T10:30:38.490 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:38.485+0000 7f0d8f7fd640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x5592b7732000 0x5592b6914b00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:30:38.490 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:38.485+0000 7f0d8f7fd640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x5592b7732000 0x5592b6914b00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.37:3300/0 says I am v2:172.21.15.37:55578/0 (socket says 172.21.15.37:55578) 2024-04-25T10:30:38.490 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:38.485+0000 7f0d8f7fd640 1 -- 172.21.15.37:0/2228638765 learned_addr learned my addr 172.21.15.37:0/2228638765 (peer_addr_for_me v2:172.21.15.37:0/0) 2024-04-25T10:30:38.490 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:38.485+0000 7f0d8effc640 1 --2- 172.21.15.37:0/2228638765 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x5592b69fba80 0x5592b6914000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:30:38.490 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:38.485+0000 7f0d8e7fb640 1 --2- 172.21.15.37:0/2228638765 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x5592b69fb180 0x5592b6914580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:30:38.490 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:38.486+0000 7f0d8f7fd640 1 -- 172.21.15.37:0/2228638765 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x5592b69fb180 msgr2=0x5592b6914580 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:30:38.490 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:38.486+0000 7f0d8f7fd640 1 --2- 172.21.15.37:0/2228638765 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x5592b69fb180 0x5592b6914580 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:30:38.490 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:38.486+0000 7f0d8f7fd640 1 -- 172.21.15.37:0/2228638765 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x5592b69fba80 msgr2=0x5592b6914000 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:30:38.490 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:38.486+0000 7f0d8f7fd640 1 --2- 172.21.15.37:0/2228638765 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x5592b69fba80 0x5592b6914000 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:30:38.491 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:38.486+0000 7f0d8f7fd640 1 -- 172.21.15.37:0/2228638765 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5592b286a780 con 0x5592b7732000 2024-04-25T10:30:38.491 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:38.486+0000 7f0d8e7fb640 1 --2- 172.21.15.37:0/2228638765 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x5592b69fb180 0x5592b6914580 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-25T10:30:38.491 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:38.486+0000 7f0d8f7fd640 1 --2- 172.21.15.37:0/2228638765 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x5592b7732000 0x5592b6914b00 secure :-1 s=READY pgs=552 cs=0 l=1 rev1=1 crypto rx=0x5592b769b3e0 tx=0x5592b6962120 comp rx=0 tx=0).ready entity=mon.0 client_cookie=52d3be5a4b9eb138 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:30:38.491 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:38.486+0000 7f0d8c7f7640 1 -- 172.21.15.37:0/2228638765 <== mon.0 v2:172.21.15.37:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x5592b285a680 con 0x5592b7732000 2024-04-25T10:30:38.491 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:38.486+0000 7f0d8c7f7640 1 -- 172.21.15.37:0/2228638765 <== mon.0 v2:172.21.15.37:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x5592b2860a80 con 0x5592b7732000 2024-04-25T10:30:38.491 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:38.486+0000 7f0d8c7f7640 1 -- 172.21.15.37:0/2228638765 <== mon.0 v2:172.21.15.37:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x5592b285ab60 con 0x5592b7732000 2024-04-25T10:30:38.491 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:38.486+0000 7f0d90850280 1 -- 172.21.15.37:0/2228638765 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5592b286a3c0 con 0x5592b7732000 2024-04-25T10:30:38.491 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:38.486+0000 7f0d90850280 1 -- 172.21.15.37:0/2228638765 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({fsmap=0}) -- 0x5592b286b680 con 0x5592b7732000 2024-04-25T10:30:38.491 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:38.486+0000 7f0d90850280 4 waiting for MDS map... 2024-04-25T10:30:38.491 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:38.486+0000 7f0d8c7f7640 1 -- 172.21.15.37:0/2228638765 <== mon.0 v2:172.21.15.37:3300/0 4 ==== osd_map(44..44 src has 1..44) ==== 6248+0+0 (secure 0 0 0) 0x5592b774e600 con 0x5592b7732000 2024-04-25T10:30:38.491 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:38.486+0000 7f0d8c7f7640 1 -- 172.21.15.37:0/2228638765 <== mon.0 v2:172.21.15.37:3300/0 5 ==== fsmap(e 22) ==== 4135+0+0 (secure 0 0 0) 0x5592b68bf180 con 0x5592b7732000 2024-04-25T10:30:38.491 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:38.486+0000 7f0d90850280 4 Got MDS map 22 2024-04-25T10:30:38.491 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:38.486+0000 7f0d90850280 10 main: main 2024-04-25T10:30:38.491 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:38.487+0000 7f0d90850280 4 main: connecting to RADOS... 2024-04-25T10:30:38.491 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:38.487+0000 7f0d90850280 1 Processor -- start 2024-04-25T10:30:38.491 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:38.487+0000 7f0d90850280 1 -- start start 2024-04-25T10:30:38.496 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:38.489+0000 7f0d90850280 1 --2- >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x5592b7732900 0x5592b6915080 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:30:38.496 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:38.489+0000 7f0d90850280 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x5592b7732d80 0x5592b6915600 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:30:38.496 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:38.489+0000 7f0d90850280 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x5592b7733200 0x5592b6915b80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:30:38.496 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:38.489+0000 7f0d90850280 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 -- 0x5592b6920300 con 0x5592b7732d80 2024-04-25T10:30:38.496 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:38.489+0000 7f0d90850280 1 -- --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_getmap magic: 0 -- 0x5592b6920000 con 0x5592b7733200 2024-04-25T10:30:38.496 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:38.489+0000 7f0d90850280 1 -- --> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] -- mon_getmap magic: 0 -- 0x5592b68c6900 con 0x5592b7732900 2024-04-25T10:30:38.496 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:38.489+0000 7f0d8effc640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x5592b7732d80 0x5592b6915600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:30:38.496 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:38.489+0000 7f0d8effc640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x5592b7732d80 0x5592b6915600 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.37:3300/0 says I am v2:172.21.15.37:55590/0 (socket says 172.21.15.37:55590) 2024-04-25T10:30:38.496 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:38.489+0000 7f0d8effc640 1 -- 172.21.15.37:0/1765478926 learned_addr learned my addr 172.21.15.37:0/1765478926 (peer_addr_for_me v2:172.21.15.37:0/0) 2024-04-25T10:30:38.496 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:38.489+0000 7f0d8e7fb640 1 --2- 172.21.15.37:0/1765478926 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x5592b7732900 0x5592b6915080 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:30:38.496 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:38.489+0000 7f0d8e7fb640 1 --2- 172.21.15.37:0/1765478926 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x5592b7733200 0x5592b6915b80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:30:38.496 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:38.489+0000 7f0d8effc640 1 -- 172.21.15.37:0/1765478926 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x5592b7732900 msgr2=0x5592b6915080 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:30:38.497 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:38.489+0000 7f0d8effc640 1 --2- 172.21.15.37:0/1765478926 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x5592b7732900 0x5592b6915080 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:30:38.497 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:38.489+0000 7f0d8effc640 1 -- 172.21.15.37:0/1765478926 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x5592b7733200 msgr2=0x5592b6915b80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:30:38.497 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:38.489+0000 7f0d8effc640 1 --2- 172.21.15.37:0/1765478926 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x5592b7733200 0x5592b6915b80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:30:38.497 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:38.489+0000 7f0d8effc640 1 -- 172.21.15.37:0/1765478926 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5592b286b4a0 con 0x5592b7732d80 2024-04-25T10:30:38.497 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:38.489+0000 7f0d8effc640 1 --2- 172.21.15.37:0/1765478926 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x5592b7732d80 0x5592b6915600 secure :-1 s=READY pgs=553 cs=0 l=1 rev1=1 crypto rx=0x5592b7656840 tx=0x5592b775cc00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f2c30615f880cf82 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:30:38.497 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:38.489+0000 7f0d8a7f3640 1 -- 172.21.15.37:0/1765478926 <== mon.0 v2:172.21.15.37:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x5592b285a680 con 0x5592b7732d80 2024-04-25T10:30:38.497 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:38.489+0000 7f0d8a7f3640 1 -- 172.21.15.37:0/1765478926 <== mon.0 v2:172.21.15.37:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x5592b2860fc0 con 0x5592b7732d80 2024-04-25T10:30:38.497 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:38.489+0000 7f0d8a7f3640 1 -- 172.21.15.37:0/1765478926 <== mon.0 v2:172.21.15.37:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x5592b285ab60 con 0x5592b7732d80 2024-04-25T10:30:38.497 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:38.490+0000 7f0d90850280 1 -- 172.21.15.37:0/1765478926 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x5592b7732d80 msgr2=0x5592b6915600 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:30:38.497 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:38.490+0000 7f0d90850280 1 --2- 172.21.15.37:0/1765478926 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x5592b7732d80 0x5592b6915600 secure :-1 s=READY pgs=553 cs=0 l=1 rev1=1 crypto rx=0x5592b7656840 tx=0x5592b775cc00 comp rx=0 tx=0).stop 2024-04-25T10:30:38.497 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:38.490+0000 7f0d90850280 1 -- 172.21.15.37:0/1765478926 shutdown_connections 2024-04-25T10:30:38.497 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:38.490+0000 7f0d90850280 1 --2- 172.21.15.37:0/1765478926 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x5592b7733200 0x5592b6915b80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:30:38.497 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:38.490+0000 7f0d90850280 1 --2- 172.21.15.37:0/1765478926 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x5592b7732d80 0x5592b6915600 unknown :-1 s=CLOSED pgs=553 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:30:38.497 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:38.490+0000 7f0d90850280 1 --2- 172.21.15.37:0/1765478926 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x5592b7732900 0x5592b6915080 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:30:38.498 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:38.490+0000 7f0d90850280 1 -- 172.21.15.37:0/1765478926 >> 172.21.15.37:0/1765478926 conn(0x5592b7732480 msgr2=0x5592b6991800 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:30:38.498 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:38.490+0000 7f0d90850280 1 -- 172.21.15.37:0/1765478926 shutdown_connections 2024-04-25T10:30:38.498 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:38.490+0000 7f0d90850280 1 -- 172.21.15.37:0/1765478926 wait complete. 2024-04-25T10:30:38.498 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:38.491+0000 7f0d90850280 1 Processor -- start 2024-04-25T10:30:38.498 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:38.491+0000 7f0d90850280 1 -- start start 2024-04-25T10:30:38.498 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:38.492+0000 7f0d90850280 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x5592b7732900 0x5592b6915080 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:30:38.498 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:38.492+0000 7f0d90850280 1 --2- >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x5592b7732d80 0x5592b6915600 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:30:38.498 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:38.492+0000 7f0d90850280 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x5592b7733200 0x5592b6915b80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:30:38.498 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:38.492+0000 7f0d90850280 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 -- 0x5592b6920000 con 0x5592b7733200 2024-04-25T10:30:38.498 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:38.492+0000 7f0d90850280 1 -- --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_getmap magic: 0 -- 0x5592b6921980 con 0x5592b7732900 2024-04-25T10:30:38.498 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:38.492+0000 7f0d90850280 1 -- --> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] -- mon_getmap magic: 0 -- 0x5592b6920300 con 0x5592b7732d80 2024-04-25T10:30:38.498 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:38.492+0000 7f0d8e7fb640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x5592b7733200 0x5592b6915b80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:30:38.499 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:38.492+0000 7f0d8e7fb640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x5592b7733200 0x5592b6915b80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.37:3300/0 says I am v2:172.21.15.37:55606/0 (socket says 172.21.15.37:55606) 2024-04-25T10:30:38.499 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:38.492+0000 7f0d8e7fb640 1 -- 172.21.15.37:0/279810063 learned_addr learned my addr 172.21.15.37:0/279810063 (peer_addr_for_me v2:172.21.15.37:0/0) 2024-04-25T10:30:38.499 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:38.492+0000 7f0d8effc640 1 --2- 172.21.15.37:0/279810063 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x5592b7732d80 0x5592b6915600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:30:38.499 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:38.492+0000 7f0d8e7fb640 1 --2- 172.21.15.37:0/279810063 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x5592b7732900 0x5592b6915080 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:30:38.499 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:38.493+0000 7f0d8e7fb640 1 -- 172.21.15.37:0/279810063 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x5592b7732d80 msgr2=0x5592b6915600 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:30:38.499 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:38.493+0000 7f0d8e7fb640 1 --2- 172.21.15.37:0/279810063 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x5592b7732d80 0x5592b6915600 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:30:38.499 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:38.493+0000 7f0d8e7fb640 1 -- 172.21.15.37:0/279810063 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x5592b7732900 msgr2=0x5592b6915080 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:30:38.499 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:38.493+0000 7f0d8e7fb640 1 --2- 172.21.15.37:0/279810063 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x5592b7732900 0x5592b6915080 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:30:38.499 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:38.493+0000 7f0d8e7fb640 1 -- 172.21.15.37:0/279810063 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5592b286a780 con 0x5592b7733200 2024-04-25T10:30:38.499 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:38.493+0000 7f0d8effc640 1 --2- 172.21.15.37:0/279810063 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x5592b7732d80 0x5592b6915600 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-25T10:30:38.499 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:38.493+0000 7f0d8e7fb640 1 --2- 172.21.15.37:0/279810063 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x5592b7733200 0x5592b6915b80 secure :-1 s=READY pgs=554 cs=0 l=1 rev1=1 crypto rx=0x5592b7657b60 tx=0x5592b6960a20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:30:38.500 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:38.493+0000 7f0d88ff0640 1 -- 172.21.15.37:0/279810063 <== mon.0 v2:172.21.15.37:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x5592b285a9c0 con 0x5592b7733200 2024-04-25T10:30:38.500 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:38.493+0000 7f0d88ff0640 1 -- 172.21.15.37:0/279810063 <== mon.0 v2:172.21.15.37:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x5592b2860fc0 con 0x5592b7733200 2024-04-25T10:30:38.500 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:38.493+0000 7f0d90850280 1 -- 172.21.15.37:0/279810063 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x5592b286a3c0 con 0x5592b7733200 2024-04-25T10:30:38.500 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:38.493+0000 7f0d88ff0640 1 -- 172.21.15.37:0/279810063 <== mon.0 v2:172.21.15.37:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x5592b285a680 con 0x5592b7733200 2024-04-25T10:30:38.500 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:38.493+0000 7f0d90850280 1 -- 172.21.15.37:0/279810063 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5592b286b680 con 0x5592b7733200 2024-04-25T10:30:38.500 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:38.493+0000 7f0d90850280 4 main: resolving pool 6 2024-04-25T10:30:38.501 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:38.494+0000 7f0d88ff0640 1 -- 172.21.15.37:0/279810063 <== mon.0 v2:172.21.15.37:3300/0 4 ==== mgrmap(e 5) ==== 96782+0+0 (secure 0 0 0) 0x5592b7794900 con 0x5592b7733200 2024-04-25T10:30:38.501 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:38.495+0000 7f0d88ff0640 1 --2- 172.21.15.37:0/279810063 >> [v2:172.21.15.203:6832/2418205237,v1:172.21.15.203:6833/2418205237] conn(0x5592b7735600 0x5592b6916100 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:30:38.502 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:38.495+0000 7f0d88ff0640 1 -- 172.21.15.37:0/279810063 <== mon.0 v2:172.21.15.37:3300/0 5 ==== osd_map(44..44 src has 1..44) ==== 6248+0+0 (secure 0 0 0) 0x5592b77bcc00 con 0x5592b7733200 2024-04-25T10:30:38.502 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:38.495+0000 7f0d90850280 4 main: creating IoCtx.. 2024-04-25T10:30:38.502 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:38.495+0000 7f0d90850280 1 --2- 172.21.15.37:0/279810063 >> [v2:172.21.15.37:6800/1690341662,v1:172.21.15.37:6801/1690341662] conn(0x5592b7733680 0x5592b6916680 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:30:38.502 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:38.495+0000 7f0d8effc640 1 --2- 172.21.15.37:0/279810063 >> [v2:172.21.15.203:6832/2418205237,v1:172.21.15.203:6833/2418205237] conn(0x5592b7735600 0x5592b6916100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:30:38.502 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:38.495+0000 7f0d90850280 1 -- 172.21.15.37:0/279810063 --> [v2:172.21.15.37:6800/1690341662,v1:172.21.15.37:6801/1690341662] -- osd_op(unknown.0.0:1 6.d 6:b50e409b:::mds_snaptable:head [writefull 0~70 in=70b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e44) -- 0x5592b77cfc00 con 0x5592b7733680 2024-04-25T10:30:38.502 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:38.495+0000 7f0d8e7fb640 1 --2- 172.21.15.37:0/279810063 >> [v2:172.21.15.37:6800/1690341662,v1:172.21.15.37:6801/1690341662] conn(0x5592b7733680 0x5592b6916680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:30:38.502 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:38.495+0000 7f0d8e7fb640 1 --2- 172.21.15.37:0/279810063 >> [v2:172.21.15.37:6800/1690341662,v1:172.21.15.37:6801/1690341662] conn(0x5592b7733680 0x5592b6916680 crc :-1 s=READY pgs=14 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:30:38.502 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:38.495+0000 7f0d8effc640 1 --2- 172.21.15.37:0/279810063 >> [v2:172.21.15.203:6832/2418205237,v1:172.21.15.203:6833/2418205237] conn(0x5592b7735600 0x5592b6916100 secure :-1 s=READY pgs=263 cs=0 l=1 rev1=1 crypto rx=0x5592b7656840 tx=0x5592b6960a80 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:30:38.502 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:38.497+0000 7f0d8e7fb640 1 -- 172.21.15.37:0/279810063 <== osd.1 v2:172.21.15.37:6800/1690341662 1 ==== osd_op_reply(1 mds_snaptable [writefull 0~70] v44'1 uv1 ondisk = 0) ==== 157+0+0 (crc 0 0 0) 0x5592b694a000 con 0x5592b7733680 2024-04-25T10:30:38.502 INFO:teuthology.orchestra.run.smithi037.stdout:{ 2024-04-25T10:30:38.503 INFO:teuthology.orchestra.run.smithi037.stdout: "result": 0 2024-04-25T10:30:38.503 INFO:teuthology.orchestra.run.smithi037.stdout:} 2024-04-25T10:30:38.503 INFO:teuthology.orchestra.run.smithi037.stdout: 2024-04-25T10:30:38.505 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:38.497+0000 7f0d90850280 1 -- 172.21.15.37:0/279810063 >> [v2:172.21.15.37:6800/1690341662,v1:172.21.15.37:6801/1690341662] conn(0x5592b7733680 msgr2=0x5592b6916680 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:30:38.505 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:38.497+0000 7f0d90850280 1 --2- 172.21.15.37:0/279810063 >> [v2:172.21.15.37:6800/1690341662,v1:172.21.15.37:6801/1690341662] conn(0x5592b7733680 0x5592b6916680 crc :-1 s=READY pgs=14 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:30:38.505 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:38.497+0000 7f0d90850280 1 -- 172.21.15.37:0/279810063 >> [v2:172.21.15.203:6832/2418205237,v1:172.21.15.203:6833/2418205237] conn(0x5592b7735600 msgr2=0x5592b6916100 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:30:38.505 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:38.497+0000 7f0d90850280 1 --2- 172.21.15.37:0/279810063 >> [v2:172.21.15.203:6832/2418205237,v1:172.21.15.203:6833/2418205237] conn(0x5592b7735600 0x5592b6916100 secure :-1 s=READY pgs=263 cs=0 l=1 rev1=1 crypto rx=0x5592b7656840 tx=0x5592b6960a80 comp rx=0 tx=0).stop 2024-04-25T10:30:38.505 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:38.497+0000 7f0d90850280 1 -- 172.21.15.37:0/279810063 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x5592b7733200 msgr2=0x5592b6915b80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:30:38.505 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:38.497+0000 7f0d90850280 1 --2- 172.21.15.37:0/279810063 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x5592b7733200 0x5592b6915b80 secure :-1 s=READY pgs=554 cs=0 l=1 rev1=1 crypto rx=0x5592b7657b60 tx=0x5592b6960a20 comp rx=0 tx=0).stop 2024-04-25T10:30:38.505 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:38.497+0000 7f0d8effc640 1 -- 172.21.15.37:0/279810063 reap_dead start 2024-04-25T10:30:38.505 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:38.497+0000 7f0d90850280 1 -- 172.21.15.37:0/279810063 shutdown_connections 2024-04-25T10:30:38.505 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:38.497+0000 7f0d90850280 1 -- 172.21.15.37:0/279810063 >> 172.21.15.37:0/279810063 conn(0x5592b7732480 msgr2=0x5592b6991800 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:30:38.505 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:38.497+0000 7f0d90850280 1 -- 172.21.15.37:0/279810063 shutdown_connections 2024-04-25T10:30:38.505 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:38.497+0000 7f0d90850280 1 -- 172.21.15.37:0/279810063 wait complete. 2024-04-25T10:30:38.505 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:38.497+0000 7f0d90850280 1 -- 172.21.15.37:0/2228638765 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x5592b7732000 msgr2=0x5592b6914b00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:30:38.506 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:38.497+0000 7f0d90850280 1 --2- 172.21.15.37:0/2228638765 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x5592b7732000 0x5592b6914b00 secure :-1 s=READY pgs=552 cs=0 l=1 rev1=1 crypto rx=0x5592b769b3e0 tx=0x5592b6962120 comp rx=0 tx=0).stop 2024-04-25T10:30:38.506 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:38.497+0000 7f0d90850280 1 -- 172.21.15.37:0/2228638765 shutdown_connections 2024-04-25T10:30:38.506 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:38.497+0000 7f0d90850280 1 --2- 172.21.15.37:0/2228638765 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x5592b7732000 0x5592b6914b00 unknown :-1 s=CLOSED pgs=552 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:30:38.506 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:38.498+0000 7f0d90850280 1 --2- 172.21.15.37:0/2228638765 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x5592b69fb180 0x5592b6914580 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:30:38.506 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:38.498+0000 7f0d90850280 1 --2- 172.21.15.37:0/2228638765 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x5592b69fba80 0x5592b6914000 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:30:38.506 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:38.498+0000 7f0d90850280 1 -- 172.21.15.37:0/2228638765 >> 172.21.15.37:0/2228638765 conn(0x5592b69fb600 msgr2=0x5592b6991000 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:30:38.506 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:38.498+0000 7f0d90850280 1 -- 172.21.15.37:0/2228638765 shutdown_connections 2024-04-25T10:30:38.506 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:38.498+0000 7f0d90850280 1 -- 172.21.15.37:0/2228638765 wait complete. 2024-04-25T10:30:38.506 DEBUG:tasks.cephfs.filesystem:Ran ['cephfs-table-tool', '--debug-mds=20', '--debug-ms=1', '--debug-objecter=1', 'recovery_fs:0', 'reset', 'snap'] in time 0:00:00.072265, result: { "result": 0 } 2024-04-25T10:30:38.506 DEBUG:teuthology.orchestra.run.smithi037:> cephfs-table-tool --debug-mds=20 --debug-ms=1 --debug-objecter=1 recovery_fs:0 reset inode 2024-04-25T10:30:38.558 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:38.556+0000 7fe7b0959280 1 -- start start 2024-04-25T10:30:38.558 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:38.556+0000 7fe7b0959280 1 Processor -- start 2024-04-25T10:30:38.561 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:38.557+0000 7fe7b0959280 1 --2- >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x55ba2a579a80 0x55ba2a496000 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:30:38.561 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:38.557+0000 7fe7b0959280 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55ba2a579180 0x55ba2a496580 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:30:38.561 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:38.557+0000 7fe7b0959280 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x55ba2b1f2000 0x55ba2a496b00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:30:38.561 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:38.557+0000 7fe7b0959280 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 -- 0x55ba2a49e600 con 0x55ba2a579180 2024-04-25T10:30:38.561 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:38.557+0000 7fe7b0959280 1 -- --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_getmap magic: 0 -- 0x55ba2a49e480 con 0x55ba2b1f2000 2024-04-25T10:30:38.561 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:38.557+0000 7fe7b0959280 1 -- --> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] -- mon_getmap magic: 0 -- 0x55ba2a49e300 con 0x55ba2a579a80 2024-04-25T10:30:38.561 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:38.557+0000 7fe7ae904640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55ba2a579180 0x55ba2a496580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:30:38.562 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:38.557+0000 7fe7ae904640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55ba2a579180 0x55ba2a496580 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.37:3300/0 says I am v2:172.21.15.37:55622/0 (socket says 172.21.15.37:55622) 2024-04-25T10:30:38.562 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:38.557+0000 7fe7ae904640 1 -- 172.21.15.37:0/574161655 learned_addr learned my addr 172.21.15.37:0/574161655 (peer_addr_for_me v2:172.21.15.37:0/0) 2024-04-25T10:30:38.562 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:38.557+0000 7fe7af906640 1 --2- 172.21.15.37:0/574161655 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x55ba2b1f2000 0x55ba2a496b00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:30:38.562 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:38.557+0000 7fe7af105640 1 --2- 172.21.15.37:0/574161655 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x55ba2a579a80 0x55ba2a496000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:30:38.562 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:38.557+0000 7fe7ae904640 1 -- 172.21.15.37:0/574161655 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x55ba2a579a80 msgr2=0x55ba2a496000 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:30:38.562 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:38.557+0000 7fe7ae904640 1 --2- 172.21.15.37:0/574161655 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x55ba2a579a80 0x55ba2a496000 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:30:38.562 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:38.557+0000 7fe7ae904640 1 -- 172.21.15.37:0/574161655 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x55ba2b1f2000 msgr2=0x55ba2a496b00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:30:38.562 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:38.557+0000 7fe7ae904640 1 --2- 172.21.15.37:0/574161655 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x55ba2b1f2000 0x55ba2a496b00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:30:38.562 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:38.557+0000 7fe7ae904640 1 -- 172.21.15.37:0/574161655 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55ba263e8780 con 0x55ba2a579180 2024-04-25T10:30:38.562 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:38.557+0000 7fe7af906640 1 --2- 172.21.15.37:0/574161655 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x55ba2b1f2000 0x55ba2a496b00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T10:30:38.562 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:38.557+0000 7fe7af105640 1 --2- 172.21.15.37:0/574161655 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x55ba2a579a80 0x55ba2a496000 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T10:30:38.562 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:38.558+0000 7fe7ae904640 1 --2- 172.21.15.37:0/574161655 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55ba2a579180 0x55ba2a496580 secure :-1 s=READY pgs=556 cs=0 l=1 rev1=1 crypto rx=0x55ba2b2193b0 tx=0x55ba2a4de8a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=2cdf4869ba999c52 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:30:38.562 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:38.558+0000 7fe7ac900640 1 -- 172.21.15.37:0/574161655 <== mon.0 v2:172.21.15.37:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x55ba263d89c0 con 0x55ba2a579180 2024-04-25T10:30:38.562 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:38.558+0000 7fe7ac900640 1 -- 172.21.15.37:0/574161655 <== mon.0 v2:172.21.15.37:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x55ba263dee00 con 0x55ba2a579180 2024-04-25T10:30:38.563 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:38.558+0000 7fe7b0959280 1 -- 172.21.15.37:0/574161655 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55ba263e94a0 con 0x55ba2a579180 2024-04-25T10:30:38.563 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:38.558+0000 7fe7ac900640 1 -- 172.21.15.37:0/574161655 <== mon.0 v2:172.21.15.37:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x55ba263d8680 con 0x55ba2a579180 2024-04-25T10:30:38.563 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:38.558+0000 7fe7b0959280 1 -- 172.21.15.37:0/574161655 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({fsmap=0}) -- 0x55ba263e8000 con 0x55ba2a579180 2024-04-25T10:30:38.563 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:38.558+0000 7fe7b0959280 4 waiting for MDS map... 2024-04-25T10:30:38.563 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:38.558+0000 7fe7ac900640 1 -- 172.21.15.37:0/574161655 <== mon.0 v2:172.21.15.37:3300/0 4 ==== osd_map(44..44 src has 1..44) ==== 6248+0+0 (secure 0 0 0) 0x55ba2b2c4600 con 0x55ba2a579180 2024-04-25T10:30:38.563 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:38.558+0000 7fe7ac900640 1 -- 172.21.15.37:0/574161655 <== mon.0 v2:172.21.15.37:3300/0 5 ==== fsmap(e 22) ==== 4135+0+0 (secure 0 0 0) 0x55ba2a43d180 con 0x55ba2a579180 2024-04-25T10:30:38.563 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:38.558+0000 7fe7b0959280 4 Got MDS map 22 2024-04-25T10:30:38.563 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:38.558+0000 7fe7b0959280 10 main: main 2024-04-25T10:30:38.563 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:38.558+0000 7fe7b0959280 4 main: connecting to RADOS... 2024-04-25T10:30:38.563 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:38.559+0000 7fe7b0959280 1 Processor -- start 2024-04-25T10:30:38.563 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:38.559+0000 7fe7b0959280 1 -- start start 2024-04-25T10:30:38.568 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:38.560+0000 7fe7b0959280 1 --2- >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x55ba2b1f2900 0x55ba2a497080 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:30:38.568 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:38.560+0000 7fe7b0959280 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x55ba2b1f2d80 0x55ba2a497600 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:30:38.568 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:38.560+0000 7fe7b0959280 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55ba2b1f3200 0x55ba2a497b80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:30:38.568 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:38.560+0000 7fe7b0959280 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 -- 0x55ba2a49e180 con 0x55ba2b1f3200 2024-04-25T10:30:38.568 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:38.560+0000 7fe7b0959280 1 -- --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_getmap magic: 0 -- 0x55ba2a444480 con 0x55ba2b1f2d80 2024-04-25T10:30:38.568 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:38.560+0000 7fe7b0959280 1 -- --> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] -- mon_getmap magic: 0 -- 0x55ba2a444900 con 0x55ba2b1f2900 2024-04-25T10:30:38.568 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:38.560+0000 7fe7ae904640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55ba2b1f3200 0x55ba2a497b80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:30:38.568 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:38.560+0000 7fe7ae904640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55ba2b1f3200 0x55ba2a497b80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.37:3300/0 says I am v2:172.21.15.37:55630/0 (socket says 172.21.15.37:55630) 2024-04-25T10:30:38.568 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:38.560+0000 7fe7ae904640 1 -- 172.21.15.37:0/3307730207 learned_addr learned my addr 172.21.15.37:0/3307730207 (peer_addr_for_me v2:172.21.15.37:0/0) 2024-04-25T10:30:38.568 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:38.560+0000 7fe7af906640 1 --2- 172.21.15.37:0/3307730207 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x55ba2b1f2900 0x55ba2a497080 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:30:38.568 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:38.560+0000 7fe7af105640 1 --2- 172.21.15.37:0/3307730207 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x55ba2b1f2d80 0x55ba2a497600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:30:38.568 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:38.561+0000 7fe7ae904640 1 -- 172.21.15.37:0/3307730207 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x55ba2b1f2900 msgr2=0x55ba2a497080 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:30:38.568 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:38.561+0000 7fe7ae904640 1 --2- 172.21.15.37:0/3307730207 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x55ba2b1f2900 0x55ba2a497080 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:30:38.569 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:38.561+0000 7fe7ae904640 1 -- 172.21.15.37:0/3307730207 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x55ba2b1f2d80 msgr2=0x55ba2a497600 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:30:38.569 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:38.561+0000 7fe7ae904640 1 --2- 172.21.15.37:0/3307730207 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x55ba2b1f2d80 0x55ba2a497600 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:30:38.569 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:38.561+0000 7fe7ae904640 1 -- 172.21.15.37:0/3307730207 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55ba263e83c0 con 0x55ba2b1f3200 2024-04-25T10:30:38.569 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:38.561+0000 7fe7af906640 1 --2- 172.21.15.37:0/3307730207 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x55ba2b1f2900 0x55ba2a497080 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T10:30:38.569 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:38.561+0000 7fe7ae904640 1 --2- 172.21.15.37:0/3307730207 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55ba2b1f3200 0x55ba2a497b80 secure :-1 s=READY pgs=557 cs=0 l=1 rev1=1 crypto rx=0x55ba2b1f0300 tx=0x55ba2a4de660 comp rx=0 tx=0).ready entity=mon.0 client_cookie=946d833071cc2150 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:30:38.569 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:38.561+0000 7fe7aa8fc640 1 -- 172.21.15.37:0/3307730207 <== mon.0 v2:172.21.15.37:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x55ba263d8b60 con 0x55ba2b1f3200 2024-04-25T10:30:38.569 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:38.561+0000 7fe7aa8fc640 1 -- 172.21.15.37:0/3307730207 <== mon.0 v2:172.21.15.37:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x55ba263dea80 con 0x55ba2b1f3200 2024-04-25T10:30:38.569 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:38.561+0000 7fe7aa8fc640 1 -- 172.21.15.37:0/3307730207 <== mon.0 v2:172.21.15.37:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x55ba263d8820 con 0x55ba2b1f3200 2024-04-25T10:30:38.569 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:38.562+0000 7fe7b0959280 1 -- 172.21.15.37:0/3307730207 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55ba2b1f3200 msgr2=0x55ba2a497b80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:30:38.569 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:38.562+0000 7fe7b0959280 1 --2- 172.21.15.37:0/3307730207 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55ba2b1f3200 0x55ba2a497b80 secure :-1 s=READY pgs=557 cs=0 l=1 rev1=1 crypto rx=0x55ba2b1f0300 tx=0x55ba2a4de660 comp rx=0 tx=0).stop 2024-04-25T10:30:38.569 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:38.562+0000 7fe7b0959280 1 -- 172.21.15.37:0/3307730207 shutdown_connections 2024-04-25T10:30:38.569 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:38.562+0000 7fe7b0959280 1 --2- 172.21.15.37:0/3307730207 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55ba2b1f3200 0x55ba2a497b80 unknown :-1 s=CLOSED pgs=557 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:30:38.569 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:38.562+0000 7fe7b0959280 1 --2- 172.21.15.37:0/3307730207 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x55ba2b1f2d80 0x55ba2a497600 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:30:38.569 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:38.562+0000 7fe7b0959280 1 --2- 172.21.15.37:0/3307730207 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x55ba2b1f2900 0x55ba2a497080 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:30:38.569 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:38.562+0000 7fe7b0959280 1 -- 172.21.15.37:0/3307730207 >> 172.21.15.37:0/3307730207 conn(0x55ba2b1f2480 msgr2=0x55ba2a50c000 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:30:38.569 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:38.562+0000 7fe7b0959280 1 -- 172.21.15.37:0/3307730207 shutdown_connections 2024-04-25T10:30:38.570 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:38.562+0000 7fe7b0959280 1 -- 172.21.15.37:0/3307730207 wait complete. 2024-04-25T10:30:38.570 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:38.562+0000 7fe7b0959280 1 Processor -- start 2024-04-25T10:30:38.570 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:38.562+0000 7fe7b0959280 1 -- start start 2024-04-25T10:30:38.570 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:38.563+0000 7fe7b0959280 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x55ba2b1f2900 0x55ba2a497080 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:30:38.570 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:38.564+0000 7fe7b0959280 1 --2- >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x55ba2b1f2d80 0x55ba2a497600 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:30:38.570 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:38.564+0000 7fe7b0959280 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55ba2b1f3200 0x55ba2a497b80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:30:38.570 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:38.564+0000 7fe7b0959280 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 -- 0x55ba2a49ec00 con 0x55ba2b1f3200 2024-04-25T10:30:38.570 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:38.564+0000 7fe7b0959280 1 -- --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_getmap magic: 0 -- 0x55ba2a49ea80 con 0x55ba2b1f2900 2024-04-25T10:30:38.570 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:38.564+0000 7fe7b0959280 1 -- --> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] -- mon_getmap magic: 0 -- 0x55ba2a49f380 con 0x55ba2b1f2d80 2024-04-25T10:30:38.570 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:38.564+0000 7fe7ae904640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55ba2b1f3200 0x55ba2a497b80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:30:38.570 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:38.564+0000 7fe7ae904640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55ba2b1f3200 0x55ba2a497b80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.37:3300/0 says I am v2:172.21.15.37:55638/0 (socket says 172.21.15.37:55638) 2024-04-25T10:30:38.570 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:38.564+0000 7fe7ae904640 1 -- 172.21.15.37:0/3661016806 learned_addr learned my addr 172.21.15.37:0/3661016806 (peer_addr_for_me v2:172.21.15.37:0/0) 2024-04-25T10:30:38.570 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:38.564+0000 7fe7af105640 1 --2- 172.21.15.37:0/3661016806 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x55ba2b1f2d80 0x55ba2a497600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:30:38.570 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:38.564+0000 7fe7af906640 1 --2- 172.21.15.37:0/3661016806 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x55ba2b1f2900 0x55ba2a497080 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:30:38.571 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:38.564+0000 7fe7ae904640 1 -- 172.21.15.37:0/3661016806 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x55ba2b1f2d80 msgr2=0x55ba2a497600 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:30:38.571 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:38.564+0000 7fe7ae904640 1 --2- 172.21.15.37:0/3661016806 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x55ba2b1f2d80 0x55ba2a497600 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:30:38.571 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:38.564+0000 7fe7ae904640 1 -- 172.21.15.37:0/3661016806 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x55ba2b1f2900 msgr2=0x55ba2a497080 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:30:38.571 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:38.564+0000 7fe7ae904640 1 --2- 172.21.15.37:0/3661016806 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x55ba2b1f2900 0x55ba2a497080 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:30:38.571 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:38.564+0000 7fe7ae904640 1 -- 172.21.15.37:0/3661016806 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55ba263e8960 con 0x55ba2b1f3200 2024-04-25T10:30:38.571 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:38.564+0000 7fe7ae904640 1 --2- 172.21.15.37:0/3661016806 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55ba2b1f3200 0x55ba2a497b80 secure :-1 s=READY pgs=558 cs=0 l=1 rev1=1 crypto rx=0x55ba2b1f0b70 tx=0x55ba2a4de9c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:30:38.571 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:38.564+0000 7fe7a90f9640 1 -- 172.21.15.37:0/3661016806 <== mon.0 v2:172.21.15.37:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x55ba263d8b60 con 0x55ba2b1f3200 2024-04-25T10:30:38.571 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:38.564+0000 7fe7a90f9640 1 -- 172.21.15.37:0/3661016806 <== mon.0 v2:172.21.15.37:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x55ba263dec40 con 0x55ba2b1f3200 2024-04-25T10:30:38.571 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:38.564+0000 7fe7b0959280 1 -- 172.21.15.37:0/3661016806 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x55ba263e9680 con 0x55ba2b1f3200 2024-04-25T10:30:38.571 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:38.564+0000 7fe7a90f9640 1 -- 172.21.15.37:0/3661016806 <== mon.0 v2:172.21.15.37:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x55ba263d8820 con 0x55ba2b1f3200 2024-04-25T10:30:38.571 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:38.564+0000 7fe7b0959280 1 -- 172.21.15.37:0/3661016806 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55ba263e9860 con 0x55ba2b1f3200 2024-04-25T10:30:38.571 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:38.564+0000 7fe7b0959280 4 main: resolving pool 6 2024-04-25T10:30:38.571 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:38.566+0000 7fe7a90f9640 1 -- 172.21.15.37:0/3661016806 <== mon.0 v2:172.21.15.37:3300/0 4 ==== mgrmap(e 5) ==== 96782+0+0 (secure 0 0 0) 0x55ba2b30c900 con 0x55ba2b1f3200 2024-04-25T10:30:38.573 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:38.566+0000 7fe7a90f9640 1 --2- 172.21.15.37:0/3661016806 >> [v2:172.21.15.203:6832/2418205237,v1:172.21.15.203:6833/2418205237] conn(0x55ba2b1f5600 0x55ba2a498100 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:30:38.573 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:38.566+0000 7fe7a90f9640 1 -- 172.21.15.37:0/3661016806 <== mon.0 v2:172.21.15.37:3300/0 5 ==== osd_map(44..44 src has 1..44) ==== 6248+0+0 (secure 0 0 0) 0x55ba2b31ac00 con 0x55ba2b1f3200 2024-04-25T10:30:38.573 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:38.566+0000 7fe7b0959280 4 main: creating IoCtx.. 2024-04-25T10:30:38.573 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:38.566+0000 7fe7b0959280 1 --2- 172.21.15.37:0/3661016806 >> [v2:172.21.15.203:6808/3688099014,v1:172.21.15.203:6809/3688099014] conn(0x55ba2b1f3680 0x55ba2a498680 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:30:38.573 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:38.566+0000 7fe7b0959280 1 -- 172.21.15.37:0/3661016806 --> [v2:172.21.15.203:6808/3688099014,v1:172.21.15.203:6809/3688099014] -- osd_op(unknown.0.0:1 6.13 6:c91d4a1d:::mds0_inotable:head [writefull 0~34 in=34b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e44) -- 0x55ba2b331800 con 0x55ba2b1f3680 2024-04-25T10:30:38.573 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:38.566+0000 7fe7af906640 1 --2- 172.21.15.37:0/3661016806 >> [v2:172.21.15.203:6832/2418205237,v1:172.21.15.203:6833/2418205237] conn(0x55ba2b1f5600 0x55ba2a498100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:30:38.573 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:38.566+0000 7fe7af105640 1 --2- 172.21.15.37:0/3661016806 >> [v2:172.21.15.203:6808/3688099014,v1:172.21.15.203:6809/3688099014] conn(0x55ba2b1f3680 0x55ba2a498680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:30:38.573 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:38.567+0000 7fe7af906640 1 --2- 172.21.15.37:0/3661016806 >> [v2:172.21.15.203:6832/2418205237,v1:172.21.15.203:6833/2418205237] conn(0x55ba2b1f5600 0x55ba2a498100 secure :-1 s=READY pgs=264 cs=0 l=1 rev1=1 crypto rx=0x55ba2a524840 tx=0x55ba2b2d4d80 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:30:38.573 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:38.567+0000 7fe7af105640 1 --2- 172.21.15.37:0/3661016806 >> [v2:172.21.15.203:6808/3688099014,v1:172.21.15.203:6809/3688099014] conn(0x55ba2b1f3680 0x55ba2a498680 crc :-1 s=READY pgs=14 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.7 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:30:38.573 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:38.568+0000 7fe7af105640 1 -- 172.21.15.37:0/3661016806 <== osd.7 v2:172.21.15.203:6808/3688099014 1 ==== osd_op_reply(1 mds0_inotable [writefull 0~34] v44'2 uv2 ondisk = 0) ==== 157+0+0 (crc 0 0 0) 0x55ba2a4c8000 con 0x55ba2b1f3680 2024-04-25T10:30:38.573 INFO:teuthology.orchestra.run.smithi037.stdout:{ 2024-04-25T10:30:38.574 INFO:teuthology.orchestra.run.smithi037.stdout: "0": { 2024-04-25T10:30:38.574 INFO:teuthology.orchestra.run.smithi037.stdout: "data": {}, 2024-04-25T10:30:38.574 INFO:teuthology.orchestra.run.smithi037.stdout: "result": 0 2024-04-25T10:30:38.574 INFO:teuthology.orchestra.run.smithi037.stdout: } 2024-04-25T10:30:38.574 INFO:teuthology.orchestra.run.smithi037.stdout:} 2024-04-25T10:30:38.574 INFO:teuthology.orchestra.run.smithi037.stdout: 2024-04-25T10:30:38.576 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:38.569+0000 7fe7b0959280 1 -- 172.21.15.37:0/3661016806 >> [v2:172.21.15.203:6808/3688099014,v1:172.21.15.203:6809/3688099014] conn(0x55ba2b1f3680 msgr2=0x55ba2a498680 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:30:38.576 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:38.569+0000 7fe7b0959280 1 --2- 172.21.15.37:0/3661016806 >> [v2:172.21.15.203:6808/3688099014,v1:172.21.15.203:6809/3688099014] conn(0x55ba2b1f3680 0x55ba2a498680 crc :-1 s=READY pgs=14 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:30:38.576 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:38.569+0000 7fe7b0959280 1 -- 172.21.15.37:0/3661016806 >> [v2:172.21.15.203:6832/2418205237,v1:172.21.15.203:6833/2418205237] conn(0x55ba2b1f5600 msgr2=0x55ba2a498100 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:30:38.576 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:38.569+0000 7fe7b0959280 1 --2- 172.21.15.37:0/3661016806 >> [v2:172.21.15.203:6832/2418205237,v1:172.21.15.203:6833/2418205237] conn(0x55ba2b1f5600 0x55ba2a498100 secure :-1 s=READY pgs=264 cs=0 l=1 rev1=1 crypto rx=0x55ba2a524840 tx=0x55ba2b2d4d80 comp rx=0 tx=0).stop 2024-04-25T10:30:38.576 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:38.569+0000 7fe7b0959280 1 -- 172.21.15.37:0/3661016806 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55ba2b1f3200 msgr2=0x55ba2a497b80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:30:38.576 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:38.569+0000 7fe7b0959280 1 --2- 172.21.15.37:0/3661016806 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55ba2b1f3200 0x55ba2a497b80 secure :-1 s=READY pgs=558 cs=0 l=1 rev1=1 crypto rx=0x55ba2b1f0b70 tx=0x55ba2a4de9c0 comp rx=0 tx=0).stop 2024-04-25T10:30:38.576 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:38.569+0000 7fe7af105640 1 -- 172.21.15.37:0/3661016806 reap_dead start 2024-04-25T10:30:38.576 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:38.569+0000 7fe7b0959280 1 -- 172.21.15.37:0/3661016806 shutdown_connections 2024-04-25T10:30:38.576 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:38.569+0000 7fe7b0959280 1 -- 172.21.15.37:0/3661016806 >> 172.21.15.37:0/3661016806 conn(0x55ba2b1f2480 msgr2=0x55ba2a50c000 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:30:38.576 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:38.569+0000 7fe7b0959280 1 -- 172.21.15.37:0/3661016806 shutdown_connections 2024-04-25T10:30:38.576 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:38.569+0000 7fe7b0959280 1 -- 172.21.15.37:0/3661016806 wait complete. 2024-04-25T10:30:38.577 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:38.569+0000 7fe7b0959280 1 -- 172.21.15.37:0/574161655 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55ba2a579180 msgr2=0x55ba2a496580 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:30:38.577 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:38.569+0000 7fe7b0959280 1 --2- 172.21.15.37:0/574161655 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55ba2a579180 0x55ba2a496580 secure :-1 s=READY pgs=556 cs=0 l=1 rev1=1 crypto rx=0x55ba2b2193b0 tx=0x55ba2a4de8a0 comp rx=0 tx=0).stop 2024-04-25T10:30:38.577 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:38.569+0000 7fe7b0959280 1 -- 172.21.15.37:0/574161655 shutdown_connections 2024-04-25T10:30:38.577 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:38.569+0000 7fe7b0959280 1 --2- 172.21.15.37:0/574161655 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x55ba2b1f2000 0x55ba2a496b00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:30:38.577 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:38.569+0000 7fe7b0959280 1 --2- 172.21.15.37:0/574161655 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55ba2a579180 0x55ba2a496580 unknown :-1 s=CLOSED pgs=556 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:30:38.577 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:38.569+0000 7fe7b0959280 1 --2- 172.21.15.37:0/574161655 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x55ba2a579a80 0x55ba2a496000 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:30:38.577 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:38.569+0000 7fe7b0959280 1 -- 172.21.15.37:0/574161655 >> 172.21.15.37:0/574161655 conn(0x55ba2a579600 msgr2=0x55ba2a50d000 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:30:38.577 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:38.569+0000 7fe7b0959280 1 -- 172.21.15.37:0/574161655 shutdown_connections 2024-04-25T10:30:38.577 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:38.569+0000 7fe7b0959280 1 -- 172.21.15.37:0/574161655 wait complete. 2024-04-25T10:30:38.577 DEBUG:tasks.cephfs.filesystem:Ran ['cephfs-table-tool', '--debug-mds=20', '--debug-ms=1', '--debug-objecter=1', 'recovery_fs:0', 'reset', 'inode'] in time 0:00:00.070809, result: { "0": { "data": {}, "result": 0 } } 2024-04-25T10:30:38.577 DEBUG:teuthology.orchestra.run.smithi037:> cephfs-data-scan --debug-mds=20 --debug-ms=1 --debug-objecter=1 scan_extents --alternate-pool recovery_fs_metadata --filesystem cephfs cephfs_data 2024-04-25T10:30:38.633 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:38.630+0000 7f7ba022e740 1 -- start start 2024-04-25T10:30:38.633 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:38.630+0000 7f7ba022e740 1 Processor -- start 2024-04-25T10:30:38.636 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:38.632+0000 7f7ba022e740 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x5633871ab680 0x563387188b00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:30:38.636 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:38.632+0000 7f7ba022e740 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x5633871abb00 0x563387188580 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:30:38.636 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:38.632+0000 7f7ba022e740 1 --2- >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x5633871abf80 0x563387188000 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:30:38.636 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:38.632+0000 7f7ba022e740 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 -- 0x5633870d0180 con 0x5633871abb00 2024-04-25T10:30:38.636 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:38.632+0000 7f7ba022e740 1 -- --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_getmap magic: 0 -- 0x563387072900 con 0x5633871ab680 2024-04-25T10:30:38.636 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:38.632+0000 7f7ba022e740 1 -- --> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] -- mon_getmap magic: 0 -- 0x563387072480 con 0x5633871abf80 2024-04-25T10:30:38.636 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:38.632+0000 7f7b9d33e640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x5633871abb00 0x563387188580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:30:38.636 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:38.632+0000 7f7b9d33e640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x5633871abb00 0x563387188580 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.37:3300/0 says I am v2:172.21.15.37:55654/0 (socket says 172.21.15.37:55654) 2024-04-25T10:30:38.636 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:38.632+0000 7f7b9d33e640 1 -- 172.21.15.37:0/1847905238 learned_addr learned my addr 172.21.15.37:0/1847905238 (peer_addr_for_me v2:172.21.15.37:0/0) 2024-04-25T10:30:38.636 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:38.632+0000 7f7b9e340640 1 --2- 172.21.15.37:0/1847905238 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x5633871abf80 0x563387188000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:30:38.637 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:38.632+0000 7f7b9db3f640 1 --2- 172.21.15.37:0/1847905238 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x5633871ab680 0x563387188b00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:30:38.637 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:38.632+0000 7f7b9d33e640 1 -- 172.21.15.37:0/1847905238 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x5633871abf80 msgr2=0x563387188000 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:30:38.637 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:38.632+0000 7f7b9d33e640 1 --2- 172.21.15.37:0/1847905238 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x5633871abf80 0x563387188000 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:30:38.637 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:38.632+0000 7f7b9d33e640 1 -- 172.21.15.37:0/1847905238 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x5633871ab680 msgr2=0x563387188b00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:30:38.637 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:38.632+0000 7f7b9d33e640 1 --2- 172.21.15.37:0/1847905238 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x5633871ab680 0x563387188b00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:30:38.637 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:38.632+0000 7f7b9d33e640 1 -- 172.21.15.37:0/1847905238 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x56338300f680 con 0x5633871abb00 2024-04-25T10:30:38.637 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:38.632+0000 7f7b9d33e640 1 --2- 172.21.15.37:0/1847905238 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x5633871abb00 0x563387188580 secure :-1 s=READY pgs=560 cs=0 l=1 rev1=1 crypto rx=0x563387e49440 tx=0x563387112c60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f1a4a76e1b4979cc server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:30:38.637 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:38.632+0000 7f7b9b33a640 1 -- 172.21.15.37:0/1847905238 <== mon.0 v2:172.21.15.37:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x5633830089c0 con 0x5633871abb00 2024-04-25T10:30:38.637 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:38.632+0000 7f7b9b33a640 1 -- 172.21.15.37:0/1847905238 <== mon.0 v2:172.21.15.37:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x56338705a1c0 con 0x5633871abb00 2024-04-25T10:30:38.637 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:38.632+0000 7f7ba022e740 1 -- 172.21.15.37:0/1847905238 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({osdmap=0}) -- 0x56338300f860 con 0x5633871abb00 2024-04-25T10:30:38.637 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:38.632+0000 7f7ba022e740 1 -- 172.21.15.37:0/1847905238 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({fsmap=0}) -- 0x56338300f2c0 con 0x5633871abb00 2024-04-25T10:30:38.637 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:38.633+0000 7f7ba022e740 4 waiting for MDS map... 2024-04-25T10:30:38.637 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:38.633+0000 7f7b9b33a640 1 -- 172.21.15.37:0/1847905238 <== mon.0 v2:172.21.15.37:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x563383008680 con 0x5633871abb00 2024-04-25T10:30:38.638 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:38.633+0000 7f7b9b33a640 1 -- 172.21.15.37:0/1847905238 <== mon.0 v2:172.21.15.37:3300/0 4 ==== osd_map(44..44 src has 1..44) ==== 6248+0+0 (secure 0 0 0) 0x563387dfc200 con 0x5633871abb00 2024-04-25T10:30:38.638 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:38.633+0000 7f7b9b33a640 1 -- 172.21.15.37:0/1847905238 <== mon.0 v2:172.21.15.37:3300/0 5 ==== fsmap(e 22) ==== 4135+0+0 (secure 0 0 0) 0x56338706b180 con 0x5633871abb00 2024-04-25T10:30:38.638 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:38.633+0000 7f7ba022e740 4 Got MDS map 22 2024-04-25T10:30:38.638 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:38.633+0000 7f7ba022e740 4 datascan.main: Using metadata pool output 2024-04-25T10:30:38.638 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:38.633+0000 7f7ba022e740 4 datascan.main: connecting to RADOS... 2024-04-25T10:30:38.638 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:38.634+0000 7f7ba022e740 1 Processor -- start 2024-04-25T10:30:38.638 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:38.634+0000 7f7ba022e740 1 -- start start 2024-04-25T10:30:38.643 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:38.635+0000 7f7ba022e740 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x5633871aad80 0x563387189080 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:30:38.643 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:38.635+0000 7f7ba022e740 1 --2- >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x5633871aa900 0x563387189600 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:30:38.643 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:38.635+0000 7f7ba022e740 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x5633871ac880 0x563387189b80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:30:38.643 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:38.635+0000 7f7ba022e740 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 -- 0x563387072180 con 0x5633871ac880 2024-04-25T10:30:38.643 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:38.635+0000 7f7ba022e740 1 -- --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_getmap magic: 0 -- 0x5633870d1500 con 0x5633871aad80 2024-04-25T10:30:38.643 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:38.635+0000 7f7ba022e740 1 -- --> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] -- mon_getmap magic: 0 -- 0x5633870d1380 con 0x5633871aa900 2024-04-25T10:30:38.643 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:38.635+0000 7f7b9d33e640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x5633871ac880 0x563387189b80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:30:38.643 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:38.635+0000 7f7b9d33e640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x5633871ac880 0x563387189b80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.37:3300/0 says I am v2:172.21.15.37:55662/0 (socket says 172.21.15.37:55662) 2024-04-25T10:30:38.643 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:38.635+0000 7f7b9db3f640 1 --2- >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x5633871aa900 0x563387189600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:30:38.643 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:38.635+0000 7f7b9d33e640 1 -- 172.21.15.37:0/978484400 learned_addr learned my addr 172.21.15.37:0/978484400 (peer_addr_for_me v2:172.21.15.37:0/0) 2024-04-25T10:30:38.643 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:38.635+0000 7f7b9e340640 1 --2- 172.21.15.37:0/978484400 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x5633871aad80 0x563387189080 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:30:38.643 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:38.636+0000 7f7b9d33e640 1 -- 172.21.15.37:0/978484400 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x5633871aa900 msgr2=0x563387189600 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:30:38.643 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:38.636+0000 7f7b9d33e640 1 --2- 172.21.15.37:0/978484400 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x5633871aa900 0x563387189600 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:30:38.644 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:38.636+0000 7f7b9d33e640 1 -- 172.21.15.37:0/978484400 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x5633871aad80 msgr2=0x563387189080 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:30:38.644 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:38.636+0000 7f7b9d33e640 1 --2- 172.21.15.37:0/978484400 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x5633871aad80 0x563387189080 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:30:38.644 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:38.636+0000 7f7b9d33e640 1 -- 172.21.15.37:0/978484400 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x56338300f4a0 con 0x5633871ac880 2024-04-25T10:30:38.644 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:38.636+0000 7f7b9db3f640 1 --2- 172.21.15.37:0/978484400 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x5633871aa900 0x563387189600 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-25T10:30:38.644 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:38.636+0000 7f7b9d33e640 1 --2- 172.21.15.37:0/978484400 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x5633871ac880 0x563387189b80 secure :-1 s=READY pgs=561 cs=0 l=1 rev1=1 crypto rx=0x563387e0c840 tx=0x563387112600 comp rx=0 tx=0).ready entity=mon.0 client_cookie=82d922976f799668 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:30:38.644 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:38.636+0000 7f7b99336640 1 -- 172.21.15.37:0/978484400 <== mon.0 v2:172.21.15.37:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x563383008b60 con 0x5633871ac880 2024-04-25T10:30:38.644 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:38.636+0000 7f7b99336640 1 -- 172.21.15.37:0/978484400 <== mon.0 v2:172.21.15.37:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x56338705ae00 con 0x5633871ac880 2024-04-25T10:30:38.644 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:38.636+0000 7f7b99336640 1 -- 172.21.15.37:0/978484400 <== mon.0 v2:172.21.15.37:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x563383008820 con 0x5633871ac880 2024-04-25T10:30:38.644 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:38.636+0000 7f7ba022e740 1 -- 172.21.15.37:0/978484400 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x5633871ac880 msgr2=0x563387189b80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:30:38.644 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:38.636+0000 7f7ba022e740 1 --2- 172.21.15.37:0/978484400 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x5633871ac880 0x563387189b80 secure :-1 s=READY pgs=561 cs=0 l=1 rev1=1 crypto rx=0x563387e0c840 tx=0x563387112600 comp rx=0 tx=0).stop 2024-04-25T10:30:38.644 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:38.636+0000 7f7ba022e740 1 -- 172.21.15.37:0/978484400 shutdown_connections 2024-04-25T10:30:38.644 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:38.636+0000 7f7ba022e740 1 --2- 172.21.15.37:0/978484400 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x5633871ac880 0x563387189b80 unknown :-1 s=CLOSED pgs=561 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:30:38.644 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:38.636+0000 7f7ba022e740 1 --2- 172.21.15.37:0/978484400 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x5633871aa900 0x563387189600 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:30:38.644 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:38.636+0000 7f7ba022e740 1 --2- 172.21.15.37:0/978484400 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x5633871aad80 0x563387189080 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:30:38.644 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:38.636+0000 7f7ba022e740 1 -- 172.21.15.37:0/978484400 >> 172.21.15.37:0/978484400 conn(0x5633871ab200 msgr2=0x5633870f9800 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:30:38.645 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:38.637+0000 7f7ba022e740 1 -- 172.21.15.37:0/978484400 shutdown_connections 2024-04-25T10:30:38.645 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:38.637+0000 7f7ba022e740 1 -- 172.21.15.37:0/978484400 wait complete. 2024-04-25T10:30:38.645 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:38.637+0000 7f7ba022e740 1 Processor -- start 2024-04-25T10:30:38.645 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:38.637+0000 7f7ba022e740 1 -- start start 2024-04-25T10:30:38.645 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:38.638+0000 7f7ba022e740 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x5633871aa900 0x563387189600 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:30:38.645 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:38.639+0000 7f7ba022e740 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x5633871aad80 0x563387189080 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:30:38.645 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:38.639+0000 7f7ba022e740 1 --2- >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x5633871ac880 0x563387189b80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:30:38.645 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:38.639+0000 7f7ba022e740 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 -- 0x5633870d0a80 con 0x5633871aa900 2024-04-25T10:30:38.645 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:38.639+0000 7f7ba022e740 1 -- --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_getmap magic: 0 -- 0x5633870d1680 con 0x5633871aad80 2024-04-25T10:30:38.645 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:38.639+0000 7f7ba022e740 1 -- --> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] -- mon_getmap magic: 0 -- 0x5633870d0900 con 0x5633871ac880 2024-04-25T10:30:38.645 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:38.639+0000 7f7b9e340640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x5633871aa900 0x563387189600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:30:38.645 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:38.639+0000 7f7b9e340640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x5633871aa900 0x563387189600 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.37:3300/0 says I am v2:172.21.15.37:55668/0 (socket says 172.21.15.37:55668) 2024-04-25T10:30:38.645 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:38.639+0000 7f7b9e340640 1 -- 172.21.15.37:0/1699050071 learned_addr learned my addr 172.21.15.37:0/1699050071 (peer_addr_for_me v2:172.21.15.37:0/0) 2024-04-25T10:30:38.645 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:38.639+0000 7f7b9db3f640 1 --2- 172.21.15.37:0/1699050071 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x5633871aad80 0x563387189080 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:30:38.646 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:38.639+0000 7f7b9d33e640 1 --2- 172.21.15.37:0/1699050071 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x5633871ac880 0x563387189b80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:30:38.646 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:38.639+0000 7f7b9e340640 1 -- 172.21.15.37:0/1699050071 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x5633871ac880 msgr2=0x563387189b80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:30:38.646 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:38.639+0000 7f7b9e340640 1 --2- 172.21.15.37:0/1699050071 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x5633871ac880 0x563387189b80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:30:38.646 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:38.639+0000 7f7b9e340640 1 -- 172.21.15.37:0/1699050071 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x5633871aad80 msgr2=0x563387189080 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:30:38.646 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:38.639+0000 7f7b9e340640 1 --2- 172.21.15.37:0/1699050071 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x5633871aad80 0x563387189080 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:30:38.646 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:38.639+0000 7f7b9e340640 1 -- 172.21.15.37:0/1699050071 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x56338300f0e0 con 0x5633871aa900 2024-04-25T10:30:38.646 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:38.639+0000 7f7b9d33e640 1 --2- 172.21.15.37:0/1699050071 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x5633871ac880 0x563387189b80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T10:30:38.646 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:38.639+0000 7f7b9e340640 1 --2- 172.21.15.37:0/1699050071 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x5633871aa900 0x563387189600 secure :-1 s=READY pgs=562 cs=0 l=1 rev1=1 crypto rx=0x563387f6e6c0 tx=0x563387ee8c00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:30:38.646 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:38.639+0000 7f7b97b33640 1 -- 172.21.15.37:0/1699050071 <== mon.0 v2:172.21.15.37:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x563383008b60 con 0x5633871aa900 2024-04-25T10:30:38.646 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:38.639+0000 7f7b97b33640 1 -- 172.21.15.37:0/1699050071 <== mon.0 v2:172.21.15.37:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x56338705a540 con 0x5633871aa900 2024-04-25T10:30:38.646 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:38.639+0000 7f7ba022e740 1 -- 172.21.15.37:0/1699050071 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x56338300fa40 con 0x5633871aa900 2024-04-25T10:30:38.646 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:38.639+0000 7f7b97b33640 1 -- 172.21.15.37:0/1699050071 <== mon.0 v2:172.21.15.37:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x563383008820 con 0x5633871aa900 2024-04-25T10:30:38.646 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:38.639+0000 7f7ba022e740 1 -- 172.21.15.37:0/1699050071 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({osdmap=0}) -- 0x56338300e3c0 con 0x5633871aa900 2024-04-25T10:30:38.647 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:38.639+0000 7f7ba022e740 4 datascan.init: forcing metadata pool 'recovery_fs_metadata' 2024-04-25T10:30:38.647 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:38.641+0000 7f7b97b33640 1 -- 172.21.15.37:0/1699050071 <== mon.0 v2:172.21.15.37:3300/0 4 ==== mgrmap(e 5) ==== 96782+0+0 (secure 0 0 0) 0x563387f32900 con 0x5633871aa900 2024-04-25T10:30:38.653 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:38.641+0000 7f7b97b33640 1 --2- 172.21.15.37:0/1699050071 >> [v2:172.21.15.203:6832/2418205237,v1:172.21.15.203:6833/2418205237] conn(0x5633871ada80 0x56338718a100 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:30:38.653 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:38.641+0000 7f7b97b33640 1 -- 172.21.15.37:0/1699050071 <== mon.0 v2:172.21.15.37:3300/0 5 ==== osd_map(44..44 src has 1..44) ==== 6248+0+0 (secure 0 0 0) 0x563387f4b000 con 0x5633871aa900 2024-04-25T10:30:38.653 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:38.641+0000 7f7ba022e740 4 datascan.main: data pool 'cephfs_data' has ID 5 2024-04-25T10:30:38.653 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:38.641+0000 7f7ba022e740 4 datascan.main: opening data pool 'cephfs_data' 2024-04-25T10:30:38.654 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:38.641+0000 7f7ba022e740 1 --2- 172.21.15.37:0/1699050071 >> [v2:172.21.15.203:6800/4051532063,v1:172.21.15.203:6801/4051532063] conn(0x5633871acd00 0x56338718a680 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:30:38.654 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:38.642+0000 7f7ba022e740 1 -- 172.21.15.37:0/1699050071 --> [v2:172.21.15.203:6800/4051532063,v1:172.21.15.203:6801/4051532063] -- osd_op(unknown.0.0:1 5.0 5:00000000::::head [pgnls start_epoch 44 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e44) -- 0x563387f70c00 con 0x5633871acd00 2024-04-25T10:30:38.654 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:38.642+0000 7f7b9db3f640 1 --2- 172.21.15.37:0/1699050071 >> [v2:172.21.15.203:6832/2418205237,v1:172.21.15.203:6833/2418205237] conn(0x5633871ada80 0x56338718a100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:30:38.654 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:38.642+0000 7f7b9d33e640 1 --2- 172.21.15.37:0/1699050071 >> [v2:172.21.15.203:6800/4051532063,v1:172.21.15.203:6801/4051532063] conn(0x5633871acd00 0x56338718a680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:30:38.654 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:38.642+0000 7f7b9db3f640 1 --2- 172.21.15.37:0/1699050071 >> [v2:172.21.15.203:6832/2418205237,v1:172.21.15.203:6833/2418205237] conn(0x5633871ada80 0x56338718a100 secure :-1 s=READY pgs=265 cs=0 l=1 rev1=1 crypto rx=0x5633871150b0 tx=0x563387ee9140 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:30:38.654 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:38.642+0000 7f7b9d33e640 1 --2- 172.21.15.37:0/1699050071 >> [v2:172.21.15.203:6800/4051532063,v1:172.21.15.203:6801/4051532063] conn(0x5633871acd00 0x56338718a680 crc :-1 s=READY pgs=14 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.4 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:30:38.654 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:38.643+0000 7f7b9d33e640 1 -- 172.21.15.37:0/1699050071 <== osd.4 v2:172.21.15.203:6800/4051532063 1 ==== osd_op_reply(1 [pgnls start_epoch 44 out=49b] v36'8 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x5633870fa000 con 0x5633871acd00 2024-04-25T10:30:38.654 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:38.643+0000 7f7b98b35640 1 --2- 172.21.15.37:0/1699050071 >> [v2:172.21.15.37:6824/1815290618,v1:172.21.15.37:6825/1815290618] conn(0x56338741e000 0x563387422000 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:30:38.654 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:38.643+0000 7f7b98b35640 1 -- 172.21.15.37:0/1699050071 --> [v2:172.21.15.37:6824/1815290618,v1:172.21.15.37:6825/1815290618] -- osd_op(unknown.0.0:2 5.20 5:04000000::::head [pgnls start_epoch 44 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e44) -- 0x563387426000 con 0x56338741e000 2024-04-25T10:30:38.654 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:38.643+0000 7f7b9e340640 1 --2- 172.21.15.37:0/1699050071 >> [v2:172.21.15.37:6824/1815290618,v1:172.21.15.37:6825/1815290618] conn(0x56338741e000 0x563387422000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:30:38.654 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:38.643+0000 7f7b9e340640 1 --2- 172.21.15.37:0/1699050071 >> [v2:172.21.15.37:6824/1815290618,v1:172.21.15.37:6825/1815290618] conn(0x56338741e000 0x563387422000 crc :-1 s=READY pgs=14 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.3 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:30:38.654 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:38.643+0000 7f7b9e340640 1 -- 172.21.15.37:0/1699050071 <== osd.3 v2:172.21.15.37:6824/1815290618 1 ==== osd_op_reply(2 [pgnls start_epoch 44 out=49b] v36'9 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x5633870fa500 con 0x56338741e000 2024-04-25T10:30:38.654 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:38.643+0000 7f7b99336640 1 --2- 172.21.15.37:0/1699050071 >> [v2:172.21.15.37:6808/2754669649,v1:172.21.15.37:6809/2754669649] conn(0x56338741e480 0x563387422580 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:30:38.654 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:38.643+0000 7f7b99336640 1 -- 172.21.15.37:0/1699050071 --> [v2:172.21.15.37:6808/2754669649,v1:172.21.15.37:6809/2754669649] -- osd_op(unknown.0.0:3 5.10 5:08000000::::head [pgnls start_epoch 44 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e44) -- 0x563387426400 con 0x56338741e480 2024-04-25T10:30:38.655 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:38.644+0000 7f7b9db3f640 1 --2- 172.21.15.37:0/1699050071 >> [v2:172.21.15.37:6808/2754669649,v1:172.21.15.37:6809/2754669649] conn(0x56338741e480 0x563387422580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:30:38.655 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:38.644+0000 7f7b9db3f640 1 --2- 172.21.15.37:0/1699050071 >> [v2:172.21.15.37:6808/2754669649,v1:172.21.15.37:6809/2754669649] conn(0x56338741e480 0x563387422580 crc :-1 s=READY pgs=13 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:30:38.655 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:38.644+0000 7f7b9db3f640 1 -- 172.21.15.37:0/1699050071 <== osd.2 v2:172.21.15.37:6808/2754669649 1 ==== osd_op_reply(3 [pgnls start_epoch 44 out=49b] v36'6 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x5633870fa780 con 0x56338741e480 2024-04-25T10:30:38.655 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:38.644+0000 7f7b98b35640 1 --2- 172.21.15.37:0/1699050071 >> [v2:172.21.15.37:6816/2746074720,v1:172.21.15.37:6817/2746074720] conn(0x56338741e900 0x563387422b00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:30:38.655 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:38.644+0000 7f7b98b35640 1 -- 172.21.15.37:0/1699050071 --> [v2:172.21.15.37:6816/2746074720,v1:172.21.15.37:6817/2746074720] -- osd_op(unknown.0.0:4 5.30 5:0c000000::::head [pgnls start_epoch 44 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e44) -- 0x563387426800 con 0x56338741e900 2024-04-25T10:30:38.655 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:38.644+0000 7f7b9d33e640 1 --2- 172.21.15.37:0/1699050071 >> [v2:172.21.15.37:6816/2746074720,v1:172.21.15.37:6817/2746074720] conn(0x56338741e900 0x563387422b00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:30:38.655 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:38.645+0000 7f7b9d33e640 1 --2- 172.21.15.37:0/1699050071 >> [v2:172.21.15.37:6816/2746074720,v1:172.21.15.37:6817/2746074720] conn(0x56338741e900 0x563387422b00 crc :-1 s=READY pgs=13 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:30:38.655 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:38.645+0000 7f7b9d33e640 1 -- 172.21.15.37:0/1699050071 <== osd.0 v2:172.21.15.37:6816/2746074720 1 ==== osd_op_reply(4 [pgnls start_epoch 44 out=49b] v36'9 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x5633870fa000 con 0x56338741e900 2024-04-25T10:30:38.655 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:38.645+0000 7f7b99336640 1 --2- 172.21.15.37:0/1699050071 >> [v2:172.21.15.203:6810/2309615014,v1:172.21.15.203:6812/2309615014] conn(0x56338741f200 0x563387423600 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:30:38.655 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:38.645+0000 7f7b99336640 1 -- 172.21.15.37:0/1699050071 --> [v2:172.21.15.203:6810/2309615014,v1:172.21.15.203:6812/2309615014] -- osd_op(unknown.0.0:5 5.8 5:10000000::::head [pgnls start_epoch 44 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e44) -- 0x563387427000 con 0x56338741f200 2024-04-25T10:30:38.655 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:38.645+0000 7f7b9e340640 1 --2- 172.21.15.37:0/1699050071 >> [v2:172.21.15.203:6810/2309615014,v1:172.21.15.203:6812/2309615014] conn(0x56338741f200 0x563387423600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:30:38.655 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:38.646+0000 7f7b9e340640 1 --2- 172.21.15.37:0/1699050071 >> [v2:172.21.15.203:6810/2309615014,v1:172.21.15.203:6812/2309615014] conn(0x56338741f200 0x563387423600 crc :-1 s=READY pgs=13 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.6 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:30:38.655 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:38.646+0000 7f7b9e340640 1 -- 172.21.15.37:0/1699050071 <== osd.6 v2:172.21.15.203:6810/2309615014 1 ==== osd_op_reply(5 [pgnls start_epoch 44 out=49b] v36'9 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x5633870fa500 con 0x56338741f200 2024-04-25T10:30:38.655 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:38.646+0000 7f7b98b35640 1 -- 172.21.15.37:0/1699050071 --> [v2:172.21.15.37:6808/2754669649,v1:172.21.15.37:6809/2754669649] -- osd_op(unknown.0.0:6 5.28 5:14000000::::head [pgnls start_epoch 44 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e44) -- 0x563387426c00 con 0x56338741e480 2024-04-25T10:30:38.656 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:38.646+0000 7f7b9db3f640 1 -- 172.21.15.37:0/1699050071 <== osd.2 v2:172.21.15.37:6808/2754669649 2 ==== osd_op_reply(6 [pgnls start_epoch 44 out=49b] v36'6 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x5633870fa780 con 0x56338741e480 2024-04-25T10:30:38.656 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:38.646+0000 7f7b99336640 1 --2- 172.21.15.37:0/1699050071 >> [v2:172.21.15.37:6800/1690341662,v1:172.21.15.37:6801/1690341662] conn(0x56338741f680 0x563387423b80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:30:38.656 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:38.646+0000 7f7b99336640 1 -- 172.21.15.37:0/1699050071 --> [v2:172.21.15.37:6800/1690341662,v1:172.21.15.37:6801/1690341662] -- osd_op(unknown.0.0:7 5.18 5:18000000::::head [pgnls start_epoch 44 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e44) -- 0x563387427400 con 0x56338741f680 2024-04-25T10:30:38.656 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:38.646+0000 7f7b9db3f640 1 --2- 172.21.15.37:0/1699050071 >> [v2:172.21.15.37:6800/1690341662,v1:172.21.15.37:6801/1690341662] conn(0x56338741f680 0x563387423b80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:30:38.656 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:38.647+0000 7f7b9db3f640 1 --2- 172.21.15.37:0/1699050071 >> [v2:172.21.15.37:6800/1690341662,v1:172.21.15.37:6801/1690341662] conn(0x56338741f680 0x563387423b80 crc :-1 s=READY pgs=15 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:30:38.656 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:38.647+0000 7f7b9db3f640 1 -- 172.21.15.37:0/1699050071 <== osd.1 v2:172.21.15.37:6800/1690341662 1 ==== osd_op_reply(7 [pgnls start_epoch 44 out=49b] v36'8 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x5633870fa780 con 0x56338741f680 2024-04-25T10:30:38.656 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:38.647+0000 7f7b98b35640 1 -- 172.21.15.37:0/1699050071 --> [v2:172.21.15.37:6808/2754669649,v1:172.21.15.37:6809/2754669649] -- osd_op(unknown.0.0:8 5.38 5:1c000000::::head [pgnls start_epoch 44 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e44) -- 0x563387426400 con 0x56338741e480 2024-04-25T10:30:38.656 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:38.647+0000 7f7b9db3f640 1 -- 172.21.15.37:0/1699050071 <== osd.2 v2:172.21.15.37:6808/2754669649 3 ==== osd_op_reply(8 [pgnls start_epoch 44 out=49b] v36'9 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x5633870fa780 con 0x56338741e480 2024-04-25T10:30:38.656 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:38.647+0000 7f7b99336640 1 -- 172.21.15.37:0/1699050071 --> [v2:172.21.15.37:6816/2746074720,v1:172.21.15.37:6817/2746074720] -- osd_op(unknown.0.0:9 5.4 5:20000000::::head [pgnls start_epoch 44 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e44) -- 0x563387427400 con 0x56338741e900 2024-04-25T10:30:38.656 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:38.648+0000 7f7b9d33e640 1 -- 172.21.15.37:0/1699050071 <== osd.0 v2:172.21.15.37:6816/2746074720 2 ==== osd_op_reply(9 [pgnls start_epoch 44 out=49b] v36'9 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x5633870fa000 con 0x56338741e900 2024-04-25T10:30:38.656 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:38.648+0000 7f7b98b35640 1 -- 172.21.15.37:0/1699050071 --> [v2:172.21.15.37:6808/2754669649,v1:172.21.15.37:6809/2754669649] -- osd_op(unknown.0.0:10 5.24 5:24000000::::head [pgnls start_epoch 44 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e44) -- 0x563387426c00 con 0x56338741e480 2024-04-25T10:30:38.656 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:38.648+0000 7f7b9db3f640 1 -- 172.21.15.37:0/1699050071 <== osd.2 v2:172.21.15.37:6808/2754669649 4 ==== osd_op_reply(10 [pgnls start_epoch 44 out=49b] v36'4 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x5633870fa780 con 0x56338741e480 2024-04-25T10:30:38.656 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:38.648+0000 7f7b99336640 1 -- 172.21.15.37:0/1699050071 --> [v2:172.21.15.203:6800/4051532063,v1:172.21.15.203:6801/4051532063] -- osd_op(unknown.0.0:11 5.14 5:28000000::::head [pgnls start_epoch 44 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e44) -- 0x563387427800 con 0x5633871acd00 2024-04-25T10:30:38.657 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:38.648+0000 7f7b9d33e640 1 -- 172.21.15.37:0/1699050071 <== osd.4 v2:172.21.15.203:6800/4051532063 2 ==== osd_op_reply(11 [pgnls start_epoch 44 out=49b] v36'11 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x5633870fa000 con 0x5633871acd00 2024-04-25T10:30:38.657 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:38.648+0000 7f7b98b35640 1 -- 172.21.15.37:0/1699050071 --> [v2:172.21.15.37:6800/1690341662,v1:172.21.15.37:6801/1690341662] -- osd_op(unknown.0.0:12 5.34 5:2c000000::::head [pgnls start_epoch 44 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e44) -- 0x563387f71c00 con 0x56338741f680 2024-04-25T10:30:38.657 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:38.648+0000 7f7b9db3f640 1 -- 172.21.15.37:0/1699050071 <== osd.1 v2:172.21.15.37:6800/1690341662 2 ==== osd_op_reply(12 [pgnls start_epoch 44 out=49b] v36'7 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x5633870fa780 con 0x56338741f680 2024-04-25T10:30:38.657 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:38.649+0000 7f7b99336640 1 --2- 172.21.15.37:0/1699050071 >> [v2:172.21.15.203:6824/1203822890,v1:172.21.15.203:6825/1203822890] conn(0x56338741fb00 0x563387424100 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:30:38.657 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:38.649+0000 7f7b99336640 1 -- 172.21.15.37:0/1699050071 --> [v2:172.21.15.203:6824/1203822890,v1:172.21.15.203:6825/1203822890] -- osd_op(unknown.0.0:13 5.c 5:30000000::::head [pgnls start_epoch 44 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e44) -- 0x563387427c00 con 0x56338741fb00 2024-04-25T10:30:38.657 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:38.649+0000 7f7b9d33e640 1 --2- 172.21.15.37:0/1699050071 >> [v2:172.21.15.203:6824/1203822890,v1:172.21.15.203:6825/1203822890] conn(0x56338741fb00 0x563387424100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:30:38.657 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:38.649+0000 7f7b9d33e640 1 --2- 172.21.15.37:0/1699050071 >> [v2:172.21.15.203:6824/1203822890,v1:172.21.15.203:6825/1203822890] conn(0x56338741fb00 0x563387424100 crc :-1 s=READY pgs=13 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.5 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:30:38.657 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:38.649+0000 7f7b9d33e640 1 -- 172.21.15.37:0/1699050071 <== osd.5 v2:172.21.15.203:6824/1203822890 1 ==== osd_op_reply(13 [pgnls start_epoch 44 out=49b] v36'6 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x5633870fa000 con 0x56338741fb00 2024-04-25T10:30:38.657 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:38.649+0000 7f7b98b35640 1 -- 172.21.15.37:0/1699050071 --> [v2:172.21.15.203:6810/2309615014,v1:172.21.15.203:6812/2309615014] -- osd_op(unknown.0.0:14 5.2c 5:34000000::::head [pgnls start_epoch 44 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e44) -- 0x563387426800 con 0x56338741f200 2024-04-25T10:30:38.657 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:38.650+0000 7f7b9e340640 1 -- 172.21.15.37:0/1699050071 <== osd.6 v2:172.21.15.203:6810/2309615014 2 ==== osd_op_reply(14 [pgnls start_epoch 44 out=49b] v36'8 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x5633870fa500 con 0x56338741f200 2024-04-25T10:30:38.657 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:38.650+0000 7f7b99336640 1 -- 172.21.15.37:0/1699050071 --> [v2:172.21.15.37:6808/2754669649,v1:172.21.15.37:6809/2754669649] -- osd_op(unknown.0.0:15 5.1c 5:38000000::::head [pgnls start_epoch 44 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e44) -- 0x563387427400 con 0x56338741e480 2024-04-25T10:30:38.657 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:38.650+0000 7f7b9db3f640 1 -- 172.21.15.37:0/1699050071 <== osd.2 v2:172.21.15.37:6808/2754669649 5 ==== osd_op_reply(15 [pgnls start_epoch 44 out=49b] v36'9 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x5633870fa780 con 0x56338741e480 2024-04-25T10:30:38.657 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:38.650+0000 7f7b98b35640 1 -- 172.21.15.37:0/1699050071 --> [v2:172.21.15.203:6800/4051532063,v1:172.21.15.203:6801/4051532063] -- osd_op(unknown.0.0:16 5.3c 5:3c000000::::head [pgnls start_epoch 44 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e44) -- 0x563387f70c00 con 0x5633871acd00 2024-04-25T10:30:38.657 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:38.650+0000 7f7b9d33e640 1 -- 172.21.15.37:0/1699050071 <== osd.4 v2:172.21.15.203:6800/4051532063 3 ==== osd_op_reply(16 [pgnls start_epoch 44 out=49b] v36'8 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x5633870fa000 con 0x5633871acd00 2024-04-25T10:30:38.658 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:38.650+0000 7f7b99336640 1 -- 172.21.15.37:0/1699050071 --> [v2:172.21.15.37:6816/2746074720,v1:172.21.15.37:6817/2746074720] -- osd_op(unknown.0.0:17 5.2 5:40000000::::head [pgnls start_epoch 44 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e44) -- 0x563387427800 con 0x56338741e900 2024-04-25T10:30:38.658 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:38.651+0000 7f7b9d33e640 1 -- 172.21.15.37:0/1699050071 <== osd.0 v2:172.21.15.37:6816/2746074720 3 ==== osd_op_reply(17 [pgnls start_epoch 44 out=49b] v36'8 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x5633870fa000 con 0x56338741e900 2024-04-25T10:30:38.658 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:38.651+0000 7f7b98b35640 1 -- 172.21.15.37:0/1699050071 --> [v2:172.21.15.37:6816/2746074720,v1:172.21.15.37:6817/2746074720] -- osd_op(unknown.0.0:18 5.22 5:44000000::::head [pgnls start_epoch 44 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e44) -- 0x563387e02c00 con 0x56338741e900 2024-04-25T10:30:38.658 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:38.651+0000 7f7b9d33e640 1 -- 172.21.15.37:0/1699050071 <== osd.0 v2:172.21.15.37:6816/2746074720 4 ==== osd_op_reply(18 [pgnls start_epoch 44 out=49b] v36'8 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x5633870fa000 con 0x56338741e900 2024-04-25T10:30:38.658 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:38.651+0000 7f7b99336640 1 -- 172.21.15.37:0/1699050071 --> [v2:172.21.15.203:6810/2309615014,v1:172.21.15.203:6812/2309615014] -- osd_op(unknown.0.0:19 5.12 5:48000000::::head [pgnls start_epoch 44 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e44) -- 0x563387427c00 con 0x56338741f200 2024-04-25T10:30:38.658 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:38.651+0000 7f7b9e340640 1 -- 172.21.15.37:0/1699050071 <== osd.6 v2:172.21.15.203:6810/2309615014 3 ==== osd_op_reply(19 [pgnls start_epoch 44 out=49b] v36'10 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x5633870fa500 con 0x56338741f200 2024-04-25T10:30:38.658 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:38.651+0000 7f7b98b35640 1 -- 172.21.15.37:0/1699050071 --> [v2:172.21.15.203:6800/4051532063,v1:172.21.15.203:6801/4051532063] -- osd_op(unknown.0.0:20 5.32 5:4c000000::::head [pgnls start_epoch 44 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e44) -- 0x563387e03c00 con 0x5633871acd00 2024-04-25T10:30:38.666 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:38.652+0000 7f7b9d33e640 1 -- 172.21.15.37:0/1699050071 <== osd.4 v2:172.21.15.203:6800/4051532063 4 ==== osd_op_reply(20 [pgnls start_epoch 44 out=49b] v36'8 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x5633870fa000 con 0x5633871acd00 2024-04-25T10:30:38.666 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:38.652+0000 7f7b99336640 1 -- 172.21.15.37:0/1699050071 --> [v2:172.21.15.203:6810/2309615014,v1:172.21.15.203:6812/2309615014] -- osd_op(unknown.0.0:21 5.a 5:50000000::::head [pgnls start_epoch 44 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e44) -- 0x563387426000 con 0x56338741f200 2024-04-25T10:30:38.666 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:38.652+0000 7f7b9e340640 1 -- 172.21.15.37:0/1699050071 <== osd.6 v2:172.21.15.203:6810/2309615014 4 ==== osd_op_reply(21 [pgnls start_epoch 44 out=49b] v36'5 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x5633870fa500 con 0x56338741f200 2024-04-25T10:30:38.666 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:38.652+0000 7f7b98b35640 1 -- 172.21.15.37:0/1699050071 --> [v2:172.21.15.37:6808/2754669649,v1:172.21.15.37:6809/2754669649] -- osd_op(unknown.0.0:22 5.2a 5:54000000::::head [pgnls start_epoch 44 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e44) -- 0x56338744e000 con 0x56338741e480 2024-04-25T10:30:38.666 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:38.652+0000 7f7b9db3f640 1 -- 172.21.15.37:0/1699050071 <== osd.2 v2:172.21.15.37:6808/2754669649 6 ==== osd_op_reply(22 [pgnls start_epoch 44 out=49b] v36'9 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x5633870fa780 con 0x56338741e480 2024-04-25T10:30:38.666 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:38.652+0000 7f7b99336640 1 -- 172.21.15.37:0/1699050071 --> [v2:172.21.15.37:6800/1690341662,v1:172.21.15.37:6801/1690341662] -- osd_op(unknown.0.0:23 5.1a 5:58000000::::head [pgnls start_epoch 44 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e44) -- 0x563387427000 con 0x56338741f680 2024-04-25T10:30:38.666 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:38.653+0000 7f7b9db3f640 1 -- 172.21.15.37:0/1699050071 <== osd.1 v2:172.21.15.37:6800/1690341662 3 ==== osd_op_reply(23 [pgnls start_epoch 44 out=49b] v36'7 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x5633870fa780 con 0x56338741f680 2024-04-25T10:30:38.666 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:38.653+0000 7f7b98b35640 1 --2- 172.21.15.37:0/1699050071 >> [v2:172.21.15.203:6808/3688099014,v1:172.21.15.203:6809/3688099014] conn(0x56338741ed80 0x563387423080 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:30:38.666 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:38.653+0000 7f7b98b35640 1 -- 172.21.15.37:0/1699050071 --> [v2:172.21.15.203:6808/3688099014,v1:172.21.15.203:6809/3688099014] -- osd_op(unknown.0.0:24 5.3a 5:5c000000::::head [pgnls start_epoch 44 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e44) -- 0x56338744e400 con 0x56338741ed80 2024-04-25T10:30:38.667 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:38.653+0000 7f7b9e340640 1 --2- 172.21.15.37:0/1699050071 >> [v2:172.21.15.203:6808/3688099014,v1:172.21.15.203:6809/3688099014] conn(0x56338741ed80 0x563387423080 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:30:38.667 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:38.653+0000 7f7b9e340640 1 --2- 172.21.15.37:0/1699050071 >> [v2:172.21.15.203:6808/3688099014,v1:172.21.15.203:6809/3688099014] conn(0x56338741ed80 0x563387423080 crc :-1 s=READY pgs=15 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.7 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:30:38.667 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:38.654+0000 7f7b9e340640 1 -- 172.21.15.37:0/1699050071 <== osd.7 v2:172.21.15.203:6808/3688099014 1 ==== osd_op_reply(24 [pgnls start_epoch 44 out=49b] v36'6 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x5633870fa500 con 0x56338741ed80 2024-04-25T10:30:38.667 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:38.654+0000 7f7b99336640 1 -- 172.21.15.37:0/1699050071 --> [v2:172.21.15.203:6808/3688099014,v1:172.21.15.203:6809/3688099014] -- osd_op(unknown.0.0:25 5.6 5:60000000::::head [pgnls start_epoch 44 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e44) -- 0x563387426800 con 0x56338741ed80 2024-04-25T10:30:38.667 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:38.654+0000 7f7b9e340640 1 -- 172.21.15.37:0/1699050071 <== osd.7 v2:172.21.15.203:6808/3688099014 2 ==== osd_op_reply(25 [pgnls start_epoch 44 out=49b] v36'7 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x5633870fa500 con 0x56338741ed80 2024-04-25T10:30:38.667 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:38.654+0000 7f7b98b35640 1 -- 172.21.15.37:0/1699050071 --> [v2:172.21.15.203:6800/4051532063,v1:172.21.15.203:6801/4051532063] -- osd_op(unknown.0.0:26 5.26 5:64000000::::head [pgnls start_epoch 44 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e44) -- 0x56338744e800 con 0x5633871acd00 2024-04-25T10:30:38.667 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:38.654+0000 7f7b9d33e640 1 -- 172.21.15.37:0/1699050071 <== osd.4 v2:172.21.15.203:6800/4051532063 5 ==== osd_op_reply(26 [pgnls start_epoch 44 out=49b] v36'15 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x5633870fa000 con 0x5633871acd00 2024-04-25T10:30:38.667 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:38.654+0000 7f7b99336640 1 -- 172.21.15.37:0/1699050071 --> [v2:172.21.15.37:6824/1815290618,v1:172.21.15.37:6825/1815290618] -- osd_op(unknown.0.0:27 5.16 5:68000000::::head [pgnls start_epoch 44 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e44) -- 0x563387427c00 con 0x56338741e000 2024-04-25T10:30:38.667 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:38.654+0000 7f7b9e340640 1 -- 172.21.15.37:0/1699050071 <== osd.3 v2:172.21.15.37:6824/1815290618 2 ==== osd_op_reply(27 [pgnls start_epoch 44 out=49b] v36'10 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x5633870fa500 con 0x56338741e000 2024-04-25T10:30:38.667 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:38.654+0000 7f7b98b35640 1 -- 172.21.15.37:0/1699050071 --> [v2:172.21.15.203:6800/4051532063,v1:172.21.15.203:6801/4051532063] -- osd_op(unknown.0.0:28 5.36 5:6c000000::::head [pgnls start_epoch 44 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e44) -- 0x56338744ec00 con 0x5633871acd00 2024-04-25T10:30:38.667 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:38.655+0000 7f7b9d33e640 1 -- 172.21.15.37:0/1699050071 <== osd.4 v2:172.21.15.203:6800/4051532063 6 ==== osd_op_reply(28 [pgnls start_epoch 44 out=49b] v36'15 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x5633870fa000 con 0x5633871acd00 2024-04-25T10:30:38.667 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:38.655+0000 7f7b99336640 1 -- 172.21.15.37:0/1699050071 --> [v2:172.21.15.37:6824/1815290618,v1:172.21.15.37:6825/1815290618] -- osd_op(unknown.0.0:29 5.e 5:70000000::::head [pgnls start_epoch 44 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e44) -- 0x56338744e000 con 0x56338741e000 2024-04-25T10:30:38.667 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:38.655+0000 7f7b9e340640 1 -- 172.21.15.37:0/1699050071 <== osd.3 v2:172.21.15.37:6824/1815290618 3 ==== osd_op_reply(29 [pgnls start_epoch 44 out=49b] v36'5 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x5633870fa500 con 0x56338741e000 2024-04-25T10:30:38.667 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:38.655+0000 7f7b98b35640 1 -- 172.21.15.37:0/1699050071 --> [v2:172.21.15.203:6808/3688099014,v1:172.21.15.203:6809/3688099014] -- osd_op(unknown.0.0:30 5.2e 5:74000000::::head [pgnls start_epoch 44 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e44) -- 0x56338744f000 con 0x56338741ed80 2024-04-25T10:30:38.668 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:38.655+0000 7f7b9e340640 1 -- 172.21.15.37:0/1699050071 <== osd.7 v2:172.21.15.203:6808/3688099014 3 ==== osd_op_reply(30 [pgnls start_epoch 44 out=49b] v36'10 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x5633870fa500 con 0x56338741ed80 2024-04-25T10:30:38.668 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:38.655+0000 7f7b99336640 1 -- 172.21.15.37:0/1699050071 --> [v2:172.21.15.203:6810/2309615014,v1:172.21.15.203:6812/2309615014] -- osd_op(unknown.0.0:31 5.1e 5:78000000::::head [pgnls start_epoch 44 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e44) -- 0x56338744f400 con 0x56338741f200 2024-04-25T10:30:38.668 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:38.656+0000 7f7b9e340640 1 -- 172.21.15.37:0/1699050071 <== osd.6 v2:172.21.15.203:6810/2309615014 5 ==== osd_op_reply(31 [pgnls start_epoch 44 out=49b] v36'5 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x5633870fa500 con 0x56338741f200 2024-04-25T10:30:38.668 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:38.656+0000 7f7b98b35640 1 -- 172.21.15.37:0/1699050071 --> [v2:172.21.15.203:6824/1203822890,v1:172.21.15.203:6825/1203822890] -- osd_op(unknown.0.0:32 5.3e 5:7c000000::::head [pgnls start_epoch 44 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e44) -- 0x56338744e400 con 0x56338741fb00 2024-04-25T10:30:38.668 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:38.656+0000 7f7b9d33e640 1 -- 172.21.15.37:0/1699050071 <== osd.5 v2:172.21.15.203:6824/1203822890 2 ==== osd_op_reply(32 [pgnls start_epoch 44 out=49b] v36'4 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x5633870fa000 con 0x56338741fb00 2024-04-25T10:30:38.668 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:38.656+0000 7f7b99336640 1 -- 172.21.15.37:0/1699050071 --> [v2:172.21.15.37:6824/1815290618,v1:172.21.15.37:6825/1815290618] -- osd_op(unknown.0.0:33 5.1 5:80000000::::head [pgnls start_epoch 44 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e44) -- 0x563387426400 con 0x56338741e000 2024-04-25T10:30:38.668 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:38.656+0000 7f7b9e340640 1 -- 172.21.15.37:0/1699050071 <== osd.3 v2:172.21.15.37:6824/1815290618 4 ==== osd_op_reply(33 [pgnls start_epoch 44 out=49b] v36'5 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x5633870fa500 con 0x56338741e000 2024-04-25T10:30:38.668 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:38.656+0000 7f7b98b35640 1 -- 172.21.15.37:0/1699050071 --> [v2:172.21.15.37:6808/2754669649,v1:172.21.15.37:6809/2754669649] -- osd_op(unknown.0.0:34 5.21 5:84000000::::head [pgnls start_epoch 44 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e44) -- 0x56338744e000 con 0x56338741e480 2024-04-25T10:30:38.668 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:38.657+0000 7f7b9db3f640 1 -- 172.21.15.37:0/1699050071 <== osd.2 v2:172.21.15.37:6808/2754669649 7 ==== osd_op_reply(34 [pgnls start_epoch 44 out=49b] v36'12 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x5633870fa780 con 0x56338741e480 2024-04-25T10:30:38.668 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:38.657+0000 7f7b99336640 1 -- 172.21.15.37:0/1699050071 --> [v2:172.21.15.37:6824/1815290618,v1:172.21.15.37:6825/1815290618] -- osd_op(unknown.0.0:35 5.11 5:88000000::::head [pgnls start_epoch 44 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e44) -- 0x563387426c00 con 0x56338741e000 2024-04-25T10:30:38.668 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:38.657+0000 7f7b9e340640 1 -- 172.21.15.37:0/1699050071 <== osd.3 v2:172.21.15.37:6824/1815290618 5 ==== osd_op_reply(35 [pgnls start_epoch 44 out=49b] v36'6 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x5633870fa500 con 0x56338741e000 2024-04-25T10:30:38.668 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:38.657+0000 7f7b98b35640 1 -- 172.21.15.37:0/1699050071 --> [v2:172.21.15.37:6816/2746074720,v1:172.21.15.37:6817/2746074720] -- osd_op(unknown.0.0:36 5.31 5:8c000000::::head [pgnls start_epoch 44 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e44) -- 0x56338744f800 con 0x56338741e900 2024-04-25T10:30:38.668 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:38.657+0000 7f7b9d33e640 1 -- 172.21.15.37:0/1699050071 <== osd.0 v2:172.21.15.37:6816/2746074720 5 ==== osd_op_reply(36 [pgnls start_epoch 44 out=49b] v36'7 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x5633870fa000 con 0x56338741e900 2024-04-25T10:30:38.668 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:38.657+0000 7f7b99336640 1 -- 172.21.15.37:0/1699050071 --> [v2:172.21.15.37:6800/1690341662,v1:172.21.15.37:6801/1690341662] -- osd_op(unknown.0.0:37 5.9 5:90000000::::head [pgnls start_epoch 44 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e44) -- 0x563387427400 con 0x56338741f680 2024-04-25T10:30:38.669 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:38.658+0000 7f7b9db3f640 1 -- 172.21.15.37:0/1699050071 <== osd.1 v2:172.21.15.37:6800/1690341662 4 ==== osd_op_reply(37 [pgnls start_epoch 44 out=49b] v36'2 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x5633870fa780 con 0x56338741f680 2024-04-25T10:30:38.669 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:38.658+0000 7f7b98b35640 1 -- 172.21.15.37:0/1699050071 --> [v2:172.21.15.203:6810/2309615014,v1:172.21.15.203:6812/2309615014] -- osd_op(unknown.0.0:38 5.29 5:94000000::::head [pgnls start_epoch 44 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e44) -- 0x563387426000 con 0x56338741f200 2024-04-25T10:30:38.669 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:38.658+0000 7f7b9e340640 1 -- 172.21.15.37:0/1699050071 <== osd.6 v2:172.21.15.203:6810/2309615014 6 ==== osd_op_reply(38 [pgnls start_epoch 44 out=49b] v36'8 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x5633870fa500 con 0x56338741f200 2024-04-25T10:30:38.669 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:38.658+0000 7f7b99336640 1 -- 172.21.15.37:0/1699050071 --> [v2:172.21.15.203:6800/4051532063,v1:172.21.15.203:6801/4051532063] -- osd_op(unknown.0.0:39 5.19 5:98000000::::head [pgnls start_epoch 44 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e44) -- 0x563387f71c00 con 0x5633871acd00 2024-04-25T10:30:38.669 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:38.658+0000 7f7b9d33e640 1 -- 172.21.15.37:0/1699050071 <== osd.4 v2:172.21.15.203:6800/4051532063 7 ==== osd_op_reply(39 [pgnls start_epoch 44 out=49b] v36'10 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x5633870fa000 con 0x5633871acd00 2024-04-25T10:30:38.669 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:38.658+0000 7f7b98b35640 1 -- 172.21.15.37:0/1699050071 --> [v2:172.21.15.37:6808/2754669649,v1:172.21.15.37:6809/2754669649] -- osd_op(unknown.0.0:40 5.39 5:9c000000::::head [pgnls start_epoch 44 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e44) -- 0x563387426800 con 0x56338741e480 2024-04-25T10:30:38.669 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:38.659+0000 7f7b9db3f640 1 -- 172.21.15.37:0/1699050071 <== osd.2 v2:172.21.15.37:6808/2754669649 8 ==== osd_op_reply(40 [pgnls start_epoch 44 out=49b] v36'5 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x5633870fa780 con 0x56338741e480 2024-04-25T10:30:38.669 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:38.659+0000 7f7b99336640 1 -- 172.21.15.37:0/1699050071 --> [v2:172.21.15.203:6808/3688099014,v1:172.21.15.203:6809/3688099014] -- osd_op(unknown.0.0:41 5.5 5:a0000000::::head [pgnls start_epoch 44 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e44) -- 0x56338744e800 con 0x56338741ed80 2024-04-25T10:30:38.669 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:38.659+0000 7f7b9e340640 1 -- 172.21.15.37:0/1699050071 <== osd.7 v2:172.21.15.203:6808/3688099014 4 ==== osd_op_reply(41 [pgnls start_epoch 44 out=49b] v36'5 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x5633870fa500 con 0x56338741ed80 2024-04-25T10:30:38.669 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:38.659+0000 7f7b98b35640 1 -- 172.21.15.37:0/1699050071 --> [v2:172.21.15.203:6824/1203822890,v1:172.21.15.203:6825/1203822890] -- osd_op(unknown.0.0:42 5.25 5:a4000000::::head [pgnls start_epoch 44 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e44) -- 0x563387427c00 con 0x56338741fb00 2024-04-25T10:30:38.669 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:38.659+0000 7f7b9d33e640 1 -- 172.21.15.37:0/1699050071 <== osd.5 v2:172.21.15.203:6824/1203822890 3 ==== osd_op_reply(42 [pgnls start_epoch 44 out=49b] v36'6 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x5633870fa000 con 0x56338741fb00 2024-04-25T10:30:38.669 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:38.659+0000 7f7b99336640 1 -- 172.21.15.37:0/1699050071 --> [v2:172.21.15.37:6824/1815290618,v1:172.21.15.37:6825/1815290618] -- osd_op(unknown.0.0:43 5.15 5:a8000000::::head [pgnls start_epoch 44 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e44) -- 0x56338744ec00 con 0x56338741e000 2024-04-25T10:30:38.669 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:38.660+0000 7f7b9e340640 1 -- 172.21.15.37:0/1699050071 <== osd.3 v2:172.21.15.37:6824/1815290618 6 ==== osd_op_reply(43 [pgnls start_epoch 44 out=49b] v36'6 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x5633870fa500 con 0x56338741e000 2024-04-25T10:30:38.669 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:38.660+0000 7f7b98b35640 1 -- 172.21.15.37:0/1699050071 --> [v2:172.21.15.37:6824/1815290618,v1:172.21.15.37:6825/1815290618] -- osd_op(unknown.0.0:44 5.35 5:ac000000::::head [pgnls start_epoch 44 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e44) -- 0x56338744f000 con 0x56338741e000 2024-04-25T10:30:38.670 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:38.660+0000 7f7b9e340640 1 -- 172.21.15.37:0/1699050071 <== osd.3 v2:172.21.15.37:6824/1815290618 7 ==== osd_op_reply(44 [pgnls start_epoch 44 out=49b] v36'6 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x5633870fa500 con 0x56338741e000 2024-04-25T10:30:38.670 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:38.660+0000 7f7b99336640 1 -- 172.21.15.37:0/1699050071 --> [v2:172.21.15.203:6808/3688099014,v1:172.21.15.203:6809/3688099014] -- osd_op(unknown.0.0:45 5.d 5:b0000000::::head [pgnls start_epoch 44 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e44) -- 0x56338744e400 con 0x56338741ed80 2024-04-25T10:30:38.670 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:38.660+0000 7f7b9e340640 1 -- 172.21.15.37:0/1699050071 <== osd.7 v2:172.21.15.203:6808/3688099014 5 ==== osd_op_reply(45 [pgnls start_epoch 44 out=49b] v36'7 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x5633870fa500 con 0x56338741ed80 2024-04-25T10:30:38.670 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:38.660+0000 7f7b98b35640 1 -- 172.21.15.37:0/1699050071 --> [v2:172.21.15.203:6808/3688099014,v1:172.21.15.203:6809/3688099014] -- osd_op(unknown.0.0:46 5.2d 5:b4000000::::head [pgnls start_epoch 44 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e44) -- 0x56338744f400 con 0x56338741ed80 2024-04-25T10:30:38.670 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:38.660+0000 7f7b9e340640 1 -- 172.21.15.37:0/1699050071 <== osd.7 v2:172.21.15.203:6808/3688099014 6 ==== osd_op_reply(46 [pgnls start_epoch 44 out=49b] v36'7 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x5633870fa500 con 0x56338741ed80 2024-04-25T10:30:38.670 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:38.660+0000 7f7b99336640 1 -- 172.21.15.37:0/1699050071 --> [v2:172.21.15.203:6800/4051532063,v1:172.21.15.203:6801/4051532063] -- osd_op(unknown.0.0:47 5.1d 5:b8000000::::head [pgnls start_epoch 44 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e44) -- 0x563387e02c00 con 0x5633871acd00 2024-04-25T10:30:38.670 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:38.661+0000 7f7b9d33e640 1 -- 172.21.15.37:0/1699050071 <== osd.4 v2:172.21.15.203:6800/4051532063 8 ==== osd_op_reply(47 [pgnls start_epoch 44 out=49b] v36'15 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x5633870fa000 con 0x5633871acd00 2024-04-25T10:30:38.670 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:38.661+0000 7f7b98b35640 1 -- 172.21.15.37:0/1699050071 --> [v2:172.21.15.37:6800/1690341662,v1:172.21.15.37:6801/1690341662] -- osd_op(unknown.0.0:48 5.3d 5:bc000000::::head [pgnls start_epoch 44 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e44) -- 0x56338744e800 con 0x56338741f680 2024-04-25T10:30:38.670 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:38.661+0000 7f7b9db3f640 1 -- 172.21.15.37:0/1699050071 <== osd.1 v2:172.21.15.37:6800/1690341662 5 ==== osd_op_reply(48 [pgnls start_epoch 44 out=49b] v36'7 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x5633870fa780 con 0x56338741f680 2024-04-25T10:30:38.670 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:38.661+0000 7f7b99336640 1 -- 172.21.15.37:0/1699050071 --> [v2:172.21.15.203:6800/4051532063,v1:172.21.15.203:6801/4051532063] -- osd_op(unknown.0.0:49 5.3 5:c0000000::::head [pgnls start_epoch 44 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e44) -- 0x563387e03c00 con 0x5633871acd00 2024-04-25T10:30:38.670 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:38.661+0000 7f7b9d33e640 1 -- 172.21.15.37:0/1699050071 <== osd.4 v2:172.21.15.203:6800/4051532063 9 ==== osd_op_reply(49 [pgnls start_epoch 44 out=49b] v36'7 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x5633870fa000 con 0x5633871acd00 2024-04-25T10:30:38.670 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:38.661+0000 7f7b98b35640 1 -- 172.21.15.37:0/1699050071 --> [v2:172.21.15.37:6816/2746074720,v1:172.21.15.37:6817/2746074720] -- osd_op(unknown.0.0:50 5.23 5:c4000000::::head [pgnls start_epoch 44 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e44) -- 0x56338744f800 con 0x56338741e900 2024-04-25T10:30:38.670 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:38.662+0000 7f7b9d33e640 1 -- 172.21.15.37:0/1699050071 <== osd.0 v2:172.21.15.37:6816/2746074720 6 ==== osd_op_reply(50 [pgnls start_epoch 44 out=49b] v36'7 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x5633870fa000 con 0x56338741e900 2024-04-25T10:30:38.670 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:38.662+0000 7f7b99336640 1 -- 172.21.15.37:0/1699050071 --> [v2:172.21.15.203:6800/4051532063,v1:172.21.15.203:6801/4051532063] -- osd_op(unknown.0.0:51 5.13 5:c8000000::::head [pgnls start_epoch 44 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e44) -- 0x563387427800 con 0x5633871acd00 2024-04-25T10:30:38.671 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:38.662+0000 7f7b9d33e640 1 -- 172.21.15.37:0/1699050071 <== osd.4 v2:172.21.15.203:6800/4051532063 10 ==== osd_op_reply(51 [pgnls start_epoch 44 out=49b] v36'15 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x5633870fa000 con 0x5633871acd00 2024-04-25T10:30:38.671 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:38.662+0000 7f7b98b35640 1 -- 172.21.15.37:0/1699050071 --> [v2:172.21.15.203:6810/2309615014,v1:172.21.15.203:6812/2309615014] -- osd_op(unknown.0.0:52 5.33 5:cc000000::::head [pgnls start_epoch 44 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e44) -- 0x563387426400 con 0x56338741f200 2024-04-25T10:30:38.671 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:38.662+0000 7f7b9e340640 1 -- 172.21.15.37:0/1699050071 <== osd.6 v2:172.21.15.203:6810/2309615014 7 ==== osd_op_reply(52 [pgnls start_epoch 44 out=49b] v36'10 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x5633870fa500 con 0x56338741f200 2024-04-25T10:30:38.671 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:38.662+0000 7f7b99336640 1 -- 172.21.15.37:0/1699050071 --> [v2:172.21.15.203:6824/1203822890,v1:172.21.15.203:6825/1203822890] -- osd_op(unknown.0.0:53 5.b 5:d0000000::::head [pgnls start_epoch 44 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e44) -- 0x563387f70c00 con 0x56338741fb00 2024-04-25T10:30:38.671 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:38.663+0000 7f7b9d33e640 1 -- 172.21.15.37:0/1699050071 <== osd.5 v2:172.21.15.203:6824/1203822890 4 ==== osd_op_reply(53 [pgnls start_epoch 44 out=49b] v36'8 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x5633870fa000 con 0x56338741fb00 2024-04-25T10:30:38.671 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:38.663+0000 7f7b98b35640 1 -- 172.21.15.37:0/1699050071 --> [v2:172.21.15.37:6816/2746074720,v1:172.21.15.37:6817/2746074720] -- osd_op(unknown.0.0:54 5.2b 5:d4000000::::head [pgnls start_epoch 44 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e44) -- 0x563387426c00 con 0x56338741e900 2024-04-25T10:30:38.671 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:38.663+0000 7f7b9d33e640 1 -- 172.21.15.37:0/1699050071 <== osd.0 v2:172.21.15.37:6816/2746074720 7 ==== osd_op_reply(54 [pgnls start_epoch 44 out=49b] v36'11 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x5633870fa000 con 0x56338741e900 2024-04-25T10:30:38.671 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:38.663+0000 7f7b99336640 1 -- 172.21.15.37:0/1699050071 --> [v2:172.21.15.37:6808/2754669649,v1:172.21.15.37:6809/2754669649] -- osd_op(unknown.0.0:55 5.1b 5:d8000000::::head [pgnls start_epoch 44 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e44) -- 0x56338744fc00 con 0x56338741e480 2024-04-25T10:30:38.671 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:38.663+0000 7f7b9db3f640 1 -- 172.21.15.37:0/1699050071 <== osd.2 v2:172.21.15.37:6808/2754669649 9 ==== osd_op_reply(55 [pgnls start_epoch 44 out=49b] v36'8 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x5633870fa780 con 0x56338741e480 2024-04-25T10:30:38.671 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:38.663+0000 7f7b98b35640 1 -- 172.21.15.37:0/1699050071 --> [v2:172.21.15.203:6824/1203822890,v1:172.21.15.203:6825/1203822890] -- osd_op(unknown.0.0:56 5.3b 5:dc000000::::head [pgnls start_epoch 44 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e44) -- 0x563387426000 con 0x56338741fb00 2024-04-25T10:30:38.671 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:38.663+0000 7f7b9d33e640 1 -- 172.21.15.37:0/1699050071 <== osd.5 v2:172.21.15.203:6824/1203822890 5 ==== osd_op_reply(56 [pgnls start_epoch 44 out=49b] v36'4 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x5633870fa000 con 0x56338741fb00 2024-04-25T10:30:38.671 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:38.663+0000 7f7b99336640 1 -- 172.21.15.37:0/1699050071 --> [v2:172.21.15.37:6824/1815290618,v1:172.21.15.37:6825/1815290618] -- osd_op(unknown.0.0:57 5.7 5:e0000000::::head [pgnls start_epoch 44 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e44) -- 0x563387427c00 con 0x56338741e000 2024-04-25T10:30:38.671 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:38.664+0000 7f7b9e340640 1 -- 172.21.15.37:0/1699050071 <== osd.3 v2:172.21.15.37:6824/1815290618 8 ==== osd_op_reply(57 [pgnls start_epoch 44 out=81b] v36'8 uv8 ondisk = 1) ==== 144+0+81 (crc 0 0 0) 0x5633870fa500 con 0x56338741e000 2024-04-25T10:30:38.671 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:38.664+0000 7f7ba022e740 1 -- 172.21.15.37:0/1699050071 --> [v2:172.21.15.37:6824/1815290618,v1:172.21.15.37:6825/1815290618] -- osd_op(unknown.0.0:58 5.7 5:e2e07930:::10000000001.00000000:head [stat] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e44) -- 0x563387f71000 con 0x56338741e000 2024-04-25T10:30:38.676 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:38.664+0000 7f7b9e340640 1 -- 172.21.15.37:0/1699050071 <== osd.3 v2:172.21.15.37:6824/1815290618 9 ==== osd_op_reply(58 10000000001.00000000 [stat out=16b] v0'0 uv8 ondisk = 0) ==== 164+0+16 (crc 0 0 0) 0x5633870fa500 con 0x56338741e000 2024-04-25T10:30:38.676 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:38.664+0000 7f7ba022e740 10 datascan.operator(): handling object 1099511627777.0 2024-04-25T10:30:38.676 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:38.664+0000 7f7ba022e740 1 -- 172.21.15.37:0/1699050071 --> [v2:172.21.15.37:6824/1815290618,v1:172.21.15.37:6825/1815290618] -- osd_op(unknown.0.0:59 5.7 5:e2e07930:::10000000001.00000000:head [call cephfs.accumulate_inode_metadata in=112b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e44) -- 0x563387426400 con 0x56338741e000 2024-04-25T10:30:38.676 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:38.666+0000 7f7b9e340640 1 -- 172.21.15.37:0/1699050071 <== osd.3 v2:172.21.15.37:6824/1815290618 10 ==== osd_op_reply(59 10000000001.00000000 [call] v44'9 uv9 ondisk = 0) ==== 164+0+0 (crc 0 0 0) 0x5633870fa500 con 0x56338741e000 2024-04-25T10:30:38.676 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:38.666+0000 7f7ba022e740 1 -- 172.21.15.37:0/1699050071 --> [v2:172.21.15.203:6800/4051532063,v1:172.21.15.203:6801/4051532063] -- osd_op(unknown.0.0:60 5.27 5:e4000000::::head [pgnls start_epoch 44 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e44) -- 0x563387427c00 con 0x5633871acd00 2024-04-25T10:30:38.676 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:38.666+0000 7f7b9d33e640 1 -- 172.21.15.37:0/1699050071 <== osd.4 v2:172.21.15.203:6800/4051532063 11 ==== osd_op_reply(60 [pgnls start_epoch 44 out=49b] v36'2 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x5633870fa000 con 0x5633871acd00 2024-04-25T10:30:38.676 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:38.666+0000 7f7b99336640 1 -- 172.21.15.37:0/1699050071 --> [v2:172.21.15.203:6800/4051532063,v1:172.21.15.203:6801/4051532063] -- osd_op(unknown.0.0:61 5.17 5:e8000000::::head [pgnls start_epoch 44 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e44) -- 0x563387427800 con 0x5633871acd00 2024-04-25T10:30:38.676 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:38.666+0000 7f7b9d33e640 1 -- 172.21.15.37:0/1699050071 <== osd.4 v2:172.21.15.203:6800/4051532063 12 ==== osd_op_reply(61 [pgnls start_epoch 44 out=49b] v36'9 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x5633870fa000 con 0x5633871acd00 2024-04-25T10:30:38.677 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:38.667+0000 7f7b98b35640 1 -- 172.21.15.37:0/1699050071 --> [v2:172.21.15.37:6800/1690341662,v1:172.21.15.37:6801/1690341662] -- osd_op(unknown.0.0:62 5.37 5:ec000000::::head [pgnls start_epoch 44 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e44) -- 0x563387427400 con 0x56338741f680 2024-04-25T10:30:38.677 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:38.667+0000 7f7b9db3f640 1 -- 172.21.15.37:0/1699050071 <== osd.1 v2:172.21.15.37:6800/1690341662 6 ==== osd_op_reply(62 [pgnls start_epoch 44 out=49b] v36'11 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x5633870fa780 con 0x56338741f680 2024-04-25T10:30:38.677 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:38.667+0000 7f7b99336640 1 -- 172.21.15.37:0/1699050071 --> [v2:172.21.15.37:6800/1690341662,v1:172.21.15.37:6801/1690341662] -- osd_op(unknown.0.0:63 5.f 5:f0000000::::head [pgnls start_epoch 44 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e44) -- 0x563387426c00 con 0x56338741f680 2024-04-25T10:30:38.677 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:38.667+0000 7f7b9db3f640 1 -- 172.21.15.37:0/1699050071 <== osd.1 v2:172.21.15.37:6800/1690341662 7 ==== osd_op_reply(63 [pgnls start_epoch 44 out=49b] v36'9 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x5633870fa780 con 0x56338741f680 2024-04-25T10:30:38.677 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:38.667+0000 7f7b98b35640 1 -- 172.21.15.37:0/1699050071 --> [v2:172.21.15.37:6824/1815290618,v1:172.21.15.37:6825/1815290618] -- osd_op(unknown.0.0:64 5.2f 5:f4000000::::head [pgnls start_epoch 44 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e44) -- 0x563387426800 con 0x56338741e000 2024-04-25T10:30:38.677 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:38.668+0000 7f7b9e340640 1 -- 172.21.15.37:0/1699050071 <== osd.3 v2:172.21.15.37:6824/1815290618 11 ==== osd_op_reply(64 [pgnls start_epoch 44 out=81b] v36'9 uv1 ondisk = 1) ==== 144+0+81 (crc 0 0 0) 0x5633870fa500 con 0x56338741e000 2024-04-25T10:30:38.677 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:38.668+0000 7f7ba022e740 1 -- 172.21.15.37:0/1699050071 --> [v2:172.21.15.37:6824/1815290618,v1:172.21.15.37:6825/1815290618] -- osd_op(unknown.0.0:65 5.2f 5:f526278d:::10000000001.00000001:head [stat] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e44) -- 0x563387f71000 con 0x56338741e000 2024-04-25T10:30:38.677 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:38.668+0000 7f7b9e340640 1 -- 172.21.15.37:0/1699050071 <== osd.3 v2:172.21.15.37:6824/1815290618 12 ==== osd_op_reply(65 10000000001.00000001 [stat out=16b] v0'0 uv1 ondisk = 0) ==== 164+0+16 (crc 0 0 0) 0x5633870fa500 con 0x56338741e000 2024-04-25T10:30:38.677 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:38.668+0000 7f7ba022e740 10 datascan.operator(): handling object 1099511627777.1 2024-04-25T10:30:38.677 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:38.668+0000 7f7ba022e740 1 -- 172.21.15.37:0/1699050071 --> [v2:172.21.15.37:6824/1815290618,v1:172.21.15.37:6825/1815290618] -- osd_op(unknown.0.0:66 5.7 5:e2e07930:::10000000001.00000000:head [call cephfs.accumulate_inode_metadata in=112b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e44) -- 0x56338745c000 con 0x56338741e000 2024-04-25T10:30:38.677 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:38.669+0000 7f7b9e340640 1 -- 172.21.15.37:0/1699050071 <== osd.3 v2:172.21.15.37:6824/1815290618 13 ==== osd_op_reply(66 10000000001.00000000 [call] v44'10 uv10 ondisk = 0) ==== 164+0+0 (crc 0 0 0) 0x5633870fa500 con 0x56338741e000 2024-04-25T10:30:38.677 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:38.669+0000 7f7ba022e740 1 -- 172.21.15.37:0/1699050071 --> [v2:172.21.15.37:6808/2754669649,v1:172.21.15.37:6809/2754669649] -- osd_op(unknown.0.0:67 5.1f 5:f8000000::::head [pgnls start_epoch 44 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e44) -- 0x56338745c400 con 0x56338741e480 2024-04-25T10:30:38.677 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:38.670+0000 7f7b9db3f640 1 -- 172.21.15.37:0/1699050071 <== osd.2 v2:172.21.15.37:6808/2754669649 10 ==== osd_op_reply(67 [pgnls start_epoch 44 out=49b] v36'6 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x5633870fa780 con 0x56338741e480 2024-04-25T10:30:38.677 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:38.670+0000 7f7b98b35640 1 -- 172.21.15.37:0/1699050071 --> [v2:172.21.15.37:6808/2754669649,v1:172.21.15.37:6809/2754669649] -- osd_op(unknown.0.0:68 5.3f 5:fc000000::::head [pgnls start_epoch 44 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e44) -- 0x56338745d000 con 0x56338741e480 2024-04-25T10:30:38.677 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:38.670+0000 7f7b9db3f640 1 -- 172.21.15.37:0/1699050071 <== osd.2 v2:172.21.15.37:6808/2754669649 11 ==== osd_op_reply(68 [pgnls start_epoch 44 out=49b] v36'8 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x5633870fa780 con 0x56338741e480 2024-04-25T10:30:38.678 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:38.670+0000 7f7ba022e740 1 -- 172.21.15.37:0/1699050071 >> [v2:172.21.15.37:6816/2746074720,v1:172.21.15.37:6817/2746074720] conn(0x56338741e900 msgr2=0x563387422b00 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:30:38.678 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:38.670+0000 7f7ba022e740 1 --2- 172.21.15.37:0/1699050071 >> [v2:172.21.15.37:6816/2746074720,v1:172.21.15.37:6817/2746074720] conn(0x56338741e900 0x563387422b00 crc :-1 s=READY pgs=13 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:30:38.678 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:38.670+0000 7f7ba022e740 1 -- 172.21.15.37:0/1699050071 >> [v2:172.21.15.37:6800/1690341662,v1:172.21.15.37:6801/1690341662] conn(0x56338741f680 msgr2=0x563387423b80 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:30:38.678 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:38.670+0000 7f7ba022e740 1 --2- 172.21.15.37:0/1699050071 >> [v2:172.21.15.37:6800/1690341662,v1:172.21.15.37:6801/1690341662] conn(0x56338741f680 0x563387423b80 crc :-1 s=READY pgs=15 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:30:38.678 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:38.670+0000 7f7ba022e740 1 -- 172.21.15.37:0/1699050071 >> [v2:172.21.15.37:6808/2754669649,v1:172.21.15.37:6809/2754669649] conn(0x56338741e480 msgr2=0x563387422580 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:30:38.678 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:38.670+0000 7f7ba022e740 1 --2- 172.21.15.37:0/1699050071 >> [v2:172.21.15.37:6808/2754669649,v1:172.21.15.37:6809/2754669649] conn(0x56338741e480 0x563387422580 crc :-1 s=READY pgs=13 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:30:38.678 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:38.670+0000 7f7ba022e740 1 -- 172.21.15.37:0/1699050071 >> [v2:172.21.15.37:6824/1815290618,v1:172.21.15.37:6825/1815290618] conn(0x56338741e000 msgr2=0x563387422000 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:30:38.678 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:38.670+0000 7f7ba022e740 1 --2- 172.21.15.37:0/1699050071 >> [v2:172.21.15.37:6824/1815290618,v1:172.21.15.37:6825/1815290618] conn(0x56338741e000 0x563387422000 crc :-1 s=READY pgs=14 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:30:38.678 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:38.670+0000 7f7ba022e740 1 -- 172.21.15.37:0/1699050071 >> [v2:172.21.15.203:6800/4051532063,v1:172.21.15.203:6801/4051532063] conn(0x5633871acd00 msgr2=0x56338718a680 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:30:38.678 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:38.670+0000 7f7ba022e740 1 --2- 172.21.15.37:0/1699050071 >> [v2:172.21.15.203:6800/4051532063,v1:172.21.15.203:6801/4051532063] conn(0x5633871acd00 0x56338718a680 crc :-1 s=READY pgs=14 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:30:38.678 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:38.670+0000 7f7ba022e740 1 -- 172.21.15.37:0/1699050071 >> [v2:172.21.15.203:6824/1203822890,v1:172.21.15.203:6825/1203822890] conn(0x56338741fb00 msgr2=0x563387424100 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:30:38.678 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:38.670+0000 7f7b9db3f640 1 -- 172.21.15.37:0/1699050071 reap_dead start 2024-04-25T10:30:38.678 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:38.670+0000 7f7ba022e740 1 --2- 172.21.15.37:0/1699050071 >> [v2:172.21.15.203:6824/1203822890,v1:172.21.15.203:6825/1203822890] conn(0x56338741fb00 0x563387424100 crc :-1 s=READY pgs=13 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:30:38.678 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:38.670+0000 7f7ba022e740 1 -- 172.21.15.37:0/1699050071 >> [v2:172.21.15.203:6810/2309615014,v1:172.21.15.203:6812/2309615014] conn(0x56338741f200 msgr2=0x563387423600 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:30:38.679 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:38.670+0000 7f7ba022e740 1 --2- 172.21.15.37:0/1699050071 >> [v2:172.21.15.203:6810/2309615014,v1:172.21.15.203:6812/2309615014] conn(0x56338741f200 0x563387423600 crc :-1 s=READY pgs=13 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:30:38.679 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:38.670+0000 7f7ba022e740 1 -- 172.21.15.37:0/1699050071 >> [v2:172.21.15.203:6808/3688099014,v1:172.21.15.203:6809/3688099014] conn(0x56338741ed80 msgr2=0x563387423080 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:30:38.679 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:38.670+0000 7f7ba022e740 1 --2- 172.21.15.37:0/1699050071 >> [v2:172.21.15.203:6808/3688099014,v1:172.21.15.203:6809/3688099014] conn(0x56338741ed80 0x563387423080 crc :-1 s=READY pgs=15 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:30:38.679 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:38.670+0000 7f7ba022e740 1 -- 172.21.15.37:0/1699050071 >> [v2:172.21.15.203:6832/2418205237,v1:172.21.15.203:6833/2418205237] conn(0x5633871ada80 msgr2=0x56338718a100 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:30:38.679 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:38.670+0000 7f7ba022e740 1 --2- 172.21.15.37:0/1699050071 >> [v2:172.21.15.203:6832/2418205237,v1:172.21.15.203:6833/2418205237] conn(0x5633871ada80 0x56338718a100 secure :-1 s=READY pgs=265 cs=0 l=1 rev1=1 crypto rx=0x5633871150b0 tx=0x563387ee9140 comp rx=0 tx=0).stop 2024-04-25T10:30:38.679 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:38.670+0000 7f7b9db3f640 1 -- 172.21.15.37:0/1699050071 reap_dead start 2024-04-25T10:30:38.679 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:38.670+0000 7f7ba022e740 1 -- 172.21.15.37:0/1699050071 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x5633871aa900 msgr2=0x563387189600 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:30:38.679 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:38.670+0000 7f7ba022e740 1 --2- 172.21.15.37:0/1699050071 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x5633871aa900 0x563387189600 secure :-1 s=READY pgs=562 cs=0 l=1 rev1=1 crypto rx=0x563387f6e6c0 tx=0x563387ee8c00 comp rx=0 tx=0).stop 2024-04-25T10:30:38.679 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:38.670+0000 7f7ba022e740 1 -- 172.21.15.37:0/1699050071 shutdown_connections 2024-04-25T10:30:38.679 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:38.670+0000 7f7ba022e740 1 --2- 172.21.15.37:0/1699050071 >> [v2:172.21.15.203:6832/2418205237,v1:172.21.15.203:6833/2418205237] conn(0x5633871ada80 0x56338718a100 unknown :-1 s=CLOSED pgs=265 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:30:38.679 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:38.670+0000 7f7ba022e740 1 --2- 172.21.15.37:0/1699050071 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x5633871aa900 0x563387189600 unknown :-1 s=CLOSED pgs=562 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:30:38.679 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:38.670+0000 7f7ba022e740 1 -- 172.21.15.37:0/1699050071 >> 172.21.15.37:0/1699050071 conn(0x5633871ab200 msgr2=0x5633870f9800 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:30:38.679 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:38.670+0000 7f7ba022e740 1 -- 172.21.15.37:0/1699050071 shutdown_connections 2024-04-25T10:30:38.680 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:38.670+0000 7f7ba022e740 1 -- 172.21.15.37:0/1699050071 wait complete. 2024-04-25T10:30:38.680 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:38.671+0000 7f7ba022e740 1 -- 172.21.15.37:0/1847905238 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x5633871abb00 msgr2=0x563387188580 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:30:38.680 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:38.671+0000 7f7ba022e740 1 --2- 172.21.15.37:0/1847905238 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x5633871abb00 0x563387188580 secure :-1 s=READY pgs=560 cs=0 l=1 rev1=1 crypto rx=0x563387e49440 tx=0x563387112c60 comp rx=0 tx=0).stop 2024-04-25T10:30:38.680 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:38.671+0000 7f7ba022e740 1 -- 172.21.15.37:0/1847905238 shutdown_connections 2024-04-25T10:30:38.680 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:38.671+0000 7f7ba022e740 1 --2- 172.21.15.37:0/1847905238 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x5633871abf80 0x563387188000 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:30:38.680 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:38.671+0000 7f7ba022e740 1 --2- 172.21.15.37:0/1847905238 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x5633871abb00 0x563387188580 unknown :-1 s=CLOSED pgs=560 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:30:38.680 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:38.671+0000 7f7ba022e740 1 --2- 172.21.15.37:0/1847905238 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x5633871ab680 0x563387188b00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:30:38.680 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:38.671+0000 7f7ba022e740 1 -- 172.21.15.37:0/1847905238 >> 172.21.15.37:0/1847905238 conn(0x5633871ac400 msgr2=0x5633870f9000 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:30:38.680 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:38.671+0000 7f7ba022e740 1 -- 172.21.15.37:0/1847905238 shutdown_connections 2024-04-25T10:30:38.680 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:38.671+0000 7f7ba022e740 1 -- 172.21.15.37:0/1847905238 wait complete. 2024-04-25T10:30:38.680 DEBUG:tasks.cephfs.filesystem:Ran ['cephfs-data-scan', '--debug-mds=20', '--debug-ms=1', '--debug-objecter=1', 'scan_extents', '--alternate-pool', 'recovery_fs_metadata', '--filesystem', 'cephfs', 'cephfs_data'] in time 0:00:00.102895, result: 2024-04-25T10:30:38.681 DEBUG:teuthology.orchestra.run.smithi037:> cephfs-data-scan --debug-mds=20 --debug-ms=1 --debug-objecter=1 scan_inodes --alternate-pool recovery_fs_metadata --filesystem cephfs --force-corrupt --force-init cephfs_data 2024-04-25T10:30:38.738 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:38.736+0000 7f08d7a22740 1 -- start start 2024-04-25T10:30:38.739 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:38.736+0000 7f08d7a22740 1 Processor -- start 2024-04-25T10:30:38.741 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:38.737+0000 7f08d7a22740 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x561d661d7680 0x561d661ac580 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:30:38.741 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:38.737+0000 7f08d7a22740 1 --2- >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x561d661d7b00 0x561d661ac000 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:30:38.741 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:38.737+0000 7f08d7a22740 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x561d661d7f80 0x561d661acb00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:30:38.741 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:38.737+0000 7f08d7a22740 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 -- 0x561d660fc180 con 0x561d661d7f80 2024-04-25T10:30:38.742 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:38.737+0000 7f08d7a22740 1 -- --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_getmap magic: 0 -- 0x561d660fc300 con 0x561d661d7680 2024-04-25T10:30:38.742 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:38.738+0000 7f08d7a22740 1 -- --> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] -- mon_getmap magic: 0 -- 0x561d660fc480 con 0x561d661d7b00 2024-04-25T10:30:38.742 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:38.738+0000 7f08d5ecf640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x561d661d7f80 0x561d661acb00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:30:38.742 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:38.738+0000 7f08d5ecf640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x561d661d7f80 0x561d661acb00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.37:3300/0 says I am v2:172.21.15.37:55676/0 (socket says 172.21.15.37:55676) 2024-04-25T10:30:38.742 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:38.738+0000 7f08d5ecf640 1 -- 172.21.15.37:0/4100771996 learned_addr learned my addr 172.21.15.37:0/4100771996 (peer_addr_for_me v2:172.21.15.37:0/0) 2024-04-25T10:30:38.742 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:38.738+0000 7f08d56ce640 1 --2- 172.21.15.37:0/4100771996 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x561d661d7680 0x561d661ac580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:30:38.742 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:38.738+0000 7f08d4ecd640 1 --2- 172.21.15.37:0/4100771996 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x561d661d7b00 0x561d661ac000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:30:38.742 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:38.738+0000 7f08d5ecf640 1 -- 172.21.15.37:0/4100771996 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x561d661d7b00 msgr2=0x561d661ac000 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:30:38.742 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:38.738+0000 7f08d5ecf640 1 --2- 172.21.15.37:0/4100771996 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x561d661d7b00 0x561d661ac000 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:30:38.742 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:38.738+0000 7f08d5ecf640 1 -- 172.21.15.37:0/4100771996 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x561d661d7680 msgr2=0x561d661ac580 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:30:38.742 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:38.738+0000 7f08d5ecf640 1 --2- 172.21.15.37:0/4100771996 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x561d661d7680 0x561d661ac580 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:30:38.742 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:38.738+0000 7f08d5ecf640 1 -- 172.21.15.37:0/4100771996 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x561d6203b680 con 0x561d661d7f80 2024-04-25T10:30:38.742 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:38.738+0000 7f08d5ecf640 1 --2- 172.21.15.37:0/4100771996 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x561d661d7f80 0x561d661acb00 secure :-1 s=READY pgs=563 cs=0 l=1 rev1=1 crypto rx=0x561d66e63cb0 tx=0x561d66142180 comp rx=0 tx=0).ready entity=mon.0 client_cookie=30ee748baa91c850 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:30:38.742 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:38.738+0000 7f08d2ec9640 1 -- 172.21.15.37:0/4100771996 <== mon.0 v2:172.21.15.37:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x561d62034680 con 0x561d661d7f80 2024-04-25T10:30:38.743 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:38.738+0000 7f08d2ec9640 1 -- 172.21.15.37:0/4100771996 <== mon.0 v2:172.21.15.37:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x561d66086540 con 0x561d661d7f80 2024-04-25T10:30:38.743 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:38.738+0000 7f08d2ec9640 1 -- 172.21.15.37:0/4100771996 <== mon.0 v2:172.21.15.37:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x561d62034b60 con 0x561d661d7f80 2024-04-25T10:30:38.743 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:38.738+0000 7f08d7a22740 1 -- 172.21.15.37:0/4100771996 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({osdmap=0}) -- 0x561d6203b860 con 0x561d661d7f80 2024-04-25T10:30:38.743 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:38.738+0000 7f08d7a22740 1 -- 172.21.15.37:0/4100771996 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({fsmap=0}) -- 0x561d6203b2c0 con 0x561d661d7f80 2024-04-25T10:30:38.743 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:38.738+0000 7f08d7a22740 4 waiting for MDS map... 2024-04-25T10:30:38.743 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:38.739+0000 7f08d2ec9640 1 -- 172.21.15.37:0/4100771996 <== mon.0 v2:172.21.15.37:3300/0 4 ==== osd_map(44..44 src has 1..44) ==== 6248+0+0 (secure 0 0 0) 0x561d66f1a600 con 0x561d661d7f80 2024-04-25T10:30:38.743 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:38.739+0000 7f08d2ec9640 1 -- 172.21.15.37:0/4100771996 <== mon.0 v2:172.21.15.37:3300/0 5 ==== fsmap(e 22) ==== 4135+0+0 (secure 0 0 0) 0x561d66097180 con 0x561d661d7f80 2024-04-25T10:30:38.743 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:38.739+0000 7f08d7a22740 4 Got MDS map 22 2024-04-25T10:30:38.743 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:38.739+0000 7f08d7a22740 4 datascan.main: Using metadata pool output 2024-04-25T10:30:38.743 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:38.739+0000 7f08d7a22740 4 datascan.main: connecting to RADOS... 2024-04-25T10:30:38.747 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:38.740+0000 7f08d7a22740 1 Processor -- start 2024-04-25T10:30:38.748 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:38.740+0000 7f08d7a22740 1 -- start start 2024-04-25T10:30:38.748 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:38.741+0000 7f08d7a22740 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x561d661d6d80 0x561d661ad080 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:30:38.748 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:38.741+0000 7f08d7a22740 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x561d661d6900 0x561d661ad600 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:30:38.748 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:38.741+0000 7f08d7a22740 1 --2- >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x561d661d8880 0x561d661adb80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:30:38.748 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:38.741+0000 7f08d7a22740 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 -- 0x561d660fc600 con 0x561d661d6d80 2024-04-25T10:30:38.748 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:38.741+0000 7f08d7a22740 1 -- --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_getmap magic: 0 -- 0x561d660fc780 con 0x561d661d6900 2024-04-25T10:30:38.748 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:38.741+0000 7f08d7a22740 1 -- --> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] -- mon_getmap magic: 0 -- 0x561d660fc900 con 0x561d661d8880 2024-04-25T10:30:38.748 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:38.741+0000 7f08d4ecd640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x561d661d6d80 0x561d661ad080 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:30:38.748 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:38.741+0000 7f08d4ecd640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x561d661d6d80 0x561d661ad080 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.37:3300/0 says I am v2:172.21.15.37:55678/0 (socket says 172.21.15.37:55678) 2024-04-25T10:30:38.748 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:38.741+0000 7f08d4ecd640 1 -- 172.21.15.37:0/4287906926 learned_addr learned my addr 172.21.15.37:0/4287906926 (peer_addr_for_me v2:172.21.15.37:0/0) 2024-04-25T10:30:38.748 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:38.741+0000 7f08d56ce640 1 --2- 172.21.15.37:0/4287906926 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x561d661d6900 0x561d661ad600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:30:38.748 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:38.741+0000 7f08d4ecd640 1 --2- 172.21.15.37:0/4287906926 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x561d661d8880 0x561d661adb80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:30:38.749 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:38.741+0000 7f08d4ecd640 1 -- 172.21.15.37:0/4287906926 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x561d661d8880 msgr2=0x561d661adb80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:30:38.749 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:38.741+0000 7f08d4ecd640 1 --2- 172.21.15.37:0/4287906926 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x561d661d8880 0x561d661adb80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:30:38.749 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:38.741+0000 7f08d4ecd640 1 -- 172.21.15.37:0/4287906926 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x561d661d6900 msgr2=0x561d661ad600 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:30:38.749 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:38.741+0000 7f08d4ecd640 1 --2- 172.21.15.37:0/4287906926 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x561d661d6900 0x561d661ad600 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:30:38.749 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:38.741+0000 7f08d4ecd640 1 -- 172.21.15.37:0/4287906926 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x561d6203b4a0 con 0x561d661d6d80 2024-04-25T10:30:38.749 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:38.741+0000 7f08d4ecd640 1 --2- 172.21.15.37:0/4287906926 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x561d661d6d80 0x561d661ad080 secure :-1 s=READY pgs=564 cs=0 l=1 rev1=1 crypto rx=0x561d66f032f0 tx=0x561d66f32c60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=61f44b6753352d7e server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:30:38.749 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:38.742+0000 7f08d0ec5640 1 -- 172.21.15.37:0/4287906926 <== mon.0 v2:172.21.15.37:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x561d62034680 con 0x561d661d6d80 2024-04-25T10:30:38.749 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:38.742+0000 7f08d0ec5640 1 -- 172.21.15.37:0/4287906926 <== mon.0 v2:172.21.15.37:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x561d66086e00 con 0x561d661d6d80 2024-04-25T10:30:38.749 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:38.742+0000 7f08d0ec5640 1 -- 172.21.15.37:0/4287906926 <== mon.0 v2:172.21.15.37:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x561d62034b60 con 0x561d661d6d80 2024-04-25T10:30:38.749 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:38.742+0000 7f08d7a22740 1 -- 172.21.15.37:0/4287906926 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x561d661d6d80 msgr2=0x561d661ad080 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:30:38.749 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:38.742+0000 7f08d7a22740 1 --2- 172.21.15.37:0/4287906926 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x561d661d6d80 0x561d661ad080 secure :-1 s=READY pgs=564 cs=0 l=1 rev1=1 crypto rx=0x561d66f032f0 tx=0x561d66f32c60 comp rx=0 tx=0).stop 2024-04-25T10:30:38.749 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:38.742+0000 7f08d7a22740 1 -- 172.21.15.37:0/4287906926 shutdown_connections 2024-04-25T10:30:38.749 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:38.742+0000 7f08d7a22740 1 --2- 172.21.15.37:0/4287906926 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x561d661d8880 0x561d661adb80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:30:38.750 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:38.742+0000 7f08d7a22740 1 --2- 172.21.15.37:0/4287906926 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x561d661d6900 0x561d661ad600 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:30:38.750 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:38.742+0000 7f08d7a22740 1 --2- 172.21.15.37:0/4287906926 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x561d661d6d80 0x561d661ad080 unknown :-1 s=CLOSED pgs=564 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:30:38.750 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:38.742+0000 7f08d7a22740 1 -- 172.21.15.37:0/4287906926 >> 172.21.15.37:0/4287906926 conn(0x561d661d7200 msgr2=0x561d66125800 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:30:38.750 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:38.742+0000 7f08d7a22740 1 -- 172.21.15.37:0/4287906926 shutdown_connections 2024-04-25T10:30:38.750 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:38.742+0000 7f08d7a22740 1 -- 172.21.15.37:0/4287906926 wait complete. 2024-04-25T10:30:38.750 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:38.743+0000 7f08d7a22740 1 Processor -- start 2024-04-25T10:30:38.750 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:38.743+0000 7f08d7a22740 1 -- start start 2024-04-25T10:30:38.750 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:38.744+0000 7f08d7a22740 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x561d661d6900 0x561d661ad600 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:30:38.750 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:38.744+0000 7f08d7a22740 1 --2- >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x561d661d6d80 0x561d661ad080 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:30:38.750 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:38.744+0000 7f08d7a22740 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x561d661d8880 0x561d661adb80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:30:38.750 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:38.744+0000 7f08d7a22740 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 -- 0x561d660fdc80 con 0x561d661d6900 2024-04-25T10:30:38.750 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:38.744+0000 7f08d7a22740 1 -- --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_getmap magic: 0 -- 0x561d660fcc00 con 0x561d661d8880 2024-04-25T10:30:38.750 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:38.744+0000 7f08d7a22740 1 -- --> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] -- mon_getmap magic: 0 -- 0x561d660fcd80 con 0x561d661d6d80 2024-04-25T10:30:38.751 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:38.744+0000 7f08d4ecd640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x561d661d6900 0x561d661ad600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:30:38.751 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:38.744+0000 7f08d4ecd640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x561d661d6900 0x561d661ad600 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.37:3300/0 says I am v2:172.21.15.37:55682/0 (socket says 172.21.15.37:55682) 2024-04-25T10:30:38.751 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:38.744+0000 7f08d4ecd640 1 -- 172.21.15.37:0/2208611052 learned_addr learned my addr 172.21.15.37:0/2208611052 (peer_addr_for_me v2:172.21.15.37:0/0) 2024-04-25T10:30:38.751 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:38.744+0000 7f08d56ce640 1 --2- 172.21.15.37:0/2208611052 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x561d661d6d80 0x561d661ad080 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:30:38.751 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:38.744+0000 7f08d4ecd640 1 --2- 172.21.15.37:0/2208611052 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x561d661d8880 0x561d661adb80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:30:38.751 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:38.745+0000 7f08d4ecd640 1 -- 172.21.15.37:0/2208611052 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x561d661d6d80 msgr2=0x561d661ad080 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:30:38.751 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:38.745+0000 7f08d4ecd640 1 --2- 172.21.15.37:0/2208611052 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x561d661d6d80 0x561d661ad080 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:30:38.751 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:38.745+0000 7f08d4ecd640 1 -- 172.21.15.37:0/2208611052 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x561d661d8880 msgr2=0x561d661adb80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:30:38.751 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:38.745+0000 7f08d4ecd640 1 --2- 172.21.15.37:0/2208611052 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x561d661d8880 0x561d661adb80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:30:38.751 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:38.745+0000 7f08d4ecd640 1 -- 172.21.15.37:0/2208611052 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x561d6203b680 con 0x561d661d6900 2024-04-25T10:30:38.751 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:38.745+0000 7f08d4ecd640 1 --2- 172.21.15.37:0/2208611052 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x561d661d6900 0x561d661ad600 secure :-1 s=READY pgs=565 cs=0 l=1 rev1=1 crypto rx=0x561d66f03fb0 tx=0x561d6613ea20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:30:38.751 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:38.745+0000 7f08cf6c2640 1 -- 172.21.15.37:0/2208611052 <== mon.0 v2:172.21.15.37:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x561d620344e0 con 0x561d661d6900 2024-04-25T10:30:38.751 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:38.745+0000 7f08cf6c2640 1 -- 172.21.15.37:0/2208611052 <== mon.0 v2:172.21.15.37:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x561d66086c40 con 0x561d661d6900 2024-04-25T10:30:38.752 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:38.745+0000 7f08d7a22740 1 -- 172.21.15.37:0/2208611052 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x561d6203b860 con 0x561d661d6900 2024-04-25T10:30:38.752 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:38.745+0000 7f08cf6c2640 1 -- 172.21.15.37:0/2208611052 <== mon.0 v2:172.21.15.37:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x561d62034680 con 0x561d661d6900 2024-04-25T10:30:38.752 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:38.745+0000 7f08d7a22740 1 -- 172.21.15.37:0/2208611052 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({osdmap=0}) -- 0x561d6203b0e0 con 0x561d661d6900 2024-04-25T10:30:38.752 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:38.745+0000 7f08d7a22740 4 datascan.init: forcing metadata pool 'recovery_fs_metadata' 2024-04-25T10:30:38.759 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:38.747+0000 7f08cf6c2640 1 -- 172.21.15.37:0/2208611052 <== mon.0 v2:172.21.15.37:3300/0 4 ==== mgrmap(e 5) ==== 96782+0+0 (secure 0 0 0) 0x561d66f70900 con 0x561d661d6900 2024-04-25T10:30:38.759 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:38.747+0000 7f08cf6c2640 1 --2- 172.21.15.37:0/2208611052 >> [v2:172.21.15.203:6832/2418205237,v1:172.21.15.203:6833/2418205237] conn(0x561d661d9a80 0x561d661ae100 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:30:38.759 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:38.747+0000 7f08cf6c2640 1 -- 172.21.15.37:0/2208611052 <== mon.0 v2:172.21.15.37:3300/0 5 ==== osd_map(44..44 src has 1..44) ==== 6248+0+0 (secure 0 0 0) 0x561d66f8aa00 con 0x561d661d6900 2024-04-25T10:30:38.760 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:38.747+0000 7f08d7a22740 4 datascan.main: data pool 'cephfs_data' has ID 5 2024-04-25T10:30:38.760 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:38.747+0000 7f08d7a22740 4 datascan.main: opening data pool 'cephfs_data' 2024-04-25T10:30:38.760 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:38.747+0000 7f08d7a22740 1 --2- 172.21.15.37:0/2208611052 >> [v2:172.21.15.203:6800/4051532063,v1:172.21.15.203:6801/4051532063] conn(0x561d661d8d00 0x561d661ae680 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:30:38.760 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:38.747+0000 7f08d7a22740 1 -- 172.21.15.37:0/2208611052 --> [v2:172.21.15.203:6800/4051532063,v1:172.21.15.203:6801/4051532063] -- osd_op(unknown.0.0:1 6.14 6:287034c4:::1.00000000.inode:head [stat] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e44) -- 0x561d66f94c00 con 0x561d661d8d00 2024-04-25T10:30:38.760 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:38.747+0000 7f08d56ce640 1 --2- 172.21.15.37:0/2208611052 >> [v2:172.21.15.203:6832/2418205237,v1:172.21.15.203:6833/2418205237] conn(0x561d661d9a80 0x561d661ae100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:30:38.760 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:38.747+0000 7f08d4ecd640 1 --2- 172.21.15.37:0/2208611052 >> [v2:172.21.15.203:6800/4051532063,v1:172.21.15.203:6801/4051532063] conn(0x561d661d8d00 0x561d661ae680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:30:38.760 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:38.748+0000 7f08d4ecd640 1 --2- 172.21.15.37:0/2208611052 >> [v2:172.21.15.203:6800/4051532063,v1:172.21.15.203:6801/4051532063] conn(0x561d661d8d00 0x561d661ae680 crc :-1 s=READY pgs=15 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.4 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:30:38.760 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:38.748+0000 7f08d56ce640 1 --2- 172.21.15.37:0/2208611052 >> [v2:172.21.15.203:6832/2418205237,v1:172.21.15.203:6833/2418205237] conn(0x561d661d9a80 0x561d661ae100 secure :-1 s=READY pgs=266 cs=0 l=1 rev1=1 crypto rx=0x561d661408a0 tx=0x561d6613e7e0 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:30:38.760 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:38.748+0000 7f08d4ecd640 1 -- 172.21.15.37:0/2208611052 <== osd.4 v2:172.21.15.203:6800/4051532063 1 ==== osd_op_reply(1 1.00000000.inode [stat out=16b] v0'0 uv1 ondisk = 0) ==== 160+0+16 (crc 0 0 0) 0x561d66126000 con 0x561d661d8d00 2024-04-25T10:30:38.760 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:38.748+0000 7f08d7a22740 1 --2- 172.21.15.37:0/2208611052 >> [v2:172.21.15.203:6810/2309615014,v1:172.21.15.203:6812/2309615014] conn(0x561d661d9180 0x561d661aec00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:30:38.760 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:38.748+0000 7f08d7a22740 1 -- 172.21.15.37:0/2208611052 --> [v2:172.21.15.203:6810/2309615014,v1:172.21.15.203:6812/2309615014] -- osd_op(unknown.0.0:2 6.1f 6:fe07bba1:::100.00000000.inode:head [stat] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e44) -- 0x561d66f95000 con 0x561d661d9180 2024-04-25T10:30:38.761 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:38.748+0000 7f08d5ecf640 1 --2- 172.21.15.37:0/2208611052 >> [v2:172.21.15.203:6810/2309615014,v1:172.21.15.203:6812/2309615014] conn(0x561d661d9180 0x561d661aec00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:30:38.761 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:38.749+0000 7f08d5ecf640 1 --2- 172.21.15.37:0/2208611052 >> [v2:172.21.15.203:6810/2309615014,v1:172.21.15.203:6812/2309615014] conn(0x561d661d9180 0x561d661aec00 crc :-1 s=READY pgs=14 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.6 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:30:38.761 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:38.749+0000 7f08d5ecf640 1 -- 172.21.15.37:0/2208611052 <== osd.6 v2:172.21.15.203:6810/2309615014 1 ==== osd_op_reply(2 100.00000000.inode [stat out=16b] v0'0 uv1 ondisk = 0) ==== 162+0+16 (crc 0 0 0) 0x561d66126500 con 0x561d661d9180 2024-04-25T10:30:38.761 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:38.749+0000 7f08d7a22740 1 -- 172.21.15.37:0/2208611052 --> [v2:172.21.15.203:6800/4051532063,v1:172.21.15.203:6801/4051532063] -- osd_op(unknown.0.0:3 5.0 5:00000000::::head [pgnls-filter start_epoch 44 in=89b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e44) -- 0x561d66f95400 con 0x561d661d8d00 2024-04-25T10:30:38.761 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:38.749+0000 7f08d4ecd640 1 -- 172.21.15.37:0/2208611052 <== osd.4 v2:172.21.15.203:6800/4051532063 2 ==== osd_op_reply(3 [pgnls-filter start_epoch 44 out=49b] v36'8 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x561d66126000 con 0x561d661d8d00 2024-04-25T10:30:38.761 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:38.749+0000 7f08d06c4640 1 --2- 172.21.15.37:0/2208611052 >> [v2:172.21.15.37:6824/1815290618,v1:172.21.15.37:6825/1815290618] conn(0x561d6644c000 0x561d66454000 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:30:38.761 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:38.749+0000 7f08d06c4640 1 -- 172.21.15.37:0/2208611052 --> [v2:172.21.15.37:6824/1815290618,v1:172.21.15.37:6825/1815290618] -- osd_op(unknown.0.0:4 5.20 5:04000000::::head [pgnls-filter start_epoch 44 in=89b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e44) -- 0x561d66458000 con 0x561d6644c000 2024-04-25T10:30:38.761 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:38.750+0000 7f08d56ce640 1 --2- 172.21.15.37:0/2208611052 >> [v2:172.21.15.37:6824/1815290618,v1:172.21.15.37:6825/1815290618] conn(0x561d6644c000 0x561d66454000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:30:38.761 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:38.750+0000 7f08d56ce640 1 --2- 172.21.15.37:0/2208611052 >> [v2:172.21.15.37:6824/1815290618,v1:172.21.15.37:6825/1815290618] conn(0x561d6644c000 0x561d66454000 crc :-1 s=READY pgs=15 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.3 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:30:38.761 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:38.750+0000 7f08d56ce640 1 -- 172.21.15.37:0/2208611052 <== osd.3 v2:172.21.15.37:6824/1815290618 1 ==== osd_op_reply(4 [pgnls-filter start_epoch 44 out=49b] v36'9 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x561d66126780 con 0x561d6644c000 2024-04-25T10:30:38.761 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:38.750+0000 7f08d0ec5640 1 --2- 172.21.15.37:0/2208611052 >> [v2:172.21.15.37:6808/2754669649,v1:172.21.15.37:6809/2754669649] conn(0x561d6644c480 0x561d66454580 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:30:38.761 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:38.750+0000 7f08d0ec5640 1 -- 172.21.15.37:0/2208611052 --> [v2:172.21.15.37:6808/2754669649,v1:172.21.15.37:6809/2754669649] -- osd_op(unknown.0.0:5 5.10 5:08000000::::head [pgnls-filter start_epoch 44 in=89b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e44) -- 0x561d66458400 con 0x561d6644c480 2024-04-25T10:30:38.761 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:38.750+0000 7f08d4ecd640 1 --2- 172.21.15.37:0/2208611052 >> [v2:172.21.15.37:6808/2754669649,v1:172.21.15.37:6809/2754669649] conn(0x561d6644c480 0x561d66454580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:30:38.761 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:38.751+0000 7f08d4ecd640 1 --2- 172.21.15.37:0/2208611052 >> [v2:172.21.15.37:6808/2754669649,v1:172.21.15.37:6809/2754669649] conn(0x561d6644c480 0x561d66454580 crc :-1 s=READY pgs=14 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:30:38.762 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:38.751+0000 7f08d4ecd640 1 -- 172.21.15.37:0/2208611052 <== osd.2 v2:172.21.15.37:6808/2754669649 1 ==== osd_op_reply(5 [pgnls-filter start_epoch 44 out=49b] v36'6 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x561d66126000 con 0x561d6644c480 2024-04-25T10:30:38.762 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:38.751+0000 7f08d06c4640 1 --2- 172.21.15.37:0/2208611052 >> [v2:172.21.15.37:6816/2746074720,v1:172.21.15.37:6817/2746074720] conn(0x561d6644c900 0x561d66454b00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:30:38.762 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:38.751+0000 7f08d06c4640 1 -- 172.21.15.37:0/2208611052 --> [v2:172.21.15.37:6816/2746074720,v1:172.21.15.37:6817/2746074720] -- osd_op(unknown.0.0:6 5.30 5:0c000000::::head [pgnls-filter start_epoch 44 in=89b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e44) -- 0x561d66458800 con 0x561d6644c900 2024-04-25T10:30:38.762 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:38.751+0000 7f08d5ecf640 1 --2- 172.21.15.37:0/2208611052 >> [v2:172.21.15.37:6816/2746074720,v1:172.21.15.37:6817/2746074720] conn(0x561d6644c900 0x561d66454b00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:30:38.762 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:38.751+0000 7f08d5ecf640 1 --2- 172.21.15.37:0/2208611052 >> [v2:172.21.15.37:6816/2746074720,v1:172.21.15.37:6817/2746074720] conn(0x561d6644c900 0x561d66454b00 crc :-1 s=READY pgs=14 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:30:38.762 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:38.752+0000 7f08d5ecf640 1 -- 172.21.15.37:0/2208611052 <== osd.0 v2:172.21.15.37:6816/2746074720 1 ==== osd_op_reply(6 [pgnls-filter start_epoch 44 out=49b] v36'9 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x561d66126500 con 0x561d6644c900 2024-04-25T10:30:38.762 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:38.752+0000 7f08d0ec5640 1 -- 172.21.15.37:0/2208611052 --> [v2:172.21.15.203:6810/2309615014,v1:172.21.15.203:6812/2309615014] -- osd_op(unknown.0.0:7 5.8 5:10000000::::head [pgnls-filter start_epoch 44 in=89b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e44) -- 0x561d66459000 con 0x561d661d9180 2024-04-25T10:30:38.762 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:38.752+0000 7f08d5ecf640 1 -- 172.21.15.37:0/2208611052 <== osd.6 v2:172.21.15.203:6810/2309615014 2 ==== osd_op_reply(7 [pgnls-filter start_epoch 44 out=49b] v36'9 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x561d66126500 con 0x561d661d9180 2024-04-25T10:30:38.762 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:38.752+0000 7f08d06c4640 1 -- 172.21.15.37:0/2208611052 --> [v2:172.21.15.37:6808/2754669649,v1:172.21.15.37:6809/2754669649] -- osd_op(unknown.0.0:8 5.28 5:14000000::::head [pgnls-filter start_epoch 44 in=89b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e44) -- 0x561d66458c00 con 0x561d6644c480 2024-04-25T10:30:38.762 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:38.752+0000 7f08d4ecd640 1 -- 172.21.15.37:0/2208611052 <== osd.2 v2:172.21.15.37:6808/2754669649 2 ==== osd_op_reply(8 [pgnls-filter start_epoch 44 out=49b] v36'6 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x561d66126000 con 0x561d6644c480 2024-04-25T10:30:38.762 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:38.752+0000 7f08d0ec5640 1 --2- 172.21.15.37:0/2208611052 >> [v2:172.21.15.37:6800/1690341662,v1:172.21.15.37:6801/1690341662] conn(0x561d6644d200 0x561d66455600 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:30:38.762 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:38.753+0000 7f08d0ec5640 1 -- 172.21.15.37:0/2208611052 --> [v2:172.21.15.37:6800/1690341662,v1:172.21.15.37:6801/1690341662] -- osd_op(unknown.0.0:9 5.18 5:18000000::::head [pgnls-filter start_epoch 44 in=89b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e44) -- 0x561d66459400 con 0x561d6644d200 2024-04-25T10:30:38.762 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:38.753+0000 7f08d56ce640 1 --2- 172.21.15.37:0/2208611052 >> [v2:172.21.15.37:6800/1690341662,v1:172.21.15.37:6801/1690341662] conn(0x561d6644d200 0x561d66455600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:30:38.763 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:38.753+0000 7f08d56ce640 1 --2- 172.21.15.37:0/2208611052 >> [v2:172.21.15.37:6800/1690341662,v1:172.21.15.37:6801/1690341662] conn(0x561d6644d200 0x561d66455600 crc :-1 s=READY pgs=16 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:30:38.763 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:38.753+0000 7f08d56ce640 1 -- 172.21.15.37:0/2208611052 <== osd.1 v2:172.21.15.37:6800/1690341662 1 ==== osd_op_reply(9 [pgnls-filter start_epoch 44 out=49b] v36'8 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x561d66126780 con 0x561d6644d200 2024-04-25T10:30:38.763 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:38.753+0000 7f08d06c4640 1 -- 172.21.15.37:0/2208611052 --> [v2:172.21.15.37:6808/2754669649,v1:172.21.15.37:6809/2754669649] -- osd_op(unknown.0.0:10 5.38 5:1c000000::::head [pgnls-filter start_epoch 44 in=89b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e44) -- 0x561d66458400 con 0x561d6644c480 2024-04-25T10:30:38.763 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:38.753+0000 7f08d4ecd640 1 -- 172.21.15.37:0/2208611052 <== osd.2 v2:172.21.15.37:6808/2754669649 3 ==== osd_op_reply(10 [pgnls-filter start_epoch 44 out=49b] v36'9 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x561d66126000 con 0x561d6644c480 2024-04-25T10:30:38.763 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:38.754+0000 7f08d0ec5640 1 -- 172.21.15.37:0/2208611052 --> [v2:172.21.15.37:6816/2746074720,v1:172.21.15.37:6817/2746074720] -- osd_op(unknown.0.0:11 5.4 5:20000000::::head [pgnls-filter start_epoch 44 in=89b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e44) -- 0x561d66458c00 con 0x561d6644c900 2024-04-25T10:30:38.763 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:38.754+0000 7f08d5ecf640 1 -- 172.21.15.37:0/2208611052 <== osd.0 v2:172.21.15.37:6816/2746074720 2 ==== osd_op_reply(11 [pgnls-filter start_epoch 44 out=49b] v36'9 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x561d66126500 con 0x561d6644c900 2024-04-25T10:30:38.763 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:38.754+0000 7f08d06c4640 1 -- 172.21.15.37:0/2208611052 --> [v2:172.21.15.37:6808/2754669649,v1:172.21.15.37:6809/2754669649] -- osd_op(unknown.0.0:12 5.24 5:24000000::::head [pgnls-filter start_epoch 44 in=89b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e44) -- 0x561d66f94c00 con 0x561d6644c480 2024-04-25T10:30:38.763 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:38.754+0000 7f08d4ecd640 1 -- 172.21.15.37:0/2208611052 <== osd.2 v2:172.21.15.37:6808/2754669649 4 ==== osd_op_reply(12 [pgnls-filter start_epoch 44 out=49b] v36'4 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x561d66126000 con 0x561d6644c480 2024-04-25T10:30:38.763 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:38.754+0000 7f08d0ec5640 1 -- 172.21.15.37:0/2208611052 --> [v2:172.21.15.203:6800/4051532063,v1:172.21.15.203:6801/4051532063] -- osd_op(unknown.0.0:13 5.14 5:28000000::::head [pgnls-filter start_epoch 44 in=89b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e44) -- 0x561d66459800 con 0x561d661d8d00 2024-04-25T10:30:38.763 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:38.754+0000 7f08d4ecd640 1 -- 172.21.15.37:0/2208611052 <== osd.4 v2:172.21.15.203:6800/4051532063 3 ==== osd_op_reply(13 [pgnls-filter start_epoch 44 out=49b] v36'11 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x561d66126000 con 0x561d661d8d00 2024-04-25T10:30:38.763 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:38.754+0000 7f08d06c4640 1 -- 172.21.15.37:0/2208611052 --> [v2:172.21.15.37:6800/1690341662,v1:172.21.15.37:6801/1690341662] -- osd_op(unknown.0.0:14 5.34 5:2c000000::::head [pgnls-filter start_epoch 44 in=89b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e44) -- 0x561d66f95400 con 0x561d6644d200 2024-04-25T10:30:38.763 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:38.755+0000 7f08d56ce640 1 -- 172.21.15.37:0/2208611052 <== osd.1 v2:172.21.15.37:6800/1690341662 2 ==== osd_op_reply(14 [pgnls-filter start_epoch 44 out=49b] v36'7 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x561d66126780 con 0x561d6644d200 2024-04-25T10:30:38.763 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:38.755+0000 7f08d0ec5640 1 --2- 172.21.15.37:0/2208611052 >> [v2:172.21.15.203:6824/1203822890,v1:172.21.15.203:6825/1203822890] conn(0x561d6644d680 0x561d66455b80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:30:38.763 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:38.755+0000 7f08d0ec5640 1 -- 172.21.15.37:0/2208611052 --> [v2:172.21.15.203:6824/1203822890,v1:172.21.15.203:6825/1203822890] -- osd_op(unknown.0.0:15 5.c 5:30000000::::head [pgnls-filter start_epoch 44 in=89b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e44) -- 0x561d66459c00 con 0x561d6644d680 2024-04-25T10:30:38.763 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:38.755+0000 7f08d4ecd640 1 --2- 172.21.15.37:0/2208611052 >> [v2:172.21.15.203:6824/1203822890,v1:172.21.15.203:6825/1203822890] conn(0x561d6644d680 0x561d66455b80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:30:38.764 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:38.755+0000 7f08d4ecd640 1 --2- 172.21.15.37:0/2208611052 >> [v2:172.21.15.203:6824/1203822890,v1:172.21.15.203:6825/1203822890] conn(0x561d6644d680 0x561d66455b80 crc :-1 s=READY pgs=14 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.5 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:30:38.764 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:38.756+0000 7f08d4ecd640 1 -- 172.21.15.37:0/2208611052 <== osd.5 v2:172.21.15.203:6824/1203822890 1 ==== osd_op_reply(15 [pgnls-filter start_epoch 44 out=49b] v36'6 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x561d66126000 con 0x561d6644d680 2024-04-25T10:30:38.764 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:38.756+0000 7f08d06c4640 1 -- 172.21.15.37:0/2208611052 --> [v2:172.21.15.203:6810/2309615014,v1:172.21.15.203:6812/2309615014] -- osd_op(unknown.0.0:16 5.2c 5:34000000::::head [pgnls-filter start_epoch 44 in=89b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e44) -- 0x561d66458800 con 0x561d661d9180 2024-04-25T10:30:38.764 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:38.756+0000 7f08d5ecf640 1 -- 172.21.15.37:0/2208611052 <== osd.6 v2:172.21.15.203:6810/2309615014 3 ==== osd_op_reply(16 [pgnls-filter start_epoch 44 out=49b] v36'8 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x561d66126500 con 0x561d661d9180 2024-04-25T10:30:38.764 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:38.756+0000 7f08d0ec5640 1 -- 172.21.15.37:0/2208611052 --> [v2:172.21.15.37:6808/2754669649,v1:172.21.15.37:6809/2754669649] -- osd_op(unknown.0.0:17 5.1c 5:38000000::::head [pgnls-filter start_epoch 44 in=89b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e44) -- 0x561d66458c00 con 0x561d6644c480 2024-04-25T10:30:38.764 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:38.756+0000 7f08d4ecd640 1 -- 172.21.15.37:0/2208611052 <== osd.2 v2:172.21.15.37:6808/2754669649 5 ==== osd_op_reply(17 [pgnls-filter start_epoch 44 out=49b] v36'9 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x561d66126000 con 0x561d6644c480 2024-04-25T10:30:38.764 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:38.757+0000 7f08d06c4640 1 -- 172.21.15.37:0/2208611052 --> [v2:172.21.15.203:6800/4051532063,v1:172.21.15.203:6801/4051532063] -- osd_op(unknown.0.0:18 5.3c 5:3c000000::::head [pgnls-filter start_epoch 44 in=89b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e44) -- 0x561d66459000 con 0x561d661d8d00 2024-04-25T10:30:38.764 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:38.757+0000 7f08d4ecd640 1 -- 172.21.15.37:0/2208611052 <== osd.4 v2:172.21.15.203:6800/4051532063 4 ==== osd_op_reply(18 [pgnls-filter start_epoch 44 out=49b] v36'8 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x561d66126000 con 0x561d661d8d00 2024-04-25T10:30:38.764 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:38.757+0000 7f08d0ec5640 1 -- 172.21.15.37:0/2208611052 --> [v2:172.21.15.37:6816/2746074720,v1:172.21.15.37:6817/2746074720] -- osd_op(unknown.0.0:19 5.2 5:40000000::::head [pgnls-filter start_epoch 44 in=89b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e44) -- 0x561d66458400 con 0x561d6644c900 2024-04-25T10:30:38.764 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:38.757+0000 7f08d5ecf640 1 -- 172.21.15.37:0/2208611052 <== osd.0 v2:172.21.15.37:6816/2746074720 3 ==== osd_op_reply(19 [pgnls-filter start_epoch 44 out=49b] v36'8 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x561d66126500 con 0x561d6644c900 2024-04-25T10:30:38.764 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:38.757+0000 7f08d06c4640 1 -- 172.21.15.37:0/2208611052 --> [v2:172.21.15.37:6816/2746074720,v1:172.21.15.37:6817/2746074720] -- osd_op(unknown.0.0:20 5.22 5:44000000::::head [pgnls-filter start_epoch 44 in=89b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e44) -- 0x561d66f95000 con 0x561d6644c900 2024-04-25T10:30:38.764 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:38.757+0000 7f08d5ecf640 1 -- 172.21.15.37:0/2208611052 <== osd.0 v2:172.21.15.37:6816/2746074720 4 ==== osd_op_reply(20 [pgnls-filter start_epoch 44 out=49b] v36'8 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x561d66126500 con 0x561d6644c900 2024-04-25T10:30:38.764 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:38.757+0000 7f08d0ec5640 1 -- 172.21.15.37:0/2208611052 --> [v2:172.21.15.203:6810/2309615014,v1:172.21.15.203:6812/2309615014] -- osd_op(unknown.0.0:21 5.12 5:48000000::::head [pgnls-filter start_epoch 44 in=89b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e44) -- 0x561d66459800 con 0x561d661d9180 2024-04-25T10:30:38.775 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:38.758+0000 7f08d5ecf640 1 -- 172.21.15.37:0/2208611052 <== osd.6 v2:172.21.15.203:6810/2309615014 4 ==== osd_op_reply(21 [pgnls-filter start_epoch 44 out=49b] v36'10 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x561d66126500 con 0x561d661d9180 2024-04-25T10:30:38.775 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:38.758+0000 7f08d06c4640 1 -- 172.21.15.37:0/2208611052 --> [v2:172.21.15.203:6800/4051532063,v1:172.21.15.203:6801/4051532063] -- osd_op(unknown.0.0:22 5.32 5:4c000000::::head [pgnls-filter start_epoch 44 in=89b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e44) -- 0x561d66f19400 con 0x561d661d8d00 2024-04-25T10:30:38.775 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:38.758+0000 7f08d4ecd640 1 -- 172.21.15.37:0/2208611052 <== osd.4 v2:172.21.15.203:6800/4051532063 5 ==== osd_op_reply(22 [pgnls-filter start_epoch 44 out=49b] v36'8 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x561d66126000 con 0x561d661d8d00 2024-04-25T10:30:38.775 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:38.758+0000 7f08d0ec5640 1 -- 172.21.15.37:0/2208611052 --> [v2:172.21.15.203:6810/2309615014,v1:172.21.15.203:6812/2309615014] -- osd_op(unknown.0.0:23 5.a 5:50000000::::head [pgnls-filter start_epoch 44 in=89b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e44) -- 0x561d66459c00 con 0x561d661d9180 2024-04-25T10:30:38.775 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:38.758+0000 7f08d5ecf640 1 -- 172.21.15.37:0/2208611052 <== osd.6 v2:172.21.15.203:6810/2309615014 5 ==== osd_op_reply(23 [pgnls-filter start_epoch 44 out=49b] v36'5 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x561d66126500 con 0x561d661d9180 2024-04-25T10:30:38.775 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:38.758+0000 7f08d06c4640 1 -- 172.21.15.37:0/2208611052 --> [v2:172.21.15.37:6808/2754669649,v1:172.21.15.37:6809/2754669649] -- osd_op(unknown.0.0:24 5.2a 5:54000000::::head [pgnls-filter start_epoch 44 in=89b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e44) -- 0x561d66472000 con 0x561d6644c480 2024-04-25T10:30:38.775 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:38.759+0000 7f08d4ecd640 1 -- 172.21.15.37:0/2208611052 <== osd.2 v2:172.21.15.37:6808/2754669649 6 ==== osd_op_reply(24 [pgnls-filter start_epoch 44 out=49b] v36'9 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x561d66126000 con 0x561d6644c480 2024-04-25T10:30:38.775 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:38.759+0000 7f08d0ec5640 1 -- 172.21.15.37:0/2208611052 --> [v2:172.21.15.37:6800/1690341662,v1:172.21.15.37:6801/1690341662] -- osd_op(unknown.0.0:25 5.1a 5:58000000::::head [pgnls-filter start_epoch 44 in=89b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e44) -- 0x561d66458c00 con 0x561d6644d200 2024-04-25T10:30:38.775 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:38.759+0000 7f08d56ce640 1 -- 172.21.15.37:0/2208611052 <== osd.1 v2:172.21.15.37:6800/1690341662 3 ==== osd_op_reply(25 [pgnls-filter start_epoch 44 out=49b] v36'7 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x561d66126780 con 0x561d6644d200 2024-04-25T10:30:38.775 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:38.759+0000 7f08d06c4640 1 --2- 172.21.15.37:0/2208611052 >> [v2:172.21.15.203:6808/3688099014,v1:172.21.15.203:6809/3688099014] conn(0x561d6644cd80 0x561d66455080 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:30:38.775 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:38.759+0000 7f08d06c4640 1 -- 172.21.15.37:0/2208611052 --> [v2:172.21.15.203:6808/3688099014,v1:172.21.15.203:6809/3688099014] -- osd_op(unknown.0.0:26 5.3a 5:5c000000::::head [pgnls-filter start_epoch 44 in=89b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e44) -- 0x561d66472400 con 0x561d6644cd80 2024-04-25T10:30:38.776 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:38.759+0000 7f08d5ecf640 1 --2- 172.21.15.37:0/2208611052 >> [v2:172.21.15.203:6808/3688099014,v1:172.21.15.203:6809/3688099014] conn(0x561d6644cd80 0x561d66455080 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:30:38.776 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:38.760+0000 7f08d5ecf640 1 --2- 172.21.15.37:0/2208611052 >> [v2:172.21.15.203:6808/3688099014,v1:172.21.15.203:6809/3688099014] conn(0x561d6644cd80 0x561d66455080 crc :-1 s=READY pgs=16 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.7 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:30:38.776 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:38.760+0000 7f08d5ecf640 1 -- 172.21.15.37:0/2208611052 <== osd.7 v2:172.21.15.203:6808/3688099014 1 ==== osd_op_reply(26 [pgnls-filter start_epoch 44 out=49b] v36'6 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x561d66126500 con 0x561d6644cd80 2024-04-25T10:30:38.776 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:38.760+0000 7f08d0ec5640 1 -- 172.21.15.37:0/2208611052 --> [v2:172.21.15.203:6808/3688099014,v1:172.21.15.203:6809/3688099014] -- osd_op(unknown.0.0:27 5.6 5:60000000::::head [pgnls-filter start_epoch 44 in=89b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e44) -- 0x561d66f94c00 con 0x561d6644cd80 2024-04-25T10:30:38.776 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:38.760+0000 7f08d5ecf640 1 -- 172.21.15.37:0/2208611052 <== osd.7 v2:172.21.15.203:6808/3688099014 2 ==== osd_op_reply(27 [pgnls-filter start_epoch 44 out=49b] v36'7 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x561d66126500 con 0x561d6644cd80 2024-04-25T10:30:38.776 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:38.760+0000 7f08d06c4640 1 -- 172.21.15.37:0/2208611052 --> [v2:172.21.15.203:6800/4051532063,v1:172.21.15.203:6801/4051532063] -- osd_op(unknown.0.0:28 5.26 5:64000000::::head [pgnls-filter start_epoch 44 in=89b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e44) -- 0x561d66472800 con 0x561d661d8d00 2024-04-25T10:30:38.776 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:38.761+0000 7f08d4ecd640 1 -- 172.21.15.37:0/2208611052 <== osd.4 v2:172.21.15.203:6800/4051532063 6 ==== osd_op_reply(28 [pgnls-filter start_epoch 44 out=49b] v36'15 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x561d66126000 con 0x561d661d8d00 2024-04-25T10:30:38.776 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:38.761+0000 7f08d0ec5640 1 -- 172.21.15.37:0/2208611052 --> [v2:172.21.15.37:6824/1815290618,v1:172.21.15.37:6825/1815290618] -- osd_op(unknown.0.0:29 5.16 5:68000000::::head [pgnls-filter start_epoch 44 in=89b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e44) -- 0x561d66472400 con 0x561d6644c000 2024-04-25T10:30:38.776 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:38.761+0000 7f08d56ce640 1 -- 172.21.15.37:0/2208611052 <== osd.3 v2:172.21.15.37:6824/1815290618 2 ==== osd_op_reply(29 [pgnls-filter start_epoch 44 out=49b] v36'10 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x561d66126780 con 0x561d6644c000 2024-04-25T10:30:38.776 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:38.761+0000 7f08d06c4640 1 -- 172.21.15.37:0/2208611052 --> [v2:172.21.15.203:6800/4051532063,v1:172.21.15.203:6801/4051532063] -- osd_op(unknown.0.0:30 5.36 5:6c000000::::head [pgnls-filter start_epoch 44 in=89b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e44) -- 0x561d66472c00 con 0x561d661d8d00 2024-04-25T10:30:38.776 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:38.761+0000 7f08d4ecd640 1 -- 172.21.15.37:0/2208611052 <== osd.4 v2:172.21.15.203:6800/4051532063 7 ==== osd_op_reply(30 [pgnls-filter start_epoch 44 out=49b] v36'15 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x561d66126000 con 0x561d661d8d00 2024-04-25T10:30:38.776 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:38.761+0000 7f08d0ec5640 1 -- 172.21.15.37:0/2208611052 --> [v2:172.21.15.37:6824/1815290618,v1:172.21.15.37:6825/1815290618] -- osd_op(unknown.0.0:31 5.e 5:70000000::::head [pgnls-filter start_epoch 44 in=89b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e44) -- 0x561d66458800 con 0x561d6644c000 2024-04-25T10:30:38.776 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:38.761+0000 7f08d56ce640 1 -- 172.21.15.37:0/2208611052 <== osd.3 v2:172.21.15.37:6824/1815290618 3 ==== osd_op_reply(31 [pgnls-filter start_epoch 44 out=49b] v36'5 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x561d66126780 con 0x561d6644c000 2024-04-25T10:30:38.776 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:38.762+0000 7f08d06c4640 1 -- 172.21.15.37:0/2208611052 --> [v2:172.21.15.203:6808/3688099014,v1:172.21.15.203:6809/3688099014] -- osd_op(unknown.0.0:32 5.2e 5:74000000::::head [pgnls-filter start_epoch 44 in=89b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e44) -- 0x561d66473000 con 0x561d6644cd80 2024-04-25T10:30:38.777 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:38.762+0000 7f08d5ecf640 1 -- 172.21.15.37:0/2208611052 <== osd.7 v2:172.21.15.203:6808/3688099014 3 ==== osd_op_reply(32 [pgnls-filter start_epoch 44 out=49b] v36'10 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x561d66126500 con 0x561d6644cd80 2024-04-25T10:30:38.777 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:38.762+0000 7f08d0ec5640 1 -- 172.21.15.37:0/2208611052 --> [v2:172.21.15.203:6810/2309615014,v1:172.21.15.203:6812/2309615014] -- osd_op(unknown.0.0:33 5.1e 5:78000000::::head [pgnls-filter start_epoch 44 in=89b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e44) -- 0x561d66458400 con 0x561d661d9180 2024-04-25T10:30:38.777 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:38.762+0000 7f08d5ecf640 1 -- 172.21.15.37:0/2208611052 <== osd.6 v2:172.21.15.203:6810/2309615014 6 ==== osd_op_reply(33 [pgnls-filter start_epoch 44 out=49b] v36'5 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x561d66126500 con 0x561d661d9180 2024-04-25T10:30:38.777 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:38.762+0000 7f08d06c4640 1 -- 172.21.15.37:0/2208611052 --> [v2:172.21.15.203:6824/1203822890,v1:172.21.15.203:6825/1203822890] -- osd_op(unknown.0.0:34 5.3e 5:7c000000::::head [pgnls-filter start_epoch 44 in=89b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e44) -- 0x561d66472400 con 0x561d6644d680 2024-04-25T10:30:38.777 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:38.763+0000 7f08d4ecd640 1 -- 172.21.15.37:0/2208611052 <== osd.5 v2:172.21.15.203:6824/1203822890 2 ==== osd_op_reply(34 [pgnls-filter start_epoch 44 out=49b] v36'4 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x561d66126000 con 0x561d6644d680 2024-04-25T10:30:38.777 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:38.763+0000 7f08d0ec5640 1 -- 172.21.15.37:0/2208611052 --> [v2:172.21.15.37:6824/1815290618,v1:172.21.15.37:6825/1815290618] -- osd_op(unknown.0.0:35 5.1 5:80000000::::head [pgnls-filter start_epoch 44 in=89b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e44) -- 0x561d66459800 con 0x561d6644c000 2024-04-25T10:30:38.777 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:38.763+0000 7f08d56ce640 1 -- 172.21.15.37:0/2208611052 <== osd.3 v2:172.21.15.37:6824/1815290618 4 ==== osd_op_reply(35 [pgnls-filter start_epoch 44 out=49b] v36'5 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x561d66126780 con 0x561d6644c000 2024-04-25T10:30:38.777 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:38.763+0000 7f08d06c4640 1 -- 172.21.15.37:0/2208611052 --> [v2:172.21.15.37:6808/2754669649,v1:172.21.15.37:6809/2754669649] -- osd_op(unknown.0.0:36 5.21 5:84000000::::head [pgnls-filter start_epoch 44 in=89b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e44) -- 0x561d66473400 con 0x561d6644c480 2024-04-25T10:30:38.777 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:38.763+0000 7f08d4ecd640 1 -- 172.21.15.37:0/2208611052 <== osd.2 v2:172.21.15.37:6808/2754669649 7 ==== osd_op_reply(36 [pgnls-filter start_epoch 44 out=49b] v36'12 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x561d66126000 con 0x561d6644c480 2024-04-25T10:30:38.777 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:38.763+0000 7f08d0ec5640 1 -- 172.21.15.37:0/2208611052 --> [v2:172.21.15.37:6824/1815290618,v1:172.21.15.37:6825/1815290618] -- osd_op(unknown.0.0:37 5.11 5:88000000::::head [pgnls-filter start_epoch 44 in=89b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e44) -- 0x561d66459c00 con 0x561d6644c000 2024-04-25T10:30:38.777 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:38.763+0000 7f08d56ce640 1 -- 172.21.15.37:0/2208611052 <== osd.3 v2:172.21.15.37:6824/1815290618 5 ==== osd_op_reply(37 [pgnls-filter start_epoch 44 out=49b] v36'6 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x561d66126780 con 0x561d6644c000 2024-04-25T10:30:38.777 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:38.763+0000 7f08d06c4640 1 -- 172.21.15.37:0/2208611052 --> [v2:172.21.15.37:6816/2746074720,v1:172.21.15.37:6817/2746074720] -- osd_op(unknown.0.0:38 5.31 5:8c000000::::head [pgnls-filter start_epoch 44 in=89b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e44) -- 0x561d66458000 con 0x561d6644c900 2024-04-25T10:30:38.777 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:38.764+0000 7f08d5ecf640 1 -- 172.21.15.37:0/2208611052 <== osd.0 v2:172.21.15.37:6816/2746074720 5 ==== osd_op_reply(38 [pgnls-filter start_epoch 44 out=49b] v36'7 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x561d66126500 con 0x561d6644c900 2024-04-25T10:30:38.777 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:38.764+0000 7f08d0ec5640 1 -- 172.21.15.37:0/2208611052 --> [v2:172.21.15.37:6800/1690341662,v1:172.21.15.37:6801/1690341662] -- osd_op(unknown.0.0:39 5.9 5:90000000::::head [pgnls-filter start_epoch 44 in=89b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e44) -- 0x561d66f95000 con 0x561d6644d200 2024-04-25T10:30:38.778 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:38.764+0000 7f08d56ce640 1 -- 172.21.15.37:0/2208611052 <== osd.1 v2:172.21.15.37:6800/1690341662 4 ==== osd_op_reply(39 [pgnls-filter start_epoch 44 out=49b] v36'2 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x561d66126780 con 0x561d6644d200 2024-04-25T10:30:38.778 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:38.764+0000 7f08d06c4640 1 -- 172.21.15.37:0/2208611052 --> [v2:172.21.15.203:6810/2309615014,v1:172.21.15.203:6812/2309615014] -- osd_op(unknown.0.0:40 5.29 5:94000000::::head [pgnls-filter start_epoch 44 in=89b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e44) -- 0x561d66459400 con 0x561d661d9180 2024-04-25T10:30:38.778 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:38.764+0000 7f08d5ecf640 1 -- 172.21.15.37:0/2208611052 <== osd.6 v2:172.21.15.203:6810/2309615014 7 ==== osd_op_reply(40 [pgnls-filter start_epoch 44 out=49b] v36'8 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x561d66126500 con 0x561d661d9180 2024-04-25T10:30:38.778 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:38.764+0000 7f08d0ec5640 1 -- 172.21.15.37:0/2208611052 --> [v2:172.21.15.203:6800/4051532063,v1:172.21.15.203:6801/4051532063] -- osd_op(unknown.0.0:41 5.19 5:98000000::::head [pgnls-filter start_epoch 44 in=89b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e44) -- 0x561d66472000 con 0x561d661d8d00 2024-04-25T10:30:38.778 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:38.765+0000 7f08d4ecd640 1 -- 172.21.15.37:0/2208611052 <== osd.4 v2:172.21.15.203:6800/4051532063 8 ==== osd_op_reply(41 [pgnls-filter start_epoch 44 out=49b] v36'10 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x561d66126000 con 0x561d661d8d00 2024-04-25T10:30:38.778 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:38.765+0000 7f08d06c4640 1 -- 172.21.15.37:0/2208611052 --> [v2:172.21.15.37:6808/2754669649,v1:172.21.15.37:6809/2754669649] -- osd_op(unknown.0.0:42 5.39 5:9c000000::::head [pgnls-filter start_epoch 44 in=89b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e44) -- 0x561d66458c00 con 0x561d6644c480 2024-04-25T10:30:38.778 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:38.765+0000 7f08d4ecd640 1 -- 172.21.15.37:0/2208611052 <== osd.2 v2:172.21.15.37:6808/2754669649 8 ==== osd_op_reply(42 [pgnls-filter start_epoch 44 out=49b] v36'5 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x561d66126000 con 0x561d6644c480 2024-04-25T10:30:38.778 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:38.765+0000 7f08d0ec5640 1 -- 172.21.15.37:0/2208611052 --> [v2:172.21.15.203:6808/3688099014,v1:172.21.15.203:6809/3688099014] -- osd_op(unknown.0.0:43 5.5 5:a0000000::::head [pgnls-filter start_epoch 44 in=89b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e44) -- 0x561d66472800 con 0x561d6644cd80 2024-04-25T10:30:38.778 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:38.765+0000 7f08d5ecf640 1 -- 172.21.15.37:0/2208611052 <== osd.7 v2:172.21.15.203:6808/3688099014 4 ==== osd_op_reply(43 [pgnls-filter start_epoch 44 out=49b] v36'5 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x561d66126500 con 0x561d6644cd80 2024-04-25T10:30:38.778 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:38.765+0000 7f08d06c4640 1 -- 172.21.15.37:0/2208611052 --> [v2:172.21.15.203:6824/1203822890,v1:172.21.15.203:6825/1203822890] -- osd_op(unknown.0.0:44 5.25 5:a4000000::::head [pgnls-filter start_epoch 44 in=89b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e44) -- 0x561d66f95400 con 0x561d6644d680 2024-04-25T10:30:38.778 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:38.766+0000 7f08d4ecd640 1 -- 172.21.15.37:0/2208611052 <== osd.5 v2:172.21.15.203:6824/1203822890 3 ==== osd_op_reply(44 [pgnls-filter start_epoch 44 out=49b] v36'6 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x561d66126000 con 0x561d6644d680 2024-04-25T10:30:38.778 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:38.766+0000 7f08d0ec5640 1 -- 172.21.15.37:0/2208611052 --> [v2:172.21.15.37:6824/1815290618,v1:172.21.15.37:6825/1815290618] -- osd_op(unknown.0.0:45 5.15 5:a8000000::::head [pgnls-filter start_epoch 44 in=89b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e44) -- 0x561d66472c00 con 0x561d6644c000 2024-04-25T10:30:38.778 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:38.766+0000 7f08d56ce640 1 -- 172.21.15.37:0/2208611052 <== osd.3 v2:172.21.15.37:6824/1815290618 6 ==== osd_op_reply(45 [pgnls-filter start_epoch 44 out=49b] v36'6 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x561d66126780 con 0x561d6644c000 2024-04-25T10:30:38.778 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:38.766+0000 7f08d06c4640 1 -- 172.21.15.37:0/2208611052 --> [v2:172.21.15.37:6824/1815290618,v1:172.21.15.37:6825/1815290618] -- osd_op(unknown.0.0:46 5.35 5:ac000000::::head [pgnls-filter start_epoch 44 in=89b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e44) -- 0x561d66473c00 con 0x561d6644c000 2024-04-25T10:30:38.779 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:38.766+0000 7f08d56ce640 1 -- 172.21.15.37:0/2208611052 <== osd.3 v2:172.21.15.37:6824/1815290618 7 ==== osd_op_reply(46 [pgnls-filter start_epoch 44 out=49b] v36'6 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x561d66126780 con 0x561d6644c000 2024-04-25T10:30:38.779 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:38.766+0000 7f08d0ec5640 1 -- 172.21.15.37:0/2208611052 --> [v2:172.21.15.203:6808/3688099014,v1:172.21.15.203:6809/3688099014] -- osd_op(unknown.0.0:47 5.d 5:b0000000::::head [pgnls-filter start_epoch 44 in=89b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e44) -- 0x561d66472400 con 0x561d6644cd80 2024-04-25T10:30:38.779 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:38.767+0000 7f08d5ecf640 1 -- 172.21.15.37:0/2208611052 <== osd.7 v2:172.21.15.203:6808/3688099014 5 ==== osd_op_reply(47 [pgnls-filter start_epoch 44 out=49b] v36'7 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x561d66126500 con 0x561d6644cd80 2024-04-25T10:30:38.779 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:38.767+0000 7f08d06c4640 1 -- 172.21.15.37:0/2208611052 --> [v2:172.21.15.203:6808/3688099014,v1:172.21.15.203:6809/3688099014] -- osd_op(unknown.0.0:48 5.2d 5:b4000000::::head [pgnls-filter start_epoch 44 in=89b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e44) -- 0x561d66f94c00 con 0x561d6644cd80 2024-04-25T10:30:38.779 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:38.767+0000 7f08d5ecf640 1 -- 172.21.15.37:0/2208611052 <== osd.7 v2:172.21.15.203:6808/3688099014 6 ==== osd_op_reply(48 [pgnls-filter start_epoch 44 out=49b] v36'7 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x561d66126500 con 0x561d6644cd80 2024-04-25T10:30:38.779 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:38.767+0000 7f08d0ec5640 1 -- 172.21.15.37:0/2208611052 --> [v2:172.21.15.203:6800/4051532063,v1:172.21.15.203:6801/4051532063] -- osd_op(unknown.0.0:49 5.1d 5:b8000000::::head [pgnls-filter start_epoch 44 in=89b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e44) -- 0x561d66473800 con 0x561d661d8d00 2024-04-25T10:30:38.779 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:38.767+0000 7f08d4ecd640 1 -- 172.21.15.37:0/2208611052 <== osd.4 v2:172.21.15.203:6800/4051532063 9 ==== osd_op_reply(49 [pgnls-filter start_epoch 44 out=49b] v36'15 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x561d66126000 con 0x561d661d8d00 2024-04-25T10:30:38.779 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:38.767+0000 7f08d06c4640 1 -- 172.21.15.37:0/2208611052 --> [v2:172.21.15.37:6800/1690341662,v1:172.21.15.37:6801/1690341662] -- osd_op(unknown.0.0:50 5.3d 5:bc000000::::head [pgnls-filter start_epoch 44 in=89b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e44) -- 0x561d66f95000 con 0x561d6644d200 2024-04-25T10:30:38.779 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:38.767+0000 7f08d56ce640 1 -- 172.21.15.37:0/2208611052 <== osd.1 v2:172.21.15.37:6800/1690341662 5 ==== osd_op_reply(50 [pgnls-filter start_epoch 44 out=49b] v36'7 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x561d66126780 con 0x561d6644d200 2024-04-25T10:30:38.779 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:38.767+0000 7f08d0ec5640 1 -- 172.21.15.37:0/2208611052 --> [v2:172.21.15.203:6800/4051532063,v1:172.21.15.203:6801/4051532063] -- osd_op(unknown.0.0:51 5.3 5:c0000000::::head [pgnls-filter start_epoch 44 in=89b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e44) -- 0x561d66f19400 con 0x561d661d8d00 2024-04-25T10:30:38.779 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:38.768+0000 7f08d4ecd640 1 -- 172.21.15.37:0/2208611052 <== osd.4 v2:172.21.15.203:6800/4051532063 10 ==== osd_op_reply(51 [pgnls-filter start_epoch 44 out=49b] v36'7 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x561d66126000 con 0x561d661d8d00 2024-04-25T10:30:38.779 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:38.768+0000 7f08d06c4640 1 -- 172.21.15.37:0/2208611052 --> [v2:172.21.15.37:6816/2746074720,v1:172.21.15.37:6817/2746074720] -- osd_op(unknown.0.0:52 5.23 5:c4000000::::head [pgnls-filter start_epoch 44 in=89b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e44) -- 0x561d66458400 con 0x561d6644c900 2024-04-25T10:30:38.779 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:38.768+0000 7f08d5ecf640 1 -- 172.21.15.37:0/2208611052 <== osd.0 v2:172.21.15.37:6816/2746074720 6 ==== osd_op_reply(52 [pgnls-filter start_epoch 44 out=49b] v36'7 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x561d66126500 con 0x561d6644c900 2024-04-25T10:30:38.779 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:38.768+0000 7f08d0ec5640 1 -- 172.21.15.37:0/2208611052 --> [v2:172.21.15.203:6800/4051532063,v1:172.21.15.203:6801/4051532063] -- osd_op(unknown.0.0:53 5.13 5:c8000000::::head [pgnls-filter start_epoch 44 in=89b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e44) -- 0x561d66459000 con 0x561d661d8d00 2024-04-25T10:30:38.779 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:38.768+0000 7f08d4ecd640 1 -- 172.21.15.37:0/2208611052 <== osd.4 v2:172.21.15.203:6800/4051532063 11 ==== osd_op_reply(53 [pgnls-filter start_epoch 44 out=49b] v36'15 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x561d66126000 con 0x561d661d8d00 2024-04-25T10:30:38.780 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:38.768+0000 7f08d06c4640 1 -- 172.21.15.37:0/2208611052 --> [v2:172.21.15.203:6810/2309615014,v1:172.21.15.203:6812/2309615014] -- osd_op(unknown.0.0:54 5.33 5:cc000000::::head [pgnls-filter start_epoch 44 in=89b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e44) -- 0x561d66458000 con 0x561d661d9180 2024-04-25T10:30:38.780 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:38.769+0000 7f08d5ecf640 1 -- 172.21.15.37:0/2208611052 <== osd.6 v2:172.21.15.203:6810/2309615014 8 ==== osd_op_reply(54 [pgnls-filter start_epoch 44 out=49b] v36'10 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x561d66126500 con 0x561d661d9180 2024-04-25T10:30:38.780 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:38.769+0000 7f08d0ec5640 1 -- 172.21.15.37:0/2208611052 --> [v2:172.21.15.203:6824/1203822890,v1:172.21.15.203:6825/1203822890] -- osd_op(unknown.0.0:55 5.b 5:d0000000::::head [pgnls-filter start_epoch 44 in=89b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e44) -- 0x561d66458c00 con 0x561d6644d680 2024-04-25T10:30:38.780 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:38.769+0000 7f08d4ecd640 1 -- 172.21.15.37:0/2208611052 <== osd.5 v2:172.21.15.203:6824/1203822890 4 ==== osd_op_reply(55 [pgnls-filter start_epoch 44 out=49b] v36'8 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x561d66126000 con 0x561d6644d680 2024-04-25T10:30:38.780 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:38.769+0000 7f08d06c4640 1 -- 172.21.15.37:0/2208611052 --> [v2:172.21.15.37:6816/2746074720,v1:172.21.15.37:6817/2746074720] -- osd_op(unknown.0.0:56 5.2b 5:d4000000::::head [pgnls-filter start_epoch 44 in=89b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e44) -- 0x561d66459400 con 0x561d6644c900 2024-04-25T10:30:38.780 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:38.769+0000 7f08d5ecf640 1 -- 172.21.15.37:0/2208611052 <== osd.0 v2:172.21.15.37:6816/2746074720 7 ==== osd_op_reply(56 [pgnls-filter start_epoch 44 out=49b] v36'11 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x561d66126500 con 0x561d6644c900 2024-04-25T10:30:38.780 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:38.769+0000 7f08d0ec5640 1 -- 172.21.15.37:0/2208611052 --> [v2:172.21.15.37:6808/2754669649,v1:172.21.15.37:6809/2754669649] -- osd_op(unknown.0.0:57 5.1b 5:d8000000::::head [pgnls-filter start_epoch 44 in=89b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e44) -- 0x561d66459000 con 0x561d6644c480 2024-04-25T10:30:38.780 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:38.770+0000 7f08d4ecd640 1 -- 172.21.15.37:0/2208611052 <== osd.2 v2:172.21.15.37:6808/2754669649 9 ==== osd_op_reply(57 [pgnls-filter start_epoch 44 out=49b] v36'8 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x561d66126000 con 0x561d6644c480 2024-04-25T10:30:38.780 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:38.770+0000 7f08d06c4640 1 -- 172.21.15.37:0/2208611052 --> [v2:172.21.15.203:6824/1203822890,v1:172.21.15.203:6825/1203822890] -- osd_op(unknown.0.0:58 5.3b 5:dc000000::::head [pgnls-filter start_epoch 44 in=89b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e44) -- 0x561d66458800 con 0x561d6644d680 2024-04-25T10:30:38.780 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:38.770+0000 7f08d4ecd640 1 -- 172.21.15.37:0/2208611052 <== osd.5 v2:172.21.15.203:6824/1203822890 5 ==== osd_op_reply(58 [pgnls-filter start_epoch 44 out=49b] v36'4 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x561d66126000 con 0x561d6644d680 2024-04-25T10:30:38.780 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:38.770+0000 7f08d0ec5640 1 -- 172.21.15.37:0/2208611052 --> [v2:172.21.15.37:6824/1815290618,v1:172.21.15.37:6825/1815290618] -- osd_op(unknown.0.0:59 5.7 5:e0000000::::head [pgnls-filter start_epoch 44 in=89b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e44) -- 0x561d66f19400 con 0x561d6644c000 2024-04-25T10:30:38.780 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:38.770+0000 7f08d56ce640 1 -- 172.21.15.37:0/2208611052 <== osd.3 v2:172.21.15.37:6824/1815290618 8 ==== osd_op_reply(59 [pgnls-filter start_epoch 44 out=81b] v44'10 uv10 ondisk = 1) ==== 144+0+81 (crc 0 0 0) 0x561d66126780 con 0x561d6644c000 2024-04-25T10:30:38.780 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:38.770+0000 7f08d7a22740 1 -- 172.21.15.37:0/2208611052 --> [v2:172.21.15.203:6800/4051532063,v1:172.21.15.203:6801/4051532063] -- osd_op(unknown.0.0:60 5.0 5:00000000::::head [pgnls-filter start_epoch 44 in=85b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e44) -- 0x561d66f95800 con 0x561d661d8d00 2024-04-25T10:30:38.780 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:38.771+0000 7f08d4ecd640 1 -- 172.21.15.37:0/2208611052 <== osd.4 v2:172.21.15.203:6800/4051532063 12 ==== osd_op_reply(60 [pgnls-filter start_epoch 44 out=49b] v36'8 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x561d66126000 con 0x561d661d8d00 2024-04-25T10:30:38.781 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:38.771+0000 7f08d0ec5640 1 -- 172.21.15.37:0/2208611052 --> [v2:172.21.15.37:6824/1815290618,v1:172.21.15.37:6825/1815290618] -- osd_op(unknown.0.0:61 5.20 5:04000000::::head [pgnls-filter start_epoch 44 in=85b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e44) -- 0x561d66f95400 con 0x561d6644c000 2024-04-25T10:30:38.781 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:38.771+0000 7f08d56ce640 1 -- 172.21.15.37:0/2208611052 <== osd.3 v2:172.21.15.37:6824/1815290618 9 ==== osd_op_reply(61 [pgnls-filter start_epoch 44 out=49b] v36'9 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x561d66126780 con 0x561d6644c000 2024-04-25T10:30:38.781 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:38.771+0000 7f08d06c4640 1 -- 172.21.15.37:0/2208611052 --> [v2:172.21.15.37:6808/2754669649,v1:172.21.15.37:6809/2754669649] -- osd_op(unknown.0.0:62 5.10 5:08000000::::head [pgnls-filter start_epoch 44 in=85b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e44) -- 0x561d66459800 con 0x561d6644c480 2024-04-25T10:30:38.781 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:38.771+0000 7f08d4ecd640 1 -- 172.21.15.37:0/2208611052 <== osd.2 v2:172.21.15.37:6808/2754669649 10 ==== osd_op_reply(62 [pgnls-filter start_epoch 44 out=49b] v36'6 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x561d66126000 con 0x561d6644c480 2024-04-25T10:30:38.781 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:38.771+0000 7f08d0ec5640 1 -- 172.21.15.37:0/2208611052 --> [v2:172.21.15.37:6816/2746074720,v1:172.21.15.37:6817/2746074720] -- osd_op(unknown.0.0:63 5.30 5:0c000000::::head [pgnls-filter start_epoch 44 in=85b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e44) -- 0x561d66472000 con 0x561d6644c900 2024-04-25T10:30:38.781 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:38.771+0000 7f08d5ecf640 1 -- 172.21.15.37:0/2208611052 <== osd.0 v2:172.21.15.37:6816/2746074720 8 ==== osd_op_reply(63 [pgnls-filter start_epoch 44 out=49b] v36'9 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x561d66126500 con 0x561d6644c900 2024-04-25T10:30:38.781 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:38.772+0000 7f08d06c4640 1 -- 172.21.15.37:0/2208611052 --> [v2:172.21.15.203:6810/2309615014,v1:172.21.15.203:6812/2309615014] -- osd_op(unknown.0.0:64 5.8 5:10000000::::head [pgnls-filter start_epoch 44 in=85b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e44) -- 0x561d66459c00 con 0x561d661d9180 2024-04-25T10:30:38.781 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:38.772+0000 7f08d5ecf640 1 -- 172.21.15.37:0/2208611052 <== osd.6 v2:172.21.15.203:6810/2309615014 9 ==== osd_op_reply(64 [pgnls-filter start_epoch 44 out=49b] v36'9 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x561d66126500 con 0x561d661d9180 2024-04-25T10:30:38.781 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:38.772+0000 7f08d0ec5640 1 -- 172.21.15.37:0/2208611052 --> [v2:172.21.15.37:6808/2754669649,v1:172.21.15.37:6809/2754669649] -- osd_op(unknown.0.0:65 5.28 5:14000000::::head [pgnls-filter start_epoch 44 in=85b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e44) -- 0x561d66472400 con 0x561d6644c480 2024-04-25T10:30:38.781 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:38.772+0000 7f08d4ecd640 1 -- 172.21.15.37:0/2208611052 <== osd.2 v2:172.21.15.37:6808/2754669649 11 ==== osd_op_reply(65 [pgnls-filter start_epoch 44 out=49b] v36'6 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x561d66126000 con 0x561d6644c480 2024-04-25T10:30:38.781 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:38.772+0000 7f08d06c4640 1 -- 172.21.15.37:0/2208611052 --> [v2:172.21.15.37:6800/1690341662,v1:172.21.15.37:6801/1690341662] -- osd_op(unknown.0.0:66 5.18 5:18000000::::head [pgnls-filter start_epoch 44 in=85b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e44) -- 0x561d6647c000 con 0x561d6644d200 2024-04-25T10:30:38.781 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:38.772+0000 7f08d56ce640 1 -- 172.21.15.37:0/2208611052 <== osd.1 v2:172.21.15.37:6800/1690341662 6 ==== osd_op_reply(66 [pgnls-filter start_epoch 44 out=49b] v36'8 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x561d66126780 con 0x561d6644d200 2024-04-25T10:30:38.781 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:38.772+0000 7f08d0ec5640 1 -- 172.21.15.37:0/2208611052 --> [v2:172.21.15.37:6808/2754669649,v1:172.21.15.37:6809/2754669649] -- osd_op(unknown.0.0:67 5.38 5:1c000000::::head [pgnls-filter start_epoch 44 in=85b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e44) -- 0x561d66472800 con 0x561d6644c480 2024-04-25T10:30:38.781 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:38.773+0000 7f08d4ecd640 1 -- 172.21.15.37:0/2208611052 <== osd.2 v2:172.21.15.37:6808/2754669649 12 ==== osd_op_reply(67 [pgnls-filter start_epoch 44 out=49b] v36'9 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x561d66126000 con 0x561d6644c480 2024-04-25T10:30:38.781 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:38.773+0000 7f08d06c4640 1 -- 172.21.15.37:0/2208611052 --> [v2:172.21.15.37:6816/2746074720,v1:172.21.15.37:6817/2746074720] -- osd_op(unknown.0.0:68 5.4 5:20000000::::head [pgnls-filter start_epoch 44 in=85b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e44) -- 0x561d6647c400 con 0x561d6644c900 2024-04-25T10:30:38.793 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:38.773+0000 7f08d5ecf640 1 -- 172.21.15.37:0/2208611052 <== osd.0 v2:172.21.15.37:6816/2746074720 9 ==== osd_op_reply(68 [pgnls-filter start_epoch 44 out=49b] v36'9 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x561d66126500 con 0x561d6644c900 2024-04-25T10:30:38.793 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:38.773+0000 7f08d0ec5640 1 -- 172.21.15.37:0/2208611052 --> [v2:172.21.15.37:6808/2754669649,v1:172.21.15.37:6809/2754669649] -- osd_op(unknown.0.0:69 5.24 5:24000000::::head [pgnls-filter start_epoch 44 in=85b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e44) -- 0x561d66472c00 con 0x561d6644c480 2024-04-25T10:30:38.793 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:38.773+0000 7f08d4ecd640 1 -- 172.21.15.37:0/2208611052 <== osd.2 v2:172.21.15.37:6808/2754669649 13 ==== osd_op_reply(69 [pgnls-filter start_epoch 44 out=49b] v36'4 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x561d66126000 con 0x561d6644c480 2024-04-25T10:30:38.793 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:38.773+0000 7f08d06c4640 1 -- 172.21.15.37:0/2208611052 --> [v2:172.21.15.203:6800/4051532063,v1:172.21.15.203:6801/4051532063] -- osd_op(unknown.0.0:70 5.14 5:28000000::::head [pgnls-filter start_epoch 44 in=85b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e44) -- 0x561d6647c800 con 0x561d661d8d00 2024-04-25T10:30:38.793 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:38.774+0000 7f08d4ecd640 1 -- 172.21.15.37:0/2208611052 <== osd.4 v2:172.21.15.203:6800/4051532063 13 ==== osd_op_reply(70 [pgnls-filter start_epoch 44 out=49b] v36'11 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x561d66126000 con 0x561d661d8d00 2024-04-25T10:30:38.793 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:38.774+0000 7f08d0ec5640 1 -- 172.21.15.37:0/2208611052 --> [v2:172.21.15.37:6800/1690341662,v1:172.21.15.37:6801/1690341662] -- osd_op(unknown.0.0:71 5.34 5:2c000000::::head [pgnls-filter start_epoch 44 in=85b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e44) -- 0x561d66473000 con 0x561d6644d200 2024-04-25T10:30:38.793 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:38.774+0000 7f08d56ce640 1 -- 172.21.15.37:0/2208611052 <== osd.1 v2:172.21.15.37:6800/1690341662 7 ==== osd_op_reply(71 [pgnls-filter start_epoch 44 out=49b] v36'7 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x561d66126780 con 0x561d6644d200 2024-04-25T10:30:38.793 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:38.774+0000 7f08d06c4640 1 -- 172.21.15.37:0/2208611052 --> [v2:172.21.15.203:6824/1203822890,v1:172.21.15.203:6825/1203822890] -- osd_op(unknown.0.0:72 5.c 5:30000000::::head [pgnls-filter start_epoch 44 in=85b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e44) -- 0x561d6647cc00 con 0x561d6644d680 2024-04-25T10:30:38.794 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:38.774+0000 7f08d4ecd640 1 -- 172.21.15.37:0/2208611052 <== osd.5 v2:172.21.15.203:6824/1203822890 6 ==== osd_op_reply(72 [pgnls-filter start_epoch 44 out=49b] v36'6 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x561d66126000 con 0x561d6644d680 2024-04-25T10:30:38.794 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:38.774+0000 7f08d0ec5640 1 -- 172.21.15.37:0/2208611052 --> [v2:172.21.15.203:6810/2309615014,v1:172.21.15.203:6812/2309615014] -- osd_op(unknown.0.0:73 5.2c 5:34000000::::head [pgnls-filter start_epoch 44 in=85b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e44) -- 0x561d66473400 con 0x561d661d9180 2024-04-25T10:30:38.794 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:38.775+0000 7f08d5ecf640 1 -- 172.21.15.37:0/2208611052 <== osd.6 v2:172.21.15.203:6810/2309615014 10 ==== osd_op_reply(73 [pgnls-filter start_epoch 44 out=49b] v36'8 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x561d66126500 con 0x561d661d9180 2024-04-25T10:30:38.794 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:38.775+0000 7f08d06c4640 1 -- 172.21.15.37:0/2208611052 --> [v2:172.21.15.37:6808/2754669649,v1:172.21.15.37:6809/2754669649] -- osd_op(unknown.0.0:74 5.1c 5:38000000::::head [pgnls-filter start_epoch 44 in=85b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e44) -- 0x561d6647d000 con 0x561d6644c480 2024-04-25T10:30:38.794 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:38.775+0000 7f08d4ecd640 1 -- 172.21.15.37:0/2208611052 <== osd.2 v2:172.21.15.37:6808/2754669649 14 ==== osd_op_reply(74 [pgnls-filter start_epoch 44 out=49b] v36'9 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x561d66126000 con 0x561d6644c480 2024-04-25T10:30:38.794 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:38.775+0000 7f08d0ec5640 1 -- 172.21.15.37:0/2208611052 --> [v2:172.21.15.203:6800/4051532063,v1:172.21.15.203:6801/4051532063] -- osd_op(unknown.0.0:75 5.3c 5:3c000000::::head [pgnls-filter start_epoch 44 in=85b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e44) -- 0x561d66473800 con 0x561d661d8d00 2024-04-25T10:30:38.794 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:38.775+0000 7f08d4ecd640 1 -- 172.21.15.37:0/2208611052 <== osd.4 v2:172.21.15.203:6800/4051532063 14 ==== osd_op_reply(75 [pgnls-filter start_epoch 44 out=49b] v36'8 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x561d66126000 con 0x561d661d8d00 2024-04-25T10:30:38.794 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:38.775+0000 7f08d06c4640 1 -- 172.21.15.37:0/2208611052 --> [v2:172.21.15.37:6816/2746074720,v1:172.21.15.37:6817/2746074720] -- osd_op(unknown.0.0:76 5.2 5:40000000::::head [pgnls-filter start_epoch 44 in=85b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e44) -- 0x561d6647d400 con 0x561d6644c900 2024-04-25T10:30:38.794 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:38.776+0000 7f08d5ecf640 1 -- 172.21.15.37:0/2208611052 <== osd.0 v2:172.21.15.37:6816/2746074720 10 ==== osd_op_reply(76 [pgnls-filter start_epoch 44 out=49b] v36'8 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x561d66126500 con 0x561d6644c900 2024-04-25T10:30:38.794 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:38.776+0000 7f08d0ec5640 1 -- 172.21.15.37:0/2208611052 --> [v2:172.21.15.37:6816/2746074720,v1:172.21.15.37:6817/2746074720] -- osd_op(unknown.0.0:77 5.22 5:44000000::::head [pgnls-filter start_epoch 44 in=85b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e44) -- 0x561d66473c00 con 0x561d6644c900 2024-04-25T10:30:38.794 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:38.776+0000 7f08d5ecf640 1 -- 172.21.15.37:0/2208611052 <== osd.0 v2:172.21.15.37:6816/2746074720 11 ==== osd_op_reply(77 [pgnls-filter start_epoch 44 out=49b] v36'8 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x561d66126500 con 0x561d6644c900 2024-04-25T10:30:38.794 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:38.776+0000 7f08d06c4640 1 -- 172.21.15.37:0/2208611052 --> [v2:172.21.15.203:6810/2309615014,v1:172.21.15.203:6812/2309615014] -- osd_op(unknown.0.0:78 5.12 5:48000000::::head [pgnls-filter start_epoch 44 in=85b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e44) -- 0x561d6647d800 con 0x561d661d9180 2024-04-25T10:30:38.794 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:38.776+0000 7f08d5ecf640 1 -- 172.21.15.37:0/2208611052 <== osd.6 v2:172.21.15.203:6810/2309615014 11 ==== osd_op_reply(78 [pgnls-filter start_epoch 44 out=49b] v36'10 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x561d66126500 con 0x561d661d9180 2024-04-25T10:30:38.794 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:38.776+0000 7f08d0ec5640 1 -- 172.21.15.37:0/2208611052 --> [v2:172.21.15.203:6800/4051532063,v1:172.21.15.203:6801/4051532063] -- osd_op(unknown.0.0:79 5.32 5:4c000000::::head [pgnls-filter start_epoch 44 in=85b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e44) -- 0x561d66472000 con 0x561d661d8d00 2024-04-25T10:30:38.795 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:38.777+0000 7f08d4ecd640 1 -- 172.21.15.37:0/2208611052 <== osd.4 v2:172.21.15.203:6800/4051532063 15 ==== osd_op_reply(79 [pgnls-filter start_epoch 44 out=49b] v36'8 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x561d66126000 con 0x561d661d8d00 2024-04-25T10:30:38.795 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:38.777+0000 7f08d06c4640 1 -- 172.21.15.37:0/2208611052 --> [v2:172.21.15.203:6810/2309615014,v1:172.21.15.203:6812/2309615014] -- osd_op(unknown.0.0:80 5.a 5:50000000::::head [pgnls-filter start_epoch 44 in=85b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e44) -- 0x561d6647dc00 con 0x561d661d9180 2024-04-25T10:30:38.795 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:38.777+0000 7f08d5ecf640 1 -- 172.21.15.37:0/2208611052 <== osd.6 v2:172.21.15.203:6810/2309615014 12 ==== osd_op_reply(80 [pgnls-filter start_epoch 44 out=49b] v36'5 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x561d66126500 con 0x561d661d9180 2024-04-25T10:30:38.795 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:38.777+0000 7f08d0ec5640 1 -- 172.21.15.37:0/2208611052 --> [v2:172.21.15.37:6808/2754669649,v1:172.21.15.37:6809/2754669649] -- osd_op(unknown.0.0:81 5.2a 5:54000000::::head [pgnls-filter start_epoch 44 in=85b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e44) -- 0x561d66473400 con 0x561d6644c480 2024-04-25T10:30:38.795 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:38.777+0000 7f08d4ecd640 1 -- 172.21.15.37:0/2208611052 <== osd.2 v2:172.21.15.37:6808/2754669649 15 ==== osd_op_reply(81 [pgnls-filter start_epoch 44 out=49b] v36'9 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x561d66126000 con 0x561d6644c480 2024-04-25T10:30:38.795 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:38.777+0000 7f08d06c4640 1 -- 172.21.15.37:0/2208611052 --> [v2:172.21.15.37:6800/1690341662,v1:172.21.15.37:6801/1690341662] -- osd_op(unknown.0.0:82 5.1a 5:58000000::::head [pgnls-filter start_epoch 44 in=85b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e44) -- 0x561d66458000 con 0x561d6644d200 2024-04-25T10:30:38.795 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:38.777+0000 7f08d56ce640 1 -- 172.21.15.37:0/2208611052 <== osd.1 v2:172.21.15.37:6800/1690341662 8 ==== osd_op_reply(82 [pgnls-filter start_epoch 44 out=49b] v36'7 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x561d66126780 con 0x561d6644d200 2024-04-25T10:30:38.795 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:38.778+0000 7f08d0ec5640 1 -- 172.21.15.37:0/2208611052 --> [v2:172.21.15.203:6808/3688099014,v1:172.21.15.203:6809/3688099014] -- osd_op(unknown.0.0:83 5.3a 5:5c000000::::head [pgnls-filter start_epoch 44 in=85b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e44) -- 0x561d66472400 con 0x561d6644cd80 2024-04-25T10:30:38.795 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:38.778+0000 7f08d5ecf640 1 -- 172.21.15.37:0/2208611052 <== osd.7 v2:172.21.15.203:6808/3688099014 7 ==== osd_op_reply(83 [pgnls-filter start_epoch 44 out=49b] v36'6 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x561d66126500 con 0x561d6644cd80 2024-04-25T10:30:38.795 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:38.778+0000 7f08d06c4640 1 -- 172.21.15.37:0/2208611052 --> [v2:172.21.15.203:6808/3688099014,v1:172.21.15.203:6809/3688099014] -- osd_op(unknown.0.0:84 5.6 5:60000000::::head [pgnls-filter start_epoch 44 in=85b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e44) -- 0x561d66458400 con 0x561d6644cd80 2024-04-25T10:30:38.795 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:38.778+0000 7f08d5ecf640 1 -- 172.21.15.37:0/2208611052 <== osd.7 v2:172.21.15.203:6808/3688099014 8 ==== osd_op_reply(84 [pgnls-filter start_epoch 44 out=49b] v36'7 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x561d66126500 con 0x561d6644cd80 2024-04-25T10:30:38.795 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:38.778+0000 7f08d0ec5640 1 -- 172.21.15.37:0/2208611052 --> [v2:172.21.15.203:6800/4051532063,v1:172.21.15.203:6801/4051532063] -- osd_op(unknown.0.0:85 5.26 5:64000000::::head [pgnls-filter start_epoch 44 in=85b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e44) -- 0x561d66472800 con 0x561d661d8d00 2024-04-25T10:30:38.795 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:38.778+0000 7f08d4ecd640 1 -- 172.21.15.37:0/2208611052 <== osd.4 v2:172.21.15.203:6800/4051532063 16 ==== osd_op_reply(85 [pgnls-filter start_epoch 44 out=49b] v36'15 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x561d66126000 con 0x561d661d8d00 2024-04-25T10:30:38.795 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:38.778+0000 7f08d06c4640 1 -- 172.21.15.37:0/2208611052 --> [v2:172.21.15.37:6824/1815290618,v1:172.21.15.37:6825/1815290618] -- osd_op(unknown.0.0:86 5.16 5:68000000::::head [pgnls-filter start_epoch 44 in=85b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e44) -- 0x561d66458800 con 0x561d6644c000 2024-04-25T10:30:38.796 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:38.779+0000 7f08d56ce640 1 -- 172.21.15.37:0/2208611052 <== osd.3 v2:172.21.15.37:6824/1815290618 10 ==== osd_op_reply(86 [pgnls-filter start_epoch 44 out=49b] v36'10 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x561d66126780 con 0x561d6644c000 2024-04-25T10:30:38.796 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:38.779+0000 7f08d0ec5640 1 -- 172.21.15.37:0/2208611052 --> [v2:172.21.15.203:6800/4051532063,v1:172.21.15.203:6801/4051532063] -- osd_op(unknown.0.0:87 5.36 5:6c000000::::head [pgnls-filter start_epoch 44 in=85b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e44) -- 0x561d66472c00 con 0x561d661d8d00 2024-04-25T10:30:38.796 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:38.779+0000 7f08d4ecd640 1 -- 172.21.15.37:0/2208611052 <== osd.4 v2:172.21.15.203:6800/4051532063 17 ==== osd_op_reply(87 [pgnls-filter start_epoch 44 out=49b] v36'15 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x561d66126000 con 0x561d661d8d00 2024-04-25T10:30:38.796 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:38.779+0000 7f08d06c4640 1 -- 172.21.15.37:0/2208611052 --> [v2:172.21.15.37:6824/1815290618,v1:172.21.15.37:6825/1815290618] -- osd_op(unknown.0.0:88 5.e 5:70000000::::head [pgnls-filter start_epoch 44 in=85b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e44) -- 0x561d66458c00 con 0x561d6644c000 2024-04-25T10:30:38.796 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:38.779+0000 7f08d56ce640 1 -- 172.21.15.37:0/2208611052 <== osd.3 v2:172.21.15.37:6824/1815290618 11 ==== osd_op_reply(88 [pgnls-filter start_epoch 44 out=49b] v36'5 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x561d66126780 con 0x561d6644c000 2024-04-25T10:30:38.796 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:38.779+0000 7f08d0ec5640 1 -- 172.21.15.37:0/2208611052 --> [v2:172.21.15.203:6808/3688099014,v1:172.21.15.203:6809/3688099014] -- osd_op(unknown.0.0:89 5.2e 5:74000000::::head [pgnls-filter start_epoch 44 in=85b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e44) -- 0x561d66f94c00 con 0x561d6644cd80 2024-04-25T10:30:38.796 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:38.780+0000 7f08d5ecf640 1 -- 172.21.15.37:0/2208611052 <== osd.7 v2:172.21.15.203:6808/3688099014 9 ==== osd_op_reply(89 [pgnls-filter start_epoch 44 out=49b] v36'10 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x561d66126500 con 0x561d6644cd80 2024-04-25T10:30:38.796 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:38.780+0000 7f08d06c4640 1 -- 172.21.15.37:0/2208611052 --> [v2:172.21.15.203:6810/2309615014,v1:172.21.15.203:6812/2309615014] -- osd_op(unknown.0.0:90 5.1e 5:78000000::::head [pgnls-filter start_epoch 44 in=85b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e44) -- 0x561d66459000 con 0x561d661d9180 2024-04-25T10:30:38.796 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:38.780+0000 7f08d5ecf640 1 -- 172.21.15.37:0/2208611052 <== osd.6 v2:172.21.15.203:6810/2309615014 13 ==== osd_op_reply(90 [pgnls-filter start_epoch 44 out=49b] v36'5 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x561d66126500 con 0x561d661d9180 2024-04-25T10:30:38.796 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:38.780+0000 7f08d0ec5640 1 -- 172.21.15.37:0/2208611052 --> [v2:172.21.15.203:6824/1203822890,v1:172.21.15.203:6825/1203822890] -- osd_op(unknown.0.0:91 5.3e 5:7c000000::::head [pgnls-filter start_epoch 44 in=85b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e44) -- 0x561d66f95800 con 0x561d6644d680 2024-04-25T10:30:38.796 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:38.780+0000 7f08d4ecd640 1 -- 172.21.15.37:0/2208611052 <== osd.5 v2:172.21.15.203:6824/1203822890 7 ==== osd_op_reply(91 [pgnls-filter start_epoch 44 out=49b] v36'4 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x561d66126000 con 0x561d6644d680 2024-04-25T10:30:38.796 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:38.780+0000 7f08d06c4640 1 -- 172.21.15.37:0/2208611052 --> [v2:172.21.15.37:6824/1815290618,v1:172.21.15.37:6825/1815290618] -- osd_op(unknown.0.0:92 5.1 5:80000000::::head [pgnls-filter start_epoch 44 in=85b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e44) -- 0x561d66459400 con 0x561d6644c000 2024-04-25T10:30:38.796 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:38.781+0000 7f08d56ce640 1 -- 172.21.15.37:0/2208611052 <== osd.3 v2:172.21.15.37:6824/1815290618 12 ==== osd_op_reply(92 [pgnls-filter start_epoch 44 out=49b] v36'5 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x561d66126780 con 0x561d6644c000 2024-04-25T10:30:38.796 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:38.781+0000 7f08d0ec5640 1 -- 172.21.15.37:0/2208611052 --> [v2:172.21.15.37:6808/2754669649,v1:172.21.15.37:6809/2754669649] -- osd_op(unknown.0.0:93 5.21 5:84000000::::head [pgnls-filter start_epoch 44 in=85b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e44) -- 0x561d6647c400 con 0x561d6644c480 2024-04-25T10:30:38.797 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:38.781+0000 7f08d4ecd640 1 -- 172.21.15.37:0/2208611052 <== osd.2 v2:172.21.15.37:6808/2754669649 16 ==== osd_op_reply(93 [pgnls-filter start_epoch 44 out=49b] v36'12 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x561d66126000 con 0x561d6644c480 2024-04-25T10:30:38.797 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:38.781+0000 7f08d06c4640 1 -- 172.21.15.37:0/2208611052 --> [v2:172.21.15.37:6824/1815290618,v1:172.21.15.37:6825/1815290618] -- osd_op(unknown.0.0:94 5.11 5:88000000::::head [pgnls-filter start_epoch 44 in=85b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e44) -- 0x561d66459800 con 0x561d6644c000 2024-04-25T10:30:38.797 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:38.781+0000 7f08d56ce640 1 -- 172.21.15.37:0/2208611052 <== osd.3 v2:172.21.15.37:6824/1815290618 13 ==== osd_op_reply(94 [pgnls-filter start_epoch 44 out=49b] v36'6 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x561d66126780 con 0x561d6644c000 2024-04-25T10:30:38.797 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:38.781+0000 7f08d0ec5640 1 -- 172.21.15.37:0/2208611052 --> [v2:172.21.15.37:6816/2746074720,v1:172.21.15.37:6817/2746074720] -- osd_op(unknown.0.0:95 5.31 5:8c000000::::head [pgnls-filter start_epoch 44 in=85b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e44) -- 0x561d66473800 con 0x561d6644c900 2024-04-25T10:30:38.797 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:38.781+0000 7f08d5ecf640 1 -- 172.21.15.37:0/2208611052 <== osd.0 v2:172.21.15.37:6816/2746074720 12 ==== osd_op_reply(95 [pgnls-filter start_epoch 44 out=49b] v36'7 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x561d66126500 con 0x561d6644c900 2024-04-25T10:30:38.797 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:38.781+0000 7f08d06c4640 1 -- 172.21.15.37:0/2208611052 --> [v2:172.21.15.37:6800/1690341662,v1:172.21.15.37:6801/1690341662] -- osd_op(unknown.0.0:96 5.9 5:90000000::::head [pgnls-filter start_epoch 44 in=85b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e44) -- 0x561d66459c00 con 0x561d6644d200 2024-04-25T10:30:38.797 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:38.782+0000 7f08d56ce640 1 -- 172.21.15.37:0/2208611052 <== osd.1 v2:172.21.15.37:6800/1690341662 9 ==== osd_op_reply(96 [pgnls-filter start_epoch 44 out=49b] v36'2 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x561d66126780 con 0x561d6644d200 2024-04-25T10:30:38.797 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:38.782+0000 7f08d0ec5640 1 -- 172.21.15.37:0/2208611052 --> [v2:172.21.15.203:6810/2309615014,v1:172.21.15.203:6812/2309615014] -- osd_op(unknown.0.0:97 5.29 5:94000000::::head [pgnls-filter start_epoch 44 in=85b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e44) -- 0x561d66472000 con 0x561d661d9180 2024-04-25T10:30:38.797 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:38.782+0000 7f08d5ecf640 1 -- 172.21.15.37:0/2208611052 <== osd.6 v2:172.21.15.203:6810/2309615014 14 ==== osd_op_reply(97 [pgnls-filter start_epoch 44 out=49b] v36'8 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x561d66126500 con 0x561d661d9180 2024-04-25T10:30:38.797 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:38.782+0000 7f08d06c4640 1 -- 172.21.15.37:0/2208611052 --> [v2:172.21.15.203:6800/4051532063,v1:172.21.15.203:6801/4051532063] -- osd_op(unknown.0.0:98 5.19 5:98000000::::head [pgnls-filter start_epoch 44 in=85b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e44) -- 0x561d66458400 con 0x561d661d8d00 2024-04-25T10:30:38.797 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:38.782+0000 7f08d4ecd640 1 -- 172.21.15.37:0/2208611052 <== osd.4 v2:172.21.15.203:6800/4051532063 18 ==== osd_op_reply(98 [pgnls-filter start_epoch 44 out=49b] v36'10 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x561d66126000 con 0x561d661d8d00 2024-04-25T10:30:38.797 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:38.782+0000 7f08d0ec5640 1 -- 172.21.15.37:0/2208611052 --> [v2:172.21.15.37:6808/2754669649,v1:172.21.15.37:6809/2754669649] -- osd_op(unknown.0.0:99 5.39 5:9c000000::::head [pgnls-filter start_epoch 44 in=85b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e44) -- 0x561d66473400 con 0x561d6644c480 2024-04-25T10:30:38.797 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:38.783+0000 7f08d4ecd640 1 -- 172.21.15.37:0/2208611052 <== osd.2 v2:172.21.15.37:6808/2754669649 17 ==== osd_op_reply(99 [pgnls-filter start_epoch 44 out=49b] v36'5 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x561d66126000 con 0x561d6644c480 2024-04-25T10:30:38.797 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:38.783+0000 7f08d06c4640 1 -- 172.21.15.37:0/2208611052 --> [v2:172.21.15.203:6808/3688099014,v1:172.21.15.203:6809/3688099014] -- osd_op(unknown.0.0:100 5.5 5:a0000000::::head [pgnls-filter start_epoch 44 in=85b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e44) -- 0x561d66473c00 con 0x561d6644cd80 2024-04-25T10:30:38.797 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:38.783+0000 7f08d5ecf640 1 -- 172.21.15.37:0/2208611052 <== osd.7 v2:172.21.15.203:6808/3688099014 10 ==== osd_op_reply(100 [pgnls-filter start_epoch 44 out=49b] v36'5 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x561d66126500 con 0x561d6644cd80 2024-04-25T10:30:38.798 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:38.783+0000 7f08d0ec5640 1 -- 172.21.15.37:0/2208611052 --> [v2:172.21.15.203:6824/1203822890,v1:172.21.15.203:6825/1203822890] -- osd_op(unknown.0.0:101 5.25 5:a4000000::::head [pgnls-filter start_epoch 44 in=85b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e44) -- 0x561d6647c800 con 0x561d6644d680 2024-04-25T10:30:38.798 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:38.783+0000 7f08d4ecd640 1 -- 172.21.15.37:0/2208611052 <== osd.5 v2:172.21.15.203:6824/1203822890 8 ==== osd_op_reply(101 [pgnls-filter start_epoch 44 out=49b] v36'6 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x561d66126000 con 0x561d6644d680 2024-04-25T10:30:38.798 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:38.783+0000 7f08d06c4640 1 -- 172.21.15.37:0/2208611052 --> [v2:172.21.15.37:6824/1815290618,v1:172.21.15.37:6825/1815290618] -- osd_op(unknown.0.0:102 5.15 5:a8000000::::head [pgnls-filter start_epoch 44 in=85b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e44) -- 0x561d66472400 con 0x561d6644c000 2024-04-25T10:30:38.798 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:38.784+0000 7f08d56ce640 1 -- 172.21.15.37:0/2208611052 <== osd.3 v2:172.21.15.37:6824/1815290618 14 ==== osd_op_reply(102 [pgnls-filter start_epoch 44 out=49b] v36'6 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x561d66126780 con 0x561d6644c000 2024-04-25T10:30:38.798 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:38.784+0000 7f08d0ec5640 1 -- 172.21.15.37:0/2208611052 --> [v2:172.21.15.37:6824/1815290618,v1:172.21.15.37:6825/1815290618] -- osd_op(unknown.0.0:103 5.35 5:ac000000::::head [pgnls-filter start_epoch 44 in=85b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e44) -- 0x561d6647cc00 con 0x561d6644c000 2024-04-25T10:30:38.798 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:38.784+0000 7f08d56ce640 1 -- 172.21.15.37:0/2208611052 <== osd.3 v2:172.21.15.37:6824/1815290618 15 ==== osd_op_reply(103 [pgnls-filter start_epoch 44 out=49b] v36'6 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x561d66126780 con 0x561d6644c000 2024-04-25T10:30:38.798 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:38.784+0000 7f08d06c4640 1 -- 172.21.15.37:0/2208611052 --> [v2:172.21.15.203:6808/3688099014,v1:172.21.15.203:6809/3688099014] -- osd_op(unknown.0.0:104 5.d 5:b0000000::::head [pgnls-filter start_epoch 44 in=85b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e44) -- 0x561d66473800 con 0x561d6644cd80 2024-04-25T10:30:38.798 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:38.784+0000 7f08d5ecf640 1 -- 172.21.15.37:0/2208611052 <== osd.7 v2:172.21.15.203:6808/3688099014 11 ==== osd_op_reply(104 [pgnls-filter start_epoch 44 out=49b] v36'7 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x561d66126500 con 0x561d6644cd80 2024-04-25T10:30:38.798 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:38.784+0000 7f08d0ec5640 1 -- 172.21.15.37:0/2208611052 --> [v2:172.21.15.203:6808/3688099014,v1:172.21.15.203:6809/3688099014] -- osd_op(unknown.0.0:105 5.2d 5:b4000000::::head [pgnls-filter start_epoch 44 in=85b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e44) -- 0x561d6647d000 con 0x561d6644cd80 2024-04-25T10:30:38.798 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:38.784+0000 7f08d5ecf640 1 -- 172.21.15.37:0/2208611052 <== osd.7 v2:172.21.15.203:6808/3688099014 12 ==== osd_op_reply(105 [pgnls-filter start_epoch 44 out=49b] v36'7 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x561d66126500 con 0x561d6644cd80 2024-04-25T10:30:38.798 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:38.784+0000 7f08d06c4640 1 -- 172.21.15.37:0/2208611052 --> [v2:172.21.15.203:6800/4051532063,v1:172.21.15.203:6801/4051532063] -- osd_op(unknown.0.0:106 5.1d 5:b8000000::::head [pgnls-filter start_epoch 44 in=85b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e44) -- 0x561d66472800 con 0x561d661d8d00 2024-04-25T10:30:38.798 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:38.785+0000 7f08d4ecd640 1 -- 172.21.15.37:0/2208611052 <== osd.4 v2:172.21.15.203:6800/4051532063 19 ==== osd_op_reply(106 [pgnls-filter start_epoch 44 out=49b] v36'15 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x561d66126000 con 0x561d661d8d00 2024-04-25T10:30:38.798 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:38.785+0000 7f08d0ec5640 1 -- 172.21.15.37:0/2208611052 --> [v2:172.21.15.37:6800/1690341662,v1:172.21.15.37:6801/1690341662] -- osd_op(unknown.0.0:107 5.3d 5:bc000000::::head [pgnls-filter start_epoch 44 in=85b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e44) -- 0x561d6647c000 con 0x561d6644d200 2024-04-25T10:30:38.799 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:38.785+0000 7f08d56ce640 1 -- 172.21.15.37:0/2208611052 <== osd.1 v2:172.21.15.37:6800/1690341662 10 ==== osd_op_reply(107 [pgnls-filter start_epoch 44 out=49b] v36'7 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x561d66126780 con 0x561d6644d200 2024-04-25T10:30:38.799 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:38.785+0000 7f08d06c4640 1 -- 172.21.15.37:0/2208611052 --> [v2:172.21.15.203:6800/4051532063,v1:172.21.15.203:6801/4051532063] -- osd_op(unknown.0.0:108 5.3 5:c0000000::::head [pgnls-filter start_epoch 44 in=85b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e44) -- 0x561d66472c00 con 0x561d661d8d00 2024-04-25T10:30:38.799 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:38.785+0000 7f08d4ecd640 1 -- 172.21.15.37:0/2208611052 <== osd.4 v2:172.21.15.203:6800/4051532063 20 ==== osd_op_reply(108 [pgnls-filter start_epoch 44 out=49b] v36'7 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x561d66126000 con 0x561d661d8d00 2024-04-25T10:30:38.799 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:38.785+0000 7f08d0ec5640 1 -- 172.21.15.37:0/2208611052 --> [v2:172.21.15.37:6816/2746074720,v1:172.21.15.37:6817/2746074720] -- osd_op(unknown.0.0:109 5.23 5:c4000000::::head [pgnls-filter start_epoch 44 in=85b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e44) -- 0x561d66f95800 con 0x561d6644c900 2024-04-25T10:30:38.799 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:38.786+0000 7f08d5ecf640 1 -- 172.21.15.37:0/2208611052 <== osd.0 v2:172.21.15.37:6816/2746074720 13 ==== osd_op_reply(109 [pgnls-filter start_epoch 44 out=49b] v36'7 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x561d66126500 con 0x561d6644c900 2024-04-25T10:30:38.799 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:38.786+0000 7f08d06c4640 1 -- 172.21.15.37:0/2208611052 --> [v2:172.21.15.203:6800/4051532063,v1:172.21.15.203:6801/4051532063] -- osd_op(unknown.0.0:110 5.13 5:c8000000::::head [pgnls-filter start_epoch 44 in=85b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e44) -- 0x561d66473000 con 0x561d661d8d00 2024-04-25T10:30:38.799 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:38.786+0000 7f08d4ecd640 1 -- 172.21.15.37:0/2208611052 <== osd.4 v2:172.21.15.203:6800/4051532063 21 ==== osd_op_reply(110 [pgnls-filter start_epoch 44 out=49b] v36'15 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x561d66126000 con 0x561d661d8d00 2024-04-25T10:30:38.799 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:38.786+0000 7f08d0ec5640 1 -- 172.21.15.37:0/2208611052 --> [v2:172.21.15.203:6810/2309615014,v1:172.21.15.203:6812/2309615014] -- osd_op(unknown.0.0:111 5.33 5:cc000000::::head [pgnls-filter start_epoch 44 in=85b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e44) -- 0x561d66f95000 con 0x561d661d9180 2024-04-25T10:30:38.799 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:38.786+0000 7f08d5ecf640 1 -- 172.21.15.37:0/2208611052 <== osd.6 v2:172.21.15.203:6810/2309615014 15 ==== osd_op_reply(111 [pgnls-filter start_epoch 44 out=49b] v36'10 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x561d66126500 con 0x561d661d9180 2024-04-25T10:30:38.799 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:38.786+0000 7f08d06c4640 1 -- 172.21.15.37:0/2208611052 --> [v2:172.21.15.203:6824/1203822890,v1:172.21.15.203:6825/1203822890] -- osd_op(unknown.0.0:112 5.b 5:d0000000::::head [pgnls-filter start_epoch 44 in=85b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e44) -- 0x561d66f94c00 con 0x561d6644d680 2024-04-25T10:30:38.799 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:38.787+0000 7f08d4ecd640 1 -- 172.21.15.37:0/2208611052 <== osd.5 v2:172.21.15.203:6824/1203822890 9 ==== osd_op_reply(112 [pgnls-filter start_epoch 44 out=49b] v36'8 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x561d66126000 con 0x561d6644d680 2024-04-25T10:30:38.799 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:38.787+0000 7f08d0ec5640 1 -- 172.21.15.37:0/2208611052 --> [v2:172.21.15.37:6816/2746074720,v1:172.21.15.37:6817/2746074720] -- osd_op(unknown.0.0:113 5.2b 5:d4000000::::head [pgnls-filter start_epoch 44 in=85b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e44) -- 0x561d66f95400 con 0x561d6644c900 2024-04-25T10:30:38.799 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:38.787+0000 7f08d5ecf640 1 -- 172.21.15.37:0/2208611052 <== osd.0 v2:172.21.15.37:6816/2746074720 14 ==== osd_op_reply(113 [pgnls-filter start_epoch 44 out=49b] v36'11 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x561d66126500 con 0x561d6644c900 2024-04-25T10:30:38.799 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:38.787+0000 7f08d06c4640 1 -- 172.21.15.37:0/2208611052 --> [v2:172.21.15.37:6808/2754669649,v1:172.21.15.37:6809/2754669649] -- osd_op(unknown.0.0:114 5.1b 5:d8000000::::head [pgnls-filter start_epoch 44 in=85b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e44) -- 0x561d6647d400 con 0x561d6644c480 2024-04-25T10:30:38.800 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:38.787+0000 7f08d4ecd640 1 -- 172.21.15.37:0/2208611052 <== osd.2 v2:172.21.15.37:6808/2754669649 18 ==== osd_op_reply(114 [pgnls-filter start_epoch 44 out=49b] v36'8 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x561d66126000 con 0x561d6644c480 2024-04-25T10:30:38.800 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:38.787+0000 7f08d0ec5640 1 -- 172.21.15.37:0/2208611052 --> [v2:172.21.15.203:6824/1203822890,v1:172.21.15.203:6825/1203822890] -- osd_op(unknown.0.0:115 5.3b 5:dc000000::::head [pgnls-filter start_epoch 44 in=85b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e44) -- 0x561d66f19400 con 0x561d6644d680 2024-04-25T10:30:38.800 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:38.788+0000 7f08d4ecd640 1 -- 172.21.15.37:0/2208611052 <== osd.5 v2:172.21.15.203:6824/1203822890 10 ==== osd_op_reply(115 [pgnls-filter start_epoch 44 out=49b] v36'4 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x561d66126000 con 0x561d6644d680 2024-04-25T10:30:38.800 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:38.788+0000 7f08d06c4640 1 -- 172.21.15.37:0/2208611052 --> [v2:172.21.15.37:6824/1815290618,v1:172.21.15.37:6825/1815290618] -- osd_op(unknown.0.0:116 5.7 5:e0000000::::head [pgnls-filter start_epoch 44 in=85b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e44) -- 0x561d6647d800 con 0x561d6644c000 2024-04-25T10:30:38.800 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:38.788+0000 7f08d56ce640 1 -- 172.21.15.37:0/2208611052 <== osd.3 v2:172.21.15.37:6824/1815290618 16 ==== osd_op_reply(116 [pgnls-filter start_epoch 44 out=81b] v44'10 uv10 ondisk = 1) ==== 144+0+81 (crc 0 0 0) 0x561d66126780 con 0x561d6644c000 2024-04-25T10:30:38.800 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:38.788+0000 7f08d7a22740 20 datascan.forall_objects: OSD matched oid 10000000001.00000000 2024-04-25T10:30:38.800 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:38.788+0000 7f08d7a22740 10 datascan.operator(): handling object 10000000001.0 2024-04-25T10:30:38.800 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:38.788+0000 7f08d7a22740 1 -- 172.21.15.37:0/2208611052 --> [v2:172.21.15.37:6824/1815290618,v1:172.21.15.37:6825/1815290618] -- osd_op(unknown.0.0:117 5.7 5:e2e07930:::10000000001.00000000:head [getxattr scan_ceiling in=12b,getxattr scan_max_size in=13b,getxattr scan_max_mtime in=14b,getxattr scan_pool_id in=12b,getxattr parent in=6b,getxattr layout in=6b,getxattr symlink in=7b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e44) -- 0x561d66472400 con 0x561d6644c000 2024-04-25T10:30:38.800 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:38.788+0000 7f08d56ce640 1 -- 172.21.15.37:0/2208611052 <== osd.3 v2:172.21.15.37:6824/1815290618 17 ==== osd_op_reply(117 10000000001.00000000 [getxattr (22) out=22b,getxattr (8) out=8b,getxattr (8) out=8b,getxattr,getxattr (95) out=95b,getxattr (30) out=30b,getxattr] v0'0 uv10 ondisk = 0) ==== 416+0+163 (crc 0 0 0) 0x561d66126780 con 0x561d6644c000 2024-04-25T10:30:38.800 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:38.788+0000 7f08d7a22740 20 datascan.operator(): loaded layout from xattr: pi: 5 os: 4194304 sc: 1 su: 4194304 2024-04-25T10:30:38.800 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:38.788+0000 7f08d7a22740 10 datascan.inject_with_backtrace: inode: 0x0x10000000001 2024-04-25T10:30:38.800 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:38.788+0000 7f08d7a22740 10 datascan.inject_with_backtrace: backptr: 0x0x10000000000/sixmegs 2024-04-25T10:30:38.800 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:38.788+0000 7f08d7a22740 20 datascan.get_frag_of: dirino=0x10000000000 target_dname=sixmegs 2024-04-25T10:30:38.800 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:38.788+0000 7f08d7a22740 1 -- 172.21.15.37:0/2208611052 --> [v2:172.21.15.203:6800/4051532063,v1:172.21.15.203:6801/4051532063] -- osd_op(unknown.0.0:118 6.10 6:0cf6ad0f:::10000000000.00000000:head [getxattr parent in=6b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e44) -- 0x561d66473400 con 0x561d661d8d00 2024-04-25T10:30:38.801 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:38.789+0000 7f08d4ecd640 1 -- 172.21.15.37:0/2208611052 <== osd.4 v2:172.21.15.203:6800/4051532063 22 ==== osd_op_reply(118 10000000000.00000000 [getxattr] v0'0 uv0 ondisk = -2 ((2) No such file or directory)) ==== 164+0+0 (crc 0 0 0) 0x561d66126000 con 0x561d661d8d00 2024-04-25T10:30:38.801 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:38.789+0000 7f08d7a22740 4 datascan.get_frag_of: Unexpected error on '10000000000.00000000': (2) No such file or directory 2024-04-25T10:30:38.801 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:38.789+0000 7f08d7a22740 20 datascan.inject_with_backtrace: don't know fragment for 0x0x10000000000/sixmegs, will insert to root 2024-04-25T10:30:38.801 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:38.789+0000 7f08d7a22740 1 -- 172.21.15.37:0/2208611052 --> [v2:172.21.15.203:6800/4051532063,v1:172.21.15.203:6801/4051532063] -- osd_op(unknown.0.0:119 6.10 6:0cf6ad0f:::10000000000.00000000:head [omap-get-header] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e44) -- 0x561d66472800 con 0x561d661d8d00 2024-04-25T10:30:38.801 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:38.789+0000 7f08d4ecd640 1 -- 172.21.15.37:0/2208611052 <== osd.4 v2:172.21.15.203:6800/4051532063 23 ==== osd_op_reply(119 10000000000.00000000 [omap-get-header] v0'0 uv0 ondisk = -2 ((2) No such file or directory)) ==== 164+0+0 (crc 0 0 0) 0x561d66126000 con 0x561d661d8d00 2024-04-25T10:30:38.801 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:38.789+0000 7f08d7a22740 10 datascan.find_or_create_dirfrag: read_version = 0 2024-04-25T10:30:38.801 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:38.789+0000 7f08d7a22740 1 -- 172.21.15.37:0/2208611052 --> [v2:172.21.15.203:6800/4051532063,v1:172.21.15.203:6801/4051532063] -- osd_op(unknown.0.0:120 6.10 6:0cf6ad0f:::10000000000.00000000:head [create,omap-set-header in=274b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e44) -- 0x561d66472c00 con 0x561d661d8d00 2024-04-25T10:30:38.801 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:38.790+0000 7f08d4ecd640 1 -- 172.21.15.37:0/2208611052 <== osd.4 v2:172.21.15.203:6800/4051532063 24 ==== osd_op_reply(120 10000000000.00000000 [create,omap-set-header] v44'1 uv1 ondisk = 0) ==== 206+0+0 (crc 0 0 0) 0x561d66126000 con 0x561d661d8d00 2024-04-25T10:30:38.801 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:38.790+0000 7f08d7a22740 10 datascan.find_or_create_dirfrag: Created dirfrag: 0x0x10000000000 2024-04-25T10:30:38.801 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:38.790+0000 7f08d7a22740 1 -- 172.21.15.37:0/2208611052 --> [v2:172.21.15.203:6800/4051532063,v1:172.21.15.203:6801/4051532063] -- osd_op(unknown.0.0:121 6.10 6:0cf6ad0f:::10000000000.00000000:head [omap-get-vals-by-keys in=20b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e44) -- 0x561d6647cc00 con 0x561d661d8d00 2024-04-25T10:30:38.801 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:38.791+0000 7f08d4ecd640 1 -- 172.21.15.37:0/2208611052 <== osd.4 v2:172.21.15.203:6800/4051532063 25 ==== osd_op_reply(121 10000000000.00000000 [omap-get-vals-by-keys out=4b] v0'0 uv1 ondisk = 0) ==== 164+0+4 (crc 0 0 0) 0x561d66126000 con 0x561d661d8d00 2024-04-25T10:30:38.801 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:38.791+0000 7f08d7a22740 20 datascan.read_dentry: oid=10000000000.00000000 dname=sixmegs frag=*, r=0 2024-04-25T10:30:38.801 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:38.791+0000 7f08d7a22740 20 datascan.read_dentry: sixmegs_head not found in result 2024-04-25T10:30:38.801 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:38.791+0000 7f08d7a22740 10 datascan.inject_with_backtrace: Linking inode 0x0x10000000001 at 0x0x10000000000/sixmegs with size=6291456 bytes 2024-04-25T10:30:38.801 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:38.791+0000 7f08d7a22740 1 -- 172.21.15.37:0/2208611052 --> [v2:172.21.15.203:6800/4051532063,v1:172.21.15.203:6801/4051532063] -- osd_op(unknown.0.0:122 6.10 6:0cf6ad0f:::10000000000.00000000:head [omap-set-vals in=508b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e44) -- 0x561d6647c000 con 0x561d661d8d00 2024-04-25T10:30:38.807 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:38.792+0000 7f08d4ecd640 1 -- 172.21.15.37:0/2208611052 <== osd.4 v2:172.21.15.203:6800/4051532063 26 ==== osd_op_reply(122 10000000000.00000000 [omap-set-vals] v44'2 uv2 ondisk = 0) ==== 164+0+0 (crc 0 0 0) 0x561d66126000 con 0x561d661d8d00 2024-04-25T10:30:38.807 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:38.792+0000 7f08d7a22740 20 datascan.inject_linkage: Injected dentry 0x0x10000000000/sixmegs pointing to 0x0x10000000001 2024-04-25T10:30:38.807 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:38.792+0000 7f08d7a22740 10 datascan.inject_with_backtrace: backptr: 0x0x1/subdir 2024-04-25T10:30:38.808 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:38.792+0000 7f08d7a22740 20 datascan.get_frag_of: dirino=0x1 target_dname=subdir 2024-04-25T10:30:38.808 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:38.792+0000 7f08d7a22740 1 -- 172.21.15.37:0/2208611052 --> [v2:172.21.15.203:6810/2309615014,v1:172.21.15.203:6812/2309615014] -- osd_op(unknown.0.0:123 6.1f 6:ff5b34d6:::1.00000000:head [getxattr parent in=6b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e44) -- 0x561d6647d800 con 0x561d661d9180 2024-04-25T10:30:38.808 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:38.792+0000 7f08d5ecf640 1 -- 172.21.15.37:0/2208611052 <== osd.6 v2:172.21.15.203:6810/2309615014 16 ==== osd_op_reply(123 1.00000000 [getxattr] v0'0 uv0 ondisk = -2 ((2) No such file or directory)) ==== 154+0+0 (crc 0 0 0) 0x561d66126500 con 0x561d661d9180 2024-04-25T10:30:38.808 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:38.792+0000 7f08d7a22740 4 datascan.get_frag_of: Unexpected error on '1.00000000': (2) No such file or directory 2024-04-25T10:30:38.808 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:38.792+0000 7f08d7a22740 20 datascan.inject_with_backtrace: don't know fragment for 0x0x1/subdir, will insert to root 2024-04-25T10:30:38.808 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:38.792+0000 7f08d7a22740 1 -- 172.21.15.37:0/2208611052 --> [v2:172.21.15.203:6810/2309615014,v1:172.21.15.203:6812/2309615014] -- osd_op(unknown.0.0:124 6.1f 6:ff5b34d6:::1.00000000:head [omap-get-header] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e44) -- 0x561d66f94c00 con 0x561d661d9180 2024-04-25T10:30:38.808 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:38.792+0000 7f08d5ecf640 1 -- 172.21.15.37:0/2208611052 <== osd.6 v2:172.21.15.203:6810/2309615014 17 ==== osd_op_reply(124 1.00000000 [omap-get-header] v0'0 uv0 ondisk = -2 ((2) No such file or directory)) ==== 154+0+0 (crc 0 0 0) 0x561d66126500 con 0x561d661d9180 2024-04-25T10:30:38.808 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:38.792+0000 7f08d7a22740 10 datascan.find_or_create_dirfrag: read_version = 0 2024-04-25T10:30:38.808 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:38.792+0000 7f08d7a22740 1 -- 172.21.15.37:0/2208611052 --> [v2:172.21.15.203:6810/2309615014,v1:172.21.15.203:6812/2309615014] -- osd_op(unknown.0.0:125 6.1f 6:ff5b34d6:::1.00000000:head [create,omap-set-header in=274b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e44) -- 0x561d66473000 con 0x561d661d9180 2024-04-25T10:30:38.808 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:38.793+0000 7f08d5ecf640 1 -- 172.21.15.37:0/2208611052 <== osd.6 v2:172.21.15.203:6810/2309615014 18 ==== osd_op_reply(125 1.00000000 [create,omap-set-header] v44'2 uv2 ondisk = 0) ==== 196+0+0 (crc 0 0 0) 0x561d66126500 con 0x561d661d9180 2024-04-25T10:30:38.808 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:38.794+0000 7f08d7a22740 10 datascan.find_or_create_dirfrag: Created dirfrag: 0x0x1 2024-04-25T10:30:38.808 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:38.794+0000 7f08d7a22740 1 -- 172.21.15.37:0/2208611052 --> [v2:172.21.15.203:6810/2309615014,v1:172.21.15.203:6812/2309615014] -- osd_op(unknown.0.0:126 6.1f 6:ff5b34d6:::1.00000000:head [omap-get-vals-by-keys in=19b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e44) -- 0x561d66472000 con 0x561d661d9180 2024-04-25T10:30:38.808 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:38.794+0000 7f08d5ecf640 1 -- 172.21.15.37:0/2208611052 <== osd.6 v2:172.21.15.203:6810/2309615014 19 ==== osd_op_reply(126 1.00000000 [omap-get-vals-by-keys out=4b] v0'0 uv2 ondisk = 0) ==== 154+0+4 (crc 0 0 0) 0x561d66126500 con 0x561d661d9180 2024-04-25T10:30:38.808 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:38.794+0000 7f08d7a22740 20 datascan.read_dentry: oid=1.00000000 dname=subdir frag=*, r=0 2024-04-25T10:30:38.809 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:38.794+0000 7f08d7a22740 20 datascan.read_dentry: subdir_head not found in result 2024-04-25T10:30:38.809 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:38.794+0000 7f08d7a22740 10 datascan.inject_with_backtrace: Linking ancestor directory of inode 0x0x10000000000 at 0x0x1:subdir 2024-04-25T10:30:38.809 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:38.794+0000 7f08d7a22740 1 -- 172.21.15.37:0/2208611052 --> [v2:172.21.15.203:6810/2309615014,v1:172.21.15.203:6812/2309615014] -- osd_op(unknown.0.0:127 6.1f 6:ff5b34d6:::1.00000000:head [omap-set-vals in=507b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e44) -- 0x561d66473c00 con 0x561d661d9180 2024-04-25T10:30:38.809 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:38.795+0000 7f08d5ecf640 1 -- 172.21.15.37:0/2208611052 <== osd.6 v2:172.21.15.203:6810/2309615014 20 ==== osd_op_reply(127 1.00000000 [omap-set-vals] v44'3 uv3 ondisk = 0) ==== 154+0+0 (crc 0 0 0) 0x561d66126500 con 0x561d661d9180 2024-04-25T10:30:38.809 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:38.795+0000 7f08d7a22740 20 datascan.inject_linkage: Injected dentry 0x0x1/subdir pointing to 0x0x10000000000 2024-04-25T10:30:38.809 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:38.795+0000 7f08d7a22740 1 -- 172.21.15.37:0/2208611052 --> [v2:172.21.15.203:6800/4051532063,v1:172.21.15.203:6801/4051532063] -- osd_op(unknown.0.0:128 5.27 5:e4000000::::head [pgnls-filter start_epoch 44 in=85b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e44) -- 0x561d66f95800 con 0x561d661d8d00 2024-04-25T10:30:38.809 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:38.795+0000 7f08d4ecd640 1 -- 172.21.15.37:0/2208611052 <== osd.4 v2:172.21.15.203:6800/4051532063 27 ==== osd_op_reply(128 [pgnls-filter start_epoch 44 out=49b] v36'2 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x561d66126000 con 0x561d661d8d00 2024-04-25T10:30:38.809 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:38.795+0000 7f08d06c4640 1 -- 172.21.15.37:0/2208611052 --> [v2:172.21.15.203:6800/4051532063,v1:172.21.15.203:6801/4051532063] -- osd_op(unknown.0.0:129 5.17 5:e8000000::::head [pgnls-filter start_epoch 44 in=85b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e44) -- 0x561d6647dc00 con 0x561d661d8d00 2024-04-25T10:30:38.809 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:38.796+0000 7f08d4ecd640 1 -- 172.21.15.37:0/2208611052 <== osd.4 v2:172.21.15.203:6800/4051532063 28 ==== osd_op_reply(129 [pgnls-filter start_epoch 44 out=49b] v36'9 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x561d66126000 con 0x561d661d8d00 2024-04-25T10:30:38.809 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:38.796+0000 7f08d0ec5640 1 -- 172.21.15.37:0/2208611052 --> [v2:172.21.15.37:6800/1690341662,v1:172.21.15.37:6801/1690341662] -- osd_op(unknown.0.0:130 5.37 5:ec000000::::head [pgnls-filter start_epoch 44 in=85b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e44) -- 0x561d66f19400 con 0x561d6644d200 2024-04-25T10:30:38.809 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:38.796+0000 7f08d56ce640 1 -- 172.21.15.37:0/2208611052 <== osd.1 v2:172.21.15.37:6800/1690341662 11 ==== osd_op_reply(130 [pgnls-filter start_epoch 44 out=49b] v36'11 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x561d66126780 con 0x561d6644d200 2024-04-25T10:30:38.809 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:38.796+0000 7f08d06c4640 1 -- 172.21.15.37:0/2208611052 --> [v2:172.21.15.37:6800/1690341662,v1:172.21.15.37:6801/1690341662] -- osd_op(unknown.0.0:131 5.f 5:f0000000::::head [pgnls-filter start_epoch 44 in=85b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e44) -- 0x561d6646e000 con 0x561d6644d200 2024-04-25T10:30:38.809 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:38.796+0000 7f08d56ce640 1 -- 172.21.15.37:0/2208611052 <== osd.1 v2:172.21.15.37:6800/1690341662 12 ==== osd_op_reply(131 [pgnls-filter start_epoch 44 out=49b] v36'9 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x561d66126780 con 0x561d6644d200 2024-04-25T10:30:38.809 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:38.796+0000 7f08d0ec5640 1 -- 172.21.15.37:0/2208611052 --> [v2:172.21.15.37:6824/1815290618,v1:172.21.15.37:6825/1815290618] -- osd_op(unknown.0.0:132 5.2f 5:f4000000::::head [pgnls-filter start_epoch 44 in=85b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e44) -- 0x561d6647d000 con 0x561d6644c000 2024-04-25T10:30:38.810 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:38.796+0000 7f08d56ce640 1 -- 172.21.15.37:0/2208611052 <== osd.3 v2:172.21.15.37:6824/1815290618 18 ==== osd_op_reply(132 [pgnls-filter start_epoch 44 out=49b] v36'9 uv1 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x561d66126780 con 0x561d6644c000 2024-04-25T10:30:38.810 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:38.797+0000 7f08d06c4640 1 -- 172.21.15.37:0/2208611052 --> [v2:172.21.15.37:6808/2754669649,v1:172.21.15.37:6809/2754669649] -- osd_op(unknown.0.0:133 5.1f 5:f8000000::::head [pgnls-filter start_epoch 44 in=85b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e44) -- 0x561d6646e400 con 0x561d6644c480 2024-04-25T10:30:38.810 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:38.797+0000 7f08d4ecd640 1 -- 172.21.15.37:0/2208611052 <== osd.2 v2:172.21.15.37:6808/2754669649 19 ==== osd_op_reply(133 [pgnls-filter start_epoch 44 out=49b] v36'6 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x561d66126000 con 0x561d6644c480 2024-04-25T10:30:38.810 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:38.797+0000 7f08d0ec5640 1 -- 172.21.15.37:0/2208611052 --> [v2:172.21.15.37:6808/2754669649,v1:172.21.15.37:6809/2754669649] -- osd_op(unknown.0.0:134 5.3f 5:fc000000::::head [pgnls-filter start_epoch 44 in=85b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e44) -- 0x561d6647d800 con 0x561d6644c480 2024-04-25T10:30:38.810 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:38.797+0000 7f08d4ecd640 1 -- 172.21.15.37:0/2208611052 <== osd.2 v2:172.21.15.37:6808/2754669649 20 ==== osd_op_reply(134 [pgnls-filter start_epoch 44 out=49b] v36'8 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x561d66126000 con 0x561d6644c480 2024-04-25T10:30:38.810 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:38.797+0000 7f08d7a22740 1 -- 172.21.15.37:0/2208611052 >> [v2:172.21.15.37:6816/2746074720,v1:172.21.15.37:6817/2746074720] conn(0x561d6644c900 msgr2=0x561d66454b00 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:30:38.810 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:38.797+0000 7f08d7a22740 1 --2- 172.21.15.37:0/2208611052 >> [v2:172.21.15.37:6816/2746074720,v1:172.21.15.37:6817/2746074720] conn(0x561d6644c900 0x561d66454b00 crc :-1 s=READY pgs=14 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:30:38.810 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:38.797+0000 7f08d7a22740 1 -- 172.21.15.37:0/2208611052 >> [v2:172.21.15.37:6800/1690341662,v1:172.21.15.37:6801/1690341662] conn(0x561d6644d200 msgr2=0x561d66455600 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:30:38.810 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:38.797+0000 7f08d7a22740 1 --2- 172.21.15.37:0/2208611052 >> [v2:172.21.15.37:6800/1690341662,v1:172.21.15.37:6801/1690341662] conn(0x561d6644d200 0x561d66455600 crc :-1 s=READY pgs=16 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:30:38.810 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:38.797+0000 7f08d7a22740 1 -- 172.21.15.37:0/2208611052 >> [v2:172.21.15.37:6808/2754669649,v1:172.21.15.37:6809/2754669649] conn(0x561d6644c480 msgr2=0x561d66454580 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:30:38.810 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:38.797+0000 7f08d7a22740 1 --2- 172.21.15.37:0/2208611052 >> [v2:172.21.15.37:6808/2754669649,v1:172.21.15.37:6809/2754669649] conn(0x561d6644c480 0x561d66454580 crc :-1 s=READY pgs=14 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:30:38.810 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:38.797+0000 7f08d7a22740 1 -- 172.21.15.37:0/2208611052 >> [v2:172.21.15.37:6824/1815290618,v1:172.21.15.37:6825/1815290618] conn(0x561d6644c000 msgr2=0x561d66454000 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:30:38.810 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:38.797+0000 7f08d7a22740 1 --2- 172.21.15.37:0/2208611052 >> [v2:172.21.15.37:6824/1815290618,v1:172.21.15.37:6825/1815290618] conn(0x561d6644c000 0x561d66454000 crc :-1 s=READY pgs=15 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:30:38.810 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:38.797+0000 7f08d7a22740 1 -- 172.21.15.37:0/2208611052 >> [v2:172.21.15.203:6800/4051532063,v1:172.21.15.203:6801/4051532063] conn(0x561d661d8d00 msgr2=0x561d661ae680 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:30:38.811 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:38.797+0000 7f08d7a22740 1 --2- 172.21.15.37:0/2208611052 >> [v2:172.21.15.203:6800/4051532063,v1:172.21.15.203:6801/4051532063] conn(0x561d661d8d00 0x561d661ae680 crc :-1 s=READY pgs=15 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:30:38.811 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:38.797+0000 7f08d7a22740 1 -- 172.21.15.37:0/2208611052 >> [v2:172.21.15.203:6824/1203822890,v1:172.21.15.203:6825/1203822890] conn(0x561d6644d680 msgr2=0x561d66455b80 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:30:38.811 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:38.797+0000 7f08d7a22740 1 --2- 172.21.15.37:0/2208611052 >> [v2:172.21.15.203:6824/1203822890,v1:172.21.15.203:6825/1203822890] conn(0x561d6644d680 0x561d66455b80 crc :-1 s=READY pgs=14 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:30:38.811 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:38.797+0000 7f08d7a22740 1 -- 172.21.15.37:0/2208611052 >> [v2:172.21.15.203:6810/2309615014,v1:172.21.15.203:6812/2309615014] conn(0x561d661d9180 msgr2=0x561d661aec00 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:30:38.811 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:38.797+0000 7f08d56ce640 1 -- 172.21.15.37:0/2208611052 reap_dead start 2024-04-25T10:30:38.811 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:38.797+0000 7f08d7a22740 1 --2- 172.21.15.37:0/2208611052 >> [v2:172.21.15.203:6810/2309615014,v1:172.21.15.203:6812/2309615014] conn(0x561d661d9180 0x561d661aec00 crc :-1 s=READY pgs=14 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:30:38.811 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:38.797+0000 7f08d56ce640 1 -- 172.21.15.37:0/2208611052 reap_dead start 2024-04-25T10:30:38.811 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:38.797+0000 7f08d7a22740 1 -- 172.21.15.37:0/2208611052 >> [v2:172.21.15.203:6808/3688099014,v1:172.21.15.203:6809/3688099014] conn(0x561d6644cd80 msgr2=0x561d66455080 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:30:38.811 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:38.797+0000 7f08d7a22740 1 --2- 172.21.15.37:0/2208611052 >> [v2:172.21.15.203:6808/3688099014,v1:172.21.15.203:6809/3688099014] conn(0x561d6644cd80 0x561d66455080 crc :-1 s=READY pgs=16 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:30:38.811 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:38.797+0000 7f08d56ce640 1 -- 172.21.15.37:0/2208611052 reap_dead start 2024-04-25T10:30:38.811 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:38.797+0000 7f08d7a22740 1 -- 172.21.15.37:0/2208611052 >> [v2:172.21.15.203:6832/2418205237,v1:172.21.15.203:6833/2418205237] conn(0x561d661d9a80 msgr2=0x561d661ae100 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:30:38.811 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:38.797+0000 7f08d7a22740 1 --2- 172.21.15.37:0/2208611052 >> [v2:172.21.15.203:6832/2418205237,v1:172.21.15.203:6833/2418205237] conn(0x561d661d9a80 0x561d661ae100 secure :-1 s=READY pgs=266 cs=0 l=1 rev1=1 crypto rx=0x561d661408a0 tx=0x561d6613e7e0 comp rx=0 tx=0).stop 2024-04-25T10:30:38.811 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:38.797+0000 7f08d7a22740 1 -- 172.21.15.37:0/2208611052 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x561d661d6900 msgr2=0x561d661ad600 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:30:38.811 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:38.797+0000 7f08d7a22740 1 --2- 172.21.15.37:0/2208611052 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x561d661d6900 0x561d661ad600 secure :-1 s=READY pgs=565 cs=0 l=1 rev1=1 crypto rx=0x561d66f03fb0 tx=0x561d6613ea20 comp rx=0 tx=0).stop 2024-04-25T10:30:38.811 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:38.797+0000 7f08d7a22740 1 -- 172.21.15.37:0/2208611052 shutdown_connections 2024-04-25T10:30:38.812 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:38.797+0000 7f08d7a22740 1 --2- 172.21.15.37:0/2208611052 >> [v2:172.21.15.203:6832/2418205237,v1:172.21.15.203:6833/2418205237] conn(0x561d661d9a80 0x561d661ae100 unknown :-1 s=CLOSED pgs=266 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:30:38.812 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:38.797+0000 7f08d7a22740 1 --2- 172.21.15.37:0/2208611052 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x561d661d6900 0x561d661ad600 unknown :-1 s=CLOSED pgs=565 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:30:38.812 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:38.797+0000 7f08d7a22740 1 -- 172.21.15.37:0/2208611052 >> 172.21.15.37:0/2208611052 conn(0x561d661d7200 msgr2=0x561d66125800 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:30:38.812 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:38.797+0000 7f08d7a22740 1 -- 172.21.15.37:0/2208611052 shutdown_connections 2024-04-25T10:30:38.812 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:38.797+0000 7f08d7a22740 1 -- 172.21.15.37:0/2208611052 wait complete. 2024-04-25T10:30:38.812 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:38.798+0000 7f08d7a22740 1 -- 172.21.15.37:0/4100771996 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x561d661d7f80 msgr2=0x561d661acb00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:30:38.812 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:38.798+0000 7f08d7a22740 1 --2- 172.21.15.37:0/4100771996 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x561d661d7f80 0x561d661acb00 secure :-1 s=READY pgs=563 cs=0 l=1 rev1=1 crypto rx=0x561d66e63cb0 tx=0x561d66142180 comp rx=0 tx=0).stop 2024-04-25T10:30:38.812 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:38.798+0000 7f08d7a22740 1 -- 172.21.15.37:0/4100771996 shutdown_connections 2024-04-25T10:30:38.812 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:38.798+0000 7f08d7a22740 1 --2- 172.21.15.37:0/4100771996 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x561d661d7f80 0x561d661acb00 unknown :-1 s=CLOSED pgs=563 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:30:38.812 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:38.798+0000 7f08d7a22740 1 --2- 172.21.15.37:0/4100771996 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x561d661d7b00 0x561d661ac000 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:30:38.812 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:38.798+0000 7f08d7a22740 1 --2- 172.21.15.37:0/4100771996 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x561d661d7680 0x561d661ac580 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:30:38.812 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:38.798+0000 7f08d7a22740 1 -- 172.21.15.37:0/4100771996 >> 172.21.15.37:0/4100771996 conn(0x561d661d8400 msgr2=0x561d66125000 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:30:38.812 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:38.798+0000 7f08d7a22740 1 -- 172.21.15.37:0/4100771996 shutdown_connections 2024-04-25T10:30:38.812 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:38.798+0000 7f08d7a22740 1 -- 172.21.15.37:0/4100771996 wait complete. 2024-04-25T10:30:38.813 DEBUG:tasks.cephfs.filesystem:Ran ['cephfs-data-scan', '--debug-mds=20', '--debug-ms=1', '--debug-objecter=1', 'scan_inodes', '--alternate-pool', 'recovery_fs_metadata', '--filesystem', 'cephfs', '--force-corrupt', '--force-init', 'cephfs_data'] in time 0:00:00.132072, result: 2024-04-25T10:30:38.813 DEBUG:teuthology.orchestra.run.smithi037:> cephfs-data-scan --debug-mds=20 --debug-ms=1 --debug-objecter=1 scan_links --filesystem recovery_fs 2024-04-25T10:30:38.865 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:38.863+0000 7f3dcc412740 1 -- start start 2024-04-25T10:30:38.866 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:38.863+0000 7f3dcc412740 1 Processor -- start 2024-04-25T10:30:38.868 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:38.864+0000 7f3dcc412740 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x56229adf1680 0x56229adce580 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:30:38.868 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:38.864+0000 7f3dcc412740 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x56229adf1b00 0x56229adce000 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:30:38.868 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:38.864+0000 7f3dcc412740 1 --2- >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x56229adf1f80 0x56229adceb00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:30:38.869 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:38.864+0000 7f3dcc412740 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 -- 0x56229ad16180 con 0x56229adf1680 2024-04-25T10:30:38.869 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:38.864+0000 7f3dcc412740 1 -- --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_getmap magic: 0 -- 0x56229ad16300 con 0x56229adf1b00 2024-04-25T10:30:38.869 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:38.864+0000 7f3dcc412740 1 -- --> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] -- mon_getmap magic: 0 -- 0x56229ad16480 con 0x56229adf1f80 2024-04-25T10:30:38.869 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:38.864+0000 7f3dca0b5640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x56229adf1680 0x56229adce580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:30:38.869 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:38.864+0000 7f3dca0b5640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x56229adf1680 0x56229adce580 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.37:3300/0 says I am v2:172.21.15.37:55700/0 (socket says 172.21.15.37:55700) 2024-04-25T10:30:38.869 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:38.864+0000 7f3dca0b5640 1 -- 172.21.15.37:0/1863835220 learned_addr learned my addr 172.21.15.37:0/1863835220 (peer_addr_for_me v2:172.21.15.37:0/0) 2024-04-25T10:30:38.869 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:38.865+0000 7f3dca8b6640 1 --2- 172.21.15.37:0/1863835220 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x56229adf1f80 0x56229adceb00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:30:38.869 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:38.865+0000 7f3dc98b4640 1 --2- 172.21.15.37:0/1863835220 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x56229adf1b00 0x56229adce000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:30:38.869 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:38.865+0000 7f3dca0b5640 1 -- 172.21.15.37:0/1863835220 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x56229adf1f80 msgr2=0x56229adceb00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:30:38.869 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:38.865+0000 7f3dca0b5640 1 --2- 172.21.15.37:0/1863835220 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x56229adf1f80 0x56229adceb00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:30:38.869 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:38.865+0000 7f3dca0b5640 1 -- 172.21.15.37:0/1863835220 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x56229adf1b00 msgr2=0x56229adce000 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:30:38.869 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:38.865+0000 7f3dca0b5640 1 --2- 172.21.15.37:0/1863835220 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x56229adf1b00 0x56229adce000 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:30:38.869 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:38.865+0000 7f3dca0b5640 1 -- 172.21.15.37:0/1863835220 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x562296c55680 con 0x56229adf1680 2024-04-25T10:30:38.869 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:38.865+0000 7f3dc98b4640 1 --2- 172.21.15.37:0/1863835220 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x56229adf1b00 0x56229adce000 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T10:30:38.869 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:38.865+0000 7f3dca0b5640 1 --2- 172.21.15.37:0/1863835220 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x56229adf1680 0x56229adce580 secure :-1 s=READY pgs=567 cs=0 l=1 rev1=1 crypto rx=0x56229bb28060 tx=0x56229ad5c900 comp rx=0 tx=0).ready entity=mon.0 client_cookie=5307031496d1da server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:30:38.870 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:38.865+0000 7f3dc78b0640 1 -- 172.21.15.37:0/1863835220 <== mon.0 v2:172.21.15.37:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x562296c4e680 con 0x56229adf1680 2024-04-25T10:30:38.870 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:38.865+0000 7f3dc78b0640 1 -- 172.21.15.37:0/1863835220 <== mon.0 v2:172.21.15.37:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x56229aca0e00 con 0x56229adf1680 2024-04-25T10:30:38.870 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:38.865+0000 7f3dc78b0640 1 -- 172.21.15.37:0/1863835220 <== mon.0 v2:172.21.15.37:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x562296c4eb60 con 0x56229adf1680 2024-04-25T10:30:38.870 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:38.865+0000 7f3dcc412740 1 -- 172.21.15.37:0/1863835220 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({osdmap=0}) -- 0x562296c55860 con 0x56229adf1680 2024-04-25T10:30:38.870 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:38.865+0000 7f3dcc412740 1 -- 172.21.15.37:0/1863835220 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({fsmap=0}) -- 0x562296c552c0 con 0x56229adf1680 2024-04-25T10:30:38.870 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:38.865+0000 7f3dcc412740 4 waiting for MDS map... 2024-04-25T10:30:38.870 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:38.865+0000 7f3dc78b0640 1 -- 172.21.15.37:0/1863835220 <== mon.0 v2:172.21.15.37:3300/0 4 ==== osd_map(44..44 src has 1..44) ==== 6248+0+0 (secure 0 0 0) 0x56229bb3ca00 con 0x56229adf1680 2024-04-25T10:30:38.870 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:38.865+0000 7f3dc78b0640 1 -- 172.21.15.37:0/1863835220 <== mon.0 v2:172.21.15.37:3300/0 5 ==== fsmap(e 22) ==== 4135+0+0 (secure 0 0 0) 0x56229acb1180 con 0x56229adf1680 2024-04-25T10:30:38.870 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:38.865+0000 7f3dcc412740 4 Got MDS map 22 2024-04-25T10:30:38.870 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:38.866+0000 7f3dcc412740 4 datascan.main: Using metadata pool output 2024-04-25T10:30:38.870 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:38.866+0000 7f3dcc412740 4 datascan.main: connecting to RADOS... 2024-04-25T10:30:38.870 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:38.866+0000 7f3dcc412740 1 Processor -- start 2024-04-25T10:30:38.870 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:38.867+0000 7f3dcc412740 1 -- start start 2024-04-25T10:30:38.874 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:38.868+0000 7f3dcc412740 1 --2- >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x56229adf0d80 0x56229adcf080 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:30:38.875 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:38.868+0000 7f3dcc412740 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x56229adf0900 0x56229adcf600 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:30:38.875 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:38.868+0000 7f3dcc412740 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x56229adf2880 0x56229adcfb80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:30:38.875 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:38.868+0000 7f3dcc412740 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 -- 0x56229ad16600 con 0x56229adf2880 2024-04-25T10:30:38.875 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:38.868+0000 7f3dcc412740 1 -- --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_getmap magic: 0 -- 0x56229ad16780 con 0x56229adf0900 2024-04-25T10:30:38.875 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:38.868+0000 7f3dcc412740 1 -- --> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] -- mon_getmap magic: 0 -- 0x56229ad16900 con 0x56229adf0d80 2024-04-25T10:30:38.875 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:38.868+0000 7f3dc98b4640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x56229adf2880 0x56229adcfb80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:30:38.875 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:38.868+0000 7f3dca8b6640 1 --2- >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x56229adf0d80 0x56229adcf080 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:30:38.875 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:38.868+0000 7f3dca0b5640 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x56229adf0900 0x56229adcf600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:30:38.875 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:38.868+0000 7f3dc98b4640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x56229adf2880 0x56229adcfb80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.37:3300/0 says I am v2:172.21.15.37:55714/0 (socket says 172.21.15.37:55714) 2024-04-25T10:30:38.875 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:38.868+0000 7f3dc98b4640 1 -- 172.21.15.37:0/1272321379 learned_addr learned my addr 172.21.15.37:0/1272321379 (peer_addr_for_me v2:172.21.15.37:0/0) 2024-04-25T10:30:38.875 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:38.868+0000 7f3dc98b4640 1 -- 172.21.15.37:0/1272321379 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x56229adf0d80 msgr2=0x56229adcf080 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:30:38.875 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:38.868+0000 7f3dc98b4640 1 --2- 172.21.15.37:0/1272321379 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x56229adf0d80 0x56229adcf080 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:30:38.876 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:38.868+0000 7f3dc98b4640 1 -- 172.21.15.37:0/1272321379 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x56229adf0900 msgr2=0x56229adcf600 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:30:38.876 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:38.868+0000 7f3dc98b4640 1 --2- 172.21.15.37:0/1272321379 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x56229adf0900 0x56229adcf600 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:30:38.876 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:38.868+0000 7f3dc98b4640 1 -- 172.21.15.37:0/1272321379 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x562296c554a0 con 0x56229adf2880 2024-04-25T10:30:38.876 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:38.868+0000 7f3dca0b5640 1 --2- 172.21.15.37:0/1272321379 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x56229adf0900 0x56229adcf600 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-25T10:30:38.876 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:38.868+0000 7f3dc98b4640 1 --2- 172.21.15.37:0/1272321379 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x56229adf2880 0x56229adcfb80 secure :-1 s=READY pgs=568 cs=0 l=1 rev1=1 crypto rx=0x56229bb29680 tx=0x56229bb48c00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=5a8ce2bac4c09e89 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:30:38.876 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:38.868+0000 7f3dc58ac640 1 -- 172.21.15.37:0/1272321379 <== mon.0 v2:172.21.15.37:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x562296c4e680 con 0x56229adf2880 2024-04-25T10:30:38.876 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:38.869+0000 7f3dc58ac640 1 -- 172.21.15.37:0/1272321379 <== mon.0 v2:172.21.15.37:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x56229aca0540 con 0x56229adf2880 2024-04-25T10:30:38.876 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:38.869+0000 7f3dc58ac640 1 -- 172.21.15.37:0/1272321379 <== mon.0 v2:172.21.15.37:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x562296c4eb60 con 0x56229adf2880 2024-04-25T10:30:38.876 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:38.869+0000 7f3dcc412740 1 -- 172.21.15.37:0/1272321379 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x56229adf2880 msgr2=0x56229adcfb80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:30:38.876 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:38.869+0000 7f3dcc412740 1 --2- 172.21.15.37:0/1272321379 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x56229adf2880 0x56229adcfb80 secure :-1 s=READY pgs=568 cs=0 l=1 rev1=1 crypto rx=0x56229bb29680 tx=0x56229bb48c00 comp rx=0 tx=0).stop 2024-04-25T10:30:38.876 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:38.869+0000 7f3dcc412740 1 -- 172.21.15.37:0/1272321379 shutdown_connections 2024-04-25T10:30:38.876 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:38.869+0000 7f3dcc412740 1 --2- 172.21.15.37:0/1272321379 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x56229adf2880 0x56229adcfb80 unknown :-1 s=CLOSED pgs=568 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:30:38.876 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:38.869+0000 7f3dcc412740 1 --2- 172.21.15.37:0/1272321379 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x56229adf0900 0x56229adcf600 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:30:38.876 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:38.869+0000 7f3dcc412740 1 --2- 172.21.15.37:0/1272321379 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x56229adf0d80 0x56229adcf080 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:30:38.876 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:38.869+0000 7f3dcc412740 1 -- 172.21.15.37:0/1272321379 >> 172.21.15.37:0/1272321379 conn(0x56229adf1200 msgr2=0x56229ad3f800 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:30:38.877 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:38.869+0000 7f3dcc412740 1 -- 172.21.15.37:0/1272321379 shutdown_connections 2024-04-25T10:30:38.877 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:38.869+0000 7f3dcc412740 1 -- 172.21.15.37:0/1272321379 wait complete. 2024-04-25T10:30:38.877 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:38.870+0000 7f3dcc412740 1 Processor -- start 2024-04-25T10:30:38.877 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:38.870+0000 7f3dcc412740 1 -- start start 2024-04-25T10:30:38.877 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:38.871+0000 7f3dcc412740 1 --2- >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x56229adf0900 0x56229adcf600 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:30:38.877 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:38.871+0000 7f3dcc412740 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x56229adf0d80 0x56229adcf080 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:30:38.877 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:38.871+0000 7f3dcc412740 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x56229adf2880 0x56229adcfb80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:30:38.877 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:38.871+0000 7f3dcc412740 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 -- 0x56229ad17e00 con 0x56229adf0d80 2024-04-25T10:30:38.877 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:38.871+0000 7f3dcc412740 1 -- --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_getmap magic: 0 -- 0x56229ad17c80 con 0x56229adf2880 2024-04-25T10:30:38.877 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:38.871+0000 7f3dcc412740 1 -- --> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] -- mon_getmap magic: 0 -- 0x56229ad16c00 con 0x56229adf0900 2024-04-25T10:30:38.877 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:38.871+0000 7f3dca0b5640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x56229adf0d80 0x56229adcf080 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:30:38.877 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:38.871+0000 7f3dca0b5640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x56229adf0d80 0x56229adcf080 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.37:3300/0 says I am v2:172.21.15.37:55718/0 (socket says 172.21.15.37:55718) 2024-04-25T10:30:38.877 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:38.871+0000 7f3dca0b5640 1 -- 172.21.15.37:0/2011009937 learned_addr learned my addr 172.21.15.37:0/2011009937 (peer_addr_for_me v2:172.21.15.37:0/0) 2024-04-25T10:30:38.877 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:38.871+0000 7f3dca8b6640 1 --2- 172.21.15.37:0/2011009937 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x56229adf0900 0x56229adcf600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:30:38.878 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:38.871+0000 7f3dc98b4640 1 --2- 172.21.15.37:0/2011009937 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x56229adf2880 0x56229adcfb80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:30:38.878 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:38.871+0000 7f3dca0b5640 1 -- 172.21.15.37:0/2011009937 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x56229adf0900 msgr2=0x56229adcf600 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:30:38.878 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:38.871+0000 7f3dca0b5640 1 --2- 172.21.15.37:0/2011009937 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x56229adf0900 0x56229adcf600 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:30:38.878 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:38.871+0000 7f3dca0b5640 1 -- 172.21.15.37:0/2011009937 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x56229adf2880 msgr2=0x56229adcfb80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:30:38.878 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:38.872+0000 7f3dca0b5640 1 --2- 172.21.15.37:0/2011009937 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x56229adf2880 0x56229adcfb80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:30:38.878 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:38.872+0000 7f3dca0b5640 1 -- 172.21.15.37:0/2011009937 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x562296c55680 con 0x56229adf0d80 2024-04-25T10:30:38.878 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:38.872+0000 7f3dca0b5640 1 --2- 172.21.15.37:0/2011009937 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x56229adf0d80 0x56229adcf080 secure :-1 s=READY pgs=569 cs=0 l=1 rev1=1 crypto rx=0x56229bbb0600 tx=0x56229ad58a20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:30:38.878 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:38.872+0000 7f3dc40a9640 1 -- 172.21.15.37:0/2011009937 <== mon.0 v2:172.21.15.37:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x562296c4e820 con 0x56229adf0d80 2024-04-25T10:30:38.878 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:38.872+0000 7f3dc40a9640 1 -- 172.21.15.37:0/2011009937 <== mon.0 v2:172.21.15.37:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x56229aca0c40 con 0x56229adf0d80 2024-04-25T10:30:38.878 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:38.872+0000 7f3dcc412740 1 -- 172.21.15.37:0/2011009937 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x562296c55860 con 0x56229adf0d80 2024-04-25T10:30:38.878 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:38.872+0000 7f3dc40a9640 1 -- 172.21.15.37:0/2011009937 <== mon.0 v2:172.21.15.37:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x562296c4e680 con 0x56229adf0d80 2024-04-25T10:30:38.878 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:38.872+0000 7f3dcc412740 1 -- 172.21.15.37:0/2011009937 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({osdmap=0}) -- 0x562296c550e0 con 0x56229adf0d80 2024-04-25T10:30:38.878 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:38.872+0000 7f3dcc412740 4 datascan.init: resolving metadata pool 6 2024-04-25T10:30:38.884 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:38.873+0000 7f3dc40a9640 1 -- 172.21.15.37:0/2011009937 <== mon.0 v2:172.21.15.37:3300/0 4 ==== mgrmap(e 5) ==== 96782+0+0 (secure 0 0 0) 0x56229bb8a900 con 0x56229adf0d80 2024-04-25T10:30:38.884 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:38.874+0000 7f3dc40a9640 1 --2- 172.21.15.37:0/2011009937 >> [v2:172.21.15.203:6832/2418205237,v1:172.21.15.203:6833/2418205237] conn(0x56229adf3a80 0x56229add0100 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:30:38.884 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:38.874+0000 7f3dc40a9640 1 -- 172.21.15.37:0/2011009937 <== mon.0 v2:172.21.15.37:3300/0 5 ==== osd_map(44..44 src has 1..44) ==== 6248+0+0 (secure 0 0 0) 0x56229bba8c00 con 0x56229adf0d80 2024-04-25T10:30:38.884 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:38.874+0000 7f3dcc412740 4 datascan.init: found metadata pool 'recovery_fs_metadata' 2024-04-25T10:30:38.884 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:38.874+0000 7f3dcc412740 4 datascan.main: resolving metadata pool 6 2024-04-25T10:30:38.884 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:38.874+0000 7f3dcc412740 1 --2- 172.21.15.37:0/2011009937 >> [v2:172.21.15.203:6808/3688099014,v1:172.21.15.203:6809/3688099014] conn(0x56229adf2d00 0x56229add0680 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:30:38.884 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:38.874+0000 7f3dca8b6640 1 --2- 172.21.15.37:0/2011009937 >> [v2:172.21.15.203:6832/2418205237,v1:172.21.15.203:6833/2418205237] conn(0x56229adf3a80 0x56229add0100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:30:38.885 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:38.874+0000 7f3dcc412740 1 -- 172.21.15.37:0/2011009937 --> [v2:172.21.15.203:6808/3688099014,v1:172.21.15.203:6809/3688099014] -- osd_op(unknown.0.0:1 6.0 6:00000000::::head [pgnls start_epoch 44 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e44) -- 0x56229bba7c00 con 0x56229adf2d00 2024-04-25T10:30:38.885 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:38.874+0000 7f3dc98b4640 1 --2- 172.21.15.37:0/2011009937 >> [v2:172.21.15.203:6808/3688099014,v1:172.21.15.203:6809/3688099014] conn(0x56229adf2d00 0x56229add0680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:30:38.885 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:38.874+0000 7f3dca8b6640 1 --2- 172.21.15.37:0/2011009937 >> [v2:172.21.15.203:6832/2418205237,v1:172.21.15.203:6833/2418205237] conn(0x56229adf3a80 0x56229add0100 secure :-1 s=READY pgs=267 cs=0 l=1 rev1=1 crypto rx=0x56229ad5a690 tx=0x56229ad58b40 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:30:38.885 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:38.874+0000 7f3dc98b4640 1 --2- 172.21.15.37:0/2011009937 >> [v2:172.21.15.203:6808/3688099014,v1:172.21.15.203:6809/3688099014] conn(0x56229adf2d00 0x56229add0680 crc :-1 s=READY pgs=17 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.7 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:30:38.885 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:38.875+0000 7f3dc98b4640 1 -- 172.21.15.37:0/2011009937 <== osd.7 v2:172.21.15.203:6808/3688099014 1 ==== osd_op_reply(1 [pgnls start_epoch 44 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x56229ad40000 con 0x56229adf2d00 2024-04-25T10:30:38.885 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:38.875+0000 7f3dc98b4640 1 --2- 172.21.15.37:0/2011009937 >> [v2:172.21.15.203:6800/4051532063,v1:172.21.15.203:6801/4051532063] conn(0x56229b056000 0x56229b05a000 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:30:38.885 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:38.875+0000 7f3dc98b4640 1 -- 172.21.15.37:0/2011009937 --> [v2:172.21.15.203:6800/4051532063,v1:172.21.15.203:6801/4051532063] -- osd_op(unknown.0.0:2 6.10 6:08000000::::head [pgnls start_epoch 44 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e44) -- 0x56229bba6800 con 0x56229b056000 2024-04-25T10:30:38.885 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:38.875+0000 7f3dca8b6640 1 --2- 172.21.15.37:0/2011009937 >> [v2:172.21.15.203:6800/4051532063,v1:172.21.15.203:6801/4051532063] conn(0x56229b056000 0x56229b05a000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:30:38.885 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:38.875+0000 7f3dca8b6640 1 --2- 172.21.15.37:0/2011009937 >> [v2:172.21.15.203:6800/4051532063,v1:172.21.15.203:6801/4051532063] conn(0x56229b056000 0x56229b05a000 crc :-1 s=READY pgs=16 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.4 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:30:38.885 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:38.876+0000 7f3dca8b6640 1 -- 172.21.15.37:0/2011009937 <== osd.4 v2:172.21.15.203:6800/4051532063 1 ==== osd_op_reply(2 [pgnls start_epoch 44 out=81b] v44'2 uv2 ondisk = 1) ==== 144+0+81 (crc 0 0 0) 0x56229ad40500 con 0x56229b056000 2024-04-25T10:30:38.885 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:38.876+0000 7f3dca8b6640 1 --2- 172.21.15.37:0/2011009937 >> [v2:172.21.15.37:6824/1815290618,v1:172.21.15.37:6825/1815290618] conn(0x56229b056480 0x56229b05a580 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:30:38.885 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:38.876+0000 7f3dca8b6640 1 -- 172.21.15.37:0/2011009937 --> [v2:172.21.15.37:6824/1815290618,v1:172.21.15.37:6825/1815290618] -- osd_op(unknown.0.0:3 6.8 6:10000000::::head [pgnls start_epoch 44 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e44) -- 0x56229bba6800 con 0x56229b056480 2024-04-25T10:30:38.885 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:38.876+0000 7f3dca0b5640 1 --2- 172.21.15.37:0/2011009937 >> [v2:172.21.15.37:6824/1815290618,v1:172.21.15.37:6825/1815290618] conn(0x56229b056480 0x56229b05a580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:30:38.885 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:38.876+0000 7f3dca0b5640 1 --2- 172.21.15.37:0/2011009937 >> [v2:172.21.15.37:6824/1815290618,v1:172.21.15.37:6825/1815290618] conn(0x56229b056480 0x56229b05a580 crc :-1 s=READY pgs=16 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.3 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:30:38.885 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:38.877+0000 7f3dca0b5640 1 -- 172.21.15.37:0/2011009937 <== osd.3 v2:172.21.15.37:6824/1815290618 1 ==== osd_op_reply(3 [pgnls start_epoch 44 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x56229ad40780 con 0x56229b056480 2024-04-25T10:30:38.886 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:38.877+0000 7f3dca0b5640 1 -- 172.21.15.37:0/2011009937 --> [v2:172.21.15.203:6800/4051532063,v1:172.21.15.203:6801/4051532063] -- osd_op(unknown.0.0:4 6.18 6:18000000::::head [pgnls start_epoch 44 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e44) -- 0x56229bba6800 con 0x56229b056000 2024-04-25T10:30:38.886 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:38.877+0000 7f3dca8b6640 1 -- 172.21.15.37:0/2011009937 <== osd.4 v2:172.21.15.203:6800/4051532063 2 ==== osd_op_reply(4 [pgnls start_epoch 44 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x56229ad40500 con 0x56229b056000 2024-04-25T10:30:38.886 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:38.877+0000 7f3dca8b6640 1 -- 172.21.15.37:0/2011009937 --> [v2:172.21.15.203:6808/3688099014,v1:172.21.15.203:6809/3688099014] -- osd_op(unknown.0.0:5 6.4 6:20000000::::head [pgnls start_epoch 44 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e44) -- 0x56229bba6800 con 0x56229adf2d00 2024-04-25T10:30:38.886 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:38.877+0000 7f3dc98b4640 1 -- 172.21.15.37:0/2011009937 <== osd.7 v2:172.21.15.203:6808/3688099014 2 ==== osd_op_reply(5 [pgnls start_epoch 44 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x56229ad40000 con 0x56229adf2d00 2024-04-25T10:30:38.886 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:38.877+0000 7f3dc98b4640 1 -- 172.21.15.37:0/2011009937 --> [v2:172.21.15.203:6800/4051532063,v1:172.21.15.203:6801/4051532063] -- osd_op(unknown.0.0:6 6.14 6:28000000::::head [pgnls start_epoch 44 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e44) -- 0x56229bba6800 con 0x56229b056000 2024-04-25T10:30:38.886 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:38.878+0000 7f3dca8b6640 1 -- 172.21.15.37:0/2011009937 <== osd.4 v2:172.21.15.203:6800/4051532063 3 ==== osd_op_reply(6 [pgnls start_epoch 44 out=77b] v44'1 uv1 ondisk = 1) ==== 144+0+77 (crc 0 0 0) 0x56229ad40500 con 0x56229b056000 2024-04-25T10:30:38.886 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:38.878+0000 7f3dca8b6640 1 --2- 172.21.15.37:0/2011009937 >> [v2:172.21.15.203:6810/2309615014,v1:172.21.15.203:6812/2309615014] conn(0x56229b056900 0x56229b05ab00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:30:38.886 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:38.878+0000 7f3dca8b6640 1 -- 172.21.15.37:0/2011009937 --> [v2:172.21.15.203:6810/2309615014,v1:172.21.15.203:6812/2309615014] -- osd_op(unknown.0.0:7 6.c 6:30000000::::head [pgnls start_epoch 44 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e44) -- 0x56229bba6800 con 0x56229b056900 2024-04-25T10:30:38.886 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:38.878+0000 7f3dc98b4640 1 --2- 172.21.15.37:0/2011009937 >> [v2:172.21.15.203:6810/2309615014,v1:172.21.15.203:6812/2309615014] conn(0x56229b056900 0x56229b05ab00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:30:38.886 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:38.878+0000 7f3dc98b4640 1 --2- 172.21.15.37:0/2011009937 >> [v2:172.21.15.203:6810/2309615014,v1:172.21.15.203:6812/2309615014] conn(0x56229b056900 0x56229b05ab00 crc :-1 s=READY pgs=15 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.6 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:30:38.886 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:38.879+0000 7f3dc98b4640 1 -- 172.21.15.37:0/2011009937 <== osd.6 v2:172.21.15.203:6810/2309615014 1 ==== osd_op_reply(7 [pgnls start_epoch 44 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x56229ad40000 con 0x56229b056900 2024-04-25T10:30:38.886 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:38.879+0000 7f3dc98b4640 1 -- 172.21.15.37:0/2011009937 --> [v2:172.21.15.37:6824/1815290618,v1:172.21.15.37:6825/1815290618] -- osd_op(unknown.0.0:8 6.1c 6:38000000::::head [pgnls start_epoch 44 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e44) -- 0x56229bba6800 con 0x56229b056480 2024-04-25T10:30:38.886 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:38.879+0000 7f3dca0b5640 1 -- 172.21.15.37:0/2011009937 <== osd.3 v2:172.21.15.37:6824/1815290618 2 ==== osd_op_reply(8 [pgnls start_epoch 44 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x56229ad40780 con 0x56229b056480 2024-04-25T10:30:38.886 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:38.879+0000 7f3dca0b5640 1 -- 172.21.15.37:0/2011009937 --> [v2:172.21.15.37:6824/1815290618,v1:172.21.15.37:6825/1815290618] -- osd_op(unknown.0.0:9 6.2 6:40000000::::head [pgnls start_epoch 44 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e44) -- 0x56229bba6800 con 0x56229b056480 2024-04-25T10:30:38.887 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:38.879+0000 7f3dca0b5640 1 -- 172.21.15.37:0/2011009937 <== osd.3 v2:172.21.15.37:6824/1815290618 3 ==== osd_op_reply(9 [pgnls start_epoch 44 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x56229ad40780 con 0x56229b056480 2024-04-25T10:30:38.887 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:38.879+0000 7f3dca0b5640 1 --2- 172.21.15.37:0/2011009937 >> [v2:172.21.15.37:6816/2746074720,v1:172.21.15.37:6817/2746074720] conn(0x56229b057200 0x56229b05b600 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:30:38.887 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:38.879+0000 7f3dca0b5640 1 -- 172.21.15.37:0/2011009937 --> [v2:172.21.15.37:6816/2746074720,v1:172.21.15.37:6817/2746074720] -- osd_op(unknown.0.0:10 6.12 6:48000000::::head [pgnls start_epoch 44 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e44) -- 0x56229bba6800 con 0x56229b057200 2024-04-25T10:30:38.887 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:38.879+0000 7f3dca8b6640 1 --2- 172.21.15.37:0/2011009937 >> [v2:172.21.15.37:6816/2746074720,v1:172.21.15.37:6817/2746074720] conn(0x56229b057200 0x56229b05b600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:30:38.887 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:38.880+0000 7f3dca8b6640 1 --2- 172.21.15.37:0/2011009937 >> [v2:172.21.15.37:6816/2746074720,v1:172.21.15.37:6817/2746074720] conn(0x56229b057200 0x56229b05b600 crc :-1 s=READY pgs=15 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:30:38.887 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:38.880+0000 7f3dca8b6640 1 -- 172.21.15.37:0/2011009937 <== osd.0 v2:172.21.15.37:6816/2746074720 1 ==== osd_op_reply(10 [pgnls start_epoch 44 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x56229ad40500 con 0x56229b057200 2024-04-25T10:30:38.887 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:38.880+0000 7f3dca8b6640 1 -- 172.21.15.37:0/2011009937 --> [v2:172.21.15.37:6816/2746074720,v1:172.21.15.37:6817/2746074720] -- osd_op(unknown.0.0:11 6.a 6:50000000::::head [pgnls start_epoch 44 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e44) -- 0x56229bba6800 con 0x56229b057200 2024-04-25T10:30:38.887 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:38.880+0000 7f3dca8b6640 1 -- 172.21.15.37:0/2011009937 <== osd.0 v2:172.21.15.37:6816/2746074720 2 ==== osd_op_reply(11 [pgnls start_epoch 44 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x56229ad40500 con 0x56229b057200 2024-04-25T10:30:38.887 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:38.880+0000 7f3dca8b6640 1 -- 172.21.15.37:0/2011009937 --> [v2:172.21.15.37:6824/1815290618,v1:172.21.15.37:6825/1815290618] -- osd_op(unknown.0.0:12 6.1a 6:58000000::::head [pgnls start_epoch 44 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e44) -- 0x56229bba6800 con 0x56229b056480 2024-04-25T10:30:38.887 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:38.880+0000 7f3dca0b5640 1 -- 172.21.15.37:0/2011009937 <== osd.3 v2:172.21.15.37:6824/1815290618 4 ==== osd_op_reply(12 [pgnls start_epoch 44 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x56229ad40780 con 0x56229b056480 2024-04-25T10:30:38.887 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:38.880+0000 7f3dca0b5640 1 --2- 172.21.15.37:0/2011009937 >> [v2:172.21.15.203:6824/1203822890,v1:172.21.15.203:6825/1203822890] conn(0x56229b057680 0x56229b05bb80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:30:38.887 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:38.880+0000 7f3dca0b5640 1 -- 172.21.15.37:0/2011009937 --> [v2:172.21.15.203:6824/1203822890,v1:172.21.15.203:6825/1203822890] -- osd_op(unknown.0.0:13 6.6 6:60000000::::head [pgnls start_epoch 44 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e44) -- 0x56229bba6800 con 0x56229b057680 2024-04-25T10:30:38.887 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:38.881+0000 7f3dca0b5640 1 --2- 172.21.15.37:0/2011009937 >> [v2:172.21.15.203:6824/1203822890,v1:172.21.15.203:6825/1203822890] conn(0x56229b057680 0x56229b05bb80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:30:38.887 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:38.881+0000 7f3dca0b5640 1 --2- 172.21.15.37:0/2011009937 >> [v2:172.21.15.203:6824/1203822890,v1:172.21.15.203:6825/1203822890] conn(0x56229b057680 0x56229b05bb80 crc :-1 s=READY pgs=15 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.5 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:30:38.887 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:38.881+0000 7f3dca0b5640 1 -- 172.21.15.37:0/2011009937 <== osd.5 v2:172.21.15.203:6824/1203822890 1 ==== osd_op_reply(13 [pgnls start_epoch 44 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x56229ad40780 con 0x56229b057680 2024-04-25T10:30:38.888 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:38.881+0000 7f3dca0b5640 1 -- 172.21.15.37:0/2011009937 --> [v2:172.21.15.203:6800/4051532063,v1:172.21.15.203:6801/4051532063] -- osd_op(unknown.0.0:14 6.16 6:68000000::::head [pgnls start_epoch 44 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e44) -- 0x56229bba6800 con 0x56229b056000 2024-04-25T10:30:38.888 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:38.882+0000 7f3dca8b6640 1 -- 172.21.15.37:0/2011009937 <== osd.4 v2:172.21.15.203:6800/4051532063 4 ==== osd_op_reply(14 [pgnls start_epoch 44 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x56229ad40500 con 0x56229b056000 2024-04-25T10:30:38.888 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:38.882+0000 7f3dca8b6640 1 --2- 172.21.15.37:0/2011009937 >> [v2:172.21.15.37:6800/1690341662,v1:172.21.15.37:6801/1690341662] conn(0x56229b056d80 0x56229b05b080 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:30:38.888 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:38.882+0000 7f3dca8b6640 1 -- 172.21.15.37:0/2011009937 --> [v2:172.21.15.37:6800/1690341662,v1:172.21.15.37:6801/1690341662] -- osd_op(unknown.0.0:15 6.e 6:70000000::::head [pgnls start_epoch 44 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e44) -- 0x56229bba6800 con 0x56229b056d80 2024-04-25T10:30:38.888 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:38.882+0000 7f3dc98b4640 1 --2- 172.21.15.37:0/2011009937 >> [v2:172.21.15.37:6800/1690341662,v1:172.21.15.37:6801/1690341662] conn(0x56229b056d80 0x56229b05b080 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:30:38.888 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:38.882+0000 7f3dc98b4640 1 --2- 172.21.15.37:0/2011009937 >> [v2:172.21.15.37:6800/1690341662,v1:172.21.15.37:6801/1690341662] conn(0x56229b056d80 0x56229b05b080 crc :-1 s=READY pgs=17 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:30:38.896 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:38.882+0000 7f3dc98b4640 1 -- 172.21.15.37:0/2011009937 <== osd.1 v2:172.21.15.37:6800/1690341662 1 ==== osd_op_reply(15 [pgnls start_epoch 44 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x56229ad40000 con 0x56229b056d80 2024-04-25T10:30:38.896 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:38.883+0000 7f3dc98b4640 1 -- 172.21.15.37:0/2011009937 --> [v2:172.21.15.37:6824/1815290618,v1:172.21.15.37:6825/1815290618] -- osd_op(unknown.0.0:16 6.1e 6:78000000::::head [pgnls start_epoch 44 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e44) -- 0x56229bba6800 con 0x56229b056480 2024-04-25T10:30:38.896 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:38.883+0000 7f3dca0b5640 1 -- 172.21.15.37:0/2011009937 <== osd.3 v2:172.21.15.37:6824/1815290618 5 ==== osd_op_reply(16 [pgnls start_epoch 44 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x56229ad40780 con 0x56229b056480 2024-04-25T10:30:38.896 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:38.883+0000 7f3dca0b5640 1 -- 172.21.15.37:0/2011009937 --> [v2:172.21.15.203:6810/2309615014,v1:172.21.15.203:6812/2309615014] -- osd_op(unknown.0.0:17 6.1 6:80000000::::head [pgnls start_epoch 44 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e44) -- 0x56229bba6800 con 0x56229b056900 2024-04-25T10:30:38.896 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:38.883+0000 7f3dc98b4640 1 -- 172.21.15.37:0/2011009937 <== osd.6 v2:172.21.15.203:6810/2309615014 2 ==== osd_op_reply(17 [pgnls start_epoch 44 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x56229ad40000 con 0x56229b056900 2024-04-25T10:30:38.896 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:38.883+0000 7f3dc98b4640 1 -- 172.21.15.37:0/2011009937 --> [v2:172.21.15.203:6810/2309615014,v1:172.21.15.203:6812/2309615014] -- osd_op(unknown.0.0:18 6.11 6:88000000::::head [pgnls start_epoch 44 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e44) -- 0x56229bba6800 con 0x56229b056900 2024-04-25T10:30:38.896 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:38.883+0000 7f3dc98b4640 1 -- 172.21.15.37:0/2011009937 <== osd.6 v2:172.21.15.203:6810/2309615014 3 ==== osd_op_reply(18 [pgnls start_epoch 44 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x56229ad40000 con 0x56229b056900 2024-04-25T10:30:38.896 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:38.883+0000 7f3dc98b4640 1 -- 172.21.15.37:0/2011009937 --> [v2:172.21.15.203:6800/4051532063,v1:172.21.15.203:6801/4051532063] -- osd_op(unknown.0.0:19 6.9 6:90000000::::head [pgnls start_epoch 44 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e44) -- 0x56229bba6800 con 0x56229b056000 2024-04-25T10:30:38.896 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:38.884+0000 7f3dca8b6640 1 -- 172.21.15.37:0/2011009937 <== osd.4 v2:172.21.15.203:6800/4051532063 5 ==== osd_op_reply(19 [pgnls start_epoch 44 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x56229ad40500 con 0x56229b056000 2024-04-25T10:30:38.897 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:38.884+0000 7f3dca8b6640 1 -- 172.21.15.37:0/2011009937 --> [v2:172.21.15.37:6816/2746074720,v1:172.21.15.37:6817/2746074720] -- osd_op(unknown.0.0:20 6.19 6:98000000::::head [pgnls start_epoch 44 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e44) -- 0x56229bba6800 con 0x56229b057200 2024-04-25T10:30:38.897 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:38.884+0000 7f3dca8b6640 1 -- 172.21.15.37:0/2011009937 <== osd.0 v2:172.21.15.37:6816/2746074720 3 ==== osd_op_reply(20 [pgnls start_epoch 44 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x56229ad40500 con 0x56229b057200 2024-04-25T10:30:38.897 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:38.884+0000 7f3dca8b6640 1 -- 172.21.15.37:0/2011009937 --> [v2:172.21.15.37:6816/2746074720,v1:172.21.15.37:6817/2746074720] -- osd_op(unknown.0.0:21 6.5 6:a0000000::::head [pgnls start_epoch 44 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e44) -- 0x56229bba6800 con 0x56229b057200 2024-04-25T10:30:38.897 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:38.884+0000 7f3dca8b6640 1 -- 172.21.15.37:0/2011009937 <== osd.0 v2:172.21.15.37:6816/2746074720 4 ==== osd_op_reply(21 [pgnls start_epoch 44 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x56229ad40500 con 0x56229b057200 2024-04-25T10:30:38.897 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:38.884+0000 7f3dca8b6640 1 --2- 172.21.15.37:0/2011009937 >> [v2:172.21.15.37:6808/2754669649,v1:172.21.15.37:6809/2754669649] conn(0x56229b057f80 0x56229b05c680 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:30:38.897 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:38.884+0000 7f3dca8b6640 1 -- 172.21.15.37:0/2011009937 --> [v2:172.21.15.37:6808/2754669649,v1:172.21.15.37:6809/2754669649] -- osd_op(unknown.0.0:22 6.15 6:a8000000::::head [pgnls start_epoch 44 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e44) -- 0x56229bba6800 con 0x56229b057f80 2024-04-25T10:30:38.897 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:38.884+0000 7f3dca8b6640 1 --2- 172.21.15.37:0/2011009937 >> [v2:172.21.15.37:6808/2754669649,v1:172.21.15.37:6809/2754669649] conn(0x56229b057f80 0x56229b05c680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:30:38.897 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:38.885+0000 7f3dca8b6640 1 --2- 172.21.15.37:0/2011009937 >> [v2:172.21.15.37:6808/2754669649,v1:172.21.15.37:6809/2754669649] conn(0x56229b057f80 0x56229b05c680 crc :-1 s=READY pgs=15 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:30:38.897 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:38.885+0000 7f3dca8b6640 1 -- 172.21.15.37:0/2011009937 <== osd.2 v2:172.21.15.37:6808/2754669649 1 ==== osd_op_reply(22 [pgnls start_epoch 44 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x56229ad40500 con 0x56229b057f80 2024-04-25T10:30:38.897 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:38.885+0000 7f3dca8b6640 1 -- 172.21.15.37:0/2011009937 --> [v2:172.21.15.37:6800/1690341662,v1:172.21.15.37:6801/1690341662] -- osd_op(unknown.0.0:23 6.d 6:b0000000::::head [pgnls start_epoch 44 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e44) -- 0x56229bba6800 con 0x56229b056d80 2024-04-25T10:30:38.897 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:38.885+0000 7f3dc98b4640 1 -- 172.21.15.37:0/2011009937 <== osd.1 v2:172.21.15.37:6800/1690341662 2 ==== osd_op_reply(23 [pgnls start_epoch 44 out=74b] v44'1 uv1 ondisk = 1) ==== 144+0+74 (crc 0 0 0) 0x56229ad40000 con 0x56229b056d80 2024-04-25T10:30:38.897 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:38.885+0000 7f3dc98b4640 1 -- 172.21.15.37:0/2011009937 --> [v2:172.21.15.203:6810/2309615014,v1:172.21.15.203:6812/2309615014] -- osd_op(unknown.0.0:24 6.1d 6:b8000000::::head [pgnls start_epoch 44 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e44) -- 0x56229bba6800 con 0x56229b056900 2024-04-25T10:30:38.897 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:38.886+0000 7f3dc98b4640 1 -- 172.21.15.37:0/2011009937 <== osd.6 v2:172.21.15.203:6810/2309615014 4 ==== osd_op_reply(24 [pgnls start_epoch 44 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x56229ad40000 con 0x56229b056900 2024-04-25T10:30:38.898 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:38.886+0000 7f3dc98b4640 1 -- 172.21.15.37:0/2011009937 --> [v2:172.21.15.37:6816/2746074720,v1:172.21.15.37:6817/2746074720] -- osd_op(unknown.0.0:25 6.3 6:c0000000::::head [pgnls start_epoch 44 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e44) -- 0x56229bba6800 con 0x56229b057200 2024-04-25T10:30:38.898 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:38.886+0000 7f3dca8b6640 1 -- 172.21.15.37:0/2011009937 <== osd.0 v2:172.21.15.37:6816/2746074720 5 ==== osd_op_reply(25 [pgnls start_epoch 44 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x56229ad40500 con 0x56229b057200 2024-04-25T10:30:38.898 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:38.886+0000 7f3dca8b6640 1 -- 172.21.15.37:0/2011009937 --> [v2:172.21.15.203:6808/3688099014,v1:172.21.15.203:6809/3688099014] -- osd_op(unknown.0.0:26 6.13 6:c8000000::::head [pgnls start_epoch 44 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e44) -- 0x56229bba6800 con 0x56229adf2d00 2024-04-25T10:30:38.898 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:38.886+0000 7f3dc98b4640 1 -- 172.21.15.37:0/2011009937 <== osd.7 v2:172.21.15.203:6808/3688099014 3 ==== osd_op_reply(26 [pgnls start_epoch 44 out=98b] v44'2 uv2 ondisk = 1) ==== 144+0+98 (crc 0 0 0) 0x56229ad40000 con 0x56229adf2d00 2024-04-25T10:30:38.898 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:38.886+0000 7f3dc98b4640 1 -- 172.21.15.37:0/2011009937 --> [v2:172.21.15.203:6800/4051532063,v1:172.21.15.203:6801/4051532063] -- osd_op(unknown.0.0:27 6.b 6:d0000000::::head [pgnls start_epoch 44 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e44) -- 0x56229bba6800 con 0x56229b056000 2024-04-25T10:30:38.898 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:38.887+0000 7f3dca8b6640 1 -- 172.21.15.37:0/2011009937 <== osd.4 v2:172.21.15.203:6800/4051532063 6 ==== osd_op_reply(27 [pgnls start_epoch 44 out=76b] v44'1 uv1 ondisk = 1) ==== 144+0+76 (crc 0 0 0) 0x56229ad40500 con 0x56229b056000 2024-04-25T10:30:38.898 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:38.887+0000 7f3dca8b6640 1 -- 172.21.15.37:0/2011009937 --> [v2:172.21.15.203:6810/2309615014,v1:172.21.15.203:6812/2309615014] -- osd_op(unknown.0.0:28 6.1b 6:d8000000::::head [pgnls start_epoch 44 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e44) -- 0x56229bba6800 con 0x56229b056900 2024-04-25T10:30:38.898 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:38.887+0000 7f3dc98b4640 1 -- 172.21.15.37:0/2011009937 <== osd.6 v2:172.21.15.203:6810/2309615014 5 ==== osd_op_reply(28 [pgnls start_epoch 44 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x56229ad40000 con 0x56229b056900 2024-04-25T10:30:38.898 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:38.887+0000 7f3dc98b4640 1 -- 172.21.15.37:0/2011009937 --> [v2:172.21.15.37:6824/1815290618,v1:172.21.15.37:6825/1815290618] -- osd_op(unknown.0.0:29 6.7 6:e0000000::::head [pgnls start_epoch 44 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e44) -- 0x56229bba6800 con 0x56229b056480 2024-04-25T10:30:38.898 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:38.887+0000 7f3dca0b5640 1 -- 172.21.15.37:0/2011009937 <== osd.3 v2:172.21.15.37:6824/1815290618 6 ==== osd_op_reply(29 [pgnls start_epoch 44 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x56229ad40780 con 0x56229b056480 2024-04-25T10:30:38.898 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:38.887+0000 7f3dca0b5640 1 -- 172.21.15.37:0/2011009937 --> [v2:172.21.15.37:6800/1690341662,v1:172.21.15.37:6801/1690341662] -- osd_op(unknown.0.0:30 6.17 6:e8000000::::head [pgnls start_epoch 44 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e44) -- 0x56229bba6800 con 0x56229b056d80 2024-04-25T10:30:38.898 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:38.888+0000 7f3dc98b4640 1 -- 172.21.15.37:0/2011009937 <== osd.1 v2:172.21.15.37:6800/1690341662 3 ==== osd_op_reply(30 [pgnls start_epoch 44 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x56229ad40000 con 0x56229b056d80 2024-04-25T10:30:38.899 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:38.888+0000 7f3dc98b4640 1 -- 172.21.15.37:0/2011009937 --> [v2:172.21.15.203:6824/1203822890,v1:172.21.15.203:6825/1203822890] -- osd_op(unknown.0.0:31 6.f 6:f0000000::::head [pgnls start_epoch 44 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e44) -- 0x56229bba6800 con 0x56229b057680 2024-04-25T10:30:38.899 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:38.888+0000 7f3dca0b5640 1 -- 172.21.15.37:0/2011009937 <== osd.5 v2:172.21.15.203:6824/1203822890 2 ==== osd_op_reply(31 [pgnls start_epoch 44 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x56229ad40780 con 0x56229b057680 2024-04-25T10:30:38.899 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:38.888+0000 7f3dca0b5640 1 -- 172.21.15.37:0/2011009937 --> [v2:172.21.15.203:6810/2309615014,v1:172.21.15.203:6812/2309615014] -- osd_op(unknown.0.0:32 6.1f 6:f8000000::::head [pgnls start_epoch 44 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e44) -- 0x56229bba6800 con 0x56229b056900 2024-04-25T10:30:38.899 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:38.888+0000 7f3dc98b4640 1 -- 172.21.15.37:0/2011009937 <== osd.6 v2:172.21.15.203:6810/2309615014 6 ==== osd_op_reply(32 [pgnls start_epoch 44 out=101b] v44'3 uv3 ondisk = 1) ==== 144+0+101 (crc 0 0 0) 0x56229ad40000 con 0x56229b056900 2024-04-25T10:30:38.899 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:38.888+0000 7f3dcc412740 10 datascan.scan_links: step 1: handling object 10000000000.00000000 2024-04-25T10:30:38.899 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:38.888+0000 7f3dcc412740 1 -- 172.21.15.37:0/2011009937 --> [v2:172.21.15.203:6800/4051532063,v1:172.21.15.203:6801/4051532063] -- osd_op(unknown.0.0:33 6.10 6:0cf6ad0f:::10000000000.00000000:head [omap-get-vals in=16b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e44) -- 0x56229b082000 con 0x56229b056000 2024-04-25T10:30:38.899 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:38.889+0000 7f3dca8b6640 1 -- 172.21.15.37:0/2011009937 <== osd.4 v2:172.21.15.203:6800/4051532063 7 ==== osd_op_reply(33 10000000000.00000000 [omap-get-vals out=509b] v0'0 uv2 ondisk = 0) ==== 164+0+509 (crc 0 0 0) 0x56229ad40500 con 0x56229b056000 2024-04-25T10:30:38.899 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:38.889+0000 7f3dcc412740 20 datascan.scan_links: injected ino detected 2024-04-25T10:30:38.899 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:38.889+0000 7f3dcc412740 10 datascan.scan_links: step 1: handling object 1.00000000.inode 2024-04-25T10:30:38.899 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:38.889+0000 7f3dcc412740 10 datascan.scan_links: Not a dirfrag: '1.00000000.inode' 2024-04-25T10:30:38.899 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:38.889+0000 7f3dcc412740 10 datascan.scan_links: step 1: handling object mds_snaptable 2024-04-25T10:30:38.899 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:38.889+0000 7f3dcc412740 10 datascan.scan_links: Not a dirfrag: 'mds_snaptable' 2024-04-25T10:30:38.899 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:38.889+0000 7f3dcc412740 10 datascan.scan_links: step 1: handling object mds0_inotable 2024-04-25T10:30:38.899 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:38.889+0000 7f3dcc412740 10 datascan.scan_links: Not a dirfrag: 'mds0_inotable' 2024-04-25T10:30:38.900 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:38.889+0000 7f3dcc412740 10 datascan.scan_links: step 1: handling object 100.00000000 2024-04-25T10:30:38.900 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:38.889+0000 7f3dcc412740 1 -- 172.21.15.37:0/2011009937 --> [v2:172.21.15.203:6808/3688099014,v1:172.21.15.203:6809/3688099014] -- osd_op(unknown.0.0:34 6.13 6:cd5a64a3:::100.00000000:head [omap-get-vals in=16b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e44) -- 0x56229b082400 con 0x56229adf2d00 2024-04-25T10:30:38.900 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:38.889+0000 7f3dc98b4640 1 -- 172.21.15.37:0/2011009937 <== osd.7 v2:172.21.15.203:6808/3688099014 4 ==== osd_op_reply(34 100.00000000 [omap-get-vals out=5b] v0'0 uv1 ondisk = 0) ==== 156+0+5 (crc 0 0 0) 0x56229ad40000 con 0x56229adf2d00 2024-04-25T10:30:38.900 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:38.889+0000 7f3dcc412740 10 datascan.scan_links: step 1: handling object mds0_sessionmap 2024-04-25T10:30:38.900 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:38.889+0000 7f3dcc412740 10 datascan.scan_links: Not a dirfrag: 'mds0_sessionmap' 2024-04-25T10:30:38.900 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:38.889+0000 7f3dcc412740 10 datascan.scan_links: step 1: handling object 100.00000000.inode 2024-04-25T10:30:38.900 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:38.889+0000 7f3dcc412740 10 datascan.scan_links: Not a dirfrag: '100.00000000.inode' 2024-04-25T10:30:38.900 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:38.889+0000 7f3dcc412740 10 datascan.scan_links: step 1: handling object 1.00000000 2024-04-25T10:30:38.900 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:38.889+0000 7f3dcc412740 1 -- 172.21.15.37:0/2011009937 --> [v2:172.21.15.203:6810/2309615014,v1:172.21.15.203:6812/2309615014] -- osd_op(unknown.0.0:35 6.1f 6:ff5b34d6:::1.00000000:head [omap-get-vals in=16b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e44) -- 0x56229b082800 con 0x56229b056900 2024-04-25T10:30:38.900 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:38.889+0000 7f3dc98b4640 1 -- 172.21.15.37:0/2011009937 <== osd.6 v2:172.21.15.203:6810/2309615014 7 ==== osd_op_reply(35 1.00000000 [omap-get-vals out=508b] v0'0 uv3 ondisk = 0) ==== 154+0+508 (crc 0 0 0) 0x56229ad40000 con 0x56229b056900 2024-04-25T10:30:38.900 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:38.889+0000 7f3dcc412740 20 datascan.scan_links: injected ino detected 2024-04-25T10:30:38.900 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:38.889+0000 7f3dcc412740 1 -- 172.21.15.37:0/2011009937 --> [v2:172.21.15.203:6808/3688099014,v1:172.21.15.203:6809/3688099014] -- osd_op(unknown.0.0:36 6.0 6:00000000::::head [pgnls start_epoch 44 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e44) -- 0x56229bba7c00 con 0x56229adf2d00 2024-04-25T10:30:38.900 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:38.890+0000 7f3dc98b4640 1 -- 172.21.15.37:0/2011009937 <== osd.7 v2:172.21.15.203:6808/3688099014 5 ==== osd_op_reply(36 [pgnls start_epoch 44 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x56229ad40000 con 0x56229adf2d00 2024-04-25T10:30:38.900 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:38.890+0000 7f3dc98b4640 1 -- 172.21.15.37:0/2011009937 --> [v2:172.21.15.203:6800/4051532063,v1:172.21.15.203:6801/4051532063] -- osd_op(unknown.0.0:37 6.10 6:08000000::::head [pgnls start_epoch 44 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e44) -- 0x56229bba7c00 con 0x56229b056000 2024-04-25T10:30:38.900 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:38.890+0000 7f3dca8b6640 1 -- 172.21.15.37:0/2011009937 <== osd.4 v2:172.21.15.203:6800/4051532063 8 ==== osd_op_reply(37 [pgnls start_epoch 44 out=81b] v44'2 uv2 ondisk = 1) ==== 144+0+81 (crc 0 0 0) 0x56229ad40500 con 0x56229b056000 2024-04-25T10:30:38.901 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:38.890+0000 7f3dca8b6640 1 -- 172.21.15.37:0/2011009937 --> [v2:172.21.15.37:6824/1815290618,v1:172.21.15.37:6825/1815290618] -- osd_op(unknown.0.0:38 6.8 6:10000000::::head [pgnls start_epoch 44 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e44) -- 0x56229bba7c00 con 0x56229b056480 2024-04-25T10:30:38.901 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:38.890+0000 7f3dca0b5640 1 -- 172.21.15.37:0/2011009937 <== osd.3 v2:172.21.15.37:6824/1815290618 7 ==== osd_op_reply(38 [pgnls start_epoch 44 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x56229ad40780 con 0x56229b056480 2024-04-25T10:30:38.901 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:38.890+0000 7f3dca0b5640 1 -- 172.21.15.37:0/2011009937 --> [v2:172.21.15.203:6800/4051532063,v1:172.21.15.203:6801/4051532063] -- osd_op(unknown.0.0:39 6.18 6:18000000::::head [pgnls start_epoch 44 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e44) -- 0x56229bba7c00 con 0x56229b056000 2024-04-25T10:30:38.901 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:38.891+0000 7f3dca8b6640 1 -- 172.21.15.37:0/2011009937 <== osd.4 v2:172.21.15.203:6800/4051532063 9 ==== osd_op_reply(39 [pgnls start_epoch 44 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x56229ad40500 con 0x56229b056000 2024-04-25T10:30:38.901 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:38.891+0000 7f3dca8b6640 1 -- 172.21.15.37:0/2011009937 --> [v2:172.21.15.203:6808/3688099014,v1:172.21.15.203:6809/3688099014] -- osd_op(unknown.0.0:40 6.4 6:20000000::::head [pgnls start_epoch 44 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e44) -- 0x56229bba7c00 con 0x56229adf2d00 2024-04-25T10:30:38.901 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:38.891+0000 7f3dc98b4640 1 -- 172.21.15.37:0/2011009937 <== osd.7 v2:172.21.15.203:6808/3688099014 6 ==== osd_op_reply(40 [pgnls start_epoch 44 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x56229ad40000 con 0x56229adf2d00 2024-04-25T10:30:38.901 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:38.891+0000 7f3dc98b4640 1 -- 172.21.15.37:0/2011009937 --> [v2:172.21.15.203:6800/4051532063,v1:172.21.15.203:6801/4051532063] -- osd_op(unknown.0.0:41 6.14 6:28000000::::head [pgnls start_epoch 44 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e44) -- 0x56229bba7c00 con 0x56229b056000 2024-04-25T10:30:38.901 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:38.891+0000 7f3dca8b6640 1 -- 172.21.15.37:0/2011009937 <== osd.4 v2:172.21.15.203:6800/4051532063 10 ==== osd_op_reply(41 [pgnls start_epoch 44 out=77b] v44'1 uv1 ondisk = 1) ==== 144+0+77 (crc 0 0 0) 0x56229ad40500 con 0x56229b056000 2024-04-25T10:30:38.901 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:38.891+0000 7f3dca8b6640 1 -- 172.21.15.37:0/2011009937 --> [v2:172.21.15.203:6810/2309615014,v1:172.21.15.203:6812/2309615014] -- osd_op(unknown.0.0:42 6.c 6:30000000::::head [pgnls start_epoch 44 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e44) -- 0x56229bba7c00 con 0x56229b056900 2024-04-25T10:30:38.901 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:38.892+0000 7f3dc98b4640 1 -- 172.21.15.37:0/2011009937 <== osd.6 v2:172.21.15.203:6810/2309615014 8 ==== osd_op_reply(42 [pgnls start_epoch 44 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x56229ad40000 con 0x56229b056900 2024-04-25T10:30:38.902 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:38.892+0000 7f3dc98b4640 1 -- 172.21.15.37:0/2011009937 --> [v2:172.21.15.37:6824/1815290618,v1:172.21.15.37:6825/1815290618] -- osd_op(unknown.0.0:43 6.1c 6:38000000::::head [pgnls start_epoch 44 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e44) -- 0x56229bba7c00 con 0x56229b056480 2024-04-25T10:30:38.902 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:38.892+0000 7f3dca0b5640 1 -- 172.21.15.37:0/2011009937 <== osd.3 v2:172.21.15.37:6824/1815290618 8 ==== osd_op_reply(43 [pgnls start_epoch 44 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x56229ad40780 con 0x56229b056480 2024-04-25T10:30:38.902 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:38.892+0000 7f3dca0b5640 1 -- 172.21.15.37:0/2011009937 --> [v2:172.21.15.37:6824/1815290618,v1:172.21.15.37:6825/1815290618] -- osd_op(unknown.0.0:44 6.2 6:40000000::::head [pgnls start_epoch 44 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e44) -- 0x56229bba7c00 con 0x56229b056480 2024-04-25T10:30:38.902 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:38.892+0000 7f3dca0b5640 1 -- 172.21.15.37:0/2011009937 <== osd.3 v2:172.21.15.37:6824/1815290618 9 ==== osd_op_reply(44 [pgnls start_epoch 44 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x56229ad40780 con 0x56229b056480 2024-04-25T10:30:38.902 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:38.892+0000 7f3dca0b5640 1 -- 172.21.15.37:0/2011009937 --> [v2:172.21.15.37:6816/2746074720,v1:172.21.15.37:6817/2746074720] -- osd_op(unknown.0.0:45 6.12 6:48000000::::head [pgnls start_epoch 44 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e44) -- 0x56229bba7c00 con 0x56229b057200 2024-04-25T10:30:38.902 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:38.892+0000 7f3dca8b6640 1 -- 172.21.15.37:0/2011009937 <== osd.0 v2:172.21.15.37:6816/2746074720 6 ==== osd_op_reply(45 [pgnls start_epoch 44 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x56229ad40500 con 0x56229b057200 2024-04-25T10:30:38.902 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:38.892+0000 7f3dca8b6640 1 -- 172.21.15.37:0/2011009937 --> [v2:172.21.15.37:6816/2746074720,v1:172.21.15.37:6817/2746074720] -- osd_op(unknown.0.0:46 6.a 6:50000000::::head [pgnls start_epoch 44 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e44) -- 0x56229bba7c00 con 0x56229b057200 2024-04-25T10:30:38.902 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:38.893+0000 7f3dca8b6640 1 -- 172.21.15.37:0/2011009937 <== osd.0 v2:172.21.15.37:6816/2746074720 7 ==== osd_op_reply(46 [pgnls start_epoch 44 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x56229ad40500 con 0x56229b057200 2024-04-25T10:30:38.902 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:38.893+0000 7f3dca8b6640 1 -- 172.21.15.37:0/2011009937 --> [v2:172.21.15.37:6824/1815290618,v1:172.21.15.37:6825/1815290618] -- osd_op(unknown.0.0:47 6.1a 6:58000000::::head [pgnls start_epoch 44 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e44) -- 0x56229bba7c00 con 0x56229b056480 2024-04-25T10:30:38.902 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:38.893+0000 7f3dca0b5640 1 -- 172.21.15.37:0/2011009937 <== osd.3 v2:172.21.15.37:6824/1815290618 10 ==== osd_op_reply(47 [pgnls start_epoch 44 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x56229ad40780 con 0x56229b056480 2024-04-25T10:30:38.902 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:38.893+0000 7f3dca0b5640 1 -- 172.21.15.37:0/2011009937 --> [v2:172.21.15.203:6824/1203822890,v1:172.21.15.203:6825/1203822890] -- osd_op(unknown.0.0:48 6.6 6:60000000::::head [pgnls start_epoch 44 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e44) -- 0x56229bba7c00 con 0x56229b057680 2024-04-25T10:30:38.902 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:38.893+0000 7f3dca0b5640 1 -- 172.21.15.37:0/2011009937 <== osd.5 v2:172.21.15.203:6824/1203822890 3 ==== osd_op_reply(48 [pgnls start_epoch 44 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x56229ad40780 con 0x56229b057680 2024-04-25T10:30:38.902 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:38.893+0000 7f3dca0b5640 1 -- 172.21.15.37:0/2011009937 --> [v2:172.21.15.203:6800/4051532063,v1:172.21.15.203:6801/4051532063] -- osd_op(unknown.0.0:49 6.16 6:68000000::::head [pgnls start_epoch 44 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e44) -- 0x56229bba7c00 con 0x56229b056000 2024-04-25T10:30:38.902 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:38.893+0000 7f3dca8b6640 1 -- 172.21.15.37:0/2011009937 <== osd.4 v2:172.21.15.203:6800/4051532063 11 ==== osd_op_reply(49 [pgnls start_epoch 44 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x56229ad40500 con 0x56229b056000 2024-04-25T10:30:38.903 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:38.893+0000 7f3dca8b6640 1 -- 172.21.15.37:0/2011009937 --> [v2:172.21.15.37:6800/1690341662,v1:172.21.15.37:6801/1690341662] -- osd_op(unknown.0.0:50 6.e 6:70000000::::head [pgnls start_epoch 44 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e44) -- 0x56229bba7c00 con 0x56229b056d80 2024-04-25T10:30:38.903 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:38.894+0000 7f3dc98b4640 1 -- 172.21.15.37:0/2011009937 <== osd.1 v2:172.21.15.37:6800/1690341662 4 ==== osd_op_reply(50 [pgnls start_epoch 44 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x56229ad40000 con 0x56229b056d80 2024-04-25T10:30:38.903 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:38.894+0000 7f3dc98b4640 1 -- 172.21.15.37:0/2011009937 --> [v2:172.21.15.37:6824/1815290618,v1:172.21.15.37:6825/1815290618] -- osd_op(unknown.0.0:51 6.1e 6:78000000::::head [pgnls start_epoch 44 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e44) -- 0x56229bba7c00 con 0x56229b056480 2024-04-25T10:30:38.903 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:38.894+0000 7f3dca0b5640 1 -- 172.21.15.37:0/2011009937 <== osd.3 v2:172.21.15.37:6824/1815290618 11 ==== osd_op_reply(51 [pgnls start_epoch 44 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x56229ad40780 con 0x56229b056480 2024-04-25T10:30:38.903 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:38.894+0000 7f3dca0b5640 1 -- 172.21.15.37:0/2011009937 --> [v2:172.21.15.203:6810/2309615014,v1:172.21.15.203:6812/2309615014] -- osd_op(unknown.0.0:52 6.1 6:80000000::::head [pgnls start_epoch 44 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e44) -- 0x56229bba7c00 con 0x56229b056900 2024-04-25T10:30:38.911 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:38.894+0000 7f3dc98b4640 1 -- 172.21.15.37:0/2011009937 <== osd.6 v2:172.21.15.203:6810/2309615014 9 ==== osd_op_reply(52 [pgnls start_epoch 44 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x56229ad40000 con 0x56229b056900 2024-04-25T10:30:38.911 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:38.894+0000 7f3dc98b4640 1 -- 172.21.15.37:0/2011009937 --> [v2:172.21.15.203:6810/2309615014,v1:172.21.15.203:6812/2309615014] -- osd_op(unknown.0.0:53 6.11 6:88000000::::head [pgnls start_epoch 44 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e44) -- 0x56229bba7c00 con 0x56229b056900 2024-04-25T10:30:38.911 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:38.895+0000 7f3dc98b4640 1 -- 172.21.15.37:0/2011009937 <== osd.6 v2:172.21.15.203:6810/2309615014 10 ==== osd_op_reply(53 [pgnls start_epoch 44 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x56229ad40000 con 0x56229b056900 2024-04-25T10:30:38.911 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:38.895+0000 7f3dc98b4640 1 -- 172.21.15.37:0/2011009937 --> [v2:172.21.15.203:6800/4051532063,v1:172.21.15.203:6801/4051532063] -- osd_op(unknown.0.0:54 6.9 6:90000000::::head [pgnls start_epoch 44 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e44) -- 0x56229bba7c00 con 0x56229b056000 2024-04-25T10:30:38.911 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:38.895+0000 7f3dca8b6640 1 -- 172.21.15.37:0/2011009937 <== osd.4 v2:172.21.15.203:6800/4051532063 12 ==== osd_op_reply(54 [pgnls start_epoch 44 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x56229ad40500 con 0x56229b056000 2024-04-25T10:30:38.911 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:38.895+0000 7f3dca8b6640 1 -- 172.21.15.37:0/2011009937 --> [v2:172.21.15.37:6816/2746074720,v1:172.21.15.37:6817/2746074720] -- osd_op(unknown.0.0:55 6.19 6:98000000::::head [pgnls start_epoch 44 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e44) -- 0x56229bba7c00 con 0x56229b057200 2024-04-25T10:30:38.911 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:38.895+0000 7f3dca8b6640 1 -- 172.21.15.37:0/2011009937 <== osd.0 v2:172.21.15.37:6816/2746074720 8 ==== osd_op_reply(55 [pgnls start_epoch 44 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x56229ad40500 con 0x56229b057200 2024-04-25T10:30:38.911 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:38.895+0000 7f3dca8b6640 1 -- 172.21.15.37:0/2011009937 --> [v2:172.21.15.37:6816/2746074720,v1:172.21.15.37:6817/2746074720] -- osd_op(unknown.0.0:56 6.5 6:a0000000::::head [pgnls start_epoch 44 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e44) -- 0x56229bba7c00 con 0x56229b057200 2024-04-25T10:30:38.911 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:38.895+0000 7f3dca8b6640 1 -- 172.21.15.37:0/2011009937 <== osd.0 v2:172.21.15.37:6816/2746074720 9 ==== osd_op_reply(56 [pgnls start_epoch 44 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x56229ad40500 con 0x56229b057200 2024-04-25T10:30:38.911 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:38.895+0000 7f3dca8b6640 1 -- 172.21.15.37:0/2011009937 --> [v2:172.21.15.37:6808/2754669649,v1:172.21.15.37:6809/2754669649] -- osd_op(unknown.0.0:57 6.15 6:a8000000::::head [pgnls start_epoch 44 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e44) -- 0x56229bba7c00 con 0x56229b057f80 2024-04-25T10:30:38.911 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:38.896+0000 7f3dca8b6640 1 -- 172.21.15.37:0/2011009937 <== osd.2 v2:172.21.15.37:6808/2754669649 2 ==== osd_op_reply(57 [pgnls start_epoch 44 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x56229ad40500 con 0x56229b057f80 2024-04-25T10:30:38.912 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:38.896+0000 7f3dca8b6640 1 -- 172.21.15.37:0/2011009937 --> [v2:172.21.15.37:6800/1690341662,v1:172.21.15.37:6801/1690341662] -- osd_op(unknown.0.0:58 6.d 6:b0000000::::head [pgnls start_epoch 44 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e44) -- 0x56229bba7c00 con 0x56229b056d80 2024-04-25T10:30:38.912 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:38.896+0000 7f3dc98b4640 1 -- 172.21.15.37:0/2011009937 <== osd.1 v2:172.21.15.37:6800/1690341662 5 ==== osd_op_reply(58 [pgnls start_epoch 44 out=74b] v44'1 uv1 ondisk = 1) ==== 144+0+74 (crc 0 0 0) 0x56229ad40000 con 0x56229b056d80 2024-04-25T10:30:38.912 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:38.896+0000 7f3dc98b4640 1 -- 172.21.15.37:0/2011009937 --> [v2:172.21.15.203:6810/2309615014,v1:172.21.15.203:6812/2309615014] -- osd_op(unknown.0.0:59 6.1d 6:b8000000::::head [pgnls start_epoch 44 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e44) -- 0x56229bba7c00 con 0x56229b056900 2024-04-25T10:30:38.912 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:38.896+0000 7f3dc98b4640 1 -- 172.21.15.37:0/2011009937 <== osd.6 v2:172.21.15.203:6810/2309615014 11 ==== osd_op_reply(59 [pgnls start_epoch 44 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x56229ad40000 con 0x56229b056900 2024-04-25T10:30:38.912 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:38.896+0000 7f3dc98b4640 1 -- 172.21.15.37:0/2011009937 --> [v2:172.21.15.37:6816/2746074720,v1:172.21.15.37:6817/2746074720] -- osd_op(unknown.0.0:60 6.3 6:c0000000::::head [pgnls start_epoch 44 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e44) -- 0x56229bba7c00 con 0x56229b057200 2024-04-25T10:30:38.912 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:38.897+0000 7f3dca8b6640 1 -- 172.21.15.37:0/2011009937 <== osd.0 v2:172.21.15.37:6816/2746074720 10 ==== osd_op_reply(60 [pgnls start_epoch 44 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x56229ad40500 con 0x56229b057200 2024-04-25T10:30:38.912 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:38.897+0000 7f3dca8b6640 1 -- 172.21.15.37:0/2011009937 --> [v2:172.21.15.203:6808/3688099014,v1:172.21.15.203:6809/3688099014] -- osd_op(unknown.0.0:61 6.13 6:c8000000::::head [pgnls start_epoch 44 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e44) -- 0x56229bba7c00 con 0x56229adf2d00 2024-04-25T10:30:38.912 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:38.897+0000 7f3dc98b4640 1 -- 172.21.15.37:0/2011009937 <== osd.7 v2:172.21.15.203:6808/3688099014 7 ==== osd_op_reply(61 [pgnls start_epoch 44 out=98b] v44'2 uv2 ondisk = 1) ==== 144+0+98 (crc 0 0 0) 0x56229ad40000 con 0x56229adf2d00 2024-04-25T10:30:38.912 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:38.897+0000 7f3dc98b4640 1 -- 172.21.15.37:0/2011009937 --> [v2:172.21.15.203:6800/4051532063,v1:172.21.15.203:6801/4051532063] -- osd_op(unknown.0.0:62 6.b 6:d0000000::::head [pgnls start_epoch 44 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e44) -- 0x56229bba7c00 con 0x56229b056000 2024-04-25T10:30:38.912 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:38.897+0000 7f3dca8b6640 1 -- 172.21.15.37:0/2011009937 <== osd.4 v2:172.21.15.203:6800/4051532063 13 ==== osd_op_reply(62 [pgnls start_epoch 44 out=76b] v44'1 uv1 ondisk = 1) ==== 144+0+76 (crc 0 0 0) 0x56229ad40500 con 0x56229b056000 2024-04-25T10:30:38.912 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:38.897+0000 7f3dca8b6640 1 -- 172.21.15.37:0/2011009937 --> [v2:172.21.15.203:6810/2309615014,v1:172.21.15.203:6812/2309615014] -- osd_op(unknown.0.0:63 6.1b 6:d8000000::::head [pgnls start_epoch 44 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e44) -- 0x56229bba7c00 con 0x56229b056900 2024-04-25T10:30:38.912 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:38.898+0000 7f3dc98b4640 1 -- 172.21.15.37:0/2011009937 <== osd.6 v2:172.21.15.203:6810/2309615014 12 ==== osd_op_reply(63 [pgnls start_epoch 44 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x56229ad40000 con 0x56229b056900 2024-04-25T10:30:38.912 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:38.898+0000 7f3dc98b4640 1 -- 172.21.15.37:0/2011009937 --> [v2:172.21.15.37:6824/1815290618,v1:172.21.15.37:6825/1815290618] -- osd_op(unknown.0.0:64 6.7 6:e0000000::::head [pgnls start_epoch 44 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e44) -- 0x56229bba7c00 con 0x56229b056480 2024-04-25T10:30:38.912 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:38.898+0000 7f3dca0b5640 1 -- 172.21.15.37:0/2011009937 <== osd.3 v2:172.21.15.37:6824/1815290618 12 ==== osd_op_reply(64 [pgnls start_epoch 44 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x56229ad40780 con 0x56229b056480 2024-04-25T10:30:38.913 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:38.898+0000 7f3dca0b5640 1 -- 172.21.15.37:0/2011009937 --> [v2:172.21.15.37:6800/1690341662,v1:172.21.15.37:6801/1690341662] -- osd_op(unknown.0.0:65 6.17 6:e8000000::::head [pgnls start_epoch 44 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e44) -- 0x56229bba7c00 con 0x56229b056d80 2024-04-25T10:30:38.913 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:38.898+0000 7f3dc98b4640 1 -- 172.21.15.37:0/2011009937 <== osd.1 v2:172.21.15.37:6800/1690341662 6 ==== osd_op_reply(65 [pgnls start_epoch 44 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x56229ad40000 con 0x56229b056d80 2024-04-25T10:30:38.913 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:38.898+0000 7f3dc98b4640 1 -- 172.21.15.37:0/2011009937 --> [v2:172.21.15.203:6824/1203822890,v1:172.21.15.203:6825/1203822890] -- osd_op(unknown.0.0:66 6.f 6:f0000000::::head [pgnls start_epoch 44 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e44) -- 0x56229bba7c00 con 0x56229b057680 2024-04-25T10:30:38.913 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:38.899+0000 7f3dca0b5640 1 -- 172.21.15.37:0/2011009937 <== osd.5 v2:172.21.15.203:6824/1203822890 4 ==== osd_op_reply(66 [pgnls start_epoch 44 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x56229ad40780 con 0x56229b057680 2024-04-25T10:30:38.913 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:38.899+0000 7f3dca0b5640 1 -- 172.21.15.37:0/2011009937 --> [v2:172.21.15.203:6810/2309615014,v1:172.21.15.203:6812/2309615014] -- osd_op(unknown.0.0:67 6.1f 6:f8000000::::head [pgnls start_epoch 44 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e44) -- 0x56229bba7c00 con 0x56229b056900 2024-04-25T10:30:38.913 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:38.899+0000 7f3dc98b4640 1 -- 172.21.15.37:0/2011009937 <== osd.6 v2:172.21.15.203:6810/2309615014 13 ==== osd_op_reply(67 [pgnls start_epoch 44 out=101b] v44'3 uv3 ondisk = 1) ==== 144+0+101 (crc 0 0 0) 0x56229ad40000 con 0x56229b056900 2024-04-25T10:30:38.913 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:38.899+0000 7f3dcc412740 10 datascan.scan_links: step 2: handling object 10000000000.00000000 2024-04-25T10:30:38.913 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:38.899+0000 7f3dcc412740 1 -- 172.21.15.37:0/2011009937 --> [v2:172.21.15.203:6800/4051532063,v1:172.21.15.203:6801/4051532063] -- osd_op(unknown.0.0:68 6.10 6:0cf6ad0f:::10000000000.00000000:head [omap-get-vals in=16b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e44) -- 0x56229bba6400 con 0x56229b056000 2024-04-25T10:30:38.913 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:38.899+0000 7f3dca8b6640 1 -- 172.21.15.37:0/2011009937 <== osd.4 v2:172.21.15.203:6800/4051532063 14 ==== osd_op_reply(68 10000000000.00000000 [omap-get-vals out=509b] v0'0 uv2 ondisk = 0) ==== 164+0+509 (crc 0 0 0) 0x56229ad40500 con 0x56229b056000 2024-04-25T10:30:38.913 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:38.899+0000 7f3dcc412740 20 datascan.scan_links: injected ino detected 2024-04-25T10:30:38.913 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:38.899+0000 7f3dcc412740 20 datascan.scan_links: adding 0x10000000001 for future processing to fix dnfirst 2024-04-25T10:30:38.913 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:38.899+0000 7f3dcc412740 10 datascan.scan_links: step 2: handling object 1.00000000.inode 2024-04-25T10:30:38.913 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:38.899+0000 7f3dcc412740 10 datascan.scan_links: Not a dirfrag: '1.00000000.inode' 2024-04-25T10:30:38.913 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:38.899+0000 7f3dcc412740 10 datascan.scan_links: step 2: handling object mds_snaptable 2024-04-25T10:30:38.913 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:38.899+0000 7f3dcc412740 10 datascan.scan_links: Not a dirfrag: 'mds_snaptable' 2024-04-25T10:30:38.914 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:38.899+0000 7f3dcc412740 10 datascan.scan_links: step 2: handling object mds0_inotable 2024-04-25T10:30:38.914 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:38.899+0000 7f3dcc412740 10 datascan.scan_links: Not a dirfrag: 'mds0_inotable' 2024-04-25T10:30:38.914 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:38.899+0000 7f3dcc412740 10 datascan.scan_links: step 2: handling object 100.00000000 2024-04-25T10:30:38.914 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:38.899+0000 7f3dcc412740 1 -- 172.21.15.37:0/2011009937 --> [v2:172.21.15.203:6808/3688099014,v1:172.21.15.203:6809/3688099014] -- osd_op(unknown.0.0:69 6.13 6:cd5a64a3:::100.00000000:head [omap-get-vals in=16b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e44) -- 0x56229bba6000 con 0x56229adf2d00 2024-04-25T10:30:38.914 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:38.900+0000 7f3dc98b4640 1 -- 172.21.15.37:0/2011009937 <== osd.7 v2:172.21.15.203:6808/3688099014 8 ==== osd_op_reply(69 100.00000000 [omap-get-vals out=5b] v0'0 uv1 ondisk = 0) ==== 156+0+5 (crc 0 0 0) 0x56229ad40000 con 0x56229adf2d00 2024-04-25T10:30:38.914 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:38.900+0000 7f3dcc412740 10 datascan.scan_links: step 2: handling object mds0_sessionmap 2024-04-25T10:30:38.914 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:38.900+0000 7f3dcc412740 10 datascan.scan_links: Not a dirfrag: 'mds0_sessionmap' 2024-04-25T10:30:38.914 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:38.900+0000 7f3dcc412740 10 datascan.scan_links: step 2: handling object 100.00000000.inode 2024-04-25T10:30:38.914 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:38.900+0000 7f3dcc412740 10 datascan.scan_links: Not a dirfrag: '100.00000000.inode' 2024-04-25T10:30:38.914 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:38.900+0000 7f3dcc412740 10 datascan.scan_links: step 2: handling object 1.00000000 2024-04-25T10:30:38.914 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:38.900+0000 7f3dcc412740 1 -- 172.21.15.37:0/2011009937 --> [v2:172.21.15.203:6810/2309615014,v1:172.21.15.203:6812/2309615014] -- osd_op(unknown.0.0:70 6.1f 6:ff5b34d6:::1.00000000:head [omap-get-vals in=16b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e44) -- 0x56229bba6800 con 0x56229b056900 2024-04-25T10:30:38.914 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:38.900+0000 7f3dc98b4640 1 -- 172.21.15.37:0/2011009937 <== osd.6 v2:172.21.15.203:6810/2309615014 14 ==== osd_op_reply(70 1.00000000 [omap-get-vals out=508b] v0'0 uv3 ondisk = 0) ==== 154+0+508 (crc 0 0 0) 0x56229ad40000 con 0x56229b056900 2024-04-25T10:30:38.914 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:38.900+0000 7f3dcc412740 20 datascan.scan_links: injected ino detected 2024-04-25T10:30:38.914 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:38.900+0000 7f3dcc412740 20 datascan.scan_links: adding 0x10000000000 for future processing to fix dnfirst 2024-04-25T10:30:38.915 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:38.900+0000 7f3dcc412740 10 datascan.scan_links: processing 0 dup_primaries, 0 remote_links 2024-04-25T10:30:38.915 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:38.900+0000 7f3dcc412740 10 datascan.scan_links: removing dup dentries from 0 objects 2024-04-25T10:30:38.915 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:38.900+0000 7f3dcc412740 10 datascan.scan_links: processing 0 bad_nlink_inos 2024-04-25T10:30:38.915 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:38.900+0000 7f3dcc412740 10 datascan.scan_links: processing 2 injected_inos 2024-04-25T10:30:38.915 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:38.900+0000 7f3dcc412740 10 datascan.scan_links: handling injected_ino 0x10000000000 2024-04-25T10:30:38.915 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:38.900+0000 7f3dcc412740 20 datascan.scan_links: fixing linkage (dnfirst) of 0x1:subdir 2024-04-25T10:30:38.915 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:38.900+0000 7f3dcc412740 1 -- 172.21.15.37:0/2011009937 --> [v2:172.21.15.203:6810/2309615014,v1:172.21.15.203:6812/2309615014] -- osd_op(unknown.0.0:71 6.1f 6:ff5b34d6:::1.00000000:head [omap-get-vals-by-keys in=19b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e44) -- 0x56229b082400 con 0x56229b056900 2024-04-25T10:30:38.915 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:38.900+0000 7f3dc98b4640 1 -- 172.21.15.37:0/2011009937 <== osd.6 v2:172.21.15.203:6810/2309615014 15 ==== osd_op_reply(71 1.00000000 [omap-get-vals-by-keys out=507b] v0'0 uv3 ondisk = 0) ==== 154+0+507 (crc 0 0 0) 0x56229ad40000 con 0x56229b056900 2024-04-25T10:30:38.915 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:38.900+0000 7f3dcc412740 20 datascan.read_dentry: oid=1.00000000 dname=subdir frag=*, r=0 2024-04-25T10:30:38.915 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:38.900+0000 7f3dcc412740 20 datascan.scan_links: first is now 2 2024-04-25T10:30:38.915 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:38.900+0000 7f3dcc412740 1 -- 172.21.15.37:0/2011009937 --> [v2:172.21.15.203:6810/2309615014,v1:172.21.15.203:6812/2309615014] -- osd_op(unknown.0.0:72 6.1f 6:ff5b34d6:::1.00000000:head [omap-set-vals in=507b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e44) -- 0x56229b082800 con 0x56229b056900 2024-04-25T10:30:38.915 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:38.902+0000 7f3dc98b4640 1 -- 172.21.15.37:0/2011009937 <== osd.6 v2:172.21.15.203:6810/2309615014 16 ==== osd_op_reply(72 1.00000000 [omap-set-vals] v44'4 uv4 ondisk = 0) ==== 154+0+0 (crc 0 0 0) 0x56229ad40000 con 0x56229b056900 2024-04-25T10:30:38.915 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:38.902+0000 7f3dcc412740 20 datascan.inject_linkage: Injected dentry 0x0x1/subdir pointing to 0x0x10000000000 2024-04-25T10:30:38.915 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:38.902+0000 7f3dcc412740 10 datascan.scan_links: handling injected_ino 0x10000000001 2024-04-25T10:30:38.916 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:38.902+0000 7f3dcc412740 20 datascan.scan_links: fixing linkage (dnfirst) of 0x10000000000:sixmegs 2024-04-25T10:30:38.916 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:38.902+0000 7f3dcc412740 1 -- 172.21.15.37:0/2011009937 --> [v2:172.21.15.203:6800/4051532063,v1:172.21.15.203:6801/4051532063] -- osd_op(unknown.0.0:73 6.10 6:0cf6ad0f:::10000000000.00000000:head [omap-get-vals-by-keys in=20b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e44) -- 0x56229b082c00 con 0x56229b056000 2024-04-25T10:30:38.916 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:38.902+0000 7f3dca8b6640 1 -- 172.21.15.37:0/2011009937 <== osd.4 v2:172.21.15.203:6800/4051532063 15 ==== osd_op_reply(73 10000000000.00000000 [omap-get-vals-by-keys out=508b] v0'0 uv2 ondisk = 0) ==== 164+0+508 (crc 0 0 0) 0x56229ad40500 con 0x56229b056000 2024-04-25T10:30:38.916 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:38.902+0000 7f3dcc412740 20 datascan.read_dentry: oid=10000000000.00000000 dname=sixmegs frag=*, r=0 2024-04-25T10:30:38.916 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:38.902+0000 7f3dcc412740 20 datascan.scan_links: first is now 2 2024-04-25T10:30:38.916 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:38.902+0000 7f3dcc412740 1 -- 172.21.15.37:0/2011009937 --> [v2:172.21.15.203:6800/4051532063,v1:172.21.15.203:6801/4051532063] -- osd_op(unknown.0.0:74 6.10 6:0cf6ad0f:::10000000000.00000000:head [omap-set-vals in=508b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e44) -- 0x56229b083000 con 0x56229b056000 2024-04-25T10:30:38.916 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:38.903+0000 7f3dca8b6640 1 -- 172.21.15.37:0/2011009937 <== osd.4 v2:172.21.15.203:6800/4051532063 16 ==== osd_op_reply(74 10000000000.00000000 [omap-set-vals] v44'3 uv3 ondisk = 0) ==== 164+0+0 (crc 0 0 0) 0x56229ad40500 con 0x56229b056000 2024-04-25T10:30:38.916 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:38.903+0000 7f3dcc412740 20 datascan.inject_linkage: Injected dentry 0x0x10000000000/sixmegs pointing to 0x0x10000000001 2024-04-25T10:30:38.916 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:38.903+0000 7f3dcc412740 10 datascan.scan_links: updating inotable 2024-04-25T10:30:38.916 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:38.903+0000 7f3dcc412740 1 -- 172.21.15.37:0/2011009937 --> [v2:172.21.15.203:6808/3688099014,v1:172.21.15.203:6809/3688099014] -- osd_op(unknown.0.0:75 6.13 6:c91d4a1d:::mds0_inotable:head [read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e44) -- 0x56229b083400 con 0x56229adf2d00 2024-04-25T10:30:38.916 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:38.904+0000 7f3dc98b4640 1 -- 172.21.15.37:0/2011009937 <== osd.7 v2:172.21.15.203:6808/3688099014 9 ==== osd_op_reply(75 mds0_inotable [read 0~34 out=34b] v0'0 uv2 ondisk = 0) ==== 157+0+34 (crc 0 0 0) 0x56229ad40000 con 0x56229adf2d00 2024-04-25T10:30:38.916 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:38.904+0000 7f3dcc412740 10 mds.0.inotable: skip_inos was [0x10000000000~0x10000000000] 2024-04-25T10:30:38.916 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:38.904+0000 7f3dcc412740 10 mds.0.inotable: skip_inos now [0x10000000002~0xfffffffffe] 2024-04-25T10:30:38.916 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:38.904+0000 7f3dcc412740 1 -- 172.21.15.37:0/2011009937 --> [v2:172.21.15.203:6808/3688099014,v1:172.21.15.203:6809/3688099014] -- osd_op(unknown.0.0:76 6.13 6:c91d4a1d:::mds0_inotable:head [writefull 0~34 in=34b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e44) -- 0x56229b083800 con 0x56229adf2d00 2024-04-25T10:30:38.916 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:38.905+0000 7f3dc98b4640 1 -- 172.21.15.37:0/2011009937 <== osd.7 v2:172.21.15.203:6808/3688099014 10 ==== osd_op_reply(76 mds0_inotable [writefull 0~34] v44'3 uv3 ondisk = 0) ==== 157+0+0 (crc 0 0 0) 0x56229ad40000 con 0x56229adf2d00 2024-04-25T10:30:38.917 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:38.905+0000 7f3dcc412740 10 datascan.scan_links: updating snaptable 2024-04-25T10:30:38.917 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:38.905+0000 7f3dcc412740 1 -- 172.21.15.37:0/2011009937 --> [v2:172.21.15.37:6800/1690341662,v1:172.21.15.37:6801/1690341662] -- osd_op(unknown.0.0:77 6.d 6:b50e409b:::mds_snaptable:head [read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e44) -- 0x56229b083c00 con 0x56229b056d80 2024-04-25T10:30:38.917 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:38.905+0000 7f3dc98b4640 1 -- 172.21.15.37:0/2011009937 <== osd.1 v2:172.21.15.37:6800/1690341662 7 ==== osd_op_reply(77 mds_snaptable [read 0~70 out=70b] v0'0 uv1 ondisk = 0) ==== 157+0+70 (crc 0 0 0) 0x56229ad40000 con 0x56229b056d80 2024-04-25T10:30:38.917 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:38.906+0000 7f3dcc412740 1 -- 172.21.15.37:0/2011009937 >> [v2:172.21.15.37:6816/2746074720,v1:172.21.15.37:6817/2746074720] conn(0x56229b057200 msgr2=0x56229b05b600 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:30:38.917 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:38.906+0000 7f3dcc412740 1 --2- 172.21.15.37:0/2011009937 >> [v2:172.21.15.37:6816/2746074720,v1:172.21.15.37:6817/2746074720] conn(0x56229b057200 0x56229b05b600 crc :-1 s=READY pgs=15 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:30:38.917 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:38.906+0000 7f3dcc412740 1 -- 172.21.15.37:0/2011009937 >> [v2:172.21.15.37:6800/1690341662,v1:172.21.15.37:6801/1690341662] conn(0x56229b056d80 msgr2=0x56229b05b080 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:30:38.917 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:38.906+0000 7f3dcc412740 1 --2- 172.21.15.37:0/2011009937 >> [v2:172.21.15.37:6800/1690341662,v1:172.21.15.37:6801/1690341662] conn(0x56229b056d80 0x56229b05b080 crc :-1 s=READY pgs=17 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:30:38.917 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:38.906+0000 7f3dcc412740 1 -- 172.21.15.37:0/2011009937 >> [v2:172.21.15.37:6808/2754669649,v1:172.21.15.37:6809/2754669649] conn(0x56229b057f80 msgr2=0x56229b05c680 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:30:38.917 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:38.906+0000 7f3dcc412740 1 --2- 172.21.15.37:0/2011009937 >> [v2:172.21.15.37:6808/2754669649,v1:172.21.15.37:6809/2754669649] conn(0x56229b057f80 0x56229b05c680 crc :-1 s=READY pgs=15 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:30:38.917 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:38.906+0000 7f3dcc412740 1 -- 172.21.15.37:0/2011009937 >> [v2:172.21.15.37:6824/1815290618,v1:172.21.15.37:6825/1815290618] conn(0x56229b056480 msgr2=0x56229b05a580 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:30:38.917 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:38.906+0000 7f3dcc412740 1 --2- 172.21.15.37:0/2011009937 >> [v2:172.21.15.37:6824/1815290618,v1:172.21.15.37:6825/1815290618] conn(0x56229b056480 0x56229b05a580 crc :-1 s=READY pgs=16 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:30:38.917 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:38.906+0000 7f3dcc412740 1 -- 172.21.15.37:0/2011009937 >> [v2:172.21.15.203:6800/4051532063,v1:172.21.15.203:6801/4051532063] conn(0x56229b056000 msgr2=0x56229b05a000 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:30:38.917 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:38.906+0000 7f3dcc412740 1 --2- 172.21.15.37:0/2011009937 >> [v2:172.21.15.203:6800/4051532063,v1:172.21.15.203:6801/4051532063] conn(0x56229b056000 0x56229b05a000 crc :-1 s=READY pgs=16 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:30:38.917 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:38.906+0000 7f3dcc412740 1 -- 172.21.15.37:0/2011009937 >> [v2:172.21.15.203:6824/1203822890,v1:172.21.15.203:6825/1203822890] conn(0x56229b057680 msgr2=0x56229b05bb80 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:30:38.918 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:38.906+0000 7f3dcc412740 1 --2- 172.21.15.37:0/2011009937 >> [v2:172.21.15.203:6824/1203822890,v1:172.21.15.203:6825/1203822890] conn(0x56229b057680 0x56229b05bb80 crc :-1 s=READY pgs=15 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:30:38.918 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:38.906+0000 7f3dc98b4640 1 -- 172.21.15.37:0/2011009937 reap_dead start 2024-04-25T10:30:38.918 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:38.906+0000 7f3dc98b4640 1 -- 172.21.15.37:0/2011009937 reap_dead start 2024-04-25T10:30:38.918 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:38.906+0000 7f3dcc412740 1 -- 172.21.15.37:0/2011009937 >> [v2:172.21.15.203:6810/2309615014,v1:172.21.15.203:6812/2309615014] conn(0x56229b056900 msgr2=0x56229b05ab00 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:30:38.918 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:38.906+0000 7f3dcc412740 1 --2- 172.21.15.37:0/2011009937 >> [v2:172.21.15.203:6810/2309615014,v1:172.21.15.203:6812/2309615014] conn(0x56229b056900 0x56229b05ab00 crc :-1 s=READY pgs=15 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:30:38.918 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:38.906+0000 7f3dcc412740 1 -- 172.21.15.37:0/2011009937 >> [v2:172.21.15.203:6808/3688099014,v1:172.21.15.203:6809/3688099014] conn(0x56229adf2d00 msgr2=0x56229add0680 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:30:38.918 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:38.906+0000 7f3dcc412740 1 --2- 172.21.15.37:0/2011009937 >> [v2:172.21.15.203:6808/3688099014,v1:172.21.15.203:6809/3688099014] conn(0x56229adf2d00 0x56229add0680 crc :-1 s=READY pgs=17 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:30:38.918 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:38.906+0000 7f3dcc412740 1 -- 172.21.15.37:0/2011009937 >> [v2:172.21.15.203:6832/2418205237,v1:172.21.15.203:6833/2418205237] conn(0x56229adf3a80 msgr2=0x56229add0100 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:30:38.918 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:38.906+0000 7f3dcc412740 1 --2- 172.21.15.37:0/2011009937 >> [v2:172.21.15.203:6832/2418205237,v1:172.21.15.203:6833/2418205237] conn(0x56229adf3a80 0x56229add0100 secure :-1 s=READY pgs=267 cs=0 l=1 rev1=1 crypto rx=0x56229ad5a690 tx=0x56229ad58b40 comp rx=0 tx=0).stop 2024-04-25T10:30:38.918 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:38.906+0000 7f3dcc412740 1 -- 172.21.15.37:0/2011009937 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x56229adf0d80 msgr2=0x56229adcf080 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:30:38.918 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:38.906+0000 7f3dcc412740 1 --2- 172.21.15.37:0/2011009937 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x56229adf0d80 0x56229adcf080 secure :-1 s=READY pgs=569 cs=0 l=1 rev1=1 crypto rx=0x56229bbb0600 tx=0x56229ad58a20 comp rx=0 tx=0).stop 2024-04-25T10:30:38.918 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:38.906+0000 7f3dcc412740 1 -- 172.21.15.37:0/2011009937 shutdown_connections 2024-04-25T10:30:38.918 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:38.906+0000 7f3dcc412740 1 --2- 172.21.15.37:0/2011009937 >> [v2:172.21.15.203:6810/2309615014,v1:172.21.15.203:6812/2309615014] conn(0x56229b056900 0x56229b05ab00 unknown :-1 s=CLOSED pgs=15 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:30:38.918 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:38.906+0000 7f3dcc412740 1 --2- 172.21.15.37:0/2011009937 >> [v2:172.21.15.203:6808/3688099014,v1:172.21.15.203:6809/3688099014] conn(0x56229adf2d00 0x56229add0680 unknown :-1 s=CLOSED pgs=17 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:30:38.918 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:38.906+0000 7f3dcc412740 1 --2- 172.21.15.37:0/2011009937 >> [v2:172.21.15.203:6832/2418205237,v1:172.21.15.203:6833/2418205237] conn(0x56229adf3a80 0x56229add0100 unknown :-1 s=CLOSED pgs=267 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:30:38.919 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:38.906+0000 7f3dcc412740 1 --2- 172.21.15.37:0/2011009937 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x56229adf0d80 0x56229adcf080 unknown :-1 s=CLOSED pgs=569 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:30:38.919 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:38.906+0000 7f3dcc412740 1 -- 172.21.15.37:0/2011009937 >> 172.21.15.37:0/2011009937 conn(0x56229adf1200 msgr2=0x56229ad3f800 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:30:38.919 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:38.906+0000 7f3dcc412740 1 -- 172.21.15.37:0/2011009937 shutdown_connections 2024-04-25T10:30:38.919 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:38.906+0000 7f3dcc412740 1 -- 172.21.15.37:0/2011009937 wait complete. 2024-04-25T10:30:38.919 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:38.906+0000 7f3dcc412740 1 -- 172.21.15.37:0/1863835220 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x56229adf1680 msgr2=0x56229adce580 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:30:38.919 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:38.906+0000 7f3dcc412740 1 --2- 172.21.15.37:0/1863835220 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x56229adf1680 0x56229adce580 secure :-1 s=READY pgs=567 cs=0 l=1 rev1=1 crypto rx=0x56229bb28060 tx=0x56229ad5c900 comp rx=0 tx=0).stop 2024-04-25T10:30:38.919 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:38.906+0000 7f3dcc412740 1 -- 172.21.15.37:0/1863835220 shutdown_connections 2024-04-25T10:30:38.919 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:38.906+0000 7f3dcc412740 1 --2- 172.21.15.37:0/1863835220 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x56229adf1f80 0x56229adceb00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:30:38.919 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:38.906+0000 7f3dcc412740 1 --2- 172.21.15.37:0/1863835220 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x56229adf1b00 0x56229adce000 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:30:38.919 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:38.906+0000 7f3dcc412740 1 --2- 172.21.15.37:0/1863835220 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x56229adf1680 0x56229adce580 unknown :-1 s=CLOSED pgs=567 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:30:38.919 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:38.906+0000 7f3dcc412740 1 -- 172.21.15.37:0/1863835220 >> 172.21.15.37:0/1863835220 conn(0x56229adf2400 msgr2=0x56229ad3f000 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:30:38.919 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:38.906+0000 7f3dcc412740 1 -- 172.21.15.37:0/1863835220 shutdown_connections 2024-04-25T10:30:38.919 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:38.906+0000 7f3dcc412740 1 -- 172.21.15.37:0/1863835220 wait complete. 2024-04-25T10:30:38.922 DEBUG:tasks.cephfs.filesystem:Ran ['cephfs-data-scan', '--debug-mds=20', '--debug-ms=1', '--debug-objecter=1', 'scan_links', '--filesystem', 'recovery_fs'] in time 0:00:00.109415, result: 2024-04-25T10:30:38.922 DEBUG:teuthology.orchestra.run.smithi037:> cephfs-journal-tool --debug-mds=20 --debug-ms=1 --debug-objecter=1 --rank recovery_fs:0 event recover_dentries list --alternate-pool recovery_fs_metadata 2024-04-25T10:30:38.977 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:38.975+0000 7fd75ba19280 1 -- start start 2024-04-25T10:30:38.978 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:38.975+0000 7fd75ba19280 1 Processor -- start 2024-04-25T10:30:38.981 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:38.976+0000 7fd75ba19280 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55cac2233680 0x55cac223a000 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:30:38.981 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:38.976+0000 7fd75ba19280 1 --2- >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x55cac2233b00 0x55cac223a580 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:30:38.981 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:38.976+0000 7fd75ba19280 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x55cac2234400 0x55cac223ab00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:30:38.981 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:38.976+0000 7fd75ba19280 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 -- 0x55cac2242600 con 0x55cac2233680 2024-04-25T10:30:38.981 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:38.976+0000 7fd75ba19280 1 -- --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_getmap magic: 0 -- 0x55cac2242480 con 0x55cac2234400 2024-04-25T10:30:38.981 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:38.976+0000 7fd75ba19280 1 -- --> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] -- mon_getmap magic: 0 -- 0x55cac2242300 con 0x55cac2233b00 2024-04-25T10:30:38.981 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:38.977+0000 7fd759553640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55cac2233680 0x55cac223a000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:30:38.981 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:38.977+0000 7fd759553640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55cac2233680 0x55cac223a000 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.37:3300/0 says I am v2:172.21.15.37:55720/0 (socket says 172.21.15.37:55720) 2024-04-25T10:30:38.981 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:38.977+0000 7fd759553640 1 -- 172.21.15.37:0/2555374190 learned_addr learned my addr 172.21.15.37:0/2555374190 (peer_addr_for_me v2:172.21.15.37:0/0) 2024-04-25T10:30:38.981 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:38.977+0000 7fd759d54640 1 --2- 172.21.15.37:0/2555374190 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x55cac2234400 0x55cac223ab00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:30:38.981 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:38.977+0000 7fd758d52640 1 --2- 172.21.15.37:0/2555374190 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x55cac2233b00 0x55cac223a580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:30:38.981 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:38.977+0000 7fd759553640 1 -- 172.21.15.37:0/2555374190 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x55cac2233b00 msgr2=0x55cac223a580 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:30:38.981 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:38.977+0000 7fd759553640 1 --2- 172.21.15.37:0/2555374190 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x55cac2233b00 0x55cac223a580 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:30:38.981 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:38.977+0000 7fd759553640 1 -- 172.21.15.37:0/2555374190 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x55cac2234400 msgr2=0x55cac223ab00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:30:38.982 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:38.977+0000 7fd759553640 1 --2- 172.21.15.37:0/2555374190 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x55cac2234400 0x55cac223ab00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:30:38.982 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:38.977+0000 7fd759553640 1 -- 172.21.15.37:0/2555374190 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55cabe188780 con 0x55cac2233680 2024-04-25T10:30:38.982 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:38.977+0000 7fd759553640 1 --2- 172.21.15.37:0/2555374190 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55cac2233680 0x55cac223a000 secure :-1 s=READY pgs=571 cs=0 l=1 rev1=1 crypto rx=0x55cac2fa5800 tx=0x55cac2282c60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=7f94e519124810fc server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:30:38.982 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:38.977+0000 7fd756d4e640 1 -- 172.21.15.37:0/2555374190 <== mon.0 v2:172.21.15.37:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x55cabe1789c0 con 0x55cac2233680 2024-04-25T10:30:38.982 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:38.977+0000 7fd756d4e640 1 -- 172.21.15.37:0/2555374190 <== mon.0 v2:172.21.15.37:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x55cabe17ee00 con 0x55cac2233680 2024-04-25T10:30:38.982 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:38.977+0000 7fd756d4e640 1 -- 172.21.15.37:0/2555374190 <== mon.0 v2:172.21.15.37:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x55cabe178680 con 0x55cac2233680 2024-04-25T10:30:38.982 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:38.977+0000 7fd75ba19280 1 -- 172.21.15.37:0/2555374190 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55cabe1894a0 con 0x55cac2233680 2024-04-25T10:30:38.982 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:38.977+0000 7fd75ba19280 1 -- 172.21.15.37:0/2555374190 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({fsmap=0}) -- 0x55cabe188000 con 0x55cac2233680 2024-04-25T10:30:38.982 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:38.977+0000 7fd75ba19280 4 waiting for MDS map... 2024-04-25T10:30:38.982 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:38.977+0000 7fd756d4e640 1 -- 172.21.15.37:0/2555374190 <== mon.0 v2:172.21.15.37:3300/0 4 ==== osd_map(44..44 src has 1..44) ==== 6248+0+0 (secure 0 0 0) 0x55cac305a600 con 0x55cac2233680 2024-04-25T10:30:38.982 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:38.978+0000 7fd756d4e640 1 -- 172.21.15.37:0/2555374190 <== mon.0 v2:172.21.15.37:3300/0 5 ==== fsmap(e 22) ==== 4135+0+0 (secure 0 0 0) 0x55cac21dd180 con 0x55cac2233680 2024-04-25T10:30:38.982 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:38.978+0000 7fd75ba19280 4 Got MDS map 22 2024-04-25T10:30:38.982 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:38.978+0000 7fd75ba19280 10 main: JournalTool::main 2024-04-25T10:30:38.982 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:38.978+0000 7fd75ba19280 4 main: JournalTool: connecting to RADOS... 2024-04-25T10:30:38.983 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:38.979+0000 7fd75ba19280 1 Processor -- start 2024-04-25T10:30:38.983 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:38.979+0000 7fd75ba19280 1 -- start start 2024-04-25T10:30:38.987 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:38.980+0000 7fd75ba19280 1 --2- >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x55cac2232d80 0x55cac223b080 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:30:38.987 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:38.980+0000 7fd75ba19280 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55cac2232900 0x55cac223b600 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:30:38.987 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:38.980+0000 7fd75ba19280 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x55cac2232480 0x55cac223bb80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:30:38.987 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:38.980+0000 7fd75ba19280 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 -- 0x55cac2242180 con 0x55cac2232900 2024-04-25T10:30:38.987 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:38.980+0000 7fd75ba19280 1 -- --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_getmap magic: 0 -- 0x55cac21e4480 con 0x55cac2232480 2024-04-25T10:30:38.987 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:38.980+0000 7fd75ba19280 1 -- --> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] -- mon_getmap magic: 0 -- 0x55cac21e4900 con 0x55cac2232d80 2024-04-25T10:30:38.987 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:38.980+0000 7fd759553640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55cac2232900 0x55cac223b600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:30:38.987 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:38.980+0000 7fd759553640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55cac2232900 0x55cac223b600 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.37:3300/0 says I am v2:172.21.15.37:55730/0 (socket says 172.21.15.37:55730) 2024-04-25T10:30:38.987 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:38.980+0000 7fd759553640 1 -- 172.21.15.37:0/3404507936 learned_addr learned my addr 172.21.15.37:0/3404507936 (peer_addr_for_me v2:172.21.15.37:0/0) 2024-04-25T10:30:38.987 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:38.980+0000 7fd759d54640 1 --2- 172.21.15.37:0/3404507936 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x55cac2232d80 0x55cac223b080 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:30:38.987 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:38.980+0000 7fd758d52640 1 --2- 172.21.15.37:0/3404507936 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x55cac2232480 0x55cac223bb80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:30:38.987 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:38.980+0000 7fd759553640 1 -- 172.21.15.37:0/3404507936 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x55cac2232d80 msgr2=0x55cac223b080 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:30:38.988 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:38.980+0000 7fd759553640 1 --2- 172.21.15.37:0/3404507936 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x55cac2232d80 0x55cac223b080 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:30:38.988 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:38.980+0000 7fd759553640 1 -- 172.21.15.37:0/3404507936 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x55cac2232480 msgr2=0x55cac223bb80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:30:38.988 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:38.980+0000 7fd759553640 1 --2- 172.21.15.37:0/3404507936 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x55cac2232480 0x55cac223bb80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:30:38.988 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:38.980+0000 7fd759553640 1 -- 172.21.15.37:0/3404507936 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55cabe1883c0 con 0x55cac2232900 2024-04-25T10:30:38.988 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:38.980+0000 7fd759553640 1 --2- 172.21.15.37:0/3404507936 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55cac2232900 0x55cac223b600 secure :-1 s=READY pgs=572 cs=0 l=1 rev1=1 crypto rx=0x55cac3060510 tx=0x55cac2249080 comp rx=0 tx=0).ready entity=mon.0 client_cookie=2043d09c075c4f7 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:30:38.988 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:38.980+0000 7fd754d4a640 1 -- 172.21.15.37:0/3404507936 <== mon.0 v2:172.21.15.37:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x55cabe178b60 con 0x55cac2232900 2024-04-25T10:30:38.988 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:38.980+0000 7fd754d4a640 1 -- 172.21.15.37:0/3404507936 <== mon.0 v2:172.21.15.37:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x55cabe17ea80 con 0x55cac2232900 2024-04-25T10:30:38.988 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:38.980+0000 7fd754d4a640 1 -- 172.21.15.37:0/3404507936 <== mon.0 v2:172.21.15.37:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x55cabe178820 con 0x55cac2232900 2024-04-25T10:30:38.988 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:38.981+0000 7fd75ba19280 1 -- 172.21.15.37:0/3404507936 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55cac2232900 msgr2=0x55cac223b600 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:30:38.988 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:38.981+0000 7fd75ba19280 1 --2- 172.21.15.37:0/3404507936 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55cac2232900 0x55cac223b600 secure :-1 s=READY pgs=572 cs=0 l=1 rev1=1 crypto rx=0x55cac3060510 tx=0x55cac2249080 comp rx=0 tx=0).stop 2024-04-25T10:30:38.988 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:38.981+0000 7fd75ba19280 1 -- 172.21.15.37:0/3404507936 shutdown_connections 2024-04-25T10:30:38.988 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:38.981+0000 7fd75ba19280 1 --2- 172.21.15.37:0/3404507936 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x55cac2232480 0x55cac223bb80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:30:38.988 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:38.981+0000 7fd75ba19280 1 --2- 172.21.15.37:0/3404507936 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55cac2232900 0x55cac223b600 unknown :-1 s=CLOSED pgs=572 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:30:38.988 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:38.981+0000 7fd75ba19280 1 --2- 172.21.15.37:0/3404507936 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x55cac2232d80 0x55cac223b080 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:30:38.989 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:38.981+0000 7fd75ba19280 1 -- 172.21.15.37:0/3404507936 >> 172.21.15.37:0/3404507936 conn(0x55cac2233200 msgr2=0x55cac2268000 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:30:38.989 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:38.981+0000 7fd75ba19280 1 -- 172.21.15.37:0/3404507936 shutdown_connections 2024-04-25T10:30:38.989 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:38.981+0000 7fd75ba19280 1 -- 172.21.15.37:0/3404507936 wait complete. 2024-04-25T10:30:38.989 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:38.982+0000 7fd75ba19280 1 Processor -- start 2024-04-25T10:30:38.989 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:38.982+0000 7fd75ba19280 1 -- start start 2024-04-25T10:30:38.989 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:38.983+0000 7fd75ba19280 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55cac2232480 0x55cac223bb80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:30:38.989 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:38.983+0000 7fd75ba19280 1 --2- >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x55cac2232900 0x55cac223b600 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:30:38.989 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:38.983+0000 7fd75ba19280 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x55cac2232d80 0x55cac223b080 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:30:38.989 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:38.983+0000 7fd75ba19280 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 -- 0x55cac2242c00 con 0x55cac2232480 2024-04-25T10:30:38.989 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:38.983+0000 7fd75ba19280 1 -- --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_getmap magic: 0 -- 0x55cac2242a80 con 0x55cac2232d80 2024-04-25T10:30:38.989 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:38.983+0000 7fd75ba19280 1 -- --> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] -- mon_getmap magic: 0 -- 0x55cac2243380 con 0x55cac2232900 2024-04-25T10:30:38.989 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:38.983+0000 7fd759d54640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55cac2232480 0x55cac223bb80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:30:38.989 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:38.983+0000 7fd759d54640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55cac2232480 0x55cac223bb80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.37:3300/0 says I am v2:172.21.15.37:55734/0 (socket says 172.21.15.37:55734) 2024-04-25T10:30:38.989 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:38.983+0000 7fd759d54640 1 -- 172.21.15.37:0/2879290984 learned_addr learned my addr 172.21.15.37:0/2879290984 (peer_addr_for_me v2:172.21.15.37:0/0) 2024-04-25T10:30:38.989 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:38.983+0000 7fd758d52640 1 --2- 172.21.15.37:0/2879290984 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x55cac2232d80 0x55cac223b080 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:30:38.990 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:38.983+0000 7fd759553640 1 --2- 172.21.15.37:0/2879290984 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x55cac2232900 0x55cac223b600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:30:38.990 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:38.983+0000 7fd759d54640 1 -- 172.21.15.37:0/2879290984 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x55cac2232900 msgr2=0x55cac223b600 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:30:38.990 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:38.983+0000 7fd759d54640 1 --2- 172.21.15.37:0/2879290984 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x55cac2232900 0x55cac223b600 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:30:38.990 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:38.983+0000 7fd759d54640 1 -- 172.21.15.37:0/2879290984 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x55cac2232d80 msgr2=0x55cac223b080 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:30:38.990 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:38.983+0000 7fd759d54640 1 --2- 172.21.15.37:0/2879290984 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x55cac2232d80 0x55cac223b080 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:30:38.990 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:38.983+0000 7fd759d54640 1 -- 172.21.15.37:0/2879290984 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55cabe188960 con 0x55cac2232480 2024-04-25T10:30:38.990 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:38.984+0000 7fd759d54640 1 --2- 172.21.15.37:0/2879290984 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55cac2232480 0x55cac223bb80 secure :-1 s=READY pgs=573 cs=0 l=1 rev1=1 crypto rx=0x55cac3061a70 tx=0x55cac3064c00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:30:38.990 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:38.984+0000 7fd753547640 1 -- 172.21.15.37:0/2879290984 <== mon.0 v2:172.21.15.37:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x55cabe178b60 con 0x55cac2232480 2024-04-25T10:30:38.990 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:38.984+0000 7fd753547640 1 -- 172.21.15.37:0/2879290984 <== mon.0 v2:172.21.15.37:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x55cabe17ec40 con 0x55cac2232480 2024-04-25T10:30:38.990 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:38.984+0000 7fd753547640 1 -- 172.21.15.37:0/2879290984 <== mon.0 v2:172.21.15.37:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x55cabe178820 con 0x55cac2232480 2024-04-25T10:30:38.990 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:38.984+0000 7fd75ba19280 1 -- 172.21.15.37:0/2879290984 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x55cabe189680 con 0x55cac2232480 2024-04-25T10:30:38.990 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:38.984+0000 7fd75ba19280 1 -- 172.21.15.37:0/2879290984 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55cabe189860 con 0x55cac2232480 2024-04-25T10:30:38.990 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:38.984+0000 7fd75ba19280 4 main: JournalTool: resolving pool 6 2024-04-25T10:30:38.994 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:38.985+0000 7fd753547640 1 -- 172.21.15.37:0/2879290984 <== mon.0 v2:172.21.15.37:3300/0 4 ==== mgrmap(e 5) ==== 96782+0+0 (secure 0 0 0) 0x55cac30a0900 con 0x55cac2232480 2024-04-25T10:30:38.994 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:38.986+0000 7fd753547640 1 --2- 172.21.15.37:0/2879290984 >> [v2:172.21.15.203:6832/2418205237,v1:172.21.15.203:6833/2418205237] conn(0x55cac2235180 0x55cac223c100 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:30:38.994 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:38.986+0000 7fd753547640 1 -- 172.21.15.37:0/2879290984 <== mon.0 v2:172.21.15.37:3300/0 5 ==== osd_map(44..44 src has 1..44) ==== 6248+0+0 (secure 0 0 0) 0x55cac30af000 con 0x55cac2232480 2024-04-25T10:30:38.994 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:38.986+0000 7fd75ba19280 4 main: JournalTool: creating IoCtx.. 2024-04-25T10:30:38.994 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:38.986+0000 7fd75ba19280 4 main: Executing for rank 0 2024-04-25T10:30:38.994 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:38.986+0000 7fd75ba19280 1 main_event: Using alternate pool recovery_fs_metadata 2024-04-25T10:30:38.994 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:38.986+0000 7fd75ba19280 1 --2- 172.21.15.37:0/2879290984 >> [v2:172.21.15.203:6824/1203822890,v1:172.21.15.203:6825/1203822890] conn(0x55cac2232000 0x55cac223c680 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:30:38.994 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:38.986+0000 7fd759553640 1 --2- 172.21.15.37:0/2879290984 >> [v2:172.21.15.203:6832/2418205237,v1:172.21.15.203:6833/2418205237] conn(0x55cac2235180 0x55cac223c100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:30:38.995 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:38.986+0000 7fd75ba19280 1 -- 172.21.15.37:0/2879290984 --> [v2:172.21.15.203:6824/1203822890,v1:172.21.15.203:6825/1203822890] -- osd_op(unknown.0.0:1 6.f 6:f1f69726:::400.00000000:head [read 0~2147483647] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e44) -- 0x55cac30c0c00 con 0x55cac2232000 2024-04-25T10:30:38.995 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:38.986+0000 7fd758d52640 1 --2- 172.21.15.37:0/2879290984 >> [v2:172.21.15.203:6824/1203822890,v1:172.21.15.203:6825/1203822890] conn(0x55cac2232000 0x55cac223c680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:30:38.995 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:38.986+0000 7fd759553640 1 --2- 172.21.15.37:0/2879290984 >> [v2:172.21.15.203:6832/2418205237,v1:172.21.15.203:6833/2418205237] conn(0x55cac2235180 0x55cac223c100 secure :-1 s=READY pgs=268 cs=0 l=1 rev1=1 crypto rx=0x55cac3060510 tx=0x55cac2282b40 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:30:38.995 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:38.987+0000 7fd758d52640 1 --2- 172.21.15.37:0/2879290984 >> [v2:172.21.15.203:6824/1203822890,v1:172.21.15.203:6825/1203822890] conn(0x55cac2232000 0x55cac223c680 crc :-1 s=READY pgs=16 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.5 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:30:38.995 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:38.987+0000 7fd758d52640 1 -- 172.21.15.37:0/2879290984 <== osd.5 v2:172.21.15.203:6824/1203822890 1 ==== osd_op_reply(1 400.00000000 [read 0~2147483647] v0'0 uv0 ondisk = -2 ((2) No such file or directory)) ==== 156+0+0 (crc 0 0 0) 0x55cac22a8000 con 0x55cac2232000 2024-04-25T10:30:38.995 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:38.987+0000 7fd75ba19280 -1 Pointer 400.00000000 is absent 2024-04-25T10:30:38.995 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:38.987+0000 7fd75ba19280 4 main_event: consumed 0 inodes 2024-04-25T10:30:38.995 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:38.987+0000 7fd75ba19280 1 --2- 172.21.15.37:0/2879290984 >> [v2:172.21.15.37:6816/2746074720,v1:172.21.15.37:6817/2746074720] conn(0x55cac2234880 0x55cac223cc00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:30:38.995 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:38.987+0000 7fd75ba19280 1 -- 172.21.15.37:0/2879290984 --> [v2:172.21.15.37:6816/2746074720,v1:172.21.15.37:6817/2746074720] -- osd_op(unknown.0.0:2 6.3 6:c0ce80d4:::4.00000000:head [omap-rm-keys in=4b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e44) -- 0x55cac30c1000 con 0x55cac2234880 2024-04-25T10:30:38.995 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:38.987+0000 7fd759d54640 1 --2- 172.21.15.37:0/2879290984 >> [v2:172.21.15.37:6816/2746074720,v1:172.21.15.37:6817/2746074720] conn(0x55cac2234880 0x55cac223cc00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:30:38.995 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:38.987+0000 7fd759d54640 1 --2- 172.21.15.37:0/2879290984 >> [v2:172.21.15.37:6816/2746074720,v1:172.21.15.37:6817/2746074720] conn(0x55cac2234880 0x55cac223cc00 crc :-1 s=READY pgs=16 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:30:38.995 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:38.988+0000 7fd759d54640 1 -- 172.21.15.37:0/2879290984 <== osd.0 v2:172.21.15.37:6816/2746074720 1 ==== osd_op_reply(2 4.00000000 [omap-rm-keys] v44'1 uv0 ondisk = -2 ((2) No such file or directory)) ==== 154+0+0 (crc 0 0 0) 0x55cac22a8500 con 0x55cac2234880 2024-04-25T10:30:38.995 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:38.988+0000 7fd75ba19280 4 0 events 2024-04-25T10:30:38.996 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:38.989+0000 7fd75ba19280 1 -- 172.21.15.37:0/2879290984 >> [v2:172.21.15.37:6816/2746074720,v1:172.21.15.37:6817/2746074720] conn(0x55cac2234880 msgr2=0x55cac223cc00 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:30:38.996 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:38.989+0000 7fd75ba19280 1 --2- 172.21.15.37:0/2879290984 >> [v2:172.21.15.37:6816/2746074720,v1:172.21.15.37:6817/2746074720] conn(0x55cac2234880 0x55cac223cc00 crc :-1 s=READY pgs=16 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:30:38.996 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:38.989+0000 7fd75ba19280 1 -- 172.21.15.37:0/2879290984 >> [v2:172.21.15.203:6824/1203822890,v1:172.21.15.203:6825/1203822890] conn(0x55cac2232000 msgr2=0x55cac223c680 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:30:38.996 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:38.989+0000 7fd75ba19280 1 --2- 172.21.15.37:0/2879290984 >> [v2:172.21.15.203:6824/1203822890,v1:172.21.15.203:6825/1203822890] conn(0x55cac2232000 0x55cac223c680 crc :-1 s=READY pgs=16 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:30:38.996 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:38.989+0000 7fd75ba19280 1 -- 172.21.15.37:0/2879290984 >> [v2:172.21.15.203:6832/2418205237,v1:172.21.15.203:6833/2418205237] conn(0x55cac2235180 msgr2=0x55cac223c100 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:30:38.996 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:38.989+0000 7fd75ba19280 1 --2- 172.21.15.37:0/2879290984 >> [v2:172.21.15.203:6832/2418205237,v1:172.21.15.203:6833/2418205237] conn(0x55cac2235180 0x55cac223c100 secure :-1 s=READY pgs=268 cs=0 l=1 rev1=1 crypto rx=0x55cac3060510 tx=0x55cac2282b40 comp rx=0 tx=0).stop 2024-04-25T10:30:38.996 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:38.989+0000 7fd75ba19280 1 -- 172.21.15.37:0/2879290984 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55cac2232480 msgr2=0x55cac223bb80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:30:38.996 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:38.989+0000 7fd75ba19280 1 --2- 172.21.15.37:0/2879290984 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55cac2232480 0x55cac223bb80 secure :-1 s=READY pgs=573 cs=0 l=1 rev1=1 crypto rx=0x55cac3061a70 tx=0x55cac3064c00 comp rx=0 tx=0).stop 2024-04-25T10:30:38.996 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:38.989+0000 7fd758d52640 1 -- 172.21.15.37:0/2879290984 reap_dead start 2024-04-25T10:30:38.996 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:38.989+0000 7fd758d52640 1 -- 172.21.15.37:0/2879290984 reap_dead start 2024-04-25T10:30:38.996 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:38.989+0000 7fd75ba19280 1 -- 172.21.15.37:0/2879290984 shutdown_connections 2024-04-25T10:30:38.996 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:38.989+0000 7fd75ba19280 1 -- 172.21.15.37:0/2879290984 >> 172.21.15.37:0/2879290984 conn(0x55cac2233200 msgr2=0x55cac2268000 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:30:38.996 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:38.989+0000 7fd75ba19280 1 -- 172.21.15.37:0/2879290984 shutdown_connections 2024-04-25T10:30:38.996 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:38.989+0000 7fd75ba19280 1 -- 172.21.15.37:0/2879290984 wait complete. 2024-04-25T10:30:38.996 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:38.989+0000 7fd75ba19280 1 -- 172.21.15.37:0/2555374190 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55cac2233680 msgr2=0x55cac223a000 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:30:38.997 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:38.989+0000 7fd75ba19280 1 --2- 172.21.15.37:0/2555374190 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55cac2233680 0x55cac223a000 secure :-1 s=READY pgs=571 cs=0 l=1 rev1=1 crypto rx=0x55cac2fa5800 tx=0x55cac2282c60 comp rx=0 tx=0).stop 2024-04-25T10:30:38.997 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:38.989+0000 7fd75ba19280 1 -- 172.21.15.37:0/2555374190 shutdown_connections 2024-04-25T10:30:38.997 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:38.989+0000 7fd75ba19280 1 --2- 172.21.15.37:0/2555374190 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x55cac2234400 0x55cac223ab00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:30:38.997 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:38.989+0000 7fd75ba19280 1 --2- 172.21.15.37:0/2555374190 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x55cac2233b00 0x55cac223a580 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:30:38.997 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:38.989+0000 7fd75ba19280 1 --2- 172.21.15.37:0/2555374190 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55cac2233680 0x55cac223a000 unknown :-1 s=CLOSED pgs=571 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:30:38.997 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:38.989+0000 7fd75ba19280 1 -- 172.21.15.37:0/2555374190 >> 172.21.15.37:0/2555374190 conn(0x55cac2233f80 msgr2=0x55cac2269000 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:30:38.997 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:38.989+0000 7fd75ba19280 1 -- 172.21.15.37:0/2555374190 shutdown_connections 2024-04-25T10:30:38.997 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:38.989+0000 7fd75ba19280 1 -- 172.21.15.37:0/2555374190 wait complete. 2024-04-25T10:30:38.997 DEBUG:tasks.cephfs.filesystem:Ran ['cephfs-journal-tool', '--debug-mds=20', '--debug-ms=1', '--debug-objecter=1', '--rank', 'recovery_fs:0', 'event', 'recover_dentries', 'list', '--alternate-pool', 'recovery_fs_metadata'] in time 0:00:00.074874, result: 2024-04-25T10:30:38.997 DEBUG:teuthology.orchestra.run.smithi037:> cephfs-journal-tool --debug-mds=20 --debug-ms=1 --debug-objecter=1 --rank recovery_fs:0 journal reset --force 2024-04-25T10:30:39.051 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:39.049+0000 7f4bf7723280 1 -- start start 2024-04-25T10:30:39.051 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:39.049+0000 7f4bf7723280 1 Processor -- start 2024-04-25T10:30:39.054 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:39.050+0000 7f4bf7723280 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x5599ffa93680 0x559a007c4000 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:30:39.054 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:39.050+0000 7f4bf7723280 1 --2- >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x5599ffa93b00 0x559a007c4580 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:30:39.054 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:39.050+0000 7f4bf7723280 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x5599ffa94400 0x559a007c4b00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:30:39.054 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:39.050+0000 7f4bf7723280 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 -- 0x5599ffaa2600 con 0x5599ffa94400 2024-04-25T10:30:39.054 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:39.050+0000 7f4bf7723280 1 -- --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_getmap magic: 0 -- 0x5599ffaa2480 con 0x5599ffa93680 2024-04-25T10:30:39.054 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:39.050+0000 7f4bf7723280 1 -- --> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] -- mon_getmap magic: 0 -- 0x5599ffaa2300 con 0x5599ffa93b00 2024-04-25T10:30:39.054 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:39.050+0000 7f4bf5e1e640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x5599ffa94400 0x559a007c4b00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:30:39.054 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:39.050+0000 7f4bf5e1e640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x5599ffa94400 0x559a007c4b00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.37:3300/0 says I am v2:172.21.15.37:55750/0 (socket says 172.21.15.37:55750) 2024-04-25T10:30:39.055 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:39.050+0000 7f4bf5e1e640 1 -- 172.21.15.37:0/3658284047 learned_addr learned my addr 172.21.15.37:0/3658284047 (peer_addr_for_me v2:172.21.15.37:0/0) 2024-04-25T10:30:39.055 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:39.050+0000 7f4bf4e1c640 1 --2- 172.21.15.37:0/3658284047 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x5599ffa93b00 0x559a007c4580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:30:39.055 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:39.050+0000 7f4bf561d640 1 --2- 172.21.15.37:0/3658284047 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x5599ffa93680 0x559a007c4000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:30:39.055 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:39.050+0000 7f4bf5e1e640 1 -- 172.21.15.37:0/3658284047 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x5599ffa93b00 msgr2=0x559a007c4580 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:30:39.055 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:39.050+0000 7f4bf5e1e640 1 --2- 172.21.15.37:0/3658284047 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x5599ffa93b00 0x559a007c4580 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:30:39.055 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:39.050+0000 7f4bf5e1e640 1 -- 172.21.15.37:0/3658284047 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x5599ffa93680 msgr2=0x559a007c4000 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:30:39.055 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:39.050+0000 7f4bf5e1e640 1 --2- 172.21.15.37:0/3658284047 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x5599ffa93680 0x559a007c4000 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:30:39.055 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:39.050+0000 7f4bf5e1e640 1 -- 172.21.15.37:0/3658284047 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5599fb9e8780 con 0x5599ffa94400 2024-04-25T10:30:39.055 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:39.050+0000 7f4bf5e1e640 1 --2- 172.21.15.37:0/3658284047 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x5599ffa94400 0x559a007c4b00 secure :-1 s=READY pgs=575 cs=0 l=1 rev1=1 crypto rx=0x559a008150b0 tx=0x5599ffae3f80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=303f7bedeec55196 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:30:39.055 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:39.051+0000 7f4bf2e18640 1 -- 172.21.15.37:0/3658284047 <== mon.0 v2:172.21.15.37:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x5599fb9d89c0 con 0x5599ffa94400 2024-04-25T10:30:39.055 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:39.051+0000 7f4bf2e18640 1 -- 172.21.15.37:0/3658284047 <== mon.0 v2:172.21.15.37:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x5599fb9dee00 con 0x5599ffa94400 2024-04-25T10:30:39.055 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:39.051+0000 7f4bf2e18640 1 -- 172.21.15.37:0/3658284047 <== mon.0 v2:172.21.15.37:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x5599fb9d8680 con 0x5599ffa94400 2024-04-25T10:30:39.055 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:39.051+0000 7f4bf7723280 1 -- 172.21.15.37:0/3658284047 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5599fb9e9680 con 0x5599ffa94400 2024-04-25T10:30:39.056 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:39.051+0000 7f4bf7723280 1 -- 172.21.15.37:0/3658284047 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({fsmap=0}) -- 0x5599fb9e94a0 con 0x5599ffa94400 2024-04-25T10:30:39.056 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:39.051+0000 7f4bf7723280 4 waiting for MDS map... 2024-04-25T10:30:39.056 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:39.051+0000 7f4bf2e18640 1 -- 172.21.15.37:0/3658284047 <== mon.0 v2:172.21.15.37:3300/0 4 ==== osd_map(44..44 src has 1..44) ==== 6248+0+0 (secure 0 0 0) 0x559a008be600 con 0x5599ffa94400 2024-04-25T10:30:39.056 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:39.051+0000 7f4bf2e18640 1 -- 172.21.15.37:0/3658284047 <== mon.0 v2:172.21.15.37:3300/0 5 ==== fsmap(e 22) ==== 4135+0+0 (secure 0 0 0) 0x5599ffa3d180 con 0x5599ffa94400 2024-04-25T10:30:39.056 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:39.051+0000 7f4bf7723280 4 Got MDS map 22 2024-04-25T10:30:39.056 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:39.051+0000 7f4bf7723280 10 main: JournalTool::main 2024-04-25T10:30:39.056 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:39.051+0000 7f4bf7723280 4 main: JournalTool: connecting to RADOS... 2024-04-25T10:30:39.056 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:39.052+0000 7f4bf7723280 1 Processor -- start 2024-04-25T10:30:39.056 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:39.052+0000 7f4bf7723280 1 -- start start 2024-04-25T10:30:39.062 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:39.053+0000 7f4bf7723280 1 --2- >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x5599ffa92d80 0x559a007c5080 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:30:39.062 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:39.053+0000 7f4bf7723280 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x5599ffa92900 0x559a007c5600 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:30:39.062 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:39.053+0000 7f4bf7723280 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x5599ffa92480 0x559a007c5b80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:30:39.063 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:39.053+0000 7f4bf7723280 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 -- 0x5599ffaa2180 con 0x5599ffa92480 2024-04-25T10:30:39.063 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:39.053+0000 7f4bf7723280 1 -- --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_getmap magic: 0 -- 0x5599ffa44480 con 0x5599ffa92900 2024-04-25T10:30:39.063 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:39.053+0000 7f4bf7723280 1 -- --> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] -- mon_getmap magic: 0 -- 0x5599ffa44900 con 0x5599ffa92d80 2024-04-25T10:30:39.063 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:39.053+0000 7f4bf4e1c640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x5599ffa92480 0x559a007c5b80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:30:39.063 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:39.053+0000 7f4bf4e1c640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x5599ffa92480 0x559a007c5b80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.37:3300/0 says I am v2:172.21.15.37:55762/0 (socket says 172.21.15.37:55762) 2024-04-25T10:30:39.063 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:39.053+0000 7f4bf4e1c640 1 -- 172.21.15.37:0/3282988426 learned_addr learned my addr 172.21.15.37:0/3282988426 (peer_addr_for_me v2:172.21.15.37:0/0) 2024-04-25T10:30:39.063 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:39.053+0000 7f4bf4e1c640 1 --2- 172.21.15.37:0/3282988426 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x5599ffa92d80 0x559a007c5080 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:30:39.063 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:39.053+0000 7f4bf561d640 1 --2- 172.21.15.37:0/3282988426 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x5599ffa92900 0x559a007c5600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:30:39.063 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:39.054+0000 7f4bf4e1c640 1 -- 172.21.15.37:0/3282988426 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x5599ffa92d80 msgr2=0x559a007c5080 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:30:39.063 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:39.054+0000 7f4bf4e1c640 1 --2- 172.21.15.37:0/3282988426 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x5599ffa92d80 0x559a007c5080 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:30:39.063 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:39.054+0000 7f4bf4e1c640 1 -- 172.21.15.37:0/3282988426 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x5599ffa92900 msgr2=0x559a007c5600 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:30:39.063 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:39.054+0000 7f4bf4e1c640 1 --2- 172.21.15.37:0/3282988426 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x5599ffa92900 0x559a007c5600 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:30:39.064 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:39.054+0000 7f4bf4e1c640 1 -- 172.21.15.37:0/3282988426 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5599fb9e8960 con 0x5599ffa92480 2024-04-25T10:30:39.064 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:39.054+0000 7f4bf4e1c640 1 --2- 172.21.15.37:0/3282988426 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x5599ffa92480 0x559a007c5b80 secure :-1 s=READY pgs=576 cs=0 l=1 rev1=1 crypto rx=0x559a008d4f90 tx=0x559a008d6cc0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=38729417f2e7d423 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:30:39.064 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:39.054+0000 7f4bf0e14640 1 -- 172.21.15.37:0/3282988426 <== mon.0 v2:172.21.15.37:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x5599fb9d89c0 con 0x5599ffa92480 2024-04-25T10:30:39.064 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:39.054+0000 7f4bf0e14640 1 -- 172.21.15.37:0/3282988426 <== mon.0 v2:172.21.15.37:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x5599fb9de1c0 con 0x5599ffa92480 2024-04-25T10:30:39.064 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:39.054+0000 7f4bf0e14640 1 -- 172.21.15.37:0/3282988426 <== mon.0 v2:172.21.15.37:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x5599fb9d8680 con 0x5599ffa92480 2024-04-25T10:30:39.064 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:39.054+0000 7f4bf7723280 1 -- 172.21.15.37:0/3282988426 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x5599ffa92480 msgr2=0x559a007c5b80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:30:39.064 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:39.054+0000 7f4bf7723280 1 --2- 172.21.15.37:0/3282988426 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x5599ffa92480 0x559a007c5b80 secure :-1 s=READY pgs=576 cs=0 l=1 rev1=1 crypto rx=0x559a008d4f90 tx=0x559a008d6cc0 comp rx=0 tx=0).stop 2024-04-25T10:30:39.064 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:39.055+0000 7f4bf7723280 1 -- 172.21.15.37:0/3282988426 shutdown_connections 2024-04-25T10:30:39.064 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:39.055+0000 7f4bf7723280 1 --2- 172.21.15.37:0/3282988426 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x5599ffa92480 0x559a007c5b80 unknown :-1 s=CLOSED pgs=576 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:30:39.064 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:39.055+0000 7f4bf7723280 1 --2- 172.21.15.37:0/3282988426 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x5599ffa92900 0x559a007c5600 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:30:39.064 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:39.055+0000 7f4bf7723280 1 --2- 172.21.15.37:0/3282988426 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x5599ffa92d80 0x559a007c5080 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:30:39.065 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:39.055+0000 7f4bf7723280 1 -- 172.21.15.37:0/3282988426 >> 172.21.15.37:0/3282988426 conn(0x5599ffa93200 msgr2=0x5599ffac8000 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:30:39.065 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:39.055+0000 7f4bf7723280 1 -- 172.21.15.37:0/3282988426 shutdown_connections 2024-04-25T10:30:39.065 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:39.055+0000 7f4bf7723280 1 -- 172.21.15.37:0/3282988426 wait complete. 2024-04-25T10:30:39.065 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:39.055+0000 7f4bf7723280 1 Processor -- start 2024-04-25T10:30:39.065 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:39.055+0000 7f4bf7723280 1 -- start start 2024-04-25T10:30:39.065 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:39.057+0000 7f4bf7723280 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x5599ffa92480 0x559a007c5b80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:30:39.065 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:39.057+0000 7f4bf7723280 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x5599ffa92900 0x559a007c5600 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:30:39.065 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:39.057+0000 7f4bf7723280 1 --2- >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x5599ffa92d80 0x559a007c5080 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:30:39.065 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:39.057+0000 7f4bf7723280 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 -- 0x5599ffaa3380 con 0x5599ffa92480 2024-04-25T10:30:39.065 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:39.057+0000 7f4bf7723280 1 -- --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_getmap magic: 0 -- 0x5599ffaa2180 con 0x5599ffa92900 2024-04-25T10:30:39.065 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:39.057+0000 7f4bf7723280 1 -- --> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] -- mon_getmap magic: 0 -- 0x5599ffaa2c00 con 0x5599ffa92d80 2024-04-25T10:30:39.066 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:39.057+0000 7f4bf4e1c640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x5599ffa92480 0x559a007c5b80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:30:39.066 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:39.057+0000 7f4bf4e1c640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x5599ffa92480 0x559a007c5b80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.37:3300/0 says I am v2:172.21.15.37:55770/0 (socket says 172.21.15.37:55770) 2024-04-25T10:30:39.066 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:39.057+0000 7f4bf4e1c640 1 -- 172.21.15.37:0/1681348478 learned_addr learned my addr 172.21.15.37:0/1681348478 (peer_addr_for_me v2:172.21.15.37:0/0) 2024-04-25T10:30:39.066 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:39.057+0000 7f4bf561d640 1 --2- 172.21.15.37:0/1681348478 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x5599ffa92900 0x559a007c5600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:30:39.066 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:39.057+0000 7f4bf4e1c640 1 --2- 172.21.15.37:0/1681348478 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x5599ffa92d80 0x559a007c5080 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:30:39.066 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:39.057+0000 7f4bf4e1c640 1 -- 172.21.15.37:0/1681348478 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x5599ffa92d80 msgr2=0x559a007c5080 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:30:39.066 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:39.057+0000 7f4bf4e1c640 1 --2- 172.21.15.37:0/1681348478 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x5599ffa92d80 0x559a007c5080 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:30:39.066 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:39.057+0000 7f4bf4e1c640 1 -- 172.21.15.37:0/1681348478 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x5599ffa92900 msgr2=0x559a007c5600 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:30:39.066 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:39.057+0000 7f4bf4e1c640 1 --2- 172.21.15.37:0/1681348478 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x5599ffa92900 0x559a007c5600 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:30:39.066 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:39.057+0000 7f4bf4e1c640 1 -- 172.21.15.37:0/1681348478 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5599fb9e8780 con 0x5599ffa92480 2024-04-25T10:30:39.066 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:39.057+0000 7f4bf561d640 1 --2- 172.21.15.37:0/1681348478 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x5599ffa92900 0x559a007c5600 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T10:30:39.067 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:39.057+0000 7f4bf4e1c640 1 --2- 172.21.15.37:0/1681348478 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x5599ffa92480 0x559a007c5b80 secure :-1 s=READY pgs=577 cs=0 l=1 rev1=1 crypto rx=0x559a008d5c50 tx=0x559a008d6c00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:30:39.067 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:39.057+0000 7f4bef611640 1 -- 172.21.15.37:0/1681348478 <== mon.0 v2:172.21.15.37:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x5599fb9d84e0 con 0x5599ffa92480 2024-04-25T10:30:39.067 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:39.057+0000 7f4bef611640 1 -- 172.21.15.37:0/1681348478 <== mon.0 v2:172.21.15.37:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x5599fb9de8c0 con 0x5599ffa92480 2024-04-25T10:30:39.067 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:39.057+0000 7f4bf7723280 1 -- 172.21.15.37:0/1681348478 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x5599fb9e83c0 con 0x5599ffa92480 2024-04-25T10:30:39.067 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:39.057+0000 7f4bef611640 1 -- 172.21.15.37:0/1681348478 <== mon.0 v2:172.21.15.37:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x5599fb9d89c0 con 0x5599ffa92480 2024-04-25T10:30:39.067 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:39.057+0000 7f4bf7723280 1 -- 172.21.15.37:0/1681348478 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5599fb9e8000 con 0x5599ffa92480 2024-04-25T10:30:39.067 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:39.057+0000 7f4bf7723280 4 main: JournalTool: resolving pool 6 2024-04-25T10:30:39.067 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:39.059+0000 7f4bef611640 1 -- 172.21.15.37:0/1681348478 <== mon.0 v2:172.21.15.37:3300/0 4 ==== mgrmap(e 5) ==== 96782+0+0 (secure 0 0 0) 0x559a00914900 con 0x5599ffa92480 2024-04-25T10:30:39.067 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:39.059+0000 7f4bef611640 1 --2- 172.21.15.37:0/1681348478 >> [v2:172.21.15.203:6832/2418205237,v1:172.21.15.203:6833/2418205237] conn(0x5599ffa95180 0x559a007c6100 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:30:39.067 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:39.059+0000 7f4bef611640 1 -- 172.21.15.37:0/1681348478 <== mon.0 v2:172.21.15.37:3300/0 5 ==== osd_map(44..44 src has 1..44) ==== 6248+0+0 (secure 0 0 0) 0x559a00932a00 con 0x5599ffa92480 2024-04-25T10:30:39.067 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:39.059+0000 7f4bf7723280 4 main: JournalTool: creating IoCtx.. 2024-04-25T10:30:39.067 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:39.059+0000 7f4bf7723280 4 main: Executing for rank 0 2024-04-25T10:30:39.067 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:39.059+0000 7f4bf561d640 1 --2- 172.21.15.37:0/1681348478 >> [v2:172.21.15.203:6832/2418205237,v1:172.21.15.203:6833/2418205237] conn(0x5599ffa95180 0x559a007c6100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:30:39.067 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:39.060+0000 7f4bf561d640 1 --2- 172.21.15.37:0/1681348478 >> [v2:172.21.15.203:6832/2418205237,v1:172.21.15.203:6833/2418205237] conn(0x5599ffa95180 0x559a007c6100 secure :-1 s=READY pgs=269 cs=0 l=1 rev1=1 crypto rx=0x5599ffaccde0 tx=0x559a008d6f00 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:30:39.067 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:39.060+0000 7f4bf7723280 1 -- start start 2024-04-25T10:30:39.068 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:39.060+0000 7f4bf7723280 1 Processor -- start 2024-04-25T10:30:39.071 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:39.061+0000 7f4bf7723280 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x5599ffa94880 0x559a007c6680 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:30:39.071 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:39.061+0000 7f4bf7723280 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x5599ffa94d00 0x559a007c6c00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:30:39.071 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:39.061+0000 7f4bf7723280 1 --2- >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x5599ffa95600 0x559a007c7180 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:30:39.071 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:39.061+0000 7f4bf7723280 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 -- 0x5599ffaa2900 con 0x5599ffa94d00 2024-04-25T10:30:39.071 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:39.061+0000 7f4bf7723280 1 -- --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_getmap magic: 0 -- 0x5599ffaa3680 con 0x5599ffa94880 2024-04-25T10:30:39.071 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:39.061+0000 7f4bf7723280 1 -- --> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] -- mon_getmap magic: 0 -- 0x5599ffa44900 con 0x5599ffa95600 2024-04-25T10:30:39.071 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:39.061+0000 7f4bf561d640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x5599ffa94d00 0x559a007c6c00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:30:39.071 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:39.061+0000 7f4bf561d640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x5599ffa94d00 0x559a007c6c00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.37:3300/0 says I am v2:172.21.15.37:55780/0 (socket says 172.21.15.37:55780) 2024-04-25T10:30:39.071 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:39.061+0000 7f4bf561d640 1 -- 172.21.15.37:0/2474076141 learned_addr learned my addr 172.21.15.37:0/2474076141 (peer_addr_for_me v2:172.21.15.37:0/0) 2024-04-25T10:30:39.071 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:39.062+0000 7f4bf5e1e640 1 --2- 172.21.15.37:0/2474076141 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x5599ffa94880 0x559a007c6680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:30:39.071 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:39.062+0000 7f4bf4e1c640 1 --2- 172.21.15.37:0/2474076141 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x5599ffa95600 0x559a007c7180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:30:39.072 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:39.062+0000 7f4bf561d640 1 -- 172.21.15.37:0/2474076141 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x5599ffa95600 msgr2=0x559a007c7180 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:30:39.072 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:39.062+0000 7f4bf561d640 1 --2- 172.21.15.37:0/2474076141 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x5599ffa95600 0x559a007c7180 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:30:39.072 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:39.062+0000 7f4bf561d640 1 -- 172.21.15.37:0/2474076141 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x5599ffa94880 msgr2=0x559a007c6680 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:30:39.072 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:39.062+0000 7f4bf561d640 1 --2- 172.21.15.37:0/2474076141 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x5599ffa94880 0x559a007c6680 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:30:39.072 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:39.062+0000 7f4bf561d640 1 -- 172.21.15.37:0/2474076141 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5599fb9e9860 con 0x5599ffa94d00 2024-04-25T10:30:39.072 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:39.062+0000 7f4bf561d640 1 --2- 172.21.15.37:0/2474076141 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x5599ffa94d00 0x559a007c6c00 secure :-1 s=READY pgs=578 cs=0 l=1 rev1=1 crypto rx=0x5599ffe42090 tx=0x5599ffae24e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=cda77faa28c24894 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:30:39.072 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:39.062+0000 7f4bec60b640 1 -- 172.21.15.37:0/2474076141 <== mon.0 v2:172.21.15.37:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x559a00948ea0 con 0x5599ffa94d00 2024-04-25T10:30:39.072 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:39.062+0000 7f4bec60b640 1 -- 172.21.15.37:0/2474076141 <== mon.0 v2:172.21.15.37:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x5599fb9de700 con 0x5599ffa94d00 2024-04-25T10:30:39.072 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:39.062+0000 7f4bec60b640 1 -- 172.21.15.37:0/2474076141 <== mon.0 v2:172.21.15.37:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x559a00953d40 con 0x5599ffa94d00 2024-04-25T10:30:39.072 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:39.062+0000 7f4bf7723280 1 -- 172.21.15.37:0/2474076141 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5599fb9e8960 con 0x5599ffa94d00 2024-04-25T10:30:39.072 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:39.062+0000 7f4bf7723280 1 -- 172.21.15.37:0/2474076141 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({fsmap=0}) -- 0x5599fb9e8780 con 0x5599ffa94d00 2024-04-25T10:30:39.072 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:39.062+0000 7f4bf7723280 4 waiting for MDS map... 2024-04-25T10:30:39.072 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:39.062+0000 7f4bec60b640 1 -- 172.21.15.37:0/2474076141 <== mon.0 v2:172.21.15.37:3300/0 4 ==== osd_map(44..44 src has 1..44) ==== 6248+0+0 (secure 0 0 0) 0x5599ffe44600 con 0x5599ffa94d00 2024-04-25T10:30:39.072 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:39.062+0000 7f4bec60b640 1 -- 172.21.15.37:0/2474076141 <== mon.0 v2:172.21.15.37:3300/0 5 ==== fsmap(e 22) ==== 4135+0+0 (secure 0 0 0) 0x5599ffa3d500 con 0x5599ffa94d00 2024-04-25T10:30:39.073 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:39.062+0000 7f4bf7723280 4 Got MDS map 22 2024-04-25T10:30:39.073 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:39.063+0000 7f4bf7723280 1 --2- 172.21.15.37:0/2474076141 >> [v2:172.21.15.203:6824/1203822890,v1:172.21.15.203:6825/1203822890] conn(0x5599ffa95a80 0x559a007c7700 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:30:39.073 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:39.063+0000 7f4bf7723280 1 -- 172.21.15.37:0/2474076141 --> [v2:172.21.15.203:6824/1203822890,v1:172.21.15.203:6825/1203822890] -- osd_op(unknown.0.0:1 6.f 6:f1f69726:::400.00000000:head [writefull 0~22 in=22b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e44) -- 0x5599ffe5e000 con 0x5599ffa95a80 2024-04-25T10:30:39.073 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:39.063+0000 7f4bf5e1e640 1 --2- 172.21.15.37:0/2474076141 >> [v2:172.21.15.203:6824/1203822890,v1:172.21.15.203:6825/1203822890] conn(0x5599ffa95a80 0x559a007c7700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:30:39.073 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:39.063+0000 7f4bf5e1e640 1 --2- 172.21.15.37:0/2474076141 >> [v2:172.21.15.203:6824/1203822890,v1:172.21.15.203:6825/1203822890] conn(0x5599ffa95a80 0x559a007c7700 crc :-1 s=READY pgs=17 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.5 client_cookie=d74cd13f0b4b268b server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:30:39.073 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:39.065+0000 7f4bf5e1e640 1 -- 172.21.15.37:0/2474076141 <== osd.5 v2:172.21.15.203:6824/1203822890 1 ==== osd_op_reply(1 400.00000000 [writefull 0~22] v44'1 uv1 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x5599ffb08000 con 0x5599ffa95a80 2024-04-25T10:30:39.073 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:39.065+0000 7f4bf7723280 1 --2- 172.21.15.37:0/2474076141 >> [v2:172.21.15.203:6800/4051532063,v1:172.21.15.203:6801/4051532063] conn(0x5599ffe64000 0x5599ffe6c000 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:30:39.073 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:39.065+0000 7f4bf7723280 1 -- 172.21.15.37:0/2474076141 --> [v2:172.21.15.203:6800/4051532063,v1:172.21.15.203:6801/4051532063] -- osd_op(unknown.0.0:2 6.14 6:292cf221:::200.00000000:head [writefull 0~90 in=90b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e44) -- 0x5599ffe5e400 con 0x5599ffe64000 2024-04-25T10:30:39.073 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:39.065+0000 7f4bf4e1c640 1 --2- 172.21.15.37:0/2474076141 >> [v2:172.21.15.203:6800/4051532063,v1:172.21.15.203:6801/4051532063] conn(0x5599ffe64000 0x5599ffe6c000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:30:39.073 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:39.065+0000 7f4bf4e1c640 1 --2- 172.21.15.37:0/2474076141 >> [v2:172.21.15.203:6800/4051532063,v1:172.21.15.203:6801/4051532063] conn(0x5599ffe64000 0x5599ffe6c000 crc :-1 s=READY pgs=17 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.4 client_cookie=830fadba6870b7be server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:30:39.073 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:39.067+0000 7f4bf4e1c640 1 -- 172.21.15.37:0/2474076141 <== osd.4 v2:172.21.15.203:6800/4051532063 1 ==== osd_op_reply(2 200.00000000 [writefull 0~90] v44'2 uv2 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x5599ffb08500 con 0x5599ffe64000 2024-04-25T10:30:39.074 INFO:teuthology.orchestra.run.smithi037.stdout:writing EResetJournal entry 2024-04-25T10:30:39.078 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:39.067+0000 7f4bf7723280 1 -- 172.21.15.37:0/2474076141 --> [v2:172.21.15.203:6800/4051532063,v1:172.21.15.203:6801/4051532063] -- osd_op(unknown.0.0:3 6.14 6:2e2fa760:::200.00000001:head [delete] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e44) -- 0x5599ffe5e800 con 0x5599ffe64000 2024-04-25T10:30:39.078 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:39.067+0000 7f4bf7723280 1 --2- 172.21.15.37:0/2474076141 >> [v2:172.21.15.203:6810/2309615014,v1:172.21.15.203:6812/2309615014] conn(0x5599ffe64480 0x5599ffe6c580 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:30:39.079 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:39.067+0000 7f4bf7723280 1 -- 172.21.15.37:0/2474076141 --> [v2:172.21.15.203:6810/2309615014,v1:172.21.15.203:6812/2309615014] -- osd_op(unknown.0.0:4 6.1b 6:ddb4e4d7:::200.00000002:head [delete] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e44) -- 0x5599ffe5ec00 con 0x5599ffe64480 2024-04-25T10:30:39.079 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:39.067+0000 7f4bf7723280 1 -- 172.21.15.37:0/2474076141 --> [v2:172.21.15.203:6800/4051532063,v1:172.21.15.203:6801/4051532063] -- osd_op(unknown.0.0:5 6.b 6:d5c7a900:::200.00000003:head [delete] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e44) -- 0x5599ffe5f000 con 0x5599ffe64000 2024-04-25T10:30:39.079 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:39.067+0000 7f4bf7723280 1 -- 172.21.15.37:0/2474076141 --> [v2:172.21.15.203:6810/2309615014,v1:172.21.15.203:6812/2309615014] -- osd_op(unknown.0.0:6 6.1f 6:f83d65d8:::200.00000004:head [delete] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e44) -- 0x5599ffe5f400 con 0x5599ffe64480 2024-04-25T10:30:39.079 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:39.067+0000 7f4bf7723280 1 --2- 172.21.15.37:0/2474076141 >> [v2:172.21.15.37:6824/1815290618,v1:172.21.15.37:6825/1815290618] conn(0x5599ffe64900 0x5599ffe6cb00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:30:39.079 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:39.067+0000 7f4bf7723280 1 -- 172.21.15.37:0/2474076141 --> [v2:172.21.15.37:6824/1815290618,v1:172.21.15.37:6825/1815290618] -- osd_op(unknown.0.0:7 6.1a 6:5e41ab69:::200.00000005:head [delete] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e44) -- 0x5599ffe5f800 con 0x5599ffe64900 2024-04-25T10:30:39.079 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:39.067+0000 7f4bf7723280 1 -- 172.21.15.37:0/2474076141 --> [v2:172.21.15.37:6824/1815290618,v1:172.21.15.37:6825/1815290618] -- osd_op(unknown.0.0:8 6.1e 6:7bf7e7b5:::200.00000006:head [delete] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e44) -- 0x5599ffe5fc00 con 0x5599ffe64900 2024-04-25T10:30:39.079 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:39.067+0000 7f4bf5e1e640 1 --2- 172.21.15.37:0/2474076141 >> [v2:172.21.15.203:6810/2309615014,v1:172.21.15.203:6812/2309615014] conn(0x5599ffe64480 0x5599ffe6c580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:30:39.079 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:39.067+0000 7f4bf561d640 1 --2- 172.21.15.37:0/2474076141 >> [v2:172.21.15.37:6824/1815290618,v1:172.21.15.37:6825/1815290618] conn(0x5599ffe64900 0x5599ffe6cb00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:30:39.079 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:39.067+0000 7f4bf561d640 1 --2- 172.21.15.37:0/2474076141 >> [v2:172.21.15.37:6824/1815290618,v1:172.21.15.37:6825/1815290618] conn(0x5599ffe64900 0x5599ffe6cb00 crc :-1 s=READY pgs=17 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.3 client_cookie=5e428a46cbffb4cf server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:30:39.079 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:39.067+0000 7f4bf5e1e640 1 --2- 172.21.15.37:0/2474076141 >> [v2:172.21.15.203:6810/2309615014,v1:172.21.15.203:6812/2309615014] conn(0x5599ffe64480 0x5599ffe6c580 crc :-1 s=READY pgs=16 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.6 client_cookie=9701a222fc732d8e server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:30:39.079 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:39.068+0000 7f4bf4e1c640 1 -- 172.21.15.37:0/2474076141 <== osd.4 v2:172.21.15.203:6800/4051532063 2 ==== osd_op_reply(3 200.00000001 [delete] v44'3 uv2 ondisk = -2 ((2) No such file or directory)) ==== 156+0+0 (crc 0 0 0) 0x5599ffb08500 con 0x5599ffe64000 2024-04-25T10:30:39.079 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:39.068+0000 7f4bf4e1c640 1 -- 172.21.15.37:0/2474076141 <== osd.4 v2:172.21.15.203:6800/4051532063 3 ==== osd_op_reply(5 200.00000003 [delete] v44'2 uv1 ondisk = -2 ((2) No such file or directory)) ==== 156+0+0 (crc 0 0 0) 0x5599ffb08500 con 0x5599ffe64000 2024-04-25T10:30:39.079 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:39.069+0000 7f4bf561d640 1 -- 172.21.15.37:0/2474076141 <== osd.3 v2:172.21.15.37:6824/1815290618 1 ==== osd_op_reply(7 200.00000005 [delete] v44'1 uv0 ondisk = -2 ((2) No such file or directory)) ==== 156+0+0 (crc 0 0 0) 0x5599ffb08780 con 0x5599ffe64900 2024-04-25T10:30:39.079 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:39.069+0000 7f4bf561d640 1 -- 172.21.15.37:0/2474076141 <== osd.3 v2:172.21.15.37:6824/1815290618 2 ==== osd_op_reply(8 200.00000006 [delete] v44'1 uv0 ondisk = -2 ((2) No such file or directory)) ==== 156+0+0 (crc 0 0 0) 0x5599ffb08780 con 0x5599ffe64900 2024-04-25T10:30:39.080 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:39.069+0000 7f4bf5e1e640 1 -- 172.21.15.37:0/2474076141 <== osd.6 v2:172.21.15.203:6810/2309615014 1 ==== osd_op_reply(4 200.00000002 [delete] v44'1 uv0 ondisk = -2 ((2) No such file or directory)) ==== 156+0+0 (crc 0 0 0) 0x5599ffb08000 con 0x5599ffe64480 2024-04-25T10:30:39.080 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:39.069+0000 7f4beae08640 1 -- 172.21.15.37:0/2474076141 --> [v2:172.21.15.203:6800/4051532063,v1:172.21.15.203:6801/4051532063] -- osd_op(unknown.0.0:9 6.14 6:2e2fa760:::200.00000001:head [write 0~48 in=48b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e44) -- 0x5599ffe7c000 con 0x5599ffe64000 2024-04-25T10:30:39.080 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:39.069+0000 7f4bf5e1e640 1 -- 172.21.15.37:0/2474076141 <== osd.6 v2:172.21.15.203:6810/2309615014 2 ==== osd_op_reply(6 200.00000004 [delete] v44'5 uv4 ondisk = -2 ((2) No such file or directory)) ==== 156+0+0 (crc 0 0 0) 0x5599ffb08000 con 0x5599ffe64480 2024-04-25T10:30:39.080 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:39.071+0000 7f4bf4e1c640 1 -- 172.21.15.37:0/2474076141 <== osd.4 v2:172.21.15.203:6800/4051532063 4 ==== osd_op_reply(9 200.00000001 [write 0~48] v44'4 uv4 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x5599ffb08500 con 0x5599ffe64000 2024-04-25T10:30:39.080 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:39.072+0000 7f4bf7723280 4 Successfully wrote new journal pointer and header for rank 3:0 2024-04-25T10:30:39.080 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:39.072+0000 7f4bf7723280 1 -- 172.21.15.37:0/2474076141 >> [v2:172.21.15.37:6824/1815290618,v1:172.21.15.37:6825/1815290618] conn(0x5599ffe64900 msgr2=0x5599ffe6cb00 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:30:39.080 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:39.072+0000 7f4bf7723280 1 --2- 172.21.15.37:0/2474076141 >> [v2:172.21.15.37:6824/1815290618,v1:172.21.15.37:6825/1815290618] conn(0x5599ffe64900 0x5599ffe6cb00 crc :-1 s=READY pgs=17 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:30:39.080 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:39.072+0000 7f4bf7723280 1 -- 172.21.15.37:0/2474076141 >> [v2:172.21.15.203:6800/4051532063,v1:172.21.15.203:6801/4051532063] conn(0x5599ffe64000 msgr2=0x5599ffe6c000 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:30:39.080 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:39.072+0000 7f4bf7723280 1 --2- 172.21.15.37:0/2474076141 >> [v2:172.21.15.203:6800/4051532063,v1:172.21.15.203:6801/4051532063] conn(0x5599ffe64000 0x5599ffe6c000 crc :-1 s=READY pgs=17 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:30:39.080 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:39.072+0000 7f4bf7723280 1 -- 172.21.15.37:0/2474076141 >> [v2:172.21.15.203:6824/1203822890,v1:172.21.15.203:6825/1203822890] conn(0x5599ffa95a80 msgr2=0x559a007c7700 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:30:39.080 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:39.072+0000 7f4bf7723280 1 --2- 172.21.15.37:0/2474076141 >> [v2:172.21.15.203:6824/1203822890,v1:172.21.15.203:6825/1203822890] conn(0x5599ffa95a80 0x559a007c7700 crc :-1 s=READY pgs=17 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:30:39.080 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:39.072+0000 7f4bf7723280 1 -- 172.21.15.37:0/2474076141 >> [v2:172.21.15.203:6810/2309615014,v1:172.21.15.203:6812/2309615014] conn(0x5599ffe64480 msgr2=0x5599ffe6c580 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:30:39.080 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:39.072+0000 7f4bf7723280 1 --2- 172.21.15.37:0/2474076141 >> [v2:172.21.15.203:6810/2309615014,v1:172.21.15.203:6812/2309615014] conn(0x5599ffe64480 0x5599ffe6c580 crc :-1 s=READY pgs=16 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:30:39.080 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:39.072+0000 7f4bf4e1c640 1 -- 172.21.15.37:0/2474076141 reap_dead start 2024-04-25T10:30:39.080 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:39.072+0000 7f4bf7723280 1 -- 172.21.15.37:0/2474076141 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x5599ffa94d00 msgr2=0x559a007c6c00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:30:39.081 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:39.072+0000 7f4bf7723280 1 --2- 172.21.15.37:0/2474076141 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x5599ffa94d00 0x559a007c6c00 secure :-1 s=READY pgs=578 cs=0 l=1 rev1=1 crypto rx=0x5599ffe42090 tx=0x5599ffae24e0 comp rx=0 tx=0).stop 2024-04-25T10:30:39.081 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:39.072+0000 7f4bf7723280 1 -- 172.21.15.37:0/2474076141 shutdown_connections 2024-04-25T10:30:39.081 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:39.072+0000 7f4bf7723280 1 --2- 172.21.15.37:0/2474076141 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x5599ffa94d00 0x559a007c6c00 unknown :-1 s=CLOSED pgs=578 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:30:39.081 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:39.072+0000 7f4bf7723280 1 -- 172.21.15.37:0/2474076141 >> 172.21.15.37:0/2474076141 conn(0x5599ffa92000 msgr2=0x5599ffde4800 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:30:39.081 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:39.072+0000 7f4bf7723280 1 -- 172.21.15.37:0/2474076141 shutdown_connections 2024-04-25T10:30:39.081 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:39.072+0000 7f4bf7723280 1 -- 172.21.15.37:0/2474076141 wait complete. 2024-04-25T10:30:39.081 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:39.072+0000 7f4bf7723280 1 -- 172.21.15.37:0/1681348478 >> [v2:172.21.15.203:6832/2418205237,v1:172.21.15.203:6833/2418205237] conn(0x5599ffa95180 msgr2=0x559a007c6100 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:30:39.081 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:39.072+0000 7f4bf7723280 1 --2- 172.21.15.37:0/1681348478 >> [v2:172.21.15.203:6832/2418205237,v1:172.21.15.203:6833/2418205237] conn(0x5599ffa95180 0x559a007c6100 secure :-1 s=READY pgs=269 cs=0 l=1 rev1=1 crypto rx=0x5599ffaccde0 tx=0x559a008d6f00 comp rx=0 tx=0).stop 2024-04-25T10:30:39.081 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:39.072+0000 7f4bf7723280 1 -- 172.21.15.37:0/1681348478 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x5599ffa92480 msgr2=0x559a007c5b80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:30:39.081 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:39.072+0000 7f4bf7723280 1 --2- 172.21.15.37:0/1681348478 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x5599ffa92480 0x559a007c5b80 secure :-1 s=READY pgs=577 cs=0 l=1 rev1=1 crypto rx=0x559a008d5c50 tx=0x559a008d6c00 comp rx=0 tx=0).stop 2024-04-25T10:30:39.081 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:39.072+0000 7f4bf7723280 1 -- 172.21.15.37:0/1681348478 shutdown_connections 2024-04-25T10:30:39.081 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:39.072+0000 7f4bf7723280 1 --2- 172.21.15.37:0/1681348478 >> [v2:172.21.15.203:6832/2418205237,v1:172.21.15.203:6833/2418205237] conn(0x5599ffa95180 0x559a007c6100 unknown :-1 s=CLOSED pgs=269 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:30:39.081 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:39.072+0000 7f4bf7723280 1 --2- 172.21.15.37:0/1681348478 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x5599ffa92d80 0x559a007c5080 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:30:39.081 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:39.072+0000 7f4bf7723280 1 --2- 172.21.15.37:0/1681348478 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x5599ffa92900 0x559a007c5600 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:30:39.082 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:39.072+0000 7f4bf7723280 1 --2- 172.21.15.37:0/1681348478 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x5599ffa92480 0x559a007c5b80 unknown :-1 s=CLOSED pgs=577 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:30:39.082 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:39.072+0000 7f4bf7723280 1 -- 172.21.15.37:0/1681348478 >> 172.21.15.37:0/1681348478 conn(0x5599ffa93200 msgr2=0x5599ffac8000 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:30:39.082 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:39.072+0000 7f4bf7723280 1 -- 172.21.15.37:0/1681348478 shutdown_connections 2024-04-25T10:30:39.082 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:39.072+0000 7f4bf7723280 1 -- 172.21.15.37:0/1681348478 wait complete. 2024-04-25T10:30:39.082 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:39.073+0000 7f4bf7723280 1 -- 172.21.15.37:0/3658284047 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x5599ffa94400 msgr2=0x559a007c4b00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:30:39.082 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:39.073+0000 7f4bf7723280 1 --2- 172.21.15.37:0/3658284047 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x5599ffa94400 0x559a007c4b00 secure :-1 s=READY pgs=575 cs=0 l=1 rev1=1 crypto rx=0x559a008150b0 tx=0x5599ffae3f80 comp rx=0 tx=0).stop 2024-04-25T10:30:39.082 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:39.073+0000 7f4bf7723280 1 -- 172.21.15.37:0/3658284047 shutdown_connections 2024-04-25T10:30:39.082 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:39.073+0000 7f4bf7723280 1 --2- 172.21.15.37:0/3658284047 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x5599ffa94400 0x559a007c4b00 unknown :-1 s=CLOSED pgs=575 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:30:39.082 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:39.073+0000 7f4bf7723280 1 --2- 172.21.15.37:0/3658284047 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x5599ffa93b00 0x559a007c4580 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:30:39.082 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:39.073+0000 7f4bf7723280 1 --2- 172.21.15.37:0/3658284047 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x5599ffa93680 0x559a007c4000 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:30:39.082 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:39.073+0000 7f4bf7723280 1 -- 172.21.15.37:0/3658284047 >> 172.21.15.37:0/3658284047 conn(0x5599ffa93f80 msgr2=0x5599ffac9000 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:30:39.082 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:39.073+0000 7f4bf7723280 1 -- 172.21.15.37:0/3658284047 shutdown_connections 2024-04-25T10:30:39.082 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:39.073+0000 7f4bf7723280 1 -- 172.21.15.37:0/3658284047 wait complete. 2024-04-25T10:30:39.083 DEBUG:tasks.cephfs.filesystem:Ran ['cephfs-journal-tool', '--debug-mds=20', '--debug-ms=1', '--debug-objecter=1', '--rank', 'recovery_fs:0', 'journal', 'reset', '--force'] in time 0:00:00.085183, result: writing EResetJournal entry 2024-04-25T10:30:39.083 DEBUG:teuthology.orchestra.run.smithi037:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs set recovery_fs joinable true 2024-04-25T10:30:39.197 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:39.195+0000 7fedc7434640 1 Processor -- start 2024-04-25T10:30:39.197 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:39.195+0000 7fedc7434640 1 -- start start 2024-04-25T10:30:39.199 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:39.196+0000 7fedc7434640 1 --2- >> v2:172.21.15.203:3300/0 conn(0x7fedc00d5430 0x7fedc00b75c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:30:39.199 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:39.196+0000 7fedc7434640 1 --2- >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7fedc0161110 0x7fedc0161880 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:30:39.199 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:39.196+0000 7fedc7434640 1 -- --> v1:172.21.15.203:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fedc00b7f10 con 0x7fedc00b7b00 2024-04-25T10:30:39.199 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:39.196+0000 7fedc7434640 1 -- --> v2:172.21.15.203:3300/0 -- mon_getmap magic: 0 -- 0x7fedc00d4420 con 0x7fedc00d5430 2024-04-25T10:30:39.199 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:39.196+0000 7fedc7434640 1 -- --> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] -- mon_getmap magic: 0 -- 0x7fedc0161520 con 0x7fedc0161110 2024-04-25T10:30:39.199 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:39.197+0000 7fedc51d0640 1 --2- >> v2:172.21.15.203:3300/0 conn(0x7fedc00d5430 0x7fedc00b75c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:30:39.199 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:39.197+0000 7fedc59d1640 1 --2- >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7fedc0161110 0x7fedc0161880 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:30:39.199 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:39.197+0000 7fedc49cf640 1 --1- >> v1:172.21.15.203:6789/0 conn(0x7fedc00b7b00 0x7fedc0160a00 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.203:6789/0 says I am v1:172.21.15.37:60394/0 (socket says 172.21.15.37:60394) 2024-04-25T10:30:39.200 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:39.197+0000 7fedc49cf640 1 -- 172.21.15.37:0/1160602803 learned_addr learned my addr 172.21.15.37:0/1160602803 (peer_addr_for_me v1:172.21.15.37:0/0) 2024-04-25T10:30:39.200 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:39.197+0000 7fedc51d0640 1 -- 172.21.15.37:0/1160602803 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7fedc0161110 msgr2=0x7fedc0161880 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:30:39.200 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:39.197+0000 7fedc51d0640 1 --2- 172.21.15.37:0/1160602803 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7fedc0161110 0x7fedc0161880 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:30:39.200 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:39.197+0000 7fedc51d0640 1 -- 172.21.15.37:0/1160602803 >> v1:172.21.15.203:6789/0 conn(0x7fedc00b7b00 legacy=0x7fedc0160a00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:30:39.200 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:39.197+0000 7fedc51d0640 1 -- 172.21.15.37:0/1160602803 --> v2:172.21.15.203:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fedc00d5840 con 0x7fedc00d5430 2024-04-25T10:30:39.201 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:39.198+0000 7fedc51d0640 1 --2- 172.21.15.37:0/1160602803 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fedc00d5430 0x7fedc00b75c0 secure :-1 s=READY pgs=223 cs=0 l=1 rev1=1 crypto rx=0x7fedbc0097a0 tx=0x7fedbc02a100 comp rx=0 tx=0).ready entity=mon.1 client_cookie=5d73340ef6f21fca server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:30:39.201 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:39.198+0000 7fedb7fff640 1 -- 172.21.15.37:0/1160602803 <== mon.1 v2:172.21.15.203:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fedbc037070 con 0x7fedc00d5430 2024-04-25T10:30:39.201 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:39.198+0000 7fedb7fff640 1 -- 172.21.15.37:0/1160602803 <== mon.1 v2:172.21.15.203:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fedbc02ae50 con 0x7fedc00d5430 2024-04-25T10:30:39.201 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:39.198+0000 7fedb7fff640 1 -- 172.21.15.37:0/1160602803 <== mon.1 v2:172.21.15.203:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fedbc032660 con 0x7fedc00d5430 2024-04-25T10:30:39.201 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:39.199+0000 7fedc7434640 1 -- 172.21.15.37:0/1160602803 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fedc00d5430 msgr2=0x7fedc00b75c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:30:39.201 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:39.199+0000 7fedc7434640 1 --2- 172.21.15.37:0/1160602803 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fedc00d5430 0x7fedc00b75c0 secure :-1 s=READY pgs=223 cs=0 l=1 rev1=1 crypto rx=0x7fedbc0097a0 tx=0x7fedbc02a100 comp rx=0 tx=0).stop 2024-04-25T10:30:39.201 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:39.199+0000 7fedc7434640 1 -- 172.21.15.37:0/1160602803 shutdown_connections 2024-04-25T10:30:39.202 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:39.199+0000 7fedc7434640 1 --2- 172.21.15.37:0/1160602803 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7fedc0161110 0x7fedc0161880 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:30:39.202 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:39.199+0000 7fedc7434640 1 --2- 172.21.15.37:0/1160602803 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fedc00d5430 0x7fedc00b75c0 unknown :-1 s=CLOSED pgs=223 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:30:39.202 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:39.199+0000 7fedc7434640 1 -- 172.21.15.37:0/1160602803 >> 172.21.15.37:0/1160602803 conn(0x7fedc00d27a0 msgr2=0x7fedc00d2bb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:30:39.202 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:39.199+0000 7fedc7434640 1 -- 172.21.15.37:0/1160602803 shutdown_connections 2024-04-25T10:30:39.202 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:39.199+0000 7fedc7434640 1 -- 172.21.15.37:0/1160602803 wait complete. 2024-04-25T10:30:39.202 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:39.200+0000 7fedc7434640 1 Processor -- start 2024-04-25T10:30:39.202 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:39.201+0000 7fedc7434640 1 -- start start 2024-04-25T10:30:39.205 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:39.202+0000 7fedc7434640 1 --2- >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7fedc00b7b00 0x7fedc0150f70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:30:39.205 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:39.202+0000 7fedc7434640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fedc00d5430 0x7fedc015bbc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:30:39.205 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:39.202+0000 7fedc7434640 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fedc0161110 0x7fedc015db80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:30:39.205 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:39.202+0000 7fedc7434640 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 -- 0x7fedc00d4c30 con 0x7fedc00d5430 2024-04-25T10:30:39.205 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:39.202+0000 7fedc7434640 1 -- --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_getmap magic: 0 -- 0x7fedc00d4950 con 0x7fedc0161110 2024-04-25T10:30:39.205 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:39.202+0000 7fedc7434640 1 -- --> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] -- mon_getmap magic: 0 -- 0x7fedc00d4f10 con 0x7fedc00b7b00 2024-04-25T10:30:39.205 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:39.202+0000 7fedc49cf640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fedc00d5430 0x7fedc015bbc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:30:39.205 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:39.202+0000 7fedc51d0640 1 --2- >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7fedc00b7b00 0x7fedc0150f70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:30:39.206 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:39.202+0000 7fedc49cf640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fedc00d5430 0x7fedc015bbc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.37:3300/0 says I am v2:172.21.15.37:55792/0 (socket says 172.21.15.37:55792) 2024-04-25T10:30:39.206 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:39.202+0000 7fedc51d0640 1 --2- >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7fedc00b7b00 0x7fedc0150f70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.203:3301/0 says I am v2:172.21.15.37:45448/0 (socket says 172.21.15.37:45448) 2024-04-25T10:30:39.206 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:39.202+0000 7fedc49cf640 1 -- 172.21.15.37:0/2985137640 learned_addr learned my addr 172.21.15.37:0/2985137640 (peer_addr_for_me v2:172.21.15.37:0/0) 2024-04-25T10:30:39.206 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:39.202+0000 7fedc59d1640 1 --2- 172.21.15.37:0/2985137640 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fedc0161110 0x7fedc015db80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:30:39.206 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:39.202+0000 7fedc51d0640 1 -- 172.21.15.37:0/2985137640 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fedc0161110 msgr2=0x7fedc015db80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:30:39.206 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:39.202+0000 7fedc51d0640 1 --2- 172.21.15.37:0/2985137640 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fedc0161110 0x7fedc015db80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:30:39.206 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:39.202+0000 7fedc51d0640 1 -- 172.21.15.37:0/2985137640 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fedc00d5430 msgr2=0x7fedc015bbc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:30:39.206 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:39.202+0000 7fedc51d0640 1 --2- 172.21.15.37:0/2985137640 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fedc00d5430 0x7fedc015bbc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:30:39.206 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:39.202+0000 7fedc51d0640 1 -- 172.21.15.37:0/2985137640 --> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fedc01f6010 con 0x7fedc00b7b00 2024-04-25T10:30:39.206 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:39.202+0000 7fedc49cf640 1 --2- 172.21.15.37:0/2985137640 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fedc00d5430 0x7fedc015bbc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-25T10:30:39.206 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:39.203+0000 7fedc51d0640 1 --2- 172.21.15.37:0/2985137640 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7fedc00b7b00 0x7fedc0150f70 secure :-1 s=READY pgs=61 cs=0 l=1 rev1=1 crypto rx=0x7fedbc031000 tx=0x7fedbc0046c0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:30:39.206 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:39.203+0000 7fedb5ffb640 1 -- 172.21.15.37:0/2985137640 <== mon.2 v2:172.21.15.203:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fedbc041070 con 0x7fedc00b7b00 2024-04-25T10:30:39.206 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:39.203+0000 7fedb5ffb640 1 -- 172.21.15.37:0/2985137640 <== mon.2 v2:172.21.15.203:3301/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fedbc032e20 con 0x7fedc00b7b00 2024-04-25T10:30:39.206 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:39.203+0000 7fedb5ffb640 1 -- 172.21.15.37:0/2985137640 <== mon.2 v2:172.21.15.203:3301/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fedbc037040 con 0x7fedc00b7b00 2024-04-25T10:30:39.207 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:39.203+0000 7fedc7434640 1 -- 172.21.15.37:0/2985137640 --> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fedc01514b0 con 0x7fedc00b7b00 2024-04-25T10:30:39.207 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:39.203+0000 7fedc7434640 1 -- 172.21.15.37:0/2985137640 --> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] -- mon_subscribe({osdmap=0}) -- 0x7fedc00ba570 con 0x7fedc00b7b00 2024-04-25T10:30:39.208 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:39.204+0000 7fedc7434640 1 -- 172.21.15.37:0/2985137640 --> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fedc006a140 con 0x7fedc00b7b00 2024-04-25T10:30:39.208 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:39.205+0000 7fedb5ffb640 1 -- 172.21.15.37:0/2985137640 <== mon.2 v2:172.21.15.203:3301/0 4 ==== mgrmap(e 5) ==== 96782+0+0 (secure 0 0 0) 0x7fedbc03a070 con 0x7fedc00b7b00 2024-04-25T10:30:39.208 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:39.205+0000 7fedb5ffb640 1 --2- 172.21.15.37:0/2985137640 >> [v2:172.21.15.203:6832/2418205237,v1:172.21.15.203:6833/2418205237] conn(0x7feda40744e0 0x7feda40769f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:30:39.208 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:39.205+0000 7fedb5ffb640 1 -- 172.21.15.37:0/2985137640 <== mon.2 v2:172.21.15.203:3301/0 5 ==== osd_map(44..44 src has 1..44) ==== 6248+0+0 (secure 0 0 0) 0x7fedbc0b55c0 con 0x7fedc00b7b00 2024-04-25T10:30:39.208 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:39.205+0000 7fedc49cf640 1 --2- 172.21.15.37:0/2985137640 >> [v2:172.21.15.203:6832/2418205237,v1:172.21.15.203:6833/2418205237] conn(0x7feda40744e0 0x7feda40769f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:30:39.208 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:39.206+0000 7fedc49cf640 1 --2- 172.21.15.37:0/2985137640 >> [v2:172.21.15.203:6832/2418205237,v1:172.21.15.203:6833/2418205237] conn(0x7feda40744e0 0x7feda40769f0 secure :-1 s=READY pgs=270 cs=0 l=1 rev1=1 crypto rx=0x7fedc00d3070 tx=0x7fedb0009040 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:30:39.212 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:39.210+0000 7fedb5ffb640 1 -- 172.21.15.37:0/2985137640 <== mon.2 v2:172.21.15.203:3301/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187246 (secure 0 0 0) 0x7fedbc07f4d0 con 0x7fedc00b7b00 2024-04-25T10:30:39.435 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:39.433+0000 7fedc7434640 1 -- 172.21.15.37:0/2985137640 --> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] -- mon_command({"prefix": "fs set", "fs_name": "recovery_fs", "var": "joinable", "val": "true"} v 0) -- 0x7fedc006cf00 con 0x7fedc00b7b00 2024-04-25T10:30:39.921 INFO:tasks.ceph.mon.a.smithi037.stderr:2024-04-25T10:30:39.918+0000 7fb24ed56640 -1 log_channel(cluster) log [ERR] : Health check update: 1 filesystem is offline (MDS_ALL_DOWN) 2024-04-25T10:30:39.921 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:39.919+0000 7fedb5ffb640 1 -- 172.21.15.37:0/2985137640 <== mon.2 v2:172.21.15.203:3301/0 7 ==== mon_command_ack([{"prefix": "fs set", "fs_name": "recovery_fs", "var": "joinable", "val": "true"}]=0 recovery_fs marked joinable; MDS may join as newly active. v23) ==== 172+0+0 (secure 0 0 0) 0x7fedbc0837d0 con 0x7fedc00b7b00 2024-04-25T10:30:39.921 INFO:teuthology.orchestra.run.smithi037.stderr:recovery_fs marked joinable; MDS may join as newly active. 2024-04-25T10:30:39.924 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:39.921+0000 7fedc7434640 1 -- 172.21.15.37:0/2985137640 >> [v2:172.21.15.203:6832/2418205237,v1:172.21.15.203:6833/2418205237] conn(0x7feda40744e0 msgr2=0x7feda40769f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:30:39.924 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:39.921+0000 7fedc7434640 1 --2- 172.21.15.37:0/2985137640 >> [v2:172.21.15.203:6832/2418205237,v1:172.21.15.203:6833/2418205237] conn(0x7feda40744e0 0x7feda40769f0 secure :-1 s=READY pgs=270 cs=0 l=1 rev1=1 crypto rx=0x7fedc00d3070 tx=0x7fedb0009040 comp rx=0 tx=0).stop 2024-04-25T10:30:39.924 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:39.921+0000 7fedc7434640 1 -- 172.21.15.37:0/2985137640 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7fedc00b7b00 msgr2=0x7fedc0150f70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:30:39.924 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:39.921+0000 7fedc7434640 1 --2- 172.21.15.37:0/2985137640 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7fedc00b7b00 0x7fedc0150f70 secure :-1 s=READY pgs=61 cs=0 l=1 rev1=1 crypto rx=0x7fedbc031000 tx=0x7fedbc0046c0 comp rx=0 tx=0).stop 2024-04-25T10:30:39.924 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:39.921+0000 7fedc7434640 1 -- 172.21.15.37:0/2985137640 shutdown_connections 2024-04-25T10:30:39.924 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:39.921+0000 7fedc7434640 1 --2- 172.21.15.37:0/2985137640 >> [v2:172.21.15.203:6832/2418205237,v1:172.21.15.203:6833/2418205237] conn(0x7feda40744e0 0x7feda40769f0 unknown :-1 s=CLOSED pgs=270 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:30:39.924 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:39.921+0000 7fedc7434640 1 --2- 172.21.15.37:0/2985137640 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fedc0161110 0x7fedc015db80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:30:39.924 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:39.921+0000 7fedc7434640 1 --2- 172.21.15.37:0/2985137640 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fedc00d5430 0x7fedc015bbc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:30:39.924 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:39.921+0000 7fedc7434640 1 --2- 172.21.15.37:0/2985137640 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7fedc00b7b00 0x7fedc0150f70 unknown :-1 s=CLOSED pgs=61 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:30:39.924 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:39.921+0000 7fedc7434640 1 -- 172.21.15.37:0/2985137640 >> 172.21.15.37:0/2985137640 conn(0x7fedc00d27a0 msgr2=0x7fedc015ea10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:30:39.924 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:39.921+0000 7fedc7434640 1 -- 172.21.15.37:0/2985137640 shutdown_connections 2024-04-25T10:30:39.924 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:39.921+0000 7fedc7434640 1 -- 172.21.15.37:0/2985137640 wait complete. 2024-04-25T10:30:39.927 INFO:tasks.ceph.mds.c.smithi037.stderr:2024-04-25T10:30:39.925+0000 7f12be3c9640 -1 mds.0.openfiles _load_finish got (2) No such file or directory 2024-04-25T10:30:39.939 DEBUG:teuthology.orchestra.run.smithi037:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-04-25T10:30:40.056 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:40.054+0000 7feb16849640 1 Processor -- start 2024-04-25T10:30:40.056 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:40.054+0000 7feb16849640 1 -- start start 2024-04-25T10:30:40.058 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:40.055+0000 7feb16849640 1 --2- >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7feb100d1c20 0x7feb100d2030 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:30:40.058 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:40.055+0000 7feb16849640 1 --2- >> v2:172.21.15.37:3300/0 conn(0x7feb100b71b0 0x7feb100b75c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:30:40.058 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:40.055+0000 7feb16849640 1 -- --> v1:172.21.15.203:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7feb100d2570 con 0x7feb1008cc90 2024-04-25T10:30:40.058 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:40.055+0000 7feb16849640 1 -- --> v2:172.21.15.37:3300/0 -- mon_getmap magic: 0 -- 0x7feb1009d2d0 con 0x7feb100b71b0 2024-04-25T10:30:40.058 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:40.055+0000 7feb16849640 1 -- --> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] -- mon_getmap magic: 0 -- 0x7feb100b8d40 con 0x7feb100d1c20 2024-04-25T10:30:40.058 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:40.055+0000 7feb14de6640 1 --2- >> v2:172.21.15.37:3300/0 conn(0x7feb100b71b0 0x7feb100b75c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:30:40.058 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:40.055+0000 7feb14de6640 1 --2- >> v2:172.21.15.37:3300/0 conn(0x7feb100b71b0 0x7feb100b75c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.37:3300/0 says I am v2:172.21.15.37:58148/0 (socket says 172.21.15.37:58148) 2024-04-25T10:30:40.058 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:40.055+0000 7feb14de6640 1 -- 172.21.15.37:0/3320653189 learned_addr learned my addr 172.21.15.37:0/3320653189 (peer_addr_for_me v2:172.21.15.37:0/0) 2024-04-25T10:30:40.058 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:40.055+0000 7feb0f7fe640 1 --2- 172.21.15.37:0/3320653189 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7feb100d1c20 0x7feb100d2030 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:30:40.058 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:40.056+0000 7feb14de6640 1 -- 172.21.15.37:0/3320653189 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7feb100d1c20 msgr2=0x7feb100d2030 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:30:40.058 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:40.056+0000 7feb14de6640 1 --2- 172.21.15.37:0/3320653189 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7feb100d1c20 0x7feb100d2030 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:30:40.059 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:40.056+0000 7feb14de6640 1 -- 172.21.15.37:0/3320653189 >> v1:172.21.15.203:6789/0 conn(0x7feb1008cc90 legacy=0x7feb1009cbc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:30:40.059 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:40.056+0000 7feb14de6640 1 -- 172.21.15.37:0/3320653189 --> v2:172.21.15.37:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7feb100d27f0 con 0x7feb100b71b0 2024-04-25T10:30:40.060 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:40.056+0000 7feb14de6640 1 --2- 172.21.15.37:0/3320653189 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7feb100b71b0 0x7feb100b75c0 secure :-1 s=READY pgs=579 cs=0 l=1 rev1=1 crypto rx=0x7feb08009e50 tx=0x7feb0802a6c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=2c717e1dec587f8f server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:30:40.060 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:40.056+0000 7feb0effd640 1 -- 172.21.15.37:0/3320653189 <== mon.0 v2:172.21.15.37:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7feb08036070 con 0x7feb100b71b0 2024-04-25T10:30:40.060 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:40.056+0000 7feb0effd640 1 -- 172.21.15.37:0/3320653189 <== mon.0 v2:172.21.15.37:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7feb08031600 con 0x7feb100b71b0 2024-04-25T10:30:40.060 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:40.056+0000 7feb0effd640 1 -- 172.21.15.37:0/3320653189 <== mon.0 v2:172.21.15.37:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7feb080319d0 con 0x7feb100b71b0 2024-04-25T10:30:40.060 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:40.057+0000 7feb16849640 1 -- 172.21.15.37:0/3320653189 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7feb100b71b0 msgr2=0x7feb100b75c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:30:40.060 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:40.057+0000 7feb16849640 1 --2- 172.21.15.37:0/3320653189 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7feb100b71b0 0x7feb100b75c0 secure :-1 s=READY pgs=579 cs=0 l=1 rev1=1 crypto rx=0x7feb08009e50 tx=0x7feb0802a6c0 comp rx=0 tx=0).stop 2024-04-25T10:30:40.060 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:40.057+0000 7feb16849640 1 -- 172.21.15.37:0/3320653189 shutdown_connections 2024-04-25T10:30:40.060 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:40.057+0000 7feb16849640 1 --2- 172.21.15.37:0/3320653189 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7feb100b71b0 0x7feb100b75c0 unknown :-1 s=CLOSED pgs=579 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:30:40.060 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:40.057+0000 7feb16849640 1 --2- 172.21.15.37:0/3320653189 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7feb100d1c20 0x7feb100d2030 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:30:40.060 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:40.057+0000 7feb16849640 1 -- 172.21.15.37:0/3320653189 >> 172.21.15.37:0/3320653189 conn(0x7feb100d03d0 msgr2=0x7feb100d07e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:30:40.060 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:40.057+0000 7feb16849640 1 -- 172.21.15.37:0/3320653189 shutdown_connections 2024-04-25T10:30:40.061 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:40.057+0000 7feb16849640 1 -- 172.21.15.37:0/3320653189 wait complete. 2024-04-25T10:30:40.061 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:40.059+0000 7feb16849640 1 Processor -- start 2024-04-25T10:30:40.061 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:40.059+0000 7feb16849640 1 -- start start 2024-04-25T10:30:40.064 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:40.060+0000 7feb16849640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7feb100b71b0 0x7feb100a0f00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:30:40.064 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:40.061+0000 7feb16849640 1 --2- >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7feb100d1c20 0x7feb100ca400 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:30:40.064 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:40.061+0000 7feb16849640 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7feb101d9400 0x7feb1009eea0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:30:40.064 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:40.061+0000 7feb16849640 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 -- 0x7feb100b93b0 con 0x7feb100b71b0 2024-04-25T10:30:40.064 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:40.061+0000 7feb16849640 1 -- --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_getmap magic: 0 -- 0x7feb100b8fd0 con 0x7feb101d9400 2024-04-25T10:30:40.065 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:40.061+0000 7feb16849640 1 -- --> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] -- mon_getmap magic: 0 -- 0x7feb100b7b00 con 0x7feb100d1c20 2024-04-25T10:30:40.065 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:40.061+0000 7feb0ffff640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7feb100b71b0 0x7feb100a0f00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:30:40.065 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:40.061+0000 7feb0ffff640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7feb100b71b0 0x7feb100a0f00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.37:3300/0 says I am v2:172.21.15.37:58160/0 (socket says 172.21.15.37:58160) 2024-04-25T10:30:40.065 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:40.061+0000 7feb0ffff640 1 -- 172.21.15.37:0/3661557760 learned_addr learned my addr 172.21.15.37:0/3661557760 (peer_addr_for_me v2:172.21.15.37:0/0) 2024-04-25T10:30:40.065 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:40.061+0000 7feb0f7fe640 1 --2- 172.21.15.37:0/3661557760 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7feb100d1c20 0x7feb100ca400 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:30:40.065 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:40.061+0000 7feb14de6640 1 --2- 172.21.15.37:0/3661557760 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7feb101d9400 0x7feb1009eea0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:30:40.065 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:40.061+0000 7feb0ffff640 1 -- 172.21.15.37:0/3661557760 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7feb100d1c20 msgr2=0x7feb100ca400 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:30:40.065 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:40.061+0000 7feb0ffff640 1 --2- 172.21.15.37:0/3661557760 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7feb100d1c20 0x7feb100ca400 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:30:40.065 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:40.061+0000 7feb0ffff640 1 -- 172.21.15.37:0/3661557760 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7feb101d9400 msgr2=0x7feb1009eea0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:30:40.065 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:40.061+0000 7feb0ffff640 1 --2- 172.21.15.37:0/3661557760 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7feb101d9400 0x7feb1009eea0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:30:40.065 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:40.061+0000 7feb0ffff640 1 -- 172.21.15.37:0/3661557760 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7feb1009d820 con 0x7feb100b71b0 2024-04-25T10:30:40.065 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:40.061+0000 7feb0ffff640 1 --2- 172.21.15.37:0/3661557760 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7feb100b71b0 0x7feb100a0f00 secure :-1 s=READY pgs=580 cs=0 l=1 rev1=1 crypto rx=0x7feb00009fd0 tx=0x7feb0000b870 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:30:40.065 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:40.061+0000 7feb0cff9640 1 -- 172.21.15.37:0/3661557760 <== mon.0 v2:172.21.15.37:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7feb00015020 con 0x7feb100b71b0 2024-04-25T10:30:40.065 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:40.061+0000 7feb0cff9640 1 -- 172.21.15.37:0/3661557760 <== mon.0 v2:172.21.15.37:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7feb00011830 con 0x7feb100b71b0 2024-04-25T10:30:40.066 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:40.061+0000 7feb16849640 1 -- 172.21.15.37:0/3661557760 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7feb1009db00 con 0x7feb100b71b0 2024-04-25T10:30:40.066 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:40.061+0000 7feb0cff9640 1 -- 172.21.15.37:0/3661557760 <== mon.0 v2:172.21.15.37:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7feb00011c00 con 0x7feb100b71b0 2024-04-25T10:30:40.066 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:40.062+0000 7feb16849640 1 -- 172.21.15.37:0/3661557760 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7feb1009e0c0 con 0x7feb100b71b0 2024-04-25T10:30:40.066 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:40.062+0000 7feb16849640 1 -- 172.21.15.37:0/3661557760 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7feb1006a140 con 0x7feb100b71b0 2024-04-25T10:30:40.067 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:40.063+0000 7feb0cff9640 1 -- 172.21.15.37:0/3661557760 <== mon.0 v2:172.21.15.37:3300/0 4 ==== mgrmap(e 5) ==== 96782+0+0 (secure 0 0 0) 0x7feb00018020 con 0x7feb100b71b0 2024-04-25T10:30:40.067 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:40.064+0000 7feb0cff9640 1 --2- 172.21.15.37:0/3661557760 >> [v2:172.21.15.203:6832/2418205237,v1:172.21.15.203:6833/2418205237] conn(0x7feafc0744b0 0x7feafc0769c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:30:40.067 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:40.064+0000 7feb0cff9640 1 -- 172.21.15.37:0/3661557760 <== mon.0 v2:172.21.15.37:3300/0 5 ==== osd_map(44..44 src has 1..44) ==== 6248+0+0 (secure 0 0 0) 0x7feb00095210 con 0x7feb100b71b0 2024-04-25T10:30:40.067 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:40.064+0000 7feb0f7fe640 1 --2- 172.21.15.37:0/3661557760 >> [v2:172.21.15.203:6832/2418205237,v1:172.21.15.203:6833/2418205237] conn(0x7feafc0744b0 0x7feafc0769c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:30:40.067 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:40.064+0000 7feb0f7fe640 1 --2- 172.21.15.37:0/3661557760 >> [v2:172.21.15.203:6832/2418205237,v1:172.21.15.203:6833/2418205237] conn(0x7feafc0744b0 0x7feafc0769c0 secure :-1 s=READY pgs=271 cs=0 l=1 rev1=1 crypto rx=0x7feb100c6f10 tx=0x7feb04008040 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:30:40.070 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:40.068+0000 7feb0cff9640 1 -- 172.21.15.37:0/3661557760 <== mon.0 v2:172.21.15.37:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187246 (secure 0 0 0) 0x7feb0001f020 con 0x7feb100b71b0 2024-04-25T10:30:40.293 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:40.291+0000 7feb16849640 1 -- 172.21.15.37:0/3661557760 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7feb100cf320 con 0x7feb100b71b0 2024-04-25T10:30:40.294 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:40.292+0000 7feb0cff9640 1 -- 172.21.15.37:0/3661557760 <== mon.0 v2:172.21.15.37:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 24 v24) ==== 94+0+6482 (secure 0 0 0) 0x7feb0005f130 con 0x7feb100b71b0 2024-04-25T10:30:40.294 INFO:teuthology.orchestra.run.smithi037.stdout: 2024-04-25T10:30:40.294 INFO:teuthology.orchestra.run.smithi037.stdout:{"epoch":24,"default_fscid":2,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":4946,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.37:6835/683655741","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.37:6834","nonce":683655741},{"type":"v1","addr":"172.21.15.37:6835","nonce":683655741}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":19},{"gid":5215,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.203:6836/3927045001","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6835","nonce":3927045001},{"type":"v1","addr":"172.21.15.203:6836","nonce":3927045001}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":19},{"gid":5218,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.203:6837/3108183946","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6834","nonce":3108183946},{"type":"v1","addr":"172.21.15.203:6837","nonce":3108183946}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":19}],"filesystems":[{"mdsmap":{"epoch":18,"flags":19,"flags_state":{"joinable":false,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-04-25T10:30:12.130473+0000","modified":"2024-04-25T10:30:24.853915+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":37,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":1,"in":[0],"up":{},"failed":[0],"damaged":[],"stopped":[],"info":{},"data_pools":[5],"metadata_pool":4,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":0,"qdb_cluster":[]},"id":2},{"mdsmap":{"epoch":24,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-04-25T10:30:28.588617+0000","modified":"2024-04-25T10:30:39.919885+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":1,"in":[0],"up":{"mds_0":5625},"failed":[],"damaged":[],"stopped":[],"info":{"gid_5625":{"gid":5625,"name":"c","rank":0,"incarnation":24,"state":"up:replay","state_seq":1,"addr":"172.21.15.37:6833/2353203722","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.37:6832","nonce":2353203722},{"type":"v1","addr":"172.21.15.37:6833","nonce":2353203722}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[5],"metadata_pool":6,"enabled":true,"fs_name":"recovery_fs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":0,"qdb_cluster":[]},"id":3}]} 2024-04-25T10:30:40.295 INFO:teuthology.orchestra.run.smithi037.stderr:dumped fsmap epoch 24 2024-04-25T10:30:40.296 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:40.294+0000 7feb16849640 1 -- 172.21.15.37:0/3661557760 >> [v2:172.21.15.203:6832/2418205237,v1:172.21.15.203:6833/2418205237] conn(0x7feafc0744b0 msgr2=0x7feafc0769c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:30:40.296 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:40.294+0000 7feb16849640 1 --2- 172.21.15.37:0/3661557760 >> [v2:172.21.15.203:6832/2418205237,v1:172.21.15.203:6833/2418205237] conn(0x7feafc0744b0 0x7feafc0769c0 secure :-1 s=READY pgs=271 cs=0 l=1 rev1=1 crypto rx=0x7feb100c6f10 tx=0x7feb04008040 comp rx=0 tx=0).stop 2024-04-25T10:30:40.296 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:40.294+0000 7feb16849640 1 -- 172.21.15.37:0/3661557760 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7feb100b71b0 msgr2=0x7feb100a0f00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:30:40.296 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:40.294+0000 7feb16849640 1 --2- 172.21.15.37:0/3661557760 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7feb100b71b0 0x7feb100a0f00 secure :-1 s=READY pgs=580 cs=0 l=1 rev1=1 crypto rx=0x7feb00009fd0 tx=0x7feb0000b870 comp rx=0 tx=0).stop 2024-04-25T10:30:40.296 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:40.294+0000 7feb16849640 1 -- 172.21.15.37:0/3661557760 shutdown_connections 2024-04-25T10:30:40.296 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:40.294+0000 7feb16849640 1 --2- 172.21.15.37:0/3661557760 >> [v2:172.21.15.203:6832/2418205237,v1:172.21.15.203:6833/2418205237] conn(0x7feafc0744b0 0x7feafc0769c0 unknown :-1 s=CLOSED pgs=271 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:30:40.297 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:40.294+0000 7feb16849640 1 --2- 172.21.15.37:0/3661557760 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7feb101d9400 0x7feb1009eea0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:30:40.297 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:40.294+0000 7feb16849640 1 --2- 172.21.15.37:0/3661557760 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7feb100d1c20 0x7feb100ca400 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:30:40.297 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:40.294+0000 7feb16849640 1 --2- 172.21.15.37:0/3661557760 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7feb100b71b0 0x7feb100a0f00 unknown :-1 s=CLOSED pgs=580 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:30:40.297 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:40.294+0000 7feb16849640 1 -- 172.21.15.37:0/3661557760 >> 172.21.15.37:0/3661557760 conn(0x7feb100d03d0 msgr2=0x7feb100cc9d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:30:40.297 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:40.294+0000 7feb16849640 1 -- 172.21.15.37:0/3661557760 shutdown_connections 2024-04-25T10:30:40.297 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:40.294+0000 7feb16849640 1 -- 172.21.15.37:0/3661557760 wait complete. 2024-04-25T10:30:40.710 DEBUG:tasks.cephfs.filesystem:are_daemons_healthy: mds map: {'epoch': 24, 'flags': 18, 'flags_state': {'joinable': True, 'allow_snaps': True, 'allow_multimds_snaps': True, 'allow_standby_replay': False, 'refuse_client_session': False, 'refuse_standby_for_another_fs': False, 'balance_automate': False}, 'ever_allowed_features': 0, 'explicitly_allowed_features': 0, 'created': '2024-04-25T10:30:28.588617+0000', 'modified': '2024-04-25T10:30:39.919885+0000', 'tableserver': 0, 'root': 0, 'session_timeout': 60, 'session_autoclose': 300, 'required_client_features': {}, 'max_file_size': 1099511627776, 'max_xattr_size': 65536, 'last_failure': 0, 'last_failure_osd_epoch': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments', 'feature_12': 'quiesce subvolumes'}}, 'max_mds': 1, 'in': [0], 'up': {'mds_0': 5625}, 'failed': [], 'damaged': [], 'stopped': [], 'info': {'gid_5625': {'gid': 5625, 'name': 'c', 'rank': 0, 'incarnation': 24, 'state': 'up:replay', 'state_seq': 1, 'addr': '172.21.15.37:6833/2353203722', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.37:6832', 'nonce': 2353203722}, {'type': 'v1', 'addr': '172.21.15.37:6833', 'nonce': 2353203722}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540701547738038271, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments', 'feature_12': 'quiesce subvolumes'}}}}, 'data_pools': [5], 'metadata_pool': 6, 'enabled': True, 'fs_name': 'recovery_fs', 'balancer': '', 'bal_rank_mask': '-1', 'standby_count_wanted': 0, 'qdb_leader': 0, 'qdb_cluster': []} 2024-04-25T10:30:40.710 WARNING:tasks.cephfs.filesystem:Unhealthy mds state gid_5625:up:replay 2024-04-25T10:30:41.712 DEBUG:teuthology.orchestra.run.smithi037:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-04-25T10:30:41.838 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:41.836+0000 7f3662851640 1 Processor -- start 2024-04-25T10:30:41.838 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:41.836+0000 7f3662851640 1 -- start start 2024-04-25T10:30:41.840 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:41.837+0000 7f3662851640 1 --2- >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f365c0b7600 0x7f365c0b7a10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:30:41.840 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:41.837+0000 7f3662851640 1 --2- >> v2:172.21.15.37:3300/0 conn(0x7f365c0d4560 0x7f365c0d4970 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:30:41.840 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:41.837+0000 7f3662851640 1 -- --> v1:172.21.15.203:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f365c0b7f50 con 0x7f365c0b4bb0 2024-04-25T10:30:41.840 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:41.837+0000 7f3662851640 1 -- --> v2:172.21.15.37:3300/0 -- mon_getmap magic: 0 -- 0x7f365c161190 con 0x7f365c0d4560 2024-04-25T10:30:41.840 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:41.837+0000 7f3662851640 1 -- --> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] -- mon_getmap magic: 0 -- 0x7f365c161470 con 0x7f365c0b7600 2024-04-25T10:30:41.840 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:41.838+0000 7f3660dee640 1 --2- >> v2:172.21.15.37:3300/0 conn(0x7f365c0d4560 0x7f365c0d4970 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:30:41.840 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:41.838+0000 7f3660dee640 1 --2- >> v2:172.21.15.37:3300/0 conn(0x7f365c0d4560 0x7f365c0d4970 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.37:3300/0 says I am v2:172.21.15.37:58168/0 (socket says 172.21.15.37:58168) 2024-04-25T10:30:41.840 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:41.838+0000 7f3660dee640 1 -- 172.21.15.37:0/607185365 learned_addr learned my addr 172.21.15.37:0/607185365 (peer_addr_for_me v2:172.21.15.37:0/0) 2024-04-25T10:30:41.840 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:41.838+0000 7f365b7fe640 1 --2- 172.21.15.37:0/607185365 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f365c0b7600 0x7f365c0b7a10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:30:41.841 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:41.838+0000 7f3660dee640 1 -- 172.21.15.37:0/607185365 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f365c0b7600 msgr2=0x7f365c0b7a10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:30:41.841 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:41.838+0000 7f3660dee640 1 --2- 172.21.15.37:0/607185365 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f365c0b7600 0x7f365c0b7a10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:30:41.841 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:41.838+0000 7f3660dee640 1 -- 172.21.15.37:0/607185365 >> v1:172.21.15.203:6789/0 conn(0x7f365c0b4bb0 legacy=0x7f365c1646b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:30:41.841 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:41.838+0000 7f3660dee640 1 -- 172.21.15.37:0/607185365 --> v2:172.21.15.37:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f365c163eb0 con 0x7f365c0d4560 2024-04-25T10:30:41.842 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:41.839+0000 7f3660dee640 1 --2- 172.21.15.37:0/607185365 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f365c0d4560 0x7f365c0d4970 secure :-1 s=READY pgs=581 cs=0 l=1 rev1=1 crypto rx=0x7f3654009e50 tx=0x7f365402a710 comp rx=0 tx=0).ready entity=mon.0 client_cookie=3da9f8f6cd891ef3 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:30:41.842 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:41.839+0000 7f365affd640 1 -- 172.21.15.37:0/607185365 <== mon.0 v2:172.21.15.37:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3654036070 con 0x7f365c0d4560 2024-04-25T10:30:41.842 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:41.839+0000 7f365affd640 1 -- 172.21.15.37:0/607185365 <== mon.0 v2:172.21.15.37:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f3654031600 con 0x7f365c0d4560 2024-04-25T10:30:41.842 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:41.839+0000 7f365affd640 1 -- 172.21.15.37:0/607185365 <== mon.0 v2:172.21.15.37:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f36540319d0 con 0x7f365c0d4560 2024-04-25T10:30:41.842 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:41.840+0000 7f3662851640 1 -- 172.21.15.37:0/607185365 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f365c0d4560 msgr2=0x7f365c0d4970 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:30:41.842 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:41.840+0000 7f3662851640 1 --2- 172.21.15.37:0/607185365 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f365c0d4560 0x7f365c0d4970 secure :-1 s=READY pgs=581 cs=0 l=1 rev1=1 crypto rx=0x7f3654009e50 tx=0x7f365402a710 comp rx=0 tx=0).stop 2024-04-25T10:30:41.842 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:41.840+0000 7f3662851640 1 -- 172.21.15.37:0/607185365 shutdown_connections 2024-04-25T10:30:41.843 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:41.840+0000 7f3662851640 1 --2- 172.21.15.37:0/607185365 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f365c0d4560 0x7f365c0d4970 unknown :-1 s=CLOSED pgs=581 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:30:41.843 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:41.840+0000 7f3662851640 1 --2- 172.21.15.37:0/607185365 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f365c0b7600 0x7f365c0b7a10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:30:41.843 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:41.840+0000 7f3662851640 1 -- 172.21.15.37:0/607185365 >> 172.21.15.37:0/607185365 conn(0x7f365c0d27e0 msgr2=0x7f365c0d2bf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:30:41.843 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:41.840+0000 7f3662851640 1 -- 172.21.15.37:0/607185365 shutdown_connections 2024-04-25T10:30:41.843 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:41.840+0000 7f3662851640 1 -- 172.21.15.37:0/607185365 wait complete. 2024-04-25T10:30:41.843 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:41.841+0000 7f3662851640 1 Processor -- start 2024-04-25T10:30:41.844 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:41.842+0000 7f3662851640 1 -- start start 2024-04-25T10:30:41.847 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:41.843+0000 7f3662851640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f365c0b4bb0 0x7f365c0ad460 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:30:41.847 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:41.843+0000 7f3662851640 1 --2- >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f365c0b7600 0x7f365c1563b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:30:41.847 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:41.843+0000 7f3662851640 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f365c0d4560 0x7f365c0ce710 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:30:41.847 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:41.843+0000 7f3662851640 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 -- 0x7f365c0d09e0 con 0x7f365c0b4bb0 2024-04-25T10:30:41.847 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:41.843+0000 7f3662851640 1 -- --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_getmap magic: 0 -- 0x7f365c0b4fc0 con 0x7f365c0d4560 2024-04-25T10:30:41.847 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:41.843+0000 7f3662851640 1 -- --> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] -- mon_getmap magic: 0 -- 0x7f365c0d0b60 con 0x7f365c0b7600 2024-04-25T10:30:41.847 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:41.843+0000 7f365bfff640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f365c0b4bb0 0x7f365c0ad460 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:30:41.847 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:41.843+0000 7f365bfff640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f365c0b4bb0 0x7f365c0ad460 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.37:3300/0 says I am v2:172.21.15.37:58184/0 (socket says 172.21.15.37:58184) 2024-04-25T10:30:41.847 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:41.843+0000 7f365bfff640 1 -- 172.21.15.37:0/40488706 learned_addr learned my addr 172.21.15.37:0/40488706 (peer_addr_for_me v2:172.21.15.37:0/0) 2024-04-25T10:30:41.847 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:41.843+0000 7f365b7fe640 1 --2- 172.21.15.37:0/40488706 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f365c0b7600 0x7f365c1563b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:30:41.847 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:41.843+0000 7f3660dee640 1 --2- 172.21.15.37:0/40488706 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f365c0d4560 0x7f365c0ce710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:30:41.847 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:41.844+0000 7f365bfff640 1 -- 172.21.15.37:0/40488706 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f365c0b7600 msgr2=0x7f365c1563b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:30:41.847 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:41.844+0000 7f365bfff640 1 --2- 172.21.15.37:0/40488706 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f365c0b7600 0x7f365c1563b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:30:41.848 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:41.844+0000 7f365bfff640 1 -- 172.21.15.37:0/40488706 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f365c0d4560 msgr2=0x7f365c0ce710 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:30:41.848 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:41.844+0000 7f365bfff640 1 --2- 172.21.15.37:0/40488706 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f365c0d4560 0x7f365c0ce710 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:30:41.848 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:41.844+0000 7f365bfff640 1 -- 172.21.15.37:0/40488706 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f365c0ac810 con 0x7f365c0b4bb0 2024-04-25T10:30:41.848 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:41.844+0000 7f365b7fe640 1 --2- 172.21.15.37:0/40488706 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f365c0b7600 0x7f365c1563b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T10:30:41.848 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:41.844+0000 7f365bfff640 1 --2- 172.21.15.37:0/40488706 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f365c0b4bb0 0x7f365c0ad460 secure :-1 s=READY pgs=582 cs=0 l=1 rev1=1 crypto rx=0x7f364c00c470 tx=0x7f364c00c930 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:30:41.848 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:41.844+0000 7f3658ff9640 1 -- 172.21.15.37:0/40488706 <== mon.0 v2:172.21.15.37:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f364c0052d0 con 0x7f365c0b4bb0 2024-04-25T10:30:41.848 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:41.844+0000 7f3658ff9640 1 -- 172.21.15.37:0/40488706 <== mon.0 v2:172.21.15.37:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f364c005490 con 0x7f365c0b4bb0 2024-04-25T10:30:41.848 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:41.844+0000 7f3658ff9640 1 -- 172.21.15.37:0/40488706 <== mon.0 v2:172.21.15.37:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f364c007260 con 0x7f365c0b4bb0 2024-04-25T10:30:41.848 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:41.844+0000 7f3662851640 1 -- 172.21.15.37:0/40488706 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f365c0cb6e0 con 0x7f365c0b4bb0 2024-04-25T10:30:41.848 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:41.844+0000 7f3662851640 1 -- 172.21.15.37:0/40488706 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f365c0cbca0 con 0x7f365c0b4bb0 2024-04-25T10:30:41.849 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:41.845+0000 7f3662851640 1 -- 172.21.15.37:0/40488706 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f365c06a140 con 0x7f365c0b4bb0 2024-04-25T10:30:41.849 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:41.846+0000 7f3658ff9640 1 -- 172.21.15.37:0/40488706 <== mon.0 v2:172.21.15.37:3300/0 4 ==== mgrmap(e 5) ==== 96782+0+0 (secure 0 0 0) 0x7f364c007400 con 0x7f365c0b4bb0 2024-04-25T10:30:41.849 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:41.846+0000 7f3658ff9640 1 --2- 172.21.15.37:0/40488706 >> [v2:172.21.15.203:6832/2418205237,v1:172.21.15.203:6833/2418205237] conn(0x7f363c0744e0 0x7f363c0769f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:30:41.849 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:41.846+0000 7f3658ff9640 1 -- 172.21.15.37:0/40488706 <== mon.0 v2:172.21.15.37:3300/0 5 ==== osd_map(44..44 src has 1..44) ==== 6248+0+0 (secure 0 0 0) 0x7f364c094920 con 0x7f365c0b4bb0 2024-04-25T10:30:41.849 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:41.846+0000 7f365b7fe640 1 --2- 172.21.15.37:0/40488706 >> [v2:172.21.15.203:6832/2418205237,v1:172.21.15.203:6833/2418205237] conn(0x7f363c0744e0 0x7f363c0769f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:30:41.849 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:41.847+0000 7f365b7fe640 1 --2- 172.21.15.37:0/40488706 >> [v2:172.21.15.203:6832/2418205237,v1:172.21.15.203:6833/2418205237] conn(0x7f363c0744e0 0x7f363c0769f0 secure :-1 s=READY pgs=272 cs=0 l=1 rev1=1 crypto rx=0x7f365c160820 tx=0x7f3644008040 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:30:41.853 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:41.851+0000 7f3658ff9640 1 -- 172.21.15.37:0/40488706 <== mon.0 v2:172.21.15.37:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187246 (secure 0 0 0) 0x7f365c0b7f50 con 0x7f365c0b4bb0 2024-04-25T10:30:42.079 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:42.077+0000 7f3662851640 1 -- 172.21.15.37:0/40488706 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f365c0d3ff0 con 0x7f365c0b4bb0 2024-04-25T10:30:42.079 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:42.077+0000 7f3658ff9640 1 -- 172.21.15.37:0/40488706 <== mon.0 v2:172.21.15.37:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 26 v26) ==== 94+0+6482 (secure 0 0 0) 0x7f364c05ee30 con 0x7f365c0b4bb0 2024-04-25T10:30:42.080 INFO:teuthology.orchestra.run.smithi037.stdout: 2024-04-25T10:30:42.080 INFO:teuthology.orchestra.run.smithi037.stdout:{"epoch":26,"default_fscid":2,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":4946,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.37:6835/683655741","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.37:6834","nonce":683655741},{"type":"v1","addr":"172.21.15.37:6835","nonce":683655741}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":19},{"gid":5215,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.203:6836/3927045001","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6835","nonce":3927045001},{"type":"v1","addr":"172.21.15.203:6836","nonce":3927045001}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":19},{"gid":5218,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.203:6837/3108183946","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6834","nonce":3108183946},{"type":"v1","addr":"172.21.15.203:6837","nonce":3108183946}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":19}],"filesystems":[{"mdsmap":{"epoch":18,"flags":19,"flags_state":{"joinable":false,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-04-25T10:30:12.130473+0000","modified":"2024-04-25T10:30:24.853915+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":37,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":1,"in":[0],"up":{},"failed":[0],"damaged":[],"stopped":[],"info":{},"data_pools":[5],"metadata_pool":4,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":0,"qdb_cluster":[]},"id":2},{"mdsmap":{"epoch":26,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-04-25T10:30:28.588617+0000","modified":"2024-04-25T10:30:40.929130+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":1,"in":[0],"up":{"mds_0":5625},"failed":[],"damaged":[],"stopped":[],"info":{"gid_5625":{"gid":5625,"name":"c","rank":0,"incarnation":24,"state":"up:rejoin","state_seq":6,"addr":"172.21.15.37:6833/2353203722","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.37:6832","nonce":2353203722},{"type":"v1","addr":"172.21.15.37:6833","nonce":2353203722}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[5],"metadata_pool":6,"enabled":true,"fs_name":"recovery_fs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":0,"qdb_cluster":[]},"id":3}]} 2024-04-25T10:30:42.080 INFO:teuthology.orchestra.run.smithi037.stderr:dumped fsmap epoch 26 2024-04-25T10:30:42.082 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:42.079+0000 7f3662851640 1 -- 172.21.15.37:0/40488706 >> [v2:172.21.15.203:6832/2418205237,v1:172.21.15.203:6833/2418205237] conn(0x7f363c0744e0 msgr2=0x7f363c0769f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:30:42.082 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:42.079+0000 7f3662851640 1 --2- 172.21.15.37:0/40488706 >> [v2:172.21.15.203:6832/2418205237,v1:172.21.15.203:6833/2418205237] conn(0x7f363c0744e0 0x7f363c0769f0 secure :-1 s=READY pgs=272 cs=0 l=1 rev1=1 crypto rx=0x7f365c160820 tx=0x7f3644008040 comp rx=0 tx=0).stop 2024-04-25T10:30:42.082 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:42.079+0000 7f3662851640 1 -- 172.21.15.37:0/40488706 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f365c0b4bb0 msgr2=0x7f365c0ad460 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:30:42.082 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:42.079+0000 7f3662851640 1 --2- 172.21.15.37:0/40488706 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f365c0b4bb0 0x7f365c0ad460 secure :-1 s=READY pgs=582 cs=0 l=1 rev1=1 crypto rx=0x7f364c00c470 tx=0x7f364c00c930 comp rx=0 tx=0).stop 2024-04-25T10:30:42.082 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:42.079+0000 7f3662851640 1 -- 172.21.15.37:0/40488706 shutdown_connections 2024-04-25T10:30:42.082 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:42.079+0000 7f3662851640 1 --2- 172.21.15.37:0/40488706 >> [v2:172.21.15.203:6832/2418205237,v1:172.21.15.203:6833/2418205237] conn(0x7f363c0744e0 0x7f363c0769f0 unknown :-1 s=CLOSED pgs=272 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:30:42.083 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:42.079+0000 7f3662851640 1 --2- 172.21.15.37:0/40488706 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f365c0d4560 0x7f365c0ce710 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:30:42.083 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:42.079+0000 7f3662851640 1 --2- 172.21.15.37:0/40488706 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f365c0b7600 0x7f365c1563b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:30:42.083 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:42.079+0000 7f3662851640 1 --2- 172.21.15.37:0/40488706 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f365c0b4bb0 0x7f365c0ad460 unknown :-1 s=CLOSED pgs=582 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:30:42.083 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:42.079+0000 7f3662851640 1 -- 172.21.15.37:0/40488706 >> 172.21.15.37:0/40488706 conn(0x7f365c0d27e0 msgr2=0x7f365c0b5ff0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:30:42.083 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:42.079+0000 7f3662851640 1 -- 172.21.15.37:0/40488706 shutdown_connections 2024-04-25T10:30:42.083 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:42.079+0000 7f3662851640 1 -- 172.21.15.37:0/40488706 wait complete. 2024-04-25T10:30:42.495 DEBUG:tasks.cephfs.filesystem:are_daemons_healthy: mds map: {'epoch': 26, 'flags': 18, 'flags_state': {'joinable': True, 'allow_snaps': True, 'allow_multimds_snaps': True, 'allow_standby_replay': False, 'refuse_client_session': False, 'refuse_standby_for_another_fs': False, 'balance_automate': False}, 'ever_allowed_features': 0, 'explicitly_allowed_features': 0, 'created': '2024-04-25T10:30:28.588617+0000', 'modified': '2024-04-25T10:30:40.929130+0000', 'tableserver': 0, 'root': 0, 'session_timeout': 60, 'session_autoclose': 300, 'required_client_features': {}, 'max_file_size': 1099511627776, 'max_xattr_size': 65536, 'last_failure': 0, 'last_failure_osd_epoch': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments', 'feature_12': 'quiesce subvolumes'}}, 'max_mds': 1, 'in': [0], 'up': {'mds_0': 5625}, 'failed': [], 'damaged': [], 'stopped': [], 'info': {'gid_5625': {'gid': 5625, 'name': 'c', 'rank': 0, 'incarnation': 24, 'state': 'up:rejoin', 'state_seq': 6, 'addr': '172.21.15.37:6833/2353203722', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.37:6832', 'nonce': 2353203722}, {'type': 'v1', 'addr': '172.21.15.37:6833', 'nonce': 2353203722}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540701547738038271, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments', 'feature_12': 'quiesce subvolumes'}}}}, 'data_pools': [5], 'metadata_pool': 6, 'enabled': True, 'fs_name': 'recovery_fs', 'balancer': '', 'bal_rank_mask': '-1', 'standby_count_wanted': 0, 'qdb_leader': 0, 'qdb_cluster': []} 2024-04-25T10:30:42.496 WARNING:tasks.cephfs.filesystem:Unhealthy mds state gid_5625:up:rejoin 2024-04-25T10:30:43.496 DEBUG:teuthology.orchestra.run.smithi037:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-04-25T10:30:43.622 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:43.620+0000 7f2d1884c640 1 Processor -- start 2024-04-25T10:30:43.622 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:43.620+0000 7f2d1884c640 1 -- start start 2024-04-25T10:30:43.624 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:43.622+0000 7f2d1884c640 1 --2- >> v2:172.21.15.203:3300/0 conn(0x7f2d140b4bb0 0x7f2d141646b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:30:43.624 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:43.622+0000 7f2d1884c640 1 --2- >> v2:172.21.15.37:3300/0 conn(0x7f2d140b7600 0x7f2d140b7a10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:30:43.624 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:43.622+0000 7f2d1884c640 1 -- --> v1:172.21.15.203:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f2d140b7f50 con 0x7f2d140d4560 2024-04-25T10:30:43.624 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:43.622+0000 7f2d1884c640 1 -- --> v2:172.21.15.37:3300/0 -- mon_getmap magic: 0 -- 0x7f2d140d5080 con 0x7f2d140b7600 2024-04-25T10:30:43.624 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:43.622+0000 7f2d1884c640 1 -- --> v2:172.21.15.203:3300/0 -- mon_getmap magic: 0 -- 0x7f2d14161190 con 0x7f2d140b4bb0 2024-04-25T10:30:43.624 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:43.622+0000 7f2d11d9b640 1 --2- >> v2:172.21.15.37:3300/0 conn(0x7f2d140b7600 0x7f2d140b7a10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:30:43.624 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:43.622+0000 7f2d11d9b640 1 --2- >> v2:172.21.15.37:3300/0 conn(0x7f2d140b7600 0x7f2d140b7a10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.37:3300/0 says I am v2:172.21.15.37:58198/0 (socket says 172.21.15.37:58198) 2024-04-25T10:30:43.624 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:43.622+0000 7f2d11d9b640 1 -- 172.21.15.37:0/2012345377 learned_addr learned my addr 172.21.15.37:0/2012345377 (peer_addr_for_me v2:172.21.15.37:0/0) 2024-04-25T10:30:43.625 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:43.622+0000 7f2d1259c640 1 --2- 172.21.15.37:0/2012345377 >> v2:172.21.15.203:3300/0 conn(0x7f2d140b4bb0 0x7f2d141646b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:30:43.625 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:43.622+0000 7f2d11d9b640 1 -- 172.21.15.37:0/2012345377 >> v2:172.21.15.203:3300/0 conn(0x7f2d140b4bb0 msgr2=0x7f2d141646b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:30:43.625 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:43.622+0000 7f2d11d9b640 1 --2- 172.21.15.37:0/2012345377 >> v2:172.21.15.203:3300/0 conn(0x7f2d140b4bb0 0x7f2d141646b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:30:43.625 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:43.622+0000 7f2d11d9b640 1 -- 172.21.15.37:0/2012345377 >> v1:172.21.15.203:6789/0 conn(0x7f2d140d4560 legacy=0x7f2d140d4970 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:30:43.625 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:43.622+0000 7f2d11d9b640 1 -- 172.21.15.37:0/2012345377 --> v2:172.21.15.37:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f2d14161470 con 0x7f2d140b7600 2024-04-25T10:30:43.626 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:43.622+0000 7f2d11d9b640 1 --2- 172.21.15.37:0/2012345377 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f2d140b7600 0x7f2d140b7a10 secure :-1 s=READY pgs=583 cs=0 l=1 rev1=1 crypto rx=0x7f2d1408e640 tx=0x7f2cfc02a270 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b3ed94c315cf8f1e server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:30:43.626 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:43.623+0000 7f2d1159a640 1 -- 172.21.15.37:0/2012345377 <== mon.0 v2:172.21.15.37:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2cfc037070 con 0x7f2d140b7600 2024-04-25T10:30:43.626 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:43.623+0000 7f2d1159a640 1 -- 172.21.15.37:0/2012345377 <== mon.0 v2:172.21.15.37:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f2cfc032440 con 0x7f2d140b7600 2024-04-25T10:30:43.626 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:43.623+0000 7f2d1159a640 1 -- 172.21.15.37:0/2012345377 <== mon.0 v2:172.21.15.37:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2cfc032810 con 0x7f2d140b7600 2024-04-25T10:30:43.626 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:43.623+0000 7f2d1884c640 1 -- 172.21.15.37:0/2012345377 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f2d140b7600 msgr2=0x7f2d140b7a10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:30:43.627 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:43.623+0000 7f2d1884c640 1 --2- 172.21.15.37:0/2012345377 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f2d140b7600 0x7f2d140b7a10 secure :-1 s=READY pgs=583 cs=0 l=1 rev1=1 crypto rx=0x7f2d1408e640 tx=0x7f2cfc02a270 comp rx=0 tx=0).stop 2024-04-25T10:30:43.627 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:43.623+0000 7f2d1884c640 1 -- 172.21.15.37:0/2012345377 shutdown_connections 2024-04-25T10:30:43.627 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:43.623+0000 7f2d1884c640 1 --2- 172.21.15.37:0/2012345377 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f2d140b7600 0x7f2d140b7a10 unknown :-1 s=CLOSED pgs=583 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:30:43.627 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:43.623+0000 7f2d1884c640 1 --2- 172.21.15.37:0/2012345377 >> v2:172.21.15.203:3300/0 conn(0x7f2d140b4bb0 0x7f2d141646b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:30:43.629 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:43.623+0000 7f2d1884c640 1 -- 172.21.15.37:0/2012345377 >> 172.21.15.37:0/2012345377 conn(0x7f2d140d27e0 msgr2=0x7f2d140d2bf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:30:43.629 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:43.623+0000 7f2d1884c640 1 -- 172.21.15.37:0/2012345377 shutdown_connections 2024-04-25T10:30:43.629 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:43.623+0000 7f2d1884c640 1 -- 172.21.15.37:0/2012345377 wait complete. 2024-04-25T10:30:43.631 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:43.625+0000 7f2d1884c640 1 Processor -- start 2024-04-25T10:30:43.632 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:43.625+0000 7f2d1884c640 1 -- start start 2024-04-25T10:30:43.632 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:43.627+0000 7f2d1884c640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f2d140b4bb0 0x7f2d14157e40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:30:43.632 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:43.627+0000 7f2d1884c640 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f2d140b7600 0x7f2d14158380 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:30:43.632 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:43.627+0000 7f2d1884c640 1 --2- >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f2d140d4560 0x7f2d141eac20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:30:43.632 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:43.627+0000 7f2d1884c640 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 -- 0x7f2d140b4fc0 con 0x7f2d140b4bb0 2024-04-25T10:30:43.632 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:43.627+0000 7f2d1884c640 1 -- --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_getmap magic: 0 -- 0x7f2d14164190 con 0x7f2d140b7600 2024-04-25T10:30:43.632 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:43.627+0000 7f2d1884c640 1 -- --> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] -- mon_getmap magic: 0 -- 0x7f2d14164bf0 con 0x7f2d140d4560 2024-04-25T10:30:43.632 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:43.627+0000 7f2d1259c640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f2d140b4bb0 0x7f2d14157e40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:30:43.632 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:43.627+0000 7f2d1259c640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f2d140b4bb0 0x7f2d14157e40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.37:3300/0 says I am v2:172.21.15.37:58214/0 (socket says 172.21.15.37:58214) 2024-04-25T10:30:43.632 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:43.627+0000 7f2d1259c640 1 -- 172.21.15.37:0/858386558 learned_addr learned my addr 172.21.15.37:0/858386558 (peer_addr_for_me v2:172.21.15.37:0/0) 2024-04-25T10:30:43.632 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:43.627+0000 7f2d11d9b640 1 --2- 172.21.15.37:0/858386558 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f2d140b7600 0x7f2d14158380 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:30:43.632 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:43.627+0000 7f2d12d9d640 1 --2- 172.21.15.37:0/858386558 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f2d140d4560 0x7f2d141eac20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:30:43.632 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:43.627+0000 7f2d1259c640 1 -- 172.21.15.37:0/858386558 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f2d140d4560 msgr2=0x7f2d141eac20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:30:43.633 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:43.627+0000 7f2d1259c640 1 --2- 172.21.15.37:0/858386558 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f2d140d4560 0x7f2d141eac20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:30:43.633 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:43.627+0000 7f2d1259c640 1 -- 172.21.15.37:0/858386558 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f2d140b7600 msgr2=0x7f2d14158380 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:30:43.633 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:43.627+0000 7f2d1259c640 1 --2- 172.21.15.37:0/858386558 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f2d140b7600 0x7f2d14158380 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:30:43.633 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:43.627+0000 7f2d1259c640 1 -- 172.21.15.37:0/858386558 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f2d141517f0 con 0x7f2d140b4bb0 2024-04-25T10:30:43.633 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:43.627+0000 7f2d12d9d640 1 --2- 172.21.15.37:0/858386558 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f2d140d4560 0x7f2d141eac20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T10:30:43.633 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:43.627+0000 7f2d1259c640 1 --2- 172.21.15.37:0/858386558 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f2d140b4bb0 0x7f2d14157e40 secure :-1 s=READY pgs=584 cs=0 l=1 rev1=1 crypto rx=0x7f2d0c00d5d0 tx=0x7f2d0c00da90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:30:43.633 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:43.627+0000 7f2d0affd640 1 -- 172.21.15.37:0/858386558 <== mon.0 v2:172.21.15.37:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2d0c004240 con 0x7f2d140b4bb0 2024-04-25T10:30:43.633 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:43.628+0000 7f2d0affd640 1 -- 172.21.15.37:0/858386558 <== mon.0 v2:172.21.15.37:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f2d0c0043e0 con 0x7f2d140b4bb0 2024-04-25T10:30:43.633 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:43.628+0000 7f2d1884c640 1 -- 172.21.15.37:0/858386558 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f2d14151ad0 con 0x7f2d140b4bb0 2024-04-25T10:30:43.633 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:43.628+0000 7f2d0affd640 1 -- 172.21.15.37:0/858386558 <== mon.0 v2:172.21.15.37:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2d0c003830 con 0x7f2d140b4bb0 2024-04-25T10:30:43.633 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:43.628+0000 7f2d1884c640 1 -- 172.21.15.37:0/858386558 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f2d14152090 con 0x7f2d140b4bb0 2024-04-25T10:30:43.633 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:43.629+0000 7f2d1884c640 1 -- 172.21.15.37:0/858386558 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f2d1406a140 con 0x7f2d140b4bb0 2024-04-25T10:30:43.633 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:43.629+0000 7f2d0affd640 1 -- 172.21.15.37:0/858386558 <== mon.0 v2:172.21.15.37:3300/0 4 ==== mgrmap(e 5) ==== 96782+0+0 (secure 0 0 0) 0x7f2d0c0039d0 con 0x7f2d140b4bb0 2024-04-25T10:30:43.633 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:43.630+0000 7f2d0affd640 1 --2- 172.21.15.37:0/858386558 >> [v2:172.21.15.203:6832/2418205237,v1:172.21.15.203:6833/2418205237] conn(0x7f2cf40744b0 0x7f2cf40769c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:30:43.633 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:43.630+0000 7f2d0affd640 1 -- 172.21.15.37:0/858386558 <== mon.0 v2:172.21.15.37:3300/0 5 ==== osd_map(44..44 src has 1..44) ==== 6248+0+0 (secure 0 0 0) 0x7f2d0c095770 con 0x7f2d140b4bb0 2024-04-25T10:30:43.634 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:43.630+0000 7f2d11d9b640 1 --2- 172.21.15.37:0/858386558 >> [v2:172.21.15.203:6832/2418205237,v1:172.21.15.203:6833/2418205237] conn(0x7f2cf40744b0 0x7f2cf40769c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:30:43.634 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:43.630+0000 7f2d11d9b640 1 --2- 172.21.15.37:0/858386558 >> [v2:172.21.15.203:6832/2418205237,v1:172.21.15.203:6833/2418205237] conn(0x7f2cf40744b0 0x7f2cf40769c0 secure :-1 s=READY pgs=273 cs=0 l=1 rev1=1 crypto rx=0x7f2cfc034000 tx=0x7f2cfc02e000 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:30:43.636 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:43.635+0000 7f2d0affd640 1 -- 172.21.15.37:0/858386558 <== mon.0 v2:172.21.15.37:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187246 (secure 0 0 0) 0x7f2d0c05fce0 con 0x7f2d140b4bb0 2024-04-25T10:30:43.861 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:43.859+0000 7f2d1884c640 1 -- 172.21.15.37:0/858386558 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f2d140d3ff0 con 0x7f2d140b4bb0 2024-04-25T10:30:43.862 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:43.860+0000 7f2d0affd640 1 -- 172.21.15.37:0/858386558 <== mon.0 v2:172.21.15.37:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 27 v27) ==== 94+0+6489 (secure 0 0 0) 0x7f2d0c05f5e0 con 0x7f2d140b4bb0 2024-04-25T10:30:43.862 INFO:teuthology.orchestra.run.smithi037.stdout: 2024-04-25T10:30:43.862 INFO:teuthology.orchestra.run.smithi037.stdout:{"epoch":27,"default_fscid":2,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":4946,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.37:6835/683655741","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.37:6834","nonce":683655741},{"type":"v1","addr":"172.21.15.37:6835","nonce":683655741}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":19},{"gid":5215,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.203:6836/3927045001","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6835","nonce":3927045001},{"type":"v1","addr":"172.21.15.203:6836","nonce":3927045001}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":19},{"gid":5218,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.203:6837/3108183946","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6834","nonce":3108183946},{"type":"v1","addr":"172.21.15.203:6837","nonce":3108183946}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":19}],"filesystems":[{"mdsmap":{"epoch":18,"flags":19,"flags_state":{"joinable":false,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-04-25T10:30:12.130473+0000","modified":"2024-04-25T10:30:24.853915+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":37,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":1,"in":[0],"up":{},"failed":[0],"damaged":[],"stopped":[],"info":{},"data_pools":[5],"metadata_pool":4,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":0,"qdb_cluster":[]},"id":2},{"mdsmap":{"epoch":27,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-04-25T10:30:28.588617+0000","modified":"2024-04-25T10:30:42.935167+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":1,"in":[0],"up":{"mds_0":5625},"failed":[],"damaged":[],"stopped":[],"info":{"gid_5625":{"gid":5625,"name":"c","rank":0,"incarnation":24,"state":"up:active","state_seq":7,"addr":"172.21.15.37:6833/2353203722","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.37:6832","nonce":2353203722},{"type":"v1","addr":"172.21.15.37:6833","nonce":2353203722}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[5],"metadata_pool":6,"enabled":true,"fs_name":"recovery_fs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":5625,"qdb_cluster":[5625]},"id":3}]} 2024-04-25T10:30:43.863 INFO:teuthology.orchestra.run.smithi037.stderr:dumped fsmap epoch 27 2024-04-25T10:30:43.865 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:43.862+0000 7f2d1884c640 1 -- 172.21.15.37:0/858386558 >> [v2:172.21.15.203:6832/2418205237,v1:172.21.15.203:6833/2418205237] conn(0x7f2cf40744b0 msgr2=0x7f2cf40769c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:30:43.865 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:43.862+0000 7f2d1884c640 1 --2- 172.21.15.37:0/858386558 >> [v2:172.21.15.203:6832/2418205237,v1:172.21.15.203:6833/2418205237] conn(0x7f2cf40744b0 0x7f2cf40769c0 secure :-1 s=READY pgs=273 cs=0 l=1 rev1=1 crypto rx=0x7f2cfc034000 tx=0x7f2cfc02e000 comp rx=0 tx=0).stop 2024-04-25T10:30:43.865 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:43.862+0000 7f2d1884c640 1 -- 172.21.15.37:0/858386558 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f2d140b4bb0 msgr2=0x7f2d14157e40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:30:43.865 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:43.862+0000 7f2d1884c640 1 --2- 172.21.15.37:0/858386558 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f2d140b4bb0 0x7f2d14157e40 secure :-1 s=READY pgs=584 cs=0 l=1 rev1=1 crypto rx=0x7f2d0c00d5d0 tx=0x7f2d0c00da90 comp rx=0 tx=0).stop 2024-04-25T10:30:43.865 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:43.862+0000 7f2d1884c640 1 -- 172.21.15.37:0/858386558 shutdown_connections 2024-04-25T10:30:43.865 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:43.862+0000 7f2d1884c640 1 --2- 172.21.15.37:0/858386558 >> [v2:172.21.15.203:6832/2418205237,v1:172.21.15.203:6833/2418205237] conn(0x7f2cf40744b0 0x7f2cf40769c0 unknown :-1 s=CLOSED pgs=273 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:30:43.865 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:43.862+0000 7f2d1884c640 1 --2- 172.21.15.37:0/858386558 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f2d140d4560 0x7f2d141eac20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:30:43.865 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:43.862+0000 7f2d1884c640 1 --2- 172.21.15.37:0/858386558 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f2d140b7600 0x7f2d14158380 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:30:43.865 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:43.862+0000 7f2d1884c640 1 --2- 172.21.15.37:0/858386558 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f2d140b4bb0 0x7f2d14157e40 unknown :-1 s=CLOSED pgs=584 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:30:43.865 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:43.862+0000 7f2d1884c640 1 -- 172.21.15.37:0/858386558 >> 172.21.15.37:0/858386558 conn(0x7f2d140d27e0 msgr2=0x7f2d141619f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:30:43.865 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:43.862+0000 7f2d1884c640 1 -- 172.21.15.37:0/858386558 shutdown_connections 2024-04-25T10:30:43.865 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:43.862+0000 7f2d1884c640 1 -- 172.21.15.37:0/858386558 wait complete. 2024-04-25T10:30:44.279 DEBUG:tasks.cephfs.filesystem:are_daemons_healthy: mds map: {'epoch': 27, 'flags': 18, 'flags_state': {'joinable': True, 'allow_snaps': True, 'allow_multimds_snaps': True, 'allow_standby_replay': False, 'refuse_client_session': False, 'refuse_standby_for_another_fs': False, 'balance_automate': False}, 'ever_allowed_features': 0, 'explicitly_allowed_features': 0, 'created': '2024-04-25T10:30:28.588617+0000', 'modified': '2024-04-25T10:30:42.935167+0000', 'tableserver': 0, 'root': 0, 'session_timeout': 60, 'session_autoclose': 300, 'required_client_features': {}, 'max_file_size': 1099511627776, 'max_xattr_size': 65536, 'last_failure': 0, 'last_failure_osd_epoch': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments', 'feature_12': 'quiesce subvolumes'}}, 'max_mds': 1, 'in': [0], 'up': {'mds_0': 5625}, 'failed': [], 'damaged': [], 'stopped': [], 'info': {'gid_5625': {'gid': 5625, 'name': 'c', 'rank': 0, 'incarnation': 24, 'state': 'up:active', 'state_seq': 7, 'addr': '172.21.15.37:6833/2353203722', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.37:6832', 'nonce': 2353203722}, {'type': 'v1', 'addr': '172.21.15.37:6833', 'nonce': 2353203722}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540701547738038271, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments', 'feature_12': 'quiesce subvolumes'}}}}, 'data_pools': [5], 'metadata_pool': 6, 'enabled': True, 'fs_name': 'recovery_fs', 'balancer': '', 'bal_rank_mask': '-1', 'standby_count_wanted': 0, 'qdb_leader': 5625, 'qdb_cluster': [5625]} 2024-04-25T10:30:44.279 DEBUG:tasks.cephfs.filesystem:are_daemons_healthy: 1/1 2024-04-25T10:30:44.280 DEBUG:teuthology.orchestra.run.smithi037:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.c status 2024-04-25T10:30:44.396 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:44.394+0000 7f252baf2640 1 Processor -- start 2024-04-25T10:30:44.396 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:44.394+0000 7f252baf2640 1 -- start start 2024-04-25T10:30:44.400 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:44.396+0000 7f252baf2640 1 --2- >> v2:172.21.15.37:3300/0 conn(0x7f25240d5160 0x7f25240b4e80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:30:44.400 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:44.396+0000 7f252baf2640 1 --2- >> v2:172.21.15.203:3300/0 conn(0x7f25240d3230 0x7f25240b9060 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:30:44.400 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:44.396+0000 7f252baf2640 1 -- --> v1:172.21.15.203:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f25240d1370 con 0x7f25240d0990 2024-04-25T10:30:44.401 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:44.396+0000 7f252baf2640 1 -- --> v2:172.21.15.37:3300/0 -- mon_getmap magic: 0 -- 0x7f25240d1650 con 0x7f25240d5160 2024-04-25T10:30:44.401 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:44.396+0000 7f252baf2640 1 -- --> v2:172.21.15.203:3300/0 -- mon_getmap magic: 0 -- 0x7f25240d4680 con 0x7f25240d3230 2024-04-25T10:30:44.401 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:44.396+0000 7f252988e640 1 --2- >> v2:172.21.15.37:3300/0 conn(0x7f25240d5160 0x7f25240b4e80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:30:44.401 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:44.396+0000 7f252988e640 1 --2- >> v2:172.21.15.37:3300/0 conn(0x7f25240d5160 0x7f25240b4e80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.37:3300/0 says I am v2:172.21.15.37:58218/0 (socket says 172.21.15.37:58218) 2024-04-25T10:30:44.401 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:44.396+0000 7f252988e640 1 -- 172.21.15.37:0/1709188736 learned_addr learned my addr 172.21.15.37:0/1709188736 (peer_addr_for_me v2:172.21.15.37:0/0) 2024-04-25T10:30:44.401 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:44.396+0000 7f252a08f640 1 --2- 172.21.15.37:0/1709188736 >> v2:172.21.15.203:3300/0 conn(0x7f25240d3230 0x7f25240b9060 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:30:44.401 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:44.396+0000 7f252988e640 1 -- 172.21.15.37:0/1709188736 >> v2:172.21.15.203:3300/0 conn(0x7f25240d3230 msgr2=0x7f25240b9060 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:30:44.402 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:44.396+0000 7f252988e640 1 --2- 172.21.15.37:0/1709188736 >> v2:172.21.15.203:3300/0 conn(0x7f25240d3230 0x7f25240b9060 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:30:44.402 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:44.396+0000 7f252988e640 1 -- 172.21.15.37:0/1709188736 >> v1:172.21.15.203:6789/0 conn(0x7f25240d0990 legacy=0x7f25240d2b20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:30:44.402 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:44.396+0000 7f252988e640 1 -- 172.21.15.37:0/1709188736 --> v2:172.21.15.37:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f25240d4960 con 0x7f25240d5160 2024-04-25T10:30:44.402 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:44.396+0000 7f252988e640 1 --2- 172.21.15.37:0/1709188736 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f25240d5160 0x7f25240b4e80 secure :-1 s=READY pgs=585 cs=0 l=1 rev1=1 crypto rx=0x7f252408f050 tx=0x7f251c02a110 comp rx=0 tx=0).ready entity=mon.0 client_cookie=dd535e28234b64b9 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:30:44.402 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:44.396+0000 7f252888c640 1 -- 172.21.15.37:0/1709188736 <== mon.0 v2:172.21.15.37:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f251c037070 con 0x7f25240d5160 2024-04-25T10:30:44.402 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:44.396+0000 7f252888c640 1 -- 172.21.15.37:0/1709188736 <== mon.0 v2:172.21.15.37:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f251c02ae20 con 0x7f25240d5160 2024-04-25T10:30:44.402 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:44.396+0000 7f252888c640 1 -- 172.21.15.37:0/1709188736 <== mon.0 v2:172.21.15.37:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f251c032660 con 0x7f25240d5160 2024-04-25T10:30:44.403 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:44.397+0000 7f252baf2640 1 -- 172.21.15.37:0/1709188736 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f25240d5160 msgr2=0x7f25240b4e80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:30:44.403 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:44.397+0000 7f252baf2640 1 --2- 172.21.15.37:0/1709188736 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f25240d5160 0x7f25240b4e80 secure :-1 s=READY pgs=585 cs=0 l=1 rev1=1 crypto rx=0x7f252408f050 tx=0x7f251c02a110 comp rx=0 tx=0).stop 2024-04-25T10:30:44.403 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:44.397+0000 7f252baf2640 1 -- 172.21.15.37:0/1709188736 shutdown_connections 2024-04-25T10:30:44.403 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:44.397+0000 7f252baf2640 1 --2- 172.21.15.37:0/1709188736 >> v2:172.21.15.203:3300/0 conn(0x7f25240d3230 0x7f25240b9060 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:30:44.403 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:44.397+0000 7f252baf2640 1 --2- 172.21.15.37:0/1709188736 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f25240d5160 0x7f25240b4e80 unknown :-1 s=CLOSED pgs=585 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:30:44.403 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:44.397+0000 7f252baf2640 1 -- 172.21.15.37:0/1709188736 >> 172.21.15.37:0/1709188736 conn(0x7f25240b7a80 msgr2=0x7f25240b7e90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:30:44.403 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:44.397+0000 7f252baf2640 1 -- 172.21.15.37:0/1709188736 shutdown_connections 2024-04-25T10:30:44.404 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:44.397+0000 7f252baf2640 1 -- 172.21.15.37:0/1709188736 wait complete. 2024-04-25T10:30:44.418 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:44.399+0000 7f252baf2640 1 Processor -- start 2024-04-25T10:30:44.418 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:44.399+0000 7f252baf2640 1 -- start start 2024-04-25T10:30:44.418 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:44.400+0000 7f252baf2640 1 --2- >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f25240d0990 0x7f252415ec90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:30:44.418 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:44.400+0000 7f252baf2640 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f25240d5160 0x7f25240cbaa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:30:44.419 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:44.400+0000 7f252baf2640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f252415ac00 0x7f252415b010 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:30:44.419 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:44.400+0000 7f252baf2640 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 -- 0x7f2524164c40 con 0x7f252415ac00 2024-04-25T10:30:44.419 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:44.400+0000 7f252baf2640 1 -- --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_getmap magic: 0 -- 0x7f2524164ac0 con 0x7f25240d5160 2024-04-25T10:30:44.419 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:44.400+0000 7f252baf2640 1 -- --> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] -- mon_getmap magic: 0 -- 0x7f25240d0da0 con 0x7f25240d0990 2024-04-25T10:30:44.419 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:44.400+0000 7f252a08f640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f252415ac00 0x7f252415b010 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:30:44.419 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:44.400+0000 7f252a08f640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f252415ac00 0x7f252415b010 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.37:3300/0 says I am v2:172.21.15.37:58234/0 (socket says 172.21.15.37:58234) 2024-04-25T10:30:44.419 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:44.400+0000 7f252a08f640 1 -- 172.21.15.37:0/2102229890 learned_addr learned my addr 172.21.15.37:0/2102229890 (peer_addr_for_me v2:172.21.15.37:0/0) 2024-04-25T10:30:44.419 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:44.400+0000 7f252988e640 1 --2- 172.21.15.37:0/2102229890 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f25240d0990 0x7f252415ec90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:30:44.419 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:44.400+0000 7f252908d640 1 --2- 172.21.15.37:0/2102229890 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f25240d5160 0x7f25240cbaa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:30:44.419 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:44.400+0000 7f252a08f640 1 -- 172.21.15.37:0/2102229890 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f25240d0990 msgr2=0x7f252415ec90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:30:44.419 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:44.400+0000 7f252a08f640 1 --2- 172.21.15.37:0/2102229890 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f25240d0990 0x7f252415ec90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:30:44.419 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:44.400+0000 7f252a08f640 1 -- 172.21.15.37:0/2102229890 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f25240d5160 msgr2=0x7f25240cbaa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:30:44.419 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:44.400+0000 7f252a08f640 1 --2- 172.21.15.37:0/2102229890 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f25240d5160 0x7f25240cbaa0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:30:44.419 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:44.400+0000 7f252a08f640 1 -- 172.21.15.37:0/2102229890 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f252415b550 con 0x7f252415ac00 2024-04-25T10:30:44.420 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:44.401+0000 7f252a08f640 1 --2- 172.21.15.37:0/2102229890 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f252415ac00 0x7f252415b010 secure :-1 s=READY pgs=586 cs=0 l=1 rev1=1 crypto rx=0x7f2514007e90 tx=0x7f251400e4b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:30:44.420 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:44.401+0000 7f25227fc640 1 -- 172.21.15.37:0/2102229890 <== mon.0 v2:172.21.15.37:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2514014420 con 0x7f252415ac00 2024-04-25T10:30:44.420 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:44.401+0000 7f25227fc640 1 -- 172.21.15.37:0/2102229890 <== mon.0 v2:172.21.15.37:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f2514019070 con 0x7f252415ac00 2024-04-25T10:30:44.420 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:44.401+0000 7f252baf2640 1 -- 172.21.15.37:0/2102229890 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f25241f9770 con 0x7f252415ac00 2024-04-25T10:30:44.420 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:44.401+0000 7f25227fc640 1 -- 172.21.15.37:0/2102229890 <== mon.0 v2:172.21.15.37:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2514014800 con 0x7f252415ac00 2024-04-25T10:30:44.420 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:44.401+0000 7f252baf2640 1 -- 172.21.15.37:0/2102229890 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f25241f9d30 con 0x7f252415ac00 2024-04-25T10:30:44.420 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:44.403+0000 7f25227fc640 1 -- 172.21.15.37:0/2102229890 <== mon.0 v2:172.21.15.37:3300/0 4 ==== mgrmap(e 5) ==== 96782+0+0 (secure 0 0 0) 0x7f25140149a0 con 0x7f252415ac00 2024-04-25T10:30:44.420 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:44.403+0000 7f25227fc640 1 --2- 172.21.15.37:0/2102229890 >> [v2:172.21.15.203:6832/2418205237,v1:172.21.15.203:6833/2418205237] conn(0x7f25100744b0 0x7f25100769c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:30:44.420 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:44.403+0000 7f25227fc640 1 -- 172.21.15.37:0/2102229890 <== mon.0 v2:172.21.15.37:3300/0 5 ==== osd_map(44..44 src has 1..44) ==== 6248+0+0 (secure 0 0 0) 0x7f2514095b90 con 0x7f252415ac00 2024-04-25T10:30:44.420 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:44.404+0000 7f252988e640 1 --2- 172.21.15.37:0/2102229890 >> [v2:172.21.15.203:6832/2418205237,v1:172.21.15.203:6833/2418205237] conn(0x7f25100744b0 0x7f25100769c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:30:44.420 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:44.404+0000 7f252988e640 1 --2- 172.21.15.37:0/2102229890 >> [v2:172.21.15.203:6832/2418205237,v1:172.21.15.203:6833/2418205237] conn(0x7f25100744b0 0x7f25100769c0 secure :-1 s=READY pgs=274 cs=0 l=1 rev1=1 crypto rx=0x7f251c034000 tx=0x7f251c02e000 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:30:44.420 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:44.404+0000 7f25329a7740 1 Processor -- start 2024-04-25T10:30:44.420 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:44.404+0000 7f25329a7740 1 -- start start 2024-04-25T10:30:44.420 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:44.405+0000 7f25329a7740 1 --2- >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x564ba7af7270 0x564ba7ec3600 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:30:44.421 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:44.405+0000 7f25329a7740 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x564ba7ace3b0 0x564ba7ebffa0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:30:44.421 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:44.406+0000 7f25329a7740 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x564ba7ad6590 0x564ba7e99260 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:30:44.421 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:44.406+0000 7f25329a7740 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 -- 0x564ba7dfd740 con 0x564ba7ace3b0 2024-04-25T10:30:44.421 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:44.406+0000 7f25329a7740 1 -- --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_getmap magic: 0 -- 0x564ba7d993a0 con 0x564ba7ad6590 2024-04-25T10:30:44.421 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:44.406+0000 7f25329a7740 1 -- --> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] -- mon_getmap magic: 0 -- 0x564ba7df9700 con 0x564ba7af7270 2024-04-25T10:30:44.421 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:44.406+0000 7f252908d640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x564ba7ace3b0 0x564ba7ebffa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:30:44.421 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:44.406+0000 7f252908d640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x564ba7ace3b0 0x564ba7ebffa0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.37:3300/0 says I am v2:172.21.15.37:58242/0 (socket says 172.21.15.37:58242) 2024-04-25T10:30:44.421 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:44.406+0000 7f252908d640 1 -- 172.21.15.37:0/2987282014 learned_addr learned my addr 172.21.15.37:0/2987282014 (peer_addr_for_me v2:172.21.15.37:0/0) 2024-04-25T10:30:44.421 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:44.406+0000 7f252988e640 1 --2- 172.21.15.37:0/2987282014 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x564ba7af7270 0x564ba7ec3600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:30:44.421 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:44.406+0000 7f252a08f640 1 --2- 172.21.15.37:0/2987282014 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x564ba7ad6590 0x564ba7e99260 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:30:44.421 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:44.406+0000 7f252908d640 1 -- 172.21.15.37:0/2987282014 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x564ba7af7270 msgr2=0x564ba7ec3600 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:30:44.421 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:44.406+0000 7f252908d640 1 --2- 172.21.15.37:0/2987282014 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x564ba7af7270 0x564ba7ec3600 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:30:44.421 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:44.406+0000 7f252908d640 1 -- 172.21.15.37:0/2987282014 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x564ba7ad6590 msgr2=0x564ba7e99260 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:30:44.421 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:44.406+0000 7f252908d640 1 --2- 172.21.15.37:0/2987282014 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x564ba7ad6590 0x564ba7e99260 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:30:44.421 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:44.406+0000 7f252908d640 1 -- 172.21.15.37:0/2987282014 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x564ba7db5750 con 0x564ba7ace3b0 2024-04-25T10:30:44.422 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:44.406+0000 7f252908d640 1 --2- 172.21.15.37:0/2987282014 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x564ba7ace3b0 0x564ba7ebffa0 secure :-1 s=READY pgs=587 cs=0 l=1 rev1=1 crypto rx=0x7f25180099d0 tx=0x7f2518009e90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=7495fe35f472dd71 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:30:44.422 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:44.406+0000 7f250b7fe640 1 -- 172.21.15.37:0/2987282014 <== mon.0 v2:172.21.15.37:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2518017070 con 0x564ba7ace3b0 2024-04-25T10:30:44.422 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:44.406+0000 7f250b7fe640 1 -- 172.21.15.37:0/2987282014 <== mon.0 v2:172.21.15.37:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f251800ea80 con 0x564ba7ace3b0 2024-04-25T10:30:44.422 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:44.406+0000 7f250b7fe640 1 -- 172.21.15.37:0/2987282014 <== mon.0 v2:172.21.15.37:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f251800ee70 con 0x564ba7ace3b0 2024-04-25T10:30:44.422 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:44.407+0000 7f25329a7740 1 -- 172.21.15.37:0/2987282014 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x564ba7ace3b0 msgr2=0x564ba7ebffa0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:30:44.422 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:44.407+0000 7f25329a7740 1 --2- 172.21.15.37:0/2987282014 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x564ba7ace3b0 0x564ba7ebffa0 secure :-1 s=READY pgs=587 cs=0 l=1 rev1=1 crypto rx=0x7f25180099d0 tx=0x7f2518009e90 comp rx=0 tx=0).stop 2024-04-25T10:30:44.422 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:44.407+0000 7f25329a7740 1 -- 172.21.15.37:0/2987282014 shutdown_connections 2024-04-25T10:30:44.422 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:44.407+0000 7f25329a7740 1 --2- 172.21.15.37:0/2987282014 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x564ba7ad6590 0x564ba7e99260 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:30:44.422 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:44.407+0000 7f25329a7740 1 --2- 172.21.15.37:0/2987282014 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x564ba7ace3b0 0x564ba7ebffa0 unknown :-1 s=CLOSED pgs=587 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:30:44.422 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:44.407+0000 7f25329a7740 1 --2- 172.21.15.37:0/2987282014 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x564ba7af7270 0x564ba7ec3600 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:30:44.422 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:44.407+0000 7f25329a7740 1 -- 172.21.15.37:0/2987282014 >> 172.21.15.37:0/2987282014 conn(0x564ba7aeb3c0 msgr2=0x564ba7e989f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:30:44.422 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:44.407+0000 7f25329a7740 1 -- 172.21.15.37:0/2987282014 shutdown_connections 2024-04-25T10:30:44.423 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:44.407+0000 7f25329a7740 1 -- 172.21.15.37:0/2987282014 wait complete. 2024-04-25T10:30:44.423 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:44.408+0000 7f25329a7740 1 -- start start 2024-04-25T10:30:44.423 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:44.408+0000 7f25329a7740 1 Processor -- start 2024-04-25T10:30:44.423 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:44.410+0000 7f25329a7740 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x564ba7ace3b0 0x564ba7ec42a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:30:44.423 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:44.410+0000 7f25329a7740 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x564ba7ad6590 0x564ba7ef07b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:30:44.424 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:44.410+0000 7f25329a7740 1 --2- >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x564ba7af7270 0x564ba7ef0cf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:30:44.424 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:44.410+0000 7f25329a7740 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 -- 0x564ba7d84fe0 con 0x564ba7ace3b0 2024-04-25T10:30:44.424 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:44.410+0000 7f25329a7740 1 -- --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_getmap magic: 0 -- 0x564ba7edd970 con 0x564ba7ad6590 2024-04-25T10:30:44.424 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:44.410+0000 7f25329a7740 1 -- --> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] -- mon_getmap magic: 0 -- 0x564ba7ebf310 con 0x564ba7af7270 2024-04-25T10:30:44.424 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:44.410+0000 7f252988e640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x564ba7ace3b0 0x564ba7ec42a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:30:44.424 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:44.410+0000 7f25329a7740 10 client.0 _ll_register_callbacks cb 0x564ba7d9b150 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-25T10:30:44.424 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:44.410+0000 7f252988e640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x564ba7ace3b0 0x564ba7ec42a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.37:3300/0 says I am v2:172.21.15.37:58250/0 (socket says 172.21.15.37:58250) 2024-04-25T10:30:44.424 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:44.410+0000 7f252988e640 1 -- 172.21.15.37:0/2356698492 learned_addr learned my addr 172.21.15.37:0/2356698492 (peer_addr_for_me v2:172.21.15.37:0/0) 2024-04-25T10:30:44.424 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:44.410+0000 7f252908d640 1 --2- 172.21.15.37:0/2356698492 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x564ba7ad6590 0x564ba7ef07b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:30:44.424 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:44.410+0000 7f252a08f640 1 --2- 172.21.15.37:0/2356698492 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x564ba7af7270 0x564ba7ef0cf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:30:44.424 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:44.410+0000 7f252988e640 1 -- 172.21.15.37:0/2356698492 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x564ba7af7270 msgr2=0x564ba7ef0cf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:30:44.424 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:44.410+0000 7f252988e640 1 --2- 172.21.15.37:0/2356698492 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x564ba7af7270 0x564ba7ef0cf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:30:44.424 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:44.410+0000 7f252988e640 1 -- 172.21.15.37:0/2356698492 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x564ba7ad6590 msgr2=0x564ba7ef07b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:30:44.424 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:44.410+0000 7f252988e640 1 --2- 172.21.15.37:0/2356698492 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x564ba7ad6590 0x564ba7ef07b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:30:44.425 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:44.410+0000 7f252988e640 1 -- 172.21.15.37:0/2356698492 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({osdmap=0}) -- 0x564ba7b0f2d0 con 0x564ba7ace3b0 2024-04-25T10:30:44.425 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:44.410+0000 7f252908d640 1 --2- 172.21.15.37:0/2356698492 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x564ba7ad6590 0x564ba7ef07b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T10:30:44.425 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:44.410+0000 7f252a08f640 1 --2- 172.21.15.37:0/2356698492 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x564ba7af7270 0x564ba7ef0cf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T10:30:44.425 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:44.410+0000 7f252988e640 1 --2- 172.21.15.37:0/2356698492 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x564ba7ace3b0 0x564ba7ec42a0 secure :-1 s=READY pgs=588 cs=0 l=1 rev1=1 crypto rx=0x7f251c02a700 tx=0x7f251c004730 comp rx=0 tx=0).ready entity=mon.0 client_cookie=12c0bcc08c4d6608 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:30:44.425 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:44.410+0000 7f2508ff9640 10 client.0 ms_handle_connect on v2:172.21.15.37:3300/0 2024-04-25T10:30:44.425 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:44.411+0000 7f2508ff9640 1 -- 172.21.15.37:0/2356698492 <== mon.0 v2:172.21.15.37:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f251c037040 con 0x564ba7ace3b0 2024-04-25T10:30:44.425 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:44.411+0000 7f2508ff9640 1 -- 172.21.15.37:0/2356698492 <== mon.0 v2:172.21.15.37:3300/0 2 ==== osd_map(44..44 src has 1..44) ==== 6248+0+0 (secure 0 0 0) 0x7f251c03ed40 con 0x564ba7ace3b0 2024-04-25T10:30:44.425 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:44.411+0000 7f25329a7740 1 -- 172.21.15.37:0/2356698492 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f251800ea80 con 0x564ba7ace3b0 2024-04-25T10:30:44.425 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:44.411+0000 7f2508ff9640 1 -- 172.21.15.37:0/2356698492 <== mon.0 v2:172.21.15.37:3300/0 3 ==== mon_get_version_reply(handle=1 version=27) ==== 24+0+0 (secure 0 0 0) 0x7f251c056050 con 0x564ba7ace3b0 2024-04-25T10:30:44.425 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:44.411+0000 7f25329a7740 10 client.5898 fetch_fsmap learned FSMap version 27 2024-04-25T10:30:44.425 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:44.411+0000 7f25329a7740 1 -- 172.21.15.37:0/2356698492 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({config=0+,fsmap=27,monmap=2+}) -- 0x564ba7bd96b0 con 0x564ba7ace3b0 2024-04-25T10:30:44.425 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:44.411+0000 7f2508ff9640 1 -- 172.21.15.37:0/2356698492 <== mon.0 v2:172.21.15.37:3300/0 4 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f251800ea80 con 0x564ba7ace3b0 2024-04-25T10:30:44.425 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:44.411+0000 7f2508ff9640 1 -- 172.21.15.37:0/2356698492 <== mon.0 v2:172.21.15.37:3300/0 5 ==== fsmap(e 27) ==== 4139+0+0 (secure 0 0 0) 0x7f251c059020 con 0x564ba7ace3b0 2024-04-25T10:30:44.426 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:44.411+0000 7f25329a7740 10 client.5898 fetch_fsmap finished waiting for FSMap version 27 2024-04-25T10:30:44.426 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:44.411+0000 7f25329a7740 10 client.5898 resolve_mds: resolved name 'c' to daemon mds.c 2024-04-25T10:30:44.426 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:44.411+0000 7f25329a7740 20 client.5898 populate_metadata read hostname 'smithi037' 2024-04-25T10:30:44.426 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:44.411+0000 7f25329a7740 1 --2- 172.21.15.37:0/2356698492 >> [v2:172.21.15.37:6832/2353203722,v1:172.21.15.37:6833/2353203722] conn(0x564ba7ab5e40 0x564ba7ec62a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:30:44.426 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:44.411+0000 7f25329a7740 4 client.5898 mds_command: new command op to 5625 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-25T10:30:44.426 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:44.411+0000 7f25329a7740 1 -- 172.21.15.37:0/2356698492 --> [v2:172.21.15.37:6832/2353203722,v1:172.21.15.37:6833/2353203722] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x564ba7e0b350 con 0x564ba7ab5e40 2024-04-25T10:30:44.427 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:44.411+0000 7f252908d640 1 --2- 172.21.15.37:0/2356698492 >> [v2:172.21.15.37:6832/2353203722,v1:172.21.15.37:6833/2353203722] conn(0x564ba7ab5e40 0x564ba7ec62a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:30:44.427 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:44.412+0000 7f2508ff9640 10 client.5898 ms_handle_connect on v2:172.21.15.37:6832/2353203722 2024-04-25T10:30:44.427 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:44.412+0000 7f252908d640 1 --2- 172.21.15.37:0/2356698492 >> [v2:172.21.15.37:6832/2353203722,v1:172.21.15.37:6833/2353203722] conn(0x564ba7ab5e40 0x564ba7ec62a0 crc :-1 s=READY pgs=12 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=5c6dbe6d602c798f server_cookie=e63def08bf3c3043 in_seq=0 out_seq=0 2024-04-25T10:30:44.427 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:44.412+0000 7f2508ff9640 1 -- 172.21.15.37:0/2356698492 <== mds.0 v2:172.21.15.37:6832/2353203722 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23648 (crc 0 0 0) 0x564ba7e0b350 con 0x564ba7ab5e40 2024-04-25T10:30:44.427 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:44.412+0000 7f2508ff9640 10 client.5898 handle_command_reply: tid=0 2024-04-25T10:30:44.427 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:44.412+0000 7f25329a7740 1 client.5898 shutdown 2024-04-25T10:30:44.428 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:44.413+0000 7f25329a7740 1 -- 172.21.15.37:0/2356698492 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x564ba7ace3b0 msgr2=0x564ba7ec42a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:30:44.428 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:44.413+0000 7f25329a7740 1 --2- 172.21.15.37:0/2356698492 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x564ba7ace3b0 0x564ba7ec42a0 secure :-1 s=READY pgs=588 cs=0 l=1 rev1=1 crypto rx=0x7f251c02a700 tx=0x7f251c004730 comp rx=0 tx=0).stop 2024-04-25T10:30:44.428 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:44.413+0000 7f25329a7740 1 -- 172.21.15.37:0/2356698492 shutdown_connections 2024-04-25T10:30:44.428 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:44.413+0000 7f25329a7740 1 --2- 172.21.15.37:0/2356698492 >> [v2:172.21.15.37:6832/2353203722,v1:172.21.15.37:6833/2353203722] conn(0x564ba7ab5e40 0x564ba7ec62a0 crc :-1 s=READY pgs=12 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:30:44.428 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:44.413+0000 7f25329a7740 1 --2- 172.21.15.37:0/2356698492 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x564ba7af7270 0x564ba7ef0cf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:30:44.428 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:44.413+0000 7f25329a7740 1 --2- 172.21.15.37:0/2356698492 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x564ba7ad6590 0x564ba7ef07b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:30:44.429 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:44.413+0000 7f25329a7740 1 --2- 172.21.15.37:0/2356698492 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x564ba7ace3b0 0x564ba7ec42a0 unknown :-1 s=CLOSED pgs=588 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:30:44.429 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:44.413+0000 7f2508ff9640 0 client.5898 ms_handle_reset on v2:172.21.15.37:6832/2353203722 2024-04-25T10:30:44.429 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:44.413+0000 7f25329a7740 1 -- 172.21.15.37:0/2356698492 >> 172.21.15.37:0/2356698492 conn(0x564ba7aeb3c0 msgr2=0x564ba7e97d90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:30:44.429 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:44.413+0000 7f25329a7740 1 -- 172.21.15.37:0/2356698492 shutdown_connections 2024-04-25T10:30:44.429 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:44.413+0000 7f25329a7740 1 -- 172.21.15.37:0/2356698492 wait complete. 2024-04-25T10:30:44.429 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:44.413+0000 7f25329a7740 20 client.5898 trim_cache size 0 max 16384 2024-04-25T10:30:44.434 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:44.432+0000 7f25329a7740 1 Processor -- start 2024-04-25T10:30:44.434 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:44.432+0000 7f25329a7740 1 -- start start 2024-04-25T10:30:44.446 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:44.433+0000 7f25329a7740 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x564ba7ace3b0 0x564ba7ec4e30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:30:44.446 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:44.433+0000 7f25329a7740 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x564ba7ad6590 0x564ba7ed3a90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:30:44.446 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:44.433+0000 7f25329a7740 1 --2- >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x564ba7af7270 0x564ba7ed3fd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:30:44.446 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:44.433+0000 7f25329a7740 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 -- 0x7f2524202c70 con 0x564ba7ad6590 2024-04-25T10:30:44.446 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:44.433+0000 7f25329a7740 1 -- --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_getmap magic: 0 -- 0x7f252415e060 con 0x564ba7ace3b0 2024-04-25T10:30:44.446 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:44.433+0000 7f25329a7740 1 -- --> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] -- mon_getmap magic: 0 -- 0x7f2524202620 con 0x564ba7af7270 2024-04-25T10:30:44.447 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:44.433+0000 7f252908d640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x564ba7ad6590 0x564ba7ed3a90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:30:44.449 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:44.433+0000 7f252908d640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x564ba7ad6590 0x564ba7ed3a90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.37:3300/0 says I am v2:172.21.15.37:58262/0 (socket says 172.21.15.37:58262) 2024-04-25T10:30:44.449 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:44.433+0000 7f252908d640 1 -- 172.21.15.37:0/112851110 learned_addr learned my addr 172.21.15.37:0/112851110 (peer_addr_for_me v2:172.21.15.37:0/0) 2024-04-25T10:30:44.450 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:44.433+0000 7f252a08f640 1 --2- 172.21.15.37:0/112851110 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x564ba7af7270 0x564ba7ed3fd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:30:44.450 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:44.434+0000 7f252988e640 1 --2- 172.21.15.37:0/112851110 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x564ba7ace3b0 0x564ba7ec4e30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:30:44.450 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:44.434+0000 7f252908d640 1 -- 172.21.15.37:0/112851110 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x564ba7af7270 msgr2=0x564ba7ed3fd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:30:44.450 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:44.434+0000 7f252908d640 1 --2- 172.21.15.37:0/112851110 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x564ba7af7270 0x564ba7ed3fd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:30:44.450 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:44.434+0000 7f252908d640 1 -- 172.21.15.37:0/112851110 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x564ba7ace3b0 msgr2=0x564ba7ec4e30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:30:44.451 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:44.434+0000 7f252908d640 1 --2- 172.21.15.37:0/112851110 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x564ba7ace3b0 0x564ba7ec4e30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:30:44.451 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:44.434+0000 7f252908d640 1 -- 172.21.15.37:0/112851110 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x564ba7ec4610 con 0x564ba7ad6590 2024-04-25T10:30:44.451 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:44.434+0000 7f252a08f640 1 --2- 172.21.15.37:0/112851110 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x564ba7af7270 0x564ba7ed3fd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T10:30:44.451 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:44.434+0000 7f252988e640 1 --2- 172.21.15.37:0/112851110 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x564ba7ace3b0 0x564ba7ec4e30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T10:30:44.451 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:44.434+0000 7f252908d640 1 --2- 172.21.15.37:0/112851110 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x564ba7ad6590 0x564ba7ed3a90 secure :-1 s=READY pgs=589 cs=0 l=1 rev1=1 crypto rx=0x7f2518016000 tx=0x7f2518008030 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a6c1aedfd07ba358 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:30:44.451 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:44.434+0000 7f250affd640 1 -- 172.21.15.37:0/112851110 <== mon.0 v2:172.21.15.37:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f251802f070 con 0x564ba7ad6590 2024-04-25T10:30:44.452 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:44.434+0000 7f250affd640 1 -- 172.21.15.37:0/112851110 <== mon.0 v2:172.21.15.37:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f2518029e50 con 0x564ba7ad6590 2024-04-25T10:30:44.452 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:44.434+0000 7f250affd640 1 -- 172.21.15.37:0/112851110 <== mon.0 v2:172.21.15.37:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2518017040 con 0x564ba7ad6590 2024-04-25T10:30:44.452 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:44.435+0000 7f25329a7740 1 -- 172.21.15.37:0/112851110 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x564ba7ad6590 msgr2=0x564ba7ed3a90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:30:44.453 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:44.435+0000 7f25329a7740 1 --2- 172.21.15.37:0/112851110 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x564ba7ad6590 0x564ba7ed3a90 secure :-1 s=READY pgs=589 cs=0 l=1 rev1=1 crypto rx=0x7f2518016000 tx=0x7f2518008030 comp rx=0 tx=0).stop 2024-04-25T10:30:44.453 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:44.435+0000 7f25329a7740 1 -- 172.21.15.37:0/112851110 shutdown_connections 2024-04-25T10:30:44.453 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:44.435+0000 7f25329a7740 1 --2- 172.21.15.37:0/112851110 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x564ba7af7270 0x564ba7ed3fd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:30:44.453 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:44.435+0000 7f25329a7740 1 --2- 172.21.15.37:0/112851110 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x564ba7ad6590 0x564ba7ed3a90 unknown :-1 s=CLOSED pgs=589 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:30:44.453 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:44.435+0000 7f25329a7740 1 --2- 172.21.15.37:0/112851110 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x564ba7ace3b0 0x564ba7ec4e30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:30:44.453 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:44.435+0000 7f25329a7740 1 -- 172.21.15.37:0/112851110 >> 172.21.15.37:0/112851110 conn(0x564ba7aeb3c0 msgr2=0x564ba7eec4c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:30:44.454 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:44.435+0000 7f25329a7740 1 -- 172.21.15.37:0/112851110 shutdown_connections 2024-04-25T10:30:44.454 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:44.435+0000 7f25329a7740 1 -- 172.21.15.37:0/112851110 wait complete. 2024-04-25T10:30:44.454 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:44.436+0000 7f25329a7740 1 -- start start 2024-04-25T10:30:44.454 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:44.436+0000 7f25329a7740 1 Processor -- start 2024-04-25T10:30:44.454 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:44.437+0000 7f25329a7740 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x564ba7ace3b0 0x564ba7f5a640 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:30:44.454 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:44.437+0000 7f25329a7740 1 --2- >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x564ba7ad6590 0x564ba7f00930 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:30:44.454 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:44.437+0000 7f25329a7740 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x564ba7af7270 0x564ba7f00e70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:30:44.454 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:44.437+0000 7f25329a7740 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 -- 0x564ba7ebefc0 con 0x564ba7ace3b0 2024-04-25T10:30:44.454 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:44.437+0000 7f25329a7740 1 -- --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_getmap magic: 0 -- 0x564ba7ec47e0 con 0x564ba7af7270 2024-04-25T10:30:44.455 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:44.437+0000 7f25329a7740 1 -- --> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] -- mon_getmap magic: 0 -- 0x564ba7edcd80 con 0x564ba7ad6590 2024-04-25T10:30:44.455 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:44.437+0000 7f25329a7740 10 client.0 _ll_register_callbacks cb 0x7f251c009590 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-25T10:30:44.455 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:44.437+0000 7f252988e640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x564ba7ace3b0 0x564ba7f5a640 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:30:44.455 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:44.437+0000 7f252988e640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x564ba7ace3b0 0x564ba7f5a640 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.37:3300/0 says I am v2:172.21.15.37:58266/0 (socket says 172.21.15.37:58266) 2024-04-25T10:30:44.455 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:44.437+0000 7f252988e640 1 -- 172.21.15.37:0/1761809535 learned_addr learned my addr 172.21.15.37:0/1761809535 (peer_addr_for_me v2:172.21.15.37:0/0) 2024-04-25T10:30:44.455 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:44.437+0000 7f252908d640 1 --2- 172.21.15.37:0/1761809535 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x564ba7ad6590 0x564ba7f00930 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:30:44.455 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:44.437+0000 7f252a08f640 1 --2- 172.21.15.37:0/1761809535 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x564ba7af7270 0x564ba7f00e70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:30:44.455 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:44.437+0000 7f252988e640 1 -- 172.21.15.37:0/1761809535 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x564ba7ad6590 msgr2=0x564ba7f00930 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:30:44.455 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:44.438+0000 7f252988e640 1 --2- 172.21.15.37:0/1761809535 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x564ba7ad6590 0x564ba7f00930 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:30:44.455 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:44.438+0000 7f252988e640 1 -- 172.21.15.37:0/1761809535 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x564ba7af7270 msgr2=0x564ba7f00e70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:30:44.455 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:44.438+0000 7f252988e640 1 --2- 172.21.15.37:0/1761809535 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x564ba7af7270 0x564ba7f00e70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:30:44.455 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:44.438+0000 7f252988e640 1 -- 172.21.15.37:0/1761809535 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({osdmap=0}) -- 0x564ba7f01680 con 0x564ba7ace3b0 2024-04-25T10:30:44.455 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:44.438+0000 7f252908d640 1 --2- 172.21.15.37:0/1761809535 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x564ba7ad6590 0x564ba7f00930 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T10:30:44.455 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:44.438+0000 7f252a08f640 1 --2- 172.21.15.37:0/1761809535 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x564ba7af7270 0x564ba7f00e70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T10:30:44.456 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:44.438+0000 7f252988e640 1 --2- 172.21.15.37:0/1761809535 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x564ba7ace3b0 0x564ba7f5a640 secure :-1 s=READY pgs=590 cs=0 l=1 rev1=1 crypto rx=0x7f251c000c00 tx=0x7f251c009240 comp rx=0 tx=0).ready entity=mon.0 client_cookie=dd3e1d913d1973fd server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:30:44.456 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:44.438+0000 7f2508ff9640 10 client.0 ms_handle_connect on v2:172.21.15.37:3300/0 2024-04-25T10:30:44.456 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:44.438+0000 7f2508ff9640 1 -- 172.21.15.37:0/1761809535 <== mon.0 v2:172.21.15.37:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f251c037070 con 0x564ba7ace3b0 2024-04-25T10:30:44.456 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:44.438+0000 7f2508ff9640 1 -- 172.21.15.37:0/1761809535 <== mon.0 v2:172.21.15.37:3300/0 2 ==== osd_map(44..44 src has 1..44) ==== 6248+0+0 (secure 0 0 0) 0x7f251c03c4d0 con 0x564ba7ace3b0 2024-04-25T10:30:44.456 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:44.438+0000 7f25329a7740 1 -- 172.21.15.37:0/1761809535 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f2518029e50 con 0x564ba7ace3b0 2024-04-25T10:30:44.456 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:44.438+0000 7f2508ff9640 1 -- 172.21.15.37:0/1761809535 <== mon.0 v2:172.21.15.37:3300/0 3 ==== mon_get_version_reply(handle=1 version=27) ==== 24+0+0 (secure 0 0 0) 0x7f251c032590 con 0x564ba7ace3b0 2024-04-25T10:30:44.456 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:44.438+0000 7f25329a7740 10 client.5904 fetch_fsmap learned FSMap version 27 2024-04-25T10:30:44.456 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:44.438+0000 7f25329a7740 1 -- 172.21.15.37:0/1761809535 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({config=0+,fsmap=27,monmap=2+}) -- 0x564ba7ec70c0 con 0x564ba7ace3b0 2024-04-25T10:30:44.456 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:44.438+0000 7f2508ff9640 1 -- 172.21.15.37:0/1761809535 <== mon.0 v2:172.21.15.37:3300/0 4 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f2518029e50 con 0x564ba7ace3b0 2024-04-25T10:30:44.456 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:44.438+0000 7f2508ff9640 1 -- 172.21.15.37:0/1761809535 <== mon.0 v2:172.21.15.37:3300/0 5 ==== fsmap(e 27) ==== 4139+0+0 (secure 0 0 0) 0x7f251c062050 con 0x564ba7ace3b0 2024-04-25T10:30:44.456 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:44.438+0000 7f25329a7740 10 client.5904 fetch_fsmap finished waiting for FSMap version 27 2024-04-25T10:30:44.456 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:44.438+0000 7f25329a7740 10 client.5904 resolve_mds: resolved name 'c' to daemon mds.c 2024-04-25T10:30:44.456 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:44.438+0000 7f25329a7740 20 client.5904 populate_metadata read hostname 'smithi037' 2024-04-25T10:30:44.456 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:44.438+0000 7f25329a7740 1 --2- 172.21.15.37:0/1761809535 >> [v2:172.21.15.37:6832/2353203722,v1:172.21.15.37:6833/2353203722] conn(0x564ba7ebdf20 0x564ba7e98e80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:30:44.457 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:44.438+0000 7f25329a7740 4 client.5904 mds_command: new command op to 5625 tid=0 multi_id=0 [{"prefix": "status"}] 2024-04-25T10:30:44.457 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:44.438+0000 7f25329a7740 1 -- 172.21.15.37:0/1761809535 --> [v2:172.21.15.37:6832/2353203722,v1:172.21.15.37:6833/2353203722] -- command(tid 0: {"prefix": "status"}) -- 0x564ba7ec5850 con 0x564ba7ebdf20 2024-04-25T10:30:44.457 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:44.439+0000 7f252908d640 1 --2- 172.21.15.37:0/1761809535 >> [v2:172.21.15.37:6832/2353203722,v1:172.21.15.37:6833/2353203722] conn(0x564ba7ebdf20 0x564ba7e98e80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:30:44.457 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:44.439+0000 7f2508ff9640 10 client.5904 ms_handle_connect on v2:172.21.15.37:6832/2353203722 2024-04-25T10:30:44.457 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:44.439+0000 7f252908d640 1 --2- 172.21.15.37:0/1761809535 >> [v2:172.21.15.37:6832/2353203722,v1:172.21.15.37:6833/2353203722] conn(0x564ba7ebdf20 0x564ba7e98e80 crc :-1 s=READY pgs=13 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=8e4f2615a360603b server_cookie=9817e19a2f5badb7 in_seq=0 out_seq=0 2024-04-25T10:30:44.457 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:44.439+0000 7f2508ff9640 1 -- 172.21.15.37:0/1761809535 <== mds.0 v2:172.21.15.37:6832/2353203722 1 ==== command_reply(tid 0: 0 ) ==== 8+0+336 (crc 0 0 0) 0x564ba7ec5850 con 0x564ba7ebdf20 2024-04-25T10:30:44.457 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:44.439+0000 7f2508ff9640 10 client.5904 handle_command_reply: tid=0 2024-04-25T10:30:44.457 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:44.439+0000 7f25329a7740 1 client.5904 shutdown 2024-04-25T10:30:44.457 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:44.439+0000 7f25329a7740 1 -- 172.21.15.37:0/1761809535 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x564ba7ace3b0 msgr2=0x564ba7f5a640 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:30:44.457 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:44.439+0000 7f25329a7740 1 --2- 172.21.15.37:0/1761809535 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x564ba7ace3b0 0x564ba7f5a640 secure :-1 s=READY pgs=590 cs=0 l=1 rev1=1 crypto rx=0x7f251c000c00 tx=0x7f251c009240 comp rx=0 tx=0).stop 2024-04-25T10:30:44.457 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:44.439+0000 7f25329a7740 1 -- 172.21.15.37:0/1761809535 shutdown_connections 2024-04-25T10:30:44.457 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:44.439+0000 7f25329a7740 1 --2- 172.21.15.37:0/1761809535 >> [v2:172.21.15.37:6832/2353203722,v1:172.21.15.37:6833/2353203722] conn(0x564ba7ebdf20 0x564ba7e98e80 crc :-1 s=READY pgs=13 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:30:44.457 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:44.439+0000 7f25329a7740 1 --2- 172.21.15.37:0/1761809535 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x564ba7af7270 0x564ba7f00e70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:30:44.457 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:44.439+0000 7f25329a7740 1 --2- 172.21.15.37:0/1761809535 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x564ba7ad6590 0x564ba7f00930 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:30:44.457 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:44.439+0000 7f2508ff9640 0 client.5904 ms_handle_reset on v2:172.21.15.37:6832/2353203722 2024-04-25T10:30:44.458 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:44.439+0000 7f25329a7740 1 --2- 172.21.15.37:0/1761809535 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x564ba7ace3b0 0x564ba7f5a640 unknown :-1 s=CLOSED pgs=590 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:30:44.458 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:44.439+0000 7f25329a7740 1 -- 172.21.15.37:0/1761809535 >> 172.21.15.37:0/1761809535 conn(0x564ba7aeb3c0 msgr2=0x564ba7ef04f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:30:44.458 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:44.439+0000 7f25329a7740 1 -- 172.21.15.37:0/1761809535 shutdown_connections 2024-04-25T10:30:44.458 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:44.439+0000 7f25329a7740 1 -- 172.21.15.37:0/1761809535 wait complete. 2024-04-25T10:30:44.458 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:44.439+0000 7f25329a7740 20 client.5904 trim_cache size 0 max 16384 2024-04-25T10:30:44.458 INFO:teuthology.orchestra.run.smithi037.stdout:{ 2024-04-25T10:30:44.458 INFO:teuthology.orchestra.run.smithi037.stdout: "cluster_fsid": "f4acda28-be34-492a-b286-994afd36f61d", 2024-04-25T10:30:44.458 INFO:teuthology.orchestra.run.smithi037.stdout: "whoami": 0, 2024-04-25T10:30:44.459 INFO:teuthology.orchestra.run.smithi037.stdout: "id": 5625, 2024-04-25T10:30:44.459 INFO:teuthology.orchestra.run.smithi037.stdout: "want_state": "up:active", 2024-04-25T10:30:44.459 INFO:teuthology.orchestra.run.smithi037.stdout: "state": "up:active", 2024-04-25T10:30:44.459 INFO:teuthology.orchestra.run.smithi037.stdout: "fs_name": "recovery_fs", 2024-04-25T10:30:44.459 INFO:teuthology.orchestra.run.smithi037.stdout: "rank_uptime": 4.5190483529999996, 2024-04-25T10:30:44.459 INFO:teuthology.orchestra.run.smithi037.stdout: "mdsmap_epoch": 27, 2024-04-25T10:30:44.459 INFO:teuthology.orchestra.run.smithi037.stdout: "osdmap_epoch": 44, 2024-04-25T10:30:44.459 INFO:teuthology.orchestra.run.smithi037.stdout: "osdmap_epoch_barrier": 44, 2024-04-25T10:30:44.459 INFO:teuthology.orchestra.run.smithi037.stdout: "uptime": 19.201478086000002 2024-04-25T10:30:44.460 INFO:teuthology.orchestra.run.smithi037.stdout:} 2024-04-25T10:30:44.461 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:44.440+0000 7f252baf2640 1 -- 172.21.15.37:0/2102229890 >> [v2:172.21.15.203:6832/2418205237,v1:172.21.15.203:6833/2418205237] conn(0x7f25100744b0 msgr2=0x7f25100769c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:30:44.461 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:44.440+0000 7f252baf2640 1 --2- 172.21.15.37:0/2102229890 >> [v2:172.21.15.203:6832/2418205237,v1:172.21.15.203:6833/2418205237] conn(0x7f25100744b0 0x7f25100769c0 secure :-1 s=READY pgs=274 cs=0 l=1 rev1=1 crypto rx=0x7f251c034000 tx=0x7f251c02e000 comp rx=0 tx=0).stop 2024-04-25T10:30:44.461 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:44.440+0000 7f252baf2640 1 -- 172.21.15.37:0/2102229890 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f252415ac00 msgr2=0x7f252415b010 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:30:44.461 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:44.440+0000 7f252baf2640 1 --2- 172.21.15.37:0/2102229890 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f252415ac00 0x7f252415b010 secure :-1 s=READY pgs=586 cs=0 l=1 rev1=1 crypto rx=0x7f2514007e90 tx=0x7f251400e4b0 comp rx=0 tx=0).stop 2024-04-25T10:30:44.462 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:44.440+0000 7f252baf2640 1 -- 172.21.15.37:0/2102229890 shutdown_connections 2024-04-25T10:30:44.462 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:44.440+0000 7f252baf2640 1 --2- 172.21.15.37:0/2102229890 >> [v2:172.21.15.203:6832/2418205237,v1:172.21.15.203:6833/2418205237] conn(0x7f25100744b0 0x7f25100769c0 unknown :-1 s=CLOSED pgs=274 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:30:44.462 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:44.440+0000 7f252baf2640 1 --2- 172.21.15.37:0/2102229890 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f252415ac00 0x7f252415b010 unknown :-1 s=CLOSED pgs=586 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:30:44.462 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:44.440+0000 7f252baf2640 1 --2- 172.21.15.37:0/2102229890 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f25240d5160 0x7f25240cbaa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:30:44.462 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:44.440+0000 7f252baf2640 1 --2- 172.21.15.37:0/2102229890 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f25240d0990 0x7f252415ec90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:30:44.462 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:44.440+0000 7f252baf2640 1 -- 172.21.15.37:0/2102229890 >> 172.21.15.37:0/2102229890 conn(0x7f25240b7a80 msgr2=0x7f25240b6c00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:30:44.462 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:44.440+0000 7f252baf2640 1 -- 172.21.15.37:0/2102229890 shutdown_connections 2024-04-25T10:30:44.462 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:44.440+0000 7f252baf2640 1 -- 172.21.15.37:0/2102229890 wait complete. 2024-04-25T10:30:44.463 DEBUG:teuthology.orchestra.run.smithi037:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph config set mds debug_mds 20 2024-04-25T10:30:44.577 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:44.576+0000 7f831f5f3640 1 Processor -- start 2024-04-25T10:30:44.577 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:44.576+0000 7f831f5f3640 1 -- start start 2024-04-25T10:30:44.579 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:44.577+0000 7f831f5f3640 1 --2- >> v2:172.21.15.37:3300/0 conn(0x7f83180b4bb0 0x7f83181646b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:30:44.580 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:44.577+0000 7f831f5f3640 1 -- --> v1:172.21.15.37:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f83180d5080 con 0x7f83180b7600 2024-04-25T10:30:44.580 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:44.577+0000 7f831f5f3640 1 -- --> v1:172.21.15.203:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f8318161190 con 0x7f83180d4560 2024-04-25T10:30:44.580 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:44.577+0000 7f831f5f3640 1 -- --> v2:172.21.15.37:3300/0 -- mon_getmap magic: 0 -- 0x7f8318161470 con 0x7f83180b4bb0 2024-04-25T10:30:44.580 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:44.577+0000 7f831d38f640 1 --2- >> v2:172.21.15.37:3300/0 conn(0x7f83180b4bb0 0x7f83181646b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:30:44.580 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:44.577+0000 7f831cb8e640 1 --1- >> v1:172.21.15.37:6789/0 conn(0x7f83180b7600 0x7f83180b7a10 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.37:6789/0 says I am v1:172.21.15.37:55662/0 (socket says 172.21.15.37:55662) 2024-04-25T10:30:44.580 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:44.577+0000 7f831d38f640 1 --2- >> v2:172.21.15.37:3300/0 conn(0x7f83180b4bb0 0x7f83181646b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.37:3300/0 says I am v2:172.21.15.37:58270/0 (socket says 172.21.15.37:58270) 2024-04-25T10:30:44.580 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:44.577+0000 7f831cb8e640 1 -- 172.21.15.37:0/3940256256 learned_addr learned my addr 172.21.15.37:0/3940256256 (peer_addr_for_me v1:172.21.15.37:0/0) 2024-04-25T10:30:44.580 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:44.578+0000 7f831d38f640 1 -- 172.21.15.37:0/3940256256 >> v1:172.21.15.203:6789/0 conn(0x7f83180d4560 legacy=0x7f83180d4970 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:30:44.580 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:44.578+0000 7f831d38f640 1 -- 172.21.15.37:0/3940256256 >> v1:172.21.15.37:6789/0 conn(0x7f83180b7600 legacy=0x7f83180b7a10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:30:44.580 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:44.578+0000 7f8317fff640 1 -- 172.21.15.37:0/3940256256 <== mon.0 v1:172.21.15.37:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 2930407965 0 0) 0x7f83180d5080 con 0x7f83180b7600 2024-04-25T10:30:44.580 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:44.578+0000 7f831d38f640 1 -- 172.21.15.37:0/3940256256 --> v2:172.21.15.37:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f8318163eb0 con 0x7f83180b4bb0 2024-04-25T10:30:44.580 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:44.578+0000 7f831d38f640 1 --2- 172.21.15.37:0/3940256256 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f83180b4bb0 0x7f83181646b0 secure :-1 s=READY pgs=592 cs=0 l=1 rev1=1 crypto rx=0x7f83100097a0 tx=0x7f831002a1e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=53824b7bd04ca8c server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:30:44.581 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:44.578+0000 7f8317fff640 1 -- 172.21.15.37:0/3940256256 <== mon.0 v2:172.21.15.37:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8310037070 con 0x7f83180b4bb0 2024-04-25T10:30:44.581 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:44.578+0000 7f8317fff640 1 -- 172.21.15.37:0/3940256256 <== mon.0 v2:172.21.15.37:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f8310032440 con 0x7f83180b4bb0 2024-04-25T10:30:44.581 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:44.579+0000 7f8317fff640 1 -- 172.21.15.37:0/3940256256 <== mon.0 v2:172.21.15.37:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8310032810 con 0x7f83180b4bb0 2024-04-25T10:30:44.582 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:44.579+0000 7f831f5f3640 1 -- 172.21.15.37:0/3940256256 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f83180b4bb0 msgr2=0x7f83181646b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:30:44.582 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:44.579+0000 7f831f5f3640 1 --2- 172.21.15.37:0/3940256256 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f83180b4bb0 0x7f83181646b0 secure :-1 s=READY pgs=592 cs=0 l=1 rev1=1 crypto rx=0x7f83100097a0 tx=0x7f831002a1e0 comp rx=0 tx=0).stop 2024-04-25T10:30:44.582 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:44.579+0000 7f831f5f3640 1 -- 172.21.15.37:0/3940256256 shutdown_connections 2024-04-25T10:30:44.582 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:44.579+0000 7f831f5f3640 1 --2- 172.21.15.37:0/3940256256 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f83180b4bb0 0x7f83181646b0 unknown :-1 s=CLOSED pgs=592 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:30:44.582 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:44.579+0000 7f831f5f3640 1 -- 172.21.15.37:0/3940256256 >> 172.21.15.37:0/3940256256 conn(0x7f83180d27e0 msgr2=0x7f83180d2bf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:30:44.582 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:44.579+0000 7f831f5f3640 1 -- 172.21.15.37:0/3940256256 shutdown_connections 2024-04-25T10:30:44.582 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:44.579+0000 7f831f5f3640 1 -- 172.21.15.37:0/3940256256 wait complete. 2024-04-25T10:30:44.583 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:44.581+0000 7f831f5f3640 1 Processor -- start 2024-04-25T10:30:44.583 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:44.581+0000 7f831f5f3640 1 -- start start 2024-04-25T10:30:44.585 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:44.582+0000 7f831f5f3640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f83180b4bb0 0x7f83181eb950 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:30:44.585 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:44.582+0000 7f831f5f3640 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f83180b7600 0x7f831815a0d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:30:44.585 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:44.582+0000 7f831f5f3640 1 --2- >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f83180d4560 0x7f83181e9b60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:30:44.585 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:44.582+0000 7f831f5f3640 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 -- 0x7f8318164bf0 con 0x7f83180b4bb0 2024-04-25T10:30:44.585 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:44.582+0000 7f831f5f3640 1 -- --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_getmap magic: 0 -- 0x7f83180b4fc0 con 0x7f83180b7600 2024-04-25T10:30:44.585 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:44.582+0000 7f831f5f3640 1 -- --> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] -- mon_getmap magic: 0 -- 0x7f83180d09e0 con 0x7f83180d4560 2024-04-25T10:30:44.585 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:44.583+0000 7f831d38f640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f83180b4bb0 0x7f83181eb950 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:30:44.585 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:44.583+0000 7f831d38f640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f83180b4bb0 0x7f83181eb950 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.37:3300/0 says I am v2:172.21.15.37:58284/0 (socket says 172.21.15.37:58284) 2024-04-25T10:30:44.585 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:44.583+0000 7f831d38f640 1 -- 172.21.15.37:0/3294705464 learned_addr learned my addr 172.21.15.37:0/3294705464 (peer_addr_for_me v2:172.21.15.37:0/0) 2024-04-25T10:30:44.586 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:44.583+0000 7f831d38f640 1 -- 172.21.15.37:0/3294705464 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f83180d4560 msgr2=0x7f83181e9b60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:30:44.586 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:44.583+0000 7f831d38f640 1 --2- 172.21.15.37:0/3294705464 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f83180d4560 0x7f83181e9b60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:30:44.586 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:44.583+0000 7f831d38f640 1 -- 172.21.15.37:0/3294705464 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f83180b7600 msgr2=0x7f831815a0d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:30:44.586 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:44.583+0000 7f831d38f640 1 --2- 172.21.15.37:0/3294705464 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f83180b7600 0x7f831815a0d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:30:44.586 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:44.583+0000 7f831d38f640 1 -- 172.21.15.37:0/3294705464 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f83181ea8b0 con 0x7f83180b4bb0 2024-04-25T10:30:44.586 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:44.583+0000 7f831d38f640 1 --2- 172.21.15.37:0/3294705464 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f83180b4bb0 0x7f83181eb950 secure :-1 s=READY pgs=593 cs=0 l=1 rev1=1 crypto rx=0x7f8310035040 tx=0x7f83100046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:30:44.586 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:44.583+0000 7f8315ffb640 1 -- 172.21.15.37:0/3294705464 <== mon.0 v2:172.21.15.37:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8310037040 con 0x7f83180b4bb0 2024-04-25T10:30:44.586 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:44.583+0000 7f8315ffb640 1 -- 172.21.15.37:0/3294705464 <== mon.0 v2:172.21.15.37:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f8310004250 con 0x7f83180b4bb0 2024-04-25T10:30:44.586 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:44.583+0000 7f831f5f3640 1 -- 172.21.15.37:0/3294705464 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f83181eab90 con 0x7f83180b4bb0 2024-04-25T10:30:44.586 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:44.583+0000 7f8315ffb640 1 -- 172.21.15.37:0/3294705464 <== mon.0 v2:172.21.15.37:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8310032e20 con 0x7f83180b4bb0 2024-04-25T10:30:44.586 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:44.584+0000 7f831f5f3640 1 -- 172.21.15.37:0/3294705464 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f83181eb150 con 0x7f83180b4bb0 2024-04-25T10:30:44.589 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:44.585+0000 7f831f5f3640 1 -- 172.21.15.37:0/3294705464 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f831806a140 con 0x7f83180b4bb0 2024-04-25T10:30:44.589 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:44.585+0000 7f8315ffb640 1 -- 172.21.15.37:0/3294705464 <== mon.0 v2:172.21.15.37:3300/0 4 ==== mgrmap(e 5) ==== 96782+0+0 (secure 0 0 0) 0x7f8310030770 con 0x7f83180b4bb0 2024-04-25T10:30:44.589 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:44.586+0000 7f8315ffb640 1 --2- 172.21.15.37:0/3294705464 >> [v2:172.21.15.203:6832/2418205237,v1:172.21.15.203:6833/2418205237] conn(0x7f83040744b0 0x7f83040769c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:30:44.589 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:44.586+0000 7f8315ffb640 1 -- 172.21.15.37:0/3294705464 <== mon.0 v2:172.21.15.37:3300/0 5 ==== osd_map(44..44 src has 1..44) ==== 6248+0+0 (secure 0 0 0) 0x7f83100b6750 con 0x7f83180b4bb0 2024-04-25T10:30:44.589 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:44.586+0000 7f831cb8e640 1 --2- 172.21.15.37:0/3294705464 >> [v2:172.21.15.203:6832/2418205237,v1:172.21.15.203:6833/2418205237] conn(0x7f83040744b0 0x7f83040769c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:30:44.589 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:44.587+0000 7f831cb8e640 1 --2- 172.21.15.37:0/3294705464 >> [v2:172.21.15.203:6832/2418205237,v1:172.21.15.203:6833/2418205237] conn(0x7f83040744b0 0x7f83040769c0 secure :-1 s=READY pgs=275 cs=0 l=1 rev1=1 crypto rx=0x7f830c007fd0 tx=0x7f830c007960 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:30:44.591 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:44.591+0000 7f8315ffb640 1 -- 172.21.15.37:0/3294705464 <== mon.0 v2:172.21.15.37:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187246 (secure 0 0 0) 0x7f8310080cc0 con 0x7f83180b4bb0 2024-04-25T10:30:44.779 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:44.777+0000 7f831f5f3640 1 -- 172.21.15.37:0/3294705464 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_command([{prefix=config set, name=debug_mds}] v 0) -- 0x7f83180d3ff0 con 0x7f83180b4bb0 2024-04-25T10:30:44.782 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:44.781+0000 7f8315ffb640 1 -- 172.21.15.37:0/3294705464 <== mon.0 v2:172.21.15.37:3300/0 7 ==== mon_command_ack([{prefix=config set, name=debug_mds}]=0 v11)=0 v11) ==== 108+0+0 (secure 0 0 0) 0x7f83100805c0 con 0x7f83180b4bb0 2024-04-25T10:30:44.785 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:44.783+0000 7f831f5f3640 1 -- 172.21.15.37:0/3294705464 >> [v2:172.21.15.203:6832/2418205237,v1:172.21.15.203:6833/2418205237] conn(0x7f83040744b0 msgr2=0x7f83040769c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:30:44.785 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:44.783+0000 7f831f5f3640 1 --2- 172.21.15.37:0/3294705464 >> [v2:172.21.15.203:6832/2418205237,v1:172.21.15.203:6833/2418205237] conn(0x7f83040744b0 0x7f83040769c0 secure :-1 s=READY pgs=275 cs=0 l=1 rev1=1 crypto rx=0x7f830c007fd0 tx=0x7f830c007960 comp rx=0 tx=0).stop 2024-04-25T10:30:44.785 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:44.783+0000 7f831f5f3640 1 -- 172.21.15.37:0/3294705464 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f83180b4bb0 msgr2=0x7f83181eb950 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:30:44.785 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:44.783+0000 7f831f5f3640 1 --2- 172.21.15.37:0/3294705464 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f83180b4bb0 0x7f83181eb950 secure :-1 s=READY pgs=593 cs=0 l=1 rev1=1 crypto rx=0x7f8310035040 tx=0x7f83100046c0 comp rx=0 tx=0).stop 2024-04-25T10:30:44.785 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:44.784+0000 7f831f5f3640 1 -- 172.21.15.37:0/3294705464 shutdown_connections 2024-04-25T10:30:44.785 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:44.784+0000 7f831f5f3640 1 --2- 172.21.15.37:0/3294705464 >> [v2:172.21.15.203:6832/2418205237,v1:172.21.15.203:6833/2418205237] conn(0x7f83040744b0 0x7f83040769c0 unknown :-1 s=CLOSED pgs=275 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:30:44.786 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:44.784+0000 7f831f5f3640 1 --2- 172.21.15.37:0/3294705464 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f83180d4560 0x7f83181e9b60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:30:44.786 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:44.784+0000 7f831f5f3640 1 --2- 172.21.15.37:0/3294705464 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f83180b7600 0x7f831815a0d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:30:44.786 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:44.784+0000 7f831f5f3640 1 --2- 172.21.15.37:0/3294705464 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f83180b4bb0 0x7f83181eb950 unknown :-1 s=CLOSED pgs=593 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:30:44.786 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:44.784+0000 7f831f5f3640 1 -- 172.21.15.37:0/3294705464 >> 172.21.15.37:0/3294705464 conn(0x7f83180d27e0 msgr2=0x7f83180d4970 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:30:44.786 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:44.784+0000 7f831f5f3640 1 -- 172.21.15.37:0/3294705464 shutdown_connections 2024-04-25T10:30:44.786 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:44.784+0000 7f831f5f3640 1 -- 172.21.15.37:0/3294705464 wait complete. 2024-04-25T10:30:44.796 DEBUG:teuthology.orchestra.run.smithi037:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.3:0 scrub start / force,recursive,repair 2024-04-25T10:30:44.913 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:44.912+0000 7f483ca84640 1 Processor -- start 2024-04-25T10:30:44.913 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:44.912+0000 7f483ca84640 1 -- start start 2024-04-25T10:30:44.915 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:44.913+0000 7f483ca84640 1 --2- >> v2:172.21.15.37:3300/0 conn(0x7f48380b79f0 0x7f48380b7e00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:30:44.915 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:44.913+0000 7f483ca84640 1 -- --> v1:172.21.15.37:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f48380d5540 con 0x7f48380d0940 2024-04-25T10:30:44.915 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:44.913+0000 7f483ca84640 1 -- --> v1:172.21.15.203:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f48380b8340 con 0x7f4838160260 2024-04-25T10:30:44.915 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:44.913+0000 7f483ca84640 1 -- --> v2:172.21.15.37:3300/0 -- mon_getmap magic: 0 -- 0x7f48380d0d50 con 0x7f48380b79f0 2024-04-25T10:30:44.915 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:44.913+0000 7f4836d9d640 1 --2- >> v2:172.21.15.37:3300/0 conn(0x7f48380b79f0 0x7f48380b7e00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:30:44.915 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:44.913+0000 7f483659c640 1 --1- >> v1:172.21.15.37:6789/0 conn(0x7f48380d0940 0x7f48381647b0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.37:6789/0 says I am v1:172.21.15.37:55666/0 (socket says 172.21.15.37:55666) 2024-04-25T10:30:44.916 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:44.914+0000 7f483659c640 1 -- 172.21.15.37:0/1019099132 learned_addr learned my addr 172.21.15.37:0/1019099132 (peer_addr_for_me v1:172.21.15.37:0/0) 2024-04-25T10:30:44.916 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:44.914+0000 7f4836d9d640 1 -- 172.21.15.37:0/1019099132 >> v1:172.21.15.203:6789/0 conn(0x7f4838160260 legacy=0x7f48380d4e30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:30:44.916 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:44.914+0000 7f4836d9d640 1 -- 172.21.15.37:0/1019099132 >> v1:172.21.15.37:6789/0 conn(0x7f48380d0940 legacy=0x7f48381647b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:30:44.916 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:44.914+0000 7f4836d9d640 1 -- 172.21.15.37:0/1019099132 --> v2:172.21.15.37:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f48380d0f70 con 0x7f48380b79f0 2024-04-25T10:30:44.916 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:44.914+0000 7f4835d9b640 1 -- 172.21.15.37:0/1019099132 <== mon.0 v1:172.21.15.37:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 2723064447 0 0) 0x7f48380d5540 con 0x7f48380d0940 2024-04-25T10:30:44.917 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:44.915+0000 7f4836d9d640 1 --2- 172.21.15.37:0/1019099132 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f48380b79f0 0x7f48380b7e00 secure :-1 s=READY pgs=595 cs=0 l=1 rev1=1 crypto rx=0x7f4830009d90 tx=0x7f483002a6a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=888d2b71360bb946 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:30:44.917 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:44.915+0000 7f4835d9b640 1 -- 172.21.15.37:0/1019099132 <== mon.0 v2:172.21.15.37:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4830007740 con 0x7f48380b79f0 2024-04-25T10:30:44.917 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:44.915+0000 7f4835d9b640 1 -- 172.21.15.37:0/1019099132 <== mon.0 v2:172.21.15.37:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f48300078e0 con 0x7f48380b79f0 2024-04-25T10:30:44.917 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:44.915+0000 7f4835d9b640 1 -- 172.21.15.37:0/1019099132 <== mon.0 v2:172.21.15.37:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4830007cb0 con 0x7f48380b79f0 2024-04-25T10:30:44.918 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:44.915+0000 7f483ca84640 1 -- 172.21.15.37:0/1019099132 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f48380b79f0 msgr2=0x7f48380b7e00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:30:44.918 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:44.915+0000 7f483ca84640 1 --2- 172.21.15.37:0/1019099132 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f48380b79f0 0x7f48380b7e00 secure :-1 s=READY pgs=595 cs=0 l=1 rev1=1 crypto rx=0x7f4830009d90 tx=0x7f483002a6a0 comp rx=0 tx=0).stop 2024-04-25T10:30:44.918 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:44.915+0000 7f483ca84640 1 -- 172.21.15.37:0/1019099132 shutdown_connections 2024-04-25T10:30:44.918 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:44.915+0000 7f483ca84640 1 --2- 172.21.15.37:0/1019099132 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f48380b79f0 0x7f48380b7e00 unknown :-1 s=CLOSED pgs=595 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:30:44.918 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:44.915+0000 7f483ca84640 1 -- 172.21.15.37:0/1019099132 >> 172.21.15.37:0/1019099132 conn(0x7f48380d5d70 msgr2=0x7f48380d6180 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:30:44.918 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:44.915+0000 7f483ca84640 1 -- 172.21.15.37:0/1019099132 shutdown_connections 2024-04-25T10:30:44.918 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:44.915+0000 7f483ca84640 1 -- 172.21.15.37:0/1019099132 wait complete. 2024-04-25T10:30:44.919 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:44.917+0000 7f483ca84640 1 Processor -- start 2024-04-25T10:30:44.919 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:44.917+0000 7f483ca84640 1 -- start start 2024-04-25T10:30:44.922 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:44.919+0000 7f483ca84640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f48380b79f0 0x7f48381f63c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:30:44.922 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:44.919+0000 7f483ca84640 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f48380d0940 0x7f48381f7dd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:30:44.922 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:44.919+0000 7f483ca84640 1 --2- >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f4838160260 0x7f48381f8310 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:30:44.923 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:44.919+0000 7f483ca84640 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 -- 0x7f48380d4910 con 0x7f48380b79f0 2024-04-25T10:30:44.923 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:44.919+0000 7f483ca84640 1 -- --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_getmap magic: 0 -- 0x7f48380d46f0 con 0x7f48380d0940 2024-04-25T10:30:44.923 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:44.919+0000 7f483ca84640 1 -- --> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] -- mon_getmap magic: 0 -- 0x7f4838160670 con 0x7f4838160260 2024-04-25T10:30:44.923 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:44.919+0000 7f483659c640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f48380b79f0 0x7f48381f63c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:30:44.923 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:44.919+0000 7f483659c640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f48380b79f0 0x7f48381f63c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.37:3300/0 says I am v2:172.21.15.37:58300/0 (socket says 172.21.15.37:58300) 2024-04-25T10:30:44.923 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:44.919+0000 7f483659c640 1 -- 172.21.15.37:0/3542185943 learned_addr learned my addr 172.21.15.37:0/3542185943 (peer_addr_for_me v2:172.21.15.37:0/0) 2024-04-25T10:30:44.923 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:44.919+0000 7f4836d9d640 1 --2- 172.21.15.37:0/3542185943 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f4838160260 0x7f48381f8310 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:30:44.923 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:44.919+0000 7f482ffff640 1 --2- 172.21.15.37:0/3542185943 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f48380d0940 0x7f48381f7dd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:30:44.923 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:44.919+0000 7f483659c640 1 -- 172.21.15.37:0/3542185943 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f4838160260 msgr2=0x7f48381f8310 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:30:44.924 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:44.919+0000 7f483659c640 1 --2- 172.21.15.37:0/3542185943 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f4838160260 0x7f48381f8310 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:30:44.924 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:44.919+0000 7f483659c640 1 -- 172.21.15.37:0/3542185943 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f48380d0940 msgr2=0x7f48381f7dd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:30:44.924 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:44.919+0000 7f483659c640 1 --2- 172.21.15.37:0/3542185943 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f48380d0940 0x7f48381f7dd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:30:44.924 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:44.919+0000 7f483659c640 1 -- 172.21.15.37:0/3542185943 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f48380cc180 con 0x7f48380b79f0 2024-04-25T10:30:44.924 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:44.919+0000 7f483659c640 1 --2- 172.21.15.37:0/3542185943 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f48380b79f0 0x7f48381f63c0 secure :-1 s=READY pgs=596 cs=0 l=1 rev1=1 crypto rx=0x7f482800c430 tx=0x7f482800c8f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:30:44.924 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:44.920+0000 7f482effd640 1 -- 172.21.15.37:0/3542185943 <== mon.0 v2:172.21.15.37:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f48280074e0 con 0x7f48380b79f0 2024-04-25T10:30:44.924 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:44.920+0000 7f482effd640 1 -- 172.21.15.37:0/3542185943 <== mon.0 v2:172.21.15.37:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f48280047e0 con 0x7f48380b79f0 2024-04-25T10:30:44.924 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:44.920+0000 7f483ca84640 1 -- 172.21.15.37:0/3542185943 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f48380cc460 con 0x7f48380b79f0 2024-04-25T10:30:44.924 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:44.920+0000 7f482effd640 1 -- 172.21.15.37:0/3542185943 <== mon.0 v2:172.21.15.37:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4828004bd0 con 0x7f48380b79f0 2024-04-25T10:30:44.924 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:44.920+0000 7f483ca84640 1 -- 172.21.15.37:0/3542185943 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f4838153d00 con 0x7f48380b79f0 2024-04-25T10:30:44.928 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:44.921+0000 7f482effd640 1 -- 172.21.15.37:0/3542185943 <== mon.0 v2:172.21.15.37:3300/0 4 ==== mgrmap(e 5) ==== 96782+0+0 (secure 0 0 0) 0x7f4828011070 con 0x7f48380b79f0 2024-04-25T10:30:44.928 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:44.922+0000 7f482effd640 1 --2- 172.21.15.37:0/3542185943 >> [v2:172.21.15.203:6832/2418205237,v1:172.21.15.203:6833/2418205237] conn(0x7f48180744b0 0x7f48180769c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:30:44.928 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:44.922+0000 7f482effd640 1 -- 172.21.15.37:0/3542185943 <== mon.0 v2:172.21.15.37:3300/0 5 ==== osd_map(44..44 src has 1..44) ==== 6248+0+0 (secure 0 0 0) 0x7f4828094e80 con 0x7f48380b79f0 2024-04-25T10:30:44.928 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:44.922+0000 7f482ffff640 1 --2- 172.21.15.37:0/3542185943 >> [v2:172.21.15.203:6832/2418205237,v1:172.21.15.203:6833/2418205237] conn(0x7f48180744b0 0x7f48180769c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:30:44.928 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:44.923+0000 7f482ffff640 1 --2- 172.21.15.37:0/3542185943 >> [v2:172.21.15.203:6832/2418205237,v1:172.21.15.203:6833/2418205237] conn(0x7f48180744b0 0x7f48180769c0 secure :-1 s=READY pgs=276 cs=0 l=1 rev1=1 crypto rx=0x7f48381638d0 tx=0x7f4820009060 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:30:44.928 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:44.923+0000 7f4843a0b740 1 Processor -- start 2024-04-25T10:30:44.929 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:44.923+0000 7f4843a0b740 1 -- start start 2024-04-25T10:30:44.929 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:44.924+0000 7f4843a0b740 1 --2- >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x55d0a73dc4d0 0x55d0a7777c50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:30:44.929 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:44.924+0000 7f4843a0b740 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55d0a73b3610 0x55d0a777b610 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:30:44.929 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:44.924+0000 7f4843a0b740 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x55d0a73bb7f0 0x55d0a77a7af0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:30:44.929 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:44.924+0000 7f4843a0b740 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 -- 0x55d0a76e2a10 con 0x55d0a73b3610 2024-04-25T10:30:44.929 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:44.924+0000 7f4843a0b740 1 -- --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_getmap magic: 0 -- 0x55d0a767e7d0 con 0x55d0a73bb7f0 2024-04-25T10:30:44.929 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:44.924+0000 7f4843a0b740 1 -- --> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] -- mon_getmap magic: 0 -- 0x55d0a76de9d0 con 0x55d0a73dc4d0 2024-04-25T10:30:44.929 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:44.925+0000 7f482ffff640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55d0a73b3610 0x55d0a777b610 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:30:44.929 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:44.925+0000 7f482ffff640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55d0a73b3610 0x55d0a777b610 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.37:3300/0 says I am v2:172.21.15.37:58304/0 (socket says 172.21.15.37:58304) 2024-04-25T10:30:44.929 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:44.925+0000 7f482ffff640 1 -- 172.21.15.37:0/3693116532 learned_addr learned my addr 172.21.15.37:0/3693116532 (peer_addr_for_me v2:172.21.15.37:0/0) 2024-04-25T10:30:44.929 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:44.925+0000 7f4836d9d640 1 --2- 172.21.15.37:0/3693116532 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x55d0a73bb7f0 0x55d0a77a7af0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:30:44.929 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:44.925+0000 7f483659c640 1 --2- 172.21.15.37:0/3693116532 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x55d0a73dc4d0 0x55d0a7777c50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:30:44.929 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:44.925+0000 7f482ffff640 1 -- 172.21.15.37:0/3693116532 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x55d0a73dc4d0 msgr2=0x55d0a7777c50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:30:44.930 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:44.925+0000 7f482ffff640 1 --2- 172.21.15.37:0/3693116532 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x55d0a73dc4d0 0x55d0a7777c50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:30:44.930 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:44.925+0000 7f482ffff640 1 -- 172.21.15.37:0/3693116532 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x55d0a73bb7f0 msgr2=0x55d0a77a7af0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:30:44.930 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:44.925+0000 7f482ffff640 1 --2- 172.21.15.37:0/3693116532 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x55d0a73bb7f0 0x55d0a77a7af0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:30:44.930 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:44.925+0000 7f482ffff640 1 -- 172.21.15.37:0/3693116532 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55d0a74c0940 con 0x55d0a73b3610 2024-04-25T10:30:44.930 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:44.925+0000 7f483659c640 1 --2- 172.21.15.37:0/3693116532 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x55d0a73dc4d0 0x55d0a7777c50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T10:30:44.930 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:44.925+0000 7f482ffff640 1 --2- 172.21.15.37:0/3693116532 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55d0a73b3610 0x55d0a777b610 secure :-1 s=READY pgs=597 cs=0 l=1 rev1=1 crypto rx=0x7f482000eb40 tx=0x7f4820011000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f146a9ae67452191 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:30:44.930 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:44.925+0000 7f4817fff640 1 -- 172.21.15.37:0/3693116532 <== mon.0 v2:172.21.15.37:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f482001b040 con 0x55d0a73b3610 2024-04-25T10:30:44.930 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:44.925+0000 7f4817fff640 1 -- 172.21.15.37:0/3693116532 <== mon.0 v2:172.21.15.37:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f4820024070 con 0x55d0a73b3610 2024-04-25T10:30:44.930 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:44.925+0000 7f4817fff640 1 -- 172.21.15.37:0/3693116532 <== mon.0 v2:172.21.15.37:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4820015c40 con 0x55d0a73b3610 2024-04-25T10:30:44.930 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:44.926+0000 7f4843a0b740 1 -- 172.21.15.37:0/3693116532 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55d0a73b3610 msgr2=0x55d0a777b610 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:30:44.930 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:44.926+0000 7f4843a0b740 1 --2- 172.21.15.37:0/3693116532 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55d0a73b3610 0x55d0a777b610 secure :-1 s=READY pgs=597 cs=0 l=1 rev1=1 crypto rx=0x7f482000eb40 tx=0x7f4820011000 comp rx=0 tx=0).stop 2024-04-25T10:30:44.930 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:44.926+0000 7f4843a0b740 1 -- 172.21.15.37:0/3693116532 shutdown_connections 2024-04-25T10:30:44.930 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:44.926+0000 7f4843a0b740 1 --2- 172.21.15.37:0/3693116532 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x55d0a73bb7f0 0x55d0a77a7af0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:30:44.931 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:44.926+0000 7f4843a0b740 1 --2- 172.21.15.37:0/3693116532 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55d0a73b3610 0x55d0a777b610 unknown :-1 s=CLOSED pgs=597 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:30:44.931 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:44.926+0000 7f4843a0b740 1 --2- 172.21.15.37:0/3693116532 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x55d0a73dc4d0 0x55d0a7777c50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:30:44.931 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:44.926+0000 7f4843a0b740 1 -- 172.21.15.37:0/3693116532 >> 172.21.15.37:0/3693116532 conn(0x55d0a73d0620 msgr2=0x55d0a777d170 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:30:44.931 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:44.926+0000 7f4843a0b740 1 -- 172.21.15.37:0/3693116532 shutdown_connections 2024-04-25T10:30:44.931 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:44.926+0000 7f4843a0b740 1 -- 172.21.15.37:0/3693116532 wait complete. 2024-04-25T10:30:44.931 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:44.927+0000 7f4843a0b740 1 -- start start 2024-04-25T10:30:44.931 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:44.927+0000 7f4843a0b740 1 Processor -- start 2024-04-25T10:30:44.935 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:44.929+0000 7f4843a0b740 1 --2- >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x55d0a73b3610 0x55d0a7776830 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:30:44.935 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:44.929+0000 7f4843a0b740 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55d0a73bb7f0 0x55d0a7776d70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:30:44.935 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:44.929+0000 7f4843a0b740 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x55d0a73dc4d0 0x55d0a77a91d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:30:44.935 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:44.929+0000 7f4843a0b740 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 -- 0x55d0a77a5560 con 0x55d0a73bb7f0 2024-04-25T10:30:44.935 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:44.929+0000 7f4843a0b740 1 -- --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_getmap magic: 0 -- 0x55d0a767c4a0 con 0x55d0a73dc4d0 2024-04-25T10:30:44.935 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:44.929+0000 7f4843a0b740 1 -- --> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] -- mon_getmap magic: 0 -- 0x55d0a7778190 con 0x55d0a73b3610 2024-04-25T10:30:44.935 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:44.929+0000 7f4843a0b740 10 client.0 _ll_register_callbacks cb 0x55d0a768add0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-25T10:30:44.935 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:44.929+0000 7f482ffff640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55d0a73bb7f0 0x55d0a7776d70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:30:44.936 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:44.929+0000 7f482ffff640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55d0a73bb7f0 0x55d0a7776d70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.37:3300/0 says I am v2:172.21.15.37:58316/0 (socket says 172.21.15.37:58316) 2024-04-25T10:30:44.936 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:44.929+0000 7f482ffff640 1 -- 172.21.15.37:0/1046230904 learned_addr learned my addr 172.21.15.37:0/1046230904 (peer_addr_for_me v2:172.21.15.37:0/0) 2024-04-25T10:30:44.936 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:44.929+0000 7f483659c640 1 --2- 172.21.15.37:0/1046230904 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x55d0a73b3610 0x55d0a7776830 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:30:44.936 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:44.929+0000 7f4836d9d640 1 --2- 172.21.15.37:0/1046230904 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x55d0a73dc4d0 0x55d0a77a91d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:30:44.936 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:44.929+0000 7f482ffff640 1 -- 172.21.15.37:0/1046230904 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x55d0a73b3610 msgr2=0x55d0a7776830 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:30:44.936 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:44.929+0000 7f482ffff640 1 --2- 172.21.15.37:0/1046230904 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x55d0a73b3610 0x55d0a7776830 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:30:44.936 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:44.929+0000 7f482ffff640 1 -- 172.21.15.37:0/1046230904 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x55d0a73dc4d0 msgr2=0x55d0a77a91d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:30:44.936 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:44.929+0000 7f482ffff640 1 --2- 172.21.15.37:0/1046230904 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x55d0a73dc4d0 0x55d0a77a91d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:30:44.936 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:44.929+0000 7f482ffff640 1 -- 172.21.15.37:0/1046230904 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55d0a75210d0 con 0x55d0a73bb7f0 2024-04-25T10:30:44.936 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:44.929+0000 7f4836d9d640 1 --2- 172.21.15.37:0/1046230904 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x55d0a73dc4d0 0x55d0a77a91d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T10:30:44.936 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:44.929+0000 7f483659c640 1 --2- 172.21.15.37:0/1046230904 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x55d0a73b3610 0x55d0a7776830 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T10:30:44.936 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:44.930+0000 7f482ffff640 1 --2- 172.21.15.37:0/1046230904 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55d0a73bb7f0 0x55d0a7776d70 secure :-1 s=READY pgs=598 cs=0 l=1 rev1=1 crypto rx=0x7f4820006d10 tx=0x7f4820004680 comp rx=0 tx=0).ready entity=mon.0 client_cookie=4ac43d72882cade3 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:30:44.936 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:44.930+0000 7f48157fa640 10 client.0 ms_handle_connect on v2:172.21.15.37:3300/0 2024-04-25T10:30:44.936 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:44.930+0000 7f48157fa640 1 -- 172.21.15.37:0/1046230904 <== mon.0 v2:172.21.15.37:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4820032630 con 0x55d0a73bb7f0 2024-04-25T10:30:44.937 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:44.930+0000 7f48157fa640 1 -- 172.21.15.37:0/1046230904 <== mon.0 v2:172.21.15.37:3300/0 2 ==== osd_map(44..44 src has 1..44) ==== 6248+0+0 (secure 0 0 0) 0x7f4820029400 con 0x55d0a73bb7f0 2024-04-25T10:30:44.937 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:44.930+0000 7f4843a0b740 1 -- 172.21.15.37:0/1046230904 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f4820024070 con 0x55d0a73bb7f0 2024-04-25T10:30:44.937 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:44.930+0000 7f48157fa640 1 -- 172.21.15.37:0/1046230904 <== mon.0 v2:172.21.15.37:3300/0 3 ==== mon_get_version_reply(handle=1 version=27) ==== 24+0+0 (secure 0 0 0) 0x7f48200327d0 con 0x55d0a73bb7f0 2024-04-25T10:30:44.937 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:44.930+0000 7f4843a0b740 10 client.5928 fetch_fsmap learned FSMap version 27 2024-04-25T10:30:44.937 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:44.930+0000 7f4843a0b740 1 -- 172.21.15.37:0/1046230904 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({config=0+,fsmap=27,monmap=2+}) -- 0x55d0a73b4230 con 0x55d0a73bb7f0 2024-04-25T10:30:44.937 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:44.930+0000 7f48157fa640 1 -- 172.21.15.37:0/1046230904 <== mon.0 v2:172.21.15.37:3300/0 4 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f4820024070 con 0x55d0a73bb7f0 2024-04-25T10:30:44.937 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:44.930+0000 7f48157fa640 1 -- 172.21.15.37:0/1046230904 <== mon.0 v2:172.21.15.37:3300/0 5 ==== fsmap(e 27) ==== 4139+0+0 (secure 0 0 0) 0x7f482001e030 con 0x55d0a73bb7f0 2024-04-25T10:30:44.937 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:44.930+0000 7f4843a0b740 10 client.5928 fetch_fsmap finished waiting for FSMap version 27 2024-04-25T10:30:44.937 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:44.930+0000 7f4843a0b740 10 client.5928 resolve_mds: resolved 3:0 to role '3:0' aka daemon mds.c 2024-04-25T10:30:44.937 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:44.930+0000 7f4843a0b740 20 client.5928 populate_metadata read hostname 'smithi037' 2024-04-25T10:30:44.937 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:44.930+0000 7f4843a0b740 1 --2- 172.21.15.37:0/1046230904 >> [v2:172.21.15.37:6832/2353203722,v1:172.21.15.37:6833/2353203722] conn(0x55d0a739b2b0 0x55d0a77a9710 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:30:44.937 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:44.930+0000 7f4843a0b740 4 client.5928 mds_command: new command op to 5625 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-25T10:30:44.937 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:44.930+0000 7f4843a0b740 1 -- 172.21.15.37:0/1046230904 --> [v2:172.21.15.37:6832/2353203722,v1:172.21.15.37:6833/2353203722] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55d0a76b63a0 con 0x55d0a739b2b0 2024-04-25T10:30:44.937 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:44.930+0000 7f483659c640 1 --2- 172.21.15.37:0/1046230904 >> [v2:172.21.15.37:6832/2353203722,v1:172.21.15.37:6833/2353203722] conn(0x55d0a739b2b0 0x55d0a77a9710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:30:44.938 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:44.931+0000 7f48157fa640 10 client.5928 ms_handle_connect on v2:172.21.15.37:6832/2353203722 2024-04-25T10:30:44.938 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:44.931+0000 7f483659c640 1 --2- 172.21.15.37:0/1046230904 >> [v2:172.21.15.37:6832/2353203722,v1:172.21.15.37:6833/2353203722] conn(0x55d0a739b2b0 0x55d0a77a9710 crc :-1 s=READY pgs=14 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=78de679b3f5049ad server_cookie=b2b28fa687227e1e in_seq=0 out_seq=0 2024-04-25T10:30:44.938 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:44.931+0000 7f48157fa640 1 -- 172.21.15.37:0/1046230904 <== mds.0 v2:172.21.15.37:6832/2353203722 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23648 (crc 0 0 0) 0x55d0a76b63a0 con 0x55d0a739b2b0 2024-04-25T10:30:44.938 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:44.931+0000 7f48157fa640 10 client.5928 handle_command_reply: tid=0 2024-04-25T10:30:44.938 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:44.931+0000 7f4843a0b740 1 client.5928 shutdown 2024-04-25T10:30:44.938 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:44.932+0000 7f4843a0b740 1 -- 172.21.15.37:0/1046230904 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55d0a73bb7f0 msgr2=0x55d0a7776d70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:30:44.938 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:44.932+0000 7f4843a0b740 1 --2- 172.21.15.37:0/1046230904 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55d0a73bb7f0 0x55d0a7776d70 secure :-1 s=READY pgs=598 cs=0 l=1 rev1=1 crypto rx=0x7f4820006d10 tx=0x7f4820004680 comp rx=0 tx=0).stop 2024-04-25T10:30:44.938 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:44.932+0000 7f4843a0b740 1 -- 172.21.15.37:0/1046230904 shutdown_connections 2024-04-25T10:30:44.938 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:44.932+0000 7f4843a0b740 1 --2- 172.21.15.37:0/1046230904 >> [v2:172.21.15.37:6832/2353203722,v1:172.21.15.37:6833/2353203722] conn(0x55d0a739b2b0 0x55d0a77a9710 crc :-1 s=READY pgs=14 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:30:44.938 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:44.932+0000 7f4843a0b740 1 --2- 172.21.15.37:0/1046230904 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x55d0a73dc4d0 0x55d0a77a91d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:30:44.938 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:44.932+0000 7f4843a0b740 1 --2- 172.21.15.37:0/1046230904 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55d0a73bb7f0 0x55d0a7776d70 unknown :-1 s=CLOSED pgs=598 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:30:44.938 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:44.932+0000 7f48157fa640 0 client.5928 ms_handle_reset on v2:172.21.15.37:6832/2353203722 2024-04-25T10:30:44.938 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:44.932+0000 7f4843a0b740 1 --2- 172.21.15.37:0/1046230904 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x55d0a73b3610 0x55d0a7776830 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:30:44.938 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:44.932+0000 7f4843a0b740 1 -- 172.21.15.37:0/1046230904 >> 172.21.15.37:0/1046230904 conn(0x55d0a73d0620 msgr2=0x55d0a777d170 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:30:44.938 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:44.932+0000 7f4843a0b740 1 -- 172.21.15.37:0/1046230904 shutdown_connections 2024-04-25T10:30:44.939 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:44.932+0000 7f4843a0b740 1 -- 172.21.15.37:0/1046230904 wait complete. 2024-04-25T10:30:44.939 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:44.932+0000 7f4843a0b740 20 client.5928 trim_cache size 0 max 16384 2024-04-25T10:30:44.952 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:44.951+0000 7f4843a0b740 1 Processor -- start 2024-04-25T10:30:44.952 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:44.951+0000 7f4843a0b740 1 -- start start 2024-04-25T10:30:44.955 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:44.952+0000 7f4843a0b740 1 --2- >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x55d0a73b3610 0x55d0a77a25e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:30:44.955 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:44.952+0000 7f4843a0b740 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x55d0a73bb7f0 0x55d0a777b510 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:30:44.955 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:44.952+0000 7f4843a0b740 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55d0a73dc4d0 0x55d0a777ba50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:30:44.955 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:44.952+0000 7f4843a0b740 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 -- 0x7f4838204400 con 0x55d0a73dc4d0 2024-04-25T10:30:44.955 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:44.952+0000 7f4843a0b740 1 -- --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_getmap magic: 0 -- 0x7f4838202b20 con 0x55d0a73bb7f0 2024-04-25T10:30:44.955 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:44.952+0000 7f4843a0b740 1 -- --> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] -- mon_getmap magic: 0 -- 0x7f48381562c0 con 0x55d0a73b3610 2024-04-25T10:30:44.955 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:44.952+0000 7f4836d9d640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55d0a73dc4d0 0x55d0a777ba50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:30:44.956 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:44.952+0000 7f4836d9d640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55d0a73dc4d0 0x55d0a777ba50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.37:3300/0 says I am v2:172.21.15.37:58320/0 (socket says 172.21.15.37:58320) 2024-04-25T10:30:44.956 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:44.952+0000 7f4836d9d640 1 -- 172.21.15.37:0/3857117358 learned_addr learned my addr 172.21.15.37:0/3857117358 (peer_addr_for_me v2:172.21.15.37:0/0) 2024-04-25T10:30:44.956 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:44.953+0000 7f482ffff640 1 --2- 172.21.15.37:0/3857117358 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x55d0a73bb7f0 0x55d0a777b510 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:30:44.956 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:44.953+0000 7f483659c640 1 --2- 172.21.15.37:0/3857117358 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x55d0a73b3610 0x55d0a77a25e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:30:44.956 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:44.953+0000 7f4836d9d640 1 -- 172.21.15.37:0/3857117358 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x55d0a73b3610 msgr2=0x55d0a77a25e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:30:44.956 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:44.953+0000 7f4836d9d640 1 --2- 172.21.15.37:0/3857117358 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x55d0a73b3610 0x55d0a77a25e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:30:44.956 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:44.953+0000 7f4836d9d640 1 -- 172.21.15.37:0/3857117358 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x55d0a73bb7f0 msgr2=0x55d0a777b510 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:30:44.956 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:44.953+0000 7f4836d9d640 1 --2- 172.21.15.37:0/3857117358 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x55d0a73bb7f0 0x55d0a777b510 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:30:44.956 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:44.953+0000 7f4836d9d640 1 -- 172.21.15.37:0/3857117358 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55d0a77a9ad0 con 0x55d0a73dc4d0 2024-04-25T10:30:44.956 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:44.953+0000 7f4836d9d640 1 --2- 172.21.15.37:0/3857117358 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55d0a73dc4d0 0x55d0a777ba50 secure :-1 s=READY pgs=599 cs=0 l=1 rev1=1 crypto rx=0x7f4830009eb0 tx=0x7f483002e000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=7cab916054fe2f85 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:30:44.956 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:44.953+0000 7f48177fe640 1 -- 172.21.15.37:0/3857117358 <== mon.0 v2:172.21.15.37:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4830007c90 con 0x55d0a73dc4d0 2024-04-25T10:30:44.956 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:44.953+0000 7f48177fe640 1 -- 172.21.15.37:0/3857117358 <== mon.0 v2:172.21.15.37:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f4830007e30 con 0x55d0a73dc4d0 2024-04-25T10:30:44.956 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:44.953+0000 7f48177fe640 1 -- 172.21.15.37:0/3857117358 <== mon.0 v2:172.21.15.37:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f483003c640 con 0x55d0a73dc4d0 2024-04-25T10:30:44.956 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:44.954+0000 7f4843a0b740 1 -- 172.21.15.37:0/3857117358 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55d0a73dc4d0 msgr2=0x55d0a777ba50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:30:44.957 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:44.954+0000 7f4843a0b740 1 --2- 172.21.15.37:0/3857117358 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55d0a73dc4d0 0x55d0a777ba50 secure :-1 s=READY pgs=599 cs=0 l=1 rev1=1 crypto rx=0x7f4830009eb0 tx=0x7f483002e000 comp rx=0 tx=0).stop 2024-04-25T10:30:44.957 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:44.954+0000 7f4843a0b740 1 -- 172.21.15.37:0/3857117358 shutdown_connections 2024-04-25T10:30:44.957 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:44.954+0000 7f4843a0b740 1 --2- 172.21.15.37:0/3857117358 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55d0a73dc4d0 0x55d0a777ba50 unknown :-1 s=CLOSED pgs=599 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:30:44.957 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:44.954+0000 7f4843a0b740 1 --2- 172.21.15.37:0/3857117358 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x55d0a73bb7f0 0x55d0a777b510 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:30:44.957 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:44.954+0000 7f4843a0b740 1 --2- 172.21.15.37:0/3857117358 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x55d0a73b3610 0x55d0a77a25e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:30:44.957 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:44.954+0000 7f4843a0b740 1 -- 172.21.15.37:0/3857117358 >> 172.21.15.37:0/3857117358 conn(0x55d0a73d0620 msgr2=0x55d0a779a580 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:30:44.957 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:44.954+0000 7f4843a0b740 1 -- 172.21.15.37:0/3857117358 shutdown_connections 2024-04-25T10:30:44.957 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:44.954+0000 7f4843a0b740 1 -- 172.21.15.37:0/3857117358 wait complete. 2024-04-25T10:30:44.961 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:44.955+0000 7f4843a0b740 1 -- start start 2024-04-25T10:30:44.961 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:44.955+0000 7f4843a0b740 1 Processor -- start 2024-04-25T10:30:44.961 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:44.956+0000 7f4843a0b740 1 --2- >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x55d0a73b3610 0x55d0a78425b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:30:44.961 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:44.956+0000 7f4843a0b740 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55d0a73bb7f0 0x55d0a7842bc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:30:44.961 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:44.956+0000 7f4843a0b740 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x55d0a73dc4d0 0x55d0a77c52c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:30:44.962 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:44.956+0000 7f4843a0b740 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 -- 0x7f48382024d0 con 0x55d0a73bb7f0 2024-04-25T10:30:44.962 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:44.956+0000 7f4843a0b740 1 -- --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_getmap magic: 0 -- 0x55d0a779b110 con 0x55d0a73dc4d0 2024-04-25T10:30:44.962 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:44.956+0000 7f4843a0b740 1 -- --> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] -- mon_getmap magic: 0 -- 0x55d0a7843430 con 0x55d0a73b3610 2024-04-25T10:30:44.962 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:44.956+0000 7f4843a0b740 10 client.0 _ll_register_callbacks cb 0x7f4830009530 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-25T10:30:44.962 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:44.956+0000 7f482ffff640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55d0a73bb7f0 0x55d0a7842bc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:30:44.962 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:44.956+0000 7f482ffff640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55d0a73bb7f0 0x55d0a7842bc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.37:3300/0 says I am v2:172.21.15.37:58336/0 (socket says 172.21.15.37:58336) 2024-04-25T10:30:44.962 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:44.956+0000 7f482ffff640 1 -- 172.21.15.37:0/2287319098 learned_addr learned my addr 172.21.15.37:0/2287319098 (peer_addr_for_me v2:172.21.15.37:0/0) 2024-04-25T10:30:44.962 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:44.957+0000 7f483659c640 1 --2- 172.21.15.37:0/2287319098 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x55d0a73b3610 0x55d0a78425b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:30:44.962 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:44.957+0000 7f4836d9d640 1 --2- 172.21.15.37:0/2287319098 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x55d0a73dc4d0 0x55d0a77c52c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:30:44.962 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:44.957+0000 7f482ffff640 1 -- 172.21.15.37:0/2287319098 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x55d0a73b3610 msgr2=0x55d0a78425b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:30:44.962 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:44.957+0000 7f482ffff640 1 --2- 172.21.15.37:0/2287319098 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x55d0a73b3610 0x55d0a78425b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:30:44.962 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:44.957+0000 7f482ffff640 1 -- 172.21.15.37:0/2287319098 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x55d0a73dc4d0 msgr2=0x55d0a77c52c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:30:44.962 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:44.957+0000 7f482ffff640 1 --2- 172.21.15.37:0/2287319098 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x55d0a73dc4d0 0x55d0a77c52c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:30:44.962 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:44.957+0000 7f482ffff640 1 -- 172.21.15.37:0/2287319098 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55d0a77bbb10 con 0x55d0a73bb7f0 2024-04-25T10:30:44.962 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:44.957+0000 7f4836d9d640 1 --2- 172.21.15.37:0/2287319098 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x55d0a73dc4d0 0x55d0a77c52c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T10:30:44.963 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:44.957+0000 7f482ffff640 1 --2- 172.21.15.37:0/2287319098 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55d0a73bb7f0 0x55d0a7842bc0 secure :-1 s=READY pgs=600 cs=0 l=1 rev1=1 crypto rx=0x7f4820010010 tx=0x7f4820015000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=84bf266f2fee3ccb server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:30:44.963 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:44.957+0000 7f48157fa640 10 client.0 ms_handle_connect on v2:172.21.15.37:3300/0 2024-04-25T10:30:44.963 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:44.957+0000 7f48157fa640 1 -- 172.21.15.37:0/2287319098 <== mon.0 v2:172.21.15.37:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4820044060 con 0x55d0a73bb7f0 2024-04-25T10:30:44.963 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:44.957+0000 7f48157fa640 1 -- 172.21.15.37:0/2287319098 <== mon.0 v2:172.21.15.37:3300/0 2 ==== osd_map(44..44 src has 1..44) ==== 6248+0+0 (secure 0 0 0) 0x7f4820004240 con 0x55d0a73bb7f0 2024-04-25T10:30:44.963 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:44.957+0000 7f4843a0b740 1 -- 172.21.15.37:0/2287319098 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f4830007e30 con 0x55d0a73bb7f0 2024-04-25T10:30:44.963 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:44.957+0000 7f48157fa640 1 -- 172.21.15.37:0/2287319098 <== mon.0 v2:172.21.15.37:3300/0 3 ==== mon_get_version_reply(handle=1 version=27) ==== 24+0+0 (secure 0 0 0) 0x7f482004a050 con 0x55d0a73bb7f0 2024-04-25T10:30:44.963 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:44.957+0000 7f4843a0b740 10 client.5934 fetch_fsmap learned FSMap version 27 2024-04-25T10:30:44.963 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:44.957+0000 7f4843a0b740 1 -- 172.21.15.37:0/2287319098 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({config=0+,fsmap=27,monmap=2+}) -- 0x55d0a7843160 con 0x55d0a73bb7f0 2024-04-25T10:30:44.963 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:44.957+0000 7f48157fa640 1 -- 172.21.15.37:0/2287319098 <== mon.0 v2:172.21.15.37:3300/0 4 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f4830007e30 con 0x55d0a73bb7f0 2024-04-25T10:30:44.963 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:44.958+0000 7f48157fa640 1 -- 172.21.15.37:0/2287319098 <== mon.0 v2:172.21.15.37:3300/0 5 ==== fsmap(e 27) ==== 4139+0+0 (secure 0 0 0) 0x7f482004d020 con 0x55d0a73bb7f0 2024-04-25T10:30:44.963 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:44.958+0000 7f4843a0b740 10 client.5934 fetch_fsmap finished waiting for FSMap version 27 2024-04-25T10:30:44.963 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:44.958+0000 7f4843a0b740 10 client.5934 resolve_mds: resolved 3:0 to role '3:0' aka daemon mds.c 2024-04-25T10:30:44.963 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:44.958+0000 7f4843a0b740 20 client.5934 populate_metadata read hostname 'smithi037' 2024-04-25T10:30:44.963 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:44.958+0000 7f4843a0b740 1 --2- 172.21.15.37:0/2287319098 >> [v2:172.21.15.37:6832/2353203722,v1:172.21.15.37:6833/2353203722] conn(0x55d0a77c6510 0x55d0a77c5800 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:30:44.963 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:44.958+0000 7f4843a0b740 4 client.5934 mds_command: new command op to 5625 tid=0 multi_id=0 [{"prefix": "scrub start", "path": "/", "scrubops": ["force", "recursive", "repair"]}] 2024-04-25T10:30:44.964 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:44.958+0000 7f4843a0b740 1 -- 172.21.15.37:0/2287319098 --> [v2:172.21.15.37:6832/2353203722,v1:172.21.15.37:6833/2353203722] -- command(tid 0: {"prefix": "scrub start", "path": "/", "scrubops": ["force", "recursive", "repair"]}) -- 0x55d0a76f0620 con 0x55d0a77c6510 2024-04-25T10:30:44.964 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:44.958+0000 7f483659c640 1 --2- 172.21.15.37:0/2287319098 >> [v2:172.21.15.37:6832/2353203722,v1:172.21.15.37:6833/2353203722] conn(0x55d0a77c6510 0x55d0a77c5800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:30:44.964 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:44.958+0000 7f48157fa640 10 client.5934 ms_handle_connect on v2:172.21.15.37:6832/2353203722 2024-04-25T10:30:44.964 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:44.958+0000 7f483659c640 1 --2- 172.21.15.37:0/2287319098 >> [v2:172.21.15.37:6832/2353203722,v1:172.21.15.37:6833/2353203722] conn(0x55d0a77c6510 0x55d0a77c5800 crc :-1 s=READY pgs=15 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=2aa5a9b399ba976e server_cookie=1bca2ddefdeabd9e in_seq=0 out_seq=0 2024-04-25T10:30:44.964 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:44.959+0000 7f48157fa640 1 -- 172.21.15.37:0/2287319098 <== mds.0 v2:172.21.15.37:6832/2353203722 1 ==== command_reply(tid 0: 0 ) ==== 8+0+110 (crc 0 0 0) 0x55d0a76f0620 con 0x55d0a77c6510 2024-04-25T10:30:44.964 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:44.959+0000 7f48157fa640 10 client.5934 handle_command_reply: tid=0 2024-04-25T10:30:44.964 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:44.959+0000 7f4843a0b740 1 client.5934 shutdown 2024-04-25T10:30:44.964 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:44.959+0000 7f4843a0b740 1 -- 172.21.15.37:0/2287319098 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55d0a73bb7f0 msgr2=0x55d0a7842bc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:30:44.964 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:44.959+0000 7f4843a0b740 1 --2- 172.21.15.37:0/2287319098 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55d0a73bb7f0 0x55d0a7842bc0 secure :-1 s=READY pgs=600 cs=0 l=1 rev1=1 crypto rx=0x7f4820010010 tx=0x7f4820015000 comp rx=0 tx=0).stop 2024-04-25T10:30:44.964 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:44.959+0000 7f4843a0b740 1 -- 172.21.15.37:0/2287319098 shutdown_connections 2024-04-25T10:30:44.964 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:44.959+0000 7f4843a0b740 1 --2- 172.21.15.37:0/2287319098 >> [v2:172.21.15.37:6832/2353203722,v1:172.21.15.37:6833/2353203722] conn(0x55d0a77c6510 0x55d0a77c5800 crc :-1 s=READY pgs=15 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:30:44.964 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:44.959+0000 7f4843a0b740 1 --2- 172.21.15.37:0/2287319098 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x55d0a73dc4d0 0x55d0a77c52c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:30:44.964 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:44.959+0000 7f4843a0b740 1 --2- 172.21.15.37:0/2287319098 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55d0a73bb7f0 0x55d0a7842bc0 unknown :-1 s=CLOSED pgs=600 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:30:44.964 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:44.959+0000 7f4843a0b740 1 --2- 172.21.15.37:0/2287319098 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x55d0a73b3610 0x55d0a78425b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:30:44.965 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:44.959+0000 7f48157fa640 0 client.5934 ms_handle_reset on v2:172.21.15.37:6832/2353203722 2024-04-25T10:30:44.965 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:44.959+0000 7f4843a0b740 1 -- 172.21.15.37:0/2287319098 >> 172.21.15.37:0/2287319098 conn(0x55d0a73d0620 msgr2=0x55d0a779a580 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:30:44.965 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:44.959+0000 7f4843a0b740 1 -- 172.21.15.37:0/2287319098 shutdown_connections 2024-04-25T10:30:44.965 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:44.959+0000 7f4843a0b740 1 -- 172.21.15.37:0/2287319098 wait complete. 2024-04-25T10:30:44.965 INFO:tasks.ceph.mds.c.smithi037.stderr:2024-04-25T10:30:44.959+0000 7f12be3c9640 -1 log_channel(cluster) log [ERR] : bad backtrace on directory inode 0x10000000000 2024-04-25T10:30:44.965 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:44.959+0000 7f4843a0b740 20 client.5934 trim_cache size 0 max 16384 2024-04-25T10:30:44.966 INFO:teuthology.orchestra.run.smithi037.stdout:{ 2024-04-25T10:30:44.966 INFO:teuthology.orchestra.run.smithi037.stdout: "return_code": 0, 2024-04-25T10:30:44.966 INFO:teuthology.orchestra.run.smithi037.stdout: "scrub_tag": "9f043681-e65c-4197-9463-0955a39659d5", 2024-04-25T10:30:44.966 INFO:teuthology.orchestra.run.smithi037.stdout: "mode": "asynchronous" 2024-04-25T10:30:44.966 INFO:teuthology.orchestra.run.smithi037.stdout:} 2024-04-25T10:30:44.966 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:44.960+0000 7f483ca84640 1 -- 172.21.15.37:0/3542185943 >> [v2:172.21.15.203:6832/2418205237,v1:172.21.15.203:6833/2418205237] conn(0x7f48180744b0 msgr2=0x7f48180769c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:30:44.967 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:44.960+0000 7f483ca84640 1 --2- 172.21.15.37:0/3542185943 >> [v2:172.21.15.203:6832/2418205237,v1:172.21.15.203:6833/2418205237] conn(0x7f48180744b0 0x7f48180769c0 secure :-1 s=READY pgs=276 cs=0 l=1 rev1=1 crypto rx=0x7f48381638d0 tx=0x7f4820009060 comp rx=0 tx=0).stop 2024-04-25T10:30:44.967 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:44.960+0000 7f483ca84640 1 -- 172.21.15.37:0/3542185943 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f48380b79f0 msgr2=0x7f48381f63c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:30:44.967 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:44.960+0000 7f483ca84640 1 --2- 172.21.15.37:0/3542185943 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f48380b79f0 0x7f48381f63c0 secure :-1 s=READY pgs=596 cs=0 l=1 rev1=1 crypto rx=0x7f482800c430 tx=0x7f482800c8f0 comp rx=0 tx=0).stop 2024-04-25T10:30:44.967 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:44.960+0000 7f483ca84640 1 -- 172.21.15.37:0/3542185943 shutdown_connections 2024-04-25T10:30:44.967 INFO:tasks.ceph.mds.c.smithi037.stderr:2024-04-25T10:30:44.960+0000 7f12be3c9640 -1 mds.0.scrubstack _validate_inode_done scrub error on inode [inode 0x10000000000 [...2,head] /subdir/ auth v1 pv4 ap=3 DIRTYPARENT f(v0 1=1+0) n() (inest lock dirty) (ifile lock->sync w=1 flushing) | dirtyscattered=2 lock=1 dirfrag=1 dirtyparent=1 waiter=1 authpin=1 scrubqueue=0 0x5603dd0f7180]: {"performed_validation":true,"passed_validation":false,"backtrace":{"checked":true,"passed":true,"read_ret_val":-61,"ondisk_value":"(-1)0x0:[]//[]","memoryvalue":"(6)0x10000000000:[<0x1/subdir v1>]//[]","error_str":"failed to read off disk; see retval"},"raw_stats":{"checked":true,"passed":false,"read_ret_val":0,"ondisk_value.dirstat":"f(v0 1=1+0)","ondisk_value.rstat":"n(v0 1=0+1)","memory_value.dirstat":"f(v0 1=1+0)","memory_value.rstat":"n()","error_str":"freshly-calculated rstats don't match existing ones (will be fixed)"},"return_code":-61} 2024-04-25T10:30:44.968 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:44.960+0000 7f483ca84640 1 --2- 172.21.15.37:0/3542185943 >> [v2:172.21.15.203:6832/2418205237,v1:172.21.15.203:6833/2418205237] conn(0x7f48180744b0 0x7f48180769c0 unknown :-1 s=CLOSED pgs=276 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:30:44.968 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:44.960+0000 7f483ca84640 1 --2- 172.21.15.37:0/3542185943 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f4838160260 0x7f48381f8310 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:30:44.968 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:44.960+0000 7f483ca84640 1 --2- 172.21.15.37:0/3542185943 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f48380d0940 0x7f48381f7dd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:30:44.968 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:44.960+0000 7f483ca84640 1 --2- 172.21.15.37:0/3542185943 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f48380b79f0 0x7f48381f63c0 unknown :-1 s=CLOSED pgs=596 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:30:44.968 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:44.960+0000 7f483ca84640 1 -- 172.21.15.37:0/3542185943 >> 172.21.15.37:0/3542185943 conn(0x7f48380d5d70 msgr2=0x7f483815eee0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:30:44.968 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:44.960+0000 7f483ca84640 1 -- 172.21.15.37:0/3542185943 shutdown_connections 2024-04-25T10:30:44.968 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:44.960+0000 7f483ca84640 1 -- 172.21.15.37:0/3542185943 wait complete. 2024-04-25T10:30:44.968 INFO:tasks.ceph.mds.c.smithi037.stderr:2024-04-25T10:30:44.961+0000 7f12be3c9640 -1 mds.0.scrubstack _validate_inode_done scrub error on inode [inode 0x10000000001 [2,head] /subdir/sixmegs auth v1 ap=1 DIRTYPARENT s=6291456 n() | dirtyparent=1 authpin=1 scrubqueue=0 0x5603dd0f5080]: {"performed_validation":true,"passed_validation":false,"backtrace":{"checked":true,"passed":false,"read_ret_val":0,"ondisk_value":"(5)0x10000000001:[<0x10000000000/sixmegs v8>,<0x1/subdir v10>]//[]","memoryvalue":"(5)0x10000000001:[<0x10000000000/sixmegs v1>,<0x1/subdir v1>]//[]","error_str":"On-disk backtrace is divergent or newer"},"raw_stats":{"checked":false,"passed":false,"read_ret_val":0,"ondisk_value.dirstat":"f()","ondisk_value.rstat":"n()","memory_value.dirstat":"f()","memory_value.rstat":"n()","error_str":""},"return_code":0} 2024-04-25T10:30:44.969 INFO:tasks.ceph.mds.c.smithi037.stderr:2024-04-25T10:30:44.968+0000 7f12be3c9640 -1 mds.0.scrubstack _validate_inode_done scrub error on inode [inode 0x1 [...2,head] / auth v4 pv6 ap=5 snaprealm=0x5603dd0e8900 DIRTYPARENT f(v1 1=0+1) n(v0 1=0+1)/n() (inest mix->lock(2) w=1 flushing) (ifile mix->sync w=1 flushing) | dirtyscattered=2 lock=2 dirfrag=1 openingsnapparents=0 dirtyparent=1 dirty=1 waiter=1 authpin=1 scrubqueue=0 0x5603dd0f4b00]: {"performed_validation":true,"passed_validation":false,"backtrace":{"checked":true,"passed":false,"read_ret_val":-61,"ondisk_value":"(-1)0x0:[]//[]","memoryvalue":"(6)0x1:[]//[]","error_str":"failed to read off disk; see retval"},"raw_stats":{"checked":true,"passed":true,"read_ret_val":0,"ondisk_value.dirstat":"f(v0 1=0+1)","ondisk_value.rstat":"n(v0 2=0+2)","memory_value.dirstat":"f(v1 1=0+1)","memory_value.rstat":"n(v0 1=0+1)","error_str":"freshly-calculated rstats don't match existing ones (will be fixed)"},"return_code":-61} 2024-04-25T10:30:44.973 DEBUG:teuthology.orchestra.run.smithi037:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-04-25T10:30:45.085 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:45.084+0000 7fdc83e70640 1 Processor -- start 2024-04-25T10:30:45.086 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:45.084+0000 7fdc83e70640 1 -- start start 2024-04-25T10:30:45.087 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:45.086+0000 7fdc83e70640 1 --2- >> v2:172.21.15.37:3300/0 conn(0x7fdc7c0b3c80 0x7fdc7c0b4090 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:30:45.088 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:45.086+0000 7fdc83e70640 1 --2- >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7fdc7c0d1ff0 0x7fdc7c0d2400 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:30:45.088 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:45.086+0000 7fdc83e70640 1 -- --> v1:172.21.15.37:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fdc7c0b8d10 con 0x7fdc7c1625b0 2024-04-25T10:30:45.088 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:45.086+0000 7fdc83e70640 1 -- --> v2:172.21.15.37:3300/0 -- mon_getmap magic: 0 -- 0x7fdc7c08ceb0 con 0x7fdc7c0b3c80 2024-04-25T10:30:45.088 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:45.086+0000 7fdc83e70640 1 -- --> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] -- mon_getmap magic: 0 -- 0x7fdc7c0b9300 con 0x7fdc7c0d1ff0 2024-04-25T10:30:45.088 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:45.086+0000 7fdc81c0c640 1 --2- >> v2:172.21.15.37:3300/0 conn(0x7fdc7c0b3c80 0x7fdc7c0b4090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:30:45.088 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:45.086+0000 7fdc8240d640 1 --1- >> v1:172.21.15.37:6789/0 conn(0x7fdc7c1625b0 0x7fdc7c1629c0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.37:6789/0 says I am v1:172.21.15.37:55678/0 (socket says 172.21.15.37:55678) 2024-04-25T10:30:45.088 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:45.086+0000 7fdc8240d640 1 -- 172.21.15.37:0/1732106106 learned_addr learned my addr 172.21.15.37:0/1732106106 (peer_addr_for_me v1:172.21.15.37:0/0) 2024-04-25T10:30:45.088 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:45.086+0000 7fdc8140b640 1 --2- 172.21.15.37:0/1732106106 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7fdc7c0d1ff0 0x7fdc7c0d2400 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:30:45.088 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:45.086+0000 7fdc81c0c640 1 -- 172.21.15.37:0/1732106106 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7fdc7c0d1ff0 msgr2=0x7fdc7c0d2400 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:30:45.088 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:45.086+0000 7fdc81c0c640 1 --2- 172.21.15.37:0/1732106106 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7fdc7c0d1ff0 0x7fdc7c0d2400 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:30:45.088 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:45.086+0000 7fdc81c0c640 1 -- 172.21.15.37:0/1732106106 >> v1:172.21.15.37:6789/0 conn(0x7fdc7c1625b0 legacy=0x7fdc7c1629c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:30:45.088 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:45.086+0000 7fdc80c0a640 1 -- 172.21.15.37:0/1732106106 <== mon.0 v1:172.21.15.37:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 18780559 0 0) 0x7fdc7c0b8d10 con 0x7fdc7c1625b0 2024-04-25T10:30:45.088 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:45.086+0000 7fdc81c0c640 1 -- 172.21.15.37:0/1732106106 --> v2:172.21.15.37:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fdc7c0b8a30 con 0x7fdc7c0b3c80 2024-04-25T10:30:45.090 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:45.087+0000 7fdc81c0c640 1 --2- 172.21.15.37:0/1732106106 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fdc7c0b3c80 0x7fdc7c0b4090 secure :-1 s=READY pgs=602 cs=0 l=1 rev1=1 crypto rx=0x7fdc740097a0 tx=0x7fdc7402a100 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c3844c2e373ce856 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:30:45.090 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:45.087+0000 7fdc80c0a640 1 -- 172.21.15.37:0/1732106106 <== mon.0 v2:172.21.15.37:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdc74037070 con 0x7fdc7c0b3c80 2024-04-25T10:30:45.090 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:45.087+0000 7fdc80c0a640 1 -- 172.21.15.37:0/1732106106 <== mon.0 v2:172.21.15.37:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fdc7402ae50 con 0x7fdc7c0b3c80 2024-04-25T10:30:45.090 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:45.087+0000 7fdc80c0a640 1 -- 172.21.15.37:0/1732106106 <== mon.0 v2:172.21.15.37:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdc74032660 con 0x7fdc7c0b3c80 2024-04-25T10:30:45.090 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:45.088+0000 7fdc83e70640 1 -- 172.21.15.37:0/1732106106 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fdc7c0b3c80 msgr2=0x7fdc7c0b4090 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:30:45.090 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:45.088+0000 7fdc83e70640 1 --2- 172.21.15.37:0/1732106106 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fdc7c0b3c80 0x7fdc7c0b4090 secure :-1 s=READY pgs=602 cs=0 l=1 rev1=1 crypto rx=0x7fdc740097a0 tx=0x7fdc7402a100 comp rx=0 tx=0).stop 2024-04-25T10:30:45.090 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:45.088+0000 7fdc83e70640 1 -- 172.21.15.37:0/1732106106 shutdown_connections 2024-04-25T10:30:45.090 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:45.088+0000 7fdc83e70640 1 --2- 172.21.15.37:0/1732106106 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7fdc7c0d1ff0 0x7fdc7c0d2400 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:30:45.090 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:45.088+0000 7fdc83e70640 1 --2- 172.21.15.37:0/1732106106 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fdc7c0b3c80 0x7fdc7c0b4090 unknown :-1 s=CLOSED pgs=602 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:30:45.090 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:45.088+0000 7fdc83e70640 1 -- 172.21.15.37:0/1732106106 >> 172.21.15.37:0/1732106106 conn(0x7fdc7c0d0390 msgr2=0x7fdc7c0d07a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:30:45.090 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:45.088+0000 7fdc83e70640 1 -- 172.21.15.37:0/1732106106 shutdown_connections 2024-04-25T10:30:45.090 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:45.088+0000 7fdc83e70640 1 -- 172.21.15.37:0/1732106106 wait complete. 2024-04-25T10:30:45.091 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:45.090+0000 7fdc83e70640 1 Processor -- start 2024-04-25T10:30:45.091 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:45.090+0000 7fdc83e70640 1 -- start start 2024-04-25T10:30:45.094 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:45.091+0000 7fdc83e70640 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fdc7c0b3c80 0x7fdc7c1f6910 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:30:45.094 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:45.091+0000 7fdc83e70640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fdc7c0d1ff0 0x7fdc7c1546b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:30:45.094 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:45.091+0000 7fdc83e70640 1 --2- >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7fdc7c1625b0 0x7fdc7c0b7d50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:30:45.094 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:45.091+0000 7fdc83e70640 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 -- 0x7fdc7c0b45d0 con 0x7fdc7c0d1ff0 2024-04-25T10:30:45.094 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:45.091+0000 7fdc83e70640 1 -- --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_getmap magic: 0 -- 0x7fdc7c09ce90 con 0x7fdc7c0b3c80 2024-04-25T10:30:45.094 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:45.091+0000 7fdc83e70640 1 -- --> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] -- mon_getmap magic: 0 -- 0x7fdc7c0d2940 con 0x7fdc7c1625b0 2024-04-25T10:30:45.094 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:45.091+0000 7fdc8140b640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fdc7c0d1ff0 0x7fdc7c1546b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:30:45.094 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:45.091+0000 7fdc8140b640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fdc7c0d1ff0 0x7fdc7c1546b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.37:3300/0 says I am v2:172.21.15.37:58364/0 (socket says 172.21.15.37:58364) 2024-04-25T10:30:45.094 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:45.091+0000 7fdc8140b640 1 -- 172.21.15.37:0/999874212 learned_addr learned my addr 172.21.15.37:0/999874212 (peer_addr_for_me v2:172.21.15.37:0/0) 2024-04-25T10:30:45.094 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:45.091+0000 7fdc81c0c640 1 --2- 172.21.15.37:0/999874212 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fdc7c0b3c80 0x7fdc7c1f6910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:30:45.094 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:45.091+0000 7fdc8240d640 1 --2- 172.21.15.37:0/999874212 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7fdc7c1625b0 0x7fdc7c0b7d50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:30:45.094 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:45.092+0000 7fdc8140b640 1 -- 172.21.15.37:0/999874212 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7fdc7c1625b0 msgr2=0x7fdc7c0b7d50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:30:45.095 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:45.092+0000 7fdc8140b640 1 --2- 172.21.15.37:0/999874212 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7fdc7c1625b0 0x7fdc7c0b7d50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:30:45.095 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:45.092+0000 7fdc8140b640 1 -- 172.21.15.37:0/999874212 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fdc7c0b3c80 msgr2=0x7fdc7c1f6910 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:30:45.095 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:45.092+0000 7fdc8140b640 1 --2- 172.21.15.37:0/999874212 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fdc7c0b3c80 0x7fdc7c1f6910 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:30:45.095 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:45.092+0000 7fdc8140b640 1 -- 172.21.15.37:0/999874212 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fdc7c154bf0 con 0x7fdc7c0d1ff0 2024-04-25T10:30:45.095 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:45.092+0000 7fdc81c0c640 1 --2- 172.21.15.37:0/999874212 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fdc7c0b3c80 0x7fdc7c1f6910 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T10:30:45.095 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:45.092+0000 7fdc8140b640 1 --2- 172.21.15.37:0/999874212 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fdc7c0d1ff0 0x7fdc7c1546b0 secure :-1 s=READY pgs=603 cs=0 l=1 rev1=1 crypto rx=0x7fdc7000d5f0 tx=0x7fdc7000dab0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:30:45.095 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:45.092+0000 7fdc7a7fc640 1 -- 172.21.15.37:0/999874212 <== mon.0 v2:172.21.15.37:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdc700125f0 con 0x7fdc7c0d1ff0 2024-04-25T10:30:45.095 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:45.092+0000 7fdc7a7fc640 1 -- 172.21.15.37:0/999874212 <== mon.0 v2:172.21.15.37:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fdc70019070 con 0x7fdc7c0d1ff0 2024-04-25T10:30:45.095 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:45.092+0000 7fdc83e70640 1 -- 172.21.15.37:0/999874212 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fdc7c06d850 con 0x7fdc7c0d1ff0 2024-04-25T10:30:45.095 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:45.092+0000 7fdc7a7fc640 1 -- 172.21.15.37:0/999874212 <== mon.0 v2:172.21.15.37:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdc700129d0 con 0x7fdc7c0d1ff0 2024-04-25T10:30:45.095 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:45.092+0000 7fdc83e70640 1 -- 172.21.15.37:0/999874212 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fdc7c06de10 con 0x7fdc7c0d1ff0 2024-04-25T10:30:45.095 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:45.093+0000 7fdc83e70640 1 -- 172.21.15.37:0/999874212 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fdc7c06a140 con 0x7fdc7c0d1ff0 2024-04-25T10:30:45.096 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:45.094+0000 7fdc7a7fc640 1 -- 172.21.15.37:0/999874212 <== mon.0 v2:172.21.15.37:3300/0 4 ==== mgrmap(e 5) ==== 96782+0+0 (secure 0 0 0) 0x7fdc70012b70 con 0x7fdc7c0d1ff0 2024-04-25T10:30:45.096 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:45.094+0000 7fdc7a7fc640 1 --2- 172.21.15.37:0/999874212 >> [v2:172.21.15.203:6832/2418205237,v1:172.21.15.203:6833/2418205237] conn(0x7fdc680744b0 0x7fdc680769c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:30:45.096 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:45.094+0000 7fdc7a7fc640 1 -- 172.21.15.37:0/999874212 <== mon.0 v2:172.21.15.37:3300/0 5 ==== osd_map(44..44 src has 1..44) ==== 6248+0+0 (secure 0 0 0) 0x7fdc700958e0 con 0x7fdc7c0d1ff0 2024-04-25T10:30:45.096 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:45.094+0000 7fdc81c0c640 1 --2- 172.21.15.37:0/999874212 >> [v2:172.21.15.203:6832/2418205237,v1:172.21.15.203:6833/2418205237] conn(0x7fdc680744b0 0x7fdc680769c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:30:45.096 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:45.095+0000 7fdc81c0c640 1 --2- 172.21.15.37:0/999874212 >> [v2:172.21.15.203:6832/2418205237,v1:172.21.15.203:6833/2418205237] conn(0x7fdc680744b0 0x7fdc680769c0 secure :-1 s=READY pgs=277 cs=0 l=1 rev1=1 crypto rx=0x7fdc74034000 tx=0x7fdc7402e000 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:30:45.100 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:45.099+0000 7fdc7a7fc640 1 -- 172.21.15.37:0/999874212 <== mon.0 v2:172.21.15.37:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187246 (secure 0 0 0) 0x7fdc7005fe50 con 0x7fdc7c0d1ff0 2024-04-25T10:30:45.327 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:45.326+0000 7fdc83e70640 1 -- 172.21.15.37:0/999874212 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7fdc7c070210 con 0x7fdc7c0d1ff0 2024-04-25T10:30:45.328 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:45.327+0000 7fdc7a7fc640 1 -- 172.21.15.37:0/999874212 <== mon.0 v2:172.21.15.37:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 27 v27) ==== 94+0+6489 (secure 0 0 0) 0x7fdc7005f750 con 0x7fdc7c0d1ff0 2024-04-25T10:30:45.328 INFO:teuthology.orchestra.run.smithi037.stdout: 2024-04-25T10:30:45.329 INFO:teuthology.orchestra.run.smithi037.stdout:{"epoch":27,"default_fscid":2,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":4946,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.37:6835/683655741","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.37:6834","nonce":683655741},{"type":"v1","addr":"172.21.15.37:6835","nonce":683655741}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":19},{"gid":5215,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.203:6836/3927045001","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6835","nonce":3927045001},{"type":"v1","addr":"172.21.15.203:6836","nonce":3927045001}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":19},{"gid":5218,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.203:6837/3108183946","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6834","nonce":3108183946},{"type":"v1","addr":"172.21.15.203:6837","nonce":3108183946}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":19}],"filesystems":[{"mdsmap":{"epoch":18,"flags":19,"flags_state":{"joinable":false,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-04-25T10:30:12.130473+0000","modified":"2024-04-25T10:30:24.853915+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":37,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":1,"in":[0],"up":{},"failed":[0],"damaged":[],"stopped":[],"info":{},"data_pools":[5],"metadata_pool":4,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":0,"qdb_cluster":[]},"id":2},{"mdsmap":{"epoch":27,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-04-25T10:30:28.588617+0000","modified":"2024-04-25T10:30:42.935167+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":1,"in":[0],"up":{"mds_0":5625},"failed":[],"damaged":[],"stopped":[],"info":{"gid_5625":{"gid":5625,"name":"c","rank":0,"incarnation":24,"state":"up:active","state_seq":7,"addr":"172.21.15.37:6833/2353203722","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.37:6832","nonce":2353203722},{"type":"v1","addr":"172.21.15.37:6833","nonce":2353203722}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[5],"metadata_pool":6,"enabled":true,"fs_name":"recovery_fs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":5625,"qdb_cluster":[5625]},"id":3}]} 2024-04-25T10:30:45.329 INFO:teuthology.orchestra.run.smithi037.stderr:dumped fsmap epoch 27 2024-04-25T10:30:45.331 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:45.329+0000 7fdc83e70640 1 -- 172.21.15.37:0/999874212 >> [v2:172.21.15.203:6832/2418205237,v1:172.21.15.203:6833/2418205237] conn(0x7fdc680744b0 msgr2=0x7fdc680769c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:30:45.331 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:45.329+0000 7fdc83e70640 1 --2- 172.21.15.37:0/999874212 >> [v2:172.21.15.203:6832/2418205237,v1:172.21.15.203:6833/2418205237] conn(0x7fdc680744b0 0x7fdc680769c0 secure :-1 s=READY pgs=277 cs=0 l=1 rev1=1 crypto rx=0x7fdc74034000 tx=0x7fdc7402e000 comp rx=0 tx=0).stop 2024-04-25T10:30:45.331 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:45.329+0000 7fdc83e70640 1 -- 172.21.15.37:0/999874212 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fdc7c0d1ff0 msgr2=0x7fdc7c1546b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:30:45.331 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:45.329+0000 7fdc83e70640 1 --2- 172.21.15.37:0/999874212 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fdc7c0d1ff0 0x7fdc7c1546b0 secure :-1 s=READY pgs=603 cs=0 l=1 rev1=1 crypto rx=0x7fdc7000d5f0 tx=0x7fdc7000dab0 comp rx=0 tx=0).stop 2024-04-25T10:30:45.331 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:45.329+0000 7fdc83e70640 1 -- 172.21.15.37:0/999874212 shutdown_connections 2024-04-25T10:30:45.331 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:45.329+0000 7fdc83e70640 1 --2- 172.21.15.37:0/999874212 >> [v2:172.21.15.203:6832/2418205237,v1:172.21.15.203:6833/2418205237] conn(0x7fdc680744b0 0x7fdc680769c0 unknown :-1 s=CLOSED pgs=277 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:30:45.331 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:45.329+0000 7fdc83e70640 1 --2- 172.21.15.37:0/999874212 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7fdc7c1625b0 0x7fdc7c0b7d50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:30:45.331 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:45.329+0000 7fdc83e70640 1 --2- 172.21.15.37:0/999874212 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fdc7c0d1ff0 0x7fdc7c1546b0 unknown :-1 s=CLOSED pgs=603 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:30:45.331 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:45.329+0000 7fdc83e70640 1 --2- 172.21.15.37:0/999874212 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fdc7c0b3c80 0x7fdc7c1f6910 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:30:45.331 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:45.329+0000 7fdc83e70640 1 -- 172.21.15.37:0/999874212 >> 172.21.15.37:0/999874212 conn(0x7fdc7c0d0390 msgr2=0x7fdc7c0712f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:30:45.331 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:45.329+0000 7fdc83e70640 1 -- 172.21.15.37:0/999874212 shutdown_connections 2024-04-25T10:30:45.332 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:45.329+0000 7fdc83e70640 1 -- 172.21.15.37:0/999874212 wait complete. 2024-04-25T10:30:45.342 INFO:tasks.cephfs.test_recovery_pool:{ "compat": { "compat": {}, "incompat": { "feature_1": "base v0.20", "feature_10": "snaprealm v2", "feature_11": "minor log segments", "feature_12": "quiesce subvolumes", "feature_2": "client writeable ranges", "feature_3": "default file layouts on dirs", "feature_4": "dir inode in separate object", "feature_5": "mds uses versioned encoding", "feature_6": "dirfrag is stored in omap", "feature_8": "no anchor table", "feature_9": "file layout v2" }, "ro_compat": {} }, "default_fscid": 2, "epoch": 27, "feature_flags": { "enable_multiple": true, "ever_enabled_multiple": true }, "filesystems": [ { "id": 2, "mdsmap": { "bal_rank_mask": "-1", "balancer": "", "compat": { "compat": {}, "incompat": { "feature_1": "base v0.20", "feature_10": "snaprealm v2", "feature_11": "minor log segments", "feature_12": "quiesce subvolumes", "feature_2": "client writeable ranges", "feature_3": "default file layouts on dirs", "feature_4": "dir inode in separate object", "feature_5": "mds uses versioned encoding", "feature_6": "dirfrag is stored in omap", "feature_7": "mds uses inline data", "feature_8": "no anchor table", "feature_9": "file layout v2" }, "ro_compat": {} }, "created": "2024-04-25T10:30:12.130473+0000", "damaged": [], "data_pools": [ 5 ], "enabled": true, "epoch": 18, "ever_allowed_features": 0, "explicitly_allowed_features": 0, "failed": [ 0 ], "flags": 19, "flags_state": { "allow_multimds_snaps": true, "allow_snaps": true, "allow_standby_replay": false, "balance_automate": false, "joinable": false, "refuse_client_session": false, "refuse_standby_for_another_fs": false }, "fs_name": "cephfs", "in": [ 0 ], "info": {}, "last_failure": 0, "last_failure_osd_epoch": 37, "max_file_size": 1099511627776, "max_mds": 1, "max_xattr_size": 65536, "metadata_pool": 4, "modified": "2024-04-25T10:30:24.853915+0000", "qdb_cluster": [], "qdb_leader": 0, "required_client_features": {}, "root": 0, "session_autoclose": 300, "session_timeout": 60, "standby_count_wanted": 0, "stopped": [], "tableserver": 0, "up": {} } }, { "id": 3, "mdsmap": { "bal_rank_mask": "-1", "balancer": "", "compat": { "compat": {}, "incompat": { "feature_1": "base v0.20", "feature_10": "snaprealm v2", "feature_11": "minor log segments", "feature_12": "quiesce subvolumes", "feature_2": "client writeable ranges", "feature_3": "default file layouts on dirs", "feature_4": "dir inode in separate object", "feature_5": "mds uses versioned encoding", "feature_6": "dirfrag is stored in omap", "feature_7": "mds uses inline data", "feature_8": "no anchor table", "feature_9": "file layout v2" }, "ro_compat": {} }, "created": "2024-04-25T10:30:28.588617+0000", "damaged": [], "data_pools": [ 5 ], "enabled": true, "epoch": 27, "ever_allowed_features": 0, "explicitly_allowed_features": 0, "failed": [], "flags": 18, "flags_state": { "allow_multimds_snaps": true, "allow_snaps": true, "allow_standby_replay": false, "balance_automate": false, "joinable": true, "refuse_client_session": false, "refuse_standby_for_another_fs": false }, "fs_name": "recovery_fs", "in": [ 0 ], "info": { "gid_5625": { "addr": "172.21.15.37:6833/2353203722", "addrs": { "addrvec": [ { "addr": "172.21.15.37:6832", "nonce": 2353203722, "type": "v2" }, { "addr": "172.21.15.37:6833", "nonce": 2353203722, "type": "v1" } ] }, "compat": { "compat": {}, "incompat": { "feature_1": "base v0.20", "feature_10": "snaprealm v2", "feature_11": "minor log segments", "feature_12": "quiesce subvolumes", "feature_2": "client writeable ranges", "feature_3": "default file layouts on dirs", "feature_4": "dir inode in separate object", "feature_5": "mds uses versioned encoding", "feature_6": "dirfrag is stored in omap", "feature_7": "mds uses inline data", "feature_8": "no anchor table", "feature_9": "file layout v2" }, "ro_compat": {} }, "export_targets": [], "features": 4540701547738038271, "flags": 0, "gid": 5625, "incarnation": 24, "join_fscid": -1, "name": "c", "rank": 0, "state": "up:active", "state_seq": 7 } }, "last_failure": 0, "last_failure_osd_epoch": 0, "max_file_size": 1099511627776, "max_mds": 1, "max_xattr_size": 65536, "metadata_pool": 6, "modified": "2024-04-25T10:30:42.935167+0000", "qdb_cluster": [ 5625 ], "qdb_leader": 5625, "required_client_features": {}, "root": 0, "session_autoclose": 300, "session_timeout": 60, "standby_count_wanted": 0, "stopped": [], "tableserver": 0, "up": { "mds_0": 5625 } } } ], "standbys": [ { "addr": "172.21.15.37:6835/683655741", "addrs": { "addrvec": [ { "addr": "172.21.15.37:6834", "nonce": 683655741, "type": "v2" }, { "addr": "172.21.15.37:6835", "nonce": 683655741, "type": "v1" } ] }, "compat": { "compat": {}, "incompat": { "feature_1": "base v0.20", "feature_10": "snaprealm v2", "feature_11": "minor log segments", "feature_12": "quiesce subvolumes", "feature_2": "client writeable ranges", "feature_3": "default file layouts on dirs", "feature_4": "dir inode in separate object", "feature_5": "mds uses versioned encoding", "feature_6": "dirfrag is stored in omap", "feature_7": "mds uses inline data", "feature_8": "no anchor table", "feature_9": "file layout v2" }, "ro_compat": {} }, "epoch": 19, "export_targets": [], "features": 4540701547738038271, "flags": 0, "gid": 4946, "incarnation": 0, "join_fscid": -1, "name": "a", "rank": -1, "state": "up:standby", "state_seq": 1 }, { "addr": "172.21.15.203:6836/3927045001", "addrs": { "addrvec": [ { "addr": "172.21.15.203:6835", "nonce": 3927045001, "type": "v2" }, { "addr": "172.21.15.203:6836", "nonce": 3927045001, "type": "v1" } ] }, "compat": { "compat": {}, "incompat": { "feature_1": "base v0.20", "feature_10": "snaprealm v2", "feature_11": "minor log segments", "feature_12": "quiesce subvolumes", "feature_2": "client writeable ranges", "feature_3": "default file layouts on dirs", "feature_4": "dir inode in separate object", "feature_5": "mds uses versioned encoding", "feature_6": "dirfrag is stored in omap", "feature_7": "mds uses inline data", "feature_8": "no anchor table", "feature_9": "file layout v2" }, "ro_compat": {} }, "epoch": 19, "export_targets": [], "features": 4540701547738038271, "flags": 0, "gid": 5215, "incarnation": 0, "join_fscid": -1, "name": "b", "rank": -1, "state": "up:standby", "state_seq": 1 }, { "addr": "172.21.15.203:6837/3108183946", "addrs": { "addrvec": [ { "addr": "172.21.15.203:6834", "nonce": 3108183946, "type": "v2" }, { "addr": "172.21.15.203:6837", "nonce": 3108183946, "type": "v1" } ] }, "compat": { "compat": {}, "incompat": { "feature_1": "base v0.20", "feature_10": "snaprealm v2", "feature_11": "minor log segments", "feature_12": "quiesce subvolumes", "feature_2": "client writeable ranges", "feature_3": "default file layouts on dirs", "feature_4": "dir inode in separate object", "feature_5": "mds uses versioned encoding", "feature_6": "dirfrag is stored in omap", "feature_7": "mds uses inline data", "feature_8": "no anchor table", "feature_9": "file layout v2" }, "ro_compat": {} }, "epoch": 19, "export_targets": [], "features": 4540701547738038271, "flags": 0, "gid": 5218, "incarnation": 0, "join_fscid": -1, "name": "d", "rank": -1, "state": "up:standby", "state_seq": 1 } ] } 2024-04-25T10:30:45.342 DEBUG:teuthology.orchestra.run.smithi037:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs ls 2024-04-25T10:30:45.454 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:45.453+0000 7ff1661c5640 1 Processor -- start 2024-04-25T10:30:45.455 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:45.453+0000 7ff1661c5640 1 -- start start 2024-04-25T10:30:45.459 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:45.454+0000 7ff1661c5640 1 --2- >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7ff1600d3230 0x7ff16016b0c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:30:45.459 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:45.454+0000 7ff1661c5640 1 -- --> v1:172.21.15.37:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7ff1600d10a0 con 0x7ff1600b4e80 2024-04-25T10:30:45.459 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:45.454+0000 7ff1661c5640 1 -- --> v1:172.21.15.203:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7ff1600d1370 con 0x7ff1600d5160 2024-04-25T10:30:45.459 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:45.454+0000 7ff1661c5640 1 -- --> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] -- mon_getmap magic: 0 -- 0x7ff1600d1650 con 0x7ff1600d3230 2024-04-25T10:30:45.459 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:45.455+0000 7ff15effd640 1 --1- >> v1:172.21.15.37:6789/0 conn(0x7ff1600b4e80 0x7ff1600d2b20 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.37:6789/0 says I am v1:172.21.15.37:55682/0 (socket says 172.21.15.37:55682) 2024-04-25T10:30:45.459 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:45.455+0000 7ff15effd640 1 -- 172.21.15.37:0/3643536226 learned_addr learned my addr 172.21.15.37:0/3643536226 (peer_addr_for_me v1:172.21.15.37:0/0) 2024-04-25T10:30:45.459 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:45.455+0000 7ff15ffff640 1 --2- 172.21.15.37:0/3643536226 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7ff1600d3230 0x7ff16016b0c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:30:45.460 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:45.455+0000 7ff15e7fc640 1 -- 172.21.15.37:0/3643536226 <== mon.0 v1:172.21.15.37:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 809967617 0 0) 0x7ff1600d10a0 con 0x7ff1600b4e80 2024-04-25T10:30:45.460 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:45.455+0000 7ff15e7fc640 1 -- 172.21.15.37:0/3643536226 --> v1:172.21.15.37:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7ff148003520 con 0x7ff1600b4e80 2024-04-25T10:30:45.460 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:45.455+0000 7ff15e7fc640 1 -- 172.21.15.37:0/3643536226 <== mon.0 v1:172.21.15.37:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 2867143946 0 0) 0x7ff148003520 con 0x7ff1600b4e80 2024-04-25T10:30:45.460 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:45.455+0000 7ff15e7fc640 1 -- 172.21.15.37:0/3643536226 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7ff1600d3230 msgr2=0x7ff16016b0c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:30:45.460 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:45.455+0000 7ff15e7fc640 1 --2- 172.21.15.37:0/3643536226 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7ff1600d3230 0x7ff16016b0c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:30:45.460 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:45.455+0000 7ff15e7fc640 1 -- 172.21.15.37:0/3643536226 >> v1:172.21.15.203:6789/0 conn(0x7ff1600d5160 legacy=0x7ff1600d0990 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:30:45.460 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:45.455+0000 7ff15e7fc640 1 -- 172.21.15.37:0/3643536226 --> v1:172.21.15.37:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ff1600d4680 con 0x7ff1600b4e80 2024-04-25T10:30:45.460 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:45.455+0000 7ff15e7fc640 1 -- 172.21.15.37:0/3643536226 <== mon.0 v1:172.21.15.37:6789/0 3 ==== mon_map magic: 0 ==== 413+0+0 (unknown 1577794600 0 0) 0x7ff154001690 con 0x7ff1600b4e80 2024-04-25T10:30:45.460 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:45.455+0000 7ff15e7fc640 1 -- 172.21.15.37:0/3643536226 >> v1:172.21.15.37:6789/0 conn(0x7ff1600b4e80 legacy=0x7ff1600d2b20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:30:45.460 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:45.455+0000 7ff15e7fc640 1 --2- 172.21.15.37:0/3643536226 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7ff1480039e0 0x7ff148003e10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:30:45.460 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:45.455+0000 7ff15e7fc640 1 --2- 172.21.15.37:0/3643536226 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7ff148004800 0x7ff148004c10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:30:45.461 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:45.455+0000 7ff15e7fc640 1 --2- 172.21.15.37:0/3643536226 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7ff148005150 0x7ff14800b9c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:30:45.461 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:45.455+0000 7ff15e7fc640 1 -- 172.21.15.37:0/3643536226 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 -- 0x7ff1600d1650 con 0x7ff1480039e0 2024-04-25T10:30:45.461 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:45.455+0000 7ff15e7fc640 1 -- 172.21.15.37:0/3643536226 --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_getmap magic: 0 -- 0x7ff14800bfb0 con 0x7ff148005150 2024-04-25T10:30:45.461 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:45.455+0000 7ff15e7fc640 1 -- 172.21.15.37:0/3643536226 --> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] -- mon_getmap magic: 0 -- 0x7ff14800c160 con 0x7ff148004800 2024-04-25T10:30:45.461 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:45.455+0000 7ff15f7fe640 1 --2- 172.21.15.37:0/3643536226 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7ff1480039e0 0x7ff148003e10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:30:45.461 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:45.456+0000 7ff15effd640 1 --2- 172.21.15.37:0/3643536226 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7ff148004800 0x7ff148004c10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:30:45.461 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:45.456+0000 7ff15ffff640 1 --2- 172.21.15.37:0/3643536226 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7ff148005150 0x7ff14800b9c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:30:45.461 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:45.456+0000 7ff15f7fe640 1 -- 172.21.15.37:0/3643536226 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7ff148004800 msgr2=0x7ff148004c10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:30:45.461 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:45.456+0000 7ff15f7fe640 1 --2- 172.21.15.37:0/3643536226 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7ff148004800 0x7ff148004c10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:30:45.461 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:45.456+0000 7ff15f7fe640 1 -- 172.21.15.37:0/3643536226 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7ff148005150 msgr2=0x7ff14800b9c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:30:45.461 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:45.456+0000 7ff15f7fe640 1 --2- 172.21.15.37:0/3643536226 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7ff148005150 0x7ff14800b9c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:30:45.461 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:45.456+0000 7ff15f7fe640 1 -- 172.21.15.37:0/3643536226 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ff148003520 con 0x7ff1480039e0 2024-04-25T10:30:45.462 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:45.456+0000 7ff15ffff640 1 --2- 172.21.15.37:0/3643536226 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7ff148005150 0x7ff14800b9c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T10:30:45.462 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:45.456+0000 7ff15f7fe640 1 --2- 172.21.15.37:0/3643536226 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7ff1480039e0 0x7ff148003e10 secure :-1 s=READY pgs=605 cs=0 l=1 rev1=1 crypto rx=0x7ff150001910 tx=0x7ff148005830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=66d0bee00a057e75 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:30:45.462 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:45.457+0000 7ff15e7fc640 1 -- 172.21.15.37:0/3643536226 <== mon.0 v2:172.21.15.37:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff150037070 con 0x7ff1480039e0 2024-04-25T10:30:45.462 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:45.457+0000 7ff15e7fc640 1 -- 172.21.15.37:0/3643536226 <== mon.0 v2:172.21.15.37:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7ff15002ac90 con 0x7ff1480039e0 2024-04-25T10:30:45.462 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:45.457+0000 7ff15e7fc640 1 -- 172.21.15.37:0/3643536226 <== mon.0 v2:172.21.15.37:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff1500014e0 con 0x7ff1480039e0 2024-04-25T10:30:45.462 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:45.457+0000 7ff1661c5640 1 -- 172.21.15.37:0/3643536226 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7ff1480039e0 msgr2=0x7ff148003e10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:30:45.462 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:45.457+0000 7ff1661c5640 1 --2- 172.21.15.37:0/3643536226 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7ff1480039e0 0x7ff148003e10 secure :-1 s=READY pgs=605 cs=0 l=1 rev1=1 crypto rx=0x7ff150001910 tx=0x7ff148005830 comp rx=0 tx=0).stop 2024-04-25T10:30:45.462 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:45.457+0000 7ff15ffff640 1 -- 172.21.15.37:0/3643536226 reap_dead start 2024-04-25T10:30:45.462 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:45.457+0000 7ff1661c5640 1 -- 172.21.15.37:0/3643536226 shutdown_connections 2024-04-25T10:30:45.462 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:45.457+0000 7ff1661c5640 1 -- 172.21.15.37:0/3643536226 >> 172.21.15.37:0/3643536226 conn(0x7ff1600b7a80 msgr2=0x7ff1600b7e90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:30:45.462 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:45.457+0000 7ff1661c5640 1 -- 172.21.15.37:0/3643536226 shutdown_connections 2024-04-25T10:30:45.463 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:45.457+0000 7ff1661c5640 1 -- 172.21.15.37:0/3643536226 wait complete. 2024-04-25T10:30:45.465 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:45.459+0000 7ff1661c5640 1 Processor -- start 2024-04-25T10:30:45.465 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:45.459+0000 7ff1661c5640 1 -- start start 2024-04-25T10:30:45.465 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:45.461+0000 7ff1661c5640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7ff1480039e0 0x7ff16015c1e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:30:45.466 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:45.461+0000 7ff1661c5640 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7ff1600cec30 0x7ff1600cf040 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:30:45.466 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:45.461+0000 7ff1661c5640 1 --2- >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7ff1600d5ae0 0x7ff1600d5ef0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:30:45.466 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:45.461+0000 7ff1661c5640 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 -- 0x7ff1600d4c40 con 0x7ff1480039e0 2024-04-25T10:30:45.466 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:45.461+0000 7ff1661c5640 1 -- --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_getmap magic: 0 -- 0x7ff1600b5290 con 0x7ff1600cec30 2024-04-25T10:30:45.466 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:45.461+0000 7ff1661c5640 1 -- --> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] -- mon_getmap magic: 0 -- 0x7ff160164ac0 con 0x7ff1600d5ae0 2024-04-25T10:30:45.466 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:45.461+0000 7ff15f7fe640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7ff1480039e0 0x7ff16015c1e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:30:45.466 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:45.461+0000 7ff15f7fe640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7ff1480039e0 0x7ff16015c1e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.37:3300/0 says I am v2:172.21.15.37:58384/0 (socket says 172.21.15.37:58384) 2024-04-25T10:30:45.466 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:45.461+0000 7ff15f7fe640 1 -- 172.21.15.37:0/3145622787 learned_addr learned my addr 172.21.15.37:0/3145622787 (peer_addr_for_me v2:172.21.15.37:0/0) 2024-04-25T10:30:45.466 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:45.461+0000 7ff15effd640 1 --2- 172.21.15.37:0/3145622787 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7ff1600cec30 0x7ff1600cf040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:30:45.466 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:45.461+0000 7ff15ffff640 1 --2- 172.21.15.37:0/3145622787 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7ff1600d5ae0 0x7ff1600d5ef0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:30:45.466 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:45.461+0000 7ff15f7fe640 1 -- 172.21.15.37:0/3145622787 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7ff1600d5ae0 msgr2=0x7ff1600d5ef0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:30:45.467 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:45.461+0000 7ff15f7fe640 1 --2- 172.21.15.37:0/3145622787 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7ff1600d5ae0 0x7ff1600d5ef0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:30:45.467 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:45.461+0000 7ff15f7fe640 1 -- 172.21.15.37:0/3145622787 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7ff1600cec30 msgr2=0x7ff1600cf040 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:30:45.467 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:45.461+0000 7ff15f7fe640 1 --2- 172.21.15.37:0/3145622787 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7ff1600cec30 0x7ff1600cf040 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:30:45.467 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:45.461+0000 7ff15f7fe640 1 -- 172.21.15.37:0/3145622787 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ff1600d6430 con 0x7ff1480039e0 2024-04-25T10:30:45.467 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:45.461+0000 7ff15f7fe640 1 --2- 172.21.15.37:0/3145622787 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7ff1480039e0 0x7ff16015c1e0 secure :-1 s=READY pgs=606 cs=0 l=1 rev1=1 crypto rx=0x7ff150001a30 tx=0x7ff1500012b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:30:45.467 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:45.461+0000 7ff147fff640 1 -- 172.21.15.37:0/3145622787 <== mon.0 v2:172.21.15.37:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff150037040 con 0x7ff1480039e0 2024-04-25T10:30:45.467 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:45.461+0000 7ff147fff640 1 -- 172.21.15.37:0/3145622787 <== mon.0 v2:172.21.15.37:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7ff1500052e0 con 0x7ff1480039e0 2024-04-25T10:30:45.467 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:45.461+0000 7ff147fff640 1 -- 172.21.15.37:0/3145622787 <== mon.0 v2:172.21.15.37:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff150007a70 con 0x7ff1480039e0 2024-04-25T10:30:45.467 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:45.461+0000 7ff1661c5640 1 -- 172.21.15.37:0/3145622787 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7ff1600cf880 con 0x7ff1480039e0 2024-04-25T10:30:45.467 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:45.462+0000 7ff1661c5640 1 -- 172.21.15.37:0/3145622787 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7ff1600cfe40 con 0x7ff1480039e0 2024-04-25T10:30:45.467 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:45.463+0000 7ff1661c5640 1 -- 172.21.15.37:0/3145622787 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7ff16006a140 con 0x7ff1480039e0 2024-04-25T10:30:45.467 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:45.463+0000 7ff147fff640 1 -- 172.21.15.37:0/3145622787 <== mon.0 v2:172.21.15.37:3300/0 4 ==== mgrmap(e 5) ==== 96782+0+0 (secure 0 0 0) 0x7ff15003a070 con 0x7ff1480039e0 2024-04-25T10:30:45.468 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:45.464+0000 7ff147fff640 1 --2- 172.21.15.37:0/3145622787 >> [v2:172.21.15.203:6832/2418205237,v1:172.21.15.203:6833/2418205237] conn(0x7ff14c0744e0 0x7ff14c0769f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:30:45.468 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:45.464+0000 7ff147fff640 1 -- 172.21.15.37:0/3145622787 <== mon.0 v2:172.21.15.37:3300/0 5 ==== osd_map(44..44 src has 1..44) ==== 6248+0+0 (secure 0 0 0) 0x7ff1500b53f0 con 0x7ff1480039e0 2024-04-25T10:30:45.468 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:45.464+0000 7ff15effd640 1 --2- 172.21.15.37:0/3145622787 >> [v2:172.21.15.203:6832/2418205237,v1:172.21.15.203:6833/2418205237] conn(0x7ff14c0744e0 0x7ff14c0769f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:30:45.468 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:45.464+0000 7ff15effd640 1 --2- 172.21.15.37:0/3145622787 >> [v2:172.21.15.203:6832/2418205237,v1:172.21.15.203:6833/2418205237] conn(0x7ff14c0744e0 0x7ff14c0769f0 secure :-1 s=READY pgs=278 cs=0 l=1 rev1=1 crypto rx=0x7ff1600b80b0 tx=0x7ff154009040 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:30:45.469 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:45.468+0000 7ff147fff640 1 -- 172.21.15.37:0/3145622787 <== mon.0 v2:172.21.15.37:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187246 (secure 0 0 0) 0x7ff1600d6430 con 0x7ff1480039e0 2024-04-25T10:30:45.693 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:45.692+0000 7ff1661c5640 1 -- 172.21.15.37:0/3145622787 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_command({"prefix": "fs ls"} v 0) -- 0x7ff1600d6990 con 0x7ff1480039e0 2024-04-25T10:30:45.694 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:45.693+0000 7ff147fff640 1 -- 172.21.15.37:0/3145622787 <== mon.0 v2:172.21.15.37:3300/0 7 ==== mon_command_ack([{"prefix": "fs ls"}]=0 v27) ==== 53+0+156 (secure 0 0 0) 0x7ff15007f960 con 0x7ff1480039e0 2024-04-25T10:30:45.694 INFO:teuthology.orchestra.run.smithi037.stdout:name: cephfs, metadata pool: cephfs_metadata, data pools: [cephfs_data ] 2024-04-25T10:30:45.694 INFO:teuthology.orchestra.run.smithi037.stdout:name: recovery_fs, metadata pool: recovery_fs_metadata, data pools: [cephfs_data ] 2024-04-25T10:30:45.696 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:45.694+0000 7ff1661c5640 1 -- 172.21.15.37:0/3145622787 >> [v2:172.21.15.203:6832/2418205237,v1:172.21.15.203:6833/2418205237] conn(0x7ff14c0744e0 msgr2=0x7ff14c0769f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:30:45.696 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:45.694+0000 7ff1661c5640 1 --2- 172.21.15.37:0/3145622787 >> [v2:172.21.15.203:6832/2418205237,v1:172.21.15.203:6833/2418205237] conn(0x7ff14c0744e0 0x7ff14c0769f0 secure :-1 s=READY pgs=278 cs=0 l=1 rev1=1 crypto rx=0x7ff1600b80b0 tx=0x7ff154009040 comp rx=0 tx=0).stop 2024-04-25T10:30:45.696 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:45.694+0000 7ff1661c5640 1 -- 172.21.15.37:0/3145622787 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7ff1480039e0 msgr2=0x7ff16015c1e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:30:45.696 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:45.694+0000 7ff1661c5640 1 --2- 172.21.15.37:0/3145622787 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7ff1480039e0 0x7ff16015c1e0 secure :-1 s=READY pgs=606 cs=0 l=1 rev1=1 crypto rx=0x7ff150001a30 tx=0x7ff1500012b0 comp rx=0 tx=0).stop 2024-04-25T10:30:45.696 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:45.694+0000 7ff1661c5640 1 -- 172.21.15.37:0/3145622787 shutdown_connections 2024-04-25T10:30:45.696 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:45.694+0000 7ff1661c5640 1 --2- 172.21.15.37:0/3145622787 >> [v2:172.21.15.203:6832/2418205237,v1:172.21.15.203:6833/2418205237] conn(0x7ff14c0744e0 0x7ff14c0769f0 unknown :-1 s=CLOSED pgs=278 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:30:45.696 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:45.694+0000 7ff1661c5640 1 --2- 172.21.15.37:0/3145622787 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7ff1600d5ae0 0x7ff1600d5ef0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:30:45.696 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:45.694+0000 7ff1661c5640 1 --2- 172.21.15.37:0/3145622787 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7ff1600cec30 0x7ff1600cf040 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:30:45.697 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:45.694+0000 7ff1661c5640 1 --2- 172.21.15.37:0/3145622787 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7ff1480039e0 0x7ff16015c1e0 unknown :-1 s=CLOSED pgs=606 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:30:45.697 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:45.694+0000 7ff1661c5640 1 -- 172.21.15.37:0/3145622787 >> 172.21.15.37:0/3145622787 conn(0x7ff1600b7a80 msgr2=0x7ff16015f2c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:30:45.697 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:45.695+0000 7ff1661c5640 1 -- 172.21.15.37:0/3145622787 shutdown_connections 2024-04-25T10:30:45.697 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:45.695+0000 7ff1661c5640 1 -- 172.21.15.37:0/3145622787 wait complete. 2024-04-25T10:30:45.707 INFO:tasks.cephfs.mount:Mounting Ceph FS recovery_fs; just confirmed its presence on cluster 2024-04-25T10:30:45.707 INFO:tasks.cephfs.mount:Mounting Ceph FS. Following are details of mount; remember "None" represents Python type None - 2024-04-25T10:30:45.707 INFO:tasks.cephfs.mount:self.client_remote.hostname = smithi037.front.sepia.ceph.com 2024-04-25T10:30:45.707 INFO:tasks.cephfs.mount:self.client.name = client.0 2024-04-25T10:30:45.707 INFO:tasks.cephfs.mount:self.hostfs_mntpt = /home/ubuntu/cephtest/mnt.0 2024-04-25T10:30:45.707 INFO:tasks.cephfs.mount:self.cephfs_name = recovery_fs 2024-04-25T10:30:45.707 INFO:tasks.cephfs.mount:self.cephfs_mntpt = None 2024-04-25T10:30:45.707 INFO:tasks.cephfs.mount:self.client_keyring_path = None 2024-04-25T10:30:45.707 INFO:tasks.cephfs.mount:Setting the 'ceph-ns--home-ubuntu-cephtest-mnt.0' netns for '/home/ubuntu/cephtest/mnt.0' 2024-04-25T10:30:45.707 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T10:30:45.708 DEBUG:teuthology.orchestra.run.smithi037:> ip addr 2024-04-25T10:30:45.723 INFO:teuthology.orchestra.run.smithi037.stdout:1: lo: mtu 65536 qdisc noqueue state UNKNOWN group default qlen 1000 2024-04-25T10:30:45.723 INFO:teuthology.orchestra.run.smithi037.stdout: link/loopback 00:00:00:00:00:00 brd 00:00:00:00:00:00 2024-04-25T10:30:45.723 INFO:teuthology.orchestra.run.smithi037.stdout: inet 127.0.0.1/8 scope host lo 2024-04-25T10:30:45.723 INFO:teuthology.orchestra.run.smithi037.stdout: valid_lft forever preferred_lft forever 2024-04-25T10:30:45.724 INFO:teuthology.orchestra.run.smithi037.stdout: inet6 ::1/128 scope host 2024-04-25T10:30:45.724 INFO:teuthology.orchestra.run.smithi037.stdout: valid_lft forever preferred_lft forever 2024-04-25T10:30:45.724 INFO:teuthology.orchestra.run.smithi037.stdout:2: eno1: mtu 1500 qdisc mq state DOWN group default qlen 1000 2024-04-25T10:30:45.724 INFO:teuthology.orchestra.run.smithi037.stdout: link/ether 0c:c4:7a:6b:fd:18 brd ff:ff:ff:ff:ff:ff 2024-04-25T10:30:45.724 INFO:teuthology.orchestra.run.smithi037.stdout: altname enp7s0f0 2024-04-25T10:30:45.724 INFO:teuthology.orchestra.run.smithi037.stdout:3: eno2: mtu 1500 qdisc mq state DOWN group default qlen 1000 2024-04-25T10:30:45.724 INFO:teuthology.orchestra.run.smithi037.stdout: link/ether 0c:c4:7a:6b:fd:19 brd ff:ff:ff:ff:ff:ff 2024-04-25T10:30:45.724 INFO:teuthology.orchestra.run.smithi037.stdout: altname enp7s0f1 2024-04-25T10:30:45.724 INFO:teuthology.orchestra.run.smithi037.stdout:4: ens1f0: mtu 1500 qdisc mq state UP group default qlen 1000 2024-04-25T10:30:45.724 INFO:teuthology.orchestra.run.smithi037.stdout: link/ether 0c:c4:7a:88:77:94 brd ff:ff:ff:ff:ff:ff 2024-04-25T10:30:45.724 INFO:teuthology.orchestra.run.smithi037.stdout: altname enp1s0f0 2024-04-25T10:30:45.724 INFO:teuthology.orchestra.run.smithi037.stdout: inet 172.21.15.37/20 brd 172.21.15.255 scope global dynamic noprefixroute ens1f0 2024-04-25T10:30:45.724 INFO:teuthology.orchestra.run.smithi037.stdout: valid_lft 42622sec preferred_lft 42622sec 2024-04-25T10:30:45.724 INFO:teuthology.orchestra.run.smithi037.stdout: inet6 fe80::ec4:7aff:fe88:7794/64 scope link noprefixroute 2024-04-25T10:30:45.725 INFO:teuthology.orchestra.run.smithi037.stdout: valid_lft forever preferred_lft forever 2024-04-25T10:30:45.725 INFO:teuthology.orchestra.run.smithi037.stdout:5: ens1f1: mtu 1500 qdisc mq state DOWN group default qlen 1000 2024-04-25T10:30:45.725 INFO:teuthology.orchestra.run.smithi037.stdout: link/ether 0c:c4:7a:88:77:95 brd ff:ff:ff:ff:ff:ff 2024-04-25T10:30:45.725 INFO:teuthology.orchestra.run.smithi037.stdout: altname enp1s0f1 2024-04-25T10:30:45.725 INFO:teuthology.orchestra.run.smithi037.stdout:6: ceph-brx: mtu 1500 qdisc noqueue state UP group default qlen 1000 2024-04-25T10:30:45.725 INFO:teuthology.orchestra.run.smithi037.stdout: link/ether ce:87:f0:58:c7:15 brd ff:ff:ff:ff:ff:ff 2024-04-25T10:30:45.725 INFO:teuthology.orchestra.run.smithi037.stdout: inet 192.168.255.254/16 brd 192.168.255.255 scope global ceph-brx 2024-04-25T10:30:45.725 INFO:teuthology.orchestra.run.smithi037.stdout: valid_lft forever preferred_lft forever 2024-04-25T10:30:45.725 INFO:teuthology.orchestra.run.smithi037.stdout: inet6 fe80::9827:f7ff:febf:df95/64 scope link 2024-04-25T10:30:45.725 INFO:teuthology.orchestra.run.smithi037.stdout: valid_lft forever preferred_lft forever 2024-04-25T10:30:45.725 INFO:teuthology.orchestra.run.smithi037.stdout:7: brx.0@if2: mtu 1500 qdisc noqueue master ceph-brx state UP group default qlen 1000 2024-04-25T10:30:45.725 INFO:teuthology.orchestra.run.smithi037.stdout: link/ether ce:87:f0:58:c7:15 brd ff:ff:ff:ff:ff:ff link-netns ceph-ns--home-ubuntu-cephtest-mnt.0 2024-04-25T10:30:45.725 INFO:teuthology.orchestra.run.smithi037.stdout: inet6 fe80::cc87:f0ff:fe58:c715/64 scope link 2024-04-25T10:30:45.725 INFO:teuthology.orchestra.run.smithi037.stdout: valid_lft forever preferred_lft forever 2024-04-25T10:30:45.725 INFO:teuthology.orchestra.run.smithi037.stdout:8: brx.1@if2: mtu 1500 qdisc noqueue master ceph-brx state UP group default qlen 1000 2024-04-25T10:30:45.725 INFO:teuthology.orchestra.run.smithi037.stdout: link/ether fa:eb:96:ac:44:0e brd ff:ff:ff:ff:ff:ff link-netns ceph-ns--home-ubuntu-cephtest-mnt.2 2024-04-25T10:30:45.725 INFO:teuthology.orchestra.run.smithi037.stdout: inet6 fe80::f8eb:96ff:feac:440e/64 scope link 2024-04-25T10:30:45.726 INFO:teuthology.orchestra.run.smithi037.stdout: valid_lft forever preferred_lft forever 2024-04-25T10:30:45.726 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T10:30:45.726 DEBUG:teuthology.orchestra.run.smithi037:> echo 1 | sudo tee /proc/sys/net/ipv4/ip_forward 2024-04-25T10:30:45.787 INFO:teuthology.orchestra.run.smithi037.stdout:1 2024-04-25T10:30:45.788 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T10:30:45.789 DEBUG:teuthology.orchestra.run.smithi037:> route 2024-04-25T10:30:45.845 INFO:teuthology.orchestra.run.smithi037.stdout:Kernel IP routing table 2024-04-25T10:30:45.845 INFO:teuthology.orchestra.run.smithi037.stdout:Destination Gateway Genmask Flags Metric Ref Use Iface 2024-04-25T10:30:45.845 INFO:teuthology.orchestra.run.smithi037.stdout:default _gateway 0.0.0.0 UG 100 0 0 ens1f0 2024-04-25T10:30:45.845 INFO:teuthology.orchestra.run.smithi037.stdout:172.21.0.0 0.0.0.0 255.255.240.0 U 100 0 0 ens1f0 2024-04-25T10:30:45.845 INFO:teuthology.orchestra.run.smithi037.stdout:192.168.0.0 0.0.0.0 255.255.0.0 U 0 0 0 ceph-brx 2024-04-25T10:30:45.846 DEBUG:teuthology.orchestra.run.smithi037:> (cd / && exec stdin-killer --timeout=300 -- bash -c ' 2024-04-25T10:30:45.846 DEBUG:teuthology.orchestra.run.smithi037:> set -e 2024-04-25T10:30:45.846 DEBUG:teuthology.orchestra.run.smithi037:> sudo iptables -A FORWARD -o ens1f0 -i ceph-brx -j ACCEPT 2024-04-25T10:30:45.846 DEBUG:teuthology.orchestra.run.smithi037:> sudo iptables -A FORWARD -i ens1f0 -o ceph-brx -j ACCEPT 2024-04-25T10:30:45.846 DEBUG:teuthology.orchestra.run.smithi037:> sudo iptables -t nat -A POSTROUTING -s 192.168.255.254/16 -o ens1f0 -j MASQUERADE 2024-04-25T10:30:45.846 DEBUG:teuthology.orchestra.run.smithi037:> ') 2024-04-25T10:30:45.866 INFO:tasks.ceph.mds.c.smithi037.stderr:2024-04-25T10:30:45.865+0000 7f12c23d1640 -1 mds.pinger is_rank_lagging: rank=0 was never sent ping request. 2024-04-25T10:30:45.914 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:45 stdin-killer INFO: expiration expected; waiting 300 seconds for command to complete 2024-04-25T10:30:45.945 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:45 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-04-25T10:30:45.951 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T10:30:45.952 DEBUG:teuthology.orchestra.run.smithi037:> ip netns list 2024-04-25T10:30:45.967 INFO:teuthology.orchestra.run.smithi037.stdout:ceph-ns--home-ubuntu-cephtest-mnt.2 (id: 1) 2024-04-25T10:30:45.967 INFO:teuthology.orchestra.run.smithi037.stdout:ceph-ns--home-ubuntu-cephtest-mnt.0 (id: 0) 2024-04-25T10:30:45.967 INFO:tasks.cephfs.mount:Resuming the 'ceph-ns--home-ubuntu-cephtest-mnt.0' netns for '/home/ubuntu/cephtest/mnt.0' 2024-04-25T10:30:45.967 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T10:30:45.967 DEBUG:teuthology.orchestra.run.smithi037:> sudo ip link set brx.0 up 2024-04-25T10:30:46.031 INFO:tasks.cephfs.fuse_mount:Client client.0 config is {} 2024-04-25T10:30:46.032 INFO:teuthology.orchestra.run:Running command with timeout 60 2024-04-25T10:30:46.032 DEBUG:teuthology.orchestra.run.smithi037:> mkdir -p -v /home/ubuntu/cephtest/mnt.0 2024-04-25T10:30:46.090 INFO:teuthology.orchestra.run.smithi037.stdout:mkdir: created directory '/home/ubuntu/cephtest/mnt.0' 2024-04-25T10:30:46.090 INFO:teuthology.orchestra.run:Running command with timeout 60 2024-04-25T10:30:46.090 DEBUG:teuthology.orchestra.run.smithi037:> chmod 0000 /home/ubuntu/cephtest/mnt.0 2024-04-25T10:30:46.145 DEBUG:teuthology.orchestra.run.smithi037:> sudo modprobe fuse 2024-04-25T10:30:46.209 DEBUG:teuthology.orchestra.run.smithi037:> cat /proc/self/mounts | awk '{print $2}' 2024-04-25T10:30:46.266 INFO:teuthology.orchestra.run.smithi037.stdout:/proc 2024-04-25T10:30:46.266 INFO:teuthology.orchestra.run.smithi037.stdout:/sys 2024-04-25T10:30:46.266 INFO:teuthology.orchestra.run.smithi037.stdout:/dev 2024-04-25T10:30:46.266 INFO:teuthology.orchestra.run.smithi037.stdout:/sys/kernel/security 2024-04-25T10:30:46.266 INFO:teuthology.orchestra.run.smithi037.stdout:/dev/shm 2024-04-25T10:30:46.266 INFO:teuthology.orchestra.run.smithi037.stdout:/dev/pts 2024-04-25T10:30:46.266 INFO:teuthology.orchestra.run.smithi037.stdout:/run 2024-04-25T10:30:46.266 INFO:teuthology.orchestra.run.smithi037.stdout:/sys/fs/cgroup 2024-04-25T10:30:46.267 INFO:teuthology.orchestra.run.smithi037.stdout:/sys/fs/pstore 2024-04-25T10:30:46.267 INFO:teuthology.orchestra.run.smithi037.stdout:/sys/fs/bpf 2024-04-25T10:30:46.267 INFO:teuthology.orchestra.run.smithi037.stdout:/ 2024-04-25T10:30:46.267 INFO:teuthology.orchestra.run.smithi037.stdout:/sys/fs/selinux 2024-04-25T10:30:46.267 INFO:teuthology.orchestra.run.smithi037.stdout:/proc/sys/fs/binfmt_misc 2024-04-25T10:30:46.267 INFO:teuthology.orchestra.run.smithi037.stdout:/dev/hugepages 2024-04-25T10:30:46.267 INFO:teuthology.orchestra.run.smithi037.stdout:/dev/mqueue 2024-04-25T10:30:46.267 INFO:teuthology.orchestra.run.smithi037.stdout:/sys/kernel/debug 2024-04-25T10:30:46.267 INFO:teuthology.orchestra.run.smithi037.stdout:/sys/kernel/tracing 2024-04-25T10:30:46.267 INFO:teuthology.orchestra.run.smithi037.stdout:/sys/fs/fuse/connections 2024-04-25T10:30:46.267 INFO:teuthology.orchestra.run.smithi037.stdout:/sys/kernel/config 2024-04-25T10:30:46.267 INFO:teuthology.orchestra.run.smithi037.stdout:/run/credentials/systemd-sysusers.service 2024-04-25T10:30:46.267 INFO:teuthology.orchestra.run.smithi037.stdout:/run/credentials/systemd-sysctl.service 2024-04-25T10:30:46.267 INFO:teuthology.orchestra.run.smithi037.stdout:/run/credentials/systemd-tmpfiles-setup-dev.service 2024-04-25T10:30:46.268 INFO:teuthology.orchestra.run.smithi037.stdout:/run/credentials/systemd-tmpfiles-setup.service 2024-04-25T10:30:46.268 INFO:teuthology.orchestra.run.smithi037.stdout:/var/lib/nfs/rpc_pipefs 2024-04-25T10:30:46.268 INFO:teuthology.orchestra.run.smithi037.stdout:/run/user/1000 2024-04-25T10:30:46.268 INFO:teuthology.orchestra.run.smithi037.stdout:/var/lib/ceph 2024-04-25T10:30:46.268 INFO:teuthology.orchestra.run.smithi037.stdout:/var/lib/ceph/osd/ceph-0 2024-04-25T10:30:46.268 INFO:teuthology.orchestra.run.smithi037.stdout:/var/lib/ceph/osd/ceph-1 2024-04-25T10:30:46.268 INFO:teuthology.orchestra.run.smithi037.stdout:/var/lib/ceph/osd/ceph-2 2024-04-25T10:30:46.268 INFO:teuthology.orchestra.run.smithi037.stdout:/var/lib/ceph/osd/ceph-3 2024-04-25T10:30:46.268 INFO:teuthology.orchestra.run.smithi037.stdout:/run/netns 2024-04-25T10:30:46.268 INFO:teuthology.orchestra.run.smithi037.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 2024-04-25T10:30:46.268 INFO:teuthology.orchestra.run.smithi037.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 2024-04-25T10:30:46.268 INFO:teuthology.orchestra.run.smithi037.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 2024-04-25T10:30:46.268 INFO:teuthology.orchestra.run.smithi037.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 2024-04-25T10:30:46.269 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T10:30:46.269 DEBUG:teuthology.orchestra.run.smithi037:> ls /sys/fs/fuse/connections 2024-04-25T10:30:46.321 INFO:tasks.cephfs.fuse_mount:Pre-mount connections: [] 2024-04-25T10:30:46.321 DEBUG:teuthology.orchestra.run.smithi037:> (cd /home/ubuntu/cephtest && exec sudo nsenter --net=/var/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-fuse -f --admin-socket '/var/run/ceph/$cluster-$name.$pid.asok' /home/ubuntu/cephtest/mnt.0 --id 0 --client_fs=recovery_fs) 2024-04-25T10:30:46.363 DEBUG:teuthology.orchestra.run.smithi037:> sudo modprobe fuse 2024-04-25T10:30:46.387 DEBUG:teuthology.orchestra.run.smithi037:> cat /proc/self/mounts | awk '{print $2}' 2024-04-25T10:30:46.446 INFO:teuthology.orchestra.run.smithi037.stdout:/proc 2024-04-25T10:30:46.446 INFO:teuthology.orchestra.run.smithi037.stdout:/sys 2024-04-25T10:30:46.446 INFO:teuthology.orchestra.run.smithi037.stdout:/dev 2024-04-25T10:30:46.446 INFO:teuthology.orchestra.run.smithi037.stdout:/sys/kernel/security 2024-04-25T10:30:46.447 INFO:teuthology.orchestra.run.smithi037.stdout:/dev/shm 2024-04-25T10:30:46.447 INFO:teuthology.orchestra.run.smithi037.stdout:/dev/pts 2024-04-25T10:30:46.447 INFO:teuthology.orchestra.run.smithi037.stdout:/run 2024-04-25T10:30:46.447 INFO:teuthology.orchestra.run.smithi037.stdout:/sys/fs/cgroup 2024-04-25T10:30:46.447 INFO:teuthology.orchestra.run.smithi037.stdout:/sys/fs/pstore 2024-04-25T10:30:46.447 INFO:teuthology.orchestra.run.smithi037.stdout:/sys/fs/bpf 2024-04-25T10:30:46.447 INFO:teuthology.orchestra.run.smithi037.stdout:/ 2024-04-25T10:30:46.447 INFO:teuthology.orchestra.run.smithi037.stdout:/sys/fs/selinux 2024-04-25T10:30:46.447 INFO:teuthology.orchestra.run.smithi037.stdout:/proc/sys/fs/binfmt_misc 2024-04-25T10:30:46.447 INFO:teuthology.orchestra.run.smithi037.stdout:/dev/hugepages 2024-04-25T10:30:46.447 INFO:teuthology.orchestra.run.smithi037.stdout:/dev/mqueue 2024-04-25T10:30:46.447 INFO:teuthology.orchestra.run.smithi037.stdout:/sys/kernel/debug 2024-04-25T10:30:46.447 INFO:teuthology.orchestra.run.smithi037.stdout:/sys/kernel/tracing 2024-04-25T10:30:46.447 INFO:teuthology.orchestra.run.smithi037.stdout:/sys/fs/fuse/connections 2024-04-25T10:30:46.448 INFO:teuthology.orchestra.run.smithi037.stdout:/sys/kernel/config 2024-04-25T10:30:46.448 INFO:teuthology.orchestra.run.smithi037.stdout:/run/credentials/systemd-sysusers.service 2024-04-25T10:30:46.448 INFO:teuthology.orchestra.run.smithi037.stdout:/run/credentials/systemd-sysctl.service 2024-04-25T10:30:46.448 INFO:teuthology.orchestra.run.smithi037.stdout:/run/credentials/systemd-tmpfiles-setup-dev.service 2024-04-25T10:30:46.448 INFO:teuthology.orchestra.run.smithi037.stdout:/run/credentials/systemd-tmpfiles-setup.service 2024-04-25T10:30:46.448 INFO:teuthology.orchestra.run.smithi037.stdout:/var/lib/nfs/rpc_pipefs 2024-04-25T10:30:46.448 INFO:teuthology.orchestra.run.smithi037.stdout:/run/user/1000 2024-04-25T10:30:46.448 INFO:teuthology.orchestra.run.smithi037.stdout:/var/lib/ceph 2024-04-25T10:30:46.448 INFO:teuthology.orchestra.run.smithi037.stdout:/var/lib/ceph/osd/ceph-0 2024-04-25T10:30:46.448 INFO:teuthology.orchestra.run.smithi037.stdout:/var/lib/ceph/osd/ceph-1 2024-04-25T10:30:46.448 INFO:teuthology.orchestra.run.smithi037.stdout:/var/lib/ceph/osd/ceph-2 2024-04-25T10:30:46.448 INFO:teuthology.orchestra.run.smithi037.stdout:/var/lib/ceph/osd/ceph-3 2024-04-25T10:30:46.448 INFO:teuthology.orchestra.run.smithi037.stdout:/run/netns 2024-04-25T10:30:46.448 INFO:teuthology.orchestra.run.smithi037.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 2024-04-25T10:30:46.449 INFO:teuthology.orchestra.run.smithi037.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 2024-04-25T10:30:46.449 INFO:teuthology.orchestra.run.smithi037.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 2024-04-25T10:30:46.449 INFO:teuthology.orchestra.run.smithi037.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 2024-04-25T10:30:46.449 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T10:30:46.449 DEBUG:teuthology.orchestra.run.smithi037:> ls /sys/fs/fuse/connections 2024-04-25T10:30:46.457 INFO:tasks.cephfs.fuse_mount.ceph-fuse.0.smithi037.stderr:2024-04-25T10:30:46.456+0000 7f273348f480 -1 init, newargv = 0x55841391c390 newargc=15 2024-04-25T10:30:46.457 INFO:tasks.cephfs.fuse_mount.ceph-fuse.0.smithi037.stderr:ceph-fuse[51764]: starting ceph client 2024-04-25T10:30:46.463 INFO:tasks.cephfs.fuse_mount.ceph-fuse.0.smithi037.stderr:ceph-fuse[51764]: starting fuse 2024-04-25T10:30:46.472 INFO:teuthology.orchestra.run.smithi037.stdout:43 2024-04-25T10:30:46.473 INFO:tasks.cephfs.fuse_mount:Post-mount connections: [43] 2024-04-25T10:30:46.473 DEBUG:teuthology.orchestra.run.smithi037:> sudo stdin-killer -- python3 -c ' 2024-04-25T10:30:46.473 DEBUG:teuthology.orchestra.run.smithi037:> import glob 2024-04-25T10:30:46.473 DEBUG:teuthology.orchestra.run.smithi037:> import re 2024-04-25T10:30:46.473 DEBUG:teuthology.orchestra.run.smithi037:> import os 2024-04-25T10:30:46.473 DEBUG:teuthology.orchestra.run.smithi037:> import subprocess 2024-04-25T10:30:46.473 DEBUG:teuthology.orchestra.run.smithi037:> 2024-04-25T10:30:46.473 DEBUG:teuthology.orchestra.run.smithi037:> def _find_admin_socket(client_name): 2024-04-25T10:30:46.473 DEBUG:teuthology.orchestra.run.smithi037:> asok_path = "/var/run/ceph/ceph-client.0.*.asok" 2024-04-25T10:30:46.473 DEBUG:teuthology.orchestra.run.smithi037:> files = glob.glob(asok_path) 2024-04-25T10:30:46.473 DEBUG:teuthology.orchestra.run.smithi037:> mountpoint = "/home/ubuntu/cephtest/mnt.0" 2024-04-25T10:30:46.473 DEBUG:teuthology.orchestra.run.smithi037:> 2024-04-25T10:30:46.473 DEBUG:teuthology.orchestra.run.smithi037:> # Given a non-glob path, it better be there 2024-04-25T10:30:46.473 DEBUG:teuthology.orchestra.run.smithi037:> if "*" not in asok_path: 2024-04-25T10:30:46.474 DEBUG:teuthology.orchestra.run.smithi037:> assert(len(files) == 1) 2024-04-25T10:30:46.474 DEBUG:teuthology.orchestra.run.smithi037:> return files[0] 2024-04-25T10:30:46.474 DEBUG:teuthology.orchestra.run.smithi037:> 2024-04-25T10:30:46.474 DEBUG:teuthology.orchestra.run.smithi037:> for f in files: 2024-04-25T10:30:46.474 DEBUG:teuthology.orchestra.run.smithi037:> pid = re.match(".*\.(\d+)\.asok$", f).group(1) 2024-04-25T10:30:46.474 DEBUG:teuthology.orchestra.run.smithi037:> if os.path.exists("/proc/{0}".format(pid)): 2024-04-25T10:30:46.474 DEBUG:teuthology.orchestra.run.smithi037:> with open("/proc/{0}/cmdline".format(pid), '"'"'r'"'"') as proc_f: 2024-04-25T10:30:46.474 DEBUG:teuthology.orchestra.run.smithi037:> contents = proc_f.read() 2024-04-25T10:30:46.474 DEBUG:teuthology.orchestra.run.smithi037:> if mountpoint in contents: 2024-04-25T10:30:46.474 DEBUG:teuthology.orchestra.run.smithi037:> return f 2024-04-25T10:30:46.475 DEBUG:teuthology.orchestra.run.smithi037:> raise RuntimeError("Client socket {0} not found".format(client_name)) 2024-04-25T10:30:46.475 DEBUG:teuthology.orchestra.run.smithi037:> 2024-04-25T10:30:46.475 DEBUG:teuthology.orchestra.run.smithi037:> print(_find_admin_socket("client.0")) 2024-04-25T10:30:46.475 DEBUG:teuthology.orchestra.run.smithi037:> ' 2024-04-25T10:30:46.592 INFO:teuthology.orchestra.run.smithi037.stdout:/var/run/ceph/ceph-client.0.51764.asok 2024-04-25T10:30:46.596 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:46 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-04-25T10:30:46.603 INFO:tasks.cephfs.fuse_mount:Found client admin socket at /var/run/ceph/ceph-client.0.51764.asok 2024-04-25T10:30:46.603 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T10:30:46.603 DEBUG:teuthology.orchestra.run.smithi037:> sudo ceph --admin-daemon /var/run/ceph/ceph-client.0.51764.asok status 2024-04-25T10:30:46.747 INFO:teuthology.orchestra.run.smithi037.stdout:{ 2024-04-25T10:30:46.748 INFO:teuthology.orchestra.run.smithi037.stdout: "metadata": { 2024-04-25T10:30:46.748 INFO:teuthology.orchestra.run.smithi037.stdout: "ceph_sha1": "994e2f20102a20fa9a27966c14b625cc01884df8", 2024-04-25T10:30:46.748 INFO:teuthology.orchestra.run.smithi037.stdout: "ceph_version": "ceph version 19.0.0-3291-g994e2f20 (994e2f20102a20fa9a27966c14b625cc01884df8) squid (dev)", 2024-04-25T10:30:46.748 INFO:teuthology.orchestra.run.smithi037.stdout: "entity_id": "0", 2024-04-25T10:30:46.748 INFO:teuthology.orchestra.run.smithi037.stdout: "hostname": "smithi037", 2024-04-25T10:30:46.748 INFO:teuthology.orchestra.run.smithi037.stdout: "mount_point": "/home/ubuntu/cephtest/mnt.0", 2024-04-25T10:30:46.748 INFO:teuthology.orchestra.run.smithi037.stdout: "pid": "51764", 2024-04-25T10:30:46.748 INFO:teuthology.orchestra.run.smithi037.stdout: "root": "/" 2024-04-25T10:30:46.748 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-04-25T10:30:46.748 INFO:teuthology.orchestra.run.smithi037.stdout: "dentry_count": 0, 2024-04-25T10:30:46.748 INFO:teuthology.orchestra.run.smithi037.stdout: "dentry_pinned_count": 0, 2024-04-25T10:30:46.748 INFO:teuthology.orchestra.run.smithi037.stdout: "id": 5955, 2024-04-25T10:30:46.748 INFO:teuthology.orchestra.run.smithi037.stdout: "inst": { 2024-04-25T10:30:46.748 INFO:teuthology.orchestra.run.smithi037.stdout: "name": { 2024-04-25T10:30:46.749 INFO:teuthology.orchestra.run.smithi037.stdout: "type": "client", 2024-04-25T10:30:46.749 INFO:teuthology.orchestra.run.smithi037.stdout: "num": 5955 2024-04-25T10:30:46.749 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-04-25T10:30:46.749 INFO:teuthology.orchestra.run.smithi037.stdout: "addr": { 2024-04-25T10:30:46.749 INFO:teuthology.orchestra.run.smithi037.stdout: "type": "v1", 2024-04-25T10:30:46.749 INFO:teuthology.orchestra.run.smithi037.stdout: "addr": "192.168.0.1:0", 2024-04-25T10:30:46.749 INFO:teuthology.orchestra.run.smithi037.stdout: "nonce": 2861119118 2024-04-25T10:30:46.749 INFO:teuthology.orchestra.run.smithi037.stdout: } 2024-04-25T10:30:46.749 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-04-25T10:30:46.749 INFO:teuthology.orchestra.run.smithi037.stdout: "addr": { 2024-04-25T10:30:46.749 INFO:teuthology.orchestra.run.smithi037.stdout: "type": "v1", 2024-04-25T10:30:46.749 INFO:teuthology.orchestra.run.smithi037.stdout: "addr": "192.168.0.1:0", 2024-04-25T10:30:46.749 INFO:teuthology.orchestra.run.smithi037.stdout: "nonce": 2861119118 2024-04-25T10:30:46.749 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-04-25T10:30:46.749 INFO:teuthology.orchestra.run.smithi037.stdout: "inst_str": "client.5955 192.168.0.1:0/2861119118", 2024-04-25T10:30:46.749 INFO:teuthology.orchestra.run.smithi037.stdout: "addr_str": "192.168.0.1:0/2861119118", 2024-04-25T10:30:46.750 INFO:teuthology.orchestra.run.smithi037.stdout: "inode_count": 1, 2024-04-25T10:30:46.750 INFO:teuthology.orchestra.run.smithi037.stdout: "mds_epoch": 27, 2024-04-25T10:30:46.750 INFO:teuthology.orchestra.run.smithi037.stdout: "osd_epoch": 44, 2024-04-25T10:30:46.750 INFO:teuthology.orchestra.run.smithi037.stdout: "osd_epoch_barrier": 0, 2024-04-25T10:30:46.750 INFO:teuthology.orchestra.run.smithi037.stdout: "blocklisted": false, 2024-04-25T10:30:46.750 INFO:teuthology.orchestra.run.smithi037.stdout: "fs_name": "recovery_fs" 2024-04-25T10:30:46.750 INFO:teuthology.orchestra.run.smithi037.stdout:} 2024-04-25T10:30:46.756 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T10:30:46.756 DEBUG:teuthology.orchestra.run.smithi037:> stat --file-system '--printf=%T 2024-04-25T10:30:46.756 DEBUG:teuthology.orchestra.run.smithi037:> ' -- /home/ubuntu/cephtest/mnt.0 2024-04-25T10:30:46.772 INFO:teuthology.orchestra.run.smithi037.stdout:fuseblk 2024-04-25T10:30:46.772 INFO:tasks.cephfs.fuse_mount:ceph-fuse is mounted on /home/ubuntu/cephtest/mnt.0 2024-04-25T10:30:46.772 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T10:30:46.772 DEBUG:teuthology.orchestra.run.smithi037:> sudo chmod 1777 /home/ubuntu/cephtest/mnt.0 2024-04-25T10:30:46.837 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T10:30:46.837 DEBUG:teuthology.orchestra.run.smithi037:> (cd /home/ubuntu/cephtest/mnt.0 && exec ls subdir) 2024-04-25T10:30:46.897 INFO:teuthology.orchestra.run.smithi037.stdout:sixmegs 2024-04-25T10:30:46.898 DEBUG:teuthology.orchestra.run.smithi037:> stdin-killer -- python3 -c ' 2024-04-25T10:30:46.898 DEBUG:teuthology.orchestra.run.smithi037:> import os 2024-04-25T10:30:46.898 DEBUG:teuthology.orchestra.run.smithi037:> import stat 2024-04-25T10:30:46.898 DEBUG:teuthology.orchestra.run.smithi037:> import json 2024-04-25T10:30:46.898 DEBUG:teuthology.orchestra.run.smithi037:> import sys 2024-04-25T10:30:46.898 DEBUG:teuthology.orchestra.run.smithi037:> 2024-04-25T10:30:46.898 DEBUG:teuthology.orchestra.run.smithi037:> try: 2024-04-25T10:30:46.899 DEBUG:teuthology.orchestra.run.smithi037:> s = os.stat('"'"'/home/ubuntu/cephtest/mnt.0/subdir/sixmegs'"'"') 2024-04-25T10:30:46.899 DEBUG:teuthology.orchestra.run.smithi037:> except OSError as e: 2024-04-25T10:30:46.899 DEBUG:teuthology.orchestra.run.smithi037:> sys.exit(e.errno) 2024-04-25T10:30:46.899 DEBUG:teuthology.orchestra.run.smithi037:> 2024-04-25T10:30:46.899 DEBUG:teuthology.orchestra.run.smithi037:> attrs = ["st_mode", "st_ino", "st_dev", "st_nlink", "st_uid", "st_gid", "st_size", "st_atime", "st_mtime", "st_ctime"] 2024-04-25T10:30:46.899 DEBUG:teuthology.orchestra.run.smithi037:> print(json.dumps( 2024-04-25T10:30:46.899 DEBUG:teuthology.orchestra.run.smithi037:> dict([(a, getattr(s, a)) for a in attrs]), 2024-04-25T10:30:46.899 DEBUG:teuthology.orchestra.run.smithi037:> indent=2)) 2024-04-25T10:30:46.899 DEBUG:teuthology.orchestra.run.smithi037:> ' 2024-04-25T10:30:47.008 INFO:teuthology.orchestra.run.smithi037.stdout:{ 2024-04-25T10:30:47.008 INFO:teuthology.orchestra.run.smithi037.stdout: "st_mode": 33088, 2024-04-25T10:30:47.008 INFO:teuthology.orchestra.run.smithi037.stdout: "st_ino": 1099511627777, 2024-04-25T10:30:47.008 INFO:teuthology.orchestra.run.smithi037.stdout: "st_dev": 43, 2024-04-25T10:30:47.008 INFO:teuthology.orchestra.run.smithi037.stdout: "st_nlink": 1, 2024-04-25T10:30:47.009 INFO:teuthology.orchestra.run.smithi037.stdout: "st_uid": 0, 2024-04-25T10:30:47.009 INFO:teuthology.orchestra.run.smithi037.stdout: "st_gid": 0, 2024-04-25T10:30:47.009 INFO:teuthology.orchestra.run.smithi037.stdout: "st_size": 6291456, 2024-04-25T10:30:47.009 INFO:teuthology.orchestra.run.smithi037.stdout: "st_atime": 1714041024.0, 2024-04-25T10:30:47.009 INFO:teuthology.orchestra.run.smithi037.stdout: "st_mtime": 1714041024.0, 2024-04-25T10:30:47.009 INFO:teuthology.orchestra.run.smithi037.stdout: "st_ctime": 1714041024.0 2024-04-25T10:30:47.009 INFO:teuthology.orchestra.run.smithi037.stdout:} 2024-04-25T10:30:47.011 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:47 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-04-25T10:30:47.018 INFO:teuthology.nuke.actions:Clearing teuthology firewall rules... 2024-04-25T10:30:47.018 DEBUG:teuthology.orchestra.run.smithi037:> sudo sh -c 'iptables-save | grep -v teuthology | iptables-restore' 2024-04-25T10:30:47.091 DEBUG:teuthology.orchestra.run.smithi203:> sudo sh -c 'iptables-save | grep -v teuthology | iptables-restore' 2024-04-25T10:30:47.144 INFO:teuthology.nuke.actions:Cleared teuthology firewall rules. 2024-04-25T10:30:47.144 INFO:tasks.cephfs.mount:Terminating background process 2024-04-25T10:30:47.144 DEBUG:teuthology.orchestra.run.smithi037:> set -ex 2024-04-25T10:30:47.144 DEBUG:teuthology.orchestra.run.smithi037:> dd if=/proc/self/mounts of=/dev/stdout 2024-04-25T10:30:47.160 DEBUG:teuthology.orchestra.run.smithi037:> set -ex 2024-04-25T10:30:47.160 DEBUG:teuthology.orchestra.run.smithi037:> dd if=/proc/self/mounts of=/dev/stdout 2024-04-25T10:30:47.217 DEBUG:teuthology.orchestra.run.smithi037:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-04-25T10:30:47.368 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:47.367+0000 7f445f427640 1 Processor -- start 2024-04-25T10:30:47.368 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:47.367+0000 7f445f427640 1 -- start start 2024-04-25T10:30:47.370 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:47.368+0000 7f445f427640 1 --2- >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f44580d5160 0x7f44580b4e80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:30:47.370 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:47.368+0000 7f445f427640 1 --2- >> v2:172.21.15.203:3300/0 conn(0x7f44580d0990 0x7f44580d2b20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:30:47.370 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:47.368+0000 7f445f427640 1 -- --> v1:172.21.15.203:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f44580d1370 con 0x7f44580d0da0 2024-04-25T10:30:47.370 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:47.368+0000 7f445f427640 1 -- --> v2:172.21.15.203:3300/0 -- mon_getmap magic: 0 -- 0x7f44580d1650 con 0x7f44580d0990 2024-04-25T10:30:47.370 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:47.368+0000 7f445f427640 1 -- --> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] -- mon_getmap magic: 0 -- 0x7f44580d4680 con 0x7f44580d5160 2024-04-25T10:30:47.370 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:47.369+0000 7f445d1c3640 1 --2- >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f44580d5160 0x7f44580b4e80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:30:47.370 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:47.369+0000 7f445c9c2640 1 --2- >> v2:172.21.15.203:3300/0 conn(0x7f44580d0990 0x7f44580d2b20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:30:47.370 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:47.369+0000 7f445d9c4640 1 --1- >> v1:172.21.15.203:6789/0 conn(0x7f44580d0da0 0x7f445806db10 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.203:6789/0 says I am v1:172.21.15.37:40572/0 (socket says 172.21.15.37:40572) 2024-04-25T10:30:47.370 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:47.369+0000 7f445d9c4640 1 -- 172.21.15.37:0/1962750643 learned_addr learned my addr 172.21.15.37:0/1962750643 (peer_addr_for_me v1:172.21.15.37:0/0) 2024-04-25T10:30:47.371 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:47.369+0000 7f445d1c3640 1 -- 172.21.15.37:0/1962750643 >> v2:172.21.15.203:3300/0 conn(0x7f44580d0990 msgr2=0x7f44580d2b20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:30:47.371 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:47.369+0000 7f445d1c3640 1 --2- 172.21.15.37:0/1962750643 >> v2:172.21.15.203:3300/0 conn(0x7f44580d0990 0x7f44580d2b20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:30:47.371 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:47.369+0000 7f445d1c3640 1 -- 172.21.15.37:0/1962750643 >> v1:172.21.15.203:6789/0 conn(0x7f44580d0da0 legacy=0x7f445806db10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:30:47.371 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:47.369+0000 7f445d1c3640 1 -- 172.21.15.37:0/1962750643 --> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f44580d4960 con 0x7f44580d5160 2024-04-25T10:30:47.371 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:47.369+0000 7f445c9c2640 1 --2- 172.21.15.37:0/1962750643 >> v2:172.21.15.203:3300/0 conn(0x7f44580d0990 0x7f44580d2b20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-25T10:30:47.371 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:47.370+0000 7f445d1c3640 1 --2- 172.21.15.37:0/1962750643 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f44580d5160 0x7f44580b4e80 secure :-1 s=READY pgs=62 cs=0 l=1 rev1=1 crypto rx=0x7f44540097a0 tx=0x7f445402a060 comp rx=0 tx=0).ready entity=mon.2 client_cookie=cf82d0f3ecc0a7e0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:30:47.371 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:47.370+0000 7f444ffff640 1 -- 172.21.15.37:0/1962750643 <== mon.2 v2:172.21.15.203:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4454037070 con 0x7f44580d5160 2024-04-25T10:30:47.372 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:47.370+0000 7f444ffff640 1 -- 172.21.15.37:0/1962750643 <== mon.2 v2:172.21.15.203:3301/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f445402adb0 con 0x7f44580d5160 2024-04-25T10:30:47.372 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:47.370+0000 7f444ffff640 1 -- 172.21.15.37:0/1962750643 <== mon.2 v2:172.21.15.203:3301/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4454032660 con 0x7f44580d5160 2024-04-25T10:30:47.372 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:47.371+0000 7f445f427640 1 -- 172.21.15.37:0/1962750643 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f44580d5160 msgr2=0x7f44580b4e80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:30:47.373 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:47.371+0000 7f445f427640 1 --2- 172.21.15.37:0/1962750643 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f44580d5160 0x7f44580b4e80 secure :-1 s=READY pgs=62 cs=0 l=1 rev1=1 crypto rx=0x7f44540097a0 tx=0x7f445402a060 comp rx=0 tx=0).stop 2024-04-25T10:30:47.373 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:47.371+0000 7f445f427640 1 -- 172.21.15.37:0/1962750643 shutdown_connections 2024-04-25T10:30:47.373 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:47.371+0000 7f445f427640 1 --2- 172.21.15.37:0/1962750643 >> v2:172.21.15.203:3300/0 conn(0x7f44580d0990 0x7f44580d2b20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:30:47.373 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:47.371+0000 7f445f427640 1 --2- 172.21.15.37:0/1962750643 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f44580d5160 0x7f44580b4e80 unknown :-1 s=CLOSED pgs=62 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:30:47.373 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:47.371+0000 7f445f427640 1 -- 172.21.15.37:0/1962750643 >> 172.21.15.37:0/1962750643 conn(0x7f44580b7a80 msgr2=0x7f44580b7e90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:30:47.373 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:47.371+0000 7f445f427640 1 -- 172.21.15.37:0/1962750643 shutdown_connections 2024-04-25T10:30:47.373 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:47.371+0000 7f445f427640 1 -- 172.21.15.37:0/1962750643 wait complete. 2024-04-25T10:30:47.374 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:47.373+0000 7f445f427640 1 Processor -- start 2024-04-25T10:30:47.374 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:47.373+0000 7f445f427640 1 -- start start 2024-04-25T10:30:47.376 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:47.374+0000 7f445f427640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f44580d0990 0x7f4458154db0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:30:47.376 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:47.374+0000 7f445f427640 1 --2- >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f44580d0da0 0x7f44581ea4e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:30:47.376 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:47.374+0000 7f445f427640 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f44580d5160 0x7f445815dee0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:30:47.376 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:47.374+0000 7f445f427640 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 -- 0x7f4458164c40 con 0x7f44580d0990 2024-04-25T10:30:47.377 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:47.374+0000 7f445f427640 1 -- --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_getmap magic: 0 -- 0x7f4458164ac0 con 0x7f44580d5160 2024-04-25T10:30:47.377 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:47.374+0000 7f445f427640 1 -- --> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] -- mon_getmap magic: 0 -- 0x7f44580d3060 con 0x7f44580d0da0 2024-04-25T10:30:47.377 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:47.374+0000 7f445d1c3640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f44580d0990 0x7f4458154db0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:30:47.377 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:47.374+0000 7f445d1c3640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f44580d0990 0x7f4458154db0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.37:3300/0 says I am v2:172.21.15.37:58390/0 (socket says 172.21.15.37:58390) 2024-04-25T10:30:47.377 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:47.374+0000 7f445d1c3640 1 -- 172.21.15.37:0/3705929019 learned_addr learned my addr 172.21.15.37:0/3705929019 (peer_addr_for_me v2:172.21.15.37:0/0) 2024-04-25T10:30:47.377 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:47.374+0000 7f445c9c2640 1 --2- 172.21.15.37:0/3705929019 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f44580d0da0 0x7f44581ea4e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:30:47.377 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:47.374+0000 7f445d9c4640 1 --2- 172.21.15.37:0/3705929019 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f44580d5160 0x7f445815dee0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:30:47.377 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:47.375+0000 7f445d1c3640 1 -- 172.21.15.37:0/3705929019 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f44580d0da0 msgr2=0x7f44581ea4e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:30:47.377 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:47.375+0000 7f445d1c3640 1 --2- 172.21.15.37:0/3705929019 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f44580d0da0 0x7f44581ea4e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:30:47.377 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:47.375+0000 7f445d1c3640 1 -- 172.21.15.37:0/3705929019 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f44580d5160 msgr2=0x7f445815dee0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:30:47.377 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:47.375+0000 7f445d1c3640 1 --2- 172.21.15.37:0/3705929019 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f44580d5160 0x7f445815dee0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:30:47.377 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:47.375+0000 7f445d1c3640 1 -- 172.21.15.37:0/3705929019 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f44581f6780 con 0x7f44580d0990 2024-04-25T10:30:47.377 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:47.375+0000 7f445d1c3640 1 --2- 172.21.15.37:0/3705929019 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f44580d0990 0x7f4458154db0 secure :-1 s=READY pgs=610 cs=0 l=1 rev1=1 crypto rx=0x7f44540098c0 tx=0x7f44540046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:30:47.377 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:47.375+0000 7f444dffb640 1 -- 172.21.15.37:0/3705929019 <== mon.0 v2:172.21.15.37:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4454042070 con 0x7f44580d0990 2024-04-25T10:30:47.377 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:47.375+0000 7f444dffb640 1 -- 172.21.15.37:0/3705929019 <== mon.0 v2:172.21.15.37:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f445403d440 con 0x7f44580d0990 2024-04-25T10:30:47.378 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:47.375+0000 7f445f427640 1 -- 172.21.15.37:0/3705929019 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f44581f6a60 con 0x7f44580d0990 2024-04-25T10:30:47.378 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:47.375+0000 7f444dffb640 1 -- 172.21.15.37:0/3705929019 <== mon.0 v2:172.21.15.37:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4454037040 con 0x7f44580d0990 2024-04-25T10:30:47.378 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:47.375+0000 7f445f427640 1 -- 172.21.15.37:0/3705929019 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f44581f7020 con 0x7f44580d0990 2024-04-25T10:30:47.378 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:47.376+0000 7f445f427640 1 -- 172.21.15.37:0/3705929019 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f445806a140 con 0x7f44580d0990 2024-04-25T10:30:47.379 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:47.377+0000 7f444dffb640 1 -- 172.21.15.37:0/3705929019 <== mon.0 v2:172.21.15.37:3300/0 4 ==== mgrmap(e 5) ==== 96782+0+0 (secure 0 0 0) 0x7f4454045070 con 0x7f44580d0990 2024-04-25T10:30:47.379 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:47.377+0000 7f444dffb640 1 --2- 172.21.15.37:0/3705929019 >> [v2:172.21.15.203:6832/2418205237,v1:172.21.15.203:6833/2418205237] conn(0x7f443c0744e0 0x7f443c0769f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:30:47.379 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:47.377+0000 7f444dffb640 1 -- 172.21.15.37:0/3705929019 <== mon.0 v2:172.21.15.37:3300/0 5 ==== osd_map(44..44 src has 1..44) ==== 6248+0+0 (secure 0 0 0) 0x7f44540b5610 con 0x7f44580d0990 2024-04-25T10:30:47.379 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:47.377+0000 7f445c9c2640 1 --2- 172.21.15.37:0/3705929019 >> [v2:172.21.15.203:6832/2418205237,v1:172.21.15.203:6833/2418205237] conn(0x7f443c0744e0 0x7f443c0769f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:30:47.379 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:47.378+0000 7f445c9c2640 1 --2- 172.21.15.37:0/3705929019 >> [v2:172.21.15.203:6832/2418205237,v1:172.21.15.203:6833/2418205237] conn(0x7f443c0744e0 0x7f443c0769f0 secure :-1 s=READY pgs=279 cs=0 l=1 rev1=1 crypto rx=0x7f44580d0810 tx=0x7f4448009190 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:30:47.383 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:47.382+0000 7f444dffb640 1 -- 172.21.15.37:0/3705929019 <== mon.0 v2:172.21.15.37:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187246 (secure 0 0 0) 0x7f445407f530 con 0x7f44580d0990 2024-04-25T10:30:47.568 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:47.567+0000 7f445f427640 1 -- 172.21.15.37:0/3705929019 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7f44580d3f50 con 0x7f44580d0990 2024-04-25T10:30:47.569 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:47.568+0000 7f444dffb640 1 -- 172.21.15.37:0/3705929019 <== mon.0 v2:172.21.15.37:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v44) ==== 74+0+18626 (secure 0 0 0) 0x7f4454083830 con 0x7f44580d0990 2024-04-25T10:30:47.569 INFO:teuthology.orchestra.run.smithi037.stdout: 2024-04-25T10:30:47.570 INFO:teuthology.orchestra.run.smithi037.stdout:{"epoch":44,"fsid":"f4acda28-be34-492a-b286-994afd36f61d","created":"2024-04-25T10:28:43.985718+0000","modified":"2024-04-25T10:30:29.873697+0000","last_up_change":"2024-04-25T10:28:54.900654+0000","last_in_change":"2024-04-25T10:28:46.216973+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":6,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-04-25T10:28:56.202153+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"19","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":4,"pool_name":"cephfs_metadata","create_time":"2024-04-25T10:30:10.008132+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"35","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.75,"score_stable":1.75,"optimal_score":1,"raw_score_acting":1.75,"raw_score_stable":1.75,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":5,"pool_name":"cephfs_data","create_time":"2024-04-25T10:30:11.084827+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"43","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"recovery_fs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":6,"pool_name":"recovery_fs_metadata","create_time":"2024-04-25T10:30:26.903737+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"44","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"recovery_fs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"a2f79028-b332-4166-9ebb-a7024a37ea22","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":42,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.37:6816","nonce":2746074720},{"type":"v1","addr":"172.21.15.37:6817","nonce":2746074720}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.37:6818","nonce":2746074720},{"type":"v1","addr":"172.21.15.37:6819","nonce":2746074720}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.37:6822","nonce":2746074720},{"type":"v1","addr":"172.21.15.37:6823","nonce":2746074720}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.37:6820","nonce":2746074720},{"type":"v1","addr":"172.21.15.37:6821","nonce":2746074720}]},"public_addr":"172.21.15.37:6817/2746074720","cluster_addr":"172.21.15.37:6819/2746074720","heartbeat_back_addr":"172.21.15.37:6823/2746074720","heartbeat_front_addr":"172.21.15.37:6821/2746074720","state":["exists","up"]},{"osd":1,"uuid":"973159de-c2ea-4083-aecc-69373d1c1132","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":42,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.37:6800","nonce":1690341662},{"type":"v1","addr":"172.21.15.37:6801","nonce":1690341662}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.37:6802","nonce":1690341662},{"type":"v1","addr":"172.21.15.37:6803","nonce":1690341662}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.37:6806","nonce":1690341662},{"type":"v1","addr":"172.21.15.37:6807","nonce":1690341662}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.37:6804","nonce":1690341662},{"type":"v1","addr":"172.21.15.37:6805","nonce":1690341662}]},"public_addr":"172.21.15.37:6801/1690341662","cluster_addr":"172.21.15.37:6803/1690341662","heartbeat_back_addr":"172.21.15.37:6807/1690341662","heartbeat_front_addr":"172.21.15.37:6805/1690341662","state":["exists","up"]},{"osd":2,"uuid":"4ae7a0e9-cb3f-484c-a75f-f3766b721c47","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":42,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.37:6808","nonce":2754669649},{"type":"v1","addr":"172.21.15.37:6809","nonce":2754669649}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.37:6810","nonce":2754669649},{"type":"v1","addr":"172.21.15.37:6811","nonce":2754669649}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.37:6814","nonce":2754669649},{"type":"v1","addr":"172.21.15.37:6815","nonce":2754669649}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.37:6812","nonce":2754669649},{"type":"v1","addr":"172.21.15.37:6813","nonce":2754669649}]},"public_addr":"172.21.15.37:6809/2754669649","cluster_addr":"172.21.15.37:6811/2754669649","heartbeat_back_addr":"172.21.15.37:6815/2754669649","heartbeat_front_addr":"172.21.15.37:6813/2754669649","state":["exists","up"]},{"osd":3,"uuid":"1ba4a746-1608-47a7-ace2-960a5cfae934","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":42,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.37:6824","nonce":1815290618},{"type":"v1","addr":"172.21.15.37:6825","nonce":1815290618}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.37:6826","nonce":1815290618},{"type":"v1","addr":"172.21.15.37:6827","nonce":1815290618}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.37:6830","nonce":1815290618},{"type":"v1","addr":"172.21.15.37:6831","nonce":1815290618}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.37:6828","nonce":1815290618},{"type":"v1","addr":"172.21.15.37:6829","nonce":1815290618}]},"public_addr":"172.21.15.37:6825/1815290618","cluster_addr":"172.21.15.37:6827/1815290618","heartbeat_back_addr":"172.21.15.37:6831/1815290618","heartbeat_front_addr":"172.21.15.37:6829/1815290618","state":["exists","up"]},{"osd":4,"uuid":"d4f59f5d-0413-4b79-87ee-bb212c1fcf62","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":42,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6800","nonce":4051532063},{"type":"v1","addr":"172.21.15.203:6801","nonce":4051532063}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6802","nonce":4051532063},{"type":"v1","addr":"172.21.15.203:6803","nonce":4051532063}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6806","nonce":4051532063},{"type":"v1","addr":"172.21.15.203:6807","nonce":4051532063}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6804","nonce":4051532063},{"type":"v1","addr":"172.21.15.203:6805","nonce":4051532063}]},"public_addr":"172.21.15.203:6801/4051532063","cluster_addr":"172.21.15.203:6803/4051532063","heartbeat_back_addr":"172.21.15.203:6807/4051532063","heartbeat_front_addr":"172.21.15.203:6805/4051532063","state":["exists","up"]},{"osd":5,"uuid":"69710a0e-41d8-4e7f-9a10-17ce540b67be","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":42,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6824","nonce":1203822890},{"type":"v1","addr":"172.21.15.203:6825","nonce":1203822890}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6826","nonce":1203822890},{"type":"v1","addr":"172.21.15.203:6827","nonce":1203822890}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6830","nonce":1203822890},{"type":"v1","addr":"172.21.15.203:6831","nonce":1203822890}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6828","nonce":1203822890},{"type":"v1","addr":"172.21.15.203:6829","nonce":1203822890}]},"public_addr":"172.21.15.203:6825/1203822890","cluster_addr":"172.21.15.203:6827/1203822890","heartbeat_back_addr":"172.21.15.203:6831/1203822890","heartbeat_front_addr":"172.21.15.203:6829/1203822890","state":["exists","up"]},{"osd":6,"uuid":"f29dd047-3b48-473d-a235-6642a6293f20","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":42,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6810","nonce":2309615014},{"type":"v1","addr":"172.21.15.203:6812","nonce":2309615014}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6814","nonce":2309615014},{"type":"v1","addr":"172.21.15.203:6816","nonce":2309615014}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6822","nonce":2309615014},{"type":"v1","addr":"172.21.15.203:6823","nonce":2309615014}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6818","nonce":2309615014},{"type":"v1","addr":"172.21.15.203:6820","nonce":2309615014}]},"public_addr":"172.21.15.203:6812/2309615014","cluster_addr":"172.21.15.203:6816/2309615014","heartbeat_back_addr":"172.21.15.203:6823/2309615014","heartbeat_front_addr":"172.21.15.203:6820/2309615014","state":["exists","up"]},{"osd":7,"uuid":"6e691644-378b-4fdc-8e86-fb5f56e31f29","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":42,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6808","nonce":3688099014},{"type":"v1","addr":"172.21.15.203:6809","nonce":3688099014}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6811","nonce":3688099014},{"type":"v1","addr":"172.21.15.203:6813","nonce":3688099014}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6819","nonce":3688099014},{"type":"v1","addr":"172.21.15.203:6821","nonce":3688099014}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6815","nonce":3688099014},{"type":"v1","addr":"172.21.15.203:6817","nonce":3688099014}]},"public_addr":"172.21.15.203:6809/3688099014","cluster_addr":"172.21.15.203:6813/3688099014","heartbeat_back_addr":"172.21.15.203:6821/3688099014","heartbeat_front_addr":"172.21.15.203:6817/3688099014","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T10:28:51.419141+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T10:28:51.366448+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T10:28:51.348276+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T10:28:51.380519+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T10:28:51.105812+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T10:28:51.384496+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T10:28:51.388755+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T10:28:51.366958+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.37:6835/311552410":"2024-04-26T10:30:24.853732+0000","172.21.15.37:6832/311552410":"2024-04-26T10:30:24.853732+0000"},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-04-25T10:30:47.571 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:47.570+0000 7f445f427640 1 -- 172.21.15.37:0/3705929019 >> [v2:172.21.15.203:6832/2418205237,v1:172.21.15.203:6833/2418205237] conn(0x7f443c0744e0 msgr2=0x7f443c0769f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:30:47.571 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:47.570+0000 7f445f427640 1 --2- 172.21.15.37:0/3705929019 >> [v2:172.21.15.203:6832/2418205237,v1:172.21.15.203:6833/2418205237] conn(0x7f443c0744e0 0x7f443c0769f0 secure :-1 s=READY pgs=279 cs=0 l=1 rev1=1 crypto rx=0x7f44580d0810 tx=0x7f4448009190 comp rx=0 tx=0).stop 2024-04-25T10:30:47.571 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:47.570+0000 7f445f427640 1 -- 172.21.15.37:0/3705929019 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f44580d0990 msgr2=0x7f4458154db0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:30:47.572 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:47.570+0000 7f445f427640 1 --2- 172.21.15.37:0/3705929019 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f44580d0990 0x7f4458154db0 secure :-1 s=READY pgs=610 cs=0 l=1 rev1=1 crypto rx=0x7f44540098c0 tx=0x7f44540046c0 comp rx=0 tx=0).stop 2024-04-25T10:30:47.572 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:47.570+0000 7f445f427640 1 -- 172.21.15.37:0/3705929019 shutdown_connections 2024-04-25T10:30:47.572 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:47.570+0000 7f445f427640 1 --2- 172.21.15.37:0/3705929019 >> [v2:172.21.15.203:6832/2418205237,v1:172.21.15.203:6833/2418205237] conn(0x7f443c0744e0 0x7f443c0769f0 unknown :-1 s=CLOSED pgs=279 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:30:47.572 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:47.570+0000 7f445f427640 1 --2- 172.21.15.37:0/3705929019 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f44580d5160 0x7f445815dee0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:30:47.572 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:47.570+0000 7f445f427640 1 --2- 172.21.15.37:0/3705929019 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f44580d0da0 0x7f44581ea4e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:30:47.572 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:47.570+0000 7f445f427640 1 --2- 172.21.15.37:0/3705929019 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f44580d0990 0x7f4458154db0 unknown :-1 s=CLOSED pgs=610 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:30:47.572 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:47.570+0000 7f445f427640 1 -- 172.21.15.37:0/3705929019 >> 172.21.15.37:0/3705929019 conn(0x7f44580b7a80 msgr2=0x7f44580b6d80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:30:47.572 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:47.570+0000 7f445f427640 1 -- 172.21.15.37:0/3705929019 shutdown_connections 2024-04-25T10:30:47.572 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:47.570+0000 7f445f427640 1 -- 172.21.15.37:0/3705929019 wait complete. 2024-04-25T10:30:47.583 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2024-04-25T10:28:56.202153+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '19', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 8, 'score_stable': 8, 'optimal_score': 0.25, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 4, 'pool_name': 'cephfs_metadata', 'create_time': '2024-04-25T10:30:10.008132+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '35', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_autoscale_bias': 4, 'pg_num_min': 16, 'recovery_priority': 5}, 'application_metadata': {'cephfs': {'metadata': 'cephfs'}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 1.75, 'score_stable': 1.75, 'optimal_score': 1, 'raw_score_acting': 1.75, 'raw_score_stable': 1.75, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 5, 'pool_name': 'cephfs_data', 'create_time': '2024-04-25T10:30:11.084827+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 64, 'pg_placement_num': 64, 'pg_placement_num_target': 64, 'pg_num_target': 64, 'pg_num_pending': 64, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '43', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.8999999761581421}, 'application_metadata': {'cephfs': {'data': 'recovery_fs'}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 1.5, 'score_stable': 1.5, 'optimal_score': 1, 'raw_score_acting': 1.5, 'raw_score_stable': 1.5, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 6, 'pool_name': 'recovery_fs_metadata', 'create_time': '2024-04-25T10:30:26.903737+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '44', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_autoscale_bias': 4, 'pg_num_min': 16, 'recovery_priority': 5}, 'application_metadata': {'cephfs': {'metadata': 'recovery_fs'}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 1.5, 'score_stable': 1.5, 'optimal_score': 1, 'raw_score_acting': 1.5, 'raw_score_stable': 1.5, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}] 2024-04-25T10:30:47.584 DEBUG:teuthology.orchestra.run.smithi037:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2024-04-25T10:30:47.699 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:47.697+0000 7fecfdcca640 1 Processor -- start 2024-04-25T10:30:47.699 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:47.697+0000 7fecfdcca640 1 -- start start 2024-04-25T10:30:47.710 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:47.699+0000 7fecfdcca640 1 --2- >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7fecf80b5fd0 0x7fecf80b2ca0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:30:47.710 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:47.699+0000 7fecfdcca640 1 --2- >> v2:172.21.15.203:3300/0 conn(0x7fecf808cc90 0x7fecf808d0a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:30:47.711 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:47.699+0000 7fecfdcca640 1 -- --> v1:172.21.15.37:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fecf80cf120 con 0x7fecf808d5e0 2024-04-25T10:30:47.711 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:47.699+0000 7fecfdcca640 1 -- --> v2:172.21.15.203:3300/0 -- mon_getmap magic: 0 -- 0x7fecf80cf400 con 0x7fecf808cc90 2024-04-25T10:30:47.711 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:47.699+0000 7fecfdcca640 1 -- --> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] -- mon_getmap magic: 0 -- 0x7fecf80cf6e0 con 0x7fecf80b5fd0 2024-04-25T10:30:47.711 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:47.699+0000 7fecf7fff640 1 --1- >> v1:172.21.15.37:6789/0 conn(0x7fecf808d5e0 0x7fecf80d18c0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.37:6789/0 says I am v1:172.21.15.37:55694/0 (socket says 172.21.15.37:55694) 2024-04-25T10:30:47.711 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:47.699+0000 7fecf7fff640 1 -- 172.21.15.37:0/2380923034 learned_addr learned my addr 172.21.15.37:0/2380923034 (peer_addr_for_me v1:172.21.15.37:0/0) 2024-04-25T10:30:47.711 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:47.699+0000 7fecf77fe640 1 --2- 172.21.15.37:0/2380923034 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7fecf80b5fd0 0x7fecf80b2ca0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:30:47.712 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:47.699+0000 7fecf6ffd640 1 --2- 172.21.15.37:0/2380923034 >> v2:172.21.15.203:3300/0 conn(0x7fecf808cc90 0x7fecf808d0a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:30:47.712 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:47.699+0000 7fecf67fc640 1 -- 172.21.15.37:0/2380923034 <== mon.0 v1:172.21.15.37:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 2789178026 0 0) 0x7fecf80cf120 con 0x7fecf808d5e0 2024-04-25T10:30:47.712 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:47.699+0000 7fecf67fc640 1 -- 172.21.15.37:0/2380923034 --> v1:172.21.15.37:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7fece0003520 con 0x7fecf808d5e0 2024-04-25T10:30:47.712 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:47.699+0000 7fecf67fc640 1 -- 172.21.15.37:0/2380923034 <== mon.0 v1:172.21.15.37:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 2025101753 0 0) 0x7fece0003520 con 0x7fecf808d5e0 2024-04-25T10:30:47.712 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:47.699+0000 7fecf67fc640 1 -- 172.21.15.37:0/2380923034 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7fecf80b5fd0 msgr2=0x7fecf80b2ca0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:30:47.712 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:47.699+0000 7fecf67fc640 1 --2- 172.21.15.37:0/2380923034 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7fecf80b5fd0 0x7fecf80b2ca0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:30:47.712 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:47.699+0000 7fecf67fc640 1 -- 172.21.15.37:0/2380923034 >> v2:172.21.15.203:3300/0 conn(0x7fecf808cc90 msgr2=0x7fecf808d0a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:30:47.713 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:47.699+0000 7fecf67fc640 1 --2- 172.21.15.37:0/2380923034 >> v2:172.21.15.203:3300/0 conn(0x7fecf808cc90 0x7fecf808d0a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:30:47.713 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:47.699+0000 7fecf67fc640 1 -- 172.21.15.37:0/2380923034 --> v1:172.21.15.37:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fecf80cf9c0 con 0x7fecf808d5e0 2024-04-25T10:30:47.713 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:47.699+0000 7fecf67fc640 1 -- 172.21.15.37:0/2380923034 <== mon.0 v1:172.21.15.37:6789/0 3 ==== mon_map magic: 0 ==== 413+0+0 (unknown 1577794600 0 0) 0x7fecf00019f0 con 0x7fecf808d5e0 2024-04-25T10:30:47.713 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:47.700+0000 7fecf67fc640 1 -- 172.21.15.37:0/2380923034 >> v1:172.21.15.37:6789/0 conn(0x7fecf808d5e0 legacy=0x7fecf80d18c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:30:47.713 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:47.700+0000 7fecf67fc640 1 --2- 172.21.15.37:0/2380923034 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7fecf80b5fd0 0x7fece00039b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:30:47.713 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:47.700+0000 7fecf67fc640 1 --2- 172.21.15.37:0/2380923034 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fece0003f30 0x7fece0004800 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:30:47.714 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:47.700+0000 7fecf67fc640 1 --2- 172.21.15.37:0/2380923034 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fece0004d40 0x7fece0005150 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:30:47.714 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:47.700+0000 7fecf67fc640 1 -- 172.21.15.37:0/2380923034 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 -- 0x7fecf80cf400 con 0x7fece0003f30 2024-04-25T10:30:47.714 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:47.700+0000 7fecf67fc640 1 -- 172.21.15.37:0/2380923034 --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_getmap magic: 0 -- 0x7fecf80cf6e0 con 0x7fece0004d40 2024-04-25T10:30:47.714 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:47.700+0000 7fecf67fc640 1 -- 172.21.15.37:0/2380923034 --> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] -- mon_getmap magic: 0 -- 0x7fece000bac0 con 0x7fecf80b5fd0 2024-04-25T10:30:47.714 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:47.700+0000 7fecf6ffd640 1 --2- 172.21.15.37:0/2380923034 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fece0003f30 0x7fece0004800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:30:47.714 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:47.700+0000 7fecf77fe640 1 --2- 172.21.15.37:0/2380923034 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7fecf80b5fd0 0x7fece00039b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:30:47.715 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:47.700+0000 7fecf7fff640 1 --2- 172.21.15.37:0/2380923034 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fece0004d40 0x7fece0005150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:30:47.715 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:47.700+0000 7fecf6ffd640 1 -- 172.21.15.37:0/2380923034 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7fecf80b5fd0 msgr2=0x7fece00039b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:30:47.715 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:47.700+0000 7fecf6ffd640 1 --2- 172.21.15.37:0/2380923034 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7fecf80b5fd0 0x7fece00039b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:30:47.715 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:47.700+0000 7fecf6ffd640 1 -- 172.21.15.37:0/2380923034 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fece0004d40 msgr2=0x7fece0005150 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:30:47.715 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:47.700+0000 7fecf6ffd640 1 --2- 172.21.15.37:0/2380923034 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fece0004d40 0x7fece0005150 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:30:47.715 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:47.700+0000 7fecf6ffd640 1 -- 172.21.15.37:0/2380923034 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fece0003520 con 0x7fece0003f30 2024-04-25T10:30:47.715 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:47.701+0000 7fecf6ffd640 1 --2- 172.21.15.37:0/2380923034 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fece0003f30 0x7fece0004800 secure :-1 s=READY pgs=612 cs=0 l=1 rev1=1 crypto rx=0x7fecec009ac0 tx=0x7fece0005830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=3e7a1f6e8d63a679 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:30:47.716 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:47.701+0000 7fecf67fc640 1 -- 172.21.15.37:0/2380923034 <== mon.0 v2:172.21.15.37:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fecec036070 con 0x7fece0003f30 2024-04-25T10:30:47.716 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:47.701+0000 7fecf67fc640 1 -- 172.21.15.37:0/2380923034 <== mon.0 v2:172.21.15.37:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fecec031400 con 0x7fece0003f30 2024-04-25T10:30:47.716 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:47.701+0000 7fecf67fc640 1 -- 172.21.15.37:0/2380923034 <== mon.0 v2:172.21.15.37:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fecec009690 con 0x7fece0003f30 2024-04-25T10:30:47.716 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:47.702+0000 7fecfdcca640 1 -- 172.21.15.37:0/2380923034 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fece0003f30 msgr2=0x7fece0004800 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:30:47.716 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:47.702+0000 7fecfdcca640 1 --2- 172.21.15.37:0/2380923034 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fece0003f30 0x7fece0004800 secure :-1 s=READY pgs=612 cs=0 l=1 rev1=1 crypto rx=0x7fecec009ac0 tx=0x7fece0005830 comp rx=0 tx=0).stop 2024-04-25T10:30:47.716 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:47.702+0000 7fecf7fff640 1 -- 172.21.15.37:0/2380923034 reap_dead start 2024-04-25T10:30:47.717 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:47.702+0000 7fecfdcca640 1 -- 172.21.15.37:0/2380923034 shutdown_connections 2024-04-25T10:30:47.717 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:47.702+0000 7fecfdcca640 1 -- 172.21.15.37:0/2380923034 >> 172.21.15.37:0/2380923034 conn(0x7fecf80b5440 msgr2=0x7fecf80b5850 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:30:47.717 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:47.702+0000 7fecfdcca640 1 -- 172.21.15.37:0/2380923034 shutdown_connections 2024-04-25T10:30:47.717 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:47.702+0000 7fecfdcca640 1 -- 172.21.15.37:0/2380923034 wait complete. 2024-04-25T10:30:47.717 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:47.703+0000 7fecfdcca640 1 Processor -- start 2024-04-25T10:30:47.717 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:47.704+0000 7fecfdcca640 1 -- start start 2024-04-25T10:30:47.717 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:47.705+0000 7fecfdcca640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fece0003f30 0x7fecf8160fc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:30:47.718 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:47.705+0000 7fecfdcca640 1 --2- >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7fecf8154d70 0x7fecf81f70f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:30:47.718 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:47.705+0000 7fecfdcca640 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fecf8155180 0x7fecf81f7630 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:30:47.718 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:47.705+0000 7fecfdcca640 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 -- 0x7fecf80b4800 con 0x7fece0003f30 2024-04-25T10:30:47.718 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:47.705+0000 7fecfdcca640 1 -- --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_getmap magic: 0 -- 0x7fecf80b4680 con 0x7fecf8155180 2024-04-25T10:30:47.718 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:47.705+0000 7fecfdcca640 1 -- --> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] -- mon_getmap magic: 0 -- 0x7fecf80b4980 con 0x7fecf8154d70 2024-04-25T10:30:47.718 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:47.705+0000 7fecf77fe640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fece0003f30 0x7fecf8160fc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:30:47.719 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:47.705+0000 7fecf77fe640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fece0003f30 0x7fecf8160fc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.37:3300/0 says I am v2:172.21.15.37:58400/0 (socket says 172.21.15.37:58400) 2024-04-25T10:30:47.719 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:47.705+0000 7fecf77fe640 1 -- 172.21.15.37:0/1917845110 learned_addr learned my addr 172.21.15.37:0/1917845110 (peer_addr_for_me v2:172.21.15.37:0/0) 2024-04-25T10:30:47.719 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:47.705+0000 7fecf7fff640 1 --2- 172.21.15.37:0/1917845110 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fecf8155180 0x7fecf81f7630 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:30:47.719 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:47.705+0000 7fecf6ffd640 1 --2- 172.21.15.37:0/1917845110 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7fecf8154d70 0x7fecf81f70f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:30:47.719 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:47.705+0000 7fecf77fe640 1 -- 172.21.15.37:0/1917845110 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7fecf8154d70 msgr2=0x7fecf81f70f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:30:47.719 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:47.705+0000 7fecf77fe640 1 --2- 172.21.15.37:0/1917845110 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7fecf8154d70 0x7fecf81f70f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:30:47.719 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:47.705+0000 7fecf77fe640 1 -- 172.21.15.37:0/1917845110 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fecf8155180 msgr2=0x7fecf81f7630 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:30:47.720 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:47.705+0000 7fecf77fe640 1 --2- 172.21.15.37:0/1917845110 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fecf8155180 0x7fecf81f7630 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:30:47.720 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:47.705+0000 7fecf77fe640 1 -- 172.21.15.37:0/1917845110 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fecf815f940 con 0x7fece0003f30 2024-04-25T10:30:47.720 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:47.706+0000 7fecf77fe640 1 --2- 172.21.15.37:0/1917845110 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fece0003f30 0x7fecf8160fc0 secure :-1 s=READY pgs=613 cs=0 l=1 rev1=1 crypto rx=0x7fece8011990 tx=0x7fece8011e50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:30:47.720 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:47.706+0000 7fecdffff640 1 -- 172.21.15.37:0/1917845110 <== mon.0 v2:172.21.15.37:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fece800fa70 con 0x7fece0003f30 2024-04-25T10:30:47.720 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:47.706+0000 7fecdffff640 1 -- 172.21.15.37:0/1917845110 <== mon.0 v2:172.21.15.37:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fece800fc10 con 0x7fece0003f30 2024-04-25T10:30:47.720 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:47.706+0000 7fecfdcca640 1 -- 172.21.15.37:0/1917845110 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fecf815fc20 con 0x7fece0003f30 2024-04-25T10:30:47.721 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:47.706+0000 7fecdffff640 1 -- 172.21.15.37:0/1917845110 <== mon.0 v2:172.21.15.37:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fece8014440 con 0x7fece0003f30 2024-04-25T10:30:47.721 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:47.706+0000 7fecfdcca640 1 -- 172.21.15.37:0/1917845110 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fecf809d8a0 con 0x7fece0003f30 2024-04-25T10:30:47.721 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:47.707+0000 7fecfdcca640 1 -- 172.21.15.37:0/1917845110 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fecf806a140 con 0x7fece0003f30 2024-04-25T10:30:47.721 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:47.708+0000 7fecdffff640 1 -- 172.21.15.37:0/1917845110 <== mon.0 v2:172.21.15.37:3300/0 4 ==== mgrmap(e 5) ==== 96782+0+0 (secure 0 0 0) 0x7fece8012040 con 0x7fece0003f30 2024-04-25T10:30:47.721 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:47.708+0000 7fecdffff640 1 --2- 172.21.15.37:0/1917845110 >> [v2:172.21.15.203:6832/2418205237,v1:172.21.15.203:6833/2418205237] conn(0x7fece40744e0 0x7fece40769f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:30:47.721 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:47.708+0000 7fecdffff640 1 -- 172.21.15.37:0/1917845110 <== mon.0 v2:172.21.15.37:3300/0 5 ==== osd_map(44..44 src has 1..44) ==== 6248+0+0 (secure 0 0 0) 0x7fece8095910 con 0x7fece0003f30 2024-04-25T10:30:47.721 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:47.708+0000 7fecf6ffd640 1 --2- 172.21.15.37:0/1917845110 >> [v2:172.21.15.203:6832/2418205237,v1:172.21.15.203:6833/2418205237] conn(0x7fece40744e0 0x7fece40769f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:30:47.722 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:47.709+0000 7fecf6ffd640 1 --2- 172.21.15.37:0/1917845110 >> [v2:172.21.15.203:6832/2418205237,v1:172.21.15.203:6833/2418205237] conn(0x7fece40744e0 0x7fece40769f0 secure :-1 s=READY pgs=280 cs=0 l=1 rev1=1 crypto rx=0x7fecec033000 tx=0x7fecec008030 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:30:47.722 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:47.713+0000 7fecdffff640 1 -- 172.21.15.37:0/1917845110 <== mon.0 v2:172.21.15.37:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187246 (secure 0 0 0) 0x7fece8019030 con 0x7fece0003f30 2024-04-25T10:30:47.900 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:47.899+0000 7fecfdcca640 1 -- 172.21.15.37:0/1917845110 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) -- 0x7fecf80b4ef0 con 0x7fece0003f30 2024-04-25T10:30:47.900 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:47.899+0000 7fecdffff640 1 -- 172.21.15.37:0/1917845110 <== mon.0 v2:172.21.15.37:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v44) ==== 93+0+10 (secure 0 0 0) 0x7fece805ffa0 con 0x7fece0003f30 2024-04-25T10:30:47.901 INFO:teuthology.orchestra.run.smithi037.stdout:pg_num: 1 2024-04-25T10:30:47.902 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:47.901+0000 7fecfdcca640 1 -- 172.21.15.37:0/1917845110 >> [v2:172.21.15.203:6832/2418205237,v1:172.21.15.203:6833/2418205237] conn(0x7fece40744e0 msgr2=0x7fece40769f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:30:47.902 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:47.901+0000 7fecfdcca640 1 --2- 172.21.15.37:0/1917845110 >> [v2:172.21.15.203:6832/2418205237,v1:172.21.15.203:6833/2418205237] conn(0x7fece40744e0 0x7fece40769f0 secure :-1 s=READY pgs=280 cs=0 l=1 rev1=1 crypto rx=0x7fecec033000 tx=0x7fecec008030 comp rx=0 tx=0).stop 2024-04-25T10:30:47.903 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:47.901+0000 7fecfdcca640 1 -- 172.21.15.37:0/1917845110 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fece0003f30 msgr2=0x7fecf8160fc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:30:47.903 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:47.901+0000 7fecfdcca640 1 --2- 172.21.15.37:0/1917845110 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fece0003f30 0x7fecf8160fc0 secure :-1 s=READY pgs=613 cs=0 l=1 rev1=1 crypto rx=0x7fece8011990 tx=0x7fece8011e50 comp rx=0 tx=0).stop 2024-04-25T10:30:47.903 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:47.901+0000 7fecfdcca640 1 -- 172.21.15.37:0/1917845110 shutdown_connections 2024-04-25T10:30:47.903 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:47.901+0000 7fecfdcca640 1 --2- 172.21.15.37:0/1917845110 >> [v2:172.21.15.203:6832/2418205237,v1:172.21.15.203:6833/2418205237] conn(0x7fece40744e0 0x7fece40769f0 unknown :-1 s=CLOSED pgs=280 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:30:47.903 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:47.901+0000 7fecfdcca640 1 --2- 172.21.15.37:0/1917845110 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fecf8155180 0x7fecf81f7630 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:30:47.903 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:47.901+0000 7fecfdcca640 1 --2- 172.21.15.37:0/1917845110 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7fecf8154d70 0x7fecf81f70f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:30:47.903 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:47.901+0000 7fecfdcca640 1 --2- 172.21.15.37:0/1917845110 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fece0003f30 0x7fecf8160fc0 unknown :-1 s=CLOSED pgs=613 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:30:47.903 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:47.901+0000 7fecfdcca640 1 -- 172.21.15.37:0/1917845110 >> 172.21.15.37:0/1917845110 conn(0x7fecf80b5440 msgr2=0x7fecf80cff60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:30:47.903 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:47.901+0000 7fecfdcca640 1 -- 172.21.15.37:0/1917845110 shutdown_connections 2024-04-25T10:30:47.903 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:47.901+0000 7fecfdcca640 1 -- 172.21.15.37:0/1917845110 wait complete. 2024-04-25T10:30:47.913 DEBUG:teuthology.orchestra.run.smithi037:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_metadata pg_num 2024-04-25T10:30:48.028 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:48.027+0000 7f9f8966e640 1 Processor -- start 2024-04-25T10:30:48.028 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:48.027+0000 7f9f8966e640 1 -- start start 2024-04-25T10:30:48.030 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:48.028+0000 7f9f8966e640 1 --2- >> v2:172.21.15.37:3300/0 conn(0x7f9f8408cc90 0x7f9f8409cbc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:30:48.030 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:48.028+0000 7f9f8966e640 1 --2- >> v2:172.21.15.203:3300/0 conn(0x7f9f840b71b0 0x7f9f840b75c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:30:48.030 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:48.028+0000 7f9f8966e640 1 -- --> v1:172.21.15.203:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f9f840b8d40 con 0x7f9f840d1c20 2024-04-25T10:30:48.030 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:48.028+0000 7f9f8966e640 1 -- --> v2:172.21.15.37:3300/0 -- mon_getmap magic: 0 -- 0x7f9f840d27f0 con 0x7f9f8408cc90 2024-04-25T10:30:48.030 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:48.028+0000 7f9f8966e640 1 -- --> v2:172.21.15.203:3300/0 -- mon_getmap magic: 0 -- 0x7f9f840d2ad0 con 0x7f9f840b71b0 2024-04-25T10:30:48.030 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:48.028+0000 7f9f82ffd640 1 --2- >> v2:172.21.15.37:3300/0 conn(0x7f9f8408cc90 0x7f9f8409cbc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:30:48.030 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:48.029+0000 7f9f82ffd640 1 --2- >> v2:172.21.15.37:3300/0 conn(0x7f9f8408cc90 0x7f9f8409cbc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.37:3300/0 says I am v2:172.21.15.37:58412/0 (socket says 172.21.15.37:58412) 2024-04-25T10:30:48.031 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:48.029+0000 7f9f82ffd640 1 -- 172.21.15.37:0/1650324413 learned_addr learned my addr 172.21.15.37:0/1650324413 (peer_addr_for_me v2:172.21.15.37:0/0) 2024-04-25T10:30:48.031 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:48.029+0000 7f9f837fe640 1 --2- 172.21.15.37:0/1650324413 >> v2:172.21.15.203:3300/0 conn(0x7f9f840b71b0 0x7f9f840b75c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:30:48.031 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:48.029+0000 7f9f82ffd640 1 -- 172.21.15.37:0/1650324413 >> v2:172.21.15.203:3300/0 conn(0x7f9f840b71b0 msgr2=0x7f9f840b75c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:30:48.031 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:48.029+0000 7f9f82ffd640 1 --2- 172.21.15.37:0/1650324413 >> v2:172.21.15.203:3300/0 conn(0x7f9f840b71b0 0x7f9f840b75c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:30:48.031 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:48.029+0000 7f9f82ffd640 1 -- 172.21.15.37:0/1650324413 >> v1:172.21.15.203:6789/0 conn(0x7f9f840d1c20 legacy=0x7f9f840d2030 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:30:48.031 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:48.029+0000 7f9f82ffd640 1 -- 172.21.15.37:0/1650324413 --> v2:172.21.15.37:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f9f840cf7a0 con 0x7f9f8408cc90 2024-04-25T10:30:48.032 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:48.030+0000 7f9f82ffd640 1 --2- 172.21.15.37:0/1650324413 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f9f8408cc90 0x7f9f8409cbc0 secure :-1 s=READY pgs=614 cs=0 l=1 rev1=1 crypto rx=0x7f9f840d38f0 tx=0x7f9f7402a230 comp rx=0 tx=0).ready entity=mon.0 client_cookie=1fc73edca4c3a522 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:30:48.032 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:48.030+0000 7f9f827fc640 1 -- 172.21.15.37:0/1650324413 <== mon.0 v2:172.21.15.37:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9f74037070 con 0x7f9f8408cc90 2024-04-25T10:30:48.032 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:48.030+0000 7f9f827fc640 1 -- 172.21.15.37:0/1650324413 <== mon.0 v2:172.21.15.37:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f9f74032440 con 0x7f9f8408cc90 2024-04-25T10:30:48.033 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:48.030+0000 7f9f827fc640 1 -- 172.21.15.37:0/1650324413 <== mon.0 v2:172.21.15.37:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9f74032810 con 0x7f9f8408cc90 2024-04-25T10:30:48.033 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:48.030+0000 7f9f8966e640 1 -- 172.21.15.37:0/1650324413 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f9f8408cc90 msgr2=0x7f9f8409cbc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:30:48.033 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:48.030+0000 7f9f8966e640 1 --2- 172.21.15.37:0/1650324413 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f9f8408cc90 0x7f9f8409cbc0 secure :-1 s=READY pgs=614 cs=0 l=1 rev1=1 crypto rx=0x7f9f840d38f0 tx=0x7f9f7402a230 comp rx=0 tx=0).stop 2024-04-25T10:30:48.033 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:48.031+0000 7f9f8966e640 1 -- 172.21.15.37:0/1650324413 shutdown_connections 2024-04-25T10:30:48.033 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:48.031+0000 7f9f8966e640 1 --2- 172.21.15.37:0/1650324413 >> v2:172.21.15.203:3300/0 conn(0x7f9f840b71b0 0x7f9f840b75c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:30:48.033 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:48.031+0000 7f9f8966e640 1 --2- 172.21.15.37:0/1650324413 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f9f8408cc90 0x7f9f8409cbc0 unknown :-1 s=CLOSED pgs=614 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:30:48.033 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:48.031+0000 7f9f8966e640 1 -- 172.21.15.37:0/1650324413 >> 172.21.15.37:0/1650324413 conn(0x7f9f840d03d0 msgr2=0x7f9f840d07e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:30:48.033 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:48.031+0000 7f9f8966e640 1 -- 172.21.15.37:0/1650324413 shutdown_connections 2024-04-25T10:30:48.033 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:48.031+0000 7f9f8966e640 1 -- 172.21.15.37:0/1650324413 wait complete. 2024-04-25T10:30:48.034 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:48.032+0000 7f9f8966e640 1 Processor -- start 2024-04-25T10:30:48.034 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:48.033+0000 7f9f8966e640 1 -- start start 2024-04-25T10:30:48.036 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:48.034+0000 7f9f8966e640 1 --2- >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f9f840b71b0 0x7f9f841e26a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:30:48.036 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:48.034+0000 7f9f8966e640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f9f840d1c20 0x7f9f840b2e10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:30:48.036 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:48.034+0000 7f9f8966e640 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f9f841e2ee0 0x7f9f840b3350 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:30:48.036 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:48.034+0000 7f9f8966e640 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 -- 0x7f9f840b7b00 con 0x7f9f840d1c20 2024-04-25T10:30:48.036 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:48.034+0000 7f9f8966e640 1 -- --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_getmap magic: 0 -- 0x7f9f840b93b0 con 0x7f9f841e2ee0 2024-04-25T10:30:48.036 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:48.034+0000 7f9f8966e640 1 -- --> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] -- mon_getmap magic: 0 -- 0x7f9f840b7c80 con 0x7f9f840b71b0 2024-04-25T10:30:48.037 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:48.034+0000 7f9f7bfff640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f9f840d1c20 0x7f9f840b2e10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:30:48.037 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:48.034+0000 7f9f7bfff640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f9f840d1c20 0x7f9f840b2e10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.37:3300/0 says I am v2:172.21.15.37:58426/0 (socket says 172.21.15.37:58426) 2024-04-25T10:30:48.037 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:48.034+0000 7f9f7bfff640 1 -- 172.21.15.37:0/2695394479 learned_addr learned my addr 172.21.15.37:0/2695394479 (peer_addr_for_me v2:172.21.15.37:0/0) 2024-04-25T10:30:48.037 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:48.034+0000 7f9f82ffd640 1 --2- 172.21.15.37:0/2695394479 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f9f840b71b0 0x7f9f841e26a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:30:48.037 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:48.034+0000 7f9f837fe640 1 --2- 172.21.15.37:0/2695394479 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f9f841e2ee0 0x7f9f840b3350 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:30:48.037 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:48.034+0000 7f9f7bfff640 1 -- 172.21.15.37:0/2695394479 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f9f840b71b0 msgr2=0x7f9f841e26a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:30:48.037 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:48.034+0000 7f9f7bfff640 1 --2- 172.21.15.37:0/2695394479 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f9f840b71b0 0x7f9f841e26a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:30:48.037 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:48.034+0000 7f9f7bfff640 1 -- 172.21.15.37:0/2695394479 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f9f841e2ee0 msgr2=0x7f9f840b3350 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:30:48.037 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:48.034+0000 7f9f7bfff640 1 --2- 172.21.15.37:0/2695394479 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f9f841e2ee0 0x7f9f840b3350 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:30:48.037 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:48.034+0000 7f9f7bfff640 1 -- 172.21.15.37:0/2695394479 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f9f841e36f0 con 0x7f9f840d1c20 2024-04-25T10:30:48.037 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:48.035+0000 7f9f7bfff640 1 --2- 172.21.15.37:0/2695394479 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f9f840d1c20 0x7f9f840b2e10 secure :-1 s=READY pgs=615 cs=0 l=1 rev1=1 crypto rx=0x7f9f6c00b3e0 tx=0x7f9f6c00b8a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:30:48.037 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:48.035+0000 7f9f7b7fe640 1 -- 172.21.15.37:0/2695394479 <== mon.0 v2:172.21.15.37:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9f6c015020 con 0x7f9f840d1c20 2024-04-25T10:30:48.038 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:48.035+0000 7f9f7b7fe640 1 -- 172.21.15.37:0/2695394479 <== mon.0 v2:172.21.15.37:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f9f6c011830 con 0x7f9f840d1c20 2024-04-25T10:30:48.038 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:48.035+0000 7f9f8966e640 1 -- 172.21.15.37:0/2695394479 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f9f840c9540 con 0x7f9f840d1c20 2024-04-25T10:30:48.038 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:48.035+0000 7f9f7b7fe640 1 -- 172.21.15.37:0/2695394479 <== mon.0 v2:172.21.15.37:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9f6c011c20 con 0x7f9f840d1c20 2024-04-25T10:30:48.038 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:48.035+0000 7f9f8966e640 1 -- 172.21.15.37:0/2695394479 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f9f840c9b00 con 0x7f9f840d1c20 2024-04-25T10:30:48.039 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:48.036+0000 7f9f8966e640 1 -- 172.21.15.37:0/2695394479 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f9f840c92b0 con 0x7f9f840d1c20 2024-04-25T10:30:48.039 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:48.036+0000 7f9f7b7fe640 1 -- 172.21.15.37:0/2695394479 <== mon.0 v2:172.21.15.37:3300/0 4 ==== mgrmap(e 5) ==== 96782+0+0 (secure 0 0 0) 0x7f9f6c018020 con 0x7f9f840d1c20 2024-04-25T10:30:48.039 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:48.037+0000 7f9f7b7fe640 1 --2- 172.21.15.37:0/2695394479 >> [v2:172.21.15.203:6832/2418205237,v1:172.21.15.203:6833/2418205237] conn(0x7f9f640744b0 0x7f9f640769c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:30:48.039 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:48.037+0000 7f9f7b7fe640 1 -- 172.21.15.37:0/2695394479 <== mon.0 v2:172.21.15.37:3300/0 5 ==== osd_map(44..44 src has 1..44) ==== 6248+0+0 (secure 0 0 0) 0x7f9f6c095080 con 0x7f9f840d1c20 2024-04-25T10:30:48.039 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:48.037+0000 7f9f82ffd640 1 --2- 172.21.15.37:0/2695394479 >> [v2:172.21.15.203:6832/2418205237,v1:172.21.15.203:6833/2418205237] conn(0x7f9f640744b0 0x7f9f640769c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:30:48.039 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:48.038+0000 7f9f82ffd640 1 --2- 172.21.15.37:0/2695394479 >> [v2:172.21.15.203:6832/2418205237,v1:172.21.15.203:6833/2418205237] conn(0x7f9f640744b0 0x7f9f640769c0 secure :-1 s=READY pgs=281 cs=0 l=1 rev1=1 crypto rx=0x7f9f74034000 tx=0x7f9f7402e000 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:30:48.042 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:48.042+0000 7f9f7b7fe640 1 -- 172.21.15.37:0/2695394479 <== mon.0 v2:172.21.15.37:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187246 (secure 0 0 0) 0x7f9f6c01f020 con 0x7f9f840d1c20 2024-04-25T10:30:48.228 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:48.227+0000 7f9f8966e640 1 -- 172.21.15.37:0/2695394479 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"} v 0) -- 0x7f9f8406a140 con 0x7f9f840d1c20 2024-04-25T10:30:48.228 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:48.227+0000 7f9f7b7fe640 1 -- 172.21.15.37:0/2695394479 <== mon.0 v2:172.21.15.37:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]=0 v44) ==== 104+0+11 (secure 0 0 0) 0x7f9f6c05f0c0 con 0x7f9f840d1c20 2024-04-25T10:30:48.229 INFO:teuthology.orchestra.run.smithi037.stdout:pg_num: 32 2024-04-25T10:30:48.231 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:48.229+0000 7f9f8966e640 1 -- 172.21.15.37:0/2695394479 >> [v2:172.21.15.203:6832/2418205237,v1:172.21.15.203:6833/2418205237] conn(0x7f9f640744b0 msgr2=0x7f9f640769c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:30:48.231 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:48.229+0000 7f9f8966e640 1 --2- 172.21.15.37:0/2695394479 >> [v2:172.21.15.203:6832/2418205237,v1:172.21.15.203:6833/2418205237] conn(0x7f9f640744b0 0x7f9f640769c0 secure :-1 s=READY pgs=281 cs=0 l=1 rev1=1 crypto rx=0x7f9f74034000 tx=0x7f9f7402e000 comp rx=0 tx=0).stop 2024-04-25T10:30:48.231 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:48.229+0000 7f9f8966e640 1 -- 172.21.15.37:0/2695394479 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f9f840d1c20 msgr2=0x7f9f840b2e10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:30:48.231 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:48.229+0000 7f9f8966e640 1 --2- 172.21.15.37:0/2695394479 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f9f840d1c20 0x7f9f840b2e10 secure :-1 s=READY pgs=615 cs=0 l=1 rev1=1 crypto rx=0x7f9f6c00b3e0 tx=0x7f9f6c00b8a0 comp rx=0 tx=0).stop 2024-04-25T10:30:48.231 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:48.229+0000 7f9f8966e640 1 -- 172.21.15.37:0/2695394479 shutdown_connections 2024-04-25T10:30:48.231 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:48.229+0000 7f9f8966e640 1 --2- 172.21.15.37:0/2695394479 >> [v2:172.21.15.203:6832/2418205237,v1:172.21.15.203:6833/2418205237] conn(0x7f9f640744b0 0x7f9f640769c0 unknown :-1 s=CLOSED pgs=281 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:30:48.231 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:48.229+0000 7f9f8966e640 1 --2- 172.21.15.37:0/2695394479 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f9f841e2ee0 0x7f9f840b3350 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:30:48.231 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:48.229+0000 7f9f8966e640 1 --2- 172.21.15.37:0/2695394479 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f9f840d1c20 0x7f9f840b2e10 unknown :-1 s=CLOSED pgs=615 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:30:48.231 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:48.229+0000 7f9f8966e640 1 --2- 172.21.15.37:0/2695394479 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f9f840b71b0 0x7f9f841e26a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:30:48.231 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:48.229+0000 7f9f8966e640 1 -- 172.21.15.37:0/2695394479 >> 172.21.15.37:0/2695394479 conn(0x7f9f840d03d0 msgr2=0x7f9f840d07e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:30:48.232 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:48.229+0000 7f9f8966e640 1 -- 172.21.15.37:0/2695394479 shutdown_connections 2024-04-25T10:30:48.232 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:48.229+0000 7f9f8966e640 1 -- 172.21.15.37:0/2695394479 wait complete. 2024-04-25T10:30:48.242 DEBUG:teuthology.orchestra.run.smithi037:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data pg_num 2024-04-25T10:30:48.356 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:48.355+0000 7f4cffb38640 1 Processor -- start 2024-04-25T10:30:48.356 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:48.355+0000 7f4cffb38640 1 -- start start 2024-04-25T10:30:48.369 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:48.356+0000 7f4cffb38640 1 --2- >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f4cf81460e0 0x7f4cf81610b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:30:48.369 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:48.356+0000 7f4cffb38640 1 --2- >> v2:172.21.15.203:3300/0 conn(0x7f4cf806f100 0x7f4cf80b6640 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:30:48.369 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:48.356+0000 7f4cffb38640 1 -- --> v1:172.21.15.37:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f4cf81616a0 con 0x7f4cf8143500 2024-04-25T10:30:48.369 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:48.356+0000 7f4cffb38640 1 -- --> v2:172.21.15.203:3300/0 -- mon_getmap magic: 0 -- 0x7f4cf806f510 con 0x7f4cf806f100 2024-04-25T10:30:48.369 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:48.356+0000 7f4cffb38640 1 -- --> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] -- mon_getmap magic: 0 -- 0x7f4cf80b5b60 con 0x7f4cf81460e0 2024-04-25T10:30:48.369 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:48.357+0000 7f4cfd8d4640 1 --1- >> v1:172.21.15.37:6789/0 conn(0x7f4cf8143500 0x7f4cf81609a0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.37:6789/0 says I am v1:172.21.15.37:55696/0 (socket says 172.21.15.37:55696) 2024-04-25T10:30:48.370 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:48.357+0000 7f4cfd8d4640 1 -- 172.21.15.37:0/421772689 learned_addr learned my addr 172.21.15.37:0/421772689 (peer_addr_for_me v1:172.21.15.37:0/0) 2024-04-25T10:30:48.370 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:48.357+0000 7f4cfc8d2640 1 -- 172.21.15.37:0/421772689 <== mon.0 v1:172.21.15.37:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 301197599 0 0) 0x7f4cf81616a0 con 0x7f4cf8143500 2024-04-25T10:30:48.370 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:48.357+0000 7f4cfe0d5640 1 --2- 172.21.15.37:0/421772689 >> v2:172.21.15.203:3300/0 conn(0x7f4cf806f100 0x7f4cf80b6640 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:30:48.370 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:48.357+0000 7f4cfd0d3640 1 --2- 172.21.15.37:0/421772689 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f4cf81460e0 0x7f4cf81610b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:30:48.370 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:48.357+0000 7f4cfc8d2640 1 -- 172.21.15.37:0/421772689 --> v1:172.21.15.37:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f4ce0005730 con 0x7f4cf8143500 2024-04-25T10:30:48.370 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:48.357+0000 7f4cfc8d2640 1 -- 172.21.15.37:0/421772689 <== mon.0 v1:172.21.15.37:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 1649110050 0 0) 0x7f4ce0005730 con 0x7f4cf8143500 2024-04-25T10:30:48.370 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:48.357+0000 7f4cfc8d2640 1 -- 172.21.15.37:0/421772689 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f4cf81460e0 msgr2=0x7f4cf81610b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:30:48.370 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:48.357+0000 7f4cfc8d2640 1 --2- 172.21.15.37:0/421772689 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f4cf81460e0 0x7f4cf81610b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:30:48.370 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:48.357+0000 7f4cfc8d2640 1 -- 172.21.15.37:0/421772689 >> v2:172.21.15.203:3300/0 conn(0x7f4cf806f100 msgr2=0x7f4cf80b6640 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:30:48.370 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:48.357+0000 7f4cfc8d2640 1 --2- 172.21.15.37:0/421772689 >> v2:172.21.15.203:3300/0 conn(0x7f4cf806f100 0x7f4cf80b6640 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:30:48.370 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:48.357+0000 7f4cfc8d2640 1 -- 172.21.15.37:0/421772689 --> v1:172.21.15.37:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f4cf80b5e40 con 0x7f4cf8143500 2024-04-25T10:30:48.370 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:48.357+0000 7f4cfc8d2640 1 -- 172.21.15.37:0/421772689 <== mon.0 v1:172.21.15.37:6789/0 3 ==== mon_map magic: 0 ==== 413+0+0 (unknown 1577794600 0 0) 0x7f4cf0001670 con 0x7f4cf8143500 2024-04-25T10:30:48.370 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:48.357+0000 7f4cfc8d2640 1 -- 172.21.15.37:0/421772689 >> v1:172.21.15.37:6789/0 conn(0x7f4cf8143500 legacy=0x7f4cf81609a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:30:48.370 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:48.357+0000 7f4cfc8d2640 1 --2- 172.21.15.37:0/421772689 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f4ce0005bc0 0x7f4ce0006030 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:30:48.371 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:48.357+0000 7f4cfc8d2640 1 --2- 172.21.15.37:0/421772689 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f4cf81460e0 0x7f4ce0006a10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:30:48.371 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:48.357+0000 7f4cfc8d2640 1 --2- 172.21.15.37:0/421772689 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f4ce0006f50 0x7f4ce0007360 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:30:48.371 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:48.357+0000 7f4cfc8d2640 1 -- 172.21.15.37:0/421772689 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 -- 0x7f4cf806f510 con 0x7f4ce0005bc0 2024-04-25T10:30:48.371 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:48.357+0000 7f4cfc8d2640 1 -- 172.21.15.37:0/421772689 --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_getmap magic: 0 -- 0x7f4cf80b5b60 con 0x7f4ce0006f50 2024-04-25T10:30:48.371 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:48.357+0000 7f4cfc8d2640 1 -- 172.21.15.37:0/421772689 --> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] -- mon_getmap magic: 0 -- 0x7f4ce000dbd0 con 0x7f4cf81460e0 2024-04-25T10:30:48.371 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:48.357+0000 7f4cfd8d4640 1 --2- 172.21.15.37:0/421772689 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f4ce0005bc0 0x7f4ce0006030 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:30:48.371 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:48.358+0000 7f4cfd0d3640 1 --2- 172.21.15.37:0/421772689 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f4cf81460e0 0x7f4ce0006a10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:30:48.371 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:48.358+0000 7f4cfe0d5640 1 --2- 172.21.15.37:0/421772689 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f4ce0006f50 0x7f4ce0007360 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:30:48.371 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:48.358+0000 7f4cfd8d4640 1 -- 172.21.15.37:0/421772689 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f4cf81460e0 msgr2=0x7f4ce0006a10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:30:48.371 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:48.358+0000 7f4cfd8d4640 1 --2- 172.21.15.37:0/421772689 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f4cf81460e0 0x7f4ce0006a10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:30:48.371 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:48.358+0000 7f4cfd8d4640 1 -- 172.21.15.37:0/421772689 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f4ce0006f50 msgr2=0x7f4ce0007360 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:30:48.371 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:48.358+0000 7f4cfd8d4640 1 --2- 172.21.15.37:0/421772689 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f4ce0006f50 0x7f4ce0007360 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:30:48.371 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:48.358+0000 7f4cfd8d4640 1 -- 172.21.15.37:0/421772689 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f4ce0005730 con 0x7f4ce0005bc0 2024-04-25T10:30:48.372 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:48.358+0000 7f4cfd8d4640 1 --2- 172.21.15.37:0/421772689 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f4ce0005bc0 0x7f4ce0006030 secure :-1 s=READY pgs=617 cs=0 l=1 rev1=1 crypto rx=0x7f4cf000c210 tx=0x7f4ce0007a40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=81fdf6da88be823 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:30:48.372 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:48.359+0000 7f4cfc8d2640 1 -- 172.21.15.37:0/421772689 <== mon.0 v2:172.21.15.37:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4cf0007d50 con 0x7f4ce0005bc0 2024-04-25T10:30:48.372 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:48.359+0000 7f4cfc8d2640 1 -- 172.21.15.37:0/421772689 <== mon.0 v2:172.21.15.37:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f4cf0005ad0 con 0x7f4ce0005bc0 2024-04-25T10:30:48.372 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:48.359+0000 7f4cfc8d2640 1 -- 172.21.15.37:0/421772689 <== mon.0 v2:172.21.15.37:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4cf0007070 con 0x7f4ce0005bc0 2024-04-25T10:30:48.372 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:48.359+0000 7f4cffb38640 1 -- 172.21.15.37:0/421772689 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f4ce0005bc0 msgr2=0x7f4ce0006030 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:30:48.372 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:48.359+0000 7f4cffb38640 1 --2- 172.21.15.37:0/421772689 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f4ce0005bc0 0x7f4ce0006030 secure :-1 s=READY pgs=617 cs=0 l=1 rev1=1 crypto rx=0x7f4cf000c210 tx=0x7f4ce0007a40 comp rx=0 tx=0).stop 2024-04-25T10:30:48.372 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:48.359+0000 7f4cfe0d5640 1 -- 172.21.15.37:0/421772689 reap_dead start 2024-04-25T10:30:48.372 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:48.359+0000 7f4cffb38640 1 -- 172.21.15.37:0/421772689 shutdown_connections 2024-04-25T10:30:48.372 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:48.359+0000 7f4cffb38640 1 -- 172.21.15.37:0/421772689 >> 172.21.15.37:0/421772689 conn(0x7f4cf80b36b0 msgr2=0x7f4cf80b3ac0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:30:48.372 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:48.359+0000 7f4cffb38640 1 -- 172.21.15.37:0/421772689 shutdown_connections 2024-04-25T10:30:48.372 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:48.359+0000 7f4cffb38640 1 -- 172.21.15.37:0/421772689 wait complete. 2024-04-25T10:30:48.372 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:48.361+0000 7f4cffb38640 1 Processor -- start 2024-04-25T10:30:48.372 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:48.361+0000 7f4cffb38640 1 -- start start 2024-04-25T10:30:48.373 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:48.362+0000 7f4cffb38640 1 --2- >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f4ce0005bc0 0x7f4cf813f3c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:30:48.373 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:48.363+0000 7f4cffb38640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f4cf8160610 0x7f4cf8140b80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:30:48.373 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:48.363+0000 7f4cffb38640 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f4cf8160a20 0x7f4cf81410c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:30:48.373 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:48.363+0000 7f4cffb38640 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 -- 0x7f4cf8161230 con 0x7f4cf8160610 2024-04-25T10:30:48.373 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:48.363+0000 7f4cffb38640 1 -- --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_getmap magic: 0 -- 0x7f4cf81610b0 con 0x7f4cf8160a20 2024-04-25T10:30:48.373 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:48.363+0000 7f4cffb38640 1 -- --> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] -- mon_getmap magic: 0 -- 0x7f4cf81613b0 con 0x7f4ce0005bc0 2024-04-25T10:30:48.373 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:48.363+0000 7f4cfd0d3640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f4cf8160610 0x7f4cf8140b80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:30:48.373 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:48.363+0000 7f4cfd0d3640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f4cf8160610 0x7f4cf8140b80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.37:3300/0 says I am v2:172.21.15.37:58436/0 (socket says 172.21.15.37:58436) 2024-04-25T10:30:48.373 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:48.363+0000 7f4cfd0d3640 1 -- 172.21.15.37:0/2214653972 learned_addr learned my addr 172.21.15.37:0/2214653972 (peer_addr_for_me v2:172.21.15.37:0/0) 2024-04-25T10:30:48.373 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:48.363+0000 7f4cfe0d5640 1 --2- 172.21.15.37:0/2214653972 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f4cf8160a20 0x7f4cf81410c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:30:48.374 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:48.363+0000 7f4cfd8d4640 1 --2- 172.21.15.37:0/2214653972 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f4ce0005bc0 0x7f4cf813f3c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:30:48.374 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:48.363+0000 7f4cfd0d3640 1 -- 172.21.15.37:0/2214653972 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f4ce0005bc0 msgr2=0x7f4cf813f3c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:30:48.374 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:48.363+0000 7f4cfd0d3640 1 --2- 172.21.15.37:0/2214653972 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f4ce0005bc0 0x7f4cf813f3c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:30:48.374 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:48.363+0000 7f4cfd0d3640 1 -- 172.21.15.37:0/2214653972 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f4cf8160a20 msgr2=0x7f4cf81410c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:30:48.374 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:48.363+0000 7f4cfd0d3640 1 --2- 172.21.15.37:0/2214653972 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f4cf8160a20 0x7f4cf81410c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:30:48.374 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:48.363+0000 7f4cfd0d3640 1 -- 172.21.15.37:0/2214653972 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f4cf81f5e60 con 0x7f4cf8160610 2024-04-25T10:30:48.374 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:48.363+0000 7f4cfd8d4640 1 --2- 172.21.15.37:0/2214653972 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f4ce0005bc0 0x7f4cf813f3c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T10:30:48.375 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:48.363+0000 7f4cfd0d3640 1 --2- 172.21.15.37:0/2214653972 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f4cf8160610 0x7f4cf8140b80 secure :-1 s=READY pgs=618 cs=0 l=1 rev1=1 crypto rx=0x7f4cec0119a0 tx=0x7f4cec011e60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:30:48.375 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:48.363+0000 7f4cf67fc640 1 -- 172.21.15.37:0/2214653972 <== mon.0 v2:172.21.15.37:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4cec00faf0 con 0x7f4cf8160610 2024-04-25T10:30:48.375 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:48.363+0000 7f4cf67fc640 1 -- 172.21.15.37:0/2214653972 <== mon.0 v2:172.21.15.37:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f4cec00fc90 con 0x7f4cf8160610 2024-04-25T10:30:48.375 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:48.363+0000 7f4cffb38640 1 -- 172.21.15.37:0/2214653972 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f4cf81f6140 con 0x7f4cf8160610 2024-04-25T10:30:48.375 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:48.363+0000 7f4cf67fc640 1 -- 172.21.15.37:0/2214653972 <== mon.0 v2:172.21.15.37:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4cec014440 con 0x7f4cf8160610 2024-04-25T10:30:48.375 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:48.364+0000 7f4cffb38640 1 -- 172.21.15.37:0/2214653972 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f4cf81f6700 con 0x7f4cf8160610 2024-04-25T10:30:48.375 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:48.365+0000 7f4cffb38640 1 -- 172.21.15.37:0/2214653972 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f4cf81f5bb0 con 0x7f4cf8160610 2024-04-25T10:30:48.375 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:48.365+0000 7f4cf67fc640 1 -- 172.21.15.37:0/2214653972 <== mon.0 v2:172.21.15.37:3300/0 4 ==== mgrmap(e 5) ==== 96782+0+0 (secure 0 0 0) 0x7f4cec012040 con 0x7f4cf8160610 2024-04-25T10:30:48.375 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:48.365+0000 7f4cf67fc640 1 --2- 172.21.15.37:0/2214653972 >> [v2:172.21.15.203:6832/2418205237,v1:172.21.15.203:6833/2418205237] conn(0x7f4ce40744e0 0x7f4ce40769f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:30:48.375 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:48.366+0000 7f4cf67fc640 1 -- 172.21.15.37:0/2214653972 <== mon.0 v2:172.21.15.37:3300/0 5 ==== osd_map(44..44 src has 1..44) ==== 6248+0+0 (secure 0 0 0) 0x7f4cec095a10 con 0x7f4cf8160610 2024-04-25T10:30:48.375 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:48.366+0000 7f4cfd8d4640 1 --2- 172.21.15.37:0/2214653972 >> [v2:172.21.15.203:6832/2418205237,v1:172.21.15.203:6833/2418205237] conn(0x7f4ce40744e0 0x7f4ce40769f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:30:48.376 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:48.366+0000 7f4cfd8d4640 1 --2- 172.21.15.37:0/2214653972 >> [v2:172.21.15.203:6832/2418205237,v1:172.21.15.203:6833/2418205237] conn(0x7f4ce40744e0 0x7f4ce40769f0 secure :-1 s=READY pgs=282 cs=0 l=1 rev1=1 crypto rx=0x7f4cf0033000 tx=0x7f4cf0035040 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:30:48.376 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:48.370+0000 7f4cf67fc640 1 -- 172.21.15.37:0/2214653972 <== mon.0 v2:172.21.15.37:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187246 (secure 0 0 0) 0x7f4cec019030 con 0x7f4cf8160610 2024-04-25T10:30:48.555 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:48.554+0000 7f4cffb38640 1 -- 172.21.15.37:0/2214653972 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"} v 0) -- 0x7f4cf806a140 con 0x7f4cf8160610 2024-04-25T10:30:48.555 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:48.554+0000 7f4cf67fc640 1 -- 172.21.15.37:0/2214653972 <== mon.0 v2:172.21.15.37:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]=0 v44) ==== 100+0+11 (secure 0 0 0) 0x7f4cec060010 con 0x7f4cf8160610 2024-04-25T10:30:48.555 INFO:teuthology.orchestra.run.smithi037.stdout:pg_num: 64 2024-04-25T10:30:48.558 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:48.556+0000 7f4cffb38640 1 -- 172.21.15.37:0/2214653972 >> [v2:172.21.15.203:6832/2418205237,v1:172.21.15.203:6833/2418205237] conn(0x7f4ce40744e0 msgr2=0x7f4ce40769f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:30:48.558 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:48.556+0000 7f4cffb38640 1 --2- 172.21.15.37:0/2214653972 >> [v2:172.21.15.203:6832/2418205237,v1:172.21.15.203:6833/2418205237] conn(0x7f4ce40744e0 0x7f4ce40769f0 secure :-1 s=READY pgs=282 cs=0 l=1 rev1=1 crypto rx=0x7f4cf0033000 tx=0x7f4cf0035040 comp rx=0 tx=0).stop 2024-04-25T10:30:48.558 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:48.556+0000 7f4cffb38640 1 -- 172.21.15.37:0/2214653972 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f4cf8160610 msgr2=0x7f4cf8140b80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:30:48.558 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:48.556+0000 7f4cffb38640 1 --2- 172.21.15.37:0/2214653972 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f4cf8160610 0x7f4cf8140b80 secure :-1 s=READY pgs=618 cs=0 l=1 rev1=1 crypto rx=0x7f4cec0119a0 tx=0x7f4cec011e60 comp rx=0 tx=0).stop 2024-04-25T10:30:48.558 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:48.556+0000 7f4cffb38640 1 -- 172.21.15.37:0/2214653972 shutdown_connections 2024-04-25T10:30:48.558 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:48.556+0000 7f4cffb38640 1 --2- 172.21.15.37:0/2214653972 >> [v2:172.21.15.203:6832/2418205237,v1:172.21.15.203:6833/2418205237] conn(0x7f4ce40744e0 0x7f4ce40769f0 unknown :-1 s=CLOSED pgs=282 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:30:48.558 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:48.556+0000 7f4cffb38640 1 --2- 172.21.15.37:0/2214653972 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f4cf8160a20 0x7f4cf81410c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:30:48.558 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:48.556+0000 7f4cffb38640 1 --2- 172.21.15.37:0/2214653972 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f4cf8160610 0x7f4cf8140b80 unknown :-1 s=CLOSED pgs=618 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:30:48.559 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:48.556+0000 7f4cffb38640 1 --2- 172.21.15.37:0/2214653972 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f4ce0005bc0 0x7f4cf813f3c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:30:48.559 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:48.556+0000 7f4cffb38640 1 -- 172.21.15.37:0/2214653972 >> 172.21.15.37:0/2214653972 conn(0x7f4cf80b36b0 msgr2=0x7f4cf806f990 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:30:48.559 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:48.556+0000 7f4cffb38640 1 -- 172.21.15.37:0/2214653972 shutdown_connections 2024-04-25T10:30:48.559 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:48.556+0000 7f4cffb38640 1 -- 172.21.15.37:0/2214653972 wait complete. 2024-04-25T10:30:48.568 DEBUG:teuthology.orchestra.run.smithi037:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get recovery_fs_metadata pg_num 2024-04-25T10:30:48.682 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:48.681+0000 7f47eb9ab640 1 Processor -- start 2024-04-25T10:30:48.683 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:48.681+0000 7f47eb9ab640 1 -- start start 2024-04-25T10:30:48.684 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:48.683+0000 7f47eb9ab640 1 --2- >> v2:172.21.15.37:3300/0 conn(0x7f47e4161020 0x7f47e416b190 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:30:48.684 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:48.683+0000 7f47eb9ab640 1 -- --> v1:172.21.15.37:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f47e4161430 con 0x7f47e415a9f0 2024-04-25T10:30:48.684 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:48.683+0000 7f47eb9ab640 1 -- --> v1:172.21.15.203:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f47e41616d0 con 0x7f47e406cc50 2024-04-25T10:30:48.684 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:48.683+0000 7f47eb9ab640 1 -- --> v2:172.21.15.37:3300/0 -- mon_getmap magic: 0 -- 0x7f47e40b6870 con 0x7f47e4161020 2024-04-25T10:30:48.684 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:48.683+0000 7f47e9f48640 1 --2- >> v2:172.21.15.37:3300/0 conn(0x7f47e4161020 0x7f47e416b190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:30:48.684 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:48.683+0000 7f47e9747640 1 --1- >> v1:172.21.15.37:6789/0 conn(0x7f47e415a9f0 0x7f47e406c540 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.37:6789/0 says I am v1:172.21.15.37:55704/0 (socket says 172.21.15.37:55704) 2024-04-25T10:30:48.684 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:48.683+0000 7f47e9747640 1 -- 172.21.15.37:0/2167825908 learned_addr learned my addr 172.21.15.37:0/2167825908 (peer_addr_for_me v1:172.21.15.37:0/0) 2024-04-25T10:30:48.685 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:48.683+0000 7f47e9f48640 1 -- 172.21.15.37:0/2167825908 >> v1:172.21.15.203:6789/0 conn(0x7f47e406cc50 legacy=0x7f47e4160910 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:30:48.685 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:48.683+0000 7f47e9f48640 1 -- 172.21.15.37:0/2167825908 >> v1:172.21.15.37:6789/0 conn(0x7f47e415a9f0 legacy=0x7f47e406c540 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:30:48.685 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:48.683+0000 7f47e9f48640 1 -- 172.21.15.37:0/2167825908 --> v2:172.21.15.37:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f47e4163e00 con 0x7f47e4161020 2024-04-25T10:30:48.685 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:48.684+0000 7f47e9f48640 1 --2- 172.21.15.37:0/2167825908 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f47e4161020 0x7f47e416b190 secure :-1 s=READY pgs=620 cs=0 l=1 rev1=1 crypto rx=0x7f47dc009d90 tx=0x7f47dc02a610 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ec4ad43a983f0cec server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:30:48.686 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:48.684+0000 7f47dbfff640 1 -- 172.21.15.37:0/2167825908 <== mon.0 v2:172.21.15.37:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f47dc031600 con 0x7f47e4161020 2024-04-25T10:30:48.686 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:48.684+0000 7f47dbfff640 1 -- 172.21.15.37:0/2167825908 <== mon.0 v2:172.21.15.37:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f47dc0317a0 con 0x7f47e4161020 2024-04-25T10:30:48.686 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:48.684+0000 7f47dbfff640 1 -- 172.21.15.37:0/2167825908 <== mon.0 v2:172.21.15.37:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f47dc031b70 con 0x7f47e4161020 2024-04-25T10:30:48.687 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:48.685+0000 7f47eb9ab640 1 -- 172.21.15.37:0/2167825908 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f47e4161020 msgr2=0x7f47e416b190 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:30:48.687 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:48.685+0000 7f47eb9ab640 1 --2- 172.21.15.37:0/2167825908 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f47e4161020 0x7f47e416b190 secure :-1 s=READY pgs=620 cs=0 l=1 rev1=1 crypto rx=0x7f47dc009d90 tx=0x7f47dc02a610 comp rx=0 tx=0).stop 2024-04-25T10:30:48.687 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:48.685+0000 7f47eb9ab640 1 -- 172.21.15.37:0/2167825908 shutdown_connections 2024-04-25T10:30:48.687 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:48.685+0000 7f47eb9ab640 1 --2- 172.21.15.37:0/2167825908 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f47e4161020 0x7f47e416b190 unknown :-1 s=CLOSED pgs=620 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:30:48.687 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:48.685+0000 7f47eb9ab640 1 -- 172.21.15.37:0/2167825908 >> 172.21.15.37:0/2167825908 conn(0x7f47e40b4560 msgr2=0x7f47e40b4970 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:30:48.687 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:48.685+0000 7f47eb9ab640 1 -- 172.21.15.37:0/2167825908 shutdown_connections 2024-04-25T10:30:48.687 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:48.685+0000 7f47eb9ab640 1 -- 172.21.15.37:0/2167825908 wait complete. 2024-04-25T10:30:48.687 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:48.687+0000 7f47eb9ab640 1 Processor -- start 2024-04-25T10:30:48.688 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:48.687+0000 7f47eb9ab640 1 -- start start 2024-04-25T10:30:48.691 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:48.688+0000 7f47eb9ab640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f47e406cc50 0x7f47e4142b30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:30:48.691 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:48.688+0000 7f47eb9ab640 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f47e415a9f0 0x7f47e413fb80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:30:48.691 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:48.688+0000 7f47eb9ab640 1 --2- >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f47e4161020 0x7f47e41f95c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:30:48.691 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:48.688+0000 7f47eb9ab640 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 -- 0x7f47e41646a0 con 0x7f47e406cc50 2024-04-25T10:30:48.691 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:48.688+0000 7f47eb9ab640 1 -- --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_getmap magic: 0 -- 0x7f47e41643c0 con 0x7f47e415a9f0 2024-04-25T10:30:48.691 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:48.688+0000 7f47eb9ab640 1 -- --> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] -- mon_getmap magic: 0 -- 0x7f47e4164bc0 con 0x7f47e4161020 2024-04-25T10:30:48.691 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:48.688+0000 7f47e9747640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f47e406cc50 0x7f47e4142b30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:30:48.691 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:48.688+0000 7f47e9747640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f47e406cc50 0x7f47e4142b30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.37:3300/0 says I am v2:172.21.15.37:58448/0 (socket says 172.21.15.37:58448) 2024-04-25T10:30:48.691 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:48.688+0000 7f47e9747640 1 -- 172.21.15.37:0/335030363 learned_addr learned my addr 172.21.15.37:0/335030363 (peer_addr_for_me v2:172.21.15.37:0/0) 2024-04-25T10:30:48.691 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:48.688+0000 7f47e9f48640 1 --2- 172.21.15.37:0/335030363 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f47e4161020 0x7f47e41f95c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:30:48.691 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:48.688+0000 7f47e8f46640 1 --2- 172.21.15.37:0/335030363 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f47e415a9f0 0x7f47e413fb80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:30:48.691 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:48.688+0000 7f47e9747640 1 -- 172.21.15.37:0/335030363 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f47e4161020 msgr2=0x7f47e41f95c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:30:48.692 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:48.688+0000 7f47e9747640 1 --2- 172.21.15.37:0/335030363 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f47e4161020 0x7f47e41f95c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:30:48.692 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:48.688+0000 7f47e9747640 1 -- 172.21.15.37:0/335030363 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f47e415a9f0 msgr2=0x7f47e413fb80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:30:48.692 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:48.688+0000 7f47e9747640 1 --2- 172.21.15.37:0/335030363 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f47e415a9f0 0x7f47e413fb80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:30:48.692 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:48.688+0000 7f47e9747640 1 -- 172.21.15.37:0/335030363 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f47e41f9b00 con 0x7f47e406cc50 2024-04-25T10:30:48.692 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:48.689+0000 7f47e9747640 1 --2- 172.21.15.37:0/335030363 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f47e406cc50 0x7f47e4142b30 secure :-1 s=READY pgs=621 cs=0 l=1 rev1=1 crypto rx=0x7f47e000c470 tx=0x7f47e000c930 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:30:48.692 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:48.689+0000 7f47d9ffb640 1 -- 172.21.15.37:0/335030363 <== mon.0 v2:172.21.15.37:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f47e0017070 con 0x7f47e406cc50 2024-04-25T10:30:48.692 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:48.689+0000 7f47d9ffb640 1 -- 172.21.15.37:0/335030363 <== mon.0 v2:172.21.15.37:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f47e0005d00 con 0x7f47e406cc50 2024-04-25T10:30:48.692 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:48.689+0000 7f47d9ffb640 1 -- 172.21.15.37:0/335030363 <== mon.0 v2:172.21.15.37:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f47e0007310 con 0x7f47e406cc50 2024-04-25T10:30:48.692 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:48.689+0000 7f47eb9ab640 1 -- 172.21.15.37:0/335030363 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f47e41f6a10 con 0x7f47e406cc50 2024-04-25T10:30:48.692 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:48.689+0000 7f47eb9ab640 1 -- 172.21.15.37:0/335030363 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f47e41f6fd0 con 0x7f47e406cc50 2024-04-25T10:30:48.692 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:48.690+0000 7f47eb9ab640 1 -- 172.21.15.37:0/335030363 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f47e406a140 con 0x7f47e406cc50 2024-04-25T10:30:48.693 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:48.690+0000 7f47d9ffb640 1 -- 172.21.15.37:0/335030363 <== mon.0 v2:172.21.15.37:3300/0 4 ==== mgrmap(e 5) ==== 96782+0+0 (secure 0 0 0) 0x7f47e001a070 con 0x7f47e406cc50 2024-04-25T10:30:48.693 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:48.691+0000 7f47d9ffb640 1 --2- 172.21.15.37:0/335030363 >> [v2:172.21.15.203:6832/2418205237,v1:172.21.15.203:6833/2418205237] conn(0x7f47c80744b0 0x7f47c80769c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:30:48.693 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:48.691+0000 7f47d9ffb640 1 -- 172.21.15.37:0/335030363 <== mon.0 v2:172.21.15.37:3300/0 5 ==== osd_map(44..44 src has 1..44) ==== 6248+0+0 (secure 0 0 0) 0x7f47e0094700 con 0x7f47e406cc50 2024-04-25T10:30:48.693 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:48.691+0000 7f47e8f46640 1 --2- 172.21.15.37:0/335030363 >> [v2:172.21.15.203:6832/2418205237,v1:172.21.15.203:6833/2418205237] conn(0x7f47c80744b0 0x7f47c80769c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:30:48.693 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:48.692+0000 7f47e8f46640 1 --2- 172.21.15.37:0/335030363 >> [v2:172.21.15.203:6832/2418205237,v1:172.21.15.203:6833/2418205237] conn(0x7f47c80744b0 0x7f47c80769c0 secure :-1 s=READY pgs=283 cs=0 l=1 rev1=1 crypto rx=0x7f47e415a2c0 tx=0x7f47d4009060 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:30:48.696 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:48.696+0000 7f47d9ffb640 1 -- 172.21.15.37:0/335030363 <== mon.0 v2:172.21.15.37:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187246 (secure 0 0 0) 0x7f47e005e610 con 0x7f47e406cc50 2024-04-25T10:30:48.889 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:48.888+0000 7f47eb9ab640 1 -- 172.21.15.37:0/335030363 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "recovery_fs_metadata", "var": "pg_num"} v 0) -- 0x7f47e4071d10 con 0x7f47e406cc50 2024-04-25T10:30:48.890 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:48.889+0000 7f47d9ffb640 1 -- 172.21.15.37:0/335030363 <== mon.0 v2:172.21.15.37:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "recovery_fs_metadata", "var": "pg_num"}]=0 v45) ==== 109+0+11 (secure 0 0 0) 0x7f47e0062910 con 0x7f47e406cc50 2024-04-25T10:30:48.890 INFO:teuthology.orchestra.run.smithi037.stdout:pg_num: 32 2024-04-25T10:30:48.892 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:48.891+0000 7f47eb9ab640 1 -- 172.21.15.37:0/335030363 >> [v2:172.21.15.203:6832/2418205237,v1:172.21.15.203:6833/2418205237] conn(0x7f47c80744b0 msgr2=0x7f47c80769c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:30:48.893 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:48.891+0000 7f47eb9ab640 1 --2- 172.21.15.37:0/335030363 >> [v2:172.21.15.203:6832/2418205237,v1:172.21.15.203:6833/2418205237] conn(0x7f47c80744b0 0x7f47c80769c0 secure :-1 s=READY pgs=283 cs=0 l=1 rev1=1 crypto rx=0x7f47e415a2c0 tx=0x7f47d4009060 comp rx=0 tx=0).stop 2024-04-25T10:30:48.893 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:48.891+0000 7f47eb9ab640 1 -- 172.21.15.37:0/335030363 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f47e406cc50 msgr2=0x7f47e4142b30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:30:48.893 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:48.891+0000 7f47eb9ab640 1 --2- 172.21.15.37:0/335030363 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f47e406cc50 0x7f47e4142b30 secure :-1 s=READY pgs=621 cs=0 l=1 rev1=1 crypto rx=0x7f47e000c470 tx=0x7f47e000c930 comp rx=0 tx=0).stop 2024-04-25T10:30:48.893 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:48.891+0000 7f47eb9ab640 1 -- 172.21.15.37:0/335030363 shutdown_connections 2024-04-25T10:30:48.893 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:48.891+0000 7f47eb9ab640 1 --2- 172.21.15.37:0/335030363 >> [v2:172.21.15.203:6832/2418205237,v1:172.21.15.203:6833/2418205237] conn(0x7f47c80744b0 0x7f47c80769c0 unknown :-1 s=CLOSED pgs=283 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:30:48.893 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:48.891+0000 7f47eb9ab640 1 --2- 172.21.15.37:0/335030363 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f47e4161020 0x7f47e41f95c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:30:48.893 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:48.891+0000 7f47eb9ab640 1 --2- 172.21.15.37:0/335030363 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f47e415a9f0 0x7f47e413fb80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:30:48.893 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:48.891+0000 7f47eb9ab640 1 --2- 172.21.15.37:0/335030363 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f47e406cc50 0x7f47e4142b30 unknown :-1 s=CLOSED pgs=621 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:30:48.893 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:48.891+0000 7f47eb9ab640 1 -- 172.21.15.37:0/335030363 >> 172.21.15.37:0/335030363 conn(0x7f47e40b4560 msgr2=0x7f47e40b6080 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:30:48.893 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:48.891+0000 7f47eb9ab640 1 -- 172.21.15.37:0/335030363 shutdown_connections 2024-04-25T10:30:48.893 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:48.891+0000 7f47eb9ab640 1 -- 172.21.15.37:0/335030363 wait complete. 2024-04-25T10:30:48.903 DEBUG:teuthology.orchestra.run.smithi037:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd blocklist ls 2024-04-25T10:30:49.016 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:49.015+0000 7fe5cb86e640 1 Processor -- start 2024-04-25T10:30:49.017 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:49.015+0000 7fe5cb86e640 1 -- start start 2024-04-25T10:30:49.020 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:49.016+0000 7fe5cb86e640 1 --2- >> v2:172.21.15.37:3300/0 conn(0x7fe5c40d5160 0x7fe5c40b4e80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:30:49.021 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:49.016+0000 7fe5cb86e640 1 --2- >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7fe5c40d0990 0x7fe5c40d2b20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:30:49.021 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:49.016+0000 7fe5cb86e640 1 --2- >> v2:172.21.15.203:3300/0 conn(0x7fe5c40d0da0 0x7fe5c40d3060 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:30:49.021 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:49.016+0000 7fe5cb86e640 1 -- --> v2:172.21.15.37:3300/0 -- mon_getmap magic: 0 -- 0x7fe5c40d1650 con 0x7fe5c40d5160 2024-04-25T10:30:49.021 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:49.016+0000 7fe5cb86e640 1 -- --> v2:172.21.15.203:3300/0 -- mon_getmap magic: 0 -- 0x7fe5c40d1370 con 0x7fe5c40d0da0 2024-04-25T10:30:49.021 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:49.016+0000 7fe5cb86e640 1 -- --> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] -- mon_getmap magic: 0 -- 0x7fe5c40d4680 con 0x7fe5c40d0990 2024-04-25T10:30:49.021 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:49.017+0000 7fe5c960a640 1 --2- >> v2:172.21.15.37:3300/0 conn(0x7fe5c40d5160 0x7fe5c40b4e80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:30:49.021 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:49.017+0000 7fe5c960a640 1 --2- >> v2:172.21.15.37:3300/0 conn(0x7fe5c40d5160 0x7fe5c40b4e80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.37:3300/0 says I am v2:172.21.15.37:58454/0 (socket says 172.21.15.37:58454) 2024-04-25T10:30:49.022 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:49.017+0000 7fe5c960a640 1 -- 172.21.15.37:0/3402428893 learned_addr learned my addr 172.21.15.37:0/3402428893 (peer_addr_for_me v2:172.21.15.37:0/0) 2024-04-25T10:30:49.022 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:49.017+0000 7fe5c8e09640 1 --2- 172.21.15.37:0/3402428893 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7fe5c40d0990 0x7fe5c40d2b20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:30:49.022 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:49.017+0000 7fe5c9e0b640 1 --2- 172.21.15.37:0/3402428893 >> v2:172.21.15.203:3300/0 conn(0x7fe5c40d0da0 0x7fe5c40d3060 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:30:49.022 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:49.017+0000 7fe5c960a640 1 -- 172.21.15.37:0/3402428893 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7fe5c40d0990 msgr2=0x7fe5c40d2b20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:30:49.022 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:49.017+0000 7fe5c960a640 1 --2- 172.21.15.37:0/3402428893 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7fe5c40d0990 0x7fe5c40d2b20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:30:49.022 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:49.017+0000 7fe5c960a640 1 -- 172.21.15.37:0/3402428893 >> v2:172.21.15.203:3300/0 conn(0x7fe5c40d0da0 msgr2=0x7fe5c40d3060 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:30:49.022 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:49.017+0000 7fe5c960a640 1 --2- 172.21.15.37:0/3402428893 >> v2:172.21.15.203:3300/0 conn(0x7fe5c40d0da0 0x7fe5c40d3060 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:30:49.023 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:49.017+0000 7fe5c960a640 1 -- 172.21.15.37:0/3402428893 --> v2:172.21.15.37:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fe5c40d4960 con 0x7fe5c40d5160 2024-04-25T10:30:49.023 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:49.018+0000 7fe5c960a640 1 --2- 172.21.15.37:0/3402428893 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fe5c40d5160 0x7fe5c40b4e80 secure :-1 s=READY pgs=622 cs=0 l=1 rev1=1 crypto rx=0x7fe5c40bba80 tx=0x7fe5b402a0b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d1d58c2cee7c079 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:30:49.023 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:49.018+0000 7fe5c3fff640 1 -- 172.21.15.37:0/3402428893 <== mon.0 v2:172.21.15.37:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe5b4037070 con 0x7fe5c40d5160 2024-04-25T10:30:49.023 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:49.018+0000 7fe5c3fff640 1 -- 172.21.15.37:0/3402428893 <== mon.0 v2:172.21.15.37:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fe5b402ae00 con 0x7fe5c40d5160 2024-04-25T10:30:49.023 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:49.018+0000 7fe5c3fff640 1 -- 172.21.15.37:0/3402428893 <== mon.0 v2:172.21.15.37:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe5b4032660 con 0x7fe5c40d5160 2024-04-25T10:30:49.023 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:49.018+0000 7fe5cb86e640 1 -- 172.21.15.37:0/3402428893 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fe5c40d5160 msgr2=0x7fe5c40b4e80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:30:49.023 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:49.018+0000 7fe5cb86e640 1 --2- 172.21.15.37:0/3402428893 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fe5c40d5160 0x7fe5c40b4e80 secure :-1 s=READY pgs=622 cs=0 l=1 rev1=1 crypto rx=0x7fe5c40bba80 tx=0x7fe5b402a0b0 comp rx=0 tx=0).stop 2024-04-25T10:30:49.023 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:49.019+0000 7fe5cb86e640 1 -- 172.21.15.37:0/3402428893 shutdown_connections 2024-04-25T10:30:49.024 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:49.019+0000 7fe5cb86e640 1 --2- 172.21.15.37:0/3402428893 >> v2:172.21.15.203:3300/0 conn(0x7fe5c40d0da0 0x7fe5c40d3060 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:30:49.024 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:49.019+0000 7fe5cb86e640 1 --2- 172.21.15.37:0/3402428893 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7fe5c40d0990 0x7fe5c40d2b20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:30:49.024 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:49.019+0000 7fe5cb86e640 1 --2- 172.21.15.37:0/3402428893 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fe5c40d5160 0x7fe5c40b4e80 unknown :-1 s=CLOSED pgs=622 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:30:49.024 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:49.019+0000 7fe5cb86e640 1 -- 172.21.15.37:0/3402428893 >> 172.21.15.37:0/3402428893 conn(0x7fe5c40b7a80 msgr2=0x7fe5c40b7e90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:30:49.024 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:49.019+0000 7fe5cb86e640 1 -- 172.21.15.37:0/3402428893 shutdown_connections 2024-04-25T10:30:49.024 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:49.019+0000 7fe5cb86e640 1 -- 172.21.15.37:0/3402428893 wait complete. 2024-04-25T10:30:49.030 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:49.020+0000 7fe5cb86e640 1 Processor -- start 2024-04-25T10:30:49.030 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:49.020+0000 7fe5cb86e640 1 -- start start 2024-04-25T10:30:49.030 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:49.022+0000 7fe5cb86e640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fe5c40d0990 0x7fe5c41f5ef0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:30:49.030 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:49.022+0000 7fe5cb86e640 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fe5c40d0da0 0x7fe5c40ca090 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:30:49.030 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:49.022+0000 7fe5cb86e640 1 --2- >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7fe5c40d5160 0x7fe5c40ca5d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:30:49.031 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:49.022+0000 7fe5cb86e640 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 -- 0x7fe5c4164c40 con 0x7fe5c40d0990 2024-04-25T10:30:49.031 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:49.022+0000 7fe5cb86e640 1 -- --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_getmap magic: 0 -- 0x7fe5c4164ac0 con 0x7fe5c40d0da0 2024-04-25T10:30:49.031 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:49.022+0000 7fe5cb86e640 1 -- --> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] -- mon_getmap magic: 0 -- 0x7fe5c406db10 con 0x7fe5c40d5160 2024-04-25T10:30:49.031 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:49.022+0000 7fe5c960a640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fe5c40d0990 0x7fe5c41f5ef0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:30:49.031 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:49.022+0000 7fe5c960a640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fe5c40d0990 0x7fe5c41f5ef0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.37:3300/0 says I am v2:172.21.15.37:58456/0 (socket says 172.21.15.37:58456) 2024-04-25T10:30:49.031 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:49.022+0000 7fe5c960a640 1 -- 172.21.15.37:0/3498887999 learned_addr learned my addr 172.21.15.37:0/3498887999 (peer_addr_for_me v2:172.21.15.37:0/0) 2024-04-25T10:30:49.031 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:49.022+0000 7fe5c8e09640 1 --2- 172.21.15.37:0/3498887999 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fe5c40d0da0 0x7fe5c40ca090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:30:49.032 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:49.022+0000 7fe5c9e0b640 1 --2- 172.21.15.37:0/3498887999 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7fe5c40d5160 0x7fe5c40ca5d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:30:49.032 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:49.022+0000 7fe5c960a640 1 -- 172.21.15.37:0/3498887999 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7fe5c40d5160 msgr2=0x7fe5c40ca5d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:30:49.032 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:49.022+0000 7fe5c960a640 1 --2- 172.21.15.37:0/3498887999 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7fe5c40d5160 0x7fe5c40ca5d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:30:49.032 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:49.022+0000 7fe5c960a640 1 -- 172.21.15.37:0/3498887999 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fe5c40d0da0 msgr2=0x7fe5c40ca090 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:30:49.032 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:49.022+0000 7fe5c960a640 1 --2- 172.21.15.37:0/3498887999 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fe5c40d0da0 0x7fe5c40ca090 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:30:49.032 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:49.022+0000 7fe5c960a640 1 -- 172.21.15.37:0/3498887999 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fe5c40cae40 con 0x7fe5c40d0990 2024-04-25T10:30:49.032 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:49.022+0000 7fe5c8e09640 1 --2- 172.21.15.37:0/3498887999 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fe5c40d0da0 0x7fe5c40ca090 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T10:30:49.033 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:49.022+0000 7fe5c960a640 1 --2- 172.21.15.37:0/3498887999 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fe5c40d0990 0x7fe5c41f5ef0 secure :-1 s=READY pgs=623 cs=0 l=1 rev1=1 crypto rx=0x7fe5b4000c00 tx=0x7fe5b40046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:30:49.033 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:49.022+0000 7fe5c1ffb640 1 -- 172.21.15.37:0/3498887999 <== mon.0 v2:172.21.15.37:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe5b4041030 con 0x7fe5c40d0990 2024-04-25T10:30:49.033 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:49.023+0000 7fe5c1ffb640 1 -- 172.21.15.37:0/3498887999 <== mon.0 v2:172.21.15.37:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fe5b402ac00 con 0x7fe5c40d0990 2024-04-25T10:30:49.033 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:49.023+0000 7fe5cb86e640 1 -- 172.21.15.37:0/3498887999 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fe5c40cb120 con 0x7fe5c40d0990 2024-04-25T10:30:49.033 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:49.023+0000 7fe5c1ffb640 1 -- 172.21.15.37:0/3498887999 <== mon.0 v2:172.21.15.37:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe5b4037040 con 0x7fe5c40d0990 2024-04-25T10:30:49.033 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:49.023+0000 7fe5cb86e640 1 -- 172.21.15.37:0/3498887999 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fe5c4150260 con 0x7fe5c40d0990 2024-04-25T10:30:49.034 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:49.024+0000 7fe5cb86e640 1 -- 172.21.15.37:0/3498887999 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fe5c414eae0 con 0x7fe5c40d0990 2024-04-25T10:30:49.034 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:49.025+0000 7fe5c1ffb640 1 -- 172.21.15.37:0/3498887999 <== mon.0 v2:172.21.15.37:3300/0 4 ==== mgrmap(e 5) ==== 96782+0+0 (secure 0 0 0) 0x7fe5b4044070 con 0x7fe5c40d0990 2024-04-25T10:30:49.034 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:49.025+0000 7fe5c1ffb640 1 --2- 172.21.15.37:0/3498887999 >> [v2:172.21.15.203:6832/2418205237,v1:172.21.15.203:6833/2418205237] conn(0x7fe5b00744e0 0x7fe5b00769f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:30:49.034 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:49.025+0000 7fe5c1ffb640 1 -- 172.21.15.37:0/3498887999 <== mon.0 v2:172.21.15.37:3300/0 5 ==== osd_map(45..45 src has 1..45) ==== 6364+0+0 (secure 0 0 0) 0x7fe5b40b57c0 con 0x7fe5c40d0990 2024-04-25T10:30:49.034 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:49.026+0000 7fe5c8e09640 1 --2- 172.21.15.37:0/3498887999 >> [v2:172.21.15.203:6832/2418205237,v1:172.21.15.203:6833/2418205237] conn(0x7fe5b00744e0 0x7fe5b00769f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:30:49.034 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:49.026+0000 7fe5c8e09640 1 --2- 172.21.15.37:0/3498887999 >> [v2:172.21.15.203:6832/2418205237,v1:172.21.15.203:6833/2418205237] conn(0x7fe5b00744e0 0x7fe5b00769f0 secure :-1 s=READY pgs=284 cs=0 l=1 rev1=1 crypto rx=0x7fe5b8006fd0 tx=0x7fe5b8008040 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:30:49.035 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:49.030+0000 7fe5c1ffb640 1 -- 172.21.15.37:0/3498887999 <== mon.0 v2:172.21.15.37:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187246 (secure 0 0 0) 0x7fe5c414eae0 con 0x7fe5c40d0990 2024-04-25T10:30:49.232 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:49.230+0000 7fe5cb86e640 1 -- 172.21.15.37:0/3498887999 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_command({"prefix": "osd blocklist ls"} v 0) -- 0x7fe5c41504c0 con 0x7fe5c40d0990 2024-04-25T10:30:49.232 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:49.231+0000 7fe5c1ffb640 1 -- 172.21.15.37:0/3498887999 <== mon.0 v2:172.21.15.37:3300/0 7 ==== mon_command_ack([{"prefix": "osd blocklist ls"}]=0 listed 2 entries v45) ==== 80+0+120 (secure 0 0 0) 0x7fe5b404b070 con 0x7fe5c40d0990 2024-04-25T10:30:49.232 INFO:teuthology.orchestra.run.smithi037.stdout:172.21.15.37:6835/311552410 2024-04-26T10:30:24.853732+0000 2024-04-25T10:30:49.232 INFO:teuthology.orchestra.run.smithi037.stdout:172.21.15.37:6832/311552410 2024-04-26T10:30:24.853732+0000 2024-04-25T10:30:49.232 INFO:teuthology.orchestra.run.smithi037.stderr:listed 2 entries 2024-04-25T10:30:49.234 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:49.233+0000 7fe5cb86e640 1 -- 172.21.15.37:0/3498887999 >> [v2:172.21.15.203:6832/2418205237,v1:172.21.15.203:6833/2418205237] conn(0x7fe5b00744e0 msgr2=0x7fe5b00769f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:30:49.234 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:49.233+0000 7fe5cb86e640 1 --2- 172.21.15.37:0/3498887999 >> [v2:172.21.15.203:6832/2418205237,v1:172.21.15.203:6833/2418205237] conn(0x7fe5b00744e0 0x7fe5b00769f0 secure :-1 s=READY pgs=284 cs=0 l=1 rev1=1 crypto rx=0x7fe5b8006fd0 tx=0x7fe5b8008040 comp rx=0 tx=0).stop 2024-04-25T10:30:49.235 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:49.233+0000 7fe5cb86e640 1 -- 172.21.15.37:0/3498887999 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fe5c40d0990 msgr2=0x7fe5c41f5ef0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:30:49.235 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:49.233+0000 7fe5cb86e640 1 --2- 172.21.15.37:0/3498887999 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fe5c40d0990 0x7fe5c41f5ef0 secure :-1 s=READY pgs=623 cs=0 l=1 rev1=1 crypto rx=0x7fe5b4000c00 tx=0x7fe5b40046c0 comp rx=0 tx=0).stop 2024-04-25T10:30:49.235 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:49.233+0000 7fe5cb86e640 1 -- 172.21.15.37:0/3498887999 shutdown_connections 2024-04-25T10:30:49.235 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:49.233+0000 7fe5cb86e640 1 --2- 172.21.15.37:0/3498887999 >> [v2:172.21.15.203:6832/2418205237,v1:172.21.15.203:6833/2418205237] conn(0x7fe5b00744e0 0x7fe5b00769f0 unknown :-1 s=CLOSED pgs=284 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:30:49.235 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:49.233+0000 7fe5cb86e640 1 --2- 172.21.15.37:0/3498887999 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7fe5c40d5160 0x7fe5c40ca5d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:30:49.235 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:49.233+0000 7fe5cb86e640 1 --2- 172.21.15.37:0/3498887999 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fe5c40d0da0 0x7fe5c40ca090 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:30:49.235 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:49.233+0000 7fe5cb86e640 1 --2- 172.21.15.37:0/3498887999 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fe5c40d0990 0x7fe5c41f5ef0 unknown :-1 s=CLOSED pgs=623 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:30:49.235 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:49.233+0000 7fe5cb86e640 1 -- 172.21.15.37:0/3498887999 >> 172.21.15.37:0/3498887999 conn(0x7fe5c40b7a80 msgr2=0x7fe5c4161ef0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:30:49.235 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:49.233+0000 7fe5cb86e640 1 -- 172.21.15.37:0/3498887999 shutdown_connections 2024-04-25T10:30:49.235 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:49.233+0000 7fe5cb86e640 1 -- 172.21.15.37:0/3498887999 wait complete. 2024-04-25T10:30:49.245 INFO:tasks.cephfs.fuse_mount:Running fusermount -u on ubuntu@smithi037.front.sepia.ceph.com... 2024-04-25T10:30:49.245 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T10:30:49.245 DEBUG:teuthology.orchestra.run.smithi037:> sudo fusermount -u /home/ubuntu/cephtest/mnt.0 2024-04-25T10:30:49.268 INFO:tasks.cephfs.fuse_mount.ceph-fuse.0.smithi037.stderr:ceph-fuse[51764]: fuse finished with error 0 and tester_r 0 2024-04-25T10:30:49.270 INFO:tasks.cephfs.mount:Cleaning up mount ubuntu@smithi037.front.sepia.ceph.com 2024-04-25T10:30:49.270 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T10:30:49.270 DEBUG:teuthology.orchestra.run.smithi037:> (cd /home/ubuntu/cephtest && exec rmdir -- /home/ubuntu/cephtest/mnt.0) 2024-04-25T10:30:49.436 INFO:tasks.cephfs.mount:Terminating background process 2024-04-25T10:30:49.436 DEBUG:teuthology.orchestra.run.smithi203:> set -ex 2024-04-25T10:30:49.436 DEBUG:teuthology.orchestra.run.smithi203:> dd if=/proc/self/mounts of=/dev/stdout 2024-04-25T10:30:49.453 DEBUG:tasks.cephfs.mount:not mounted; /proc/self/mounts is: proc /proc proc rw,nosuid,nodev,noexec,relatime 0 0 sysfs /sys sysfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 devtmpfs /dev devtmpfs rw,seclabel,nosuid,size=4096k,nr_inodes=4056088,mode=755,inode64 0 0 securityfs /sys/kernel/security securityfs rw,nosuid,nodev,noexec,relatime 0 0 tmpfs /dev/shm tmpfs rw,seclabel,nosuid,nodev,inode64 0 0 devpts /dev/pts devpts rw,seclabel,nosuid,noexec,relatime,gid=5,mode=620,ptmxmode=000 0 0 tmpfs /run tmpfs rw,seclabel,nosuid,nodev,size=6497660k,nr_inodes=819200,mode=755,inode64 0 0 cgroup2 /sys/fs/cgroup cgroup2 rw,seclabel,nosuid,nodev,noexec,relatime,nsdelegate,memory_recursiveprot 0 0 pstore /sys/fs/pstore pstore rw,seclabel,nosuid,nodev,noexec,relatime 0 0 bpf /sys/fs/bpf bpf rw,nosuid,nodev,noexec,relatime,mode=700 0 0 /dev/sda1 / ext4 rw,seclabel,relatime 0 0 selinuxfs /sys/fs/selinux selinuxfs rw,nosuid,noexec,relatime 0 0 systemd-1 /proc/sys/fs/binfmt_misc autofs rw,relatime,fd=29,pgrp=1,timeout=0,minproto=5,maxproto=5,direct,pipe_ino=181 0 0 hugetlbfs /dev/hugepages hugetlbfs rw,seclabel,relatime,pagesize=2M 0 0 mqueue /dev/mqueue mqueue rw,seclabel,nosuid,nodev,noexec,relatime 0 0 debugfs /sys/kernel/debug debugfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 tracefs /sys/kernel/tracing tracefs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 configfs /sys/kernel/config configfs rw,nosuid,nodev,noexec,relatime 0 0 none /run/credentials/systemd-sysusers.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 fusectl /sys/fs/fuse/connections fusectl rw,nosuid,nodev,noexec,relatime 0 0 none /run/credentials/systemd-sysctl.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 none /run/credentials/systemd-tmpfiles-setup-dev.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 none /run/credentials/systemd-tmpfiles-setup.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 sunrpc /var/lib/nfs/rpc_pipefs rpc_pipefs rw,relatime 0 0 tmpfs /run/user/1000 tmpfs rw,seclabel,nosuid,nodev,relatime,size=3248828k,nr_inodes=812207,mode=700,uid=1000,gid=1316,inode64 0 0 /dev/mapper/vg_nvme-lv_5 /var/lib/ceph xfs rw,seclabel,relatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_1 /var/lib/ceph/osd/ceph-4 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_2 /var/lib/ceph/osd/ceph-5 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_3 /var/lib/ceph/osd/ceph-6 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_4 /var/lib/ceph/osd/ceph-7 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 tmpfs /run/netns tmpfs rw,seclabel,nosuid,nodev,size=6497660k,nr_inodes=819200,mode=755,inode64 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 nsfs rw 0 0 2024-04-25T10:30:49.453 INFO:tasks.cephfs.mount:Terminating background process 2024-04-25T10:30:49.453 DEBUG:teuthology.orchestra.run.smithi037:> set -ex 2024-04-25T10:30:49.454 DEBUG:teuthology.orchestra.run.smithi037:> dd if=/proc/self/mounts of=/dev/stdout 2024-04-25T10:30:49.493 DEBUG:tasks.cephfs.mount:not mounted; /proc/self/mounts is: proc /proc proc rw,nosuid,nodev,noexec,relatime 0 0 sysfs /sys sysfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 devtmpfs /dev devtmpfs rw,seclabel,nosuid,size=4096k,nr_inodes=4056882,mode=755,inode64 0 0 securityfs /sys/kernel/security securityfs rw,nosuid,nodev,noexec,relatime 0 0 tmpfs /dev/shm tmpfs rw,seclabel,nosuid,nodev,inode64 0 0 devpts /dev/pts devpts rw,seclabel,nosuid,noexec,relatime,gid=5,mode=620,ptmxmode=000 0 0 tmpfs /run tmpfs rw,seclabel,nosuid,nodev,size=6498932k,nr_inodes=819200,mode=755,inode64 0 0 cgroup2 /sys/fs/cgroup cgroup2 rw,seclabel,nosuid,nodev,noexec,relatime,nsdelegate,memory_recursiveprot 0 0 pstore /sys/fs/pstore pstore rw,seclabel,nosuid,nodev,noexec,relatime 0 0 bpf /sys/fs/bpf bpf rw,nosuid,nodev,noexec,relatime,mode=700 0 0 /dev/sda1 / ext4 rw,seclabel,relatime 0 0 selinuxfs /sys/fs/selinux selinuxfs rw,nosuid,noexec,relatime 0 0 systemd-1 /proc/sys/fs/binfmt_misc autofs rw,relatime,fd=29,pgrp=1,timeout=0,minproto=5,maxproto=5,direct,pipe_ino=21000 0 0 hugetlbfs /dev/hugepages hugetlbfs rw,seclabel,relatime,pagesize=2M 0 0 mqueue /dev/mqueue mqueue rw,seclabel,nosuid,nodev,noexec,relatime 0 0 debugfs /sys/kernel/debug debugfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 tracefs /sys/kernel/tracing tracefs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 fusectl /sys/fs/fuse/connections fusectl rw,nosuid,nodev,noexec,relatime 0 0 configfs /sys/kernel/config configfs rw,nosuid,nodev,noexec,relatime 0 0 none /run/credentials/systemd-sysusers.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 none /run/credentials/systemd-sysctl.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 none /run/credentials/systemd-tmpfiles-setup-dev.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 none /run/credentials/systemd-tmpfiles-setup.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 sunrpc /var/lib/nfs/rpc_pipefs rpc_pipefs rw,relatime 0 0 tmpfs /run/user/1000 tmpfs rw,seclabel,nosuid,nodev,relatime,size=3249464k,nr_inodes=812366,mode=700,uid=1000,gid=1316,inode64 0 0 /dev/mapper/vg_nvme-lv_5 /var/lib/ceph xfs rw,seclabel,relatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_1 /var/lib/ceph/osd/ceph-0 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_2 /var/lib/ceph/osd/ceph-1 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_3 /var/lib/ceph/osd/ceph-2 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_4 /var/lib/ceph/osd/ceph-3 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 tmpfs /run/netns tmpfs rw,seclabel,nosuid,nodev,size=6498932k,nr_inodes=819200,mode=755,inode64 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 nsfs rw 0 0 2024-04-25T10:30:49.493 INFO:tasks.cephfs.mount:Terminating background process 2024-04-25T10:30:49.493 DEBUG:teuthology.orchestra.run.smithi203:> set -ex 2024-04-25T10:30:49.493 DEBUG:teuthology.orchestra.run.smithi203:> dd if=/proc/self/mounts of=/dev/stdout 2024-04-25T10:30:49.509 DEBUG:tasks.cephfs.mount:not mounted; /proc/self/mounts is: proc /proc proc rw,nosuid,nodev,noexec,relatime 0 0 sysfs /sys sysfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 devtmpfs /dev devtmpfs rw,seclabel,nosuid,size=4096k,nr_inodes=4056088,mode=755,inode64 0 0 securityfs /sys/kernel/security securityfs rw,nosuid,nodev,noexec,relatime 0 0 tmpfs /dev/shm tmpfs rw,seclabel,nosuid,nodev,inode64 0 0 devpts /dev/pts devpts rw,seclabel,nosuid,noexec,relatime,gid=5,mode=620,ptmxmode=000 0 0 tmpfs /run tmpfs rw,seclabel,nosuid,nodev,size=6497660k,nr_inodes=819200,mode=755,inode64 0 0 cgroup2 /sys/fs/cgroup cgroup2 rw,seclabel,nosuid,nodev,noexec,relatime,nsdelegate,memory_recursiveprot 0 0 pstore /sys/fs/pstore pstore rw,seclabel,nosuid,nodev,noexec,relatime 0 0 bpf /sys/fs/bpf bpf rw,nosuid,nodev,noexec,relatime,mode=700 0 0 /dev/sda1 / ext4 rw,seclabel,relatime 0 0 selinuxfs /sys/fs/selinux selinuxfs rw,nosuid,noexec,relatime 0 0 systemd-1 /proc/sys/fs/binfmt_misc autofs rw,relatime,fd=29,pgrp=1,timeout=0,minproto=5,maxproto=5,direct,pipe_ino=181 0 0 hugetlbfs /dev/hugepages hugetlbfs rw,seclabel,relatime,pagesize=2M 0 0 mqueue /dev/mqueue mqueue rw,seclabel,nosuid,nodev,noexec,relatime 0 0 debugfs /sys/kernel/debug debugfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 tracefs /sys/kernel/tracing tracefs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 configfs /sys/kernel/config configfs rw,nosuid,nodev,noexec,relatime 0 0 none /run/credentials/systemd-sysusers.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 fusectl /sys/fs/fuse/connections fusectl rw,nosuid,nodev,noexec,relatime 0 0 none /run/credentials/systemd-sysctl.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 none /run/credentials/systemd-tmpfiles-setup-dev.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 none /run/credentials/systemd-tmpfiles-setup.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 sunrpc /var/lib/nfs/rpc_pipefs rpc_pipefs rw,relatime 0 0 tmpfs /run/user/1000 tmpfs rw,seclabel,nosuid,nodev,relatime,size=3248828k,nr_inodes=812207,mode=700,uid=1000,gid=1316,inode64 0 0 /dev/mapper/vg_nvme-lv_5 /var/lib/ceph xfs rw,seclabel,relatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_1 /var/lib/ceph/osd/ceph-4 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_2 /var/lib/ceph/osd/ceph-5 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_3 /var/lib/ceph/osd/ceph-6 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_4 /var/lib/ceph/osd/ceph-7 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 tmpfs /run/netns tmpfs rw,seclabel,nosuid,nodev,size=6497660k,nr_inodes=819200,mode=755,inode64 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 nsfs rw 0 0 2024-04-25T10:30:49.509 DEBUG:teuthology.orchestra.run.smithi037:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-04-25T10:30:49.644 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:49.643+0000 7fa1354d9640 1 Processor -- start 2024-04-25T10:30:49.644 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:49.643+0000 7fa1354d9640 1 -- start start 2024-04-25T10:30:49.646 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:49.644+0000 7fa1354d9640 1 --2- >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7fa13015a9f0 0x7fa13006c540 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:30:49.646 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:49.644+0000 7fa1354d9640 1 --2- >> v2:172.21.15.203:3300/0 conn(0x7fa130161020 0x7fa1300b5860 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:30:49.647 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:49.644+0000 7fa1354d9640 1 -- --> v1:172.21.15.203:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fa130161430 con 0x7fa13006ca80 2024-04-25T10:30:49.647 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:49.645+0000 7fa1354d9640 1 -- --> v2:172.21.15.203:3300/0 -- mon_getmap magic: 0 -- 0x7fa13006ce90 con 0x7fa130161020 2024-04-25T10:30:49.647 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:49.645+0000 7fa1354d9640 1 -- --> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] -- mon_getmap magic: 0 -- 0x7fa1301616d0 con 0x7fa13015a9f0 2024-04-25T10:30:49.647 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:49.645+0000 7fa12effd640 1 --2- >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7fa13015a9f0 0x7fa13006c540 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:30:49.647 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:49.645+0000 7fa12f7fe640 1 --2- >> v2:172.21.15.203:3300/0 conn(0x7fa130161020 0x7fa1300b5860 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:30:49.647 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:49.645+0000 7fa12e7fc640 1 --1- >> v1:172.21.15.203:6789/0 conn(0x7fa13006ca80 0x7fa130160910 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.203:6789/0 says I am v1:172.21.15.37:37366/0 (socket says 172.21.15.37:37366) 2024-04-25T10:30:49.647 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:49.645+0000 7fa12e7fc640 1 -- 172.21.15.37:0/4190340421 learned_addr learned my addr 172.21.15.37:0/4190340421 (peer_addr_for_me v1:172.21.15.37:0/0) 2024-04-25T10:30:49.647 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:49.645+0000 7fa12effd640 1 -- 172.21.15.37:0/4190340421 >> v2:172.21.15.203:3300/0 conn(0x7fa130161020 msgr2=0x7fa1300b5860 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:30:49.647 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:49.645+0000 7fa12effd640 1 --2- 172.21.15.37:0/4190340421 >> v2:172.21.15.203:3300/0 conn(0x7fa130161020 0x7fa1300b5860 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:30:49.647 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:49.645+0000 7fa12effd640 1 -- 172.21.15.37:0/4190340421 >> v1:172.21.15.203:6789/0 conn(0x7fa13006ca80 legacy=0x7fa130160910 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:30:49.647 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:49.645+0000 7fa12effd640 1 -- 172.21.15.37:0/4190340421 --> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fa130163e00 con 0x7fa13015a9f0 2024-04-25T10:30:49.647 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:49.645+0000 7fa12dffb640 1 -- 172.21.15.37:0/4190340421 <== mon.1 v1:172.21.15.203:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 1718250168 0 0) 0x7fa130161430 con 0x7fa13006ca80 2024-04-25T10:30:49.647 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:49.645+0000 7fa12f7fe640 1 --2- 172.21.15.37:0/4190340421 >> v2:172.21.15.203:3300/0 conn(0x7fa130161020 0x7fa1300b5860 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-25T10:30:49.649 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:49.646+0000 7fa12effd640 1 --2- 172.21.15.37:0/4190340421 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7fa13015a9f0 0x7fa13006c540 secure :-1 s=READY pgs=63 cs=0 l=1 rev1=1 crypto rx=0x7fa1280097a0 tx=0x7fa12802a180 comp rx=0 tx=0).ready entity=mon.2 client_cookie=d0af1e4cef238fce server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:30:49.649 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:49.646+0000 7fa12dffb640 1 -- 172.21.15.37:0/4190340421 <== mon.2 v2:172.21.15.203:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa128037070 con 0x7fa13015a9f0 2024-04-25T10:30:49.649 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:49.646+0000 7fa12dffb640 1 -- 172.21.15.37:0/4190340421 <== mon.2 v2:172.21.15.203:3301/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fa128032440 con 0x7fa13015a9f0 2024-04-25T10:30:49.649 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:49.646+0000 7fa12dffb640 1 -- 172.21.15.37:0/4190340421 <== mon.2 v2:172.21.15.203:3301/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa128032810 con 0x7fa13015a9f0 2024-04-25T10:30:49.649 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:49.647+0000 7fa1354d9640 1 -- 172.21.15.37:0/4190340421 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7fa13015a9f0 msgr2=0x7fa13006c540 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:30:49.649 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:49.647+0000 7fa1354d9640 1 --2- 172.21.15.37:0/4190340421 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7fa13015a9f0 0x7fa13006c540 secure :-1 s=READY pgs=63 cs=0 l=1 rev1=1 crypto rx=0x7fa1280097a0 tx=0x7fa12802a180 comp rx=0 tx=0).stop 2024-04-25T10:30:49.649 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:49.647+0000 7fa1354d9640 1 -- 172.21.15.37:0/4190340421 shutdown_connections 2024-04-25T10:30:49.649 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:49.647+0000 7fa1354d9640 1 --2- 172.21.15.37:0/4190340421 >> v2:172.21.15.203:3300/0 conn(0x7fa130161020 0x7fa1300b5860 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:30:49.650 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:49.647+0000 7fa1354d9640 1 --2- 172.21.15.37:0/4190340421 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7fa13015a9f0 0x7fa13006c540 unknown :-1 s=CLOSED pgs=63 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:30:49.650 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:49.647+0000 7fa1354d9640 1 -- 172.21.15.37:0/4190340421 >> 172.21.15.37:0/4190340421 conn(0x7fa1300b4560 msgr2=0x7fa1300b4970 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:30:49.650 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:49.647+0000 7fa1354d9640 1 -- 172.21.15.37:0/4190340421 shutdown_connections 2024-04-25T10:30:49.650 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:49.647+0000 7fa1354d9640 1 -- 172.21.15.37:0/4190340421 wait complete. 2024-04-25T10:30:49.650 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:49.649+0000 7fa1354d9640 1 Processor -- start 2024-04-25T10:30:49.650 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:49.649+0000 7fa1354d9640 1 -- start start 2024-04-25T10:30:49.652 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:49.650+0000 7fa1354d9640 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fa13006ca80 0x7fa1300ce860 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:30:49.652 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:49.650+0000 7fa1354d9640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fa13015a9f0 0x7fa130159750 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:30:49.652 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:49.650+0000 7fa1354d9640 1 --2- >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7fa130161020 0x7fa13015bf80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:30:49.652 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:49.650+0000 7fa1354d9640 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 -- 0x7fa1301646a0 con 0x7fa13015a9f0 2024-04-25T10:30:49.652 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:49.650+0000 7fa1354d9640 1 -- --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_getmap magic: 0 -- 0x7fa1301643c0 con 0x7fa13006ca80 2024-04-25T10:30:49.653 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:49.650+0000 7fa1354d9640 1 -- --> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] -- mon_getmap magic: 0 -- 0x7fa130164bc0 con 0x7fa130161020 2024-04-25T10:30:49.653 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:49.650+0000 7fa12e7fc640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fa13015a9f0 0x7fa130159750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:30:49.653 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:49.650+0000 7fa12e7fc640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fa13015a9f0 0x7fa130159750 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.37:3300/0 says I am v2:172.21.15.37:59344/0 (socket says 172.21.15.37:59344) 2024-04-25T10:30:49.653 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:49.650+0000 7fa12e7fc640 1 -- 172.21.15.37:0/1395349429 learned_addr learned my addr 172.21.15.37:0/1395349429 (peer_addr_for_me v2:172.21.15.37:0/0) 2024-04-25T10:30:49.653 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:49.650+0000 7fa12effd640 1 --2- 172.21.15.37:0/1395349429 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fa13006ca80 0x7fa1300ce860 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:30:49.653 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:49.650+0000 7fa12f7fe640 1 --2- 172.21.15.37:0/1395349429 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7fa130161020 0x7fa13015bf80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:30:49.653 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:49.651+0000 7fa12e7fc640 1 -- 172.21.15.37:0/1395349429 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7fa130161020 msgr2=0x7fa13015bf80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:30:49.653 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:49.651+0000 7fa12e7fc640 1 --2- 172.21.15.37:0/1395349429 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7fa130161020 0x7fa13015bf80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:30:49.653 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:49.651+0000 7fa12e7fc640 1 -- 172.21.15.37:0/1395349429 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fa13006ca80 msgr2=0x7fa1300ce860 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:30:49.653 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:49.651+0000 7fa12e7fc640 1 --2- 172.21.15.37:0/1395349429 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fa13006ca80 0x7fa1300ce860 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:30:49.653 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:49.651+0000 7fa12e7fc640 1 -- 172.21.15.37:0/1395349429 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fa1300cd7c0 con 0x7fa13015a9f0 2024-04-25T10:30:49.653 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:49.651+0000 7fa12effd640 1 --2- 172.21.15.37:0/1395349429 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fa13006ca80 0x7fa1300ce860 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-25T10:30:49.653 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:49.651+0000 7fa12e7fc640 1 --2- 172.21.15.37:0/1395349429 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fa13015a9f0 0x7fa130159750 secure :-1 s=READY pgs=624 cs=0 l=1 rev1=1 crypto rx=0x7fa11800c460 tx=0x7fa11800c860 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:30:49.654 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:49.651+0000 7fa1277fe640 1 -- 172.21.15.37:0/1395349429 <== mon.0 v2:172.21.15.37:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa1180074e0 con 0x7fa13015a9f0 2024-04-25T10:30:49.654 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:49.651+0000 7fa1277fe640 1 -- 172.21.15.37:0/1395349429 <== mon.0 v2:172.21.15.37:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fa1180047e0 con 0x7fa13015a9f0 2024-04-25T10:30:49.654 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:49.651+0000 7fa1354d9640 1 -- 172.21.15.37:0/1395349429 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fa1300cdaa0 con 0x7fa13015a9f0 2024-04-25T10:30:49.654 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:49.651+0000 7fa1277fe640 1 -- 172.21.15.37:0/1395349429 <== mon.0 v2:172.21.15.37:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa118004bd0 con 0x7fa13015a9f0 2024-04-25T10:30:49.654 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:49.651+0000 7fa1354d9640 1 -- 172.21.15.37:0/1395349429 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fa1300ce060 con 0x7fa13015a9f0 2024-04-25T10:30:49.655 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:49.652+0000 7fa1354d9640 1 -- 172.21.15.37:0/1395349429 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fa13006a140 con 0x7fa13015a9f0 2024-04-25T10:30:49.655 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:49.653+0000 7fa1277fe640 1 -- 172.21.15.37:0/1395349429 <== mon.0 v2:172.21.15.37:3300/0 4 ==== mgrmap(e 5) ==== 96782+0+0 (secure 0 0 0) 0x7fa118011070 con 0x7fa13015a9f0 2024-04-25T10:30:49.655 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:49.653+0000 7fa1277fe640 1 --2- 172.21.15.37:0/1395349429 >> [v2:172.21.15.203:6832/2418205237,v1:172.21.15.203:6833/2418205237] conn(0x7fa1100744e0 0x7fa1100769f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:30:49.655 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:49.653+0000 7fa1277fe640 1 -- 172.21.15.37:0/1395349429 <== mon.0 v2:172.21.15.37:3300/0 5 ==== osd_map(45..45 src has 1..45) ==== 6364+0+0 (secure 0 0 0) 0x7fa118094f60 con 0x7fa13015a9f0 2024-04-25T10:30:49.655 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:49.653+0000 7fa12effd640 1 --2- 172.21.15.37:0/1395349429 >> [v2:172.21.15.203:6832/2418205237,v1:172.21.15.203:6833/2418205237] conn(0x7fa1100744e0 0x7fa1100769f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:30:49.655 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:49.654+0000 7fa12effd640 1 --2- 172.21.15.37:0/1395349429 >> [v2:172.21.15.203:6832/2418205237,v1:172.21.15.203:6833/2418205237] conn(0x7fa1100744e0 0x7fa1100769f0 secure :-1 s=READY pgs=285 cs=0 l=1 rev1=1 crypto rx=0x7fa128034000 tx=0x7fa12802e000 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:30:49.659 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:49.658+0000 7fa1277fe640 1 -- 172.21.15.37:0/1395349429 <== mon.0 v2:172.21.15.37:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187246 (secure 0 0 0) 0x7fa11805f460 con 0x7fa13015a9f0 2024-04-25T10:30:49.887 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:49.886+0000 7fa1354d9640 1 -- 172.21.15.37:0/1395349429 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7fa1300cd550 con 0x7fa13015a9f0 2024-04-25T10:30:49.888 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:49.887+0000 7fa1277fe640 1 -- 172.21.15.37:0/1395349429 <== mon.0 v2:172.21.15.37:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 27 v27) ==== 94+0+6489 (secure 0 0 0) 0x7fa11805ed60 con 0x7fa13015a9f0 2024-04-25T10:30:49.888 INFO:teuthology.orchestra.run.smithi037.stdout: 2024-04-25T10:30:49.888 INFO:teuthology.orchestra.run.smithi037.stdout:{"epoch":27,"default_fscid":2,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":4946,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.37:6835/683655741","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.37:6834","nonce":683655741},{"type":"v1","addr":"172.21.15.37:6835","nonce":683655741}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":19},{"gid":5215,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.203:6836/3927045001","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6835","nonce":3927045001},{"type":"v1","addr":"172.21.15.203:6836","nonce":3927045001}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":19},{"gid":5218,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.203:6837/3108183946","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6834","nonce":3108183946},{"type":"v1","addr":"172.21.15.203:6837","nonce":3108183946}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":19}],"filesystems":[{"mdsmap":{"epoch":18,"flags":19,"flags_state":{"joinable":false,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-04-25T10:30:12.130473+0000","modified":"2024-04-25T10:30:24.853915+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":37,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":1,"in":[0],"up":{},"failed":[0],"damaged":[],"stopped":[],"info":{},"data_pools":[5],"metadata_pool":4,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":0,"qdb_cluster":[]},"id":2},{"mdsmap":{"epoch":27,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-04-25T10:30:28.588617+0000","modified":"2024-04-25T10:30:42.935167+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":1,"in":[0],"up":{"mds_0":5625},"failed":[],"damaged":[],"stopped":[],"info":{"gid_5625":{"gid":5625,"name":"c","rank":0,"incarnation":24,"state":"up:active","state_seq":7,"addr":"172.21.15.37:6833/2353203722","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.37:6832","nonce":2353203722},{"type":"v1","addr":"172.21.15.37:6833","nonce":2353203722}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[5],"metadata_pool":6,"enabled":true,"fs_name":"recovery_fs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":5625,"qdb_cluster":[5625]},"id":3}]} 2024-04-25T10:30:49.889 INFO:teuthology.orchestra.run.smithi037.stderr:dumped fsmap epoch 27 2024-04-25T10:30:49.891 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:49.889+0000 7fa1354d9640 1 -- 172.21.15.37:0/1395349429 >> [v2:172.21.15.203:6832/2418205237,v1:172.21.15.203:6833/2418205237] conn(0x7fa1100744e0 msgr2=0x7fa1100769f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:30:49.891 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:49.889+0000 7fa1354d9640 1 --2- 172.21.15.37:0/1395349429 >> [v2:172.21.15.203:6832/2418205237,v1:172.21.15.203:6833/2418205237] conn(0x7fa1100744e0 0x7fa1100769f0 secure :-1 s=READY pgs=285 cs=0 l=1 rev1=1 crypto rx=0x7fa128034000 tx=0x7fa12802e000 comp rx=0 tx=0).stop 2024-04-25T10:30:49.891 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:49.889+0000 7fa1354d9640 1 -- 172.21.15.37:0/1395349429 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fa13015a9f0 msgr2=0x7fa130159750 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:30:49.891 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:49.889+0000 7fa1354d9640 1 --2- 172.21.15.37:0/1395349429 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fa13015a9f0 0x7fa130159750 secure :-1 s=READY pgs=624 cs=0 l=1 rev1=1 crypto rx=0x7fa11800c460 tx=0x7fa11800c860 comp rx=0 tx=0).stop 2024-04-25T10:30:49.891 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:49.889+0000 7fa1354d9640 1 -- 172.21.15.37:0/1395349429 shutdown_connections 2024-04-25T10:30:49.891 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:49.889+0000 7fa1354d9640 1 --2- 172.21.15.37:0/1395349429 >> [v2:172.21.15.203:6832/2418205237,v1:172.21.15.203:6833/2418205237] conn(0x7fa1100744e0 0x7fa1100769f0 unknown :-1 s=CLOSED pgs=285 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:30:49.891 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:49.889+0000 7fa1354d9640 1 --2- 172.21.15.37:0/1395349429 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7fa130161020 0x7fa13015bf80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:30:49.891 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:49.889+0000 7fa1354d9640 1 --2- 172.21.15.37:0/1395349429 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fa13015a9f0 0x7fa130159750 unknown :-1 s=CLOSED pgs=624 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:30:49.891 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:49.889+0000 7fa1354d9640 1 --2- 172.21.15.37:0/1395349429 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fa13006ca80 0x7fa1300ce860 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:30:49.892 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:49.889+0000 7fa1354d9640 1 -- 172.21.15.37:0/1395349429 >> 172.21.15.37:0/1395349429 conn(0x7fa1300b4560 msgr2=0x7fa1300b24b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:30:49.892 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:49.889+0000 7fa1354d9640 1 -- 172.21.15.37:0/1395349429 shutdown_connections 2024-04-25T10:30:49.892 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:49.889+0000 7fa1354d9640 1 -- 172.21.15.37:0/1395349429 wait complete. 2024-04-25T10:30:49.902 DEBUG:teuthology.orchestra.run.smithi037:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-04-25T10:30:50.013 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:50.012+0000 7fa5197b2640 1 Processor -- start 2024-04-25T10:30:50.013 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:50.012+0000 7fa5197b2640 1 -- start start 2024-04-25T10:30:50.015 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:50.013+0000 7fa5197b2640 1 --2- >> v2:172.21.15.37:3300/0 conn(0x7fa5140d3230 0x7fa51416b0c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:30:50.015 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:50.013+0000 7fa5197b2640 1 -- --> v1:172.21.15.37:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fa5140d10a0 con 0x7fa5140b4e80 2024-04-25T10:30:50.015 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:50.013+0000 7fa5197b2640 1 -- --> v1:172.21.15.203:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fa5140d1370 con 0x7fa5140d5160 2024-04-25T10:30:50.015 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:50.013+0000 7fa5197b2640 1 -- --> v2:172.21.15.37:3300/0 -- mon_getmap magic: 0 -- 0x7fa5140d1650 con 0x7fa5140d3230 2024-04-25T10:30:50.015 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:50.013+0000 7fa5137fe640 1 --2- >> v2:172.21.15.37:3300/0 conn(0x7fa5140d3230 0x7fa51416b0c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:30:50.015 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:50.013+0000 7fa50bfff640 1 --1- >> v1:172.21.15.37:6789/0 conn(0x7fa5140b4e80 0x7fa5140d2b20 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.37:6789/0 says I am v1:172.21.15.37:38666/0 (socket says 172.21.15.37:38666) 2024-04-25T10:30:50.015 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:50.014+0000 7fa50bfff640 1 -- 172.21.15.37:0/893521511 learned_addr learned my addr 172.21.15.37:0/893521511 (peer_addr_for_me v1:172.21.15.37:0/0) 2024-04-25T10:30:50.015 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:50.014+0000 7fa5137fe640 1 -- 172.21.15.37:0/893521511 >> v1:172.21.15.203:6789/0 conn(0x7fa5140d5160 legacy=0x7fa5140d0990 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:30:50.015 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:50.014+0000 7fa5137fe640 1 -- 172.21.15.37:0/893521511 >> v1:172.21.15.37:6789/0 conn(0x7fa5140b4e80 legacy=0x7fa5140d2b20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:30:50.016 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:50.014+0000 7fa5137fe640 1 -- 172.21.15.37:0/893521511 --> v2:172.21.15.37:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fa5140d4680 con 0x7fa5140d3230 2024-04-25T10:30:50.017 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:50.014+0000 7fa5137fe640 1 --2- 172.21.15.37:0/893521511 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fa5140d3230 0x7fa51416b0c0 secure :-1 s=READY pgs=626 cs=0 l=1 rev1=1 crypto rx=0x7fa50c009d90 tx=0x7fa50c02a610 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ae2e5ee313f85679 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:30:50.017 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:50.015+0000 7fa5127fc640 1 -- 172.21.15.37:0/893521511 <== mon.0 v2:172.21.15.37:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa50c031600 con 0x7fa5140d3230 2024-04-25T10:30:50.017 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:50.015+0000 7fa5127fc640 1 -- 172.21.15.37:0/893521511 <== mon.0 v2:172.21.15.37:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fa50c0317a0 con 0x7fa5140d3230 2024-04-25T10:30:50.017 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:50.015+0000 7fa5127fc640 1 -- 172.21.15.37:0/893521511 <== mon.0 v2:172.21.15.37:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa50c031b70 con 0x7fa5140d3230 2024-04-25T10:30:50.017 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:50.015+0000 7fa5197b2640 1 -- 172.21.15.37:0/893521511 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fa5140d3230 msgr2=0x7fa51416b0c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:30:50.017 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:50.015+0000 7fa5197b2640 1 --2- 172.21.15.37:0/893521511 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fa5140d3230 0x7fa51416b0c0 secure :-1 s=READY pgs=626 cs=0 l=1 rev1=1 crypto rx=0x7fa50c009d90 tx=0x7fa50c02a610 comp rx=0 tx=0).stop 2024-04-25T10:30:50.017 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:50.015+0000 7fa5197b2640 1 -- 172.21.15.37:0/893521511 shutdown_connections 2024-04-25T10:30:50.017 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:50.015+0000 7fa5197b2640 1 --2- 172.21.15.37:0/893521511 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fa5140d3230 0x7fa51416b0c0 unknown :-1 s=CLOSED pgs=626 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:30:50.017 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:50.015+0000 7fa5197b2640 1 -- 172.21.15.37:0/893521511 >> 172.21.15.37:0/893521511 conn(0x7fa5140b7a80 msgr2=0x7fa5140b7e90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:30:50.017 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:50.015+0000 7fa5197b2640 1 -- 172.21.15.37:0/893521511 shutdown_connections 2024-04-25T10:30:50.017 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:50.015+0000 7fa5197b2640 1 -- 172.21.15.37:0/893521511 wait complete. 2024-04-25T10:30:50.018 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:50.017+0000 7fa5197b2640 1 Processor -- start 2024-04-25T10:30:50.018 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:50.017+0000 7fa5197b2640 1 -- start start 2024-04-25T10:30:50.021 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:50.018+0000 7fa5197b2640 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fa5140b4e80 0x7fa514154e20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:30:50.021 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:50.019+0000 7fa5197b2640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fa5140d5160 0x7fa514161f10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:30:50.021 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:50.019+0000 7fa5197b2640 1 --2- >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7fa514157da0 0x7fa514162450 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:30:50.021 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:50.019+0000 7fa5197b2640 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 -- 0x7fa5140d4c40 con 0x7fa5140d5160 2024-04-25T10:30:50.021 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:50.019+0000 7fa5197b2640 1 -- --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_getmap magic: 0 -- 0x7fa5140b5290 con 0x7fa5140b4e80 2024-04-25T10:30:50.021 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:50.019+0000 7fa5197b2640 1 -- --> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] -- mon_getmap magic: 0 -- 0x7fa514164ac0 con 0x7fa514157da0 2024-04-25T10:30:50.021 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:50.019+0000 7fa50bfff640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fa5140d5160 0x7fa514161f10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:30:50.021 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:50.019+0000 7fa50bfff640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fa5140d5160 0x7fa514161f10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.37:3300/0 says I am v2:172.21.15.37:59372/0 (socket says 172.21.15.37:59372) 2024-04-25T10:30:50.022 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:50.019+0000 7fa50bfff640 1 -- 172.21.15.37:0/4119111309 learned_addr learned my addr 172.21.15.37:0/4119111309 (peer_addr_for_me v2:172.21.15.37:0/0) 2024-04-25T10:30:50.022 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:50.019+0000 7fa5137fe640 1 --2- 172.21.15.37:0/4119111309 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7fa514157da0 0x7fa514162450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:30:50.022 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:50.019+0000 7fa512ffd640 1 --2- 172.21.15.37:0/4119111309 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fa5140b4e80 0x7fa514154e20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:30:50.022 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:50.019+0000 7fa50bfff640 1 -- 172.21.15.37:0/4119111309 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7fa514157da0 msgr2=0x7fa514162450 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:30:50.022 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:50.019+0000 7fa50bfff640 1 --2- 172.21.15.37:0/4119111309 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7fa514157da0 0x7fa514162450 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:30:50.022 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:50.019+0000 7fa50bfff640 1 -- 172.21.15.37:0/4119111309 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fa5140b4e80 msgr2=0x7fa514154e20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:30:50.022 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:50.019+0000 7fa50bfff640 1 --2- 172.21.15.37:0/4119111309 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fa5140b4e80 0x7fa514154e20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:30:50.022 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:50.019+0000 7fa50bfff640 1 -- 172.21.15.37:0/4119111309 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fa5140b3fc0 con 0x7fa5140d5160 2024-04-25T10:30:50.022 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:50.019+0000 7fa50bfff640 1 --2- 172.21.15.37:0/4119111309 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fa5140d5160 0x7fa514161f10 secure :-1 s=READY pgs=627 cs=0 l=1 rev1=1 crypto rx=0x7fa4fc00c3f0 tx=0x7fa4fc00c8b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:30:50.022 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:50.019+0000 7fa50b7fe640 1 -- 172.21.15.37:0/4119111309 <== mon.0 v2:172.21.15.37:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa4fc017070 con 0x7fa5140d5160 2024-04-25T10:30:50.022 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:50.019+0000 7fa50b7fe640 1 -- 172.21.15.37:0/4119111309 <== mon.0 v2:172.21.15.37:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fa4fc005ce0 con 0x7fa5140d5160 2024-04-25T10:30:50.022 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:50.020+0000 7fa5197b2640 1 -- 172.21.15.37:0/4119111309 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fa5140b4260 con 0x7fa5140d5160 2024-04-25T10:30:50.022 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:50.020+0000 7fa50b7fe640 1 -- 172.21.15.37:0/4119111309 <== mon.0 v2:172.21.15.37:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa4fc007310 con 0x7fa5140d5160 2024-04-25T10:30:50.023 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:50.020+0000 7fa5197b2640 1 -- 172.21.15.37:0/4119111309 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fa5141f68e0 con 0x7fa5140d5160 2024-04-25T10:30:50.023 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:50.020+0000 7fa5197b2640 1 -- 172.21.15.37:0/4119111309 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fa514157880 con 0x7fa5140d5160 2024-04-25T10:30:50.024 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:50.021+0000 7fa50b7fe640 1 -- 172.21.15.37:0/4119111309 <== mon.0 v2:172.21.15.37:3300/0 4 ==== mgrmap(e 5) ==== 96782+0+0 (secure 0 0 0) 0x7fa4fc01a070 con 0x7fa5140d5160 2024-04-25T10:30:50.024 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:50.022+0000 7fa50b7fe640 1 --2- 172.21.15.37:0/4119111309 >> [v2:172.21.15.203:6832/2418205237,v1:172.21.15.203:6833/2418205237] conn(0x7fa4f40744e0 0x7fa4f40769f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:30:50.024 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:50.022+0000 7fa50b7fe640 1 -- 172.21.15.37:0/4119111309 <== mon.0 v2:172.21.15.37:3300/0 5 ==== osd_map(46..46 src has 1..46) ==== 6364+0+0 (secure 0 0 0) 0x7fa4fc094760 con 0x7fa5140d5160 2024-04-25T10:30:50.024 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:50.022+0000 7fa512ffd640 1 --2- 172.21.15.37:0/4119111309 >> [v2:172.21.15.203:6832/2418205237,v1:172.21.15.203:6833/2418205237] conn(0x7fa4f40744e0 0x7fa4f40769f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:30:50.024 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:50.022+0000 7fa512ffd640 1 --2- 172.21.15.37:0/4119111309 >> [v2:172.21.15.203:6832/2418205237,v1:172.21.15.203:6833/2418205237] conn(0x7fa4f40744e0 0x7fa4f40769f0 secure :-1 s=READY pgs=286 cs=0 l=1 rev1=1 crypto rx=0x7fa504006fd0 tx=0x7fa504009060 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:30:50.027 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:50.026+0000 7fa50b7fe640 1 -- 172.21.15.37:0/4119111309 <== mon.0 v2:172.21.15.37:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187246 (secure 0 0 0) 0x7fa4fc05e610 con 0x7fa5140d5160 2024-04-25T10:30:50.215 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:50.214+0000 7fa5197b2640 1 -- 172.21.15.37:0/4119111309 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7fa51406a140 con 0x7fa5140d5160 2024-04-25T10:30:50.217 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:50.216+0000 7fa50b7fe640 1 -- 172.21.15.37:0/4119111309 <== mon.0 v2:172.21.15.37:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v46) ==== 74+0+18876 (secure 0 0 0) 0x7fa4fc092e20 con 0x7fa5140d5160 2024-04-25T10:30:50.217 INFO:teuthology.orchestra.run.smithi037.stdout: 2024-04-25T10:30:50.217 INFO:teuthology.orchestra.run.smithi037.stdout:{"epoch":46,"fsid":"f4acda28-be34-492a-b286-994afd36f61d","created":"2024-04-25T10:28:43.985718+0000","modified":"2024-04-25T10:30:49.793339+0000","last_up_change":"2024-04-25T10:28:54.900654+0000","last_in_change":"2024-04-25T10:28:46.216973+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":6,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"luminous","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-04-25T10:28:56.202153+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"19","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":4,"pool_name":"cephfs_metadata","create_time":"2024-04-25T10:30:10.008132+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"35","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.75,"score_stable":1.75,"optimal_score":1,"raw_score_acting":1.75,"raw_score_stable":1.75,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":5,"pool_name":"cephfs_data","create_time":"2024-04-25T10:30:11.084827+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"43","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"recovery_fs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.3799999952316284,"score_stable":1.3799999952316284,"optimal_score":1,"raw_score_acting":1.3799999952316284,"raw_score_stable":1.3799999952316284,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":6,"pool_name":"recovery_fs_metadata","create_time":"2024-04-25T10:30:26.903737+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"44","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"recovery_fs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"a2f79028-b332-4166-9ebb-a7024a37ea22","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":42,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.37:6816","nonce":2746074720},{"type":"v1","addr":"172.21.15.37:6817","nonce":2746074720}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.37:6818","nonce":2746074720},{"type":"v1","addr":"172.21.15.37:6819","nonce":2746074720}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.37:6822","nonce":2746074720},{"type":"v1","addr":"172.21.15.37:6823","nonce":2746074720}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.37:6820","nonce":2746074720},{"type":"v1","addr":"172.21.15.37:6821","nonce":2746074720}]},"public_addr":"172.21.15.37:6817/2746074720","cluster_addr":"172.21.15.37:6819/2746074720","heartbeat_back_addr":"172.21.15.37:6823/2746074720","heartbeat_front_addr":"172.21.15.37:6821/2746074720","state":["exists","up"]},{"osd":1,"uuid":"973159de-c2ea-4083-aecc-69373d1c1132","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":45,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.37:6800","nonce":1690341662},{"type":"v1","addr":"172.21.15.37:6801","nonce":1690341662}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.37:6802","nonce":1690341662},{"type":"v1","addr":"172.21.15.37:6803","nonce":1690341662}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.37:6806","nonce":1690341662},{"type":"v1","addr":"172.21.15.37:6807","nonce":1690341662}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.37:6804","nonce":1690341662},{"type":"v1","addr":"172.21.15.37:6805","nonce":1690341662}]},"public_addr":"172.21.15.37:6801/1690341662","cluster_addr":"172.21.15.37:6803/1690341662","heartbeat_back_addr":"172.21.15.37:6807/1690341662","heartbeat_front_addr":"172.21.15.37:6805/1690341662","state":["exists","up"]},{"osd":2,"uuid":"4ae7a0e9-cb3f-484c-a75f-f3766b721c47","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":42,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.37:6808","nonce":2754669649},{"type":"v1","addr":"172.21.15.37:6809","nonce":2754669649}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.37:6810","nonce":2754669649},{"type":"v1","addr":"172.21.15.37:6811","nonce":2754669649}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.37:6814","nonce":2754669649},{"type":"v1","addr":"172.21.15.37:6815","nonce":2754669649}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.37:6812","nonce":2754669649},{"type":"v1","addr":"172.21.15.37:6813","nonce":2754669649}]},"public_addr":"172.21.15.37:6809/2754669649","cluster_addr":"172.21.15.37:6811/2754669649","heartbeat_back_addr":"172.21.15.37:6815/2754669649","heartbeat_front_addr":"172.21.15.37:6813/2754669649","state":["exists","up"]},{"osd":3,"uuid":"1ba4a746-1608-47a7-ace2-960a5cfae934","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":45,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.37:6824","nonce":1815290618},{"type":"v1","addr":"172.21.15.37:6825","nonce":1815290618}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.37:6826","nonce":1815290618},{"type":"v1","addr":"172.21.15.37:6827","nonce":1815290618}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.37:6830","nonce":1815290618},{"type":"v1","addr":"172.21.15.37:6831","nonce":1815290618}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.37:6828","nonce":1815290618},{"type":"v1","addr":"172.21.15.37:6829","nonce":1815290618}]},"public_addr":"172.21.15.37:6825/1815290618","cluster_addr":"172.21.15.37:6827/1815290618","heartbeat_back_addr":"172.21.15.37:6831/1815290618","heartbeat_front_addr":"172.21.15.37:6829/1815290618","state":["exists","up"]},{"osd":4,"uuid":"d4f59f5d-0413-4b79-87ee-bb212c1fcf62","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":42,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6800","nonce":4051532063},{"type":"v1","addr":"172.21.15.203:6801","nonce":4051532063}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6802","nonce":4051532063},{"type":"v1","addr":"172.21.15.203:6803","nonce":4051532063}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6806","nonce":4051532063},{"type":"v1","addr":"172.21.15.203:6807","nonce":4051532063}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6804","nonce":4051532063},{"type":"v1","addr":"172.21.15.203:6805","nonce":4051532063}]},"public_addr":"172.21.15.203:6801/4051532063","cluster_addr":"172.21.15.203:6803/4051532063","heartbeat_back_addr":"172.21.15.203:6807/4051532063","heartbeat_front_addr":"172.21.15.203:6805/4051532063","state":["exists","up"]},{"osd":5,"uuid":"69710a0e-41d8-4e7f-9a10-17ce540b67be","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":45,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6824","nonce":1203822890},{"type":"v1","addr":"172.21.15.203:6825","nonce":1203822890}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6826","nonce":1203822890},{"type":"v1","addr":"172.21.15.203:6827","nonce":1203822890}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6830","nonce":1203822890},{"type":"v1","addr":"172.21.15.203:6831","nonce":1203822890}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6828","nonce":1203822890},{"type":"v1","addr":"172.21.15.203:6829","nonce":1203822890}]},"public_addr":"172.21.15.203:6825/1203822890","cluster_addr":"172.21.15.203:6827/1203822890","heartbeat_back_addr":"172.21.15.203:6831/1203822890","heartbeat_front_addr":"172.21.15.203:6829/1203822890","state":["exists","up"]},{"osd":6,"uuid":"f29dd047-3b48-473d-a235-6642a6293f20","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":42,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6810","nonce":2309615014},{"type":"v1","addr":"172.21.15.203:6812","nonce":2309615014}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6814","nonce":2309615014},{"type":"v1","addr":"172.21.15.203:6816","nonce":2309615014}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6822","nonce":2309615014},{"type":"v1","addr":"172.21.15.203:6823","nonce":2309615014}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6818","nonce":2309615014},{"type":"v1","addr":"172.21.15.203:6820","nonce":2309615014}]},"public_addr":"172.21.15.203:6812/2309615014","cluster_addr":"172.21.15.203:6816/2309615014","heartbeat_back_addr":"172.21.15.203:6823/2309615014","heartbeat_front_addr":"172.21.15.203:6820/2309615014","state":["exists","up"]},{"osd":7,"uuid":"6e691644-378b-4fdc-8e86-fb5f56e31f29","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":42,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6808","nonce":3688099014},{"type":"v1","addr":"172.21.15.203:6809","nonce":3688099014}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6811","nonce":3688099014},{"type":"v1","addr":"172.21.15.203:6813","nonce":3688099014}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6819","nonce":3688099014},{"type":"v1","addr":"172.21.15.203:6821","nonce":3688099014}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6815","nonce":3688099014},{"type":"v1","addr":"172.21.15.203:6817","nonce":3688099014}]},"public_addr":"172.21.15.203:6809/3688099014","cluster_addr":"172.21.15.203:6813/3688099014","heartbeat_back_addr":"172.21.15.203:6821/3688099014","heartbeat_front_addr":"172.21.15.203:6817/3688099014","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T10:28:51.419141+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T10:28:51.366448+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T10:28:51.348276+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T10:28:51.380519+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T10:28:51.105812+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T10:28:51.384496+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T10:28:51.388755+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T10:28:51.366958+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[{"pgid":"5.15","mappings":[{"from":4,"to":5}]},{"pgid":"5.1a","mappings":[{"from":4,"to":5}]},{"pgid":"5.26","mappings":[{"from":4,"to":5}]},{"pgid":"5.27","mappings":[{"from":4,"to":5}]}],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.37:6835/311552410":"2024-04-26T10:30:24.853732+0000","172.21.15.37:6832/311552410":"2024-04-26T10:30:24.853732+0000"},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-04-25T10:30:50.219 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:50.218+0000 7fa5197b2640 1 -- 172.21.15.37:0/4119111309 >> [v2:172.21.15.203:6832/2418205237,v1:172.21.15.203:6833/2418205237] conn(0x7fa4f40744e0 msgr2=0x7fa4f40769f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:30:50.220 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:50.218+0000 7fa5197b2640 1 --2- 172.21.15.37:0/4119111309 >> [v2:172.21.15.203:6832/2418205237,v1:172.21.15.203:6833/2418205237] conn(0x7fa4f40744e0 0x7fa4f40769f0 secure :-1 s=READY pgs=286 cs=0 l=1 rev1=1 crypto rx=0x7fa504006fd0 tx=0x7fa504009060 comp rx=0 tx=0).stop 2024-04-25T10:30:50.220 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:50.218+0000 7fa5197b2640 1 -- 172.21.15.37:0/4119111309 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fa5140d5160 msgr2=0x7fa514161f10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:30:50.220 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:50.218+0000 7fa5197b2640 1 --2- 172.21.15.37:0/4119111309 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fa5140d5160 0x7fa514161f10 secure :-1 s=READY pgs=627 cs=0 l=1 rev1=1 crypto rx=0x7fa4fc00c3f0 tx=0x7fa4fc00c8b0 comp rx=0 tx=0).stop 2024-04-25T10:30:50.220 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:50.218+0000 7fa5197b2640 1 -- 172.21.15.37:0/4119111309 shutdown_connections 2024-04-25T10:30:50.220 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:50.218+0000 7fa5197b2640 1 --2- 172.21.15.37:0/4119111309 >> [v2:172.21.15.203:6832/2418205237,v1:172.21.15.203:6833/2418205237] conn(0x7fa4f40744e0 0x7fa4f40769f0 unknown :-1 s=CLOSED pgs=286 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:30:50.220 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:50.218+0000 7fa5197b2640 1 --2- 172.21.15.37:0/4119111309 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7fa514157da0 0x7fa514162450 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:30:50.220 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:50.218+0000 7fa5197b2640 1 --2- 172.21.15.37:0/4119111309 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fa5140d5160 0x7fa514161f10 unknown :-1 s=CLOSED pgs=627 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:30:50.220 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:50.218+0000 7fa5197b2640 1 --2- 172.21.15.37:0/4119111309 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fa5140b4e80 0x7fa514154e20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:30:50.220 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:50.218+0000 7fa5197b2640 1 -- 172.21.15.37:0/4119111309 >> 172.21.15.37:0/4119111309 conn(0x7fa5140b7a80 msgr2=0x7fa5140b5fb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:30:50.220 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:50.218+0000 7fa5197b2640 1 -- 172.21.15.37:0/4119111309 shutdown_connections 2024-04-25T10:30:50.220 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:50.218+0000 7fa5197b2640 1 -- 172.21.15.37:0/4119111309 wait complete. 2024-04-25T10:30:50.231 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2024-04-25T10:28:56.202153+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '19', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 8, 'score_stable': 8, 'optimal_score': 0.25, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 4, 'pool_name': 'cephfs_metadata', 'create_time': '2024-04-25T10:30:10.008132+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '35', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_autoscale_bias': 4, 'pg_num_min': 16, 'recovery_priority': 5}, 'application_metadata': {'cephfs': {'metadata': 'cephfs'}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 1.75, 'score_stable': 1.75, 'optimal_score': 1, 'raw_score_acting': 1.75, 'raw_score_stable': 1.75, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 5, 'pool_name': 'cephfs_data', 'create_time': '2024-04-25T10:30:11.084827+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 64, 'pg_placement_num': 64, 'pg_placement_num_target': 64, 'pg_num_target': 64, 'pg_num_pending': 64, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '43', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.8999999761581421}, 'application_metadata': {'cephfs': {'data': 'recovery_fs'}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 1.3799999952316284, 'score_stable': 1.3799999952316284, 'optimal_score': 1, 'raw_score_acting': 1.3799999952316284, 'raw_score_stable': 1.3799999952316284, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 6, 'pool_name': 'recovery_fs_metadata', 'create_time': '2024-04-25T10:30:26.903737+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '44', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_autoscale_bias': 4, 'pg_num_min': 16, 'recovery_priority': 5}, 'application_metadata': {'cephfs': {'metadata': 'recovery_fs'}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 1.5, 'score_stable': 1.5, 'optimal_score': 1, 'raw_score_acting': 1.5, 'raw_score_stable': 1.5, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}] 2024-04-25T10:30:50.231 DEBUG:teuthology.orchestra.run.smithi037:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2024-04-25T10:30:50.347 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:50.346+0000 7f3fea6ee640 1 Processor -- start 2024-04-25T10:30:50.347 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:50.346+0000 7f3fea6ee640 1 -- start start 2024-04-25T10:30:50.348 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:50.347+0000 7f3fea6ee640 1 --2- >> v2:172.21.15.37:3300/0 conn(0x7f3fe40b4c90 0x7f3fe40d29e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:30:50.349 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:50.347+0000 7f3fea6ee640 1 -- --> v1:172.21.15.37:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f3fe409ce70 con 0x7f3fe406e260 2024-04-25T10:30:50.349 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:50.347+0000 7f3fea6ee640 1 -- --> v1:172.21.15.203:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f3fe409d150 con 0x7f3fe40d1ec0 2024-04-25T10:30:50.349 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:50.347+0000 7f3fea6ee640 1 -- --> v2:172.21.15.37:3300/0 -- mon_getmap magic: 0 -- 0x7f3fe40d3370 con 0x7f3fe40b4c90 2024-04-25T10:30:50.349 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:50.347+0000 7f3fe8c8b640 1 --2- >> v2:172.21.15.37:3300/0 conn(0x7f3fe40b4c90 0x7f3fe40d29e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:30:50.349 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:50.347+0000 7f3fe3fff640 1 --1- >> v1:172.21.15.37:6789/0 conn(0x7f3fe406e260 0x7f3fe406e670 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.37:6789/0 says I am v1:172.21.15.37:38674/0 (socket says 172.21.15.37:38674) 2024-04-25T10:30:50.349 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:50.347+0000 7f3fe3fff640 1 -- 172.21.15.37:0/3515550057 learned_addr learned my addr 172.21.15.37:0/3515550057 (peer_addr_for_me v1:172.21.15.37:0/0) 2024-04-25T10:30:50.349 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:50.347+0000 7f3fe8c8b640 1 -- 172.21.15.37:0/3515550057 >> v1:172.21.15.203:6789/0 conn(0x7f3fe40d1ec0 legacy=0x7f3fe40d22d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:30:50.349 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:50.347+0000 7f3fe8c8b640 1 -- 172.21.15.37:0/3515550057 >> v1:172.21.15.37:6789/0 conn(0x7f3fe406e260 legacy=0x7f3fe406e670 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:30:50.349 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:50.347+0000 7f3fe8c8b640 1 -- 172.21.15.37:0/3515550057 --> v2:172.21.15.37:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f3fe406ed80 con 0x7f3fe40b4c90 2024-04-25T10:30:50.351 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:50.348+0000 7f3fe8c8b640 1 --2- 172.21.15.37:0/3515550057 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f3fe40b4c90 0x7f3fe40d29e0 secure :-1 s=READY pgs=629 cs=0 l=1 rev1=1 crypto rx=0x7f3fdc009d90 tx=0x7f3fdc02a610 comp rx=0 tx=0).ready entity=mon.0 client_cookie=94a047ba7b0b8f68 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:30:50.351 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:50.348+0000 7f3fe2ffd640 1 -- 172.21.15.37:0/3515550057 <== mon.0 v2:172.21.15.37:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3fdc031600 con 0x7f3fe40b4c90 2024-04-25T10:30:50.351 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:50.348+0000 7f3fe2ffd640 1 -- 172.21.15.37:0/3515550057 <== mon.0 v2:172.21.15.37:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f3fdc0317a0 con 0x7f3fe40b4c90 2024-04-25T10:30:50.351 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:50.348+0000 7f3fe2ffd640 1 -- 172.21.15.37:0/3515550057 <== mon.0 v2:172.21.15.37:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3fdc031b70 con 0x7f3fe40b4c90 2024-04-25T10:30:50.351 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:50.349+0000 7f3fea6ee640 1 -- 172.21.15.37:0/3515550057 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f3fe40b4c90 msgr2=0x7f3fe40d29e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:30:50.351 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:50.349+0000 7f3fea6ee640 1 --2- 172.21.15.37:0/3515550057 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f3fe40b4c90 0x7f3fe40d29e0 secure :-1 s=READY pgs=629 cs=0 l=1 rev1=1 crypto rx=0x7f3fdc009d90 tx=0x7f3fdc02a610 comp rx=0 tx=0).stop 2024-04-25T10:30:50.351 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:50.349+0000 7f3fea6ee640 1 -- 172.21.15.37:0/3515550057 shutdown_connections 2024-04-25T10:30:50.351 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:50.349+0000 7f3fea6ee640 1 --2- 172.21.15.37:0/3515550057 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f3fe40b4c90 0x7f3fe40d29e0 unknown :-1 s=CLOSED pgs=629 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:30:50.351 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:50.349+0000 7f3fea6ee640 1 -- 172.21.15.37:0/3515550057 >> 172.21.15.37:0/3515550057 conn(0x7f3fe40d0390 msgr2=0x7f3fe40d07a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:30:50.352 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:50.349+0000 7f3fea6ee640 1 -- 172.21.15.37:0/3515550057 shutdown_connections 2024-04-25T10:30:50.352 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:50.349+0000 7f3fea6ee640 1 -- 172.21.15.37:0/3515550057 wait complete. 2024-04-25T10:30:50.352 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:50.351+0000 7f3fea6ee640 1 Processor -- start 2024-04-25T10:30:50.352 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:50.351+0000 7f3fea6ee640 1 -- start start 2024-04-25T10:30:50.355 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:50.352+0000 7f3fea6ee640 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f3fe406e260 0x7f3fe41d64e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:30:50.355 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:50.352+0000 7f3fea6ee640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f3fe40b4c90 0x7f3fe41f5cb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:30:50.355 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:50.352+0000 7f3fea6ee640 1 --2- >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f3fe40d1ec0 0x7f3fe40a0f70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:30:50.355 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:50.352+0000 7f3fea6ee640 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 -- 0x7f3fe406d830 con 0x7f3fe40b4c90 2024-04-25T10:30:50.355 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:50.352+0000 7f3fea6ee640 1 -- --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_getmap magic: 0 -- 0x7f3fe40d3090 con 0x7f3fe406e260 2024-04-25T10:30:50.355 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:50.352+0000 7f3fea6ee640 1 -- --> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] -- mon_getmap magic: 0 -- 0x7f3fe406db30 con 0x7f3fe40d1ec0 2024-04-25T10:30:50.355 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:50.352+0000 7f3fe37fe640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f3fe40b4c90 0x7f3fe41f5cb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:30:50.355 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:50.352+0000 7f3fe37fe640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f3fe40b4c90 0x7f3fe41f5cb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.37:3300/0 says I am v2:172.21.15.37:59388/0 (socket says 172.21.15.37:59388) 2024-04-25T10:30:50.355 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:50.352+0000 7f3fe37fe640 1 -- 172.21.15.37:0/2530106390 learned_addr learned my addr 172.21.15.37:0/2530106390 (peer_addr_for_me v2:172.21.15.37:0/0) 2024-04-25T10:30:50.355 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:50.352+0000 7f3fe3fff640 1 --2- 172.21.15.37:0/2530106390 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f3fe406e260 0x7f3fe41d64e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:30:50.355 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:50.352+0000 7f3fe8c8b640 1 --2- 172.21.15.37:0/2530106390 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f3fe40d1ec0 0x7f3fe40a0f70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:30:50.356 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:50.353+0000 7f3fe37fe640 1 -- 172.21.15.37:0/2530106390 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f3fe40d1ec0 msgr2=0x7f3fe40a0f70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:30:50.356 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:50.353+0000 7f3fe37fe640 1 --2- 172.21.15.37:0/2530106390 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f3fe40d1ec0 0x7f3fe40a0f70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:30:50.356 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:50.353+0000 7f3fe37fe640 1 -- 172.21.15.37:0/2530106390 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f3fe406e260 msgr2=0x7f3fe41d64e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:30:50.356 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:50.353+0000 7f3fe37fe640 1 --2- 172.21.15.37:0/2530106390 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f3fe406e260 0x7f3fe41d64e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:30:50.356 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:50.353+0000 7f3fe37fe640 1 -- 172.21.15.37:0/2530106390 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f3fe41dc8d0 con 0x7f3fe40b4c90 2024-04-25T10:30:50.356 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:50.353+0000 7f3fe37fe640 1 --2- 172.21.15.37:0/2530106390 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f3fe40b4c90 0x7f3fe41f5cb0 secure :-1 s=READY pgs=630 cs=0 l=1 rev1=1 crypto rx=0x7f3fd800b420 tx=0x7f3fd800b8e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:30:50.356 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:50.353+0000 7f3fe0ff9640 1 -- 172.21.15.37:0/2530106390 <== mon.0 v2:172.21.15.37:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3fd8011830 con 0x7f3fe40b4c90 2024-04-25T10:30:50.356 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:50.353+0000 7f3fe0ff9640 1 -- 172.21.15.37:0/2530106390 <== mon.0 v2:172.21.15.37:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f3fd8016070 con 0x7f3fe40b4c90 2024-04-25T10:30:50.356 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:50.353+0000 7f3fea6ee640 1 -- 172.21.15.37:0/2530106390 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f3fe40a14b0 con 0x7f3fe40b4c90 2024-04-25T10:30:50.356 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:50.353+0000 7f3fe0ff9640 1 -- 172.21.15.37:0/2530106390 <== mon.0 v2:172.21.15.37:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3fd8011c10 con 0x7f3fe40b4c90 2024-04-25T10:30:50.356 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:50.353+0000 7f3fea6ee640 1 -- 172.21.15.37:0/2530106390 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f3fe409d7b0 con 0x7f3fe40b4c90 2024-04-25T10:30:50.357 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:50.354+0000 7f3fea6ee640 1 -- 172.21.15.37:0/2530106390 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f3fe41ebd50 con 0x7f3fe40b4c90 2024-04-25T10:30:50.357 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:50.355+0000 7f3fe0ff9640 1 -- 172.21.15.37:0/2530106390 <== mon.0 v2:172.21.15.37:3300/0 4 ==== mgrmap(e 5) ==== 96782+0+0 (secure 0 0 0) 0x7f3fd801b460 con 0x7f3fe40b4c90 2024-04-25T10:30:50.358 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:50.355+0000 7f3fe0ff9640 1 --2- 172.21.15.37:0/2530106390 >> [v2:172.21.15.203:6832/2418205237,v1:172.21.15.203:6833/2418205237] conn(0x7f3fd00744e0 0x7f3fd00769f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:30:50.358 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:50.355+0000 7f3fe0ff9640 1 -- 172.21.15.37:0/2530106390 <== mon.0 v2:172.21.15.37:3300/0 5 ==== osd_map(46..46 src has 1..46) ==== 6364+0+0 (secure 0 0 0) 0x7f3fd8094df0 con 0x7f3fe40b4c90 2024-04-25T10:30:50.358 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:50.355+0000 7f3fe3fff640 1 --2- 172.21.15.37:0/2530106390 >> [v2:172.21.15.203:6832/2418205237,v1:172.21.15.203:6833/2418205237] conn(0x7f3fd00744e0 0x7f3fd00769f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:30:50.358 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:50.356+0000 7f3fe3fff640 1 --2- 172.21.15.37:0/2530106390 >> [v2:172.21.15.203:6832/2418205237,v1:172.21.15.203:6833/2418205237] conn(0x7f3fd00744e0 0x7f3fd00769f0 secure :-1 s=READY pgs=287 cs=0 l=1 rev1=1 crypto rx=0x7f3fd4007fd0 tx=0x7f3fd4007e80 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:30:50.361 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:50.360+0000 7f3fe0ff9640 1 -- 172.21.15.37:0/2530106390 <== mon.0 v2:172.21.15.37:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187246 (secure 0 0 0) 0x7f3fd8020020 con 0x7f3fe40b4c90 2024-04-25T10:30:50.548 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:50.547+0000 7f3fea6ee640 1 -- 172.21.15.37:0/2530106390 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) -- 0x7f3fe406a140 con 0x7f3fe40b4c90 2024-04-25T10:30:50.549 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:50.548+0000 7f3fe0ff9640 1 -- 172.21.15.37:0/2530106390 <== mon.0 v2:172.21.15.37:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v46) ==== 93+0+10 (secure 0 0 0) 0x7f3fe41dc8d0 con 0x7f3fe40b4c90 2024-04-25T10:30:50.549 INFO:teuthology.orchestra.run.smithi037.stdout:pg_num: 1 2024-04-25T10:30:50.551 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:50.550+0000 7f3fea6ee640 1 -- 172.21.15.37:0/2530106390 >> [v2:172.21.15.203:6832/2418205237,v1:172.21.15.203:6833/2418205237] conn(0x7f3fd00744e0 msgr2=0x7f3fd00769f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:30:50.551 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:50.550+0000 7f3fea6ee640 1 --2- 172.21.15.37:0/2530106390 >> [v2:172.21.15.203:6832/2418205237,v1:172.21.15.203:6833/2418205237] conn(0x7f3fd00744e0 0x7f3fd00769f0 secure :-1 s=READY pgs=287 cs=0 l=1 rev1=1 crypto rx=0x7f3fd4007fd0 tx=0x7f3fd4007e80 comp rx=0 tx=0).stop 2024-04-25T10:30:50.551 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:50.550+0000 7f3fea6ee640 1 -- 172.21.15.37:0/2530106390 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f3fe40b4c90 msgr2=0x7f3fe41f5cb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:30:50.552 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:50.550+0000 7f3fea6ee640 1 --2- 172.21.15.37:0/2530106390 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f3fe40b4c90 0x7f3fe41f5cb0 secure :-1 s=READY pgs=630 cs=0 l=1 rev1=1 crypto rx=0x7f3fd800b420 tx=0x7f3fd800b8e0 comp rx=0 tx=0).stop 2024-04-25T10:30:50.552 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:50.550+0000 7f3fea6ee640 1 -- 172.21.15.37:0/2530106390 shutdown_connections 2024-04-25T10:30:50.552 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:50.550+0000 7f3fea6ee640 1 --2- 172.21.15.37:0/2530106390 >> [v2:172.21.15.203:6832/2418205237,v1:172.21.15.203:6833/2418205237] conn(0x7f3fd00744e0 0x7f3fd00769f0 unknown :-1 s=CLOSED pgs=287 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:30:50.552 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:50.550+0000 7f3fea6ee640 1 --2- 172.21.15.37:0/2530106390 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f3fe40d1ec0 0x7f3fe40a0f70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:30:50.552 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:50.550+0000 7f3fea6ee640 1 --2- 172.21.15.37:0/2530106390 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f3fe40b4c90 0x7f3fe41f5cb0 unknown :-1 s=CLOSED pgs=630 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:30:50.552 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:50.550+0000 7f3fea6ee640 1 --2- 172.21.15.37:0/2530106390 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f3fe406e260 0x7f3fe41d64e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:30:50.552 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:50.550+0000 7f3fea6ee640 1 -- 172.21.15.37:0/2530106390 >> 172.21.15.37:0/2530106390 conn(0x7f3fe40d0390 msgr2=0x7f3fe40b8770 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:30:50.552 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:50.550+0000 7f3fea6ee640 1 -- 172.21.15.37:0/2530106390 shutdown_connections 2024-04-25T10:30:50.552 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:50.550+0000 7f3fea6ee640 1 -- 172.21.15.37:0/2530106390 wait complete. 2024-04-25T10:30:50.562 DEBUG:teuthology.orchestra.run.smithi037:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_metadata pg_num 2024-04-25T10:30:50.677 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:50.676+0000 7f8f1a88c640 1 Processor -- start 2024-04-25T10:30:50.677 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:50.676+0000 7f8f1a88c640 1 -- start start 2024-04-25T10:30:50.679 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:50.677+0000 7f8f1a88c640 1 --2- >> v2:172.21.15.37:3300/0 conn(0x7f8f140d2860 0x7f8f1406e200 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:30:50.679 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:50.677+0000 7f8f1a88c640 1 -- --> v1:172.21.15.37:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f8f140d2e40 con 0x7f8f140ce560 2024-04-25T10:30:50.679 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:50.677+0000 7f8f1a88c640 1 -- --> v1:172.21.15.203:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f8f140d3120 con 0x7f8f140b2dc0 2024-04-25T10:30:50.679 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:50.677+0000 7f8f1a88c640 1 -- --> v2:172.21.15.37:3300/0 -- mon_getmap magic: 0 -- 0x7f8f140b92c0 con 0x7f8f140d2860 2024-04-25T10:30:50.679 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:50.677+0000 7f8f137fe640 1 --2- >> v2:172.21.15.37:3300/0 conn(0x7f8f140d2860 0x7f8f1406e200 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:30:50.679 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:50.677+0000 7f8f18e29640 1 --1- >> v1:172.21.15.37:6789/0 conn(0x7f8f140ce560 0x7f8f140ce970 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.37:6789/0 says I am v1:172.21.15.37:38684/0 (socket says 172.21.15.37:38684) 2024-04-25T10:30:50.679 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:50.677+0000 7f8f18e29640 1 -- 172.21.15.37:0/966115232 learned_addr learned my addr 172.21.15.37:0/966115232 (peer_addr_for_me v1:172.21.15.37:0/0) 2024-04-25T10:30:50.679 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:50.677+0000 7f8f137fe640 1 -- 172.21.15.37:0/966115232 >> v1:172.21.15.203:6789/0 conn(0x7f8f140b2dc0 legacy=0x7f8f1406daf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:30:50.679 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:50.677+0000 7f8f137fe640 1 -- 172.21.15.37:0/966115232 >> v1:172.21.15.37:6789/0 conn(0x7f8f140ce560 legacy=0x7f8f140ce970 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:30:50.679 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:50.677+0000 7f8f12ffd640 1 -- 172.21.15.37:0/966115232 <== mon.0 v1:172.21.15.37:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 3193828504 0 0) 0x7f8f140d2e40 con 0x7f8f140ce560 2024-04-25T10:30:50.679 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:50.678+0000 7f8f137fe640 1 -- 172.21.15.37:0/966115232 --> v2:172.21.15.37:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f8f140d3400 con 0x7f8f140d2860 2024-04-25T10:30:50.681 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:50.678+0000 7f8f137fe640 1 --2- 172.21.15.37:0/966115232 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f8f140d2860 0x7f8f1406e200 secure :-1 s=READY pgs=632 cs=0 l=1 rev1=1 crypto rx=0x7f8f080097a0 tx=0x7f8f0802a1e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=fc3169c62f6cfbb4 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:30:50.681 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:50.678+0000 7f8f12ffd640 1 -- 172.21.15.37:0/966115232 <== mon.0 v2:172.21.15.37:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8f08037070 con 0x7f8f140d2860 2024-04-25T10:30:50.681 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:50.678+0000 7f8f12ffd640 1 -- 172.21.15.37:0/966115232 <== mon.0 v2:172.21.15.37:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f8f08032440 con 0x7f8f140d2860 2024-04-25T10:30:50.681 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:50.678+0000 7f8f12ffd640 1 -- 172.21.15.37:0/966115232 <== mon.0 v2:172.21.15.37:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8f08032810 con 0x7f8f140d2860 2024-04-25T10:30:50.681 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:50.679+0000 7f8f1a88c640 1 -- 172.21.15.37:0/966115232 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f8f140d2860 msgr2=0x7f8f1406e200 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:30:50.681 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:50.679+0000 7f8f1a88c640 1 --2- 172.21.15.37:0/966115232 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f8f140d2860 0x7f8f1406e200 secure :-1 s=READY pgs=632 cs=0 l=1 rev1=1 crypto rx=0x7f8f080097a0 tx=0x7f8f0802a1e0 comp rx=0 tx=0).stop 2024-04-25T10:30:50.681 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:50.679+0000 7f8f1a88c640 1 -- 172.21.15.37:0/966115232 shutdown_connections 2024-04-25T10:30:50.681 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:50.679+0000 7f8f1a88c640 1 --2- 172.21.15.37:0/966115232 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f8f140d2860 0x7f8f1406e200 unknown :-1 s=CLOSED pgs=632 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:30:50.681 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:50.679+0000 7f8f1a88c640 1 -- 172.21.15.37:0/966115232 >> 172.21.15.37:0/966115232 conn(0x7f8f140b5440 msgr2=0x7f8f140b5850 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:30:50.681 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:50.679+0000 7f8f1a88c640 1 -- 172.21.15.37:0/966115232 shutdown_connections 2024-04-25T10:30:50.681 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:50.679+0000 7f8f1a88c640 1 -- 172.21.15.37:0/966115232 wait complete. 2024-04-25T10:30:50.682 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:50.681+0000 7f8f1a88c640 1 Processor -- start 2024-04-25T10:30:50.682 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:50.681+0000 7f8f1a88c640 1 -- start start 2024-04-25T10:30:50.685 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:50.682+0000 7f8f1a88c640 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f8f140b2dc0 0x7f8f14154f00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:30:50.685 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:50.682+0000 7f8f1a88c640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f8f140ce560 0x7f8f140c73b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:30:50.685 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:50.682+0000 7f8f1a88c640 1 --2- >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f8f140d2860 0x7f8f140c9730 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:30:50.685 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:50.682+0000 7f8f1a88c640 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 -- 0x7f8f1406e740 con 0x7f8f140ce560 2024-04-25T10:30:50.685 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:50.682+0000 7f8f1a88c640 1 -- --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_getmap magic: 0 -- 0x7f8f140ce0d0 con 0x7f8f140b2dc0 2024-04-25T10:30:50.685 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:50.682+0000 7f8f1a88c640 1 -- --> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] -- mon_getmap magic: 0 -- 0x7f8f1406e8c0 con 0x7f8f140d2860 2024-04-25T10:30:50.685 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:50.682+0000 7f8f137fe640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f8f140ce560 0x7f8f140c73b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:30:50.685 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:50.682+0000 7f8f137fe640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f8f140ce560 0x7f8f140c73b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.37:3300/0 says I am v2:172.21.15.37:59416/0 (socket says 172.21.15.37:59416) 2024-04-25T10:30:50.686 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:50.682+0000 7f8f137fe640 1 -- 172.21.15.37:0/2952495096 learned_addr learned my addr 172.21.15.37:0/2952495096 (peer_addr_for_me v2:172.21.15.37:0/0) 2024-04-25T10:30:50.686 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:50.683+0000 7f8f18e29640 1 --2- 172.21.15.37:0/2952495096 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f8f140d2860 0x7f8f140c9730 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:30:50.686 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:50.683+0000 7f8f137fe640 1 -- 172.21.15.37:0/2952495096 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f8f140d2860 msgr2=0x7f8f140c9730 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:30:50.686 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:50.683+0000 7f8f13fff640 1 --2- 172.21.15.37:0/2952495096 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f8f140b2dc0 0x7f8f14154f00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:30:50.686 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:50.683+0000 7f8f137fe640 1 --2- 172.21.15.37:0/2952495096 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f8f140d2860 0x7f8f140c9730 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:30:50.686 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:50.683+0000 7f8f137fe640 1 -- 172.21.15.37:0/2952495096 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f8f140b2dc0 msgr2=0x7f8f14154f00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:30:50.686 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:50.683+0000 7f8f137fe640 1 --2- 172.21.15.37:0/2952495096 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f8f140b2dc0 0x7f8f14154f00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:30:50.686 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:50.683+0000 7f8f137fe640 1 -- 172.21.15.37:0/2952495096 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f8f14161450 con 0x7f8f140ce560 2024-04-25T10:30:50.686 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:50.683+0000 7f8f18e29640 1 --2- 172.21.15.37:0/2952495096 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f8f140d2860 0x7f8f140c9730 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T10:30:50.686 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:50.683+0000 7f8f137fe640 1 --2- 172.21.15.37:0/2952495096 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f8f140ce560 0x7f8f140c73b0 secure :-1 s=READY pgs=633 cs=0 l=1 rev1=1 crypto rx=0x7f8f080097a0 tx=0x7f8f080046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:30:50.686 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:50.683+0000 7f8f10ff9640 1 -- 172.21.15.37:0/2952495096 <== mon.0 v2:172.21.15.37:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8f08041070 con 0x7f8f140ce560 2024-04-25T10:30:50.686 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:50.683+0000 7f8f10ff9640 1 -- 172.21.15.37:0/2952495096 <== mon.0 v2:172.21.15.37:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f8f0803c870 con 0x7f8f140ce560 2024-04-25T10:30:50.687 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:50.683+0000 7f8f1a88c640 1 -- 172.21.15.37:0/2952495096 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f8f14161730 con 0x7f8f140ce560 2024-04-25T10:30:50.687 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:50.683+0000 7f8f10ff9640 1 -- 172.21.15.37:0/2952495096 <== mon.0 v2:172.21.15.37:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8f08037040 con 0x7f8f140ce560 2024-04-25T10:30:50.687 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:50.683+0000 7f8f1a88c640 1 -- 172.21.15.37:0/2952495096 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f8f14161cf0 con 0x7f8f140ce560 2024-04-25T10:30:50.688 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:50.684+0000 7f8f1a88c640 1 -- 172.21.15.37:0/2952495096 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f8f1406a140 con 0x7f8f140ce560 2024-04-25T10:30:50.688 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:50.685+0000 7f8f10ff9640 1 -- 172.21.15.37:0/2952495096 <== mon.0 v2:172.21.15.37:3300/0 4 ==== mgrmap(e 5) ==== 96782+0+0 (secure 0 0 0) 0x7f8f08044070 con 0x7f8f140ce560 2024-04-25T10:30:50.688 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:50.686+0000 7f8f10ff9640 1 --2- 172.21.15.37:0/2952495096 >> [v2:172.21.15.203:6832/2418205237,v1:172.21.15.203:6833/2418205237] conn(0x7f8f000744b0 0x7f8f000769c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:30:50.688 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:50.686+0000 7f8f10ff9640 1 -- 172.21.15.37:0/2952495096 <== mon.0 v2:172.21.15.37:3300/0 5 ==== osd_map(46..46 src has 1..46) ==== 6364+0+0 (secure 0 0 0) 0x7f8f080b5d40 con 0x7f8f140ce560 2024-04-25T10:30:50.688 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:50.686+0000 7f8f13fff640 1 --2- 172.21.15.37:0/2952495096 >> [v2:172.21.15.203:6832/2418205237,v1:172.21.15.203:6833/2418205237] conn(0x7f8f000744b0 0x7f8f000769c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:30:50.688 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:50.686+0000 7f8f13fff640 1 --2- 172.21.15.37:0/2952495096 >> [v2:172.21.15.203:6832/2418205237,v1:172.21.15.203:6833/2418205237] conn(0x7f8f000744b0 0x7f8f000769c0 secure :-1 s=READY pgs=288 cs=0 l=1 rev1=1 crypto rx=0x7f8f14074cc0 tx=0x7f8f04008040 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:30:50.691 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:50.690+0000 7f8f10ff9640 1 -- 172.21.15.37:0/2952495096 <== mon.0 v2:172.21.15.37:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187246 (secure 0 0 0) 0x7f8f080802c0 con 0x7f8f140ce560 2024-04-25T10:30:50.876 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:50.875+0000 7f8f1a88c640 1 -- 172.21.15.37:0/2952495096 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"} v 0) -- 0x7f8f1406d7b0 con 0x7f8f140ce560 2024-04-25T10:30:50.877 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:50.876+0000 7f8f10ff9640 1 -- 172.21.15.37:0/2952495096 <== mon.0 v2:172.21.15.37:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]=0 v46) ==== 104+0+11 (secure 0 0 0) 0x7f8f0807fbc0 con 0x7f8f140ce560 2024-04-25T10:30:50.877 INFO:teuthology.orchestra.run.smithi037.stdout:pg_num: 32 2024-04-25T10:30:50.879 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:50.877+0000 7f8f1a88c640 1 -- 172.21.15.37:0/2952495096 >> [v2:172.21.15.203:6832/2418205237,v1:172.21.15.203:6833/2418205237] conn(0x7f8f000744b0 msgr2=0x7f8f000769c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:30:50.879 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:50.877+0000 7f8f1a88c640 1 --2- 172.21.15.37:0/2952495096 >> [v2:172.21.15.203:6832/2418205237,v1:172.21.15.203:6833/2418205237] conn(0x7f8f000744b0 0x7f8f000769c0 secure :-1 s=READY pgs=288 cs=0 l=1 rev1=1 crypto rx=0x7f8f14074cc0 tx=0x7f8f04008040 comp rx=0 tx=0).stop 2024-04-25T10:30:50.879 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:50.877+0000 7f8f1a88c640 1 -- 172.21.15.37:0/2952495096 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f8f140ce560 msgr2=0x7f8f140c73b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:30:50.879 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:50.877+0000 7f8f1a88c640 1 --2- 172.21.15.37:0/2952495096 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f8f140ce560 0x7f8f140c73b0 secure :-1 s=READY pgs=633 cs=0 l=1 rev1=1 crypto rx=0x7f8f080097a0 tx=0x7f8f080046c0 comp rx=0 tx=0).stop 2024-04-25T10:30:50.879 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:50.877+0000 7f8f1a88c640 1 -- 172.21.15.37:0/2952495096 shutdown_connections 2024-04-25T10:30:50.879 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:50.877+0000 7f8f1a88c640 1 --2- 172.21.15.37:0/2952495096 >> [v2:172.21.15.203:6832/2418205237,v1:172.21.15.203:6833/2418205237] conn(0x7f8f000744b0 0x7f8f000769c0 unknown :-1 s=CLOSED pgs=288 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:30:50.879 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:50.878+0000 7f8f1a88c640 1 --2- 172.21.15.37:0/2952495096 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f8f140d2860 0x7f8f140c9730 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:30:50.880 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:50.878+0000 7f8f1a88c640 1 --2- 172.21.15.37:0/2952495096 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f8f140ce560 0x7f8f140c73b0 unknown :-1 s=CLOSED pgs=633 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:30:50.880 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:50.878+0000 7f8f1a88c640 1 --2- 172.21.15.37:0/2952495096 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f8f140b2dc0 0x7f8f14154f00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:30:50.880 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:50.878+0000 7f8f1a88c640 1 -- 172.21.15.37:0/2952495096 >> 172.21.15.37:0/2952495096 conn(0x7f8f140b5440 msgr2=0x7f8f14071fc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:30:50.880 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:50.878+0000 7f8f1a88c640 1 -- 172.21.15.37:0/2952495096 shutdown_connections 2024-04-25T10:30:50.880 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:50.878+0000 7f8f1a88c640 1 -- 172.21.15.37:0/2952495096 wait complete. 2024-04-25T10:30:50.890 DEBUG:teuthology.orchestra.run.smithi037:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data pg_num 2024-04-25T10:30:51.004 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:51.003+0000 7ff3f2884640 1 Processor -- start 2024-04-25T10:30:51.004 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:51.003+0000 7ff3f2884640 1 -- start start 2024-04-25T10:30:51.006 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:51.004+0000 7ff3f2884640 1 --2- >> v2:172.21.15.37:3300/0 conn(0x7ff3ec0cf390 0x7ff3ec06f990 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:30:51.006 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:51.004+0000 7ff3f2884640 1 --2- >> v2:172.21.15.203:3300/0 conn(0x7ff3ec06fed0 0x7ff3ec0b49b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:30:51.006 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:51.004+0000 7ff3f2884640 1 -- --> v1:172.21.15.203:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7ff3ec0702e0 con 0x7ff3ec156c30 2024-04-25T10:30:51.006 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:51.004+0000 7ff3f2884640 1 -- --> v2:172.21.15.37:3300/0 -- mon_getmap magic: 0 -- 0x7ff3ec0b6f00 con 0x7ff3ec0cf390 2024-04-25T10:30:51.006 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:51.004+0000 7ff3f2884640 1 -- --> v2:172.21.15.203:3300/0 -- mon_getmap magic: 0 -- 0x7ff3ec15d850 con 0x7ff3ec06fed0 2024-04-25T10:30:51.006 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:51.004+0000 7ff3eb7fe640 1 --2- >> v2:172.21.15.37:3300/0 conn(0x7ff3ec0cf390 0x7ff3ec06f990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:30:51.006 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:51.004+0000 7ff3eb7fe640 1 --2- >> v2:172.21.15.37:3300/0 conn(0x7ff3ec0cf390 0x7ff3ec06f990 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.37:3300/0 says I am v2:172.21.15.37:59418/0 (socket says 172.21.15.37:59418) 2024-04-25T10:30:51.006 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:51.004+0000 7ff3eb7fe640 1 -- 172.21.15.37:0/849258631 learned_addr learned my addr 172.21.15.37:0/849258631 (peer_addr_for_me v2:172.21.15.37:0/0) 2024-04-25T10:30:51.006 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:51.004+0000 7ff3f0e21640 1 --2- 172.21.15.37:0/849258631 >> v2:172.21.15.203:3300/0 conn(0x7ff3ec06fed0 0x7ff3ec0b49b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:30:51.006 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:51.004+0000 7ff3eb7fe640 1 -- 172.21.15.37:0/849258631 >> v2:172.21.15.203:3300/0 conn(0x7ff3ec06fed0 msgr2=0x7ff3ec0b49b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:30:51.007 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:51.004+0000 7ff3eb7fe640 1 --2- 172.21.15.37:0/849258631 >> v2:172.21.15.203:3300/0 conn(0x7ff3ec06fed0 0x7ff3ec0b49b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:30:51.007 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:51.004+0000 7ff3eb7fe640 1 -- 172.21.15.37:0/849258631 >> v1:172.21.15.203:6789/0 conn(0x7ff3ec156c30 legacy=0x7ff3ec0cec80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:30:51.007 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:51.005+0000 7ff3eb7fe640 1 -- 172.21.15.37:0/849258631 --> v2:172.21.15.37:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ff3ec15cf20 con 0x7ff3ec0cf390 2024-04-25T10:30:51.007 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:51.005+0000 7ff3f0e21640 1 --2- 172.21.15.37:0/849258631 >> v2:172.21.15.203:3300/0 conn(0x7ff3ec06fed0 0x7ff3ec0b49b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T10:30:51.008 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:51.005+0000 7ff3eb7fe640 1 --2- 172.21.15.37:0/849258631 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7ff3ec0cf390 0x7ff3ec06f990 secure :-1 s=READY pgs=634 cs=0 l=1 rev1=1 crypto rx=0x7ff3ec071300 tx=0x7ff3d402a230 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ccc9f8660c816209 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:30:51.008 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:51.005+0000 7ff3eaffd640 1 -- 172.21.15.37:0/849258631 <== mon.0 v2:172.21.15.37:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff3d4037070 con 0x7ff3ec0cf390 2024-04-25T10:30:51.008 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:51.005+0000 7ff3eaffd640 1 -- 172.21.15.37:0/849258631 <== mon.0 v2:172.21.15.37:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7ff3d4032440 con 0x7ff3ec0cf390 2024-04-25T10:30:51.008 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:51.005+0000 7ff3eaffd640 1 -- 172.21.15.37:0/849258631 <== mon.0 v2:172.21.15.37:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff3d4032810 con 0x7ff3ec0cf390 2024-04-25T10:30:51.008 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:51.006+0000 7ff3f2884640 1 -- 172.21.15.37:0/849258631 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7ff3ec0cf390 msgr2=0x7ff3ec06f990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:30:51.008 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:51.006+0000 7ff3f2884640 1 --2- 172.21.15.37:0/849258631 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7ff3ec0cf390 0x7ff3ec06f990 secure :-1 s=READY pgs=634 cs=0 l=1 rev1=1 crypto rx=0x7ff3ec071300 tx=0x7ff3d402a230 comp rx=0 tx=0).stop 2024-04-25T10:30:51.008 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:51.006+0000 7ff3f2884640 1 -- 172.21.15.37:0/849258631 shutdown_connections 2024-04-25T10:30:51.009 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:51.006+0000 7ff3f2884640 1 --2- 172.21.15.37:0/849258631 >> v2:172.21.15.203:3300/0 conn(0x7ff3ec06fed0 0x7ff3ec0b49b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:30:51.009 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:51.006+0000 7ff3f2884640 1 --2- 172.21.15.37:0/849258631 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7ff3ec0cf390 0x7ff3ec06f990 unknown :-1 s=CLOSED pgs=634 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:30:51.009 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:51.006+0000 7ff3f2884640 1 -- 172.21.15.37:0/849258631 >> 172.21.15.37:0/849258631 conn(0x7ff3ec0b36b0 msgr2=0x7ff3ec0b3ac0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:30:51.009 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:51.006+0000 7ff3f2884640 1 -- 172.21.15.37:0/849258631 shutdown_connections 2024-04-25T10:30:51.009 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:51.006+0000 7ff3f2884640 1 -- 172.21.15.37:0/849258631 wait complete. 2024-04-25T10:30:51.009 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:51.008+0000 7ff3f2884640 1 Processor -- start 2024-04-25T10:30:51.010 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:51.008+0000 7ff3f2884640 1 -- start start 2024-04-25T10:30:51.012 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:51.009+0000 7ff3f2884640 1 --2- >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7ff3ec06fed0 0x7ff3ec1f41c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:30:51.012 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:51.009+0000 7ff3f2884640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7ff3ec0cf390 0x7ff3ec1f17c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:30:51.012 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:51.009+0000 7ff3f2884640 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7ff3ec156c30 0x7ff3ec14da80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:30:51.012 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:51.009+0000 7ff3f2884640 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 -- 0x7ff3ec157040 con 0x7ff3ec0cf390 2024-04-25T10:30:51.012 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:51.009+0000 7ff3f2884640 1 -- --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_getmap magic: 0 -- 0x7ff3ec15d4e0 con 0x7ff3ec156c30 2024-04-25T10:30:51.012 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:51.009+0000 7ff3f2884640 1 -- --> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] -- mon_getmap magic: 0 -- 0x7ff3ec15cbd0 con 0x7ff3ec06fed0 2024-04-25T10:30:51.012 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:51.009+0000 7ff3eb7fe640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7ff3ec0cf390 0x7ff3ec1f17c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:30:51.012 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:51.010+0000 7ff3eb7fe640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7ff3ec0cf390 0x7ff3ec1f17c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.37:3300/0 says I am v2:172.21.15.37:59432/0 (socket says 172.21.15.37:59432) 2024-04-25T10:30:51.012 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:51.010+0000 7ff3eb7fe640 1 -- 172.21.15.37:0/2131285595 learned_addr learned my addr 172.21.15.37:0/2131285595 (peer_addr_for_me v2:172.21.15.37:0/0) 2024-04-25T10:30:51.012 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:51.010+0000 7ff3ebfff640 1 --2- 172.21.15.37:0/2131285595 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7ff3ec06fed0 0x7ff3ec1f41c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:30:51.012 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:51.010+0000 7ff3f0e21640 1 --2- 172.21.15.37:0/2131285595 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7ff3ec156c30 0x7ff3ec14da80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:30:51.012 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:51.010+0000 7ff3eb7fe640 1 -- 172.21.15.37:0/2131285595 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7ff3ec06fed0 msgr2=0x7ff3ec1f41c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:30:51.013 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:51.010+0000 7ff3eb7fe640 1 --2- 172.21.15.37:0/2131285595 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7ff3ec06fed0 0x7ff3ec1f41c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:30:51.013 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:51.010+0000 7ff3eb7fe640 1 -- 172.21.15.37:0/2131285595 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7ff3ec156c30 msgr2=0x7ff3ec14da80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:30:51.013 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:51.010+0000 7ff3eb7fe640 1 --2- 172.21.15.37:0/2131285595 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7ff3ec156c30 0x7ff3ec14da80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:30:51.013 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:51.010+0000 7ff3eb7fe640 1 -- 172.21.15.37:0/2131285595 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ff3ec1f4a00 con 0x7ff3ec0cf390 2024-04-25T10:30:51.013 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:51.010+0000 7ff3eb7fe640 1 --2- 172.21.15.37:0/2131285595 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7ff3ec0cf390 0x7ff3ec1f17c0 secure :-1 s=READY pgs=635 cs=0 l=1 rev1=1 crypto rx=0x7ff3d4000c00 tx=0x7ff3d40046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:30:51.013 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:51.010+0000 7ff3e8ff9640 1 -- 172.21.15.37:0/2131285595 <== mon.0 v2:172.21.15.37:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff3d4041070 con 0x7ff3ec0cf390 2024-04-25T10:30:51.013 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:51.010+0000 7ff3e8ff9640 1 -- 172.21.15.37:0/2131285595 <== mon.0 v2:172.21.15.37:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7ff3d403c870 con 0x7ff3ec0cf390 2024-04-25T10:30:51.013 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:51.010+0000 7ff3e8ff9640 1 -- 172.21.15.37:0/2131285595 <== mon.0 v2:172.21.15.37:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff3d4037040 con 0x7ff3ec0cf390 2024-04-25T10:30:51.013 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:51.010+0000 7ff3f2884640 1 -- 172.21.15.37:0/2131285595 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7ff3ec1f4ce0 con 0x7ff3ec0cf390 2024-04-25T10:30:51.013 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:51.010+0000 7ff3f2884640 1 -- 172.21.15.37:0/2131285595 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7ff3ec14b7d0 con 0x7ff3ec0cf390 2024-04-25T10:30:51.014 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:51.011+0000 7ff3f2884640 1 -- 172.21.15.37:0/2131285595 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7ff3ec1f4750 con 0x7ff3ec0cf390 2024-04-25T10:30:51.014 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:51.012+0000 7ff3e8ff9640 1 -- 172.21.15.37:0/2131285595 <== mon.0 v2:172.21.15.37:3300/0 4 ==== mgrmap(e 5) ==== 96782+0+0 (secure 0 0 0) 0x7ff3d4044070 con 0x7ff3ec0cf390 2024-04-25T10:30:51.014 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:51.012+0000 7ff3e8ff9640 1 --2- 172.21.15.37:0/2131285595 >> [v2:172.21.15.203:6832/2418205237,v1:172.21.15.203:6833/2418205237] conn(0x7ff3cc0744e0 0x7ff3cc0769f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:30:51.014 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:51.013+0000 7ff3e8ff9640 1 -- 172.21.15.37:0/2131285595 <== mon.0 v2:172.21.15.37:3300/0 5 ==== osd_map(46..46 src has 1..46) ==== 6364+0+0 (secure 0 0 0) 0x7ff3d40b60f0 con 0x7ff3ec0cf390 2024-04-25T10:30:51.014 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:51.013+0000 7ff3ebfff640 1 --2- 172.21.15.37:0/2131285595 >> [v2:172.21.15.203:6832/2418205237,v1:172.21.15.203:6833/2418205237] conn(0x7ff3cc0744e0 0x7ff3cc0769f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:30:51.014 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:51.013+0000 7ff3ebfff640 1 --2- 172.21.15.37:0/2131285595 >> [v2:172.21.15.203:6832/2418205237,v1:172.21.15.203:6833/2418205237] conn(0x7ff3cc0744e0 0x7ff3cc0769f0 secure :-1 s=READY pgs=289 cs=0 l=1 rev1=1 crypto rx=0x7ff3dc006fd0 tx=0x7ff3dc008040 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:30:51.018 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:51.017+0000 7ff3e8ff9640 1 -- 172.21.15.37:0/2131285595 <== mon.0 v2:172.21.15.37:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187246 (secure 0 0 0) 0x7ff3ec0702e0 con 0x7ff3ec0cf390 2024-04-25T10:30:51.205 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:51.204+0000 7ff3f2884640 1 -- 172.21.15.37:0/2131285595 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"} v 0) -- 0x7ff3ec06a140 con 0x7ff3ec0cf390 2024-04-25T10:30:51.205 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:51.204+0000 7ff3e8ff9640 1 -- 172.21.15.37:0/2131285595 <== mon.0 v2:172.21.15.37:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]=0 v46) ==== 100+0+11 (secure 0 0 0) 0x7ff3d40805e0 con 0x7ff3ec0cf390 2024-04-25T10:30:51.206 INFO:teuthology.orchestra.run.smithi037.stdout:pg_num: 64 2024-04-25T10:30:51.208 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:51.206+0000 7ff3f2884640 1 -- 172.21.15.37:0/2131285595 >> [v2:172.21.15.203:6832/2418205237,v1:172.21.15.203:6833/2418205237] conn(0x7ff3cc0744e0 msgr2=0x7ff3cc0769f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:30:51.208 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:51.206+0000 7ff3f2884640 1 --2- 172.21.15.37:0/2131285595 >> [v2:172.21.15.203:6832/2418205237,v1:172.21.15.203:6833/2418205237] conn(0x7ff3cc0744e0 0x7ff3cc0769f0 secure :-1 s=READY pgs=289 cs=0 l=1 rev1=1 crypto rx=0x7ff3dc006fd0 tx=0x7ff3dc008040 comp rx=0 tx=0).stop 2024-04-25T10:30:51.208 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:51.206+0000 7ff3f2884640 1 -- 172.21.15.37:0/2131285595 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7ff3ec0cf390 msgr2=0x7ff3ec1f17c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:30:51.208 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:51.206+0000 7ff3f2884640 1 --2- 172.21.15.37:0/2131285595 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7ff3ec0cf390 0x7ff3ec1f17c0 secure :-1 s=READY pgs=635 cs=0 l=1 rev1=1 crypto rx=0x7ff3d4000c00 tx=0x7ff3d40046c0 comp rx=0 tx=0).stop 2024-04-25T10:30:51.208 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:51.206+0000 7ff3f2884640 1 -- 172.21.15.37:0/2131285595 shutdown_connections 2024-04-25T10:30:51.208 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:51.206+0000 7ff3f2884640 1 --2- 172.21.15.37:0/2131285595 >> [v2:172.21.15.203:6832/2418205237,v1:172.21.15.203:6833/2418205237] conn(0x7ff3cc0744e0 0x7ff3cc0769f0 unknown :-1 s=CLOSED pgs=289 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:30:51.208 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:51.206+0000 7ff3f2884640 1 --2- 172.21.15.37:0/2131285595 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7ff3ec156c30 0x7ff3ec14da80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:30:51.208 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:51.206+0000 7ff3f2884640 1 --2- 172.21.15.37:0/2131285595 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7ff3ec0cf390 0x7ff3ec1f17c0 unknown :-1 s=CLOSED pgs=635 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:30:51.209 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:51.206+0000 7ff3f2884640 1 --2- 172.21.15.37:0/2131285595 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7ff3ec06fed0 0x7ff3ec1f41c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:30:51.209 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:51.206+0000 7ff3f2884640 1 -- 172.21.15.37:0/2131285595 >> 172.21.15.37:0/2131285595 conn(0x7ff3ec0b36b0 msgr2=0x7ff3ec14ab20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:30:51.209 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:51.206+0000 7ff3f2884640 1 -- 172.21.15.37:0/2131285595 shutdown_connections 2024-04-25T10:30:51.209 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:51.206+0000 7ff3f2884640 1 -- 172.21.15.37:0/2131285595 wait complete. 2024-04-25T10:30:51.219 DEBUG:teuthology.orchestra.run.smithi037:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get recovery_fs_metadata pg_num 2024-04-25T10:30:51.333 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:51.332+0000 7f4aef3a8640 1 Processor -- start 2024-04-25T10:30:51.333 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:51.332+0000 7f4aef3a8640 1 -- start start 2024-04-25T10:30:51.338 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:51.334+0000 7f4aef3a8640 1 --2- >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f4ae80d2860 0x7f4ae806e200 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:30:51.338 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:51.334+0000 7f4aef3a8640 1 -- --> v1:172.21.15.37:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f4ae80d2e40 con 0x7f4ae80b2dc0 2024-04-25T10:30:51.338 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:51.334+0000 7f4aef3a8640 1 -- --> v1:172.21.15.203:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f4ae80d3120 con 0x7f4ae80ce560 2024-04-25T10:30:51.338 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:51.334+0000 7f4aef3a8640 1 -- --> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] -- mon_getmap magic: 0 -- 0x7f4ae80b92c0 con 0x7f4ae80d2860 2024-04-25T10:30:51.338 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:51.334+0000 7f4aed144640 1 --1- >> v1:172.21.15.37:6789/0 conn(0x7f4ae80b2dc0 0x7f4ae806daf0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.37:6789/0 says I am v1:172.21.15.37:38700/0 (socket says 172.21.15.37:38700) 2024-04-25T10:30:51.338 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:51.334+0000 7f4aed144640 1 -- 172.21.15.37:0/3645023416 learned_addr learned my addr 172.21.15.37:0/3645023416 (peer_addr_for_me v1:172.21.15.37:0/0) 2024-04-25T10:30:51.338 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:51.334+0000 7f4aec943640 1 --2- 172.21.15.37:0/3645023416 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f4ae80d2860 0x7f4ae806e200 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:30:51.338 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:51.334+0000 7f4ae7fff640 1 -- 172.21.15.37:0/3645023416 <== mon.0 v1:172.21.15.37:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 2024292278 0 0) 0x7f4ae80d2e40 con 0x7f4ae80b2dc0 2024-04-25T10:30:51.338 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:51.334+0000 7f4ae7fff640 1 -- 172.21.15.37:0/3645023416 --> v1:172.21.15.37:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f4ad0003520 con 0x7f4ae80b2dc0 2024-04-25T10:30:51.338 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:51.334+0000 7f4ae7fff640 1 -- 172.21.15.37:0/3645023416 <== mon.0 v1:172.21.15.37:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 1793669525 0 0) 0x7f4ad0003520 con 0x7f4ae80b2dc0 2024-04-25T10:30:51.338 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:51.334+0000 7f4ae7fff640 1 -- 172.21.15.37:0/3645023416 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f4ae80d2860 msgr2=0x7f4ae806e200 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:30:51.339 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:51.334+0000 7f4ae7fff640 1 --2- 172.21.15.37:0/3645023416 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f4ae80d2860 0x7f4ae806e200 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:30:51.339 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:51.334+0000 7f4ae7fff640 1 -- 172.21.15.37:0/3645023416 >> v1:172.21.15.203:6789/0 conn(0x7f4ae80ce560 legacy=0x7f4ae80ce970 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:30:51.339 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:51.334+0000 7f4ae7fff640 1 -- 172.21.15.37:0/3645023416 --> v1:172.21.15.37:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f4ae80d3400 con 0x7f4ae80b2dc0 2024-04-25T10:30:51.339 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:51.334+0000 7f4ae7fff640 1 -- 172.21.15.37:0/3645023416 <== mon.0 v1:172.21.15.37:6789/0 3 ==== mon_map magic: 0 ==== 413+0+0 (unknown 1577794600 0 0) 0x7f4ae00017f0 con 0x7f4ae80b2dc0 2024-04-25T10:30:51.339 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:51.334+0000 7f4ae7fff640 1 -- 172.21.15.37:0/3645023416 >> v1:172.21.15.37:6789/0 conn(0x7f4ae80b2dc0 legacy=0x7f4ae806daf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:30:51.339 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:51.334+0000 7f4ae7fff640 1 --2- 172.21.15.37:0/3645023416 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f4ad00039e0 0x7f4ad0003e10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:30:51.339 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:51.334+0000 7f4ae7fff640 1 --2- 172.21.15.37:0/3645023416 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f4ad0004800 0x7f4ad0004c10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:30:51.339 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:51.334+0000 7f4ae7fff640 1 --2- 172.21.15.37:0/3645023416 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f4ae80d2860 0x7f4ad0005170 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:30:51.339 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:51.335+0000 7f4ae7fff640 1 -- 172.21.15.37:0/3645023416 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 -- 0x7f4ad000ba70 con 0x7f4ad0004800 2024-04-25T10:30:51.339 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:51.335+0000 7f4ae7fff640 1 -- 172.21.15.37:0/3645023416 --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_getmap magic: 0 -- 0x7f4ad000bc20 con 0x7f4ad00039e0 2024-04-25T10:30:51.339 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:51.335+0000 7f4ae7fff640 1 -- 172.21.15.37:0/3645023416 --> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] -- mon_getmap magic: 0 -- 0x7f4ad000be20 con 0x7f4ae80d2860 2024-04-25T10:30:51.339 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:51.335+0000 7f4aec943640 1 --2- 172.21.15.37:0/3645023416 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f4ad0004800 0x7f4ad0004c10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:30:51.339 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:51.335+0000 7f4aed144640 1 --2- 172.21.15.37:0/3645023416 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f4ad00039e0 0x7f4ad0003e10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:30:51.339 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:51.335+0000 7f4aed945640 1 --2- 172.21.15.37:0/3645023416 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f4ae80d2860 0x7f4ad0005170 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:30:51.340 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:51.335+0000 7f4aec943640 1 -- 172.21.15.37:0/3645023416 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f4ae80d2860 msgr2=0x7f4ad0005170 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:30:51.340 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:51.335+0000 7f4aec943640 1 --2- 172.21.15.37:0/3645023416 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f4ae80d2860 0x7f4ad0005170 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:30:51.340 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:51.335+0000 7f4aec943640 1 -- 172.21.15.37:0/3645023416 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f4ad00039e0 msgr2=0x7f4ad0003e10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:30:51.340 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:51.335+0000 7f4aec943640 1 --2- 172.21.15.37:0/3645023416 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f4ad00039e0 0x7f4ad0003e10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:30:51.340 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:51.335+0000 7f4aec943640 1 -- 172.21.15.37:0/3645023416 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f4ad0003520 con 0x7f4ad0004800 2024-04-25T10:30:51.340 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:51.335+0000 7f4aed945640 1 --2- 172.21.15.37:0/3645023416 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f4ae80d2860 0x7f4ad0005170 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T10:30:51.340 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:51.336+0000 7f4aec943640 1 --2- 172.21.15.37:0/3645023416 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f4ad0004800 0x7f4ad0004c10 secure :-1 s=READY pgs=637 cs=0 l=1 rev1=1 crypto rx=0x7f4adc009ac0 tx=0x7f4ad0005830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=6b2817fc7ddedaa6 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:30:51.340 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:51.336+0000 7f4ae7fff640 1 -- 172.21.15.37:0/3645023416 <== mon.0 v2:172.21.15.37:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4adc036070 con 0x7f4ad0004800 2024-04-25T10:30:51.340 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:51.336+0000 7f4ae7fff640 1 -- 172.21.15.37:0/3645023416 <== mon.0 v2:172.21.15.37:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f4adc031400 con 0x7f4ad0004800 2024-04-25T10:30:51.340 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:51.336+0000 7f4ae7fff640 1 -- 172.21.15.37:0/3645023416 <== mon.0 v2:172.21.15.37:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4adc009690 con 0x7f4ad0004800 2024-04-25T10:30:51.340 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:51.336+0000 7f4aef3a8640 1 -- 172.21.15.37:0/3645023416 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f4ad0004800 msgr2=0x7f4ad0004c10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:30:51.340 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:51.336+0000 7f4aef3a8640 1 --2- 172.21.15.37:0/3645023416 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f4ad0004800 0x7f4ad0004c10 secure :-1 s=READY pgs=637 cs=0 l=1 rev1=1 crypto rx=0x7f4adc009ac0 tx=0x7f4ad0005830 comp rx=0 tx=0).stop 2024-04-25T10:30:51.340 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:51.336+0000 7f4aed945640 1 -- 172.21.15.37:0/3645023416 reap_dead start 2024-04-25T10:30:51.340 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:51.336+0000 7f4aef3a8640 1 -- 172.21.15.37:0/3645023416 shutdown_connections 2024-04-25T10:30:51.341 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:51.336+0000 7f4aef3a8640 1 -- 172.21.15.37:0/3645023416 >> 172.21.15.37:0/3645023416 conn(0x7f4ae80b5440 msgr2=0x7f4ae80b5850 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:30:51.341 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:51.337+0000 7f4aef3a8640 1 -- 172.21.15.37:0/3645023416 shutdown_connections 2024-04-25T10:30:51.341 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:51.337+0000 7f4aef3a8640 1 -- 172.21.15.37:0/3645023416 wait complete. 2024-04-25T10:30:51.343 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:51.338+0000 7f4aef3a8640 1 Processor -- start 2024-04-25T10:30:51.343 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:51.338+0000 7f4aef3a8640 1 -- start start 2024-04-25T10:30:51.343 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:51.340+0000 7f4aef3a8640 1 --2- >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f4ad0004800 0x7f4ae81f5c80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:30:51.343 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:51.340+0000 7f4aef3a8640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f4ae81f61c0 0x7f4ae8162b10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:30:51.343 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:51.340+0000 7f4aef3a8640 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f4ae8163050 0x7f4ae8148780 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:30:51.343 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:51.340+0000 7f4aef3a8640 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 -- 0x7f4ae806e200 con 0x7f4ae81f61c0 2024-04-25T10:30:51.343 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:51.340+0000 7f4aef3a8640 1 -- --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_getmap magic: 0 -- 0x7f4ae80ce0d0 con 0x7f4ae8163050 2024-04-25T10:30:51.343 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:51.340+0000 7f4aef3a8640 1 -- --> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] -- mon_getmap magic: 0 -- 0x7f4ae806e380 con 0x7f4ad0004800 2024-04-25T10:30:51.343 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:51.340+0000 7f4aec943640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f4ae81f61c0 0x7f4ae8162b10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:30:51.343 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:51.340+0000 7f4aec943640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f4ae81f61c0 0x7f4ae8162b10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.37:3300/0 says I am v2:172.21.15.37:59440/0 (socket says 172.21.15.37:59440) 2024-04-25T10:30:51.343 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:51.340+0000 7f4aec943640 1 -- 172.21.15.37:0/2933212230 learned_addr learned my addr 172.21.15.37:0/2933212230 (peer_addr_for_me v2:172.21.15.37:0/0) 2024-04-25T10:30:51.344 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:51.340+0000 7f4aed144640 1 --2- 172.21.15.37:0/2933212230 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f4ad0004800 0x7f4ae81f5c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:30:51.344 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:51.340+0000 7f4aec943640 1 -- 172.21.15.37:0/2933212230 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f4ad0004800 msgr2=0x7f4ae81f5c80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:30:51.344 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:51.340+0000 7f4aec943640 1 --2- 172.21.15.37:0/2933212230 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f4ad0004800 0x7f4ae81f5c80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:30:51.344 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:51.340+0000 7f4aec943640 1 -- 172.21.15.37:0/2933212230 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f4ae8163050 msgr2=0x7f4ae8148780 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:30:51.344 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:51.340+0000 7f4aec943640 1 --2- 172.21.15.37:0/2933212230 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f4ae8163050 0x7f4ae8148780 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:30:51.344 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:51.340+0000 7f4aec943640 1 -- 172.21.15.37:0/2933212230 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f4ae814e630 con 0x7f4ae81f61c0 2024-04-25T10:30:51.344 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:51.340+0000 7f4aec943640 1 --2- 172.21.15.37:0/2933212230 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f4ae81f61c0 0x7f4ae8162b10 secure :-1 s=READY pgs=638 cs=0 l=1 rev1=1 crypto rx=0x7f4adc034040 tx=0x7f4adc004730 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:30:51.344 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:51.340+0000 7f4ae5ffb640 1 -- 172.21.15.37:0/2933212230 <== mon.0 v2:172.21.15.37:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4adc041020 con 0x7f4ae81f61c0 2024-04-25T10:30:51.344 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:51.340+0000 7f4ae5ffb640 1 -- 172.21.15.37:0/2933212230 <== mon.0 v2:172.21.15.37:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f4adc03c470 con 0x7f4ae81f61c0 2024-04-25T10:30:51.344 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:51.340+0000 7f4aef3a8640 1 -- 172.21.15.37:0/2933212230 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f4ae80cd230 con 0x7f4ae81f61c0 2024-04-25T10:30:51.344 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:51.340+0000 7f4ae5ffb640 1 -- 172.21.15.37:0/2933212230 <== mon.0 v2:172.21.15.37:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4adc036040 con 0x7f4ae81f61c0 2024-04-25T10:30:51.344 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:51.341+0000 7f4aef3a8640 1 -- 172.21.15.37:0/2933212230 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f4ae80cc470 con 0x7f4ae81f61c0 2024-04-25T10:30:51.344 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:51.342+0000 7f4aef3a8640 1 -- 172.21.15.37:0/2933212230 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f4ae80b5ba0 con 0x7f4ae81f61c0 2024-04-25T10:30:51.345 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:51.342+0000 7f4ae5ffb640 1 -- 172.21.15.37:0/2933212230 <== mon.0 v2:172.21.15.37:3300/0 4 ==== mgrmap(e 5) ==== 96782+0+0 (secure 0 0 0) 0x7f4adc044020 con 0x7f4ae81f61c0 2024-04-25T10:30:51.345 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:51.343+0000 7f4ae5ffb640 1 --2- 172.21.15.37:0/2933212230 >> [v2:172.21.15.203:6832/2418205237,v1:172.21.15.203:6833/2418205237] conn(0x7f4ad40744b0 0x7f4ad40769c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:30:51.345 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:51.343+0000 7f4ae5ffb640 1 -- 172.21.15.37:0/2933212230 <== mon.0 v2:172.21.15.37:3300/0 5 ==== osd_map(46..46 src has 1..46) ==== 6364+0+0 (secure 0 0 0) 0x7f4adc0b5d70 con 0x7f4ae81f61c0 2024-04-25T10:30:51.345 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:51.343+0000 7f4aed144640 1 --2- 172.21.15.37:0/2933212230 >> [v2:172.21.15.203:6832/2418205237,v1:172.21.15.203:6833/2418205237] conn(0x7f4ad40744b0 0x7f4ad40769c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:30:51.346 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:51.344+0000 7f4aed144640 1 --2- 172.21.15.37:0/2933212230 >> [v2:172.21.15.203:6832/2418205237,v1:172.21.15.203:6833/2418205237] conn(0x7f4ad40744b0 0x7f4ad40769c0 secure :-1 s=READY pgs=290 cs=0 l=1 rev1=1 crypto rx=0x7f4ae0000ef0 tx=0x7f4ae0009040 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:30:51.349 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:51.348+0000 7f4ae5ffb640 1 -- 172.21.15.37:0/2933212230 <== mon.0 v2:172.21.15.37:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187246 (secure 0 0 0) 0x7f4adc04b030 con 0x7f4ae81f61c0 2024-04-25T10:30:51.534 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:51.533+0000 7f4aef3a8640 1 -- 172.21.15.37:0/2933212230 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "recovery_fs_metadata", "var": "pg_num"} v 0) -- 0x7f4ae806a140 con 0x7f4ae81f61c0 2024-04-25T10:30:51.534 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:51.533+0000 7f4ae5ffb640 1 -- 172.21.15.37:0/2933212230 <== mon.0 v2:172.21.15.37:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "recovery_fs_metadata", "var": "pg_num"}]=0 v46) ==== 109+0+11 (secure 0 0 0) 0x7f4adc080270 con 0x7f4ae81f61c0 2024-04-25T10:30:51.534 INFO:teuthology.orchestra.run.smithi037.stdout:pg_num: 32 2024-04-25T10:30:51.537 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:51.535+0000 7f4aef3a8640 1 -- 172.21.15.37:0/2933212230 >> [v2:172.21.15.203:6832/2418205237,v1:172.21.15.203:6833/2418205237] conn(0x7f4ad40744b0 msgr2=0x7f4ad40769c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:30:51.537 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:51.535+0000 7f4aef3a8640 1 --2- 172.21.15.37:0/2933212230 >> [v2:172.21.15.203:6832/2418205237,v1:172.21.15.203:6833/2418205237] conn(0x7f4ad40744b0 0x7f4ad40769c0 secure :-1 s=READY pgs=290 cs=0 l=1 rev1=1 crypto rx=0x7f4ae0000ef0 tx=0x7f4ae0009040 comp rx=0 tx=0).stop 2024-04-25T10:30:51.537 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:51.535+0000 7f4aef3a8640 1 -- 172.21.15.37:0/2933212230 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f4ae81f61c0 msgr2=0x7f4ae8162b10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:30:51.537 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:51.535+0000 7f4aef3a8640 1 --2- 172.21.15.37:0/2933212230 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f4ae81f61c0 0x7f4ae8162b10 secure :-1 s=READY pgs=638 cs=0 l=1 rev1=1 crypto rx=0x7f4adc034040 tx=0x7f4adc004730 comp rx=0 tx=0).stop 2024-04-25T10:30:51.537 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:51.535+0000 7f4aef3a8640 1 -- 172.21.15.37:0/2933212230 shutdown_connections 2024-04-25T10:30:51.537 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:51.535+0000 7f4aef3a8640 1 --2- 172.21.15.37:0/2933212230 >> [v2:172.21.15.203:6832/2418205237,v1:172.21.15.203:6833/2418205237] conn(0x7f4ad40744b0 0x7f4ad40769c0 unknown :-1 s=CLOSED pgs=290 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:30:51.537 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:51.535+0000 7f4aef3a8640 1 --2- 172.21.15.37:0/2933212230 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f4ae8163050 0x7f4ae8148780 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:30:51.537 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:51.535+0000 7f4aef3a8640 1 --2- 172.21.15.37:0/2933212230 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f4ae81f61c0 0x7f4ae8162b10 unknown :-1 s=CLOSED pgs=638 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:30:51.537 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:51.535+0000 7f4aef3a8640 1 --2- 172.21.15.37:0/2933212230 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f4ad0004800 0x7f4ae81f5c80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:30:51.537 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:51.535+0000 7f4aef3a8640 1 -- 172.21.15.37:0/2933212230 >> 172.21.15.37:0/2933212230 conn(0x7f4ae80b5440 msgr2=0x7f4ae808d350 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:30:51.538 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:51.535+0000 7f4aef3a8640 1 -- 172.21.15.37:0/2933212230 shutdown_connections 2024-04-25T10:30:51.538 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:51.535+0000 7f4aef3a8640 1 -- 172.21.15.37:0/2933212230 wait complete. 2024-04-25T10:30:51.548 DEBUG:teuthology.orchestra.run.smithi037:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-04-25T10:30:51.660 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:51.658+0000 7f909bb7d640 1 Processor -- start 2024-04-25T10:30:51.660 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:51.658+0000 7f909bb7d640 1 -- start start 2024-04-25T10:30:51.664 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:51.660+0000 7f909bb7d640 1 --2- >> v2:172.21.15.37:3300/0 conn(0x7f90940b2dc0 0x7f90940d2860 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:30:51.664 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:51.660+0000 7f909bb7d640 1 --2- >> v2:172.21.15.203:3300/0 conn(0x7f909406e610 0x7f90940ce560 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:30:51.664 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:51.660+0000 7f909bb7d640 1 -- --> v1:172.21.15.203:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f90940d2e40 con 0x7f909406daf0 2024-04-25T10:30:51.664 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:51.660+0000 7f909bb7d640 1 -- --> v2:172.21.15.37:3300/0 -- mon_getmap magic: 0 -- 0x7f90940b92c0 con 0x7f90940b2dc0 2024-04-25T10:30:51.665 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:51.660+0000 7f909bb7d640 1 -- --> v2:172.21.15.203:3300/0 -- mon_getmap magic: 0 -- 0x7f90940ce0d0 con 0x7f909406e610 2024-04-25T10:30:51.665 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:51.660+0000 7f9099919640 1 --2- >> v2:172.21.15.37:3300/0 conn(0x7f90940b2dc0 0x7f90940d2860 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:30:51.665 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:51.660+0000 7f9099919640 1 --2- >> v2:172.21.15.37:3300/0 conn(0x7f90940b2dc0 0x7f90940d2860 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.37:3300/0 says I am v2:172.21.15.37:59446/0 (socket says 172.21.15.37:59446) 2024-04-25T10:30:51.665 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:51.660+0000 7f9099919640 1 -- 172.21.15.37:0/3267502776 learned_addr learned my addr 172.21.15.37:0/3267502776 (peer_addr_for_me v2:172.21.15.37:0/0) 2024-04-25T10:30:51.665 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:51.660+0000 7f909a11a640 1 --2- 172.21.15.37:0/3267502776 >> v2:172.21.15.203:3300/0 conn(0x7f909406e610 0x7f90940ce560 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:30:51.665 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:51.660+0000 7f9099919640 1 -- 172.21.15.37:0/3267502776 >> v2:172.21.15.203:3300/0 conn(0x7f909406e610 msgr2=0x7f90940ce560 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:30:51.666 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:51.660+0000 7f9099919640 1 --2- 172.21.15.37:0/3267502776 >> v2:172.21.15.203:3300/0 conn(0x7f909406e610 0x7f90940ce560 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:30:51.666 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:51.660+0000 7f9099919640 1 -- 172.21.15.37:0/3267502776 >> v1:172.21.15.203:6789/0 conn(0x7f909406daf0 legacy=0x7f909406df00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:30:51.666 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:51.660+0000 7f9099919640 1 -- 172.21.15.37:0/3267502776 --> v2:172.21.15.37:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f90940d3120 con 0x7f90940b2dc0 2024-04-25T10:30:51.666 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:51.661+0000 7f9099919640 1 --2- 172.21.15.37:0/3267502776 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f90940b2dc0 0x7f90940d2860 secure :-1 s=READY pgs=639 cs=0 l=1 rev1=1 crypto rx=0x7f909408e1b0 tx=0x7f909002a270 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d3e852f7239fca40 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:30:51.666 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:51.661+0000 7f9098917640 1 -- 172.21.15.37:0/3267502776 <== mon.0 v2:172.21.15.37:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9090004030 con 0x7f90940b2dc0 2024-04-25T10:30:51.667 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:51.661+0000 7f9098917640 1 -- 172.21.15.37:0/3267502776 <== mon.0 v2:172.21.15.37:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f90900041d0 con 0x7f90940b2dc0 2024-04-25T10:30:51.667 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:51.661+0000 7f9098917640 1 -- 172.21.15.37:0/3267502776 <== mon.0 v2:172.21.15.37:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9090003610 con 0x7f90940b2dc0 2024-04-25T10:30:51.667 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:51.661+0000 7f909bb7d640 1 -- 172.21.15.37:0/3267502776 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f90940b2dc0 msgr2=0x7f90940d2860 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:30:51.667 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:51.661+0000 7f909bb7d640 1 --2- 172.21.15.37:0/3267502776 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f90940b2dc0 0x7f90940d2860 secure :-1 s=READY pgs=639 cs=0 l=1 rev1=1 crypto rx=0x7f909408e1b0 tx=0x7f909002a270 comp rx=0 tx=0).stop 2024-04-25T10:30:51.667 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:51.662+0000 7f909bb7d640 1 -- 172.21.15.37:0/3267502776 shutdown_connections 2024-04-25T10:30:51.667 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:51.662+0000 7f909bb7d640 1 --2- 172.21.15.37:0/3267502776 >> v2:172.21.15.203:3300/0 conn(0x7f909406e610 0x7f90940ce560 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:30:51.667 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:51.662+0000 7f909bb7d640 1 --2- 172.21.15.37:0/3267502776 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f90940b2dc0 0x7f90940d2860 unknown :-1 s=CLOSED pgs=639 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:30:51.668 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:51.662+0000 7f909bb7d640 1 -- 172.21.15.37:0/3267502776 >> 172.21.15.37:0/3267502776 conn(0x7f90940b5440 msgr2=0x7f90940b5850 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:30:51.668 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:51.662+0000 7f909bb7d640 1 -- 172.21.15.37:0/3267502776 shutdown_connections 2024-04-25T10:30:51.668 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:51.662+0000 7f909bb7d640 1 -- 172.21.15.37:0/3267502776 wait complete. 2024-04-25T10:30:51.673 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:51.663+0000 7f909bb7d640 1 Processor -- start 2024-04-25T10:30:51.673 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:51.663+0000 7f909bb7d640 1 -- start start 2024-04-25T10:30:51.673 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:51.665+0000 7f909bb7d640 1 --2- >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f909406daf0 0x7f9094163270 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:30:51.673 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:51.665+0000 7f909bb7d640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f909406e610 0x7f909414fe20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:30:51.674 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:51.665+0000 7f909bb7d640 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f90940b2dc0 0x7f909415e7a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:30:51.674 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:51.665+0000 7f909bb7d640 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 -- 0x7f90940ceaa0 con 0x7f909406e610 2024-04-25T10:30:51.674 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:51.665+0000 7f909bb7d640 1 -- --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_getmap magic: 0 -- 0x7f90940d36e0 con 0x7f90940b2dc0 2024-04-25T10:30:51.674 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:51.665+0000 7f909bb7d640 1 -- --> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] -- mon_getmap magic: 0 -- 0x7f90940cec20 con 0x7f909406daf0 2024-04-25T10:30:51.674 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:51.665+0000 7f9099118640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f909406e610 0x7f909414fe20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:30:51.674 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:51.665+0000 7f9099118640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f909406e610 0x7f909414fe20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.37:3300/0 says I am v2:172.21.15.37:59450/0 (socket says 172.21.15.37:59450) 2024-04-25T10:30:51.675 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:51.665+0000 7f9099118640 1 -- 172.21.15.37:0/3314443942 learned_addr learned my addr 172.21.15.37:0/3314443942 (peer_addr_for_me v2:172.21.15.37:0/0) 2024-04-25T10:30:51.675 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:51.665+0000 7f9099919640 1 --2- 172.21.15.37:0/3314443942 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f909406daf0 0x7f9094163270 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:30:51.675 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:51.665+0000 7f909a11a640 1 --2- 172.21.15.37:0/3314443942 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f90940b2dc0 0x7f909415e7a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:30:51.675 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:51.665+0000 7f9099118640 1 -- 172.21.15.37:0/3314443942 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f909406daf0 msgr2=0x7f9094163270 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:30:51.675 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:51.665+0000 7f9099118640 1 --2- 172.21.15.37:0/3314443942 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f909406daf0 0x7f9094163270 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:30:51.675 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:51.665+0000 7f9099118640 1 -- 172.21.15.37:0/3314443942 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f90940b2dc0 msgr2=0x7f909415e7a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:30:51.676 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:51.665+0000 7f9099118640 1 --2- 172.21.15.37:0/3314443942 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f90940b2dc0 0x7f909415e7a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:30:51.676 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:51.665+0000 7f9099118640 1 -- 172.21.15.37:0/3314443942 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f90940cbd00 con 0x7f909406e610 2024-04-25T10:30:51.676 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:51.665+0000 7f9099919640 1 --2- 172.21.15.37:0/3314443942 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f909406daf0 0x7f9094163270 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T10:30:51.676 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:51.665+0000 7f9099118640 1 --2- 172.21.15.37:0/3314443942 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f909406e610 0x7f909414fe20 secure :-1 s=READY pgs=640 cs=0 l=1 rev1=1 crypto rx=0x7f908400c440 tx=0x7f908400c900 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:30:51.676 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:51.665+0000 7f908a7fc640 1 -- 172.21.15.37:0/3314443942 <== mon.0 v2:172.21.15.37:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f90840052d0 con 0x7f909406e610 2024-04-25T10:30:51.676 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:51.666+0000 7f908a7fc640 1 -- 172.21.15.37:0/3314443942 <== mon.0 v2:172.21.15.37:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f9084005470 con 0x7f909406e610 2024-04-25T10:30:51.677 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:51.666+0000 7f908a7fc640 1 -- 172.21.15.37:0/3314443942 <== mon.0 v2:172.21.15.37:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9084007260 con 0x7f909406e610 2024-04-25T10:30:51.677 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:51.666+0000 7f909bb7d640 1 -- 172.21.15.37:0/3314443942 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f90940cbfe0 con 0x7f909406e610 2024-04-25T10:30:51.677 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:51.666+0000 7f909bb7d640 1 -- 172.21.15.37:0/3314443942 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f90940cc5a0 con 0x7f909406e610 2024-04-25T10:30:51.677 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:51.666+0000 7f909bb7d640 1 -- 172.21.15.37:0/3314443942 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f90940729d0 con 0x7f909406e610 2024-04-25T10:30:51.677 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:51.667+0000 7f908a7fc640 1 -- 172.21.15.37:0/3314443942 <== mon.0 v2:172.21.15.37:3300/0 4 ==== mgrmap(e 5) ==== 96782+0+0 (secure 0 0 0) 0x7f9084007400 con 0x7f909406e610 2024-04-25T10:30:51.677 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:51.668+0000 7f908a7fc640 1 --2- 172.21.15.37:0/3314443942 >> [v2:172.21.15.203:6832/2418205237,v1:172.21.15.203:6833/2418205237] conn(0x7f90780744b0 0x7f90780769c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:30:51.678 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:51.668+0000 7f908a7fc640 1 -- 172.21.15.37:0/3314443942 <== mon.0 v2:172.21.15.37:3300/0 5 ==== osd_map(46..46 src has 1..46) ==== 6364+0+0 (secure 0 0 0) 0x7f90840947f0 con 0x7f909406e610 2024-04-25T10:30:51.678 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:51.668+0000 7f9099919640 1 --2- 172.21.15.37:0/3314443942 >> [v2:172.21.15.203:6832/2418205237,v1:172.21.15.203:6833/2418205237] conn(0x7f90780744b0 0x7f90780769c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:30:51.678 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:51.669+0000 7f9099919640 1 --2- 172.21.15.37:0/3314443942 >> [v2:172.21.15.203:6832/2418205237,v1:172.21.15.203:6833/2418205237] conn(0x7f90780744b0 0x7f90780769c0 secure :-1 s=READY pgs=291 cs=0 l=1 rev1=1 crypto rx=0x7f9090033000 tx=0x7f909002e000 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:30:51.679 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:51.672+0000 7f908a7fc640 1 -- 172.21.15.37:0/3314443942 <== mon.0 v2:172.21.15.37:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187246 (secure 0 0 0) 0x7f90940729d0 con 0x7f909406e610 2024-04-25T10:30:51.898 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:51.897+0000 7f909bb7d640 1 -- 172.21.15.37:0/3314443942 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f90940cde20 con 0x7f909406e610 2024-04-25T10:30:51.899 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:51.898+0000 7f908a7fc640 1 -- 172.21.15.37:0/3314443942 <== mon.0 v2:172.21.15.37:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 27 v27) ==== 94+0+6489 (secure 0 0 0) 0x7f90940cde20 con 0x7f909406e610 2024-04-25T10:30:51.899 INFO:teuthology.orchestra.run.smithi037.stdout: 2024-04-25T10:30:51.899 INFO:teuthology.orchestra.run.smithi037.stdout:{"epoch":27,"default_fscid":2,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":4946,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.37:6835/683655741","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.37:6834","nonce":683655741},{"type":"v1","addr":"172.21.15.37:6835","nonce":683655741}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":19},{"gid":5215,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.203:6836/3927045001","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6835","nonce":3927045001},{"type":"v1","addr":"172.21.15.203:6836","nonce":3927045001}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":19},{"gid":5218,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.203:6837/3108183946","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6834","nonce":3108183946},{"type":"v1","addr":"172.21.15.203:6837","nonce":3108183946}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":19}],"filesystems":[{"mdsmap":{"epoch":18,"flags":19,"flags_state":{"joinable":false,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-04-25T10:30:12.130473+0000","modified":"2024-04-25T10:30:24.853915+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":37,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":1,"in":[0],"up":{},"failed":[0],"damaged":[],"stopped":[],"info":{},"data_pools":[5],"metadata_pool":4,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":0,"qdb_cluster":[]},"id":2},{"mdsmap":{"epoch":27,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-04-25T10:30:28.588617+0000","modified":"2024-04-25T10:30:42.935167+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":1,"in":[0],"up":{"mds_0":5625},"failed":[],"damaged":[],"stopped":[],"info":{"gid_5625":{"gid":5625,"name":"c","rank":0,"incarnation":24,"state":"up:active","state_seq":7,"addr":"172.21.15.37:6833/2353203722","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.37:6832","nonce":2353203722},{"type":"v1","addr":"172.21.15.37:6833","nonce":2353203722}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[5],"metadata_pool":6,"enabled":true,"fs_name":"recovery_fs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":5625,"qdb_cluster":[5625]},"id":3}]} 2024-04-25T10:30:51.899 INFO:teuthology.orchestra.run.smithi037.stderr:dumped fsmap epoch 27 2024-04-25T10:30:51.901 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:51.900+0000 7f909bb7d640 1 -- 172.21.15.37:0/3314443942 >> [v2:172.21.15.203:6832/2418205237,v1:172.21.15.203:6833/2418205237] conn(0x7f90780744b0 msgr2=0x7f90780769c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:30:51.901 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:51.900+0000 7f909bb7d640 1 --2- 172.21.15.37:0/3314443942 >> [v2:172.21.15.203:6832/2418205237,v1:172.21.15.203:6833/2418205237] conn(0x7f90780744b0 0x7f90780769c0 secure :-1 s=READY pgs=291 cs=0 l=1 rev1=1 crypto rx=0x7f9090033000 tx=0x7f909002e000 comp rx=0 tx=0).stop 2024-04-25T10:30:51.901 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:51.900+0000 7f909bb7d640 1 -- 172.21.15.37:0/3314443942 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f909406e610 msgr2=0x7f909414fe20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:30:51.901 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:51.900+0000 7f909bb7d640 1 --2- 172.21.15.37:0/3314443942 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f909406e610 0x7f909414fe20 secure :-1 s=READY pgs=640 cs=0 l=1 rev1=1 crypto rx=0x7f908400c440 tx=0x7f908400c900 comp rx=0 tx=0).stop 2024-04-25T10:30:51.901 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:51.900+0000 7f909bb7d640 1 -- 172.21.15.37:0/3314443942 shutdown_connections 2024-04-25T10:30:51.901 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:51.900+0000 7f909bb7d640 1 --2- 172.21.15.37:0/3314443942 >> [v2:172.21.15.203:6832/2418205237,v1:172.21.15.203:6833/2418205237] conn(0x7f90780744b0 0x7f90780769c0 unknown :-1 s=CLOSED pgs=291 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:30:51.901 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:51.900+0000 7f909bb7d640 1 --2- 172.21.15.37:0/3314443942 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f90940b2dc0 0x7f909415e7a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:30:51.901 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:51.900+0000 7f909bb7d640 1 --2- 172.21.15.37:0/3314443942 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f909406e610 0x7f909414fe20 unknown :-1 s=CLOSED pgs=640 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:30:51.901 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:51.900+0000 7f909bb7d640 1 --2- 172.21.15.37:0/3314443942 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f909406daf0 0x7f9094163270 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:30:51.902 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:51.900+0000 7f909bb7d640 1 -- 172.21.15.37:0/3314443942 >> 172.21.15.37:0/3314443942 conn(0x7f90940b5440 msgr2=0x7f90940cf730 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:30:51.902 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:51.900+0000 7f909bb7d640 1 -- 172.21.15.37:0/3314443942 shutdown_connections 2024-04-25T10:30:51.902 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:51.900+0000 7f909bb7d640 1 -- 172.21.15.37:0/3314443942 wait complete. 2024-04-25T10:30:51.912 DEBUG:teuthology.orchestra.run.smithi037:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-04-25T10:30:52.024 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:52.023+0000 7fc8f6630640 1 Processor -- start 2024-04-25T10:30:52.024 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:52.023+0000 7fc8f6630640 1 -- start start 2024-04-25T10:30:52.026 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:52.024+0000 7fc8f6630640 1 --2- >> v2:172.21.15.203:3300/0 conn(0x7fc8f00b3c80 0x7fc8f00b4090 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:30:52.026 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:52.024+0000 7fc8f6630640 1 --2- >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7fc8f00d1ff0 0x7fc8f00d2400 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:30:52.026 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:52.024+0000 7fc8f6630640 1 --2- >> v2:172.21.15.37:3300/0 conn(0x7fc8f01625b0 0x7fc8f01629c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:30:52.026 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:52.024+0000 7fc8f6630640 1 -- --> v2:172.21.15.37:3300/0 -- mon_getmap magic: 0 -- 0x7fc8f008ceb0 con 0x7fc8f01625b0 2024-04-25T10:30:52.026 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:52.024+0000 7fc8f6630640 1 -- --> v2:172.21.15.203:3300/0 -- mon_getmap magic: 0 -- 0x7fc8f00b9300 con 0x7fc8f00b3c80 2024-04-25T10:30:52.026 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:52.024+0000 7fc8f6630640 1 -- --> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] -- mon_getmap magic: 0 -- 0x7fc8f009ce90 con 0x7fc8f00d1ff0 2024-04-25T10:30:52.026 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:52.024+0000 7fc8f4bcd640 1 --2- >> v2:172.21.15.37:3300/0 conn(0x7fc8f01625b0 0x7fc8f01629c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:30:52.026 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:52.024+0000 7fc8f4bcd640 1 --2- >> v2:172.21.15.37:3300/0 conn(0x7fc8f01625b0 0x7fc8f01629c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.37:3300/0 says I am v2:172.21.15.37:59452/0 (socket says 172.21.15.37:59452) 2024-04-25T10:30:52.026 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:52.024+0000 7fc8f4bcd640 1 -- 172.21.15.37:0/3036728406 learned_addr learned my addr 172.21.15.37:0/3036728406 (peer_addr_for_me v2:172.21.15.37:0/0) 2024-04-25T10:30:52.026 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:52.024+0000 7fc8effff640 1 --2- 172.21.15.37:0/3036728406 >> v2:172.21.15.203:3300/0 conn(0x7fc8f00b3c80 0x7fc8f00b4090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:30:52.026 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:52.024+0000 7fc8ef7fe640 1 --2- 172.21.15.37:0/3036728406 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7fc8f00d1ff0 0x7fc8f00d2400 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:30:52.027 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:52.025+0000 7fc8f4bcd640 1 -- 172.21.15.37:0/3036728406 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7fc8f00d1ff0 msgr2=0x7fc8f00d2400 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:30:52.027 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:52.025+0000 7fc8f4bcd640 1 --2- 172.21.15.37:0/3036728406 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7fc8f00d1ff0 0x7fc8f00d2400 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:30:52.027 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:52.025+0000 7fc8f4bcd640 1 -- 172.21.15.37:0/3036728406 >> v2:172.21.15.203:3300/0 conn(0x7fc8f00b3c80 msgr2=0x7fc8f00b4090 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:30:52.027 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:52.025+0000 7fc8f4bcd640 1 --2- 172.21.15.37:0/3036728406 >> v2:172.21.15.203:3300/0 conn(0x7fc8f00b3c80 0x7fc8f00b4090 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:30:52.027 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:52.025+0000 7fc8f4bcd640 1 -- 172.21.15.37:0/3036728406 --> v2:172.21.15.37:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fc8f00b8d10 con 0x7fc8f01625b0 2024-04-25T10:30:52.028 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:52.025+0000 7fc8f4bcd640 1 --2- 172.21.15.37:0/3036728406 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fc8f01625b0 0x7fc8f01629c0 secure :-1 s=READY pgs=641 cs=0 l=1 rev1=1 crypto rx=0x7fc8f006d040 tx=0x7fc8e802a610 comp rx=0 tx=0).ready entity=mon.0 client_cookie=4df443b8074411bf server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:30:52.028 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:52.025+0000 7fc8eeffd640 1 -- 172.21.15.37:0/3036728406 <== mon.0 v2:172.21.15.37:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc8e8036070 con 0x7fc8f01625b0 2024-04-25T10:30:52.028 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:52.025+0000 7fc8eeffd640 1 -- 172.21.15.37:0/3036728406 <== mon.0 v2:172.21.15.37:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fc8e8031600 con 0x7fc8f01625b0 2024-04-25T10:30:52.028 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:52.025+0000 7fc8eeffd640 1 -- 172.21.15.37:0/3036728406 <== mon.0 v2:172.21.15.37:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc8e80319d0 con 0x7fc8f01625b0 2024-04-25T10:30:52.028 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:52.026+0000 7fc8f6630640 1 -- 172.21.15.37:0/3036728406 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fc8f01625b0 msgr2=0x7fc8f01629c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:30:52.029 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:52.026+0000 7fc8f6630640 1 --2- 172.21.15.37:0/3036728406 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fc8f01625b0 0x7fc8f01629c0 secure :-1 s=READY pgs=641 cs=0 l=1 rev1=1 crypto rx=0x7fc8f006d040 tx=0x7fc8e802a610 comp rx=0 tx=0).stop 2024-04-25T10:30:52.029 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:52.026+0000 7fc8f6630640 1 -- 172.21.15.37:0/3036728406 shutdown_connections 2024-04-25T10:30:52.029 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:52.026+0000 7fc8f6630640 1 --2- 172.21.15.37:0/3036728406 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fc8f01625b0 0x7fc8f01629c0 unknown :-1 s=CLOSED pgs=641 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:30:52.029 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:52.026+0000 7fc8f6630640 1 --2- 172.21.15.37:0/3036728406 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7fc8f00d1ff0 0x7fc8f00d2400 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:30:52.029 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:52.026+0000 7fc8f6630640 1 --2- 172.21.15.37:0/3036728406 >> v2:172.21.15.203:3300/0 conn(0x7fc8f00b3c80 0x7fc8f00b4090 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:30:52.029 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:52.026+0000 7fc8f6630640 1 -- 172.21.15.37:0/3036728406 >> 172.21.15.37:0/3036728406 conn(0x7fc8f00d0390 msgr2=0x7fc8f00d07a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:30:52.029 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:52.026+0000 7fc8f6630640 1 -- 172.21.15.37:0/3036728406 shutdown_connections 2024-04-25T10:30:52.029 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:52.026+0000 7fc8f6630640 1 -- 172.21.15.37:0/3036728406 wait complete. 2024-04-25T10:30:52.029 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:52.028+0000 7fc8f6630640 1 Processor -- start 2024-04-25T10:30:52.030 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:52.028+0000 7fc8f6630640 1 -- start start 2024-04-25T10:30:52.031 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:52.029+0000 7fc8f6630640 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fc8f00b3c80 0x7fc8f006e880 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:30:52.032 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:52.029+0000 7fc8f6630640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fc8f00d1ff0 0x7fc8f015dc00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:30:52.032 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:52.029+0000 7fc8f6630640 1 --2- >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7fc8f01625b0 0x7fc8f015fd10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:30:52.032 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:52.029+0000 7fc8f6630640 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 -- 0x7fc8f00b45d0 con 0x7fc8f00d1ff0 2024-04-25T10:30:52.032 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:52.029+0000 7fc8f6630640 1 -- --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_getmap magic: 0 -- 0x7fc8f00b9020 con 0x7fc8f00b3c80 2024-04-25T10:30:52.032 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:52.029+0000 7fc8f6630640 1 -- --> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] -- mon_getmap magic: 0 -- 0x7fc8f00d2940 con 0x7fc8f01625b0 2024-04-25T10:30:52.032 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:52.030+0000 7fc8ef7fe640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fc8f00d1ff0 0x7fc8f015dc00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:30:52.032 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:52.030+0000 7fc8ef7fe640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fc8f00d1ff0 0x7fc8f015dc00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.37:3300/0 says I am v2:172.21.15.37:59458/0 (socket says 172.21.15.37:59458) 2024-04-25T10:30:52.032 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:52.030+0000 7fc8ef7fe640 1 -- 172.21.15.37:0/2840650853 learned_addr learned my addr 172.21.15.37:0/2840650853 (peer_addr_for_me v2:172.21.15.37:0/0) 2024-04-25T10:30:52.032 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:52.030+0000 7fc8effff640 1 --2- 172.21.15.37:0/2840650853 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fc8f00b3c80 0x7fc8f006e880 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:30:52.032 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:52.030+0000 7fc8ef7fe640 1 -- 172.21.15.37:0/2840650853 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7fc8f01625b0 msgr2=0x7fc8f015fd10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:30:52.032 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:52.030+0000 7fc8ef7fe640 1 --2- 172.21.15.37:0/2840650853 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7fc8f01625b0 0x7fc8f015fd10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:30:52.032 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:52.030+0000 7fc8ef7fe640 1 -- 172.21.15.37:0/2840650853 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fc8f00b3c80 msgr2=0x7fc8f006e880 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:30:52.032 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:52.030+0000 7fc8ef7fe640 1 --2- 172.21.15.37:0/2840650853 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fc8f00b3c80 0x7fc8f006e880 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:30:52.032 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:52.030+0000 7fc8ef7fe640 1 -- 172.21.15.37:0/2840650853 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fc8f006edc0 con 0x7fc8f00d1ff0 2024-04-25T10:30:52.033 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:52.030+0000 7fc8effff640 1 --2- 172.21.15.37:0/2840650853 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fc8f00b3c80 0x7fc8f006e880 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T10:30:52.033 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:52.030+0000 7fc8ef7fe640 1 --2- 172.21.15.37:0/2840650853 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fc8f00d1ff0 0x7fc8f015dc00 secure :-1 s=READY pgs=642 cs=0 l=1 rev1=1 crypto rx=0x7fc8d800e8a0 tx=0x7fc8d800ed60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:30:52.033 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:52.030+0000 7fc8ecff9640 1 -- 172.21.15.37:0/2840650853 <== mon.0 v2:172.21.15.37:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc8d8017070 con 0x7fc8f00d1ff0 2024-04-25T10:30:52.033 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:52.030+0000 7fc8ecff9640 1 -- 172.21.15.37:0/2840650853 <== mon.0 v2:172.21.15.37:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fc8d800c940 con 0x7fc8f00d1ff0 2024-04-25T10:30:52.033 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:52.030+0000 7fc8f6630640 1 -- 172.21.15.37:0/2840650853 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fc8f01e84c0 con 0x7fc8f00d1ff0 2024-04-25T10:30:52.033 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:52.030+0000 7fc8ecff9640 1 -- 172.21.15.37:0/2840650853 <== mon.0 v2:172.21.15.37:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc8d800cd30 con 0x7fc8f00d1ff0 2024-04-25T10:30:52.033 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:52.030+0000 7fc8f6630640 1 -- 172.21.15.37:0/2840650853 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fc8f01e8a80 con 0x7fc8f00d1ff0 2024-04-25T10:30:52.034 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:52.032+0000 7fc8f6630640 1 -- 172.21.15.37:0/2840650853 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fc8f006a140 con 0x7fc8f00d1ff0 2024-04-25T10:30:52.034 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:52.032+0000 7fc8ecff9640 1 -- 172.21.15.37:0/2840650853 <== mon.0 v2:172.21.15.37:3300/0 4 ==== mgrmap(e 5) ==== 96782+0+0 (secure 0 0 0) 0x7fc8d801a070 con 0x7fc8f00d1ff0 2024-04-25T10:30:52.034 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:52.032+0000 7fc8ecff9640 1 --2- 172.21.15.37:0/2840650853 >> [v2:172.21.15.203:6832/2418205237,v1:172.21.15.203:6833/2418205237] conn(0x7fc8d00744b0 0x7fc8d00769c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:30:52.034 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:52.032+0000 7fc8ecff9640 1 -- 172.21.15.37:0/2840650853 <== mon.0 v2:172.21.15.37:3300/0 5 ==== osd_map(46..46 src has 1..46) ==== 6364+0+0 (secure 0 0 0) 0x7fc8d8095710 con 0x7fc8f00d1ff0 2024-04-25T10:30:52.034 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:52.033+0000 7fc8effff640 1 --2- 172.21.15.37:0/2840650853 >> [v2:172.21.15.203:6832/2418205237,v1:172.21.15.203:6833/2418205237] conn(0x7fc8d00744b0 0x7fc8d00769c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:30:52.034 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:52.033+0000 7fc8effff640 1 --2- 172.21.15.37:0/2840650853 >> [v2:172.21.15.203:6832/2418205237,v1:172.21.15.203:6833/2418205237] conn(0x7fc8d00744b0 0x7fc8d00769c0 secure :-1 s=READY pgs=292 cs=0 l=1 rev1=1 crypto rx=0x7fc8e00099a0 tx=0x7fc8e0008040 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:30:52.038 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:52.037+0000 7fc8ecff9640 1 -- 172.21.15.37:0/2840650853 <== mon.0 v2:172.21.15.37:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187246 (secure 0 0 0) 0x7fc8d805fc90 con 0x7fc8f00d1ff0 2024-04-25T10:30:52.226 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:52.224+0000 7fc8f6630640 1 -- 172.21.15.37:0/2840650853 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7fc8f00d1ba0 con 0x7fc8f00d1ff0 2024-04-25T10:30:52.226 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:52.225+0000 7fc8ecff9640 1 -- 172.21.15.37:0/2840650853 <== mon.0 v2:172.21.15.37:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v46) ==== 74+0+18876 (secure 0 0 0) 0x7fc8d805f590 con 0x7fc8f00d1ff0 2024-04-25T10:30:52.227 INFO:teuthology.orchestra.run.smithi037.stdout: 2024-04-25T10:30:52.227 INFO:teuthology.orchestra.run.smithi037.stdout:{"epoch":46,"fsid":"f4acda28-be34-492a-b286-994afd36f61d","created":"2024-04-25T10:28:43.985718+0000","modified":"2024-04-25T10:30:49.793339+0000","last_up_change":"2024-04-25T10:28:54.900654+0000","last_in_change":"2024-04-25T10:28:46.216973+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":6,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"luminous","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-04-25T10:28:56.202153+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"19","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":4,"pool_name":"cephfs_metadata","create_time":"2024-04-25T10:30:10.008132+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"35","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.75,"score_stable":1.75,"optimal_score":1,"raw_score_acting":1.75,"raw_score_stable":1.75,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":5,"pool_name":"cephfs_data","create_time":"2024-04-25T10:30:11.084827+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"43","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"recovery_fs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.3799999952316284,"score_stable":1.3799999952316284,"optimal_score":1,"raw_score_acting":1.3799999952316284,"raw_score_stable":1.3799999952316284,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":6,"pool_name":"recovery_fs_metadata","create_time":"2024-04-25T10:30:26.903737+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"44","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"recovery_fs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"a2f79028-b332-4166-9ebb-a7024a37ea22","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":42,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.37:6816","nonce":2746074720},{"type":"v1","addr":"172.21.15.37:6817","nonce":2746074720}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.37:6818","nonce":2746074720},{"type":"v1","addr":"172.21.15.37:6819","nonce":2746074720}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.37:6822","nonce":2746074720},{"type":"v1","addr":"172.21.15.37:6823","nonce":2746074720}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.37:6820","nonce":2746074720},{"type":"v1","addr":"172.21.15.37:6821","nonce":2746074720}]},"public_addr":"172.21.15.37:6817/2746074720","cluster_addr":"172.21.15.37:6819/2746074720","heartbeat_back_addr":"172.21.15.37:6823/2746074720","heartbeat_front_addr":"172.21.15.37:6821/2746074720","state":["exists","up"]},{"osd":1,"uuid":"973159de-c2ea-4083-aecc-69373d1c1132","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":45,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.37:6800","nonce":1690341662},{"type":"v1","addr":"172.21.15.37:6801","nonce":1690341662}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.37:6802","nonce":1690341662},{"type":"v1","addr":"172.21.15.37:6803","nonce":1690341662}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.37:6806","nonce":1690341662},{"type":"v1","addr":"172.21.15.37:6807","nonce":1690341662}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.37:6804","nonce":1690341662},{"type":"v1","addr":"172.21.15.37:6805","nonce":1690341662}]},"public_addr":"172.21.15.37:6801/1690341662","cluster_addr":"172.21.15.37:6803/1690341662","heartbeat_back_addr":"172.21.15.37:6807/1690341662","heartbeat_front_addr":"172.21.15.37:6805/1690341662","state":["exists","up"]},{"osd":2,"uuid":"4ae7a0e9-cb3f-484c-a75f-f3766b721c47","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":42,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.37:6808","nonce":2754669649},{"type":"v1","addr":"172.21.15.37:6809","nonce":2754669649}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.37:6810","nonce":2754669649},{"type":"v1","addr":"172.21.15.37:6811","nonce":2754669649}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.37:6814","nonce":2754669649},{"type":"v1","addr":"172.21.15.37:6815","nonce":2754669649}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.37:6812","nonce":2754669649},{"type":"v1","addr":"172.21.15.37:6813","nonce":2754669649}]},"public_addr":"172.21.15.37:6809/2754669649","cluster_addr":"172.21.15.37:6811/2754669649","heartbeat_back_addr":"172.21.15.37:6815/2754669649","heartbeat_front_addr":"172.21.15.37:6813/2754669649","state":["exists","up"]},{"osd":3,"uuid":"1ba4a746-1608-47a7-ace2-960a5cfae934","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":45,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.37:6824","nonce":1815290618},{"type":"v1","addr":"172.21.15.37:6825","nonce":1815290618}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.37:6826","nonce":1815290618},{"type":"v1","addr":"172.21.15.37:6827","nonce":1815290618}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.37:6830","nonce":1815290618},{"type":"v1","addr":"172.21.15.37:6831","nonce":1815290618}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.37:6828","nonce":1815290618},{"type":"v1","addr":"172.21.15.37:6829","nonce":1815290618}]},"public_addr":"172.21.15.37:6825/1815290618","cluster_addr":"172.21.15.37:6827/1815290618","heartbeat_back_addr":"172.21.15.37:6831/1815290618","heartbeat_front_addr":"172.21.15.37:6829/1815290618","state":["exists","up"]},{"osd":4,"uuid":"d4f59f5d-0413-4b79-87ee-bb212c1fcf62","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":42,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6800","nonce":4051532063},{"type":"v1","addr":"172.21.15.203:6801","nonce":4051532063}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6802","nonce":4051532063},{"type":"v1","addr":"172.21.15.203:6803","nonce":4051532063}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6806","nonce":4051532063},{"type":"v1","addr":"172.21.15.203:6807","nonce":4051532063}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6804","nonce":4051532063},{"type":"v1","addr":"172.21.15.203:6805","nonce":4051532063}]},"public_addr":"172.21.15.203:6801/4051532063","cluster_addr":"172.21.15.203:6803/4051532063","heartbeat_back_addr":"172.21.15.203:6807/4051532063","heartbeat_front_addr":"172.21.15.203:6805/4051532063","state":["exists","up"]},{"osd":5,"uuid":"69710a0e-41d8-4e7f-9a10-17ce540b67be","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":45,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6824","nonce":1203822890},{"type":"v1","addr":"172.21.15.203:6825","nonce":1203822890}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6826","nonce":1203822890},{"type":"v1","addr":"172.21.15.203:6827","nonce":1203822890}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6830","nonce":1203822890},{"type":"v1","addr":"172.21.15.203:6831","nonce":1203822890}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6828","nonce":1203822890},{"type":"v1","addr":"172.21.15.203:6829","nonce":1203822890}]},"public_addr":"172.21.15.203:6825/1203822890","cluster_addr":"172.21.15.203:6827/1203822890","heartbeat_back_addr":"172.21.15.203:6831/1203822890","heartbeat_front_addr":"172.21.15.203:6829/1203822890","state":["exists","up"]},{"osd":6,"uuid":"f29dd047-3b48-473d-a235-6642a6293f20","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":42,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6810","nonce":2309615014},{"type":"v1","addr":"172.21.15.203:6812","nonce":2309615014}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6814","nonce":2309615014},{"type":"v1","addr":"172.21.15.203:6816","nonce":2309615014}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6822","nonce":2309615014},{"type":"v1","addr":"172.21.15.203:6823","nonce":2309615014}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6818","nonce":2309615014},{"type":"v1","addr":"172.21.15.203:6820","nonce":2309615014}]},"public_addr":"172.21.15.203:6812/2309615014","cluster_addr":"172.21.15.203:6816/2309615014","heartbeat_back_addr":"172.21.15.203:6823/2309615014","heartbeat_front_addr":"172.21.15.203:6820/2309615014","state":["exists","up"]},{"osd":7,"uuid":"6e691644-378b-4fdc-8e86-fb5f56e31f29","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":42,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6808","nonce":3688099014},{"type":"v1","addr":"172.21.15.203:6809","nonce":3688099014}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6811","nonce":3688099014},{"type":"v1","addr":"172.21.15.203:6813","nonce":3688099014}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6819","nonce":3688099014},{"type":"v1","addr":"172.21.15.203:6821","nonce":3688099014}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6815","nonce":3688099014},{"type":"v1","addr":"172.21.15.203:6817","nonce":3688099014}]},"public_addr":"172.21.15.203:6809/3688099014","cluster_addr":"172.21.15.203:6813/3688099014","heartbeat_back_addr":"172.21.15.203:6821/3688099014","heartbeat_front_addr":"172.21.15.203:6817/3688099014","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T10:28:51.419141+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T10:28:51.366448+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T10:28:51.348276+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T10:28:51.380519+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T10:28:51.105812+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T10:28:51.384496+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T10:28:51.388755+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T10:28:51.366958+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[{"pgid":"5.15","mappings":[{"from":4,"to":5}]},{"pgid":"5.1a","mappings":[{"from":4,"to":5}]},{"pgid":"5.26","mappings":[{"from":4,"to":5}]},{"pgid":"5.27","mappings":[{"from":4,"to":5}]}],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.37:6835/311552410":"2024-04-26T10:30:24.853732+0000","172.21.15.37:6832/311552410":"2024-04-26T10:30:24.853732+0000"},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-04-25T10:30:52.229 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:52.227+0000 7fc8f6630640 1 -- 172.21.15.37:0/2840650853 >> [v2:172.21.15.203:6832/2418205237,v1:172.21.15.203:6833/2418205237] conn(0x7fc8d00744b0 msgr2=0x7fc8d00769c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:30:52.229 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:52.227+0000 7fc8f6630640 1 --2- 172.21.15.37:0/2840650853 >> [v2:172.21.15.203:6832/2418205237,v1:172.21.15.203:6833/2418205237] conn(0x7fc8d00744b0 0x7fc8d00769c0 secure :-1 s=READY pgs=292 cs=0 l=1 rev1=1 crypto rx=0x7fc8e00099a0 tx=0x7fc8e0008040 comp rx=0 tx=0).stop 2024-04-25T10:30:52.229 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:52.227+0000 7fc8f6630640 1 -- 172.21.15.37:0/2840650853 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fc8f00d1ff0 msgr2=0x7fc8f015dc00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:30:52.229 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:52.227+0000 7fc8f6630640 1 --2- 172.21.15.37:0/2840650853 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fc8f00d1ff0 0x7fc8f015dc00 secure :-1 s=READY pgs=642 cs=0 l=1 rev1=1 crypto rx=0x7fc8d800e8a0 tx=0x7fc8d800ed60 comp rx=0 tx=0).stop 2024-04-25T10:30:52.229 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:52.227+0000 7fc8f6630640 1 -- 172.21.15.37:0/2840650853 shutdown_connections 2024-04-25T10:30:52.229 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:52.227+0000 7fc8f6630640 1 --2- 172.21.15.37:0/2840650853 >> [v2:172.21.15.203:6832/2418205237,v1:172.21.15.203:6833/2418205237] conn(0x7fc8d00744b0 0x7fc8d00769c0 unknown :-1 s=CLOSED pgs=292 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:30:52.229 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:52.227+0000 7fc8f6630640 1 --2- 172.21.15.37:0/2840650853 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7fc8f01625b0 0x7fc8f015fd10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:30:52.229 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:52.227+0000 7fc8f6630640 1 --2- 172.21.15.37:0/2840650853 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fc8f00d1ff0 0x7fc8f015dc00 unknown :-1 s=CLOSED pgs=642 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:30:52.230 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:52.227+0000 7fc8f6630640 1 --2- 172.21.15.37:0/2840650853 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fc8f00b3c80 0x7fc8f006e880 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:30:52.230 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:52.227+0000 7fc8f6630640 1 -- 172.21.15.37:0/2840650853 >> 172.21.15.37:0/2840650853 conn(0x7fc8f00d0390 msgr2=0x7fc8f0072b60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:30:52.230 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:52.228+0000 7fc8f6630640 1 -- 172.21.15.37:0/2840650853 shutdown_connections 2024-04-25T10:30:52.230 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:52.228+0000 7fc8f6630640 1 -- 172.21.15.37:0/2840650853 wait complete. 2024-04-25T10:30:52.240 INFO:tasks.cephfs.filesystem:Destroying file system cephfs and related pools 2024-04-25T10:30:52.240 DEBUG:teuthology.orchestra.run.smithi037:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-04-25T10:30:52.352 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:52.350+0000 7f1e8aeee640 1 Processor -- start 2024-04-25T10:30:52.353 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:52.350+0000 7f1e8aeee640 1 -- start start 2024-04-25T10:30:52.356 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:52.351+0000 7f1e8aeee640 1 --2- >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f1e840b7600 0x7f1e840b7a10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:30:52.356 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:52.351+0000 7f1e8aeee640 1 -- --> v1:172.21.15.37:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f1e840b7f50 con 0x7f1e840d4560 2024-04-25T10:30:52.356 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:52.352+0000 7f1e8aeee640 1 -- --> v1:172.21.15.203:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f1e840d5080 con 0x7f1e840b4bb0 2024-04-25T10:30:52.356 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:52.352+0000 7f1e8aeee640 1 -- --> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] -- mon_getmap magic: 0 -- 0x7f1e84161190 con 0x7f1e840b7600 2024-04-25T10:30:52.356 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:52.352+0000 7f1e8948b640 1 --1- >> v1:172.21.15.37:6789/0 conn(0x7f1e840d4560 0x7f1e840d4970 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.37:6789/0 says I am v1:172.21.15.37:38714/0 (socket says 172.21.15.37:38714) 2024-04-25T10:30:52.356 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:52.352+0000 7f1e8948b640 1 -- 172.21.15.37:0/3129608009 learned_addr learned my addr 172.21.15.37:0/3129608009 (peer_addr_for_me v1:172.21.15.37:0/0) 2024-04-25T10:30:52.356 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:52.352+0000 7f1e83fff640 1 --2- 172.21.15.37:0/3129608009 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f1e840b7600 0x7f1e840b7a10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:30:52.356 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:52.352+0000 7f1e837fe640 1 -- 172.21.15.37:0/3129608009 <== mon.0 v1:172.21.15.37:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 965017194 0 0) 0x7f1e74001110 con 0x7f1e840d4560 2024-04-25T10:30:52.356 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:52.352+0000 7f1e837fe640 1 -- 172.21.15.37:0/3129608009 --> v1:172.21.15.37:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f1e6c003520 con 0x7f1e840d4560 2024-04-25T10:30:52.356 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:52.352+0000 7f1e837fe640 1 -- 172.21.15.37:0/3129608009 <== mon.1 v1:172.21.15.203:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 4002519042 0 0) 0x7f1e840d5080 con 0x7f1e840b4bb0 2024-04-25T10:30:52.356 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:52.352+0000 7f1e837fe640 1 -- 172.21.15.37:0/3129608009 --> v1:172.21.15.203:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f1e74001110 con 0x7f1e840b4bb0 2024-04-25T10:30:52.357 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:52.352+0000 7f1e837fe640 1 -- 172.21.15.37:0/3129608009 <== mon.0 v1:172.21.15.37:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 1892761618 0 0) 0x7f1e6c003520 con 0x7f1e840d4560 2024-04-25T10:30:52.357 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:52.352+0000 7f1e837fe640 1 -- 172.21.15.37:0/3129608009 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f1e840b7600 msgr2=0x7f1e840b7a10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:30:52.357 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:52.352+0000 7f1e837fe640 1 --2- 172.21.15.37:0/3129608009 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f1e840b7600 0x7f1e840b7a10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:30:52.357 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:52.352+0000 7f1e837fe640 1 -- 172.21.15.37:0/3129608009 >> v1:172.21.15.203:6789/0 conn(0x7f1e840b4bb0 legacy=0x7f1e841646b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:30:52.357 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:52.352+0000 7f1e837fe640 1 -- 172.21.15.37:0/3129608009 --> v1:172.21.15.37:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f1e84161470 con 0x7f1e840d4560 2024-04-25T10:30:52.357 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:52.352+0000 7f1e837fe640 1 -- 172.21.15.37:0/3129608009 <== mon.0 v1:172.21.15.37:6789/0 3 ==== mon_map magic: 0 ==== 413+0+0 (unknown 1577794600 0 0) 0x7f1e74001840 con 0x7f1e840d4560 2024-04-25T10:30:52.357 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:52.352+0000 7f1e83fff640 1 --2- 172.21.15.37:0/3129608009 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f1e840b7600 0x7f1e840b7a10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-25T10:30:52.357 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:52.352+0000 7f1e837fe640 1 -- 172.21.15.37:0/3129608009 >> v1:172.21.15.37:6789/0 conn(0x7f1e840d4560 legacy=0x7f1e840d4970 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:30:52.357 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:52.352+0000 7f1e837fe640 1 --2- 172.21.15.37:0/3129608009 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f1e6c003b70 0x7f1e6c003fc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:30:52.357 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:52.352+0000 7f1e837fe640 1 --2- 172.21.15.37:0/3129608009 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f1e840b7600 0x7f1e6c004800 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:30:52.357 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:52.352+0000 7f1e837fe640 1 --2- 172.21.15.37:0/3129608009 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f1e6c004d40 0x7f1e6c005150 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:30:52.357 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:52.352+0000 7f1e837fe640 1 -- 172.21.15.37:0/3129608009 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 -- 0x7f1e84161190 con 0x7f1e6c004d40 2024-04-25T10:30:52.357 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:52.352+0000 7f1e837fe640 1 -- 172.21.15.37:0/3129608009 --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_getmap magic: 0 -- 0x7f1e6c00bad0 con 0x7f1e6c003b70 2024-04-25T10:30:52.357 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:52.352+0000 7f1e837fe640 1 -- 172.21.15.37:0/3129608009 --> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] -- mon_getmap magic: 0 -- 0x7f1e6c00bcd0 con 0x7f1e840b7600 2024-04-25T10:30:52.357 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:52.352+0000 7f1e8948b640 1 --2- 172.21.15.37:0/3129608009 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f1e6c004d40 0x7f1e6c005150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:30:52.358 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:52.353+0000 7f1e83fff640 1 --2- 172.21.15.37:0/3129608009 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f1e840b7600 0x7f1e6c004800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:30:52.358 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:52.353+0000 7f1e88c8a640 1 --2- 172.21.15.37:0/3129608009 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f1e6c003b70 0x7f1e6c003fc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:30:52.358 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:52.353+0000 7f1e8948b640 1 -- 172.21.15.37:0/3129608009 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f1e840b7600 msgr2=0x7f1e6c004800 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:30:52.358 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:52.353+0000 7f1e8948b640 1 --2- 172.21.15.37:0/3129608009 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f1e840b7600 0x7f1e6c004800 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:30:52.358 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:52.353+0000 7f1e8948b640 1 -- 172.21.15.37:0/3129608009 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f1e6c003b70 msgr2=0x7f1e6c003fc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:30:52.358 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:52.353+0000 7f1e8948b640 1 --2- 172.21.15.37:0/3129608009 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f1e6c003b70 0x7f1e6c003fc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:30:52.358 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:52.353+0000 7f1e8948b640 1 -- 172.21.15.37:0/3129608009 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f1e6c003520 con 0x7f1e6c004d40 2024-04-25T10:30:52.358 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:52.354+0000 7f1e8948b640 1 --2- 172.21.15.37:0/3129608009 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f1e6c004d40 0x7f1e6c005150 secure :-1 s=READY pgs=644 cs=0 l=1 rev1=1 crypto rx=0x7f1e7400c490 tx=0x7f1e6c005830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b70197f9b73da52d server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:30:52.358 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:52.354+0000 7f1e837fe640 1 -- 172.21.15.37:0/3129608009 <== mon.0 v2:172.21.15.37:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1e74035070 con 0x7f1e6c004d40 2024-04-25T10:30:52.358 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:52.354+0000 7f1e837fe640 1 -- 172.21.15.37:0/3129608009 <== mon.0 v2:172.21.15.37:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f1e740058d0 con 0x7f1e6c004d40 2024-04-25T10:30:52.358 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:52.354+0000 7f1e837fe640 1 -- 172.21.15.37:0/3129608009 <== mon.0 v2:172.21.15.37:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1e7400c0e0 con 0x7f1e6c004d40 2024-04-25T10:30:52.361 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:52.354+0000 7f1e8aeee640 1 -- 172.21.15.37:0/3129608009 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f1e6c004d40 msgr2=0x7f1e6c005150 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:30:52.361 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:52.354+0000 7f1e8aeee640 1 --2- 172.21.15.37:0/3129608009 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f1e6c004d40 0x7f1e6c005150 secure :-1 s=READY pgs=644 cs=0 l=1 rev1=1 crypto rx=0x7f1e7400c490 tx=0x7f1e6c005830 comp rx=0 tx=0).stop 2024-04-25T10:30:52.361 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:52.354+0000 7f1e8948b640 1 -- 172.21.15.37:0/3129608009 reap_dead start 2024-04-25T10:30:52.361 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:52.354+0000 7f1e8aeee640 1 -- 172.21.15.37:0/3129608009 shutdown_connections 2024-04-25T10:30:52.361 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:52.354+0000 7f1e8aeee640 1 -- 172.21.15.37:0/3129608009 >> 172.21.15.37:0/3129608009 conn(0x7f1e840d27e0 msgr2=0x7f1e840d2bf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:30:52.362 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:52.354+0000 7f1e8aeee640 1 -- 172.21.15.37:0/3129608009 shutdown_connections 2024-04-25T10:30:52.362 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:52.354+0000 7f1e8aeee640 1 -- 172.21.15.37:0/3129608009 wait complete. 2024-04-25T10:30:52.362 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:52.356+0000 7f1e8aeee640 1 Processor -- start 2024-04-25T10:30:52.362 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:52.356+0000 7f1e8aeee640 1 -- start start 2024-04-25T10:30:52.362 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:52.358+0000 7f1e8aeee640 1 --2- >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f1e841f7fb0 0x7f1e840cd970 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:30:52.362 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:52.358+0000 7f1e8aeee640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f1e841f9d90 0x7f1e841f7870 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:30:52.362 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:52.358+0000 7f1e8aeee640 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f1e840b54c0 0x7f1e840b58d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:30:52.362 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:52.358+0000 7f1e8aeee640 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 -- 0x7f1e840b4fc0 con 0x7f1e841f9d90 2024-04-25T10:30:52.362 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:52.358+0000 7f1e8aeee640 1 -- --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_getmap magic: 0 -- 0x7f1e84164190 con 0x7f1e840b54c0 2024-04-25T10:30:52.362 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:52.358+0000 7f1e8aeee640 1 -- --> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] -- mon_getmap magic: 0 -- 0x7f1e840d09e0 con 0x7f1e841f7fb0 2024-04-25T10:30:52.362 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:52.358+0000 7f1e83fff640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f1e841f9d90 0x7f1e841f7870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:30:52.362 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:52.358+0000 7f1e83fff640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f1e841f9d90 0x7f1e841f7870 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.37:3300/0 says I am v2:172.21.15.37:59468/0 (socket says 172.21.15.37:59468) 2024-04-25T10:30:52.362 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:52.358+0000 7f1e83fff640 1 -- 172.21.15.37:0/3115933321 learned_addr learned my addr 172.21.15.37:0/3115933321 (peer_addr_for_me v2:172.21.15.37:0/0) 2024-04-25T10:30:52.362 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:52.358+0000 7f1e8948b640 1 --2- 172.21.15.37:0/3115933321 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f1e840b54c0 0x7f1e840b58d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:30:52.363 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:52.358+0000 7f1e88c8a640 1 --2- 172.21.15.37:0/3115933321 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f1e841f7fb0 0x7f1e840cd970 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:30:52.363 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:52.358+0000 7f1e83fff640 1 -- 172.21.15.37:0/3115933321 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f1e841f7fb0 msgr2=0x7f1e840cd970 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:30:52.363 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:52.358+0000 7f1e83fff640 1 --2- 172.21.15.37:0/3115933321 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f1e841f7fb0 0x7f1e840cd970 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:30:52.363 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:52.358+0000 7f1e83fff640 1 -- 172.21.15.37:0/3115933321 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f1e840b54c0 msgr2=0x7f1e840b58d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:30:52.363 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:52.358+0000 7f1e83fff640 1 --2- 172.21.15.37:0/3115933321 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f1e840b54c0 0x7f1e840b58d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:30:52.363 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:52.358+0000 7f1e83fff640 1 -- 172.21.15.37:0/3115933321 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f1e841fa1a0 con 0x7f1e841f9d90 2024-04-25T10:30:52.363 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:52.358+0000 7f1e8948b640 1 --2- 172.21.15.37:0/3115933321 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f1e840b54c0 0x7f1e840b58d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T10:30:52.363 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:52.358+0000 7f1e83fff640 1 --2- 172.21.15.37:0/3115933321 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f1e841f9d90 0x7f1e841f7870 secure :-1 s=READY pgs=645 cs=0 l=1 rev1=1 crypto rx=0x7f1e78011a00 tx=0x7f1e78011ec0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:30:52.363 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:52.359+0000 7f1e817fa640 1 -- 172.21.15.37:0/3115933321 <== mon.0 v2:172.21.15.37:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1e7800faf0 con 0x7f1e841f9d90 2024-04-25T10:30:52.363 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:52.359+0000 7f1e817fa640 1 -- 172.21.15.37:0/3115933321 <== mon.0 v2:172.21.15.37:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f1e7800fc90 con 0x7f1e841f9d90 2024-04-25T10:30:52.363 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:52.359+0000 7f1e817fa640 1 -- 172.21.15.37:0/3115933321 <== mon.0 v2:172.21.15.37:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1e78004030 con 0x7f1e841f9d90 2024-04-25T10:30:52.363 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:52.359+0000 7f1e8aeee640 1 -- 172.21.15.37:0/3115933321 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f1e841f7db0 con 0x7f1e841f9d90 2024-04-25T10:30:52.363 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:52.359+0000 7f1e8aeee640 1 -- 172.21.15.37:0/3115933321 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f1e840cdf50 con 0x7f1e841f9d90 2024-04-25T10:30:52.364 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:52.360+0000 7f1e8aeee640 1 -- 172.21.15.37:0/3115933321 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f1e84163c60 con 0x7f1e841f9d90 2024-04-25T10:30:52.364 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:52.360+0000 7f1e817fa640 1 -- 172.21.15.37:0/3115933321 <== mon.0 v2:172.21.15.37:3300/0 4 ==== mgrmap(e 5) ==== 96782+0+0 (secure 0 0 0) 0x7f1e78012040 con 0x7f1e841f9d90 2024-04-25T10:30:52.364 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:52.361+0000 7f1e817fa640 1 --2- 172.21.15.37:0/3115933321 >> [v2:172.21.15.203:6832/2418205237,v1:172.21.15.203:6833/2418205237] conn(0x7f1e700744b0 0x7f1e700769c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:30:52.364 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:52.361+0000 7f1e817fa640 1 -- 172.21.15.37:0/3115933321 <== mon.0 v2:172.21.15.37:3300/0 5 ==== osd_map(46..46 src has 1..46) ==== 6364+0+0 (secure 0 0 0) 0x7f1e78095740 con 0x7f1e841f9d90 2024-04-25T10:30:52.365 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:52.361+0000 7f1e88c8a640 1 --2- 172.21.15.37:0/3115933321 >> [v2:172.21.15.203:6832/2418205237,v1:172.21.15.203:6833/2418205237] conn(0x7f1e700744b0 0x7f1e700769c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:30:52.365 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:52.362+0000 7f1e88c8a640 1 --2- 172.21.15.37:0/3115933321 >> [v2:172.21.15.203:6832/2418205237,v1:172.21.15.203:6833/2418205237] conn(0x7f1e700744b0 0x7f1e700769c0 secure :-1 s=READY pgs=293 cs=0 l=1 rev1=1 crypto rx=0x7f1e841602f0 tx=0x7f1e7c009040 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:30:52.367 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:52.365+0000 7f1e817fa640 1 -- 172.21.15.37:0/3115933321 <== mon.0 v2:172.21.15.37:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187246 (secure 0 0 0) 0x7f1e84163c60 con 0x7f1e841f9d90 2024-04-25T10:30:52.589 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:52.587+0000 7f1e8aeee640 1 -- 172.21.15.37:0/3115933321 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f1e8406a140 con 0x7f1e841f9d90 2024-04-25T10:30:52.590 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:52.588+0000 7f1e817fa640 1 -- 172.21.15.37:0/3115933321 <== mon.0 v2:172.21.15.37:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 27 v27) ==== 94+0+6489 (secure 0 0 0) 0x7f1e841f7db0 con 0x7f1e841f9d90 2024-04-25T10:30:52.590 INFO:teuthology.orchestra.run.smithi037.stdout: 2024-04-25T10:30:52.590 INFO:teuthology.orchestra.run.smithi037.stdout:{"epoch":27,"default_fscid":2,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":4946,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.37:6835/683655741","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.37:6834","nonce":683655741},{"type":"v1","addr":"172.21.15.37:6835","nonce":683655741}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":19},{"gid":5215,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.203:6836/3927045001","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6835","nonce":3927045001},{"type":"v1","addr":"172.21.15.203:6836","nonce":3927045001}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":19},{"gid":5218,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.203:6837/3108183946","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6834","nonce":3108183946},{"type":"v1","addr":"172.21.15.203:6837","nonce":3108183946}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":19}],"filesystems":[{"mdsmap":{"epoch":18,"flags":19,"flags_state":{"joinable":false,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-04-25T10:30:12.130473+0000","modified":"2024-04-25T10:30:24.853915+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":37,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":1,"in":[0],"up":{},"failed":[0],"damaged":[],"stopped":[],"info":{},"data_pools":[5],"metadata_pool":4,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":0,"qdb_cluster":[]},"id":2},{"mdsmap":{"epoch":27,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-04-25T10:30:28.588617+0000","modified":"2024-04-25T10:30:42.935167+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":1,"in":[0],"up":{"mds_0":5625},"failed":[],"damaged":[],"stopped":[],"info":{"gid_5625":{"gid":5625,"name":"c","rank":0,"incarnation":24,"state":"up:active","state_seq":7,"addr":"172.21.15.37:6833/2353203722","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.37:6832","nonce":2353203722},{"type":"v1","addr":"172.21.15.37:6833","nonce":2353203722}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[5],"metadata_pool":6,"enabled":true,"fs_name":"recovery_fs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":5625,"qdb_cluster":[5625]},"id":3}]} 2024-04-25T10:30:52.591 INFO:teuthology.orchestra.run.smithi037.stderr:dumped fsmap epoch 27 2024-04-25T10:30:52.592 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:52.590+0000 7f1e8aeee640 1 -- 172.21.15.37:0/3115933321 >> [v2:172.21.15.203:6832/2418205237,v1:172.21.15.203:6833/2418205237] conn(0x7f1e700744b0 msgr2=0x7f1e700769c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:30:52.592 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:52.590+0000 7f1e8aeee640 1 --2- 172.21.15.37:0/3115933321 >> [v2:172.21.15.203:6832/2418205237,v1:172.21.15.203:6833/2418205237] conn(0x7f1e700744b0 0x7f1e700769c0 secure :-1 s=READY pgs=293 cs=0 l=1 rev1=1 crypto rx=0x7f1e841602f0 tx=0x7f1e7c009040 comp rx=0 tx=0).stop 2024-04-25T10:30:52.592 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:52.590+0000 7f1e8aeee640 1 -- 172.21.15.37:0/3115933321 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f1e841f9d90 msgr2=0x7f1e841f7870 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:30:52.592 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:52.590+0000 7f1e8aeee640 1 --2- 172.21.15.37:0/3115933321 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f1e841f9d90 0x7f1e841f7870 secure :-1 s=READY pgs=645 cs=0 l=1 rev1=1 crypto rx=0x7f1e78011a00 tx=0x7f1e78011ec0 comp rx=0 tx=0).stop 2024-04-25T10:30:52.593 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:52.590+0000 7f1e8aeee640 1 -- 172.21.15.37:0/3115933321 shutdown_connections 2024-04-25T10:30:52.593 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:52.590+0000 7f1e8aeee640 1 --2- 172.21.15.37:0/3115933321 >> [v2:172.21.15.203:6832/2418205237,v1:172.21.15.203:6833/2418205237] conn(0x7f1e700744b0 0x7f1e700769c0 unknown :-1 s=CLOSED pgs=293 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:30:52.593 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:52.590+0000 7f1e8aeee640 1 --2- 172.21.15.37:0/3115933321 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f1e840b54c0 0x7f1e840b58d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:30:52.593 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:52.590+0000 7f1e8aeee640 1 --2- 172.21.15.37:0/3115933321 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f1e841f9d90 0x7f1e841f7870 unknown :-1 s=CLOSED pgs=645 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:30:52.593 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:52.590+0000 7f1e8aeee640 1 --2- 172.21.15.37:0/3115933321 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f1e841f7fb0 0x7f1e840cd970 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:30:52.593 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:52.590+0000 7f1e8aeee640 1 -- 172.21.15.37:0/3115933321 >> 172.21.15.37:0/3115933321 conn(0x7f1e840d27e0 msgr2=0x7f1e84163120 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:30:52.593 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:52.590+0000 7f1e8aeee640 1 -- 172.21.15.37:0/3115933321 shutdown_connections 2024-04-25T10:30:52.593 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:52.590+0000 7f1e8aeee640 1 -- 172.21.15.37:0/3115933321 wait complete. 2024-04-25T10:30:52.604 DEBUG:teuthology.orchestra.run.smithi037:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-04-25T10:30:52.717 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:52.715+0000 7f2204f4d640 1 Processor -- start 2024-04-25T10:30:52.717 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:52.715+0000 7f2204f4d640 1 -- start start 2024-04-25T10:30:52.719 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:52.716+0000 7f2204f4d640 1 --2- >> v2:172.21.15.37:3300/0 conn(0x7f22000b4bb0 0x7f22001646b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:30:52.719 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:52.716+0000 7f2204f4d640 1 --2- >> v2:172.21.15.203:3300/0 conn(0x7f22000d4560 0x7f22000d4970 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:30:52.719 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:52.716+0000 7f2204f4d640 1 -- --> v1:172.21.15.203:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f2200161190 con 0x7f22000b7600 2024-04-25T10:30:52.719 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:52.716+0000 7f2204f4d640 1 -- --> v2:172.21.15.37:3300/0 -- mon_getmap magic: 0 -- 0x7f2200161470 con 0x7f22000b4bb0 2024-04-25T10:30:52.719 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:52.716+0000 7f2204f4d640 1 -- --> v2:172.21.15.203:3300/0 -- mon_getmap magic: 0 -- 0x7f2200163eb0 con 0x7f22000d4560 2024-04-25T10:30:52.719 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:52.716+0000 7f21fe59c640 1 --2- >> v2:172.21.15.37:3300/0 conn(0x7f22000b4bb0 0x7f22001646b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:30:52.719 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:52.716+0000 7f21fe59c640 1 --2- >> v2:172.21.15.37:3300/0 conn(0x7f22000b4bb0 0x7f22001646b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.37:3300/0 says I am v2:172.21.15.37:59482/0 (socket says 172.21.15.37:59482) 2024-04-25T10:30:52.719 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:52.716+0000 7f21fe59c640 1 -- 172.21.15.37:0/2339346437 learned_addr learned my addr 172.21.15.37:0/2339346437 (peer_addr_for_me v2:172.21.15.37:0/0) 2024-04-25T10:30:52.719 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:52.716+0000 7f21fed9d640 1 --2- 172.21.15.37:0/2339346437 >> v2:172.21.15.203:3300/0 conn(0x7f22000d4560 0x7f22000d4970 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:30:52.719 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:52.717+0000 7f21fe59c640 1 -- 172.21.15.37:0/2339346437 >> v2:172.21.15.203:3300/0 conn(0x7f22000d4560 msgr2=0x7f22000d4970 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:30:52.720 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:52.717+0000 7f21fe59c640 1 --2- 172.21.15.37:0/2339346437 >> v2:172.21.15.203:3300/0 conn(0x7f22000d4560 0x7f22000d4970 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:30:52.720 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:52.717+0000 7f21fe59c640 1 -- 172.21.15.37:0/2339346437 >> v1:172.21.15.203:6789/0 conn(0x7f22000b7600 legacy=0x7f22000b7a10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:30:52.720 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:52.717+0000 7f21fe59c640 1 -- 172.21.15.37:0/2339346437 --> v2:172.21.15.37:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f2200164190 con 0x7f22000b4bb0 2024-04-25T10:30:52.721 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:52.717+0000 7f21fe59c640 1 --2- 172.21.15.37:0/2339346437 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f22000b4bb0 0x7f22001646b0 secure :-1 s=READY pgs=646 cs=0 l=1 rev1=1 crypto rx=0x7f22000d3b00 tx=0x7f21f802a270 comp rx=0 tx=0).ready entity=mon.0 client_cookie=dc927440b3d65088 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:30:52.721 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:52.718+0000 7f21fd59a640 1 -- 172.21.15.37:0/2339346437 <== mon.0 v2:172.21.15.37:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f21f8037070 con 0x7f22000b4bb0 2024-04-25T10:30:52.721 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:52.718+0000 7f21fd59a640 1 -- 172.21.15.37:0/2339346437 <== mon.0 v2:172.21.15.37:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f21f8032440 con 0x7f22000b4bb0 2024-04-25T10:30:52.721 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:52.718+0000 7f21fd59a640 1 -- 172.21.15.37:0/2339346437 <== mon.0 v2:172.21.15.37:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f21f8032810 con 0x7f22000b4bb0 2024-04-25T10:30:52.721 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:52.718+0000 7f2204f4d640 1 -- 172.21.15.37:0/2339346437 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f22000b4bb0 msgr2=0x7f22001646b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:30:52.721 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:52.718+0000 7f2204f4d640 1 --2- 172.21.15.37:0/2339346437 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f22000b4bb0 0x7f22001646b0 secure :-1 s=READY pgs=646 cs=0 l=1 rev1=1 crypto rx=0x7f22000d3b00 tx=0x7f21f802a270 comp rx=0 tx=0).stop 2024-04-25T10:30:52.721 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:52.718+0000 7f2204f4d640 1 -- 172.21.15.37:0/2339346437 shutdown_connections 2024-04-25T10:30:52.721 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:52.718+0000 7f2204f4d640 1 --2- 172.21.15.37:0/2339346437 >> v2:172.21.15.203:3300/0 conn(0x7f22000d4560 0x7f22000d4970 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:30:52.721 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:52.718+0000 7f2204f4d640 1 --2- 172.21.15.37:0/2339346437 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f22000b4bb0 0x7f22001646b0 unknown :-1 s=CLOSED pgs=646 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:30:52.722 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:52.718+0000 7f2204f4d640 1 -- 172.21.15.37:0/2339346437 >> 172.21.15.37:0/2339346437 conn(0x7f22000d27e0 msgr2=0x7f22000d2bf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:30:52.722 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:52.718+0000 7f2204f4d640 1 -- 172.21.15.37:0/2339346437 shutdown_connections 2024-04-25T10:30:52.722 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:52.718+0000 7f2204f4d640 1 -- 172.21.15.37:0/2339346437 wait complete. 2024-04-25T10:30:52.722 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:52.720+0000 7f2204f4d640 1 Processor -- start 2024-04-25T10:30:52.722 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:52.720+0000 7f2204f4d640 1 -- start start 2024-04-25T10:30:52.725 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:52.722+0000 7f2204f4d640 1 --2- >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f22000b4bb0 0x7f22001530d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:30:52.725 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:52.722+0000 7f2204f4d640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f22000b7600 0x7f22001ea610 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:30:52.725 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:52.722+0000 7f2204f4d640 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f22000d4560 0x7f22000b9160 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:30:52.725 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:52.722+0000 7f2204f4d640 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 -- 0x7f22000d09e0 con 0x7f22000b7600 2024-04-25T10:30:52.726 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:52.722+0000 7f2204f4d640 1 -- --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_getmap magic: 0 -- 0x7f2200164bf0 con 0x7f22000d4560 2024-04-25T10:30:52.726 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:52.722+0000 7f2204f4d640 1 -- --> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] -- mon_getmap magic: 0 -- 0x7f22000d0b60 con 0x7f22000b4bb0 2024-04-25T10:30:52.726 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:52.722+0000 7f21fdd9b640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f22000b7600 0x7f22001ea610 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:30:52.726 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:52.722+0000 7f21fdd9b640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f22000b7600 0x7f22001ea610 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.37:3300/0 says I am v2:172.21.15.37:59492/0 (socket says 172.21.15.37:59492) 2024-04-25T10:30:52.726 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:52.722+0000 7f21fdd9b640 1 -- 172.21.15.37:0/2026214090 learned_addr learned my addr 172.21.15.37:0/2026214090 (peer_addr_for_me v2:172.21.15.37:0/0) 2024-04-25T10:30:52.726 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:52.722+0000 7f21fe59c640 1 --2- 172.21.15.37:0/2026214090 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f22000b4bb0 0x7f22001530d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:30:52.726 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:52.722+0000 7f21fed9d640 1 --2- 172.21.15.37:0/2026214090 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f22000d4560 0x7f22000b9160 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:30:52.726 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:52.722+0000 7f21fdd9b640 1 -- 172.21.15.37:0/2026214090 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f22000b4bb0 msgr2=0x7f22001530d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:30:52.726 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:52.722+0000 7f21fdd9b640 1 --2- 172.21.15.37:0/2026214090 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f22000b4bb0 0x7f22001530d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:30:52.726 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:52.722+0000 7f21fdd9b640 1 -- 172.21.15.37:0/2026214090 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f22000d4560 msgr2=0x7f22000b9160 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:30:52.726 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:52.722+0000 7f21fdd9b640 1 --2- 172.21.15.37:0/2026214090 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f22000d4560 0x7f22000b9160 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:30:52.726 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:52.722+0000 7f21fdd9b640 1 -- 172.21.15.37:0/2026214090 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f22000ca040 con 0x7f22000b7600 2024-04-25T10:30:52.726 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:52.722+0000 7f21fed9d640 1 --2- 172.21.15.37:0/2026214090 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f22000d4560 0x7f22000b9160 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T10:30:52.727 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:52.722+0000 7f21fdd9b640 1 --2- 172.21.15.37:0/2026214090 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f22000b7600 0x7f22001ea610 secure :-1 s=READY pgs=647 cs=0 l=1 rev1=1 crypto rx=0x7f21e800c3f0 tx=0x7f21e800c8b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:30:52.727 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:52.722+0000 7f21f6ffd640 1 -- 172.21.15.37:0/2026214090 <== mon.0 v2:172.21.15.37:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f21e80052d0 con 0x7f22000b7600 2024-04-25T10:30:52.727 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:52.722+0000 7f21f6ffd640 1 -- 172.21.15.37:0/2026214090 <== mon.0 v2:172.21.15.37:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f21e8005490 con 0x7f22000b7600 2024-04-25T10:30:52.727 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:52.723+0000 7f2204f4d640 1 -- 172.21.15.37:0/2026214090 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f22000ca580 con 0x7f22000b7600 2024-04-25T10:30:52.727 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:52.723+0000 7f21f6ffd640 1 -- 172.21.15.37:0/2026214090 <== mon.0 v2:172.21.15.37:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f21e8007260 con 0x7f22000b7600 2024-04-25T10:30:52.727 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:52.723+0000 7f2204f4d640 1 -- 172.21.15.37:0/2026214090 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f22000cab40 con 0x7f22000b7600 2024-04-25T10:30:52.727 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:52.723+0000 7f2204f4d640 1 -- 172.21.15.37:0/2026214090 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f22000c90e0 con 0x7f22000b7600 2024-04-25T10:30:52.728 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:52.724+0000 7f21f6ffd640 1 -- 172.21.15.37:0/2026214090 <== mon.0 v2:172.21.15.37:3300/0 4 ==== mgrmap(e 5) ==== 96782+0+0 (secure 0 0 0) 0x7f21e8007400 con 0x7f22000b7600 2024-04-25T10:30:52.728 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:52.725+0000 7f21f6ffd640 1 --2- 172.21.15.37:0/2026214090 >> [v2:172.21.15.203:6832/2418205237,v1:172.21.15.203:6833/2418205237] conn(0x7f21e00744e0 0x7f21e00769f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:30:52.728 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:52.725+0000 7f21f6ffd640 1 -- 172.21.15.37:0/2026214090 <== mon.0 v2:172.21.15.37:3300/0 5 ==== osd_map(46..46 src has 1..46) ==== 6364+0+0 (secure 0 0 0) 0x7f21e80948d0 con 0x7f22000b7600 2024-04-25T10:30:52.728 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:52.725+0000 7f21fe59c640 1 --2- 172.21.15.37:0/2026214090 >> [v2:172.21.15.203:6832/2418205237,v1:172.21.15.203:6833/2418205237] conn(0x7f21e00744e0 0x7f21e00769f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:30:52.728 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:52.725+0000 7f21fe59c640 1 --2- 172.21.15.37:0/2026214090 >> [v2:172.21.15.203:6832/2418205237,v1:172.21.15.203:6833/2418205237] conn(0x7f21e00744e0 0x7f21e00769f0 secure :-1 s=READY pgs=294 cs=0 l=1 rev1=1 crypto rx=0x7f21f8034000 tx=0x7f21f802e000 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:30:52.731 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:52.729+0000 7f21f6ffd640 1 -- 172.21.15.37:0/2026214090 <== mon.0 v2:172.21.15.37:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187246 (secure 0 0 0) 0x7f22000c90e0 con 0x7f22000b7600 2024-04-25T10:30:52.964 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:52.961+0000 7f2204f4d640 1 -- 172.21.15.37:0/2026214090 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f22000ca310 con 0x7f22000b7600 2024-04-25T10:30:52.964 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:52.962+0000 7f21f6ffd640 1 -- 172.21.15.37:0/2026214090 <== mon.0 v2:172.21.15.37:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 27 v27) ==== 94+0+6489 (secure 0 0 0) 0x7f21e805e780 con 0x7f22000b7600 2024-04-25T10:30:52.964 INFO:teuthology.orchestra.run.smithi037.stdout: 2024-04-25T10:30:52.965 INFO:teuthology.orchestra.run.smithi037.stdout:{"epoch":27,"default_fscid":2,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":4946,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.37:6835/683655741","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.37:6834","nonce":683655741},{"type":"v1","addr":"172.21.15.37:6835","nonce":683655741}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":19},{"gid":5215,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.203:6836/3927045001","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6835","nonce":3927045001},{"type":"v1","addr":"172.21.15.203:6836","nonce":3927045001}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":19},{"gid":5218,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.203:6837/3108183946","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6834","nonce":3108183946},{"type":"v1","addr":"172.21.15.203:6837","nonce":3108183946}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":19}],"filesystems":[{"mdsmap":{"epoch":18,"flags":19,"flags_state":{"joinable":false,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-04-25T10:30:12.130473+0000","modified":"2024-04-25T10:30:24.853915+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":37,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":1,"in":[0],"up":{},"failed":[0],"damaged":[],"stopped":[],"info":{},"data_pools":[5],"metadata_pool":4,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":0,"qdb_cluster":[]},"id":2},{"mdsmap":{"epoch":27,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-04-25T10:30:28.588617+0000","modified":"2024-04-25T10:30:42.935167+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":1,"in":[0],"up":{"mds_0":5625},"failed":[],"damaged":[],"stopped":[],"info":{"gid_5625":{"gid":5625,"name":"c","rank":0,"incarnation":24,"state":"up:active","state_seq":7,"addr":"172.21.15.37:6833/2353203722","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.37:6832","nonce":2353203722},{"type":"v1","addr":"172.21.15.37:6833","nonce":2353203722}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[5],"metadata_pool":6,"enabled":true,"fs_name":"recovery_fs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":5625,"qdb_cluster":[5625]},"id":3}]} 2024-04-25T10:30:52.965 INFO:teuthology.orchestra.run.smithi037.stderr:dumped fsmap epoch 27 2024-04-25T10:30:52.967 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:52.964+0000 7f2204f4d640 1 -- 172.21.15.37:0/2026214090 >> [v2:172.21.15.203:6832/2418205237,v1:172.21.15.203:6833/2418205237] conn(0x7f21e00744e0 msgr2=0x7f21e00769f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:30:52.967 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:52.964+0000 7f2204f4d640 1 --2- 172.21.15.37:0/2026214090 >> [v2:172.21.15.203:6832/2418205237,v1:172.21.15.203:6833/2418205237] conn(0x7f21e00744e0 0x7f21e00769f0 secure :-1 s=READY pgs=294 cs=0 l=1 rev1=1 crypto rx=0x7f21f8034000 tx=0x7f21f802e000 comp rx=0 tx=0).stop 2024-04-25T10:30:52.967 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:52.964+0000 7f2204f4d640 1 -- 172.21.15.37:0/2026214090 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f22000b7600 msgr2=0x7f22001ea610 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:30:52.967 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:52.964+0000 7f2204f4d640 1 --2- 172.21.15.37:0/2026214090 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f22000b7600 0x7f22001ea610 secure :-1 s=READY pgs=647 cs=0 l=1 rev1=1 crypto rx=0x7f21e800c3f0 tx=0x7f21e800c8b0 comp rx=0 tx=0).stop 2024-04-25T10:30:52.967 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:52.964+0000 7f2204f4d640 1 -- 172.21.15.37:0/2026214090 shutdown_connections 2024-04-25T10:30:52.967 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:52.964+0000 7f2204f4d640 1 --2- 172.21.15.37:0/2026214090 >> [v2:172.21.15.203:6832/2418205237,v1:172.21.15.203:6833/2418205237] conn(0x7f21e00744e0 0x7f21e00769f0 unknown :-1 s=CLOSED pgs=294 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:30:52.967 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:52.964+0000 7f2204f4d640 1 --2- 172.21.15.37:0/2026214090 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f22000d4560 0x7f22000b9160 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:30:52.967 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:52.964+0000 7f2204f4d640 1 --2- 172.21.15.37:0/2026214090 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f22000b7600 0x7f22001ea610 unknown :-1 s=CLOSED pgs=647 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:30:52.967 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:52.964+0000 7f2204f4d640 1 --2- 172.21.15.37:0/2026214090 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f22000b4bb0 0x7f22001530d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:30:52.967 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:52.964+0000 7f2204f4d640 1 -- 172.21.15.37:0/2026214090 >> 172.21.15.37:0/2026214090 conn(0x7f22000d27e0 msgr2=0x7f22000cfe50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:30:52.967 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:52.964+0000 7f2204f4d640 1 -- 172.21.15.37:0/2026214090 shutdown_connections 2024-04-25T10:30:52.967 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:52.964+0000 7f2204f4d640 1 -- 172.21.15.37:0/2026214090 wait complete. 2024-04-25T10:30:52.978 DEBUG:teuthology.orchestra.run.smithi037:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-04-25T10:30:53.090 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:53.088+0000 7f558e251640 1 Processor -- start 2024-04-25T10:30:53.091 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:53.088+0000 7f558e251640 1 -- start start 2024-04-25T10:30:53.093 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:53.089+0000 7f558e251640 1 --2- >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f55880b7600 0x7f55880b7a10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:30:53.094 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:53.089+0000 7f558e251640 1 --2- >> v2:172.21.15.37:3300/0 conn(0x7f55880d4560 0x7f55880d4970 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:30:53.094 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:53.089+0000 7f558e251640 1 -- --> v1:172.21.15.37:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f55880b7f50 con 0x7f55880b4bb0 2024-04-25T10:30:53.094 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:53.089+0000 7f558e251640 1 -- --> v2:172.21.15.37:3300/0 -- mon_getmap magic: 0 -- 0x7f5588161190 con 0x7f55880d4560 2024-04-25T10:30:53.094 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:53.089+0000 7f558e251640 1 -- --> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] -- mon_getmap magic: 0 -- 0x7f5588161470 con 0x7f55880b7600 2024-04-25T10:30:53.094 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:53.089+0000 7f5587fff640 1 --2- >> v2:172.21.15.37:3300/0 conn(0x7f55880d4560 0x7f55880d4970 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:30:53.094 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:53.089+0000 7f55877fe640 1 --1- >> v1:172.21.15.37:6789/0 conn(0x7f55880b4bb0 0x7f55881646b0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.37:6789/0 says I am v1:172.21.15.37:38724/0 (socket says 172.21.15.37:38724) 2024-04-25T10:30:53.094 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:53.089+0000 7f55877fe640 1 -- 172.21.15.37:0/1423612954 learned_addr learned my addr 172.21.15.37:0/1423612954 (peer_addr_for_me v1:172.21.15.37:0/0) 2024-04-25T10:30:53.094 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:53.090+0000 7f5586ffd640 1 --2- 172.21.15.37:0/1423612954 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f55880b7600 0x7f55880b7a10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:30:53.094 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:53.090+0000 7f5587fff640 1 -- 172.21.15.37:0/1423612954 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f55880b7600 msgr2=0x7f55880b7a10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:30:53.094 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:53.090+0000 7f5587fff640 1 --2- 172.21.15.37:0/1423612954 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f55880b7600 0x7f55880b7a10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:30:53.094 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:53.090+0000 7f5587fff640 1 -- 172.21.15.37:0/1423612954 >> v1:172.21.15.37:6789/0 conn(0x7f55880b4bb0 legacy=0x7f55881646b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:30:53.094 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:53.090+0000 7f5587fff640 1 -- 172.21.15.37:0/1423612954 --> v2:172.21.15.37:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f5588163eb0 con 0x7f55880d4560 2024-04-25T10:30:53.095 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:53.090+0000 7f5587fff640 1 --2- 172.21.15.37:0/1423612954 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f55880d4560 0x7f55880d4970 secure :-1 s=READY pgs=649 cs=0 l=1 rev1=1 crypto rx=0x7f5580009d90 tx=0x7f558002a660 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d5237910c3381d08 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:30:53.095 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:53.091+0000 7f55867fc640 1 -- 172.21.15.37:0/1423612954 <== mon.0 v2:172.21.15.37:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5580007740 con 0x7f55880d4560 2024-04-25T10:30:53.095 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:53.091+0000 7f55867fc640 1 -- 172.21.15.37:0/1423612954 <== mon.0 v2:172.21.15.37:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f55800078e0 con 0x7f55880d4560 2024-04-25T10:30:53.095 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:53.091+0000 7f55867fc640 1 -- 172.21.15.37:0/1423612954 <== mon.0 v2:172.21.15.37:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5580007cb0 con 0x7f55880d4560 2024-04-25T10:30:53.095 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:53.091+0000 7f558e251640 1 -- 172.21.15.37:0/1423612954 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f55880d4560 msgr2=0x7f55880d4970 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:30:53.095 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:53.091+0000 7f558e251640 1 --2- 172.21.15.37:0/1423612954 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f55880d4560 0x7f55880d4970 secure :-1 s=READY pgs=649 cs=0 l=1 rev1=1 crypto rx=0x7f5580009d90 tx=0x7f558002a660 comp rx=0 tx=0).stop 2024-04-25T10:30:53.095 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:53.091+0000 7f558e251640 1 -- 172.21.15.37:0/1423612954 shutdown_connections 2024-04-25T10:30:53.095 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:53.091+0000 7f558e251640 1 --2- 172.21.15.37:0/1423612954 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f55880d4560 0x7f55880d4970 unknown :-1 s=CLOSED pgs=649 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:30:53.095 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:53.091+0000 7f558e251640 1 --2- 172.21.15.37:0/1423612954 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f55880b7600 0x7f55880b7a10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:30:53.095 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:53.091+0000 7f558e251640 1 -- 172.21.15.37:0/1423612954 >> 172.21.15.37:0/1423612954 conn(0x7f55880d27e0 msgr2=0x7f55880d2bf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:30:53.095 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:53.091+0000 7f558e251640 1 -- 172.21.15.37:0/1423612954 shutdown_connections 2024-04-25T10:30:53.095 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:53.091+0000 7f558e251640 1 -- 172.21.15.37:0/1423612954 wait complete. 2024-04-25T10:30:53.096 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:53.093+0000 7f558e251640 1 Processor -- start 2024-04-25T10:30:53.096 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:53.093+0000 7f558e251640 1 -- start start 2024-04-25T10:30:53.098 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:53.095+0000 7f558e251640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f55880b4bb0 0x7f558815d340 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:30:53.099 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:53.095+0000 7f558e251640 1 --2- >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f55880b7600 0x7f558815bbf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:30:53.099 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:53.095+0000 7f558e251640 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f55880d4560 0x7f558815a810 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:30:53.099 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:53.095+0000 7f558e251640 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 -- 0x7f55880d09e0 con 0x7f55880b4bb0 2024-04-25T10:30:53.099 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:53.095+0000 7f558e251640 1 -- --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_getmap magic: 0 -- 0x7f55880b4fc0 con 0x7f55880d4560 2024-04-25T10:30:53.099 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:53.095+0000 7f558e251640 1 -- --> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] -- mon_getmap magic: 0 -- 0x7f55880d0b60 con 0x7f55880b7600 2024-04-25T10:30:53.099 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:53.095+0000 7f55877fe640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f55880b4bb0 0x7f558815d340 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:30:53.099 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:53.095+0000 7f55877fe640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f55880b4bb0 0x7f558815d340 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.37:3300/0 says I am v2:172.21.15.37:59512/0 (socket says 172.21.15.37:59512) 2024-04-25T10:30:53.099 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:53.095+0000 7f55877fe640 1 -- 172.21.15.37:0/1119648692 learned_addr learned my addr 172.21.15.37:0/1119648692 (peer_addr_for_me v2:172.21.15.37:0/0) 2024-04-25T10:30:53.099 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:53.095+0000 7f5587fff640 1 --2- 172.21.15.37:0/1119648692 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f55880d4560 0x7f558815a810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:30:53.099 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:53.095+0000 7f5586ffd640 1 --2- 172.21.15.37:0/1119648692 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f55880b7600 0x7f558815bbf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:30:53.100 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:53.095+0000 7f55877fe640 1 -- 172.21.15.37:0/1119648692 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f55880b7600 msgr2=0x7f558815bbf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:30:53.100 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:53.095+0000 7f55877fe640 1 --2- 172.21.15.37:0/1119648692 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f55880b7600 0x7f558815bbf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:30:53.100 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:53.095+0000 7f55877fe640 1 -- 172.21.15.37:0/1119648692 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f55880d4560 msgr2=0x7f558815a810 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:30:53.100 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:53.095+0000 7f55877fe640 1 --2- 172.21.15.37:0/1119648692 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f55880d4560 0x7f558815a810 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:30:53.100 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:53.095+0000 7f55877fe640 1 -- 172.21.15.37:0/1119648692 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f558815c580 con 0x7f55880b4bb0 2024-04-25T10:30:53.100 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:53.095+0000 7f5587fff640 1 --2- 172.21.15.37:0/1119648692 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f55880d4560 0x7f558815a810 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T10:30:53.100 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:53.095+0000 7f5586ffd640 1 --2- 172.21.15.37:0/1119648692 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f55880b7600 0x7f558815bbf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T10:30:53.100 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:53.095+0000 7f55877fe640 1 --2- 172.21.15.37:0/1119648692 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f55880b4bb0 0x7f558815d340 secure :-1 s=READY pgs=650 cs=0 l=1 rev1=1 crypto rx=0x7f557c00c470 tx=0x7f557c00c930 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:30:53.100 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:53.095+0000 7f556ffff640 1 -- 172.21.15.37:0/1119648692 <== mon.0 v2:172.21.15.37:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f557c017070 con 0x7f55880b4bb0 2024-04-25T10:30:53.100 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:53.095+0000 7f556ffff640 1 -- 172.21.15.37:0/1119648692 <== mon.0 v2:172.21.15.37:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f557c005ce0 con 0x7f55880b4bb0 2024-04-25T10:30:53.100 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:53.095+0000 7f558e251640 1 -- 172.21.15.37:0/1119648692 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f558815c860 con 0x7f55880b4bb0 2024-04-25T10:30:53.100 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:53.095+0000 7f556ffff640 1 -- 172.21.15.37:0/1119648692 <== mon.0 v2:172.21.15.37:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f557c0072d0 con 0x7f55880b4bb0 2024-04-25T10:30:53.100 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:53.096+0000 7f558e251640 1 -- 172.21.15.37:0/1119648692 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f558815ce20 con 0x7f55880b4bb0 2024-04-25T10:30:53.100 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:53.097+0000 7f558e251640 1 -- 172.21.15.37:0/1119648692 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f558806a140 con 0x7f55880b4bb0 2024-04-25T10:30:53.101 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:53.097+0000 7f556ffff640 1 -- 172.21.15.37:0/1119648692 <== mon.0 v2:172.21.15.37:3300/0 4 ==== mgrmap(e 5) ==== 96782+0+0 (secure 0 0 0) 0x7f557c01a070 con 0x7f55880b4bb0 2024-04-25T10:30:53.101 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:53.098+0000 7f556ffff640 1 --2- 172.21.15.37:0/1119648692 >> [v2:172.21.15.203:6832/2418205237,v1:172.21.15.203:6833/2418205237] conn(0x7f55740744b0 0x7f55740769c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:30:53.101 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:53.098+0000 7f556ffff640 1 -- 172.21.15.37:0/1119648692 <== mon.0 v2:172.21.15.37:3300/0 5 ==== osd_map(46..46 src has 1..46) ==== 6364+0+0 (secure 0 0 0) 0x7f557c094720 con 0x7f55880b4bb0 2024-04-25T10:30:53.101 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:53.098+0000 7f5586ffd640 1 --2- 172.21.15.37:0/1119648692 >> [v2:172.21.15.203:6832/2418205237,v1:172.21.15.203:6833/2418205237] conn(0x7f55740744b0 0x7f55740769c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:30:53.101 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:53.098+0000 7f5586ffd640 1 --2- 172.21.15.37:0/1119648692 >> [v2:172.21.15.203:6832/2418205237,v1:172.21.15.203:6833/2418205237] conn(0x7f55740744b0 0x7f55740769c0 secure :-1 s=READY pgs=295 cs=0 l=1 rev1=1 crypto rx=0x7f5578006fd0 tx=0x7f5578008040 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:30:53.104 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:53.103+0000 7f556ffff640 1 -- 172.21.15.37:0/1119648692 <== mon.0 v2:172.21.15.37:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187246 (secure 0 0 0) 0x7f55880b7f50 con 0x7f55880b4bb0 2024-04-25T10:30:53.289 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:53.286+0000 7f558e251640 1 -- 172.21.15.37:0/1119648692 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7f55880d3ff0 con 0x7f55880b4bb0 2024-04-25T10:30:53.290 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:53.288+0000 7f556ffff640 1 -- 172.21.15.37:0/1119648692 <== mon.0 v2:172.21.15.37:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v46) ==== 74+0+18876 (secure 0 0 0) 0x7f557c05e530 con 0x7f55880b4bb0 2024-04-25T10:30:53.290 INFO:teuthology.orchestra.run.smithi037.stdout: 2024-04-25T10:30:53.290 INFO:teuthology.orchestra.run.smithi037.stdout:{"epoch":46,"fsid":"f4acda28-be34-492a-b286-994afd36f61d","created":"2024-04-25T10:28:43.985718+0000","modified":"2024-04-25T10:30:49.793339+0000","last_up_change":"2024-04-25T10:28:54.900654+0000","last_in_change":"2024-04-25T10:28:46.216973+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":6,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"luminous","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-04-25T10:28:56.202153+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"19","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":4,"pool_name":"cephfs_metadata","create_time":"2024-04-25T10:30:10.008132+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"35","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.75,"score_stable":1.75,"optimal_score":1,"raw_score_acting":1.75,"raw_score_stable":1.75,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":5,"pool_name":"cephfs_data","create_time":"2024-04-25T10:30:11.084827+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"43","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"recovery_fs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.3799999952316284,"score_stable":1.3799999952316284,"optimal_score":1,"raw_score_acting":1.3799999952316284,"raw_score_stable":1.3799999952316284,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":6,"pool_name":"recovery_fs_metadata","create_time":"2024-04-25T10:30:26.903737+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"44","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"recovery_fs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"a2f79028-b332-4166-9ebb-a7024a37ea22","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":42,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.37:6816","nonce":2746074720},{"type":"v1","addr":"172.21.15.37:6817","nonce":2746074720}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.37:6818","nonce":2746074720},{"type":"v1","addr":"172.21.15.37:6819","nonce":2746074720}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.37:6822","nonce":2746074720},{"type":"v1","addr":"172.21.15.37:6823","nonce":2746074720}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.37:6820","nonce":2746074720},{"type":"v1","addr":"172.21.15.37:6821","nonce":2746074720}]},"public_addr":"172.21.15.37:6817/2746074720","cluster_addr":"172.21.15.37:6819/2746074720","heartbeat_back_addr":"172.21.15.37:6823/2746074720","heartbeat_front_addr":"172.21.15.37:6821/2746074720","state":["exists","up"]},{"osd":1,"uuid":"973159de-c2ea-4083-aecc-69373d1c1132","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":45,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.37:6800","nonce":1690341662},{"type":"v1","addr":"172.21.15.37:6801","nonce":1690341662}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.37:6802","nonce":1690341662},{"type":"v1","addr":"172.21.15.37:6803","nonce":1690341662}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.37:6806","nonce":1690341662},{"type":"v1","addr":"172.21.15.37:6807","nonce":1690341662}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.37:6804","nonce":1690341662},{"type":"v1","addr":"172.21.15.37:6805","nonce":1690341662}]},"public_addr":"172.21.15.37:6801/1690341662","cluster_addr":"172.21.15.37:6803/1690341662","heartbeat_back_addr":"172.21.15.37:6807/1690341662","heartbeat_front_addr":"172.21.15.37:6805/1690341662","state":["exists","up"]},{"osd":2,"uuid":"4ae7a0e9-cb3f-484c-a75f-f3766b721c47","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":42,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.37:6808","nonce":2754669649},{"type":"v1","addr":"172.21.15.37:6809","nonce":2754669649}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.37:6810","nonce":2754669649},{"type":"v1","addr":"172.21.15.37:6811","nonce":2754669649}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.37:6814","nonce":2754669649},{"type":"v1","addr":"172.21.15.37:6815","nonce":2754669649}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.37:6812","nonce":2754669649},{"type":"v1","addr":"172.21.15.37:6813","nonce":2754669649}]},"public_addr":"172.21.15.37:6809/2754669649","cluster_addr":"172.21.15.37:6811/2754669649","heartbeat_back_addr":"172.21.15.37:6815/2754669649","heartbeat_front_addr":"172.21.15.37:6813/2754669649","state":["exists","up"]},{"osd":3,"uuid":"1ba4a746-1608-47a7-ace2-960a5cfae934","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":45,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.37:6824","nonce":1815290618},{"type":"v1","addr":"172.21.15.37:6825","nonce":1815290618}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.37:6826","nonce":1815290618},{"type":"v1","addr":"172.21.15.37:6827","nonce":1815290618}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.37:6830","nonce":1815290618},{"type":"v1","addr":"172.21.15.37:6831","nonce":1815290618}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.37:6828","nonce":1815290618},{"type":"v1","addr":"172.21.15.37:6829","nonce":1815290618}]},"public_addr":"172.21.15.37:6825/1815290618","cluster_addr":"172.21.15.37:6827/1815290618","heartbeat_back_addr":"172.21.15.37:6831/1815290618","heartbeat_front_addr":"172.21.15.37:6829/1815290618","state":["exists","up"]},{"osd":4,"uuid":"d4f59f5d-0413-4b79-87ee-bb212c1fcf62","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":42,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6800","nonce":4051532063},{"type":"v1","addr":"172.21.15.203:6801","nonce":4051532063}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6802","nonce":4051532063},{"type":"v1","addr":"172.21.15.203:6803","nonce":4051532063}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6806","nonce":4051532063},{"type":"v1","addr":"172.21.15.203:6807","nonce":4051532063}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6804","nonce":4051532063},{"type":"v1","addr":"172.21.15.203:6805","nonce":4051532063}]},"public_addr":"172.21.15.203:6801/4051532063","cluster_addr":"172.21.15.203:6803/4051532063","heartbeat_back_addr":"172.21.15.203:6807/4051532063","heartbeat_front_addr":"172.21.15.203:6805/4051532063","state":["exists","up"]},{"osd":5,"uuid":"69710a0e-41d8-4e7f-9a10-17ce540b67be","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":45,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6824","nonce":1203822890},{"type":"v1","addr":"172.21.15.203:6825","nonce":1203822890}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6826","nonce":1203822890},{"type":"v1","addr":"172.21.15.203:6827","nonce":1203822890}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6830","nonce":1203822890},{"type":"v1","addr":"172.21.15.203:6831","nonce":1203822890}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6828","nonce":1203822890},{"type":"v1","addr":"172.21.15.203:6829","nonce":1203822890}]},"public_addr":"172.21.15.203:6825/1203822890","cluster_addr":"172.21.15.203:6827/1203822890","heartbeat_back_addr":"172.21.15.203:6831/1203822890","heartbeat_front_addr":"172.21.15.203:6829/1203822890","state":["exists","up"]},{"osd":6,"uuid":"f29dd047-3b48-473d-a235-6642a6293f20","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":42,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6810","nonce":2309615014},{"type":"v1","addr":"172.21.15.203:6812","nonce":2309615014}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6814","nonce":2309615014},{"type":"v1","addr":"172.21.15.203:6816","nonce":2309615014}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6822","nonce":2309615014},{"type":"v1","addr":"172.21.15.203:6823","nonce":2309615014}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6818","nonce":2309615014},{"type":"v1","addr":"172.21.15.203:6820","nonce":2309615014}]},"public_addr":"172.21.15.203:6812/2309615014","cluster_addr":"172.21.15.203:6816/2309615014","heartbeat_back_addr":"172.21.15.203:6823/2309615014","heartbeat_front_addr":"172.21.15.203:6820/2309615014","state":["exists","up"]},{"osd":7,"uuid":"6e691644-378b-4fdc-8e86-fb5f56e31f29","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":42,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6808","nonce":3688099014},{"type":"v1","addr":"172.21.15.203:6809","nonce":3688099014}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6811","nonce":3688099014},{"type":"v1","addr":"172.21.15.203:6813","nonce":3688099014}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6819","nonce":3688099014},{"type":"v1","addr":"172.21.15.203:6821","nonce":3688099014}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6815","nonce":3688099014},{"type":"v1","addr":"172.21.15.203:6817","nonce":3688099014}]},"public_addr":"172.21.15.203:6809/3688099014","cluster_addr":"172.21.15.203:6813/3688099014","heartbeat_back_addr":"172.21.15.203:6821/3688099014","heartbeat_front_addr":"172.21.15.203:6817/3688099014","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T10:28:51.419141+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T10:28:51.366448+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T10:28:51.348276+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T10:28:51.380519+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T10:28:51.105812+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T10:28:51.384496+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T10:28:51.388755+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T10:28:51.366958+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[{"pgid":"5.15","mappings":[{"from":4,"to":5}]},{"pgid":"5.1a","mappings":[{"from":4,"to":5}]},{"pgid":"5.26","mappings":[{"from":4,"to":5}]},{"pgid":"5.27","mappings":[{"from":4,"to":5}]}],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.37:6835/311552410":"2024-04-26T10:30:24.853732+0000","172.21.15.37:6832/311552410":"2024-04-26T10:30:24.853732+0000"},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-04-25T10:30:53.292 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:53.290+0000 7f558e251640 1 -- 172.21.15.37:0/1119648692 >> [v2:172.21.15.203:6832/2418205237,v1:172.21.15.203:6833/2418205237] conn(0x7f55740744b0 msgr2=0x7f55740769c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:30:53.292 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:53.290+0000 7f558e251640 1 --2- 172.21.15.37:0/1119648692 >> [v2:172.21.15.203:6832/2418205237,v1:172.21.15.203:6833/2418205237] conn(0x7f55740744b0 0x7f55740769c0 secure :-1 s=READY pgs=295 cs=0 l=1 rev1=1 crypto rx=0x7f5578006fd0 tx=0x7f5578008040 comp rx=0 tx=0).stop 2024-04-25T10:30:53.292 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:53.290+0000 7f558e251640 1 -- 172.21.15.37:0/1119648692 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f55880b4bb0 msgr2=0x7f558815d340 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:30:53.292 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:53.290+0000 7f558e251640 1 --2- 172.21.15.37:0/1119648692 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f55880b4bb0 0x7f558815d340 secure :-1 s=READY pgs=650 cs=0 l=1 rev1=1 crypto rx=0x7f557c00c470 tx=0x7f557c00c930 comp rx=0 tx=0).stop 2024-04-25T10:30:53.292 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:53.290+0000 7f558e251640 1 -- 172.21.15.37:0/1119648692 shutdown_connections 2024-04-25T10:30:53.293 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:53.290+0000 7f558e251640 1 --2- 172.21.15.37:0/1119648692 >> [v2:172.21.15.203:6832/2418205237,v1:172.21.15.203:6833/2418205237] conn(0x7f55740744b0 0x7f55740769c0 unknown :-1 s=CLOSED pgs=295 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:30:53.293 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:53.290+0000 7f558e251640 1 --2- 172.21.15.37:0/1119648692 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f55880d4560 0x7f558815a810 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:30:53.293 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:53.290+0000 7f558e251640 1 --2- 172.21.15.37:0/1119648692 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f55880b7600 0x7f558815bbf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:30:53.293 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:53.290+0000 7f558e251640 1 --2- 172.21.15.37:0/1119648692 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f55880b4bb0 0x7f558815d340 unknown :-1 s=CLOSED pgs=650 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:30:53.293 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:53.290+0000 7f558e251640 1 -- 172.21.15.37:0/1119648692 >> 172.21.15.37:0/1119648692 conn(0x7f55880d27e0 msgr2=0x7f55880d4970 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:30:53.293 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:53.290+0000 7f558e251640 1 -- 172.21.15.37:0/1119648692 shutdown_connections 2024-04-25T10:30:53.293 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:53.290+0000 7f558e251640 1 -- 172.21.15.37:0/1119648692 wait complete. 2024-04-25T10:30:53.304 DEBUG:teuthology.orchestra.run.smithi037:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs fail cephfs 2024-04-25T10:30:53.416 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:53.414+0000 7f1d6c5c0640 1 Processor -- start 2024-04-25T10:30:53.417 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:53.414+0000 7f1d6c5c0640 1 -- start start 2024-04-25T10:30:53.419 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:53.416+0000 7f1d6c5c0640 1 --2- >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f1d640b3c80 0x7f1d640b4090 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:30:53.419 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:53.416+0000 7f1d6c5c0640 1 --2- >> v2:172.21.15.37:3300/0 conn(0x7f1d640ce750 0x7f1d640ceb60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:30:53.419 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:53.416+0000 7f1d6c5c0640 1 -- --> v1:172.21.15.37:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f1d640b8d10 con 0x7f1d640d1ff0 2024-04-25T10:30:53.419 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:53.416+0000 7f1d6c5c0640 1 -- --> v2:172.21.15.37:3300/0 -- mon_getmap magic: 0 -- 0x7f1d6408ceb0 con 0x7f1d640ce750 2024-04-25T10:30:53.419 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:53.416+0000 7f1d6c5c0640 1 -- --> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] -- mon_getmap magic: 0 -- 0x7f1d640b9300 con 0x7f1d640b3c80 2024-04-25T10:30:53.419 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:53.416+0000 7f1d6ab5d640 1 --2- >> v2:172.21.15.37:3300/0 conn(0x7f1d640ce750 0x7f1d640ceb60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:30:53.419 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:53.416+0000 7f1d69b5b640 1 --1- >> v1:172.21.15.37:6789/0 conn(0x7f1d640d1ff0 0x7f1d640d2400 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.37:6789/0 says I am v1:172.21.15.37:38740/0 (socket says 172.21.15.37:38740) 2024-04-25T10:30:53.419 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:53.416+0000 7f1d69b5b640 1 -- 172.21.15.37:0/530060490 learned_addr learned my addr 172.21.15.37:0/530060490 (peer_addr_for_me v1:172.21.15.37:0/0) 2024-04-25T10:30:53.419 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:53.416+0000 7f1d6a35c640 1 --2- 172.21.15.37:0/530060490 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f1d640b3c80 0x7f1d640b4090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:30:53.419 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:53.416+0000 7f1d6935a640 1 -- 172.21.15.37:0/530060490 <== mon.0 v1:172.21.15.37:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 3763926125 0 0) 0x7f1d640b8d10 con 0x7f1d640d1ff0 2024-04-25T10:30:53.419 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:53.416+0000 7f1d6ab5d640 1 -- 172.21.15.37:0/530060490 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f1d640b3c80 msgr2=0x7f1d640b4090 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:30:53.419 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:53.416+0000 7f1d6ab5d640 1 --2- 172.21.15.37:0/530060490 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f1d640b3c80 0x7f1d640b4090 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:30:53.419 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:53.416+0000 7f1d6ab5d640 1 -- 172.21.15.37:0/530060490 >> v1:172.21.15.37:6789/0 conn(0x7f1d640d1ff0 legacy=0x7f1d640d2400 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:30:53.419 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:53.416+0000 7f1d6ab5d640 1 -- 172.21.15.37:0/530060490 --> v2:172.21.15.37:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f1d640b8a30 con 0x7f1d640ce750 2024-04-25T10:30:53.421 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:53.417+0000 7f1d6ab5d640 1 --2- 172.21.15.37:0/530060490 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f1d640ce750 0x7f1d640ceb60 secure :-1 s=READY pgs=652 cs=0 l=1 rev1=1 crypto rx=0x7f1d54009d90 tx=0x7f1d5402a5d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=967b15e97bcd37e5 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:30:53.421 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:53.417+0000 7f1d6935a640 1 -- 172.21.15.37:0/530060490 <== mon.0 v2:172.21.15.37:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1d54031600 con 0x7f1d640ce750 2024-04-25T10:30:53.421 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:53.417+0000 7f1d6935a640 1 -- 172.21.15.37:0/530060490 <== mon.0 v2:172.21.15.37:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f1d540317a0 con 0x7f1d640ce750 2024-04-25T10:30:53.421 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:53.417+0000 7f1d6935a640 1 -- 172.21.15.37:0/530060490 <== mon.0 v2:172.21.15.37:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1d54031b70 con 0x7f1d640ce750 2024-04-25T10:30:53.421 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:53.418+0000 7f1d6c5c0640 1 -- 172.21.15.37:0/530060490 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f1d640ce750 msgr2=0x7f1d640ceb60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:30:53.421 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:53.418+0000 7f1d6c5c0640 1 --2- 172.21.15.37:0/530060490 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f1d640ce750 0x7f1d640ceb60 secure :-1 s=READY pgs=652 cs=0 l=1 rev1=1 crypto rx=0x7f1d54009d90 tx=0x7f1d5402a5d0 comp rx=0 tx=0).stop 2024-04-25T10:30:53.421 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:53.418+0000 7f1d6c5c0640 1 -- 172.21.15.37:0/530060490 shutdown_connections 2024-04-25T10:30:53.421 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:53.418+0000 7f1d6c5c0640 1 --2- 172.21.15.37:0/530060490 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f1d640ce750 0x7f1d640ceb60 unknown :-1 s=CLOSED pgs=652 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:30:53.421 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:53.418+0000 7f1d6c5c0640 1 --2- 172.21.15.37:0/530060490 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f1d640b3c80 0x7f1d640b4090 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:30:53.421 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:53.418+0000 7f1d6c5c0640 1 -- 172.21.15.37:0/530060490 >> 172.21.15.37:0/530060490 conn(0x7f1d640d0390 msgr2=0x7f1d640d07a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:30:53.421 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:53.418+0000 7f1d6c5c0640 1 -- 172.21.15.37:0/530060490 shutdown_connections 2024-04-25T10:30:53.421 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:53.418+0000 7f1d6c5c0640 1 -- 172.21.15.37:0/530060490 wait complete. 2024-04-25T10:30:53.422 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:53.420+0000 7f1d6c5c0640 1 Processor -- start 2024-04-25T10:30:53.422 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:53.420+0000 7f1d6c5c0640 1 -- start start 2024-04-25T10:30:53.424 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:53.421+0000 7f1d6c5c0640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f1d640b3c80 0x7f1d641d69a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:30:53.424 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:53.421+0000 7f1d6c5c0640 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f1d640ce750 0x7f1d6406e5b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:30:53.424 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:53.421+0000 7f1d6c5c0640 1 --2- >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f1d640d1ff0 0x7f1d6406eaf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:30:53.425 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:53.421+0000 7f1d6c5c0640 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 -- 0x7f1d640d2b10 con 0x7f1d640b3c80 2024-04-25T10:30:53.425 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:53.421+0000 7f1d6c5c0640 1 -- --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_getmap magic: 0 -- 0x7f1d6409ce90 con 0x7f1d640ce750 2024-04-25T10:30:53.425 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:53.421+0000 7f1d6c5c0640 1 -- --> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] -- mon_getmap magic: 0 -- 0x7f1d640b45d0 con 0x7f1d640d1ff0 2024-04-25T10:30:53.425 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:53.421+0000 7f1d6a35c640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f1d640b3c80 0x7f1d641d69a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:30:53.425 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:53.421+0000 7f1d6a35c640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f1d640b3c80 0x7f1d641d69a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.37:3300/0 says I am v2:172.21.15.37:59542/0 (socket says 172.21.15.37:59542) 2024-04-25T10:30:53.425 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:53.421+0000 7f1d6a35c640 1 -- 172.21.15.37:0/1269094928 learned_addr learned my addr 172.21.15.37:0/1269094928 (peer_addr_for_me v2:172.21.15.37:0/0) 2024-04-25T10:30:53.425 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:53.421+0000 7f1d6ab5d640 1 --2- 172.21.15.37:0/1269094928 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f1d640d1ff0 0x7f1d6406eaf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:30:53.425 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:53.421+0000 7f1d69b5b640 1 --2- 172.21.15.37:0/1269094928 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f1d640ce750 0x7f1d6406e5b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:30:53.425 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:53.422+0000 7f1d6a35c640 1 -- 172.21.15.37:0/1269094928 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f1d640d1ff0 msgr2=0x7f1d6406eaf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:30:53.425 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:53.422+0000 7f1d6a35c640 1 --2- 172.21.15.37:0/1269094928 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f1d640d1ff0 0x7f1d6406eaf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:30:53.425 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:53.422+0000 7f1d6a35c640 1 -- 172.21.15.37:0/1269094928 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f1d640ce750 msgr2=0x7f1d6406e5b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:30:53.425 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:53.422+0000 7f1d6a35c640 1 --2- 172.21.15.37:0/1269094928 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f1d640ce750 0x7f1d6406e5b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:30:53.425 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:53.422+0000 7f1d6a35c640 1 -- 172.21.15.37:0/1269094928 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f1d6406d7f0 con 0x7f1d640b3c80 2024-04-25T10:30:53.425 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:53.422+0000 7f1d6a35c640 1 --2- 172.21.15.37:0/1269094928 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f1d640b3c80 0x7f1d641d69a0 secure :-1 s=READY pgs=653 cs=0 l=1 rev1=1 crypto rx=0x7f1d5c00e8a0 tx=0x7f1d5c00ed60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:30:53.426 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:53.422+0000 7f1d62ffd640 1 -- 172.21.15.37:0/1269094928 <== mon.0 v2:172.21.15.37:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1d5c017070 con 0x7f1d640b3c80 2024-04-25T10:30:53.426 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:53.422+0000 7f1d62ffd640 1 -- 172.21.15.37:0/1269094928 <== mon.0 v2:172.21.15.37:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f1d5c00c940 con 0x7f1d640b3c80 2024-04-25T10:30:53.426 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:53.422+0000 7f1d62ffd640 1 -- 172.21.15.37:0/1269094928 <== mon.0 v2:172.21.15.37:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1d5c00cd30 con 0x7f1d640b3c80 2024-04-25T10:30:53.426 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:53.422+0000 7f1d6c5c0640 1 -- 172.21.15.37:0/1269094928 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f1d6406dad0 con 0x7f1d640b3c80 2024-04-25T10:30:53.426 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:53.422+0000 7f1d6c5c0640 1 -- 172.21.15.37:0/1269094928 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f1d6406e090 con 0x7f1d640b3c80 2024-04-25T10:30:53.427 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:53.423+0000 7f1d6c5c0640 1 -- 172.21.15.37:0/1269094928 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f1d640d2e10 con 0x7f1d640b3c80 2024-04-25T10:30:53.427 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:53.424+0000 7f1d62ffd640 1 -- 172.21.15.37:0/1269094928 <== mon.0 v2:172.21.15.37:3300/0 4 ==== mgrmap(e 5) ==== 96782+0+0 (secure 0 0 0) 0x7f1d5c01a070 con 0x7f1d640b3c80 2024-04-25T10:30:53.427 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:53.424+0000 7f1d62ffd640 1 --2- 172.21.15.37:0/1269094928 >> [v2:172.21.15.203:6832/2418205237,v1:172.21.15.203:6833/2418205237] conn(0x7f1d500744b0 0x7f1d500769c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:30:53.427 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:53.424+0000 7f1d62ffd640 1 -- 172.21.15.37:0/1269094928 <== mon.0 v2:172.21.15.37:3300/0 5 ==== osd_map(46..46 src has 1..46) ==== 6364+0+0 (secure 0 0 0) 0x7f1d5c095750 con 0x7f1d640b3c80 2024-04-25T10:30:53.427 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:53.424+0000 7f1d69b5b640 1 --2- 172.21.15.37:0/1269094928 >> [v2:172.21.15.203:6832/2418205237,v1:172.21.15.203:6833/2418205237] conn(0x7f1d500744b0 0x7f1d500769c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:30:53.427 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:53.425+0000 7f1d69b5b640 1 --2- 172.21.15.37:0/1269094928 >> [v2:172.21.15.203:6832/2418205237,v1:172.21.15.203:6833/2418205237] conn(0x7f1d500744b0 0x7f1d500769c0 secure :-1 s=READY pgs=296 cs=0 l=1 rev1=1 crypto rx=0x7f1d640ccbb0 tx=0x7f1d58007c00 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:30:53.431 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:53.429+0000 7f1d62ffd640 1 -- 172.21.15.37:0/1269094928 <== mon.0 v2:172.21.15.37:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187246 (secure 0 0 0) 0x7f1d640d2e10 con 0x7f1d640b3c80 2024-04-25T10:30:53.652 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:53.650+0000 7f1d6c5c0640 1 -- 172.21.15.37:0/1269094928 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_command({"prefix": "fs fail", "fs_name": "cephfs"} v 0) -- 0x7f1d6406a140 con 0x7f1d640b3c80 2024-04-25T10:30:53.814 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:53.812+0000 7f1d62ffd640 1 -- 172.21.15.37:0/1269094928 <== mon.0 v2:172.21.15.37:3300/0 7 ==== mon_command_ack([{"prefix": "fs fail", "fs_name": "cephfs"}]=0 cephfs marked not joinable; MDS cannot join the cluster. All MDS ranks marked failed. v28) ==== 161+0+0 (secure 0 0 0) 0x7f1d5c05fc50 con 0x7f1d640b3c80 2024-04-25T10:30:53.814 INFO:teuthology.orchestra.run.smithi037.stderr:cephfs marked not joinable; MDS cannot join the cluster. All MDS ranks marked failed. 2024-04-25T10:30:53.816 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:53.814+0000 7f1d6c5c0640 1 -- 172.21.15.37:0/1269094928 >> [v2:172.21.15.203:6832/2418205237,v1:172.21.15.203:6833/2418205237] conn(0x7f1d500744b0 msgr2=0x7f1d500769c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:30:53.816 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:53.814+0000 7f1d6c5c0640 1 --2- 172.21.15.37:0/1269094928 >> [v2:172.21.15.203:6832/2418205237,v1:172.21.15.203:6833/2418205237] conn(0x7f1d500744b0 0x7f1d500769c0 secure :-1 s=READY pgs=296 cs=0 l=1 rev1=1 crypto rx=0x7f1d640ccbb0 tx=0x7f1d58007c00 comp rx=0 tx=0).stop 2024-04-25T10:30:53.816 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:53.814+0000 7f1d6c5c0640 1 -- 172.21.15.37:0/1269094928 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f1d640b3c80 msgr2=0x7f1d641d69a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:30:53.816 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:53.814+0000 7f1d6c5c0640 1 --2- 172.21.15.37:0/1269094928 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f1d640b3c80 0x7f1d641d69a0 secure :-1 s=READY pgs=653 cs=0 l=1 rev1=1 crypto rx=0x7f1d5c00e8a0 tx=0x7f1d5c00ed60 comp rx=0 tx=0).stop 2024-04-25T10:30:53.817 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:53.814+0000 7f1d6c5c0640 1 -- 172.21.15.37:0/1269094928 shutdown_connections 2024-04-25T10:30:53.817 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:53.814+0000 7f1d6c5c0640 1 --2- 172.21.15.37:0/1269094928 >> [v2:172.21.15.203:6832/2418205237,v1:172.21.15.203:6833/2418205237] conn(0x7f1d500744b0 0x7f1d500769c0 unknown :-1 s=CLOSED pgs=296 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:30:53.817 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:53.814+0000 7f1d6c5c0640 1 --2- 172.21.15.37:0/1269094928 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f1d640d1ff0 0x7f1d6406eaf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:30:53.817 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:53.814+0000 7f1d6c5c0640 1 --2- 172.21.15.37:0/1269094928 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f1d640ce750 0x7f1d6406e5b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:30:53.817 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:53.814+0000 7f1d6c5c0640 1 --2- 172.21.15.37:0/1269094928 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f1d640b3c80 0x7f1d641d69a0 unknown :-1 s=CLOSED pgs=653 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:30:53.817 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:53.814+0000 7f1d6c5c0640 1 -- 172.21.15.37:0/1269094928 >> 172.21.15.37:0/1269094928 conn(0x7f1d640d0390 msgr2=0x7f1d640d07a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:30:53.817 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:53.814+0000 7f1d6c5c0640 1 -- 172.21.15.37:0/1269094928 shutdown_connections 2024-04-25T10:30:53.817 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:53.814+0000 7f1d6c5c0640 1 -- 172.21.15.37:0/1269094928 wait complete. 2024-04-25T10:30:53.828 DEBUG:teuthology.orchestra.run.smithi037:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs rm cephfs --yes-i-really-mean-it 2024-04-25T10:30:53.941 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:53.939+0000 7fbdab93f640 1 Processor -- start 2024-04-25T10:30:53.942 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:53.939+0000 7fbdab93f640 1 -- start start 2024-04-25T10:30:53.944 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:53.941+0000 7fbdab93f640 1 --2- >> v2:172.21.15.203:3300/0 conn(0x7fbda40b4e80 0x7fbda40d2b20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:30:53.944 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:53.941+0000 7fbdab93f640 1 --2- >> v2:172.21.15.37:3300/0 conn(0x7fbda40d3060 0x7fbda40b9060 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:30:53.944 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:53.941+0000 7fbdab93f640 1 -- --> v1:172.21.15.37:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fbda40d3470 con 0x7fbda40d5160 2024-04-25T10:30:53.944 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:53.941+0000 7fbdab93f640 1 -- --> v2:172.21.15.37:3300/0 -- mon_getmap magic: 0 -- 0x7fbda40d1650 con 0x7fbda40d3060 2024-04-25T10:30:53.944 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:53.941+0000 7fbdab93f640 1 -- --> v2:172.21.15.203:3300/0 -- mon_getmap magic: 0 -- 0x7fbda40b5290 con 0x7fbda40b4e80 2024-04-25T10:30:53.944 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:53.941+0000 7fbda96db640 1 --1- >> v1:172.21.15.37:6789/0 conn(0x7fbda40d5160 0x7fbda40d0990 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.37:6789/0 says I am v1:172.21.15.37:38754/0 (socket says 172.21.15.37:38754) 2024-04-25T10:30:53.945 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:53.941+0000 7fbda96db640 1 -- 172.21.15.37:0/2057012840 learned_addr learned my addr 172.21.15.37:0/2057012840 (peer_addr_for_me v1:172.21.15.37:0/0) 2024-04-25T10:30:53.945 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:53.941+0000 7fbda9edc640 1 --2- 172.21.15.37:0/2057012840 >> v2:172.21.15.37:3300/0 conn(0x7fbda40d3060 0x7fbda40b9060 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:30:53.945 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:53.941+0000 7fbda3fff640 1 -- 172.21.15.37:0/2057012840 <== mon.0 v1:172.21.15.37:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 3785210991 0 0) 0x7fbd9c001490 con 0x7fbda40d5160 2024-04-25T10:30:53.945 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:53.941+0000 7fbda8eda640 1 --2- 172.21.15.37:0/2057012840 >> v2:172.21.15.203:3300/0 conn(0x7fbda40b4e80 0x7fbda40d2b20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:30:53.945 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:53.941+0000 7fbda3fff640 1 -- 172.21.15.37:0/2057012840 --> v1:172.21.15.37:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7fbd8c003520 con 0x7fbda40d5160 2024-04-25T10:30:53.945 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:53.941+0000 7fbda9edc640 1 -- 172.21.15.37:0/2057012840 >> v2:172.21.15.203:3300/0 conn(0x7fbda40b4e80 msgr2=0x7fbda40d2b20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:30:53.945 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:53.941+0000 7fbda9edc640 1 --2- 172.21.15.37:0/2057012840 >> v2:172.21.15.203:3300/0 conn(0x7fbda40b4e80 0x7fbda40d2b20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:30:53.945 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:53.941+0000 7fbda9edc640 1 -- 172.21.15.37:0/2057012840 >> v1:172.21.15.37:6789/0 conn(0x7fbda40d5160 legacy=0x7fbda40d0990 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:30:53.945 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:53.941+0000 7fbda9edc640 1 -- 172.21.15.37:0/2057012840 --> v2:172.21.15.37:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fbda40d10a0 con 0x7fbda40d3060 2024-04-25T10:30:53.945 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:53.942+0000 7fbda9edc640 1 --2- 172.21.15.37:0/2057012840 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fbda40d3060 0x7fbda40b9060 secure :-1 s=READY pgs=655 cs=0 l=1 rev1=1 crypto rx=0x7fbda40d6b80 tx=0x7fbd9402a110 comp rx=0 tx=0).ready entity=mon.0 client_cookie=60ed7ee98c99d7aa server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:30:53.945 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:53.942+0000 7fbda3fff640 1 -- 172.21.15.37:0/2057012840 <== mon.0 v2:172.21.15.37:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbd94037070 con 0x7fbda40d3060 2024-04-25T10:30:53.945 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:53.942+0000 7fbda3fff640 1 -- 172.21.15.37:0/2057012840 <== mon.0 v2:172.21.15.37:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fbd94032440 con 0x7fbda40d3060 2024-04-25T10:30:53.945 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:53.942+0000 7fbda3fff640 1 -- 172.21.15.37:0/2057012840 <== mon.0 v2:172.21.15.37:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbd9402ae60 con 0x7fbda40d3060 2024-04-25T10:30:53.946 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:53.943+0000 7fbdab93f640 1 -- 172.21.15.37:0/2057012840 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fbda40d3060 msgr2=0x7fbda40b9060 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:30:53.947 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:53.943+0000 7fbdab93f640 1 --2- 172.21.15.37:0/2057012840 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fbda40d3060 0x7fbda40b9060 secure :-1 s=READY pgs=655 cs=0 l=1 rev1=1 crypto rx=0x7fbda40d6b80 tx=0x7fbd9402a110 comp rx=0 tx=0).stop 2024-04-25T10:30:53.947 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:53.943+0000 7fbdab93f640 1 -- 172.21.15.37:0/2057012840 shutdown_connections 2024-04-25T10:30:53.947 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:53.943+0000 7fbdab93f640 1 --2- 172.21.15.37:0/2057012840 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fbda40d3060 0x7fbda40b9060 unknown :-1 s=CLOSED pgs=655 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:30:53.947 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:53.943+0000 7fbdab93f640 1 --2- 172.21.15.37:0/2057012840 >> v2:172.21.15.203:3300/0 conn(0x7fbda40b4e80 0x7fbda40d2b20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:30:53.947 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:53.943+0000 7fbdab93f640 1 -- 172.21.15.37:0/2057012840 >> 172.21.15.37:0/2057012840 conn(0x7fbda40b7a80 msgr2=0x7fbda40b7e90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:30:53.947 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:53.943+0000 7fbdab93f640 1 -- 172.21.15.37:0/2057012840 shutdown_connections 2024-04-25T10:30:53.947 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:53.943+0000 7fbdab93f640 1 -- 172.21.15.37:0/2057012840 wait complete. 2024-04-25T10:30:53.947 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:53.945+0000 7fbdab93f640 1 Processor -- start 2024-04-25T10:30:53.947 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:53.945+0000 7fbdab93f640 1 -- start start 2024-04-25T10:30:53.949 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:53.946+0000 7fbdab93f640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fbda40b4e80 0x7fbda4155230 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:30:53.949 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:53.946+0000 7fbdab93f640 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fbda40d3060 0x7fbda41613a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:30:53.949 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:53.946+0000 7fbdab93f640 1 --2- >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7fbda40d5160 0x7fbda40cfca0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:30:53.949 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:53.946+0000 7fbdab93f640 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 -- 0x7fbda40d4960 con 0x7fbda40b4e80 2024-04-25T10:30:53.949 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:53.946+0000 7fbdab93f640 1 -- --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_getmap magic: 0 -- 0x7fbda40d4680 con 0x7fbda40d3060 2024-04-25T10:30:53.950 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:53.946+0000 7fbdab93f640 1 -- --> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] -- mon_getmap magic: 0 -- 0x7fbda40d4c40 con 0x7fbda40d5160 2024-04-25T10:30:53.950 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:53.946+0000 7fbda96db640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fbda40b4e80 0x7fbda4155230 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:30:53.950 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:53.946+0000 7fbda96db640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fbda40b4e80 0x7fbda4155230 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.37:3300/0 says I am v2:172.21.15.37:59568/0 (socket says 172.21.15.37:59568) 2024-04-25T10:30:53.950 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:53.946+0000 7fbda96db640 1 -- 172.21.15.37:0/320073479 learned_addr learned my addr 172.21.15.37:0/320073479 (peer_addr_for_me v2:172.21.15.37:0/0) 2024-04-25T10:30:53.950 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:53.946+0000 7fbda9edc640 1 --2- 172.21.15.37:0/320073479 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7fbda40d5160 0x7fbda40cfca0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:30:53.950 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:53.946+0000 7fbda8eda640 1 --2- 172.21.15.37:0/320073479 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fbda40d3060 0x7fbda41613a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:30:53.950 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:53.946+0000 7fbda96db640 1 -- 172.21.15.37:0/320073479 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7fbda40d5160 msgr2=0x7fbda40cfca0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:30:53.950 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:53.946+0000 7fbda96db640 1 --2- 172.21.15.37:0/320073479 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7fbda40d5160 0x7fbda40cfca0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:30:53.950 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:53.946+0000 7fbda96db640 1 -- 172.21.15.37:0/320073479 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fbda40d3060 msgr2=0x7fbda41613a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:30:53.950 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:53.946+0000 7fbda96db640 1 --2- 172.21.15.37:0/320073479 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fbda40d3060 0x7fbda41613a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:30:53.950 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:53.946+0000 7fbda96db640 1 -- 172.21.15.37:0/320073479 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fbda4162360 con 0x7fbda40b4e80 2024-04-25T10:30:53.950 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:53.946+0000 7fbda9edc640 1 --2- 172.21.15.37:0/320073479 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7fbda40d5160 0x7fbda40cfca0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T10:30:53.951 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:53.946+0000 7fbda8eda640 1 --2- 172.21.15.37:0/320073479 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fbda40d3060 0x7fbda41613a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T10:30:53.951 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:53.946+0000 7fbda96db640 1 --2- 172.21.15.37:0/320073479 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fbda40b4e80 0x7fbda4155230 secure :-1 s=READY pgs=656 cs=0 l=1 rev1=1 crypto rx=0x7fbd9c00cb10 tx=0x7fbd9c00a060 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:30:53.951 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:53.947+0000 7fbda1ffb640 1 -- 172.21.15.37:0/320073479 <== mon.0 v2:172.21.15.37:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbd9c017070 con 0x7fbda40b4e80 2024-04-25T10:30:53.951 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:53.947+0000 7fbda1ffb640 1 -- 172.21.15.37:0/320073479 <== mon.0 v2:172.21.15.37:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fbd9c00ac10 con 0x7fbda40b4e80 2024-04-25T10:30:53.951 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:53.947+0000 7fbdab93f640 1 -- 172.21.15.37:0/320073479 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fbda4162640 con 0x7fbda40b4e80 2024-04-25T10:30:53.951 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:53.947+0000 7fbda1ffb640 1 -- 172.21.15.37:0/320073479 <== mon.0 v2:172.21.15.37:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbd9c012400 con 0x7fbda40b4e80 2024-04-25T10:30:53.951 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:53.947+0000 7fbdab93f640 1 -- 172.21.15.37:0/320073479 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fbda4156310 con 0x7fbda40b4e80 2024-04-25T10:30:53.952 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:53.948+0000 7fbdab93f640 1 -- 172.21.15.37:0/320073479 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fbda4156570 con 0x7fbda40b4e80 2024-04-25T10:30:53.952 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:53.949+0000 7fbda1ffb640 1 -- 172.21.15.37:0/320073479 <== mon.0 v2:172.21.15.37:3300/0 4 ==== mgrmap(e 5) ==== 96782+0+0 (secure 0 0 0) 0x7fbd9c01a070 con 0x7fbda40b4e80 2024-04-25T10:30:53.952 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:53.949+0000 7fbda1ffb640 1 --2- 172.21.15.37:0/320073479 >> [v2:172.21.15.203:6832/2418205237,v1:172.21.15.203:6833/2418205237] conn(0x7fbd900744b0 0x7fbd900769c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:30:53.952 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:53.949+0000 7fbda1ffb640 1 -- 172.21.15.37:0/320073479 <== mon.0 v2:172.21.15.37:3300/0 5 ==== osd_map(46..46 src has 1..46) ==== 6364+0+0 (secure 0 0 0) 0x7fbd9c095180 con 0x7fbda40b4e80 2024-04-25T10:30:53.952 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:53.949+0000 7fbda8eda640 1 --2- 172.21.15.37:0/320073479 >> [v2:172.21.15.203:6832/2418205237,v1:172.21.15.203:6833/2418205237] conn(0x7fbd900744b0 0x7fbd900769c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:30:53.952 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:53.950+0000 7fbda8eda640 1 --2- 172.21.15.37:0/320073479 >> [v2:172.21.15.203:6832/2418205237,v1:172.21.15.203:6833/2418205237] conn(0x7fbd900744b0 0x7fbd900769c0 secure :-1 s=READY pgs=297 cs=0 l=1 rev1=1 crypto rx=0x7fbd98006fd0 tx=0x7fbd98008040 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:30:53.956 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:53.954+0000 7fbda1ffb640 1 -- 172.21.15.37:0/320073479 <== mon.0 v2:172.21.15.37:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187246 (secure 0 0 0) 0x7fbda40d3470 con 0x7fbda40b4e80 2024-04-25T10:30:54.186 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:54.184+0000 7fbdab93f640 1 -- 172.21.15.37:0/320073479 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_command({"prefix": "fs rm", "fs_name": "cephfs", "yes_i_really_mean_it": true} v 0) -- 0x7fbda406a140 con 0x7fbda40b4e80 2024-04-25T10:30:54.828 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:54.826+0000 7fbda1ffb640 1 -- 172.21.15.37:0/320073479 <== mon.0 v2:172.21.15.37:3300/0 7 ==== mon_command_ack([{"prefix": "fs rm", "fs_name": "cephfs", "yes_i_really_mean_it": true}]=0 v29) ==== 104+0+0 (secure 0 0 0) 0x7fbd9c05f7a0 con 0x7fbda40b4e80 2024-04-25T10:30:54.831 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:54.829+0000 7fbdab93f640 1 -- 172.21.15.37:0/320073479 >> [v2:172.21.15.203:6832/2418205237,v1:172.21.15.203:6833/2418205237] conn(0x7fbd900744b0 msgr2=0x7fbd900769c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:30:54.831 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:54.829+0000 7fbdab93f640 1 --2- 172.21.15.37:0/320073479 >> [v2:172.21.15.203:6832/2418205237,v1:172.21.15.203:6833/2418205237] conn(0x7fbd900744b0 0x7fbd900769c0 secure :-1 s=READY pgs=297 cs=0 l=1 rev1=1 crypto rx=0x7fbd98006fd0 tx=0x7fbd98008040 comp rx=0 tx=0).stop 2024-04-25T10:30:54.831 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:54.829+0000 7fbdab93f640 1 -- 172.21.15.37:0/320073479 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fbda40b4e80 msgr2=0x7fbda4155230 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:30:54.831 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:54.829+0000 7fbdab93f640 1 --2- 172.21.15.37:0/320073479 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fbda40b4e80 0x7fbda4155230 secure :-1 s=READY pgs=656 cs=0 l=1 rev1=1 crypto rx=0x7fbd9c00cb10 tx=0x7fbd9c00a060 comp rx=0 tx=0).stop 2024-04-25T10:30:54.832 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:54.829+0000 7fbdab93f640 1 -- 172.21.15.37:0/320073479 shutdown_connections 2024-04-25T10:30:54.832 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:54.829+0000 7fbdab93f640 1 --2- 172.21.15.37:0/320073479 >> [v2:172.21.15.203:6832/2418205237,v1:172.21.15.203:6833/2418205237] conn(0x7fbd900744b0 0x7fbd900769c0 unknown :-1 s=CLOSED pgs=297 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:30:54.832 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:54.829+0000 7fbdab93f640 1 --2- 172.21.15.37:0/320073479 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7fbda40d5160 0x7fbda40cfca0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:30:54.832 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:54.829+0000 7fbdab93f640 1 --2- 172.21.15.37:0/320073479 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fbda40d3060 0x7fbda41613a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:30:54.832 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:54.829+0000 7fbdab93f640 1 --2- 172.21.15.37:0/320073479 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fbda40b4e80 0x7fbda4155230 unknown :-1 s=CLOSED pgs=656 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:30:54.832 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:54.829+0000 7fbdab93f640 1 -- 172.21.15.37:0/320073479 >> 172.21.15.37:0/320073479 conn(0x7fbda40b7a80 msgr2=0x7fbda40b54d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:30:54.832 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:54.829+0000 7fbdab93f640 1 -- 172.21.15.37:0/320073479 shutdown_connections 2024-04-25T10:30:54.832 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:54.829+0000 7fbdab93f640 1 -- 172.21.15.37:0/320073479 wait complete. 2024-04-25T10:30:54.845 DEBUG:teuthology.orchestra.run.smithi037:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool rm cephfs_metadata cephfs_metadata --yes-i-really-really-mean-it 2024-04-25T10:30:54.968 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:54.966+0000 7fc59e78f640 1 Processor -- start 2024-04-25T10:30:54.968 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:54.966+0000 7fc59e78f640 1 -- start start 2024-04-25T10:30:54.970 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:54.967+0000 7fc59e78f640 1 --2- >> v2:172.21.15.37:3300/0 conn(0x7fc5981648e0 0x7fc5980b75c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:30:54.971 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:54.967+0000 7fc59e78f640 1 --2- >> v2:172.21.15.203:3300/0 conn(0x7fc5980b7b00 0x7fc59806db90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:30:54.971 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:54.967+0000 7fc59e78f640 1 -- --> v1:172.21.15.203:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fc5980b7f10 con 0x7fc59806e0d0 2024-04-25T10:30:54.971 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:54.967+0000 7fc59e78f640 1 -- --> v2:172.21.15.37:3300/0 -- mon_getmap magic: 0 -- 0x7fc5980d0b20 con 0x7fc5981648e0 2024-04-25T10:30:54.971 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:54.967+0000 7fc59e78f640 1 -- --> v2:172.21.15.203:3300/0 -- mon_getmap magic: 0 -- 0x7fc598161330 con 0x7fc5980b7b00 2024-04-25T10:30:54.971 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:54.967+0000 7fc597fff640 1 --2- >> v2:172.21.15.37:3300/0 conn(0x7fc5981648e0 0x7fc5980b75c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:30:54.971 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:54.967+0000 7fc597fff640 1 --2- >> v2:172.21.15.37:3300/0 conn(0x7fc5981648e0 0x7fc5980b75c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.37:3300/0 says I am v2:172.21.15.37:59580/0 (socket says 172.21.15.37:59580) 2024-04-25T10:30:54.971 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:54.967+0000 7fc597fff640 1 -- 172.21.15.37:0/3255274276 learned_addr learned my addr 172.21.15.37:0/3255274276 (peer_addr_for_me v2:172.21.15.37:0/0) 2024-04-25T10:30:54.971 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:54.967+0000 7fc5977fe640 1 --2- 172.21.15.37:0/3255274276 >> v2:172.21.15.203:3300/0 conn(0x7fc5980b7b00 0x7fc59806db90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:30:54.971 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:54.967+0000 7fc597fff640 1 -- 172.21.15.37:0/3255274276 >> v2:172.21.15.203:3300/0 conn(0x7fc5980b7b00 msgr2=0x7fc59806db90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:30:54.971 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:54.967+0000 7fc597fff640 1 --2- 172.21.15.37:0/3255274276 >> v2:172.21.15.203:3300/0 conn(0x7fc5980b7b00 0x7fc59806db90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:30:54.971 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:54.968+0000 7fc597fff640 1 -- 172.21.15.37:0/3255274276 >> v1:172.21.15.203:6789/0 conn(0x7fc59806e0d0 legacy=0x7fc5980d4450 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:30:54.971 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:54.968+0000 7fc597fff640 1 -- 172.21.15.37:0/3255274276 --> v2:172.21.15.37:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fc598163e00 con 0x7fc5981648e0 2024-04-25T10:30:54.971 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:54.968+0000 7fc597fff640 1 --2- 172.21.15.37:0/3255274276 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fc5981648e0 0x7fc5980b75c0 secure :-1 s=READY pgs=657 cs=0 l=1 rev1=1 crypto rx=0x7fc59806ec80 tx=0x7fc58802a270 comp rx=0 tx=0).ready entity=mon.0 client_cookie=6115dfc2c1d4e18d server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:30:54.971 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:54.968+0000 7fc596ffd640 1 -- 172.21.15.37:0/3255274276 <== mon.0 v2:172.21.15.37:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc588037070 con 0x7fc5981648e0 2024-04-25T10:30:54.972 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:54.969+0000 7fc596ffd640 1 -- 172.21.15.37:0/3255274276 <== mon.0 v2:172.21.15.37:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fc588032440 con 0x7fc5981648e0 2024-04-25T10:30:54.972 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:54.969+0000 7fc596ffd640 1 -- 172.21.15.37:0/3255274276 <== mon.0 v2:172.21.15.37:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc588032810 con 0x7fc5981648e0 2024-04-25T10:30:54.972 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:54.969+0000 7fc59e78f640 1 -- 172.21.15.37:0/3255274276 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fc5981648e0 msgr2=0x7fc5980b75c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:30:54.973 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:54.969+0000 7fc59e78f640 1 --2- 172.21.15.37:0/3255274276 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fc5981648e0 0x7fc5980b75c0 secure :-1 s=READY pgs=657 cs=0 l=1 rev1=1 crypto rx=0x7fc59806ec80 tx=0x7fc58802a270 comp rx=0 tx=0).stop 2024-04-25T10:30:54.973 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:54.969+0000 7fc59e78f640 1 -- 172.21.15.37:0/3255274276 shutdown_connections 2024-04-25T10:30:54.973 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:54.969+0000 7fc59e78f640 1 --2- 172.21.15.37:0/3255274276 >> v2:172.21.15.203:3300/0 conn(0x7fc5980b7b00 0x7fc59806db90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:30:54.973 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:54.969+0000 7fc59e78f640 1 --2- 172.21.15.37:0/3255274276 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fc5981648e0 0x7fc5980b75c0 unknown :-1 s=CLOSED pgs=657 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:30:54.973 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:54.969+0000 7fc59e78f640 1 -- 172.21.15.37:0/3255274276 >> 172.21.15.37:0/3255274276 conn(0x7fc5980d27a0 msgr2=0x7fc5980d2bb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:30:54.973 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:54.969+0000 7fc59e78f640 1 -- 172.21.15.37:0/3255274276 shutdown_connections 2024-04-25T10:30:54.973 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:54.969+0000 7fc59e78f640 1 -- 172.21.15.37:0/3255274276 wait complete. 2024-04-25T10:30:54.973 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:54.971+0000 7fc59e78f640 1 Processor -- start 2024-04-25T10:30:54.974 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:54.971+0000 7fc59e78f640 1 -- start start 2024-04-25T10:30:54.977 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:54.973+0000 7fc59e78f640 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fc59806e0d0 0x7fc598156300 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:30:54.977 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:54.973+0000 7fc59e78f640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fc5980b7b00 0x7fc5981e9a80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:30:54.977 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:54.973+0000 7fc59e78f640 1 --2- >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7fc5981648e0 0x7fc598158090 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:30:54.977 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:54.973+0000 7fc59e78f640 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 -- 0x7fc5980b4fb0 con 0x7fc5980b7b00 2024-04-25T10:30:54.977 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:54.973+0000 7fc59e78f640 1 -- --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_getmap magic: 0 -- 0x7fc5981643c0 con 0x7fc59806e0d0 2024-04-25T10:30:54.977 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:54.973+0000 7fc59e78f640 1 -- --> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] -- mon_getmap magic: 0 -- 0x7fc5980b5130 con 0x7fc5981648e0 2024-04-25T10:30:54.977 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:54.973+0000 7fc5977fe640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fc5980b7b00 0x7fc5981e9a80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:30:54.977 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:54.973+0000 7fc5977fe640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fc5980b7b00 0x7fc5981e9a80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.37:3300/0 says I am v2:172.21.15.37:59596/0 (socket says 172.21.15.37:59596) 2024-04-25T10:30:54.977 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:54.973+0000 7fc5977fe640 1 -- 172.21.15.37:0/2421850954 learned_addr learned my addr 172.21.15.37:0/2421850954 (peer_addr_for_me v2:172.21.15.37:0/0) 2024-04-25T10:30:54.977 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:54.973+0000 7fc59cd2c640 1 --2- 172.21.15.37:0/2421850954 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7fc5981648e0 0x7fc598158090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:30:54.978 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:54.973+0000 7fc597fff640 1 --2- 172.21.15.37:0/2421850954 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fc59806e0d0 0x7fc598156300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:30:54.978 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:54.973+0000 7fc5977fe640 1 -- 172.21.15.37:0/2421850954 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7fc5981648e0 msgr2=0x7fc598158090 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:30:54.978 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:54.973+0000 7fc5977fe640 1 --2- 172.21.15.37:0/2421850954 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7fc5981648e0 0x7fc598158090 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:30:54.978 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:54.973+0000 7fc5977fe640 1 -- 172.21.15.37:0/2421850954 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fc59806e0d0 msgr2=0x7fc598156300 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:30:54.978 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:54.973+0000 7fc5977fe640 1 --2- 172.21.15.37:0/2421850954 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fc59806e0d0 0x7fc598156300 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:30:54.978 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:54.973+0000 7fc5977fe640 1 -- 172.21.15.37:0/2421850954 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fc5981ea500 con 0x7fc5980b7b00 2024-04-25T10:30:54.978 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:54.973+0000 7fc5977fe640 1 --2- 172.21.15.37:0/2421850954 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fc5980b7b00 0x7fc5981e9a80 secure :-1 s=READY pgs=658 cs=0 l=1 rev1=1 crypto rx=0x7fc58c00d600 tx=0x7fc58c00dac0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:30:54.978 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:54.973+0000 7fc594ff9640 1 -- 172.21.15.37:0/2421850954 <== mon.0 v2:172.21.15.37:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc58c0125f0 con 0x7fc5980b7b00 2024-04-25T10:30:54.978 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:54.973+0000 7fc594ff9640 1 -- 172.21.15.37:0/2421850954 <== mon.0 v2:172.21.15.37:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fc58c019070 con 0x7fc5980b7b00 2024-04-25T10:30:54.978 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:54.973+0000 7fc59e78f640 1 -- 172.21.15.37:0/2421850954 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fc5981ea7e0 con 0x7fc5980b7b00 2024-04-25T10:30:54.978 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:54.973+0000 7fc594ff9640 1 -- 172.21.15.37:0/2421850954 <== mon.0 v2:172.21.15.37:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc58c0129f0 con 0x7fc5980b7b00 2024-04-25T10:30:54.978 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:54.974+0000 7fc59e78f640 1 -- 172.21.15.37:0/2421850954 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fc5981eada0 con 0x7fc5980b7b00 2024-04-25T10:30:54.978 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:54.974+0000 7fc59e78f640 1 -- 172.21.15.37:0/2421850954 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fc5981ea270 con 0x7fc5980b7b00 2024-04-25T10:30:54.979 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:54.975+0000 7fc594ff9640 1 -- 172.21.15.37:0/2421850954 <== mon.0 v2:172.21.15.37:3300/0 4 ==== mgrmap(e 5) ==== 96782+0+0 (secure 0 0 0) 0x7fc58c012b90 con 0x7fc5980b7b00 2024-04-25T10:30:54.979 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:54.976+0000 7fc594ff9640 1 --2- 172.21.15.37:0/2421850954 >> [v2:172.21.15.203:6832/2418205237,v1:172.21.15.203:6833/2418205237] conn(0x7fc5840744b0 0x7fc5840769c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:30:54.979 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:54.976+0000 7fc594ff9640 1 -- 172.21.15.37:0/2421850954 <== mon.0 v2:172.21.15.37:3300/0 5 ==== osd_map(46..46 src has 1..46) ==== 6364+0+0 (secure 0 0 0) 0x7fc58c095950 con 0x7fc5980b7b00 2024-04-25T10:30:54.979 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:54.976+0000 7fc597fff640 1 --2- 172.21.15.37:0/2421850954 >> [v2:172.21.15.203:6832/2418205237,v1:172.21.15.203:6833/2418205237] conn(0x7fc5840744b0 0x7fc5840769c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:30:54.979 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:54.977+0000 7fc597fff640 1 --2- 172.21.15.37:0/2421850954 >> [v2:172.21.15.203:6832/2418205237,v1:172.21.15.203:6833/2418205237] conn(0x7fc5840744b0 0x7fc5840769c0 secure :-1 s=READY pgs=298 cs=0 l=1 rev1=1 crypto rx=0x7fc588034000 tx=0x7fc58802e000 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:30:54.982 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:54.980+0000 7fc594ff9640 1 -- 172.21.15.37:0/2421850954 <== mon.0 v2:172.21.15.37:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187246 (secure 0 0 0) 0x7fc58c05fe50 con 0x7fc5980b7b00 2024-04-25T10:30:55.165 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:55.163+0000 7fc59e78f640 1 -- 172.21.15.37:0/2421850954 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_command({"prefix": "osd pool rm", "pool": "cephfs_metadata", "pool2": "cephfs_metadata", "yes_i_really_really_mean_it": true} v 0) -- 0x7fc59806a140 con 0x7fc5980b7b00 2024-04-25T10:30:55.830 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:55.828+0000 7fc594ff9640 1 -- 172.21.15.37:0/2421850954 <== mon.0 v2:172.21.15.37:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool rm", "pool": "cephfs_metadata", "pool2": "cephfs_metadata", "yes_i_really_really_mean_it": true}]=0 pool 'cephfs_metadata' removed v47) ==== 181+0+0 (secure 0 0 0) 0x7fc58c05f750 con 0x7fc5980b7b00 2024-04-25T10:30:55.830 INFO:teuthology.orchestra.run.smithi037.stderr:pool 'cephfs_metadata' removed 2024-04-25T10:30:55.834 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:55.830+0000 7fc59e78f640 1 -- 172.21.15.37:0/2421850954 >> [v2:172.21.15.203:6832/2418205237,v1:172.21.15.203:6833/2418205237] conn(0x7fc5840744b0 msgr2=0x7fc5840769c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:30:55.834 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:55.830+0000 7fc59e78f640 1 --2- 172.21.15.37:0/2421850954 >> [v2:172.21.15.203:6832/2418205237,v1:172.21.15.203:6833/2418205237] conn(0x7fc5840744b0 0x7fc5840769c0 secure :-1 s=READY pgs=298 cs=0 l=1 rev1=1 crypto rx=0x7fc588034000 tx=0x7fc58802e000 comp rx=0 tx=0).stop 2024-04-25T10:30:55.834 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:55.830+0000 7fc59e78f640 1 -- 172.21.15.37:0/2421850954 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fc5980b7b00 msgr2=0x7fc5981e9a80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:30:55.834 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:55.830+0000 7fc59e78f640 1 --2- 172.21.15.37:0/2421850954 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fc5980b7b00 0x7fc5981e9a80 secure :-1 s=READY pgs=658 cs=0 l=1 rev1=1 crypto rx=0x7fc58c00d600 tx=0x7fc58c00dac0 comp rx=0 tx=0).stop 2024-04-25T10:30:55.834 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:55.830+0000 7fc59e78f640 1 -- 172.21.15.37:0/2421850954 shutdown_connections 2024-04-25T10:30:55.834 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:55.830+0000 7fc59e78f640 1 --2- 172.21.15.37:0/2421850954 >> [v2:172.21.15.203:6832/2418205237,v1:172.21.15.203:6833/2418205237] conn(0x7fc5840744b0 0x7fc5840769c0 unknown :-1 s=CLOSED pgs=298 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:30:55.834 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:55.830+0000 7fc59e78f640 1 --2- 172.21.15.37:0/2421850954 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7fc5981648e0 0x7fc598158090 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:30:55.835 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:55.830+0000 7fc59e78f640 1 --2- 172.21.15.37:0/2421850954 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fc5980b7b00 0x7fc5981e9a80 unknown :-1 s=CLOSED pgs=658 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:30:55.835 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:55.830+0000 7fc59e78f640 1 --2- 172.21.15.37:0/2421850954 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fc59806e0d0 0x7fc598156300 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:30:55.835 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:55.830+0000 7fc59e78f640 1 -- 172.21.15.37:0/2421850954 >> 172.21.15.37:0/2421850954 conn(0x7fc5980d27a0 msgr2=0x7fc5980b6060 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:30:55.835 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:55.831+0000 7fc59e78f640 1 -- 172.21.15.37:0/2421850954 shutdown_connections 2024-04-25T10:30:55.835 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:55.831+0000 7fc59e78f640 1 -- 172.21.15.37:0/2421850954 wait complete. 2024-04-25T10:30:55.850 DEBUG:teuthology.orchestra.run.smithi037:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool rm cephfs_data cephfs_data --yes-i-really-really-mean-it 2024-04-25T10:30:55.972 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:55.970+0000 7f911ee84640 1 Processor -- start 2024-04-25T10:30:55.972 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:55.970+0000 7f911ee84640 1 -- start start 2024-04-25T10:30:55.975 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:55.971+0000 7f911ee84640 1 --2- >> v2:172.21.15.203:3300/0 conn(0x7f9118145a20 0x7f91180702e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:30:55.975 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:55.971+0000 7f911ee84640 1 --2- >> v2:172.21.15.37:3300/0 conn(0x7f9118070820 0x7f91180b4bd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:30:55.975 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:55.971+0000 7f911ee84640 1 -- --> v1:172.21.15.37:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f9118070c30 con 0x7f91180d13c0 2024-04-25T10:30:55.975 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:55.971+0000 7f911ee84640 1 -- --> v2:172.21.15.37:3300/0 -- mon_getmap magic: 0 -- 0x7f91180d0f30 con 0x7f9118070820 2024-04-25T10:30:55.975 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:55.971+0000 7f911ee84640 1 -- --> v2:172.21.15.203:3300/0 -- mon_getmap magic: 0 -- 0x7f91180d0320 con 0x7f9118145a20 2024-04-25T10:30:55.975 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:55.971+0000 7f911cc20640 1 --1- >> v1:172.21.15.37:6789/0 conn(0x7f91180d13c0 0x7f9118073b80 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.37:6789/0 says I am v1:172.21.15.37:38756/0 (socket says 172.21.15.37:38756) 2024-04-25T10:30:55.975 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:55.971+0000 7f911cc20640 1 -- 172.21.15.37:0/1455424373 learned_addr learned my addr 172.21.15.37:0/1455424373 (peer_addr_for_me v1:172.21.15.37:0/0) 2024-04-25T10:30:55.975 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:55.971+0000 7f911d421640 1 --2- 172.21.15.37:0/1455424373 >> v2:172.21.15.37:3300/0 conn(0x7f9118070820 0x7f91180b4bd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:30:55.975 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:55.972+0000 7f9117fff640 1 --2- 172.21.15.37:0/1455424373 >> v2:172.21.15.203:3300/0 conn(0x7f9118145a20 0x7f91180702e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:30:55.975 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:55.972+0000 7f91177fe640 1 -- 172.21.15.37:0/1455424373 <== mon.0 v1:172.21.15.37:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 3895103817 0 0) 0x7f910c001110 con 0x7f91180d13c0 2024-04-25T10:30:55.975 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:55.972+0000 7f91177fe640 1 -- 172.21.15.37:0/1455424373 --> v1:172.21.15.37:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f9108003520 con 0x7f91180d13c0 2024-04-25T10:30:55.975 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:55.972+0000 7f911d421640 1 -- 172.21.15.37:0/1455424373 >> v2:172.21.15.203:3300/0 conn(0x7f9118145a20 msgr2=0x7f91180702e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:30:55.976 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:55.972+0000 7f911d421640 1 --2- 172.21.15.37:0/1455424373 >> v2:172.21.15.203:3300/0 conn(0x7f9118145a20 0x7f91180702e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:30:55.976 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:55.972+0000 7f911d421640 1 -- 172.21.15.37:0/1455424373 >> v1:172.21.15.37:6789/0 conn(0x7f91180d13c0 legacy=0x7f9118073b80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:30:55.976 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:55.972+0000 7f911d421640 1 -- 172.21.15.37:0/1455424373 --> v2:172.21.15.37:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f91180d0600 con 0x7f9118070820 2024-04-25T10:30:55.976 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:55.973+0000 7f911d421640 1 --2- 172.21.15.37:0/1455424373 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f9118070820 0x7f91180b4bd0 secure :-1 s=READY pgs=660 cs=0 l=1 rev1=1 crypto rx=0x7f911806eb90 tx=0x7f911002a5f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=44c47fe4ee407455 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:30:55.976 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:55.973+0000 7f91177fe640 1 -- 172.21.15.37:0/1455424373 <== mon.0 v2:172.21.15.37:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9110007740 con 0x7f9118070820 2024-04-25T10:30:55.976 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:55.973+0000 7f91177fe640 1 -- 172.21.15.37:0/1455424373 <== mon.0 v2:172.21.15.37:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f91100078e0 con 0x7f9118070820 2024-04-25T10:30:55.976 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:55.973+0000 7f91177fe640 1 -- 172.21.15.37:0/1455424373 <== mon.0 v2:172.21.15.37:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9110007cb0 con 0x7f9118070820 2024-04-25T10:30:55.977 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:55.973+0000 7f911ee84640 1 -- 172.21.15.37:0/1455424373 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f9118070820 msgr2=0x7f91180b4bd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:30:55.977 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:55.973+0000 7f911ee84640 1 --2- 172.21.15.37:0/1455424373 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f9118070820 0x7f91180b4bd0 secure :-1 s=READY pgs=660 cs=0 l=1 rev1=1 crypto rx=0x7f911806eb90 tx=0x7f911002a5f0 comp rx=0 tx=0).stop 2024-04-25T10:30:55.977 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:55.974+0000 7f911ee84640 1 -- 172.21.15.37:0/1455424373 shutdown_connections 2024-04-25T10:30:55.977 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:55.974+0000 7f911ee84640 1 --2- 172.21.15.37:0/1455424373 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f9118070820 0x7f91180b4bd0 unknown :-1 s=CLOSED pgs=660 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:30:55.977 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:55.974+0000 7f911ee84640 1 --2- 172.21.15.37:0/1455424373 >> v2:172.21.15.203:3300/0 conn(0x7f9118145a20 0x7f91180702e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:30:55.977 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:55.974+0000 7f911ee84640 1 -- 172.21.15.37:0/1455424373 >> 172.21.15.37:0/1455424373 conn(0x7f91180b35c0 msgr2=0x7f91180b39d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:30:55.977 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:55.974+0000 7f911ee84640 1 -- 172.21.15.37:0/1455424373 shutdown_connections 2024-04-25T10:30:55.977 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:55.974+0000 7f911ee84640 1 -- 172.21.15.37:0/1455424373 wait complete. 2024-04-25T10:30:55.978 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:55.975+0000 7f911ee84640 1 Processor -- start 2024-04-25T10:30:55.978 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:55.975+0000 7f911ee84640 1 -- start start 2024-04-25T10:30:55.980 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:55.977+0000 7f911ee84640 1 --2- >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f9118070820 0x7f911814dd50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:30:55.980 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:55.977+0000 7f911ee84640 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f9118145a20 0x7f911814e290 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:30:55.980 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:55.977+0000 7f911ee84640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f9118153e20 0x7f9118154230 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:30:55.980 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:55.977+0000 7f911ee84640 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 -- 0x7f91180cffd0 con 0x7f9118153e20 2024-04-25T10:30:55.980 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:55.977+0000 7f911ee84640 1 -- --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_getmap magic: 0 -- 0x7f91180d0bc0 con 0x7f9118145a20 2024-04-25T10:30:55.980 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:55.977+0000 7f911ee84640 1 -- --> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] -- mon_getmap magic: 0 -- 0x7f9118145e30 con 0x7f9118070820 2024-04-25T10:30:55.980 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:55.977+0000 7f911d421640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f9118153e20 0x7f9118154230 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:30:55.980 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:55.977+0000 7f911d421640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f9118153e20 0x7f9118154230 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.37:3300/0 says I am v2:172.21.15.37:59618/0 (socket says 172.21.15.37:59618) 2024-04-25T10:30:55.980 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:55.977+0000 7f911d421640 1 -- 172.21.15.37:0/1578649592 learned_addr learned my addr 172.21.15.37:0/1578649592 (peer_addr_for_me v2:172.21.15.37:0/0) 2024-04-25T10:30:55.981 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:55.977+0000 7f9117fff640 1 --2- 172.21.15.37:0/1578649592 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f9118145a20 0x7f911814e290 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:30:55.981 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:55.977+0000 7f911cc20640 1 --2- 172.21.15.37:0/1578649592 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f9118070820 0x7f911814dd50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:30:55.981 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:55.977+0000 7f911d421640 1 -- 172.21.15.37:0/1578649592 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f9118070820 msgr2=0x7f911814dd50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:30:55.981 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:55.977+0000 7f911d421640 1 --2- 172.21.15.37:0/1578649592 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f9118070820 0x7f911814dd50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:30:55.981 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:55.977+0000 7f911d421640 1 -- 172.21.15.37:0/1578649592 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f9118145a20 msgr2=0x7f911814e290 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:30:55.981 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:55.977+0000 7f911d421640 1 --2- 172.21.15.37:0/1578649592 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f9118145a20 0x7f911814e290 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:30:55.981 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:55.977+0000 7f911d421640 1 -- 172.21.15.37:0/1578649592 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f91180abb40 con 0x7f9118153e20 2024-04-25T10:30:55.981 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:55.978+0000 7f911d421640 1 --2- 172.21.15.37:0/1578649592 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f9118153e20 0x7f9118154230 secure :-1 s=READY pgs=661 cs=0 l=1 rev1=1 crypto rx=0x7f9110009e50 tx=0x7f9110004a40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:30:55.981 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:55.978+0000 7f91157fa640 1 -- 172.21.15.37:0/1578649592 <== mon.0 v2:172.21.15.37:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f911003f070 con 0x7f9118153e20 2024-04-25T10:30:55.981 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:55.978+0000 7f91157fa640 1 -- 172.21.15.37:0/1578649592 <== mon.0 v2:172.21.15.37:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f9110031bf0 con 0x7f9118153e20 2024-04-25T10:30:55.981 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:55.978+0000 7f911ee84640 1 -- 172.21.15.37:0/1578649592 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f9118154770 con 0x7f9118153e20 2024-04-25T10:30:55.981 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:55.978+0000 7f91157fa640 1 -- 172.21.15.37:0/1578649592 <== mon.0 v2:172.21.15.37:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9110009450 con 0x7f9118153e20 2024-04-25T10:30:55.981 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:55.978+0000 7f911ee84640 1 -- 172.21.15.37:0/1578649592 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f91180ab580 con 0x7f9118153e20 2024-04-25T10:30:55.982 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:55.979+0000 7f911ee84640 1 -- 172.21.15.37:0/1578649592 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f911806a140 con 0x7f9118153e20 2024-04-25T10:30:55.982 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:55.979+0000 7f91157fa640 1 -- 172.21.15.37:0/1578649592 <== mon.0 v2:172.21.15.37:3300/0 4 ==== mgrmap(e 5) ==== 96782+0+0 (secure 0 0 0) 0x7f9110003990 con 0x7f9118153e20 2024-04-25T10:30:55.982 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:55.980+0000 7f91157fa640 1 --2- 172.21.15.37:0/1578649592 >> [v2:172.21.15.203:6832/2418205237,v1:172.21.15.203:6833/2418205237] conn(0x7f90fc0744e0 0x7f90fc0769f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:30:55.983 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:55.980+0000 7f91157fa640 1 -- 172.21.15.37:0/1578649592 <== mon.0 v2:172.21.15.37:3300/0 5 ==== osd_map(47..47 src has 1..47) ==== 5926+0+0 (secure 0 0 0) 0x7f91100b6a80 con 0x7f9118153e20 2024-04-25T10:30:55.983 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:55.980+0000 7f911cc20640 1 --2- 172.21.15.37:0/1578649592 >> [v2:172.21.15.203:6832/2418205237,v1:172.21.15.203:6833/2418205237] conn(0x7f90fc0744e0 0x7f90fc0769f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:30:55.983 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:55.981+0000 7f911cc20640 1 --2- 172.21.15.37:0/1578649592 >> [v2:172.21.15.203:6832/2418205237,v1:172.21.15.203:6833/2418205237] conn(0x7f90fc0744e0 0x7f90fc0769f0 secure :-1 s=READY pgs=299 cs=0 l=1 rev1=1 crypto rx=0x7f910c007fd0 tx=0x7f910c009040 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:30:55.986 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:55.984+0000 7f91157fa640 1 -- 172.21.15.37:0/1578649592 <== mon.0 v2:172.21.15.37:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187246 (secure 0 0 0) 0x7f9110049060 con 0x7f9118153e20 2024-04-25T10:30:56.170 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:56.167+0000 7f911ee84640 1 -- 172.21.15.37:0/1578649592 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_command({"prefix": "osd pool rm", "pool": "cephfs_data", "pool2": "cephfs_data", "yes_i_really_really_mean_it": true} v 0) -- 0x7f91180cece0 con 0x7f9118153e20 2024-04-25T10:30:56.170 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:56.168+0000 7f91157fa640 1 -- 172.21.15.37:0/1578649592 <== mon.0 v2:172.21.15.37:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool rm", "pool": "cephfs_data", "pool2": "cephfs_data", "yes_i_really_really_mean_it": true}]=-16 pool 'cephfs_data' is in use by CephFS v47) ==== 181+0+0 (secure 0 0 0) 0x7f91100811c0 con 0x7f9118153e20 2024-04-25T10:30:56.170 INFO:teuthology.orchestra.run.smithi037.stderr:Error EBUSY: pool 'cephfs_data' is in use by CephFS 2024-04-25T10:30:56.175 DEBUG:teuthology.orchestra.run:got remote process result: 16 2024-04-25T10:30:56.176 DEBUG:teuthology.orchestra.run.smithi037:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-04-25T10:30:56.292 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:56.289+0000 7f010f6d1640 1 Processor -- start 2024-04-25T10:30:56.292 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:56.289+0000 7f010f6d1640 1 -- start start 2024-04-25T10:30:56.296 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:56.291+0000 7f010f6d1640 1 --2- >> v2:172.21.15.203:3300/0 conn(0x7f010806e260 0x7f010806e670 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:30:56.296 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:56.291+0000 7f010f6d1640 1 --2- >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f0108164ce0 0x7f01080b3c80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:30:56.296 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:56.291+0000 7f010f6d1640 1 -- --> v1:172.21.15.37:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f010809ce70 con 0x7f010806ebb0 2024-04-25T10:30:56.296 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:56.291+0000 7f010f6d1640 1 -- --> v2:172.21.15.203:3300/0 -- mon_getmap magic: 0 -- 0x7f01080d3370 con 0x7f010806e260 2024-04-25T10:30:56.296 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:56.291+0000 7f010f6d1640 1 -- --> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] -- mon_getmap magic: 0 -- 0x7f01080d3090 con 0x7f0108164ce0 2024-04-25T10:30:56.296 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:56.291+0000 7f010cc6c640 1 --1- >> v1:172.21.15.37:6789/0 conn(0x7f010806ebb0 0x7f01081645d0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.37:6789/0 says I am v1:172.21.15.37:38768/0 (socket says 172.21.15.37:38768) 2024-04-25T10:30:56.296 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:56.291+0000 7f010cc6c640 1 -- 172.21.15.37:0/3182852802 learned_addr learned my addr 172.21.15.37:0/3182852802 (peer_addr_for_me v1:172.21.15.37:0/0) 2024-04-25T10:30:56.296 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:56.291+0000 7f010d46d640 1 --2- 172.21.15.37:0/3182852802 >> v2:172.21.15.203:3300/0 conn(0x7f010806e260 0x7f010806e670 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:30:56.297 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:56.291+0000 7f010dc6e640 1 --2- 172.21.15.37:0/3182852802 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f0108164ce0 0x7f01080b3c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:30:56.297 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:56.291+0000 7f00fffff640 1 -- 172.21.15.37:0/3182852802 <== mon.0 v1:172.21.15.37:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 1172257460 0 0) 0x7f010809ce70 con 0x7f010806ebb0 2024-04-25T10:30:56.297 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:56.291+0000 7f00fffff640 1 -- 172.21.15.37:0/3182852802 --> v1:172.21.15.37:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f00f4003520 con 0x7f010806ebb0 2024-04-25T10:30:56.297 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:56.291+0000 7f00fffff640 1 -- 172.21.15.37:0/3182852802 <== mon.0 v1:172.21.15.37:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 3609793068 0 0) 0x7f00f4003520 con 0x7f010806ebb0 2024-04-25T10:30:56.297 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:56.291+0000 7f00fffff640 1 -- 172.21.15.37:0/3182852802 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f0108164ce0 msgr2=0x7f01080b3c80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:30:56.297 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:56.291+0000 7f00fffff640 1 --2- 172.21.15.37:0/3182852802 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f0108164ce0 0x7f01080b3c80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:30:56.297 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:56.291+0000 7f00fffff640 1 -- 172.21.15.37:0/3182852802 >> v2:172.21.15.203:3300/0 conn(0x7f010806e260 msgr2=0x7f010806e670 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:30:56.297 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:56.291+0000 7f00fffff640 1 --2- 172.21.15.37:0/3182852802 >> v2:172.21.15.203:3300/0 conn(0x7f010806e260 0x7f010806e670 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:30:56.297 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:56.292+0000 7f00fffff640 1 -- 172.21.15.37:0/3182852802 --> v1:172.21.15.37:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f010809d150 con 0x7f010806ebb0 2024-04-25T10:30:56.297 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:56.292+0000 7f00fffff640 1 -- 172.21.15.37:0/3182852802 <== mon.0 v1:172.21.15.37:6789/0 3 ==== mon_map magic: 0 ==== 413+0+0 (unknown 1577794600 0 0) 0x7f00f8001670 con 0x7f010806ebb0 2024-04-25T10:30:56.297 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:56.292+0000 7f00fffff640 1 -- 172.21.15.37:0/3182852802 >> v1:172.21.15.37:6789/0 conn(0x7f010806ebb0 legacy=0x7f01081645d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:30:56.297 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:56.292+0000 7f00fffff640 1 --2- 172.21.15.37:0/3182852802 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f00f40039b0 0x7f00f4003e20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:30:56.297 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:56.292+0000 7f00fffff640 1 --2- 172.21.15.37:0/3182852802 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f00f4004800 0x7f00f4004c10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:30:56.297 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:56.292+0000 7f00fffff640 1 --2- 172.21.15.37:0/3182852802 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f0108164ce0 0x7f00f4005150 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:30:56.297 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:56.292+0000 7f00fffff640 1 -- 172.21.15.37:0/3182852802 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 -- 0x7f01080d3090 con 0x7f00f4004800 2024-04-25T10:30:56.298 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:56.292+0000 7f00fffff640 1 -- 172.21.15.37:0/3182852802 --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_getmap magic: 0 -- 0x7f00f400ba70 con 0x7f00f40039b0 2024-04-25T10:30:56.298 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:56.292+0000 7f00fffff640 1 -- 172.21.15.37:0/3182852802 --> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] -- mon_getmap magic: 0 -- 0x7f00f400bbf0 con 0x7f0108164ce0 2024-04-25T10:30:56.298 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:56.292+0000 7f010cc6c640 1 --2- 172.21.15.37:0/3182852802 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f00f4004800 0x7f00f4004c10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:30:56.298 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:56.292+0000 7f010d46d640 1 --2- 172.21.15.37:0/3182852802 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f00f40039b0 0x7f00f4003e20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:30:56.298 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:56.292+0000 7f010dc6e640 1 --2- 172.21.15.37:0/3182852802 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f0108164ce0 0x7f00f4005150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:30:56.298 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:56.292+0000 7f010cc6c640 1 -- 172.21.15.37:0/3182852802 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f0108164ce0 msgr2=0x7f00f4005150 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:30:56.298 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:56.292+0000 7f010cc6c640 1 --2- 172.21.15.37:0/3182852802 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f0108164ce0 0x7f00f4005150 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:30:56.298 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:56.292+0000 7f010cc6c640 1 -- 172.21.15.37:0/3182852802 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f00f40039b0 msgr2=0x7f00f4003e20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:30:56.298 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:56.292+0000 7f010cc6c640 1 --2- 172.21.15.37:0/3182852802 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f00f40039b0 0x7f00f4003e20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:30:56.298 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:56.292+0000 7f010cc6c640 1 -- 172.21.15.37:0/3182852802 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f00f4003520 con 0x7f00f4004800 2024-04-25T10:30:56.298 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:56.293+0000 7f010cc6c640 1 --2- 172.21.15.37:0/3182852802 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f00f4004800 0x7f00f4004c10 secure :-1 s=READY pgs=663 cs=0 l=1 rev1=1 crypto rx=0x7f00f800c210 tx=0x7f00f4005830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ffb1cb9b649945dd server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:30:56.298 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:56.293+0000 7f00fffff640 1 -- 172.21.15.37:0/3182852802 <== mon.0 v2:172.21.15.37:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f00f8007d50 con 0x7f00f4004800 2024-04-25T10:30:56.298 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:56.293+0000 7f00fffff640 1 -- 172.21.15.37:0/3182852802 <== mon.0 v2:172.21.15.37:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f00f8005af0 con 0x7f00f4004800 2024-04-25T10:30:56.298 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:56.293+0000 7f00fffff640 1 -- 172.21.15.37:0/3182852802 <== mon.0 v2:172.21.15.37:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f00f8007070 con 0x7f00f4004800 2024-04-25T10:30:56.299 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:56.294+0000 7f010f6d1640 1 -- 172.21.15.37:0/3182852802 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f00f4004800 msgr2=0x7f00f4004c10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:30:56.299 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:56.294+0000 7f010f6d1640 1 --2- 172.21.15.37:0/3182852802 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f00f4004800 0x7f00f4004c10 secure :-1 s=READY pgs=663 cs=0 l=1 rev1=1 crypto rx=0x7f00f800c210 tx=0x7f00f4005830 comp rx=0 tx=0).stop 2024-04-25T10:30:56.299 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:56.294+0000 7f010dc6e640 1 -- 172.21.15.37:0/3182852802 reap_dead start 2024-04-25T10:30:56.299 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:56.294+0000 7f010f6d1640 1 -- 172.21.15.37:0/3182852802 shutdown_connections 2024-04-25T10:30:56.299 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:56.294+0000 7f010f6d1640 1 -- 172.21.15.37:0/3182852802 >> 172.21.15.37:0/3182852802 conn(0x7f01080d0390 msgr2=0x7f01080d07a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:30:56.299 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:56.294+0000 7f010f6d1640 1 -- 172.21.15.37:0/3182852802 shutdown_connections 2024-04-25T10:30:56.299 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:56.294+0000 7f010f6d1640 1 -- 172.21.15.37:0/3182852802 wait complete. 2024-04-25T10:30:56.303 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:56.295+0000 7f010f6d1640 1 Processor -- start 2024-04-25T10:30:56.303 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:56.296+0000 7f010f6d1640 1 -- start start 2024-04-25T10:30:56.303 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:56.297+0000 7f010f6d1640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f00f4004800 0x7f010815d960 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:30:56.303 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:56.297+0000 7f010f6d1640 1 --2- >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f0108160de0 0x7f010815e810 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:30:56.303 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:56.297+0000 7f010f6d1640 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f01081611f0 0x7f01080b6760 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:30:56.303 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:56.297+0000 7f010f6d1640 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 -- 0x7f010806db30 con 0x7f00f4004800 2024-04-25T10:30:56.303 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:56.297+0000 7f010f6d1640 1 -- --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_getmap magic: 0 -- 0x7f010806d830 con 0x7f01081611f0 2024-04-25T10:30:56.303 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:56.297+0000 7f010f6d1640 1 -- --> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] -- mon_getmap magic: 0 -- 0x7f010806dcb0 con 0x7f0108160de0 2024-04-25T10:30:56.303 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:56.297+0000 7f010d46d640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f00f4004800 0x7f010815d960 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:30:56.303 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:56.297+0000 7f010d46d640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f00f4004800 0x7f010815d960 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.37:3300/0 says I am v2:172.21.15.37:59624/0 (socket says 172.21.15.37:59624) 2024-04-25T10:30:56.303 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:56.297+0000 7f010d46d640 1 -- 172.21.15.37:0/3051450502 learned_addr learned my addr 172.21.15.37:0/3051450502 (peer_addr_for_me v2:172.21.15.37:0/0) 2024-04-25T10:30:56.303 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:56.297+0000 7f010cc6c640 1 --2- 172.21.15.37:0/3051450502 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f0108160de0 0x7f010815e810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:30:56.303 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:56.297+0000 7f010dc6e640 1 --2- 172.21.15.37:0/3051450502 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f01081611f0 0x7f01080b6760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:30:56.304 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:56.297+0000 7f010d46d640 1 -- 172.21.15.37:0/3051450502 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f0108160de0 msgr2=0x7f010815e810 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:30:56.304 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:56.297+0000 7f010d46d640 1 --2- 172.21.15.37:0/3051450502 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f0108160de0 0x7f010815e810 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:30:56.304 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:56.297+0000 7f010d46d640 1 -- 172.21.15.37:0/3051450502 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f01081611f0 msgr2=0x7f01080b6760 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:30:56.304 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:56.297+0000 7f010d46d640 1 --2- 172.21.15.37:0/3051450502 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f01081611f0 0x7f01080b6760 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:30:56.304 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:56.297+0000 7f010d46d640 1 -- 172.21.15.37:0/3051450502 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f010815aed0 con 0x7f00f4004800 2024-04-25T10:30:56.304 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:56.297+0000 7f010dc6e640 1 --2- 172.21.15.37:0/3051450502 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f01081611f0 0x7f01080b6760 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T10:30:56.304 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:56.297+0000 7f010d46d640 1 --2- 172.21.15.37:0/3051450502 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f00f4004800 0x7f010815d960 secure :-1 s=READY pgs=664 cs=0 l=1 rev1=1 crypto rx=0x7f0104009f50 tx=0x7f010400f580 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:30:56.304 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:56.297+0000 7f00fdffb640 1 -- 172.21.15.37:0/3051450502 <== mon.0 v2:172.21.15.37:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0104015430 con 0x7f00f4004800 2024-04-25T10:30:56.304 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:56.298+0000 7f00fdffb640 1 -- 172.21.15.37:0/3051450502 <== mon.0 v2:172.21.15.37:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f01040155d0 con 0x7f00f4004800 2024-04-25T10:30:56.304 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:56.298+0000 7f010f6d1640 1 -- 172.21.15.37:0/3051450502 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f01081502a0 con 0x7f00f4004800 2024-04-25T10:30:56.304 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:56.298+0000 7f00fdffb640 1 -- 172.21.15.37:0/3051450502 <== mon.0 v2:172.21.15.37:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0104015980 con 0x7f00f4004800 2024-04-25T10:30:56.304 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:56.298+0000 7f010f6d1640 1 -- 172.21.15.37:0/3051450502 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f010815cba0 con 0x7f00f4004800 2024-04-25T10:30:56.304 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:56.299+0000 7f010f6d1640 1 -- 172.21.15.37:0/3051450502 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f010815d5a0 con 0x7f00f4004800 2024-04-25T10:30:56.304 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:56.299+0000 7f00fdffb640 1 -- 172.21.15.37:0/3051450502 <== mon.0 v2:172.21.15.37:3300/0 4 ==== mgrmap(e 5) ==== 96782+0+0 (secure 0 0 0) 0x7f0104015b20 con 0x7f00f4004800 2024-04-25T10:30:56.305 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:56.300+0000 7f00fdffb640 1 --2- 172.21.15.37:0/3051450502 >> [v2:172.21.15.203:6832/2418205237,v1:172.21.15.203:6833/2418205237] conn(0x7f00ec0744e0 0x7f00ec0769f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:30:56.305 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:56.300+0000 7f00fdffb640 1 -- 172.21.15.37:0/3051450502 <== mon.0 v2:172.21.15.37:3300/0 5 ==== osd_map(47..47 src has 1..47) ==== 5926+0+0 (secure 0 0 0) 0x7f0104096360 con 0x7f00f4004800 2024-04-25T10:30:56.305 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:56.300+0000 7f010cc6c640 1 --2- 172.21.15.37:0/3051450502 >> [v2:172.21.15.203:6832/2418205237,v1:172.21.15.203:6833/2418205237] conn(0x7f00ec0744e0 0x7f00ec0769f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:30:56.305 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:56.301+0000 7f010cc6c640 1 --2- 172.21.15.37:0/3051450502 >> [v2:172.21.15.203:6832/2418205237,v1:172.21.15.203:6833/2418205237] conn(0x7f00ec0744e0 0x7f00ec0769f0 secure :-1 s=READY pgs=300 cs=0 l=1 rev1=1 crypto rx=0x7f00f800c210 tx=0x7f00f8009030 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:30:56.307 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:56.305+0000 7f00fdffb640 1 -- 172.21.15.37:0/3051450502 <== mon.0 v2:172.21.15.37:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187246 (secure 0 0 0) 0x7f010815aed0 con 0x7f00f4004800 2024-04-25T10:30:56.491 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:56.489+0000 7f010f6d1640 1 -- 172.21.15.37:0/3051450502 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7f010806a140 con 0x7f00f4004800 2024-04-25T10:30:56.492 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:56.491+0000 7f00fdffb640 1 -- 172.21.15.37:0/3051450502 <== mon.0 v2:172.21.15.37:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v47) ==== 74+0+17009 (secure 0 0 0) 0x7f0104019030 con 0x7f00f4004800 2024-04-25T10:30:56.493 INFO:teuthology.orchestra.run.smithi037.stdout: 2024-04-25T10:30:56.493 INFO:teuthology.orchestra.run.smithi037.stdout:{"epoch":47,"fsid":"f4acda28-be34-492a-b286-994afd36f61d","created":"2024-04-25T10:28:43.985718+0000","modified":"2024-04-25T10:30:55.827846+0000","last_up_change":"2024-04-25T10:28:54.900654+0000","last_in_change":"2024-04-25T10:28:46.216973+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":6,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"luminous","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-04-25T10:28:56.202153+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"19","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":5,"pool_name":"cephfs_data","create_time":"2024-04-25T10:30:11.084827+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"43","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"recovery_fs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.3799999952316284,"score_stable":1.3799999952316284,"optimal_score":1,"raw_score_acting":1.3799999952316284,"raw_score_stable":1.3799999952316284,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":6,"pool_name":"recovery_fs_metadata","create_time":"2024-04-25T10:30:26.903737+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"44","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"recovery_fs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"a2f79028-b332-4166-9ebb-a7024a37ea22","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":42,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.37:6816","nonce":2746074720},{"type":"v1","addr":"172.21.15.37:6817","nonce":2746074720}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.37:6818","nonce":2746074720},{"type":"v1","addr":"172.21.15.37:6819","nonce":2746074720}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.37:6822","nonce":2746074720},{"type":"v1","addr":"172.21.15.37:6823","nonce":2746074720}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.37:6820","nonce":2746074720},{"type":"v1","addr":"172.21.15.37:6821","nonce":2746074720}]},"public_addr":"172.21.15.37:6817/2746074720","cluster_addr":"172.21.15.37:6819/2746074720","heartbeat_back_addr":"172.21.15.37:6823/2746074720","heartbeat_front_addr":"172.21.15.37:6821/2746074720","state":["exists","up"]},{"osd":1,"uuid":"973159de-c2ea-4083-aecc-69373d1c1132","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":45,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.37:6800","nonce":1690341662},{"type":"v1","addr":"172.21.15.37:6801","nonce":1690341662}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.37:6802","nonce":1690341662},{"type":"v1","addr":"172.21.15.37:6803","nonce":1690341662}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.37:6806","nonce":1690341662},{"type":"v1","addr":"172.21.15.37:6807","nonce":1690341662}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.37:6804","nonce":1690341662},{"type":"v1","addr":"172.21.15.37:6805","nonce":1690341662}]},"public_addr":"172.21.15.37:6801/1690341662","cluster_addr":"172.21.15.37:6803/1690341662","heartbeat_back_addr":"172.21.15.37:6807/1690341662","heartbeat_front_addr":"172.21.15.37:6805/1690341662","state":["exists","up"]},{"osd":2,"uuid":"4ae7a0e9-cb3f-484c-a75f-f3766b721c47","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":42,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.37:6808","nonce":2754669649},{"type":"v1","addr":"172.21.15.37:6809","nonce":2754669649}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.37:6810","nonce":2754669649},{"type":"v1","addr":"172.21.15.37:6811","nonce":2754669649}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.37:6814","nonce":2754669649},{"type":"v1","addr":"172.21.15.37:6815","nonce":2754669649}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.37:6812","nonce":2754669649},{"type":"v1","addr":"172.21.15.37:6813","nonce":2754669649}]},"public_addr":"172.21.15.37:6809/2754669649","cluster_addr":"172.21.15.37:6811/2754669649","heartbeat_back_addr":"172.21.15.37:6815/2754669649","heartbeat_front_addr":"172.21.15.37:6813/2754669649","state":["exists","up"]},{"osd":3,"uuid":"1ba4a746-1608-47a7-ace2-960a5cfae934","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":45,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.37:6824","nonce":1815290618},{"type":"v1","addr":"172.21.15.37:6825","nonce":1815290618}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.37:6826","nonce":1815290618},{"type":"v1","addr":"172.21.15.37:6827","nonce":1815290618}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.37:6830","nonce":1815290618},{"type":"v1","addr":"172.21.15.37:6831","nonce":1815290618}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.37:6828","nonce":1815290618},{"type":"v1","addr":"172.21.15.37:6829","nonce":1815290618}]},"public_addr":"172.21.15.37:6825/1815290618","cluster_addr":"172.21.15.37:6827/1815290618","heartbeat_back_addr":"172.21.15.37:6831/1815290618","heartbeat_front_addr":"172.21.15.37:6829/1815290618","state":["exists","up"]},{"osd":4,"uuid":"d4f59f5d-0413-4b79-87ee-bb212c1fcf62","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":42,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6800","nonce":4051532063},{"type":"v1","addr":"172.21.15.203:6801","nonce":4051532063}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6802","nonce":4051532063},{"type":"v1","addr":"172.21.15.203:6803","nonce":4051532063}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6806","nonce":4051532063},{"type":"v1","addr":"172.21.15.203:6807","nonce":4051532063}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6804","nonce":4051532063},{"type":"v1","addr":"172.21.15.203:6805","nonce":4051532063}]},"public_addr":"172.21.15.203:6801/4051532063","cluster_addr":"172.21.15.203:6803/4051532063","heartbeat_back_addr":"172.21.15.203:6807/4051532063","heartbeat_front_addr":"172.21.15.203:6805/4051532063","state":["exists","up"]},{"osd":5,"uuid":"69710a0e-41d8-4e7f-9a10-17ce540b67be","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":45,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6824","nonce":1203822890},{"type":"v1","addr":"172.21.15.203:6825","nonce":1203822890}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6826","nonce":1203822890},{"type":"v1","addr":"172.21.15.203:6827","nonce":1203822890}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6830","nonce":1203822890},{"type":"v1","addr":"172.21.15.203:6831","nonce":1203822890}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6828","nonce":1203822890},{"type":"v1","addr":"172.21.15.203:6829","nonce":1203822890}]},"public_addr":"172.21.15.203:6825/1203822890","cluster_addr":"172.21.15.203:6827/1203822890","heartbeat_back_addr":"172.21.15.203:6831/1203822890","heartbeat_front_addr":"172.21.15.203:6829/1203822890","state":["exists","up"]},{"osd":6,"uuid":"f29dd047-3b48-473d-a235-6642a6293f20","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":42,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6810","nonce":2309615014},{"type":"v1","addr":"172.21.15.203:6812","nonce":2309615014}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6814","nonce":2309615014},{"type":"v1","addr":"172.21.15.203:6816","nonce":2309615014}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6822","nonce":2309615014},{"type":"v1","addr":"172.21.15.203:6823","nonce":2309615014}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6818","nonce":2309615014},{"type":"v1","addr":"172.21.15.203:6820","nonce":2309615014}]},"public_addr":"172.21.15.203:6812/2309615014","cluster_addr":"172.21.15.203:6816/2309615014","heartbeat_back_addr":"172.21.15.203:6823/2309615014","heartbeat_front_addr":"172.21.15.203:6820/2309615014","state":["exists","up"]},{"osd":7,"uuid":"6e691644-378b-4fdc-8e86-fb5f56e31f29","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":42,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6808","nonce":3688099014},{"type":"v1","addr":"172.21.15.203:6809","nonce":3688099014}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6811","nonce":3688099014},{"type":"v1","addr":"172.21.15.203:6813","nonce":3688099014}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6819","nonce":3688099014},{"type":"v1","addr":"172.21.15.203:6821","nonce":3688099014}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6815","nonce":3688099014},{"type":"v1","addr":"172.21.15.203:6817","nonce":3688099014}]},"public_addr":"172.21.15.203:6809/3688099014","cluster_addr":"172.21.15.203:6813/3688099014","heartbeat_back_addr":"172.21.15.203:6821/3688099014","heartbeat_front_addr":"172.21.15.203:6817/3688099014","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T10:28:51.419141+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T10:28:51.366448+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T10:28:51.348276+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T10:28:51.380519+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T10:28:51.105812+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T10:28:51.384496+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T10:28:51.388755+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T10:28:51.366958+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[{"pgid":"5.15","mappings":[{"from":4,"to":5}]},{"pgid":"5.1a","mappings":[{"from":4,"to":5}]},{"pgid":"5.26","mappings":[{"from":4,"to":5}]},{"pgid":"5.27","mappings":[{"from":4,"to":5}]}],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.37:6835/311552410":"2024-04-26T10:30:24.853732+0000","172.21.15.37:6832/311552410":"2024-04-26T10:30:24.853732+0000"},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-04-25T10:30:56.495 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:56.493+0000 7f010f6d1640 1 -- 172.21.15.37:0/3051450502 >> [v2:172.21.15.203:6832/2418205237,v1:172.21.15.203:6833/2418205237] conn(0x7f00ec0744e0 msgr2=0x7f00ec0769f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:30:56.495 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:56.493+0000 7f010f6d1640 1 --2- 172.21.15.37:0/3051450502 >> [v2:172.21.15.203:6832/2418205237,v1:172.21.15.203:6833/2418205237] conn(0x7f00ec0744e0 0x7f00ec0769f0 secure :-1 s=READY pgs=300 cs=0 l=1 rev1=1 crypto rx=0x7f00f800c210 tx=0x7f00f8009030 comp rx=0 tx=0).stop 2024-04-25T10:30:56.495 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:56.493+0000 7f010f6d1640 1 -- 172.21.15.37:0/3051450502 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f00f4004800 msgr2=0x7f010815d960 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:30:56.495 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:56.493+0000 7f010f6d1640 1 --2- 172.21.15.37:0/3051450502 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f00f4004800 0x7f010815d960 secure :-1 s=READY pgs=664 cs=0 l=1 rev1=1 crypto rx=0x7f0104009f50 tx=0x7f010400f580 comp rx=0 tx=0).stop 2024-04-25T10:30:56.495 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:56.493+0000 7f010f6d1640 1 -- 172.21.15.37:0/3051450502 shutdown_connections 2024-04-25T10:30:56.495 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:56.493+0000 7f010f6d1640 1 --2- 172.21.15.37:0/3051450502 >> [v2:172.21.15.203:6832/2418205237,v1:172.21.15.203:6833/2418205237] conn(0x7f00ec0744e0 0x7f00ec0769f0 unknown :-1 s=CLOSED pgs=300 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:30:56.496 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:56.493+0000 7f010f6d1640 1 --2- 172.21.15.37:0/3051450502 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f01081611f0 0x7f01080b6760 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:30:56.496 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:56.493+0000 7f010f6d1640 1 --2- 172.21.15.37:0/3051450502 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f0108160de0 0x7f010815e810 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:30:56.496 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:56.493+0000 7f010f6d1640 1 --2- 172.21.15.37:0/3051450502 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f00f4004800 0x7f010815d960 unknown :-1 s=CLOSED pgs=664 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:30:56.496 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:56.493+0000 7f010f6d1640 1 -- 172.21.15.37:0/3051450502 >> 172.21.15.37:0/3051450502 conn(0x7f01080d0390 msgr2=0x7f0108071df0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:30:56.496 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:56.493+0000 7f010f6d1640 1 -- 172.21.15.37:0/3051450502 shutdown_connections 2024-04-25T10:30:56.496 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:56.493+0000 7f010f6d1640 1 -- 172.21.15.37:0/3051450502 wait complete. 2024-04-25T10:30:56.507 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2024-04-25T10:28:56.202153+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '19', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 8, 'score_stable': 8, 'optimal_score': 0.25, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 5, 'pool_name': 'cephfs_data', 'create_time': '2024-04-25T10:30:11.084827+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 64, 'pg_placement_num': 64, 'pg_placement_num_target': 64, 'pg_num_target': 64, 'pg_num_pending': 64, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '43', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.8999999761581421}, 'application_metadata': {'cephfs': {'data': 'recovery_fs'}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 1.3799999952316284, 'score_stable': 1.3799999952316284, 'optimal_score': 1, 'raw_score_acting': 1.3799999952316284, 'raw_score_stable': 1.3799999952316284, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 6, 'pool_name': 'recovery_fs_metadata', 'create_time': '2024-04-25T10:30:26.903737+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '44', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_autoscale_bias': 4, 'pg_num_min': 16, 'recovery_priority': 5}, 'application_metadata': {'cephfs': {'metadata': 'recovery_fs'}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 1.5, 'score_stable': 1.5, 'optimal_score': 1, 'raw_score_acting': 1.5, 'raw_score_stable': 1.5, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}] 2024-04-25T10:30:56.508 DEBUG:teuthology.orchestra.run.smithi037:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2024-04-25T10:30:56.622 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:56.620+0000 7f97d6fce640 1 Processor -- start 2024-04-25T10:30:56.622 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:56.620+0000 7f97d6fce640 1 -- start start 2024-04-25T10:30:56.624 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:56.621+0000 7f97d6fce640 1 --2- >> v2:172.21.15.37:3300/0 conn(0x7f97d00b4bb0 0x7f97d01646b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:30:56.625 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:56.621+0000 7f97d6fce640 1 --2- >> v2:172.21.15.203:3300/0 conn(0x7f97d00b7600 0x7f97d00b7a10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:30:56.625 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:56.621+0000 7f97d6fce640 1 -- --> v1:172.21.15.203:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f97d00b7f50 con 0x7f97d00d4560 2024-04-25T10:30:56.625 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:56.621+0000 7f97d6fce640 1 -- --> v2:172.21.15.37:3300/0 -- mon_getmap magic: 0 -- 0x7f97d00d5080 con 0x7f97d00b4bb0 2024-04-25T10:30:56.625 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:56.621+0000 7f97d6fce640 1 -- --> v2:172.21.15.203:3300/0 -- mon_getmap magic: 0 -- 0x7f97d0161190 con 0x7f97d00b7600 2024-04-25T10:30:56.625 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:56.621+0000 7f97d4d6a640 1 --2- >> v2:172.21.15.37:3300/0 conn(0x7f97d00b4bb0 0x7f97d01646b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:30:56.625 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:56.621+0000 7f97d4d6a640 1 --2- >> v2:172.21.15.37:3300/0 conn(0x7f97d00b4bb0 0x7f97d01646b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.37:3300/0 says I am v2:172.21.15.37:59634/0 (socket says 172.21.15.37:59634) 2024-04-25T10:30:56.625 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:56.621+0000 7f97d4d6a640 1 -- 172.21.15.37:0/4235308338 learned_addr learned my addr 172.21.15.37:0/4235308338 (peer_addr_for_me v2:172.21.15.37:0/0) 2024-04-25T10:30:56.625 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:56.621+0000 7f97cffff640 1 --2- 172.21.15.37:0/4235308338 >> v2:172.21.15.203:3300/0 conn(0x7f97d00b7600 0x7f97d00b7a10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:30:56.625 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:56.622+0000 7f97d4d6a640 1 -- 172.21.15.37:0/4235308338 >> v2:172.21.15.203:3300/0 conn(0x7f97d00b7600 msgr2=0x7f97d00b7a10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:30:56.625 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:56.622+0000 7f97d4d6a640 1 --2- 172.21.15.37:0/4235308338 >> v2:172.21.15.203:3300/0 conn(0x7f97d00b7600 0x7f97d00b7a10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:30:56.625 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:56.622+0000 7f97d4d6a640 1 -- 172.21.15.37:0/4235308338 >> v1:172.21.15.203:6789/0 conn(0x7f97d00d4560 legacy=0x7f97d00d4970 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:30:56.625 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:56.622+0000 7f97d4d6a640 1 -- 172.21.15.37:0/4235308338 --> v2:172.21.15.37:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f97d0161470 con 0x7f97d00b4bb0 2024-04-25T10:30:56.625 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:56.622+0000 7f97d4d6a640 1 --2- 172.21.15.37:0/4235308338 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f97d00b4bb0 0x7f97d01646b0 secure :-1 s=READY pgs=665 cs=0 l=1 rev1=1 crypto rx=0x7f97d00ccbe0 tx=0x7f97c802a730 comp rx=0 tx=0).ready entity=mon.0 client_cookie=beff9563ff562c1 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:30:56.626 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:56.623+0000 7f97cf7fe640 1 -- 172.21.15.37:0/4235308338 <== mon.0 v2:172.21.15.37:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f97c8036070 con 0x7f97d00b4bb0 2024-04-25T10:30:56.626 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:56.623+0000 7f97cf7fe640 1 -- 172.21.15.37:0/4235308338 <== mon.0 v2:172.21.15.37:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f97c8031600 con 0x7f97d00b4bb0 2024-04-25T10:30:56.626 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:56.623+0000 7f97cf7fe640 1 -- 172.21.15.37:0/4235308338 <== mon.0 v2:172.21.15.37:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f97c80319d0 con 0x7f97d00b4bb0 2024-04-25T10:30:56.626 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:56.623+0000 7f97d6fce640 1 -- 172.21.15.37:0/4235308338 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f97d00b4bb0 msgr2=0x7f97d01646b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:30:56.627 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:56.623+0000 7f97d6fce640 1 --2- 172.21.15.37:0/4235308338 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f97d00b4bb0 0x7f97d01646b0 secure :-1 s=READY pgs=665 cs=0 l=1 rev1=1 crypto rx=0x7f97d00ccbe0 tx=0x7f97c802a730 comp rx=0 tx=0).stop 2024-04-25T10:30:56.627 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:56.623+0000 7f97d6fce640 1 -- 172.21.15.37:0/4235308338 shutdown_connections 2024-04-25T10:30:56.627 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:56.623+0000 7f97d6fce640 1 --2- 172.21.15.37:0/4235308338 >> v2:172.21.15.203:3300/0 conn(0x7f97d00b7600 0x7f97d00b7a10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:30:56.627 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:56.623+0000 7f97d6fce640 1 --2- 172.21.15.37:0/4235308338 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f97d00b4bb0 0x7f97d01646b0 unknown :-1 s=CLOSED pgs=665 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:30:56.627 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:56.623+0000 7f97d6fce640 1 -- 172.21.15.37:0/4235308338 >> 172.21.15.37:0/4235308338 conn(0x7f97d00d27e0 msgr2=0x7f97d00d2bf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:30:56.627 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:56.623+0000 7f97d6fce640 1 -- 172.21.15.37:0/4235308338 shutdown_connections 2024-04-25T10:30:56.627 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:56.623+0000 7f97d6fce640 1 -- 172.21.15.37:0/4235308338 wait complete. 2024-04-25T10:30:56.627 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:56.625+0000 7f97d6fce640 1 Processor -- start 2024-04-25T10:30:56.628 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:56.625+0000 7f97d6fce640 1 -- start start 2024-04-25T10:30:56.630 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:56.626+0000 7f97d6fce640 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f97d00b4bb0 0x7f97d01fa360 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:30:56.630 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:56.626+0000 7f97d6fce640 1 --2- >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f97d00b7600 0x7f97d00d0850 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:30:56.630 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:56.626+0000 7f97d6fce640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f97d00d4560 0x7f97d015a090 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:30:56.630 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:56.626+0000 7f97d6fce640 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 -- 0x7f97d00b4fc0 con 0x7f97d00d4560 2024-04-25T10:30:56.630 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:56.626+0000 7f97d6fce640 1 -- --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_getmap magic: 0 -- 0x7f97d0164190 con 0x7f97d00b4bb0 2024-04-25T10:30:56.630 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:56.626+0000 7f97d6fce640 1 -- --> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] -- mon_getmap magic: 0 -- 0x7f97d0164bf0 con 0x7f97d00b7600 2024-04-25T10:30:56.630 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:56.627+0000 7f97d556b640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f97d00d4560 0x7f97d015a090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:30:56.630 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:56.627+0000 7f97d556b640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f97d00d4560 0x7f97d015a090 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.37:3300/0 says I am v2:172.21.15.37:59636/0 (socket says 172.21.15.37:59636) 2024-04-25T10:30:56.630 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:56.627+0000 7f97d556b640 1 -- 172.21.15.37:0/38959966 learned_addr learned my addr 172.21.15.37:0/38959966 (peer_addr_for_me v2:172.21.15.37:0/0) 2024-04-25T10:30:56.631 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:56.627+0000 7f97cffff640 1 --2- 172.21.15.37:0/38959966 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f97d00b7600 0x7f97d00d0850 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:30:56.631 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:56.627+0000 7f97d4d6a640 1 --2- 172.21.15.37:0/38959966 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f97d00b4bb0 0x7f97d01fa360 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:30:56.631 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:56.627+0000 7f97d556b640 1 -- 172.21.15.37:0/38959966 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f97d00b7600 msgr2=0x7f97d00d0850 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:30:56.631 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:56.627+0000 7f97d556b640 1 --2- 172.21.15.37:0/38959966 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f97d00b7600 0x7f97d00d0850 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:30:56.631 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:56.627+0000 7f97d556b640 1 -- 172.21.15.37:0/38959966 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f97d00b4bb0 msgr2=0x7f97d01fa360 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:30:56.631 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:56.627+0000 7f97d556b640 1 --2- 172.21.15.37:0/38959966 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f97d00b4bb0 0x7f97d01fa360 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:30:56.631 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:56.627+0000 7f97d556b640 1 -- 172.21.15.37:0/38959966 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f97d00d0d90 con 0x7f97d00d4560 2024-04-25T10:30:56.631 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:56.627+0000 7f97d556b640 1 --2- 172.21.15.37:0/38959966 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f97d00d4560 0x7f97d015a090 secure :-1 s=READY pgs=666 cs=0 l=1 rev1=1 crypto rx=0x7f97c000c3c0 tx=0x7f97c000c7c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:30:56.631 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:56.627+0000 7f97cd7fa640 1 -- 172.21.15.37:0/38959966 <== mon.0 v2:172.21.15.37:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f97c0017070 con 0x7f97d00d4560 2024-04-25T10:30:56.631 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:56.627+0000 7f97cd7fa640 1 -- 172.21.15.37:0/38959966 <== mon.0 v2:172.21.15.37:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f97c00052d0 con 0x7f97d00d4560 2024-04-25T10:30:56.631 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:56.627+0000 7f97d6fce640 1 -- 172.21.15.37:0/38959966 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f97d01f65e0 con 0x7f97d00d4560 2024-04-25T10:30:56.631 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:56.627+0000 7f97cd7fa640 1 -- 172.21.15.37:0/38959966 <== mon.0 v2:172.21.15.37:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f97c0003830 con 0x7f97d00d4560 2024-04-25T10:30:56.631 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:56.627+0000 7f97d6fce640 1 -- 172.21.15.37:0/38959966 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f97d01f6ba0 con 0x7f97d00d4560 2024-04-25T10:30:56.632 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:56.628+0000 7f97d6fce640 1 -- 172.21.15.37:0/38959966 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f97d006a140 con 0x7f97d00d4560 2024-04-25T10:30:56.632 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:56.629+0000 7f97cd7fa640 1 -- 172.21.15.37:0/38959966 <== mon.0 v2:172.21.15.37:3300/0 4 ==== mgrmap(e 5) ==== 96782+0+0 (secure 0 0 0) 0x7f97c001a070 con 0x7f97d00d4560 2024-04-25T10:30:56.632 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:56.630+0000 7f97cd7fa640 1 --2- 172.21.15.37:0/38959966 >> [v2:172.21.15.203:6832/2418205237,v1:172.21.15.203:6833/2418205237] conn(0x7f97b00744e0 0x7f97b00769f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:30:56.632 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:56.630+0000 7f97cd7fa640 1 -- 172.21.15.37:0/38959966 <== mon.0 v2:172.21.15.37:3300/0 5 ==== osd_map(47..47 src has 1..47) ==== 5926+0+0 (secure 0 0 0) 0x7f97c0094590 con 0x7f97d00d4560 2024-04-25T10:30:56.633 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:56.630+0000 7f97d4d6a640 1 --2- 172.21.15.37:0/38959966 >> [v2:172.21.15.203:6832/2418205237,v1:172.21.15.203:6833/2418205237] conn(0x7f97b00744e0 0x7f97b00769f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:30:56.633 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:56.630+0000 7f97d4d6a640 1 --2- 172.21.15.37:0/38959966 >> [v2:172.21.15.203:6832/2418205237,v1:172.21.15.203:6833/2418205237] conn(0x7f97b00744e0 0x7f97b00769f0 secure :-1 s=READY pgs=301 cs=0 l=1 rev1=1 crypto rx=0x7f97c8033000 tx=0x7f97c8009000 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:30:56.636 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:56.634+0000 7f97cd7fa640 1 -- 172.21.15.37:0/38959966 <== mon.0 v2:172.21.15.37:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187246 (secure 0 0 0) 0x7f97d00d0d90 con 0x7f97d00d4560 2024-04-25T10:30:56.819 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:56.817+0000 7f97d6fce640 1 -- 172.21.15.37:0/38959966 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) -- 0x7f97d01f6360 con 0x7f97d00d4560 2024-04-25T10:30:56.820 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:56.818+0000 7f97cd7fa640 1 -- 172.21.15.37:0/38959966 <== mon.0 v2:172.21.15.37:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v47) ==== 93+0+10 (secure 0 0 0) 0x7f97d01f6360 con 0x7f97d00d4560 2024-04-25T10:30:56.820 INFO:teuthology.orchestra.run.smithi037.stdout:pg_num: 1 2024-04-25T10:30:56.822 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:56.819+0000 7f97d6fce640 1 -- 172.21.15.37:0/38959966 >> [v2:172.21.15.203:6832/2418205237,v1:172.21.15.203:6833/2418205237] conn(0x7f97b00744e0 msgr2=0x7f97b00769f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:30:56.822 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:56.819+0000 7f97d6fce640 1 --2- 172.21.15.37:0/38959966 >> [v2:172.21.15.203:6832/2418205237,v1:172.21.15.203:6833/2418205237] conn(0x7f97b00744e0 0x7f97b00769f0 secure :-1 s=READY pgs=301 cs=0 l=1 rev1=1 crypto rx=0x7f97c8033000 tx=0x7f97c8009000 comp rx=0 tx=0).stop 2024-04-25T10:30:56.822 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:56.819+0000 7f97d6fce640 1 -- 172.21.15.37:0/38959966 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f97d00d4560 msgr2=0x7f97d015a090 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:30:56.822 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:56.819+0000 7f97d6fce640 1 --2- 172.21.15.37:0/38959966 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f97d00d4560 0x7f97d015a090 secure :-1 s=READY pgs=666 cs=0 l=1 rev1=1 crypto rx=0x7f97c000c3c0 tx=0x7f97c000c7c0 comp rx=0 tx=0).stop 2024-04-25T10:30:56.822 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:56.820+0000 7f97d6fce640 1 -- 172.21.15.37:0/38959966 shutdown_connections 2024-04-25T10:30:56.822 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:56.820+0000 7f97d6fce640 1 --2- 172.21.15.37:0/38959966 >> [v2:172.21.15.203:6832/2418205237,v1:172.21.15.203:6833/2418205237] conn(0x7f97b00744e0 0x7f97b00769f0 unknown :-1 s=CLOSED pgs=301 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:30:56.822 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:56.820+0000 7f97d6fce640 1 --2- 172.21.15.37:0/38959966 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f97d00d4560 0x7f97d015a090 unknown :-1 s=CLOSED pgs=666 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:30:56.822 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:56.820+0000 7f97d6fce640 1 --2- 172.21.15.37:0/38959966 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f97d00b7600 0x7f97d00d0850 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:30:56.822 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:56.820+0000 7f97d6fce640 1 --2- 172.21.15.37:0/38959966 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f97d00b4bb0 0x7f97d01fa360 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:30:56.823 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:56.820+0000 7f97d6fce640 1 -- 172.21.15.37:0/38959966 >> 172.21.15.37:0/38959966 conn(0x7f97d00d27e0 msgr2=0x7f97d015f590 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:30:56.823 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:56.820+0000 7f97d6fce640 1 -- 172.21.15.37:0/38959966 shutdown_connections 2024-04-25T10:30:56.823 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:56.820+0000 7f97d6fce640 1 -- 172.21.15.37:0/38959966 wait complete. 2024-04-25T10:30:56.834 DEBUG:teuthology.orchestra.run.smithi037:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data pg_num 2024-04-25T10:30:56.948 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:56.946+0000 7f366fa9f640 1 Processor -- start 2024-04-25T10:30:56.949 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:56.946+0000 7f366fa9f640 1 -- start start 2024-04-25T10:30:56.953 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:56.948+0000 7f366fa9f640 1 --2- >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f36680b7600 0x7f36680b7a10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:30:56.953 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:56.948+0000 7f366fa9f640 1 --2- >> v2:172.21.15.203:3300/0 conn(0x7f36680d4560 0x7f36680d4970 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:30:56.953 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:56.948+0000 7f366fa9f640 1 -- --> v1:172.21.15.37:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f36680b7f50 con 0x7f36680b4bb0 2024-04-25T10:30:56.953 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:56.948+0000 7f366fa9f640 1 -- --> v2:172.21.15.203:3300/0 -- mon_getmap magic: 0 -- 0x7f3668161190 con 0x7f36680d4560 2024-04-25T10:30:56.953 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:56.948+0000 7f366fa9f640 1 -- --> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] -- mon_getmap magic: 0 -- 0x7f3668161470 con 0x7f36680b7600 2024-04-25T10:30:56.953 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:56.948+0000 7f366d83b640 1 --1- >> v1:172.21.15.37:6789/0 conn(0x7f36680b4bb0 0x7f36681646b0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.37:6789/0 says I am v1:172.21.15.37:38770/0 (socket says 172.21.15.37:38770) 2024-04-25T10:30:56.953 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:56.948+0000 7f366d83b640 1 -- 172.21.15.37:0/3474499835 learned_addr learned my addr 172.21.15.37:0/3474499835 (peer_addr_for_me v1:172.21.15.37:0/0) 2024-04-25T10:30:56.953 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:56.948+0000 7f366c839640 1 -- 172.21.15.37:0/3474499835 <== mon.0 v1:172.21.15.37:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 3446604017 0 0) 0x7f3660001110 con 0x7f36680b4bb0 2024-04-25T10:30:56.953 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:56.948+0000 7f366c839640 1 -- 172.21.15.37:0/3474499835 --> v1:172.21.15.37:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f3650003520 con 0x7f36680b4bb0 2024-04-25T10:30:56.953 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:56.948+0000 7f366d03a640 1 --2- 172.21.15.37:0/3474499835 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f36680b7600 0x7f36680b7a10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:30:56.953 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:56.948+0000 7f366e03c640 1 --2- 172.21.15.37:0/3474499835 >> v2:172.21.15.203:3300/0 conn(0x7f36680d4560 0x7f36680d4970 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:30:56.954 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:56.948+0000 7f366c839640 1 -- 172.21.15.37:0/3474499835 <== mon.0 v1:172.21.15.37:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 2834363601 0 0) 0x7f3650003520 con 0x7f36680b4bb0 2024-04-25T10:30:56.954 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:56.948+0000 7f366c839640 1 -- 172.21.15.37:0/3474499835 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f36680b7600 msgr2=0x7f36680b7a10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:30:56.954 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:56.948+0000 7f366c839640 1 --2- 172.21.15.37:0/3474499835 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f36680b7600 0x7f36680b7a10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:30:56.954 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:56.948+0000 7f366c839640 1 -- 172.21.15.37:0/3474499835 >> v2:172.21.15.203:3300/0 conn(0x7f36680d4560 msgr2=0x7f36680d4970 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:30:56.954 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:56.948+0000 7f366c839640 1 --2- 172.21.15.37:0/3474499835 >> v2:172.21.15.203:3300/0 conn(0x7f36680d4560 0x7f36680d4970 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:30:56.954 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:56.948+0000 7f366c839640 1 -- 172.21.15.37:0/3474499835 --> v1:172.21.15.37:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f3668163eb0 con 0x7f36680b4bb0 2024-04-25T10:30:56.954 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:56.948+0000 7f366c839640 1 -- 172.21.15.37:0/3474499835 <== mon.0 v1:172.21.15.37:6789/0 3 ==== mon_map magic: 0 ==== 413+0+0 (unknown 1577794600 0 0) 0x7f3660001840 con 0x7f36680b4bb0 2024-04-25T10:30:56.954 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:56.948+0000 7f366d03a640 1 --2- 172.21.15.37:0/3474499835 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f36680b7600 0x7f36680b7a10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T10:30:56.954 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:56.948+0000 7f366c839640 1 -- 172.21.15.37:0/3474499835 >> v1:172.21.15.37:6789/0 conn(0x7f36680b4bb0 legacy=0x7f36681646b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:30:56.954 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:56.948+0000 7f366c839640 1 --2- 172.21.15.37:0/3474499835 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f36500039b0 0x7f3650003e20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:30:56.954 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:56.948+0000 7f366c839640 1 --2- 172.21.15.37:0/3474499835 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f3650004800 0x7f3650004c10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:30:56.954 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:56.948+0000 7f366c839640 1 --2- 172.21.15.37:0/3474499835 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f36680b7600 0x7f3650005150 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:30:56.954 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:56.948+0000 7f366c839640 1 -- 172.21.15.37:0/3474499835 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 -- 0x7f3668161190 con 0x7f3650004800 2024-04-25T10:30:56.954 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:56.948+0000 7f366c839640 1 -- 172.21.15.37:0/3474499835 --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_getmap magic: 0 -- 0x7f3668161470 con 0x7f36500039b0 2024-04-25T10:30:56.955 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:56.948+0000 7f366c839640 1 -- 172.21.15.37:0/3474499835 --> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] -- mon_getmap magic: 0 -- 0x7f365000baf0 con 0x7f36680b7600 2024-04-25T10:30:56.955 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:56.949+0000 7f366d03a640 1 --2- 172.21.15.37:0/3474499835 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f3650004800 0x7f3650004c10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:30:56.955 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:56.949+0000 7f366d83b640 1 --2- 172.21.15.37:0/3474499835 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f36500039b0 0x7f3650003e20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:30:56.955 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:56.949+0000 7f366e03c640 1 --2- 172.21.15.37:0/3474499835 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f36680b7600 0x7f3650005150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:30:56.955 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:56.949+0000 7f366d03a640 1 -- 172.21.15.37:0/3474499835 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f36680b7600 msgr2=0x7f3650005150 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:30:56.955 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:56.949+0000 7f366d03a640 1 --2- 172.21.15.37:0/3474499835 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f36680b7600 0x7f3650005150 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:30:56.955 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:56.949+0000 7f366d03a640 1 -- 172.21.15.37:0/3474499835 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f36500039b0 msgr2=0x7f3650003e20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:30:56.955 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:56.949+0000 7f366d03a640 1 --2- 172.21.15.37:0/3474499835 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f36500039b0 0x7f3650003e20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:30:56.955 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:56.949+0000 7f366d03a640 1 -- 172.21.15.37:0/3474499835 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f3650003520 con 0x7f3650004800 2024-04-25T10:30:56.955 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:56.950+0000 7f366d03a640 1 --2- 172.21.15.37:0/3474499835 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f3650004800 0x7f3650004c10 secure :-1 s=READY pgs=668 cs=0 l=1 rev1=1 crypto rx=0x7f365c0098a0 tx=0x7f3650005830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d628c79ac5f16ff8 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:30:56.955 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:56.950+0000 7f366c839640 1 -- 172.21.15.37:0/3474499835 <== mon.0 v2:172.21.15.37:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f365c00f040 con 0x7f3650004800 2024-04-25T10:30:56.955 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:56.950+0000 7f366c839640 1 -- 172.21.15.37:0/3474499835 <== mon.0 v2:172.21.15.37:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f365c02ace0 con 0x7f3650004800 2024-04-25T10:30:56.955 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:56.950+0000 7f366c839640 1 -- 172.21.15.37:0/3474499835 <== mon.0 v2:172.21.15.37:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f365c004250 con 0x7f3650004800 2024-04-25T10:30:56.955 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:56.950+0000 7f366fa9f640 1 -- 172.21.15.37:0/3474499835 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f3650004800 msgr2=0x7f3650004c10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:30:56.956 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:56.950+0000 7f366fa9f640 1 --2- 172.21.15.37:0/3474499835 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f3650004800 0x7f3650004c10 secure :-1 s=READY pgs=668 cs=0 l=1 rev1=1 crypto rx=0x7f365c0098a0 tx=0x7f3650005830 comp rx=0 tx=0).stop 2024-04-25T10:30:56.956 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:56.950+0000 7f366e03c640 1 -- 172.21.15.37:0/3474499835 reap_dead start 2024-04-25T10:30:56.956 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:56.950+0000 7f366fa9f640 1 -- 172.21.15.37:0/3474499835 shutdown_connections 2024-04-25T10:30:56.956 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:56.950+0000 7f366fa9f640 1 -- 172.21.15.37:0/3474499835 >> 172.21.15.37:0/3474499835 conn(0x7f36680d27e0 msgr2=0x7f36680d2bf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:30:56.956 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:56.951+0000 7f366fa9f640 1 -- 172.21.15.37:0/3474499835 shutdown_connections 2024-04-25T10:30:56.956 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:56.951+0000 7f366fa9f640 1 -- 172.21.15.37:0/3474499835 wait complete. 2024-04-25T10:30:56.958 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:56.952+0000 7f366fa9f640 1 Processor -- start 2024-04-25T10:30:56.958 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:56.952+0000 7f366fa9f640 1 -- start start 2024-04-25T10:30:56.958 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:56.954+0000 7f366fa9f640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f3650004800 0x7f366815a280 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:30:56.958 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:56.954+0000 7f366fa9f640 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f36681f6510 0x7f36680cafb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:30:56.958 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:56.954+0000 7f366fa9f640 1 --2- >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f36681f6920 0x7f36680cb4f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:30:56.958 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:56.954+0000 7f366fa9f640 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 -- 0x7f36680d09e0 con 0x7f3650004800 2024-04-25T10:30:56.959 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:56.954+0000 7f366fa9f640 1 -- --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_getmap magic: 0 -- 0x7f36680b4fc0 con 0x7f36681f6510 2024-04-25T10:30:56.959 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:56.954+0000 7f366fa9f640 1 -- --> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] -- mon_getmap magic: 0 -- 0x7f36680d0b60 con 0x7f36681f6920 2024-04-25T10:30:56.959 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:56.954+0000 7f366d83b640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f3650004800 0x7f366815a280 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:30:56.959 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:56.954+0000 7f366d83b640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f3650004800 0x7f366815a280 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.37:3300/0 says I am v2:172.21.15.37:59658/0 (socket says 172.21.15.37:59658) 2024-04-25T10:30:56.959 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:56.954+0000 7f366d83b640 1 -- 172.21.15.37:0/3823710916 learned_addr learned my addr 172.21.15.37:0/3823710916 (peer_addr_for_me v2:172.21.15.37:0/0) 2024-04-25T10:30:56.959 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:56.954+0000 7f366d03a640 1 --2- 172.21.15.37:0/3823710916 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f36681f6510 0x7f36680cafb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:30:56.959 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:56.954+0000 7f366e03c640 1 --2- 172.21.15.37:0/3823710916 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f36681f6920 0x7f36680cb4f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:30:56.959 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:56.954+0000 7f366d83b640 1 -- 172.21.15.37:0/3823710916 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f36681f6920 msgr2=0x7f36680cb4f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:30:56.959 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:56.954+0000 7f366d83b640 1 --2- 172.21.15.37:0/3823710916 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f36681f6920 0x7f36680cb4f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:30:56.959 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:56.954+0000 7f366d83b640 1 -- 172.21.15.37:0/3823710916 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f36681f6510 msgr2=0x7f36680cafb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:30:56.959 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:56.954+0000 7f366d83b640 1 --2- 172.21.15.37:0/3823710916 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f36681f6510 0x7f36680cafb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:30:56.959 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:56.954+0000 7f366d83b640 1 -- 172.21.15.37:0/3823710916 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f36681f70f0 con 0x7f3650004800 2024-04-25T10:30:56.959 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:56.954+0000 7f366d83b640 1 --2- 172.21.15.37:0/3823710916 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f3650004800 0x7f366815a280 secure :-1 s=READY pgs=669 cs=0 l=1 rev1=1 crypto rx=0x7f3660001d80 tx=0x7f36600077a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:30:56.959 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:56.954+0000 7f36667fc640 1 -- 172.21.15.37:0/3823710916 <== mon.0 v2:172.21.15.37:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f366000f9c0 con 0x7f3650004800 2024-04-25T10:30:56.960 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:56.954+0000 7f36667fc640 1 -- 172.21.15.37:0/3823710916 <== mon.0 v2:172.21.15.37:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f3660018070 con 0x7f3650004800 2024-04-25T10:30:56.960 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:56.954+0000 7f366fa9f640 1 -- 172.21.15.37:0/3823710916 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f36681f73d0 con 0x7f3650004800 2024-04-25T10:30:56.960 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:56.954+0000 7f36667fc640 1 -- 172.21.15.37:0/3823710916 <== mon.0 v2:172.21.15.37:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f366000fdc0 con 0x7f3650004800 2024-04-25T10:30:56.960 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:56.955+0000 7f366fa9f640 1 -- 172.21.15.37:0/3823710916 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f36681e9ea0 con 0x7f3650004800 2024-04-25T10:30:56.960 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:56.955+0000 7f366fa9f640 1 -- 172.21.15.37:0/3823710916 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f366806a140 con 0x7f3650004800 2024-04-25T10:30:56.960 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:56.956+0000 7f36667fc640 1 -- 172.21.15.37:0/3823710916 <== mon.0 v2:172.21.15.37:3300/0 4 ==== mgrmap(e 5) ==== 96782+0+0 (secure 0 0 0) 0x7f366001d460 con 0x7f3650004800 2024-04-25T10:30:56.961 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:56.957+0000 7f36667fc640 1 --2- 172.21.15.37:0/3823710916 >> [v2:172.21.15.203:6832/2418205237,v1:172.21.15.203:6833/2418205237] conn(0x7f36540744e0 0x7f36540769f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:30:56.961 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:56.957+0000 7f36667fc640 1 -- 172.21.15.37:0/3823710916 <== mon.0 v2:172.21.15.37:3300/0 5 ==== osd_map(47..47 src has 1..47) ==== 5926+0+0 (secure 0 0 0) 0x7f3660095400 con 0x7f3650004800 2024-04-25T10:30:56.961 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:56.957+0000 7f366d03a640 1 --2- 172.21.15.37:0/3823710916 >> [v2:172.21.15.203:6832/2418205237,v1:172.21.15.203:6833/2418205237] conn(0x7f36540744e0 0x7f36540769f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:30:56.961 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:56.958+0000 7f366d03a640 1 --2- 172.21.15.37:0/3823710916 >> [v2:172.21.15.203:6832/2418205237,v1:172.21.15.203:6833/2418205237] conn(0x7f36540744e0 0x7f36540769f0 secure :-1 s=READY pgs=302 cs=0 l=1 rev1=1 crypto rx=0x7f365c033000 tx=0x7f365c02e000 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:30:56.963 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:56.961+0000 7f36667fc640 1 -- 172.21.15.37:0/3823710916 <== mon.0 v2:172.21.15.37:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187246 (secure 0 0 0) 0x7f36680b7f50 con 0x7f3650004800 2024-04-25T10:30:57.150 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:57.147+0000 7f366fa9f640 1 -- 172.21.15.37:0/3823710916 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"} v 0) -- 0x7f36680d3ff0 con 0x7f3650004800 2024-04-25T10:30:57.150 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:57.148+0000 7f36667fc640 1 -- 172.21.15.37:0/3823710916 <== mon.0 v2:172.21.15.37:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]=0 v47) ==== 100+0+11 (secure 0 0 0) 0x7f366005fab0 con 0x7f3650004800 2024-04-25T10:30:57.150 INFO:teuthology.orchestra.run.smithi037.stdout:pg_num: 64 2024-04-25T10:30:57.152 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:57.150+0000 7f366fa9f640 1 -- 172.21.15.37:0/3823710916 >> [v2:172.21.15.203:6832/2418205237,v1:172.21.15.203:6833/2418205237] conn(0x7f36540744e0 msgr2=0x7f36540769f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:30:57.152 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:57.150+0000 7f366fa9f640 1 --2- 172.21.15.37:0/3823710916 >> [v2:172.21.15.203:6832/2418205237,v1:172.21.15.203:6833/2418205237] conn(0x7f36540744e0 0x7f36540769f0 secure :-1 s=READY pgs=302 cs=0 l=1 rev1=1 crypto rx=0x7f365c033000 tx=0x7f365c02e000 comp rx=0 tx=0).stop 2024-04-25T10:30:57.152 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:57.150+0000 7f366fa9f640 1 -- 172.21.15.37:0/3823710916 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f3650004800 msgr2=0x7f366815a280 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:30:57.152 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:57.150+0000 7f366fa9f640 1 --2- 172.21.15.37:0/3823710916 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f3650004800 0x7f366815a280 secure :-1 s=READY pgs=669 cs=0 l=1 rev1=1 crypto rx=0x7f3660001d80 tx=0x7f36600077a0 comp rx=0 tx=0).stop 2024-04-25T10:30:57.152 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:57.150+0000 7f366fa9f640 1 -- 172.21.15.37:0/3823710916 shutdown_connections 2024-04-25T10:30:57.152 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:57.150+0000 7f366fa9f640 1 --2- 172.21.15.37:0/3823710916 >> [v2:172.21.15.203:6832/2418205237,v1:172.21.15.203:6833/2418205237] conn(0x7f36540744e0 0x7f36540769f0 unknown :-1 s=CLOSED pgs=302 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:30:57.153 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:57.150+0000 7f366fa9f640 1 --2- 172.21.15.37:0/3823710916 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f36681f6920 0x7f36680cb4f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:30:57.153 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:57.150+0000 7f366fa9f640 1 --2- 172.21.15.37:0/3823710916 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f36681f6510 0x7f36680cafb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:30:57.153 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:57.150+0000 7f366fa9f640 1 --2- 172.21.15.37:0/3823710916 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f3650004800 0x7f366815a280 unknown :-1 s=CLOSED pgs=669 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:30:57.153 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:57.150+0000 7f366fa9f640 1 -- 172.21.15.37:0/3823710916 >> 172.21.15.37:0/3823710916 conn(0x7f36680d27e0 msgr2=0x7f36680b61f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:30:57.153 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:57.150+0000 7f366fa9f640 1 -- 172.21.15.37:0/3823710916 shutdown_connections 2024-04-25T10:30:57.153 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:57.150+0000 7f366fa9f640 1 -- 172.21.15.37:0/3823710916 wait complete. 2024-04-25T10:30:57.163 DEBUG:teuthology.orchestra.run.smithi037:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get recovery_fs_metadata pg_num 2024-04-25T10:30:57.277 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:57.275+0000 7f3fc984c640 1 Processor -- start 2024-04-25T10:30:57.278 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:57.275+0000 7f3fc984c640 1 -- start start 2024-04-25T10:30:57.278 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:57.277+0000 7f3fc984c640 1 --2- >> v2:172.21.15.203:3300/0 conn(0x7f3fc40d5430 0x7f3fc40b75c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:30:57.279 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:57.277+0000 7f3fc984c640 1 --2- >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f3fc40b7b00 0x7f3fc4160a00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:30:57.279 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:57.277+0000 7f3fc984c640 1 -- --> v1:172.21.15.203:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f3fc40b7f10 con 0x7f3fc4160f40 2024-04-25T10:30:57.279 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:57.277+0000 7f3fc984c640 1 -- --> v2:172.21.15.203:3300/0 -- mon_getmap magic: 0 -- 0x7f3fc40d4420 con 0x7f3fc40d5430 2024-04-25T10:30:57.279 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:57.277+0000 7f3fc984c640 1 -- --> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] -- mon_getmap magic: 0 -- 0x7f3fc40d5840 con 0x7f3fc40b7b00 2024-04-25T10:30:57.280 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:57.277+0000 7f3fc27fc640 1 --2- >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f3fc40b7b00 0x7f3fc4160a00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:30:57.280 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:57.277+0000 7f3fc27fc640 1 --2- >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f3fc40b7b00 0x7f3fc4160a00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.203:3301/0 says I am v2:172.21.15.37:59000/0 (socket says 172.21.15.37:59000) 2024-04-25T10:30:57.281 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:57.277+0000 7f3fc2ffd640 1 --2- >> v2:172.21.15.203:3300/0 conn(0x7f3fc40d5430 0x7f3fc40b75c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:30:57.281 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:57.277+0000 7f3fc27fc640 1 -- 172.21.15.37:0/3269201695 learned_addr learned my addr 172.21.15.37:0/3269201695 (peer_addr_for_me v2:172.21.15.37:0/0) 2024-04-25T10:30:57.281 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:57.277+0000 7f3fc1ffb640 1 -- 172.21.15.37:0/3269201695 <== mon.1 v1:172.21.15.203:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 1487328398 0 0) 0x7f3fb4001490 con 0x7f3fc4160f40 2024-04-25T10:30:57.281 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:57.277+0000 7f3fc1ffb640 1 -- 172.21.15.37:0/3269201695 --> v1:172.21.15.203:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f3fb0003520 con 0x7f3fc4160f40 2024-04-25T10:30:57.281 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:57.277+0000 7f3fc2ffd640 1 -- 172.21.15.37:0/3269201695 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f3fc40b7b00 msgr2=0x7f3fc4160a00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:30:57.281 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:57.277+0000 7f3fc2ffd640 1 --2- 172.21.15.37:0/3269201695 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f3fc40b7b00 0x7f3fc4160a00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:30:57.281 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:57.277+0000 7f3fc2ffd640 1 -- 172.21.15.37:0/3269201695 >> v1:172.21.15.203:6789/0 conn(0x7f3fc4160f40 legacy=0x7f3fc4161880 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:30:57.281 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:57.277+0000 7f3fc2ffd640 1 -- 172.21.15.37:0/3269201695 --> v2:172.21.15.203:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f3fc40d4670 con 0x7f3fc40d5430 2024-04-25T10:30:57.281 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:57.277+0000 7f3fc27fc640 1 --2- 172.21.15.37:0/3269201695 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f3fc40b7b00 0x7f3fc4160a00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-25T10:30:57.281 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:57.278+0000 7f3fc2ffd640 1 --2- 172.21.15.37:0/3269201695 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f3fc40d5430 0x7f3fc40b75c0 secure :-1 s=READY pgs=245 cs=0 l=1 rev1=1 crypto rx=0x7f3fbc0097a0 tx=0x7f3fbc02a100 comp rx=0 tx=0).ready entity=mon.1 client_cookie=57f0c1ea9ed46e16 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:30:57.281 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:57.278+0000 7f3fc1ffb640 1 -- 172.21.15.37:0/3269201695 <== mon.1 v2:172.21.15.203:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3fbc037070 con 0x7f3fc40d5430 2024-04-25T10:30:57.281 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:57.278+0000 7f3fc1ffb640 1 -- 172.21.15.37:0/3269201695 <== mon.1 v2:172.21.15.203:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f3fbc02ae50 con 0x7f3fc40d5430 2024-04-25T10:30:57.281 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:57.278+0000 7f3fc1ffb640 1 -- 172.21.15.37:0/3269201695 <== mon.1 v2:172.21.15.203:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3fbc032640 con 0x7f3fc40d5430 2024-04-25T10:30:57.282 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:57.279+0000 7f3fc984c640 1 -- 172.21.15.37:0/3269201695 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f3fc40d5430 msgr2=0x7f3fc40b75c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:30:57.282 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:57.279+0000 7f3fc984c640 1 --2- 172.21.15.37:0/3269201695 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f3fc40d5430 0x7f3fc40b75c0 secure :-1 s=READY pgs=245 cs=0 l=1 rev1=1 crypto rx=0x7f3fbc0097a0 tx=0x7f3fbc02a100 comp rx=0 tx=0).stop 2024-04-25T10:30:57.282 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:57.279+0000 7f3fc984c640 1 -- 172.21.15.37:0/3269201695 shutdown_connections 2024-04-25T10:30:57.282 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:57.279+0000 7f3fc984c640 1 --2- 172.21.15.37:0/3269201695 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f3fc40b7b00 0x7f3fc4160a00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:30:57.282 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:57.279+0000 7f3fc984c640 1 --2- 172.21.15.37:0/3269201695 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f3fc40d5430 0x7f3fc40b75c0 unknown :-1 s=CLOSED pgs=245 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:30:57.283 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:57.279+0000 7f3fc984c640 1 -- 172.21.15.37:0/3269201695 >> 172.21.15.37:0/3269201695 conn(0x7f3fc40d27a0 msgr2=0x7f3fc40d2bb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:30:57.283 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:57.279+0000 7f3fc984c640 1 -- 172.21.15.37:0/3269201695 shutdown_connections 2024-04-25T10:30:57.283 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:57.279+0000 7f3fc984c640 1 -- 172.21.15.37:0/3269201695 wait complete. 2024-04-25T10:30:57.283 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:57.281+0000 7f3fc984c640 1 Processor -- start 2024-04-25T10:30:57.283 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:57.281+0000 7f3fc984c640 1 -- start start 2024-04-25T10:30:57.286 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:57.282+0000 7f3fc984c640 1 --2- >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f3fc40b7b00 0x7f3fc40cd740 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:30:57.286 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:57.282+0000 7f3fc984c640 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f3fc40d5430 0x7f3fc41f9300 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:30:57.286 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:57.282+0000 7f3fc984c640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f3fc4160f40 0x7f3fc41f6630 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:30:57.286 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:57.282+0000 7f3fc984c640 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 -- 0x7f3fc40d4f10 con 0x7f3fc4160f40 2024-04-25T10:30:57.286 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:57.282+0000 7f3fc984c640 1 -- --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_getmap magic: 0 -- 0x7f3fc40d4c30 con 0x7f3fc40d5430 2024-04-25T10:30:57.286 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:57.282+0000 7f3fc984c640 1 -- --> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] -- mon_getmap magic: 0 -- 0x7f3fc4161350 con 0x7f3fc40b7b00 2024-04-25T10:30:57.287 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:57.283+0000 7f3fc37fe640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f3fc4160f40 0x7f3fc41f6630 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:30:57.287 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:57.283+0000 7f3fc37fe640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f3fc4160f40 0x7f3fc41f6630 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.37:3300/0 says I am v2:172.21.15.37:59666/0 (socket says 172.21.15.37:59666) 2024-04-25T10:30:57.287 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:57.283+0000 7f3fc37fe640 1 -- 172.21.15.37:0/4254892228 learned_addr learned my addr 172.21.15.37:0/4254892228 (peer_addr_for_me v2:172.21.15.37:0/0) 2024-04-25T10:30:57.287 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:57.283+0000 7f3fc2ffd640 1 --2- 172.21.15.37:0/4254892228 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f3fc40b7b00 0x7f3fc40cd740 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:30:57.287 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:57.283+0000 7f3fc27fc640 1 --2- 172.21.15.37:0/4254892228 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f3fc40d5430 0x7f3fc41f9300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:30:57.287 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:57.283+0000 7f3fc37fe640 1 -- 172.21.15.37:0/4254892228 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f3fc40b7b00 msgr2=0x7f3fc40cd740 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:30:57.287 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:57.283+0000 7f3fc37fe640 1 --2- 172.21.15.37:0/4254892228 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f3fc40b7b00 0x7f3fc40cd740 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:30:57.287 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:57.283+0000 7f3fc37fe640 1 -- 172.21.15.37:0/4254892228 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f3fc40d5430 msgr2=0x7f3fc41f9300 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:30:57.287 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:57.283+0000 7f3fc37fe640 1 --2- 172.21.15.37:0/4254892228 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f3fc40d5430 0x7f3fc41f9300 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:30:57.287 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:57.283+0000 7f3fc37fe640 1 -- 172.21.15.37:0/4254892228 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f3fc41f9840 con 0x7f3fc4160f40 2024-04-25T10:30:57.287 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:57.283+0000 7f3fc37fe640 1 --2- 172.21.15.37:0/4254892228 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f3fc4160f40 0x7f3fc41f6630 secure :-1 s=READY pgs=670 cs=0 l=1 rev1=1 crypto rx=0x7f3fb400c7e0 tx=0x7f3fb400cca0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:30:57.287 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:57.283+0000 7f3fbbfff640 1 -- 172.21.15.37:0/4254892228 <== mon.0 v2:172.21.15.37:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3fb4016020 con 0x7f3fc4160f40 2024-04-25T10:30:57.287 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:57.283+0000 7f3fbbfff640 1 -- 172.21.15.37:0/4254892228 <== mon.0 v2:172.21.15.37:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f3fb4006d50 con 0x7f3fc4160f40 2024-04-25T10:30:57.287 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:57.283+0000 7f3fc984c640 1 -- 172.21.15.37:0/4254892228 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f3fc40ce100 con 0x7f3fc4160f40 2024-04-25T10:30:57.288 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:57.283+0000 7f3fbbfff640 1 -- 172.21.15.37:0/4254892228 <== mon.0 v2:172.21.15.37:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3fb4003bb0 con 0x7f3fc4160f40 2024-04-25T10:30:57.288 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:57.283+0000 7f3fc984c640 1 -- 172.21.15.37:0/4254892228 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f3fc40ce6c0 con 0x7f3fc4160f40 2024-04-25T10:30:57.289 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:57.284+0000 7f3fc984c640 1 -- 172.21.15.37:0/4254892228 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f3fc406a140 con 0x7f3fc4160f40 2024-04-25T10:30:57.289 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:57.285+0000 7f3fbbfff640 1 -- 172.21.15.37:0/4254892228 <== mon.0 v2:172.21.15.37:3300/0 4 ==== mgrmap(e 5) ==== 96782+0+0 (secure 0 0 0) 0x7f3fb400a850 con 0x7f3fc4160f40 2024-04-25T10:30:57.289 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:57.285+0000 7f3fbbfff640 1 --2- 172.21.15.37:0/4254892228 >> [v2:172.21.15.203:6832/2418205237,v1:172.21.15.203:6833/2418205237] conn(0x7f3fa40744b0 0x7f3fa40769c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:30:57.289 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:57.286+0000 7f3fbbfff640 1 -- 172.21.15.37:0/4254892228 <== mon.0 v2:172.21.15.37:3300/0 5 ==== osd_map(47..47 src has 1..47) ==== 5926+0+0 (secure 0 0 0) 0x7f3fb4094e30 con 0x7f3fc4160f40 2024-04-25T10:30:57.289 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:57.286+0000 7f3fc2ffd640 1 --2- 172.21.15.37:0/4254892228 >> [v2:172.21.15.203:6832/2418205237,v1:172.21.15.203:6833/2418205237] conn(0x7f3fa40744b0 0x7f3fa40769c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:30:57.289 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:57.286+0000 7f3fc2ffd640 1 --2- 172.21.15.37:0/4254892228 >> [v2:172.21.15.203:6832/2418205237,v1:172.21.15.203:6833/2418205237] conn(0x7f3fa40744b0 0x7f3fa40769c0 secure :-1 s=READY pgs=303 cs=0 l=1 rev1=1 crypto rx=0x7f3fbc034000 tx=0x7f3fbc02e000 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:30:57.292 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:57.290+0000 7f3fbbfff640 1 -- 172.21.15.37:0/4254892228 <== mon.0 v2:172.21.15.37:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187246 (secure 0 0 0) 0x7f3fb4020020 con 0x7f3fc4160f40 2024-04-25T10:30:57.474 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:57.472+0000 7f3fc984c640 1 -- 172.21.15.37:0/4254892228 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "recovery_fs_metadata", "var": "pg_num"} v 0) -- 0x7f3fc40d3f20 con 0x7f3fc4160f40 2024-04-25T10:30:57.475 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:57.473+0000 7f3fbbfff640 1 -- 172.21.15.37:0/4254892228 <== mon.0 v2:172.21.15.37:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "recovery_fs_metadata", "var": "pg_num"}]=0 v47) ==== 109+0+11 (secure 0 0 0) 0x7f3fc40b7f10 con 0x7f3fc4160f40 2024-04-25T10:30:57.475 INFO:teuthology.orchestra.run.smithi037.stdout:pg_num: 32 2024-04-25T10:30:57.477 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:57.475+0000 7f3fc984c640 1 -- 172.21.15.37:0/4254892228 >> [v2:172.21.15.203:6832/2418205237,v1:172.21.15.203:6833/2418205237] conn(0x7f3fa40744b0 msgr2=0x7f3fa40769c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:30:57.477 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:57.475+0000 7f3fc984c640 1 --2- 172.21.15.37:0/4254892228 >> [v2:172.21.15.203:6832/2418205237,v1:172.21.15.203:6833/2418205237] conn(0x7f3fa40744b0 0x7f3fa40769c0 secure :-1 s=READY pgs=303 cs=0 l=1 rev1=1 crypto rx=0x7f3fbc034000 tx=0x7f3fbc02e000 comp rx=0 tx=0).stop 2024-04-25T10:30:57.477 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:57.475+0000 7f3fc984c640 1 -- 172.21.15.37:0/4254892228 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f3fc4160f40 msgr2=0x7f3fc41f6630 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:30:57.477 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:57.475+0000 7f3fc984c640 1 --2- 172.21.15.37:0/4254892228 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f3fc4160f40 0x7f3fc41f6630 secure :-1 s=READY pgs=670 cs=0 l=1 rev1=1 crypto rx=0x7f3fb400c7e0 tx=0x7f3fb400cca0 comp rx=0 tx=0).stop 2024-04-25T10:30:57.478 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:57.475+0000 7f3fc984c640 1 -- 172.21.15.37:0/4254892228 shutdown_connections 2024-04-25T10:30:57.478 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:57.475+0000 7f3fc984c640 1 --2- 172.21.15.37:0/4254892228 >> [v2:172.21.15.203:6832/2418205237,v1:172.21.15.203:6833/2418205237] conn(0x7f3fa40744b0 0x7f3fa40769c0 unknown :-1 s=CLOSED pgs=303 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:30:57.478 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:57.475+0000 7f3fc984c640 1 --2- 172.21.15.37:0/4254892228 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f3fc4160f40 0x7f3fc41f6630 unknown :-1 s=CLOSED pgs=670 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:30:57.478 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:57.475+0000 7f3fc984c640 1 --2- 172.21.15.37:0/4254892228 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f3fc40d5430 0x7f3fc41f9300 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:30:57.478 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:57.475+0000 7f3fc984c640 1 --2- 172.21.15.37:0/4254892228 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f3fc40b7b00 0x7f3fc40cd740 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:30:57.478 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:57.475+0000 7f3fc984c640 1 -- 172.21.15.37:0/4254892228 >> 172.21.15.37:0/4254892228 conn(0x7f3fc40d27a0 msgr2=0x7f3fc40bb240 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:30:57.478 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:57.475+0000 7f3fc984c640 1 -- 172.21.15.37:0/4254892228 shutdown_connections 2024-04-25T10:30:57.478 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:57.475+0000 7f3fc984c640 1 -- 172.21.15.37:0/4254892228 wait complete. 2024-04-25T10:30:57.488 DEBUG:teuthology.orchestra.run.smithi037:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-04-25T10:30:57.600 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:57.598+0000 7f309ae92640 1 Processor -- start 2024-04-25T10:30:57.600 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:57.598+0000 7f309ae92640 1 -- start start 2024-04-25T10:30:57.603 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:57.599+0000 7f309ae92640 1 --2- >> v2:172.21.15.203:3300/0 conn(0x7f30940b4bb0 0x7f30941646b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:30:57.603 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:57.599+0000 7f309ae92640 1 --2- >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f30940d4560 0x7f30940d4970 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:30:57.604 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:57.599+0000 7f309ae92640 1 -- --> v1:172.21.15.37:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f3094161190 con 0x7f30940b7600 2024-04-25T10:30:57.604 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:57.599+0000 7f309ae92640 1 -- --> v2:172.21.15.203:3300/0 -- mon_getmap magic: 0 -- 0x7f3094161470 con 0x7f30940b4bb0 2024-04-25T10:30:57.604 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:57.599+0000 7f309ae92640 1 -- --> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] -- mon_getmap magic: 0 -- 0x7f3094163eb0 con 0x7f30940d4560 2024-04-25T10:30:57.604 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:57.599+0000 7f3093fff640 1 --1- >> v1:172.21.15.37:6789/0 conn(0x7f30940b7600 0x7f30940b7a10 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.37:6789/0 says I am v1:172.21.15.37:38774/0 (socket says 172.21.15.37:38774) 2024-04-25T10:30:57.604 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:57.599+0000 7f3093fff640 1 -- 172.21.15.37:0/2460259476 learned_addr learned my addr 172.21.15.37:0/2460259476 (peer_addr_for_me v1:172.21.15.37:0/0) 2024-04-25T10:30:57.604 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:57.599+0000 7f309942f640 1 --2- 172.21.15.37:0/2460259476 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f30940d4560 0x7f30940d4970 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:30:57.604 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:57.599+0000 7f3098c2e640 1 --2- 172.21.15.37:0/2460259476 >> v2:172.21.15.203:3300/0 conn(0x7f30940b4bb0 0x7f30941646b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:30:57.604 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:57.599+0000 7f30937fe640 1 -- 172.21.15.37:0/2460259476 <== mon.0 v1:172.21.15.37:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 2820560977 0 0) 0x7f3094161190 con 0x7f30940b7600 2024-04-25T10:30:57.604 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:57.600+0000 7f30937fe640 1 -- 172.21.15.37:0/2460259476 --> v1:172.21.15.37:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f307c003520 con 0x7f30940b7600 2024-04-25T10:30:57.604 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:57.600+0000 7f30937fe640 1 -- 172.21.15.37:0/2460259476 <== mon.0 v1:172.21.15.37:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 2299750713 0 0) 0x7f307c003520 con 0x7f30940b7600 2024-04-25T10:30:57.604 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:57.600+0000 7f30937fe640 1 -- 172.21.15.37:0/2460259476 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f30940d4560 msgr2=0x7f30940d4970 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:30:57.604 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:57.600+0000 7f30937fe640 1 --2- 172.21.15.37:0/2460259476 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f30940d4560 0x7f30940d4970 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:30:57.604 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:57.600+0000 7f30937fe640 1 -- 172.21.15.37:0/2460259476 >> v2:172.21.15.203:3300/0 conn(0x7f30940b4bb0 msgr2=0x7f30941646b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:30:57.605 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:57.600+0000 7f30937fe640 1 --2- 172.21.15.37:0/2460259476 >> v2:172.21.15.203:3300/0 conn(0x7f30940b4bb0 0x7f30941646b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:30:57.605 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:57.600+0000 7f30937fe640 1 -- 172.21.15.37:0/2460259476 --> v1:172.21.15.37:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f3094164190 con 0x7f30940b7600 2024-04-25T10:30:57.605 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:57.600+0000 7f30937fe640 1 -- 172.21.15.37:0/2460259476 <== mon.0 v1:172.21.15.37:6789/0 3 ==== mon_map magic: 0 ==== 413+0+0 (unknown 1577794600 0 0) 0x7f3088001670 con 0x7f30940b7600 2024-04-25T10:30:57.605 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:57.600+0000 7f30937fe640 1 -- 172.21.15.37:0/2460259476 >> v1:172.21.15.37:6789/0 conn(0x7f30940b7600 legacy=0x7f30940b7a10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:30:57.605 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:57.600+0000 7f30937fe640 1 --2- 172.21.15.37:0/2460259476 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f307c0039b0 0x7f307c003e20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:30:57.605 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:57.600+0000 7f30937fe640 1 --2- 172.21.15.37:0/2460259476 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f307c004800 0x7f307c004c10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:30:57.605 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:57.600+0000 7f30937fe640 1 --2- 172.21.15.37:0/2460259476 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f30940d4560 0x7f307c005150 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:30:57.605 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:57.600+0000 7f30937fe640 1 -- 172.21.15.37:0/2460259476 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 -- 0x7f3094161470 con 0x7f307c004800 2024-04-25T10:30:57.605 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:57.600+0000 7f30937fe640 1 -- 172.21.15.37:0/2460259476 --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_getmap magic: 0 -- 0x7f3094163eb0 con 0x7f307c0039b0 2024-04-25T10:30:57.605 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:57.600+0000 7f30937fe640 1 -- 172.21.15.37:0/2460259476 --> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] -- mon_getmap magic: 0 -- 0x7f307c00b9c0 con 0x7f30940d4560 2024-04-25T10:30:57.605 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:57.600+0000 7f3093fff640 1 --2- 172.21.15.37:0/2460259476 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f307c004800 0x7f307c004c10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:30:57.605 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:57.600+0000 7f3098c2e640 1 --2- 172.21.15.37:0/2460259476 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f307c0039b0 0x7f307c003e20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:30:57.605 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:57.600+0000 7f309942f640 1 --2- 172.21.15.37:0/2460259476 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f30940d4560 0x7f307c005150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:30:57.605 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:57.600+0000 7f3093fff640 1 -- 172.21.15.37:0/2460259476 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f30940d4560 msgr2=0x7f307c005150 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:30:57.605 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:57.600+0000 7f3093fff640 1 --2- 172.21.15.37:0/2460259476 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f30940d4560 0x7f307c005150 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:30:57.605 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:57.600+0000 7f3093fff640 1 -- 172.21.15.37:0/2460259476 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f307c0039b0 msgr2=0x7f307c003e20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:30:57.606 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:57.600+0000 7f3093fff640 1 --2- 172.21.15.37:0/2460259476 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f307c0039b0 0x7f307c003e20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:30:57.606 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:57.600+0000 7f3093fff640 1 -- 172.21.15.37:0/2460259476 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f307c003520 con 0x7f307c004800 2024-04-25T10:30:57.606 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:57.600+0000 7f309942f640 1 --2- 172.21.15.37:0/2460259476 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f30940d4560 0x7f307c005150 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T10:30:57.606 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:57.601+0000 7f3093fff640 1 --2- 172.21.15.37:0/2460259476 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f307c004800 0x7f307c004c10 secure :-1 s=READY pgs=672 cs=0 l=1 rev1=1 crypto rx=0x7f3088007fd0 tx=0x7f307c005830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=4bc033d6c01713cd server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:30:57.606 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:57.601+0000 7f30937fe640 1 -- 172.21.15.37:0/2460259476 <== mon.0 v2:172.21.15.37:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3088007d50 con 0x7f307c004800 2024-04-25T10:30:57.606 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:57.601+0000 7f30937fe640 1 -- 172.21.15.37:0/2460259476 <== mon.0 v2:172.21.15.37:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f3088005ad0 con 0x7f307c004800 2024-04-25T10:30:57.606 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:57.601+0000 7f30937fe640 1 -- 172.21.15.37:0/2460259476 <== mon.0 v2:172.21.15.37:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3088007070 con 0x7f307c004800 2024-04-25T10:30:57.609 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:57.602+0000 7f309ae92640 1 -- 172.21.15.37:0/2460259476 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f307c004800 msgr2=0x7f307c004c10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:30:57.609 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:57.602+0000 7f309ae92640 1 --2- 172.21.15.37:0/2460259476 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f307c004800 0x7f307c004c10 secure :-1 s=READY pgs=672 cs=0 l=1 rev1=1 crypto rx=0x7f3088007fd0 tx=0x7f307c005830 comp rx=0 tx=0).stop 2024-04-25T10:30:57.609 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:57.602+0000 7f309942f640 1 -- 172.21.15.37:0/2460259476 reap_dead start 2024-04-25T10:30:57.609 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:57.602+0000 7f309ae92640 1 -- 172.21.15.37:0/2460259476 shutdown_connections 2024-04-25T10:30:57.609 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:57.602+0000 7f309ae92640 1 -- 172.21.15.37:0/2460259476 >> 172.21.15.37:0/2460259476 conn(0x7f30940d27e0 msgr2=0x7f30940d2bf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:30:57.610 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:57.602+0000 7f309ae92640 1 -- 172.21.15.37:0/2460259476 shutdown_connections 2024-04-25T10:30:57.610 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:57.602+0000 7f309ae92640 1 -- 172.21.15.37:0/2460259476 wait complete. 2024-04-25T10:30:57.610 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:57.604+0000 7f309ae92640 1 Processor -- start 2024-04-25T10:30:57.610 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:57.604+0000 7f309ae92640 1 -- start start 2024-04-25T10:30:57.610 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:57.605+0000 7f309ae92640 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f307c004800 0x7f30940ca3b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:30:57.610 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:57.605+0000 7f309ae92640 1 --2- >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f3094158f80 0x7f309415c630 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:30:57.610 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:57.605+0000 7f309ae92640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f309414a610 0x7f309414aa20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:30:57.610 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:57.605+0000 7f309ae92640 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 -- 0x7f30940d09e0 con 0x7f309414a610 2024-04-25T10:30:57.610 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:57.605+0000 7f309ae92640 1 -- --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_getmap magic: 0 -- 0x7f3094164bf0 con 0x7f307c004800 2024-04-25T10:30:57.610 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:57.605+0000 7f309ae92640 1 -- --> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] -- mon_getmap magic: 0 -- 0x7f30940d0b60 con 0x7f3094158f80 2024-04-25T10:30:57.610 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:57.605+0000 7f309942f640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f309414a610 0x7f309414aa20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:30:57.610 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:57.605+0000 7f309942f640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f309414a610 0x7f309414aa20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.37:3300/0 says I am v2:172.21.15.37:59686/0 (socket says 172.21.15.37:59686) 2024-04-25T10:30:57.610 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:57.605+0000 7f309942f640 1 -- 172.21.15.37:0/1101082921 learned_addr learned my addr 172.21.15.37:0/1101082921 (peer_addr_for_me v2:172.21.15.37:0/0) 2024-04-25T10:30:57.610 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:57.606+0000 7f3098c2e640 1 --2- 172.21.15.37:0/1101082921 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f307c004800 0x7f30940ca3b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:30:57.610 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:57.606+0000 7f3093fff640 1 --2- 172.21.15.37:0/1101082921 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f3094158f80 0x7f309415c630 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:30:57.611 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:57.606+0000 7f309942f640 1 -- 172.21.15.37:0/1101082921 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f3094158f80 msgr2=0x7f309415c630 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:30:57.611 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:57.606+0000 7f309942f640 1 --2- 172.21.15.37:0/1101082921 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f3094158f80 0x7f309415c630 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:30:57.611 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:57.606+0000 7f309942f640 1 -- 172.21.15.37:0/1101082921 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f307c004800 msgr2=0x7f30940ca3b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:30:57.611 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:57.606+0000 7f309942f640 1 --2- 172.21.15.37:0/1101082921 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f307c004800 0x7f30940ca3b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:30:57.611 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:57.606+0000 7f309942f640 1 -- 172.21.15.37:0/1101082921 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f3094149570 con 0x7f309414a610 2024-04-25T10:30:57.611 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:57.606+0000 7f309942f640 1 --2- 172.21.15.37:0/1101082921 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f309414a610 0x7f309414aa20 secure :-1 s=READY pgs=673 cs=0 l=1 rev1=1 crypto rx=0x7f30940d2050 tx=0x7f3084011920 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:30:57.611 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:57.606+0000 7f30917fa640 1 -- 172.21.15.37:0/1101082921 <== mon.0 v2:172.21.15.37:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3084003610 con 0x7f309414a610 2024-04-25T10:30:57.611 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:57.606+0000 7f30917fa640 1 -- 172.21.15.37:0/1101082921 <== mon.0 v2:172.21.15.37:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f308401b070 con 0x7f309414a610 2024-04-25T10:30:57.611 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:57.606+0000 7f309ae92640 1 -- 172.21.15.37:0/1101082921 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f3094149850 con 0x7f309414a610 2024-04-25T10:30:57.611 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:57.606+0000 7f30917fa640 1 -- 172.21.15.37:0/1101082921 <== mon.0 v2:172.21.15.37:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3084003a10 con 0x7f309414a610 2024-04-25T10:30:57.611 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:57.606+0000 7f309ae92640 1 -- 172.21.15.37:0/1101082921 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f3094149e10 con 0x7f309414a610 2024-04-25T10:30:57.611 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:57.607+0000 7f309ae92640 1 -- 172.21.15.37:0/1101082921 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f30941492e0 con 0x7f309414a610 2024-04-25T10:30:57.612 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:57.608+0000 7f30917fa640 1 -- 172.21.15.37:0/1101082921 <== mon.0 v2:172.21.15.37:3300/0 4 ==== mgrmap(e 5) ==== 96782+0+0 (secure 0 0 0) 0x7f3084003bb0 con 0x7f309414a610 2024-04-25T10:30:57.612 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:57.608+0000 7f30917fa640 1 --2- 172.21.15.37:0/1101082921 >> [v2:172.21.15.203:6832/2418205237,v1:172.21.15.203:6833/2418205237] conn(0x7f30800744b0 0x7f30800769c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:30:57.612 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:57.608+0000 7f30917fa640 1 -- 172.21.15.37:0/1101082921 <== mon.0 v2:172.21.15.37:3300/0 5 ==== osd_map(47..47 src has 1..47) ==== 5926+0+0 (secure 0 0 0) 0x7f3084095390 con 0x7f309414a610 2024-04-25T10:30:57.612 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:57.609+0000 7f3098c2e640 1 --2- 172.21.15.37:0/1101082921 >> [v2:172.21.15.203:6832/2418205237,v1:172.21.15.203:6833/2418205237] conn(0x7f30800744b0 0x7f30800769c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:30:57.612 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:57.609+0000 7f3098c2e640 1 --2- 172.21.15.37:0/1101082921 >> [v2:172.21.15.203:6832/2418205237,v1:172.21.15.203:6833/2418205237] conn(0x7f30800744b0 0x7f30800769c0 secure :-1 s=READY pgs=304 cs=0 l=1 rev1=1 crypto rx=0x7f30940d6c00 tx=0x7f308c00b040 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:30:57.615 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:57.613+0000 7f30917fa640 1 -- 172.21.15.37:0/1101082921 <== mon.0 v2:172.21.15.37:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187246 (secure 0 0 0) 0x7f3084025020 con 0x7f309414a610 2024-04-25T10:30:57.838 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:57.836+0000 7f309ae92640 1 -- 172.21.15.37:0/1101082921 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f309406a140 con 0x7f309414a610 2024-04-25T10:30:57.839 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:57.837+0000 7f30917fa640 1 -- 172.21.15.37:0/1101082921 <== mon.0 v2:172.21.15.37:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 29 v29) ==== 94+0+5190 (secure 0 0 0) 0x7f308405fad0 con 0x7f309414a610 2024-04-25T10:30:57.840 INFO:teuthology.orchestra.run.smithi037.stdout: 2024-04-25T10:30:57.840 INFO:teuthology.orchestra.run.smithi037.stdout:{"epoch":29,"default_fscid":-1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":4946,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.37:6835/683655741","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.37:6834","nonce":683655741},{"type":"v1","addr":"172.21.15.37:6835","nonce":683655741}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":19},{"gid":5215,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.203:6836/3927045001","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6835","nonce":3927045001},{"type":"v1","addr":"172.21.15.203:6836","nonce":3927045001}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":19},{"gid":5218,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.203:6837/3108183946","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6834","nonce":3108183946},{"type":"v1","addr":"172.21.15.203:6837","nonce":3108183946}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":19}],"filesystems":[{"mdsmap":{"epoch":27,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-04-25T10:30:28.588617+0000","modified":"2024-04-25T10:30:42.935167+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":1,"in":[0],"up":{"mds_0":5625},"failed":[],"damaged":[],"stopped":[],"info":{"gid_5625":{"gid":5625,"name":"c","rank":0,"incarnation":24,"state":"up:active","state_seq":7,"addr":"172.21.15.37:6833/2353203722","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.37:6832","nonce":2353203722},{"type":"v1","addr":"172.21.15.37:6833","nonce":2353203722}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[5],"metadata_pool":6,"enabled":true,"fs_name":"recovery_fs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":5625,"qdb_cluster":[5625]},"id":3}]} 2024-04-25T10:30:57.840 INFO:teuthology.orchestra.run.smithi037.stderr:dumped fsmap epoch 29 2024-04-25T10:30:57.842 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:57.839+0000 7f309ae92640 1 -- 172.21.15.37:0/1101082921 >> [v2:172.21.15.203:6832/2418205237,v1:172.21.15.203:6833/2418205237] conn(0x7f30800744b0 msgr2=0x7f30800769c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:30:57.842 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:57.839+0000 7f309ae92640 1 --2- 172.21.15.37:0/1101082921 >> [v2:172.21.15.203:6832/2418205237,v1:172.21.15.203:6833/2418205237] conn(0x7f30800744b0 0x7f30800769c0 secure :-1 s=READY pgs=304 cs=0 l=1 rev1=1 crypto rx=0x7f30940d6c00 tx=0x7f308c00b040 comp rx=0 tx=0).stop 2024-04-25T10:30:57.842 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:57.839+0000 7f309ae92640 1 -- 172.21.15.37:0/1101082921 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f309414a610 msgr2=0x7f309414aa20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:30:57.842 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:57.839+0000 7f309ae92640 1 --2- 172.21.15.37:0/1101082921 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f309414a610 0x7f309414aa20 secure :-1 s=READY pgs=673 cs=0 l=1 rev1=1 crypto rx=0x7f30940d2050 tx=0x7f3084011920 comp rx=0 tx=0).stop 2024-04-25T10:30:57.842 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:57.840+0000 7f309ae92640 1 -- 172.21.15.37:0/1101082921 shutdown_connections 2024-04-25T10:30:57.842 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:57.840+0000 7f309ae92640 1 --2- 172.21.15.37:0/1101082921 >> [v2:172.21.15.203:6832/2418205237,v1:172.21.15.203:6833/2418205237] conn(0x7f30800744b0 0x7f30800769c0 unknown :-1 s=CLOSED pgs=304 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:30:57.842 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:57.840+0000 7f309ae92640 1 --2- 172.21.15.37:0/1101082921 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f309414a610 0x7f309414aa20 unknown :-1 s=CLOSED pgs=673 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:30:57.842 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:57.840+0000 7f309ae92640 1 --2- 172.21.15.37:0/1101082921 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f3094158f80 0x7f309415c630 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:30:57.842 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:57.840+0000 7f309ae92640 1 --2- 172.21.15.37:0/1101082921 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f307c004800 0x7f30940ca3b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:30:57.843 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:57.840+0000 7f309ae92640 1 -- 172.21.15.37:0/1101082921 >> 172.21.15.37:0/1101082921 conn(0x7f30940d27e0 msgr2=0x7f3094162920 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:30:57.843 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:57.840+0000 7f309ae92640 1 -- 172.21.15.37:0/1101082921 shutdown_connections 2024-04-25T10:30:57.843 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:57.840+0000 7f309ae92640 1 -- 172.21.15.37:0/1101082921 wait complete. 2024-04-25T10:30:57.853 DEBUG:teuthology.orchestra.run.smithi037:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-04-25T10:30:57.964 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:57.962+0000 7fe6a3825640 1 Processor -- start 2024-04-25T10:30:57.964 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:57.962+0000 7fe6a3825640 1 -- start start 2024-04-25T10:30:57.968 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:57.963+0000 7fe6a3825640 1 --2- >> v2:172.21.15.203:3300/0 conn(0x7fe69c0b7600 0x7fe69c0b7a10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:30:57.968 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:57.963+0000 7fe6a3825640 1 -- --> v1:172.21.15.37:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fe69c0b7f50 con 0x7fe69c0d4560 2024-04-25T10:30:57.968 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:57.963+0000 7fe6a3825640 1 -- --> v1:172.21.15.203:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fe69c0d5080 con 0x7fe69c0b4bb0 2024-04-25T10:30:57.968 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:57.963+0000 7fe6a3825640 1 -- --> v2:172.21.15.203:3300/0 -- mon_getmap magic: 0 -- 0x7fe69c161190 con 0x7fe69c0b7600 2024-04-25T10:30:57.968 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:57.963+0000 7fe6a1dc2640 1 --1- >> v1:172.21.15.37:6789/0 conn(0x7fe69c0d4560 0x7fe69c0d4970 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.37:6789/0 says I am v1:172.21.15.37:38778/0 (socket says 172.21.15.37:38778) 2024-04-25T10:30:57.968 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:57.963+0000 7fe6a1dc2640 1 -- 172.21.15.37:0/2021620146 learned_addr learned my addr 172.21.15.37:0/2021620146 (peer_addr_for_me v1:172.21.15.37:0/0) 2024-04-25T10:30:57.968 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:57.964+0000 7fe6a0dc0640 1 --2- 172.21.15.37:0/2021620146 >> v2:172.21.15.203:3300/0 conn(0x7fe69c0b7600 0x7fe69c0b7a10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:30:57.968 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:57.964+0000 7fe693fff640 1 -- 172.21.15.37:0/2021620146 <== mon.0 v1:172.21.15.37:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 3790642962 0 0) 0x7fe694001490 con 0x7fe69c0d4560 2024-04-25T10:30:57.968 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:57.964+0000 7fe693fff640 1 -- 172.21.15.37:0/2021620146 --> v1:172.21.15.37:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7fe688003520 con 0x7fe69c0d4560 2024-04-25T10:30:57.968 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:57.964+0000 7fe693fff640 1 -- 172.21.15.37:0/2021620146 <== mon.0 v1:172.21.15.37:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 4275203357 0 0) 0x7fe688003520 con 0x7fe69c0d4560 2024-04-25T10:30:57.968 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:57.964+0000 7fe693fff640 1 -- 172.21.15.37:0/2021620146 >> v2:172.21.15.203:3300/0 conn(0x7fe69c0b7600 msgr2=0x7fe69c0b7a10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:30:57.968 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:57.964+0000 7fe693fff640 1 --2- 172.21.15.37:0/2021620146 >> v2:172.21.15.203:3300/0 conn(0x7fe69c0b7600 0x7fe69c0b7a10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:30:57.968 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:57.964+0000 7fe693fff640 1 -- 172.21.15.37:0/2021620146 >> v1:172.21.15.203:6789/0 conn(0x7fe69c0b4bb0 legacy=0x7fe69c1646b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:30:57.969 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:57.964+0000 7fe693fff640 1 -- 172.21.15.37:0/2021620146 --> v1:172.21.15.37:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fe69c161470 con 0x7fe69c0d4560 2024-04-25T10:30:57.969 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:57.964+0000 7fe693fff640 1 -- 172.21.15.37:0/2021620146 <== mon.0 v1:172.21.15.37:6789/0 3 ==== mon_map magic: 0 ==== 413+0+0 (unknown 1577794600 0 0) 0x7fe694001bc0 con 0x7fe69c0d4560 2024-04-25T10:30:57.969 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:57.964+0000 7fe693fff640 1 -- 172.21.15.37:0/2021620146 >> v1:172.21.15.37:6789/0 conn(0x7fe69c0d4560 legacy=0x7fe69c0d4970 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:30:57.969 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:57.964+0000 7fe693fff640 1 --2- 172.21.15.37:0/2021620146 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7fe6880039b0 0x7fe688003e20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:30:57.969 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:57.964+0000 7fe693fff640 1 --2- 172.21.15.37:0/2021620146 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fe688004800 0x7fe688004c10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:30:57.969 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:57.964+0000 7fe693fff640 1 --2- 172.21.15.37:0/2021620146 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fe6880051c0 0x7fe68800b9c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:30:57.969 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:57.964+0000 7fe693fff640 1 -- 172.21.15.37:0/2021620146 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 -- 0x7fe69c161190 con 0x7fe6880051c0 2024-04-25T10:30:57.969 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:57.964+0000 7fe693fff640 1 -- 172.21.15.37:0/2021620146 --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_getmap magic: 0 -- 0x7fe68800c160 con 0x7fe688004800 2024-04-25T10:30:57.969 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:57.964+0000 7fe693fff640 1 -- 172.21.15.37:0/2021620146 --> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] -- mon_getmap magic: 0 -- 0x7fe68800c360 con 0x7fe6880039b0 2024-04-25T10:30:57.969 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:57.964+0000 7fe6a1dc2640 1 --2- 172.21.15.37:0/2021620146 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fe6880051c0 0x7fe68800b9c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:30:57.969 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:57.964+0000 7fe6a0dc0640 1 --2- 172.21.15.37:0/2021620146 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fe688004800 0x7fe688004c10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:30:57.969 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:57.964+0000 7fe6a15c1640 1 --2- 172.21.15.37:0/2021620146 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7fe6880039b0 0x7fe688003e20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:30:57.969 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:57.964+0000 7fe6a1dc2640 1 -- 172.21.15.37:0/2021620146 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7fe6880039b0 msgr2=0x7fe688003e20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:30:57.969 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:57.964+0000 7fe6a1dc2640 1 --2- 172.21.15.37:0/2021620146 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7fe6880039b0 0x7fe688003e20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:30:57.970 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:57.964+0000 7fe6a1dc2640 1 -- 172.21.15.37:0/2021620146 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fe688004800 msgr2=0x7fe688004c10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:30:57.970 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:57.964+0000 7fe6a1dc2640 1 --2- 172.21.15.37:0/2021620146 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fe688004800 0x7fe688004c10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:30:57.970 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:57.965+0000 7fe6a1dc2640 1 -- 172.21.15.37:0/2021620146 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fe688003520 con 0x7fe6880051c0 2024-04-25T10:30:57.970 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:57.965+0000 7fe6a1dc2640 1 -- 172.21.15.37:0/2021620146 reap_dead start 2024-04-25T10:30:57.970 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:57.965+0000 7fe6a1dc2640 1 --2- 172.21.15.37:0/2021620146 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fe6880051c0 0x7fe68800b9c0 secure :-1 s=READY pgs=675 cs=0 l=1 rev1=1 crypto rx=0x7fe69400aa00 tx=0x7fe688005830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=38be0519a38d159 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:30:57.970 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:57.965+0000 7fe693fff640 1 -- 172.21.15.37:0/2021620146 <== mon.0 v2:172.21.15.37:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe69400a650 con 0x7fe6880051c0 2024-04-25T10:30:57.970 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:57.965+0000 7fe693fff640 1 -- 172.21.15.37:0/2021620146 <== mon.0 v2:172.21.15.37:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fe69402f070 con 0x7fe6880051c0 2024-04-25T10:30:57.970 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:57.965+0000 7fe693fff640 1 -- 172.21.15.37:0/2021620146 <== mon.0 v2:172.21.15.37:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe69402bdb0 con 0x7fe6880051c0 2024-04-25T10:30:57.973 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:57.966+0000 7fe6a3825640 1 -- 172.21.15.37:0/2021620146 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fe6880051c0 msgr2=0x7fe68800b9c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:30:57.973 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:57.966+0000 7fe6a3825640 1 --2- 172.21.15.37:0/2021620146 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fe6880051c0 0x7fe68800b9c0 secure :-1 s=READY pgs=675 cs=0 l=1 rev1=1 crypto rx=0x7fe69400aa00 tx=0x7fe688005830 comp rx=0 tx=0).stop 2024-04-25T10:30:57.973 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:57.966+0000 7fe6a3825640 1 -- 172.21.15.37:0/2021620146 shutdown_connections 2024-04-25T10:30:57.974 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:57.966+0000 7fe6a3825640 1 --2- 172.21.15.37:0/2021620146 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fe6880051c0 0x7fe68800b9c0 unknown :-1 s=CLOSED pgs=675 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:30:57.974 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:57.966+0000 7fe6a3825640 1 -- 172.21.15.37:0/2021620146 >> 172.21.15.37:0/2021620146 conn(0x7fe69c0d27e0 msgr2=0x7fe69c0d2bf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:30:57.974 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:57.966+0000 7fe6a3825640 1 -- 172.21.15.37:0/2021620146 shutdown_connections 2024-04-25T10:30:57.974 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:57.966+0000 7fe6a3825640 1 -- 172.21.15.37:0/2021620146 wait complete. 2024-04-25T10:30:57.974 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:57.968+0000 7fe6a3825640 1 Processor -- start 2024-04-25T10:30:57.974 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:57.968+0000 7fe6a3825640 1 -- start start 2024-04-25T10:30:57.974 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:57.969+0000 7fe6a3825640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fe6880051c0 0x7fe69c1f6400 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:30:57.974 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:57.969+0000 7fe6a3825640 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fe69c15b6c0 0x7fe69c15c380 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:30:57.974 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:57.969+0000 7fe6a3825640 1 --2- >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7fe69c15c8c0 0x7fe69c0ba0e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:30:57.974 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:57.969+0000 7fe6a3825640 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 -- 0x7fe69c0b4fc0 con 0x7fe6880051c0 2024-04-25T10:30:57.974 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:57.969+0000 7fe6a3825640 1 -- --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_getmap magic: 0 -- 0x7fe69c164190 con 0x7fe69c15b6c0 2024-04-25T10:30:57.974 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:57.969+0000 7fe6a3825640 1 -- --> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] -- mon_getmap magic: 0 -- 0x7fe69c0d09e0 con 0x7fe69c15c8c0 2024-04-25T10:30:57.974 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:57.969+0000 7fe6a15c1640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fe6880051c0 0x7fe69c1f6400 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:30:57.975 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:57.969+0000 7fe6a15c1640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fe6880051c0 0x7fe69c1f6400 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.37:3300/0 says I am v2:172.21.15.37:59698/0 (socket says 172.21.15.37:59698) 2024-04-25T10:30:57.975 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:57.969+0000 7fe6a15c1640 1 -- 172.21.15.37:0/2712463981 learned_addr learned my addr 172.21.15.37:0/2712463981 (peer_addr_for_me v2:172.21.15.37:0/0) 2024-04-25T10:30:57.975 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:57.970+0000 7fe6a1dc2640 1 --2- 172.21.15.37:0/2712463981 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7fe69c15c8c0 0x7fe69c0ba0e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:30:57.975 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:57.970+0000 7fe6a0dc0640 1 --2- 172.21.15.37:0/2712463981 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fe69c15b6c0 0x7fe69c15c380 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:30:57.975 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:57.970+0000 7fe6a15c1640 1 -- 172.21.15.37:0/2712463981 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7fe69c15c8c0 msgr2=0x7fe69c0ba0e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:30:57.975 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:57.970+0000 7fe6a15c1640 1 --2- 172.21.15.37:0/2712463981 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7fe69c15c8c0 0x7fe69c0ba0e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:30:57.975 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:57.970+0000 7fe6a15c1640 1 -- 172.21.15.37:0/2712463981 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fe69c15b6c0 msgr2=0x7fe69c15c380 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:30:57.975 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:57.970+0000 7fe6a15c1640 1 --2- 172.21.15.37:0/2712463981 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fe69c15b6c0 0x7fe69c15c380 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:30:57.975 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:57.970+0000 7fe6a15c1640 1 -- 172.21.15.37:0/2712463981 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fe69c15ccd0 con 0x7fe6880051c0 2024-04-25T10:30:57.975 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:57.970+0000 7fe6a1dc2640 1 --2- 172.21.15.37:0/2712463981 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7fe69c15c8c0 0x7fe69c0ba0e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T10:30:57.975 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:57.970+0000 7fe6a15c1640 1 --2- 172.21.15.37:0/2712463981 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fe6880051c0 0x7fe69c1f6400 secure :-1 s=READY pgs=676 cs=0 l=1 rev1=1 crypto rx=0x7fe69800f5f0 tx=0x7fe69800fab0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:30:57.975 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:57.970+0000 7fe691ffb640 1 -- 172.21.15.37:0/2712463981 <== mon.0 v2:172.21.15.37:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe6980052e0 con 0x7fe6880051c0 2024-04-25T10:30:57.976 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:57.970+0000 7fe6a3825640 1 -- 172.21.15.37:0/2712463981 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fe69c1f75c0 con 0x7fe6880051c0 2024-04-25T10:30:57.976 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:57.970+0000 7fe691ffb640 1 -- 172.21.15.37:0/2712463981 <== mon.0 v2:172.21.15.37:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fe698018070 con 0x7fe6880051c0 2024-04-25T10:30:57.976 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:57.970+0000 7fe691ffb640 1 -- 172.21.15.37:0/2712463981 <== mon.0 v2:172.21.15.37:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe698003830 con 0x7fe6880051c0 2024-04-25T10:30:57.976 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:57.970+0000 7fe6a3825640 1 -- 172.21.15.37:0/2712463981 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fe69c1f6e50 con 0x7fe6880051c0 2024-04-25T10:30:57.976 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:57.971+0000 7fe6a3825640 1 -- 172.21.15.37:0/2712463981 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fe69c1f7790 con 0x7fe6880051c0 2024-04-25T10:30:57.977 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:57.972+0000 7fe691ffb640 1 -- 172.21.15.37:0/2712463981 <== mon.0 v2:172.21.15.37:3300/0 4 ==== mgrmap(e 5) ==== 96782+0+0 (secure 0 0 0) 0x7fe6980039d0 con 0x7fe6880051c0 2024-04-25T10:30:57.977 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:57.973+0000 7fe691ffb640 1 --2- 172.21.15.37:0/2712463981 >> [v2:172.21.15.203:6832/2418205237,v1:172.21.15.203:6833/2418205237] conn(0x7fe6800744e0 0x7fe6800769f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:30:57.977 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:57.973+0000 7fe691ffb640 1 -- 172.21.15.37:0/2712463981 <== mon.0 v2:172.21.15.37:3300/0 5 ==== osd_map(47..47 src has 1..47) ==== 5926+0+0 (secure 0 0 0) 0x7fe698095040 con 0x7fe6880051c0 2024-04-25T10:30:57.977 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:57.973+0000 7fe6a0dc0640 1 --2- 172.21.15.37:0/2712463981 >> [v2:172.21.15.203:6832/2418205237,v1:172.21.15.203:6833/2418205237] conn(0x7fe6800744e0 0x7fe6800769f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:30:57.977 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:57.973+0000 7fe6a0dc0640 1 --2- 172.21.15.37:0/2712463981 >> [v2:172.21.15.203:6832/2418205237,v1:172.21.15.203:6833/2418205237] conn(0x7fe6800744e0 0x7fe6800769f0 secure :-1 s=READY pgs=305 cs=0 l=1 rev1=1 crypto rx=0x7fe69c0cd1a0 tx=0x7fe68c008040 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:30:57.979 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:57.977+0000 7fe691ffb640 1 -- 172.21.15.37:0/2712463981 <== mon.0 v2:172.21.15.37:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187246 (secure 0 0 0) 0x7fe69c15ccd0 con 0x7fe6880051c0 2024-04-25T10:30:58.166 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:58.164+0000 7fe6a3825640 1 -- 172.21.15.37:0/2712463981 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7fe69c06a140 con 0x7fe6880051c0 2024-04-25T10:30:58.168 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:58.165+0000 7fe691ffb640 1 -- 172.21.15.37:0/2712463981 <== mon.0 v2:172.21.15.37:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v47) ==== 74+0+17009 (secure 0 0 0) 0x7fe69805f6e0 con 0x7fe6880051c0 2024-04-25T10:30:58.168 INFO:teuthology.orchestra.run.smithi037.stdout: 2024-04-25T10:30:58.168 INFO:teuthology.orchestra.run.smithi037.stdout:{"epoch":47,"fsid":"f4acda28-be34-492a-b286-994afd36f61d","created":"2024-04-25T10:28:43.985718+0000","modified":"2024-04-25T10:30:55.827846+0000","last_up_change":"2024-04-25T10:28:54.900654+0000","last_in_change":"2024-04-25T10:28:46.216973+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":6,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"luminous","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-04-25T10:28:56.202153+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"19","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":5,"pool_name":"cephfs_data","create_time":"2024-04-25T10:30:11.084827+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"43","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"recovery_fs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.3799999952316284,"score_stable":1.3799999952316284,"optimal_score":1,"raw_score_acting":1.3799999952316284,"raw_score_stable":1.3799999952316284,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":6,"pool_name":"recovery_fs_metadata","create_time":"2024-04-25T10:30:26.903737+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"44","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"recovery_fs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"a2f79028-b332-4166-9ebb-a7024a37ea22","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":42,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.37:6816","nonce":2746074720},{"type":"v1","addr":"172.21.15.37:6817","nonce":2746074720}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.37:6818","nonce":2746074720},{"type":"v1","addr":"172.21.15.37:6819","nonce":2746074720}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.37:6822","nonce":2746074720},{"type":"v1","addr":"172.21.15.37:6823","nonce":2746074720}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.37:6820","nonce":2746074720},{"type":"v1","addr":"172.21.15.37:6821","nonce":2746074720}]},"public_addr":"172.21.15.37:6817/2746074720","cluster_addr":"172.21.15.37:6819/2746074720","heartbeat_back_addr":"172.21.15.37:6823/2746074720","heartbeat_front_addr":"172.21.15.37:6821/2746074720","state":["exists","up"]},{"osd":1,"uuid":"973159de-c2ea-4083-aecc-69373d1c1132","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":45,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.37:6800","nonce":1690341662},{"type":"v1","addr":"172.21.15.37:6801","nonce":1690341662}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.37:6802","nonce":1690341662},{"type":"v1","addr":"172.21.15.37:6803","nonce":1690341662}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.37:6806","nonce":1690341662},{"type":"v1","addr":"172.21.15.37:6807","nonce":1690341662}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.37:6804","nonce":1690341662},{"type":"v1","addr":"172.21.15.37:6805","nonce":1690341662}]},"public_addr":"172.21.15.37:6801/1690341662","cluster_addr":"172.21.15.37:6803/1690341662","heartbeat_back_addr":"172.21.15.37:6807/1690341662","heartbeat_front_addr":"172.21.15.37:6805/1690341662","state":["exists","up"]},{"osd":2,"uuid":"4ae7a0e9-cb3f-484c-a75f-f3766b721c47","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":42,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.37:6808","nonce":2754669649},{"type":"v1","addr":"172.21.15.37:6809","nonce":2754669649}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.37:6810","nonce":2754669649},{"type":"v1","addr":"172.21.15.37:6811","nonce":2754669649}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.37:6814","nonce":2754669649},{"type":"v1","addr":"172.21.15.37:6815","nonce":2754669649}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.37:6812","nonce":2754669649},{"type":"v1","addr":"172.21.15.37:6813","nonce":2754669649}]},"public_addr":"172.21.15.37:6809/2754669649","cluster_addr":"172.21.15.37:6811/2754669649","heartbeat_back_addr":"172.21.15.37:6815/2754669649","heartbeat_front_addr":"172.21.15.37:6813/2754669649","state":["exists","up"]},{"osd":3,"uuid":"1ba4a746-1608-47a7-ace2-960a5cfae934","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":45,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.37:6824","nonce":1815290618},{"type":"v1","addr":"172.21.15.37:6825","nonce":1815290618}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.37:6826","nonce":1815290618},{"type":"v1","addr":"172.21.15.37:6827","nonce":1815290618}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.37:6830","nonce":1815290618},{"type":"v1","addr":"172.21.15.37:6831","nonce":1815290618}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.37:6828","nonce":1815290618},{"type":"v1","addr":"172.21.15.37:6829","nonce":1815290618}]},"public_addr":"172.21.15.37:6825/1815290618","cluster_addr":"172.21.15.37:6827/1815290618","heartbeat_back_addr":"172.21.15.37:6831/1815290618","heartbeat_front_addr":"172.21.15.37:6829/1815290618","state":["exists","up"]},{"osd":4,"uuid":"d4f59f5d-0413-4b79-87ee-bb212c1fcf62","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":42,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6800","nonce":4051532063},{"type":"v1","addr":"172.21.15.203:6801","nonce":4051532063}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6802","nonce":4051532063},{"type":"v1","addr":"172.21.15.203:6803","nonce":4051532063}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6806","nonce":4051532063},{"type":"v1","addr":"172.21.15.203:6807","nonce":4051532063}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6804","nonce":4051532063},{"type":"v1","addr":"172.21.15.203:6805","nonce":4051532063}]},"public_addr":"172.21.15.203:6801/4051532063","cluster_addr":"172.21.15.203:6803/4051532063","heartbeat_back_addr":"172.21.15.203:6807/4051532063","heartbeat_front_addr":"172.21.15.203:6805/4051532063","state":["exists","up"]},{"osd":5,"uuid":"69710a0e-41d8-4e7f-9a10-17ce540b67be","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":45,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6824","nonce":1203822890},{"type":"v1","addr":"172.21.15.203:6825","nonce":1203822890}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6826","nonce":1203822890},{"type":"v1","addr":"172.21.15.203:6827","nonce":1203822890}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6830","nonce":1203822890},{"type":"v1","addr":"172.21.15.203:6831","nonce":1203822890}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6828","nonce":1203822890},{"type":"v1","addr":"172.21.15.203:6829","nonce":1203822890}]},"public_addr":"172.21.15.203:6825/1203822890","cluster_addr":"172.21.15.203:6827/1203822890","heartbeat_back_addr":"172.21.15.203:6831/1203822890","heartbeat_front_addr":"172.21.15.203:6829/1203822890","state":["exists","up"]},{"osd":6,"uuid":"f29dd047-3b48-473d-a235-6642a6293f20","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":42,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6810","nonce":2309615014},{"type":"v1","addr":"172.21.15.203:6812","nonce":2309615014}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6814","nonce":2309615014},{"type":"v1","addr":"172.21.15.203:6816","nonce":2309615014}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6822","nonce":2309615014},{"type":"v1","addr":"172.21.15.203:6823","nonce":2309615014}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6818","nonce":2309615014},{"type":"v1","addr":"172.21.15.203:6820","nonce":2309615014}]},"public_addr":"172.21.15.203:6812/2309615014","cluster_addr":"172.21.15.203:6816/2309615014","heartbeat_back_addr":"172.21.15.203:6823/2309615014","heartbeat_front_addr":"172.21.15.203:6820/2309615014","state":["exists","up"]},{"osd":7,"uuid":"6e691644-378b-4fdc-8e86-fb5f56e31f29","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":42,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6808","nonce":3688099014},{"type":"v1","addr":"172.21.15.203:6809","nonce":3688099014}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6811","nonce":3688099014},{"type":"v1","addr":"172.21.15.203:6813","nonce":3688099014}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6819","nonce":3688099014},{"type":"v1","addr":"172.21.15.203:6821","nonce":3688099014}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6815","nonce":3688099014},{"type":"v1","addr":"172.21.15.203:6817","nonce":3688099014}]},"public_addr":"172.21.15.203:6809/3688099014","cluster_addr":"172.21.15.203:6813/3688099014","heartbeat_back_addr":"172.21.15.203:6821/3688099014","heartbeat_front_addr":"172.21.15.203:6817/3688099014","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T10:28:51.419141+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T10:28:51.366448+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T10:28:51.348276+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T10:28:51.380519+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T10:28:51.105812+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T10:28:51.384496+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T10:28:51.388755+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T10:28:51.366958+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[{"pgid":"5.15","mappings":[{"from":4,"to":5}]},{"pgid":"5.1a","mappings":[{"from":4,"to":5}]},{"pgid":"5.26","mappings":[{"from":4,"to":5}]},{"pgid":"5.27","mappings":[{"from":4,"to":5}]}],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.37:6835/311552410":"2024-04-26T10:30:24.853732+0000","172.21.15.37:6832/311552410":"2024-04-26T10:30:24.853732+0000"},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-04-25T10:30:58.170 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:58.168+0000 7fe6a3825640 1 -- 172.21.15.37:0/2712463981 >> [v2:172.21.15.203:6832/2418205237,v1:172.21.15.203:6833/2418205237] conn(0x7fe6800744e0 msgr2=0x7fe6800769f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:30:58.170 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:58.168+0000 7fe6a3825640 1 --2- 172.21.15.37:0/2712463981 >> [v2:172.21.15.203:6832/2418205237,v1:172.21.15.203:6833/2418205237] conn(0x7fe6800744e0 0x7fe6800769f0 secure :-1 s=READY pgs=305 cs=0 l=1 rev1=1 crypto rx=0x7fe69c0cd1a0 tx=0x7fe68c008040 comp rx=0 tx=0).stop 2024-04-25T10:30:58.170 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:58.168+0000 7fe6a3825640 1 -- 172.21.15.37:0/2712463981 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fe6880051c0 msgr2=0x7fe69c1f6400 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:30:58.170 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:58.168+0000 7fe6a3825640 1 --2- 172.21.15.37:0/2712463981 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fe6880051c0 0x7fe69c1f6400 secure :-1 s=READY pgs=676 cs=0 l=1 rev1=1 crypto rx=0x7fe69800f5f0 tx=0x7fe69800fab0 comp rx=0 tx=0).stop 2024-04-25T10:30:58.170 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:58.168+0000 7fe6a3825640 1 -- 172.21.15.37:0/2712463981 shutdown_connections 2024-04-25T10:30:58.171 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:58.168+0000 7fe6a3825640 1 --2- 172.21.15.37:0/2712463981 >> [v2:172.21.15.203:6832/2418205237,v1:172.21.15.203:6833/2418205237] conn(0x7fe6800744e0 0x7fe6800769f0 unknown :-1 s=CLOSED pgs=305 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:30:58.171 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:58.168+0000 7fe6a3825640 1 --2- 172.21.15.37:0/2712463981 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7fe69c15c8c0 0x7fe69c0ba0e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:30:58.171 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:58.168+0000 7fe6a3825640 1 --2- 172.21.15.37:0/2712463981 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fe69c15b6c0 0x7fe69c15c380 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:30:58.171 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:58.168+0000 7fe6a3825640 1 --2- 172.21.15.37:0/2712463981 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fe6880051c0 0x7fe69c1f6400 unknown :-1 s=CLOSED pgs=676 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:30:58.171 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:58.168+0000 7fe6a3825640 1 -- 172.21.15.37:0/2712463981 >> 172.21.15.37:0/2712463981 conn(0x7fe69c0d27e0 msgr2=0x7fe69c162ad0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:30:58.171 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:58.168+0000 7fe6a3825640 1 -- 172.21.15.37:0/2712463981 shutdown_connections 2024-04-25T10:30:58.171 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:58.168+0000 7fe6a3825640 1 -- 172.21.15.37:0/2712463981 wait complete. 2024-04-25T10:30:58.181 INFO:tasks.cephfs.filesystem:Destroying file system recovery_fs and related pools 2024-04-25T10:30:58.182 DEBUG:teuthology.orchestra.run.smithi037:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-04-25T10:30:58.294 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:58.292+0000 7f862d205640 1 Processor -- start 2024-04-25T10:30:58.294 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:58.292+0000 7f862d205640 1 -- start start 2024-04-25T10:30:58.295 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:58.293+0000 7f862d205640 1 --2- >> v2:172.21.15.203:3300/0 conn(0x7f86280d5160 0x7f86280b4e80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:30:58.295 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:58.293+0000 7f862d205640 1 --2- >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f86280d3230 0x7f86280b9060 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:30:58.296 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:58.293+0000 7f862d205640 1 -- --> v1:172.21.15.203:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f86280d1370 con 0x7f86280d0990 2024-04-25T10:30:58.296 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:58.293+0000 7f862d205640 1 -- --> v2:172.21.15.203:3300/0 -- mon_getmap magic: 0 -- 0x7f86280d1650 con 0x7f86280d5160 2024-04-25T10:30:58.296 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:58.293+0000 7f862d205640 1 -- --> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] -- mon_getmap magic: 0 -- 0x7f86280d4680 con 0x7f86280d3230 2024-04-25T10:30:58.297 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:58.294+0000 7f862759e640 1 --2- >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f86280d3230 0x7f86280b9060 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:30:58.298 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:58.294+0000 7f8626d9d640 1 --2- >> v2:172.21.15.203:3300/0 conn(0x7f86280d5160 0x7f86280b4e80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:30:58.298 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:58.294+0000 7f8626d9d640 1 --2- >> v2:172.21.15.203:3300/0 conn(0x7f86280d5160 0x7f86280b4e80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.203:3300/0 says I am v2:172.21.15.37:57484/0 (socket says 172.21.15.37:57484) 2024-04-25T10:30:58.298 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:58.294+0000 7f862659c640 1 --1- >> v1:172.21.15.203:6789/0 conn(0x7f86280d0990 0x7f86280d2b20 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.203:6789/0 says I am v1:172.21.15.37:37490/0 (socket says 172.21.15.37:37490) 2024-04-25T10:30:58.298 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:58.294+0000 7f8626d9d640 1 -- 172.21.15.37:0/69640095 learned_addr learned my addr 172.21.15.37:0/69640095 (peer_addr_for_me v2:172.21.15.37:0/0) 2024-04-25T10:30:58.298 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:58.294+0000 7f862759e640 1 --2- >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f86280d3230 0x7f86280b9060 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.203:3301/0 says I am v2:172.21.15.37:59044/0 (socket says 172.21.15.37:59044) 2024-04-25T10:30:58.298 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:58.294+0000 7f8626d9d640 1 -- 172.21.15.37:0/69640095 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f86280d3230 msgr2=0x7f86280b9060 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:30:58.298 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:58.294+0000 7f8626d9d640 1 --2- 172.21.15.37:0/69640095 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f86280d3230 0x7f86280b9060 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:30:58.298 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:58.294+0000 7f8626d9d640 1 -- 172.21.15.37:0/69640095 >> v1:172.21.15.203:6789/0 conn(0x7f86280d0990 legacy=0x7f86280d2b20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:30:58.298 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:58.294+0000 7f8626d9d640 1 -- 172.21.15.37:0/69640095 --> v2:172.21.15.203:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f86280d4960 con 0x7f86280d5160 2024-04-25T10:30:58.298 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:58.294+0000 7f8625d9b640 1 -- 172.21.15.37:0/69640095 <== mon.1 v1:172.21.15.203:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 2822192574 0 0) 0x7f86280d1370 con 0x7f86280d0990 2024-04-25T10:30:58.298 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:58.295+0000 7f8626d9d640 1 --2- 172.21.15.37:0/69640095 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f86280d5160 0x7f86280b4e80 secure :-1 s=READY pgs=248 cs=0 l=1 rev1=1 crypto rx=0x7f8618009870 tx=0x7f861802a130 comp rx=0 tx=0).ready entity=mon.1 client_cookie=d3c8e8ec074781f2 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:30:58.298 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:58.295+0000 7f8625d9b640 1 -- 172.21.15.37:0/69640095 <== mon.1 v2:172.21.15.203:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8618037070 con 0x7f86280d5160 2024-04-25T10:30:58.299 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:58.295+0000 7f8625d9b640 1 -- 172.21.15.37:0/69640095 <== mon.1 v2:172.21.15.203:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f8618032440 con 0x7f86280d5160 2024-04-25T10:30:58.299 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:58.295+0000 7f8625d9b640 1 -- 172.21.15.37:0/69640095 <== mon.1 v2:172.21.15.203:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8618032810 con 0x7f86280d5160 2024-04-25T10:30:58.300 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:58.296+0000 7f862d205640 1 -- 172.21.15.37:0/69640095 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f86280d5160 msgr2=0x7f86280b4e80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:30:58.300 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:58.296+0000 7f862d205640 1 --2- 172.21.15.37:0/69640095 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f86280d5160 0x7f86280b4e80 secure :-1 s=READY pgs=248 cs=0 l=1 rev1=1 crypto rx=0x7f8618009870 tx=0x7f861802a130 comp rx=0 tx=0).stop 2024-04-25T10:30:58.300 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:58.296+0000 7f862d205640 1 -- 172.21.15.37:0/69640095 shutdown_connections 2024-04-25T10:30:58.300 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:58.296+0000 7f862d205640 1 --2- 172.21.15.37:0/69640095 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f86280d3230 0x7f86280b9060 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:30:58.300 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:58.296+0000 7f862d205640 1 --2- 172.21.15.37:0/69640095 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f86280d5160 0x7f86280b4e80 unknown :-1 s=CLOSED pgs=248 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:30:58.300 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:58.296+0000 7f862d205640 1 -- 172.21.15.37:0/69640095 >> 172.21.15.37:0/69640095 conn(0x7f86280b7a80 msgr2=0x7f86280b7e90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:30:58.300 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:58.296+0000 7f862d205640 1 -- 172.21.15.37:0/69640095 shutdown_connections 2024-04-25T10:30:58.300 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:58.296+0000 7f862d205640 1 -- 172.21.15.37:0/69640095 wait complete. 2024-04-25T10:30:58.301 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:58.298+0000 7f862d205640 1 Processor -- start 2024-04-25T10:30:58.301 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:58.298+0000 7f862d205640 1 -- start start 2024-04-25T10:30:58.303 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:58.300+0000 7f862d205640 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f86280d0990 0x7f8628152100 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:30:58.303 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:58.300+0000 7f862d205640 1 --2- >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f86280d5160 0x7f86280cbc70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:30:58.303 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:58.300+0000 7f862d205640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f86281f7520 0x7f86280cc1b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:30:58.303 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:58.300+0000 7f862d205640 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 -- 0x7f8628164c40 con 0x7f86281f7520 2024-04-25T10:30:58.303 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:58.300+0000 7f862d205640 1 -- --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_getmap magic: 0 -- 0x7f8628164ac0 con 0x7f86280d0990 2024-04-25T10:30:58.304 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:58.300+0000 7f862d205640 1 -- --> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] -- mon_getmap magic: 0 -- 0x7f86280d0da0 con 0x7f86280d5160 2024-04-25T10:30:58.304 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:58.300+0000 7f862759e640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f86281f7520 0x7f86280cc1b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:30:58.304 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:58.300+0000 7f862759e640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f86281f7520 0x7f86280cc1b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.37:3300/0 says I am v2:172.21.15.37:59708/0 (socket says 172.21.15.37:59708) 2024-04-25T10:30:58.304 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:58.300+0000 7f862759e640 1 -- 172.21.15.37:0/1378987546 learned_addr learned my addr 172.21.15.37:0/1378987546 (peer_addr_for_me v2:172.21.15.37:0/0) 2024-04-25T10:30:58.304 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:58.300+0000 7f8626d9d640 1 --2- 172.21.15.37:0/1378987546 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f86280d0990 0x7f8628152100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:30:58.304 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:58.300+0000 7f862659c640 1 --2- 172.21.15.37:0/1378987546 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f86280d5160 0x7f86280cbc70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:30:58.304 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:58.300+0000 7f862759e640 1 -- 172.21.15.37:0/1378987546 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f86280d5160 msgr2=0x7f86280cbc70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:30:58.304 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:58.300+0000 7f862759e640 1 --2- 172.21.15.37:0/1378987546 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f86280d5160 0x7f86280cbc70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:30:58.305 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:58.300+0000 7f862759e640 1 -- 172.21.15.37:0/1378987546 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f86280d0990 msgr2=0x7f8628152100 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:30:58.305 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:58.300+0000 7f862759e640 1 --2- 172.21.15.37:0/1378987546 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f86280d0990 0x7f8628152100 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:30:58.305 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:58.300+0000 7f862759e640 1 -- 172.21.15.37:0/1378987546 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f8628152a20 con 0x7f86281f7520 2024-04-25T10:30:58.305 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:58.300+0000 7f862759e640 1 --2- 172.21.15.37:0/1378987546 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f86281f7520 0x7f86280cc1b0 secure :-1 s=READY pgs=677 cs=0 l=1 rev1=1 crypto rx=0x7f862000ef90 tx=0x7f862000c490 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:30:58.305 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:58.301+0000 7f860f7fe640 1 -- 172.21.15.37:0/1378987546 <== mon.0 v2:172.21.15.37:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8620018020 con 0x7f86281f7520 2024-04-25T10:30:58.305 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:58.301+0000 7f860f7fe640 1 -- 172.21.15.37:0/1378987546 <== mon.0 v2:172.21.15.37:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f8620014480 con 0x7f86281f7520 2024-04-25T10:30:58.305 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:58.301+0000 7f860f7fe640 1 -- 172.21.15.37:0/1378987546 <== mon.0 v2:172.21.15.37:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8620009e60 con 0x7f86281f7520 2024-04-25T10:30:58.305 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:58.301+0000 7f862d205640 1 -- 172.21.15.37:0/1378987546 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f8628152d00 con 0x7f86281f7520 2024-04-25T10:30:58.305 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:58.301+0000 7f862d205640 1 -- 172.21.15.37:0/1378987546 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f86281532c0 con 0x7f86281f7520 2024-04-25T10:30:58.306 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:58.302+0000 7f862d205640 1 -- 172.21.15.37:0/1378987546 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f862806a140 con 0x7f86281f7520 2024-04-25T10:30:58.306 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:58.303+0000 7f860f7fe640 1 -- 172.21.15.37:0/1378987546 <== mon.0 v2:172.21.15.37:3300/0 4 ==== mgrmap(e 5) ==== 96782+0+0 (secure 0 0 0) 0x7f8620005d10 con 0x7f86281f7520 2024-04-25T10:30:58.306 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:58.303+0000 7f860f7fe640 1 --2- 172.21.15.37:0/1378987546 >> [v2:172.21.15.203:6832/2418205237,v1:172.21.15.203:6833/2418205237] conn(0x7f86140744b0 0x7f86140769c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:30:58.306 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:58.303+0000 7f860f7fe640 1 -- 172.21.15.37:0/1378987546 <== mon.0 v2:172.21.15.37:3300/0 5 ==== osd_map(47..47 src has 1..47) ==== 5926+0+0 (secure 0 0 0) 0x7f8620095f20 con 0x7f86281f7520 2024-04-25T10:30:58.306 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:58.303+0000 7f8626d9d640 1 --2- 172.21.15.37:0/1378987546 >> [v2:172.21.15.203:6832/2418205237,v1:172.21.15.203:6833/2418205237] conn(0x7f86140744b0 0x7f86140769c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:30:58.307 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:58.304+0000 7f8626d9d640 1 --2- 172.21.15.37:0/1378987546 >> [v2:172.21.15.203:6832/2418205237,v1:172.21.15.203:6833/2418205237] conn(0x7f86140744b0 0x7f86140769c0 secure :-1 s=READY pgs=306 cs=0 l=1 rev1=1 crypto rx=0x7f8618034000 tx=0x7f861802e000 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:30:58.310 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:58.308+0000 7f860f7fe640 1 -- 172.21.15.37:0/1378987546 <== mon.0 v2:172.21.15.37:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187246 (secure 0 0 0) 0x7f8620022030 con 0x7f86281f7520 2024-04-25T10:30:58.531 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:58.529+0000 7f862d205640 1 -- 172.21.15.37:0/1378987546 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f86280cc770 con 0x7f86281f7520 2024-04-25T10:30:58.532 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:58.529+0000 7f860f7fe640 1 -- 172.21.15.37:0/1378987546 <== mon.0 v2:172.21.15.37:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 29 v29) ==== 94+0+5190 (secure 0 0 0) 0x7f8620010040 con 0x7f86281f7520 2024-04-25T10:30:58.532 INFO:teuthology.orchestra.run.smithi037.stdout: 2024-04-25T10:30:58.532 INFO:teuthology.orchestra.run.smithi037.stdout:{"epoch":29,"default_fscid":-1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":4946,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.37:6835/683655741","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.37:6834","nonce":683655741},{"type":"v1","addr":"172.21.15.37:6835","nonce":683655741}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":19},{"gid":5215,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.203:6836/3927045001","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6835","nonce":3927045001},{"type":"v1","addr":"172.21.15.203:6836","nonce":3927045001}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":19},{"gid":5218,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.203:6837/3108183946","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6834","nonce":3108183946},{"type":"v1","addr":"172.21.15.203:6837","nonce":3108183946}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":19}],"filesystems":[{"mdsmap":{"epoch":27,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-04-25T10:30:28.588617+0000","modified":"2024-04-25T10:30:42.935167+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":1,"in":[0],"up":{"mds_0":5625},"failed":[],"damaged":[],"stopped":[],"info":{"gid_5625":{"gid":5625,"name":"c","rank":0,"incarnation":24,"state":"up:active","state_seq":7,"addr":"172.21.15.37:6833/2353203722","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.37:6832","nonce":2353203722},{"type":"v1","addr":"172.21.15.37:6833","nonce":2353203722}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[5],"metadata_pool":6,"enabled":true,"fs_name":"recovery_fs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":5625,"qdb_cluster":[5625]},"id":3}]} 2024-04-25T10:30:58.532 INFO:teuthology.orchestra.run.smithi037.stderr:dumped fsmap epoch 29 2024-04-25T10:30:58.534 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:58.531+0000 7f862d205640 1 -- 172.21.15.37:0/1378987546 >> [v2:172.21.15.203:6832/2418205237,v1:172.21.15.203:6833/2418205237] conn(0x7f86140744b0 msgr2=0x7f86140769c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:30:58.534 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:58.531+0000 7f862d205640 1 --2- 172.21.15.37:0/1378987546 >> [v2:172.21.15.203:6832/2418205237,v1:172.21.15.203:6833/2418205237] conn(0x7f86140744b0 0x7f86140769c0 secure :-1 s=READY pgs=306 cs=0 l=1 rev1=1 crypto rx=0x7f8618034000 tx=0x7f861802e000 comp rx=0 tx=0).stop 2024-04-25T10:30:58.534 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:58.531+0000 7f862d205640 1 -- 172.21.15.37:0/1378987546 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f86281f7520 msgr2=0x7f86280cc1b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:30:58.534 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:58.531+0000 7f862d205640 1 --2- 172.21.15.37:0/1378987546 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f86281f7520 0x7f86280cc1b0 secure :-1 s=READY pgs=677 cs=0 l=1 rev1=1 crypto rx=0x7f862000ef90 tx=0x7f862000c490 comp rx=0 tx=0).stop 2024-04-25T10:30:58.534 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:58.531+0000 7f862d205640 1 -- 172.21.15.37:0/1378987546 shutdown_connections 2024-04-25T10:30:58.534 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:58.531+0000 7f862d205640 1 --2- 172.21.15.37:0/1378987546 >> [v2:172.21.15.203:6832/2418205237,v1:172.21.15.203:6833/2418205237] conn(0x7f86140744b0 0x7f86140769c0 unknown :-1 s=CLOSED pgs=306 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:30:58.534 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:58.531+0000 7f862d205640 1 --2- 172.21.15.37:0/1378987546 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f86281f7520 0x7f86280cc1b0 unknown :-1 s=CLOSED pgs=677 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:30:58.534 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:58.531+0000 7f862d205640 1 --2- 172.21.15.37:0/1378987546 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f86280d5160 0x7f86280cbc70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:30:58.534 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:58.531+0000 7f862d205640 1 --2- 172.21.15.37:0/1378987546 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f86280d0990 0x7f8628152100 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:30:58.535 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:58.531+0000 7f862d205640 1 -- 172.21.15.37:0/1378987546 >> 172.21.15.37:0/1378987546 conn(0x7f86280b7a80 msgr2=0x7f8628161550 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:30:58.535 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:58.531+0000 7f862d205640 1 -- 172.21.15.37:0/1378987546 shutdown_connections 2024-04-25T10:30:58.535 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:58.531+0000 7f862d205640 1 -- 172.21.15.37:0/1378987546 wait complete. 2024-04-25T10:30:58.545 DEBUG:teuthology.orchestra.run.smithi037:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-04-25T10:30:58.656 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:58.654+0000 7fb3aa5e3640 1 Processor -- start 2024-04-25T10:30:58.656 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:58.654+0000 7fb3aa5e3640 1 -- start start 2024-04-25T10:30:58.658 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:58.655+0000 7fb3aa5e3640 1 --2- >> v2:172.21.15.37:3300/0 conn(0x7fb3a40b4bb0 0x7fb3a41646b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:30:58.658 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:58.655+0000 7fb3aa5e3640 1 --2- >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7fb3a40d4560 0x7fb3a40d4970 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:30:58.658 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:58.655+0000 7fb3aa5e3640 1 -- --> v1:172.21.15.203:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fb3a4161190 con 0x7fb3a40b7600 2024-04-25T10:30:58.658 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:58.655+0000 7fb3aa5e3640 1 -- --> v2:172.21.15.37:3300/0 -- mon_getmap magic: 0 -- 0x7fb3a4161470 con 0x7fb3a40b4bb0 2024-04-25T10:30:58.658 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:58.655+0000 7fb3aa5e3640 1 -- --> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] -- mon_getmap magic: 0 -- 0x7fb3a4163eb0 con 0x7fb3a40d4560 2024-04-25T10:30:58.658 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:58.655+0000 7fb3a3fff640 1 --2- >> v2:172.21.15.37:3300/0 conn(0x7fb3a40b4bb0 0x7fb3a41646b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:30:58.658 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:58.655+0000 7fb3a3fff640 1 --2- >> v2:172.21.15.37:3300/0 conn(0x7fb3a40b4bb0 0x7fb3a41646b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.37:3300/0 says I am v2:172.21.15.37:59710/0 (socket says 172.21.15.37:59710) 2024-04-25T10:30:58.658 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:58.655+0000 7fb3a3fff640 1 -- 172.21.15.37:0/3098057381 learned_addr learned my addr 172.21.15.37:0/3098057381 (peer_addr_for_me v2:172.21.15.37:0/0) 2024-04-25T10:30:58.658 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:58.655+0000 7fb3a8b80640 1 --2- 172.21.15.37:0/3098057381 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7fb3a40d4560 0x7fb3a40d4970 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:30:58.659 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:58.656+0000 7fb3a3fff640 1 -- 172.21.15.37:0/3098057381 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7fb3a40d4560 msgr2=0x7fb3a40d4970 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:30:58.659 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:58.656+0000 7fb3a3fff640 1 --2- 172.21.15.37:0/3098057381 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7fb3a40d4560 0x7fb3a40d4970 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:30:58.659 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:58.656+0000 7fb3a3fff640 1 -- 172.21.15.37:0/3098057381 >> v1:172.21.15.203:6789/0 conn(0x7fb3a40b7600 legacy=0x7fb3a40b7a10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:30:58.659 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:58.656+0000 7fb3a3fff640 1 -- 172.21.15.37:0/3098057381 --> v2:172.21.15.37:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fb3a4164190 con 0x7fb3a40b4bb0 2024-04-25T10:30:58.660 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:58.656+0000 7fb3a3fff640 1 --2- 172.21.15.37:0/3098057381 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fb3a40b4bb0 0x7fb3a41646b0 secure :-1 s=READY pgs=678 cs=0 l=1 rev1=1 crypto rx=0x7fb394009870 tx=0x7fb39402a270 comp rx=0 tx=0).ready entity=mon.0 client_cookie=897c17333dc0264f server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:30:58.660 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:58.656+0000 7fb3a2ffd640 1 -- 172.21.15.37:0/3098057381 <== mon.0 v2:172.21.15.37:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb394037070 con 0x7fb3a40b4bb0 2024-04-25T10:30:58.660 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:58.657+0000 7fb3a2ffd640 1 -- 172.21.15.37:0/3098057381 <== mon.0 v2:172.21.15.37:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fb394032440 con 0x7fb3a40b4bb0 2024-04-25T10:30:58.660 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:58.657+0000 7fb3a2ffd640 1 -- 172.21.15.37:0/3098057381 <== mon.0 v2:172.21.15.37:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb394032810 con 0x7fb3a40b4bb0 2024-04-25T10:30:58.660 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:58.657+0000 7fb3aa5e3640 1 -- 172.21.15.37:0/3098057381 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fb3a40b4bb0 msgr2=0x7fb3a41646b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:30:58.660 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:58.657+0000 7fb3aa5e3640 1 --2- 172.21.15.37:0/3098057381 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fb3a40b4bb0 0x7fb3a41646b0 secure :-1 s=READY pgs=678 cs=0 l=1 rev1=1 crypto rx=0x7fb394009870 tx=0x7fb39402a270 comp rx=0 tx=0).stop 2024-04-25T10:30:58.660 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:58.657+0000 7fb3aa5e3640 1 -- 172.21.15.37:0/3098057381 shutdown_connections 2024-04-25T10:30:58.660 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:58.657+0000 7fb3aa5e3640 1 --2- 172.21.15.37:0/3098057381 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7fb3a40d4560 0x7fb3a40d4970 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:30:58.661 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:58.657+0000 7fb3aa5e3640 1 --2- 172.21.15.37:0/3098057381 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fb3a40b4bb0 0x7fb3a41646b0 unknown :-1 s=CLOSED pgs=678 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:30:58.661 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:58.657+0000 7fb3aa5e3640 1 -- 172.21.15.37:0/3098057381 >> 172.21.15.37:0/3098057381 conn(0x7fb3a40d27e0 msgr2=0x7fb3a40d2bf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:30:58.661 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:58.657+0000 7fb3aa5e3640 1 -- 172.21.15.37:0/3098057381 shutdown_connections 2024-04-25T10:30:58.661 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:58.657+0000 7fb3aa5e3640 1 -- 172.21.15.37:0/3098057381 wait complete. 2024-04-25T10:30:58.661 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:58.659+0000 7fb3aa5e3640 1 Processor -- start 2024-04-25T10:30:58.661 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:58.659+0000 7fb3aa5e3640 1 -- start start 2024-04-25T10:30:58.663 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:58.660+0000 7fb3aa5e3640 1 --2- >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7fb3a40b4bb0 0x7fb3a415a540 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:30:58.664 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:58.660+0000 7fb3aa5e3640 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fb3a40b7600 0x7fb3a415c030 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:30:58.664 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:58.660+0000 7fb3aa5e3640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fb3a40d4560 0x7fb3a40d4970 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:30:58.664 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:58.660+0000 7fb3aa5e3640 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 -- 0x7fb3a40d09e0 con 0x7fb3a40d4560 2024-04-25T10:30:58.664 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:58.660+0000 7fb3aa5e3640 1 -- --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_getmap magic: 0 -- 0x7fb3a4164bf0 con 0x7fb3a40b7600 2024-04-25T10:30:58.664 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:58.660+0000 7fb3aa5e3640 1 -- --> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] -- mon_getmap magic: 0 -- 0x7fb3a40d0b60 con 0x7fb3a40b4bb0 2024-04-25T10:30:58.664 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:58.661+0000 7fb3a8b80640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fb3a40d4560 0x7fb3a40d4970 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:30:58.664 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:58.661+0000 7fb3a8b80640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fb3a40d4560 0x7fb3a40d4970 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.37:3300/0 says I am v2:172.21.15.37:59712/0 (socket says 172.21.15.37:59712) 2024-04-25T10:30:58.664 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:58.661+0000 7fb3a8b80640 1 -- 172.21.15.37:0/2753911234 learned_addr learned my addr 172.21.15.37:0/2753911234 (peer_addr_for_me v2:172.21.15.37:0/0) 2024-04-25T10:30:58.664 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:58.661+0000 7fb3a3fff640 1 --2- 172.21.15.37:0/2753911234 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7fb3a40b4bb0 0x7fb3a415a540 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:30:58.664 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:58.661+0000 7fb3a37fe640 1 --2- 172.21.15.37:0/2753911234 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fb3a40b7600 0x7fb3a415c030 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:30:58.664 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:58.661+0000 7fb3a8b80640 1 -- 172.21.15.37:0/2753911234 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7fb3a40b4bb0 msgr2=0x7fb3a415a540 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:30:58.664 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:58.661+0000 7fb3a8b80640 1 --2- 172.21.15.37:0/2753911234 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7fb3a40b4bb0 0x7fb3a415a540 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:30:58.664 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:58.661+0000 7fb3a8b80640 1 -- 172.21.15.37:0/2753911234 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fb3a40b7600 msgr2=0x7fb3a415c030 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:30:58.665 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:58.661+0000 7fb3a8b80640 1 --2- 172.21.15.37:0/2753911234 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fb3a40b7600 0x7fb3a415c030 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:30:58.665 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:58.661+0000 7fb3a8b80640 1 -- 172.21.15.37:0/2753911234 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fb3a415aa80 con 0x7fb3a40d4560 2024-04-25T10:30:58.665 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:58.661+0000 7fb3a8b80640 1 --2- 172.21.15.37:0/2753911234 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fb3a40d4560 0x7fb3a40d4970 secure :-1 s=READY pgs=679 cs=0 l=1 rev1=1 crypto rx=0x7fb39c007e60 tx=0x7fb39c00e4b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:30:58.665 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:58.661+0000 7fb3a0ff9640 1 -- 172.21.15.37:0/2753911234 <== mon.0 v2:172.21.15.37:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb39c014420 con 0x7fb3a40d4560 2024-04-25T10:30:58.665 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:58.661+0000 7fb3a0ff9640 1 -- 172.21.15.37:0/2753911234 <== mon.0 v2:172.21.15.37:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fb39c019070 con 0x7fb3a40d4560 2024-04-25T10:30:58.665 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:58.661+0000 7fb3aa5e3640 1 -- 172.21.15.37:0/2753911234 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fb3a415af90 con 0x7fb3a40d4560 2024-04-25T10:30:58.665 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:58.661+0000 7fb3a0ff9640 1 -- 172.21.15.37:0/2753911234 <== mon.0 v2:172.21.15.37:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb39c014820 con 0x7fb3a40d4560 2024-04-25T10:30:58.665 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:58.661+0000 7fb3aa5e3640 1 -- 172.21.15.37:0/2753911234 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fb3a415b550 con 0x7fb3a40d4560 2024-04-25T10:30:58.666 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:58.662+0000 7fb3aa5e3640 1 -- 172.21.15.37:0/2753911234 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fb3a406a140 con 0x7fb3a40d4560 2024-04-25T10:30:58.666 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:58.663+0000 7fb3a0ff9640 1 -- 172.21.15.37:0/2753911234 <== mon.0 v2:172.21.15.37:3300/0 4 ==== mgrmap(e 5) ==== 96782+0+0 (secure 0 0 0) 0x7fb39c0149c0 con 0x7fb3a40d4560 2024-04-25T10:30:58.666 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:58.664+0000 7fb3a0ff9640 1 --2- 172.21.15.37:0/2753911234 >> [v2:172.21.15.203:6832/2418205237,v1:172.21.15.203:6833/2418205237] conn(0x7fb3900744b0 0x7fb3900769c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:30:58.666 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:58.664+0000 7fb3a0ff9640 1 -- 172.21.15.37:0/2753911234 <== mon.0 v2:172.21.15.37:3300/0 5 ==== osd_map(47..47 src has 1..47) ==== 5926+0+0 (secure 0 0 0) 0x7fb39c095b80 con 0x7fb3a40d4560 2024-04-25T10:30:58.666 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:58.664+0000 7fb3a3fff640 1 --2- 172.21.15.37:0/2753911234 >> [v2:172.21.15.203:6832/2418205237,v1:172.21.15.203:6833/2418205237] conn(0x7fb3900744b0 0x7fb3900769c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:30:58.666 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:58.664+0000 7fb3a3fff640 1 --2- 172.21.15.37:0/2753911234 >> [v2:172.21.15.203:6832/2418205237,v1:172.21.15.203:6833/2418205237] conn(0x7fb3900744b0 0x7fb3900769c0 secure :-1 s=READY pgs=307 cs=0 l=1 rev1=1 crypto rx=0x7fb394034000 tx=0x7fb39402e000 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:30:58.670 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:58.668+0000 7fb3a0ff9640 1 -- 172.21.15.37:0/2753911234 <== mon.0 v2:172.21.15.37:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187246 (secure 0 0 0) 0x7fb39c060220 con 0x7fb3a40d4560 2024-04-25T10:30:58.895 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:58.893+0000 7fb3aa5e3640 1 -- 172.21.15.37:0/2753911234 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7fb3a40d3ff0 con 0x7fb3a40d4560 2024-04-25T10:30:58.896 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:58.894+0000 7fb3a0ff9640 1 -- 172.21.15.37:0/2753911234 <== mon.0 v2:172.21.15.37:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 29 v29) ==== 94+0+5190 (secure 0 0 0) 0x7fb39c05fb20 con 0x7fb3a40d4560 2024-04-25T10:30:58.896 INFO:teuthology.orchestra.run.smithi037.stdout: 2024-04-25T10:30:58.896 INFO:teuthology.orchestra.run.smithi037.stdout:{"epoch":29,"default_fscid":-1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":4946,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.37:6835/683655741","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.37:6834","nonce":683655741},{"type":"v1","addr":"172.21.15.37:6835","nonce":683655741}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":19},{"gid":5215,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.203:6836/3927045001","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6835","nonce":3927045001},{"type":"v1","addr":"172.21.15.203:6836","nonce":3927045001}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":19},{"gid":5218,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.203:6837/3108183946","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6834","nonce":3108183946},{"type":"v1","addr":"172.21.15.203:6837","nonce":3108183946}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":19}],"filesystems":[{"mdsmap":{"epoch":27,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-04-25T10:30:28.588617+0000","modified":"2024-04-25T10:30:42.935167+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":1,"in":[0],"up":{"mds_0":5625},"failed":[],"damaged":[],"stopped":[],"info":{"gid_5625":{"gid":5625,"name":"c","rank":0,"incarnation":24,"state":"up:active","state_seq":7,"addr":"172.21.15.37:6833/2353203722","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.37:6832","nonce":2353203722},{"type":"v1","addr":"172.21.15.37:6833","nonce":2353203722}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[5],"metadata_pool":6,"enabled":true,"fs_name":"recovery_fs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":5625,"qdb_cluster":[5625]},"id":3}]} 2024-04-25T10:30:58.896 INFO:teuthology.orchestra.run.smithi037.stderr:dumped fsmap epoch 29 2024-04-25T10:30:58.898 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:58.896+0000 7fb3aa5e3640 1 -- 172.21.15.37:0/2753911234 >> [v2:172.21.15.203:6832/2418205237,v1:172.21.15.203:6833/2418205237] conn(0x7fb3900744b0 msgr2=0x7fb3900769c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:30:58.898 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:58.896+0000 7fb3aa5e3640 1 --2- 172.21.15.37:0/2753911234 >> [v2:172.21.15.203:6832/2418205237,v1:172.21.15.203:6833/2418205237] conn(0x7fb3900744b0 0x7fb3900769c0 secure :-1 s=READY pgs=307 cs=0 l=1 rev1=1 crypto rx=0x7fb394034000 tx=0x7fb39402e000 comp rx=0 tx=0).stop 2024-04-25T10:30:58.898 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:58.896+0000 7fb3aa5e3640 1 -- 172.21.15.37:0/2753911234 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fb3a40d4560 msgr2=0x7fb3a40d4970 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:30:58.899 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:58.896+0000 7fb3aa5e3640 1 --2- 172.21.15.37:0/2753911234 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fb3a40d4560 0x7fb3a40d4970 secure :-1 s=READY pgs=679 cs=0 l=1 rev1=1 crypto rx=0x7fb39c007e60 tx=0x7fb39c00e4b0 comp rx=0 tx=0).stop 2024-04-25T10:30:58.899 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:58.896+0000 7fb3aa5e3640 1 -- 172.21.15.37:0/2753911234 shutdown_connections 2024-04-25T10:30:58.899 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:58.896+0000 7fb3aa5e3640 1 --2- 172.21.15.37:0/2753911234 >> [v2:172.21.15.203:6832/2418205237,v1:172.21.15.203:6833/2418205237] conn(0x7fb3900744b0 0x7fb3900769c0 unknown :-1 s=CLOSED pgs=307 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:30:58.899 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:58.896+0000 7fb3aa5e3640 1 --2- 172.21.15.37:0/2753911234 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fb3a40d4560 0x7fb3a40d4970 unknown :-1 s=CLOSED pgs=679 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:30:58.899 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:58.896+0000 7fb3aa5e3640 1 --2- 172.21.15.37:0/2753911234 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fb3a40b7600 0x7fb3a415c030 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:30:58.899 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:58.896+0000 7fb3aa5e3640 1 --2- 172.21.15.37:0/2753911234 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7fb3a40b4bb0 0x7fb3a415a540 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:30:58.899 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:58.896+0000 7fb3aa5e3640 1 -- 172.21.15.37:0/2753911234 >> 172.21.15.37:0/2753911234 conn(0x7fb3a40d27e0 msgr2=0x7fb3a4162fe0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:30:58.899 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:58.896+0000 7fb3aa5e3640 1 -- 172.21.15.37:0/2753911234 shutdown_connections 2024-04-25T10:30:58.899 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:58.896+0000 7fb3aa5e3640 1 -- 172.21.15.37:0/2753911234 wait complete. 2024-04-25T10:30:58.909 DEBUG:teuthology.orchestra.run.smithi037:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-04-25T10:30:59.020 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:59.018+0000 7f7baf2af640 1 Processor -- start 2024-04-25T10:30:59.021 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:59.018+0000 7f7baf2af640 1 -- start start 2024-04-25T10:30:59.022 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:59.020+0000 7f7baf2af640 1 --2- >> v2:172.21.15.37:3300/0 conn(0x7f7ba80d5430 0x7f7ba80b75c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:30:59.022 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:59.020+0000 7f7baf2af640 1 -- --> v1:172.21.15.37:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f7ba80b7f10 con 0x7f7ba80b7b00 2024-04-25T10:30:59.022 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:59.020+0000 7f7baf2af640 1 -- --> v1:172.21.15.203:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f7ba8161520 con 0x7f7ba8161110 2024-04-25T10:30:59.023 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:59.020+0000 7f7baf2af640 1 -- --> v2:172.21.15.37:3300/0 -- mon_getmap magic: 0 -- 0x7f7ba80d4420 con 0x7f7ba80d5430 2024-04-25T10:30:59.023 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:59.020+0000 7f7bad04b640 1 --2- >> v2:172.21.15.37:3300/0 conn(0x7f7ba80d5430 0x7f7ba80b75c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:30:59.023 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:59.020+0000 7f7bac84a640 1 --1- >> v1:172.21.15.37:6789/0 conn(0x7f7ba80b7b00 0x7f7ba8160a00 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.37:6789/0 says I am v1:172.21.15.37:38782/0 (socket says 172.21.15.37:38782) 2024-04-25T10:30:59.023 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:59.020+0000 7f7bac84a640 1 -- 172.21.15.37:0/3880949690 learned_addr learned my addr 172.21.15.37:0/3880949690 (peer_addr_for_me v1:172.21.15.37:0/0) 2024-04-25T10:30:59.023 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:59.020+0000 7f7ba7fff640 1 -- 172.21.15.37:0/3880949690 <== mon.0 v1:172.21.15.37:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 964916972 0 0) 0x7f7b9c001110 con 0x7f7ba80b7b00 2024-04-25T10:30:59.023 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:59.020+0000 7f7ba7fff640 1 -- 172.21.15.37:0/3880949690 --> v1:172.21.15.37:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f7b90003520 con 0x7f7ba80b7b00 2024-04-25T10:30:59.023 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:59.020+0000 7f7bad04b640 1 -- 172.21.15.37:0/3880949690 >> v1:172.21.15.203:6789/0 conn(0x7f7ba8161110 legacy=0x7f7ba8161880 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:30:59.023 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:59.020+0000 7f7bad04b640 1 -- 172.21.15.37:0/3880949690 >> v1:172.21.15.37:6789/0 conn(0x7f7ba80b7b00 legacy=0x7f7ba8160a00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:30:59.023 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:59.020+0000 7f7bad04b640 1 -- 172.21.15.37:0/3880949690 --> v2:172.21.15.37:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f7ba80d5840 con 0x7f7ba80d5430 2024-04-25T10:30:59.024 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:59.021+0000 7f7bad04b640 1 --2- 172.21.15.37:0/3880949690 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f7ba80d5430 0x7f7ba80b75c0 secure :-1 s=READY pgs=681 cs=0 l=1 rev1=1 crypto rx=0x7f7ba81616f0 tx=0x7f7ba002a180 comp rx=0 tx=0).ready entity=mon.0 client_cookie=1f81068f21a7b5f3 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:30:59.024 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:59.021+0000 7f7ba7fff640 1 -- 172.21.15.37:0/3880949690 <== mon.0 v2:172.21.15.37:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7ba0037070 con 0x7f7ba80d5430 2024-04-25T10:30:59.024 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:59.021+0000 7f7ba7fff640 1 -- 172.21.15.37:0/3880949690 <== mon.0 v2:172.21.15.37:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f7ba0032440 con 0x7f7ba80d5430 2024-04-25T10:30:59.025 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:59.021+0000 7f7ba7fff640 1 -- 172.21.15.37:0/3880949690 <== mon.0 v2:172.21.15.37:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7ba0032810 con 0x7f7ba80d5430 2024-04-25T10:30:59.025 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:59.022+0000 7f7baf2af640 1 -- 172.21.15.37:0/3880949690 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f7ba80d5430 msgr2=0x7f7ba80b75c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:30:59.025 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:59.022+0000 7f7baf2af640 1 --2- 172.21.15.37:0/3880949690 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f7ba80d5430 0x7f7ba80b75c0 secure :-1 s=READY pgs=681 cs=0 l=1 rev1=1 crypto rx=0x7f7ba81616f0 tx=0x7f7ba002a180 comp rx=0 tx=0).stop 2024-04-25T10:30:59.025 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:59.022+0000 7f7baf2af640 1 -- 172.21.15.37:0/3880949690 shutdown_connections 2024-04-25T10:30:59.025 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:59.022+0000 7f7baf2af640 1 --2- 172.21.15.37:0/3880949690 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f7ba80d5430 0x7f7ba80b75c0 unknown :-1 s=CLOSED pgs=681 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:30:59.025 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:59.022+0000 7f7baf2af640 1 -- 172.21.15.37:0/3880949690 >> 172.21.15.37:0/3880949690 conn(0x7f7ba80d27a0 msgr2=0x7f7ba80d2bb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:30:59.025 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:59.022+0000 7f7baf2af640 1 -- 172.21.15.37:0/3880949690 shutdown_connections 2024-04-25T10:30:59.025 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:59.022+0000 7f7baf2af640 1 -- 172.21.15.37:0/3880949690 wait complete. 2024-04-25T10:30:59.026 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:59.024+0000 7f7baf2af640 1 Processor -- start 2024-04-25T10:30:59.026 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:59.024+0000 7f7baf2af640 1 -- start start 2024-04-25T10:30:59.028 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:59.025+0000 7f7baf2af640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f7ba80b7b00 0x7f7ba80cb650 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:30:59.028 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:59.025+0000 7f7baf2af640 1 --2- >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f7ba80d5430 0x7f7ba80cc980 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:30:59.028 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:59.025+0000 7f7baf2af640 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f7ba8161110 0x7f7ba80d5b20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:30:59.029 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:59.025+0000 7f7baf2af640 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 -- 0x7f7ba80d4c30 con 0x7f7ba80b7b00 2024-04-25T10:30:59.029 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:59.025+0000 7f7baf2af640 1 -- --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_getmap magic: 0 -- 0x7f7ba80d4950 con 0x7f7ba8161110 2024-04-25T10:30:59.029 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:59.025+0000 7f7baf2af640 1 -- --> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] -- mon_getmap magic: 0 -- 0x7f7ba80d4f10 con 0x7f7ba80d5430 2024-04-25T10:30:59.029 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:59.025+0000 7f7bad04b640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f7ba80b7b00 0x7f7ba80cb650 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:30:59.029 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:59.025+0000 7f7bad04b640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f7ba80b7b00 0x7f7ba80cb650 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.37:3300/0 says I am v2:172.21.15.37:59744/0 (socket says 172.21.15.37:59744) 2024-04-25T10:30:59.029 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:59.025+0000 7f7bad04b640 1 -- 172.21.15.37:0/1911951398 learned_addr learned my addr 172.21.15.37:0/1911951398 (peer_addr_for_me v2:172.21.15.37:0/0) 2024-04-25T10:30:59.029 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:59.025+0000 7f7bac84a640 1 --2- 172.21.15.37:0/1911951398 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f7ba80d5430 0x7f7ba80cc980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:30:59.029 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:59.025+0000 7f7bad84c640 1 --2- 172.21.15.37:0/1911951398 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f7ba8161110 0x7f7ba80d5b20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:30:59.029 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:59.025+0000 7f7bad04b640 1 -- 172.21.15.37:0/1911951398 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f7ba80d5430 msgr2=0x7f7ba80cc980 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:30:59.029 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:59.025+0000 7f7bad04b640 1 --2- 172.21.15.37:0/1911951398 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f7ba80d5430 0x7f7ba80cc980 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:30:59.029 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:59.025+0000 7f7bad04b640 1 -- 172.21.15.37:0/1911951398 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f7ba8161110 msgr2=0x7f7ba80d5b20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:30:59.029 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:59.025+0000 7f7bad04b640 1 --2- 172.21.15.37:0/1911951398 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f7ba8161110 0x7f7ba80d5b20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:30:59.029 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:59.025+0000 7f7bad04b640 1 -- 172.21.15.37:0/1911951398 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f7ba80cbb90 con 0x7f7ba80b7b00 2024-04-25T10:30:59.029 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:59.026+0000 7f7bad04b640 1 --2- 172.21.15.37:0/1911951398 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f7ba80b7b00 0x7f7ba80cb650 secure :-1 s=READY pgs=682 cs=0 l=1 rev1=1 crypto rx=0x7f7ba0000c00 tx=0x7f7ba00046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:30:59.030 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:59.026+0000 7f7ba5ffb640 1 -- 172.21.15.37:0/1911951398 <== mon.0 v2:172.21.15.37:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7ba0037040 con 0x7f7ba80b7b00 2024-04-25T10:30:59.030 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:59.026+0000 7f7ba5ffb640 1 -- 172.21.15.37:0/1911951398 <== mon.0 v2:172.21.15.37:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f7ba0042070 con 0x7f7ba80b7b00 2024-04-25T10:30:59.030 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:59.026+0000 7f7baf2af640 1 -- 172.21.15.37:0/1911951398 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f7ba80cd620 con 0x7f7ba80b7b00 2024-04-25T10:30:59.030 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:59.026+0000 7f7ba5ffb640 1 -- 172.21.15.37:0/1911951398 <== mon.0 v2:172.21.15.37:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7ba0030960 con 0x7f7ba80b7b00 2024-04-25T10:30:59.030 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:59.026+0000 7f7baf2af640 1 -- 172.21.15.37:0/1911951398 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f7ba80cdbe0 con 0x7f7ba80b7b00 2024-04-25T10:30:59.031 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:59.027+0000 7f7baf2af640 1 -- 172.21.15.37:0/1911951398 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f7ba806a140 con 0x7f7ba80b7b00 2024-04-25T10:30:59.031 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:59.028+0000 7f7ba5ffb640 1 -- 172.21.15.37:0/1911951398 <== mon.0 v2:172.21.15.37:3300/0 4 ==== mgrmap(e 5) ==== 96782+0+0 (secure 0 0 0) 0x7f7ba0030b00 con 0x7f7ba80b7b00 2024-04-25T10:30:59.031 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:59.028+0000 7f7ba5ffb640 1 --2- 172.21.15.37:0/1911951398 >> [v2:172.21.15.203:6832/2418205237,v1:172.21.15.203:6833/2418205237] conn(0x7f7b940744e0 0x7f7b940769f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:30:59.031 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:59.028+0000 7f7ba5ffb640 1 -- 172.21.15.37:0/1911951398 <== mon.0 v2:172.21.15.37:3300/0 5 ==== osd_map(47..47 src has 1..47) ==== 5926+0+0 (secure 0 0 0) 0x7f7ba00b5ab0 con 0x7f7ba80b7b00 2024-04-25T10:30:59.031 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:59.028+0000 7f7bac84a640 1 --2- 172.21.15.37:0/1911951398 >> [v2:172.21.15.203:6832/2418205237,v1:172.21.15.203:6833/2418205237] conn(0x7f7b940744e0 0x7f7b940769f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:30:59.031 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:59.029+0000 7f7bac84a640 1 --2- 172.21.15.37:0/1911951398 >> [v2:172.21.15.203:6832/2418205237,v1:172.21.15.203:6833/2418205237] conn(0x7f7b940744e0 0x7f7b940769f0 secure :-1 s=READY pgs=308 cs=0 l=1 rev1=1 crypto rx=0x7f7b9c007fd0 tx=0x7f7b9c009040 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:30:59.035 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:59.033+0000 7f7ba5ffb640 1 -- 172.21.15.37:0/1911951398 <== mon.0 v2:172.21.15.37:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187246 (secure 0 0 0) 0x7f7ba004c020 con 0x7f7ba80b7b00 2024-04-25T10:30:59.225 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:59.223+0000 7f7baf2af640 1 -- 172.21.15.37:0/1911951398 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7f7ba80cff70 con 0x7f7ba80b7b00 2024-04-25T10:30:59.226 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:59.224+0000 7f7ba5ffb640 1 -- 172.21.15.37:0/1911951398 <== mon.0 v2:172.21.15.37:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v47) ==== 74+0+17009 (secure 0 0 0) 0x7f7ba80cbb90 con 0x7f7ba80b7b00 2024-04-25T10:30:59.226 INFO:teuthology.orchestra.run.smithi037.stdout: 2024-04-25T10:30:59.226 INFO:teuthology.orchestra.run.smithi037.stdout:{"epoch":47,"fsid":"f4acda28-be34-492a-b286-994afd36f61d","created":"2024-04-25T10:28:43.985718+0000","modified":"2024-04-25T10:30:55.827846+0000","last_up_change":"2024-04-25T10:28:54.900654+0000","last_in_change":"2024-04-25T10:28:46.216973+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":6,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"luminous","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-04-25T10:28:56.202153+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"19","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":5,"pool_name":"cephfs_data","create_time":"2024-04-25T10:30:11.084827+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"43","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"recovery_fs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.3799999952316284,"score_stable":1.3799999952316284,"optimal_score":1,"raw_score_acting":1.3799999952316284,"raw_score_stable":1.3799999952316284,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":6,"pool_name":"recovery_fs_metadata","create_time":"2024-04-25T10:30:26.903737+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"44","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"recovery_fs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"a2f79028-b332-4166-9ebb-a7024a37ea22","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":42,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.37:6816","nonce":2746074720},{"type":"v1","addr":"172.21.15.37:6817","nonce":2746074720}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.37:6818","nonce":2746074720},{"type":"v1","addr":"172.21.15.37:6819","nonce":2746074720}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.37:6822","nonce":2746074720},{"type":"v1","addr":"172.21.15.37:6823","nonce":2746074720}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.37:6820","nonce":2746074720},{"type":"v1","addr":"172.21.15.37:6821","nonce":2746074720}]},"public_addr":"172.21.15.37:6817/2746074720","cluster_addr":"172.21.15.37:6819/2746074720","heartbeat_back_addr":"172.21.15.37:6823/2746074720","heartbeat_front_addr":"172.21.15.37:6821/2746074720","state":["exists","up"]},{"osd":1,"uuid":"973159de-c2ea-4083-aecc-69373d1c1132","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":45,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.37:6800","nonce":1690341662},{"type":"v1","addr":"172.21.15.37:6801","nonce":1690341662}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.37:6802","nonce":1690341662},{"type":"v1","addr":"172.21.15.37:6803","nonce":1690341662}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.37:6806","nonce":1690341662},{"type":"v1","addr":"172.21.15.37:6807","nonce":1690341662}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.37:6804","nonce":1690341662},{"type":"v1","addr":"172.21.15.37:6805","nonce":1690341662}]},"public_addr":"172.21.15.37:6801/1690341662","cluster_addr":"172.21.15.37:6803/1690341662","heartbeat_back_addr":"172.21.15.37:6807/1690341662","heartbeat_front_addr":"172.21.15.37:6805/1690341662","state":["exists","up"]},{"osd":2,"uuid":"4ae7a0e9-cb3f-484c-a75f-f3766b721c47","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":42,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.37:6808","nonce":2754669649},{"type":"v1","addr":"172.21.15.37:6809","nonce":2754669649}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.37:6810","nonce":2754669649},{"type":"v1","addr":"172.21.15.37:6811","nonce":2754669649}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.37:6814","nonce":2754669649},{"type":"v1","addr":"172.21.15.37:6815","nonce":2754669649}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.37:6812","nonce":2754669649},{"type":"v1","addr":"172.21.15.37:6813","nonce":2754669649}]},"public_addr":"172.21.15.37:6809/2754669649","cluster_addr":"172.21.15.37:6811/2754669649","heartbeat_back_addr":"172.21.15.37:6815/2754669649","heartbeat_front_addr":"172.21.15.37:6813/2754669649","state":["exists","up"]},{"osd":3,"uuid":"1ba4a746-1608-47a7-ace2-960a5cfae934","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":45,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.37:6824","nonce":1815290618},{"type":"v1","addr":"172.21.15.37:6825","nonce":1815290618}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.37:6826","nonce":1815290618},{"type":"v1","addr":"172.21.15.37:6827","nonce":1815290618}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.37:6830","nonce":1815290618},{"type":"v1","addr":"172.21.15.37:6831","nonce":1815290618}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.37:6828","nonce":1815290618},{"type":"v1","addr":"172.21.15.37:6829","nonce":1815290618}]},"public_addr":"172.21.15.37:6825/1815290618","cluster_addr":"172.21.15.37:6827/1815290618","heartbeat_back_addr":"172.21.15.37:6831/1815290618","heartbeat_front_addr":"172.21.15.37:6829/1815290618","state":["exists","up"]},{"osd":4,"uuid":"d4f59f5d-0413-4b79-87ee-bb212c1fcf62","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":42,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6800","nonce":4051532063},{"type":"v1","addr":"172.21.15.203:6801","nonce":4051532063}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6802","nonce":4051532063},{"type":"v1","addr":"172.21.15.203:6803","nonce":4051532063}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6806","nonce":4051532063},{"type":"v1","addr":"172.21.15.203:6807","nonce":4051532063}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6804","nonce":4051532063},{"type":"v1","addr":"172.21.15.203:6805","nonce":4051532063}]},"public_addr":"172.21.15.203:6801/4051532063","cluster_addr":"172.21.15.203:6803/4051532063","heartbeat_back_addr":"172.21.15.203:6807/4051532063","heartbeat_front_addr":"172.21.15.203:6805/4051532063","state":["exists","up"]},{"osd":5,"uuid":"69710a0e-41d8-4e7f-9a10-17ce540b67be","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":45,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6824","nonce":1203822890},{"type":"v1","addr":"172.21.15.203:6825","nonce":1203822890}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6826","nonce":1203822890},{"type":"v1","addr":"172.21.15.203:6827","nonce":1203822890}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6830","nonce":1203822890},{"type":"v1","addr":"172.21.15.203:6831","nonce":1203822890}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6828","nonce":1203822890},{"type":"v1","addr":"172.21.15.203:6829","nonce":1203822890}]},"public_addr":"172.21.15.203:6825/1203822890","cluster_addr":"172.21.15.203:6827/1203822890","heartbeat_back_addr":"172.21.15.203:6831/1203822890","heartbeat_front_addr":"172.21.15.203:6829/1203822890","state":["exists","up"]},{"osd":6,"uuid":"f29dd047-3b48-473d-a235-6642a6293f20","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":42,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6810","nonce":2309615014},{"type":"v1","addr":"172.21.15.203:6812","nonce":2309615014}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6814","nonce":2309615014},{"type":"v1","addr":"172.21.15.203:6816","nonce":2309615014}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6822","nonce":2309615014},{"type":"v1","addr":"172.21.15.203:6823","nonce":2309615014}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6818","nonce":2309615014},{"type":"v1","addr":"172.21.15.203:6820","nonce":2309615014}]},"public_addr":"172.21.15.203:6812/2309615014","cluster_addr":"172.21.15.203:6816/2309615014","heartbeat_back_addr":"172.21.15.203:6823/2309615014","heartbeat_front_addr":"172.21.15.203:6820/2309615014","state":["exists","up"]},{"osd":7,"uuid":"6e691644-378b-4fdc-8e86-fb5f56e31f29","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":42,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6808","nonce":3688099014},{"type":"v1","addr":"172.21.15.203:6809","nonce":3688099014}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6811","nonce":3688099014},{"type":"v1","addr":"172.21.15.203:6813","nonce":3688099014}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6819","nonce":3688099014},{"type":"v1","addr":"172.21.15.203:6821","nonce":3688099014}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6815","nonce":3688099014},{"type":"v1","addr":"172.21.15.203:6817","nonce":3688099014}]},"public_addr":"172.21.15.203:6809/3688099014","cluster_addr":"172.21.15.203:6813/3688099014","heartbeat_back_addr":"172.21.15.203:6821/3688099014","heartbeat_front_addr":"172.21.15.203:6817/3688099014","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T10:28:51.419141+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T10:28:51.366448+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T10:28:51.348276+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T10:28:51.380519+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T10:28:51.105812+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T10:28:51.384496+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T10:28:51.388755+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T10:28:51.366958+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[{"pgid":"5.15","mappings":[{"from":4,"to":5}]},{"pgid":"5.1a","mappings":[{"from":4,"to":5}]},{"pgid":"5.26","mappings":[{"from":4,"to":5}]},{"pgid":"5.27","mappings":[{"from":4,"to":5}]}],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.37:6835/311552410":"2024-04-26T10:30:24.853732+0000","172.21.15.37:6832/311552410":"2024-04-26T10:30:24.853732+0000"},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-04-25T10:30:59.229 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:59.226+0000 7f7baf2af640 1 -- 172.21.15.37:0/1911951398 >> [v2:172.21.15.203:6832/2418205237,v1:172.21.15.203:6833/2418205237] conn(0x7f7b940744e0 msgr2=0x7f7b940769f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:30:59.229 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:59.226+0000 7f7baf2af640 1 --2- 172.21.15.37:0/1911951398 >> [v2:172.21.15.203:6832/2418205237,v1:172.21.15.203:6833/2418205237] conn(0x7f7b940744e0 0x7f7b940769f0 secure :-1 s=READY pgs=308 cs=0 l=1 rev1=1 crypto rx=0x7f7b9c007fd0 tx=0x7f7b9c009040 comp rx=0 tx=0).stop 2024-04-25T10:30:59.229 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:59.226+0000 7f7baf2af640 1 -- 172.21.15.37:0/1911951398 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f7ba80b7b00 msgr2=0x7f7ba80cb650 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:30:59.229 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:59.226+0000 7f7baf2af640 1 --2- 172.21.15.37:0/1911951398 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f7ba80b7b00 0x7f7ba80cb650 secure :-1 s=READY pgs=682 cs=0 l=1 rev1=1 crypto rx=0x7f7ba0000c00 tx=0x7f7ba00046c0 comp rx=0 tx=0).stop 2024-04-25T10:30:59.229 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:59.226+0000 7f7baf2af640 1 -- 172.21.15.37:0/1911951398 shutdown_connections 2024-04-25T10:30:59.229 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:59.226+0000 7f7baf2af640 1 --2- 172.21.15.37:0/1911951398 >> [v2:172.21.15.203:6832/2418205237,v1:172.21.15.203:6833/2418205237] conn(0x7f7b940744e0 0x7f7b940769f0 unknown :-1 s=CLOSED pgs=308 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:30:59.229 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:59.226+0000 7f7baf2af640 1 --2- 172.21.15.37:0/1911951398 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f7ba8161110 0x7f7ba80d5b20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:30:59.229 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:59.226+0000 7f7baf2af640 1 --2- 172.21.15.37:0/1911951398 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f7ba80d5430 0x7f7ba80cc980 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:30:59.229 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:59.226+0000 7f7baf2af640 1 --2- 172.21.15.37:0/1911951398 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f7ba80b7b00 0x7f7ba80cb650 unknown :-1 s=CLOSED pgs=682 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:30:59.230 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:59.226+0000 7f7baf2af640 1 -- 172.21.15.37:0/1911951398 >> 172.21.15.37:0/1911951398 conn(0x7f7ba80d27a0 msgr2=0x7f7ba815ea10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:30:59.230 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:59.226+0000 7f7baf2af640 1 -- 172.21.15.37:0/1911951398 shutdown_connections 2024-04-25T10:30:59.230 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:59.227+0000 7f7baf2af640 1 -- 172.21.15.37:0/1911951398 wait complete. 2024-04-25T10:30:59.240 DEBUG:teuthology.orchestra.run.smithi037:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs fail recovery_fs 2024-04-25T10:30:59.352 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:59.350+0000 7f1be778f640 1 Processor -- start 2024-04-25T10:30:59.353 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:59.350+0000 7f1be778f640 1 -- start start 2024-04-25T10:30:59.355 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:59.352+0000 7f1be778f640 1 --2- >> v2:172.21.15.203:3300/0 conn(0x7f1be00b5fd0 0x7f1be00b2ca0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:30:59.355 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:59.352+0000 7f1be778f640 1 --2- >> v2:172.21.15.37:3300/0 conn(0x7f1be008cc90 0x7f1be008d0a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:30:59.355 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:59.352+0000 7f1be778f640 1 --2- >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f1be008d5e0 0x7f1be00d18c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:30:59.355 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:59.352+0000 7f1be778f640 1 -- --> v2:172.21.15.37:3300/0 -- mon_getmap magic: 0 -- 0x7f1be00cf120 con 0x7f1be008cc90 2024-04-25T10:30:59.355 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:59.352+0000 7f1be778f640 1 -- --> v2:172.21.15.203:3300/0 -- mon_getmap magic: 0 -- 0x7f1be00cf400 con 0x7f1be00b5fd0 2024-04-25T10:30:59.356 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:59.352+0000 7f1be778f640 1 -- --> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] -- mon_getmap magic: 0 -- 0x7f1be00cf6e0 con 0x7f1be008d5e0 2024-04-25T10:30:59.356 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:59.352+0000 7f1be4d2a640 1 --2- >> v2:172.21.15.37:3300/0 conn(0x7f1be008cc90 0x7f1be008d0a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:30:59.356 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:59.352+0000 7f1be4d2a640 1 --2- >> v2:172.21.15.37:3300/0 conn(0x7f1be008cc90 0x7f1be008d0a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.37:3300/0 says I am v2:172.21.15.37:59750/0 (socket says 172.21.15.37:59750) 2024-04-25T10:30:59.356 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:59.352+0000 7f1be4d2a640 1 -- 172.21.15.37:0/4020297477 learned_addr learned my addr 172.21.15.37:0/4020297477 (peer_addr_for_me v2:172.21.15.37:0/0) 2024-04-25T10:30:59.356 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:59.352+0000 7f1be552b640 1 --2- 172.21.15.37:0/4020297477 >> v2:172.21.15.203:3300/0 conn(0x7f1be00b5fd0 0x7f1be00b2ca0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:30:59.356 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:59.352+0000 7f1be5d2c640 1 --2- 172.21.15.37:0/4020297477 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f1be008d5e0 0x7f1be00d18c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:30:59.356 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:59.352+0000 7f1be4d2a640 1 -- 172.21.15.37:0/4020297477 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f1be008d5e0 msgr2=0x7f1be00d18c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:30:59.356 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:59.352+0000 7f1be4d2a640 1 --2- 172.21.15.37:0/4020297477 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f1be008d5e0 0x7f1be00d18c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:30:59.356 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:59.352+0000 7f1be4d2a640 1 -- 172.21.15.37:0/4020297477 >> v2:172.21.15.203:3300/0 conn(0x7f1be00b5fd0 msgr2=0x7f1be00b2ca0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:30:59.356 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:59.352+0000 7f1be4d2a640 1 --2- 172.21.15.37:0/4020297477 >> v2:172.21.15.203:3300/0 conn(0x7f1be00b5fd0 0x7f1be00b2ca0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:30:59.356 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:59.352+0000 7f1be4d2a640 1 -- 172.21.15.37:0/4020297477 --> v2:172.21.15.37:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f1be00cf9c0 con 0x7f1be008cc90 2024-04-25T10:30:59.356 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:59.352+0000 7f1be552b640 1 --2- 172.21.15.37:0/4020297477 >> v2:172.21.15.203:3300/0 conn(0x7f1be00b5fd0 0x7f1be00b2ca0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T10:30:59.356 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:59.353+0000 7f1be4d2a640 1 --2- 172.21.15.37:0/4020297477 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f1be008cc90 0x7f1be008d0a0 secure :-1 s=READY pgs=683 cs=0 l=1 rev1=1 crypto rx=0x7f1be0070b40 tx=0x7f1bd402a140 comp rx=0 tx=0).ready entity=mon.0 client_cookie=329004dd501f77b1 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:30:59.356 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:59.353+0000 7f1bdffff640 1 -- 172.21.15.37:0/4020297477 <== mon.0 v2:172.21.15.37:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1bd4037070 con 0x7f1be008cc90 2024-04-25T10:30:59.357 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:59.353+0000 7f1bdffff640 1 -- 172.21.15.37:0/4020297477 <== mon.0 v2:172.21.15.37:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f1bd4032440 con 0x7f1be008cc90 2024-04-25T10:30:59.357 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:59.353+0000 7f1bdffff640 1 -- 172.21.15.37:0/4020297477 <== mon.0 v2:172.21.15.37:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1bd4032810 con 0x7f1be008cc90 2024-04-25T10:30:59.361 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:59.354+0000 7f1be778f640 1 -- 172.21.15.37:0/4020297477 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f1be008cc90 msgr2=0x7f1be008d0a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:30:59.361 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:59.354+0000 7f1be778f640 1 --2- 172.21.15.37:0/4020297477 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f1be008cc90 0x7f1be008d0a0 secure :-1 s=READY pgs=683 cs=0 l=1 rev1=1 crypto rx=0x7f1be0070b40 tx=0x7f1bd402a140 comp rx=0 tx=0).stop 2024-04-25T10:30:59.361 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:59.354+0000 7f1be778f640 1 -- 172.21.15.37:0/4020297477 shutdown_connections 2024-04-25T10:30:59.361 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:59.354+0000 7f1be778f640 1 --2- 172.21.15.37:0/4020297477 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f1be008cc90 0x7f1be008d0a0 unknown :-1 s=CLOSED pgs=683 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:30:59.361 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:59.354+0000 7f1be778f640 1 --2- 172.21.15.37:0/4020297477 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f1be008d5e0 0x7f1be00d18c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:30:59.361 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:59.354+0000 7f1be778f640 1 --2- 172.21.15.37:0/4020297477 >> v2:172.21.15.203:3300/0 conn(0x7f1be00b5fd0 0x7f1be00b2ca0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:30:59.361 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:59.354+0000 7f1be778f640 1 -- 172.21.15.37:0/4020297477 >> 172.21.15.37:0/4020297477 conn(0x7f1be00b5440 msgr2=0x7f1be00b5850 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:30:59.361 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:59.354+0000 7f1be778f640 1 -- 172.21.15.37:0/4020297477 shutdown_connections 2024-04-25T10:30:59.361 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:59.354+0000 7f1be778f640 1 -- 172.21.15.37:0/4020297477 wait complete. 2024-04-25T10:30:59.361 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:59.356+0000 7f1be778f640 1 Processor -- start 2024-04-25T10:30:59.361 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:59.356+0000 7f1be778f640 1 -- start start 2024-04-25T10:30:59.361 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:59.357+0000 7f1be778f640 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f1be008cc90 0x7f1be015d8a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:30:59.362 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:59.357+0000 7f1be778f640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f1be008d5e0 0x7f1be015cb30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:30:59.362 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:59.357+0000 7f1be778f640 1 --2- >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f1be00b5fd0 0x7f1be009d650 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:30:59.362 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:59.357+0000 7f1be778f640 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 -- 0x7f1be00d1f80 con 0x7f1be008d5e0 2024-04-25T10:30:59.362 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:59.357+0000 7f1be778f640 1 -- --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_getmap magic: 0 -- 0x7f1be00d1e00 con 0x7f1be008cc90 2024-04-25T10:30:59.362 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:59.357+0000 7f1be778f640 1 -- --> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] -- mon_getmap magic: 0 -- 0x7f1be00d2100 con 0x7f1be00b5fd0 2024-04-25T10:30:59.362 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:59.357+0000 7f1be4d2a640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f1be008d5e0 0x7f1be015cb30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:30:59.362 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:59.357+0000 7f1be4d2a640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f1be008d5e0 0x7f1be015cb30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.37:3300/0 says I am v2:172.21.15.37:59752/0 (socket says 172.21.15.37:59752) 2024-04-25T10:30:59.362 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:59.357+0000 7f1be4d2a640 1 -- 172.21.15.37:0/2494620569 learned_addr learned my addr 172.21.15.37:0/2494620569 (peer_addr_for_me v2:172.21.15.37:0/0) 2024-04-25T10:30:59.362 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:59.357+0000 7f1be5d2c640 1 --2- 172.21.15.37:0/2494620569 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f1be00b5fd0 0x7f1be009d650 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:30:59.362 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:59.357+0000 7f1be552b640 1 --2- 172.21.15.37:0/2494620569 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f1be008cc90 0x7f1be015d8a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:30:59.362 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:59.358+0000 7f1be4d2a640 1 -- 172.21.15.37:0/2494620569 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f1be00b5fd0 msgr2=0x7f1be009d650 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:30:59.362 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:59.358+0000 7f1be4d2a640 1 --2- 172.21.15.37:0/2494620569 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f1be00b5fd0 0x7f1be009d650 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:30:59.362 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:59.358+0000 7f1be4d2a640 1 -- 172.21.15.37:0/2494620569 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f1be008cc90 msgr2=0x7f1be015d8a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:30:59.362 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:59.358+0000 7f1be4d2a640 1 --2- 172.21.15.37:0/2494620569 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f1be008cc90 0x7f1be015d8a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:30:59.363 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:59.358+0000 7f1be4d2a640 1 -- 172.21.15.37:0/2494620569 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f1be015d660 con 0x7f1be008d5e0 2024-04-25T10:30:59.363 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:59.358+0000 7f1be552b640 1 --2- 172.21.15.37:0/2494620569 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f1be008cc90 0x7f1be015d8a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T10:30:59.363 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:59.358+0000 7f1be4d2a640 1 --2- 172.21.15.37:0/2494620569 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f1be008d5e0 0x7f1be015cb30 secure :-1 s=READY pgs=684 cs=0 l=1 rev1=1 crypto rx=0x7f1bd4000c00 tx=0x7f1bd40046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:30:59.363 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:59.358+0000 7f1bddffb640 1 -- 172.21.15.37:0/2494620569 <== mon.0 v2:172.21.15.37:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1bd4037040 con 0x7f1be008d5e0 2024-04-25T10:30:59.363 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:59.358+0000 7f1bddffb640 1 -- 172.21.15.37:0/2494620569 <== mon.0 v2:172.21.15.37:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f1bd4042070 con 0x7f1be008d5e0 2024-04-25T10:30:59.363 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:59.358+0000 7f1be778f640 1 -- 172.21.15.37:0/2494620569 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f1be01603e0 con 0x7f1be008d5e0 2024-04-25T10:30:59.363 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:59.358+0000 7f1bddffb640 1 -- 172.21.15.37:0/2494620569 <== mon.0 v2:172.21.15.37:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1bd4030bb0 con 0x7f1be008d5e0 2024-04-25T10:30:59.363 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:59.358+0000 7f1be778f640 1 -- 172.21.15.37:0/2494620569 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f1be00a0480 con 0x7f1be008d5e0 2024-04-25T10:30:59.364 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:59.359+0000 7f1be778f640 1 -- 172.21.15.37:0/2494620569 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f1be006a140 con 0x7f1be008d5e0 2024-04-25T10:30:59.364 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:59.360+0000 7f1bddffb640 1 -- 172.21.15.37:0/2494620569 <== mon.0 v2:172.21.15.37:3300/0 4 ==== mgrmap(e 5) ==== 96782+0+0 (secure 0 0 0) 0x7f1bd4047460 con 0x7f1be008d5e0 2024-04-25T10:30:59.364 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:59.360+0000 7f1bddffb640 1 --2- 172.21.15.37:0/2494620569 >> [v2:172.21.15.203:6832/2418205237,v1:172.21.15.203:6833/2418205237] conn(0x7f1bcc0744b0 0x7f1bcc0769c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:30:59.364 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:59.360+0000 7f1bddffb640 1 -- 172.21.15.37:0/2494620569 <== mon.0 v2:172.21.15.37:3300/0 5 ==== osd_map(47..47 src has 1..47) ==== 5926+0+0 (secure 0 0 0) 0x7f1bd40b5f80 con 0x7f1be008d5e0 2024-04-25T10:30:59.364 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:59.361+0000 7f1be552b640 1 --2- 172.21.15.37:0/2494620569 >> [v2:172.21.15.203:6832/2418205237,v1:172.21.15.203:6833/2418205237] conn(0x7f1bcc0744b0 0x7f1bcc0769c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:30:59.364 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:59.361+0000 7f1be552b640 1 --2- 172.21.15.37:0/2494620569 >> [v2:172.21.15.203:6832/2418205237,v1:172.21.15.203:6833/2418205237] conn(0x7f1bcc0744b0 0x7f1bcc0769c0 secure :-1 s=READY pgs=309 cs=0 l=1 rev1=1 crypto rx=0x7f1be0073c80 tx=0x7f1bd0007a80 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:30:59.367 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:59.365+0000 7f1bddffb640 1 -- 172.21.15.37:0/2494620569 <== mon.0 v2:172.21.15.37:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187246 (secure 0 0 0) 0x7f1bd4080620 con 0x7f1be008d5e0 2024-04-25T10:30:59.588 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:59.586+0000 7f1be778f640 1 -- 172.21.15.37:0/2494620569 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_command({"prefix": "fs fail", "fs_name": "recovery_fs"} v 0) -- 0x7f1be0160170 con 0x7f1be008d5e0 2024-04-25T10:30:59.589 INFO:tasks.ceph.mon.a.smithi037.stderr:2024-04-25T10:30:59.587+0000 7fb25155b640 -1 log_channel(cluster) log [ERR] : Health check failed: 1 filesystem is offline (MDS_ALL_DOWN) 2024-04-25T10:30:59.590 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:59.588+0000 7f1bddffb640 1 -- 172.21.15.37:0/2494620569 <== mon.0 v2:172.21.15.37:3300/0 7 ==== mon_command_ack([{"prefix": "fs fail", "fs_name": "recovery_fs"}]=0 recovery_fs marked not joinable; MDS cannot join the cluster. All MDS ranks marked failed. v30) ==== 171+0+0 (secure 0 0 0) 0x7f1bd407ff20 con 0x7f1be008d5e0 2024-04-25T10:30:59.590 INFO:teuthology.orchestra.run.smithi037.stderr:recovery_fs marked not joinable; MDS cannot join the cluster. All MDS ranks marked failed. 2024-04-25T10:30:59.594 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:59.591+0000 7f1be778f640 1 -- 172.21.15.37:0/2494620569 >> [v2:172.21.15.203:6832/2418205237,v1:172.21.15.203:6833/2418205237] conn(0x7f1bcc0744b0 msgr2=0x7f1bcc0769c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:30:59.594 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:59.591+0000 7f1be778f640 1 --2- 172.21.15.37:0/2494620569 >> [v2:172.21.15.203:6832/2418205237,v1:172.21.15.203:6833/2418205237] conn(0x7f1bcc0744b0 0x7f1bcc0769c0 secure :-1 s=READY pgs=309 cs=0 l=1 rev1=1 crypto rx=0x7f1be0073c80 tx=0x7f1bd0007a80 comp rx=0 tx=0).stop 2024-04-25T10:30:59.594 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:59.591+0000 7f1be778f640 1 -- 172.21.15.37:0/2494620569 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f1be008d5e0 msgr2=0x7f1be015cb30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:30:59.594 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:59.591+0000 7f1be778f640 1 --2- 172.21.15.37:0/2494620569 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f1be008d5e0 0x7f1be015cb30 secure :-1 s=READY pgs=684 cs=0 l=1 rev1=1 crypto rx=0x7f1bd4000c00 tx=0x7f1bd40046c0 comp rx=0 tx=0).stop 2024-04-25T10:30:59.594 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:59.591+0000 7f1be778f640 1 -- 172.21.15.37:0/2494620569 shutdown_connections 2024-04-25T10:30:59.594 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:59.591+0000 7f1be778f640 1 --2- 172.21.15.37:0/2494620569 >> [v2:172.21.15.203:6832/2418205237,v1:172.21.15.203:6833/2418205237] conn(0x7f1bcc0744b0 0x7f1bcc0769c0 unknown :-1 s=CLOSED pgs=309 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:30:59.594 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:59.591+0000 7f1be778f640 1 --2- 172.21.15.37:0/2494620569 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f1be00b5fd0 0x7f1be009d650 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:30:59.594 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:59.591+0000 7f1be778f640 1 --2- 172.21.15.37:0/2494620569 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f1be008d5e0 0x7f1be015cb30 unknown :-1 s=CLOSED pgs=684 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:30:59.594 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:59.591+0000 7f1be778f640 1 --2- 172.21.15.37:0/2494620569 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f1be008cc90 0x7f1be015d8a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:30:59.594 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:59.591+0000 7f1be778f640 1 -- 172.21.15.37:0/2494620569 >> 172.21.15.37:0/2494620569 conn(0x7f1be00b5440 msgr2=0x7f1be0072960 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:30:59.595 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:59.591+0000 7f1be778f640 1 -- 172.21.15.37:0/2494620569 shutdown_connections 2024-04-25T10:30:59.595 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:59.591+0000 7f1be778f640 1 -- 172.21.15.37:0/2494620569 wait complete. 2024-04-25T10:30:59.595 INFO:tasks.ceph.mds.c.smithi037.stderr: -3716> 2024-04-25T10:30:39.925+0000 7f12be3c9640 -1 mds.0.openfiles _load_finish got (2) No such file or directory 2024-04-25T10:30:59.597 INFO:tasks.ceph.mds.c.smithi037.stderr: -2058> 2024-04-25T10:30:44.959+0000 7f12be3c9640 -1 log_channel(cluster) log [ERR] : bad backtrace on directory inode 0x10000000000 2024-04-25T10:30:59.597 INFO:tasks.ceph.mds.c.smithi037.stderr: -1911> 2024-04-25T10:30:44.960+0000 7f12be3c9640 -1 mds.0.scrubstack _validate_inode_done scrub error on inode [inode 0x10000000000 [...2,head] /subdir/ auth v1 pv4 ap=3 DIRTYPARENT f(v0 1=1+0) n() (inest lock dirty) (ifile lock->sync w=1 flushing) | dirtyscattered=2 lock=1 dirfrag=1 dirtyparent=1 waiter=1 authpin=1 scrubqueue=0 0x5603dd0f7180]: {"performed_validation":true,"passed_validation":false,"backtrace":{"checked":true,"passed":true,"read_ret_val":-61,"ondisk_value":"(-1)0x0:[]//[]","memoryvalue":"(6)0x10000000000:[<0x1/subdir v1>]//[]","error_str":"failed to read off disk; see retval"},"raw_stats":{"checked":true,"passed":false,"read_ret_val":0,"ondisk_value.dirstat":"f(v0 1=1+0)","ondisk_value.rstat":"n(v0 1=0+1)","memory_value.dirstat":"f(v0 1=1+0)","memory_value.rstat":"n()","error_str":"freshly-calculated rstats don't match existing ones (will be fixed)"},"return_code":-61} 2024-04-25T10:30:59.597 INFO:tasks.ceph.mds.c.smithi037.stderr: -1834> 2024-04-25T10:30:44.961+0000 7f12be3c9640 -1 mds.0.scrubstack _validate_inode_done scrub error on inode [inode 0x10000000001 [2,head] /subdir/sixmegs auth v1 ap=1 DIRTYPARENT s=6291456 n() | dirtyparent=1 authpin=1 scrubqueue=0 0x5603dd0f5080]: {"performed_validation":true,"passed_validation":false,"backtrace":{"checked":true,"passed":false,"read_ret_val":0,"ondisk_value":"(5)0x10000000001:[<0x10000000000/sixmegs v8>,<0x1/subdir v10>]//[]","memoryvalue":"(5)0x10000000001:[<0x10000000000/sixmegs v1>,<0x1/subdir v1>]//[]","error_str":"On-disk backtrace is divergent or newer"},"raw_stats":{"checked":false,"passed":false,"read_ret_val":0,"ondisk_value.dirstat":"f()","ondisk_value.rstat":"n()","memory_value.dirstat":"f()","memory_value.rstat":"n()","error_str":""},"return_code":0} 2024-04-25T10:30:59.598 INFO:tasks.ceph.mds.c.smithi037.stderr: -1492> 2024-04-25T10:30:44.968+0000 7f12be3c9640 -1 mds.0.scrubstack _validate_inode_done scrub error on inode [inode 0x1 [...2,head] / auth v4 pv6 ap=5 snaprealm=0x5603dd0e8900 DIRTYPARENT f(v1 1=0+1) n(v0 1=0+1)/n() (inest mix->lock(2) w=1 flushing) (ifile mix->sync w=1 flushing) | dirtyscattered=2 lock=2 dirfrag=1 openingsnapparents=0 dirtyparent=1 dirty=1 waiter=1 authpin=1 scrubqueue=0 0x5603dd0f4b00]: {"performed_validation":true,"passed_validation":false,"backtrace":{"checked":true,"passed":false,"read_ret_val":-61,"ondisk_value":"(-1)0x0:[]//[]","memoryvalue":"(6)0x1:[]//[]","error_str":"failed to read off disk; see retval"},"raw_stats":{"checked":true,"passed":true,"read_ret_val":0,"ondisk_value.dirstat":"f(v0 1=0+1)","ondisk_value.rstat":"n(v0 2=0+2)","memory_value.dirstat":"f(v1 1=0+1)","memory_value.rstat":"n(v0 1=0+1)","error_str":"freshly-calculated rstats don't match existing ones (will be fixed)"},"return_code":-61} 2024-04-25T10:30:59.598 INFO:tasks.ceph.mds.c.smithi037.stderr: -1342> 2024-04-25T10:30:45.865+0000 7f12c23d1640 -1 mds.pinger is_rank_lagging: rank=0 was never sent ping request. 2024-04-25T10:30:59.606 DEBUG:teuthology.orchestra.run.smithi037:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs rm recovery_fs --yes-i-really-mean-it 2024-04-25T10:30:59.651 INFO:tasks.ceph.mds.c.smithi037.stdout:starting mds.c at 2024-04-25T10:30:59.722 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:59.720+0000 7f1ef4415640 1 Processor -- start 2024-04-25T10:30:59.722 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:59.720+0000 7f1ef4415640 1 -- start start 2024-04-25T10:30:59.727 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:59.721+0000 7f1ef4415640 1 --2- >> v2:172.21.15.203:3300/0 conn(0x7f1eec0b4e80 0x7f1eec0d2b20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:30:59.727 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:59.721+0000 7f1ef4415640 1 -- --> v1:172.21.15.37:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f1eec0d3470 con 0x7f1eec0d5160 2024-04-25T10:30:59.727 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:59.721+0000 7f1ef4415640 1 -- --> v1:172.21.15.203:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f1eec0d10a0 con 0x7f1eec0d3060 2024-04-25T10:30:59.727 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:59.721+0000 7f1ef4415640 1 -- --> v2:172.21.15.203:3300/0 -- mon_getmap magic: 0 -- 0x7f1eec0d1650 con 0x7f1eec0b4e80 2024-04-25T10:30:59.727 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:59.722+0000 7f1ef21b1640 1 --1- >> v1:172.21.15.37:6789/0 conn(0x7f1eec0d5160 0x7f1eec0d0990 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.37:6789/0 says I am v1:172.21.15.37:47040/0 (socket says 172.21.15.37:47040) 2024-04-25T10:30:59.727 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:59.722+0000 7f1ef21b1640 1 -- 172.21.15.37:0/3460414321 learned_addr learned my addr 172.21.15.37:0/3460414321 (peer_addr_for_me v1:172.21.15.37:0/0) 2024-04-25T10:30:59.727 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:59.722+0000 7f1ef19b0640 1 --2- 172.21.15.37:0/3460414321 >> v2:172.21.15.203:3300/0 conn(0x7f1eec0b4e80 0x7f1eec0d2b20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:30:59.728 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:59.722+0000 7f1ef11af640 1 -- 172.21.15.37:0/3460414321 <== mon.0 v1:172.21.15.37:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 3993316832 0 0) 0x7f1ee8001110 con 0x7f1eec0d5160 2024-04-25T10:30:59.728 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:59.722+0000 7f1ef11af640 1 -- 172.21.15.37:0/3460414321 --> v1:172.21.15.37:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f1ed8003520 con 0x7f1eec0d5160 2024-04-25T10:30:59.728 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:59.722+0000 7f1ef11af640 1 -- 172.21.15.37:0/3460414321 <== mon.0 v1:172.21.15.37:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 1620316110 0 0) 0x7f1ed8003520 con 0x7f1eec0d5160 2024-04-25T10:30:59.728 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:59.722+0000 7f1ef11af640 1 -- 172.21.15.37:0/3460414321 >> v2:172.21.15.203:3300/0 conn(0x7f1eec0b4e80 msgr2=0x7f1eec0d2b20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:30:59.728 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:59.722+0000 7f1ef11af640 1 --2- 172.21.15.37:0/3460414321 >> v2:172.21.15.203:3300/0 conn(0x7f1eec0b4e80 0x7f1eec0d2b20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:30:59.728 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:59.722+0000 7f1ef11af640 1 -- 172.21.15.37:0/3460414321 >> v1:172.21.15.203:6789/0 conn(0x7f1eec0d3060 legacy=0x7f1eec0b9060 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:30:59.728 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:59.722+0000 7f1ef11af640 1 -- 172.21.15.37:0/3460414321 --> v1:172.21.15.37:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f1eec0d1370 con 0x7f1eec0d5160 2024-04-25T10:30:59.728 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:59.722+0000 7f1ef11af640 1 -- 172.21.15.37:0/3460414321 <== mon.0 v1:172.21.15.37:6789/0 3 ==== mon_map magic: 0 ==== 413+0+0 (unknown 1577794600 0 0) 0x7f1ee8001840 con 0x7f1eec0d5160 2024-04-25T10:30:59.728 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:59.722+0000 7f1ef11af640 1 -- 172.21.15.37:0/3460414321 >> v1:172.21.15.37:6789/0 conn(0x7f1eec0d5160 legacy=0x7f1eec0d0990 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:30:59.728 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:59.722+0000 7f1ef11af640 1 --2- 172.21.15.37:0/3460414321 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f1ed80039b0 0x7f1ed8003e20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:30:59.728 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:59.722+0000 7f1ef11af640 1 --2- 172.21.15.37:0/3460414321 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f1ed8004800 0x7f1ed8004c10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:30:59.729 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:59.722+0000 7f1ef11af640 1 --2- 172.21.15.37:0/3460414321 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f1ed8005250 0x7f1ed800b9c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:30:59.729 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:59.722+0000 7f1ef11af640 1 -- 172.21.15.37:0/3460414321 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 -- 0x7f1eec0d1650 con 0x7f1ed8005250 2024-04-25T10:30:59.729 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:59.722+0000 7f1ef11af640 1 -- 172.21.15.37:0/3460414321 --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_getmap magic: 0 -- 0x7f1ed800c1e0 con 0x7f1ed8004800 2024-04-25T10:30:59.729 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:59.722+0000 7f1ef11af640 1 -- 172.21.15.37:0/3460414321 --> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] -- mon_getmap magic: 0 -- 0x7f1ed800c3e0 con 0x7f1ed80039b0 2024-04-25T10:30:59.729 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:59.722+0000 7f1ef29b2640 1 --2- 172.21.15.37:0/3460414321 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f1ed8005250 0x7f1ed800b9c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:30:59.729 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:59.722+0000 7f1ef21b1640 1 --2- 172.21.15.37:0/3460414321 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f1ed80039b0 0x7f1ed8003e20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:30:59.729 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:59.723+0000 7f1ef19b0640 1 --2- 172.21.15.37:0/3460414321 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f1ed8004800 0x7f1ed8004c10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:30:59.729 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:59.723+0000 7f1ef29b2640 1 -- 172.21.15.37:0/3460414321 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f1ed80039b0 msgr2=0x7f1ed8003e20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:30:59.729 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:59.723+0000 7f1ef29b2640 1 --2- 172.21.15.37:0/3460414321 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f1ed80039b0 0x7f1ed8003e20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:30:59.729 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:59.723+0000 7f1ef29b2640 1 -- 172.21.15.37:0/3460414321 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f1ed8004800 msgr2=0x7f1ed8004c10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:30:59.729 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:59.723+0000 7f1ef29b2640 1 --2- 172.21.15.37:0/3460414321 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f1ed8004800 0x7f1ed8004c10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:30:59.729 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:59.723+0000 7f1ef29b2640 1 -- 172.21.15.37:0/3460414321 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f1ed8003520 con 0x7f1ed8005250 2024-04-25T10:30:59.730 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:59.723+0000 7f1ef29b2640 1 -- 172.21.15.37:0/3460414321 reap_dead start 2024-04-25T10:30:59.730 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:59.724+0000 7f1ef29b2640 1 --2- 172.21.15.37:0/3460414321 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f1ed8005250 0x7f1ed800b9c0 secure :-1 s=READY pgs=688 cs=0 l=1 rev1=1 crypto rx=0x7f1ee4001e50 tx=0x7f1ed8005830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a13f00c7599ec11c server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:30:59.730 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:59.724+0000 7f1ef11af640 1 -- 172.21.15.37:0/3460414321 <== mon.0 v2:172.21.15.37:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1ee4001a70 con 0x7f1ed8005250 2024-04-25T10:30:59.730 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:59.724+0000 7f1ef11af640 1 -- 172.21.15.37:0/3460414321 <== mon.0 v2:172.21.15.37:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f1ee4003970 con 0x7f1ed8005250 2024-04-25T10:30:59.730 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:59.724+0000 7f1ef11af640 1 -- 172.21.15.37:0/3460414321 <== mon.0 v2:172.21.15.37:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1ee4003d60 con 0x7f1ed8005250 2024-04-25T10:30:59.730 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:59.724+0000 7f1ef4415640 1 -- 172.21.15.37:0/3460414321 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f1ed8005250 msgr2=0x7f1ed800b9c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:30:59.730 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:59.724+0000 7f1ef4415640 1 --2- 172.21.15.37:0/3460414321 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f1ed8005250 0x7f1ed800b9c0 secure :-1 s=READY pgs=688 cs=0 l=1 rev1=1 crypto rx=0x7f1ee4001e50 tx=0x7f1ed8005830 comp rx=0 tx=0).stop 2024-04-25T10:30:59.730 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:59.724+0000 7f1ef4415640 1 -- 172.21.15.37:0/3460414321 shutdown_connections 2024-04-25T10:30:59.730 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:59.724+0000 7f1ef4415640 1 --2- 172.21.15.37:0/3460414321 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f1ed8005250 0x7f1ed800b9c0 unknown :-1 s=CLOSED pgs=688 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:30:59.731 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:59.724+0000 7f1ef4415640 1 -- 172.21.15.37:0/3460414321 >> 172.21.15.37:0/3460414321 conn(0x7f1eec0b7a80 msgr2=0x7f1eec0b7e90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:30:59.731 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:59.725+0000 7f1ef4415640 1 -- 172.21.15.37:0/3460414321 shutdown_connections 2024-04-25T10:30:59.731 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:59.725+0000 7f1ef4415640 1 -- 172.21.15.37:0/3460414321 wait complete. 2024-04-25T10:30:59.734 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:59.726+0000 7f1ef4415640 1 Processor -- start 2024-04-25T10:30:59.734 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:59.726+0000 7f1ef4415640 1 -- start start 2024-04-25T10:30:59.734 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:59.728+0000 7f1ef4415640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f1ed8005250 0x7f1eec0d0380 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:30:59.734 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:59.728+0000 7f1ef4415640 1 --2- >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f1eec142de0 0x7f1eec1eb410 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:30:59.734 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:59.728+0000 7f1ef4415640 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f1eec161ee0 0x7f1eec1622f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:30:59.735 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:59.728+0000 7f1ef4415640 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 -- 0x7f1eec0d4960 con 0x7f1ed8005250 2024-04-25T10:30:59.735 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:59.728+0000 7f1ef4415640 1 -- --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_getmap magic: 0 -- 0x7f1eec0b5290 con 0x7f1eec161ee0 2024-04-25T10:30:59.735 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:59.728+0000 7f1ef4415640 1 -- --> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] -- mon_getmap magic: 0 -- 0x7f1eec0d4c40 con 0x7f1eec142de0 2024-04-25T10:30:59.735 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:59.728+0000 7f1ef21b1640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f1ed8005250 0x7f1eec0d0380 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:30:59.735 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:59.728+0000 7f1ef21b1640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f1ed8005250 0x7f1eec0d0380 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.37:3300/0 says I am v2:172.21.15.37:34456/0 (socket says 172.21.15.37:34456) 2024-04-25T10:30:59.735 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:59.728+0000 7f1ef21b1640 1 -- 172.21.15.37:0/4229701839 learned_addr learned my addr 172.21.15.37:0/4229701839 (peer_addr_for_me v2:172.21.15.37:0/0) 2024-04-25T10:30:59.735 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:59.728+0000 7f1ef19b0640 1 --2- 172.21.15.37:0/4229701839 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f1eec142de0 0x7f1eec1eb410 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:30:59.735 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:59.728+0000 7f1ef29b2640 1 --2- 172.21.15.37:0/4229701839 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f1eec161ee0 0x7f1eec1622f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:30:59.735 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:59.728+0000 7f1ef21b1640 1 -- 172.21.15.37:0/4229701839 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f1eec142de0 msgr2=0x7f1eec1eb410 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:30:59.735 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:59.728+0000 7f1ef21b1640 1 --2- 172.21.15.37:0/4229701839 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f1eec142de0 0x7f1eec1eb410 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:30:59.736 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:59.728+0000 7f1ef21b1640 1 -- 172.21.15.37:0/4229701839 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f1eec161ee0 msgr2=0x7f1eec1622f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:30:59.736 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:59.728+0000 7f1ef21b1640 1 --2- 172.21.15.37:0/4229701839 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f1eec161ee0 0x7f1eec1622f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:30:59.736 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:59.728+0000 7f1ef21b1640 1 -- 172.21.15.37:0/4229701839 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f1eec162830 con 0x7f1ed8005250 2024-04-25T10:30:59.736 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:59.728+0000 7f1ef21b1640 1 --2- 172.21.15.37:0/4229701839 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f1ed8005250 0x7f1eec0d0380 secure :-1 s=READY pgs=689 cs=0 l=1 rev1=1 crypto rx=0x7f1ee8007290 tx=0x7f1ee8007750 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:30:59.736 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:59.728+0000 7f1ee2ffd640 1 -- 172.21.15.37:0/4229701839 <== mon.0 v2:172.21.15.37:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1ee801a070 con 0x7f1ed8005250 2024-04-25T10:30:59.736 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:59.728+0000 7f1ee2ffd640 1 -- 172.21.15.37:0/4229701839 <== mon.0 v2:172.21.15.37:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f1ee800f9c0 con 0x7f1ed8005250 2024-04-25T10:30:59.736 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:59.728+0000 7f1ef4415640 1 -- 172.21.15.37:0/4229701839 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f1eec155e80 con 0x7f1ed8005250 2024-04-25T10:30:59.736 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:59.728+0000 7f1ee2ffd640 1 -- 172.21.15.37:0/4229701839 <== mon.0 v2:172.21.15.37:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1ee800fd90 con 0x7f1ed8005250 2024-04-25T10:30:59.736 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:59.728+0000 7f1ef4415640 1 -- 172.21.15.37:0/4229701839 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f1eec156440 con 0x7f1ed8005250 2024-04-25T10:30:59.736 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:59.729+0000 7f1ef4415640 1 -- 172.21.15.37:0/4229701839 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f1eec06a140 con 0x7f1ed8005250 2024-04-25T10:30:59.736 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:59.730+0000 7f1ee2ffd640 1 -- 172.21.15.37:0/4229701839 <== mon.0 v2:172.21.15.37:3300/0 4 ==== mgrmap(e 5) ==== 96782+0+0 (secure 0 0 0) 0x7f1ee801d070 con 0x7f1ed8005250 2024-04-25T10:30:59.736 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:59.731+0000 7f1ee2ffd640 1 --2- 172.21.15.37:0/4229701839 >> [v2:172.21.15.203:6832/2418205237,v1:172.21.15.203:6833/2418205237] conn(0x7f1ed00744b0 0x7f1ed00769c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:30:59.736 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:59.731+0000 7f1ee2ffd640 1 -- 172.21.15.37:0/4229701839 <== mon.0 v2:172.21.15.37:3300/0 5 ==== osd_map(48..48 src has 1..48) ==== 6012+0+0 (secure 0 0 0) 0x7f1ee8094de0 con 0x7f1ed8005250 2024-04-25T10:30:59.736 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:59.731+0000 7f1ef19b0640 1 --2- 172.21.15.37:0/4229701839 >> [v2:172.21.15.203:6832/2418205237,v1:172.21.15.203:6833/2418205237] conn(0x7f1ed00744b0 0x7f1ed00769c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:30:59.737 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:59.732+0000 7f1ef19b0640 1 --2- 172.21.15.37:0/4229701839 >> [v2:172.21.15.203:6832/2418205237,v1:172.21.15.203:6833/2418205237] conn(0x7f1ed00744b0 0x7f1ed00769c0 secure :-1 s=READY pgs=310 cs=0 l=1 rev1=1 crypto rx=0x7f1eec142cb0 tx=0x7f1edc008040 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:30:59.737 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:59.736+0000 7f1ee2ffd640 1 -- 172.21.15.37:0/4229701839 <== mon.0 v2:172.21.15.37:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187246 (secure 0 0 0) 0x7f1eec162830 con 0x7f1ed8005250 2024-04-25T10:30:59.965 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:30:59.963+0000 7f1ef4415640 1 -- 172.21.15.37:0/4229701839 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_command({"prefix": "fs rm", "fs_name": "recovery_fs", "yes_i_really_mean_it": true} v 0) -- 0x7f1eec0ba210 con 0x7f1ed8005250 2024-04-25T10:31:00.592 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:31:00.590+0000 7f1ee2ffd640 1 -- 172.21.15.37:0/4229701839 <== mon.0 v2:172.21.15.37:3300/0 7 ==== mon_command_ack([{"prefix": "fs rm", "fs_name": "recovery_fs", "yes_i_really_mean_it": true}]=0 v31) ==== 109+0+0 (secure 0 0 0) 0x7f1eec0d3470 con 0x7f1ed8005250 2024-04-25T10:31:00.595 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:31:00.593+0000 7f1ef4415640 1 -- 172.21.15.37:0/4229701839 >> [v2:172.21.15.203:6832/2418205237,v1:172.21.15.203:6833/2418205237] conn(0x7f1ed00744b0 msgr2=0x7f1ed00769c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:31:00.595 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:31:00.593+0000 7f1ef4415640 1 --2- 172.21.15.37:0/4229701839 >> [v2:172.21.15.203:6832/2418205237,v1:172.21.15.203:6833/2418205237] conn(0x7f1ed00744b0 0x7f1ed00769c0 secure :-1 s=READY pgs=310 cs=0 l=1 rev1=1 crypto rx=0x7f1eec142cb0 tx=0x7f1edc008040 comp rx=0 tx=0).stop 2024-04-25T10:31:00.595 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:31:00.593+0000 7f1ef4415640 1 -- 172.21.15.37:0/4229701839 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f1ed8005250 msgr2=0x7f1eec0d0380 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:31:00.595 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:31:00.593+0000 7f1ef4415640 1 --2- 172.21.15.37:0/4229701839 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f1ed8005250 0x7f1eec0d0380 secure :-1 s=READY pgs=689 cs=0 l=1 rev1=1 crypto rx=0x7f1ee8007290 tx=0x7f1ee8007750 comp rx=0 tx=0).stop 2024-04-25T10:31:00.596 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:31:00.593+0000 7f1ef4415640 1 -- 172.21.15.37:0/4229701839 shutdown_connections 2024-04-25T10:31:00.596 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:31:00.593+0000 7f1ef4415640 1 --2- 172.21.15.37:0/4229701839 >> [v2:172.21.15.203:6832/2418205237,v1:172.21.15.203:6833/2418205237] conn(0x7f1ed00744b0 0x7f1ed00769c0 unknown :-1 s=CLOSED pgs=310 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:31:00.596 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:31:00.593+0000 7f1ef4415640 1 --2- 172.21.15.37:0/4229701839 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f1eec161ee0 0x7f1eec1622f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:31:00.596 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:31:00.593+0000 7f1ef4415640 1 --2- 172.21.15.37:0/4229701839 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f1eec142de0 0x7f1eec1eb410 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:31:00.596 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:31:00.593+0000 7f1ef4415640 1 --2- 172.21.15.37:0/4229701839 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f1ed8005250 0x7f1eec0d0380 unknown :-1 s=CLOSED pgs=689 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:31:00.596 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:31:00.593+0000 7f1ef4415640 1 -- 172.21.15.37:0/4229701839 >> 172.21.15.37:0/4229701839 conn(0x7f1eec0b7a80 msgr2=0x7f1eec0bac40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:31:00.596 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:31:00.593+0000 7f1ef4415640 1 -- 172.21.15.37:0/4229701839 shutdown_connections 2024-04-25T10:31:00.596 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:31:00.593+0000 7f1ef4415640 1 -- 172.21.15.37:0/4229701839 wait complete. 2024-04-25T10:31:00.609 DEBUG:teuthology.orchestra.run.smithi037:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool rm recovery_fs_metadata recovery_fs_metadata --yes-i-really-really-mean-it 2024-04-25T10:31:00.730 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:31:00.728+0000 7f0216108640 1 Processor -- start 2024-04-25T10:31:00.731 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:31:00.728+0000 7f0216108640 1 -- start start 2024-04-25T10:31:00.732 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:31:00.729+0000 7f0216108640 1 --2- >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f0210160b30 0x7f02100cded0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:31:00.732 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:31:00.729+0000 7f0216108640 1 --2- >> v2:172.21.15.37:3300/0 conn(0x7f02100cf710 0x7f02100cfb20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:31:00.732 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:31:00.729+0000 7f0216108640 1 -- --> v1:172.21.15.203:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f02100d0060 con 0x7f0210070b40 2024-04-25T10:31:00.732 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:31:00.729+0000 7f0216108640 1 -- --> v2:172.21.15.37:3300/0 -- mon_getmap magic: 0 -- 0x7f02100b6bb0 con 0x7f02100cf710 2024-04-25T10:31:00.733 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:31:00.729+0000 7f0216108640 1 -- --> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] -- mon_getmap magic: 0 -- 0x7f02100b65f0 con 0x7f0210160b30 2024-04-25T10:31:00.733 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:31:00.730+0000 7f020ffff640 1 --2- >> v2:172.21.15.37:3300/0 conn(0x7f02100cf710 0x7f02100cfb20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:31:00.733 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:31:00.730+0000 7f020ffff640 1 --2- >> v2:172.21.15.37:3300/0 conn(0x7f02100cf710 0x7f02100cfb20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.37:3300/0 says I am v2:172.21.15.37:34458/0 (socket says 172.21.15.37:34458) 2024-04-25T10:31:00.733 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:31:00.730+0000 7f020ffff640 1 -- 172.21.15.37:0/399830625 learned_addr learned my addr 172.21.15.37:0/399830625 (peer_addr_for_me v2:172.21.15.37:0/0) 2024-04-25T10:31:00.733 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:31:00.730+0000 7f020f7fe640 1 --2- 172.21.15.37:0/399830625 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f0210160b30 0x7f02100cded0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:31:00.733 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:31:00.730+0000 7f020ffff640 1 -- 172.21.15.37:0/399830625 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f0210160b30 msgr2=0x7f02100cded0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:31:00.733 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:31:00.730+0000 7f020ffff640 1 --2- 172.21.15.37:0/399830625 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f0210160b30 0x7f02100cded0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:31:00.733 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:31:00.730+0000 7f020ffff640 1 -- 172.21.15.37:0/399830625 >> v1:172.21.15.203:6789/0 conn(0x7f0210070b40 legacy=0x7f021015d1f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:31:00.733 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:31:00.730+0000 7f020ffff640 1 -- 172.21.15.37:0/399830625 --> v2:172.21.15.37:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f02100b68d0 con 0x7f02100cf710 2024-04-25T10:31:00.734 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:31:00.731+0000 7f020ffff640 1 --2- 172.21.15.37:0/399830625 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f02100cf710 0x7f02100cfb20 secure :-1 s=READY pgs=690 cs=0 l=1 rev1=1 crypto rx=0x7f0208009e50 tx=0x7f020802a710 comp rx=0 tx=0).ready entity=mon.0 client_cookie=fd4b3d61382c78a3 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:31:00.735 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:31:00.731+0000 7f020e7fc640 1 -- 172.21.15.37:0/399830625 <== mon.0 v2:172.21.15.37:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0208036070 con 0x7f02100cf710 2024-04-25T10:31:00.735 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:31:00.731+0000 7f020e7fc640 1 -- 172.21.15.37:0/399830625 <== mon.0 v2:172.21.15.37:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f0208031600 con 0x7f02100cf710 2024-04-25T10:31:00.735 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:31:00.731+0000 7f020e7fc640 1 -- 172.21.15.37:0/399830625 <== mon.0 v2:172.21.15.37:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f02080319d0 con 0x7f02100cf710 2024-04-25T10:31:00.735 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:31:00.732+0000 7f0216108640 1 -- 172.21.15.37:0/399830625 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f02100cf710 msgr2=0x7f02100cfb20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:31:00.735 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:31:00.732+0000 7f0216108640 1 --2- 172.21.15.37:0/399830625 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f02100cf710 0x7f02100cfb20 secure :-1 s=READY pgs=690 cs=0 l=1 rev1=1 crypto rx=0x7f0208009e50 tx=0x7f020802a710 comp rx=0 tx=0).stop 2024-04-25T10:31:00.735 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:31:00.732+0000 7f0216108640 1 -- 172.21.15.37:0/399830625 shutdown_connections 2024-04-25T10:31:00.735 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:31:00.732+0000 7f0216108640 1 --2- 172.21.15.37:0/399830625 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f02100cf710 0x7f02100cfb20 unknown :-1 s=CLOSED pgs=690 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:31:00.735 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:31:00.732+0000 7f0216108640 1 --2- 172.21.15.37:0/399830625 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f0210160b30 0x7f02100cded0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:31:00.735 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:31:00.732+0000 7f0216108640 1 -- 172.21.15.37:0/399830625 >> 172.21.15.37:0/399830625 conn(0x7f02100b35c0 msgr2=0x7f02100b39d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:31:00.735 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:31:00.732+0000 7f0216108640 1 -- 172.21.15.37:0/399830625 shutdown_connections 2024-04-25T10:31:00.735 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:31:00.732+0000 7f0216108640 1 -- 172.21.15.37:0/399830625 wait complete. 2024-04-25T10:31:00.736 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:31:00.733+0000 7f0216108640 1 Processor -- start 2024-04-25T10:31:00.736 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:31:00.734+0000 7f0216108640 1 -- start start 2024-04-25T10:31:00.738 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:31:00.735+0000 7f0216108640 1 --2- >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f0210070b40 0x7f0210141720 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:31:00.738 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:31:00.735+0000 7f0216108640 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f02100cf710 0x7f0210150de0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:31:00.738 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:31:00.735+0000 7f0216108640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f0210160b30 0x7f0210151320 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:31:00.738 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:31:00.735+0000 7f0216108640 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 -- 0x7f021015d900 con 0x7f0210160b30 2024-04-25T10:31:00.739 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:31:00.735+0000 7f0216108640 1 -- --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_getmap magic: 0 -- 0x7f0210070f50 con 0x7f02100cf710 2024-04-25T10:31:00.739 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:31:00.735+0000 7f0216108640 1 -- --> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] -- mon_getmap magic: 0 -- 0x7f021015da80 con 0x7f0210070b40 2024-04-25T10:31:00.739 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:31:00.735+0000 7f020ffff640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f0210160b30 0x7f0210151320 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:31:00.739 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:31:00.735+0000 7f020ffff640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f0210160b30 0x7f0210151320 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.37:3300/0 says I am v2:172.21.15.37:34460/0 (socket says 172.21.15.37:34460) 2024-04-25T10:31:00.739 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:31:00.735+0000 7f020ffff640 1 -- 172.21.15.37:0/3587399964 learned_addr learned my addr 172.21.15.37:0/3587399964 (peer_addr_for_me v2:172.21.15.37:0/0) 2024-04-25T10:31:00.739 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:31:00.735+0000 7f020f7fe640 1 --2- 172.21.15.37:0/3587399964 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f0210070b40 0x7f0210141720 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:31:00.739 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:31:00.735+0000 7f020effd640 1 --2- 172.21.15.37:0/3587399964 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f02100cf710 0x7f0210150de0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:31:00.739 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:31:00.735+0000 7f020ffff640 1 -- 172.21.15.37:0/3587399964 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f0210070b40 msgr2=0x7f0210141720 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:31:00.739 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:31:00.735+0000 7f020ffff640 1 --2- 172.21.15.37:0/3587399964 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f0210070b40 0x7f0210141720 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:31:00.739 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:31:00.735+0000 7f020ffff640 1 -- 172.21.15.37:0/3587399964 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f02100cf710 msgr2=0x7f0210150de0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:31:00.739 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:31:00.735+0000 7f020ffff640 1 --2- 172.21.15.37:0/3587399964 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f02100cf710 0x7f0210150de0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:31:00.739 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:31:00.735+0000 7f020ffff640 1 -- 172.21.15.37:0/3587399964 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f02101580c0 con 0x7f0210160b30 2024-04-25T10:31:00.740 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:31:00.735+0000 7f020effd640 1 --2- 172.21.15.37:0/3587399964 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f02100cf710 0x7f0210150de0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T10:31:00.740 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:31:00.736+0000 7f020ffff640 1 --2- 172.21.15.37:0/3587399964 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f0210160b30 0x7f0210151320 secure :-1 s=READY pgs=691 cs=0 l=1 rev1=1 crypto rx=0x7f0208009e50 tx=0x7f0208004a40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:31:00.740 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:31:00.736+0000 7f01f7fff640 1 -- 172.21.15.37:0/3587399964 <== mon.0 v2:172.21.15.37:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0208036050 con 0x7f0210160b30 2024-04-25T10:31:00.740 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:31:00.736+0000 7f01f7fff640 1 -- 172.21.15.37:0/3587399964 <== mon.0 v2:172.21.15.37:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f0208042070 con 0x7f0210160b30 2024-04-25T10:31:00.740 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:31:00.736+0000 7f0216108640 1 -- 172.21.15.37:0/3587399964 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f02101583a0 con 0x7f0210160b30 2024-04-25T10:31:00.740 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:31:00.736+0000 7f01f7fff640 1 -- 172.21.15.37:0/3587399964 <== mon.0 v2:172.21.15.37:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0208009510 con 0x7f0210160b30 2024-04-25T10:31:00.740 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:31:00.736+0000 7f0216108640 1 -- 172.21.15.37:0/3587399964 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f0210158960 con 0x7f0210160b30 2024-04-25T10:31:00.741 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:31:00.737+0000 7f0216108640 1 -- 172.21.15.37:0/3587399964 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f021006a140 con 0x7f0210160b30 2024-04-25T10:31:00.741 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:31:00.737+0000 7f01f7fff640 1 -- 172.21.15.37:0/3587399964 <== mon.0 v2:172.21.15.37:3300/0 4 ==== mgrmap(e 5) ==== 96782+0+0 (secure 0 0 0) 0x7f0208007bf0 con 0x7f0210160b30 2024-04-25T10:31:00.741 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:31:00.738+0000 7f01f7fff640 1 --2- 172.21.15.37:0/3587399964 >> [v2:172.21.15.203:6832/2418205237,v1:172.21.15.203:6833/2418205237] conn(0x7f01fc0744b0 0x7f01fc0769c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:31:00.741 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:31:00.738+0000 7f01f7fff640 1 -- 172.21.15.37:0/3587399964 <== mon.0 v2:172.21.15.37:3300/0 5 ==== osd_map(48..48 src has 1..48) ==== 6012+0+0 (secure 0 0 0) 0x7f02080b6170 con 0x7f0210160b30 2024-04-25T10:31:00.741 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:31:00.738+0000 7f020f7fe640 1 --2- 172.21.15.37:0/3587399964 >> [v2:172.21.15.203:6832/2418205237,v1:172.21.15.203:6833/2418205237] conn(0x7f01fc0744b0 0x7f01fc0769c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:31:00.741 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:31:00.739+0000 7f020f7fe640 1 --2- 172.21.15.37:0/3587399964 >> [v2:172.21.15.203:6832/2418205237,v1:172.21.15.203:6833/2418205237] conn(0x7f01fc0744b0 0x7f01fc0769c0 secure :-1 s=READY pgs=312 cs=0 l=1 rev1=1 crypto rx=0x7f0210073120 tx=0x7f0200008040 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:31:00.745 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:31:00.743+0000 7f01f7fff640 1 -- 172.21.15.37:0/3587399964 <== mon.0 v2:172.21.15.37:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187246 (secure 0 0 0) 0x7f02080807d0 con 0x7f0210160b30 2024-04-25T10:31:00.936 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:31:00.934+0000 7f0216108640 1 -- 172.21.15.37:0/3587399964 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_command({"prefix": "osd pool rm", "pool": "recovery_fs_metadata", "pool2": "recovery_fs_metadata", "yes_i_really_really_mean_it": true} v 0) -- 0x7f02100b48c0 con 0x7f0210160b30 2024-04-25T10:31:01.861 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:31:01.858+0000 7f01f7fff640 1 -- 172.21.15.37:0/3587399964 <== mon.0 v2:172.21.15.37:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool rm", "pool": "recovery_fs_metadata", "pool2": "recovery_fs_metadata", "yes_i_really_really_mean_it": true}]=0 pool 'recovery_fs_metadata' removed v49) ==== 196+0+0 (secure 0 0 0) 0x7f02080800d0 con 0x7f0210160b30 2024-04-25T10:31:01.861 INFO:teuthology.orchestra.run.smithi037.stderr:pool 'recovery_fs_metadata' removed 2024-04-25T10:31:01.864 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:31:01.861+0000 7f0216108640 1 -- 172.21.15.37:0/3587399964 >> [v2:172.21.15.203:6832/2418205237,v1:172.21.15.203:6833/2418205237] conn(0x7f01fc0744b0 msgr2=0x7f01fc0769c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:31:01.864 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:31:01.861+0000 7f0216108640 1 --2- 172.21.15.37:0/3587399964 >> [v2:172.21.15.203:6832/2418205237,v1:172.21.15.203:6833/2418205237] conn(0x7f01fc0744b0 0x7f01fc0769c0 secure :-1 s=READY pgs=312 cs=0 l=1 rev1=1 crypto rx=0x7f0210073120 tx=0x7f0200008040 comp rx=0 tx=0).stop 2024-04-25T10:31:01.864 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:31:01.861+0000 7f0216108640 1 -- 172.21.15.37:0/3587399964 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f0210160b30 msgr2=0x7f0210151320 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:31:01.864 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:31:01.861+0000 7f0216108640 1 --2- 172.21.15.37:0/3587399964 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f0210160b30 0x7f0210151320 secure :-1 s=READY pgs=691 cs=0 l=1 rev1=1 crypto rx=0x7f0208009e50 tx=0x7f0208004a40 comp rx=0 tx=0).stop 2024-04-25T10:31:01.864 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:31:01.861+0000 7f0216108640 1 -- 172.21.15.37:0/3587399964 shutdown_connections 2024-04-25T10:31:01.864 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:31:01.861+0000 7f0216108640 1 --2- 172.21.15.37:0/3587399964 >> [v2:172.21.15.203:6832/2418205237,v1:172.21.15.203:6833/2418205237] conn(0x7f01fc0744b0 0x7f01fc0769c0 unknown :-1 s=CLOSED pgs=312 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:31:01.864 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:31:01.861+0000 7f0216108640 1 --2- 172.21.15.37:0/3587399964 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f0210160b30 0x7f0210151320 unknown :-1 s=CLOSED pgs=691 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:31:01.864 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:31:01.861+0000 7f0216108640 1 --2- 172.21.15.37:0/3587399964 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f02100cf710 0x7f0210150de0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:31:01.864 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:31:01.861+0000 7f0216108640 1 --2- 172.21.15.37:0/3587399964 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f0210070b40 0x7f0210141720 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:31:01.864 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:31:01.861+0000 7f0216108640 1 -- 172.21.15.37:0/3587399964 >> 172.21.15.37:0/3587399964 conn(0x7f02100b35c0 msgr2=0x7f0210156440 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:31:01.864 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:31:01.862+0000 7f0216108640 1 -- 172.21.15.37:0/3587399964 shutdown_connections 2024-04-25T10:31:01.864 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:31:01.862+0000 7f0216108640 1 -- 172.21.15.37:0/3587399964 wait complete. 2024-04-25T10:31:01.880 DEBUG:teuthology.orchestra.run.smithi037:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool rm cephfs_data cephfs_data --yes-i-really-really-mean-it 2024-04-25T10:31:02.001 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:31:01.999+0000 7efc502b1640 1 Processor -- start 2024-04-25T10:31:02.002 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:31:01.999+0000 7efc502b1640 1 -- start start 2024-04-25T10:31:02.004 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:31:02.001+0000 7efc502b1640 1 --2- >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7efc480bb690 0x7efc480d4e60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:31:02.004 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:31:02.001+0000 7efc502b1640 1 --2- >> v2:172.21.15.37:3300/0 conn(0x7efc480b7cb0 0x7efc48164080 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:31:02.004 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:31:02.001+0000 7efc502b1640 1 -- --> v1:172.21.15.203:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7efc480bbaa0 con 0x7efc48163c70 2024-04-25T10:31:02.004 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:31:02.001+0000 7efc502b1640 1 -- --> v2:172.21.15.37:3300/0 -- mon_getmap magic: 0 -- 0x7efc480d49d0 con 0x7efc480b7cb0 2024-04-25T10:31:02.004 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:31:02.001+0000 7efc502b1640 1 -- --> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] -- mon_getmap magic: 0 -- 0x7efc48164730 con 0x7efc480bb690 2024-04-25T10:31:02.004 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:31:02.001+0000 7efc4e84e640 1 --2- >> v2:172.21.15.37:3300/0 conn(0x7efc480b7cb0 0x7efc48164080 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:31:02.004 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:31:02.001+0000 7efc4e84e640 1 --2- >> v2:172.21.15.37:3300/0 conn(0x7efc480b7cb0 0x7efc48164080 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.37:3300/0 says I am v2:172.21.15.37:34476/0 (socket says 172.21.15.37:34476) 2024-04-25T10:31:02.004 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:31:02.001+0000 7efc4e84e640 1 -- 172.21.15.37:0/4184368113 learned_addr learned my addr 172.21.15.37:0/4184368113 (peer_addr_for_me v2:172.21.15.37:0/0) 2024-04-25T10:31:02.004 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:31:02.001+0000 7efc4e04d640 1 --2- 172.21.15.37:0/4184368113 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7efc480bb690 0x7efc480d4e60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:31:02.005 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:31:02.001+0000 7efc4e84e640 1 -- 172.21.15.37:0/4184368113 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7efc480bb690 msgr2=0x7efc480d4e60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:31:02.005 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:31:02.001+0000 7efc4e84e640 1 --2- 172.21.15.37:0/4184368113 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7efc480bb690 0x7efc480d4e60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:31:02.005 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:31:02.001+0000 7efc4e84e640 1 -- 172.21.15.37:0/4184368113 >> v1:172.21.15.203:6789/0 conn(0x7efc48163c70 legacy=0x7efc480b75a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:31:02.005 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:31:02.001+0000 7efc4e84e640 1 -- 172.21.15.37:0/4184368113 --> v2:172.21.15.37:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7efc48164a10 con 0x7efc480b7cb0 2024-04-25T10:31:02.005 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:31:02.002+0000 7efc4e84e640 1 --2- 172.21.15.37:0/4184368113 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7efc480b7cb0 0x7efc48164080 secure :-1 s=READY pgs=692 cs=0 l=1 rev1=1 crypto rx=0x7efc40009e50 tx=0x7efc4002a6a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=34c6c26776efe1da server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:31:02.005 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:31:02.002+0000 7efc4d04b640 1 -- 172.21.15.37:0/4184368113 <== mon.0 v2:172.21.15.37:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7efc40036070 con 0x7efc480b7cb0 2024-04-25T10:31:02.005 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:31:02.002+0000 7efc4d04b640 1 -- 172.21.15.37:0/4184368113 <== mon.0 v2:172.21.15.37:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7efc40031600 con 0x7efc480b7cb0 2024-04-25T10:31:02.005 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:31:02.002+0000 7efc4d04b640 1 -- 172.21.15.37:0/4184368113 <== mon.0 v2:172.21.15.37:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7efc400319d0 con 0x7efc480b7cb0 2024-04-25T10:31:02.006 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:31:02.003+0000 7efc502b1640 1 -- 172.21.15.37:0/4184368113 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7efc480b7cb0 msgr2=0x7efc48164080 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:31:02.006 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:31:02.003+0000 7efc502b1640 1 --2- 172.21.15.37:0/4184368113 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7efc480b7cb0 0x7efc48164080 secure :-1 s=READY pgs=692 cs=0 l=1 rev1=1 crypto rx=0x7efc40009e50 tx=0x7efc4002a6a0 comp rx=0 tx=0).stop 2024-04-25T10:31:02.006 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:31:02.003+0000 7efc502b1640 1 -- 172.21.15.37:0/4184368113 shutdown_connections 2024-04-25T10:31:02.006 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:31:02.003+0000 7efc502b1640 1 --2- 172.21.15.37:0/4184368113 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7efc480b7cb0 0x7efc48164080 unknown :-1 s=CLOSED pgs=692 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:31:02.006 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:31:02.003+0000 7efc502b1640 1 --2- 172.21.15.37:0/4184368113 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7efc480bb690 0x7efc480d4e60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:31:02.006 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:31:02.003+0000 7efc502b1640 1 -- 172.21.15.37:0/4184368113 >> 172.21.15.37:0/4184368113 conn(0x7efc480d2780 msgr2=0x7efc480d2b90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:31:02.006 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:31:02.003+0000 7efc502b1640 1 -- 172.21.15.37:0/4184368113 shutdown_connections 2024-04-25T10:31:02.006 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:31:02.003+0000 7efc502b1640 1 -- 172.21.15.37:0/4184368113 wait complete. 2024-04-25T10:31:02.007 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:31:02.005+0000 7efc502b1640 1 Processor -- start 2024-04-25T10:31:02.007 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:31:02.005+0000 7efc502b1640 1 -- start start 2024-04-25T10:31:02.009 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:31:02.006+0000 7efc502b1640 1 --2- >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7efc480bb690 0x7efc4814e720 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:31:02.009 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:31:02.006+0000 7efc502b1640 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7efc48163c70 0x7efc480cf930 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:31:02.009 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:31:02.006+0000 7efc502b1640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7efc480b9610 0x7efc480b9a20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:31:02.009 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:31:02.006+0000 7efc502b1640 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 -- 0x7efc480d53a0 con 0x7efc480b9610 2024-04-25T10:31:02.010 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:31:02.006+0000 7efc502b1640 1 -- --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_getmap magic: 0 -- 0x7efc480d09f0 con 0x7efc48163c70 2024-04-25T10:31:02.010 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:31:02.006+0000 7efc502b1640 1 -- --> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] -- mon_getmap magic: 0 -- 0x7efc480d5520 con 0x7efc480bb690 2024-04-25T10:31:02.010 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:31:02.006+0000 7efc4e84e640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7efc480b9610 0x7efc480b9a20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:31:02.010 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:31:02.006+0000 7efc4e84e640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7efc480b9610 0x7efc480b9a20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.37:3300/0 says I am v2:172.21.15.37:34492/0 (socket says 172.21.15.37:34492) 2024-04-25T10:31:02.010 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:31:02.006+0000 7efc4e04d640 1 --2- >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7efc480bb690 0x7efc4814e720 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:31:02.010 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:31:02.006+0000 7efc4e84e640 1 -- 172.21.15.37:0/2016857025 learned_addr learned my addr 172.21.15.37:0/2016857025 (peer_addr_for_me v2:172.21.15.37:0/0) 2024-04-25T10:31:02.010 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:31:02.006+0000 7efc4d84c640 1 --2- 172.21.15.37:0/2016857025 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7efc48163c70 0x7efc480cf930 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:31:02.010 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:31:02.006+0000 7efc4e84e640 1 -- 172.21.15.37:0/2016857025 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7efc480bb690 msgr2=0x7efc4814e720 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:31:02.010 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:31:02.006+0000 7efc4e84e640 1 --2- 172.21.15.37:0/2016857025 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7efc480bb690 0x7efc4814e720 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:31:02.010 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:31:02.006+0000 7efc4e84e640 1 -- 172.21.15.37:0/2016857025 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7efc48163c70 msgr2=0x7efc480cf930 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:31:02.010 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:31:02.006+0000 7efc4e84e640 1 --2- 172.21.15.37:0/2016857025 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7efc48163c70 0x7efc480cf930 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:31:02.010 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:31:02.006+0000 7efc4e84e640 1 -- 172.21.15.37:0/2016857025 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7efc480b9f60 con 0x7efc480b9610 2024-04-25T10:31:02.011 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:31:02.007+0000 7efc4e84e640 1 --2- 172.21.15.37:0/2016857025 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7efc480b9610 0x7efc480b9a20 secure :-1 s=READY pgs=693 cs=0 l=1 rev1=1 crypto rx=0x7efc40009e50 tx=0x7efc40004a40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:31:02.011 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:31:02.007+0000 7efc3effd640 1 -- 172.21.15.37:0/2016857025 <== mon.0 v2:172.21.15.37:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7efc40036050 con 0x7efc480b9610 2024-04-25T10:31:02.011 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:31:02.007+0000 7efc3effd640 1 -- 172.21.15.37:0/2016857025 <== mon.0 v2:172.21.15.37:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7efc40042070 con 0x7efc480b9610 2024-04-25T10:31:02.011 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:31:02.007+0000 7efc502b1640 1 -- 172.21.15.37:0/2016857025 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7efc480cfe70 con 0x7efc480b9610 2024-04-25T10:31:02.011 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:31:02.007+0000 7efc3effd640 1 -- 172.21.15.37:0/2016857025 <== mon.0 v2:172.21.15.37:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7efc40007b90 con 0x7efc480b9610 2024-04-25T10:31:02.011 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:31:02.007+0000 7efc502b1640 1 -- 172.21.15.37:0/2016857025 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7efc4814f280 con 0x7efc480b9610 2024-04-25T10:31:02.012 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:31:02.008+0000 7efc502b1640 1 -- 172.21.15.37:0/2016857025 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7efc480d0040 con 0x7efc480b9610 2024-04-25T10:31:02.012 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:31:02.009+0000 7efc3effd640 1 -- 172.21.15.37:0/2016857025 <== mon.0 v2:172.21.15.37:3300/0 4 ==== mgrmap(e 5) ==== 96782+0+0 (secure 0 0 0) 0x7efc40047460 con 0x7efc480b9610 2024-04-25T10:31:02.012 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:31:02.009+0000 7efc3effd640 1 --2- 172.21.15.37:0/2016857025 >> [v2:172.21.15.203:6832/2418205237,v1:172.21.15.203:6833/2418205237] conn(0x7efc2c0744b0 0x7efc2c0769c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:31:02.012 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:31:02.009+0000 7efc3effd640 1 -- 172.21.15.37:0/2016857025 <== mon.0 v2:172.21.15.37:3300/0 5 ==== osd_map(49..49 src has 1..49) ==== 5564+0+0 (secure 0 0 0) 0x7efc400b5f50 con 0x7efc480b9610 2024-04-25T10:31:02.012 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:31:02.009+0000 7efc4e04d640 1 --2- 172.21.15.37:0/2016857025 >> [v2:172.21.15.203:6832/2418205237,v1:172.21.15.203:6833/2418205237] conn(0x7efc2c0744b0 0x7efc2c0769c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:31:02.012 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:31:02.010+0000 7efc4e04d640 1 --2- 172.21.15.37:0/2016857025 >> [v2:172.21.15.203:6832/2418205237,v1:172.21.15.203:6833/2418205237] conn(0x7efc2c0744b0 0x7efc2c0769c0 secure :-1 s=READY pgs=313 cs=0 l=1 rev1=1 crypto rx=0x7efc440079a0 tx=0x7efc44008040 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:31:02.015 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:31:02.014+0000 7efc3effd640 1 -- 172.21.15.37:0/2016857025 <== mon.0 v2:172.21.15.37:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187246 (secure 0 0 0) 0x7efc480b9f60 con 0x7efc480b9610 2024-04-25T10:31:02.201 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:31:02.199+0000 7efc502b1640 1 -- 172.21.15.37:0/2016857025 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_command({"prefix": "osd pool rm", "pool": "cephfs_data", "pool2": "cephfs_data", "yes_i_really_really_mean_it": true} v 0) -- 0x7efc4806a140 con 0x7efc480b9610 2024-04-25T10:31:02.867 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:31:02.864+0000 7efc3effd640 1 -- 172.21.15.37:0/2016857025 <== mon.0 v2:172.21.15.37:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool rm", "pool": "cephfs_data", "pool2": "cephfs_data", "yes_i_really_really_mean_it": true}]=0 pool 'cephfs_data' removed v50) ==== 169+0+0 (secure 0 0 0) 0x7efc40080120 con 0x7efc480b9610 2024-04-25T10:31:02.867 INFO:teuthology.orchestra.run.smithi037.stderr:pool 'cephfs_data' removed 2024-04-25T10:31:02.875 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:31:02.870+0000 7efc502b1640 1 -- 172.21.15.37:0/2016857025 >> [v2:172.21.15.203:6832/2418205237,v1:172.21.15.203:6833/2418205237] conn(0x7efc2c0744b0 msgr2=0x7efc2c0769c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:31:02.875 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:31:02.870+0000 7efc502b1640 1 --2- 172.21.15.37:0/2016857025 >> [v2:172.21.15.203:6832/2418205237,v1:172.21.15.203:6833/2418205237] conn(0x7efc2c0744b0 0x7efc2c0769c0 secure :-1 s=READY pgs=313 cs=0 l=1 rev1=1 crypto rx=0x7efc440079a0 tx=0x7efc44008040 comp rx=0 tx=0).stop 2024-04-25T10:31:02.875 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:31:02.871+0000 7efc502b1640 1 -- 172.21.15.37:0/2016857025 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7efc480b9610 msgr2=0x7efc480b9a20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:31:02.875 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:31:02.871+0000 7efc502b1640 1 --2- 172.21.15.37:0/2016857025 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7efc480b9610 0x7efc480b9a20 secure :-1 s=READY pgs=693 cs=0 l=1 rev1=1 crypto rx=0x7efc40009e50 tx=0x7efc40004a40 comp rx=0 tx=0).stop 2024-04-25T10:31:02.875 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:31:02.872+0000 7efc502b1640 1 -- 172.21.15.37:0/2016857025 shutdown_connections 2024-04-25T10:31:02.875 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:31:02.872+0000 7efc502b1640 1 --2- 172.21.15.37:0/2016857025 >> [v2:172.21.15.203:6832/2418205237,v1:172.21.15.203:6833/2418205237] conn(0x7efc2c0744b0 0x7efc2c0769c0 unknown :-1 s=CLOSED pgs=313 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:31:02.875 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:31:02.872+0000 7efc502b1640 1 --2- 172.21.15.37:0/2016857025 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7efc480b9610 0x7efc480b9a20 unknown :-1 s=CLOSED pgs=693 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:31:02.875 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:31:02.872+0000 7efc502b1640 1 --2- 172.21.15.37:0/2016857025 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7efc48163c70 0x7efc480cf930 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:31:02.876 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:31:02.872+0000 7efc502b1640 1 --2- 172.21.15.37:0/2016857025 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7efc480bb690 0x7efc4814e720 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:31:02.876 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:31:02.872+0000 7efc502b1640 1 -- 172.21.15.37:0/2016857025 >> 172.21.15.37:0/2016857025 conn(0x7efc480d2780 msgr2=0x7efc480b6e20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:31:02.876 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:31:02.873+0000 7efc502b1640 1 -- 172.21.15.37:0/2016857025 shutdown_connections 2024-04-25T10:31:02.876 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:31:02.873+0000 7efc502b1640 1 -- 172.21.15.37:0/2016857025 wait complete. 2024-04-25T10:31:02.891 DEBUG:teuthology.orchestra.run.smithi037:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph config rm mds debug_mds 2024-04-25T10:31:03.011 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:31:03.008+0000 7f150b5b2640 1 Processor -- start 2024-04-25T10:31:03.011 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:31:03.009+0000 7f150b5b2640 1 -- start start 2024-04-25T10:31:03.013 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:31:03.010+0000 7f150b5b2640 1 --2- >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f15040d5160 0x7f15040b4e80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:31:03.013 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:31:03.010+0000 7f150b5b2640 1 --2- >> v2:172.21.15.37:3300/0 conn(0x7f15040d3230 0x7f15040b9060 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:31:03.013 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:31:03.010+0000 7f150b5b2640 1 -- --> v1:172.21.15.203:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f15040d1370 con 0x7f15040d0990 2024-04-25T10:31:03.013 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:31:03.010+0000 7f150b5b2640 1 -- --> v2:172.21.15.37:3300/0 -- mon_getmap magic: 0 -- 0x7f15040d1650 con 0x7f15040d3230 2024-04-25T10:31:03.013 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:31:03.010+0000 7f150b5b2640 1 -- --> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] -- mon_getmap magic: 0 -- 0x7f15040d4680 con 0x7f15040d5160 2024-04-25T10:31:03.013 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:31:03.010+0000 7f1509b4f640 1 --2- >> v2:172.21.15.37:3300/0 conn(0x7f15040d3230 0x7f15040b9060 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:31:03.013 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:31:03.010+0000 7f1509b4f640 1 --2- >> v2:172.21.15.37:3300/0 conn(0x7f15040d3230 0x7f15040b9060 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.37:3300/0 says I am v2:172.21.15.37:34508/0 (socket says 172.21.15.37:34508) 2024-04-25T10:31:03.013 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:31:03.010+0000 7f1509b4f640 1 -- 172.21.15.37:0/4273328942 learned_addr learned my addr 172.21.15.37:0/4273328942 (peer_addr_for_me v2:172.21.15.37:0/0) 2024-04-25T10:31:03.013 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:31:03.010+0000 7f150934e640 1 --2- 172.21.15.37:0/4273328942 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f15040d5160 0x7f15040b4e80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:31:03.013 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:31:03.010+0000 7f1509b4f640 1 -- 172.21.15.37:0/4273328942 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f15040d5160 msgr2=0x7f15040b4e80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:31:03.014 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:31:03.010+0000 7f1509b4f640 1 --2- 172.21.15.37:0/4273328942 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f15040d5160 0x7f15040b4e80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:31:03.014 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:31:03.010+0000 7f1509b4f640 1 -- 172.21.15.37:0/4273328942 >> v1:172.21.15.203:6789/0 conn(0x7f15040d0990 legacy=0x7f15040d2b20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:31:03.014 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:31:03.010+0000 7f1509b4f640 1 -- 172.21.15.37:0/4273328942 --> v2:172.21.15.37:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f15040d4960 con 0x7f15040d3230 2024-04-25T10:31:03.015 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:31:03.011+0000 7f1509b4f640 1 --2- 172.21.15.37:0/4273328942 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f15040d3230 0x7f15040b9060 secure :-1 s=READY pgs=694 cs=0 l=1 rev1=1 crypto rx=0x7f14f4009e50 tx=0x7f14f402a610 comp rx=0 tx=0).ready entity=mon.0 client_cookie=85ec01d9fb2d6bee server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:31:03.015 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:31:03.011+0000 7f1503fff640 1 -- 172.21.15.37:0/4273328942 <== mon.0 v2:172.21.15.37:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f14f4036070 con 0x7f15040d3230 2024-04-25T10:31:03.015 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:31:03.011+0000 7f1503fff640 1 -- 172.21.15.37:0/4273328942 <== mon.0 v2:172.21.15.37:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f14f4031600 con 0x7f15040d3230 2024-04-25T10:31:03.015 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:31:03.011+0000 7f1503fff640 1 -- 172.21.15.37:0/4273328942 <== mon.0 v2:172.21.15.37:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f14f40319d0 con 0x7f15040d3230 2024-04-25T10:31:03.015 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:31:03.012+0000 7f150b5b2640 1 -- 172.21.15.37:0/4273328942 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f15040d3230 msgr2=0x7f15040b9060 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:31:03.015 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:31:03.012+0000 7f150b5b2640 1 --2- 172.21.15.37:0/4273328942 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f15040d3230 0x7f15040b9060 secure :-1 s=READY pgs=694 cs=0 l=1 rev1=1 crypto rx=0x7f14f4009e50 tx=0x7f14f402a610 comp rx=0 tx=0).stop 2024-04-25T10:31:03.016 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:31:03.012+0000 7f150b5b2640 1 -- 172.21.15.37:0/4273328942 shutdown_connections 2024-04-25T10:31:03.016 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:31:03.012+0000 7f150b5b2640 1 --2- 172.21.15.37:0/4273328942 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f15040d3230 0x7f15040b9060 unknown :-1 s=CLOSED pgs=694 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:31:03.016 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:31:03.012+0000 7f150b5b2640 1 --2- 172.21.15.37:0/4273328942 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f15040d5160 0x7f15040b4e80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:31:03.016 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:31:03.012+0000 7f150b5b2640 1 -- 172.21.15.37:0/4273328942 >> 172.21.15.37:0/4273328942 conn(0x7f15040b7a80 msgr2=0x7f15040b7e90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:31:03.016 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:31:03.012+0000 7f150b5b2640 1 -- 172.21.15.37:0/4273328942 shutdown_connections 2024-04-25T10:31:03.016 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:31:03.012+0000 7f150b5b2640 1 -- 172.21.15.37:0/4273328942 wait complete. 2024-04-25T10:31:03.016 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:31:03.014+0000 7f150b5b2640 1 Processor -- start 2024-04-25T10:31:03.016 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:31:03.014+0000 7f150b5b2640 1 -- start start 2024-04-25T10:31:03.019 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:31:03.015+0000 7f150b5b2640 1 --2- >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f15040d0990 0x7f15040ac5a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:31:03.019 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:31:03.015+0000 7f150b5b2640 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f15040d5160 0x7f15040cfc40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:31:03.019 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:31:03.015+0000 7f150b5b2640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f150415e8b0 0x7f150415ecc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:31:03.019 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:31:03.015+0000 7f150b5b2640 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 -- 0x7f1504164c40 con 0x7f150415e8b0 2024-04-25T10:31:03.019 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:31:03.015+0000 7f150b5b2640 1 -- --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_getmap magic: 0 -- 0x7f1504164ac0 con 0x7f15040d5160 2024-04-25T10:31:03.019 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:31:03.015+0000 7f150b5b2640 1 -- --> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] -- mon_getmap magic: 0 -- 0x7f15040d0da0 con 0x7f15040d0990 2024-04-25T10:31:03.019 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:31:03.015+0000 7f1509b4f640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f150415e8b0 0x7f150415ecc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:31:03.019 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:31:03.015+0000 7f1509b4f640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f150415e8b0 0x7f150415ecc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.37:3300/0 says I am v2:172.21.15.37:34522/0 (socket says 172.21.15.37:34522) 2024-04-25T10:31:03.019 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:31:03.015+0000 7f1509b4f640 1 -- 172.21.15.37:0/984690224 learned_addr learned my addr 172.21.15.37:0/984690224 (peer_addr_for_me v2:172.21.15.37:0/0) 2024-04-25T10:31:03.019 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:31:03.016+0000 7f150934e640 1 --2- 172.21.15.37:0/984690224 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f15040d0990 0x7f15040ac5a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:31:03.019 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:31:03.016+0000 7f1508b4d640 1 --2- 172.21.15.37:0/984690224 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f15040d5160 0x7f15040cfc40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:31:03.019 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:31:03.016+0000 7f1509b4f640 1 -- 172.21.15.37:0/984690224 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f15040d0990 msgr2=0x7f15040ac5a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:31:03.020 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:31:03.016+0000 7f1509b4f640 1 --2- 172.21.15.37:0/984690224 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f15040d0990 0x7f15040ac5a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:31:03.020 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:31:03.016+0000 7f1509b4f640 1 -- 172.21.15.37:0/984690224 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f15040d5160 msgr2=0x7f15040cfc40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:31:03.020 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:31:03.016+0000 7f1509b4f640 1 --2- 172.21.15.37:0/984690224 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f15040d5160 0x7f15040cfc40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:31:03.020 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:31:03.016+0000 7f1509b4f640 1 -- 172.21.15.37:0/984690224 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f150415f200 con 0x7f150415e8b0 2024-04-25T10:31:03.020 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:31:03.016+0000 7f1508b4d640 1 --2- 172.21.15.37:0/984690224 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f15040d5160 0x7f15040cfc40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T10:31:03.020 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:31:03.016+0000 7f1509b4f640 1 --2- 172.21.15.37:0/984690224 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f150415e8b0 0x7f150415ecc0 secure :-1 s=READY pgs=695 cs=0 l=1 rev1=1 crypto rx=0x7f14f4009f70 tx=0x7f14f4004a40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:31:03.020 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:31:03.016+0000 7f1501ffb640 1 -- 172.21.15.37:0/984690224 <== mon.0 v2:172.21.15.37:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f14f4036050 con 0x7f150415e8b0 2024-04-25T10:31:03.020 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:31:03.016+0000 7f1501ffb640 1 -- 172.21.15.37:0/984690224 <== mon.0 v2:172.21.15.37:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f14f4042070 con 0x7f150415e8b0 2024-04-25T10:31:03.020 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:31:03.016+0000 7f150b5b2640 1 -- 172.21.15.37:0/984690224 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f15041592b0 con 0x7f150415e8b0 2024-04-25T10:31:03.020 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:31:03.016+0000 7f1501ffb640 1 -- 172.21.15.37:0/984690224 <== mon.0 v2:172.21.15.37:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f14f4007990 con 0x7f150415e8b0 2024-04-25T10:31:03.020 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:31:03.016+0000 7f150b5b2640 1 -- 172.21.15.37:0/984690224 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f1504159870 con 0x7f150415e8b0 2024-04-25T10:31:03.021 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:31:03.017+0000 7f150b5b2640 1 -- 172.21.15.37:0/984690224 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f150406a140 con 0x7f150415e8b0 2024-04-25T10:31:03.021 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:31:03.018+0000 7f1501ffb640 1 -- 172.21.15.37:0/984690224 <== mon.0 v2:172.21.15.37:3300/0 4 ==== mgrmap(e 5) ==== 96782+0+0 (secure 0 0 0) 0x7f14f4007b30 con 0x7f150415e8b0 2024-04-25T10:31:03.021 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:31:03.018+0000 7f1501ffb640 1 --2- 172.21.15.37:0/984690224 >> [v2:172.21.15.203:6832/2418205237,v1:172.21.15.203:6833/2418205237] conn(0x7f14f00744b0 0x7f14f00769c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:31:03.021 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:31:03.018+0000 7f1501ffb640 1 -- 172.21.15.37:0/984690224 <== mon.0 v2:172.21.15.37:3300/0 5 ==== osd_map(50..50 src has 1..50) ==== 5029+0+0 (secure 0 0 0) 0x7f14f40b5e20 con 0x7f150415e8b0 2024-04-25T10:31:03.021 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:31:03.019+0000 7f150934e640 1 --2- 172.21.15.37:0/984690224 >> [v2:172.21.15.203:6832/2418205237,v1:172.21.15.203:6833/2418205237] conn(0x7f14f00744b0 0x7f14f00769c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:31:03.022 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:31:03.019+0000 7f150934e640 1 --2- 172.21.15.37:0/984690224 >> [v2:172.21.15.203:6832/2418205237,v1:172.21.15.203:6833/2418205237] conn(0x7f14f00744b0 0x7f14f00769c0 secure :-1 s=READY pgs=314 cs=0 l=1 rev1=1 crypto rx=0x7f14fc006fd0 tx=0x7f14fc008040 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:31:03.025 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:31:03.023+0000 7f1501ffb640 1 -- 172.21.15.37:0/984690224 <== mon.0 v2:172.21.15.37:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187246 (secure 0 0 0) 0x7f150415f200 con 0x7f150415e8b0 2024-04-25T10:31:03.220 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:31:03.218+0000 7f150b5b2640 1 -- 172.21.15.37:0/984690224 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_command({"prefix": "config rm", "who": "mds", "name": "debug_mds"} v 0) -- 0x7f15040b6ee0 con 0x7f150415e8b0 2024-04-25T10:31:03.223 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:31:03.221+0000 7f1501ffb640 1 -- 172.21.15.37:0/984690224 <== mon.0 v2:172.21.15.37:3300/0 7 ==== mon_command_ack([{"prefix": "config rm", "who": "mds", "name": "debug_mds"}]=0 v12) ==== 92+0+0 (secure 0 0 0) 0x7f14f4007e20 con 0x7f150415e8b0 2024-04-25T10:31:03.226 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:31:03.224+0000 7f150b5b2640 1 -- 172.21.15.37:0/984690224 >> [v2:172.21.15.203:6832/2418205237,v1:172.21.15.203:6833/2418205237] conn(0x7f14f00744b0 msgr2=0x7f14f00769c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:31:03.226 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:31:03.224+0000 7f150b5b2640 1 --2- 172.21.15.37:0/984690224 >> [v2:172.21.15.203:6832/2418205237,v1:172.21.15.203:6833/2418205237] conn(0x7f14f00744b0 0x7f14f00769c0 secure :-1 s=READY pgs=314 cs=0 l=1 rev1=1 crypto rx=0x7f14fc006fd0 tx=0x7f14fc008040 comp rx=0 tx=0).stop 2024-04-25T10:31:03.227 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:31:03.224+0000 7f150b5b2640 1 -- 172.21.15.37:0/984690224 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f150415e8b0 msgr2=0x7f150415ecc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:31:03.227 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:31:03.224+0000 7f150b5b2640 1 --2- 172.21.15.37:0/984690224 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f150415e8b0 0x7f150415ecc0 secure :-1 s=READY pgs=695 cs=0 l=1 rev1=1 crypto rx=0x7f14f4009f70 tx=0x7f14f4004a40 comp rx=0 tx=0).stop 2024-04-25T10:31:03.227 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:31:03.224+0000 7f150b5b2640 1 -- 172.21.15.37:0/984690224 shutdown_connections 2024-04-25T10:31:03.227 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:31:03.224+0000 7f150b5b2640 1 --2- 172.21.15.37:0/984690224 >> [v2:172.21.15.203:6832/2418205237,v1:172.21.15.203:6833/2418205237] conn(0x7f14f00744b0 0x7f14f00769c0 unknown :-1 s=CLOSED pgs=314 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:31:03.227 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:31:03.224+0000 7f150b5b2640 1 --2- 172.21.15.37:0/984690224 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f150415e8b0 0x7f150415ecc0 unknown :-1 s=CLOSED pgs=695 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:31:03.227 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:31:03.224+0000 7f150b5b2640 1 --2- 172.21.15.37:0/984690224 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f15040d5160 0x7f15040cfc40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:31:03.227 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:31:03.224+0000 7f150b5b2640 1 --2- 172.21.15.37:0/984690224 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f15040d0990 0x7f15040ac5a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:31:03.227 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:31:03.224+0000 7f150b5b2640 1 -- 172.21.15.37:0/984690224 >> 172.21.15.37:0/984690224 conn(0x7f15040b7a80 msgr2=0x7f15040b7e90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:31:03.227 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:31:03.224+0000 7f150b5b2640 1 -- 172.21.15.37:0/984690224 shutdown_connections 2024-04-25T10:31:03.227 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:31:03.224+0000 7f150b5b2640 1 -- 172.21.15.37:0/984690224 wait complete. 2024-04-25T10:31:03.237 DEBUG:teuthology.orchestra.run.smithi037:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph config rm mon mon_allow_pool_delete 2024-04-25T10:31:03.350 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:31:03.348+0000 7f176efe3640 1 Processor -- start 2024-04-25T10:31:03.350 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:31:03.348+0000 7f176efe3640 1 -- start start 2024-04-25T10:31:03.352 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:31:03.349+0000 7f176efe3640 1 --2- >> v2:172.21.15.203:3300/0 conn(0x7f176815a9f0 0x7f176806c540 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:31:03.352 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:31:03.349+0000 7f176efe3640 1 --2- >> v2:172.21.15.37:3300/0 conn(0x7f176806ca80 0x7f1768160910 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:31:03.352 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:31:03.349+0000 7f176efe3640 1 -- --> v1:172.21.15.37:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f176806ce90 con 0x7f1768160e50 2024-04-25T10:31:03.352 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:31:03.349+0000 7f176efe3640 1 -- --> v2:172.21.15.37:3300/0 -- mon_getmap magic: 0 -- 0x7f17680b6870 con 0x7f176806ca80 2024-04-25T10:31:03.353 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:31:03.349+0000 7f176efe3640 1 -- --> v2:172.21.15.203:3300/0 -- mon_getmap magic: 0 -- 0x7f17681616d0 con 0x7f176815a9f0 2024-04-25T10:31:03.353 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:31:03.350+0000 7f175ffff640 1 --2- >> v2:172.21.15.37:3300/0 conn(0x7f176806ca80 0x7f1768160910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:31:03.353 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:31:03.350+0000 7f175ffff640 1 --2- >> v2:172.21.15.37:3300/0 conn(0x7f176806ca80 0x7f1768160910 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.37:3300/0 says I am v2:172.21.15.37:34534/0 (socket says 172.21.15.37:34534) 2024-04-25T10:31:03.353 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:31:03.350+0000 7f175ffff640 1 -- 172.21.15.37:0/1959887498 learned_addr learned my addr 172.21.15.37:0/1959887498 (peer_addr_for_me v2:172.21.15.37:0/0) 2024-04-25T10:31:03.353 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:31:03.350+0000 7f176cd7f640 1 --2- 172.21.15.37:0/1959887498 >> v2:172.21.15.203:3300/0 conn(0x7f176815a9f0 0x7f176806c540 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:31:03.353 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:31:03.350+0000 7f175ffff640 1 -- 172.21.15.37:0/1959887498 >> v2:172.21.15.203:3300/0 conn(0x7f176815a9f0 msgr2=0x7f176806c540 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:31:03.353 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:31:03.350+0000 7f175ffff640 1 --2- 172.21.15.37:0/1959887498 >> v2:172.21.15.203:3300/0 conn(0x7f176815a9f0 0x7f176806c540 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:31:03.353 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:31:03.350+0000 7f175ffff640 1 -- 172.21.15.37:0/1959887498 >> v1:172.21.15.37:6789/0 conn(0x7f1768160e50 legacy=0x7f1768145a30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:31:03.353 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:31:03.350+0000 7f175ffff640 1 -- 172.21.15.37:0/1959887498 --> v2:172.21.15.37:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f1768163e00 con 0x7f176806ca80 2024-04-25T10:31:03.354 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:31:03.351+0000 7f175ffff640 1 --2- 172.21.15.37:0/1959887498 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f176806ca80 0x7f1768160910 secure :-1 s=READY pgs=697 cs=0 l=1 rev1=1 crypto rx=0x7f1768071cc0 tx=0x7f175802a100 comp rx=0 tx=0).ready entity=mon.0 client_cookie=dee60b6a265a416a server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:31:03.354 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:31:03.351+0000 7f175f7fe640 1 -- 172.21.15.37:0/1959887498 <== mon.0 v2:172.21.15.37:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1758037070 con 0x7f176806ca80 2024-04-25T10:31:03.354 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:31:03.351+0000 7f175f7fe640 1 -- 172.21.15.37:0/1959887498 <== mon.0 v2:172.21.15.37:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f175802ad90 con 0x7f176806ca80 2024-04-25T10:31:03.355 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:31:03.351+0000 7f175f7fe640 1 -- 172.21.15.37:0/1959887498 <== mon.0 v2:172.21.15.37:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1758032660 con 0x7f176806ca80 2024-04-25T10:31:03.355 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:31:03.352+0000 7f176efe3640 1 -- 172.21.15.37:0/1959887498 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f176806ca80 msgr2=0x7f1768160910 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:31:03.355 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:31:03.352+0000 7f176efe3640 1 --2- 172.21.15.37:0/1959887498 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f176806ca80 0x7f1768160910 secure :-1 s=READY pgs=697 cs=0 l=1 rev1=1 crypto rx=0x7f1768071cc0 tx=0x7f175802a100 comp rx=0 tx=0).stop 2024-04-25T10:31:03.355 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:31:03.352+0000 7f176efe3640 1 -- 172.21.15.37:0/1959887498 shutdown_connections 2024-04-25T10:31:03.355 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:31:03.352+0000 7f176efe3640 1 --2- 172.21.15.37:0/1959887498 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f176806ca80 0x7f1768160910 unknown :-1 s=CLOSED pgs=697 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:31:03.355 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:31:03.352+0000 7f176efe3640 1 --2- 172.21.15.37:0/1959887498 >> v2:172.21.15.203:3300/0 conn(0x7f176815a9f0 0x7f176806c540 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:31:03.355 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:31:03.352+0000 7f176efe3640 1 -- 172.21.15.37:0/1959887498 >> 172.21.15.37:0/1959887498 conn(0x7f17680b4560 msgr2=0x7f17680b4970 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:31:03.355 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:31:03.352+0000 7f176efe3640 1 -- 172.21.15.37:0/1959887498 shutdown_connections 2024-04-25T10:31:03.355 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:31:03.352+0000 7f176efe3640 1 -- 172.21.15.37:0/1959887498 wait complete. 2024-04-25T10:31:03.356 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:31:03.354+0000 7f176efe3640 1 Processor -- start 2024-04-25T10:31:03.356 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:31:03.354+0000 7f176efe3640 1 -- start start 2024-04-25T10:31:03.359 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:31:03.355+0000 7f176efe3640 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f176806ca80 0x7f176814dc20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:31:03.359 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:31:03.355+0000 7f176efe3640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f176815a9f0 0x7f17681571f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:31:03.359 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:31:03.355+0000 7f176efe3640 1 --2- >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f1768160e50 0x7f1768161f00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:31:03.359 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:31:03.355+0000 7f176efe3640 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 -- 0x7f17681646a0 con 0x7f176815a9f0 2024-04-25T10:31:03.359 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:31:03.355+0000 7f176efe3640 1 -- --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_getmap magic: 0 -- 0x7f17681643c0 con 0x7f176806ca80 2024-04-25T10:31:03.359 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:31:03.355+0000 7f176efe3640 1 -- --> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] -- mon_getmap magic: 0 -- 0x7f1768164bc0 con 0x7f1768160e50 2024-04-25T10:31:03.359 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:31:03.355+0000 7f175ffff640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f176815a9f0 0x7f17681571f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:31:03.359 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:31:03.355+0000 7f175ffff640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f176815a9f0 0x7f17681571f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.37:3300/0 says I am v2:172.21.15.37:34538/0 (socket says 172.21.15.37:34538) 2024-04-25T10:31:03.359 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:31:03.355+0000 7f175ffff640 1 -- 172.21.15.37:0/1795842482 learned_addr learned my addr 172.21.15.37:0/1795842482 (peer_addr_for_me v2:172.21.15.37:0/0) 2024-04-25T10:31:03.359 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:31:03.355+0000 7f175ffff640 1 -- 172.21.15.37:0/1795842482 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f1768160e50 msgr2=0x7f1768161f00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:31:03.360 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:31:03.355+0000 7f176d580640 1 --2- 172.21.15.37:0/1795842482 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f1768160e50 0x7f1768161f00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:31:03.360 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:31:03.355+0000 7f176cd7f640 1 --2- 172.21.15.37:0/1795842482 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f176806ca80 0x7f176814dc20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:31:03.360 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:31:03.356+0000 7f175ffff640 1 --2- 172.21.15.37:0/1795842482 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f1768160e50 0x7f1768161f00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:31:03.360 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:31:03.356+0000 7f175ffff640 1 -- 172.21.15.37:0/1795842482 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f176806ca80 msgr2=0x7f176814dc20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:31:03.360 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:31:03.356+0000 7f175ffff640 1 --2- 172.21.15.37:0/1795842482 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f176806ca80 0x7f176814dc20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:31:03.360 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:31:03.356+0000 7f175ffff640 1 -- 172.21.15.37:0/1795842482 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f17681f9b70 con 0x7f176815a9f0 2024-04-25T10:31:03.360 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:31:03.356+0000 7f176d580640 1 --2- 172.21.15.37:0/1795842482 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f1768160e50 0x7f1768161f00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T10:31:03.360 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:31:03.356+0000 7f175ffff640 1 --2- 172.21.15.37:0/1795842482 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f176815a9f0 0x7f17681571f0 secure :-1 s=READY pgs=698 cs=0 l=1 rev1=1 crypto rx=0x7f1758000c00 tx=0x7f17580046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:31:03.360 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:31:03.356+0000 7f175d7fa640 1 -- 172.21.15.37:0/1795842482 <== mon.0 v2:172.21.15.37:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1758037040 con 0x7f176815a9f0 2024-04-25T10:31:03.360 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:31:03.356+0000 7f175d7fa640 1 -- 172.21.15.37:0/1795842482 <== mon.0 v2:172.21.15.37:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f1758004030 con 0x7f176815a9f0 2024-04-25T10:31:03.360 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:31:03.356+0000 7f175d7fa640 1 -- 172.21.15.37:0/1795842482 <== mon.0 v2:172.21.15.37:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1758004400 con 0x7f176815a9f0 2024-04-25T10:31:03.360 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:31:03.356+0000 7f176efe3640 1 -- 172.21.15.37:0/1795842482 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f17681fa070 con 0x7f176815a9f0 2024-04-25T10:31:03.360 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:31:03.356+0000 7f176efe3640 1 -- 172.21.15.37:0/1795842482 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f17681fa630 con 0x7f176815a9f0 2024-04-25T10:31:03.360 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:31:03.357+0000 7f176efe3640 1 -- 172.21.15.37:0/1795842482 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f17681f9de0 con 0x7f176815a9f0 2024-04-25T10:31:03.361 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:31:03.358+0000 7f175d7fa640 1 -- 172.21.15.37:0/1795842482 <== mon.0 v2:172.21.15.37:3300/0 4 ==== mgrmap(e 5) ==== 96782+0+0 (secure 0 0 0) 0x7f1758003610 con 0x7f176815a9f0 2024-04-25T10:31:03.362 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:31:03.358+0000 7f175d7fa640 1 --2- 172.21.15.37:0/1795842482 >> [v2:172.21.15.203:6832/2418205237,v1:172.21.15.203:6833/2418205237] conn(0x7f17540744b0 0x7f17540769c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:31:03.362 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:31:03.358+0000 7f175d7fa640 1 -- 172.21.15.37:0/1795842482 <== mon.0 v2:172.21.15.37:3300/0 5 ==== osd_map(50..50 src has 1..50) ==== 5029+0+0 (secure 0 0 0) 0x7f17580b5830 con 0x7f176815a9f0 2024-04-25T10:31:03.362 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:31:03.358+0000 7f176cd7f640 1 --2- 172.21.15.37:0/1795842482 >> [v2:172.21.15.203:6832/2418205237,v1:172.21.15.203:6833/2418205237] conn(0x7f17540744b0 0x7f17540769c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:31:03.362 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:31:03.359+0000 7f176cd7f640 1 --2- 172.21.15.37:0/1795842482 >> [v2:172.21.15.203:6832/2418205237,v1:172.21.15.203:6833/2418205237] conn(0x7f17540744b0 0x7f17540769c0 secure :-1 s=READY pgs=315 cs=0 l=1 rev1=1 crypto rx=0x7f1764006fd0 tx=0x7f1764008040 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:31:03.365 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:31:03.363+0000 7f175d7fa640 1 -- 172.21.15.37:0/1795842482 <== mon.0 v2:172.21.15.37:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187246 (secure 0 0 0) 0x7f17580802f0 con 0x7f176815a9f0 2024-04-25T10:31:03.550 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:31:03.548+0000 7f176efe3640 1 -- 172.21.15.37:0/1795842482 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_command({"prefix": "config rm", "who": "mon", "name": "mon_allow_pool_delete"} v 0) -- 0x7f17681f9de0 con 0x7f176815a9f0 2024-04-25T10:31:03.554 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:31:03.552+0000 7f175d7fa640 1 -- 172.21.15.37:0/1795842482 <== mon.0 v2:172.21.15.37:3300/0 7 ==== mon_command_ack([{"prefix": "config rm", "who": "mon", "name": "mon_allow_pool_delete"}]=0 v13) ==== 104+0+0 (secure 0 0 0) 0x7f175807fbf0 con 0x7f176815a9f0 2024-04-25T10:31:03.556 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:31:03.553+0000 7f176efe3640 1 -- 172.21.15.37:0/1795842482 >> [v2:172.21.15.203:6832/2418205237,v1:172.21.15.203:6833/2418205237] conn(0x7f17540744b0 msgr2=0x7f17540769c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:31:03.556 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:31:03.554+0000 7f176efe3640 1 --2- 172.21.15.37:0/1795842482 >> [v2:172.21.15.203:6832/2418205237,v1:172.21.15.203:6833/2418205237] conn(0x7f17540744b0 0x7f17540769c0 secure :-1 s=READY pgs=315 cs=0 l=1 rev1=1 crypto rx=0x7f1764006fd0 tx=0x7f1764008040 comp rx=0 tx=0).stop 2024-04-25T10:31:03.556 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:31:03.554+0000 7f176efe3640 1 -- 172.21.15.37:0/1795842482 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f176815a9f0 msgr2=0x7f17681571f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:31:03.556 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:31:03.554+0000 7f176efe3640 1 --2- 172.21.15.37:0/1795842482 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f176815a9f0 0x7f17681571f0 secure :-1 s=READY pgs=698 cs=0 l=1 rev1=1 crypto rx=0x7f1758000c00 tx=0x7f17580046c0 comp rx=0 tx=0).stop 2024-04-25T10:31:03.556 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:31:03.554+0000 7f176efe3640 1 -- 172.21.15.37:0/1795842482 shutdown_connections 2024-04-25T10:31:03.556 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:31:03.554+0000 7f176efe3640 1 --2- 172.21.15.37:0/1795842482 >> [v2:172.21.15.203:6832/2418205237,v1:172.21.15.203:6833/2418205237] conn(0x7f17540744b0 0x7f17540769c0 unknown :-1 s=CLOSED pgs=315 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:31:03.557 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:31:03.554+0000 7f176efe3640 1 --2- 172.21.15.37:0/1795842482 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f1768160e50 0x7f1768161f00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:31:03.557 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:31:03.554+0000 7f176efe3640 1 --2- 172.21.15.37:0/1795842482 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f176815a9f0 0x7f17681571f0 unknown :-1 s=CLOSED pgs=698 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:31:03.557 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:31:03.554+0000 7f176efe3640 1 --2- 172.21.15.37:0/1795842482 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f176806ca80 0x7f176814dc20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:31:03.557 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:31:03.554+0000 7f176efe3640 1 -- 172.21.15.37:0/1795842482 >> 172.21.15.37:0/1795842482 conn(0x7f17680b4560 msgr2=0x7f17680b4970 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:31:03.557 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:31:03.554+0000 7f176efe3640 1 -- 172.21.15.37:0/1795842482 shutdown_connections 2024-04-25T10:31:03.557 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:31:03.554+0000 7f176efe3640 1 -- 172.21.15.37:0/1795842482 wait complete. 2024-04-25T10:31:03.567 DEBUG:teuthology.orchestra.run.smithi037:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph log 'Ended test tasks.cephfs.test_recovery_pool.TestRecoveryPool.test_rebuild_simple' 2024-04-25T10:31:03.678 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:31:03.676+0000 7f48c5300640 1 Processor -- start 2024-04-25T10:31:03.679 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:31:03.676+0000 7f48c5300640 1 -- start start 2024-04-25T10:31:03.680 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:31:03.678+0000 7f48c5300640 1 --2- >> v2:172.21.15.37:3300/0 conn(0x7f48c00b4e80 0x7f48c00d2b20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:31:03.680 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:31:03.678+0000 7f48c5300640 1 --2- >> v2:172.21.15.203:3300/0 conn(0x7f48c00d3060 0x7f48c00b9060 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:31:03.680 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:31:03.678+0000 7f48c5300640 1 -- --> v1:172.21.15.203:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f48c00d3470 con 0x7f48c00d5160 2024-04-25T10:31:03.680 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:31:03.678+0000 7f48c5300640 1 -- --> v2:172.21.15.37:3300/0 -- mon_getmap magic: 0 -- 0x7f48c00d1650 con 0x7f48c00b4e80 2024-04-25T10:31:03.681 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:31:03.678+0000 7f48c5300640 1 -- --> v2:172.21.15.203:3300/0 -- mon_getmap magic: 0 -- 0x7f48c00b5290 con 0x7f48c00d3060 2024-04-25T10:31:03.681 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:31:03.678+0000 7f48be7fc640 1 --2- >> v2:172.21.15.37:3300/0 conn(0x7f48c00b4e80 0x7f48c00d2b20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:31:03.681 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:31:03.678+0000 7f48be7fc640 1 --2- >> v2:172.21.15.37:3300/0 conn(0x7f48c00b4e80 0x7f48c00d2b20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.37:3300/0 says I am v2:172.21.15.37:34542/0 (socket says 172.21.15.37:34542) 2024-04-25T10:31:03.681 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:31:03.678+0000 7f48be7fc640 1 -- 172.21.15.37:0/746939727 learned_addr learned my addr 172.21.15.37:0/746939727 (peer_addr_for_me v2:172.21.15.37:0/0) 2024-04-25T10:31:03.681 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:31:03.678+0000 7f48bf7fe640 1 --2- 172.21.15.37:0/746939727 >> v2:172.21.15.203:3300/0 conn(0x7f48c00d3060 0x7f48c00b9060 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:31:03.681 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:31:03.678+0000 7f48be7fc640 1 -- 172.21.15.37:0/746939727 >> v2:172.21.15.203:3300/0 conn(0x7f48c00d3060 msgr2=0x7f48c00b9060 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:31:03.681 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:31:03.678+0000 7f48be7fc640 1 --2- 172.21.15.37:0/746939727 >> v2:172.21.15.203:3300/0 conn(0x7f48c00d3060 0x7f48c00b9060 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:31:03.681 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:31:03.678+0000 7f48be7fc640 1 -- 172.21.15.37:0/746939727 >> v1:172.21.15.203:6789/0 conn(0x7f48c00d5160 legacy=0x7f48c00d0990 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:31:03.681 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:31:03.678+0000 7f48be7fc640 1 -- 172.21.15.37:0/746939727 --> v2:172.21.15.37:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f48c00d10a0 con 0x7f48c00b4e80 2024-04-25T10:31:03.681 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:31:03.678+0000 7f48bf7fe640 1 --2- 172.21.15.37:0/746939727 >> v2:172.21.15.203:3300/0 conn(0x7f48c00d3060 0x7f48c00b9060 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T10:31:03.682 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:31:03.679+0000 7f48be7fc640 1 --2- 172.21.15.37:0/746939727 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f48c00b4e80 0x7f48c00d2b20 secure :-1 s=READY pgs=699 cs=0 l=1 rev1=1 crypto rx=0x7f48c008f050 tx=0x7f48a802a1a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=86c952e07b9d2c32 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:31:03.682 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:31:03.679+0000 7f48bdffb640 1 -- 172.21.15.37:0/746939727 <== mon.0 v2:172.21.15.37:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f48a8037070 con 0x7f48c00b4e80 2024-04-25T10:31:03.683 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:31:03.679+0000 7f48bdffb640 1 -- 172.21.15.37:0/746939727 <== mon.0 v2:172.21.15.37:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f48a8032440 con 0x7f48c00b4e80 2024-04-25T10:31:03.683 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:31:03.679+0000 7f48bdffb640 1 -- 172.21.15.37:0/746939727 <== mon.0 v2:172.21.15.37:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f48a8032810 con 0x7f48c00b4e80 2024-04-25T10:31:03.683 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:31:03.680+0000 7f48c5300640 1 -- 172.21.15.37:0/746939727 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f48c00b4e80 msgr2=0x7f48c00d2b20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:31:03.683 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:31:03.680+0000 7f48c5300640 1 --2- 172.21.15.37:0/746939727 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f48c00b4e80 0x7f48c00d2b20 secure :-1 s=READY pgs=699 cs=0 l=1 rev1=1 crypto rx=0x7f48c008f050 tx=0x7f48a802a1a0 comp rx=0 tx=0).stop 2024-04-25T10:31:03.683 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:31:03.680+0000 7f48c5300640 1 -- 172.21.15.37:0/746939727 shutdown_connections 2024-04-25T10:31:03.683 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:31:03.680+0000 7f48c5300640 1 --2- 172.21.15.37:0/746939727 >> v2:172.21.15.203:3300/0 conn(0x7f48c00d3060 0x7f48c00b9060 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:31:03.683 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:31:03.680+0000 7f48c5300640 1 --2- 172.21.15.37:0/746939727 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f48c00b4e80 0x7f48c00d2b20 unknown :-1 s=CLOSED pgs=699 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:31:03.683 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:31:03.680+0000 7f48c5300640 1 -- 172.21.15.37:0/746939727 >> 172.21.15.37:0/746939727 conn(0x7f48c00b7a80 msgr2=0x7f48c00b7e90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:31:03.683 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:31:03.680+0000 7f48c5300640 1 -- 172.21.15.37:0/746939727 shutdown_connections 2024-04-25T10:31:03.683 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:31:03.680+0000 7f48c5300640 1 -- 172.21.15.37:0/746939727 wait complete. 2024-04-25T10:31:03.684 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:31:03.682+0000 7f48c5300640 1 Processor -- start 2024-04-25T10:31:03.684 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:31:03.682+0000 7f48c5300640 1 -- start start 2024-04-25T10:31:03.687 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:31:03.683+0000 7f48c5300640 1 --2- >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f48c00b4e80 0x7f48c0154b00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:31:03.687 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:31:03.683+0000 7f48c5300640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f48c00d3060 0x7f48c0156560 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:31:03.687 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:31:03.683+0000 7f48c5300640 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f48c00d5160 0x7f48c0156aa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:31:03.687 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:31:03.683+0000 7f48c5300640 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 -- 0x7f48c00d4960 con 0x7f48c00d3060 2024-04-25T10:31:03.687 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:31:03.683+0000 7f48c5300640 1 -- --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_getmap magic: 0 -- 0x7f48c00d4680 con 0x7f48c00d5160 2024-04-25T10:31:03.687 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:31:03.683+0000 7f48c5300640 1 -- --> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] -- mon_getmap magic: 0 -- 0x7f48c00d4c40 con 0x7f48c00b4e80 2024-04-25T10:31:03.687 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:31:03.683+0000 7f48be7fc640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f48c00d3060 0x7f48c0156560 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:31:03.687 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:31:03.683+0000 7f48be7fc640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f48c00d3060 0x7f48c0156560 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.37:3300/0 says I am v2:172.21.15.37:34546/0 (socket says 172.21.15.37:34546) 2024-04-25T10:31:03.687 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:31:03.683+0000 7f48be7fc640 1 -- 172.21.15.37:0/436698702 learned_addr learned my addr 172.21.15.37:0/436698702 (peer_addr_for_me v2:172.21.15.37:0/0) 2024-04-25T10:31:03.687 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:31:03.683+0000 7f48beffd640 1 --2- 172.21.15.37:0/436698702 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f48c00b4e80 0x7f48c0154b00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:31:03.687 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:31:03.683+0000 7f48bf7fe640 1 --2- 172.21.15.37:0/436698702 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f48c00d5160 0x7f48c0156aa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:31:03.687 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:31:03.683+0000 7f48be7fc640 1 -- 172.21.15.37:0/436698702 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f48c00b4e80 msgr2=0x7f48c0154b00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:31:03.687 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:31:03.683+0000 7f48be7fc640 1 --2- 172.21.15.37:0/436698702 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f48c00b4e80 0x7f48c0154b00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:31:03.688 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:31:03.683+0000 7f48be7fc640 1 -- 172.21.15.37:0/436698702 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f48c00d5160 msgr2=0x7f48c0156aa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:31:03.688 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:31:03.683+0000 7f48be7fc640 1 --2- 172.21.15.37:0/436698702 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f48c00d5160 0x7f48c0156aa0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:31:03.688 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:31:03.683+0000 7f48be7fc640 1 -- 172.21.15.37:0/436698702 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f48c0156fe0 con 0x7f48c00d3060 2024-04-25T10:31:03.688 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:31:03.683+0000 7f48bf7fe640 1 --2- 172.21.15.37:0/436698702 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f48c00d5160 0x7f48c0156aa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T10:31:03.688 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:31:03.683+0000 7f48beffd640 1 --2- 172.21.15.37:0/436698702 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f48c00b4e80 0x7f48c0154b00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T10:31:03.688 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:31:03.684+0000 7f48be7fc640 1 --2- 172.21.15.37:0/436698702 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f48c00d3060 0x7f48c0156560 secure :-1 s=READY pgs=700 cs=0 l=1 rev1=1 crypto rx=0x7f48a8000c00 tx=0x7f48a80046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:31:03.688 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:31:03.684+0000 7f48b77fe640 1 -- 172.21.15.37:0/436698702 <== mon.0 v2:172.21.15.37:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f48a8041020 con 0x7f48c00d3060 2024-04-25T10:31:03.688 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:31:03.684+0000 7f48b77fe640 1 -- 172.21.15.37:0/436698702 <== mon.0 v2:172.21.15.37:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f48a8030970 con 0x7f48c00d3060 2024-04-25T10:31:03.688 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:31:03.684+0000 7f48c5300640 1 -- 172.21.15.37:0/436698702 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f48c01ea150 con 0x7f48c00d3060 2024-04-25T10:31:03.688 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:31:03.684+0000 7f48b77fe640 1 -- 172.21.15.37:0/436698702 <== mon.0 v2:172.21.15.37:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f48a8037040 con 0x7f48c00d3060 2024-04-25T10:31:03.688 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:31:03.684+0000 7f48c5300640 1 -- 172.21.15.37:0/436698702 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f48c01ea710 con 0x7f48c00d3060 2024-04-25T10:31:03.688 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:31:03.685+0000 7f48c5300640 1 -- 172.21.15.37:0/436698702 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f48c006a140 con 0x7f48c00d3060 2024-04-25T10:31:03.689 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:31:03.685+0000 7f48b77fe640 1 -- 172.21.15.37:0/436698702 <== mon.0 v2:172.21.15.37:3300/0 4 ==== mgrmap(e 5) ==== 96782+0+0 (secure 0 0 0) 0x7f48a8044020 con 0x7f48c00d3060 2024-04-25T10:31:03.689 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:31:03.686+0000 7f48b77fe640 1 --2- 172.21.15.37:0/436698702 >> [v2:172.21.15.203:6832/2418205237,v1:172.21.15.203:6833/2418205237] conn(0x7f48a00744b0 0x7f48a00769c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:31:03.689 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:31:03.686+0000 7f48b77fe640 1 -- 172.21.15.37:0/436698702 <== mon.0 v2:172.21.15.37:3300/0 5 ==== osd_map(50..50 src has 1..50) ==== 5029+0+0 (secure 0 0 0) 0x7f48a80b59e0 con 0x7f48c00d3060 2024-04-25T10:31:03.689 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:31:03.686+0000 7f48beffd640 1 --2- 172.21.15.37:0/436698702 >> [v2:172.21.15.203:6832/2418205237,v1:172.21.15.203:6833/2418205237] conn(0x7f48a00744b0 0x7f48a00769c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:31:03.689 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:31:03.687+0000 7f48beffd640 1 --2- 172.21.15.37:0/436698702 >> [v2:172.21.15.203:6832/2418205237,v1:172.21.15.203:6833/2418205237] conn(0x7f48a00744b0 0x7f48a00769c0 secure :-1 s=READY pgs=316 cs=0 l=1 rev1=1 crypto rx=0x7f48b8006fd0 tx=0x7f48b8008040 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:31:03.693 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:31:03.690+0000 7f48b77fe640 1 -- 172.21.15.37:0/436698702 <== mon.0 v2:172.21.15.37:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187246 (secure 0 0 0) 0x7f48a804b030 con 0x7f48c00d3060 2024-04-25T10:31:03.952 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:31:03.950+0000 7f48c5300640 1 -- 172.21.15.37:0/436698702 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_command({"prefix": "log", "logtext": ["Ended test tasks.cephfs.test_recovery_pool.TestRecoveryPool.test_rebuild_simple"]} v 0) -- 0x7f48c015ada0 con 0x7f48c00d3060 2024-04-25T10:31:04.873 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:31:04.871+0000 7f48b77fe640 1 -- 172.21.15.37:0/436698702 <== mon.0 v2:172.21.15.37:3300/0 7 ==== mon_command_ack([{"prefix": "log", "logtext": ["Ended test tasks.cephfs.test_recovery_pool.TestRecoveryPool.test_rebuild_simple"]}]=0 v131) ==== 147+0+0 (secure 0 0 0) 0x7f48c00d3470 con 0x7f48c00d3060 2024-04-25T10:31:04.876 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:31:04.874+0000 7f48c5300640 1 -- 172.21.15.37:0/436698702 >> [v2:172.21.15.203:6832/2418205237,v1:172.21.15.203:6833/2418205237] conn(0x7f48a00744b0 msgr2=0x7f48a00769c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:31:04.876 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:31:04.874+0000 7f48c5300640 1 --2- 172.21.15.37:0/436698702 >> [v2:172.21.15.203:6832/2418205237,v1:172.21.15.203:6833/2418205237] conn(0x7f48a00744b0 0x7f48a00769c0 secure :-1 s=READY pgs=316 cs=0 l=1 rev1=1 crypto rx=0x7f48b8006fd0 tx=0x7f48b8008040 comp rx=0 tx=0).stop 2024-04-25T10:31:04.876 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:31:04.874+0000 7f48c5300640 1 -- 172.21.15.37:0/436698702 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f48c00d3060 msgr2=0x7f48c0156560 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:31:04.877 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:31:04.874+0000 7f48c5300640 1 --2- 172.21.15.37:0/436698702 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f48c00d3060 0x7f48c0156560 secure :-1 s=READY pgs=700 cs=0 l=1 rev1=1 crypto rx=0x7f48a8000c00 tx=0x7f48a80046c0 comp rx=0 tx=0).stop 2024-04-25T10:31:04.877 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:31:04.874+0000 7f48c5300640 1 -- 172.21.15.37:0/436698702 shutdown_connections 2024-04-25T10:31:04.877 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:31:04.874+0000 7f48c5300640 1 --2- 172.21.15.37:0/436698702 >> [v2:172.21.15.203:6832/2418205237,v1:172.21.15.203:6833/2418205237] conn(0x7f48a00744b0 0x7f48a00769c0 unknown :-1 s=CLOSED pgs=316 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:31:04.877 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:31:04.874+0000 7f48c5300640 1 --2- 172.21.15.37:0/436698702 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f48c00d5160 0x7f48c0156aa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:31:04.877 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:31:04.874+0000 7f48c5300640 1 --2- 172.21.15.37:0/436698702 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f48c00d3060 0x7f48c0156560 unknown :-1 s=CLOSED pgs=700 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:31:04.877 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:31:04.874+0000 7f48c5300640 1 --2- 172.21.15.37:0/436698702 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f48c00b4e80 0x7f48c0154b00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:31:04.877 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:31:04.874+0000 7f48c5300640 1 -- 172.21.15.37:0/436698702 >> 172.21.15.37:0/436698702 conn(0x7f48c00b7a80 msgr2=0x7f48c015ec10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:31:04.877 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:31:04.874+0000 7f48c5300640 1 -- 172.21.15.37:0/436698702 shutdown_connections 2024-04-25T10:31:04.877 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:31:04.874+0000 7f48c5300640 1 -- 172.21.15.37:0/436698702 wait complete. 2024-04-25T10:31:04.890 INFO:tasks.cephfs_test_runner:test_rebuild_simple (tasks.cephfs.test_recovery_pool.TestRecoveryPool) ... ok 2024-04-25T10:31:04.891 INFO:tasks.cephfs_test_runner: 2024-04-25T10:31:04.891 INFO:tasks.cephfs_test_runner:---------------------------------------------------------------------- 2024-04-25T10:31:04.891 INFO:tasks.cephfs_test_runner:Ran 1 test in 83.547s 2024-04-25T10:31:04.891 INFO:tasks.cephfs_test_runner: 2024-04-25T10:31:04.891 INFO:tasks.cephfs_test_runner:OK 2024-04-25T10:31:04.891 DEBUG:teuthology.run_tasks:Unwinding manager cephfs_test_runner 2024-04-25T10:31:04.901 DEBUG:teuthology.run_tasks:Unwinding manager ceph-fuse 2024-04-25T10:31:04.909 INFO:tasks.ceph_fuse:Unmounting ceph-fuse clients... 2024-04-25T10:31:04.909 DEBUG:teuthology.orchestra.run.smithi037:> set -ex 2024-04-25T10:31:04.909 DEBUG:teuthology.orchestra.run.smithi037:> dd if=/proc/self/mounts of=/dev/stdout 2024-04-25T10:31:04.927 DEBUG:tasks.cephfs.mount:not mounted; /proc/self/mounts is: proc /proc proc rw,nosuid,nodev,noexec,relatime 0 0 sysfs /sys sysfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 devtmpfs /dev devtmpfs rw,seclabel,nosuid,size=4096k,nr_inodes=4056882,mode=755,inode64 0 0 securityfs /sys/kernel/security securityfs rw,nosuid,nodev,noexec,relatime 0 0 tmpfs /dev/shm tmpfs rw,seclabel,nosuid,nodev,inode64 0 0 devpts /dev/pts devpts rw,seclabel,nosuid,noexec,relatime,gid=5,mode=620,ptmxmode=000 0 0 tmpfs /run tmpfs rw,seclabel,nosuid,nodev,size=6498932k,nr_inodes=819200,mode=755,inode64 0 0 cgroup2 /sys/fs/cgroup cgroup2 rw,seclabel,nosuid,nodev,noexec,relatime,nsdelegate,memory_recursiveprot 0 0 pstore /sys/fs/pstore pstore rw,seclabel,nosuid,nodev,noexec,relatime 0 0 bpf /sys/fs/bpf bpf rw,nosuid,nodev,noexec,relatime,mode=700 0 0 /dev/sda1 / ext4 rw,seclabel,relatime 0 0 selinuxfs /sys/fs/selinux selinuxfs rw,nosuid,noexec,relatime 0 0 systemd-1 /proc/sys/fs/binfmt_misc autofs rw,relatime,fd=29,pgrp=1,timeout=0,minproto=5,maxproto=5,direct,pipe_ino=21000 0 0 hugetlbfs /dev/hugepages hugetlbfs rw,seclabel,relatime,pagesize=2M 0 0 mqueue /dev/mqueue mqueue rw,seclabel,nosuid,nodev,noexec,relatime 0 0 debugfs /sys/kernel/debug debugfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 tracefs /sys/kernel/tracing tracefs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 fusectl /sys/fs/fuse/connections fusectl rw,nosuid,nodev,noexec,relatime 0 0 configfs /sys/kernel/config configfs rw,nosuid,nodev,noexec,relatime 0 0 none /run/credentials/systemd-sysusers.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 none /run/credentials/systemd-sysctl.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 none /run/credentials/systemd-tmpfiles-setup-dev.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 none /run/credentials/systemd-tmpfiles-setup.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 sunrpc /var/lib/nfs/rpc_pipefs rpc_pipefs rw,relatime 0 0 tmpfs /run/user/1000 tmpfs rw,seclabel,nosuid,nodev,relatime,size=3249464k,nr_inodes=812366,mode=700,uid=1000,gid=1316,inode64 0 0 /dev/mapper/vg_nvme-lv_5 /var/lib/ceph xfs rw,seclabel,relatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_1 /var/lib/ceph/osd/ceph-0 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_2 /var/lib/ceph/osd/ceph-1 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_3 /var/lib/ceph/osd/ceph-2 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_4 /var/lib/ceph/osd/ceph-3 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 tmpfs /run/netns tmpfs rw,seclabel,nosuid,nodev,size=6498932k,nr_inodes=819200,mode=755,inode64 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 nsfs rw 0 0 2024-04-25T10:31:04.927 DEBUG:teuthology.orchestra.run.smithi037:> set -ex 2024-04-25T10:31:04.927 DEBUG:teuthology.orchestra.run.smithi037:> dd if=/proc/self/mounts of=/dev/stdout 2024-04-25T10:31:04.985 DEBUG:tasks.cephfs.mount:not mounted; /proc/self/mounts is: proc /proc proc rw,nosuid,nodev,noexec,relatime 0 0 sysfs /sys sysfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 devtmpfs /dev devtmpfs rw,seclabel,nosuid,size=4096k,nr_inodes=4056882,mode=755,inode64 0 0 securityfs /sys/kernel/security securityfs rw,nosuid,nodev,noexec,relatime 0 0 tmpfs /dev/shm tmpfs rw,seclabel,nosuid,nodev,inode64 0 0 devpts /dev/pts devpts rw,seclabel,nosuid,noexec,relatime,gid=5,mode=620,ptmxmode=000 0 0 tmpfs /run tmpfs rw,seclabel,nosuid,nodev,size=6498932k,nr_inodes=819200,mode=755,inode64 0 0 cgroup2 /sys/fs/cgroup cgroup2 rw,seclabel,nosuid,nodev,noexec,relatime,nsdelegate,memory_recursiveprot 0 0 pstore /sys/fs/pstore pstore rw,seclabel,nosuid,nodev,noexec,relatime 0 0 bpf /sys/fs/bpf bpf rw,nosuid,nodev,noexec,relatime,mode=700 0 0 /dev/sda1 / ext4 rw,seclabel,relatime 0 0 selinuxfs /sys/fs/selinux selinuxfs rw,nosuid,noexec,relatime 0 0 systemd-1 /proc/sys/fs/binfmt_misc autofs rw,relatime,fd=29,pgrp=1,timeout=0,minproto=5,maxproto=5,direct,pipe_ino=21000 0 0 hugetlbfs /dev/hugepages hugetlbfs rw,seclabel,relatime,pagesize=2M 0 0 mqueue /dev/mqueue mqueue rw,seclabel,nosuid,nodev,noexec,relatime 0 0 debugfs /sys/kernel/debug debugfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 tracefs /sys/kernel/tracing tracefs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 fusectl /sys/fs/fuse/connections fusectl rw,nosuid,nodev,noexec,relatime 0 0 configfs /sys/kernel/config configfs rw,nosuid,nodev,noexec,relatime 0 0 none /run/credentials/systemd-sysusers.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 none /run/credentials/systemd-sysctl.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 none /run/credentials/systemd-tmpfiles-setup-dev.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 none /run/credentials/systemd-tmpfiles-setup.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 sunrpc /var/lib/nfs/rpc_pipefs rpc_pipefs rw,relatime 0 0 tmpfs /run/user/1000 tmpfs rw,seclabel,nosuid,nodev,relatime,size=3249464k,nr_inodes=812366,mode=700,uid=1000,gid=1316,inode64 0 0 /dev/mapper/vg_nvme-lv_5 /var/lib/ceph xfs rw,seclabel,relatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_1 /var/lib/ceph/osd/ceph-0 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_2 /var/lib/ceph/osd/ceph-1 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_3 /var/lib/ceph/osd/ceph-2 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_4 /var/lib/ceph/osd/ceph-3 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 tmpfs /run/netns tmpfs rw,seclabel,nosuid,nodev,size=6498932k,nr_inodes=819200,mode=755,inode64 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 nsfs rw 0 0 2024-04-25T10:31:04.985 DEBUG:teuthology.orchestra.run.smithi203:> set -ex 2024-04-25T10:31:04.985 DEBUG:teuthology.orchestra.run.smithi203:> dd if=/proc/self/mounts of=/dev/stdout 2024-04-25T10:31:05.004 DEBUG:tasks.cephfs.mount:not mounted; /proc/self/mounts is: proc /proc proc rw,nosuid,nodev,noexec,relatime 0 0 sysfs /sys sysfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 devtmpfs /dev devtmpfs rw,seclabel,nosuid,size=4096k,nr_inodes=4056088,mode=755,inode64 0 0 securityfs /sys/kernel/security securityfs rw,nosuid,nodev,noexec,relatime 0 0 tmpfs /dev/shm tmpfs rw,seclabel,nosuid,nodev,inode64 0 0 devpts /dev/pts devpts rw,seclabel,nosuid,noexec,relatime,gid=5,mode=620,ptmxmode=000 0 0 tmpfs /run tmpfs rw,seclabel,nosuid,nodev,size=6497660k,nr_inodes=819200,mode=755,inode64 0 0 cgroup2 /sys/fs/cgroup cgroup2 rw,seclabel,nosuid,nodev,noexec,relatime,nsdelegate,memory_recursiveprot 0 0 pstore /sys/fs/pstore pstore rw,seclabel,nosuid,nodev,noexec,relatime 0 0 bpf /sys/fs/bpf bpf rw,nosuid,nodev,noexec,relatime,mode=700 0 0 /dev/sda1 / ext4 rw,seclabel,relatime 0 0 selinuxfs /sys/fs/selinux selinuxfs rw,nosuid,noexec,relatime 0 0 systemd-1 /proc/sys/fs/binfmt_misc autofs rw,relatime,fd=29,pgrp=1,timeout=0,minproto=5,maxproto=5,direct,pipe_ino=181 0 0 hugetlbfs /dev/hugepages hugetlbfs rw,seclabel,relatime,pagesize=2M 0 0 mqueue /dev/mqueue mqueue rw,seclabel,nosuid,nodev,noexec,relatime 0 0 debugfs /sys/kernel/debug debugfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 tracefs /sys/kernel/tracing tracefs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 configfs /sys/kernel/config configfs rw,nosuid,nodev,noexec,relatime 0 0 none /run/credentials/systemd-sysusers.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 fusectl /sys/fs/fuse/connections fusectl rw,nosuid,nodev,noexec,relatime 0 0 none /run/credentials/systemd-sysctl.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 none /run/credentials/systemd-tmpfiles-setup-dev.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 none /run/credentials/systemd-tmpfiles-setup.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 sunrpc /var/lib/nfs/rpc_pipefs rpc_pipefs rw,relatime 0 0 tmpfs /run/user/1000 tmpfs rw,seclabel,nosuid,nodev,relatime,size=3248828k,nr_inodes=812207,mode=700,uid=1000,gid=1316,inode64 0 0 /dev/mapper/vg_nvme-lv_5 /var/lib/ceph xfs rw,seclabel,relatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_1 /var/lib/ceph/osd/ceph-4 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_2 /var/lib/ceph/osd/ceph-5 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_3 /var/lib/ceph/osd/ceph-6 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_4 /var/lib/ceph/osd/ceph-7 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 tmpfs /run/netns tmpfs rw,seclabel,nosuid,nodev,size=6497660k,nr_inodes=819200,mode=755,inode64 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 nsfs rw 0 0 2024-04-25T10:31:05.005 DEBUG:teuthology.orchestra.run.smithi203:> set -ex 2024-04-25T10:31:05.005 DEBUG:teuthology.orchestra.run.smithi203:> dd if=/proc/self/mounts of=/dev/stdout 2024-04-25T10:31:05.063 DEBUG:tasks.cephfs.mount:not mounted; /proc/self/mounts is: proc /proc proc rw,nosuid,nodev,noexec,relatime 0 0 sysfs /sys sysfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 devtmpfs /dev devtmpfs rw,seclabel,nosuid,size=4096k,nr_inodes=4056088,mode=755,inode64 0 0 securityfs /sys/kernel/security securityfs rw,nosuid,nodev,noexec,relatime 0 0 tmpfs /dev/shm tmpfs rw,seclabel,nosuid,nodev,inode64 0 0 devpts /dev/pts devpts rw,seclabel,nosuid,noexec,relatime,gid=5,mode=620,ptmxmode=000 0 0 tmpfs /run tmpfs rw,seclabel,nosuid,nodev,size=6497660k,nr_inodes=819200,mode=755,inode64 0 0 cgroup2 /sys/fs/cgroup cgroup2 rw,seclabel,nosuid,nodev,noexec,relatime,nsdelegate,memory_recursiveprot 0 0 pstore /sys/fs/pstore pstore rw,seclabel,nosuid,nodev,noexec,relatime 0 0 bpf /sys/fs/bpf bpf rw,nosuid,nodev,noexec,relatime,mode=700 0 0 /dev/sda1 / ext4 rw,seclabel,relatime 0 0 selinuxfs /sys/fs/selinux selinuxfs rw,nosuid,noexec,relatime 0 0 systemd-1 /proc/sys/fs/binfmt_misc autofs rw,relatime,fd=29,pgrp=1,timeout=0,minproto=5,maxproto=5,direct,pipe_ino=181 0 0 hugetlbfs /dev/hugepages hugetlbfs rw,seclabel,relatime,pagesize=2M 0 0 mqueue /dev/mqueue mqueue rw,seclabel,nosuid,nodev,noexec,relatime 0 0 debugfs /sys/kernel/debug debugfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 tracefs /sys/kernel/tracing tracefs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 configfs /sys/kernel/config configfs rw,nosuid,nodev,noexec,relatime 0 0 none /run/credentials/systemd-sysusers.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 fusectl /sys/fs/fuse/connections fusectl rw,nosuid,nodev,noexec,relatime 0 0 none /run/credentials/systemd-sysctl.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 none /run/credentials/systemd-tmpfiles-setup-dev.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 none /run/credentials/systemd-tmpfiles-setup.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 sunrpc /var/lib/nfs/rpc_pipefs rpc_pipefs rw,relatime 0 0 tmpfs /run/user/1000 tmpfs rw,seclabel,nosuid,nodev,relatime,size=3248828k,nr_inodes=812207,mode=700,uid=1000,gid=1316,inode64 0 0 /dev/mapper/vg_nvme-lv_5 /var/lib/ceph xfs rw,seclabel,relatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_1 /var/lib/ceph/osd/ceph-4 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_2 /var/lib/ceph/osd/ceph-5 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_3 /var/lib/ceph/osd/ceph-6 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_4 /var/lib/ceph/osd/ceph-7 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 tmpfs /run/netns tmpfs rw,seclabel,nosuid,nodev,size=6497660k,nr_inodes=819200,mode=755,inode64 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 nsfs rw 0 0 2024-04-25T10:31:05.064 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T10:31:05.064 DEBUG:teuthology.orchestra.run.smithi037:> ip netns list 2024-04-25T10:31:05.081 INFO:teuthology.orchestra.run.smithi037.stdout:ceph-ns--home-ubuntu-cephtest-mnt.2 (id: 1) 2024-04-25T10:31:05.082 INFO:teuthology.orchestra.run.smithi037.stdout:ceph-ns--home-ubuntu-cephtest-mnt.0 (id: 0) 2024-04-25T10:31:05.082 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T10:31:05.082 DEBUG:teuthology.orchestra.run.smithi037:> sudo ip netns delete ceph-ns--home-ubuntu-cephtest-mnt.2 2024-04-25T10:31:05.149 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T10:31:05.149 DEBUG:teuthology.orchestra.run.smithi037:> sudo ip netns delete ceph-ns--home-ubuntu-cephtest-mnt.0 2024-04-25T10:31:05.217 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T10:31:05.217 DEBUG:teuthology.orchestra.run.smithi037:> sudo ip link delete ceph-brx 2024-04-25T10:31:05.331 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T10:31:05.332 DEBUG:teuthology.orchestra.run.smithi203:> ip netns list 2024-04-25T10:31:05.350 INFO:teuthology.orchestra.run.smithi203.stdout:ceph-ns--home-ubuntu-cephtest-mnt.3 (id: 1) 2024-04-25T10:31:05.350 INFO:teuthology.orchestra.run.smithi203.stdout:ceph-ns--home-ubuntu-cephtest-mnt.1 (id: 0) 2024-04-25T10:31:05.350 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T10:31:05.350 DEBUG:teuthology.orchestra.run.smithi203:> sudo ip netns delete ceph-ns--home-ubuntu-cephtest-mnt.3 2024-04-25T10:31:05.418 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T10:31:05.418 DEBUG:teuthology.orchestra.run.smithi203:> sudo ip netns delete ceph-ns--home-ubuntu-cephtest-mnt.1 2024-04-25T10:31:05.486 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T10:31:05.487 DEBUG:teuthology.orchestra.run.smithi203:> sudo ip link delete ceph-brx 2024-04-25T10:31:05.590 DEBUG:teuthology.run_tasks:Unwinding manager ceph 2024-04-25T10:31:05.600 INFO:tasks.ceph.ceph_manager.ceph:waiting for clean 2024-04-25T10:31:05.600 DEBUG:teuthology.orchestra.run.smithi037:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-04-25T10:31:05.718 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:31:05.716+0000 7f7ea0a84640 1 Processor -- start 2024-04-25T10:31:05.718 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:31:05.716+0000 7f7ea0a84640 1 -- start start 2024-04-25T10:31:05.720 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:31:05.717+0000 7f7ea0a84640 1 --2- >> v2:172.21.15.37:3300/0 conn(0x7f7e9c06f100 0x7f7e9c1610b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:31:05.720 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:31:05.717+0000 7f7ea0a84640 1 -- --> v1:172.21.15.37:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f7e9c161730 con 0x7f7e9c143500 2024-04-25T10:31:05.720 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:31:05.717+0000 7f7ea0a84640 1 -- --> v1:172.21.15.203:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f7e9c0b5b60 con 0x7f7e9c1460e0 2024-04-25T10:31:05.720 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:31:05.717+0000 7f7ea0a84640 1 -- --> v2:172.21.15.37:3300/0 -- mon_getmap magic: 0 -- 0x7f7e9c06f510 con 0x7f7e9c06f100 2024-04-25T10:31:05.720 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:31:05.717+0000 7f7e9ad9d640 1 --2- >> v2:172.21.15.37:3300/0 conn(0x7f7e9c06f100 0x7f7e9c1610b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:31:05.720 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:31:05.717+0000 7f7e9ad9d640 1 --2- >> v2:172.21.15.37:3300/0 conn(0x7f7e9c06f100 0x7f7e9c1610b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.37:3300/0 says I am v2:172.21.15.37:34558/0 (socket says 172.21.15.37:34558) 2024-04-25T10:31:05.720 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:31:05.717+0000 7f7e9ad9d640 1 -- 172.21.15.37:0/447552246 learned_addr learned my addr 172.21.15.37:0/447552246 (peer_addr_for_me v2:172.21.15.37:0/0) 2024-04-25T10:31:05.720 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:31:05.717+0000 7f7e9a59c640 1 --1- 172.21.15.37:0/447552246 >> v1:172.21.15.37:6789/0 conn(0x7f7e9c143500 0x7f7e9c1609a0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.37:6789/0 says I am v1:172.21.15.37:47064/0 (socket says 172.21.15.37:47064) 2024-04-25T10:31:05.721 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:31:05.718+0000 7f7e9ad9d640 1 -- 172.21.15.37:0/447552246 >> v1:172.21.15.203:6789/0 conn(0x7f7e9c1460e0 legacy=0x7f7e9c143b30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:31:05.721 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:31:05.718+0000 7f7e9ad9d640 1 -- 172.21.15.37:0/447552246 >> v1:172.21.15.37:6789/0 conn(0x7f7e9c143500 legacy=0x7f7e9c1609a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:31:05.721 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:31:05.718+0000 7f7e99d9b640 1 -- 172.21.15.37:0/447552246 <== mon.0 v1:172.21.15.37:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 2814608522 0 0) 0x7f7e9c161730 con 0x7f7e9c143500 2024-04-25T10:31:05.721 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:31:05.718+0000 7f7e9ad9d640 1 -- 172.21.15.37:0/447552246 --> v2:172.21.15.37:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f7e9c0b5e40 con 0x7f7e9c06f100 2024-04-25T10:31:05.722 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:31:05.719+0000 7f7e9ad9d640 1 --2- 172.21.15.37:0/447552246 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f7e9c06f100 0x7f7e9c1610b0 secure :-1 s=READY pgs=702 cs=0 l=1 rev1=1 crypto rx=0x7f7e9400d150 tx=0x7f7e9402c9c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c5cfa6eff0766cc5 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:31:05.722 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:31:05.719+0000 7f7e99d9b640 1 -- 172.21.15.37:0/447552246 <== mon.0 v2:172.21.15.37:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7e94035020 con 0x7f7e9c06f100 2024-04-25T10:31:05.722 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:31:05.719+0000 7f7e99d9b640 1 -- 172.21.15.37:0/447552246 <== mon.0 v2:172.21.15.37:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f7e9400b2d0 con 0x7f7e9c06f100 2024-04-25T10:31:05.722 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:31:05.719+0000 7f7e99d9b640 1 -- 172.21.15.37:0/447552246 <== mon.0 v2:172.21.15.37:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7e9400b480 con 0x7f7e9c06f100 2024-04-25T10:31:05.722 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:31:05.719+0000 7f7ea0a84640 1 -- 172.21.15.37:0/447552246 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f7e9c06f100 msgr2=0x7f7e9c1610b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:31:05.722 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:31:05.719+0000 7f7ea0a84640 1 --2- 172.21.15.37:0/447552246 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f7e9c06f100 0x7f7e9c1610b0 secure :-1 s=READY pgs=702 cs=0 l=1 rev1=1 crypto rx=0x7f7e9400d150 tx=0x7f7e9402c9c0 comp rx=0 tx=0).stop 2024-04-25T10:31:05.722 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:31:05.719+0000 7f7ea0a84640 1 -- 172.21.15.37:0/447552246 shutdown_connections 2024-04-25T10:31:05.722 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:31:05.719+0000 7f7ea0a84640 1 --2- 172.21.15.37:0/447552246 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f7e9c06f100 0x7f7e9c1610b0 unknown :-1 s=CLOSED pgs=702 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:31:05.722 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:31:05.719+0000 7f7ea0a84640 1 -- 172.21.15.37:0/447552246 >> 172.21.15.37:0/447552246 conn(0x7f7e9c0b36b0 msgr2=0x7f7e9c0b3ac0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:31:05.723 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:31:05.719+0000 7f7ea0a84640 1 -- 172.21.15.37:0/447552246 shutdown_connections 2024-04-25T10:31:05.723 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:31:05.719+0000 7f7ea0a84640 1 -- 172.21.15.37:0/447552246 wait complete. 2024-04-25T10:31:05.723 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:31:05.721+0000 7f7ea0a84640 1 Processor -- start 2024-04-25T10:31:05.723 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:31:05.721+0000 7f7ea0a84640 1 -- start start 2024-04-25T10:31:05.726 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:31:05.723+0000 7f7ea0a84640 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f7e9c06f100 0x7f7e9c0aa160 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:31:05.727 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:31:05.723+0000 7f7ea0a84640 1 --2- >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f7e9c1460e0 0x7f7e9c1410a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:31:05.727 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:31:05.723+0000 7f7ea0a84640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f7e9c1415e0 0x7f7e9c15d490 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:31:05.727 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:31:05.723+0000 7f7ea0a84640 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 -- 0x7f7e9c0b67c0 con 0x7f7e9c1415e0 2024-04-25T10:31:05.727 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:31:05.723+0000 7f7ea0a84640 1 -- --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_getmap magic: 0 -- 0x7f7e9c0b6640 con 0x7f7e9c06f100 2024-04-25T10:31:05.727 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:31:05.723+0000 7f7ea0a84640 1 -- --> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] -- mon_getmap magic: 0 -- 0x7f7e9c0b6940 con 0x7f7e9c1460e0 2024-04-25T10:31:05.727 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:31:05.723+0000 7f7e9ad9d640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f7e9c1415e0 0x7f7e9c15d490 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:31:05.727 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:31:05.723+0000 7f7e9ad9d640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f7e9c1415e0 0x7f7e9c15d490 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.37:3300/0 says I am v2:172.21.15.37:34560/0 (socket says 172.21.15.37:34560) 2024-04-25T10:31:05.727 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:31:05.723+0000 7f7e9ad9d640 1 -- 172.21.15.37:0/3803461570 learned_addr learned my addr 172.21.15.37:0/3803461570 (peer_addr_for_me v2:172.21.15.37:0/0) 2024-04-25T10:31:05.727 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:31:05.723+0000 7f7e9a59c640 1 --2- 172.21.15.37:0/3803461570 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f7e9c06f100 0x7f7e9c0aa160 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:31:05.727 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:31:05.723+0000 7f7e93fff640 1 --2- 172.21.15.37:0/3803461570 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f7e9c1460e0 0x7f7e9c1410a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:31:05.727 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:31:05.723+0000 7f7e9ad9d640 1 -- 172.21.15.37:0/3803461570 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f7e9c1460e0 msgr2=0x7f7e9c1410a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:31:05.727 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:31:05.723+0000 7f7e9ad9d640 1 --2- 172.21.15.37:0/3803461570 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f7e9c1460e0 0x7f7e9c1410a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:31:05.728 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:31:05.723+0000 7f7e9ad9d640 1 -- 172.21.15.37:0/3803461570 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f7e9c06f100 msgr2=0x7f7e9c0aa160 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:31:05.728 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:31:05.723+0000 7f7e9ad9d640 1 --2- 172.21.15.37:0/3803461570 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f7e9c06f100 0x7f7e9c0aa160 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:31:05.728 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:31:05.723+0000 7f7e9ad9d640 1 -- 172.21.15.37:0/3803461570 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f7e9c0aa6a0 con 0x7f7e9c1415e0 2024-04-25T10:31:05.728 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:31:05.723+0000 7f7e9ad9d640 1 --2- 172.21.15.37:0/3803461570 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f7e9c1415e0 0x7f7e9c15d490 secure :-1 s=READY pgs=703 cs=0 l=1 rev1=1 crypto rx=0x7f7e9400d150 tx=0x7f7e94004a40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:31:05.728 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:31:05.724+0000 7f7e92ffd640 1 -- 172.21.15.37:0/3803461570 <== mon.0 v2:172.21.15.37:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7e94035040 con 0x7f7e9c1415e0 2024-04-25T10:31:05.728 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:31:05.724+0000 7f7e92ffd640 1 -- 172.21.15.37:0/3803461570 <== mon.0 v2:172.21.15.37:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f7e94031a50 con 0x7f7e9c1415e0 2024-04-25T10:31:05.728 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:31:05.724+0000 7f7ea0a84640 1 -- 172.21.15.37:0/3803461570 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f7e9c12e200 con 0x7f7e9c1415e0 2024-04-25T10:31:05.728 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:31:05.724+0000 7f7e92ffd640 1 -- 172.21.15.37:0/3803461570 <== mon.0 v2:172.21.15.37:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7e9400b480 con 0x7f7e9c1415e0 2024-04-25T10:31:05.728 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:31:05.724+0000 7f7ea0a84640 1 -- 172.21.15.37:0/3803461570 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f7e9c12e7c0 con 0x7f7e9c1415e0 2024-04-25T10:31:05.728 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:31:05.725+0000 7f7ea0a84640 1 -- 172.21.15.37:0/3803461570 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f7e9c06a140 con 0x7f7e9c1415e0 2024-04-25T10:31:05.729 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:31:05.725+0000 7f7e92ffd640 1 -- 172.21.15.37:0/3803461570 <== mon.0 v2:172.21.15.37:3300/0 4 ==== mgrmap(e 5) ==== 96782+0+0 (secure 0 0 0) 0x7f7e94038020 con 0x7f7e9c1415e0 2024-04-25T10:31:05.729 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:31:05.726+0000 7f7e92ffd640 1 --2- 172.21.15.37:0/3803461570 >> [v2:172.21.15.203:6832/2418205237,v1:172.21.15.203:6833/2418205237] conn(0x7f7e7c0744b0 0x7f7e7c0769c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:31:05.729 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:31:05.726+0000 7f7e92ffd640 1 -- 172.21.15.37:0/3803461570 <== mon.0 v2:172.21.15.37:3300/0 5 ==== osd_map(50..50 src has 1..50) ==== 5029+0+0 (secure 0 0 0) 0x7f7e940b7db0 con 0x7f7e9c1415e0 2024-04-25T10:31:05.729 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:31:05.726+0000 7f7e9a59c640 1 --2- 172.21.15.37:0/3803461570 >> [v2:172.21.15.203:6832/2418205237,v1:172.21.15.203:6833/2418205237] conn(0x7f7e7c0744b0 0x7f7e7c0769c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:31:05.729 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:31:05.727+0000 7f7e9a59c640 1 --2- 172.21.15.37:0/3803461570 >> [v2:172.21.15.203:6832/2418205237,v1:172.21.15.203:6833/2418205237] conn(0x7f7e7c0744b0 0x7f7e7c0769c0 secure :-1 s=READY pgs=317 cs=0 l=1 rev1=1 crypto rx=0x7f7e8c000f60 tx=0x7f7e8c007e40 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:31:05.733 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:31:05.731+0000 7f7e92ffd640 1 -- 172.21.15.37:0/3803461570 <== mon.0 v2:172.21.15.37:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187246 (secure 0 0 0) 0x7f7e940bc070 con 0x7f7e9c1415e0 2024-04-25T10:31:05.915 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:31:05.913+0000 7f7ea0a84640 1 -- 172.21.15.37:0/3803461570 --> [v2:172.21.15.203:6832/2418205237,v1:172.21.15.203:6833/2418205237] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) -- 0x7f7e9c130e70 con 0x7f7e7c0744b0 2024-04-25T10:31:05.916 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:31:05.914+0000 7f7e92ffd640 1 -- 172.21.15.37:0/3803461570 <== mgr.4100 v2:172.21.15.203:6832/2418205237 1 ==== mgr_command_reply(tid 0: 0 dumped all) ==== 18+0+13497 (secure 0 0 0) 0x7f7e9c130e70 con 0x7f7e7c0744b0 2024-04-25T10:31:05.916 INFO:teuthology.orchestra.run.smithi037.stdout: 2024-04-25T10:31:05.916 INFO:teuthology.orchestra.run.smithi037.stderr:dumped all 2024-04-25T10:31:05.918 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:31:05.916+0000 7f7ea0a84640 1 -- 172.21.15.37:0/3803461570 >> [v2:172.21.15.203:6832/2418205237,v1:172.21.15.203:6833/2418205237] conn(0x7f7e7c0744b0 msgr2=0x7f7e7c0769c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:31:05.919 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:31:05.916+0000 7f7ea0a84640 1 --2- 172.21.15.37:0/3803461570 >> [v2:172.21.15.203:6832/2418205237,v1:172.21.15.203:6833/2418205237] conn(0x7f7e7c0744b0 0x7f7e7c0769c0 secure :-1 s=READY pgs=317 cs=0 l=1 rev1=1 crypto rx=0x7f7e8c000f60 tx=0x7f7e8c007e40 comp rx=0 tx=0).stop 2024-04-25T10:31:05.919 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:31:05.916+0000 7f7ea0a84640 1 -- 172.21.15.37:0/3803461570 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f7e9c1415e0 msgr2=0x7f7e9c15d490 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:31:05.919 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:31:05.916+0000 7f7ea0a84640 1 --2- 172.21.15.37:0/3803461570 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f7e9c1415e0 0x7f7e9c15d490 secure :-1 s=READY pgs=703 cs=0 l=1 rev1=1 crypto rx=0x7f7e9400d150 tx=0x7f7e94004a40 comp rx=0 tx=0).stop 2024-04-25T10:31:05.919 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:31:05.916+0000 7f7ea0a84640 1 -- 172.21.15.37:0/3803461570 shutdown_connections 2024-04-25T10:31:05.919 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:31:05.916+0000 7f7ea0a84640 1 --2- 172.21.15.37:0/3803461570 >> [v2:172.21.15.203:6832/2418205237,v1:172.21.15.203:6833/2418205237] conn(0x7f7e7c0744b0 0x7f7e7c0769c0 unknown :-1 s=CLOSED pgs=317 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:31:05.919 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:31:05.916+0000 7f7ea0a84640 1 --2- 172.21.15.37:0/3803461570 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f7e9c1415e0 0x7f7e9c15d490 unknown :-1 s=CLOSED pgs=703 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:31:05.919 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:31:05.916+0000 7f7ea0a84640 1 --2- 172.21.15.37:0/3803461570 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f7e9c1460e0 0x7f7e9c1410a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:31:05.919 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:31:05.916+0000 7f7ea0a84640 1 --2- 172.21.15.37:0/3803461570 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f7e9c06f100 0x7f7e9c0aa160 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:31:05.919 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:31:05.916+0000 7f7ea0a84640 1 -- 172.21.15.37:0/3803461570 >> 172.21.15.37:0/3803461570 conn(0x7f7e9c0b36b0 msgr2=0x7f7e9c0703a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:31:05.920 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:31:05.916+0000 7f7ea0a84640 1 -- 172.21.15.37:0/3803461570 shutdown_connections 2024-04-25T10:31:05.920 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:31:05.916+0000 7f7ea0a84640 1 -- 172.21.15.37:0/3803461570 wait complete. 2024-04-25T10:31:05.929 INFO:teuthology.orchestra.run.smithi037.stdout:{"pg_ready":true,"pg_map":{"version":109,"stamp":"2024-04-25T10:31:04.215830+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":66,"num_read_kb":55,"num_write":101,"num_write_kb":1628,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":72,"ondisk_log_size":72,"up":2,"acting":2,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":194,"num_osds":8,"num_per_pool_osds":8,"num_per_pool_omap_osds":7,"kb":754974720,"kb_used":238116,"kb_used_data":17796,"kb_used_omap":12,"kb_used_meta":220211,"kb_avail":754736604,"statfs":{"total":773094113280,"available":772850282496,"internally_reserved":0,"allocated":18223104,"data_stored":15638395,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":12752,"internal_metadata":225496624},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":14,"apply_latency_ms":14,"commit_latency_ns":14000000,"apply_latency_ns":14000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":-6322167,"num_objects":-15,"num_object_clones":0,"num_object_copies":-30,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":-15,"num_whiteouts":0,"num_read":-53,"num_read_kb":-39,"num_write":-50,"num_write_kb":-6193,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":-5,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"6.001408"},"pg_stats":[{"pgid":"1.0","version":"18'72","reported_seq":156,"reported_epoch":48,"state":"active+clean","last_fresh":"2024-04-25T10:30:59.592109+0000","last_change":"2024-04-25T10:28:57.972696+0000","last_active":"2024-04-25T10:30:59.592109+0000","last_peered":"2024-04-25T10:30:59.592109+0000","last_clean":"2024-04-25T10:30:59.592109+0000","last_became_active":"2024-04-25T10:28:57.972373+0000","last_became_peered":"2024-04-25T10:28:57.972373+0000","last_unstale":"2024-04-25T10:30:59.592109+0000","last_undegraded":"2024-04-25T10:30:59.592109+0000","last_fullsized":"2024-04-25T10:30:59.592109+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:28:56.959439+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:28:56.959439+0000","last_clean_scrub_stamp":"2024-04-25T10:28:56.959439+0000","objects_scrubbed":0,"log_size":72,"log_dups_size":0,"ondisk_log_size":72,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T17:34:34.057408+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":66,"num_read_kb":55,"num_write":101,"num_write_kb":1628,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]}],"pool_stats":[{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":66,"num_read_kb":55,"num_write":101,"num_write_kb":1628,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1187840,"data_stored":1180736,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":72,"ondisk_log_size":72,"up":2,"acting":2,"num_store_stats":2}],"osd_stats":[{"osd":7,"up_from":15,"seq":64424509468,"num_pgs":19,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":31648,"kb_used_data":4628,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94340192,"statfs":{"total":96636764160,"available":96604356608,"internally_reserved":0,"allocated":4739072,"data_stored":4421615,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1590,"internal_metadata":27654602},"hb_peers":[0,1,2,3,4,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]},{"osd":6,"up_from":15,"seq":64424509469,"num_pgs":26,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27556,"kb_used_data":536,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344284,"statfs":{"total":96636764160,"available":96608546816,"internally_reserved":0,"allocated":548864,"data_stored":227893,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":27654598},"hb_peers":[0,1,2,3,4,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]},{"osd":5,"up_from":15,"seq":64424509470,"num_pgs":20,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":28132,"kb_used_data":1112,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94343708,"statfs":{"total":96636764160,"available":96607956992,"internally_reserved":0,"allocated":1138688,"data_stored":817667,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]},{"osd":3,"up_from":15,"seq":64424509470,"num_pgs":22,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":33732,"kb_used_data":6712,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94338108,"statfs":{"total":96636764160,"available":96602222592,"internally_reserved":0,"allocated":6873088,"data_stored":6548646,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1593,"internal_metadata":27654599},"hb_peers":[0,1,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[]},{"osd":2,"up_from":15,"seq":64424509470,"num_pgs":25,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":29600,"kb_used_data":2580,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94342240,"statfs":{"total":96636764160,"available":96606453760,"internally_reserved":0,"allocated":2641920,"data_stored":2324519,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]},{"osd":1,"up_from":15,"seq":64424509468,"num_pgs":26,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":28144,"kb_used_data":1124,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94343696,"statfs":{"total":96636764160,"available":96607944704,"internally_reserved":0,"allocated":1150976,"data_stored":824013,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]},{"osd":0,"up_from":15,"seq":64424509469,"num_pgs":25,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27568,"kb_used_data":548,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344272,"statfs":{"total":96636764160,"available":96608534528,"internally_reserved":0,"allocated":561152,"data_stored":234247,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[1,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]},{"osd":4,"up_from":13,"seq":55834574877,"num_pgs":31,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":0,"kb":94371840,"kb_used":31736,"kb_used_data":556,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94340104,"statfs":{"total":96636764160,"available":96604266496,"internally_reserved":0,"allocated":569344,"data_stored":239795,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[0,1,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-04-25T10:31:05.930 DEBUG:teuthology.orchestra.run.smithi037:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-04-25T10:31:06.042 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:31:06.040+0000 7f52b034e640 1 Processor -- start 2024-04-25T10:31:06.042 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:31:06.040+0000 7f52b034e640 1 -- start start 2024-04-25T10:31:06.047 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:31:06.041+0000 7f52b034e640 1 --2- >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f52a80b4bb0 0x7f52a81646b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:31:06.047 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:31:06.041+0000 7f52b034e640 1 --2- >> v2:172.21.15.203:3300/0 conn(0x7f52a80b7600 0x7f52a80b7a10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:31:06.047 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:31:06.041+0000 7f52b034e640 1 -- --> v1:172.21.15.37:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f52a80b7f50 con 0x7f52a80d4560 2024-04-25T10:31:06.047 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:31:06.041+0000 7f52b034e640 1 -- --> v2:172.21.15.203:3300/0 -- mon_getmap magic: 0 -- 0x7f52a80d5080 con 0x7f52a80b7600 2024-04-25T10:31:06.047 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:31:06.041+0000 7f52b034e640 1 -- --> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] -- mon_getmap magic: 0 -- 0x7f52a8161190 con 0x7f52a80b4bb0 2024-04-25T10:31:06.047 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:31:06.041+0000 7f52ae8eb640 1 --1- >> v1:172.21.15.37:6789/0 conn(0x7f52a80d4560 0x7f52a80d4970 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.37:6789/0 says I am v1:172.21.15.37:47076/0 (socket says 172.21.15.37:47076) 2024-04-25T10:31:06.047 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:31:06.041+0000 7f52ae8eb640 1 -- 172.21.15.37:0/2920425110 learned_addr learned my addr 172.21.15.37:0/2920425110 (peer_addr_for_me v1:172.21.15.37:0/0) 2024-04-25T10:31:06.047 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:31:06.041+0000 7f52ae0ea640 1 --2- 172.21.15.37:0/2920425110 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f52a80b4bb0 0x7f52a81646b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:31:06.047 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:31:06.041+0000 7f52ad8e9640 1 --2- 172.21.15.37:0/2920425110 >> v2:172.21.15.203:3300/0 conn(0x7f52a80b7600 0x7f52a80b7a10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:31:06.047 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:31:06.042+0000 7f52ad0e8640 1 -- 172.21.15.37:0/2920425110 <== mon.0 v1:172.21.15.37:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 865627971 0 0) 0x7f52a4001490 con 0x7f52a80d4560 2024-04-25T10:31:06.048 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:31:06.042+0000 7f52ad0e8640 1 -- 172.21.15.37:0/2920425110 --> v1:172.21.15.37:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f5294003520 con 0x7f52a80d4560 2024-04-25T10:31:06.048 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:31:06.042+0000 7f52ad0e8640 1 -- 172.21.15.37:0/2920425110 <== mon.0 v1:172.21.15.37:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 40274536 0 0) 0x7f5294003520 con 0x7f52a80d4560 2024-04-25T10:31:06.048 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:31:06.042+0000 7f52ad0e8640 1 -- 172.21.15.37:0/2920425110 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f52a80b4bb0 msgr2=0x7f52a81646b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:31:06.048 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:31:06.042+0000 7f52ad0e8640 1 --2- 172.21.15.37:0/2920425110 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f52a80b4bb0 0x7f52a81646b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:31:06.048 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:31:06.042+0000 7f52ad0e8640 1 -- 172.21.15.37:0/2920425110 >> v2:172.21.15.203:3300/0 conn(0x7f52a80b7600 msgr2=0x7f52a80b7a10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:31:06.048 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:31:06.042+0000 7f52ad0e8640 1 --2- 172.21.15.37:0/2920425110 >> v2:172.21.15.203:3300/0 conn(0x7f52a80b7600 0x7f52a80b7a10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:31:06.048 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:31:06.042+0000 7f52ad0e8640 1 -- 172.21.15.37:0/2920425110 --> v1:172.21.15.37:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f52a8161470 con 0x7f52a80d4560 2024-04-25T10:31:06.048 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:31:06.042+0000 7f52ad0e8640 1 -- 172.21.15.37:0/2920425110 <== mon.0 v1:172.21.15.37:6789/0 3 ==== mon_map magic: 0 ==== 413+0+0 (unknown 1577794600 0 0) 0x7f52a4001bc0 con 0x7f52a80d4560 2024-04-25T10:31:06.048 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:31:06.042+0000 7f52ad0e8640 1 -- 172.21.15.37:0/2920425110 >> v1:172.21.15.37:6789/0 conn(0x7f52a80d4560 legacy=0x7f52a80d4970 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:31:06.048 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:31:06.042+0000 7f52ad0e8640 1 --2- 172.21.15.37:0/2920425110 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f52940039b0 0x7f5294003e20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:31:06.048 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:31:06.042+0000 7f52ad0e8640 1 --2- 172.21.15.37:0/2920425110 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f5294004800 0x7f5294004c10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:31:06.048 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:31:06.042+0000 7f52ad0e8640 1 --2- 172.21.15.37:0/2920425110 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f52a80b4bb0 0x7f5294005150 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:31:06.048 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:31:06.042+0000 7f52ad0e8640 1 -- 172.21.15.37:0/2920425110 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 -- 0x7f52a80d5080 con 0x7f52940039b0 2024-04-25T10:31:06.048 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:31:06.042+0000 7f52ad0e8640 1 -- 172.21.15.37:0/2920425110 --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_getmap magic: 0 -- 0x7f52a8161190 con 0x7f5294004800 2024-04-25T10:31:06.049 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:31:06.042+0000 7f52ad0e8640 1 -- 172.21.15.37:0/2920425110 --> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] -- mon_getmap magic: 0 -- 0x7f529400b9c0 con 0x7f52a80b4bb0 2024-04-25T10:31:06.049 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:31:06.042+0000 7f52ae0ea640 1 --2- 172.21.15.37:0/2920425110 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f52940039b0 0x7f5294003e20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:31:06.049 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:31:06.042+0000 7f52ad8e9640 1 --2- 172.21.15.37:0/2920425110 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f5294004800 0x7f5294004c10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:31:06.049 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:31:06.042+0000 7f52ae8eb640 1 --2- 172.21.15.37:0/2920425110 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f52a80b4bb0 0x7f5294005150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:31:06.049 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:31:06.042+0000 7f52ae0ea640 1 -- 172.21.15.37:0/2920425110 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f52a80b4bb0 msgr2=0x7f5294005150 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:31:06.049 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:31:06.042+0000 7f52ae0ea640 1 --2- 172.21.15.37:0/2920425110 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f52a80b4bb0 0x7f5294005150 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:31:06.049 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:31:06.042+0000 7f52ae0ea640 1 -- 172.21.15.37:0/2920425110 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f5294004800 msgr2=0x7f5294004c10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:31:06.049 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:31:06.042+0000 7f52ae0ea640 1 --2- 172.21.15.37:0/2920425110 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f5294004800 0x7f5294004c10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:31:06.049 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:31:06.042+0000 7f52ae0ea640 1 -- 172.21.15.37:0/2920425110 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f5294003520 con 0x7f52940039b0 2024-04-25T10:31:06.049 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:31:06.042+0000 7f52ae8eb640 1 --2- 172.21.15.37:0/2920425110 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f52a80b4bb0 0x7f5294005150 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T10:31:06.049 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:31:06.043+0000 7f52ae0ea640 1 --2- 172.21.15.37:0/2920425110 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f52940039b0 0x7f5294003e20 secure :-1 s=READY pgs=705 cs=0 l=1 rev1=1 crypto rx=0x7f52a0009a90 tx=0x7f5294005830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f2dc5168715bccf5 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:31:06.049 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:31:06.043+0000 7f52ad0e8640 1 -- 172.21.15.37:0/2920425110 <== mon.0 v2:172.21.15.37:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f52a0036070 con 0x7f52940039b0 2024-04-25T10:31:06.049 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:31:06.043+0000 7f52ad0e8640 1 -- 172.21.15.37:0/2920425110 <== mon.0 v2:172.21.15.37:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f52a0031400 con 0x7f52940039b0 2024-04-25T10:31:06.049 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:31:06.043+0000 7f52ad0e8640 1 -- 172.21.15.37:0/2920425110 <== mon.0 v2:172.21.15.37:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f52a0009690 con 0x7f52940039b0 2024-04-25T10:31:06.050 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:31:06.044+0000 7f52b034e640 1 -- 172.21.15.37:0/2920425110 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f52940039b0 msgr2=0x7f5294003e20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:31:06.050 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:31:06.044+0000 7f52b034e640 1 --2- 172.21.15.37:0/2920425110 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f52940039b0 0x7f5294003e20 secure :-1 s=READY pgs=705 cs=0 l=1 rev1=1 crypto rx=0x7f52a0009a90 tx=0x7f5294005830 comp rx=0 tx=0).stop 2024-04-25T10:31:06.050 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:31:06.044+0000 7f52ae8eb640 1 -- 172.21.15.37:0/2920425110 reap_dead start 2024-04-25T10:31:06.050 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:31:06.044+0000 7f52b034e640 1 -- 172.21.15.37:0/2920425110 shutdown_connections 2024-04-25T10:31:06.050 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:31:06.044+0000 7f52b034e640 1 -- 172.21.15.37:0/2920425110 >> 172.21.15.37:0/2920425110 conn(0x7f52a80d27e0 msgr2=0x7f52a80d2bf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:31:06.050 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:31:06.044+0000 7f52b034e640 1 -- 172.21.15.37:0/2920425110 shutdown_connections 2024-04-25T10:31:06.050 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:31:06.044+0000 7f52b034e640 1 -- 172.21.15.37:0/2920425110 wait complete. 2024-04-25T10:31:06.053 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:31:06.046+0000 7f52b034e640 1 Processor -- start 2024-04-25T10:31:06.053 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:31:06.046+0000 7f52b034e640 1 -- start start 2024-04-25T10:31:06.053 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:31:06.047+0000 7f52b034e640 1 --2- >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f52940039b0 0x7f52a81fa1f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:31:06.053 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:31:06.047+0000 7f52b034e640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f52a815eef0 0x7f52a806dbe0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:31:06.053 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:31:06.047+0000 7f52b034e640 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f52a815f300 0x7f52a806e120 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:31:06.053 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:31:06.047+0000 7f52b034e640 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 -- 0x7f52a80b4fc0 con 0x7f52a815eef0 2024-04-25T10:31:06.053 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:31:06.047+0000 7f52b034e640 1 -- --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_getmap magic: 0 -- 0x7f52a8164190 con 0x7f52a815f300 2024-04-25T10:31:06.054 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:31:06.047+0000 7f52b034e640 1 -- --> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] -- mon_getmap magic: 0 -- 0x7f52a80d09e0 con 0x7f52940039b0 2024-04-25T10:31:06.054 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:31:06.047+0000 7f52ad8e9640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f52a815eef0 0x7f52a806dbe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:31:06.054 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:31:06.047+0000 7f52ad8e9640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f52a815eef0 0x7f52a806dbe0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.37:3300/0 says I am v2:172.21.15.37:34574/0 (socket says 172.21.15.37:34574) 2024-04-25T10:31:06.054 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:31:06.047+0000 7f52ad8e9640 1 -- 172.21.15.37:0/3981296218 learned_addr learned my addr 172.21.15.37:0/3981296218 (peer_addr_for_me v2:172.21.15.37:0/0) 2024-04-25T10:31:06.054 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:31:06.048+0000 7f52ae0ea640 1 --2- 172.21.15.37:0/3981296218 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f52940039b0 0x7f52a81fa1f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:31:06.054 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:31:06.048+0000 7f52ae8eb640 1 --2- 172.21.15.37:0/3981296218 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f52a815f300 0x7f52a806e120 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:31:06.054 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:31:06.048+0000 7f52ad8e9640 1 -- 172.21.15.37:0/3981296218 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f52940039b0 msgr2=0x7f52a81fa1f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:31:06.054 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:31:06.048+0000 7f52ad8e9640 1 --2- 172.21.15.37:0/3981296218 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f52940039b0 0x7f52a81fa1f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:31:06.054 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:31:06.048+0000 7f52ad8e9640 1 -- 172.21.15.37:0/3981296218 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f52a815f300 msgr2=0x7f52a806e120 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:31:06.054 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:31:06.048+0000 7f52ad8e9640 1 --2- 172.21.15.37:0/3981296218 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f52a815f300 0x7f52a806e120 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:31:06.054 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:31:06.048+0000 7f52ad8e9640 1 -- 172.21.15.37:0/3981296218 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f52a81f79b0 con 0x7f52a815eef0 2024-04-25T10:31:06.054 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:31:06.048+0000 7f52ae8eb640 1 --2- 172.21.15.37:0/3981296218 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f52a815f300 0x7f52a806e120 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T10:31:06.054 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:31:06.048+0000 7f52ad8e9640 1 --2- 172.21.15.37:0/3981296218 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f52a815eef0 0x7f52a806dbe0 secure :-1 s=READY pgs=706 cs=0 l=1 rev1=1 crypto rx=0x7f5298011aa0 tx=0x7f5298011f60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:31:06.054 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:31:06.048+0000 7f529effd640 1 -- 172.21.15.37:0/3981296218 <== mon.0 v2:172.21.15.37:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f529800fb80 con 0x7f52a815eef0 2024-04-25T10:31:06.055 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:31:06.048+0000 7f529effd640 1 -- 172.21.15.37:0/3981296218 <== mon.0 v2:172.21.15.37:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f529800fd20 con 0x7f52a815eef0 2024-04-25T10:31:06.055 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:31:06.048+0000 7f52b034e640 1 -- 172.21.15.37:0/3981296218 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f52a81f7c90 con 0x7f52a815eef0 2024-04-25T10:31:06.055 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:31:06.048+0000 7f529effd640 1 -- 172.21.15.37:0/3981296218 <== mon.0 v2:172.21.15.37:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5298014660 con 0x7f52a815eef0 2024-04-25T10:31:06.055 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:31:06.048+0000 7f52b034e640 1 -- 172.21.15.37:0/3981296218 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f52a81f8250 con 0x7f52a815eef0 2024-04-25T10:31:06.055 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:31:06.049+0000 7f52b034e640 1 -- 172.21.15.37:0/3981296218 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f52a806a140 con 0x7f52a815eef0 2024-04-25T10:31:06.055 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:31:06.050+0000 7f529effd640 1 -- 172.21.15.37:0/3981296218 <== mon.0 v2:172.21.15.37:3300/0 4 ==== mgrmap(e 5) ==== 96782+0+0 (secure 0 0 0) 0x7f5298012040 con 0x7f52a815eef0 2024-04-25T10:31:06.055 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:31:06.050+0000 7f529effd640 1 --2- 172.21.15.37:0/3981296218 >> [v2:172.21.15.203:6832/2418205237,v1:172.21.15.203:6833/2418205237] conn(0x7f528c0744b0 0x7f528c0769c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:31:06.055 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:31:06.050+0000 7f529effd640 1 -- 172.21.15.37:0/3981296218 <== mon.0 v2:172.21.15.37:3300/0 5 ==== osd_map(50..50 src has 1..50) ==== 5029+0+0 (secure 0 0 0) 0x7f52980956f0 con 0x7f52a815eef0 2024-04-25T10:31:06.055 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:31:06.050+0000 7f52ae0ea640 1 --2- 172.21.15.37:0/3981296218 >> [v2:172.21.15.203:6832/2418205237,v1:172.21.15.203:6833/2418205237] conn(0x7f528c0744b0 0x7f528c0769c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:31:06.055 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:31:06.051+0000 7f52ae0ea640 1 --2- 172.21.15.37:0/3981296218 >> [v2:172.21.15.203:6832/2418205237,v1:172.21.15.203:6833/2418205237] conn(0x7f528c0744b0 0x7f528c0769c0 secure :-1 s=READY pgs=318 cs=0 l=1 rev1=1 crypto rx=0x7f52a0033000 tx=0x7f52a0008030 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:31:06.057 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:31:06.055+0000 7f529effd640 1 -- 172.21.15.37:0/3981296218 <== mon.0 v2:172.21.15.37:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187246 (secure 0 0 0) 0x7f5298019030 con 0x7f52a815eef0 2024-04-25T10:31:06.237 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:31:06.235+0000 7f52b034e640 1 -- 172.21.15.37:0/3981296218 --> [v2:172.21.15.203:6832/2418205237,v1:172.21.15.203:6833/2418205237] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) -- 0x7f52a80d3ff0 con 0x7f528c0744b0 2024-04-25T10:31:06.238 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:31:06.236+0000 7f529effd640 1 -- 172.21.15.37:0/3981296218 <== mgr.4100 v2:172.21.15.203:6832/2418205237 1 ==== mgr_command_reply(tid 0: 0 dumped all) ==== 18+0+13479 (secure 0 0 0) 0x7f52a80d3ff0 con 0x7f528c0744b0 2024-04-25T10:31:06.238 INFO:teuthology.orchestra.run.smithi037.stdout: 2024-04-25T10:31:06.238 INFO:teuthology.orchestra.run.smithi037.stderr:dumped all 2024-04-25T10:31:06.240 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:31:06.238+0000 7f52b034e640 1 -- 172.21.15.37:0/3981296218 >> [v2:172.21.15.203:6832/2418205237,v1:172.21.15.203:6833/2418205237] conn(0x7f528c0744b0 msgr2=0x7f528c0769c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:31:06.240 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:31:06.238+0000 7f52b034e640 1 --2- 172.21.15.37:0/3981296218 >> [v2:172.21.15.203:6832/2418205237,v1:172.21.15.203:6833/2418205237] conn(0x7f528c0744b0 0x7f528c0769c0 secure :-1 s=READY pgs=318 cs=0 l=1 rev1=1 crypto rx=0x7f52a0033000 tx=0x7f52a0008030 comp rx=0 tx=0).stop 2024-04-25T10:31:06.240 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:31:06.238+0000 7f52b034e640 1 -- 172.21.15.37:0/3981296218 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f52a815eef0 msgr2=0x7f52a806dbe0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:31:06.240 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:31:06.238+0000 7f52b034e640 1 --2- 172.21.15.37:0/3981296218 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f52a815eef0 0x7f52a806dbe0 secure :-1 s=READY pgs=706 cs=0 l=1 rev1=1 crypto rx=0x7f5298011aa0 tx=0x7f5298011f60 comp rx=0 tx=0).stop 2024-04-25T10:31:06.240 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:31:06.238+0000 7f52b034e640 1 -- 172.21.15.37:0/3981296218 shutdown_connections 2024-04-25T10:31:06.241 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:31:06.238+0000 7f52b034e640 1 --2- 172.21.15.37:0/3981296218 >> [v2:172.21.15.203:6832/2418205237,v1:172.21.15.203:6833/2418205237] conn(0x7f528c0744b0 0x7f528c0769c0 unknown :-1 s=CLOSED pgs=318 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:31:06.241 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:31:06.238+0000 7f52b034e640 1 --2- 172.21.15.37:0/3981296218 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f52a815f300 0x7f52a806e120 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:31:06.241 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:31:06.238+0000 7f52b034e640 1 --2- 172.21.15.37:0/3981296218 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f52a815eef0 0x7f52a806dbe0 unknown :-1 s=CLOSED pgs=706 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:31:06.241 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:31:06.238+0000 7f52b034e640 1 --2- 172.21.15.37:0/3981296218 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f52940039b0 0x7f52a81fa1f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:31:06.241 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:31:06.238+0000 7f52b034e640 1 -- 172.21.15.37:0/3981296218 >> 172.21.15.37:0/3981296218 conn(0x7f52a80d27e0 msgr2=0x7f52a815ffd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:31:06.241 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:31:06.238+0000 7f52b034e640 1 -- 172.21.15.37:0/3981296218 shutdown_connections 2024-04-25T10:31:06.241 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:31:06.238+0000 7f52b034e640 1 -- 172.21.15.37:0/3981296218 wait complete. 2024-04-25T10:31:06.252 INFO:teuthology.orchestra.run.smithi037.stdout:{"pg_ready":true,"pg_map":{"version":110,"stamp":"2024-04-25T10:31:06.216214+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":66,"num_read_kb":55,"num_write":101,"num_write_kb":1628,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":72,"ondisk_log_size":72,"up":2,"acting":2,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":68,"num_osds":8,"num_per_pool_osds":8,"num_per_pool_omap_osds":2,"kb":754974720,"kb_used":226048,"kb_used_data":5720,"kb_used_omap":12,"kb_used_meta":220211,"kb_avail":754748672,"statfs":{"total":773094113280,"available":772862640128,"internally_reserved":0,"allocated":5857280,"data_stored":3139164,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":12752,"internal_metadata":225496624},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":15,"apply_latency_ms":15,"commit_latency_ns":15000000,"apply_latency_ns":15000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":-6322167,"num_objects":-15,"num_object_clones":0,"num_object_copies":-30,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":-15,"num_whiteouts":0,"num_read":-53,"num_read_kb":-39,"num_write":-50,"num_write_kb":-6193,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":-5,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"6.622730"},"pg_stats":[{"pgid":"1.0","version":"18'72","reported_seq":160,"reported_epoch":50,"state":"active+clean","last_fresh":"2024-04-25T10:31:03.120107+0000","last_change":"2024-04-25T10:28:57.972696+0000","last_active":"2024-04-25T10:31:03.120107+0000","last_peered":"2024-04-25T10:31:03.120107+0000","last_clean":"2024-04-25T10:31:03.120107+0000","last_became_active":"2024-04-25T10:28:57.972373+0000","last_became_peered":"2024-04-25T10:28:57.972373+0000","last_unstale":"2024-04-25T10:31:03.120107+0000","last_undegraded":"2024-04-25T10:31:03.120107+0000","last_fullsized":"2024-04-25T10:31:03.120107+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:28:56.959439+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:28:56.959439+0000","last_clean_scrub_stamp":"2024-04-25T10:28:56.959439+0000","objects_scrubbed":0,"log_size":72,"log_dups_size":0,"ondisk_log_size":72,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T17:34:34.057408+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":66,"num_read_kb":55,"num_write":101,"num_write_kb":1628,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]}],"pool_stats":[{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":66,"num_read_kb":55,"num_write":101,"num_write_kb":1628,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1187840,"data_stored":1180736,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":72,"ondisk_log_size":72,"up":2,"acting":2,"num_store_stats":2}],"osd_stats":[{"osd":7,"up_from":15,"seq":64424509469,"num_pgs":19,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":0,"kb":94371840,"kb_used":27592,"kb_used_data":572,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344248,"statfs":{"total":96636764160,"available":96608509952,"internally_reserved":0,"allocated":585728,"data_stored":245519,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1590,"internal_metadata":27654602},"hb_peers":[0,1,2,3,4,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[]},{"osd":6,"up_from":15,"seq":64424509470,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":0,"kb":94371840,"kb_used":27592,"kb_used_data":572,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344248,"statfs":{"total":96636764160,"available":96608509952,"internally_reserved":0,"allocated":585728,"data_stored":245519,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":27654598},"hb_peers":[0,1,2,3,4,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]},{"osd":5,"up_from":15,"seq":64424509471,"num_pgs":1,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":28172,"kb_used_data":1152,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94343668,"statfs":{"total":96636764160,"available":96607916032,"internally_reserved":0,"allocated":1179648,"data_stored":835887,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[]},{"osd":3,"up_from":15,"seq":64424509471,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":0,"kb":94371840,"kb_used":27592,"kb_used_data":572,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344248,"statfs":{"total":96636764160,"available":96608509952,"internally_reserved":0,"allocated":585728,"data_stored":245519,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1593,"internal_metadata":27654599},"hb_peers":[0,1,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]},{"osd":2,"up_from":15,"seq":64424509471,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":0,"kb":94371840,"kb_used":27596,"kb_used_data":572,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344244,"statfs":{"total":96636764160,"available":96608505856,"internally_reserved":0,"allocated":585728,"data_stored":245519,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]},{"osd":1,"up_from":15,"seq":64424509469,"num_pgs":1,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":28176,"kb_used_data":1152,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94343664,"statfs":{"total":96636764160,"available":96607911936,"internally_reserved":0,"allocated":1179648,"data_stored":835887,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]},{"osd":0,"up_from":15,"seq":64424509470,"num_pgs":16,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":0,"kb":94371840,"kb_used":27592,"kb_used_data":572,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344248,"statfs":{"total":96636764160,"available":96608509952,"internally_reserved":0,"allocated":585728,"data_stored":245519,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[1,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]},{"osd":4,"up_from":13,"seq":55834574877,"num_pgs":31,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":0,"kb":94371840,"kb_used":31736,"kb_used_data":556,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94340104,"statfs":{"total":96636764160,"available":96604266496,"internally_reserved":0,"allocated":569344,"data_stored":239795,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[0,1,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-04-25T10:31:06.252 INFO:tasks.ceph.ceph_manager.ceph:clean! 2024-04-25T10:31:06.252 DEBUG:teuthology.orchestra.run.smithi037:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-04-25T10:31:06.364 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:31:06.361+0000 7f3d3b785640 1 Processor -- start 2024-04-25T10:31:06.364 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:31:06.362+0000 7f3d3b785640 1 -- start start 2024-04-25T10:31:06.366 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:31:06.363+0000 7f3d3b785640 1 --2- >> v2:172.21.15.203:3300/0 conn(0x7f3d340d5430 0x7f3d340b75c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:31:06.366 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:31:06.363+0000 7f3d3b785640 1 --2- >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f3d34161110 0x7f3d34161880 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:31:06.366 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:31:06.363+0000 7f3d3b785640 1 -- --> v1:172.21.15.203:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f3d340b7f10 con 0x7f3d340b7b00 2024-04-25T10:31:06.366 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:31:06.363+0000 7f3d3b785640 1 -- --> v2:172.21.15.203:3300/0 -- mon_getmap magic: 0 -- 0x7f3d340d4420 con 0x7f3d340d5430 2024-04-25T10:31:06.366 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:31:06.363+0000 7f3d3b785640 1 -- --> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] -- mon_getmap magic: 0 -- 0x7f3d34161520 con 0x7f3d34161110 2024-04-25T10:31:06.366 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:31:06.363+0000 7f3d39d22640 1 --2- >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f3d34161110 0x7f3d34161880 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:31:06.366 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:31:06.363+0000 7f3d39521640 1 --2- >> v2:172.21.15.203:3300/0 conn(0x7f3d340d5430 0x7f3d340b75c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:31:06.366 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:31:06.363+0000 7f3d38d20640 1 --1- >> v1:172.21.15.203:6789/0 conn(0x7f3d340b7b00 0x7f3d34160a00 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.203:6789/0 says I am v1:172.21.15.37:34854/0 (socket says 172.21.15.37:34854) 2024-04-25T10:31:06.367 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:31:06.363+0000 7f3d38d20640 1 -- 172.21.15.37:0/676830140 learned_addr learned my addr 172.21.15.37:0/676830140 (peer_addr_for_me v1:172.21.15.37:0/0) 2024-04-25T10:31:06.367 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:31:06.364+0000 7f3d39d22640 1 -- 172.21.15.37:0/676830140 >> v2:172.21.15.203:3300/0 conn(0x7f3d340d5430 msgr2=0x7f3d340b75c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:31:06.367 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:31:06.364+0000 7f3d39d22640 1 --2- 172.21.15.37:0/676830140 >> v2:172.21.15.203:3300/0 conn(0x7f3d340d5430 0x7f3d340b75c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:31:06.367 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:31:06.364+0000 7f3d39d22640 1 -- 172.21.15.37:0/676830140 >> v1:172.21.15.203:6789/0 conn(0x7f3d340b7b00 legacy=0x7f3d34160a00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:31:06.367 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:31:06.364+0000 7f3d39d22640 1 -- 172.21.15.37:0/676830140 --> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f3d340d5840 con 0x7f3d34161110 2024-04-25T10:31:06.367 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:31:06.364+0000 7f3d39521640 1 --2- 172.21.15.37:0/676830140 >> v2:172.21.15.203:3300/0 conn(0x7f3d340d5430 0x7f3d340b75c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-25T10:31:06.368 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:31:06.364+0000 7f3d39d22640 1 --2- 172.21.15.37:0/676830140 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f3d34161110 0x7f3d34161880 secure :-1 s=READY pgs=64 cs=0 l=1 rev1=1 crypto rx=0x7f3d30009d90 tx=0x7f3d3002a5d0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=da0bfcb5a9cb9ec3 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:31:06.368 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:31:06.365+0000 7f3d2bfff640 1 -- 172.21.15.37:0/676830140 <== mon.2 v2:172.21.15.203:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3d30031600 con 0x7f3d34161110 2024-04-25T10:31:06.368 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:31:06.365+0000 7f3d2bfff640 1 -- 172.21.15.37:0/676830140 <== mon.2 v2:172.21.15.203:3301/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f3d300317a0 con 0x7f3d34161110 2024-04-25T10:31:06.368 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:31:06.365+0000 7f3d2bfff640 1 -- 172.21.15.37:0/676830140 <== mon.2 v2:172.21.15.203:3301/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3d30031b70 con 0x7f3d34161110 2024-04-25T10:31:06.368 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:31:06.365+0000 7f3d3b785640 1 -- 172.21.15.37:0/676830140 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f3d34161110 msgr2=0x7f3d34161880 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:31:06.369 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:31:06.365+0000 7f3d3b785640 1 --2- 172.21.15.37:0/676830140 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f3d34161110 0x7f3d34161880 secure :-1 s=READY pgs=64 cs=0 l=1 rev1=1 crypto rx=0x7f3d30009d90 tx=0x7f3d3002a5d0 comp rx=0 tx=0).stop 2024-04-25T10:31:06.369 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:31:06.365+0000 7f3d3b785640 1 -- 172.21.15.37:0/676830140 shutdown_connections 2024-04-25T10:31:06.369 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:31:06.365+0000 7f3d3b785640 1 --2- 172.21.15.37:0/676830140 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f3d34161110 0x7f3d34161880 unknown :-1 s=CLOSED pgs=64 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:31:06.369 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:31:06.365+0000 7f3d3b785640 1 --2- 172.21.15.37:0/676830140 >> v2:172.21.15.203:3300/0 conn(0x7f3d340d5430 0x7f3d340b75c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:31:06.369 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:31:06.365+0000 7f3d3b785640 1 -- 172.21.15.37:0/676830140 >> 172.21.15.37:0/676830140 conn(0x7f3d340d27a0 msgr2=0x7f3d340d2bb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:31:06.369 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:31:06.365+0000 7f3d3b785640 1 -- 172.21.15.37:0/676830140 shutdown_connections 2024-04-25T10:31:06.369 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:31:06.365+0000 7f3d3b785640 1 -- 172.21.15.37:0/676830140 wait complete. 2024-04-25T10:31:06.369 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:31:06.367+0000 7f3d3b785640 1 Processor -- start 2024-04-25T10:31:06.369 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:31:06.367+0000 7f3d3b785640 1 -- start start 2024-04-25T10:31:06.372 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:31:06.368+0000 7f3d3b785640 1 --2- >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f3d340b7b00 0x7f3d341fa500 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:31:06.372 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:31:06.368+0000 7f3d3b785640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f3d340d5430 0x7f3d3415a820 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:31:06.372 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:31:06.368+0000 7f3d3b785640 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f3d34161110 0x7f3d340cf070 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:31:06.372 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:31:06.368+0000 7f3d3b785640 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 -- 0x7f3d340d4c30 con 0x7f3d340d5430 2024-04-25T10:31:06.372 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:31:06.368+0000 7f3d3b785640 1 -- --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_getmap magic: 0 -- 0x7f3d340d4950 con 0x7f3d34161110 2024-04-25T10:31:06.372 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:31:06.368+0000 7f3d3b785640 1 -- --> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] -- mon_getmap magic: 0 -- 0x7f3d340d4f10 con 0x7f3d340b7b00 2024-04-25T10:31:06.372 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:31:06.369+0000 7f3d38d20640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f3d340d5430 0x7f3d3415a820 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:31:06.372 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:31:06.369+0000 7f3d38d20640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f3d340d5430 0x7f3d3415a820 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.37:3300/0 says I am v2:172.21.15.37:34580/0 (socket says 172.21.15.37:34580) 2024-04-25T10:31:06.372 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:31:06.369+0000 7f3d38d20640 1 -- 172.21.15.37:0/2904094750 learned_addr learned my addr 172.21.15.37:0/2904094750 (peer_addr_for_me v2:172.21.15.37:0/0) 2024-04-25T10:31:06.372 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:31:06.369+0000 7f3d39d22640 1 --2- 172.21.15.37:0/2904094750 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f3d34161110 0x7f3d340cf070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:31:06.372 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:31:06.369+0000 7f3d39521640 1 --2- 172.21.15.37:0/2904094750 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f3d340b7b00 0x7f3d341fa500 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:31:06.373 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:31:06.369+0000 7f3d38d20640 1 -- 172.21.15.37:0/2904094750 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f3d340b7b00 msgr2=0x7f3d341fa500 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:31:06.373 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:31:06.369+0000 7f3d38d20640 1 --2- 172.21.15.37:0/2904094750 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f3d340b7b00 0x7f3d341fa500 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:31:06.373 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:31:06.369+0000 7f3d38d20640 1 -- 172.21.15.37:0/2904094750 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f3d34161110 msgr2=0x7f3d340cf070 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:31:06.373 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:31:06.369+0000 7f3d38d20640 1 --2- 172.21.15.37:0/2904094750 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f3d34161110 0x7f3d340cf070 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:31:06.373 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:31:06.369+0000 7f3d38d20640 1 -- 172.21.15.37:0/2904094750 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f3d3415ad60 con 0x7f3d340d5430 2024-04-25T10:31:06.373 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:31:06.369+0000 7f3d38d20640 1 --2- 172.21.15.37:0/2904094750 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f3d340d5430 0x7f3d3415a820 secure :-1 s=READY pgs=707 cs=0 l=1 rev1=1 crypto rx=0x7f3d2400c3f0 tx=0x7f3d2400c8b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:31:06.373 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:31:06.369+0000 7f3d29ffb640 1 -- 172.21.15.37:0/2904094750 <== mon.0 v2:172.21.15.37:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3d24017070 con 0x7f3d340d5430 2024-04-25T10:31:06.373 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:31:06.369+0000 7f3d29ffb640 1 -- 172.21.15.37:0/2904094750 <== mon.0 v2:172.21.15.37:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f3d24005ce0 con 0x7f3d340d5430 2024-04-25T10:31:06.373 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:31:06.369+0000 7f3d3b785640 1 -- 172.21.15.37:0/2904094750 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f3d340cce40 con 0x7f3d340d5430 2024-04-25T10:31:06.373 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:31:06.369+0000 7f3d29ffb640 1 -- 172.21.15.37:0/2904094750 <== mon.0 v2:172.21.15.37:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3d240072f0 con 0x7f3d340d5430 2024-04-25T10:31:06.373 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:31:06.370+0000 7f3d3b785640 1 -- 172.21.15.37:0/2904094750 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f3d34155e70 con 0x7f3d340d5430 2024-04-25T10:31:06.374 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:31:06.371+0000 7f3d3b785640 1 -- 172.21.15.37:0/2904094750 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f3d340cd360 con 0x7f3d340d5430 2024-04-25T10:31:06.374 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:31:06.371+0000 7f3d29ffb640 1 -- 172.21.15.37:0/2904094750 <== mon.0 v2:172.21.15.37:3300/0 4 ==== mgrmap(e 5) ==== 96782+0+0 (secure 0 0 0) 0x7f3d2401a070 con 0x7f3d340d5430 2024-04-25T10:31:06.374 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:31:06.372+0000 7f3d29ffb640 1 --2- 172.21.15.37:0/2904094750 >> [v2:172.21.15.203:6832/2418205237,v1:172.21.15.203:6833/2418205237] conn(0x7f3d180744b0 0x7f3d180769c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:31:06.374 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:31:06.372+0000 7f3d29ffb640 1 -- 172.21.15.37:0/2904094750 <== mon.0 v2:172.21.15.37:3300/0 5 ==== osd_map(50..50 src has 1..50) ==== 5029+0+0 (secure 0 0 0) 0x7f3d24094240 con 0x7f3d340d5430 2024-04-25T10:31:06.374 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:31:06.372+0000 7f3d39521640 1 --2- 172.21.15.37:0/2904094750 >> [v2:172.21.15.203:6832/2418205237,v1:172.21.15.203:6833/2418205237] conn(0x7f3d180744b0 0x7f3d180769c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:31:06.375 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:31:06.373+0000 7f3d39521640 1 --2- 172.21.15.37:0/2904094750 >> [v2:172.21.15.203:6832/2418205237,v1:172.21.15.203:6833/2418205237] conn(0x7f3d180744b0 0x7f3d180769c0 secure :-1 s=READY pgs=319 cs=0 l=1 rev1=1 crypto rx=0x7f3d2c009560 tx=0x7f3d2c009190 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:31:06.378 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:31:06.377+0000 7f3d29ffb640 1 -- 172.21.15.37:0/2904094750 <== mon.0 v2:172.21.15.37:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187246 (secure 0 0 0) 0x7f3d340cd360 con 0x7f3d340d5430 2024-04-25T10:31:06.559 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:31:06.556+0000 7f3d3b785640 1 -- 172.21.15.37:0/2904094750 --> [v2:172.21.15.203:6832/2418205237,v1:172.21.15.203:6833/2418205237] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) -- 0x7f3d341faac0 con 0x7f3d180744b0 2024-04-25T10:31:06.559 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:31:06.558+0000 7f3d29ffb640 1 -- 172.21.15.37:0/2904094750 <== mgr.4100 v2:172.21.15.203:6832/2418205237 1 ==== mgr_command_reply(tid 0: 0 dumped all) ==== 18+0+13479 (secure 0 0 0) 0x7f3d341faac0 con 0x7f3d180744b0 2024-04-25T10:31:06.560 INFO:teuthology.orchestra.run.smithi037.stdout: 2024-04-25T10:31:06.560 INFO:teuthology.orchestra.run.smithi037.stderr:dumped all 2024-04-25T10:31:06.562 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:31:06.560+0000 7f3d3b785640 1 -- 172.21.15.37:0/2904094750 >> [v2:172.21.15.203:6832/2418205237,v1:172.21.15.203:6833/2418205237] conn(0x7f3d180744b0 msgr2=0x7f3d180769c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:31:06.562 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:31:06.560+0000 7f3d3b785640 1 --2- 172.21.15.37:0/2904094750 >> [v2:172.21.15.203:6832/2418205237,v1:172.21.15.203:6833/2418205237] conn(0x7f3d180744b0 0x7f3d180769c0 secure :-1 s=READY pgs=319 cs=0 l=1 rev1=1 crypto rx=0x7f3d2c009560 tx=0x7f3d2c009190 comp rx=0 tx=0).stop 2024-04-25T10:31:06.562 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:31:06.560+0000 7f3d3b785640 1 -- 172.21.15.37:0/2904094750 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f3d340d5430 msgr2=0x7f3d3415a820 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:31:06.562 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:31:06.560+0000 7f3d3b785640 1 --2- 172.21.15.37:0/2904094750 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f3d340d5430 0x7f3d3415a820 secure :-1 s=READY pgs=707 cs=0 l=1 rev1=1 crypto rx=0x7f3d2400c3f0 tx=0x7f3d2400c8b0 comp rx=0 tx=0).stop 2024-04-25T10:31:06.562 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:31:06.560+0000 7f3d3b785640 1 -- 172.21.15.37:0/2904094750 shutdown_connections 2024-04-25T10:31:06.562 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:31:06.560+0000 7f3d3b785640 1 --2- 172.21.15.37:0/2904094750 >> [v2:172.21.15.203:6832/2418205237,v1:172.21.15.203:6833/2418205237] conn(0x7f3d180744b0 0x7f3d180769c0 unknown :-1 s=CLOSED pgs=319 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:31:06.563 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:31:06.560+0000 7f3d3b785640 1 --2- 172.21.15.37:0/2904094750 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f3d34161110 0x7f3d340cf070 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:31:06.563 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:31:06.560+0000 7f3d3b785640 1 --2- 172.21.15.37:0/2904094750 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f3d340d5430 0x7f3d3415a820 unknown :-1 s=CLOSED pgs=707 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:31:06.563 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:31:06.560+0000 7f3d3b785640 1 --2- 172.21.15.37:0/2904094750 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f3d340b7b00 0x7f3d341fa500 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:31:06.563 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:31:06.560+0000 7f3d3b785640 1 -- 172.21.15.37:0/2904094750 >> 172.21.15.37:0/2904094750 conn(0x7f3d340d27a0 msgr2=0x7f3d34162060 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:31:06.563 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:31:06.560+0000 7f3d3b785640 1 -- 172.21.15.37:0/2904094750 shutdown_connections 2024-04-25T10:31:06.563 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:31:06.560+0000 7f3d3b785640 1 -- 172.21.15.37:0/2904094750 wait complete. 2024-04-25T10:31:06.573 INFO:teuthology.orchestra.run.smithi037.stdout:{"pg_ready":true,"pg_map":{"version":110,"stamp":"2024-04-25T10:31:06.216214+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":66,"num_read_kb":55,"num_write":101,"num_write_kb":1628,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":72,"ondisk_log_size":72,"up":2,"acting":2,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":68,"num_osds":8,"num_per_pool_osds":8,"num_per_pool_omap_osds":2,"kb":754974720,"kb_used":226048,"kb_used_data":5720,"kb_used_omap":12,"kb_used_meta":220211,"kb_avail":754748672,"statfs":{"total":773094113280,"available":772862640128,"internally_reserved":0,"allocated":5857280,"data_stored":3139164,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":12752,"internal_metadata":225496624},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":15,"apply_latency_ms":15,"commit_latency_ns":15000000,"apply_latency_ns":15000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":-6322167,"num_objects":-15,"num_object_clones":0,"num_object_copies":-30,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":-15,"num_whiteouts":0,"num_read":-53,"num_read_kb":-39,"num_write":-50,"num_write_kb":-6193,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":-5,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"6.622730"},"pg_stats":[{"pgid":"1.0","version":"18'72","reported_seq":160,"reported_epoch":50,"state":"active+clean","last_fresh":"2024-04-25T10:31:03.120107+0000","last_change":"2024-04-25T10:28:57.972696+0000","last_active":"2024-04-25T10:31:03.120107+0000","last_peered":"2024-04-25T10:31:03.120107+0000","last_clean":"2024-04-25T10:31:03.120107+0000","last_became_active":"2024-04-25T10:28:57.972373+0000","last_became_peered":"2024-04-25T10:28:57.972373+0000","last_unstale":"2024-04-25T10:31:03.120107+0000","last_undegraded":"2024-04-25T10:31:03.120107+0000","last_fullsized":"2024-04-25T10:31:03.120107+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:28:56.959439+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:28:56.959439+0000","last_clean_scrub_stamp":"2024-04-25T10:28:56.959439+0000","objects_scrubbed":0,"log_size":72,"log_dups_size":0,"ondisk_log_size":72,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T17:34:34.057408+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":66,"num_read_kb":55,"num_write":101,"num_write_kb":1628,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]}],"pool_stats":[{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":66,"num_read_kb":55,"num_write":101,"num_write_kb":1628,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1187840,"data_stored":1180736,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":72,"ondisk_log_size":72,"up":2,"acting":2,"num_store_stats":2}],"osd_stats":[{"osd":7,"up_from":15,"seq":64424509469,"num_pgs":19,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":0,"kb":94371840,"kb_used":27592,"kb_used_data":572,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344248,"statfs":{"total":96636764160,"available":96608509952,"internally_reserved":0,"allocated":585728,"data_stored":245519,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1590,"internal_metadata":27654602},"hb_peers":[0,1,2,3,4,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[]},{"osd":6,"up_from":15,"seq":64424509470,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":0,"kb":94371840,"kb_used":27592,"kb_used_data":572,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344248,"statfs":{"total":96636764160,"available":96608509952,"internally_reserved":0,"allocated":585728,"data_stored":245519,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":27654598},"hb_peers":[0,1,2,3,4,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]},{"osd":5,"up_from":15,"seq":64424509471,"num_pgs":1,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":28172,"kb_used_data":1152,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94343668,"statfs":{"total":96636764160,"available":96607916032,"internally_reserved":0,"allocated":1179648,"data_stored":835887,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[]},{"osd":3,"up_from":15,"seq":64424509471,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":0,"kb":94371840,"kb_used":27592,"kb_used_data":572,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344248,"statfs":{"total":96636764160,"available":96608509952,"internally_reserved":0,"allocated":585728,"data_stored":245519,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1593,"internal_metadata":27654599},"hb_peers":[0,1,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]},{"osd":2,"up_from":15,"seq":64424509471,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":0,"kb":94371840,"kb_used":27596,"kb_used_data":572,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344244,"statfs":{"total":96636764160,"available":96608505856,"internally_reserved":0,"allocated":585728,"data_stored":245519,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]},{"osd":1,"up_from":15,"seq":64424509469,"num_pgs":1,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":28176,"kb_used_data":1152,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94343664,"statfs":{"total":96636764160,"available":96607911936,"internally_reserved":0,"allocated":1179648,"data_stored":835887,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]},{"osd":0,"up_from":15,"seq":64424509470,"num_pgs":16,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":0,"kb":94371840,"kb_used":27592,"kb_used_data":572,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344248,"statfs":{"total":96636764160,"available":96608509952,"internally_reserved":0,"allocated":585728,"data_stored":245519,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[1,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]},{"osd":4,"up_from":13,"seq":55834574877,"num_pgs":31,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":0,"kb":94371840,"kb_used":31736,"kb_used_data":556,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94340104,"statfs":{"total":96636764160,"available":96604266496,"internally_reserved":0,"allocated":569344,"data_stored":239795,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[0,1,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-04-25T10:31:06.574 DEBUG:teuthology.orchestra.run.smithi037:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-04-25T10:31:06.685 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:31:06.682+0000 7f1ed5abf640 1 Processor -- start 2024-04-25T10:31:06.685 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:31:06.682+0000 7f1ed5abf640 1 -- start start 2024-04-25T10:31:06.686 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:31:06.684+0000 7f1ed5abf640 1 --2- >> v2:172.21.15.37:3300/0 conn(0x7f1ed006cc50 0x7f1ed0160910 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:31:06.686 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:31:06.684+0000 7f1ed5abf640 1 --2- >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f1ed0160e50 0x7f1ed00b5860 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:31:06.686 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:31:06.684+0000 7f1ed5abf640 1 -- --> v1:172.21.15.37:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f1ed01616d0 con 0x7f1ed015a9f0 2024-04-25T10:31:06.686 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:31:06.684+0000 7f1ed5abf640 1 -- --> v2:172.21.15.37:3300/0 -- mon_getmap magic: 0 -- 0x7f1ed0163e00 con 0x7f1ed006cc50 2024-04-25T10:31:06.687 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:31:06.684+0000 7f1ed5abf640 1 -- --> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] -- mon_getmap magic: 0 -- 0x7f1ed01640e0 con 0x7f1ed0160e50 2024-04-25T10:31:06.687 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:31:06.684+0000 7f1eceffd640 1 --2- >> v2:172.21.15.37:3300/0 conn(0x7f1ed006cc50 0x7f1ed0160910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:31:06.687 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:31:06.684+0000 7f1ecf7fe640 1 --1- >> v1:172.21.15.37:6789/0 conn(0x7f1ed015a9f0 0x7f1ed006c540 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.37:6789/0 says I am v1:172.21.15.37:47092/0 (socket says 172.21.15.37:47092) 2024-04-25T10:31:06.687 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:31:06.684+0000 7f1ecf7fe640 1 -- 172.21.15.37:0/2751481024 learned_addr learned my addr 172.21.15.37:0/2751481024 (peer_addr_for_me v1:172.21.15.37:0/0) 2024-04-25T10:31:06.687 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:31:06.684+0000 7f1ecffff640 1 --2- 172.21.15.37:0/2751481024 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f1ed0160e50 0x7f1ed00b5860 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:31:06.687 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:31:06.684+0000 7f1eceffd640 1 -- 172.21.15.37:0/2751481024 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f1ed0160e50 msgr2=0x7f1ed00b5860 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:31:06.687 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:31:06.684+0000 7f1eceffd640 1 --2- 172.21.15.37:0/2751481024 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f1ed0160e50 0x7f1ed00b5860 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:31:06.687 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:31:06.684+0000 7f1eceffd640 1 -- 172.21.15.37:0/2751481024 >> v1:172.21.15.37:6789/0 conn(0x7f1ed015a9f0 legacy=0x7f1ed006c540 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:31:06.687 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:31:06.684+0000 7f1eceffd640 1 -- 172.21.15.37:0/2751481024 --> v2:172.21.15.37:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f1ed01643c0 con 0x7f1ed006cc50 2024-04-25T10:31:06.687 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:31:06.684+0000 7f1ece7fc640 1 -- 172.21.15.37:0/2751481024 <== mon.0 v1:172.21.15.37:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 3847584344 0 0) 0x7f1ed01616d0 con 0x7f1ed015a9f0 2024-04-25T10:31:06.688 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:31:06.685+0000 7f1eceffd640 1 --2- 172.21.15.37:0/2751481024 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f1ed006cc50 0x7f1ed0160910 secure :-1 s=READY pgs=709 cs=0 l=1 rev1=1 crypto rx=0x7f1ec40097a0 tx=0x7f1ec402a180 comp rx=0 tx=0).ready entity=mon.0 client_cookie=7bac70c97d917dd9 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:31:06.688 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:31:06.685+0000 7f1ece7fc640 1 -- 172.21.15.37:0/2751481024 <== mon.0 v2:172.21.15.37:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1ec4037070 con 0x7f1ed006cc50 2024-04-25T10:31:06.688 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:31:06.685+0000 7f1ece7fc640 1 -- 172.21.15.37:0/2751481024 <== mon.0 v2:172.21.15.37:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f1ec4032440 con 0x7f1ed006cc50 2024-04-25T10:31:06.689 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:31:06.685+0000 7f1ece7fc640 1 -- 172.21.15.37:0/2751481024 <== mon.0 v2:172.21.15.37:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1ec4032810 con 0x7f1ed006cc50 2024-04-25T10:31:06.689 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:31:06.686+0000 7f1ed5abf640 1 -- 172.21.15.37:0/2751481024 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f1ed006cc50 msgr2=0x7f1ed0160910 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:31:06.689 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:31:06.686+0000 7f1ed5abf640 1 --2- 172.21.15.37:0/2751481024 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f1ed006cc50 0x7f1ed0160910 secure :-1 s=READY pgs=709 cs=0 l=1 rev1=1 crypto rx=0x7f1ec40097a0 tx=0x7f1ec402a180 comp rx=0 tx=0).stop 2024-04-25T10:31:06.689 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:31:06.686+0000 7f1ed5abf640 1 -- 172.21.15.37:0/2751481024 shutdown_connections 2024-04-25T10:31:06.689 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:31:06.686+0000 7f1ed5abf640 1 --2- 172.21.15.37:0/2751481024 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f1ed0160e50 0x7f1ed00b5860 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:31:06.689 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:31:06.686+0000 7f1ed5abf640 1 --2- 172.21.15.37:0/2751481024 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f1ed006cc50 0x7f1ed0160910 unknown :-1 s=CLOSED pgs=709 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:31:06.689 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:31:06.686+0000 7f1ed5abf640 1 -- 172.21.15.37:0/2751481024 >> 172.21.15.37:0/2751481024 conn(0x7f1ed00b4560 msgr2=0x7f1ed00b4970 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:31:06.689 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:31:06.686+0000 7f1ed5abf640 1 -- 172.21.15.37:0/2751481024 shutdown_connections 2024-04-25T10:31:06.689 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:31:06.686+0000 7f1ed5abf640 1 -- 172.21.15.37:0/2751481024 wait complete. 2024-04-25T10:31:06.690 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:31:06.688+0000 7f1ed5abf640 1 Processor -- start 2024-04-25T10:31:06.690 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:31:06.688+0000 7f1ed5abf640 1 -- start start 2024-04-25T10:31:06.693 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:31:06.689+0000 7f1ed5abf640 1 --2- >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f1ed006cc50 0x7f1ed0148e00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:31:06.693 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:31:06.689+0000 7f1ed5abf640 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f1ed015a9f0 0x7f1ed00cddb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:31:06.693 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:31:06.689+0000 7f1ed5abf640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f1ed0160e50 0x7f1ed00ce2f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:31:06.693 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:31:06.689+0000 7f1ed5abf640 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 -- 0x7f1ed0164bc0 con 0x7f1ed0160e50 2024-04-25T10:31:06.693 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:31:06.689+0000 7f1ed5abf640 1 -- --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_getmap magic: 0 -- 0x7f1ed0161260 con 0x7f1ed015a9f0 2024-04-25T10:31:06.693 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:31:06.689+0000 7f1ed5abf640 1 -- --> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] -- mon_getmap magic: 0 -- 0x7f1ed01fa8b0 con 0x7f1ed006cc50 2024-04-25T10:31:06.693 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:31:06.689+0000 7f1ecffff640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f1ed0160e50 0x7f1ed00ce2f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:31:06.693 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:31:06.689+0000 7f1ecffff640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f1ed0160e50 0x7f1ed00ce2f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.37:3300/0 says I am v2:172.21.15.37:34590/0 (socket says 172.21.15.37:34590) 2024-04-25T10:31:06.694 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:31:06.689+0000 7f1ecffff640 1 -- 172.21.15.37:0/1209795219 learned_addr learned my addr 172.21.15.37:0/1209795219 (peer_addr_for_me v2:172.21.15.37:0/0) 2024-04-25T10:31:06.694 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:31:06.690+0000 7f1ecf7fe640 1 --2- 172.21.15.37:0/1209795219 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f1ed006cc50 0x7f1ed0148e00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:31:06.694 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:31:06.690+0000 7f1eceffd640 1 --2- 172.21.15.37:0/1209795219 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f1ed015a9f0 0x7f1ed00cddb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:31:06.694 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:31:06.690+0000 7f1ecffff640 1 -- 172.21.15.37:0/1209795219 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f1ed006cc50 msgr2=0x7f1ed0148e00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:31:06.694 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:31:06.690+0000 7f1ecffff640 1 --2- 172.21.15.37:0/1209795219 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f1ed006cc50 0x7f1ed0148e00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:31:06.694 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:31:06.690+0000 7f1ecffff640 1 -- 172.21.15.37:0/1209795219 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f1ed015a9f0 msgr2=0x7f1ed00cddb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:31:06.694 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:31:06.690+0000 7f1ecffff640 1 --2- 172.21.15.37:0/1209795219 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f1ed015a9f0 0x7f1ed00cddb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:31:06.694 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:31:06.690+0000 7f1ecffff640 1 -- 172.21.15.37:0/1209795219 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f1ed01f9af0 con 0x7f1ed0160e50 2024-04-25T10:31:06.694 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:31:06.690+0000 7f1eceffd640 1 --2- 172.21.15.37:0/1209795219 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f1ed015a9f0 0x7f1ed00cddb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T10:31:06.694 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:31:06.690+0000 7f1ecffff640 1 --2- 172.21.15.37:0/1209795219 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f1ed0160e50 0x7f1ed00ce2f0 secure :-1 s=READY pgs=710 cs=0 l=1 rev1=1 crypto rx=0x7f1ec8007d80 tx=0x7f1ec800e4b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:31:06.694 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:31:06.690+0000 7f1eb7fff640 1 -- 172.21.15.37:0/1209795219 <== mon.0 v2:172.21.15.37:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1ec8014420 con 0x7f1ed0160e50 2024-04-25T10:31:06.694 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:31:06.690+0000 7f1eb7fff640 1 -- 172.21.15.37:0/1209795219 <== mon.0 v2:172.21.15.37:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f1ec8019070 con 0x7f1ed0160e50 2024-04-25T10:31:06.694 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:31:06.690+0000 7f1ed5abf640 1 -- 172.21.15.37:0/1209795219 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f1ed01f9dd0 con 0x7f1ed0160e50 2024-04-25T10:31:06.694 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:31:06.690+0000 7f1eb7fff640 1 -- 172.21.15.37:0/1209795219 <== mon.0 v2:172.21.15.37:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1ec8014800 con 0x7f1ed0160e50 2024-04-25T10:31:06.695 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:31:06.690+0000 7f1ed5abf640 1 -- 172.21.15.37:0/1209795219 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f1ed014a690 con 0x7f1ed0160e50 2024-04-25T10:31:06.695 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:31:06.691+0000 7f1ed5abf640 1 -- 172.21.15.37:0/1209795219 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f1ed014c530 con 0x7f1ed0160e50 2024-04-25T10:31:06.695 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:31:06.692+0000 7f1eb7fff640 1 -- 172.21.15.37:0/1209795219 <== mon.0 v2:172.21.15.37:3300/0 4 ==== mgrmap(e 5) ==== 96782+0+0 (secure 0 0 0) 0x7f1ec80149a0 con 0x7f1ed0160e50 2024-04-25T10:31:06.695 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:31:06.692+0000 7f1eb7fff640 1 --2- 172.21.15.37:0/1209795219 >> [v2:172.21.15.203:6832/2418205237,v1:172.21.15.203:6833/2418205237] conn(0x7f1ebc0744e0 0x7f1ebc0769f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:31:06.696 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:31:06.692+0000 7f1eb7fff640 1 -- 172.21.15.37:0/1209795219 <== mon.0 v2:172.21.15.37:3300/0 5 ==== osd_map(50..50 src has 1..50) ==== 5029+0+0 (secure 0 0 0) 0x7f1ec8095760 con 0x7f1ed0160e50 2024-04-25T10:31:06.696 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:31:06.692+0000 7f1ecf7fe640 1 --2- 172.21.15.37:0/1209795219 >> [v2:172.21.15.203:6832/2418205237,v1:172.21.15.203:6833/2418205237] conn(0x7f1ebc0744e0 0x7f1ebc0769f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:31:06.696 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:31:06.693+0000 7f1ecf7fe640 1 --2- 172.21.15.37:0/1209795219 >> [v2:172.21.15.203:6832/2418205237,v1:172.21.15.203:6833/2418205237] conn(0x7f1ebc0744e0 0x7f1ebc0769f0 secure :-1 s=READY pgs=320 cs=0 l=1 rev1=1 crypto rx=0x7f1ec0000f60 tx=0x7f1ec0007c00 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:31:06.698 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:31:06.697+0000 7f1eb7fff640 1 -- 172.21.15.37:0/1209795219 <== mon.0 v2:172.21.15.37:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187246 (secure 0 0 0) 0x7f1ec8060190 con 0x7f1ed0160e50 2024-04-25T10:31:06.883 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:31:06.881+0000 7f1ed5abf640 1 -- 172.21.15.37:0/1209795219 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7f1ed006a140 con 0x7f1ed0160e50 2024-04-25T10:31:06.883 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:31:06.881+0000 7f1eb7fff640 1 -- 172.21.15.37:0/1209795219 <== mon.0 v2:172.21.15.37:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v50) ==== 74+0+13160 (secure 0 0 0) 0x7f1ec805fa90 con 0x7f1ed0160e50 2024-04-25T10:31:06.884 INFO:teuthology.orchestra.run.smithi037.stdout: 2024-04-25T10:31:06.884 INFO:teuthology.orchestra.run.smithi037.stdout:{"epoch":50,"fsid":"f4acda28-be34-492a-b286-994afd36f61d","created":"2024-04-25T10:28:43.985718+0000","modified":"2024-04-25T10:31:02.860123+0000","last_up_change":"2024-04-25T10:28:54.900654+0000","last_in_change":"2024-04-25T10:28:46.216973+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":6,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-04-25T10:28:56.202153+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"19","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"a2f79028-b332-4166-9ebb-a7024a37ea22","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":42,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.37:6816","nonce":2746074720},{"type":"v1","addr":"172.21.15.37:6817","nonce":2746074720}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.37:6818","nonce":2746074720},{"type":"v1","addr":"172.21.15.37:6819","nonce":2746074720}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.37:6822","nonce":2746074720},{"type":"v1","addr":"172.21.15.37:6823","nonce":2746074720}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.37:6820","nonce":2746074720},{"type":"v1","addr":"172.21.15.37:6821","nonce":2746074720}]},"public_addr":"172.21.15.37:6817/2746074720","cluster_addr":"172.21.15.37:6819/2746074720","heartbeat_back_addr":"172.21.15.37:6823/2746074720","heartbeat_front_addr":"172.21.15.37:6821/2746074720","state":["exists","up"]},{"osd":1,"uuid":"973159de-c2ea-4083-aecc-69373d1c1132","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":45,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.37:6800","nonce":1690341662},{"type":"v1","addr":"172.21.15.37:6801","nonce":1690341662}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.37:6802","nonce":1690341662},{"type":"v1","addr":"172.21.15.37:6803","nonce":1690341662}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.37:6806","nonce":1690341662},{"type":"v1","addr":"172.21.15.37:6807","nonce":1690341662}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.37:6804","nonce":1690341662},{"type":"v1","addr":"172.21.15.37:6805","nonce":1690341662}]},"public_addr":"172.21.15.37:6801/1690341662","cluster_addr":"172.21.15.37:6803/1690341662","heartbeat_back_addr":"172.21.15.37:6807/1690341662","heartbeat_front_addr":"172.21.15.37:6805/1690341662","state":["exists","up"]},{"osd":2,"uuid":"4ae7a0e9-cb3f-484c-a75f-f3766b721c47","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":42,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.37:6808","nonce":2754669649},{"type":"v1","addr":"172.21.15.37:6809","nonce":2754669649}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.37:6810","nonce":2754669649},{"type":"v1","addr":"172.21.15.37:6811","nonce":2754669649}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.37:6814","nonce":2754669649},{"type":"v1","addr":"172.21.15.37:6815","nonce":2754669649}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.37:6812","nonce":2754669649},{"type":"v1","addr":"172.21.15.37:6813","nonce":2754669649}]},"public_addr":"172.21.15.37:6809/2754669649","cluster_addr":"172.21.15.37:6811/2754669649","heartbeat_back_addr":"172.21.15.37:6815/2754669649","heartbeat_front_addr":"172.21.15.37:6813/2754669649","state":["exists","up"]},{"osd":3,"uuid":"1ba4a746-1608-47a7-ace2-960a5cfae934","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":45,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.37:6824","nonce":1815290618},{"type":"v1","addr":"172.21.15.37:6825","nonce":1815290618}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.37:6826","nonce":1815290618},{"type":"v1","addr":"172.21.15.37:6827","nonce":1815290618}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.37:6830","nonce":1815290618},{"type":"v1","addr":"172.21.15.37:6831","nonce":1815290618}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.37:6828","nonce":1815290618},{"type":"v1","addr":"172.21.15.37:6829","nonce":1815290618}]},"public_addr":"172.21.15.37:6825/1815290618","cluster_addr":"172.21.15.37:6827/1815290618","heartbeat_back_addr":"172.21.15.37:6831/1815290618","heartbeat_front_addr":"172.21.15.37:6829/1815290618","state":["exists","up"]},{"osd":4,"uuid":"d4f59f5d-0413-4b79-87ee-bb212c1fcf62","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":42,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6800","nonce":4051532063},{"type":"v1","addr":"172.21.15.203:6801","nonce":4051532063}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6802","nonce":4051532063},{"type":"v1","addr":"172.21.15.203:6803","nonce":4051532063}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6806","nonce":4051532063},{"type":"v1","addr":"172.21.15.203:6807","nonce":4051532063}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6804","nonce":4051532063},{"type":"v1","addr":"172.21.15.203:6805","nonce":4051532063}]},"public_addr":"172.21.15.203:6801/4051532063","cluster_addr":"172.21.15.203:6803/4051532063","heartbeat_back_addr":"172.21.15.203:6807/4051532063","heartbeat_front_addr":"172.21.15.203:6805/4051532063","state":["exists","up"]},{"osd":5,"uuid":"69710a0e-41d8-4e7f-9a10-17ce540b67be","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":45,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6824","nonce":1203822890},{"type":"v1","addr":"172.21.15.203:6825","nonce":1203822890}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6826","nonce":1203822890},{"type":"v1","addr":"172.21.15.203:6827","nonce":1203822890}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6830","nonce":1203822890},{"type":"v1","addr":"172.21.15.203:6831","nonce":1203822890}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6828","nonce":1203822890},{"type":"v1","addr":"172.21.15.203:6829","nonce":1203822890}]},"public_addr":"172.21.15.203:6825/1203822890","cluster_addr":"172.21.15.203:6827/1203822890","heartbeat_back_addr":"172.21.15.203:6831/1203822890","heartbeat_front_addr":"172.21.15.203:6829/1203822890","state":["exists","up"]},{"osd":6,"uuid":"f29dd047-3b48-473d-a235-6642a6293f20","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":42,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6810","nonce":2309615014},{"type":"v1","addr":"172.21.15.203:6812","nonce":2309615014}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6814","nonce":2309615014},{"type":"v1","addr":"172.21.15.203:6816","nonce":2309615014}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6822","nonce":2309615014},{"type":"v1","addr":"172.21.15.203:6823","nonce":2309615014}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6818","nonce":2309615014},{"type":"v1","addr":"172.21.15.203:6820","nonce":2309615014}]},"public_addr":"172.21.15.203:6812/2309615014","cluster_addr":"172.21.15.203:6816/2309615014","heartbeat_back_addr":"172.21.15.203:6823/2309615014","heartbeat_front_addr":"172.21.15.203:6820/2309615014","state":["exists","up"]},{"osd":7,"uuid":"6e691644-378b-4fdc-8e86-fb5f56e31f29","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":42,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6808","nonce":3688099014},{"type":"v1","addr":"172.21.15.203:6809","nonce":3688099014}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6811","nonce":3688099014},{"type":"v1","addr":"172.21.15.203:6813","nonce":3688099014}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6819","nonce":3688099014},{"type":"v1","addr":"172.21.15.203:6821","nonce":3688099014}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6815","nonce":3688099014},{"type":"v1","addr":"172.21.15.203:6817","nonce":3688099014}]},"public_addr":"172.21.15.203:6809/3688099014","cluster_addr":"172.21.15.203:6813/3688099014","heartbeat_back_addr":"172.21.15.203:6821/3688099014","heartbeat_front_addr":"172.21.15.203:6817/3688099014","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T10:28:51.419141+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T10:28:51.366448+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T10:28:51.348276+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T10:28:51.380519+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T10:28:51.105812+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T10:28:51.384496+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T10:28:51.388755+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T10:28:51.366958+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.37:6833/2353203722":"2024-04-26T10:30:59.588340+0000","172.21.15.37:6832/2353203722":"2024-04-26T10:30:59.588340+0000","172.21.15.37:6835/311552410":"2024-04-26T10:30:24.853732+0000","172.21.15.37:6832/311552410":"2024-04-26T10:30:24.853732+0000"},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-04-25T10:31:06.886 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:31:06.884+0000 7f1ed5abf640 1 -- 172.21.15.37:0/1209795219 >> [v2:172.21.15.203:6832/2418205237,v1:172.21.15.203:6833/2418205237] conn(0x7f1ebc0744e0 msgr2=0x7f1ebc0769f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:31:06.886 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:31:06.884+0000 7f1ed5abf640 1 --2- 172.21.15.37:0/1209795219 >> [v2:172.21.15.203:6832/2418205237,v1:172.21.15.203:6833/2418205237] conn(0x7f1ebc0744e0 0x7f1ebc0769f0 secure :-1 s=READY pgs=320 cs=0 l=1 rev1=1 crypto rx=0x7f1ec0000f60 tx=0x7f1ec0007c00 comp rx=0 tx=0).stop 2024-04-25T10:31:06.886 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:31:06.884+0000 7f1ed5abf640 1 -- 172.21.15.37:0/1209795219 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f1ed0160e50 msgr2=0x7f1ed00ce2f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:31:06.886 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:31:06.884+0000 7f1ed5abf640 1 --2- 172.21.15.37:0/1209795219 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f1ed0160e50 0x7f1ed00ce2f0 secure :-1 s=READY pgs=710 cs=0 l=1 rev1=1 crypto rx=0x7f1ec8007d80 tx=0x7f1ec800e4b0 comp rx=0 tx=0).stop 2024-04-25T10:31:06.887 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:31:06.884+0000 7f1ed5abf640 1 -- 172.21.15.37:0/1209795219 shutdown_connections 2024-04-25T10:31:06.887 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:31:06.884+0000 7f1ed5abf640 1 --2- 172.21.15.37:0/1209795219 >> [v2:172.21.15.203:6832/2418205237,v1:172.21.15.203:6833/2418205237] conn(0x7f1ebc0744e0 0x7f1ebc0769f0 unknown :-1 s=CLOSED pgs=320 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:31:06.887 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:31:06.884+0000 7f1ed5abf640 1 --2- 172.21.15.37:0/1209795219 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f1ed0160e50 0x7f1ed00ce2f0 unknown :-1 s=CLOSED pgs=710 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:31:06.887 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:31:06.884+0000 7f1ed5abf640 1 --2- 172.21.15.37:0/1209795219 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f1ed015a9f0 0x7f1ed00cddb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:31:06.887 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:31:06.884+0000 7f1ed5abf640 1 --2- 172.21.15.37:0/1209795219 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f1ed006cc50 0x7f1ed0148e00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:31:06.887 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:31:06.884+0000 7f1ed5abf640 1 -- 172.21.15.37:0/1209795219 >> 172.21.15.37:0/1209795219 conn(0x7f1ed00b4560 msgr2=0x7f1ed01620f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:31:06.887 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:31:06.884+0000 7f1ed5abf640 1 -- 172.21.15.37:0/1209795219 shutdown_connections 2024-04-25T10:31:06.887 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:31:06.884+0000 7f1ed5abf640 1 -- 172.21.15.37:0/1209795219 wait complete. 2024-04-25T10:31:07.898 INFO:tasks.ceph:Scrubbing osd.0 2024-04-25T10:31:07.898 DEBUG:teuthology.orchestra.run.smithi037:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.0 config set osd_debug_deep_scrub_sleep 0 2024-04-25T10:31:08.024 DEBUG:teuthology.orchestra.run.smithi037:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-25T10:31:08.026 DEBUG:teuthology.orchestra.run.smithi203:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-25T10:31:08.030 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:31:08.027+0000 7fe41999d640 1 Processor -- start 2024-04-25T10:31:08.030 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:31:08.028+0000 7fe41999d640 1 -- start start 2024-04-25T10:31:08.031 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:31:08.029+0000 7fe41999d640 1 --2- >> v2:172.21.15.37:3300/0 conn(0x7fe4140cf740 0x7fe4140b3da0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:31:08.031 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:31:08.029+0000 7fe41999d640 1 -- --> v1:172.21.15.37:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fe4140d33c0 con 0x7fe4140b7cf0 2024-04-25T10:31:08.031 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:31:08.029+0000 7fe41999d640 1 -- --> v1:172.21.15.203:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fe4140b23b0 con 0x7fe4140b1b40 2024-04-25T10:31:08.031 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:31:08.029+0000 7fe41999d640 1 -- --> v2:172.21.15.37:3300/0 -- mon_getmap magic: 0 -- 0x7fe4140d36a0 con 0x7fe4140cf740 2024-04-25T10:31:08.031 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:31:08.029+0000 7fe4137fe640 1 --2- >> v2:172.21.15.37:3300/0 conn(0x7fe4140cf740 0x7fe4140b3da0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:31:08.031 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:31:08.029+0000 7fe4127fc640 1 --1- >> v1:172.21.15.37:6789/0 conn(0x7fe4140b7cf0 0x7fe41406dee0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.37:6789/0 says I am v1:172.21.15.37:47104/0 (socket says 172.21.15.37:47104) 2024-04-25T10:31:08.031 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:31:08.029+0000 7fe4127fc640 1 -- 172.21.15.37:0/3234749901 learned_addr learned my addr 172.21.15.37:0/3234749901 (peer_addr_for_me v1:172.21.15.37:0/0) 2024-04-25T10:31:08.032 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:31:08.030+0000 7fe4137fe640 1 -- 172.21.15.37:0/3234749901 >> v1:172.21.15.203:6789/0 conn(0x7fe4140b1b40 legacy=0x7fe4140b75e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:31:08.032 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:31:08.030+0000 7fe4137fe640 1 -- 172.21.15.37:0/3234749901 >> v1:172.21.15.37:6789/0 conn(0x7fe4140b7cf0 legacy=0x7fe41406dee0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:31:08.032 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:31:08.030+0000 7fe411ffb640 1 -- 172.21.15.37:0/3234749901 <== mon.1 v1:172.21.15.203:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 2792983153 0 0) 0x7fe4140b23b0 con 0x7fe4140b1b40 2024-04-25T10:31:08.032 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:31:08.030+0000 7fe4137fe640 1 -- 172.21.15.37:0/3234749901 --> v2:172.21.15.37:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fe4140b2690 con 0x7fe4140cf740 2024-04-25T10:31:08.033 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:31:08.030+0000 7fe4137fe640 1 --2- 172.21.15.37:0/3234749901 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fe4140cf740 0x7fe4140b3da0 secure :-1 s=READY pgs=712 cs=0 l=1 rev1=1 crypto rx=0x7fe40c009d90 tx=0x7fe40c02a4d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=5d4d2534725804fa server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:31:08.033 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:31:08.031+0000 7fe411ffb640 1 -- 172.21.15.37:0/3234749901 <== mon.0 v2:172.21.15.37:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe40c007740 con 0x7fe4140cf740 2024-04-25T10:31:08.033 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:31:08.031+0000 7fe411ffb640 1 -- 172.21.15.37:0/3234749901 <== mon.0 v2:172.21.15.37:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fe40c0078e0 con 0x7fe4140cf740 2024-04-25T10:31:08.033 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:31:08.031+0000 7fe411ffb640 1 -- 172.21.15.37:0/3234749901 <== mon.0 v2:172.21.15.37:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe40c007cb0 con 0x7fe4140cf740 2024-04-25T10:31:08.034 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:31:08.031+0000 7fe41999d640 1 -- 172.21.15.37:0/3234749901 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fe4140cf740 msgr2=0x7fe4140b3da0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:31:08.034 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:31:08.031+0000 7fe41999d640 1 --2- 172.21.15.37:0/3234749901 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fe4140cf740 0x7fe4140b3da0 secure :-1 s=READY pgs=712 cs=0 l=1 rev1=1 crypto rx=0x7fe40c009d90 tx=0x7fe40c02a4d0 comp rx=0 tx=0).stop 2024-04-25T10:31:08.034 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:31:08.032+0000 7fe41999d640 1 -- 172.21.15.37:0/3234749901 shutdown_connections 2024-04-25T10:31:08.034 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:31:08.032+0000 7fe41999d640 1 --2- 172.21.15.37:0/3234749901 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fe4140cf740 0x7fe4140b3da0 unknown :-1 s=CLOSED pgs=712 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:31:08.034 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:31:08.032+0000 7fe41999d640 1 -- 172.21.15.37:0/3234749901 >> 172.21.15.37:0/3234749901 conn(0x7fe41409b700 msgr2=0x7fe41409bb10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:31:08.034 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:31:08.032+0000 7fe41999d640 1 -- 172.21.15.37:0/3234749901 shutdown_connections 2024-04-25T10:31:08.034 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:31:08.032+0000 7fe41999d640 1 -- 172.21.15.37:0/3234749901 wait complete. 2024-04-25T10:31:08.037 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:31:08.035+0000 7fe41999d640 1 Processor -- start 2024-04-25T10:31:08.037 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:31:08.035+0000 7fe41999d640 1 -- start start 2024-04-25T10:31:08.040 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:31:08.037+0000 7fe41999d640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fe4140b1b40 0x7fe4140c5aa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:31:08.041 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:31:08.037+0000 7fe41999d640 1 --2- >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7fe4140cf740 0x7fe414157fb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:31:08.041 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:31:08.037+0000 7fe41999d640 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fe414155a50 0x7fe4141584f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:31:08.041 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:31:08.037+0000 7fe41999d640 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 -- 0x7fe41415cf90 con 0x7fe4140b1b40 2024-04-25T10:31:08.041 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:31:08.037+0000 7fe41999d640 1 -- --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_getmap magic: 0 -- 0x7fe4140b2970 con 0x7fe414155a50 2024-04-25T10:31:08.041 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:31:08.037+0000 7fe41999d640 1 -- --> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] -- mon_getmap magic: 0 -- 0x7fe41415d110 con 0x7fe4140cf740 2024-04-25T10:31:08.041 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:31:08.037+0000 7fe412ffd640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fe4140b1b40 0x7fe4140c5aa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:31:08.041 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:31:08.037+0000 7fe412ffd640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fe4140b1b40 0x7fe4140c5aa0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.37:3300/0 says I am v2:172.21.15.37:34598/0 (socket says 172.21.15.37:34598) 2024-04-25T10:31:08.041 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:31:08.037+0000 7fe412ffd640 1 -- 172.21.15.37:0/3456167946 learned_addr learned my addr 172.21.15.37:0/3456167946 (peer_addr_for_me v2:172.21.15.37:0/0) 2024-04-25T10:31:08.041 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:31:08.038+0000 7fe4137fe640 1 --2- 172.21.15.37:0/3456167946 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fe414155a50 0x7fe4141584f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:31:08.041 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:31:08.038+0000 7fe4127fc640 1 --2- 172.21.15.37:0/3456167946 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7fe4140cf740 0x7fe414157fb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:31:08.041 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:31:08.038+0000 7fe412ffd640 1 -- 172.21.15.37:0/3456167946 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7fe4140cf740 msgr2=0x7fe414157fb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:31:08.041 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:31:08.038+0000 7fe412ffd640 1 --2- 172.21.15.37:0/3456167946 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7fe4140cf740 0x7fe414157fb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:31:08.041 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:31:08.038+0000 7fe412ffd640 1 -- 172.21.15.37:0/3456167946 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fe414155a50 msgr2=0x7fe4141584f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:31:08.041 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:31:08.038+0000 7fe412ffd640 1 --2- 172.21.15.37:0/3456167946 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fe414155a50 0x7fe4141584f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:31:08.042 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:31:08.038+0000 7fe412ffd640 1 -- 172.21.15.37:0/3456167946 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fe414143a00 con 0x7fe4140b1b40 2024-04-25T10:31:08.042 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:31:08.038+0000 7fe412ffd640 1 --2- 172.21.15.37:0/3456167946 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fe4140b1b40 0x7fe4140c5aa0 secure :-1 s=READY pgs=713 cs=0 l=1 rev1=1 crypto rx=0x7fe40400c440 tx=0x7fe40400c900 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:31:08.042 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:31:08.038+0000 7fe3fbfff640 1 -- 172.21.15.37:0/3456167946 <== mon.0 v2:172.21.15.37:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe404017070 con 0x7fe4140b1b40 2024-04-25T10:31:08.042 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:31:08.038+0000 7fe3fbfff640 1 -- 172.21.15.37:0/3456167946 <== mon.0 v2:172.21.15.37:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fe404005480 con 0x7fe4140b1b40 2024-04-25T10:31:08.042 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:31:08.038+0000 7fe41999d640 1 -- 172.21.15.37:0/3456167946 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fe4140d0a90 con 0x7fe4140b1b40 2024-04-25T10:31:08.042 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:31:08.038+0000 7fe3fbfff640 1 -- 172.21.15.37:0/3456167946 <== mon.0 v2:172.21.15.37:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe404003830 con 0x7fe4140b1b40 2024-04-25T10:31:08.042 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:31:08.038+0000 7fe41999d640 1 -- 172.21.15.37:0/3456167946 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fe4140d1050 con 0x7fe4140b1b40 2024-04-25T10:31:08.042 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:31:08.039+0000 7fe41999d640 1 -- 172.21.15.37:0/3456167946 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_get_version(what=osdmap handle=1) -- 0x7fe41406a140 con 0x7fe4140b1b40 2024-04-25T10:31:08.043 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:31:08.040+0000 7fe3fbfff640 1 -- 172.21.15.37:0/3456167946 <== mon.0 v2:172.21.15.37:3300/0 4 ==== mgrmap(e 5) ==== 96782+0+0 (secure 0 0 0) 0x7fe40401a070 con 0x7fe4140b1b40 2024-04-25T10:31:08.043 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:31:08.041+0000 7fe3fbfff640 1 --2- 172.21.15.37:0/3456167946 >> [v2:172.21.15.203:6832/2418205237,v1:172.21.15.203:6833/2418205237] conn(0x7fe4000744e0 0x7fe4000769f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:31:08.044 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:31:08.041+0000 7fe3fbfff640 1 -- 172.21.15.37:0/3456167946 <== mon.0 v2:172.21.15.37:3300/0 5 ==== osd_map(50..50 src has 1..50) ==== 5029+0+0 (secure 0 0 0) 0x7fe4040949f0 con 0x7fe4140b1b40 2024-04-25T10:31:08.044 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:31:08.041+0000 7fe4127fc640 1 --2- 172.21.15.37:0/3456167946 >> [v2:172.21.15.203:6832/2418205237,v1:172.21.15.203:6833/2418205237] conn(0x7fe4000744e0 0x7fe4000769f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:31:08.044 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:31:08.041+0000 7fe3fbfff640 1 --2- 172.21.15.37:0/3456167946 >> [v2:172.21.15.37:6816/2746074720,v1:172.21.15.37:6817/2746074720] conn(0x7fe40007a490 0x7fe40007c8e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:31:08.044 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:31:08.041+0000 7fe3fbfff640 1 -- 172.21.15.37:0/3456167946 --> [v2:172.21.15.37:6816/2746074720,v1:172.21.15.37:6817/2746074720] -- command(tid 1: {"prefix": "get_command_descriptions"}) -- 0x7fe40401dec0 con 0x7fe40007a490 2024-04-25T10:31:08.044 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:31:08.041+0000 7fe3fbfff640 1 -- 172.21.15.37:0/3456167946 <== mon.0 v2:172.21.15.37:3300/0 6 ==== mon_get_version_reply(handle=1 version=50) ==== 24+0+0 (secure 0 0 0) 0x7fe404094e80 con 0x7fe4140b1b40 2024-04-25T10:31:08.044 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:31:08.042+0000 7fe4137fe640 1 --2- 172.21.15.37:0/3456167946 >> [v2:172.21.15.37:6816/2746074720,v1:172.21.15.37:6817/2746074720] conn(0x7fe40007a490 0x7fe40007c8e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:31:08.044 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:31:08.042+0000 7fe4127fc640 1 --2- 172.21.15.37:0/3456167946 >> [v2:172.21.15.203:6832/2418205237,v1:172.21.15.203:6833/2418205237] conn(0x7fe4000744e0 0x7fe4000769f0 secure :-1 s=READY pgs=321 cs=0 l=1 rev1=1 crypto rx=0x7fe408000f60 tx=0x7fe408007e20 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:31:08.044 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:31:08.042+0000 7fe4137fe640 1 --2- 172.21.15.37:0/3456167946 >> [v2:172.21.15.37:6816/2746074720,v1:172.21.15.37:6817/2746074720] conn(0x7fe40007a490 0x7fe40007c8e0 crc :-1 s=READY pgs=17 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:31:08.045 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:31:08.043+0000 7fe3fbfff640 1 -- 172.21.15.37:0/3456167946 <== osd.0 v2:172.21.15.37:6816/2746074720 1 ==== command_reply(tid 1: 0 ) ==== 8+0+27193 (crc 0 0 0) 0x7fe40401dec0 con 0x7fe40007a490 2024-04-25T10:31:08.060 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:31:08.058+0000 7fe41999d640 1 -- 172.21.15.37:0/3456167946 --> [v2:172.21.15.37:6816/2746074720,v1:172.21.15.37:6817/2746074720] -- command(tid 2: {"prefix": "config set", "var": "osd_debug_deep_scrub_sleep", "val": ["0"]}) -- 0x7fe4140b54f0 con 0x7fe40007a490 2024-04-25T10:31:08.061 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:31:08.059+0000 7fe3fbfff640 1 -- 172.21.15.37:0/3456167946 <== osd.0 v2:172.21.15.37:6816/2746074720 2 ==== command_reply(tid 2: 0 ) ==== 8+0+97 (crc 0 0 0) 0x7fe4140b54f0 con 0x7fe40007a490 2024-04-25T10:31:08.061 INFO:teuthology.orchestra.run.smithi037.stdout:{ 2024-04-25T10:31:08.061 INFO:teuthology.orchestra.run.smithi037.stdout: "success": "osd_debug_deep_scrub_sleep = '' (not observed, change may require restart) " 2024-04-25T10:31:08.061 INFO:teuthology.orchestra.run.smithi037.stdout:} 2024-04-25T10:31:08.062 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:31:08.059+0000 7fe41999d640 1 -- 172.21.15.37:0/3456167946 >> [v2:172.21.15.37:6816/2746074720,v1:172.21.15.37:6817/2746074720] conn(0x7fe40007a490 msgr2=0x7fe40007c8e0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:31:08.062 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:31:08.059+0000 7fe41999d640 1 --2- 172.21.15.37:0/3456167946 >> [v2:172.21.15.37:6816/2746074720,v1:172.21.15.37:6817/2746074720] conn(0x7fe40007a490 0x7fe40007c8e0 crc :-1 s=READY pgs=17 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:31:08.062 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:31:08.059+0000 7fe41999d640 1 -- 172.21.15.37:0/3456167946 >> [v2:172.21.15.203:6832/2418205237,v1:172.21.15.203:6833/2418205237] conn(0x7fe4000744e0 msgr2=0x7fe4000769f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:31:08.062 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:31:08.059+0000 7fe41999d640 1 --2- 172.21.15.37:0/3456167946 >> [v2:172.21.15.203:6832/2418205237,v1:172.21.15.203:6833/2418205237] conn(0x7fe4000744e0 0x7fe4000769f0 secure :-1 s=READY pgs=321 cs=0 l=1 rev1=1 crypto rx=0x7fe408000f60 tx=0x7fe408007e20 comp rx=0 tx=0).stop 2024-04-25T10:31:08.062 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:31:08.059+0000 7fe41999d640 1 -- 172.21.15.37:0/3456167946 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fe4140b1b40 msgr2=0x7fe4140c5aa0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:31:08.062 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:31:08.059+0000 7fe41999d640 1 --2- 172.21.15.37:0/3456167946 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fe4140b1b40 0x7fe4140c5aa0 secure :-1 s=READY pgs=713 cs=0 l=1 rev1=1 crypto rx=0x7fe40400c440 tx=0x7fe40400c900 comp rx=0 tx=0).stop 2024-04-25T10:31:08.062 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:31:08.059+0000 7fe4137fe640 1 -- 172.21.15.37:0/3456167946 reap_dead start 2024-04-25T10:31:08.062 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:31:08.059+0000 7fe41999d640 1 -- 172.21.15.37:0/3456167946 shutdown_connections 2024-04-25T10:31:08.062 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:31:08.059+0000 7fe41999d640 1 -- 172.21.15.37:0/3456167946 >> 172.21.15.37:0/3456167946 conn(0x7fe41409b700 msgr2=0x7fe414071180 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:31:08.062 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:31:08.059+0000 7fe41999d640 1 -- 172.21.15.37:0/3456167946 shutdown_connections 2024-04-25T10:31:08.062 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:31:08.059+0000 7fe41999d640 1 -- 172.21.15.37:0/3456167946 wait complete. 2024-04-25T10:31:08.072 DEBUG:teuthology.orchestra.run.smithi037:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd deep-scrub 0 2024-04-25T10:31:08.230 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:31:08.228+0000 7fc5c24e0640 1 Processor -- start 2024-04-25T10:31:08.231 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:31:08.228+0000 7fc5c24e0640 1 -- start start 2024-04-25T10:31:08.232 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:31:08.230+0000 7fc5c24e0640 1 --2- >> v2:172.21.15.37:3300/0 conn(0x7fc5bc0b4bb0 0x7fc5bc1646b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:31:08.232 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:31:08.230+0000 7fc5c24e0640 1 --2- >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7fc5bc0d4560 0x7fc5bc0d4970 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:31:08.233 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:31:08.230+0000 7fc5c24e0640 1 -- --> v1:172.21.15.37:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fc5bc161190 con 0x7fc5bc0b7600 2024-04-25T10:31:08.233 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:31:08.230+0000 7fc5c24e0640 1 -- --> v2:172.21.15.37:3300/0 -- mon_getmap magic: 0 -- 0x7fc5bc161470 con 0x7fc5bc0b4bb0 2024-04-25T10:31:08.233 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:31:08.230+0000 7fc5c24e0640 1 -- --> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] -- mon_getmap magic: 0 -- 0x7fc5bc163eb0 con 0x7fc5bc0d4560 2024-04-25T10:31:08.233 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:31:08.230+0000 7fc5bbfff640 1 --2- >> v2:172.21.15.37:3300/0 conn(0x7fc5bc0b4bb0 0x7fc5bc1646b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:31:08.233 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:31:08.230+0000 7fc5bb7fe640 1 --1- >> v1:172.21.15.37:6789/0 conn(0x7fc5bc0b7600 0x7fc5bc0b7a10 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.37:6789/0 says I am v1:172.21.15.37:47114/0 (socket says 172.21.15.37:47114) 2024-04-25T10:31:08.233 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:31:08.230+0000 7fc5bbfff640 1 --2- >> v2:172.21.15.37:3300/0 conn(0x7fc5bc0b4bb0 0x7fc5bc1646b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.37:3300/0 says I am v2:172.21.15.37:34612/0 (socket says 172.21.15.37:34612) 2024-04-25T10:31:08.233 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:31:08.230+0000 7fc5bb7fe640 1 -- 172.21.15.37:0/1452180043 learned_addr learned my addr 172.21.15.37:0/1452180043 (peer_addr_for_me v1:172.21.15.37:0/0) 2024-04-25T10:31:08.233 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:31:08.230+0000 7fc5c0a7d640 1 --2- 172.21.15.37:0/1452180043 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7fc5bc0d4560 0x7fc5bc0d4970 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:31:08.233 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:31:08.230+0000 7fc5bbfff640 1 -- 172.21.15.37:0/1452180043 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7fc5bc0d4560 msgr2=0x7fc5bc0d4970 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:31:08.233 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:31:08.230+0000 7fc5bbfff640 1 --2- 172.21.15.37:0/1452180043 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7fc5bc0d4560 0x7fc5bc0d4970 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:31:08.233 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:31:08.230+0000 7fc5bbfff640 1 -- 172.21.15.37:0/1452180043 >> v1:172.21.15.37:6789/0 conn(0x7fc5bc0b7600 legacy=0x7fc5bc0b7a10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:31:08.233 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:31:08.230+0000 7fc5baffd640 1 -- 172.21.15.37:0/1452180043 <== mon.0 v1:172.21.15.37:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 586961248 0 0) 0x7fc5bc161190 con 0x7fc5bc0b7600 2024-04-25T10:31:08.233 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:31:08.230+0000 7fc5bbfff640 1 -- 172.21.15.37:0/1452180043 --> v2:172.21.15.37:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fc5bc164190 con 0x7fc5bc0b4bb0 2024-04-25T10:31:08.233 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:31:08.230+0000 7fc5c0a7d640 1 --2- 172.21.15.37:0/1452180043 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7fc5bc0d4560 0x7fc5bc0d4970 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T10:31:08.235 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:31:08.231+0000 7fc5bbfff640 1 --2- 172.21.15.37:0/1452180043 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fc5bc0b4bb0 0x7fc5bc1646b0 secure :-1 s=READY pgs=715 cs=0 l=1 rev1=1 crypto rx=0x7fc5a40097a0 tx=0x7fc5a402a180 comp rx=0 tx=0).ready entity=mon.0 client_cookie=9020ed7c830dbadf server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:31:08.235 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:31:08.231+0000 7fc5baffd640 1 -- 172.21.15.37:0/1452180043 <== mon.0 v2:172.21.15.37:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc5a4037070 con 0x7fc5bc0b4bb0 2024-04-25T10:31:08.235 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:31:08.231+0000 7fc5baffd640 1 -- 172.21.15.37:0/1452180043 <== mon.0 v2:172.21.15.37:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fc5a4032440 con 0x7fc5bc0b4bb0 2024-04-25T10:31:08.235 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:31:08.231+0000 7fc5baffd640 1 -- 172.21.15.37:0/1452180043 <== mon.0 v2:172.21.15.37:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc5a4032810 con 0x7fc5bc0b4bb0 2024-04-25T10:31:08.235 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:31:08.232+0000 7fc5c24e0640 1 -- 172.21.15.37:0/1452180043 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fc5bc0b4bb0 msgr2=0x7fc5bc1646b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:31:08.235 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:31:08.232+0000 7fc5c24e0640 1 --2- 172.21.15.37:0/1452180043 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fc5bc0b4bb0 0x7fc5bc1646b0 secure :-1 s=READY pgs=715 cs=0 l=1 rev1=1 crypto rx=0x7fc5a40097a0 tx=0x7fc5a402a180 comp rx=0 tx=0).stop 2024-04-25T10:31:08.235 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:31:08.232+0000 7fc5c24e0640 1 -- 172.21.15.37:0/1452180043 shutdown_connections 2024-04-25T10:31:08.235 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:31:08.232+0000 7fc5c24e0640 1 --2- 172.21.15.37:0/1452180043 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7fc5bc0d4560 0x7fc5bc0d4970 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:31:08.235 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:31:08.232+0000 7fc5c24e0640 1 --2- 172.21.15.37:0/1452180043 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fc5bc0b4bb0 0x7fc5bc1646b0 unknown :-1 s=CLOSED pgs=715 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:31:08.235 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:31:08.232+0000 7fc5c24e0640 1 -- 172.21.15.37:0/1452180043 >> 172.21.15.37:0/1452180043 conn(0x7fc5bc0d27e0 msgr2=0x7fc5bc0d2bf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:31:08.235 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:31:08.232+0000 7fc5c24e0640 1 -- 172.21.15.37:0/1452180043 shutdown_connections 2024-04-25T10:31:08.235 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:31:08.232+0000 7fc5c24e0640 1 -- 172.21.15.37:0/1452180043 wait complete. 2024-04-25T10:31:08.236 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:31:08.234+0000 7fc5c24e0640 1 Processor -- start 2024-04-25T10:31:08.236 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:31:08.234+0000 7fc5c24e0640 1 -- start start 2024-04-25T10:31:08.239 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:31:08.235+0000 7fc5c24e0640 1 --2- >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7fc5bc0b4bb0 0x7fc5bc0cdc60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:31:08.239 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:31:08.235+0000 7fc5c24e0640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fc5bc0b7600 0x7fc5bc0ce1a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:31:08.239 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:31:08.235+0000 7fc5c24e0640 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fc5bc0d4560 0x7fc5bc0ce6e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:31:08.239 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:31:08.235+0000 7fc5c24e0640 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 -- 0x7fc5bc0d09e0 con 0x7fc5bc0b7600 2024-04-25T10:31:08.239 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:31:08.235+0000 7fc5c24e0640 1 -- --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_getmap magic: 0 -- 0x7fc5bc164bf0 con 0x7fc5bc0d4560 2024-04-25T10:31:08.239 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:31:08.235+0000 7fc5c24e0640 1 -- --> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] -- mon_getmap magic: 0 -- 0x7fc5bc0d0b60 con 0x7fc5bc0b4bb0 2024-04-25T10:31:08.239 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:31:08.235+0000 7fc5bb7fe640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fc5bc0b7600 0x7fc5bc0ce1a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:31:08.239 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:31:08.235+0000 7fc5bb7fe640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fc5bc0b7600 0x7fc5bc0ce1a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.37:3300/0 says I am v2:172.21.15.37:34622/0 (socket says 172.21.15.37:34622) 2024-04-25T10:31:08.239 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:31:08.235+0000 7fc5bb7fe640 1 -- 172.21.15.37:0/4218846879 learned_addr learned my addr 172.21.15.37:0/4218846879 (peer_addr_for_me v2:172.21.15.37:0/0) 2024-04-25T10:31:08.239 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:31:08.236+0000 7fc5bbfff640 1 --2- 172.21.15.37:0/4218846879 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7fc5bc0b4bb0 0x7fc5bc0cdc60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:31:08.239 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:31:08.236+0000 7fc5c0a7d640 1 --2- 172.21.15.37:0/4218846879 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fc5bc0d4560 0x7fc5bc0ce6e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:31:08.239 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:31:08.236+0000 7fc5bb7fe640 1 -- 172.21.15.37:0/4218846879 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7fc5bc0b4bb0 msgr2=0x7fc5bc0cdc60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:31:08.240 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:31:08.236+0000 7fc5bb7fe640 1 --2- 172.21.15.37:0/4218846879 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7fc5bc0b4bb0 0x7fc5bc0cdc60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:31:08.240 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:31:08.236+0000 7fc5bb7fe640 1 -- 172.21.15.37:0/4218846879 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fc5bc0d4560 msgr2=0x7fc5bc0ce6e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:31:08.240 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:31:08.236+0000 7fc5bb7fe640 1 --2- 172.21.15.37:0/4218846879 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fc5bc0d4560 0x7fc5bc0ce6e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:31:08.240 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:31:08.236+0000 7fc5bb7fe640 1 -- 172.21.15.37:0/4218846879 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fc5bc0c9be0 con 0x7fc5bc0b7600 2024-04-25T10:31:08.240 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:31:08.236+0000 7fc5c0a7d640 1 --2- 172.21.15.37:0/4218846879 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fc5bc0d4560 0x7fc5bc0ce6e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T10:31:08.240 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:31:08.236+0000 7fc5bb7fe640 1 --2- 172.21.15.37:0/4218846879 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fc5bc0b7600 0x7fc5bc0ce1a0 secure :-1 s=READY pgs=716 cs=0 l=1 rev1=1 crypto rx=0x7fc5ac00c430 tx=0x7fc5ac00c8f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:31:08.240 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:31:08.236+0000 7fc5b8ff9640 1 -- 172.21.15.37:0/4218846879 <== mon.0 v2:172.21.15.37:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc5ac0074e0 con 0x7fc5bc0b7600 2024-04-25T10:31:08.240 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:31:08.236+0000 7fc5b8ff9640 1 -- 172.21.15.37:0/4218846879 <== mon.0 v2:172.21.15.37:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fc5ac0047e0 con 0x7fc5bc0b7600 2024-04-25T10:31:08.240 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:31:08.236+0000 7fc5c24e0640 1 -- 172.21.15.37:0/4218846879 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fc5bc0c9ec0 con 0x7fc5bc0b7600 2024-04-25T10:31:08.240 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:31:08.236+0000 7fc5b8ff9640 1 -- 172.21.15.37:0/4218846879 <== mon.0 v2:172.21.15.37:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc5ac004bd0 con 0x7fc5bc0b7600 2024-04-25T10:31:08.240 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:31:08.236+0000 7fc5c24e0640 1 -- 172.21.15.37:0/4218846879 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fc5bc0ca480 con 0x7fc5bc0b7600 2024-04-25T10:31:08.241 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:31:08.237+0000 7fc5c24e0640 1 -- 172.21.15.37:0/4218846879 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fc5bc06a140 con 0x7fc5bc0b7600 2024-04-25T10:31:08.241 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:31:08.238+0000 7fc5b8ff9640 1 -- 172.21.15.37:0/4218846879 <== mon.0 v2:172.21.15.37:3300/0 4 ==== mgrmap(e 5) ==== 96782+0+0 (secure 0 0 0) 0x7fc5ac011070 con 0x7fc5bc0b7600 2024-04-25T10:31:08.241 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:31:08.238+0000 7fc5b8ff9640 1 --2- 172.21.15.37:0/4218846879 >> [v2:172.21.15.203:6832/2418205237,v1:172.21.15.203:6833/2418205237] conn(0x7fc59c0744b0 0x7fc59c0769c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:31:08.241 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:31:08.239+0000 7fc5b8ff9640 1 -- 172.21.15.37:0/4218846879 <== mon.0 v2:172.21.15.37:3300/0 5 ==== osd_map(50..50 src has 1..50) ==== 5029+0+0 (secure 0 0 0) 0x7fc5ac094af0 con 0x7fc5bc0b7600 2024-04-25T10:31:08.241 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:31:08.239+0000 7fc5bbfff640 1 --2- 172.21.15.37:0/4218846879 >> [v2:172.21.15.203:6832/2418205237,v1:172.21.15.203:6833/2418205237] conn(0x7fc59c0744b0 0x7fc59c0769c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:31:08.242 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:31:08.240+0000 7fc5bbfff640 1 --2- 172.21.15.37:0/4218846879 >> [v2:172.21.15.203:6832/2418205237,v1:172.21.15.203:6833/2418205237] conn(0x7fc59c0744b0 0x7fc59c0769c0 secure :-1 s=READY pgs=322 cs=0 l=1 rev1=1 crypto rx=0x7fc5a4034000 tx=0x7fc5a402e000 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:31:08.245 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:31:08.243+0000 7fc5b8ff9640 1 -- 172.21.15.37:0/4218846879 <== mon.0 v2:172.21.15.37:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187246 (secure 0 0 0) 0x7fc5ac05f5b0 con 0x7fc5bc0b7600 2024-04-25T10:31:08.436 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:31:08.434+0000 7fc5c24e0640 1 -- 172.21.15.37:0/4218846879 --> [v2:172.21.15.203:6832/2418205237,v1:172.21.15.203:6833/2418205237] -- mgr_command(tid 0: {"prefix": "osd deep-scrub", "who": "0", "target": ["mon-mgr", ""]}) -- 0x7fc5bc0aca00 con 0x7fc59c0744b0 2024-04-25T10:31:08.436 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:31:08.434+0000 7fc5b8ff9640 1 -- 172.21.15.37:0/4218846879 <== mgr.4100 v2:172.21.15.203:6832/2418205237 1 ==== mgr_command_reply(tid 0: 0 instructed osd(s) 0 to deep-scrub) ==== 41+0+0 (secure 0 0 0) 0x7fc5bc0aca00 con 0x7fc59c0744b0 2024-04-25T10:31:08.436 INFO:teuthology.orchestra.run.smithi037.stderr:instructed osd(s) 0 to deep-scrub 2024-04-25T10:31:08.439 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:31:08.436+0000 7fc5c24e0640 1 -- 172.21.15.37:0/4218846879 >> [v2:172.21.15.203:6832/2418205237,v1:172.21.15.203:6833/2418205237] conn(0x7fc59c0744b0 msgr2=0x7fc59c0769c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:31:08.439 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:31:08.436+0000 7fc5c24e0640 1 --2- 172.21.15.37:0/4218846879 >> [v2:172.21.15.203:6832/2418205237,v1:172.21.15.203:6833/2418205237] conn(0x7fc59c0744b0 0x7fc59c0769c0 secure :-1 s=READY pgs=322 cs=0 l=1 rev1=1 crypto rx=0x7fc5a4034000 tx=0x7fc5a402e000 comp rx=0 tx=0).stop 2024-04-25T10:31:08.439 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:31:08.436+0000 7fc5c24e0640 1 -- 172.21.15.37:0/4218846879 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fc5bc0b7600 msgr2=0x7fc5bc0ce1a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:31:08.439 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:31:08.436+0000 7fc5c24e0640 1 --2- 172.21.15.37:0/4218846879 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fc5bc0b7600 0x7fc5bc0ce1a0 secure :-1 s=READY pgs=716 cs=0 l=1 rev1=1 crypto rx=0x7fc5ac00c430 tx=0x7fc5ac00c8f0 comp rx=0 tx=0).stop 2024-04-25T10:31:08.439 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:31:08.436+0000 7fc5c24e0640 1 -- 172.21.15.37:0/4218846879 shutdown_connections 2024-04-25T10:31:08.439 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:31:08.436+0000 7fc5c24e0640 1 --2- 172.21.15.37:0/4218846879 >> [v2:172.21.15.203:6832/2418205237,v1:172.21.15.203:6833/2418205237] conn(0x7fc59c0744b0 0x7fc59c0769c0 unknown :-1 s=CLOSED pgs=322 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:31:08.439 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:31:08.436+0000 7fc5c24e0640 1 --2- 172.21.15.37:0/4218846879 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fc5bc0d4560 0x7fc5bc0ce6e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:31:08.439 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:31:08.436+0000 7fc5c24e0640 1 --2- 172.21.15.37:0/4218846879 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fc5bc0b7600 0x7fc5bc0ce1a0 unknown :-1 s=CLOSED pgs=716 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:31:08.439 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:31:08.436+0000 7fc5c24e0640 1 --2- 172.21.15.37:0/4218846879 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7fc5bc0b4bb0 0x7fc5bc0cdc60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:31:08.439 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:31:08.436+0000 7fc5c24e0640 1 -- 172.21.15.37:0/4218846879 >> 172.21.15.37:0/4218846879 conn(0x7fc5bc0d27e0 msgr2=0x7fc5bc161830 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:31:08.440 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:31:08.436+0000 7fc5c24e0640 1 -- 172.21.15.37:0/4218846879 shutdown_connections 2024-04-25T10:31:08.440 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:31:08.436+0000 7fc5c24e0640 1 -- 172.21.15.37:0/4218846879 wait complete. 2024-04-25T10:31:08.450 INFO:tasks.ceph:Scrubbing osd.1 2024-04-25T10:31:08.450 DEBUG:teuthology.orchestra.run.smithi037:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.1 config set osd_debug_deep_scrub_sleep 0 2024-04-25T10:31:08.567 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:31:08.565+0000 7f6b8d4e0640 1 Processor -- start 2024-04-25T10:31:08.567 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:31:08.565+0000 7f6b8d4e0640 1 -- start start 2024-04-25T10:31:08.571 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:31:08.566+0000 7f6b8d4e0640 1 --2- >> v2:172.21.15.37:3300/0 conn(0x7f6b881603c0 0x7f6b880d13f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:31:08.571 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:31:08.566+0000 7f6b8d4e0640 1 --2- >> v2:172.21.15.203:3300/0 conn(0x7f6b880b4840 0x7f6b88073c70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:31:08.571 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:31:08.566+0000 7f6b8d4e0640 1 -- --> v1:172.21.15.37:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f6b880745c0 con 0x7f6b880741b0 2024-04-25T10:31:08.572 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:31:08.566+0000 7f6b8d4e0640 1 -- --> v2:172.21.15.37:3300/0 -- mon_getmap magic: 0 -- 0x7f6b88164c80 con 0x7f6b881603c0 2024-04-25T10:31:08.572 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:31:08.566+0000 7f6b8d4e0640 1 -- --> v2:172.21.15.203:3300/0 -- mon_getmap magic: 0 -- 0x7f6b88163ec0 con 0x7f6b880b4840 2024-04-25T10:31:08.572 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:31:08.566+0000 7f6b86ffd640 1 --2- >> v2:172.21.15.37:3300/0 conn(0x7f6b881603c0 0x7f6b880d13f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:31:08.572 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:31:08.566+0000 7f6b877fe640 1 --1- >> v1:172.21.15.37:6789/0 conn(0x7f6b880741b0 0x7f6b8806dc20 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.37:6789/0 says I am v1:172.21.15.37:47118/0 (socket says 172.21.15.37:47118) 2024-04-25T10:31:08.572 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:31:08.566+0000 7f6b877fe640 1 -- 172.21.15.37:0/2664861294 learned_addr learned my addr 172.21.15.37:0/2664861294 (peer_addr_for_me v1:172.21.15.37:0/0) 2024-04-25T10:31:08.572 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:31:08.567+0000 7f6b867fc640 1 --2- 172.21.15.37:0/2664861294 >> v2:172.21.15.203:3300/0 conn(0x7f6b880b4840 0x7f6b88073c70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:31:08.572 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:31:08.567+0000 7f6b86ffd640 1 -- 172.21.15.37:0/2664861294 >> v2:172.21.15.203:3300/0 conn(0x7f6b880b4840 msgr2=0x7f6b88073c70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:31:08.572 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:31:08.567+0000 7f6b86ffd640 1 --2- 172.21.15.37:0/2664861294 >> v2:172.21.15.203:3300/0 conn(0x7f6b880b4840 0x7f6b88073c70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:31:08.573 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:31:08.567+0000 7f6b86ffd640 1 -- 172.21.15.37:0/2664861294 >> v1:172.21.15.37:6789/0 conn(0x7f6b880741b0 legacy=0x7f6b8806dc20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:31:08.573 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:31:08.567+0000 7f6b86ffd640 1 -- 172.21.15.37:0/2664861294 --> v2:172.21.15.37:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f6b881641a0 con 0x7f6b881603c0 2024-04-25T10:31:08.573 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:31:08.567+0000 7f6b85ffb640 1 -- 172.21.15.37:0/2664861294 <== mon.0 v1:172.21.15.37:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 643706261 0 0) 0x7f6b80001490 con 0x7f6b880741b0 2024-04-25T10:31:08.573 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:31:08.567+0000 7f6b86ffd640 1 --2- 172.21.15.37:0/2664861294 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f6b881603c0 0x7f6b880d13f0 secure :-1 s=READY pgs=718 cs=0 l=1 rev1=1 crypto rx=0x7f6b880d5b50 tx=0x7f6b7802a1a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=706b2f5158341d9e server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:31:08.573 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:31:08.568+0000 7f6b85ffb640 1 -- 172.21.15.37:0/2664861294 <== mon.0 v2:172.21.15.37:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6b78037070 con 0x7f6b881603c0 2024-04-25T10:31:08.573 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:31:08.568+0000 7f6b85ffb640 1 -- 172.21.15.37:0/2664861294 <== mon.0 v2:172.21.15.37:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f6b78032440 con 0x7f6b881603c0 2024-04-25T10:31:08.573 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:31:08.568+0000 7f6b85ffb640 1 -- 172.21.15.37:0/2664861294 <== mon.0 v2:172.21.15.37:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6b78032810 con 0x7f6b881603c0 2024-04-25T10:31:08.573 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:31:08.568+0000 7f6b8d4e0640 1 -- 172.21.15.37:0/2664861294 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f6b881603c0 msgr2=0x7f6b880d13f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:31:08.573 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:31:08.568+0000 7f6b8d4e0640 1 --2- 172.21.15.37:0/2664861294 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f6b881603c0 0x7f6b880d13f0 secure :-1 s=READY pgs=718 cs=0 l=1 rev1=1 crypto rx=0x7f6b880d5b50 tx=0x7f6b7802a1a0 comp rx=0 tx=0).stop 2024-04-25T10:31:08.574 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:31:08.569+0000 7f6b8d4e0640 1 -- 172.21.15.37:0/2664861294 shutdown_connections 2024-04-25T10:31:08.574 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:31:08.569+0000 7f6b8d4e0640 1 --2- 172.21.15.37:0/2664861294 >> v2:172.21.15.203:3300/0 conn(0x7f6b880b4840 0x7f6b88073c70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:31:08.574 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:31:08.569+0000 7f6b8d4e0640 1 --2- 172.21.15.37:0/2664861294 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f6b881603c0 0x7f6b880d13f0 unknown :-1 s=CLOSED pgs=718 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:31:08.574 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:31:08.569+0000 7f6b8d4e0640 1 -- 172.21.15.37:0/2664861294 >> 172.21.15.37:0/2664861294 conn(0x7f6b880d5d60 msgr2=0x7f6b880d6170 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:31:08.574 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:31:08.569+0000 7f6b8d4e0640 1 -- 172.21.15.37:0/2664861294 shutdown_connections 2024-04-25T10:31:08.574 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:31:08.569+0000 7f6b8d4e0640 1 -- 172.21.15.37:0/2664861294 wait complete. 2024-04-25T10:31:08.580 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:31:08.571+0000 7f6b8d4e0640 1 Processor -- start 2024-04-25T10:31:08.580 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:31:08.571+0000 7f6b8d4e0640 1 -- start start 2024-04-25T10:31:08.580 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:31:08.572+0000 7f6b8d4e0640 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f6b880741b0 0x7f6b881f9800 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:31:08.580 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:31:08.572+0000 7f6b8d4e0640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f6b880b4840 0x7f6b881ea530 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:31:08.580 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:31:08.572+0000 7f6b8d4e0640 1 --2- >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f6b881603c0 0x7f6b88162aa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:31:08.580 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:31:08.572+0000 7f6b8d4e0640 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 -- 0x7f6b880b9140 con 0x7f6b880b4840 2024-04-25T10:31:08.580 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:31:08.572+0000 7f6b8d4e0640 1 -- --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_getmap magic: 0 -- 0x7f6b88164760 con 0x7f6b880741b0 2024-04-25T10:31:08.580 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:31:08.572+0000 7f6b8d4e0640 1 -- --> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] -- mon_getmap magic: 0 -- 0x7f6b880b92c0 con 0x7f6b881603c0 2024-04-25T10:31:08.581 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:31:08.572+0000 7f6b867fc640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f6b880b4840 0x7f6b881ea530 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:31:08.581 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:31:08.572+0000 7f6b867fc640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f6b880b4840 0x7f6b881ea530 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.37:3300/0 says I am v2:172.21.15.37:34638/0 (socket says 172.21.15.37:34638) 2024-04-25T10:31:08.581 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:31:08.572+0000 7f6b867fc640 1 -- 172.21.15.37:0/743640566 learned_addr learned my addr 172.21.15.37:0/743640566 (peer_addr_for_me v2:172.21.15.37:0/0) 2024-04-25T10:31:08.581 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:31:08.572+0000 7f6b877fe640 1 --2- 172.21.15.37:0/743640566 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f6b881603c0 0x7f6b88162aa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:31:08.581 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:31:08.572+0000 7f6b86ffd640 1 --2- 172.21.15.37:0/743640566 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f6b880741b0 0x7f6b881f9800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:31:08.581 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:31:08.572+0000 7f6b867fc640 1 -- 172.21.15.37:0/743640566 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f6b881603c0 msgr2=0x7f6b88162aa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:31:08.581 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:31:08.572+0000 7f6b867fc640 1 --2- 172.21.15.37:0/743640566 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f6b881603c0 0x7f6b88162aa0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:31:08.581 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:31:08.572+0000 7f6b867fc640 1 -- 172.21.15.37:0/743640566 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f6b880741b0 msgr2=0x7f6b881f9800 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:31:08.582 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:31:08.572+0000 7f6b867fc640 1 --2- 172.21.15.37:0/743640566 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f6b880741b0 0x7f6b881f9800 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:31:08.582 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:31:08.572+0000 7f6b867fc640 1 -- 172.21.15.37:0/743640566 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f6b881eaea0 con 0x7f6b880b4840 2024-04-25T10:31:08.582 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:31:08.573+0000 7f6b867fc640 1 --2- 172.21.15.37:0/743640566 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f6b880b4840 0x7f6b881ea530 secure :-1 s=READY pgs=719 cs=0 l=1 rev1=1 crypto rx=0x7f6b7c00e870 tx=0x7f6b7c00ed30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:31:08.582 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:31:08.573+0000 7f6b6f7fe640 1 -- 172.21.15.37:0/743640566 <== mon.0 v2:172.21.15.37:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6b7c017070 con 0x7f6b880b4840 2024-04-25T10:31:08.582 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:31:08.573+0000 7f6b6f7fe640 1 -- 172.21.15.37:0/743640566 <== mon.0 v2:172.21.15.37:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f6b7c00c8e0 con 0x7f6b880b4840 2024-04-25T10:31:08.582 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:31:08.573+0000 7f6b8d4e0640 1 -- 172.21.15.37:0/743640566 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f6b88160a50 con 0x7f6b880b4840 2024-04-25T10:31:08.582 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:31:08.573+0000 7f6b6f7fe640 1 -- 172.21.15.37:0/743640566 <== mon.0 v2:172.21.15.37:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6b7c00ccd0 con 0x7f6b880b4840 2024-04-25T10:31:08.582 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:31:08.573+0000 7f6b8d4e0640 1 -- 172.21.15.37:0/743640566 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f6b881f68b0 con 0x7f6b880b4840 2024-04-25T10:31:08.583 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:31:08.574+0000 7f6b8d4e0640 1 -- 172.21.15.37:0/743640566 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_get_version(what=osdmap handle=1) -- 0x7f6b880d46a0 con 0x7f6b880b4840 2024-04-25T10:31:08.583 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:31:08.575+0000 7f6b6f7fe640 1 -- 172.21.15.37:0/743640566 <== mon.0 v2:172.21.15.37:3300/0 4 ==== mgrmap(e 5) ==== 96782+0+0 (secure 0 0 0) 0x7f6b7c01a070 con 0x7f6b880b4840 2024-04-25T10:31:08.583 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:31:08.575+0000 7f6b6f7fe640 1 --2- 172.21.15.37:0/743640566 >> [v2:172.21.15.203:6832/2418205237,v1:172.21.15.203:6833/2418205237] conn(0x7f6b740744b0 0x7f6b740769c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:31:08.583 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:31:08.575+0000 7f6b6f7fe640 1 -- 172.21.15.37:0/743640566 <== mon.0 v2:172.21.15.37:3300/0 5 ==== osd_map(50..50 src has 1..50) ==== 5029+0+0 (secure 0 0 0) 0x7f6b7c095530 con 0x7f6b880b4840 2024-04-25T10:31:08.583 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:31:08.575+0000 7f6b6f7fe640 1 --2- 172.21.15.37:0/743640566 >> [v2:172.21.15.37:6800/1690341662,v1:172.21.15.37:6801/1690341662] conn(0x7f6b7407a460 0x7f6b7407c8b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:31:08.583 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:31:08.575+0000 7f6b6f7fe640 1 -- 172.21.15.37:0/743640566 --> [v2:172.21.15.37:6800/1690341662,v1:172.21.15.37:6801/1690341662] -- command(tid 1: {"prefix": "get_command_descriptions"}) -- 0x7f6b7c001d30 con 0x7f6b7407a460 2024-04-25T10:31:08.583 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:31:08.575+0000 7f6b6f7fe640 1 -- 172.21.15.37:0/743640566 <== mon.0 v2:172.21.15.37:3300/0 6 ==== mon_get_version_reply(handle=1 version=50) ==== 24+0+0 (secure 0 0 0) 0x7f6b7c0959c0 con 0x7f6b880b4840 2024-04-25T10:31:08.583 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:31:08.576+0000 7f6b877fe640 1 --2- 172.21.15.37:0/743640566 >> [v2:172.21.15.37:6800/1690341662,v1:172.21.15.37:6801/1690341662] conn(0x7f6b7407a460 0x7f6b7407c8b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:31:08.583 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:31:08.576+0000 7f6b86ffd640 1 --2- 172.21.15.37:0/743640566 >> [v2:172.21.15.203:6832/2418205237,v1:172.21.15.203:6833/2418205237] conn(0x7f6b740744b0 0x7f6b740769c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:31:08.583 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:31:08.576+0000 7f6b877fe640 1 --2- 172.21.15.37:0/743640566 >> [v2:172.21.15.37:6800/1690341662,v1:172.21.15.37:6801/1690341662] conn(0x7f6b7407a460 0x7f6b7407c8b0 crc :-1 s=READY pgs=19 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:31:08.584 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:31:08.576+0000 7f6b86ffd640 1 --2- 172.21.15.37:0/743640566 >> [v2:172.21.15.203:6832/2418205237,v1:172.21.15.203:6833/2418205237] conn(0x7f6b740744b0 0x7f6b740769c0 secure :-1 s=READY pgs=323 cs=0 l=1 rev1=1 crypto rx=0x7f6b78034000 tx=0x7f6b7802e000 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:31:08.584 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:31:08.577+0000 7f6b6f7fe640 1 -- 172.21.15.37:0/743640566 <== osd.1 v2:172.21.15.37:6800/1690341662 1 ==== command_reply(tid 1: 0 ) ==== 8+0+27193 (crc 0 0 0) 0x7f6b7c001d30 con 0x7f6b7407a460 2024-04-25T10:31:08.592 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:31:08.590+0000 7f6b8d4e0640 1 -- 172.21.15.37:0/743640566 --> [v2:172.21.15.37:6800/1690341662,v1:172.21.15.37:6801/1690341662] -- command(tid 2: {"prefix": "config set", "var": "osd_debug_deep_scrub_sleep", "val": ["0"]}) -- 0x7f6b881eac50 con 0x7f6b7407a460 2024-04-25T10:31:08.593 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:31:08.590+0000 7f6b6f7fe640 1 -- 172.21.15.37:0/743640566 <== osd.1 v2:172.21.15.37:6800/1690341662 2 ==== command_reply(tid 2: 0 ) ==== 8+0+97 (crc 0 0 0) 0x7f6b881eac50 con 0x7f6b7407a460 2024-04-25T10:31:08.593 INFO:teuthology.orchestra.run.smithi037.stdout:{ 2024-04-25T10:31:08.593 INFO:teuthology.orchestra.run.smithi037.stdout: "success": "osd_debug_deep_scrub_sleep = '' (not observed, change may require restart) " 2024-04-25T10:31:08.593 INFO:teuthology.orchestra.run.smithi037.stdout:} 2024-04-25T10:31:08.594 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:31:08.590+0000 7f6b8d4e0640 1 -- 172.21.15.37:0/743640566 >> [v2:172.21.15.37:6800/1690341662,v1:172.21.15.37:6801/1690341662] conn(0x7f6b7407a460 msgr2=0x7f6b7407c8b0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:31:08.594 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:31:08.591+0000 7f6b8d4e0640 1 --2- 172.21.15.37:0/743640566 >> [v2:172.21.15.37:6800/1690341662,v1:172.21.15.37:6801/1690341662] conn(0x7f6b7407a460 0x7f6b7407c8b0 crc :-1 s=READY pgs=19 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:31:08.594 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:31:08.591+0000 7f6b8d4e0640 1 -- 172.21.15.37:0/743640566 >> [v2:172.21.15.203:6832/2418205237,v1:172.21.15.203:6833/2418205237] conn(0x7f6b740744b0 msgr2=0x7f6b740769c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:31:08.594 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:31:08.591+0000 7f6b8d4e0640 1 --2- 172.21.15.37:0/743640566 >> [v2:172.21.15.203:6832/2418205237,v1:172.21.15.203:6833/2418205237] conn(0x7f6b740744b0 0x7f6b740769c0 secure :-1 s=READY pgs=323 cs=0 l=1 rev1=1 crypto rx=0x7f6b78034000 tx=0x7f6b7802e000 comp rx=0 tx=0).stop 2024-04-25T10:31:08.594 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:31:08.591+0000 7f6b8d4e0640 1 -- 172.21.15.37:0/743640566 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f6b880b4840 msgr2=0x7f6b881ea530 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:31:08.595 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:31:08.591+0000 7f6b8d4e0640 1 --2- 172.21.15.37:0/743640566 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f6b880b4840 0x7f6b881ea530 secure :-1 s=READY pgs=719 cs=0 l=1 rev1=1 crypto rx=0x7f6b7c00e870 tx=0x7f6b7c00ed30 comp rx=0 tx=0).stop 2024-04-25T10:31:08.595 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:31:08.591+0000 7f6b877fe640 1 -- 172.21.15.37:0/743640566 reap_dead start 2024-04-25T10:31:08.595 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:31:08.591+0000 7f6b8d4e0640 1 -- 172.21.15.37:0/743640566 shutdown_connections 2024-04-25T10:31:08.595 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:31:08.591+0000 7f6b8d4e0640 1 -- 172.21.15.37:0/743640566 >> 172.21.15.37:0/743640566 conn(0x7f6b880d5d60 msgr2=0x7f6b880b6b20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:31:08.595 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:31:08.591+0000 7f6b8d4e0640 1 -- 172.21.15.37:0/743640566 shutdown_connections 2024-04-25T10:31:08.595 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:31:08.591+0000 7f6b8d4e0640 1 -- 172.21.15.37:0/743640566 wait complete. 2024-04-25T10:31:08.604 DEBUG:teuthology.orchestra.run.smithi037:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd deep-scrub 1 2024-04-25T10:31:08.716 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:31:08.714+0000 7fed490e0640 1 Processor -- start 2024-04-25T10:31:08.717 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:31:08.714+0000 7fed490e0640 1 -- start start 2024-04-25T10:31:08.719 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:31:08.716+0000 7fed490e0640 1 --2- >> v2:172.21.15.37:3300/0 conn(0x7fed44156c30 0x7fed440cec80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:31:08.719 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:31:08.716+0000 7fed490e0640 1 --2- >> v2:172.21.15.203:3300/0 conn(0x7fed440cf1c0 0x7fed4406f990 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:31:08.719 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:31:08.716+0000 7fed490e0640 1 -- --> v1:172.21.15.203:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fed440cf5d0 con 0x7fed4406fed0 2024-04-25T10:31:08.719 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:31:08.716+0000 7fed490e0640 1 -- --> v2:172.21.15.37:3300/0 -- mon_getmap magic: 0 -- 0x7fed440b6f00 con 0x7fed44156c30 2024-04-25T10:31:08.719 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:31:08.716+0000 7fed490e0640 1 -- --> v2:172.21.15.203:3300/0 -- mon_getmap magic: 0 -- 0x7fed4415d850 con 0x7fed440cf1c0 2024-04-25T10:31:08.719 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:31:08.716+0000 7fed42d9d640 1 --2- >> v2:172.21.15.37:3300/0 conn(0x7fed44156c30 0x7fed440cec80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:31:08.720 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:31:08.716+0000 7fed42d9d640 1 --2- >> v2:172.21.15.37:3300/0 conn(0x7fed44156c30 0x7fed440cec80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.37:3300/0 says I am v2:172.21.15.37:34640/0 (socket says 172.21.15.37:34640) 2024-04-25T10:31:08.720 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:31:08.716+0000 7fed42d9d640 1 -- 172.21.15.37:0/1650533791 learned_addr learned my addr 172.21.15.37:0/1650533791 (peer_addr_for_me v2:172.21.15.37:0/0) 2024-04-25T10:31:08.720 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:31:08.716+0000 7fed4259c640 1 --2- 172.21.15.37:0/1650533791 >> v2:172.21.15.203:3300/0 conn(0x7fed440cf1c0 0x7fed4406f990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:31:08.720 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:31:08.716+0000 7fed42d9d640 1 -- 172.21.15.37:0/1650533791 >> v2:172.21.15.203:3300/0 conn(0x7fed440cf1c0 msgr2=0x7fed4406f990 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:31:08.720 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:31:08.716+0000 7fed42d9d640 1 --2- 172.21.15.37:0/1650533791 >> v2:172.21.15.203:3300/0 conn(0x7fed440cf1c0 0x7fed4406f990 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:31:08.720 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:31:08.716+0000 7fed42d9d640 1 -- 172.21.15.37:0/1650533791 >> v1:172.21.15.203:6789/0 conn(0x7fed4406fed0 legacy=0x7fed441601c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:31:08.720 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:31:08.716+0000 7fed42d9d640 1 -- 172.21.15.37:0/1650533791 --> v2:172.21.15.37:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fed440702e0 con 0x7fed44156c30 2024-04-25T10:31:08.720 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:31:08.716+0000 7fed4259c640 1 --2- 172.21.15.37:0/1650533791 >> v2:172.21.15.203:3300/0 conn(0x7fed440cf1c0 0x7fed4406f990 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T10:31:08.720 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:31:08.717+0000 7fed42d9d640 1 --2- 172.21.15.37:0/1650533791 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fed44156c30 0x7fed440cec80 secure :-1 s=READY pgs=720 cs=0 l=1 rev1=1 crypto rx=0x7fed44073090 tx=0x7fed3802a230 comp rx=0 tx=0).ready entity=mon.0 client_cookie=1d109373623f195a server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:31:08.720 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:31:08.717+0000 7fed41d9b640 1 -- 172.21.15.37:0/1650533791 <== mon.0 v2:172.21.15.37:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fed38037070 con 0x7fed44156c30 2024-04-25T10:31:08.720 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:31:08.717+0000 7fed41d9b640 1 -- 172.21.15.37:0/1650533791 <== mon.0 v2:172.21.15.37:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fed38032440 con 0x7fed44156c30 2024-04-25T10:31:08.720 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:31:08.717+0000 7fed41d9b640 1 -- 172.21.15.37:0/1650533791 <== mon.0 v2:172.21.15.37:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fed38032810 con 0x7fed44156c30 2024-04-25T10:31:08.721 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:31:08.718+0000 7fed490e0640 1 -- 172.21.15.37:0/1650533791 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fed44156c30 msgr2=0x7fed440cec80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:31:08.721 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:31:08.718+0000 7fed490e0640 1 --2- 172.21.15.37:0/1650533791 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fed44156c30 0x7fed440cec80 secure :-1 s=READY pgs=720 cs=0 l=1 rev1=1 crypto rx=0x7fed44073090 tx=0x7fed3802a230 comp rx=0 tx=0).stop 2024-04-25T10:31:08.721 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:31:08.718+0000 7fed490e0640 1 -- 172.21.15.37:0/1650533791 shutdown_connections 2024-04-25T10:31:08.721 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:31:08.718+0000 7fed490e0640 1 --2- 172.21.15.37:0/1650533791 >> v2:172.21.15.203:3300/0 conn(0x7fed440cf1c0 0x7fed4406f990 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:31:08.721 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:31:08.718+0000 7fed490e0640 1 --2- 172.21.15.37:0/1650533791 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fed44156c30 0x7fed440cec80 unknown :-1 s=CLOSED pgs=720 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:31:08.722 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:31:08.718+0000 7fed490e0640 1 -- 172.21.15.37:0/1650533791 >> 172.21.15.37:0/1650533791 conn(0x7fed440b36b0 msgr2=0x7fed440b3ac0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:31:08.722 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:31:08.718+0000 7fed490e0640 1 -- 172.21.15.37:0/1650533791 shutdown_connections 2024-04-25T10:31:08.722 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:31:08.718+0000 7fed490e0640 1 -- 172.21.15.37:0/1650533791 wait complete. 2024-04-25T10:31:08.722 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:31:08.720+0000 7fed490e0640 1 Processor -- start 2024-04-25T10:31:08.722 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:31:08.720+0000 7fed490e0640 1 -- start start 2024-04-25T10:31:08.725 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:31:08.721+0000 7fed490e0640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fed4406fed0 0x7fed440b1df0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:31:08.725 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:31:08.721+0000 7fed490e0640 1 --2- >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7fed440cf1c0 0x7fed44144cb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:31:08.725 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:31:08.721+0000 7fed490e0640 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fed44156c30 0x7fed4415b630 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:31:08.725 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:31:08.721+0000 7fed490e0640 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 -- 0x7fed4415d4e0 con 0x7fed4406fed0 2024-04-25T10:31:08.725 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:31:08.721+0000 7fed490e0640 1 -- --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_getmap magic: 0 -- 0x7fed4415d200 con 0x7fed44156c30 2024-04-25T10:31:08.725 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:31:08.721+0000 7fed490e0640 1 -- --> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] -- mon_getmap magic: 0 -- 0x7fed44157040 con 0x7fed440cf1c0 2024-04-25T10:31:08.725 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:31:08.721+0000 7fed42d9d640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fed4406fed0 0x7fed440b1df0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:31:08.725 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:31:08.722+0000 7fed42d9d640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fed4406fed0 0x7fed440b1df0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.37:3300/0 says I am v2:172.21.15.37:34656/0 (socket says 172.21.15.37:34656) 2024-04-25T10:31:08.725 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:31:08.722+0000 7fed42d9d640 1 -- 172.21.15.37:0/3073336843 learned_addr learned my addr 172.21.15.37:0/3073336843 (peer_addr_for_me v2:172.21.15.37:0/0) 2024-04-25T10:31:08.726 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:31:08.722+0000 7fed4359e640 1 --2- 172.21.15.37:0/3073336843 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fed44156c30 0x7fed4415b630 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:31:08.726 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:31:08.722+0000 7fed4259c640 1 --2- 172.21.15.37:0/3073336843 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7fed440cf1c0 0x7fed44144cb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:31:08.726 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:31:08.722+0000 7fed42d9d640 1 -- 172.21.15.37:0/3073336843 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7fed440cf1c0 msgr2=0x7fed44144cb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:31:08.726 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:31:08.722+0000 7fed42d9d640 1 --2- 172.21.15.37:0/3073336843 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7fed440cf1c0 0x7fed44144cb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:31:08.726 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:31:08.722+0000 7fed42d9d640 1 -- 172.21.15.37:0/3073336843 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fed44156c30 msgr2=0x7fed4415b630 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:31:08.726 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:31:08.722+0000 7fed42d9d640 1 --2- 172.21.15.37:0/3073336843 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fed44156c30 0x7fed4415b630 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:31:08.726 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:31:08.722+0000 7fed42d9d640 1 -- 172.21.15.37:0/3073336843 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fed44154c20 con 0x7fed4406fed0 2024-04-25T10:31:08.726 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:31:08.722+0000 7fed42d9d640 1 --2- 172.21.15.37:0/3073336843 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fed4406fed0 0x7fed440b1df0 secure :-1 s=READY pgs=721 cs=0 l=1 rev1=1 crypto rx=0x7fed3802a270 tx=0x7fed380046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:31:08.726 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:31:08.722+0000 7fed2b7fe640 1 -- 172.21.15.37:0/3073336843 <== mon.0 v2:172.21.15.37:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fed38037040 con 0x7fed4406fed0 2024-04-25T10:31:08.726 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:31:08.722+0000 7fed2b7fe640 1 -- 172.21.15.37:0/3073336843 <== mon.0 v2:172.21.15.37:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fed38042070 con 0x7fed4406fed0 2024-04-25T10:31:08.726 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:31:08.722+0000 7fed490e0640 1 -- 172.21.15.37:0/3073336843 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fed44154f00 con 0x7fed4406fed0 2024-04-25T10:31:08.726 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:31:08.722+0000 7fed2b7fe640 1 -- 172.21.15.37:0/3073336843 <== mon.0 v2:172.21.15.37:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fed38030b90 con 0x7fed4406fed0 2024-04-25T10:31:08.726 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:31:08.723+0000 7fed490e0640 1 -- 172.21.15.37:0/3073336843 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fed441554c0 con 0x7fed4406fed0 2024-04-25T10:31:08.727 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:31:08.724+0000 7fed490e0640 1 -- 172.21.15.37:0/3073336843 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fed4406a140 con 0x7fed4406fed0 2024-04-25T10:31:08.727 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:31:08.724+0000 7fed2b7fe640 1 -- 172.21.15.37:0/3073336843 <== mon.0 v2:172.21.15.37:3300/0 4 ==== mgrmap(e 5) ==== 96782+0+0 (secure 0 0 0) 0x7fed38047460 con 0x7fed4406fed0 2024-04-25T10:31:08.727 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:31:08.725+0000 7fed2b7fe640 1 --2- 172.21.15.37:0/3073336843 >> [v2:172.21.15.203:6832/2418205237,v1:172.21.15.203:6833/2418205237] conn(0x7fed300744b0 0x7fed300769c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:31:08.728 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:31:08.725+0000 7fed2b7fe640 1 -- 172.21.15.37:0/3073336843 <== mon.0 v2:172.21.15.37:3300/0 5 ==== osd_map(50..50 src has 1..50) ==== 5029+0+0 (secure 0 0 0) 0x7fed380b5c20 con 0x7fed4406fed0 2024-04-25T10:31:08.728 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:31:08.725+0000 7fed4259c640 1 --2- 172.21.15.37:0/3073336843 >> [v2:172.21.15.203:6832/2418205237,v1:172.21.15.203:6833/2418205237] conn(0x7fed300744b0 0x7fed300769c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:31:08.728 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:31:08.725+0000 7fed4259c640 1 --2- 172.21.15.37:0/3073336843 >> [v2:172.21.15.203:6832/2418205237,v1:172.21.15.203:6833/2418205237] conn(0x7fed300744b0 0x7fed300769c0 secure :-1 s=READY pgs=324 cs=0 l=1 rev1=1 crypto rx=0x7fed4414ebe0 tx=0x7fed34008040 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:31:08.735 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:31:08.733+0000 7fed2b7fe640 1 -- 172.21.15.37:0/3073336843 <== mon.0 v2:172.21.15.37:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187246 (secure 0 0 0) 0x7fed440702e0 con 0x7fed4406fed0 2024-04-25T10:31:08.923 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:31:08.921+0000 7fed490e0640 1 -- 172.21.15.37:0/3073336843 --> [v2:172.21.15.203:6832/2418205237,v1:172.21.15.203:6833/2418205237] -- mgr_command(tid 0: {"prefix": "osd deep-scrub", "who": "1", "target": ["mon-mgr", ""]}) -- 0x7fed441451f0 con 0x7fed300744b0 2024-04-25T10:31:08.923 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:31:08.921+0000 7fed2b7fe640 1 -- 172.21.15.37:0/3073336843 <== mgr.4100 v2:172.21.15.203:6832/2418205237 1 ==== mgr_command_reply(tid 0: 0 instructed osd(s) 1 to deep-scrub) ==== 41+0+0 (secure 0 0 0) 0x7fed441451f0 con 0x7fed300744b0 2024-04-25T10:31:08.923 INFO:teuthology.orchestra.run.smithi037.stderr:instructed osd(s) 1 to deep-scrub 2024-04-25T10:31:08.926 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:31:08.924+0000 7fed490e0640 1 -- 172.21.15.37:0/3073336843 >> [v2:172.21.15.203:6832/2418205237,v1:172.21.15.203:6833/2418205237] conn(0x7fed300744b0 msgr2=0x7fed300769c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:31:08.926 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:31:08.924+0000 7fed490e0640 1 --2- 172.21.15.37:0/3073336843 >> [v2:172.21.15.203:6832/2418205237,v1:172.21.15.203:6833/2418205237] conn(0x7fed300744b0 0x7fed300769c0 secure :-1 s=READY pgs=324 cs=0 l=1 rev1=1 crypto rx=0x7fed4414ebe0 tx=0x7fed34008040 comp rx=0 tx=0).stop 2024-04-25T10:31:08.926 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:31:08.924+0000 7fed490e0640 1 -- 172.21.15.37:0/3073336843 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fed4406fed0 msgr2=0x7fed440b1df0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:31:08.926 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:31:08.924+0000 7fed490e0640 1 --2- 172.21.15.37:0/3073336843 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fed4406fed0 0x7fed440b1df0 secure :-1 s=READY pgs=721 cs=0 l=1 rev1=1 crypto rx=0x7fed3802a270 tx=0x7fed380046c0 comp rx=0 tx=0).stop 2024-04-25T10:31:08.926 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:31:08.924+0000 7fed490e0640 1 -- 172.21.15.37:0/3073336843 shutdown_connections 2024-04-25T10:31:08.926 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:31:08.924+0000 7fed490e0640 1 --2- 172.21.15.37:0/3073336843 >> [v2:172.21.15.203:6832/2418205237,v1:172.21.15.203:6833/2418205237] conn(0x7fed300744b0 0x7fed300769c0 unknown :-1 s=CLOSED pgs=324 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:31:08.926 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:31:08.924+0000 7fed490e0640 1 --2- 172.21.15.37:0/3073336843 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fed44156c30 0x7fed4415b630 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:31:08.926 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:31:08.924+0000 7fed490e0640 1 --2- 172.21.15.37:0/3073336843 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7fed440cf1c0 0x7fed44144cb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:31:08.927 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:31:08.924+0000 7fed490e0640 1 --2- 172.21.15.37:0/3073336843 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fed4406fed0 0x7fed440b1df0 unknown :-1 s=CLOSED pgs=721 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:31:08.927 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:31:08.924+0000 7fed490e0640 1 -- 172.21.15.37:0/3073336843 >> 172.21.15.37:0/3073336843 conn(0x7fed440b36b0 msgr2=0x7fed441524b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:31:08.927 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:31:08.924+0000 7fed490e0640 1 -- 172.21.15.37:0/3073336843 shutdown_connections 2024-04-25T10:31:08.927 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:31:08.924+0000 7fed490e0640 1 -- 172.21.15.37:0/3073336843 wait complete. 2024-04-25T10:31:08.937 INFO:tasks.ceph:Scrubbing osd.2 2024-04-25T10:31:08.937 DEBUG:teuthology.orchestra.run.smithi037:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.2 config set osd_debug_deep_scrub_sleep 0 2024-04-25T10:31:09.053 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:31:09.051+0000 7f675157d640 1 Processor -- start 2024-04-25T10:31:09.053 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:31:09.051+0000 7f675157d640 1 -- start start 2024-04-25T10:31:09.057 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:31:09.052+0000 7f675157d640 1 --2- >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f674c08cc90 0x7f674c09bd00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:31:09.057 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:31:09.052+0000 7f675157d640 1 --2- >> v2:172.21.15.37:3300/0 conn(0x7f674c09c240 0x7f674c06d9f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:31:09.057 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:31:09.052+0000 7f675157d640 1 --2- >> v2:172.21.15.203:3300/0 conn(0x7f674c0b9260 0x7f674c06df30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:31:09.057 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:31:09.052+0000 7f675157d640 1 -- --> v2:172.21.15.37:3300/0 -- mon_getmap magic: 0 -- 0x7f674c09ca20 con 0x7f674c09c240 2024-04-25T10:31:09.057 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:31:09.052+0000 7f675157d640 1 -- --> v2:172.21.15.203:3300/0 -- mon_getmap magic: 0 -- 0x7f674c0b4c70 con 0x7f674c0b9260 2024-04-25T10:31:09.057 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:31:09.052+0000 7f675157d640 1 -- --> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] -- mon_getmap magic: 0 -- 0x7f674c09ccc0 con 0x7f674c08cc90 2024-04-25T10:31:09.057 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:31:09.052+0000 7f674a7fc640 1 --2- >> v2:172.21.15.37:3300/0 conn(0x7f674c09c240 0x7f674c06d9f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:31:09.057 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:31:09.052+0000 7f674a7fc640 1 --2- >> v2:172.21.15.37:3300/0 conn(0x7f674c09c240 0x7f674c06d9f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.37:3300/0 says I am v2:172.21.15.37:34664/0 (socket says 172.21.15.37:34664) 2024-04-25T10:31:09.057 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:31:09.052+0000 7f674a7fc640 1 -- 172.21.15.37:0/62604203 learned_addr learned my addr 172.21.15.37:0/62604203 (peer_addr_for_me v2:172.21.15.37:0/0) 2024-04-25T10:31:09.057 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:31:09.052+0000 7f674b7fe640 1 --2- 172.21.15.37:0/62604203 >> v2:172.21.15.203:3300/0 conn(0x7f674c0b9260 0x7f674c06df30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:31:09.057 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:31:09.052+0000 7f674affd640 1 --2- 172.21.15.37:0/62604203 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f674c08cc90 0x7f674c09bd00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:31:09.058 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:31:09.053+0000 7f674a7fc640 1 -- 172.21.15.37:0/62604203 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f674c08cc90 msgr2=0x7f674c09bd00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:31:09.058 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:31:09.053+0000 7f674a7fc640 1 --2- 172.21.15.37:0/62604203 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f674c08cc90 0x7f674c09bd00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:31:09.058 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:31:09.053+0000 7f674a7fc640 1 -- 172.21.15.37:0/62604203 >> v2:172.21.15.203:3300/0 conn(0x7f674c0b9260 msgr2=0x7f674c06df30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:31:09.058 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:31:09.053+0000 7f674a7fc640 1 --2- 172.21.15.37:0/62604203 >> v2:172.21.15.203:3300/0 conn(0x7f674c0b9260 0x7f674c06df30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:31:09.058 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:31:09.053+0000 7f674a7fc640 1 -- 172.21.15.37:0/62604203 --> v2:172.21.15.37:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f674c0d2790 con 0x7f674c09c240 2024-04-25T10:31:09.058 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:31:09.053+0000 7f674affd640 1 --2- 172.21.15.37:0/62604203 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f674c08cc90 0x7f674c09bd00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T10:31:09.058 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:31:09.053+0000 7f674a7fc640 1 --2- 172.21.15.37:0/62604203 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f674c09c240 0x7f674c06d9f0 secure :-1 s=READY pgs=722 cs=0 l=1 rev1=1 crypto rx=0x7f674c09d3b0 tx=0x7f674002a1e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=3b7ff4a72d2f2f40 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:31:09.058 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:31:09.054+0000 7f6749ffb640 1 -- 172.21.15.37:0/62604203 <== mon.0 v2:172.21.15.37:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6740037070 con 0x7f674c09c240 2024-04-25T10:31:09.058 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:31:09.054+0000 7f6749ffb640 1 -- 172.21.15.37:0/62604203 <== mon.0 v2:172.21.15.37:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f6740032440 con 0x7f674c09c240 2024-04-25T10:31:09.058 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:31:09.054+0000 7f6749ffb640 1 -- 172.21.15.37:0/62604203 <== mon.0 v2:172.21.15.37:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6740032810 con 0x7f674c09c240 2024-04-25T10:31:09.058 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:31:09.054+0000 7f675157d640 1 -- 172.21.15.37:0/62604203 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f674c09c240 msgr2=0x7f674c06d9f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:31:09.058 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:31:09.054+0000 7f675157d640 1 --2- 172.21.15.37:0/62604203 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f674c09c240 0x7f674c06d9f0 secure :-1 s=READY pgs=722 cs=0 l=1 rev1=1 crypto rx=0x7f674c09d3b0 tx=0x7f674002a1e0 comp rx=0 tx=0).stop 2024-04-25T10:31:09.059 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:31:09.054+0000 7f675157d640 1 -- 172.21.15.37:0/62604203 shutdown_connections 2024-04-25T10:31:09.059 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:31:09.054+0000 7f675157d640 1 --2- 172.21.15.37:0/62604203 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f674c09c240 0x7f674c06d9f0 unknown :-1 s=CLOSED pgs=722 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:31:09.059 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:31:09.054+0000 7f675157d640 1 --2- 172.21.15.37:0/62604203 >> v2:172.21.15.203:3300/0 conn(0x7f674c0b9260 0x7f674c06df30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:31:09.059 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:31:09.054+0000 7f675157d640 1 --2- 172.21.15.37:0/62604203 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f674c08cc90 0x7f674c09bd00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:31:09.059 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:31:09.054+0000 7f675157d640 1 -- 172.21.15.37:0/62604203 >> 172.21.15.37:0/62604203 conn(0x7f674c0b54d0 msgr2=0x7f674c0b58e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:31:09.059 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:31:09.054+0000 7f675157d640 1 -- 172.21.15.37:0/62604203 shutdown_connections 2024-04-25T10:31:09.059 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:31:09.054+0000 7f675157d640 1 -- 172.21.15.37:0/62604203 wait complete. 2024-04-25T10:31:09.061 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:31:09.056+0000 7f675157d640 1 Processor -- start 2024-04-25T10:31:09.061 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:31:09.056+0000 7f675157d640 1 -- start start 2024-04-25T10:31:09.061 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:31:09.058+0000 7f675157d640 1 --2- >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f674c09c240 0x7f674c0b3180 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:31:09.061 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:31:09.058+0000 7f675157d640 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f674c0b9260 0x7f674c1dbdc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:31:09.061 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:31:09.058+0000 7f675157d640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f674c0cba80 0x7f674c1ea820 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:31:09.062 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:31:09.058+0000 7f675157d640 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 -- 0x7f674c0d3030 con 0x7f674c0cba80 2024-04-25T10:31:09.062 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:31:09.058+0000 7f675157d640 1 -- --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_getmap magic: 0 -- 0x7f674c0d2d50 con 0x7f674c0b9260 2024-04-25T10:31:09.062 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:31:09.058+0000 7f675157d640 1 -- --> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] -- mon_getmap magic: 0 -- 0x7f674c0d00d0 con 0x7f674c09c240 2024-04-25T10:31:09.062 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:31:09.058+0000 7f674b7fe640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f674c0cba80 0x7f674c1ea820 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:31:09.062 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:31:09.058+0000 7f674b7fe640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f674c0cba80 0x7f674c1ea820 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.37:3300/0 says I am v2:172.21.15.37:34674/0 (socket says 172.21.15.37:34674) 2024-04-25T10:31:09.062 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:31:09.058+0000 7f674b7fe640 1 -- 172.21.15.37:0/382532049 learned_addr learned my addr 172.21.15.37:0/382532049 (peer_addr_for_me v2:172.21.15.37:0/0) 2024-04-25T10:31:09.062 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:31:09.058+0000 7f674a7fc640 1 --2- 172.21.15.37:0/382532049 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f674c0b9260 0x7f674c1dbdc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:31:09.062 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:31:09.058+0000 7f674affd640 1 --2- 172.21.15.37:0/382532049 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f674c09c240 0x7f674c0b3180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:31:09.062 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:31:09.058+0000 7f674b7fe640 1 -- 172.21.15.37:0/382532049 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f674c09c240 msgr2=0x7f674c0b3180 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:31:09.062 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:31:09.058+0000 7f674b7fe640 1 --2- 172.21.15.37:0/382532049 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f674c09c240 0x7f674c0b3180 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:31:09.062 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:31:09.058+0000 7f674b7fe640 1 -- 172.21.15.37:0/382532049 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f674c0b9260 msgr2=0x7f674c1dbdc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:31:09.062 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:31:09.058+0000 7f674b7fe640 1 --2- 172.21.15.37:0/382532049 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f674c0b9260 0x7f674c1dbdc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:31:09.062 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:31:09.058+0000 7f674b7fe640 1 -- 172.21.15.37:0/382532049 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f674c1dc300 con 0x7f674c0cba80 2024-04-25T10:31:09.062 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:31:09.058+0000 7f674a7fc640 1 --2- 172.21.15.37:0/382532049 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f674c0b9260 0x7f674c1dbdc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-25T10:31:09.063 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:31:09.058+0000 7f674b7fe640 1 --2- 172.21.15.37:0/382532049 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f674c0cba80 0x7f674c1ea820 secure :-1 s=READY pgs=723 cs=0 l=1 rev1=1 crypto rx=0x7f674400efc0 tx=0x7f674400c490 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:31:09.063 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:31:09.059+0000 7f67337fe640 1 -- 172.21.15.37:0/382532049 <== mon.0 v2:172.21.15.37:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6744018020 con 0x7f674c0cba80 2024-04-25T10:31:09.063 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:31:09.059+0000 7f67337fe640 1 -- 172.21.15.37:0/382532049 <== mon.0 v2:172.21.15.37:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f6744014480 con 0x7f674c0cba80 2024-04-25T10:31:09.063 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:31:09.059+0000 7f67337fe640 1 -- 172.21.15.37:0/382532049 <== mon.0 v2:172.21.15.37:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6744014870 con 0x7f674c0cba80 2024-04-25T10:31:09.063 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:31:09.059+0000 7f675157d640 1 -- 172.21.15.37:0/382532049 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f674c0cacc0 con 0x7f674c0cba80 2024-04-25T10:31:09.063 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:31:09.059+0000 7f675157d640 1 -- 172.21.15.37:0/382532049 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f674c0cb280 con 0x7f674c0cba80 2024-04-25T10:31:09.064 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:31:09.060+0000 7f675157d640 1 -- 172.21.15.37:0/382532049 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_get_version(what=osdmap handle=1) -- 0x7f674c0caa70 con 0x7f674c0cba80 2024-04-25T10:31:09.064 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:31:09.060+0000 7f67337fe640 1 -- 172.21.15.37:0/382532049 <== mon.0 v2:172.21.15.37:3300/0 4 ==== mgrmap(e 5) ==== 96782+0+0 (secure 0 0 0) 0x7f6744005d10 con 0x7f674c0cba80 2024-04-25T10:31:09.065 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:31:09.061+0000 7f67337fe640 1 --2- 172.21.15.37:0/382532049 >> [v2:172.21.15.203:6832/2418205237,v1:172.21.15.203:6833/2418205237] conn(0x7f67380744b0 0x7f67380769c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:31:09.065 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:31:09.061+0000 7f67337fe640 1 -- 172.21.15.37:0/382532049 <== mon.0 v2:172.21.15.37:3300/0 5 ==== osd_map(50..50 src has 1..50) ==== 5029+0+0 (secure 0 0 0) 0x7f6744095920 con 0x7f674c0cba80 2024-04-25T10:31:09.065 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:31:09.061+0000 7f67337fe640 1 --2- 172.21.15.37:0/382532049 >> [v2:172.21.15.37:6808/2754669649,v1:172.21.15.37:6809/2754669649] conn(0x7f673807a460 0x7f673807c8b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:31:09.065 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:31:09.061+0000 7f67337fe640 1 -- 172.21.15.37:0/382532049 --> [v2:172.21.15.37:6808/2754669649,v1:172.21.15.37:6809/2754669649] -- command(tid 1: {"prefix": "get_command_descriptions"}) -- 0x7f674400be50 con 0x7f673807a460 2024-04-25T10:31:09.065 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:31:09.061+0000 7f674affd640 1 --2- 172.21.15.37:0/382532049 >> [v2:172.21.15.203:6832/2418205237,v1:172.21.15.203:6833/2418205237] conn(0x7f67380744b0 0x7f67380769c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:31:09.065 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:31:09.061+0000 7f67337fe640 1 -- 172.21.15.37:0/382532049 <== mon.0 v2:172.21.15.37:3300/0 6 ==== mon_get_version_reply(handle=1 version=50) ==== 24+0+0 (secure 0 0 0) 0x7f674409a2c0 con 0x7f674c0cba80 2024-04-25T10:31:09.065 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:31:09.061+0000 7f674a7fc640 1 --2- 172.21.15.37:0/382532049 >> [v2:172.21.15.37:6808/2754669649,v1:172.21.15.37:6809/2754669649] conn(0x7f673807a460 0x7f673807c8b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:31:09.065 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:31:09.062+0000 7f674a7fc640 1 --2- 172.21.15.37:0/382532049 >> [v2:172.21.15.37:6808/2754669649,v1:172.21.15.37:6809/2754669649] conn(0x7f673807a460 0x7f673807c8b0 crc :-1 s=READY pgs=17 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:31:09.065 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:31:09.062+0000 7f674affd640 1 --2- 172.21.15.37:0/382532049 >> [v2:172.21.15.203:6832/2418205237,v1:172.21.15.203:6833/2418205237] conn(0x7f67380744b0 0x7f67380769c0 secure :-1 s=READY pgs=325 cs=0 l=1 rev1=1 crypto rx=0x7f674c0b53b0 tx=0x7f673c008040 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:31:09.065 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:31:09.062+0000 7f67337fe640 1 -- 172.21.15.37:0/382532049 <== osd.2 v2:172.21.15.37:6808/2754669649 1 ==== command_reply(tid 1: 0 ) ==== 8+0+27193 (crc 0 0 0) 0x7f674400be50 con 0x7f673807a460 2024-04-25T10:31:09.078 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:31:09.075+0000 7f675157d640 1 -- 172.21.15.37:0/382532049 --> [v2:172.21.15.37:6808/2754669649,v1:172.21.15.37:6809/2754669649] -- command(tid 2: {"prefix": "config set", "var": "osd_debug_deep_scrub_sleep", "val": ["0"]}) -- 0x7f674c0b2c30 con 0x7f673807a460 2024-04-25T10:31:09.078 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:31:09.076+0000 7f67337fe640 1 -- 172.21.15.37:0/382532049 <== osd.2 v2:172.21.15.37:6808/2754669649 2 ==== command_reply(tid 2: 0 ) ==== 8+0+97 (crc 0 0 0) 0x7f674c0b2c30 con 0x7f673807a460 2024-04-25T10:31:09.078 INFO:teuthology.orchestra.run.smithi037.stdout:{ 2024-04-25T10:31:09.078 INFO:teuthology.orchestra.run.smithi037.stdout: "success": "osd_debug_deep_scrub_sleep = '' (not observed, change may require restart) " 2024-04-25T10:31:09.078 INFO:teuthology.orchestra.run.smithi037.stdout:} 2024-04-25T10:31:09.079 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:31:09.076+0000 7f675157d640 1 -- 172.21.15.37:0/382532049 >> [v2:172.21.15.37:6808/2754669649,v1:172.21.15.37:6809/2754669649] conn(0x7f673807a460 msgr2=0x7f673807c8b0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:31:09.079 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:31:09.076+0000 7f675157d640 1 --2- 172.21.15.37:0/382532049 >> [v2:172.21.15.37:6808/2754669649,v1:172.21.15.37:6809/2754669649] conn(0x7f673807a460 0x7f673807c8b0 crc :-1 s=READY pgs=17 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:31:09.079 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:31:09.076+0000 7f675157d640 1 -- 172.21.15.37:0/382532049 >> [v2:172.21.15.203:6832/2418205237,v1:172.21.15.203:6833/2418205237] conn(0x7f67380744b0 msgr2=0x7f67380769c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:31:09.079 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:31:09.076+0000 7f675157d640 1 --2- 172.21.15.37:0/382532049 >> [v2:172.21.15.203:6832/2418205237,v1:172.21.15.203:6833/2418205237] conn(0x7f67380744b0 0x7f67380769c0 secure :-1 s=READY pgs=325 cs=0 l=1 rev1=1 crypto rx=0x7f674c0b53b0 tx=0x7f673c008040 comp rx=0 tx=0).stop 2024-04-25T10:31:09.079 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:31:09.076+0000 7f675157d640 1 -- 172.21.15.37:0/382532049 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f674c0cba80 msgr2=0x7f674c1ea820 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:31:09.080 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:31:09.076+0000 7f675157d640 1 --2- 172.21.15.37:0/382532049 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f674c0cba80 0x7f674c1ea820 secure :-1 s=READY pgs=723 cs=0 l=1 rev1=1 crypto rx=0x7f674400efc0 tx=0x7f674400c490 comp rx=0 tx=0).stop 2024-04-25T10:31:09.080 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:31:09.076+0000 7f674b7fe640 1 -- 172.21.15.37:0/382532049 reap_dead start 2024-04-25T10:31:09.080 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:31:09.076+0000 7f675157d640 1 -- 172.21.15.37:0/382532049 shutdown_connections 2024-04-25T10:31:09.080 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:31:09.076+0000 7f675157d640 1 -- 172.21.15.37:0/382532049 >> 172.21.15.37:0/382532049 conn(0x7f674c0b54d0 msgr2=0x7f674c0cf2a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:31:09.080 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:31:09.076+0000 7f675157d640 1 -- 172.21.15.37:0/382532049 shutdown_connections 2024-04-25T10:31:09.080 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:31:09.076+0000 7f675157d640 1 -- 172.21.15.37:0/382532049 wait complete. 2024-04-25T10:31:09.089 DEBUG:teuthology.orchestra.run.smithi037:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd deep-scrub 2 2024-04-25T10:31:09.202 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:31:09.200+0000 7fa1d4205640 1 Processor -- start 2024-04-25T10:31:09.202 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:31:09.200+0000 7fa1d4205640 1 -- start start 2024-04-25T10:31:09.204 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:31:09.201+0000 7fa1d4205640 1 --2- >> v2:172.21.15.37:3300/0 conn(0x7fa1cc0b7600 0x7fa1cc0b7a10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:31:09.204 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:31:09.201+0000 7fa1d4205640 1 --2- >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7fa1cc0d4560 0x7fa1cc0d4970 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:31:09.204 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:31:09.201+0000 7fa1d4205640 1 -- --> v1:172.21.15.203:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fa1cc0b7f50 con 0x7fa1cc0b4bb0 2024-04-25T10:31:09.204 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:31:09.201+0000 7fa1d4205640 1 -- --> v2:172.21.15.37:3300/0 -- mon_getmap magic: 0 -- 0x7fa1cc161190 con 0x7fa1cc0b7600 2024-04-25T10:31:09.205 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:31:09.201+0000 7fa1d4205640 1 -- --> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] -- mon_getmap magic: 0 -- 0x7fa1cc161470 con 0x7fa1cc0d4560 2024-04-25T10:31:09.205 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:31:09.201+0000 7fa1d17a0640 1 --2- >> v2:172.21.15.37:3300/0 conn(0x7fa1cc0b7600 0x7fa1cc0b7a10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:31:09.205 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:31:09.202+0000 7fa1d17a0640 1 --2- >> v2:172.21.15.37:3300/0 conn(0x7fa1cc0b7600 0x7fa1cc0b7a10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.37:3300/0 says I am v2:172.21.15.37:34678/0 (socket says 172.21.15.37:34678) 2024-04-25T10:31:09.205 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:31:09.202+0000 7fa1d17a0640 1 -- 172.21.15.37:0/3459056901 learned_addr learned my addr 172.21.15.37:0/3459056901 (peer_addr_for_me v2:172.21.15.37:0/0) 2024-04-25T10:31:09.205 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:31:09.202+0000 7fa1d27a2640 1 --2- 172.21.15.37:0/3459056901 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7fa1cc0d4560 0x7fa1cc0d4970 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:31:09.205 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:31:09.202+0000 7fa1d17a0640 1 -- 172.21.15.37:0/3459056901 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7fa1cc0d4560 msgr2=0x7fa1cc0d4970 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:31:09.205 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:31:09.202+0000 7fa1d17a0640 1 --2- 172.21.15.37:0/3459056901 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7fa1cc0d4560 0x7fa1cc0d4970 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:31:09.205 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:31:09.202+0000 7fa1d17a0640 1 -- 172.21.15.37:0/3459056901 >> v1:172.21.15.203:6789/0 conn(0x7fa1cc0b4bb0 legacy=0x7fa1cc1646b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:31:09.205 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:31:09.202+0000 7fa1d17a0640 1 -- 172.21.15.37:0/3459056901 --> v2:172.21.15.37:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fa1cc163eb0 con 0x7fa1cc0b7600 2024-04-25T10:31:09.206 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:31:09.203+0000 7fa1d17a0640 1 --2- 172.21.15.37:0/3459056901 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fa1cc0b7600 0x7fa1cc0b7a10 secure :-1 s=READY pgs=724 cs=0 l=1 rev1=1 crypto rx=0x7fa1bc009870 tx=0x7fa1bc02a270 comp rx=0 tx=0).ready entity=mon.0 client_cookie=424642be1c77cf80 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:31:09.206 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:31:09.203+0000 7fa1d0f9f640 1 -- 172.21.15.37:0/3459056901 <== mon.0 v2:172.21.15.37:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa1bc037070 con 0x7fa1cc0b7600 2024-04-25T10:31:09.206 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:31:09.203+0000 7fa1d0f9f640 1 -- 172.21.15.37:0/3459056901 <== mon.0 v2:172.21.15.37:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fa1bc032440 con 0x7fa1cc0b7600 2024-04-25T10:31:09.206 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:31:09.203+0000 7fa1d0f9f640 1 -- 172.21.15.37:0/3459056901 <== mon.0 v2:172.21.15.37:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa1bc032810 con 0x7fa1cc0b7600 2024-04-25T10:31:09.207 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:31:09.203+0000 7fa1d4205640 1 -- 172.21.15.37:0/3459056901 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fa1cc0b7600 msgr2=0x7fa1cc0b7a10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:31:09.207 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:31:09.203+0000 7fa1d4205640 1 --2- 172.21.15.37:0/3459056901 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fa1cc0b7600 0x7fa1cc0b7a10 secure :-1 s=READY pgs=724 cs=0 l=1 rev1=1 crypto rx=0x7fa1bc009870 tx=0x7fa1bc02a270 comp rx=0 tx=0).stop 2024-04-25T10:31:09.207 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:31:09.204+0000 7fa1d4205640 1 -- 172.21.15.37:0/3459056901 shutdown_connections 2024-04-25T10:31:09.207 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:31:09.204+0000 7fa1d4205640 1 --2- 172.21.15.37:0/3459056901 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7fa1cc0d4560 0x7fa1cc0d4970 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:31:09.207 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:31:09.204+0000 7fa1d4205640 1 --2- 172.21.15.37:0/3459056901 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fa1cc0b7600 0x7fa1cc0b7a10 unknown :-1 s=CLOSED pgs=724 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:31:09.207 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:31:09.204+0000 7fa1d4205640 1 -- 172.21.15.37:0/3459056901 >> 172.21.15.37:0/3459056901 conn(0x7fa1cc0d27e0 msgr2=0x7fa1cc0d2bf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:31:09.207 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:31:09.204+0000 7fa1d4205640 1 -- 172.21.15.37:0/3459056901 shutdown_connections 2024-04-25T10:31:09.207 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:31:09.204+0000 7fa1d4205640 1 -- 172.21.15.37:0/3459056901 wait complete. 2024-04-25T10:31:09.207 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:31:09.205+0000 7fa1d4205640 1 Processor -- start 2024-04-25T10:31:09.208 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:31:09.205+0000 7fa1d4205640 1 -- start start 2024-04-25T10:31:09.210 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:31:09.207+0000 7fa1d4205640 1 --2- >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7fa1cc0b4bb0 0x7fa1cc14fde0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:31:09.210 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:31:09.207+0000 7fa1d4205640 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fa1cc0b7600 0x7fa1cc1590a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:31:09.211 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:31:09.207+0000 7fa1d4205640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fa1cc0d4560 0x7fa1cc0ba1b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:31:09.211 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:31:09.207+0000 7fa1d4205640 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 -- 0x7fa1cc0d09e0 con 0x7fa1cc0d4560 2024-04-25T10:31:09.211 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:31:09.207+0000 7fa1d4205640 1 -- --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_getmap magic: 0 -- 0x7fa1cc0b4fc0 con 0x7fa1cc0b7600 2024-04-25T10:31:09.211 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:31:09.207+0000 7fa1d4205640 1 -- --> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] -- mon_getmap magic: 0 -- 0x7fa1cc0d0b60 con 0x7fa1cc0b4bb0 2024-04-25T10:31:09.211 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:31:09.207+0000 7fa1d27a2640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fa1cc0d4560 0x7fa1cc0ba1b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:31:09.211 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:31:09.207+0000 7fa1d27a2640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fa1cc0d4560 0x7fa1cc0ba1b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.37:3300/0 says I am v2:172.21.15.37:34680/0 (socket says 172.21.15.37:34680) 2024-04-25T10:31:09.211 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:31:09.207+0000 7fa1d27a2640 1 -- 172.21.15.37:0/4041179613 learned_addr learned my addr 172.21.15.37:0/4041179613 (peer_addr_for_me v2:172.21.15.37:0/0) 2024-04-25T10:31:09.211 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:31:09.207+0000 7fa1d1fa1640 1 --2- 172.21.15.37:0/4041179613 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7fa1cc0b4bb0 0x7fa1cc14fde0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:31:09.211 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:31:09.207+0000 7fa1d17a0640 1 --2- 172.21.15.37:0/4041179613 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fa1cc0b7600 0x7fa1cc1590a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:31:09.211 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:31:09.207+0000 7fa1d27a2640 1 -- 172.21.15.37:0/4041179613 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7fa1cc0b4bb0 msgr2=0x7fa1cc14fde0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:31:09.211 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:31:09.207+0000 7fa1d27a2640 1 --2- 172.21.15.37:0/4041179613 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7fa1cc0b4bb0 0x7fa1cc14fde0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:31:09.211 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:31:09.207+0000 7fa1d27a2640 1 -- 172.21.15.37:0/4041179613 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fa1cc0b7600 msgr2=0x7fa1cc1590a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:31:09.211 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:31:09.207+0000 7fa1d27a2640 1 --2- 172.21.15.37:0/4041179613 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fa1cc0b7600 0x7fa1cc1590a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:31:09.211 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:31:09.207+0000 7fa1d27a2640 1 -- 172.21.15.37:0/4041179613 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fa1cc157740 con 0x7fa1cc0d4560 2024-04-25T10:31:09.212 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:31:09.207+0000 7fa1d27a2640 1 --2- 172.21.15.37:0/4041179613 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fa1cc0d4560 0x7fa1cc0ba1b0 secure :-1 s=READY pgs=725 cs=0 l=1 rev1=1 crypto rx=0x7fa1c4007e60 tx=0x7fa1c400e4b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:31:09.212 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:31:09.208+0000 7fa1c27fc640 1 -- 172.21.15.37:0/4041179613 <== mon.0 v2:172.21.15.37:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa1c4014420 con 0x7fa1cc0d4560 2024-04-25T10:31:09.212 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:31:09.208+0000 7fa1c27fc640 1 -- 172.21.15.37:0/4041179613 <== mon.0 v2:172.21.15.37:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fa1c4019070 con 0x7fa1cc0d4560 2024-04-25T10:31:09.212 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:31:09.208+0000 7fa1c27fc640 1 -- 172.21.15.37:0/4041179613 <== mon.0 v2:172.21.15.37:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa1c4014820 con 0x7fa1cc0d4560 2024-04-25T10:31:09.212 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:31:09.208+0000 7fa1d4205640 1 -- 172.21.15.37:0/4041179613 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fa1cc157a20 con 0x7fa1cc0d4560 2024-04-25T10:31:09.212 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:31:09.208+0000 7fa1d4205640 1 -- 172.21.15.37:0/4041179613 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fa1cc157fe0 con 0x7fa1cc0d4560 2024-04-25T10:31:09.212 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:31:09.209+0000 7fa1d4205640 1 -- 172.21.15.37:0/4041179613 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fa1cc06a140 con 0x7fa1cc0d4560 2024-04-25T10:31:09.213 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:31:09.209+0000 7fa1c27fc640 1 -- 172.21.15.37:0/4041179613 <== mon.0 v2:172.21.15.37:3300/0 4 ==== mgrmap(e 5) ==== 96782+0+0 (secure 0 0 0) 0x7fa1c40149c0 con 0x7fa1cc0d4560 2024-04-25T10:31:09.213 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:31:09.210+0000 7fa1c27fc640 1 --2- 172.21.15.37:0/4041179613 >> [v2:172.21.15.203:6832/2418205237,v1:172.21.15.203:6833/2418205237] conn(0x7fa1b00744b0 0x7fa1b00769c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:31:09.213 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:31:09.210+0000 7fa1c27fc640 1 -- 172.21.15.37:0/4041179613 <== mon.0 v2:172.21.15.37:3300/0 5 ==== osd_map(50..50 src has 1..50) ==== 5029+0+0 (secure 0 0 0) 0x7fa1c40956c0 con 0x7fa1cc0d4560 2024-04-25T10:31:09.213 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:31:09.210+0000 7fa1d1fa1640 1 --2- 172.21.15.37:0/4041179613 >> [v2:172.21.15.203:6832/2418205237,v1:172.21.15.203:6833/2418205237] conn(0x7fa1b00744b0 0x7fa1b00769c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:31:09.213 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:31:09.211+0000 7fa1d1fa1640 1 --2- 172.21.15.37:0/4041179613 >> [v2:172.21.15.203:6832/2418205237,v1:172.21.15.203:6833/2418205237] conn(0x7fa1b00744b0 0x7fa1b00769c0 secure :-1 s=READY pgs=326 cs=0 l=1 rev1=1 crypto rx=0x7fa1c8000f60 tx=0x7fa1c8009040 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:31:09.216 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:31:09.215+0000 7fa1c27fc640 1 -- 172.21.15.37:0/4041179613 <== mon.0 v2:172.21.15.37:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187246 (secure 0 0 0) 0x7fa1c405faa0 con 0x7fa1cc0d4560 2024-04-25T10:31:09.400 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:31:09.398+0000 7fa1d4205640 1 -- 172.21.15.37:0/4041179613 --> [v2:172.21.15.203:6832/2418205237,v1:172.21.15.203:6833/2418205237] -- mgr_command(tid 0: {"prefix": "osd deep-scrub", "who": "2", "target": ["mon-mgr", ""]}) -- 0x7fa1cc1549c0 con 0x7fa1b00744b0 2024-04-25T10:31:09.400 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:31:09.398+0000 7fa1c27fc640 1 -- 172.21.15.37:0/4041179613 <== mgr.4100 v2:172.21.15.203:6832/2418205237 1 ==== mgr_command_reply(tid 0: 0 instructed osd(s) 2 to deep-scrub) ==== 41+0+0 (secure 0 0 0) 0x7fa1c8001d10 con 0x7fa1b00744b0 2024-04-25T10:31:09.400 INFO:teuthology.orchestra.run.smithi037.stderr:instructed osd(s) 2 to deep-scrub 2024-04-25T10:31:09.402 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:31:09.400+0000 7fa1d4205640 1 -- 172.21.15.37:0/4041179613 >> [v2:172.21.15.203:6832/2418205237,v1:172.21.15.203:6833/2418205237] conn(0x7fa1b00744b0 msgr2=0x7fa1b00769c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:31:09.403 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:31:09.400+0000 7fa1d4205640 1 --2- 172.21.15.37:0/4041179613 >> [v2:172.21.15.203:6832/2418205237,v1:172.21.15.203:6833/2418205237] conn(0x7fa1b00744b0 0x7fa1b00769c0 secure :-1 s=READY pgs=326 cs=0 l=1 rev1=1 crypto rx=0x7fa1c8000f60 tx=0x7fa1c8009040 comp rx=0 tx=0).stop 2024-04-25T10:31:09.403 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:31:09.400+0000 7fa1d4205640 1 -- 172.21.15.37:0/4041179613 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fa1cc0d4560 msgr2=0x7fa1cc0ba1b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:31:09.403 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:31:09.400+0000 7fa1d4205640 1 --2- 172.21.15.37:0/4041179613 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fa1cc0d4560 0x7fa1cc0ba1b0 secure :-1 s=READY pgs=725 cs=0 l=1 rev1=1 crypto rx=0x7fa1c4007e60 tx=0x7fa1c400e4b0 comp rx=0 tx=0).stop 2024-04-25T10:31:09.403 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:31:09.400+0000 7fa1d4205640 1 -- 172.21.15.37:0/4041179613 shutdown_connections 2024-04-25T10:31:09.403 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:31:09.400+0000 7fa1d4205640 1 --2- 172.21.15.37:0/4041179613 >> [v2:172.21.15.203:6832/2418205237,v1:172.21.15.203:6833/2418205237] conn(0x7fa1b00744b0 0x7fa1b00769c0 unknown :-1 s=CLOSED pgs=326 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:31:09.403 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:31:09.400+0000 7fa1d4205640 1 --2- 172.21.15.37:0/4041179613 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fa1cc0d4560 0x7fa1cc0ba1b0 unknown :-1 s=CLOSED pgs=725 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:31:09.403 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:31:09.400+0000 7fa1d4205640 1 --2- 172.21.15.37:0/4041179613 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fa1cc0b7600 0x7fa1cc1590a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:31:09.403 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:31:09.400+0000 7fa1d4205640 1 --2- 172.21.15.37:0/4041179613 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7fa1cc0b4bb0 0x7fa1cc14fde0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:31:09.403 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:31:09.400+0000 7fa1d4205640 1 -- 172.21.15.37:0/4041179613 >> 172.21.15.37:0/4041179613 conn(0x7fa1cc0d27e0 msgr2=0x7fa1cc0b54c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:31:09.403 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:31:09.400+0000 7fa1d4205640 1 -- 172.21.15.37:0/4041179613 shutdown_connections 2024-04-25T10:31:09.403 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:31:09.400+0000 7fa1d4205640 1 -- 172.21.15.37:0/4041179613 wait complete. 2024-04-25T10:31:09.413 INFO:tasks.ceph:Scrubbing osd.3 2024-04-25T10:31:09.413 DEBUG:teuthology.orchestra.run.smithi037:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.3 config set osd_debug_deep_scrub_sleep 0 2024-04-25T10:31:09.530 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:31:09.527+0000 7fde6217d640 1 Processor -- start 2024-04-25T10:31:09.530 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:31:09.527+0000 7fde6217d640 1 -- start start 2024-04-25T10:31:09.533 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:31:09.529+0000 7fde6217d640 1 --2- >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7fde5c0cf2c0 0x7fde5c16b8e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:31:09.533 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:31:09.529+0000 7fde6217d640 1 -- --> v1:172.21.15.37:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fde5c0cf6d0 con 0x7fde5c0d20b0 2024-04-25T10:31:09.533 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:31:09.529+0000 7fde6217d640 1 -- --> v1:172.21.15.203:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fde5c06f450 con 0x7fde5c0ce7a0 2024-04-25T10:31:09.533 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:31:09.529+0000 7fde6217d640 1 -- --> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] -- mon_getmap magic: 0 -- 0x7fde5c0d2bd0 con 0x7fde5c0cf2c0 2024-04-25T10:31:09.533 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:31:09.529+0000 7fde5b7fe640 1 --1- >> v1:172.21.15.37:6789/0 conn(0x7fde5c0d20b0 0x7fde5c0d24c0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.37:6789/0 says I am v1:172.21.15.37:47124/0 (socket says 172.21.15.37:47124) 2024-04-25T10:31:09.533 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:31:09.529+0000 7fde5b7fe640 1 -- 172.21.15.37:0/3863814203 learned_addr learned my addr 172.21.15.37:0/3863814203 (peer_addr_for_me v1:172.21.15.37:0/0) 2024-04-25T10:31:09.533 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:31:09.529+0000 7fde5bfff640 1 --2- 172.21.15.37:0/3863814203 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7fde5c0cf2c0 0x7fde5c16b8e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:31:09.533 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:31:09.529+0000 7fde5a7fc640 1 -- 172.21.15.37:0/3863814203 <== mon.0 v1:172.21.15.37:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 1351601430 0 0) 0x7fde4c001290 con 0x7fde5c0d20b0 2024-04-25T10:31:09.533 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:31:09.529+0000 7fde5a7fc640 1 -- 172.21.15.37:0/3863814203 --> v1:172.21.15.37:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7fde44003520 con 0x7fde5c0d20b0 2024-04-25T10:31:09.533 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:31:09.530+0000 7fde5a7fc640 1 -- 172.21.15.37:0/3863814203 <== mon.0 v1:172.21.15.37:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 639230840 0 0) 0x7fde44003520 con 0x7fde5c0d20b0 2024-04-25T10:31:09.534 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:31:09.530+0000 7fde5a7fc640 1 -- 172.21.15.37:0/3863814203 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7fde5c0cf2c0 msgr2=0x7fde5c16b8e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:31:09.534 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:31:09.530+0000 7fde5a7fc640 1 --2- 172.21.15.37:0/3863814203 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7fde5c0cf2c0 0x7fde5c16b8e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:31:09.534 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:31:09.530+0000 7fde5a7fc640 1 -- 172.21.15.37:0/3863814203 >> v1:172.21.15.203:6789/0 conn(0x7fde5c0ce7a0 legacy=0x7fde5c0cebb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:31:09.534 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:31:09.530+0000 7fde5a7fc640 1 -- 172.21.15.37:0/3863814203 --> v1:172.21.15.37:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fde5c0cf950 con 0x7fde5c0d20b0 2024-04-25T10:31:09.534 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:31:09.530+0000 7fde5a7fc640 1 -- 172.21.15.37:0/3863814203 <== mon.0 v1:172.21.15.37:6789/0 3 ==== mon_map magic: 0 ==== 413+0+0 (unknown 1577794600 0 0) 0x7fde4c0019c0 con 0x7fde5c0d20b0 2024-04-25T10:31:09.534 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:31:09.530+0000 7fde5a7fc640 1 -- 172.21.15.37:0/3863814203 >> v1:172.21.15.37:6789/0 conn(0x7fde5c0d20b0 legacy=0x7fde5c0d24c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:31:09.534 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:31:09.530+0000 7fde5a7fc640 1 --2- 172.21.15.37:0/3863814203 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fde440039e0 0x7fde44003e10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:31:09.534 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:31:09.530+0000 7fde5a7fc640 1 --2- 172.21.15.37:0/3863814203 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fde44004800 0x7fde44004c10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:31:09.534 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:31:09.530+0000 7fde5a7fc640 1 --2- 172.21.15.37:0/3863814203 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7fde5c0cf2c0 0x7fde44005200 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:31:09.534 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:31:09.530+0000 7fde5a7fc640 1 -- 172.21.15.37:0/3863814203 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 -- 0x7fde5c0d2bd0 con 0x7fde440039e0 2024-04-25T10:31:09.535 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:31:09.530+0000 7fde5a7fc640 1 -- 172.21.15.37:0/3863814203 --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_getmap magic: 0 -- 0x7fde4400baf0 con 0x7fde44004800 2024-04-25T10:31:09.535 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:31:09.530+0000 7fde5a7fc640 1 -- 172.21.15.37:0/3863814203 --> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] -- mon_getmap magic: 0 -- 0x7fde4400bcf0 con 0x7fde5c0cf2c0 2024-04-25T10:31:09.535 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:31:09.530+0000 7fde5b7fe640 1 --2- 172.21.15.37:0/3863814203 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fde440039e0 0x7fde44003e10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:31:09.535 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:31:09.530+0000 7fde5affd640 1 --2- 172.21.15.37:0/3863814203 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fde44004800 0x7fde44004c10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:31:09.535 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:31:09.530+0000 7fde5bfff640 1 --2- 172.21.15.37:0/3863814203 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7fde5c0cf2c0 0x7fde44005200 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:31:09.535 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:31:09.530+0000 7fde5b7fe640 1 -- 172.21.15.37:0/3863814203 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7fde5c0cf2c0 msgr2=0x7fde44005200 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:31:09.536 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:31:09.530+0000 7fde5b7fe640 1 --2- 172.21.15.37:0/3863814203 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7fde5c0cf2c0 0x7fde44005200 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:31:09.536 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:31:09.530+0000 7fde5b7fe640 1 -- 172.21.15.37:0/3863814203 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fde44004800 msgr2=0x7fde44004c10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:31:09.536 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:31:09.530+0000 7fde5b7fe640 1 --2- 172.21.15.37:0/3863814203 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fde44004800 0x7fde44004c10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:31:09.536 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:31:09.530+0000 7fde5b7fe640 1 -- 172.21.15.37:0/3863814203 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fde44003520 con 0x7fde440039e0 2024-04-25T10:31:09.536 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:31:09.530+0000 7fde5bfff640 1 --2- 172.21.15.37:0/3863814203 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7fde5c0cf2c0 0x7fde44005200 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T10:31:09.536 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:31:09.531+0000 7fde5b7fe640 1 --2- 172.21.15.37:0/3863814203 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fde440039e0 0x7fde44003e10 secure :-1 s=READY pgs=727 cs=0 l=1 rev1=1 crypto rx=0x7fde4c00c490 tx=0x7fde44005830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d85625d91e486a1f server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:31:09.536 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:31:09.531+0000 7fde5a7fc640 1 -- 172.21.15.37:0/3863814203 <== mon.0 v2:172.21.15.37:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fde4c00c0e0 con 0x7fde440039e0 2024-04-25T10:31:09.536 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:31:09.531+0000 7fde5a7fc640 1 -- 172.21.15.37:0/3863814203 <== mon.0 v2:172.21.15.37:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fde4c034070 con 0x7fde440039e0 2024-04-25T10:31:09.536 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:31:09.531+0000 7fde5a7fc640 1 -- 172.21.15.37:0/3863814203 <== mon.0 v2:172.21.15.37:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fde4c0074b0 con 0x7fde440039e0 2024-04-25T10:31:09.539 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:31:09.532+0000 7fde6217d640 1 -- 172.21.15.37:0/3863814203 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fde440039e0 msgr2=0x7fde44003e10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:31:09.539 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:31:09.532+0000 7fde6217d640 1 --2- 172.21.15.37:0/3863814203 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fde440039e0 0x7fde44003e10 secure :-1 s=READY pgs=727 cs=0 l=1 rev1=1 crypto rx=0x7fde4c00c490 tx=0x7fde44005830 comp rx=0 tx=0).stop 2024-04-25T10:31:09.539 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:31:09.532+0000 7fde5bfff640 1 -- 172.21.15.37:0/3863814203 reap_dead start 2024-04-25T10:31:09.539 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:31:09.532+0000 7fde6217d640 1 -- 172.21.15.37:0/3863814203 shutdown_connections 2024-04-25T10:31:09.539 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:31:09.532+0000 7fde6217d640 1 -- 172.21.15.37:0/3863814203 >> 172.21.15.37:0/3863814203 conn(0x7fde5c0d03e0 msgr2=0x7fde5c0d07f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:31:09.540 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:31:09.532+0000 7fde6217d640 1 -- 172.21.15.37:0/3863814203 shutdown_connections 2024-04-25T10:31:09.540 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:31:09.532+0000 7fde6217d640 1 -- 172.21.15.37:0/3863814203 wait complete. 2024-04-25T10:31:09.540 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:31:09.534+0000 7fde6217d640 1 Processor -- start 2024-04-25T10:31:09.540 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:31:09.534+0000 7fde6217d640 1 -- start start 2024-04-25T10:31:09.540 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:31:09.535+0000 7fde6217d640 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fde440039e0 0x7fde5c0a0650 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:31:09.540 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:31:09.535+0000 7fde6217d640 1 --2- >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7fde5c1d5690 0x7fde5c1fa870 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:31:09.540 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:31:09.535+0000 7fde6217d640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fde5c0b0610 0x7fde5c09eb50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:31:09.540 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:31:09.535+0000 7fde6217d640 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 -- 0x7fde5c0d0120 con 0x7fde5c0b0610 2024-04-25T10:31:09.540 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:31:09.535+0000 7fde6217d640 1 -- --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_getmap magic: 0 -- 0x7fde5c0b67b0 con 0x7fde440039e0 2024-04-25T10:31:09.540 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:31:09.535+0000 7fde6217d640 1 -- --> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] -- mon_getmap magic: 0 -- 0x7fde5c0d4770 con 0x7fde5c1d5690 2024-04-25T10:31:09.540 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:31:09.535+0000 7fde5bfff640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fde5c0b0610 0x7fde5c09eb50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:31:09.540 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:31:09.535+0000 7fde5bfff640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fde5c0b0610 0x7fde5c09eb50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.37:3300/0 says I am v2:172.21.15.37:34710/0 (socket says 172.21.15.37:34710) 2024-04-25T10:31:09.541 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:31:09.535+0000 7fde5bfff640 1 -- 172.21.15.37:0/913712825 learned_addr learned my addr 172.21.15.37:0/913712825 (peer_addr_for_me v2:172.21.15.37:0/0) 2024-04-25T10:31:09.541 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:31:09.535+0000 7fde5bfff640 1 -- 172.21.15.37:0/913712825 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7fde5c1d5690 msgr2=0x7fde5c1fa870 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:31:09.541 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:31:09.535+0000 7fde5bfff640 1 --2- 172.21.15.37:0/913712825 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7fde5c1d5690 0x7fde5c1fa870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:31:09.541 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:31:09.535+0000 7fde5bfff640 1 -- 172.21.15.37:0/913712825 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fde440039e0 msgr2=0x7fde5c0a0650 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:31:09.541 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:31:09.535+0000 7fde5bfff640 1 --2- 172.21.15.37:0/913712825 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fde440039e0 0x7fde5c0a0650 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:31:09.541 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:31:09.535+0000 7fde5bfff640 1 -- 172.21.15.37:0/913712825 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fde5c09dab0 con 0x7fde5c0b0610 2024-04-25T10:31:09.541 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:31:09.536+0000 7fde5bfff640 1 --2- 172.21.15.37:0/913712825 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fde5c0b0610 0x7fde5c09eb50 secure :-1 s=READY pgs=728 cs=0 l=1 rev1=1 crypto rx=0x7fde5c074410 tx=0x7fde54009f90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:31:09.541 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:31:09.536+0000 7fde43fff640 1 -- 172.21.15.37:0/913712825 <== mon.0 v2:172.21.15.37:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fde54010e80 con 0x7fde5c0b0610 2024-04-25T10:31:09.541 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:31:09.536+0000 7fde43fff640 1 -- 172.21.15.37:0/913712825 <== mon.0 v2:172.21.15.37:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fde54003990 con 0x7fde5c0b0610 2024-04-25T10:31:09.541 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:31:09.536+0000 7fde43fff640 1 -- 172.21.15.37:0/913712825 <== mon.0 v2:172.21.15.37:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fde54003da0 con 0x7fde5c0b0610 2024-04-25T10:31:09.541 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:31:09.536+0000 7fde6217d640 1 -- 172.21.15.37:0/913712825 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fde5c09dd90 con 0x7fde5c0b0610 2024-04-25T10:31:09.541 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:31:09.536+0000 7fde6217d640 1 -- 172.21.15.37:0/913712825 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fde5c09e350 con 0x7fde5c0b0610 2024-04-25T10:31:09.541 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:31:09.537+0000 7fde6217d640 1 -- 172.21.15.37:0/913712825 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_get_version(what=osdmap handle=1) -- 0x7fde5c06a140 con 0x7fde5c0b0610 2024-04-25T10:31:09.543 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:31:09.538+0000 7fde43fff640 1 -- 172.21.15.37:0/913712825 <== mon.0 v2:172.21.15.37:3300/0 4 ==== mgrmap(e 5) ==== 96782+0+0 (secure 0 0 0) 0x7fde54005d10 con 0x7fde5c0b0610 2024-04-25T10:31:09.543 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:31:09.538+0000 7fde43fff640 1 --2- 172.21.15.37:0/913712825 >> [v2:172.21.15.203:6832/2418205237,v1:172.21.15.203:6833/2418205237] conn(0x7fde480744b0 0x7fde480769c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:31:09.543 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:31:09.538+0000 7fde43fff640 1 -- 172.21.15.37:0/913712825 <== mon.0 v2:172.21.15.37:3300/0 5 ==== osd_map(50..50 src has 1..50) ==== 5029+0+0 (secure 0 0 0) 0x7fde54096470 con 0x7fde5c0b0610 2024-04-25T10:31:09.543 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:31:09.538+0000 7fde43fff640 1 --2- 172.21.15.37:0/913712825 >> [v2:172.21.15.37:6824/1815290618,v1:172.21.15.37:6825/1815290618] conn(0x7fde4807a460 0x7fde4807c8b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:31:09.543 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:31:09.538+0000 7fde43fff640 1 -- 172.21.15.37:0/913712825 --> [v2:172.21.15.37:6824/1815290618,v1:172.21.15.37:6825/1815290618] -- command(tid 1: {"prefix": "get_command_descriptions"}) -- 0x7fde5401fe50 con 0x7fde4807a460 2024-04-25T10:31:09.543 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:31:09.538+0000 7fde43fff640 1 -- 172.21.15.37:0/913712825 <== mon.0 v2:172.21.15.37:3300/0 6 ==== mon_get_version_reply(handle=1 version=50) ==== 24+0+0 (secure 0 0 0) 0x7fde54096900 con 0x7fde5c0b0610 2024-04-25T10:31:09.543 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:31:09.538+0000 7fde5affd640 1 --2- 172.21.15.37:0/913712825 >> [v2:172.21.15.37:6824/1815290618,v1:172.21.15.37:6825/1815290618] conn(0x7fde4807a460 0x7fde4807c8b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:31:09.543 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:31:09.538+0000 7fde5b7fe640 1 --2- 172.21.15.37:0/913712825 >> [v2:172.21.15.203:6832/2418205237,v1:172.21.15.203:6833/2418205237] conn(0x7fde480744b0 0x7fde480769c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:31:09.543 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:31:09.539+0000 7fde5affd640 1 --2- 172.21.15.37:0/913712825 >> [v2:172.21.15.37:6824/1815290618,v1:172.21.15.37:6825/1815290618] conn(0x7fde4807a460 0x7fde4807c8b0 crc :-1 s=READY pgs=19 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.3 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:31:09.543 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:31:09.539+0000 7fde5b7fe640 1 --2- 172.21.15.37:0/913712825 >> [v2:172.21.15.203:6832/2418205237,v1:172.21.15.203:6833/2418205237] conn(0x7fde480744b0 0x7fde480769c0 secure :-1 s=READY pgs=327 cs=0 l=1 rev1=1 crypto rx=0x7fde4c033000 tx=0x7fde4c009030 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:31:09.543 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:31:09.539+0000 7fde43fff640 1 -- 172.21.15.37:0/913712825 <== osd.3 v2:172.21.15.37:6824/1815290618 1 ==== command_reply(tid 1: 0 ) ==== 8+0+27193 (crc 0 0 0) 0x7fde5401fe50 con 0x7fde4807a460 2024-04-25T10:31:09.554 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:31:09.552+0000 7fde6217d640 1 -- 172.21.15.37:0/913712825 --> [v2:172.21.15.37:6824/1815290618,v1:172.21.15.37:6825/1815290618] -- command(tid 2: {"prefix": "config set", "var": "osd_debug_deep_scrub_sleep", "val": ["0"]}) -- 0x7fde5c0a0dc0 con 0x7fde4807a460 2024-04-25T10:31:09.555 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:31:09.553+0000 7fde43fff640 1 -- 172.21.15.37:0/913712825 <== osd.3 v2:172.21.15.37:6824/1815290618 2 ==== command_reply(tid 2: 0 ) ==== 8+0+97 (crc 0 0 0) 0x7fde5c0a0dc0 con 0x7fde4807a460 2024-04-25T10:31:09.555 INFO:teuthology.orchestra.run.smithi037.stdout:{ 2024-04-25T10:31:09.555 INFO:teuthology.orchestra.run.smithi037.stdout: "success": "osd_debug_deep_scrub_sleep = '' (not observed, change may require restart) " 2024-04-25T10:31:09.555 INFO:teuthology.orchestra.run.smithi037.stdout:} 2024-04-25T10:31:09.556 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:31:09.553+0000 7fde6217d640 1 -- 172.21.15.37:0/913712825 >> [v2:172.21.15.37:6824/1815290618,v1:172.21.15.37:6825/1815290618] conn(0x7fde4807a460 msgr2=0x7fde4807c8b0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:31:09.556 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:31:09.553+0000 7fde6217d640 1 --2- 172.21.15.37:0/913712825 >> [v2:172.21.15.37:6824/1815290618,v1:172.21.15.37:6825/1815290618] conn(0x7fde4807a460 0x7fde4807c8b0 crc :-1 s=READY pgs=19 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:31:09.556 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:31:09.553+0000 7fde6217d640 1 -- 172.21.15.37:0/913712825 >> [v2:172.21.15.203:6832/2418205237,v1:172.21.15.203:6833/2418205237] conn(0x7fde480744b0 msgr2=0x7fde480769c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:31:09.556 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:31:09.553+0000 7fde6217d640 1 --2- 172.21.15.37:0/913712825 >> [v2:172.21.15.203:6832/2418205237,v1:172.21.15.203:6833/2418205237] conn(0x7fde480744b0 0x7fde480769c0 secure :-1 s=READY pgs=327 cs=0 l=1 rev1=1 crypto rx=0x7fde4c033000 tx=0x7fde4c009030 comp rx=0 tx=0).stop 2024-04-25T10:31:09.556 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:31:09.553+0000 7fde6217d640 1 -- 172.21.15.37:0/913712825 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fde5c0b0610 msgr2=0x7fde5c09eb50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:31:09.556 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:31:09.553+0000 7fde6217d640 1 --2- 172.21.15.37:0/913712825 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fde5c0b0610 0x7fde5c09eb50 secure :-1 s=READY pgs=728 cs=0 l=1 rev1=1 crypto rx=0x7fde5c074410 tx=0x7fde54009f90 comp rx=0 tx=0).stop 2024-04-25T10:31:09.556 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:31:09.553+0000 7fde5bfff640 1 -- 172.21.15.37:0/913712825 reap_dead start 2024-04-25T10:31:09.556 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:31:09.553+0000 7fde6217d640 1 -- 172.21.15.37:0/913712825 shutdown_connections 2024-04-25T10:31:09.557 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:31:09.553+0000 7fde6217d640 1 -- 172.21.15.37:0/913712825 >> 172.21.15.37:0/913712825 conn(0x7fde5c0d03e0 msgr2=0x7fde5c0b6930 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:31:09.557 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:31:09.553+0000 7fde6217d640 1 -- 172.21.15.37:0/913712825 shutdown_connections 2024-04-25T10:31:09.557 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:31:09.553+0000 7fde6217d640 1 -- 172.21.15.37:0/913712825 wait complete. 2024-04-25T10:31:09.566 DEBUG:teuthology.orchestra.run.smithi037:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd deep-scrub 3 2024-04-25T10:31:09.678 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:31:09.676+0000 7fdef435e640 1 Processor -- start 2024-04-25T10:31:09.678 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:31:09.676+0000 7fdef435e640 1 -- start start 2024-04-25T10:31:09.680 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:31:09.678+0000 7fdef435e640 1 --2- >> v2:172.21.15.203:3300/0 conn(0x7fdeec0b4e80 0x7fdeec0d2b20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:31:09.680 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:31:09.678+0000 7fdef435e640 1 --2- >> v2:172.21.15.37:3300/0 conn(0x7fdeec0d3060 0x7fdeec0b9060 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:31:09.681 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:31:09.678+0000 7fdef435e640 1 -- --> v1:172.21.15.37:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fdeec0d3470 con 0x7fdeec0d5160 2024-04-25T10:31:09.681 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:31:09.678+0000 7fdef435e640 1 -- --> v2:172.21.15.37:3300/0 -- mon_getmap magic: 0 -- 0x7fdeec0d1650 con 0x7fdeec0d3060 2024-04-25T10:31:09.681 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:31:09.678+0000 7fdef435e640 1 -- --> v2:172.21.15.203:3300/0 -- mon_getmap magic: 0 -- 0x7fdeec0b5290 con 0x7fdeec0b4e80 2024-04-25T10:31:09.681 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:31:09.678+0000 7fdef28fb640 1 --2- >> v2:172.21.15.37:3300/0 conn(0x7fdeec0d3060 0x7fdeec0b9060 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:31:09.681 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:31:09.678+0000 7fdef28fb640 1 --2- >> v2:172.21.15.37:3300/0 conn(0x7fdeec0d3060 0x7fdeec0b9060 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.37:3300/0 says I am v2:172.21.15.37:59524/0 (socket says 172.21.15.37:59524) 2024-04-25T10:31:09.681 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:31:09.678+0000 7fdef28fb640 1 -- 172.21.15.37:0/3045015326 learned_addr learned my addr 172.21.15.37:0/3045015326 (peer_addr_for_me v2:172.21.15.37:0/0) 2024-04-25T10:31:09.681 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:31:09.678+0000 7fdef20fa640 1 --1- >> v1:172.21.15.37:6789/0 conn(0x7fdeec0d5160 0x7fdeec0d0990 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.37:6789/0 says I am v1:172.21.15.37:56402/0 (socket says 172.21.15.37:56402) 2024-04-25T10:31:09.681 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:31:09.678+0000 7fdef18f9640 1 --2- 172.21.15.37:0/3045015326 >> v2:172.21.15.203:3300/0 conn(0x7fdeec0b4e80 0x7fdeec0d2b20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:31:09.681 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:31:09.678+0000 7fdef10f8640 1 -- 172.21.15.37:0/3045015326 <== mon.0 v1:172.21.15.37:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 2284428346 0 0) 0x7fdee8001110 con 0x7fdeec0d5160 2024-04-25T10:31:09.681 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:31:09.678+0000 7fdef10f8640 1 -- 172.21.15.37:0/3045015326 --> v1:172.21.15.37:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7fded8003520 con 0x7fdeec0d5160 2024-04-25T10:31:09.681 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:31:09.678+0000 7fdef28fb640 1 -- 172.21.15.37:0/3045015326 >> v2:172.21.15.203:3300/0 conn(0x7fdeec0b4e80 msgr2=0x7fdeec0d2b20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:31:09.681 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:31:09.678+0000 7fdef28fb640 1 --2- 172.21.15.37:0/3045015326 >> v2:172.21.15.203:3300/0 conn(0x7fdeec0b4e80 0x7fdeec0d2b20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:31:09.681 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:31:09.678+0000 7fdef28fb640 1 -- 172.21.15.37:0/3045015326 >> v1:172.21.15.37:6789/0 conn(0x7fdeec0d5160 legacy=0x7fdeec0d0990 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:31:09.681 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:31:09.678+0000 7fdef28fb640 1 -- 172.21.15.37:0/3045015326 --> v2:172.21.15.37:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fdeec0d10a0 con 0x7fdeec0d3060 2024-04-25T10:31:09.683 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:31:09.679+0000 7fdef28fb640 1 --2- 172.21.15.37:0/3045015326 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fdeec0d3060 0x7fdeec0b9060 secure :-1 s=READY pgs=730 cs=0 l=1 rev1=1 crypto rx=0x7fdeec0d6b80 tx=0x7fdee402a6a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=30b3e24ca0cd44de server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:31:09.683 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:31:09.679+0000 7fdef10f8640 1 -- 172.21.15.37:0/3045015326 <== mon.0 v2:172.21.15.37:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdee4036070 con 0x7fdeec0d3060 2024-04-25T10:31:09.683 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:31:09.679+0000 7fdef10f8640 1 -- 172.21.15.37:0/3045015326 <== mon.0 v2:172.21.15.37:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fdee4031600 con 0x7fdeec0d3060 2024-04-25T10:31:09.683 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:31:09.679+0000 7fdef10f8640 1 -- 172.21.15.37:0/3045015326 <== mon.0 v2:172.21.15.37:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdee40319b0 con 0x7fdeec0d3060 2024-04-25T10:31:09.683 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:31:09.680+0000 7fdef435e640 1 -- 172.21.15.37:0/3045015326 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fdeec0d3060 msgr2=0x7fdeec0b9060 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:31:09.683 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:31:09.680+0000 7fdef435e640 1 --2- 172.21.15.37:0/3045015326 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fdeec0d3060 0x7fdeec0b9060 secure :-1 s=READY pgs=730 cs=0 l=1 rev1=1 crypto rx=0x7fdeec0d6b80 tx=0x7fdee402a6a0 comp rx=0 tx=0).stop 2024-04-25T10:31:09.683 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:31:09.680+0000 7fdef435e640 1 -- 172.21.15.37:0/3045015326 shutdown_connections 2024-04-25T10:31:09.683 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:31:09.680+0000 7fdef435e640 1 --2- 172.21.15.37:0/3045015326 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fdeec0d3060 0x7fdeec0b9060 unknown :-1 s=CLOSED pgs=730 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:31:09.683 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:31:09.680+0000 7fdef435e640 1 --2- 172.21.15.37:0/3045015326 >> v2:172.21.15.203:3300/0 conn(0x7fdeec0b4e80 0x7fdeec0d2b20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:31:09.683 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:31:09.680+0000 7fdef435e640 1 -- 172.21.15.37:0/3045015326 >> 172.21.15.37:0/3045015326 conn(0x7fdeec0b7a80 msgr2=0x7fdeec0b7e90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:31:09.683 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:31:09.680+0000 7fdef435e640 1 -- 172.21.15.37:0/3045015326 shutdown_connections 2024-04-25T10:31:09.684 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:31:09.680+0000 7fdef435e640 1 -- 172.21.15.37:0/3045015326 wait complete. 2024-04-25T10:31:09.684 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:31:09.682+0000 7fdef435e640 1 Processor -- start 2024-04-25T10:31:09.684 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:31:09.682+0000 7fdef435e640 1 -- start start 2024-04-25T10:31:09.686 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:31:09.683+0000 7fdef435e640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fdeec0b4e80 0x7fdeec1f7160 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:31:09.686 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:31:09.683+0000 7fdef435e640 1 --2- >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7fdeec0d3060 0x7fdeec0d0080 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:31:09.686 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:31:09.683+0000 7fdef435e640 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fdeec0d5160 0x7fdeec0b9740 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:31:09.686 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:31:09.683+0000 7fdef435e640 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 -- 0x7fdeec0d4960 con 0x7fdeec0b4e80 2024-04-25T10:31:09.686 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:31:09.683+0000 7fdef435e640 1 -- --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_getmap magic: 0 -- 0x7fdeec0d4680 con 0x7fdeec0d5160 2024-04-25T10:31:09.686 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:31:09.683+0000 7fdef435e640 1 -- --> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] -- mon_getmap magic: 0 -- 0x7fdeec0d4c40 con 0x7fdeec0d3060 2024-04-25T10:31:09.687 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:31:09.683+0000 7fdef20fa640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fdeec0b4e80 0x7fdeec1f7160 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:31:09.687 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:31:09.683+0000 7fdef20fa640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fdeec0b4e80 0x7fdeec1f7160 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.37:3300/0 says I am v2:172.21.15.37:59536/0 (socket says 172.21.15.37:59536) 2024-04-25T10:31:09.687 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:31:09.683+0000 7fdef20fa640 1 -- 172.21.15.37:0/2580860634 learned_addr learned my addr 172.21.15.37:0/2580860634 (peer_addr_for_me v2:172.21.15.37:0/0) 2024-04-25T10:31:09.687 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:31:09.683+0000 7fdef28fb640 1 --2- 172.21.15.37:0/2580860634 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fdeec0d5160 0x7fdeec0b9740 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:31:09.687 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:31:09.683+0000 7fdef18f9640 1 --2- 172.21.15.37:0/2580860634 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7fdeec0d3060 0x7fdeec0d0080 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:31:09.687 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:31:09.683+0000 7fdef20fa640 1 -- 172.21.15.37:0/2580860634 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7fdeec0d3060 msgr2=0x7fdeec0d0080 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:31:09.687 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:31:09.683+0000 7fdef20fa640 1 --2- 172.21.15.37:0/2580860634 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7fdeec0d3060 0x7fdeec0d0080 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:31:09.687 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:31:09.683+0000 7fdef20fa640 1 -- 172.21.15.37:0/2580860634 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fdeec0d5160 msgr2=0x7fdeec0b9740 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:31:09.687 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:31:09.683+0000 7fdef20fa640 1 --2- 172.21.15.37:0/2580860634 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fdeec0d5160 0x7fdeec0b9740 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:31:09.687 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:31:09.683+0000 7fdef20fa640 1 -- 172.21.15.37:0/2580860634 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fdeec1f76a0 con 0x7fdeec0b4e80 2024-04-25T10:31:09.687 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:31:09.684+0000 7fdef20fa640 1 --2- 172.21.15.37:0/2580860634 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fdeec0b4e80 0x7fdeec1f7160 secure :-1 s=READY pgs=731 cs=0 l=1 rev1=1 crypto rx=0x7fdee800c3f0 tx=0x7fdee800c8b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:31:09.687 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:31:09.684+0000 7fdee2ffd640 1 -- 172.21.15.37:0/2580860634 <== mon.0 v2:172.21.15.37:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdee80074e0 con 0x7fdeec0b4e80 2024-04-25T10:31:09.687 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:31:09.684+0000 7fdee2ffd640 1 -- 172.21.15.37:0/2580860634 <== mon.0 v2:172.21.15.37:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fdee80047e0 con 0x7fdeec0b4e80 2024-04-25T10:31:09.687 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:31:09.684+0000 7fdef435e640 1 -- 172.21.15.37:0/2580860634 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fdeec162b60 con 0x7fdeec0b4e80 2024-04-25T10:31:09.687 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:31:09.684+0000 7fdee2ffd640 1 -- 172.21.15.37:0/2580860634 <== mon.0 v2:172.21.15.37:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdee8004bd0 con 0x7fdeec0b4e80 2024-04-25T10:31:09.688 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:31:09.684+0000 7fdef435e640 1 -- 172.21.15.37:0/2580860634 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fdeec163120 con 0x7fdeec0b4e80 2024-04-25T10:31:09.688 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:31:09.685+0000 7fdef435e640 1 -- 172.21.15.37:0/2580860634 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fdeec06a140 con 0x7fdeec0b4e80 2024-04-25T10:31:09.688 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:31:09.685+0000 7fdee2ffd640 1 -- 172.21.15.37:0/2580860634 <== mon.0 v2:172.21.15.37:3300/0 4 ==== mgrmap(e 5) ==== 96782+0+0 (secure 0 0 0) 0x7fdee801a020 con 0x7fdeec0b4e80 2024-04-25T10:31:09.689 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:31:09.686+0000 7fdee2ffd640 1 --2- 172.21.15.37:0/2580860634 >> [v2:172.21.15.203:6832/2418205237,v1:172.21.15.203:6833/2418205237] conn(0x7fded00744b0 0x7fded00769c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:31:09.689 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:31:09.686+0000 7fdee2ffd640 1 -- 172.21.15.37:0/2580860634 <== mon.0 v2:172.21.15.37:3300/0 5 ==== osd_map(50..50 src has 1..50) ==== 5029+0+0 (secure 0 0 0) 0x7fdee80946c0 con 0x7fdeec0b4e80 2024-04-25T10:31:09.689 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:31:09.686+0000 7fdef18f9640 1 --2- 172.21.15.37:0/2580860634 >> [v2:172.21.15.203:6832/2418205237,v1:172.21.15.203:6833/2418205237] conn(0x7fded00744b0 0x7fded00769c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:31:09.689 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:31:09.687+0000 7fdef18f9640 1 --2- 172.21.15.37:0/2580860634 >> [v2:172.21.15.203:6832/2418205237,v1:172.21.15.203:6833/2418205237] conn(0x7fded00744b0 0x7fded00769c0 secure :-1 s=READY pgs=328 cs=0 l=1 rev1=1 crypto rx=0x7fdeec0b46f0 tx=0x7fdedc008040 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:31:09.693 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:31:09.691+0000 7fdee2ffd640 1 -- 172.21.15.37:0/2580860634 <== mon.0 v2:172.21.15.37:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187246 (secure 0 0 0) 0x7fdeec1f76a0 con 0x7fdeec0b4e80 2024-04-25T10:31:09.878 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:31:09.876+0000 7fdef435e640 1 -- 172.21.15.37:0/2580860634 --> [v2:172.21.15.203:6832/2418205237,v1:172.21.15.203:6833/2418205237] -- mgr_command(tid 0: {"prefix": "osd deep-scrub", "who": "3", "target": ["mon-mgr", ""]}) -- 0x7fdeec162910 con 0x7fded00744b0 2024-04-25T10:31:09.879 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:31:09.877+0000 7fdee2ffd640 1 -- 172.21.15.37:0/2580860634 <== mgr.4100 v2:172.21.15.203:6832/2418205237 1 ==== mgr_command_reply(tid 0: 0 instructed osd(s) 3 to deep-scrub) ==== 41+0+0 (secure 0 0 0) 0x7fdeec162910 con 0x7fded00744b0 2024-04-25T10:31:09.879 INFO:teuthology.orchestra.run.smithi037.stderr:instructed osd(s) 3 to deep-scrub 2024-04-25T10:31:09.881 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:31:09.879+0000 7fdef435e640 1 -- 172.21.15.37:0/2580860634 >> [v2:172.21.15.203:6832/2418205237,v1:172.21.15.203:6833/2418205237] conn(0x7fded00744b0 msgr2=0x7fded00769c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:31:09.881 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:31:09.879+0000 7fdef435e640 1 --2- 172.21.15.37:0/2580860634 >> [v2:172.21.15.203:6832/2418205237,v1:172.21.15.203:6833/2418205237] conn(0x7fded00744b0 0x7fded00769c0 secure :-1 s=READY pgs=328 cs=0 l=1 rev1=1 crypto rx=0x7fdeec0b46f0 tx=0x7fdedc008040 comp rx=0 tx=0).stop 2024-04-25T10:31:09.881 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:31:09.879+0000 7fdef435e640 1 -- 172.21.15.37:0/2580860634 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fdeec0b4e80 msgr2=0x7fdeec1f7160 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:31:09.881 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:31:09.879+0000 7fdef435e640 1 --2- 172.21.15.37:0/2580860634 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fdeec0b4e80 0x7fdeec1f7160 secure :-1 s=READY pgs=731 cs=0 l=1 rev1=1 crypto rx=0x7fdee800c3f0 tx=0x7fdee800c8b0 comp rx=0 tx=0).stop 2024-04-25T10:31:09.881 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:31:09.879+0000 7fdef435e640 1 -- 172.21.15.37:0/2580860634 shutdown_connections 2024-04-25T10:31:09.881 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:31:09.879+0000 7fdef435e640 1 --2- 172.21.15.37:0/2580860634 >> [v2:172.21.15.203:6832/2418205237,v1:172.21.15.203:6833/2418205237] conn(0x7fded00744b0 0x7fded00769c0 unknown :-1 s=CLOSED pgs=328 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:31:09.881 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:31:09.879+0000 7fdef435e640 1 --2- 172.21.15.37:0/2580860634 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fdeec0d5160 0x7fdeec0b9740 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:31:09.881 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:31:09.879+0000 7fdef435e640 1 --2- 172.21.15.37:0/2580860634 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7fdeec0d3060 0x7fdeec0d0080 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:31:09.882 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:31:09.879+0000 7fdef435e640 1 --2- 172.21.15.37:0/2580860634 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fdeec0b4e80 0x7fdeec1f7160 unknown :-1 s=CLOSED pgs=731 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:31:09.882 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:31:09.879+0000 7fdef435e640 1 -- 172.21.15.37:0/2580860634 >> 172.21.15.37:0/2580860634 conn(0x7fdeec0b7a80 msgr2=0x7fdeec0b54d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:31:09.882 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:31:09.879+0000 7fdef435e640 1 -- 172.21.15.37:0/2580860634 shutdown_connections 2024-04-25T10:31:09.882 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:31:09.879+0000 7fdef435e640 1 -- 172.21.15.37:0/2580860634 wait complete. 2024-04-25T10:31:09.892 INFO:tasks.ceph:Scrubbing osd.4 2024-04-25T10:31:09.892 DEBUG:teuthology.orchestra.run.smithi037:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.4 config set osd_debug_deep_scrub_sleep 0 2024-04-25T10:31:10.007 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:31:10.005+0000 7fa254c4c640 1 Processor -- start 2024-04-25T10:31:10.007 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:31:10.005+0000 7fa254c4c640 1 -- start start 2024-04-25T10:31:10.010 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:31:10.007+0000 7fa254c4c640 1 --2- >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7fa2500bb690 0x7fa2500d4e60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:31:10.010 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:31:10.007+0000 7fa254c4c640 1 --2- >> v2:172.21.15.37:3300/0 conn(0x7fa250163c70 0x7fa250164080 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:31:10.010 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:31:10.007+0000 7fa254c4c640 1 -- --> v1:172.21.15.203:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fa2500bbaa0 con 0x7fa2500b75a0 2024-04-25T10:31:10.010 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:31:10.007+0000 7fa254c4c640 1 -- --> v2:172.21.15.37:3300/0 -- mon_getmap magic: 0 -- 0x7fa2500d49d0 con 0x7fa250163c70 2024-04-25T10:31:10.010 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:31:10.007+0000 7fa254c4c640 1 -- --> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] -- mon_getmap magic: 0 -- 0x7fa250164730 con 0x7fa2500bb690 2024-04-25T10:31:10.010 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:31:10.007+0000 7fa24dd9b640 1 --2- >> v2:172.21.15.37:3300/0 conn(0x7fa250163c70 0x7fa250164080 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:31:10.010 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:31:10.007+0000 7fa24dd9b640 1 --2- >> v2:172.21.15.37:3300/0 conn(0x7fa250163c70 0x7fa250164080 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.37:3300/0 says I am v2:172.21.15.37:59546/0 (socket says 172.21.15.37:59546) 2024-04-25T10:31:10.011 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:31:10.007+0000 7fa24dd9b640 1 -- 172.21.15.37:0/4108788396 learned_addr learned my addr 172.21.15.37:0/4108788396 (peer_addr_for_me v2:172.21.15.37:0/0) 2024-04-25T10:31:10.011 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:31:10.007+0000 7fa24e59c640 1 --2- 172.21.15.37:0/4108788396 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7fa2500bb690 0x7fa2500d4e60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:31:10.011 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:31:10.007+0000 7fa24dd9b640 1 -- 172.21.15.37:0/4108788396 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7fa2500bb690 msgr2=0x7fa2500d4e60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:31:10.011 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:31:10.007+0000 7fa24dd9b640 1 --2- 172.21.15.37:0/4108788396 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7fa2500bb690 0x7fa2500d4e60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:31:10.011 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:31:10.007+0000 7fa24dd9b640 1 -- 172.21.15.37:0/4108788396 >> v1:172.21.15.203:6789/0 conn(0x7fa2500b75a0 legacy=0x7fa2500b79b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:31:10.011 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:31:10.007+0000 7fa24dd9b640 1 -- 172.21.15.37:0/4108788396 --> v2:172.21.15.37:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fa250164a10 con 0x7fa250163c70 2024-04-25T10:31:10.011 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:31:10.008+0000 7fa24dd9b640 1 --2- 172.21.15.37:0/4108788396 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fa250163c70 0x7fa250164080 secure :-1 s=READY pgs=732 cs=0 l=1 rev1=1 crypto rx=0x7fa238009870 tx=0x7fa23802a180 comp rx=0 tx=0).ready entity=mon.0 client_cookie=97c252b8ce82fbb0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:31:10.011 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:31:10.008+0000 7fa24d59a640 1 -- 172.21.15.37:0/4108788396 <== mon.0 v2:172.21.15.37:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa238037070 con 0x7fa250163c70 2024-04-25T10:31:10.011 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:31:10.008+0000 7fa24d59a640 1 -- 172.21.15.37:0/4108788396 <== mon.0 v2:172.21.15.37:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fa238032440 con 0x7fa250163c70 2024-04-25T10:31:10.011 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:31:10.008+0000 7fa24d59a640 1 -- 172.21.15.37:0/4108788396 <== mon.0 v2:172.21.15.37:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa238032810 con 0x7fa250163c70 2024-04-25T10:31:10.012 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:31:10.009+0000 7fa254c4c640 1 -- 172.21.15.37:0/4108788396 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fa250163c70 msgr2=0x7fa250164080 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:31:10.012 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:31:10.009+0000 7fa254c4c640 1 --2- 172.21.15.37:0/4108788396 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fa250163c70 0x7fa250164080 secure :-1 s=READY pgs=732 cs=0 l=1 rev1=1 crypto rx=0x7fa238009870 tx=0x7fa23802a180 comp rx=0 tx=0).stop 2024-04-25T10:31:10.012 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:31:10.009+0000 7fa254c4c640 1 -- 172.21.15.37:0/4108788396 shutdown_connections 2024-04-25T10:31:10.012 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:31:10.009+0000 7fa254c4c640 1 --2- 172.21.15.37:0/4108788396 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fa250163c70 0x7fa250164080 unknown :-1 s=CLOSED pgs=732 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:31:10.012 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:31:10.009+0000 7fa254c4c640 1 --2- 172.21.15.37:0/4108788396 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7fa2500bb690 0x7fa2500d4e60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:31:10.012 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:31:10.009+0000 7fa254c4c640 1 -- 172.21.15.37:0/4108788396 >> 172.21.15.37:0/4108788396 conn(0x7fa2500d2780 msgr2=0x7fa2500d2b90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:31:10.012 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:31:10.009+0000 7fa254c4c640 1 -- 172.21.15.37:0/4108788396 shutdown_connections 2024-04-25T10:31:10.012 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:31:10.009+0000 7fa254c4c640 1 -- 172.21.15.37:0/4108788396 wait complete. 2024-04-25T10:31:10.013 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:31:10.011+0000 7fa254c4c640 1 Processor -- start 2024-04-25T10:31:10.013 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:31:10.011+0000 7fa254c4c640 1 -- start start 2024-04-25T10:31:10.016 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:31:10.012+0000 7fa254c4c640 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fa2500b75a0 0x7fa25015c290 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:31:10.016 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:31:10.012+0000 7fa254c4c640 1 --2- >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7fa2500bb690 0x7fa250157f30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:31:10.016 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:31:10.012+0000 7fa254c4c640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fa250163c70 0x7fa2500ceda0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:31:10.016 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:31:10.012+0000 7fa254c4c640 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 -- 0x7fa2500d53a0 con 0x7fa250163c70 2024-04-25T10:31:10.016 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:31:10.012+0000 7fa254c4c640 1 -- --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_getmap magic: 0 -- 0x7fa2500d09f0 con 0x7fa2500b75a0 2024-04-25T10:31:10.016 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:31:10.012+0000 7fa254c4c640 1 -- --> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] -- mon_getmap magic: 0 -- 0x7fa2500d5520 con 0x7fa2500bb690 2024-04-25T10:31:10.016 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:31:10.012+0000 7fa24ed9d640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fa250163c70 0x7fa2500ceda0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:31:10.016 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:31:10.013+0000 7fa24ed9d640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fa250163c70 0x7fa2500ceda0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.37:3300/0 says I am v2:172.21.15.37:59558/0 (socket says 172.21.15.37:59558) 2024-04-25T10:31:10.016 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:31:10.013+0000 7fa24ed9d640 1 -- 172.21.15.37:0/1125233779 learned_addr learned my addr 172.21.15.37:0/1125233779 (peer_addr_for_me v2:172.21.15.37:0/0) 2024-04-25T10:31:10.016 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:31:10.013+0000 7fa24e59c640 1 --2- 172.21.15.37:0/1125233779 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fa2500b75a0 0x7fa25015c290 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:31:10.016 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:31:10.013+0000 7fa24dd9b640 1 --2- 172.21.15.37:0/1125233779 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7fa2500bb690 0x7fa250157f30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:31:10.016 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:31:10.013+0000 7fa24ed9d640 1 -- 172.21.15.37:0/1125233779 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7fa2500bb690 msgr2=0x7fa250157f30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:31:10.017 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:31:10.013+0000 7fa24ed9d640 1 --2- 172.21.15.37:0/1125233779 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7fa2500bb690 0x7fa250157f30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:31:10.017 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:31:10.013+0000 7fa24ed9d640 1 -- 172.21.15.37:0/1125233779 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fa2500b75a0 msgr2=0x7fa25015c290 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:31:10.017 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:31:10.013+0000 7fa24ed9d640 1 --2- 172.21.15.37:0/1125233779 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fa2500b75a0 0x7fa25015c290 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:31:10.017 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:31:10.013+0000 7fa24ed9d640 1 -- 172.21.15.37:0/1125233779 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fa25015a710 con 0x7fa250163c70 2024-04-25T10:31:10.017 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:31:10.013+0000 7fa24ed9d640 1 --2- 172.21.15.37:0/1125233779 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fa250163c70 0x7fa2500ceda0 secure :-1 s=READY pgs=733 cs=0 l=1 rev1=1 crypto rx=0x7fa24800b440 tx=0x7fa24800b900 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:31:10.017 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:31:10.013+0000 7fa246ffd640 1 -- 172.21.15.37:0/1125233779 <== mon.0 v2:172.21.15.37:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa248015020 con 0x7fa250163c70 2024-04-25T10:31:10.017 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:31:10.013+0000 7fa246ffd640 1 -- 172.21.15.37:0/1125233779 <== mon.0 v2:172.21.15.37:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fa248011830 con 0x7fa250163c70 2024-04-25T10:31:10.017 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:31:10.013+0000 7fa254c4c640 1 -- 172.21.15.37:0/1125233779 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fa2500cacc0 con 0x7fa250163c70 2024-04-25T10:31:10.017 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:31:10.013+0000 7fa246ffd640 1 -- 172.21.15.37:0/1125233779 <== mon.0 v2:172.21.15.37:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa248011c20 con 0x7fa250163c70 2024-04-25T10:31:10.017 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:31:10.013+0000 7fa254c4c640 1 -- 172.21.15.37:0/1125233779 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fa25015a9f0 con 0x7fa250163c70 2024-04-25T10:31:10.018 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:31:10.014+0000 7fa254c4c640 1 -- 172.21.15.37:0/1125233779 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_get_version(what=osdmap handle=1) -- 0x7fa250158470 con 0x7fa250163c70 2024-04-25T10:31:10.019 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:31:10.015+0000 7fa246ffd640 1 -- 172.21.15.37:0/1125233779 <== mon.0 v2:172.21.15.37:3300/0 4 ==== mgrmap(e 5) ==== 96782+0+0 (secure 0 0 0) 0x7fa248018020 con 0x7fa250163c70 2024-04-25T10:31:10.019 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:31:10.015+0000 7fa246ffd640 1 --2- 172.21.15.37:0/1125233779 >> [v2:172.21.15.203:6832/2418205237,v1:172.21.15.203:6833/2418205237] conn(0x7fa2300744b0 0x7fa2300769c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:31:10.019 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:31:10.015+0000 7fa246ffd640 1 -- 172.21.15.37:0/1125233779 <== mon.0 v2:172.21.15.37:3300/0 5 ==== osd_map(50..50 src has 1..50) ==== 5029+0+0 (secure 0 0 0) 0x7fa248094a10 con 0x7fa250163c70 2024-04-25T10:31:10.019 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:31:10.016+0000 7fa246ffd640 1 --2- 172.21.15.37:0/1125233779 >> [v2:172.21.15.203:6800/4051532063,v1:172.21.15.203:6801/4051532063] conn(0x7fa23007a460 0x7fa23007c8b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:31:10.019 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:31:10.016+0000 7fa246ffd640 1 -- 172.21.15.37:0/1125233779 --> [v2:172.21.15.203:6800/4051532063,v1:172.21.15.203:6801/4051532063] -- command(tid 1: {"prefix": "get_command_descriptions"}) -- 0x7fa23007cf80 con 0x7fa23007a460 2024-04-25T10:31:10.019 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:31:10.016+0000 7fa246ffd640 1 -- 172.21.15.37:0/1125233779 <== mon.0 v2:172.21.15.37:3300/0 6 ==== mon_get_version_reply(handle=1 version=50) ==== 24+0+0 (secure 0 0 0) 0x7fa2480992c0 con 0x7fa250163c70 2024-04-25T10:31:10.019 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:31:10.016+0000 7fa24e59c640 1 --2- 172.21.15.37:0/1125233779 >> [v2:172.21.15.203:6832/2418205237,v1:172.21.15.203:6833/2418205237] conn(0x7fa2300744b0 0x7fa2300769c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:31:10.019 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:31:10.016+0000 7fa24dd9b640 1 --2- 172.21.15.37:0/1125233779 >> [v2:172.21.15.203:6800/4051532063,v1:172.21.15.203:6801/4051532063] conn(0x7fa23007a460 0x7fa23007c8b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:31:10.019 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:31:10.016+0000 7fa24e59c640 1 --2- 172.21.15.37:0/1125233779 >> [v2:172.21.15.203:6832/2418205237,v1:172.21.15.203:6833/2418205237] conn(0x7fa2300744b0 0x7fa2300769c0 secure :-1 s=READY pgs=329 cs=0 l=1 rev1=1 crypto rx=0x7fa24000f360 tx=0x7fa240007710 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:31:10.019 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:31:10.016+0000 7fa24dd9b640 1 --2- 172.21.15.37:0/1125233779 >> [v2:172.21.15.203:6800/4051532063,v1:172.21.15.203:6801/4051532063] conn(0x7fa23007a460 0x7fa23007c8b0 crc :-1 s=READY pgs=19 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.4 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:31:10.020 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:31:10.017+0000 7fa246ffd640 1 -- 172.21.15.37:0/1125233779 <== osd.4 v2:172.21.15.203:6800/4051532063 1 ==== command_reply(tid 1: 0 ) ==== 8+0+27193 (crc 0 0 0) 0x7fa23007cf80 con 0x7fa23007a460 2024-04-25T10:31:10.032 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:31:10.030+0000 7fa254c4c640 1 -- 172.21.15.37:0/1125233779 --> [v2:172.21.15.203:6800/4051532063,v1:172.21.15.203:6801/4051532063] -- command(tid 2: {"prefix": "config set", "var": "osd_debug_deep_scrub_sleep", "val": ["0"]}) -- 0x7fa25015a4c0 con 0x7fa23007a460 2024-04-25T10:31:10.033 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:31:10.031+0000 7fa246ffd640 1 -- 172.21.15.37:0/1125233779 <== osd.4 v2:172.21.15.203:6800/4051532063 2 ==== command_reply(tid 2: 0 ) ==== 8+0+97 (crc 0 0 0) 0x7fa25015a4c0 con 0x7fa23007a460 2024-04-25T10:31:10.033 INFO:teuthology.orchestra.run.smithi037.stdout:{ 2024-04-25T10:31:10.033 INFO:teuthology.orchestra.run.smithi037.stdout: "success": "osd_debug_deep_scrub_sleep = '' (not observed, change may require restart) " 2024-04-25T10:31:10.033 INFO:teuthology.orchestra.run.smithi037.stdout:} 2024-04-25T10:31:10.034 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:31:10.031+0000 7fa254c4c640 1 -- 172.21.15.37:0/1125233779 >> [v2:172.21.15.203:6800/4051532063,v1:172.21.15.203:6801/4051532063] conn(0x7fa23007a460 msgr2=0x7fa23007c8b0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:31:10.034 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:31:10.031+0000 7fa254c4c640 1 --2- 172.21.15.37:0/1125233779 >> [v2:172.21.15.203:6800/4051532063,v1:172.21.15.203:6801/4051532063] conn(0x7fa23007a460 0x7fa23007c8b0 crc :-1 s=READY pgs=19 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:31:10.035 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:31:10.031+0000 7fa254c4c640 1 -- 172.21.15.37:0/1125233779 >> [v2:172.21.15.203:6832/2418205237,v1:172.21.15.203:6833/2418205237] conn(0x7fa2300744b0 msgr2=0x7fa2300769c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:31:10.035 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:31:10.031+0000 7fa254c4c640 1 --2- 172.21.15.37:0/1125233779 >> [v2:172.21.15.203:6832/2418205237,v1:172.21.15.203:6833/2418205237] conn(0x7fa2300744b0 0x7fa2300769c0 secure :-1 s=READY pgs=329 cs=0 l=1 rev1=1 crypto rx=0x7fa24000f360 tx=0x7fa240007710 comp rx=0 tx=0).stop 2024-04-25T10:31:10.035 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:31:10.031+0000 7fa254c4c640 1 -- 172.21.15.37:0/1125233779 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fa250163c70 msgr2=0x7fa2500ceda0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:31:10.035 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:31:10.031+0000 7fa254c4c640 1 --2- 172.21.15.37:0/1125233779 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fa250163c70 0x7fa2500ceda0 secure :-1 s=READY pgs=733 cs=0 l=1 rev1=1 crypto rx=0x7fa24800b440 tx=0x7fa24800b900 comp rx=0 tx=0).stop 2024-04-25T10:31:10.035 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:31:10.031+0000 7fa24ed9d640 1 -- 172.21.15.37:0/1125233779 reap_dead start 2024-04-25T10:31:10.035 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:31:10.031+0000 7fa254c4c640 1 -- 172.21.15.37:0/1125233779 shutdown_connections 2024-04-25T10:31:10.035 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:31:10.031+0000 7fa254c4c640 1 -- 172.21.15.37:0/1125233779 >> 172.21.15.37:0/1125233779 conn(0x7fa2500d2780 msgr2=0x7fa2500d0c90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:31:10.035 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:31:10.031+0000 7fa254c4c640 1 -- 172.21.15.37:0/1125233779 shutdown_connections 2024-04-25T10:31:10.035 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:31:10.031+0000 7fa254c4c640 1 -- 172.21.15.37:0/1125233779 wait complete. 2024-04-25T10:31:10.044 DEBUG:teuthology.orchestra.run.smithi037:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd deep-scrub 4 2024-04-25T10:31:10.155 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:31:10.153+0000 7f444904a640 1 Processor -- start 2024-04-25T10:31:10.156 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:31:10.153+0000 7f444904a640 1 -- start start 2024-04-25T10:31:10.159 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:31:10.155+0000 7f444904a640 1 --2- >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f44440d0c60 0x7f44440ce8d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:31:10.159 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:31:10.155+0000 7f444904a640 1 --2- >> v2:172.21.15.37:3300/0 conn(0x7f44440cee10 0x7f444406f3a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:31:10.159 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:31:10.155+0000 7f444904a640 1 -- --> v1:172.21.15.203:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f4444144700 con 0x7f44441442f0 2024-04-25T10:31:10.159 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:31:10.155+0000 7f444904a640 1 -- --> v2:172.21.15.37:3300/0 -- mon_getmap magic: 0 -- 0x7f44440cf600 con 0x7f44440cee10 2024-04-25T10:31:10.159 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:31:10.155+0000 7f444904a640 1 -- --> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] -- mon_getmap magic: 0 -- 0x7f44440cf8e0 con 0x7f44440d0c60 2024-04-25T10:31:10.159 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:31:10.155+0000 7f444359e640 1 --2- >> v2:172.21.15.37:3300/0 conn(0x7f44440cee10 0x7f444406f3a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:31:10.159 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:31:10.155+0000 7f444359e640 1 --2- >> v2:172.21.15.37:3300/0 conn(0x7f44440cee10 0x7f444406f3a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.37:3300/0 says I am v2:172.21.15.37:59562/0 (socket says 172.21.15.37:59562) 2024-04-25T10:31:10.159 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:31:10.155+0000 7f444359e640 1 -- 172.21.15.37:0/1810537 learned_addr learned my addr 172.21.15.37:0/1810537 (peer_addr_for_me v2:172.21.15.37:0/0) 2024-04-25T10:31:10.160 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:31:10.155+0000 7f444259c640 1 --2- 172.21.15.37:0/1810537 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f44440d0c60 0x7f44440ce8d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:31:10.160 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:31:10.155+0000 7f444359e640 1 -- 172.21.15.37:0/1810537 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f44440d0c60 msgr2=0x7f44440ce8d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:31:10.160 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:31:10.155+0000 7f444359e640 1 --2- 172.21.15.37:0/1810537 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f44440d0c60 0x7f44440ce8d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:31:10.160 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:31:10.155+0000 7f444359e640 1 -- 172.21.15.37:0/1810537 >> v1:172.21.15.203:6789/0 conn(0x7f44441442f0 legacy=0x7f44440d0550 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:31:10.160 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:31:10.155+0000 7f444359e640 1 -- 172.21.15.37:0/1810537 --> v2:172.21.15.37:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f44440cfbc0 con 0x7f44440cee10 2024-04-25T10:31:10.160 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:31:10.156+0000 7f444359e640 1 --2- 172.21.15.37:0/1810537 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f44440cee10 0x7f444406f3a0 secure :-1 s=READY pgs=734 cs=0 l=1 rev1=1 crypto rx=0x7f443c009870 tx=0x7f443c02a270 comp rx=0 tx=0).ready entity=mon.0 client_cookie=3e4829e46e4cccbc server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:31:10.160 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:31:10.156+0000 7f4441d9b640 1 -- 172.21.15.37:0/1810537 <== mon.0 v2:172.21.15.37:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f443c037070 con 0x7f44440cee10 2024-04-25T10:31:10.160 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:31:10.156+0000 7f4441d9b640 1 -- 172.21.15.37:0/1810537 <== mon.0 v2:172.21.15.37:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f443c032440 con 0x7f44440cee10 2024-04-25T10:31:10.160 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:31:10.156+0000 7f4441d9b640 1 -- 172.21.15.37:0/1810537 <== mon.0 v2:172.21.15.37:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f443c032810 con 0x7f44440cee10 2024-04-25T10:31:10.160 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:31:10.157+0000 7f444904a640 1 -- 172.21.15.37:0/1810537 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f44440cee10 msgr2=0x7f444406f3a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:31:10.160 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:31:10.157+0000 7f444904a640 1 --2- 172.21.15.37:0/1810537 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f44440cee10 0x7f444406f3a0 secure :-1 s=READY pgs=734 cs=0 l=1 rev1=1 crypto rx=0x7f443c009870 tx=0x7f443c02a270 comp rx=0 tx=0).stop 2024-04-25T10:31:10.160 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:31:10.157+0000 7f444904a640 1 -- 172.21.15.37:0/1810537 shutdown_connections 2024-04-25T10:31:10.160 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:31:10.157+0000 7f444904a640 1 --2- 172.21.15.37:0/1810537 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f44440cee10 0x7f444406f3a0 unknown :-1 s=CLOSED pgs=734 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:31:10.160 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:31:10.157+0000 7f444904a640 1 --2- 172.21.15.37:0/1810537 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f44440d0c60 0x7f44440ce8d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:31:10.160 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:31:10.157+0000 7f444904a640 1 -- 172.21.15.37:0/1810537 >> 172.21.15.37:0/1810537 conn(0x7f44440b36b0 msgr2=0x7f44440b3ac0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:31:10.161 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:31:10.157+0000 7f444904a640 1 -- 172.21.15.37:0/1810537 shutdown_connections 2024-04-25T10:31:10.161 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:31:10.157+0000 7f444904a640 1 -- 172.21.15.37:0/1810537 wait complete. 2024-04-25T10:31:10.161 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:31:10.159+0000 7f444904a640 1 Processor -- start 2024-04-25T10:31:10.161 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:31:10.159+0000 7f444904a640 1 -- start start 2024-04-25T10:31:10.164 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:31:10.160+0000 7f444904a640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f44440cee10 0x7f444415ba00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:31:10.164 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:31:10.160+0000 7f444904a640 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f44441442f0 0x7f4444138e30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:31:10.164 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:31:10.160+0000 7f444904a640 1 --2- >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f44441378f0 0x7f4444137d00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:31:10.164 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:31:10.160+0000 7f444904a640 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 -- 0x7f44440cf220 con 0x7f44440cee10 2024-04-25T10:31:10.164 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:31:10.160+0000 7f444904a640 1 -- --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_getmap magic: 0 -- 0x7f44440d0180 con 0x7f44441442f0 2024-04-25T10:31:10.164 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:31:10.160+0000 7f444904a640 1 -- --> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] -- mon_getmap magic: 0 -- 0x7f44440cf3a0 con 0x7f44441378f0 2024-04-25T10:31:10.164 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:31:10.160+0000 7f4442d9d640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f44440cee10 0x7f444415ba00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:31:10.164 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:31:10.160+0000 7f4442d9d640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f44440cee10 0x7f444415ba00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.37:3300/0 says I am v2:172.21.15.37:59578/0 (socket says 172.21.15.37:59578) 2024-04-25T10:31:10.164 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:31:10.160+0000 7f4442d9d640 1 -- 172.21.15.37:0/1845975003 learned_addr learned my addr 172.21.15.37:0/1845975003 (peer_addr_for_me v2:172.21.15.37:0/0) 2024-04-25T10:31:10.164 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:31:10.160+0000 7f444259c640 1 --2- 172.21.15.37:0/1845975003 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f44441442f0 0x7f4444138e30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:31:10.164 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:31:10.161+0000 7f444359e640 1 --2- 172.21.15.37:0/1845975003 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f44441378f0 0x7f4444137d00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:31:10.164 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:31:10.161+0000 7f4442d9d640 1 -- 172.21.15.37:0/1845975003 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f44441378f0 msgr2=0x7f4444137d00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:31:10.164 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:31:10.161+0000 7f4442d9d640 1 --2- 172.21.15.37:0/1845975003 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f44441378f0 0x7f4444137d00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:31:10.164 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:31:10.161+0000 7f4442d9d640 1 -- 172.21.15.37:0/1845975003 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f44441442f0 msgr2=0x7f4444138e30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:31:10.165 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:31:10.161+0000 7f4442d9d640 1 --2- 172.21.15.37:0/1845975003 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f44441442f0 0x7f4444138e30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:31:10.165 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:31:10.161+0000 7f4442d9d640 1 -- 172.21.15.37:0/1845975003 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f4444147560 con 0x7f44440cee10 2024-04-25T10:31:10.165 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:31:10.161+0000 7f4442d9d640 1 --2- 172.21.15.37:0/1845975003 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f44440cee10 0x7f444415ba00 secure :-1 s=READY pgs=735 cs=0 l=1 rev1=1 crypto rx=0x7f443400c6b0 tx=0x7f443400cb70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:31:10.165 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:31:10.161+0000 7f442b7fe640 1 -- 172.21.15.37:0/1845975003 <== mon.0 v2:172.21.15.37:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4434016020 con 0x7f44440cee10 2024-04-25T10:31:10.165 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:31:10.161+0000 7f442b7fe640 1 -- 172.21.15.37:0/1845975003 <== mon.0 v2:172.21.15.37:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f4434006d50 con 0x7f44440cee10 2024-04-25T10:31:10.165 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:31:10.161+0000 7f444904a640 1 -- 172.21.15.37:0/1845975003 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f4444138240 con 0x7f44440cee10 2024-04-25T10:31:10.165 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:31:10.161+0000 7f442b7fe640 1 -- 172.21.15.37:0/1845975003 <== mon.0 v2:172.21.15.37:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4434003bb0 con 0x7f44440cee10 2024-04-25T10:31:10.165 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:31:10.161+0000 7f444904a640 1 -- 172.21.15.37:0/1845975003 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f4444145fe0 con 0x7f44440cee10 2024-04-25T10:31:10.166 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:31:10.162+0000 7f444904a640 1 -- 172.21.15.37:0/1845975003 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f444406a140 con 0x7f44440cee10 2024-04-25T10:31:10.166 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:31:10.163+0000 7f442b7fe640 1 -- 172.21.15.37:0/1845975003 <== mon.0 v2:172.21.15.37:3300/0 4 ==== mgrmap(e 5) ==== 96782+0+0 (secure 0 0 0) 0x7f44340050d0 con 0x7f44440cee10 2024-04-25T10:31:10.166 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:31:10.163+0000 7f442b7fe640 1 --2- 172.21.15.37:0/1845975003 >> [v2:172.21.15.203:6832/2418205237,v1:172.21.15.203:6833/2418205237] conn(0x7f44300744e0 0x7f44300769f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:31:10.166 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:31:10.163+0000 7f442b7fe640 1 -- 172.21.15.37:0/1845975003 <== mon.0 v2:172.21.15.37:3300/0 5 ==== osd_map(50..50 src has 1..50) ==== 5029+0+0 (secure 0 0 0) 0x7f44340949d0 con 0x7f44440cee10 2024-04-25T10:31:10.166 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:31:10.164+0000 7f444259c640 1 --2- 172.21.15.37:0/1845975003 >> [v2:172.21.15.203:6832/2418205237,v1:172.21.15.203:6833/2418205237] conn(0x7f44300744e0 0x7f44300769f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:31:10.166 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:31:10.164+0000 7f444259c640 1 --2- 172.21.15.37:0/1845975003 >> [v2:172.21.15.203:6832/2418205237,v1:172.21.15.203:6833/2418205237] conn(0x7f44300744e0 0x7f44300769f0 secure :-1 s=READY pgs=330 cs=0 l=1 rev1=1 crypto rx=0x7f444413d3a0 tx=0x7f4438009190 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:31:10.170 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:31:10.168+0000 7f442b7fe640 1 -- 172.21.15.37:0/1845975003 <== mon.0 v2:172.21.15.37:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187246 (secure 0 0 0) 0x7f4434020020 con 0x7f44440cee10 2024-04-25T10:31:10.353 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:31:10.351+0000 7f444904a640 1 -- 172.21.15.37:0/1845975003 --> [v2:172.21.15.203:6832/2418205237,v1:172.21.15.203:6833/2418205237] -- mgr_command(tid 0: {"prefix": "osd deep-scrub", "who": "4", "target": ["mon-mgr", ""]}) -- 0x7f44440b4580 con 0x7f44300744e0 2024-04-25T10:31:10.353 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:31:10.351+0000 7f442b7fe640 1 -- 172.21.15.37:0/1845975003 <== mgr.4100 v2:172.21.15.203:6832/2418205237 1 ==== mgr_command_reply(tid 0: 0 instructed osd(s) 4 to deep-scrub) ==== 41+0+0 (secure 0 0 0) 0x7f44440b4580 con 0x7f44300744e0 2024-04-25T10:31:10.353 INFO:teuthology.orchestra.run.smithi037.stderr:instructed osd(s) 4 to deep-scrub 2024-04-25T10:31:10.356 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:31:10.353+0000 7f444904a640 1 -- 172.21.15.37:0/1845975003 >> [v2:172.21.15.203:6832/2418205237,v1:172.21.15.203:6833/2418205237] conn(0x7f44300744e0 msgr2=0x7f44300769f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:31:10.356 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:31:10.353+0000 7f444904a640 1 --2- 172.21.15.37:0/1845975003 >> [v2:172.21.15.203:6832/2418205237,v1:172.21.15.203:6833/2418205237] conn(0x7f44300744e0 0x7f44300769f0 secure :-1 s=READY pgs=330 cs=0 l=1 rev1=1 crypto rx=0x7f444413d3a0 tx=0x7f4438009190 comp rx=0 tx=0).stop 2024-04-25T10:31:10.356 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:31:10.353+0000 7f444904a640 1 -- 172.21.15.37:0/1845975003 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f44440cee10 msgr2=0x7f444415ba00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:31:10.356 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:31:10.353+0000 7f444904a640 1 --2- 172.21.15.37:0/1845975003 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f44440cee10 0x7f444415ba00 secure :-1 s=READY pgs=735 cs=0 l=1 rev1=1 crypto rx=0x7f443400c6b0 tx=0x7f443400cb70 comp rx=0 tx=0).stop 2024-04-25T10:31:10.356 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:31:10.353+0000 7f444904a640 1 -- 172.21.15.37:0/1845975003 shutdown_connections 2024-04-25T10:31:10.356 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:31:10.353+0000 7f444904a640 1 --2- 172.21.15.37:0/1845975003 >> [v2:172.21.15.203:6832/2418205237,v1:172.21.15.203:6833/2418205237] conn(0x7f44300744e0 0x7f44300769f0 unknown :-1 s=CLOSED pgs=330 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:31:10.356 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:31:10.353+0000 7f444904a640 1 --2- 172.21.15.37:0/1845975003 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f44441378f0 0x7f4444137d00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:31:10.356 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:31:10.353+0000 7f444904a640 1 --2- 172.21.15.37:0/1845975003 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f44441442f0 0x7f4444138e30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:31:10.356 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:31:10.353+0000 7f444904a640 1 --2- 172.21.15.37:0/1845975003 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f44440cee10 0x7f444415ba00 unknown :-1 s=CLOSED pgs=735 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:31:10.356 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:31:10.353+0000 7f444904a640 1 -- 172.21.15.37:0/1845975003 >> 172.21.15.37:0/1845975003 conn(0x7f44440b36b0 msgr2=0x7f444413eab0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:31:10.356 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:31:10.353+0000 7f444904a640 1 -- 172.21.15.37:0/1845975003 shutdown_connections 2024-04-25T10:31:10.356 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:31:10.353+0000 7f444904a640 1 -- 172.21.15.37:0/1845975003 wait complete. 2024-04-25T10:31:10.366 INFO:tasks.ceph:Scrubbing osd.5 2024-04-25T10:31:10.366 DEBUG:teuthology.orchestra.run.smithi037:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.5 config set osd_debug_deep_scrub_sleep 0 2024-04-25T10:31:10.483 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:31:10.480+0000 7f7dbbcd8640 1 Processor -- start 2024-04-25T10:31:10.484 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:31:10.480+0000 7f7dbbcd8640 1 -- start start 2024-04-25T10:31:10.497 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:31:10.481+0000 7f7dbbcd8640 1 --2- >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f7db40d2020 0x7f7db40d2430 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:31:10.497 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:31:10.481+0000 7f7dbbcd8640 1 --2- >> v2:172.21.15.37:3300/0 conn(0x7f7db40ce890 0x7f7db40ceca0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:31:10.498 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:31:10.481+0000 7f7dbbcd8640 1 -- --> v1:172.21.15.37:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f7db406e700 con 0x7f7db40b9250 2024-04-25T10:31:10.498 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:31:10.481+0000 7f7dbbcd8640 1 -- --> v2:172.21.15.37:3300/0 -- mon_getmap magic: 0 -- 0x7f7db40d3180 con 0x7f7db40ce890 2024-04-25T10:31:10.498 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:31:10.481+0000 7f7dbbcd8640 1 -- --> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] -- mon_getmap magic: 0 -- 0x7f7db408ce90 con 0x7f7db40d2020 2024-04-25T10:31:10.498 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:31:10.481+0000 7f7db9a74640 1 --1- >> v1:172.21.15.37:6789/0 conn(0x7f7db40b9250 0x7f7db40d2a70 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.37:6789/0 says I am v1:172.21.15.37:56412/0 (socket says 172.21.15.37:56412) 2024-04-25T10:31:10.498 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:31:10.481+0000 7f7db9a74640 1 -- 172.21.15.37:0/1635332477 learned_addr learned my addr 172.21.15.37:0/1635332477 (peer_addr_for_me v1:172.21.15.37:0/0) 2024-04-25T10:31:10.498 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:31:10.481+0000 7f7dba275640 1 --2- 172.21.15.37:0/1635332477 >> v2:172.21.15.37:3300/0 conn(0x7f7db40ce890 0x7f7db40ceca0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:31:10.499 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:31:10.482+0000 7f7db9273640 1 --2- 172.21.15.37:0/1635332477 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f7db40d2020 0x7f7db40d2430 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:31:10.499 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:31:10.482+0000 7f7dba275640 1 -- 172.21.15.37:0/1635332477 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f7db40d2020 msgr2=0x7f7db40d2430 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:31:10.499 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:31:10.482+0000 7f7dba275640 1 --2- 172.21.15.37:0/1635332477 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f7db40d2020 0x7f7db40d2430 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:31:10.499 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:31:10.482+0000 7f7dba275640 1 -- 172.21.15.37:0/1635332477 >> v1:172.21.15.37:6789/0 conn(0x7f7db40b9250 legacy=0x7f7db40d2a70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:31:10.499 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:31:10.482+0000 7f7dba275640 1 -- 172.21.15.37:0/1635332477 --> v2:172.21.15.37:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f7db409ce70 con 0x7f7db40ce890 2024-04-25T10:31:10.499 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:31:10.482+0000 7f7db8a72640 1 -- 172.21.15.37:0/1635332477 <== mon.0 v1:172.21.15.37:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 2660567019 0 0) 0x7f7db406e700 con 0x7f7db40b9250 2024-04-25T10:31:10.499 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:31:10.483+0000 7f7dba275640 1 --2- 172.21.15.37:0/1635332477 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f7db40ce890 0x7f7db40ceca0 secure :-1 s=READY pgs=737 cs=0 l=1 rev1=1 crypto rx=0x7f7dac009d90 tx=0x7f7dac02a5d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=bf0b1f4bea3dc11f server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:31:10.499 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:31:10.483+0000 7f7db8a72640 1 -- 172.21.15.37:0/1635332477 <== mon.0 v2:172.21.15.37:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7dac031600 con 0x7f7db40ce890 2024-04-25T10:31:10.499 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:31:10.483+0000 7f7db8a72640 1 -- 172.21.15.37:0/1635332477 <== mon.0 v2:172.21.15.37:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f7dac0317a0 con 0x7f7db40ce890 2024-04-25T10:31:10.500 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:31:10.483+0000 7f7db8a72640 1 -- 172.21.15.37:0/1635332477 <== mon.0 v2:172.21.15.37:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7dac031b70 con 0x7f7db40ce890 2024-04-25T10:31:10.500 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:31:10.483+0000 7f7dbbcd8640 1 -- 172.21.15.37:0/1635332477 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f7db40ce890 msgr2=0x7f7db40ceca0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:31:10.500 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:31:10.483+0000 7f7dbbcd8640 1 --2- 172.21.15.37:0/1635332477 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f7db40ce890 0x7f7db40ceca0 secure :-1 s=READY pgs=737 cs=0 l=1 rev1=1 crypto rx=0x7f7dac009d90 tx=0x7f7dac02a5d0 comp rx=0 tx=0).stop 2024-04-25T10:31:10.500 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:31:10.483+0000 7f7dbbcd8640 1 -- 172.21.15.37:0/1635332477 shutdown_connections 2024-04-25T10:31:10.500 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:31:10.483+0000 7f7dbbcd8640 1 --2- 172.21.15.37:0/1635332477 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f7db40ce890 0x7f7db40ceca0 unknown :-1 s=CLOSED pgs=737 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:31:10.500 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:31:10.483+0000 7f7dbbcd8640 1 --2- 172.21.15.37:0/1635332477 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f7db40d2020 0x7f7db40d2430 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:31:10.500 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:31:10.483+0000 7f7dbbcd8640 1 -- 172.21.15.37:0/1635332477 >> 172.21.15.37:0/1635332477 conn(0x7f7db40d0450 msgr2=0x7f7db40d0860 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:31:10.500 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:31:10.484+0000 7f7dbbcd8640 1 -- 172.21.15.37:0/1635332477 shutdown_connections 2024-04-25T10:31:10.500 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:31:10.484+0000 7f7dbbcd8640 1 -- 172.21.15.37:0/1635332477 wait complete. 2024-04-25T10:31:10.500 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:31:10.485+0000 7f7dbbcd8640 1 Processor -- start 2024-04-25T10:31:10.500 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:31:10.485+0000 7f7dbbcd8640 1 -- start start 2024-04-25T10:31:10.500 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:31:10.487+0000 7f7dbbcd8640 1 --2- >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f7db40b9250 0x7f7db41d74b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:31:10.501 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:31:10.487+0000 7f7dbbcd8640 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f7db40ce890 0x7f7db41f9ed0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:31:10.501 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:31:10.487+0000 7f7dbbcd8640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f7db40d2020 0x7f7db41fa410 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:31:10.501 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:31:10.487+0000 7f7dbbcd8640 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 -- 0x7f7db40b2c70 con 0x7f7db40d2020 2024-04-25T10:31:10.501 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:31:10.487+0000 7f7dbbcd8640 1 -- --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_getmap magic: 0 -- 0x7f7db409d150 con 0x7f7db40ce890 2024-04-25T10:31:10.501 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:31:10.487+0000 7f7dbbcd8640 1 -- --> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] -- mon_getmap magic: 0 -- 0x7f7db40b2df0 con 0x7f7db40b9250 2024-04-25T10:31:10.501 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:31:10.487+0000 7f7dba275640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f7db40d2020 0x7f7db41fa410 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:31:10.501 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:31:10.487+0000 7f7dba275640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f7db40d2020 0x7f7db41fa410 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.37:3300/0 says I am v2:172.21.15.37:59594/0 (socket says 172.21.15.37:59594) 2024-04-25T10:31:10.502 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:31:10.487+0000 7f7dba275640 1 -- 172.21.15.37:0/3534661094 learned_addr learned my addr 172.21.15.37:0/3534661094 (peer_addr_for_me v2:172.21.15.37:0/0) 2024-04-25T10:31:10.502 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:31:10.487+0000 7f7db9273640 1 --2- 172.21.15.37:0/3534661094 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f7db40ce890 0x7f7db41f9ed0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:31:10.502 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:31:10.487+0000 7f7db9a74640 1 --2- 172.21.15.37:0/3534661094 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f7db40b9250 0x7f7db41d74b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:31:10.502 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:31:10.487+0000 7f7dba275640 1 -- 172.21.15.37:0/3534661094 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f7db40b9250 msgr2=0x7f7db41d74b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:31:10.502 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:31:10.487+0000 7f7dba275640 1 --2- 172.21.15.37:0/3534661094 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f7db40b9250 0x7f7db41d74b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:31:10.502 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:31:10.487+0000 7f7dba275640 1 -- 172.21.15.37:0/3534661094 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f7db40ce890 msgr2=0x7f7db41f9ed0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:31:10.502 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:31:10.487+0000 7f7dba275640 1 --2- 172.21.15.37:0/3534661094 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f7db40ce890 0x7f7db41f9ed0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:31:10.502 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:31:10.487+0000 7f7dba275640 1 -- 172.21.15.37:0/3534661094 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f7db41d9cb0 con 0x7f7db40d2020 2024-04-25T10:31:10.502 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:31:10.487+0000 7f7dba275640 1 --2- 172.21.15.37:0/3534661094 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f7db40d2020 0x7f7db41fa410 secure :-1 s=READY pgs=738 cs=0 l=1 rev1=1 crypto rx=0x7f7dac009d90 tx=0x7f7dac004a40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:31:10.503 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:31:10.487+0000 7f7daa7fc640 1 -- 172.21.15.37:0/3534661094 <== mon.0 v2:172.21.15.37:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7dac03f070 con 0x7f7db40d2020 2024-04-25T10:31:10.503 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:31:10.488+0000 7f7daa7fc640 1 -- 172.21.15.37:0/3534661094 <== mon.0 v2:172.21.15.37:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f7dac030b60 con 0x7f7db40d2020 2024-04-25T10:31:10.503 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:31:10.488+0000 7f7dbbcd8640 1 -- 172.21.15.37:0/3534661094 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f7db41fac70 con 0x7f7db40d2020 2024-04-25T10:31:10.503 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:31:10.488+0000 7f7daa7fc640 1 -- 172.21.15.37:0/3534661094 <== mon.0 v2:172.21.15.37:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7dac009450 con 0x7f7db40d2020 2024-04-25T10:31:10.503 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:31:10.488+0000 7f7dbbcd8640 1 -- 172.21.15.37:0/3534661094 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f7db41da250 con 0x7f7db40d2020 2024-04-25T10:31:10.503 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:31:10.489+0000 7f7dbbcd8640 1 -- 172.21.15.37:0/3534661094 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_get_version(what=osdmap handle=1) -- 0x7f7db4050740 con 0x7f7db40d2020 2024-04-25T10:31:10.503 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:31:10.489+0000 7f7daa7fc640 1 -- 172.21.15.37:0/3534661094 <== mon.0 v2:172.21.15.37:3300/0 4 ==== mgrmap(e 5) ==== 96782+0+0 (secure 0 0 0) 0x7f7dac03b410 con 0x7f7db40d2020 2024-04-25T10:31:10.503 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:31:10.490+0000 7f7daa7fc640 1 --2- 172.21.15.37:0/3534661094 >> [v2:172.21.15.203:6832/2418205237,v1:172.21.15.203:6833/2418205237] conn(0x7f7d980744b0 0x7f7d980769c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:31:10.503 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:31:10.490+0000 7f7daa7fc640 1 -- 172.21.15.37:0/3534661094 <== mon.0 v2:172.21.15.37:3300/0 5 ==== osd_map(50..50 src has 1..50) ==== 5029+0+0 (secure 0 0 0) 0x7f7dac0b60f0 con 0x7f7db40d2020 2024-04-25T10:31:10.503 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:31:10.490+0000 7f7daa7fc640 1 --2- 172.21.15.37:0/3534661094 >> [v2:172.21.15.203:6824/1203822890,v1:172.21.15.203:6825/1203822890] conn(0x7f7d9807a460 0x7f7d9807c8b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:31:10.503 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:31:10.490+0000 7f7daa7fc640 1 -- 172.21.15.37:0/3534661094 --> [v2:172.21.15.203:6824/1203822890,v1:172.21.15.203:6825/1203822890] -- command(tid 1: {"prefix": "get_command_descriptions"}) -- 0x7f7dac00aea0 con 0x7f7d9807a460 2024-04-25T10:31:10.503 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:31:10.490+0000 7f7db9a74640 1 --2- 172.21.15.37:0/3534661094 >> [v2:172.21.15.203:6832/2418205237,v1:172.21.15.203:6833/2418205237] conn(0x7f7d980744b0 0x7f7d980769c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:31:10.503 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:31:10.490+0000 7f7daa7fc640 1 -- 172.21.15.37:0/3534661094 <== mon.0 v2:172.21.15.37:3300/0 6 ==== mon_get_version_reply(handle=1 version=50) ==== 24+0+0 (secure 0 0 0) 0x7f7dac0bd050 con 0x7f7db40d2020 2024-04-25T10:31:10.504 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:31:10.490+0000 7f7db9273640 1 --2- 172.21.15.37:0/3534661094 >> [v2:172.21.15.203:6824/1203822890,v1:172.21.15.203:6825/1203822890] conn(0x7f7d9807a460 0x7f7d9807c8b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:31:10.504 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:31:10.491+0000 7f7db9a74640 1 --2- 172.21.15.37:0/3534661094 >> [v2:172.21.15.203:6832/2418205237,v1:172.21.15.203:6833/2418205237] conn(0x7f7d980744b0 0x7f7d980769c0 secure :-1 s=READY pgs=331 cs=0 l=1 rev1=1 crypto rx=0x7f7db0007fd0 tx=0x7f7db0007de0 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:31:10.504 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:31:10.491+0000 7f7db9273640 1 --2- 172.21.15.37:0/3534661094 >> [v2:172.21.15.203:6824/1203822890,v1:172.21.15.203:6825/1203822890] conn(0x7f7d9807a460 0x7f7d9807c8b0 crc :-1 s=READY pgs=19 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.5 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:31:10.504 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:31:10.492+0000 7f7daa7fc640 1 -- 172.21.15.37:0/3534661094 <== osd.5 v2:172.21.15.203:6824/1203822890 1 ==== command_reply(tid 1: 0 ) ==== 8+0+27193 (crc 0 0 0) 0x7f7dac00aea0 con 0x7f7d9807a460 2024-04-25T10:31:10.507 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:31:10.505+0000 7f7dbbcd8640 1 -- 172.21.15.37:0/3534661094 --> [v2:172.21.15.203:6824/1203822890,v1:172.21.15.203:6825/1203822890] -- command(tid 2: {"prefix": "config set", "var": "osd_debug_deep_scrub_sleep", "val": ["0"]}) -- 0x7f7db41faa00 con 0x7f7d9807a460 2024-04-25T10:31:10.508 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:31:10.505+0000 7f7daa7fc640 1 -- 172.21.15.37:0/3534661094 <== osd.5 v2:172.21.15.203:6824/1203822890 2 ==== command_reply(tid 2: 0 ) ==== 8+0+97 (crc 0 0 0) 0x7f7db41faa00 con 0x7f7d9807a460 2024-04-25T10:31:10.509 INFO:teuthology.orchestra.run.smithi037.stdout:{ 2024-04-25T10:31:10.509 INFO:teuthology.orchestra.run.smithi037.stdout: "success": "osd_debug_deep_scrub_sleep = '' (not observed, change may require restart) " 2024-04-25T10:31:10.509 INFO:teuthology.orchestra.run.smithi037.stdout:} 2024-04-25T10:31:10.510 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:31:10.506+0000 7f7dbbcd8640 1 -- 172.21.15.37:0/3534661094 >> [v2:172.21.15.203:6824/1203822890,v1:172.21.15.203:6825/1203822890] conn(0x7f7d9807a460 msgr2=0x7f7d9807c8b0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:31:10.511 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:31:10.506+0000 7f7dbbcd8640 1 --2- 172.21.15.37:0/3534661094 >> [v2:172.21.15.203:6824/1203822890,v1:172.21.15.203:6825/1203822890] conn(0x7f7d9807a460 0x7f7d9807c8b0 crc :-1 s=READY pgs=19 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:31:10.511 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:31:10.506+0000 7f7dbbcd8640 1 -- 172.21.15.37:0/3534661094 >> [v2:172.21.15.203:6832/2418205237,v1:172.21.15.203:6833/2418205237] conn(0x7f7d980744b0 msgr2=0x7f7d980769c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:31:10.511 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:31:10.506+0000 7f7dbbcd8640 1 --2- 172.21.15.37:0/3534661094 >> [v2:172.21.15.203:6832/2418205237,v1:172.21.15.203:6833/2418205237] conn(0x7f7d980744b0 0x7f7d980769c0 secure :-1 s=READY pgs=331 cs=0 l=1 rev1=1 crypto rx=0x7f7db0007fd0 tx=0x7f7db0007de0 comp rx=0 tx=0).stop 2024-04-25T10:31:10.511 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:31:10.506+0000 7f7dbbcd8640 1 -- 172.21.15.37:0/3534661094 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f7db40d2020 msgr2=0x7f7db41fa410 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:31:10.511 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:31:10.506+0000 7f7dbbcd8640 1 --2- 172.21.15.37:0/3534661094 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f7db40d2020 0x7f7db41fa410 secure :-1 s=READY pgs=738 cs=0 l=1 rev1=1 crypto rx=0x7f7dac009d90 tx=0x7f7dac004a40 comp rx=0 tx=0).stop 2024-04-25T10:31:10.511 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:31:10.506+0000 7f7dba275640 1 -- 172.21.15.37:0/3534661094 reap_dead start 2024-04-25T10:31:10.511 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:31:10.506+0000 7f7dbbcd8640 1 -- 172.21.15.37:0/3534661094 shutdown_connections 2024-04-25T10:31:10.511 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:31:10.506+0000 7f7dbbcd8640 1 -- 172.21.15.37:0/3534661094 >> 172.21.15.37:0/3534661094 conn(0x7f7db40d0450 msgr2=0x7f7db41e60c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:31:10.511 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:31:10.506+0000 7f7dbbcd8640 1 -- 172.21.15.37:0/3534661094 shutdown_connections 2024-04-25T10:31:10.511 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:31:10.506+0000 7f7dbbcd8640 1 -- 172.21.15.37:0/3534661094 wait complete. 2024-04-25T10:31:10.519 DEBUG:teuthology.orchestra.run.smithi037:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd deep-scrub 5 2024-04-25T10:31:10.635 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:31:10.631+0000 7f27f7634640 1 Processor -- start 2024-04-25T10:31:10.635 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:31:10.631+0000 7f27f7634640 1 -- start start 2024-04-25T10:31:10.635 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:31:10.632+0000 7f27f7634640 1 --2- >> v2:172.21.15.37:3300/0 conn(0x7f27f015aa20 0x7f27f0161a80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:31:10.635 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:31:10.632+0000 7f27f7634640 1 --2- >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f27f00b7750 0x7f27f00b7b60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:31:10.635 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:31:10.632+0000 7f27f7634640 1 -- --> v1:172.21.15.203:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f27f00b80a0 con 0x7f27f00d0d10 2024-04-25T10:31:10.635 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:31:10.632+0000 7f27f7634640 1 -- --> v2:172.21.15.37:3300/0 -- mon_getmap magic: 0 -- 0x7f27f00d4b30 con 0x7f27f015aa20 2024-04-25T10:31:10.635 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:31:10.632+0000 7f27f7634640 1 -- --> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] -- mon_getmap magic: 0 -- 0x7f27f00d4240 con 0x7f27f00b7750 2024-04-25T10:31:10.635 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:31:10.632+0000 7f27f53d0640 1 --2- >> v2:172.21.15.37:3300/0 conn(0x7f27f015aa20 0x7f27f0161a80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:31:10.635 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:31:10.632+0000 7f27f53d0640 1 --2- >> v2:172.21.15.37:3300/0 conn(0x7f27f015aa20 0x7f27f0161a80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.37:3300/0 says I am v2:172.21.15.37:59608/0 (socket says 172.21.15.37:59608) 2024-04-25T10:31:10.636 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:31:10.632+0000 7f27f53d0640 1 -- 172.21.15.37:0/3209895551 learned_addr learned my addr 172.21.15.37:0/3209895551 (peer_addr_for_me v2:172.21.15.37:0/0) 2024-04-25T10:31:10.636 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:31:10.633+0000 7f27f4bcf640 1 --2- 172.21.15.37:0/3209895551 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f27f00b7750 0x7f27f00b7b60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:31:10.636 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:31:10.633+0000 7f27f53d0640 1 -- 172.21.15.37:0/3209895551 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f27f00b7750 msgr2=0x7f27f00b7b60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:31:10.636 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:31:10.633+0000 7f27f53d0640 1 --2- 172.21.15.37:0/3209895551 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f27f00b7750 0x7f27f00b7b60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:31:10.636 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:31:10.633+0000 7f27f53d0640 1 -- 172.21.15.37:0/3209895551 >> v1:172.21.15.203:6789/0 conn(0x7f27f00d0d10 legacy=0x7f27f00d1120 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:31:10.636 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:31:10.633+0000 7f27f53d0640 1 -- 172.21.15.37:0/3209895551 --> v2:172.21.15.37:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f27f00d1a00 con 0x7f27f015aa20 2024-04-25T10:31:10.638 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:31:10.633+0000 7f27f53d0640 1 --2- 172.21.15.37:0/3209895551 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f27f015aa20 0x7f27f0161a80 secure :-1 s=READY pgs=739 cs=0 l=1 rev1=1 crypto rx=0x7f27e8009870 tx=0x7f27e802a210 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a37a66bc63f69b38 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:31:10.638 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:31:10.634+0000 7f27effff640 1 -- 172.21.15.37:0/3209895551 <== mon.0 v2:172.21.15.37:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f27e8037070 con 0x7f27f015aa20 2024-04-25T10:31:10.638 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:31:10.634+0000 7f27effff640 1 -- 172.21.15.37:0/3209895551 <== mon.0 v2:172.21.15.37:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f27e8032440 con 0x7f27f015aa20 2024-04-25T10:31:10.639 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:31:10.634+0000 7f27effff640 1 -- 172.21.15.37:0/3209895551 <== mon.0 v2:172.21.15.37:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f27e8032810 con 0x7f27f015aa20 2024-04-25T10:31:10.639 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:31:10.634+0000 7f27f7634640 1 -- 172.21.15.37:0/3209895551 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f27f015aa20 msgr2=0x7f27f0161a80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:31:10.639 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:31:10.634+0000 7f27f7634640 1 --2- 172.21.15.37:0/3209895551 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f27f015aa20 0x7f27f0161a80 secure :-1 s=READY pgs=739 cs=0 l=1 rev1=1 crypto rx=0x7f27e8009870 tx=0x7f27e802a210 comp rx=0 tx=0).stop 2024-04-25T10:31:10.639 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:31:10.634+0000 7f27f7634640 1 -- 172.21.15.37:0/3209895551 shutdown_connections 2024-04-25T10:31:10.639 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:31:10.634+0000 7f27f7634640 1 --2- 172.21.15.37:0/3209895551 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f27f00b7750 0x7f27f00b7b60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:31:10.639 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:31:10.634+0000 7f27f7634640 1 --2- 172.21.15.37:0/3209895551 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f27f015aa20 0x7f27f0161a80 unknown :-1 s=CLOSED pgs=739 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:31:10.639 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:31:10.634+0000 7f27f7634640 1 -- 172.21.15.37:0/3209895551 >> 172.21.15.37:0/3209895551 conn(0x7f27f00d2750 msgr2=0x7f27f00d2b60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:31:10.639 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:31:10.634+0000 7f27f7634640 1 -- 172.21.15.37:0/3209895551 shutdown_connections 2024-04-25T10:31:10.640 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:31:10.634+0000 7f27f7634640 1 -- 172.21.15.37:0/3209895551 wait complete. 2024-04-25T10:31:10.640 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:31:10.636+0000 7f27f7634640 1 Processor -- start 2024-04-25T10:31:10.640 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:31:10.636+0000 7f27f7634640 1 -- start start 2024-04-25T10:31:10.642 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:31:10.637+0000 7f27f7634640 1 --2- >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f27f00b7750 0x7f27f0153a90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:31:10.642 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:31:10.637+0000 7f27f7634640 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f27f00d0d10 0x7f27f00c97e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:31:10.643 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:31:10.637+0000 7f27f7634640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f27f015aa20 0x7f27f00cd7d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:31:10.643 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:31:10.637+0000 7f27f7634640 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 -- 0x7f27f0161fc0 con 0x7f27f015aa20 2024-04-25T10:31:10.643 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:31:10.637+0000 7f27f7634640 1 -- --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_getmap magic: 0 -- 0x7f27f00d47c0 con 0x7f27f00d0d10 2024-04-25T10:31:10.643 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:31:10.637+0000 7f27f7634640 1 -- --> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] -- mon_getmap magic: 0 -- 0x7f27f00d4fc0 con 0x7f27f00b7750 2024-04-25T10:31:10.643 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:31:10.638+0000 7f27f5bd1640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f27f015aa20 0x7f27f00cd7d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:31:10.643 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:31:10.638+0000 7f27f5bd1640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f27f015aa20 0x7f27f00cd7d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.37:3300/0 says I am v2:172.21.15.37:59622/0 (socket says 172.21.15.37:59622) 2024-04-25T10:31:10.643 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:31:10.638+0000 7f27f5bd1640 1 -- 172.21.15.37:0/3108019166 learned_addr learned my addr 172.21.15.37:0/3108019166 (peer_addr_for_me v2:172.21.15.37:0/0) 2024-04-25T10:31:10.643 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:31:10.638+0000 7f27f53d0640 1 --2- 172.21.15.37:0/3108019166 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f27f00b7750 0x7f27f0153a90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:31:10.643 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:31:10.638+0000 7f27f4bcf640 1 --2- 172.21.15.37:0/3108019166 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f27f00d0d10 0x7f27f00c97e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:31:10.643 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:31:10.638+0000 7f27f5bd1640 1 -- 172.21.15.37:0/3108019166 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f27f00b7750 msgr2=0x7f27f0153a90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:31:10.643 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:31:10.638+0000 7f27f5bd1640 1 --2- 172.21.15.37:0/3108019166 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f27f00b7750 0x7f27f0153a90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:31:10.643 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:31:10.638+0000 7f27f5bd1640 1 -- 172.21.15.37:0/3108019166 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f27f00d0d10 msgr2=0x7f27f00c97e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:31:10.643 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:31:10.638+0000 7f27f5bd1640 1 --2- 172.21.15.37:0/3108019166 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f27f00d0d10 0x7f27f00c97e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:31:10.644 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:31:10.638+0000 7f27f5bd1640 1 -- 172.21.15.37:0/3108019166 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f27f00cc1b0 con 0x7f27f015aa20 2024-04-25T10:31:10.644 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:31:10.638+0000 7f27f5bd1640 1 --2- 172.21.15.37:0/3108019166 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f27f015aa20 0x7f27f00cd7d0 secure :-1 s=READY pgs=740 cs=0 l=1 rev1=1 crypto rx=0x7f27e000b9e0 tx=0x7f27e000bea0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:31:10.644 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:31:10.638+0000 7f27edffb640 1 -- 172.21.15.37:0/3108019166 <== mon.0 v2:172.21.15.37:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f27e0003b70 con 0x7f27f015aa20 2024-04-25T10:31:10.644 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:31:10.638+0000 7f27edffb640 1 -- 172.21.15.37:0/3108019166 <== mon.0 v2:172.21.15.37:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f27e0003d10 con 0x7f27f015aa20 2024-04-25T10:31:10.644 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:31:10.638+0000 7f27f7634640 1 -- 172.21.15.37:0/3108019166 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f27f00cc490 con 0x7f27f015aa20 2024-04-25T10:31:10.644 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:31:10.638+0000 7f27edffb640 1 -- 172.21.15.37:0/3108019166 <== mon.0 v2:172.21.15.37:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f27e000cb60 con 0x7f27f015aa20 2024-04-25T10:31:10.644 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:31:10.639+0000 7f27f7634640 1 -- 172.21.15.37:0/3108019166 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f27f0153fd0 con 0x7f27f015aa20 2024-04-25T10:31:10.645 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:31:10.640+0000 7f27f7634640 1 -- 172.21.15.37:0/3108019166 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f27f00b8c60 con 0x7f27f015aa20 2024-04-25T10:31:10.645 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:31:10.640+0000 7f27edffb640 1 -- 172.21.15.37:0/3108019166 <== mon.0 v2:172.21.15.37:3300/0 4 ==== mgrmap(e 5) ==== 96782+0+0 (secure 0 0 0) 0x7f27e000cd00 con 0x7f27f015aa20 2024-04-25T10:31:10.645 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:31:10.641+0000 7f27edffb640 1 --2- 172.21.15.37:0/3108019166 >> [v2:172.21.15.203:6832/2418205237,v1:172.21.15.203:6833/2418205237] conn(0x7f27dc0744b0 0x7f27dc0769c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:31:10.645 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:31:10.641+0000 7f27edffb640 1 -- 172.21.15.37:0/3108019166 <== mon.0 v2:172.21.15.37:3300/0 5 ==== osd_map(50..50 src has 1..50) ==== 5029+0+0 (secure 0 0 0) 0x7f27e0094ca0 con 0x7f27f015aa20 2024-04-25T10:31:10.646 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:31:10.641+0000 7f27f53d0640 1 --2- 172.21.15.37:0/3108019166 >> [v2:172.21.15.203:6832/2418205237,v1:172.21.15.203:6833/2418205237] conn(0x7f27dc0744b0 0x7f27dc0769c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:31:10.646 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:31:10.641+0000 7f27f53d0640 1 --2- 172.21.15.37:0/3108019166 >> [v2:172.21.15.203:6832/2418205237,v1:172.21.15.203:6833/2418205237] conn(0x7f27dc0744b0 0x7f27dc0769c0 secure :-1 s=READY pgs=332 cs=0 l=1 rev1=1 crypto rx=0x7f27e8034000 tx=0x7f27e802e000 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:31:10.647 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:31:10.646+0000 7f27edffb640 1 -- 172.21.15.37:0/3108019166 <== mon.0 v2:172.21.15.37:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187246 (secure 0 0 0) 0x7f27f00b8c60 con 0x7f27f015aa20 2024-04-25T10:31:10.834 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:31:10.832+0000 7f27f7634640 1 -- 172.21.15.37:0/3108019166 --> [v2:172.21.15.203:6832/2418205237,v1:172.21.15.203:6833/2418205237] -- mgr_command(tid 0: {"prefix": "osd deep-scrub", "who": "5", "target": ["mon-mgr", ""]}) -- 0x7f27f006a140 con 0x7f27dc0744b0 2024-04-25T10:31:10.834 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:31:10.832+0000 7f27edffb640 1 -- 172.21.15.37:0/3108019166 <== mgr.4100 v2:172.21.15.203:6832/2418205237 1 ==== mgr_command_reply(tid 0: 0 instructed osd(s) 5 to deep-scrub) ==== 41+0+0 (secure 0 0 0) 0x7f27f006a140 con 0x7f27dc0744b0 2024-04-25T10:31:10.835 INFO:teuthology.orchestra.run.smithi037.stderr:instructed osd(s) 5 to deep-scrub 2024-04-25T10:31:10.837 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:31:10.834+0000 7f27f7634640 1 -- 172.21.15.37:0/3108019166 >> [v2:172.21.15.203:6832/2418205237,v1:172.21.15.203:6833/2418205237] conn(0x7f27dc0744b0 msgr2=0x7f27dc0769c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:31:10.837 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:31:10.834+0000 7f27f7634640 1 --2- 172.21.15.37:0/3108019166 >> [v2:172.21.15.203:6832/2418205237,v1:172.21.15.203:6833/2418205237] conn(0x7f27dc0744b0 0x7f27dc0769c0 secure :-1 s=READY pgs=332 cs=0 l=1 rev1=1 crypto rx=0x7f27e8034000 tx=0x7f27e802e000 comp rx=0 tx=0).stop 2024-04-25T10:31:10.837 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:31:10.834+0000 7f27f7634640 1 -- 172.21.15.37:0/3108019166 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f27f015aa20 msgr2=0x7f27f00cd7d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:31:10.837 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:31:10.834+0000 7f27f7634640 1 --2- 172.21.15.37:0/3108019166 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f27f015aa20 0x7f27f00cd7d0 secure :-1 s=READY pgs=740 cs=0 l=1 rev1=1 crypto rx=0x7f27e000b9e0 tx=0x7f27e000bea0 comp rx=0 tx=0).stop 2024-04-25T10:31:10.837 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:31:10.834+0000 7f27f7634640 1 -- 172.21.15.37:0/3108019166 shutdown_connections 2024-04-25T10:31:10.837 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:31:10.834+0000 7f27f7634640 1 --2- 172.21.15.37:0/3108019166 >> [v2:172.21.15.203:6832/2418205237,v1:172.21.15.203:6833/2418205237] conn(0x7f27dc0744b0 0x7f27dc0769c0 unknown :-1 s=CLOSED pgs=332 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:31:10.837 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:31:10.834+0000 7f27f7634640 1 --2- 172.21.15.37:0/3108019166 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f27f015aa20 0x7f27f00cd7d0 unknown :-1 s=CLOSED pgs=740 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:31:10.837 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:31:10.834+0000 7f27f7634640 1 --2- 172.21.15.37:0/3108019166 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f27f00d0d10 0x7f27f00c97e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:31:10.837 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:31:10.834+0000 7f27f7634640 1 --2- 172.21.15.37:0/3108019166 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f27f00b7750 0x7f27f0153a90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:31:10.837 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:31:10.834+0000 7f27f7634640 1 -- 172.21.15.37:0/3108019166 >> 172.21.15.37:0/3108019166 conn(0x7f27f00d2750 msgr2=0x7f27f00d53f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:31:10.837 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:31:10.834+0000 7f27f7634640 1 -- 172.21.15.37:0/3108019166 shutdown_connections 2024-04-25T10:31:10.837 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:31:10.834+0000 7f27f7634640 1 -- 172.21.15.37:0/3108019166 wait complete. 2024-04-25T10:31:10.848 INFO:tasks.ceph:Scrubbing osd.6 2024-04-25T10:31:10.848 DEBUG:teuthology.orchestra.run.smithi037:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.6 config set osd_debug_deep_scrub_sleep 0 2024-04-25T10:31:10.963 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:31:10.961+0000 7fe723430640 1 Processor -- start 2024-04-25T10:31:10.963 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:31:10.961+0000 7fe723430640 1 -- start start 2024-04-25T10:31:10.965 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:31:10.962+0000 7fe723430640 1 --2- >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7fe71c06ddc0 0x7fe71c0b2cd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:31:10.965 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:31:10.962+0000 7fe723430640 1 --2- >> v2:172.21.15.37:3300/0 conn(0x7fe71c09c230 0x7fe71c0b40a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:31:10.965 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:31:10.962+0000 7fe723430640 1 -- --> v1:172.21.15.203:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fe71c160820 con 0x7fe71c06ec70 2024-04-25T10:31:10.965 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:31:10.962+0000 7fe723430640 1 -- --> v2:172.21.15.37:3300/0 -- mon_getmap magic: 0 -- 0x7fe71c06f570 con 0x7fe71c09c230 2024-04-25T10:31:10.965 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:31:10.962+0000 7fe723430640 1 -- --> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] -- mon_getmap magic: 0 -- 0x7fe71c0b32e0 con 0x7fe71c06ddc0 2024-04-25T10:31:10.965 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:31:10.962+0000 7fe7219cd640 1 --2- >> v2:172.21.15.37:3300/0 conn(0x7fe71c09c230 0x7fe71c0b40a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:31:10.965 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:31:10.962+0000 7fe7219cd640 1 --2- >> v2:172.21.15.37:3300/0 conn(0x7fe71c09c230 0x7fe71c0b40a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.37:3300/0 says I am v2:172.21.15.37:59628/0 (socket says 172.21.15.37:59628) 2024-04-25T10:31:10.965 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:31:10.962+0000 7fe7219cd640 1 -- 172.21.15.37:0/627020588 learned_addr learned my addr 172.21.15.37:0/627020588 (peer_addr_for_me v2:172.21.15.37:0/0) 2024-04-25T10:31:10.965 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:31:10.963+0000 7fe7211cc640 1 --2- 172.21.15.37:0/627020588 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7fe71c06ddc0 0x7fe71c0b2cd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:31:10.966 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:31:10.963+0000 7fe7219cd640 1 -- 172.21.15.37:0/627020588 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7fe71c06ddc0 msgr2=0x7fe71c0b2cd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:31:10.966 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:31:10.963+0000 7fe7219cd640 1 --2- 172.21.15.37:0/627020588 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7fe71c06ddc0 0x7fe71c0b2cd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:31:10.966 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:31:10.963+0000 7fe7219cd640 1 -- 172.21.15.37:0/627020588 >> v1:172.21.15.203:6789/0 conn(0x7fe71c06ec70 legacy=0x7fe71c09bb20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:31:10.966 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:31:10.963+0000 7fe7219cd640 1 -- 172.21.15.37:0/627020588 --> v2:172.21.15.37:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fe71c0b35c0 con 0x7fe71c09c230 2024-04-25T10:31:10.967 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:31:10.963+0000 7fe7219cd640 1 --2- 172.21.15.37:0/627020588 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fe71c09c230 0x7fe71c0b40a0 secure :-1 s=READY pgs=741 cs=0 l=1 rev1=1 crypto rx=0x7fe714009e50 tx=0x7fe71402a710 comp rx=0 tx=0).ready entity=mon.0 client_cookie=11d3966ced239672 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:31:10.967 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:31:10.964+0000 7fe713fff640 1 -- 172.21.15.37:0/627020588 <== mon.0 v2:172.21.15.37:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe714036070 con 0x7fe71c09c230 2024-04-25T10:31:10.967 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:31:10.964+0000 7fe713fff640 1 -- 172.21.15.37:0/627020588 <== mon.0 v2:172.21.15.37:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fe714031600 con 0x7fe71c09c230 2024-04-25T10:31:10.967 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:31:10.964+0000 7fe713fff640 1 -- 172.21.15.37:0/627020588 <== mon.0 v2:172.21.15.37:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe7140319d0 con 0x7fe71c09c230 2024-04-25T10:31:10.967 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:31:10.964+0000 7fe723430640 1 -- 172.21.15.37:0/627020588 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fe71c09c230 msgr2=0x7fe71c0b40a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:31:10.967 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:31:10.964+0000 7fe723430640 1 --2- 172.21.15.37:0/627020588 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fe71c09c230 0x7fe71c0b40a0 secure :-1 s=READY pgs=741 cs=0 l=1 rev1=1 crypto rx=0x7fe714009e50 tx=0x7fe71402a710 comp rx=0 tx=0).stop 2024-04-25T10:31:10.967 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:31:10.964+0000 7fe723430640 1 -- 172.21.15.37:0/627020588 shutdown_connections 2024-04-25T10:31:10.967 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:31:10.964+0000 7fe723430640 1 --2- 172.21.15.37:0/627020588 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fe71c09c230 0x7fe71c0b40a0 unknown :-1 s=CLOSED pgs=741 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:31:10.968 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:31:10.964+0000 7fe723430640 1 --2- 172.21.15.37:0/627020588 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7fe71c06ddc0 0x7fe71c0b2cd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:31:10.968 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:31:10.964+0000 7fe723430640 1 -- 172.21.15.37:0/627020588 >> 172.21.15.37:0/627020588 conn(0x7fe71c0b5470 msgr2=0x7fe71c0b5880 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:31:10.968 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:31:10.964+0000 7fe723430640 1 -- 172.21.15.37:0/627020588 shutdown_connections 2024-04-25T10:31:10.968 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:31:10.964+0000 7fe723430640 1 -- 172.21.15.37:0/627020588 wait complete. 2024-04-25T10:31:10.968 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:31:10.966+0000 7fe723430640 1 Processor -- start 2024-04-25T10:31:10.968 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:31:10.966+0000 7fe723430640 1 -- start start 2024-04-25T10:31:10.971 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:31:10.967+0000 7fe723430640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fe71c06ddc0 0x7fe71c1609f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:31:10.971 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:31:10.967+0000 7fe723430640 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fe71c06ec70 0x7fe71c0c9510 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:31:10.971 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:31:10.967+0000 7fe723430640 1 --2- >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7fe71c155a10 0x7fe71c15e260 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:31:10.971 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:31:10.967+0000 7fe723430640 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 -- 0x7fe71c06f080 con 0x7fe71c06ddc0 2024-04-25T10:31:10.971 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:31:10.967+0000 7fe723430640 1 -- --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_getmap magic: 0 -- 0x7fe71c0b3b80 con 0x7fe71c06ec70 2024-04-25T10:31:10.971 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:31:10.967+0000 7fe723430640 1 -- --> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] -- mon_getmap magic: 0 -- 0x7fe71c06f200 con 0x7fe71c155a10 2024-04-25T10:31:10.971 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:31:10.968+0000 7fe7211cc640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fe71c06ddc0 0x7fe71c1609f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:31:10.971 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:31:10.968+0000 7fe7211cc640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fe71c06ddc0 0x7fe71c1609f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.37:3300/0 says I am v2:172.21.15.37:59638/0 (socket says 172.21.15.37:59638) 2024-04-25T10:31:10.971 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:31:10.968+0000 7fe7211cc640 1 -- 172.21.15.37:0/2170188188 learned_addr learned my addr 172.21.15.37:0/2170188188 (peer_addr_for_me v2:172.21.15.37:0/0) 2024-04-25T10:31:10.971 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:31:10.968+0000 7fe7209cb640 1 --2- 172.21.15.37:0/2170188188 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fe71c06ec70 0x7fe71c0c9510 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:31:10.971 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:31:10.968+0000 7fe7219cd640 1 --2- 172.21.15.37:0/2170188188 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7fe71c155a10 0x7fe71c15e260 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:31:10.971 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:31:10.968+0000 7fe7211cc640 1 -- 172.21.15.37:0/2170188188 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7fe71c155a10 msgr2=0x7fe71c15e260 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:31:10.971 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:31:10.968+0000 7fe7211cc640 1 --2- 172.21.15.37:0/2170188188 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7fe71c155a10 0x7fe71c15e260 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:31:10.971 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:31:10.968+0000 7fe7211cc640 1 -- 172.21.15.37:0/2170188188 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fe71c06ec70 msgr2=0x7fe71c0c9510 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:31:10.972 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:31:10.968+0000 7fe7211cc640 1 --2- 172.21.15.37:0/2170188188 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fe71c06ec70 0x7fe71c0c9510 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:31:10.972 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:31:10.968+0000 7fe7211cc640 1 -- 172.21.15.37:0/2170188188 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fe71c0c7770 con 0x7fe71c06ddc0 2024-04-25T10:31:10.972 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:31:10.968+0000 7fe7211cc640 1 --2- 172.21.15.37:0/2170188188 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fe71c06ddc0 0x7fe71c1609f0 secure :-1 s=READY pgs=742 cs=0 l=1 rev1=1 crypto rx=0x7fe71800d830 tx=0x7fe71800dcf0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:31:10.972 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:31:10.968+0000 7fe711ffb640 1 -- 172.21.15.37:0/2170188188 <== mon.0 v2:172.21.15.37:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe718005990 con 0x7fe71c06ddc0 2024-04-25T10:31:10.972 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:31:10.968+0000 7fe711ffb640 1 -- 172.21.15.37:0/2170188188 <== mon.0 v2:172.21.15.37:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fe718005b50 con 0x7fe71c06ddc0 2024-04-25T10:31:10.972 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:31:10.968+0000 7fe723430640 1 -- 172.21.15.37:0/2170188188 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fe71c0c9a50 con 0x7fe71c06ddc0 2024-04-25T10:31:10.972 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:31:10.968+0000 7fe711ffb640 1 -- 172.21.15.37:0/2170188188 <== mon.0 v2:172.21.15.37:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe7180096a0 con 0x7fe71c06ddc0 2024-04-25T10:31:10.972 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:31:10.968+0000 7fe723430640 1 -- 172.21.15.37:0/2170188188 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fe71c161490 con 0x7fe71c06ddc0 2024-04-25T10:31:10.973 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:31:10.969+0000 7fe723430640 1 -- 172.21.15.37:0/2170188188 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_get_version(what=osdmap handle=1) -- 0x7fe71c06a140 con 0x7fe71c06ddc0 2024-04-25T10:31:10.973 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:31:10.970+0000 7fe711ffb640 1 -- 172.21.15.37:0/2170188188 <== mon.0 v2:172.21.15.37:3300/0 4 ==== mgrmap(e 5) ==== 96782+0+0 (secure 0 0 0) 0x7fe718009a50 con 0x7fe71c06ddc0 2024-04-25T10:31:10.974 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:31:10.971+0000 7fe711ffb640 1 --2- 172.21.15.37:0/2170188188 >> [v2:172.21.15.203:6832/2418205237,v1:172.21.15.203:6833/2418205237] conn(0x7fe7000744b0 0x7fe7000769c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:31:10.974 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:31:10.971+0000 7fe711ffb640 1 -- 172.21.15.37:0/2170188188 <== mon.0 v2:172.21.15.37:3300/0 5 ==== osd_map(50..50 src has 1..50) ==== 5029+0+0 (secure 0 0 0) 0x7fe718095420 con 0x7fe71c06ddc0 2024-04-25T10:31:10.974 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:31:10.971+0000 7fe711ffb640 1 --2- 172.21.15.37:0/2170188188 >> [v2:172.21.15.203:6810/2309615014,v1:172.21.15.203:6812/2309615014] conn(0x7fe70007a460 0x7fe70007c8b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:31:10.974 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:31:10.971+0000 7fe711ffb640 1 -- 172.21.15.37:0/2170188188 --> [v2:172.21.15.203:6810/2309615014,v1:172.21.15.203:6812/2309615014] -- command(tid 1: {"prefix": "get_command_descriptions"}) -- 0x7fe71801ae70 con 0x7fe70007a460 2024-04-25T10:31:10.974 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:31:10.971+0000 7fe711ffb640 1 -- 172.21.15.37:0/2170188188 <== mon.0 v2:172.21.15.37:3300/0 6 ==== mon_get_version_reply(handle=1 version=50) ==== 24+0+0 (secure 0 0 0) 0x7fe71809c050 con 0x7fe71c06ddc0 2024-04-25T10:31:10.974 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:31:10.971+0000 7fe7209cb640 1 --2- 172.21.15.37:0/2170188188 >> [v2:172.21.15.203:6832/2418205237,v1:172.21.15.203:6833/2418205237] conn(0x7fe7000744b0 0x7fe7000769c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:31:10.974 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:31:10.971+0000 7fe7219cd640 1 --2- 172.21.15.37:0/2170188188 >> [v2:172.21.15.203:6810/2309615014,v1:172.21.15.203:6812/2309615014] conn(0x7fe70007a460 0x7fe70007c8b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:31:10.974 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:31:10.971+0000 7fe7209cb640 1 --2- 172.21.15.37:0/2170188188 >> [v2:172.21.15.203:6832/2418205237,v1:172.21.15.203:6833/2418205237] conn(0x7fe7000744b0 0x7fe7000769c0 secure :-1 s=READY pgs=333 cs=0 l=1 rev1=1 crypto rx=0x7fe70c007fd0 tx=0x7fe70c009040 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:31:10.974 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:31:10.972+0000 7fe7219cd640 1 --2- 172.21.15.37:0/2170188188 >> [v2:172.21.15.203:6810/2309615014,v1:172.21.15.203:6812/2309615014] conn(0x7fe70007a460 0x7fe70007c8b0 crc :-1 s=READY pgs=18 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.6 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:31:10.975 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:31:10.972+0000 7fe711ffb640 1 -- 172.21.15.37:0/2170188188 <== osd.6 v2:172.21.15.203:6810/2309615014 1 ==== command_reply(tid 1: 0 ) ==== 8+0+27193 (crc 0 0 0) 0x7fe71801ae70 con 0x7fe70007a460 2024-04-25T10:31:10.987 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:31:10.986+0000 7fe723430640 1 -- 172.21.15.37:0/2170188188 --> [v2:172.21.15.203:6810/2309615014,v1:172.21.15.203:6812/2309615014] -- command(tid 2: {"prefix": "config set", "var": "osd_debug_deep_scrub_sleep", "val": ["0"]}) -- 0x7fe71c0b4420 con 0x7fe70007a460 2024-04-25T10:31:10.988 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:31:10.986+0000 7fe711ffb640 1 -- 172.21.15.37:0/2170188188 <== osd.6 v2:172.21.15.203:6810/2309615014 2 ==== command_reply(tid 2: 0 ) ==== 8+0+97 (crc 0 0 0) 0x7fe71c0b4420 con 0x7fe70007a460 2024-04-25T10:31:10.988 INFO:teuthology.orchestra.run.smithi037.stdout:{ 2024-04-25T10:31:10.988 INFO:teuthology.orchestra.run.smithi037.stdout: "success": "osd_debug_deep_scrub_sleep = '' (not observed, change may require restart) " 2024-04-25T10:31:10.988 INFO:teuthology.orchestra.run.smithi037.stdout:} 2024-04-25T10:31:10.989 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:31:10.986+0000 7fe723430640 1 -- 172.21.15.37:0/2170188188 >> [v2:172.21.15.203:6810/2309615014,v1:172.21.15.203:6812/2309615014] conn(0x7fe70007a460 msgr2=0x7fe70007c8b0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:31:10.990 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:31:10.986+0000 7fe723430640 1 --2- 172.21.15.37:0/2170188188 >> [v2:172.21.15.203:6810/2309615014,v1:172.21.15.203:6812/2309615014] conn(0x7fe70007a460 0x7fe70007c8b0 crc :-1 s=READY pgs=18 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:31:10.990 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:31:10.986+0000 7fe723430640 1 -- 172.21.15.37:0/2170188188 >> [v2:172.21.15.203:6832/2418205237,v1:172.21.15.203:6833/2418205237] conn(0x7fe7000744b0 msgr2=0x7fe7000769c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:31:10.990 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:31:10.986+0000 7fe723430640 1 --2- 172.21.15.37:0/2170188188 >> [v2:172.21.15.203:6832/2418205237,v1:172.21.15.203:6833/2418205237] conn(0x7fe7000744b0 0x7fe7000769c0 secure :-1 s=READY pgs=333 cs=0 l=1 rev1=1 crypto rx=0x7fe70c007fd0 tx=0x7fe70c009040 comp rx=0 tx=0).stop 2024-04-25T10:31:10.990 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:31:10.986+0000 7fe723430640 1 -- 172.21.15.37:0/2170188188 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fe71c06ddc0 msgr2=0x7fe71c1609f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:31:10.990 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:31:10.986+0000 7fe723430640 1 --2- 172.21.15.37:0/2170188188 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fe71c06ddc0 0x7fe71c1609f0 secure :-1 s=READY pgs=742 cs=0 l=1 rev1=1 crypto rx=0x7fe71800d830 tx=0x7fe71800dcf0 comp rx=0 tx=0).stop 2024-04-25T10:31:10.990 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:31:10.987+0000 7fe7219cd640 1 -- 172.21.15.37:0/2170188188 reap_dead start 2024-04-25T10:31:10.990 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:31:10.987+0000 7fe723430640 1 -- 172.21.15.37:0/2170188188 shutdown_connections 2024-04-25T10:31:10.990 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:31:10.987+0000 7fe723430640 1 -- 172.21.15.37:0/2170188188 >> 172.21.15.37:0/2170188188 conn(0x7fe71c0b5470 msgr2=0x7fe71c0b4940 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:31:10.990 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:31:10.987+0000 7fe723430640 1 -- 172.21.15.37:0/2170188188 shutdown_connections 2024-04-25T10:31:10.990 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:31:10.987+0000 7fe723430640 1 -- 172.21.15.37:0/2170188188 wait complete. 2024-04-25T10:31:10.999 DEBUG:teuthology.orchestra.run.smithi037:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd deep-scrub 6 2024-04-25T10:31:11.111 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:31:11.109+0000 7fd75186e640 1 Processor -- start 2024-04-25T10:31:11.111 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:31:11.109+0000 7fd75186e640 1 -- start start 2024-04-25T10:31:11.115 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:31:11.110+0000 7fd75186e640 1 --2- >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7fd74c06e260 0x7fd74c06e670 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:31:11.115 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:31:11.111+0000 7fd75186e640 1 -- --> v1:172.21.15.37:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fd74c09ce70 con 0x7fd74c06ebb0 2024-04-25T10:31:11.115 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:31:11.111+0000 7fd75186e640 1 -- --> v1:172.21.15.203:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fd74c09d150 con 0x7fd74c164ce0 2024-04-25T10:31:11.115 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:31:11.111+0000 7fd75186e640 1 -- --> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] -- mon_getmap magic: 0 -- 0x7fd74c0d3370 con 0x7fd74c06e260 2024-04-25T10:31:11.115 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:31:11.111+0000 7fd74a7fc640 1 --1- >> v1:172.21.15.37:6789/0 conn(0x7fd74c06ebb0 0x7fd74c1645d0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.37:6789/0 says I am v1:172.21.15.37:56416/0 (socket says 172.21.15.37:56416) 2024-04-25T10:31:11.115 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:31:11.111+0000 7fd74a7fc640 1 -- 172.21.15.37:0/2663157366 learned_addr learned my addr 172.21.15.37:0/2663157366 (peer_addr_for_me v1:172.21.15.37:0/0) 2024-04-25T10:31:11.115 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:31:11.111+0000 7fd74affd640 1 --2- 172.21.15.37:0/2663157366 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7fd74c06e260 0x7fd74c06e670 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:31:11.115 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:31:11.111+0000 7fd749ffb640 1 -- 172.21.15.37:0/2663157366 <== mon.0 v1:172.21.15.37:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 3820076080 0 0) 0x7fd74c09ce70 con 0x7fd74c06ebb0 2024-04-25T10:31:11.115 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:31:11.111+0000 7fd749ffb640 1 -- 172.21.15.37:0/2663157366 --> v1:172.21.15.37:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7fd734003520 con 0x7fd74c06ebb0 2024-04-25T10:31:11.116 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:31:11.111+0000 7fd749ffb640 1 -- 172.21.15.37:0/2663157366 <== mon.0 v1:172.21.15.37:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 2333741047 0 0) 0x7fd734003520 con 0x7fd74c06ebb0 2024-04-25T10:31:11.116 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:31:11.111+0000 7fd749ffb640 1 -- 172.21.15.37:0/2663157366 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7fd74c06e260 msgr2=0x7fd74c06e670 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:31:11.116 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:31:11.111+0000 7fd749ffb640 1 --2- 172.21.15.37:0/2663157366 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7fd74c06e260 0x7fd74c06e670 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:31:11.116 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:31:11.111+0000 7fd749ffb640 1 -- 172.21.15.37:0/2663157366 >> v1:172.21.15.203:6789/0 conn(0x7fd74c164ce0 legacy=0x7fd74c0b3c80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:31:11.116 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:31:11.111+0000 7fd749ffb640 1 -- 172.21.15.37:0/2663157366 --> v1:172.21.15.37:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fd74c0b9000 con 0x7fd74c06ebb0 2024-04-25T10:31:11.116 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:31:11.111+0000 7fd749ffb640 1 -- 172.21.15.37:0/2663157366 <== mon.0 v1:172.21.15.37:6789/0 3 ==== mon_map magic: 0 ==== 413+0+0 (unknown 1577794600 0 0) 0x7fd740001670 con 0x7fd74c06ebb0 2024-04-25T10:31:11.116 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:31:11.111+0000 7fd74affd640 1 --2- 172.21.15.37:0/2663157366 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7fd74c06e260 0x7fd74c06e670 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-25T10:31:11.116 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:31:11.111+0000 7fd749ffb640 1 -- 172.21.15.37:0/2663157366 >> v1:172.21.15.37:6789/0 conn(0x7fd74c06ebb0 legacy=0x7fd74c1645d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:31:11.116 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:31:11.111+0000 7fd749ffb640 1 --2- 172.21.15.37:0/2663157366 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fd7340039e0 0x7fd734003e10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:31:11.116 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:31:11.111+0000 7fd749ffb640 1 --2- 172.21.15.37:0/2663157366 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fd734004800 0x7fd734004c10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:31:11.116 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:31:11.111+0000 7fd749ffb640 1 --2- 172.21.15.37:0/2663157366 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7fd74c06e260 0x7fd7340051c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:31:11.116 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:31:11.111+0000 7fd749ffb640 1 -- 172.21.15.37:0/2663157366 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 -- 0x7fd73400ba90 con 0x7fd7340039e0 2024-04-25T10:31:11.116 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:31:11.111+0000 7fd749ffb640 1 -- 172.21.15.37:0/2663157366 --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_getmap magic: 0 -- 0x7fd73400bc40 con 0x7fd734004800 2024-04-25T10:31:11.117 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:31:11.111+0000 7fd749ffb640 1 -- 172.21.15.37:0/2663157366 --> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] -- mon_getmap magic: 0 -- 0x7fd73400be40 con 0x7fd74c06e260 2024-04-25T10:31:11.117 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:31:11.112+0000 7fd74affd640 1 --2- 172.21.15.37:0/2663157366 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fd7340039e0 0x7fd734003e10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:31:11.117 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:31:11.112+0000 7fd74a7fc640 1 --2- 172.21.15.37:0/2663157366 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fd734004800 0x7fd734004c10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:31:11.117 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:31:11.112+0000 7fd74b7fe640 1 --2- 172.21.15.37:0/2663157366 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7fd74c06e260 0x7fd7340051c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:31:11.117 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:31:11.112+0000 7fd74affd640 1 -- 172.21.15.37:0/2663157366 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7fd74c06e260 msgr2=0x7fd7340051c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:31:11.117 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:31:11.112+0000 7fd74affd640 1 --2- 172.21.15.37:0/2663157366 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7fd74c06e260 0x7fd7340051c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:31:11.117 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:31:11.112+0000 7fd74affd640 1 -- 172.21.15.37:0/2663157366 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fd734004800 msgr2=0x7fd734004c10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:31:11.117 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:31:11.112+0000 7fd74affd640 1 --2- 172.21.15.37:0/2663157366 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fd734004800 0x7fd734004c10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:31:11.117 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:31:11.112+0000 7fd74affd640 1 -- 172.21.15.37:0/2663157366 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fd734003520 con 0x7fd7340039e0 2024-04-25T10:31:11.117 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:31:11.112+0000 7fd74a7fc640 1 --2- 172.21.15.37:0/2663157366 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fd734004800 0x7fd734004c10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T10:31:11.117 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:31:11.113+0000 7fd74affd640 1 --2- 172.21.15.37:0/2663157366 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fd7340039e0 0x7fd734003e10 secure :-1 s=READY pgs=744 cs=0 l=1 rev1=1 crypto rx=0x7fd73c00e320 tx=0x7fd734005830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=1b8e9c78714b4240 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:31:11.117 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:31:11.113+0000 7fd749ffb640 1 -- 172.21.15.37:0/2663157366 <== mon.0 v2:172.21.15.37:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd73c009de0 con 0x7fd7340039e0 2024-04-25T10:31:11.117 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:31:11.113+0000 7fd749ffb640 1 -- 172.21.15.37:0/2663157366 <== mon.0 v2:172.21.15.37:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fd73c003b90 con 0x7fd7340039e0 2024-04-25T10:31:11.117 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:31:11.113+0000 7fd749ffb640 1 -- 172.21.15.37:0/2663157366 <== mon.0 v2:172.21.15.37:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd73c004020 con 0x7fd7340039e0 2024-04-25T10:31:11.120 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:31:11.113+0000 7fd75186e640 1 -- 172.21.15.37:0/2663157366 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fd7340039e0 msgr2=0x7fd734003e10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:31:11.120 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:31:11.113+0000 7fd75186e640 1 --2- 172.21.15.37:0/2663157366 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fd7340039e0 0x7fd734003e10 secure :-1 s=READY pgs=744 cs=0 l=1 rev1=1 crypto rx=0x7fd73c00e320 tx=0x7fd734005830 comp rx=0 tx=0).stop 2024-04-25T10:31:11.120 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:31:11.113+0000 7fd74b7fe640 1 -- 172.21.15.37:0/2663157366 reap_dead start 2024-04-25T10:31:11.120 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:31:11.113+0000 7fd75186e640 1 -- 172.21.15.37:0/2663157366 shutdown_connections 2024-04-25T10:31:11.120 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:31:11.113+0000 7fd75186e640 1 -- 172.21.15.37:0/2663157366 >> 172.21.15.37:0/2663157366 conn(0x7fd74c0d0390 msgr2=0x7fd74c0d07a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:31:11.120 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:31:11.113+0000 7fd75186e640 1 -- 172.21.15.37:0/2663157366 shutdown_connections 2024-04-25T10:31:11.120 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:31:11.113+0000 7fd75186e640 1 -- 172.21.15.37:0/2663157366 wait complete. 2024-04-25T10:31:11.121 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:31:11.115+0000 7fd75186e640 1 Processor -- start 2024-04-25T10:31:11.121 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:31:11.115+0000 7fd75186e640 1 -- start start 2024-04-25T10:31:11.121 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:31:11.117+0000 7fd75186e640 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fd7340039e0 0x7fd74c0a0190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:31:11.121 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:31:11.117+0000 7fd75186e640 1 --2- >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7fd74c154d10 0x7fd74c155120 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:31:11.121 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:31:11.117+0000 7fd75186e640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fd74c0b77b0 0x7fd74c0b7bc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:31:11.121 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:31:11.117+0000 7fd75186e640 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 -- 0x7fd74c0b91f0 con 0x7fd74c0b77b0 2024-04-25T10:31:11.121 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:31:11.117+0000 7fd75186e640 1 -- --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_getmap magic: 0 -- 0x7fd74c0d3090 con 0x7fd7340039e0 2024-04-25T10:31:11.121 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:31:11.117+0000 7fd75186e640 1 -- --> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] -- mon_getmap magic: 0 -- 0x7fd74c06d830 con 0x7fd74c154d10 2024-04-25T10:31:11.121 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:31:11.117+0000 7fd74b7fe640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fd74c0b77b0 0x7fd74c0b7bc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:31:11.121 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:31:11.117+0000 7fd74b7fe640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fd74c0b77b0 0x7fd74c0b7bc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.37:3300/0 says I am v2:172.21.15.37:59646/0 (socket says 172.21.15.37:59646) 2024-04-25T10:31:11.121 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:31:11.117+0000 7fd74b7fe640 1 -- 172.21.15.37:0/651264739 learned_addr learned my addr 172.21.15.37:0/651264739 (peer_addr_for_me v2:172.21.15.37:0/0) 2024-04-25T10:31:11.121 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:31:11.117+0000 7fd74a7fc640 1 --2- 172.21.15.37:0/651264739 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7fd74c154d10 0x7fd74c155120 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:31:11.121 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:31:11.117+0000 7fd74affd640 1 --2- 172.21.15.37:0/651264739 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fd7340039e0 0x7fd74c0a0190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:31:11.121 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:31:11.117+0000 7fd74b7fe640 1 -- 172.21.15.37:0/651264739 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7fd74c154d10 msgr2=0x7fd74c155120 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:31:11.121 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:31:11.117+0000 7fd74b7fe640 1 --2- 172.21.15.37:0/651264739 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7fd74c154d10 0x7fd74c155120 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:31:11.122 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:31:11.117+0000 7fd74b7fe640 1 -- 172.21.15.37:0/651264739 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fd7340039e0 msgr2=0x7fd74c0a0190 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:31:11.122 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:31:11.117+0000 7fd74b7fe640 1 --2- 172.21.15.37:0/651264739 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fd7340039e0 0x7fd74c0a0190 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:31:11.122 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:31:11.117+0000 7fd74b7fe640 1 -- 172.21.15.37:0/651264739 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fd74c0b8100 con 0x7fd74c0b77b0 2024-04-25T10:31:11.122 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:31:11.117+0000 7fd74b7fe640 1 --2- 172.21.15.37:0/651264739 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fd74c0b77b0 0x7fd74c0b7bc0 secure :-1 s=READY pgs=745 cs=0 l=1 rev1=1 crypto rx=0x7fd74c08cfe0 tx=0x7fd74400fc00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:31:11.122 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:31:11.118+0000 7fd733fff640 1 -- 172.21.15.37:0/651264739 <== mon.0 v2:172.21.15.37:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd744019070 con 0x7fd74c0b77b0 2024-04-25T10:31:11.122 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:31:11.118+0000 7fd733fff640 1 -- 172.21.15.37:0/651264739 <== mon.0 v2:172.21.15.37:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fd744005030 con 0x7fd74c0b77b0 2024-04-25T10:31:11.122 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:31:11.118+0000 7fd75186e640 1 -- 172.21.15.37:0/651264739 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fd74c155a80 con 0x7fd74c0b77b0 2024-04-25T10:31:11.122 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:31:11.118+0000 7fd733fff640 1 -- 172.21.15.37:0/651264739 <== mon.0 v2:172.21.15.37:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd744005400 con 0x7fd74c0b77b0 2024-04-25T10:31:11.122 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:31:11.118+0000 7fd75186e640 1 -- 172.21.15.37:0/651264739 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fd74c156040 con 0x7fd74c0b77b0 2024-04-25T10:31:11.123 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:31:11.119+0000 7fd75186e640 1 -- 172.21.15.37:0/651264739 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fd74c06a140 con 0x7fd74c0b77b0 2024-04-25T10:31:11.123 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:31:11.119+0000 7fd733fff640 1 -- 172.21.15.37:0/651264739 <== mon.0 v2:172.21.15.37:3300/0 4 ==== mgrmap(e 5) ==== 96782+0+0 (secure 0 0 0) 0x7fd74401c070 con 0x7fd74c0b77b0 2024-04-25T10:31:11.123 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:31:11.120+0000 7fd733fff640 1 --2- 172.21.15.37:0/651264739 >> [v2:172.21.15.203:6832/2418205237,v1:172.21.15.203:6833/2418205237] conn(0x7fd7380744b0 0x7fd7380769c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:31:11.123 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:31:11.120+0000 7fd733fff640 1 -- 172.21.15.37:0/651264739 <== mon.0 v2:172.21.15.37:3300/0 5 ==== osd_map(50..50 src has 1..50) ==== 5029+0+0 (secure 0 0 0) 0x7fd744093e20 con 0x7fd74c0b77b0 2024-04-25T10:31:11.123 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:31:11.120+0000 7fd74affd640 1 --2- 172.21.15.37:0/651264739 >> [v2:172.21.15.203:6832/2418205237,v1:172.21.15.203:6833/2418205237] conn(0x7fd7380744b0 0x7fd7380769c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:31:11.123 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:31:11.121+0000 7fd74affd640 1 --2- 172.21.15.37:0/651264739 >> [v2:172.21.15.203:6832/2418205237,v1:172.21.15.203:6833/2418205237] conn(0x7fd7380744b0 0x7fd7380769c0 secure :-1 s=READY pgs=334 cs=0 l=1 rev1=1 crypto rx=0x7fd73c034000 tx=0x7fd73c038040 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:31:11.127 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:31:11.125+0000 7fd733fff640 1 -- 172.21.15.37:0/651264739 <== mon.0 v2:172.21.15.37:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187246 (secure 0 0 0) 0x7fd74c0b8100 con 0x7fd74c0b77b0 2024-04-25T10:31:11.311 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:31:11.309+0000 7fd75186e640 1 -- 172.21.15.37:0/651264739 --> [v2:172.21.15.203:6832/2418205237,v1:172.21.15.203:6833/2418205237] -- mgr_command(tid 0: {"prefix": "osd deep-scrub", "who": "6", "target": ["mon-mgr", ""]}) -- 0x7fd74c155810 con 0x7fd7380744b0 2024-04-25T10:31:11.311 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:31:11.309+0000 7fd733fff640 1 -- 172.21.15.37:0/651264739 <== mgr.4100 v2:172.21.15.203:6832/2418205237 1 ==== mgr_command_reply(tid 0: 0 instructed osd(s) 6 to deep-scrub) ==== 41+0+0 (secure 0 0 0) 0x7fd74c155810 con 0x7fd7380744b0 2024-04-25T10:31:11.311 INFO:teuthology.orchestra.run.smithi037.stderr:instructed osd(s) 6 to deep-scrub 2024-04-25T10:31:11.314 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:31:11.311+0000 7fd75186e640 1 -- 172.21.15.37:0/651264739 >> [v2:172.21.15.203:6832/2418205237,v1:172.21.15.203:6833/2418205237] conn(0x7fd7380744b0 msgr2=0x7fd7380769c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:31:11.314 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:31:11.311+0000 7fd75186e640 1 --2- 172.21.15.37:0/651264739 >> [v2:172.21.15.203:6832/2418205237,v1:172.21.15.203:6833/2418205237] conn(0x7fd7380744b0 0x7fd7380769c0 secure :-1 s=READY pgs=334 cs=0 l=1 rev1=1 crypto rx=0x7fd73c034000 tx=0x7fd73c038040 comp rx=0 tx=0).stop 2024-04-25T10:31:11.314 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:31:11.311+0000 7fd75186e640 1 -- 172.21.15.37:0/651264739 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fd74c0b77b0 msgr2=0x7fd74c0b7bc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:31:11.314 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:31:11.311+0000 7fd75186e640 1 --2- 172.21.15.37:0/651264739 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fd74c0b77b0 0x7fd74c0b7bc0 secure :-1 s=READY pgs=745 cs=0 l=1 rev1=1 crypto rx=0x7fd74c08cfe0 tx=0x7fd74400fc00 comp rx=0 tx=0).stop 2024-04-25T10:31:11.314 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:31:11.311+0000 7fd75186e640 1 -- 172.21.15.37:0/651264739 shutdown_connections 2024-04-25T10:31:11.314 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:31:11.311+0000 7fd75186e640 1 --2- 172.21.15.37:0/651264739 >> [v2:172.21.15.203:6832/2418205237,v1:172.21.15.203:6833/2418205237] conn(0x7fd7380744b0 0x7fd7380769c0 unknown :-1 s=CLOSED pgs=334 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:31:11.314 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:31:11.311+0000 7fd75186e640 1 --2- 172.21.15.37:0/651264739 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fd74c0b77b0 0x7fd74c0b7bc0 unknown :-1 s=CLOSED pgs=745 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:31:11.314 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:31:11.311+0000 7fd75186e640 1 --2- 172.21.15.37:0/651264739 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7fd74c154d10 0x7fd74c155120 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:31:11.314 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:31:11.311+0000 7fd75186e640 1 --2- 172.21.15.37:0/651264739 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fd7340039e0 0x7fd74c0a0190 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:31:11.314 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:31:11.311+0000 7fd75186e640 1 -- 172.21.15.37:0/651264739 >> 172.21.15.37:0/651264739 conn(0x7fd74c0d0390 msgr2=0x7fd74c0b2a10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:31:11.314 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:31:11.311+0000 7fd75186e640 1 -- 172.21.15.37:0/651264739 shutdown_connections 2024-04-25T10:31:11.314 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:31:11.311+0000 7fd75186e640 1 -- 172.21.15.37:0/651264739 wait complete. 2024-04-25T10:31:11.324 INFO:tasks.ceph:Scrubbing osd.7 2024-04-25T10:31:11.324 DEBUG:teuthology.orchestra.run.smithi037:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.7 config set osd_debug_deep_scrub_sleep 0 2024-04-25T10:31:11.440 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:31:11.437+0000 7fdde2fce640 1 Processor -- start 2024-04-25T10:31:11.440 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:31:11.438+0000 7fdde2fce640 1 -- start start 2024-04-25T10:31:11.441 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:31:11.439+0000 7fdde2fce640 1 --2- >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7fdddc0d20b0 0x7fdddc0d24c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:31:11.441 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:31:11.439+0000 7fdde2fce640 1 --2- >> v2:172.21.15.203:3300/0 conn(0x7fdddc165140 0x7fdddc06e440 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:31:11.441 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:31:11.439+0000 7fdde2fce640 1 -- --> v1:172.21.15.203:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fdddc0cf950 con 0x7fdddc164620 2024-04-25T10:31:11.441 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:31:11.439+0000 7fdde2fce640 1 -- --> v2:172.21.15.203:3300/0 -- mon_getmap magic: 0 -- 0x7fdddc09cf10 con 0x7fdddc165140 2024-04-25T10:31:11.441 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:31:11.439+0000 7fdde2fce640 1 -- --> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] -- mon_getmap magic: 0 -- 0x7fdddc0b67b0 con 0x7fdddc0d20b0 2024-04-25T10:31:11.441 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:31:11.439+0000 7fdde0d6a640 1 --2- >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7fdddc0d20b0 0x7fdddc0d24c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:31:11.442 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:31:11.439+0000 7fdde156b640 1 --2- >> v2:172.21.15.203:3300/0 conn(0x7fdddc165140 0x7fdddc06e440 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:31:11.442 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:31:11.439+0000 7fdde0d6a640 1 --2- >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7fdddc0d20b0 0x7fdddc0d24c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.203:3301/0 says I am v2:172.21.15.37:47140/0 (socket says 172.21.15.37:47140) 2024-04-25T10:31:11.442 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:31:11.439+0000 7fdde0d6a640 1 -- 172.21.15.37:0/4000624860 learned_addr learned my addr 172.21.15.37:0/4000624860 (peer_addr_for_me v2:172.21.15.37:0/0) 2024-04-25T10:31:11.443 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:31:11.440+0000 7fdde0d6a640 1 -- 172.21.15.37:0/4000624860 >> v2:172.21.15.203:3300/0 conn(0x7fdddc165140 msgr2=0x7fdddc06e440 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:31:11.443 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:31:11.440+0000 7fdde0d6a640 1 --2- 172.21.15.37:0/4000624860 >> v2:172.21.15.203:3300/0 conn(0x7fdddc165140 0x7fdddc06e440 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:31:11.443 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:31:11.440+0000 7fdde0d6a640 1 -- 172.21.15.37:0/4000624860 >> v1:172.21.15.203:6789/0 conn(0x7fdddc164620 legacy=0x7fdddc164a30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:31:11.443 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:31:11.440+0000 7fdde0d6a640 1 -- 172.21.15.37:0/4000624860 --> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fdddc0d0120 con 0x7fdddc0d20b0 2024-04-25T10:31:11.443 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:31:11.440+0000 7fdde0d6a640 1 --2- 172.21.15.37:0/4000624860 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7fdddc0d20b0 0x7fdddc0d24c0 secure :-1 s=READY pgs=65 cs=0 l=1 rev1=1 crypto rx=0x7fddd4009850 tx=0x7fddd402a1a0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=73b409e3131bc38b server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:31:11.443 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:31:11.441+0000 7fdddb7fe640 1 -- 172.21.15.37:0/4000624860 <== mon.2 v2:172.21.15.203:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fddd4004030 con 0x7fdddc0d20b0 2024-04-25T10:31:11.443 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:31:11.441+0000 7fdddb7fe640 1 -- 172.21.15.37:0/4000624860 <== mon.2 v2:172.21.15.203:3301/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fddd40041d0 con 0x7fdddc0d20b0 2024-04-25T10:31:11.443 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:31:11.441+0000 7fdddb7fe640 1 -- 172.21.15.37:0/4000624860 <== mon.2 v2:172.21.15.203:3301/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fddd4003610 con 0x7fdddc0d20b0 2024-04-25T10:31:11.444 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:31:11.441+0000 7fdde2fce640 1 -- 172.21.15.37:0/4000624860 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7fdddc0d20b0 msgr2=0x7fdddc0d24c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:31:11.444 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:31:11.441+0000 7fdde2fce640 1 --2- 172.21.15.37:0/4000624860 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7fdddc0d20b0 0x7fdddc0d24c0 secure :-1 s=READY pgs=65 cs=0 l=1 rev1=1 crypto rx=0x7fddd4009850 tx=0x7fddd402a1a0 comp rx=0 tx=0).stop 2024-04-25T10:31:11.444 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:31:11.441+0000 7fdde2fce640 1 -- 172.21.15.37:0/4000624860 shutdown_connections 2024-04-25T10:31:11.444 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:31:11.441+0000 7fdde2fce640 1 --2- 172.21.15.37:0/4000624860 >> v2:172.21.15.203:3300/0 conn(0x7fdddc165140 0x7fdddc06e440 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:31:11.444 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:31:11.441+0000 7fdde2fce640 1 --2- 172.21.15.37:0/4000624860 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7fdddc0d20b0 0x7fdddc0d24c0 unknown :-1 s=CLOSED pgs=65 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:31:11.444 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:31:11.441+0000 7fdde2fce640 1 -- 172.21.15.37:0/4000624860 >> 172.21.15.37:0/4000624860 conn(0x7fdddc0d03e0 msgr2=0x7fdddc0d07f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:31:11.444 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:31:11.441+0000 7fdde2fce640 1 -- 172.21.15.37:0/4000624860 shutdown_connections 2024-04-25T10:31:11.444 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:31:11.441+0000 7fdde2fce640 1 -- 172.21.15.37:0/4000624860 wait complete. 2024-04-25T10:31:11.445 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:31:11.443+0000 7fdde2fce640 1 Processor -- start 2024-04-25T10:31:11.445 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:31:11.443+0000 7fdde2fce640 1 -- start start 2024-04-25T10:31:11.449 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:31:11.445+0000 7fdde2fce640 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fdddc0d20b0 0x7fdddc0cb420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:31:11.450 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:31:11.445+0000 7fdde2fce640 1 --2- >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7fdddc164620 0x7fdddc158e50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:31:11.450 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:31:11.445+0000 7fdde2fce640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fdddc165140 0x7fdddc0cc840 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:31:11.450 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:31:11.445+0000 7fdde2fce640 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 -- 0x7fdddc0d2b80 con 0x7fdddc165140 2024-04-25T10:31:11.450 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:31:11.445+0000 7fdde2fce640 1 -- --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_getmap magic: 0 -- 0x7fdddc0d2a00 con 0x7fdddc0d20b0 2024-04-25T10:31:11.450 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:31:11.445+0000 7fdde2fce640 1 -- --> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] -- mon_getmap magic: 0 -- 0x7fdddc0d1980 con 0x7fdddc164620 2024-04-25T10:31:11.450 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:31:11.445+0000 7fdde156b640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fdddc165140 0x7fdddc0cc840 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:31:11.450 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:31:11.445+0000 7fdde156b640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fdddc165140 0x7fdddc0cc840 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.37:3300/0 says I am v2:172.21.15.37:59660/0 (socket says 172.21.15.37:59660) 2024-04-25T10:31:11.450 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:31:11.445+0000 7fdde156b640 1 -- 172.21.15.37:0/2265474412 learned_addr learned my addr 172.21.15.37:0/2265474412 (peer_addr_for_me v2:172.21.15.37:0/0) 2024-04-25T10:31:11.450 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:31:11.445+0000 7fdddbfff640 1 --2- 172.21.15.37:0/2265474412 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7fdddc164620 0x7fdddc158e50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:31:11.450 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:31:11.445+0000 7fdde0d6a640 1 --2- 172.21.15.37:0/2265474412 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fdddc0d20b0 0x7fdddc0cb420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:31:11.450 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:31:11.445+0000 7fdde156b640 1 -- 172.21.15.37:0/2265474412 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7fdddc164620 msgr2=0x7fdddc158e50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:31:11.450 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:31:11.445+0000 7fdde156b640 1 --2- 172.21.15.37:0/2265474412 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7fdddc164620 0x7fdddc158e50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:31:11.450 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:31:11.445+0000 7fdde156b640 1 -- 172.21.15.37:0/2265474412 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fdddc0d20b0 msgr2=0x7fdddc0cb420 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:31:11.450 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:31:11.445+0000 7fdde156b640 1 --2- 172.21.15.37:0/2265474412 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fdddc0d20b0 0x7fdddc0cb420 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:31:11.451 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:31:11.445+0000 7fdde156b640 1 -- 172.21.15.37:0/2265474412 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fdddc0c7490 con 0x7fdddc165140 2024-04-25T10:31:11.451 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:31:11.445+0000 7fdddbfff640 1 --2- 172.21.15.37:0/2265474412 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7fdddc164620 0x7fdddc158e50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-25T10:31:11.451 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:31:11.445+0000 7fdde0d6a640 1 --2- 172.21.15.37:0/2265474412 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fdddc0d20b0 0x7fdddc0cb420 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-25T10:31:11.451 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:31:11.446+0000 7fdde156b640 1 --2- 172.21.15.37:0/2265474412 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fdddc165140 0x7fdddc0cc840 secure :-1 s=READY pgs=746 cs=0 l=1 rev1=1 crypto rx=0x7fddc400e7f0 tx=0x7fddc400ebf0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:31:11.451 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:31:11.446+0000 7fddd97fa640 1 -- 172.21.15.37:0/2265474412 <== mon.0 v2:172.21.15.37:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fddc4017070 con 0x7fdddc165140 2024-04-25T10:31:11.451 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:31:11.446+0000 7fddd97fa640 1 -- 172.21.15.37:0/2265474412 <== mon.0 v2:172.21.15.37:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fddc400ee40 con 0x7fdddc165140 2024-04-25T10:31:11.451 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:31:11.446+0000 7fdde2fce640 1 -- 172.21.15.37:0/2265474412 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fdddc0cbd50 con 0x7fdddc165140 2024-04-25T10:31:11.451 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:31:11.446+0000 7fddd97fa640 1 -- 172.21.15.37:0/2265474412 <== mon.0 v2:172.21.15.37:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fddc400cb80 con 0x7fdddc165140 2024-04-25T10:31:11.451 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:31:11.446+0000 7fdde2fce640 1 -- 172.21.15.37:0/2265474412 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fdddc0cc310 con 0x7fdddc165140 2024-04-25T10:31:11.451 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:31:11.446+0000 7fdde2fce640 1 -- 172.21.15.37:0/2265474412 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_get_version(what=osdmap handle=1) -- 0x7fdddc06a140 con 0x7fdddc165140 2024-04-25T10:31:11.451 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:31:11.447+0000 7fddd97fa640 1 -- 172.21.15.37:0/2265474412 <== mon.0 v2:172.21.15.37:3300/0 4 ==== mgrmap(e 5) ==== 96782+0+0 (secure 0 0 0) 0x7fddc401a070 con 0x7fdddc165140 2024-04-25T10:31:11.452 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:31:11.448+0000 7fddd97fa640 1 --2- 172.21.15.37:0/2265474412 >> [v2:172.21.15.203:6832/2418205237,v1:172.21.15.203:6833/2418205237] conn(0x7fddbc0744b0 0x7fddbc0769c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:31:11.452 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:31:11.448+0000 7fddd97fa640 1 -- 172.21.15.37:0/2265474412 <== mon.0 v2:172.21.15.37:3300/0 5 ==== osd_map(50..50 src has 1..50) ==== 5029+0+0 (secure 0 0 0) 0x7fddc4094e20 con 0x7fdddc165140 2024-04-25T10:31:11.453 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:31:11.448+0000 7fddd97fa640 1 --2- 172.21.15.37:0/2265474412 >> [v2:172.21.15.203:6808/3688099014,v1:172.21.15.203:6809/3688099014] conn(0x7fddbc07a460 0x7fddbc07c8b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:31:11.453 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:31:11.448+0000 7fddd97fa640 1 -- 172.21.15.37:0/2265474412 --> [v2:172.21.15.203:6808/3688099014,v1:172.21.15.203:6809/3688099014] -- command(tid 1: {"prefix": "get_command_descriptions"}) -- 0x7fddc4021ec0 con 0x7fddbc07a460 2024-04-25T10:31:11.453 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:31:11.448+0000 7fddd97fa640 1 -- 172.21.15.37:0/2265474412 <== mon.0 v2:172.21.15.37:3300/0 6 ==== mon_get_version_reply(handle=1 version=50) ==== 24+0+0 (secure 0 0 0) 0x7fddc4097520 con 0x7fdddc165140 2024-04-25T10:31:11.453 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:31:11.448+0000 7fdde0d6a640 1 --2- 172.21.15.37:0/2265474412 >> [v2:172.21.15.203:6832/2418205237,v1:172.21.15.203:6833/2418205237] conn(0x7fddbc0744b0 0x7fddbc0769c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:31:11.453 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:31:11.448+0000 7fdddbfff640 1 --2- 172.21.15.37:0/2265474412 >> [v2:172.21.15.203:6808/3688099014,v1:172.21.15.203:6809/3688099014] conn(0x7fddbc07a460 0x7fddbc07c8b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:31:11.453 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:31:11.449+0000 7fdde0d6a640 1 --2- 172.21.15.37:0/2265474412 >> [v2:172.21.15.203:6832/2418205237,v1:172.21.15.203:6833/2418205237] conn(0x7fddbc0744b0 0x7fddbc0769c0 secure :-1 s=READY pgs=335 cs=0 l=1 rev1=1 crypto rx=0x7fddd4033000 tx=0x7fddd402e000 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:31:11.453 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:31:11.449+0000 7fdddbfff640 1 --2- 172.21.15.37:0/2265474412 >> [v2:172.21.15.203:6808/3688099014,v1:172.21.15.203:6809/3688099014] conn(0x7fddbc07a460 0x7fddbc07c8b0 crc :-1 s=READY pgs=19 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.7 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:31:11.453 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:31:11.450+0000 7fddd97fa640 1 -- 172.21.15.37:0/2265474412 <== osd.7 v2:172.21.15.203:6808/3688099014 1 ==== command_reply(tid 1: 0 ) ==== 8+0+27193 (crc 0 0 0) 0x7fddc4021ec0 con 0x7fddbc07a460 2024-04-25T10:31:11.466 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:31:11.463+0000 7fdde2fce640 1 -- 172.21.15.37:0/2265474412 --> [v2:172.21.15.203:6808/3688099014,v1:172.21.15.203:6809/3688099014] -- command(tid 2: {"prefix": "config set", "var": "osd_debug_deep_scrub_sleep", "val": ["0"]}) -- 0x7fdddc0cbb00 con 0x7fddbc07a460 2024-04-25T10:31:11.466 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:31:11.464+0000 7fddd97fa640 1 -- 172.21.15.37:0/2265474412 <== osd.7 v2:172.21.15.203:6808/3688099014 2 ==== command_reply(tid 2: 0 ) ==== 8+0+97 (crc 0 0 0) 0x7fdddc0cbb00 con 0x7fddbc07a460 2024-04-25T10:31:11.466 INFO:teuthology.orchestra.run.smithi037.stdout:{ 2024-04-25T10:31:11.467 INFO:teuthology.orchestra.run.smithi037.stdout: "success": "osd_debug_deep_scrub_sleep = '' (not observed, change may require restart) " 2024-04-25T10:31:11.467 INFO:teuthology.orchestra.run.smithi037.stdout:} 2024-04-25T10:31:11.468 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:31:11.464+0000 7fdde2fce640 1 -- 172.21.15.37:0/2265474412 >> [v2:172.21.15.203:6808/3688099014,v1:172.21.15.203:6809/3688099014] conn(0x7fddbc07a460 msgr2=0x7fddbc07c8b0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:31:11.468 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:31:11.464+0000 7fdde2fce640 1 --2- 172.21.15.37:0/2265474412 >> [v2:172.21.15.203:6808/3688099014,v1:172.21.15.203:6809/3688099014] conn(0x7fddbc07a460 0x7fddbc07c8b0 crc :-1 s=READY pgs=19 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:31:11.468 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:31:11.464+0000 7fdde2fce640 1 -- 172.21.15.37:0/2265474412 >> [v2:172.21.15.203:6832/2418205237,v1:172.21.15.203:6833/2418205237] conn(0x7fddbc0744b0 msgr2=0x7fddbc0769c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:31:11.468 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:31:11.464+0000 7fdde2fce640 1 --2- 172.21.15.37:0/2265474412 >> [v2:172.21.15.203:6832/2418205237,v1:172.21.15.203:6833/2418205237] conn(0x7fddbc0744b0 0x7fddbc0769c0 secure :-1 s=READY pgs=335 cs=0 l=1 rev1=1 crypto rx=0x7fddd4033000 tx=0x7fddd402e000 comp rx=0 tx=0).stop 2024-04-25T10:31:11.468 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:31:11.464+0000 7fdde2fce640 1 -- 172.21.15.37:0/2265474412 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fdddc165140 msgr2=0x7fdddc0cc840 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:31:11.468 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:31:11.464+0000 7fdde2fce640 1 --2- 172.21.15.37:0/2265474412 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fdddc165140 0x7fdddc0cc840 secure :-1 s=READY pgs=746 cs=0 l=1 rev1=1 crypto rx=0x7fddc400e7f0 tx=0x7fddc400ebf0 comp rx=0 tx=0).stop 2024-04-25T10:31:11.468 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:31:11.464+0000 7fdde156b640 1 -- 172.21.15.37:0/2265474412 reap_dead start 2024-04-25T10:31:11.468 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:31:11.464+0000 7fdde2fce640 1 -- 172.21.15.37:0/2265474412 shutdown_connections 2024-04-25T10:31:11.468 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:31:11.464+0000 7fdde2fce640 1 -- 172.21.15.37:0/2265474412 >> 172.21.15.37:0/2265474412 conn(0x7fdddc0d03e0 msgr2=0x7fdddc06e340 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:31:11.468 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:31:11.464+0000 7fdde2fce640 1 -- 172.21.15.37:0/2265474412 shutdown_connections 2024-04-25T10:31:11.468 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:31:11.464+0000 7fdde2fce640 1 -- 172.21.15.37:0/2265474412 wait complete. 2024-04-25T10:31:11.478 DEBUG:teuthology.orchestra.run.smithi037:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd deep-scrub 7 2024-04-25T10:31:11.590 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:31:11.588+0000 7fe57a6b6640 1 Processor -- start 2024-04-25T10:31:11.590 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:31:11.588+0000 7fe57a6b6640 1 -- start start 2024-04-25T10:31:11.593 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:31:11.589+0000 7fe57a6b6640 1 --2- >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7fe5740b4bb0 0x7fe5741646b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:31:11.593 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:31:11.589+0000 7fe57a6b6640 1 --2- >> v2:172.21.15.203:3300/0 conn(0x7fe5740d4560 0x7fe5740d4970 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:31:11.593 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:31:11.589+0000 7fe57a6b6640 1 -- --> v1:172.21.15.37:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fe574161190 con 0x7fe5740b7600 2024-04-25T10:31:11.593 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:31:11.589+0000 7fe57a6b6640 1 -- --> v2:172.21.15.203:3300/0 -- mon_getmap magic: 0 -- 0x7fe574161470 con 0x7fe5740d4560 2024-04-25T10:31:11.593 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:31:11.589+0000 7fe57a6b6640 1 -- --> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] -- mon_getmap magic: 0 -- 0x7fe574163eb0 con 0x7fe5740b4bb0 2024-04-25T10:31:11.594 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:31:11.589+0000 7fe5737fe640 1 --1- >> v1:172.21.15.37:6789/0 conn(0x7fe5740b7600 0x7fe5740b7a10 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.37:6789/0 says I am v1:172.21.15.37:56428/0 (socket says 172.21.15.37:56428) 2024-04-25T10:31:11.594 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:31:11.589+0000 7fe5737fe640 1 -- 172.21.15.37:0/4235890824 learned_addr learned my addr 172.21.15.37:0/4235890824 (peer_addr_for_me v1:172.21.15.37:0/0) 2024-04-25T10:31:11.594 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:31:11.590+0000 7fe573fff640 1 --2- 172.21.15.37:0/4235890824 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7fe5740b4bb0 0x7fe5741646b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:31:11.594 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:31:11.590+0000 7fe578c53640 1 --2- 172.21.15.37:0/4235890824 >> v2:172.21.15.203:3300/0 conn(0x7fe5740d4560 0x7fe5740d4970 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:31:11.594 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:31:11.590+0000 7fe572ffd640 1 -- 172.21.15.37:0/4235890824 <== mon.0 v1:172.21.15.37:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 3144681309 0 0) 0x7fe574161190 con 0x7fe5740b7600 2024-04-25T10:31:11.594 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:31:11.590+0000 7fe572ffd640 1 -- 172.21.15.37:0/4235890824 --> v1:172.21.15.37:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7fe55c003520 con 0x7fe5740b7600 2024-04-25T10:31:11.594 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:31:11.590+0000 7fe573fff640 1 -- 172.21.15.37:0/4235890824 >> v2:172.21.15.203:3300/0 conn(0x7fe5740d4560 msgr2=0x7fe5740d4970 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:31:11.594 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:31:11.590+0000 7fe573fff640 1 --2- 172.21.15.37:0/4235890824 >> v2:172.21.15.203:3300/0 conn(0x7fe5740d4560 0x7fe5740d4970 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:31:11.594 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:31:11.590+0000 7fe573fff640 1 -- 172.21.15.37:0/4235890824 >> v1:172.21.15.37:6789/0 conn(0x7fe5740b7600 legacy=0x7fe5740b7a10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:31:11.595 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:31:11.590+0000 7fe573fff640 1 -- 172.21.15.37:0/4235890824 --> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fe574164190 con 0x7fe5740b4bb0 2024-04-25T10:31:11.595 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:31:11.591+0000 7fe573fff640 1 --2- 172.21.15.37:0/4235890824 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7fe5740b4bb0 0x7fe5741646b0 secure :-1 s=READY pgs=66 cs=0 l=1 rev1=1 crypto rx=0x7fe5680097a0 tx=0x7fe56802a180 comp rx=0 tx=0).ready entity=mon.2 client_cookie=c9b2bd49448a717b server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:31:11.595 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:31:11.591+0000 7fe572ffd640 1 -- 172.21.15.37:0/4235890824 <== mon.2 v2:172.21.15.203:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe568037070 con 0x7fe5740b4bb0 2024-04-25T10:31:11.595 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:31:11.591+0000 7fe572ffd640 1 -- 172.21.15.37:0/4235890824 <== mon.2 v2:172.21.15.203:3301/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fe568032440 con 0x7fe5740b4bb0 2024-04-25T10:31:11.595 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:31:11.591+0000 7fe572ffd640 1 -- 172.21.15.37:0/4235890824 <== mon.2 v2:172.21.15.203:3301/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe5680327f0 con 0x7fe5740b4bb0 2024-04-25T10:31:11.598 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:31:11.592+0000 7fe57a6b6640 1 -- 172.21.15.37:0/4235890824 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7fe5740b4bb0 msgr2=0x7fe5741646b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:31:11.598 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:31:11.592+0000 7fe57a6b6640 1 --2- 172.21.15.37:0/4235890824 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7fe5740b4bb0 0x7fe5741646b0 secure :-1 s=READY pgs=66 cs=0 l=1 rev1=1 crypto rx=0x7fe5680097a0 tx=0x7fe56802a180 comp rx=0 tx=0).stop 2024-04-25T10:31:11.598 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:31:11.592+0000 7fe57a6b6640 1 -- 172.21.15.37:0/4235890824 shutdown_connections 2024-04-25T10:31:11.598 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:31:11.592+0000 7fe57a6b6640 1 --2- 172.21.15.37:0/4235890824 >> v2:172.21.15.203:3300/0 conn(0x7fe5740d4560 0x7fe5740d4970 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:31:11.599 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:31:11.592+0000 7fe57a6b6640 1 --2- 172.21.15.37:0/4235890824 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7fe5740b4bb0 0x7fe5741646b0 unknown :-1 s=CLOSED pgs=66 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:31:11.599 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:31:11.592+0000 7fe57a6b6640 1 -- 172.21.15.37:0/4235890824 >> 172.21.15.37:0/4235890824 conn(0x7fe5740d27e0 msgr2=0x7fe5740d2bf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:31:11.599 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:31:11.592+0000 7fe57a6b6640 1 -- 172.21.15.37:0/4235890824 shutdown_connections 2024-04-25T10:31:11.599 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:31:11.592+0000 7fe57a6b6640 1 -- 172.21.15.37:0/4235890824 wait complete. 2024-04-25T10:31:11.599 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:31:11.593+0000 7fe57a6b6640 1 Processor -- start 2024-04-25T10:31:11.599 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:31:11.594+0000 7fe57a6b6640 1 -- start start 2024-04-25T10:31:11.599 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:31:11.595+0000 7fe57a6b6640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fe5740b4bb0 0x7fe5740cd5f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:31:11.599 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:31:11.595+0000 7fe57a6b6640 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fe5740b7600 0x7fe57414ef90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:31:11.599 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:31:11.595+0000 7fe57a6b6640 1 --2- >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7fe5740d4560 0x7fe57415f330 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:31:11.599 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:31:11.595+0000 7fe57a6b6640 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 -- 0x7fe5740d09e0 con 0x7fe5740b4bb0 2024-04-25T10:31:11.599 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:31:11.595+0000 7fe57a6b6640 1 -- --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_getmap magic: 0 -- 0x7fe574164bf0 con 0x7fe5740b7600 2024-04-25T10:31:11.599 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:31:11.595+0000 7fe57a6b6640 1 -- --> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] -- mon_getmap magic: 0 -- 0x7fe5740d0b60 con 0x7fe5740d4560 2024-04-25T10:31:11.599 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:31:11.595+0000 7fe573fff640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fe5740b4bb0 0x7fe5740cd5f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:31:11.599 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:31:11.595+0000 7fe573fff640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fe5740b4bb0 0x7fe5740cd5f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.37:3300/0 says I am v2:172.21.15.37:59668/0 (socket says 172.21.15.37:59668) 2024-04-25T10:31:11.599 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:31:11.595+0000 7fe573fff640 1 -- 172.21.15.37:0/1780443258 learned_addr learned my addr 172.21.15.37:0/1780443258 (peer_addr_for_me v2:172.21.15.37:0/0) 2024-04-25T10:31:11.600 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:31:11.595+0000 7fe578c53640 1 --2- 172.21.15.37:0/1780443258 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7fe5740d4560 0x7fe57415f330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:31:11.600 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:31:11.595+0000 7fe5737fe640 1 --2- 172.21.15.37:0/1780443258 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fe5740b7600 0x7fe57414ef90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:31:11.600 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:31:11.595+0000 7fe573fff640 1 -- 172.21.15.37:0/1780443258 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7fe5740d4560 msgr2=0x7fe57415f330 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:31:11.600 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:31:11.595+0000 7fe573fff640 1 --2- 172.21.15.37:0/1780443258 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7fe5740d4560 0x7fe57415f330 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:31:11.600 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:31:11.595+0000 7fe573fff640 1 -- 172.21.15.37:0/1780443258 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fe5740b7600 msgr2=0x7fe57414ef90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:31:11.600 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:31:11.595+0000 7fe573fff640 1 --2- 172.21.15.37:0/1780443258 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fe5740b7600 0x7fe57414ef90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:31:11.600 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:31:11.595+0000 7fe573fff640 1 -- 172.21.15.37:0/1780443258 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fe5740cdb30 con 0x7fe5740b4bb0 2024-04-25T10:31:11.600 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:31:11.595+0000 7fe578c53640 1 --2- 172.21.15.37:0/1780443258 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7fe5740d4560 0x7fe57415f330 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T10:31:11.600 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:31:11.596+0000 7fe573fff640 1 --2- 172.21.15.37:0/1780443258 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fe5740b4bb0 0x7fe5740cd5f0 secure :-1 s=READY pgs=748 cs=0 l=1 rev1=1 crypto rx=0x7fe56802a1c0 tx=0x7fe5680046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:31:11.600 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:31:11.596+0000 7fe570ff9640 1 -- 172.21.15.37:0/1780443258 <== mon.0 v2:172.21.15.37:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe568037040 con 0x7fe5740b4bb0 2024-04-25T10:31:11.600 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:31:11.596+0000 7fe570ff9640 1 -- 172.21.15.37:0/1780443258 <== mon.0 v2:172.21.15.37:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fe568042070 con 0x7fe5740b4bb0 2024-04-25T10:31:11.600 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:31:11.596+0000 7fe57a6b6640 1 -- 172.21.15.37:0/1780443258 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fe57414f4d0 con 0x7fe5740b4bb0 2024-04-25T10:31:11.600 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:31:11.596+0000 7fe570ff9640 1 -- 172.21.15.37:0/1780443258 <== mon.0 v2:172.21.15.37:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe56803b930 con 0x7fe5740b4bb0 2024-04-25T10:31:11.600 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:31:11.596+0000 7fe57a6b6640 1 -- 172.21.15.37:0/1780443258 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fe5740ce300 con 0x7fe5740b4bb0 2024-04-25T10:31:11.601 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:31:11.597+0000 7fe57a6b6640 1 -- 172.21.15.37:0/1780443258 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fe57406a140 con 0x7fe5740b4bb0 2024-04-25T10:31:11.602 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:31:11.598+0000 7fe570ff9640 1 -- 172.21.15.37:0/1780443258 <== mon.0 v2:172.21.15.37:3300/0 4 ==== mgrmap(e 5) ==== 96782+0+0 (secure 0 0 0) 0x7fe56803bad0 con 0x7fe5740b4bb0 2024-04-25T10:31:11.602 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:31:11.598+0000 7fe570ff9640 1 --2- 172.21.15.37:0/1780443258 >> [v2:172.21.15.203:6832/2418205237,v1:172.21.15.203:6833/2418205237] conn(0x7fe5600744b0 0x7fe5600769c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:31:11.602 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:31:11.598+0000 7fe570ff9640 1 -- 172.21.15.37:0/1780443258 <== mon.0 v2:172.21.15.37:3300/0 5 ==== osd_map(50..50 src has 1..50) ==== 5029+0+0 (secure 0 0 0) 0x7fe5680b5860 con 0x7fe5740b4bb0 2024-04-25T10:31:11.602 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:31:11.598+0000 7fe5737fe640 1 --2- 172.21.15.37:0/1780443258 >> [v2:172.21.15.203:6832/2418205237,v1:172.21.15.203:6833/2418205237] conn(0x7fe5600744b0 0x7fe5600769c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:31:11.602 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:31:11.599+0000 7fe5737fe640 1 --2- 172.21.15.37:0/1780443258 >> [v2:172.21.15.203:6832/2418205237,v1:172.21.15.203:6833/2418205237] conn(0x7fe5600744b0 0x7fe5600769c0 secure :-1 s=READY pgs=336 cs=0 l=1 rev1=1 crypto rx=0x7fe574160500 tx=0x7fe564009040 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:31:11.604 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:31:11.603+0000 7fe570ff9640 1 -- 172.21.15.37:0/1780443258 <== mon.0 v2:172.21.15.37:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187246 (secure 0 0 0) 0x7fe568080290 con 0x7fe5740b4bb0 2024-04-25T10:31:11.792 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:31:11.790+0000 7fe57a6b6640 1 -- 172.21.15.37:0/1780443258 --> [v2:172.21.15.203:6832/2418205237,v1:172.21.15.203:6833/2418205237] -- mgr_command(tid 0: {"prefix": "osd deep-scrub", "who": "7", "target": ["mon-mgr", ""]}) -- 0x7fe5740d3ff0 con 0x7fe5600744b0 2024-04-25T10:31:11.792 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:31:11.790+0000 7fe570ff9640 1 -- 172.21.15.37:0/1780443258 <== mgr.4100 v2:172.21.15.203:6832/2418205237 1 ==== mgr_command_reply(tid 0: 0 instructed osd(s) 7 to deep-scrub) ==== 41+0+0 (secure 0 0 0) 0x7fe5740d3ff0 con 0x7fe5600744b0 2024-04-25T10:31:11.792 INFO:teuthology.orchestra.run.smithi037.stderr:instructed osd(s) 7 to deep-scrub 2024-04-25T10:31:11.795 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:31:11.793+0000 7fe57a6b6640 1 -- 172.21.15.37:0/1780443258 >> [v2:172.21.15.203:6832/2418205237,v1:172.21.15.203:6833/2418205237] conn(0x7fe5600744b0 msgr2=0x7fe5600769c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:31:11.795 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:31:11.793+0000 7fe57a6b6640 1 --2- 172.21.15.37:0/1780443258 >> [v2:172.21.15.203:6832/2418205237,v1:172.21.15.203:6833/2418205237] conn(0x7fe5600744b0 0x7fe5600769c0 secure :-1 s=READY pgs=336 cs=0 l=1 rev1=1 crypto rx=0x7fe574160500 tx=0x7fe564009040 comp rx=0 tx=0).stop 2024-04-25T10:31:11.795 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:31:11.793+0000 7fe57a6b6640 1 -- 172.21.15.37:0/1780443258 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fe5740b4bb0 msgr2=0x7fe5740cd5f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:31:11.795 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:31:11.793+0000 7fe57a6b6640 1 --2- 172.21.15.37:0/1780443258 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fe5740b4bb0 0x7fe5740cd5f0 secure :-1 s=READY pgs=748 cs=0 l=1 rev1=1 crypto rx=0x7fe56802a1c0 tx=0x7fe5680046c0 comp rx=0 tx=0).stop 2024-04-25T10:31:11.795 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:31:11.793+0000 7fe57a6b6640 1 -- 172.21.15.37:0/1780443258 shutdown_connections 2024-04-25T10:31:11.796 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:31:11.793+0000 7fe57a6b6640 1 --2- 172.21.15.37:0/1780443258 >> [v2:172.21.15.203:6832/2418205237,v1:172.21.15.203:6833/2418205237] conn(0x7fe5600744b0 0x7fe5600769c0 unknown :-1 s=CLOSED pgs=336 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:31:11.796 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:31:11.793+0000 7fe57a6b6640 1 --2- 172.21.15.37:0/1780443258 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7fe5740d4560 0x7fe57415f330 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:31:11.796 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:31:11.793+0000 7fe57a6b6640 1 --2- 172.21.15.37:0/1780443258 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fe5740b7600 0x7fe57414ef90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:31:11.796 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:31:11.793+0000 7fe57a6b6640 1 --2- 172.21.15.37:0/1780443258 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fe5740b4bb0 0x7fe5740cd5f0 unknown :-1 s=CLOSED pgs=748 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:31:11.796 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:31:11.793+0000 7fe57a6b6640 1 -- 172.21.15.37:0/1780443258 >> 172.21.15.37:0/1780443258 conn(0x7fe5740d27e0 msgr2=0x7fe5741608e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:31:11.796 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:31:11.793+0000 7fe57a6b6640 1 -- 172.21.15.37:0/1780443258 shutdown_connections 2024-04-25T10:31:11.796 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:31:11.793+0000 7fe57a6b6640 1 -- 172.21.15.37:0/1780443258 wait complete. 2024-04-25T10:31:11.806 DEBUG:teuthology.orchestra.run.smithi037:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-04-25T10:31:11.918 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:31:11.916+0000 7f9de796b640 1 Processor -- start 2024-04-25T10:31:11.918 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:31:11.916+0000 7f9de796b640 1 -- start start 2024-04-25T10:31:11.921 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:31:11.917+0000 7f9de796b640 1 --2- >> v2:172.21.15.203:3300/0 conn(0x7f9de00d1ec0 0x7f9de00d22d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:31:11.921 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:31:11.917+0000 7f9de796b640 1 --2- >> v2:172.21.15.37:3300/0 conn(0x7f9de00d2810 0x7f9de00b3c80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:31:11.921 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:31:11.917+0000 7f9de796b640 1 -- --> v1:172.21.15.203:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f9de009ce70 con 0x7f9de006e260 2024-04-25T10:31:11.921 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:31:11.917+0000 7f9de796b640 1 -- --> v2:172.21.15.37:3300/0 -- mon_getmap magic: 0 -- 0x7f9de00d3370 con 0x7f9de00d2810 2024-04-25T10:31:11.921 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:31:11.917+0000 7f9de796b640 1 -- --> v2:172.21.15.203:3300/0 -- mon_getmap magic: 0 -- 0x7f9de00d3090 con 0x7f9de00d1ec0 2024-04-25T10:31:11.921 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:31:11.918+0000 7f9de5f08640 1 --2- >> v2:172.21.15.37:3300/0 conn(0x7f9de00d2810 0x7f9de00b3c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:31:11.921 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:31:11.918+0000 7f9de5f08640 1 --2- >> v2:172.21.15.37:3300/0 conn(0x7f9de00d2810 0x7f9de00b3c80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.37:3300/0 says I am v2:172.21.15.37:59676/0 (socket says 172.21.15.37:59676) 2024-04-25T10:31:11.921 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:31:11.918+0000 7f9de5f08640 1 -- 172.21.15.37:0/446851821 learned_addr learned my addr 172.21.15.37:0/446851821 (peer_addr_for_me v2:172.21.15.37:0/0) 2024-04-25T10:31:11.921 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:31:11.918+0000 7f9de4f06640 1 --2- 172.21.15.37:0/446851821 >> v2:172.21.15.203:3300/0 conn(0x7f9de00d1ec0 0x7f9de00d22d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:31:11.921 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:31:11.918+0000 7f9de5f08640 1 -- 172.21.15.37:0/446851821 >> v2:172.21.15.203:3300/0 conn(0x7f9de00d1ec0 msgr2=0x7f9de00d22d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:31:11.922 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:31:11.918+0000 7f9de5f08640 1 --2- 172.21.15.37:0/446851821 >> v2:172.21.15.203:3300/0 conn(0x7f9de00d1ec0 0x7f9de00d22d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:31:11.922 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:31:11.918+0000 7f9de5f08640 1 -- 172.21.15.37:0/446851821 >> v1:172.21.15.203:6789/0 conn(0x7f9de006e260 legacy=0x7f9de006e670 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:31:11.922 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:31:11.918+0000 7f9de5f08640 1 -- 172.21.15.37:0/446851821 --> v2:172.21.15.37:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f9de009d150 con 0x7f9de00d2810 2024-04-25T10:31:11.922 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:31:11.918+0000 7f9de4f06640 1 --2- 172.21.15.37:0/446851821 >> v2:172.21.15.203:3300/0 conn(0x7f9de00d1ec0 0x7f9de00d22d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T10:31:11.922 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:31:11.919+0000 7f9de5f08640 1 --2- 172.21.15.37:0/446851821 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f9de00d2810 0x7f9de00b3c80 secure :-1 s=READY pgs=749 cs=0 l=1 rev1=1 crypto rx=0x7f9de0074b20 tx=0x7f9dd802a6a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=86cc528a054ecfb4 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:31:11.922 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:31:11.919+0000 7f9dd7fff640 1 -- 172.21.15.37:0/446851821 <== mon.0 v2:172.21.15.37:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9dd8036070 con 0x7f9de00d2810 2024-04-25T10:31:11.922 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:31:11.919+0000 7f9dd7fff640 1 -- 172.21.15.37:0/446851821 <== mon.0 v2:172.21.15.37:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f9dd8031600 con 0x7f9de00d2810 2024-04-25T10:31:11.922 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:31:11.919+0000 7f9dd7fff640 1 -- 172.21.15.37:0/446851821 <== mon.0 v2:172.21.15.37:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9dd80319d0 con 0x7f9de00d2810 2024-04-25T10:31:11.923 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:31:11.919+0000 7f9de796b640 1 -- 172.21.15.37:0/446851821 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f9de00d2810 msgr2=0x7f9de00b3c80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:31:11.923 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:31:11.919+0000 7f9de796b640 1 --2- 172.21.15.37:0/446851821 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f9de00d2810 0x7f9de00b3c80 secure :-1 s=READY pgs=749 cs=0 l=1 rev1=1 crypto rx=0x7f9de0074b20 tx=0x7f9dd802a6a0 comp rx=0 tx=0).stop 2024-04-25T10:31:11.923 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:31:11.920+0000 7f9de796b640 1 -- 172.21.15.37:0/446851821 shutdown_connections 2024-04-25T10:31:11.924 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:31:11.920+0000 7f9de796b640 1 --2- 172.21.15.37:0/446851821 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f9de00d2810 0x7f9de00b3c80 unknown :-1 s=CLOSED pgs=749 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:31:11.924 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:31:11.920+0000 7f9de796b640 1 --2- 172.21.15.37:0/446851821 >> v2:172.21.15.203:3300/0 conn(0x7f9de00d1ec0 0x7f9de00d22d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:31:11.924 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:31:11.920+0000 7f9de796b640 1 -- 172.21.15.37:0/446851821 >> 172.21.15.37:0/446851821 conn(0x7f9de00d0390 msgr2=0x7f9de00d07a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:31:11.924 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:31:11.920+0000 7f9de796b640 1 -- 172.21.15.37:0/446851821 shutdown_connections 2024-04-25T10:31:11.924 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:31:11.920+0000 7f9de796b640 1 -- 172.21.15.37:0/446851821 wait complete. 2024-04-25T10:31:11.924 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:31:11.921+0000 7f9de796b640 1 Processor -- start 2024-04-25T10:31:11.924 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:31:11.922+0000 7f9de796b640 1 -- start start 2024-04-25T10:31:11.926 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:31:11.923+0000 7f9de796b640 1 --2- >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f9de006e260 0x7f9de009f300 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:31:11.926 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:31:11.923+0000 7f9de796b640 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f9de00d1ec0 0x7f9de01dbea0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:31:11.926 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:31:11.923+0000 7f9de796b640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f9de00d2810 0x7f9de01ead00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:31:11.926 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:31:11.923+0000 7f9de796b640 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 -- 0x7f9de00b9000 con 0x7f9de00d2810 2024-04-25T10:31:11.926 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:31:11.923+0000 7f9de796b640 1 -- --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_getmap magic: 0 -- 0x7f9de006d830 con 0x7f9de00d1ec0 2024-04-25T10:31:11.927 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:31:11.923+0000 7f9de796b640 1 -- --> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] -- mon_getmap magic: 0 -- 0x7f9de00b9180 con 0x7f9de006e260 2024-04-25T10:31:11.927 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:31:11.923+0000 7f9de5f08640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f9de00d2810 0x7f9de01ead00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:31:11.927 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:31:11.923+0000 7f9de5f08640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f9de00d2810 0x7f9de01ead00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.37:3300/0 says I am v2:172.21.15.37:59690/0 (socket says 172.21.15.37:59690) 2024-04-25T10:31:11.927 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:31:11.923+0000 7f9de5f08640 1 -- 172.21.15.37:0/884466930 learned_addr learned my addr 172.21.15.37:0/884466930 (peer_addr_for_me v2:172.21.15.37:0/0) 2024-04-25T10:31:11.927 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:31:11.923+0000 7f9de4f06640 1 --2- 172.21.15.37:0/884466930 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f9de00d1ec0 0x7f9de01dbea0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:31:11.927 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:31:11.923+0000 7f9de5707640 1 --2- 172.21.15.37:0/884466930 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f9de006e260 0x7f9de009f300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:31:11.927 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:31:11.923+0000 7f9de5f08640 1 -- 172.21.15.37:0/884466930 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f9de006e260 msgr2=0x7f9de009f300 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:31:11.927 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:31:11.923+0000 7f9de5f08640 1 --2- 172.21.15.37:0/884466930 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f9de006e260 0x7f9de009f300 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:31:11.927 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:31:11.923+0000 7f9de5f08640 1 -- 172.21.15.37:0/884466930 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f9de00d1ec0 msgr2=0x7f9de01dbea0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:31:11.927 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:31:11.923+0000 7f9de5f08640 1 --2- 172.21.15.37:0/884466930 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f9de00d1ec0 0x7f9de01dbea0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:31:11.927 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:31:11.923+0000 7f9de5f08640 1 -- 172.21.15.37:0/884466930 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f9de009fda0 con 0x7f9de00d2810 2024-04-25T10:31:11.927 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:31:11.924+0000 7f9de5f08640 1 --2- 172.21.15.37:0/884466930 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f9de00d2810 0x7f9de01ead00 secure :-1 s=READY pgs=750 cs=0 l=1 rev1=1 crypto rx=0x7f9dd8034040 tx=0x7f9dd8004a40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:31:11.927 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:31:11.924+0000 7f9dd5ffb640 1 -- 172.21.15.37:0/884466930 <== mon.0 v2:172.21.15.37:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9dd8036050 con 0x7f9de00d2810 2024-04-25T10:31:11.927 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:31:11.924+0000 7f9dd5ffb640 1 -- 172.21.15.37:0/884466930 <== mon.0 v2:172.21.15.37:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f9dd8042070 con 0x7f9de00d2810 2024-04-25T10:31:11.928 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:31:11.924+0000 7f9de796b640 1 -- 172.21.15.37:0/884466930 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f9de00a0080 con 0x7f9de00d2810 2024-04-25T10:31:11.928 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:31:11.924+0000 7f9dd5ffb640 1 -- 172.21.15.37:0/884466930 <== mon.0 v2:172.21.15.37:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9dd8009510 con 0x7f9de00d2810 2024-04-25T10:31:11.928 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:31:11.924+0000 7f9de796b640 1 -- 172.21.15.37:0/884466930 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f9de00a0640 con 0x7f9de00d2810 2024-04-25T10:31:11.928 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:31:11.925+0000 7f9de796b640 1 -- 172.21.15.37:0/884466930 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f9de00d2c20 con 0x7f9de00d2810 2024-04-25T10:31:11.929 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:31:11.926+0000 7f9dd5ffb640 1 -- 172.21.15.37:0/884466930 <== mon.0 v2:172.21.15.37:3300/0 4 ==== mgrmap(e 5) ==== 96782+0+0 (secure 0 0 0) 0x7f9dd8007bd0 con 0x7f9de00d2810 2024-04-25T10:31:11.929 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:31:11.926+0000 7f9dd5ffb640 1 --2- 172.21.15.37:0/884466930 >> [v2:172.21.15.203:6832/2418205237,v1:172.21.15.203:6833/2418205237] conn(0x7f9dc40744b0 0x7f9dc40769c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:31:11.929 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:31:11.926+0000 7f9dd5ffb640 1 -- 172.21.15.37:0/884466930 <== mon.0 v2:172.21.15.37:3300/0 5 ==== osd_map(50..50 src has 1..50) ==== 5029+0+0 (secure 0 0 0) 0x7f9dd80b5d80 con 0x7f9de00d2810 2024-04-25T10:31:11.929 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:31:11.927+0000 7f9de5707640 1 --2- 172.21.15.37:0/884466930 >> [v2:172.21.15.203:6832/2418205237,v1:172.21.15.203:6833/2418205237] conn(0x7f9dc40744b0 0x7f9dc40769c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:31:11.929 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:31:11.927+0000 7f9de5707640 1 --2- 172.21.15.37:0/884466930 >> [v2:172.21.15.203:6832/2418205237,v1:172.21.15.203:6833/2418205237] conn(0x7f9dc40744b0 0x7f9dc40769c0 secure :-1 s=READY pgs=337 cs=0 l=1 rev1=1 crypto rx=0x7f9de00d1490 tx=0x7f9ddc009040 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:31:11.933 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:31:11.931+0000 7f9dd5ffb640 1 -- 172.21.15.37:0/884466930 <== mon.0 v2:172.21.15.37:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187246 (secure 0 0 0) 0x7f9de00d2c20 con 0x7f9de00d2810 2024-04-25T10:31:12.114 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:31:12.112+0000 7f9de796b640 1 -- 172.21.15.37:0/884466930 --> [v2:172.21.15.203:6832/2418205237,v1:172.21.15.203:6833/2418205237] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) -- 0x7f9de006a140 con 0x7f9dc40744b0 2024-04-25T10:31:12.115 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:31:12.113+0000 7f9dd5ffb640 1 -- 172.21.15.37:0/884466930 <== mgr.4100 v2:172.21.15.203:6832/2418205237 1 ==== mgr_command_reply(tid 0: 0 dumped all) ==== 18+0+13431 (secure 0 0 0) 0x7f9de006a140 con 0x7f9dc40744b0 2024-04-25T10:31:12.116 INFO:teuthology.orchestra.run.smithi037.stdout: 2024-04-25T10:31:12.116 INFO:teuthology.orchestra.run.smithi037.stderr:dumped all 2024-04-25T10:31:12.118 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:31:12.115+0000 7f9de796b640 1 -- 172.21.15.37:0/884466930 >> [v2:172.21.15.203:6832/2418205237,v1:172.21.15.203:6833/2418205237] conn(0x7f9dc40744b0 msgr2=0x7f9dc40769c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:31:12.118 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:31:12.115+0000 7f9de796b640 1 --2- 172.21.15.37:0/884466930 >> [v2:172.21.15.203:6832/2418205237,v1:172.21.15.203:6833/2418205237] conn(0x7f9dc40744b0 0x7f9dc40769c0 secure :-1 s=READY pgs=337 cs=0 l=1 rev1=1 crypto rx=0x7f9de00d1490 tx=0x7f9ddc009040 comp rx=0 tx=0).stop 2024-04-25T10:31:12.118 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:31:12.115+0000 7f9de796b640 1 -- 172.21.15.37:0/884466930 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f9de00d2810 msgr2=0x7f9de01ead00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:31:12.118 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:31:12.115+0000 7f9de796b640 1 --2- 172.21.15.37:0/884466930 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f9de00d2810 0x7f9de01ead00 secure :-1 s=READY pgs=750 cs=0 l=1 rev1=1 crypto rx=0x7f9dd8034040 tx=0x7f9dd8004a40 comp rx=0 tx=0).stop 2024-04-25T10:31:12.118 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:31:12.115+0000 7f9de796b640 1 -- 172.21.15.37:0/884466930 shutdown_connections 2024-04-25T10:31:12.118 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:31:12.115+0000 7f9de796b640 1 --2- 172.21.15.37:0/884466930 >> [v2:172.21.15.203:6832/2418205237,v1:172.21.15.203:6833/2418205237] conn(0x7f9dc40744b0 0x7f9dc40769c0 unknown :-1 s=CLOSED pgs=337 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:31:12.118 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:31:12.115+0000 7f9de796b640 1 --2- 172.21.15.37:0/884466930 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f9de00d2810 0x7f9de01ead00 unknown :-1 s=CLOSED pgs=750 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:31:12.118 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:31:12.115+0000 7f9de796b640 1 --2- 172.21.15.37:0/884466930 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f9de00d1ec0 0x7f9de01dbea0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:31:12.118 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:31:12.115+0000 7f9de796b640 1 --2- 172.21.15.37:0/884466930 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f9de006e260 0x7f9de009f300 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:31:12.118 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:31:12.115+0000 7f9de796b640 1 -- 172.21.15.37:0/884466930 >> 172.21.15.37:0/884466930 conn(0x7f9de00d0390 msgr2=0x7f9de00b8770 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:31:12.118 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:31:12.115+0000 7f9de796b640 1 -- 172.21.15.37:0/884466930 shutdown_connections 2024-04-25T10:31:12.119 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:31:12.115+0000 7f9de796b640 1 -- 172.21.15.37:0/884466930 wait complete. 2024-04-25T10:31:12.128 INFO:teuthology.orchestra.run.smithi037.stdout:{"pg_ready":true,"pg_map":{"version":112,"stamp":"2024-04-25T10:31:10.217287+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":66,"num_read_kb":55,"num_write":101,"num_write_kb":1628,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":72,"ondisk_log_size":72,"up":2,"acting":2,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":18,"num_osds":8,"num_per_pool_osds":8,"num_per_pool_omap_osds":2,"kb":754974720,"kb_used":226068,"kb_used_data":5736,"kb_used_omap":12,"kb_used_meta":220211,"kb_avail":754748652,"statfs":{"total":773094113280,"available":772862619648,"internally_reserved":0,"allocated":5873664,"data_stored":3144888,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":12752,"internal_metadata":225496624},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":8,"apply_latency_ms":8,"commit_latency_ns":8000000,"apply_latency_ns":8000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":-6291456,"num_objects":-2,"num_object_clones":0,"num_object_copies":-4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":-2,"num_whiteouts":0,"num_read":-16,"num_read_kb":-9,"num_write":-8,"num_write_kb":-6144,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"8.354154"},"pg_stats":[{"pgid":"1.0","version":"18'72","reported_seq":160,"reported_epoch":50,"state":"active+clean","last_fresh":"2024-04-25T10:31:03.120107+0000","last_change":"2024-04-25T10:28:57.972696+0000","last_active":"2024-04-25T10:31:03.120107+0000","last_peered":"2024-04-25T10:31:03.120107+0000","last_clean":"2024-04-25T10:31:03.120107+0000","last_became_active":"2024-04-25T10:28:57.972373+0000","last_became_peered":"2024-04-25T10:28:57.972373+0000","last_unstale":"2024-04-25T10:31:03.120107+0000","last_undegraded":"2024-04-25T10:31:03.120107+0000","last_fullsized":"2024-04-25T10:31:03.120107+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:28:56.959439+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:28:56.959439+0000","last_clean_scrub_stamp":"2024-04-25T10:28:56.959439+0000","objects_scrubbed":0,"log_size":72,"log_dups_size":0,"ondisk_log_size":72,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T17:34:34.057408+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":66,"num_read_kb":55,"num_write":101,"num_write_kb":1628,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]}],"pool_stats":[{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":66,"num_read_kb":55,"num_write":101,"num_write_kb":1628,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1187840,"data_stored":1180736,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":72,"ondisk_log_size":72,"up":2,"acting":2,"num_store_stats":2}],"osd_stats":[{"osd":7,"up_from":15,"seq":64424509470,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":0,"kb":94371840,"kb_used":27592,"kb_used_data":572,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344248,"statfs":{"total":96636764160,"available":96608509952,"internally_reserved":0,"allocated":585728,"data_stored":245519,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1590,"internal_metadata":27654602},"hb_peers":[0,1,2,3,4,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":6,"up_from":15,"seq":64424509470,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":0,"kb":94371840,"kb_used":27592,"kb_used_data":572,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344248,"statfs":{"total":96636764160,"available":96608509952,"internally_reserved":0,"allocated":585728,"data_stored":245519,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":27654598},"hb_peers":[0,1,2,3,4,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]},{"osd":5,"up_from":15,"seq":64424509472,"num_pgs":1,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":28172,"kb_used_data":1152,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94343668,"statfs":{"total":96636764160,"available":96607916032,"internally_reserved":0,"allocated":1179648,"data_stored":835887,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":3,"up_from":15,"seq":64424509471,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":0,"kb":94371840,"kb_used":27592,"kb_used_data":572,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344248,"statfs":{"total":96636764160,"available":96608509952,"internally_reserved":0,"allocated":585728,"data_stored":245519,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1593,"internal_metadata":27654599},"hb_peers":[0,1,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]},{"osd":2,"up_from":15,"seq":64424509471,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":0,"kb":94371840,"kb_used":27596,"kb_used_data":572,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344244,"statfs":{"total":96636764160,"available":96608505856,"internally_reserved":0,"allocated":585728,"data_stored":245519,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]},{"osd":1,"up_from":15,"seq":64424509469,"num_pgs":1,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":28176,"kb_used_data":1152,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94343664,"statfs":{"total":96636764160,"available":96607911936,"internally_reserved":0,"allocated":1179648,"data_stored":835887,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]},{"osd":0,"up_from":15,"seq":64424509470,"num_pgs":16,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":0,"kb":94371840,"kb_used":27592,"kb_used_data":572,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344248,"statfs":{"total":96636764160,"available":96608509952,"internally_reserved":0,"allocated":585728,"data_stored":245519,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[1,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]},{"osd":4,"up_from":13,"seq":55834574878,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":0,"kb":94371840,"kb_used":31756,"kb_used_data":572,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94340084,"statfs":{"total":96636764160,"available":96604246016,"internally_reserved":0,"allocated":585728,"data_stored":245519,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[0,1,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-04-25T10:31:12.133 INFO:tasks.ceph:pgid 1.0 last_scrub_stamp 2024-04-25T10:28:56.959439+0000 time.struct_time(tm_year=2024, tm_mon=4, tm_mday=25, tm_hour=10, tm_min=28, tm_sec=56, tm_wday=3, tm_yday=116, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=4, tm_mday=25, tm_hour=10, tm_min=31, tm_sec=6, tm_wday=3, tm_yday=116, tm_isdst=0) 2024-04-25T10:31:12.133 INFO:tasks.ceph:Still waiting for all pgs to be scrubbed. 2024-04-25T10:31:32.134 DEBUG:teuthology.orchestra.run.smithi037:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-04-25T10:31:32.261 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:31:32.259+0000 7f72c2af2640 1 Processor -- start 2024-04-25T10:31:32.261 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:31:32.259+0000 7f72c2af2640 1 -- start start 2024-04-25T10:31:32.263 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:31:32.260+0000 7f72c2af2640 1 --2- >> v2:172.21.15.203:3300/0 conn(0x7f72bc140040 0x7f72bc15de20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:31:32.263 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:31:32.260+0000 7f72c2af2640 1 --2- >> v2:172.21.15.37:3300/0 conn(0x7f72bc160a00 0x7f72bc160e10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:31:32.263 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:31:32.260+0000 7f72c2af2640 1 -- --> v1:172.21.15.203:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f72bc161430 con 0x7f72bc073810 2024-04-25T10:31:32.263 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:31:32.260+0000 7f72c2af2640 1 -- --> v2:172.21.15.37:3300/0 -- mon_getmap magic: 0 -- 0x7f72bc15d990 con 0x7f72bc160a00 2024-04-25T10:31:32.263 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:31:32.260+0000 7f72c2af2640 1 -- --> v2:172.21.15.203:3300/0 -- mon_getmap magic: 0 -- 0x7f72bc15d3c0 con 0x7f72bc140040 2024-04-25T10:31:32.263 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:31:32.260+0000 7f72bbfff640 1 --2- >> v2:172.21.15.37:3300/0 conn(0x7f72bc160a00 0x7f72bc160e10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:31:32.263 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:31:32.260+0000 7f72bbfff640 1 --2- >> v2:172.21.15.37:3300/0 conn(0x7f72bc160a00 0x7f72bc160e10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.37:3300/0 says I am v2:172.21.15.37:46270/0 (socket says 172.21.15.37:46270) 2024-04-25T10:31:32.264 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:31:32.260+0000 7f72bbfff640 1 -- 172.21.15.37:0/2372114835 learned_addr learned my addr 172.21.15.37:0/2372114835 (peer_addr_for_me v2:172.21.15.37:0/0) 2024-04-25T10:31:32.264 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:31:32.261+0000 7f72c088e640 1 --2- 172.21.15.37:0/2372114835 >> v2:172.21.15.203:3300/0 conn(0x7f72bc140040 0x7f72bc15de20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:31:32.264 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:31:32.261+0000 7f72bbfff640 1 -- 172.21.15.37:0/2372114835 >> v2:172.21.15.203:3300/0 conn(0x7f72bc140040 msgr2=0x7f72bc15de20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:31:32.264 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:31:32.261+0000 7f72bbfff640 1 --2- 172.21.15.37:0/2372114835 >> v2:172.21.15.203:3300/0 conn(0x7f72bc140040 0x7f72bc15de20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:31:32.264 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:31:32.261+0000 7f72bbfff640 1 -- 172.21.15.37:0/2372114835 >> v1:172.21.15.203:6789/0 conn(0x7f72bc073810 legacy=0x7f72bc073c20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:31:32.264 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:31:32.261+0000 7f72bbfff640 1 -- 172.21.15.37:0/2372114835 --> v2:172.21.15.37:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f72bc161710 con 0x7f72bc160a00 2024-04-25T10:31:32.265 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:31:32.261+0000 7f72bbfff640 1 --2- 172.21.15.37:0/2372114835 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f72bc160a00 0x7f72bc160e10 secure :-1 s=READY pgs=751 cs=0 l=1 rev1=1 crypto rx=0x7f72bc070710 tx=0x7f72a402a230 comp rx=0 tx=0).ready entity=mon.0 client_cookie=6f7291add7abe0dc server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:31:32.265 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:31:32.262+0000 7f72bb7fe640 1 -- 172.21.15.37:0/2372114835 <== mon.0 v2:172.21.15.37:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f72a4037070 con 0x7f72bc160a00 2024-04-25T10:31:32.265 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:31:32.262+0000 7f72bb7fe640 1 -- 172.21.15.37:0/2372114835 <== mon.0 v2:172.21.15.37:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f72a4032440 con 0x7f72bc160a00 2024-04-25T10:31:32.265 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:31:32.262+0000 7f72bb7fe640 1 -- 172.21.15.37:0/2372114835 <== mon.0 v2:172.21.15.37:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f72a4032810 con 0x7f72bc160a00 2024-04-25T10:31:32.266 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:31:32.262+0000 7f72c2af2640 1 -- 172.21.15.37:0/2372114835 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f72bc160a00 msgr2=0x7f72bc160e10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:31:32.266 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:31:32.262+0000 7f72c2af2640 1 --2- 172.21.15.37:0/2372114835 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f72bc160a00 0x7f72bc160e10 secure :-1 s=READY pgs=751 cs=0 l=1 rev1=1 crypto rx=0x7f72bc070710 tx=0x7f72a402a230 comp rx=0 tx=0).stop 2024-04-25T10:31:32.266 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:31:32.262+0000 7f72c2af2640 1 -- 172.21.15.37:0/2372114835 shutdown_connections 2024-04-25T10:31:32.266 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:31:32.262+0000 7f72c2af2640 1 --2- 172.21.15.37:0/2372114835 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f72bc160a00 0x7f72bc160e10 unknown :-1 s=CLOSED pgs=751 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:31:32.266 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:31:32.262+0000 7f72c2af2640 1 --2- 172.21.15.37:0/2372114835 >> v2:172.21.15.203:3300/0 conn(0x7f72bc140040 0x7f72bc15de20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:31:32.266 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:31:32.262+0000 7f72c2af2640 1 -- 172.21.15.37:0/2372114835 >> 172.21.15.37:0/2372114835 conn(0x7f72bc0b3e20 msgr2=0x7f72bc0b4260 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:31:32.266 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:31:32.262+0000 7f72c2af2640 1 -- 172.21.15.37:0/2372114835 shutdown_connections 2024-04-25T10:31:32.266 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:31:32.262+0000 7f72c2af2640 1 -- 172.21.15.37:0/2372114835 wait complete. 2024-04-25T10:31:32.267 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:31:32.264+0000 7f72c2af2640 1 Processor -- start 2024-04-25T10:31:32.267 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:31:32.264+0000 7f72c2af2640 1 -- start start 2024-04-25T10:31:32.269 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:31:32.266+0000 7f72c2af2640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f72bc073810 0x7f72bc1f8390 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:31:32.269 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:31:32.266+0000 7f72c2af2640 1 --2- >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f72bc160a00 0x7f72bc153810 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:31:32.269 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:31:32.266+0000 7f72c2af2640 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f72bc157580 0x7f72bc141090 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:31:32.269 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:31:32.266+0000 7f72c2af2640 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 -- 0x7f72bc1427b0 con 0x7f72bc073810 2024-04-25T10:31:32.270 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:31:32.266+0000 7f72c2af2640 1 -- --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_getmap magic: 0 -- 0x7f72bc15e360 con 0x7f72bc157580 2024-04-25T10:31:32.270 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:31:32.266+0000 7f72c2af2640 1 -- --> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] -- mon_getmap magic: 0 -- 0x7f72bc142930 con 0x7f72bc160a00 2024-04-25T10:31:32.270 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:31:32.266+0000 7f72c088e640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f72bc073810 0x7f72bc1f8390 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:31:32.270 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:31:32.266+0000 7f72c088e640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f72bc073810 0x7f72bc1f8390 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.37:3300/0 says I am v2:172.21.15.37:46272/0 (socket says 172.21.15.37:46272) 2024-04-25T10:31:32.270 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:31:32.266+0000 7f72c088e640 1 -- 172.21.15.37:0/1396210448 learned_addr learned my addr 172.21.15.37:0/1396210448 (peer_addr_for_me v2:172.21.15.37:0/0) 2024-04-25T10:31:32.270 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:31:32.266+0000 7f72c108f640 1 --2- 172.21.15.37:0/1396210448 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f72bc157580 0x7f72bc141090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:31:32.270 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:31:32.266+0000 7f72bbfff640 1 --2- 172.21.15.37:0/1396210448 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f72bc160a00 0x7f72bc153810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:31:32.270 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:31:32.266+0000 7f72c088e640 1 -- 172.21.15.37:0/1396210448 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f72bc160a00 msgr2=0x7f72bc153810 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:31:32.270 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:31:32.266+0000 7f72c088e640 1 --2- 172.21.15.37:0/1396210448 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f72bc160a00 0x7f72bc153810 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:31:32.270 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:31:32.266+0000 7f72c088e640 1 -- 172.21.15.37:0/1396210448 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f72bc157580 msgr2=0x7f72bc141090 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:31:32.270 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:31:32.266+0000 7f72c088e640 1 --2- 172.21.15.37:0/1396210448 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f72bc157580 0x7f72bc141090 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:31:32.270 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:31:32.266+0000 7f72c088e640 1 -- 172.21.15.37:0/1396210448 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f72bc159320 con 0x7f72bc073810 2024-04-25T10:31:32.270 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:31:32.267+0000 7f72c088e640 1 --2- 172.21.15.37:0/1396210448 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f72bc073810 0x7f72bc1f8390 secure :-1 s=READY pgs=752 cs=0 l=1 rev1=1 crypto rx=0x7f72b4007e90 tx=0x7f72b400e4b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:31:32.271 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:31:32.267+0000 7f72b97fa640 1 -- 172.21.15.37:0/1396210448 <== mon.0 v2:172.21.15.37:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f72b401a070 con 0x7f72bc073810 2024-04-25T10:31:32.271 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:31:32.267+0000 7f72b97fa640 1 -- 172.21.15.37:0/1396210448 <== mon.0 v2:172.21.15.37:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f72b4003b90 con 0x7f72bc073810 2024-04-25T10:31:32.271 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:31:32.267+0000 7f72c2af2640 1 -- 172.21.15.37:0/1396210448 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f72bc159600 con 0x7f72bc073810 2024-04-25T10:31:32.271 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:31:32.267+0000 7f72b97fa640 1 -- 172.21.15.37:0/1396210448 <== mon.0 v2:172.21.15.37:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f72b400ee70 con 0x7f72bc073810 2024-04-25T10:31:32.271 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:31:32.267+0000 7f72c2af2640 1 -- 172.21.15.37:0/1396210448 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f72bc1f4420 con 0x7f72bc073810 2024-04-25T10:31:32.272 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:31:32.268+0000 7f72c2af2640 1 -- 172.21.15.37:0/1396210448 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f72bc1f4650 con 0x7f72bc073810 2024-04-25T10:31:32.272 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:31:32.269+0000 7f72b97fa640 1 -- 172.21.15.37:0/1396210448 <== mon.0 v2:172.21.15.37:3300/0 4 ==== mgrmap(e 5) ==== 96782+0+0 (secure 0 0 0) 0x7f72b401d070 con 0x7f72bc073810 2024-04-25T10:31:32.272 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:31:32.269+0000 7f72b97fa640 1 --2- 172.21.15.37:0/1396210448 >> [v2:172.21.15.203:6832/2418205237,v1:172.21.15.203:6833/2418205237] conn(0x7f729c0744e0 0x7f729c0769f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:31:32.272 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:31:32.269+0000 7f72b97fa640 1 -- 172.21.15.37:0/1396210448 <== mon.0 v2:172.21.15.37:3300/0 5 ==== osd_map(50..50 src has 1..50) ==== 5029+0+0 (secure 0 0 0) 0x7f72b4094fa0 con 0x7f72bc073810 2024-04-25T10:31:32.272 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:31:32.270+0000 7f72bbfff640 1 --2- 172.21.15.37:0/1396210448 >> [v2:172.21.15.203:6832/2418205237,v1:172.21.15.203:6833/2418205237] conn(0x7f729c0744e0 0x7f729c0769f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:31:32.272 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:31:32.270+0000 7f72bbfff640 1 --2- 172.21.15.37:0/1396210448 >> [v2:172.21.15.203:6832/2418205237,v1:172.21.15.203:6833/2418205237] conn(0x7f729c0744e0 0x7f729c0769f0 secure :-1 s=READY pgs=338 cs=0 l=1 rev1=1 crypto rx=0x7f72a4034000 tx=0x7f72a402e000 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:31:32.276 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:31:32.274+0000 7f72b97fa640 1 -- 172.21.15.37:0/1396210448 <== mon.0 v2:172.21.15.37:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187246 (secure 0 0 0) 0x7f72b405f4a0 con 0x7f72bc073810 2024-04-25T10:31:32.458 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:31:32.455+0000 7f72c2af2640 1 -- 172.21.15.37:0/1396210448 --> [v2:172.21.15.203:6832/2418205237,v1:172.21.15.203:6833/2418205237] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) -- 0x7f72bc156d50 con 0x7f729c0744e0 2024-04-25T10:31:32.459 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:31:32.457+0000 7f72b97fa640 1 -- 172.21.15.37:0/1396210448 <== mgr.4100 v2:172.21.15.203:6832/2418205237 1 ==== mgr_command_reply(tid 0: 0 dumped all) ==== 18+0+13344 (secure 0 0 0) 0x7f72a4045430 con 0x7f729c0744e0 2024-04-25T10:31:32.459 INFO:teuthology.orchestra.run.smithi037.stdout: 2024-04-25T10:31:32.459 INFO:teuthology.orchestra.run.smithi037.stderr:dumped all 2024-04-25T10:31:32.461 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:31:32.459+0000 7f72c2af2640 1 -- 172.21.15.37:0/1396210448 >> [v2:172.21.15.203:6832/2418205237,v1:172.21.15.203:6833/2418205237] conn(0x7f729c0744e0 msgr2=0x7f729c0769f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:31:32.461 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:31:32.459+0000 7f72c2af2640 1 --2- 172.21.15.37:0/1396210448 >> [v2:172.21.15.203:6832/2418205237,v1:172.21.15.203:6833/2418205237] conn(0x7f729c0744e0 0x7f729c0769f0 secure :-1 s=READY pgs=338 cs=0 l=1 rev1=1 crypto rx=0x7f72a4034000 tx=0x7f72a402e000 comp rx=0 tx=0).stop 2024-04-25T10:31:32.461 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:31:32.459+0000 7f72c2af2640 1 -- 172.21.15.37:0/1396210448 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f72bc073810 msgr2=0x7f72bc1f8390 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:31:32.461 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:31:32.459+0000 7f72c2af2640 1 --2- 172.21.15.37:0/1396210448 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f72bc073810 0x7f72bc1f8390 secure :-1 s=READY pgs=752 cs=0 l=1 rev1=1 crypto rx=0x7f72b4007e90 tx=0x7f72b400e4b0 comp rx=0 tx=0).stop 2024-04-25T10:31:32.461 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:31:32.459+0000 7f72c2af2640 1 -- 172.21.15.37:0/1396210448 shutdown_connections 2024-04-25T10:31:32.461 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:31:32.459+0000 7f72c2af2640 1 --2- 172.21.15.37:0/1396210448 >> [v2:172.21.15.203:6832/2418205237,v1:172.21.15.203:6833/2418205237] conn(0x7f729c0744e0 0x7f729c0769f0 unknown :-1 s=CLOSED pgs=338 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:31:32.461 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:31:32.459+0000 7f72c2af2640 1 --2- 172.21.15.37:0/1396210448 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f72bc157580 0x7f72bc141090 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:31:32.461 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:31:32.459+0000 7f72c2af2640 1 --2- 172.21.15.37:0/1396210448 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f72bc160a00 0x7f72bc153810 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:31:32.461 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:31:32.459+0000 7f72c2af2640 1 --2- 172.21.15.37:0/1396210448 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f72bc073810 0x7f72bc1f8390 unknown :-1 s=CLOSED pgs=752 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:31:32.462 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:31:32.459+0000 7f72c2af2640 1 -- 172.21.15.37:0/1396210448 >> 172.21.15.37:0/1396210448 conn(0x7f72bc0b3e20 msgr2=0x7f72bc0b2560 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:31:32.462 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:31:32.459+0000 7f72c2af2640 1 -- 172.21.15.37:0/1396210448 shutdown_connections 2024-04-25T10:31:32.462 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:31:32.459+0000 7f72c2af2640 1 -- 172.21.15.37:0/1396210448 wait complete. 2024-04-25T10:31:32.472 INFO:teuthology.orchestra.run.smithi037.stdout:{"pg_ready":true,"pg_map":{"version":123,"stamp":"2024-04-25T10:31:32.222032+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":66,"num_read_kb":55,"num_write":101,"num_write_kb":1628,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":74,"ondisk_log_size":74,"up":2,"acting":2,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":2,"num_osds":8,"num_per_pool_osds":8,"num_per_pool_omap_osds":2,"kb":754974720,"kb_used":226056,"kb_used_data":5736,"kb_used_omap":12,"kb_used_meta":220211,"kb_avail":754748664,"statfs":{"total":773094113280,"available":772862631936,"internally_reserved":0,"allocated":5873664,"data_stored":3144888,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":12752,"internal_metadata":225496624},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"12.002727"},"pg_stats":[{"pgid":"1.0","version":"50'74","reported_seq":171,"reported_epoch":50,"state":"active+clean","last_fresh":"2024-04-25T10:31:09.639686+0000","last_change":"2024-04-25T10:31:09.639733+0000","last_active":"2024-04-25T10:31:09.639686+0000","last_peered":"2024-04-25T10:31:09.639686+0000","last_clean":"2024-04-25T10:31:09.639686+0000","last_became_active":"2024-04-25T10:28:57.972373+0000","last_became_peered":"2024-04-25T10:28:57.972373+0000","last_unstale":"2024-04-25T10:31:09.639686+0000","last_undegraded":"2024-04-25T10:31:09.639686+0000","last_fullsized":"2024-04-25T10:31:09.639686+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"50'74","last_scrub_stamp":"2024-04-25T10:31:09.639572+0000","last_deep_scrub":"50'74","last_deep_scrub_stamp":"2024-04-25T10:31:09.639572+0000","last_clean_scrub_stamp":"2024-04-25T10:31:09.639572+0000","objects_scrubbed":2,"log_size":74,"log_dups_size":0,"ondisk_log_size":74,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T22:09:59.795122+0000","scrub_duration":6,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":66,"num_read_kb":55,"num_write":101,"num_write_kb":1628,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]}],"pool_stats":[{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":66,"num_read_kb":55,"num_write":101,"num_write_kb":1628,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1187840,"data_stored":1180736,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":74,"ondisk_log_size":74,"up":2,"acting":2,"num_store_stats":2}],"osd_stats":[{"osd":7,"up_from":15,"seq":64424509474,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":0,"kb":94371840,"kb_used":27592,"kb_used_data":572,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344248,"statfs":{"total":96636764160,"available":96608509952,"internally_reserved":0,"allocated":585728,"data_stored":245519,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1590,"internal_metadata":27654602},"hb_peers":[0,1,2,3,4,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":6,"up_from":15,"seq":64424509475,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":0,"kb":94371840,"kb_used":27592,"kb_used_data":572,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344248,"statfs":{"total":96636764160,"available":96608509952,"internally_reserved":0,"allocated":585728,"data_stored":245519,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":27654598},"hb_peers":[0,1,2,3,4,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":5,"up_from":15,"seq":64424509476,"num_pgs":1,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":28168,"kb_used_data":1152,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94343672,"statfs":{"total":96636764160,"available":96607920128,"internally_reserved":0,"allocated":1179648,"data_stored":835887,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":3,"up_from":15,"seq":64424509476,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":0,"kb":94371840,"kb_used":27592,"kb_used_data":572,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344248,"statfs":{"total":96636764160,"available":96608509952,"internally_reserved":0,"allocated":585728,"data_stored":245519,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1593,"internal_metadata":27654599},"hb_peers":[0,1,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":2,"up_from":15,"seq":64424509476,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":0,"kb":94371840,"kb_used":27596,"kb_used_data":572,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344244,"statfs":{"total":96636764160,"available":96608505856,"internally_reserved":0,"allocated":585728,"data_stored":245519,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":1,"up_from":15,"seq":64424509474,"num_pgs":1,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":28168,"kb_used_data":1152,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94343672,"statfs":{"total":96636764160,"available":96607920128,"internally_reserved":0,"allocated":1179648,"data_stored":835887,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":0,"up_from":15,"seq":64424509475,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":0,"kb":94371840,"kb_used":27592,"kb_used_data":572,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344248,"statfs":{"total":96636764160,"available":96608509952,"internally_reserved":0,"allocated":585728,"data_stored":245519,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[1,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":4,"up_from":13,"seq":55834574882,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":0,"kb":94371840,"kb_used":31756,"kb_used_data":572,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94340084,"statfs":{"total":96636764160,"available":96604246016,"internally_reserved":0,"allocated":585728,"data_stored":245519,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[0,1,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-04-25T10:31:32.473 DEBUG:teuthology.orchestra.run.smithi037:> sudo ceph --cluster ceph config set global mon_health_to_clog false 2024-04-25T10:31:32.584 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:31:32.582+0000 7fc50df08640 1 Processor -- start 2024-04-25T10:31:32.585 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:31:32.582+0000 7fc50df08640 1 -- start start 2024-04-25T10:31:32.587 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:31:32.584+0000 7fc50df08640 1 --2- >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7fc5080b4e80 0x7fc5080d2b20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:31:32.587 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:31:32.584+0000 7fc50df08640 1 -- --> v1:172.21.15.37:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fc5080d3470 con 0x7fc5080d3060 2024-04-25T10:31:32.587 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:31:32.584+0000 7fc50df08640 1 -- --> v1:172.21.15.203:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fc5080d10a0 con 0x7fc5080d5160 2024-04-25T10:31:32.587 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:31:32.584+0000 7fc50df08640 1 -- --> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] -- mon_getmap magic: 0 -- 0x7fc5080d1650 con 0x7fc5080b4e80 2024-04-25T10:31:32.587 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:31:32.584+0000 7fc507fff640 1 --1- >> v1:172.21.15.37:6789/0 conn(0x7fc5080d3060 0x7fc5080b9060 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.37:6789/0 says I am v1:172.21.15.37:48320/0 (socket says 172.21.15.37:48320) 2024-04-25T10:31:32.587 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:31:32.584+0000 7fc507fff640 1 -- 172.21.15.37:0/56757036 learned_addr learned my addr 172.21.15.37:0/56757036 (peer_addr_for_me v1:172.21.15.37:0/0) 2024-04-25T10:31:32.587 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:31:32.584+0000 7fc506ffd640 1 --2- 172.21.15.37:0/56757036 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7fc5080b4e80 0x7fc5080d2b20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:31:32.587 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:31:32.584+0000 7fc5067fc640 1 -- 172.21.15.37:0/56757036 <== mon.0 v1:172.21.15.37:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 2962618993 0 0) 0x7fc500001110 con 0x7fc5080d3060 2024-04-25T10:31:32.587 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:31:32.584+0000 7fc5067fc640 1 -- 172.21.15.37:0/56757036 --> v1:172.21.15.37:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7fc4f0003520 con 0x7fc5080d3060 2024-04-25T10:31:32.587 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:31:32.584+0000 7fc5067fc640 1 -- 172.21.15.37:0/56757036 <== mon.1 v1:172.21.15.203:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 148009167 0 0) 0x7fc5080d10a0 con 0x7fc5080d5160 2024-04-25T10:31:32.587 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:31:32.584+0000 7fc5067fc640 1 -- 172.21.15.37:0/56757036 --> v1:172.21.15.203:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7fc500001110 con 0x7fc5080d5160 2024-04-25T10:31:32.588 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:31:32.584+0000 7fc506ffd640 1 -- 172.21.15.37:0/56757036 >> v1:172.21.15.203:6789/0 conn(0x7fc5080d5160 legacy=0x7fc5080d0990 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:31:32.588 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:31:32.584+0000 7fc5067fc640 1 -- 172.21.15.37:0/56757036 <== mon.0 v1:172.21.15.37:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 3352469212 0 0) 0x7fc4f0003520 con 0x7fc5080d3060 2024-04-25T10:31:32.588 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:31:32.584+0000 7fc506ffd640 1 -- 172.21.15.37:0/56757036 >> v1:172.21.15.37:6789/0 conn(0x7fc5080d3060 legacy=0x7fc5080b9060 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:31:32.588 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:31:32.584+0000 7fc506ffd640 1 -- 172.21.15.37:0/56757036 --> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fc5080d1370 con 0x7fc5080b4e80 2024-04-25T10:31:32.588 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:31:32.585+0000 7fc506ffd640 1 --2- 172.21.15.37:0/56757036 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7fc5080b4e80 0x7fc5080d2b20 secure :-1 s=READY pgs=67 cs=0 l=1 rev1=1 crypto rx=0x7fc4fc0097d0 tx=0x7fc4fc02a100 comp rx=0 tx=0).ready entity=mon.2 client_cookie=172e875b82b74c61 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:31:32.589 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:31:32.585+0000 7fc5067fc640 1 -- 172.21.15.37:0/56757036 <== mon.2 v2:172.21.15.203:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc4fc037070 con 0x7fc5080b4e80 2024-04-25T10:31:32.589 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:31:32.585+0000 7fc5067fc640 1 -- 172.21.15.37:0/56757036 <== mon.2 v2:172.21.15.203:3301/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fc4fc02ab90 con 0x7fc5080b4e80 2024-04-25T10:31:32.589 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:31:32.585+0000 7fc5067fc640 1 -- 172.21.15.37:0/56757036 <== mon.2 v2:172.21.15.203:3301/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc500001860 con 0x7fc5080b4e80 2024-04-25T10:31:32.589 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:31:32.586+0000 7fc50df08640 1 -- 172.21.15.37:0/56757036 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7fc5080b4e80 msgr2=0x7fc5080d2b20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:31:32.589 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:31:32.586+0000 7fc50df08640 1 --2- 172.21.15.37:0/56757036 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7fc5080b4e80 0x7fc5080d2b20 secure :-1 s=READY pgs=67 cs=0 l=1 rev1=1 crypto rx=0x7fc4fc0097d0 tx=0x7fc4fc02a100 comp rx=0 tx=0).stop 2024-04-25T10:31:32.589 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:31:32.586+0000 7fc50df08640 1 -- 172.21.15.37:0/56757036 shutdown_connections 2024-04-25T10:31:32.589 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:31:32.586+0000 7fc50df08640 1 --2- 172.21.15.37:0/56757036 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7fc5080b4e80 0x7fc5080d2b20 unknown :-1 s=CLOSED pgs=67 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:31:32.589 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:31:32.586+0000 7fc50df08640 1 -- 172.21.15.37:0/56757036 >> 172.21.15.37:0/56757036 conn(0x7fc5080b7a80 msgr2=0x7fc5080b7e90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:31:32.589 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:31:32.586+0000 7fc50df08640 1 -- 172.21.15.37:0/56757036 shutdown_connections 2024-04-25T10:31:32.590 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:31:32.586+0000 7fc50df08640 1 -- 172.21.15.37:0/56757036 wait complete. 2024-04-25T10:31:32.590 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:31:32.588+0000 7fc50df08640 1 Processor -- start 2024-04-25T10:31:32.590 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:31:32.588+0000 7fc50df08640 1 -- start start 2024-04-25T10:31:32.593 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:31:32.589+0000 7fc50df08640 1 --2- >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7fc5080b4e80 0x7fc50815b940 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:31:32.593 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:31:32.589+0000 7fc50df08640 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fc5080d3060 0x7fc5081f9c20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:31:32.593 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:31:32.589+0000 7fc50df08640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fc5080d5160 0x7fc5080cb760 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:31:32.593 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:31:32.589+0000 7fc50df08640 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 -- 0x7fc5080d4960 con 0x7fc5080d5160 2024-04-25T10:31:32.593 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:31:32.589+0000 7fc50df08640 1 -- --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_getmap magic: 0 -- 0x7fc5080b5290 con 0x7fc5080d3060 2024-04-25T10:31:32.593 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:31:32.589+0000 7fc50df08640 1 -- --> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] -- mon_getmap magic: 0 -- 0x7fc5080d4c40 con 0x7fc5080b4e80 2024-04-25T10:31:32.593 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:31:32.589+0000 7fc507fff640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fc5080d5160 0x7fc5080cb760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:31:32.593 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:31:32.589+0000 7fc507fff640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fc5080d5160 0x7fc5080cb760 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.37:3300/0 says I am v2:172.21.15.37:46274/0 (socket says 172.21.15.37:46274) 2024-04-25T10:31:32.593 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:31:32.589+0000 7fc507fff640 1 -- 172.21.15.37:0/4214231484 learned_addr learned my addr 172.21.15.37:0/4214231484 (peer_addr_for_me v2:172.21.15.37:0/0) 2024-04-25T10:31:32.593 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:31:32.590+0000 7fc5077fe640 1 --2- 172.21.15.37:0/4214231484 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7fc5080b4e80 0x7fc50815b940 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:31:32.594 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:31:32.590+0000 7fc506ffd640 1 --2- 172.21.15.37:0/4214231484 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fc5080d3060 0x7fc5081f9c20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:31:32.594 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:31:32.590+0000 7fc507fff640 1 -- 172.21.15.37:0/4214231484 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7fc5080b4e80 msgr2=0x7fc50815b940 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:31:32.594 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:31:32.590+0000 7fc507fff640 1 --2- 172.21.15.37:0/4214231484 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7fc5080b4e80 0x7fc50815b940 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:31:32.594 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:31:32.590+0000 7fc507fff640 1 -- 172.21.15.37:0/4214231484 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fc5080d3060 msgr2=0x7fc5081f9c20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:31:32.594 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:31:32.590+0000 7fc507fff640 1 --2- 172.21.15.37:0/4214231484 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fc5080d3060 0x7fc5081f9c20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:31:32.594 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:31:32.590+0000 7fc507fff640 1 -- 172.21.15.37:0/4214231484 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fc5081f65f0 con 0x7fc5080d5160 2024-04-25T10:31:32.594 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:31:32.590+0000 7fc506ffd640 1 --2- 172.21.15.37:0/4214231484 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fc5080d3060 0x7fc5081f9c20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T10:31:32.594 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:31:32.590+0000 7fc507fff640 1 --2- 172.21.15.37:0/4214231484 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fc5080d5160 0x7fc5080cb760 secure :-1 s=READY pgs=754 cs=0 l=1 rev1=1 crypto rx=0x7fc50000c750 tx=0x7fc50000cc10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:31:32.594 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:31:32.590+0000 7fc4effff640 1 -- 172.21.15.37:0/4214231484 <== mon.0 v2:172.21.15.37:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc500018070 con 0x7fc5080d5160 2024-04-25T10:31:32.594 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:31:32.590+0000 7fc4effff640 1 -- 172.21.15.37:0/4214231484 <== mon.0 v2:172.21.15.37:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fc5000072d0 con 0x7fc5080d5160 2024-04-25T10:31:32.594 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:31:32.590+0000 7fc50df08640 1 -- 172.21.15.37:0/4214231484 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fc5081fa990 con 0x7fc5080d5160 2024-04-25T10:31:32.594 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:31:32.590+0000 7fc4effff640 1 -- 172.21.15.37:0/4214231484 <== mon.0 v2:172.21.15.37:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc5000047e0 con 0x7fc5080d5160 2024-04-25T10:31:32.594 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:31:32.590+0000 7fc50df08640 1 -- 172.21.15.37:0/4214231484 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fc5080c9bd0 con 0x7fc5080d5160 2024-04-25T10:31:32.595 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:31:32.591+0000 7fc50df08640 1 -- 172.21.15.37:0/4214231484 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fc5081fa700 con 0x7fc5080d5160 2024-04-25T10:31:32.595 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:31:32.592+0000 7fc4effff640 1 -- 172.21.15.37:0/4214231484 <== mon.0 v2:172.21.15.37:3300/0 4 ==== mgrmap(e 5) ==== 96782+0+0 (secure 0 0 0) 0x7fc50001b070 con 0x7fc5080d5160 2024-04-25T10:31:32.595 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:31:32.592+0000 7fc4effff640 1 --2- 172.21.15.37:0/4214231484 >> [v2:172.21.15.203:6832/2418205237,v1:172.21.15.203:6833/2418205237] conn(0x7fc4f40744b0 0x7fc4f40769c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:31:32.595 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:31:32.593+0000 7fc4effff640 1 -- 172.21.15.37:0/4214231484 <== mon.0 v2:172.21.15.37:3300/0 5 ==== osd_map(50..50 src has 1..50) ==== 5029+0+0 (secure 0 0 0) 0x7fc500094430 con 0x7fc5080d5160 2024-04-25T10:31:32.596 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:31:32.593+0000 7fc5077fe640 1 --2- 172.21.15.37:0/4214231484 >> [v2:172.21.15.203:6832/2418205237,v1:172.21.15.203:6833/2418205237] conn(0x7fc4f40744b0 0x7fc4f40769c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:31:32.596 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:31:32.593+0000 7fc5077fe640 1 --2- 172.21.15.37:0/4214231484 >> [v2:172.21.15.203:6832/2418205237,v1:172.21.15.203:6833/2418205237] conn(0x7fc4f40744b0 0x7fc4f40769c0 secure :-1 s=READY pgs=339 cs=0 l=1 rev1=1 crypto rx=0x7fc4f8001560 tx=0x7fc4f8009040 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:31:32.599 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:31:32.597+0000 7fc4effff640 1 -- 172.21.15.37:0/4214231484 <== mon.0 v2:172.21.15.37:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187246 (secure 0 0 0) 0x7fc5080d3470 con 0x7fc5080d5160 2024-04-25T10:31:32.787 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:31:32.784+0000 7fc50df08640 1 -- 172.21.15.37:0/4214231484 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_command([{prefix=config set, name=mon_health_to_clog}] v 0) -- 0x7fc50806a140 con 0x7fc5080d5160 2024-04-25T10:31:32.790 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:31:32.788+0000 7fc4effff640 1 -- 172.21.15.37:0/4214231484 <== mon.0 v2:172.21.15.37:3300/0 7 ==== mon_command_ack([{prefix=config set, name=mon_health_to_clog}]=0 v14)=0 v14) ==== 123+0+0 (secure 0 0 0) 0x7fc50005ee50 con 0x7fc5080d5160 2024-04-25T10:31:32.791 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:31:32.789+0000 7fc4effff640 1 -- 172.21.15.37:0/4214231484 <== mon.0 v2:172.21.15.37:3300/0 8 ==== config(1 keys) ==== 35+0+0 (secure 0 0 0) 0x7fc50005e260 con 0x7fc5080d5160 2024-04-25T10:31:32.793 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:31:32.790+0000 7fc50df08640 1 -- 172.21.15.37:0/4214231484 >> [v2:172.21.15.203:6832/2418205237,v1:172.21.15.203:6833/2418205237] conn(0x7fc4f40744b0 msgr2=0x7fc4f40769c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:31:32.793 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:31:32.790+0000 7fc50df08640 1 --2- 172.21.15.37:0/4214231484 >> [v2:172.21.15.203:6832/2418205237,v1:172.21.15.203:6833/2418205237] conn(0x7fc4f40744b0 0x7fc4f40769c0 secure :-1 s=READY pgs=339 cs=0 l=1 rev1=1 crypto rx=0x7fc4f8001560 tx=0x7fc4f8009040 comp rx=0 tx=0).stop 2024-04-25T10:31:32.793 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:31:32.791+0000 7fc50df08640 1 -- 172.21.15.37:0/4214231484 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fc5080d5160 msgr2=0x7fc5080cb760 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:31:32.794 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:31:32.791+0000 7fc50df08640 1 --2- 172.21.15.37:0/4214231484 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fc5080d5160 0x7fc5080cb760 secure :-1 s=READY pgs=754 cs=0 l=1 rev1=1 crypto rx=0x7fc50000c750 tx=0x7fc50000cc10 comp rx=0 tx=0).stop 2024-04-25T10:31:32.794 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:31:32.791+0000 7fc50df08640 1 -- 172.21.15.37:0/4214231484 shutdown_connections 2024-04-25T10:31:32.794 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:31:32.791+0000 7fc50df08640 1 --2- 172.21.15.37:0/4214231484 >> [v2:172.21.15.203:6832/2418205237,v1:172.21.15.203:6833/2418205237] conn(0x7fc4f40744b0 0x7fc4f40769c0 unknown :-1 s=CLOSED pgs=339 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:31:32.794 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:31:32.791+0000 7fc50df08640 1 --2- 172.21.15.37:0/4214231484 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fc5080d5160 0x7fc5080cb760 unknown :-1 s=CLOSED pgs=754 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:31:32.794 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:31:32.791+0000 7fc50df08640 1 --2- 172.21.15.37:0/4214231484 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fc5080d3060 0x7fc5081f9c20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:31:32.794 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:31:32.791+0000 7fc50df08640 1 --2- 172.21.15.37:0/4214231484 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7fc5080b4e80 0x7fc50815b940 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:31:32.794 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:31:32.791+0000 7fc50df08640 1 -- 172.21.15.37:0/4214231484 >> 172.21.15.37:0/4214231484 conn(0x7fc5080b7a80 msgr2=0x7fc5080ba7e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:31:32.794 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:31:32.791+0000 7fc50df08640 1 -- 172.21.15.37:0/4214231484 shutdown_connections 2024-04-25T10:31:32.794 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:31:32.791+0000 7fc50df08640 1 -- 172.21.15.37:0/4214231484 wait complete. 2024-04-25T10:31:32.804 INFO:tasks.cephfs.filesystem:Destroying file system cephfs and related pools 2024-04-25T10:31:32.804 DEBUG:teuthology.orchestra.run.smithi037:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-04-25T10:31:32.916 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:31:32.914+0000 7f2b2a17a640 1 Processor -- start 2024-04-25T10:31:32.916 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:31:32.914+0000 7f2b2a17a640 1 -- start start 2024-04-25T10:31:32.918 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:31:32.915+0000 7f2b2a17a640 1 --2- >> v2:172.21.15.203:3300/0 conn(0x7f2b240d5160 0x7f2b240b4e80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:31:32.918 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:31:32.915+0000 7f2b2a17a640 1 --2- >> v2:172.21.15.37:3300/0 conn(0x7f2b240d0990 0x7f2b240d2b20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:31:32.918 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:31:32.915+0000 7f2b2a17a640 1 --2- >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f2b240d0da0 0x7f2b240d3060 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:31:32.918 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:31:32.915+0000 7f2b2a17a640 1 -- --> v2:172.21.15.37:3300/0 -- mon_getmap magic: 0 -- 0x7f2b240d1650 con 0x7f2b240d0990 2024-04-25T10:31:32.919 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:31:32.915+0000 7f2b2a17a640 1 -- --> v2:172.21.15.203:3300/0 -- mon_getmap magic: 0 -- 0x7f2b240d1370 con 0x7f2b240d5160 2024-04-25T10:31:32.919 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:31:32.915+0000 7f2b2a17a640 1 -- --> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] -- mon_getmap magic: 0 -- 0x7f2b240d4680 con 0x7f2b240d0da0 2024-04-25T10:31:32.919 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:31:32.915+0000 7f2b22ffd640 1 --2- >> v2:172.21.15.37:3300/0 conn(0x7f2b240d0990 0x7f2b240d2b20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:31:32.919 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:31:32.915+0000 7f2b22ffd640 1 --2- >> v2:172.21.15.37:3300/0 conn(0x7f2b240d0990 0x7f2b240d2b20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.37:3300/0 says I am v2:172.21.15.37:46276/0 (socket says 172.21.15.37:46276) 2024-04-25T10:31:32.919 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:31:32.915+0000 7f2b22ffd640 1 -- 172.21.15.37:0/2919131286 learned_addr learned my addr 172.21.15.37:0/2919131286 (peer_addr_for_me v2:172.21.15.37:0/0) 2024-04-25T10:31:32.919 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:31:32.915+0000 7f2b23fff640 1 --2- 172.21.15.37:0/2919131286 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f2b240d0da0 0x7f2b240d3060 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:31:32.919 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:31:32.915+0000 7f2b237fe640 1 --2- 172.21.15.37:0/2919131286 >> v2:172.21.15.203:3300/0 conn(0x7f2b240d5160 0x7f2b240b4e80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:31:32.919 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:31:32.915+0000 7f2b22ffd640 1 -- 172.21.15.37:0/2919131286 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f2b240d0da0 msgr2=0x7f2b240d3060 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:31:32.919 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:31:32.915+0000 7f2b22ffd640 1 --2- 172.21.15.37:0/2919131286 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f2b240d0da0 0x7f2b240d3060 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:31:32.919 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:31:32.915+0000 7f2b22ffd640 1 -- 172.21.15.37:0/2919131286 >> v2:172.21.15.203:3300/0 conn(0x7f2b240d5160 msgr2=0x7f2b240b4e80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:31:32.919 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:31:32.915+0000 7f2b22ffd640 1 --2- 172.21.15.37:0/2919131286 >> v2:172.21.15.203:3300/0 conn(0x7f2b240d5160 0x7f2b240b4e80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:31:32.919 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:31:32.915+0000 7f2b22ffd640 1 -- 172.21.15.37:0/2919131286 --> v2:172.21.15.37:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f2b240d4960 con 0x7f2b240d0990 2024-04-25T10:31:32.919 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:31:32.915+0000 7f2b23fff640 1 --2- 172.21.15.37:0/2919131286 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f2b240d0da0 0x7f2b240d3060 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T10:31:32.920 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:31:32.916+0000 7f2b22ffd640 1 --2- 172.21.15.37:0/2919131286 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f2b240d0990 0x7f2b240d2b20 secure :-1 s=READY pgs=755 cs=0 l=1 rev1=1 crypto rx=0x7f2b240b86b0 tx=0x7f2b1802a0b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=5aae12ce49f27ece server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:31:32.920 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:31:32.916+0000 7f2b227fc640 1 -- 172.21.15.37:0/2919131286 <== mon.0 v2:172.21.15.37:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2b18037070 con 0x7f2b240d0990 2024-04-25T10:31:32.920 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:31:32.916+0000 7f2b227fc640 1 -- 172.21.15.37:0/2919131286 <== mon.0 v2:172.21.15.37:3300/0 2 ==== config(1 keys) ==== 35+0+0 (secure 0 0 0) 0x7f2b1802ae00 con 0x7f2b240d0990 2024-04-25T10:31:32.920 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:31:32.916+0000 7f2b227fc640 1 -- 172.21.15.37:0/2919131286 <== mon.0 v2:172.21.15.37:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2b18032680 con 0x7f2b240d0990 2024-04-25T10:31:32.921 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:31:32.917+0000 7f2b2a17a640 1 -- 172.21.15.37:0/2919131286 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f2b240d0990 msgr2=0x7f2b240d2b20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:31:32.921 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:31:32.917+0000 7f2b2a17a640 1 --2- 172.21.15.37:0/2919131286 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f2b240d0990 0x7f2b240d2b20 secure :-1 s=READY pgs=755 cs=0 l=1 rev1=1 crypto rx=0x7f2b240b86b0 tx=0x7f2b1802a0b0 comp rx=0 tx=0).stop 2024-04-25T10:31:32.921 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:31:32.917+0000 7f2b2a17a640 1 -- 172.21.15.37:0/2919131286 shutdown_connections 2024-04-25T10:31:32.921 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:31:32.917+0000 7f2b2a17a640 1 --2- 172.21.15.37:0/2919131286 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f2b240d0990 0x7f2b240d2b20 unknown :-1 s=CLOSED pgs=755 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:31:32.921 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:31:32.917+0000 7f2b2a17a640 1 --2- 172.21.15.37:0/2919131286 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f2b240d0da0 0x7f2b240d3060 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:31:32.921 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:31:32.917+0000 7f2b2a17a640 1 --2- 172.21.15.37:0/2919131286 >> v2:172.21.15.203:3300/0 conn(0x7f2b240d5160 0x7f2b240b4e80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:31:32.921 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:31:32.917+0000 7f2b2a17a640 1 -- 172.21.15.37:0/2919131286 >> 172.21.15.37:0/2919131286 conn(0x7f2b240b7a80 msgr2=0x7f2b240b7e90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:31:32.922 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:31:32.917+0000 7f2b2a17a640 1 -- 172.21.15.37:0/2919131286 shutdown_connections 2024-04-25T10:31:32.922 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:31:32.917+0000 7f2b2a17a640 1 -- 172.21.15.37:0/2919131286 wait complete. 2024-04-25T10:31:32.922 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:31:32.919+0000 7f2b2a17a640 1 Processor -- start 2024-04-25T10:31:32.922 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:31:32.919+0000 7f2b2a17a640 1 -- start start 2024-04-25T10:31:32.924 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:31:32.921+0000 7f2b2a17a640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f2b240d0990 0x7f2b24152b70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:31:32.924 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:31:32.921+0000 7f2b2a17a640 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f2b240d0da0 0x7f2b240ce390 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:31:32.924 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:31:32.921+0000 7f2b2a17a640 1 --2- >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f2b240d5160 0x7f2b241623c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:31:32.924 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:31:32.921+0000 7f2b2a17a640 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 -- 0x7f2b24164c40 con 0x7f2b240d0990 2024-04-25T10:31:32.924 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:31:32.921+0000 7f2b2a17a640 1 -- --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_getmap magic: 0 -- 0x7f2b24164ac0 con 0x7f2b240d0da0 2024-04-25T10:31:32.924 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:31:32.921+0000 7f2b2a17a640 1 -- --> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] -- mon_getmap magic: 0 -- 0x7f2b2406db10 con 0x7f2b240d5160 2024-04-25T10:31:32.924 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:31:32.921+0000 7f2b237fe640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f2b240d0990 0x7f2b24152b70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:31:32.925 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:31:32.921+0000 7f2b237fe640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f2b240d0990 0x7f2b24152b70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.37:3300/0 says I am v2:172.21.15.37:46278/0 (socket says 172.21.15.37:46278) 2024-04-25T10:31:32.925 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:31:32.921+0000 7f2b237fe640 1 -- 172.21.15.37:0/3480675258 learned_addr learned my addr 172.21.15.37:0/3480675258 (peer_addr_for_me v2:172.21.15.37:0/0) 2024-04-25T10:31:32.925 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:31:32.921+0000 7f2b22ffd640 1 --2- 172.21.15.37:0/3480675258 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f2b240d0da0 0x7f2b240ce390 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:31:32.925 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:31:32.921+0000 7f2b23fff640 1 --2- 172.21.15.37:0/3480675258 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f2b240d5160 0x7f2b241623c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:31:32.925 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:31:32.921+0000 7f2b237fe640 1 -- 172.21.15.37:0/3480675258 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f2b240d5160 msgr2=0x7f2b241623c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:31:32.925 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:31:32.921+0000 7f2b237fe640 1 --2- 172.21.15.37:0/3480675258 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f2b240d5160 0x7f2b241623c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:31:32.925 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:31:32.921+0000 7f2b237fe640 1 -- 172.21.15.37:0/3480675258 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f2b240d0da0 msgr2=0x7f2b240ce390 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:31:32.925 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:31:32.921+0000 7f2b237fe640 1 --2- 172.21.15.37:0/3480675258 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f2b240d0da0 0x7f2b240ce390 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:31:32.925 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:31:32.921+0000 7f2b237fe640 1 -- 172.21.15.37:0/3480675258 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f2b240ced00 con 0x7f2b240d0990 2024-04-25T10:31:32.925 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:31:32.921+0000 7f2b237fe640 1 --2- 172.21.15.37:0/3480675258 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f2b240d0990 0x7f2b24152b70 secure :-1 s=READY pgs=756 cs=0 l=1 rev1=1 crypto rx=0x7f2b1400e870 tx=0x7f2b1400ed30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:31:32.925 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:31:32.921+0000 7f2b0bfff640 1 -- 172.21.15.37:0/3480675258 <== mon.0 v2:172.21.15.37:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2b14017070 con 0x7f2b240d0990 2024-04-25T10:31:32.925 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:31:32.921+0000 7f2b0bfff640 1 -- 172.21.15.37:0/3480675258 <== mon.0 v2:172.21.15.37:3300/0 2 ==== config(1 keys) ==== 35+0+0 (secure 0 0 0) 0x7f2b1400c940 con 0x7f2b240d0990 2024-04-25T10:31:32.925 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:31:32.921+0000 7f2b2a17a640 1 -- 172.21.15.37:0/3480675258 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f2b240cefe0 con 0x7f2b240d0990 2024-04-25T10:31:32.926 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:31:32.922+0000 7f2b0bfff640 1 -- 172.21.15.37:0/3480675258 <== mon.0 v2:172.21.15.37:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2b1400cd50 con 0x7f2b240d0990 2024-04-25T10:31:32.926 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:31:32.922+0000 7f2b2a17a640 1 -- 172.21.15.37:0/3480675258 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f2b240cf5a0 con 0x7f2b240d0990 2024-04-25T10:31:32.926 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:31:32.923+0000 7f2b2a17a640 1 -- 172.21.15.37:0/3480675258 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f2b240ba3b0 con 0x7f2b240d0990 2024-04-25T10:31:32.927 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:31:32.923+0000 7f2b0bfff640 1 -- 172.21.15.37:0/3480675258 <== mon.0 v2:172.21.15.37:3300/0 4 ==== mgrmap(e 5) ==== 96782+0+0 (secure 0 0 0) 0x7f2b1401a070 con 0x7f2b240d0990 2024-04-25T10:31:32.927 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:31:32.924+0000 7f2b0bfff640 1 --2- 172.21.15.37:0/3480675258 >> [v2:172.21.15.203:6832/2418205237,v1:172.21.15.203:6833/2418205237] conn(0x7f2b100744e0 0x7f2b100769f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:31:32.927 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:31:32.924+0000 7f2b0bfff640 1 -- 172.21.15.37:0/3480675258 <== mon.0 v2:172.21.15.37:3300/0 5 ==== osd_map(50..50 src has 1..50) ==== 5029+0+0 (secure 0 0 0) 0x7f2b14095140 con 0x7f2b240d0990 2024-04-25T10:31:32.927 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:31:32.924+0000 7f2b22ffd640 1 --2- 172.21.15.37:0/3480675258 >> [v2:172.21.15.203:6832/2418205237,v1:172.21.15.203:6833/2418205237] conn(0x7f2b100744e0 0x7f2b100769f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:31:32.927 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:31:32.924+0000 7f2b22ffd640 1 --2- 172.21.15.37:0/3480675258 >> [v2:172.21.15.203:6832/2418205237,v1:172.21.15.203:6833/2418205237] conn(0x7f2b100744e0 0x7f2b100769f0 secure :-1 s=READY pgs=340 cs=0 l=1 rev1=1 crypto rx=0x7f2b18034000 tx=0x7f2b1802e000 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:31:32.930 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:31:32.928+0000 7f2b0bfff640 1 -- 172.21.15.37:0/3480675258 <== mon.0 v2:172.21.15.37:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187246 (secure 0 0 0) 0x7f2b240ba3b0 con 0x7f2b240d0990 2024-04-25T10:31:33.155 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:31:33.153+0000 7f2b2a17a640 1 -- 172.21.15.37:0/3480675258 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f2b2406a140 con 0x7f2b240d0990 2024-04-25T10:31:33.155 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:31:33.153+0000 7f2b0bfff640 1 -- 172.21.15.37:0/3480675258 <== mon.0 v2:172.21.15.37:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 31 v31) ==== 94+0+3872 (secure 0 0 0) 0x7f2b1405f520 con 0x7f2b240d0990 2024-04-25T10:31:33.156 INFO:teuthology.orchestra.run.smithi037.stdout: 2024-04-25T10:31:33.156 INFO:teuthology.orchestra.run.smithi037.stdout:{"epoch":31,"default_fscid":-1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":4946,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.37:6835/683655741","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.37:6834","nonce":683655741},{"type":"v1","addr":"172.21.15.37:6835","nonce":683655741}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":19},{"gid":5215,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.203:6836/3927045001","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6835","nonce":3927045001},{"type":"v1","addr":"172.21.15.203:6836","nonce":3927045001}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":19},{"gid":5218,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.203:6837/3108183946","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6834","nonce":3108183946},{"type":"v1","addr":"172.21.15.203:6837","nonce":3108183946}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":19},{"gid":6162,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.37:6833/1209328775","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.37:6832","nonce":1209328775},{"type":"v1","addr":"172.21.15.37:6833","nonce":1209328775}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":31}],"filesystems":[]} 2024-04-25T10:31:33.156 INFO:teuthology.orchestra.run.smithi037.stderr:dumped fsmap epoch 31 2024-04-25T10:31:33.158 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:31:33.155+0000 7f2b2a17a640 1 -- 172.21.15.37:0/3480675258 >> [v2:172.21.15.203:6832/2418205237,v1:172.21.15.203:6833/2418205237] conn(0x7f2b100744e0 msgr2=0x7f2b100769f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:31:33.158 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:31:33.155+0000 7f2b2a17a640 1 --2- 172.21.15.37:0/3480675258 >> [v2:172.21.15.203:6832/2418205237,v1:172.21.15.203:6833/2418205237] conn(0x7f2b100744e0 0x7f2b100769f0 secure :-1 s=READY pgs=340 cs=0 l=1 rev1=1 crypto rx=0x7f2b18034000 tx=0x7f2b1802e000 comp rx=0 tx=0).stop 2024-04-25T10:31:33.158 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:31:33.155+0000 7f2b2a17a640 1 -- 172.21.15.37:0/3480675258 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f2b240d0990 msgr2=0x7f2b24152b70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:31:33.158 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:31:33.155+0000 7f2b2a17a640 1 --2- 172.21.15.37:0/3480675258 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f2b240d0990 0x7f2b24152b70 secure :-1 s=READY pgs=756 cs=0 l=1 rev1=1 crypto rx=0x7f2b1400e870 tx=0x7f2b1400ed30 comp rx=0 tx=0).stop 2024-04-25T10:31:33.158 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:31:33.155+0000 7f2b2a17a640 1 -- 172.21.15.37:0/3480675258 shutdown_connections 2024-04-25T10:31:33.158 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:31:33.155+0000 7f2b2a17a640 1 --2- 172.21.15.37:0/3480675258 >> [v2:172.21.15.203:6832/2418205237,v1:172.21.15.203:6833/2418205237] conn(0x7f2b100744e0 0x7f2b100769f0 unknown :-1 s=CLOSED pgs=340 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:31:33.158 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:31:33.155+0000 7f2b2a17a640 1 --2- 172.21.15.37:0/3480675258 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f2b240d5160 0x7f2b241623c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:31:33.158 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:31:33.155+0000 7f2b2a17a640 1 --2- 172.21.15.37:0/3480675258 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f2b240d0da0 0x7f2b240ce390 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:31:33.159 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:31:33.155+0000 7f2b2a17a640 1 --2- 172.21.15.37:0/3480675258 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f2b240d0990 0x7f2b24152b70 unknown :-1 s=CLOSED pgs=756 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:31:33.159 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:31:33.155+0000 7f2b2a17a640 1 -- 172.21.15.37:0/3480675258 >> 172.21.15.37:0/3480675258 conn(0x7f2b240b7a80 msgr2=0x7f2b240b6d40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:31:33.159 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:31:33.156+0000 7f2b2a17a640 1 -- 172.21.15.37:0/3480675258 shutdown_connections 2024-04-25T10:31:33.159 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T10:31:33.156+0000 7f2b2a17a640 1 -- 172.21.15.37:0/3480675258 wait complete. 2024-04-25T10:31:33.169 DEBUG:tasks.cephfs.filesystem:already dead... 2024-04-25T10:31:33.169 INFO:teuthology.misc:Shutting down mds daemons... 2024-04-25T10:31:33.169 DEBUG:tasks.ceph.mds.a:waiting for process to exit 2024-04-25T10:31:33.169 INFO:teuthology.orchestra.run:waiting for 300 2024-04-25T10:31:33.179 INFO:tasks.ceph.mds.a:Stopped 2024-04-25T10:31:33.179 DEBUG:tasks.ceph.mds.c:waiting for process to exit 2024-04-25T10:31:33.179 INFO:teuthology.orchestra.run:waiting for 300 2024-04-25T10:31:33.231 INFO:tasks.ceph.mds.c:Stopped 2024-04-25T10:31:33.231 DEBUG:tasks.ceph.mds.b:waiting for process to exit 2024-04-25T10:31:33.231 INFO:teuthology.orchestra.run:waiting for 300 2024-04-25T10:31:33.243 INFO:tasks.ceph.mds.b:Stopped 2024-04-25T10:31:33.243 DEBUG:tasks.ceph.mds.d:waiting for process to exit 2024-04-25T10:31:33.243 INFO:teuthology.orchestra.run:waiting for 300 2024-04-25T10:31:33.296 INFO:tasks.ceph.mds.d:Stopped 2024-04-25T10:31:33.297 INFO:teuthology.misc:Shutting down osd daemons... 2024-04-25T10:31:33.297 DEBUG:tasks.ceph.osd.0:waiting for process to exit 2024-04-25T10:31:33.297 INFO:teuthology.orchestra.run:waiting for 300 2024-04-25T10:31:33.326 INFO:tasks.ceph.osd.0:Stopped 2024-04-25T10:31:33.326 DEBUG:tasks.ceph.osd.1:waiting for process to exit 2024-04-25T10:31:33.326 INFO:teuthology.orchestra.run:waiting for 300 2024-04-25T10:31:33.392 INFO:tasks.ceph.osd.1:Stopped 2024-04-25T10:31:33.392 DEBUG:tasks.ceph.osd.2:waiting for process to exit 2024-04-25T10:31:33.392 INFO:teuthology.orchestra.run:waiting for 300 2024-04-25T10:31:33.458 INFO:tasks.ceph.osd.2:Stopped 2024-04-25T10:31:33.458 DEBUG:tasks.ceph.osd.3:waiting for process to exit 2024-04-25T10:31:33.458 INFO:teuthology.orchestra.run:waiting for 300 2024-04-25T10:31:33.527 INFO:tasks.ceph.osd.3:Stopped 2024-04-25T10:31:33.527 DEBUG:tasks.ceph.osd.4:waiting for process to exit 2024-04-25T10:31:33.527 INFO:teuthology.orchestra.run:waiting for 300 2024-04-25T10:31:33.558 INFO:tasks.ceph.osd.4:Stopped 2024-04-25T10:31:33.558 DEBUG:tasks.ceph.osd.5:waiting for process to exit 2024-04-25T10:31:33.558 INFO:teuthology.orchestra.run:waiting for 300 2024-04-25T10:31:33.628 INFO:tasks.ceph.osd.5:Stopped 2024-04-25T10:31:33.628 DEBUG:tasks.ceph.osd.6:waiting for process to exit 2024-04-25T10:31:33.628 INFO:teuthology.orchestra.run:waiting for 300 2024-04-25T10:31:33.701 INFO:tasks.ceph.osd.6:Stopped 2024-04-25T10:31:33.701 DEBUG:tasks.ceph.osd.7:waiting for process to exit 2024-04-25T10:31:33.702 INFO:teuthology.orchestra.run:waiting for 300 2024-04-25T10:31:33.774 INFO:tasks.ceph.osd.7:Stopped 2024-04-25T10:31:33.775 INFO:teuthology.misc:Shutting down mgr daemons... 2024-04-25T10:31:33.775 DEBUG:tasks.ceph.mgr.y:waiting for process to exit 2024-04-25T10:31:33.775 INFO:teuthology.orchestra.run:waiting for 300 2024-04-25T10:31:33.800 INFO:tasks.ceph.mgr.y:Stopped 2024-04-25T10:31:33.801 DEBUG:tasks.ceph.mgr.x:waiting for process to exit 2024-04-25T10:31:33.801 INFO:teuthology.orchestra.run:waiting for 300 2024-04-25T10:31:33.844 INFO:tasks.ceph.mgr.x:Stopped 2024-04-25T10:31:33.844 INFO:teuthology.misc:Shutting down mon daemons... 2024-04-25T10:31:33.845 DEBUG:tasks.ceph.mon.a:waiting for process to exit 2024-04-25T10:31:33.845 INFO:teuthology.orchestra.run:waiting for 300 2024-04-25T10:31:33.857 INFO:tasks.ceph.mon.a:Stopped 2024-04-25T10:31:33.857 DEBUG:tasks.ceph.mon.b:waiting for process to exit 2024-04-25T10:31:33.857 INFO:teuthology.orchestra.run:waiting for 300 2024-04-25T10:31:33.897 INFO:tasks.ceph.mon.b:Stopped 2024-04-25T10:31:33.897 DEBUG:tasks.ceph.mon.c:waiting for process to exit 2024-04-25T10:31:33.897 INFO:teuthology.orchestra.run:waiting for 300 2024-04-25T10:31:33.950 INFO:tasks.ceph.mon.c:Stopped 2024-04-25T10:31:33.950 INFO:tasks.ceph:Checking cluster log for badness... 2024-04-25T10:31:33.950 DEBUG:teuthology.orchestra.run.smithi037:> sudo egrep '\[ERR\]|\[WRN\]|\[SEC\]' /var/log/ceph/ceph.log | egrep -v '\(MDS_ALL_DOWN\)' | egrep -v '\(MDS_UP_LESS_THAN_MAX\)' | egrep -v FS_DEGRADED | egrep -v FS_INLINE_DATA_DEPRECATED | egrep -v FS_WITH_FAILED_MDS | egrep -v MDS_ALL_DOWN | egrep -v MDS_DAMAGE | egrep -v MDS_DEGRADED | egrep -v MDS_FAILED | egrep -v MDS_INSUFFICIENT_STANDBY | egrep -v MDS_UP_LESS_THAN_MAX | egrep -v 'filesystem is online with fewer MDS than max_mds' | egrep -v POOL_APP_NOT_ENABLED | egrep -v 'do not have an application enabled' | egrep -v 'overall HEALTH_' | egrep -v 'Replacing daemon' | egrep -v 'deprecated feature inline_data' | egrep -v 'overall HEALTH_' | egrep -v '\(OSD_DOWN\)' | egrep -v '\(OSD_' | egrep -v 'but it is still running' | egrep -v 'is not responding' | egrep -v PG_AVAILABILITY | egrep -v PG_DEGRADED | egrep -v 'Reduced data availability' | egrep -v 'bad backtrace' | egrep -v 'object missing on disk' | egrep -v 'error reading table object' | egrep -v 'error reading sessionmap' | egrep -v 'unmatched fragstat' | egrep -v 'unmatched rstat' | egrep -v 'was unreadable, recreating it now' | egrep -v 'Scrub error on inode' | egrep -v 'Metadata damage detected' | egrep -v MDS_FAILED | egrep -v MDS_DAMAGE | head -n 1 2024-04-25T10:31:33.984 INFO:tasks.ceph:Unmounting /var/lib/ceph/osd/ceph-0 on ubuntu@smithi037.front.sepia.ceph.com 2024-04-25T10:31:33.985 DEBUG:teuthology.orchestra.run.smithi037:> sync && sudo umount -f /var/lib/ceph/osd/ceph-0 2024-04-25T10:31:34.184 INFO:tasks.ceph:Unmounting /var/lib/ceph/osd/ceph-1 on ubuntu@smithi037.front.sepia.ceph.com 2024-04-25T10:31:34.185 DEBUG:teuthology.orchestra.run.smithi037:> sync && sudo umount -f /var/lib/ceph/osd/ceph-1 2024-04-25T10:31:34.255 INFO:tasks.ceph:Unmounting /var/lib/ceph/osd/ceph-2 on ubuntu@smithi037.front.sepia.ceph.com 2024-04-25T10:31:34.255 DEBUG:teuthology.orchestra.run.smithi037:> sync && sudo umount -f /var/lib/ceph/osd/ceph-2 2024-04-25T10:31:34.296 INFO:tasks.ceph:Unmounting /var/lib/ceph/osd/ceph-3 on ubuntu@smithi037.front.sepia.ceph.com 2024-04-25T10:31:34.297 DEBUG:teuthology.orchestra.run.smithi037:> sync && sudo umount -f /var/lib/ceph/osd/ceph-3 2024-04-25T10:31:34.381 INFO:tasks.ceph:Unmounting /var/lib/ceph/osd/ceph-4 on ubuntu@smithi203.front.sepia.ceph.com 2024-04-25T10:31:34.381 DEBUG:teuthology.orchestra.run.smithi203:> sync && sudo umount -f /var/lib/ceph/osd/ceph-4 2024-04-25T10:31:34.517 INFO:tasks.ceph:Unmounting /var/lib/ceph/osd/ceph-5 on ubuntu@smithi203.front.sepia.ceph.com 2024-04-25T10:31:34.517 DEBUG:teuthology.orchestra.run.smithi203:> sync && sudo umount -f /var/lib/ceph/osd/ceph-5 2024-04-25T10:31:34.564 INFO:tasks.ceph:Unmounting /var/lib/ceph/osd/ceph-6 on ubuntu@smithi203.front.sepia.ceph.com 2024-04-25T10:31:34.564 DEBUG:teuthology.orchestra.run.smithi203:> sync && sudo umount -f /var/lib/ceph/osd/ceph-6 2024-04-25T10:31:34.664 INFO:tasks.ceph:Unmounting /var/lib/ceph/osd/ceph-7 on ubuntu@smithi203.front.sepia.ceph.com 2024-04-25T10:31:34.664 DEBUG:teuthology.orchestra.run.smithi203:> sync && sudo umount -f /var/lib/ceph/osd/ceph-7 2024-04-25T10:31:34.722 INFO:tasks.ceph:Archiving mon data... 2024-04-25T10:31:34.723 DEBUG:teuthology.misc:Transferring archived files from smithi037:/var/lib/ceph/mon/ceph-a to /home/teuthworker/archive/pdonnell-2024-04-25_10:00:23-fs-wip-pdonnell-testing-20240425.015853-debug-distro-default-smithi/7673076/data/mon.a.tgz 2024-04-25T10:31:34.723 DEBUG:teuthology.orchestra.run.smithi037:> mktemp 2024-04-25T10:31:34.739 INFO:teuthology.orchestra.run.smithi037.stdout:/tmp/tmp.4RZ1qdCiMW 2024-04-25T10:31:34.739 DEBUG:teuthology.orchestra.run.smithi037:> sudo tar cz -f - -C /var/lib/ceph/mon/ceph-a -- . > /tmp/tmp.4RZ1qdCiMW 2024-04-25T10:31:34.886 DEBUG:teuthology.orchestra.run.smithi037:> sudo chmod 0666 /tmp/tmp.4RZ1qdCiMW 2024-04-25T10:31:34.972 DEBUG:teuthology.orchestra.remote:smithi037:/tmp/tmp.4RZ1qdCiMW is 327KB 2024-04-25T10:31:35.070 DEBUG:teuthology.orchestra.run.smithi037:> rm -fr /tmp/tmp.4RZ1qdCiMW 2024-04-25T10:31:35.085 DEBUG:teuthology.misc:Transferring archived files from smithi203:/var/lib/ceph/mon/ceph-b to /home/teuthworker/archive/pdonnell-2024-04-25_10:00:23-fs-wip-pdonnell-testing-20240425.015853-debug-distro-default-smithi/7673076/data/mon.b.tgz 2024-04-25T10:31:35.085 DEBUG:teuthology.orchestra.run.smithi203:> mktemp 2024-04-25T10:31:35.102 INFO:teuthology.orchestra.run.smithi203.stdout:/tmp/tmp.BdgggU2Z2H 2024-04-25T10:31:35.103 DEBUG:teuthology.orchestra.run.smithi203:> sudo tar cz -f - -C /var/lib/ceph/mon/ceph-b -- . > /tmp/tmp.BdgggU2Z2H 2024-04-25T10:31:35.263 DEBUG:teuthology.orchestra.run.smithi203:> sudo chmod 0666 /tmp/tmp.BdgggU2Z2H 2024-04-25T10:31:35.345 DEBUG:teuthology.orchestra.remote:smithi203:/tmp/tmp.BdgggU2Z2H is 435KB 2024-04-25T10:31:35.445 DEBUG:teuthology.orchestra.run.smithi203:> rm -fr /tmp/tmp.BdgggU2Z2H 2024-04-25T10:31:35.460 DEBUG:teuthology.misc:Transferring archived files from smithi203:/var/lib/ceph/mon/ceph-c to /home/teuthworker/archive/pdonnell-2024-04-25_10:00:23-fs-wip-pdonnell-testing-20240425.015853-debug-distro-default-smithi/7673076/data/mon.c.tgz 2024-04-25T10:31:35.460 DEBUG:teuthology.orchestra.run.smithi203:> mktemp 2024-04-25T10:31:35.513 INFO:teuthology.orchestra.run.smithi203.stdout:/tmp/tmp.cqDRtjHgGb 2024-04-25T10:31:35.513 DEBUG:teuthology.orchestra.run.smithi203:> sudo tar cz -f - -C /var/lib/ceph/mon/ceph-c -- . > /tmp/tmp.cqDRtjHgGb 2024-04-25T10:31:35.658 DEBUG:teuthology.orchestra.run.smithi203:> sudo chmod 0666 /tmp/tmp.cqDRtjHgGb 2024-04-25T10:31:35.741 DEBUG:teuthology.orchestra.remote:smithi203:/tmp/tmp.cqDRtjHgGb is 435KB 2024-04-25T10:31:35.845 DEBUG:teuthology.orchestra.run.smithi203:> rm -fr /tmp/tmp.cqDRtjHgGb 2024-04-25T10:31:35.859 INFO:tasks.ceph:Cleaning ceph cluster... 2024-04-25T10:31:35.860 DEBUG:teuthology.orchestra.run.smithi037:> sudo rm -rf -- /etc/ceph/ceph.conf /etc/ceph/ceph.keyring /home/ubuntu/cephtest/ceph.data /home/ubuntu/cephtest/ceph.monmap /home/ubuntu/cephtest/../*.pid 2024-04-25T10:31:35.862 DEBUG:teuthology.orchestra.run.smithi203:> sudo rm -rf -- /etc/ceph/ceph.conf /etc/ceph/ceph.keyring /home/ubuntu/cephtest/ceph.data /home/ubuntu/cephtest/ceph.monmap /home/ubuntu/cephtest/../*.pid 2024-04-25T10:31:35.945 INFO:teuthology.util.scanner:summary_data or yaml_file is empty! 2024-04-25T10:31:35.978 INFO:teuthology.util.scanner:summary_data or yaml_file is empty! 2024-04-25T10:31:35.978 INFO:tasks.ceph:Archiving crash dumps... 2024-04-25T10:31:35.980 DEBUG:teuthology.misc:Transferring archived files from smithi037:/var/lib/ceph/crash to /home/teuthworker/archive/pdonnell-2024-04-25_10:00:23-fs-wip-pdonnell-testing-20240425.015853-debug-distro-default-smithi/7673076/remote/smithi037/crash 2024-04-25T10:31:35.980 DEBUG:teuthology.orchestra.run.smithi037:> sudo tar c -f - -C /var/lib/ceph/crash -- . 2024-04-25T10:31:36.013 DEBUG:teuthology.misc:Transferring archived files from smithi203:/var/lib/ceph/crash to /home/teuthworker/archive/pdonnell-2024-04-25_10:00:23-fs-wip-pdonnell-testing-20240425.015853-debug-distro-default-smithi/7673076/remote/smithi203/crash 2024-04-25T10:31:36.014 DEBUG:teuthology.orchestra.run.smithi203:> sudo tar c -f - -C /var/lib/ceph/crash -- . 2024-04-25T10:31:36.040 INFO:tasks.ceph:Shutting down logrotate 2024-04-25T10:31:36.041 DEBUG:teuthology.orchestra.run.smithi037:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-25T10:31:36.059 DEBUG:teuthology.orchestra.run.smithi203:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-25T10:31:36.136 DEBUG:teuthology.orchestra.run.smithi037:> sudo rm /etc/logrotate.d/ceph-test.conf 2024-04-25T10:31:36.162 DEBUG:teuthology.orchestra.run.smithi203:> sudo rm /etc/logrotate.d/ceph-test.conf 2024-04-25T10:31:36.186 INFO:tasks.ceph:Compressing logs... 2024-04-25T10:31:36.186 DEBUG:teuthology.orchestra.run.smithi037:> time sudo find /var/log/ceph -name '*.log' -print0 | sudo xargs --max-args=1 --max-procs=0 --verbose -0 --no-run-if-empty -- gzip -5 --verbose -- 2024-04-25T10:31:36.205 DEBUG:teuthology.orchestra.run.smithi203:> time sudo find /var/log/ceph -name '*.log' -print0 | sudo xargs --max-args=1 --max-procs=0 --verbose -0 --no-run-if-empty -- gzip -5 --verbose -- 2024-04-25T10:31:36.229 INFO:teuthology.orchestra.run.smithi037.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.46095.log 2024-04-25T10:31:36.229 INFO:teuthology.orchestra.run.smithi037.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.52311.log 2024-04-25T10:31:36.229 INFO:teuthology.orchestra.run.smithi037.stderr:/var/log/ceph/ceph-client.admin.46095.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.53827.log 2024-04-25T10:31:36.265 INFO:teuthology.orchestra.run.smithi203.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.35053.log 2024-04-25T10:31:36.265 INFO:teuthology.orchestra.run.smithi203.stderr:gzip -5 --verbose -- /var/log/ceph/ceph.log 2024-04-25T10:31:36.265 INFO:teuthology.orchestra.run.smithi203.stderr:/var/log/ceph/ceph-client.admin.35053.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.34801.log 2024-04-25T10:31:36.265 INFO:teuthology.orchestra.run.smithi203.stderr:/var/log/ceph/ceph.log: gzip -5 --verbose -- /var/log/ceph/ceph.audit.log 2024-04-25T10:31:36.265 INFO:teuthology.orchestra.run.smithi203.stderr: 88.3% -- replaced with /var/log/ceph/ceph-client.admin.35053.log.gz 2024-04-25T10:31:36.265 INFO:teuthology.orchestra.run.smithi203.stderr:/var/log/ceph/ceph-client.admin.34801.log: gzip -5 --verbose -- /var/log/ceph/ceph-osd.7.log 2024-04-25T10:31:36.265 INFO:teuthology.orchestra.run.smithi203.stderr: 88.5%/var/log/ceph/ceph.audit.log: -- replaced with /var/log/ceph/ceph-client.admin.34801.log.gz 2024-04-25T10:31:36.265 INFO:teuthology.orchestra.run.smithi203.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-mds.b.log 2024-04-25T10:31:36.265 INFO:teuthology.orchestra.run.smithi203.stderr:/var/log/ceph/ceph-osd.7.log: gzip -5 --verbose -- /var/log/ceph/ceph-osd.5.log 2024-04-25T10:31:36.266 INFO:teuthology.orchestra.run.smithi203.stderr:/var/log/ceph/ceph-mds.b.log: 91.7% -- replaced with /var/log/ceph/ceph.log.gz 2024-04-25T10:31:36.266 INFO:teuthology.orchestra.run.smithi203.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-osd.4.log 2024-04-25T10:31:36.266 INFO:teuthology.orchestra.run.smithi203.stderr:/var/log/ceph/ceph-osd.5.log: gzip -5 --verbose -- /var/log/ceph/ceph-mgr.x.log 2024-04-25T10:31:36.266 INFO:teuthology.orchestra.run.smithi203.stderr:/var/log/ceph/ceph-osd.4.log: gzip -5 --verbose -- /var/log/ceph/ceph-mon.c.log 2024-04-25T10:31:36.266 INFO:teuthology.orchestra.run.smithi203.stderr: 92.2% -- replaced with /var/log/ceph/ceph-mds.b.log.gz 2024-04-25T10:31:36.266 INFO:teuthology.orchestra.run.smithi203.stderr:/var/log/ceph/ceph-mgr.x.log: gzip -5 --verbose -- /var/log/ceph/ceph-osd.6.log 2024-04-25T10:31:36.266 INFO:teuthology.orchestra.run.smithi203.stderr:gzip/var/log/ceph/ceph-mon.c.log: -5 --verbose -- /var/log/ceph/ceph-client.3.41240.log 2024-04-25T10:31:36.266 INFO:teuthology.orchestra.run.smithi203.stderr:/var/log/ceph/ceph-osd.6.log: 93.9%gzip -5 --verbose -- /var/log/ceph/ceph-client.1.40791.log 2024-04-25T10:31:36.266 INFO:teuthology.orchestra.run.smithi203.stderr: -- replaced with /var/log/ceph/ceph.audit.log.gz 2024-04-25T10:31:36.266 INFO:teuthology.orchestra.run.smithi203.stderr:/var/log/ceph/ceph-client.3.41240.log: 90.4% -- replaced with /var/log/ceph/ceph-client.3.41240.log.gz 2024-04-25T10:31:36.266 INFO:teuthology.orchestra.run.smithi203.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.35013.log 2024-04-25T10:31:36.266 INFO:teuthology.orchestra.run.smithi203.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.34841.log 2024-04-25T10:31:36.266 INFO:teuthology.orchestra.run.smithi203.stderr:/var/log/ceph/ceph-client.1.40791.log: /var/log/ceph/ceph-client.admin.35013.log: 88.1% -- replaced with /var/log/ceph/ceph-client.admin.35013.log.gz 2024-04-25T10:31:36.267 INFO:teuthology.orchestra.run.smithi203.stderr: 89.4% -- replaced with /var/log/ceph/ceph-client.1.40791.log.gz 2024-04-25T10:31:36.267 INFO:teuthology.orchestra.run.smithi203.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.34973.log 2024-04-25T10:31:36.267 INFO:teuthology.orchestra.run.smithi203.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.34761.log 2024-04-25T10:31:36.267 INFO:teuthology.orchestra.run.smithi203.stderr:/var/log/ceph/ceph-client.admin.34973.log: /var/log/ceph/ceph-client.admin.34841.log: 87.4% 88.1% -- replaced with /var/log/ceph/ceph-client.admin.34841.log.gz 2024-04-25T10:31:36.267 INFO:teuthology.orchestra.run.smithi203.stderr: -- replaced with /var/log/ceph/ceph-client.admin.34973.log.gz 2024-04-25T10:31:36.267 INFO:teuthology.orchestra.run.smithi203.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-mds.d.log 2024-04-25T10:31:36.267 INFO:teuthology.orchestra.run.smithi203.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-mon.b.log 2024-04-25T10:31:36.267 INFO:teuthology.orchestra.run.smithi203.stderr:/var/log/ceph/ceph-client.admin.34761.log: /var/log/ceph/ceph-mds.d.log: 88.3% -- replaced with /var/log/ceph/ceph-client.admin.34761.log.gz 2024-04-25T10:31:36.267 INFO:teuthology.orchestra.run.smithi203.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.34933.log 2024-04-25T10:31:36.273 INFO:teuthology.orchestra.run.smithi203.stderr:/var/log/ceph/ceph-mon.b.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.34891.log 2024-04-25T10:31:36.273 INFO:teuthology.orchestra.run.smithi203.stderr:/var/log/ceph/ceph-client.admin.34933.log: 86.9% -- replaced with /var/log/ceph/ceph-client.admin.34933.log.gz 2024-04-25T10:31:36.287 INFO:teuthology.orchestra.run.smithi203.stderr: 91.3% -- replaced with /var/log/ceph/ceph-mds.d.log.gz 2024-04-25T10:31:36.292 INFO:teuthology.orchestra.run.smithi203.stderr:/var/log/ceph/ceph-client.admin.34891.log: 88.6% -- replaced with /var/log/ceph/ceph-client.admin.34891.log.gz 2024-04-25T10:31:36.305 INFO:teuthology.orchestra.run.smithi203.stderr: 91.4% -- replaced with /var/log/ceph/ceph-mgr.x.log.gz 2024-04-25T10:31:36.380 INFO:teuthology.orchestra.run.smithi203.stderr: 91.5% -- replaced with /var/log/ceph/ceph-mon.b.log.gz 2024-04-25T10:31:36.387 INFO:teuthology.orchestra.run.smithi203.stderr: 91.4% -- replaced with /var/log/ceph/ceph-mon.c.log.gz 2024-04-25T10:31:36.389 INFO:teuthology.orchestra.run.smithi037.stderr:/var/log/ceph/ceph-client.admin.52311.log: 89.1% -- replaced with /var/log/ceph/ceph-client.admin.46095.log.gz 2024-04-25T10:31:36.389 INFO:teuthology.orchestra.run.smithi037.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.46693.log 2024-04-25T10:31:36.389 INFO:teuthology.orchestra.run.smithi037.stderr: 88.3% -- replaced with /var/log/ceph/ceph-client.admin.52311.log.gz 2024-04-25T10:31:36.389 INFO:teuthology.orchestra.run.smithi037.stderr:/var/log/ceph/ceph-client.admin.53827.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.52605.log 2024-04-25T10:31:36.389 INFO:teuthology.orchestra.run.smithi037.stderr:/var/log/ceph/ceph-client.admin.46693.log: 87.9% -- replaced with /var/log/ceph/ceph-client.admin.53827.log.gzgzip 2024-04-25T10:31:36.389 INFO:teuthology.orchestra.run.smithi037.stderr: -5 --verbose -- /var/log/ceph/ceph-client.admin.47156.log 2024-04-25T10:31:36.389 INFO:teuthology.orchestra.run.smithi037.stderr:/var/log/ceph/ceph-client.admin.52605.log: 88.7% -- replaced with /var/log/ceph/ceph-client.admin.46693.log.gz 2024-04-25T10:31:36.389 INFO:teuthology.orchestra.run.smithi037.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.48487.log 2024-04-25T10:31:36.389 INFO:teuthology.orchestra.run.smithi037.stderr: 89.2% -- replaced with /var/log/ceph/ceph-client.admin.52605.log.gz 2024-04-25T10:31:36.390 INFO:teuthology.orchestra.run.smithi037.stderr:/var/log/ceph/ceph-client.admin.47156.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.44392.log 2024-04-25T10:31:36.390 INFO:teuthology.orchestra.run.smithi037.stderr:/var/log/ceph/ceph-client.admin.48487.log: 88.4% -- replaced with /var/log/ceph/ceph-client.admin.47156.log.gz 2024-04-25T10:31:36.390 INFO:teuthology.orchestra.run.smithi037.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.52176.log 2024-04-25T10:31:36.390 INFO:teuthology.orchestra.run.smithi037.stderr:/var/log/ceph/ceph-client.admin.44392.log: 88.5% -- replaced with /var/log/ceph/ceph-client.admin.48487.log.gz 2024-04-25T10:31:36.390 INFO:teuthology.orchestra.run.smithi037.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.50451.log 2024-04-25T10:31:36.390 INFO:teuthology.orchestra.run.smithi037.stderr: 88.3% -- replaced with /var/log/ceph/ceph-client.admin.44392.log.gz 2024-04-25T10:31:36.390 INFO:teuthology.orchestra.run.smithi037.stderr:/var/log/ceph/ceph-client.admin.52176.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.47446.log 2024-04-25T10:31:36.390 INFO:teuthology.orchestra.run.smithi037.stderr:/var/log/ceph/ceph-client.admin.50451.log: 87.7% -- replaced with /var/log/ceph/ceph-client.admin.52176.log.gz 2024-04-25T10:31:36.390 INFO:teuthology.orchestra.run.smithi037.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.53150.log 2024-04-25T10:31:36.390 INFO:teuthology.orchestra.run.smithi037.stderr: 88.5% -- replaced with /var/log/ceph/ceph-client.admin.50451.log.gz 2024-04-25T10:31:36.390 INFO:teuthology.orchestra.run.smithi037.stderr:/var/log/ceph/ceph-client.admin.47446.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.2.43722.log 2024-04-25T10:31:36.390 INFO:teuthology.orchestra.run.smithi037.stderr: 88.6% -- replaced with /var/log/ceph/ceph-client.admin.47446.log.gz 2024-04-25T10:31:36.390 INFO:teuthology.orchestra.run.smithi037.stderr:/var/log/ceph/ceph-client.admin.53150.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.48403.log 2024-04-25T10:31:36.391 INFO:teuthology.orchestra.run.smithi037.stderr:/var/log/ceph/ceph-client.2.43722.log: 88.9%gzip -- replaced with /var/log/ceph/ceph-client.admin.53150.log.gz 2024-04-25T10:31:36.391 INFO:teuthology.orchestra.run.smithi037.stderr: -5 --verbose -- /var/log/ceph/ceph-client.admin.49987.log 2024-04-25T10:31:36.391 INFO:teuthology.orchestra.run.smithi037.stderr:/var/log/ceph/ceph-client.admin.48403.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.53463.log 2024-04-25T10:31:36.391 INFO:teuthology.orchestra.run.smithi037.stderr: 87.7% -- replaced with /var/log/ceph/ceph-client.admin.48403.log.gz 2024-04-25T10:31:36.391 INFO:teuthology.orchestra.run.smithi037.stderr:/var/log/ceph/ceph-client.admin.49987.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.49945.log 2024-04-25T10:31:36.391 INFO:teuthology.orchestra.run.smithi037.stderr: 90.8% -- replaced with /var/log/ceph/ceph-client.2.43722.log.gz 2024-04-25T10:31:36.391 INFO:teuthology.orchestra.run.smithi037.stderr: 87.8% -- replaced with /var/log/ceph/ceph-client.admin.49987.log.gz 2024-04-25T10:31:36.391 INFO:teuthology.orchestra.run.smithi037.stderr:/var/log/ceph/ceph-client.admin.53463.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.47610.log 2024-04-25T10:31:36.391 INFO:teuthology.orchestra.run.smithi037.stderr:/var/log/ceph/ceph-client.admin.49945.log: 88.5% -- replaced with /var/log/ceph/ceph-client.admin.53463.log.gz 2024-04-25T10:31:36.391 INFO:teuthology.orchestra.run.smithi037.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.50493.log 2024-04-25T10:31:36.391 INFO:teuthology.orchestra.run.smithi037.stderr: 88.2% -- replaced with /var/log/ceph/ceph-client.admin.49945.log.gz 2024-04-25T10:31:36.391 INFO:teuthology.orchestra.run.smithi037.stderr:/var/log/ceph/ceph-client.admin.47610.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.48105.log 2024-04-25T10:31:36.391 INFO:teuthology.orchestra.run.smithi037.stderr: 88.4% -- replaced with /var/log/ceph/ceph-client.admin.47610.log.gz 2024-04-25T10:31:36.391 INFO:teuthology.orchestra.run.smithi037.stderr:/var/log/ceph/ceph-client.admin.50493.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.53589.log 2024-04-25T10:31:36.391 INFO:teuthology.orchestra.run.smithi037.stderr: 88.4% -- replaced with /var/log/ceph/ceph-client.admin.50493.log.gz 2024-04-25T10:31:36.392 INFO:teuthology.orchestra.run.smithi037.stderr:/var/log/ceph/ceph-client.admin.48105.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.45727.log 2024-04-25T10:31:36.392 INFO:teuthology.orchestra.run.smithi037.stderr: 88.4% -- replaced with /var/log/ceph/ceph-client.admin.48105.log.gz 2024-04-25T10:31:36.392 INFO:teuthology.orchestra.run.smithi037.stderr:/var/log/ceph/ceph-client.admin.53589.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.47240.log 2024-04-25T10:31:36.392 INFO:teuthology.orchestra.run.smithi037.stderr:/var/log/ceph/ceph-client.admin.45727.log: 88.6% -- replaced with /var/log/ceph/ceph-client.admin.53589.log.gz 2024-04-25T10:31:36.392 INFO:teuthology.orchestra.run.smithi037.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.50996.log 2024-04-25T10:31:36.392 INFO:teuthology.orchestra.run.smithi037.stderr: 88.7% -- replaced with /var/log/ceph/ceph-client.admin.45727.log.gz 2024-04-25T10:31:36.392 INFO:teuthology.orchestra.run.smithi037.stderr:/var/log/ceph/ceph-client.admin.47240.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.54022.log 2024-04-25T10:31:36.392 INFO:teuthology.orchestra.run.smithi037.stderr: 87.2% -- replaced with /var/log/ceph/ceph-client.admin.47240.log.gz 2024-04-25T10:31:36.392 INFO:teuthology.orchestra.run.smithi037.stderr:/var/log/ceph/ceph-client.admin.50996.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.46237.log 2024-04-25T10:31:36.392 INFO:teuthology.orchestra.run.smithi037.stderr:/var/log/ceph/ceph-client.admin.54022.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.41020.log 2024-04-25T10:31:36.392 INFO:teuthology.orchestra.run.smithi037.stderr: 90.2% -- replaced with /var/log/ceph/ceph-client.admin.50996.log.gz 2024-04-25T10:31:36.392 INFO:teuthology.orchestra.run.smithi037.stderr: 87.7% -- replaced with /var/log/ceph/ceph-client.admin.54022.log.gz 2024-04-25T10:31:36.392 INFO:teuthology.orchestra.run.smithi037.stderr:/var/log/ceph/ceph-client.admin.46237.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.45044.log 2024-04-25T10:31:36.392 INFO:teuthology.orchestra.run.smithi037.stderr: 88.2% -- replaced with /var/log/ceph/ceph-client.admin.46237.log.gz 2024-04-25T10:31:36.393 INFO:teuthology.orchestra.run.smithi037.stderr:/var/log/ceph/ceph-client.admin.41020.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.41870.log 2024-04-25T10:31:36.393 INFO:teuthology.orchestra.run.smithi037.stderr:/var/log/ceph/ceph-client.admin.45044.log: 88.1% -- replaced with /var/log/ceph/ceph-client.admin.41020.log.gz 2024-04-25T10:31:36.393 INFO:teuthology.orchestra.run.smithi037.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.52773.log 2024-04-25T10:31:36.393 INFO:teuthology.orchestra.run.smithi037.stderr:/var/log/ceph/ceph-client.admin.41870.log: 89.0% -- replaced with /var/log/ceph/ceph-client.admin.45044.log.gz 2024-04-25T10:31:36.393 INFO:teuthology.orchestra.run.smithi037.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.52521.log 2024-04-25T10:31:36.393 INFO:teuthology.orchestra.run.smithi037.stderr: 87.6% -- replaced with /var/log/ceph/ceph-client.admin.41870.log.gz 2024-04-25T10:31:36.393 INFO:teuthology.orchestra.run.smithi037.stderr:/var/log/ceph/ceph-client.admin.52773.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.48529.log 2024-04-25T10:31:36.393 INFO:teuthology.orchestra.run.smithi037.stderr:/var/log/ceph/ceph-client.admin.52521.log: 87.9% -- replaced with /var/log/ceph/ceph-client.admin.52773.log.gz 2024-04-25T10:31:36.393 INFO:teuthology.orchestra.run.smithi037.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.46011.log 2024-04-25T10:31:36.393 INFO:teuthology.orchestra.run.smithi037.stderr:/var/log/ceph/ceph-client.admin.48529.log: 88.7% -- replaced with /var/log/ceph/ceph-client.admin.52521.log.gz 2024-04-25T10:31:36.393 INFO:teuthology.orchestra.run.smithi037.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.41190.log 2024-04-25T10:31:36.393 INFO:teuthology.orchestra.run.smithi037.stderr:gzip/var/log/ceph/ceph-client.admin.46011.log: -5 --verbose -- /var/log/ceph/ceph-client.admin.52353.log 2024-04-25T10:31:36.393 INFO:teuthology.orchestra.run.smithi037.stderr:/var/log/ceph/ceph-client.admin.41190.log: 91.1% -- replaced with /var/log/ceph/ceph-client.admin.48529.log.gz 2024-04-25T10:31:36.394 INFO:teuthology.orchestra.run.smithi037.stderr:gzip -5 --verbose -- 87.6% /var/log/ceph/ceph.log 2024-04-25T10:31:36.394 INFO:teuthology.orchestra.run.smithi037.stderr: -- replaced with /var/log/ceph/ceph-client.admin.46011.log.gz 2024-04-25T10:31:36.394 INFO:teuthology.orchestra.run.smithi037.stderr:/var/log/ceph/ceph-client.admin.52353.log: 88.4% -- replaced with /var/log/ceph/ceph-client.admin.41190.log.gz 2024-04-25T10:31:36.394 INFO:teuthology.orchestra.run.smithi037.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.49735.log 2024-04-25T10:31:36.394 INFO:teuthology.orchestra.run.smithi037.stderr:/var/log/ceph/ceph.log: 87.6% -- replaced with /var/log/ceph/ceph-client.admin.52353.log.gz 2024-04-25T10:31:36.394 INFO:teuthology.orchestra.run.smithi037.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.53869.log 2024-04-25T10:31:36.394 INFO:teuthology.orchestra.run.smithi037.stderr:gzip/var/log/ceph/ceph-client.admin.49735.log: -5 --verbose -- /var/log/ceph/ceph-client.admin.47882.log 2024-04-25T10:31:36.394 INFO:teuthology.orchestra.run.smithi037.stderr:/var/log/ceph/ceph-client.admin.53869.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.44308.log 2024-04-25T10:31:36.394 INFO:teuthology.orchestra.run.smithi037.stderr: 85.1% -- replaced with /var/log/ceph/ceph.log.gz 2024-04-25T10:31:36.394 INFO:teuthology.orchestra.run.smithi037.stderr: 88.3% -- replaced with /var/log/ceph/ceph-client.admin.53869.log.gz 2024-04-25T10:31:36.394 INFO:teuthology.orchestra.run.smithi037.stderr:/var/log/ceph/ceph-client.admin.47882.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.45397.log 2024-04-25T10:31:36.394 INFO:teuthology.orchestra.run.smithi037.stderr: 91.6% -- replaced with /var/log/ceph/ceph-client.admin.49735.log.gz 2024-04-25T10:31:36.394 INFO:teuthology.orchestra.run.smithi037.stderr:/var/log/ceph/ceph-client.admin.44308.log: 88.4% -- replaced with /var/log/ceph/ceph-client.admin.47882.log.gz 2024-04-25T10:31:36.394 INFO:teuthology.orchestra.run.smithi037.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-osd.2.log 2024-04-25T10:31:36.394 INFO:teuthology.orchestra.run.smithi037.stderr: 88.7% -- replaced with /var/log/ceph/ceph-client.admin.44308.log.gz 2024-04-25T10:31:36.395 INFO:teuthology.orchestra.run.smithi037.stderr:/var/log/ceph/ceph-client.admin.45397.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.50870.log 2024-04-25T10:31:36.395 INFO:teuthology.orchestra.run.smithi037.stderr: 87.8% -- replaced with /var/log/ceph/ceph-client.admin.45397.log.gz 2024-04-25T10:31:36.395 INFO:teuthology.orchestra.run.smithi037.stderr:/var/log/ceph/ceph-osd.2.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.51504.log 2024-04-25T10:31:36.395 INFO:teuthology.orchestra.run.smithi037.stderr:/var/log/ceph/ceph-client.admin.50870.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.47756.log 2024-04-25T10:31:36.395 INFO:teuthology.orchestra.run.smithi037.stderr:/var/log/ceph/ceph-client.admin.51504.log: gzip -5 --verbose -- /var/log/ceph/ceph.tmp-client.admin.28365.log 2024-04-25T10:31:36.395 INFO:teuthology.orchestra.run.smithi037.stderr:/var/log/ceph/ceph-client.admin.47756.log: 89.0% -- replaced with /var/log/ceph/ceph-client.admin.51504.log.gz 2024-04-25T10:31:36.395 INFO:teuthology.orchestra.run.smithi037.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.50029.log 2024-04-25T10:31:36.395 INFO:teuthology.orchestra.run.smithi037.stderr: 89.2% -- replaced with /var/log/ceph/ceph-client.admin.47756.log.gz 2024-04-25T10:31:36.395 INFO:teuthology.orchestra.run.smithi037.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.44224.log 2024-04-25T10:31:36.395 INFO:teuthology.orchestra.run.smithi037.stderr:/var/log/ceph/ceph.tmp-client.admin.28365.log: 0.0% -- replaced with /var/log/ceph/ceph.tmp-client.admin.28365.log.gz 2024-04-25T10:31:36.395 INFO:teuthology.orchestra.run.smithi037.stderr:/var/log/ceph/ceph-client.admin.50029.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.49490.log 92.4% 2024-04-25T10:31:36.395 INFO:teuthology.orchestra.run.smithi037.stderr: -- replaced with /var/log/ceph/ceph-client.admin.50870.log.gz 2024-04-25T10:31:36.395 INFO:teuthology.orchestra.run.smithi037.stderr:/var/log/ceph/ceph-client.admin.44224.log: 87.5% -- replaced with /var/log/ceph/ceph-client.admin.50029.log.gz 2024-04-25T10:31:36.395 INFO:teuthology.orchestra.run.smithi037.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.40768.log 2024-04-25T10:31:36.396 INFO:teuthology.orchestra.run.smithi037.stderr:/var/log/ceph/ceph-client.admin.49490.log: 88.1% -- replaced with /var/log/ceph/ceph-client.admin.44224.log.gz 2024-04-25T10:31:36.396 INFO:teuthology.orchestra.run.smithi037.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.34659.log 2024-04-25T10:31:36.396 INFO:teuthology.orchestra.run.smithi037.stderr:/var/log/ceph/ceph-client.admin.40768.log: 87.6% -- replaced with /var/log/ceph/ceph-client.admin.49490.log.gz 2024-04-25T10:31:36.396 INFO:teuthology.orchestra.run.smithi037.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.49364.log 2024-04-25T10:31:36.396 INFO:teuthology.orchestra.run.smithi037.stderr:/var/log/ceph/ceph-client.admin.34659.log: 87.4% -- replaced with /var/log/ceph/ceph-client.admin.40768.log.gz 2024-04-25T10:31:36.396 INFO:teuthology.orchestra.run.smithi037.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.45601.log 2024-04-25T10:31:36.396 INFO:teuthology.orchestra.run.smithi037.stderr: 87.8% -- replaced with /var/log/ceph/ceph-client.admin.34659.log.gz 2024-04-25T10:31:36.396 INFO:teuthology.orchestra.run.smithi037.stderr:/var/log/ceph/ceph-client.admin.49364.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.41872.log 2024-04-25T10:31:36.396 INFO:teuthology.orchestra.run.smithi037.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.46987.log 2024-04-25T10:31:36.396 INFO:teuthology.orchestra.run.smithi037.stderr: 87.6% -- replaced with /var/log/ceph/ceph-client.admin.49364.log.gz 2024-04-25T10:31:36.396 INFO:teuthology.orchestra.run.smithi037.stderr:/var/log/ceph/ceph-client.admin.45601.log: /var/log/ceph/ceph-client.admin.41872.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.54487.log 2024-04-25T10:31:36.396 INFO:teuthology.orchestra.run.smithi037.stderr: 89.3% -- replaced with /var/log/ceph/ceph-client.admin.45601.log.gz 2024-04-25T10:31:36.396 INFO:teuthology.orchestra.run.smithi037.stderr:/var/log/ceph/ceph-client.admin.46987.log: 88.2% -- replaced with /var/log/ceph/ceph-client.admin.41872.log.gz 2024-04-25T10:31:36.396 INFO:teuthology.orchestra.run.smithi037.stderr:gzip -5 --verbose -- /var/log/ceph/ceph.audit.log 2024-04-25T10:31:36.396 INFO:teuthology.orchestra.run.smithi037.stderr: 88.4% -- replaced with /var/log/ceph/ceph-client.admin.46987.log.gz 2024-04-25T10:31:36.397 INFO:teuthology.orchestra.run.smithi037.stderr:gzip/var/log/ceph/ceph-client.admin.54487.log: -5 --verbose -- /var/log/ceph/ceph-client.admin.47366.log 2024-04-25T10:31:36.397 INFO:teuthology.orchestra.run.smithi037.stderr:/var/log/ceph/ceph.audit.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.49901.log 2024-04-25T10:31:36.397 INFO:teuthology.orchestra.run.smithi037.stderr: 88.6% -- replaced with /var/log/ceph/ceph-client.admin.54487.log.gz 2024-04-25T10:31:36.397 INFO:teuthology.orchestra.run.smithi037.stderr:/var/log/ceph/ceph-client.admin.47366.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.42802.log 2024-04-25T10:31:36.397 INFO:teuthology.orchestra.run.smithi037.stderr: 87.7% -- replaced with /var/log/ceph/ceph-client.admin.47366.log.gz 2024-04-25T10:31:36.397 INFO:teuthology.orchestra.run.smithi037.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.44350.log 2024-04-25T10:31:36.397 INFO:teuthology.orchestra.run.smithi037.stderr:/var/log/ceph/ceph-client.admin.49901.log: /var/log/ceph/ceph-client.admin.42802.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.53421.log 2024-04-25T10:31:36.397 INFO:teuthology.orchestra.run.smithi037.stderr: 89.4% -- replaced with /var/log/ceph/ceph-client.admin.49901.log.gz 2024-04-25T10:31:36.397 INFO:teuthology.orchestra.run.smithi037.stderr: 88.5% -- replaced with /var/log/ceph/ceph-client.admin.42802.log.gz 2024-04-25T10:31:36.397 INFO:teuthology.orchestra.run.smithi037.stderr:/var/log/ceph/ceph-client.admin.44350.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.48189.log 2024-04-25T10:31:36.397 INFO:teuthology.orchestra.run.smithi037.stderr: 89.3% -- replaced with /var/log/ceph/ceph.audit.log.gz 2024-04-25T10:31:36.397 INFO:teuthology.orchestra.run.smithi037.stderr: 88.4% -- replaced with /var/log/ceph/ceph-client.admin.44350.log.gz 2024-04-25T10:31:36.397 INFO:teuthology.orchestra.run.smithi037.stderr:/var/log/ceph/ceph-client.admin.53421.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.42574.log 2024-04-25T10:31:36.397 INFO:teuthology.orchestra.run.smithi037.stderr: 88.6% -- replaced with /var/log/ceph/ceph-client.admin.53421.log.gz 2024-04-25T10:31:36.398 INFO:teuthology.orchestra.run.smithi037.stderr:/var/log/ceph/ceph-client.admin.48189.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.50661.log 2024-04-25T10:31:36.398 INFO:teuthology.orchestra.run.smithi037.stderr:/var/log/ceph/ceph-client.admin.42574.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.53318.log 2024-04-25T10:31:36.398 INFO:teuthology.orchestra.run.smithi037.stderr: 87.5% -- replaced with /var/log/ceph/ceph-client.admin.48189.log.gz 2024-04-25T10:31:36.398 INFO:teuthology.orchestra.run.smithi037.stderr:/var/log/ceph/ceph-client.admin.50661.log: 88.1% -- replaced with /var/log/ceph/ceph-client.admin.42574.log.gz 2024-04-25T10:31:36.398 INFO:teuthology.orchestra.run.smithi037.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.40158.log 2024-04-25T10:31:36.398 INFO:teuthology.orchestra.run.smithi037.stderr: 88.4% -- replaced with /var/log/ceph/ceph-client.admin.50661.log.gz 2024-04-25T10:31:36.398 INFO:teuthology.orchestra.run.smithi037.stderr:/var/log/ceph/ceph-client.admin.53318.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.50115.log 2024-04-25T10:31:36.398 INFO:teuthology.orchestra.run.smithi037.stderr: 89.2%/var/log/ceph/ceph-client.admin.40158.log: -- replaced with /var/log/ceph/ceph-client.admin.53318.log.gz 2024-04-25T10:31:36.398 INFO:teuthology.orchestra.run.smithi037.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.47071.log 2024-04-25T10:31:36.398 INFO:teuthology.orchestra.run.smithi037.stderr:/var/log/ceph/ceph-client.admin.50115.log: gzip -5 88.4% --verbose -- -- replaced with /var/log/ceph/ceph-client.admin.40158.log.gz /var/log/ceph/ceph-client.admin.51288.log 2024-04-25T10:31:36.398 INFO:teuthology.orchestra.run.smithi037.stderr: 2024-04-25T10:31:36.398 INFO:teuthology.orchestra.run.smithi037.stderr: 88.0% -- replaced with /var/log/ceph/ceph-client.admin.50115.log.gz 2024-04-25T10:31:36.398 INFO:teuthology.orchestra.run.smithi037.stderr:/var/log/ceph/ceph-client.admin.47071.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.53972.log 2024-04-25T10:31:36.398 INFO:teuthology.orchestra.run.smithi037.stderr:/var/log/ceph/ceph-client.admin.51288.log: 89.2% -- replaced with /var/log/ceph/ceph-client.admin.47071.log.gz 2024-04-25T10:31:36.399 INFO:teuthology.orchestra.run.smithi037.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.49448.log 2024-04-25T10:31:36.399 INFO:teuthology.orchestra.run.smithi037.stderr:/var/log/ceph/ceph-client.admin.53972.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.0.43223.log 2024-04-25T10:31:36.399 INFO:teuthology.orchestra.run.smithi037.stderr: 91.3% -- replaced with /var/log/ceph/ceph-client.admin.51288.log.gz 2024-04-25T10:31:36.399 INFO:teuthology.orchestra.run.smithi037.stderr: 88.2% -- replaced with /var/log/ceph/ceph-client.admin.53972.log.gz/var/log/ceph/ceph-client.admin.49448.log: 2024-04-25T10:31:36.399 INFO:teuthology.orchestra.run.smithi037.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-osd.3.log 2024-04-25T10:31:36.399 INFO:teuthology.orchestra.run.smithi037.stderr: 89.3% -- replaced with /var/log/ceph/ceph-client.admin.49448.log.gz 2024-04-25T10:31:36.399 INFO:teuthology.orchestra.run.smithi037.stderr:/var/log/ceph/ceph-client.0.43223.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.51246.log 2024-04-25T10:31:36.399 INFO:teuthology.orchestra.run.smithi037.stderr:/var/log/ceph/ceph-osd.3.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.46945.log 2024-04-25T10:31:36.399 INFO:teuthology.orchestra.run.smithi037.stderr:/var/log/ceph/ceph-client.admin.51246.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.44518.log 2024-04-25T10:31:36.399 INFO:teuthology.orchestra.run.smithi037.stderr: 88.6% -- replaced with /var/log/ceph/ceph-client.admin.51246.log.gz 2024-04-25T10:31:36.399 INFO:teuthology.orchestra.run.smithi037.stderr:/var/log/ceph/ceph-client.admin.46945.log: 90.4% -- replaced with /var/log/ceph/ceph-client.0.43223.log.gz 2024-04-25T10:31:36.399 INFO:teuthology.orchestra.run.smithi037.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.46819.log 2024-04-25T10:31:36.399 INFO:teuthology.orchestra.run.smithi037.stderr: 88.0% -- replaced with /var/log/ceph/ceph-client.admin.46945.log.gz 2024-04-25T10:31:36.399 INFO:teuthology.orchestra.run.smithi037.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.40328.log 2024-04-25T10:31:36.400 INFO:teuthology.orchestra.run.smithi037.stderr:/var/log/ceph/ceph-client.admin.44518.log: /var/log/ceph/ceph-client.admin.46819.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.53785.log 2024-04-25T10:31:36.400 INFO:teuthology.orchestra.run.smithi037.stderr: 88.8% -- replaced with /var/log/ceph/ceph-client.admin.44518.log.gz 2024-04-25T10:31:36.400 INFO:teuthology.orchestra.run.smithi037.stderr:/var/log/ceph/ceph-client.admin.40328.log: 88.8%gzip -- replaced with /var/log/ceph/ceph-client.admin.46819.log.gz -5 2024-04-25T10:31:36.400 INFO:teuthology.orchestra.run.smithi037.stderr: --verbose -- /var/log/ceph/ceph-client.admin.46053.log 2024-04-25T10:31:36.400 INFO:teuthology.orchestra.run.smithi037.stderr: 87.4% -- replaced with /var/log/ceph/ceph-client.admin.40328.log.gz 2024-04-25T10:31:36.400 INFO:teuthology.orchestra.run.smithi037.stderr:/var/log/ceph/ceph-client.admin.53785.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.47568.log 2024-04-25T10:31:36.400 INFO:teuthology.orchestra.run.smithi037.stderr:/var/log/ceph/ceph-client.admin.46053.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.40600.log 2024-04-25T10:31:36.400 INFO:teuthology.orchestra.run.smithi037.stderr: 88.9% -- replaced with /var/log/ceph/ceph-client.admin.53785.log.gz 2024-04-25T10:31:36.400 INFO:teuthology.orchestra.run.smithi037.stderr: 88.2% -- replaced with /var/log/ceph/ceph-client.admin.46053.log.gz 2024-04-25T10:31:36.400 INFO:teuthology.orchestra.run.smithi037.stderr:/var/log/ceph/ceph-client.admin.47568.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.53192.log 2024-04-25T10:31:36.400 INFO:teuthology.orchestra.run.smithi037.stderr:/var/log/ceph/ceph-client.admin.40600.log: 87.6% -- replaced with /var/log/ceph/ceph-client.admin.47568.log.gz 2024-04-25T10:31:36.400 INFO:teuthology.orchestra.run.smithi037.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.54445.log 2024-04-25T10:31:36.400 INFO:teuthology.orchestra.run.smithi037.stderr: 88.7% -- replaced with /var/log/ceph/ceph-client.admin.40600.log.gz 2024-04-25T10:31:36.400 INFO:teuthology.orchestra.run.smithi037.stderr:/var/log/ceph/ceph-client.admin.53192.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.44164.log 2024-04-25T10:31:36.401 INFO:teuthology.orchestra.run.smithi037.stderr: 88.4% -- replaced with /var/log/ceph/ceph-client.admin.53192.log.gz 2024-04-25T10:31:36.401 INFO:teuthology.orchestra.run.smithi037.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.50744.log 2024-04-25T10:31:36.401 INFO:teuthology.orchestra.run.smithi037.stderr:/var/log/ceph/ceph-client.admin.54445.log: /var/log/ceph/ceph-client.admin.44164.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.42322.log 2024-04-25T10:31:36.401 INFO:teuthology.orchestra.run.smithi037.stderr: 88.3% -- replaced with /var/log/ceph/ceph-client.admin.54445.log.gz 2024-04-25T10:31:36.401 INFO:teuthology.orchestra.run.smithi037.stderr: 89.3% -- replaced with /var/log/ceph/ceph-client.admin.44164.log.gz 2024-04-25T10:31:36.401 INFO:teuthology.orchestra.run.smithi037.stderr:/var/log/ceph/ceph-client.admin.50744.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.42532.log 2024-04-25T10:31:36.401 INFO:teuthology.orchestra.run.smithi037.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.41200.log 2024-04-25T10:31:36.401 INFO:teuthology.orchestra.run.smithi037.stderr:/var/log/ceph/ceph-client.admin.42322.log: /var/log/ceph/ceph-client.admin.42532.log: 90.5% -- replaced with /var/log/ceph/ceph-client.admin.50744.log.gz 2024-04-25T10:31:36.401 INFO:teuthology.orchestra.run.smithi037.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.40432.log 2024-04-25T10:31:36.401 INFO:teuthology.orchestra.run.smithi037.stderr: 87.5% -- replaced with /var/log/ceph/ceph-client.admin.42322.log.gz 2024-04-25T10:31:36.401 INFO:teuthology.orchestra.run.smithi037.stderr: 87.9% -- replaced with /var/log/ceph/ceph-client.admin.42532.log.gz 2024-04-25T10:31:36.401 INFO:teuthology.orchestra.run.smithi037.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.47114.log 2024-04-25T10:31:36.401 INFO:teuthology.orchestra.run.smithi037.stderr:/var/log/ceph/ceph-client.admin.41200.log: /var/log/ceph/ceph-client.admin.40432.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.50367.log 2024-04-25T10:31:36.401 INFO:teuthology.orchestra.run.smithi037.stderr: 88.8% -- replaced with /var/log/ceph/ceph-client.admin.41200.log.gz 2024-04-25T10:31:36.402 INFO:teuthology.orchestra.run.smithi037.stderr: 88.1% -- replaced with /var/log/ceph/ceph-client.admin.40432.log.gz 2024-04-25T10:31:36.402 INFO:teuthology.orchestra.run.smithi037.stderr:/var/log/ceph/ceph-client.admin.47114.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.45769.log 2024-04-25T10:31:36.402 INFO:teuthology.orchestra.run.smithi037.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.46861.log 2024-04-25T10:31:36.402 INFO:teuthology.orchestra.run.smithi037.stderr:/var/log/ceph/ceph-client.admin.50367.log: 88.8% -- replaced with /var/log/ceph/ceph-client.admin.47114.log.gz 2024-04-25T10:31:36.402 INFO:teuthology.orchestra.run.smithi037.stderr:/var/log/ceph/ceph-client.admin.45769.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.44266.log 2024-04-25T10:31:36.402 INFO:teuthology.orchestra.run.smithi037.stderr: 89.4% -- replaced with /var/log/ceph/ceph-client.admin.50367.log.gz 2024-04-25T10:31:36.402 INFO:teuthology.orchestra.run.smithi037.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-mds.c.log 2024-04-25T10:31:36.402 INFO:teuthology.orchestra.run.smithi037.stderr:/var/log/ceph/ceph-client.admin.46861.log: 89.2% -- replaced with /var/log/ceph/ceph-client.admin.45769.log.gz 2024-04-25T10:31:36.402 INFO:teuthology.orchestra.run.smithi037.stderr:/var/log/ceph/ceph-client.admin.44266.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.53743.log 2024-04-25T10:31:36.402 INFO:teuthology.orchestra.run.smithi037.stderr: 88.8% -- replaced with /var/log/ceph/ceph-client.admin.46861.log.gz 2024-04-25T10:31:36.402 INFO:teuthology.orchestra.run.smithi037.stderr: 88.4% -- replaced with /var/log/ceph/ceph-client.admin.44266.log.gz 2024-04-25T10:31:36.402 INFO:teuthology.orchestra.run.smithi037.stderr:/var/log/ceph/ceph-mds.c.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.40242.log 2024-04-25T10:31:36.402 INFO:teuthology.orchestra.run.smithi037.stderr:/var/log/ceph/ceph-client.admin.53743.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.42616.log 2024-04-25T10:31:36.403 INFO:teuthology.orchestra.run.smithi037.stderr: 87.5% -- replaced with /var/log/ceph/ceph-client.admin.53743.log.gz 2024-04-25T10:31:36.403 INFO:teuthology.orchestra.run.smithi037.stderr:/var/log/ceph/ceph-client.admin.40242.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.50325.log 2024-04-25T10:31:36.403 INFO:teuthology.orchestra.run.smithi037.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.41535.log 88.7% 2024-04-25T10:31:36.403 INFO:teuthology.orchestra.run.smithi037.stderr: -- replaced with /var/log/ceph/ceph-client.admin.40242.log.gz 2024-04-25T10:31:36.403 INFO:teuthology.orchestra.run.smithi037.stderr:/var/log/ceph/ceph-client.admin.42616.log: /var/log/ceph/ceph-client.admin.50325.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.47526.log 2024-04-25T10:31:36.403 INFO:teuthology.orchestra.run.smithi037.stderr: 87.8% -- replaced with /var/log/ceph/ceph-client.admin.42616.log.gz 2024-04-25T10:31:36.403 INFO:teuthology.orchestra.run.smithi037.stderr:/var/log/ceph/ceph-client.admin.41535.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.40894.log 2024-04-25T10:31:36.403 INFO:teuthology.orchestra.run.smithi037.stderr: 88.9% -- replaced with /var/log/ceph/ceph-client.admin.50325.log.gz 2024-04-25T10:31:36.403 INFO:teuthology.orchestra.run.smithi037.stderr:/var/log/ceph/ceph-client.admin.47526.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.45355.log 2024-04-25T10:31:36.403 INFO:teuthology.orchestra.run.smithi037.stderr: 88.0% -- replaced with /var/log/ceph/ceph-client.admin.41535.log.gz 2024-04-25T10:31:36.403 INFO:teuthology.orchestra.run.smithi037.stderr:/var/log/ceph/ceph-client.admin.40894.log: gzip -5 --verbose -- /var/log/ceph/ceph-osd.1.log 2024-04-25T10:31:36.403 INFO:teuthology.orchestra.run.smithi037.stderr: 88.5% -- replaced with /var/log/ceph/ceph-client.admin.47526.log.gz 2024-04-25T10:31:36.403 INFO:teuthology.orchestra.run.smithi037.stderr:/var/log/ceph/ceph-client.admin.45355.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.52899.log 2024-04-25T10:31:36.403 INFO:teuthology.orchestra.run.smithi037.stderr: 88.0% -- replaced with /var/log/ceph/ceph-client.admin.40894.log.gz 2024-04-25T10:31:36.404 INFO:teuthology.orchestra.run.smithi037.stderr: 88.2% -- replaced with /var/log/ceph/ceph-client.admin.45355.log.gz 2024-04-25T10:31:36.404 INFO:teuthology.orchestra.run.smithi037.stderr:/var/log/ceph/ceph-osd.1.log: gzip -5 --verbose -- /var/log/ceph/ceph-mgr.y.log 2024-04-25T10:31:36.404 INFO:teuthology.orchestra.run.smithi037.stderr:gzip/var/log/ceph/ceph-client.admin.52899.log: -5 --verbose -- /var/log/ceph/ceph-client.admin.42364.log 2024-04-25T10:31:36.404 INFO:teuthology.orchestra.run.smithi037.stderr:/var/log/ceph/ceph-mgr.y.log: 86.5% -- replaced with /var/log/ceph/ceph-client.admin.52899.log.gz 2024-04-25T10:31:36.404 INFO:teuthology.orchestra.run.smithi037.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-osd.0.log 2024-04-25T10:31:36.404 INFO:teuthology.orchestra.run.smithi037.stderr:/var/log/ceph/ceph-client.admin.42364.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.43996.log 2024-04-25T10:31:36.404 INFO:teuthology.orchestra.run.smithi037.stderr:/var/log/ceph/ceph-osd.0.log: 89.0% -- replaced with /var/log/ceph/ceph-client.admin.42364.log.gz 2024-04-25T10:31:36.404 INFO:teuthology.orchestra.run.smithi037.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.40073.log 2024-04-25T10:31:36.404 INFO:teuthology.orchestra.run.smithi037.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.54613.log 2024-04-25T10:31:36.404 INFO:teuthology.orchestra.run.smithi037.stderr:/var/log/ceph/ceph-client.admin.43996.log: /var/log/ceph/ceph-client.admin.40073.log: 88.7% -- replaced with /var/log/ceph/ceph-client.admin.43996.log.gz 2024-04-25T10:31:36.404 INFO:teuthology.orchestra.run.smithi037.stderr: 88.6% -- replaced with /var/log/ceph/ceph-client.admin.40073.log.gzgzip -5 --verbose -- /var/log/ceph/ceph-client.admin.43886.log 2024-04-25T10:31:36.404 INFO:teuthology.orchestra.run.smithi037.stderr: 2024-04-25T10:31:36.404 INFO:teuthology.orchestra.run.smithi037.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.49222.log 2024-04-25T10:31:36.404 INFO:teuthology.orchestra.run.smithi037.stderr:/var/log/ceph/ceph-client.admin.54613.log: /var/log/ceph/ceph-client.admin.43886.log: 88.3% -- replaced with /var/log/ceph/ceph-client.admin.54613.log.gz 2024-04-25T10:31:36.405 INFO:teuthology.orchestra.run.smithi037.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.40116.log 2024-04-25T10:31:36.405 INFO:teuthology.orchestra.run.smithi037.stderr: 88.8% -- replaced with /var/log/ceph/ceph-client.admin.43886.log.gz 2024-04-25T10:31:36.405 INFO:teuthology.orchestra.run.smithi037.stderr:/var/log/ceph/ceph-client.admin.49222.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.54737.log 2024-04-25T10:31:36.405 INFO:teuthology.orchestra.run.smithi037.stderr:/var/log/ceph/ceph-client.admin.40116.log: 92.9% -- replaced with /var/log/ceph/ceph-mgr.y.log.gz 2024-04-25T10:31:36.405 INFO:teuthology.orchestra.run.smithi037.stderr: 87.8% -- replaced with /var/log/ceph/ceph-client.admin.49222.log.gz 2024-04-25T10:31:36.405 INFO:teuthology.orchestra.run.smithi037.stderr: 89.3% -- replaced with /var/log/ceph/ceph-client.admin.40116.log.gz 2024-04-25T10:31:36.405 INFO:teuthology.orchestra.run.smithi037.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.52050.log 2024-04-25T10:31:36.405 INFO:teuthology.orchestra.run.smithi037.stderr:/var/log/ceph/ceph-client.admin.54737.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.52689.log 2024-04-25T10:31:36.405 INFO:teuthology.orchestra.run.smithi037.stderr: 89.4%/var/log/ceph/ceph-client.admin.52050.log: -- replaced with /var/log/ceph/ceph-client.admin.54737.log.gz 2024-04-25T10:31:36.405 INFO:teuthology.orchestra.run.smithi037.stderr:gzip -5 --verbose -- 89.2% -- replaced with /var/log/ceph/ceph-client.admin.52050.log.gz 2024-04-25T10:31:36.405 INFO:teuthology.orchestra.run.smithi037.stderr: /var/log/ceph/ceph-client.admin.45559.log 2024-04-25T10:31:36.405 INFO:teuthology.orchestra.run.smithi037.stderr:/var/log/ceph/ceph-client.admin.52689.log: gzip -5 88.6% -- replaced with /var/log/ceph/ceph-client.admin.52689.log.gz 2024-04-25T10:31:36.405 INFO:teuthology.orchestra.run.smithi037.stderr: --verbose -- /var/log/ceph/ceph-client.admin.50912.log 2024-04-25T10:31:36.405 INFO:teuthology.orchestra.run.smithi037.stderr:/var/log/ceph/ceph-client.admin.45559.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.41531.log 2024-04-25T10:31:36.406 INFO:teuthology.orchestra.run.smithi037.stderr: 87.4% -- replaced with /var/log/ceph/ceph-client.admin.45559.log.gz 2024-04-25T10:31:36.406 INFO:teuthology.orchestra.run.smithi037.stderr:/var/log/ceph/ceph-client.admin.50912.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.46609.log 2024-04-25T10:31:36.406 INFO:teuthology.orchestra.run.smithi037.stderr:/var/log/ceph/ceph-client.admin.41531.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.47198.log 2024-04-25T10:31:36.406 INFO:teuthology.orchestra.run.smithi037.stderr: 87.9% -- replaced with /var/log/ceph/ceph-client.admin.41531.log.gz 2024-04-25T10:31:36.406 INFO:teuthology.orchestra.run.smithi037.stderr:/var/log/ceph/ceph-client.admin.46609.log: 88.6% -- replaced with /var/log/ceph/ceph-client.admin.46609.log.gzgzip -5 --verbose -- 92.4% -- replaced with /var/log/ceph/ceph-client.admin.50912.log.gz 2024-04-25T10:31:36.406 INFO:teuthology.orchestra.run.smithi037.stderr: /var/log/ceph/ceph-client.admin.45685.log 2024-04-25T10:31:36.406 INFO:teuthology.orchestra.run.smithi037.stderr:/var/log/ceph/ceph-client.admin.47198.log: 2024-04-25T10:31:36.406 INFO:teuthology.orchestra.run.smithi037.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.52092.log 2024-04-25T10:31:36.406 INFO:teuthology.orchestra.run.smithi037.stderr: 87.9% -- replaced with /var/log/ceph/ceph-client.admin.47198.log.gz 2024-04-25T10:31:36.406 INFO:teuthology.orchestra.run.smithi037.stderr:/var/log/ceph/ceph-client.admin.45685.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.49568.log 88.8% -- replaced with /var/log/ceph/ceph-client.admin.45685.log.gz 2024-04-25T10:31:36.406 INFO:teuthology.orchestra.run.smithi037.stderr: 2024-04-25T10:31:36.406 INFO:teuthology.orchestra.run.smithi037.stderr:/var/log/ceph/ceph-client.admin.52092.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.54571.log 2024-04-25T10:31:36.406 INFO:teuthology.orchestra.run.smithi037.stderr: 88.6% -- replaced with /var/log/ceph/ceph-client.admin.52092.log.gz 2024-04-25T10:31:36.407 INFO:teuthology.orchestra.run.smithi037.stderr:/var/log/ceph/ceph-client.admin.49568.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.41199.log 2024-04-25T10:31:36.407 INFO:teuthology.orchestra.run.smithi037.stderr: 89.1% -- replaced with /var/log/ceph/ceph-client.admin.49568.log.gz/var/log/ceph/ceph-client.admin.54571.log: 2024-04-25T10:31:36.407 INFO:teuthology.orchestra.run.smithi037.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.50409.log 2024-04-25T10:31:36.407 INFO:teuthology.orchestra.run.smithi037.stderr: 87.7% -- replaced with /var/log/ceph/ceph-client.admin.54571.log.gz/var/log/ceph/ceph-client.admin.41199.log: 2024-04-25T10:31:36.407 INFO:teuthology.orchestra.run.smithi037.stderr:gzip -5 --verbose -- 87.9% -- replaced with /var/log/ceph/ceph-client.admin.41199.log.gz 2024-04-25T10:31:36.407 INFO:teuthology.orchestra.run.smithi037.stderr: /var/log/ceph/ceph-client.admin.44770.log 2024-04-25T10:31:36.407 INFO:teuthology.orchestra.run.smithi037.stderr:/var/log/ceph/ceph-client.admin.50409.log: gzip -5 --verbose -- 88.7% -- replaced with /var/log/ceph/ceph-client.admin.50409.log.gz 2024-04-25T10:31:36.407 INFO:teuthology.orchestra.run.smithi037.stderr: /var/log/ceph/ceph-client.admin.46651.log 2024-04-25T10:31:36.407 INFO:teuthology.orchestra.run.smithi037.stderr:/var/log/ceph/ceph-client.admin.44770.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.34762.log 88.6% -- replaced with /var/log/ceph/ceph-client.admin.44770.log.gz 2024-04-25T10:31:36.407 INFO:teuthology.orchestra.run.smithi037.stderr: 2024-04-25T10:31:36.407 INFO:teuthology.orchestra.run.smithi037.stderr:/var/log/ceph/ceph-client.admin.46651.log: gzip -5 --verbose 88.7% -- replaced with /var/log/ceph/ceph-client.admin.46651.log.gz 2024-04-25T10:31:36.407 INFO:teuthology.orchestra.run.smithi037.stderr: -- /var/log/ceph/ceph-client.admin.45313.log 2024-04-25T10:31:36.407 INFO:teuthology.orchestra.run.smithi037.stderr:/var/log/ceph/ceph-client.admin.34762.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.53108.log 88.1% -- replaced with /var/log/ceph/ceph-client.admin.34762.log.gz 2024-04-25T10:31:36.407 INFO:teuthology.orchestra.run.smithi037.stderr: 2024-04-25T10:31:36.407 INFO:teuthology.orchestra.run.smithi037.stderr:/var/log/ceph/ceph-client.admin.45313.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.45927.log 88.2% -- replaced with /var/log/ceph/ceph-client.admin.45313.log.gz 2024-04-25T10:31:36.408 INFO:teuthology.orchestra.run.smithi037.stderr: 2024-04-25T10:31:36.408 INFO:teuthology.orchestra.run.smithi037.stderr:/var/log/ceph/ceph-client.admin.53108.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.54277.log 2024-04-25T10:31:36.408 INFO:teuthology.orchestra.run.smithi037.stderr: 89.1% -- replaced with /var/log/ceph/ceph-client.admin.53108.log.gz 2024-04-25T10:31:36.408 INFO:teuthology.orchestra.run.smithi037.stderr:/var/log/ceph/ceph-client.admin.45927.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.54150.log 2024-04-25T10:31:36.408 INFO:teuthology.orchestra.run.smithi037.stderr: 87.4% -- replaced with /var/log/ceph/ceph-client.admin.45927.log.gz 2024-04-25T10:31:36.408 INFO:teuthology.orchestra.run.smithi037.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.50702.log 2024-04-25T10:31:36.408 INFO:teuthology.orchestra.run.smithi037.stderr:/var/log/ceph/ceph-client.admin.54277.log: /var/log/ceph/ceph-client.admin.54150.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.42112.log 2024-04-25T10:31:36.408 INFO:teuthology.orchestra.run.smithi037.stderr: 88.5% -- replaced with /var/log/ceph/ceph-client.admin.54277.log.gz 2024-04-25T10:31:36.408 INFO:teuthology.orchestra.run.smithi037.stderr:/var/log/ceph/ceph-client.admin.50702.log: 88.4% -- replaced with /var/log/ceph/ceph-client.admin.54150.log.gz 2024-04-25T10:31:36.408 INFO:teuthology.orchestra.run.smithi037.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.44434.log 90.9% -- replaced with /var/log/ceph/ceph-client.admin.50702.log.gz 2024-04-25T10:31:36.408 INFO:teuthology.orchestra.run.smithi037.stderr: 2024-04-25T10:31:36.408 INFO:teuthology.orchestra.run.smithi037.stderr:/var/log/ceph/ceph-client.admin.42112.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.47282.log 2024-04-25T10:31:36.408 INFO:teuthology.orchestra.run.smithi037.stderr: 88.2% -- replaced with /var/log/ceph/ceph-client.admin.42112.log.gz 2024-04-25T10:31:36.408 INFO:teuthology.orchestra.run.smithi037.stderr:/var/log/ceph/ceph-client.admin.44434.log: gzip -5 --verbose 88.3% -- replaced with /var/log/ceph/ceph-client.admin.44434.log.gz 2024-04-25T10:31:36.409 INFO:teuthology.orchestra.run.smithi037.stderr: -- /var/log/ceph/ceph-client.admin.51105.log 2024-04-25T10:31:36.409 INFO:teuthology.orchestra.run.smithi037.stderr:/var/log/ceph/ceph-client.admin.47282.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.54403.log 2024-04-25T10:31:36.409 INFO:teuthology.orchestra.run.smithi037.stderr: 88.8% -- replaced with /var/log/ceph/ceph-client.admin.47282.log.gz 2024-04-25T10:31:36.409 INFO:teuthology.orchestra.run.smithi037.stderr:/var/log/ceph/ceph-client.admin.51105.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.40852.log 2024-04-25T10:31:36.409 INFO:teuthology.orchestra.run.smithi037.stderr: 88.3% -- replaced with /var/log/ceph/ceph-client.admin.51105.log.gz 2024-04-25T10:31:36.409 INFO:teuthology.orchestra.run.smithi037.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.44686.log 2024-04-25T10:31:36.409 INFO:teuthology.orchestra.run.smithi037.stderr:/var/log/ceph/ceph-client.admin.54403.log: /var/log/ceph/ceph-client.admin.40852.log: gzip -5 --verbose -- /var/log/ceph/ceph-mon.a.log 2024-04-25T10:31:36.409 INFO:teuthology.orchestra.run.smithi037.stderr: 88.5% -- replaced with /var/log/ceph/ceph-client.admin.54403.log.gz 2024-04-25T10:31:36.409 INFO:teuthology.orchestra.run.smithi037.stderr: 88.1%/var/log/ceph/ceph-client.admin.44686.log: -- replaced with /var/log/ceph/ceph-client.admin.40852.log.gz 2024-04-25T10:31:36.409 INFO:teuthology.orchestra.run.smithi037.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.45271.log 2024-04-25T10:31:36.409 INFO:teuthology.orchestra.run.smithi037.stderr: 89.3% -- replaced with /var/log/ceph/ceph-client.admin.44686.log.gz 2024-04-25T10:31:36.409 INFO:teuthology.orchestra.run.smithi037.stderr:/var/log/ceph/ceph-mon.a.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.54529.log 2024-04-25T10:31:36.409 INFO:teuthology.orchestra.run.smithi037.stderr:/var/log/ceph/ceph-client.admin.45271.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.49406.log 2024-04-25T10:31:36.409 INFO:teuthology.orchestra.run.smithi037.stderr: 87.6%/var/log/ceph/ceph-client.admin.54529.log: -- replaced with /var/log/ceph/ceph-client.admin.45271.log.gz 2024-04-25T10:31:36.409 INFO:teuthology.orchestra.run.smithi037.stderr: 88.5% 91.9% -- replaced with /var/log/ceph/ceph-mds.c.log.gz 2024-04-25T10:31:36.410 INFO:teuthology.orchestra.run.smithi037.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.42448.log 2024-04-25T10:31:36.410 INFO:teuthology.orchestra.run.smithi037.stderr: -- replaced with /var/log/ceph/ceph-client.admin.54529.log.gz 2024-04-25T10:31:36.410 INFO:teuthology.orchestra.run.smithi037.stderr:/var/log/ceph/ceph-client.admin.49406.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.52218.log 2024-04-25T10:31:36.410 INFO:teuthology.orchestra.run.smithi037.stderr: 88.9%/var/log/ceph/ceph-client.admin.42448.log: -- replaced with /var/log/ceph/ceph-client.admin.49406.log.gz 2024-04-25T10:31:36.410 INFO:teuthology.orchestra.run.smithi037.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.46195.log 2024-04-25T10:31:36.410 INFO:teuthology.orchestra.run.smithi037.stderr: 88.0% -- replaced with /var/log/ceph/ceph-client.admin.42448.log.gz 2024-04-25T10:31:36.410 INFO:teuthology.orchestra.run.smithi037.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.49859.log 2024-04-25T10:31:36.410 INFO:teuthology.orchestra.run.smithi037.stderr:/var/log/ceph/ceph-client.admin.52218.log: /var/log/ceph/ceph-client.admin.46195.log: 89.4% -- replaced with /var/log/ceph/ceph-client.admin.52218.log.gz 2024-04-25T10:31:36.410 INFO:teuthology.orchestra.run.smithi037.stderr: 88.4% -- replaced with /var/log/ceph/ceph-client.admin.46195.log.gz 2024-04-25T10:31:36.410 INFO:teuthology.orchestra.run.smithi037.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.52982.log 2024-04-25T10:31:36.410 INFO:teuthology.orchestra.run.smithi037.stderr:/var/log/ceph/ceph-client.admin.49859.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.47798.log 2024-04-25T10:31:36.410 INFO:teuthology.orchestra.run.smithi037.stderr: 87.7% -- replaced with /var/log/ceph/ceph-client.admin.49859.log.gz 2024-04-25T10:31:36.410 INFO:teuthology.orchestra.run.smithi037.stderr:/var/log/ceph/ceph-client.admin.52982.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.42280.log 2024-04-25T10:31:36.410 INFO:teuthology.orchestra.run.smithi037.stderr: 88.5%/var/log/ceph/ceph-client.admin.47798.log: -- replaced with /var/log/ceph/ceph-client.admin.52982.log.gz 2024-04-25T10:31:36.411 INFO:teuthology.orchestra.run.smithi037.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.41538.log 2024-04-25T10:31:36.411 INFO:teuthology.orchestra.run.smithi037.stderr: 88.7% -- replaced with /var/log/ceph/ceph-client.admin.47798.log.gz 2024-04-25T10:31:36.411 INFO:teuthology.orchestra.run.smithi037.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.41873.log 2024-04-25T10:31:36.411 INFO:teuthology.orchestra.run.smithi037.stderr:/var/log/ceph/ceph-client.admin.42280.log: /var/log/ceph/ceph-client.admin.41538.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.41530.log 2024-04-25T10:31:36.411 INFO:teuthology.orchestra.run.smithi037.stderr:/var/log/ceph/ceph-client.admin.41873.log: 89.2% -- replaced with /var/log/ceph/ceph-client.admin.42280.log.gz 2024-04-25T10:31:36.411 INFO:teuthology.orchestra.run.smithi037.stderr: 87.8% -- replaced with /var/log/ceph/ceph-client.admin.41538.log.gz 2024-04-25T10:31:36.411 INFO:teuthology.orchestra.run.smithi037.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.40516.log 2024-04-25T10:31:36.411 INFO:teuthology.orchestra.run.smithi037.stderr: 89.1% -- replaced with /var/log/ceph/ceph-client.admin.41873.log.gz 2024-04-25T10:31:36.411 INFO:teuthology.orchestra.run.smithi037.stderr:gzip -5/var/log/ceph/ceph-client.admin.41530.log: --verbose -- /var/log/ceph/ceph-client.admin.46483.log 2024-04-25T10:31:36.411 INFO:teuthology.orchestra.run.smithi037.stderr:/var/log/ceph/ceph-client.admin.40516.log: 87.7% -- replaced with /var/log/ceph/ceph-client.admin.41530.log.gz 2024-04-25T10:31:36.411 INFO:teuthology.orchestra.run.smithi037.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.49096.log 2024-04-25T10:31:36.411 INFO:teuthology.orchestra.run.smithi037.stderr: 88.9% -- replaced with /var/log/ceph/ceph-client.admin.40516.log.gz 2024-04-25T10:31:36.411 INFO:teuthology.orchestra.run.smithi037.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.48010.log 2024-04-25T10:31:36.411 INFO:teuthology.orchestra.run.smithi037.stderr:/var/log/ceph/ceph-client.admin.46483.log: /var/log/ceph/ceph-client.admin.49096.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.54108.log 2024-04-25T10:31:36.412 INFO:teuthology.orchestra.run.smithi037.stderr:/var/log/ceph/ceph-client.admin.48010.log: 87.6% -- replaced with /var/log/ceph/ceph-client.admin.46483.log.gz 2024-04-25T10:31:36.412 INFO:teuthology.orchestra.run.smithi037.stderr: 88.5% -- replaced with /var/log/ceph/ceph-client.admin.49096.log.gz 2024-04-25T10:31:36.412 INFO:teuthology.orchestra.run.smithi037.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.50577.log 2024-04-25T10:31:36.412 INFO:teuthology.orchestra.run.smithi037.stderr: 88.9% -- replaced with /var/log/ceph/ceph-client.admin.48010.log.gz 2024-04-25T10:31:36.412 INFO:teuthology.orchestra.run.smithi037.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.39950.log 2024-04-25T10:31:36.412 INFO:teuthology.orchestra.run.smithi037.stderr:/var/log/ceph/ceph-client.admin.54108.log: /var/log/ceph/ceph-client.admin.50577.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.42406.log 2024-04-25T10:31:36.412 INFO:teuthology.orchestra.run.smithi037.stderr:/var/log/ceph/ceph-client.admin.39950.log: 89.0% -- replaced with /var/log/ceph/ceph-client.admin.54108.log.gz 2024-04-25T10:31:36.412 INFO:teuthology.orchestra.run.smithi037.stderr: 89.0% -- replaced with /var/log/ceph/ceph-client.admin.50577.log.gz 2024-04-25T10:31:36.412 INFO:teuthology.orchestra.run.smithi037.stderr: 87.6% -- replaced with /var/log/ceph/ceph-client.admin.39950.log.gzgzip 2024-04-25T10:31:36.412 INFO:teuthology.orchestra.run.smithi037.stderr: -5 --verbose -- /var/log/ceph/ceph-client.admin.46153.log 2024-04-25T10:31:36.412 INFO:teuthology.orchestra.run.smithi037.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.45129.log 2024-04-25T10:31:36.412 INFO:teuthology.orchestra.run.smithi037.stderr:/var/log/ceph/ceph-client.admin.42406.log: /var/log/ceph/ceph-client.admin.46153.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.54695.log 2024-04-25T10:31:36.412 INFO:teuthology.orchestra.run.smithi037.stderr:/var/log/ceph/ceph-client.admin.45129.log: 88.9% -- replaced with /var/log/ceph/ceph-client.admin.42406.log.gz 2024-04-25T10:31:36.412 INFO:teuthology.orchestra.run.smithi037.stderr: 88.2% -- replaced with /var/log/ceph/ceph-client.admin.46153.log.gz 2024-04-25T10:31:36.413 INFO:teuthology.orchestra.run.smithi037.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.53912.log 2024-04-25T10:31:36.413 INFO:teuthology.orchestra.run.smithi037.stderr: 88.0% -- replaced with /var/log/ceph/ceph-client.admin.45129.log.gz 2024-04-25T10:31:36.413 INFO:teuthology.orchestra.run.smithi037.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.41534.log 2024-04-25T10:31:36.413 INFO:teuthology.orchestra.run.smithi037.stderr:/var/log/ceph/ceph-client.admin.54695.log: /var/log/ceph/ceph-client.admin.53912.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.52563.log 2024-04-25T10:31:36.413 INFO:teuthology.orchestra.run.smithi037.stderr:/var/log/ceph/ceph-client.admin.41534.log: 87.6% -- replaced with /var/log/ceph/ceph-client.admin.54695.log.gz 2024-04-25T10:31:36.413 INFO:teuthology.orchestra.run.smithi037.stderr: 87.3% -- replaced with /var/log/ceph/ceph-client.admin.53912.log.gz 2024-04-25T10:31:36.413 INFO:teuthology.orchestra.run.smithi037.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.47966.log 2024-04-25T10:31:36.413 INFO:teuthology.orchestra.run.smithi037.stderr: 88.3% -- replaced with /var/log/ceph/ceph-client.admin.41534.log.gz 2024-04-25T10:31:36.413 INFO:teuthology.orchestra.run.smithi037.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.46441.log 2024-04-25T10:31:36.413 INFO:teuthology.orchestra.run.smithi037.stderr:/var/log/ceph/ceph-client.admin.52563.log: gzip/var/log/ceph/ceph-client.admin.47966.log: -5 --verbose -- /var/log/ceph/ceph-client.admin.34394.log 2024-04-25T10:31:36.413 INFO:teuthology.orchestra.run.smithi037.stderr:/var/log/ceph/ceph-client.admin.46441.log: 88.0% -- replaced with /var/log/ceph/ceph-client.admin.47966.log.gz 2024-04-25T10:31:36.413 INFO:teuthology.orchestra.run.smithi037.stderr: 88.7% -- replaced with /var/log/ceph/ceph-client.admin.52563.log.gz 2024-04-25T10:31:36.413 INFO:teuthology.orchestra.run.smithi037.stderr: 88.8%gzip -- replaced with /var/log/ceph/ceph-client.admin.46441.log.gz 2024-04-25T10:31:36.413 INFO:teuthology.orchestra.run.smithi037.stderr: -5 --verbose -- /var/log/ceph/ceph-client.admin.50073.log 2024-04-25T10:31:36.414 INFO:teuthology.orchestra.run.smithi037.stderr:/var/log/ceph/ceph-client.admin.34394.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.41191.log 2024-04-25T10:31:36.414 INFO:teuthology.orchestra.run.smithi037.stderr: 89.0% -- replaced with /var/log/ceph/ceph-client.admin.34394.log.gz/var/log/ceph/ceph-client.admin.50073.log: 2024-04-25T10:31:36.414 INFO:teuthology.orchestra.run.smithi037.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.41201.log 2024-04-25T10:31:36.414 INFO:teuthology.orchestra.run.smithi037.stderr: 89.0% -- replaced with /var/log/ceph/ceph-client.admin.50073.log.gzgzip -5 --verbose -- /var/log/ceph/ceph-client.admin.50786.log 2024-04-25T10:31:36.414 INFO:teuthology.orchestra.run.smithi037.stderr: 2024-04-25T10:31:36.414 INFO:teuthology.orchestra.run.smithi037.stderr:/var/log/ceph/ceph-client.admin.41191.log: /var/log/ceph/ceph-client.admin.41201.log: 88.4% -- replaced with /var/log/ceph/ceph-client.admin.41191.log.gz 2024-04-25T10:31:36.414 INFO:teuthology.orchestra.run.smithi037.stderr: 88.4% -- replaced with /var/log/ceph/ceph-client.admin.41201.log.gz 2024-04-25T10:31:36.414 INFO:teuthology.orchestra.run.smithi037.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.52857.log 2024-04-25T10:31:36.414 INFO:teuthology.orchestra.run.smithi037.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.44728.log 2024-04-25T10:31:36.414 INFO:teuthology.orchestra.run.smithi037.stderr:/var/log/ceph/ceph-client.admin.50786.log: /var/log/ceph/ceph-client.admin.52857.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.39773.log 2024-04-25T10:31:36.414 INFO:teuthology.orchestra.run.smithi037.stderr:/var/log/ceph/ceph-client.admin.44728.log: 88.4% -- replaced with /var/log/ceph/ceph-client.admin.52857.log.gz 2024-04-25T10:31:36.414 INFO:teuthology.orchestra.run.smithi037.stderr: 90.7% -- replaced with /var/log/ceph/ceph-client.admin.50786.log.gz 2024-04-25T10:31:36.414 INFO:teuthology.orchestra.run.smithi037.stderr: 89.5% -- replaced with /var/log/ceph/ceph-client.admin.44728.log.gz 2024-04-25T10:31:36.414 INFO:teuthology.orchestra.run.smithi037.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.41868.log 2024-04-25T10:31:36.415 INFO:teuthology.orchestra.run.smithi037.stderr:/var/log/ceph/ceph-client.admin.39773.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.54235.log 2024-04-25T10:31:36.415 INFO:teuthology.orchestra.run.smithi037.stderr: 88.7%gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.51462.log 2024-04-25T10:31:36.415 INFO:teuthology.orchestra.run.smithi037.stderr: -- replaced with /var/log/ceph/ceph-client.admin.39773.log.gz 2024-04-25T10:31:36.415 INFO:teuthology.orchestra.run.smithi037.stderr:/var/log/ceph/ceph-client.admin.41868.log: /var/log/ceph/ceph-client.admin.54235.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.47840.log 2024-04-25T10:31:36.415 INFO:teuthology.orchestra.run.smithi037.stderr:/var/log/ceph/ceph-client.admin.51462.log: 88.0% -- replaced with /var/log/ceph/ceph-client.admin.54235.log.gz 2024-04-25T10:31:36.415 INFO:teuthology.orchestra.run.smithi037.stderr: 88.8% -- replaced with /var/log/ceph/ceph-client.admin.41868.log.gz 2024-04-25T10:31:36.415 INFO:teuthology.orchestra.run.smithi037.stderr:gzip -5 88.2% -- replaced with /var/log/ceph/ceph-client.admin.51462.log.gz 2024-04-25T10:31:36.415 INFO:teuthology.orchestra.run.smithi037.stderr: --verbose -- /var/log/ceph/ceph-client.admin.47029.log 2024-04-25T10:31:36.415 INFO:teuthology.orchestra.run.smithi037.stderr:/var/log/ceph/ceph-client.admin.47840.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.40390.log 2024-04-25T10:31:36.415 INFO:teuthology.orchestra.run.smithi037.stderr: 88.4% -- replaced with /var/log/ceph/ceph-client.admin.47840.log.gz 2024-04-25T10:31:36.415 INFO:teuthology.orchestra.run.smithi037.stderr:/var/log/ceph/ceph-client.admin.47029.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.40200.log 88.2% 2024-04-25T10:31:36.415 INFO:teuthology.orchestra.run.smithi037.stderr: -- replaced with /var/log/ceph/ceph-client.admin.47029.log.gz 2024-04-25T10:31:36.415 INFO:teuthology.orchestra.run.smithi037.stderr:/var/log/ceph/ceph-client.admin.40390.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.50535.log 2024-04-25T10:31:36.415 INFO:teuthology.orchestra.run.smithi037.stderr: 89.0%gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.44854.log 2024-04-25T10:31:36.415 INFO:teuthology.orchestra.run.smithi037.stderr: -- replaced with /var/log/ceph/ceph-client.admin.40390.log.gz 2024-04-25T10:31:36.416 INFO:teuthology.orchestra.run.smithi037.stderr:/var/log/ceph/ceph-client.admin.40200.log: /var/log/ceph/ceph-client.admin.50535.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.51396.log 2024-04-25T10:31:36.416 INFO:teuthology.orchestra.run.smithi037.stderr:/var/log/ceph/ceph-client.admin.44854.log: 88.6% -- replaced with /var/log/ceph/ceph-client.admin.40200.log.gz 2024-04-25T10:31:36.416 INFO:teuthology.orchestra.run.smithi037.stderr: 88.2% -- replaced with /var/log/ceph/ceph-client.admin.50535.log.gz 2024-04-25T10:31:36.416 INFO:teuthology.orchestra.run.smithi037.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.53505.log 2024-04-25T10:31:36.416 INFO:teuthology.orchestra.run.smithi037.stderr: 87.4% -- replaced with /var/log/ceph/ceph-client.admin.44854.log.gz 2024-04-25T10:31:36.416 INFO:teuthology.orchestra.run.smithi037.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.45087.log 2024-04-25T10:31:36.416 INFO:teuthology.orchestra.run.smithi037.stderr:/var/log/ceph/ceph-client.admin.51396.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.41536.log 2024-04-25T10:31:36.416 INFO:teuthology.orchestra.run.smithi037.stderr:/var/log/ceph/ceph-client.admin.53505.log: 88.8% -- replaced with /var/log/ceph/ceph-client.admin.53505.log.gz/var/log/ceph/ceph-client.admin.45087.log: 2024-04-25T10:31:36.416 INFO:teuthology.orchestra.run.smithi037.stderr: 90.8% -- replaced with /var/log/ceph/ceph-client.admin.51396.log.gz 2024-04-25T10:31:36.416 INFO:teuthology.orchestra.run.smithi037.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.50199.log 2024-04-25T10:31:36.416 INFO:teuthology.orchestra.run.smithi037.stderr: 88.1% -- replaced with /var/log/ceph/ceph-client.admin.45087.log.gz 2024-04-25T10:31:36.416 INFO:teuthology.orchestra.run.smithi037.stderr:/var/log/ceph/ceph-client.admin.41536.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.43844.log 2024-04-25T10:31:36.416 INFO:teuthology.orchestra.run.smithi037.stderr: 87.9%/var/log/ceph/ceph-client.admin.50199.log: -- replaced with /var/log/ceph/ceph-client.admin.41536.log.gz 2024-04-25T10:31:36.416 INFO:teuthology.orchestra.run.smithi037.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.52134.log 2024-04-25T10:31:36.417 INFO:teuthology.orchestra.run.smithi037.stderr: 88.1% -- replaced with /var/log/ceph/ceph-client.admin.50199.log.gz 2024-04-25T10:31:36.417 INFO:teuthology.orchestra.run.smithi037.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.41532.log 2024-04-25T10:31:36.417 INFO:teuthology.orchestra.run.smithi037.stderr:/var/log/ceph/ceph-client.admin.43844.log: /var/log/ceph/ceph-client.admin.52134.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.40936.log 2024-04-25T10:31:36.417 INFO:teuthology.orchestra.run.smithi037.stderr:/var/log/ceph/ceph-client.admin.41532.log: 87.9% -- replaced with /var/log/ceph/ceph-client.admin.43844.log.gz 89.1% -- replaced with /var/log/ceph/ceph-client.admin.52134.log.gz 2024-04-25T10:31:36.417 INFO:teuthology.orchestra.run.smithi037.stderr: 2024-04-25T10:31:36.417 INFO:teuthology.orchestra.run.smithi037.stderr: 88.0% -- replaced with /var/log/ceph/ceph-client.admin.41532.log.gz 2024-04-25T10:31:36.417 INFO:teuthology.orchestra.run.smithi037.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.51158.log 2024-04-25T10:31:36.417 INFO:teuthology.orchestra.run.smithi037.stderr:/var/log/ceph/ceph-client.admin.40936.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.50954.log 2024-04-25T10:31:36.417 INFO:teuthology.orchestra.run.smithi037.stderr: 87.6%/var/log/ceph/ceph-client.admin.51158.log: -- replaced with /var/log/ceph/ceph-client.admin.40936.log.gz 2024-04-25T10:31:36.417 INFO:teuthology.orchestra.run.smithi037.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.44812.log 2024-04-25T10:31:36.417 INFO:teuthology.orchestra.run.smithi037.stderr: 88.7% -- replaced with /var/log/ceph/ceph-client.admin.51158.log.gz 2024-04-25T10:31:36.417 INFO:teuthology.orchestra.run.smithi037.stderr:/var/log/ceph/ceph-client.admin.50954.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.44560.log 2024-04-25T10:31:36.417 INFO:teuthology.orchestra.run.smithi037.stderr:/var/log/ceph/ceph-client.admin.44812.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.44897.log 2024-04-25T10:31:36.417 INFO:teuthology.orchestra.run.smithi037.stderr: 87.6% -- replaced with /var/log/ceph/ceph-client.admin.44812.log.gz 2024-04-25T10:31:36.418 INFO:teuthology.orchestra.run.smithi037.stderr:/var/log/ceph/ceph-client.admin.44560.log: 92.4% -- replaced with /var/log/ceph/ceph-client.admin.50954.log.gz 2024-04-25T10:31:36.418 INFO:teuthology.orchestra.run.smithi037.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.48595.log 2024-04-25T10:31:36.418 INFO:teuthology.orchestra.run.smithi037.stderr: 87.6% -- replaced with /var/log/ceph/ceph-client.admin.44560.log.gz 2024-04-25T10:31:36.418 INFO:teuthology.orchestra.run.smithi037.stderr:/var/log/ceph/ceph-client.admin.44897.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.42070.log 2024-04-25T10:31:36.418 INFO:teuthology.orchestra.run.smithi037.stderr: 88.7%/var/log/ceph/ceph-client.admin.48595.log: -- replaced with /var/log/ceph/ceph-client.admin.44897.log.gz 2024-04-25T10:31:36.418 INFO:teuthology.orchestra.run.smithi037.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.51354.log 2024-04-25T10:31:36.418 INFO:teuthology.orchestra.run.smithi037.stderr: 88.0% -- replaced with /var/log/ceph/ceph-client.admin.48595.log.gz 2024-04-25T10:31:36.418 INFO:teuthology.orchestra.run.smithi037.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.45475.log 2024-04-25T10:31:36.418 INFO:teuthology.orchestra.run.smithi037.stderr:/var/log/ceph/ceph-client.admin.42070.log: /var/log/ceph/ceph-client.admin.51354.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.51200.log 2024-04-25T10:31:36.418 INFO:teuthology.orchestra.run.smithi037.stderr: 88.1% -- replaced with /var/log/ceph/ceph-client.admin.42070.log.gz 2024-04-25T10:31:36.418 INFO:teuthology.orchestra.run.smithi037.stderr: 87.4% -- replaced with /var/log/ceph/ceph-client.admin.51354.log.gz 2024-04-25T10:31:36.418 INFO:teuthology.orchestra.run.smithi037.stderr:/var/log/ceph/ceph-client.admin.45475.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.41734.log 2024-04-25T10:31:36.418 INFO:teuthology.orchestra.run.smithi037.stderr: 89.0% -- replaced with /var/log/ceph/ceph-client.admin.45475.log.gz 2024-04-25T10:31:36.418 INFO:teuthology.orchestra.run.smithi037.stderr:/var/log/ceph/ceph-client.admin.51200.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.44644.log 88.6% -- replaced with /var/log/ceph/ceph-client.admin.51200.log.gz 2024-04-25T10:31:36.419 INFO:teuthology.orchestra.run.smithi037.stderr: 2024-04-25T10:31:36.419 INFO:teuthology.orchestra.run.smithi037.stderr:/var/log/ceph/ceph-client.admin.41734.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.42490.log 2024-04-25T10:31:36.419 INFO:teuthology.orchestra.run.smithi037.stderr: 87.8% -- replaced with /var/log/ceph/ceph-client.admin.41734.log.gz 2024-04-25T10:31:36.419 INFO:teuthology.orchestra.run.smithi037.stderr:/var/log/ceph/ceph-client.admin.44644.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.45853.log 2024-04-25T10:31:36.419 INFO:teuthology.orchestra.run.smithi037.stderr: 87.9% -- replaced with /var/log/ceph/ceph-client.admin.44644.log.gz 2024-04-25T10:31:36.419 INFO:teuthology.orchestra.run.smithi037.stderr:/var/log/ceph/ceph-client.admin.42490.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.53547.log 2024-04-25T10:31:36.419 INFO:teuthology.orchestra.run.smithi037.stderr: 88.4% -- replaced with /var/log/ceph/ceph-client.admin.42490.log.gz/var/log/ceph/ceph-client.admin.45853.log: 2024-04-25T10:31:36.419 INFO:teuthology.orchestra.run.smithi037.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.42700.log 87.7% -- replaced with /var/log/ceph/ceph-client.admin.45853.log.gz 2024-04-25T10:31:36.419 INFO:teuthology.orchestra.run.smithi037.stderr: 2024-04-25T10:31:36.419 INFO:teuthology.orchestra.run.smithi037.stderr:/var/log/ceph/ceph-client.admin.53547.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.40684.log 2024-04-25T10:31:36.419 INFO:teuthology.orchestra.run.smithi037.stderr: 88.7% -- replaced with /var/log/ceph/ceph-client.admin.53547.log.gz 2024-04-25T10:31:36.419 INFO:teuthology.orchestra.run.smithi037.stderr:/var/log/ceph/ceph-client.admin.42700.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.53234.log 2024-04-25T10:31:36.419 INFO:teuthology.orchestra.run.smithi037.stderr:/var/log/ceph/ceph-client.admin.40684.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.50828.log 2024-04-25T10:31:36.419 INFO:teuthology.orchestra.run.smithi037.stderr: 90.9% -- replaced with /var/log/ceph/ceph-client.admin.42700.log.gz 2024-04-25T10:31:36.419 INFO:teuthology.orchestra.run.smithi037.stderr: 88.3% -- replaced with /var/log/ceph/ceph-client.admin.40684.log.gz 2024-04-25T10:31:36.420 INFO:teuthology.orchestra.run.smithi037.stderr:/var/log/ceph/ceph-client.admin.53234.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.53360.log 2024-04-25T10:31:36.420 INFO:teuthology.orchestra.run.smithi037.stderr:/var/log/ceph/ceph-client.admin.50828.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.54193.log 2024-04-25T10:31:36.420 INFO:teuthology.orchestra.run.smithi037.stderr: 88.8% -- replaced with /var/log/ceph/ceph-client.admin.53234.log.gz 2024-04-25T10:31:36.420 INFO:teuthology.orchestra.run.smithi037.stderr:/var/log/ceph/ceph-client.admin.53360.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.52940.log 2024-04-25T10:31:36.420 INFO:teuthology.orchestra.run.smithi037.stderr: 90.4% -- replaced with /var/log/ceph/ceph-client.admin.50828.log.gz 2024-04-25T10:31:36.420 INFO:teuthology.orchestra.run.smithi037.stderr: 88.9% -- replaced with /var/log/ceph/ceph-client.admin.53360.log.gz 2024-04-25T10:31:36.420 INFO:teuthology.orchestra.run.smithi037.stderr:/var/log/ceph/ceph-client.admin.54193.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.45229.log 2024-04-25T10:31:36.420 INFO:teuthology.orchestra.run.smithi037.stderr: 88.5% -- replaced with /var/log/ceph/ceph-client.admin.54193.log.gz 2024-04-25T10:31:36.420 INFO:teuthology.orchestra.run.smithi037.stderr:/var/log/ceph/ceph-client.admin.52940.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.52395.log 2024-04-25T10:31:36.420 INFO:teuthology.orchestra.run.smithi037.stderr: 89.2% -- replaced with /var/log/ceph/ceph-client.admin.52940.log.gz 2024-04-25T10:31:36.420 INFO:teuthology.orchestra.run.smithi037.stderr:/var/log/ceph/ceph-client.admin.45229.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.54066.log 2024-04-25T10:31:36.420 INFO:teuthology.orchestra.run.smithi037.stderr: 89.0% -- replaced with /var/log/ceph/ceph-client.admin.45229.log.gz 2024-04-25T10:31:36.420 INFO:teuthology.orchestra.run.smithi037.stderr:/var/log/ceph/ceph-client.admin.52395.log: gzip -5 --verbose -- 87.7% /var/log/ceph/ceph-client.admin.40031.log 2024-04-25T10:31:36.420 INFO:teuthology.orchestra.run.smithi037.stderr: -- replaced with /var/log/ceph/ceph-client.admin.52395.log.gz 2024-04-25T10:31:36.421 INFO:teuthology.orchestra.run.smithi037.stderr:/var/log/ceph/ceph-client.admin.54066.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.48361.log 2024-04-25T10:31:36.421 INFO:teuthology.orchestra.run.smithi037.stderr: 88.3%gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.40474.log 2024-04-25T10:31:36.421 INFO:teuthology.orchestra.run.smithi037.stderr: -- replaced with /var/log/ceph/ceph-client.admin.54066.log.gz 2024-04-25T10:31:36.421 INFO:teuthology.orchestra.run.smithi037.stderr:/var/log/ceph/ceph-client.admin.40031.log: /var/log/ceph/ceph-client.admin.48361.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.41202.log 2024-04-25T10:31:36.421 INFO:teuthology.orchestra.run.smithi037.stderr:/var/log/ceph/ceph-client.admin.40474.log: 88.0% -- replaced with /var/log/ceph/ceph-client.admin.40031.log.gz 2024-04-25T10:31:36.421 INFO:teuthology.orchestra.run.smithi037.stderr: 88.5% -- replaced with /var/log/ceph/ceph-client.admin.48361.log.gz 2024-04-25T10:31:36.421 INFO:teuthology.orchestra.run.smithi037.stderr:gzip 88.3% -- replaced with /var/log/ceph/ceph-client.admin.40474.log.gz 2024-04-25T10:31:36.421 INFO:teuthology.orchestra.run.smithi037.stderr: -5 --verbose -- /var/log/ceph/ceph-client.admin.52815.log 2024-04-25T10:31:36.421 INFO:teuthology.orchestra.run.smithi037.stderr:/var/log/ceph/ceph-client.admin.41202.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.49264.log 87.5% -- replaced with /var/log/ceph/ceph-client.admin.41202.log.gz 2024-04-25T10:31:36.421 INFO:teuthology.orchestra.run.smithi037.stderr: 2024-04-25T10:31:36.421 INFO:teuthology.orchestra.run.smithi037.stderr:/var/log/ceph/ceph-client.admin.52815.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.40642.log 2024-04-25T10:31:36.421 INFO:teuthology.orchestra.run.smithi037.stderr: 88.2% -- replaced with /var/log/ceph/ceph-client.admin.52815.log.gz 2024-04-25T10:31:36.421 INFO:teuthology.orchestra.run.smithi037.stderr:/var/log/ceph/ceph-client.admin.49264.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.46525.log 2024-04-25T10:31:36.421 INFO:teuthology.orchestra.run.smithi037.stderr: 88.9% -- replaced with /var/log/ceph/ceph-client.admin.49264.log.gz 2024-04-25T10:31:36.422 INFO:teuthology.orchestra.run.smithi037.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.49630.log 2024-04-25T10:31:36.422 INFO:teuthology.orchestra.run.smithi037.stderr:/var/log/ceph/ceph-client.admin.40642.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.54319.log 2024-04-25T10:31:36.422 INFO:teuthology.orchestra.run.smithi037.stderr: 88.4% -- replaced with /var/log/ceph/ceph-client.admin.40642.log.gz 2024-04-25T10:31:36.422 INFO:teuthology.orchestra.run.smithi037.stderr:/var/log/ceph/ceph-client.admin.46525.log: /var/log/ceph/ceph-client.admin.49630.log: 88.5% -- replaced with /var/log/ceph/ceph-client.admin.46525.log.gz 2024-04-25T10:31:36.422 INFO:teuthology.orchestra.run.smithi037.stderr: 89.1% -- replaced with /var/log/ceph/ceph-client.admin.49630.log.gz 2024-04-25T10:31:36.422 INFO:teuthology.orchestra.run.smithi037.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.48445.log 2024-04-25T10:31:36.422 INFO:teuthology.orchestra.run.smithi037.stderr:/var/log/ceph/ceph-client.admin.54319.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.45002.log 2024-04-25T10:31:36.422 INFO:teuthology.orchestra.run.smithi037.stderr:/var/log/ceph/ceph-client.admin.48445.log: 88.5% -- replaced with /var/log/ceph/ceph-client.admin.54319.log.gz 2024-04-25T10:31:36.422 INFO:teuthology.orchestra.run.smithi037.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.42658.log 2024-04-25T10:31:36.422 INFO:teuthology.orchestra.run.smithi037.stderr: 88.5% -- replaced with /var/log/ceph/ceph-client.admin.48445.log.gzgzip -5 --verbose -- /var/log/ceph/ceph-client.admin.41196.log 2024-04-25T10:31:36.422 INFO:teuthology.orchestra.run.smithi037.stderr: 2024-04-25T10:31:36.422 INFO:teuthology.orchestra.run.smithi037.stderr:/var/log/ceph/ceph-client.admin.45002.log: /var/log/ceph/ceph-client.admin.42658.log: 88.7% -- replaced with /var/log/ceph/ceph-client.admin.45002.log.gz 2024-04-25T10:31:36.422 INFO:teuthology.orchestra.run.smithi037.stderr: 88.6% -- replaced with /var/log/ceph/ceph-client.admin.42658.log.gz 2024-04-25T10:31:36.423 INFO:teuthology.orchestra.run.smithi037.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.43346.log 2024-04-25T10:31:36.423 INFO:teuthology.orchestra.run.smithi037.stderr:/var/log/ceph/ceph-client.admin.41196.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.45969.log 2024-04-25T10:31:36.423 INFO:teuthology.orchestra.run.smithi037.stderr: 88.2% -- replaced with /var/log/ceph/ceph-client.admin.41196.log.gz 2024-04-25T10:31:36.423 INFO:teuthology.orchestra.run.smithi037.stderr:/var/log/ceph/ceph-client.admin.43346.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.42154.log 2024-04-25T10:31:36.423 INFO:teuthology.orchestra.run.smithi037.stderr: 87.7% -- replaced with /var/log/ceph/ceph-client.admin.43346.log.gz/var/log/ceph/ceph-client.admin.45969.log: 2024-04-25T10:31:36.423 INFO:teuthology.orchestra.run.smithi037.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.41195.log 2024-04-25T10:31:36.423 INFO:teuthology.orchestra.run.smithi037.stderr: 88.3% -- replaced with /var/log/ceph/ceph-client.admin.45969.log.gz 2024-04-25T10:31:36.423 INFO:teuthology.orchestra.run.smithi037.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.53276.log 2024-04-25T10:31:36.423 INFO:teuthology.orchestra.run.smithi037.stderr:/var/log/ceph/ceph-client.admin.42154.log: /var/log/ceph/ceph-client.admin.41195.log: 88.3% -- replaced with /var/log/ceph/ceph-client.admin.42154.log.gz 2024-04-25T10:31:36.423 INFO:teuthology.orchestra.run.smithi037.stderr: 89.0% -- replaced with /var/log/ceph/ceph-client.admin.41195.log.gz 2024-04-25T10:31:36.423 INFO:teuthology.orchestra.run.smithi037.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.47696.log 2024-04-25T10:31:36.423 INFO:teuthology.orchestra.run.smithi037.stderr:/var/log/ceph/ceph-client.admin.53276.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.45171.log 87.6% -- replaced with /var/log/ceph/ceph-client.admin.53276.log.gz 2024-04-25T10:31:36.423 INFO:teuthology.orchestra.run.smithi037.stderr: 2024-04-25T10:31:36.423 INFO:teuthology.orchestra.run.smithi037.stderr:/var/log/ceph/ceph-client.admin.47696.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.40810.log 2024-04-25T10:31:36.424 INFO:teuthology.orchestra.run.smithi037.stderr: 87.2% -- replaced with /var/log/ceph/ceph-client.admin.47696.log.gz 2024-04-25T10:31:36.424 INFO:teuthology.orchestra.run.smithi037.stderr:/var/log/ceph/ceph-client.admin.45171.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.49322.log 2024-04-25T10:31:36.424 INFO:teuthology.orchestra.run.smithi037.stderr: 88.5% -- replaced with /var/log/ceph/ceph-client.admin.45171.log.gz 2024-04-25T10:31:36.424 INFO:teuthology.orchestra.run.smithi037.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.53024.log 2024-04-25T10:31:36.424 INFO:teuthology.orchestra.run.smithi037.stderr:/var/log/ceph/ceph-client.admin.40810.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.46777.log 2024-04-25T10:31:36.424 INFO:teuthology.orchestra.run.smithi037.stderr:/var/log/ceph/ceph-client.admin.49322.log: 88.6% -- replaced with /var/log/ceph/ceph-client.admin.40810.log.gz 2024-04-25T10:31:36.424 INFO:teuthology.orchestra.run.smithi037.stderr:/var/log/ceph/ceph-client.admin.53024.log: 89.2% -- replaced with /var/log/ceph/ceph-client.admin.49322.log.gz 2024-04-25T10:31:36.424 INFO:teuthology.orchestra.run.smithi037.stderr: 89.0% -- replaced with /var/log/ceph/ceph-client.admin.53024.log.gz 2024-04-25T10:31:36.424 INFO:teuthology.orchestra.run.smithi037.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.50157.log 2024-04-25T10:31:36.424 INFO:teuthology.orchestra.run.smithi037.stderr:/var/log/ceph/ceph-client.admin.46777.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.48277.log 2024-04-25T10:31:36.424 INFO:teuthology.orchestra.run.smithi037.stderr:gzip -5 89.4% --verbose -- /var/log/ceph/ceph-mds.a.log 2024-04-25T10:31:36.424 INFO:teuthology.orchestra.run.smithi037.stderr:/var/log/ceph/ceph-client.admin.50157.log: -- replaced with /var/log/ceph/ceph-client.admin.46777.log.gz 2024-04-25T10:31:36.424 INFO:teuthology.orchestra.run.smithi037.stderr: 88.8%/var/log/ceph/ceph-client.admin.48277.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.40978.log 2024-04-25T10:31:36.424 INFO:teuthology.orchestra.run.smithi037.stderr: -- replaced with /var/log/ceph/ceph-client.admin.50157.log.gz 2024-04-25T10:31:36.425 INFO:teuthology.orchestra.run.smithi037.stderr:/var/log/ceph/ceph-mds.a.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.48147.log 2024-04-25T10:31:36.425 INFO:teuthology.orchestra.run.smithi037.stderr: 88.7% -- replaced with /var/log/ceph/ceph-client.admin.48277.log.gz 2024-04-25T10:31:36.425 INFO:teuthology.orchestra.run.smithi037.stderr:/var/log/ceph/ceph-client.admin.40978.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.45643.log 2024-04-25T10:31:36.425 INFO:teuthology.orchestra.run.smithi037.stderr: 88.6% -- replaced with /var/log/ceph/ceph-client.admin.40978.log.gz/var/log/ceph/ceph-client.admin.48147.log: 2024-04-25T10:31:36.425 INFO:teuthology.orchestra.run.smithi037.stderr: 88.9% -- replaced with /var/log/ceph/ceph-client.admin.48147.log.gz 2024-04-25T10:31:36.425 INFO:teuthology.orchestra.run.smithi037.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.52647.log 2024-04-25T10:31:36.425 INFO:teuthology.orchestra.run.smithi037.stderr:/var/log/ceph/ceph-client.admin.45643.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.0.48855.log 2024-04-25T10:31:36.425 INFO:teuthology.orchestra.run.smithi037.stderr: 88.9% -- replaced with /var/log/ceph/ceph-client.admin.45643.log.gz/var/log/ceph/ceph-client.admin.52647.log: 2024-04-25T10:31:36.425 INFO:teuthology.orchestra.run.smithi037.stderr: 89.0%gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.48054.log 2024-04-25T10:31:36.425 INFO:teuthology.orchestra.run.smithi037.stderr: -- replaced with /var/log/ceph/ceph-client.admin.52647.log.gz 2024-04-25T10:31:36.425 INFO:teuthology.orchestra.run.smithi037.stderr:/var/log/ceph/ceph-client.0.48855.log: 94.0% -- replaced with /var/log/ceph/ceph-client.0.48855.log.gz 2024-04-25T10:31:36.425 INFO:teuthology.orchestra.run.smithi037.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.40726.log 2024-04-25T10:31:36.425 INFO:teuthology.orchestra.run.smithi037.stderr:/var/log/ceph/ceph-client.admin.48054.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.44476.log 2024-04-25T10:31:36.425 INFO:teuthology.orchestra.run.smithi037.stderr: 88.7% -- replaced with /var/log/ceph/ceph-client.admin.48054.log.gz/var/log/ceph/ceph-client.admin.40726.log: 2024-04-25T10:31:36.426 INFO:teuthology.orchestra.run.smithi037.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.46279.log 2024-04-25T10:31:36.426 INFO:teuthology.orchestra.run.smithi037.stderr: 87.4%/var/log/ceph/ceph-client.admin.44476.log: -- replaced with /var/log/ceph/ceph-client.admin.40726.log.gz 2024-04-25T10:31:36.426 INFO:teuthology.orchestra.run.smithi037.stderr: 88.6% -- replaced with /var/log/ceph/ceph-client.admin.44476.log.gz 2024-04-25T10:31:36.426 INFO:teuthology.orchestra.run.smithi037.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.48319.log 2024-04-25T10:31:36.426 INFO:teuthology.orchestra.run.smithi037.stderr:/var/log/ceph/ceph-client.admin.46279.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.0.51764.log 2024-04-25T10:31:36.426 INFO:teuthology.orchestra.run.smithi037.stderr: 89.1% -- replaced with /var/log/ceph/ceph-client.admin.46279.log.gz 2024-04-25T10:31:36.426 INFO:teuthology.orchestra.run.smithi037.stderr:/var/log/ceph/ceph-client.admin.48319.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.46321.log 2024-04-25T10:31:36.426 INFO:teuthology.orchestra.run.smithi037.stderr: 88.4% -- replaced with /var/log/ceph/ceph-client.admin.48319.log.gz 2024-04-25T10:31:36.426 INFO:teuthology.orchestra.run.smithi037.stderr:/var/log/ceph/ceph-client.0.51764.log: 91.6% -- replaced with /var/log/ceph/ceph-client.0.51764.log.gz 2024-04-25T10:31:36.426 INFO:teuthology.orchestra.run.smithi037.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.46567.log 2024-04-25T10:31:36.426 INFO:teuthology.orchestra.run.smithi037.stderr:/var/log/ceph/ceph-client.admin.46321.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.52479.log 2024-04-25T10:31:36.426 INFO:teuthology.orchestra.run.smithi037.stderr: 88.2% -- replaced with /var/log/ceph/ceph-client.admin.46321.log.gz 2024-04-25T10:31:36.426 INFO:teuthology.orchestra.run.smithi037.stderr:/var/log/ceph/ceph-client.admin.46567.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.46399.log 2024-04-25T10:31:36.426 INFO:teuthology.orchestra.run.smithi037.stderr: 89.4% -- replaced with /var/log/ceph/ceph-client.admin.46567.log.gz 2024-04-25T10:31:36.427 INFO:teuthology.orchestra.run.smithi037.stderr:/var/log/ceph/ceph-client.admin.52479.log: 88.6% -- replaced with /var/log/ceph/ceph-client.admin.52479.log.gz 2024-04-25T10:31:36.427 INFO:teuthology.orchestra.run.smithi037.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.41871.log 2024-04-25T10:31:36.427 INFO:teuthology.orchestra.run.smithi037.stderr:/var/log/ceph/ceph-client.admin.46399.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.50283.log 2024-04-25T10:31:36.427 INFO:teuthology.orchestra.run.smithi037.stderr: 88.4% -- replaced with /var/log/ceph/ceph-client.admin.46399.log.gz 2024-04-25T10:31:36.427 INFO:teuthology.orchestra.run.smithi037.stderr:/var/log/ceph/ceph-client.admin.41871.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.41874.log 2024-04-25T10:31:36.427 INFO:teuthology.orchestra.run.smithi037.stderr:/var/log/ceph/ceph-client.admin.50283.log: 88.1% -- replaced with /var/log/ceph/ceph-client.admin.41871.log.gz 2024-04-25T10:31:36.427 INFO:teuthology.orchestra.run.smithi037.stderr: 89.1% -- replaced with /var/log/ceph/ceph-client.admin.50283.log.gz 2024-04-25T10:31:36.427 INFO:teuthology.orchestra.run.smithi037.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.44038.log 2024-04-25T10:31:36.427 INFO:teuthology.orchestra.run.smithi037.stderr:/var/log/ceph/ceph-client.admin.41874.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.41869.log 2024-04-25T10:31:36.427 INFO:teuthology.orchestra.run.smithi037.stderr: 88.2% -- replaced with /var/log/ceph/ceph-client.admin.41874.log.gz 2024-04-25T10:31:36.427 INFO:teuthology.orchestra.run.smithi037.stderr:/var/log/ceph/ceph-client.admin.44038.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.51038.log 2024-04-25T10:31:36.427 INFO:teuthology.orchestra.run.smithi037.stderr:/var/log/ceph/ceph-client.admin.41869.log: 87.7% -- replaced with /var/log/ceph/ceph-client.admin.44038.log.gz 2024-04-25T10:31:36.427 INFO:teuthology.orchestra.run.smithi037.stderr: 88.0% -- replaced with /var/log/ceph/ceph-client.admin.41869.log.gz 2024-04-25T10:31:36.427 INFO:teuthology.orchestra.run.smithi037.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.47654.log 2024-04-25T10:31:36.428 INFO:teuthology.orchestra.run.smithi037.stderr:/var/log/ceph/ceph-client.admin.51038.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.49138.log 2024-04-25T10:31:36.428 INFO:teuthology.orchestra.run.smithi037.stderr:/var/log/ceph/ceph-client.admin.47654.log: 91.1% -- replaced with /var/log/ceph/ceph-client.admin.51038.log.gz 2024-04-25T10:31:36.428 INFO:teuthology.orchestra.run.smithi037.stderr: 88.8% -- replaced with /var/log/ceph/ceph-client.admin.47654.log.gz 2024-04-25T10:31:36.428 INFO:teuthology.orchestra.run.smithi037.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.50619.log 2024-04-25T10:31:36.428 INFO:teuthology.orchestra.run.smithi037.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.41537.log 2024-04-25T10:31:36.428 INFO:teuthology.orchestra.run.smithi037.stderr:/var/log/ceph/ceph-client.admin.49138.log: /var/log/ceph/ceph-client.admin.50619.log: 89.1% -- replaced with /var/log/ceph/ceph-client.admin.49138.log.gz 2024-04-25T10:31:36.428 INFO:teuthology.orchestra.run.smithi037.stderr: 87.5% -- replaced with /var/log/ceph/ceph-client.admin.50619.log.gz 2024-04-25T10:31:36.428 INFO:teuthology.orchestra.run.smithi037.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.52437.log 2024-04-25T10:31:36.428 INFO:teuthology.orchestra.run.smithi037.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.52008.log 2024-04-25T10:31:36.428 INFO:teuthology.orchestra.run.smithi037.stderr:/var/log/ceph/ceph-client.admin.52437.log: /var/log/ceph/ceph-client.admin.41537.log: 87.5% 88.5% -- replaced with /var/log/ceph/ceph-client.admin.41537.log.gz 2024-04-25T10:31:36.428 INFO:teuthology.orchestra.run.smithi037.stderr: -- replaced with /var/log/ceph/ceph-client.admin.52437.log.gz 2024-04-25T10:31:36.428 INFO:teuthology.orchestra.run.smithi037.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.50241.log 2024-04-25T10:31:36.428 INFO:teuthology.orchestra.run.smithi037.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.42238.log 2024-04-25T10:31:36.428 INFO:teuthology.orchestra.run.smithi037.stderr:/var/log/ceph/ceph-client.admin.52008.log: /var/log/ceph/ceph-client.admin.50241.log: 88.2% -- replaced with /var/log/ceph/ceph-client.admin.52008.log.gz 2024-04-25T10:31:36.429 INFO:teuthology.orchestra.run.smithi037.stderr: 87.7% -- replaced with /var/log/ceph/ceph-client.admin.50241.log.gzgzip -5 --verbose -- /var/log/ceph/ceph-client.admin.53066.log 2024-04-25T10:31:36.429 INFO:teuthology.orchestra.run.smithi037.stderr: 2024-04-25T10:31:36.429 INFO:teuthology.orchestra.run.smithi037.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.44602.log 2024-04-25T10:31:36.429 INFO:teuthology.orchestra.run.smithi037.stderr:/var/log/ceph/ceph-client.admin.42238.log: /var/log/ceph/ceph-client.admin.53066.log: 87.6% -- replaced with /var/log/ceph/ceph-client.admin.42238.log.gz 2024-04-25T10:31:36.429 INFO:teuthology.orchestra.run.smithi037.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.54361.log 2024-04-25T10:31:36.429 INFO:teuthology.orchestra.run.smithi037.stderr: 88.4% -- replaced with /var/log/ceph/ceph-client.admin.53066.log.gz 2024-04-25T10:31:36.429 INFO:teuthology.orchestra.run.smithi037.stderr:/var/log/ceph/ceph-client.admin.44602.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.40558.log 2024-04-25T10:31:36.429 INFO:teuthology.orchestra.run.smithi037.stderr:/var/log/ceph/ceph-client.admin.54361.log: 88.5% -- replaced with /var/log/ceph/ceph-client.admin.44602.log.gz 2024-04-25T10:31:36.429 INFO:teuthology.orchestra.run.smithi037.stderr: 88.0% -- replaced with /var/log/ceph/ceph-client.admin.54361.log.gz 2024-04-25T10:31:36.429 INFO:teuthology.orchestra.run.smithi037.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.45517.log 2024-04-25T10:31:36.429 INFO:teuthology.orchestra.run.smithi037.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.48235.log 2024-04-25T10:31:36.429 INFO:teuthology.orchestra.run.smithi037.stderr:/var/log/ceph/ceph-client.admin.40558.log: /var/log/ceph/ceph-client.admin.45517.log: 87.6% -- replaced with /var/log/ceph/ceph-client.admin.40558.log.gz 2024-04-25T10:31:36.429 INFO:teuthology.orchestra.run.smithi037.stderr: 87.8%gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.45811.log 2024-04-25T10:31:36.429 INFO:teuthology.orchestra.run.smithi037.stderr: -- replaced with /var/log/ceph/ceph-client.admin.45517.log.gz 2024-04-25T10:31:36.430 INFO:teuthology.orchestra.run.smithi037.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.54655.log 2024-04-25T10:31:36.430 INFO:teuthology.orchestra.run.smithi037.stderr:/var/log/ceph/ceph-client.admin.48235.log: /var/log/ceph/ceph-client.admin.45811.log: 88.2% 88.3% -- replaced with /var/log/ceph/ceph-client.admin.48235.log.gz 2024-04-25T10:31:36.430 INFO:teuthology.orchestra.run.smithi037.stderr: -- replaced with /var/log/ceph/ceph-client.admin.45811.log.gz 2024-04-25T10:31:36.430 INFO:teuthology.orchestra.run.smithi037.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.46903.log 2024-04-25T10:31:36.430 INFO:teuthology.orchestra.run.smithi037.stderr:gzip -5 --verbose/var/log/ceph/ceph-client.admin.54655.log: -- /var/log/ceph/ceph-client.admin.40284.log 2024-04-25T10:31:36.430 INFO:teuthology.orchestra.run.smithi037.stderr: 88.3% -- replaced with /var/log/ceph/ceph-client.admin.54655.log.gz 2024-04-25T10:31:36.430 INFO:teuthology.orchestra.run.smithi037.stderr:/var/log/ceph/ceph-client.admin.46903.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.49180.log 2024-04-25T10:31:36.430 INFO:teuthology.orchestra.run.smithi037.stderr:/var/log/ceph/ceph-client.admin.40284.log: 92.0% -- replaced with /var/log/ceph/ceph-mds.a.log.gz 2024-04-25T10:31:36.430 INFO:teuthology.orchestra.run.smithi037.stderr: 87.7% -- replaced with /var/log/ceph/ceph-client.admin.46903.log.gz 2024-04-25T10:31:36.430 INFO:teuthology.orchestra.run.smithi037.stderr: 88.4% -- replaced with /var/log/ceph/ceph-client.admin.40284.log.gz 2024-04-25T10:31:36.430 INFO:teuthology.orchestra.run.smithi037.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.47324.log 2024-04-25T10:31:36.430 INFO:teuthology.orchestra.run.smithi037.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.44080.log 2024-04-25T10:31:36.430 INFO:teuthology.orchestra.run.smithi037.stderr:/var/log/ceph/ceph-client.admin.49180.log: /var/log/ceph/ceph-client.admin.47324.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.47924.log 2024-04-25T10:31:36.431 INFO:teuthology.orchestra.run.smithi037.stderr:/var/log/ceph/ceph-client.admin.44080.log: 89.5% -- replaced with /var/log/ceph/ceph-client.admin.49180.log.gz 2024-04-25T10:31:36.431 INFO:teuthology.orchestra.run.smithi037.stderr: 89.1% -- replaced with /var/log/ceph/ceph-client.admin.47324.log.gz 2024-04-25T10:31:36.431 INFO:teuthology.orchestra.run.smithi037.stderr: 87.6% -- replaced with /var/log/ceph/ceph-client.admin.44080.log.gz 2024-04-25T10:31:36.431 INFO:teuthology.orchestra.run.smithi037.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.39854.log 2024-04-25T10:31:36.431 INFO:teuthology.orchestra.run.smithi037.stderr:/var/log/ceph/ceph-client.admin.47924.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.44122.log 2024-04-25T10:31:36.431 INFO:teuthology.orchestra.run.smithi037.stderr: 88.3%/var/log/ceph/ceph-client.admin.39854.log: -- replaced with /var/log/ceph/ceph-client.admin.47924.log.gz 2024-04-25T10:31:36.431 INFO:teuthology.orchestra.run.smithi037.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.46735.log 2024-04-25T10:31:36.431 INFO:teuthology.orchestra.run.smithi037.stderr: 89.3% -- replaced with /var/log/ceph/ceph-client.admin.39854.log.gz 2024-04-25T10:31:36.431 INFO:teuthology.orchestra.run.smithi037.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.42196.log 2024-04-25T10:31:36.431 INFO:teuthology.orchestra.run.smithi037.stderr:/var/log/ceph/ceph-client.admin.44122.log: /var/log/ceph/ceph-client.admin.46735.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.52731.log 2024-04-25T10:31:36.431 INFO:teuthology.orchestra.run.smithi037.stderr: 87.8% -- replaced with /var/log/ceph/ceph-client.admin.44122.log.gz 2024-04-25T10:31:36.431 INFO:teuthology.orchestra.run.smithi037.stderr:/var/log/ceph/ceph-client.admin.42196.log: 88.8% -- replaced with /var/log/ceph/ceph-client.admin.46735.log.gz 2024-04-25T10:31:36.431 INFO:teuthology.orchestra.run.smithi037.stderr: 88.6% -- replaced with /var/log/ceph/ceph-client.admin.42196.log.gz 2024-04-25T10:31:36.431 INFO:teuthology.orchestra.run.smithi037.stderr:/var/log/ceph/ceph-client.admin.52731.log: 88.5% -- replaced with /var/log/ceph/ceph-client.admin.52731.log.gz 2024-04-25T10:31:36.432 INFO:teuthology.orchestra.run.smithi037.stderr: 93.9% -- replaced with /var/log/ceph/ceph-osd.2.log.gz 2024-04-25T10:31:36.433 INFO:teuthology.orchestra.run.smithi203.stderr: 93.5% -- replaced with /var/log/ceph/ceph-osd.5.log.gz 2024-04-25T10:31:36.433 INFO:teuthology.orchestra.run.smithi203.stderr: 93.6% -- replaced with /var/log/ceph/ceph-osd.7.log.gz 2024-04-25T10:31:36.433 INFO:teuthology.orchestra.run.smithi203.stderr: 93.8% -- replaced with /var/log/ceph/ceph-osd.6.log.gz 2024-04-25T10:31:36.433 INFO:teuthology.orchestra.run.smithi203.stderr: 93.9% -- replaced with /var/log/ceph/ceph-osd.4.log.gz 2024-04-25T10:31:36.434 INFO:teuthology.orchestra.run.smithi037.stderr: 93.7% -- replaced with /var/log/ceph/ceph-osd.1.log.gz 2024-04-25T10:31:36.435 INFO:teuthology.orchestra.run.smithi203.stderr: 2024-04-25T10:31:36.435 INFO:teuthology.orchestra.run.smithi203.stderr:real 0m0.195s 2024-04-25T10:31:36.435 INFO:teuthology.orchestra.run.smithi203.stderr:user 0m0.970s 2024-04-25T10:31:36.435 INFO:teuthology.orchestra.run.smithi203.stderr:sys 0m0.060s 2024-04-25T10:31:36.458 INFO:teuthology.orchestra.run.smithi037.stderr: 93.8% -- replaced with /var/log/ceph/ceph-osd.0.log.gz 2024-04-25T10:31:36.464 INFO:teuthology.orchestra.run.smithi037.stderr: 93.9% -- replaced with /var/log/ceph/ceph-osd.3.log.gz 2024-04-25T10:31:36.526 INFO:teuthology.orchestra.run.smithi037.stderr: 90.8% -- replaced with /var/log/ceph/ceph-mon.a.log.gz 2024-04-25T10:31:36.528 INFO:teuthology.orchestra.run.smithi037.stderr: 2024-04-25T10:31:36.528 INFO:teuthology.orchestra.run.smithi037.stderr:real 0m0.310s 2024-04-25T10:31:36.528 INFO:teuthology.orchestra.run.smithi037.stderr:user 0m1.176s 2024-04-25T10:31:36.528 INFO:teuthology.orchestra.run.smithi037.stderr:sys 0m0.244s 2024-04-25T10:31:36.529 INFO:tasks.ceph:Archiving logs... 2024-04-25T10:31:36.529 DEBUG:teuthology.misc:Transferring archived files from smithi037:/var/log/ceph to /home/teuthworker/archive/pdonnell-2024-04-25_10:00:23-fs-wip-pdonnell-testing-20240425.015853-debug-distro-default-smithi/7673076/remote/smithi037/log 2024-04-25T10:31:36.530 DEBUG:teuthology.orchestra.run.smithi037:> sudo tar c -f - -C /var/log/ceph -- . 2024-04-25T10:31:36.997 DEBUG:teuthology.misc:Transferring archived files from smithi203:/var/log/ceph to /home/teuthworker/archive/pdonnell-2024-04-25_10:00:23-fs-wip-pdonnell-testing-20240425.015853-debug-distro-default-smithi/7673076/remote/smithi203/log 2024-04-25T10:31:36.998 DEBUG:teuthology.orchestra.run.smithi203:> sudo tar c -f - -C /var/log/ceph -- . 2024-04-25T10:31:37.157 DEBUG:teuthology.run_tasks:Unwinding manager install 2024-04-25T10:31:37.167 INFO:teuthology.task.install.util:Removing shipped files: /home/ubuntu/cephtest/valgrind.supp /usr/bin/daemon-helper /usr/bin/adjust-ulimits /usr/bin/stdin-killer... 2024-04-25T10:31:37.168 DEBUG:teuthology.orchestra.run.smithi037:> sudo rm -f -- /home/ubuntu/cephtest/valgrind.supp /usr/bin/daemon-helper /usr/bin/adjust-ulimits /usr/bin/stdin-killer 2024-04-25T10:31:37.170 DEBUG:teuthology.orchestra.run.smithi203:> sudo rm -f -- /home/ubuntu/cephtest/valgrind.supp /usr/bin/daemon-helper /usr/bin/adjust-ulimits /usr/bin/stdin-killer 2024-04-25T10:31:37.254 DEBUG:teuthology.run_tasks:Unwinding manager clock 2024-04-25T10:31:37.264 INFO:teuthology.task.clock:Checking final clock skew... 2024-04-25T10:31:37.264 DEBUG:teuthology.orchestra.run.smithi037:> PATH=/usr/bin:/usr/sbin ntpq -p || PATH=/usr/bin:/usr/sbin chronyc sources || true 2024-04-25T10:31:37.267 DEBUG:teuthology.orchestra.run.smithi203:> PATH=/usr/bin:/usr/sbin ntpq -p || PATH=/usr/bin:/usr/sbin chronyc sources || true 2024-04-25T10:31:37.280 INFO:teuthology.orchestra.run.smithi037.stderr:bash: line 1: ntpq: command not found 2024-04-25T10:31:37.281 INFO:teuthology.orchestra.run.smithi203.stderr:bash: line 1: ntpq: command not found 2024-04-25T10:31:37.284 INFO:teuthology.orchestra.run.smithi037.stdout:MS Name/IP address Stratum Poll Reach LastRx Last sample 2024-04-25T10:31:37.284 INFO:teuthology.orchestra.run.smithi037.stdout:=============================================================================== 2024-04-25T10:31:37.285 INFO:teuthology.orchestra.run.smithi037.stdout:^* hv01.front.sepia.ceph.com 3 6 377 50 +1194us[+1207us] +/- 56ms 2024-04-25T10:31:37.285 INFO:teuthology.orchestra.run.smithi037.stdout:^+ hv02.front.sepia.ceph.com 3 6 377 50 -766us[ -753us] +/- 63ms 2024-04-25T10:31:37.285 INFO:teuthology.orchestra.run.smithi037.stdout:^+ hv03.front.sepia.ceph.com 3 6 377 52 -711us[ -698us] +/- 76ms 2024-04-25T10:31:37.285 INFO:teuthology.orchestra.run.smithi037.stdout:^? hv04.front.sepia.ceph.com 0 8 0 - +0ns[ +0ns] +/- 0ns 2024-04-25T10:31:37.286 INFO:teuthology.orchestra.run.smithi203.stdout:MS Name/IP address Stratum Poll Reach LastRx Last sample 2024-04-25T10:31:37.286 INFO:teuthology.orchestra.run.smithi203.stdout:=============================================================================== 2024-04-25T10:31:37.287 INFO:teuthology.orchestra.run.smithi203.stdout:^* hv01.front.sepia.ceph.com 3 6 377 51 +1197us[+1196us] +/- 56ms 2024-04-25T10:31:37.287 INFO:teuthology.orchestra.run.smithi203.stdout:^+ hv02.front.sepia.ceph.com 3 6 377 50 -778us[ -778us] +/- 63ms 2024-04-25T10:31:37.287 INFO:teuthology.orchestra.run.smithi203.stdout:^+ hv03.front.sepia.ceph.com 3 6 377 50 -712us[ -712us] +/- 76ms 2024-04-25T10:31:37.287 INFO:teuthology.orchestra.run.smithi203.stdout:^? hv04.front.sepia.ceph.com 0 8 0 - +0ns[ +0ns] +/- 0ns 2024-04-25T10:31:37.287 DEBUG:teuthology.run_tasks:Unwinding manager ansible.cephlab 2024-04-25T10:31:37.297 INFO:teuthology.task.ansible:Skipping ansible cleanup... 2024-04-25T10:31:37.298 DEBUG:teuthology.run_tasks:Unwinding manager selinux 2024-04-25T10:31:37.311 DEBUG:teuthology.orchestra.run.smithi037:> mkdir /home/ubuntu/cephtest/archive/audit && sudo cp /var/log/audit/audit.log /home/ubuntu/cephtest/archive/audit && sudo chown $USER /home/ubuntu/cephtest/archive/audit/audit.log && gzip /home/ubuntu/cephtest/archive/audit/audit.log 2024-04-25T10:31:37.395 DEBUG:teuthology.orchestra.run.smithi203:> mkdir /home/ubuntu/cephtest/archive/audit && sudo cp /var/log/audit/audit.log /home/ubuntu/cephtest/archive/audit && sudo chown $USER /home/ubuntu/cephtest/archive/audit/audit.log && gzip /home/ubuntu/cephtest/archive/audit/audit.log 2024-04-25T10:31:37.455 DEBUG:teuthology.orchestra.run.smithi037:> sudo grep -a 'avc: .*denied' /var/log/audit/audit.log | grep -av -e 'comm="dmidecode"' -e chronyd.service -e 'name="cephtest"' -e scontext=system_u:system_r:nrpe_t:s0 -e scontext=system_u:system_r:pcp_pmlogger_t -e scontext=system_u:system_r:pcp_pmcd_t:s0 -e 'comm="rhsmd"' -e scontext=system_u:system_r:syslogd_t:s0 -e tcontext=system_u:system_r:nrpe_t:s0 -e 'comm="updatedb"' -e 'comm="smartd"' -e 'comm="rhsmcertd-worke"' -e 'comm="setroubleshootd"' -e 'comm="rpm"' -e tcontext=system_u:object_r:container_runtime_exec_t:s0 -e 'comm="ksmtuned"' -e 'comm="sssd"' -e 'comm="sss_cache"' -e context=system_u:system_r:NetworkManager_dispatcher_t:s0 -e scontext=system_u:system_r:getty_t:s0 2024-04-25T10:31:37.483 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-04-25T10:31:37.483 DEBUG:teuthology.orchestra.run.smithi203:> sudo grep -a 'avc: .*denied' /var/log/audit/audit.log | grep -av -e 'comm="dmidecode"' -e chronyd.service -e 'name="cephtest"' -e scontext=system_u:system_r:nrpe_t:s0 -e scontext=system_u:system_r:pcp_pmlogger_t -e scontext=system_u:system_r:pcp_pmcd_t:s0 -e 'comm="rhsmd"' -e scontext=system_u:system_r:syslogd_t:s0 -e tcontext=system_u:system_r:nrpe_t:s0 -e 'comm="updatedb"' -e 'comm="smartd"' -e 'comm="rhsmcertd-worke"' -e 'comm="setroubleshootd"' -e 'comm="rpm"' -e tcontext=system_u:object_r:container_runtime_exec_t:s0 -e 'comm="ksmtuned"' -e 'comm="sssd"' -e 'comm="sss_cache"' -e context=system_u:system_r:NetworkManager_dispatcher_t:s0 -e scontext=system_u:system_r:getty_t:s0 2024-04-25T10:31:37.509 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-04-25T10:31:37.510 DEBUG:teuthology.run_tasks:Unwinding manager pcp 2024-04-25T10:31:37.520 DEBUG:teuthology.run_tasks:Unwinding manager internal.timer 2024-04-25T10:31:37.530 INFO:teuthology.task.internal:Duration was 528.871981 seconds 2024-04-25T10:31:37.530 DEBUG:teuthology.run_tasks:Unwinding manager internal.syslog 2024-04-25T10:31:37.542 INFO:teuthology.task.internal.syslog:Shutting down syslog monitoring... 2024-04-25T10:31:37.542 DEBUG:teuthology.orchestra.run.smithi037:> sudo rm -f -- /etc/rsyslog.d/80-cephtest.conf && sudo service rsyslog restart 2024-04-25T10:31:37.545 DEBUG:teuthology.orchestra.run.smithi203:> sudo rm -f -- /etc/rsyslog.d/80-cephtest.conf && sudo service rsyslog restart 2024-04-25T10:31:37.580 INFO:teuthology.orchestra.run.smithi037.stderr:Redirecting to /bin/systemctl restart rsyslog.service 2024-04-25T10:31:37.587 INFO:teuthology.orchestra.run.smithi203.stderr:Redirecting to /bin/systemctl restart rsyslog.service 2024-04-25T10:31:37.900 INFO:teuthology.task.internal.syslog:Checking logs for errors... 2024-04-25T10:31:37.900 DEBUG:teuthology.task.internal.syslog:Checking ubuntu@smithi037.front.sepia.ceph.com 2024-04-25T10:31:37.901 DEBUG:teuthology.orchestra.run.smithi037:> egrep --binary-files=text '\bBUG\b|\bINFO\b|\bDEADLOCK\b' /home/ubuntu/cephtest/archive/syslog/kern.log | grep -v 'task .* blocked for more than .* seconds' | grep -v 'lockdep is turned off' | grep -v 'trying to register non-static key' | grep -v 'DEBUG: fsize' | grep -v CRON | grep -v 'BUG: bad unlock balance detected' | grep -v 'inconsistent lock state' | grep -v '*** DEADLOCK ***' | grep -v 'INFO: possible irq lock inversion dependency detected' | grep -v 'INFO: NMI handler (perf_event_nmi_handler) took too long to run' | grep -v 'INFO: recovery required on readonly' | grep -v 'ceph-create-keys: INFO' | grep -v INFO:ceph-create-keys | grep -v 'Loaded datasource DataSourceOpenStack' | grep -v 'container-storage-setup: INFO: Volume group backing root filesystem could not be determined' | egrep -v '\bsalt-master\b|\bsalt-minion\b|\bsalt-api\b' | grep -v ceph-crash | egrep -v '\btcmu-runner\b.*\bINFO\b' | head -n 1 2024-04-25T10:31:37.961 DEBUG:teuthology.task.internal.syslog:Checking ubuntu@smithi203.front.sepia.ceph.com 2024-04-25T10:31:37.961 DEBUG:teuthology.orchestra.run.smithi203:> egrep --binary-files=text '\bBUG\b|\bINFO\b|\bDEADLOCK\b' /home/ubuntu/cephtest/archive/syslog/kern.log | grep -v 'task .* blocked for more than .* seconds' | grep -v 'lockdep is turned off' | grep -v 'trying to register non-static key' | grep -v 'DEBUG: fsize' | grep -v CRON | grep -v 'BUG: bad unlock balance detected' | grep -v 'inconsistent lock state' | grep -v '*** DEADLOCK ***' | grep -v 'INFO: possible irq lock inversion dependency detected' | grep -v 'INFO: NMI handler (perf_event_nmi_handler) took too long to run' | grep -v 'INFO: recovery required on readonly' | grep -v 'ceph-create-keys: INFO' | grep -v INFO:ceph-create-keys | grep -v 'Loaded datasource DataSourceOpenStack' | grep -v 'container-storage-setup: INFO: Volume group backing root filesystem could not be determined' | egrep -v '\bsalt-master\b|\bsalt-minion\b|\bsalt-api\b' | grep -v ceph-crash | egrep -v '\btcmu-runner\b.*\bINFO\b' | head -n 1 2024-04-25T10:31:37.981 INFO:teuthology.task.internal.syslog:Compressing syslogs... 2024-04-25T10:31:37.981 DEBUG:teuthology.orchestra.run.smithi037:> find /home/ubuntu/cephtest/archive/syslog -name '*.log' -print0 | sudo xargs -0 --no-run-if-empty -- gzip -- 2024-04-25T10:31:38.004 DEBUG:teuthology.orchestra.run.smithi203:> find /home/ubuntu/cephtest/archive/syslog -name '*.log' -print0 | sudo xargs -0 --no-run-if-empty -- gzip -- 2024-04-25T10:31:38.047 INFO:teuthology.task.internal.syslog:Gathering journactl -b0... 2024-04-25T10:31:38.047 DEBUG:teuthology.orchestra.run.smithi037:> sudo journalctl -b0 | gzip -9 > /home/ubuntu/cephtest/archive/syslog/journalctl-b0.gz 2024-04-25T10:31:38.070 DEBUG:teuthology.orchestra.run.smithi203:> sudo journalctl -b0 | gzip -9 > /home/ubuntu/cephtest/archive/syslog/journalctl-b0.gz 2024-04-25T10:31:38.182 DEBUG:teuthology.run_tasks:Unwinding manager internal.sudo 2024-04-25T10:31:38.193 INFO:teuthology.task.internal:Restoring /etc/sudoers... 2024-04-25T10:31:38.194 DEBUG:teuthology.orchestra.run.smithi037:> sudo mv -f /etc/sudoers.orig.teuthology /etc/sudoers 2024-04-25T10:31:38.217 DEBUG:teuthology.orchestra.run.smithi203:> sudo mv -f /etc/sudoers.orig.teuthology /etc/sudoers 2024-04-25T10:31:38.241 DEBUG:teuthology.run_tasks:Unwinding manager internal.coredump 2024-04-25T10:31:38.302 DEBUG:teuthology.orchestra.run.smithi037:> sudo sysctl -w kernel.core_pattern=core && sudo bash -c 'for f in `find /home/ubuntu/cephtest/archive/coredump -type f`; do file $f | grep -q systemd-sysusers && rm $f || true ; done' && rmdir --ignore-fail-on-non-empty -- /home/ubuntu/cephtest/archive/coredump 2024-04-25T10:31:38.305 DEBUG:teuthology.orchestra.run.smithi203:> sudo sysctl -w kernel.core_pattern=core && sudo bash -c 'for f in `find /home/ubuntu/cephtest/archive/coredump -type f`; do file $f | grep -q systemd-sysusers && rm $f || true ; done' && rmdir --ignore-fail-on-non-empty -- /home/ubuntu/cephtest/archive/coredump 2024-04-25T10:31:38.325 INFO:teuthology.orchestra.run.smithi037.stdout:kernel.core_pattern = core 2024-04-25T10:31:38.327 INFO:teuthology.orchestra.run.smithi203.stdout:kernel.core_pattern = core 2024-04-25T10:31:38.339 DEBUG:teuthology.orchestra.run.smithi037:> test -e /home/ubuntu/cephtest/archive/coredump 2024-04-25T10:31:38.392 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-04-25T10:31:38.392 DEBUG:teuthology.orchestra.run.smithi203:> test -e /home/ubuntu/cephtest/archive/coredump 2024-04-25T10:31:38.406 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-04-25T10:31:38.406 DEBUG:teuthology.run_tasks:Unwinding manager internal.archive 2024-04-25T10:31:38.418 INFO:teuthology.task.internal:Transferring archived files... 2024-04-25T10:31:38.418 DEBUG:teuthology.misc:Transferring archived files from smithi037:/home/ubuntu/cephtest/archive to /home/teuthworker/archive/pdonnell-2024-04-25_10:00:23-fs-wip-pdonnell-testing-20240425.015853-debug-distro-default-smithi/7673076/remote/smithi037 2024-04-25T10:31:38.419 DEBUG:teuthology.orchestra.run.smithi037:> sudo tar c -f - -C /home/ubuntu/cephtest/archive -- . 2024-04-25T10:31:38.467 DEBUG:teuthology.misc:Transferring archived files from smithi203:/home/ubuntu/cephtest/archive to /home/teuthworker/archive/pdonnell-2024-04-25_10:00:23-fs-wip-pdonnell-testing-20240425.015853-debug-distro-default-smithi/7673076/remote/smithi203 2024-04-25T10:31:38.467 DEBUG:teuthology.orchestra.run.smithi203:> sudo tar c -f - -C /home/ubuntu/cephtest/archive -- . 2024-04-25T10:31:38.500 INFO:teuthology.task.internal:Removing archive directory... 2024-04-25T10:31:38.500 DEBUG:teuthology.orchestra.run.smithi037:> rm -rf -- /home/ubuntu/cephtest/archive 2024-04-25T10:31:38.503 DEBUG:teuthology.orchestra.run.smithi203:> rm -rf -- /home/ubuntu/cephtest/archive 2024-04-25T10:31:38.548 DEBUG:teuthology.run_tasks:Unwinding manager internal.archive_upload 2024-04-25T10:31:38.565 INFO:teuthology.task.internal:Not uploading archives. 2024-04-25T10:31:38.565 DEBUG:teuthology.run_tasks:Unwinding manager internal.base 2024-04-25T10:31:38.617 INFO:teuthology.task.internal:Tidying up after the test... 2024-04-25T10:31:38.617 DEBUG:teuthology.orchestra.run.smithi037:> find /home/ubuntu/cephtest -ls ; rmdir -- /home/ubuntu/cephtest 2024-04-25T10:31:38.620 DEBUG:teuthology.orchestra.run.smithi203:> find /home/ubuntu/cephtest -ls ; rmdir -- /home/ubuntu/cephtest 2024-04-25T10:31:38.632 INFO:teuthology.orchestra.run.smithi037.stdout: 262213 4 drwxr-xr-x 2 ubuntu ubuntu 4096 Apr 25 10:31 /home/ubuntu/cephtest 2024-04-25T10:31:38.634 INFO:teuthology.orchestra.run.smithi203.stdout: 262213 4 drwxr-xr-x 2 ubuntu ubuntu 4096 Apr 25 10:31 /home/ubuntu/cephtest 2024-04-25T10:31:38.636 DEBUG:teuthology.run_tasks:Unwinding manager kernel 2024-04-25T10:31:38.647 DEBUG:teuthology.run_tasks:Unwinding manager console_log 2024-04-25T10:31:38.714 INFO:teuthology.run:Summary data: description: fs/functional/{begin/{0-install 1-ceph 2-logrotate 3-modules} clusters/1a3s-mds-4c-client conf/{client mds mgr mon osd} distro/{centos_latest} mount/fuse objectstore/bluestore-bitmap overrides/{ignorelist_health ignorelist_wrongly_marked_down no_client_pidfile pg_health} subvol_versions/create_subvol_version_v1 tasks/alternate-pool} duration: 528.8719809055328 flavor: default owner: scheduled_pdonnell@teuthology success: true 2024-04-25T10:31:38.714 DEBUG:teuthology.report:Pushing job info to https://paddles.front.sepia.ceph.com/ 2024-04-25T10:31:38.787 INFO:teuthology.run:pass